2024-09-11T21:36:12.947 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-11T21:36:12.974 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-11T21:36:13.059 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901408 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/fs/misc}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7901408' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount - mds.dir_split install: ceph: flavor: default sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - wsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 481 sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc sleep_before_teardown: 0 subset: 57/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi001.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBZOJwZiXUjpaJpxnOy2ox7FvSnqgSGQ6dSdxPkEb8jK6dqBcH0IyARgInCpfB1Su2nrBgjvyGQpw8RkOOga3/Y= smithi053.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAMYCtJ/m4/spxWON4fDvHD32qc54v0uaAa+SezBuzdujCitDVilk5oKQQezQ+itVtcrT7lcDssNa/C8nK0Fy8s= smithi161.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF2LkQuY5kbuTlhXG5KP6zvFXoFTEHUyMioeQ6vWz2mZIXNDogKMWvHrd4Njw5H6cvGLxk/c+GsPmndC3fGvzSE= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph config set mgr mgr/snap_schedule/log_level debug - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - fs/misc teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-11_21:24:17 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-11T21:36:13.059 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa; will attempt to use it 2024-09-11T21:36:13.060 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks 2024-09-11T21:36:13.060 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-11T21:36:13.061 INFO:teuthology.task.internal:Checking packages... 2024-09-11T21:36:13.083 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc' 2024-09-11T21:36:13.083 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-11T21:36:13.083 INFO:teuthology.packaging:ref: None 2024-09-11T21:36:13.083 INFO:teuthology.packaging:tag: None 2024-09-11T21:36:13.083 INFO:teuthology.packaging:branch: squid 2024-09-11T21:36:13.083 INFO:teuthology.packaging:sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T21:36:13.083 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-11T21:36:13.330 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-314.g086c737f 2024-09-11T21:36:13.331 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-11T21:36:13.339 INFO:teuthology.task.internal:no buildpackages task found 2024-09-11T21:36:13.339 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-11T21:36:13.390 INFO:teuthology.task.internal:Saving configuration 2024-09-11T21:36:13.410 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-11T21:36:13.443 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-11T21:36:13.469 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi001.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901408', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-11 21:30:54.345445', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBZOJwZiXUjpaJpxnOy2ox7FvSnqgSGQ6dSdxPkEb8jK6dqBcH0IyARgInCpfB1Su2nrBgjvyGQpw8RkOOga3/Y='} 2024-09-11T21:36:13.491 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi053.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901408', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-11 21:30:54.347142', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAMYCtJ/m4/spxWON4fDvHD32qc54v0uaAa+SezBuzdujCitDVilk5oKQQezQ+itVtcrT7lcDssNa/C8nK0Fy8s='} 2024-09-11T21:36:13.514 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi161.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901408', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-11 21:30:54.348322', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF2LkQuY5kbuTlhXG5KP6zvFXoFTEHUyMioeQ6vWz2mZIXNDogKMWvHrd4Njw5H6cvGLxk/c+GsPmndC3fGvzSE='} 2024-09-11T21:36:13.514 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-11T21:36:13.522 INFO:teuthology.task.internal:roles: ubuntu@smithi001.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-11T21:36:13.522 INFO:teuthology.task.internal:roles: ubuntu@smithi053.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-11T21:36:13.522 INFO:teuthology.task.internal:roles: ubuntu@smithi161.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-11T21:36:13.522 INFO:teuthology.run_tasks:Running task console_log... 2024-09-11T21:36:13.612 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fcba5887be0>, signals=[15]) 2024-09-11T21:36:13.612 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-11T21:36:13.618 INFO:teuthology.task.internal:Opening connections... 2024-09-11T21:36:13.618 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi001.front.sepia.ceph.com 2024-09-11T21:36:13.619 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi001.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T21:36:13.699 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi053.front.sepia.ceph.com 2024-09-11T21:36:13.700 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T21:36:13.774 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi161.front.sepia.ceph.com 2024-09-11T21:36:13.775 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T21:36:13.850 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-11T21:36:13.859 DEBUG:teuthology.orchestra.run.smithi001:> uname -m 2024-09-11T21:36:13.877 INFO:teuthology.orchestra.run.smithi001.stdout:x86_64 2024-09-11T21:36:13.878 DEBUG:teuthology.orchestra.run.smithi001:> cat /etc/os-release 2024-09-11T21:36:13.937 INFO:teuthology.orchestra.run.smithi001.stdout:NAME="CentOS Stream" 2024-09-11T21:36:13.937 INFO:teuthology.orchestra.run.smithi001.stdout:VERSION="9" 2024-09-11T21:36:13.938 INFO:teuthology.orchestra.run.smithi001.stdout:ID="centos" 2024-09-11T21:36:13.938 INFO:teuthology.orchestra.run.smithi001.stdout:ID_LIKE="rhel fedora" 2024-09-11T21:36:13.938 INFO:teuthology.orchestra.run.smithi001.stdout:VERSION_ID="9" 2024-09-11T21:36:13.938 INFO:teuthology.orchestra.run.smithi001.stdout:PLATFORM_ID="platform:el9" 2024-09-11T21:36:13.938 INFO:teuthology.orchestra.run.smithi001.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-11T21:36:13.938 INFO:teuthology.orchestra.run.smithi001.stdout:ANSI_COLOR="0;31" 2024-09-11T21:36:13.938 INFO:teuthology.orchestra.run.smithi001.stdout:LOGO="fedora-logo-icon" 2024-09-11T21:36:13.938 INFO:teuthology.orchestra.run.smithi001.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-11T21:36:13.938 INFO:teuthology.orchestra.run.smithi001.stdout:HOME_URL="https://centos.org/" 2024-09-11T21:36:13.938 INFO:teuthology.orchestra.run.smithi001.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-11T21:36:13.938 INFO:teuthology.orchestra.run.smithi001.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-11T21:36:13.938 INFO:teuthology.orchestra.run.smithi001.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-11T21:36:13.939 INFO:teuthology.lock.ops:Updating smithi001.front.sepia.ceph.com on lock server 2024-09-11T21:36:13.962 DEBUG:teuthology.orchestra.run.smithi053:> uname -m 2024-09-11T21:36:13.980 INFO:teuthology.orchestra.run.smithi053.stdout:x86_64 2024-09-11T21:36:13.980 DEBUG:teuthology.orchestra.run.smithi053:> cat /etc/os-release 2024-09-11T21:36:14.038 INFO:teuthology.orchestra.run.smithi053.stdout:NAME="CentOS Stream" 2024-09-11T21:36:14.039 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION="9" 2024-09-11T21:36:14.039 INFO:teuthology.orchestra.run.smithi053.stdout:ID="centos" 2024-09-11T21:36:14.039 INFO:teuthology.orchestra.run.smithi053.stdout:ID_LIKE="rhel fedora" 2024-09-11T21:36:14.039 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION_ID="9" 2024-09-11T21:36:14.039 INFO:teuthology.orchestra.run.smithi053.stdout:PLATFORM_ID="platform:el9" 2024-09-11T21:36:14.039 INFO:teuthology.orchestra.run.smithi053.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-11T21:36:14.039 INFO:teuthology.orchestra.run.smithi053.stdout:ANSI_COLOR="0;31" 2024-09-11T21:36:14.039 INFO:teuthology.orchestra.run.smithi053.stdout:LOGO="fedora-logo-icon" 2024-09-11T21:36:14.040 INFO:teuthology.orchestra.run.smithi053.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-11T21:36:14.040 INFO:teuthology.orchestra.run.smithi053.stdout:HOME_URL="https://centos.org/" 2024-09-11T21:36:14.040 INFO:teuthology.orchestra.run.smithi053.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-11T21:36:14.040 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-11T21:36:14.040 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-11T21:36:14.040 INFO:teuthology.lock.ops:Updating smithi053.front.sepia.ceph.com on lock server 2024-09-11T21:36:14.062 DEBUG:teuthology.orchestra.run.smithi161:> uname -m 2024-09-11T21:36:14.080 INFO:teuthology.orchestra.run.smithi161.stdout:x86_64 2024-09-11T21:36:14.080 DEBUG:teuthology.orchestra.run.smithi161:> cat /etc/os-release 2024-09-11T21:36:14.137 INFO:teuthology.orchestra.run.smithi161.stdout:NAME="CentOS Stream" 2024-09-11T21:36:14.137 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION="9" 2024-09-11T21:36:14.138 INFO:teuthology.orchestra.run.smithi161.stdout:ID="centos" 2024-09-11T21:36:14.138 INFO:teuthology.orchestra.run.smithi161.stdout:ID_LIKE="rhel fedora" 2024-09-11T21:36:14.138 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION_ID="9" 2024-09-11T21:36:14.138 INFO:teuthology.orchestra.run.smithi161.stdout:PLATFORM_ID="platform:el9" 2024-09-11T21:36:14.138 INFO:teuthology.orchestra.run.smithi161.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-11T21:36:14.138 INFO:teuthology.orchestra.run.smithi161.stdout:ANSI_COLOR="0;31" 2024-09-11T21:36:14.138 INFO:teuthology.orchestra.run.smithi161.stdout:LOGO="fedora-logo-icon" 2024-09-11T21:36:14.138 INFO:teuthology.orchestra.run.smithi161.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-11T21:36:14.138 INFO:teuthology.orchestra.run.smithi161.stdout:HOME_URL="https://centos.org/" 2024-09-11T21:36:14.138 INFO:teuthology.orchestra.run.smithi161.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-11T21:36:14.138 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-11T21:36:14.138 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-11T21:36:14.139 INFO:teuthology.lock.ops:Updating smithi161.front.sepia.ceph.com on lock server 2024-09-11T21:36:14.161 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-11T21:36:14.171 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-11T21:36:14.242 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-11T21:36:14.243 DEBUG:teuthology.orchestra.run.smithi001:> test '!' -e /home/ubuntu/cephtest 2024-09-11T21:36:14.245 DEBUG:teuthology.orchestra.run.smithi053:> test '!' -e /home/ubuntu/cephtest 2024-09-11T21:36:14.247 DEBUG:teuthology.orchestra.run.smithi161:> test '!' -e /home/ubuntu/cephtest 2024-09-11T21:36:14.264 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-11T21:36:14.362 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-11T21:36:14.363 DEBUG:teuthology.orchestra.run.smithi001:> test -z $(ls -A /var/lib/ceph) 2024-09-11T21:36:14.365 DEBUG:teuthology.orchestra.run.smithi053:> test -z $(ls -A /var/lib/ceph) 2024-09-11T21:36:14.367 DEBUG:teuthology.orchestra.run.smithi161:> test -z $(ls -A /var/lib/ceph) 2024-09-11T21:36:14.460 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-11T21:36:14.539 INFO:teuthology.run_tasks:Running task kernel... 2024-09-11T21:36:14.557 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-11T21:36:14.557 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-09-11T21:36:14.558 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-09-11T21:36:14.558 DEBUG:teuthology.orchestra.run.smithi001:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-11T21:36:14.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T21:36:14.575 DEBUG:teuthology.orchestra.run.smithi001:> uname -r 2024-09-11T21:36:14.632 INFO:teuthology.orchestra.run.smithi001.stdout:5.14.0-503.el9.x86_64 2024-09-11T21:36:14.632 INFO:teuthology.task.kernel:Running kernel on smithi001: 5.14.0-503.el9.x86_64 2024-09-11T21:36:14.632 DEBUG:teuthology.orchestra.run.smithi001:> sudo yum install -y kernel 2024-09-11T21:36:18.450 INFO:teuthology.orchestra.run.smithi001.stdout:CentOS Stream 9 - BaseOS 4.0 MB/s | 8.2 MB 00:02 2024-09-11T21:36:21.710 INFO:teuthology.orchestra.run.smithi001.stdout:CentOS Stream 9 - AppStream 9.8 MB/s | 20 MB 00:02 2024-09-11T21:36:27.085 INFO:teuthology.orchestra.run.smithi001.stdout:CentOS Stream 9 - CRB 4.4 MB/s | 6.5 MB 00:01 2024-09-11T21:36:28.910 INFO:teuthology.orchestra.run.smithi001.stdout:CentOS Stream 9 - Extras packages 33 kB/s | 19 kB 00:00 2024-09-11T21:36:30.073 INFO:teuthology.orchestra.run.smithi001.stdout:Extra Packages for Enterprise Linux 22 MB/s | 23 MB 00:01 2024-09-11T21:36:35.662 INFO:teuthology.orchestra.run.smithi001.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-11T21:36:37.777 INFO:teuthology.orchestra.run.smithi001.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-11T21:36:37.778 INFO:teuthology.orchestra.run.smithi001.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-11T21:36:37.868 INFO:teuthology.orchestra.run.smithi001.stdout:Dependencies resolved. 2024-09-11T21:36:37.884 INFO:teuthology.orchestra.run.smithi001.stdout:================================================================================ 2024-09-11T21:36:37.885 INFO:teuthology.orchestra.run.smithi001.stdout: Package Architecture Version Repository Size 2024-09-11T21:36:37.885 INFO:teuthology.orchestra.run.smithi001.stdout:================================================================================ 2024-09-11T21:36:37.885 INFO:teuthology.orchestra.run.smithi001.stdout:Installing: 2024-09-11T21:36:37.885 INFO:teuthology.orchestra.run.smithi001.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-11T21:36:37.885 INFO:teuthology.orchestra.run.smithi001.stdout:Installing dependencies: 2024-09-11T21:36:37.885 INFO:teuthology.orchestra.run.smithi001.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-11T21:36:37.885 INFO:teuthology.orchestra.run.smithi001.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-11T21:36:37.885 INFO:teuthology.orchestra.run.smithi001.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-11T21:36:37.885 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:36:37.885 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction Summary 2024-09-11T21:36:37.885 INFO:teuthology.orchestra.run.smithi001.stdout:================================================================================ 2024-09-11T21:36:37.885 INFO:teuthology.orchestra.run.smithi001.stdout:Install 4 Packages 2024-09-11T21:36:37.885 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:36:37.886 INFO:teuthology.orchestra.run.smithi001.stdout:Total download size: 79 M 2024-09-11T21:36:37.886 INFO:teuthology.orchestra.run.smithi001.stdout:Installed size: 126 M 2024-09-11T21:36:37.886 INFO:teuthology.orchestra.run.smithi001.stdout:Downloading Packages: 2024-09-11T21:36:38.597 INFO:teuthology.orchestra.run.smithi001.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 38 kB/s | 23 kB 00:00 2024-09-11T21:36:40.031 INFO:teuthology.orchestra.run.smithi001.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 7.7 MB/s | 16 MB 00:02 2024-09-11T21:36:41.064 INFO:teuthology.orchestra.run.smithi001.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 11 MB/s | 35 MB 00:03 2024-09-11T21:36:42.056 INFO:teuthology.orchestra.run.smithi001.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 8.3 MB/s | 29 MB 00:03 2024-09-11T21:36:42.056 INFO:teuthology.orchestra.run.smithi001.stdout:-------------------------------------------------------------------------------- 2024-09-11T21:36:42.057 INFO:teuthology.orchestra.run.smithi001.stdout:Total 19 MB/s | 79 MB 00:04 2024-09-11T21:36:42.767 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction check 2024-09-11T21:36:43.164 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction check succeeded. 2024-09-11T21:36:43.164 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction test 2024-09-11T21:36:44.133 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction test succeeded. 2024-09-11T21:36:44.134 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction 2024-09-11T21:36:46.742 INFO:teuthology.orchestra.run.smithi001.stdout: Preparing : 1/1 2024-09-11T21:36:47.424 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-11T21:36:47.514 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T21:36:50.305 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T21:36:50.452 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T21:36:55.576 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T21:36:55.611 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T21:37:00.761 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T21:37:39.620 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T21:37:39.662 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-11T21:37:40.030 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T21:37:40.030 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-11T21:37:40.030 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T21:37:40.031 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T21:37:40.927 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T21:37:40.927 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:37:40.927 INFO:teuthology.orchestra.run.smithi001.stdout:Installed: 2024-09-11T21:37:40.927 INFO:teuthology.orchestra.run.smithi001.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-11T21:37:40.927 INFO:teuthology.orchestra.run.smithi001.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-11T21:37:40.927 INFO:teuthology.orchestra.run.smithi001.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-11T21:37:40.927 INFO:teuthology.orchestra.run.smithi001.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-11T21:37:40.928 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:37:40.928 INFO:teuthology.orchestra.run.smithi001.stdout:Complete! 2024-09-11T21:37:41.260 DEBUG:teuthology.orchestra.run.smithi001:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T21:37:41.383 INFO:teuthology.orchestra.run.smithi001.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T21:37:41.383 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T21:37:41.383 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-11T21:37:41.383 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-11T21:37:41.383 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-11T21:37:41.383 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-11T21:37:41.384 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi001.front.sepia.ceph.com, path=None, version=distro) 2024-09-11T21:37:41.384 DEBUG:teuthology.orchestra.run.smithi001:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T21:37:41.441 INFO:teuthology.orchestra.run.smithi001.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T21:37:41.442 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T21:37:41.442 DEBUG:teuthology.orchestra.run.smithi001:> sudo rpm -qi grub2-tools 2024-09-11T21:37:41.487 INFO:teuthology.orchestra.run.smithi001.stdout:Name : grub2-tools 2024-09-11T21:37:41.487 INFO:teuthology.orchestra.run.smithi001.stdout:Epoch : 1 2024-09-11T21:37:41.487 INFO:teuthology.orchestra.run.smithi001.stdout:Version : 2.06 2024-09-11T21:37:41.487 INFO:teuthology.orchestra.run.smithi001.stdout:Release : 82.el9 2024-09-11T21:37:41.487 INFO:teuthology.orchestra.run.smithi001.stdout:Architecture: x86_64 2024-09-11T21:37:41.487 INFO:teuthology.orchestra.run.smithi001.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-11T21:37:41.487 INFO:teuthology.orchestra.run.smithi001.stdout:Group : Unspecified 2024-09-11T21:37:41.487 INFO:teuthology.orchestra.run.smithi001.stdout:Size : 8274670 2024-09-11T21:37:41.487 INFO:teuthology.orchestra.run.smithi001.stdout:License : GPLv3+ 2024-09-11T21:37:41.487 INFO:teuthology.orchestra.run.smithi001.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-11T21:37:41.487 INFO:teuthology.orchestra.run.smithi001.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-11T21:37:41.487 INFO:teuthology.orchestra.run.smithi001.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-11T21:37:41.487 INFO:teuthology.orchestra.run.smithi001.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-11T21:37:41.488 INFO:teuthology.orchestra.run.smithi001.stdout:Packager : builder@centos.org 2024-09-11T21:37:41.488 INFO:teuthology.orchestra.run.smithi001.stdout:Vendor : CentOS 2024-09-11T21:37:41.488 INFO:teuthology.orchestra.run.smithi001.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-11T21:37:41.488 INFO:teuthology.orchestra.run.smithi001.stdout:Summary : Support tools for GRUB. 2024-09-11T21:37:41.488 INFO:teuthology.orchestra.run.smithi001.stdout:Description : 2024-09-11T21:37:41.488 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:37:41.488 INFO:teuthology.orchestra.run.smithi001.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-11T21:37:41.488 INFO:teuthology.orchestra.run.smithi001.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-11T21:37:41.488 INFO:teuthology.orchestra.run.smithi001.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-11T21:37:41.488 INFO:teuthology.orchestra.run.smithi001.stdout:hardware devices. 2024-09-11T21:37:41.488 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:37:41.488 INFO:teuthology.orchestra.run.smithi001.stdout:This subpackage provides tools for support of all platforms. 2024-09-11T21:37:41.489 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-11T21:37:41.489 INFO:teuthology.task.kernel:Updating grub on smithi001 to boot 5.14.0-505.el9.x86_64 2024-09-11T21:37:41.490 DEBUG:teuthology.orchestra.run.smithi001:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-11T21:37:42.173 INFO:teuthology.orchestra.run.smithi001.stderr:Generating grub configuration file ... 2024-09-11T21:37:43.634 INFO:teuthology.orchestra.run.smithi001.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-11T21:37:43.659 INFO:teuthology.orchestra.run.smithi001.stderr:done 2024-09-11T21:37:43.661 DEBUG:teuthology.orchestra.run.smithi001:> mktemp 2024-09-11T21:37:43.676 INFO:teuthology.orchestra.run.smithi001.stdout:/tmp/tmp.F0p6ezJRow 2024-09-11T21:37:43.676 DEBUG:teuthology.orchestra.run.smithi001:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.F0p6ezJRow 2024-09-11T21:37:43.742 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 0666 /tmp/tmp.F0p6ezJRow 2024-09-11T21:37:43.879 DEBUG:teuthology.orchestra.remote:smithi001:/tmp/tmp.F0p6ezJRow is 6KB 2024-09-11T21:37:43.927 DEBUG:teuthology.orchestra.run.smithi001:> rm -fr /tmp/tmp.F0p6ezJRow 2024-09-11T21:37:43.941 DEBUG:teuthology.orchestra.run.smithi001:> sudo /bin/ls /boot/loader/entries || true 2024-09-11T21:37:44.003 INFO:teuthology.orchestra.run.smithi001.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-11T21:37:44.004 INFO:teuthology.orchestra.run.smithi001.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-11T21:37:44.004 INFO:teuthology.orchestra.run.smithi001.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-11T21:37:44.004 INFO:teuthology.orchestra.run.smithi001.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-11T21:37:44.005 DEBUG:teuthology.orchestra.run.smithi001:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-11T21:37:44.162 DEBUG:teuthology.orchestra.run.smithi001:> sudo shutdown -r now 2024-09-11T21:38:14.165 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-11T21:38:14.166 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi001.front.sepia.ceph.com' 2024-09-11T21:38:14.167 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi001.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T21:38:32.637 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.1 2024-09-11T21:38:41.638 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi001.front.sepia.ceph.com' 2024-09-11T21:38:41.639 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi001.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T21:38:44.705 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.1 2024-09-11T21:38:56.706 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi001.front.sepia.ceph.com' 2024-09-11T21:38:56.707 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi001.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T21:38:59.773 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.1 2024-09-11T21:39:14.774 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi001.front.sepia.ceph.com' 2024-09-11T21:39:14.774 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi001.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T21:39:17.853 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.1 2024-09-11T21:39:35.854 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi001.front.sepia.ceph.com' 2024-09-11T21:39:35.855 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi001.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T21:39:54.301 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.1 2024-09-11T21:40:15.302 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi001.front.sepia.ceph.com' 2024-09-11T21:40:15.303 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi001.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T21:40:15.585 DEBUG:teuthology.orchestra.run.smithi001:> true 2024-09-11T21:40:16.002 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi001.front.sepia.ceph.com' 2024-09-11T21:40:16.002 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-505.el9.x86_64"... 2024-09-11T21:40:16.002 DEBUG:teuthology.orchestra.run.smithi001:> uname -r 2024-09-11T21:40:16.019 INFO:teuthology.orchestra.run.smithi001.stdout:5.14.0-505.el9.x86_64 2024-09-11T21:40:16.019 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-11T21:40:16.020 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-11T21:40:16.020 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-11T21:40:17.021 DEBUG:teuthology.parallel:result is None 2024-09-11T21:40:17.021 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-11T21:40:17.032 INFO:teuthology.task.internal:Creating test directory... 2024-09-11T21:40:17.033 DEBUG:teuthology.orchestra.run.smithi001:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-11T21:40:17.035 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-11T21:40:17.038 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-11T21:40:17.057 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-11T21:40:17.066 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-11T21:40:17.080 INFO:teuthology.task.internal:Creating archive directory... 2024-09-11T21:40:17.081 DEBUG:teuthology.orchestra.run.smithi001:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-11T21:40:17.094 DEBUG:teuthology.orchestra.run.smithi053:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-11T21:40:17.097 DEBUG:teuthology.orchestra.run.smithi161:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-11T21:40:17.154 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-11T21:40:17.166 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-11T21:40:17.166 DEBUG:teuthology.orchestra.run.smithi053:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-11T21:40:17.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T21:40:17.183 DEBUG:teuthology.orchestra.run.smithi161:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-11T21:40:17.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T21:40:17.201 DEBUG:teuthology.orchestra.run.smithi001:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-11T21:40:17.204 DEBUG:teuthology.orchestra.run.smithi053:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-11T21:40:17.226 DEBUG:teuthology.orchestra.run.smithi161:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-11T21:40:17.269 INFO:teuthology.orchestra.run.smithi053.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T21:40:17.288 INFO:teuthology.orchestra.run.smithi053.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T21:40:17.296 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T21:40:17.306 INFO:teuthology.orchestra.run.smithi001.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T21:40:17.316 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T21:40:17.337 INFO:teuthology.orchestra.run.smithi001.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T21:40:17.339 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-11T21:40:17.348 INFO:teuthology.task.internal:Configuring sudo... 2024-09-11T21:40:17.348 DEBUG:teuthology.orchestra.run.smithi001:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-11T21:40:17.350 DEBUG:teuthology.orchestra.run.smithi053:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-11T21:40:17.354 DEBUG:teuthology.orchestra.run.smithi161:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-11T21:40:17.382 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-11T21:40:17.438 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-11T21:40:17.438 DEBUG:teuthology.orchestra.run.smithi001:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-11T21:40:17.440 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-11T21:40:17.443 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-11T21:40:17.459 DEBUG:teuthology.orchestra.run.smithi001:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T21:40:17.543 DEBUG:teuthology.orchestra.run.smithi001:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T21:40:17.597 DEBUG:teuthology.orchestra.run.smithi001:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T21:40:17.680 DEBUG:teuthology.orchestra.run.smithi001:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T21:40:17.724 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-11T21:40:17.724 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-11T21:40:17.791 DEBUG:teuthology.orchestra.run.smithi053:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T21:40:17.835 DEBUG:teuthology.orchestra.run.smithi053:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T21:40:17.889 DEBUG:teuthology.orchestra.run.smithi053:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T21:40:17.973 DEBUG:teuthology.orchestra.run.smithi053:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T21:40:18.016 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-11T21:40:18.016 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-11T21:40:18.085 DEBUG:teuthology.orchestra.run.smithi161:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T21:40:18.130 DEBUG:teuthology.orchestra.run.smithi161:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T21:40:18.186 DEBUG:teuthology.orchestra.run.smithi161:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T21:40:18.272 DEBUG:teuthology.orchestra.run.smithi161:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T21:40:18.316 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-09-11T21:40:18.316 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-11T21:40:18.385 DEBUG:teuthology.orchestra.run.smithi001:> sudo service rsyslog restart 2024-09-11T21:40:18.388 DEBUG:teuthology.orchestra.run.smithi053:> sudo service rsyslog restart 2024-09-11T21:40:18.390 DEBUG:teuthology.orchestra.run.smithi161:> sudo service rsyslog restart 2024-09-11T21:40:18.430 INFO:teuthology.orchestra.run.smithi053.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T21:40:18.431 INFO:teuthology.orchestra.run.smithi001.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T21:40:18.468 INFO:teuthology.orchestra.run.smithi161.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T21:40:18.952 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-11T21:40:18.960 INFO:teuthology.task.internal:Starting timer... 2024-09-11T21:40:18.960 INFO:teuthology.run_tasks:Running task pcp... 2024-09-11T21:40:18.972 INFO:teuthology.run_tasks:Running task selinux... 2024-09-11T21:40:18.980 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-11T21:40:18.980 DEBUG:teuthology.orchestra.run.smithi001:> sudo service auditd rotate 2024-09-11T21:40:19.037 INFO:teuthology.orchestra.run.smithi001.stdout:Rotating logs: 2024-09-11T21:40:19.039 DEBUG:teuthology.orchestra.run.smithi053:> sudo service auditd rotate 2024-09-11T21:40:19.098 INFO:teuthology.orchestra.run.smithi053.stdout:Rotating logs: 2024-09-11T21:40:19.100 DEBUG:teuthology.orchestra.run.smithi161:> sudo service auditd rotate 2024-09-11T21:40:19.144 INFO:teuthology.orchestra.run.smithi161.stdout:Rotating logs: 2024-09-11T21:40:19.146 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-11T21:40:19.146 DEBUG:teuthology.orchestra.run.smithi001:> /usr/sbin/getenforce 2024-09-11T21:40:19.171 INFO:teuthology.orchestra.run.smithi001.stdout:Permissive 2024-09-11T21:40:19.172 DEBUG:teuthology.orchestra.run.smithi053:> /usr/sbin/getenforce 2024-09-11T21:40:19.198 INFO:teuthology.orchestra.run.smithi053.stdout:Permissive 2024-09-11T21:40:19.198 DEBUG:teuthology.orchestra.run.smithi161:> /usr/sbin/getenforce 2024-09-11T21:40:19.221 INFO:teuthology.orchestra.run.smithi161.stdout:Permissive 2024-09-11T21:40:19.222 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi001.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi053.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi161.front.sepia.ceph.com': 'permissive'} 2024-09-11T21:40:19.222 DEBUG:teuthology.orchestra.run.smithi001:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-11T21:40:19.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T21:40:19.250 DEBUG:teuthology.orchestra.run.smithi053:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-11T21:40:19.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T21:40:19.279 DEBUG:teuthology.orchestra.run.smithi161:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-11T21:40:19.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T21:40:19.310 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-11T21:40:19.310 DEBUG:teuthology.orchestra.run.smithi001:> sudo /usr/sbin/setenforce permissive 2024-09-11T21:40:19.343 DEBUG:teuthology.orchestra.run.smithi053:> sudo /usr/sbin/setenforce permissive 2024-09-11T21:40:19.376 DEBUG:teuthology.orchestra.run.smithi161:> sudo /usr/sbin/setenforce permissive 2024-09-11T21:40:19.408 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-11T21:40:19.418 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-11T21:40:19.431 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-11T21:40:19.432 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi001.front.sepia.ceph.com,smithi053.front.sepia.ceph.com,smithi161.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-11T21:44:21.536 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi001.front.sepia.ceph.com'), Remote(name='ubuntu@smithi053.front.sepia.ceph.com'), Remote(name='ubuntu@smithi161.front.sepia.ceph.com')] 2024-09-11T21:44:21.537 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi001.front.sepia.ceph.com' 2024-09-11T21:44:21.539 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi001.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T21:44:21.617 DEBUG:teuthology.orchestra.run.smithi001:> true 2024-09-11T21:44:21.701 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi001.front.sepia.ceph.com' 2024-09-11T21:44:21.702 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-09-11T21:44:21.702 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T21:44:21.781 DEBUG:teuthology.orchestra.run.smithi053:> true 2024-09-11T21:44:21.865 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-09-11T21:44:21.866 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-09-11T21:44:21.866 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T21:44:21.944 DEBUG:teuthology.orchestra.run.smithi161:> true 2024-09-11T21:44:22.024 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-09-11T21:44:22.025 INFO:teuthology.run_tasks:Running task clock... 2024-09-11T21:44:22.037 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-11T21:44:22.038 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-11T21:44:22.038 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T21:44:22.041 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-11T21:44:22.041 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T21:44:22.043 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-11T21:44:22.044 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T21:44:22.068 INFO:teuthology.orchestra.run.smithi001.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-11T21:44:22.074 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-11T21:44:22.083 INFO:teuthology.orchestra.run.smithi001.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-11T21:44:22.091 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-11T21:44:22.092 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-11T21:44:22.107 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-11T21:44:22.109 INFO:teuthology.orchestra.run.smithi001.stderr:sudo: ntpd: command not found 2024-09-11T21:44:22.117 INFO:teuthology.orchestra.run.smithi053.stderr:sudo: ntpd: command not found 2024-09-11T21:44:22.121 INFO:teuthology.orchestra.run.smithi001.stdout:506 Cannot talk to daemon 2024-09-11T21:44:22.128 INFO:teuthology.orchestra.run.smithi053.stdout:506 Cannot talk to daemon 2024-09-11T21:44:22.131 INFO:teuthology.orchestra.run.smithi161.stderr:sudo: ntpd: command not found 2024-09-11T21:44:22.136 INFO:teuthology.orchestra.run.smithi001.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-11T21:44:22.143 INFO:teuthology.orchestra.run.smithi161.stdout:506 Cannot talk to daemon 2024-09-11T21:44:22.143 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-11T21:44:22.150 INFO:teuthology.orchestra.run.smithi001.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-11T21:44:22.157 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-11T21:44:22.158 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-11T21:44:22.171 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-11T21:44:22.207 INFO:teuthology.orchestra.run.smithi001.stderr:bash: line 1: ntpq: command not found 2024-09-11T21:44:22.211 INFO:teuthology.orchestra.run.smithi001.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T21:44:22.211 INFO:teuthology.orchestra.run.smithi001.stdout:=============================================================================== 2024-09-11T21:44:22.211 INFO:teuthology.orchestra.run.smithi001.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T21:44:22.211 INFO:teuthology.orchestra.run.smithi001.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T21:44:22.211 INFO:teuthology.orchestra.run.smithi001.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T21:44:22.211 INFO:teuthology.orchestra.run.smithi001.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T21:44:22.212 INFO:teuthology.orchestra.run.smithi053.stderr:bash: line 1: ntpq: command not found 2024-09-11T21:44:22.215 INFO:teuthology.orchestra.run.smithi053.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T21:44:22.215 INFO:teuthology.orchestra.run.smithi053.stdout:=============================================================================== 2024-09-11T21:44:22.216 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T21:44:22.216 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T21:44:22.216 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T21:44:22.216 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T21:44:22.234 INFO:teuthology.orchestra.run.smithi161.stderr:bash: line 1: ntpq: command not found 2024-09-11T21:44:22.238 INFO:teuthology.orchestra.run.smithi161.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T21:44:22.238 INFO:teuthology.orchestra.run.smithi161.stdout:=============================================================================== 2024-09-11T21:44:22.238 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T21:44:22.238 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T21:44:22.238 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T21:44:22.238 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T21:44:22.239 INFO:teuthology.run_tasks:Running task install... 2024-09-11T21:44:22.256 DEBUG:teuthology.task.install:project ceph 2024-09-11T21:44:22.256 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'}} 2024-09-11T21:44:22.257 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'} 2024-09-11T21:44:22.257 INFO:teuthology.task.install:Using flavor: default 2024-09-11T21:44:22.262 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-11T21:44:22.262 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-11T21:44:22.263 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-11T21:44:22.263 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T21:44:22.265 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-11T21:44:22.265 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T21:44:22.266 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-11T21:44:22.266 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T21:44:22.405 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-11T21:44:22.405 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-11T21:44:22.418 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-11T21:44:22.418 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-11T21:44:22.423 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-11T21:44:22.424 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-11T21:44:22.516 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-11T21:44:22.516 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-11T21:44:22.516 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-11T21:44:22.535 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-11T21:44:22.535 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-11T21:44:22.535 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-11T21:44:22.537 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-11T21:44:22.537 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-09-11T21:44:22.537 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-11T21:44:22.544 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-11T21:44:22.544 DEBUG:teuthology.orchestra.run.smithi001:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-11T21:44:22.563 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-11T21:44:22.563 DEBUG:teuthology.orchestra.run.smithi053:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-11T21:44:22.564 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-11T21:44:22.565 DEBUG:teuthology.orchestra.run.smithi161:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-11T21:44:22.611 DEBUG:teuthology.orchestra.run.smithi001:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-11T21:44:22.630 DEBUG:teuthology.orchestra.run.smithi053:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-11T21:44:22.632 DEBUG:teuthology.orchestra.run.smithi161:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-11T21:44:22.688 DEBUG:teuthology.orchestra.run.smithi001:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-11T21:44:22.705 DEBUG:teuthology.orchestra.run.smithi053:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-11T21:44:22.707 DEBUG:teuthology.orchestra.run.smithi161:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-11T21:44:22.753 INFO:teuthology.orchestra.run.smithi001.stdout:check_obsoletes = 1 2024-09-11T21:44:22.755 DEBUG:teuthology.orchestra.run.smithi001:> sudo yum clean all 2024-09-11T21:44:22.769 INFO:teuthology.orchestra.run.smithi053.stdout:check_obsoletes = 1 2024-09-11T21:44:22.771 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum clean all 2024-09-11T21:44:22.772 INFO:teuthology.orchestra.run.smithi161.stdout:check_obsoletes = 1 2024-09-11T21:44:22.773 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum clean all 2024-09-11T21:44:22.992 INFO:teuthology.orchestra.run.smithi001.stdout:45 files removed 2024-09-11T21:44:23.000 INFO:teuthology.orchestra.run.smithi161.stdout:45 files removed 2024-09-11T21:44:23.001 INFO:teuthology.orchestra.run.smithi053.stdout:45 files removed 2024-09-11T21:44:23.019 DEBUG:teuthology.orchestra.run.smithi001:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-11T21:44:23.028 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-11T21:44:23.029 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-11T21:44:23.695 INFO:teuthology.orchestra.run.smithi053.stdout:ceph packages for x86_64 219 kB/s | 84 kB 00:00 2024-09-11T21:44:23.698 INFO:teuthology.orchestra.run.smithi161.stdout:ceph packages for x86_64 214 kB/s | 84 kB 00:00 2024-09-11T21:44:23.718 INFO:teuthology.orchestra.run.smithi001.stdout:ceph packages for x86_64 209 kB/s | 84 kB 00:00 2024-09-11T21:44:24.020 INFO:teuthology.orchestra.run.smithi053.stdout:ceph noarch packages 42 kB/s | 12 kB 00:00 2024-09-11T21:44:24.024 INFO:teuthology.orchestra.run.smithi161.stdout:ceph noarch packages 42 kB/s | 12 kB 00:00 2024-09-11T21:44:24.044 INFO:teuthology.orchestra.run.smithi001.stdout:ceph noarch packages 42 kB/s | 12 kB 00:00 2024-09-11T21:44:24.429 INFO:teuthology.orchestra.run.smithi053.stdout:ceph source packages 5.1 kB/s | 1.9 kB 00:00 2024-09-11T21:44:24.433 INFO:teuthology.orchestra.run.smithi161.stdout:ceph source packages 5.1 kB/s | 1.9 kB 00:00 2024-09-11T21:44:24.503 INFO:teuthology.orchestra.run.smithi001.stdout:ceph source packages 4.5 kB/s | 1.9 kB 00:00 2024-09-11T21:44:24.970 INFO:teuthology.orchestra.run.smithi001.stdout:CentOS Stream 9 - BaseOS 19 MB/s | 8.2 MB 00:00 2024-09-11T21:44:26.116 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS Stream 9 - BaseOS 5.0 MB/s | 8.2 MB 00:01 2024-09-11T21:44:26.229 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - BaseOS 4.7 MB/s | 8.2 MB 00:01 2024-09-11T21:44:29.381 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - AppStream 10 MB/s | 20 MB 00:02 2024-09-11T21:44:29.421 INFO:teuthology.orchestra.run.smithi001.stdout:CentOS Stream 9 - AppStream 6.2 MB/s | 20 MB 00:03 2024-09-11T21:44:32.108 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS Stream 9 - AppStream 4.2 MB/s | 20 MB 00:04 2024-09-11T21:44:33.864 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.5 MB 00:00 2024-09-11T21:44:35.941 INFO:teuthology.orchestra.run.smithi001.stdout:CentOS Stream 9 - CRB 2.6 MB/s | 6.5 MB 00:02 2024-09-11T21:44:36.373 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - Extras packages 15 kB/s | 19 kB 00:01 2024-09-11T21:44:36.767 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS Stream 9 - CRB 8.1 MB/s | 6.5 MB 00:00 2024-09-11T21:44:37.344 INFO:teuthology.orchestra.run.smithi053.stdout:Extra Packages for Enterprise Linux 27 MB/s | 23 MB 00:00 2024-09-11T21:44:37.741 INFO:teuthology.orchestra.run.smithi001.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 19 kB 00:00 2024-09-11T21:44:38.134 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS Stream 9 - Extras packages 115 kB/s | 19 kB 00:00 2024-09-11T21:44:38.712 INFO:teuthology.orchestra.run.smithi001.stdout:Extra Packages for Enterprise Linux 27 MB/s | 23 MB 00:00 2024-09-11T21:44:38.939 INFO:teuthology.orchestra.run.smithi161.stdout:Extra Packages for Enterprise Linux 33 MB/s | 23 MB 00:00 2024-09-11T21:44:43.000 INFO:teuthology.orchestra.run.smithi053.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-11T21:44:44.368 INFO:teuthology.orchestra.run.smithi001.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-09-11T21:44:44.419 INFO:teuthology.orchestra.run.smithi161.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-11T21:44:45.380 INFO:teuthology.orchestra.run.smithi053.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-11T21:44:45.381 INFO:teuthology.orchestra.run.smithi053.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-11T21:44:45.395 INFO:teuthology.orchestra.run.smithi053.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-11T21:44:45.396 INFO:teuthology.orchestra.run.smithi053.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-11T21:44:45.397 INFO:teuthology.orchestra.run.smithi053.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-11T21:44:45.398 INFO:teuthology.orchestra.run.smithi053.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-11T21:44:45.400 INFO:teuthology.orchestra.run.smithi053.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-11T21:44:45.403 INFO:teuthology.orchestra.run.smithi053.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-11T21:44:45.404 INFO:teuthology.orchestra.run.smithi053.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-11T21:44:45.472 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-11T21:44:45.492 INFO:teuthology.orchestra.run.smithi053.stdout:====================================================================================== 2024-09-11T21:44:45.492 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repository Size 2024-09-11T21:44:45.492 INFO:teuthology.orchestra.run.smithi053.stdout:====================================================================================== 2024-09-11T21:44:45.492 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-09-11T21:44:45.492 INFO:teuthology.orchestra.run.smithi053.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-11T21:44:45.492 INFO:teuthology.orchestra.run.smithi053.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-11T21:44:45.492 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-11T21:44:45.492 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-11T21:44:45.492 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-11T21:44:45.492 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-11T21:44:45.493 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-11T21:44:45.493 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-11T21:44:45.493 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-11T21:44:45.493 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-11T21:44:45.493 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-11T21:44:45.493 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-11T21:44:45.493 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-11T21:44:45.493 INFO:teuthology.orchestra.run.smithi053.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-11T21:44:45.493 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 228 k 2024-09-11T21:44:45.493 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-shell x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 27 k 2024-09-11T21:44:45.493 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-top noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 18 k 2024-09-11T21:44:45.494 INFO:teuthology.orchestra.run.smithi053.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-11T21:44:45.494 INFO:teuthology.orchestra.run.smithi053.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-11T21:44:45.494 INFO:teuthology.orchestra.run.smithi053.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-11T21:44:45.494 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-11T21:44:45.494 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-11T21:44:45.494 INFO:teuthology.orchestra.run.smithi053.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-11T21:44:45.494 INFO:teuthology.orchestra.run.smithi053.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-11T21:44:45.494 INFO:teuthology.orchestra.run.smithi053.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-11T21:44:45.494 INFO:teuthology.orchestra.run.smithi053.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-11T21:44:45.494 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-11T21:44:45.494 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-11T21:44:45.494 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-11T21:44:45.495 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-11T21:44:45.495 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-11T21:44:45.495 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-11T21:44:45.495 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-11T21:44:45.495 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-11T21:44:45.495 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-11T21:44:45.495 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-11T21:44:45.495 INFO:teuthology.orchestra.run.smithi053.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-11T21:44:45.495 INFO:teuthology.orchestra.run.smithi053.stdout:Upgrading: 2024-09-11T21:44:45.495 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-11T21:44:45.495 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-11T21:44:45.495 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-11T21:44:45.496 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-11T21:44:45.496 INFO:teuthology.orchestra.run.smithi053.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-11T21:44:45.496 INFO:teuthology.orchestra.run.smithi053.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-11T21:44:45.496 INFO:teuthology.orchestra.run.smithi053.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-11T21:44:45.496 INFO:teuthology.orchestra.run.smithi053.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-11T21:44:45.496 INFO:teuthology.orchestra.run.smithi053.stdout:Installing dependencies: 2024-09-11T21:44:45.496 INFO:teuthology.orchestra.run.smithi053.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-11T21:44:45.496 INFO:teuthology.orchestra.run.smithi053.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-11T21:44:45.496 INFO:teuthology.orchestra.run.smithi053.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-11T21:44:45.496 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-11T21:44:45.496 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-11T21:44:45.496 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-11T21:44:45.497 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-11T21:44:45.497 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-11T21:44:45.497 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-11T21:44:45.497 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-11T21:44:45.497 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-11T21:44:45.497 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-11T21:44:45.497 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-11T21:44:45.497 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-11T21:44:45.497 INFO:teuthology.orchestra.run.smithi053.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-11T21:44:45.497 INFO:teuthology.orchestra.run.smithi053.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-11T21:44:45.497 INFO:teuthology.orchestra.run.smithi053.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-11T21:44:45.497 INFO:teuthology.orchestra.run.smithi053.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-11T21:44:45.498 INFO:teuthology.orchestra.run.smithi053.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-11T21:44:45.498 INFO:teuthology.orchestra.run.smithi053.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-11T21:44:45.498 INFO:teuthology.orchestra.run.smithi053.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-11T21:44:45.498 INFO:teuthology.orchestra.run.smithi053.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-11T21:44:45.498 INFO:teuthology.orchestra.run.smithi053.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-11T21:44:45.498 INFO:teuthology.orchestra.run.smithi053.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-11T21:44:45.498 INFO:teuthology.orchestra.run.smithi053.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-11T21:44:45.498 INFO:teuthology.orchestra.run.smithi053.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-11T21:44:45.498 INFO:teuthology.orchestra.run.smithi053.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-11T21:44:45.498 INFO:teuthology.orchestra.run.smithi053.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-11T21:44:45.498 INFO:teuthology.orchestra.run.smithi053.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-11T21:44:45.499 INFO:teuthology.orchestra.run.smithi053.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-11T21:44:45.499 INFO:teuthology.orchestra.run.smithi053.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-11T21:44:45.499 INFO:teuthology.orchestra.run.smithi053.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-11T21:44:45.499 INFO:teuthology.orchestra.run.smithi053.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-11T21:44:45.499 INFO:teuthology.orchestra.run.smithi053.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-11T21:44:45.499 INFO:teuthology.orchestra.run.smithi053.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-11T21:44:45.499 INFO:teuthology.orchestra.run.smithi053.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-11T21:44:45.499 INFO:teuthology.orchestra.run.smithi053.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-11T21:44:45.499 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-11T21:44:45.499 INFO:teuthology.orchestra.run.smithi053.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-11T21:44:45.499 INFO:teuthology.orchestra.run.smithi053.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-11T21:44:45.499 INFO:teuthology.orchestra.run.smithi053.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-11T21:44:45.499 INFO:teuthology.orchestra.run.smithi053.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-11T21:44:45.500 INFO:teuthology.orchestra.run.smithi053.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-11T21:44:45.500 INFO:teuthology.orchestra.run.smithi053.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-11T21:44:45.500 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-11T21:44:45.500 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-11T21:44:45.500 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-11T21:44:45.500 INFO:teuthology.orchestra.run.smithi053.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-11T21:44:45.500 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-11T21:44:45.500 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-11T21:44:45.500 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-11T21:44:45.500 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-11T21:44:45.500 INFO:teuthology.orchestra.run.smithi053.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-11T21:44:45.500 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-11T21:44:45.501 INFO:teuthology.orchestra.run.smithi053.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-11T21:44:45.501 INFO:teuthology.orchestra.run.smithi053.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-11T21:44:45.501 INFO:teuthology.orchestra.run.smithi053.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-11T21:44:45.501 INFO:teuthology.orchestra.run.smithi053.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-11T21:44:45.501 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-11T21:44:45.501 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-11T21:44:45.501 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-11T21:44:45.501 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-11T21:44:45.501 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-11T21:44:45.501 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-11T21:44:45.501 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-11T21:44:45.501 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-11T21:44:45.501 INFO:teuthology.orchestra.run.smithi053.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-11T21:44:45.502 INFO:teuthology.orchestra.run.smithi053.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-11T21:44:45.502 INFO:teuthology.orchestra.run.smithi053.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-11T21:44:45.502 INFO:teuthology.orchestra.run.smithi053.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-11T21:44:45.502 INFO:teuthology.orchestra.run.smithi053.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-11T21:44:45.502 INFO:teuthology.orchestra.run.smithi053.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-11T21:44:45.502 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-11T21:44:45.502 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-11T21:44:45.502 INFO:teuthology.orchestra.run.smithi053.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-11T21:44:45.502 INFO:teuthology.orchestra.run.smithi053.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-11T21:44:45.502 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-11T21:44:45.502 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-11T21:44:45.502 INFO:teuthology.orchestra.run.smithi053.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-11T21:44:45.502 INFO:teuthology.orchestra.run.smithi053.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-11T21:44:45.503 INFO:teuthology.orchestra.run.smithi053.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-11T21:44:45.503 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-11T21:44:45.503 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-11T21:44:45.503 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-11T21:44:45.503 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-11T21:44:45.503 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-11T21:44:45.503 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-11T21:44:45.503 INFO:teuthology.orchestra.run.smithi053.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-11T21:44:45.503 INFO:teuthology.orchestra.run.smithi053.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-11T21:44:45.503 INFO:teuthology.orchestra.run.smithi053.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-11T21:44:45.503 INFO:teuthology.orchestra.run.smithi053.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-11T21:44:45.503 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-11T21:44:45.504 INFO:teuthology.orchestra.run.smithi053.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-11T21:44:45.504 INFO:teuthology.orchestra.run.smithi053.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-11T21:44:45.504 INFO:teuthology.orchestra.run.smithi053.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-11T21:44:45.504 INFO:teuthology.orchestra.run.smithi053.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-11T21:44:45.504 INFO:teuthology.orchestra.run.smithi053.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-11T21:44:45.504 INFO:teuthology.orchestra.run.smithi053.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-11T21:44:45.504 INFO:teuthology.orchestra.run.smithi053.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-11T21:44:45.504 INFO:teuthology.orchestra.run.smithi053.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-11T21:44:45.504 INFO:teuthology.orchestra.run.smithi053.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-11T21:44:45.504 INFO:teuthology.orchestra.run.smithi053.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-11T21:44:45.504 INFO:teuthology.orchestra.run.smithi053.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-11T21:44:45.504 INFO:teuthology.orchestra.run.smithi053.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-11T21:44:45.505 INFO:teuthology.orchestra.run.smithi053.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-11T21:44:45.505 INFO:teuthology.orchestra.run.smithi053.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-11T21:44:45.505 INFO:teuthology.orchestra.run.smithi053.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-11T21:44:45.505 INFO:teuthology.orchestra.run.smithi053.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-11T21:44:45.505 INFO:teuthology.orchestra.run.smithi053.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-11T21:44:45.505 INFO:teuthology.orchestra.run.smithi053.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-11T21:44:45.505 INFO:teuthology.orchestra.run.smithi053.stdout:Installing weak dependencies: 2024-09-11T21:44:45.505 INFO:teuthology.orchestra.run.smithi053.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-11T21:44:45.505 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-11T21:44:45.505 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-09-11T21:44:45.505 INFO:teuthology.orchestra.run.smithi053.stdout:====================================================================================== 2024-09-11T21:44:45.506 INFO:teuthology.orchestra.run.smithi053.stdout:Install 148 Packages 2024-09-11T21:44:45.506 INFO:teuthology.orchestra.run.smithi053.stdout:Upgrade 8 Packages 2024-09-11T21:44:45.506 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-11T21:44:45.506 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 226 M 2024-09-11T21:44:45.506 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-09-11T21:44:46.401 INFO:teuthology.orchestra.run.smithi053.stdout:(1/156): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.r 51 kB/s | 6.5 kB 00:00 2024-09-11T21:44:46.627 INFO:teuthology.orchestra.run.smithi053.stdout:(2/156): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86 3.9 MB/s | 897 kB 00:00 2024-09-11T21:44:46.710 INFO:teuthology.orchestra.run.smithi053.stdout:(3/156): ceph-immutable-object-cache-19.1.1-298 1.7 MB/s | 145 kB 00:00 2024-09-11T21:44:46.779 INFO:teuthology.orchestra.run.smithi001.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-11T21:44:46.780 INFO:teuthology.orchestra.run.smithi001.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-11T21:44:46.785 INFO:teuthology.orchestra.run.smithi053.stdout:(4/156): ceph-base-19.1.1-298.gb53b6d3e.el9.x86 11 MB/s | 5.4 MB 00:00 2024-09-11T21:44:46.791 INFO:teuthology.orchestra.run.smithi001.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-11T21:44:46.792 INFO:teuthology.orchestra.run.smithi001.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-11T21:44:46.793 INFO:teuthology.orchestra.run.smithi001.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-11T21:44:46.794 INFO:teuthology.orchestra.run.smithi001.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-11T21:44:46.796 INFO:teuthology.orchestra.run.smithi001.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-11T21:44:46.798 INFO:teuthology.orchestra.run.smithi001.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-11T21:44:46.799 INFO:teuthology.orchestra.run.smithi001.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-11T21:44:46.813 INFO:teuthology.orchestra.run.smithi161.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-11T21:44:46.813 INFO:teuthology.orchestra.run.smithi161.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-11T21:44:46.824 INFO:teuthology.orchestra.run.smithi161.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-11T21:44:46.825 INFO:teuthology.orchestra.run.smithi161.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-11T21:44:46.826 INFO:teuthology.orchestra.run.smithi161.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-11T21:44:46.827 INFO:teuthology.orchestra.run.smithi161.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-11T21:44:46.829 INFO:teuthology.orchestra.run.smithi161.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-11T21:44:46.831 INFO:teuthology.orchestra.run.smithi161.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-11T21:44:46.832 INFO:teuthology.orchestra.run.smithi161.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-11T21:44:46.866 INFO:teuthology.orchestra.run.smithi001.stdout:Dependencies resolved. 2024-09-11T21:44:46.869 INFO:teuthology.orchestra.run.smithi053.stdout:(5/156): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_ 18 MB/s | 1.5 MB 00:00 2024-09-11T21:44:46.883 INFO:teuthology.orchestra.run.smithi001.stdout:====================================================================================== 2024-09-11T21:44:46.883 INFO:teuthology.orchestra.run.smithi001.stdout: Package Arch Version Repository Size 2024-09-11T21:44:46.883 INFO:teuthology.orchestra.run.smithi001.stdout:====================================================================================== 2024-09-11T21:44:46.883 INFO:teuthology.orchestra.run.smithi001.stdout:Installing: 2024-09-11T21:44:46.883 INFO:teuthology.orchestra.run.smithi001.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-11T21:44:46.884 INFO:teuthology.orchestra.run.smithi001.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-11T21:44:46.884 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-11T21:44:46.884 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-11T21:44:46.884 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-11T21:44:46.884 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-11T21:44:46.884 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-11T21:44:46.884 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-11T21:44:46.884 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-11T21:44:46.884 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-11T21:44:46.884 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-11T21:44:46.884 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-11T21:44:46.884 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-11T21:44:46.885 INFO:teuthology.orchestra.run.smithi001.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-11T21:44:46.885 INFO:teuthology.orchestra.run.smithi001.stdout: cephfs-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 228 k 2024-09-11T21:44:46.885 INFO:teuthology.orchestra.run.smithi001.stdout: cephfs-shell x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 27 k 2024-09-11T21:44:46.885 INFO:teuthology.orchestra.run.smithi001.stdout: cephfs-top noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 18 k 2024-09-11T21:44:46.885 INFO:teuthology.orchestra.run.smithi001.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-11T21:44:46.885 INFO:teuthology.orchestra.run.smithi001.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-11T21:44:46.885 INFO:teuthology.orchestra.run.smithi001.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-11T21:44:46.885 INFO:teuthology.orchestra.run.smithi001.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-11T21:44:46.885 INFO:teuthology.orchestra.run.smithi001.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-11T21:44:46.885 INFO:teuthology.orchestra.run.smithi001.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-11T21:44:46.885 INFO:teuthology.orchestra.run.smithi001.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-11T21:44:46.885 INFO:teuthology.orchestra.run.smithi001.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-11T21:44:46.885 INFO:teuthology.orchestra.run.smithi001.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-11T21:44:46.885 INFO:teuthology.orchestra.run.smithi001.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-11T21:44:46.886 INFO:teuthology.orchestra.run.smithi001.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-11T21:44:46.886 INFO:teuthology.orchestra.run.smithi001.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-11T21:44:46.886 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-11T21:44:46.886 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-11T21:44:46.886 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-11T21:44:46.886 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-11T21:44:46.886 INFO:teuthology.orchestra.run.smithi001.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-11T21:44:46.886 INFO:teuthology.orchestra.run.smithi001.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-11T21:44:46.886 INFO:teuthology.orchestra.run.smithi001.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-11T21:44:46.886 INFO:teuthology.orchestra.run.smithi001.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-11T21:44:46.886 INFO:teuthology.orchestra.run.smithi001.stdout:Upgrading: 2024-09-11T21:44:46.886 INFO:teuthology.orchestra.run.smithi001.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-11T21:44:46.886 INFO:teuthology.orchestra.run.smithi001.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-11T21:44:46.886 INFO:teuthology.orchestra.run.smithi001.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-11T21:44:46.887 INFO:teuthology.orchestra.run.smithi001.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-11T21:44:46.887 INFO:teuthology.orchestra.run.smithi001.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-11T21:44:46.887 INFO:teuthology.orchestra.run.smithi001.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-11T21:44:46.887 INFO:teuthology.orchestra.run.smithi001.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-11T21:44:46.887 INFO:teuthology.orchestra.run.smithi001.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-11T21:44:46.887 INFO:teuthology.orchestra.run.smithi001.stdout:Installing dependencies: 2024-09-11T21:44:46.887 INFO:teuthology.orchestra.run.smithi001.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-11T21:44:46.887 INFO:teuthology.orchestra.run.smithi001.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-11T21:44:46.887 INFO:teuthology.orchestra.run.smithi001.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-11T21:44:46.887 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-11T21:44:46.887 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-11T21:44:46.887 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-11T21:44:46.887 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-11T21:44:46.887 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-11T21:44:46.888 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-11T21:44:46.888 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-11T21:44:46.888 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-11T21:44:46.888 INFO:teuthology.orchestra.run.smithi001.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-11T21:44:46.888 INFO:teuthology.orchestra.run.smithi001.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-11T21:44:46.888 INFO:teuthology.orchestra.run.smithi001.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-11T21:44:46.888 INFO:teuthology.orchestra.run.smithi001.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-11T21:44:46.888 INFO:teuthology.orchestra.run.smithi001.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-11T21:44:46.888 INFO:teuthology.orchestra.run.smithi001.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-11T21:44:46.888 INFO:teuthology.orchestra.run.smithi001.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-11T21:44:46.888 INFO:teuthology.orchestra.run.smithi001.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-11T21:44:46.888 INFO:teuthology.orchestra.run.smithi001.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-11T21:44:46.888 INFO:teuthology.orchestra.run.smithi001.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-11T21:44:46.889 INFO:teuthology.orchestra.run.smithi001.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-11T21:44:46.889 INFO:teuthology.orchestra.run.smithi001.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-11T21:44:46.889 INFO:teuthology.orchestra.run.smithi001.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-11T21:44:46.889 INFO:teuthology.orchestra.run.smithi001.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-11T21:44:46.889 INFO:teuthology.orchestra.run.smithi001.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-11T21:44:46.889 INFO:teuthology.orchestra.run.smithi001.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-11T21:44:46.889 INFO:teuthology.orchestra.run.smithi001.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-11T21:44:46.889 INFO:teuthology.orchestra.run.smithi001.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-11T21:44:46.889 INFO:teuthology.orchestra.run.smithi001.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-11T21:44:46.889 INFO:teuthology.orchestra.run.smithi001.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-11T21:44:46.889 INFO:teuthology.orchestra.run.smithi001.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-11T21:44:46.889 INFO:teuthology.orchestra.run.smithi001.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-11T21:44:46.889 INFO:teuthology.orchestra.run.smithi001.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-11T21:44:46.890 INFO:teuthology.orchestra.run.smithi001.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-11T21:44:46.890 INFO:teuthology.orchestra.run.smithi001.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-11T21:44:46.890 INFO:teuthology.orchestra.run.smithi001.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-11T21:44:46.890 INFO:teuthology.orchestra.run.smithi001.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-11T21:44:46.890 INFO:teuthology.orchestra.run.smithi001.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-11T21:44:46.890 INFO:teuthology.orchestra.run.smithi001.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-11T21:44:46.890 INFO:teuthology.orchestra.run.smithi001.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-11T21:44:46.890 INFO:teuthology.orchestra.run.smithi001.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-11T21:44:46.890 INFO:teuthology.orchestra.run.smithi001.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-11T21:44:46.890 INFO:teuthology.orchestra.run.smithi001.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-11T21:44:46.890 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-11T21:44:46.891 INFO:teuthology.orchestra.run.smithi001.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-11T21:44:46.891 INFO:teuthology.orchestra.run.smithi001.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-11T21:44:46.891 INFO:teuthology.orchestra.run.smithi001.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-11T21:44:46.891 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-11T21:44:46.891 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-11T21:44:46.891 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-11T21:44:46.891 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-11T21:44:46.891 INFO:teuthology.orchestra.run.smithi001.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-11T21:44:46.891 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-11T21:44:46.891 INFO:teuthology.orchestra.run.smithi001.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-11T21:44:46.891 INFO:teuthology.orchestra.run.smithi001.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-11T21:44:46.891 INFO:teuthology.orchestra.run.smithi001.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-11T21:44:46.891 INFO:teuthology.orchestra.run.smithi001.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-11T21:44:46.892 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-11T21:44:46.892 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-11T21:44:46.892 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-11T21:44:46.892 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-11T21:44:46.892 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-11T21:44:46.892 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-11T21:44:46.892 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-11T21:44:46.892 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-11T21:44:46.892 INFO:teuthology.orchestra.run.smithi001.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-11T21:44:46.892 INFO:teuthology.orchestra.run.smithi001.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-11T21:44:46.892 INFO:teuthology.orchestra.run.smithi001.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-11T21:44:46.893 INFO:teuthology.orchestra.run.smithi001.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-11T21:44:46.893 INFO:teuthology.orchestra.run.smithi001.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-11T21:44:46.893 INFO:teuthology.orchestra.run.smithi001.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-11T21:44:46.893 INFO:teuthology.orchestra.run.smithi001.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-11T21:44:46.893 INFO:teuthology.orchestra.run.smithi001.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-11T21:44:46.893 INFO:teuthology.orchestra.run.smithi001.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-11T21:44:46.893 INFO:teuthology.orchestra.run.smithi001.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-11T21:44:46.893 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-11T21:44:46.893 INFO:teuthology.orchestra.run.smithi001.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-11T21:44:46.893 INFO:teuthology.orchestra.run.smithi001.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-11T21:44:46.893 INFO:teuthology.orchestra.run.smithi001.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-11T21:44:46.893 INFO:teuthology.orchestra.run.smithi001.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-11T21:44:46.893 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-11T21:44:46.894 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-11T21:44:46.894 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-11T21:44:46.894 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-11T21:44:46.894 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-11T21:44:46.894 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-11T21:44:46.894 INFO:teuthology.orchestra.run.smithi001.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-11T21:44:46.894 INFO:teuthology.orchestra.run.smithi001.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-11T21:44:46.894 INFO:teuthology.orchestra.run.smithi001.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-11T21:44:46.894 INFO:teuthology.orchestra.run.smithi001.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-11T21:44:46.894 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-11T21:44:46.894 INFO:teuthology.orchestra.run.smithi001.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-11T21:44:46.894 INFO:teuthology.orchestra.run.smithi001.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-11T21:44:46.894 INFO:teuthology.orchestra.run.smithi001.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-11T21:44:46.894 INFO:teuthology.orchestra.run.smithi001.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-11T21:44:46.895 INFO:teuthology.orchestra.run.smithi001.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-11T21:44:46.895 INFO:teuthology.orchestra.run.smithi001.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-11T21:44:46.895 INFO:teuthology.orchestra.run.smithi001.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-11T21:44:46.895 INFO:teuthology.orchestra.run.smithi001.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-11T21:44:46.895 INFO:teuthology.orchestra.run.smithi001.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-11T21:44:46.895 INFO:teuthology.orchestra.run.smithi001.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-11T21:44:46.895 INFO:teuthology.orchestra.run.smithi001.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-11T21:44:46.895 INFO:teuthology.orchestra.run.smithi001.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-11T21:44:46.895 INFO:teuthology.orchestra.run.smithi001.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-11T21:44:46.895 INFO:teuthology.orchestra.run.smithi001.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-11T21:44:46.895 INFO:teuthology.orchestra.run.smithi001.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-11T21:44:46.895 INFO:teuthology.orchestra.run.smithi001.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-11T21:44:46.895 INFO:teuthology.orchestra.run.smithi001.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-11T21:44:46.896 INFO:teuthology.orchestra.run.smithi001.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-11T21:44:46.896 INFO:teuthology.orchestra.run.smithi001.stdout:Installing weak dependencies: 2024-09-11T21:44:46.896 INFO:teuthology.orchestra.run.smithi001.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-11T21:44:46.896 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:44:46.896 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction Summary 2024-09-11T21:44:46.896 INFO:teuthology.orchestra.run.smithi001.stdout:====================================================================================== 2024-09-11T21:44:46.896 INFO:teuthology.orchestra.run.smithi001.stdout:Install 148 Packages 2024-09-11T21:44:46.896 INFO:teuthology.orchestra.run.smithi001.stdout:Upgrade 8 Packages 2024-09-11T21:44:46.896 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:44:46.897 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2024-09-11T21:44:46.897 INFO:teuthology.orchestra.run.smithi001.stdout:Total download size: 226 M 2024-09-11T21:44:46.897 INFO:teuthology.orchestra.run.smithi001.stdout:Downloading Packages: 2024-09-11T21:44:46.904 INFO:teuthology.orchestra.run.smithi161.stdout:====================================================================================== 2024-09-11T21:44:46.904 INFO:teuthology.orchestra.run.smithi161.stdout: Package Arch Version Repository Size 2024-09-11T21:44:46.905 INFO:teuthology.orchestra.run.smithi161.stdout:====================================================================================== 2024-09-11T21:44:46.905 INFO:teuthology.orchestra.run.smithi161.stdout:Installing: 2024-09-11T21:44:46.905 INFO:teuthology.orchestra.run.smithi161.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-11T21:44:46.905 INFO:teuthology.orchestra.run.smithi161.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-11T21:44:46.905 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-11T21:44:46.905 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-11T21:44:46.905 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-11T21:44:46.905 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-11T21:44:46.905 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-11T21:44:46.905 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-11T21:44:46.905 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-11T21:44:46.905 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-11T21:44:46.905 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-11T21:44:46.905 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-11T21:44:46.906 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-11T21:44:46.906 INFO:teuthology.orchestra.run.smithi161.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-11T21:44:46.906 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 228 k 2024-09-11T21:44:46.906 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-shell x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 27 k 2024-09-11T21:44:46.906 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-top noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 18 k 2024-09-11T21:44:46.906 INFO:teuthology.orchestra.run.smithi161.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-11T21:44:46.906 INFO:teuthology.orchestra.run.smithi161.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-11T21:44:46.906 INFO:teuthology.orchestra.run.smithi161.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-11T21:44:46.906 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-11T21:44:46.906 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-11T21:44:46.906 INFO:teuthology.orchestra.run.smithi161.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-11T21:44:46.906 INFO:teuthology.orchestra.run.smithi161.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-11T21:44:46.906 INFO:teuthology.orchestra.run.smithi161.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-11T21:44:46.906 INFO:teuthology.orchestra.run.smithi161.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-11T21:44:46.906 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-11T21:44:46.907 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-11T21:44:46.907 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-11T21:44:46.907 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-11T21:44:46.907 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-11T21:44:46.907 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-11T21:44:46.907 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-11T21:44:46.907 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-11T21:44:46.907 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-11T21:44:46.907 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-11T21:44:46.907 INFO:teuthology.orchestra.run.smithi161.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-11T21:44:46.907 INFO:teuthology.orchestra.run.smithi161.stdout:Upgrading: 2024-09-11T21:44:46.907 INFO:teuthology.orchestra.run.smithi161.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-11T21:44:46.907 INFO:teuthology.orchestra.run.smithi161.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-11T21:44:46.908 INFO:teuthology.orchestra.run.smithi161.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-11T21:44:46.908 INFO:teuthology.orchestra.run.smithi161.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-11T21:44:46.908 INFO:teuthology.orchestra.run.smithi161.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-11T21:44:46.908 INFO:teuthology.orchestra.run.smithi161.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-11T21:44:46.908 INFO:teuthology.orchestra.run.smithi161.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-11T21:44:46.908 INFO:teuthology.orchestra.run.smithi161.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-11T21:44:46.908 INFO:teuthology.orchestra.run.smithi161.stdout:Installing dependencies: 2024-09-11T21:44:46.908 INFO:teuthology.orchestra.run.smithi161.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-11T21:44:46.908 INFO:teuthology.orchestra.run.smithi161.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-11T21:44:46.908 INFO:teuthology.orchestra.run.smithi161.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-11T21:44:46.908 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-11T21:44:46.909 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-11T21:44:46.909 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-11T21:44:46.909 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-11T21:44:46.909 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-11T21:44:46.909 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-11T21:44:46.909 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-11T21:44:46.909 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-11T21:44:46.909 INFO:teuthology.orchestra.run.smithi161.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-11T21:44:46.909 INFO:teuthology.orchestra.run.smithi161.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-11T21:44:46.909 INFO:teuthology.orchestra.run.smithi161.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-11T21:44:46.909 INFO:teuthology.orchestra.run.smithi161.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-11T21:44:46.909 INFO:teuthology.orchestra.run.smithi161.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-11T21:44:46.909 INFO:teuthology.orchestra.run.smithi161.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-11T21:44:46.910 INFO:teuthology.orchestra.run.smithi161.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-11T21:44:46.910 INFO:teuthology.orchestra.run.smithi161.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-11T21:44:46.910 INFO:teuthology.orchestra.run.smithi161.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-11T21:44:46.910 INFO:teuthology.orchestra.run.smithi161.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-11T21:44:46.910 INFO:teuthology.orchestra.run.smithi161.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-11T21:44:46.910 INFO:teuthology.orchestra.run.smithi161.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-11T21:44:46.910 INFO:teuthology.orchestra.run.smithi161.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-11T21:44:46.910 INFO:teuthology.orchestra.run.smithi161.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-11T21:44:46.910 INFO:teuthology.orchestra.run.smithi161.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-11T21:44:46.910 INFO:teuthology.orchestra.run.smithi161.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-11T21:44:46.910 INFO:teuthology.orchestra.run.smithi161.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-11T21:44:46.910 INFO:teuthology.orchestra.run.smithi161.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-11T21:44:46.911 INFO:teuthology.orchestra.run.smithi161.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-11T21:44:46.911 INFO:teuthology.orchestra.run.smithi161.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-11T21:44:46.911 INFO:teuthology.orchestra.run.smithi161.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-11T21:44:46.911 INFO:teuthology.orchestra.run.smithi161.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-11T21:44:46.911 INFO:teuthology.orchestra.run.smithi161.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-11T21:44:46.911 INFO:teuthology.orchestra.run.smithi161.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-11T21:44:46.911 INFO:teuthology.orchestra.run.smithi161.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-11T21:44:46.911 INFO:teuthology.orchestra.run.smithi161.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-11T21:44:46.911 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-11T21:44:46.911 INFO:teuthology.orchestra.run.smithi161.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-11T21:44:46.911 INFO:teuthology.orchestra.run.smithi161.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-11T21:44:46.912 INFO:teuthology.orchestra.run.smithi161.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-11T21:44:46.912 INFO:teuthology.orchestra.run.smithi161.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-11T21:44:46.912 INFO:teuthology.orchestra.run.smithi161.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-11T21:44:46.912 INFO:teuthology.orchestra.run.smithi161.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-11T21:44:46.912 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-11T21:44:46.912 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-11T21:44:46.912 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-11T21:44:46.912 INFO:teuthology.orchestra.run.smithi161.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-11T21:44:46.912 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-11T21:44:46.912 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-11T21:44:46.912 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-11T21:44:46.912 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-11T21:44:46.912 INFO:teuthology.orchestra.run.smithi161.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-11T21:44:46.913 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-11T21:44:46.913 INFO:teuthology.orchestra.run.smithi161.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-11T21:44:46.913 INFO:teuthology.orchestra.run.smithi161.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-11T21:44:46.913 INFO:teuthology.orchestra.run.smithi161.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-11T21:44:46.913 INFO:teuthology.orchestra.run.smithi161.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-11T21:44:46.913 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-11T21:44:46.913 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-11T21:44:46.913 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-11T21:44:46.913 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-11T21:44:46.913 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-11T21:44:46.913 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-11T21:44:46.913 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-11T21:44:46.914 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-11T21:44:46.914 INFO:teuthology.orchestra.run.smithi161.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-11T21:44:46.914 INFO:teuthology.orchestra.run.smithi161.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-11T21:44:46.914 INFO:teuthology.orchestra.run.smithi161.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-11T21:44:46.914 INFO:teuthology.orchestra.run.smithi161.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-11T21:44:46.914 INFO:teuthology.orchestra.run.smithi161.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-11T21:44:46.914 INFO:teuthology.orchestra.run.smithi161.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-11T21:44:46.914 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-11T21:44:46.914 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-11T21:44:46.914 INFO:teuthology.orchestra.run.smithi161.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-11T21:44:46.914 INFO:teuthology.orchestra.run.smithi161.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-11T21:44:46.914 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-11T21:44:46.914 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-11T21:44:46.914 INFO:teuthology.orchestra.run.smithi161.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-11T21:44:46.914 INFO:teuthology.orchestra.run.smithi161.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-11T21:44:46.915 INFO:teuthology.orchestra.run.smithi161.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-11T21:44:46.915 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-11T21:44:46.915 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-11T21:44:46.915 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-11T21:44:46.915 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-11T21:44:46.915 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-11T21:44:46.915 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-11T21:44:46.915 INFO:teuthology.orchestra.run.smithi161.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-11T21:44:46.915 INFO:teuthology.orchestra.run.smithi161.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-11T21:44:46.915 INFO:teuthology.orchestra.run.smithi161.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-11T21:44:46.915 INFO:teuthology.orchestra.run.smithi161.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-11T21:44:46.915 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-11T21:44:46.915 INFO:teuthology.orchestra.run.smithi161.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-11T21:44:46.915 INFO:teuthology.orchestra.run.smithi161.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-11T21:44:46.915 INFO:teuthology.orchestra.run.smithi161.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-11T21:44:46.916 INFO:teuthology.orchestra.run.smithi161.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-11T21:44:46.916 INFO:teuthology.orchestra.run.smithi161.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-11T21:44:46.916 INFO:teuthology.orchestra.run.smithi161.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-11T21:44:46.916 INFO:teuthology.orchestra.run.smithi161.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-11T21:44:46.916 INFO:teuthology.orchestra.run.smithi161.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-11T21:44:46.916 INFO:teuthology.orchestra.run.smithi161.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-11T21:44:46.916 INFO:teuthology.orchestra.run.smithi161.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-11T21:44:46.916 INFO:teuthology.orchestra.run.smithi161.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-11T21:44:46.916 INFO:teuthology.orchestra.run.smithi161.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-11T21:44:46.916 INFO:teuthology.orchestra.run.smithi161.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-11T21:44:46.916 INFO:teuthology.orchestra.run.smithi161.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-11T21:44:46.916 INFO:teuthology.orchestra.run.smithi161.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-11T21:44:46.916 INFO:teuthology.orchestra.run.smithi161.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-11T21:44:46.916 INFO:teuthology.orchestra.run.smithi161.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-11T21:44:46.916 INFO:teuthology.orchestra.run.smithi161.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-11T21:44:46.917 INFO:teuthology.orchestra.run.smithi161.stdout:Installing weak dependencies: 2024-09-11T21:44:46.917 INFO:teuthology.orchestra.run.smithi161.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-11T21:44:46.917 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-11T21:44:46.917 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2024-09-11T21:44:46.917 INFO:teuthology.orchestra.run.smithi161.stdout:====================================================================================== 2024-09-11T21:44:46.917 INFO:teuthology.orchestra.run.smithi161.stdout:Install 148 Packages 2024-09-11T21:44:46.917 INFO:teuthology.orchestra.run.smithi161.stdout:Upgrade 8 Packages 2024-09-11T21:44:46.917 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-11T21:44:46.917 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 226 M 2024-09-11T21:44:46.918 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading Packages: 2024-09-11T21:44:46.944 INFO:teuthology.orchestra.run.smithi053.stdout:(6/156): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_ 10 MB/s | 2.4 MB 00:00 2024-09-11T21:44:47.186 INFO:teuthology.orchestra.run.smithi053.stdout:(7/156): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_ 15 MB/s | 4.7 MB 00:00 2024-09-11T21:44:47.278 INFO:teuthology.orchestra.run.smithi001.stdout:(1/156): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.r 40 kB/s | 6.5 kB 00:00 2024-09-11T21:44:47.403 INFO:teuthology.orchestra.run.smithi053.stdout:(8/156): ceph-common-19.1.1-298.gb53b6d3e.el9.x 19 MB/s | 21 MB 00:01 2024-09-11T21:44:47.470 INFO:teuthology.orchestra.run.smithi053.stdout:(9/156): ceph-selinux-19.1.1-298.gb53b6d3e.el9. 379 kB/s | 25 kB 00:00 2024-09-11T21:44:47.787 INFO:teuthology.orchestra.run.smithi161.stdout:(1/156): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.r 24 kB/s | 6.5 kB 00:00 2024-09-11T21:44:47.820 INFO:teuthology.orchestra.run.smithi001.stdout:(2/156): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.6 MB/s | 897 kB 00:00 2024-09-11T21:44:48.012 INFO:teuthology.orchestra.run.smithi001.stdout:(3/156): ceph-immutable-object-cache-19.1.1-298 757 kB/s | 145 kB 00:00 2024-09-11T21:44:48.079 INFO:teuthology.orchestra.run.smithi001.stdout:(4/156): ceph-base-19.1.1-298.gb53b6d3e.el9.x86 5.6 MB/s | 5.4 MB 00:00 2024-09-11T21:44:48.196 INFO:teuthology.orchestra.run.smithi161.stdout:(2/156): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86 2.1 MB/s | 897 kB 00:00 2024-09-11T21:44:48.204 INFO:teuthology.orchestra.run.smithi053.stdout:(10/156): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86 13 MB/s | 17 MB 00:01 2024-09-11T21:44:48.220 INFO:teuthology.orchestra.run.smithi001.stdout:(5/156): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_ 11 MB/s | 1.5 MB 00:00 2024-09-11T21:44:48.246 INFO:teuthology.orchestra.run.smithi161.stdout:(3/156): ceph-immutable-object-cache-19.1.1-298 2.9 MB/s | 145 kB 00:00 2024-09-11T21:44:48.321 INFO:teuthology.orchestra.run.smithi053.stdout:(11/156): ceph-radosgw-19.1.1-298.gb53b6d3e.el9 9.3 MB/s | 11 MB 00:01 2024-09-11T21:44:48.346 INFO:teuthology.orchestra.run.smithi053.stdout:(12/156): cephfs-mirror-19.1.1-298.gb53b6d3e.el 1.6 MB/s | 228 kB 00:00 2024-09-11T21:44:48.379 INFO:teuthology.orchestra.run.smithi053.stdout:(13/156): cephfs-shell-19.1.1-298.gb53b6d3e.el9 464 kB/s | 27 kB 00:00 2024-09-11T21:44:48.413 INFO:teuthology.orchestra.run.smithi053.stdout:(14/156): libcephfs-devel-19.1.1-298.gb53b6d3e. 474 kB/s | 32 kB 00:00 2024-09-11T21:44:48.496 INFO:teuthology.orchestra.run.smithi053.stdout:(15/156): libcephfs2-19.1.1-298.gb53b6d3e.el9.x 6.3 MB/s | 752 kB 00:00 2024-09-11T21:44:48.571 INFO:teuthology.orchestra.run.smithi001.stdout:(6/156): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_ 13 MB/s | 4.7 MB 00:00 2024-09-11T21:44:48.580 INFO:teuthology.orchestra.run.smithi053.stdout:(16/156): librados-devel-19.1.1-298.gb53b6d3e.e 1.5 MB/s | 127 kB 00:00 2024-09-11T21:44:48.646 INFO:teuthology.orchestra.run.smithi161.stdout:(4/156): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_ 6.0 MB/s | 2.4 MB 00:00 2024-09-11T21:44:48.680 INFO:teuthology.orchestra.run.smithi053.stdout:(17/156): libradosstriper1-19.1.1-298.gb53b6d3e 5.0 MB/s | 506 kB 00:00 2024-09-11T21:44:48.846 INFO:teuthology.orchestra.run.smithi001.stdout:(7/156): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_ 2.9 MB/s | 2.4 MB 00:00 2024-09-11T21:44:49.256 INFO:teuthology.orchestra.run.smithi053.stdout:(18/156): librgw2-19.1.1-298.gb53b6d3e.el9.x86_ 9.3 MB/s | 5.3 MB 00:00 2024-09-11T21:44:49.289 INFO:teuthology.orchestra.run.smithi053.stdout:(19/156): libcephsqlite-19.1.1-298.gb53b6d3e.el 187 kB/s | 163 kB 00:00 2024-09-11T21:44:49.339 INFO:teuthology.orchestra.run.smithi053.stdout:(20/156): python3-ceph-argparse-19.1.1-298.gb53 541 kB/s | 45 kB 00:00 2024-09-11T21:44:49.355 INFO:teuthology.orchestra.run.smithi161.stdout:(5/156): ceph-base-19.1.1-298.gb53b6d3e.el9.x86 2.9 MB/s | 5.4 MB 00:01 2024-09-11T21:44:49.364 INFO:teuthology.orchestra.run.smithi053.stdout:(21/156): python3-ceph-common-19.1.1-298.gb53b6 1.8 MB/s | 136 kB 00:00 2024-09-11T21:44:49.423 INFO:teuthology.orchestra.run.smithi053.stdout:(22/156): python3-cephfs-19.1.1-298.gb53b6d3e.e 1.9 MB/s | 162 kB 00:00 2024-09-11T21:44:49.438 INFO:teuthology.orchestra.run.smithi161.stdout:(6/156): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_ 1.9 MB/s | 1.5 MB 00:00 2024-09-11T21:44:49.465 INFO:teuthology.orchestra.run.smithi053.stdout:(23/156): python3-rados-19.1.1-298.gb53b6d3e.el 3.1 MB/s | 321 kB 00:00 2024-09-11T21:44:49.516 INFO:teuthology.orchestra.run.smithi053.stdout:(24/156): python3-rbd-19.1.1-298.gb53b6d3e.el9. 3.2 MB/s | 301 kB 00:00 2024-09-11T21:44:49.557 INFO:teuthology.orchestra.run.smithi053.stdout:(25/156): python3-rgw-19.1.1-298.gb53b6d3e.el9. 1.1 MB/s | 100 kB 00:00 2024-09-11T21:44:49.583 INFO:teuthology.orchestra.run.smithi053.stdout:(26/156): rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.2 MB/s | 85 kB 00:00 2024-09-11T21:44:49.683 INFO:teuthology.orchestra.run.smithi053.stdout:(27/156): rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_ 1.7 MB/s | 171 kB 00:00 2024-09-11T21:44:49.688 INFO:teuthology.orchestra.run.smithi001.stdout:(8/156): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86_ 15 MB/s | 17 MB 00:01 2024-09-11T21:44:49.783 INFO:teuthology.orchestra.run.smithi053.stdout:(28/156): ceph-grafana-dashboards-19.1.1-298.gb 318 kB/s | 32 kB 00:00 2024-09-11T21:44:49.788 INFO:teuthology.orchestra.run.smithi001.stdout:(9/156): ceph-selinux-19.1.1-298.gb53b6d3e.el9. 252 kB/s | 25 kB 00:00 2024-09-11T21:44:49.884 INFO:teuthology.orchestra.run.smithi053.stdout:(29/156): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e 1.4 MB/s | 146 kB 00:00 2024-09-11T21:44:50.064 INFO:teuthology.orchestra.run.smithi161.stdout:(7/156): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_ 6.6 MB/s | 4.7 MB 00:00 2024-09-11T21:44:50.159 INFO:teuthology.orchestra.run.smithi053.stdout:(30/156): rbd-mirror-19.1.1-298.gb53b6d3e.el9.x 5.2 MB/s | 3.1 MB 00:00 2024-09-11T21:44:50.568 INFO:teuthology.orchestra.run.smithi053.stdout:(31/156): ceph-mgr-dashboard-19.1.1-298.gb53b6d 5.2 MB/s | 3.6 MB 00:00 2024-09-11T21:44:50.659 INFO:teuthology.orchestra.run.smithi053.stdout:(32/156): ceph-mgr-modules-core-19.1.1-298.gb53 2.7 MB/s | 249 kB 00:00 2024-09-11T21:44:50.743 INFO:teuthology.orchestra.run.smithi053.stdout:(33/156): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el 591 kB/s | 49 kB 00:00 2024-09-11T21:44:50.835 INFO:teuthology.orchestra.run.smithi053.stdout:(34/156): ceph-prometheus-alerts-19.1.1-298.gb5 179 kB/s | 16 kB 00:00 2024-09-11T21:44:51.294 INFO:teuthology.orchestra.run.smithi053.stdout:(35/156): ceph-test-19.1.1-298.gb53b6d3e.el9.x8 13 MB/s | 49 MB 00:03 2024-09-11T21:44:51.319 INFO:teuthology.orchestra.run.smithi053.stdout:(36/156): ceph-volume-19.1.1-298.gb53b6d3e.el9. 590 kB/s | 285 kB 00:00 2024-09-11T21:44:51.322 INFO:teuthology.orchestra.run.smithi001.stdout:(10/156): ceph-radosgw-19.1.1-298.gb53b6d3e.el9 4.3 MB/s | 11 MB 00:02 2024-09-11T21:44:51.377 INFO:teuthology.orchestra.run.smithi053.stdout:(37/156): cephfs-top-19.1.1-298.gb53b6d3e.el9.n 313 kB/s | 18 kB 00:00 2024-09-11T21:44:51.397 INFO:teuthology.orchestra.run.smithi001.stdout:(11/156): cephfs-mirror-19.1.1-298.gb53b6d3e.el 3.0 MB/s | 228 kB 00:00 2024-09-11T21:44:51.445 INFO:teuthology.orchestra.run.smithi053.stdout:(38/156): cephadm-19.1.1-298.gb53b6d3e.el9.noar 5.0 MB/s | 765 kB 00:00 2024-09-11T21:44:51.464 INFO:teuthology.orchestra.run.smithi001.stdout:(12/156): cephfs-shell-19.1.1-298.gb53b6d3e.el9 406 kB/s | 27 kB 00:00 2024-09-11T21:44:51.502 INFO:teuthology.orchestra.run.smithi053.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 642 kB/s | 80 kB 00:00 2024-09-11T21:44:51.531 INFO:teuthology.orchestra.run.smithi001.stdout:(13/156): libcephfs-devel-19.1.1-298.gb53b6d3e. 477 kB/s | 32 kB 00:00 2024-09-11T21:44:51.555 INFO:teuthology.orchestra.run.smithi161.stdout:(8/156): ceph-common-19.1.1-298.gb53b6d3e.el9.x 5.2 MB/s | 21 MB 00:04 2024-09-11T21:44:51.594 INFO:teuthology.orchestra.run.smithi053.stdout:(40/156): libgfortran-11.5.0-2.el9.x86_64.rpm 5.3 MB/s | 801 kB 00:00 2024-09-11T21:44:51.628 INFO:teuthology.orchestra.run.smithi053.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 1.5 MB/s | 191 kB 00:00 2024-09-11T21:44:51.681 INFO:teuthology.orchestra.run.smithi161.stdout:(9/156): ceph-radosgw-19.1.1-298.gb53b6d3e.el9. 6.6 MB/s | 11 MB 00:01 2024-09-11T21:44:51.697 INFO:teuthology.orchestra.run.smithi161.stdout:(10/156): ceph-selinux-19.1.1-298.gb53b6d3e.el9 177 kB/s | 25 kB 00:00 2024-09-11T21:44:51.719 INFO:teuthology.orchestra.run.smithi053.stdout:(42/156): ceph-mgr-diskprediction-local-19.1.1- 4.7 MB/s | 7.4 MB 00:01 2024-09-11T21:44:51.731 INFO:teuthology.orchestra.run.smithi001.stdout:(14/156): ceph-common-19.1.1-298.gb53b6d3e.el9. 4.6 MB/s | 21 MB 00:04 2024-09-11T21:44:51.745 INFO:teuthology.orchestra.run.smithi053.stdout:(43/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.6 MB/s | 253 kB 00:00 2024-09-11T21:44:51.747 INFO:teuthology.orchestra.run.smithi161.stdout:(11/156): cephfs-mirror-19.1.1-298.gb53b6d3e.el 4.5 MB/s | 228 kB 00:00 2024-09-11T21:44:51.765 INFO:teuthology.orchestra.run.smithi001.stdout:(15/156): libcephfs2-19.1.1-298.gb53b6d3e.el9.x 3.1 MB/s | 752 kB 00:00 2024-09-11T21:44:51.778 INFO:teuthology.orchestra.run.smithi053.stdout:(44/156): python3-cryptography-36.0.1-4.el9.x86 8.1 MB/s | 1.2 MB 00:00 2024-09-11T21:44:51.790 INFO:teuthology.orchestra.run.smithi001.stdout:(16/156): libcephsqlite-19.1.1-298.gb53b6d3e.el 2.8 MB/s | 163 kB 00:00 2024-09-11T21:44:51.798 INFO:teuthology.orchestra.run.smithi161.stdout:(12/156): cephfs-shell-19.1.1-298.gb53b6d3e.el9 541 kB/s | 27 kB 00:00 2024-09-11T21:44:51.820 INFO:teuthology.orchestra.run.smithi053.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 1.8 MB/s | 135 kB 00:00 2024-09-11T21:44:51.823 INFO:teuthology.orchestra.run.smithi001.stdout:(17/156): librados-devel-19.1.1-298.gb53b6d3e.e 2.1 MB/s | 127 kB 00:00 2024-09-11T21:44:51.848 INFO:teuthology.orchestra.run.smithi161.stdout:(13/156): libcephfs-devel-19.1.1-298.gb53b6d3e. 633 kB/s | 32 kB 00:00 2024-09-11T21:44:51.862 INFO:teuthology.orchestra.run.smithi053.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-09-11T21:44:51.888 INFO:teuthology.orchestra.run.smithi053.stdout:(47/156): python3-ply-3.11-14.el9.noarch.rpm 636 kB/s | 106 kB 00:00 2024-09-11T21:44:51.891 INFO:teuthology.orchestra.run.smithi001.stdout:(18/156): libradosstriper1-19.1.1-298.gb53b6d3e 4.9 MB/s | 506 kB 00:00 2024-09-11T21:44:51.913 INFO:teuthology.orchestra.run.smithi053.stdout:(48/156): python3-urllib3-1.26.5-6.el9.noarch.r 2.3 MB/s | 215 kB 00:00 2024-09-11T21:44:51.957 INFO:teuthology.orchestra.run.smithi001.stdout:(19/156): python3-ceph-argparse-19.1.1-298.gb53 678 kB/s | 45 kB 00:00 2024-09-11T21:44:51.973 INFO:teuthology.orchestra.run.smithi161.stdout:(14/156): libcephfs2-19.1.1-298.gb53b6d3e.el9.x 5.9 MB/s | 752 kB 00:00 2024-09-11T21:44:52.024 INFO:teuthology.orchestra.run.smithi001.stdout:(20/156): python3-ceph-common-19.1.1-298.gb53b6 2.0 MB/s | 136 kB 00:00 2024-09-11T21:44:52.030 INFO:teuthology.orchestra.run.smithi053.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 918 kB/s | 107 kB 00:00 2024-09-11T21:44:52.049 INFO:teuthology.orchestra.run.smithi161.stdout:(15/156): libcephsqlite-19.1.1-298.gb53b6d3e.el 2.1 MB/s | 163 kB 00:00 2024-09-11T21:44:52.063 INFO:teuthology.orchestra.run.smithi053.stdout:(50/156): automake-1.16.2-8.el9.noarch.rpm 3.4 MB/s | 689 kB 00:00 2024-09-11T21:44:52.080 INFO:teuthology.orchestra.run.smithi053.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 433 kB/s | 22 kB 00:00 2024-09-11T21:44:52.091 INFO:teuthology.orchestra.run.smithi001.stdout:(21/156): python3-cephfs-19.1.1-298.gb53b6d3e.e 2.4 MB/s | 162 kB 00:00 2024-09-11T21:44:52.114 INFO:teuthology.orchestra.run.smithi053.stdout:(52/156): bison-3.7.4-5.el9.x86_64.rpm 4.1 MB/s | 943 kB 00:00 2024-09-11T21:44:52.124 INFO:teuthology.orchestra.run.smithi161.stdout:(16/156): librados-devel-19.1.1-298.gb53b6d3e.e 1.6 MB/s | 127 kB 00:00 2024-09-11T21:44:52.131 INFO:teuthology.orchestra.run.smithi053.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 653 kB/s | 33 kB 00:00 2024-09-11T21:44:52.156 INFO:teuthology.orchestra.run.smithi053.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 3.3 MB/s | 313 kB 00:00 2024-09-11T21:44:52.181 INFO:teuthology.orchestra.run.smithi053.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 363 kB/s | 18 kB 00:00 2024-09-11T21:44:52.207 INFO:teuthology.orchestra.run.smithi053.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-09-11T21:44:52.208 INFO:teuthology.orchestra.run.smithi161.stdout:(17/156): libradosstriper1-19.1.1-298.gb53b6d3e 5.9 MB/s | 506 kB 00:00 2024-09-11T21:44:52.217 INFO:teuthology.orchestra.run.smithi001.stdout:(22/156): python3-rados-19.1.1-298.gb53b6d3e.el 2.5 MB/s | 321 kB 00:00 2024-09-11T21:44:52.232 INFO:teuthology.orchestra.run.smithi053.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-09-11T21:44:52.257 INFO:teuthology.orchestra.run.smithi053.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 903 kB/s | 45 kB 00:00 2024-09-11T21:44:52.299 INFO:teuthology.orchestra.run.smithi053.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 662 kB 00:00 2024-09-11T21:44:52.308 INFO:teuthology.orchestra.run.smithi001.stdout:(23/156): python3-rbd-19.1.1-298.gb53b6d3e.el9. 3.2 MB/s | 301 kB 00:00 2024-09-11T21:44:52.374 INFO:teuthology.orchestra.run.smithi053.stdout:(60/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 12 MB/s | 3.0 MB 00:00 2024-09-11T21:44:52.392 INFO:teuthology.orchestra.run.smithi001.stdout:(24/156): python3-rgw-19.1.1-298.gb53b6d3e.el9. 1.2 MB/s | 100 kB 00:00 2024-09-11T21:44:52.401 INFO:teuthology.orchestra.run.smithi053.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 244 kB/s | 24 kB 00:00 2024-09-11T21:44:52.425 INFO:teuthology.orchestra.run.smithi053.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-09-11T21:44:52.442 INFO:teuthology.orchestra.run.smithi001.stdout:(25/156): rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-11T21:44:52.450 INFO:teuthology.orchestra.run.smithi053.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-09-11T21:44:52.475 INFO:teuthology.orchestra.run.smithi053.stdout:(64/156): libtool-2.4.6-46.el9.x86_64.rpm 2.6 MB/s | 581 kB 00:00 2024-09-11T21:44:52.500 INFO:teuthology.orchestra.run.smithi053.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 769 kB/s | 38 kB 00:00 2024-09-11T21:44:52.516 INFO:teuthology.orchestra.run.smithi161.stdout:(18/156): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86 5.5 MB/s | 17 MB 00:03 2024-09-11T21:44:52.526 INFO:teuthology.orchestra.run.smithi053.stdout:(66/156): lua-5.4.4-4.el9.x86_64.rpm 1.8 MB/s | 188 kB 00:00 2024-09-11T21:44:52.592 INFO:teuthology.orchestra.run.smithi161.stdout:(19/156): python3-ceph-argparse-19.1.1-298.gb53 603 kB/s | 45 kB 00:00 2024-09-11T21:44:52.593 INFO:teuthology.orchestra.run.smithi053.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 24 MB/s | 1.6 MB 00:00 2024-09-11T21:44:52.660 INFO:teuthology.orchestra.run.smithi053.stdout:(68/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 28 MB/s | 4.4 MB 00:00 2024-09-11T21:44:52.717 INFO:teuthology.orchestra.run.smithi161.stdout:(20/156): python3-ceph-common-19.1.1-298.gb53b6 1.1 MB/s | 136 kB 00:00 2024-09-11T21:44:52.727 INFO:teuthology.orchestra.run.smithi053.stdout:(69/156): openblas-openmp-0.3.26-2.el9.x86_64.r 20 MB/s | 4.9 MB 00:00 2024-09-11T21:44:52.752 INFO:teuthology.orchestra.run.smithi053.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 1.5 MB/s | 137 kB 00:00 2024-09-11T21:44:52.794 INFO:teuthology.orchestra.run.smithi053.stdout:(71/156): postgresql-contrib-13.16-1.el9.x86_64 4.4 MB/s | 910 kB 00:00 2024-09-11T21:44:52.827 INFO:teuthology.orchestra.run.smithi053.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 13 MB/s | 1.0 MB 00:00 2024-09-11T21:44:52.834 INFO:teuthology.orchestra.run.smithi001.stdout:(26/156): librgw2-19.1.1-298.gb53b6d3e.el9.x86_ 5.3 MB/s | 5.3 MB 00:01 2024-09-11T21:44:52.850 INFO:teuthology.orchestra.run.smithi161.stdout:(21/156): librgw2-19.1.1-298.gb53b6d3e.el9.x86_ 8.3 MB/s | 5.3 MB 00:00 2024-09-11T21:44:52.853 INFO:teuthology.orchestra.run.smithi053.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-09-11T21:44:52.876 INFO:teuthology.orchestra.run.smithi161.stdout:(22/156): python3-cephfs-19.1.1-298.gb53b6d3e.e 1.0 MB/s | 162 kB 00:00 2024-09-11T21:44:52.895 INFO:teuthology.orchestra.run.smithi053.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-09-11T21:44:52.901 INFO:teuthology.orchestra.run.smithi001.stdout:(27/156): rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_ 2.5 MB/s | 171 kB 00:00 2024-09-11T21:44:52.934 INFO:teuthology.orchestra.run.smithi161.stdout:(23/156): python3-rados-19.1.1-298.gb53b6d3e.el 3.8 MB/s | 321 kB 00:00 2024-09-11T21:44:52.960 INFO:teuthology.orchestra.run.smithi161.stdout:(24/156): python3-rbd-19.1.1-298.gb53b6d3e.el9. 3.5 MB/s | 301 kB 00:00 2024-09-11T21:44:52.978 INFO:teuthology.orchestra.run.smithi053.stdout:(75/156): postgresql-server-13.16-1.el9.x86_64. 23 MB/s | 5.8 MB 00:00 2024-09-11T21:44:53.001 INFO:teuthology.orchestra.run.smithi001.stdout:(28/156): ceph-grafana-dashboards-19.1.1-298.gb 318 kB/s | 32 kB 00:00 2024-09-11T21:44:53.003 INFO:teuthology.orchestra.run.smithi053.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.2 MB/s | 245 kB 00:00 2024-09-11T21:44:53.010 INFO:teuthology.orchestra.run.smithi161.stdout:(25/156): python3-rgw-19.1.1-298.gb53b6d3e.el9. 1.3 MB/s | 100 kB 00:00 2024-09-11T21:44:53.028 INFO:teuthology.orchestra.run.smithi053.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-09-11T21:44:53.035 INFO:teuthology.orchestra.run.smithi161.stdout:(26/156): rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.1 MB/s | 85 kB 00:00 2024-09-11T21:44:53.054 INFO:teuthology.orchestra.run.smithi053.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 512 kB/s | 26 kB 00:00 2024-09-11T21:44:53.060 INFO:teuthology.orchestra.run.smithi001.stdout:(29/156): rbd-mirror-19.1.1-298.gb53b6d3e.el9.x 5.0 MB/s | 3.1 MB 00:00 2024-09-11T21:44:53.085 INFO:teuthology.orchestra.run.smithi001.stdout:(30/156): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e 1.7 MB/s | 146 kB 00:00 2024-09-11T21:44:53.087 INFO:teuthology.orchestra.run.smithi053.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 325 kB/s | 19 kB 00:00 2024-09-11T21:44:53.113 INFO:teuthology.orchestra.run.smithi053.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-09-11T21:44:53.119 INFO:teuthology.orchestra.run.smithi161.stdout:(27/156): rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_ 2.0 MB/s | 171 kB 00:00 2024-09-11T21:44:53.138 INFO:teuthology.orchestra.run.smithi053.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 693 kB/s | 35 kB 00:00 2024-09-11T21:44:53.211 INFO:teuthology.orchestra.run.smithi161.stdout:(28/156): ceph-grafana-dashboards-19.1.1-298.gb 348 kB/s | 32 kB 00:00 2024-09-11T21:44:53.213 INFO:teuthology.orchestra.run.smithi053.stdout:(82/156): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-09-11T21:44:53.239 INFO:teuthology.orchestra.run.smithi053.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.4 MB/s | 250 kB 00:00 2024-09-11T21:44:53.264 INFO:teuthology.orchestra.run.smithi053.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-09-11T21:44:53.289 INFO:teuthology.orchestra.run.smithi053.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-11T21:44:53.302 INFO:teuthology.orchestra.run.smithi161.stdout:(29/156): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e 1.6 MB/s | 146 kB 00:00 2024-09-11T21:44:53.314 INFO:teuthology.orchestra.run.smithi053.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 843 kB/s | 42 kB 00:00 2024-09-11T21:44:53.340 INFO:teuthology.orchestra.run.smithi053.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 5.3 MB/s | 269 kB 00:00 2024-09-11T21:44:53.365 INFO:teuthology.orchestra.run.smithi053.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-11T21:44:53.390 INFO:teuthology.orchestra.run.smithi053.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-09-11T21:44:53.403 INFO:teuthology.orchestra.run.smithi161.stdout:(30/156): rbd-mirror-19.1.1-298.gb53b6d3e.el9.x 7.9 MB/s | 3.1 MB 00:00 2024-09-11T21:44:53.407 INFO:teuthology.orchestra.run.smithi053.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-09-11T21:44:53.424 INFO:teuthology.orchestra.run.smithi053.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.6 MB/s | 54 kB 00:00 2024-09-11T21:44:53.458 INFO:teuthology.orchestra.run.smithi053.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 1.2 MB/s | 42 kB 00:00 2024-09-11T21:44:53.525 INFO:teuthology.orchestra.run.smithi053.stdout:(93/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-09-11T21:44:53.541 INFO:teuthology.orchestra.run.smithi053.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 530 kB/s | 44 kB 00:00 2024-09-11T21:44:53.575 INFO:teuthology.orchestra.run.smithi053.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-11T21:44:53.600 INFO:teuthology.orchestra.run.smithi053.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 613 kB/s | 36 kB 00:00 2024-09-11T21:44:53.617 INFO:teuthology.orchestra.run.smithi053.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-09-11T21:44:53.643 INFO:teuthology.orchestra.run.smithi053.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 7.1 MB/s | 304 kB 00:00 2024-09-11T21:44:53.659 INFO:teuthology.orchestra.run.smithi053.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 1.4 MB/s | 58 kB 00:00 2024-09-11T21:44:53.693 INFO:teuthology.orchestra.run.smithi053.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-11T21:44:53.718 INFO:teuthology.orchestra.run.smithi053.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-09-11T21:44:53.743 INFO:teuthology.orchestra.run.smithi053.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 893 kB/s | 45 kB 00:00 2024-09-11T21:44:53.836 INFO:teuthology.orchestra.run.smithi001.stdout:(31/156): ceph-mgr-dashboard-19.1.1-298.gb53b6d 4.6 MB/s | 3.6 MB 00:00 2024-09-11T21:44:53.911 INFO:teuthology.orchestra.run.smithi001.stdout:(32/156): ceph-mgr-modules-core-19.1.1-298.gb53 3.3 MB/s | 249 kB 00:00 2024-09-11T21:44:53.961 INFO:teuthology.orchestra.run.smithi053.stdout:(103/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 29 MB/s | 16 MB 00:00 2024-09-11T21:44:53.969 INFO:teuthology.orchestra.run.smithi001.stdout:(33/156): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el 845 kB/s | 49 kB 00:00 2024-09-11T21:44:54.003 INFO:teuthology.orchestra.run.smithi161.stdout:(31/156): ceph-mgr-dashboard-19.1.1-298.gb53b6d 5.1 MB/s | 3.6 MB 00:00 2024-09-11T21:44:54.011 INFO:teuthology.orchestra.run.smithi053.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-11T21:44:54.028 INFO:teuthology.orchestra.run.smithi001.stdout:(34/156): ceph-prometheus-alerts-19.1.1-298.gb5 283 kB/s | 16 kB 00:00 2024-09-11T21:44:54.044 INFO:teuthology.orchestra.run.smithi053.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 9.0 MB/s | 309 kB 00:00 2024-09-11T21:44:54.070 INFO:teuthology.orchestra.run.smithi053.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 779 kB/s | 19 kB 00:00 2024-09-11T21:44:54.103 INFO:teuthology.orchestra.run.smithi161.stdout:(32/156): ceph-mgr-modules-core-19.1.1-298.gb53 2.4 MB/s | 249 kB 00:00 2024-09-11T21:44:54.128 INFO:teuthology.orchestra.run.smithi001.stdout:(35/156): ceph-volume-19.1.1-298.gb53b6d3e.el9. 2.8 MB/s | 285 kB 00:00 2024-09-11T21:44:54.187 INFO:teuthology.orchestra.run.smithi161.stdout:(33/156): ceph-mgr-diskprediction-local-19.1.1- 9.4 MB/s | 7.4 MB 00:00 2024-09-11T21:44:54.212 INFO:teuthology.orchestra.run.smithi161.stdout:(34/156): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el 454 kB/s | 49 kB 00:00 2024-09-11T21:44:54.271 INFO:teuthology.orchestra.run.smithi161.stdout:(35/156): ceph-prometheus-alerts-19.1.1-298.gb5 197 kB/s | 16 kB 00:00 2024-09-11T21:44:54.286 INFO:teuthology.orchestra.run.smithi053.stdout:(107/156): libarrow-9.0.0-11.el9.x86_64.rpm 20 MB/s | 4.4 MB 00:00 2024-09-11T21:44:54.296 INFO:teuthology.orchestra.run.smithi161.stdout:(36/156): ceph-volume-19.1.1-298.gb53b6d3e.el9. 3.3 MB/s | 285 kB 00:00 2024-09-11T21:44:54.312 INFO:teuthology.orchestra.run.smithi053.stdout:(108/156): lua-devel-5.4.4-4.el9.x86_64.rpm 38 kB/s | 22 kB 00:00 2024-09-11T21:44:54.329 INFO:teuthology.orchestra.run.smithi001.stdout:(36/156): cephadm-19.1.1-298.gb53b6d3e.el9.noar 3.7 MB/s | 765 kB 00:00 2024-09-11T21:44:54.345 INFO:teuthology.orchestra.run.smithi053.stdout:(109/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 445 kB/s | 26 kB 00:00 2024-09-11T21:44:54.370 INFO:teuthology.orchestra.run.smithi053.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.9 MB/s | 67 kB 00:00 2024-09-11T21:44:54.404 INFO:teuthology.orchestra.run.smithi053.stdout:(111/156): liboath-2.6.7-2.el9.x86_64.rpm 527 kB/s | 49 kB 00:00 2024-09-11T21:44:54.421 INFO:teuthology.orchestra.run.smithi001.stdout:(37/156): cephfs-top-19.1.1-298.gb53b6d3e.el9.n 199 kB/s | 18 kB 00:00 2024-09-11T21:44:54.430 INFO:teuthology.orchestra.run.smithi053.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-09-11T21:44:54.438 INFO:teuthology.orchestra.run.smithi161.stdout:(37/156): cephfs-top-19.1.1-298.gb53b6d3e.el9.n 128 kB/s | 18 kB 00:00 2024-09-11T21:44:54.463 INFO:teuthology.orchestra.run.smithi161.stdout:(38/156): cephadm-19.1.1-298.gb53b6d3e.el9.noar 3.9 MB/s | 765 kB 00:00 2024-09-11T21:44:54.471 INFO:teuthology.orchestra.run.smithi053.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 13 MB/s | 548 kB 00:00 2024-09-11T21:44:54.497 INFO:teuthology.orchestra.run.smithi053.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-11T21:44:54.530 INFO:teuthology.orchestra.run.smithi053.stdout:(115/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 6.5 MB/s | 839 kB 00:00 2024-09-11T21:44:54.556 INFO:teuthology.orchestra.run.smithi053.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 550 kB/s | 32 kB 00:00 2024-09-11T21:44:54.581 INFO:teuthology.orchestra.run.smithi053.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 281 kB/s | 14 kB 00:00 2024-09-11T21:44:54.606 INFO:teuthology.orchestra.run.smithi053.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-11T21:44:54.630 INFO:teuthology.orchestra.run.smithi161.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 416 kB/s | 80 kB 00:00 2024-09-11T21:44:54.638 INFO:teuthology.orchestra.run.smithi001.stdout:(38/156): ceph-mgr-diskprediction-local-19.1.1- 4.8 MB/s | 7.4 MB 00:01 2024-09-11T21:44:54.640 INFO:teuthology.orchestra.run.smithi053.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-11T21:44:54.665 INFO:teuthology.orchestra.run.smithi053.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-09-11T21:44:54.690 INFO:teuthology.orchestra.run.smithi053.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-11T21:44:54.715 INFO:teuthology.orchestra.run.smithi053.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2024-09-11T21:44:54.738 INFO:teuthology.orchestra.run.smithi161.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 1.7 MB/s | 191 kB 00:00 2024-09-11T21:44:54.741 INFO:teuthology.orchestra.run.smithi053.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 428 kB/s | 11 kB 00:00 2024-09-11T21:44:54.764 INFO:teuthology.orchestra.run.smithi161.stdout:(41/156): libgfortran-11.5.0-2.el9.x86_64.rpm 2.6 MB/s | 801 kB 00:00 2024-09-11T21:44:54.766 INFO:teuthology.orchestra.run.smithi053.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 709 kB/s | 18 kB 00:00 2024-09-11T21:44:54.783 INFO:teuthology.orchestra.run.smithi053.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 1.4 MB/s | 23 kB 00:00 2024-09-11T21:44:54.817 INFO:teuthology.orchestra.run.smithi053.stdout:(126/156): protobuf-compiler-3.14.0-13.el9.x86_ 805 kB/s | 863 kB 00:01 2024-09-11T21:44:54.822 INFO:teuthology.orchestra.run.smithi161.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.0 MB/s | 253 kB 00:00 2024-09-11T21:44:54.833 INFO:teuthology.orchestra.run.smithi053.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-09-11T21:44:54.873 INFO:teuthology.orchestra.run.smithi161.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-11T21:44:54.879 INFO:teuthology.orchestra.run.smithi001.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 174 kB/s | 80 kB 00:00 2024-09-11T21:44:54.892 INFO:teuthology.orchestra.run.smithi053.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 261 kB/s | 20 kB 00:00 2024-09-11T21:44:54.917 INFO:teuthology.orchestra.run.smithi053.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-11T21:44:54.931 INFO:teuthology.orchestra.run.smithi161.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-09-11T21:44:54.943 INFO:teuthology.orchestra.run.smithi053.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-11T21:44:54.976 INFO:teuthology.orchestra.run.smithi053.stdout:(131/156): python3-kubernetes-26.1.0-2.el9.noar 7.6 MB/s | 1.0 MB 00:00 2024-09-11T21:44:54.982 INFO:teuthology.orchestra.run.smithi161.stdout:(45/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-11T21:44:55.001 INFO:teuthology.orchestra.run.smithi053.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 988 kB/s | 58 kB 00:00 2024-09-11T21:44:55.027 INFO:teuthology.orchestra.run.smithi053.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 665 kB/s | 16 kB 00:00 2024-09-11T21:44:55.041 INFO:teuthology.orchestra.run.smithi161.stdout:(46/156): python3-urllib3-1.26.5-6.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-09-11T21:44:55.052 INFO:teuthology.orchestra.run.smithi053.stdout:(134/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-09-11T21:44:55.094 INFO:teuthology.orchestra.run.smithi053.stdout:(135/156): python3-grpcio-1.46.7-10.el9.x86_64. 4.8 MB/s | 2.0 MB 00:00 2024-09-11T21:44:55.119 INFO:teuthology.orchestra.run.smithi053.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 422 kB/s | 28 kB 00:00 2024-09-11T21:44:55.144 INFO:teuthology.orchestra.run.smithi053.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-09-11T21:44:55.172 INFO:teuthology.orchestra.run.smithi001.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 655 kB/s | 191 kB 00:00 2024-09-11T21:44:55.178 INFO:teuthology.orchestra.run.smithi053.stdout:(138/156): python3-pecan-1.4.2-3.el9.noarch.rpm 1.5 MB/s | 272 kB 00:00 2024-09-11T21:44:55.182 INFO:teuthology.orchestra.run.smithi161.stdout:(47/156): python3-cryptography-36.0.1-4.el9.x86 2.9 MB/s | 1.2 MB 00:00 2024-09-11T21:44:55.203 INFO:teuthology.orchestra.run.smithi053.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-09-11T21:44:55.228 INFO:teuthology.orchestra.run.smithi053.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 715 kB/s | 36 kB 00:00 2024-09-11T21:44:55.254 INFO:teuthology.orchestra.run.smithi053.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.5 MB/s | 74 kB 00:00 2024-09-11T21:44:55.265 INFO:teuthology.orchestra.run.smithi161.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 3.0 MB/s | 689 kB 00:00 2024-09-11T21:44:55.279 INFO:teuthology.orchestra.run.smithi053.stdout:(142/156): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-09-11T21:44:55.304 INFO:teuthology.orchestra.run.smithi053.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-11T21:44:55.308 INFO:teuthology.orchestra.run.smithi161.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 2.5 MB/s | 107 kB 00:00 2024-09-11T21:44:55.330 INFO:teuthology.orchestra.run.smithi053.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 808 kB/s | 20 kB 00:00 2024-09-11T21:44:55.341 INFO:teuthology.orchestra.run.smithi161.stdout:(50/156): elfutils-libelf-devel-0.191-4.el9.x86 649 kB/s | 22 kB 00:00 2024-09-11T21:44:55.355 INFO:teuthology.orchestra.run.smithi053.stdout:(145/156): python3-webob-1.8.8-2.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-09-11T21:44:55.372 INFO:teuthology.orchestra.run.smithi001.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.2 MB/s | 253 kB 00:00 2024-09-11T21:44:55.387 INFO:teuthology.orchestra.run.smithi053.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-11T21:44:55.391 INFO:teuthology.orchestra.run.smithi161.stdout:(51/156): bison-3.7.4-5.el9.x86_64.rpm 4.4 MB/s | 943 kB 00:00 2024-09-11T21:44:55.414 INFO:teuthology.orchestra.run.smithi053.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-09-11T21:44:55.417 INFO:teuthology.orchestra.run.smithi161.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 4.1 MB/s | 313 kB 00:00 2024-09-11T21:44:55.434 INFO:teuthology.orchestra.run.smithi161.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 777 kB/s | 33 kB 00:00 2024-09-11T21:44:55.464 INFO:teuthology.orchestra.run.smithi053.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-11T21:44:55.467 INFO:teuthology.orchestra.run.smithi161.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 546 kB/s | 18 kB 00:00 2024-09-11T21:44:55.480 INFO:teuthology.orchestra.run.smithi001.stdout:(42/156): libgfortran-11.5.0-2.el9.x86_64.rpm 951 kB/s | 801 kB 00:00 2024-09-11T21:44:55.510 INFO:teuthology.orchestra.run.smithi161.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 266 kB/s | 11 kB 00:00 2024-09-11T21:44:55.576 INFO:teuthology.orchestra.run.smithi161.stdout:(56/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.4 MB/s | 160 kB 00:00 2024-09-11T21:44:55.581 INFO:teuthology.orchestra.run.smithi053.stdout:(149/156): NetworkManager-1.49.90-1.el9.x86_64. 20 MB/s | 2.3 MB 00:00 2024-09-11T21:44:55.597 INFO:teuthology.orchestra.run.smithi001.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 913 kB/s | 106 kB 00:00 2024-09-11T21:44:55.610 INFO:teuthology.orchestra.run.smithi161.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.3 MB/s | 45 kB 00:00 2024-09-11T21:44:55.690 INFO:teuthology.orchestra.run.smithi053.stdout:(150/156): NetworkManager-libnm-1.49.90-1.el9.x 17 MB/s | 1.8 MB 00:00 2024-09-11T21:44:55.697 INFO:teuthology.orchestra.run.smithi001.stdout:(44/156): python3-cryptography-36.0.1-4.el9.x86 3.8 MB/s | 1.2 MB 00:00 2024-09-11T21:44:55.702 INFO:teuthology.orchestra.run.smithi161.stdout:(58/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 11 MB/s | 3.0 MB 00:00 2024-09-11T21:44:55.723 INFO:teuthology.orchestra.run.smithi001.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 1.1 MB/s | 135 kB 00:00 2024-09-11T21:44:55.765 INFO:teuthology.orchestra.run.smithi053.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 544 kB/s | 41 kB 00:00 2024-09-11T21:44:55.786 INFO:teuthology.orchestra.run.smithi161.stdout:(59/156): libtool-2.4.6-46.el9.x86_64.rpm 6.8 MB/s | 581 kB 00:00 2024-09-11T21:44:55.823 INFO:teuthology.orchestra.run.smithi001.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 996 kB/s | 125 kB 00:00 2024-09-11T21:44:55.827 INFO:teuthology.orchestra.run.smithi161.stdout:(60/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 588 kB/s | 24 kB 00:00 2024-09-11T21:44:55.848 INFO:teuthology.orchestra.run.smithi001.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 1.7 MB/s | 215 kB 00:00 2024-09-11T21:44:55.857 INFO:teuthology.orchestra.run.smithi053.stdout:(152/156): NetworkManager-tui-1.49.90-1.el9.x86 2.7 MB/s | 249 kB 00:00 2024-09-11T21:44:55.869 INFO:teuthology.orchestra.run.smithi161.stdout:(61/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-09-11T21:44:55.895 INFO:teuthology.orchestra.run.smithi161.stdout:(62/156): librdkafka-1.6.1-102.el9.x86_64.rpm 2.3 MB/s | 662 kB 00:00 2024-09-11T21:44:55.920 INFO:teuthology.orchestra.run.smithi161.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-09-11T21:44:55.962 INFO:teuthology.orchestra.run.smithi161.stdout:(64/156): openblas-0.3.26-2.el9.x86_64.rpm 923 kB/s | 38 kB 00:00 2024-09-11T21:44:55.965 INFO:teuthology.orchestra.run.smithi053.stdout:(153/156): openssl-3.2.2-6.el9.x86_64.rpm 13 MB/s | 1.4 MB 00:00 2024-09-11T21:44:55.987 INFO:teuthology.orchestra.run.smithi161.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 2.0 MB/s | 188 kB 00:00 2024-09-11T21:44:56.024 INFO:teuthology.orchestra.run.smithi001.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 3.4 MB/s | 689 kB 00:00 2024-09-11T21:44:56.057 INFO:teuthology.orchestra.run.smithi001.stdout:(49/156): bison-3.7.4-5.el9.x86_64.rpm 4.4 MB/s | 943 kB 00:00 2024-09-11T21:44:56.083 INFO:teuthology.orchestra.run.smithi001.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-09-11T21:44:56.108 INFO:teuthology.orchestra.run.smithi001.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 431 kB/s | 22 kB 00:00 2024-09-11T21:44:56.115 INFO:teuthology.orchestra.run.smithi053.stdout:(154/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 16 MB/s | 2.4 MB 00:00 2024-09-11T21:44:56.133 INFO:teuthology.orchestra.run.smithi001.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-09-11T21:44:56.158 INFO:teuthology.orchestra.run.smithi001.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 654 kB/s | 33 kB 00:00 2024-09-11T21:44:56.200 INFO:teuthology.orchestra.run.smithi001.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 437 kB/s | 18 kB 00:00 2024-09-11T21:44:56.242 INFO:teuthology.orchestra.run.smithi001.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 268 kB/s | 11 kB 00:00 2024-09-11T21:44:56.282 INFO:teuthology.orchestra.run.smithi053.stdout:(155/156): librados2-19.1.1-298.gb53b6d3e.el9.x 3.8 MB/s | 3.4 MB 00:00 2024-09-11T21:44:56.293 INFO:teuthology.orchestra.run.smithi001.stdout:(56/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 19 MB/s | 3.0 MB 00:00 2024-09-11T21:44:56.318 INFO:teuthology.orchestra.run.smithi001.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 160 kB 00:00 2024-09-11T21:44:56.343 INFO:teuthology.orchestra.run.smithi001.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 904 kB/s | 45 kB 00:00 2024-09-11T21:44:56.377 INFO:teuthology.orchestra.run.smithi001.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-09-11T21:44:56.419 INFO:teuthology.orchestra.run.smithi001.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 7.5 MB/s | 581 kB 00:00 2024-09-11T21:44:56.444 INFO:teuthology.orchestra.run.smithi001.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 366 kB/s | 24 kB 00:00 2024-09-11T21:44:56.457 INFO:teuthology.orchestra.run.smithi053.stdout:(156/156): librbd1-19.1.1-298.gb53b6d3e.el9.x86 3.0 MB/s | 3.2 MB 00:01 2024-09-11T21:44:56.464 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-09-11T21:44:56.464 INFO:teuthology.orchestra.run.smithi053.stdout:Total 21 MB/s | 226 MB 00:10 2024-09-11T21:44:56.469 INFO:teuthology.orchestra.run.smithi001.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 948 kB/s | 47 kB 00:00 2024-09-11T21:44:56.488 INFO:teuthology.orchestra.run.smithi161.stdout:(66/156): ceph-test-19.1.1-298.gb53b6d3e.el9.x8 10 MB/s | 49 MB 00:04 2024-09-11T21:44:56.494 INFO:teuthology.orchestra.run.smithi001.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-09-11T21:44:56.520 INFO:teuthology.orchestra.run.smithi001.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-09-11T21:44:56.537 INFO:teuthology.orchestra.run.smithi001.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 918 kB/s | 38 kB 00:00 2024-09-11T21:44:56.588 INFO:teuthology.orchestra.run.smithi161.stdout:(67/156): openblas-openmp-0.3.26-2.el9.x86_64.r 7.9 MB/s | 4.9 MB 00:00 2024-09-11T21:44:56.637 INFO:teuthology.orchestra.run.smithi001.stdout:(66/156): openblas-openmp-0.3.26-2.el9.x86_64.r 42 MB/s | 4.9 MB 00:00 2024-09-11T21:44:56.663 INFO:teuthology.orchestra.run.smithi161.stdout:(68/156): postgresql-contrib-13.16-1.el9.x86_64 12 MB/s | 910 kB 00:00 2024-09-11T21:44:56.705 INFO:teuthology.orchestra.run.smithi161.stdout:(69/156): postgresql-private-libs-13.16-1.el9.x 3.2 MB/s | 137 kB 00:00 2024-09-11T21:44:56.729 INFO:teuthology.orchestra.run.smithi001.stdout:(67/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 23 MB/s | 4.4 MB 00:00 2024-09-11T21:44:56.780 INFO:teuthology.orchestra.run.smithi001.stdout:(68/156): postgresql-13.16-1.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-09-11T21:44:56.821 INFO:teuthology.orchestra.run.smithi001.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 9.7 MB/s | 910 kB 00:00 2024-09-11T21:44:56.838 INFO:teuthology.orchestra.run.smithi001.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 2.3 MB/s | 137 kB 00:00 2024-09-11T21:44:56.909 INFO:teuthology.orchestra.run.smithi001.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-09-11T21:44:56.956 INFO:teuthology.orchestra.run.smithi001.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-09-11T21:44:56.972 INFO:teuthology.orchestra.run.smithi161.stdout:(70/156): postgresql-13.16-1.el9.x86_64.rpm 3.3 MB/s | 1.6 MB 00:00 2024-09-11T21:44:57.031 INFO:teuthology.orchestra.run.smithi001.stdout:(73/156): postgresql-server-13.16-1.el9.x86_64. 28 MB/s | 5.8 MB 00:00 2024-09-11T21:44:57.065 INFO:teuthology.orchestra.run.smithi001.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.8 MB/s | 61 kB 00:00 2024-09-11T21:44:57.106 INFO:teuthology.orchestra.run.smithi161.stdout:(71/156): postgresql-server-13.16-1.el9.x86_64. 14 MB/s | 5.8 MB 00:00 2024-09-11T21:44:57.147 INFO:teuthology.orchestra.run.smithi161.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 2.1 MB/s | 87 kB 00:00 2024-09-11T21:44:57.148 INFO:teuthology.orchestra.run.smithi001.stdout:(75/156): python3-babel-2.9.1-2.el9.noarch.rpm 31 MB/s | 6.0 MB 00:00 2024-09-11T21:44:57.173 INFO:teuthology.orchestra.run.smithi001.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.2 MB/s | 245 kB 00:00 2024-09-11T21:44:57.199 INFO:teuthology.orchestra.run.smithi001.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-09-11T21:44:57.216 INFO:teuthology.orchestra.run.smithi001.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 612 kB/s | 26 kB 00:00 2024-09-11T21:44:57.232 INFO:teuthology.orchestra.run.smithi001.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 574 kB/s | 19 kB 00:00 2024-09-11T21:44:57.266 INFO:teuthology.orchestra.run.smithi001.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-09-11T21:44:57.281 INFO:teuthology.orchestra.run.smithi161.stdout:(73/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 3.4 MB/s | 4.4 MB 00:01 2024-09-11T21:44:57.291 INFO:teuthology.orchestra.run.smithi001.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 594 kB/s | 35 kB 00:00 2024-09-11T21:44:57.314 INFO:teuthology.orchestra.run.smithi161.stdout:(74/156): protobuf-3.14.0-13.el9.x86_64.rpm 2.9 MB/s | 1.0 MB 00:00 2024-09-11T21:44:57.340 INFO:teuthology.orchestra.run.smithi161.stdout:(75/156): python3-colorama-0.4.6-3.el9.noarch.r 1.0 MB/s | 61 kB 00:00 2024-09-11T21:44:57.342 INFO:teuthology.orchestra.run.smithi001.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.9 MB/s | 250 kB 00:00 2024-09-11T21:44:57.407 INFO:teuthology.orchestra.run.smithi161.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.6 MB/s | 245 kB 00:00 2024-09-11T21:44:57.417 INFO:teuthology.orchestra.run.smithi001.stdout:(83/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 34 MB/s | 5.1 MB 00:00 2024-09-11T21:44:57.442 INFO:teuthology.orchestra.run.smithi001.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-09-11T21:44:57.448 INFO:teuthology.orchestra.run.smithi161.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 2.2 MB/s | 248 kB 00:00 2024-09-11T21:44:57.465 INFO:teuthology.orchestra.run.smithi161.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 440 kB/s | 26 kB 00:00 2024-09-11T21:44:57.467 INFO:teuthology.orchestra.run.smithi001.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-11T21:44:57.491 INFO:teuthology.orchestra.run.smithi161.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 457 kB/s | 19 kB 00:00 2024-09-11T21:44:57.493 INFO:teuthology.orchestra.run.smithi001.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 846 kB/s | 42 kB 00:00 2024-09-11T21:44:57.518 INFO:teuthology.orchestra.run.smithi001.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-09-11T21:44:57.543 INFO:teuthology.orchestra.run.smithi001.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-11T21:44:57.575 INFO:teuthology.orchestra.run.smithi161.stdout:(80/156): python3-babel-2.9.1-2.el9.noarch.rpm 14 MB/s | 6.0 MB 00:00 2024-09-11T21:44:57.585 INFO:teuthology.orchestra.run.smithi001.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 279 kB 00:00 2024-09-11T21:44:57.591 INFO:teuthology.orchestra.run.smithi161.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 347 kB/s | 35 kB 00:00 2024-09-11T21:44:57.610 INFO:teuthology.orchestra.run.smithi001.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 764 kB/s | 51 kB 00:00 2024-09-11T21:44:57.617 INFO:teuthology.orchestra.run.smithi161.stdout:(82/156): python3-mako-1.1.4-6.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2024-09-11T21:44:57.636 INFO:teuthology.orchestra.run.smithi001.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-11T21:44:57.684 INFO:teuthology.orchestra.run.smithi161.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 3.3 MB/s | 222 kB 00:00 2024-09-11T21:44:57.709 INFO:teuthology.orchestra.run.smithi161.stdout:(84/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.1 MB/s | 250 kB 00:00 2024-09-11T21:44:57.709 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-09-11T21:44:57.734 INFO:teuthology.orchestra.run.smithi161.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-11T21:44:57.751 INFO:teuthology.orchestra.run.smithi161.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-09-11T21:44:57.826 INFO:teuthology.orchestra.run.smithi161.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-09-11T21:44:57.852 INFO:teuthology.orchestra.run.smithi161.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 2.2 MB/s | 269 kB 00:00 2024-09-11T21:44:57.875 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-09-11T21:44:57.875 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-09-11T21:44:57.919 INFO:teuthology.orchestra.run.smithi161.stdout:(89/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 15 MB/s | 5.1 MB 00:00 2024-09-11T21:44:57.935 INFO:teuthology.orchestra.run.smithi161.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 612 kB/s | 51 kB 00:00 2024-09-11T21:44:57.952 INFO:teuthology.orchestra.run.smithi161.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.6 MB/s | 54 kB 00:00 2024-09-11T21:44:57.978 INFO:teuthology.orchestra.run.smithi161.stdout:(92/156): python3-pyasn1-modules-0.4.8-6.el9.no 1.8 MB/s | 279 kB 00:00 2024-09-11T21:44:57.994 INFO:teuthology.orchestra.run.smithi161.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 995 kB/s | 42 kB 00:00 2024-09-11T21:44:58.011 INFO:teuthology.orchestra.run.smithi161.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.3 MB/s | 44 kB 00:00 2024-09-11T21:44:58.037 INFO:teuthology.orchestra.run.smithi161.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 5.1 MB/s | 221 kB 00:00 2024-09-11T21:44:58.053 INFO:teuthology.orchestra.run.smithi161.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 855 kB/s | 36 kB 00:00 2024-09-11T21:44:58.069 INFO:teuthology.orchestra.run.smithi001.stdout:(92/156): ceph-test-19.1.1-298.gb53b6d3e.el9.x8 5.9 MB/s | 49 MB 00:08 2024-09-11T21:44:58.079 INFO:teuthology.orchestra.run.smithi161.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-09-11T21:44:58.095 INFO:teuthology.orchestra.run.smithi001.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 91 kB/s | 42 kB 00:00 2024-09-11T21:44:58.104 INFO:teuthology.orchestra.run.smithi161.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-09-11T21:44:58.121 INFO:teuthology.orchestra.run.smithi161.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 1.4 MB/s | 58 kB 00:00 2024-09-11T21:44:58.145 INFO:teuthology.orchestra.run.smithi001.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-11T21:44:58.146 INFO:teuthology.orchestra.run.smithi161.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 2.2 MB/s | 96 kB 00:00 2024-09-11T21:44:58.163 INFO:teuthology.orchestra.run.smithi161.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-09-11T21:44:58.179 INFO:teuthology.orchestra.run.smithi001.stdout:(95/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 407 kB/s | 44 kB 00:00 2024-09-11T21:44:58.188 INFO:teuthology.orchestra.run.smithi161.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 1.0 MB/s | 45 kB 00:00 2024-09-11T21:44:58.204 INFO:teuthology.orchestra.run.smithi001.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 613 kB/s | 36 kB 00:00 2024-09-11T21:44:58.238 INFO:teuthology.orchestra.run.smithi001.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 810 kB/s | 47 kB 00:00 2024-09-11T21:44:58.239 INFO:teuthology.orchestra.run.smithi161.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 297 kB/s | 22 kB 00:00 2024-09-11T21:44:58.263 INFO:teuthology.orchestra.run.smithi001.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 5.1 MB/s | 304 kB 00:00 2024-09-11T21:44:58.297 INFO:teuthology.orchestra.run.smithi001.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 988 kB/s | 58 kB 00:00 2024-09-11T21:44:58.322 INFO:teuthology.orchestra.run.smithi001.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.6 MB/s | 96 kB 00:00 2024-09-11T21:44:58.331 INFO:teuthology.orchestra.run.smithi161.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 5.8 MB/s | 548 kB 00:00 2024-09-11T21:44:58.355 INFO:teuthology.orchestra.run.smithi001.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-09-11T21:44:58.372 INFO:teuthology.orchestra.run.smithi161.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 7.3 MB/s | 309 kB 00:00 2024-09-11T21:44:58.381 INFO:teuthology.orchestra.run.smithi001.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 767 kB/s | 45 kB 00:00 2024-09-11T21:44:58.398 INFO:teuthology.orchestra.run.smithi161.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 780 kB/s | 19 kB 00:00 2024-09-11T21:44:58.548 INFO:teuthology.orchestra.run.smithi001.stdout:(103/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 17 MB/s | 16 MB 00:00 2024-09-11T21:44:58.557 INFO:teuthology.orchestra.run.smithi161.stdout:(107/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 26 MB/s | 16 MB 00:00 2024-09-11T21:44:58.598 INFO:teuthology.orchestra.run.smithi001.stdout:(104/156): lua-devel-5.4.4-4.el9.x86_64.rpm 92 kB/s | 22 kB 00:00 2024-09-11T21:44:58.640 INFO:teuthology.orchestra.run.smithi001.stdout:(105/156): abseil-cpp-20211102.0-3.el9.x86_64.r 5.8 MB/s | 548 kB 00:00 2024-09-11T21:44:58.665 INFO:teuthology.orchestra.run.smithi161.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 241 kB/s | 26 kB 00:00 2024-09-11T21:44:58.665 INFO:teuthology.orchestra.run.smithi001.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 851 kB/s | 19 kB 00:00 2024-09-11T21:44:58.690 INFO:teuthology.orchestra.run.smithi161.stdout:(109/156): protobuf-compiler-3.14.0-13.el9.x86_ 1.7 MB/s | 863 kB 00:00 2024-09-11T21:44:58.699 INFO:teuthology.orchestra.run.smithi001.stdout:(107/156): gperftools-libs-2.9.1-2.el9.x86_64.r 3.0 MB/s | 309 kB 00:00 2024-09-11T21:44:58.716 INFO:teuthology.orchestra.run.smithi161.stdout:(110/156): liboath-2.6.7-2.el9.x86_64.rpm 957 kB/s | 49 kB 00:00 2024-09-11T21:44:58.724 INFO:teuthology.orchestra.run.smithi001.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-11T21:44:58.757 INFO:teuthology.orchestra.run.smithi001.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.4 MB/s | 49 kB 00:00 2024-09-11T21:44:58.758 INFO:teuthology.orchestra.run.smithi161.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 4.3 MB/s | 151 kB 00:00 2024-09-11T21:44:58.783 INFO:teuthology.orchestra.run.smithi161.stdout:(112/156): libunwind-1.6.2-1.el9.x86_64.rpm 729 kB/s | 67 kB 00:00 2024-09-11T21:44:58.791 INFO:teuthology.orchestra.run.smithi001.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.0 MB/s | 67 kB 00:00 2024-09-11T21:44:58.825 INFO:teuthology.orchestra.run.smithi001.stdout:(111/156): protobuf-compiler-3.14.0-13.el9.x86_ 1.9 MB/s | 863 kB 00:00 2024-09-11T21:44:58.825 INFO:teuthology.orchestra.run.smithi161.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 12 MB/s | 839 kB 00:00 2024-09-11T21:44:58.859 INFO:teuthology.orchestra.run.smithi161.stdout:(114/156): python3-asyncssh-2.13.2-5.el9.noarch 7.1 MB/s | 548 kB 00:00 2024-09-11T21:44:58.867 INFO:teuthology.orchestra.run.smithi001.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-09-11T21:44:58.884 INFO:teuthology.orchestra.run.smithi161.stdout:(115/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 742 kB/s | 43 kB 00:00 2024-09-11T21:44:58.909 INFO:teuthology.orchestra.run.smithi161.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 643 kB/s | 32 kB 00:00 2024-09-11T21:44:58.934 INFO:teuthology.orchestra.run.smithi161.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 280 kB/s | 14 kB 00:00 2024-09-11T21:44:58.942 INFO:teuthology.orchestra.run.smithi001.stdout:(113/156): libarrow-9.0.0-11.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-09-11T21:44:58.960 INFO:teuthology.orchestra.run.smithi161.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-11T21:44:58.967 INFO:teuthology.orchestra.run.smithi001.stdout:(114/156): python3-asyncssh-2.13.2-5.el9.noarch 5.5 MB/s | 548 kB 00:00 2024-09-11T21:44:58.985 INFO:teuthology.orchestra.run.smithi161.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-09-11T21:44:58.992 INFO:teuthology.orchestra.run.smithi001.stdout:(115/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 870 kB/s | 43 kB 00:00 2024-09-11T21:44:59.010 INFO:teuthology.orchestra.run.smithi161.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-09-11T21:44:59.017 INFO:teuthology.orchestra.run.smithi001.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 642 kB/s | 32 kB 00:00 2024-09-11T21:44:59.043 INFO:teuthology.orchestra.run.smithi001.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 280 kB/s | 14 kB 00:00 2024-09-11T21:44:59.077 INFO:teuthology.orchestra.run.smithi001.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-09-11T21:44:59.089 INFO:teuthology.orchestra.run.smithi161.stdout:(121/156): libarrow-9.0.0-11.el9.x86_64.rpm 6.4 MB/s | 4.4 MB 00:00 2024-09-11T21:44:59.102 INFO:teuthology.orchestra.run.smithi001.stdout:(119/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.0 MB/s | 839 kB 00:00 2024-09-11T21:44:59.119 INFO:teuthology.orchestra.run.smithi161.stdout:(122/156): python3-google-auth-2.34.0-1.el9.noa 1.6 MB/s | 220 kB 00:00 2024-09-11T21:44:59.127 INFO:teuthology.orchestra.run.smithi001.stdout:(120/156): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-09-11T21:44:59.144 INFO:teuthology.orchestra.run.smithi161.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 2.6 MB/s | 144 kB 00:00 2024-09-11T21:44:59.152 INFO:teuthology.orchestra.run.smithi001.stdout:(121/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-09-11T21:44:59.170 INFO:teuthology.orchestra.run.smithi161.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 213 kB/s | 11 kB 00:00 2024-09-11T21:44:59.178 INFO:teuthology.orchestra.run.smithi001.stdout:(122/156): python3-google-auth-2.34.0-1.el9.noa 2.8 MB/s | 220 kB 00:00 2024-09-11T21:44:59.195 INFO:teuthology.orchestra.run.smithi161.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 351 kB/s | 18 kB 00:00 2024-09-11T21:44:59.203 INFO:teuthology.orchestra.run.smithi001.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-09-11T21:44:59.220 INFO:teuthology.orchestra.run.smithi161.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 461 kB/s | 23 kB 00:00 2024-09-11T21:44:59.228 INFO:teuthology.orchestra.run.smithi001.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 213 kB/s | 11 kB 00:00 2024-09-11T21:44:59.246 INFO:teuthology.orchestra.run.smithi161.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 388 kB/s | 19 kB 00:00 2024-09-11T21:44:59.253 INFO:teuthology.orchestra.run.smithi001.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-09-11T21:44:59.271 INFO:teuthology.orchestra.run.smithi161.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-09-11T21:44:59.278 INFO:teuthology.orchestra.run.smithi001.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 461 kB/s | 23 kB 00:00 2024-09-11T21:44:59.296 INFO:teuthology.orchestra.run.smithi161.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-11T21:44:59.304 INFO:teuthology.orchestra.run.smithi001.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-09-11T21:44:59.338 INFO:teuthology.orchestra.run.smithi001.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 334 kB/s | 20 kB 00:00 2024-09-11T21:44:59.338 INFO:teuthology.orchestra.run.smithi161.stdout:(130/156): python3-kubernetes-26.1.0-2.el9.noar 11 MB/s | 1.0 MB 00:00 2024-09-11T21:44:59.365 INFO:teuthology.orchestra.run.smithi001.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-11T21:44:59.365 INFO:teuthology.orchestra.run.smithi161.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 1.2 MB/s | 79 kB 00:00 2024-09-11T21:44:59.389 INFO:teuthology.orchestra.run.smithi161.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-09-11T21:44:59.397 INFO:teuthology.orchestra.run.smithi001.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-09-11T21:44:59.414 INFO:teuthology.orchestra.run.smithi161.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 5.3 MB/s | 272 kB 00:00 2024-09-11T21:44:59.431 INFO:teuthology.orchestra.run.smithi161.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 392 kB/s | 16 kB 00:00 2024-09-11T21:44:59.439 INFO:teuthology.orchestra.run.smithi001.stdout:(131/156): python3-kubernetes-26.1.0-2.el9.noar 7.6 MB/s | 1.0 MB 00:00 2024-09-11T21:44:59.448 INFO:teuthology.orchestra.run.smithi161.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 2.6 MB/s | 90 kB 00:00 2024-09-11T21:44:59.464 INFO:teuthology.orchestra.run.smithi001.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 862 kB/s | 58 kB 00:00 2024-09-11T21:44:59.465 INFO:teuthology.orchestra.run.smithi161.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 844 kB/s | 28 kB 00:00 2024-09-11T21:44:59.482 INFO:teuthology.orchestra.run.smithi161.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 920 kB/s | 31 kB 00:00 2024-09-11T21:44:59.489 INFO:teuthology.orchestra.run.smithi001.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 656 kB/s | 16 kB 00:00 2024-09-11T21:44:59.514 INFO:teuthology.orchestra.run.smithi001.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-09-11T21:44:59.532 INFO:teuthology.orchestra.run.smithi161.stdout:(138/156): python3-grpcio-1.46.7-10.el9.x86_64. 3.9 MB/s | 2.0 MB 00:00 2024-09-11T21:44:59.556 INFO:teuthology.orchestra.run.smithi001.stdout:(135/156): python3-grpcio-1.46.7-10.el9.x86_64. 4.8 MB/s | 2.0 MB 00:00 2024-09-11T21:44:59.557 INFO:teuthology.orchestra.run.smithi161.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.0 MB/s | 188 kB 00:00 2024-09-11T21:44:59.582 INFO:teuthology.orchestra.run.smithi001.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 422 kB/s | 28 kB 00:00 2024-09-11T21:44:59.582 INFO:teuthology.orchestra.run.smithi161.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 710 kB/s | 36 kB 00:00 2024-09-11T21:44:59.607 INFO:teuthology.orchestra.run.smithi001.stdout:(137/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 766 kB/s | 90 kB 00:00 2024-09-11T21:44:59.608 INFO:teuthology.orchestra.run.smithi161.stdout:(141/156): python3-rsa-4.9-2.el9.noarch.rpm 469 kB/s | 59 kB 00:00 2024-09-11T21:44:59.625 INFO:teuthology.orchestra.run.smithi161.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 1.1 MB/s | 74 kB 00:00 2024-09-11T21:44:59.632 INFO:teuthology.orchestra.run.smithi001.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 407 kB/s | 31 kB 00:00 2024-09-11T21:44:59.650 INFO:teuthology.orchestra.run.smithi161.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 2.1 MB/s | 90 kB 00:00 2024-09-11T21:44:59.657 INFO:teuthology.orchestra.run.smithi001.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-09-11T21:44:59.675 INFO:teuthology.orchestra.run.smithi161.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-09-11T21:44:59.683 INFO:teuthology.orchestra.run.smithi001.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 714 kB/s | 36 kB 00:00 2024-09-11T21:44:59.701 INFO:teuthology.orchestra.run.smithi161.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 399 kB/s | 20 kB 00:00 2024-09-11T21:44:59.708 INFO:teuthology.orchestra.run.smithi001.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-11T21:44:59.726 INFO:teuthology.orchestra.run.smithi161.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-11T21:44:59.733 INFO:teuthology.orchestra.run.smithi001.stdout:(142/156): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-09-11T21:44:59.751 INFO:teuthology.orchestra.run.smithi161.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-11T21:44:59.760 INFO:teuthology.orchestra.run.smithi001.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-11T21:44:59.775 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-09-11T21:44:59.775 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-09-11T21:44:59.784 INFO:teuthology.orchestra.run.smithi001.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-11T21:44:59.801 INFO:teuthology.orchestra.run.smithi001.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 477 kB/s | 20 kB 00:00 2024-09-11T21:44:59.810 INFO:teuthology.orchestra.run.smithi161.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-11T21:44:59.826 INFO:teuthology.orchestra.run.smithi001.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 4.5 MB/s | 191 kB 00:00 2024-09-11T21:44:59.851 INFO:teuthology.orchestra.run.smithi001.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.5 MB/s | 427 kB 00:00 2024-09-11T21:44:59.901 INFO:teuthology.orchestra.run.smithi001.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-11T21:45:00.027 INFO:teuthology.orchestra.run.smithi161.stdout:(149/156): librbd1-19.1.1-298.gb53b6d3e.el9.x86 11 MB/s | 3.2 MB 00:00 2024-09-11T21:45:00.143 INFO:teuthology.orchestra.run.smithi161.stdout:(150/156): librados2-19.1.1-298.gb53b6d3e.el9.x 8.2 MB/s | 3.4 MB 00:00 2024-09-11T21:45:00.244 INFO:teuthology.orchestra.run.smithi001.stdout:(149/156): librbd1-19.1.1-298.gb53b6d3e.el9.x86 8.1 MB/s | 3.2 MB 00:00 2024-09-11T21:45:00.252 INFO:teuthology.orchestra.run.smithi161.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 376 kB/s | 41 kB 00:00 2024-09-11T21:45:00.302 INFO:teuthology.orchestra.run.smithi001.stdout:(150/156): librados2-19.1.1-298.gb53b6d3e.el9.x 7.2 MB/s | 3.4 MB 00:00 2024-09-11T21:45:00.361 INFO:teuthology.orchestra.run.smithi161.stdout:(152/156): NetworkManager-1.49.90-1.el9.x86_64. 4.2 MB/s | 2.3 MB 00:00 2024-09-11T21:45:00.386 INFO:teuthology.orchestra.run.smithi161.stdout:(153/156): NetworkManager-tui-1.49.90-1.el9.x86 1.8 MB/s | 249 kB 00:00 2024-09-11T21:45:00.428 INFO:teuthology.orchestra.run.smithi161.stdout:(154/156): NetworkManager-libnm-1.49.90-1.el9.x 4.5 MB/s | 1.8 MB 00:00 2024-09-11T21:45:00.486 INFO:teuthology.orchestra.run.smithi161.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 11 MB/s | 1.4 MB 00:00 2024-09-11T21:45:00.561 INFO:teuthology.orchestra.run.smithi161.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 14 MB/s | 2.4 MB 00:00 2024-09-11T21:45:00.567 INFO:teuthology.orchestra.run.smithi161.stdout:-------------------------------------------------------------------------------- 2024-09-11T21:45:00.567 INFO:teuthology.orchestra.run.smithi161.stdout:Total 17 MB/s | 226 MB 00:13 2024-09-11T21:45:00.594 INFO:teuthology.orchestra.run.smithi001.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 140 kB/s | 41 kB 00:00 2024-09-11T21:45:01.069 INFO:teuthology.orchestra.run.smithi001.stdout:(152/156): NetworkManager-tui-1.49.90-1.el9.x86 524 kB/s | 249 kB 00:00 2024-09-11T21:45:01.136 INFO:teuthology.orchestra.run.smithi001.stdout:(153/156): NetworkManager-1.49.90-1.el9.x86_64. 1.9 MB/s | 2.3 MB 00:01 2024-09-11T21:45:01.195 INFO:teuthology.orchestra.run.smithi001.stdout:(154/156): NetworkManager-libnm-1.49.90-1.el9.x 1.9 MB/s | 1.8 MB 00:00 2024-09-11T21:45:01.503 INFO:teuthology.orchestra.run.smithi001.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 3.1 MB/s | 1.4 MB 00:00 2024-09-11T21:45:01.788 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2024-09-11T21:45:01.939 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction check succeeded. 2024-09-11T21:45:01.939 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2024-09-11T21:45:02.337 INFO:teuthology.orchestra.run.smithi001.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 2.0 MB/s | 2.4 MB 00:01 2024-09-11T21:45:02.343 INFO:teuthology.orchestra.run.smithi001.stdout:-------------------------------------------------------------------------------- 2024-09-11T21:45:02.343 INFO:teuthology.orchestra.run.smithi001.stdout:Total 15 MB/s | 226 MB 00:15 2024-09-11T21:45:03.560 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction check 2024-09-11T21:45:03.728 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction check succeeded. 2024-09-11T21:45:03.728 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction test 2024-09-11T21:45:03.764 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded. 2024-09-11T21:45:03.764 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2024-09-11T21:45:06.097 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction test succeeded. 2024-09-11T21:45:06.098 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction 2024-09-11T21:45:06.112 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-09-11T21:45:06.211 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-11T21:45:06.326 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-11T21:45:06.498 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-11T21:45:06.538 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-11T21:45:06.563 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-11T21:45:06.608 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-11T21:45:07.016 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-11T21:45:07.072 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T21:45:07.197 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T21:45:07.231 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T21:45:07.323 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T21:45:07.352 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-11T21:45:07.517 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-11T21:45:07.574 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-11T21:45:07.601 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T21:45:07.707 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T21:45:07.727 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-11T21:45:07.790 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-11T21:45:07.812 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-11T21:45:07.833 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-11T21:45:08.021 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-11T21:45:08.162 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-11T21:45:08.285 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-11T21:45:08.373 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T21:45:08.444 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T21:45:08.670 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T21:45:08.780 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-11T21:45:08.874 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-11T21:45:08.908 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-11T21:45:08.943 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-11T21:45:09.018 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-11T21:45:09.069 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-11T21:45:09.109 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-11T21:45:09.153 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-11T21:45:09.185 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-11T21:45:09.238 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-11T21:45:09.312 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-11T21:45:09.372 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-11T21:45:09.402 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-11T21:45:09.437 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-11T21:45:09.516 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-11T21:45:09.557 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 34/164 2024-09-11T21:45:09.600 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 35/164 2024-09-11T21:45:09.628 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/164 2024-09-11T21:45:09.701 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-11T21:45:09.743 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-11T21:45:09.820 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-11T21:45:09.957 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-11T21:45:10.000 INFO:teuthology.orchestra.run.smithi161.stdout: Preparing : 1/1 2024-09-11T21:45:10.013 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-11T21:45:10.115 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-11T21:45:10.154 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-11T21:45:10.206 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-11T21:45:10.288 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-11T21:45:10.468 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-11T21:45:10.568 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-11T21:45:10.569 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-11T21:45:10.651 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-11T21:45:10.695 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-11T21:45:10.738 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-11T21:45:10.776 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 46/164 2024-09-11T21:45:10.818 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-11T21:45:10.900 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-11T21:45:10.970 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-11T21:45:11.009 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T21:45:11.021 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-11T21:45:11.076 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-11T21:45:11.131 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T21:45:11.150 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-11T21:45:11.159 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T21:45:11.188 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-11T21:45:11.225 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-11T21:45:11.250 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T21:45:11.279 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-11T21:45:11.280 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-11T21:45:11.322 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-11T21:45:11.366 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-11T21:45:11.408 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-11T21:45:11.427 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-11T21:45:11.477 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-11T21:45:11.505 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T21:45:11.549 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-11T21:45:11.592 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T21:45:11.614 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-11T21:45:11.661 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-11T21:45:11.682 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-11T21:45:11.707 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-11T21:45:11.717 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-11T21:45:11.760 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-11T21:45:11.859 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-11T21:45:11.866 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-11T21:45:11.900 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-11T21:45:11.940 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-11T21:45:11.965 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-11T21:45:11.990 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-11T21:45:11.999 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-11T21:45:12.060 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-11T21:45:12.131 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-11T21:45:12.188 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-11T21:45:12.227 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T21:45:12.230 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-11T21:45:12.259 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-11T21:45:12.290 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T21:45:12.293 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-11T21:45:12.332 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-11T21:45:12.372 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-11T21:45:12.402 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-11T21:45:12.436 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-11T21:45:12.470 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-11T21:45:12.512 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-11T21:45:12.528 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T21:45:12.546 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-11T21:45:12.621 INFO:teuthology.orchestra.run.smithi001.stdout: Preparing : 1/1 2024-09-11T21:45:12.633 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-11T21:45:12.635 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-11T21:45:12.693 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-11T21:45:12.724 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-11T21:45:12.727 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-11T21:45:12.749 INFO:teuthology.orchestra.run.smithi001.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-11T21:45:12.762 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-11T21:45:12.797 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-11T21:45:12.831 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-11T21:45:12.831 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-11T21:45:12.871 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-11T21:45:12.884 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-11T21:45:12.923 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-11T21:45:12.963 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-11T21:45:13.007 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-11T21:45:13.039 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-11T21:45:13.057 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-11T21:45:13.067 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-11T21:45:13.110 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-11T21:45:13.134 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-11T21:45:13.137 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-11T21:45:13.185 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-11T21:45:13.191 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-11T21:45:13.215 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-11T21:45:13.259 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-11T21:45:13.346 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-11T21:45:13.377 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 34/164 2024-09-11T21:45:13.421 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 35/164 2024-09-11T21:45:13.449 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/164 2024-09-11T21:45:13.520 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-11T21:45:13.541 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-11T21:45:13.564 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-11T21:45:13.639 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-11T21:45:13.647 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-11T21:45:13.705 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-11T21:45:13.710 INFO:teuthology.orchestra.run.smithi001.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T21:45:13.739 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-11T21:45:13.782 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-11T21:45:13.834 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-11T21:45:13.835 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T21:45:13.841 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-11T21:45:13.861 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T21:45:13.894 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-11T21:45:13.940 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-11T21:45:13.954 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T21:45:13.982 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-11T21:45:14.029 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-11T21:45:14.066 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-11T21:45:14.147 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-11T21:45:14.205 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-11T21:45:14.232 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T21:45:14.257 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-11T21:45:14.288 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-11T21:45:14.321 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T21:45:14.349 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-11T21:45:14.361 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-11T21:45:14.368 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-11T21:45:14.395 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-11T21:45:14.417 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-11T21:45:14.439 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-11T21:45:14.444 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-11T21:45:14.483 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-11T21:45:14.526 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-11T21:45:14.564 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 46/164 2024-09-11T21:45:14.606 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-11T21:45:14.645 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-11T21:45:14.765 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-11T21:45:14.806 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-11T21:45:14.817 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-11T21:45:14.871 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-11T21:45:14.932 INFO:teuthology.orchestra.run.smithi001.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-11T21:45:14.945 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-11T21:45:14.976 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-11T21:45:15.012 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-11T21:45:15.028 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T21:45:15.045 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-11T21:45:15.059 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-11T21:45:15.103 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-11T21:45:15.146 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-11T21:45:15.158 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-11T21:45:15.188 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-11T21:45:15.236 INFO:teuthology.orchestra.run.smithi001.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T21:45:15.325 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-11T21:45:15.365 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-11T21:45:15.406 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-11T21:45:15.472 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T21:45:15.510 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-11T21:45:15.585 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-11T21:45:15.679 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-11T21:45:15.722 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-11T21:45:15.732 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-11T21:45:15.757 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-11T21:45:15.772 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-11T21:45:15.806 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-11T21:45:15.830 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-11T21:45:15.832 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-11T21:45:15.884 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-11T21:45:15.908 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-11T21:45:15.915 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-11T21:45:15.958 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-11T21:45:15.994 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-11T21:45:16.032 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-11T21:45:16.035 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-11T21:45:16.078 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-11T21:45:16.107 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-11T21:45:16.107 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-11T21:45:16.141 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-11T21:45:16.158 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-11T21:45:16.180 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-11T21:45:16.188 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-11T21:45:16.212 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-11T21:45:16.232 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-11T21:45:16.241 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-11T21:45:16.276 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-11T21:45:16.310 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-11T21:45:16.318 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-11T21:45:16.342 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 34/164 2024-09-11T21:45:16.359 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-11T21:45:16.386 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 35/164 2024-09-11T21:45:16.393 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-11T21:45:16.414 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/164 2024-09-11T21:45:16.446 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-11T21:45:16.480 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-11T21:45:16.487 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-11T21:45:16.524 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-11T21:45:16.529 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-11T21:45:16.555 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-11T21:45:16.585 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-11T21:45:16.608 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-11T21:45:16.621 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-11T21:45:16.654 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-11T21:45:16.660 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-11T21:45:16.722 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-11T21:45:16.752 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-11T21:45:16.808 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-11T21:45:16.982 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-11T21:45:17.025 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-11T21:45:17.100 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-11T21:45:17.130 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 102/164 2024-09-11T21:45:17.276 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-11T21:45:17.365 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-11T21:45:17.455 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-11T21:45:17.455 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-11T21:45:17.489 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-11T21:45:17.513 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-11T21:45:17.532 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-11T21:45:17.547 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-11T21:45:17.561 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 46/164 2024-09-11T21:45:17.604 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-11T21:45:17.647 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-11T21:45:17.693 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-11T21:45:17.738 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-11T21:45:17.759 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-11T21:45:17.781 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-11T21:45:17.811 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-11T21:45:17.815 INFO:teuthology.orchestra.run.smithi001.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-11T21:45:17.826 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-11T21:45:17.866 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-11T21:45:17.871 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-11T21:45:17.882 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-11T21:45:17.913 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 106/164 2024-09-11T21:45:17.946 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-11T21:45:17.982 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-11T21:45:18.018 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-11T21:45:18.050 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-11T21:45:18.080 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-11T21:45:18.158 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-11T21:45:18.159 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-11T21:45:18.193 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-11T21:45:18.237 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-11T21:45:18.279 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-11T21:45:18.422 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-11T21:45:18.463 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-11T21:45:18.505 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-11T21:45:18.612 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-11T21:45:18.655 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-11T21:45:18.695 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-11T21:45:18.728 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-11T21:45:18.753 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-11T21:45:18.808 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-11T21:45:18.841 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-11T21:45:18.920 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-11T21:45:18.976 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-11T21:45:19.010 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-11T21:45:19.039 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-11T21:45:19.078 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-11T21:45:19.120 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-11T21:45:19.163 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-11T21:45:19.193 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-11T21:45:19.227 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-11T21:45:19.260 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-11T21:45:19.303 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-11T21:45:19.337 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-11T21:45:19.426 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-11T21:45:19.476 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-11T21:45:19.506 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-11T21:45:19.614 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-11T21:45:19.675 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-11T21:45:20.152 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-11T21:45:20.242 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-11T21:45:20.269 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-11T21:45:20.303 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-11T21:45:20.438 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-11T21:45:20.480 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-11T21:45:20.513 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-11T21:45:20.586 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T21:45:20.613 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-11T21:45:20.616 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-11T21:45:20.626 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T21:45:20.665 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-11T21:45:20.677 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-11T21:45:20.722 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-11T21:45:20.756 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-11T21:45:20.787 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 102/164 2024-09-11T21:45:20.814 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-11T21:45:20.823 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-11T21:45:20.858 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-11T21:45:21.049 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-11T21:45:21.088 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-11T21:45:21.168 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-11T21:45:21.203 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T21:45:21.248 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-11T21:45:21.306 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-11T21:45:21.335 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-11T21:45:21.340 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-11T21:45:21.340 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-11T21:45:21.340 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-11T21:45:21.341 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-11T21:45:21.374 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-11T21:45:21.406 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-11T21:45:21.428 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 106/164 2024-09-11T21:45:21.858 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-11T21:45:21.983 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-11T21:45:23.314 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-11T21:45:23.400 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-11T21:45:23.429 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-11T21:45:23.462 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-11T21:45:23.721 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T21:45:23.749 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T21:45:23.794 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-11T21:45:23.832 INFO:teuthology.orchestra.run.smithi001.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-11T21:45:23.925 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-11T21:45:23.954 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 102/164 2024-09-11T21:45:23.996 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-11T21:45:24.294 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T21:45:24.328 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-11T21:45:24.413 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-11T21:45:24.413 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-11T21:45:24.413 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-11T21:45:24.413 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-11T21:45:24.446 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-11T21:45:24.505 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-11T21:45:24.677 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-11T21:45:24.748 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-11T21:45:24.778 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 106/164 2024-09-11T21:45:27.206 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T21:45:27.242 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T21:45:27.837 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T21:45:27.870 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-11T21:45:27.968 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-11T21:45:27.968 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-11T21:45:27.968 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-11T21:45:27.968 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:45:28.004 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-11T21:45:33.635 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-11T21:45:33.635 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /sys 2024-09-11T21:45:33.636 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /proc 2024-09-11T21:45:33.636 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /mnt 2024-09-11T21:45:33.636 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /var/tmp 2024-09-11T21:45:33.636 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /home 2024-09-11T21:45:33.636 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /root 2024-09-11T21:45:33.636 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /tmp 2024-09-11T21:45:33.636 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-11T21:45:33.818 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-11T21:45:33.834 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-11T21:45:33.835 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T21:45:33.835 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-11T21:45:33.835 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-11T21:45:35.379 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-11T21:45:35.406 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-11T21:45:35.406 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T21:45:35.406 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-11T21:45:35.406 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T21:45:35.406 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T21:45:35.407 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-11T21:45:35.646 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-11T21:45:35.670 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-11T21:45:35.670 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T21:45:35.670 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-11T21:45:35.670 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T21:45:35.670 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T21:45:35.670 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-11T21:45:36.130 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-11T21:45:36.153 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-11T21:45:36.154 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T21:45:36.154 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-11T21:45:36.154 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T21:45:36.154 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T21:45:36.154 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-11T21:45:36.261 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-11T21:45:36.319 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-11T21:45:36.391 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-11T21:45:36.513 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-11T21:45:36.513 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /sys 2024-09-11T21:45:36.513 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /proc 2024-09-11T21:45:36.513 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /mnt 2024-09-11T21:45:36.513 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /var/tmp 2024-09-11T21:45:36.514 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /home 2024-09-11T21:45:36.514 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /root 2024-09-11T21:45:36.514 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /tmp 2024-09-11T21:45:36.514 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-11T21:45:36.584 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-11T21:45:36.632 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-11T21:45:36.688 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-11T21:45:36.705 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-11T21:45:36.705 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T21:45:36.705 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-11T21:45:36.705 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-11T21:45:36.712 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-11T21:45:36.826 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-11T21:45:36.968 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-11T21:45:37.015 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-11T21:45:37.046 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-11T21:45:37.073 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-11T21:45:37.526 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-11T21:45:37.556 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-11T21:45:37.587 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 126/164 2024-09-11T21:45:37.644 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 127/164 2024-09-11T21:45:37.670 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-11T21:45:38.191 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-11T21:45:38.218 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-11T21:45:38.219 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T21:45:38.219 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-11T21:45:38.219 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T21:45:38.219 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T21:45:38.219 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-11T21:45:38.467 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-11T21:45:38.491 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-11T21:45:38.491 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T21:45:38.492 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-11T21:45:38.492 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T21:45:38.492 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T21:45:38.492 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-11T21:45:38.775 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-11T21:45:38.813 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-11T21:45:38.935 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-11T21:45:38.959 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-11T21:45:38.959 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T21:45:38.959 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-11T21:45:38.959 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T21:45:38.959 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T21:45:38.960 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-11T21:45:39.065 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-11T21:45:39.122 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-11T21:45:39.178 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-11T21:45:39.364 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-11T21:45:39.411 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-11T21:45:39.489 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-11T21:45:39.577 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-11T21:45:39.725 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-11T21:45:39.778 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-11T21:45:39.810 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-11T21:45:39.836 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-11T21:45:39.837 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-11T21:45:39.864 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-11T21:45:39.971 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-11T21:45:40.150 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 131/164 2024-09-11T21:45:40.182 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-11T21:45:40.195 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-11T21:45:40.195 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /sys 2024-09-11T21:45:40.196 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /proc 2024-09-11T21:45:40.196 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /mnt 2024-09-11T21:45:40.196 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /var/tmp 2024-09-11T21:45:40.196 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /home 2024-09-11T21:45:40.196 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /root 2024-09-11T21:45:40.196 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /tmp 2024-09-11T21:45:40.196 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:45:40.207 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-11T21:45:40.207 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T21:45:40.207 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-11T21:45:40.207 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T21:45:40.207 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T21:45:40.207 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-11T21:45:40.258 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-11T21:45:40.269 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-11T21:45:40.296 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-11T21:45:40.328 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-11T21:45:40.359 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 126/164 2024-09-11T21:45:40.372 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 134/164 2024-09-11T21:45:40.376 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-11T21:45:40.392 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-11T21:45:40.393 INFO:teuthology.orchestra.run.smithi001.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T21:45:40.393 INFO:teuthology.orchestra.run.smithi001.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-11T21:45:40.393 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:45:40.421 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 127/164 2024-09-11T21:45:40.424 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 135/164 2024-09-11T21:45:40.450 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-11T21:45:41.405 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-11T21:45:41.444 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-11T21:45:41.467 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-11T21:45:41.467 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T21:45:41.467 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-11T21:45:41.467 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T21:45:41.467 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T21:45:41.467 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-11T21:45:41.503 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-11T21:45:41.511 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-11T21:45:41.533 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-11T21:45:41.534 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T21:45:41.534 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-11T21:45:41.534 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-11T21:45:41.535 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-11T21:45:41.586 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-11T21:45:41.609 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-11T21:45:41.610 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T21:45:41.610 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-11T21:45:41.610 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-11T21:45:41.610 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-11T21:45:41.610 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-11T21:45:41.929 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-11T21:45:41.937 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-11T21:45:41.952 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-11T21:45:41.952 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T21:45:41.952 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-11T21:45:41.952 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T21:45:41.952 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T21:45:41.952 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-11T21:45:41.965 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-11T21:45:41.965 INFO:teuthology.orchestra.run.smithi001.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T21:45:41.965 INFO:teuthology.orchestra.run.smithi001.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-11T21:45:41.965 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T21:45:41.965 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T21:45:41.965 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:45:42.204 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-11T21:45:42.229 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-11T21:45:42.229 INFO:teuthology.orchestra.run.smithi001.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T21:45:42.229 INFO:teuthology.orchestra.run.smithi001.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-11T21:45:42.229 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T21:45:42.229 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T21:45:42.230 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:45:42.504 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-11T21:45:42.527 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-11T21:45:42.630 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-11T21:45:42.688 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-11T21:45:42.713 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-11T21:45:42.713 INFO:teuthology.orchestra.run.smithi001.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T21:45:42.713 INFO:teuthology.orchestra.run.smithi001.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-11T21:45:42.713 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T21:45:42.713 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T21:45:42.713 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:45:42.789 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 131/164 2024-09-11T21:45:42.819 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-11T21:45:42.821 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-11T21:45:42.845 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-11T21:45:42.845 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T21:45:42.845 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-11T21:45:42.846 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T21:45:42.846 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T21:45:42.846 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-11T21:45:42.877 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-11T21:45:42.888 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-11T21:45:42.900 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-11T21:45:42.940 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-11T21:45:42.998 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 134/164 2024-09-11T21:45:43.053 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 135/164 2024-09-11T21:45:43.134 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-11T21:45:43.181 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-11T21:45:43.261 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-11T21:45:43.352 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-11T21:45:43.502 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-11T21:45:43.540 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-11T21:45:43.579 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-11T21:45:43.607 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-11T21:45:43.980 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-11T21:45:44.000 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-11T21:45:44.007 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-11T21:45:44.030 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-11T21:45:44.031 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-11T21:45:44.031 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T21:45:44.031 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-11T21:45:44.031 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T21:45:44.031 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T21:45:44.031 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-11T21:45:44.062 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 126/164 2024-09-11T21:45:44.075 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-11T21:45:44.098 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-11T21:45:44.099 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T21:45:44.099 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-11T21:45:44.099 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-11T21:45:44.126 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 127/164 2024-09-11T21:45:44.150 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-11T21:45:44.174 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-11T21:45:44.174 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T21:45:44.174 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-11T21:45:44.174 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-11T21:45:44.174 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-11T21:45:44.174 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-11T21:45:44.178 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-11T21:45:44.476 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-11T21:45:44.500 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-11T21:45:44.500 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T21:45:44.500 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-11T21:45:44.500 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T21:45:44.500 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T21:45:44.500 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-11T21:45:45.284 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-11T21:45:45.330 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-11T21:45:46.350 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-11T21:45:46.380 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-11T21:45:46.488 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-11T21:45:46.658 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 131/164 2024-09-11T21:45:46.691 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-11T21:45:46.715 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-11T21:45:46.716 INFO:teuthology.orchestra.run.smithi001.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T21:45:46.716 INFO:teuthology.orchestra.run.smithi001.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-11T21:45:46.716 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T21:45:46.716 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T21:45:46.716 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:45:46.766 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-11T21:45:46.778 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-11T21:45:46.783 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 141/164 2024-09-11T21:45:46.833 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 142/164 2024-09-11T21:45:46.874 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 143/164 2024-09-11T21:45:46.888 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 134/164 2024-09-11T21:45:46.905 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-11T21:45:46.932 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-11T21:45:46.953 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 135/164 2024-09-11T21:45:47.119 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 146/164 2024-09-11T21:45:47.162 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-11T21:45:47.192 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 148/164 2024-09-11T21:45:47.222 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-11T21:45:47.268 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-11T21:45:47.844 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T21:45:47.882 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T21:45:47.903 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T21:45:47.929 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-11T21:45:47.961 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-11T21:45:47.986 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-11T21:45:47.986 INFO:teuthology.orchestra.run.smithi001.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T21:45:47.986 INFO:teuthology.orchestra.run.smithi001.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-11T21:45:47.986 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T21:45:47.986 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T21:45:47.986 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:45:48.028 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-11T21:45:48.051 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-11T21:45:48.052 INFO:teuthology.orchestra.run.smithi001.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T21:45:48.052 INFO:teuthology.orchestra.run.smithi001.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-11T21:45:48.052 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:45:48.103 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-11T21:45:48.127 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-11T21:45:48.128 INFO:teuthology.orchestra.run.smithi001.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T21:45:48.128 INFO:teuthology.orchestra.run.smithi001.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-11T21:45:48.128 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-11T21:45:48.128 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-11T21:45:48.128 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:45:48.446 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-11T21:45:48.470 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-11T21:45:48.471 INFO:teuthology.orchestra.run.smithi001.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T21:45:48.471 INFO:teuthology.orchestra.run.smithi001.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-11T21:45:48.471 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T21:45:48.471 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T21:45:48.471 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:45:48.809 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 152/164 2024-09-11T21:45:48.913 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-11T21:45:48.953 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-11T21:45:49.036 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-11T21:45:49.054 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 141/164 2024-09-11T21:45:49.072 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-11T21:45:49.096 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 142/164 2024-09-11T21:45:49.098 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-11T21:45:49.124 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-11T21:45:49.124 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-11T21:45:49.137 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 143/164 2024-09-11T21:45:49.169 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-11T21:45:49.191 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-11T21:45:49.191 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-11T21:45:49.196 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-11T21:45:49.275 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-11T21:45:49.300 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-11T21:45:49.306 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T21:45:49.306 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T21:45:49.377 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 146/164 2024-09-11T21:45:49.417 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-11T21:45:49.448 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 148/164 2024-09-11T21:45:49.477 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-11T21:45:49.494 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T21:45:49.494 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-11T21:45:49.523 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-11T21:45:49.578 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-11T21:45:49.644 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T21:45:50.146 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T21:45:50.179 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T21:45:50.200 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T21:45:51.056 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 152/164 2024-09-11T21:45:51.142 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-11T21:45:51.183 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-11T21:45:51.264 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-11T21:45:51.295 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-11T21:45:51.320 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-11T21:45:51.346 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-11T21:45:51.346 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-11T21:45:51.439 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-11T21:45:51.439 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-11T21:45:51.531 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-11T21:45:51.555 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-11T21:45:51.561 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T21:45:51.561 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T21:45:51.741 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T21:45:51.741 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-11T21:45:51.850 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-11T21:45:51.921 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T21:45:53.325 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 141/164 2024-09-11T21:45:53.367 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 142/164 2024-09-11T21:45:53.424 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 143/164 2024-09-11T21:45:53.456 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-11T21:45:53.483 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-11T21:45:53.701 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 146/164 2024-09-11T21:45:53.746 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-11T21:45:53.776 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 148/164 2024-09-11T21:45:53.806 INFO:teuthology.orchestra.run.smithi001.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-11T21:45:53.844 INFO:teuthology.orchestra.run.smithi001.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-11T21:45:54.489 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T21:45:54.533 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T21:45:54.554 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T21:45:55.083 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T21:45:55.083 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/164 2024-09-11T21:45:55.084 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/164 2024-09-11T21:45:55.084 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/164 2024-09-11T21:45:55.084 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/164 2024-09-11T21:45:55.084 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 5/164 2024-09-11T21:45:55.084 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/164 2024-09-11T21:45:55.084 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/164 2024-09-11T21:45:55.084 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T21:45:55.084 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T21:45:55.084 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-11T21:45:55.084 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/164 2024-09-11T21:45:55.085 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/164 2024-09-11T21:45:55.085 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T21:45:55.085 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 14/164 2024-09-11T21:45:55.085 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 15/164 2024-09-11T21:45:55.085 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 16/164 2024-09-11T21:45:55.085 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 17/164 2024-09-11T21:45:55.085 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 18/164 2024-09-11T21:45:55.085 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 19/164 2024-09-11T21:45:55.085 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 20/164 2024-09-11T21:45:55.085 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 21/164 2024-09-11T21:45:55.085 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 22/164 2024-09-11T21:45:55.085 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/164 2024-09-11T21:45:55.086 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 24/164 2024-09-11T21:45:55.086 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 25/164 2024-09-11T21:45:55.086 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/164 2024-09-11T21:45:55.086 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/164 2024-09-11T21:45:55.086 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 28/164 2024-09-11T21:45:55.086 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/164 2024-09-11T21:45:55.099 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 30/164 2024-09-11T21:45:55.099 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 31/164 2024-09-11T21:45:55.100 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 32/164 2024-09-11T21:45:55.100 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 33/164 2024-09-11T21:45:55.100 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 34/164 2024-09-11T21:45:55.100 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 35/164 2024-09-11T21:45:55.100 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 36/164 2024-09-11T21:45:55.100 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 37/164 2024-09-11T21:45:55.100 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 38/164 2024-09-11T21:45:55.100 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 39/164 2024-09-11T21:45:55.100 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-11T21:45:55.100 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-11T21:45:55.100 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-11T21:45:55.100 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-11T21:45:55.101 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-11T21:45:55.101 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-11T21:45:55.101 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-11T21:45:55.101 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-11T21:45:55.101 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-11T21:45:55.101 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-11T21:45:55.101 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-11T21:45:55.101 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-11T21:45:55.101 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-11T21:45:55.101 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-11T21:45:55.101 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-11T21:45:55.101 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-11T21:45:55.102 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-11T21:45:55.102 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-11T21:45:55.103 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-11T21:45:55.103 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-11T21:45:55.103 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-11T21:45:55.103 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-11T21:45:55.103 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-11T21:45:55.103 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-11T21:45:55.103 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-11T21:45:55.103 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-11T21:45:55.103 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-11T21:45:55.103 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-11T21:45:55.103 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-11T21:45:55.103 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-11T21:45:55.104 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-11T21:45:55.104 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-11T21:45:55.104 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-11T21:45:55.104 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-11T21:45:55.104 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-11T21:45:55.104 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-11T21:45:55.104 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-11T21:45:55.104 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-11T21:45:55.104 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-11T21:45:55.104 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-11T21:45:55.104 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-11T21:45:55.104 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-11T21:45:55.105 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-11T21:45:55.105 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-11T21:45:55.105 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-11T21:45:55.105 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-11T21:45:55.105 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-11T21:45:55.105 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-11T21:45:55.105 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-11T21:45:55.105 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-11T21:45:55.105 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-11T21:45:55.105 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-11T21:45:55.105 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-11T21:45:55.105 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-11T21:45:55.105 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-11T21:45:55.105 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-11T21:45:55.106 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-11T21:45:55.106 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-11T21:45:55.106 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-11T21:45:55.106 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-11T21:45:55.106 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-11T21:45:55.106 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-11T21:45:55.106 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-11T21:45:55.106 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-11T21:45:55.106 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-11T21:45:55.106 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-11T21:45:55.106 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-11T21:45:55.106 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-11T21:45:55.106 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-11T21:45:55.107 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-11T21:45:55.107 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-11T21:45:55.107 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-11T21:45:55.107 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-11T21:45:55.107 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-11T21:45:55.107 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-11T21:45:55.107 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-11T21:45:55.107 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-11T21:45:55.107 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-11T21:45:55.107 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-11T21:45:55.107 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-11T21:45:55.107 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-11T21:45:55.107 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-11T21:45:55.107 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-11T21:45:55.108 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-11T21:45:55.108 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-11T21:45:55.108 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-11T21:45:55.108 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-11T21:45:55.108 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-11T21:45:55.108 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-11T21:45:55.108 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-11T21:45:55.108 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-11T21:45:55.108 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-11T21:45:55.108 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-11T21:45:55.108 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-11T21:45:55.108 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-11T21:45:55.109 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-11T21:45:55.109 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-11T21:45:55.109 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-11T21:45:55.109 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-11T21:45:55.109 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-11T21:45:55.109 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-11T21:45:55.109 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-11T21:45:55.109 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-11T21:45:55.109 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-11T21:45:55.109 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-11T21:45:55.109 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-11T21:45:55.109 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-11T21:45:55.109 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-11T21:45:55.110 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-11T21:45:55.110 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 149/164 2024-09-11T21:45:55.110 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-11T21:45:55.110 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 151/164 2024-09-11T21:45:55.110 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-11T21:45:55.110 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-11T21:45:55.110 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-11T21:45:55.110 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-11T21:45:55.110 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-11T21:45:55.110 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-11T21:45:55.110 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-11T21:45:55.110 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-11T21:45:55.111 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-11T21:45:55.111 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-11T21:45:55.111 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-11T21:45:55.111 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-11T21:45:55.460 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 152/164 2024-09-11T21:45:55.563 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-11T21:45:55.610 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-11T21:45:55.705 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-11T21:45:55.740 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-11T21:45:55.771 INFO:teuthology.orchestra.run.smithi001.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-11T21:45:55.800 INFO:teuthology.orchestra.run.smithi001.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-11T21:45:55.800 INFO:teuthology.orchestra.run.smithi001.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-11T21:45:55.942 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-11T21:45:55.942 INFO:teuthology.orchestra.run.smithi001.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-11T21:45:56.027 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-11T21:45:56.052 INFO:teuthology.orchestra.run.smithi001.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-11T21:45:56.061 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T21:45:56.061 INFO:teuthology.orchestra.run.smithi001.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T21:45:56.245 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T21:45:56.245 INFO:teuthology.orchestra.run.smithi001.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-11T21:45:56.329 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-11T21:45:56.397 INFO:teuthology.orchestra.run.smithi001.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T21:45:57.337 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T21:45:57.337 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/164 2024-09-11T21:45:57.337 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/164 2024-09-11T21:45:57.337 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/164 2024-09-11T21:45:57.337 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/164 2024-09-11T21:45:57.337 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 5/164 2024-09-11T21:45:57.338 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/164 2024-09-11T21:45:57.338 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/164 2024-09-11T21:45:57.338 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T21:45:57.338 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T21:45:57.338 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-11T21:45:57.338 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/164 2024-09-11T21:45:57.338 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/164 2024-09-11T21:45:57.338 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T21:45:57.338 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 14/164 2024-09-11T21:45:57.338 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 15/164 2024-09-11T21:45:57.339 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 16/164 2024-09-11T21:45:57.339 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 17/164 2024-09-11T21:45:57.339 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 18/164 2024-09-11T21:45:57.339 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 19/164 2024-09-11T21:45:57.339 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 20/164 2024-09-11T21:45:57.339 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 21/164 2024-09-11T21:45:57.339 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 22/164 2024-09-11T21:45:57.339 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/164 2024-09-11T21:45:57.339 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 24/164 2024-09-11T21:45:57.340 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 25/164 2024-09-11T21:45:57.340 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/164 2024-09-11T21:45:57.340 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/164 2024-09-11T21:45:57.340 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 28/164 2024-09-11T21:45:57.340 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/164 2024-09-11T21:45:57.340 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 30/164 2024-09-11T21:45:57.340 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 31/164 2024-09-11T21:45:57.340 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 32/164 2024-09-11T21:45:57.340 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 33/164 2024-09-11T21:45:57.341 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 34/164 2024-09-11T21:45:57.341 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 35/164 2024-09-11T21:45:57.341 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 36/164 2024-09-11T21:45:57.341 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 37/164 2024-09-11T21:45:57.341 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 38/164 2024-09-11T21:45:57.341 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 39/164 2024-09-11T21:45:57.341 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-11T21:45:57.341 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-11T21:45:57.341 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-11T21:45:57.341 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-11T21:45:57.342 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-11T21:45:57.342 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-11T21:45:57.342 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-11T21:45:57.342 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-11T21:45:57.342 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-11T21:45:57.342 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-11T21:45:57.342 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-11T21:45:57.342 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-11T21:45:57.342 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-11T21:45:57.342 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-11T21:45:57.343 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-11T21:45:57.343 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-11T21:45:57.343 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-11T21:45:57.343 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-11T21:45:57.343 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-11T21:45:57.343 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-11T21:45:57.343 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-11T21:45:57.343 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-11T21:45:57.343 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-11T21:45:57.343 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-11T21:45:57.344 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-11T21:45:57.344 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-11T21:45:57.344 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-11T21:45:57.344 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-11T21:45:57.344 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-11T21:45:57.344 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-11T21:45:57.344 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-11T21:45:57.344 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-11T21:45:57.344 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-11T21:45:57.344 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-11T21:45:57.345 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-11T21:45:57.345 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-11T21:45:57.345 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-11T21:45:57.345 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-11T21:45:57.345 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-11T21:45:57.345 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-11T21:45:57.345 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-11T21:45:57.345 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-11T21:45:57.345 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-11T21:45:57.345 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-11T21:45:57.345 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-11T21:45:57.346 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-11T21:45:57.346 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-11T21:45:57.346 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-11T21:45:57.346 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-11T21:45:57.346 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-11T21:45:57.346 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-11T21:45:57.346 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-11T21:45:57.346 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-11T21:45:57.346 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-11T21:45:57.346 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-11T21:45:57.347 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-11T21:45:57.347 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-11T21:45:57.347 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-11T21:45:57.347 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-11T21:45:57.347 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-11T21:45:57.347 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-11T21:45:57.347 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-11T21:45:57.347 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-11T21:45:57.347 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-11T21:45:57.347 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-11T21:45:57.347 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-11T21:45:57.348 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-11T21:45:57.348 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-11T21:45:57.348 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-11T21:45:57.348 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-11T21:45:57.348 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-11T21:45:57.348 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-11T21:45:57.348 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-11T21:45:57.348 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-11T21:45:57.348 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-11T21:45:57.349 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-11T21:45:57.349 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-11T21:45:57.349 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-11T21:45:57.349 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-11T21:45:57.349 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-11T21:45:57.349 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-11T21:45:57.349 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-11T21:45:57.349 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-11T21:45:57.349 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-11T21:45:57.349 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-11T21:45:57.350 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-11T21:45:57.350 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-11T21:45:57.350 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-11T21:45:57.350 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-11T21:45:57.350 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-11T21:45:57.350 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-11T21:45:57.350 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-11T21:45:57.350 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-11T21:45:57.350 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-11T21:45:57.350 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-11T21:45:57.350 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-11T21:45:57.351 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-11T21:45:57.351 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-11T21:45:57.351 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-11T21:45:57.351 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-11T21:45:57.351 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-11T21:45:57.351 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-11T21:45:57.351 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-11T21:45:57.351 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-11T21:45:57.351 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-11T21:45:57.351 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-11T21:45:57.351 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-11T21:45:57.351 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-11T21:45:57.351 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-11T21:45:57.352 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 149/164 2024-09-11T21:45:57.352 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-11T21:45:57.352 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 151/164 2024-09-11T21:45:57.352 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-11T21:45:57.352 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-11T21:45:57.352 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-11T21:45:57.352 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-11T21:45:57.352 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-11T21:45:57.352 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-11T21:45:57.352 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-11T21:45:57.352 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-11T21:45:57.352 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-11T21:45:57.353 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-11T21:45:57.353 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-11T21:45:57.353 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-11T21:45:57.996 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T21:45:57.997 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-11T21:45:57.997 INFO:teuthology.orchestra.run.smithi053.stdout:Upgraded: 2024-09-11T21:45:57.997 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-11T21:45:57.997 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-11T21:45:57.997 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-11T21:45:57.997 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-11T21:45:57.997 INFO:teuthology.orchestra.run.smithi053.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:45:57.997 INFO:teuthology.orchestra.run.smithi053.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:45:57.997 INFO:teuthology.orchestra.run.smithi053.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-11T21:45:57.997 INFO:teuthology.orchestra.run.smithi053.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-11T21:45:57.997 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-09-11T21:45:57.998 INFO:teuthology.orchestra.run.smithi053.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-11T21:45:57.998 INFO:teuthology.orchestra.run.smithi053.stdout: automake-1.16.2-8.el9.noarch 2024-09-11T21:45:57.998 INFO:teuthology.orchestra.run.smithi053.stdout: bison-3.7.4-5.el9.x86_64 2024-09-11T21:45:57.998 INFO:teuthology.orchestra.run.smithi053.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-11T21:45:57.998 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:45:57.998 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:45:57.998 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:45:57.998 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:45:57.998 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T21:45:57.998 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:45:57.998 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:45:57.998 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:45:57.998 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T21:45:57.999 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T21:45:57.999 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T21:45:57.999 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T21:45:57.999 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T21:45:57.999 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:45:57.999 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:45:57.999 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T21:45:57.999 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:45:57.999 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:45:57.999 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:45:57.999 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T21:45:57.999 INFO:teuthology.orchestra.run.smithi053.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T21:45:57.999 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:45:58.000 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:45:58.000 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T21:45:58.000 INFO:teuthology.orchestra.run.smithi053.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-11T21:45:58.000 INFO:teuthology.orchestra.run.smithi053.stdout: flex-2.6.4-9.el9.x86_64 2024-09-11T21:45:58.000 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-11T21:45:58.000 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-11T21:45:58.000 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-11T21:45:58.000 INFO:teuthology.orchestra.run.smithi053.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-11T21:45:58.000 INFO:teuthology.orchestra.run.smithi053.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-11T21:45:58.000 INFO:teuthology.orchestra.run.smithi053.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-11T21:45:58.000 INFO:teuthology.orchestra.run.smithi053.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-11T21:45:58.000 INFO:teuthology.orchestra.run.smithi053.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-11T21:45:58.001 INFO:teuthology.orchestra.run.smithi053.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-11T21:45:58.001 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:45:58.001 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:45:58.001 INFO:teuthology.orchestra.run.smithi053.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:45:58.001 INFO:teuthology.orchestra.run.smithi053.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-11T21:45:58.001 INFO:teuthology.orchestra.run.smithi053.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-11T21:45:58.001 INFO:teuthology.orchestra.run.smithi053.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-11T21:45:58.001 INFO:teuthology.orchestra.run.smithi053.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-11T21:45:58.001 INFO:teuthology.orchestra.run.smithi053.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-11T21:45:58.001 INFO:teuthology.orchestra.run.smithi053.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:45:58.001 INFO:teuthology.orchestra.run.smithi053.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:45:58.001 INFO:teuthology.orchestra.run.smithi053.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-11T21:45:58.001 INFO:teuthology.orchestra.run.smithi053.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:45:58.002 INFO:teuthology.orchestra.run.smithi053.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-11T21:45:58.002 INFO:teuthology.orchestra.run.smithi053.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-11T21:45:58.002 INFO:teuthology.orchestra.run.smithi053.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-11T21:45:58.002 INFO:teuthology.orchestra.run.smithi053.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-11T21:45:58.002 INFO:teuthology.orchestra.run.smithi053.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-11T21:45:58.002 INFO:teuthology.orchestra.run.smithi053.stdout: lua-5.4.4-4.el9.x86_64 2024-09-11T21:45:58.002 INFO:teuthology.orchestra.run.smithi053.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-11T21:45:58.002 INFO:teuthology.orchestra.run.smithi053.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-11T21:45:58.002 INFO:teuthology.orchestra.run.smithi053.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-11T21:45:58.002 INFO:teuthology.orchestra.run.smithi053.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-11T21:45:58.002 INFO:teuthology.orchestra.run.smithi053.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-11T21:45:58.002 INFO:teuthology.orchestra.run.smithi053.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-11T21:45:58.003 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-13.16-1.el9.x86_64 2024-09-11T21:45:58.003 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-11T21:45:58.003 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-11T21:45:58.003 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-11T21:45:58.003 INFO:teuthology.orchestra.run.smithi053.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-11T21:45:58.003 INFO:teuthology.orchestra.run.smithi053.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-11T21:45:58.003 INFO:teuthology.orchestra.run.smithi053.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-11T21:45:58.003 INFO:teuthology.orchestra.run.smithi053.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-11T21:45:58.003 INFO:teuthology.orchestra.run.smithi053.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-11T21:45:58.003 INFO:teuthology.orchestra.run.smithi053.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-11T21:45:58.003 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-11T21:45:58.003 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:45:58.003 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:45:58.003 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:45:58.004 INFO:teuthology.orchestra.run.smithi053.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-11T21:45:58.004 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-11T21:45:58.004 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-11T21:45:58.004 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-11T21:45:58.004 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-11T21:45:58.004 INFO:teuthology.orchestra.run.smithi053.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-11T21:45:58.004 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-11T21:45:58.004 INFO:teuthology.orchestra.run.smithi053.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-11T21:45:58.004 INFO:teuthology.orchestra.run.smithi053.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-11T21:45:58.004 INFO:teuthology.orchestra.run.smithi053.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-11T21:45:58.004 INFO:teuthology.orchestra.run.smithi053.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-11T21:45:58.004 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-11T21:45:58.005 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-11T21:45:58.005 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-11T21:45:58.005 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-11T21:45:58.005 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-11T21:45:58.005 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-11T21:45:58.005 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-11T21:45:58.005 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-11T21:45:58.005 INFO:teuthology.orchestra.run.smithi053.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-11T21:45:58.005 INFO:teuthology.orchestra.run.smithi053.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-11T21:45:58.005 INFO:teuthology.orchestra.run.smithi053.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-11T21:45:58.005 INFO:teuthology.orchestra.run.smithi053.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-11T21:45:58.005 INFO:teuthology.orchestra.run.smithi053.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-11T21:45:58.005 INFO:teuthology.orchestra.run.smithi053.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-11T21:45:58.006 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-11T21:45:58.006 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-11T21:45:58.006 INFO:teuthology.orchestra.run.smithi053.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-11T21:45:58.006 INFO:teuthology.orchestra.run.smithi053.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-11T21:45:58.006 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-11T21:45:58.006 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ply-3.11-14.el9.noarch 2024-09-11T21:45:58.006 INFO:teuthology.orchestra.run.smithi053.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-11T21:45:58.006 INFO:teuthology.orchestra.run.smithi053.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-11T21:45:58.006 INFO:teuthology.orchestra.run.smithi053.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-11T21:45:58.006 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-11T21:45:58.006 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-11T21:45:58.006 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-11T21:45:58.007 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-11T21:45:58.007 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-11T21:45:58.007 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-11T21:45:58.007 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:45:58.007 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:45:58.007 INFO:teuthology.orchestra.run.smithi053.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-11T21:45:58.007 INFO:teuthology.orchestra.run.smithi053.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-11T21:45:58.007 INFO:teuthology.orchestra.run.smithi053.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-11T21:45:58.007 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:45:58.007 INFO:teuthology.orchestra.run.smithi053.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-11T21:45:58.007 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-11T21:45:58.007 INFO:teuthology.orchestra.run.smithi053.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-11T21:45:58.008 INFO:teuthology.orchestra.run.smithi053.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-11T21:45:58.008 INFO:teuthology.orchestra.run.smithi053.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-11T21:45:58.008 INFO:teuthology.orchestra.run.smithi053.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-11T21:45:58.008 INFO:teuthology.orchestra.run.smithi053.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-11T21:45:58.008 INFO:teuthology.orchestra.run.smithi053.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-11T21:45:58.008 INFO:teuthology.orchestra.run.smithi053.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-11T21:45:58.008 INFO:teuthology.orchestra.run.smithi053.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-11T21:45:58.008 INFO:teuthology.orchestra.run.smithi053.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-11T21:45:58.008 INFO:teuthology.orchestra.run.smithi053.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-11T21:45:58.008 INFO:teuthology.orchestra.run.smithi053.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-11T21:45:58.008 INFO:teuthology.orchestra.run.smithi053.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-11T21:45:58.008 INFO:teuthology.orchestra.run.smithi053.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-11T21:45:58.008 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:45:58.008 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:45:58.009 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:45:58.009 INFO:teuthology.orchestra.run.smithi053.stdout: re2-1:20211101-20.el9.x86_64 2024-09-11T21:45:58.009 INFO:teuthology.orchestra.run.smithi053.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-11T21:45:58.009 INFO:teuthology.orchestra.run.smithi053.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-11T21:45:58.009 INFO:teuthology.orchestra.run.smithi053.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-11T21:45:58.009 INFO:teuthology.orchestra.run.smithi053.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-11T21:45:58.009 INFO:teuthology.orchestra.run.smithi053.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-11T21:45:58.009 INFO:teuthology.orchestra.run.smithi053.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-11T21:45:58.010 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-11T21:45:58.010 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-11T21:45:58.399 DEBUG:teuthology.parallel:result is None 2024-09-11T21:46:00.122 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T21:46:00.122 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-11T21:46:00.122 INFO:teuthology.orchestra.run.smithi161.stdout:Upgraded: 2024-09-11T21:46:00.122 INFO:teuthology.orchestra.run.smithi161.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-11T21:46:00.123 INFO:teuthology.orchestra.run.smithi161.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-11T21:46:00.123 INFO:teuthology.orchestra.run.smithi161.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-11T21:46:00.123 INFO:teuthology.orchestra.run.smithi161.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-11T21:46:00.123 INFO:teuthology.orchestra.run.smithi161.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:46:00.123 INFO:teuthology.orchestra.run.smithi161.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:46:00.123 INFO:teuthology.orchestra.run.smithi161.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-11T21:46:00.123 INFO:teuthology.orchestra.run.smithi161.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-11T21:46:00.123 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2024-09-11T21:46:00.123 INFO:teuthology.orchestra.run.smithi161.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-11T21:46:00.123 INFO:teuthology.orchestra.run.smithi161.stdout: automake-1.16.2-8.el9.noarch 2024-09-11T21:46:00.123 INFO:teuthology.orchestra.run.smithi161.stdout: bison-3.7.4-5.el9.x86_64 2024-09-11T21:46:00.123 INFO:teuthology.orchestra.run.smithi161.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-11T21:46:00.124 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:46:00.124 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:46:00.124 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:46:00.124 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:46:00.124 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T21:46:00.124 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:46:00.124 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:46:00.124 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:46:00.124 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T21:46:00.124 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T21:46:00.124 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T21:46:00.125 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T21:46:00.125 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T21:46:00.125 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:46:00.128 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:46:00.128 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T21:46:00.128 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:46:00.128 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:46:00.129 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:46:00.129 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T21:46:00.129 INFO:teuthology.orchestra.run.smithi161.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T21:46:00.129 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:46:00.129 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:46:00.129 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T21:46:00.129 INFO:teuthology.orchestra.run.smithi161.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-11T21:46:00.129 INFO:teuthology.orchestra.run.smithi161.stdout: flex-2.6.4-9.el9.x86_64 2024-09-11T21:46:00.129 INFO:teuthology.orchestra.run.smithi161.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-11T21:46:00.129 INFO:teuthology.orchestra.run.smithi161.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-11T21:46:00.129 INFO:teuthology.orchestra.run.smithi161.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-11T21:46:00.129 INFO:teuthology.orchestra.run.smithi161.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-11T21:46:00.130 INFO:teuthology.orchestra.run.smithi161.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-11T21:46:00.130 INFO:teuthology.orchestra.run.smithi161.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-11T21:46:00.130 INFO:teuthology.orchestra.run.smithi161.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-11T21:46:00.130 INFO:teuthology.orchestra.run.smithi161.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-11T21:46:00.130 INFO:teuthology.orchestra.run.smithi161.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-11T21:46:00.130 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:46:00.130 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:46:00.130 INFO:teuthology.orchestra.run.smithi161.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:46:00.130 INFO:teuthology.orchestra.run.smithi161.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-11T21:46:00.130 INFO:teuthology.orchestra.run.smithi161.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-11T21:46:00.130 INFO:teuthology.orchestra.run.smithi161.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-11T21:46:00.130 INFO:teuthology.orchestra.run.smithi161.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-11T21:46:00.131 INFO:teuthology.orchestra.run.smithi161.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-11T21:46:00.131 INFO:teuthology.orchestra.run.smithi161.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:46:00.131 INFO:teuthology.orchestra.run.smithi161.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:46:00.131 INFO:teuthology.orchestra.run.smithi161.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-11T21:46:00.131 INFO:teuthology.orchestra.run.smithi161.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:46:00.131 INFO:teuthology.orchestra.run.smithi161.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-11T21:46:00.131 INFO:teuthology.orchestra.run.smithi161.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-11T21:46:00.131 INFO:teuthology.orchestra.run.smithi161.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-11T21:46:00.131 INFO:teuthology.orchestra.run.smithi161.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-11T21:46:00.131 INFO:teuthology.orchestra.run.smithi161.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-11T21:46:00.131 INFO:teuthology.orchestra.run.smithi161.stdout: lua-5.4.4-4.el9.x86_64 2024-09-11T21:46:00.131 INFO:teuthology.orchestra.run.smithi161.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-11T21:46:00.131 INFO:teuthology.orchestra.run.smithi161.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-11T21:46:00.131 INFO:teuthology.orchestra.run.smithi161.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-11T21:46:00.132 INFO:teuthology.orchestra.run.smithi161.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-11T21:46:00.132 INFO:teuthology.orchestra.run.smithi161.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-11T21:46:00.132 INFO:teuthology.orchestra.run.smithi161.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-11T21:46:00.132 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-13.16-1.el9.x86_64 2024-09-11T21:46:00.132 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-11T21:46:00.132 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-11T21:46:00.132 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-11T21:46:00.132 INFO:teuthology.orchestra.run.smithi161.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-11T21:46:00.132 INFO:teuthology.orchestra.run.smithi161.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-11T21:46:00.132 INFO:teuthology.orchestra.run.smithi161.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-11T21:46:00.132 INFO:teuthology.orchestra.run.smithi161.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-11T21:46:00.133 INFO:teuthology.orchestra.run.smithi161.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-11T21:46:00.133 INFO:teuthology.orchestra.run.smithi161.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-11T21:46:00.133 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-11T21:46:00.133 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:46:00.133 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:46:00.133 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:46:00.133 INFO:teuthology.orchestra.run.smithi161.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-11T21:46:00.133 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-11T21:46:00.133 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-11T21:46:00.134 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-11T21:46:00.134 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-11T21:46:00.134 INFO:teuthology.orchestra.run.smithi161.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-11T21:46:00.134 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-11T21:46:00.134 INFO:teuthology.orchestra.run.smithi161.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-11T21:46:00.134 INFO:teuthology.orchestra.run.smithi161.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-11T21:46:00.134 INFO:teuthology.orchestra.run.smithi161.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-11T21:46:00.134 INFO:teuthology.orchestra.run.smithi161.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-11T21:46:00.134 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-11T21:46:00.134 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-11T21:46:00.134 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-11T21:46:00.134 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-11T21:46:00.135 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-11T21:46:00.135 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-11T21:46:00.135 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-11T21:46:00.135 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-11T21:46:00.135 INFO:teuthology.orchestra.run.smithi161.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-11T21:46:00.135 INFO:teuthology.orchestra.run.smithi161.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-11T21:46:00.135 INFO:teuthology.orchestra.run.smithi161.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-11T21:46:00.135 INFO:teuthology.orchestra.run.smithi161.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-11T21:46:00.135 INFO:teuthology.orchestra.run.smithi161.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-11T21:46:00.135 INFO:teuthology.orchestra.run.smithi161.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-11T21:46:00.135 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-11T21:46:00.135 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-11T21:46:00.136 INFO:teuthology.orchestra.run.smithi161.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-11T21:46:00.136 INFO:teuthology.orchestra.run.smithi161.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-11T21:46:00.136 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-11T21:46:00.136 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ply-3.11-14.el9.noarch 2024-09-11T21:46:00.136 INFO:teuthology.orchestra.run.smithi161.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-11T21:46:00.136 INFO:teuthology.orchestra.run.smithi161.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-11T21:46:00.136 INFO:teuthology.orchestra.run.smithi161.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-11T21:46:00.136 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-11T21:46:00.136 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-11T21:46:00.136 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-11T21:46:00.136 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-11T21:46:00.136 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-11T21:46:00.137 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-11T21:46:00.137 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:46:00.137 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:46:00.137 INFO:teuthology.orchestra.run.smithi161.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-11T21:46:00.137 INFO:teuthology.orchestra.run.smithi161.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-11T21:46:00.137 INFO:teuthology.orchestra.run.smithi161.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-11T21:46:00.137 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:46:00.137 INFO:teuthology.orchestra.run.smithi161.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-11T21:46:00.137 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-11T21:46:00.137 INFO:teuthology.orchestra.run.smithi161.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-11T21:46:00.138 INFO:teuthology.orchestra.run.smithi161.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-11T21:46:00.138 INFO:teuthology.orchestra.run.smithi161.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-11T21:46:00.138 INFO:teuthology.orchestra.run.smithi161.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-11T21:46:00.138 INFO:teuthology.orchestra.run.smithi161.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-11T21:46:00.138 INFO:teuthology.orchestra.run.smithi161.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-11T21:46:00.138 INFO:teuthology.orchestra.run.smithi161.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-11T21:46:00.138 INFO:teuthology.orchestra.run.smithi161.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-11T21:46:00.138 INFO:teuthology.orchestra.run.smithi161.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-11T21:46:00.138 INFO:teuthology.orchestra.run.smithi161.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-11T21:46:00.138 INFO:teuthology.orchestra.run.smithi161.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-11T21:46:00.138 INFO:teuthology.orchestra.run.smithi161.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-11T21:46:00.138 INFO:teuthology.orchestra.run.smithi161.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-11T21:46:00.138 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:46:00.139 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:46:00.139 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:46:00.139 INFO:teuthology.orchestra.run.smithi161.stdout: re2-1:20211101-20.el9.x86_64 2024-09-11T21:46:00.139 INFO:teuthology.orchestra.run.smithi161.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-11T21:46:00.139 INFO:teuthology.orchestra.run.smithi161.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-11T21:46:00.139 INFO:teuthology.orchestra.run.smithi161.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-11T21:46:00.139 INFO:teuthology.orchestra.run.smithi161.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-11T21:46:00.139 INFO:teuthology.orchestra.run.smithi161.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-11T21:46:00.140 INFO:teuthology.orchestra.run.smithi161.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-11T21:46:00.140 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-11T21:46:00.140 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2024-09-11T21:46:00.555 DEBUG:teuthology.parallel:result is None 2024-09-11T21:46:01.891 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T21:46:01.891 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/164 2024-09-11T21:46:01.891 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/164 2024-09-11T21:46:01.891 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/164 2024-09-11T21:46:01.891 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/164 2024-09-11T21:46:01.891 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 5/164 2024-09-11T21:46:01.891 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/164 2024-09-11T21:46:01.891 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/164 2024-09-11T21:46:01.891 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T21:46:01.892 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T21:46:01.892 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-11T21:46:01.892 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/164 2024-09-11T21:46:01.892 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/164 2024-09-11T21:46:01.892 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T21:46:01.892 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 14/164 2024-09-11T21:46:01.892 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 15/164 2024-09-11T21:46:01.892 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 16/164 2024-09-11T21:46:01.892 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 17/164 2024-09-11T21:46:01.892 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 18/164 2024-09-11T21:46:01.892 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 19/164 2024-09-11T21:46:01.892 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 20/164 2024-09-11T21:46:01.893 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 21/164 2024-09-11T21:46:01.893 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 22/164 2024-09-11T21:46:01.893 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/164 2024-09-11T21:46:01.893 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 24/164 2024-09-11T21:46:01.893 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 25/164 2024-09-11T21:46:01.893 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/164 2024-09-11T21:46:01.893 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/164 2024-09-11T21:46:01.906 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 28/164 2024-09-11T21:46:01.906 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/164 2024-09-11T21:46:01.906 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 30/164 2024-09-11T21:46:01.906 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 31/164 2024-09-11T21:46:01.906 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 32/164 2024-09-11T21:46:01.906 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 33/164 2024-09-11T21:46:01.906 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 34/164 2024-09-11T21:46:01.906 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 35/164 2024-09-11T21:46:01.906 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 36/164 2024-09-11T21:46:01.906 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 37/164 2024-09-11T21:46:01.907 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 38/164 2024-09-11T21:46:01.907 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 39/164 2024-09-11T21:46:01.907 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-11T21:46:01.907 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-11T21:46:01.907 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-11T21:46:01.907 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-11T21:46:01.907 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-11T21:46:01.907 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-11T21:46:01.908 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-11T21:46:01.908 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-11T21:46:01.908 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-11T21:46:01.908 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-11T21:46:01.908 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-11T21:46:01.908 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-11T21:46:01.908 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-11T21:46:01.908 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-11T21:46:01.908 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-11T21:46:01.909 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-11T21:46:01.909 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-11T21:46:01.909 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-11T21:46:01.909 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-11T21:46:01.909 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-11T21:46:01.909 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-11T21:46:01.909 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-11T21:46:01.909 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-11T21:46:01.909 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-11T21:46:01.909 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-11T21:46:01.909 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-11T21:46:01.909 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-11T21:46:01.909 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-11T21:46:01.910 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-11T21:46:01.910 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-11T21:46:01.910 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-11T21:46:01.910 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-11T21:46:01.910 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-11T21:46:01.910 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-11T21:46:01.911 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-11T21:46:01.911 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-11T21:46:01.911 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-11T21:46:01.911 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-11T21:46:01.911 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-11T21:46:01.912 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-11T21:46:01.912 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-11T21:46:01.912 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-11T21:46:01.912 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-11T21:46:01.912 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-11T21:46:01.912 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-11T21:46:01.912 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-11T21:46:01.912 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-11T21:46:01.912 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-11T21:46:01.912 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-11T21:46:01.912 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-11T21:46:01.912 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-11T21:46:01.912 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-11T21:46:01.913 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-11T21:46:01.913 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-11T21:46:01.913 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-11T21:46:01.913 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-11T21:46:01.913 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-11T21:46:01.913 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-11T21:46:01.913 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-11T21:46:01.913 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-11T21:46:01.913 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-11T21:46:01.913 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-11T21:46:01.913 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-11T21:46:01.913 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-11T21:46:01.913 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-11T21:46:01.914 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-11T21:46:01.914 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-11T21:46:01.914 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-11T21:46:01.914 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-11T21:46:01.914 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-11T21:46:01.914 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-11T21:46:01.914 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-11T21:46:01.914 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-11T21:46:01.914 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-11T21:46:01.914 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-11T21:46:01.914 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-11T21:46:01.914 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-11T21:46:01.914 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-11T21:46:01.915 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-11T21:46:01.915 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-11T21:46:01.915 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-11T21:46:01.915 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-11T21:46:01.915 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-11T21:46:01.915 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-11T21:46:01.915 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-11T21:46:01.915 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-11T21:46:01.915 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-11T21:46:01.915 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-11T21:46:01.915 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-11T21:46:01.915 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-11T21:46:01.915 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-11T21:46:01.916 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-11T21:46:01.916 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-11T21:46:01.916 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-11T21:46:01.916 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-11T21:46:01.916 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-11T21:46:01.916 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-11T21:46:01.916 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-11T21:46:01.916 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-11T21:46:01.916 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-11T21:46:01.916 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-11T21:46:01.916 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-11T21:46:01.916 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-11T21:46:01.916 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-11T21:46:01.917 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-11T21:46:01.917 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-11T21:46:01.917 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-11T21:46:01.917 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-11T21:46:01.917 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-11T21:46:01.917 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 149/164 2024-09-11T21:46:01.917 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-11T21:46:01.917 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 151/164 2024-09-11T21:46:01.918 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-11T21:46:01.918 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-11T21:46:01.918 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-11T21:46:01.918 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-11T21:46:01.918 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-11T21:46:01.918 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-11T21:46:01.918 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-11T21:46:01.918 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-11T21:46:01.919 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-11T21:46:01.950 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-11T21:46:01.950 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-11T21:46:01.951 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-11T21:46:04.729 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T21:46:04.729 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:46:04.729 INFO:teuthology.orchestra.run.smithi001.stdout:Upgraded: 2024-09-11T21:46:04.729 INFO:teuthology.orchestra.run.smithi001.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-11T21:46:04.729 INFO:teuthology.orchestra.run.smithi001.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-11T21:46:04.729 INFO:teuthology.orchestra.run.smithi001.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-11T21:46:04.730 INFO:teuthology.orchestra.run.smithi001.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-11T21:46:04.730 INFO:teuthology.orchestra.run.smithi001.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:46:04.730 INFO:teuthology.orchestra.run.smithi001.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:46:04.730 INFO:teuthology.orchestra.run.smithi001.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-11T21:46:04.730 INFO:teuthology.orchestra.run.smithi001.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-11T21:46:04.730 INFO:teuthology.orchestra.run.smithi001.stdout:Installed: 2024-09-11T21:46:04.730 INFO:teuthology.orchestra.run.smithi001.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-11T21:46:04.730 INFO:teuthology.orchestra.run.smithi001.stdout: automake-1.16.2-8.el9.noarch 2024-09-11T21:46:04.730 INFO:teuthology.orchestra.run.smithi001.stdout: bison-3.7.4-5.el9.x86_64 2024-09-11T21:46:04.730 INFO:teuthology.orchestra.run.smithi001.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-11T21:46:04.730 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:46:04.730 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:46:04.731 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:46:04.731 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:46:04.731 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T21:46:04.731 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:46:04.731 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:46:04.731 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:46:04.731 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T21:46:04.731 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T21:46:04.731 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T21:46:04.731 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T21:46:04.731 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T21:46:04.731 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:46:04.732 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:46:04.732 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T21:46:04.732 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:46:04.732 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:46:04.732 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:46:04.732 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T21:46:04.732 INFO:teuthology.orchestra.run.smithi001.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T21:46:04.732 INFO:teuthology.orchestra.run.smithi001.stdout: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:46:04.732 INFO:teuthology.orchestra.run.smithi001.stdout: cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:46:04.732 INFO:teuthology.orchestra.run.smithi001.stdout: cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T21:46:04.732 INFO:teuthology.orchestra.run.smithi001.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-11T21:46:04.733 INFO:teuthology.orchestra.run.smithi001.stdout: flex-2.6.4-9.el9.x86_64 2024-09-11T21:46:04.733 INFO:teuthology.orchestra.run.smithi001.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-11T21:46:04.733 INFO:teuthology.orchestra.run.smithi001.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-11T21:46:04.733 INFO:teuthology.orchestra.run.smithi001.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-11T21:46:04.733 INFO:teuthology.orchestra.run.smithi001.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-11T21:46:04.733 INFO:teuthology.orchestra.run.smithi001.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-11T21:46:04.733 INFO:teuthology.orchestra.run.smithi001.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-11T21:46:04.733 INFO:teuthology.orchestra.run.smithi001.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-11T21:46:04.733 INFO:teuthology.orchestra.run.smithi001.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-11T21:46:04.733 INFO:teuthology.orchestra.run.smithi001.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-11T21:46:04.733 INFO:teuthology.orchestra.run.smithi001.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:46:04.733 INFO:teuthology.orchestra.run.smithi001.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:46:04.733 INFO:teuthology.orchestra.run.smithi001.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:46:04.734 INFO:teuthology.orchestra.run.smithi001.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-11T21:46:04.734 INFO:teuthology.orchestra.run.smithi001.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-11T21:46:04.734 INFO:teuthology.orchestra.run.smithi001.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-11T21:46:04.734 INFO:teuthology.orchestra.run.smithi001.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-11T21:46:04.734 INFO:teuthology.orchestra.run.smithi001.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-11T21:46:04.734 INFO:teuthology.orchestra.run.smithi001.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:46:04.734 INFO:teuthology.orchestra.run.smithi001.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:46:04.734 INFO:teuthology.orchestra.run.smithi001.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-11T21:46:04.734 INFO:teuthology.orchestra.run.smithi001.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:46:04.734 INFO:teuthology.orchestra.run.smithi001.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-11T21:46:04.734 INFO:teuthology.orchestra.run.smithi001.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-11T21:46:04.735 INFO:teuthology.orchestra.run.smithi001.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-11T21:46:04.735 INFO:teuthology.orchestra.run.smithi001.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-11T21:46:04.735 INFO:teuthology.orchestra.run.smithi001.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-11T21:46:04.735 INFO:teuthology.orchestra.run.smithi001.stdout: lua-5.4.4-4.el9.x86_64 2024-09-11T21:46:04.735 INFO:teuthology.orchestra.run.smithi001.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-11T21:46:04.735 INFO:teuthology.orchestra.run.smithi001.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-11T21:46:04.735 INFO:teuthology.orchestra.run.smithi001.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-11T21:46:04.735 INFO:teuthology.orchestra.run.smithi001.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-11T21:46:04.735 INFO:teuthology.orchestra.run.smithi001.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-11T21:46:04.735 INFO:teuthology.orchestra.run.smithi001.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-11T21:46:04.735 INFO:teuthology.orchestra.run.smithi001.stdout: postgresql-13.16-1.el9.x86_64 2024-09-11T21:46:04.735 INFO:teuthology.orchestra.run.smithi001.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-11T21:46:04.736 INFO:teuthology.orchestra.run.smithi001.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-11T21:46:04.736 INFO:teuthology.orchestra.run.smithi001.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-11T21:46:04.736 INFO:teuthology.orchestra.run.smithi001.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-11T21:46:04.736 INFO:teuthology.orchestra.run.smithi001.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-11T21:46:04.736 INFO:teuthology.orchestra.run.smithi001.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-11T21:46:04.736 INFO:teuthology.orchestra.run.smithi001.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-11T21:46:04.736 INFO:teuthology.orchestra.run.smithi001.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-11T21:46:04.736 INFO:teuthology.orchestra.run.smithi001.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-11T21:46:04.736 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-11T21:46:04.736 INFO:teuthology.orchestra.run.smithi001.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:46:04.736 INFO:teuthology.orchestra.run.smithi001.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:46:04.736 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:46:04.737 INFO:teuthology.orchestra.run.smithi001.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-11T21:46:04.737 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-11T21:46:04.737 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-11T21:46:04.737 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-11T21:46:04.737 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-11T21:46:04.737 INFO:teuthology.orchestra.run.smithi001.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-11T21:46:04.737 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-11T21:46:04.737 INFO:teuthology.orchestra.run.smithi001.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-11T21:46:04.737 INFO:teuthology.orchestra.run.smithi001.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-11T21:46:04.737 INFO:teuthology.orchestra.run.smithi001.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-11T21:46:04.737 INFO:teuthology.orchestra.run.smithi001.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-11T21:46:04.737 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-11T21:46:04.737 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-11T21:46:04.738 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-11T21:46:04.738 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-11T21:46:04.738 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-11T21:46:04.738 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-11T21:46:04.738 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-11T21:46:04.738 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-11T21:46:04.738 INFO:teuthology.orchestra.run.smithi001.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-11T21:46:04.738 INFO:teuthology.orchestra.run.smithi001.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-11T21:46:04.738 INFO:teuthology.orchestra.run.smithi001.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-11T21:46:04.738 INFO:teuthology.orchestra.run.smithi001.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-11T21:46:04.738 INFO:teuthology.orchestra.run.smithi001.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-11T21:46:04.738 INFO:teuthology.orchestra.run.smithi001.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-11T21:46:04.739 INFO:teuthology.orchestra.run.smithi001.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-11T21:46:04.739 INFO:teuthology.orchestra.run.smithi001.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-11T21:46:04.739 INFO:teuthology.orchestra.run.smithi001.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-11T21:46:04.739 INFO:teuthology.orchestra.run.smithi001.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-11T21:46:04.739 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-11T21:46:04.739 INFO:teuthology.orchestra.run.smithi001.stdout: python3-ply-3.11-14.el9.noarch 2024-09-11T21:46:04.739 INFO:teuthology.orchestra.run.smithi001.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-11T21:46:04.739 INFO:teuthology.orchestra.run.smithi001.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-11T21:46:04.739 INFO:teuthology.orchestra.run.smithi001.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-11T21:46:04.739 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-11T21:46:04.739 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-11T21:46:04.739 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-11T21:46:04.739 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-11T21:46:04.740 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-11T21:46:04.740 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-11T21:46:04.740 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:46:04.740 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:46:04.740 INFO:teuthology.orchestra.run.smithi001.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-11T21:46:04.740 INFO:teuthology.orchestra.run.smithi001.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-11T21:46:04.740 INFO:teuthology.orchestra.run.smithi001.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-11T21:46:04.740 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:46:04.740 INFO:teuthology.orchestra.run.smithi001.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-11T21:46:04.740 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-11T21:46:04.740 INFO:teuthology.orchestra.run.smithi001.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-11T21:46:04.740 INFO:teuthology.orchestra.run.smithi001.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-11T21:46:04.741 INFO:teuthology.orchestra.run.smithi001.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-11T21:46:04.741 INFO:teuthology.orchestra.run.smithi001.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-11T21:46:04.741 INFO:teuthology.orchestra.run.smithi001.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-11T21:46:04.741 INFO:teuthology.orchestra.run.smithi001.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-11T21:46:04.741 INFO:teuthology.orchestra.run.smithi001.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-11T21:46:04.741 INFO:teuthology.orchestra.run.smithi001.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-11T21:46:04.741 INFO:teuthology.orchestra.run.smithi001.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-11T21:46:04.741 INFO:teuthology.orchestra.run.smithi001.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-11T21:46:04.741 INFO:teuthology.orchestra.run.smithi001.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-11T21:46:04.741 INFO:teuthology.orchestra.run.smithi001.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-11T21:46:04.741 INFO:teuthology.orchestra.run.smithi001.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-11T21:46:04.741 INFO:teuthology.orchestra.run.smithi001.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:46:04.741 INFO:teuthology.orchestra.run.smithi001.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:46:04.742 INFO:teuthology.orchestra.run.smithi001.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T21:46:04.742 INFO:teuthology.orchestra.run.smithi001.stdout: re2-1:20211101-20.el9.x86_64 2024-09-11T21:46:04.742 INFO:teuthology.orchestra.run.smithi001.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-11T21:46:04.742 INFO:teuthology.orchestra.run.smithi001.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-11T21:46:04.742 INFO:teuthology.orchestra.run.smithi001.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-11T21:46:04.742 INFO:teuthology.orchestra.run.smithi001.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-11T21:46:04.742 INFO:teuthology.orchestra.run.smithi001.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-11T21:46:04.742 INFO:teuthology.orchestra.run.smithi001.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-11T21:46:04.742 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:46:04.743 INFO:teuthology.orchestra.run.smithi001.stdout:Complete! 2024-09-11T21:46:05.220 DEBUG:teuthology.parallel:result is None 2024-09-11T21:46:05.221 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T21:46:05.363 DEBUG:teuthology.orchestra.run.smithi001:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-11T21:46:05.387 INFO:teuthology.orchestra.run.smithi001.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-11T21:46:05.388 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-11T21:46:05.388 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-11T21:46:05.390 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T21:46:05.538 DEBUG:teuthology.orchestra.run.smithi053:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-11T21:46:05.564 INFO:teuthology.orchestra.run.smithi053.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-11T21:46:05.564 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-11T21:46:05.564 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-11T21:46:05.566 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T21:46:05.706 DEBUG:teuthology.orchestra.run.smithi161:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-11T21:46:05.733 INFO:teuthology.orchestra.run.smithi161.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-11T21:46:05.733 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-11T21:46:05.733 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-11T21:46:05.735 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-11T21:46:05.735 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-11T21:46:05.735 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-11T21:46:05.768 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-11T21:46:05.768 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-11T21:46:05.801 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-09-11T21:46:05.801 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-11T21:46:05.832 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-11T21:46:05.833 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-11T21:46:05.833 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/usr/bin/daemon-helper 2024-09-11T21:46:05.862 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-11T21:46:05.931 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-11T21:46:05.931 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/usr/bin/daemon-helper 2024-09-11T21:46:05.961 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-11T21:46:06.030 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-09-11T21:46:06.030 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/usr/bin/daemon-helper 2024-09-11T21:46:06.060 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-11T21:46:06.129 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-11T21:46:06.129 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-11T21:46:06.129 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-11T21:46:06.158 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-11T21:46:06.227 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-11T21:46:06.227 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-11T21:46:06.256 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-11T21:46:06.324 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-09-11T21:46:06.324 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-11T21:46:06.355 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-11T21:46:06.423 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-11T21:46:06.423 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-11T21:46:06.423 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/usr/bin/stdin-killer 2024-09-11T21:46:06.457 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-11T21:46:06.527 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-11T21:46:06.528 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/usr/bin/stdin-killer 2024-09-11T21:46:06.557 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-11T21:46:06.627 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-09-11T21:46:06.627 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/usr/bin/stdin-killer 2024-09-11T21:46:06.655 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-11T21:46:06.723 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-11T21:46:06.818 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-09-11T21:46:06.818 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T21:46:06.819 INFO:tasks.cephadm:Cluster fsid is 3ffca83c-7087-11ef-bcea-c7b262605968 2024-09-11T21:46:06.819 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-11T21:46:06.819 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.1', 'mon.b': '172.21.15.53', 'mon.c': '172.21.15.161'} 2024-09-11T21:46:06.819 INFO:tasks.cephadm:First mon is mon.a on smithi001 2024-09-11T21:46:06.820 INFO:tasks.cephadm:First mgr is x 2024-09-11T21:46:06.820 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-11T21:46:06.820 DEBUG:teuthology.orchestra.run.smithi001:> sudo hostname $(hostname -s) 2024-09-11T21:46:06.851 DEBUG:teuthology.orchestra.run.smithi053:> sudo hostname $(hostname -s) 2024-09-11T21:46:06.881 DEBUG:teuthology.orchestra.run.smithi161:> sudo hostname $(hostname -s) 2024-09-11T21:46:06.910 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-11T21:46:06.910 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T21:46:07.053 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/56038/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-298-gb53b6d3e', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-298.gb53b6d3e'}, 'url': 'https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-09 17:24:58.599585', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-09-11T21:46:07.192 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref squid, sha1 b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T21:46:07.194 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm 2024-09-11T21:46:07.194 INFO:tasks.cephadm:Downloading cephadm from url: https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm 2024-09-11T21:46:07.194 DEBUG:teuthology.orchestra.run.smithi001:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-11T21:46:07.567 INFO:teuthology.orchestra.run.smithi001.stdout:-rw-r--r--. 1 ubuntu ubuntu 784498 Sep 11 21:46 /home/ubuntu/cephtest/cephadm 2024-09-11T21:46:07.567 DEBUG:teuthology.orchestra.run.smithi053:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-11T21:46:07.937 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-r--r--. 1 ubuntu ubuntu 784498 Sep 11 21:46 /home/ubuntu/cephtest/cephadm 2024-09-11T21:46:07.938 DEBUG:teuthology.orchestra.run.smithi161:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-11T21:46:08.307 INFO:teuthology.orchestra.run.smithi161.stdout:-rw-r--r--. 1 ubuntu ubuntu 784498 Sep 11 21:46 /home/ubuntu/cephtest/cephadm 2024-09-11T21:46:08.307 DEBUG:teuthology.orchestra.run.smithi001:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-11T21:46:08.328 DEBUG:teuthology.orchestra.run.smithi053:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-11T21:46:08.347 DEBUG:teuthology.orchestra.run.smithi161:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-11T21:46:08.382 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc on all hosts... 2024-09-11T21:46:08.382 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc pull 2024-09-11T21:46:08.385 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc pull 2024-09-11T21:46:08.389 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc pull 2024-09-11T21:46:10.034 INFO:teuthology.orchestra.run.smithi053.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-11T21:46:10.039 INFO:teuthology.orchestra.run.smithi001.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-11T21:46:10.039 INFO:teuthology.orchestra.run.smithi161.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-11T21:46:54.212 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-11T21:46:54.212 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-11T21:46:54.212 INFO:teuthology.orchestra.run.smithi053.stdout: "image_id": "54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594", 2024-09-11T21:46:54.212 INFO:teuthology.orchestra.run.smithi053.stdout: "repo_digests": [ 2024-09-11T21:46:54.212 INFO:teuthology.orchestra.run.smithi053.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66" 2024-09-11T21:46:54.212 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-11T21:46:54.212 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-11T21:47:05.740 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-11T21:47:05.741 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-11T21:47:05.741 INFO:teuthology.orchestra.run.smithi161.stdout: "image_id": "54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594", 2024-09-11T21:47:05.741 INFO:teuthology.orchestra.run.smithi161.stdout: "repo_digests": [ 2024-09-11T21:47:05.741 INFO:teuthology.orchestra.run.smithi161.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66" 2024-09-11T21:47:05.741 INFO:teuthology.orchestra.run.smithi161.stdout: ] 2024-09-11T21:47:05.741 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-11T21:47:19.825 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-11T21:47:19.825 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-11T21:47:19.825 INFO:teuthology.orchestra.run.smithi001.stdout: "image_id": "54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594", 2024-09-11T21:47:19.825 INFO:teuthology.orchestra.run.smithi001.stdout: "repo_digests": [ 2024-09-11T21:47:19.825 INFO:teuthology.orchestra.run.smithi001.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66" 2024-09-11T21:47:19.825 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-11T21:47:19.825 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-11T21:47:19.851 DEBUG:teuthology.orchestra.run.smithi001:> sudo mkdir -p /etc/ceph 2024-09-11T21:47:19.895 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /etc/ceph 2024-09-11T21:47:19.929 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /etc/ceph 2024-09-11T21:47:19.963 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 777 /etc/ceph 2024-09-11T21:47:19.989 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 777 /etc/ceph 2024-09-11T21:47:20.017 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 777 /etc/ceph 2024-09-11T21:47:20.046 INFO:tasks.cephadm:Writing seed config... 2024-09-11T21:47:20.047 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-11T21:47:20.048 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-11T21:47:20.048 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-11T21:47:20.048 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-11T21:47:20.048 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-11T21:47:20.048 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-11T21:47:20.048 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-11T21:47:20.048 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-11T21:47:20.048 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-11T21:47:20.048 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-09-11T21:47:20.048 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-09-11T21:47:20.048 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-09-11T21:47:20.049 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-09-11T21:47:20.049 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-11T21:47:20.049 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-11T21:47:20.049 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-11T21:47:20.049 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-11T21:47:20.049 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-09-11T21:47:20.049 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-11T21:47:20.049 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-11T21:47:20.049 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-11T21:47:20.049 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-11T21:47:20.049 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-09-11T21:47:20.049 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-11T21:47:20.049 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-11T21:47:20.050 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-11T21:47:20.050 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-11T21:47:20.050 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-11T21:47:20.050 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-11T21:47:20.050 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-11T21:47:20.050 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-11T21:47:20.050 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-11T21:47:20.050 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-11T21:47:20.050 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-09-11T21:47:20.050 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-11T21:47:20.050 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-11T21:47:20.050 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-11T21:47:20.050 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-11T21:47:20.051 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-11T21:47:20.051 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-11T21:47:20.051 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-11T21:47:20.051 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-11T21:47:20.051 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-11T21:47:20.051 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-11T21:47:20.051 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-11T21:47:20.051 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-11T21:47:20.051 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-11T21:47:20.051 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-09-11T21:47:20.052 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-11T21:47:20.053 DEBUG:teuthology.orchestra.run.smithi001:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-11T21:47:20.068 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 3ffca83c-7087-11ef-bcea-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-11T21:47:20.069 DEBUG:teuthology.orchestra.run.smithi001:mon.a> sudo journalctl -f -n 0 -u ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mon.a.service 2024-09-11T21:47:20.111 DEBUG:teuthology.orchestra.run.smithi001:mgr.x> sudo journalctl -f -n 0 -u ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mgr.x.service 2024-09-11T21:47:20.154 INFO:tasks.cephadm:Bootstrapping... 2024-09-11T21:47:20.154 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc -v bootstrap --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.1 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-11T21:47:20.363 INFO:teuthology.orchestra.run.smithi001.stdout:-------------------------------------------------------------------------------- 2024-09-11T21:47:20.363 INFO:teuthology.orchestra.run.smithi001.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', '-v', 'bootstrap', '--fsid', '3ffca83c-7087-11ef-bcea-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.1', '--skip-admin-label'] 2024-09-11T21:47:20.363 INFO:teuthology.orchestra.run.smithi001.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-11T21:47:20.364 INFO:teuthology.orchestra.run.smithi001.stdout:Verifying podman|docker is present... 2024-09-11T21:47:20.397 INFO:teuthology.orchestra.run.smithi001.stdout:/bin/podman: stdout 5.2.2 2024-09-11T21:47:20.397 INFO:teuthology.orchestra.run.smithi001.stdout:Verifying lvm2 is present... 2024-09-11T21:47:20.397 INFO:teuthology.orchestra.run.smithi001.stdout:Verifying time synchronization is in place... 2024-09-11T21:47:20.405 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-11T21:47:20.405 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-11T21:47:20.412 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-11T21:47:20.412 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout inactive 2024-09-11T21:47:20.419 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout enabled 2024-09-11T21:47:20.425 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout active 2024-09-11T21:47:20.425 INFO:teuthology.orchestra.run.smithi001.stdout:Unit chronyd.service is enabled and running 2024-09-11T21:47:20.425 INFO:teuthology.orchestra.run.smithi001.stdout:Repeating the final host check... 2024-09-11T21:47:20.448 INFO:teuthology.orchestra.run.smithi001.stdout:/bin/podman: stdout 5.2.2 2024-09-11T21:47:20.448 INFO:teuthology.orchestra.run.smithi001.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-11T21:47:20.448 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl is present 2024-09-11T21:47:20.448 INFO:teuthology.orchestra.run.smithi001.stdout:lvcreate is present 2024-09-11T21:47:20.455 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-11T21:47:20.455 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-11T21:47:20.461 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-11T21:47:20.461 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout inactive 2024-09-11T21:47:20.468 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout enabled 2024-09-11T21:47:20.475 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout active 2024-09-11T21:47:20.475 INFO:teuthology.orchestra.run.smithi001.stdout:Unit chronyd.service is enabled and running 2024-09-11T21:47:20.475 INFO:teuthology.orchestra.run.smithi001.stdout:Host looks OK 2024-09-11T21:47:20.476 INFO:teuthology.orchestra.run.smithi001.stdout:Cluster fsid: 3ffca83c-7087-11ef-bcea-c7b262605968 2024-09-11T21:47:20.476 INFO:teuthology.orchestra.run.smithi001.stdout:Acquiring lock 140326779704320 on /run/cephadm/3ffca83c-7087-11ef-bcea-c7b262605968.lock 2024-09-11T21:47:20.476 INFO:teuthology.orchestra.run.smithi001.stdout:Lock 140326779704320 acquired on /run/cephadm/3ffca83c-7087-11ef-bcea-c7b262605968.lock 2024-09-11T21:47:20.476 INFO:teuthology.orchestra.run.smithi001.stdout:Verifying IP 172.21.15.1 port 3300 ... 2024-09-11T21:47:20.476 INFO:teuthology.orchestra.run.smithi001.stdout:Verifying IP 172.21.15.1 port 6789 ... 2024-09-11T21:47:20.476 INFO:teuthology.orchestra.run.smithi001.stdout:Base mon IP(s) is [172.21.15.1:3300, 172.21.15.1:6789], mon addrv is [v2:172.21.15.1:3300,v1:172.21.15.1:6789] 2024-09-11T21:47:20.479 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.1 metric 100 2024-09-11T21:47:20.479 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.1 metric 100 2024-09-11T21:47:20.482 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-11T21:47:20.482 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-09-11T21:47:20.482 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1731sec hoplimit 64 pref medium 2024-09-11T21:47:20.485 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-11T21:47:20.485 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-11T21:47:20.485 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-11T21:47:20.485 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-09-11T21:47:20.485 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:15e8/64 scope link 2024-09-11T21:47:20.485 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-11T21:47:20.486 INFO:teuthology.orchestra.run.smithi001.stdout:Mon IP `172.21.15.1` is in CIDR network `172.21.0.0/20` 2024-09-11T21:47:20.486 INFO:teuthology.orchestra.run.smithi001.stdout:Mon IP `172.21.15.1` is in CIDR network `172.21.0.0/20` 2024-09-11T21:47:20.486 INFO:teuthology.orchestra.run.smithi001.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-11T21:47:20.486 INFO:teuthology.orchestra.run.smithi001.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-11T21:47:20.487 INFO:teuthology.orchestra.run.smithi001.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-11T21:47:21.027 INFO:teuthology.orchestra.run.smithi001.stdout:/bin/podman: stdout 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 2024-09-11T21:47:21.027 INFO:teuthology.orchestra.run.smithi001.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-11T21:47:21.028 INFO:teuthology.orchestra.run.smithi001.stdout:/bin/podman: stderr Getting image source signatures 2024-09-11T21:47:21.028 INFO:teuthology.orchestra.run.smithi001.stdout:/bin/podman: stderr Copying blob sha256:60180723200353b4e13b39fd162aecaf1bc097c859c7c6537324229e97c78567 2024-09-11T21:47:21.028 INFO:teuthology.orchestra.run.smithi001.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-11T21:47:21.028 INFO:teuthology.orchestra.run.smithi001.stdout:/bin/podman: stderr Copying config sha256:54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 2024-09-11T21:47:21.028 INFO:teuthology.orchestra.run.smithi001.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-11T21:47:21.924 INFO:teuthology.orchestra.run.smithi001.stdout:ceph: stdout ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable) 2024-09-11T21:47:21.924 INFO:teuthology.orchestra.run.smithi001.stdout:Ceph version: ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable) 2024-09-11T21:47:21.924 INFO:teuthology.orchestra.run.smithi001.stdout:Extracting ceph user uid/gid from container image... 2024-09-11T21:47:22.880 INFO:teuthology.orchestra.run.smithi001.stdout:stat: stdout 167 167 2024-09-11T21:47:22.880 INFO:teuthology.orchestra.run.smithi001.stdout:Creating initial keys... 2024-09-11T21:47:23.778 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph-authtool: stdout AQDrD+Jmen5BCxAAcZLXftIyzxKZhn6Uvq6CRA== 2024-09-11T21:47:24.658 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph-authtool: stdout AQDsD+JmuYUnBxAA5Gb/08zVTo4FzG+yK7OXsg== 2024-09-11T21:47:25.588 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph-authtool: stdout AQDsD+JmhKipORAALx7k00BdiRg9tvdDnKshCA== 2024-09-11T21:47:25.589 INFO:teuthology.orchestra.run.smithi001.stdout:Creating initial monmap... 2024-09-11T21:47:26.453 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-11T21:47:26.454 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-11T21:47:26.454 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 3ffca83c-7087-11ef-bcea-c7b262605968 2024-09-11T21:47:26.454 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-11T21:47:26.454 INFO:teuthology.orchestra.run.smithi001.stdout:monmaptool for a [v2:172.21.15.1:3300,v1:172.21.15.1:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-11T21:47:26.454 INFO:teuthology.orchestra.run.smithi001.stdout:setting min_mon_release = quincy 2024-09-11T21:47:26.454 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/monmaptool: set fsid to 3ffca83c-7087-11ef-bcea-c7b262605968 2024-09-11T21:47:26.454 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-11T21:47:26.454 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:47:26.454 INFO:teuthology.orchestra.run.smithi001.stdout:Creating mon... 2024-09-11T21:47:27.385 INFO:teuthology.orchestra.run.smithi001.stdout:create mon.a on 2024-09-11T21:47:27.596 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-11T21:47:27.763 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-11T21:47:27.965 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-3ffca83c-7087-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-3ffca83c-7087-11ef-bcea-c7b262605968.target. 2024-09-11T21:47:27.965 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-3ffca83c-7087-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-3ffca83c-7087-11ef-bcea-c7b262605968.target. 2024-09-11T21:47:28.218 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mon.a 2024-09-11T21:47:28.219 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stderr Failed to reset failed state of unit ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mon.a.service: Unit ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mon.a.service not loaded. 2024-09-11T21:47:28.397 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-3ffca83c-7087-11ef-bcea-c7b262605968.target.wants/ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mon.a.service → /etc/systemd/system/ceph-3ffca83c-7087-11ef-bcea-c7b262605968@.service. 2024-09-11T21:47:28.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:28 smithi001 systemd[1]: Starting Ceph mon.a for 3ffca83c-7087-11ef-bcea-c7b262605968... 2024-09-11T21:47:28.722 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:28 smithi001 podman[30548]: 2024-09-11 21:47:28.625544718 +0000 UTC m=+0.101936590 container create 29ad5a4ee3579b336933d58b32f64a33fb6b4daef24e9f5123e8090ad353f459 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mon-a, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-11T21:47:28.722 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:28 smithi001 podman[30548]: 2024-09-11 21:47:28.544049913 +0000 UTC m=+0.020441795 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T21:47:28.925 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T21:47:28.925 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout disabled 2024-09-11T21:47:28.932 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T21:47:28.932 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout inactive 2024-09-11T21:47:28.932 INFO:teuthology.orchestra.run.smithi001.stdout:firewalld.service is not enabled 2024-09-11T21:47:28.932 INFO:teuthology.orchestra.run.smithi001.stdout:Not possible to enable service . firewalld.service is not available 2024-09-11T21:47:28.933 INFO:teuthology.orchestra.run.smithi001.stdout:Waiting for mon to start... 2024-09-11T21:47:28.933 INFO:teuthology.orchestra.run.smithi001.stdout:Waiting for mon... 2024-09-11T21:47:29.017 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:28 smithi001 podman[30548]: 2024-09-11 21:47:28.827750739 +0000 UTC m=+0.304142620 container init 29ad5a4ee3579b336933d58b32f64a33fb6b4daef24e9f5123e8090ad353f459 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-11T21:47:29.017 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:28 smithi001 podman[30548]: 2024-09-11 21:47:28.831102579 +0000 UTC m=+0.307494453 container start 29ad5a4ee3579b336933d58b32f64a33fb6b4daef24e9f5123e8090ad353f459 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mon-a, ceph=True, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T21:47:29.017 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:28 smithi001 ceph-mon[30567]: mkfs 3ffca83c-7087-11ef-bcea-c7b262605968 2024-09-11T21:47:29.017 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:28 smithi001 ceph-mon[30567]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-11T21:47:29.017 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:28 smithi001 bash[30548]: 29ad5a4ee3579b336933d58b32f64a33fb6b4daef24e9f5123e8090ad353f459 2024-09-11T21:47:29.018 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:28 smithi001 systemd[1]: Started Ceph mon.a for 3ffca83c-7087-11ef-bcea-c7b262605968. 2024-09-11T21:47:30.162 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:29 smithi001 ceph-mon[30567]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-11T21:47:30.162 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:29 smithi001 ceph-mon[30567]: monmap epoch 1 2024-09-11T21:47:30.162 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:29 smithi001 ceph-mon[30567]: fsid 3ffca83c-7087-11ef-bcea-c7b262605968 2024-09-11T21:47:30.162 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:29 smithi001 ceph-mon[30567]: last_changed 2024-09-11T21:47:25.953679+0000 2024-09-11T21:47:30.162 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:29 smithi001 ceph-mon[30567]: created 2024-09-11T21:47:25.953679+0000 2024-09-11T21:47:30.162 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:29 smithi001 ceph-mon[30567]: min_mon_release 19 (squid) 2024-09-11T21:47:30.162 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:29 smithi001 ceph-mon[30567]: election_strategy: 1 2024-09-11T21:47:30.162 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:29 smithi001 ceph-mon[30567]: 0: [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] mon.a 2024-09-11T21:47:30.162 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:29 smithi001 ceph-mon[30567]: fsmap 2024-09-11T21:47:30.162 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:29 smithi001 ceph-mon[30567]: osdmap e1: 0 total, 0 up, 0 in 2024-09-11T21:47:30.163 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:29 smithi001 ceph-mon[30567]: mgrmap e1: no daemons active 2024-09-11T21:47:31.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:30 smithi001 ceph-mon[30567]: from='client.? 172.21.15.1:0/1748137915' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-11T21:47:36.722 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout cluster: 2024-09-11T21:47:36.722 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout id: 3ffca83c-7087-11ef-bcea-c7b262605968 2024-09-11T21:47:36.722 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-11T21:47:36.722 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-09-11T21:47:36.722 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout services: 2024-09-11T21:47:36.723 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 1.45794s) 2024-09-11T21:47:36.723 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-11T21:47:36.723 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-11T21:47:36.723 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-09-11T21:47:36.723 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout data: 2024-09-11T21:47:36.723 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-11T21:47:36.723 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-11T21:47:36.723 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-11T21:47:36.723 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout pgs: 2024-09-11T21:47:36.723 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-09-11T21:47:36.723 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:30.251+0000 7f1749f20640 1 Processor -- start 2024-09-11T21:47:36.723 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:30.251+0000 7f1749f20640 1 -- start start 2024-09-11T21:47:36.724 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:30.252+0000 7f1749f20640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1744108d30 0x7f1744109130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:47:36.724 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:30.252+0000 7f1749f20640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1744109700 con 0x7f1744108d30 2024-09-11T21:47:36.724 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:30.252+0000 7f17437fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1744108d30 0x7f1744109130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:47:36.724 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:30.252+0000 7f17437fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1744108d30 0x7f1744109130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43906/0 (socket says 172.21.15.1:43906) 2024-09-11T21:47:36.724 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:30.252+0000 7f17437fe640 1 -- 172.21.15.1:0/1841257382 learned_addr learned my addr 172.21.15.1:0/1841257382 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:47:36.724 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:30.252+0000 7f17437fe640 1 -- 172.21.15.1:0/1841257382 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1744109880 con 0x7f1744108d30 2024-09-11T21:47:36.724 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:30.254+0000 7f17437fe640 1 --2- 172.21.15.1:0/1841257382 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1744108d30 0x7f1744109130 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1734009820 tx=0x7f173402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dfebecb4747d7d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:47:36.724 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:30.255+0000 7f17427fc640 1 -- 172.21.15.1:0/1841257382 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1734037070 con 0x7f1744108d30 2024-09-11T21:47:36.724 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:30.255+0000 7f17427fc640 1 -- 172.21.15.1:0/1841257382 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1734032440 con 0x7f1744108d30 2024-09-11T21:47:36.724 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:30.255+0000 7f17427fc640 1 -- 172.21.15.1:0/1841257382 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1734032740 con 0x7f1744108d30 2024-09-11T21:47:36.724 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:30.255+0000 7f1749f20640 1 -- 172.21.15.1:0/1841257382 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1744108d30 msgr2=0x7f1744109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:47:36.725 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:30.255+0000 7f1749f20640 1 --2- 172.21.15.1:0/1841257382 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1744108d30 0x7f1744109130 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1734009820 tx=0x7f173402a270 comp rx=0 tx=0).stop 2024-09-11T21:47:36.725 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:30.255+0000 7f1749f20640 1 -- 172.21.15.1:0/1841257382 shutdown_connections 2024-09-11T21:47:36.725 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:30.255+0000 7f1749f20640 1 --2- 172.21.15.1:0/1841257382 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1744108d30 0x7f1744109130 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:47:36.725 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:30.255+0000 7f1749f20640 1 -- 172.21.15.1:0/1841257382 >> 172.21.15.1:0/1841257382 conn(0x7f1744101560 msgr2=0x7f1744103980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:47:36.725 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:30.255+0000 7f1749f20640 1 -- 172.21.15.1:0/1841257382 shutdown_connections 2024-09-11T21:47:36.725 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:30.255+0000 7f1749f20640 1 -- 172.21.15.1:0/1841257382 wait complete. 2024-09-11T21:47:36.725 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:30.256+0000 7f1749f20640 1 Processor -- start 2024-09-11T21:47:36.725 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:30.256+0000 7f1749f20640 1 -- start start 2024-09-11T21:47:36.725 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:30.256+0000 7f1749f20640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1744108d30 0x7f1744108350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:47:36.725 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:30.256+0000 7f1749f20640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1744105ac0 con 0x7f1744108d30 2024-09-11T21:47:36.725 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:30.257+0000 7f17437fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1744108d30 0x7f1744108350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:47:36.726 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:30.257+0000 7f17437fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1744108d30 0x7f1744108350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43918/0 (socket says 172.21.15.1:43918) 2024-09-11T21:47:36.726 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:30.257+0000 7f17437fe640 1 -- 172.21.15.1:0/1748137915 learned_addr learned my addr 172.21.15.1:0/1748137915 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:47:36.726 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:30.257+0000 7f17437fe640 1 -- 172.21.15.1:0/1748137915 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1744108890 con 0x7f1744108d30 2024-09-11T21:47:36.726 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:30.257+0000 7f17437fe640 1 --2- 172.21.15.1:0/1748137915 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1744108d30 0x7f1744108350 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f17340090b0 tx=0x7f17340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:47:36.726 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:30.257+0000 7f1740ff9640 1 -- 172.21.15.1:0/1748137915 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1734041070 con 0x7f1744108d30 2024-09-11T21:47:36.726 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:30.257+0000 7f1740ff9640 1 -- 172.21.15.1:0/1748137915 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f173403c7a0 con 0x7f1744108d30 2024-09-11T21:47:36.726 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:30.257+0000 7f1749f20640 1 -- 172.21.15.1:0/1748137915 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1744106a80 con 0x7f1744108d30 2024-09-11T21:47:36.726 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:30.257+0000 7f1740ff9640 1 -- 172.21.15.1:0/1748137915 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1734037040 con 0x7f1744108d30 2024-09-11T21:47:36.726 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:30.257+0000 7f1749f20640 1 -- 172.21.15.1:0/1748137915 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1744106fc0 con 0x7f1744108d30 2024-09-11T21:47:36.726 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:30.258+0000 7f1740ff9640 1 -- 172.21.15.1:0/1748137915 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1734044070 con 0x7f1744108d30 2024-09-11T21:47:36.726 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:30.258+0000 7f1740ff9640 1 -- 172.21.15.1:0/1748137915 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f173403d3f0 con 0x7f1744108d30 2024-09-11T21:47:36.726 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:30.258+0000 7f1749f20640 1 -- 172.21.15.1:0/1748137915 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17441091b0 con 0x7f1744108d30 2024-09-11T21:47:36.727 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:30.261+0000 7f1740ff9640 1 -- 172.21.15.1:0/1748137915 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f1734046450 con 0x7f1744108d30 2024-09-11T21:47:36.727 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:30.324+0000 7f1749f20640 1 -- 172.21.15.1:0/1748137915 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f1744193e10 con 0x7f1744108d30 2024-09-11T21:47:36.727 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:30.324+0000 7f1740ff9640 1 -- 172.21.15.1:0/1748137915 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+316 (secure 0 0 0) 0x7f1734046630 con 0x7f1744108d30 2024-09-11T21:47:36.727 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:30.324+0000 7f1749f20640 1 -- 172.21.15.1:0/1748137915 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1744108d30 msgr2=0x7f1744108350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:47:36.727 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:30.324+0000 7f1749f20640 1 --2- 172.21.15.1:0/1748137915 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1744108d30 0x7f1744108350 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f17340090b0 tx=0x7f17340046c0 comp rx=0 tx=0).stop 2024-09-11T21:47:36.727 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:30.325+0000 7f1749f20640 1 -- 172.21.15.1:0/1748137915 shutdown_connections 2024-09-11T21:47:36.727 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:30.325+0000 7f1749f20640 1 --2- 172.21.15.1:0/1748137915 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1744108d30 0x7f1744108350 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:47:36.727 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:30.325+0000 7f1749f20640 1 -- 172.21.15.1:0/1748137915 >> 172.21.15.1:0/1748137915 conn(0x7f1744101560 msgr2=0x7f174418f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:47:36.727 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:30.325+0000 7f1749f20640 1 -- 172.21.15.1:0/1748137915 shutdown_connections 2024-09-11T21:47:36.727 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:30.325+0000 7f1749f20640 1 -- 172.21.15.1:0/1748137915 wait complete. 2024-09-11T21:47:36.727 INFO:teuthology.orchestra.run.smithi001.stdout:mon is available 2024-09-11T21:47:36.728 INFO:teuthology.orchestra.run.smithi001.stdout:Assimilating anything we can from ceph.conf... 2024-09-11T21:47:39.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:39 smithi001 ceph-mon[30567]: from='client.? 172.21.15.1:0/1344586169' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-11T21:47:39.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:39 smithi001 ceph-mon[30567]: from='client.? 172.21.15.1:0/1344586169' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-11T21:47:40.947 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-09-11T21:47:40.947 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout [global] 2024-09-11T21:47:40.947 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout fsid = 3ffca83c-7087-11ef-bcea-c7b262605968 2024-09-11T21:47:40.948 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.1:3300,v1:172.21.15.1:6789] 2024-09-11T21:47:40.948 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-11T21:47:40.948 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-11T21:47:40.948 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-11T21:47:40.948 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-11T21:47:40.948 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-09-11T21:47:40.948 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-11T21:47:40.948 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-11T21:47:40.948 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-09-11T21:47:40.948 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout [osd] 2024-09-11T21:47:40.948 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-11T21:47:40.949 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-11T21:47:40.949 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-11T21:47:40.949 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-11T21:47:40.949 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-11T21:47:40.949 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:39.694+0000 7f78c003d640 1 Processor -- start 2024-09-11T21:47:40.949 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:39.694+0000 7f78c003d640 1 -- start start 2024-09-11T21:47:40.949 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:39.695+0000 7f78c003d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f78b8106610 0x7f78b8106a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:47:40.949 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:39.695+0000 7f78c003d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f78b8106fe0 con 0x7f78b8106610 2024-09-11T21:47:40.949 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:39.695+0000 7f78bddd9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f78b8106610 0x7f78b8106a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:47:40.949 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:39.695+0000 7f78bddd9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f78b8106610 0x7f78b8106a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59434/0 (socket says 172.21.15.1:59434) 2024-09-11T21:47:40.949 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:39.695+0000 7f78bddd9640 1 -- 172.21.15.1:0/409062151 learned_addr learned my addr 172.21.15.1:0/409062151 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:47:40.949 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:39.695+0000 7f78bddd9640 1 -- 172.21.15.1:0/409062151 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78b8107810 con 0x7f78b8106610 2024-09-11T21:47:40.950 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:39.696+0000 7f78bddd9640 1 --2- 172.21.15.1:0/409062151 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f78b8106610 0x7f78b8106a10 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f78b4009e80 tx=0x7f78b402a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bc4c4c2179280fd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:47:40.950 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:39.696+0000 7f78bcdd7640 1 -- 172.21.15.1:0/409062151 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f78b4036070 con 0x7f78b8106610 2024-09-11T21:47:40.950 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:39.696+0000 7f78bcdd7640 1 -- 172.21.15.1:0/409062151 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78b4031400 con 0x7f78b8106610 2024-09-11T21:47:40.950 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:39.696+0000 7f78bcdd7640 1 -- 172.21.15.1:0/409062151 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f78b4031700 con 0x7f78b8106610 2024-09-11T21:47:40.950 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:39.697+0000 7f78c003d640 1 -- 172.21.15.1:0/409062151 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f78b8106610 msgr2=0x7f78b8106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:47:40.950 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:39.697+0000 7f78c003d640 1 --2- 172.21.15.1:0/409062151 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f78b8106610 0x7f78b8106a10 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f78b4009e80 tx=0x7f78b402a7a0 comp rx=0 tx=0).stop 2024-09-11T21:47:40.950 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:39.697+0000 7f78c003d640 1 -- 172.21.15.1:0/409062151 shutdown_connections 2024-09-11T21:47:40.950 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:39.697+0000 7f78c003d640 1 --2- 172.21.15.1:0/409062151 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f78b8106610 0x7f78b8106a10 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:47:40.950 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:39.697+0000 7f78c003d640 1 -- 172.21.15.1:0/409062151 >> 172.21.15.1:0/409062151 conn(0x7f78b8101da0 msgr2=0x7f78b81041e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:47:40.950 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:39.697+0000 7f78c003d640 1 -- 172.21.15.1:0/409062151 shutdown_connections 2024-09-11T21:47:40.950 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:39.697+0000 7f78c003d640 1 -- 172.21.15.1:0/409062151 wait complete. 2024-09-11T21:47:40.950 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:39.697+0000 7f78c003d640 1 Processor -- start 2024-09-11T21:47:40.950 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:39.698+0000 7f78c003d640 1 -- start start 2024-09-11T21:47:40.951 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:39.698+0000 7f78c003d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f78b8106610 0x7f78b818f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:47:40.951 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:39.698+0000 7f78c003d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f78b8107db0 con 0x7f78b8106610 2024-09-11T21:47:40.951 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:39.698+0000 7f78bddd9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f78b8106610 0x7f78b818f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:47:40.951 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:39.698+0000 7f78bddd9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f78b8106610 0x7f78b818f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59438/0 (socket says 172.21.15.1:59438) 2024-09-11T21:47:40.951 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:39.698+0000 7f78bddd9640 1 -- 172.21.15.1:0/1344586169 learned_addr learned my addr 172.21.15.1:0/1344586169 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:47:40.951 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:39.698+0000 7f78bddd9640 1 -- 172.21.15.1:0/1344586169 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78b818f710 con 0x7f78b8106610 2024-09-11T21:47:40.951 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:39.698+0000 7f78bddd9640 1 --2- 172.21.15.1:0/1344586169 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f78b8106610 0x7f78b818f1d0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f78b4034040 tx=0x7f78b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:47:40.951 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:39.698+0000 7f78aeffd640 1 -- 172.21.15.1:0/1344586169 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f78b4041070 con 0x7f78b8106610 2024-09-11T21:47:40.951 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:39.698+0000 7f78aeffd640 1 -- 172.21.15.1:0/1344586169 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78b403c650 con 0x7f78b8106610 2024-09-11T21:47:40.951 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:39.698+0000 7f78aeffd640 1 -- 172.21.15.1:0/1344586169 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f78b4036040 con 0x7f78b8106610 2024-09-11T21:47:40.951 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:39.698+0000 7f78c003d640 1 -- 172.21.15.1:0/1344586169 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78b818d900 con 0x7f78b8106610 2024-09-11T21:47:40.951 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:39.699+0000 7f78c003d640 1 -- 172.21.15.1:0/1344586169 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78b818de40 con 0x7f78b8106610 2024-09-11T21:47:40.951 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:39.699+0000 7f78aeffd640 1 -- 172.21.15.1:0/1344586169 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f78b4044070 con 0x7f78b8106610 2024-09-11T21:47:40.951 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:39.699+0000 7f78aeffd640 1 -- 172.21.15.1:0/1344586169 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f78b4046c10 con 0x7f78b8106610 2024-09-11T21:47:40.952 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:39.700+0000 7f78c003d640 1 -- 172.21.15.1:0/1344586169 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78b8108400 con 0x7f78b8106610 2024-09-11T21:47:40.952 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:39.702+0000 7f78aeffd640 1 -- 172.21.15.1:0/1344586169 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f78b403c950 con 0x7f78b8106610 2024-09-11T21:47:40.952 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:39.762+0000 7f78c003d640 1 -- 172.21.15.1:0/1344586169 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f78b8194800 con 0x7f78b8106610 2024-09-11T21:47:40.952 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:39.765+0000 7f78aeffd640 1 -- 172.21.15.1:0/1344586169 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+427 (secure 0 0 0) 0x7f78b403cb30 con 0x7f78b8106610 2024-09-11T21:47:40.952 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:39.766+0000 7f78aeffd640 1 -- 172.21.15.1:0/1344586169 <== mon.0 v2:172.21.15.1:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f78b4056070 con 0x7f78b8106610 2024-09-11T21:47:40.952 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:39.767+0000 7f78c003d640 1 -- 172.21.15.1:0/1344586169 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f78b8106610 msgr2=0x7f78b818f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:47:40.952 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:39.767+0000 7f78c003d640 1 --2- 172.21.15.1:0/1344586169 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f78b8106610 0x7f78b818f1d0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f78b4034040 tx=0x7f78b4004a40 comp rx=0 tx=0).stop 2024-09-11T21:47:40.952 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:39.767+0000 7f78c003d640 1 -- 172.21.15.1:0/1344586169 shutdown_connections 2024-09-11T21:47:40.952 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:39.767+0000 7f78c003d640 1 --2- 172.21.15.1:0/1344586169 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f78b8106610 0x7f78b818f1d0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:47:40.952 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:39.767+0000 7f78c003d640 1 -- 172.21.15.1:0/1344586169 >> 172.21.15.1:0/1344586169 conn(0x7f78b8101da0 msgr2=0x7f78b8191bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:47:40.952 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:39.767+0000 7f78c003d640 1 -- 172.21.15.1:0/1344586169 shutdown_connections 2024-09-11T21:47:40.952 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:39.767+0000 7f78c003d640 1 -- 172.21.15.1:0/1344586169 wait complete. 2024-09-11T21:47:40.953 INFO:teuthology.orchestra.run.smithi001.stdout:Generating new minimal ceph.conf... 2024-09-11T21:47:41.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:41 smithi001 ceph-mon[30567]: from='client.? 172.21.15.1:0/2380816856' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:47:41.984 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:41.341+0000 7efcb08e0640 1 Processor -- start 2024-09-11T21:47:41.985 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:41.341+0000 7efcb08e0640 1 -- start start 2024-09-11T21:47:41.985 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:41.342+0000 7efcb08e0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efcac109960 0x7efcac109d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:47:41.985 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:41.342+0000 7efcb08e0640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7efcac10a330 con 0x7efcac109960 2024-09-11T21:47:41.985 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:41.342+0000 7efcaa59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efcac109960 0x7efcac109d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:47:41.985 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:41.342+0000 7efcaa59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efcac109960 0x7efcac109d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59440/0 (socket says 172.21.15.1:59440) 2024-09-11T21:47:41.985 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:41.342+0000 7efcaa59c640 1 -- 172.21.15.1:0/1766644377 learned_addr learned my addr 172.21.15.1:0/1766644377 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:47:41.985 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:41.342+0000 7efcaa59c640 1 -- 172.21.15.1:0/1766644377 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcac10aad0 con 0x7efcac109960 2024-09-11T21:47:41.985 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:41.343+0000 7efcaa59c640 1 --2- 172.21.15.1:0/1766644377 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efcac109960 0x7efcac109d60 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7efca4009e80 tx=0x7efca402a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e690a0d4a81ef990 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:47:41.985 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:41.343+0000 7efca959a640 1 -- 172.21.15.1:0/1766644377 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efca4036070 con 0x7efcac109960 2024-09-11T21:47:41.985 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:41.343+0000 7efca959a640 1 -- 172.21.15.1:0/1766644377 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7efca40318c0 con 0x7efcac109960 2024-09-11T21:47:41.985 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:41.343+0000 7efca959a640 1 -- 172.21.15.1:0/1766644377 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efca4007ab0 con 0x7efcac109960 2024-09-11T21:47:41.986 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:41.344+0000 7efcb08e0640 1 -- 172.21.15.1:0/1766644377 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efcac109960 msgr2=0x7efcac109d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:47:41.986 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:41.344+0000 7efcb08e0640 1 --2- 172.21.15.1:0/1766644377 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efcac109960 0x7efcac109d60 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7efca4009e80 tx=0x7efca402a7a0 comp rx=0 tx=0).stop 2024-09-11T21:47:41.986 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:41.344+0000 7efcb08e0640 1 -- 172.21.15.1:0/1766644377 shutdown_connections 2024-09-11T21:47:41.986 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:41.344+0000 7efcb08e0640 1 --2- 172.21.15.1:0/1766644377 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efcac109960 0x7efcac109d60 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:47:41.986 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:41.344+0000 7efcb08e0640 1 -- 172.21.15.1:0/1766644377 >> 172.21.15.1:0/1766644377 conn(0x7efcac07cf30 msgr2=0x7efcac07d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:47:41.986 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:41.344+0000 7efcb08e0640 1 -- 172.21.15.1:0/1766644377 shutdown_connections 2024-09-11T21:47:41.986 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:41.344+0000 7efcb08e0640 1 -- 172.21.15.1:0/1766644377 wait complete. 2024-09-11T21:47:41.986 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:41.345+0000 7efcb08e0640 1 Processor -- start 2024-09-11T21:47:41.986 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:41.345+0000 7efcb08e0640 1 -- start start 2024-09-11T21:47:41.986 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:41.345+0000 7efcb08e0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efcac109960 0x7efcac19d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:47:41.986 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:41.345+0000 7efcb08e0640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7efcac10de80 con 0x7efcac109960 2024-09-11T21:47:41.986 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:41.345+0000 7efcaa59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efcac109960 0x7efcac19d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:47:41.987 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:41.345+0000 7efcaa59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efcac109960 0x7efcac19d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59442/0 (socket says 172.21.15.1:59442) 2024-09-11T21:47:41.987 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:41.345+0000 7efcaa59c640 1 -- 172.21.15.1:0/2380816856 learned_addr learned my addr 172.21.15.1:0/2380816856 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:47:41.987 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:41.345+0000 7efcaa59c640 1 -- 172.21.15.1:0/2380816856 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcac19de60 con 0x7efcac109960 2024-09-11T21:47:41.987 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:41.345+0000 7efcaa59c640 1 --2- 172.21.15.1:0/2380816856 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efcac109960 0x7efcac19d920 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7efca4007ce0 tx=0x7efca4031ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:47:41.987 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:41.346+0000 7efca37fe640 1 -- 172.21.15.1:0/2380816856 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efca4036050 con 0x7efcac109960 2024-09-11T21:47:41.987 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:41.346+0000 7efca37fe640 1 -- 172.21.15.1:0/2380816856 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7efca4042070 con 0x7efcac109960 2024-09-11T21:47:41.987 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:41.346+0000 7efcb08e0640 1 -- 172.21.15.1:0/2380816856 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcac19e0f0 con 0x7efcac109960 2024-09-11T21:47:41.987 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:41.346+0000 7efca37fe640 1 -- 172.21.15.1:0/2380816856 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efca403d8a0 con 0x7efcac109960 2024-09-11T21:47:41.987 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:41.346+0000 7efcb08e0640 1 -- 172.21.15.1:0/2380816856 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcac1a0de0 con 0x7efcac109960 2024-09-11T21:47:41.987 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:41.346+0000 7efca37fe640 1 -- 172.21.15.1:0/2380816856 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7efca403db50 con 0x7efcac109960 2024-09-11T21:47:41.987 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:41.346+0000 7efca37fe640 1 -- 172.21.15.1:0/2380816856 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7efca4007e00 con 0x7efcac109960 2024-09-11T21:47:41.987 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:41.347+0000 7efcb08e0640 1 -- 172.21.15.1:0/2380816856 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcac10e000 con 0x7efcac109960 2024-09-11T21:47:41.988 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:41.349+0000 7efca37fe640 1 -- 172.21.15.1:0/2380816856 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7efca4007310 con 0x7efcac109960 2024-09-11T21:47:41.988 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:41.409+0000 7efcb08e0640 1 -- 172.21.15.1:0/2380816856 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7efcac1a1370 con 0x7efcac109960 2024-09-11T21:47:41.988 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:41.410+0000 7efca37fe640 1 -- 172.21.15.1:0/2380816856 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+173 (secure 0 0 0) 0x7efca4047460 con 0x7efcac109960 2024-09-11T21:47:41.988 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:41.411+0000 7efcb08e0640 1 -- 172.21.15.1:0/2380816856 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efcac109960 msgr2=0x7efcac19d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:47:41.988 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:41.411+0000 7efcb08e0640 1 --2- 172.21.15.1:0/2380816856 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efcac109960 0x7efcac19d920 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7efca4007ce0 tx=0x7efca4031ca0 comp rx=0 tx=0).stop 2024-09-11T21:47:41.988 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:41.411+0000 7efcb08e0640 1 -- 172.21.15.1:0/2380816856 shutdown_connections 2024-09-11T21:47:41.988 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:41.411+0000 7efcb08e0640 1 --2- 172.21.15.1:0/2380816856 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efcac109960 0x7efcac19d920 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:47:41.988 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:41.411+0000 7efcb08e0640 1 -- 172.21.15.1:0/2380816856 >> 172.21.15.1:0/2380816856 conn(0x7efcac07cf30 msgr2=0x7efcac107570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:47:41.988 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:41.411+0000 7efcb08e0640 1 -- 172.21.15.1:0/2380816856 shutdown_connections 2024-09-11T21:47:41.988 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:41.411+0000 7efcb08e0640 1 -- 172.21.15.1:0/2380816856 wait complete. 2024-09-11T21:47:41.988 INFO:teuthology.orchestra.run.smithi001.stdout:Restarting the monitor... 2024-09-11T21:47:42.265 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:42 smithi001 systemd[1]: Stopping Ceph mon.a for 3ffca83c-7087-11ef-bcea-c7b262605968... 2024-09-11T21:47:42.564 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:42 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mon-a[30563]: 2024-09-11T21:47:42.263+0000 7f86e88b4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T21:47:42.564 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:42 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mon-a[30563]: 2024-09-11T21:47:42.263+0000 7f86e88b4640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-11T21:47:42.564 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:42 smithi001 podman[30810]: 2024-09-11 21:47:42.335927101 +0000 UTC m=+0.179282847 container died 29ad5a4ee3579b336933d58b32f64a33fb6b4daef24e9f5123e8090ad353f459 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-11T21:47:42.848 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:42 smithi001 podman[30810]: 2024-09-11 21:47:42.672043017 +0000 UTC m=+0.515398771 container cleanup 29ad5a4ee3579b336933d58b32f64a33fb6b4daef24e9f5123e8090ad353f459 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mon-a, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-11T21:47:42.849 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:42 smithi001 bash[30810]: ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mon-a 2024-09-11T21:47:43.099 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:42 smithi001 podman[30822]: 2024-09-11 21:47:42.847984483 +0000 UTC m=+0.509814675 container remove 29ad5a4ee3579b336933d58b32f64a33fb6b4daef24e9f5123e8090ad353f459 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mon-a, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-11T21:47:43.099 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 systemd[1]: ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-11T21:47:43.099 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 systemd[1]: Stopped Ceph mon.a for 3ffca83c-7087-11ef-bcea-c7b262605968. 2024-09-11T21:47:43.366 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 systemd[1]: Starting Ceph mon.a for 3ffca83c-7087-11ef-bcea-c7b262605968... 2024-09-11T21:47:43.367 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 podman[30916]: 2024-09-11 21:47:43.282423577 +0000 UTC m=+0.084577510 container create f1d1cf7d2034ea29ba929dbc44cd9681238de202ff525f06a6580de4455f2057 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-11T21:47:43.367 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 podman[30916]: 2024-09-11 21:47:43.214038106 +0000 UTC m=+0.016192011 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T21:47:43.548 INFO:teuthology.orchestra.run.smithi001.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-11T21:47:43.644 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 podman[30916]: 2024-09-11 21:47:43.468013002 +0000 UTC m=+0.270166900 container init f1d1cf7d2034ea29ba929dbc44cd9681238de202ff525f06a6580de4455f2057 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mon-a, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T21:47:43.645 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 podman[30916]: 2024-09-11 21:47:43.471149232 +0000 UTC m=+0.273303120 container start f1d1cf7d2034ea29ba929dbc44cd9681238de202ff525f06a6580de4455f2057 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T21:47:43.645 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: set uid:gid to 167:167 (ceph:ceph) 2024-09-11T21:47:43.645 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable), process ceph-mon, pid 2 2024-09-11T21:47:43.645 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: pidfile_write: ignore empty --pid-file 2024-09-11T21:47:43.645 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: load: jerasure load: lrc 2024-09-11T21:47:43.645 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: RocksDB version: 7.9.2 2024-09-11T21:47:43.645 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Git sha 0 2024-09-11T21:47:43.645 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Compile date 2024-09-09 16:23:28 2024-09-11T21:47:43.645 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: DB SUMMARY 2024-09-11T21:47:43.645 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: DB Session ID: ZHOO5SQY9EORM99EITJD 2024-09-11T21:47:43.645 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: CURRENT file: CURRENT 2024-09-11T21:47:43.645 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: IDENTITY file: IDENTITY 2024-09-11T21:47:43.646 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-11T21:47:43.646 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-11T21:47:43.646 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 106095 ; 2024-09-11T21:47:43.646 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.error_if_exists: 0 2024-09-11T21:47:43.646 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.create_if_missing: 0 2024-09-11T21:47:43.646 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.paranoid_checks: 1 2024-09-11T21:47:43.646 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-11T21:47:43.646 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-11T21:47:43.646 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-11T21:47:43.646 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.env: 0x556d31f95940 2024-09-11T21:47:43.646 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.fs: PosixFileSystem 2024-09-11T21:47:43.646 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.info_log: 0x556d332076e0 2024-09-11T21:47:43.647 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.max_file_opening_threads: 16 2024-09-11T21:47:43.647 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.statistics: (nil) 2024-09-11T21:47:43.647 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.use_fsync: 0 2024-09-11T21:47:43.647 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.max_log_file_size: 0 2024-09-11T21:47:43.647 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-11T21:47:43.647 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-11T21:47:43.647 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.keep_log_file_num: 1000 2024-09-11T21:47:43.647 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.recycle_log_file_num: 0 2024-09-11T21:47:43.647 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.allow_fallocate: 1 2024-09-11T21:47:43.647 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.allow_mmap_reads: 0 2024-09-11T21:47:43.647 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.allow_mmap_writes: 0 2024-09-11T21:47:43.648 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.use_direct_reads: 0 2024-09-11T21:47:43.648 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-11T21:47:43.648 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.create_missing_column_families: 0 2024-09-11T21:47:43.648 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.db_log_dir: 2024-09-11T21:47:43.648 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.wal_dir: 2024-09-11T21:47:43.648 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-11T21:47:43.648 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-11T21:47:43.648 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-11T21:47:43.648 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-11T21:47:43.648 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-11T21:47:43.648 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-11T21:47:43.648 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.advise_random_on_open: 1 2024-09-11T21:47:43.649 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.db_write_buffer_size: 0 2024-09-11T21:47:43.649 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.write_buffer_manager: 0x556d33216dc0 2024-09-11T21:47:43.649 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-11T21:47:43.649 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-11T21:47:43.649 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-11T21:47:43.649 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.rate_limiter: (nil) 2024-09-11T21:47:43.649 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-11T21:47:43.649 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.wal_recovery_mode: 2 2024-09-11T21:47:43.649 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.enable_thread_tracking: 0 2024-09-11T21:47:43.649 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.enable_pipelined_write: 0 2024-09-11T21:47:43.649 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.unordered_write: 0 2024-09-11T21:47:43.649 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-11T21:47:43.649 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-11T21:47:43.650 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-11T21:47:43.650 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-11T21:47:43.650 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.row_cache: None 2024-09-11T21:47:43.650 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.wal_filter: None 2024-09-11T21:47:43.650 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-11T21:47:43.651 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.allow_ingest_behind: 0 2024-09-11T21:47:43.651 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.two_write_queues: 0 2024-09-11T21:47:43.651 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.manual_wal_flush: 0 2024-09-11T21:47:43.651 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.wal_compression: 0 2024-09-11T21:47:43.652 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.atomic_flush: 0 2024-09-11T21:47:43.652 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-11T21:47:43.652 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-11T21:47:43.652 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-11T21:47:43.652 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.log_readahead_size: 0 2024-09-11T21:47:43.652 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-11T21:47:43.652 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.best_efforts_recovery: 0 2024-09-11T21:47:43.652 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-11T21:47:43.652 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-11T21:47:43.652 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.allow_data_in_errors: 0 2024-09-11T21:47:43.652 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.db_host_id: __hostname__ 2024-09-11T21:47:43.652 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-11T21:47:43.652 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.max_background_jobs: 2 2024-09-11T21:47:43.653 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.max_background_compactions: -1 2024-09-11T21:47:43.653 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.max_subcompactions: 1 2024-09-11T21:47:43.653 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-11T21:47:43.653 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-11T21:47:43.653 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-11T21:47:43.653 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.max_total_wal_size: 0 2024-09-11T21:47:43.653 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-11T21:47:43.653 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-11T21:47:43.653 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-11T21:47:43.653 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-11T21:47:43.653 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.max_open_files: -1 2024-09-11T21:47:43.654 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.bytes_per_sync: 0 2024-09-11T21:47:43.654 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-11T21:47:43.654 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-11T21:47:43.654 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.compaction_readahead_size: 0 2024-09-11T21:47:43.654 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.max_background_flushes: -1 2024-09-11T21:47:43.654 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Compression algorithms supported: 2024-09-11T21:47:43.654 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: kZSTD supported: 0 2024-09-11T21:47:43.654 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: kXpressCompression supported: 0 2024-09-11T21:47:43.654 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: kBZip2Compression supported: 0 2024-09-11T21:47:43.654 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-11T21:47:43.654 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: kLZ4Compression supported: 1 2024-09-11T21:47:43.654 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: kZlibCompression supported: 1 2024-09-11T21:47:43.654 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: kLZ4HCCompression supported: 1 2024-09-11T21:47:43.655 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: kSnappyCompression supported: 1 2024-09-11T21:47:43.655 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-11T21:47:43.655 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-11T21:47:43.655 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-11T21:47:43.655 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-11T21:47:43.655 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-11T21:47:43.655 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.merge_operator: 2024-09-11T21:47:43.655 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.compaction_filter: None 2024-09-11T21:47:43.655 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.compaction_filter_factory: None 2024-09-11T21:47:43.655 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.sst_partitioner_factory: None 2024-09-11T21:47:43.655 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-11T21:47:43.655 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-11T21:47:43.655 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x556d33207320) 2024-09-11T21:47:43.656 INFO:journalctl@ceph.mon.a.smithi001.stdout: cache_index_and_filter_blocks: 1 2024-09-11T21:47:43.656 INFO:journalctl@ceph.mon.a.smithi001.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-11T21:47:43.656 INFO:journalctl@ceph.mon.a.smithi001.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-11T21:47:43.656 INFO:journalctl@ceph.mon.a.smithi001.stdout: pin_top_level_index_and_filter: 1 2024-09-11T21:47:43.656 INFO:journalctl@ceph.mon.a.smithi001.stdout: index_type: 0 2024-09-11T21:47:43.656 INFO:journalctl@ceph.mon.a.smithi001.stdout: data_block_index_type: 0 2024-09-11T21:47:43.656 INFO:journalctl@ceph.mon.a.smithi001.stdout: index_shortening: 1 2024-09-11T21:47:43.656 INFO:journalctl@ceph.mon.a.smithi001.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-11T21:47:43.656 INFO:journalctl@ceph.mon.a.smithi001.stdout: checksum: 4 2024-09-11T21:47:43.656 INFO:journalctl@ceph.mon.a.smithi001.stdout: no_block_cache: 0 2024-09-11T21:47:43.656 INFO:journalctl@ceph.mon.a.smithi001.stdout: block_cache: 0x556d331fb1f0 2024-09-11T21:47:43.656 INFO:journalctl@ceph.mon.a.smithi001.stdout: block_cache_name: BinnedLRUCache 2024-09-11T21:47:43.656 INFO:journalctl@ceph.mon.a.smithi001.stdout: block_cache_options: 2024-09-11T21:47:43.657 INFO:journalctl@ceph.mon.a.smithi001.stdout: capacity : 536870912 2024-09-11T21:47:43.657 INFO:journalctl@ceph.mon.a.smithi001.stdout: num_shard_bits : 4 2024-09-11T21:47:43.657 INFO:journalctl@ceph.mon.a.smithi001.stdout: strict_capacity_limit : 0 2024-09-11T21:47:43.657 INFO:journalctl@ceph.mon.a.smithi001.stdout: high_pri_pool_ratio: 0.000 2024-09-11T21:47:43.657 INFO:journalctl@ceph.mon.a.smithi001.stdout: block_cache_compressed: (nil) 2024-09-11T21:47:43.657 INFO:journalctl@ceph.mon.a.smithi001.stdout: persistent_cache: (nil) 2024-09-11T21:47:43.657 INFO:journalctl@ceph.mon.a.smithi001.stdout: block_size: 4096 2024-09-11T21:47:43.657 INFO:journalctl@ceph.mon.a.smithi001.stdout: block_size_deviation: 10 2024-09-11T21:47:43.657 INFO:journalctl@ceph.mon.a.smithi001.stdout: block_restart_interval: 16 2024-09-11T21:47:43.657 INFO:journalctl@ceph.mon.a.smithi001.stdout: index_block_restart_interval: 1 2024-09-11T21:47:43.657 INFO:journalctl@ceph.mon.a.smithi001.stdout: metadata_block_size: 4096 2024-09-11T21:47:43.657 INFO:journalctl@ceph.mon.a.smithi001.stdout: partition_filters: 0 2024-09-11T21:47:43.658 INFO:journalctl@ceph.mon.a.smithi001.stdout: use_delta_encoding: 1 2024-09-11T21:47:43.658 INFO:journalctl@ceph.mon.a.smithi001.stdout: filter_policy: bloomfilter 2024-09-11T21:47:43.658 INFO:journalctl@ceph.mon.a.smithi001.stdout: whole_key_filtering: 1 2024-09-11T21:47:43.658 INFO:journalctl@ceph.mon.a.smithi001.stdout: verify_compression: 0 2024-09-11T21:47:43.658 INFO:journalctl@ceph.mon.a.smithi001.stdout: read_amp_bytes_per_bit: 0 2024-09-11T21:47:43.658 INFO:journalctl@ceph.mon.a.smithi001.stdout: format_version: 5 2024-09-11T21:47:43.658 INFO:journalctl@ceph.mon.a.smithi001.stdout: enable_index_compression: 1 2024-09-11T21:47:43.658 INFO:journalctl@ceph.mon.a.smithi001.stdout: block_align: 0 2024-09-11T21:47:43.658 INFO:journalctl@ceph.mon.a.smithi001.stdout: max_auto_readahead_size: 262144 2024-09-11T21:47:43.658 INFO:journalctl@ceph.mon.a.smithi001.stdout: prepopulate_block_cache: 0 2024-09-11T21:47:43.658 INFO:journalctl@ceph.mon.a.smithi001.stdout: initial_auto_readahead_size: 8192 2024-09-11T21:47:43.658 INFO:journalctl@ceph.mon.a.smithi001.stdout: num_file_reads_for_auto_readahead: 2 2024-09-11T21:47:43.658 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.write_buffer_size: 33554432 2024-09-11T21:47:43.659 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.max_write_buffer_number: 2 2024-09-11T21:47:43.659 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.compression: NoCompression 2024-09-11T21:47:43.659 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.bottommost_compression: Disabled 2024-09-11T21:47:43.659 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.prefix_extractor: nullptr 2024-09-11T21:47:43.659 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-11T21:47:43.659 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.num_levels: 7 2024-09-11T21:47:43.659 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-11T21:47:43.659 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-11T21:47:43.659 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-11T21:47:43.659 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-11T21:47:43.659 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-11T21:47:43.659 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-11T21:47:43.659 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-11T21:47:43.660 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-11T21:47:43.660 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-11T21:47:43.660 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-11T21:47:43.660 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-11T21:47:43.660 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-11T21:47:43.660 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-11T21:47:43.660 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.compression_opts.level: 32767 2024-09-11T21:47:43.660 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.compression_opts.strategy: 0 2024-09-11T21:47:43.660 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-11T21:47:43.660 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-11T21:47:43.660 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-11T21:47:43.660 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-11T21:47:43.660 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.compression_opts.enabled: false 2024-09-11T21:47:43.660 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-11T21:47:43.661 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-11T21:47:43.661 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-11T21:47:43.661 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-11T21:47:43.661 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.target_file_size_base: 67108864 2024-09-11T21:47:43.661 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-11T21:47:43.661 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-11T21:47:43.661 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-11T21:47:43.661 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-11T21:47:43.661 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-11T21:47:43.661 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-11T21:47:43.661 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-11T21:47:43.661 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-11T21:47:43.661 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-11T21:47:43.662 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-11T21:47:43.662 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-11T21:47:43.662 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-11T21:47:43.662 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-11T21:47:43.662 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-11T21:47:43.662 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.arena_block_size: 1048576 2024-09-11T21:47:43.662 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-11T21:47:43.662 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-11T21:47:43.662 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.disable_auto_compactions: 0 2024-09-11T21:47:43.662 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-11T21:47:43.662 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-11T21:47:43.662 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-11T21:47:43.662 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-11T21:47:43.662 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-11T21:47:43.662 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-11T21:47:43.663 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-11T21:47:43.663 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-11T21:47:43.663 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-11T21:47:43.663 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-11T21:47:43.663 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-11T21:47:43.663 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.inplace_update_support: 0 2024-09-11T21:47:43.663 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-11T21:47:43.663 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-11T21:47:43.663 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-11T21:47:43.663 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-11T21:47:43.663 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.bloom_locality: 0 2024-09-11T21:47:43.663 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.max_successive_merges: 0 2024-09-11T21:47:43.664 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-11T21:47:43.664 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.paranoid_file_checks: 0 2024-09-11T21:47:43.664 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.force_consistency_checks: 1 2024-09-11T21:47:43.664 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.report_bg_io_stats: 0 2024-09-11T21:47:43.664 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.ttl: 2592000 2024-09-11T21:47:43.664 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-11T21:47:43.664 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-11T21:47:43.664 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-11T21:47:43.664 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.enable_blob_files: false 2024-09-11T21:47:43.664 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.min_blob_size: 0 2024-09-11T21:47:43.664 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.blob_file_size: 268435456 2024-09-11T21:47:43.664 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-11T21:47:43.665 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-11T21:47:43.665 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-11T21:47:43.665 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-11T21:47:43.665 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-11T21:47:43.665 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.blob_file_starting_level: 0 2024-09-11T21:47:43.665 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-11T21:47:43.665 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-11T21:47:43.665 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-11T21:47:43.665 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 9670f833-d60d-4f29-8406-347520352fef 2024-09-11T21:47:43.665 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726091263498002, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-11T21:47:43.665 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-11T21:47:43.665 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726091263499084, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100958, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 336, "table_properties": {"data_size": 98843, "index_size": 307, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14453, "raw_average_key_size": 49, "raw_value_size": 91023, "raw_average_value_size": 312, "num_data_blocks": 12, "num_entries": 291, "num_filter_entries": 291, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726091263, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "9670f833-d60d-4f29-8406-347520352fef", "db_session_id": "ZHOO5SQY9EORM99EITJD", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-11T21:47:43.665 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726091263499181, "job": 1, "event": "recovery_finished"} 2024-09-11T21:47:43.666 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-11T21:47:43.666 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-11T21:47:43.666 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x556d33228e00 2024-09-11T21:47:43.666 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: DB pointer 0x556d33324000 2024-09-11T21:47:43.666 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-11T21:47:43.666 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-11T21:47:43.666 INFO:journalctl@ceph.mon.a.smithi001.stdout: ** DB Stats ** 2024-09-11T21:47:43.666 INFO:journalctl@ceph.mon.a.smithi001.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T21:47:43.666 INFO:journalctl@ceph.mon.a.smithi001.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-11T21:47:43.666 INFO:journalctl@ceph.mon.a.smithi001.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-11T21:47:43.666 INFO:journalctl@ceph.mon.a.smithi001.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-11T21:47:43.666 INFO:journalctl@ceph.mon.a.smithi001.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-11T21:47:43.667 INFO:journalctl@ceph.mon.a.smithi001.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-11T21:47:43.667 INFO:journalctl@ceph.mon.a.smithi001.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-11T21:47:43.667 INFO:journalctl@ceph.mon.a.smithi001.stdout: 2024-09-11T21:47:43.667 INFO:journalctl@ceph.mon.a.smithi001.stdout: ** Compaction Stats [default] ** 2024-09-11T21:47:43.667 INFO:journalctl@ceph.mon.a.smithi001.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-11T21:47:43.667 INFO:journalctl@ceph.mon.a.smithi001.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-11T21:47:43.667 INFO:journalctl@ceph.mon.a.smithi001.stdout: L0 2/0 100.45 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 143.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-11T21:47:43.667 INFO:journalctl@ceph.mon.a.smithi001.stdout: Sum 2/0 100.45 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 143.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-11T21:47:43.667 INFO:journalctl@ceph.mon.a.smithi001.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 143.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-11T21:47:43.667 INFO:journalctl@ceph.mon.a.smithi001.stdout: 2024-09-11T21:47:43.667 INFO:journalctl@ceph.mon.a.smithi001.stdout: ** Compaction Stats [default] ** 2024-09-11T21:47:43.667 INFO:journalctl@ceph.mon.a.smithi001.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-11T21:47:43.668 INFO:journalctl@ceph.mon.a.smithi001.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T21:47:43.668 INFO:journalctl@ceph.mon.a.smithi001.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 143.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-11T21:47:43.668 INFO:journalctl@ceph.mon.a.smithi001.stdout: 2024-09-11T21:47:43.668 INFO:journalctl@ceph.mon.a.smithi001.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-11T21:47:43.668 INFO:journalctl@ceph.mon.a.smithi001.stdout: 2024-09-11T21:47:43.668 INFO:journalctl@ceph.mon.a.smithi001.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T21:47:43.668 INFO:journalctl@ceph.mon.a.smithi001.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-11T21:47:43.668 INFO:journalctl@ceph.mon.a.smithi001.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-11T21:47:43.668 INFO:journalctl@ceph.mon.a.smithi001.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-11T21:47:43.668 INFO:journalctl@ceph.mon.a.smithi001.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-11T21:47:43.668 INFO:journalctl@ceph.mon.a.smithi001.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-11T21:47:43.668 INFO:journalctl@ceph.mon.a.smithi001.stdout: Cumulative compaction: 0.00 GB write, 29.29 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T21:47:43.669 INFO:journalctl@ceph.mon.a.smithi001.stdout: Interval compaction: 0.00 GB write, 29.29 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T21:47:43.669 INFO:journalctl@ceph.mon.a.smithi001.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-11T21:47:43.669 INFO:journalctl@ceph.mon.a.smithi001.stdout: Block cache BinnedLRUCache@0x556d331fb1f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-11T21:47:43.669 INFO:journalctl@ceph.mon.a.smithi001.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-09-11T21:47:43.669 INFO:journalctl@ceph.mon.a.smithi001.stdout: 2024-09-11T21:47:43.669 INFO:journalctl@ceph.mon.a.smithi001.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-11T21:47:43.669 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: starting mon.a rank 0 at public addrs [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] at bind addrs [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 3ffca83c-7087-11ef-bcea-c7b262605968 2024-09-11T21:47:43.669 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: mon.a@-1(???) e1 preinit fsid 3ffca83c-7087-11ef-bcea-c7b262605968 2024-09-11T21:47:43.669 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: mon.a@-1(???).mds e1 new map 2024-09-11T21:47:43.669 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: mon.a@-1(???).mds e1 print_map 2024-09-11T21:47:43.669 INFO:journalctl@ceph.mon.a.smithi001.stdout: e1 2024-09-11T21:47:43.669 INFO:journalctl@ceph.mon.a.smithi001.stdout: btime 2024-09-11T21:47:28:868551+0000 2024-09-11T21:47:43.670 INFO:journalctl@ceph.mon.a.smithi001.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-11T21:47:43.670 INFO:journalctl@ceph.mon.a.smithi001.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-11T21:47:43.670 INFO:journalctl@ceph.mon.a.smithi001.stdout: legacy client fscid: -1 2024-09-11T21:47:43.670 INFO:journalctl@ceph.mon.a.smithi001.stdout: 2024-09-11T21:47:43.670 INFO:journalctl@ceph.mon.a.smithi001.stdout: No filesystems configured 2024-09-11T21:47:43.670 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-11T21:47:43.670 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T21:47:43.670 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T21:47:43.670 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T21:47:43.670 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 ceph-mon[30935]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-11T21:47:43.670 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 bash[30916]: f1d1cf7d2034ea29ba929dbc44cd9681238de202ff525f06a6580de4455f2057 2024-09-11T21:47:43.670 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:43 smithi001 systemd[1]: Started Ceph mon.a for 3ffca83c-7087-11ef-bcea-c7b262605968. 2024-09-11T21:47:44.625 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:44 smithi001 ceph-mon[30935]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-11T21:47:44.625 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:44 smithi001 ceph-mon[30935]: monmap epoch 1 2024-09-11T21:47:44.625 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:44 smithi001 ceph-mon[30935]: fsid 3ffca83c-7087-11ef-bcea-c7b262605968 2024-09-11T21:47:44.626 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:44 smithi001 ceph-mon[30935]: last_changed 2024-09-11T21:47:25.953679+0000 2024-09-11T21:47:44.626 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:44 smithi001 ceph-mon[30935]: created 2024-09-11T21:47:25.953679+0000 2024-09-11T21:47:44.626 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:44 smithi001 ceph-mon[30935]: min_mon_release 19 (squid) 2024-09-11T21:47:44.626 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:44 smithi001 ceph-mon[30935]: election_strategy: 1 2024-09-11T21:47:44.626 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:44 smithi001 ceph-mon[30935]: 0: [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] mon.a 2024-09-11T21:47:44.626 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:44 smithi001 ceph-mon[30935]: fsmap 2024-09-11T21:47:44.626 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:44 smithi001 ceph-mon[30935]: osdmap e1: 0 total, 0 up, 0 in 2024-09-11T21:47:44.626 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:44 smithi001 ceph-mon[30935]: mgrmap e1: no daemons active 2024-09-11T21:47:44.626 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:44 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/2382949042' entity='client.admin' 2024-09-11T21:47:44.628 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:43.954+0000 7f22f5bba640 1 Processor -- start 2024-09-11T21:47:44.632 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:43.954+0000 7f22f5bba640 1 -- start start 2024-09-11T21:47:44.632 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:43.954+0000 7f22f5bba640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f22f0107720 0x7f22f0107b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:47:44.632 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:43.954+0000 7f22f5bba640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f22f01080f0 con 0x7f22f0107720 2024-09-11T21:47:44.632 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:43.955+0000 7f22ef7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f22f0107720 0x7f22f0107b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:47:44.632 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:43.955+0000 7f22ef7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f22f0107720 0x7f22f0107b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59444/0 (socket says 172.21.15.1:59444) 2024-09-11T21:47:44.632 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:43.955+0000 7f22ef7fe640 1 -- 172.21.15.1:0/2297961021 learned_addr learned my addr 172.21.15.1:0/2297961021 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:47:44.633 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:43.955+0000 7f22ef7fe640 1 -- 172.21.15.1:0/2297961021 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22f01088d0 con 0x7f22f0107720 2024-09-11T21:47:44.633 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:43.957+0000 7f22ef7fe640 1 --2- 172.21.15.1:0/2297961021 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f22f0107720 0x7f22f0107b20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f22e0009820 tx=0x7f22e002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10030f3931ace38d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:47:44.633 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:43.957+0000 7f22eeffd640 1 -- 172.21.15.1:0/2297961021 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f22e0037070 con 0x7f22f0107720 2024-09-11T21:47:44.633 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:43.957+0000 7f22eeffd640 1 -- 172.21.15.1:0/2297961021 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f22e0032900 con 0x7f22f0107720 2024-09-11T21:47:44.633 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:43.957+0000 7f22eeffd640 1 -- 172.21.15.1:0/2297961021 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f22e0030850 con 0x7f22f0107720 2024-09-11T21:47:44.633 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:43.958+0000 7f22f5bba640 1 -- 172.21.15.1:0/2297961021 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f22f0107720 msgr2=0x7f22f0107b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:47:44.633 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:43.958+0000 7f22f5bba640 1 --2- 172.21.15.1:0/2297961021 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f22f0107720 0x7f22f0107b20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f22e0009820 tx=0x7f22e002a270 comp rx=0 tx=0).stop 2024-09-11T21:47:44.633 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:43.958+0000 7f22f5bba640 1 -- 172.21.15.1:0/2297961021 shutdown_connections 2024-09-11T21:47:44.633 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:43.958+0000 7f22f5bba640 1 --2- 172.21.15.1:0/2297961021 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f22f0107720 0x7f22f0107b20 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:47:44.633 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:43.958+0000 7f22f5bba640 1 -- 172.21.15.1:0/2297961021 >> 172.21.15.1:0/2297961021 conn(0x7f22f0102ef0 msgr2=0x7f22f0105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:47:44.633 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:43.958+0000 7f22f5bba640 1 -- 172.21.15.1:0/2297961021 shutdown_connections 2024-09-11T21:47:44.634 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:43.958+0000 7f22f5bba640 1 -- 172.21.15.1:0/2297961021 wait complete. 2024-09-11T21:47:44.634 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:43.958+0000 7f22f5bba640 1 Processor -- start 2024-09-11T21:47:44.634 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:43.958+0000 7f22f5bba640 1 -- start start 2024-09-11T21:47:44.634 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:43.959+0000 7f22f5bba640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f22f0107720 0x7f22f019b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:47:44.634 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:43.959+0000 7f22f5bba640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f22f010bc80 con 0x7f22f0107720 2024-09-11T21:47:44.634 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:43.959+0000 7f22ef7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f22f0107720 0x7f22f019b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:47:44.634 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:43.959+0000 7f22ef7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f22f0107720 0x7f22f019b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59458/0 (socket says 172.21.15.1:59458) 2024-09-11T21:47:44.634 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:43.959+0000 7f22ef7fe640 1 -- 172.21.15.1:0/2382949042 learned_addr learned my addr 172.21.15.1:0/2382949042 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:47:44.634 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:43.959+0000 7f22ef7fe640 1 -- 172.21.15.1:0/2382949042 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22f019bc00 con 0x7f22f0107720 2024-09-11T21:47:44.634 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:43.959+0000 7f22ef7fe640 1 --2- 172.21.15.1:0/2382949042 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f22f0107720 0x7f22f019b6c0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f22e00097f0 tx=0x7f22e0030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:47:44.634 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:43.959+0000 7f22ed7fa640 1 -- 172.21.15.1:0/2382949042 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f22e0037040 con 0x7f22f0107720 2024-09-11T21:47:44.634 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:43.959+0000 7f22ed7fa640 1 -- 172.21.15.1:0/2382949042 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f22e0040070 con 0x7f22f0107720 2024-09-11T21:47:44.634 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:43.959+0000 7f22f5bba640 1 -- 172.21.15.1:0/2382949042 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22f019be90 con 0x7f22f0107720 2024-09-11T21:47:44.635 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:43.959+0000 7f22ed7fa640 1 -- 172.21.15.1:0/2382949042 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f22e003a390 con 0x7f22f0107720 2024-09-11T21:47:44.635 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:43.960+0000 7f22f5bba640 1 -- 172.21.15.1:0/2382949042 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22f019eb80 con 0x7f22f0107720 2024-09-11T21:47:44.635 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:43.960+0000 7f22ed7fa640 1 -- 172.21.15.1:0/2382949042 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f22e003a5e0 con 0x7f22f0107720 2024-09-11T21:47:44.635 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:43.960+0000 7f22ed7fa640 1 -- 172.21.15.1:0/2382949042 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f22e0045950 con 0x7f22f0107720 2024-09-11T21:47:44.635 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:43.961+0000 7f22f5bba640 1 -- 172.21.15.1:0/2382949042 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22f0107ba0 con 0x7f22f0107720 2024-09-11T21:47:44.635 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:43.963+0000 7f22ed7fa640 1 -- 172.21.15.1:0/2382949042 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f22e0045460 con 0x7f22f0107720 2024-09-11T21:47:44.635 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:44.022+0000 7f22f5bba640 1 -- 172.21.15.1:0/2382949042 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f22f019c310 con 0x7f22f0107720 2024-09-11T21:47:44.635 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:44.024+0000 7f22ed7fa640 1 -- 172.21.15.1:0/2382949042 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f22e003b430 con 0x7f22f0107720 2024-09-11T21:47:44.635 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:44.025+0000 7f22f5bba640 1 -- 172.21.15.1:0/2382949042 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f22f0107720 msgr2=0x7f22f019b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:47:44.635 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:44.025+0000 7f22f5bba640 1 --2- 172.21.15.1:0/2382949042 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f22f0107720 0x7f22f019b6c0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f22e00097f0 tx=0x7f22e0030d80 comp rx=0 tx=0).stop 2024-09-11T21:47:44.635 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:44.025+0000 7f22f5bba640 1 -- 172.21.15.1:0/2382949042 shutdown_connections 2024-09-11T21:47:44.636 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:44.025+0000 7f22f5bba640 1 --2- 172.21.15.1:0/2382949042 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f22f0107720 0x7f22f019b6c0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:47:44.636 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:44.025+0000 7f22f5bba640 1 -- 172.21.15.1:0/2382949042 >> 172.21.15.1:0/2382949042 conn(0x7f22f0102ef0 msgr2=0x7f22f0103a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:47:44.636 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:44.025+0000 7f22f5bba640 1 -- 172.21.15.1:0/2382949042 shutdown_connections 2024-09-11T21:47:44.636 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:44.025+0000 7f22f5bba640 1 -- 172.21.15.1:0/2382949042 wait complete. 2024-09-11T21:47:44.636 INFO:teuthology.orchestra.run.smithi001.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-11T21:47:44.658 INFO:teuthology.orchestra.run.smithi001.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-11T21:47:44.658 INFO:teuthology.orchestra.run.smithi001.stdout:Creating mgr... 2024-09-11T21:47:44.659 INFO:teuthology.orchestra.run.smithi001.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-11T21:47:44.659 INFO:teuthology.orchestra.run.smithi001.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-11T21:47:44.906 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mgr.x 2024-09-11T21:47:44.906 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stderr Failed to reset failed state of unit ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mgr.x.service: Unit ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mgr.x.service not loaded. 2024-09-11T21:47:45.064 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-3ffca83c-7087-11ef-bcea-c7b262605968.target.wants/ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-3ffca83c-7087-11ef-bcea-c7b262605968@.service. 2024-09-11T21:47:45.193 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:47:45 smithi001 systemd[1]: Starting Ceph mgr.x for 3ffca83c-7087-11ef-bcea-c7b262605968... 2024-09-11T21:47:45.627 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:47:45 smithi001 podman[31146]: 2024-09-11 21:47:45.285560575 +0000 UTC m=+0.101517979 container create 4bf5fcbd71e5d307481af2d58c5ea88a95d452fe77949423dcca832fe91c6b02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-11T21:47:45.627 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:47:45 smithi001 podman[31146]: 2024-09-11 21:47:45.200754513 +0000 UTC m=+0.016711928 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T21:47:45.627 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:47:45 smithi001 podman[31146]: 2024-09-11 21:47:45.47938935 +0000 UTC m=+0.295346757 container init 4bf5fcbd71e5d307481af2d58c5ea88a95d452fe77949423dcca832fe91c6b02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-11T21:47:45.627 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:47:45 smithi001 podman[31146]: 2024-09-11 21:47:45.482277785 +0000 UTC m=+0.298235183 container start 4bf5fcbd71e5d307481af2d58c5ea88a95d452fe77949423dcca832fe91c6b02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-11T21:47:45.627 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:47:45 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:47:45.606+0000 7fe51ce9a040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T21:47:45.892 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:47:45 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:47:45.665+0000 7fe51ce9a040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T21:47:45.892 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:47:45 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:47:45.822+0000 7fe51ce9a040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-11T21:47:45.892 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:47:45 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:47:45.891+0000 7fe51ce9a040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T21:47:46.167 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:47:46 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:47:46.046+0000 7fe51ce9a040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T21:47:46.182 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T21:47:46.182 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout disabled 2024-09-11T21:47:46.190 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T21:47:46.190 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout inactive 2024-09-11T21:47:46.190 INFO:teuthology.orchestra.run.smithi001.stdout:firewalld.service is not enabled 2024-09-11T21:47:46.190 INFO:teuthology.orchestra.run.smithi001.stdout:Not possible to enable service . firewalld.service is not available 2024-09-11T21:47:46.199 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T21:47:46.199 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout disabled 2024-09-11T21:47:46.206 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T21:47:46.206 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout inactive 2024-09-11T21:47:46.206 INFO:teuthology.orchestra.run.smithi001.stdout:firewalld.service is not enabled 2024-09-11T21:47:46.207 INFO:teuthology.orchestra.run.smithi001.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-11T21:47:46.207 INFO:teuthology.orchestra.run.smithi001.stdout:Waiting for mgr to start... 2024-09-11T21:47:46.207 INFO:teuthology.orchestra.run.smithi001.stdout:Waiting for mgr... 2024-09-11T21:47:46.443 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:47:46 smithi001 bash[31146]: 4bf5fcbd71e5d307481af2d58c5ea88a95d452fe77949423dcca832fe91c6b02 2024-09-11T21:47:46.443 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:47:46 smithi001 systemd[1]: Started Ceph mgr.x for 3ffca83c-7087-11ef-bcea-c7b262605968. 2024-09-11T21:47:46.443 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:47:46 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:47:46.193+0000 7fe51ce9a040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T21:47:46.802 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:46 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/2286311066' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-11T21:47:46.803 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:47:46 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:47:46.537+0000 7fe51ce9a040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T21:47:46.803 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:47:46 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:47:46.604+0000 7fe51ce9a040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T21:47:47.193 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:47:46 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:47:46.802+0000 7fe51ce9a040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T21:47:47.238 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-09-11T21:47:47.238 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout { 2024-09-11T21:47:47.238 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "fsid": "3ffca83c-7087-11ef-bcea-c7b262605968", 2024-09-11T21:47:47.238 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "health": { 2024-09-11T21:47:47.239 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-11T21:47:47.239 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-11T21:47:47.239 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-11T21:47:47.239 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-09-11T21:47:47.239 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-11T21:47:47.239 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-11T21:47:47.239 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 0 2024-09-11T21:47:47.239 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2024-09-11T21:47:47.239 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-11T21:47:47.239 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "a" 2024-09-11T21:47:47.239 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2024-09-11T21:47:47.239 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-09-11T21:47:47.240 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-11T21:47:47.240 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T21:47:47.240 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-11T21:47:47.240 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-11T21:47:47.240 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-09-11T21:47:47.240 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-11T21:47:47.240 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T21:47:47.240 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-11T21:47:47.240 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-11T21:47:47.251 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-11T21:47:47.251 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-11T21:47:47.251 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-11T21:47:47.251 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-11T21:47:47.251 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-09-11T21:47:47.252 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-11T21:47:47.252 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-11T21:47:47.252 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-11T21:47:47.252 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-11T21:47:47.252 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-11T21:47:47.252 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-11T21:47:47.252 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-11T21:47:47.252 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-11T21:47:47.252 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-11T21:47:47.252 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-09-11T21:47:47.252 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-11T21:47:47.252 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T21:47:47.252 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "btime": "2024-09-11T21:47:28:868551+0000", 2024-09-11T21:47:47.253 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-11T21:47:47.253 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-11T21:47:47.253 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-09-11T21:47:47.253 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-11T21:47:47.253 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-11T21:47:47.253 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-11T21:47:47.253 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-11T21:47:47.253 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-11T21:47:47.253 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-11T21:47:47.253 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "restful" 2024-09-11T21:47:47.253 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2024-09-11T21:47:47.253 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T21:47:47.254 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-09-11T21:47:47.254 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-11T21:47:47.254 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T21:47:47.254 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "modified": "2024-09-11T21:47:28.869189+0000", 2024-09-11T21:47:47.254 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T21:47:47.254 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-09-11T21:47:47.254 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-11T21:47:47.254 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout } 2024-09-11T21:47:47.254 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:46.576+0000 7fd292c33640 1 Processor -- start 2024-09-11T21:47:47.254 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:46.576+0000 7fd292c33640 1 -- start start 2024-09-11T21:47:47.254 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:46.576+0000 7fd292c33640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd28c108260 0x7fd28c10a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:47:47.254 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:46.576+0000 7fd292c33640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd28c07bdd0 con 0x7fd28c108260 2024-09-11T21:47:47.255 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:46.576+0000 7fd2909cf640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd28c108260 0x7fd28c10a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:47:47.255 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:46.577+0000 7fd2909cf640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd28c108260 0x7fd28c10a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36532/0 (socket says 172.21.15.1:36532) 2024-09-11T21:47:47.255 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:46.577+0000 7fd2909cf640 1 -- 172.21.15.1:0/2307141668 learned_addr learned my addr 172.21.15.1:0/2307141668 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:47:47.255 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:46.577+0000 7fd2909cf640 1 -- 172.21.15.1:0/2307141668 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd28c10ab90 con 0x7fd28c108260 2024-09-11T21:47:47.255 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:46.577+0000 7fd2909cf640 1 --2- 172.21.15.1:0/2307141668 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd28c108260 0x7fd28c10a650 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fd280009820 tx=0x7fd28002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b52443126ad0a1d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:47:47.255 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:46.578+0000 7fd28b7fe640 1 -- 172.21.15.1:0/2307141668 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd280037070 con 0x7fd28c108260 2024-09-11T21:47:47.255 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:46.578+0000 7fd28b7fe640 1 -- 172.21.15.1:0/2307141668 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd280032900 con 0x7fd28c108260 2024-09-11T21:47:47.255 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:46.578+0000 7fd28b7fe640 1 -- 172.21.15.1:0/2307141668 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd280030850 con 0x7fd28c108260 2024-09-11T21:47:47.255 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:46.578+0000 7fd292c33640 1 -- 172.21.15.1:0/2307141668 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd28c108260 msgr2=0x7fd28c10a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:47:47.255 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:46.578+0000 7fd292c33640 1 --2- 172.21.15.1:0/2307141668 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd28c108260 0x7fd28c10a650 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fd280009820 tx=0x7fd28002a270 comp rx=0 tx=0).stop 2024-09-11T21:47:47.255 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:46.579+0000 7fd292c33640 1 -- 172.21.15.1:0/2307141668 shutdown_connections 2024-09-11T21:47:47.255 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:46.579+0000 7fd292c33640 1 --2- 172.21.15.1:0/2307141668 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd28c108260 0x7fd28c10a650 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:47:47.256 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:46.579+0000 7fd292c33640 1 -- 172.21.15.1:0/2307141668 >> 172.21.15.1:0/2307141668 conn(0x7fd28c101da0 msgr2=0x7fd28c1041e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:47:47.256 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:46.579+0000 7fd292c33640 1 -- 172.21.15.1:0/2307141668 shutdown_connections 2024-09-11T21:47:47.256 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:46.579+0000 7fd292c33640 1 -- 172.21.15.1:0/2307141668 wait complete. 2024-09-11T21:47:47.256 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:46.579+0000 7fd292c33640 1 Processor -- start 2024-09-11T21:47:47.256 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:46.579+0000 7fd292c33640 1 -- start start 2024-09-11T21:47:47.256 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:46.579+0000 7fd292c33640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd28c108260 0x7fd28c19b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:47:47.256 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:46.579+0000 7fd292c33640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd28c10bb10 con 0x7fd28c108260 2024-09-11T21:47:47.256 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:46.579+0000 7fd2909cf640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd28c108260 0x7fd28c19b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:47:47.256 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:46.580+0000 7fd2909cf640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd28c108260 0x7fd28c19b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36540/0 (socket says 172.21.15.1:36540) 2024-09-11T21:47:47.256 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:46.580+0000 7fd2909cf640 1 -- 172.21.15.1:0/2286311066 learned_addr learned my addr 172.21.15.1:0/2286311066 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:47:47.256 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:46.580+0000 7fd2909cf640 1 -- 172.21.15.1:0/2286311066 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd28c19bc60 con 0x7fd28c108260 2024-09-11T21:47:47.256 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:46.580+0000 7fd2909cf640 1 --2- 172.21.15.1:0/2286311066 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd28c108260 0x7fd28c19b720 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd2800097f0 tx=0x7fd280030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:47:47.257 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:46.580+0000 7fd289ffb640 1 -- 172.21.15.1:0/2286311066 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd280042070 con 0x7fd28c108260 2024-09-11T21:47:47.257 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:46.580+0000 7fd289ffb640 1 -- 172.21.15.1:0/2286311066 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd280032ca0 con 0x7fd28c108260 2024-09-11T21:47:47.257 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:46.580+0000 7fd292c33640 1 -- 172.21.15.1:0/2286311066 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd28c19bef0 con 0x7fd28c108260 2024-09-11T21:47:47.257 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:46.580+0000 7fd289ffb640 1 -- 172.21.15.1:0/2286311066 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd280037040 con 0x7fd28c108260 2024-09-11T21:47:47.257 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:46.580+0000 7fd292c33640 1 -- 172.21.15.1:0/2286311066 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd28c19ebe0 con 0x7fd28c108260 2024-09-11T21:47:47.257 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:46.581+0000 7fd289ffb640 1 -- 172.21.15.1:0/2286311066 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd280045070 con 0x7fd28c108260 2024-09-11T21:47:47.257 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:46.581+0000 7fd289ffb640 1 -- 172.21.15.1:0/2286311066 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd280047af0 con 0x7fd28c108260 2024-09-11T21:47:47.257 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:46.581+0000 7fd292c33640 1 -- 172.21.15.1:0/2286311066 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd28c1086c0 con 0x7fd28c108260 2024-09-11T21:47:47.257 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:46.585+0000 7fd289ffb640 1 -- 172.21.15.1:0/2286311066 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fd28003db10 con 0x7fd28c108260 2024-09-11T21:47:47.257 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:46.648+0000 7fd292c33640 1 -- 172.21.15.1:0/2286311066 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fd28c19f160 con 0x7fd28c108260 2024-09-11T21:47:47.257 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:46.649+0000 7fd289ffb640 1 -- 172.21.15.1:0/2286311066 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7fd28003d430 con 0x7fd28c108260 2024-09-11T21:47:47.257 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:46.649+0000 7fd292c33640 1 -- 172.21.15.1:0/2286311066 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd28c108260 msgr2=0x7fd28c19b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:47:47.258 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:46.649+0000 7fd292c33640 1 --2- 172.21.15.1:0/2286311066 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd28c108260 0x7fd28c19b720 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd2800097f0 tx=0x7fd280030ec0 comp rx=0 tx=0).stop 2024-09-11T21:47:47.258 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:46.649+0000 7fd292c33640 1 -- 172.21.15.1:0/2286311066 shutdown_connections 2024-09-11T21:47:47.258 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:46.650+0000 7fd292c33640 1 --2- 172.21.15.1:0/2286311066 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd28c108260 0x7fd28c19b720 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:47:47.258 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:46.650+0000 7fd292c33640 1 -- 172.21.15.1:0/2286311066 >> 172.21.15.1:0/2286311066 conn(0x7fd28c101da0 msgr2=0x7fd28c1029a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:47:47.258 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:46.650+0000 7fd292c33640 1 -- 172.21.15.1:0/2286311066 shutdown_connections 2024-09-11T21:47:47.258 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:46.650+0000 7fd292c33640 1 -- 172.21.15.1:0/2286311066 wait complete. 2024-09-11T21:47:47.258 INFO:teuthology.orchestra.run.smithi001.stdout:mgr not available, waiting (1/15)... 2024-09-11T21:47:48.193 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:47:47 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:47:47.735+0000 7fe51ce9a040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T21:47:48.193 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:47:47 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:47:47.805+0000 7fe51ce9a040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T21:47:48.849 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:47:48 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:47:48.570+0000 7fe51ce9a040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T21:47:48.849 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:47:48 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:47:48.695+0000 7fe51ce9a040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T21:47:49.143 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:47:48 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:47:48.847+0000 7fe51ce9a040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T21:47:49.143 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:47:48 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:47:48.912+0000 7fe51ce9a040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T21:47:49.143 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:47:49 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:47:49.070+0000 7fe51ce9a040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T21:47:49.144 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:47:49 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:47:49.142+0000 7fe51ce9a040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T21:47:49.443 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:47:49 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:47:49.205+0000 7fe51ce9a040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T21:47:49.443 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:47:49 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:47:49.377+0000 7fe51ce9a040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T21:47:49.700 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:47:49 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:47:49.446+0000 7fe51ce9a040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T21:47:49.700 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:47:49 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:47:49.519+0000 7fe51ce9a040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T21:47:49.700 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:47:49 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:47:49.607+0000 7fe51ce9a040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T21:47:49.700 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:47:49 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:47:49.672+0000 7fe51ce9a040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T21:47:50.080 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:49 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/4166422466' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-11T21:47:50.081 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:47:49 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:47:49.748+0000 7fe51ce9a040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T21:47:50.344 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-09-11T21:47:50.345 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout { 2024-09-11T21:47:50.345 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "fsid": "3ffca83c-7087-11ef-bcea-c7b262605968", 2024-09-11T21:47:50.345 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "health": { 2024-09-11T21:47:50.345 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-11T21:47:50.345 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-11T21:47:50.345 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-11T21:47:50.345 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-09-11T21:47:50.345 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-11T21:47:50.346 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-11T21:47:50.346 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 0 2024-09-11T21:47:50.346 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2024-09-11T21:47:50.346 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-11T21:47:50.346 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "a" 2024-09-11T21:47:50.346 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2024-09-11T21:47:50.346 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-09-11T21:47:50.346 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-11T21:47:50.346 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T21:47:50.346 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-11T21:47:50.346 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-11T21:47:50.346 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-09-11T21:47:50.346 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-11T21:47:50.346 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T21:47:50.347 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-11T21:47:50.347 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-11T21:47:50.347 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-11T21:47:50.354 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-11T21:47:50.354 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-11T21:47:50.354 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-11T21:47:50.354 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-09-11T21:47:50.354 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-11T21:47:50.354 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-11T21:47:50.355 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-11T21:47:50.355 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-11T21:47:50.355 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-11T21:47:50.355 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-11T21:47:50.355 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-11T21:47:50.355 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-11T21:47:50.355 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-11T21:47:50.355 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-09-11T21:47:50.355 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-11T21:47:50.355 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T21:47:50.355 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "btime": "2024-09-11T21:47:28:868551+0000", 2024-09-11T21:47:50.355 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-11T21:47:50.355 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-11T21:47:50.355 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-09-11T21:47:50.355 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-11T21:47:50.356 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-11T21:47:50.356 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-11T21:47:50.356 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-11T21:47:50.356 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-11T21:47:50.356 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-11T21:47:50.356 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "restful" 2024-09-11T21:47:50.356 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2024-09-11T21:47:50.356 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T21:47:50.356 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-09-11T21:47:50.356 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-11T21:47:50.356 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T21:47:50.356 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "modified": "2024-09-11T21:47:28.869189+0000", 2024-09-11T21:47:50.357 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T21:47:50.357 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-09-11T21:47:50.357 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-11T21:47:50.357 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout } 2024-09-11T21:47:50.357 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:49.699+0000 7f9223f7f640 1 Processor -- start 2024-09-11T21:47:50.357 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:49.699+0000 7f9223f7f640 1 -- start start 2024-09-11T21:47:50.357 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:49.699+0000 7f9223f7f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f921c105ff0 0x7f921c1063f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:47:50.357 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:49.699+0000 7f9223f7f640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f921c07be10 con 0x7f921c105ff0 2024-09-11T21:47:50.357 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:49.700+0000 7f9221d1b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f921c105ff0 0x7f921c1063f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:47:50.357 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:49.700+0000 7f9221d1b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f921c105ff0 0x7f921c1063f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36546/0 (socket says 172.21.15.1:36546) 2024-09-11T21:47:50.357 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:49.700+0000 7f9221d1b640 1 -- 172.21.15.1:0/446283047 learned_addr learned my addr 172.21.15.1:0/446283047 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:47:50.357 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:49.700+0000 7f9221d1b640 1 -- 172.21.15.1:0/446283047 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f921c106930 con 0x7f921c105ff0 2024-09-11T21:47:50.358 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:49.701+0000 7f9221d1b640 1 --2- 172.21.15.1:0/446283047 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f921c105ff0 0x7f921c1063f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9218009820 tx=0x7f921802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76910a93fe588ca5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:47:50.358 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:49.701+0000 7f9220d19640 1 -- 172.21.15.1:0/446283047 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9218037070 con 0x7f921c105ff0 2024-09-11T21:47:50.358 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:49.701+0000 7f9220d19640 1 -- 172.21.15.1:0/446283047 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9218032900 con 0x7f921c105ff0 2024-09-11T21:47:50.358 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:49.701+0000 7f9220d19640 1 -- 172.21.15.1:0/446283047 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9218030850 con 0x7f921c105ff0 2024-09-11T21:47:50.358 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:49.702+0000 7f9223f7f640 1 -- 172.21.15.1:0/446283047 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f921c105ff0 msgr2=0x7f921c1063f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:47:50.358 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:49.702+0000 7f9223f7f640 1 --2- 172.21.15.1:0/446283047 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f921c105ff0 0x7f921c1063f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9218009820 tx=0x7f921802a270 comp rx=0 tx=0).stop 2024-09-11T21:47:50.358 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:49.702+0000 7f9223f7f640 1 -- 172.21.15.1:0/446283047 shutdown_connections 2024-09-11T21:47:50.358 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:49.702+0000 7f9223f7f640 1 --2- 172.21.15.1:0/446283047 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f921c105ff0 0x7f921c1063f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:47:50.358 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:49.702+0000 7f9223f7f640 1 -- 172.21.15.1:0/446283047 >> 172.21.15.1:0/446283047 conn(0x7f921c101e20 msgr2=0x7f921c104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:47:50.358 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:49.702+0000 7f9223f7f640 1 -- 172.21.15.1:0/446283047 shutdown_connections 2024-09-11T21:47:50.358 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:49.702+0000 7f9223f7f640 1 -- 172.21.15.1:0/446283047 wait complete. 2024-09-11T21:47:50.358 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:49.703+0000 7f9223f7f640 1 Processor -- start 2024-09-11T21:47:50.359 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:49.703+0000 7f9223f7f640 1 -- start start 2024-09-11T21:47:50.359 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:49.703+0000 7f9223f7f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f921c105ff0 0x7f921c19d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:47:50.359 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:49.703+0000 7f9223f7f640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f921c107a30 con 0x7f921c105ff0 2024-09-11T21:47:50.359 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:49.703+0000 7f9221d1b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f921c105ff0 0x7f921c19d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:47:50.359 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:49.703+0000 7f9221d1b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f921c105ff0 0x7f921c19d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36554/0 (socket says 172.21.15.1:36554) 2024-09-11T21:47:50.359 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:49.703+0000 7f9221d1b640 1 -- 172.21.15.1:0/4166422466 learned_addr learned my addr 172.21.15.1:0/4166422466 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:47:50.359 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:49.703+0000 7f9221d1b640 1 -- 172.21.15.1:0/4166422466 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f921c19de00 con 0x7f921c105ff0 2024-09-11T21:47:50.359 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:49.704+0000 7f9221d1b640 1 --2- 172.21.15.1:0/4166422466 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f921c105ff0 0x7f921c19d8c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9218031000 tx=0x7f92180326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:47:50.359 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:49.704+0000 7f9212ffd640 1 -- 172.21.15.1:0/4166422466 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9218042070 con 0x7f921c105ff0 2024-09-11T21:47:50.359 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:49.704+0000 7f9212ffd640 1 -- 172.21.15.1:0/4166422466 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9218032be0 con 0x7f921c105ff0 2024-09-11T21:47:50.359 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:49.704+0000 7f9223f7f640 1 -- 172.21.15.1:0/4166422466 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f921c19e090 con 0x7f921c105ff0 2024-09-11T21:47:50.359 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:49.704+0000 7f9223f7f640 1 -- 172.21.15.1:0/4166422466 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f921c1a0d80 con 0x7f921c105ff0 2024-09-11T21:47:50.359 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:49.704+0000 7f9212ffd640 1 -- 172.21.15.1:0/4166422466 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9218037040 con 0x7f921c105ff0 2024-09-11T21:47:50.359 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:49.704+0000 7f9212ffd640 1 -- 172.21.15.1:0/4166422466 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9218045070 con 0x7f921c105ff0 2024-09-11T21:47:50.360 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:49.704+0000 7f9212ffd640 1 -- 172.21.15.1:0/4166422466 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9218047b40 con 0x7f921c105ff0 2024-09-11T21:47:50.360 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:49.705+0000 7f9223f7f640 1 -- 172.21.15.1:0/4166422466 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92000052c0 con 0x7f921c105ff0 2024-09-11T21:47:50.360 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:49.707+0000 7f9212ffd640 1 -- 172.21.15.1:0/4166422466 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f921803dc40 con 0x7f921c105ff0 2024-09-11T21:47:50.360 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:49.784+0000 7f9223f7f640 1 -- 172.21.15.1:0/4166422466 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f9200005880 con 0x7f921c105ff0 2024-09-11T21:47:50.360 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:49.784+0000 7f9212ffd640 1 -- 172.21.15.1:0/4166422466 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f9218032d90 con 0x7f921c105ff0 2024-09-11T21:47:50.360 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:49.785+0000 7f9223f7f640 1 -- 172.21.15.1:0/4166422466 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f921c105ff0 msgr2=0x7f921c19d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:47:50.360 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:49.785+0000 7f9223f7f640 1 --2- 172.21.15.1:0/4166422466 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f921c105ff0 0x7f921c19d8c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9218031000 tx=0x7f92180326e0 comp rx=0 tx=0).stop 2024-09-11T21:47:50.360 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:49.785+0000 7f9223f7f640 1 -- 172.21.15.1:0/4166422466 shutdown_connections 2024-09-11T21:47:50.360 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:49.785+0000 7f9223f7f640 1 --2- 172.21.15.1:0/4166422466 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f921c105ff0 0x7f921c19d8c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:47:50.360 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:49.785+0000 7f9223f7f640 1 -- 172.21.15.1:0/4166422466 >> 172.21.15.1:0/4166422466 conn(0x7f921c101e20 msgr2=0x7f921c1945a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:47:50.360 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:49.786+0000 7f9223f7f640 1 -- 172.21.15.1:0/4166422466 shutdown_connections 2024-09-11T21:47:50.360 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:49.786+0000 7f9223f7f640 1 -- 172.21.15.1:0/4166422466 wait complete. 2024-09-11T21:47:50.360 INFO:teuthology.orchestra.run.smithi001.stdout:mgr not available, waiting (2/15)... 2024-09-11T21:47:50.693 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:47:50 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:47:50.587+0000 7fe51ce9a040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T21:47:51.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:50 smithi001 ceph-mon[30935]: Activating manager daemon x 2024-09-11T21:47:51.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:50 smithi001 ceph-mon[30935]: mgrmap e2: x(active, starting, since 0.00333246s) 2024-09-11T21:47:51.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:50 smithi001 ceph-mon[30935]: from='mgr.14100 172.21.15.1:0/4142873155' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T21:47:51.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:50 smithi001 ceph-mon[30935]: from='mgr.14100 172.21.15.1:0/4142873155' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T21:47:51.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:50 smithi001 ceph-mon[30935]: from='mgr.14100 172.21.15.1:0/4142873155' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T21:47:51.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:50 smithi001 ceph-mon[30935]: from='mgr.14100 172.21.15.1:0/4142873155' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T21:47:51.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:50 smithi001 ceph-mon[30935]: from='mgr.14100 172.21.15.1:0/4142873155' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T21:47:51.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:50 smithi001 ceph-mon[30935]: Manager daemon x is now available 2024-09-11T21:47:51.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:50 smithi001 ceph-mon[30935]: from='mgr.14100 172.21.15.1:0/4142873155' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-11T21:47:51.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:50 smithi001 ceph-mon[30935]: from='mgr.14100 172.21.15.1:0/4142873155' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-11T21:47:51.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:50 smithi001 ceph-mon[30935]: from='mgr.14100 172.21.15.1:0/4142873155' entity='mgr.x' 2024-09-11T21:47:51.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:50 smithi001 ceph-mon[30935]: from='mgr.14100 172.21.15.1:0/4142873155' entity='mgr.x' 2024-09-11T21:47:51.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:50 smithi001 ceph-mon[30935]: from='mgr.14100 172.21.15.1:0/4142873155' entity='mgr.x' 2024-09-11T21:47:52.751 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:52 smithi001 ceph-mon[30935]: mgrmap e3: x(active, since 1.00751s) 2024-09-11T21:47:53.551 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-09-11T21:47:53.551 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout { 2024-09-11T21:47:53.551 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "fsid": "3ffca83c-7087-11ef-bcea-c7b262605968", 2024-09-11T21:47:53.551 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "health": { 2024-09-11T21:47:53.551 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-11T21:47:53.552 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-11T21:47:53.552 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-11T21:47:53.552 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-09-11T21:47:53.552 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-11T21:47:53.552 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-11T21:47:53.552 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 0 2024-09-11T21:47:53.552 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2024-09-11T21:47:53.552 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-11T21:47:53.552 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "a" 2024-09-11T21:47:53.552 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2024-09-11T21:47:53.552 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-09-11T21:47:53.552 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-11T21:47:53.553 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T21:47:53.553 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-11T21:47:53.553 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-11T21:47:53.553 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-09-11T21:47:53.553 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-11T21:47:53.553 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T21:47:53.553 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-11T21:47:53.553 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-11T21:47:53.553 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-11T21:47:53.553 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-11T21:47:53.553 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-11T21:47:53.553 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-11T21:47:53.553 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-09-11T21:47:53.554 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-11T21:47:53.554 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-11T21:47:53.554 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-11T21:47:53.554 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-11T21:47:53.554 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-11T21:47:53.554 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-11T21:47:53.554 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-11T21:47:53.554 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-11T21:47:53.554 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-11T21:47:53.554 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-09-11T21:47:53.554 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-11T21:47:53.554 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T21:47:53.555 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "btime": "2024-09-11T21:47:28:868551+0000", 2024-09-11T21:47:53.555 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-11T21:47:53.555 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-11T21:47:53.555 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-09-11T21:47:53.555 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-11T21:47:53.555 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-11T21:47:53.555 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-11T21:47:53.555 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-11T21:47:53.555 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-11T21:47:53.555 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-11T21:47:53.555 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "restful" 2024-09-11T21:47:53.555 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2024-09-11T21:47:53.556 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T21:47:53.556 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-09-11T21:47:53.556 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-11T21:47:53.556 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T21:47:53.556 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "modified": "2024-09-11T21:47:28.869189+0000", 2024-09-11T21:47:53.556 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T21:47:53.556 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-09-11T21:47:53.556 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-11T21:47:53.556 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout } 2024-09-11T21:47:53.556 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:52.753+0000 7f1b19da4640 1 Processor -- start 2024-09-11T21:47:53.556 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:52.753+0000 7f1b19da4640 1 -- start start 2024-09-11T21:47:53.556 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:52.753+0000 7f1b19da4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b14107720 0x7f1b14107b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:47:53.557 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:52.753+0000 7f1b19da4640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1b141080f0 con 0x7f1b14107720 2024-09-11T21:47:53.557 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:52.753+0000 7f1b137fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b14107720 0x7f1b14107b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:47:53.557 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:52.753+0000 7f1b137fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b14107720 0x7f1b14107b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36642/0 (socket says 172.21.15.1:36642) 2024-09-11T21:47:53.557 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:52.753+0000 7f1b137fe640 1 -- 172.21.15.1:0/3913308240 learned_addr learned my addr 172.21.15.1:0/3913308240 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:47:53.557 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:52.754+0000 7f1b137fe640 1 -- 172.21.15.1:0/3913308240 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b141088d0 con 0x7f1b14107720 2024-09-11T21:47:53.557 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:52.754+0000 7f1b137fe640 1 --2- 172.21.15.1:0/3913308240 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b14107720 0x7f1b14107b20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1b04009820 tx=0x7f1b0402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8cd8d621ac6cf14 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:47:53.557 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:52.754+0000 7f1b127fc640 1 -- 172.21.15.1:0/3913308240 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b04037070 con 0x7f1b14107720 2024-09-11T21:47:53.557 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:52.754+0000 7f1b127fc640 1 -- 172.21.15.1:0/3913308240 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1b04032900 con 0x7f1b14107720 2024-09-11T21:47:53.557 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:52.754+0000 7f1b127fc640 1 -- 172.21.15.1:0/3913308240 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b04030850 con 0x7f1b14107720 2024-09-11T21:47:53.557 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:52.755+0000 7f1b19da4640 1 -- 172.21.15.1:0/3913308240 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b14107720 msgr2=0x7f1b14107b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:47:53.557 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:52.755+0000 7f1b19da4640 1 --2- 172.21.15.1:0/3913308240 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b14107720 0x7f1b14107b20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1b04009820 tx=0x7f1b0402a270 comp rx=0 tx=0).stop 2024-09-11T21:47:53.558 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:52.755+0000 7f1b19da4640 1 -- 172.21.15.1:0/3913308240 shutdown_connections 2024-09-11T21:47:53.558 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:52.755+0000 7f1b19da4640 1 --2- 172.21.15.1:0/3913308240 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b14107720 0x7f1b14107b20 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:47:53.558 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:52.755+0000 7f1b19da4640 1 -- 172.21.15.1:0/3913308240 >> 172.21.15.1:0/3913308240 conn(0x7f1b14102ef0 msgr2=0x7f1b14105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:47:53.558 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:52.755+0000 7f1b19da4640 1 -- 172.21.15.1:0/3913308240 shutdown_connections 2024-09-11T21:47:53.558 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:52.755+0000 7f1b19da4640 1 -- 172.21.15.1:0/3913308240 wait complete. 2024-09-11T21:47:53.558 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:52.756+0000 7f1b19da4640 1 Processor -- start 2024-09-11T21:47:53.558 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:52.756+0000 7f1b19da4640 1 -- start start 2024-09-11T21:47:53.558 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:52.756+0000 7f1b19da4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b14107720 0x7f1b1419b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:47:53.558 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:52.756+0000 7f1b19da4640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1b1410bc80 con 0x7f1b14107720 2024-09-11T21:47:53.558 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:52.756+0000 7f1b137fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b14107720 0x7f1b1419b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:47:53.558 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:52.756+0000 7f1b137fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b14107720 0x7f1b1419b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36652/0 (socket says 172.21.15.1:36652) 2024-09-11T21:47:53.558 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:52.756+0000 7f1b137fe640 1 -- 172.21.15.1:0/2720566402 learned_addr learned my addr 172.21.15.1:0/2720566402 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:47:53.559 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:52.757+0000 7f1b137fe640 1 -- 172.21.15.1:0/2720566402 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b1419bc00 con 0x7f1b14107720 2024-09-11T21:47:53.559 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:52.757+0000 7f1b137fe640 1 --2- 172.21.15.1:0/2720566402 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b14107720 0x7f1b1419b6c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1b040097f0 tx=0x7f1b04030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:47:53.559 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:52.757+0000 7f1b10ff9640 1 -- 172.21.15.1:0/2720566402 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b04037040 con 0x7f1b14107720 2024-09-11T21:47:53.559 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:52.757+0000 7f1b19da4640 1 -- 172.21.15.1:0/2720566402 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b1419be90 con 0x7f1b14107720 2024-09-11T21:47:53.559 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:52.757+0000 7f1b10ff9640 1 -- 172.21.15.1:0/2720566402 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1b04040070 con 0x7f1b14107720 2024-09-11T21:47:53.559 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:52.757+0000 7f1b10ff9640 1 -- 172.21.15.1:0/2720566402 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b0403a390 con 0x7f1b14107720 2024-09-11T21:47:53.559 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:52.757+0000 7f1b19da4640 1 -- 172.21.15.1:0/2720566402 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b1419eb80 con 0x7f1b14107720 2024-09-11T21:47:53.559 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:52.758+0000 7f1b10ff9640 1 -- 172.21.15.1:0/2720566402 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f1b0403a630 con 0x7f1b14107720 2024-09-11T21:47:53.559 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:52.758+0000 7f1b19da4640 1 -- 172.21.15.1:0/2720566402 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b14107ba0 con 0x7f1b14107720 2024-09-11T21:47:53.559 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:52.758+0000 7f1b10ff9640 1 --2- 172.21.15.1:0/2720566402 >> [v2:172.21.15.1:6800/2311309446,v1:172.21.15.1:6801/2311309446] conn(0x7f1afc038f20 0x7f1afc03b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:47:53.559 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:52.758+0000 7f1b10ff9640 1 -- 172.21.15.1:0/2720566402 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1b0406c100 con 0x7f1b14107720 2024-09-11T21:47:53.559 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:52.758+0000 7f1b12ffd640 1 --2- 172.21.15.1:0/2720566402 >> [v2:172.21.15.1:6800/2311309446,v1:172.21.15.1:6801/2311309446] conn(0x7f1afc038f20 0x7f1afc03b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:47:53.560 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:52.759+0000 7f1b12ffd640 1 --2- 172.21.15.1:0/2720566402 >> [v2:172.21.15.1:6800/2311309446,v1:172.21.15.1:6801/2311309446] conn(0x7f1afc038f20 0x7f1afc03b3e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1b080098c0 tx=0x7f1b08006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:47:53.560 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:52.764+0000 7f1b10ff9640 1 -- 172.21.15.1:0/2720566402 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1b0406be60 con 0x7f1b14107720 2024-09-11T21:47:53.560 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:53.008+0000 7f1b19da4640 1 -- 172.21.15.1:0/2720566402 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f1b1419c310 con 0x7f1b14107720 2024-09-11T21:47:53.560 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:53.009+0000 7f1b10ff9640 1 -- 172.21.15.1:0/2720566402 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1290 (secure 0 0 0) 0x7f1b04043760 con 0x7f1b14107720 2024-09-11T21:47:53.560 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:53.011+0000 7f1b19da4640 1 -- 172.21.15.1:0/2720566402 >> [v2:172.21.15.1:6800/2311309446,v1:172.21.15.1:6801/2311309446] conn(0x7f1afc038f20 msgr2=0x7f1afc03b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:47:53.560 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:53.011+0000 7f1b19da4640 1 --2- 172.21.15.1:0/2720566402 >> [v2:172.21.15.1:6800/2311309446,v1:172.21.15.1:6801/2311309446] conn(0x7f1afc038f20 0x7f1afc03b3e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1b080098c0 tx=0x7f1b08006db0 comp rx=0 tx=0).stop 2024-09-11T21:47:53.560 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:53.011+0000 7f1b19da4640 1 -- 172.21.15.1:0/2720566402 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b14107720 msgr2=0x7f1b1419b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:47:53.560 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:53.011+0000 7f1b19da4640 1 --2- 172.21.15.1:0/2720566402 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b14107720 0x7f1b1419b6c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1b040097f0 tx=0x7f1b04030d80 comp rx=0 tx=0).stop 2024-09-11T21:47:53.560 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:53.011+0000 7f1b19da4640 1 -- 172.21.15.1:0/2720566402 shutdown_connections 2024-09-11T21:47:53.560 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:53.011+0000 7f1b19da4640 1 --2- 172.21.15.1:0/2720566402 >> [v2:172.21.15.1:6800/2311309446,v1:172.21.15.1:6801/2311309446] conn(0x7f1afc038f20 0x7f1afc03b3e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:47:53.561 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:53.011+0000 7f1b19da4640 1 --2- 172.21.15.1:0/2720566402 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b14107720 0x7f1b1419b6c0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:47:53.561 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:53.011+0000 7f1b19da4640 1 -- 172.21.15.1:0/2720566402 >> 172.21.15.1:0/2720566402 conn(0x7f1b14102ef0 msgr2=0x7f1b14103a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:47:53.561 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:53.011+0000 7f1b19da4640 1 -- 172.21.15.1:0/2720566402 shutdown_connections 2024-09-11T21:47:53.561 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:53.011+0000 7f1b19da4640 1 -- 172.21.15.1:0/2720566402 wait complete. 2024-09-11T21:47:53.561 INFO:teuthology.orchestra.run.smithi001.stdout:mgr is available 2024-09-11T21:47:53.858 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:53 smithi001 ceph-mon[30935]: mgrmap e4: x(active, since 2s) 2024-09-11T21:47:53.858 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:53 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/2720566402' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-11T21:47:54.645 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-09-11T21:47:54.645 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout [global] 2024-09-11T21:47:54.645 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout fsid = 3ffca83c-7087-11ef-bcea-c7b262605968 2024-09-11T21:47:54.646 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.1:3300,v1:172.21.15.1:6789] 2024-09-11T21:47:54.646 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-11T21:47:54.646 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-11T21:47:54.646 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-11T21:47:54.646 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-11T21:47:54.646 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-09-11T21:47:54.646 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-11T21:47:54.646 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-11T21:47:54.646 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-09-11T21:47:54.646 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout [osd] 2024-09-11T21:47:54.646 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-11T21:47:54.646 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-11T21:47:54.646 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-11T21:47:54.647 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-11T21:47:54.647 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-11T21:47:54.647 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:53.936+0000 7f6e45310640 1 Processor -- start 2024-09-11T21:47:54.647 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:53.936+0000 7f6e45310640 1 -- start start 2024-09-11T21:47:54.647 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:53.936+0000 7f6e45310640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e401076f0 0x7f6e40107af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:47:54.647 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:53.936+0000 7f6e45310640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6e401080c0 con 0x7f6e401076f0 2024-09-11T21:47:54.647 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:53.936+0000 7f6e3effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e401076f0 0x7f6e40107af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:47:54.647 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:53.936+0000 7f6e3effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e401076f0 0x7f6e40107af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36666/0 (socket says 172.21.15.1:36666) 2024-09-11T21:47:54.647 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:53.936+0000 7f6e3effd640 1 -- 172.21.15.1:0/49921507 learned_addr learned my addr 172.21.15.1:0/49921507 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:47:54.647 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:53.937+0000 7f6e3effd640 1 -- 172.21.15.1:0/49921507 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e401088f0 con 0x7f6e401076f0 2024-09-11T21:47:54.647 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:53.937+0000 7f6e3effd640 1 --2- 172.21.15.1:0/49921507 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e401076f0 0x7f6e40107af0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6e30009820 tx=0x7f6e3002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2f2e894873f03c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:47:54.647 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:53.937+0000 7f6e3e7fc640 1 -- 172.21.15.1:0/49921507 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e30037070 con 0x7f6e401076f0 2024-09-11T21:47:54.648 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:53.938+0000 7f6e3e7fc640 1 -- 172.21.15.1:0/49921507 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6e30032900 con 0x7f6e401076f0 2024-09-11T21:47:54.648 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:53.938+0000 7f6e3e7fc640 1 -- 172.21.15.1:0/49921507 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e30030850 con 0x7f6e401076f0 2024-09-11T21:47:54.648 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:53.938+0000 7f6e45310640 1 -- 172.21.15.1:0/49921507 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e401076f0 msgr2=0x7f6e40107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:47:54.648 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:53.938+0000 7f6e45310640 1 --2- 172.21.15.1:0/49921507 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e401076f0 0x7f6e40107af0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6e30009820 tx=0x7f6e3002a270 comp rx=0 tx=0).stop 2024-09-11T21:47:54.648 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:53.938+0000 7f6e45310640 1 -- 172.21.15.1:0/49921507 shutdown_connections 2024-09-11T21:47:54.648 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:53.938+0000 7f6e45310640 1 --2- 172.21.15.1:0/49921507 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e401076f0 0x7f6e40107af0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:47:54.648 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:53.938+0000 7f6e45310640 1 -- 172.21.15.1:0/49921507 >> 172.21.15.1:0/49921507 conn(0x7f6e40102ec0 msgr2=0x7f6e401052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:47:54.648 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:53.938+0000 7f6e45310640 1 -- 172.21.15.1:0/49921507 shutdown_connections 2024-09-11T21:47:54.648 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:53.938+0000 7f6e45310640 1 -- 172.21.15.1:0/49921507 wait complete. 2024-09-11T21:47:54.648 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:53.939+0000 7f6e45310640 1 Processor -- start 2024-09-11T21:47:54.648 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:53.939+0000 7f6e45310640 1 -- start start 2024-09-11T21:47:54.648 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:53.939+0000 7f6e45310640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e401076f0 0x7f6e4019b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:47:54.649 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:53.939+0000 7f6e45310640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6e4010bca0 con 0x7f6e401076f0 2024-09-11T21:47:54.649 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:53.939+0000 7f6e3effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e401076f0 0x7f6e4019b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:47:54.649 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:53.939+0000 7f6e3effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e401076f0 0x7f6e4019b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36672/0 (socket says 172.21.15.1:36672) 2024-09-11T21:47:54.649 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:53.939+0000 7f6e3effd640 1 -- 172.21.15.1:0/3967614927 learned_addr learned my addr 172.21.15.1:0/3967614927 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:47:54.649 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:53.939+0000 7f6e3effd640 1 -- 172.21.15.1:0/3967614927 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e4019bc20 con 0x7f6e401076f0 2024-09-11T21:47:54.649 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:53.939+0000 7f6e3effd640 1 --2- 172.21.15.1:0/3967614927 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e401076f0 0x7f6e4019b6e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6e300097f0 tx=0x7f6e30030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:47:54.649 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:53.940+0000 7f6e3cff9640 1 -- 172.21.15.1:0/3967614927 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e30037040 con 0x7f6e401076f0 2024-09-11T21:47:54.649 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:53.940+0000 7f6e45310640 1 -- 172.21.15.1:0/3967614927 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e4019beb0 con 0x7f6e401076f0 2024-09-11T21:47:54.649 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:53.940+0000 7f6e3cff9640 1 -- 172.21.15.1:0/3967614927 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6e30040070 con 0x7f6e401076f0 2024-09-11T21:47:54.649 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:53.940+0000 7f6e3cff9640 1 -- 172.21.15.1:0/3967614927 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e3003a390 con 0x7f6e401076f0 2024-09-11T21:47:54.649 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:53.940+0000 7f6e45310640 1 -- 172.21.15.1:0/3967614927 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e4019eba0 con 0x7f6e401076f0 2024-09-11T21:47:54.649 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:53.940+0000 7f6e3cff9640 1 -- 172.21.15.1:0/3967614927 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f6e3003a630 con 0x7f6e401076f0 2024-09-11T21:47:54.649 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:53.941+0000 7f6e3cff9640 1 --2- 172.21.15.1:0/3967614927 >> [v2:172.21.15.1:6800/2311309446,v1:172.21.15.1:6801/2311309446] conn(0x7f6e2c038f20 0x7f6e2c03b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:47:54.650 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:53.941+0000 7f6e3cff9640 1 -- 172.21.15.1:0/3967614927 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6e3006c0c0 con 0x7f6e401076f0 2024-09-11T21:47:54.650 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:53.941+0000 7f6e45310640 1 -- 172.21.15.1:0/3967614927 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e40107b70 con 0x7f6e401076f0 2024-09-11T21:47:54.650 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:53.941+0000 7f6e37fff640 1 --2- 172.21.15.1:0/3967614927 >> [v2:172.21.15.1:6800/2311309446,v1:172.21.15.1:6801/2311309446] conn(0x7f6e2c038f20 0x7f6e2c03b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:47:54.650 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:53.941+0000 7f6e37fff640 1 --2- 172.21.15.1:0/3967614927 >> [v2:172.21.15.1:6800/2311309446,v1:172.21.15.1:6801/2311309446] conn(0x7f6e2c038f20 0x7f6e2c03b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6e280098c0 tx=0x7f6e28006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:47:54.650 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:53.945+0000 7f6e3cff9640 1 -- 172.21.15.1:0/3967614927 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6e30042310 con 0x7f6e401076f0 2024-09-11T21:47:54.650 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:54.120+0000 7f6e45310640 1 -- 172.21.15.1:0/3967614927 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f6e4019c330 con 0x7f6e401076f0 2024-09-11T21:47:54.650 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:54.121+0000 7f6e3cff9640 1 -- 172.21.15.1:0/3967614927 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+427 (secure 0 0 0) 0x7f6e300371e0 con 0x7f6e401076f0 2024-09-11T21:47:54.650 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:54.124+0000 7f6e45310640 1 -- 172.21.15.1:0/3967614927 >> [v2:172.21.15.1:6800/2311309446,v1:172.21.15.1:6801/2311309446] conn(0x7f6e2c038f20 msgr2=0x7f6e2c03b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:47:54.650 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:54.124+0000 7f6e45310640 1 --2- 172.21.15.1:0/3967614927 >> [v2:172.21.15.1:6800/2311309446,v1:172.21.15.1:6801/2311309446] conn(0x7f6e2c038f20 0x7f6e2c03b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6e280098c0 tx=0x7f6e28006db0 comp rx=0 tx=0).stop 2024-09-11T21:47:54.650 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:54.124+0000 7f6e45310640 1 -- 172.21.15.1:0/3967614927 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e401076f0 msgr2=0x7f6e4019b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:47:54.650 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:54.124+0000 7f6e45310640 1 --2- 172.21.15.1:0/3967614927 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e401076f0 0x7f6e4019b6e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6e300097f0 tx=0x7f6e30030d80 comp rx=0 tx=0).stop 2024-09-11T21:47:54.650 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:54.124+0000 7f6e45310640 1 -- 172.21.15.1:0/3967614927 shutdown_connections 2024-09-11T21:47:54.651 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:54.124+0000 7f6e45310640 1 --2- 172.21.15.1:0/3967614927 >> [v2:172.21.15.1:6800/2311309446,v1:172.21.15.1:6801/2311309446] conn(0x7f6e2c038f20 0x7f6e2c03b3e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:47:54.651 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:54.124+0000 7f6e45310640 1 --2- 172.21.15.1:0/3967614927 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e401076f0 0x7f6e4019b6e0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:47:54.651 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:54.124+0000 7f6e45310640 1 -- 172.21.15.1:0/3967614927 >> 172.21.15.1:0/3967614927 conn(0x7f6e40102ec0 msgr2=0x7f6e401039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:47:54.651 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:54.124+0000 7f6e45310640 1 -- 172.21.15.1:0/3967614927 shutdown_connections 2024-09-11T21:47:54.651 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:54.124+0000 7f6e45310640 1 -- 172.21.15.1:0/3967614927 wait complete. 2024-09-11T21:47:54.651 INFO:teuthology.orchestra.run.smithi001.stdout:Enabling cephadm module... 2024-09-11T21:47:54.863 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:54 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/3967614927' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-11T21:47:55.900 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:55 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/3375260393' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-11T21:47:55.900 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:47:55 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: ignoring --setuser ceph since I am not root 2024-09-11T21:47:55.901 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:47:55 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: ignoring --setgroup ceph since I am not root 2024-09-11T21:47:55.901 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:47:55 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:47:55.744+0000 7f420be6b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T21:47:55.901 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:47:55 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:47:55.803+0000 7f420be6b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T21:47:56.150 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:55.037+0000 7fc4f18db640 1 Processor -- start 2024-09-11T21:47:56.150 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:55.037+0000 7fc4f18db640 1 -- start start 2024-09-11T21:47:56.150 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:55.037+0000 7fc4f18db640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4ec108160 0x7fc4ec10a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:47:56.150 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:55.037+0000 7fc4f18db640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ec07beb0 con 0x7fc4ec108160 2024-09-11T21:47:56.151 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:55.038+0000 7fc4eaffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4ec108160 0x7fc4ec10a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:47:56.151 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:55.038+0000 7fc4eaffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4ec108160 0x7fc4ec10a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36684/0 (socket says 172.21.15.1:36684) 2024-09-11T21:47:56.151 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:55.038+0000 7fc4eaffd640 1 -- 172.21.15.1:0/3681531256 learned_addr learned my addr 172.21.15.1:0/3681531256 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:47:56.151 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:55.038+0000 7fc4eaffd640 1 -- 172.21.15.1:0/3681531256 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4ec10aa90 con 0x7fc4ec108160 2024-09-11T21:47:56.151 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:55.039+0000 7fc4eaffd640 1 --2- 172.21.15.1:0/3681531256 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4ec108160 0x7fc4ec10a550 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc009e80 tx=0x7fc4dc02a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81051a4de7f192c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:47:56.151 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:55.039+0000 7fc4e9ffb640 1 -- 172.21.15.1:0/3681531256 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4dc036070 con 0x7fc4ec108160 2024-09-11T21:47:56.151 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:55.039+0000 7fc4e9ffb640 1 -- 172.21.15.1:0/3681531256 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc4dc0318c0 con 0x7fc4ec108160 2024-09-11T21:47:56.151 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:55.039+0000 7fc4e9ffb640 1 -- 172.21.15.1:0/3681531256 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4dc007ab0 con 0x7fc4ec108160 2024-09-11T21:47:56.151 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:55.039+0000 7fc4f18db640 1 -- 172.21.15.1:0/3681531256 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4ec108160 msgr2=0x7fc4ec10a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:47:56.151 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:55.039+0000 7fc4f18db640 1 --2- 172.21.15.1:0/3681531256 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4ec108160 0x7fc4ec10a550 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc009e80 tx=0x7fc4dc02a7a0 comp rx=0 tx=0).stop 2024-09-11T21:47:56.151 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:55.040+0000 7fc4f18db640 1 -- 172.21.15.1:0/3681531256 shutdown_connections 2024-09-11T21:47:56.152 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:55.040+0000 7fc4f18db640 1 --2- 172.21.15.1:0/3681531256 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4ec108160 0x7fc4ec10a550 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:47:56.152 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:55.040+0000 7fc4f18db640 1 -- 172.21.15.1:0/3681531256 >> 172.21.15.1:0/3681531256 conn(0x7fc4ec101e20 msgr2=0x7fc4ec104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:47:56.152 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:55.040+0000 7fc4f18db640 1 -- 172.21.15.1:0/3681531256 shutdown_connections 2024-09-11T21:47:56.152 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:55.040+0000 7fc4f18db640 1 -- 172.21.15.1:0/3681531256 wait complete. 2024-09-11T21:47:56.152 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:55.040+0000 7fc4f18db640 1 Processor -- start 2024-09-11T21:47:56.152 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:55.040+0000 7fc4f18db640 1 -- start start 2024-09-11T21:47:56.152 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:55.041+0000 7fc4f18db640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4ec108160 0x7fc4ec19b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:47:56.152 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:55.041+0000 7fc4f18db640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ec10bb30 con 0x7fc4ec108160 2024-09-11T21:47:56.152 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:55.041+0000 7fc4eaffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4ec108160 0x7fc4ec19b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:47:56.152 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:55.041+0000 7fc4eaffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4ec108160 0x7fc4ec19b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36692/0 (socket says 172.21.15.1:36692) 2024-09-11T21:47:56.152 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:55.041+0000 7fc4eaffd640 1 -- 172.21.15.1:0/3375260393 learned_addr learned my addr 172.21.15.1:0/3375260393 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:47:56.152 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:55.041+0000 7fc4eaffd640 1 -- 172.21.15.1:0/3375260393 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4ec19bc20 con 0x7fc4ec108160 2024-09-11T21:47:56.152 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:55.041+0000 7fc4eaffd640 1 --2- 172.21.15.1:0/3375260393 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4ec108160 0x7fc4ec19b6e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc02acc0 tx=0x7fc4dc031ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:47:56.153 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:55.041+0000 7fc4f08d9640 1 -- 172.21.15.1:0/3375260393 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4dc036050 con 0x7fc4ec108160 2024-09-11T21:47:56.153 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:55.041+0000 7fc4f08d9640 1 -- 172.21.15.1:0/3375260393 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc4dc037040 con 0x7fc4ec108160 2024-09-11T21:47:56.153 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:55.041+0000 7fc4f08d9640 1 -- 172.21.15.1:0/3375260393 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4dc03d920 con 0x7fc4ec108160 2024-09-11T21:47:56.153 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:55.041+0000 7fc4f18db640 1 -- 172.21.15.1:0/3375260393 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4ec19beb0 con 0x7fc4ec108160 2024-09-11T21:47:56.153 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:55.041+0000 7fc4f18db640 1 -- 172.21.15.1:0/3375260393 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4ec19eba0 con 0x7fc4ec108160 2024-09-11T21:47:56.153 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:55.042+0000 7fc4f08d9640 1 -- 172.21.15.1:0/3375260393 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7fc4dc044020 con 0x7fc4ec108160 2024-09-11T21:47:56.153 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:55.042+0000 7fc4f18db640 1 -- 172.21.15.1:0/3375260393 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4ec1085c0 con 0x7fc4ec108160 2024-09-11T21:47:56.153 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:55.043+0000 7fc4f08d9640 1 --2- 172.21.15.1:0/3375260393 >> [v2:172.21.15.1:6800/2311309446,v1:172.21.15.1:6801/2311309446] conn(0x7fc4d8038f70 0x7fc4d803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:47:56.153 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:55.043+0000 7fc4f08d9640 1 -- 172.21.15.1:0/3375260393 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc4dc042020 con 0x7fc4ec108160 2024-09-11T21:47:56.153 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:55.043+0000 7fc4ea7fc640 1 --2- 172.21.15.1:0/3375260393 >> [v2:172.21.15.1:6800/2311309446,v1:172.21.15.1:6801/2311309446] conn(0x7fc4d8038f70 0x7fc4d803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:47:56.153 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:55.043+0000 7fc4ea7fc640 1 --2- 172.21.15.1:0/3375260393 >> [v2:172.21.15.1:6800/2311309446,v1:172.21.15.1:6801/2311309446] conn(0x7fc4d8038f70 0x7fc4d803b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc4d40098c0 tx=0x7fc4d4006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:47:56.153 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:55.047+0000 7fc4f08d9640 1 -- 172.21.15.1:0/3375260393 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc4dc04b020 con 0x7fc4ec108160 2024-09-11T21:47:56.153 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:55.270+0000 7fc4f18db640 1 -- 172.21.15.1:0/3375260393 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7fc4ec19f220 con 0x7fc4ec108160 2024-09-11T21:47:56.154 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:55.599+0000 7fc4f08d9640 1 -- 172.21.15.1:0/3375260393 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7fc4dc0072f0 con 0x7fc4ec108160 2024-09-11T21:47:56.154 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:55.601+0000 7fc4f08d9640 1 -- 172.21.15.1:0/3375260393 <== mon.0 v2:172.21.15.1:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7fc4dc046790 con 0x7fc4ec108160 2024-09-11T21:47:56.154 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:55.602+0000 7fc4f18db640 1 -- 172.21.15.1:0/3375260393 >> [v2:172.21.15.1:6800/2311309446,v1:172.21.15.1:6801/2311309446] conn(0x7fc4d8038f70 msgr2=0x7fc4d803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:47:56.154 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:55.602+0000 7fc4f18db640 1 --2- 172.21.15.1:0/3375260393 >> [v2:172.21.15.1:6800/2311309446,v1:172.21.15.1:6801/2311309446] conn(0x7fc4d8038f70 0x7fc4d803b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc4d40098c0 tx=0x7fc4d4006db0 comp rx=0 tx=0).stop 2024-09-11T21:47:56.154 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:55.602+0000 7fc4f18db640 1 -- 172.21.15.1:0/3375260393 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4ec108160 msgr2=0x7fc4ec19b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:47:56.154 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:55.602+0000 7fc4f18db640 1 --2- 172.21.15.1:0/3375260393 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4ec108160 0x7fc4ec19b6e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc02acc0 tx=0x7fc4dc031ca0 comp rx=0 tx=0).stop 2024-09-11T21:47:56.154 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:55.602+0000 7fc4f18db640 1 -- 172.21.15.1:0/3375260393 shutdown_connections 2024-09-11T21:47:56.154 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:55.602+0000 7fc4f18db640 1 --2- 172.21.15.1:0/3375260393 >> [v2:172.21.15.1:6800/2311309446,v1:172.21.15.1:6801/2311309446] conn(0x7fc4d8038f70 0x7fc4d803b430 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:47:56.154 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:55.602+0000 7fc4f18db640 1 --2- 172.21.15.1:0/3375260393 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4ec108160 0x7fc4ec19b6e0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:47:56.154 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:55.602+0000 7fc4f18db640 1 -- 172.21.15.1:0/3375260393 >> 172.21.15.1:0/3375260393 conn(0x7fc4ec101e20 msgr2=0x7fc4ec1029e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:47:56.154 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:55.603+0000 7fc4f18db640 1 -- 172.21.15.1:0/3375260393 shutdown_connections 2024-09-11T21:47:56.154 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:55.603+0000 7fc4f18db640 1 -- 172.21.15.1:0/3375260393 wait complete. 2024-09-11T21:47:56.186 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:47:55 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:47:55.959+0000 7f420be6b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-11T21:47:56.186 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:47:56 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:47:56.028+0000 7f420be6b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T21:47:56.186 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:47:56 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:47:56.185+0000 7f420be6b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T21:47:56.443 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:47:56 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:47:56.332+0000 7f420be6b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T21:47:56.730 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:56 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/3375260393' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-11T21:47:56.730 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:56 smithi001 ceph-mon[30935]: mgrmap e5: x(active, since 5s) 2024-09-11T21:47:56.730 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:47:56 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:47:56.666+0000 7f420be6b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T21:47:56.730 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:47:56 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:47:56.729+0000 7f420be6b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T21:47:57.007 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:47:56 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:47:56.929+0000 7f420be6b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T21:47:57.292 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout { 2024-09-11T21:47:57.292 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-11T21:47:57.292 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-11T21:47:57.292 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-11T21:47:57.292 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-11T21:47:57.292 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout } 2024-09-11T21:47:57.292 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:56.541+0000 7ffb5f44d640 1 Processor -- start 2024-09-11T21:47:57.293 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:56.541+0000 7ffb5f44d640 1 -- start start 2024-09-11T21:47:57.293 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:56.541+0000 7ffb5f44d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb58108160 0x7ffb5810a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:47:57.293 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:56.541+0000 7ffb5f44d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ffb5807beb0 con 0x7ffb58108160 2024-09-11T21:47:57.293 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:56.541+0000 7ffb5d1e9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb58108160 0x7ffb5810a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:47:57.293 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:56.542+0000 7ffb5d1e9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb58108160 0x7ffb5810a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53400/0 (socket says 172.21.15.1:53400) 2024-09-11T21:47:57.293 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:56.542+0000 7ffb5d1e9640 1 -- 172.21.15.1:0/141963575 learned_addr learned my addr 172.21.15.1:0/141963575 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:47:57.293 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:56.542+0000 7ffb5d1e9640 1 -- 172.21.15.1:0/141963575 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb5810aa90 con 0x7ffb58108160 2024-09-11T21:47:57.293 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:56.543+0000 7ffb5d1e9640 1 --2- 172.21.15.1:0/141963575 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb58108160 0x7ffb5810a550 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ffb54009820 tx=0x7ffb5402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c2d5324b6a70ac9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:47:57.293 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:56.543+0000 7ffb4ffff640 1 -- 172.21.15.1:0/141963575 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb54037070 con 0x7ffb58108160 2024-09-11T21:47:57.293 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:56.543+0000 7ffb4ffff640 1 -- 172.21.15.1:0/141963575 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ffb54032900 con 0x7ffb58108160 2024-09-11T21:47:57.293 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:56.543+0000 7ffb4ffff640 1 -- 172.21.15.1:0/141963575 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb54030850 con 0x7ffb58108160 2024-09-11T21:47:57.293 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:56.543+0000 7ffb5f44d640 1 -- 172.21.15.1:0/141963575 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb58108160 msgr2=0x7ffb5810a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:47:57.294 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:56.543+0000 7ffb5f44d640 1 --2- 172.21.15.1:0/141963575 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb58108160 0x7ffb5810a550 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ffb54009820 tx=0x7ffb5402a270 comp rx=0 tx=0).stop 2024-09-11T21:47:57.294 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:56.544+0000 7ffb5f44d640 1 -- 172.21.15.1:0/141963575 shutdown_connections 2024-09-11T21:47:57.294 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:56.544+0000 7ffb5f44d640 1 --2- 172.21.15.1:0/141963575 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb58108160 0x7ffb5810a550 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:47:57.294 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:56.544+0000 7ffb5f44d640 1 -- 172.21.15.1:0/141963575 >> 172.21.15.1:0/141963575 conn(0x7ffb58101e20 msgr2=0x7ffb58104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:47:57.294 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:56.544+0000 7ffb5f44d640 1 -- 172.21.15.1:0/141963575 shutdown_connections 2024-09-11T21:47:57.294 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:56.544+0000 7ffb5f44d640 1 -- 172.21.15.1:0/141963575 wait complete. 2024-09-11T21:47:57.294 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:56.544+0000 7ffb5f44d640 1 Processor -- start 2024-09-11T21:47:57.294 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:56.544+0000 7ffb5f44d640 1 -- start start 2024-09-11T21:47:57.294 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:56.544+0000 7ffb5f44d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb58108160 0x7ffb5819d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:47:57.294 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:56.544+0000 7ffb5f44d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ffb5810bb30 con 0x7ffb58108160 2024-09-11T21:47:57.294 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:56.545+0000 7ffb5d1e9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb58108160 0x7ffb5819d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:47:57.294 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:56.545+0000 7ffb5d1e9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb58108160 0x7ffb5819d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53412/0 (socket says 172.21.15.1:53412) 2024-09-11T21:47:57.295 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:56.545+0000 7ffb5d1e9640 1 -- 172.21.15.1:0/3736473820 learned_addr learned my addr 172.21.15.1:0/3736473820 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:47:57.295 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:56.545+0000 7ffb5d1e9640 1 -- 172.21.15.1:0/3736473820 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb5819de20 con 0x7ffb58108160 2024-09-11T21:47:57.295 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:56.545+0000 7ffb5d1e9640 1 --2- 172.21.15.1:0/3736473820 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb58108160 0x7ffb5819d8e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ffb54031000 tx=0x7ffb540326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:47:57.295 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:56.545+0000 7ffb4e7fc640 1 -- 172.21.15.1:0/3736473820 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb54042070 con 0x7ffb58108160 2024-09-11T21:47:57.295 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:56.545+0000 7ffb4e7fc640 1 -- 172.21.15.1:0/3736473820 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ffb54032be0 con 0x7ffb58108160 2024-09-11T21:47:57.295 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:56.545+0000 7ffb5f44d640 1 -- 172.21.15.1:0/3736473820 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb5819e0b0 con 0x7ffb58108160 2024-09-11T21:47:57.295 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:56.545+0000 7ffb4e7fc640 1 -- 172.21.15.1:0/3736473820 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb54037040 con 0x7ffb58108160 2024-09-11T21:47:57.295 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:56.545+0000 7ffb5f44d640 1 -- 172.21.15.1:0/3736473820 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb581a0da0 con 0x7ffb58108160 2024-09-11T21:47:57.295 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:56.546+0000 7ffb4e7fc640 1 -- 172.21.15.1:0/3736473820 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7ffb54045070 con 0x7ffb58108160 2024-09-11T21:47:57.295 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:56.546+0000 7ffb4e7fc640 1 --2- 172.21.15.1:0/3736473820 >> [v2:172.21.15.1:6800/2311309446,v1:172.21.15.1:6801/2311309446] conn(0x7ffb44038fc0 0x7ffb4403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:47:57.295 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:56.546+0000 7ffb5f44d640 1 -- 172.21.15.1:0/3736473820 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb581085c0 con 0x7ffb58108160 2024-09-11T21:47:57.295 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:56.546+0000 7ffb4e7fc640 1 -- 172.21.15.1:0/3736473820 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ffb5406bf40 con 0x7ffb58108160 2024-09-11T21:47:57.296 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:56.547+0000 7ffb5c9e8640 1 -- 172.21.15.1:0/3736473820 >> [v2:172.21.15.1:6800/2311309446,v1:172.21.15.1:6801/2311309446] conn(0x7ffb44038fc0 msgr2=0x7ffb4403b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/2311309446 2024-09-11T21:47:57.296 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:56.547+0000 7ffb5c9e8640 1 --2- 172.21.15.1:0/3736473820 >> [v2:172.21.15.1:6800/2311309446,v1:172.21.15.1:6801/2311309446] conn(0x7ffb44038fc0 0x7ffb4403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T21:47:57.296 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:56.551+0000 7ffb4e7fc640 1 -- 172.21.15.1:0/3736473820 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffb54045360 con 0x7ffb58108160 2024-09-11T21:47:57.296 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:56.748+0000 7ffb5c9e8640 1 -- 172.21.15.1:0/3736473820 >> [v2:172.21.15.1:6800/2311309446,v1:172.21.15.1:6801/2311309446] conn(0x7ffb44038fc0 msgr2=0x7ffb4403b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/2311309446 2024-09-11T21:47:57.296 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:56.748+0000 7ffb5c9e8640 1 --2- 172.21.15.1:0/3736473820 >> [v2:172.21.15.1:6800/2311309446,v1:172.21.15.1:6801/2311309446] conn(0x7ffb44038fc0 0x7ffb4403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-11T21:47:57.296 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:56.778+0000 7ffb5f44d640 1 -- 172.21.15.1:0/3736473820 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7ffb581a1010 con 0x7ffb58108160 2024-09-11T21:47:57.296 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:56.779+0000 7ffb4e7fc640 1 -- 172.21.15.1:0/3736473820 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7ffb581a1010 con 0x7ffb58108160 2024-09-11T21:47:57.296 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:56.781+0000 7ffb5f44d640 1 -- 172.21.15.1:0/3736473820 >> [v2:172.21.15.1:6800/2311309446,v1:172.21.15.1:6801/2311309446] conn(0x7ffb44038fc0 msgr2=0x7ffb4403b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T21:47:57.296 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:56.781+0000 7ffb5f44d640 1 --2- 172.21.15.1:0/3736473820 >> [v2:172.21.15.1:6800/2311309446,v1:172.21.15.1:6801/2311309446] conn(0x7ffb44038fc0 0x7ffb4403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:47:57.296 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:56.781+0000 7ffb5f44d640 1 -- 172.21.15.1:0/3736473820 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb58108160 msgr2=0x7ffb5819d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:47:57.296 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:56.781+0000 7ffb5f44d640 1 --2- 172.21.15.1:0/3736473820 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb58108160 0x7ffb5819d8e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ffb54031000 tx=0x7ffb540326e0 comp rx=0 tx=0).stop 2024-09-11T21:47:57.296 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:56.781+0000 7ffb5f44d640 1 -- 172.21.15.1:0/3736473820 shutdown_connections 2024-09-11T21:47:57.296 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:56.781+0000 7ffb5f44d640 1 --2- 172.21.15.1:0/3736473820 >> [v2:172.21.15.1:6800/2311309446,v1:172.21.15.1:6801/2311309446] conn(0x7ffb44038fc0 0x7ffb4403b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:47:57.297 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:56.781+0000 7ffb5f44d640 1 --2- 172.21.15.1:0/3736473820 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb58108160 0x7ffb5819d8e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:47:57.297 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:56.781+0000 7ffb5f44d640 1 -- 172.21.15.1:0/3736473820 >> 172.21.15.1:0/3736473820 conn(0x7ffb58101e20 msgr2=0x7ffb58102950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:47:57.297 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:56.781+0000 7ffb5f44d640 1 -- 172.21.15.1:0/3736473820 shutdown_connections 2024-09-11T21:47:57.297 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:56.781+0000 7ffb5f44d640 1 -- 172.21.15.1:0/3736473820 wait complete. 2024-09-11T21:47:57.297 INFO:teuthology.orchestra.run.smithi001.stdout:Waiting for the mgr to restart... 2024-09-11T21:47:57.297 INFO:teuthology.orchestra.run.smithi001.stdout:Waiting for mgr epoch 5... 2024-09-11T21:47:57.863 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:47:57 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/3736473820' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-11T21:47:58.193 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:47:57 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:47:57.862+0000 7f420be6b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T21:47:58.193 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:47:57 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:47:57.933+0000 7f420be6b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T21:47:58.943 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:47:58 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:47:58.677+0000 7f420be6b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T21:47:58.943 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:47:58 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:47:58.797+0000 7f420be6b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T21:47:59.239 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:47:58 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:47:58.946+0000 7f420be6b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T21:47:59.239 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:47:59 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:47:59.012+0000 7f420be6b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T21:47:59.239 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:47:59 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:47:59.167+0000 7f420be6b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T21:47:59.239 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:47:59 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:47:59.238+0000 7f420be6b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T21:47:59.552 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:47:59 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:47:59.301+0000 7f420be6b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T21:47:59.552 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:47:59 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:47:59.466+0000 7f420be6b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T21:47:59.552 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:47:59 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:47:59.551+0000 7f420be6b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T21:47:59.847 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:47:59 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:47:59.637+0000 7f420be6b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T21:47:59.848 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:47:59 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:47:59.723+0000 7f420be6b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T21:47:59.848 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:47:59 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:47:59.785+0000 7f420be6b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T21:47:59.848 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:47:59 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:47:59.846+0000 7f420be6b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T21:48:00.943 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:00 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:48:00.672+0000 7f420be6b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T21:48:00.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:00 smithi001 ceph-mon[30935]: Active manager daemon x restarted 2024-09-11T21:48:00.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:00 smithi001 ceph-mon[30935]: Activating manager daemon x 2024-09-11T21:48:00.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:00 smithi001 ceph-mon[30935]: osdmap e2: 0 total, 0 up, 0 in 2024-09-11T21:48:00.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:00 smithi001 ceph-mon[30935]: mgrmap e6: x(active, starting, since 0.00362813s) 2024-09-11T21:48:00.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:00 smithi001 ceph-mon[30935]: from='mgr.14118 172.21.15.1:0/779986856' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T21:48:00.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:00 smithi001 ceph-mon[30935]: from='mgr.14118 172.21.15.1:0/779986856' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T21:48:00.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:00 smithi001 ceph-mon[30935]: from='mgr.14118 172.21.15.1:0/779986856' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T21:48:00.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:00 smithi001 ceph-mon[30935]: from='mgr.14118 172.21.15.1:0/779986856' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T21:48:00.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:00 smithi001 ceph-mon[30935]: from='mgr.14118 172.21.15.1:0/779986856' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T21:48:00.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:00 smithi001 ceph-mon[30935]: Manager daemon x is now available 2024-09-11T21:48:00.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:00 smithi001 ceph-mon[30935]: from='mgr.14118 172.21.15.1:0/779986856' entity='mgr.x' 2024-09-11T21:48:00.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:00 smithi001 ceph-mon[30935]: from='mgr.14118 172.21.15.1:0/779986856' entity='mgr.x' 2024-09-11T21:48:00.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:00 smithi001 ceph-mon[30935]: from='mgr.14118 172.21.15.1:0/779986856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:48:00.945 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:00 smithi001 ceph-mon[30935]: from='mgr.14118 172.21.15.1:0/779986856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:48:00.945 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:00 smithi001 ceph-mon[30935]: from='mgr.14118 172.21.15.1:0/779986856' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-11T21:48:00.945 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:00 smithi001 ceph-mon[30935]: from='mgr.14118 172.21.15.1:0/779986856' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-11T21:48:02.257 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout { 2024-09-11T21:48:02.258 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-11T21:48:02.258 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-11T21:48:02.258 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout } 2024-09-11T21:48:02.258 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:57.707+0000 7f53cc834640 1 Processor -- start 2024-09-11T21:48:02.258 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:57.707+0000 7f53cc834640 1 -- start start 2024-09-11T21:48:02.258 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:57.707+0000 7f53cc834640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f53c8108190 0x7f53c810a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:02.258 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:57.707+0000 7f53cc834640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f53c807be60 con 0x7f53c8108190 2024-09-11T21:48:02.258 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:57.708+0000 7f53c659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f53c8108190 0x7f53c810a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:48:02.258 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:57.708+0000 7f53c659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f53c8108190 0x7f53c810a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53428/0 (socket says 172.21.15.1:53428) 2024-09-11T21:48:02.258 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:57.708+0000 7f53c659c640 1 -- 172.21.15.1:0/3916368217 learned_addr learned my addr 172.21.15.1:0/3916368217 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:48:02.259 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:57.708+0000 7f53c659c640 1 -- 172.21.15.1:0/3916368217 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53c810aac0 con 0x7f53c8108190 2024-09-11T21:48:02.259 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:57.709+0000 7f53c659c640 1 --2- 172.21.15.1:0/3916368217 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f53c8108190 0x7f53c810a580 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f53b8009820 tx=0x7f53b802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d2281f6d47cbf00 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:48:02.259 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:57.709+0000 7f53c5d9b640 1 -- 172.21.15.1:0/3916368217 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f53b8037070 con 0x7f53c8108190 2024-09-11T21:48:02.259 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:57.709+0000 7f53c5d9b640 1 -- 172.21.15.1:0/3916368217 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f53b8032900 con 0x7f53c8108190 2024-09-11T21:48:02.259 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:57.709+0000 7f53c5d9b640 1 -- 172.21.15.1:0/3916368217 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f53b8030850 con 0x7f53c8108190 2024-09-11T21:48:02.259 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:57.710+0000 7f53cc834640 1 -- 172.21.15.1:0/3916368217 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f53c8108190 msgr2=0x7f53c810a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:02.259 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:57.710+0000 7f53cc834640 1 --2- 172.21.15.1:0/3916368217 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f53c8108190 0x7f53c810a580 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f53b8009820 tx=0x7f53b802a270 comp rx=0 tx=0).stop 2024-09-11T21:48:02.259 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:57.710+0000 7f53cc834640 1 -- 172.21.15.1:0/3916368217 shutdown_connections 2024-09-11T21:48:02.259 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:57.710+0000 7f53cc834640 1 --2- 172.21.15.1:0/3916368217 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f53c8108190 0x7f53c810a580 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:02.259 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:57.710+0000 7f53cc834640 1 -- 172.21.15.1:0/3916368217 >> 172.21.15.1:0/3916368217 conn(0x7f53c8101db0 msgr2=0x7f53c81041f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:48:02.259 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:57.710+0000 7f53cc834640 1 -- 172.21.15.1:0/3916368217 shutdown_connections 2024-09-11T21:48:02.259 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:57.710+0000 7f53cc834640 1 -- 172.21.15.1:0/3916368217 wait complete. 2024-09-11T21:48:02.260 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:57.711+0000 7f53cc834640 1 Processor -- start 2024-09-11T21:48:02.260 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:57.711+0000 7f53cc834640 1 -- start start 2024-09-11T21:48:02.260 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:57.711+0000 7f53cc834640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f53c8108190 0x7f53c819b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:02.260 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:57.711+0000 7f53cc834640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f53c810bb00 con 0x7f53c8108190 2024-09-11T21:48:02.260 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:57.711+0000 7f53c659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f53c8108190 0x7f53c819b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:48:02.260 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:57.711+0000 7f53c659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f53c8108190 0x7f53c819b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53432/0 (socket says 172.21.15.1:53432) 2024-09-11T21:48:02.260 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:57.711+0000 7f53c659c640 1 -- 172.21.15.1:0/291278394 learned_addr learned my addr 172.21.15.1:0/291278394 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:48:02.260 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:57.712+0000 7f53c659c640 1 -- 172.21.15.1:0/291278394 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53c819bc10 con 0x7f53c8108190 2024-09-11T21:48:02.260 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:57.712+0000 7f53c659c640 1 --2- 172.21.15.1:0/291278394 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f53c8108190 0x7f53c819b6d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f53b8031000 tx=0x7f53b80326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:48:02.260 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:57.712+0000 7f53bf7fe640 1 -- 172.21.15.1:0/291278394 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f53b8042070 con 0x7f53c8108190 2024-09-11T21:48:02.260 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:57.712+0000 7f53bf7fe640 1 -- 172.21.15.1:0/291278394 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f53b8032be0 con 0x7f53c8108190 2024-09-11T21:48:02.261 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:57.712+0000 7f53cc834640 1 -- 172.21.15.1:0/291278394 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53c819bea0 con 0x7f53c8108190 2024-09-11T21:48:02.261 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:57.712+0000 7f53bf7fe640 1 -- 172.21.15.1:0/291278394 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f53b8037040 con 0x7f53c8108190 2024-09-11T21:48:02.261 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:57.712+0000 7f53cc834640 1 -- 172.21.15.1:0/291278394 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53c819eb90 con 0x7f53c8108190 2024-09-11T21:48:02.261 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:57.713+0000 7f53bf7fe640 1 -- 172.21.15.1:0/291278394 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f53b8045070 con 0x7f53c8108190 2024-09-11T21:48:02.261 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:57.713+0000 7f53bf7fe640 1 --2- 172.21.15.1:0/291278394 >> [v2:172.21.15.1:6800/2311309446,v1:172.21.15.1:6801/2311309446] conn(0x7f53b4038fc0 0x7f53b403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:02.261 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:57.713+0000 7f53bf7fe640 1 -- 172.21.15.1:0/291278394 --> [v2:172.21.15.1:6800/2311309446,v1:172.21.15.1:6801/2311309446] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f53b403bb90 con 0x7f53b4038fc0 2024-09-11T21:48:02.261 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:57.713+0000 7f53bf7fe640 1 -- 172.21.15.1:0/291278394 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f53b806bf50 con 0x7f53c8108190 2024-09-11T21:48:02.261 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:57.713+0000 7f53bffff640 1 -- 172.21.15.1:0/291278394 >> [v2:172.21.15.1:6800/2311309446,v1:172.21.15.1:6801/2311309446] conn(0x7f53b4038fc0 msgr2=0x7f53b403b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/2311309446 2024-09-11T21:48:02.261 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:57.713+0000 7f53bffff640 1 --2- 172.21.15.1:0/291278394 >> [v2:172.21.15.1:6800/2311309446,v1:172.21.15.1:6801/2311309446] conn(0x7f53b4038fc0 0x7f53b403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T21:48:02.262 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:57.914+0000 7f53bffff640 1 -- 172.21.15.1:0/291278394 >> [v2:172.21.15.1:6800/2311309446,v1:172.21.15.1:6801/2311309446] conn(0x7f53b4038fc0 msgr2=0x7f53b403b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/2311309446 2024-09-11T21:48:02.262 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:57.914+0000 7f53bffff640 1 --2- 172.21.15.1:0/291278394 >> [v2:172.21.15.1:6800/2311309446,v1:172.21.15.1:6801/2311309446] conn(0x7f53b4038fc0 0x7f53b403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-11T21:48:02.262 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:58.315+0000 7f53bffff640 1 -- 172.21.15.1:0/291278394 >> [v2:172.21.15.1:6800/2311309446,v1:172.21.15.1:6801/2311309446] conn(0x7f53b4038fc0 msgr2=0x7f53b403b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/2311309446 2024-09-11T21:48:02.262 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:58.315+0000 7f53bffff640 1 --2- 172.21.15.1:0/291278394 >> [v2:172.21.15.1:6800/2311309446,v1:172.21.15.1:6801/2311309446] conn(0x7f53b4038fc0 0x7f53b403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-11T21:48:02.262 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:59.116+0000 7f53bffff640 1 -- 172.21.15.1:0/291278394 >> [v2:172.21.15.1:6800/2311309446,v1:172.21.15.1:6801/2311309446] conn(0x7f53b4038fc0 msgr2=0x7f53b403b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/2311309446 2024-09-11T21:48:02.262 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:47:59.116+0000 7f53bffff640 1 --2- 172.21.15.1:0/291278394 >> [v2:172.21.15.1:6800/2311309446,v1:172.21.15.1:6801/2311309446] conn(0x7f53b4038fc0 0x7f53b403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-11T21:48:02.262 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:00.676+0000 7f53bf7fe640 1 -- 172.21.15.1:0/291278394 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7f53b806aa10 con 0x7f53c8108190 2024-09-11T21:48:02.262 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:00.676+0000 7f53bf7fe640 1 -- 172.21.15.1:0/291278394 >> [v2:172.21.15.1:6800/2311309446,v1:172.21.15.1:6801/2311309446] conn(0x7f53b4038fc0 msgr2=0x7f53b403b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T21:48:02.262 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:00.676+0000 7f53bf7fe640 1 --2- 172.21.15.1:0/291278394 >> [v2:172.21.15.1:6800/2311309446,v1:172.21.15.1:6801/2311309446] conn(0x7f53b4038fc0 0x7f53b403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:02.262 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:01.676+0000 7f53bf7fe640 1 -- 172.21.15.1:0/291278394 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f53b803d430 con 0x7f53c8108190 2024-09-11T21:48:02.262 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:01.677+0000 7f53bf7fe640 1 --2- 172.21.15.1:0/291278394 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7f53b403c8f0 0x7f53b403ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:02.262 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:01.677+0000 7f53bf7fe640 1 -- 172.21.15.1:0/291278394 --> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f53b803ee50 con 0x7f53b403c8f0 2024-09-11T21:48:02.263 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:01.677+0000 7f53bffff640 1 --2- 172.21.15.1:0/291278394 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7f53b403c8f0 0x7f53b403ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:48:02.263 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:01.678+0000 7f53bffff640 1 --2- 172.21.15.1:0/291278394 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7f53b403c8f0 0x7f53b403ece0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f53b0003980 tx=0x7f53b00092b0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:48:02.263 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:01.679+0000 7f53bf7fe640 1 -- 172.21.15.1:0/291278394 <== mgr.14118 v2:172.21.15.1:6800/2821761445 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f53b803ee50 con 0x7f53b403c8f0 2024-09-11T21:48:02.263 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:01.686+0000 7f53cc834640 1 -- 172.21.15.1:0/291278394 --> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f53c81085f0 con 0x7f53b403c8f0 2024-09-11T21:48:02.263 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:01.686+0000 7f53bf7fe640 1 -- 172.21.15.1:0/291278394 <== mgr.14118 v2:172.21.15.1:6800/2821761445 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f53c81085f0 con 0x7f53b403c8f0 2024-09-11T21:48:02.263 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:01.686+0000 7f53cc834640 1 -- 172.21.15.1:0/291278394 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7f53b403c8f0 msgr2=0x7f53b403ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:02.263 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:01.686+0000 7f53cc834640 1 --2- 172.21.15.1:0/291278394 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7f53b403c8f0 0x7f53b403ece0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f53b0003980 tx=0x7f53b00092b0 comp rx=0 tx=0).stop 2024-09-11T21:48:02.263 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:01.686+0000 7f53cc834640 1 -- 172.21.15.1:0/291278394 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f53c8108190 msgr2=0x7f53c819b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:02.263 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:01.686+0000 7f53cc834640 1 --2- 172.21.15.1:0/291278394 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f53c8108190 0x7f53c819b6d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f53b8031000 tx=0x7f53b80326e0 comp rx=0 tx=0).stop 2024-09-11T21:48:02.263 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:01.686+0000 7f53cc834640 1 -- 172.21.15.1:0/291278394 shutdown_connections 2024-09-11T21:48:02.263 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:01.686+0000 7f53cc834640 1 --2- 172.21.15.1:0/291278394 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7f53b403c8f0 0x7f53b403ece0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:02.264 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:01.686+0000 7f53cc834640 1 --2- 172.21.15.1:0/291278394 >> [v2:172.21.15.1:6800/2311309446,v1:172.21.15.1:6801/2311309446] conn(0x7f53b4038fc0 0x7f53b403b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:02.264 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:01.686+0000 7f53cc834640 1 --2- 172.21.15.1:0/291278394 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f53c8108190 0x7f53c819b6d0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:02.264 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:01.686+0000 7f53cc834640 1 -- 172.21.15.1:0/291278394 >> 172.21.15.1:0/291278394 conn(0x7f53c8101db0 msgr2=0x7f53c81029b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:48:02.264 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:01.686+0000 7f53cc834640 1 -- 172.21.15.1:0/291278394 shutdown_connections 2024-09-11T21:48:02.264 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:01.686+0000 7f53cc834640 1 -- 172.21.15.1:0/291278394 wait complete. 2024-09-11T21:48:02.264 INFO:teuthology.orchestra.run.smithi001.stdout:mgr epoch 5 is available 2024-09-11T21:48:02.264 INFO:teuthology.orchestra.run.smithi001.stdout:Setting orchestrator backend to cephadm... 2024-09-11T21:48:02.910 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:02 smithi001 ceph-mon[30935]: Found migration_current of "None". Setting to last migration. 2024-09-11T21:48:02.910 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:02 smithi001 ceph-mon[30935]: from='mgr.14118 172.21.15.1:0/779986856' entity='mgr.x' 2024-09-11T21:48:02.910 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:02 smithi001 ceph-mon[30935]: from='mgr.14118 172.21.15.1:0/779986856' entity='mgr.x' 2024-09-11T21:48:02.910 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:02 smithi001 ceph-mon[30935]: mgrmap e7: x(active, since 1.00536s) 2024-09-11T21:48:02.910 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:02 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:48:02.668+0000 7f41789d0640 -1 log_channel(cephadm) log [ERR] : [11/Sep/2024:21:48:02] ENGINE Error in HTTPServer.serve 2024-09-11T21:48:02.910 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:02 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: Traceback (most recent call last): 2024-09-11T21:48:02.911 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:02 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T21:48:02.911 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:02 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: self._connections.run(self.expiration_interval) 2024-09-11T21:48:02.911 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:02 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T21:48:02.911 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:02 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: self._run(expiration_interval) 2024-09-11T21:48:02.911 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:02 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T21:48:02.911 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:02 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T21:48:02.911 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:02 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T21:48:02.911 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:02 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T21:48:02.911 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:02 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T21:48:02.911 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:02 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: s = self.context.wrap_socket( 2024-09-11T21:48:02.911 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:02 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T21:48:02.911 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:02 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: return self.sslsocket_class._create( 2024-09-11T21:48:02.911 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:02 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T21:48:02.911 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:02 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: self.do_handshake() 2024-09-11T21:48:02.912 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:02 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T21:48:02.912 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:02 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: self._sslobj.do_handshake() 2024-09-11T21:48:02.912 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:02 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T21:48:02.912 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:02 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:48:03.436 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:02.707+0000 7fdc0cc01640 1 Processor -- start 2024-09-11T21:48:03.436 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:02.707+0000 7fdc0cc01640 1 -- start start 2024-09-11T21:48:03.436 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:02.708+0000 7fdc0cc01640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdc081076f0 0x7fdc08107af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:03.436 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:02.708+0000 7fdc0cc01640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdc081080c0 con 0x7fdc081076f0 2024-09-11T21:48:03.436 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:02.708+0000 7fdc0659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdc081076f0 0x7fdc08107af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:48:03.436 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:02.708+0000 7fdc0659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdc081076f0 0x7fdc08107af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53504/0 (socket says 172.21.15.1:53504) 2024-09-11T21:48:03.436 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:02.708+0000 7fdc0659c640 1 -- 172.21.15.1:0/3036032424 learned_addr learned my addr 172.21.15.1:0/3036032424 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:48:03.436 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:02.708+0000 7fdc0659c640 1 -- 172.21.15.1:0/3036032424 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc081088f0 con 0x7fdc081076f0 2024-09-11T21:48:03.436 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:02.709+0000 7fdc0659c640 1 --2- 172.21.15.1:0/3036032424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdc081076f0 0x7fdc08107af0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fdbf0009820 tx=0x7fdbf002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b01b2444cb809a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:48:03.436 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:02.709+0000 7fdc0559a640 1 -- 172.21.15.1:0/3036032424 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdbf0037070 con 0x7fdc081076f0 2024-09-11T21:48:03.436 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:02.709+0000 7fdc0559a640 1 -- 172.21.15.1:0/3036032424 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdbf0032900 con 0x7fdc081076f0 2024-09-11T21:48:03.436 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:02.709+0000 7fdc0559a640 1 -- 172.21.15.1:0/3036032424 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdbf0030850 con 0x7fdc081076f0 2024-09-11T21:48:03.437 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:02.709+0000 7fdc0cc01640 1 -- 172.21.15.1:0/3036032424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdc081076f0 msgr2=0x7fdc08107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:03.437 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:02.709+0000 7fdc0cc01640 1 --2- 172.21.15.1:0/3036032424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdc081076f0 0x7fdc08107af0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fdbf0009820 tx=0x7fdbf002a270 comp rx=0 tx=0).stop 2024-09-11T21:48:03.437 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:02.710+0000 7fdc0cc01640 1 -- 172.21.15.1:0/3036032424 shutdown_connections 2024-09-11T21:48:03.437 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:02.710+0000 7fdc0cc01640 1 --2- 172.21.15.1:0/3036032424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdc081076f0 0x7fdc08107af0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:03.437 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:02.710+0000 7fdc0cc01640 1 -- 172.21.15.1:0/3036032424 >> 172.21.15.1:0/3036032424 conn(0x7fdc08102ea0 msgr2=0x7fdc081052c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:48:03.437 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:02.710+0000 7fdc0cc01640 1 -- 172.21.15.1:0/3036032424 shutdown_connections 2024-09-11T21:48:03.437 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:02.710+0000 7fdc0cc01640 1 -- 172.21.15.1:0/3036032424 wait complete. 2024-09-11T21:48:03.437 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:02.710+0000 7fdc0cc01640 1 Processor -- start 2024-09-11T21:48:03.437 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:02.711+0000 7fdc0cc01640 1 -- start start 2024-09-11T21:48:03.437 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:02.711+0000 7fdc0cc01640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdc081076f0 0x7fdc0819b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:03.437 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:02.711+0000 7fdc0cc01640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0810bca0 con 0x7fdc081076f0 2024-09-11T21:48:03.437 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:02.711+0000 7fdc0659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdc081076f0 0x7fdc0819b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:48:03.437 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:02.711+0000 7fdc0659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdc081076f0 0x7fdc0819b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53512/0 (socket says 172.21.15.1:53512) 2024-09-11T21:48:03.438 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:02.711+0000 7fdc0659c640 1 -- 172.21.15.1:0/3573996406 learned_addr learned my addr 172.21.15.1:0/3573996406 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:48:03.438 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:02.711+0000 7fdc0659c640 1 -- 172.21.15.1:0/3573996406 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc0819bbf0 con 0x7fdc081076f0 2024-09-11T21:48:03.438 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:02.711+0000 7fdc0659c640 1 --2- 172.21.15.1:0/3573996406 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdc081076f0 0x7fdc0819b6b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fdbf0009940 tx=0x7fdbf0032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:48:03.438 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:02.711+0000 7fdbff7fe640 1 -- 172.21.15.1:0/3573996406 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdbf0042070 con 0x7fdc081076f0 2024-09-11T21:48:03.438 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:02.711+0000 7fdc0cc01640 1 -- 172.21.15.1:0/3573996406 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc0819be80 con 0x7fdc081076f0 2024-09-11T21:48:03.438 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:02.711+0000 7fdbff7fe640 1 -- 172.21.15.1:0/3573996406 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdbf00308c0 con 0x7fdc081076f0 2024-09-11T21:48:03.438 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:02.711+0000 7fdbff7fe640 1 -- 172.21.15.1:0/3573996406 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdbf0037040 con 0x7fdc081076f0 2024-09-11T21:48:03.438 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:02.712+0000 7fdc0cc01640 1 -- 172.21.15.1:0/3573996406 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc0819eb70 con 0x7fdc081076f0 2024-09-11T21:48:03.438 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:02.712+0000 7fdbff7fe640 1 -- 172.21.15.1:0/3573996406 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7fdbf0045070 con 0x7fdc081076f0 2024-09-11T21:48:03.438 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:02.712+0000 7fdc0cc01640 1 -- 172.21.15.1:0/3573996406 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc0810be20 con 0x7fdc081076f0 2024-09-11T21:48:03.438 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:02.713+0000 7fdbff7fe640 1 --2- 172.21.15.1:0/3573996406 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7fdbf4038e60 0x7fdbf403b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:03.438 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:02.713+0000 7fdbff7fe640 1 -- 172.21.15.1:0/3573996406 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fdbf006c0f0 con 0x7fdc081076f0 2024-09-11T21:48:03.438 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:02.713+0000 7fdc05d9b640 1 --2- 172.21.15.1:0/3573996406 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7fdbf4038e60 0x7fdbf403b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:48:03.439 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:02.713+0000 7fdc05d9b640 1 --2- 172.21.15.1:0/3573996406 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7fdbf4038e60 0x7fdbf403b320 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fdbf80098c0 tx=0x7fdbf8006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:48:03.439 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:02.718+0000 7fdbff7fe640 1 -- 172.21.15.1:0/3573996406 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdbf0045520 con 0x7fdc081076f0 2024-09-11T21:48:03.439 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:02.909+0000 7fdc0cc01640 1 -- 172.21.15.1:0/3573996406 --> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7fdc081040a0 con 0x7fdbf4038e60 2024-09-11T21:48:03.439 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:02.914+0000 7fdbff7fe640 1 -- 172.21.15.1:0/3573996406 <== mgr.14118 v2:172.21.15.1:6800/2821761445 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fdc081040a0 con 0x7fdbf4038e60 2024-09-11T21:48:03.439 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:02.916+0000 7fdc0cc01640 1 -- 172.21.15.1:0/3573996406 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7fdbf4038e60 msgr2=0x7fdbf403b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:03.439 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:02.916+0000 7fdc0cc01640 1 --2- 172.21.15.1:0/3573996406 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7fdbf4038e60 0x7fdbf403b320 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fdbf80098c0 tx=0x7fdbf8006db0 comp rx=0 tx=0).stop 2024-09-11T21:48:03.439 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:02.916+0000 7fdc0cc01640 1 -- 172.21.15.1:0/3573996406 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdc081076f0 msgr2=0x7fdc0819b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:03.439 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:02.916+0000 7fdc0cc01640 1 --2- 172.21.15.1:0/3573996406 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdc081076f0 0x7fdc0819b6b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fdbf0009940 tx=0x7fdbf0032ce0 comp rx=0 tx=0).stop 2024-09-11T21:48:03.439 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:02.916+0000 7fdc0cc01640 1 -- 172.21.15.1:0/3573996406 shutdown_connections 2024-09-11T21:48:03.439 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:02.916+0000 7fdc0cc01640 1 --2- 172.21.15.1:0/3573996406 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7fdbf4038e60 0x7fdbf403b320 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:03.439 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:02.916+0000 7fdc0cc01640 1 --2- 172.21.15.1:0/3573996406 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdc081076f0 0x7fdc0819b6b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:03.439 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:02.916+0000 7fdc0cc01640 1 -- 172.21.15.1:0/3573996406 >> 172.21.15.1:0/3573996406 conn(0x7fdc08102ea0 msgr2=0x7fdc08103990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:48:03.439 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:02.916+0000 7fdc0cc01640 1 -- 172.21.15.1:0/3573996406 shutdown_connections 2024-09-11T21:48:03.439 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:02.916+0000 7fdc0cc01640 1 -- 172.21.15.1:0/3573996406 wait complete. 2024-09-11T21:48:03.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:03 smithi001 ceph-mon[30935]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-11T21:48:03.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:03 smithi001 ceph-mon[30935]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-11T21:48:03.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:03 smithi001 ceph-mon[30935]: [11/Sep/2024:21:48:02] ENGINE Bus STARTING 2024-09-11T21:48:03.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:03 smithi001 ceph-mon[30935]: from='mgr.14118 172.21.15.1:0/779986856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:48:03.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:03 smithi001 ceph-mon[30935]: from='mgr.14118 172.21.15.1:0/779986856' entity='mgr.x' 2024-09-11T21:48:03.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:03 smithi001 ceph-mon[30935]: from='mgr.14118 172.21.15.1:0/779986856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:48:04.551 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-11T21:48:04.551 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:03.800+0000 7f54599b6640 1 Processor -- start 2024-09-11T21:48:04.551 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:03.801+0000 7f54599b6640 1 -- start start 2024-09-11T21:48:04.551 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:03.801+0000 7f54599b6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54541077a0 0x7f5454107ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:04.551 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:03.801+0000 7f54599b6640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5454108170 con 0x7f54541077a0 2024-09-11T21:48:04.552 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:03.801+0000 7f5452ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54541077a0 0x7f5454107ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:48:04.552 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:03.801+0000 7f5452ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54541077a0 0x7f5454107ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53522/0 (socket says 172.21.15.1:53522) 2024-09-11T21:48:04.552 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:03.801+0000 7f5452ffd640 1 -- 172.21.15.1:0/1782128147 learned_addr learned my addr 172.21.15.1:0/1782128147 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:48:04.552 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:03.801+0000 7f5452ffd640 1 -- 172.21.15.1:0/1782128147 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5454108950 con 0x7f54541077a0 2024-09-11T21:48:04.552 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:03.802+0000 7f5452ffd640 1 --2- 172.21.15.1:0/1782128147 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54541077a0 0x7f5454107ba0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5444009820 tx=0x7f544402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bf696b604687a02 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:48:04.552 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:03.802+0000 7f54527fc640 1 -- 172.21.15.1:0/1782128147 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5444037070 con 0x7f54541077a0 2024-09-11T21:48:04.552 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:03.802+0000 7f54527fc640 1 -- 172.21.15.1:0/1782128147 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5444032900 con 0x7f54541077a0 2024-09-11T21:48:04.552 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:03.802+0000 7f54527fc640 1 -- 172.21.15.1:0/1782128147 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5444030850 con 0x7f54541077a0 2024-09-11T21:48:04.552 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:03.803+0000 7f54599b6640 1 -- 172.21.15.1:0/1782128147 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54541077a0 msgr2=0x7f5454107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:04.552 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:03.803+0000 7f54599b6640 1 --2- 172.21.15.1:0/1782128147 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54541077a0 0x7f5454107ba0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5444009820 tx=0x7f544402a270 comp rx=0 tx=0).stop 2024-09-11T21:48:04.552 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:03.803+0000 7f54599b6640 1 -- 172.21.15.1:0/1782128147 shutdown_connections 2024-09-11T21:48:04.552 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:03.803+0000 7f54599b6640 1 --2- 172.21.15.1:0/1782128147 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54541077a0 0x7f5454107ba0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:04.553 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:03.803+0000 7f54599b6640 1 -- 172.21.15.1:0/1782128147 >> 172.21.15.1:0/1782128147 conn(0x7f5454102fb0 msgr2=0x7f54541053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:48:04.553 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:03.803+0000 7f54599b6640 1 -- 172.21.15.1:0/1782128147 shutdown_connections 2024-09-11T21:48:04.553 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:03.803+0000 7f54599b6640 1 -- 172.21.15.1:0/1782128147 wait complete. 2024-09-11T21:48:04.553 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:03.803+0000 7f54599b6640 1 Processor -- start 2024-09-11T21:48:04.553 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:03.803+0000 7f54599b6640 1 -- start start 2024-09-11T21:48:04.553 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:03.804+0000 7f54599b6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54541077a0 0x7f545419b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:04.553 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:03.804+0000 7f54599b6640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f545410bd00 con 0x7f54541077a0 2024-09-11T21:48:04.553 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:03.804+0000 7f5452ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54541077a0 0x7f545419b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:48:04.553 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:03.804+0000 7f5452ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54541077a0 0x7f545419b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53532/0 (socket says 172.21.15.1:53532) 2024-09-11T21:48:04.553 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:03.804+0000 7f5452ffd640 1 -- 172.21.15.1:0/896650201 learned_addr learned my addr 172.21.15.1:0/896650201 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:48:04.553 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:03.804+0000 7f5452ffd640 1 -- 172.21.15.1:0/896650201 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f545419bc40 con 0x7f54541077a0 2024-09-11T21:48:04.553 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:03.804+0000 7f5452ffd640 1 --2- 172.21.15.1:0/896650201 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54541077a0 0x7f545419b700 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f54440097f0 tx=0x7f5444030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:48:04.554 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:03.804+0000 7f54589b4640 1 -- 172.21.15.1:0/896650201 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5444042070 con 0x7f54541077a0 2024-09-11T21:48:04.554 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:03.804+0000 7f54589b4640 1 -- 172.21.15.1:0/896650201 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5444032ca0 con 0x7f54541077a0 2024-09-11T21:48:04.554 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:03.804+0000 7f54589b4640 1 -- 172.21.15.1:0/896650201 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5444037040 con 0x7f54541077a0 2024-09-11T21:48:04.554 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:03.804+0000 7f54599b6640 1 -- 172.21.15.1:0/896650201 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f545419bed0 con 0x7f54541077a0 2024-09-11T21:48:04.554 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:03.804+0000 7f54599b6640 1 -- 172.21.15.1:0/896650201 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f545419ebc0 con 0x7f54541077a0 2024-09-11T21:48:04.554 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:03.805+0000 7f54589b4640 1 -- 172.21.15.1:0/896650201 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f5444045070 con 0x7f54541077a0 2024-09-11T21:48:04.554 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:03.805+0000 7f54599b6640 1 -- 172.21.15.1:0/896650201 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5454107c20 con 0x7f54541077a0 2024-09-11T21:48:04.554 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:03.806+0000 7f54589b4640 1 --2- 172.21.15.1:0/896650201 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7f5440038f70 0x7f544003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:04.554 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:03.806+0000 7f54589b4640 1 -- 172.21.15.1:0/896650201 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f544406beb0 con 0x7f54541077a0 2024-09-11T21:48:04.554 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:03.806+0000 7f544bfff640 1 --2- 172.21.15.1:0/896650201 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7f5440038f70 0x7f544003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:48:04.554 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:03.806+0000 7f544bfff640 1 --2- 172.21.15.1:0/896650201 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7f5440038f70 0x7f544003b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f543c0098c0 tx=0x7f543c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:48:04.554 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:03.810+0000 7f54589b4640 1 -- 172.21.15.1:0/896650201 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f544404b170 con 0x7f54541077a0 2024-09-11T21:48:04.554 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:03.991+0000 7f54599b6640 1 -- 172.21.15.1:0/896650201 --> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f54541041f0 con 0x7f5440038f70 2024-09-11T21:48:04.554 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:03.992+0000 7f54589b4640 1 -- 172.21.15.1:0/896650201 <== mgr.14118 v2:172.21.15.1:6800/2821761445 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f54541041f0 con 0x7f5440038f70 2024-09-11T21:48:04.555 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:03.994+0000 7f54599b6640 1 -- 172.21.15.1:0/896650201 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7f5440038f70 msgr2=0x7f544003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:04.555 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:03.994+0000 7f54599b6640 1 --2- 172.21.15.1:0/896650201 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7f5440038f70 0x7f544003b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f543c0098c0 tx=0x7f543c006db0 comp rx=0 tx=0).stop 2024-09-11T21:48:04.555 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:03.994+0000 7f54599b6640 1 -- 172.21.15.1:0/896650201 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54541077a0 msgr2=0x7f545419b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:04.555 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:03.994+0000 7f54599b6640 1 --2- 172.21.15.1:0/896650201 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54541077a0 0x7f545419b700 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f54440097f0 tx=0x7f5444030ec0 comp rx=0 tx=0).stop 2024-09-11T21:48:04.555 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:03.995+0000 7f54599b6640 1 -- 172.21.15.1:0/896650201 shutdown_connections 2024-09-11T21:48:04.555 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:03.995+0000 7f54599b6640 1 --2- 172.21.15.1:0/896650201 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7f5440038f70 0x7f544003b430 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:04.555 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:03.995+0000 7f54599b6640 1 --2- 172.21.15.1:0/896650201 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54541077a0 0x7f545419b700 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:04.555 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:03.995+0000 7f54599b6640 1 -- 172.21.15.1:0/896650201 >> 172.21.15.1:0/896650201 conn(0x7f5454102fb0 msgr2=0x7f5454103ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:48:04.555 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:03.995+0000 7f54599b6640 1 -- 172.21.15.1:0/896650201 shutdown_connections 2024-09-11T21:48:04.555 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:03.995+0000 7f54599b6640 1 -- 172.21.15.1:0/896650201 wait complete. 2024-09-11T21:48:04.555 INFO:teuthology.orchestra.run.smithi001.stdout:Generating ssh key... 2024-09-11T21:48:04.798 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:04 smithi001 ceph-mon[30935]: [11/Sep/2024:21:48:02] ENGINE Serving on https://172.21.15.1:7150 2024-09-11T21:48:04.798 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:04 smithi001 ceph-mon[30935]: [11/Sep/2024:21:48:02] ENGINE Error in HTTPServer.serve 2024-09-11T21:48:04.798 INFO:journalctl@ceph.mon.a.smithi001.stdout: Traceback (most recent call last): 2024-09-11T21:48:04.798 INFO:journalctl@ceph.mon.a.smithi001.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T21:48:04.798 INFO:journalctl@ceph.mon.a.smithi001.stdout: self._connections.run(self.expiration_interval) 2024-09-11T21:48:04.798 INFO:journalctl@ceph.mon.a.smithi001.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T21:48:04.798 INFO:journalctl@ceph.mon.a.smithi001.stdout: self._run(expiration_interval) 2024-09-11T21:48:04.798 INFO:journalctl@ceph.mon.a.smithi001.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T21:48:04.798 INFO:journalctl@ceph.mon.a.smithi001.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T21:48:04.799 INFO:journalctl@ceph.mon.a.smithi001.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T21:48:04.799 INFO:journalctl@ceph.mon.a.smithi001.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T21:48:04.799 INFO:journalctl@ceph.mon.a.smithi001.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T21:48:04.799 INFO:journalctl@ceph.mon.a.smithi001.stdout: s = self.context.wrap_socket( 2024-09-11T21:48:04.799 INFO:journalctl@ceph.mon.a.smithi001.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T21:48:04.799 INFO:journalctl@ceph.mon.a.smithi001.stdout: return self.sslsocket_class._create( 2024-09-11T21:48:04.799 INFO:journalctl@ceph.mon.a.smithi001.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T21:48:04.799 INFO:journalctl@ceph.mon.a.smithi001.stdout: self.do_handshake() 2024-09-11T21:48:04.799 INFO:journalctl@ceph.mon.a.smithi001.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T21:48:04.799 INFO:journalctl@ceph.mon.a.smithi001.stdout: self._sslobj.do_handshake() 2024-09-11T21:48:04.799 INFO:journalctl@ceph.mon.a.smithi001.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T21:48:04.799 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:04 smithi001 ceph-mon[30935]: [11/Sep/2024:21:48:02] ENGINE Serving on http://172.21.15.1:8765 2024-09-11T21:48:04.799 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:04 smithi001 ceph-mon[30935]: [11/Sep/2024:21:48:02] ENGINE Bus STARTED 2024-09-11T21:48:04.800 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:04 smithi001 ceph-mon[30935]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:48:04.800 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:04 smithi001 ceph-mon[30935]: mgrmap e8: x(active, since 2s) 2024-09-11T21:48:05.808 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:05 smithi001 ceph-mon[30935]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:48:05.808 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:05 smithi001 ceph-mon[30935]: from='mgr.14118 172.21.15.1:0/779986856' entity='mgr.x' 2024-09-11T21:48:05.808 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:05 smithi001 ceph-mon[30935]: from='mgr.14118 172.21.15.1:0/779986856' entity='mgr.x' 2024-09-11T21:48:05.809 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:05 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: Generating public/private rsa key pair. 2024-09-11T21:48:05.809 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:05 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: Your identification has been saved in /tmp/tmpjz1zkon0/key 2024-09-11T21:48:05.809 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:05 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: Your public key has been saved in /tmp/tmpjz1zkon0/key.pub 2024-09-11T21:48:05.809 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:05 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: The key fingerprint is: 2024-09-11T21:48:05.809 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:05 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: SHA256:IhabvtH+vf/ys4F1PgUgRHVM8CYs4PeD3R6T8bMfY5g ceph-3ffca83c-7087-11ef-bcea-c7b262605968 2024-09-11T21:48:05.809 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:05 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: The key's randomart image is: 2024-09-11T21:48:05.809 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:05 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: +---[RSA 3072]----+ 2024-09-11T21:48:05.809 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:05 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: | .o+.++o | 2024-09-11T21:48:05.809 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:05 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: | . . o +. | 2024-09-11T21:48:05.809 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:05 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: | . . o o = | 2024-09-11T21:48:05.809 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:05 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: | + . = + = | 2024-09-11T21:48:05.810 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:05 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: | = . S . + *.+| 2024-09-11T21:48:05.810 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:05 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: | o o . =o=+| 2024-09-11T21:48:05.810 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:05 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: | o . .Eo*.| 2024-09-11T21:48:05.810 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:05 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: | + . . o.=| 2024-09-11T21:48:05.810 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:05 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: | . ... oo.++o.| 2024-09-11T21:48:05.810 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:05 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: +----[SHA256]-----+ 2024-09-11T21:48:06.116 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:04.966+0000 7fd528b18640 1 Processor -- start 2024-09-11T21:48:06.116 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:04.966+0000 7fd528b18640 1 -- start start 2024-09-11T21:48:06.116 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:04.966+0000 7fd528b18640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd524107720 0x7fd524107b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:06.116 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:04.966+0000 7fd528b18640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd5241080f0 con 0x7fd524107720 2024-09-11T21:48:06.116 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:04.966+0000 7fd52259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd524107720 0x7fd524107b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:48:06.117 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:04.966+0000 7fd52259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd524107720 0x7fd524107b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53544/0 (socket says 172.21.15.1:53544) 2024-09-11T21:48:06.117 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:04.966+0000 7fd52259c640 1 -- 172.21.15.1:0/1897328780 learned_addr learned my addr 172.21.15.1:0/1897328780 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:48:06.117 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:04.967+0000 7fd52259c640 1 -- 172.21.15.1:0/1897328780 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5241088d0 con 0x7fd524107720 2024-09-11T21:48:06.117 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:04.967+0000 7fd52259c640 1 --2- 172.21.15.1:0/1897328780 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd524107720 0x7fd524107b20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd514009820 tx=0x7fd51402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bfb8073696ff2e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:48:06.117 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:04.968+0000 7fd521d9b640 1 -- 172.21.15.1:0/1897328780 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd514037070 con 0x7fd524107720 2024-09-11T21:48:06.117 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:04.968+0000 7fd521d9b640 1 -- 172.21.15.1:0/1897328780 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd514032900 con 0x7fd524107720 2024-09-11T21:48:06.117 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:04.968+0000 7fd521d9b640 1 -- 172.21.15.1:0/1897328780 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd514030850 con 0x7fd524107720 2024-09-11T21:48:06.117 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:04.968+0000 7fd528b18640 1 -- 172.21.15.1:0/1897328780 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd524107720 msgr2=0x7fd524107b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:06.117 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:04.968+0000 7fd528b18640 1 --2- 172.21.15.1:0/1897328780 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd524107720 0x7fd524107b20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd514009820 tx=0x7fd51402a270 comp rx=0 tx=0).stop 2024-09-11T21:48:06.117 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:04.968+0000 7fd528b18640 1 -- 172.21.15.1:0/1897328780 shutdown_connections 2024-09-11T21:48:06.117 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:04.968+0000 7fd528b18640 1 --2- 172.21.15.1:0/1897328780 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd524107720 0x7fd524107b20 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:06.117 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:04.968+0000 7fd528b18640 1 -- 172.21.15.1:0/1897328780 >> 172.21.15.1:0/1897328780 conn(0x7fd524102ef0 msgr2=0x7fd524105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:48:06.118 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:04.968+0000 7fd528b18640 1 -- 172.21.15.1:0/1897328780 shutdown_connections 2024-09-11T21:48:06.118 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:04.968+0000 7fd528b18640 1 -- 172.21.15.1:0/1897328780 wait complete. 2024-09-11T21:48:06.118 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:04.969+0000 7fd528b18640 1 Processor -- start 2024-09-11T21:48:06.118 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:04.969+0000 7fd528b18640 1 -- start start 2024-09-11T21:48:06.118 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:04.970+0000 7fd528b18640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd524107720 0x7fd52419d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:06.118 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:04.970+0000 7fd528b18640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd52410bc80 con 0x7fd524107720 2024-09-11T21:48:06.118 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:04.970+0000 7fd52259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd524107720 0x7fd52419d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:48:06.118 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:04.970+0000 7fd52259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd524107720 0x7fd52419d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53554/0 (socket says 172.21.15.1:53554) 2024-09-11T21:48:06.118 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:04.970+0000 7fd52259c640 1 -- 172.21.15.1:0/1263101318 learned_addr learned my addr 172.21.15.1:0/1263101318 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:48:06.118 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:04.970+0000 7fd52259c640 1 -- 172.21.15.1:0/1263101318 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd52419de30 con 0x7fd524107720 2024-09-11T21:48:06.118 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:04.970+0000 7fd52259c640 1 --2- 172.21.15.1:0/1263101318 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd524107720 0x7fd52419d8f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd514009940 tx=0x7fd514032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:48:06.118 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:04.970+0000 7fd51bfff640 1 -- 172.21.15.1:0/1263101318 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd514042070 con 0x7fd524107720 2024-09-11T21:48:06.119 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:04.970+0000 7fd51bfff640 1 -- 172.21.15.1:0/1263101318 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd5140308c0 con 0x7fd524107720 2024-09-11T21:48:06.119 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:04.970+0000 7fd528b18640 1 -- 172.21.15.1:0/1263101318 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd52419e0c0 con 0x7fd524107720 2024-09-11T21:48:06.119 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:04.971+0000 7fd51bfff640 1 -- 172.21.15.1:0/1263101318 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd514037040 con 0x7fd524107720 2024-09-11T21:48:06.119 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:04.971+0000 7fd528b18640 1 -- 172.21.15.1:0/1263101318 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5241a0db0 con 0x7fd524107720 2024-09-11T21:48:06.119 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:04.971+0000 7fd51bfff640 1 -- 172.21.15.1:0/1263101318 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fd514045070 con 0x7fd524107720 2024-09-11T21:48:06.119 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:04.972+0000 7fd51bfff640 1 --2- 172.21.15.1:0/1263101318 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7fd510038f70 0x7fd51003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:06.119 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:04.972+0000 7fd51bfff640 1 -- 172.21.15.1:0/1263101318 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd51406c090 con 0x7fd524107720 2024-09-11T21:48:06.119 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:04.972+0000 7fd528b18640 1 -- 172.21.15.1:0/1263101318 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd52410be00 con 0x7fd524107720 2024-09-11T21:48:06.119 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:04.972+0000 7fd519d9b640 1 --2- 172.21.15.1:0/1263101318 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7fd510038f70 0x7fd51003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:48:06.119 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:04.972+0000 7fd519d9b640 1 --2- 172.21.15.1:0/1263101318 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7fd510038f70 0x7fd51003b430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd50c0098c0 tx=0x7fd50c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:48:06.119 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:04.977+0000 7fd51bfff640 1 -- 172.21.15.1:0/1263101318 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd514045520 con 0x7fd524107720 2024-09-11T21:48:06.119 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:05.156+0000 7fd528b18640 1 -- 172.21.15.1:0/1263101318 --> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7fd5241040e0 con 0x7fd510038f70 2024-09-11T21:48:06.120 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:05.498+0000 7fd51bfff640 1 -- 172.21.15.1:0/1263101318 <== mgr.14118 v2:172.21.15.1:6800/2821761445 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd5241040e0 con 0x7fd510038f70 2024-09-11T21:48:06.120 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:05.500+0000 7fd528b18640 1 -- 172.21.15.1:0/1263101318 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7fd510038f70 msgr2=0x7fd51003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:06.120 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:05.500+0000 7fd528b18640 1 --2- 172.21.15.1:0/1263101318 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7fd510038f70 0x7fd51003b430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd50c0098c0 tx=0x7fd50c006db0 comp rx=0 tx=0).stop 2024-09-11T21:48:06.120 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:05.500+0000 7fd528b18640 1 -- 172.21.15.1:0/1263101318 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd524107720 msgr2=0x7fd52419d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:06.120 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:05.500+0000 7fd528b18640 1 --2- 172.21.15.1:0/1263101318 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd524107720 0x7fd52419d8f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd514009940 tx=0x7fd514032ce0 comp rx=0 tx=0).stop 2024-09-11T21:48:06.120 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:05.500+0000 7fd528b18640 1 -- 172.21.15.1:0/1263101318 shutdown_connections 2024-09-11T21:48:06.120 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:05.500+0000 7fd528b18640 1 --2- 172.21.15.1:0/1263101318 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7fd510038f70 0x7fd51003b430 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:06.120 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:05.500+0000 7fd528b18640 1 --2- 172.21.15.1:0/1263101318 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd524107720 0x7fd52419d8f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:06.120 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:05.500+0000 7fd528b18640 1 -- 172.21.15.1:0/1263101318 >> 172.21.15.1:0/1263101318 conn(0x7fd524102ef0 msgr2=0x7fd5241039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:48:06.120 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:05.500+0000 7fd528b18640 1 -- 172.21.15.1:0/1263101318 shutdown_connections 2024-09-11T21:48:06.120 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:05.500+0000 7fd528b18640 1 -- 172.21.15.1:0/1263101318 wait complete. 2024-09-11T21:48:06.677 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:06 smithi001 ceph-mon[30935]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:48:06.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:06 smithi001 ceph-mon[30935]: Generating ssh key... 2024-09-11T21:48:07.240 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCmSKiSRGV/mNwNumHfq5hGkh9DG302w4QnOziWzEe5RG6HBp3yzlWsDKhhZepa5kKwtVzLDlvOL7tnxpfj7diqEu3JplRVuT3iFJhsBotcjmerodp8UTkxJnQfIbFG3tVLEyuGX7S8xvuzvhhbmhHLcvquJZPvkPqI1qEE5bFz/U71/Z0ZG9ftsIPEIf3RcuJIDrEQBU9Af0wkTgM4i/PiTplm2UT2DTVEvSWg4Hx0Ck4anmX4lK78h54U71cDCRlqkWaQ/sazb+g482Coek+LPB6YpkSiL1FfImndQSiwOTGSnWBfWEGxHjsnfXjMEzOBgi0JogVWjktEyBdCPhYeltyxBfCMIXo9sUJCqk6bNwT7rfTbstRSdtmYAAH4snVMVqnOfvHt8nWuTIrDH3WlOwLeNim/3IwW3p7+OzQlmNKiAT+3I3+c9Ugn12/OOeL9LPnmiIUrR0HUGyGTY/CTA+LASfNGd6yXn7Y19ieA0AjPtlZWA0EHZlkQUhPOXTU= ceph-3ffca83c-7087-11ef-bcea-c7b262605968 2024-09-11T21:48:07.241 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:06.491+0000 7fbd116d1640 1 Processor -- start 2024-09-11T21:48:07.241 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:06.491+0000 7fbd116d1640 1 -- start start 2024-09-11T21:48:07.241 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:06.491+0000 7fbd116d1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbd0c1076f0 0x7fbd0c107af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:07.241 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:06.492+0000 7fbd116d1640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0c1080c0 con 0x7fbd0c1076f0 2024-09-11T21:48:07.241 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:06.492+0000 7fbd0affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbd0c1076f0 0x7fbd0c107af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:48:07.241 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:06.492+0000 7fbd0affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbd0c1076f0 0x7fbd0c107af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58590/0 (socket says 172.21.15.1:58590) 2024-09-11T21:48:07.241 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:06.492+0000 7fbd0affd640 1 -- 172.21.15.1:0/4243400012 learned_addr learned my addr 172.21.15.1:0/4243400012 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:48:07.241 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:06.492+0000 7fbd0affd640 1 -- 172.21.15.1:0/4243400012 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd0c1088f0 con 0x7fbd0c1076f0 2024-09-11T21:48:07.241 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:06.493+0000 7fbd0affd640 1 --2- 172.21.15.1:0/4243400012 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbd0c1076f0 0x7fbd0c107af0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc009820 tx=0x7fbcfc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c5353c613f58c0f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:48:07.242 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:06.493+0000 7fbd09ffb640 1 -- 172.21.15.1:0/4243400012 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbcfc037070 con 0x7fbd0c1076f0 2024-09-11T21:48:07.242 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:06.493+0000 7fbd09ffb640 1 -- 172.21.15.1:0/4243400012 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbcfc032900 con 0x7fbd0c1076f0 2024-09-11T21:48:07.242 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:06.493+0000 7fbd09ffb640 1 -- 172.21.15.1:0/4243400012 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbcfc030850 con 0x7fbd0c1076f0 2024-09-11T21:48:07.242 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:06.493+0000 7fbd116d1640 1 -- 172.21.15.1:0/4243400012 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbd0c1076f0 msgr2=0x7fbd0c107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:07.242 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:06.493+0000 7fbd116d1640 1 --2- 172.21.15.1:0/4243400012 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbd0c1076f0 0x7fbd0c107af0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc009820 tx=0x7fbcfc02a270 comp rx=0 tx=0).stop 2024-09-11T21:48:07.242 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:06.494+0000 7fbd116d1640 1 -- 172.21.15.1:0/4243400012 shutdown_connections 2024-09-11T21:48:07.242 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:06.494+0000 7fbd116d1640 1 --2- 172.21.15.1:0/4243400012 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbd0c1076f0 0x7fbd0c107af0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:07.242 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:06.494+0000 7fbd116d1640 1 -- 172.21.15.1:0/4243400012 >> 172.21.15.1:0/4243400012 conn(0x7fbd0c102ec0 msgr2=0x7fbd0c1052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:48:07.242 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:06.494+0000 7fbd116d1640 1 -- 172.21.15.1:0/4243400012 shutdown_connections 2024-09-11T21:48:07.242 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:06.494+0000 7fbd116d1640 1 -- 172.21.15.1:0/4243400012 wait complete. 2024-09-11T21:48:07.242 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:06.494+0000 7fbd116d1640 1 Processor -- start 2024-09-11T21:48:07.242 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:06.494+0000 7fbd116d1640 1 -- start start 2024-09-11T21:48:07.243 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:06.495+0000 7fbd116d1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbd0c1076f0 0x7fbd0c19b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:07.243 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:06.495+0000 7fbd116d1640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0c10bca0 con 0x7fbd0c1076f0 2024-09-11T21:48:07.243 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:06.495+0000 7fbd0affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbd0c1076f0 0x7fbd0c19b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:48:07.243 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:06.495+0000 7fbd0affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbd0c1076f0 0x7fbd0c19b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58600/0 (socket says 172.21.15.1:58600) 2024-09-11T21:48:07.243 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:06.495+0000 7fbd0affd640 1 -- 172.21.15.1:0/4252942358 learned_addr learned my addr 172.21.15.1:0/4252942358 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:48:07.243 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:06.495+0000 7fbd0affd640 1 -- 172.21.15.1:0/4252942358 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd0c19bc20 con 0x7fbd0c1076f0 2024-09-11T21:48:07.243 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:06.495+0000 7fbd0affd640 1 --2- 172.21.15.1:0/4252942358 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbd0c1076f0 0x7fbd0c19b6e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc009940 tx=0x7fbcfc032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:48:07.243 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:06.495+0000 7fbcf3fff640 1 -- 172.21.15.1:0/4252942358 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbcfc042070 con 0x7fbd0c1076f0 2024-09-11T21:48:07.243 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:06.495+0000 7fbcf3fff640 1 -- 172.21.15.1:0/4252942358 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbcfc0308c0 con 0x7fbd0c1076f0 2024-09-11T21:48:07.243 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:06.495+0000 7fbd116d1640 1 -- 172.21.15.1:0/4252942358 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd0c19beb0 con 0x7fbd0c1076f0 2024-09-11T21:48:07.243 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:06.495+0000 7fbcf3fff640 1 -- 172.21.15.1:0/4252942358 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbcfc037040 con 0x7fbd0c1076f0 2024-09-11T21:48:07.243 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:06.495+0000 7fbd116d1640 1 -- 172.21.15.1:0/4252942358 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd0c19eba0 con 0x7fbd0c1076f0 2024-09-11T21:48:07.243 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:06.496+0000 7fbcf3fff640 1 -- 172.21.15.1:0/4252942358 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fbcfc045070 con 0x7fbd0c1076f0 2024-09-11T21:48:07.244 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:06.496+0000 7fbcf3fff640 1 --2- 172.21.15.1:0/4252942358 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7fbcf4038fc0 0x7fbcf403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:07.244 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:06.496+0000 7fbd116d1640 1 -- 172.21.15.1:0/4252942358 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd0c10be20 con 0x7fbd0c1076f0 2024-09-11T21:48:07.244 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:06.496+0000 7fbcf3fff640 1 -- 172.21.15.1:0/4252942358 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbcfc06c030 con 0x7fbd0c1076f0 2024-09-11T21:48:07.244 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:06.496+0000 7fbd0a7fc640 1 --2- 172.21.15.1:0/4252942358 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7fbcf4038fc0 0x7fbcf403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:48:07.244 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:06.497+0000 7fbd0a7fc640 1 --2- 172.21.15.1:0/4252942358 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7fbcf4038fc0 0x7fbcf403b480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbd000098c0 tx=0x7fbd00006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:48:07.244 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:06.501+0000 7fbcf3fff640 1 -- 172.21.15.1:0/4252942358 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbcfc045360 con 0x7fbd0c1076f0 2024-09-11T21:48:07.244 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:06.677+0000 7fbd116d1640 1 -- 172.21.15.1:0/4252942358 --> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7fbd0c1040e0 con 0x7fbcf4038fc0 2024-09-11T21:48:07.244 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:06.677+0000 7fbcf3fff640 1 -- 172.21.15.1:0/4252942358 <== mgr.14118 v2:172.21.15.1:6800/2821761445 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7fbd0c1040e0 con 0x7fbcf4038fc0 2024-09-11T21:48:07.244 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:06.679+0000 7fbd116d1640 1 -- 172.21.15.1:0/4252942358 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7fbcf4038fc0 msgr2=0x7fbcf403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:07.244 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:06.679+0000 7fbd116d1640 1 --2- 172.21.15.1:0/4252942358 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7fbcf4038fc0 0x7fbcf403b480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbd000098c0 tx=0x7fbd00006db0 comp rx=0 tx=0).stop 2024-09-11T21:48:07.244 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:06.679+0000 7fbd116d1640 1 -- 172.21.15.1:0/4252942358 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbd0c1076f0 msgr2=0x7fbd0c19b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:07.245 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:06.679+0000 7fbd116d1640 1 --2- 172.21.15.1:0/4252942358 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbd0c1076f0 0x7fbd0c19b6e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc009940 tx=0x7fbcfc032ce0 comp rx=0 tx=0).stop 2024-09-11T21:48:07.245 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:06.679+0000 7fbd116d1640 1 -- 172.21.15.1:0/4252942358 shutdown_connections 2024-09-11T21:48:07.245 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:06.679+0000 7fbd116d1640 1 --2- 172.21.15.1:0/4252942358 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7fbcf4038fc0 0x7fbcf403b480 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:07.245 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:06.679+0000 7fbd116d1640 1 --2- 172.21.15.1:0/4252942358 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbd0c1076f0 0x7fbd0c19b6e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:07.245 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:06.679+0000 7fbd116d1640 1 -- 172.21.15.1:0/4252942358 >> 172.21.15.1:0/4252942358 conn(0x7fbd0c102ec0 msgr2=0x7fbd0c1039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:48:07.245 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:06.680+0000 7fbd116d1640 1 -- 172.21.15.1:0/4252942358 shutdown_connections 2024-09-11T21:48:07.245 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:06.680+0000 7fbd116d1640 1 -- 172.21.15.1:0/4252942358 wait complete. 2024-09-11T21:48:07.245 INFO:teuthology.orchestra.run.smithi001.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-11T21:48:07.245 INFO:teuthology.orchestra.run.smithi001.stdout:Adding key to root@localhost authorized_keys... 2024-09-11T21:48:07.247 INFO:teuthology.orchestra.run.smithi001.stdout:Adding host smithi001... 2024-09-11T21:48:07.820 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:07 smithi001 ceph-mon[30935]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:48:08.870 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:08 smithi001 ceph-mon[30935]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi001", "addr": "172.21.15.1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:48:09.793 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:09 smithi001 ceph-mon[30935]: Deploying cephadm binary to smithi001 2024-09-11T21:48:12.322 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout Added host 'smithi001' with addr '172.21.15.1' 2024-09-11T21:48:12.322 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:07.629+0000 7f688b67a640 1 Processor -- start 2024-09-11T21:48:12.322 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:07.629+0000 7f688b67a640 1 -- start start 2024-09-11T21:48:12.322 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:07.630+0000 7f688b67a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68841059f0 0x7f6884105df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:12.322 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:07.630+0000 7f688b67a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f68841063c0 con 0x7f68841059f0 2024-09-11T21:48:12.322 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:07.630+0000 7f6889416640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68841059f0 0x7f6884105df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:48:12.322 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:07.630+0000 7f6889416640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68841059f0 0x7f6884105df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58602/0 (socket says 172.21.15.1:58602) 2024-09-11T21:48:12.322 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:07.630+0000 7f6889416640 1 -- 172.21.15.1:0/2933220865 learned_addr learned my addr 172.21.15.1:0/2933220865 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:48:12.323 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:07.630+0000 7f6889416640 1 -- 172.21.15.1:0/2933220865 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68841089d0 con 0x7f68841059f0 2024-09-11T21:48:12.323 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:07.631+0000 7f6889416640 1 --2- 172.21.15.1:0/2933220865 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68841059f0 0x7f6884105df0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6880009820 tx=0x7f688002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8574b7f54e1902b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:48:12.323 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:07.631+0000 7f687bfff640 1 -- 172.21.15.1:0/2933220865 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6880037070 con 0x7f68841059f0 2024-09-11T21:48:12.323 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:07.631+0000 7f687bfff640 1 -- 172.21.15.1:0/2933220865 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6880032900 con 0x7f68841059f0 2024-09-11T21:48:12.323 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:07.631+0000 7f687bfff640 1 -- 172.21.15.1:0/2933220865 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6880030850 con 0x7f68841059f0 2024-09-11T21:48:12.323 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:07.632+0000 7f688b67a640 1 -- 172.21.15.1:0/2933220865 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68841059f0 msgr2=0x7f6884105df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:12.323 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:07.632+0000 7f688b67a640 1 --2- 172.21.15.1:0/2933220865 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68841059f0 0x7f6884105df0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6880009820 tx=0x7f688002a270 comp rx=0 tx=0).stop 2024-09-11T21:48:12.323 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:07.632+0000 7f688b67a640 1 -- 172.21.15.1:0/2933220865 shutdown_connections 2024-09-11T21:48:12.323 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:07.632+0000 7f688b67a640 1 --2- 172.21.15.1:0/2933220865 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68841059f0 0x7f6884105df0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:12.323 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:07.632+0000 7f688b67a640 1 -- 172.21.15.1:0/2933220865 >> 172.21.15.1:0/2933220865 conn(0x7f6884101560 msgr2=0x7f6884103980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:48:12.323 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:07.632+0000 7f688b67a640 1 -- 172.21.15.1:0/2933220865 shutdown_connections 2024-09-11T21:48:12.323 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:07.632+0000 7f688b67a640 1 -- 172.21.15.1:0/2933220865 wait complete. 2024-09-11T21:48:12.324 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:07.632+0000 7f688b67a640 1 Processor -- start 2024-09-11T21:48:12.324 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:07.633+0000 7f688b67a640 1 -- start start 2024-09-11T21:48:12.324 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:07.633+0000 7f688b67a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68841059f0 0x7f688419b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:12.324 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:07.633+0000 7f688b67a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f688410bc00 con 0x7f68841059f0 2024-09-11T21:48:12.324 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:07.633+0000 7f6889416640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68841059f0 0x7f688419b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:48:12.324 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:07.633+0000 7f6889416640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68841059f0 0x7f688419b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58604/0 (socket says 172.21.15.1:58604) 2024-09-11T21:48:12.324 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:07.633+0000 7f6889416640 1 -- 172.21.15.1:0/3603120888 learned_addr learned my addr 172.21.15.1:0/3603120888 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:48:12.324 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:07.633+0000 7f6889416640 1 -- 172.21.15.1:0/3603120888 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f688419bc80 con 0x7f68841059f0 2024-09-11T21:48:12.324 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:07.633+0000 7f6889416640 1 --2- 172.21.15.1:0/3603120888 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68841059f0 0x7f688419b740 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6880009940 tx=0x7f68800326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:48:12.324 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:07.633+0000 7f687a7fc640 1 -- 172.21.15.1:0/3603120888 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6880042070 con 0x7f68841059f0 2024-09-11T21:48:12.324 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:07.633+0000 7f687a7fc640 1 -- 172.21.15.1:0/3603120888 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6880032be0 con 0x7f68841059f0 2024-09-11T21:48:12.324 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:07.633+0000 7f688b67a640 1 -- 172.21.15.1:0/3603120888 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f688419bf10 con 0x7f68841059f0 2024-09-11T21:48:12.325 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:07.633+0000 7f687a7fc640 1 -- 172.21.15.1:0/3603120888 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6880037040 con 0x7f68841059f0 2024-09-11T21:48:12.325 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:07.634+0000 7f688b67a640 1 -- 172.21.15.1:0/3603120888 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f688419ec00 con 0x7f68841059f0 2024-09-11T21:48:12.325 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:07.634+0000 7f687a7fc640 1 -- 172.21.15.1:0/3603120888 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f6880045070 con 0x7f68841059f0 2024-09-11T21:48:12.325 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:07.635+0000 7f688b67a640 1 -- 172.21.15.1:0/3603120888 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6884105e70 con 0x7f68841059f0 2024-09-11T21:48:12.325 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:07.635+0000 7f687a7fc640 1 --2- 172.21.15.1:0/3603120888 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7f6870038fc0 0x7f687003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:12.325 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:07.635+0000 7f687a7fc640 1 -- 172.21.15.1:0/3603120888 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f688006c020 con 0x7f68841059f0 2024-09-11T21:48:12.325 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:07.635+0000 7f6888c15640 1 --2- 172.21.15.1:0/3603120888 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7f6870038fc0 0x7f687003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:48:12.325 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:07.635+0000 7f6888c15640 1 --2- 172.21.15.1:0/3603120888 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7f6870038fc0 0x7f687003b480 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f68740098c0 tx=0x7f6874006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:48:12.325 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:07.639+0000 7f687a7fc640 1 -- 172.21.15.1:0/3603120888 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f688003ac60 con 0x7f68841059f0 2024-09-11T21:48:12.325 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:07.820+0000 7f688b67a640 1 -- 172.21.15.1:0/3603120888 --> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi001", "addr": "172.21.15.1", "target": ["mon-mgr", ""]}) -- 0x7f6884102820 con 0x7f6870038fc0 2024-09-11T21:48:12.325 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:11.746+0000 7f687a7fc640 1 -- 172.21.15.1:0/3603120888 <== mgr.14118 v2:172.21.15.1:6800/2821761445 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+47 (secure 0 0 0) 0x7f6884102820 con 0x7f6870038fc0 2024-09-11T21:48:12.325 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:11.749+0000 7f688b67a640 1 -- 172.21.15.1:0/3603120888 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7f6870038fc0 msgr2=0x7f687003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:12.326 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:11.749+0000 7f688b67a640 1 --2- 172.21.15.1:0/3603120888 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7f6870038fc0 0x7f687003b480 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f68740098c0 tx=0x7f6874006db0 comp rx=0 tx=0).stop 2024-09-11T21:48:12.326 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:11.749+0000 7f688b67a640 1 -- 172.21.15.1:0/3603120888 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68841059f0 msgr2=0x7f688419b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:12.326 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:11.749+0000 7f688b67a640 1 --2- 172.21.15.1:0/3603120888 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68841059f0 0x7f688419b740 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6880009940 tx=0x7f68800326e0 comp rx=0 tx=0).stop 2024-09-11T21:48:12.326 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:11.750+0000 7f688b67a640 1 -- 172.21.15.1:0/3603120888 shutdown_connections 2024-09-11T21:48:12.326 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:11.750+0000 7f688b67a640 1 --2- 172.21.15.1:0/3603120888 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7f6870038fc0 0x7f687003b480 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:12.326 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:11.750+0000 7f688b67a640 1 --2- 172.21.15.1:0/3603120888 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68841059f0 0x7f688419b740 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:12.326 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:11.750+0000 7f688b67a640 1 -- 172.21.15.1:0/3603120888 >> 172.21.15.1:0/3603120888 conn(0x7f6884101560 msgr2=0x7f6884102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:48:12.326 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:11.750+0000 7f688b67a640 1 -- 172.21.15.1:0/3603120888 shutdown_connections 2024-09-11T21:48:12.326 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:11.750+0000 7f688b67a640 1 -- 172.21.15.1:0/3603120888 wait complete. 2024-09-11T21:48:12.326 INFO:teuthology.orchestra.run.smithi001.stdout:Deploying unmanaged mon service... 2024-09-11T21:48:12.897 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:12 smithi001 ceph-mon[30935]: from='mgr.14118 172.21.15.1:0/779986856' entity='mgr.x' 2024-09-11T21:48:12.897 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:12 smithi001 ceph-mon[30935]: Added host smithi001 2024-09-11T21:48:12.897 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:12 smithi001 ceph-mon[30935]: from='mgr.14118 172.21.15.1:0/779986856' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:48:13.640 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-11T21:48:13.640 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:12.693+0000 7f2a0ad23640 1 Processor -- start 2024-09-11T21:48:13.640 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:12.693+0000 7f2a0ad23640 1 -- start start 2024-09-11T21:48:13.640 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:12.693+0000 7f2a0ad23640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a04105a10 0x7f2a04105e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:13.640 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:12.693+0000 7f2a0ad23640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2a041063e0 con 0x7f2a04105a10 2024-09-11T21:48:13.640 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:12.694+0000 7f2a08abf640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a04105a10 0x7f2a04105e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:48:13.640 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:12.694+0000 7f2a08abf640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a04105a10 0x7f2a04105e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58616/0 (socket says 172.21.15.1:58616) 2024-09-11T21:48:13.640 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:12.694+0000 7f2a08abf640 1 -- 172.21.15.1:0/129317671 learned_addr learned my addr 172.21.15.1:0/129317671 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:48:13.641 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:12.694+0000 7f2a08abf640 1 -- 172.21.15.1:0/129317671 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a04106560 con 0x7f2a04105a10 2024-09-11T21:48:13.641 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:12.695+0000 7f2a08abf640 1 --2- 172.21.15.1:0/129317671 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a04105a10 0x7f2a04105e10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f29f4009820 tx=0x7f29f402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7aaac2bcd72c03f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:48:13.641 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:12.695+0000 7f2a037fe640 1 -- 172.21.15.1:0/129317671 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f29f4037070 con 0x7f2a04105a10 2024-09-11T21:48:13.641 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:12.695+0000 7f2a037fe640 1 -- 172.21.15.1:0/129317671 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f29f4032900 con 0x7f2a04105a10 2024-09-11T21:48:13.641 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:12.695+0000 7f2a037fe640 1 -- 172.21.15.1:0/129317671 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f29f4030850 con 0x7f2a04105a10 2024-09-11T21:48:13.641 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:12.695+0000 7f2a0ad23640 1 -- 172.21.15.1:0/129317671 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a04105a10 msgr2=0x7f2a04105e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:13.641 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:12.695+0000 7f2a0ad23640 1 --2- 172.21.15.1:0/129317671 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a04105a10 0x7f2a04105e10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f29f4009820 tx=0x7f29f402a270 comp rx=0 tx=0).stop 2024-09-11T21:48:13.641 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:12.695+0000 7f2a0ad23640 1 -- 172.21.15.1:0/129317671 shutdown_connections 2024-09-11T21:48:13.641 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:12.695+0000 7f2a0ad23640 1 --2- 172.21.15.1:0/129317671 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a04105a10 0x7f2a04105e10 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:13.641 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:12.695+0000 7f2a0ad23640 1 -- 172.21.15.1:0/129317671 >> 172.21.15.1:0/129317671 conn(0x7f2a04101560 msgr2=0x7f2a04103980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:48:13.641 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:12.695+0000 7f2a0ad23640 1 -- 172.21.15.1:0/129317671 shutdown_connections 2024-09-11T21:48:13.641 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:12.696+0000 7f2a0ad23640 1 -- 172.21.15.1:0/129317671 wait complete. 2024-09-11T21:48:13.642 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:12.696+0000 7f2a0ad23640 1 Processor -- start 2024-09-11T21:48:13.642 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:12.696+0000 7f2a0ad23640 1 -- start start 2024-09-11T21:48:13.642 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:12.696+0000 7f2a0ad23640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a04105a10 0x7f2a04108310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:13.642 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:12.696+0000 7f2a0ad23640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2a041098f0 con 0x7f2a04105a10 2024-09-11T21:48:13.642 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:12.696+0000 7f2a08abf640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a04105a10 0x7f2a04108310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:48:13.642 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:12.696+0000 7f2a08abf640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a04105a10 0x7f2a04108310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58620/0 (socket says 172.21.15.1:58620) 2024-09-11T21:48:13.642 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:12.696+0000 7f2a08abf640 1 -- 172.21.15.1:0/1467675572 learned_addr learned my addr 172.21.15.1:0/1467675572 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:48:13.642 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:12.697+0000 7f2a08abf640 1 -- 172.21.15.1:0/1467675572 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a04108850 con 0x7f2a04105a10 2024-09-11T21:48:13.642 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:12.697+0000 7f2a08abf640 1 --2- 172.21.15.1:0/1467675572 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a04105a10 0x7f2a04108310 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f29f4009940 tx=0x7f29f40326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:48:13.642 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:12.697+0000 7f2a01ffb640 1 -- 172.21.15.1:0/1467675572 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f29f4042070 con 0x7f2a04105a10 2024-09-11T21:48:13.642 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:12.697+0000 7f2a01ffb640 1 -- 172.21.15.1:0/1467675572 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f29f4032be0 con 0x7f2a04105a10 2024-09-11T21:48:13.642 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:12.697+0000 7f2a01ffb640 1 -- 172.21.15.1:0/1467675572 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f29f4037040 con 0x7f2a04105a10 2024-09-11T21:48:13.642 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:12.697+0000 7f2a0ad23640 1 -- 172.21.15.1:0/1467675572 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a04106a40 con 0x7f2a04105a10 2024-09-11T21:48:13.643 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:12.697+0000 7f2a0ad23640 1 -- 172.21.15.1:0/1467675572 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a04106f80 con 0x7f2a04105a10 2024-09-11T21:48:13.643 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:12.698+0000 7f2a01ffb640 1 -- 172.21.15.1:0/1467675572 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f29f4045070 con 0x7f2a04105a10 2024-09-11T21:48:13.643 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:12.698+0000 7f2a0ad23640 1 -- 172.21.15.1:0/1467675572 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a04105e90 con 0x7f2a04105a10 2024-09-11T21:48:13.643 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:12.698+0000 7f2a01ffb640 1 --2- 172.21.15.1:0/1467675572 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7f29ec038f70 0x7f29ec03b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:13.643 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:12.698+0000 7f2a01ffb640 1 -- 172.21.15.1:0/1467675572 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f29f406bee0 con 0x7f2a04105a10 2024-09-11T21:48:13.643 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:12.698+0000 7f2a03fff640 1 --2- 172.21.15.1:0/1467675572 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7f29ec038f70 0x7f29ec03b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:48:13.643 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:12.699+0000 7f2a03fff640 1 --2- 172.21.15.1:0/1467675572 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7f29ec038f70 0x7f29ec03b430 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f29f80098c0 tx=0x7f29f8006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:48:13.643 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:12.703+0000 7f2a01ffb640 1 -- 172.21.15.1:0/1467675572 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f29f404b170 con 0x7f2a04105a10 2024-09-11T21:48:13.644 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:12.896+0000 7f2a0ad23640 1 -- 172.21.15.1:0/1467675572 --> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f2a04102820 con 0x7f29ec038f70 2024-09-11T21:48:13.644 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:12.900+0000 7f2a01ffb640 1 -- 172.21.15.1:0/1467675572 <== mgr.14118 v2:172.21.15.1:6800/2821761445 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f2a04102820 con 0x7f29ec038f70 2024-09-11T21:48:13.644 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:12.902+0000 7f2a0ad23640 1 -- 172.21.15.1:0/1467675572 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7f29ec038f70 msgr2=0x7f29ec03b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:13.644 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:12.902+0000 7f2a0ad23640 1 --2- 172.21.15.1:0/1467675572 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7f29ec038f70 0x7f29ec03b430 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f29f80098c0 tx=0x7f29f8006db0 comp rx=0 tx=0).stop 2024-09-11T21:48:13.644 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:12.902+0000 7f2a0ad23640 1 -- 172.21.15.1:0/1467675572 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a04105a10 msgr2=0x7f2a04108310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:13.644 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:12.902+0000 7f2a0ad23640 1 --2- 172.21.15.1:0/1467675572 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a04105a10 0x7f2a04108310 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f29f4009940 tx=0x7f29f40326e0 comp rx=0 tx=0).stop 2024-09-11T21:48:13.644 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:12.902+0000 7f2a0ad23640 1 -- 172.21.15.1:0/1467675572 shutdown_connections 2024-09-11T21:48:13.644 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:12.902+0000 7f2a0ad23640 1 --2- 172.21.15.1:0/1467675572 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7f29ec038f70 0x7f29ec03b430 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:13.644 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:12.902+0000 7f2a0ad23640 1 --2- 172.21.15.1:0/1467675572 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a04105a10 0x7f2a04108310 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:13.644 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:12.902+0000 7f2a0ad23640 1 -- 172.21.15.1:0/1467675572 >> 172.21.15.1:0/1467675572 conn(0x7f2a04101560 msgr2=0x7f2a04102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:48:13.644 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:12.903+0000 7f2a0ad23640 1 -- 172.21.15.1:0/1467675572 shutdown_connections 2024-09-11T21:48:13.644 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:12.903+0000 7f2a0ad23640 1 -- 172.21.15.1:0/1467675572 wait complete. 2024-09-11T21:48:13.644 INFO:teuthology.orchestra.run.smithi001.stdout:Deploying unmanaged mgr service... 2024-09-11T21:48:14.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:13 smithi001 ceph-mon[30935]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:48:14.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:13 smithi001 ceph-mon[30935]: Saving service mon spec with placement count:5 2024-09-11T21:48:14.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:13 smithi001 ceph-mon[30935]: from='mgr.14118 172.21.15.1:0/779986856' entity='mgr.x' 2024-09-11T21:48:14.861 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-11T21:48:14.861 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:14.017+0000 7fa0a426d640 1 Processor -- start 2024-09-11T21:48:14.861 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:14.017+0000 7fa0a426d640 1 -- start start 2024-09-11T21:48:14.861 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:14.017+0000 7fa0a426d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa09c108160 0x7fa09c10a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:14.861 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:14.017+0000 7fa0a426d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa09c07beb0 con 0x7fa09c108160 2024-09-11T21:48:14.861 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:14.018+0000 7fa0a2009640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa09c108160 0x7fa09c10a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:48:14.861 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:14.018+0000 7fa0a2009640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa09c108160 0x7fa09c10a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58622/0 (socket says 172.21.15.1:58622) 2024-09-11T21:48:14.862 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:14.018+0000 7fa0a2009640 1 -- 172.21.15.1:0/3629588626 learned_addr learned my addr 172.21.15.1:0/3629588626 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:48:14.862 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:14.018+0000 7fa0a2009640 1 -- 172.21.15.1:0/3629588626 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa09c10aa90 con 0x7fa09c108160 2024-09-11T21:48:14.862 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:14.019+0000 7fa0a2009640 1 --2- 172.21.15.1:0/3629588626 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa09c108160 0x7fa09c10a550 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa098009820 tx=0x7fa09802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=418bd9e702ef7b9b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:48:14.862 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:14.019+0000 7fa0a1007640 1 -- 172.21.15.1:0/3629588626 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa098037070 con 0x7fa09c108160 2024-09-11T21:48:14.862 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:14.019+0000 7fa0a1007640 1 -- 172.21.15.1:0/3629588626 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa098032900 con 0x7fa09c108160 2024-09-11T21:48:14.862 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:14.019+0000 7fa0a1007640 1 -- 172.21.15.1:0/3629588626 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa098030850 con 0x7fa09c108160 2024-09-11T21:48:14.862 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:14.019+0000 7fa0a426d640 1 -- 172.21.15.1:0/3629588626 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa09c108160 msgr2=0x7fa09c10a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:14.862 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:14.019+0000 7fa0a426d640 1 --2- 172.21.15.1:0/3629588626 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa09c108160 0x7fa09c10a550 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa098009820 tx=0x7fa09802a270 comp rx=0 tx=0).stop 2024-09-11T21:48:14.862 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:14.020+0000 7fa0a426d640 1 -- 172.21.15.1:0/3629588626 shutdown_connections 2024-09-11T21:48:14.862 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:14.020+0000 7fa0a426d640 1 --2- 172.21.15.1:0/3629588626 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa09c108160 0x7fa09c10a550 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:14.862 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:14.020+0000 7fa0a426d640 1 -- 172.21.15.1:0/3629588626 >> 172.21.15.1:0/3629588626 conn(0x7fa09c101e20 msgr2=0x7fa09c104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:48:14.862 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:14.020+0000 7fa0a426d640 1 -- 172.21.15.1:0/3629588626 shutdown_connections 2024-09-11T21:48:14.862 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:14.020+0000 7fa0a426d640 1 -- 172.21.15.1:0/3629588626 wait complete. 2024-09-11T21:48:14.863 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:14.020+0000 7fa0a426d640 1 Processor -- start 2024-09-11T21:48:14.863 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:14.020+0000 7fa0a426d640 1 -- start start 2024-09-11T21:48:14.863 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:14.020+0000 7fa0a426d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa09c108160 0x7fa09c19b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:14.863 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:14.020+0000 7fa0a426d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa09c10bb30 con 0x7fa09c108160 2024-09-11T21:48:14.863 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:14.020+0000 7fa0a2009640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa09c108160 0x7fa09c19b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:48:14.863 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:14.020+0000 7fa0a2009640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa09c108160 0x7fa09c19b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58630/0 (socket says 172.21.15.1:58630) 2024-09-11T21:48:14.863 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:14.020+0000 7fa0a2009640 1 -- 172.21.15.1:0/2398511806 learned_addr learned my addr 172.21.15.1:0/2398511806 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:48:14.863 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:14.021+0000 7fa0a2009640 1 -- 172.21.15.1:0/2398511806 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa09c19bc20 con 0x7fa09c108160 2024-09-11T21:48:14.863 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:14.021+0000 7fa0a2009640 1 --2- 172.21.15.1:0/2398511806 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa09c108160 0x7fa09c19b6e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa098031000 tx=0x7fa0980326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:48:14.863 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:14.021+0000 7fa0937fe640 1 -- 172.21.15.1:0/2398511806 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa098042070 con 0x7fa09c108160 2024-09-11T21:48:14.863 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:14.021+0000 7fa0937fe640 1 -- 172.21.15.1:0/2398511806 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa098032be0 con 0x7fa09c108160 2024-09-11T21:48:14.863 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:14.021+0000 7fa0a426d640 1 -- 172.21.15.1:0/2398511806 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa09c19beb0 con 0x7fa09c108160 2024-09-11T21:48:14.863 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:14.021+0000 7fa0937fe640 1 -- 172.21.15.1:0/2398511806 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa098037040 con 0x7fa09c108160 2024-09-11T21:48:14.864 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:14.021+0000 7fa0a426d640 1 -- 172.21.15.1:0/2398511806 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa09c19eba0 con 0x7fa09c108160 2024-09-11T21:48:14.864 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:14.022+0000 7fa0937fe640 1 -- 172.21.15.1:0/2398511806 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fa098045070 con 0x7fa09c108160 2024-09-11T21:48:14.864 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:14.022+0000 7fa0a426d640 1 -- 172.21.15.1:0/2398511806 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa09c1085c0 con 0x7fa09c108160 2024-09-11T21:48:14.864 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:14.022+0000 7fa0937fe640 1 --2- 172.21.15.1:0/2398511806 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7fa088038f70 0x7fa08803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:14.864 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:14.022+0000 7fa0937fe640 1 -- 172.21.15.1:0/2398511806 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa09806bff0 con 0x7fa09c108160 2024-09-11T21:48:14.864 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:14.023+0000 7fa0a1808640 1 --2- 172.21.15.1:0/2398511806 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7fa088038f70 0x7fa08803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:48:14.864 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:14.023+0000 7fa0a1808640 1 --2- 172.21.15.1:0/2398511806 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7fa088038f70 0x7fa08803b430 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa08c0098c0 tx=0x7fa08c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:48:14.864 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:14.027+0000 7fa0937fe640 1 -- 172.21.15.1:0/2398511806 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa098032d90 con 0x7fa09c108160 2024-09-11T21:48:14.864 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:14.216+0000 7fa0a426d640 1 -- 172.21.15.1:0/2398511806 --> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fa09c1030f0 con 0x7fa088038f70 2024-09-11T21:48:14.864 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:14.221+0000 7fa0937fe640 1 -- 172.21.15.1:0/2398511806 <== mgr.14118 v2:172.21.15.1:6800/2821761445 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fa09c1030f0 con 0x7fa088038f70 2024-09-11T21:48:14.864 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:14.222+0000 7fa0a426d640 1 -- 172.21.15.1:0/2398511806 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7fa088038f70 msgr2=0x7fa08803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:14.864 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:14.222+0000 7fa0a426d640 1 --2- 172.21.15.1:0/2398511806 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7fa088038f70 0x7fa08803b430 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa08c0098c0 tx=0x7fa08c006db0 comp rx=0 tx=0).stop 2024-09-11T21:48:14.865 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:14.222+0000 7fa0a426d640 1 -- 172.21.15.1:0/2398511806 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa09c108160 msgr2=0x7fa09c19b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:14.865 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:14.223+0000 7fa0a426d640 1 --2- 172.21.15.1:0/2398511806 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa09c108160 0x7fa09c19b6e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa098031000 tx=0x7fa0980326e0 comp rx=0 tx=0).stop 2024-09-11T21:48:14.865 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:14.223+0000 7fa0a426d640 1 -- 172.21.15.1:0/2398511806 shutdown_connections 2024-09-11T21:48:14.865 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:14.223+0000 7fa0a426d640 1 --2- 172.21.15.1:0/2398511806 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7fa088038f70 0x7fa08803b430 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:14.865 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:14.223+0000 7fa0a426d640 1 --2- 172.21.15.1:0/2398511806 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa09c108160 0x7fa09c19b6e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:14.865 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:14.223+0000 7fa0a426d640 1 -- 172.21.15.1:0/2398511806 >> 172.21.15.1:0/2398511806 conn(0x7fa09c101e20 msgr2=0x7fa09c1029e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:48:14.865 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:14.223+0000 7fa0a426d640 1 -- 172.21.15.1:0/2398511806 shutdown_connections 2024-09-11T21:48:14.865 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:14.223+0000 7fa0a426d640 1 -- 172.21.15.1:0/2398511806 wait complete. 2024-09-11T21:48:15.638 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:15 smithi001 ceph-mon[30935]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:48:15.638 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:15 smithi001 ceph-mon[30935]: Saving service mgr spec with placement count:2 2024-09-11T21:48:15.638 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:15 smithi001 ceph-mon[30935]: from='mgr.14118 172.21.15.1:0/779986856' entity='mgr.x' 2024-09-11T21:48:16.828 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:16 smithi001 ceph-mon[30935]: from='mgr.14118 172.21.15.1:0/779986856' entity='mgr.x' 2024-09-11T21:48:16.828 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:16 smithi001 ceph-mon[30935]: from='mgr.14118 172.21.15.1:0/779986856' entity='mgr.x' 2024-09-11T21:48:16.828 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:16 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/349394690' entity='client.admin' 2024-09-11T21:48:16.836 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:16.068+0000 7fbb8197b640 1 Processor -- start 2024-09-11T21:48:16.836 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:16.068+0000 7fbb8197b640 1 -- start start 2024-09-11T21:48:16.836 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:16.068+0000 7fbb8197b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbb7c1065f0 0x7fbb7c1069f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:16.836 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:16.068+0000 7fbb8197b640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fbb7c106fc0 con 0x7fbb7c1065f0 2024-09-11T21:48:16.836 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:16.068+0000 7fbb7affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbb7c1065f0 0x7fbb7c1069f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:48:16.836 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:16.068+0000 7fbb7affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbb7c1065f0 0x7fbb7c1069f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58640/0 (socket says 172.21.15.1:58640) 2024-09-11T21:48:16.836 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:16.068+0000 7fbb7affd640 1 -- 172.21.15.1:0/1624098384 learned_addr learned my addr 172.21.15.1:0/1624098384 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:48:16.836 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:16.068+0000 7fbb7affd640 1 -- 172.21.15.1:0/1624098384 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb7c1077f0 con 0x7fbb7c1065f0 2024-09-11T21:48:16.836 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:16.069+0000 7fbb7affd640 1 --2- 172.21.15.1:0/1624098384 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbb7c1065f0 0x7fbb7c1069f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c009820 tx=0x7fbb6c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70ff91c7f17b37f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:48:16.837 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:16.069+0000 7fbb7a7fc640 1 -- 172.21.15.1:0/1624098384 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb6c037070 con 0x7fbb7c1065f0 2024-09-11T21:48:16.837 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:16.069+0000 7fbb7a7fc640 1 -- 172.21.15.1:0/1624098384 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb6c032920 con 0x7fbb7c1065f0 2024-09-11T21:48:16.837 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:16.069+0000 7fbb7a7fc640 1 -- 172.21.15.1:0/1624098384 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb6c030810 con 0x7fbb7c1065f0 2024-09-11T21:48:16.837 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:16.070+0000 7fbb8197b640 1 -- 172.21.15.1:0/1624098384 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbb7c1065f0 msgr2=0x7fbb7c1069f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:16.837 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:16.070+0000 7fbb8197b640 1 --2- 172.21.15.1:0/1624098384 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbb7c1065f0 0x7fbb7c1069f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c009820 tx=0x7fbb6c02a270 comp rx=0 tx=0).stop 2024-09-11T21:48:16.837 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:16.070+0000 7fbb8197b640 1 -- 172.21.15.1:0/1624098384 shutdown_connections 2024-09-11T21:48:16.837 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:16.070+0000 7fbb8197b640 1 --2- 172.21.15.1:0/1624098384 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbb7c1065f0 0x7fbb7c1069f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:16.837 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:16.070+0000 7fbb8197b640 1 -- 172.21.15.1:0/1624098384 >> 172.21.15.1:0/1624098384 conn(0x7fbb7c101e20 msgr2=0x7fbb7c104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:48:16.837 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:16.071+0000 7fbb8197b640 1 -- 172.21.15.1:0/1624098384 shutdown_connections 2024-09-11T21:48:16.838 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:16.071+0000 7fbb8197b640 1 -- 172.21.15.1:0/1624098384 wait complete. 2024-09-11T21:48:16.838 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:16.071+0000 7fbb8197b640 1 Processor -- start 2024-09-11T21:48:16.838 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:16.071+0000 7fbb8197b640 1 -- start start 2024-09-11T21:48:16.838 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:16.071+0000 7fbb8197b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbb7c1065f0 0x7fbb7c19d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:16.838 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:16.071+0000 7fbb8197b640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fbb7c10aba0 con 0x7fbb7c1065f0 2024-09-11T21:48:16.838 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:16.071+0000 7fbb7affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbb7c1065f0 0x7fbb7c19d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:48:16.838 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:16.071+0000 7fbb7affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbb7c1065f0 0x7fbb7c19d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58648/0 (socket says 172.21.15.1:58648) 2024-09-11T21:48:16.838 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:16.071+0000 7fbb7affd640 1 -- 172.21.15.1:0/349394690 learned_addr learned my addr 172.21.15.1:0/349394690 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:48:16.838 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:16.072+0000 7fbb7affd640 1 -- 172.21.15.1:0/349394690 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb7c19dec0 con 0x7fbb7c1065f0 2024-09-11T21:48:16.838 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:16.072+0000 7fbb7affd640 1 --2- 172.21.15.1:0/349394690 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbb7c1065f0 0x7fbb7c19d980 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c0097f0 tx=0x7fbb6c0327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:48:16.838 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:16.072+0000 7fbb80979640 1 -- 172.21.15.1:0/349394690 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb6c037040 con 0x7fbb7c1065f0 2024-09-11T21:48:16.838 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:16.072+0000 7fbb8197b640 1 -- 172.21.15.1:0/349394690 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb7c19e150 con 0x7fbb7c1065f0 2024-09-11T21:48:16.839 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:16.072+0000 7fbb80979640 1 -- 172.21.15.1:0/349394690 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb6c030c40 con 0x7fbb7c1065f0 2024-09-11T21:48:16.839 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:16.072+0000 7fbb80979640 1 -- 172.21.15.1:0/349394690 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb6c03a5b0 con 0x7fbb7c1065f0 2024-09-11T21:48:16.839 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:16.072+0000 7fbb8197b640 1 -- 172.21.15.1:0/349394690 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb7c1a0e40 con 0x7fbb7c1065f0 2024-09-11T21:48:16.839 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:16.073+0000 7fbb80979640 1 -- 172.21.15.1:0/349394690 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fbb6c03a7a0 con 0x7fbb7c1065f0 2024-09-11T21:48:16.839 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:16.073+0000 7fbb80979640 1 --2- 172.21.15.1:0/349394690 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7fbb68038fc0 0x7fbb6803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:16.839 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:16.073+0000 7fbb8197b640 1 -- 172.21.15.1:0/349394690 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb7c10ad20 con 0x7fbb7c1065f0 2024-09-11T21:48:16.839 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:16.073+0000 7fbb80979640 1 -- 172.21.15.1:0/349394690 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbb6c06c370 con 0x7fbb7c1065f0 2024-09-11T21:48:16.839 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:16.074+0000 7fbb73fff640 1 --2- 172.21.15.1:0/349394690 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7fbb68038fc0 0x7fbb6803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:48:16.839 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:16.074+0000 7fbb73fff640 1 --2- 172.21.15.1:0/349394690 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7fbb68038fc0 0x7fbb6803b480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbb640098c0 tx=0x7fbb64006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:48:16.839 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:16.078+0000 7fbb80979640 1 -- 172.21.15.1:0/349394690 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbb6c0371e0 con 0x7fbb7c1065f0 2024-09-11T21:48:16.839 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:16.262+0000 7fbb8197b640 1 -- 172.21.15.1:0/349394690 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7fbb7c106a70 con 0x7fbb7c1065f0 2024-09-11T21:48:16.840 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:16.265+0000 7fbb80979640 1 -- 172.21.15.1:0/349394690 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7fbb6c02c030 con 0x7fbb7c1065f0 2024-09-11T21:48:16.840 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:16.267+0000 7fbb8197b640 1 -- 172.21.15.1:0/349394690 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7fbb68038fc0 msgr2=0x7fbb6803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:16.840 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:16.267+0000 7fbb8197b640 1 --2- 172.21.15.1:0/349394690 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7fbb68038fc0 0x7fbb6803b480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbb640098c0 tx=0x7fbb64006db0 comp rx=0 tx=0).stop 2024-09-11T21:48:16.840 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:16.267+0000 7fbb8197b640 1 -- 172.21.15.1:0/349394690 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbb7c1065f0 msgr2=0x7fbb7c19d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:16.840 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:16.267+0000 7fbb8197b640 1 --2- 172.21.15.1:0/349394690 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbb7c1065f0 0x7fbb7c19d980 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c0097f0 tx=0x7fbb6c0327b0 comp rx=0 tx=0).stop 2024-09-11T21:48:16.840 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:16.267+0000 7fbb8197b640 1 -- 172.21.15.1:0/349394690 shutdown_connections 2024-09-11T21:48:16.840 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:16.267+0000 7fbb8197b640 1 --2- 172.21.15.1:0/349394690 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7fbb68038fc0 0x7fbb6803b480 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:16.840 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:16.267+0000 7fbb8197b640 1 --2- 172.21.15.1:0/349394690 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbb7c1065f0 0x7fbb7c19d980 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:16.840 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:16.267+0000 7fbb8197b640 1 -- 172.21.15.1:0/349394690 >> 172.21.15.1:0/349394690 conn(0x7fbb7c101e20 msgr2=0x7fbb7c102d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:48:16.840 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:16.268+0000 7fbb8197b640 1 -- 172.21.15.1:0/349394690 shutdown_connections 2024-09-11T21:48:16.840 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:16.268+0000 7fbb8197b640 1 -- 172.21.15.1:0/349394690 wait complete. 2024-09-11T21:48:18.596 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:18 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/2933352645' entity='client.admin' 2024-09-11T21:48:18.597 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:18 smithi001 ceph-mon[30935]: from='mgr.14118 172.21.15.1:0/779986856' entity='mgr.x' 2024-09-11T21:48:18.604 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:17.223+0000 7f8c8d10a640 1 Processor -- start 2024-09-11T21:48:18.604 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:17.223+0000 7f8c8d10a640 1 -- start start 2024-09-11T21:48:18.604 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:17.224+0000 7f8c8d10a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8c881065f0 0x7f8c881069f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:18.604 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:17.224+0000 7f8c8d10a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8c88106fc0 con 0x7f8c881065f0 2024-09-11T21:48:18.604 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:17.224+0000 7f8c86d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8c881065f0 0x7f8c881069f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:48:18.604 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:17.224+0000 7f8c86d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8c881065f0 0x7f8c881069f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37450/0 (socket says 172.21.15.1:37450) 2024-09-11T21:48:18.604 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:17.224+0000 7f8c86d9d640 1 -- 172.21.15.1:0/1788782179 learned_addr learned my addr 172.21.15.1:0/1788782179 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:48:18.604 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:17.224+0000 7f8c86d9d640 1 -- 172.21.15.1:0/1788782179 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c881077f0 con 0x7f8c881065f0 2024-09-11T21:48:18.604 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:17.225+0000 7f8c86d9d640 1 --2- 172.21.15.1:0/1788782179 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8c881065f0 0x7f8c881069f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8c78009820 tx=0x7f8c7802c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c48538c17a90c681 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:48:18.604 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:17.225+0000 7f8c85d9b640 1 -- 172.21.15.1:0/1788782179 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c78038070 con 0x7f8c881065f0 2024-09-11T21:48:18.605 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:17.225+0000 7f8c85d9b640 1 -- 172.21.15.1:0/1788782179 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c78031b20 con 0x7f8c881065f0 2024-09-11T21:48:18.605 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:17.225+0000 7f8c85d9b640 1 -- 172.21.15.1:0/1788782179 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c78033cb0 con 0x7f8c881065f0 2024-09-11T21:48:18.605 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:17.226+0000 7f8c8d10a640 1 -- 172.21.15.1:0/1788782179 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8c881065f0 msgr2=0x7f8c881069f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:18.605 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:17.226+0000 7f8c8d10a640 1 --2- 172.21.15.1:0/1788782179 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8c881065f0 0x7f8c881069f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8c78009820 tx=0x7f8c7802c480 comp rx=0 tx=0).stop 2024-09-11T21:48:18.605 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:17.226+0000 7f8c8d10a640 1 -- 172.21.15.1:0/1788782179 shutdown_connections 2024-09-11T21:48:18.605 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:17.226+0000 7f8c8d10a640 1 --2- 172.21.15.1:0/1788782179 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8c881065f0 0x7f8c881069f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:18.605 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:17.226+0000 7f8c8d10a640 1 -- 172.21.15.1:0/1788782179 >> 172.21.15.1:0/1788782179 conn(0x7f8c88101e20 msgr2=0x7f8c88104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:48:18.605 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:17.226+0000 7f8c8d10a640 1 -- 172.21.15.1:0/1788782179 shutdown_connections 2024-09-11T21:48:18.605 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:17.226+0000 7f8c8d10a640 1 -- 172.21.15.1:0/1788782179 wait complete. 2024-09-11T21:48:18.605 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:17.227+0000 7f8c8d10a640 1 Processor -- start 2024-09-11T21:48:18.605 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:17.227+0000 7f8c8d10a640 1 -- start start 2024-09-11T21:48:18.605 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:17.227+0000 7f8c8d10a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8c881065f0 0x7f8c8819d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:18.605 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:17.227+0000 7f8c8d10a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8c8807b150 con 0x7f8c881065f0 2024-09-11T21:48:18.606 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:17.228+0000 7f8c86d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8c881065f0 0x7f8c8819d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:48:18.606 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:17.228+0000 7f8c86d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8c881065f0 0x7f8c8819d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37460/0 (socket says 172.21.15.1:37460) 2024-09-11T21:48:18.606 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:17.228+0000 7f8c86d9d640 1 -- 172.21.15.1:0/2933352645 learned_addr learned my addr 172.21.15.1:0/2933352645 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:48:18.606 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:17.228+0000 7f8c86d9d640 1 -- 172.21.15.1:0/2933352645 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c8819de60 con 0x7f8c881065f0 2024-09-11T21:48:18.606 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:17.228+0000 7f8c86d9d640 1 --2- 172.21.15.1:0/2933352645 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8c881065f0 0x7f8c8819d920 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8c780337e0 tx=0x7f8c78031740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:48:18.606 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:17.228+0000 7f8c6ffff640 1 -- 172.21.15.1:0/2933352645 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c78038070 con 0x7f8c881065f0 2024-09-11T21:48:18.606 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:17.228+0000 7f8c8d10a640 1 -- 172.21.15.1:0/2933352645 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c8819e0f0 con 0x7f8c881065f0 2024-09-11T21:48:18.606 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:17.228+0000 7f8c6ffff640 1 -- 172.21.15.1:0/2933352645 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c78031950 con 0x7f8c881065f0 2024-09-11T21:48:18.606 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:17.228+0000 7f8c6ffff640 1 -- 172.21.15.1:0/2933352645 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c7803e8a0 con 0x7f8c881065f0 2024-09-11T21:48:18.606 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:17.228+0000 7f8c8d10a640 1 -- 172.21.15.1:0/2933352645 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c881a0de0 con 0x7f8c881065f0 2024-09-11T21:48:18.606 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:17.229+0000 7f8c6ffff640 1 -- 172.21.15.1:0/2933352645 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f8c7803eb60 con 0x7f8c881065f0 2024-09-11T21:48:18.606 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:17.229+0000 7f8c6ffff640 1 --2- 172.21.15.1:0/2933352645 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7f8c70038f70 0x7f8c7003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:18.606 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:17.229+0000 7f8c6ffff640 1 -- 172.21.15.1:0/2933352645 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8c7806da70 con 0x7f8c881065f0 2024-09-11T21:48:18.607 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:17.229+0000 7f8c8659c640 1 --2- 172.21.15.1:0/2933352645 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7f8c70038f70 0x7f8c7003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:48:18.607 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:17.230+0000 7f8c8d10a640 1 -- 172.21.15.1:0/2933352645 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c881069f0 con 0x7f8c881065f0 2024-09-11T21:48:18.607 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:17.230+0000 7f8c8659c640 1 --2- 172.21.15.1:0/2933352645 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7f8c70038f70 0x7f8c7003b430 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c00ad30 tx=0x7f8c7c0093f0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:48:18.607 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:17.235+0000 7f8c6ffff640 1 -- 172.21.15.1:0/2933352645 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8c7803c340 con 0x7f8c881065f0 2024-09-11T21:48:18.607 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:17.418+0000 7f8c8d10a640 1 -- 172.21.15.1:0/2933352645 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f8c88192980 con 0x7f8c881065f0 2024-09-11T21:48:18.607 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:17.420+0000 7f8c6ffff640 1 -- 172.21.15.1:0/2933352645 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f8c78038210 con 0x7f8c881065f0 2024-09-11T21:48:18.607 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:17.422+0000 7f8c8d10a640 1 -- 172.21.15.1:0/2933352645 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7f8c70038f70 msgr2=0x7f8c7003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:18.607 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:17.422+0000 7f8c8d10a640 1 --2- 172.21.15.1:0/2933352645 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7f8c70038f70 0x7f8c7003b430 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c00ad30 tx=0x7f8c7c0093f0 comp rx=0 tx=0).stop 2024-09-11T21:48:18.607 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:17.423+0000 7f8c8d10a640 1 -- 172.21.15.1:0/2933352645 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8c881065f0 msgr2=0x7f8c8819d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:18.607 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:17.423+0000 7f8c8d10a640 1 --2- 172.21.15.1:0/2933352645 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8c881065f0 0x7f8c8819d920 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8c780337e0 tx=0x7f8c78031740 comp rx=0 tx=0).stop 2024-09-11T21:48:18.607 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:17.423+0000 7f8c8d10a640 1 -- 172.21.15.1:0/2933352645 shutdown_connections 2024-09-11T21:48:18.607 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:17.423+0000 7f8c8d10a640 1 --2- 172.21.15.1:0/2933352645 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7f8c70038f70 0x7f8c7003b430 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:18.608 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:17.423+0000 7f8c8d10a640 1 --2- 172.21.15.1:0/2933352645 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8c881065f0 0x7f8c8819d920 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:18.608 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:17.423+0000 7f8c8d10a640 1 -- 172.21.15.1:0/2933352645 >> 172.21.15.1:0/2933352645 conn(0x7f8c88101e20 msgr2=0x7f8c8818fad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:48:18.608 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:17.423+0000 7f8c8d10a640 1 -- 172.21.15.1:0/2933352645 shutdown_connections 2024-09-11T21:48:18.608 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:17.423+0000 7f8c8d10a640 1 -- 172.21.15.1:0/2933352645 wait complete. 2024-09-11T21:48:18.608 INFO:teuthology.orchestra.run.smithi001.stdout:Enabling the dashboard module... 2024-09-11T21:48:19.478 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:19 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/1064587630' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-11T21:48:19.728 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:19 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: ignoring --setuser ceph since I am not root 2024-09-11T21:48:19.728 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:19 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: ignoring --setgroup ceph since I am not root 2024-09-11T21:48:19.728 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:19 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:48:19.586+0000 7f76a6a86040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T21:48:19.728 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:19 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:48:19.646+0000 7f76a6a86040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T21:48:20.043 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:19 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:48:19.814+0000 7f76a6a86040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-11T21:48:20.043 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:19 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:48:19.884+0000 7f76a6a86040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T21:48:20.043 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:20 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:48:20.042+0000 7f76a6a86040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T21:48:20.071 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:18.971+0000 7f62b4232640 1 Processor -- start 2024-09-11T21:48:20.071 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:18.972+0000 7f62b4232640 1 -- start start 2024-09-11T21:48:20.072 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:18.972+0000 7f62b4232640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62ac108180 0x7f62ac10a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:20.072 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:18.972+0000 7f62b4232640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f62ac07beb0 con 0x7f62ac108180 2024-09-11T21:48:20.072 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:18.972+0000 7f62b1fce640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62ac108180 0x7f62ac10a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:48:20.072 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:18.972+0000 7f62b1fce640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62ac108180 0x7f62ac10a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37468/0 (socket says 172.21.15.1:37468) 2024-09-11T21:48:20.072 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:18.972+0000 7f62b1fce640 1 -- 172.21.15.1:0/2506515463 learned_addr learned my addr 172.21.15.1:0/2506515463 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:48:20.072 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:18.972+0000 7f62b1fce640 1 -- 172.21.15.1:0/2506515463 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62ac10aab0 con 0x7f62ac108180 2024-09-11T21:48:20.072 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:18.973+0000 7f62b1fce640 1 --2- 172.21.15.1:0/2506515463 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62ac108180 0x7f62ac10a570 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f62a8009820 tx=0x7f62a802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=429f2129aff91d6e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:48:20.072 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:18.973+0000 7f62b0fcc640 1 -- 172.21.15.1:0/2506515463 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f62a8037070 con 0x7f62ac108180 2024-09-11T21:48:20.072 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:18.973+0000 7f62b0fcc640 1 -- 172.21.15.1:0/2506515463 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62a8032920 con 0x7f62ac108180 2024-09-11T21:48:20.072 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:18.973+0000 7f62b0fcc640 1 -- 172.21.15.1:0/2506515463 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f62a8030810 con 0x7f62ac108180 2024-09-11T21:48:20.072 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:18.974+0000 7f62b4232640 1 -- 172.21.15.1:0/2506515463 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62ac108180 msgr2=0x7f62ac10a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:20.073 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:18.974+0000 7f62b4232640 1 --2- 172.21.15.1:0/2506515463 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62ac108180 0x7f62ac10a570 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f62a8009820 tx=0x7f62a802a270 comp rx=0 tx=0).stop 2024-09-11T21:48:20.073 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:18.974+0000 7f62b4232640 1 -- 172.21.15.1:0/2506515463 shutdown_connections 2024-09-11T21:48:20.073 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:18.974+0000 7f62b4232640 1 --2- 172.21.15.1:0/2506515463 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62ac108180 0x7f62ac10a570 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:20.073 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:18.974+0000 7f62b4232640 1 -- 172.21.15.1:0/2506515463 >> 172.21.15.1:0/2506515463 conn(0x7f62ac101da0 msgr2=0x7f62ac1041e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:48:20.073 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:18.974+0000 7f62b4232640 1 -- 172.21.15.1:0/2506515463 shutdown_connections 2024-09-11T21:48:20.073 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:18.974+0000 7f62b4232640 1 -- 172.21.15.1:0/2506515463 wait complete. 2024-09-11T21:48:20.073 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:18.975+0000 7f62b4232640 1 Processor -- start 2024-09-11T21:48:20.073 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:18.975+0000 7f62b4232640 1 -- start start 2024-09-11T21:48:20.073 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:18.975+0000 7f62b4232640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62ac108180 0x7f62ac19b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:20.073 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:18.975+0000 7f62b4232640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f62ac10bb90 con 0x7f62ac108180 2024-09-11T21:48:20.073 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:18.975+0000 7f62b1fce640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62ac108180 0x7f62ac19b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:48:20.074 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:18.975+0000 7f62b1fce640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62ac108180 0x7f62ac19b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37484/0 (socket says 172.21.15.1:37484) 2024-09-11T21:48:20.074 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:18.975+0000 7f62b1fce640 1 -- 172.21.15.1:0/1064587630 learned_addr learned my addr 172.21.15.1:0/1064587630 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:48:20.074 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:18.975+0000 7f62b1fce640 1 -- 172.21.15.1:0/1064587630 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62ac19bc40 con 0x7f62ac108180 2024-09-11T21:48:20.074 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:18.976+0000 7f62b1fce640 1 --2- 172.21.15.1:0/1064587630 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62ac108180 0x7f62ac19b700 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f62a8030420 tx=0x7f62a8032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:48:20.074 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:18.976+0000 7f62a2ffd640 1 -- 172.21.15.1:0/1064587630 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f62a8037040 con 0x7f62ac108180 2024-09-11T21:48:20.074 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:18.976+0000 7f62a2ffd640 1 -- 172.21.15.1:0/1064587630 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62a8040070 con 0x7f62ac108180 2024-09-11T21:48:20.074 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:18.976+0000 7f62b4232640 1 -- 172.21.15.1:0/1064587630 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62ac19bed0 con 0x7f62ac108180 2024-09-11T21:48:20.074 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:18.976+0000 7f62a2ffd640 1 -- 172.21.15.1:0/1064587630 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f62a803a6e0 con 0x7f62ac108180 2024-09-11T21:48:20.074 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:18.976+0000 7f62b4232640 1 -- 172.21.15.1:0/1064587630 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62ac19ebc0 con 0x7f62ac108180 2024-09-11T21:48:20.074 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:18.977+0000 7f62a2ffd640 1 -- 172.21.15.1:0/1064587630 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f62a803a9e0 con 0x7f62ac108180 2024-09-11T21:48:20.075 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:18.977+0000 7f62b4232640 1 -- 172.21.15.1:0/1064587630 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62ac1085e0 con 0x7f62ac108180 2024-09-11T21:48:20.075 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:18.977+0000 7f62a2ffd640 1 --2- 172.21.15.1:0/1064587630 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7f6298038f70 0x7f629803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:20.075 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:18.977+0000 7f62a2ffd640 1 -- 172.21.15.1:0/1064587630 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f62a806c750 con 0x7f62ac108180 2024-09-11T21:48:20.075 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:18.977+0000 7f62b17cd640 1 --2- 172.21.15.1:0/1064587630 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7f6298038f70 0x7f629803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:48:20.075 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:18.978+0000 7f62b17cd640 1 --2- 172.21.15.1:0/1064587630 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7f6298038f70 0x7f629803b430 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f629c0098c0 tx=0x7f629c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:48:20.075 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:18.982+0000 7f62a2ffd640 1 -- 172.21.15.1:0/1064587630 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f62a803add0 con 0x7f62ac108180 2024-09-11T21:48:20.075 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:19.225+0000 7f62b4232640 1 -- 172.21.15.1:0/1064587630 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f62ac19ed90 con 0x7f62ac108180 2024-09-11T21:48:20.075 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:19.424+0000 7f62a2ffd640 1 -- 172.21.15.1:0/1064587630 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f62a806ecd0 con 0x7f62ac108180 2024-09-11T21:48:20.075 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:19.426+0000 7f62a2ffd640 1 -- 172.21.15.1:0/1064587630 <== mon.0 v2:172.21.15.1:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f62a8045980 con 0x7f62ac108180 2024-09-11T21:48:20.075 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:19.427+0000 7f62b4232640 1 -- 172.21.15.1:0/1064587630 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7f6298038f70 msgr2=0x7f629803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:20.075 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:19.427+0000 7f62b4232640 1 --2- 172.21.15.1:0/1064587630 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7f6298038f70 0x7f629803b430 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f629c0098c0 tx=0x7f629c006db0 comp rx=0 tx=0).stop 2024-09-11T21:48:20.076 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:19.427+0000 7f62b4232640 1 -- 172.21.15.1:0/1064587630 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62ac108180 msgr2=0x7f62ac19b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:20.076 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:19.427+0000 7f62b4232640 1 --2- 172.21.15.1:0/1064587630 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62ac108180 0x7f62ac19b700 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f62a8030420 tx=0x7f62a8032670 comp rx=0 tx=0).stop 2024-09-11T21:48:20.076 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:19.427+0000 7f62b4232640 1 -- 172.21.15.1:0/1064587630 shutdown_connections 2024-09-11T21:48:20.076 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:19.427+0000 7f62b4232640 1 --2- 172.21.15.1:0/1064587630 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7f6298038f70 0x7f629803b430 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:20.076 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:19.427+0000 7f62b4232640 1 --2- 172.21.15.1:0/1064587630 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62ac108180 0x7f62ac19b700 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:20.076 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:19.427+0000 7f62b4232640 1 -- 172.21.15.1:0/1064587630 >> 172.21.15.1:0/1064587630 conn(0x7f62ac101da0 msgr2=0x7f62ac102910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:48:20.076 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:19.428+0000 7f62b4232640 1 -- 172.21.15.1:0/1064587630 shutdown_connections 2024-09-11T21:48:20.076 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:19.428+0000 7f62b4232640 1 -- 172.21.15.1:0/1064587630 wait complete. 2024-09-11T21:48:20.327 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:20 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:48:20.198+0000 7f76a6a86040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T21:48:20.578 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:20 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/1064587630' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-11T21:48:20.578 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:20 smithi001 ceph-mon[30935]: mgrmap e9: x(active, since 18s) 2024-09-11T21:48:20.578 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:20 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:48:20.535+0000 7f76a6a86040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T21:48:20.943 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:20 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:48:20.606+0000 7f76a6a86040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T21:48:20.943 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:20 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:48:20.801+0000 7f76a6a86040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T21:48:21.321 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout { 2024-09-11T21:48:21.327 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-11T21:48:21.327 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-11T21:48:21.327 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-11T21:48:21.327 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-11T21:48:21.327 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout } 2024-09-11T21:48:21.327 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:20.572+0000 7f190c0b8640 1 Processor -- start 2024-09-11T21:48:21.327 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:20.573+0000 7f190c0b8640 1 -- start start 2024-09-11T21:48:21.327 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:20.573+0000 7f190c0b8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1904107740 0x7f1904107b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:21.327 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:20.573+0000 7f190c0b8640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1904108110 con 0x7f1904107740 2024-09-11T21:48:21.328 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:20.573+0000 7f1909e54640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1904107740 0x7f1904107b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:48:21.328 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:20.573+0000 7f1909e54640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1904107740 0x7f1904107b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37504/0 (socket says 172.21.15.1:37504) 2024-09-11T21:48:21.328 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:20.573+0000 7f1909e54640 1 -- 172.21.15.1:0/1953703507 learned_addr learned my addr 172.21.15.1:0/1953703507 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:48:21.328 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:20.573+0000 7f1909e54640 1 -- 172.21.15.1:0/1953703507 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19041088f0 con 0x7f1904107740 2024-09-11T21:48:21.328 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:20.574+0000 7f1909e54640 1 --2- 172.21.15.1:0/1953703507 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1904107740 0x7f1904107b40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f18fc009820 tx=0x7f18fc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e6753e869d0049c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:48:21.328 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:20.574+0000 7f1908e52640 1 -- 172.21.15.1:0/1953703507 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18fc037070 con 0x7f1904107740 2024-09-11T21:48:21.328 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:20.574+0000 7f1908e52640 1 -- 172.21.15.1:0/1953703507 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18fc032920 con 0x7f1904107740 2024-09-11T21:48:21.328 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:20.574+0000 7f1908e52640 1 -- 172.21.15.1:0/1953703507 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18fc030810 con 0x7f1904107740 2024-09-11T21:48:21.328 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:20.575+0000 7f190c0b8640 1 -- 172.21.15.1:0/1953703507 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1904107740 msgr2=0x7f1904107b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:21.328 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:20.575+0000 7f190c0b8640 1 --2- 172.21.15.1:0/1953703507 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1904107740 0x7f1904107b40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f18fc009820 tx=0x7f18fc02a270 comp rx=0 tx=0).stop 2024-09-11T21:48:21.328 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:20.575+0000 7f190c0b8640 1 -- 172.21.15.1:0/1953703507 shutdown_connections 2024-09-11T21:48:21.328 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:20.575+0000 7f190c0b8640 1 --2- 172.21.15.1:0/1953703507 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1904107740 0x7f1904107b40 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:21.328 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:20.575+0000 7f190c0b8640 1 -- 172.21.15.1:0/1953703507 >> 172.21.15.1:0/1953703507 conn(0x7f1904102ef0 msgr2=0x7f1904105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:48:21.329 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:20.575+0000 7f190c0b8640 1 -- 172.21.15.1:0/1953703507 shutdown_connections 2024-09-11T21:48:21.329 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:20.575+0000 7f190c0b8640 1 -- 172.21.15.1:0/1953703507 wait complete. 2024-09-11T21:48:21.329 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:20.576+0000 7f190c0b8640 1 Processor -- start 2024-09-11T21:48:21.329 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:20.576+0000 7f190c0b8640 1 -- start start 2024-09-11T21:48:21.329 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:20.576+0000 7f190c0b8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1904107740 0x7f190407db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:21.329 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:20.576+0000 7f190c0b8640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f190410bca0 con 0x7f1904107740 2024-09-11T21:48:21.329 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:20.576+0000 7f1909e54640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1904107740 0x7f190407db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:48:21.329 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:20.576+0000 7f1909e54640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1904107740 0x7f190407db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37510/0 (socket says 172.21.15.1:37510) 2024-09-11T21:48:21.329 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:20.576+0000 7f1909e54640 1 -- 172.21.15.1:0/4188136833 learned_addr learned my addr 172.21.15.1:0/4188136833 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:48:21.329 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:20.577+0000 7f1909e54640 1 -- 172.21.15.1:0/4188136833 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f190407e0b0 con 0x7f1904107740 2024-09-11T21:48:21.329 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:20.577+0000 7f1909e54640 1 --2- 172.21.15.1:0/4188136833 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1904107740 0x7f190407db70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f18fc0097f0 tx=0x7f18fc0327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:48:21.329 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:20.577+0000 7f1902ffd640 1 -- 172.21.15.1:0/4188136833 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18fc037040 con 0x7f1904107740 2024-09-11T21:48:21.329 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:20.577+0000 7f1902ffd640 1 -- 172.21.15.1:0/4188136833 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18fc030c40 con 0x7f1904107740 2024-09-11T21:48:21.330 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:20.577+0000 7f190c0b8640 1 -- 172.21.15.1:0/4188136833 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f190407c2a0 con 0x7f1904107740 2024-09-11T21:48:21.330 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:20.577+0000 7f1902ffd640 1 -- 172.21.15.1:0/4188136833 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18fc03a5b0 con 0x7f1904107740 2024-09-11T21:48:21.330 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:20.577+0000 7f190c0b8640 1 -- 172.21.15.1:0/4188136833 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f190407c7e0 con 0x7f1904107740 2024-09-11T21:48:21.330 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:20.578+0000 7f190c0b8640 1 -- 172.21.15.1:0/4188136833 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f190410be20 con 0x7f1904107740 2024-09-11T21:48:21.330 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:20.578+0000 7f1902ffd640 1 -- 172.21.15.1:0/4188136833 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f18fc03a7a0 con 0x7f1904107740 2024-09-11T21:48:21.330 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:20.579+0000 7f1902ffd640 1 --2- 172.21.15.1:0/4188136833 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7f18ec039020 0x7f18ec03b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:21.330 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:20.579+0000 7f1902ffd640 1 -- 172.21.15.1:0/4188136833 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f18fc06c440 con 0x7f1904107740 2024-09-11T21:48:21.330 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:20.579+0000 7f1909653640 1 -- 172.21.15.1:0/4188136833 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7f18ec039020 msgr2=0x7f18ec03b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/2821761445 2024-09-11T21:48:21.330 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:20.579+0000 7f1909653640 1 --2- 172.21.15.1:0/4188136833 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7f18ec039020 0x7f18ec03b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T21:48:21.330 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:20.587+0000 7f1902ffd640 1 -- 172.21.15.1:0/4188136833 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f18fc0371e0 con 0x7f1904107740 2024-09-11T21:48:21.330 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:20.780+0000 7f1909653640 1 -- 172.21.15.1:0/4188136833 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7f18ec039020 msgr2=0x7f18ec03b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/2821761445 2024-09-11T21:48:21.330 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:20.780+0000 7f1909653640 1 --2- 172.21.15.1:0/4188136833 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7f18ec039020 0x7f18ec03b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-11T21:48:21.330 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:20.812+0000 7f190c0b8640 1 -- 172.21.15.1:0/4188136833 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f190407cf70 con 0x7f1904107740 2024-09-11T21:48:21.330 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:20.813+0000 7f1902ffd640 1 -- 172.21.15.1:0/4188136833 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f18fc06be10 con 0x7f1904107740 2024-09-11T21:48:21.331 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:20.815+0000 7f190c0b8640 1 -- 172.21.15.1:0/4188136833 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7f18ec039020 msgr2=0x7f18ec03b4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T21:48:21.331 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:20.815+0000 7f190c0b8640 1 --2- 172.21.15.1:0/4188136833 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7f18ec039020 0x7f18ec03b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:21.331 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:20.815+0000 7f190c0b8640 1 -- 172.21.15.1:0/4188136833 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1904107740 msgr2=0x7f190407db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:21.331 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:20.815+0000 7f190c0b8640 1 --2- 172.21.15.1:0/4188136833 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1904107740 0x7f190407db70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f18fc0097f0 tx=0x7f18fc0327b0 comp rx=0 tx=0).stop 2024-09-11T21:48:21.331 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:20.815+0000 7f190c0b8640 1 -- 172.21.15.1:0/4188136833 shutdown_connections 2024-09-11T21:48:21.331 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:20.815+0000 7f190c0b8640 1 --2- 172.21.15.1:0/4188136833 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7f18ec039020 0x7f18ec03b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:21.331 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:20.815+0000 7f190c0b8640 1 --2- 172.21.15.1:0/4188136833 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1904107740 0x7f190407db70 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:21.331 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:20.815+0000 7f190c0b8640 1 -- 172.21.15.1:0/4188136833 >> 172.21.15.1:0/4188136833 conn(0x7f1904102ef0 msgr2=0x7f1904103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:48:21.331 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:20.815+0000 7f190c0b8640 1 -- 172.21.15.1:0/4188136833 shutdown_connections 2024-09-11T21:48:21.331 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:20.815+0000 7f190c0b8640 1 -- 172.21.15.1:0/4188136833 wait complete. 2024-09-11T21:48:21.331 INFO:teuthology.orchestra.run.smithi001.stdout:Waiting for the mgr to restart... 2024-09-11T21:48:21.331 INFO:teuthology.orchestra.run.smithi001.stdout:Waiting for mgr epoch 9... 2024-09-11T21:48:21.540 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:21 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/4188136833' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-11T21:48:21.815 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:21 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:48:21.744+0000 7f76a6a86040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T21:48:22.193 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:21 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:48:21.814+0000 7f76a6a86040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T21:48:22.680 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:22 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:48:22.559+0000 7f76a6a86040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T21:48:22.680 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:22 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:48:22.679+0000 7f76a6a86040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T21:48:22.943 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:22 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:48:22.829+0000 7f76a6a86040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T21:48:22.943 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:22 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:48:22.892+0000 7f76a6a86040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T21:48:23.349 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:23 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:48:23.049+0000 7f76a6a86040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T21:48:23.349 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:23 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:48:23.120+0000 7f76a6a86040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T21:48:23.350 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:23 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:48:23.182+0000 7f76a6a86040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T21:48:23.350 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:23 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:48:23.348+0000 7f76a6a86040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T21:48:23.635 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:23 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:48:23.416+0000 7f76a6a86040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T21:48:23.635 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:23 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:48:23.486+0000 7f76a6a86040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T21:48:23.635 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:23 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:48:23.572+0000 7f76a6a86040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T21:48:23.635 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:23 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:48:23.634+0000 7f76a6a86040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T21:48:23.943 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:23 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:48:23.696+0000 7f76a6a86040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T21:48:24.778 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:24 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:48:24.522+0000 7f76a6a86040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T21:48:24.779 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:24 smithi001 ceph-mon[30935]: Active manager daemon x restarted 2024-09-11T21:48:24.779 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:24 smithi001 ceph-mon[30935]: Activating manager daemon x 2024-09-11T21:48:24.779 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:24 smithi001 ceph-mon[30935]: osdmap e3: 0 total, 0 up, 0 in 2024-09-11T21:48:24.779 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:24 smithi001 ceph-mon[30935]: mgrmap e10: x(active, starting, since 0.00343499s) 2024-09-11T21:48:24.779 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:24 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T21:48:24.779 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:24 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T21:48:24.779 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:24 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T21:48:24.779 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:24 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T21:48:24.779 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:24 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T21:48:24.780 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:24 smithi001 ceph-mon[30935]: Manager daemon x is now available 2024-09-11T21:48:24.780 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:24 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:48:24.780 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:24 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-11T21:48:24.780 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:24 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-11T21:48:26.443 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:26 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:48:26.423+0000 7f760c54f640 -1 log_channel(cephadm) log [ERR] : [11/Sep/2024:21:48:26] ENGINE Error in HTTPServer.serve 2024-09-11T21:48:26.444 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:26 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: Traceback (most recent call last): 2024-09-11T21:48:26.444 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:26 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T21:48:26.444 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:26 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: self._connections.run(self.expiration_interval) 2024-09-11T21:48:26.444 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:26 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T21:48:26.444 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:26 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: self._run(expiration_interval) 2024-09-11T21:48:26.444 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:26 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T21:48:26.444 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:26 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T21:48:26.444 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:26 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T21:48:26.444 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:26 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T21:48:26.444 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:26 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T21:48:26.445 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:26 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: s = self.context.wrap_socket( 2024-09-11T21:48:26.445 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:26 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T21:48:26.445 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:26 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: return self.sslsocket_class._create( 2024-09-11T21:48:26.445 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:26 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T21:48:26.445 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:26 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: self.do_handshake() 2024-09-11T21:48:26.445 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:26 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T21:48:26.445 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:26 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: self._sslobj.do_handshake() 2024-09-11T21:48:26.445 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:26 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T21:48:26.445 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:48:26 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:48:26.480 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout { 2024-09-11T21:48:26.487 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-11T21:48:26.488 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-11T21:48:26.488 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout } 2024-09-11T21:48:26.488 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:21.711+0000 7fd6b8f46640 1 Processor -- start 2024-09-11T21:48:26.488 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:21.711+0000 7fd6b8f46640 1 -- start start 2024-09-11T21:48:26.488 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:21.711+0000 7fd6b8f46640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd6b4106620 0x7fd6b4106a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:26.488 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:21.711+0000 7fd6b8f46640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b4106ff0 con 0x7fd6b4106620 2024-09-11T21:48:26.488 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:21.711+0000 7fd6b259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd6b4106620 0x7fd6b4106a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:48:26.488 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:21.711+0000 7fd6b259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd6b4106620 0x7fd6b4106a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37520/0 (socket says 172.21.15.1:37520) 2024-09-11T21:48:26.488 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:21.711+0000 7fd6b259c640 1 -- 172.21.15.1:0/968028500 learned_addr learned my addr 172.21.15.1:0/968028500 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:48:26.488 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:21.712+0000 7fd6b259c640 1 -- 172.21.15.1:0/968028500 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6b41077d0 con 0x7fd6b4106620 2024-09-11T21:48:26.489 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:21.712+0000 7fd6b259c640 1 --2- 172.21.15.1:0/968028500 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd6b4106620 0x7fd6b4106a20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd6a4009820 tx=0x7fd6a402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed7c3066b2ae36dd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:48:26.489 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:21.712+0000 7fd6b1d9b640 1 -- 172.21.15.1:0/968028500 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd6a4037070 con 0x7fd6b4106620 2024-09-11T21:48:26.489 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:21.712+0000 7fd6b1d9b640 1 -- 172.21.15.1:0/968028500 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6a4032920 con 0x7fd6b4106620 2024-09-11T21:48:26.489 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:21.713+0000 7fd6b1d9b640 1 -- 172.21.15.1:0/968028500 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd6a4030810 con 0x7fd6b4106620 2024-09-11T21:48:26.489 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:21.713+0000 7fd6b8f46640 1 -- 172.21.15.1:0/968028500 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd6b4106620 msgr2=0x7fd6b4106a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:26.489 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:21.713+0000 7fd6b8f46640 1 --2- 172.21.15.1:0/968028500 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd6b4106620 0x7fd6b4106a20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd6a4009820 tx=0x7fd6a402a270 comp rx=0 tx=0).stop 2024-09-11T21:48:26.489 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:21.713+0000 7fd6b8f46640 1 -- 172.21.15.1:0/968028500 shutdown_connections 2024-09-11T21:48:26.489 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:21.713+0000 7fd6b8f46640 1 --2- 172.21.15.1:0/968028500 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd6b4106620 0x7fd6b4106a20 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:26.489 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:21.713+0000 7fd6b8f46640 1 -- 172.21.15.1:0/968028500 >> 172.21.15.1:0/968028500 conn(0x7fd6b4101db0 msgr2=0x7fd6b41041f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:48:26.489 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:21.713+0000 7fd6b8f46640 1 -- 172.21.15.1:0/968028500 shutdown_connections 2024-09-11T21:48:26.490 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:21.713+0000 7fd6b8f46640 1 -- 172.21.15.1:0/968028500 wait complete. 2024-09-11T21:48:26.490 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:21.714+0000 7fd6b8f46640 1 Processor -- start 2024-09-11T21:48:26.490 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:21.714+0000 7fd6b8f46640 1 -- start start 2024-09-11T21:48:26.490 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:21.714+0000 7fd6b8f46640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd6b4106620 0x7fd6b419b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:26.490 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:21.714+0000 7fd6b8f46640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b410ab80 con 0x7fd6b4106620 2024-09-11T21:48:26.490 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:21.714+0000 7fd6b259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd6b4106620 0x7fd6b419b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:48:26.490 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:21.714+0000 7fd6b259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd6b4106620 0x7fd6b419b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37524/0 (socket says 172.21.15.1:37524) 2024-09-11T21:48:26.490 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:21.714+0000 7fd6b259c640 1 -- 172.21.15.1:0/1795353933 learned_addr learned my addr 172.21.15.1:0/1795353933 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:48:26.490 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:21.715+0000 7fd6b259c640 1 -- 172.21.15.1:0/1795353933 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6b419bc80 con 0x7fd6b4106620 2024-09-11T21:48:26.490 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:21.715+0000 7fd6b259c640 1 --2- 172.21.15.1:0/1795353933 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd6b4106620 0x7fd6b419b740 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd6a4030560 tx=0x7fd6a4032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:48:26.490 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:21.715+0000 7fd6ab7fe640 1 -- 172.21.15.1:0/1795353933 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd6a4037040 con 0x7fd6b4106620 2024-09-11T21:48:26.490 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:21.715+0000 7fd6b8f46640 1 -- 172.21.15.1:0/1795353933 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6b419bf10 con 0x7fd6b4106620 2024-09-11T21:48:26.490 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:21.715+0000 7fd6ab7fe640 1 -- 172.21.15.1:0/1795353933 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6a4040070 con 0x7fd6b4106620 2024-09-11T21:48:26.491 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:21.715+0000 7fd6ab7fe640 1 -- 172.21.15.1:0/1795353933 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd6a403a6d0 con 0x7fd6b4106620 2024-09-11T21:48:26.491 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:21.715+0000 7fd6b8f46640 1 -- 172.21.15.1:0/1795353933 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6b419ec00 con 0x7fd6b4106620 2024-09-11T21:48:26.491 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:21.716+0000 7fd6ab7fe640 1 -- 172.21.15.1:0/1795353933 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fd6a403a940 con 0x7fd6b4106620 2024-09-11T21:48:26.491 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:21.716+0000 7fd6ab7fe640 1 --2- 172.21.15.1:0/1795353933 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7fd6a0038fc0 0x7fd6a003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:26.491 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:21.716+0000 7fd6ab7fe640 1 -- 172.21.15.1:0/1795353933 --> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd6a003bb90 con 0x7fd6a0038fc0 2024-09-11T21:48:26.491 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:21.716+0000 7fd6ab7fe640 1 -- 172.21.15.1:0/1795353933 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd6a406c580 con 0x7fd6b4106620 2024-09-11T21:48:26.491 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:21.716+0000 7fd6abfff640 1 -- 172.21.15.1:0/1795353933 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7fd6a0038fc0 msgr2=0x7fd6a003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/2821761445 2024-09-11T21:48:26.491 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:21.716+0000 7fd6abfff640 1 --2- 172.21.15.1:0/1795353933 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7fd6a0038fc0 0x7fd6a003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T21:48:26.491 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:21.917+0000 7fd6abfff640 1 -- 172.21.15.1:0/1795353933 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7fd6a0038fc0 msgr2=0x7fd6a003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/2821761445 2024-09-11T21:48:26.491 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:21.917+0000 7fd6abfff640 1 --2- 172.21.15.1:0/1795353933 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7fd6a0038fc0 0x7fd6a003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-11T21:48:26.491 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:22.318+0000 7fd6abfff640 1 -- 172.21.15.1:0/1795353933 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7fd6a0038fc0 msgr2=0x7fd6a003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/2821761445 2024-09-11T21:48:26.491 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:22.318+0000 7fd6abfff640 1 --2- 172.21.15.1:0/1795353933 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7fd6a0038fc0 0x7fd6a003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-11T21:48:26.491 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:23.119+0000 7fd6abfff640 1 -- 172.21.15.1:0/1795353933 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7fd6a0038fc0 msgr2=0x7fd6a003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/2821761445 2024-09-11T21:48:26.492 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:23.119+0000 7fd6abfff640 1 --2- 172.21.15.1:0/1795353933 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7fd6a0038fc0 0x7fd6a003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-11T21:48:26.492 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:24.526+0000 7fd6ab7fe640 1 -- 172.21.15.1:0/1795353933 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7fd6a406b030 con 0x7fd6b4106620 2024-09-11T21:48:26.492 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:24.526+0000 7fd6ab7fe640 1 -- 172.21.15.1:0/1795353933 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7fd6a0038fc0 msgr2=0x7fd6a003b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T21:48:26.492 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:24.526+0000 7fd6ab7fe640 1 --2- 172.21.15.1:0/1795353933 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7fd6a0038fc0 0x7fd6a003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:26.492 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:25.528+0000 7fd6ab7fe640 1 -- 172.21.15.1:0/1795353933 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7fd6a4045460 con 0x7fd6b4106620 2024-09-11T21:48:26.492 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:25.528+0000 7fd6ab7fe640 1 --2- 172.21.15.1:0/1795353933 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fd6a003ca80 0x7fd6a003ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:26.492 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:25.528+0000 7fd6ab7fe640 1 -- 172.21.15.1:0/1795353933 --> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd6a003bb90 con 0x7fd6a003ca80 2024-09-11T21:48:26.492 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:25.528+0000 7fd6abfff640 1 --2- 172.21.15.1:0/1795353933 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fd6a003ca80 0x7fd6a003ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:48:26.492 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:25.529+0000 7fd6abfff640 1 --2- 172.21.15.1:0/1795353933 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fd6a003ca80 0x7fd6a003ee70 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd69c003980 tx=0x7fd69c006e10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:48:26.492 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:25.529+0000 7fd6ab7fe640 1 -- 172.21.15.1:0/1795353933 <== mgr.14150 v2:172.21.15.1:6800/3890350886 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fd6a003bb90 con 0x7fd6a003ca80 2024-09-11T21:48:26.492 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:25.537+0000 7fd6b8f46640 1 -- 172.21.15.1:0/1795353933 --> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fd6b410ad00 con 0x7fd6a003ca80 2024-09-11T21:48:26.492 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:25.537+0000 7fd6ab7fe640 1 -- 172.21.15.1:0/1795353933 <== mgr.14150 v2:172.21.15.1:6800/3890350886 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7fd6b410ad00 con 0x7fd6a003ca80 2024-09-11T21:48:26.492 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:25.537+0000 7fd6b8f46640 1 -- 172.21.15.1:0/1795353933 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fd6a003ca80 msgr2=0x7fd6a003ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:26.493 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:25.537+0000 7fd6b8f46640 1 --2- 172.21.15.1:0/1795353933 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fd6a003ca80 0x7fd6a003ee70 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd69c003980 tx=0x7fd69c006e10 comp rx=0 tx=0).stop 2024-09-11T21:48:26.493 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:25.537+0000 7fd6b8f46640 1 -- 172.21.15.1:0/1795353933 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd6b4106620 msgr2=0x7fd6b419b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:26.493 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:25.537+0000 7fd6b8f46640 1 --2- 172.21.15.1:0/1795353933 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd6b4106620 0x7fd6b419b740 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd6a4030560 tx=0x7fd6a4032670 comp rx=0 tx=0).stop 2024-09-11T21:48:26.493 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:25.537+0000 7fd6b8f46640 1 -- 172.21.15.1:0/1795353933 shutdown_connections 2024-09-11T21:48:26.493 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:25.537+0000 7fd6b8f46640 1 --2- 172.21.15.1:0/1795353933 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fd6a003ca80 0x7fd6a003ee70 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:26.493 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:25.537+0000 7fd6b8f46640 1 --2- 172.21.15.1:0/1795353933 >> [v2:172.21.15.1:6800/2821761445,v1:172.21.15.1:6801/2821761445] conn(0x7fd6a0038fc0 0x7fd6a003b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:26.493 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:25.537+0000 7fd6b8f46640 1 --2- 172.21.15.1:0/1795353933 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd6b4106620 0x7fd6b419b740 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:26.493 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:25.537+0000 7fd6b8f46640 1 -- 172.21.15.1:0/1795353933 >> 172.21.15.1:0/1795353933 conn(0x7fd6b4101db0 msgr2=0x7fd6b4102d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:48:26.493 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:25.537+0000 7fd6b8f46640 1 -- 172.21.15.1:0/1795353933 shutdown_connections 2024-09-11T21:48:26.493 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:25.537+0000 7fd6b8f46640 1 -- 172.21.15.1:0/1795353933 wait complete. 2024-09-11T21:48:26.493 INFO:teuthology.orchestra.run.smithi001.stdout:mgr epoch 9 is available 2024-09-11T21:48:26.493 INFO:teuthology.orchestra.run.smithi001.stdout:Generating a dashboard self-signed certificate... 2024-09-11T21:48:26.798 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:26 smithi001 ceph-mon[30935]: mgrmap e11: x(active, since 1.00636s) 2024-09-11T21:48:26.798 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:26 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:48:26.798 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:26 smithi001 ceph-mon[30935]: [11/Sep/2024:21:48:26] ENGINE Bus STARTING 2024-09-11T21:48:26.798 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:26 smithi001 ceph-mon[30935]: [11/Sep/2024:21:48:26] ENGINE Serving on http://172.21.15.1:8765 2024-09-11T21:48:26.799 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:26 smithi001 ceph-mon[30935]: [11/Sep/2024:21:48:26] ENGINE Serving on https://172.21.15.1:7150 2024-09-11T21:48:26.799 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:26 smithi001 ceph-mon[30935]: [11/Sep/2024:21:48:26] ENGINE Bus STARTED 2024-09-11T21:48:26.799 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:26 smithi001 ceph-mon[30935]: [11/Sep/2024:21:48:26] ENGINE Error in HTTPServer.serve 2024-09-11T21:48:26.799 INFO:journalctl@ceph.mon.a.smithi001.stdout: Traceback (most recent call last): 2024-09-11T21:48:26.799 INFO:journalctl@ceph.mon.a.smithi001.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T21:48:26.799 INFO:journalctl@ceph.mon.a.smithi001.stdout: self._connections.run(self.expiration_interval) 2024-09-11T21:48:26.799 INFO:journalctl@ceph.mon.a.smithi001.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T21:48:26.799 INFO:journalctl@ceph.mon.a.smithi001.stdout: self._run(expiration_interval) 2024-09-11T21:48:26.799 INFO:journalctl@ceph.mon.a.smithi001.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T21:48:26.799 INFO:journalctl@ceph.mon.a.smithi001.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T21:48:26.799 INFO:journalctl@ceph.mon.a.smithi001.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T21:48:26.799 INFO:journalctl@ceph.mon.a.smithi001.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T21:48:26.799 INFO:journalctl@ceph.mon.a.smithi001.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T21:48:26.799 INFO:journalctl@ceph.mon.a.smithi001.stdout: s = self.context.wrap_socket( 2024-09-11T21:48:26.800 INFO:journalctl@ceph.mon.a.smithi001.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T21:48:26.800 INFO:journalctl@ceph.mon.a.smithi001.stdout: return self.sslsocket_class._create( 2024-09-11T21:48:26.800 INFO:journalctl@ceph.mon.a.smithi001.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T21:48:26.800 INFO:journalctl@ceph.mon.a.smithi001.stdout: self.do_handshake() 2024-09-11T21:48:26.800 INFO:journalctl@ceph.mon.a.smithi001.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T21:48:26.800 INFO:journalctl@ceph.mon.a.smithi001.stdout: self._sslobj.do_handshake() 2024-09-11T21:48:26.800 INFO:journalctl@ceph.mon.a.smithi001.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T21:48:27.826 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-11T21:48:27.826 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:26.876+0000 7f0ebf340640 1 Processor -- start 2024-09-11T21:48:27.826 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:26.877+0000 7f0ebf340640 1 -- start start 2024-09-11T21:48:27.826 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:26.877+0000 7f0ebf340640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0eb81081e0 0x7f0eb810a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:27.827 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:26.877+0000 7f0ebf340640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb807be50 con 0x7f0eb81081e0 2024-09-11T21:48:27.827 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:26.877+0000 7f0ebd0dc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0eb81081e0 0x7f0eb810a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:48:27.827 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:26.877+0000 7f0ebd0dc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0eb81081e0 0x7f0eb810a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51514/0 (socket says 172.21.15.1:51514) 2024-09-11T21:48:27.827 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:26.877+0000 7f0ebd0dc640 1 -- 172.21.15.1:0/815386212 learned_addr learned my addr 172.21.15.1:0/815386212 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:48:27.827 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:26.877+0000 7f0ebd0dc640 1 -- 172.21.15.1:0/815386212 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0eb810ab10 con 0x7f0eb81081e0 2024-09-11T21:48:27.827 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:26.878+0000 7f0ebd0dc640 1 --2- 172.21.15.1:0/815386212 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0eb81081e0 0x7f0eb810a5d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0eb4009820 tx=0x7f0eb402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5146feb4d30a382 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:48:27.827 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:26.878+0000 7f0eabfff640 1 -- 172.21.15.1:0/815386212 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0eb4037070 con 0x7f0eb81081e0 2024-09-11T21:48:27.827 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:26.878+0000 7f0eabfff640 1 -- 172.21.15.1:0/815386212 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0eb4032920 con 0x7f0eb81081e0 2024-09-11T21:48:27.827 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:26.878+0000 7f0eabfff640 1 -- 172.21.15.1:0/815386212 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0eb4030810 con 0x7f0eb81081e0 2024-09-11T21:48:27.827 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:26.879+0000 7f0ebf340640 1 -- 172.21.15.1:0/815386212 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0eb81081e0 msgr2=0x7f0eb810a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:27.827 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:26.879+0000 7f0ebf340640 1 --2- 172.21.15.1:0/815386212 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0eb81081e0 0x7f0eb810a5d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0eb4009820 tx=0x7f0eb402a270 comp rx=0 tx=0).stop 2024-09-11T21:48:27.827 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:26.879+0000 7f0ebf340640 1 -- 172.21.15.1:0/815386212 shutdown_connections 2024-09-11T21:48:27.828 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:26.879+0000 7f0ebf340640 1 --2- 172.21.15.1:0/815386212 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0eb81081e0 0x7f0eb810a5d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:27.828 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:26.879+0000 7f0ebf340640 1 -- 172.21.15.1:0/815386212 >> 172.21.15.1:0/815386212 conn(0x7f0eb8101da0 msgr2=0x7f0eb81041e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:48:27.828 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:26.879+0000 7f0ebf340640 1 -- 172.21.15.1:0/815386212 shutdown_connections 2024-09-11T21:48:27.828 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:26.879+0000 7f0ebf340640 1 -- 172.21.15.1:0/815386212 wait complete. 2024-09-11T21:48:27.828 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:26.879+0000 7f0ebf340640 1 Processor -- start 2024-09-11T21:48:27.828 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:26.880+0000 7f0ebf340640 1 -- start start 2024-09-11T21:48:27.828 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:26.880+0000 7f0ebf340640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0eb81081e0 0x7f0eb819b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:27.828 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:26.880+0000 7f0ebf340640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb810bb90 con 0x7f0eb81081e0 2024-09-11T21:48:27.828 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:26.880+0000 7f0ebd0dc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0eb81081e0 0x7f0eb819b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:48:27.828 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:26.880+0000 7f0ebd0dc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0eb81081e0 0x7f0eb819b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51516/0 (socket says 172.21.15.1:51516) 2024-09-11T21:48:27.828 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:26.880+0000 7f0ebd0dc640 1 -- 172.21.15.1:0/1928474108 learned_addr learned my addr 172.21.15.1:0/1928474108 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:48:27.828 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:26.880+0000 7f0ebd0dc640 1 -- 172.21.15.1:0/1928474108 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0eb819bc70 con 0x7f0eb81081e0 2024-09-11T21:48:27.828 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:26.880+0000 7f0ebd0dc640 1 --2- 172.21.15.1:0/1928474108 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0eb81081e0 0x7f0eb819b730 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0eb4030420 tx=0x7f0eb4032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:48:27.829 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:26.880+0000 7f0eaa7fc640 1 -- 172.21.15.1:0/1928474108 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0eb4037040 con 0x7f0eb81081e0 2024-09-11T21:48:27.829 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:26.880+0000 7f0eaa7fc640 1 -- 172.21.15.1:0/1928474108 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0eb4040070 con 0x7f0eb81081e0 2024-09-11T21:48:27.829 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:26.880+0000 7f0ebf340640 1 -- 172.21.15.1:0/1928474108 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0eb819bf00 con 0x7f0eb81081e0 2024-09-11T21:48:27.829 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:26.880+0000 7f0eaa7fc640 1 -- 172.21.15.1:0/1928474108 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0eb403a6e0 con 0x7f0eb81081e0 2024-09-11T21:48:27.829 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:26.881+0000 7f0ebf340640 1 -- 172.21.15.1:0/1928474108 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0eb819ebf0 con 0x7f0eb81081e0 2024-09-11T21:48:27.829 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:26.881+0000 7f0eaa7fc640 1 -- 172.21.15.1:0/1928474108 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f0eb403a990 con 0x7f0eb81081e0 2024-09-11T21:48:27.829 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:26.882+0000 7f0eaa7fc640 1 --2- 172.21.15.1:0/1928474108 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f0ea4038f00 0x7f0ea403b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:27.829 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:26.882+0000 7f0eaa7fc640 1 -- 172.21.15.1:0/1928474108 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0eb406c4d0 con 0x7f0eb81081e0 2024-09-11T21:48:27.829 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:26.882+0000 7f0ebc8db640 1 --2- 172.21.15.1:0/1928474108 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f0ea4038f00 0x7f0ea403b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:48:27.829 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:26.882+0000 7f0ebf340640 1 -- 172.21.15.1:0/1928474108 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0eb8108640 con 0x7f0eb81081e0 2024-09-11T21:48:27.829 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:26.882+0000 7f0ebc8db640 1 --2- 172.21.15.1:0/1928474108 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f0ea4038f00 0x7f0ea403b3c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0eac0098c0 tx=0x7f0eac006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:48:27.830 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:26.887+0000 7f0eaa7fc640 1 -- 172.21.15.1:0/1928474108 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0eb404a200 con 0x7f0eb81081e0 2024-09-11T21:48:27.830 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:26.890+0000 7f0eaa7fc640 1 -- 172.21.15.1:0/1928474108 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f0eb403bb20 con 0x7f0eb81081e0 2024-09-11T21:48:27.830 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:27.088+0000 7f0ebf340640 1 -- 172.21.15.1:0/1928474108 --> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f0eb8103020 con 0x7f0ea4038f00 2024-09-11T21:48:27.830 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:27.226+0000 7f0eaa7fc640 1 -- 172.21.15.1:0/1928474108 <== mgr.14150 v2:172.21.15.1:6800/3890350886 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f0eb8103020 con 0x7f0ea4038f00 2024-09-11T21:48:27.830 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:27.229+0000 7f0ebf340640 1 -- 172.21.15.1:0/1928474108 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f0ea4038f00 msgr2=0x7f0ea403b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:27.830 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:27.229+0000 7f0ebf340640 1 --2- 172.21.15.1:0/1928474108 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f0ea4038f00 0x7f0ea403b3c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0eac0098c0 tx=0x7f0eac006db0 comp rx=0 tx=0).stop 2024-09-11T21:48:27.830 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:27.229+0000 7f0ebf340640 1 -- 172.21.15.1:0/1928474108 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0eb81081e0 msgr2=0x7f0eb819b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:27.830 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:27.229+0000 7f0ebf340640 1 --2- 172.21.15.1:0/1928474108 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0eb81081e0 0x7f0eb819b730 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0eb4030420 tx=0x7f0eb4032670 comp rx=0 tx=0).stop 2024-09-11T21:48:27.830 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:27.229+0000 7f0ebf340640 1 -- 172.21.15.1:0/1928474108 shutdown_connections 2024-09-11T21:48:27.830 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:27.229+0000 7f0ebf340640 1 --2- 172.21.15.1:0/1928474108 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f0ea4038f00 0x7f0ea403b3c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:27.830 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:27.229+0000 7f0ebf340640 1 --2- 172.21.15.1:0/1928474108 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0eb81081e0 0x7f0eb819b730 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:27.830 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:27.229+0000 7f0ebf340640 1 -- 172.21.15.1:0/1928474108 >> 172.21.15.1:0/1928474108 conn(0x7f0eb8101da0 msgr2=0x7f0eb8102910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:48:27.830 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:27.229+0000 7f0ebf340640 1 -- 172.21.15.1:0/1928474108 shutdown_connections 2024-09-11T21:48:27.831 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:27.229+0000 7f0ebf340640 1 -- 172.21.15.1:0/1928474108 wait complete. 2024-09-11T21:48:27.831 INFO:teuthology.orchestra.run.smithi001.stdout:Creating initial admin user... 2024-09-11T21:48:28.188 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:27 smithi001 ceph-mon[30935]: mgrmap e12: x(active, since 2s) 2024-09-11T21:48:28.188 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:27 smithi001 ceph-mon[30935]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:48:28.188 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:27 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:48:28.189 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:27 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:48:28.189 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:27 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:48:29.180 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:28 smithi001 ceph-mon[30935]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:48:29.180 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:28 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:48:29.814 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$scQsQidO1nIt1gJCf.rqlOZQyaVXPZk5g57o.sU/x2i24v44FWEE6", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726091308, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-11T21:48:29.814 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:28.196+0000 7ff74c031640 1 Processor -- start 2024-09-11T21:48:29.815 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:28.196+0000 7ff74c031640 1 -- start start 2024-09-11T21:48:29.815 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:28.197+0000 7ff74c031640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff7441081a0 0x7ff74410a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:29.815 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:28.197+0000 7ff74c031640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff74407be70 con 0x7ff7441081a0 2024-09-11T21:48:29.815 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:28.197+0000 7ff749dcd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff7441081a0 0x7ff74410a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:48:29.815 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:28.197+0000 7ff749dcd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff7441081a0 0x7ff74410a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51522/0 (socket says 172.21.15.1:51522) 2024-09-11T21:48:29.815 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:28.197+0000 7ff749dcd640 1 -- 172.21.15.1:0/4022095090 learned_addr learned my addr 172.21.15.1:0/4022095090 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:48:29.815 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:28.197+0000 7ff749dcd640 1 -- 172.21.15.1:0/4022095090 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff74410aad0 con 0x7ff7441081a0 2024-09-11T21:48:29.815 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:28.198+0000 7ff749dcd640 1 --2- 172.21.15.1:0/4022095090 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff7441081a0 0x7ff74410a590 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff740009820 tx=0x7ff74002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1307d93c84a6fd9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:48:29.815 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:28.198+0000 7ff748dcb640 1 -- 172.21.15.1:0/4022095090 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff740037070 con 0x7ff7441081a0 2024-09-11T21:48:29.815 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:28.198+0000 7ff748dcb640 1 -- 172.21.15.1:0/4022095090 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff740032920 con 0x7ff7441081a0 2024-09-11T21:48:29.815 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:28.198+0000 7ff748dcb640 1 -- 172.21.15.1:0/4022095090 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff740030810 con 0x7ff7441081a0 2024-09-11T21:48:29.816 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:28.199+0000 7ff74c031640 1 -- 172.21.15.1:0/4022095090 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff7441081a0 msgr2=0x7ff74410a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:29.816 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:28.199+0000 7ff74c031640 1 --2- 172.21.15.1:0/4022095090 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff7441081a0 0x7ff74410a590 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff740009820 tx=0x7ff74002a270 comp rx=0 tx=0).stop 2024-09-11T21:48:29.816 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:28.199+0000 7ff74c031640 1 -- 172.21.15.1:0/4022095090 shutdown_connections 2024-09-11T21:48:29.816 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:28.199+0000 7ff74c031640 1 --2- 172.21.15.1:0/4022095090 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff7441081a0 0x7ff74410a590 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:29.816 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:28.199+0000 7ff74c031640 1 -- 172.21.15.1:0/4022095090 >> 172.21.15.1:0/4022095090 conn(0x7ff744101e20 msgr2=0x7ff744104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:48:29.816 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:28.199+0000 7ff74c031640 1 -- 172.21.15.1:0/4022095090 shutdown_connections 2024-09-11T21:48:29.816 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:28.199+0000 7ff74c031640 1 -- 172.21.15.1:0/4022095090 wait complete. 2024-09-11T21:48:29.816 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:28.199+0000 7ff74c031640 1 Processor -- start 2024-09-11T21:48:29.816 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:28.200+0000 7ff74c031640 1 -- start start 2024-09-11T21:48:29.816 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:28.200+0000 7ff74c031640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff7441081a0 0x7ff74419d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:29.817 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:28.200+0000 7ff74c031640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff74410bb50 con 0x7ff7441081a0 2024-09-11T21:48:29.817 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:28.200+0000 7ff749dcd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff7441081a0 0x7ff74419d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:48:29.820 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:28.200+0000 7ff749dcd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff7441081a0 0x7ff74419d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51532/0 (socket says 172.21.15.1:51532) 2024-09-11T21:48:29.821 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:28.200+0000 7ff749dcd640 1 -- 172.21.15.1:0/2864031490 learned_addr learned my addr 172.21.15.1:0/2864031490 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:48:29.821 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:28.200+0000 7ff749dcd640 1 -- 172.21.15.1:0/2864031490 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff74419de20 con 0x7ff7441081a0 2024-09-11T21:48:29.821 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:28.200+0000 7ff749dcd640 1 --2- 172.21.15.1:0/2864031490 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff7441081a0 0x7ff74419d8e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff740030420 tx=0x7ff740032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:48:29.821 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:28.201+0000 7ff73affd640 1 -- 172.21.15.1:0/2864031490 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff740037040 con 0x7ff7441081a0 2024-09-11T21:48:29.821 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:28.201+0000 7ff73affd640 1 -- 172.21.15.1:0/2864031490 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff740040070 con 0x7ff7441081a0 2024-09-11T21:48:29.821 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:28.201+0000 7ff74c031640 1 -- 172.21.15.1:0/2864031490 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff74419e0b0 con 0x7ff7441081a0 2024-09-11T21:48:29.821 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:28.201+0000 7ff73affd640 1 -- 172.21.15.1:0/2864031490 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff74003a6e0 con 0x7ff7441081a0 2024-09-11T21:48:29.821 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:28.201+0000 7ff74c031640 1 -- 172.21.15.1:0/2864031490 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7441a0da0 con 0x7ff7441081a0 2024-09-11T21:48:29.821 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:28.202+0000 7ff73affd640 1 -- 172.21.15.1:0/2864031490 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7ff74003a880 con 0x7ff7441081a0 2024-09-11T21:48:29.821 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:28.202+0000 7ff74c031640 1 -- 172.21.15.1:0/2864031490 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff744108600 con 0x7ff7441081a0 2024-09-11T21:48:29.821 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:28.202+0000 7ff73affd640 1 --2- 172.21.15.1:0/2864031490 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7ff730039020 0x7ff73003b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:29.821 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:28.202+0000 7ff73affd640 1 -- 172.21.15.1:0/2864031490 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff74006c880 con 0x7ff7441081a0 2024-09-11T21:48:29.822 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:28.202+0000 7ff7495cc640 1 --2- 172.21.15.1:0/2864031490 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7ff730039020 0x7ff73003b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:48:29.822 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:28.203+0000 7ff7495cc640 1 --2- 172.21.15.1:0/2864031490 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7ff730039020 0x7ff73003b4e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff7340098c0 tx=0x7ff734006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:48:29.822 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:28.207+0000 7ff73affd640 1 -- 172.21.15.1:0/2864031490 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff740044c70 con 0x7ff7441081a0 2024-09-11T21:48:29.822 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:28.401+0000 7ff74c031640 1 -- 172.21.15.1:0/2864031490 --> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7ff7441a1030 con 0x7ff730039020 2024-09-11T21:48:29.822 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:28.650+0000 7ff73affd640 1 -- 172.21.15.1:0/2864031490 <== mgr.14150 v2:172.21.15.1:6800/3890350886 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7ff7441a1030 con 0x7ff730039020 2024-09-11T21:48:29.822 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:28.652+0000 7ff74c031640 1 -- 172.21.15.1:0/2864031490 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7ff730039020 msgr2=0x7ff73003b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:29.822 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:28.652+0000 7ff74c031640 1 --2- 172.21.15.1:0/2864031490 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7ff730039020 0x7ff73003b4e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff7340098c0 tx=0x7ff734006db0 comp rx=0 tx=0).stop 2024-09-11T21:48:29.822 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:28.652+0000 7ff74c031640 1 -- 172.21.15.1:0/2864031490 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff7441081a0 msgr2=0x7ff74419d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:29.822 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:28.652+0000 7ff74c031640 1 --2- 172.21.15.1:0/2864031490 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff7441081a0 0x7ff74419d8e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff740030420 tx=0x7ff740032670 comp rx=0 tx=0).stop 2024-09-11T21:48:29.822 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:28.652+0000 7ff74c031640 1 -- 172.21.15.1:0/2864031490 shutdown_connections 2024-09-11T21:48:29.822 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:28.652+0000 7ff74c031640 1 --2- 172.21.15.1:0/2864031490 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7ff730039020 0x7ff73003b4e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:29.822 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:28.652+0000 7ff74c031640 1 --2- 172.21.15.1:0/2864031490 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff7441081a0 0x7ff74419d8e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:29.822 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:28.652+0000 7ff74c031640 1 -- 172.21.15.1:0/2864031490 >> 172.21.15.1:0/2864031490 conn(0x7ff744101e20 msgr2=0x7ff744102930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:48:29.823 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:28.652+0000 7ff74c031640 1 -- 172.21.15.1:0/2864031490 shutdown_connections 2024-09-11T21:48:29.823 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:28.652+0000 7ff74c031640 1 -- 172.21.15.1:0/2864031490 wait complete. 2024-09-11T21:48:29.823 INFO:teuthology.orchestra.run.smithi001.stdout:Fetching dashboard port number... 2024-09-11T21:48:30.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:30 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/1697913200' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-11T21:48:31.104 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 8443 2024-09-11T21:48:31.104 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:30.265+0000 7f3761cc5640 1 Processor -- start 2024-09-11T21:48:31.104 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:30.265+0000 7f3761cc5640 1 -- start start 2024-09-11T21:48:31.105 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:30.265+0000 7f3761cc5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f375c108200 0x7f375c10a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:31.105 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:30.265+0000 7f3761cc5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f375c07be30 con 0x7f375c108200 2024-09-11T21:48:31.105 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:30.265+0000 7f375b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f375c108200 0x7f375c10a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:48:31.105 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:30.265+0000 7f375b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f375c108200 0x7f375c10a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51548/0 (socket says 172.21.15.1:51548) 2024-09-11T21:48:31.105 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:30.265+0000 7f375b7fe640 1 -- 172.21.15.1:0/2526836654 learned_addr learned my addr 172.21.15.1:0/2526836654 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:48:31.105 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:30.266+0000 7f375b7fe640 1 -- 172.21.15.1:0/2526836654 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f375c10ab30 con 0x7f375c108200 2024-09-11T21:48:31.105 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:30.266+0000 7f375b7fe640 1 --2- 172.21.15.1:0/2526836654 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f375c108200 0x7f375c10a5f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f374c009820 tx=0x7f374c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e683b6fe08d4848 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:48:31.105 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:30.267+0000 7f375a7fc640 1 -- 172.21.15.1:0/2526836654 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f374c037070 con 0x7f375c108200 2024-09-11T21:48:31.105 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:30.267+0000 7f375a7fc640 1 -- 172.21.15.1:0/2526836654 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f374c032920 con 0x7f375c108200 2024-09-11T21:48:31.105 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:30.267+0000 7f375a7fc640 1 -- 172.21.15.1:0/2526836654 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f374c030810 con 0x7f375c108200 2024-09-11T21:48:31.105 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:30.267+0000 7f3761cc5640 1 -- 172.21.15.1:0/2526836654 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f375c108200 msgr2=0x7f375c10a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:31.105 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:30.267+0000 7f3761cc5640 1 --2- 172.21.15.1:0/2526836654 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f375c108200 0x7f375c10a5f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f374c009820 tx=0x7f374c02a270 comp rx=0 tx=0).stop 2024-09-11T21:48:31.106 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:30.267+0000 7f3761cc5640 1 -- 172.21.15.1:0/2526836654 shutdown_connections 2024-09-11T21:48:31.106 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:30.267+0000 7f3761cc5640 1 --2- 172.21.15.1:0/2526836654 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f375c108200 0x7f375c10a5f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:31.106 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:30.267+0000 7f3761cc5640 1 -- 172.21.15.1:0/2526836654 >> 172.21.15.1:0/2526836654 conn(0x7f375c101da0 msgr2=0x7f375c1041e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:48:31.106 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:30.267+0000 7f3761cc5640 1 -- 172.21.15.1:0/2526836654 shutdown_connections 2024-09-11T21:48:31.106 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:30.267+0000 7f3761cc5640 1 -- 172.21.15.1:0/2526836654 wait complete. 2024-09-11T21:48:31.106 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:30.268+0000 7f3761cc5640 1 Processor -- start 2024-09-11T21:48:31.106 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:30.268+0000 7f3761cc5640 1 -- start start 2024-09-11T21:48:31.106 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:30.268+0000 7f3761cc5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f375c108200 0x7f375c107a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:31.106 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:30.268+0000 7f3761cc5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f375c10bb50 con 0x7f375c108200 2024-09-11T21:48:31.106 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:30.269+0000 7f375b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f375c108200 0x7f375c107a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:48:31.106 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:30.269+0000 7f375b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f375c108200 0x7f375c107a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51552/0 (socket says 172.21.15.1:51552) 2024-09-11T21:48:31.106 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:30.269+0000 7f375b7fe640 1 -- 172.21.15.1:0/1697913200 learned_addr learned my addr 172.21.15.1:0/1697913200 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:48:31.106 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:30.269+0000 7f375b7fe640 1 -- 172.21.15.1:0/1697913200 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f375c107fc0 con 0x7f375c108200 2024-09-11T21:48:31.107 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:30.269+0000 7f375b7fe640 1 --2- 172.21.15.1:0/1697913200 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f375c108200 0x7f375c107a80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f374c0097f0 tx=0x7f374c0327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:48:31.107 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:30.269+0000 7f3758ff9640 1 -- 172.21.15.1:0/1697913200 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f374c037040 con 0x7f375c108200 2024-09-11T21:48:31.107 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:30.269+0000 7f3758ff9640 1 -- 172.21.15.1:0/1697913200 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f374c030c40 con 0x7f375c108200 2024-09-11T21:48:31.107 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:30.269+0000 7f3761cc5640 1 -- 172.21.15.1:0/1697913200 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f375c1061b0 con 0x7f375c108200 2024-09-11T21:48:31.107 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:30.269+0000 7f3758ff9640 1 -- 172.21.15.1:0/1697913200 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f374c03a5d0 con 0x7f375c108200 2024-09-11T21:48:31.107 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:30.269+0000 7f3761cc5640 1 -- 172.21.15.1:0/1697913200 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f375c1066f0 con 0x7f375c108200 2024-09-11T21:48:31.107 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:30.270+0000 7f3758ff9640 1 -- 172.21.15.1:0/1697913200 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f374c03a7c0 con 0x7f375c108200 2024-09-11T21:48:31.107 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:30.270+0000 7f3761cc5640 1 -- 172.21.15.1:0/1697913200 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f375c108660 con 0x7f375c108200 2024-09-11T21:48:31.107 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:30.270+0000 7f3758ff9640 1 --2- 172.21.15.1:0/1697913200 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f3744039020 0x7f374403b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:31.107 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:30.270+0000 7f3758ff9640 1 -- 172.21.15.1:0/1697913200 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f374c06c540 con 0x7f375c108200 2024-09-11T21:48:31.107 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:30.270+0000 7f375affd640 1 --2- 172.21.15.1:0/1697913200 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f3744039020 0x7f374403b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:48:31.107 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:30.271+0000 7f375affd640 1 --2- 172.21.15.1:0/1697913200 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f3744039020 0x7f374403b4e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f37500098c0 tx=0x7f3750006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:48:31.108 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:30.275+0000 7f3758ff9640 1 -- 172.21.15.1:0/1697913200 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f374c03aab0 con 0x7f375c108200 2024-09-11T21:48:31.108 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:30.450+0000 7f3761cc5640 1 -- 172.21.15.1:0/1697913200 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f375c106970 con 0x7f375c108200 2024-09-11T21:48:31.108 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:30.450+0000 7f3758ff9640 1 -- 172.21.15.1:0/1697913200 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f374c03b9d0 con 0x7f375c108200 2024-09-11T21:48:31.108 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:30.452+0000 7f3761cc5640 1 -- 172.21.15.1:0/1697913200 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f3744039020 msgr2=0x7f374403b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:31.108 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:30.452+0000 7f3761cc5640 1 --2- 172.21.15.1:0/1697913200 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f3744039020 0x7f374403b4e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f37500098c0 tx=0x7f3750006db0 comp rx=0 tx=0).stop 2024-09-11T21:48:31.108 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:30.452+0000 7f3761cc5640 1 -- 172.21.15.1:0/1697913200 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f375c108200 msgr2=0x7f375c107a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:31.108 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:30.452+0000 7f3761cc5640 1 --2- 172.21.15.1:0/1697913200 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f375c108200 0x7f375c107a80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f374c0097f0 tx=0x7f374c0327b0 comp rx=0 tx=0).stop 2024-09-11T21:48:31.108 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:30.452+0000 7f3761cc5640 1 -- 172.21.15.1:0/1697913200 shutdown_connections 2024-09-11T21:48:31.108 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:30.452+0000 7f3761cc5640 1 --2- 172.21.15.1:0/1697913200 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f3744039020 0x7f374403b4e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:31.108 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:30.452+0000 7f3761cc5640 1 --2- 172.21.15.1:0/1697913200 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f375c108200 0x7f375c107a80 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:31.108 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:30.452+0000 7f3761cc5640 1 -- 172.21.15.1:0/1697913200 >> 172.21.15.1:0/1697913200 conn(0x7f375c101da0 msgr2=0x7f375c102950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:48:31.108 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:30.453+0000 7f3761cc5640 1 -- 172.21.15.1:0/1697913200 shutdown_connections 2024-09-11T21:48:31.109 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:30.453+0000 7f3761cc5640 1 -- 172.21.15.1:0/1697913200 wait complete. 2024-09-11T21:48:31.111 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T21:48:31.111 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout disabled 2024-09-11T21:48:31.118 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T21:48:31.118 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout inactive 2024-09-11T21:48:31.118 INFO:teuthology.orchestra.run.smithi001.stdout:firewalld.service is not enabled 2024-09-11T21:48:31.118 INFO:teuthology.orchestra.run.smithi001.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-11T21:48:31.120 INFO:teuthology.orchestra.run.smithi001.stdout:Ceph Dashboard is now available at: 2024-09-11T21:48:31.120 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:48:31.120 INFO:teuthology.orchestra.run.smithi001.stdout: URL: https://smithi001.front.sepia.ceph.com:8443/ 2024-09-11T21:48:31.121 INFO:teuthology.orchestra.run.smithi001.stdout: User: admin 2024-09-11T21:48:31.121 INFO:teuthology.orchestra.run.smithi001.stdout: Password: dwtg5c8xsf 2024-09-11T21:48:31.121 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:48:31.121 INFO:teuthology.orchestra.run.smithi001.stdout:Saving cluster configuration to /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/config directory 2024-09-11T21:48:32.237 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:31.463+0000 7f24e8106640 1 Processor -- start 2024-09-11T21:48:32.237 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:31.463+0000 7f24e8106640 1 -- start start 2024-09-11T21:48:32.237 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:31.463+0000 7f24e8106640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f24e0109960 0x7f24e0109d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:32.237 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:31.463+0000 7f24e8106640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f24e010a330 con 0x7f24e0109960 2024-09-11T21:48:32.237 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:31.463+0000 7f24e5ea2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f24e0109960 0x7f24e0109d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:48:32.237 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:31.463+0000 7f24e5ea2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f24e0109960 0x7f24e0109d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51562/0 (socket says 172.21.15.1:51562) 2024-09-11T21:48:32.237 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:31.463+0000 7f24e5ea2640 1 -- 172.21.15.1:0/265203165 learned_addr learned my addr 172.21.15.1:0/265203165 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:48:32.238 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:31.464+0000 7f24e5ea2640 1 -- 172.21.15.1:0/265203165 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24e010aad0 con 0x7f24e0109960 2024-09-11T21:48:32.238 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:31.464+0000 7f24e5ea2640 1 --2- 172.21.15.1:0/265203165 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f24e0109960 0x7f24e0109d60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f24dc009820 tx=0x7f24dc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef203d3b1eaaab79 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:48:32.238 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:31.465+0000 7f24e4ea0640 1 -- 172.21.15.1:0/265203165 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24dc037070 con 0x7f24e0109960 2024-09-11T21:48:32.238 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:31.465+0000 7f24e4ea0640 1 -- 172.21.15.1:0/265203165 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24dc032920 con 0x7f24e0109960 2024-09-11T21:48:32.238 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:31.465+0000 7f24e4ea0640 1 -- 172.21.15.1:0/265203165 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24dc030810 con 0x7f24e0109960 2024-09-11T21:48:32.238 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:31.465+0000 7f24e8106640 1 -- 172.21.15.1:0/265203165 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f24e0109960 msgr2=0x7f24e0109d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:32.238 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:31.465+0000 7f24e8106640 1 --2- 172.21.15.1:0/265203165 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f24e0109960 0x7f24e0109d60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f24dc009820 tx=0x7f24dc02a270 comp rx=0 tx=0).stop 2024-09-11T21:48:32.238 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:31.465+0000 7f24e8106640 1 -- 172.21.15.1:0/265203165 shutdown_connections 2024-09-11T21:48:32.238 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:31.465+0000 7f24e8106640 1 --2- 172.21.15.1:0/265203165 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f24e0109960 0x7f24e0109d60 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:32.238 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:31.465+0000 7f24e8106640 1 -- 172.21.15.1:0/265203165 >> 172.21.15.1:0/265203165 conn(0x7f24e007cf30 msgr2=0x7f24e007d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:48:32.238 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:31.465+0000 7f24e8106640 1 -- 172.21.15.1:0/265203165 shutdown_connections 2024-09-11T21:48:32.239 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:31.465+0000 7f24e8106640 1 -- 172.21.15.1:0/265203165 wait complete. 2024-09-11T21:48:32.239 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:31.466+0000 7f24e8106640 1 Processor -- start 2024-09-11T21:48:32.239 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:31.466+0000 7f24e8106640 1 -- start start 2024-09-11T21:48:32.239 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:31.466+0000 7f24e8106640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f24e0109960 0x7f24e019d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:32.239 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:31.466+0000 7f24e8106640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f24e010de80 con 0x7f24e0109960 2024-09-11T21:48:32.239 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:31.466+0000 7f24e5ea2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f24e0109960 0x7f24e019d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:48:32.239 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:31.466+0000 7f24e5ea2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f24e0109960 0x7f24e019d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51570/0 (socket says 172.21.15.1:51570) 2024-09-11T21:48:32.239 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:31.466+0000 7f24e5ea2640 1 -- 172.21.15.1:0/914306014 learned_addr learned my addr 172.21.15.1:0/914306014 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:48:32.239 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:31.466+0000 7f24e5ea2640 1 -- 172.21.15.1:0/914306014 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24e019de60 con 0x7f24e0109960 2024-09-11T21:48:32.239 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:31.467+0000 7f24e5ea2640 1 --2- 172.21.15.1:0/914306014 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f24e0109960 0x7f24e019d920 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f24dc030560 tx=0x7f24dc032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:48:32.239 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:31.467+0000 7f24d6ffd640 1 -- 172.21.15.1:0/914306014 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24dc037040 con 0x7f24e0109960 2024-09-11T21:48:32.240 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:31.467+0000 7f24d6ffd640 1 -- 172.21.15.1:0/914306014 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24dc040070 con 0x7f24e0109960 2024-09-11T21:48:32.240 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:31.467+0000 7f24e8106640 1 -- 172.21.15.1:0/914306014 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24e019e0f0 con 0x7f24e0109960 2024-09-11T21:48:32.240 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:31.467+0000 7f24d6ffd640 1 -- 172.21.15.1:0/914306014 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24dc03a6d0 con 0x7f24e0109960 2024-09-11T21:48:32.240 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:31.467+0000 7f24e8106640 1 -- 172.21.15.1:0/914306014 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24e01a0de0 con 0x7f24e0109960 2024-09-11T21:48:32.240 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:31.468+0000 7f24d6ffd640 1 -- 172.21.15.1:0/914306014 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f24dc03b420 con 0x7f24e0109960 2024-09-11T21:48:32.240 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:31.468+0000 7f24e8106640 1 -- 172.21.15.1:0/914306014 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24e010e000 con 0x7f24e0109960 2024-09-11T21:48:32.240 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:31.468+0000 7f24d6ffd640 1 --2- 172.21.15.1:0/914306014 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f24cc038fc0 0x7f24cc03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:32.240 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:31.468+0000 7f24d6ffd640 1 -- 172.21.15.1:0/914306014 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f24dc06c650 con 0x7f24e0109960 2024-09-11T21:48:32.240 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:31.468+0000 7f24e56a1640 1 --2- 172.21.15.1:0/914306014 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f24cc038fc0 0x7f24cc03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:48:32.240 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:31.469+0000 7f24e56a1640 1 --2- 172.21.15.1:0/914306014 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f24cc038fc0 0x7f24cc03b480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f24d00098c0 tx=0x7f24d0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:48:32.240 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:31.473+0000 7f24d6ffd640 1 -- 172.21.15.1:0/914306014 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f24dc04a200 con 0x7f24e0109960 2024-09-11T21:48:32.240 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:31.503+0000 7f24d6ffd640 1 -- 172.21.15.1:0/914306014 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f24dc03a940 con 0x7f24e0109960 2024-09-11T21:48:32.240 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:31.739+0000 7f24e8106640 1 -- 172.21.15.1:0/914306014 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f24e01a0fb0 con 0x7f24e0109960 2024-09-11T21:48:32.241 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:31.741+0000 7f24d6ffd640 1 -- 172.21.15.1:0/914306014 <== mon.0 v2:172.21.15.1:3300/0 8 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) ==== 153+0+0 (secure 0 0 0) 0x7f24dc043c60 con 0x7f24e0109960 2024-09-11T21:48:32.241 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-11T21:48:32.241 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:31.743+0000 7f24e8106640 1 -- 172.21.15.1:0/914306014 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f24cc038fc0 msgr2=0x7f24cc03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:32.241 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:31.743+0000 7f24e8106640 1 --2- 172.21.15.1:0/914306014 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f24cc038fc0 0x7f24cc03b480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f24d00098c0 tx=0x7f24d0006db0 comp rx=0 tx=0).stop 2024-09-11T21:48:32.241 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:31.743+0000 7f24e8106640 1 -- 172.21.15.1:0/914306014 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f24e0109960 msgr2=0x7f24e019d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:32.241 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:31.743+0000 7f24e8106640 1 --2- 172.21.15.1:0/914306014 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f24e0109960 0x7f24e019d920 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f24dc030560 tx=0x7f24dc032670 comp rx=0 tx=0).stop 2024-09-11T21:48:32.241 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:31.743+0000 7f24e8106640 1 -- 172.21.15.1:0/914306014 shutdown_connections 2024-09-11T21:48:32.241 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:31.743+0000 7f24e8106640 1 --2- 172.21.15.1:0/914306014 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f24cc038fc0 0x7f24cc03b480 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:32.241 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:31.743+0000 7f24e8106640 1 --2- 172.21.15.1:0/914306014 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f24e0109960 0x7f24e019d920 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:32.241 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:31.743+0000 7f24e8106640 1 -- 172.21.15.1:0/914306014 >> 172.21.15.1:0/914306014 conn(0x7f24e007cf30 msgr2=0x7f24e0107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:48:32.241 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:31.743+0000 7f24e8106640 1 -- 172.21.15.1:0/914306014 shutdown_connections 2024-09-11T21:48:32.241 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-09-11T21:48:31.743+0000 7f24e8106640 1 -- 172.21.15.1:0/914306014 wait complete. 2024-09-11T21:48:32.241 INFO:teuthology.orchestra.run.smithi001.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-11T21:48:32.242 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:48:32.242 INFO:teuthology.orchestra.run.smithi001.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-11T21:48:32.242 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:48:32.242 INFO:teuthology.orchestra.run.smithi001.stdout:Or, if you are only running a single cluster on this host: 2024-09-11T21:48:32.242 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:48:32.242 INFO:teuthology.orchestra.run.smithi001.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-11T21:48:32.242 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:48:32.242 INFO:teuthology.orchestra.run.smithi001.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-11T21:48:32.242 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:48:32.242 INFO:teuthology.orchestra.run.smithi001.stdout: ceph telemetry on 2024-09-11T21:48:32.242 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:48:32.242 INFO:teuthology.orchestra.run.smithi001.stdout:For more information see: 2024-09-11T21:48:32.242 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:48:32.242 INFO:teuthology.orchestra.run.smithi001.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-11T21:48:32.243 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:48:32.243 INFO:teuthology.orchestra.run.smithi001.stdout:Bootstrap complete. 2024-09-11T21:48:32.278 INFO:tasks.cephadm:Fetching config... 2024-09-11T21:48:32.279 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-11T21:48:32.279 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-11T21:48:32.297 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-11T21:48:32.297 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-11T21:48:32.298 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-11T21:48:32.354 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-11T21:48:32.354 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-11T21:48:32.354 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-11T21:48:32.422 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-11T21:48:32.422 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-11T21:48:32.422 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-11T21:48:32.479 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-11T21:48:32.479 DEBUG:teuthology.orchestra.run.smithi001:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCmSKiSRGV/mNwNumHfq5hGkh9DG302w4QnOziWzEe5RG6HBp3yzlWsDKhhZepa5kKwtVzLDlvOL7tnxpfj7diqEu3JplRVuT3iFJhsBotcjmerodp8UTkxJnQfIbFG3tVLEyuGX7S8xvuzvhhbmhHLcvquJZPvkPqI1qEE5bFz/U71/Z0ZG9ftsIPEIf3RcuJIDrEQBU9Af0wkTgM4i/PiTplm2UT2DTVEvSWg4Hx0Ck4anmX4lK78h54U71cDCRlqkWaQ/sazb+g482Coek+LPB6YpkSiL1FfImndQSiwOTGSnWBfWEGxHjsnfXjMEzOBgi0JogVWjktEyBdCPhYeltyxBfCMIXo9sUJCqk6bNwT7rfTbstRSdtmYAAH4snVMVqnOfvHt8nWuTIrDH3WlOwLeNim/3IwW3p7+OzQlmNKiAT+3I3+c9Ugn12/OOeL9LPnmiIUrR0HUGyGTY/CTA+LASfNGd6yXn7Y19ieA0AjPtlZWA0EHZlkQUhPOXTU= ceph-3ffca83c-7087-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-11T21:48:32.522 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:32 smithi001 ceph-mon[30935]: mgrmap e13: x(active, since 6s) 2024-09-11T21:48:32.523 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:32 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/914306014' entity='client.admin' 2024-09-11T21:48:32.557 INFO:teuthology.orchestra.run.smithi001.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCmSKiSRGV/mNwNumHfq5hGkh9DG302w4QnOziWzEe5RG6HBp3yzlWsDKhhZepa5kKwtVzLDlvOL7tnxpfj7diqEu3JplRVuT3iFJhsBotcjmerodp8UTkxJnQfIbFG3tVLEyuGX7S8xvuzvhhbmhHLcvquJZPvkPqI1qEE5bFz/U71/Z0ZG9ftsIPEIf3RcuJIDrEQBU9Af0wkTgM4i/PiTplm2UT2DTVEvSWg4Hx0Ck4anmX4lK78h54U71cDCRlqkWaQ/sazb+g482Coek+LPB6YpkSiL1FfImndQSiwOTGSnWBfWEGxHjsnfXjMEzOBgi0JogVWjktEyBdCPhYeltyxBfCMIXo9sUJCqk6bNwT7rfTbstRSdtmYAAH4snVMVqnOfvHt8nWuTIrDH3WlOwLeNim/3IwW3p7+OzQlmNKiAT+3I3+c9Ugn12/OOeL9LPnmiIUrR0HUGyGTY/CTA+LASfNGd6yXn7Y19ieA0AjPtlZWA0EHZlkQUhPOXTU= ceph-3ffca83c-7087-11ef-bcea-c7b262605968 2024-09-11T21:48:32.569 DEBUG:teuthology.orchestra.run.smithi053:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCmSKiSRGV/mNwNumHfq5hGkh9DG302w4QnOziWzEe5RG6HBp3yzlWsDKhhZepa5kKwtVzLDlvOL7tnxpfj7diqEu3JplRVuT3iFJhsBotcjmerodp8UTkxJnQfIbFG3tVLEyuGX7S8xvuzvhhbmhHLcvquJZPvkPqI1qEE5bFz/U71/Z0ZG9ftsIPEIf3RcuJIDrEQBU9Af0wkTgM4i/PiTplm2UT2DTVEvSWg4Hx0Ck4anmX4lK78h54U71cDCRlqkWaQ/sazb+g482Coek+LPB6YpkSiL1FfImndQSiwOTGSnWBfWEGxHjsnfXjMEzOBgi0JogVWjktEyBdCPhYeltyxBfCMIXo9sUJCqk6bNwT7rfTbstRSdtmYAAH4snVMVqnOfvHt8nWuTIrDH3WlOwLeNim/3IwW3p7+OzQlmNKiAT+3I3+c9Ugn12/OOeL9LPnmiIUrR0HUGyGTY/CTA+LASfNGd6yXn7Y19ieA0AjPtlZWA0EHZlkQUhPOXTU= ceph-3ffca83c-7087-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-11T21:48:32.611 INFO:teuthology.orchestra.run.smithi053.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCmSKiSRGV/mNwNumHfq5hGkh9DG302w4QnOziWzEe5RG6HBp3yzlWsDKhhZepa5kKwtVzLDlvOL7tnxpfj7diqEu3JplRVuT3iFJhsBotcjmerodp8UTkxJnQfIbFG3tVLEyuGX7S8xvuzvhhbmhHLcvquJZPvkPqI1qEE5bFz/U71/Z0ZG9ftsIPEIf3RcuJIDrEQBU9Af0wkTgM4i/PiTplm2UT2DTVEvSWg4Hx0Ck4anmX4lK78h54U71cDCRlqkWaQ/sazb+g482Coek+LPB6YpkSiL1FfImndQSiwOTGSnWBfWEGxHjsnfXjMEzOBgi0JogVWjktEyBdCPhYeltyxBfCMIXo9sUJCqk6bNwT7rfTbstRSdtmYAAH4snVMVqnOfvHt8nWuTIrDH3WlOwLeNim/3IwW3p7+OzQlmNKiAT+3I3+c9Ugn12/OOeL9LPnmiIUrR0HUGyGTY/CTA+LASfNGd6yXn7Y19ieA0AjPtlZWA0EHZlkQUhPOXTU= ceph-3ffca83c-7087-11ef-bcea-c7b262605968 2024-09-11T21:48:32.623 DEBUG:teuthology.orchestra.run.smithi161:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCmSKiSRGV/mNwNumHfq5hGkh9DG302w4QnOziWzEe5RG6HBp3yzlWsDKhhZepa5kKwtVzLDlvOL7tnxpfj7diqEu3JplRVuT3iFJhsBotcjmerodp8UTkxJnQfIbFG3tVLEyuGX7S8xvuzvhhbmhHLcvquJZPvkPqI1qEE5bFz/U71/Z0ZG9ftsIPEIf3RcuJIDrEQBU9Af0wkTgM4i/PiTplm2UT2DTVEvSWg4Hx0Ck4anmX4lK78h54U71cDCRlqkWaQ/sazb+g482Coek+LPB6YpkSiL1FfImndQSiwOTGSnWBfWEGxHjsnfXjMEzOBgi0JogVWjktEyBdCPhYeltyxBfCMIXo9sUJCqk6bNwT7rfTbstRSdtmYAAH4snVMVqnOfvHt8nWuTIrDH3WlOwLeNim/3IwW3p7+OzQlmNKiAT+3I3+c9Ugn12/OOeL9LPnmiIUrR0HUGyGTY/CTA+LASfNGd6yXn7Y19ieA0AjPtlZWA0EHZlkQUhPOXTU= ceph-3ffca83c-7087-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-11T21:48:32.665 INFO:teuthology.orchestra.run.smithi161.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCmSKiSRGV/mNwNumHfq5hGkh9DG302w4QnOziWzEe5RG6HBp3yzlWsDKhhZepa5kKwtVzLDlvOL7tnxpfj7diqEu3JplRVuT3iFJhsBotcjmerodp8UTkxJnQfIbFG3tVLEyuGX7S8xvuzvhhbmhHLcvquJZPvkPqI1qEE5bFz/U71/Z0ZG9ftsIPEIf3RcuJIDrEQBU9Af0wkTgM4i/PiTplm2UT2DTVEvSWg4Hx0Ck4anmX4lK78h54U71cDCRlqkWaQ/sazb+g482Coek+LPB6YpkSiL1FfImndQSiwOTGSnWBfWEGxHjsnfXjMEzOBgi0JogVWjktEyBdCPhYeltyxBfCMIXo9sUJCqk6bNwT7rfTbstRSdtmYAAH4snVMVqnOfvHt8nWuTIrDH3WlOwLeNim/3IwW3p7+OzQlmNKiAT+3I3+c9Ugn12/OOeL9LPnmiIUrR0HUGyGTY/CTA+LASfNGd6yXn7Y19ieA0AjPtlZWA0EHZlkQUhPOXTU= ceph-3ffca83c-7087-11ef-bcea-c7b262605968 2024-09-11T21:48:32.678 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-11T21:48:32.932 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:48:34.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:33.402+0000 7fdac0d07640 1 -- 172.21.15.1:0/3548281757 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdabc1015e0 msgr2=0x7fdabc1019e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:34.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:33.402+0000 7fdac0d07640 1 --2- 172.21.15.1:0/3548281757 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdabc1015e0 0x7fdabc1019e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdab4009c20 tx=0x7fdab402aa90 comp rx=0 tx=0).stop 2024-09-11T21:48:34.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:33.402+0000 7fdac0d07640 1 -- 172.21.15.1:0/3548281757 shutdown_connections 2024-09-11T21:48:34.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:33.402+0000 7fdac0d07640 1 --2- 172.21.15.1:0/3548281757 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdabc1015e0 0x7fdabc1019e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:34.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:33.402+0000 7fdac0d07640 1 -- 172.21.15.1:0/3548281757 >> 172.21.15.1:0/3548281757 conn(0x7fdabc0fcd70 msgr2=0x7fdabc0ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:48:34.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:33.403+0000 7fdac0d07640 1 -- 172.21.15.1:0/3548281757 shutdown_connections 2024-09-11T21:48:34.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:33.403+0000 7fdac0d07640 1 -- 172.21.15.1:0/3548281757 wait complete. 2024-09-11T21:48:34.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:33.403+0000 7fdac0d07640 1 Processor -- start 2024-09-11T21:48:34.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:33.403+0000 7fdac0d07640 1 -- start start 2024-09-11T21:48:34.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:33.403+0000 7fdac0d07640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdabc1015e0 0x7fdabc198f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:34.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:33.403+0000 7fdac0d07640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdabc1076f0 con 0x7fdabc1015e0 2024-09-11T21:48:34.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:33.404+0000 7fdaba59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdabc1015e0 0x7fdabc198f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:48:34.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:33.404+0000 7fdaba59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdabc1015e0 0x7fdabc198f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51586/0 (socket says 172.21.15.1:51586) 2024-09-11T21:48:34.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:33.404+0000 7fdaba59c640 1 -- 172.21.15.1:0/3518837122 learned_addr learned my addr 172.21.15.1:0/3518837122 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:48:34.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:33.404+0000 7fdaba59c640 1 -- 172.21.15.1:0/3518837122 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdabc199490 con 0x7fdabc1015e0 2024-09-11T21:48:34.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:33.404+0000 7fdaba59c640 1 --2- 172.21.15.1:0/3518837122 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdabc1015e0 0x7fdabc198f50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fdab4005cc0 tx=0x7fdab402fc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:48:34.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:33.404+0000 7fdab37fe640 1 -- 172.21.15.1:0/3518837122 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdab4047070 con 0x7fdabc1015e0 2024-09-11T21:48:34.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:33.404+0000 7fdac0d07640 1 -- 172.21.15.1:0/3518837122 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdabc199720 con 0x7fdabc1015e0 2024-09-11T21:48:34.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:33.404+0000 7fdab37fe640 1 -- 172.21.15.1:0/3518837122 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdab4003e50 con 0x7fdabc1015e0 2024-09-11T21:48:34.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:33.404+0000 7fdab37fe640 1 -- 172.21.15.1:0/3518837122 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdab4033050 con 0x7fdabc1015e0 2024-09-11T21:48:34.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:33.404+0000 7fdac0d07640 1 -- 172.21.15.1:0/3518837122 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdabc19c410 con 0x7fdabc1015e0 2024-09-11T21:48:34.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:33.405+0000 7fdab37fe640 1 -- 172.21.15.1:0/3518837122 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7fdab404a070 con 0x7fdabc1015e0 2024-09-11T21:48:34.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:33.406+0000 7fdac0d07640 1 -- 172.21.15.1:0/3518837122 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdabc107e70 con 0x7fdabc1015e0 2024-09-11T21:48:34.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:33.406+0000 7fdab37fe640 1 --2- 172.21.15.1:0/3518837122 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fdaa8039060 0x7fdaa803b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:34.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:33.406+0000 7fdab37fe640 1 -- 172.21.15.1:0/3518837122 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fdab403e030 con 0x7fdabc1015e0 2024-09-11T21:48:34.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:33.406+0000 7fdab9d9b640 1 --2- 172.21.15.1:0/3518837122 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fdaa8039060 0x7fdaa803b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:48:34.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:33.406+0000 7fdab9d9b640 1 --2- 172.21.15.1:0/3518837122 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fdaa8039060 0x7fdaa803b520 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fdaa40098c0 tx=0x7fdaa4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:48:34.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:33.411+0000 7fdab37fe640 1 -- 172.21.15.1:0/3518837122 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdab40373b0 con 0x7fdabc1015e0 2024-09-11T21:48:34.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:33.592+0000 7fdac0d07640 1 -- 172.21.15.1:0/3518837122 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fdabc02d830 con 0x7fdabc1015e0 2024-09-11T21:48:34.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:33.594+0000 7fdab37fe640 1 -- 172.21.15.1:0/3518837122 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fdab40510c0 con 0x7fdabc1015e0 2024-09-11T21:48:34.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:33.597+0000 7fdac0d07640 1 -- 172.21.15.1:0/3518837122 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fdaa8039060 msgr2=0x7fdaa803b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:34.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:33.597+0000 7fdac0d07640 1 --2- 172.21.15.1:0/3518837122 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fdaa8039060 0x7fdaa803b520 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fdaa40098c0 tx=0x7fdaa4006db0 comp rx=0 tx=0).stop 2024-09-11T21:48:34.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:33.597+0000 7fdac0d07640 1 -- 172.21.15.1:0/3518837122 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdabc1015e0 msgr2=0x7fdabc198f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:34.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:33.597+0000 7fdac0d07640 1 --2- 172.21.15.1:0/3518837122 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdabc1015e0 0x7fdabc198f50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fdab4005cc0 tx=0x7fdab402fc10 comp rx=0 tx=0).stop 2024-09-11T21:48:34.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:33.597+0000 7fdac0d07640 1 -- 172.21.15.1:0/3518837122 shutdown_connections 2024-09-11T21:48:34.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:33.597+0000 7fdac0d07640 1 --2- 172.21.15.1:0/3518837122 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fdaa8039060 0x7fdaa803b520 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:34.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:33.597+0000 7fdac0d07640 1 --2- 172.21.15.1:0/3518837122 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdabc1015e0 0x7fdabc198f50 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:34.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:33.597+0000 7fdac0d07640 1 -- 172.21.15.1:0/3518837122 >> 172.21.15.1:0/3518837122 conn(0x7fdabc0fcd70 msgr2=0x7fdabc0ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:48:34.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:33.597+0000 7fdac0d07640 1 -- 172.21.15.1:0/3518837122 shutdown_connections 2024-09-11T21:48:34.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:33.597+0000 7fdac0d07640 1 -- 172.21.15.1:0/3518837122 wait complete. 2024-09-11T21:48:34.642 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:34 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/3518837122' entity='client.admin' 2024-09-11T21:48:34.642 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:34 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:48:34.642 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:34 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:48:34.642 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:34 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi001", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:48:34.643 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:34 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:48:34.643 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:34 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:48:34.643 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:34 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:48:34.669 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-11T21:48:34.670 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-11T21:48:34.920 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:48:35.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:35.331+0000 7f28b1758640 1 -- 172.21.15.1:0/1125959023 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f28ac101b00 msgr2=0x7f28ac101f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:35.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:35.331+0000 7f28b1758640 1 --2- 172.21.15.1:0/1125959023 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f28ac101b00 0x7f28ac101f00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f289c009880 tx=0x7f289c02a5d0 comp rx=0 tx=0).stop 2024-09-11T21:48:35.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:35.331+0000 7f28b1758640 1 -- 172.21.15.1:0/1125959023 shutdown_connections 2024-09-11T21:48:35.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:35.331+0000 7f28b1758640 1 --2- 172.21.15.1:0/1125959023 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f28ac101b00 0x7f28ac101f00 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:35.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:35.331+0000 7f28b1758640 1 -- 172.21.15.1:0/1125959023 >> 172.21.15.1:0/1125959023 conn(0x7f28ac0fb400 msgr2=0x7f28ac0fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:48:35.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:35.331+0000 7f28b1758640 1 -- 172.21.15.1:0/1125959023 shutdown_connections 2024-09-11T21:48:35.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:35.331+0000 7f28b1758640 1 -- 172.21.15.1:0/1125959023 wait complete. 2024-09-11T21:48:35.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:35.332+0000 7f28b1758640 1 Processor -- start 2024-09-11T21:48:35.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:35.332+0000 7f28b1758640 1 -- start start 2024-09-11T21:48:35.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:35.332+0000 7f28b1758640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f28ac101b00 0x7f28ac1000d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:35.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:35.332+0000 7f28b1758640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f28ac106600 con 0x7f28ac101b00 2024-09-11T21:48:35.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:35.333+0000 7f28aaffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f28ac101b00 0x7f28ac1000d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:48:35.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:35.333+0000 7f28aaffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f28ac101b00 0x7f28ac1000d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51606/0 (socket says 172.21.15.1:51606) 2024-09-11T21:48:35.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:35.333+0000 7f28aaffd640 1 -- 172.21.15.1:0/3855469168 learned_addr learned my addr 172.21.15.1:0/3855469168 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:48:35.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:35.333+0000 7f28aaffd640 1 -- 172.21.15.1:0/3855469168 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28ac199b00 con 0x7f28ac101b00 2024-09-11T21:48:35.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:35.333+0000 7f28aaffd640 1 --2- 172.21.15.1:0/3855469168 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f28ac101b00 0x7f28ac1000d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f289c004f60 tx=0x7f289c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:48:35.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:35.333+0000 7f2893fff640 1 -- 172.21.15.1:0/3855469168 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f289c03f070 con 0x7f28ac101b00 2024-09-11T21:48:35.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:35.333+0000 7f2893fff640 1 -- 172.21.15.1:0/3855469168 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f289c004d40 con 0x7f28ac101b00 2024-09-11T21:48:35.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:35.333+0000 7f28b1758640 1 -- 172.21.15.1:0/3855469168 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28ac199cd0 con 0x7f28ac101b00 2024-09-11T21:48:35.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:35.333+0000 7f2893fff640 1 -- 172.21.15.1:0/3855469168 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f289c035030 con 0x7f28ac101b00 2024-09-11T21:48:35.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:35.333+0000 7f28b1758640 1 -- 172.21.15.1:0/3855469168 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28ac19a070 con 0x7f28ac101b00 2024-09-11T21:48:35.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:35.334+0000 7f2893fff640 1 -- 172.21.15.1:0/3855469168 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f289c042070 con 0x7f28ac101b00 2024-09-11T21:48:35.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:35.334+0000 7f28b1758640 1 -- 172.21.15.1:0/3855469168 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28ac106d80 con 0x7f28ac101b00 2024-09-11T21:48:35.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:35.334+0000 7f2893fff640 1 --2- 172.21.15.1:0/3855469168 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f28940390c0 0x7f289403b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:35.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:35.334+0000 7f2893fff640 1 -- 172.21.15.1:0/3855469168 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f289c06cf10 con 0x7f28ac101b00 2024-09-11T21:48:35.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:35.335+0000 7f28aa7fc640 1 --2- 172.21.15.1:0/3855469168 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f28940390c0 0x7f289403b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:48:35.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:35.335+0000 7f28aa7fc640 1 --2- 172.21.15.1:0/3855469168 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f28940390c0 0x7f289403b580 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f28a00098c0 tx=0x7f28a0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:48:35.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:35.339+0000 7f2893fff640 1 -- 172.21.15.1:0/3855469168 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f289c039350 con 0x7f28ac101b00 2024-09-11T21:48:35.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:35.517+0000 7f28b1758640 1 -- 172.21.15.1:0/3855469168 --> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f28ac02d930 con 0x7f28940390c0 2024-09-11T21:48:35.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:35.521+0000 7f2893fff640 1 -- 172.21.15.1:0/3855469168 <== mgr.14150 v2:172.21.15.1:6800/3890350886 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f28ac02d930 con 0x7f28940390c0 2024-09-11T21:48:35.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:35.523+0000 7f28b1758640 1 -- 172.21.15.1:0/3855469168 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f28940390c0 msgr2=0x7f289403b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:35.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:35.523+0000 7f28b1758640 1 --2- 172.21.15.1:0/3855469168 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f28940390c0 0x7f289403b580 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f28a00098c0 tx=0x7f28a0006db0 comp rx=0 tx=0).stop 2024-09-11T21:48:35.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:35.523+0000 7f28b1758640 1 -- 172.21.15.1:0/3855469168 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f28ac101b00 msgr2=0x7f28ac1000d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:35.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:35.523+0000 7f28b1758640 1 --2- 172.21.15.1:0/3855469168 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f28ac101b00 0x7f28ac1000d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f289c004f60 tx=0x7f289c0048f0 comp rx=0 tx=0).stop 2024-09-11T21:48:35.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:35.523+0000 7f28b1758640 1 -- 172.21.15.1:0/3855469168 shutdown_connections 2024-09-11T21:48:35.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:35.523+0000 7f28b1758640 1 --2- 172.21.15.1:0/3855469168 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f28940390c0 0x7f289403b580 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:35.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:35.523+0000 7f28b1758640 1 --2- 172.21.15.1:0/3855469168 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f28ac101b00 0x7f28ac1000d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:35.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:35.523+0000 7f28b1758640 1 -- 172.21.15.1:0/3855469168 >> 172.21.15.1:0/3855469168 conn(0x7f28ac0fb400 msgr2=0x7f28ac0fbe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:48:35.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:35.523+0000 7f28b1758640 1 -- 172.21.15.1:0/3855469168 shutdown_connections 2024-09-11T21:48:35.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:35.523+0000 7f28b1758640 1 -- 172.21.15.1:0/3855469168 wait complete. 2024-09-11T21:48:36.250 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi053 2024-09-11T21:48:36.250 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-11T21:48:36.250 DEBUG:teuthology.orchestra.run.smithi053:> dd of=/etc/ceph/ceph.conf 2024-09-11T21:48:36.270 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-11T21:48:36.270 DEBUG:teuthology.orchestra.run.smithi053:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T21:48:36.328 INFO:tasks.cephadm:Adding host smithi053 to orchestrator... 2024-09-11T21:48:36.328 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph orch host add smithi053 2024-09-11T21:48:36.583 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:48:36.777 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:36 smithi001 ceph-mon[30935]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:48:36.778 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:36 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:48:36.778 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:36 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:48:36.778 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:36 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:48:36.778 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:36 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:48:36.778 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:36 smithi001 ceph-mon[30935]: Updating smithi001:/etc/ceph/ceph.conf 2024-09-11T21:48:36.778 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:36 smithi001 ceph-mon[30935]: Updating smithi001:/var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T21:48:36.778 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:36 smithi001 ceph-mon[30935]: Updating smithi001:/etc/ceph/ceph.client.admin.keyring 2024-09-11T21:48:36.778 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:36 smithi001 ceph-mon[30935]: Updating smithi001:/var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T21:48:36.778 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:36 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:48:36.778 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:36 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:48:36.778 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:36 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:48:36.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:36.960+0000 7f68b7909640 1 -- 172.21.15.1:0/2223090442 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68b0106540 msgr2=0x7f68b0106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:36.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:36.960+0000 7f68b7909640 1 --2- 172.21.15.1:0/2223090442 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68b0106540 0x7f68b0106920 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f68ac009810 tx=0x7f68ac02a540 comp rx=0 tx=0).stop 2024-09-11T21:48:36.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:36.961+0000 7f68b7909640 1 -- 172.21.15.1:0/2223090442 shutdown_connections 2024-09-11T21:48:36.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:36.961+0000 7f68b7909640 1 --2- 172.21.15.1:0/2223090442 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68b0106540 0x7f68b0106920 secure :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f68ac009810 tx=0x7f68ac02a540 comp rx=0 tx=0).stop 2024-09-11T21:48:36.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:36.961+0000 7f68b7909640 1 -- 172.21.15.1:0/2223090442 >> 172.21.15.1:0/2223090442 conn(0x7f68b0077310 msgr2=0x7f68b00fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:48:36.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:36.961+0000 7f68b7909640 1 -- 172.21.15.1:0/2223090442 shutdown_connections 2024-09-11T21:48:36.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:36.961+0000 7f68b7909640 1 -- 172.21.15.1:0/2223090442 wait complete. 2024-09-11T21:48:36.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:36.962+0000 7f68b7909640 1 Processor -- start 2024-09-11T21:48:36.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:36.962+0000 7f68b7909640 1 -- start start 2024-09-11T21:48:36.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:36.962+0000 7f68b7909640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68b00fc9b0 0x7f68b00fcd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:36.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:36.962+0000 7f68b7909640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f68b010ab70 con 0x7f68b00fc9b0 2024-09-11T21:48:36.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:36.962+0000 7f68b56a5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68b00fc9b0 0x7f68b00fcd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:48:36.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:36.962+0000 7f68b56a5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68b00fc9b0 0x7f68b00fcd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42060/0 (socket says 172.21.15.1:42060) 2024-09-11T21:48:36.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:36.962+0000 7f68b56a5640 1 -- 172.21.15.1:0/2448250784 learned_addr learned my addr 172.21.15.1:0/2448250784 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:48:36.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:36.962+0000 7f68b56a5640 1 -- 172.21.15.1:0/2448250784 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68b00fe8a0 con 0x7f68b00fc9b0 2024-09-11T21:48:36.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:36.962+0000 7f68b56a5640 1 --2- 172.21.15.1:0/2448250784 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68b00fc9b0 0x7f68b00fcd90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f68ac030940 tx=0x7f68ac030f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:48:36.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:36.963+0000 7f68a67fc640 1 -- 172.21.15.1:0/2448250784 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f68ac040070 con 0x7f68b00fc9b0 2024-09-11T21:48:36.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:36.963+0000 7f68a67fc640 1 -- 172.21.15.1:0/2448250784 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68ac0318e0 con 0x7f68b00fc9b0 2024-09-11T21:48:36.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:36.963+0000 7f68b7909640 1 -- 172.21.15.1:0/2448250784 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68b00fd2d0 con 0x7f68b00fc9b0 2024-09-11T21:48:36.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:36.963+0000 7f68a67fc640 1 -- 172.21.15.1:0/2448250784 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f68ac0047c0 con 0x7f68b00fc9b0 2024-09-11T21:48:36.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:36.963+0000 7f68b7909640 1 -- 172.21.15.1:0/2448250784 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68b00fd790 con 0x7f68b00fc9b0 2024-09-11T21:48:36.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:36.964+0000 7f68a67fc640 1 -- 172.21.15.1:0/2448250784 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f68ac039040 con 0x7f68b00fc9b0 2024-09-11T21:48:36.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:36.964+0000 7f68b7909640 1 -- 172.21.15.1:0/2448250784 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68b0101bb0 con 0x7f68b00fc9b0 2024-09-11T21:48:36.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:36.964+0000 7f68a67fc640 1 --2- 172.21.15.1:0/2448250784 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f689c039060 0x7f689c03b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:36.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:36.964+0000 7f68a67fc640 1 -- 172.21.15.1:0/2448250784 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f68ac06cf20 con 0x7f68b00fc9b0 2024-09-11T21:48:36.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:36.964+0000 7f68b4ea4640 1 --2- 172.21.15.1:0/2448250784 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f689c039060 0x7f689c03b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:48:36.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:36.964+0000 7f68b4ea4640 1 --2- 172.21.15.1:0/2448250784 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f689c039060 0x7f689c03b520 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f68a00098c0 tx=0x7f68a0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:48:36.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:36.968+0000 7f68a67fc640 1 -- 172.21.15.1:0/2448250784 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f68ac041cb0 con 0x7f68b00fc9b0 2024-09-11T21:48:37.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:37.149+0000 7f68b7909640 1 -- 172.21.15.1:0/2448250784 --> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi053", "target": ["mon-mgr", ""]}) -- 0x7f68b00fdf50 con 0x7f689c039060 2024-09-11T21:48:37.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:37 smithi001 ceph-mon[30935]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi053", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:48:38.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:38 smithi001 ceph-mon[30935]: Deploying cephadm binary to smithi053 2024-09-11T21:48:41.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:41.071+0000 7f68a67fc640 1 -- 172.21.15.1:0/2448250784 <== mgr.14150 v2:172.21.15.1:6800/3890350886 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f68b00fdf50 con 0x7f689c039060 2024-09-11T21:48:41.077 INFO:teuthology.orchestra.run.smithi001.stdout:Added host 'smithi053' with addr '172.21.15.53' 2024-09-11T21:48:41.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:41.073+0000 7f68b7909640 1 -- 172.21.15.1:0/2448250784 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f689c039060 msgr2=0x7f689c03b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:41.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:41.073+0000 7f68b7909640 1 --2- 172.21.15.1:0/2448250784 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f689c039060 0x7f689c03b520 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f68a00098c0 tx=0x7f68a0006db0 comp rx=0 tx=0).stop 2024-09-11T21:48:41.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:41.073+0000 7f68b7909640 1 -- 172.21.15.1:0/2448250784 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68b00fc9b0 msgr2=0x7f68b00fcd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:41.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:41.073+0000 7f68b7909640 1 --2- 172.21.15.1:0/2448250784 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68b00fc9b0 0x7f68b00fcd90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f68ac030940 tx=0x7f68ac030f00 comp rx=0 tx=0).stop 2024-09-11T21:48:41.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:41.073+0000 7f68b7909640 1 -- 172.21.15.1:0/2448250784 shutdown_connections 2024-09-11T21:48:41.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:41.073+0000 7f68b7909640 1 --2- 172.21.15.1:0/2448250784 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f689c039060 0x7f689c03b520 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:41.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:41.073+0000 7f68b7909640 1 --2- 172.21.15.1:0/2448250784 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68b00fc9b0 0x7f68b00fcd90 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:41.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:41.073+0000 7f68b7909640 1 -- 172.21.15.1:0/2448250784 >> 172.21.15.1:0/2448250784 conn(0x7f68b0077310 msgr2=0x7f68b00698f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:48:41.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:41.073+0000 7f68b7909640 1 -- 172.21.15.1:0/2448250784 shutdown_connections 2024-09-11T21:48:41.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:41.073+0000 7f68b7909640 1 -- 172.21.15.1:0/2448250784 wait complete. 2024-09-11T21:48:41.631 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-11T21:48:41.880 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:48:42.270 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:42 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:48:42.270 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:42 smithi001 ceph-mon[30935]: Added host smithi053 2024-09-11T21:48:42.270 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:42 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:48:42.270 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:42 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:48:42.270 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:42 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:48:42.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:42.302+0000 7fd5ff556640 1 -- 172.21.15.1:0/1295481891 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5f8106560 msgr2=0x7fd5f8106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:42.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:42.302+0000 7fd5ff556640 1 --2- 172.21.15.1:0/1295481891 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5f8106560 0x7fd5f8106940 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4009880 tx=0x7fd5f402a5d0 comp rx=0 tx=0).stop 2024-09-11T21:48:42.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:42.302+0000 7fd5ff556640 1 -- 172.21.15.1:0/1295481891 shutdown_connections 2024-09-11T21:48:42.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:42.302+0000 7fd5ff556640 1 --2- 172.21.15.1:0/1295481891 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5f8106560 0x7fd5f8106940 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:42.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:42.302+0000 7fd5ff556640 1 -- 172.21.15.1:0/1295481891 >> 172.21.15.1:0/1295481891 conn(0x7fd5f80fc1b0 msgr2=0x7fd5f80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:48:42.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:42.302+0000 7fd5ff556640 1 -- 172.21.15.1:0/1295481891 shutdown_connections 2024-09-11T21:48:42.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:42.302+0000 7fd5ff556640 1 -- 172.21.15.1:0/1295481891 wait complete. 2024-09-11T21:48:42.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:42.303+0000 7fd5ff556640 1 Processor -- start 2024-09-11T21:48:42.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:42.303+0000 7fd5ff556640 1 -- start start 2024-09-11T21:48:42.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:42.303+0000 7fd5ff556640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5f8106560 0x7fd5f8197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:42.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:42.303+0000 7fd5ff556640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f810ab90 con 0x7fd5f8106560 2024-09-11T21:48:42.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:42.303+0000 7fd5fd2f2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5f8106560 0x7fd5f8197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:48:42.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:42.303+0000 7fd5fd2f2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5f8106560 0x7fd5f8197f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42064/0 (socket says 172.21.15.1:42064) 2024-09-11T21:48:42.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:42.303+0000 7fd5fd2f2640 1 -- 172.21.15.1:0/195061734 learned_addr learned my addr 172.21.15.1:0/195061734 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:48:42.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:42.303+0000 7fd5fd2f2640 1 -- 172.21.15.1:0/195061734 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5f81984b0 con 0x7fd5f8106560 2024-09-11T21:48:42.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:42.304+0000 7fd5fd2f2640 1 --2- 172.21.15.1:0/195061734 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5f8106560 0x7fd5f8197f70 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4009230 tx=0x7fd5f4003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:48:42.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:42.304+0000 7fd5ee7fc640 1 -- 172.21.15.1:0/195061734 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5f403f070 con 0x7fd5f8106560 2024-09-11T21:48:42.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:42.304+0000 7fd5ee7fc640 1 -- 172.21.15.1:0/195061734 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5f403bc80 con 0x7fd5f8106560 2024-09-11T21:48:42.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:42.304+0000 7fd5ff556640 1 -- 172.21.15.1:0/195061734 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5f8198740 con 0x7fd5f8106560 2024-09-11T21:48:42.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:42.304+0000 7fd5ee7fc640 1 -- 172.21.15.1:0/195061734 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5f4035030 con 0x7fd5f8106560 2024-09-11T21:48:42.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:42.304+0000 7fd5ff556640 1 -- 172.21.15.1:0/195061734 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5f8198ba0 con 0x7fd5f8106560 2024-09-11T21:48:42.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:42.305+0000 7fd5ee7fc640 1 -- 172.21.15.1:0/195061734 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7fd5f4042070 con 0x7fd5f8106560 2024-09-11T21:48:42.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:42.305+0000 7fd5ff556640 1 -- 172.21.15.1:0/195061734 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5f8101bd0 con 0x7fd5f8106560 2024-09-11T21:48:42.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:42.305+0000 7fd5ee7fc640 1 --2- 172.21.15.1:0/195061734 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fd5e4039060 0x7fd5e403b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:42.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:42.305+0000 7fd5ee7fc640 1 -- 172.21.15.1:0/195061734 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd5f406c9e0 con 0x7fd5f8106560 2024-09-11T21:48:42.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:42.305+0000 7fd5fcaf1640 1 --2- 172.21.15.1:0/195061734 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fd5e4039060 0x7fd5e403b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:48:42.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:42.306+0000 7fd5fcaf1640 1 --2- 172.21.15.1:0/195061734 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fd5e4039060 0x7fd5e403b520 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd5e80098c0 tx=0x7fd5e8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:48:42.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:42.310+0000 7fd5ee7fc640 1 -- 172.21.15.1:0/195061734 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd5f4031570 con 0x7fd5f8106560 2024-09-11T21:48:42.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:42.490+0000 7fd5ff556640 1 -- 172.21.15.1:0/195061734 --> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd5f81069c0 con 0x7fd5e4039060 2024-09-11T21:48:42.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:42.491+0000 7fd5ee7fc640 1 -- 172.21.15.1:0/195061734 <== mgr.14150 v2:172.21.15.1:6800/3890350886 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+158 (secure 0 0 0) 0x7fd5f81069c0 con 0x7fd5e4039060 2024-09-11T21:48:42.493 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:48:42.493 INFO:teuthology.orchestra.run.smithi001.stdout:[{"addr": "172.21.15.1", "hostname": "smithi001", "labels": [], "status": ""}, {"addr": "172.21.15.53", "hostname": "smithi053", "labels": [], "status": ""}] 2024-09-11T21:48:42.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:42.493+0000 7fd5ff556640 1 -- 172.21.15.1:0/195061734 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fd5e4039060 msgr2=0x7fd5e403b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:42.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:42.493+0000 7fd5ff556640 1 --2- 172.21.15.1:0/195061734 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fd5e4039060 0x7fd5e403b520 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd5e80098c0 tx=0x7fd5e8006db0 comp rx=0 tx=0).stop 2024-09-11T21:48:42.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:42.493+0000 7fd5ff556640 1 -- 172.21.15.1:0/195061734 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5f8106560 msgr2=0x7fd5f8197f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:42.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:42.493+0000 7fd5ff556640 1 --2- 172.21.15.1:0/195061734 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5f8106560 0x7fd5f8197f70 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4009230 tx=0x7fd5f4003d40 comp rx=0 tx=0).stop 2024-09-11T21:48:42.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:42.493+0000 7fd5ff556640 1 -- 172.21.15.1:0/195061734 shutdown_connections 2024-09-11T21:48:42.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:42.493+0000 7fd5ff556640 1 --2- 172.21.15.1:0/195061734 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fd5e4039060 0x7fd5e403b520 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:42.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:42.493+0000 7fd5ff556640 1 --2- 172.21.15.1:0/195061734 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5f8106560 0x7fd5f8197f70 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:42.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:42.493+0000 7fd5ff556640 1 -- 172.21.15.1:0/195061734 >> 172.21.15.1:0/195061734 conn(0x7fd5f80fc1b0 msgr2=0x7fd5f80fc590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:48:42.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:42.493+0000 7fd5ff556640 1 -- 172.21.15.1:0/195061734 shutdown_connections 2024-09-11T21:48:42.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:42.494+0000 7fd5ff556640 1 -- 172.21.15.1:0/195061734 wait complete. 2024-09-11T21:48:43.062 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi161 2024-09-11T21:48:43.062 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-09-11T21:48:43.062 DEBUG:teuthology.orchestra.run.smithi161:> dd of=/etc/ceph/ceph.conf 2024-09-11T21:48:43.082 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-09-11T21:48:43.082 DEBUG:teuthology.orchestra.run.smithi161:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T21:48:43.140 INFO:tasks.cephadm:Adding host smithi161 to orchestrator... 2024-09-11T21:48:43.140 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph orch host add smithi161 2024-09-11T21:48:43.390 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:48:43.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:43 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:48:43.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:43.800+0000 7f17ff919640 1 -- 172.21.15.1:0/4081594719 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17f8107660 msgr2=0x7f17f8107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:43.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:43.800+0000 7f17ff919640 1 --2- 172.21.15.1:0/4081594719 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17f8107660 0x7f17f8107a40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f17f4009880 tx=0x7f17f402a5d0 comp rx=0 tx=0).stop 2024-09-11T21:48:43.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:43.800+0000 7f17ff919640 1 -- 172.21.15.1:0/4081594719 shutdown_connections 2024-09-11T21:48:43.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:43.800+0000 7f17ff919640 1 --2- 172.21.15.1:0/4081594719 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17f8107660 0x7f17f8107a40 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:43.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:43.800+0000 7f17ff919640 1 -- 172.21.15.1:0/4081594719 >> 172.21.15.1:0/4081594719 conn(0x7f17f80fd290 msgr2=0x7f17f80ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:48:43.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:43.801+0000 7f17ff919640 1 -- 172.21.15.1:0/4081594719 shutdown_connections 2024-09-11T21:48:43.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:43.801+0000 7f17ff919640 1 -- 172.21.15.1:0/4081594719 wait complete. 2024-09-11T21:48:43.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:43.801+0000 7f17ff919640 1 Processor -- start 2024-09-11T21:48:43.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:43.802+0000 7f17ff919640 1 -- start start 2024-09-11T21:48:43.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:43.802+0000 7f17ff919640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17f8107660 0x7f17f81970c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:43.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:43.802+0000 7f17ff919640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f17f810bc90 con 0x7f17f8107660 2024-09-11T21:48:43.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:43.802+0000 7f17fd6b5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17f8107660 0x7f17f81970c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:48:43.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:43.802+0000 7f17fd6b5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17f8107660 0x7f17f81970c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42090/0 (socket says 172.21.15.1:42090) 2024-09-11T21:48:43.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:43.802+0000 7f17fd6b5640 1 -- 172.21.15.1:0/3071554145 learned_addr learned my addr 172.21.15.1:0/3071554145 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:48:43.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:43.802+0000 7f17fd6b5640 1 -- 172.21.15.1:0/3071554145 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17f8197600 con 0x7f17f8107660 2024-09-11T21:48:43.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:43.802+0000 7f17fd6b5640 1 --2- 172.21.15.1:0/3071554145 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17f8107660 0x7f17f81970c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f17f4009230 tx=0x7f17f4003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:48:43.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:43.803+0000 7f17ee7fc640 1 -- 172.21.15.1:0/3071554145 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f17f403f070 con 0x7f17f8107660 2024-09-11T21:48:43.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:43.803+0000 7f17ff919640 1 -- 172.21.15.1:0/3071554145 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17f8197890 con 0x7f17f8107660 2024-09-11T21:48:43.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:43.803+0000 7f17ee7fc640 1 -- 172.21.15.1:0/3071554145 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17f403bc80 con 0x7f17f8107660 2024-09-11T21:48:43.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:43.803+0000 7f17ee7fc640 1 -- 172.21.15.1:0/3071554145 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f17f4035030 con 0x7f17f8107660 2024-09-11T21:48:43.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:43.803+0000 7f17ff919640 1 -- 172.21.15.1:0/3071554145 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17f819b990 con 0x7f17f8107660 2024-09-11T21:48:43.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:43.803+0000 7f17ee7fc640 1 -- 172.21.15.1:0/3071554145 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f17f4042070 con 0x7f17f8107660 2024-09-11T21:48:43.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:43.804+0000 7f17ee7fc640 1 --2- 172.21.15.1:0/3071554145 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f17e4039060 0x7f17e403b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:43.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:43.804+0000 7f17ff919640 1 -- 172.21.15.1:0/3071554145 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17f8102cd0 con 0x7f17f8107660 2024-09-11T21:48:43.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:43.804+0000 7f17ee7fc640 1 -- 172.21.15.1:0/3071554145 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f17f406cc90 con 0x7f17f8107660 2024-09-11T21:48:43.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:43.804+0000 7f17fceb4640 1 --2- 172.21.15.1:0/3071554145 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f17e4039060 0x7f17e403b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:48:43.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:43.804+0000 7f17fceb4640 1 --2- 172.21.15.1:0/3071554145 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f17e4039060 0x7f17e403b520 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f17e80098c0 tx=0x7f17e8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:48:43.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:43.809+0000 7f17ee7fc640 1 -- 172.21.15.1:0/3071554145 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f17f404c350 con 0x7f17f8107660 2024-09-11T21:48:43.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:43.992+0000 7f17ff919640 1 -- 172.21.15.1:0/3071554145 --> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi161", "target": ["mon-mgr", ""]}) -- 0x7f17f8107ac0 con 0x7f17e4039060 2024-09-11T21:48:44.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:44 smithi001 ceph-mon[30935]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T21:48:45.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:45 smithi001 ceph-mon[30935]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi161", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:48:45.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:45 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:48:45.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:45 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:48:45.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:45 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:48:45.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:45 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:48:45.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:45 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi053", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:48:45.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:45 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:48:45.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:45 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:48:46.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:46 smithi001 ceph-mon[30935]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:48:46.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:46 smithi001 ceph-mon[30935]: Updating smithi053:/etc/ceph/ceph.conf 2024-09-11T21:48:46.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:46 smithi001 ceph-mon[30935]: Deploying cephadm binary to smithi161 2024-09-11T21:48:46.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:46 smithi001 ceph-mon[30935]: Updating smithi053:/var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T21:48:46.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:46 smithi001 ceph-mon[30935]: Updating smithi053:/etc/ceph/ceph.client.admin.keyring 2024-09-11T21:48:46.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:46 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:48:46.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:46 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:48:46.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:46 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:48:47.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:47 smithi001 ceph-mon[30935]: Updating smithi053:/var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T21:48:47.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:47.908+0000 7f17ee7fc640 1 -- 172.21.15.1:0/3071554145 <== mgr.14150 v2:172.21.15.1:6800/3890350886 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f17f8107ac0 con 0x7f17e4039060 2024-09-11T21:48:47.910 INFO:teuthology.orchestra.run.smithi001.stdout:Added host 'smithi161' with addr '172.21.15.161' 2024-09-11T21:48:47.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:47.911+0000 7f17ff919640 1 -- 172.21.15.1:0/3071554145 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f17e4039060 msgr2=0x7f17e403b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:47.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:47.911+0000 7f17ff919640 1 --2- 172.21.15.1:0/3071554145 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f17e4039060 0x7f17e403b520 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f17e80098c0 tx=0x7f17e8006db0 comp rx=0 tx=0).stop 2024-09-11T21:48:47.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:47.911+0000 7f17ff919640 1 -- 172.21.15.1:0/3071554145 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17f8107660 msgr2=0x7f17f81970c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:47.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:47.911+0000 7f17ff919640 1 --2- 172.21.15.1:0/3071554145 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17f8107660 0x7f17f81970c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f17f4009230 tx=0x7f17f4003d40 comp rx=0 tx=0).stop 2024-09-11T21:48:47.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:47.911+0000 7f17ff919640 1 -- 172.21.15.1:0/3071554145 shutdown_connections 2024-09-11T21:48:47.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:47.911+0000 7f17ff919640 1 --2- 172.21.15.1:0/3071554145 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f17e4039060 0x7f17e403b520 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:47.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:47.911+0000 7f17ff919640 1 --2- 172.21.15.1:0/3071554145 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17f8107660 0x7f17f81970c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:47.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:47.911+0000 7f17ff919640 1 -- 172.21.15.1:0/3071554145 >> 172.21.15.1:0/3071554145 conn(0x7f17f80fd290 msgr2=0x7f17f80febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:48:47.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:47.911+0000 7f17ff919640 1 -- 172.21.15.1:0/3071554145 shutdown_connections 2024-09-11T21:48:47.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:47.911+0000 7f17ff919640 1 -- 172.21.15.1:0/3071554145 wait complete. 2024-09-11T21:48:48.514 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:48 smithi001 ceph-mon[30935]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:48:48.514 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:48 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:48:48.514 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:48 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:48:48.515 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:48 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:48:48.546 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-11T21:48:48.795 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:48:49.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:49.225+0000 7f8332304640 1 -- 172.21.15.1:0/2160093736 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f832c1027b0 msgr2=0x7f832c102b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:49.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:49.225+0000 7f8332304640 1 --2- 172.21.15.1:0/2160093736 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f832c1027b0 0x7f832c102b90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f831c009810 tx=0x7f831c02a540 comp rx=0 tx=0).stop 2024-09-11T21:48:49.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:49.226+0000 7f8332304640 1 -- 172.21.15.1:0/2160093736 shutdown_connections 2024-09-11T21:48:49.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:49.226+0000 7f8332304640 1 --2- 172.21.15.1:0/2160093736 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f832c1027b0 0x7f832c102b90 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:49.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:49.226+0000 7f8332304640 1 -- 172.21.15.1:0/2160093736 >> 172.21.15.1:0/2160093736 conn(0x7f832c077310 msgr2=0x7f832c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:48:49.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:49.226+0000 7f8332304640 1 -- 172.21.15.1:0/2160093736 shutdown_connections 2024-09-11T21:48:49.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:49.226+0000 7f8332304640 1 -- 172.21.15.1:0/2160093736 wait complete. 2024-09-11T21:48:49.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:49.226+0000 7f8332304640 1 Processor -- start 2024-09-11T21:48:49.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:49.226+0000 7f8332304640 1 -- start start 2024-09-11T21:48:49.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:49.227+0000 7f8332304640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f832c1027b0 0x7f832c1971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:49.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:49.227+0000 7f8332304640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f832c10bc80 con 0x7f832c1027b0 2024-09-11T21:48:49.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:49.227+0000 7f832bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f832c1027b0 0x7f832c1971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:48:49.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:49.227+0000 7f832bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f832c1027b0 0x7f832c1971b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55450/0 (socket says 172.21.15.1:55450) 2024-09-11T21:48:49.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:49.227+0000 7f832bfff640 1 -- 172.21.15.1:0/1730227194 learned_addr learned my addr 172.21.15.1:0/1730227194 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:48:49.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:49.227+0000 7f832bfff640 1 -- 172.21.15.1:0/1730227194 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f832c1976f0 con 0x7f832c1027b0 2024-09-11T21:48:49.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:49.227+0000 7f832bfff640 1 --2- 172.21.15.1:0/1730227194 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f832c1027b0 0x7f832c1971b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f831c0045e0 tx=0x7f831c004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:48:49.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:49.227+0000 7f83297fa640 1 -- 172.21.15.1:0/1730227194 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f831c030e70 con 0x7f832c1027b0 2024-09-11T21:48:49.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:49.227+0000 7f83297fa640 1 -- 172.21.15.1:0/1730227194 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f831c0318e0 con 0x7f832c1027b0 2024-09-11T21:48:49.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:49.227+0000 7f8332304640 1 -- 172.21.15.1:0/1730227194 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f832c197980 con 0x7f832c1027b0 2024-09-11T21:48:49.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:49.227+0000 7f83297fa640 1 -- 172.21.15.1:0/1730227194 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f831c0049c0 con 0x7f832c1027b0 2024-09-11T21:48:49.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:49.227+0000 7f8332304640 1 -- 172.21.15.1:0/1730227194 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f832c197de0 con 0x7f832c1027b0 2024-09-11T21:48:49.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:49.228+0000 7f83297fa640 1 -- 172.21.15.1:0/1730227194 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f831c039040 con 0x7f832c1027b0 2024-09-11T21:48:49.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:49.228+0000 7f8332304640 1 -- 172.21.15.1:0/1730227194 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f832c069f10 con 0x7f832c1027b0 2024-09-11T21:48:49.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:49.229+0000 7f83297fa640 1 --2- 172.21.15.1:0/1730227194 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f8314039060 0x7f831403b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:49.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:49.229+0000 7f83297fa640 1 -- 172.21.15.1:0/1730227194 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f831c06d370 con 0x7f832c1027b0 2024-09-11T21:48:49.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:49.229+0000 7f832b7fe640 1 --2- 172.21.15.1:0/1730227194 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f8314039060 0x7f831403b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:48:49.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:49.229+0000 7f832b7fe640 1 --2- 172.21.15.1:0/1730227194 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f8314039060 0x7f831403b520 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f83200098c0 tx=0x7f8320006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:48:49.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:49.234+0000 7f83297fa640 1 -- 172.21.15.1:0/1730227194 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f831c040030 con 0x7f832c1027b0 2024-09-11T21:48:49.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:49.417+0000 7f8332304640 1 -- 172.21.15.1:0/1730227194 --> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f832c198510 con 0x7f8314039060 2024-09-11T21:48:49.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:49.418+0000 7f83297fa640 1 -- 172.21.15.1:0/1730227194 <== mgr.14150 v2:172.21.15.1:6800/3890350886 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+238 (secure 0 0 0) 0x7f832c198510 con 0x7f8314039060 2024-09-11T21:48:49.420 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:48:49.420 INFO:teuthology.orchestra.run.smithi001.stdout:[{"addr": "172.21.15.1", "hostname": "smithi001", "labels": [], "status": ""}, {"addr": "172.21.15.53", "hostname": "smithi053", "labels": [], "status": ""}, {"addr": "172.21.15.161", "hostname": "smithi161", "labels": [], "status": ""}] 2024-09-11T21:48:49.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:49.420+0000 7f8332304640 1 -- 172.21.15.1:0/1730227194 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f8314039060 msgr2=0x7f831403b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:49.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:49.420+0000 7f8332304640 1 --2- 172.21.15.1:0/1730227194 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f8314039060 0x7f831403b520 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f83200098c0 tx=0x7f8320006db0 comp rx=0 tx=0).stop 2024-09-11T21:48:49.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:49.420+0000 7f8332304640 1 -- 172.21.15.1:0/1730227194 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f832c1027b0 msgr2=0x7f832c1971b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:49.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:49.420+0000 7f8332304640 1 --2- 172.21.15.1:0/1730227194 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f832c1027b0 0x7f832c1971b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f831c0045e0 tx=0x7f831c004610 comp rx=0 tx=0).stop 2024-09-11T21:48:49.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:49.420+0000 7f8332304640 1 -- 172.21.15.1:0/1730227194 shutdown_connections 2024-09-11T21:48:49.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:49.420+0000 7f8332304640 1 --2- 172.21.15.1:0/1730227194 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f8314039060 0x7f831403b520 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:49.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:49.420+0000 7f8332304640 1 --2- 172.21.15.1:0/1730227194 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f832c1027b0 0x7f832c1971b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:49.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:49.420+0000 7f8332304640 1 -- 172.21.15.1:0/1730227194 >> 172.21.15.1:0/1730227194 conn(0x7f832c077310 msgr2=0x7f832c105540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:48:49.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:49.420+0000 7f8332304640 1 -- 172.21.15.1:0/1730227194 shutdown_connections 2024-09-11T21:48:49.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:49.421+0000 7f8332304640 1 -- 172.21.15.1:0/1730227194 wait complete. 2024-09-11T21:48:49.853 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:49 smithi001 ceph-mon[30935]: Added host smithi161 2024-09-11T21:48:49.853 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:49 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:48:49.854 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:49 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:48:50.098 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-11T21:48:50.099 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph osd crush tunables default 2024-09-11T21:48:50.349 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:48:50.724 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:50 smithi001 ceph-mon[30935]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:48:50.724 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:50 smithi001 ceph-mon[30935]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T21:48:50.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:50.806+0000 7fe9c7ea5640 1 -- 172.21.15.1:0/568676230 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe9c0106560 msgr2=0x7fe9c0106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:50.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:50.806+0000 7fe9c7ea5640 1 --2- 172.21.15.1:0/568676230 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe9c0106560 0x7fe9c0106940 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc009880 tx=0x7fe9bc02a5d0 comp rx=0 tx=0).stop 2024-09-11T21:48:50.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:50.806+0000 7fe9c7ea5640 1 -- 172.21.15.1:0/568676230 shutdown_connections 2024-09-11T21:48:50.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:50.806+0000 7fe9c7ea5640 1 --2- 172.21.15.1:0/568676230 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe9c0106560 0x7fe9c0106940 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:50.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:50.806+0000 7fe9c7ea5640 1 -- 172.21.15.1:0/568676230 >> 172.21.15.1:0/568676230 conn(0x7fe9c00fc1b0 msgr2=0x7fe9c00fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:48:50.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:50.806+0000 7fe9c7ea5640 1 -- 172.21.15.1:0/568676230 shutdown_connections 2024-09-11T21:48:50.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:50.806+0000 7fe9c7ea5640 1 -- 172.21.15.1:0/568676230 wait complete. 2024-09-11T21:48:50.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:50.807+0000 7fe9c7ea5640 1 Processor -- start 2024-09-11T21:48:50.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:50.807+0000 7fe9c7ea5640 1 -- start start 2024-09-11T21:48:50.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:50.807+0000 7fe9c7ea5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe9c0106560 0x7fe9c0197120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:50.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:50.807+0000 7fe9c7ea5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c010ab90 con 0x7fe9c0106560 2024-09-11T21:48:50.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:50.807+0000 7fe9c5c41640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe9c0106560 0x7fe9c0197120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:48:50.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:50.807+0000 7fe9c5c41640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe9c0106560 0x7fe9c0197120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55464/0 (socket says 172.21.15.1:55464) 2024-09-11T21:48:50.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:50.807+0000 7fe9c5c41640 1 -- 172.21.15.1:0/3135701706 learned_addr learned my addr 172.21.15.1:0/3135701706 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:48:50.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:50.807+0000 7fe9c5c41640 1 -- 172.21.15.1:0/3135701706 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9c0197660 con 0x7fe9c0106560 2024-09-11T21:48:50.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:50.808+0000 7fe9c5c41640 1 --2- 172.21.15.1:0/3135701706 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe9c0106560 0x7fe9c0197120 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc009230 tx=0x7fe9bc003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:48:50.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:50.808+0000 7fe9b6ffd640 1 -- 172.21.15.1:0/3135701706 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9bc03f070 con 0x7fe9c0106560 2024-09-11T21:48:50.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:50.808+0000 7fe9b6ffd640 1 -- 172.21.15.1:0/3135701706 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9bc03bc80 con 0x7fe9c0106560 2024-09-11T21:48:50.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:50.808+0000 7fe9c7ea5640 1 -- 172.21.15.1:0/3135701706 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9c01978f0 con 0x7fe9c0106560 2024-09-11T21:48:50.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:50.808+0000 7fe9b6ffd640 1 -- 172.21.15.1:0/3135701706 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9bc035030 con 0x7fe9c0106560 2024-09-11T21:48:50.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:50.808+0000 7fe9c7ea5640 1 -- 172.21.15.1:0/3135701706 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9c019b9f0 con 0x7fe9c0106560 2024-09-11T21:48:50.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:50.809+0000 7fe9b6ffd640 1 -- 172.21.15.1:0/3135701706 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7fe9bc042070 con 0x7fe9c0106560 2024-09-11T21:48:50.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:50.809+0000 7fe9c7ea5640 1 -- 172.21.15.1:0/3135701706 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9c0101bd0 con 0x7fe9c0106560 2024-09-11T21:48:50.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:50.809+0000 7fe9b6ffd640 1 --2- 172.21.15.1:0/3135701706 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fe9ac0390c0 0x7fe9ac03b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:50.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:50.809+0000 7fe9b6ffd640 1 -- 172.21.15.1:0/3135701706 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe9bc06c8e0 con 0x7fe9c0106560 2024-09-11T21:48:50.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:50.809+0000 7fe9c5440640 1 --2- 172.21.15.1:0/3135701706 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fe9ac0390c0 0x7fe9ac03b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:48:50.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:50.810+0000 7fe9c5440640 1 --2- 172.21.15.1:0/3135701706 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fe9ac0390c0 0x7fe9ac03b580 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe9b00098c0 tx=0x7fe9b0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:48:50.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:50.814+0000 7fe9b6ffd640 1 -- 172.21.15.1:0/3135701706 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe9bc042360 con 0x7fe9c0106560 2024-09-11T21:48:50.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:50.995+0000 7fe9c7ea5640 1 -- 172.21.15.1:0/3135701706 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fe9c01069c0 con 0x7fe9c0106560 2024-09-11T21:48:51.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:51.498+0000 7fe9b6ffd640 1 -- 172.21.15.1:0/3135701706 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fe9bc031e20 con 0x7fe9c0106560 2024-09-11T21:48:51.500 INFO:teuthology.orchestra.run.smithi001.stderr:adjusted tunables profile to default 2024-09-11T21:48:51.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:51.500+0000 7fe9c7ea5640 1 -- 172.21.15.1:0/3135701706 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fe9ac0390c0 msgr2=0x7fe9ac03b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:51.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:51.500+0000 7fe9c7ea5640 1 --2- 172.21.15.1:0/3135701706 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fe9ac0390c0 0x7fe9ac03b580 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe9b00098c0 tx=0x7fe9b0006db0 comp rx=0 tx=0).stop 2024-09-11T21:48:51.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:51.500+0000 7fe9c7ea5640 1 -- 172.21.15.1:0/3135701706 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe9c0106560 msgr2=0x7fe9c0197120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:51.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:51.500+0000 7fe9c7ea5640 1 --2- 172.21.15.1:0/3135701706 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe9c0106560 0x7fe9c0197120 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc009230 tx=0x7fe9bc003d40 comp rx=0 tx=0).stop 2024-09-11T21:48:51.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:51.500+0000 7fe9c7ea5640 1 -- 172.21.15.1:0/3135701706 shutdown_connections 2024-09-11T21:48:51.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:51.500+0000 7fe9c7ea5640 1 --2- 172.21.15.1:0/3135701706 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fe9ac0390c0 0x7fe9ac03b580 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:51.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:51.500+0000 7fe9c7ea5640 1 --2- 172.21.15.1:0/3135701706 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe9c0106560 0x7fe9c0197120 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:51.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:51.500+0000 7fe9c7ea5640 1 -- 172.21.15.1:0/3135701706 >> 172.21.15.1:0/3135701706 conn(0x7fe9c00fc1b0 msgr2=0x7fe9c0069ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:48:51.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:51.500+0000 7fe9c7ea5640 1 -- 172.21.15.1:0/3135701706 shutdown_connections 2024-09-11T21:48:51.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:48:51.500+0000 7fe9c7ea5640 1 -- 172.21.15.1:0/3135701706 wait complete. 2024-09-11T21:48:51.802 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:51 smithi001 ceph-mon[30935]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:48:51.802 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:51 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/3135701706' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-11T21:48:52.088 INFO:tasks.cephadm:Adding mon.a on smithi001 2024-09-11T21:48:52.088 INFO:tasks.cephadm:Adding mon.b on smithi053 2024-09-11T21:48:52.088 INFO:tasks.cephadm:Adding mon.c on smithi161 2024-09-11T21:48:52.088 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph orch apply mon '3;smithi001:172.21.15.1=a;smithi053:172.21.15.53=b;smithi161:172.21.15.161=c' 2024-09-11T21:48:52.359 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-11T21:48:52.404 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-11T21:48:52.928 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:52.879+0000 7f3342024640 1 -- 172.21.15.161:0/3197363188 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f333c100510 msgr2=0x7f333c100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:52.929 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:52.879+0000 7f3342024640 1 --2- 172.21.15.161:0/3197363188 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f333c100510 0x7f333c100910 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f332c009880 tx=0x7f332c02a5d0 comp rx=0 tx=0).stop 2024-09-11T21:48:52.929 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:52.879+0000 7f3342024640 1 -- 172.21.15.161:0/3197363188 shutdown_connections 2024-09-11T21:48:52.929 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:52.879+0000 7f3342024640 1 --2- 172.21.15.161:0/3197363188 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f333c100510 0x7f333c100910 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:52.929 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:52.879+0000 7f3342024640 1 -- 172.21.15.161:0/3197363188 >> 172.21.15.161:0/3197363188 conn(0x7f333c0fbc80 msgr2=0x7f333c0fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:48:52.929 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:52.879+0000 7f3342024640 1 -- 172.21.15.161:0/3197363188 shutdown_connections 2024-09-11T21:48:52.929 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:52.879+0000 7f3342024640 1 -- 172.21.15.161:0/3197363188 wait complete. 2024-09-11T21:48:52.929 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:52.880+0000 7f3342024640 1 Processor -- start 2024-09-11T21:48:52.929 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:52.880+0000 7f3342024640 1 -- start start 2024-09-11T21:48:52.929 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:52.880+0000 7f3342024640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f333c100510 0x7f333c195c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:52.929 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:52.880+0000 7f3342024640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f333c106620 con 0x7f333c100510 2024-09-11T21:48:52.930 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:52.880+0000 7f333b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f333c100510 0x7f333c195c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:48:52.930 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:52.880+0000 7f333b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f333c100510 0x7f333c195c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.161:56458/0 (socket says 172.21.15.161:56458) 2024-09-11T21:48:52.930 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:52.880+0000 7f333b7fe640 1 -- 172.21.15.161:0/1141645377 learned_addr learned my addr 172.21.15.161:0/1141645377 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-09-11T21:48:52.930 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:52.881+0000 7f333b7fe640 1 -- 172.21.15.161:0/1141645377 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f333c1961d0 con 0x7f333c100510 2024-09-11T21:48:52.930 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:52.881+0000 7f333b7fe640 1 --2- 172.21.15.161:0/1141645377 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f333c100510 0x7f333c195c90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f332c004f60 tx=0x7f332c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:48:52.930 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:52.881+0000 7f3338ff9640 1 -- 172.21.15.161:0/1141645377 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f332c03f070 con 0x7f333c100510 2024-09-11T21:48:52.930 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:52.881+0000 7f3342024640 1 -- 172.21.15.161:0/1141645377 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f333c196460 con 0x7f333c100510 2024-09-11T21:48:52.930 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:52.881+0000 7f3338ff9640 1 -- 172.21.15.161:0/1141645377 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f332c004cc0 con 0x7f333c100510 2024-09-11T21:48:52.930 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:52.881+0000 7f3338ff9640 1 -- 172.21.15.161:0/1141645377 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f332c035030 con 0x7f333c100510 2024-09-11T21:48:52.930 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:52.881+0000 7f3342024640 1 -- 172.21.15.161:0/1141645377 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f333c199150 con 0x7f333c100510 2024-09-11T21:48:52.930 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:52.882+0000 7f3338ff9640 1 -- 172.21.15.161:0/1141645377 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f332c042070 con 0x7f333c100510 2024-09-11T21:48:52.930 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:52.882+0000 7f3342024640 1 -- 172.21.15.161:0/1141645377 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f333c106da0 con 0x7f333c100510 2024-09-11T21:48:52.931 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:52.882+0000 7f3338ff9640 1 --2- 172.21.15.161:0/1141645377 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f3324039060 0x7f332403b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:52.931 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:52.882+0000 7f3338ff9640 1 -- 172.21.15.161:0/1141645377 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f332c06cf60 con 0x7f333c100510 2024-09-11T21:48:52.931 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:52.882+0000 7f333affd640 1 --2- 172.21.15.161:0/1141645377 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f3324039060 0x7f332403b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:48:52.931 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:52.883+0000 7f333affd640 1 --2- 172.21.15.161:0/1141645377 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f3324039060 0x7f332403b520 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f33300098c0 tx=0x7f3330006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:48:52.931 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:52.887+0000 7f3338ff9640 1 -- 172.21.15.161:0/1141645377 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f332c003dd0 con 0x7f333c100510 2024-09-11T21:48:52.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:52 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/3135701706' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-11T21:48:52.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:52 smithi001 ceph-mon[30935]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T21:48:52.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:52 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:48:52.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:52 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:48:52.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:52 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:48:52.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:52 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:48:52.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:52 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi161", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:48:52.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:52 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:48:52.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:52 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:48:52.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:52 smithi001 ceph-mon[30935]: Updating smithi161:/etc/ceph/ceph.conf 2024-09-11T21:48:52.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:52 smithi001 ceph-mon[30935]: Updating smithi161:/var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T21:48:53.075 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:53.073+0000 7f3342024640 1 -- 172.21.15.161:0/1141645377 --> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi001:172.21.15.1=a;smithi053:172.21.15.53=b;smithi161:172.21.15.161=c", "target": ["mon-mgr", ""]}) -- 0x7f333c02d8e0 con 0x7f3324039060 2024-09-11T21:48:53.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:53.078+0000 7f3338ff9640 1 -- 172.21.15.161:0/1141645377 <== mgr.14150 v2:172.21.15.1:6800/3890350886 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f333c02d8e0 con 0x7f3324039060 2024-09-11T21:48:53.080 INFO:teuthology.orchestra.run.smithi161.stdout:Scheduled mon update... 2024-09-11T21:48:53.082 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:53.081+0000 7f3342024640 1 -- 172.21.15.161:0/1141645377 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f3324039060 msgr2=0x7f332403b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:53.082 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:53.081+0000 7f3342024640 1 --2- 172.21.15.161:0/1141645377 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f3324039060 0x7f332403b520 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f33300098c0 tx=0x7f3330006db0 comp rx=0 tx=0).stop 2024-09-11T21:48:53.082 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:53.081+0000 7f3342024640 1 -- 172.21.15.161:0/1141645377 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f333c100510 msgr2=0x7f333c195c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:53.082 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:53.081+0000 7f3342024640 1 --2- 172.21.15.161:0/1141645377 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f333c100510 0x7f333c195c90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f332c004f60 tx=0x7f332c0048f0 comp rx=0 tx=0).stop 2024-09-11T21:48:53.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:53.081+0000 7f3342024640 1 -- 172.21.15.161:0/1141645377 shutdown_connections 2024-09-11T21:48:53.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:53.081+0000 7f3342024640 1 --2- 172.21.15.161:0/1141645377 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f3324039060 0x7f332403b520 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:53.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:53.081+0000 7f3342024640 1 --2- 172.21.15.161:0/1141645377 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f333c100510 0x7f333c195c90 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:53.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:53.081+0000 7f3342024640 1 -- 172.21.15.161:0/1141645377 >> 172.21.15.161:0/1141645377 conn(0x7f333c0fbc80 msgr2=0x7f333c0fc6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:48:53.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:53.081+0000 7f3342024640 1 -- 172.21.15.161:0/1141645377 shutdown_connections 2024-09-11T21:48:53.083 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:53.081+0000 7f3342024640 1 -- 172.21.15.161:0/1141645377 wait complete. 2024-09-11T21:48:53.635 DEBUG:teuthology.orchestra.run.smithi053:mon.b> sudo journalctl -f -n 0 -u ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mon.b.service 2024-09-11T21:48:53.638 DEBUG:teuthology.orchestra.run.smithi161:mon.c> sudo journalctl -f -n 0 -u ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mon.c.service 2024-09-11T21:48:53.641 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T21:48:53.641 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T21:48:53.911 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T21:48:54.340 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:54.338+0000 7fadbabf6640 1 -- 172.21.15.161:0/4178058308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fadb4106540 msgr2=0x7fadb4106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:54.340 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:54.338+0000 7fadbabf6640 1 --2- 172.21.15.161:0/4178058308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fadb4106540 0x7fadb4106920 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fada4009810 tx=0x7fada402a540 comp rx=0 tx=0).stop 2024-09-11T21:48:54.340 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:54.338+0000 7fadbabf6640 1 -- 172.21.15.161:0/4178058308 shutdown_connections 2024-09-11T21:48:54.340 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:54.338+0000 7fadbabf6640 1 --2- 172.21.15.161:0/4178058308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fadb4106540 0x7fadb4106920 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:54.340 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:54.338+0000 7fadbabf6640 1 -- 172.21.15.161:0/4178058308 >> 172.21.15.161:0/4178058308 conn(0x7fadb4077310 msgr2=0x7fadb40fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:48:54.340 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:54.338+0000 7fadbabf6640 1 -- 172.21.15.161:0/4178058308 shutdown_connections 2024-09-11T21:48:54.340 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:54.338+0000 7fadbabf6640 1 -- 172.21.15.161:0/4178058308 wait complete. 2024-09-11T21:48:54.341 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:54.339+0000 7fadbabf6640 1 Processor -- start 2024-09-11T21:48:54.341 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:54.339+0000 7fadbabf6640 1 -- start start 2024-09-11T21:48:54.342 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:54.339+0000 7fadbabf6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fadb4106540 0x7fadb4197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:54.342 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:54.339+0000 7fadbabf6640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fadb410ab70 con 0x7fadb4106540 2024-09-11T21:48:54.342 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:54.339+0000 7fadb8992640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fadb4106540 0x7fadb4197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:48:54.342 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:54.339+0000 7fadb8992640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fadb4106540 0x7fadb4197180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.161:56474/0 (socket says 172.21.15.161:56474) 2024-09-11T21:48:54.342 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:54.339+0000 7fadb8992640 1 -- 172.21.15.161:0/229598613 learned_addr learned my addr 172.21.15.161:0/229598613 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-09-11T21:48:54.342 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:54.340+0000 7fadb8992640 1 -- 172.21.15.161:0/229598613 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadb41976c0 con 0x7fadb4106540 2024-09-11T21:48:54.342 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:54.340+0000 7fadb8992640 1 --2- 172.21.15.161:0/229598613 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fadb4106540 0x7fadb4197180 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fada40045e0 tx=0x7fada4004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:48:54.342 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:54.340+0000 7fadb1ffb640 1 -- 172.21.15.161:0/229598613 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fada4030e70 con 0x7fadb4106540 2024-09-11T21:48:54.342 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:54.340+0000 7fadbabf6640 1 -- 172.21.15.161:0/229598613 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadb4197950 con 0x7fadb4106540 2024-09-11T21:48:54.342 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:54.340+0000 7fadbabf6640 1 -- 172.21.15.161:0/229598613 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadb4197db0 con 0x7fadb4106540 2024-09-11T21:48:54.343 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:54.340+0000 7fadb1ffb640 1 -- 172.21.15.161:0/229598613 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fada40318e0 con 0x7fadb4106540 2024-09-11T21:48:54.343 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:54.340+0000 7fadb1ffb640 1 -- 172.21.15.161:0/229598613 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fada4049060 con 0x7fadb4106540 2024-09-11T21:48:54.343 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:54.341+0000 7fadbabf6640 1 -- 172.21.15.161:0/229598613 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadb4101bb0 con 0x7fadb4106540 2024-09-11T21:48:54.344 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:54.341+0000 7fadb1ffb640 1 -- 172.21.15.161:0/229598613 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7fada4039040 con 0x7fadb4106540 2024-09-11T21:48:54.344 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:54.342+0000 7fadb1ffb640 1 --2- 172.21.15.161:0/229598613 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fad9c039060 0x7fad9c03b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:54.344 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:54.342+0000 7fadb1ffb640 1 -- 172.21.15.161:0/229598613 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fada406cf50 con 0x7fadb4106540 2024-09-11T21:48:54.344 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:54.342+0000 7fadb3fff640 1 --2- 172.21.15.161:0/229598613 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fad9c039060 0x7fad9c03b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:48:54.344 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:54.342+0000 7fadb3fff640 1 --2- 172.21.15.161:0/229598613 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fad9c039060 0x7fad9c03b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fada80098c0 tx=0x7fada8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:48:54.348 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:54.347+0000 7fadb1ffb640 1 -- 172.21.15.161:0/229598613 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fada4040030 con 0x7fadb4106540 2024-09-11T21:48:54.351 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:53 smithi001 ceph-mon[30935]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:48:54.351 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:53 smithi001 ceph-mon[30935]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2024-09-11T21:48:54.351 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:53 smithi001 ceph-mon[30935]: Updating smithi161:/var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T21:48:54.351 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:53 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:48:54.351 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:53 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:48:54.352 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:53 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:48:54.352 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:53 smithi001 ceph-mon[30935]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi001:172.21.15.1=a;smithi053:172.21.15.53=b;smithi161:172.21.15.161=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:48:54.352 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:53 smithi001 ceph-mon[30935]: Saving service mon spec with placement smithi001:172.21.15.1=a;smithi053:172.21.15.53=b;smithi161:172.21.15.161=c;count:3 2024-09-11T21:48:54.352 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:53 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:48:54.352 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:53 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:48:54.352 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:53 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:48:54.352 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:53 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:48:54.352 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:53 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:48:54.352 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:53 smithi001 ceph-mon[30935]: Reconfiguring mon.a (unknown last config time)... 2024-09-11T21:48:54.352 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:53 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T21:48:54.352 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:53 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T21:48:54.352 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:53 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:48:54.352 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:53 smithi001 ceph-mon[30935]: Reconfiguring daemon mon.a on smithi001 2024-09-11T21:48:54.588 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:54.587+0000 7fadbabf6640 1 -- 172.21.15.161:0/229598613 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fadb4198540 con 0x7fadb4106540 2024-09-11T21:48:54.589 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:54.587+0000 7fadb1ffb640 1 -- 172.21.15.161:0/229598613 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+741 (secure 0 0 0) 0x7fada40433b0 con 0x7fadb4106540 2024-09-11T21:48:54.589 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-11T21:48:54.589 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":1,"fsid":"3ffca83c-7087-11ef-bcea-c7b262605968","modified":"2024-09-11T21:47:25.953679Z","created":"2024-09-11T21:47:25.953679Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:3300","nonce":0},{"type":"v1","addr":"172.21.15.1:6789","nonce":0}]},"addr":"172.21.15.1:6789/0","public_addr":"172.21.15.1:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-11T21:48:54.589 INFO:teuthology.orchestra.run.smithi161.stderr:dumped monmap epoch 1 2024-09-11T21:48:54.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:54.590+0000 7fadbabf6640 1 -- 172.21.15.161:0/229598613 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fad9c039060 msgr2=0x7fad9c03b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:54.592 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:54.590+0000 7fadbabf6640 1 --2- 172.21.15.161:0/229598613 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fad9c039060 0x7fad9c03b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fada80098c0 tx=0x7fada8006db0 comp rx=0 tx=0).stop 2024-09-11T21:48:54.592 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:54.590+0000 7fadbabf6640 1 -- 172.21.15.161:0/229598613 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fadb4106540 msgr2=0x7fadb4197180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:54.592 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:54.590+0000 7fadbabf6640 1 --2- 172.21.15.161:0/229598613 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fadb4106540 0x7fadb4197180 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fada40045e0 tx=0x7fada4004610 comp rx=0 tx=0).stop 2024-09-11T21:48:54.592 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:54.590+0000 7fadbabf6640 1 -- 172.21.15.161:0/229598613 shutdown_connections 2024-09-11T21:48:54.592 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:54.590+0000 7fadbabf6640 1 --2- 172.21.15.161:0/229598613 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fad9c039060 0x7fad9c03b520 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:54.592 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:54.590+0000 7fadbabf6640 1 --2- 172.21.15.161:0/229598613 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fadb4106540 0x7fadb4197180 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:54.592 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:54.590+0000 7fadbabf6640 1 -- 172.21.15.161:0/229598613 >> 172.21.15.161:0/229598613 conn(0x7fadb4077310 msgr2=0x7fadb40fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:48:54.592 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:54.590+0000 7fadbabf6640 1 -- 172.21.15.161:0/229598613 shutdown_connections 2024-09-11T21:48:54.592 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:54.590+0000 7fadbabf6640 1 -- 172.21.15.161:0/229598613 wait complete. 2024-09-11T21:48:55.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:55 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:48:55.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:55 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:48:55.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:55 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:48:55.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:55 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:48:55.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:55 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:48:55.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:55 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:48:55.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:55 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T21:48:55.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:55 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:48:55.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:55 smithi001 ceph-mon[30935]: Deploying daemon mon.c on smithi161 2024-09-11T21:48:55.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:55 smithi001 ceph-mon[30935]: from='client.? 172.21.15.161:0/229598613' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T21:48:56.167 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T21:48:56.167 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T21:48:56.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:56 smithi001 ceph-mon[30935]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:48:56.892 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T21:48:57.308 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:57.305+0000 7f956e6bb640 1 -- 172.21.15.161:0/3345411491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9568106540 msgr2=0x7f9568106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:57.308 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:57.305+0000 7f956e6bb640 1 --2- 172.21.15.161:0/3345411491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9568106540 0x7f9568106920 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9558009810 tx=0x7f955802a540 comp rx=0 tx=0).stop 2024-09-11T21:48:57.308 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:57.305+0000 7f956e6bb640 1 -- 172.21.15.161:0/3345411491 shutdown_connections 2024-09-11T21:48:57.308 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:57.305+0000 7f956e6bb640 1 --2- 172.21.15.161:0/3345411491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9568106540 0x7f9568106920 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:57.308 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:57.306+0000 7f956e6bb640 1 -- 172.21.15.161:0/3345411491 >> 172.21.15.161:0/3345411491 conn(0x7f9568077310 msgr2=0x7f95680fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:48:57.308 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:57.306+0000 7f956e6bb640 1 -- 172.21.15.161:0/3345411491 shutdown_connections 2024-09-11T21:48:57.308 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:57.306+0000 7f956e6bb640 1 -- 172.21.15.161:0/3345411491 wait complete. 2024-09-11T21:48:57.308 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:57.306+0000 7f956e6bb640 1 Processor -- start 2024-09-11T21:48:57.308 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:57.306+0000 7f956e6bb640 1 -- start start 2024-09-11T21:48:57.309 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:57.306+0000 7f956e6bb640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9568106540 0x7f9568197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:57.309 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:57.306+0000 7f956e6bb640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f956810ab70 con 0x7f9568106540 2024-09-11T21:48:57.310 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:57.307+0000 7f9567fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9568106540 0x7f9568197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:48:57.310 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:57.307+0000 7f9567fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9568106540 0x7f9568197180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.161:56488/0 (socket says 172.21.15.161:56488) 2024-09-11T21:48:57.310 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:57.307+0000 7f9567fff640 1 -- 172.21.15.161:0/4243359582 learned_addr learned my addr 172.21.15.161:0/4243359582 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-09-11T21:48:57.310 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:57.307+0000 7f9567fff640 1 -- 172.21.15.161:0/4243359582 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95681976c0 con 0x7f9568106540 2024-09-11T21:48:57.310 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:57.308+0000 7f9567fff640 1 --2- 172.21.15.161:0/4243359582 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9568106540 0x7f9568197180 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9558009930 tx=0x7f9558004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:48:57.310 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:57.308+0000 7f9565ffb640 1 -- 172.21.15.161:0/4243359582 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9558031400 con 0x7f9568106540 2024-09-11T21:48:57.311 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:57.308+0000 7f956e6bb640 1 -- 172.21.15.161:0/4243359582 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9568197950 con 0x7f9568106540 2024-09-11T21:48:57.311 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:57.308+0000 7f9565ffb640 1 -- 172.21.15.161:0/4243359582 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95580035f0 con 0x7f9568106540 2024-09-11T21:48:57.311 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:57.308+0000 7f9565ffb640 1 -- 172.21.15.161:0/4243359582 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f955803c440 con 0x7f9568106540 2024-09-11T21:48:57.311 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:57.308+0000 7f956e6bb640 1 -- 172.21.15.161:0/4243359582 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9568197db0 con 0x7f9568106540 2024-09-11T21:48:57.311 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:57.309+0000 7f9565ffb640 1 -- 172.21.15.161:0/4243359582 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f9558039040 con 0x7f9568106540 2024-09-11T21:48:57.311 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:57.309+0000 7f956e6bb640 1 -- 172.21.15.161:0/4243359582 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9568101bb0 con 0x7f9568106540 2024-09-11T21:48:57.311 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:57.309+0000 7f9565ffb640 1 --2- 172.21.15.161:0/4243359582 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f95540390c0 0x7f955403b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:48:57.311 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:57.309+0000 7f9565ffb640 1 -- 172.21.15.161:0/4243359582 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f955806d2d0 con 0x7f9568106540 2024-09-11T21:48:57.311 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:57.310+0000 7f955ffff640 1 --2- 172.21.15.161:0/4243359582 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f95540390c0 0x7f955403b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:48:57.312 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:57.311+0000 7f955ffff640 1 --2- 172.21.15.161:0/4243359582 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f95540390c0 0x7f955403b580 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f95500098c0 tx=0x7f9550006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:48:57.316 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:57.315+0000 7f9565ffb640 1 -- 172.21.15.161:0/4243359582 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9558040030 con 0x7f9568106540 2024-09-11T21:48:57.552 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:57.550+0000 7f956e6bb640 1 -- 172.21.15.161:0/4243359582 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f9568198540 con 0x7f9568106540 2024-09-11T21:48:57.553 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:57.551+0000 7f9565ffb640 1 -- 172.21.15.161:0/4243359582 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+741 (secure 0 0 0) 0x7f955804a530 con 0x7f9568106540 2024-09-11T21:48:57.553 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-11T21:48:57.553 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":1,"fsid":"3ffca83c-7087-11ef-bcea-c7b262605968","modified":"2024-09-11T21:47:25.953679Z","created":"2024-09-11T21:47:25.953679Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:3300","nonce":0},{"type":"v1","addr":"172.21.15.1:6789","nonce":0}]},"addr":"172.21.15.1:6789/0","public_addr":"172.21.15.1:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-11T21:48:57.553 INFO:teuthology.orchestra.run.smithi161.stderr:dumped monmap epoch 1 2024-09-11T21:48:57.555 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:57.553+0000 7f956e6bb640 1 -- 172.21.15.161:0/4243359582 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f95540390c0 msgr2=0x7f955403b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:57.555 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:57.553+0000 7f956e6bb640 1 --2- 172.21.15.161:0/4243359582 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f95540390c0 0x7f955403b580 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f95500098c0 tx=0x7f9550006db0 comp rx=0 tx=0).stop 2024-09-11T21:48:57.555 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:57.553+0000 7f956e6bb640 1 -- 172.21.15.161:0/4243359582 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9568106540 msgr2=0x7f9568197180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:48:57.555 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:57.553+0000 7f956e6bb640 1 --2- 172.21.15.161:0/4243359582 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9568106540 0x7f9568197180 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9558009930 tx=0x7f9558004570 comp rx=0 tx=0).stop 2024-09-11T21:48:57.555 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:57.553+0000 7f956e6bb640 1 -- 172.21.15.161:0/4243359582 shutdown_connections 2024-09-11T21:48:57.555 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:57.553+0000 7f956e6bb640 1 --2- 172.21.15.161:0/4243359582 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f95540390c0 0x7f955403b580 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:57.555 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:57.553+0000 7f956e6bb640 1 --2- 172.21.15.161:0/4243359582 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9568106540 0x7f9568197180 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:48:57.555 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:57.553+0000 7f956e6bb640 1 -- 172.21.15.161:0/4243359582 >> 172.21.15.161:0/4243359582 conn(0x7f9568077310 msgr2=0x7f95680fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:48:57.556 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:57.553+0000 7f956e6bb640 1 -- 172.21.15.161:0/4243359582 shutdown_connections 2024-09-11T21:48:57.556 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:48:57.554+0000 7f956e6bb640 1 -- 172.21.15.161:0/4243359582 wait complete. 2024-09-11T21:48:58.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:58 smithi001 ceph-mon[30935]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:48:58.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:48:58 smithi001 ceph-mon[30935]: from='client.? 172.21.15.161:0/4243359582' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T21:48:59.369 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T21:48:59.370 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T21:48:59.627 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T21:49:04.459 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:04.456+0000 7fb1e06b9640 1 -- 172.21.15.161:0/2764092806 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1c4003530 msgr2=0x7fb1c40059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:49:04.459 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:04.456+0000 7fb1e06b9640 1 --2- 172.21.15.161:0/2764092806 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1c4003530 0x7fb1c40059c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb1d8101170 tx=0x7fb1c802cbf0 comp rx=0 tx=0).stop 2024-09-11T21:49:04.459 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:04.456+0000 7fb1e06b9640 1 -- 172.21.15.161:0/2764092806 shutdown_connections 2024-09-11T21:49:04.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:04.456+0000 7fb1e06b9640 1 --2- 172.21.15.161:0/2764092806 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1c4003530 0x7fb1c40059c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:49:04.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:04.456+0000 7fb1e06b9640 1 -- 172.21.15.161:0/2764092806 >> 172.21.15.161:0/2764092806 conn(0x7fb1d80fb9a0 msgr2=0x7fb1d80fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:49:04.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:04.456+0000 7fb1e06b9640 1 -- 172.21.15.161:0/2764092806 shutdown_connections 2024-09-11T21:49:04.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:04.456+0000 7fb1e06b9640 1 -- 172.21.15.161:0/2764092806 wait complete. 2024-09-11T21:49:04.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:04.457+0000 7fb1e06b9640 1 Processor -- start 2024-09-11T21:49:04.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:04.457+0000 7fb1e06b9640 1 -- start start 2024-09-11T21:49:04.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:04.457+0000 7fb1e06b9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1c4003530 0x7fb1d81972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:49:04.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:04.457+0000 7fb1e06b9640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1d81977e0 0x7fb1d819bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:49:04.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:04.457+0000 7fb1e06b9640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d8107580 con 0x7fb1c4003530 2024-09-11T21:49:04.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:04.457+0000 7fb1e06b9640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d8107400 con 0x7fb1d81977e0 2024-09-11T21:49:04.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:04.458+0000 7fb1ddc54640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1d81977e0 0x7fb1d819bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:49:04.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:04.458+0000 7fb1ddc54640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1d81977e0 0x7fb1d819bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:41460/0 (socket says 172.21.15.161:41460) 2024-09-11T21:49:04.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:04.458+0000 7fb1ddc54640 1 -- 172.21.15.161:0/2884137950 learned_addr learned my addr 172.21.15.161:0/2884137950 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-09-11T21:49:04.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:04.458+0000 7fb1de455640 1 --2- 172.21.15.161:0/2884137950 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1c4003530 0x7fb1d81972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:49:04.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:04.458+0000 7fb1ddc54640 1 -- 172.21.15.161:0/2884137950 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1d81977e0 msgr2=0x7fb1d819bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-11T21:49:04.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:04.458+0000 7fb1ddc54640 1 -- 172.21.15.161:0/2884137950 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1d81977e0 msgr2=0x7fb1d819bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-11T21:49:04.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:04.458+0000 7fb1ddc54640 1 --2- 172.21.15.161:0/2884137950 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1d81977e0 0x7fb1d819bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-11T21:49:04.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:04.458+0000 7fb1ddc54640 1 --2- 172.21.15.161:0/2884137950 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1d81977e0 0x7fb1d819bb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T21:49:04.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:04.458+0000 7fb1de455640 1 -- 172.21.15.161:0/2884137950 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1d81977e0 msgr2=0x7fb1d819bb70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T21:49:04.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:04.458+0000 7fb1de455640 1 --2- 172.21.15.161:0/2884137950 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1d81977e0 0x7fb1d819bb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:49:04.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:04.458+0000 7fb1de455640 1 -- 172.21.15.161:0/2884137950 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1d8197e00 con 0x7fb1c4003530 2024-09-11T21:49:04.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:04.458+0000 7fb1de455640 1 --2- 172.21.15.161:0/2884137950 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1c4003530 0x7fb1d81972a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb1c800ec10 tx=0x7fb1c802e8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:49:04.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:04.459+0000 7fb1cf7fe640 1 -- 172.21.15.161:0/2884137950 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fb1c802ee40 con 0x7fb1c4003530 2024-09-11T21:49:04.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:04.459+0000 7fb1cf7fe640 1 -- 172.21.15.161:0/2884137950 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1c80388e0 con 0x7fb1c4003530 2024-09-11T21:49:04.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:04.459+0000 7fb1e06b9640 1 -- 172.21.15.161:0/2884137950 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1d819c0b0 con 0x7fb1c4003530 2024-09-11T21:49:04.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:04.459+0000 7fb1cf7fe640 1 -- 172.21.15.161:0/2884137950 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fb1c803f930 con 0x7fb1c4003530 2024-09-11T21:49:04.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:04.459+0000 7fb1e06b9640 1 -- 172.21.15.161:0/2884137950 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1d819c560 con 0x7fb1c4003530 2024-09-11T21:49:04.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:04.460+0000 7fb1cf7fe640 1 -- 172.21.15.161:0/2884137950 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7fb1c8047060 con 0x7fb1c4003530 2024-09-11T21:49:04.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:04.460+0000 7fb1e06b9640 1 -- 172.21.15.161:0/2884137950 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1d8101420 con 0x7fb1c4003530 2024-09-11T21:49:04.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:04.460+0000 7fb1cf7fe640 1 --2- 172.21.15.161:0/2884137950 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fb1bc038e00 0x7fb1bc03b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:49:04.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:04.460+0000 7fb1cf7fe640 1 -- 172.21.15.161:0/2884137950 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb1c806db40 con 0x7fb1c4003530 2024-09-11T21:49:04.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:04.460+0000 7fb1ddc54640 1 --2- 172.21.15.161:0/2884137950 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fb1bc038e00 0x7fb1bc03b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:49:04.463 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:04.461+0000 7fb1ddc54640 1 --2- 172.21.15.161:0/2884137950 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fb1bc038e00 0x7fb1bc03b2c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb1d4009100 tx=0x7fb1d4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:49:04.467 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:04.467+0000 7fb1cf7fe640 1 -- 172.21.15.161:0/2884137950 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb1c8034030 con 0x7fb1c4003530 2024-09-11T21:49:04.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:04 smithi001 ceph-mon[30935]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:49:04.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:04 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T21:49:04.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:04 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T21:49:04.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:04 smithi001 ceph-mon[30935]: mon.a calling monitor election 2024-09-11T21:49:04.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:04 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T21:49:04.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:04 smithi001 ceph-mon[30935]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:49:04.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:04 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T21:49:04.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:04 smithi001 ceph-mon[30935]: mon.c calling monitor election 2024-09-11T21:49:04.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:04 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T21:49:04.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:04 smithi001 ceph-mon[30935]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:49:04.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:04 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T21:49:04.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:04 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T21:49:04.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:04 smithi001 ceph-mon[30935]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-11T21:49:04.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:04 smithi001 ceph-mon[30935]: monmap epoch 2 2024-09-11T21:49:04.695 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:04 smithi001 ceph-mon[30935]: fsid 3ffca83c-7087-11ef-bcea-c7b262605968 2024-09-11T21:49:04.695 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:04 smithi001 ceph-mon[30935]: last_changed 2024-09-11T21:48:59.209131+0000 2024-09-11T21:49:04.695 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:04 smithi001 ceph-mon[30935]: created 2024-09-11T21:47:25.953679+0000 2024-09-11T21:49:04.695 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:04 smithi001 ceph-mon[30935]: min_mon_release 19 (squid) 2024-09-11T21:49:04.695 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:04 smithi001 ceph-mon[30935]: election_strategy: 1 2024-09-11T21:49:04.695 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:04 smithi001 ceph-mon[30935]: 0: [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] mon.a 2024-09-11T21:49:04.695 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:04 smithi001 ceph-mon[30935]: 1: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.c 2024-09-11T21:49:04.695 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:04 smithi001 ceph-mon[30935]: fsmap 2024-09-11T21:49:04.695 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:04 smithi001 ceph-mon[30935]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T21:49:04.695 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:04 smithi001 ceph-mon[30935]: mgrmap e13: x(active, since 39s) 2024-09-11T21:49:04.695 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:04 smithi001 ceph-mon[30935]: overall HEALTH_OK 2024-09-11T21:49:04.695 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:04 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:04.695 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:04 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:04.695 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:04 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:04.696 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:04 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T21:49:04.696 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:04 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:49:04.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:04.701+0000 7fb1e06b9640 1 -- 172.21.15.161:0/2884137950 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fb1d818ed40 con 0x7fb1c4003530 2024-09-11T21:49:04.702 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:04.702+0000 7fb1cf7fe640 1 -- 172.21.15.161:0/2884137950 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1011 (secure 0 0 0) 0x7fb1c8037e90 con 0x7fb1c4003530 2024-09-11T21:49:04.703 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-11T21:49:04.703 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":2,"fsid":"3ffca83c-7087-11ef-bcea-c7b262605968","modified":"2024-09-11T21:48:59.209131Z","created":"2024-09-11T21:47:25.953679Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:3300","nonce":0},{"type":"v1","addr":"172.21.15.1:6789","nonce":0}]},"addr":"172.21.15.1:6789/0","public_addr":"172.21.15.1:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:3300","nonce":0},{"type":"v1","addr":"172.21.15.161:6789","nonce":0}]},"addr":"172.21.15.161:6789/0","public_addr":"172.21.15.161:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-11T21:49:04.703 INFO:teuthology.orchestra.run.smithi161.stderr:dumped monmap epoch 2 2024-09-11T21:49:04.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:04.704+0000 7fb1e06b9640 1 -- 172.21.15.161:0/2884137950 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fb1bc038e00 msgr2=0x7fb1bc03b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:49:04.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:04.704+0000 7fb1e06b9640 1 --2- 172.21.15.161:0/2884137950 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fb1bc038e00 0x7fb1bc03b2c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb1d4009100 tx=0x7fb1d4008040 comp rx=0 tx=0).stop 2024-09-11T21:49:04.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:04.704+0000 7fb1e06b9640 1 -- 172.21.15.161:0/2884137950 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1c4003530 msgr2=0x7fb1d81972a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:49:04.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:04.704+0000 7fb1e06b9640 1 --2- 172.21.15.161:0/2884137950 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1c4003530 0x7fb1d81972a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb1c800ec10 tx=0x7fb1c802e8a0 comp rx=0 tx=0).stop 2024-09-11T21:49:04.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:04.704+0000 7fb1e06b9640 1 -- 172.21.15.161:0/2884137950 shutdown_connections 2024-09-11T21:49:04.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:04.704+0000 7fb1e06b9640 1 --2- 172.21.15.161:0/2884137950 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fb1bc038e00 0x7fb1bc03b2c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:49:04.705 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:04.704+0000 7fb1e06b9640 1 --2- 172.21.15.161:0/2884137950 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1d81977e0 0x7fb1d819bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:49:04.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:04.704+0000 7fb1e06b9640 1 --2- 172.21.15.161:0/2884137950 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1c4003530 0x7fb1d81972a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:49:04.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:04.704+0000 7fb1e06b9640 1 -- 172.21.15.161:0/2884137950 >> 172.21.15.161:0/2884137950 conn(0x7fb1d80fb9a0 msgr2=0x7fb1d80fcb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:49:04.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:04.704+0000 7fb1e06b9640 1 -- 172.21.15.161:0/2884137950 shutdown_connections 2024-09-11T21:49:04.706 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:04.704+0000 7fb1e06b9640 1 -- 172.21.15.161:0/2884137950 wait complete. 2024-09-11T21:49:05.367 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:05 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T21:49:05.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:05 smithi001 ceph-mon[30935]: Deploying daemon mon.b on smithi053 2024-09-11T21:49:05.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:05 smithi001 ceph-mon[30935]: from='client.? 172.21.15.161:0/2884137950' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T21:49:05.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:05 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T21:49:06.397 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T21:49:06.397 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T21:49:06.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:06 smithi161 ceph-mon[35615]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:49:06.651 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T21:49:06.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:06 smithi001 ceph-mon[30935]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:49:06.693 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:49:06 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:49:06.215+0000 7f764578f640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-11T21:49:07.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:07.084+0000 7f153cd8e640 1 -- 172.21.15.161:0/965648685 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1538107670 msgr2=0x7f1538107a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:49:07.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:07.084+0000 7f153cd8e640 1 --2- 172.21.15.161:0/965648685 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1538107670 0x7f1538107a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1528009880 tx=0x7f152802a5d0 comp rx=0 tx=0).stop 2024-09-11T21:49:07.086 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:07.084+0000 7f153cd8e640 1 -- 172.21.15.161:0/965648685 shutdown_connections 2024-09-11T21:49:07.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:07.084+0000 7f153cd8e640 1 --2- 172.21.15.161:0/965648685 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1538107670 0x7f1538107a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:49:07.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:07.084+0000 7f153cd8e640 1 -- 172.21.15.161:0/965648685 >> 172.21.15.161:0/965648685 conn(0x7f15380fd330 msgr2=0x7f15380ff750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:49:07.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:07.084+0000 7f153cd8e640 1 -- 172.21.15.161:0/965648685 shutdown_connections 2024-09-11T21:49:07.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:07.084+0000 7f153cd8e640 1 -- 172.21.15.161:0/965648685 wait complete. 2024-09-11T21:49:07.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:07.085+0000 7f153cd8e640 1 Processor -- start 2024-09-11T21:49:07.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:07.085+0000 7f153cd8e640 1 -- start start 2024-09-11T21:49:07.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:07.085+0000 7f153cd8e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1538107670 0x7f1538197460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:49:07.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:07.085+0000 7f153cd8e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15381979a0 0x7f153819bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:49:07.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:07.085+0000 7f153cd8e640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f153810bd30 con 0x7f15381979a0 2024-09-11T21:49:07.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:07.085+0000 7f153cd8e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f153810bbb0 con 0x7f1538107670 2024-09-11T21:49:07.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:07.086+0000 7f153659c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1538107670 0x7f1538197460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:49:07.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:07.086+0000 7f153659c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1538107670 0x7f1538197460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:41462/0 (socket says 172.21.15.161:41462) 2024-09-11T21:49:07.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:07.086+0000 7f153659c640 1 -- 172.21.15.161:0/527045112 learned_addr learned my addr 172.21.15.161:0/527045112 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-09-11T21:49:07.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:07.086+0000 7f153659c640 1 -- 172.21.15.161:0/527045112 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1538107670 msgr2=0x7f1538197460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-11T21:49:07.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:07.086+0000 7f153659c640 1 -- 172.21.15.161:0/527045112 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1538107670 msgr2=0x7f1538197460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-11T21:49:07.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:07.086+0000 7f153659c640 1 --2- 172.21.15.161:0/527045112 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1538107670 0x7f1538197460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-11T21:49:07.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:07.086+0000 7f153659c640 1 --2- 172.21.15.161:0/527045112 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1538107670 0x7f1538197460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T21:49:07.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:07.086+0000 7f1535d9b640 1 --2- 172.21.15.161:0/527045112 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15381979a0 0x7f153819bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:49:07.089 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:07.086+0000 7f1535d9b640 1 -- 172.21.15.161:0/527045112 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1538107670 msgr2=0x7f1538197460 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T21:49:07.089 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:07.086+0000 7f1535d9b640 1 --2- 172.21.15.161:0/527045112 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1538107670 0x7f1538197460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:49:07.090 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:07.086+0000 7f1535d9b640 1 -- 172.21.15.161:0/527045112 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1538197fc0 con 0x7f15381979a0 2024-09-11T21:49:07.090 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:07.087+0000 7f1535d9b640 1 --2- 172.21.15.161:0/527045112 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15381979a0 0x7f153819bd30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f152c00b3d0 tx=0x7f152c00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:49:07.090 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:07.087+0000 7f15277fe640 1 -- 172.21.15.161:0/527045112 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f152c011630 con 0x7f15381979a0 2024-09-11T21:49:07.090 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:07.087+0000 7f15277fe640 1 -- 172.21.15.161:0/527045112 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f152c011cb0 con 0x7f15381979a0 2024-09-11T21:49:07.090 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:07.087+0000 7f153cd8e640 1 -- 172.21.15.161:0/527045112 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f153819c270 con 0x7f15381979a0 2024-09-11T21:49:07.090 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:07.087+0000 7f15277fe640 1 -- 172.21.15.161:0/527045112 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f152c010df0 con 0x7f15381979a0 2024-09-11T21:49:07.090 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:07.087+0000 7f153cd8e640 1 -- 172.21.15.161:0/527045112 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f153819c7b0 con 0x7f15381979a0 2024-09-11T21:49:07.090 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:07.088+0000 7f153cd8e640 1 -- 172.21.15.161:0/527045112 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1538102db0 con 0x7f15381979a0 2024-09-11T21:49:07.090 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:07.088+0000 7f15277fe640 1 -- 172.21.15.161:0/527045112 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f152c018020 con 0x7f15381979a0 2024-09-11T21:49:07.090 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:07.088+0000 7f15277fe640 1 --2- 172.21.15.161:0/527045112 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f1520038e50 0x7f152003b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:49:07.090 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:07.088+0000 7f15277fe640 1 -- 172.21.15.161:0/527045112 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f152c04bce0 con 0x7f15381979a0 2024-09-11T21:49:07.090 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:07.089+0000 7f153659c640 1 --2- 172.21.15.161:0/527045112 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f1520038e50 0x7f152003b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:49:07.091 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:07.089+0000 7f153659c640 1 --2- 172.21.15.161:0/527045112 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f1520038e50 0x7f152003b310 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f15280096c0 tx=0x7f1528031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:49:07.094 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:07.094+0000 7f15277fe640 1 -- 172.21.15.161:0/527045112 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f152c01f020 con 0x7f15381979a0 2024-09-11T21:49:07.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:07.323+0000 7f153cd8e640 1 -- 172.21.15.161:0/527045112 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f153819cac0 con 0x7f15381979a0 2024-09-11T21:49:07.324 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:07.324+0000 7f15277fe640 1 -- 172.21.15.161:0/527045112 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1011 (secure 0 0 0) 0x7f152c0172c0 con 0x7f15381979a0 2024-09-11T21:49:07.324 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-11T21:49:07.325 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":2,"fsid":"3ffca83c-7087-11ef-bcea-c7b262605968","modified":"2024-09-11T21:48:59.209131Z","created":"2024-09-11T21:47:25.953679Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:3300","nonce":0},{"type":"v1","addr":"172.21.15.1:6789","nonce":0}]},"addr":"172.21.15.1:6789/0","public_addr":"172.21.15.1:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:3300","nonce":0},{"type":"v1","addr":"172.21.15.161:6789","nonce":0}]},"addr":"172.21.15.161:6789/0","public_addr":"172.21.15.161:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-11T21:49:07.325 INFO:teuthology.orchestra.run.smithi161.stderr:dumped monmap epoch 2 2024-09-11T21:49:07.326 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:07.326+0000 7f153cd8e640 1 -- 172.21.15.161:0/527045112 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f1520038e50 msgr2=0x7f152003b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:49:07.326 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:07.326+0000 7f153cd8e640 1 --2- 172.21.15.161:0/527045112 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f1520038e50 0x7f152003b310 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f15280096c0 tx=0x7f1528031400 comp rx=0 tx=0).stop 2024-09-11T21:49:07.327 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:07.326+0000 7f153cd8e640 1 -- 172.21.15.161:0/527045112 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15381979a0 msgr2=0x7f153819bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:49:07.327 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:07.326+0000 7f153cd8e640 1 --2- 172.21.15.161:0/527045112 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15381979a0 0x7f153819bd30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f152c00b3d0 tx=0x7f152c00b890 comp rx=0 tx=0).stop 2024-09-11T21:49:07.327 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:07.326+0000 7f153cd8e640 1 -- 172.21.15.161:0/527045112 shutdown_connections 2024-09-11T21:49:07.327 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:07.326+0000 7f153cd8e640 1 --2- 172.21.15.161:0/527045112 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f1520038e50 0x7f152003b310 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:49:07.327 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:07.326+0000 7f153cd8e640 1 --2- 172.21.15.161:0/527045112 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15381979a0 0x7f153819bd30 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:49:07.327 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:07.326+0000 7f153cd8e640 1 --2- 172.21.15.161:0/527045112 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1538107670 0x7f1538197460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:49:07.327 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:07.326+0000 7f153cd8e640 1 -- 172.21.15.161:0/527045112 >> 172.21.15.161:0/527045112 conn(0x7f15380fd330 msgr2=0x7f15380fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:49:07.327 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:07.326+0000 7f153cd8e640 1 -- 172.21.15.161:0/527045112 shutdown_connections 2024-09-11T21:49:07.327 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:07.326+0000 7f153cd8e640 1 -- 172.21.15.161:0/527045112 wait complete. 2024-09-11T21:49:08.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:07 smithi053 systemd[1]: Started Ceph mon.b for 3ffca83c-7087-11ef-bcea-c7b262605968. 2024-09-11T21:49:08.893 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T21:49:08.893 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T21:49:09.144 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T21:49:12.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:12.773+0000 7fc310090640 1 -- 172.21.15.161:0/77568743 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc2f4003530 msgr2=0x7fc2f40059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:49:12.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:12.773+0000 7fc310090640 1 --2- 172.21.15.161:0/77568743 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc2f4003530 0x7fc2f40059c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc308102b00 tx=0x7fc30402c150 comp rx=0 tx=0).stop 2024-09-11T21:49:12.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:12.773+0000 7fc310090640 1 -- 172.21.15.161:0/77568743 shutdown_connections 2024-09-11T21:49:12.776 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:12.773+0000 7fc310090640 1 --2- 172.21.15.161:0/77568743 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc2f4003530 0x7fc2f40059c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:49:12.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:12.773+0000 7fc310090640 1 -- 172.21.15.161:0/77568743 >> 172.21.15.161:0/77568743 conn(0x7fc3080fd350 msgr2=0x7fc3080ff770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:49:12.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:12.773+0000 7fc310090640 1 -- 172.21.15.161:0/77568743 shutdown_connections 2024-09-11T21:49:12.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:12.773+0000 7fc310090640 1 -- 172.21.15.161:0/77568743 wait complete. 2024-09-11T21:49:12.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:12.774+0000 7fc310090640 1 Processor -- start 2024-09-11T21:49:12.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:12.774+0000 7fc310090640 1 -- start start 2024-09-11T21:49:12.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:12.774+0000 7fc310090640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2f4003530 0x7fc308197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:49:12.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:12.774+0000 7fc310090640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc308197850 0x7fc30819bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:49:12.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:12.774+0000 7fc310090640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc30819c120 0x7fc30819c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:49:12.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:12.774+0000 7fc310090640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc308109790 con 0x7fc308197850 2024-09-11T21:49:12.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:12.774+0000 7fc310090640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc308109610 con 0x7fc2f4003530 2024-09-11T21:49:12.777 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:12.774+0000 7fc310090640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc308109910 con 0x7fc30819c120 2024-09-11T21:49:12.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:12.775+0000 7fc30e62d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc30819c120 0x7fc30819c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:49:12.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:12.775+0000 7fc30de2c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2f4003530 0x7fc308197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:49:12.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:12.775+0000 7fc30e62d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc30819c120 0x7fc30819c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:47738/0 (socket says 172.21.15.161:47738) 2024-09-11T21:49:12.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:12.775+0000 7fc30e62d640 1 -- 172.21.15.161:0/1101974365 learned_addr learned my addr 172.21.15.161:0/1101974365 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-09-11T21:49:12.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:12.775+0000 7fc30de2c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2f4003530 0x7fc308197310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.161:35676/0 (socket says 172.21.15.161:35676) 2024-09-11T21:49:12.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:12.775+0000 7fc30d62b640 1 --2- 172.21.15.161:0/1101974365 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc308197850 0x7fc30819bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:49:12.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:12.775+0000 7fc30de2c640 1 -- 172.21.15.161:0/1101974365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2f4003530 msgr2=0x7fc308197310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-11T21:49:12.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:12.775+0000 7fc30de2c640 1 -- 172.21.15.161:0/1101974365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2f4003530 msgr2=0x7fc308197310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-11T21:49:12.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:12.775+0000 7fc30de2c640 1 --2- 172.21.15.161:0/1101974365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2f4003530 0x7fc308197310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-11T21:49:12.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:12.775+0000 7fc30e62d640 1 -- 172.21.15.161:0/1101974365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc30819c120 msgr2=0x7fc30819c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-09-11T21:49:12.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:12.775+0000 7fc30de2c640 1 --2- 172.21.15.161:0/1101974365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2f4003530 0x7fc308197310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T21:49:12.778 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:12.775+0000 7fc30e62d640 1 -- 172.21.15.161:0/1101974365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc30819c120 msgr2=0x7fc30819c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-11T21:49:12.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:12.775+0000 7fc30e62d640 1 --2- 172.21.15.161:0/1101974365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc30819c120 0x7fc30819c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-11T21:49:12.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:12.775+0000 7fc30e62d640 1 --2- 172.21.15.161:0/1101974365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc30819c120 0x7fc30819c5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T21:49:12.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:12.775+0000 7fc30d62b640 1 -- 172.21.15.161:0/1101974365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc30819c120 msgr2=0x7fc30819c5d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T21:49:12.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:12.775+0000 7fc30d62b640 1 --2- 172.21.15.161:0/1101974365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc30819c120 0x7fc30819c5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:49:12.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:12.775+0000 7fc30d62b640 1 -- 172.21.15.161:0/1101974365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2f4003530 msgr2=0x7fc308197310 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T21:49:12.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:12.775+0000 7fc30d62b640 1 --2- 172.21.15.161:0/1101974365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2f4003530 0x7fc308197310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:49:12.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:12.775+0000 7fc30d62b640 1 -- 172.21.15.161:0/1101974365 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3081a3e90 con 0x7fc308197850 2024-09-11T21:49:12.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:12.775+0000 7fc30d62b640 1 --2- 172.21.15.161:0/1101974365 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc308197850 0x7fc30819bbe0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc2f800b450 tx=0x7fc2f800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:49:12.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:12.775+0000 7fc2feffd640 1 -- 172.21.15.161:0/1101974365 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f8004230 con 0x7fc308197850 2024-09-11T21:49:12.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:12.776+0000 7fc2feffd640 1 -- 172.21.15.161:0/1101974365 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2f80043d0 con 0x7fc308197850 2024-09-11T21:49:12.779 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:12.776+0000 7fc310090640 1 -- 172.21.15.161:0/1101974365 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3081a40c0 con 0x7fc308197850 2024-09-11T21:49:12.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:12.776+0000 7fc2feffd640 1 -- 172.21.15.161:0/1101974365 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f8003cb0 con 0x7fc308197850 2024-09-11T21:49:12.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:12.776+0000 7fc310090640 1 -- 172.21.15.161:0/1101974365 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3081a4620 con 0x7fc308197850 2024-09-11T21:49:12.780 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:12.776+0000 7fc2feffd640 1 -- 172.21.15.161:0/1101974365 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7fc2f800f050 con 0x7fc308197850 2024-09-11T21:49:12.781 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:12.777+0000 7fc2feffd640 1 --2- 172.21.15.161:0/1101974365 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fc2ec0394f0 0x7fc2ec03b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:49:12.781 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:12.777+0000 7fc2feffd640 1 -- 172.21.15.161:0/1101974365 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc2f804ce60 con 0x7fc308197850 2024-09-11T21:49:12.781 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:12.777+0000 7fc30de2c640 1 --2- 172.21.15.161:0/1101974365 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fc2ec0394f0 0x7fc2ec03b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:49:12.781 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:12.777+0000 7fc310090640 1 -- 172.21.15.161:0/1101974365 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc30810a8c0 con 0x7fc308197850 2024-09-11T21:49:12.781 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:12.777+0000 7fc30de2c640 1 --2- 172.21.15.161:0/1101974365 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fc2ec0394f0 0x7fc2ec03b9b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc304005dc0 tx=0x7fc304005d10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:49:12.783 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:12.783+0000 7fc2feffd640 1 -- 172.21.15.161:0/1101974365 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc2f800d040 con 0x7fc308197850 2024-09-11T21:49:13.024 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:13.024+0000 7fc310090640 1 -- 172.21.15.161:0/1101974365 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fc30802d7c0 con 0x7fc308197850 2024-09-11T21:49:13.025 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:12 smithi161 ceph-mon[35615]: mon.c calling monitor election 2024-09-11T21:49:13.025 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:12 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T21:49:13.025 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:12 smithi161 ceph-mon[35615]: mon.a calling monitor election 2024-09-11T21:49:13.025 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:12 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T21:49:13.025 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:12 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T21:49:13.026 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:12 smithi161 ceph-mon[35615]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:49:13.026 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:12 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T21:49:13.026 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:12 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T21:49:13.026 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:12 smithi161 ceph-mon[35615]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:49:13.026 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:12 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T21:49:13.026 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:12 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T21:49:13.026 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:12 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T21:49:13.026 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:12 smithi161 ceph-mon[35615]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-11T21:49:13.026 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:12 smithi161 ceph-mon[35615]: monmap epoch 3 2024-09-11T21:49:13.026 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:12 smithi161 ceph-mon[35615]: fsid 3ffca83c-7087-11ef-bcea-c7b262605968 2024-09-11T21:49:13.026 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:12 smithi161 ceph-mon[35615]: last_changed 2024-09-11T21:49:07.698583+0000 2024-09-11T21:49:13.026 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:12 smithi161 ceph-mon[35615]: created 2024-09-11T21:47:25.953679+0000 2024-09-11T21:49:13.026 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:12 smithi161 ceph-mon[35615]: min_mon_release 19 (squid) 2024-09-11T21:49:13.027 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:12 smithi161 ceph-mon[35615]: election_strategy: 1 2024-09-11T21:49:13.027 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:12 smithi161 ceph-mon[35615]: 0: [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] mon.a 2024-09-11T21:49:13.027 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:12 smithi161 ceph-mon[35615]: 1: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.c 2024-09-11T21:49:13.027 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:12 smithi161 ceph-mon[35615]: 2: [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] mon.b 2024-09-11T21:49:13.027 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:12 smithi161 ceph-mon[35615]: fsmap 2024-09-11T21:49:13.027 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:12 smithi161 ceph-mon[35615]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T21:49:13.027 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:12 smithi161 ceph-mon[35615]: mgrmap e13: x(active, since 48s) 2024-09-11T21:49:13.027 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:12 smithi161 ceph-mon[35615]: overall HEALTH_OK 2024-09-11T21:49:13.027 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:12 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:13.027 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:12 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:13.027 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:12 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:13.027 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:12 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:13.028 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:13.024+0000 7fc2feffd640 1 -- 172.21.15.161:0/1101974365 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1275 (secure 0 0 0) 0x7fc2f8011d00 con 0x7fc308197850 2024-09-11T21:49:13.028 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-11T21:49:13.028 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":3,"fsid":"3ffca83c-7087-11ef-bcea-c7b262605968","modified":"2024-09-11T21:49:07.698583Z","created":"2024-09-11T21:47:25.953679Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:3300","nonce":0},{"type":"v1","addr":"172.21.15.1:6789","nonce":0}]},"addr":"172.21.15.1:6789/0","public_addr":"172.21.15.1:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:3300","nonce":0},{"type":"v1","addr":"172.21.15.161:6789","nonce":0}]},"addr":"172.21.15.161:6789/0","public_addr":"172.21.15.161:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:3300","nonce":0},{"type":"v1","addr":"172.21.15.53:6789","nonce":0}]},"addr":"172.21.15.53:6789/0","public_addr":"172.21.15.53:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-11T21:49:13.029 INFO:teuthology.orchestra.run.smithi161.stderr:dumped monmap epoch 3 2024-09-11T21:49:13.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:13.028+0000 7fc310090640 1 -- 172.21.15.161:0/1101974365 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fc2ec0394f0 msgr2=0x7fc2ec03b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:49:13.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:13.028+0000 7fc310090640 1 --2- 172.21.15.161:0/1101974365 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fc2ec0394f0 0x7fc2ec03b9b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc304005dc0 tx=0x7fc304005d10 comp rx=0 tx=0).stop 2024-09-11T21:49:13.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:13.028+0000 7fc310090640 1 -- 172.21.15.161:0/1101974365 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc308197850 msgr2=0x7fc30819bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:49:13.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:13.028+0000 7fc310090640 1 --2- 172.21.15.161:0/1101974365 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc308197850 0x7fc30819bbe0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc2f800b450 tx=0x7fc2f800b910 comp rx=0 tx=0).stop 2024-09-11T21:49:13.029 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:13.028+0000 7fc310090640 1 -- 172.21.15.161:0/1101974365 shutdown_connections 2024-09-11T21:49:13.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:13.028+0000 7fc310090640 1 --2- 172.21.15.161:0/1101974365 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fc2ec0394f0 0x7fc2ec03b9b0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:49:13.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:13.028+0000 7fc310090640 1 --2- 172.21.15.161:0/1101974365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc30819c120 0x7fc30819c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:49:13.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:13.028+0000 7fc310090640 1 --2- 172.21.15.161:0/1101974365 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc308197850 0x7fc30819bbe0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:49:13.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:13.028+0000 7fc310090640 1 --2- 172.21.15.161:0/1101974365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc2f4003530 0x7fc308197310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:49:13.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:13.028+0000 7fc310090640 1 -- 172.21.15.161:0/1101974365 >> 172.21.15.161:0/1101974365 conn(0x7fc3080fd350 msgr2=0x7fc3080ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:49:13.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:13.028+0000 7fc310090640 1 -- 172.21.15.161:0/1101974365 shutdown_connections 2024-09-11T21:49:13.030 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:13.028+0000 7fc310090640 1 -- 172.21.15.161:0/1101974365 wait complete. 2024-09-11T21:49:13.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:12 smithi001 ceph-mon[30935]: mon.c calling monitor election 2024-09-11T21:49:13.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:12 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T21:49:13.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:12 smithi001 ceph-mon[30935]: mon.a calling monitor election 2024-09-11T21:49:13.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:12 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T21:49:13.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:12 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T21:49:13.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:12 smithi001 ceph-mon[30935]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:49:13.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:12 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T21:49:13.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:12 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T21:49:13.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:12 smithi001 ceph-mon[30935]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:49:13.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:12 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T21:49:13.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:12 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T21:49:13.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:12 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T21:49:13.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:12 smithi001 ceph-mon[30935]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-11T21:49:13.195 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:12 smithi001 ceph-mon[30935]: monmap epoch 3 2024-09-11T21:49:13.195 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:12 smithi001 ceph-mon[30935]: fsid 3ffca83c-7087-11ef-bcea-c7b262605968 2024-09-11T21:49:13.195 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:12 smithi001 ceph-mon[30935]: last_changed 2024-09-11T21:49:07.698583+0000 2024-09-11T21:49:13.195 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:12 smithi001 ceph-mon[30935]: created 2024-09-11T21:47:25.953679+0000 2024-09-11T21:49:13.195 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:12 smithi001 ceph-mon[30935]: min_mon_release 19 (squid) 2024-09-11T21:49:13.195 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:12 smithi001 ceph-mon[30935]: election_strategy: 1 2024-09-11T21:49:13.195 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:12 smithi001 ceph-mon[30935]: 0: [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] mon.a 2024-09-11T21:49:13.195 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:12 smithi001 ceph-mon[30935]: 1: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.c 2024-09-11T21:49:13.195 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:12 smithi001 ceph-mon[30935]: 2: [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] mon.b 2024-09-11T21:49:13.195 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:12 smithi001 ceph-mon[30935]: fsmap 2024-09-11T21:49:13.195 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:12 smithi001 ceph-mon[30935]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T21:49:13.195 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:12 smithi001 ceph-mon[30935]: mgrmap e13: x(active, since 48s) 2024-09-11T21:49:13.195 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:12 smithi001 ceph-mon[30935]: overall HEALTH_OK 2024-09-11T21:49:13.195 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:12 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:13.196 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:12 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:13.196 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:12 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:13.196 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:12 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:13.604 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-11T21:49:13.605 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph config generate-minimal-conf 2024-09-11T21:49:13.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:13 smithi161 ceph-mon[35615]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:49:13.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:13 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:49:13.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:13 smithi161 ceph-mon[35615]: from='client.? 172.21.15.161:0/1101974365' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T21:49:13.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:13 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T21:49:13.872 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:49:13.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:13 smithi001 ceph-mon[30935]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:49:13.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:13 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:49:13.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:13 smithi001 ceph-mon[30935]: from='client.? 172.21.15.161:0/1101974365' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T21:49:13.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:13 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T21:49:14.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:14.293+0000 7f9379d60640 1 -- 172.21.15.1:0/3045220198 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9374102820 msgr2=0x7f9374102c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:49:14.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:14.293+0000 7f9379d60640 1 --2- 172.21.15.1:0/3045220198 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9374102820 0x7f9374102c00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9364009880 tx=0x7f936402a670 comp rx=0 tx=0).stop 2024-09-11T21:49:14.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:14.293+0000 7f9379d60640 1 -- 172.21.15.1:0/3045220198 shutdown_connections 2024-09-11T21:49:14.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:14.294+0000 7f9379d60640 1 --2- 172.21.15.1:0/3045220198 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9374102820 0x7f9374102c00 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:49:14.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:14.294+0000 7f9379d60640 1 -- 172.21.15.1:0/3045220198 >> 172.21.15.1:0/3045220198 conn(0x7f93740fc220 msgr2=0x7f93740fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:49:14.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:14.294+0000 7f9379d60640 1 -- 172.21.15.1:0/3045220198 shutdown_connections 2024-09-11T21:49:14.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:14.294+0000 7f9379d60640 1 -- 172.21.15.1:0/3045220198 wait complete. 2024-09-11T21:49:14.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:14.294+0000 7f9379d60640 1 Processor -- start 2024-09-11T21:49:14.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:14.294+0000 7f9379d60640 1 -- start start 2024-09-11T21:49:14.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:14.295+0000 7f9379d60640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9374102820 0x7f9374197510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:49:14.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:14.295+0000 7f9379d60640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9374197a50 0x7f937419bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:49:14.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:14.295+0000 7f9379d60640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f937419c320 0x7f937419c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:49:14.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:14.295+0000 7f9379d60640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f937410bd70 con 0x7f9374197a50 2024-09-11T21:49:14.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:14.295+0000 7f9379d60640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f937410bbf0 con 0x7f937419c320 2024-09-11T21:49:14.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:14.295+0000 7f9379d60640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f937410bef0 con 0x7f9374102820 2024-09-11T21:49:14.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:14.295+0000 7f9372ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9374197a50 0x7f937419bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:49:14.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:14.295+0000 7f9372ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9374197a50 0x7f937419bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60130/0 (socket says 172.21.15.1:60130) 2024-09-11T21:49:14.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:14.295+0000 7f9372ffd640 1 -- 172.21.15.1:0/2858259826 learned_addr learned my addr 172.21.15.1:0/2858259826 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:49:14.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:14.295+0000 7f93737fe640 1 --2- 172.21.15.1:0/2858259826 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9374102820 0x7f9374197510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:49:14.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:14.295+0000 7f9373fff640 1 --2- 172.21.15.1:0/2858259826 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f937419c320 0x7f937419c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:49:14.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:14.295+0000 7f9372ffd640 1 -- 172.21.15.1:0/2858259826 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9374102820 msgr2=0x7f9374197510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:49:14.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:14.295+0000 7f9372ffd640 1 --2- 172.21.15.1:0/2858259826 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9374102820 0x7f9374197510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:49:14.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:14.295+0000 7f9372ffd640 1 -- 172.21.15.1:0/2858259826 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f937419c320 msgr2=0x7f937419c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:49:14.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:14.295+0000 7f9372ffd640 1 --2- 172.21.15.1:0/2858259826 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f937419c320 0x7f937419c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:49:14.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:14.295+0000 7f9372ffd640 1 -- 172.21.15.1:0/2858259826 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93741a4090 con 0x7f9374197a50 2024-09-11T21:49:14.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:14.295+0000 7f9373fff640 1 --2- 172.21.15.1:0/2858259826 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f937419c320 0x7f937419c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T21:49:14.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:14.295+0000 7f93737fe640 1 --2- 172.21.15.1:0/2858259826 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9374102820 0x7f9374197510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T21:49:14.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:14.295+0000 7f9372ffd640 1 --2- 172.21.15.1:0/2858259826 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9374197a50 0x7f937419bde0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f936800b520 tx=0x7f936800b9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:49:14.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:14.295+0000 7f9370ff9640 1 -- 172.21.15.1:0/2858259826 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9368004230 con 0x7f9374197a50 2024-09-11T21:49:14.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:14.296+0000 7f9379d60640 1 -- 172.21.15.1:0/2858259826 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93741a42c0 con 0x7f9374197a50 2024-09-11T21:49:14.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:14.296+0000 7f9370ff9640 1 -- 172.21.15.1:0/2858259826 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93680043d0 con 0x7f9374197a50 2024-09-11T21:49:14.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:14.296+0000 7f9370ff9640 1 -- 172.21.15.1:0/2858259826 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9368003d60 con 0x7f9374197a50 2024-09-11T21:49:14.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:14.296+0000 7f9379d60640 1 -- 172.21.15.1:0/2858259826 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93741a47d0 con 0x7f9374197a50 2024-09-11T21:49:14.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:14.297+0000 7f9370ff9640 1 -- 172.21.15.1:0/2858259826 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f936800e070 con 0x7f9374197a50 2024-09-11T21:49:14.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:14.297+0000 7f9379d60640 1 -- 172.21.15.1:0/2858259826 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9374102c80 con 0x7f9374197a50 2024-09-11T21:49:14.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:14.297+0000 7f9370ff9640 1 --2- 172.21.15.1:0/2858259826 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f935c0394f0 0x7f935c03b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:49:14.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:14.297+0000 7f9370ff9640 1 -- 172.21.15.1:0/2858259826 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f936804baa0 con 0x7f9374197a50 2024-09-11T21:49:14.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:14.297+0000 7f93737fe640 1 --2- 172.21.15.1:0/2858259826 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f935c0394f0 0x7f935c03b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:49:14.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:14.297+0000 7f93737fe640 1 --2- 172.21.15.1:0/2858259826 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f935c0394f0 0x7f935c03b9b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9364005ab0 tx=0x7f9364005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:49:14.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:14.302+0000 7f9370ff9640 1 -- 172.21.15.1:0/2858259826 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9368014020 con 0x7f9374197a50 2024-09-11T21:49:14.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:14.484+0000 7f9379d60640 1 -- 172.21.15.1:0/2858259826 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f93741a4c20 con 0x7f9374197a50 2024-09-11T21:49:14.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:14.485+0000 7f9370ff9640 1 -- 172.21.15.1:0/2858259826 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+271 (secure 0 0 0) 0x7f9368015260 con 0x7f9374197a50 2024-09-11T21:49:14.486 INFO:teuthology.orchestra.run.smithi001.stdout:# minimal ceph.conf for 3ffca83c-7087-11ef-bcea-c7b262605968 2024-09-11T21:49:14.486 INFO:teuthology.orchestra.run.smithi001.stdout:[global] 2024-09-11T21:49:14.486 INFO:teuthology.orchestra.run.smithi001.stdout: fsid = 3ffca83c-7087-11ef-bcea-c7b262605968 2024-09-11T21:49:14.486 INFO:teuthology.orchestra.run.smithi001.stdout: mon_host = [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] 2024-09-11T21:49:14.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:14.487+0000 7f9379d60640 1 -- 172.21.15.1:0/2858259826 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f935c0394f0 msgr2=0x7f935c03b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:49:14.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:14.487+0000 7f9379d60640 1 --2- 172.21.15.1:0/2858259826 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f935c0394f0 0x7f935c03b9b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9364005ab0 tx=0x7f9364005a20 comp rx=0 tx=0).stop 2024-09-11T21:49:14.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:14.487+0000 7f9379d60640 1 -- 172.21.15.1:0/2858259826 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9374197a50 msgr2=0x7f937419bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:49:14.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:14.487+0000 7f9379d60640 1 --2- 172.21.15.1:0/2858259826 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9374197a50 0x7f937419bde0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f936800b520 tx=0x7f936800b9e0 comp rx=0 tx=0).stop 2024-09-11T21:49:14.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:14.487+0000 7f9379d60640 1 -- 172.21.15.1:0/2858259826 shutdown_connections 2024-09-11T21:49:14.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:14.487+0000 7f9379d60640 1 --2- 172.21.15.1:0/2858259826 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f935c0394f0 0x7f935c03b9b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:49:14.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:14.487+0000 7f9379d60640 1 --2- 172.21.15.1:0/2858259826 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f937419c320 0x7f937419c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:49:14.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:14.487+0000 7f9379d60640 1 --2- 172.21.15.1:0/2858259826 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9374197a50 0x7f937419bde0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:49:14.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:14.487+0000 7f9379d60640 1 --2- 172.21.15.1:0/2858259826 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9374102820 0x7f9374197510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:49:14.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:14.487+0000 7f9379d60640 1 -- 172.21.15.1:0/2858259826 >> 172.21.15.1:0/2858259826 conn(0x7f93740fc220 msgr2=0x7f93741055e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:49:14.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:14.487+0000 7f9379d60640 1 -- 172.21.15.1:0/2858259826 shutdown_connections 2024-09-11T21:49:14.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:14.487+0000 7f9379d60640 1 -- 172.21.15.1:0/2858259826 wait complete. 2024-09-11T21:49:15.048 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:14 smithi053 ceph-mon[34908]: mon.c calling monitor election 2024-09-11T21:49:15.049 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:14 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T21:49:15.049 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:14 smithi053 ceph-mon[34908]: mon.a calling monitor election 2024-09-11T21:49:15.049 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:14 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T21:49:15.049 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:14 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T21:49:15.049 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:14 smithi053 ceph-mon[34908]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:49:15.049 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:14 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T21:49:15.049 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:14 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T21:49:15.049 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:14 smithi053 ceph-mon[34908]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:49:15.049 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:14 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T21:49:15.049 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:14 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T21:49:15.049 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:14 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T21:49:15.050 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:14 smithi053 ceph-mon[34908]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-11T21:49:15.050 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:14 smithi053 ceph-mon[34908]: monmap epoch 3 2024-09-11T21:49:15.050 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:14 smithi053 ceph-mon[34908]: fsid 3ffca83c-7087-11ef-bcea-c7b262605968 2024-09-11T21:49:15.050 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:14 smithi053 ceph-mon[34908]: last_changed 2024-09-11T21:49:07.698583+0000 2024-09-11T21:49:15.050 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:14 smithi053 ceph-mon[34908]: created 2024-09-11T21:47:25.953679+0000 2024-09-11T21:49:15.050 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:14 smithi053 ceph-mon[34908]: min_mon_release 19 (squid) 2024-09-11T21:49:15.050 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:14 smithi053 ceph-mon[34908]: election_strategy: 1 2024-09-11T21:49:15.050 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:14 smithi053 ceph-mon[34908]: 0: [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] mon.a 2024-09-11T21:49:15.050 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:14 smithi053 ceph-mon[34908]: 1: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.c 2024-09-11T21:49:15.050 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:14 smithi053 ceph-mon[34908]: 2: [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] mon.b 2024-09-11T21:49:15.050 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:14 smithi053 ceph-mon[34908]: fsmap 2024-09-11T21:49:15.050 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:14 smithi053 ceph-mon[34908]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T21:49:15.050 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:14 smithi053 ceph-mon[34908]: mgrmap e13: x(active, since 48s) 2024-09-11T21:49:15.051 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:14 smithi053 ceph-mon[34908]: overall HEALTH_OK 2024-09-11T21:49:15.051 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:14 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:15.051 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:14 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:15.051 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:14 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:15.051 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:14 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:15.051 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:14 smithi053 ceph-mon[34908]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:49:15.051 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:14 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:49:15.051 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:14 smithi053 ceph-mon[34908]: from='client.? 172.21.15.161:0/1101974365' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T21:49:15.051 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:14 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T21:49:15.240 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-11T21:49:15.240 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-11T21:49:15.240 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/etc/ceph/ceph.conf 2024-09-11T21:49:15.270 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-11T21:49:15.271 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T21:49:15.337 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-11T21:49:15.338 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/ceph/ceph.conf 2024-09-11T21:49:15.365 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-11T21:49:15.365 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T21:49:15.431 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-09-11T21:49:15.431 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/ceph/ceph.conf 2024-09-11T21:49:15.458 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-09-11T21:49:15.458 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T21:49:15.524 INFO:tasks.cephadm:Adding mgr.x on smithi001 2024-09-11T21:49:15.524 INFO:tasks.cephadm:Adding mgr.y on smithi053 2024-09-11T21:49:15.524 INFO:tasks.cephadm:Adding mgr.z on smithi161 2024-09-11T21:49:15.524 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph orch apply mgr '3;smithi001=x;smithi053=y;smithi161=z' 2024-09-11T21:49:15.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:15 smithi053 ceph-mon[34908]: mon.b calling monitor election 2024-09-11T21:49:15.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:15 smithi053 ceph-mon[34908]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:49:15.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:15 smithi053 ceph-mon[34908]: Updating smithi161:/var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T21:49:15.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:15 smithi053 ceph-mon[34908]: Updating smithi053:/var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T21:49:15.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:15 smithi053 ceph-mon[34908]: Updating smithi001:/var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T21:49:15.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:15 smithi053 ceph-mon[34908]: mon.b calling monitor election 2024-09-11T21:49:15.799 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:15 smithi053 ceph-mon[34908]: mon.c calling monitor election 2024-09-11T21:49:15.799 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:15 smithi053 ceph-mon[34908]: mon.a calling monitor election 2024-09-11T21:49:15.799 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:15 smithi053 ceph-mon[34908]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-11T21:49:15.799 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:15 smithi053 ceph-mon[34908]: monmap epoch 3 2024-09-11T21:49:15.799 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:15 smithi053 ceph-mon[34908]: fsid 3ffca83c-7087-11ef-bcea-c7b262605968 2024-09-11T21:49:15.799 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:15 smithi053 ceph-mon[34908]: last_changed 2024-09-11T21:49:07.698583+0000 2024-09-11T21:49:15.799 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:15 smithi053 ceph-mon[34908]: created 2024-09-11T21:47:25.953679+0000 2024-09-11T21:49:15.799 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:15 smithi053 ceph-mon[34908]: min_mon_release 19 (squid) 2024-09-11T21:49:15.799 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:15 smithi053 ceph-mon[34908]: election_strategy: 1 2024-09-11T21:49:15.799 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:15 smithi053 ceph-mon[34908]: 0: [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] mon.a 2024-09-11T21:49:15.799 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:15 smithi053 ceph-mon[34908]: 1: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.c 2024-09-11T21:49:15.799 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:15 smithi053 ceph-mon[34908]: 2: [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] mon.b 2024-09-11T21:49:15.799 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:15 smithi053 ceph-mon[34908]: fsmap 2024-09-11T21:49:15.800 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:15 smithi053 ceph-mon[34908]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T21:49:15.800 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:15 smithi053 ceph-mon[34908]: mgrmap e13: x(active, since 50s) 2024-09-11T21:49:15.800 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:15 smithi053 ceph-mon[34908]: overall HEALTH_OK 2024-09-11T21:49:15.800 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:15 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:15.800 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:15 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:15.800 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:15 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:15.800 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:15 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:15.800 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:15 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:15.800 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:15 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:15.800 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:15 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:15.800 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:15 smithi053 ceph-mon[34908]: Reconfiguring mon.a (monmap changed)... 2024-09-11T21:49:15.800 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:15 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T21:49:15.800 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:15 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T21:49:15.800 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:15 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:49:15.801 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:15 smithi053 ceph-mon[34908]: Reconfiguring daemon mon.a on smithi001 2024-09-11T21:49:15.801 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T21:49:15.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:15 smithi161 ceph-mon[35615]: mon.b calling monitor election 2024-09-11T21:49:15.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:15 smithi161 ceph-mon[35615]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:49:15.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:15 smithi161 ceph-mon[35615]: Updating smithi161:/var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T21:49:15.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:15 smithi161 ceph-mon[35615]: Updating smithi053:/var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T21:49:15.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:15 smithi161 ceph-mon[35615]: Updating smithi001:/var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T21:49:15.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:15 smithi161 ceph-mon[35615]: mon.b calling monitor election 2024-09-11T21:49:15.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:15 smithi161 ceph-mon[35615]: mon.c calling monitor election 2024-09-11T21:49:15.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:15 smithi161 ceph-mon[35615]: mon.a calling monitor election 2024-09-11T21:49:15.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:15 smithi161 ceph-mon[35615]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-11T21:49:15.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:15 smithi161 ceph-mon[35615]: monmap epoch 3 2024-09-11T21:49:15.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:15 smithi161 ceph-mon[35615]: fsid 3ffca83c-7087-11ef-bcea-c7b262605968 2024-09-11T21:49:15.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:15 smithi161 ceph-mon[35615]: last_changed 2024-09-11T21:49:07.698583+0000 2024-09-11T21:49:15.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:15 smithi161 ceph-mon[35615]: created 2024-09-11T21:47:25.953679+0000 2024-09-11T21:49:15.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:15 smithi161 ceph-mon[35615]: min_mon_release 19 (squid) 2024-09-11T21:49:15.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:15 smithi161 ceph-mon[35615]: election_strategy: 1 2024-09-11T21:49:15.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:15 smithi161 ceph-mon[35615]: 0: [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] mon.a 2024-09-11T21:49:15.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:15 smithi161 ceph-mon[35615]: 1: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.c 2024-09-11T21:49:15.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:15 smithi161 ceph-mon[35615]: 2: [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] mon.b 2024-09-11T21:49:15.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:15 smithi161 ceph-mon[35615]: fsmap 2024-09-11T21:49:15.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:15 smithi161 ceph-mon[35615]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T21:49:15.844 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:15 smithi161 ceph-mon[35615]: mgrmap e13: x(active, since 50s) 2024-09-11T21:49:15.844 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:15 smithi161 ceph-mon[35615]: overall HEALTH_OK 2024-09-11T21:49:15.844 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:15 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:15.844 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:15 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:15.844 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:15 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:15.844 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:15 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:15.844 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:15 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:15.844 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:15 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:15.844 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:15 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:15.844 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:15 smithi161 ceph-mon[35615]: Reconfiguring mon.a (monmap changed)... 2024-09-11T21:49:15.844 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:15 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T21:49:15.844 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:15 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T21:49:15.844 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:15 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:49:15.845 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:15 smithi161 ceph-mon[35615]: Reconfiguring daemon mon.a on smithi001 2024-09-11T21:49:15.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:15 smithi001 ceph-mon[30935]: mon.b calling monitor election 2024-09-11T21:49:15.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:15 smithi001 ceph-mon[30935]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:49:15.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:15 smithi001 ceph-mon[30935]: Updating smithi161:/var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T21:49:15.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:15 smithi001 ceph-mon[30935]: Updating smithi053:/var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T21:49:15.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:15 smithi001 ceph-mon[30935]: Updating smithi001:/var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T21:49:15.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:15 smithi001 ceph-mon[30935]: mon.b calling monitor election 2024-09-11T21:49:15.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:15 smithi001 ceph-mon[30935]: mon.c calling monitor election 2024-09-11T21:49:15.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:15 smithi001 ceph-mon[30935]: mon.a calling monitor election 2024-09-11T21:49:15.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:15 smithi001 ceph-mon[30935]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-11T21:49:15.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:15 smithi001 ceph-mon[30935]: monmap epoch 3 2024-09-11T21:49:15.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:15 smithi001 ceph-mon[30935]: fsid 3ffca83c-7087-11ef-bcea-c7b262605968 2024-09-11T21:49:15.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:15 smithi001 ceph-mon[30935]: last_changed 2024-09-11T21:49:07.698583+0000 2024-09-11T21:49:15.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:15 smithi001 ceph-mon[30935]: created 2024-09-11T21:47:25.953679+0000 2024-09-11T21:49:15.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:15 smithi001 ceph-mon[30935]: min_mon_release 19 (squid) 2024-09-11T21:49:15.945 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:15 smithi001 ceph-mon[30935]: election_strategy: 1 2024-09-11T21:49:15.945 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:15 smithi001 ceph-mon[30935]: 0: [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] mon.a 2024-09-11T21:49:15.945 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:15 smithi001 ceph-mon[30935]: 1: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.c 2024-09-11T21:49:15.945 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:15 smithi001 ceph-mon[30935]: 2: [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] mon.b 2024-09-11T21:49:15.945 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:15 smithi001 ceph-mon[30935]: fsmap 2024-09-11T21:49:15.945 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:15 smithi001 ceph-mon[30935]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T21:49:15.945 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:15 smithi001 ceph-mon[30935]: mgrmap e13: x(active, since 50s) 2024-09-11T21:49:15.945 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:15 smithi001 ceph-mon[30935]: overall HEALTH_OK 2024-09-11T21:49:15.945 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:15 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:15.945 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:15 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:15.945 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:15 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:15.945 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:15 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:15.945 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:15 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:15.946 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:15 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:15.946 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:15 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:15.946 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:15 smithi001 ceph-mon[30935]: Reconfiguring mon.a (monmap changed)... 2024-09-11T21:49:15.946 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:15 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T21:49:15.946 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:15 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T21:49:15.946 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:15 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:49:15.946 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:15 smithi001 ceph-mon[30935]: Reconfiguring daemon mon.a on smithi001 2024-09-11T21:49:16.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:16.266+0000 7f83d41be640 1 -- 172.21.15.161:0/3433575530 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83cc1005f0 msgr2=0x7f83cc1009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:49:16.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:16.266+0000 7f83d41be640 1 --2- 172.21.15.161:0/3433575530 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83cc1005f0 0x7f83cc1009f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f83c4009880 tx=0x7f83c402a5d0 comp rx=0 tx=0).stop 2024-09-11T21:49:16.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:16.266+0000 7f83d41be640 1 -- 172.21.15.161:0/3433575530 shutdown_connections 2024-09-11T21:49:16.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:16.266+0000 7f83d41be640 1 --2- 172.21.15.161:0/3433575530 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83cc1005f0 0x7f83cc1009f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:49:16.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:16.266+0000 7f83d41be640 1 -- 172.21.15.161:0/3433575530 >> 172.21.15.161:0/3433575530 conn(0x7f83cc0fbda0 msgr2=0x7f83cc0fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:49:16.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:16.266+0000 7f83d41be640 1 -- 172.21.15.161:0/3433575530 shutdown_connections 2024-09-11T21:49:16.268 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:16.266+0000 7f83d41be640 1 -- 172.21.15.161:0/3433575530 wait complete. 2024-09-11T21:49:16.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:16.266+0000 7f83d41be640 1 Processor -- start 2024-09-11T21:49:16.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:16.266+0000 7f83d41be640 1 -- start start 2024-09-11T21:49:16.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:16.267+0000 7f83d41be640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83cc1005f0 0x7f83cc0fdc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:49:16.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:16.267+0000 7f83d41be640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83cc0fc340 0x7f83cc0fc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:49:16.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:16.267+0000 7f83d41be640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83cc0fcd00 0x7f83cc1a1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:49:16.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:16.267+0000 7f83d41be640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f83cc106700 con 0x7f83cc0fc340 2024-09-11T21:49:16.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:16.267+0000 7f83d41be640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f83cc106580 con 0x7f83cc0fcd00 2024-09-11T21:49:16.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:16.267+0000 7f83d41be640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f83cc106880 con 0x7f83cc1005f0 2024-09-11T21:49:16.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:16.267+0000 7f83d1f5a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83cc1005f0 0x7f83cc0fdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:49:16.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:16.267+0000 7f83d1759640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83cc0fc340 0x7f83cc0fc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:49:16.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:16.267+0000 7f83d1f5a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83cc1005f0 0x7f83cc0fdc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:47746/0 (socket says 172.21.15.161:47746) 2024-09-11T21:49:16.269 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:16.267+0000 7f83d1f5a640 1 -- 172.21.15.161:0/2347199725 learned_addr learned my addr 172.21.15.161:0/2347199725 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-09-11T21:49:16.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:16.267+0000 7f83d275b640 1 --2- 172.21.15.161:0/2347199725 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83cc0fcd00 0x7f83cc1a1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:49:16.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:16.267+0000 7f83d1759640 1 -- 172.21.15.161:0/2347199725 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83cc1005f0 msgr2=0x7f83cc0fdc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:49:16.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:16.267+0000 7f83d1759640 1 --2- 172.21.15.161:0/2347199725 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83cc1005f0 0x7f83cc0fdc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:49:16.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:16.267+0000 7f83d1759640 1 -- 172.21.15.161:0/2347199725 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83cc0fcd00 msgr2=0x7f83cc1a1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:49:16.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:16.267+0000 7f83d1759640 1 --2- 172.21.15.161:0/2347199725 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83cc0fcd00 0x7f83cc1a1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:49:16.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:16.267+0000 7f83d1759640 1 -- 172.21.15.161:0/2347199725 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83cc1a15c0 con 0x7f83cc0fc340 2024-09-11T21:49:16.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:16.267+0000 7f83d1759640 1 --2- 172.21.15.161:0/2347199725 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83cc0fc340 0x7f83cc0fc7c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f83c000b450 tx=0x7f83c000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:49:16.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:16.267+0000 7f83caffd640 1 -- 172.21.15.161:0/2347199725 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c0004230 con 0x7f83cc0fc340 2024-09-11T21:49:16.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:16.267+0000 7f83caffd640 1 -- 172.21.15.161:0/2347199725 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83c00043d0 con 0x7f83cc0fc340 2024-09-11T21:49:16.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:16.267+0000 7f83caffd640 1 -- 172.21.15.161:0/2347199725 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c0003cb0 con 0x7f83cc0fc340 2024-09-11T21:49:16.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:16.268+0000 7f83d41be640 1 -- 172.21.15.161:0/2347199725 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83cc1a18b0 con 0x7f83cc0fc340 2024-09-11T21:49:16.270 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:16.268+0000 7f83d41be640 1 -- 172.21.15.161:0/2347199725 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83cc1a1e40 con 0x7f83cc0fc340 2024-09-11T21:49:16.271 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:16.268+0000 7f83caffd640 1 -- 172.21.15.161:0/2347199725 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f83c0011630 con 0x7f83cc0fc340 2024-09-11T21:49:16.271 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:16.268+0000 7f83d41be640 1 -- 172.21.15.161:0/2347199725 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83cc1907d0 con 0x7f83cc0fc340 2024-09-11T21:49:16.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:16.269+0000 7f83caffd640 1 --2- 172.21.15.161:0/2347199725 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f83b40394f0 0x7f83b403b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:49:16.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:16.269+0000 7f83caffd640 1 -- 172.21.15.161:0/2347199725 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f83c004bdc0 con 0x7f83cc0fc340 2024-09-11T21:49:16.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:16.269+0000 7f83d1f5a640 1 --2- 172.21.15.161:0/2347199725 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f83b40394f0 0x7f83b403b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:49:16.272 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:16.269+0000 7f83d1f5a640 1 --2- 172.21.15.161:0/2347199725 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f83b40394f0 0x7f83b403b9b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f83c40096c0 tx=0x7f83c40315d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:49:16.276 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:16.274+0000 7f83caffd640 1 -- 172.21.15.161:0/2347199725 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f83c0003e50 con 0x7f83cc0fc340 2024-09-11T21:49:16.454 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:16.453+0000 7f83d41be640 1 -- 172.21.15.161:0/2347199725 --> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi001=x;smithi053=y;smithi161=z", "target": ["mon-mgr", ""]}) -- 0x7f83cc062630 con 0x7f83b40394f0 2024-09-11T21:49:16.458 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:16.458+0000 7f83caffd640 1 -- 172.21.15.161:0/2347199725 <== mgr.14150 v2:172.21.15.1:6800/3890350886 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f83cc062630 con 0x7f83b40394f0 2024-09-11T21:49:16.459 INFO:teuthology.orchestra.run.smithi161.stdout:Scheduled mgr update... 2024-09-11T21:49:16.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:16.460+0000 7f83d41be640 1 -- 172.21.15.161:0/2347199725 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f83b40394f0 msgr2=0x7f83b403b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:49:16.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:16.460+0000 7f83d41be640 1 --2- 172.21.15.161:0/2347199725 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f83b40394f0 0x7f83b403b9b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f83c40096c0 tx=0x7f83c40315d0 comp rx=0 tx=0).stop 2024-09-11T21:49:16.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:16.460+0000 7f83d41be640 1 -- 172.21.15.161:0/2347199725 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83cc0fc340 msgr2=0x7f83cc0fc7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:49:16.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:16.460+0000 7f83d41be640 1 --2- 172.21.15.161:0/2347199725 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83cc0fc340 0x7f83cc0fc7c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f83c000b450 tx=0x7f83c000b910 comp rx=0 tx=0).stop 2024-09-11T21:49:16.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:16.460+0000 7f83d41be640 1 -- 172.21.15.161:0/2347199725 shutdown_connections 2024-09-11T21:49:16.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:16.460+0000 7f83d41be640 1 --2- 172.21.15.161:0/2347199725 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f83b40394f0 0x7f83b403b9b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:49:16.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:16.460+0000 7f83d41be640 1 --2- 172.21.15.161:0/2347199725 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83cc0fcd00 0x7f83cc1a1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:49:16.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:16.460+0000 7f83d41be640 1 --2- 172.21.15.161:0/2347199725 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83cc0fc340 0x7f83cc0fc7c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:49:16.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:16.460+0000 7f83d41be640 1 --2- 172.21.15.161:0/2347199725 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83cc1005f0 0x7f83cc0fdc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:49:16.461 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:16.460+0000 7f83d41be640 1 -- 172.21.15.161:0/2347199725 >> 172.21.15.161:0/2347199725 conn(0x7f83cc0fbda0 msgr2=0x7f83cc06a950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:49:16.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:16.460+0000 7f83d41be640 1 -- 172.21.15.161:0/2347199725 shutdown_connections 2024-09-11T21:49:16.462 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:49:16.460+0000 7f83d41be640 1 -- 172.21.15.161:0/2347199725 wait complete. 2024-09-11T21:49:16.748 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:16 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T21:49:16.748 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:16 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:16.748 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:16 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:16.748 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:16 smithi053 ceph-mon[34908]: Reconfiguring mon.b (monmap changed)... 2024-09-11T21:49:16.748 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:16 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T21:49:16.748 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:16 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T21:49:16.749 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:16 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:49:16.749 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:16 smithi053 ceph-mon[34908]: Reconfiguring daemon mon.b on smithi053 2024-09-11T21:49:16.749 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:16 smithi053 ceph-mon[34908]: from='client.14214 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi001=x;smithi053=y;smithi161=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:49:16.749 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:16 smithi053 ceph-mon[34908]: Saving service mgr spec with placement smithi001=x;smithi053=y;smithi161=z;count:3 2024-09-11T21:49:16.749 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:16 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:16.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:16 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T21:49:16.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:16 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:16.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:16 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:16.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:16 smithi161 ceph-mon[35615]: Reconfiguring mon.b (monmap changed)... 2024-09-11T21:49:16.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:16 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T21:49:16.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:16 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T21:49:16.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:16 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:49:16.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:16 smithi161 ceph-mon[35615]: Reconfiguring daemon mon.b on smithi053 2024-09-11T21:49:16.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:16 smithi161 ceph-mon[35615]: from='client.14214 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi001=x;smithi053=y;smithi161=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:49:16.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:16 smithi161 ceph-mon[35615]: Saving service mgr spec with placement smithi001=x;smithi053=y;smithi161=z;count:3 2024-09-11T21:49:16.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:16 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:16.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:16 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T21:49:16.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:16 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:16.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:16 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:16.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:16 smithi001 ceph-mon[30935]: Reconfiguring mon.b (monmap changed)... 2024-09-11T21:49:16.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:16 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T21:49:16.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:16 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T21:49:16.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:16 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:49:16.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:16 smithi001 ceph-mon[30935]: Reconfiguring daemon mon.b on smithi053 2024-09-11T21:49:16.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:16 smithi001 ceph-mon[30935]: from='client.14214 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi001=x;smithi053=y;smithi161=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:49:16.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:16 smithi001 ceph-mon[30935]: Saving service mgr spec with placement smithi001=x;smithi053=y;smithi161=z;count:3 2024-09-11T21:49:16.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:16 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:17.074 DEBUG:teuthology.orchestra.run.smithi053:mgr.y> sudo journalctl -f -n 0 -u ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mgr.y.service 2024-09-11T21:49:17.077 DEBUG:teuthology.orchestra.run.smithi161:mgr.z> sudo journalctl -f -n 0 -u ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mgr.z.service 2024-09-11T21:49:17.080 INFO:tasks.cephadm:Deploying OSDs... 2024-09-11T21:49:17.080 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-11T21:49:17.080 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/scratch_devs of=/dev/stdout 2024-09-11T21:49:17.100 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-11T21:49:17.100 DEBUG:teuthology.orchestra.run.smithi001:> stat /dev/vg_nvme/lv_1 2024-09-11T21:49:17.159 INFO:teuthology.orchestra.run.smithi001.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-11T21:49:17.159 INFO:teuthology.orchestra.run.smithi001.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T21:49:17.159 INFO:teuthology.orchestra.run.smithi001.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-11T21:49:17.159 INFO:teuthology.orchestra.run.smithi001.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T21:49:17.160 INFO:teuthology.orchestra.run.smithi001.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T21:49:17.160 INFO:teuthology.orchestra.run.smithi001.stdout:Access: 2024-09-11 21:48:33.078990092 +0000 2024-09-11T21:49:17.160 INFO:teuthology.orchestra.run.smithi001.stdout:Modify: 2024-09-11 21:43:48.816141670 +0000 2024-09-11T21:49:17.160 INFO:teuthology.orchestra.run.smithi001.stdout:Change: 2024-09-11 21:43:48.816141670 +0000 2024-09-11T21:49:17.160 INFO:teuthology.orchestra.run.smithi001.stdout: Birth: 2024-09-11 21:43:48.816141670 +0000 2024-09-11T21:49:17.160 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-11T21:49:17.229 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records in 2024-09-11T21:49:17.229 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records out 2024-09-11T21:49:17.229 INFO:teuthology.orchestra.run.smithi001.stderr:512 bytes copied, 0.000130337 s, 3.9 MB/s 2024-09-11T21:49:17.230 DEBUG:teuthology.orchestra.run.smithi001:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-11T21:49:17.291 DEBUG:teuthology.orchestra.run.smithi001:> stat /dev/vg_nvme/lv_2 2024-09-11T21:49:17.349 INFO:teuthology.orchestra.run.smithi001.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-11T21:49:17.349 INFO:teuthology.orchestra.run.smithi001.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T21:49:17.350 INFO:teuthology.orchestra.run.smithi001.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-11T21:49:17.350 INFO:teuthology.orchestra.run.smithi001.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T21:49:17.350 INFO:teuthology.orchestra.run.smithi001.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T21:49:17.350 INFO:teuthology.orchestra.run.smithi001.stdout:Access: 2024-09-11 21:48:33.107989731 +0000 2024-09-11T21:49:17.350 INFO:teuthology.orchestra.run.smithi001.stdout:Modify: 2024-09-11 21:43:49.137137720 +0000 2024-09-11T21:49:17.350 INFO:teuthology.orchestra.run.smithi001.stdout:Change: 2024-09-11 21:43:49.137137720 +0000 2024-09-11T21:49:17.350 INFO:teuthology.orchestra.run.smithi001.stdout: Birth: 2024-09-11 21:43:49.137137720 +0000 2024-09-11T21:49:17.350 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-11T21:49:17.416 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records in 2024-09-11T21:49:17.416 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records out 2024-09-11T21:49:17.417 INFO:teuthology.orchestra.run.smithi001.stderr:512 bytes copied, 0.000127401 s, 4.0 MB/s 2024-09-11T21:49:17.418 DEBUG:teuthology.orchestra.run.smithi001:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-11T21:49:17.477 DEBUG:teuthology.orchestra.run.smithi001:> stat /dev/vg_nvme/lv_3 2024-09-11T21:49:17.535 INFO:teuthology.orchestra.run.smithi001.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-11T21:49:17.535 INFO:teuthology.orchestra.run.smithi001.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T21:49:17.535 INFO:teuthology.orchestra.run.smithi001.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-11T21:49:17.535 INFO:teuthology.orchestra.run.smithi001.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T21:49:17.535 INFO:teuthology.orchestra.run.smithi001.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T21:49:17.535 INFO:teuthology.orchestra.run.smithi001.stdout:Access: 2024-09-11 21:48:33.137989357 +0000 2024-09-11T21:49:17.535 INFO:teuthology.orchestra.run.smithi001.stdout:Modify: 2024-09-11 21:43:49.456133794 +0000 2024-09-11T21:49:17.535 INFO:teuthology.orchestra.run.smithi001.stdout:Change: 2024-09-11 21:43:49.456133794 +0000 2024-09-11T21:49:17.535 INFO:teuthology.orchestra.run.smithi001.stdout: Birth: 2024-09-11 21:43:49.456133794 +0000 2024-09-11T21:49:17.536 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-11T21:49:17.602 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records in 2024-09-11T21:49:17.602 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records out 2024-09-11T21:49:17.602 INFO:teuthology.orchestra.run.smithi001.stderr:512 bytes copied, 0.000111703 s, 4.6 MB/s 2024-09-11T21:49:17.603 DEBUG:teuthology.orchestra.run.smithi001:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-11T21:49:17.661 DEBUG:teuthology.orchestra.run.smithi001:> stat /dev/vg_nvme/lv_4 2024-09-11T21:49:17.718 INFO:teuthology.orchestra.run.smithi001.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-11T21:49:17.718 INFO:teuthology.orchestra.run.smithi001.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T21:49:17.718 INFO:teuthology.orchestra.run.smithi001.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-11T21:49:17.718 INFO:teuthology.orchestra.run.smithi001.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T21:49:17.718 INFO:teuthology.orchestra.run.smithi001.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T21:49:17.718 INFO:teuthology.orchestra.run.smithi001.stdout:Access: 2024-09-11 21:48:33.167988984 +0000 2024-09-11T21:49:17.718 INFO:teuthology.orchestra.run.smithi001.stdout:Modify: 2024-09-11 21:43:49.776129857 +0000 2024-09-11T21:49:17.719 INFO:teuthology.orchestra.run.smithi001.stdout:Change: 2024-09-11 21:43:49.776129857 +0000 2024-09-11T21:49:17.719 INFO:teuthology.orchestra.run.smithi001.stdout: Birth: 2024-09-11 21:43:49.776129857 +0000 2024-09-11T21:49:17.719 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-11T21:49:17.783 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records in 2024-09-11T21:49:17.784 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records out 2024-09-11T21:49:17.784 INFO:teuthology.orchestra.run.smithi001.stderr:512 bytes copied, 9.2538e-05 s, 5.5 MB/s 2024-09-11T21:49:17.785 DEBUG:teuthology.orchestra.run.smithi001:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-11T21:49:17.842 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-11T21:49:17.842 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/scratch_devs of=/dev/stdout 2024-09-11T21:49:17.859 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-11T21:49:17.859 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_1 2024-09-11T21:49:17.915 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-11T21:49:17.916 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T21:49:17.916 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-09-11T21:49:17.916 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T21:49:17.916 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T21:49:17.916 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2024-09-11 21:49:07.150756768 +0000 2024-09-11T21:49:17.916 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2024-09-11 21:44:03.205837509 +0000 2024-09-11T21:49:17.916 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2024-09-11 21:44:03.205837509 +0000 2024-09-11T21:49:17.916 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: 2024-09-11 21:44:03.205837509 +0000 2024-09-11T21:49:17.917 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-11T21:49:17.981 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2024-09-11T21:49:17.981 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2024-09-11T21:49:17.981 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.000111913 s, 4.6 MB/s 2024-09-11T21:49:17.982 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-11T21:49:18.039 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_2 2024-09-11T21:49:18.095 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-11T21:49:18.095 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T21:49:18.095 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-11T21:49:18.095 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T21:49:18.095 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T21:49:18.096 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2024-09-11 21:49:07.150756768 +0000 2024-09-11T21:49:18.096 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2024-09-11 21:44:03.523830908 +0000 2024-09-11T21:49:18.096 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2024-09-11 21:44:03.523830908 +0000 2024-09-11T21:49:18.096 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: 2024-09-11 21:44:03.523830908 +0000 2024-09-11T21:49:18.096 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-11T21:49:18.159 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2024-09-11T21:49:18.159 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2024-09-11T21:49:18.159 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.000109076 s, 4.7 MB/s 2024-09-11T21:49:18.160 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-11T21:49:18.217 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_3 2024-09-11T21:49:18.273 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-11T21:49:18.273 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T21:49:18.273 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-11T21:49:18.273 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T21:49:18.273 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T21:49:18.273 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2024-09-11 21:49:07.150756768 +0000 2024-09-11T21:49:18.273 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2024-09-11 21:44:03.841824307 +0000 2024-09-11T21:49:18.273 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2024-09-11 21:44:03.841824307 +0000 2024-09-11T21:49:18.273 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: 2024-09-11 21:44:03.841824307 +0000 2024-09-11T21:49:18.274 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-11T21:49:18.339 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2024-09-11T21:49:18.339 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2024-09-11T21:49:18.339 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.000105667 s, 4.8 MB/s 2024-09-11T21:49:18.340 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-11T21:49:18.396 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_4 2024-09-11T21:49:18.451 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-11T21:49:18.452 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T21:49:18.452 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-11T21:49:18.452 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T21:49:18.452 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T21:49:18.452 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2024-09-11 21:49:07.151756747 +0000 2024-09-11T21:49:18.452 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2024-09-11 21:44:04.159817707 +0000 2024-09-11T21:49:18.452 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2024-09-11 21:44:04.159817707 +0000 2024-09-11T21:49:18.452 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: 2024-09-11 21:44:04.159817707 +0000 2024-09-11T21:49:18.452 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-11T21:49:18.515 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2024-09-11T21:49:18.515 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2024-09-11T21:49:18.515 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.000106877 s, 4.8 MB/s 2024-09-11T21:49:18.516 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-11T21:49:18.572 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-09-11T21:49:18.573 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/scratch_devs of=/dev/stdout 2024-09-11T21:49:18.590 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-11T21:49:18.590 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_1 2024-09-11T21:49:18.646 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-11T21:49:18.646 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T21:49:18.646 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-11T21:49:18.646 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T21:49:18.646 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T21:49:18.646 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-09-11 21:48:58.683626866 +0000 2024-09-11T21:49:18.646 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-09-11 21:43:56.120782254 +0000 2024-09-11T21:49:18.647 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-09-11 21:43:56.120782254 +0000 2024-09-11T21:49:18.647 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: 2024-09-11 21:43:56.120782254 +0000 2024-09-11T21:49:18.647 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-11T21:49:18.710 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-09-11T21:49:18.711 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-09-11T21:49:18.711 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000263842 s, 1.9 MB/s 2024-09-11T21:49:18.711 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:18 smithi161 ceph-mon[35615]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:49:18.712 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:18 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:18.712 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:18 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:18.712 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:18 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T21:49:18.712 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:18 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T21:49:18.712 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:18 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:49:18.714 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-11T21:49:18.770 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_2 2024-09-11T21:49:18.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:18 smithi053 ceph-mon[34908]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:49:18.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:18 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:18.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:18 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:18.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:18 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T21:49:18.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:18 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T21:49:18.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:18 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:49:18.826 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-11T21:49:18.826 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T21:49:18.827 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-11T21:49:18.827 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T21:49:18.827 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T21:49:18.827 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-09-11 21:48:58.683626866 +0000 2024-09-11T21:49:18.827 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-09-11 21:43:56.441775251 +0000 2024-09-11T21:49:18.827 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-09-11 21:43:56.441775251 +0000 2024-09-11T21:49:18.827 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: 2024-09-11 21:43:56.441775251 +0000 2024-09-11T21:49:18.827 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-11T21:49:18.845 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:18 smithi001 ceph-mon[30935]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:49:18.845 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:18 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:18.845 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:18 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:18.845 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:18 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T21:49:18.845 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:18 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T21:49:18.845 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:18 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:49:18.891 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-09-11T21:49:18.892 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-09-11T21:49:18.892 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000208988 s, 2.4 MB/s 2024-09-11T21:49:18.893 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-11T21:49:18.949 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_3 2024-09-11T21:49:19.006 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-11T21:49:19.006 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T21:49:19.006 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-09-11T21:49:19.006 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T21:49:19.006 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T21:49:19.006 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-09-11 21:48:58.684626845 +0000 2024-09-11T21:49:19.006 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-09-11 21:43:56.768768118 +0000 2024-09-11T21:49:19.006 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-09-11 21:43:56.768768118 +0000 2024-09-11T21:49:19.006 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: 2024-09-11 21:43:56.768768118 +0000 2024-09-11T21:49:19.007 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-11T21:49:19.070 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-09-11T21:49:19.070 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-09-11T21:49:19.070 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000223829 s, 2.3 MB/s 2024-09-11T21:49:19.072 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-11T21:49:19.129 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_4 2024-09-11T21:49:19.185 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-11T21:49:19.186 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T21:49:19.186 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-09-11T21:49:19.186 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T21:49:19.186 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T21:49:19.186 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-09-11 21:48:58.684626845 +0000 2024-09-11T21:49:19.186 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-09-11 21:43:57.082761269 +0000 2024-09-11T21:49:19.186 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-09-11 21:43:57.082761269 +0000 2024-09-11T21:49:19.186 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: 2024-09-11 21:43:57.082761269 +0000 2024-09-11T21:49:19.186 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-11T21:49:19.253 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-09-11T21:49:19.254 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-09-11T21:49:19.254 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000240087 s, 2.1 MB/s 2024-09-11T21:49:19.255 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-11T21:49:19.311 INFO:tasks.cephadm:Deploying osd.0 on smithi001 with /dev/vg_nvme/lv_4... 2024-09-11T21:49:19.311 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-11T21:49:19.578 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:49:19.630 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:19 smithi161 ceph-mon[35615]: Reconfiguring mon.c (monmap changed)... 2024-09-11T21:49:19.631 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:19 smithi161 ceph-mon[35615]: Reconfiguring daemon mon.c on smithi161 2024-09-11T21:49:19.631 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:19 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:19.631 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:19 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:19.631 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:19 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:49:19.631 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:19 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:49:19.631 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:19 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:49:19.631 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:19 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:19.631 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:19 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T21:49:19.631 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:19 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T21:49:19.631 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:19 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T21:49:19.631 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:19 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:49:19.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:19 smithi001 ceph-mon[30935]: Reconfiguring mon.c (monmap changed)... 2024-09-11T21:49:19.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:19 smithi001 ceph-mon[30935]: Reconfiguring daemon mon.c on smithi161 2024-09-11T21:49:19.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:19 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:19.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:19 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:19.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:19 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:49:19.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:19 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:49:19.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:19 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:49:19.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:19 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:19.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:19 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T21:49:19.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:19 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T21:49:19.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:19 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T21:49:19.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:19 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:49:19.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:19 smithi053 ceph-mon[34908]: Reconfiguring mon.c (monmap changed)... 2024-09-11T21:49:19.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:19 smithi053 ceph-mon[34908]: Reconfiguring daemon mon.c on smithi161 2024-09-11T21:49:19.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:19 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:19.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:19 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:19.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:19 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:49:19.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:19 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:49:19.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:19 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:49:19.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:19 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:19.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:19 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T21:49:19.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:19 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T21:49:19.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:19 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T21:49:19.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:19 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:49:20.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:20 smithi053 ceph-mon[34908]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:49:20.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:20 smithi053 ceph-mon[34908]: Deploying daemon mgr.z on smithi161 2024-09-11T21:49:20.808 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:20 smithi161 ceph-mon[35615]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:49:20.808 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:20 smithi161 ceph-mon[35615]: Deploying daemon mgr.z on smithi161 2024-09-11T21:49:20.808 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 11 21:49:20 smithi161 systemd[1]: Starting Ceph mgr.z for 3ffca83c-7087-11ef-bcea-c7b262605968... 2024-09-11T21:49:20.821 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:20 smithi001 ceph-mon[30935]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:49:20.822 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:20 smithi001 ceph-mon[30935]: Deploying daemon mgr.z on smithi161 2024-09-11T21:49:21.088 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 11 21:49:20 smithi161 podman[37401]: 2024-09-11 21:49:20.710119125 +0000 UTC m=+0.016175582 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T21:49:21.088 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 11 21:49:20 smithi161 podman[37401]: 2024-09-11 21:49:20.827142095 +0000 UTC m=+0.133198545 container create 57051efff98290a971d35f0fab755f4dfaa93b5030c769aabd1c1f25e0cab756 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T21:49:21.088 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 11 21:49:21 smithi161 podman[37401]: 2024-09-11 21:49:21.021121759 +0000 UTC m=+0.327178207 container init 57051efff98290a971d35f0fab755f4dfaa93b5030c769aabd1c1f25e0cab756 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-11T21:49:21.088 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 11 21:49:21 smithi161 podman[37401]: 2024-09-11 21:49:21.02393388 +0000 UTC m=+0.329990339 container start 57051efff98290a971d35f0fab755f4dfaa93b5030c769aabd1c1f25e0cab756 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-z, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-11T21:49:21.088 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 11 21:49:21 smithi161 ceph-mgr[37420]: -- 172.21.15.161:0/2110795139 <== mon.1 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a27d2e10e0 con 0x55a27d2bf400 2024-09-11T21:49:21.088 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 11 21:49:21 smithi161 bash[37401]: 57051efff98290a971d35f0fab755f4dfaa93b5030c769aabd1c1f25e0cab756 2024-09-11T21:49:21.341 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 11 21:49:21 smithi161 systemd[1]: Started Ceph mgr.z for 3ffca83c-7087-11ef-bcea-c7b262605968. 2024-09-11T21:49:21.342 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 11 21:49:21 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-z[37416]: 2024-09-11T21:49:21.150+0000 7ffb2f951040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T21:49:21.342 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 11 21:49:21 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-z[37416]: 2024-09-11T21:49:21.208+0000 7ffb2f951040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T21:49:21.709 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 11 21:49:21 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-z[37416]: 2024-09-11T21:49:21.354+0000 7ffb2f951040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-11T21:49:21.709 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 11 21:49:21 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-z[37416]: 2024-09-11T21:49:21.420+0000 7ffb2f951040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T21:49:21.709 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 11 21:49:21 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-z[37416]: 2024-09-11T21:49:21.567+0000 7ffb2f951040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T21:49:21.709 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 11 21:49:21 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-z[37416]: 2024-09-11T21:49:21.708+0000 7ffb2f951040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T21:49:21.815 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:49:21.841 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi001:vg_nvme/lv_4 2024-09-11T21:49:22.093 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:49:22.342 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 11 21:49:22 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-z[37416]: 2024-09-11T21:49:22.019+0000 7ffb2f951040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T21:49:22.342 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 11 21:49:22 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-z[37416]: 2024-09-11T21:49:22.080+0000 7ffb2f951040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T21:49:22.342 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 11 21:49:22 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-z[37416]: 2024-09-11T21:49:22.264+0000 7ffb2f951040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T21:49:22.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:22 smithi161 ceph-mon[35615]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:49:22.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:22 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:22.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:22 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:22.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:22 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:22.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:22 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T21:49:22.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:22 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T21:49:22.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:22 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T21:49:22.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:22 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:49:22.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:22 smithi161 ceph-mon[35615]: Deploying daemon mgr.y on smithi053 2024-09-11T21:49:22.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:22 smithi001 ceph-mon[30935]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:49:22.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:22 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:22.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:22 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:22.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:22 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:22.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:22 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T21:49:22.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:22 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T21:49:22.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:22 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T21:49:22.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:22 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:49:22.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:22 smithi001 ceph-mon[30935]: Deploying daemon mgr.y on smithi053 2024-09-11T21:49:22.485 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:22 smithi053 ceph-mon[34908]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:49:22.485 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:22 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:22.485 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:22 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:22.485 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:22 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:22.485 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:22 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T21:49:22.485 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:22 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T21:49:22.485 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:22 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T21:49:22.485 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:22 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:49:22.485 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:22 smithi053 ceph-mon[34908]: Deploying daemon mgr.y on smithi053 2024-09-11T21:49:22.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:22.533+0000 7fce512db640 1 -- 172.21.15.1:0/1035047396 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fce4c102e00 msgr2=0x7fce4c103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:49:22.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:22.533+0000 7fce512db640 1 --2- 172.21.15.1:0/1035047396 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fce4c102e00 0x7fce4c103280 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fce34009900 tx=0x7fce3402a410 comp rx=0 tx=0).stop 2024-09-11T21:49:22.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:22.533+0000 7fce512db640 1 -- 172.21.15.1:0/1035047396 shutdown_connections 2024-09-11T21:49:22.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:22.533+0000 7fce512db640 1 --2- 172.21.15.1:0/1035047396 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce4c1037c0 0x7fce4c10a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:49:22.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:22.533+0000 7fce512db640 1 --2- 172.21.15.1:0/1035047396 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fce4c102e00 0x7fce4c103280 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:49:22.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:22.533+0000 7fce512db640 1 --2- 172.21.15.1:0/1035047396 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce4c101c00 0x7fce4c102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:49:22.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:22.533+0000 7fce512db640 1 -- 172.21.15.1:0/1035047396 >> 172.21.15.1:0/1035047396 conn(0x7fce4c0fd390 msgr2=0x7fce4c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:49:22.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:22.534+0000 7fce512db640 1 -- 172.21.15.1:0/1035047396 shutdown_connections 2024-09-11T21:49:22.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:22.534+0000 7fce512db640 1 -- 172.21.15.1:0/1035047396 wait complete. 2024-09-11T21:49:22.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:22.534+0000 7fce512db640 1 Processor -- start 2024-09-11T21:49:22.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:22.534+0000 7fce512db640 1 -- start start 2024-09-11T21:49:22.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:22.534+0000 7fce512db640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce4c101c00 0x7fce4c075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:49:22.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:22.534+0000 7fce512db640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fce4c102e00 0x7fce4c0745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:49:22.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:22.534+0000 7fce512db640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce4c1037c0 0x7fce4c074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:49:22.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:22.534+0000 7fce512db640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fce4c10d900 con 0x7fce4c102e00 2024-09-11T21:49:22.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:22.534+0000 7fce512db640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fce4c10d780 con 0x7fce4c101c00 2024-09-11T21:49:22.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:22.534+0000 7fce512db640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fce4c10da80 con 0x7fce4c1037c0 2024-09-11T21:49:22.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:22.535+0000 7fce43fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fce4c102e00 0x7fce4c0745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:49:22.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:22.535+0000 7fce43fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fce4c102e00 0x7fce4c0745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39226/0 (socket says 172.21.15.1:39226) 2024-09-11T21:49:22.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:22.535+0000 7fce43fff640 1 -- 172.21.15.1:0/3792608538 learned_addr learned my addr 172.21.15.1:0/3792608538 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:49:22.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:22.535+0000 7fce4b7fe640 1 --2- 172.21.15.1:0/3792608538 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce4c1037c0 0x7fce4c074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:49:22.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:22.535+0000 7fce4affd640 1 --2- 172.21.15.1:0/3792608538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce4c101c00 0x7fce4c075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:49:22.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:22.535+0000 7fce43fff640 1 -- 172.21.15.1:0/3792608538 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce4c1037c0 msgr2=0x7fce4c074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:49:22.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:22.535+0000 7fce43fff640 1 --2- 172.21.15.1:0/3792608538 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce4c1037c0 0x7fce4c074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:49:22.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:22.535+0000 7fce43fff640 1 -- 172.21.15.1:0/3792608538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce4c101c00 msgr2=0x7fce4c075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:49:22.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:22.535+0000 7fce43fff640 1 --2- 172.21.15.1:0/3792608538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce4c101c00 0x7fce4c075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:49:22.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:22.535+0000 7fce43fff640 1 -- 172.21.15.1:0/3792608538 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce4c0753e0 con 0x7fce4c102e00 2024-09-11T21:49:22.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:22.535+0000 7fce43fff640 1 --2- 172.21.15.1:0/3792608538 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fce4c102e00 0x7fce4c0745e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fce34009230 tx=0x7fce34031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:49:22.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:22.535+0000 7fce48ff9640 1 -- 172.21.15.1:0/3792608538 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce3403f030 con 0x7fce4c102e00 2024-09-11T21:49:22.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:22.535+0000 7fce48ff9640 1 -- 172.21.15.1:0/3792608538 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce34003ad0 con 0x7fce4c102e00 2024-09-11T21:49:22.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:22.535+0000 7fce512db640 1 -- 172.21.15.1:0/3792608538 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce4c1a20b0 con 0x7fce4c102e00 2024-09-11T21:49:22.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:22.535+0000 7fce48ff9640 1 -- 172.21.15.1:0/3792608538 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce34035070 con 0x7fce4c102e00 2024-09-11T21:49:22.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:22.535+0000 7fce512db640 1 -- 172.21.15.1:0/3792608538 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce4c1a2630 con 0x7fce4c102e00 2024-09-11T21:49:22.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:22.536+0000 7fce48ff9640 1 -- 172.21.15.1:0/3792608538 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7fce34042030 con 0x7fce4c102e00 2024-09-11T21:49:22.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:22.536+0000 7fce512db640 1 -- 172.21.15.1:0/3792608538 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce4c108fa0 con 0x7fce4c102e00 2024-09-11T21:49:22.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:22.536+0000 7fce48ff9640 1 --2- 172.21.15.1:0/3792608538 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fce38039290 0x7fce3803b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:49:22.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:22.537+0000 7fce48ff9640 1 -- 172.21.15.1:0/3792608538 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fce3406c8e0 con 0x7fce4c102e00 2024-09-11T21:49:22.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:22.537+0000 7fce4affd640 1 --2- 172.21.15.1:0/3792608538 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fce38039290 0x7fce3803b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:49:22.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:22.537+0000 7fce4affd640 1 --2- 172.21.15.1:0/3792608538 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fce38039290 0x7fce3803b750 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fce4c102c60 tx=0x7fce3c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:49:22.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:22.542+0000 7fce48ff9640 1 -- 172.21.15.1:0/3792608538 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fce34030c30 con 0x7fce4c102e00 2024-09-11T21:49:22.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:22.725+0000 7fce512db640 1 -- 172.21.15.1:0/3792608538 --> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fce4c062630 con 0x7fce38039290 2024-09-11T21:49:23.418 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:23 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T21:49:23.419 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:23 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T21:49:23.419 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:23 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:49:23.443 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:23 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T21:49:23.444 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:23 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T21:49:23.444 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:23 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:49:23.444 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 11 21:49:23 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-y[36280]: 2024-09-11T21:49:23.381+0000 7fc8fcd84040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T21:49:23.444 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 11 21:49:23 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-y[36280]: 2024-09-11T21:49:23.442+0000 7fc8fcd84040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T21:49:23.592 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 11 21:49:23 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-z[37416]: 2024-09-11T21:49:23.144+0000 7ffb2f951040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T21:49:23.592 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 11 21:49:23 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-z[37416]: 2024-09-11T21:49:23.211+0000 7ffb2f951040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T21:49:23.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:23 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T21:49:23.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:23 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T21:49:23.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:23 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:49:23.797 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 11 21:49:23 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-y[36280]: 2024-09-11T21:49:23.600+0000 7fc8fcd84040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-11T21:49:23.797 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 11 21:49:23 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-y[36280]: 2024-09-11T21:49:23.670+0000 7fc8fcd84040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T21:49:24.136 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 11 21:49:23 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-y[36280]: 2024-09-11T21:49:23.840+0000 7fc8fcd84040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T21:49:24.136 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 11 21:49:23 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-y[36280]: 2024-09-11T21:49:23.994+0000 7fc8fcd84040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T21:49:24.239 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 11 21:49:23 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-z[37416]: 2024-09-11T21:49:23.974+0000 7ffb2f951040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T21:49:24.239 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 11 21:49:24 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-z[37416]: 2024-09-11T21:49:24.090+0000 7ffb2f951040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T21:49:24.239 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 11 21:49:24 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-z[37416]: 2024-09-11T21:49:24.239+0000 7ffb2f951040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T21:49:24.394 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 11 21:49:24 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-y[36280]: 2024-09-11T21:49:24.330+0000 7fc8fcd84040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T21:49:24.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:24 smithi001 ceph-mon[30935]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:49:24.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:24 smithi001 ceph-mon[30935]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:49:24.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:24 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:24.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:24 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:24.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:24 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:24.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:24 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:24.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:24 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:49:24.526 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 11 21:49:24 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-z[37416]: 2024-09-11T21:49:24.306+0000 7ffb2f951040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T21:49:24.526 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 11 21:49:24 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-z[37416]: 2024-09-11T21:49:24.457+0000 7ffb2f951040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T21:49:24.526 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 11 21:49:24 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-z[37416]: 2024-09-11T21:49:24.526+0000 7ffb2f951040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T21:49:24.527 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:24 smithi161 ceph-mon[35615]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:49:24.527 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:24 smithi161 ceph-mon[35615]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:49:24.527 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:24 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:24.527 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:24 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:24.527 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:24 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:24.527 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:24 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:24.527 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:24 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:49:24.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:24 smithi053 ceph-mon[34908]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:49:24.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:24 smithi053 ceph-mon[34908]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:49:24.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:24 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:24.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:24 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:24.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:24 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:24.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:24 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:24.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:24 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:49:24.798 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 11 21:49:24 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-y[36280]: 2024-09-11T21:49:24.393+0000 7fc8fcd84040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T21:49:24.798 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 11 21:49:24 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-y[36280]: 2024-09-11T21:49:24.591+0000 7fc8fcd84040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T21:49:24.825 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 11 21:49:24 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-z[37416]: 2024-09-11T21:49:24.587+0000 7ffb2f951040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T21:49:24.825 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 11 21:49:24 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-z[37416]: 2024-09-11T21:49:24.759+0000 7ffb2f951040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T21:49:24.825 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 11 21:49:24 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-z[37416]: 2024-09-11T21:49:24.824+0000 7ffb2f951040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T21:49:25.089 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 11 21:49:24 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-z[37416]: 2024-09-11T21:49:24.890+0000 7ffb2f951040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T21:49:25.089 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 11 21:49:24 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-z[37416]: 2024-09-11T21:49:24.971+0000 7ffb2f951040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T21:49:25.089 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 11 21:49:25 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-z[37416]: 2024-09-11T21:49:25.030+0000 7ffb2f951040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T21:49:25.089 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 11 21:49:25 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-z[37416]: 2024-09-11T21:49:25.088+0000 7ffb2f951040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T21:49:25.798 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 11 21:49:25 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-y[36280]: 2024-09-11T21:49:25.514+0000 7fc8fcd84040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T21:49:25.798 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 11 21:49:25 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-y[36280]: 2024-09-11T21:49:25.584+0000 7fc8fcd84040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T21:49:25.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:25 smithi053 ceph-mon[34908]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:49:25.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:25 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:25.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:25 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:25.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:25 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:25.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:25 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:25.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:25 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:25.799 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:25 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:49:25.799 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:25 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:49:25.799 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:25 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:25.799 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:25 smithi053 ceph-mon[34908]: Reconfiguring mgr.x (unknown last config time)... 2024-09-11T21:49:25.799 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:25 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T21:49:25.799 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:25 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T21:49:25.799 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:25 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:49:25.799 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:25 smithi053 ceph-mon[34908]: Reconfiguring daemon mgr.x on smithi001 2024-09-11T21:49:25.799 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:25 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/312120736' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "19e56504-18f1-4234-b2d6-f193ff47d8eb"}]: dispatch 2024-09-11T21:49:25.799 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:25 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/312120736' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "19e56504-18f1-4234-b2d6-f193ff47d8eb"}]': finished 2024-09-11T21:49:25.799 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:25 smithi053 ceph-mon[34908]: osdmap e5: 1 total, 0 up, 1 in 2024-09-11T21:49:25.799 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:25 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T21:49:25.799 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:25 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/687623493' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T21:49:25.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:25 smithi161 ceph-mon[35615]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:49:25.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:25 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:25.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:25 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:25.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:25 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:25.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:25 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:25.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:25 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:25.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:25 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:49:25.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:25 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:49:25.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:25 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:25.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:25 smithi161 ceph-mon[35615]: Reconfiguring mgr.x (unknown last config time)... 2024-09-11T21:49:25.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:25 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T21:49:25.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:25 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T21:49:25.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:25 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:49:25.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:25 smithi161 ceph-mon[35615]: Reconfiguring daemon mgr.x on smithi001 2024-09-11T21:49:25.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:25 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/312120736' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "19e56504-18f1-4234-b2d6-f193ff47d8eb"}]: dispatch 2024-09-11T21:49:25.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:25 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/312120736' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "19e56504-18f1-4234-b2d6-f193ff47d8eb"}]': finished 2024-09-11T21:49:25.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:25 smithi161 ceph-mon[35615]: osdmap e5: 1 total, 0 up, 1 in 2024-09-11T21:49:25.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:25 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T21:49:25.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:25 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/687623493' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T21:49:25.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:25 smithi001 ceph-mon[30935]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:49:25.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:25 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:25.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:25 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:25.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:25 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:25.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:25 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:25.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:25 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:25.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:25 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:49:25.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:25 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:49:25.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:25 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:25.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:25 smithi001 ceph-mon[30935]: Reconfiguring mgr.x (unknown last config time)... 2024-09-11T21:49:25.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:25 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T21:49:25.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:25 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T21:49:25.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:25 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:49:25.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:25 smithi001 ceph-mon[30935]: Reconfiguring daemon mgr.x on smithi001 2024-09-11T21:49:25.945 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:25 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/312120736' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "19e56504-18f1-4234-b2d6-f193ff47d8eb"}]: dispatch 2024-09-11T21:49:25.945 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:25 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/312120736' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "19e56504-18f1-4234-b2d6-f193ff47d8eb"}]': finished 2024-09-11T21:49:25.945 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:25 smithi001 ceph-mon[30935]: osdmap e5: 1 total, 0 up, 1 in 2024-09-11T21:49:25.945 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:25 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T21:49:25.945 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:25 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/687623493' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T21:49:26.342 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 11 21:49:25 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-z[37416]: 2024-09-11T21:49:25.872+0000 7ffb2f951040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T21:49:26.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:26.588+0000 7fce48ff9640 1 -- 172.21.15.1:0/3792608538 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mgrmap(e 14) ==== 92059+0+0 (secure 0 0 0) 0x7fce3403b460 con 0x7fce4c102e00 2024-09-11T21:49:26.605 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 11 21:49:26 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-y[36280]: 2024-09-11T21:49:26.336+0000 7fc8fcd84040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T21:49:26.605 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 11 21:49:26 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-y[36280]: 2024-09-11T21:49:26.455+0000 7fc8fcd84040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T21:49:26.605 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:26 smithi053 ceph-mon[34908]: Standby manager daemon z started 2024-09-11T21:49:26.605 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:26 smithi053 ceph-mon[34908]: from='mgr.? 172.21.15.161:0/1723501002' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-11T21:49:26.605 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:26 smithi053 ceph-mon[34908]: from='mgr.? 172.21.15.161:0/1723501002' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T21:49:26.605 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:26 smithi053 ceph-mon[34908]: from='mgr.? 172.21.15.161:0/1723501002' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-11T21:49:26.606 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:26 smithi053 ceph-mon[34908]: from='mgr.? 172.21.15.161:0/1723501002' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T21:49:26.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:26 smithi161 ceph-mon[35615]: Standby manager daemon z started 2024-09-11T21:49:26.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:26 smithi161 ceph-mon[35615]: from='mgr.? 172.21.15.161:0/1723501002' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-11T21:49:26.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:26 smithi161 ceph-mon[35615]: from='mgr.? 172.21.15.161:0/1723501002' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T21:49:26.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:26 smithi161 ceph-mon[35615]: from='mgr.? 172.21.15.161:0/1723501002' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-11T21:49:26.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:26 smithi161 ceph-mon[35615]: from='mgr.? 172.21.15.161:0/1723501002' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T21:49:26.893 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 11 21:49:26 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-y[36280]: 2024-09-11T21:49:26.604+0000 7fc8fcd84040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T21:49:26.894 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 11 21:49:26 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-y[36280]: 2024-09-11T21:49:26.667+0000 7fc8fcd84040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T21:49:26.894 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 11 21:49:26 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-y[36280]: 2024-09-11T21:49:26.822+0000 7fc8fcd84040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T21:49:26.894 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 11 21:49:26 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-y[36280]: 2024-09-11T21:49:26.892+0000 7fc8fcd84040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T21:49:26.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:26 smithi001 ceph-mon[30935]: Standby manager daemon z started 2024-09-11T21:49:26.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:26 smithi001 ceph-mon[30935]: from='mgr.? 172.21.15.161:0/1723501002' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-11T21:49:26.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:26 smithi001 ceph-mon[30935]: from='mgr.? 172.21.15.161:0/1723501002' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T21:49:26.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:26 smithi001 ceph-mon[30935]: from='mgr.? 172.21.15.161:0/1723501002' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-11T21:49:26.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:26 smithi001 ceph-mon[30935]: from='mgr.? 172.21.15.161:0/1723501002' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T21:49:27.187 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 11 21:49:26 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-y[36280]: 2024-09-11T21:49:26.954+0000 7fc8fcd84040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T21:49:27.188 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 11 21:49:27 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-y[36280]: 2024-09-11T21:49:27.119+0000 7fc8fcd84040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T21:49:27.188 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 11 21:49:27 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-y[36280]: 2024-09-11T21:49:27.186+0000 7fc8fcd84040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T21:49:27.465 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 11 21:49:27 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-y[36280]: 2024-09-11T21:49:27.255+0000 7fc8fcd84040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T21:49:27.465 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 11 21:49:27 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-y[36280]: 2024-09-11T21:49:27.341+0000 7fc8fcd84040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T21:49:27.465 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 11 21:49:27 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-y[36280]: 2024-09-11T21:49:27.402+0000 7fc8fcd84040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T21:49:27.465 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 11 21:49:27 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-y[36280]: 2024-09-11T21:49:27.463+0000 7fc8fcd84040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T21:49:27.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:27 smithi053 ceph-mon[34908]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:49:27.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:27 smithi053 ceph-mon[34908]: mgrmap e14: x(active, since 62s), standbys: z 2024-09-11T21:49:27.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:27 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-11T21:49:27.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:27 smithi161 ceph-mon[35615]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:49:27.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:27 smithi161 ceph-mon[35615]: mgrmap e14: x(active, since 62s), standbys: z 2024-09-11T21:49:27.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:27 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-11T21:49:27.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:27 smithi001 ceph-mon[30935]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:49:27.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:27 smithi001 ceph-mon[30935]: mgrmap e14: x(active, since 62s), standbys: z 2024-09-11T21:49:27.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:27 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-11T21:49:28.548 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 11 21:49:28 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-y[36280]: 2024-09-11T21:49:28.287+0000 7fc8fcd84040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T21:49:28.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:28.598+0000 7fce48ff9640 1 -- 172.21.15.1:0/3792608538 <== mon.0 v2:172.21.15.1:3300/0 8 ==== mgrmap(e 15) ==== 137776+0+0 (secure 0 0 0) 0x7fce3403bcb0 con 0x7fce4c102e00 2024-09-11T21:49:28.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:28 smithi001 ceph-mon[30935]: Standby manager daemon y started 2024-09-11T21:49:28.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:28 smithi001 ceph-mon[30935]: from='mgr.? 172.21.15.53:0/2518985733' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-11T21:49:28.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:28 smithi001 ceph-mon[30935]: from='mgr.? 172.21.15.53:0/2518985733' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T21:49:28.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:28 smithi001 ceph-mon[30935]: from='mgr.? 172.21.15.53:0/2518985733' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-11T21:49:28.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:28 smithi001 ceph-mon[30935]: from='mgr.? 172.21.15.53:0/2518985733' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T21:49:29.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:28 smithi053 ceph-mon[34908]: Standby manager daemon y started 2024-09-11T21:49:29.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:28 smithi053 ceph-mon[34908]: from='mgr.? 172.21.15.53:0/2518985733' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-11T21:49:29.048 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:28 smithi053 ceph-mon[34908]: from='mgr.? 172.21.15.53:0/2518985733' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T21:49:29.048 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:28 smithi053 ceph-mon[34908]: from='mgr.? 172.21.15.53:0/2518985733' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-11T21:49:29.048 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:28 smithi053 ceph-mon[34908]: from='mgr.? 172.21.15.53:0/2518985733' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T21:49:29.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:28 smithi161 ceph-mon[35615]: Standby manager daemon y started 2024-09-11T21:49:29.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:28 smithi161 ceph-mon[35615]: from='mgr.? 172.21.15.53:0/2518985733' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-11T21:49:29.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:28 smithi161 ceph-mon[35615]: from='mgr.? 172.21.15.53:0/2518985733' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T21:49:29.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:28 smithi161 ceph-mon[35615]: from='mgr.? 172.21.15.53:0/2518985733' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-11T21:49:29.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:28 smithi161 ceph-mon[35615]: from='mgr.? 172.21.15.53:0/2518985733' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T21:49:29.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:29 smithi001 ceph-mon[30935]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:49:29.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:29 smithi001 ceph-mon[30935]: mgrmap e15: x(active, since 64s), standbys: z, y 2024-09-11T21:49:29.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:29 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T21:49:30.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:29 smithi053 ceph-mon[34908]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:49:30.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:29 smithi053 ceph-mon[34908]: mgrmap e15: x(active, since 64s), standbys: z, y 2024-09-11T21:49:30.048 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:29 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T21:49:30.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:29 smithi161 ceph-mon[35615]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:49:30.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:29 smithi161 ceph-mon[35615]: mgrmap e15: x(active, since 64s), standbys: z, y 2024-09-11T21:49:30.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:29 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T21:49:31.923 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:31 smithi001 ceph-mon[30935]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:49:31.923 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:31 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:31.924 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:31 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:31.924 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:31 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:49:32.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:31 smithi053 ceph-mon[34908]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:49:32.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:31 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:32.048 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:31 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:32.048 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:31 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:49:32.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:31 smithi161 ceph-mon[35615]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:49:32.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:31 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:32.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:31 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:32.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:31 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:49:32.814 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:32 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:49:32.814 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:32 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:49:32.814 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:32 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:33.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:32 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:49:33.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:32 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:49:33.048 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:32 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:33.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:32 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:49:33.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:32 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:49:33.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:32 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:34.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:33 smithi053 ceph-mon[34908]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:49:34.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:33 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-11T21:49:34.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:33 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:49:34.311 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:33 smithi001 ceph-mon[30935]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:49:34.311 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:33 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-11T21:49:34.311 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:33 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:49:34.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:33 smithi161 ceph-mon[35615]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:49:34.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:33 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-11T21:49:34.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:33 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:49:35.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:34 smithi161 ceph-mon[35615]: Deploying daemon osd.0 on smithi001 2024-09-11T21:49:35.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:34 smithi053 ceph-mon[34908]: Deploying daemon osd.0 on smithi001 2024-09-11T21:49:35.322 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:34 smithi001 ceph-mon[30935]: Deploying daemon osd.0 on smithi001 2024-09-11T21:49:36.258 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:35 smithi161 ceph-mon[35615]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:49:36.300 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:35 smithi053 ceph-mon[34908]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:49:36.396 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:35 smithi001 ceph-mon[30935]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:49:38.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:37 smithi053 ceph-mon[34908]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:49:38.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:37 smithi161 ceph-mon[35615]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:49:38.399 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:37 smithi001 ceph-mon[30935]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:49:40.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:39 smithi053 ceph-mon[34908]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:49:40.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:39 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:49:40.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:39 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:40.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:39 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:40.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:39 smithi161 ceph-mon[35615]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:49:40.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:39 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:49:40.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:39 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:40.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:39 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:40.416 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:39 smithi001 ceph-mon[30935]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:49:40.416 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:39 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:49:40.416 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:39 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:40.417 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:39 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:42.249 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:41 smithi001 ceph-mon[30935]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:49:42.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:41 smithi053 ceph-mon[34908]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:49:42.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:41 smithi161 ceph-mon[35615]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:49:43.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:43 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:43.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:43 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:43.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:43 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:49:43.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:43 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:49:43.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:43 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:43.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:43 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:43.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:43 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:43.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:43 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:49:43.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:43 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:49:43.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:43 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:43.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:43 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:43.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:43 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:43.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:43 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:49:43.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:43 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:49:43.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:43 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:44.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:44.126+0000 7fce48ff9640 1 -- 172.21.15.1:0/3792608538 <== mgr.14150 v2:172.21.15.1:6800/3890350886 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fce4c062630 con 0x7fce38039290 2024-09-11T21:49:44.127 INFO:teuthology.orchestra.run.smithi001.stdout:Created osd(s) 0 on host 'smithi001' 2024-09-11T21:49:44.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:44.128+0000 7fce512db640 1 -- 172.21.15.1:0/3792608538 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fce38039290 msgr2=0x7fce3803b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:49:44.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:44.128+0000 7fce512db640 1 --2- 172.21.15.1:0/3792608538 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fce38039290 0x7fce3803b750 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fce4c102c60 tx=0x7fce3c005f90 comp rx=0 tx=0).stop 2024-09-11T21:49:44.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:44.128+0000 7fce512db640 1 -- 172.21.15.1:0/3792608538 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fce4c102e00 msgr2=0x7fce4c0745e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:49:44.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:44.128+0000 7fce512db640 1 --2- 172.21.15.1:0/3792608538 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fce4c102e00 0x7fce4c0745e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fce34009230 tx=0x7fce34031b60 comp rx=0 tx=0).stop 2024-09-11T21:49:44.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:44.128+0000 7fce512db640 1 -- 172.21.15.1:0/3792608538 shutdown_connections 2024-09-11T21:49:44.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:44.128+0000 7fce512db640 1 --2- 172.21.15.1:0/3792608538 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fce38039290 0x7fce3803b750 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:49:44.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:44.128+0000 7fce512db640 1 --2- 172.21.15.1:0/3792608538 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce4c1037c0 0x7fce4c074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:49:44.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:44.128+0000 7fce512db640 1 --2- 172.21.15.1:0/3792608538 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fce4c102e00 0x7fce4c0745e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:49:44.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:44.128+0000 7fce512db640 1 --2- 172.21.15.1:0/3792608538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce4c101c00 0x7fce4c075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:49:44.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:44.128+0000 7fce512db640 1 -- 172.21.15.1:0/3792608538 >> 172.21.15.1:0/3792608538 conn(0x7fce4c0fd390 msgr2=0x7fce4c106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:49:44.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:44.128+0000 7fce512db640 1 -- 172.21.15.1:0/3792608538 shutdown_connections 2024-09-11T21:49:44.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:49:44.128+0000 7fce512db640 1 -- 172.21.15.1:0/3792608538 wait complete. 2024-09-11T21:49:44.421 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:44 smithi001 ceph-mon[30935]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:49:44.421 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:44 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:49:44.422 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:44 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:44.422 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:44 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:44.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:44 smithi053 ceph-mon[34908]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:49:44.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:44 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:49:44.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:44 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:44.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:44 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:44.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:44 smithi161 ceph-mon[35615]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:49:44.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:44 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:49:44.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:44 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:44.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:44 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:44.722 DEBUG:teuthology.orchestra.run.smithi001:osd.0> sudo journalctl -f -n 0 -u ceph-3ffca83c-7087-11ef-bcea-c7b262605968@osd.0.service 2024-09-11T21:49:44.724 INFO:tasks.cephadm:Deploying osd.1 on smithi053 with /dev/vg_nvme/lv_4... 2024-09-11T21:49:44.724 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-11T21:49:44.969 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T21:49:45.862 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 11 21:49:45 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-0[42460]: 2024-09-11T21:49:45.532+0000 7f9c0a4f5640 -1 osd.0 0 log_to_monitors true 2024-09-11T21:49:46.335 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:46 smithi053 ceph-mon[34908]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:49:46.335 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:46 smithi053 ceph-mon[34908]: from='osd.0 [v2:172.21.15.1:6802/4012142023,v1:172.21.15.1:6803/4012142023]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T21:49:46.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:46 smithi001 ceph-mon[30935]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:49:46.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:46 smithi001 ceph-mon[30935]: from='osd.0 [v2:172.21.15.1:6802/4012142023,v1:172.21.15.1:6803/4012142023]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T21:49:46.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:46 smithi161 ceph-mon[35615]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:49:46.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:46 smithi161 ceph-mon[35615]: from='osd.0 [v2:172.21.15.1:6802/4012142023,v1:172.21.15.1:6803/4012142023]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T21:49:47.371 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:47 smithi053 ceph-mon[34908]: from='osd.0 [v2:172.21.15.1:6802/4012142023,v1:172.21.15.1:6803/4012142023]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-11T21:49:47.372 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:47 smithi053 ceph-mon[34908]: osdmap e6: 1 total, 0 up, 1 in 2024-09-11T21:49:47.372 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:47 smithi053 ceph-mon[34908]: from='osd.0 [v2:172.21.15.1:6802/4012142023,v1:172.21.15.1:6803/4012142023]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-09-11T21:49:47.372 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:47 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T21:49:47.372 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:47 smithi053 ceph-mon[34908]: from='osd.0 [v2:172.21.15.1:6802/4012142023,v1:172.21.15.1:6803/4012142023]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-09-11T21:49:47.375 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-11T21:49:47.401 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi053:vg_nvme/lv_4 2024-09-11T21:49:47.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:47 smithi001 ceph-mon[30935]: from='osd.0 [v2:172.21.15.1:6802/4012142023,v1:172.21.15.1:6803/4012142023]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-11T21:49:47.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:47 smithi001 ceph-mon[30935]: osdmap e6: 1 total, 0 up, 1 in 2024-09-11T21:49:47.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:47 smithi001 ceph-mon[30935]: from='osd.0 [v2:172.21.15.1:6802/4012142023,v1:172.21.15.1:6803/4012142023]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-09-11T21:49:47.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:47 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T21:49:47.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:47 smithi001 ceph-mon[30935]: from='osd.0 [v2:172.21.15.1:6802/4012142023,v1:172.21.15.1:6803/4012142023]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-09-11T21:49:47.444 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 11 21:49:47 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-0[42460]: 2024-09-11T21:49:47.293+0000 7f9c06478640 -1 osd.0 0 waiting for initial osdmap 2024-09-11T21:49:47.444 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 11 21:49:47 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-0[42460]: 2024-09-11T21:49:47.297+0000 7f9c022a0640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T21:49:47.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:47 smithi161 ceph-mon[35615]: from='osd.0 [v2:172.21.15.1:6802/4012142023,v1:172.21.15.1:6803/4012142023]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-11T21:49:47.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:47 smithi161 ceph-mon[35615]: osdmap e6: 1 total, 0 up, 1 in 2024-09-11T21:49:47.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:47 smithi161 ceph-mon[35615]: from='osd.0 [v2:172.21.15.1:6802/4012142023,v1:172.21.15.1:6803/4012142023]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-09-11T21:49:47.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:47 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T21:49:47.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:47 smithi161 ceph-mon[35615]: from='osd.0 [v2:172.21.15.1:6802/4012142023,v1:172.21.15.1:6803/4012142023]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-09-11T21:49:47.652 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T21:49:48.100 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:49:48.093+0000 7efe702ed640 1 -- 172.21.15.53:0/2784743179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe68100ad0 msgr2=0x7efe68100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:49:48.100 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:49:48.093+0000 7efe702ed640 1 --2- 172.21.15.53:0/2784743179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe68100ad0 0x7efe68100ed0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7efe58009ca0 tx=0x7efe5802a8c0 comp rx=0 tx=0).stop 2024-09-11T21:49:48.100 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:49:48.093+0000 7efe702ed640 1 -- 172.21.15.53:0/2784743179 shutdown_connections 2024-09-11T21:49:48.100 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:49:48.093+0000 7efe702ed640 1 --2- 172.21.15.53:0/2784743179 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe68102690 0x7efe68108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:49:48.100 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:49:48.093+0000 7efe702ed640 1 --2- 172.21.15.53:0/2784743179 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe68101cd0 0x7efe68102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:49:48.100 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:49:48.093+0000 7efe702ed640 1 --2- 172.21.15.53:0/2784743179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe68100ad0 0x7efe68100ed0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:49:48.100 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:49:48.093+0000 7efe702ed640 1 -- 172.21.15.53:0/2784743179 >> 172.21.15.53:0/2784743179 conn(0x7efe680fc280 msgr2=0x7efe680fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:49:48.100 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:49:48.093+0000 7efe702ed640 1 -- 172.21.15.53:0/2784743179 shutdown_connections 2024-09-11T21:49:48.100 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:49:48.093+0000 7efe702ed640 1 -- 172.21.15.53:0/2784743179 wait complete. 2024-09-11T21:49:48.101 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:49:48.094+0000 7efe702ed640 1 Processor -- start 2024-09-11T21:49:48.101 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:49:48.094+0000 7efe702ed640 1 -- start start 2024-09-11T21:49:48.101 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:49:48.094+0000 7efe702ed640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe68100ad0 0x7efe68104a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:49:48.101 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:49:48.094+0000 7efe702ed640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe68101cd0 0x7efe681030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:49:48.101 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:49:48.094+0000 7efe702ed640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe68102690 0x7efe681035f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:49:48.101 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:49:48.094+0000 7efe702ed640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7efe6810c780 con 0x7efe68100ad0 2024-09-11T21:49:48.101 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:49:48.094+0000 7efe702ed640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7efe6810c600 con 0x7efe68102690 2024-09-11T21:49:48.101 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:49:48.094+0000 7efe702ed640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7efe6810c900 con 0x7efe68101cd0 2024-09-11T21:49:48.101 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:49:48.094+0000 7efe6e88a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe68102690 0x7efe681035f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:49:48.101 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:49:48.094+0000 7efe6e089640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe68100ad0 0x7efe68104a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:49:48.101 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:49:48.094+0000 7efe6e88a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe68102690 0x7efe681035f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:50994/0 (socket says 172.21.15.53:50994) 2024-09-11T21:49:48.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:49:48.094+0000 7efe6e88a640 1 -- 172.21.15.53:0/2818953200 learned_addr learned my addr 172.21.15.53:0/2818953200 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-11T21:49:48.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:49:48.094+0000 7efe6e089640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe68100ad0 0x7efe68104a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.53:55014/0 (socket says 172.21.15.53:55014) 2024-09-11T21:49:48.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:49:48.094+0000 7efe6d888640 1 --2- 172.21.15.53:0/2818953200 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe68101cd0 0x7efe681030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:49:48.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:49:48.095+0000 7efe6e88a640 1 -- 172.21.15.53:0/2818953200 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe68101cd0 msgr2=0x7efe681030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:49:48.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:49:48.095+0000 7efe6e88a640 1 --2- 172.21.15.53:0/2818953200 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe68101cd0 0x7efe681030b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:49:48.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:49:48.095+0000 7efe6e88a640 1 -- 172.21.15.53:0/2818953200 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe68100ad0 msgr2=0x7efe68104a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:49:48.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:49:48.095+0000 7efe6e88a640 1 --2- 172.21.15.53:0/2818953200 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe68100ad0 0x7efe68104a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:49:48.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:49:48.095+0000 7efe6e88a640 1 -- 172.21.15.53:0/2818953200 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe68103eb0 con 0x7efe68102690 2024-09-11T21:49:48.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:49:48.095+0000 7efe6e88a640 1 --2- 172.21.15.53:0/2818953200 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe68102690 0x7efe681035f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7efe6000e970 tx=0x7efe6000ed70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:49:48.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:49:48.095+0000 7efe677fe640 1 -- 172.21.15.53:0/2818953200 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6000ccc0 con 0x7efe68102690 2024-09-11T21:49:48.102 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:49:48.095+0000 7efe677fe640 1 -- 172.21.15.53:0/2818953200 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe6000ce60 con 0x7efe68102690 2024-09-11T21:49:48.103 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:49:48.095+0000 7efe702ed640 1 -- 172.21.15.53:0/2818953200 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe681a20d0 con 0x7efe68102690 2024-09-11T21:49:48.103 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:49:48.095+0000 7efe677fe640 1 -- 172.21.15.53:0/2818953200 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe60010490 con 0x7efe68102690 2024-09-11T21:49:48.103 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:49:48.095+0000 7efe702ed640 1 -- 172.21.15.53:0/2818953200 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe681a2590 con 0x7efe68102690 2024-09-11T21:49:48.103 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:49:48.096+0000 7efe702ed640 1 -- 172.21.15.53:0/2818953200 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe681a2820 con 0x7efe68102690 2024-09-11T21:49:48.103 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:49:48.097+0000 7efe677fe640 1 -- 172.21.15.53:0/2818953200 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 15) ==== 137776+0+0 (secure 0 0 0) 0x7efe60005020 con 0x7efe68102690 2024-09-11T21:49:48.103 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:49:48.098+0000 7efe677fe640 1 --2- 172.21.15.53:0/2818953200 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7efe500a2af0 0x7efe500a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:49:48.103 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:49:48.098+0000 7efe677fe640 1 -- 172.21.15.53:0/2818953200 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1487+0+0 (secure 0 0 0) 0x7efe600cd970 con 0x7efe68102690 2024-09-11T21:49:48.103 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:49:48.098+0000 7efe6e089640 1 --2- 172.21.15.53:0/2818953200 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7efe500a2af0 0x7efe500a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:49:48.103 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:49:48.098+0000 7efe6e089640 1 --2- 172.21.15.53:0/2818953200 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7efe500a2af0 0x7efe500a4fb0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efe58000f80 tx=0x7efe58005f30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:49:48.103 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:49:48.103+0000 7efe677fe640 1 -- 172.21.15.53:0/2818953200 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efe6009c460 con 0x7efe68102690 2024-09-11T21:49:48.290 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:49:48.289+0000 7efe702ed640 1 -- 172.21.15.53:0/2818953200 --> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7efe681a29f0 con 0x7efe500a2af0 2024-09-11T21:49:48.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:48 smithi053 ceph-mon[34908]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:49:48.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:48 smithi053 ceph-mon[34908]: osdmap e7: 1 total, 0 up, 1 in 2024-09-11T21:49:48.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:48 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T21:49:48.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:48 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T21:49:48.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:48 smithi053 ceph-mon[34908]: Detected new or changed devices on smithi001 2024-09-11T21:49:48.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:48 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:48.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:48 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:48.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:48 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:49:48.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:48 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:48.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:48 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:49:48.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:48 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:49:48.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:48 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:48.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:48 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T21:49:48.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:48 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T21:49:48.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:48 smithi161 ceph-mon[35615]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:49:48.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:48 smithi161 ceph-mon[35615]: osdmap e7: 1 total, 0 up, 1 in 2024-09-11T21:49:48.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:48 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T21:49:48.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:48 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T21:49:48.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:48 smithi161 ceph-mon[35615]: Detected new or changed devices on smithi001 2024-09-11T21:49:48.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:48 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:48.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:48 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:48.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:48 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:49:48.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:48 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:48.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:48 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:49:48.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:48 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:49:48.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:48 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:48.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:48 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T21:49:48.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:48 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T21:49:48.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:48 smithi001 ceph-mon[30935]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:49:48.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:48 smithi001 ceph-mon[30935]: osdmap e7: 1 total, 0 up, 1 in 2024-09-11T21:49:48.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:48 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T21:49:48.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:48 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T21:49:48.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:48 smithi001 ceph-mon[30935]: Detected new or changed devices on smithi001 2024-09-11T21:49:48.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:48 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:48.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:48 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:48.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:48 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:49:48.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:48 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:48.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:48 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:49:48.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:48 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:49:48.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:48 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:49:48.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:48 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T21:49:48.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:48 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T21:49:49.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:49 smithi161 ceph-mon[35615]: purged_snaps scrub starts 2024-09-11T21:49:49.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:49 smithi161 ceph-mon[35615]: purged_snaps scrub ok 2024-09-11T21:49:49.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:49 smithi161 ceph-mon[35615]: Adjusting osd_memory_target on smithi001 to 17092M 2024-09-11T21:49:49.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:49 smithi161 ceph-mon[35615]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:49:49.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:49 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T21:49:49.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:49 smithi161 ceph-mon[35615]: osd.0 [v2:172.21.15.1:6802/4012142023,v1:172.21.15.1:6803/4012142023] boot 2024-09-11T21:49:49.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:49 smithi161 ceph-mon[35615]: osdmap e8: 1 total, 1 up, 1 in 2024-09-11T21:49:49.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:49 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T21:49:49.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:49 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:49:49.640 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:49 smithi053 ceph-mon[34908]: purged_snaps scrub starts 2024-09-11T21:49:49.640 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:49 smithi053 ceph-mon[34908]: purged_snaps scrub ok 2024-09-11T21:49:49.640 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:49 smithi053 ceph-mon[34908]: Adjusting osd_memory_target on smithi001 to 17092M 2024-09-11T21:49:49.640 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:49 smithi053 ceph-mon[34908]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:49:49.640 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:49 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T21:49:49.641 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:49 smithi053 ceph-mon[34908]: osd.0 [v2:172.21.15.1:6802/4012142023,v1:172.21.15.1:6803/4012142023] boot 2024-09-11T21:49:49.641 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:49 smithi053 ceph-mon[34908]: osdmap e8: 1 total, 1 up, 1 in 2024-09-11T21:49:49.641 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:49 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T21:49:49.641 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:49 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:49:49.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:49 smithi001 ceph-mon[30935]: purged_snaps scrub starts 2024-09-11T21:49:49.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:49 smithi001 ceph-mon[30935]: purged_snaps scrub ok 2024-09-11T21:49:49.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:49 smithi001 ceph-mon[30935]: Adjusting osd_memory_target on smithi001 to 17092M 2024-09-11T21:49:49.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:49 smithi001 ceph-mon[30935]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:49:49.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:49 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T21:49:49.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:49 smithi001 ceph-mon[30935]: osd.0 [v2:172.21.15.1:6802/4012142023,v1:172.21.15.1:6803/4012142023] boot 2024-09-11T21:49:49.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:49 smithi001 ceph-mon[30935]: osdmap e8: 1 total, 1 up, 1 in 2024-09-11T21:49:49.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:49 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T21:49:49.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:49 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:49:50.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:50 smithi053 ceph-mon[34908]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:49:50.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:50 smithi053 ceph-mon[34908]: osdmap e9: 1 total, 1 up, 1 in 2024-09-11T21:49:50.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:50 smithi161 ceph-mon[35615]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:49:50.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:50 smithi161 ceph-mon[35615]: osdmap e9: 1 total, 1 up, 1 in 2024-09-11T21:49:50.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:50 smithi001 ceph-mon[30935]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T21:49:50.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:50 smithi001 ceph-mon[30935]: osdmap e9: 1 total, 1 up, 1 in 2024-09-11T21:49:51.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:51 smithi053 ceph-mon[34908]: from='client.? 172.21.15.53:0/946795180' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "996c9f0c-c5a4-43b8-b8e0-d712f27c5dfa"}]: dispatch 2024-09-11T21:49:51.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:51 smithi053 ceph-mon[34908]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "996c9f0c-c5a4-43b8-b8e0-d712f27c5dfa"}]: dispatch 2024-09-11T21:49:51.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:51 smithi053 ceph-mon[34908]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "996c9f0c-c5a4-43b8-b8e0-d712f27c5dfa"}]': finished 2024-09-11T21:49:51.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:51 smithi053 ceph-mon[34908]: osdmap e10: 2 total, 1 up, 2 in 2024-09-11T21:49:51.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:51 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T21:49:51.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:51 smithi053 ceph-mon[34908]: from='client.? 172.21.15.53:0/659005926' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T21:49:51.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:51 smithi161 ceph-mon[35615]: from='client.? 172.21.15.53:0/946795180' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "996c9f0c-c5a4-43b8-b8e0-d712f27c5dfa"}]: dispatch 2024-09-11T21:49:51.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:51 smithi161 ceph-mon[35615]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "996c9f0c-c5a4-43b8-b8e0-d712f27c5dfa"}]: dispatch 2024-09-11T21:49:51.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:51 smithi161 ceph-mon[35615]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "996c9f0c-c5a4-43b8-b8e0-d712f27c5dfa"}]': finished 2024-09-11T21:49:51.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:51 smithi161 ceph-mon[35615]: osdmap e10: 2 total, 1 up, 2 in 2024-09-11T21:49:51.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:51 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T21:49:51.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:51 smithi161 ceph-mon[35615]: from='client.? 172.21.15.53:0/659005926' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T21:49:51.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:51 smithi001 ceph-mon[30935]: from='client.? 172.21.15.53:0/946795180' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "996c9f0c-c5a4-43b8-b8e0-d712f27c5dfa"}]: dispatch 2024-09-11T21:49:51.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:51 smithi001 ceph-mon[30935]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "996c9f0c-c5a4-43b8-b8e0-d712f27c5dfa"}]: dispatch 2024-09-11T21:49:51.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:51 smithi001 ceph-mon[30935]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "996c9f0c-c5a4-43b8-b8e0-d712f27c5dfa"}]': finished 2024-09-11T21:49:51.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:51 smithi001 ceph-mon[30935]: osdmap e10: 2 total, 1 up, 2 in 2024-09-11T21:49:51.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:51 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T21:49:51.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:51 smithi001 ceph-mon[30935]: from='client.? 172.21.15.53:0/659005926' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T21:49:52.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:52 smithi161 ceph-mon[35615]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T21:49:52.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:52 smithi001 ceph-mon[30935]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T21:49:52.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:52 smithi053 ceph-mon[34908]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T21:49:54.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:54 smithi161 ceph-mon[35615]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T21:49:54.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:54 smithi001 ceph-mon[30935]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T21:49:54.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:54 smithi053 ceph-mon[34908]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T21:49:56.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:56 smithi161 ceph-mon[35615]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T21:49:56.682 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:56 smithi053 ceph-mon[34908]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T21:49:56.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:56 smithi001 ceph-mon[30935]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T21:49:58.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:58 smithi161 ceph-mon[35615]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T21:49:58.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:58 smithi001 ceph-mon[30935]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T21:49:58.749 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:58 smithi053 ceph-mon[34908]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T21:49:59.546 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:59 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-11T21:49:59.546 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:49:59 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:49:59.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:59 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-11T21:49:59.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:49:59 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:49:59.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:59 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-11T21:49:59.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:49:59 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:50:00.362 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:00 smithi053 ceph-mon[34908]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T21:50:00.362 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:00 smithi053 ceph-mon[34908]: Deploying daemon osd.1 on smithi053 2024-09-11T21:50:00.362 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:00 smithi053 ceph-mon[34908]: overall HEALTH_OK 2024-09-11T21:50:00.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:00 smithi161 ceph-mon[35615]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T21:50:00.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:00 smithi161 ceph-mon[35615]: Deploying daemon osd.1 on smithi053 2024-09-11T21:50:00.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:00 smithi161 ceph-mon[35615]: overall HEALTH_OK 2024-09-11T21:50:00.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:00 smithi001 ceph-mon[30935]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T21:50:00.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:00 smithi001 ceph-mon[30935]: Deploying daemon osd.1 on smithi053 2024-09-11T21:50:00.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:00 smithi001 ceph-mon[30935]: overall HEALTH_OK 2024-09-11T21:50:02.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:02 smithi161 ceph-mon[35615]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T21:50:02.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:02 smithi001 ceph-mon[30935]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T21:50:02.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:02 smithi053 ceph-mon[34908]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T21:50:04.506 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:04 smithi053 ceph-mon[34908]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T21:50:04.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:04 smithi161 ceph-mon[35615]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T21:50:04.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:04 smithi001 ceph-mon[30935]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T21:50:05.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:05 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:50:05.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:05 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:05.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:05 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:05.593 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:05 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:50:05.594 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:05 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:05.594 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:05 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:05.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:05 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:50:05.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:05 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:05.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:05 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:06.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:06 smithi161 ceph-mon[35615]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T21:50:06.617 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:06 smithi053 ceph-mon[34908]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T21:50:06.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:06 smithi001 ceph-mon[30935]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T21:50:08.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:08 smithi161 ceph-mon[35615]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T21:50:08.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:08 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:08.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:08 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:08.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:08 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:50:08.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:08 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:50:08.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:08 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:08.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:08 smithi001 ceph-mon[30935]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T21:50:08.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:08 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:08.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:08 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:08.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:08 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:50:08.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:08 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:50:08.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:08 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:08.755 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:08 smithi053 ceph-mon[34908]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T21:50:08.756 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:08 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:08.756 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:08 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:08.756 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:08 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:50:08.756 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:08 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:50:08.756 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:08 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:09.371 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:50:09.370+0000 7efe677fe640 1 -- 172.21.15.53:0/2818953200 <== mgr.14150 v2:172.21.15.1:6800/3890350886 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7efe681a29f0 con 0x7efe500a2af0 2024-09-11T21:50:09.372 INFO:teuthology.orchestra.run.smithi053.stdout:Created osd(s) 1 on host 'smithi053' 2024-09-11T21:50:09.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:50:09.373+0000 7efe702ed640 1 -- 172.21.15.53:0/2818953200 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7efe500a2af0 msgr2=0x7efe500a4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:50:09.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:50:09.373+0000 7efe702ed640 1 --2- 172.21.15.53:0/2818953200 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7efe500a2af0 0x7efe500a4fb0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efe58000f80 tx=0x7efe58005f30 comp rx=0 tx=0).stop 2024-09-11T21:50:09.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:50:09.373+0000 7efe702ed640 1 -- 172.21.15.53:0/2818953200 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe68102690 msgr2=0x7efe681035f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:50:09.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:50:09.373+0000 7efe702ed640 1 --2- 172.21.15.53:0/2818953200 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe68102690 0x7efe681035f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7efe6000e970 tx=0x7efe6000ed70 comp rx=0 tx=0).stop 2024-09-11T21:50:09.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:50:09.373+0000 7efe702ed640 1 -- 172.21.15.53:0/2818953200 shutdown_connections 2024-09-11T21:50:09.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:50:09.373+0000 7efe702ed640 1 --2- 172.21.15.53:0/2818953200 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7efe500a2af0 0x7efe500a4fb0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:50:09.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:50:09.373+0000 7efe702ed640 1 --2- 172.21.15.53:0/2818953200 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe68102690 0x7efe681035f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:50:09.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:50:09.373+0000 7efe702ed640 1 --2- 172.21.15.53:0/2818953200 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe68101cd0 0x7efe681030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:50:09.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:50:09.373+0000 7efe702ed640 1 --2- 172.21.15.53:0/2818953200 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe68100ad0 0x7efe68104a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:50:09.374 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:50:09.373+0000 7efe702ed640 1 -- 172.21.15.53:0/2818953200 >> 172.21.15.53:0/2818953200 conn(0x7efe680fc280 msgr2=0x7efe6806a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:50:09.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:50:09.373+0000 7efe702ed640 1 -- 172.21.15.53:0/2818953200 shutdown_connections 2024-09-11T21:50:09.375 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:50:09.373+0000 7efe702ed640 1 -- 172.21.15.53:0/2818953200 wait complete. 2024-09-11T21:50:10.014 DEBUG:teuthology.orchestra.run.smithi053:osd.1> sudo journalctl -f -n 0 -u ceph-3ffca83c-7087-11ef-bcea-c7b262605968@osd.1.service 2024-09-11T21:50:10.017 INFO:tasks.cephadm:Deploying osd.2 on smithi161 with /dev/vg_nvme/lv_4... 2024-09-11T21:50:10.017 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-11T21:50:10.273 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T21:50:10.582 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:10 smithi161 ceph-mon[35615]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T21:50:10.582 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:10 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:50:10.582 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:10 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:10.582 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:10 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:10.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:10 smithi001 ceph-mon[30935]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T21:50:10.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:10 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:50:10.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:10 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:10.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:10 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:10.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:10 smithi053 ceph-mon[34908]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T21:50:10.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:10 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:50:10.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:10 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:10.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:10 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:11.233 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 11 21:50:10 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-1[39386]: 2024-09-11T21:50:10.938+0000 7f6c731e6640 -1 osd.1 0 log_to_monitors true 2024-09-11T21:50:11.498 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:11 smithi053 ceph-mon[34908]: from='osd.1 [v2:172.21.15.53:6800/3039391924,v1:172.21.15.53:6801/3039391924]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T21:50:11.498 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:11 smithi053 ceph-mon[34908]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T21:50:11.572 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:11 smithi161 ceph-mon[35615]: from='osd.1 [v2:172.21.15.53:6800/3039391924,v1:172.21.15.53:6801/3039391924]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T21:50:11.572 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:11 smithi161 ceph-mon[35615]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T21:50:11.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:11 smithi001 ceph-mon[30935]: from='osd.1 [v2:172.21.15.53:6800/3039391924,v1:172.21.15.53:6801/3039391924]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T21:50:11.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:11 smithi001 ceph-mon[30935]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T21:50:12.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:12 smithi053 ceph-mon[34908]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T21:50:12.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:12 smithi053 ceph-mon[34908]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-11T21:50:12.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:12 smithi053 ceph-mon[34908]: from='osd.1 [v2:172.21.15.53:6800/3039391924,v1:172.21.15.53:6801/3039391924]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-09-11T21:50:12.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:12 smithi053 ceph-mon[34908]: osdmap e11: 2 total, 1 up, 2 in 2024-09-11T21:50:12.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:12 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T21:50:12.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:12 smithi053 ceph-mon[34908]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-09-11T21:50:12.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:12 smithi161 ceph-mon[35615]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T21:50:12.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:12 smithi161 ceph-mon[35615]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-11T21:50:12.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:12 smithi161 ceph-mon[35615]: from='osd.1 [v2:172.21.15.53:6800/3039391924,v1:172.21.15.53:6801/3039391924]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-09-11T21:50:12.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:12 smithi161 ceph-mon[35615]: osdmap e11: 2 total, 1 up, 2 in 2024-09-11T21:50:12.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:12 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T21:50:12.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:12 smithi161 ceph-mon[35615]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-09-11T21:50:12.595 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-11T21:50:12.622 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi161:vg_nvme/lv_4 2024-09-11T21:50:12.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:12 smithi001 ceph-mon[30935]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T21:50:12.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:12 smithi001 ceph-mon[30935]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-11T21:50:12.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:12 smithi001 ceph-mon[30935]: from='osd.1 [v2:172.21.15.53:6800/3039391924,v1:172.21.15.53:6801/3039391924]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-09-11T21:50:12.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:12 smithi001 ceph-mon[30935]: osdmap e11: 2 total, 1 up, 2 in 2024-09-11T21:50:12.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:12 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T21:50:12.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:12 smithi001 ceph-mon[30935]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-09-11T21:50:12.872 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T21:50:12.879 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 11 21:50:12 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-1[39386]: 2024-09-11T21:50:12.737+0000 7f6c6f169640 -1 osd.1 0 waiting for initial osdmap 2024-09-11T21:50:12.879 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 11 21:50:12 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-1[39386]: 2024-09-11T21:50:12.740+0000 7f6c66f89640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T21:50:13.394 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:50:13.362+0000 7f84fcf24640 1 -- 172.21.15.161:0/2931590429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84f8101cd0 msgr2=0x7f84f8102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:50:13.394 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:50:13.362+0000 7f84fcf24640 1 --2- 172.21.15.161:0/2931590429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84f8101cd0 0x7f84f8102150 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f84e8009900 tx=0x7f84e802a410 comp rx=0 tx=0).stop 2024-09-11T21:50:13.394 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:50:13.362+0000 7f84fcf24640 1 -- 172.21.15.161:0/2931590429 shutdown_connections 2024-09-11T21:50:13.394 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:50:13.362+0000 7f84fcf24640 1 --2- 172.21.15.161:0/2931590429 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84f8102690 0x7f84f8108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:50:13.394 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:50:13.362+0000 7f84fcf24640 1 --2- 172.21.15.161:0/2931590429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84f8101cd0 0x7f84f8102150 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:50:13.394 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:50:13.362+0000 7f84fcf24640 1 --2- 172.21.15.161:0/2931590429 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84f8100ad0 0x7f84f8100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:50:13.394 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:50:13.362+0000 7f84fcf24640 1 -- 172.21.15.161:0/2931590429 >> 172.21.15.161:0/2931590429 conn(0x7f84f80fc280 msgr2=0x7f84f80fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:50:13.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:50:13.362+0000 7f84fcf24640 1 -- 172.21.15.161:0/2931590429 shutdown_connections 2024-09-11T21:50:13.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:50:13.362+0000 7f84fcf24640 1 -- 172.21.15.161:0/2931590429 wait complete. 2024-09-11T21:50:13.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:50:13.363+0000 7f84fcf24640 1 Processor -- start 2024-09-11T21:50:13.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:50:13.363+0000 7f84fcf24640 1 -- start start 2024-09-11T21:50:13.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:50:13.363+0000 7f84fcf24640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84f8100ad0 0x7f84f8196e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:50:13.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:50:13.363+0000 7f84fcf24640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84f8101cd0 0x7f84f8197370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:50:13.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:50:13.363+0000 7f84fcf24640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84f8102690 0x7f84f819e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:50:13.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:50:13.363+0000 7f84fcf24640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f84f810c7b0 con 0x7f84f8102690 2024-09-11T21:50:13.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:50:13.363+0000 7f84fcf24640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f84f810c630 con 0x7f84f8100ad0 2024-09-11T21:50:13.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:50:13.363+0000 7f84fcf24640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f84f810c930 con 0x7f84f8101cd0 2024-09-11T21:50:13.395 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:50:13.363+0000 7f84f5d9b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84f8101cd0 0x7f84f8197370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:50:13.396 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:50:13.363+0000 7f84f5d9b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84f8101cd0 0x7f84f8197370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55208/0 (socket says 172.21.15.161:55208) 2024-09-11T21:50:13.396 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:50:13.363+0000 7f84f5d9b640 1 -- 172.21.15.161:0/3347359056 learned_addr learned my addr 172.21.15.161:0/3347359056 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-09-11T21:50:13.396 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:50:13.363+0000 7f84f659c640 1 --2- 172.21.15.161:0/3347359056 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84f8100ad0 0x7f84f8196e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:50:13.396 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:50:13.363+0000 7f84f6d9d640 1 --2- 172.21.15.161:0/3347359056 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84f8102690 0x7f84f819e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:50:13.396 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:50:13.363+0000 7f84f5d9b640 1 -- 172.21.15.161:0/3347359056 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84f8100ad0 msgr2=0x7f84f8196e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:50:13.396 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:50:13.363+0000 7f84f5d9b640 1 --2- 172.21.15.161:0/3347359056 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84f8100ad0 0x7f84f8196e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:50:13.396 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:50:13.363+0000 7f84f5d9b640 1 -- 172.21.15.161:0/3347359056 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84f8102690 msgr2=0x7f84f819e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:50:13.396 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:50:13.363+0000 7f84f5d9b640 1 --2- 172.21.15.161:0/3347359056 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84f8102690 0x7f84f819e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:50:13.396 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:50:13.363+0000 7f84f5d9b640 1 -- 172.21.15.161:0/3347359056 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84f819ea60 con 0x7f84f8101cd0 2024-09-11T21:50:13.396 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:50:13.364+0000 7f84f5d9b640 1 --2- 172.21.15.161:0/3347359056 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84f8101cd0 0x7f84f8197370 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f84e8009230 tx=0x7f84e8004360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:50:13.396 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:50:13.364+0000 7f84ef7fe640 1 -- 172.21.15.161:0/3347359056 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e803f030 con 0x7f84f8101cd0 2024-09-11T21:50:13.397 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:50:13.364+0000 7f84ef7fe640 1 -- 172.21.15.161:0/3347359056 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84e8003ce0 con 0x7f84f8101cd0 2024-09-11T21:50:13.397 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:50:13.364+0000 7f84fcf24640 1 -- 172.21.15.161:0/3347359056 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84f819ecf0 con 0x7f84f8101cd0 2024-09-11T21:50:13.397 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:50:13.364+0000 7f84ef7fe640 1 -- 172.21.15.161:0/3347359056 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e8035070 con 0x7f84f8101cd0 2024-09-11T21:50:13.397 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:50:13.364+0000 7f84fcf24640 1 -- 172.21.15.161:0/3347359056 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84f819f1c0 con 0x7f84f8101cd0 2024-09-11T21:50:13.397 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:50:13.365+0000 7f84fcf24640 1 -- 172.21.15.161:0/3347359056 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84f8107e70 con 0x7f84f8101cd0 2024-09-11T21:50:13.397 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:50:13.366+0000 7f84ef7fe640 1 -- 172.21.15.161:0/3347359056 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 15) ==== 137776+0+0 (secure 0 0 0) 0x7f84e8042030 con 0x7f84f8101cd0 2024-09-11T21:50:13.397 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:50:13.367+0000 7f84ef7fe640 1 --2- 172.21.15.161:0/3347359056 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f84e40a2ba0 0x7f84e40a5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:50:13.397 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:50:13.367+0000 7f84ef7fe640 1 -- 172.21.15.161:0/3347359056 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2029+0+0 (secure 0 0 0) 0x7f84e80edbe0 con 0x7f84f8101cd0 2024-09-11T21:50:13.397 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:50:13.367+0000 7f84f659c640 1 --2- 172.21.15.161:0/3347359056 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f84e40a2ba0 0x7f84e40a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:50:13.397 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:50:13.367+0000 7f84f659c640 1 --2- 172.21.15.161:0/3347359056 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f84e40a2ba0 0x7f84e40a5060 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f84e0005fd0 tx=0x7f84e000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:50:13.397 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:50:13.371+0000 7f84ef7fe640 1 -- 172.21.15.161:0/3347359056 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f84e80bbe60 con 0x7f84f8101cd0 2024-09-11T21:50:13.546 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:50:13.545+0000 7f84fcf24640 1 -- 172.21.15.161:0/3347359056 --> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f84f8062630 con 0x7f84e40a2ba0 2024-09-11T21:50:13.546 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:13 smithi161 ceph-mon[35615]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-09-11T21:50:13.546 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:13 smithi161 ceph-mon[35615]: osdmap e12: 2 total, 1 up, 2 in 2024-09-11T21:50:13.546 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:13 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T21:50:13.546 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:13 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T21:50:13.546 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:13 smithi161 ceph-mon[35615]: from='osd.1 ' entity='osd.1' 2024-09-11T21:50:13.547 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:13 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:13.547 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:13 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:13.547 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:13 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:50:13.547 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:13 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:13.547 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:13 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:50:13.547 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:13 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:50:13.547 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:13 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:13.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:13 smithi001 ceph-mon[30935]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-09-11T21:50:13.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:13 smithi001 ceph-mon[30935]: osdmap e12: 2 total, 1 up, 2 in 2024-09-11T21:50:13.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:13 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T21:50:13.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:13 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T21:50:13.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:13 smithi001 ceph-mon[30935]: from='osd.1 ' entity='osd.1' 2024-09-11T21:50:13.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:13 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:13.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:13 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:13.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:13 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:50:13.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:13 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:13.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:13 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:50:13.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:13 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:50:13.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:13 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:13.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:13 smithi053 ceph-mon[34908]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-09-11T21:50:13.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:13 smithi053 ceph-mon[34908]: osdmap e12: 2 total, 1 up, 2 in 2024-09-11T21:50:13.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:13 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T21:50:13.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:13 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T21:50:13.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:13 smithi053 ceph-mon[34908]: from='osd.1 ' entity='osd.1' 2024-09-11T21:50:13.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:13 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:13.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:13 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:13.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:13 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:50:13.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:13 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:13.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:13 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:50:13.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:13 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:50:13.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:13 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:14.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:14 smithi161 ceph-mon[35615]: purged_snaps scrub starts 2024-09-11T21:50:14.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:14 smithi161 ceph-mon[35615]: purged_snaps scrub ok 2024-09-11T21:50:14.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:14 smithi161 ceph-mon[35615]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T21:50:14.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:14 smithi161 ceph-mon[35615]: Detected new or changed devices on smithi053 2024-09-11T21:50:14.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:14 smithi161 ceph-mon[35615]: Adjusting osd_memory_target on smithi053 to 17092M 2024-09-11T21:50:14.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:14 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T21:50:14.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:14 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T21:50:14.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:14 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T21:50:14.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:14 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:50:14.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:14 smithi161 ceph-mon[35615]: osd.1 [v2:172.21.15.53:6800/3039391924,v1:172.21.15.53:6801/3039391924] boot 2024-09-11T21:50:14.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:14 smithi161 ceph-mon[35615]: osdmap e13: 2 total, 2 up, 2 in 2024-09-11T21:50:14.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:14 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T21:50:14.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:14 smithi001 ceph-mon[30935]: purged_snaps scrub starts 2024-09-11T21:50:14.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:14 smithi001 ceph-mon[30935]: purged_snaps scrub ok 2024-09-11T21:50:14.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:14 smithi001 ceph-mon[30935]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T21:50:14.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:14 smithi001 ceph-mon[30935]: Detected new or changed devices on smithi053 2024-09-11T21:50:14.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:14 smithi001 ceph-mon[30935]: Adjusting osd_memory_target on smithi053 to 17092M 2024-09-11T21:50:14.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:14 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T21:50:14.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:14 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T21:50:14.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:14 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T21:50:14.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:14 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:50:14.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:14 smithi001 ceph-mon[30935]: osd.1 [v2:172.21.15.53:6800/3039391924,v1:172.21.15.53:6801/3039391924] boot 2024-09-11T21:50:14.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:14 smithi001 ceph-mon[30935]: osdmap e13: 2 total, 2 up, 2 in 2024-09-11T21:50:14.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:14 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T21:50:14.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:14 smithi053 ceph-mon[34908]: purged_snaps scrub starts 2024-09-11T21:50:14.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:14 smithi053 ceph-mon[34908]: purged_snaps scrub ok 2024-09-11T21:50:14.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:14 smithi053 ceph-mon[34908]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T21:50:14.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:14 smithi053 ceph-mon[34908]: Detected new or changed devices on smithi053 2024-09-11T21:50:14.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:14 smithi053 ceph-mon[34908]: Adjusting osd_memory_target on smithi053 to 17092M 2024-09-11T21:50:14.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:14 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T21:50:14.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:14 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T21:50:14.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:14 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T21:50:14.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:14 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:50:14.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:14 smithi053 ceph-mon[34908]: osd.1 [v2:172.21.15.53:6800/3039391924,v1:172.21.15.53:6801/3039391924] boot 2024-09-11T21:50:14.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:14 smithi053 ceph-mon[34908]: osdmap e13: 2 total, 2 up, 2 in 2024-09-11T21:50:14.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:14 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T21:50:15.552 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:15 smithi161 ceph-mon[35615]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:50:15.552 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:15 smithi161 ceph-mon[35615]: osdmap e14: 2 total, 2 up, 2 in 2024-09-11T21:50:15.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:15 smithi001 ceph-mon[30935]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:50:15.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:15 smithi001 ceph-mon[30935]: osdmap e14: 2 total, 2 up, 2 in 2024-09-11T21:50:15.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:15 smithi053 ceph-mon[34908]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:50:15.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:15 smithi053 ceph-mon[34908]: osdmap e14: 2 total, 2 up, 2 in 2024-09-11T21:50:16.442 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:16 smithi161 ceph-mon[35615]: pgmap v58: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T21:50:16.442 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:16 smithi161 ceph-mon[35615]: from='client.? 172.21.15.161:0/4190513348' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "578d4260-41e1-4645-b9b1-1c5445712c13"}]: dispatch 2024-09-11T21:50:16.442 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:16 smithi161 ceph-mon[35615]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "578d4260-41e1-4645-b9b1-1c5445712c13"}]: dispatch 2024-09-11T21:50:16.442 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:16 smithi161 ceph-mon[35615]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "578d4260-41e1-4645-b9b1-1c5445712c13"}]': finished 2024-09-11T21:50:16.443 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:16 smithi161 ceph-mon[35615]: osdmap e15: 3 total, 2 up, 3 in 2024-09-11T21:50:16.443 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:16 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T21:50:16.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:16 smithi001 ceph-mon[30935]: pgmap v58: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T21:50:16.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:16 smithi001 ceph-mon[30935]: from='client.? 172.21.15.161:0/4190513348' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "578d4260-41e1-4645-b9b1-1c5445712c13"}]: dispatch 2024-09-11T21:50:16.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:16 smithi001 ceph-mon[30935]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "578d4260-41e1-4645-b9b1-1c5445712c13"}]: dispatch 2024-09-11T21:50:16.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:16 smithi001 ceph-mon[30935]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "578d4260-41e1-4645-b9b1-1c5445712c13"}]': finished 2024-09-11T21:50:16.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:16 smithi001 ceph-mon[30935]: osdmap e15: 3 total, 2 up, 3 in 2024-09-11T21:50:16.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:16 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T21:50:16.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:16 smithi053 ceph-mon[34908]: pgmap v58: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T21:50:16.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:16 smithi053 ceph-mon[34908]: from='client.? 172.21.15.161:0/4190513348' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "578d4260-41e1-4645-b9b1-1c5445712c13"}]: dispatch 2024-09-11T21:50:16.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:16 smithi053 ceph-mon[34908]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "578d4260-41e1-4645-b9b1-1c5445712c13"}]: dispatch 2024-09-11T21:50:16.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:16 smithi053 ceph-mon[34908]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "578d4260-41e1-4645-b9b1-1c5445712c13"}]': finished 2024-09-11T21:50:16.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:16 smithi053 ceph-mon[34908]: osdmap e15: 3 total, 2 up, 3 in 2024-09-11T21:50:16.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:16 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T21:50:17.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:17 smithi001 ceph-mon[30935]: from='client.? 172.21.15.161:0/846505625' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T21:50:17.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:17 smithi053 ceph-mon[34908]: from='client.? 172.21.15.161:0/846505625' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T21:50:17.841 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:17 smithi161 ceph-mon[35615]: from='client.? 172.21.15.161:0/846505625' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T21:50:18.621 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:18 smithi001 ceph-mon[30935]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T21:50:18.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:18 smithi053 ceph-mon[34908]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T21:50:18.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:18 smithi161 ceph-mon[35615]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T21:50:20.580 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:20 smithi161 ceph-mon[35615]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T21:50:20.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:20 smithi001 ceph-mon[30935]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T21:50:20.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:20 smithi053 ceph-mon[34908]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T21:50:22.549 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:22 smithi161 ceph-mon[35615]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T21:50:22.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:22 smithi001 ceph-mon[30935]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T21:50:22.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:22 smithi053 ceph-mon[34908]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T21:50:24.464 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:24 smithi161 ceph-mon[35615]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T21:50:24.464 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:24 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-11T21:50:24.464 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:24 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:50:24.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:24 smithi001 ceph-mon[30935]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T21:50:24.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:24 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-11T21:50:24.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:24 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:50:24.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:24 smithi053 ceph-mon[34908]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T21:50:24.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:24 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-11T21:50:24.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:24 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:50:25.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:25 smithi161 ceph-mon[35615]: Deploying daemon osd.2 on smithi161 2024-09-11T21:50:25.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:25 smithi001 ceph-mon[30935]: Deploying daemon osd.2 on smithi161 2024-09-11T21:50:25.800 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:25 smithi053 ceph-mon[34908]: Deploying daemon osd.2 on smithi161 2024-09-11T21:50:26.504 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:26 smithi161 ceph-mon[35615]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T21:50:26.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:26 smithi001 ceph-mon[30935]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T21:50:26.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:26 smithi053 ceph-mon[34908]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T21:50:28.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:28 smithi001 ceph-mon[30935]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T21:50:28.748 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:28 smithi161 ceph-mon[35615]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T21:50:28.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:28 smithi053 ceph-mon[34908]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T21:50:30.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:30 smithi001 ceph-mon[30935]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T21:50:30.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:30 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:50:30.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:30 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:30.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:30 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:30.698 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:30 smithi161 ceph-mon[35615]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T21:50:30.698 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:30 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:50:30.698 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:30 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:30.698 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:30 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:30.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:30 smithi053 ceph-mon[34908]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T21:50:30.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:30 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:50:30.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:30 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:30.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:30 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:32.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:32 smithi001 ceph-mon[30935]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T21:50:32.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:32 smithi053 ceph-mon[34908]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T21:50:32.841 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:32 smithi161 ceph-mon[35615]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T21:50:33.828 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:33 smithi161 ceph-mon[35615]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T21:50:33.828 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:33 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:33.828 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:33 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:33.828 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:33 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:50:33.828 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:33 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:50:33.828 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:33 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:33.869 INFO:teuthology.orchestra.run.smithi161.stdout:Created osd(s) 2 on host 'smithi161' 2024-09-11T21:50:33.869 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:50:33.867+0000 7f84ef7fe640 1 -- 172.21.15.161:0/3347359056 <== mgr.14150 v2:172.21.15.1:6800/3890350886 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f84f8062630 con 0x7f84e40a2ba0 2024-09-11T21:50:33.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:50:33.869+0000 7f84fcf24640 1 -- 172.21.15.161:0/3347359056 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f84e40a2ba0 msgr2=0x7f84e40a5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:50:33.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:50:33.869+0000 7f84fcf24640 1 --2- 172.21.15.161:0/3347359056 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f84e40a2ba0 0x7f84e40a5060 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f84e0005fd0 tx=0x7f84e000b410 comp rx=0 tx=0).stop 2024-09-11T21:50:33.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:50:33.870+0000 7f84fcf24640 1 -- 172.21.15.161:0/3347359056 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84f8101cd0 msgr2=0x7f84f8197370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:50:33.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:50:33.870+0000 7f84fcf24640 1 --2- 172.21.15.161:0/3347359056 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84f8101cd0 0x7f84f8197370 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f84e8009230 tx=0x7f84e8004360 comp rx=0 tx=0).stop 2024-09-11T21:50:33.871 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:50:33.870+0000 7f84fcf24640 1 -- 172.21.15.161:0/3347359056 shutdown_connections 2024-09-11T21:50:33.872 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:50:33.870+0000 7f84fcf24640 1 --2- 172.21.15.161:0/3347359056 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f84e40a2ba0 0x7f84e40a5060 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:50:33.872 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:50:33.870+0000 7f84fcf24640 1 --2- 172.21.15.161:0/3347359056 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84f8102690 0x7f84f819e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:50:33.872 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:50:33.870+0000 7f84fcf24640 1 --2- 172.21.15.161:0/3347359056 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84f8101cd0 0x7f84f8197370 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:50:33.872 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:50:33.870+0000 7f84fcf24640 1 --2- 172.21.15.161:0/3347359056 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84f8100ad0 0x7f84f8196e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:50:33.872 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:50:33.870+0000 7f84fcf24640 1 -- 172.21.15.161:0/3347359056 >> 172.21.15.161:0/3347359056 conn(0x7f84f80fc280 msgr2=0x7f84f8105040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:50:33.872 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:50:33.870+0000 7f84fcf24640 1 -- 172.21.15.161:0/3347359056 shutdown_connections 2024-09-11T21:50:33.872 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:50:33.870+0000 7f84fcf24640 1 -- 172.21.15.161:0/3347359056 wait complete. 2024-09-11T21:50:33.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:33 smithi001 ceph-mon[30935]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T21:50:33.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:33 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:33.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:33 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:33.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:33 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:50:33.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:33 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:50:33.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:33 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:34.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:33 smithi053 ceph-mon[34908]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T21:50:34.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:33 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:34.048 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:33 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:34.048 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:33 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:50:34.048 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:33 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:50:34.048 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:33 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:34.491 DEBUG:teuthology.orchestra.run.smithi161:osd.2> sudo journalctl -f -n 0 -u ceph-3ffca83c-7087-11ef-bcea-c7b262605968@osd.2.service 2024-09-11T21:50:34.494 INFO:tasks.cephadm:Deploying osd.3 on smithi001 with /dev/vg_nvme/lv_3... 2024-09-11T21:50:34.494 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-11T21:50:34.732 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:34 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:50:34.732 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:34 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:34.732 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:34 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:34.744 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:50:34.866 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:34 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:50:34.866 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:34 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:34.866 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:34 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:34.949 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:34 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:50:34.949 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:34 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:34.949 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:34 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:36.143 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:35 smithi001 ceph-mon[30935]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T21:50:36.228 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:35 smithi053 ceph-mon[34908]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T21:50:36.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:35 smithi161 ceph-mon[35615]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T21:50:36.342 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 11 21:50:35 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-2[40567]: 2024-09-11T21:50:35.971+0000 7fc56f585640 -1 osd.2 0 log_to_monitors true 2024-09-11T21:50:37.116 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:36 smithi001 ceph-mon[30935]: from='osd.2 [v2:172.21.15.161:6800/1707914885,v1:172.21.15.161:6801/1707914885]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T21:50:37.116 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:36 smithi001 ceph-mon[30935]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T21:50:37.128 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:36 smithi161 ceph-mon[35615]: from='osd.2 [v2:172.21.15.161:6800/1707914885,v1:172.21.15.161:6801/1707914885]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T21:50:37.128 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:36 smithi161 ceph-mon[35615]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T21:50:37.216 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:36 smithi053 ceph-mon[34908]: from='osd.2 [v2:172.21.15.161:6800/1707914885,v1:172.21.15.161:6801/1707914885]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T21:50:37.217 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:36 smithi053 ceph-mon[34908]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T21:50:37.429 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:50:37.459 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi001:vg_nvme/lv_3 2024-09-11T21:50:37.717 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:50:37.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:37 smithi001 ceph-mon[30935]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T21:50:37.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:37 smithi001 ceph-mon[30935]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-11T21:50:37.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:37 smithi001 ceph-mon[30935]: osdmap e16: 3 total, 2 up, 3 in 2024-09-11T21:50:37.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:37 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T21:50:37.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:37 smithi001 ceph-mon[30935]: from='osd.2 [v2:172.21.15.161:6800/1707914885,v1:172.21.15.161:6801/1707914885]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-09-11T21:50:37.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:37 smithi001 ceph-mon[30935]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-09-11T21:50:37.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:37 smithi001 ceph-mon[30935]: Detected new or changed devices on smithi161 2024-09-11T21:50:37.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:37 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:37.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:37 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:37.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:37 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:50:37.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:37 smithi001 ceph-mon[30935]: Adjusting osd_memory_target on smithi161 to 17088M 2024-09-11T21:50:37.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:37 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:37.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:37 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:50:37.945 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:37 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:50:37.945 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:37 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:38.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:50:38.207+0000 7f57e441a640 1 -- 172.21.15.1:0/1057193655 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57dc102690 msgr2=0x7f57dc108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:50:38.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:50:38.207+0000 7f57e441a640 1 --2- 172.21.15.1:0/1057193655 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57dc102690 0x7f57dc108f20 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f57d4009ca0 tx=0x7f57d402a8c0 comp rx=0 tx=0).stop 2024-09-11T21:50:38.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:50:38.207+0000 7f57e441a640 1 -- 172.21.15.1:0/1057193655 shutdown_connections 2024-09-11T21:50:38.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:50:38.207+0000 7f57e441a640 1 --2- 172.21.15.1:0/1057193655 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57dc102690 0x7f57dc108f20 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:50:38.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:50:38.207+0000 7f57e441a640 1 --2- 172.21.15.1:0/1057193655 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f57dc101cd0 0x7f57dc102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:50:38.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:50:38.207+0000 7f57e441a640 1 --2- 172.21.15.1:0/1057193655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57dc100ad0 0x7f57dc100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:50:38.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:50:38.207+0000 7f57e441a640 1 -- 172.21.15.1:0/1057193655 >> 172.21.15.1:0/1057193655 conn(0x7f57dc0fc280 msgr2=0x7f57dc0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:50:38.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:50:38.207+0000 7f57e441a640 1 -- 172.21.15.1:0/1057193655 shutdown_connections 2024-09-11T21:50:38.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:50:38.207+0000 7f57e441a640 1 -- 172.21.15.1:0/1057193655 wait complete. 2024-09-11T21:50:38.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:50:38.208+0000 7f57e441a640 1 Processor -- start 2024-09-11T21:50:38.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:50:38.208+0000 7f57e441a640 1 -- start start 2024-09-11T21:50:38.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:50:38.208+0000 7f57e441a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f57dc100ad0 0x7f57dc19a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:50:38.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:50:38.208+0000 7f57e441a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57dc101cd0 0x7f57dc19a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:50:38.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:50:38.208+0000 7f57e441a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57dc102690 0x7f57dc1a1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:50:38.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:50:38.208+0000 7f57e441a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f57dc10c750 con 0x7f57dc101cd0 2024-09-11T21:50:38.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:50:38.208+0000 7f57e441a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f57dc10c5d0 con 0x7f57dc102690 2024-09-11T21:50:38.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:50:38.208+0000 7f57e441a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f57dc10c8d0 con 0x7f57dc100ad0 2024-09-11T21:50:38.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:50:38.208+0000 7f57e19b5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57dc101cd0 0x7f57dc19a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:50:38.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:50:38.208+0000 7f57e19b5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57dc101cd0 0x7f57dc19a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40320/0 (socket says 172.21.15.1:40320) 2024-09-11T21:50:38.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:50:38.208+0000 7f57e19b5640 1 -- 172.21.15.1:0/3764037575 learned_addr learned my addr 172.21.15.1:0/3764037575 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:50:38.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:50:38.208+0000 7f57e21b6640 1 --2- 172.21.15.1:0/3764037575 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f57dc100ad0 0x7f57dc19a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:50:38.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:50:38.208+0000 7f57e29b7640 1 --2- 172.21.15.1:0/3764037575 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57dc102690 0x7f57dc1a1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:50:38.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:50:38.208+0000 7f57e19b5640 1 -- 172.21.15.1:0/3764037575 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f57dc100ad0 msgr2=0x7f57dc19a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:50:38.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:50:38.208+0000 7f57e19b5640 1 --2- 172.21.15.1:0/3764037575 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f57dc100ad0 0x7f57dc19a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:50:38.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:50:38.208+0000 7f57e19b5640 1 -- 172.21.15.1:0/3764037575 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57dc102690 msgr2=0x7f57dc1a1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:50:38.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:50:38.208+0000 7f57e19b5640 1 --2- 172.21.15.1:0/3764037575 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57dc102690 0x7f57dc1a1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:50:38.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:50:38.209+0000 7f57e19b5640 1 -- 172.21.15.1:0/3764037575 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57dc1a1cd0 con 0x7f57dc101cd0 2024-09-11T21:50:38.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:50:38.209+0000 7f57e29b7640 1 --2- 172.21.15.1:0/3764037575 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57dc102690 0x7f57dc1a1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T21:50:38.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:50:38.209+0000 7f57e19b5640 1 --2- 172.21.15.1:0/3764037575 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57dc101cd0 0x7f57dc19a5e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f57d000e9f0 tx=0x7f57d000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:50:38.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:50:38.209+0000 7f57cf7fe640 1 -- 172.21.15.1:0/3764037575 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d000cd80 con 0x7f57dc101cd0 2024-09-11T21:50:38.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:50:38.209+0000 7f57cf7fe640 1 -- 172.21.15.1:0/3764037575 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57d0005500 con 0x7f57dc101cd0 2024-09-11T21:50:38.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:50:38.209+0000 7f57e441a640 1 -- 172.21.15.1:0/3764037575 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57dc1a1fc0 con 0x7f57dc101cd0 2024-09-11T21:50:38.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:50:38.209+0000 7f57cf7fe640 1 -- 172.21.15.1:0/3764037575 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d0010690 con 0x7f57dc101cd0 2024-09-11T21:50:38.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:50:38.209+0000 7f57e441a640 1 -- 172.21.15.1:0/3764037575 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57dc1a2470 con 0x7f57dc101cd0 2024-09-11T21:50:38.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:50:38.210+0000 7f57e441a640 1 -- 172.21.15.1:0/3764037575 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57dc1a2700 con 0x7f57dc101cd0 2024-09-11T21:50:38.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:50:38.212+0000 7f57cf7fe640 1 -- 172.21.15.1:0/3764037575 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 15) ==== 137776+0+0 (secure 0 0 0) 0x7f57d00046c0 con 0x7f57dc101cd0 2024-09-11T21:50:38.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:50:38.213+0000 7f57cf7fe640 1 --2- 172.21.15.1:0/3764037575 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f57c80a2ba0 0x7f57c80a5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:50:38.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:50:38.213+0000 7f57e21b6640 1 --2- 172.21.15.1:0/3764037575 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f57c80a2ba0 0x7f57c80a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:50:38.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:50:38.213+0000 7f57cf7fe640 1 -- 172.21.15.1:0/3764037575 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 2571+0+0 (secure 0 0 0) 0x7f57d00cdc90 con 0x7f57dc101cd0 2024-09-11T21:50:38.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:50:38.213+0000 7f57e21b6640 1 --2- 172.21.15.1:0/3764037575 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f57c80a2ba0 0x7f57c80a5060 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f57dc101b30 tx=0x7f57d8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:50:38.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:50:38.218+0000 7f57cf7fe640 1 -- 172.21.15.1:0/3764037575 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f57d009c320 con 0x7f57dc101cd0 2024-09-11T21:50:38.257 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:37 smithi161 ceph-mon[35615]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T21:50:38.258 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:37 smithi161 ceph-mon[35615]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-11T21:50:38.258 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:37 smithi161 ceph-mon[35615]: osdmap e16: 3 total, 2 up, 3 in 2024-09-11T21:50:38.258 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:37 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T21:50:38.258 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:37 smithi161 ceph-mon[35615]: from='osd.2 [v2:172.21.15.161:6800/1707914885,v1:172.21.15.161:6801/1707914885]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-09-11T21:50:38.258 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:37 smithi161 ceph-mon[35615]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-09-11T21:50:38.258 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:37 smithi161 ceph-mon[35615]: Detected new or changed devices on smithi161 2024-09-11T21:50:38.258 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:37 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:38.258 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:37 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:38.258 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:37 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:50:38.258 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:37 smithi161 ceph-mon[35615]: Adjusting osd_memory_target on smithi161 to 17088M 2024-09-11T21:50:38.258 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:37 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:38.258 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:37 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:50:38.259 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:37 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:50:38.259 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:37 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:38.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:37 smithi053 ceph-mon[34908]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T21:50:38.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:37 smithi053 ceph-mon[34908]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-11T21:50:38.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:37 smithi053 ceph-mon[34908]: osdmap e16: 3 total, 2 up, 3 in 2024-09-11T21:50:38.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:37 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T21:50:38.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:37 smithi053 ceph-mon[34908]: from='osd.2 [v2:172.21.15.161:6800/1707914885,v1:172.21.15.161:6801/1707914885]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-09-11T21:50:38.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:37 smithi053 ceph-mon[34908]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-09-11T21:50:38.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:37 smithi053 ceph-mon[34908]: Detected new or changed devices on smithi161 2024-09-11T21:50:38.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:37 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:38.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:37 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:38.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:37 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:50:38.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:37 smithi053 ceph-mon[34908]: Adjusting osd_memory_target on smithi161 to 17088M 2024-09-11T21:50:38.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:37 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:38.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:37 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:50:38.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:37 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:50:38.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:37 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:38.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:50:38.405+0000 7f57e441a640 1 -- 172.21.15.1:0/3764037575 --> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f57dc1a28d0 con 0x7f57c80a2ba0 2024-09-11T21:50:38.591 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 11 21:50:38 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-2[40567]: 2024-09-11T21:50:38.256+0000 7fc56bd1c640 -1 osd.2 0 waiting for initial osdmap 2024-09-11T21:50:38.592 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 11 21:50:38 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-2[40567]: 2024-09-11T21:50:38.260+0000 7fc563328640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T21:50:39.040 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:38 smithi001 ceph-mon[30935]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-09-11T21:50:39.040 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:38 smithi001 ceph-mon[30935]: osdmap e17: 3 total, 2 up, 3 in 2024-09-11T21:50:39.040 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:38 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T21:50:39.040 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:38 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T21:50:39.041 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:38 smithi001 ceph-mon[30935]: from='osd.2 ' entity='osd.2' 2024-09-11T21:50:39.041 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:38 smithi001 ceph-mon[30935]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:50:39.041 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:38 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T21:50:39.041 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:38 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T21:50:39.041 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:38 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:50:39.301 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:38 smithi053 ceph-mon[34908]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-09-11T21:50:39.301 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:38 smithi053 ceph-mon[34908]: osdmap e17: 3 total, 2 up, 3 in 2024-09-11T21:50:39.301 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:38 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T21:50:39.301 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:38 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T21:50:39.301 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:38 smithi053 ceph-mon[34908]: from='osd.2 ' entity='osd.2' 2024-09-11T21:50:39.301 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:38 smithi053 ceph-mon[34908]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:50:39.301 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:38 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T21:50:39.301 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:38 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T21:50:39.301 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:38 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:50:39.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:38 smithi161 ceph-mon[35615]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-09-11T21:50:39.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:38 smithi161 ceph-mon[35615]: osdmap e17: 3 total, 2 up, 3 in 2024-09-11T21:50:39.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:38 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T21:50:39.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:38 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T21:50:39.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:38 smithi161 ceph-mon[35615]: from='osd.2 ' entity='osd.2' 2024-09-11T21:50:39.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:38 smithi161 ceph-mon[35615]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:50:39.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:38 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T21:50:39.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:38 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T21:50:39.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:38 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:50:40.030 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:39 smithi001 ceph-mon[30935]: purged_snaps scrub starts 2024-09-11T21:50:40.030 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:39 smithi001 ceph-mon[30935]: purged_snaps scrub ok 2024-09-11T21:50:40.030 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:39 smithi001 ceph-mon[30935]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T21:50:40.030 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:39 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T21:50:40.030 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:39 smithi001 ceph-mon[30935]: osd.2 [v2:172.21.15.161:6800/1707914885,v1:172.21.15.161:6801/1707914885] boot 2024-09-11T21:50:40.030 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:39 smithi001 ceph-mon[30935]: osdmap e18: 3 total, 3 up, 3 in 2024-09-11T21:50:40.030 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:39 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T21:50:40.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:39 smithi053 ceph-mon[34908]: purged_snaps scrub starts 2024-09-11T21:50:40.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:39 smithi053 ceph-mon[34908]: purged_snaps scrub ok 2024-09-11T21:50:40.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:39 smithi053 ceph-mon[34908]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T21:50:40.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:39 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T21:50:40.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:39 smithi053 ceph-mon[34908]: osd.2 [v2:172.21.15.161:6800/1707914885,v1:172.21.15.161:6801/1707914885] boot 2024-09-11T21:50:40.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:39 smithi053 ceph-mon[34908]: osdmap e18: 3 total, 3 up, 3 in 2024-09-11T21:50:40.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:39 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T21:50:40.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:39 smithi161 ceph-mon[35615]: purged_snaps scrub starts 2024-09-11T21:50:40.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:39 smithi161 ceph-mon[35615]: purged_snaps scrub ok 2024-09-11T21:50:40.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:39 smithi161 ceph-mon[35615]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-11T21:50:40.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:39 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T21:50:40.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:39 smithi161 ceph-mon[35615]: osd.2 [v2:172.21.15.161:6800/1707914885,v1:172.21.15.161:6801/1707914885] boot 2024-09-11T21:50:40.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:39 smithi161 ceph-mon[35615]: osdmap e18: 3 total, 3 up, 3 in 2024-09-11T21:50:40.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:39 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T21:50:41.344 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:41 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-11T21:50:41.344 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:41 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/999253184' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73f78766-9662-4aa3-841c-fe2ce654ee9c"}]: dispatch 2024-09-11T21:50:41.345 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:41 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-11T21:50:41.345 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:41 smithi161 ceph-mon[35615]: osdmap e19: 3 total, 3 up, 3 in 2024-09-11T21:50:41.345 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:41 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T21:50:41.345 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:41 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/999253184' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "73f78766-9662-4aa3-841c-fe2ce654ee9c"}]': finished 2024-09-11T21:50:41.345 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:41 smithi161 ceph-mon[35615]: osdmap e20: 4 total, 3 up, 4 in 2024-09-11T21:50:41.345 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:41 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T21:50:41.356 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:41 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-11T21:50:41.356 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:41 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/999253184' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73f78766-9662-4aa3-841c-fe2ce654ee9c"}]: dispatch 2024-09-11T21:50:41.356 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:41 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-11T21:50:41.356 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:41 smithi001 ceph-mon[30935]: osdmap e19: 3 total, 3 up, 3 in 2024-09-11T21:50:41.356 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:41 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T21:50:41.356 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:41 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/999253184' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "73f78766-9662-4aa3-841c-fe2ce654ee9c"}]': finished 2024-09-11T21:50:41.356 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:41 smithi001 ceph-mon[30935]: osdmap e20: 4 total, 3 up, 4 in 2024-09-11T21:50:41.356 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:41 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T21:50:41.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:41 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-11T21:50:41.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:41 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/999253184' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73f78766-9662-4aa3-841c-fe2ce654ee9c"}]: dispatch 2024-09-11T21:50:41.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:41 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-11T21:50:41.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:41 smithi053 ceph-mon[34908]: osdmap e19: 3 total, 3 up, 3 in 2024-09-11T21:50:41.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:41 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T21:50:41.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:41 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/999253184' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "73f78766-9662-4aa3-841c-fe2ce654ee9c"}]': finished 2024-09-11T21:50:41.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:41 smithi053 ceph-mon[34908]: osdmap e20: 4 total, 3 up, 4 in 2024-09-11T21:50:41.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:41 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T21:50:42.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:42 smithi161 ceph-mon[35615]: pgmap v76: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-11T21:50:42.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:42 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/3744466421' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T21:50:42.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:42 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-11T21:50:42.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:42 smithi161 ceph-mon[35615]: osdmap e21: 4 total, 3 up, 4 in 2024-09-11T21:50:42.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:42 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T21:50:42.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:42 smithi001 ceph-mon[30935]: pgmap v76: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-11T21:50:42.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:42 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/3744466421' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T21:50:42.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:42 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-11T21:50:42.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:42 smithi001 ceph-mon[30935]: osdmap e21: 4 total, 3 up, 4 in 2024-09-11T21:50:42.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:42 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T21:50:42.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:42 smithi053 ceph-mon[34908]: pgmap v76: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-11T21:50:42.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:42 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/3744466421' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T21:50:42.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:42 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-11T21:50:42.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:42 smithi053 ceph-mon[34908]: osdmap e21: 4 total, 3 up, 4 in 2024-09-11T21:50:42.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:42 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T21:50:42.943 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 11 21:50:42 smithi001 sudo[47117]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T21:50:42.943 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 11 21:50:42 smithi001 sudo[47117]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T21:50:42.943 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 11 21:50:42 smithi001 sudo[47117]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T21:50:42.943 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 11 21:50:42 smithi001 sudo[47117]: pam_unix(sudo:session): session closed for user root 2024-09-11T21:50:43.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:50:43.093+0000 7f57cf7fe640 1 -- 172.21.15.1:0/3764037575 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f57d0098290 con 0x7f57dc101cd0 2024-09-11T21:50:43.443 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 11 21:50:43 smithi001 sudo[47486]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T21:50:43.443 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 11 21:50:43 smithi001 sudo[47486]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T21:50:43.443 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 11 21:50:43 smithi001 sudo[47486]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T21:50:43.443 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 11 21:50:43 smithi001 sudo[47486]: pam_unix(sudo:session): session closed for user root 2024-09-11T21:50:43.604 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 11 21:50:43 smithi053 sudo[43232]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T21:50:43.605 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 11 21:50:43 smithi053 sudo[43232]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T21:50:43.605 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 11 21:50:43 smithi053 sudo[43232]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T21:50:43.605 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 11 21:50:43 smithi053 sudo[43232]: pam_unix(sudo:session): session closed for user root 2024-09-11T21:50:44.039 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 11 21:50:43 smithi053 sudo[43235]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T21:50:44.039 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 11 21:50:43 smithi053 sudo[43235]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T21:50:44.039 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 11 21:50:43 smithi053 sudo[43235]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T21:50:44.039 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 11 21:50:43 smithi053 sudo[43235]: pam_unix(sudo:session): session closed for user root 2024-09-11T21:50:44.129 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 11 21:50:43 smithi161 sudo[44637]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T21:50:44.130 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 11 21:50:43 smithi161 sudo[44637]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T21:50:44.130 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 11 21:50:43 smithi161 sudo[44637]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T21:50:44.130 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 11 21:50:43 smithi161 sudo[44637]: pam_unix(sudo:session): session closed for user root 2024-09-11T21:50:44.130 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:44 smithi161 ceph-mon[35615]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-11T21:50:44.130 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:44 smithi161 ceph-mon[35615]: osdmap e22: 4 total, 3 up, 4 in 2024-09-11T21:50:44.130 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:44 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T21:50:44.130 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:44 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:44.130 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:44 smithi161 ceph-mon[35615]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-11T21:50:44.130 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:44 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:44.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:44 smithi053 ceph-mon[34908]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-11T21:50:44.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:44 smithi053 ceph-mon[34908]: osdmap e22: 4 total, 3 up, 4 in 2024-09-11T21:50:44.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:44 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T21:50:44.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:44 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:44.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:44 smithi053 ceph-mon[34908]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-11T21:50:44.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:44 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:44.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:44 smithi001 ceph-mon[30935]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-11T21:50:44.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:44 smithi001 ceph-mon[30935]: osdmap e22: 4 total, 3 up, 4 in 2024-09-11T21:50:44.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:44 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T21:50:44.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:44 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:44.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:44 smithi001 ceph-mon[30935]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-11T21:50:44.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:44 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:44.592 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 11 21:50:44 smithi161 sudo[44640]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T21:50:44.592 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 11 21:50:44 smithi161 sudo[44640]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T21:50:44.592 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 11 21:50:44 smithi161 sudo[44640]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T21:50:44.592 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 11 21:50:44 smithi161 sudo[44640]: pam_unix(sudo:session): session closed for user root 2024-09-11T21:50:44.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:44 smithi001 sudo[47861]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T21:50:44.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:44 smithi001 sudo[47861]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T21:50:44.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:44 smithi001 sudo[47861]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T21:50:44.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:44 smithi001 sudo[47861]: pam_unix(sudo:session): session closed for user root 2024-09-11T21:50:45.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:45 smithi001 sudo[47925]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T21:50:45.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:45 smithi001 sudo[47925]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T21:50:45.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:45 smithi001 sudo[47925]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T21:50:45.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:45 smithi001 sudo[47925]: pam_unix(sudo:session): session closed for user root 2024-09-11T21:50:45.549 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:45 smithi161 sudo[44643]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T21:50:45.550 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:45 smithi161 sudo[44643]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T21:50:45.550 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:45 smithi161 sudo[44643]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T21:50:45.550 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:45 smithi161 sudo[44643]: pam_unix(sudo:session): session closed for user root 2024-09-11T21:50:45.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:45 smithi161 sudo[44646]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T21:50:45.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:45 smithi161 sudo[44646]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T21:50:45.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:45 smithi161 sudo[44646]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T21:50:45.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:45 smithi161 sudo[44646]: pam_unix(sudo:session): session closed for user root 2024-09-11T21:50:45.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:45 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:45.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:45 smithi161 ceph-mon[35615]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T21:50:45.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:45 smithi161 ceph-mon[35615]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T21:50:45.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:45 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T21:50:45.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:45 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T21:50:45.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:45 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T21:50:45.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:45 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T21:50:45.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:45 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T21:50:45.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:45 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T21:50:45.845 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:45 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:45.845 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:45 smithi001 ceph-mon[30935]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T21:50:45.845 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:45 smithi001 ceph-mon[30935]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T21:50:45.845 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:45 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T21:50:45.845 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:45 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T21:50:45.845 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:45 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T21:50:45.846 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:45 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T21:50:45.846 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:45 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T21:50:45.846 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:45 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T21:50:46.076 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:45 smithi053 sudo[43238]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T21:50:46.076 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:45 smithi053 sudo[43238]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T21:50:46.076 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:45 smithi053 sudo[43238]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T21:50:46.076 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:45 smithi053 sudo[43238]: pam_unix(sudo:session): session closed for user root 2024-09-11T21:50:46.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:46 smithi053 sudo[43241]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T21:50:46.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:46 smithi053 sudo[43241]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T21:50:46.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:46 smithi053 sudo[43241]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T21:50:46.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:46 smithi053 sudo[43241]: pam_unix(sudo:session): session closed for user root 2024-09-11T21:50:46.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:46 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:46.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:46 smithi053 ceph-mon[34908]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T21:50:46.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:46 smithi053 ceph-mon[34908]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T21:50:46.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:46 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T21:50:46.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:46 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T21:50:46.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:46 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T21:50:46.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:46 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T21:50:46.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:46 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T21:50:46.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:46 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T21:50:46.787 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:46 smithi001 ceph-mon[30935]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-11T21:50:46.787 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:46 smithi001 ceph-mon[30935]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T21:50:46.787 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:46 smithi001 ceph-mon[30935]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T21:50:46.788 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:46 smithi001 ceph-mon[30935]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T21:50:46.788 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:46 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T21:50:46.788 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:46 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T21:50:46.788 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:46 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T21:50:46.788 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:46 smithi001 ceph-mon[30935]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T21:50:46.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:46 smithi161 ceph-mon[35615]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-11T21:50:46.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:46 smithi161 ceph-mon[35615]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T21:50:46.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:46 smithi161 ceph-mon[35615]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T21:50:46.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:46 smithi161 ceph-mon[35615]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T21:50:46.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:46 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T21:50:46.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:46 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T21:50:46.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:46 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T21:50:46.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:46 smithi161 ceph-mon[35615]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T21:50:47.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:46 smithi053 ceph-mon[34908]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-11T21:50:47.048 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:46 smithi053 ceph-mon[34908]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T21:50:47.048 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:46 smithi053 ceph-mon[34908]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T21:50:47.048 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:46 smithi053 ceph-mon[34908]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T21:50:47.048 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:46 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T21:50:47.048 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:46 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T21:50:47.048 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:46 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T21:50:47.048 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:46 smithi053 ceph-mon[34908]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T21:50:47.841 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:47 smithi161 ceph-mon[35615]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-11T21:50:47.876 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:47 smithi001 ceph-mon[30935]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-11T21:50:48.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:47 smithi053 ceph-mon[34908]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-11T21:50:49.657 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:49 smithi001 ceph-mon[30935]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-11T21:50:49.657 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:49 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-11T21:50:49.657 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:49 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:50:49.657 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:49 smithi001 ceph-mon[30935]: Deploying daemon osd.3 on smithi001 2024-09-11T21:50:49.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:49 smithi161 ceph-mon[35615]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-11T21:50:49.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:49 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-11T21:50:49.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:49 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:50:49.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:49 smithi161 ceph-mon[35615]: Deploying daemon osd.3 on smithi001 2024-09-11T21:50:50.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:49 smithi053 ceph-mon[34908]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-11T21:50:50.048 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:49 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-11T21:50:50.048 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:49 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:50:50.048 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:49 smithi053 ceph-mon[34908]: Deploying daemon osd.3 on smithi001 2024-09-11T21:50:51.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:51 smithi161 ceph-mon[35615]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-09-11T21:50:51.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:51 smithi001 ceph-mon[30935]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-09-11T21:50:52.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:51 smithi053 ceph-mon[34908]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-09-11T21:50:53.846 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:53 smithi161 ceph-mon[35615]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-09-11T21:50:53.946 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:53 smithi001 ceph-mon[30935]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-09-11T21:50:54.048 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:53 smithi053 ceph-mon[34908]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-09-11T21:50:55.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:55 smithi001 ceph-mon[30935]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-11T21:50:55.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:55 smithi161 ceph-mon[35615]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-11T21:50:56.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:55 smithi053 ceph-mon[34908]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-11T21:50:56.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:56 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:50:56.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:56 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:56.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:56 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:56.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:56 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:50:56.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:56 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:56.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:56 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:57.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:56 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:50:57.048 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:56 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:57.048 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:56 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:57.613 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:57 smithi001 ceph-mon[30935]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-11T21:50:58.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:57 smithi053 ceph-mon[34908]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-11T21:50:58.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:57 smithi161 ceph-mon[35615]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-11T21:50:59.554 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:59 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:59.554 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:59 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:59.554 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:59 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:50:59.554 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:59 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:50:59.554 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:50:59 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:59.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:50:59.591+0000 7f57cf7fe640 1 -- 172.21.15.1:0/3764037575 <== mgr.14150 v2:172.21.15.1:6800/3890350886 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f57dc1a28d0 con 0x7f57c80a2ba0 2024-09-11T21:50:59.592 INFO:teuthology.orchestra.run.smithi001.stdout:Created osd(s) 3 on host 'smithi001' 2024-09-11T21:50:59.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:50:59.594+0000 7f57e441a640 1 -- 172.21.15.1:0/3764037575 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f57c80a2ba0 msgr2=0x7f57c80a5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:50:59.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:50:59.594+0000 7f57e441a640 1 --2- 172.21.15.1:0/3764037575 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f57c80a2ba0 0x7f57c80a5060 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f57dc101b30 tx=0x7f57d8005f90 comp rx=0 tx=0).stop 2024-09-11T21:50:59.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:50:59.594+0000 7f57e441a640 1 -- 172.21.15.1:0/3764037575 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57dc101cd0 msgr2=0x7f57dc19a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:50:59.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:50:59.594+0000 7f57e441a640 1 --2- 172.21.15.1:0/3764037575 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57dc101cd0 0x7f57dc19a5e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f57d000e9f0 tx=0x7f57d000edf0 comp rx=0 tx=0).stop 2024-09-11T21:50:59.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:50:59.595+0000 7f57e441a640 1 -- 172.21.15.1:0/3764037575 shutdown_connections 2024-09-11T21:50:59.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:50:59.595+0000 7f57e441a640 1 --2- 172.21.15.1:0/3764037575 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f57c80a2ba0 0x7f57c80a5060 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:50:59.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:50:59.595+0000 7f57e441a640 1 --2- 172.21.15.1:0/3764037575 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57dc102690 0x7f57dc1a1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:50:59.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:50:59.595+0000 7f57e441a640 1 --2- 172.21.15.1:0/3764037575 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57dc101cd0 0x7f57dc19a5e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:50:59.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:50:59.595+0000 7f57e441a640 1 --2- 172.21.15.1:0/3764037575 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f57dc100ad0 0x7f57dc19a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:50:59.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:50:59.595+0000 7f57e441a640 1 -- 172.21.15.1:0/3764037575 >> 172.21.15.1:0/3764037575 conn(0x7f57dc0fc280 msgr2=0x7f57dc0fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:50:59.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:50:59.595+0000 7f57e441a640 1 -- 172.21.15.1:0/3764037575 shutdown_connections 2024-09-11T21:50:59.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:50:59.595+0000 7f57e441a640 1 -- 172.21.15.1:0/3764037575 wait complete. 2024-09-11T21:50:59.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:59 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:59.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:59 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:59.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:59 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:50:59.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:59 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:50:59.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:50:59 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:59.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:59 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:59.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:59 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:50:59.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:59 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:50:59.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:59 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:50:59.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:50:59 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:00.277 DEBUG:teuthology.orchestra.run.smithi001:osd.3> sudo journalctl -f -n 0 -u ceph-3ffca83c-7087-11ef-bcea-c7b262605968@osd.3.service 2024-09-11T21:51:00.279 INFO:tasks.cephadm:Deploying osd.4 on smithi053 with /dev/vg_nvme/lv_3... 2024-09-11T21:51:00.279 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-11T21:51:00.540 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T21:51:00.557 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:00 smithi001 ceph-mon[30935]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-11T21:51:00.557 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:00 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:51:00.557 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:00 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:00.557 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:00 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:00.660 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:00 smithi053 ceph-mon[34908]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-11T21:51:00.661 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:00 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:51:00.661 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:00 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:00.661 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:00 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:00.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:00 smithi161 ceph-mon[35615]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-11T21:51:00.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:00 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:51:00.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:00 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:00.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:00 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:01.541 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:01 smithi001 ceph-mon[30935]: from='osd.3 [v2:172.21.15.1:6810/1541142398,v1:172.21.15.1:6811/1541142398]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-11T21:51:01.542 INFO:journalctl@ceph.osd.3.smithi001.stdout:Sep 11 21:51:01 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-3[49085]: 2024-09-11T21:51:01.201+0000 7eff3c1bd640 -1 osd.3 0 log_to_monitors true 2024-09-11T21:51:01.667 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:01 smithi053 ceph-mon[34908]: from='osd.3 [v2:172.21.15.1:6810/1541142398,v1:172.21.15.1:6811/1541142398]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-11T21:51:01.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:01 smithi161 ceph-mon[35615]: from='osd.3 [v2:172.21.15.1:6810/1541142398,v1:172.21.15.1:6811/1541142398]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-11T21:51:02.578 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:02 smithi053 ceph-mon[34908]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-11T21:51:02.578 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:02 smithi053 ceph-mon[34908]: from='osd.3 [v2:172.21.15.1:6810/1541142398,v1:172.21.15.1:6811/1541142398]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-11T21:51:02.579 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:02 smithi053 ceph-mon[34908]: osdmap e23: 4 total, 3 up, 4 in 2024-09-11T21:51:02.579 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:02 smithi053 ceph-mon[34908]: from='osd.3 [v2:172.21.15.1:6810/1541142398,v1:172.21.15.1:6811/1541142398]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-09-11T21:51:02.579 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:02 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T21:51:02.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:02 smithi001 ceph-mon[30935]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-11T21:51:02.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:02 smithi001 ceph-mon[30935]: from='osd.3 [v2:172.21.15.1:6810/1541142398,v1:172.21.15.1:6811/1541142398]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-11T21:51:02.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:02 smithi001 ceph-mon[30935]: osdmap e23: 4 total, 3 up, 4 in 2024-09-11T21:51:02.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:02 smithi001 ceph-mon[30935]: from='osd.3 [v2:172.21.15.1:6810/1541142398,v1:172.21.15.1:6811/1541142398]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-09-11T21:51:02.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:02 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T21:51:02.694 INFO:journalctl@ceph.osd.3.smithi001.stdout:Sep 11 21:51:02 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-3[49085]: 2024-09-11T21:51:02.375+0000 7eff38140640 -1 osd.3 0 waiting for initial osdmap 2024-09-11T21:51:02.694 INFO:journalctl@ceph.osd.3.smithi001.stdout:Sep 11 21:51:02 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-3[49085]: 2024-09-11T21:51:02.379+0000 7eff33f68640 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T21:51:02.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:02 smithi161 ceph-mon[35615]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-11T21:51:02.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:02 smithi161 ceph-mon[35615]: from='osd.3 [v2:172.21.15.1:6810/1541142398,v1:172.21.15.1:6811/1541142398]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-11T21:51:02.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:02 smithi161 ceph-mon[35615]: osdmap e23: 4 total, 3 up, 4 in 2024-09-11T21:51:02.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:02 smithi161 ceph-mon[35615]: from='osd.3 [v2:172.21.15.1:6810/1541142398,v1:172.21.15.1:6811/1541142398]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-09-11T21:51:02.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:02 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T21:51:03.319 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-11T21:51:03.345 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi053:vg_nvme/lv_3 2024-09-11T21:51:03.584 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:03 smithi053 ceph-mon[34908]: from='osd.3 [v2:172.21.15.1:6810/1541142398,v1:172.21.15.1:6811/1541142398]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-09-11T21:51:03.584 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:03 smithi053 ceph-mon[34908]: osdmap e24: 4 total, 3 up, 4 in 2024-09-11T21:51:03.584 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:03 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T21:51:03.584 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:03 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T21:51:03.584 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:03 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:03.584 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:03 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:03.584 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:03 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:51:03.584 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:03 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:51:03.584 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:03 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:03.585 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:03 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:51:03.585 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:03 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:51:03.585 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:03 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:03.611 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T21:51:03.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:03 smithi001 ceph-mon[30935]: from='osd.3 [v2:172.21.15.1:6810/1541142398,v1:172.21.15.1:6811/1541142398]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-09-11T21:51:03.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:03 smithi001 ceph-mon[30935]: osdmap e24: 4 total, 3 up, 4 in 2024-09-11T21:51:03.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:03 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T21:51:03.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:03 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T21:51:03.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:03 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:03.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:03 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:03.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:03 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:51:03.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:03 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:51:03.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:03 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:03.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:03 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:51:03.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:03 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:51:03.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:03 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:03.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:03 smithi161 ceph-mon[35615]: from='osd.3 [v2:172.21.15.1:6810/1541142398,v1:172.21.15.1:6811/1541142398]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-09-11T21:51:03.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:03 smithi161 ceph-mon[35615]: osdmap e24: 4 total, 3 up, 4 in 2024-09-11T21:51:03.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:03 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T21:51:03.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:03 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T21:51:03.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:03 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:03.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:03 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:03.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:03 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:51:03.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:03 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:51:03.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:03 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:03.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:03 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:51:03.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:03 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:51:03.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:03 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:04.103 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:51:04.096+0000 7f83c35f3640 1 -- 172.21.15.53:0/619214283 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83bc1017a0 msgr2=0x7f83bc109040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:51:04.103 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:51:04.096+0000 7f83c35f3640 1 --2- 172.21.15.53:0/619214283 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83bc1017a0 0x7f83bc109040 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f83b4009ca0 tx=0x7f83b402a8c0 comp rx=0 tx=0).stop 2024-09-11T21:51:04.103 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:51:04.096+0000 7f83c35f3640 1 -- 172.21.15.53:0/619214283 shutdown_connections 2024-09-11T21:51:04.103 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:51:04.096+0000 7f83c35f3640 1 --2- 172.21.15.53:0/619214283 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83bc1017a0 0x7f83bc109040 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:51:04.103 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:51:04.096+0000 7f83c35f3640 1 --2- 172.21.15.53:0/619214283 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83bc100de0 0x7f83bc101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:51:04.103 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:51:04.096+0000 7f83c35f3640 1 --2- 172.21.15.53:0/619214283 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83bc1020a0 0x7f83bc1024a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:51:04.103 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:51:04.096+0000 7f83c35f3640 1 -- 172.21.15.53:0/619214283 >> 172.21.15.53:0/619214283 conn(0x7f83bc0fba40 msgr2=0x7f83bc0fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:51:04.103 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:51:04.096+0000 7f83c35f3640 1 -- 172.21.15.53:0/619214283 shutdown_connections 2024-09-11T21:51:04.103 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:51:04.096+0000 7f83c35f3640 1 -- 172.21.15.53:0/619214283 wait complete. 2024-09-11T21:51:04.104 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:51:04.097+0000 7f83c35f3640 1 Processor -- start 2024-09-11T21:51:04.104 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:51:04.097+0000 7f83c35f3640 1 -- start start 2024-09-11T21:51:04.104 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:51:04.097+0000 7f83c35f3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83bc100de0 0x7f83bc1980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:51:04.104 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:51:04.097+0000 7f83c35f3640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83bc1017a0 0x7f83bc198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:51:04.104 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:51:04.097+0000 7f83c35f3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83bc1020a0 0x7f83bc19f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:51:04.104 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:51:04.097+0000 7f83c35f3640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f83bc10d9b0 con 0x7f83bc1020a0 2024-09-11T21:51:04.104 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:51:04.097+0000 7f83c35f3640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f83bc10d830 con 0x7f83bc100de0 2024-09-11T21:51:04.104 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:51:04.097+0000 7f83c35f3640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f83bc10db30 con 0x7f83bc1017a0 2024-09-11T21:51:04.104 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:51:04.097+0000 7f83c138f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83bc100de0 0x7f83bc1980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:51:04.104 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:51:04.097+0000 7f83c138f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83bc100de0 0x7f83bc1980f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:57056/0 (socket says 172.21.15.53:57056) 2024-09-11T21:51:04.104 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:51:04.097+0000 7f83c138f640 1 -- 172.21.15.53:0/268783359 learned_addr learned my addr 172.21.15.53:0/268783359 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-11T21:51:04.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:51:04.097+0000 7f83c0b8e640 1 --2- 172.21.15.53:0/268783359 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83bc1017a0 0x7f83bc198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:51:04.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:51:04.097+0000 7f83c1b90640 1 --2- 172.21.15.53:0/268783359 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83bc1020a0 0x7f83bc19f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:51:04.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:51:04.097+0000 7f83c138f640 1 -- 172.21.15.53:0/268783359 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83bc1017a0 msgr2=0x7f83bc198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:51:04.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:51:04.097+0000 7f83c138f640 1 --2- 172.21.15.53:0/268783359 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83bc1017a0 0x7f83bc198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:51:04.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:51:04.097+0000 7f83c138f640 1 -- 172.21.15.53:0/268783359 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83bc1020a0 msgr2=0x7f83bc19f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:51:04.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:51:04.097+0000 7f83c138f640 1 --2- 172.21.15.53:0/268783359 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83bc1020a0 0x7f83bc19f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:51:04.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:51:04.097+0000 7f83c138f640 1 -- 172.21.15.53:0/268783359 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83bc19fd20 con 0x7f83bc100de0 2024-09-11T21:51:04.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:51:04.097+0000 7f83c1b90640 1 --2- 172.21.15.53:0/268783359 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83bc1020a0 0x7f83bc19f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T21:51:04.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:51:04.098+0000 7f83c138f640 1 --2- 172.21.15.53:0/268783359 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83bc100de0 0x7f83bc1980f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f83b800b450 tx=0x7f83b800b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:51:04.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:51:04.098+0000 7f83ae7fc640 1 -- 172.21.15.53:0/268783359 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b8015020 con 0x7f83bc100de0 2024-09-11T21:51:04.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:51:04.098+0000 7f83ae7fc640 1 -- 172.21.15.53:0/268783359 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83b8011d70 con 0x7f83bc100de0 2024-09-11T21:51:04.105 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:51:04.098+0000 7f83c35f3640 1 -- 172.21.15.53:0/268783359 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83bc19ffb0 con 0x7f83bc100de0 2024-09-11T21:51:04.106 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:51:04.098+0000 7f83ae7fc640 1 -- 172.21.15.53:0/268783359 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b800f340 con 0x7f83bc100de0 2024-09-11T21:51:04.106 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:51:04.098+0000 7f83c35f3640 1 -- 172.21.15.53:0/268783359 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83bc1a04e0 con 0x7f83bc100de0 2024-09-11T21:51:04.106 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:51:04.099+0000 7f83c35f3640 1 -- 172.21.15.53:0/268783359 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83bc108080 con 0x7f83bc100de0 2024-09-11T21:51:04.106 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:51:04.100+0000 7f83ae7fc640 1 -- 172.21.15.53:0/268783359 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f83b8018020 con 0x7f83bc100de0 2024-09-11T21:51:04.106 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:51:04.101+0000 7f83ae7fc640 1 --2- 172.21.15.53:0/268783359 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f83a80a2bb0 0x7f83a80a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:51:04.106 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:51:04.101+0000 7f83ae7fc640 1 -- 172.21.15.53:0/268783359 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 3698+0+0 (secure 0 0 0) 0x7f83b80cd910 con 0x7f83bc100de0 2024-09-11T21:51:04.106 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:51:04.101+0000 7f83c0b8e640 1 --2- 172.21.15.53:0/268783359 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f83a80a2bb0 0x7f83a80a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:51:04.106 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:51:04.102+0000 7f83c0b8e640 1 --2- 172.21.15.53:0/268783359 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f83a80a2bb0 0x7f83a80a5070 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f83bc199610 tx=0x7f83b0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:51:04.107 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:51:04.105+0000 7f83ae7fc640 1 -- 172.21.15.53:0/268783359 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f83b801f020 con 0x7f83bc100de0 2024-09-11T21:51:04.291 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:51:04.290+0000 7f83c35f3640 1 -- 172.21.15.53:0/268783359 --> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f83bc062630 con 0x7f83a80a2bb0 2024-09-11T21:51:04.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:04 smithi053 ceph-mon[34908]: purged_snaps scrub starts 2024-09-11T21:51:04.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:04 smithi053 ceph-mon[34908]: purged_snaps scrub ok 2024-09-11T21:51:04.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:04 smithi053 ceph-mon[34908]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-11T21:51:04.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:04 smithi053 ceph-mon[34908]: Detected new or changed devices on smithi001 2024-09-11T21:51:04.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:04 smithi053 ceph-mon[34908]: Adjusting osd_memory_target on smithi001 to 8546M 2024-09-11T21:51:04.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:04 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T21:51:04.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:04 smithi053 ceph-mon[34908]: osd.3 [v2:172.21.15.1:6810/1541142398,v1:172.21.15.1:6811/1541142398] boot 2024-09-11T21:51:04.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:04 smithi053 ceph-mon[34908]: osdmap e25: 4 total, 4 up, 4 in 2024-09-11T21:51:04.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:04 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T21:51:04.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:04 smithi053 ceph-mon[34908]: osdmap e26: 4 total, 4 up, 4 in 2024-09-11T21:51:04.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:04 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T21:51:04.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:04 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T21:51:04.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:04 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:51:04.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:04 smithi001 ceph-mon[30935]: purged_snaps scrub starts 2024-09-11T21:51:04.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:04 smithi001 ceph-mon[30935]: purged_snaps scrub ok 2024-09-11T21:51:04.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:04 smithi001 ceph-mon[30935]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-11T21:51:04.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:04 smithi001 ceph-mon[30935]: Detected new or changed devices on smithi001 2024-09-11T21:51:04.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:04 smithi001 ceph-mon[30935]: Adjusting osd_memory_target on smithi001 to 8546M 2024-09-11T21:51:04.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:04 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T21:51:04.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:04 smithi001 ceph-mon[30935]: osd.3 [v2:172.21.15.1:6810/1541142398,v1:172.21.15.1:6811/1541142398] boot 2024-09-11T21:51:04.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:04 smithi001 ceph-mon[30935]: osdmap e25: 4 total, 4 up, 4 in 2024-09-11T21:51:04.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:04 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T21:51:04.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:04 smithi001 ceph-mon[30935]: osdmap e26: 4 total, 4 up, 4 in 2024-09-11T21:51:04.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:04 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T21:51:04.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:04 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T21:51:04.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:04 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:51:04.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:04 smithi161 ceph-mon[35615]: purged_snaps scrub starts 2024-09-11T21:51:04.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:04 smithi161 ceph-mon[35615]: purged_snaps scrub ok 2024-09-11T21:51:04.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:04 smithi161 ceph-mon[35615]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-11T21:51:04.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:04 smithi161 ceph-mon[35615]: Detected new or changed devices on smithi001 2024-09-11T21:51:04.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:04 smithi161 ceph-mon[35615]: Adjusting osd_memory_target on smithi001 to 8546M 2024-09-11T21:51:04.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:04 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T21:51:04.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:04 smithi161 ceph-mon[35615]: osd.3 [v2:172.21.15.1:6810/1541142398,v1:172.21.15.1:6811/1541142398] boot 2024-09-11T21:51:04.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:04 smithi161 ceph-mon[35615]: osdmap e25: 4 total, 4 up, 4 in 2024-09-11T21:51:04.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:04 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T21:51:04.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:04 smithi161 ceph-mon[35615]: osdmap e26: 4 total, 4 up, 4 in 2024-09-11T21:51:04.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:04 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T21:51:04.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:04 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T21:51:04.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:04 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:51:05.676 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:05 smithi053 ceph-mon[34908]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:51:05.676 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:05 smithi053 ceph-mon[34908]: osdmap e27: 4 total, 4 up, 4 in 2024-09-11T21:51:05.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:05 smithi001 ceph-mon[30935]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:51:05.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:05 smithi001 ceph-mon[30935]: osdmap e27: 4 total, 4 up, 4 in 2024-09-11T21:51:05.743 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:05 smithi161 ceph-mon[35615]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:51:05.743 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:05 smithi161 ceph-mon[35615]: osdmap e27: 4 total, 4 up, 4 in 2024-09-11T21:51:06.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:06 smithi001 ceph-mon[30935]: pgmap v97: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-11T21:51:06.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:06 smithi001 ceph-mon[30935]: osdmap e28: 4 total, 4 up, 4 in 2024-09-11T21:51:06.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:06 smithi053 ceph-mon[34908]: pgmap v97: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-11T21:51:06.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:06 smithi053 ceph-mon[34908]: osdmap e28: 4 total, 4 up, 4 in 2024-09-11T21:51:06.841 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:06 smithi161 ceph-mon[35615]: pgmap v97: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-11T21:51:06.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:06 smithi161 ceph-mon[35615]: osdmap e28: 4 total, 4 up, 4 in 2024-09-11T21:51:07.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:07 smithi161 ceph-mon[35615]: osdmap e29: 4 total, 4 up, 4 in 2024-09-11T21:51:07.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:07 smithi161 ceph-mon[35615]: pgmap v100: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-11T21:51:07.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:07 smithi161 ceph-mon[35615]: from='client.? 172.21.15.53:0/1665416058' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56c808bb-d4cd-4969-b49e-adef49ecd0f5"}]: dispatch 2024-09-11T21:51:07.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:07 smithi161 ceph-mon[35615]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56c808bb-d4cd-4969-b49e-adef49ecd0f5"}]: dispatch 2024-09-11T21:51:07.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:07 smithi161 ceph-mon[35615]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "56c808bb-d4cd-4969-b49e-adef49ecd0f5"}]': finished 2024-09-11T21:51:07.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:07 smithi161 ceph-mon[35615]: osdmap e30: 5 total, 4 up, 5 in 2024-09-11T21:51:07.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:07 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T21:51:07.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:07 smithi161 ceph-mon[35615]: from='client.? 172.21.15.53:0/2477226644' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T21:51:07.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:07 smithi001 ceph-mon[30935]: osdmap e29: 4 total, 4 up, 4 in 2024-09-11T21:51:07.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:07 smithi001 ceph-mon[30935]: pgmap v100: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-11T21:51:07.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:07 smithi001 ceph-mon[30935]: from='client.? 172.21.15.53:0/1665416058' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56c808bb-d4cd-4969-b49e-adef49ecd0f5"}]: dispatch 2024-09-11T21:51:07.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:07 smithi001 ceph-mon[30935]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56c808bb-d4cd-4969-b49e-adef49ecd0f5"}]: dispatch 2024-09-11T21:51:07.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:07 smithi001 ceph-mon[30935]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "56c808bb-d4cd-4969-b49e-adef49ecd0f5"}]': finished 2024-09-11T21:51:07.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:07 smithi001 ceph-mon[30935]: osdmap e30: 5 total, 4 up, 5 in 2024-09-11T21:51:07.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:07 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T21:51:07.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:07 smithi001 ceph-mon[30935]: from='client.? 172.21.15.53:0/2477226644' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T21:51:08.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:07 smithi053 ceph-mon[34908]: osdmap e29: 4 total, 4 up, 4 in 2024-09-11T21:51:08.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:07 smithi053 ceph-mon[34908]: pgmap v100: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-11T21:51:08.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:07 smithi053 ceph-mon[34908]: from='client.? 172.21.15.53:0/1665416058' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56c808bb-d4cd-4969-b49e-adef49ecd0f5"}]: dispatch 2024-09-11T21:51:08.048 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:07 smithi053 ceph-mon[34908]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56c808bb-d4cd-4969-b49e-adef49ecd0f5"}]: dispatch 2024-09-11T21:51:08.048 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:07 smithi053 ceph-mon[34908]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "56c808bb-d4cd-4969-b49e-adef49ecd0f5"}]': finished 2024-09-11T21:51:08.048 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:07 smithi053 ceph-mon[34908]: osdmap e30: 5 total, 4 up, 5 in 2024-09-11T21:51:08.048 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:07 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T21:51:08.048 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:07 smithi053 ceph-mon[34908]: from='client.? 172.21.15.53:0/2477226644' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T21:51:09.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:09 smithi001 ceph-mon[30935]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-11T21:51:10.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:09 smithi053 ceph-mon[34908]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-11T21:51:10.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:09 smithi161 ceph-mon[35615]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-11T21:51:11.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:11 smithi053 ceph-mon[34908]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T21:51:11.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:11 smithi001 ceph-mon[30935]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T21:51:12.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:11 smithi161 ceph-mon[35615]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T21:51:13.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:13 smithi001 ceph-mon[30935]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-11T21:51:14.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:13 smithi053 ceph-mon[34908]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-11T21:51:14.091 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:13 smithi161 ceph-mon[35615]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-11T21:51:15.875 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:15 smithi053 ceph-mon[34908]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T21:51:15.875 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:15 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-11T21:51:15.875 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:15 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:51:15.875 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:15 smithi053 ceph-mon[34908]: Deploying daemon osd.4 on smithi053 2024-09-11T21:51:15.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:15 smithi001 ceph-mon[30935]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T21:51:15.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:15 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-11T21:51:15.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:15 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:51:15.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:15 smithi001 ceph-mon[30935]: Deploying daemon osd.4 on smithi053 2024-09-11T21:51:16.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:15 smithi161 ceph-mon[35615]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T21:51:16.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:15 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-11T21:51:16.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:15 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:51:16.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:15 smithi161 ceph-mon[35615]: Deploying daemon osd.4 on smithi053 2024-09-11T21:51:17.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:17 smithi001 ceph-mon[30935]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T21:51:18.048 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:17 smithi053 ceph-mon[34908]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T21:51:18.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:17 smithi161 ceph-mon[35615]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T21:51:19.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:19 smithi001 ceph-mon[30935]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T21:51:20.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:19 smithi053 ceph-mon[34908]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T21:51:20.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:19 smithi161 ceph-mon[35615]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T21:51:21.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:21 smithi001 ceph-mon[30935]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T21:51:22.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:21 smithi053 ceph-mon[34908]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T21:51:22.091 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:21 smithi161 ceph-mon[35615]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T21:51:22.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:22 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:51:22.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:22 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:22.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:22 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:22.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:22 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:51:22.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:22 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:22.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:22 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:23.091 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:22 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:51:23.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:22 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:23.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:22 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:24.085 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:23 smithi053 ceph-mon[34908]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-11T21:51:24.091 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:23 smithi161 ceph-mon[35615]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-11T21:51:24.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:23 smithi001 ceph-mon[30935]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-11T21:51:25.785 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:25 smithi053 ceph-mon[34908]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-11T21:51:25.785 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:25 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:25.785 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:25 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:25.786 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:25 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:51:25.786 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:25 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:51:25.786 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:25 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:25.837 INFO:teuthology.orchestra.run.smithi053.stdout:Created osd(s) 4 on host 'smithi053' 2024-09-11T21:51:25.837 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:51:25.835+0000 7f83ae7fc640 1 -- 172.21.15.53:0/268783359 <== mgr.14150 v2:172.21.15.1:6800/3890350886 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f83bc062630 con 0x7f83a80a2bb0 2024-09-11T21:51:25.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:51:25.838+0000 7f83c35f3640 1 -- 172.21.15.53:0/268783359 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f83a80a2bb0 msgr2=0x7f83a80a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:51:25.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:51:25.838+0000 7f83c35f3640 1 --2- 172.21.15.53:0/268783359 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f83a80a2bb0 0x7f83a80a5070 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f83bc199610 tx=0x7f83b0008040 comp rx=0 tx=0).stop 2024-09-11T21:51:25.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:51:25.838+0000 7f83c35f3640 1 -- 172.21.15.53:0/268783359 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83bc100de0 msgr2=0x7f83bc1980f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:51:25.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:51:25.838+0000 7f83c35f3640 1 --2- 172.21.15.53:0/268783359 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83bc100de0 0x7f83bc1980f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f83b800b450 tx=0x7f83b800b910 comp rx=0 tx=0).stop 2024-09-11T21:51:25.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:51:25.838+0000 7f83c35f3640 1 -- 172.21.15.53:0/268783359 shutdown_connections 2024-09-11T21:51:25.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:51:25.838+0000 7f83c35f3640 1 --2- 172.21.15.53:0/268783359 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f83a80a2bb0 0x7f83a80a5070 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:51:25.839 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:51:25.838+0000 7f83c35f3640 1 --2- 172.21.15.53:0/268783359 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83bc1020a0 0x7f83bc19f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:51:25.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:51:25.838+0000 7f83c35f3640 1 --2- 172.21.15.53:0/268783359 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83bc1017a0 0x7f83bc198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:51:25.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:51:25.838+0000 7f83c35f3640 1 --2- 172.21.15.53:0/268783359 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f83bc100de0 0x7f83bc1980f0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:51:25.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:51:25.838+0000 7f83c35f3640 1 -- 172.21.15.53:0/268783359 >> 172.21.15.53:0/268783359 conn(0x7f83bc0fba40 msgr2=0x7f83bc105250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:51:25.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:51:25.838+0000 7f83c35f3640 1 -- 172.21.15.53:0/268783359 shutdown_connections 2024-09-11T21:51:25.840 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:51:25.838+0000 7f83c35f3640 1 -- 172.21.15.53:0/268783359 wait complete. 2024-09-11T21:51:25.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:25 smithi161 ceph-mon[35615]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-11T21:51:25.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:25 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:25.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:25 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:25.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:25 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:51:25.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:25 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:51:25.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:25 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:25.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:25 smithi001 ceph-mon[30935]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-11T21:51:25.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:25 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:25.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:25 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:25.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:25 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:51:25.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:25 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:51:25.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:25 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:26.604 DEBUG:teuthology.orchestra.run.smithi053:osd.4> sudo journalctl -f -n 0 -u ceph-3ffca83c-7087-11ef-bcea-c7b262605968@osd.4.service 2024-09-11T21:51:26.606 INFO:tasks.cephadm:Deploying osd.5 on smithi161 with /dev/vg_nvme/lv_3... 2024-09-11T21:51:26.607 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-11T21:51:26.841 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:26 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:51:26.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:26 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:26.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:26 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:26.860 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T21:51:26.863 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:26 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:51:26.863 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:26 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:26.863 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:26 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:26.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:26 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:51:26.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:26 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:26.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:26 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:27.298 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 11 21:51:26 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-4[45992]: 2024-09-11T21:51:26.880+0000 7fef08f09640 -1 osd.4 0 log_to_monitors true 2024-09-11T21:51:27.995 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:27 smithi053 ceph-mon[34908]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-11T21:51:27.995 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:27 smithi053 ceph-mon[34908]: from='osd.4 [v2:172.21.15.53:6808/3081402681,v1:172.21.15.53:6809/3081402681]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T21:51:27.995 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:27 smithi053 ceph-mon[34908]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T21:51:28.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:27 smithi161 ceph-mon[35615]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-11T21:51:28.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:27 smithi161 ceph-mon[35615]: from='osd.4 [v2:172.21.15.53:6808/3081402681,v1:172.21.15.53:6809/3081402681]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T21:51:28.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:27 smithi161 ceph-mon[35615]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T21:51:28.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:27 smithi001 ceph-mon[30935]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-11T21:51:28.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:27 smithi001 ceph-mon[30935]: from='osd.4 [v2:172.21.15.53:6808/3081402681,v1:172.21.15.53:6809/3081402681]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T21:51:28.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:27 smithi001 ceph-mon[30935]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T21:51:29.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:28 smithi053 ceph-mon[34908]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-11T21:51:29.048 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:28 smithi053 ceph-mon[34908]: from='osd.4 [v2:172.21.15.53:6808/3081402681,v1:172.21.15.53:6809/3081402681]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-09-11T21:51:29.048 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:28 smithi053 ceph-mon[34908]: osdmap e31: 5 total, 4 up, 5 in 2024-09-11T21:51:29.048 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:28 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T21:51:29.048 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:28 smithi053 ceph-mon[34908]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-09-11T21:51:29.048 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 11 21:51:28 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-4[45992]: 2024-09-11T21:51:28.842+0000 7fef04e8c640 -1 osd.4 0 waiting for initial osdmap 2024-09-11T21:51:29.048 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 11 21:51:28 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-4[45992]: 2024-09-11T21:51:28.848+0000 7fef00cb4640 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T21:51:29.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:28 smithi161 ceph-mon[35615]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-11T21:51:29.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:28 smithi161 ceph-mon[35615]: from='osd.4 [v2:172.21.15.53:6808/3081402681,v1:172.21.15.53:6809/3081402681]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-09-11T21:51:29.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:28 smithi161 ceph-mon[35615]: osdmap e31: 5 total, 4 up, 5 in 2024-09-11T21:51:29.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:28 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T21:51:29.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:28 smithi161 ceph-mon[35615]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-09-11T21:51:29.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:28 smithi001 ceph-mon[30935]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-11T21:51:29.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:28 smithi001 ceph-mon[30935]: from='osd.4 [v2:172.21.15.53:6808/3081402681,v1:172.21.15.53:6809/3081402681]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-09-11T21:51:29.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:28 smithi001 ceph-mon[30935]: osdmap e31: 5 total, 4 up, 5 in 2024-09-11T21:51:29.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:28 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T21:51:29.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:28 smithi001 ceph-mon[30935]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-09-11T21:51:29.400 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-11T21:51:29.426 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi161:vg_nvme/lv_3 2024-09-11T21:51:29.684 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T21:51:29.965 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:29 smithi161 ceph-mon[35615]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-11T21:51:29.965 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:29 smithi161 ceph-mon[35615]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-09-11T21:51:29.965 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:29 smithi161 ceph-mon[35615]: osdmap e32: 5 total, 4 up, 5 in 2024-09-11T21:51:29.965 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:29 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T21:51:29.966 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:29 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T21:51:29.966 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:29 smithi161 ceph-mon[35615]: Detected new or changed devices on smithi053 2024-09-11T21:51:29.966 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:29 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:29.966 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:29 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:29.966 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:29 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:51:29.966 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:29 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:51:29.967 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:29 smithi161 ceph-mon[35615]: Adjusting osd_memory_target on smithi053 to 8546M 2024-09-11T21:51:29.967 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:29 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:29.967 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:29 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:51:29.967 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:29 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:51:29.967 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:29 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:30.192 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:51:30.178+0000 7f0d2b147640 1 -- 172.21.15.161:0/2251102257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d24102e20 msgr2=0x7f0d241032a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:51:30.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:51:30.178+0000 7f0d2b147640 1 --2- 172.21.15.161:0/2251102257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d24102e20 0x7f0d241032a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0d10009900 tx=0x7f0d1002a410 comp rx=0 tx=0).stop 2024-09-11T21:51:30.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:51:30.178+0000 7f0d2b147640 1 -- 172.21.15.161:0/2251102257 shutdown_connections 2024-09-11T21:51:30.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:51:30.178+0000 7f0d2b147640 1 --2- 172.21.15.161:0/2251102257 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d241037e0 0x7f0d2410a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:51:30.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:51:30.178+0000 7f0d2b147640 1 --2- 172.21.15.161:0/2251102257 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d24102e20 0x7f0d241032a0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:51:30.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:51:30.178+0000 7f0d2b147640 1 --2- 172.21.15.161:0/2251102257 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d24101c20 0x7f0d24102020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:51:30.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:51:30.178+0000 7f0d2b147640 1 -- 172.21.15.161:0/2251102257 >> 172.21.15.161:0/2251102257 conn(0x7f0d240fd3b0 msgr2=0x7f0d240ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:51:30.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:51:30.178+0000 7f0d2b147640 1 -- 172.21.15.161:0/2251102257 shutdown_connections 2024-09-11T21:51:30.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:51:30.178+0000 7f0d2b147640 1 -- 172.21.15.161:0/2251102257 wait complete. 2024-09-11T21:51:30.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:51:30.179+0000 7f0d2b147640 1 Processor -- start 2024-09-11T21:51:30.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:51:30.179+0000 7f0d2b147640 1 -- start start 2024-09-11T21:51:30.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:51:30.179+0000 7f0d2b147640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d24101c20 0x7f0d24197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:51:30.193 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:51:30.179+0000 7f0d2b147640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d24102e20 0x7f0d24198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:51:30.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:51:30.179+0000 7f0d2b147640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d241037e0 0x7f0d2419f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:51:30.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:51:30.179+0000 7f0d2b147640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0d2410d8a0 con 0x7f0d24102e20 2024-09-11T21:51:30.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:51:30.179+0000 7f0d2b147640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0d2410d720 con 0x7f0d24101c20 2024-09-11T21:51:30.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:51:30.179+0000 7f0d2b147640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0d2410da20 con 0x7f0d241037e0 2024-09-11T21:51:30.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:51:30.179+0000 7f0d296e4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d241037e0 0x7f0d2419f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:51:30.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:51:30.179+0000 7f0d296e4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d241037e0 0x7f0d2419f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:42276/0 (socket says 172.21.15.161:42276) 2024-09-11T21:51:30.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:51:30.179+0000 7f0d296e4640 1 -- 172.21.15.161:0/1943932784 learned_addr learned my addr 172.21.15.161:0/1943932784 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-09-11T21:51:30.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:51:30.179+0000 7f0d28ee3640 1 --2- 172.21.15.161:0/1943932784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d24101c20 0x7f0d24197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:51:30.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:51:30.179+0000 7f0d23fff640 1 --2- 172.21.15.161:0/1943932784 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d24102e20 0x7f0d24198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:51:30.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:51:30.179+0000 7f0d296e4640 1 -- 172.21.15.161:0/1943932784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d24101c20 msgr2=0x7f0d24197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:51:30.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:51:30.179+0000 7f0d296e4640 1 --2- 172.21.15.161:0/1943932784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d24101c20 0x7f0d24197ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:51:30.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:51:30.179+0000 7f0d296e4640 1 -- 172.21.15.161:0/1943932784 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d24102e20 msgr2=0x7f0d24198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:51:30.194 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:51:30.179+0000 7f0d296e4640 1 --2- 172.21.15.161:0/1943932784 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d24102e20 0x7f0d24198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:51:30.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:51:30.179+0000 7f0d296e4640 1 -- 172.21.15.161:0/1943932784 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d2419fb20 con 0x7f0d241037e0 2024-09-11T21:51:30.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:51:30.179+0000 7f0d296e4640 1 --2- 172.21.15.161:0/1943932784 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d241037e0 0x7f0d2419f4b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c00eaa0 tx=0x7f0d1c00ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:51:30.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:51:30.180+0000 7f0d21ffb640 1 -- 172.21.15.161:0/1943932784 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d1c00cdb0 con 0x7f0d241037e0 2024-09-11T21:51:30.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:51:30.180+0000 7f0d21ffb640 1 -- 172.21.15.161:0/1943932784 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d1c005020 con 0x7f0d241037e0 2024-09-11T21:51:30.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:51:30.180+0000 7f0d2b147640 1 -- 172.21.15.161:0/1943932784 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d2419fe10 con 0x7f0d241037e0 2024-09-11T21:51:30.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:51:30.180+0000 7f0d21ffb640 1 -- 172.21.15.161:0/1943932784 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d1c0116b0 con 0x7f0d241037e0 2024-09-11T21:51:30.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:51:30.180+0000 7f0d2b147640 1 -- 172.21.15.161:0/1943932784 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d241a0310 con 0x7f0d241037e0 2024-09-11T21:51:30.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:51:30.181+0000 7f0d2b147640 1 -- 172.21.15.161:0/1943932784 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d24108ff0 con 0x7f0d241037e0 2024-09-11T21:51:30.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:51:30.182+0000 7f0d21ffb640 1 -- 172.21.15.161:0/1943932784 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f0d1c005560 con 0x7f0d241037e0 2024-09-11T21:51:30.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:51:30.182+0000 7f0d21ffb640 1 --2- 172.21.15.161:0/1943932784 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f0d140a2c30 0x7f0d140a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:51:30.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:51:30.183+0000 7f0d21ffb640 1 -- 172.21.15.161:0/1943932784 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4101+0+0 (secure 0 0 0) 0x7f0d1c0ce0a0 con 0x7f0d241037e0 2024-09-11T21:51:30.195 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:51:30.183+0000 7f0d28ee3640 1 --2- 172.21.15.161:0/1943932784 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f0d140a2c30 0x7f0d140a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:51:30.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:51:30.183+0000 7f0d28ee3640 1 --2- 172.21.15.161:0/1943932784 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f0d140a2c30 0x7f0d140a50f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0d24102c80 tx=0x7f0d1800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:51:30.196 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:51:30.187+0000 7f0d21ffb640 1 -- 172.21.15.161:0/1943932784 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0d1c016030 con 0x7f0d241037e0 2024-09-11T21:51:30.196 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:29 smithi001 ceph-mon[30935]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-11T21:51:30.196 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:29 smithi001 ceph-mon[30935]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-09-11T21:51:30.196 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:29 smithi001 ceph-mon[30935]: osdmap e32: 5 total, 4 up, 5 in 2024-09-11T21:51:30.196 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:29 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T21:51:30.196 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:29 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T21:51:30.197 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:29 smithi001 ceph-mon[30935]: Detected new or changed devices on smithi053 2024-09-11T21:51:30.197 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:29 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:30.197 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:29 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:30.197 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:29 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:51:30.197 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:29 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:51:30.197 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:29 smithi001 ceph-mon[30935]: Adjusting osd_memory_target on smithi053 to 8546M 2024-09-11T21:51:30.197 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:29 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:30.197 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:29 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:51:30.197 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:29 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:51:30.197 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:29 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:30.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:29 smithi053 ceph-mon[34908]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-11T21:51:30.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:29 smithi053 ceph-mon[34908]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-09-11T21:51:30.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:29 smithi053 ceph-mon[34908]: osdmap e32: 5 total, 4 up, 5 in 2024-09-11T21:51:30.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:29 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T21:51:30.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:29 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T21:51:30.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:29 smithi053 ceph-mon[34908]: Detected new or changed devices on smithi053 2024-09-11T21:51:30.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:29 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:30.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:29 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:30.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:29 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:51:30.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:29 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:51:30.300 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:29 smithi053 ceph-mon[34908]: Adjusting osd_memory_target on smithi053 to 8546M 2024-09-11T21:51:30.300 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:29 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:30.300 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:29 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:51:30.300 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:29 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:51:30.300 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:29 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:30.371 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:51:30.370+0000 7f0d2b147640 1 -- 172.21.15.161:0/1943932784 --> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f0d24062630 con 0x7f0d140a2c30 2024-09-11T21:51:31.053 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:30 smithi161 ceph-mon[35615]: purged_snaps scrub starts 2024-09-11T21:51:31.054 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:30 smithi161 ceph-mon[35615]: purged_snaps scrub ok 2024-09-11T21:51:31.054 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:30 smithi161 ceph-mon[35615]: osd.4 [v2:172.21.15.53:6808/3081402681,v1:172.21.15.53:6809/3081402681] boot 2024-09-11T21:51:31.054 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:30 smithi161 ceph-mon[35615]: osdmap e33: 5 total, 5 up, 5 in 2024-09-11T21:51:31.054 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:30 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T21:51:31.054 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:30 smithi161 ceph-mon[35615]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:51:31.054 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:30 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T21:51:31.055 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:30 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T21:51:31.055 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:30 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:51:31.117 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:30 smithi001 ceph-mon[30935]: purged_snaps scrub starts 2024-09-11T21:51:31.117 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:30 smithi001 ceph-mon[30935]: purged_snaps scrub ok 2024-09-11T21:51:31.117 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:30 smithi001 ceph-mon[30935]: osd.4 [v2:172.21.15.53:6808/3081402681,v1:172.21.15.53:6809/3081402681] boot 2024-09-11T21:51:31.117 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:30 smithi001 ceph-mon[30935]: osdmap e33: 5 total, 5 up, 5 in 2024-09-11T21:51:31.117 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:30 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T21:51:31.117 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:30 smithi001 ceph-mon[30935]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:51:31.117 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:30 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T21:51:31.117 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:30 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T21:51:31.117 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:30 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:51:31.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:30 smithi053 ceph-mon[34908]: purged_snaps scrub starts 2024-09-11T21:51:31.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:30 smithi053 ceph-mon[34908]: purged_snaps scrub ok 2024-09-11T21:51:31.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:30 smithi053 ceph-mon[34908]: osd.4 [v2:172.21.15.53:6808/3081402681,v1:172.21.15.53:6809/3081402681] boot 2024-09-11T21:51:31.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:30 smithi053 ceph-mon[34908]: osdmap e33: 5 total, 5 up, 5 in 2024-09-11T21:51:31.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:30 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T21:51:31.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:30 smithi053 ceph-mon[34908]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:51:31.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:30 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T21:51:31.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:30 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T21:51:31.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:30 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:51:32.091 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:31 smithi161 ceph-mon[35615]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-11T21:51:32.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:31 smithi161 ceph-mon[35615]: osdmap e34: 5 total, 5 up, 5 in 2024-09-11T21:51:32.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:31 smithi001 ceph-mon[30935]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-11T21:51:32.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:31 smithi001 ceph-mon[30935]: osdmap e34: 5 total, 5 up, 5 in 2024-09-11T21:51:32.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:31 smithi053 ceph-mon[34908]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-11T21:51:32.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:31 smithi053 ceph-mon[34908]: osdmap e34: 5 total, 5 up, 5 in 2024-09-11T21:51:33.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:32 smithi001 ceph-mon[30935]: from='client.? 172.21.15.161:0/2252758926' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b8266d6-b507-47c9-b4fa-ff16c4a74d35"}]: dispatch 2024-09-11T21:51:33.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:32 smithi001 ceph-mon[30935]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b8266d6-b507-47c9-b4fa-ff16c4a74d35"}]: dispatch 2024-09-11T21:51:33.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:32 smithi001 ceph-mon[30935]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7b8266d6-b507-47c9-b4fa-ff16c4a74d35"}]': finished 2024-09-11T21:51:33.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:32 smithi001 ceph-mon[30935]: osdmap e35: 6 total, 5 up, 6 in 2024-09-11T21:51:33.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:32 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T21:51:33.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:32 smithi053 ceph-mon[34908]: from='client.? 172.21.15.161:0/2252758926' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b8266d6-b507-47c9-b4fa-ff16c4a74d35"}]: dispatch 2024-09-11T21:51:33.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:32 smithi053 ceph-mon[34908]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b8266d6-b507-47c9-b4fa-ff16c4a74d35"}]: dispatch 2024-09-11T21:51:33.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:32 smithi053 ceph-mon[34908]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7b8266d6-b507-47c9-b4fa-ff16c4a74d35"}]': finished 2024-09-11T21:51:33.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:32 smithi053 ceph-mon[34908]: osdmap e35: 6 total, 5 up, 6 in 2024-09-11T21:51:33.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:32 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T21:51:33.314 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:32 smithi161 ceph-mon[35615]: from='client.? 172.21.15.161:0/2252758926' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b8266d6-b507-47c9-b4fa-ff16c4a74d35"}]: dispatch 2024-09-11T21:51:33.314 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:32 smithi161 ceph-mon[35615]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b8266d6-b507-47c9-b4fa-ff16c4a74d35"}]: dispatch 2024-09-11T21:51:33.314 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:32 smithi161 ceph-mon[35615]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7b8266d6-b507-47c9-b4fa-ff16c4a74d35"}]': finished 2024-09-11T21:51:33.315 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:32 smithi161 ceph-mon[35615]: osdmap e35: 6 total, 5 up, 6 in 2024-09-11T21:51:33.315 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:32 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T21:51:34.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:33 smithi001 ceph-mon[30935]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T21:51:34.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:33 smithi001 ceph-mon[30935]: from='client.? 172.21.15.161:0/2058342929' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T21:51:34.198 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:33 smithi053 ceph-mon[34908]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T21:51:34.198 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:33 smithi053 ceph-mon[34908]: from='client.? 172.21.15.161:0/2058342929' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T21:51:34.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:33 smithi161 ceph-mon[35615]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T21:51:34.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:33 smithi161 ceph-mon[35615]: from='client.? 172.21.15.161:0/2058342929' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T21:51:36.144 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:35 smithi161 ceph-mon[35615]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T21:51:36.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:35 smithi001 ceph-mon[30935]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T21:51:36.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:35 smithi053 ceph-mon[34908]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T21:51:38.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:37 smithi001 ceph-mon[30935]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T21:51:38.274 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:37 smithi161 ceph-mon[35615]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T21:51:38.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:37 smithi053 ceph-mon[34908]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T21:51:40.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:39 smithi001 ceph-mon[30935]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T21:51:40.209 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:39 smithi161 ceph-mon[35615]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T21:51:40.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:39 smithi053 ceph-mon[34908]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T21:51:42.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:41 smithi001 ceph-mon[30935]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T21:51:42.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:41 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-11T21:51:42.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:41 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:51:42.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:41 smithi001 ceph-mon[30935]: Deploying daemon osd.5 on smithi161 2024-09-11T21:51:42.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:41 smithi053 ceph-mon[34908]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T21:51:42.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:41 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-11T21:51:42.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:41 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:51:42.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:41 smithi053 ceph-mon[34908]: Deploying daemon osd.5 on smithi161 2024-09-11T21:51:42.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:41 smithi161 ceph-mon[35615]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T21:51:42.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:41 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-11T21:51:42.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:41 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:51:42.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:41 smithi161 ceph-mon[35615]: Deploying daemon osd.5 on smithi161 2024-09-11T21:51:44.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:43 smithi001 ceph-mon[30935]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T21:51:44.230 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:43 smithi161 ceph-mon[35615]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T21:51:44.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:43 smithi053 ceph-mon[34908]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T21:51:46.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:45 smithi161 ceph-mon[35615]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T21:51:46.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:45 smithi001 ceph-mon[30935]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T21:51:46.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:45 smithi053 ceph-mon[34908]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T21:51:48.041 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:47 smithi161 ceph-mon[35615]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T21:51:48.041 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:47 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:51:48.041 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:47 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:48.041 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:47 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:48.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:47 smithi001 ceph-mon[30935]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T21:51:48.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:47 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:51:48.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:47 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:48.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:47 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:48.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:47 smithi053 ceph-mon[34908]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T21:51:48.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:47 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:51:48.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:47 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:48.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:47 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:49.976 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:49 smithi161 ceph-mon[35615]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T21:51:50.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:49 smithi001 ceph-mon[30935]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T21:51:50.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:49 smithi053 ceph-mon[34908]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T21:51:51.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:51 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:51.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:51 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:51.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:51 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:51:51.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:51 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:51:51.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:51 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:51.463 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:51 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:51.463 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:51 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:51.463 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:51 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:51:51.463 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:51 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:51:51.464 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:51 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:51.502 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:51:51.501+0000 7f0d21ffb640 1 -- 172.21.15.161:0/1943932784 <== mgr.14150 v2:172.21.15.1:6800/3890350886 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0d24062630 con 0x7f0d140a2c30 2024-09-11T21:51:51.503 INFO:teuthology.orchestra.run.smithi161.stdout:Created osd(s) 5 on host 'smithi161' 2024-09-11T21:51:51.505 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:51:51.503+0000 7f0d2b147640 1 -- 172.21.15.161:0/1943932784 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f0d140a2c30 msgr2=0x7f0d140a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:51:51.505 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:51:51.503+0000 7f0d2b147640 1 --2- 172.21.15.161:0/1943932784 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f0d140a2c30 0x7f0d140a50f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0d24102c80 tx=0x7f0d1800b390 comp rx=0 tx=0).stop 2024-09-11T21:51:51.505 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:51:51.503+0000 7f0d2b147640 1 -- 172.21.15.161:0/1943932784 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d241037e0 msgr2=0x7f0d2419f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:51:51.505 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:51:51.503+0000 7f0d2b147640 1 --2- 172.21.15.161:0/1943932784 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d241037e0 0x7f0d2419f4b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c00eaa0 tx=0x7f0d1c00ef60 comp rx=0 tx=0).stop 2024-09-11T21:51:51.505 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:51:51.503+0000 7f0d2b147640 1 -- 172.21.15.161:0/1943932784 shutdown_connections 2024-09-11T21:51:51.505 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:51:51.503+0000 7f0d2b147640 1 --2- 172.21.15.161:0/1943932784 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f0d140a2c30 0x7f0d140a50f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:51:51.506 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:51:51.503+0000 7f0d2b147640 1 --2- 172.21.15.161:0/1943932784 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d241037e0 0x7f0d2419f4b0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:51:51.506 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:51:51.503+0000 7f0d2b147640 1 --2- 172.21.15.161:0/1943932784 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d24102e20 0x7f0d24198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:51:51.506 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:51:51.503+0000 7f0d2b147640 1 --2- 172.21.15.161:0/1943932784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d24101c20 0x7f0d24197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:51:51.506 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:51:51.503+0000 7f0d2b147640 1 -- 172.21.15.161:0/1943932784 >> 172.21.15.161:0/1943932784 conn(0x7f0d240fd3b0 msgr2=0x7f0d241061c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:51:51.506 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:51:51.503+0000 7f0d2b147640 1 -- 172.21.15.161:0/1943932784 shutdown_connections 2024-09-11T21:51:51.506 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:51:51.503+0000 7f0d2b147640 1 -- 172.21.15.161:0/1943932784 wait complete. 2024-09-11T21:51:51.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:51 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:51.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:51 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:51.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:51 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:51:51.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:51 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:51:51.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:51 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:52.144 DEBUG:teuthology.orchestra.run.smithi161:osd.5> sudo journalctl -f -n 0 -u ceph-3ffca83c-7087-11ef-bcea-c7b262605968@osd.5.service 2024-09-11T21:51:52.146 INFO:tasks.cephadm:Deploying osd.6 on smithi001 with /dev/vg_nvme/lv_2... 2024-09-11T21:51:52.147 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-11T21:51:52.414 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:51:52.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:52 smithi161 ceph-mon[35615]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T21:51:52.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:52 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:51:52.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:52 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:52.428 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:52 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:52.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:52 smithi001 ceph-mon[30935]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T21:51:52.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:52 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:51:52.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:52 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:52.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:52 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:52.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:52 smithi053 ceph-mon[34908]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T21:51:52.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:52 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:51:52.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:52 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:52.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:52 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:53.819 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 11 21:51:53 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-5[47459]: 2024-09-11T21:51:53.658+0000 7ff954e4f640 -1 osd.5 0 log_to_monitors true 2024-09-11T21:51:54.411 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:54 smithi001 ceph-mon[30935]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T21:51:54.411 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:54 smithi001 ceph-mon[30935]: from='osd.5 [v2:172.21.15.161:6808/1622081327,v1:172.21.15.161:6809/1622081327]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T21:51:54.411 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:54 smithi001 ceph-mon[30935]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T21:51:54.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:54 smithi053 ceph-mon[34908]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T21:51:54.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:54 smithi053 ceph-mon[34908]: from='osd.5 [v2:172.21.15.161:6808/1622081327,v1:172.21.15.161:6809/1622081327]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T21:51:54.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:54 smithi053 ceph-mon[34908]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T21:51:54.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:54 smithi161 ceph-mon[35615]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T21:51:54.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:54 smithi161 ceph-mon[35615]: from='osd.5 [v2:172.21.15.161:6808/1622081327,v1:172.21.15.161:6809/1622081327]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T21:51:54.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:54 smithi161 ceph-mon[35615]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T21:51:55.091 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:51:55.117 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi001:vg_nvme/lv_2 2024-09-11T21:51:55.342 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 11 21:51:55 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-5[47459]: 2024-09-11T21:51:55.159+0000 7ff950dd2640 -1 osd.5 0 waiting for initial osdmap 2024-09-11T21:51:55.342 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 11 21:51:55 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-5[47459]: 2024-09-11T21:51:55.165+0000 7ff94cbfa640 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T21:51:55.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:55 smithi161 ceph-mon[35615]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-11T21:51:55.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:55 smithi161 ceph-mon[35615]: osdmap e36: 6 total, 5 up, 6 in 2024-09-11T21:51:55.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:55 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T21:51:55.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:55 smithi161 ceph-mon[35615]: from='osd.5 [v2:172.21.15.161:6808/1622081327,v1:172.21.15.161:6809/1622081327]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-09-11T21:51:55.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:55 smithi161 ceph-mon[35615]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-09-11T21:51:55.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:55 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:55.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:55 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:55.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:55 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:51:55.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:55 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:51:55.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:55 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:55.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:55 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:51:55.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:55 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:51:55.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:55 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:55.372 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:51:55.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:55 smithi001 ceph-mon[30935]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-11T21:51:55.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:55 smithi001 ceph-mon[30935]: osdmap e36: 6 total, 5 up, 6 in 2024-09-11T21:51:55.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:55 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T21:51:55.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:55 smithi001 ceph-mon[30935]: from='osd.5 [v2:172.21.15.161:6808/1622081327,v1:172.21.15.161:6809/1622081327]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-09-11T21:51:55.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:55 smithi001 ceph-mon[30935]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-09-11T21:51:55.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:55 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:55.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:55 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:55.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:55 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:51:55.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:55 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:51:55.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:55 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:55.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:55 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:51:55.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:55 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:51:55.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:55 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:55.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:55 smithi053 ceph-mon[34908]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-11T21:51:55.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:55 smithi053 ceph-mon[34908]: osdmap e36: 6 total, 5 up, 6 in 2024-09-11T21:51:55.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:55 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T21:51:55.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:55 smithi053 ceph-mon[34908]: from='osd.5 [v2:172.21.15.161:6808/1622081327,v1:172.21.15.161:6809/1622081327]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-09-11T21:51:55.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:55 smithi053 ceph-mon[34908]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-09-11T21:51:55.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:55 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:55.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:55 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:55.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:55 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:51:55.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:55 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:51:55.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:55 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:55.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:55 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:51:55.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:55 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:51:55.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:55 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:51:55.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:51:55.852+0000 7f230a992640 1 -- 172.21.15.1:0/2535486714 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2304101d10 msgr2=0x7f2304102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:51:55.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:51:55.852+0000 7f230a992640 1 --2- 172.21.15.1:0/2535486714 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2304101d10 0x7f2304102190 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f22ec009900 tx=0x7f22ec02a410 comp rx=0 tx=0).stop 2024-09-11T21:51:55.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:51:55.852+0000 7f230a992640 1 -- 172.21.15.1:0/2535486714 shutdown_connections 2024-09-11T21:51:55.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:51:55.852+0000 7f230a992640 1 --2- 172.21.15.1:0/2535486714 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23041026d0 0x7f2304108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:51:55.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:51:55.852+0000 7f230a992640 1 --2- 172.21.15.1:0/2535486714 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2304101d10 0x7f2304102190 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:51:55.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:51:55.852+0000 7f230a992640 1 --2- 172.21.15.1:0/2535486714 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2304100b10 0x7f2304100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:51:55.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:51:55.852+0000 7f230a992640 1 -- 172.21.15.1:0/2535486714 >> 172.21.15.1:0/2535486714 conn(0x7f23040fc2a0 msgr2=0x7f23040fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:51:55.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:51:55.852+0000 7f230a992640 1 -- 172.21.15.1:0/2535486714 shutdown_connections 2024-09-11T21:51:55.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:51:55.852+0000 7f230a992640 1 -- 172.21.15.1:0/2535486714 wait complete. 2024-09-11T21:51:55.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:51:55.852+0000 7f230a992640 1 Processor -- start 2024-09-11T21:51:55.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:51:55.852+0000 7f230a992640 1 -- start start 2024-09-11T21:51:55.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:51:55.853+0000 7f230a992640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2304100b10 0x7f230406b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:51:55.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:51:55.853+0000 7f230a992640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2304101d10 0x7f230419cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:51:55.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:51:55.853+0000 7f230a992640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23041026d0 0x7f230419f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:51:55.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:51:55.853+0000 7f230a992640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f230410c790 con 0x7f2304100b10 2024-09-11T21:51:55.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:51:55.853+0000 7f230a992640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f230410c610 con 0x7f2304101d10 2024-09-11T21:51:55.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:51:55.853+0000 7f230a992640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f230410c910 con 0x7f23041026d0 2024-09-11T21:51:55.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:51:55.853+0000 7f2303fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2304100b10 0x7f230406b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:51:55.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:51:55.853+0000 7f2303fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2304100b10 0x7f230406b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:34978/0 (socket says 172.21.15.1:34978) 2024-09-11T21:51:55.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:51:55.853+0000 7f2303fff640 1 -- 172.21.15.1:0/1926599597 learned_addr learned my addr 172.21.15.1:0/1926599597 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:51:55.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:51:55.853+0000 7f2308f2f640 1 --2- 172.21.15.1:0/1926599597 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23041026d0 0x7f230419f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:51:55.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:51:55.853+0000 7f22fbfff640 1 --2- 172.21.15.1:0/1926599597 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2304101d10 0x7f230419cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:51:55.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:51:55.853+0000 7f2303fff640 1 -- 172.21.15.1:0/1926599597 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23041026d0 msgr2=0x7f230419f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:51:55.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:51:55.853+0000 7f2303fff640 1 --2- 172.21.15.1:0/1926599597 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23041026d0 0x7f230419f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:51:55.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:51:55.853+0000 7f2303fff640 1 -- 172.21.15.1:0/1926599597 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2304101d10 msgr2=0x7f230419cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:51:55.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:51:55.853+0000 7f2303fff640 1 --2- 172.21.15.1:0/1926599597 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2304101d10 0x7f230419cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:51:55.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:51:55.853+0000 7f2303fff640 1 -- 172.21.15.1:0/1926599597 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f230419fba0 con 0x7f2304100b10 2024-09-11T21:51:55.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:51:55.853+0000 7f2303fff640 1 --2- 172.21.15.1:0/1926599597 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2304100b10 0x7f230406b2f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f22f400d770 tx=0x7f22f400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:51:55.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:51:55.854+0000 7f2301ffb640 1 -- 172.21.15.1:0/1926599597 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f4017070 con 0x7f2304100b10 2024-09-11T21:51:55.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:51:55.854+0000 7f2301ffb640 1 -- 172.21.15.1:0/1926599597 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22f4009c30 con 0x7f2304100b10 2024-09-11T21:51:55.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:51:55.854+0000 7f230a992640 1 -- 172.21.15.1:0/1926599597 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f230419fe30 con 0x7f2304100b10 2024-09-11T21:51:55.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:51:55.854+0000 7f2301ffb640 1 -- 172.21.15.1:0/1926599597 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f4003dd0 con 0x7f2304100b10 2024-09-11T21:51:55.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:51:55.854+0000 7f230a992640 1 -- 172.21.15.1:0/1926599597 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23041a0280 con 0x7f2304100b10 2024-09-11T21:51:55.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:51:55.855+0000 7f230a992640 1 -- 172.21.15.1:0/1926599597 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2304107f10 con 0x7f2304100b10 2024-09-11T21:51:55.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:51:55.856+0000 7f2301ffb640 1 -- 172.21.15.1:0/1926599597 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f22f4003f90 con 0x7f2304100b10 2024-09-11T21:51:55.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:51:55.856+0000 7f2301ffb640 1 --2- 172.21.15.1:0/1926599597 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f22f00a2b80 0x7f22f00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:51:55.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:51:55.857+0000 7f22fbfff640 1 --2- 172.21.15.1:0/1926599597 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f22f00a2b80 0x7f22f00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:51:55.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:51:55.857+0000 7f2301ffb640 1 -- 172.21.15.1:0/1926599597 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4253+0+0 (secure 0 0 0) 0x7f22f40cdcc0 con 0x7f2304100b10 2024-09-11T21:51:55.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:51:55.857+0000 7f22fbfff640 1 --2- 172.21.15.1:0/1926599597 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f22f00a2b80 0x7f22f00a5040 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f22ec0314e0 tx=0x7f22ec000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:51:55.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:51:55.860+0000 7f2301ffb640 1 -- 172.21.15.1:0/1926599597 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f22f4009de0 con 0x7f2304100b10 2024-09-11T21:51:56.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:51:56.043+0000 7f230a992640 1 -- 172.21.15.1:0/1926599597 --> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f2304062630 con 0x7f22f00a2b80 2024-09-11T21:51:56.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:56 smithi001 ceph-mon[30935]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T21:51:56.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:56 smithi001 ceph-mon[30935]: Detected new or changed devices on smithi161 2024-09-11T21:51:56.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:56 smithi001 ceph-mon[30935]: Adjusting osd_memory_target on smithi161 to 8544M 2024-09-11T21:51:56.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:56 smithi001 ceph-mon[30935]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-09-11T21:51:56.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:56 smithi001 ceph-mon[30935]: osdmap e37: 6 total, 5 up, 6 in 2024-09-11T21:51:56.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:56 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T21:51:56.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:56 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T21:51:56.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:56 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T21:51:56.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:56 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T21:51:56.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:56 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:51:56.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:56 smithi053 ceph-mon[34908]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T21:51:56.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:56 smithi053 ceph-mon[34908]: Detected new or changed devices on smithi161 2024-09-11T21:51:56.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:56 smithi053 ceph-mon[34908]: Adjusting osd_memory_target on smithi161 to 8544M 2024-09-11T21:51:56.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:56 smithi053 ceph-mon[34908]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-09-11T21:51:56.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:56 smithi053 ceph-mon[34908]: osdmap e37: 6 total, 5 up, 6 in 2024-09-11T21:51:56.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:56 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T21:51:56.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:56 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T21:51:56.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:56 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T21:51:56.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:56 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T21:51:56.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:56 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:51:56.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:56 smithi161 ceph-mon[35615]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T21:51:56.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:56 smithi161 ceph-mon[35615]: Detected new or changed devices on smithi161 2024-09-11T21:51:56.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:56 smithi161 ceph-mon[35615]: Adjusting osd_memory_target on smithi161 to 8544M 2024-09-11T21:51:56.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:56 smithi161 ceph-mon[35615]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-09-11T21:51:56.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:56 smithi161 ceph-mon[35615]: osdmap e37: 6 total, 5 up, 6 in 2024-09-11T21:51:56.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:56 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T21:51:56.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:56 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T21:51:56.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:56 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T21:51:56.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:56 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T21:51:56.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:56 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:51:57.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:57 smithi001 ceph-mon[30935]: purged_snaps scrub starts 2024-09-11T21:51:57.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:57 smithi001 ceph-mon[30935]: purged_snaps scrub ok 2024-09-11T21:51:57.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:57 smithi001 ceph-mon[30935]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:51:57.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:57 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T21:51:57.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:57 smithi001 ceph-mon[30935]: osd.5 [v2:172.21.15.161:6808/1622081327,v1:172.21.15.161:6809/1622081327] boot 2024-09-11T21:51:57.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:57 smithi001 ceph-mon[30935]: osdmap e38: 6 total, 6 up, 6 in 2024-09-11T21:51:57.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:57 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T21:51:57.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:57 smithi053 ceph-mon[34908]: purged_snaps scrub starts 2024-09-11T21:51:57.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:57 smithi053 ceph-mon[34908]: purged_snaps scrub ok 2024-09-11T21:51:57.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:57 smithi053 ceph-mon[34908]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:51:57.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:57 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T21:51:57.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:57 smithi053 ceph-mon[34908]: osd.5 [v2:172.21.15.161:6808/1622081327,v1:172.21.15.161:6809/1622081327] boot 2024-09-11T21:51:57.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:57 smithi053 ceph-mon[34908]: osdmap e38: 6 total, 6 up, 6 in 2024-09-11T21:51:57.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:57 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T21:51:57.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:57 smithi161 ceph-mon[35615]: purged_snaps scrub starts 2024-09-11T21:51:57.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:57 smithi161 ceph-mon[35615]: purged_snaps scrub ok 2024-09-11T21:51:57.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:57 smithi161 ceph-mon[35615]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:51:57.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:57 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T21:51:57.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:57 smithi161 ceph-mon[35615]: osd.5 [v2:172.21.15.161:6808/1622081327,v1:172.21.15.161:6809/1622081327] boot 2024-09-11T21:51:57.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:57 smithi161 ceph-mon[35615]: osdmap e38: 6 total, 6 up, 6 in 2024-09-11T21:51:57.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:57 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T21:51:58.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:58 smithi001 ceph-mon[30935]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T21:51:58.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:58 smithi001 ceph-mon[30935]: osdmap e39: 6 total, 6 up, 6 in 2024-09-11T21:51:58.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:58 smithi053 ceph-mon[34908]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T21:51:58.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:58 smithi053 ceph-mon[34908]: osdmap e39: 6 total, 6 up, 6 in 2024-09-11T21:51:58.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:58 smithi161 ceph-mon[35615]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T21:51:58.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:58 smithi161 ceph-mon[35615]: osdmap e39: 6 total, 6 up, 6 in 2024-09-11T21:51:59.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:59 smithi001 ceph-mon[30935]: osdmap e40: 6 total, 6 up, 6 in 2024-09-11T21:51:59.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:59 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/2380089515' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3af5925d-a749-4849-a076-80580222cad4"}]: dispatch 2024-09-11T21:51:59.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:59 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/2380089515' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3af5925d-a749-4849-a076-80580222cad4"}]': finished 2024-09-11T21:51:59.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:59 smithi001 ceph-mon[30935]: osdmap e41: 7 total, 6 up, 7 in 2024-09-11T21:51:59.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:59 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T21:51:59.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:59 smithi001 ceph-mon[30935]: osdmap e42: 7 total, 6 up, 7 in 2024-09-11T21:51:59.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:59 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T21:51:59.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:51:59 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/1334876502' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T21:51:59.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:59 smithi053 ceph-mon[34908]: osdmap e40: 6 total, 6 up, 6 in 2024-09-11T21:51:59.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:59 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/2380089515' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3af5925d-a749-4849-a076-80580222cad4"}]: dispatch 2024-09-11T21:51:59.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:59 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/2380089515' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3af5925d-a749-4849-a076-80580222cad4"}]': finished 2024-09-11T21:51:59.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:59 smithi053 ceph-mon[34908]: osdmap e41: 7 total, 6 up, 7 in 2024-09-11T21:51:59.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:59 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T21:51:59.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:59 smithi053 ceph-mon[34908]: osdmap e42: 7 total, 6 up, 7 in 2024-09-11T21:51:59.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:59 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T21:51:59.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:51:59 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/1334876502' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T21:51:59.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:59 smithi161 ceph-mon[35615]: osdmap e40: 6 total, 6 up, 6 in 2024-09-11T21:51:59.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:59 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/2380089515' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3af5925d-a749-4849-a076-80580222cad4"}]: dispatch 2024-09-11T21:51:59.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:59 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/2380089515' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3af5925d-a749-4849-a076-80580222cad4"}]': finished 2024-09-11T21:51:59.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:59 smithi161 ceph-mon[35615]: osdmap e41: 7 total, 6 up, 7 in 2024-09-11T21:51:59.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:59 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T21:51:59.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:59 smithi161 ceph-mon[35615]: osdmap e42: 7 total, 6 up, 7 in 2024-09-11T21:51:59.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:59 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T21:51:59.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:51:59 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/1334876502' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T21:52:00.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:00 smithi001 ceph-mon[30935]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail 2024-09-11T21:52:00.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:00 smithi001 ceph-mon[30935]: osdmap e43: 7 total, 6 up, 7 in 2024-09-11T21:52:00.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:00 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T21:52:00.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:00 smithi053 ceph-mon[34908]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail 2024-09-11T21:52:00.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:00 smithi053 ceph-mon[34908]: osdmap e43: 7 total, 6 up, 7 in 2024-09-11T21:52:00.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:00 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T21:52:00.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:00 smithi161 ceph-mon[35615]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail 2024-09-11T21:52:00.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:00 smithi161 ceph-mon[35615]: osdmap e43: 7 total, 6 up, 7 in 2024-09-11T21:52:00.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:00 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T21:52:02.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:02 smithi001 ceph-mon[30935]: pgmap v141: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 170 KiB/s, 0 objects/s recovering 2024-09-11T21:52:02.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:02 smithi053 ceph-mon[34908]: pgmap v141: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 170 KiB/s, 0 objects/s recovering 2024-09-11T21:52:02.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:02 smithi161 ceph-mon[35615]: pgmap v141: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 170 KiB/s, 0 objects/s recovering 2024-09-11T21:52:04.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:04 smithi001 ceph-mon[30935]: pgmap v142: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 131 KiB/s, 0 objects/s recovering 2024-09-11T21:52:04.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:04 smithi053 ceph-mon[34908]: pgmap v142: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 131 KiB/s, 0 objects/s recovering 2024-09-11T21:52:04.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:04 smithi161 ceph-mon[35615]: pgmap v142: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 131 KiB/s, 0 objects/s recovering 2024-09-11T21:52:06.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:06 smithi001 ceph-mon[30935]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-09-11T21:52:06.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:06 smithi053 ceph-mon[34908]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-09-11T21:52:06.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:06 smithi161 ceph-mon[35615]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-09-11T21:52:07.293 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:07 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-11T21:52:07.293 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:07 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:52:07.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:07 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-11T21:52:07.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:07 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:52:07.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:07 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-11T21:52:07.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:07 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:52:08.275 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:08 smithi001 ceph-mon[30935]: Deploying daemon osd.6 on smithi001 2024-09-11T21:52:08.276 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:08 smithi001 ceph-mon[30935]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T21:52:08.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:08 smithi053 ceph-mon[34908]: Deploying daemon osd.6 on smithi001 2024-09-11T21:52:08.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:08 smithi053 ceph-mon[34908]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T21:52:08.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:08 smithi161 ceph-mon[35615]: Deploying daemon osd.6 on smithi001 2024-09-11T21:52:08.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:08 smithi161 ceph-mon[35615]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T21:52:10.261 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:10 smithi001 ceph-mon[30935]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T21:52:10.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:10 smithi053 ceph-mon[34908]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T21:52:10.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:10 smithi161 ceph-mon[35615]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T21:52:12.321 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:12 smithi001 ceph-mon[30935]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-11T21:52:12.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:12 smithi053 ceph-mon[34908]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-11T21:52:12.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:12 smithi161 ceph-mon[35615]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-11T21:52:13.340 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:13 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:52:13.340 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:13 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:52:13.341 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:13 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:52:13.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:13 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:52:13.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:13 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:52:13.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:13 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:52:13.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:13 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:52:13.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:13 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:52:13.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:13 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:52:14.272 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:14 smithi001 ceph-mon[30935]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-11T21:52:14.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:14 smithi053 ceph-mon[34908]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-11T21:52:14.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:14 smithi161 ceph-mon[35615]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-11T21:52:16.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:52:16.374+0000 7f2301ffb640 1 -- 172.21.15.1:0/1926599597 <== mgr.14150 v2:172.21.15.1:6800/3890350886 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2304062630 con 0x7f22f00a2b80 2024-09-11T21:52:16.375 INFO:teuthology.orchestra.run.smithi001.stdout:Created osd(s) 6 on host 'smithi001' 2024-09-11T21:52:16.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:52:16.376+0000 7f230a992640 1 -- 172.21.15.1:0/1926599597 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f22f00a2b80 msgr2=0x7f22f00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:52:16.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:52:16.376+0000 7f230a992640 1 --2- 172.21.15.1:0/1926599597 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f22f00a2b80 0x7f22f00a5040 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f22ec0314e0 tx=0x7f22ec000bc0 comp rx=0 tx=0).stop 2024-09-11T21:52:16.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:52:16.376+0000 7f230a992640 1 -- 172.21.15.1:0/1926599597 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2304100b10 msgr2=0x7f230406b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:52:16.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:52:16.376+0000 7f230a992640 1 --2- 172.21.15.1:0/1926599597 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2304100b10 0x7f230406b2f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f22f400d770 tx=0x7f22f400dc30 comp rx=0 tx=0).stop 2024-09-11T21:52:16.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:52:16.376+0000 7f230a992640 1 -- 172.21.15.1:0/1926599597 shutdown_connections 2024-09-11T21:52:16.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:52:16.376+0000 7f230a992640 1 --2- 172.21.15.1:0/1926599597 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f22f00a2b80 0x7f22f00a5040 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:52:16.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:52:16.376+0000 7f230a992640 1 --2- 172.21.15.1:0/1926599597 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23041026d0 0x7f230419f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:52:16.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:52:16.376+0000 7f230a992640 1 --2- 172.21.15.1:0/1926599597 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2304101d10 0x7f230419cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:52:16.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:52:16.376+0000 7f230a992640 1 --2- 172.21.15.1:0/1926599597 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2304100b10 0x7f230406b2f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:52:16.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:52:16.376+0000 7f230a992640 1 -- 172.21.15.1:0/1926599597 >> 172.21.15.1:0/1926599597 conn(0x7f23040fc2a0 msgr2=0x7f23041050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:52:16.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:52:16.377+0000 7f230a992640 1 -- 172.21.15.1:0/1926599597 shutdown_connections 2024-09-11T21:52:16.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:52:16.377+0000 7f230a992640 1 -- 172.21.15.1:0/1926599597 wait complete. 2024-09-11T21:52:16.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:16 smithi001 ceph-mon[30935]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-11T21:52:16.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:16 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:52:16.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:16 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:52:16.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:16 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:52:16.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:16 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:52:16.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:16 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:52:16.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:16 smithi053 ceph-mon[34908]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-11T21:52:16.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:16 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:52:16.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:16 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:52:16.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:16 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:52:16.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:16 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:52:16.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:16 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:52:16.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:16 smithi161 ceph-mon[35615]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-11T21:52:16.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:16 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:52:16.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:16 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:52:16.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:16 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:52:16.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:16 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:52:16.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:16 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:52:16.998 DEBUG:teuthology.orchestra.run.smithi001:osd.6> sudo journalctl -f -n 0 -u ceph-3ffca83c-7087-11ef-bcea-c7b262605968@osd.6.service 2024-09-11T21:52:17.001 INFO:tasks.cephadm:Deploying osd.7 on smithi053 with /dev/vg_nvme/lv_2... 2024-09-11T21:52:17.002 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-11T21:52:17.264 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T21:52:17.472 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:17 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:52:17.472 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:17 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:52:17.472 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:17 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:52:17.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:17 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:52:17.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:17 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:52:17.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:17 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:52:17.631 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:17 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:52:17.631 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:17 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:52:17.631 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:17 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:52:18.160 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 11 21:52:17 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-6[55973]: 2024-09-11T21:52:17.927+0000 7f9c9285e640 -1 osd.6 0 log_to_monitors true 2024-09-11T21:52:18.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:18 smithi001 ceph-mon[30935]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-11T21:52:18.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:18 smithi001 ceph-mon[30935]: from='osd.6 [v2:172.21.15.1:6818/158434853,v1:172.21.15.1:6819/158434853]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T21:52:18.759 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:18 smithi053 ceph-mon[34908]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-11T21:52:18.759 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:18 smithi053 ceph-mon[34908]: from='osd.6 [v2:172.21.15.1:6818/158434853,v1:172.21.15.1:6819/158434853]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T21:52:18.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:18 smithi161 ceph-mon[35615]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-11T21:52:18.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:18 smithi161 ceph-mon[35615]: from='osd.6 [v2:172.21.15.1:6818/158434853,v1:172.21.15.1:6819/158434853]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T21:52:19.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:19 smithi001 ceph-mon[30935]: from='osd.6 [v2:172.21.15.1:6818/158434853,v1:172.21.15.1:6819/158434853]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-11T21:52:19.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:19 smithi001 ceph-mon[30935]: osdmap e44: 7 total, 6 up, 7 in 2024-09-11T21:52:19.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:19 smithi001 ceph-mon[30935]: from='osd.6 [v2:172.21.15.1:6818/158434853,v1:172.21.15.1:6819/158434853]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-09-11T21:52:19.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:19 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T21:52:19.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:19 smithi001 ceph-mon[30935]: from='osd.6 [v2:172.21.15.1:6818/158434853,v1:172.21.15.1:6819/158434853]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-09-11T21:52:19.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:19 smithi001 ceph-mon[30935]: osdmap e45: 7 total, 6 up, 7 in 2024-09-11T21:52:19.444 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 11 21:52:19 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-6[55973]: 2024-09-11T21:52:19.381+0000 7f9c8e7e1640 -1 osd.6 0 waiting for initial osdmap 2024-09-11T21:52:19.444 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 11 21:52:19 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-6[55973]: 2024-09-11T21:52:19.391+0000 7f9c8a609640 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T21:52:19.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:19 smithi053 ceph-mon[34908]: from='osd.6 [v2:172.21.15.1:6818/158434853,v1:172.21.15.1:6819/158434853]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-11T21:52:19.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:19 smithi053 ceph-mon[34908]: osdmap e44: 7 total, 6 up, 7 in 2024-09-11T21:52:19.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:19 smithi053 ceph-mon[34908]: from='osd.6 [v2:172.21.15.1:6818/158434853,v1:172.21.15.1:6819/158434853]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-09-11T21:52:19.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:19 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T21:52:19.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:19 smithi053 ceph-mon[34908]: from='osd.6 [v2:172.21.15.1:6818/158434853,v1:172.21.15.1:6819/158434853]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-09-11T21:52:19.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:19 smithi053 ceph-mon[34908]: osdmap e45: 7 total, 6 up, 7 in 2024-09-11T21:52:19.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:19 smithi161 ceph-mon[35615]: from='osd.6 [v2:172.21.15.1:6818/158434853,v1:172.21.15.1:6819/158434853]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-11T21:52:19.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:19 smithi161 ceph-mon[35615]: osdmap e44: 7 total, 6 up, 7 in 2024-09-11T21:52:19.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:19 smithi161 ceph-mon[35615]: from='osd.6 [v2:172.21.15.1:6818/158434853,v1:172.21.15.1:6819/158434853]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-09-11T21:52:19.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:19 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T21:52:19.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:19 smithi161 ceph-mon[35615]: from='osd.6 [v2:172.21.15.1:6818/158434853,v1:172.21.15.1:6819/158434853]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-09-11T21:52:19.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:19 smithi161 ceph-mon[35615]: osdmap e45: 7 total, 6 up, 7 in 2024-09-11T21:52:20.222 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-11T21:52:20.247 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi053:vg_nvme/lv_2 2024-09-11T21:52:20.496 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T21:52:20.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:20 smithi053 ceph-mon[34908]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-11T21:52:20.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:20 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T21:52:20.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:20 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T21:52:20.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:20 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:52:20.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:20 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:52:20.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:20 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:52:20.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:20 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:52:20.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:20 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:52:20.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:20 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:52:20.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:20 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:52:20.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:20 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:52:20.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:20 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:52:20.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:20 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T21:52:20.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:20 smithi001 ceph-mon[30935]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-11T21:52:20.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:20 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T21:52:20.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:20 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T21:52:20.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:20 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:52:20.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:20 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:52:20.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:20 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:52:20.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:20 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:52:20.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:20 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:52:20.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:20 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:52:20.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:20 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:52:20.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:20 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:52:20.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:20 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:52:20.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:20 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T21:52:20.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:20 smithi161 ceph-mon[35615]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-11T21:52:20.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:20 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T21:52:20.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:20 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T21:52:20.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:20 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:52:20.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:20 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:52:20.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:20 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:52:20.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:20 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:52:20.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:20 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:52:20.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:20 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:52:20.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:20 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:52:20.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:20 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:52:20.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:20 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:52:20.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:20 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T21:52:20.963 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:52:20.948+0000 7fe00c77a640 1 -- 172.21.15.53:0/1103523605 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0041078c0 msgr2=0x7fe004109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:52:20.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:52:20.948+0000 7fe00c77a640 1 --2- 172.21.15.53:0/1103523605 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0041078c0 0x7fe004109d10 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdffc009900 tx=0x7fdffc02a410 comp rx=0 tx=0).stop 2024-09-11T21:52:20.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:52:20.948+0000 7fe00c77a640 1 -- 172.21.15.53:0/1103523605 shutdown_connections 2024-09-11T21:52:20.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:52:20.948+0000 7fe00c77a640 1 --2- 172.21.15.53:0/1103523605 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe0041078c0 0x7fe004109d10 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:52:20.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:52:20.948+0000 7fe00c77a640 1 --2- 172.21.15.53:0/1103523605 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe004104f90 0x7fe004107380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:52:20.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:52:20.948+0000 7fe00c77a640 1 --2- 172.21.15.53:0/1103523605 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe004102660 0x7fe004104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:52:20.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:52:20.948+0000 7fe00c77a640 1 -- 172.21.15.53:0/1103523605 >> 172.21.15.53:0/1103523605 conn(0x7fe0040fc280 msgr2=0x7fe0040fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:52:20.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:52:20.948+0000 7fe00c77a640 1 -- 172.21.15.53:0/1103523605 shutdown_connections 2024-09-11T21:52:20.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:52:20.948+0000 7fe00c77a640 1 -- 172.21.15.53:0/1103523605 wait complete. 2024-09-11T21:52:20.964 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:52:20.948+0000 7fe00c77a640 1 Processor -- start 2024-09-11T21:52:20.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:52:20.948+0000 7fe00c77a640 1 -- start start 2024-09-11T21:52:20.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:52:20.948+0000 7fe00c77a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe004102660 0x7fe004197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:52:20.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:52:20.948+0000 7fe00c77a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe004104f90 0x7fe0041984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:52:20.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:52:20.948+0000 7fe00c77a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe0041078c0 0x7fe00419f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:52:20.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:52:20.948+0000 7fe00c77a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe00410d900 con 0x7fe004104f90 2024-09-11T21:52:20.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:52:20.949+0000 7fe00c77a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe00410d780 con 0x7fe004102660 2024-09-11T21:52:20.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:52:20.949+0000 7fe00c77a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe00410da80 con 0x7fe0041078c0 2024-09-11T21:52:20.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:52:20.949+0000 7fe00a516640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe004102660 0x7fe004197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:52:20.965 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:52:20.949+0000 7fe00a516640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe004102660 0x7fe004197f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:55104/0 (socket says 172.21.15.53:55104) 2024-09-11T21:52:20.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:52:20.949+0000 7fe00a516640 1 -- 172.21.15.53:0/3328162337 learned_addr learned my addr 172.21.15.53:0/3328162337 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-11T21:52:20.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:52:20.949+0000 7fe00ad17640 1 --2- 172.21.15.53:0/3328162337 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe0041078c0 0x7fe00419f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:52:20.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:52:20.949+0000 7fe00a516640 1 -- 172.21.15.53:0/3328162337 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe0041078c0 msgr2=0x7fe00419f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:52:20.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:52:20.949+0000 7fe00a516640 1 --2- 172.21.15.53:0/3328162337 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe0041078c0 0x7fe00419f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:52:20.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:52:20.949+0000 7fe00a516640 1 -- 172.21.15.53:0/3328162337 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe004104f90 msgr2=0x7fe0041984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:52:20.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:52:20.949+0000 7fe00a516640 1 --2- 172.21.15.53:0/3328162337 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe004104f90 0x7fe0041984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:52:20.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:52:20.949+0000 7fe00a516640 1 -- 172.21.15.53:0/3328162337 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe00419fb90 con 0x7fe004102660 2024-09-11T21:52:20.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:52:20.949+0000 7fe00ad17640 1 --2- 172.21.15.53:0/3328162337 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe0041078c0 0x7fe00419f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:52:20.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:52:20.949+0000 7fe00a516640 1 --2- 172.21.15.53:0/3328162337 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe004102660 0x7fe004197f60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdff400ba50 tx=0x7fdff400bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:52:20.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:52:20.949+0000 7fe0037fe640 1 -- 172.21.15.53:0/3328162337 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff4019070 con 0x7fe004102660 2024-09-11T21:52:20.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:52:20.949+0000 7fe0037fe640 1 -- 172.21.15.53:0/3328162337 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdff400cbe0 con 0x7fe004102660 2024-09-11T21:52:20.966 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:52:20.949+0000 7fe00c77a640 1 -- 172.21.15.53:0/3328162337 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe00419fe80 con 0x7fe004102660 2024-09-11T21:52:20.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:52:20.949+0000 7fe0037fe640 1 -- 172.21.15.53:0/3328162337 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff40076e0 con 0x7fe004102660 2024-09-11T21:52:20.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:52:20.949+0000 7fe00c77a640 1 -- 172.21.15.53:0/3328162337 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0041a03b0 con 0x7fe004102660 2024-09-11T21:52:20.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:52:20.950+0000 7fe00c77a640 1 -- 172.21.15.53:0/3328162337 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0040fdd70 con 0x7fe004102660 2024-09-11T21:52:20.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:52:20.951+0000 7fe0037fe640 1 -- 172.21.15.53:0/3328162337 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fdff401c070 con 0x7fe004102660 2024-09-11T21:52:20.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:52:20.952+0000 7fe0037fe640 1 --2- 172.21.15.53:0/3328162337 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fdfec0a2bb0 0x7fdfec0a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:52:20.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:52:20.952+0000 7fe0037fe640 1 -- 172.21.15.53:0/3328162337 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 4965+0+0 (secure 0 0 0) 0x7fdff40cd940 con 0x7fe004102660 2024-09-11T21:52:20.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:52:20.952+0000 7fe009d15640 1 --2- 172.21.15.53:0/3328162337 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fdfec0a2bb0 0x7fdfec0a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:52:20.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:52:20.953+0000 7fe009d15640 1 --2- 172.21.15.53:0/3328162337 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fdfec0a2bb0 0x7fdfec0a5070 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdff8005fd0 tx=0x7fdff8005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:52:20.967 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:52:20.956+0000 7fe0037fe640 1 -- 172.21.15.53:0/3328162337 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdff409b5b0 con 0x7fe004102660 2024-09-11T21:52:21.141 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:52:21.140+0000 7fe00c77a640 1 -- 172.21.15.53:0/3328162337 --> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fe004062630 con 0x7fdfec0a2bb0 2024-09-11T21:52:21.644 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:21 smithi053 ceph-mon[34908]: purged_snaps scrub starts 2024-09-11T21:52:21.645 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:21 smithi053 ceph-mon[34908]: purged_snaps scrub ok 2024-09-11T21:52:21.645 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:21 smithi053 ceph-mon[34908]: Detected new or changed devices on smithi001 2024-09-11T21:52:21.645 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:21 smithi053 ceph-mon[34908]: Adjusting osd_memory_target on smithi001 to 5697M 2024-09-11T21:52:21.645 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:21 smithi053 ceph-mon[34908]: osd.6 [v2:172.21.15.1:6818/158434853,v1:172.21.15.1:6819/158434853] boot 2024-09-11T21:52:21.645 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:21 smithi053 ceph-mon[34908]: osdmap e46: 7 total, 7 up, 7 in 2024-09-11T21:52:21.645 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:21 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T21:52:21.645 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:21 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T21:52:21.645 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:21 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T21:52:21.645 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:21 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:52:21.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:21 smithi001 ceph-mon[30935]: purged_snaps scrub starts 2024-09-11T21:52:21.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:21 smithi001 ceph-mon[30935]: purged_snaps scrub ok 2024-09-11T21:52:21.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:21 smithi001 ceph-mon[30935]: Detected new or changed devices on smithi001 2024-09-11T21:52:21.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:21 smithi001 ceph-mon[30935]: Adjusting osd_memory_target on smithi001 to 5697M 2024-09-11T21:52:21.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:21 smithi001 ceph-mon[30935]: osd.6 [v2:172.21.15.1:6818/158434853,v1:172.21.15.1:6819/158434853] boot 2024-09-11T21:52:21.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:21 smithi001 ceph-mon[30935]: osdmap e46: 7 total, 7 up, 7 in 2024-09-11T21:52:21.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:21 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T21:52:21.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:21 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T21:52:21.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:21 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T21:52:21.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:21 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:52:21.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:21 smithi161 ceph-mon[35615]: purged_snaps scrub starts 2024-09-11T21:52:21.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:21 smithi161 ceph-mon[35615]: purged_snaps scrub ok 2024-09-11T21:52:21.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:21 smithi161 ceph-mon[35615]: Detected new or changed devices on smithi001 2024-09-11T21:52:21.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:21 smithi161 ceph-mon[35615]: Adjusting osd_memory_target on smithi001 to 5697M 2024-09-11T21:52:21.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:21 smithi161 ceph-mon[35615]: osd.6 [v2:172.21.15.1:6818/158434853,v1:172.21.15.1:6819/158434853] boot 2024-09-11T21:52:21.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:21 smithi161 ceph-mon[35615]: osdmap e46: 7 total, 7 up, 7 in 2024-09-11T21:52:21.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:21 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T21:52:21.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:21 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T21:52:21.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:21 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T21:52:21.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:21 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:52:22.529 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:22 smithi053 ceph-mon[34908]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-11T21:52:22.529 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:22 smithi053 ceph-mon[34908]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:52:22.530 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:22 smithi053 ceph-mon[34908]: osdmap e47: 7 total, 7 up, 7 in 2024-09-11T21:52:22.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:22 smithi001 ceph-mon[30935]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-11T21:52:22.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:22 smithi001 ceph-mon[30935]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:52:22.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:22 smithi001 ceph-mon[30935]: osdmap e47: 7 total, 7 up, 7 in 2024-09-11T21:52:22.841 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:22 smithi161 ceph-mon[35615]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-11T21:52:22.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:22 smithi161 ceph-mon[35615]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:52:22.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:22 smithi161 ceph-mon[35615]: osdmap e47: 7 total, 7 up, 7 in 2024-09-11T21:52:24.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:24 smithi001 ceph-mon[30935]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T21:52:24.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:24 smithi001 ceph-mon[30935]: from='client.? 172.21.15.53:0/3257597860' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8f26c3c-17e9-4e89-8392-289796e0c8a6"}]: dispatch 2024-09-11T21:52:24.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:24 smithi001 ceph-mon[30935]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8f26c3c-17e9-4e89-8392-289796e0c8a6"}]: dispatch 2024-09-11T21:52:24.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:24 smithi001 ceph-mon[30935]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f8f26c3c-17e9-4e89-8392-289796e0c8a6"}]': finished 2024-09-11T21:52:24.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:24 smithi001 ceph-mon[30935]: osdmap e48: 8 total, 7 up, 8 in 2024-09-11T21:52:24.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:24 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T21:52:24.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:24 smithi001 ceph-mon[30935]: from='client.? 172.21.15.53:0/736440668' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T21:52:24.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:24 smithi053 ceph-mon[34908]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T21:52:24.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:24 smithi053 ceph-mon[34908]: from='client.? 172.21.15.53:0/3257597860' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8f26c3c-17e9-4e89-8392-289796e0c8a6"}]: dispatch 2024-09-11T21:52:24.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:24 smithi053 ceph-mon[34908]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8f26c3c-17e9-4e89-8392-289796e0c8a6"}]: dispatch 2024-09-11T21:52:24.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:24 smithi053 ceph-mon[34908]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f8f26c3c-17e9-4e89-8392-289796e0c8a6"}]': finished 2024-09-11T21:52:24.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:24 smithi053 ceph-mon[34908]: osdmap e48: 8 total, 7 up, 8 in 2024-09-11T21:52:24.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:24 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T21:52:24.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:24 smithi053 ceph-mon[34908]: from='client.? 172.21.15.53:0/736440668' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T21:52:24.841 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:24 smithi161 ceph-mon[35615]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T21:52:24.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:24 smithi161 ceph-mon[35615]: from='client.? 172.21.15.53:0/3257597860' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8f26c3c-17e9-4e89-8392-289796e0c8a6"}]: dispatch 2024-09-11T21:52:24.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:24 smithi161 ceph-mon[35615]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f8f26c3c-17e9-4e89-8392-289796e0c8a6"}]: dispatch 2024-09-11T21:52:24.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:24 smithi161 ceph-mon[35615]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f8f26c3c-17e9-4e89-8392-289796e0c8a6"}]': finished 2024-09-11T21:52:24.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:24 smithi161 ceph-mon[35615]: osdmap e48: 8 total, 7 up, 8 in 2024-09-11T21:52:24.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:24 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T21:52:24.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:24 smithi161 ceph-mon[35615]: from='client.? 172.21.15.53:0/736440668' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T21:52:26.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:26 smithi001 ceph-mon[30935]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T21:52:26.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:26 smithi053 ceph-mon[34908]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T21:52:26.841 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:26 smithi161 ceph-mon[35615]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T21:52:28.648 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:28 smithi053 ceph-mon[34908]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T21:52:28.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:28 smithi001 ceph-mon[30935]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T21:52:28.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:28 smithi161 ceph-mon[35615]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T21:52:30.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:30 smithi001 ceph-mon[30935]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T21:52:30.796 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:30 smithi053 ceph-mon[34908]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T21:52:30.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:30 smithi161 ceph-mon[35615]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T21:52:32.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:32 smithi001 ceph-mon[30935]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T21:52:32.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:32 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-11T21:52:32.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:32 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:52:32.756 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:32 smithi053 ceph-mon[34908]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T21:52:32.757 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:32 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-11T21:52:32.757 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:32 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:52:32.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:32 smithi161 ceph-mon[35615]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T21:52:32.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:32 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-11T21:52:32.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:32 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:52:33.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:33 smithi001 ceph-mon[30935]: Deploying daemon osd.7 on smithi053 2024-09-11T21:52:33.718 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:33 smithi053 ceph-mon[34908]: Deploying daemon osd.7 on smithi053 2024-09-11T21:52:33.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:33 smithi161 ceph-mon[35615]: Deploying daemon osd.7 on smithi053 2024-09-11T21:52:34.565 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:34 smithi053 ceph-mon[34908]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T21:52:34.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:34 smithi001 ceph-mon[30935]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T21:52:34.841 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:34 smithi161 ceph-mon[35615]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T21:52:36.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:36 smithi001 ceph-mon[30935]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T21:52:36.757 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:36 smithi053 ceph-mon[34908]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T21:52:36.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:36 smithi161 ceph-mon[35615]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T21:52:38.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:38 smithi053 ceph-mon[34908]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T21:52:38.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:38 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:52:38.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:38 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:52:38.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:38 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:52:38.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:38 smithi001 ceph-mon[30935]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T21:52:38.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:38 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:52:38.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:38 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:52:38.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:38 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:52:38.841 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:38 smithi161 ceph-mon[35615]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T21:52:38.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:38 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:52:38.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:38 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:52:38.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:38 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:52:40.475 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:40 smithi053 ceph-mon[34908]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T21:52:40.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:40 smithi001 ceph-mon[30935]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T21:52:40.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:40 smithi161 ceph-mon[35615]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T21:52:42.159 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:52:42.158+0000 7fe0037fe640 1 -- 172.21.15.53:0/3328162337 <== mgr.14150 v2:172.21.15.1:6800/3890350886 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe004062630 con 0x7fdfec0a2bb0 2024-09-11T21:52:42.159 INFO:teuthology.orchestra.run.smithi053.stdout:Created osd(s) 7 on host 'smithi053' 2024-09-11T21:52:42.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:52:42.160+0000 7fe00c77a640 1 -- 172.21.15.53:0/3328162337 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fdfec0a2bb0 msgr2=0x7fdfec0a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:52:42.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:52:42.160+0000 7fe00c77a640 1 --2- 172.21.15.53:0/3328162337 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fdfec0a2bb0 0x7fdfec0a5070 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdff8005fd0 tx=0x7fdff8005850 comp rx=0 tx=0).stop 2024-09-11T21:52:42.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:52:42.160+0000 7fe00c77a640 1 -- 172.21.15.53:0/3328162337 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe004102660 msgr2=0x7fe004197f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:52:42.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:52:42.160+0000 7fe00c77a640 1 --2- 172.21.15.53:0/3328162337 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe004102660 0x7fe004197f60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdff400ba50 tx=0x7fdff400bf10 comp rx=0 tx=0).stop 2024-09-11T21:52:42.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:52:42.160+0000 7fe00c77a640 1 -- 172.21.15.53:0/3328162337 shutdown_connections 2024-09-11T21:52:42.161 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:52:42.160+0000 7fe00c77a640 1 --2- 172.21.15.53:0/3328162337 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fdfec0a2bb0 0x7fdfec0a5070 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:52:42.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:52:42.160+0000 7fe00c77a640 1 --2- 172.21.15.53:0/3328162337 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe0041078c0 0x7fe00419f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:52:42.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:52:42.160+0000 7fe00c77a640 1 --2- 172.21.15.53:0/3328162337 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe004104f90 0x7fe0041984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:52:42.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:52:42.160+0000 7fe00c77a640 1 --2- 172.21.15.53:0/3328162337 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe004102660 0x7fe004197f60 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:52:42.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:52:42.160+0000 7fe00c77a640 1 -- 172.21.15.53:0/3328162337 >> 172.21.15.53:0/3328162337 conn(0x7fe0040fc280 msgr2=0x7fe004102f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:52:42.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:52:42.160+0000 7fe00c77a640 1 -- 172.21.15.53:0/3328162337 shutdown_connections 2024-09-11T21:52:42.162 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:52:42.160+0000 7fe00c77a640 1 -- 172.21.15.53:0/3328162337 wait complete. 2024-09-11T21:52:42.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:42 smithi001 ceph-mon[30935]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T21:52:42.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:42 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:52:42.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:42 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:52:42.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:42 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:52:42.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:42 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:52:42.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:42 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:52:42.528 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:42 smithi053 ceph-mon[34908]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T21:52:42.528 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:42 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:52:42.529 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:42 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:52:42.529 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:42 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:52:42.529 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:42 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:52:42.529 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:42 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:52:42.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:42 smithi161 ceph-mon[35615]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T21:52:42.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:42 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:52:42.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:42 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:52:42.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:42 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:52:42.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:42 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:52:42.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:42 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:52:42.869 DEBUG:teuthology.orchestra.run.smithi053:osd.7> sudo journalctl -f -n 0 -u ceph-3ffca83c-7087-11ef-bcea-c7b262605968@osd.7.service 2024-09-11T21:52:42.874 INFO:tasks.cephadm:Deploying osd.8 on smithi161 with /dev/vg_nvme/lv_2... 2024-09-11T21:52:42.874 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-11T21:52:43.131 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T21:52:43.252 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:43 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:52:43.252 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:43 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:52:43.252 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:43 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:52:43.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:43 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:52:43.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:43 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:52:43.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:43 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:52:43.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:43 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:52:43.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:43 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:52:43.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:43 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:52:44.020 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 21:52:43 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-7[52494]: 2024-09-11T21:52:43.914+0000 7f29fba80640 -1 osd.7 0 log_to_monitors true 2024-09-11T21:52:44.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:44 smithi053 ceph-mon[34908]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T21:52:44.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:44 smithi053 ceph-mon[34908]: from='osd.7 [v2:172.21.15.53:6816/2132057697,v1:172.21.15.53:6817/2132057697]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T21:52:44.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:44 smithi053 ceph-mon[34908]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T21:52:44.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:44 smithi001 ceph-mon[30935]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T21:52:44.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:44 smithi001 ceph-mon[30935]: from='osd.7 [v2:172.21.15.53:6816/2132057697,v1:172.21.15.53:6817/2132057697]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T21:52:44.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:44 smithi001 ceph-mon[30935]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T21:52:44.520 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:44 smithi161 ceph-mon[35615]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T21:52:44.521 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:44 smithi161 ceph-mon[35615]: from='osd.7 [v2:172.21.15.53:6816/2132057697,v1:172.21.15.53:6817/2132057697]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T21:52:44.521 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:44 smithi161 ceph-mon[35615]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T21:52:45.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:45 smithi053 ceph-mon[34908]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-11T21:52:45.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:45 smithi053 ceph-mon[34908]: from='osd.7 [v2:172.21.15.53:6816/2132057697,v1:172.21.15.53:6817/2132057697]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-09-11T21:52:45.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:45 smithi053 ceph-mon[34908]: osdmap e49: 8 total, 7 up, 8 in 2024-09-11T21:52:45.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:45 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T21:52:45.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:45 smithi053 ceph-mon[34908]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-09-11T21:52:45.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:45 smithi053 ceph-mon[34908]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-09-11T21:52:45.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:45 smithi161 ceph-mon[35615]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-11T21:52:45.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:45 smithi161 ceph-mon[35615]: from='osd.7 [v2:172.21.15.53:6816/2132057697,v1:172.21.15.53:6817/2132057697]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-09-11T21:52:45.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:45 smithi161 ceph-mon[35615]: osdmap e49: 8 total, 7 up, 8 in 2024-09-11T21:52:45.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:45 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T21:52:45.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:45 smithi161 ceph-mon[35615]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-09-11T21:52:45.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:45 smithi161 ceph-mon[35615]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-09-11T21:52:45.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:45 smithi001 ceph-mon[30935]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-11T21:52:45.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:45 smithi001 ceph-mon[30935]: from='osd.7 [v2:172.21.15.53:6816/2132057697,v1:172.21.15.53:6817/2132057697]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-09-11T21:52:45.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:45 smithi001 ceph-mon[30935]: osdmap e49: 8 total, 7 up, 8 in 2024-09-11T21:52:45.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:45 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T21:52:45.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:45 smithi001 ceph-mon[30935]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-09-11T21:52:45.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:45 smithi001 ceph-mon[30935]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-09-11T21:52:45.745 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 21:52:45 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-7[52494]: 2024-09-11T21:52:45.567+0000 7f29f8217640 -1 osd.7 0 waiting for initial osdmap 2024-09-11T21:52:45.745 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 21:52:45 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-7[52494]: 2024-09-11T21:52:45.576+0000 7f29ef823640 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T21:52:45.822 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-11T21:52:45.848 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi161:vg_nvme/lv_2 2024-09-11T21:52:46.108 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T21:52:46.433 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:46 smithi161 ceph-mon[35615]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T21:52:46.433 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:46 smithi161 ceph-mon[35615]: osdmap e50: 8 total, 7 up, 8 in 2024-09-11T21:52:46.433 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:46 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T21:52:46.433 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:46 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T21:52:46.433 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:46 smithi161 ceph-mon[35615]: from='osd.7 ' entity='osd.7' 2024-09-11T21:52:46.433 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:46 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:52:46.433 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:46 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:52:46.433 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:46 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:52:46.433 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:46 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:52:46.433 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:46 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:52:46.433 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:46 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:52:46.434 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:46 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:52:46.434 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:46 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:52:46.434 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:46 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:52:46.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:46 smithi001 ceph-mon[30935]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T21:52:46.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:46 smithi001 ceph-mon[30935]: osdmap e50: 8 total, 7 up, 8 in 2024-09-11T21:52:46.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:46 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T21:52:46.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:46 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T21:52:46.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:46 smithi001 ceph-mon[30935]: from='osd.7 ' entity='osd.7' 2024-09-11T21:52:46.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:46 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:52:46.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:46 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:52:46.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:46 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:52:46.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:46 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:52:46.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:46 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:52:46.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:46 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:52:46.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:46 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:52:46.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:46 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:52:46.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:46 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:52:46.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:46 smithi053 ceph-mon[34908]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-11T21:52:46.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:46 smithi053 ceph-mon[34908]: osdmap e50: 8 total, 7 up, 8 in 2024-09-11T21:52:46.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:46 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T21:52:46.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:46 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T21:52:46.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:46 smithi053 ceph-mon[34908]: from='osd.7 ' entity='osd.7' 2024-09-11T21:52:46.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:46 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:52:46.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:46 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:52:46.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:46 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:52:46.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:46 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:52:46.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:46 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:52:46.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:46 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:52:46.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:46 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:52:46.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:46 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:52:46.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:46 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:52:46.589 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:52:46.570+0000 7f3a97870640 1 -- 172.21.15.161:0/886359295 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a90101c00 msgr2=0x7f3a90102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:52:46.589 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:52:46.570+0000 7f3a97870640 1 --2- 172.21.15.161:0/886359295 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a90101c00 0x7f3a90102000 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c009900 tx=0x7f3a8c02a410 comp rx=0 tx=0).stop 2024-09-11T21:52:46.589 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:52:46.570+0000 7f3a97870640 1 -- 172.21.15.161:0/886359295 shutdown_connections 2024-09-11T21:52:46.589 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:52:46.570+0000 7f3a97870640 1 --2- 172.21.15.161:0/886359295 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a901037c0 0x7f3a9010a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:52:46.590 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:52:46.570+0000 7f3a97870640 1 --2- 172.21.15.161:0/886359295 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3a90102e00 0x7f3a90103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:52:46.590 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:52:46.570+0000 7f3a97870640 1 --2- 172.21.15.161:0/886359295 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a90101c00 0x7f3a90102000 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:52:46.590 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:52:46.570+0000 7f3a97870640 1 -- 172.21.15.161:0/886359295 >> 172.21.15.161:0/886359295 conn(0x7f3a900fd390 msgr2=0x7f3a900ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:52:46.590 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:52:46.570+0000 7f3a97870640 1 -- 172.21.15.161:0/886359295 shutdown_connections 2024-09-11T21:52:46.590 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:52:46.570+0000 7f3a97870640 1 -- 172.21.15.161:0/886359295 wait complete. 2024-09-11T21:52:46.590 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:52:46.571+0000 7f3a97870640 1 Processor -- start 2024-09-11T21:52:46.590 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:52:46.571+0000 7f3a97870640 1 -- start start 2024-09-11T21:52:46.590 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:52:46.571+0000 7f3a97870640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a90101c00 0x7f3a90197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:52:46.590 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:52:46.571+0000 7f3a97870640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3a90102e00 0x7f3a901983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:52:46.590 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:52:46.571+0000 7f3a97870640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a901037c0 0x7f3a9019f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:52:46.590 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:52:46.571+0000 7f3a97870640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3a9010d860 con 0x7f3a90102e00 2024-09-11T21:52:46.590 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:52:46.571+0000 7f3a97870640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3a9010d6e0 con 0x7f3a90101c00 2024-09-11T21:52:46.590 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:52:46.571+0000 7f3a97870640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3a9010d9e0 con 0x7f3a901037c0 2024-09-11T21:52:46.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:52:46.571+0000 7f3a95e0d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a901037c0 0x7f3a9019f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:52:46.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:52:46.571+0000 7f3a95e0d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a901037c0 0x7f3a9019f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:39276/0 (socket says 172.21.15.161:39276) 2024-09-11T21:52:46.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:52:46.571+0000 7f3a95e0d640 1 -- 172.21.15.161:0/215957392 learned_addr learned my addr 172.21.15.161:0/215957392 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-09-11T21:52:46.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:52:46.572+0000 7f3a9560c640 1 --2- 172.21.15.161:0/215957392 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a90101c00 0x7f3a90197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:52:46.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:52:46.572+0000 7f3a95e0d640 1 -- 172.21.15.161:0/215957392 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a90101c00 msgr2=0x7f3a90197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:52:46.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:52:46.572+0000 7f3a94e0b640 1 --2- 172.21.15.161:0/215957392 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3a90102e00 0x7f3a901983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:52:46.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:52:46.572+0000 7f3a95e0d640 1 --2- 172.21.15.161:0/215957392 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a90101c00 0x7f3a90197eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:52:46.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:52:46.572+0000 7f3a95e0d640 1 -- 172.21.15.161:0/215957392 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3a90102e00 msgr2=0x7f3a901983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:52:46.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:52:46.572+0000 7f3a95e0d640 1 --2- 172.21.15.161:0/215957392 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3a90102e00 0x7f3a901983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:52:46.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:52:46.572+0000 7f3a95e0d640 1 -- 172.21.15.161:0/215957392 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a9019fae0 con 0x7f3a901037c0 2024-09-11T21:52:46.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:52:46.572+0000 7f3a95e0d640 1 --2- 172.21.15.161:0/215957392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a901037c0 0x7f3a9019f470 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3a8800cad0 tx=0x7f3a8800cf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:52:46.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:52:46.572+0000 7f3a867fc640 1 -- 172.21.15.161:0/215957392 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a8800ae20 con 0x7f3a901037c0 2024-09-11T21:52:46.591 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:52:46.572+0000 7f3a867fc640 1 -- 172.21.15.161:0/215957392 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a88005020 con 0x7f3a901037c0 2024-09-11T21:52:46.592 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:52:46.572+0000 7f3a97870640 1 -- 172.21.15.161:0/215957392 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a9019fdd0 con 0x7f3a901037c0 2024-09-11T21:52:46.592 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:52:46.572+0000 7f3a867fc640 1 -- 172.21.15.161:0/215957392 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a8800f6f0 con 0x7f3a901037c0 2024-09-11T21:52:46.592 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:52:46.572+0000 7f3a97870640 1 -- 172.21.15.161:0/215957392 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a901a0230 con 0x7f3a901037c0 2024-09-11T21:52:46.592 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:52:46.573+0000 7f3a97870640 1 -- 172.21.15.161:0/215957392 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a90109020 con 0x7f3a901037c0 2024-09-11T21:52:46.592 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:52:46.574+0000 7f3a867fc640 1 -- 172.21.15.161:0/215957392 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f3a880051d0 con 0x7f3a901037c0 2024-09-11T21:52:46.592 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:52:46.575+0000 7f3a867fc640 1 --2- 172.21.15.161:0/215957392 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f3a7c0a2bb0 0x7f3a7c0a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:52:46.592 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:52:46.575+0000 7f3a867fc640 1 -- 172.21.15.161:0/215957392 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5430+0+0 (secure 0 0 0) 0x7f3a880cea40 con 0x7f3a901037c0 2024-09-11T21:52:46.592 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:52:46.575+0000 7f3a9560c640 1 --2- 172.21.15.161:0/215957392 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f3a7c0a2bb0 0x7f3a7c0a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:52:46.592 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:52:46.575+0000 7f3a9560c640 1 --2- 172.21.15.161:0/215957392 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f3a7c0a2bb0 0x7f3a7c0a5070 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c006010 tx=0x7f3a8c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:52:46.592 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:52:46.578+0000 7f3a867fc640 1 -- 172.21.15.161:0/215957392 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3a88014030 con 0x7f3a901037c0 2024-09-11T21:52:46.753 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:52:46.752+0000 7f3a97870640 1 -- 172.21.15.161:0/215957392 --> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f3a90062630 con 0x7f3a7c0a2bb0 2024-09-11T21:52:47.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:47 smithi001 ceph-mon[30935]: purged_snaps scrub starts 2024-09-11T21:52:47.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:47 smithi001 ceph-mon[30935]: purged_snaps scrub ok 2024-09-11T21:52:47.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:47 smithi001 ceph-mon[30935]: Detected new or changed devices on smithi053 2024-09-11T21:52:47.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:47 smithi001 ceph-mon[30935]: Adjusting osd_memory_target on smithi053 to 5697M 2024-09-11T21:52:47.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:47 smithi001 ceph-mon[30935]: osd.7 [v2:172.21.15.53:6816/2132057697,v1:172.21.15.53:6817/2132057697] boot 2024-09-11T21:52:47.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:47 smithi001 ceph-mon[30935]: osdmap e51: 8 total, 8 up, 8 in 2024-09-11T21:52:47.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:47 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T21:52:47.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:47 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T21:52:47.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:47 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T21:52:47.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:47 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:52:47.500 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:47 smithi161 ceph-mon[35615]: purged_snaps scrub starts 2024-09-11T21:52:47.501 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:47 smithi161 ceph-mon[35615]: purged_snaps scrub ok 2024-09-11T21:52:47.501 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:47 smithi161 ceph-mon[35615]: Detected new or changed devices on smithi053 2024-09-11T21:52:47.501 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:47 smithi161 ceph-mon[35615]: Adjusting osd_memory_target on smithi053 to 5697M 2024-09-11T21:52:47.501 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:47 smithi161 ceph-mon[35615]: osd.7 [v2:172.21.15.53:6816/2132057697,v1:172.21.15.53:6817/2132057697] boot 2024-09-11T21:52:47.501 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:47 smithi161 ceph-mon[35615]: osdmap e51: 8 total, 8 up, 8 in 2024-09-11T21:52:47.501 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:47 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T21:52:47.501 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:47 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T21:52:47.501 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:47 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T21:52:47.501 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:47 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:52:47.506 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:47 smithi053 ceph-mon[34908]: purged_snaps scrub starts 2024-09-11T21:52:47.506 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:47 smithi053 ceph-mon[34908]: purged_snaps scrub ok 2024-09-11T21:52:47.506 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:47 smithi053 ceph-mon[34908]: Detected new or changed devices on smithi053 2024-09-11T21:52:47.506 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:47 smithi053 ceph-mon[34908]: Adjusting osd_memory_target on smithi053 to 5697M 2024-09-11T21:52:47.506 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:47 smithi053 ceph-mon[34908]: osd.7 [v2:172.21.15.53:6816/2132057697,v1:172.21.15.53:6817/2132057697] boot 2024-09-11T21:52:47.506 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:47 smithi053 ceph-mon[34908]: osdmap e51: 8 total, 8 up, 8 in 2024-09-11T21:52:47.506 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:47 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T21:52:47.507 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:47 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T21:52:47.507 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:47 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T21:52:47.507 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:47 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:52:48.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:48 smithi001 ceph-mon[30935]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-11T21:52:48.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:48 smithi001 ceph-mon[30935]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:52:48.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:48 smithi001 ceph-mon[30935]: osdmap e52: 8 total, 8 up, 8 in 2024-09-11T21:52:48.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:48 smithi001 ceph-mon[30935]: osdmap e53: 8 total, 8 up, 8 in 2024-09-11T21:52:48.518 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:48 smithi161 ceph-mon[35615]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-11T21:52:48.518 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:48 smithi161 ceph-mon[35615]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:52:48.519 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:48 smithi161 ceph-mon[35615]: osdmap e52: 8 total, 8 up, 8 in 2024-09-11T21:52:48.519 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:48 smithi161 ceph-mon[35615]: osdmap e53: 8 total, 8 up, 8 in 2024-09-11T21:52:48.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:48 smithi053 ceph-mon[34908]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-11T21:52:48.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:48 smithi053 ceph-mon[34908]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:52:48.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:48 smithi053 ceph-mon[34908]: osdmap e52: 8 total, 8 up, 8 in 2024-09-11T21:52:48.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:48 smithi053 ceph-mon[34908]: osdmap e53: 8 total, 8 up, 8 in 2024-09-11T21:52:50.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:50 smithi001 ceph-mon[30935]: pgmap v175: 1 pgs: 1 remapped+peering; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-11T21:52:50.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:50 smithi001 ceph-mon[30935]: osdmap e54: 8 total, 8 up, 8 in 2024-09-11T21:52:50.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:50 smithi001 ceph-mon[30935]: from='client.? 172.21.15.161:0/71201034' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "36ea401b-3c93-4b9d-8880-298d1eb6e30b"}]: dispatch 2024-09-11T21:52:50.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:50 smithi001 ceph-mon[30935]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "36ea401b-3c93-4b9d-8880-298d1eb6e30b"}]: dispatch 2024-09-11T21:52:50.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:50 smithi001 ceph-mon[30935]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "36ea401b-3c93-4b9d-8880-298d1eb6e30b"}]': finished 2024-09-11T21:52:50.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:50 smithi001 ceph-mon[30935]: osdmap e55: 9 total, 8 up, 9 in 2024-09-11T21:52:50.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:50 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T21:52:50.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:50 smithi001 ceph-mon[30935]: from='client.? 172.21.15.161:0/942107154' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T21:52:50.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:50 smithi053 ceph-mon[34908]: pgmap v175: 1 pgs: 1 remapped+peering; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-11T21:52:50.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:50 smithi053 ceph-mon[34908]: osdmap e54: 8 total, 8 up, 8 in 2024-09-11T21:52:50.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:50 smithi053 ceph-mon[34908]: from='client.? 172.21.15.161:0/71201034' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "36ea401b-3c93-4b9d-8880-298d1eb6e30b"}]: dispatch 2024-09-11T21:52:50.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:50 smithi053 ceph-mon[34908]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "36ea401b-3c93-4b9d-8880-298d1eb6e30b"}]: dispatch 2024-09-11T21:52:50.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:50 smithi053 ceph-mon[34908]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "36ea401b-3c93-4b9d-8880-298d1eb6e30b"}]': finished 2024-09-11T21:52:50.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:50 smithi053 ceph-mon[34908]: osdmap e55: 9 total, 8 up, 9 in 2024-09-11T21:52:50.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:50 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T21:52:50.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:50 smithi053 ceph-mon[34908]: from='client.? 172.21.15.161:0/942107154' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T21:52:50.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:50 smithi161 ceph-mon[35615]: pgmap v175: 1 pgs: 1 remapped+peering; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-11T21:52:50.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:50 smithi161 ceph-mon[35615]: osdmap e54: 8 total, 8 up, 8 in 2024-09-11T21:52:50.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:50 smithi161 ceph-mon[35615]: from='client.? 172.21.15.161:0/71201034' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "36ea401b-3c93-4b9d-8880-298d1eb6e30b"}]: dispatch 2024-09-11T21:52:50.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:50 smithi161 ceph-mon[35615]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "36ea401b-3c93-4b9d-8880-298d1eb6e30b"}]: dispatch 2024-09-11T21:52:50.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:50 smithi161 ceph-mon[35615]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "36ea401b-3c93-4b9d-8880-298d1eb6e30b"}]': finished 2024-09-11T21:52:50.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:50 smithi161 ceph-mon[35615]: osdmap e55: 9 total, 8 up, 9 in 2024-09-11T21:52:50.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:50 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T21:52:50.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:50 smithi161 ceph-mon[35615]: from='client.? 172.21.15.161:0/942107154' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T21:52:52.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:52 smithi001 ceph-mon[30935]: pgmap v178: 1 pgs: 1 remapped+peering; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-11T21:52:52.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:52 smithi053 ceph-mon[34908]: pgmap v178: 1 pgs: 1 remapped+peering; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-11T21:52:52.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:52 smithi161 ceph-mon[35615]: pgmap v178: 1 pgs: 1 remapped+peering; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-11T21:52:54.246 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:54 smithi161 ceph-mon[35615]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T21:52:54.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:54 smithi001 ceph-mon[30935]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T21:52:54.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:54 smithi053 ceph-mon[34908]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T21:52:56.240 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:56 smithi161 ceph-mon[35615]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T21:52:56.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:56 smithi001 ceph-mon[30935]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T21:52:56.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:56 smithi053 ceph-mon[34908]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T21:52:58.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:58 smithi161 ceph-mon[35615]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T21:52:58.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:58 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-11T21:52:58.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:58 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:52:58.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:52:58 smithi161 ceph-mon[35615]: Deploying daemon osd.8 on smithi161 2024-09-11T21:52:58.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:58 smithi001 ceph-mon[30935]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T21:52:58.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:58 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-11T21:52:58.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:58 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:52:58.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:52:58 smithi001 ceph-mon[30935]: Deploying daemon osd.8 on smithi161 2024-09-11T21:52:58.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:58 smithi053 ceph-mon[34908]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T21:52:58.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:58 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-11T21:52:58.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:58 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:52:58.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:52:58 smithi053 ceph-mon[34908]: Deploying daemon osd.8 on smithi161 2024-09-11T21:53:00.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:00 smithi161 ceph-mon[35615]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T21:53:00.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:00 smithi001 ceph-mon[30935]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T21:53:00.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:00 smithi053 ceph-mon[34908]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T21:53:02.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:02 smithi161 ceph-mon[35615]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T21:53:02.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:02 smithi053 ceph-mon[34908]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T21:53:02.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:02 smithi001 ceph-mon[30935]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T21:53:04.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:04 smithi161 ceph-mon[35615]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T21:53:04.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:04 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:53:04.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:04 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:04.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:04 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:04.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:04 smithi053 ceph-mon[34908]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T21:53:04.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:04 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:53:04.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:04 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:04.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:04 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:04.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:04 smithi001 ceph-mon[30935]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T21:53:04.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:04 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:53:04.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:04 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:04.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:04 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:06.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:06 smithi053 ceph-mon[34908]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T21:53:06.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:06 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:06.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:06 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:06.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:06 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:53:06.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:06 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:53:06.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:06 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:06.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:06 smithi161 ceph-mon[35615]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T21:53:06.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:06 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:06.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:06 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:06.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:06 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:53:06.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:06 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:53:06.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:06 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:06.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:06 smithi001 ceph-mon[30935]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T21:53:06.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:06 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:06.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:06 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:06.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:06 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:53:06.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:06 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:53:06.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:06 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:07.085 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:53:07.084+0000 7f3a867fc640 1 -- 172.21.15.161:0/215957392 <== mgr.14150 v2:172.21.15.1:6800/3890350886 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3a90062630 con 0x7f3a7c0a2bb0 2024-09-11T21:53:07.085 INFO:teuthology.orchestra.run.smithi161.stdout:Created osd(s) 8 on host 'smithi161' 2024-09-11T21:53:07.087 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:53:07.087+0000 7f3a97870640 1 -- 172.21.15.161:0/215957392 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f3a7c0a2bb0 msgr2=0x7f3a7c0a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:53:07.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:53:07.087+0000 7f3a97870640 1 --2- 172.21.15.161:0/215957392 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f3a7c0a2bb0 0x7f3a7c0a5070 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c006010 tx=0x7f3a8c000bc0 comp rx=0 tx=0).stop 2024-09-11T21:53:07.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:53:07.087+0000 7f3a97870640 1 -- 172.21.15.161:0/215957392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a901037c0 msgr2=0x7f3a9019f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:53:07.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:53:07.087+0000 7f3a97870640 1 --2- 172.21.15.161:0/215957392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a901037c0 0x7f3a9019f470 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3a8800cad0 tx=0x7f3a8800cf90 comp rx=0 tx=0).stop 2024-09-11T21:53:07.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:53:07.087+0000 7f3a97870640 1 -- 172.21.15.161:0/215957392 shutdown_connections 2024-09-11T21:53:07.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:53:07.087+0000 7f3a97870640 1 --2- 172.21.15.161:0/215957392 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f3a7c0a2bb0 0x7f3a7c0a5070 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:53:07.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:53:07.087+0000 7f3a97870640 1 --2- 172.21.15.161:0/215957392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a901037c0 0x7f3a9019f470 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:53:07.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:53:07.087+0000 7f3a97870640 1 --2- 172.21.15.161:0/215957392 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3a90102e00 0x7f3a901983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:53:07.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:53:07.087+0000 7f3a97870640 1 --2- 172.21.15.161:0/215957392 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3a90101c00 0x7f3a90197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:53:07.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:53:07.087+0000 7f3a97870640 1 -- 172.21.15.161:0/215957392 >> 172.21.15.161:0/215957392 conn(0x7f3a900fd390 msgr2=0x7f3a901061f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:53:07.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:53:07.087+0000 7f3a97870640 1 -- 172.21.15.161:0/215957392 shutdown_connections 2024-09-11T21:53:07.088 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:53:07.087+0000 7f3a97870640 1 -- 172.21.15.161:0/215957392 wait complete. 2024-09-11T21:53:07.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:07 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:53:07.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:07 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:07.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:07 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:07.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:07 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:53:07.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:07 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:07.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:07 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:07.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:07 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:53:07.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:07 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:07.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:07 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:07.747 DEBUG:teuthology.orchestra.run.smithi161:osd.8> sudo journalctl -f -n 0 -u ceph-3ffca83c-7087-11ef-bcea-c7b262605968@osd.8.service 2024-09-11T21:53:07.749 INFO:tasks.cephadm:Deploying osd.9 on smithi001 with /dev/vg_nvme/lv_1... 2024-09-11T21:53:07.750 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-11T21:53:08.016 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:53:08.316 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:08 smithi161 ceph-mon[35615]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T21:53:08.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:08 smithi001 ceph-mon[30935]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T21:53:08.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:08 smithi053 ceph-mon[34908]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T21:53:09.468 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 21:53:09 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T21:53:09.340+0000 7f3512ac1640 -1 osd.8 0 log_to_monitors true 2024-09-11T21:53:10.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:10 smithi001 ceph-mon[30935]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T21:53:10.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:10 smithi001 ceph-mon[30935]: from='osd.8 [v2:172.21.15.161:6816/1838248669,v1:172.21.15.161:6817/1838248669]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-11T21:53:10.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:10 smithi001 ceph-mon[30935]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-11T21:53:10.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:10 smithi053 ceph-mon[34908]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T21:53:10.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:10 smithi053 ceph-mon[34908]: from='osd.8 [v2:172.21.15.161:6816/1838248669,v1:172.21.15.161:6817/1838248669]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-11T21:53:10.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:10 smithi053 ceph-mon[34908]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-11T21:53:10.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:10 smithi161 ceph-mon[35615]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T21:53:10.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:10 smithi161 ceph-mon[35615]: from='osd.8 [v2:172.21.15.161:6816/1838248669,v1:172.21.15.161:6817/1838248669]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-11T21:53:10.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:10 smithi161 ceph-mon[35615]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-11T21:53:10.807 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:53:10.834 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi001:vg_nvme/lv_1 2024-09-11T21:53:11.099 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:53:11.480 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:11 smithi001 ceph-mon[30935]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-11T21:53:11.480 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:11 smithi001 ceph-mon[30935]: osdmap e56: 9 total, 8 up, 9 in 2024-09-11T21:53:11.480 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:11 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T21:53:11.481 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:11 smithi001 ceph-mon[30935]: from='osd.8 [v2:172.21.15.161:6816/1838248669,v1:172.21.15.161:6817/1838248669]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-09-11T21:53:11.481 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:11 smithi001 ceph-mon[30935]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-09-11T21:53:11.481 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:11 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:11.481 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:11 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:11.481 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:11 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:53:11.481 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:11 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:53:11.481 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:11 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:53:11.481 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:11 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:11.481 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:11 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:53:11.481 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:11 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:53:11.481 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:11 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:11.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:11 smithi053 ceph-mon[34908]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-11T21:53:11.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:11 smithi053 ceph-mon[34908]: osdmap e56: 9 total, 8 up, 9 in 2024-09-11T21:53:11.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:11 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T21:53:11.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:11 smithi053 ceph-mon[34908]: from='osd.8 [v2:172.21.15.161:6816/1838248669,v1:172.21.15.161:6817/1838248669]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-09-11T21:53:11.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:11 smithi053 ceph-mon[34908]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-09-11T21:53:11.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:11 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:11.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:11 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:11.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:11 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:53:11.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:11 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:53:11.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:11 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:53:11.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:11 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:11.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:11 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:53:11.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:11 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:53:11.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:11 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:11.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:11 smithi161 ceph-mon[35615]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-11T21:53:11.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:11 smithi161 ceph-mon[35615]: osdmap e56: 9 total, 8 up, 9 in 2024-09-11T21:53:11.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:11 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T21:53:11.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:11 smithi161 ceph-mon[35615]: from='osd.8 [v2:172.21.15.161:6816/1838248669,v1:172.21.15.161:6817/1838248669]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-09-11T21:53:11.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:11 smithi161 ceph-mon[35615]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-09-11T21:53:11.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:11 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:11.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:11 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:11.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:11 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:53:11.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:11 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:53:11.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:11 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:53:11.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:11 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:11.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:11 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:53:11.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:11 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:53:11.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:11 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:11.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:53:11.610+0000 7fdaa7684640 1 -- 172.21.15.1:0/4099000578 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdaa0101c10 msgr2=0x7fdaa0102010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:53:11.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:53:11.610+0000 7fdaa7684640 1 --2- 172.21.15.1:0/4099000578 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdaa0101c10 0x7fdaa0102010 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fda9c009900 tx=0x7fda9c02a410 comp rx=0 tx=0).stop 2024-09-11T21:53:11.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:53:11.610+0000 7fdaa7684640 1 -- 172.21.15.1:0/4099000578 shutdown_connections 2024-09-11T21:53:11.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:53:11.610+0000 7fdaa7684640 1 --2- 172.21.15.1:0/4099000578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdaa01037d0 0x7fdaa010a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:53:11.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:53:11.610+0000 7fdaa7684640 1 --2- 172.21.15.1:0/4099000578 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdaa0102e10 0x7fdaa0103290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:53:11.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:53:11.610+0000 7fdaa7684640 1 --2- 172.21.15.1:0/4099000578 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdaa0101c10 0x7fdaa0102010 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:53:11.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:53:11.610+0000 7fdaa7684640 1 -- 172.21.15.1:0/4099000578 >> 172.21.15.1:0/4099000578 conn(0x7fdaa00fd3c0 msgr2=0x7fdaa00ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:53:11.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:53:11.610+0000 7fdaa7684640 1 -- 172.21.15.1:0/4099000578 shutdown_connections 2024-09-11T21:53:11.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:53:11.610+0000 7fdaa7684640 1 -- 172.21.15.1:0/4099000578 wait complete. 2024-09-11T21:53:11.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:53:11.611+0000 7fdaa7684640 1 Processor -- start 2024-09-11T21:53:11.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:53:11.611+0000 7fdaa7684640 1 -- start start 2024-09-11T21:53:11.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:53:11.611+0000 7fdaa7684640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdaa0101c10 0x7fdaa0197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:53:11.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:53:11.611+0000 7fdaa7684640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdaa0102e10 0x7fdaa0198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:53:11.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:53:11.611+0000 7fdaa7684640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdaa01037d0 0x7fdaa019f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:53:11.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:53:11.611+0000 7fdaa7684640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa010d870 con 0x7fdaa0102e10 2024-09-11T21:53:11.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:53:11.611+0000 7fdaa7684640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa010d6f0 con 0x7fdaa0101c10 2024-09-11T21:53:11.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:53:11.611+0000 7fdaa7684640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa010d9f0 con 0x7fdaa01037d0 2024-09-11T21:53:11.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:53:11.611+0000 7fdaa4c1f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdaa0102e10 0x7fdaa0198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:53:11.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:53:11.611+0000 7fdaa4c1f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdaa0102e10 0x7fdaa0198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59832/0 (socket says 172.21.15.1:59832) 2024-09-11T21:53:11.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:53:11.611+0000 7fdaa4c1f640 1 -- 172.21.15.1:0/1558331154 learned_addr learned my addr 172.21.15.1:0/1558331154 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:53:11.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:53:11.611+0000 7fdaa5c21640 1 --2- 172.21.15.1:0/1558331154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdaa01037d0 0x7fdaa019f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:53:11.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:53:11.611+0000 7fdaa5420640 1 --2- 172.21.15.1:0/1558331154 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdaa0101c10 0x7fdaa0197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:53:11.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:53:11.611+0000 7fdaa4c1f640 1 -- 172.21.15.1:0/1558331154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdaa01037d0 msgr2=0x7fdaa019f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:53:11.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:53:11.611+0000 7fdaa4c1f640 1 --2- 172.21.15.1:0/1558331154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdaa01037d0 0x7fdaa019f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:53:11.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:53:11.611+0000 7fdaa4c1f640 1 -- 172.21.15.1:0/1558331154 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdaa0101c10 msgr2=0x7fdaa0197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:53:11.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:53:11.611+0000 7fdaa4c1f640 1 --2- 172.21.15.1:0/1558331154 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdaa0101c10 0x7fdaa0197ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:53:11.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:53:11.611+0000 7fdaa4c1f640 1 -- 172.21.15.1:0/1558331154 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaa019fb20 con 0x7fdaa0102e10 2024-09-11T21:53:11.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:53:11.611+0000 7fdaa5420640 1 --2- 172.21.15.1:0/1558331154 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdaa0101c10 0x7fdaa0197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:53:11.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:53:11.612+0000 7fdaa4c1f640 1 --2- 172.21.15.1:0/1558331154 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdaa0102e10 0x7fdaa0198430 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fda9000d950 tx=0x7fda9000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:53:11.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:53:11.612+0000 7fda967fc640 1 -- 172.21.15.1:0/1558331154 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda90017030 con 0x7fdaa0102e10 2024-09-11T21:53:11.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:53:11.612+0000 7fda967fc640 1 -- 172.21.15.1:0/1558331154 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda90004020 con 0x7fdaa0102e10 2024-09-11T21:53:11.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:53:11.612+0000 7fdaa7684640 1 -- 172.21.15.1:0/1558331154 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaa019fe10 con 0x7fdaa0102e10 2024-09-11T21:53:11.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:53:11.612+0000 7fda967fc640 1 -- 172.21.15.1:0/1558331154 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda900047a0 con 0x7fdaa0102e10 2024-09-11T21:53:11.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:53:11.612+0000 7fdaa7684640 1 -- 172.21.15.1:0/1558331154 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaa01a02c0 con 0x7fdaa0102e10 2024-09-11T21:53:11.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:53:11.613+0000 7fdaa7684640 1 -- 172.21.15.1:0/1558331154 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaa0109060 con 0x7fdaa0102e10 2024-09-11T21:53:11.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:53:11.614+0000 7fda967fc640 1 -- 172.21.15.1:0/1558331154 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fda900041d0 con 0x7fdaa0102e10 2024-09-11T21:53:11.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:53:11.615+0000 7fda967fc640 1 --2- 172.21.15.1:0/1558331154 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fda8c0a2bb0 0x7fda8c0a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:53:11.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:53:11.615+0000 7fdaa5420640 1 --2- 172.21.15.1:0/1558331154 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fda8c0a2bb0 0x7fda8c0a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:53:11.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:53:11.615+0000 7fda967fc640 1 -- 172.21.15.1:0/1558331154 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5549+0+0 (secure 0 0 0) 0x7fda900ce030 con 0x7fdaa0102e10 2024-09-11T21:53:11.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:53:11.615+0000 7fdaa5420640 1 --2- 172.21.15.1:0/1558331154 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fda8c0a2bb0 0x7fda8c0a5070 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fda9c006010 tx=0x7fda9c032dd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:53:11.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:53:11.619+0000 7fda967fc640 1 -- 172.21.15.1:0/1558331154 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fda9009b9c0 con 0x7fdaa0102e10 2024-09-11T21:53:11.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:53:11.810+0000 7fdaa7684640 1 -- 172.21.15.1:0/1558331154 --> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fdaa0062630 con 0x7fda8c0a2bb0 2024-09-11T21:53:12.092 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 21:53:11 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T21:53:11.625+0000 7f350f258640 -1 osd.8 0 waiting for initial osdmap 2024-09-11T21:53:12.092 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 21:53:11 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T21:53:11.634+0000 7f3506864640 -1 osd.8 57 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T21:53:12.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:12 smithi053 ceph-mon[34908]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T21:53:12.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:12 smithi053 ceph-mon[34908]: Detected new or changed devices on smithi161 2024-09-11T21:53:12.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:12 smithi053 ceph-mon[34908]: Adjusting osd_memory_target on smithi161 to 5696M 2024-09-11T21:53:12.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:12 smithi053 ceph-mon[34908]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-09-11T21:53:12.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:12 smithi053 ceph-mon[34908]: osdmap e57: 9 total, 8 up, 9 in 2024-09-11T21:53:12.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:12 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T21:53:12.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:12 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T21:53:12.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:12 smithi053 ceph-mon[34908]: from='osd.8 ' entity='osd.8' 2024-09-11T21:53:12.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:12 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T21:53:12.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:12 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T21:53:12.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:12 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:53:12.553 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:12 smithi001 ceph-mon[30935]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T21:53:12.553 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:12 smithi001 ceph-mon[30935]: Detected new or changed devices on smithi161 2024-09-11T21:53:12.553 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:12 smithi001 ceph-mon[30935]: Adjusting osd_memory_target on smithi161 to 5696M 2024-09-11T21:53:12.553 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:12 smithi001 ceph-mon[30935]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-09-11T21:53:12.553 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:12 smithi001 ceph-mon[30935]: osdmap e57: 9 total, 8 up, 9 in 2024-09-11T21:53:12.553 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:12 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T21:53:12.554 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:12 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T21:53:12.554 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:12 smithi001 ceph-mon[30935]: from='osd.8 ' entity='osd.8' 2024-09-11T21:53:12.554 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:12 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T21:53:12.554 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:12 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T21:53:12.554 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:12 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:53:12.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:12 smithi161 ceph-mon[35615]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T21:53:12.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:12 smithi161 ceph-mon[35615]: Detected new or changed devices on smithi161 2024-09-11T21:53:12.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:12 smithi161 ceph-mon[35615]: Adjusting osd_memory_target on smithi161 to 5696M 2024-09-11T21:53:12.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:12 smithi161 ceph-mon[35615]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-09-11T21:53:12.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:12 smithi161 ceph-mon[35615]: osdmap e57: 9 total, 8 up, 9 in 2024-09-11T21:53:12.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:12 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T21:53:12.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:12 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T21:53:12.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:12 smithi161 ceph-mon[35615]: from='osd.8 ' entity='osd.8' 2024-09-11T21:53:12.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:12 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T21:53:12.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:12 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T21:53:12.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:12 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:53:13.484 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:13 smithi001 ceph-mon[30935]: purged_snaps scrub starts 2024-09-11T21:53:13.484 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:13 smithi001 ceph-mon[30935]: purged_snaps scrub ok 2024-09-11T21:53:13.484 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:13 smithi001 ceph-mon[30935]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:53:13.485 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:13 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T21:53:13.485 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:13 smithi001 ceph-mon[30935]: osd.8 [v2:172.21.15.161:6816/1838248669,v1:172.21.15.161:6817/1838248669] boot 2024-09-11T21:53:13.485 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:13 smithi001 ceph-mon[30935]: osdmap e58: 9 total, 9 up, 9 in 2024-09-11T21:53:13.485 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:13 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T21:53:13.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:13 smithi053 ceph-mon[34908]: purged_snaps scrub starts 2024-09-11T21:53:13.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:13 smithi053 ceph-mon[34908]: purged_snaps scrub ok 2024-09-11T21:53:13.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:13 smithi053 ceph-mon[34908]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:53:13.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:13 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T21:53:13.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:13 smithi053 ceph-mon[34908]: osd.8 [v2:172.21.15.161:6816/1838248669,v1:172.21.15.161:6817/1838248669] boot 2024-09-11T21:53:13.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:13 smithi053 ceph-mon[34908]: osdmap e58: 9 total, 9 up, 9 in 2024-09-11T21:53:13.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:13 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T21:53:13.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:13 smithi161 ceph-mon[35615]: purged_snaps scrub starts 2024-09-11T21:53:13.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:13 smithi161 ceph-mon[35615]: purged_snaps scrub ok 2024-09-11T21:53:13.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:13 smithi161 ceph-mon[35615]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:53:13.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:13 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T21:53:13.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:13 smithi161 ceph-mon[35615]: osd.8 [v2:172.21.15.161:6816/1838248669,v1:172.21.15.161:6817/1838248669] boot 2024-09-11T21:53:13.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:13 smithi161 ceph-mon[35615]: osdmap e58: 9 total, 9 up, 9 in 2024-09-11T21:53:13.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:13 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T21:53:14.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:14 smithi001 ceph-mon[30935]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T21:53:14.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:14 smithi001 ceph-mon[30935]: osdmap e59: 9 total, 9 up, 9 in 2024-09-11T21:53:14.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:14 smithi053 ceph-mon[34908]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T21:53:14.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:14 smithi053 ceph-mon[34908]: osdmap e59: 9 total, 9 up, 9 in 2024-09-11T21:53:14.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:14 smithi161 ceph-mon[35615]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T21:53:14.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:14 smithi161 ceph-mon[35615]: osdmap e59: 9 total, 9 up, 9 in 2024-09-11T21:53:15.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:15 smithi053 ceph-mon[34908]: osdmap e60: 9 total, 9 up, 9 in 2024-09-11T21:53:15.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:15 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/3781262359' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b788d3f-a3bc-4463-bd99-95fdb7a52768"}]: dispatch 2024-09-11T21:53:15.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:15 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/3781262359' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4b788d3f-a3bc-4463-bd99-95fdb7a52768"}]': finished 2024-09-11T21:53:15.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:15 smithi053 ceph-mon[34908]: osdmap e61: 10 total, 9 up, 10 in 2024-09-11T21:53:15.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:15 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T21:53:15.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:15 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/1453628285' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T21:53:15.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:15 smithi161 ceph-mon[35615]: osdmap e60: 9 total, 9 up, 9 in 2024-09-11T21:53:15.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:15 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/3781262359' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b788d3f-a3bc-4463-bd99-95fdb7a52768"}]: dispatch 2024-09-11T21:53:15.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:15 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/3781262359' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4b788d3f-a3bc-4463-bd99-95fdb7a52768"}]': finished 2024-09-11T21:53:15.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:15 smithi161 ceph-mon[35615]: osdmap e61: 10 total, 9 up, 10 in 2024-09-11T21:53:15.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:15 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T21:53:15.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:15 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/1453628285' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T21:53:15.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:15 smithi001 ceph-mon[30935]: osdmap e60: 9 total, 9 up, 9 in 2024-09-11T21:53:15.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:15 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/3781262359' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b788d3f-a3bc-4463-bd99-95fdb7a52768"}]: dispatch 2024-09-11T21:53:15.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:15 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/3781262359' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4b788d3f-a3bc-4463-bd99-95fdb7a52768"}]': finished 2024-09-11T21:53:15.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:15 smithi001 ceph-mon[30935]: osdmap e61: 10 total, 9 up, 10 in 2024-09-11T21:53:15.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:15 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T21:53:15.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:15 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/1453628285' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T21:53:16.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:16 smithi001 ceph-mon[30935]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-11T21:53:16.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:16 smithi001 ceph-mon[30935]: osdmap e62: 10 total, 9 up, 10 in 2024-09-11T21:53:16.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:16 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T21:53:16.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:16 smithi053 ceph-mon[34908]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-11T21:53:16.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:16 smithi053 ceph-mon[34908]: osdmap e62: 10 total, 9 up, 10 in 2024-09-11T21:53:16.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:16 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T21:53:16.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:16 smithi161 ceph-mon[35615]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-11T21:53:16.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:16 smithi161 ceph-mon[35615]: osdmap e62: 10 total, 9 up, 10 in 2024-09-11T21:53:16.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:16 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T21:53:17.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:17 smithi001 ceph-mon[30935]: osdmap e63: 10 total, 9 up, 10 in 2024-09-11T21:53:17.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:17 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T21:53:17.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:17 smithi053 ceph-mon[34908]: osdmap e63: 10 total, 9 up, 10 in 2024-09-11T21:53:17.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:17 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T21:53:17.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:17 smithi161 ceph-mon[35615]: osdmap e63: 10 total, 9 up, 10 in 2024-09-11T21:53:17.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:17 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T21:53:18.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:18 smithi001 ceph-mon[30935]: pgmap v199: 1 pgs: 1 peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 172 KiB/s, 0 objects/s recovering 2024-09-11T21:53:18.789 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:18 smithi161 ceph-mon[35615]: pgmap v199: 1 pgs: 1 peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 172 KiB/s, 0 objects/s recovering 2024-09-11T21:53:18.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:18 smithi053 ceph-mon[34908]: pgmap v199: 1 pgs: 1 peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 172 KiB/s, 0 objects/s recovering 2024-09-11T21:53:20.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:20 smithi001 ceph-mon[30935]: pgmap v200: 1 pgs: 1 peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 133 KiB/s, 0 objects/s recovering 2024-09-11T21:53:20.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:20 smithi053 ceph-mon[34908]: pgmap v200: 1 pgs: 1 peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 133 KiB/s, 0 objects/s recovering 2024-09-11T21:53:20.841 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:20 smithi161 ceph-mon[35615]: pgmap v200: 1 pgs: 1 peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 133 KiB/s, 0 objects/s recovering 2024-09-11T21:53:22.660 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:22 smithi001 ceph-mon[30935]: pgmap v201: 1 pgs: 1 peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-09-11T21:53:22.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:22 smithi053 ceph-mon[34908]: pgmap v201: 1 pgs: 1 peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-09-11T21:53:22.841 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:22 smithi161 ceph-mon[35615]: pgmap v201: 1 pgs: 1 peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-09-11T21:53:23.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:23 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-11T21:53:23.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:23 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:53:23.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:23 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-11T21:53:23.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:23 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:53:23.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:23 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-11T21:53:23.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:23 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:53:24.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:24 smithi001 ceph-mon[30935]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T21:53:24.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:24 smithi001 ceph-mon[30935]: Deploying daemon osd.9 on smithi001 2024-09-11T21:53:24.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:24 smithi053 ceph-mon[34908]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T21:53:24.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:24 smithi053 ceph-mon[34908]: Deploying daemon osd.9 on smithi001 2024-09-11T21:53:24.841 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:24 smithi161 ceph-mon[35615]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T21:53:24.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:24 smithi161 ceph-mon[35615]: Deploying daemon osd.9 on smithi001 2024-09-11T21:53:26.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:26 smithi001 ceph-mon[30935]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-11T21:53:26.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:26 smithi053 ceph-mon[34908]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-11T21:53:26.841 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:26 smithi161 ceph-mon[35615]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-11T21:53:28.634 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:28 smithi001 ceph-mon[30935]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-11T21:53:28.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:28 smithi053 ceph-mon[34908]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-11T21:53:28.841 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:28 smithi161 ceph-mon[35615]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-11T21:53:29.621 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:29 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:53:29.621 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:29 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:29.621 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:29 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:29.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:29 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:53:29.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:29 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:29.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:29 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:29.841 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:29 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:53:29.841 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:29 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:29.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:29 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:30.663 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:30 smithi001 ceph-mon[30935]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-11T21:53:30.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:30 smithi053 ceph-mon[34908]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-11T21:53:30.841 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:30 smithi161 ceph-mon[35615]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-11T21:53:32.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:32 smithi001 ceph-mon[30935]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-11T21:53:32.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:32 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:32.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:32 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:32.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:32 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:53:32.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:32 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:53:32.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:32 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:32.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:53:32.594+0000 7fda967fc640 1 -- 172.21.15.1:0/1558331154 <== mgr.14150 v2:172.21.15.1:6800/3890350886 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fdaa0062630 con 0x7fda8c0a2bb0 2024-09-11T21:53:32.595 INFO:teuthology.orchestra.run.smithi001.stdout:Created osd(s) 9 on host 'smithi001' 2024-09-11T21:53:32.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:53:32.596+0000 7fdaa7684640 1 -- 172.21.15.1:0/1558331154 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fda8c0a2bb0 msgr2=0x7fda8c0a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:53:32.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:53:32.596+0000 7fdaa7684640 1 --2- 172.21.15.1:0/1558331154 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fda8c0a2bb0 0x7fda8c0a5070 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fda9c006010 tx=0x7fda9c032dd0 comp rx=0 tx=0).stop 2024-09-11T21:53:32.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:53:32.596+0000 7fdaa7684640 1 -- 172.21.15.1:0/1558331154 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdaa0102e10 msgr2=0x7fdaa0198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:53:32.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:53:32.596+0000 7fdaa7684640 1 --2- 172.21.15.1:0/1558331154 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdaa0102e10 0x7fdaa0198430 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fda9000d950 tx=0x7fda9000de10 comp rx=0 tx=0).stop 2024-09-11T21:53:32.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:53:32.596+0000 7fdaa7684640 1 -- 172.21.15.1:0/1558331154 shutdown_connections 2024-09-11T21:53:32.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:53:32.596+0000 7fdaa7684640 1 --2- 172.21.15.1:0/1558331154 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fda8c0a2bb0 0x7fda8c0a5070 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:53:32.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:53:32.596+0000 7fdaa7684640 1 --2- 172.21.15.1:0/1558331154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdaa01037d0 0x7fdaa019f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:53:32.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:53:32.596+0000 7fdaa7684640 1 --2- 172.21.15.1:0/1558331154 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdaa0102e10 0x7fdaa0198430 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:53:32.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:53:32.596+0000 7fdaa7684640 1 --2- 172.21.15.1:0/1558331154 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdaa0101c10 0x7fdaa0197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:53:32.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:53:32.596+0000 7fdaa7684640 1 -- 172.21.15.1:0/1558331154 >> 172.21.15.1:0/1558331154 conn(0x7fdaa00fd3c0 msgr2=0x7fdaa0106230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:53:32.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:53:32.596+0000 7fdaa7684640 1 -- 172.21.15.1:0/1558331154 shutdown_connections 2024-09-11T21:53:32.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:53:32.596+0000 7fdaa7684640 1 -- 172.21.15.1:0/1558331154 wait complete. 2024-09-11T21:53:32.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:32 smithi053 ceph-mon[34908]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-11T21:53:32.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:32 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:32.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:32 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:32.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:32 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:53:32.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:32 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:53:32.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:32 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:32.841 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:32 smithi161 ceph-mon[35615]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-11T21:53:32.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:32 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:32.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:32 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:32.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:32 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:53:32.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:32 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:53:32.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:32 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:33.407 DEBUG:teuthology.orchestra.run.smithi001:osd.9> sudo journalctl -f -n 0 -u ceph-3ffca83c-7087-11ef-bcea-c7b262605968@osd.9.service 2024-09-11T21:53:33.410 INFO:tasks.cephadm:Deploying osd.10 on smithi053 with /dev/vg_nvme/lv_1... 2024-09-11T21:53:33.410 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-11T21:53:33.662 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:33 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:53:33.663 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:33 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:33.663 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:33 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:33.668 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T21:53:33.789 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:33 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:53:33.789 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:33 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:33.789 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:33 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:33.841 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:33 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:53:33.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:33 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:33.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:33 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:34.663 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:34 smithi001 ceph-mon[30935]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-11T21:53:34.663 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 21:53:34 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T21:53:34.441+0000 7fac65d02640 -1 osd.9 0 log_to_monitors true 2024-09-11T21:53:34.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:34 smithi053 ceph-mon[34908]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-11T21:53:34.841 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:34 smithi161 ceph-mon[35615]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-11T21:53:35.687 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:35 smithi053 ceph-mon[34908]: from='osd.9 [v2:172.21.15.1:6826/2806274589,v1:172.21.15.1:6827/2806274589]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-11T21:53:35.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:35 smithi001 ceph-mon[30935]: from='osd.9 [v2:172.21.15.1:6826/2806274589,v1:172.21.15.1:6827/2806274589]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-11T21:53:35.841 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:35 smithi161 ceph-mon[35615]: from='osd.9 [v2:172.21.15.1:6826/2806274589,v1:172.21.15.1:6827/2806274589]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-11T21:53:36.289 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-11T21:53:36.318 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi053:vg_nvme/lv_1 2024-09-11T21:53:36.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:36 smithi053 ceph-mon[34908]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-11T21:53:36.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:36 smithi053 ceph-mon[34908]: from='osd.9 [v2:172.21.15.1:6826/2806274589,v1:172.21.15.1:6827/2806274589]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-11T21:53:36.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:36 smithi053 ceph-mon[34908]: osdmap e64: 10 total, 9 up, 10 in 2024-09-11T21:53:36.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:36 smithi053 ceph-mon[34908]: from='osd.9 [v2:172.21.15.1:6826/2806274589,v1:172.21.15.1:6827/2806274589]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-09-11T21:53:36.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:36 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T21:53:36.587 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T21:53:36.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:36 smithi001 ceph-mon[30935]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-11T21:53:36.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:36 smithi001 ceph-mon[30935]: from='osd.9 [v2:172.21.15.1:6826/2806274589,v1:172.21.15.1:6827/2806274589]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-11T21:53:36.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:36 smithi001 ceph-mon[30935]: osdmap e64: 10 total, 9 up, 10 in 2024-09-11T21:53:36.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:36 smithi001 ceph-mon[30935]: from='osd.9 [v2:172.21.15.1:6826/2806274589,v1:172.21.15.1:6827/2806274589]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-09-11T21:53:36.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:36 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T21:53:36.743 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:36 smithi161 ceph-mon[35615]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-11T21:53:36.743 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:36 smithi161 ceph-mon[35615]: from='osd.9 [v2:172.21.15.1:6826/2806274589,v1:172.21.15.1:6827/2806274589]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-11T21:53:36.743 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:36 smithi161 ceph-mon[35615]: osdmap e64: 10 total, 9 up, 10 in 2024-09-11T21:53:36.743 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:36 smithi161 ceph-mon[35615]: from='osd.9 [v2:172.21.15.1:6826/2806274589,v1:172.21.15.1:6827/2806274589]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-09-11T21:53:36.743 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:36 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T21:53:37.123 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:53:37.091+0000 7f050b8f0640 1 -- 172.21.15.53:0/4130066297 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0504104f90 msgr2=0x7f0504107380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:53:37.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:53:37.091+0000 7f050b8f0640 1 --2- 172.21.15.53:0/4130066297 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0504104f90 0x7f0504107380 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f04f4009900 tx=0x7f04f402a410 comp rx=0 tx=0).stop 2024-09-11T21:53:37.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:53:37.091+0000 7f050b8f0640 1 -- 172.21.15.53:0/4130066297 shutdown_connections 2024-09-11T21:53:37.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:53:37.091+0000 7f050b8f0640 1 --2- 172.21.15.53:0/4130066297 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05041078c0 0x7f0504109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:53:37.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:53:37.091+0000 7f050b8f0640 1 --2- 172.21.15.53:0/4130066297 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0504104f90 0x7f0504107380 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:53:37.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:53:37.091+0000 7f050b8f0640 1 --2- 172.21.15.53:0/4130066297 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0504102660 0x7f0504104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:53:37.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:53:37.091+0000 7f050b8f0640 1 -- 172.21.15.53:0/4130066297 >> 172.21.15.53:0/4130066297 conn(0x7f05040fc280 msgr2=0x7f05040fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:53:37.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:53:37.091+0000 7f050b8f0640 1 -- 172.21.15.53:0/4130066297 shutdown_connections 2024-09-11T21:53:37.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:53:37.091+0000 7f050b8f0640 1 -- 172.21.15.53:0/4130066297 wait complete. 2024-09-11T21:53:37.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:53:37.092+0000 7f050b8f0640 1 Processor -- start 2024-09-11T21:53:37.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:53:37.092+0000 7f050b8f0640 1 -- start start 2024-09-11T21:53:37.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:53:37.092+0000 7f050b8f0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0504102660 0x7f0504101fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:53:37.124 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:53:37.092+0000 7f050b8f0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0504104f90 0x7f0504100610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:53:37.125 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:53:37.092+0000 7f050b8f0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05041078c0 0x7f0504100b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:53:37.125 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:53:37.092+0000 7f050b8f0640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f050410d8b0 con 0x7f0504104f90 2024-09-11T21:53:37.125 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:53:37.092+0000 7f050b8f0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f050410d730 con 0x7f0504102660 2024-09-11T21:53:37.125 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:53:37.092+0000 7f050b8f0640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f050410da30 con 0x7f05041078c0 2024-09-11T21:53:37.125 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:53:37.092+0000 7f050968c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0504102660 0x7f0504101fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:53:37.125 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:53:37.092+0000 7f050968c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0504102660 0x7f0504101fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58960/0 (socket says 172.21.15.53:58960) 2024-09-11T21:53:37.125 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:53:37.092+0000 7f050968c640 1 -- 172.21.15.53:0/2070751764 learned_addr learned my addr 172.21.15.53:0/2070751764 (peer_addr_for_me v2:172.21.15.53:0/0) 2024-09-11T21:53:37.125 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:53:37.092+0000 7f0509e8d640 1 --2- 172.21.15.53:0/2070751764 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05041078c0 0x7f0504100b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:53:37.125 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:53:37.092+0000 7f0508e8b640 1 --2- 172.21.15.53:0/2070751764 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0504104f90 0x7f0504100610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:53:37.125 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:53:37.093+0000 7f050968c640 1 -- 172.21.15.53:0/2070751764 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05041078c0 msgr2=0x7f0504100b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:53:37.126 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:53:37.093+0000 7f050968c640 1 --2- 172.21.15.53:0/2070751764 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05041078c0 0x7f0504100b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:53:37.126 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:53:37.093+0000 7f050968c640 1 -- 172.21.15.53:0/2070751764 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0504104f90 msgr2=0x7f0504100610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:53:37.126 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:53:37.093+0000 7f050968c640 1 --2- 172.21.15.53:0/2070751764 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0504104f90 0x7f0504100610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:53:37.126 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:53:37.093+0000 7f050968c640 1 -- 172.21.15.53:0/2070751764 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0504101410 con 0x7f0504102660 2024-09-11T21:53:37.126 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:53:37.093+0000 7f050968c640 1 --2- 172.21.15.53:0/2070751764 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0504102660 0x7f0504101fc0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f050000c920 tx=0x7f050000cde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:53:37.126 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:53:37.093+0000 7f04fa7fc640 1 -- 172.21.15.53:0/2070751764 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0500007c70 con 0x7f0504102660 2024-09-11T21:53:37.126 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:53:37.093+0000 7f050b8f0640 1 -- 172.21.15.53:0/2070751764 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05041a20a0 con 0x7f0504102660 2024-09-11T21:53:37.126 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:53:37.093+0000 7f04fa7fc640 1 -- 172.21.15.53:0/2070751764 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0500007e10 con 0x7f0504102660 2024-09-11T21:53:37.126 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:53:37.093+0000 7f04fa7fc640 1 -- 172.21.15.53:0/2070751764 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0500018890 con 0x7f0504102660 2024-09-11T21:53:37.126 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:53:37.093+0000 7f050b8f0640 1 -- 172.21.15.53:0/2070751764 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05041a2630 con 0x7f0504102660 2024-09-11T21:53:37.126 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:53:37.094+0000 7f050b8f0640 1 -- 172.21.15.53:0/2070751764 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05040fdd70 con 0x7f0504102660 2024-09-11T21:53:37.127 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:53:37.095+0000 7f04fa7fc640 1 -- 172.21.15.53:0/2070751764 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f0500018a50 con 0x7f0504102660 2024-09-11T21:53:37.127 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:53:37.096+0000 7f04fa7fc640 1 --2- 172.21.15.53:0/2070751764 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f04f00a2b80 0x7f04f00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:53:37.127 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:53:37.096+0000 7f04fa7fc640 1 -- 172.21.15.53:0/2070751764 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5981+0+0 (secure 0 0 0) 0x7f05000ceaf0 con 0x7f0504102660 2024-09-11T21:53:37.127 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:53:37.096+0000 7f0508e8b640 1 --2- 172.21.15.53:0/2070751764 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f04f00a2b80 0x7f04f00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:53:37.127 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:53:37.097+0000 7f0508e8b640 1 --2- 172.21.15.53:0/2070751764 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f04f00a2b80 0x7f04f00a5040 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f04f4005ab0 tx=0x7f04f4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:53:37.127 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:53:37.099+0000 7f04fa7fc640 1 -- 172.21.15.53:0/2070751764 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0500014030 con 0x7f0504102660 2024-09-11T21:53:37.193 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 21:53:36 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T21:53:36.873+0000 7fac62499640 -1 osd.9 0 waiting for initial osdmap 2024-09-11T21:53:37.193 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 21:53:36 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T21:53:36.888+0000 7fac59aa5640 -1 osd.9 65 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T21:53:37.291 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:53:37.290+0000 7f050b8f0640 1 -- 172.21.15.53:0/2070751764 --> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f0504062630 con 0x7f04f00a2b80 2024-09-11T21:53:37.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:37 smithi001 ceph-mon[30935]: from='osd.9 [v2:172.21.15.1:6826/2806274589,v1:172.21.15.1:6827/2806274589]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-09-11T21:53:37.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:37 smithi001 ceph-mon[30935]: osdmap e65: 10 total, 9 up, 10 in 2024-09-11T21:53:37.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:37 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T21:53:37.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:37 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T21:53:37.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:37 smithi001 ceph-mon[30935]: Detected new or changed devices on smithi001 2024-09-11T21:53:37.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:37 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:37.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:37 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:37.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:37 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:53:37.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:37 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:53:37.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:37 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:53:37.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:37 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:53:37.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:37 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:37.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:37 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:53:37.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:37 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:53:37.695 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:37 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:37.695 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:37 smithi001 ceph-mon[30935]: from='osd.9 [v2:172.21.15.1:6826/2806274589,v1:172.21.15.1:6827/2806274589]' entity='osd.9' 2024-09-11T21:53:37.695 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:37 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T21:53:37.695 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:37 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T21:53:37.695 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:37 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:53:37.705 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:37 smithi053 ceph-mon[34908]: from='osd.9 [v2:172.21.15.1:6826/2806274589,v1:172.21.15.1:6827/2806274589]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-09-11T21:53:37.706 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:37 smithi053 ceph-mon[34908]: osdmap e65: 10 total, 9 up, 10 in 2024-09-11T21:53:37.706 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:37 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T21:53:37.706 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:37 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T21:53:37.706 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:37 smithi053 ceph-mon[34908]: Detected new or changed devices on smithi001 2024-09-11T21:53:37.706 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:37 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:37.706 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:37 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:37.706 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:37 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:53:37.706 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:37 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:53:37.706 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:37 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:53:37.706 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:37 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:53:37.706 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:37 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:37.707 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:37 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:53:37.707 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:37 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:53:37.707 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:37 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:37.707 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:37 smithi053 ceph-mon[34908]: from='osd.9 [v2:172.21.15.1:6826/2806274589,v1:172.21.15.1:6827/2806274589]' entity='osd.9' 2024-09-11T21:53:37.707 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:37 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T21:53:37.707 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:37 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T21:53:37.707 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:37 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:53:37.764 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:37 smithi161 ceph-mon[35615]: from='osd.9 [v2:172.21.15.1:6826/2806274589,v1:172.21.15.1:6827/2806274589]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-09-11T21:53:37.764 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:37 smithi161 ceph-mon[35615]: osdmap e65: 10 total, 9 up, 10 in 2024-09-11T21:53:37.764 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:37 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T21:53:37.764 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:37 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T21:53:37.764 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:37 smithi161 ceph-mon[35615]: Detected new or changed devices on smithi001 2024-09-11T21:53:37.764 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:37 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:37.764 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:37 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:37.765 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:37 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:53:37.765 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:37 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:53:37.765 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:37 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:53:37.765 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:37 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:53:37.765 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:37 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:37.765 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:37 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:53:37.765 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:37 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:53:37.765 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:37 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:37.765 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:37 smithi161 ceph-mon[35615]: from='osd.9 [v2:172.21.15.1:6826/2806274589,v1:172.21.15.1:6827/2806274589]' entity='osd.9' 2024-09-11T21:53:37.765 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:37 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T21:53:37.765 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:37 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T21:53:37.765 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:37 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:53:38.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:38 smithi001 ceph-mon[30935]: purged_snaps scrub starts 2024-09-11T21:53:38.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:38 smithi001 ceph-mon[30935]: purged_snaps scrub ok 2024-09-11T21:53:38.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:38 smithi001 ceph-mon[30935]: Adjusting osd_memory_target on smithi001 to 4273M 2024-09-11T21:53:38.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:38 smithi001 ceph-mon[30935]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-11T21:53:38.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:38 smithi001 ceph-mon[30935]: from='client.24335 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:53:38.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:38 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T21:53:38.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:38 smithi001 ceph-mon[30935]: osd.9 [v2:172.21.15.1:6826/2806274589,v1:172.21.15.1:6827/2806274589] boot 2024-09-11T21:53:38.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:38 smithi001 ceph-mon[30935]: osdmap e66: 10 total, 10 up, 10 in 2024-09-11T21:53:38.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:38 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T21:53:38.731 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:38 smithi053 ceph-mon[34908]: purged_snaps scrub starts 2024-09-11T21:53:38.731 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:38 smithi053 ceph-mon[34908]: purged_snaps scrub ok 2024-09-11T21:53:38.731 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:38 smithi053 ceph-mon[34908]: Adjusting osd_memory_target on smithi001 to 4273M 2024-09-11T21:53:38.731 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:38 smithi053 ceph-mon[34908]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-11T21:53:38.731 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:38 smithi053 ceph-mon[34908]: from='client.24335 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:53:38.731 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:38 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T21:53:38.731 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:38 smithi053 ceph-mon[34908]: osd.9 [v2:172.21.15.1:6826/2806274589,v1:172.21.15.1:6827/2806274589] boot 2024-09-11T21:53:38.731 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:38 smithi053 ceph-mon[34908]: osdmap e66: 10 total, 10 up, 10 in 2024-09-11T21:53:38.731 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:38 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T21:53:38.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:38 smithi161 ceph-mon[35615]: purged_snaps scrub starts 2024-09-11T21:53:38.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:38 smithi161 ceph-mon[35615]: purged_snaps scrub ok 2024-09-11T21:53:38.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:38 smithi161 ceph-mon[35615]: Adjusting osd_memory_target on smithi001 to 4273M 2024-09-11T21:53:38.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:38 smithi161 ceph-mon[35615]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-11T21:53:38.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:38 smithi161 ceph-mon[35615]: from='client.24335 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:53:38.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:38 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T21:53:38.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:38 smithi161 ceph-mon[35615]: osd.9 [v2:172.21.15.1:6826/2806274589,v1:172.21.15.1:6827/2806274589] boot 2024-09-11T21:53:38.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:38 smithi161 ceph-mon[35615]: osdmap e66: 10 total, 10 up, 10 in 2024-09-11T21:53:38.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:38 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T21:53:40.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:39 smithi001 ceph-mon[30935]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-11T21:53:40.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:39 smithi001 ceph-mon[30935]: osdmap e67: 10 total, 10 up, 10 in 2024-09-11T21:53:40.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:39 smithi001 ceph-mon[30935]: from='client.? 172.21.15.53:0/2370113431' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70549796-5270-4ae2-ac3f-f61f5de105d4"}]: dispatch 2024-09-11T21:53:40.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:39 smithi001 ceph-mon[30935]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70549796-5270-4ae2-ac3f-f61f5de105d4"}]: dispatch 2024-09-11T21:53:40.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:39 smithi001 ceph-mon[30935]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "70549796-5270-4ae2-ac3f-f61f5de105d4"}]': finished 2024-09-11T21:53:40.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:39 smithi001 ceph-mon[30935]: osdmap e68: 11 total, 10 up, 11 in 2024-09-11T21:53:40.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:39 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T21:53:40.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:39 smithi053 ceph-mon[34908]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-11T21:53:40.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:39 smithi053 ceph-mon[34908]: osdmap e67: 10 total, 10 up, 10 in 2024-09-11T21:53:40.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:39 smithi053 ceph-mon[34908]: from='client.? 172.21.15.53:0/2370113431' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70549796-5270-4ae2-ac3f-f61f5de105d4"}]: dispatch 2024-09-11T21:53:40.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:39 smithi053 ceph-mon[34908]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70549796-5270-4ae2-ac3f-f61f5de105d4"}]: dispatch 2024-09-11T21:53:40.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:39 smithi053 ceph-mon[34908]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "70549796-5270-4ae2-ac3f-f61f5de105d4"}]': finished 2024-09-11T21:53:40.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:39 smithi053 ceph-mon[34908]: osdmap e68: 11 total, 10 up, 11 in 2024-09-11T21:53:40.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:39 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T21:53:40.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:39 smithi161 ceph-mon[35615]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-11T21:53:40.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:39 smithi161 ceph-mon[35615]: osdmap e67: 10 total, 10 up, 10 in 2024-09-11T21:53:40.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:39 smithi161 ceph-mon[35615]: from='client.? 172.21.15.53:0/2370113431' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70549796-5270-4ae2-ac3f-f61f5de105d4"}]: dispatch 2024-09-11T21:53:40.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:39 smithi161 ceph-mon[35615]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70549796-5270-4ae2-ac3f-f61f5de105d4"}]: dispatch 2024-09-11T21:53:40.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:39 smithi161 ceph-mon[35615]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "70549796-5270-4ae2-ac3f-f61f5de105d4"}]': finished 2024-09-11T21:53:40.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:39 smithi161 ceph-mon[35615]: osdmap e68: 11 total, 10 up, 11 in 2024-09-11T21:53:40.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:39 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T21:53:41.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:40 smithi001 ceph-mon[30935]: from='client.? 172.21.15.53:0/83413817' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T21:53:41.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:40 smithi053 ceph-mon[34908]: from='client.? 172.21.15.53:0/83413817' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T21:53:41.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:40 smithi161 ceph-mon[35615]: from='client.? 172.21.15.53:0/83413817' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T21:53:42.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:41 smithi001 ceph-mon[30935]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-11T21:53:42.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:41 smithi053 ceph-mon[34908]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-11T21:53:42.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:41 smithi161 ceph-mon[35615]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-11T21:53:44.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:43 smithi001 ceph-mon[30935]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-11T21:53:44.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:43 smithi053 ceph-mon[34908]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-11T21:53:44.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:43 smithi161 ceph-mon[35615]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-11T21:53:46.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:45 smithi001 ceph-mon[30935]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-11T21:53:46.293 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:45 smithi053 ceph-mon[34908]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-11T21:53:46.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:45 smithi161 ceph-mon[35615]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-11T21:53:48.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:47 smithi053 ceph-mon[34908]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-11T21:53:48.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:47 smithi001 ceph-mon[30935]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-11T21:53:48.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:47 smithi161 ceph-mon[35615]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-11T21:53:49.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:48 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-11T21:53:49.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:48 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:53:49.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:48 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-11T21:53:49.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:48 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:53:49.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:48 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-11T21:53:49.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:48 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:53:50.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:49 smithi001 ceph-mon[30935]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-11T21:53:50.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:49 smithi001 ceph-mon[30935]: Deploying daemon osd.10 on smithi053 2024-09-11T21:53:50.232 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:49 smithi053 ceph-mon[34908]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-11T21:53:50.232 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:49 smithi053 ceph-mon[34908]: Deploying daemon osd.10 on smithi053 2024-09-11T21:53:50.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:49 smithi161 ceph-mon[35615]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-11T21:53:50.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:49 smithi161 ceph-mon[35615]: Deploying daemon osd.10 on smithi053 2024-09-11T21:53:52.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:51 smithi001 ceph-mon[30935]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-11T21:53:52.258 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:51 smithi053 ceph-mon[34908]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-11T21:53:52.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:51 smithi161 ceph-mon[35615]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-11T21:53:54.048 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:53 smithi053 ceph-mon[34908]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-11T21:53:54.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:53 smithi001 ceph-mon[30935]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-11T21:53:54.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:53 smithi161 ceph-mon[35615]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-11T21:53:55.187 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:54 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:53:55.188 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:54 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:55.188 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:54 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:55.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:54 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:53:55.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:54 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:55.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:54 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:55.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:54 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:53:55.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:54 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:55.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:54 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:56.188 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:55 smithi053 ceph-mon[34908]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-11T21:53:56.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:55 smithi001 ceph-mon[30935]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-11T21:53:56.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:55 smithi161 ceph-mon[35615]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-11T21:53:58.323 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:53:58.322+0000 7f04fa7fc640 1 -- 172.21.15.53:0/2070751764 <== mgr.14150 v2:172.21.15.1:6800/3890350886 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f0504062630 con 0x7f04f00a2b80 2024-09-11T21:53:58.323 INFO:teuthology.orchestra.run.smithi053.stdout:Created osd(s) 10 on host 'smithi053' 2024-09-11T21:53:58.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:53:58.325+0000 7f050b8f0640 1 -- 172.21.15.53:0/2070751764 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f04f00a2b80 msgr2=0x7f04f00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:53:58.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:53:58.325+0000 7f050b8f0640 1 --2- 172.21.15.53:0/2070751764 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f04f00a2b80 0x7f04f00a5040 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f04f4005ab0 tx=0x7f04f4000bc0 comp rx=0 tx=0).stop 2024-09-11T21:53:58.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:53:58.325+0000 7f050b8f0640 1 -- 172.21.15.53:0/2070751764 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0504102660 msgr2=0x7f0504101fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:53:58.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:53:58.325+0000 7f050b8f0640 1 --2- 172.21.15.53:0/2070751764 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0504102660 0x7f0504101fc0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f050000c920 tx=0x7f050000cde0 comp rx=0 tx=0).stop 2024-09-11T21:53:58.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:53:58.325+0000 7f050b8f0640 1 -- 172.21.15.53:0/2070751764 shutdown_connections 2024-09-11T21:53:58.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:53:58.325+0000 7f050b8f0640 1 --2- 172.21.15.53:0/2070751764 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f04f00a2b80 0x7f04f00a5040 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:53:58.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:53:58.325+0000 7f050b8f0640 1 --2- 172.21.15.53:0/2070751764 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05041078c0 0x7f0504100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:53:58.326 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:53:58.325+0000 7f050b8f0640 1 --2- 172.21.15.53:0/2070751764 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0504104f90 0x7f0504100610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:53:58.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:53:58.325+0000 7f050b8f0640 1 --2- 172.21.15.53:0/2070751764 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0504102660 0x7f0504101fc0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:53:58.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:53:58.325+0000 7f050b8f0640 1 -- 172.21.15.53:0/2070751764 >> 172.21.15.53:0/2070751764 conn(0x7f05040fc280 msgr2=0x7f0504102f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:53:58.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:53:58.325+0000 7f050b8f0640 1 -- 172.21.15.53:0/2070751764 shutdown_connections 2024-09-11T21:53:58.327 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-11T21:53:58.325+0000 7f050b8f0640 1 -- 172.21.15.53:0/2070751764 wait complete. 2024-09-11T21:53:58.703 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:58 smithi053 ceph-mon[34908]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-11T21:53:58.703 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:58 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:58.703 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:58 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:58.703 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:58 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:53:58.703 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:58 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:53:58.704 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:58 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:58.704 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:58 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:53:58.704 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:58 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:58.704 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:53:58 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:58.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:58 smithi161 ceph-mon[35615]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-11T21:53:58.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:58 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:58.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:58 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:58.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:58 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:53:58.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:58 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:53:58.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:58 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:58.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:58 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:53:58.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:58 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:58.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:53:58 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:58.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:58 smithi001 ceph-mon[30935]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-11T21:53:58.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:58 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:58.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:58 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:58.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:58 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:53:58.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:58 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:53:58.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:58 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:58.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:58 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:53:58.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:58 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:58.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:53:58 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:53:59.058 DEBUG:teuthology.orchestra.run.smithi053:osd.10> sudo journalctl -f -n 0 -u ceph-3ffca83c-7087-11ef-bcea-c7b262605968@osd.10.service 2024-09-11T21:53:59.061 INFO:tasks.cephadm:Deploying osd.11 on smithi161 with /dev/vg_nvme/lv_1... 2024-09-11T21:53:59.061 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-11T21:53:59.318 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T21:54:00.743 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:00 smithi161 ceph-mon[35615]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-11T21:54:00.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:00 smithi053 ceph-mon[34908]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-11T21:54:00.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 21:54:00 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T21:54:00.521+0000 7f5cf554a640 -1 osd.10 0 log_to_monitors true 2024-09-11T21:54:00.869 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:00 smithi001 ceph-mon[30935]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-11T21:54:01.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:01 smithi053 ceph-mon[34908]: from='osd.10 [v2:172.21.15.53:6824/761923580,v1:172.21.15.53:6825/761923580]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-11T21:54:01.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:01 smithi053 ceph-mon[34908]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-11T21:54:01.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:01 smithi161 ceph-mon[35615]: from='osd.10 [v2:172.21.15.53:6824/761923580,v1:172.21.15.53:6825/761923580]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-11T21:54:01.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:01 smithi161 ceph-mon[35615]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-11T21:54:01.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:01 smithi001 ceph-mon[30935]: from='osd.10 [v2:172.21.15.53:6824/761923580,v1:172.21.15.53:6825/761923580]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-11T21:54:01.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:01 smithi001 ceph-mon[30935]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-11T21:54:02.072 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-09-11T21:54:02.098 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi161:vg_nvme/lv_1 2024-09-11T21:54:02.347 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T21:54:02.740 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:02 smithi161 ceph-mon[35615]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-11T21:54:02.740 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:02 smithi161 ceph-mon[35615]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-11T21:54:02.740 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:02 smithi161 ceph-mon[35615]: osdmap e69: 11 total, 10 up, 11 in 2024-09-11T21:54:02.740 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:02 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T21:54:02.740 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:02 smithi161 ceph-mon[35615]: from='osd.10 [v2:172.21.15.53:6824/761923580,v1:172.21.15.53:6825/761923580]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-09-11T21:54:02.740 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:02 smithi161 ceph-mon[35615]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-09-11T21:54:02.741 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:02 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:02.741 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:02 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:02.741 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:02 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:54:02.741 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:02 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:54:02.741 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:02 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:54:02.741 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:02 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:54:02.741 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:02 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:02.741 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:02 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:54:02.741 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:02 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:54:02.741 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:02 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:02.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:02 smithi053 ceph-mon[34908]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-11T21:54:02.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:02 smithi053 ceph-mon[34908]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-11T21:54:02.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:02 smithi053 ceph-mon[34908]: osdmap e69: 11 total, 10 up, 11 in 2024-09-11T21:54:02.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:02 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T21:54:02.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:02 smithi053 ceph-mon[34908]: from='osd.10 [v2:172.21.15.53:6824/761923580,v1:172.21.15.53:6825/761923580]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-09-11T21:54:02.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:02 smithi053 ceph-mon[34908]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-09-11T21:54:02.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:02 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:02.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:02 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:02.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:02 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:54:02.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:02 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:54:02.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:02 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:54:02.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:02 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:54:02.799 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:02 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:02.799 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:02 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:54:02.799 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:02 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:54:02.799 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:02 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:02.840 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:02.822+0000 7ff4f6af5640 1 -- 172.21.15.161:0/2612135924 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4f01026d0 msgr2=0x7ff4f0108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:54:02.840 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:02.822+0000 7ff4f6af5640 1 --2- 172.21.15.161:0/2612135924 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4f01026d0 0x7ff4f0108f60 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8009ca0 tx=0x7ff4e802a8c0 comp rx=0 tx=0).stop 2024-09-11T21:54:02.840 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:02.822+0000 7ff4f6af5640 1 -- 172.21.15.161:0/2612135924 shutdown_connections 2024-09-11T21:54:02.840 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:02.822+0000 7ff4f6af5640 1 --2- 172.21.15.161:0/2612135924 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4f01026d0 0x7ff4f0108f60 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:02.840 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:02.822+0000 7ff4f6af5640 1 --2- 172.21.15.161:0/2612135924 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4f0101d10 0x7ff4f0102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:02.840 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:02.822+0000 7ff4f6af5640 1 --2- 172.21.15.161:0/2612135924 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff4f0100b10 0x7ff4f0100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:02.840 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:02.822+0000 7ff4f6af5640 1 -- 172.21.15.161:0/2612135924 >> 172.21.15.161:0/2612135924 conn(0x7ff4f00fc2a0 msgr2=0x7ff4f00fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:54:02.840 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:02.822+0000 7ff4f6af5640 1 -- 172.21.15.161:0/2612135924 shutdown_connections 2024-09-11T21:54:02.841 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:02.822+0000 7ff4f6af5640 1 -- 172.21.15.161:0/2612135924 wait complete. 2024-09-11T21:54:02.841 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:02.823+0000 7ff4f6af5640 1 Processor -- start 2024-09-11T21:54:02.841 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:02.823+0000 7ff4f6af5640 1 -- start start 2024-09-11T21:54:02.841 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:02.823+0000 7ff4f6af5640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4f0100b10 0x7ff4f0197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:54:02.841 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:02.823+0000 7ff4f6af5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4f0101d10 0x7ff4f0198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:54:02.841 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:02.823+0000 7ff4f6af5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff4f01026d0 0x7ff4f019f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:54:02.841 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:02.823+0000 7ff4f6af5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f010d880 con 0x7ff4f01026d0 2024-09-11T21:54:02.841 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:02.823+0000 7ff4f6af5640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f010d700 con 0x7ff4f0101d10 2024-09-11T21:54:02.841 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:02.823+0000 7ff4f6af5640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f010da00 con 0x7ff4f0100b10 2024-09-11T21:54:02.841 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:02.823+0000 7ff4f4891640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4f0100b10 0x7ff4f0197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:54:02.842 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:02.823+0000 7ff4f4891640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4f0100b10 0x7ff4f0197de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:59088/0 (socket says 172.21.15.161:59088) 2024-09-11T21:54:02.842 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:02.823+0000 7ff4f4891640 1 -- 172.21.15.161:0/390849707 learned_addr learned my addr 172.21.15.161:0/390849707 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-09-11T21:54:02.842 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:02.823+0000 7ff4effff640 1 --2- 172.21.15.161:0/390849707 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4f0101d10 0x7ff4f0198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:54:02.842 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:02.823+0000 7ff4f5092640 1 --2- 172.21.15.161:0/390849707 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff4f01026d0 0x7ff4f019f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:54:02.842 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:02.823+0000 7ff4f4891640 1 -- 172.21.15.161:0/390849707 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4f0101d10 msgr2=0x7ff4f0198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:54:02.842 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:02.823+0000 7ff4f4891640 1 --2- 172.21.15.161:0/390849707 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4f0101d10 0x7ff4f0198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:02.842 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:02.823+0000 7ff4f4891640 1 -- 172.21.15.161:0/390849707 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff4f01026d0 msgr2=0x7ff4f019f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:54:02.842 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:02.823+0000 7ff4f4891640 1 --2- 172.21.15.161:0/390849707 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff4f01026d0 0x7ff4f019f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:02.842 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:02.823+0000 7ff4f4891640 1 -- 172.21.15.161:0/390849707 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4f019fa10 con 0x7ff4f0100b10 2024-09-11T21:54:02.842 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:02.823+0000 7ff4f5092640 1 --2- 172.21.15.161:0/390849707 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff4f01026d0 0x7ff4f019f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:54:02.843 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:02.824+0000 7ff4f4891640 1 --2- 172.21.15.161:0/390849707 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4f0100b10 0x7ff4f0197de0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff4e400c970 tx=0x7ff4e400ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:54:02.843 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:02.824+0000 7ff4edffb640 1 -- 172.21.15.161:0/390849707 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e4007c10 con 0x7ff4f0100b10 2024-09-11T21:54:02.843 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:02.824+0000 7ff4edffb640 1 -- 172.21.15.161:0/390849707 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4e4007db0 con 0x7ff4f0100b10 2024-09-11T21:54:02.843 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:02.824+0000 7ff4f6af5640 1 -- 172.21.15.161:0/390849707 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4f019fd00 con 0x7ff4f0100b10 2024-09-11T21:54:02.843 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:02.824+0000 7ff4edffb640 1 -- 172.21.15.161:0/390849707 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e400f940 con 0x7ff4f0100b10 2024-09-11T21:54:02.843 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:02.824+0000 7ff4f6af5640 1 -- 172.21.15.161:0/390849707 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4f01a0190 con 0x7ff4f0100b10 2024-09-11T21:54:02.843 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:02.825+0000 7ff4f6af5640 1 -- 172.21.15.161:0/390849707 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4f0107e40 con 0x7ff4f0100b10 2024-09-11T21:54:02.843 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:02.826+0000 7ff4edffb640 1 -- 172.21.15.161:0/390849707 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7ff4e400fae0 con 0x7ff4f0100b10 2024-09-11T21:54:02.843 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:02.826+0000 7ff4edffb640 1 --2- 172.21.15.161:0/390849707 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7ff4e00a2bb0 0x7ff4e00a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:54:02.843 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:02.827+0000 7ff4edffb640 1 -- 172.21.15.161:0/390849707 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6414+0+0 (secure 0 0 0) 0x7ff4e40cec60 con 0x7ff4f0100b10 2024-09-11T21:54:02.843 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:02.827+0000 7ff4effff640 1 --2- 172.21.15.161:0/390849707 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7ff4e00a2bb0 0x7ff4e00a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:54:02.844 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:02.827+0000 7ff4effff640 1 --2- 172.21.15.161:0/390849707 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7ff4e00a2bb0 0x7ff4e00a5070 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0199300 tx=0x7ff4dc00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:54:02.844 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:02.831+0000 7ff4edffb640 1 -- 172.21.15.161:0/390849707 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff4e409c3c0 con 0x7ff4f0100b10 2024-09-11T21:54:02.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:02 smithi001 ceph-mon[30935]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-11T21:54:02.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:02 smithi001 ceph-mon[30935]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-11T21:54:02.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:02 smithi001 ceph-mon[30935]: osdmap e69: 11 total, 10 up, 11 in 2024-09-11T21:54:02.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:02 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T21:54:02.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:02 smithi001 ceph-mon[30935]: from='osd.10 [v2:172.21.15.53:6824/761923580,v1:172.21.15.53:6825/761923580]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-09-11T21:54:02.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:02 smithi001 ceph-mon[30935]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2024-09-11T21:54:02.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:02 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:02.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:02 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:02.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:02 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:54:02.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:02 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:54:02.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:02 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:54:02.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:02 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:54:02.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:02 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:02.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:02 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:54:02.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:02 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:54:02.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:02 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:03.008 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:03.007+0000 7ff4f6af5640 1 -- 172.21.15.161:0/390849707 --> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7ff4f0062630 con 0x7ff4e00a2bb0 2024-09-11T21:54:03.297 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 21:54:02 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T21:54:02.921+0000 7f5cf14cd640 -1 osd.10 0 waiting for initial osdmap 2024-09-11T21:54:03.298 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 21:54:02 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T21:54:02.935+0000 7f5ce92ed640 -1 osd.10 70 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T21:54:03.701 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:03 smithi161 ceph-mon[35615]: purged_snaps scrub starts 2024-09-11T21:54:03.701 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:03 smithi161 ceph-mon[35615]: purged_snaps scrub ok 2024-09-11T21:54:03.701 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:03 smithi161 ceph-mon[35615]: Detected new or changed devices on smithi053 2024-09-11T21:54:03.702 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:03 smithi161 ceph-mon[35615]: Adjusting osd_memory_target on smithi053 to 4273M 2024-09-11T21:54:03.702 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:03 smithi161 ceph-mon[35615]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-09-11T21:54:03.702 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:03 smithi161 ceph-mon[35615]: osdmap e70: 11 total, 10 up, 11 in 2024-09-11T21:54:03.702 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:03 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T21:54:03.702 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:03 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T21:54:03.702 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:03 smithi161 ceph-mon[35615]: from='osd.10 ' entity='osd.10' 2024-09-11T21:54:03.702 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:03 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T21:54:03.702 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:03 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T21:54:03.702 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:03 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:54:03.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:03 smithi053 ceph-mon[34908]: purged_snaps scrub starts 2024-09-11T21:54:03.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:03 smithi053 ceph-mon[34908]: purged_snaps scrub ok 2024-09-11T21:54:03.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:03 smithi053 ceph-mon[34908]: Detected new or changed devices on smithi053 2024-09-11T21:54:03.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:03 smithi053 ceph-mon[34908]: Adjusting osd_memory_target on smithi053 to 4273M 2024-09-11T21:54:03.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:03 smithi053 ceph-mon[34908]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-09-11T21:54:03.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:03 smithi053 ceph-mon[34908]: osdmap e70: 11 total, 10 up, 11 in 2024-09-11T21:54:03.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:03 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T21:54:03.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:03 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T21:54:03.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:03 smithi053 ceph-mon[34908]: from='osd.10 ' entity='osd.10' 2024-09-11T21:54:03.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:03 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T21:54:03.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:03 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T21:54:03.799 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:03 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:54:03.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:03 smithi001 ceph-mon[30935]: purged_snaps scrub starts 2024-09-11T21:54:03.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:03 smithi001 ceph-mon[30935]: purged_snaps scrub ok 2024-09-11T21:54:03.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:03 smithi001 ceph-mon[30935]: Detected new or changed devices on smithi053 2024-09-11T21:54:03.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:03 smithi001 ceph-mon[30935]: Adjusting osd_memory_target on smithi053 to 4273M 2024-09-11T21:54:03.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:03 smithi001 ceph-mon[30935]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2024-09-11T21:54:03.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:03 smithi001 ceph-mon[30935]: osdmap e70: 11 total, 10 up, 11 in 2024-09-11T21:54:03.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:03 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T21:54:03.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:03 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T21:54:03.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:03 smithi001 ceph-mon[30935]: from='osd.10 ' entity='osd.10' 2024-09-11T21:54:03.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:03 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T21:54:03.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:03 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T21:54:03.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:03 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:54:04.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:04 smithi053 ceph-mon[34908]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-11T21:54:04.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:04 smithi053 ceph-mon[34908]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:54:04.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:04 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T21:54:04.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:04 smithi053 ceph-mon[34908]: osd.10 [v2:172.21.15.53:6824/761923580,v1:172.21.15.53:6825/761923580] boot 2024-09-11T21:54:04.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:04 smithi053 ceph-mon[34908]: osdmap e71: 11 total, 11 up, 11 in 2024-09-11T21:54:04.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:04 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T21:54:04.826 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:04 smithi161 ceph-mon[35615]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-11T21:54:04.826 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:04 smithi161 ceph-mon[35615]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:54:04.826 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:04 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T21:54:04.826 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:04 smithi161 ceph-mon[35615]: osd.10 [v2:172.21.15.53:6824/761923580,v1:172.21.15.53:6825/761923580] boot 2024-09-11T21:54:04.826 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:04 smithi161 ceph-mon[35615]: osdmap e71: 11 total, 11 up, 11 in 2024-09-11T21:54:04.827 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:04 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T21:54:04.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:04 smithi001 ceph-mon[30935]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-09-11T21:54:04.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:04 smithi001 ceph-mon[30935]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi161:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:54:04.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:04 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T21:54:04.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:04 smithi001 ceph-mon[30935]: osd.10 [v2:172.21.15.53:6824/761923580,v1:172.21.15.53:6825/761923580] boot 2024-09-11T21:54:04.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:04 smithi001 ceph-mon[30935]: osdmap e71: 11 total, 11 up, 11 in 2024-09-11T21:54:04.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:04 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T21:54:05.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:05 smithi161 ceph-mon[35615]: osdmap e72: 11 total, 11 up, 11 in 2024-09-11T21:54:05.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:05 smithi161 ceph-mon[35615]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-09-11T21:54:05.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:05 smithi161 ceph-mon[35615]: from='client.? 172.21.15.161:0/1379861790' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "48f17c4a-370a-493e-956e-bf6ad3dcc594"}]: dispatch 2024-09-11T21:54:05.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:05 smithi161 ceph-mon[35615]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "48f17c4a-370a-493e-956e-bf6ad3dcc594"}]: dispatch 2024-09-11T21:54:05.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:05 smithi161 ceph-mon[35615]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "48f17c4a-370a-493e-956e-bf6ad3dcc594"}]': finished 2024-09-11T21:54:05.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:05 smithi161 ceph-mon[35615]: osdmap e73: 12 total, 11 up, 12 in 2024-09-11T21:54:05.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:05 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T21:54:05.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:05 smithi001 ceph-mon[30935]: osdmap e72: 11 total, 11 up, 11 in 2024-09-11T21:54:05.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:05 smithi001 ceph-mon[30935]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-09-11T21:54:05.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:05 smithi001 ceph-mon[30935]: from='client.? 172.21.15.161:0/1379861790' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "48f17c4a-370a-493e-956e-bf6ad3dcc594"}]: dispatch 2024-09-11T21:54:05.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:05 smithi001 ceph-mon[30935]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "48f17c4a-370a-493e-956e-bf6ad3dcc594"}]: dispatch 2024-09-11T21:54:05.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:05 smithi001 ceph-mon[30935]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "48f17c4a-370a-493e-956e-bf6ad3dcc594"}]': finished 2024-09-11T21:54:05.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:05 smithi001 ceph-mon[30935]: osdmap e73: 12 total, 11 up, 12 in 2024-09-11T21:54:05.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:05 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T21:54:05.959 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:05 smithi053 ceph-mon[34908]: osdmap e72: 11 total, 11 up, 11 in 2024-09-11T21:54:05.959 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:05 smithi053 ceph-mon[34908]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-09-11T21:54:05.959 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:05 smithi053 ceph-mon[34908]: from='client.? 172.21.15.161:0/1379861790' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "48f17c4a-370a-493e-956e-bf6ad3dcc594"}]: dispatch 2024-09-11T21:54:05.959 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:05 smithi053 ceph-mon[34908]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "48f17c4a-370a-493e-956e-bf6ad3dcc594"}]: dispatch 2024-09-11T21:54:05.960 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:05 smithi053 ceph-mon[34908]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "48f17c4a-370a-493e-956e-bf6ad3dcc594"}]': finished 2024-09-11T21:54:05.960 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:05 smithi053 ceph-mon[34908]: osdmap e73: 12 total, 11 up, 12 in 2024-09-11T21:54:05.960 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:05 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T21:54:06.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:06 smithi161 ceph-mon[35615]: from='client.? 172.21.15.161:0/958377520' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T21:54:06.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:06 smithi161 ceph-mon[35615]: osdmap e74: 12 total, 11 up, 12 in 2024-09-11T21:54:06.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:06 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T21:54:06.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:06 smithi001 ceph-mon[30935]: from='client.? 172.21.15.161:0/958377520' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T21:54:06.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:06 smithi001 ceph-mon[30935]: osdmap e74: 12 total, 11 up, 12 in 2024-09-11T21:54:06.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:06 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T21:54:07.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:06 smithi053 ceph-mon[34908]: from='client.? 172.21.15.161:0/958377520' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T21:54:07.048 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:06 smithi053 ceph-mon[34908]: osdmap e74: 12 total, 11 up, 12 in 2024-09-11T21:54:07.048 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:06 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T21:54:08.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:08 smithi053 ceph-mon[34908]: pgmap v235: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-11T21:54:08.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:08 smithi053 ceph-mon[34908]: osdmap e75: 12 total, 11 up, 12 in 2024-09-11T21:54:08.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:08 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T21:54:08.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:08 smithi161 ceph-mon[35615]: pgmap v235: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-11T21:54:08.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:08 smithi161 ceph-mon[35615]: osdmap e75: 12 total, 11 up, 12 in 2024-09-11T21:54:08.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:08 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T21:54:08.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:08 smithi001 ceph-mon[30935]: pgmap v235: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-11T21:54:08.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:08 smithi001 ceph-mon[30935]: osdmap e75: 12 total, 11 up, 12 in 2024-09-11T21:54:08.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:08 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T21:54:10.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:10 smithi053 ceph-mon[34908]: pgmap v237: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 143 KiB/s, 0 objects/s recovering 2024-09-11T21:54:10.841 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:10 smithi161 ceph-mon[35615]: pgmap v237: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 143 KiB/s, 0 objects/s recovering 2024-09-11T21:54:10.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:10 smithi001 ceph-mon[30935]: pgmap v237: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 143 KiB/s, 0 objects/s recovering 2024-09-11T21:54:11.687 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:11 smithi161 ceph-mon[35615]: pgmap v238: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T21:54:11.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:11 smithi001 ceph-mon[30935]: pgmap v238: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T21:54:12.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:11 smithi053 ceph-mon[34908]: pgmap v238: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T21:54:13.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:13 smithi161 ceph-mon[35615]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-11T21:54:13.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:13 smithi001 ceph-mon[30935]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-11T21:54:14.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:13 smithi053 ceph-mon[34908]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-11T21:54:14.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:14 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-11T21:54:14.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:14 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:54:14.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:14 smithi001 ceph-mon[30935]: Deploying daemon osd.11 on smithi161 2024-09-11T21:54:15.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:14 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-11T21:54:15.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:14 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:54:15.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:14 smithi053 ceph-mon[34908]: Deploying daemon osd.11 on smithi161 2024-09-11T21:54:15.091 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:14 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-11T21:54:15.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:14 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:54:15.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:14 smithi161 ceph-mon[35615]: Deploying daemon osd.11 on smithi161 2024-09-11T21:54:15.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:15 smithi053 ceph-mon[34908]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-09-11T21:54:15.912 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:15 smithi161 ceph-mon[35615]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-09-11T21:54:15.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:15 smithi001 ceph-mon[30935]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-09-11T21:54:17.930 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:17 smithi161 ceph-mon[35615]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-09-11T21:54:17.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:17 smithi001 ceph-mon[30935]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-09-11T21:54:18.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:17 smithi053 ceph-mon[34908]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-09-11T21:54:19.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:19 smithi161 ceph-mon[35615]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-09-11T21:54:19.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:19 smithi001 ceph-mon[30935]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-09-11T21:54:20.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:19 smithi053 ceph-mon[34908]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-09-11T21:54:20.883 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:20 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:54:20.883 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:20 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:20.883 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:20 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:20.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:20 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:54:20.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:20 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:20.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:20 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:21.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:20 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:54:21.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:20 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:21.048 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:20 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:22.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:22 smithi001 ceph-mon[30935]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-09-11T21:54:22.503 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:22 smithi161 ceph-mon[35615]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-09-11T21:54:22.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:22 smithi053 ceph-mon[34908]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-09-11T21:54:23.591 INFO:teuthology.orchestra.run.smithi161.stdout:Created osd(s) 11 on host 'smithi161' 2024-09-11T21:54:23.592 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:23.589+0000 7ff4edffb640 1 -- 172.21.15.161:0/390849707 <== mgr.14150 v2:172.21.15.1:6800/3890350886 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7ff4f0062630 con 0x7ff4e00a2bb0 2024-09-11T21:54:23.594 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:23.592+0000 7ff4f6af5640 1 -- 172.21.15.161:0/390849707 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7ff4e00a2bb0 msgr2=0x7ff4e00a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:54:23.594 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:23.592+0000 7ff4f6af5640 1 --2- 172.21.15.161:0/390849707 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7ff4e00a2bb0 0x7ff4e00a5070 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0199300 tx=0x7ff4dc00b410 comp rx=0 tx=0).stop 2024-09-11T21:54:23.594 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:23.592+0000 7ff4f6af5640 1 -- 172.21.15.161:0/390849707 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4f0100b10 msgr2=0x7ff4f0197de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:54:23.594 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:23.592+0000 7ff4f6af5640 1 --2- 172.21.15.161:0/390849707 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4f0100b10 0x7ff4f0197de0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff4e400c970 tx=0x7ff4e400ce30 comp rx=0 tx=0).stop 2024-09-11T21:54:23.594 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:23.592+0000 7ff4f6af5640 1 -- 172.21.15.161:0/390849707 shutdown_connections 2024-09-11T21:54:23.595 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:23.592+0000 7ff4f6af5640 1 --2- 172.21.15.161:0/390849707 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7ff4e00a2bb0 0x7ff4e00a5070 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:23.595 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:23.592+0000 7ff4f6af5640 1 --2- 172.21.15.161:0/390849707 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff4f01026d0 0x7ff4f019f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:23.595 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:23.592+0000 7ff4f6af5640 1 --2- 172.21.15.161:0/390849707 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff4f0101d10 0x7ff4f0198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:23.595 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:23.592+0000 7ff4f6af5640 1 --2- 172.21.15.161:0/390849707 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4f0100b10 0x7ff4f0197de0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:23.595 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:23.592+0000 7ff4f6af5640 1 -- 172.21.15.161:0/390849707 >> 172.21.15.161:0/390849707 conn(0x7ff4f00fc2a0 msgr2=0x7ff4f0105010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:54:23.596 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:23.592+0000 7ff4f6af5640 1 -- 172.21.15.161:0/390849707 shutdown_connections 2024-09-11T21:54:23.596 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:23.592+0000 7ff4f6af5640 1 -- 172.21.15.161:0/390849707 wait complete. 2024-09-11T21:54:24.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:23 smithi001 ceph-mon[30935]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-09-11T21:54:24.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:23 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:24.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:23 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:24.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:23 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:54:24.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:23 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:54:24.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:23 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:24.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:23 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:54:24.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:23 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:24.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:23 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:24.204 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:23 smithi161 ceph-mon[35615]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-09-11T21:54:24.204 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:23 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:24.204 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:23 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:24.205 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:23 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:54:24.205 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:23 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:54:24.205 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:23 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:24.205 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:23 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:54:24.205 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:23 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:24.205 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:23 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:24.234 DEBUG:teuthology.orchestra.run.smithi161:osd.11> sudo journalctl -f -n 0 -u ceph-3ffca83c-7087-11ef-bcea-c7b262605968@osd.11.service 2024-09-11T21:54:24.237 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-11T21:54:24.237 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-11T21:54:24.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:23 smithi053 ceph-mon[34908]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-09-11T21:54:24.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:23 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:24.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:23 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:24.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:23 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:54:24.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:23 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:54:24.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:23 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:24.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:23 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:54:24.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:23 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:24.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:23 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:24.505 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:54:25.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:25.033+0000 7f8a0940a640 1 -- 172.21.15.1:0/2362346077 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a04101c00 msgr2=0x7f8a04102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:54:25.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:25.033+0000 7f8a0940a640 1 --2- 172.21.15.1:0/2362346077 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a04101c00 0x7f8a04102000 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f89ec009900 tx=0x7f89ec02a410 comp rx=0 tx=0).stop 2024-09-11T21:54:25.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:25.033+0000 7f8a0940a640 1 -- 172.21.15.1:0/2362346077 shutdown_connections 2024-09-11T21:54:25.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:25.033+0000 7f8a0940a640 1 --2- 172.21.15.1:0/2362346077 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a041037c0 0x7f8a0410a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:25.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:25.033+0000 7f8a0940a640 1 --2- 172.21.15.1:0/2362346077 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a04102e00 0x7f8a04103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:25.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:25.033+0000 7f8a0940a640 1 --2- 172.21.15.1:0/2362346077 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a04101c00 0x7f8a04102000 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:25.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:25.033+0000 7f8a0940a640 1 -- 172.21.15.1:0/2362346077 >> 172.21.15.1:0/2362346077 conn(0x7f8a040fd390 msgr2=0x7f8a040ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:54:25.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:25.034+0000 7f8a0940a640 1 -- 172.21.15.1:0/2362346077 shutdown_connections 2024-09-11T21:54:25.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:25.034+0000 7f8a0940a640 1 -- 172.21.15.1:0/2362346077 wait complete. 2024-09-11T21:54:25.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:25.034+0000 7f8a0940a640 1 Processor -- start 2024-09-11T21:54:25.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:25.034+0000 7f8a0940a640 1 -- start start 2024-09-11T21:54:25.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:25.034+0000 7f8a0940a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a04101c00 0x7f8a04075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:54:25.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:25.034+0000 7f8a0940a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a04102e00 0x7f8a040745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:54:25.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:25.034+0000 7f8a0940a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a041037c0 0x7f8a04074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:54:25.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:25.034+0000 7f8a0940a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0410d8e0 con 0x7f8a041037c0 2024-09-11T21:54:25.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:25.034+0000 7f8a0940a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0410d760 con 0x7f8a04101c00 2024-09-11T21:54:25.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:25.034+0000 7f8a0940a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0410da60 con 0x7f8a04102e00 2024-09-11T21:54:25.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:25.035+0000 7f8a037fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a041037c0 0x7f8a04074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:54:25.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:25.035+0000 7f8a027fc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a04102e00 0x7f8a040745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:54:25.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:25.035+0000 7f8a037fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a041037c0 0x7f8a04074b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41700/0 (socket says 172.21.15.1:41700) 2024-09-11T21:54:25.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:25.035+0000 7f8a037fe640 1 -- 172.21.15.1:0/2712823819 learned_addr learned my addr 172.21.15.1:0/2712823819 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:54:25.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:25.035+0000 7f8a02ffd640 1 --2- 172.21.15.1:0/2712823819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a04101c00 0x7f8a04075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:54:25.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:25.035+0000 7f8a037fe640 1 -- 172.21.15.1:0/2712823819 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a04102e00 msgr2=0x7f8a040745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:54:25.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:25.035+0000 7f8a037fe640 1 --2- 172.21.15.1:0/2712823819 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a04102e00 0x7f8a040745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:25.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:25.035+0000 7f8a037fe640 1 -- 172.21.15.1:0/2712823819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a04101c00 msgr2=0x7f8a04075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:54:25.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:25.035+0000 7f8a037fe640 1 --2- 172.21.15.1:0/2712823819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a04101c00 0x7f8a04075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:25.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:25.035+0000 7f8a037fe640 1 -- 172.21.15.1:0/2712823819 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a040753e0 con 0x7f8a041037c0 2024-09-11T21:54:25.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:25.035+0000 7f8a037fe640 1 --2- 172.21.15.1:0/2712823819 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a041037c0 0x7f8a04074b20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f89fc00eaa0 tx=0x7f89fc00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:54:25.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:25.035+0000 7f89fbfff640 1 -- 172.21.15.1:0/2712823819 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89fc004d20 con 0x7f8a041037c0 2024-09-11T21:54:25.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:25.035+0000 7f89fbfff640 1 -- 172.21.15.1:0/2712823819 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89fc005020 con 0x7f8a041037c0 2024-09-11T21:54:25.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:25.035+0000 7f8a0940a640 1 -- 172.21.15.1:0/2712823819 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a041a20f0 con 0x7f8a041037c0 2024-09-11T21:54:25.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:25.035+0000 7f89fbfff640 1 -- 172.21.15.1:0/2712823819 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89fc0116d0 con 0x7f8a041037c0 2024-09-11T21:54:25.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:25.035+0000 7f8a0940a640 1 -- 172.21.15.1:0/2712823819 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a041a2630 con 0x7f8a041037c0 2024-09-11T21:54:25.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:25.037+0000 7f8a0940a640 1 -- 172.21.15.1:0/2712823819 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a04109070 con 0x7f8a041037c0 2024-09-11T21:54:25.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:25.038+0000 7f89fbfff640 1 -- 172.21.15.1:0/2712823819 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f89fc0051d0 con 0x7f8a041037c0 2024-09-11T21:54:25.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:25.038+0000 7f89fbfff640 1 --2- 172.21.15.1:0/2712823819 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f89f00a2b80 0x7f89f00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:54:25.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:25.038+0000 7f8a02ffd640 1 --2- 172.21.15.1:0/2712823819 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f89f00a2b80 0x7f89f00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:54:25.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:25.039+0000 7f89fbfff640 1 -- 172.21.15.1:0/2712823819 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6809+0+0 (secure 0 0 0) 0x7f89fc0cecb0 con 0x7f8a041037c0 2024-09-11T21:54:25.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:25.039+0000 7f8a02ffd640 1 --2- 172.21.15.1:0/2712823819 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f89f00a2b80 0x7f89f00a5040 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f89ec006010 tx=0x7f89ec000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:54:25.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:25.043+0000 7f89fbfff640 1 -- 172.21.15.1:0/2712823819 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f89fc016030 con 0x7f8a041037c0 2024-09-11T21:54:25.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:25.218+0000 7f8a0940a640 1 -- 172.21.15.1:0/2712823819 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f8a04062df0 con 0x7f8a041037c0 2024-09-11T21:54:25.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:25.218+0000 7f89fbfff640 1 -- 172.21.15.1:0/2712823819 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v75) ==== 74+0+133 (secure 0 0 0) 0x7f89fc0ce020 con 0x7f8a041037c0 2024-09-11T21:54:25.219 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:54:25.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:25.221+0000 7f8a0940a640 1 -- 172.21.15.1:0/2712823819 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f89f00a2b80 msgr2=0x7f89f00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:54:25.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:25.221+0000 7f8a0940a640 1 --2- 172.21.15.1:0/2712823819 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f89f00a2b80 0x7f89f00a5040 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f89ec006010 tx=0x7f89ec000bc0 comp rx=0 tx=0).stop 2024-09-11T21:54:25.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:25.221+0000 7f8a0940a640 1 -- 172.21.15.1:0/2712823819 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a041037c0 msgr2=0x7f8a04074b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:54:25.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:25.221+0000 7f8a0940a640 1 --2- 172.21.15.1:0/2712823819 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a041037c0 0x7f8a04074b20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f89fc00eaa0 tx=0x7f89fc00eea0 comp rx=0 tx=0).stop 2024-09-11T21:54:25.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:25.221+0000 7f8a0940a640 1 -- 172.21.15.1:0/2712823819 shutdown_connections 2024-09-11T21:54:25.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:25.221+0000 7f8a0940a640 1 --2- 172.21.15.1:0/2712823819 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f89f00a2b80 0x7f89f00a5040 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:25.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:25.221+0000 7f8a0940a640 1 --2- 172.21.15.1:0/2712823819 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a041037c0 0x7f8a04074b20 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:25.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:25.221+0000 7f8a0940a640 1 --2- 172.21.15.1:0/2712823819 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a04102e00 0x7f8a040745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:25.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:25.221+0000 7f8a0940a640 1 --2- 172.21.15.1:0/2712823819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a04101c00 0x7f8a04075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:25.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:25.221+0000 7f8a0940a640 1 -- 172.21.15.1:0/2712823819 >> 172.21.15.1:0/2712823819 conn(0x7f8a040fd390 msgr2=0x7f8a04106240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:54:25.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:25.221+0000 7f8a0940a640 1 -- 172.21.15.1:0/2712823819 shutdown_connections 2024-09-11T21:54:25.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:25.221+0000 7f8a0940a640 1 -- 172.21.15.1:0/2712823819 wait complete. 2024-09-11T21:54:25.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:25 smithi001 ceph-mon[30935]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-09-11T21:54:25.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:25 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/2712823819' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T21:54:26.080 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":75,"num_osds":12,"num_up_osds":11,"osd_up_since":1726091643,"num_in_osds":12,"osd_in_since":1726091645,"num_remapped_pgs":0} 2024-09-11T21:54:26.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:25 smithi161 ceph-mon[35615]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-09-11T21:54:26.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:25 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/2712823819' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T21:54:26.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 21:54:25 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T21:54:25.949+0000 7f67ae54c640 -1 osd.11 0 log_to_monitors true 2024-09-11T21:54:26.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:25 smithi053 ceph-mon[34908]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-09-11T21:54:26.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:25 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/2712823819' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T21:54:27.063 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:26 smithi161 ceph-mon[35615]: from='osd.11 [v2:172.21.15.161:6824/1586352190,v1:172.21.15.161:6825/1586352190]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-11T21:54:27.063 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:26 smithi161 ceph-mon[35615]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-11T21:54:27.081 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-11T21:54:27.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:26 smithi001 ceph-mon[30935]: from='osd.11 [v2:172.21.15.161:6824/1586352190,v1:172.21.15.161:6825/1586352190]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-11T21:54:27.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:26 smithi001 ceph-mon[30935]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-11T21:54:27.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:26 smithi053 ceph-mon[34908]: from='osd.11 [v2:172.21.15.161:6824/1586352190,v1:172.21.15.161:6825/1586352190]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-11T21:54:27.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:26 smithi053 ceph-mon[34908]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-11T21:54:27.360 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:54:27.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:27.817+0000 7f49daf42640 1 -- 172.21.15.1:0/2711909593 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49d4101cd0 msgr2=0x7f49d4102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:54:27.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:27.817+0000 7f49daf42640 1 --2- 172.21.15.1:0/2711909593 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49d4101cd0 0x7f49d4102150 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f49c0009900 tx=0x7f49c002a410 comp rx=0 tx=0).stop 2024-09-11T21:54:27.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:27.817+0000 7f49daf42640 1 -- 172.21.15.1:0/2711909593 shutdown_connections 2024-09-11T21:54:27.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:27.817+0000 7f49daf42640 1 --2- 172.21.15.1:0/2711909593 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49d4102690 0x7f49d4108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:27.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:27.817+0000 7f49daf42640 1 --2- 172.21.15.1:0/2711909593 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49d4101cd0 0x7f49d4102150 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:27.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:27.817+0000 7f49daf42640 1 --2- 172.21.15.1:0/2711909593 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49d4100ad0 0x7f49d4100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:27.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:27.817+0000 7f49daf42640 1 -- 172.21.15.1:0/2711909593 >> 172.21.15.1:0/2711909593 conn(0x7f49d40fc280 msgr2=0x7f49d40fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:54:27.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:27.817+0000 7f49daf42640 1 -- 172.21.15.1:0/2711909593 shutdown_connections 2024-09-11T21:54:27.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:27.817+0000 7f49daf42640 1 -- 172.21.15.1:0/2711909593 wait complete. 2024-09-11T21:54:27.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:27.818+0000 7f49daf42640 1 Processor -- start 2024-09-11T21:54:27.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:27.818+0000 7f49daf42640 1 -- start start 2024-09-11T21:54:27.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:27.818+0000 7f49daf42640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49d4100ad0 0x7f49d4197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:54:27.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:27.818+0000 7f49daf42640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49d4101cd0 0x7f49d41984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:54:27.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:27.818+0000 7f49daf42640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49d4102690 0x7f49d419f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:54:27.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:27.818+0000 7f49daf42640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f49d410d8c0 con 0x7f49d4101cd0 2024-09-11T21:54:27.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:27.818+0000 7f49daf42640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f49d410d740 con 0x7f49d4100ad0 2024-09-11T21:54:27.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:27.818+0000 7f49daf42640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f49d410da40 con 0x7f49d4102690 2024-09-11T21:54:27.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:27.818+0000 7f49d3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49d4101cd0 0x7f49d41984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:54:27.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:27.818+0000 7f49d3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49d4101cd0 0x7f49d41984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41380/0 (socket says 172.21.15.1:41380) 2024-09-11T21:54:27.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:27.818+0000 7f49d3fff640 1 -- 172.21.15.1:0/1247609319 learned_addr learned my addr 172.21.15.1:0/1247609319 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:54:27.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:27.818+0000 7f49d8cde640 1 --2- 172.21.15.1:0/1247609319 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49d4100ad0 0x7f49d4197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:54:27.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:27.818+0000 7f49d94df640 1 --2- 172.21.15.1:0/1247609319 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49d4102690 0x7f49d419f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:54:27.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:27.818+0000 7f49d3fff640 1 -- 172.21.15.1:0/1247609319 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49d4102690 msgr2=0x7f49d419f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:54:27.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:27.818+0000 7f49d3fff640 1 --2- 172.21.15.1:0/1247609319 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49d4102690 0x7f49d419f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:27.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:27.818+0000 7f49d3fff640 1 -- 172.21.15.1:0/1247609319 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49d4100ad0 msgr2=0x7f49d4197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:54:27.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:27.818+0000 7f49d3fff640 1 --2- 172.21.15.1:0/1247609319 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49d4100ad0 0x7f49d4197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:27.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:27.818+0000 7f49d3fff640 1 -- 172.21.15.1:0/1247609319 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49d419fb90 con 0x7f49d4101cd0 2024-09-11T21:54:27.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:27.819+0000 7f49d3fff640 1 --2- 172.21.15.1:0/1247609319 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49d4101cd0 0x7f49d41984a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f49c0009230 tx=0x7f49c0004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:54:27.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:27.819+0000 7f49d1ffb640 1 -- 172.21.15.1:0/1247609319 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c003f030 con 0x7f49d4101cd0 2024-09-11T21:54:27.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:27.819+0000 7f49daf42640 1 -- 172.21.15.1:0/1247609319 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49d419fe20 con 0x7f49d4101cd0 2024-09-11T21:54:27.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:27.819+0000 7f49d1ffb640 1 -- 172.21.15.1:0/1247609319 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49c0003ce0 con 0x7f49d4101cd0 2024-09-11T21:54:27.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:27.819+0000 7f49d1ffb640 1 -- 172.21.15.1:0/1247609319 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c0035070 con 0x7f49d4101cd0 2024-09-11T21:54:27.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:27.819+0000 7f49daf42640 1 -- 172.21.15.1:0/1247609319 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49d41a0280 con 0x7f49d4101cd0 2024-09-11T21:54:27.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:27.820+0000 7f49daf42640 1 -- 172.21.15.1:0/1247609319 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49d4107ed0 con 0x7f49d4101cd0 2024-09-11T21:54:27.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:27.821+0000 7f49d1ffb640 1 -- 172.21.15.1:0/1247609319 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f49c0042030 con 0x7f49d4101cd0 2024-09-11T21:54:27.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:27.822+0000 7f49d1ffb640 1 --2- 172.21.15.1:0/1247609319 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f49c40a2bb0 0x7f49c40a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:54:27.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:27.822+0000 7f49d8cde640 1 --2- 172.21.15.1:0/1247609319 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f49c40a2bb0 0x7f49c40a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:54:27.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:27.822+0000 7f49d1ffb640 1 -- 172.21.15.1:0/1247609319 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6831+0+0 (secure 0 0 0) 0x7f49c00ef120 con 0x7f49d4101cd0 2024-09-11T21:54:27.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:27.823+0000 7f49d8cde640 1 --2- 172.21.15.1:0/1247609319 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f49c40a2bb0 0x7f49c40a5070 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f49c8005fd0 tx=0x7f49c800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:54:27.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:27.826+0000 7f49d1ffb640 1 -- 172.21.15.1:0/1247609319 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f49c00bc5b0 con 0x7f49d4101cd0 2024-09-11T21:54:28.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:28.015+0000 7f49daf42640 1 -- 172.21.15.1:0/1247609319 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f49d4062e20 con 0x7f49d4101cd0 2024-09-11T21:54:28.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:28.016+0000 7f49d1ffb640 1 -- 172.21.15.1:0/1247609319 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v77) ==== 74+0+133 (secure 0 0 0) 0x7f49c00bbeb0 con 0x7f49d4101cd0 2024-09-11T21:54:28.017 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:27 smithi001 ceph-mon[30935]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-09-11T21:54:28.017 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:27 smithi001 ceph-mon[30935]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-11T21:54:28.017 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:27 smithi001 ceph-mon[30935]: osdmap e76: 12 total, 11 up, 12 in 2024-09-11T21:54:28.017 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:27 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T21:54:28.017 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:27 smithi001 ceph-mon[30935]: from='osd.11 [v2:172.21.15.161:6824/1586352190,v1:172.21.15.161:6825/1586352190]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-09-11T21:54:28.017 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:27 smithi001 ceph-mon[30935]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-09-11T21:54:28.017 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:27 smithi001 ceph-mon[30935]: Detected new or changed devices on smithi161 2024-09-11T21:54:28.017 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:27 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:28.017 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:27 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:28.017 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:27 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:54:28.017 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:27 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:54:28.018 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:27 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:54:28.018 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:27 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:54:28.018 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:27 smithi001 ceph-mon[30935]: Adjusting osd_memory_target on smithi161 to 4272M 2024-09-11T21:54:28.018 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:27 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:28.018 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:27 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:54:28.018 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:27 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:54:28.018 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:27 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:28.018 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:54:28.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:28.018+0000 7f49daf42640 1 -- 172.21.15.1:0/1247609319 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f49c40a2bb0 msgr2=0x7f49c40a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:54:28.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:28.018+0000 7f49daf42640 1 --2- 172.21.15.1:0/1247609319 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f49c40a2bb0 0x7f49c40a5070 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f49c8005fd0 tx=0x7f49c800b410 comp rx=0 tx=0).stop 2024-09-11T21:54:28.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:28.018+0000 7f49daf42640 1 -- 172.21.15.1:0/1247609319 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49d4101cd0 msgr2=0x7f49d41984a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:54:28.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:28.018+0000 7f49daf42640 1 --2- 172.21.15.1:0/1247609319 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49d4101cd0 0x7f49d41984a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f49c0009230 tx=0x7f49c0004360 comp rx=0 tx=0).stop 2024-09-11T21:54:28.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:28.018+0000 7f49daf42640 1 -- 172.21.15.1:0/1247609319 shutdown_connections 2024-09-11T21:54:28.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:28.018+0000 7f49daf42640 1 --2- 172.21.15.1:0/1247609319 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f49c40a2bb0 0x7f49c40a5070 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:28.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:28.018+0000 7f49daf42640 1 --2- 172.21.15.1:0/1247609319 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49d4102690 0x7f49d419f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:28.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:28.018+0000 7f49daf42640 1 --2- 172.21.15.1:0/1247609319 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49d4101cd0 0x7f49d41984a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:28.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:28.018+0000 7f49daf42640 1 --2- 172.21.15.1:0/1247609319 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49d4100ad0 0x7f49d4197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:28.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:28.018+0000 7f49daf42640 1 -- 172.21.15.1:0/1247609319 >> 172.21.15.1:0/1247609319 conn(0x7f49d40fc280 msgr2=0x7f49d41050a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:54:28.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:28.019+0000 7f49daf42640 1 -- 172.21.15.1:0/1247609319 shutdown_connections 2024-09-11T21:54:28.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:28.019+0000 7f49daf42640 1 -- 172.21.15.1:0/1247609319 wait complete. 2024-09-11T21:54:28.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:27 smithi053 ceph-mon[34908]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-09-11T21:54:28.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:27 smithi053 ceph-mon[34908]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-11T21:54:28.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:27 smithi053 ceph-mon[34908]: osdmap e76: 12 total, 11 up, 12 in 2024-09-11T21:54:28.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:27 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T21:54:28.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:27 smithi053 ceph-mon[34908]: from='osd.11 [v2:172.21.15.161:6824/1586352190,v1:172.21.15.161:6825/1586352190]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-09-11T21:54:28.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:27 smithi053 ceph-mon[34908]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-09-11T21:54:28.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:27 smithi053 ceph-mon[34908]: Detected new or changed devices on smithi161 2024-09-11T21:54:28.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:27 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:28.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:27 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:28.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:27 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:54:28.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:27 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:54:28.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:27 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:54:28.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:27 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:54:28.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:27 smithi053 ceph-mon[34908]: Adjusting osd_memory_target on smithi161 to 4272M 2024-09-11T21:54:28.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:27 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:28.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:27 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:54:28.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:27 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:54:28.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:27 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:28.318 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:27 smithi161 ceph-mon[35615]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-09-11T21:54:28.318 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:27 smithi161 ceph-mon[35615]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-11T21:54:28.318 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:27 smithi161 ceph-mon[35615]: osdmap e76: 12 total, 11 up, 12 in 2024-09-11T21:54:28.318 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:27 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T21:54:28.318 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:27 smithi161 ceph-mon[35615]: from='osd.11 [v2:172.21.15.161:6824/1586352190,v1:172.21.15.161:6825/1586352190]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-09-11T21:54:28.318 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:27 smithi161 ceph-mon[35615]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]: dispatch 2024-09-11T21:54:28.318 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:27 smithi161 ceph-mon[35615]: Detected new or changed devices on smithi161 2024-09-11T21:54:28.318 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:27 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:28.318 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:27 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:28.318 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:27 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:54:28.319 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:27 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:54:28.319 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:27 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:54:28.319 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:27 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:54:28.319 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:27 smithi161 ceph-mon[35615]: Adjusting osd_memory_target on smithi161 to 4272M 2024-09-11T21:54:28.319 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:27 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:28.319 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:27 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:54:28.319 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:27 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:54:28.319 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:27 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:28.592 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 21:54:28 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T21:54:28.316+0000 7f67aa4cf640 -1 osd.11 0 waiting for initial osdmap 2024-09-11T21:54:28.592 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 21:54:28 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T21:54:28.329+0000 7f67a22ef640 -1 osd.11 77 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T21:54:28.650 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":77,"num_osds":12,"num_up_osds":11,"osd_up_since":1726091643,"num_in_osds":12,"osd_in_since":1726091645,"num_remapped_pgs":0} 2024-09-11T21:54:29.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:28 smithi001 ceph-mon[30935]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-09-11T21:54:29.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:28 smithi001 ceph-mon[30935]: osdmap e77: 12 total, 11 up, 12 in 2024-09-11T21:54:29.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:28 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T21:54:29.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:28 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T21:54:29.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:28 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/1247609319' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T21:54:29.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:28 smithi001 ceph-mon[30935]: from='osd.11 ' entity='osd.11' 2024-09-11T21:54:29.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:28 smithi053 ceph-mon[34908]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-09-11T21:54:29.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:28 smithi053 ceph-mon[34908]: osdmap e77: 12 total, 11 up, 12 in 2024-09-11T21:54:29.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:28 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T21:54:29.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:28 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T21:54:29.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:28 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/1247609319' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T21:54:29.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:28 smithi053 ceph-mon[34908]: from='osd.11 ' entity='osd.11' 2024-09-11T21:54:29.344 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:28 smithi161 ceph-mon[35615]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi161", "root=default"]}]': finished 2024-09-11T21:54:29.344 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:28 smithi161 ceph-mon[35615]: osdmap e77: 12 total, 11 up, 12 in 2024-09-11T21:54:29.344 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:28 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T21:54:29.344 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:28 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T21:54:29.345 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:28 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/1247609319' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T21:54:29.345 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:28 smithi161 ceph-mon[35615]: from='osd.11 ' entity='osd.11' 2024-09-11T21:54:29.651 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-11T21:54:29.919 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:54:29.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:29 smithi001 ceph-mon[30935]: purged_snaps scrub starts 2024-09-11T21:54:29.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:29 smithi001 ceph-mon[30935]: purged_snaps scrub ok 2024-09-11T21:54:29.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:29 smithi001 ceph-mon[30935]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-09-11T21:54:29.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:29 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T21:54:29.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:29 smithi001 ceph-mon[30935]: osd.11 [v2:172.21.15.161:6824/1586352190,v1:172.21.15.161:6825/1586352190] boot 2024-09-11T21:54:29.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:29 smithi001 ceph-mon[30935]: osdmap e78: 12 total, 12 up, 12 in 2024-09-11T21:54:29.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:29 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T21:54:30.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:29 smithi053 ceph-mon[34908]: purged_snaps scrub starts 2024-09-11T21:54:30.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:29 smithi053 ceph-mon[34908]: purged_snaps scrub ok 2024-09-11T21:54:30.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:29 smithi053 ceph-mon[34908]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-09-11T21:54:30.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:29 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T21:54:30.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:29 smithi053 ceph-mon[34908]: osd.11 [v2:172.21.15.161:6824/1586352190,v1:172.21.15.161:6825/1586352190] boot 2024-09-11T21:54:30.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:29 smithi053 ceph-mon[34908]: osdmap e78: 12 total, 12 up, 12 in 2024-09-11T21:54:30.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:29 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T21:54:30.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:29 smithi161 ceph-mon[35615]: purged_snaps scrub starts 2024-09-11T21:54:30.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:29 smithi161 ceph-mon[35615]: purged_snaps scrub ok 2024-09-11T21:54:30.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:29 smithi161 ceph-mon[35615]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-09-11T21:54:30.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:29 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T21:54:30.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:29 smithi161 ceph-mon[35615]: osd.11 [v2:172.21.15.161:6824/1586352190,v1:172.21.15.161:6825/1586352190] boot 2024-09-11T21:54:30.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:29 smithi161 ceph-mon[35615]: osdmap e78: 12 total, 12 up, 12 in 2024-09-11T21:54:30.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:29 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T21:54:30.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:30.408+0000 7fb32a91b640 1 -- 172.21.15.1:0/1160577394 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb324107900 msgr2=0x7fb324109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:54:30.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:30.408+0000 7fb32a91b640 1 --2- 172.21.15.1:0/1160577394 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb324107900 0x7fb324109d50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb31c009ca0 tx=0x7fb31c02a8c0 comp rx=0 tx=0).stop 2024-09-11T21:54:30.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:30.408+0000 7fb32a91b640 1 -- 172.21.15.1:0/1160577394 shutdown_connections 2024-09-11T21:54:30.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:30.408+0000 7fb32a91b640 1 --2- 172.21.15.1:0/1160577394 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb324107900 0x7fb324109d50 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:30.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:30.408+0000 7fb32a91b640 1 --2- 172.21.15.1:0/1160577394 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb324104fd0 0x7fb3241073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:30.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:30.408+0000 7fb32a91b640 1 --2- 172.21.15.1:0/1160577394 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb3241026a0 0x7fb324104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:30.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:30.408+0000 7fb32a91b640 1 -- 172.21.15.1:0/1160577394 >> 172.21.15.1:0/1160577394 conn(0x7fb3240fc2a0 msgr2=0x7fb3240fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:54:30.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:30.408+0000 7fb32a91b640 1 -- 172.21.15.1:0/1160577394 shutdown_connections 2024-09-11T21:54:30.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:30.408+0000 7fb32a91b640 1 -- 172.21.15.1:0/1160577394 wait complete. 2024-09-11T21:54:30.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:30.409+0000 7fb32a91b640 1 Processor -- start 2024-09-11T21:54:30.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:30.409+0000 7fb32a91b640 1 -- start start 2024-09-11T21:54:30.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:30.409+0000 7fb32a91b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3241026a0 0x7fb324102000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:54:30.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:30.409+0000 7fb32a91b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb324104fd0 0x7fb324100650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:54:30.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:30.409+0000 7fb32a91b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb324107900 0x7fb324100b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:54:30.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:30.409+0000 7fb32a91b640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb32410d8c0 con 0x7fb324107900 2024-09-11T21:54:30.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:30.409+0000 7fb32a91b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb32410d740 con 0x7fb3241026a0 2024-09-11T21:54:30.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:30.409+0000 7fb32a91b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb32410da40 con 0x7fb324104fd0 2024-09-11T21:54:30.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:30.409+0000 7fb328eb8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb324107900 0x7fb324100b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:54:30.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:30.409+0000 7fb328eb8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb324107900 0x7fb324100b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41400/0 (socket says 172.21.15.1:41400) 2024-09-11T21:54:30.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:30.409+0000 7fb328eb8640 1 -- 172.21.15.1:0/2559822178 learned_addr learned my addr 172.21.15.1:0/2559822178 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:54:30.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:30.409+0000 7fb323fff640 1 --2- 172.21.15.1:0/2559822178 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3241026a0 0x7fb324102000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:54:30.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:30.410+0000 7fb3237fe640 1 --2- 172.21.15.1:0/2559822178 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb324104fd0 0x7fb324100650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:54:30.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:30.410+0000 7fb328eb8640 1 -- 172.21.15.1:0/2559822178 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb324104fd0 msgr2=0x7fb324100650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:54:30.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:30.410+0000 7fb328eb8640 1 --2- 172.21.15.1:0/2559822178 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb324104fd0 0x7fb324100650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:30.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:30.410+0000 7fb328eb8640 1 -- 172.21.15.1:0/2559822178 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3241026a0 msgr2=0x7fb324102000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:54:30.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:30.410+0000 7fb328eb8640 1 --2- 172.21.15.1:0/2559822178 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3241026a0 0x7fb324102000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:30.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:30.410+0000 7fb328eb8640 1 -- 172.21.15.1:0/2559822178 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb324101450 con 0x7fb324107900 2024-09-11T21:54:30.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:30.410+0000 7fb328eb8640 1 --2- 172.21.15.1:0/2559822178 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb324107900 0x7fb324100b90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb31c031ae0 tx=0x7fb31c031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:54:30.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:30.410+0000 7fb3217fa640 1 -- 172.21.15.1:0/2559822178 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb31c003c30 con 0x7fb324107900 2024-09-11T21:54:30.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:30.410+0000 7fb3217fa640 1 -- 172.21.15.1:0/2559822178 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb31c003dd0 con 0x7fb324107900 2024-09-11T21:54:30.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:30.410+0000 7fb32a91b640 1 -- 172.21.15.1:0/2559822178 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3241a2020 con 0x7fb324107900 2024-09-11T21:54:30.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:30.410+0000 7fb3217fa640 1 -- 172.21.15.1:0/2559822178 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb31c02fcb0 con 0x7fb324107900 2024-09-11T21:54:30.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:30.410+0000 7fb32a91b640 1 -- 172.21.15.1:0/2559822178 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3241a2580 con 0x7fb324107900 2024-09-11T21:54:30.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:30.411+0000 7fb32a91b640 1 -- 172.21.15.1:0/2559822178 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3240fdd90 con 0x7fb324107900 2024-09-11T21:54:30.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:30.412+0000 7fb3217fa640 1 -- 172.21.15.1:0/2559822178 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fb31c004050 con 0x7fb324107900 2024-09-11T21:54:30.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:30.413+0000 7fb3217fa640 1 --2- 172.21.15.1:0/2559822178 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fb30c0a2c00 0x7fb30c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:54:30.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:30.414+0000 7fb323fff640 1 --2- 172.21.15.1:0/2559822178 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fb30c0a2c00 0x7fb30c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:54:30.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:30.414+0000 7fb3217fa640 1 -- 172.21.15.1:0/2559822178 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7fb31c0efd50 con 0x7fb324107900 2024-09-11T21:54:30.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:30.414+0000 7fb323fff640 1 --2- 172.21.15.1:0/2559822178 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fb30c0a2c00 0x7fb30c0a50c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb314009bd0 tx=0x7fb314005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:54:30.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:30.418+0000 7fb3217fa640 1 -- 172.21.15.1:0/2559822178 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb31c0f3030 con 0x7fb324107900 2024-09-11T21:54:30.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:30.593+0000 7fb32a91b640 1 -- 172.21.15.1:0/2559822178 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fb324062df0 con 0x7fb324107900 2024-09-11T21:54:30.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:30.593+0000 7fb3217fa640 1 -- 172.21.15.1:0/2559822178 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v78) ==== 74+0+133 (secure 0 0 0) 0x7fb31c0bd270 con 0x7fb324107900 2024-09-11T21:54:30.594 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:54:30.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:30.596+0000 7fb32a91b640 1 -- 172.21.15.1:0/2559822178 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fb30c0a2c00 msgr2=0x7fb30c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:54:30.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:30.596+0000 7fb32a91b640 1 --2- 172.21.15.1:0/2559822178 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fb30c0a2c00 0x7fb30c0a50c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb314009bd0 tx=0x7fb314005f90 comp rx=0 tx=0).stop 2024-09-11T21:54:30.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:30.596+0000 7fb32a91b640 1 -- 172.21.15.1:0/2559822178 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb324107900 msgr2=0x7fb324100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:54:30.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:30.596+0000 7fb32a91b640 1 --2- 172.21.15.1:0/2559822178 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb324107900 0x7fb324100b90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb31c031ae0 tx=0x7fb31c031ee0 comp rx=0 tx=0).stop 2024-09-11T21:54:30.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:30.596+0000 7fb32a91b640 1 -- 172.21.15.1:0/2559822178 shutdown_connections 2024-09-11T21:54:30.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:30.596+0000 7fb32a91b640 1 --2- 172.21.15.1:0/2559822178 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fb30c0a2c00 0x7fb30c0a50c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:30.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:30.596+0000 7fb32a91b640 1 --2- 172.21.15.1:0/2559822178 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb324107900 0x7fb324100b90 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:30.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:30.596+0000 7fb32a91b640 1 --2- 172.21.15.1:0/2559822178 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb324104fd0 0x7fb324100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:30.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:30.596+0000 7fb32a91b640 1 --2- 172.21.15.1:0/2559822178 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3241026a0 0x7fb324102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:30.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:30.596+0000 7fb32a91b640 1 -- 172.21.15.1:0/2559822178 >> 172.21.15.1:0/2559822178 conn(0x7fb3240fc2a0 msgr2=0x7fb324102f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:54:30.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:30.596+0000 7fb32a91b640 1 -- 172.21.15.1:0/2559822178 shutdown_connections 2024-09-11T21:54:30.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:30.596+0000 7fb32a91b640 1 -- 172.21.15.1:0/2559822178 wait complete. 2024-09-11T21:54:31.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:30 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/2559822178' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T21:54:31.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:30 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/2559822178' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T21:54:31.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:30 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/2559822178' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T21:54:31.352 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":78,"num_osds":12,"num_up_osds":12,"osd_up_since":1726091669,"num_in_osds":12,"osd_in_since":1726091645,"num_remapped_pgs":0} 2024-09-11T21:54:31.352 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T21:54:31.614 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:54:32.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:32.086+0000 7f073f7e3640 1 -- 172.21.15.1:0/1514045062 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0738074620 msgr2=0x7f0738074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:54:32.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:32.086+0000 7f073f7e3640 1 --2- 172.21.15.1:0/1514045062 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0738074620 0x7f0738074a00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0734009900 tx=0x7f073402a480 comp rx=0 tx=0).stop 2024-09-11T21:54:32.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:32.086+0000 7f073f7e3640 1 -- 172.21.15.1:0/1514045062 shutdown_connections 2024-09-11T21:54:32.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:32.086+0000 7f073f7e3640 1 --2- 172.21.15.1:0/1514045062 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f073810e3c0 0x7f0738110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:32.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:32.086+0000 7f073f7e3640 1 --2- 172.21.15.1:0/1514045062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0738074f40 0x7f073810de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:32.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:32.086+0000 7f073f7e3640 1 --2- 172.21.15.1:0/1514045062 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0738074620 0x7f0738074a00 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:32.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:32.086+0000 7f073f7e3640 1 -- 172.21.15.1:0/1514045062 >> 172.21.15.1:0/1514045062 conn(0x7f07380fd8c0 msgr2=0x7f07380ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:54:32.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:32.086+0000 7f073f7e3640 1 -- 172.21.15.1:0/1514045062 shutdown_connections 2024-09-11T21:54:32.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:32.086+0000 7f073f7e3640 1 -- 172.21.15.1:0/1514045062 wait complete. 2024-09-11T21:54:32.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:32.087+0000 7f073f7e3640 1 Processor -- start 2024-09-11T21:54:32.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:32.087+0000 7f073f7e3640 1 -- start start 2024-09-11T21:54:32.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:32.087+0000 7f073f7e3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0738074620 0x7f07381a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:54:32.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:32.087+0000 7f073f7e3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0738074f40 0x7f07381a0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:54:32.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:32.087+0000 7f073f7e3640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f073810e3c0 0x7f073819a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:54:32.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:32.087+0000 7f073f7e3640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0738114230 con 0x7f0738074f40 2024-09-11T21:54:32.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:32.087+0000 7f073f7e3640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f07381140b0 con 0x7f0738074620 2024-09-11T21:54:32.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:32.087+0000 7f073f7e3640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f07381143b0 con 0x7f073810e3c0 2024-09-11T21:54:32.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:32.088+0000 7f073dd80640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f073810e3c0 0x7f073819a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:54:32.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:32.088+0000 7f073dd80640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f073810e3c0 0x7f073819a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:57926/0 (socket says 172.21.15.1:57926) 2024-09-11T21:54:32.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:32.088+0000 7f073dd80640 1 -- 172.21.15.1:0/1160790392 learned_addr learned my addr 172.21.15.1:0/1160790392 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:54:32.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:32.088+0000 7f073cd7e640 1 --2- 172.21.15.1:0/1160790392 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0738074f40 0x7f07381a0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:54:32.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:32.088+0000 7f073d57f640 1 --2- 172.21.15.1:0/1160790392 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0738074620 0x7f07381a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:54:32.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:32.088+0000 7f073dd80640 1 -- 172.21.15.1:0/1160790392 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0738074620 msgr2=0x7f07381a0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:54:32.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:32.088+0000 7f073dd80640 1 --2- 172.21.15.1:0/1160790392 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0738074620 0x7f07381a0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:32.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:32.088+0000 7f073dd80640 1 -- 172.21.15.1:0/1160790392 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0738074f40 msgr2=0x7f07381a0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:54:32.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:32.088+0000 7f073dd80640 1 --2- 172.21.15.1:0/1160790392 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0738074f40 0x7f07381a0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:32.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:32.088+0000 7f073dd80640 1 -- 172.21.15.1:0/1160790392 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f073819ad00 con 0x7f073810e3c0 2024-09-11T21:54:32.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:32.088+0000 7f073cd7e640 1 --2- 172.21.15.1:0/1160790392 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0738074f40 0x7f07381a0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T21:54:32.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:32.088+0000 7f073dd80640 1 --2- 172.21.15.1:0/1160790392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f073810e3c0 0x7f073819a5d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f073000b970 tx=0x7f073000bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:54:32.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:32.089+0000 7f072e7fc640 1 -- 172.21.15.1:0/1160790392 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0730019070 con 0x7f073810e3c0 2024-09-11T21:54:32.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:32.089+0000 7f073f7e3640 1 -- 172.21.15.1:0/1160790392 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f073819aff0 con 0x7f073810e3c0 2024-09-11T21:54:32.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:32.089+0000 7f073f7e3640 1 -- 172.21.15.1:0/1160790392 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07381a72e0 con 0x7f073810e3c0 2024-09-11T21:54:32.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:32.089+0000 7f072e7fc640 1 -- 172.21.15.1:0/1160790392 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0730014440 con 0x7f073810e3c0 2024-09-11T21:54:32.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:32.089+0000 7f072e7fc640 1 -- 172.21.15.1:0/1160790392 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f073001d610 con 0x7f073810e3c0 2024-09-11T21:54:32.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:32.090+0000 7f073f7e3640 1 -- 172.21.15.1:0/1160790392 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f071c005290 con 0x7f073810e3c0 2024-09-11T21:54:32.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:32.091+0000 7f072e7fc640 1 -- 172.21.15.1:0/1160790392 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f073001c070 con 0x7f073810e3c0 2024-09-11T21:54:32.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:32.092+0000 7f072e7fc640 1 --2- 172.21.15.1:0/1160790392 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f07240a2b80 0x7f07240a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:54:32.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:32.092+0000 7f073d57f640 1 --2- 172.21.15.1:0/1160790392 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f07240a2b80 0x7f07240a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:54:32.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:32.092+0000 7f072e7fc640 1 -- 172.21.15.1:0/1160790392 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f07300ce580 con 0x7f073810e3c0 2024-09-11T21:54:32.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:32.092+0000 7f073d57f640 1 --2- 172.21.15.1:0/1160790392 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f07240a2b80 0x7f07240a5040 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0734000c00 tx=0x7f07340058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:54:32.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:32.095+0000 7f072e7fc640 1 -- 172.21.15.1:0/1160790392 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f073009b290 con 0x7f073810e3c0 2024-09-11T21:54:32.189 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:31 smithi001 ceph-mon[30935]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:54:32.189 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:31 smithi001 ceph-mon[30935]: osdmap e79: 12 total, 12 up, 12 in 2024-09-11T21:54:32.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:32.275+0000 7f073f7e3640 1 -- 172.21.15.1:0/1160790392 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f071c005850 con 0x7f073810e3c0 2024-09-11T21:54:32.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:32.276+0000 7f072e7fc640 1 -- 172.21.15.1:0/1160790392 <== mon.1 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+18387 (secure 0 0 0) 0x7f07300b9e20 con 0x7f073810e3c0 2024-09-11T21:54:32.277 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:54:32.277 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":79,"fsid":"3ffca83c-7087-11ef-bcea-c7b262605968","created":"2024-09-11T21:47:28.868707+0000","modified":"2024-09-11T21:54:30.924742+0000","last_up_change":"2024-09-11T21:54:29.319441+0000","last_in_change":"2024-09-11T21:54:05.532281+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T21:50:41.026490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"19e56504-18f1-4234-b2d6-f193ff47d8eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":4012142023},{"type":"v1","addr":"172.21.15.1:6803","nonce":4012142023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":4012142023},{"type":"v1","addr":"172.21.15.1:6805","nonce":4012142023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":4012142023},{"type":"v1","addr":"172.21.15.1:6809","nonce":4012142023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":4012142023},{"type":"v1","addr":"172.21.15.1:6807","nonce":4012142023}]},"public_addr":"172.21.15.1:6803/4012142023","cluster_addr":"172.21.15.1:6805/4012142023","heartbeat_back_addr":"172.21.15.1:6809/4012142023","heartbeat_front_addr":"172.21.15.1:6807/4012142023","state":["exists","up"]},{"osd":1,"uuid":"996c9f0c-c5a4-43b8-b8e0-d712f27c5dfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3039391924},{"type":"v1","addr":"172.21.15.53:6801","nonce":3039391924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3039391924},{"type":"v1","addr":"172.21.15.53:6803","nonce":3039391924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3039391924},{"type":"v1","addr":"172.21.15.53:6807","nonce":3039391924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3039391924},{"type":"v1","addr":"172.21.15.53:6805","nonce":3039391924}]},"public_addr":"172.21.15.53:6801/3039391924","cluster_addr":"172.21.15.53:6803/3039391924","heartbeat_back_addr":"172.21.15.53:6807/3039391924","heartbeat_front_addr":"172.21.15.53:6805/3039391924","state":["exists","up"]},{"osd":2,"uuid":"578d4260-41e1-4645-b9b1-1c5445712c13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1707914885},{"type":"v1","addr":"172.21.15.161:6801","nonce":1707914885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1707914885},{"type":"v1","addr":"172.21.15.161:6803","nonce":1707914885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1707914885},{"type":"v1","addr":"172.21.15.161:6807","nonce":1707914885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1707914885},{"type":"v1","addr":"172.21.15.161:6805","nonce":1707914885}]},"public_addr":"172.21.15.161:6801/1707914885","cluster_addr":"172.21.15.161:6803/1707914885","heartbeat_back_addr":"172.21.15.161:6807/1707914885","heartbeat_front_addr":"172.21.15.161:6805/1707914885","state":["exists","up"]},{"osd":3,"uuid":"73f78766-9662-4aa3-841c-fe2ce654ee9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1541142398},{"type":"v1","addr":"172.21.15.1:6811","nonce":1541142398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1541142398},{"type":"v1","addr":"172.21.15.1:6813","nonce":1541142398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1541142398},{"type":"v1","addr":"172.21.15.1:6817","nonce":1541142398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1541142398},{"type":"v1","addr":"172.21.15.1:6815","nonce":1541142398}]},"public_addr":"172.21.15.1:6811/1541142398","cluster_addr":"172.21.15.1:6813/1541142398","heartbeat_back_addr":"172.21.15.1:6817/1541142398","heartbeat_front_addr":"172.21.15.1:6815/1541142398","state":["exists","up"]},{"osd":4,"uuid":"56c808bb-d4cd-4969-b49e-adef49ecd0f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":3081402681},{"type":"v1","addr":"172.21.15.53:6809","nonce":3081402681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3081402681},{"type":"v1","addr":"172.21.15.53:6811","nonce":3081402681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3081402681},{"type":"v1","addr":"172.21.15.53:6815","nonce":3081402681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3081402681},{"type":"v1","addr":"172.21.15.53:6813","nonce":3081402681}]},"public_addr":"172.21.15.53:6809/3081402681","cluster_addr":"172.21.15.53:6811/3081402681","heartbeat_back_addr":"172.21.15.53:6815/3081402681","heartbeat_front_addr":"172.21.15.53:6813/3081402681","state":["exists","up"]},{"osd":5,"uuid":"7b8266d6-b507-47c9-b4fa-ff16c4a74d35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":1622081327},{"type":"v1","addr":"172.21.15.161:6809","nonce":1622081327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":1622081327},{"type":"v1","addr":"172.21.15.161:6811","nonce":1622081327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":1622081327},{"type":"v1","addr":"172.21.15.161:6815","nonce":1622081327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":1622081327},{"type":"v1","addr":"172.21.15.161:6813","nonce":1622081327}]},"public_addr":"172.21.15.161:6809/1622081327","cluster_addr":"172.21.15.161:6811/1622081327","heartbeat_back_addr":"172.21.15.161:6815/1622081327","heartbeat_front_addr":"172.21.15.161:6813/1622081327","state":["exists","up"]},{"osd":6,"uuid":"3af5925d-a749-4849-a076-80580222cad4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":158434853},{"type":"v1","addr":"172.21.15.1:6819","nonce":158434853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":158434853},{"type":"v1","addr":"172.21.15.1:6821","nonce":158434853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":158434853},{"type":"v1","addr":"172.21.15.1:6825","nonce":158434853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":158434853},{"type":"v1","addr":"172.21.15.1:6823","nonce":158434853}]},"public_addr":"172.21.15.1:6819/158434853","cluster_addr":"172.21.15.1:6821/158434853","heartbeat_back_addr":"172.21.15.1:6825/158434853","heartbeat_front_addr":"172.21.15.1:6823/158434853","state":["exists","up"]},{"osd":7,"uuid":"f8f26c3c-17e9-4e89-8392-289796e0c8a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2132057697},{"type":"v1","addr":"172.21.15.53:6817","nonce":2132057697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2132057697},{"type":"v1","addr":"172.21.15.53:6819","nonce":2132057697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2132057697},{"type":"v1","addr":"172.21.15.53:6823","nonce":2132057697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2132057697},{"type":"v1","addr":"172.21.15.53:6821","nonce":2132057697}]},"public_addr":"172.21.15.53:6817/2132057697","cluster_addr":"172.21.15.53:6819/2132057697","heartbeat_back_addr":"172.21.15.53:6823/2132057697","heartbeat_front_addr":"172.21.15.53:6821/2132057697","state":["exists","up"]},{"osd":8,"uuid":"36ea401b-3c93-4b9d-8880-298d1eb6e30b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1838248669},{"type":"v1","addr":"172.21.15.161:6817","nonce":1838248669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1838248669},{"type":"v1","addr":"172.21.15.161:6819","nonce":1838248669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1838248669},{"type":"v1","addr":"172.21.15.161:6823","nonce":1838248669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1838248669},{"type":"v1","addr":"172.21.15.161:6821","nonce":1838248669}]},"public_addr":"172.21.15.161:6817/1838248669","cluster_addr":"172.21.15.161:6819/1838248669","heartbeat_back_addr":"172.21.15.161:6823/1838248669","heartbeat_front_addr":"172.21.15.161:6821/1838248669","state":["exists","up"]},{"osd":9,"uuid":"4b788d3f-a3bc-4463-bd99-95fdb7a52768","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2806274589},{"type":"v1","addr":"172.21.15.1:6827","nonce":2806274589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2806274589},{"type":"v1","addr":"172.21.15.1:6829","nonce":2806274589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":2806274589},{"type":"v1","addr":"172.21.15.1:6833","nonce":2806274589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2806274589},{"type":"v1","addr":"172.21.15.1:6831","nonce":2806274589}]},"public_addr":"172.21.15.1:6827/2806274589","cluster_addr":"172.21.15.1:6829/2806274589","heartbeat_back_addr":"172.21.15.1:6833/2806274589","heartbeat_front_addr":"172.21.15.1:6831/2806274589","state":["exists","up"]},{"osd":10,"uuid":"70549796-5270-4ae2-ac3f-f61f5de105d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":761923580},{"type":"v1","addr":"172.21.15.53:6825","nonce":761923580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":761923580},{"type":"v1","addr":"172.21.15.53:6827","nonce":761923580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":761923580},{"type":"v1","addr":"172.21.15.53:6831","nonce":761923580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":761923580},{"type":"v1","addr":"172.21.15.53:6829","nonce":761923580}]},"public_addr":"172.21.15.53:6825/761923580","cluster_addr":"172.21.15.53:6827/761923580","heartbeat_back_addr":"172.21.15.53:6831/761923580","heartbeat_front_addr":"172.21.15.53:6829/761923580","state":["exists","up"]},{"osd":11,"uuid":"48f17c4a-370a-493e-956e-bf6ad3dcc594","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1586352190},{"type":"v1","addr":"172.21.15.161:6825","nonce":1586352190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":1586352190},{"type":"v1","addr":"172.21.15.161:6827","nonce":1586352190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":1586352190},{"type":"v1","addr":"172.21.15.161:6831","nonce":1586352190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":1586352190},{"type":"v1","addr":"172.21.15.161:6829","nonce":1586352190}]},"public_addr":"172.21.15.161:6825/1586352190","cluster_addr":"172.21.15.161:6827/1586352190","heartbeat_back_addr":"172.21.15.161:6831/1586352190","heartbeat_front_addr":"172.21.15.161:6829/1586352190","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:49:46.539613+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:50:11.912300+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:50:36.955490+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:51:02.200554+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:51:27.860585+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:51:54.694099+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:52:18.962351+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:52:44.881923+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:53:10.301390+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:53:35.417647+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:54:01.492219+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:54:26.938338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/1749020850":"2024-09-12T21:48:24.524913+0000","172.21.15.1:6801/2821761445":"2024-09-12T21:48:24.524913+0000","172.21.15.1:0/1699984777":"2024-09-12T21:48:24.524913+0000","172.21.15.1:0/276770268":"2024-09-12T21:48:00.674728+0000","172.21.15.1:0/1278877732":"2024-09-12T21:48:24.524913+0000","172.21.15.1:0/2599895977":"2024-09-12T21:48:00.674728+0000","172.21.15.1:6801/2311309446":"2024-09-12T21:48:00.674728+0000","172.21.15.1:6800/2821761445":"2024-09-12T21:48:24.524913+0000","172.21.15.1:6800/2311309446":"2024-09-12T21:48:00.674728+0000","172.21.15.1:0/3202962495":"2024-09-12T21:48:00.674728+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T21:54:32.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:32.278+0000 7f073f7e3640 1 -- 172.21.15.1:0/1160790392 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f07240a2b80 msgr2=0x7f07240a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:54:32.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:32.278+0000 7f073f7e3640 1 --2- 172.21.15.1:0/1160790392 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f07240a2b80 0x7f07240a5040 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0734000c00 tx=0x7f07340058d0 comp rx=0 tx=0).stop 2024-09-11T21:54:32.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:32.278+0000 7f073f7e3640 1 -- 172.21.15.1:0/1160790392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f073810e3c0 msgr2=0x7f073819a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:54:32.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:32.278+0000 7f073f7e3640 1 --2- 172.21.15.1:0/1160790392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f073810e3c0 0x7f073819a5d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f073000b970 tx=0x7f073000bd70 comp rx=0 tx=0).stop 2024-09-11T21:54:32.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:32.279+0000 7f073f7e3640 1 -- 172.21.15.1:0/1160790392 shutdown_connections 2024-09-11T21:54:32.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:32.279+0000 7f073f7e3640 1 --2- 172.21.15.1:0/1160790392 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f07240a2b80 0x7f07240a5040 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:32.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:32.279+0000 7f073f7e3640 1 --2- 172.21.15.1:0/1160790392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f073810e3c0 0x7f073819a5d0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:32.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:32.279+0000 7f073f7e3640 1 --2- 172.21.15.1:0/1160790392 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0738074f40 0x7f07381a0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:32.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:32.279+0000 7f073f7e3640 1 --2- 172.21.15.1:0/1160790392 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0738074620 0x7f07381a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:32.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:32.279+0000 7f073f7e3640 1 -- 172.21.15.1:0/1160790392 >> 172.21.15.1:0/1160790392 conn(0x7f07380fd8c0 msgr2=0x7f07380ff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:54:32.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:32.279+0000 7f073f7e3640 1 -- 172.21.15.1:0/1160790392 shutdown_connections 2024-09-11T21:54:32.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:32.279+0000 7f073f7e3640 1 -- 172.21.15.1:0/1160790392 wait complete. 2024-09-11T21:54:32.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:31 smithi053 ceph-mon[34908]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:54:32.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:31 smithi053 ceph-mon[34908]: osdmap e79: 12 total, 12 up, 12 in 2024-09-11T21:54:32.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:31 smithi161 ceph-mon[35615]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:54:32.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:31 smithi161 ceph-mon[35615]: osdmap e79: 12 total, 12 up, 12 in 2024-09-11T21:54:32.930 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T21:50:41.026490+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-11T21:54:32.930 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-11T21:54:33.184 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:54:33.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:32 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/1160790392' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T21:54:33.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:32 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/1160790392' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T21:54:33.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:32 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/1160790392' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T21:54:33.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:33.672+0000 7fdb763ec640 1 -- 172.21.15.1:0/321232882 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb7010c170 msgr2=0x7fdb7010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:54:33.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:33.672+0000 7fdb763ec640 1 --2- 172.21.15.1:0/321232882 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb7010c170 0x7fdb7010e610 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fdb68009ca0 tx=0x7fdb6802a950 comp rx=0 tx=0).stop 2024-09-11T21:54:33.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:33.672+0000 7fdb763ec640 1 -- 172.21.15.1:0/321232882 shutdown_connections 2024-09-11T21:54:33.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:33.672+0000 7fdb763ec640 1 --2- 172.21.15.1:0/321232882 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb7010c170 0x7fdb7010e610 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:33.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:33.672+0000 7fdb763ec640 1 --2- 172.21.15.1:0/321232882 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb70103660 0x7fdb7010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:33.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:33.672+0000 7fdb763ec640 1 --2- 172.21.15.1:0/321232882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdb70102d40 0x7fdb70103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:33.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:33.672+0000 7fdb763ec640 1 -- 172.21.15.1:0/321232882 >> 172.21.15.1:0/321232882 conn(0x7fdb700fc7a0 msgr2=0x7fdb700febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:54:33.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:33.672+0000 7fdb763ec640 1 -- 172.21.15.1:0/321232882 shutdown_connections 2024-09-11T21:54:33.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:33.672+0000 7fdb763ec640 1 -- 172.21.15.1:0/321232882 wait complete. 2024-09-11T21:54:33.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:33.672+0000 7fdb763ec640 1 Processor -- start 2024-09-11T21:54:33.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:33.672+0000 7fdb763ec640 1 -- start start 2024-09-11T21:54:33.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:33.673+0000 7fdb763ec640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb70102d40 0x7fdb70198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:54:33.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:33.673+0000 7fdb763ec640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdb70103660 0x7fdb701989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:54:33.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:33.673+0000 7fdb763ec640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb7010c170 0x7fdb7019cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:54:33.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:33.673+0000 7fdb763ec640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdb70111fb0 con 0x7fdb7010c170 2024-09-11T21:54:33.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:33.673+0000 7fdb763ec640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fdb70111e30 con 0x7fdb70102d40 2024-09-11T21:54:33.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:33.673+0000 7fdb763ec640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fdb70112130 con 0x7fdb70103660 2024-09-11T21:54:33.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:33.673+0000 7fdb74989640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb7010c170 0x7fdb7019cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:54:33.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:33.673+0000 7fdb74989640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb7010c170 0x7fdb7019cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41438/0 (socket says 172.21.15.1:41438) 2024-09-11T21:54:33.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:33.673+0000 7fdb74989640 1 -- 172.21.15.1:0/1652740260 learned_addr learned my addr 172.21.15.1:0/1652740260 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:54:33.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:33.673+0000 7fdb6ffff640 1 --2- 172.21.15.1:0/1652740260 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb70102d40 0x7fdb70198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:54:33.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:33.673+0000 7fdb6f7fe640 1 --2- 172.21.15.1:0/1652740260 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdb70103660 0x7fdb701989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:54:33.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:33.673+0000 7fdb74989640 1 -- 172.21.15.1:0/1652740260 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdb70103660 msgr2=0x7fdb701989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:54:33.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:33.673+0000 7fdb74989640 1 --2- 172.21.15.1:0/1652740260 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdb70103660 0x7fdb701989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:33.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:33.673+0000 7fdb74989640 1 -- 172.21.15.1:0/1652740260 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb70102d40 msgr2=0x7fdb70198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:54:33.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:33.673+0000 7fdb74989640 1 --2- 172.21.15.1:0/1652740260 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb70102d40 0x7fdb70198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:33.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:33.673+0000 7fdb74989640 1 -- 172.21.15.1:0/1652740260 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb7019d420 con 0x7fdb7010c170 2024-09-11T21:54:33.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:33.673+0000 7fdb6f7fe640 1 --2- 172.21.15.1:0/1652740260 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdb70103660 0x7fdb701989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:54:33.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:33.673+0000 7fdb74989640 1 --2- 172.21.15.1:0/1652740260 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb7010c170 0x7fdb7019cd40 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fdb68006010 tx=0x7fdb6802f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:54:33.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:33.674+0000 7fdb6d7fa640 1 -- 172.21.15.1:0/1652740260 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb68034040 con 0x7fdb7010c170 2024-09-11T21:54:33.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:33.674+0000 7fdb763ec640 1 -- 172.21.15.1:0/1652740260 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb7019d6b0 con 0x7fdb7010c170 2024-09-11T21:54:33.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:33.674+0000 7fdb6d7fa640 1 -- 172.21.15.1:0/1652740260 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb68003e00 con 0x7fdb7010c170 2024-09-11T21:54:33.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:33.674+0000 7fdb6d7fa640 1 -- 172.21.15.1:0/1652740260 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb68045480 con 0x7fdb7010c170 2024-09-11T21:54:33.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:33.674+0000 7fdb763ec640 1 -- 172.21.15.1:0/1652740260 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb701a4ff0 con 0x7fdb7010c170 2024-09-11T21:54:33.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:33.675+0000 7fdb763ec640 1 -- 172.21.15.1:0/1652740260 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb70103120 con 0x7fdb7010c170 2024-09-11T21:54:33.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:33.676+0000 7fdb6d7fa640 1 -- 172.21.15.1:0/1652740260 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fdb68037020 con 0x7fdb7010c170 2024-09-11T21:54:33.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:33.676+0000 7fdb6d7fa640 1 --2- 172.21.15.1:0/1652740260 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fdb580a2b80 0x7fdb580a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:54:33.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:33.676+0000 7fdb6ffff640 1 --2- 172.21.15.1:0/1652740260 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fdb580a2b80 0x7fdb580a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:54:33.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:33.677+0000 7fdb6d7fa640 1 -- 172.21.15.1:0/1652740260 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7fdb680efd10 con 0x7fdb7010c170 2024-09-11T21:54:33.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:33.677+0000 7fdb6ffff640 1 --2- 172.21.15.1:0/1652740260 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fdb580a2b80 0x7fdb580a5040 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fdb60009990 tx=0x7fdb60009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:54:33.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:33.681+0000 7fdb6d7fa640 1 -- 172.21.15.1:0/1652740260 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdb680bd110 con 0x7fdb7010c170 2024-09-11T21:54:33.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:33.855+0000 7fdb763ec640 1 -- 172.21.15.1:0/1652740260 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdb701a5220 con 0x7fdb7010c170 2024-09-11T21:54:33.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:33.855+0000 7fdb6d7fa640 1 -- 172.21.15.1:0/1652740260 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) ==== 93+0+10 (secure 0 0 0) 0x7fdb680bca10 con 0x7fdb7010c170 2024-09-11T21:54:33.856 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-09-11T21:54:33.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:33.857+0000 7fdb763ec640 1 -- 172.21.15.1:0/1652740260 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fdb580a2b80 msgr2=0x7fdb580a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:54:33.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:33.857+0000 7fdb763ec640 1 --2- 172.21.15.1:0/1652740260 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fdb580a2b80 0x7fdb580a5040 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fdb60009990 tx=0x7fdb60009110 comp rx=0 tx=0).stop 2024-09-11T21:54:33.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:33.857+0000 7fdb763ec640 1 -- 172.21.15.1:0/1652740260 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb7010c170 msgr2=0x7fdb7019cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:54:33.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:33.857+0000 7fdb763ec640 1 --2- 172.21.15.1:0/1652740260 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb7010c170 0x7fdb7019cd40 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fdb68006010 tx=0x7fdb6802f710 comp rx=0 tx=0).stop 2024-09-11T21:54:33.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:33.857+0000 7fdb763ec640 1 -- 172.21.15.1:0/1652740260 shutdown_connections 2024-09-11T21:54:33.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:33.857+0000 7fdb763ec640 1 --2- 172.21.15.1:0/1652740260 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fdb580a2b80 0x7fdb580a5040 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:33.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:33.857+0000 7fdb763ec640 1 --2- 172.21.15.1:0/1652740260 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb7010c170 0x7fdb7019cd40 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:33.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:33.857+0000 7fdb763ec640 1 --2- 172.21.15.1:0/1652740260 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdb70103660 0x7fdb701989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:33.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:33.857+0000 7fdb763ec640 1 --2- 172.21.15.1:0/1652740260 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb70102d40 0x7fdb70198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:33.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:33.857+0000 7fdb763ec640 1 -- 172.21.15.1:0/1652740260 >> 172.21.15.1:0/1652740260 conn(0x7fdb700fc7a0 msgr2=0x7fdb700fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:54:33.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:33.858+0000 7fdb763ec640 1 -- 172.21.15.1:0/1652740260 shutdown_connections 2024-09-11T21:54:33.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:33.858+0000 7fdb763ec640 1 -- 172.21.15.1:0/1652740260 wait complete. 2024-09-11T21:54:34.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:33 smithi001 ceph-mon[30935]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:54:34.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:33 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/1652740260' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T21:54:34.198 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:33 smithi053 ceph-mon[34908]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:54:34.198 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:33 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/1652740260' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T21:54:34.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:33 smithi161 ceph-mon[35615]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:54:34.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:33 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/1652740260' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T21:54:34.614 INFO:tasks.cephadm:Adding mds.a on smithi001 2024-09-11T21:54:34.614 INFO:tasks.cephadm:Adding mds.d on smithi001 2024-09-11T21:54:34.614 INFO:tasks.cephadm:Adding mds.g on smithi001 2024-09-11T21:54:34.614 INFO:tasks.cephadm:Adding mds.j on smithi001 2024-09-11T21:54:34.614 INFO:tasks.cephadm:Adding mds.b on smithi053 2024-09-11T21:54:34.614 INFO:tasks.cephadm:Adding mds.e on smithi053 2024-09-11T21:54:34.614 INFO:tasks.cephadm:Adding mds.h on smithi053 2024-09-11T21:54:34.614 INFO:tasks.cephadm:Adding mds.k on smithi053 2024-09-11T21:54:34.614 INFO:tasks.cephadm:Adding mds.c on smithi161 2024-09-11T21:54:34.614 INFO:tasks.cephadm:Adding mds.f on smithi161 2024-09-11T21:54:34.614 INFO:tasks.cephadm:Adding mds.i on smithi161 2024-09-11T21:54:34.614 INFO:tasks.cephadm:Adding mds.l on smithi161 2024-09-11T21:54:34.615 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph orch apply mds all '12;smithi001=a;smithi001=d;smithi001=g;smithi001=j;smithi053=b;smithi053=e;smithi053=h;smithi053=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l' 2024-09-11T21:54:34.874 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T21:54:35.401 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:35.368+0000 7f8bb6cd8640 1 -- 172.21.15.161:0/3248365569 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8bb0102e00 msgr2=0x7f8bb0103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:54:35.401 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:35.368+0000 7f8bb6cd8640 1 --2- 172.21.15.161:0/3248365569 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8bb0102e00 0x7f8bb0103280 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c009900 tx=0x7f8b9c02a410 comp rx=0 tx=0).stop 2024-09-11T21:54:35.401 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:35.368+0000 7f8bb6cd8640 1 -- 172.21.15.161:0/3248365569 shutdown_connections 2024-09-11T21:54:35.401 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:35.368+0000 7f8bb6cd8640 1 --2- 172.21.15.161:0/3248365569 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8bb01037c0 0x7f8bb010a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:35.401 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:35.368+0000 7f8bb6cd8640 1 --2- 172.21.15.161:0/3248365569 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8bb0102e00 0x7f8bb0103280 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:35.401 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:35.368+0000 7f8bb6cd8640 1 --2- 172.21.15.161:0/3248365569 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bb0101c00 0x7f8bb0102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:35.401 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:35.368+0000 7f8bb6cd8640 1 -- 172.21.15.161:0/3248365569 >> 172.21.15.161:0/3248365569 conn(0x7f8bb00fd390 msgr2=0x7f8bb00ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:54:35.401 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:35.368+0000 7f8bb6cd8640 1 -- 172.21.15.161:0/3248365569 shutdown_connections 2024-09-11T21:54:35.401 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:35.368+0000 7f8bb6cd8640 1 -- 172.21.15.161:0/3248365569 wait complete. 2024-09-11T21:54:35.402 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:35.369+0000 7f8bb6cd8640 1 Processor -- start 2024-09-11T21:54:35.402 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:35.369+0000 7f8bb6cd8640 1 -- start start 2024-09-11T21:54:35.402 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:35.369+0000 7f8bb6cd8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8bb0101c00 0x7f8bb0197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:54:35.402 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:35.369+0000 7f8bb6cd8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bb0102e00 0x7f8bb0198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:54:35.402 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:35.369+0000 7f8bb6cd8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8bb01037c0 0x7f8bb019f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:54:35.402 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:35.369+0000 7f8bb6cd8640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb010d8d0 con 0x7f8bb0102e00 2024-09-11T21:54:35.402 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:35.369+0000 7f8bb6cd8640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb010d750 con 0x7f8bb01037c0 2024-09-11T21:54:35.402 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:35.369+0000 7f8bb6cd8640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb010da50 con 0x7f8bb0101c00 2024-09-11T21:54:35.402 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:35.369+0000 7f8bb4a74640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8bb0101c00 0x7f8bb0197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:54:35.402 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:35.369+0000 7f8bb4a74640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8bb0101c00 0x7f8bb0197ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:47386/0 (socket says 172.21.15.161:47386) 2024-09-11T21:54:35.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:35.369+0000 7f8bb4a74640 1 -- 172.21.15.161:0/3994935471 learned_addr learned my addr 172.21.15.161:0/3994935471 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-09-11T21:54:35.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:35.370+0000 7f8baffff640 1 --2- 172.21.15.161:0/3994935471 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bb0102e00 0x7f8bb0198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:54:35.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:35.370+0000 7f8bb5275640 1 --2- 172.21.15.161:0/3994935471 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8bb01037c0 0x7f8bb019f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:54:35.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:35.370+0000 7f8bb4a74640 1 -- 172.21.15.161:0/3994935471 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8bb01037c0 msgr2=0x7f8bb019f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:54:35.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:35.370+0000 7f8bb4a74640 1 --2- 172.21.15.161:0/3994935471 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8bb01037c0 0x7f8bb019f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:35.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:35.370+0000 7f8bb4a74640 1 -- 172.21.15.161:0/3994935471 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bb0102e00 msgr2=0x7f8bb0198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:54:35.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:35.370+0000 7f8bb4a74640 1 --2- 172.21.15.161:0/3994935471 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bb0102e00 0x7f8bb0198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:35.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:35.370+0000 7f8bb4a74640 1 -- 172.21.15.161:0/3994935471 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bb019fb00 con 0x7f8bb0101c00 2024-09-11T21:54:35.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:35.370+0000 7f8bb4a74640 1 --2- 172.21.15.161:0/3994935471 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8bb0101c00 0x7f8bb0197ed0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8ba400e9c0 tx=0x7f8ba400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:54:35.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:35.370+0000 7f8badffb640 1 -- 172.21.15.161:0/3994935471 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba400cd80 con 0x7f8bb0101c00 2024-09-11T21:54:35.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:35.370+0000 7f8bb6cd8640 1 -- 172.21.15.161:0/3994935471 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bb019fdf0 con 0x7f8bb0101c00 2024-09-11T21:54:35.403 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:35.370+0000 7f8badffb640 1 -- 172.21.15.161:0/3994935471 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ba40035f0 con 0x7f8bb0101c00 2024-09-11T21:54:35.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:35.370+0000 7f8badffb640 1 -- 172.21.15.161:0/3994935471 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba4010690 con 0x7f8bb0101c00 2024-09-11T21:54:35.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:35.370+0000 7f8bb6cd8640 1 -- 172.21.15.161:0/3994935471 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bb01a0250 con 0x7f8bb0101c00 2024-09-11T21:54:35.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:35.371+0000 7f8bb6cd8640 1 -- 172.21.15.161:0/3994935471 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bb0109000 con 0x7f8bb0101c00 2024-09-11T21:54:35.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:35.374+0000 7f8badffb640 1 -- 172.21.15.161:0/3994935471 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f8ba40037a0 con 0x7f8bb0101c00 2024-09-11T21:54:35.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:35.375+0000 7f8badffb640 1 --2- 172.21.15.161:0/3994935471 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f8ba00a2bb0 0x7f8ba00a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:54:35.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:35.375+0000 7f8badffb640 1 -- 172.21.15.161:0/3994935471 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f8ba40cf150 con 0x7f8bb0101c00 2024-09-11T21:54:35.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:35.376+0000 7f8baffff640 1 --2- 172.21.15.161:0/3994935471 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f8ba00a2bb0 0x7f8ba00a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:54:35.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:35.377+0000 7f8badffb640 1 -- 172.21.15.161:0/3994935471 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8ba409c5e0 con 0x7f8bb0101c00 2024-09-11T21:54:35.404 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:35.378+0000 7f8baffff640 1 --2- 172.21.15.161:0/3994935471 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f8ba00a2bb0 0x7f8ba00a5070 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c031610 tx=0x7f8b9c005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:54:35.561 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:35.560+0000 7f8bb6cd8640 1 -- 172.21.15.161:0/3994935471 --> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi001=a;smithi001=d;smithi001=g;smithi001=j;smithi053=b;smithi053=e;smithi053=h;smithi053=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l", "target": ["mon-mgr", ""]}) -- 0x7f8bb0062730 con 0x7f8ba00a2bb0 2024-09-11T21:54:35.569 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:35.568+0000 7f8badffb640 1 -- 172.21.15.161:0/3994935471 <== mgr.14150 v2:172.21.15.1:6800/3890350886 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f8bb0062730 con 0x7f8ba00a2bb0 2024-09-11T21:54:35.569 INFO:teuthology.orchestra.run.smithi161.stdout:Scheduled mds.all update... 2024-09-11T21:54:35.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:35.571+0000 7f8bb6cd8640 1 -- 172.21.15.161:0/3994935471 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f8ba00a2bb0 msgr2=0x7f8ba00a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:54:35.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:35.571+0000 7f8bb6cd8640 1 --2- 172.21.15.161:0/3994935471 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f8ba00a2bb0 0x7f8ba00a5070 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c031610 tx=0x7f8b9c005fd0 comp rx=0 tx=0).stop 2024-09-11T21:54:35.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:35.571+0000 7f8bb6cd8640 1 -- 172.21.15.161:0/3994935471 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8bb0101c00 msgr2=0x7f8bb0197ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:54:35.571 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:35.571+0000 7f8bb6cd8640 1 --2- 172.21.15.161:0/3994935471 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8bb0101c00 0x7f8bb0197ed0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8ba400e9c0 tx=0x7f8ba400ee80 comp rx=0 tx=0).stop 2024-09-11T21:54:35.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:35.571+0000 7f8bb6cd8640 1 -- 172.21.15.161:0/3994935471 shutdown_connections 2024-09-11T21:54:35.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:35.571+0000 7f8bb6cd8640 1 --2- 172.21.15.161:0/3994935471 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f8ba00a2bb0 0x7f8ba00a5070 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:35.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:35.571+0000 7f8bb6cd8640 1 --2- 172.21.15.161:0/3994935471 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8bb01037c0 0x7f8bb019f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:35.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:35.571+0000 7f8bb6cd8640 1 --2- 172.21.15.161:0/3994935471 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bb0102e00 0x7f8bb0198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:35.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:35.571+0000 7f8bb6cd8640 1 --2- 172.21.15.161:0/3994935471 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8bb0101c00 0x7f8bb0197ed0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:35.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:35.571+0000 7f8bb6cd8640 1 -- 172.21.15.161:0/3994935471 >> 172.21.15.161:0/3994935471 conn(0x7f8bb00fd390 msgr2=0x7f8bb01061d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:54:35.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:35.571+0000 7f8bb6cd8640 1 -- 172.21.15.161:0/3994935471 shutdown_connections 2024-09-11T21:54:35.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-09-11T21:54:35.571+0000 7f8bb6cd8640 1 -- 172.21.15.161:0/3994935471 wait complete. 2024-09-11T21:54:36.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:35 smithi001 ceph-mon[30935]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:54:36.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:35 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:36.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:35 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:54:36.264 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:35 smithi161 ceph-mon[35615]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:54:36.264 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:35 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:36.264 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:35 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:54:36.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:35 smithi053 ceph-mon[34908]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:54:36.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:35 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:36.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:35 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:54:36.303 DEBUG:teuthology.orchestra.run.smithi001:mds.a> sudo journalctl -f -n 0 -u ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mds.a.service 2024-09-11T21:54:36.306 DEBUG:teuthology.orchestra.run.smithi001:mds.d> sudo journalctl -f -n 0 -u ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mds.d.service 2024-09-11T21:54:36.349 DEBUG:teuthology.orchestra.run.smithi001:mds.g> sudo journalctl -f -n 0 -u ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mds.g.service 2024-09-11T21:54:36.392 DEBUG:teuthology.orchestra.run.smithi001:mds.j> sudo journalctl -f -n 0 -u ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mds.j.service 2024-09-11T21:54:36.435 DEBUG:teuthology.orchestra.run.smithi053:mds.b> sudo journalctl -f -n 0 -u ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mds.b.service 2024-09-11T21:54:36.438 DEBUG:teuthology.orchestra.run.smithi053:mds.e> sudo journalctl -f -n 0 -u ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mds.e.service 2024-09-11T21:54:36.481 DEBUG:teuthology.orchestra.run.smithi053:mds.h> sudo journalctl -f -n 0 -u ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mds.h.service 2024-09-11T21:54:36.524 DEBUG:teuthology.orchestra.run.smithi053:mds.k> sudo journalctl -f -n 0 -u ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mds.k.service 2024-09-11T21:54:36.566 DEBUG:teuthology.orchestra.run.smithi161:mds.c> sudo journalctl -f -n 0 -u ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mds.c.service 2024-09-11T21:54:36.568 DEBUG:teuthology.orchestra.run.smithi161:mds.f> sudo journalctl -f -n 0 -u ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mds.f.service 2024-09-11T21:54:36.611 DEBUG:teuthology.orchestra.run.smithi161:mds.i> sudo journalctl -f -n 0 -u ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mds.i.service 2024-09-11T21:54:36.654 DEBUG:teuthology.orchestra.run.smithi161:mds.l> sudo journalctl -f -n 0 -u ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mds.l.service 2024-09-11T21:54:36.697 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-11T21:54:36.699 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T21:54:36.959 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:54:37.081 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:36 smithi001 ceph-mon[30935]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi001=a;smithi001=d;smithi001=g;smithi001=j;smithi053=b;smithi053=e;smithi053=h;smithi053=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:54:37.081 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:36 smithi001 ceph-mon[30935]: Saving service mds.all spec with placement smithi001=a;smithi001=d;smithi001=g;smithi001=j;smithi053=b;smithi053=e;smithi053=h;smithi053=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l;count:12 2024-09-11T21:54:37.081 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:36 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:54:37.081 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:36 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:54:37.081 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:36 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:37.081 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:36 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T21:54:37.081 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:36 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T21:54:37.081 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:36 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:54:37.082 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:36 smithi001 ceph-mon[30935]: Deploying daemon mds.l on smithi161 2024-09-11T21:54:37.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:36 smithi053 ceph-mon[34908]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi001=a;smithi001=d;smithi001=g;smithi001=j;smithi053=b;smithi053=e;smithi053=h;smithi053=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:54:37.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:36 smithi053 ceph-mon[34908]: Saving service mds.all spec with placement smithi001=a;smithi001=d;smithi001=g;smithi001=j;smithi053=b;smithi053=e;smithi053=h;smithi053=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l;count:12 2024-09-11T21:54:37.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:36 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:54:37.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:36 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:54:37.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:36 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:37.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:36 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T21:54:37.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:36 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T21:54:37.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:36 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:54:37.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:36 smithi053 ceph-mon[34908]: Deploying daemon mds.l on smithi161 2024-09-11T21:54:37.338 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:36 smithi161 ceph-mon[35615]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi001=a;smithi001=d;smithi001=g;smithi001=j;smithi053=b;smithi053=e;smithi053=h;smithi053=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:54:37.338 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:36 smithi161 ceph-mon[35615]: Saving service mds.all spec with placement smithi001=a;smithi001=d;smithi001=g;smithi001=j;smithi053=b;smithi053=e;smithi053=h;smithi053=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l;count:12 2024-09-11T21:54:37.338 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:36 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:54:37.338 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:36 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:54:37.338 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:36 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:37.338 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:36 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T21:54:37.338 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:36 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T21:54:37.339 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:36 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:54:37.339 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:36 smithi161 ceph-mon[35615]: Deploying daemon mds.l on smithi161 2024-09-11T21:54:37.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:37.436+0000 7f9660d46640 1 -- 172.21.15.1:0/3632933399 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f965c106f50 msgr2=0x7f965c10eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:54:37.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:37.436+0000 7f9660d46640 1 --2- 172.21.15.1:0/3632933399 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f965c106f50 0x7f965c10eaf0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9654009cd0 tx=0x7f965402a8e0 comp rx=0 tx=0).stop 2024-09-11T21:54:37.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:37.436+0000 7f9660d46640 1 -- 172.21.15.1:0/3632933399 shutdown_connections 2024-09-11T21:54:37.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:37.436+0000 7f9660d46640 1 --2- 172.21.15.1:0/3632933399 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f965c106f50 0x7f965c10eaf0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:37.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:37.436+0000 7f9660d46640 1 --2- 172.21.15.1:0/3632933399 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f965c100ca0 0x7f965c106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:37.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:37.436+0000 7f9660d46640 1 --2- 172.21.15.1:0/3632933399 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f965c1002f0 0x7f965c1006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:37.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:37.436+0000 7f9660d46640 1 -- 172.21.15.1:0/3632933399 >> 172.21.15.1:0/3632933399 conn(0x7f965c077210 msgr2=0x7f965c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:54:37.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:37.436+0000 7f9660d46640 1 -- 172.21.15.1:0/3632933399 shutdown_connections 2024-09-11T21:54:37.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:37.436+0000 7f9660d46640 1 -- 172.21.15.1:0/3632933399 wait complete. 2024-09-11T21:54:37.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:37.437+0000 7f9660d46640 1 Processor -- start 2024-09-11T21:54:37.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:37.437+0000 7f9660d46640 1 -- start start 2024-09-11T21:54:37.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:37.437+0000 7f9660d46640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f965c1002f0 0x7f965c1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:54:37.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:37.437+0000 7f9660d46640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f965c100ca0 0x7f965c198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:54:37.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:37.437+0000 7f9660d46640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f965c106f50 0x7f965c19cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:54:37.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:37.437+0000 7f9660d46640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f965c111fa0 con 0x7f965c100ca0 2024-09-11T21:54:37.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:37.437+0000 7f9660d46640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f965c111e20 con 0x7f965c1002f0 2024-09-11T21:54:37.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:37.437+0000 7f9660d46640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f965c112120 con 0x7f965c106f50 2024-09-11T21:54:37.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:37.437+0000 7f9659d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f965c100ca0 0x7f965c198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:54:37.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:37.437+0000 7f9659d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f965c100ca0 0x7f965c198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33952/0 (socket says 172.21.15.1:33952) 2024-09-11T21:54:37.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:37.437+0000 7f9659d9b640 1 -- 172.21.15.1:0/746486250 learned_addr learned my addr 172.21.15.1:0/746486250 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:54:37.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:37.437+0000 7f965a59c640 1 --2- 172.21.15.1:0/746486250 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f965c1002f0 0x7f965c1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:54:37.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:37.437+0000 7f965ad9d640 1 --2- 172.21.15.1:0/746486250 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f965c106f50 0x7f965c19cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:54:37.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:37.438+0000 7f9659d9b640 1 -- 172.21.15.1:0/746486250 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f965c106f50 msgr2=0x7f965c19cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:54:37.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:37.438+0000 7f9659d9b640 1 --2- 172.21.15.1:0/746486250 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f965c106f50 0x7f965c19cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:37.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:37.438+0000 7f9659d9b640 1 -- 172.21.15.1:0/746486250 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f965c1002f0 msgr2=0x7f965c1983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:54:37.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:37.438+0000 7f9659d9b640 1 --2- 172.21.15.1:0/746486250 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f965c1002f0 0x7f965c1983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:37.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:37.438+0000 7f9659d9b640 1 -- 172.21.15.1:0/746486250 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f965c19d380 con 0x7f965c100ca0 2024-09-11T21:54:37.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:37.438+0000 7f9659d9b640 1 --2- 172.21.15.1:0/746486250 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f965c100ca0 0x7f965c198910 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f965000c970 tx=0x7f965000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:54:37.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:37.438+0000 7f964b7fe640 1 -- 172.21.15.1:0/746486250 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9650014070 con 0x7f965c100ca0 2024-09-11T21:54:37.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:37.438+0000 7f9660d46640 1 -- 172.21.15.1:0/746486250 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f965c19d670 con 0x7f965c100ca0 2024-09-11T21:54:37.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:37.438+0000 7f964b7fe640 1 -- 172.21.15.1:0/746486250 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9650004440 con 0x7f965c100ca0 2024-09-11T21:54:37.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:37.438+0000 7f964b7fe640 1 -- 172.21.15.1:0/746486250 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965000f450 con 0x7f965c100ca0 2024-09-11T21:54:37.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:37.438+0000 7f9660d46640 1 -- 172.21.15.1:0/746486250 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f965c1a4f50 con 0x7f965c100ca0 2024-09-11T21:54:37.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:37.439+0000 7f9660d46640 1 -- 172.21.15.1:0/746486250 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f965c1006d0 con 0x7f965c100ca0 2024-09-11T21:54:37.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:37.440+0000 7f964b7fe640 1 -- 172.21.15.1:0/746486250 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f9650017070 con 0x7f965c100ca0 2024-09-11T21:54:37.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:37.441+0000 7f964b7fe640 1 --2- 172.21.15.1:0/746486250 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f96440a2b80 0x7f96440a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:54:37.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:37.441+0000 7f965a59c640 1 --2- 172.21.15.1:0/746486250 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f96440a2b80 0x7f96440a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:54:37.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:37.441+0000 7f964b7fe640 1 -- 172.21.15.1:0/746486250 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f96500ce8e0 con 0x7f965c100ca0 2024-09-11T21:54:37.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:37.441+0000 7f965a59c640 1 --2- 172.21.15.1:0/746486250 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f96440a2b80 0x7f96440a5040 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f964c00bbe0 tx=0x7f964c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:54:37.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:37.445+0000 7f964b7fe640 1 -- 172.21.15.1:0/746486250 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f965001e020 con 0x7f965c100ca0 2024-09-11T21:54:37.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:37.659+0000 7f9660d46640 1 -- 172.21.15.1:0/746486250 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f965c02d7c0 con 0x7f965c100ca0 2024-09-11T21:54:37.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:37.660+0000 7f964b7fe640 1 -- 172.21.15.1:0/746486250 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7f965009bce0 con 0x7f965c100ca0 2024-09-11T21:54:37.661 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:54:37.661 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":1,"btime":"2024-09-11T21:47:28:868551+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-11T21:54:37.661 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 1 2024-09-11T21:54:37.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:37.662+0000 7f9660d46640 1 -- 172.21.15.1:0/746486250 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f96440a2b80 msgr2=0x7f96440a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:54:37.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:37.662+0000 7f9660d46640 1 --2- 172.21.15.1:0/746486250 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f96440a2b80 0x7f96440a5040 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f964c00bbe0 tx=0x7f964c00b390 comp rx=0 tx=0).stop 2024-09-11T21:54:37.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:37.662+0000 7f9660d46640 1 -- 172.21.15.1:0/746486250 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f965c100ca0 msgr2=0x7f965c198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:54:37.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:37.662+0000 7f9660d46640 1 --2- 172.21.15.1:0/746486250 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f965c100ca0 0x7f965c198910 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f965000c970 tx=0x7f965000ce30 comp rx=0 tx=0).stop 2024-09-11T21:54:37.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:37.662+0000 7f9660d46640 1 -- 172.21.15.1:0/746486250 shutdown_connections 2024-09-11T21:54:37.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:37.662+0000 7f9660d46640 1 --2- 172.21.15.1:0/746486250 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f96440a2b80 0x7f96440a5040 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:37.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:37.662+0000 7f9660d46640 1 --2- 172.21.15.1:0/746486250 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f965c106f50 0x7f965c19cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:37.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:37.662+0000 7f9660d46640 1 --2- 172.21.15.1:0/746486250 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f965c100ca0 0x7f965c198910 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:37.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:37.662+0000 7f9660d46640 1 --2- 172.21.15.1:0/746486250 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f965c1002f0 0x7f965c1983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:37.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:37.662+0000 7f9660d46640 1 -- 172.21.15.1:0/746486250 >> 172.21.15.1:0/746486250 conn(0x7f965c077210 msgr2=0x7f965c105810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:54:37.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:37.662+0000 7f9660d46640 1 -- 172.21.15.1:0/746486250 shutdown_connections 2024-09-11T21:54:37.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:37.662+0000 7f9660d46640 1 -- 172.21.15.1:0/746486250 wait complete. 2024-09-11T21:54:38.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:37 smithi053 ceph-mon[34908]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:54:38.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:37 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/746486250' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:54:38.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:37 smithi161 ceph-mon[35615]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:54:38.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:37 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/746486250' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:54:38.342 INFO:journalctl@ceph.mds.l.smithi161.stdout:Sep 11 21:54:38 smithi161 systemd[1]: Starting Ceph mds.l for 3ffca83c-7087-11ef-bcea-c7b262605968... 2024-09-11T21:54:38.370 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:37 smithi001 ceph-mon[30935]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:54:38.370 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:37 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/746486250' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:54:38.635 INFO:journalctl@ceph.mds.l.smithi161.stdout:Sep 11 21:54:38 smithi161 podman[65044]: 2024-09-11 21:54:38.28243292 +0000 UTC m=+0.015915449 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T21:54:38.636 INFO:journalctl@ceph.mds.l.smithi161.stdout:Sep 11 21:54:38 smithi161 podman[65044]: 2024-09-11 21:54:38.391633604 +0000 UTC m=+0.125116126 container create c8172226b47872aa8ed04235a4bed705e5eebae99826fb76141579bab1a1d985 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-l, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3) 2024-09-11T21:54:39.092 INFO:journalctl@ceph.mds.l.smithi161.stdout:Sep 11 21:54:38 smithi161 podman[65044]: 2024-09-11 21:54:38.635434968 +0000 UTC m=+0.368917489 container init c8172226b47872aa8ed04235a4bed705e5eebae99826fb76141579bab1a1d985 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-l, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3) 2024-09-11T21:54:39.092 INFO:journalctl@ceph.mds.l.smithi161.stdout:Sep 11 21:54:38 smithi161 podman[65044]: 2024-09-11 21:54:38.638318664 +0000 UTC m=+0.371801204 container start c8172226b47872aa8ed04235a4bed705e5eebae99826fb76141579bab1a1d985 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-l, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-11T21:54:39.092 INFO:journalctl@ceph.mds.l.smithi161.stdout:Sep 11 21:54:38 smithi161 ceph-mds[65064]: -- 172.21.15.161:0/4202986577 <== mon.1 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d68400d860 con 0x55d683feb400 2024-09-11T21:54:39.092 INFO:journalctl@ceph.mds.l.smithi161.stdout:Sep 11 21:54:38 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-l[65060]: starting mds.l at 2024-09-11T21:54:39.092 INFO:journalctl@ceph.mds.l.smithi161.stdout:Sep 11 21:54:38 smithi161 bash[65044]: c8172226b47872aa8ed04235a4bed705e5eebae99826fb76141579bab1a1d985 2024-09-11T21:54:39.092 INFO:journalctl@ceph.mds.l.smithi161.stdout:Sep 11 21:54:38 smithi161 systemd[1]: Started Ceph mds.l for 3ffca83c-7087-11ef-bcea-c7b262605968. 2024-09-11T21:54:39.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:39 smithi053 ceph-mon[34908]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:54:39.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:39 smithi053 ceph-mon[34908]: mds.? [v2:172.21.15.161:6832/872410024,v1:172.21.15.161:6833/872410024] up:boot 2024-09-11T21:54:39.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:39 smithi053 ceph-mon[34908]: fsmap 1 up:standby 2024-09-11T21:54:39.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:39 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-11T21:54:39.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:39 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:39.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:39 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:39.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:39 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:39.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:39 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T21:54:39.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:39 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T21:54:39.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:39 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:54:39.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:39 smithi053 ceph-mon[34908]: Deploying daemon mds.k on smithi053 2024-09-11T21:54:39.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:39 smithi001 ceph-mon[30935]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:54:39.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:39 smithi001 ceph-mon[30935]: mds.? [v2:172.21.15.161:6832/872410024,v1:172.21.15.161:6833/872410024] up:boot 2024-09-11T21:54:39.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:39 smithi001 ceph-mon[30935]: fsmap 1 up:standby 2024-09-11T21:54:39.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:39 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-11T21:54:39.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:39 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:39.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:39 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:39.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:39 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:39.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:39 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T21:54:39.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:39 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T21:54:39.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:39 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:54:39.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:39 smithi001 ceph-mon[30935]: Deploying daemon mds.k on smithi053 2024-09-11T21:54:40.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:39 smithi161 ceph-mon[35615]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:54:40.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:39 smithi161 ceph-mon[35615]: mds.? [v2:172.21.15.161:6832/872410024,v1:172.21.15.161:6833/872410024] up:boot 2024-09-11T21:54:40.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:39 smithi161 ceph-mon[35615]: fsmap 1 up:standby 2024-09-11T21:54:40.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:39 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-11T21:54:40.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:39 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:40.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:39 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:40.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:39 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:40.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:39 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T21:54:40.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:39 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T21:54:40.093 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:39 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:54:40.093 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:39 smithi161 ceph-mon[35615]: Deploying daemon mds.k on smithi053 2024-09-11T21:54:40.403 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T21:54:40.671 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:54:41.047 INFO:journalctl@ceph.mds.k.smithi053.stdout:Sep 11 21:54:40 smithi053 systemd[1]: Starting Ceph mds.k for 3ffca83c-7087-11ef-bcea-c7b262605968... 2024-09-11T21:54:41.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:41.130+0000 7fb3539bf640 1 -- 172.21.15.1:0/3477876070 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb34c101de0 msgr2=0x7fb34c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:54:41.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:41.130+0000 7fb3539bf640 1 --2- 172.21.15.1:0/3477876070 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb34c101de0 0x7fb34c1021c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb348009900 tx=0x7fb34802a410 comp rx=0 tx=0).stop 2024-09-11T21:54:41.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:41.130+0000 7fb3539bf640 1 -- 172.21.15.1:0/3477876070 shutdown_connections 2024-09-11T21:54:41.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:41.130+0000 7fb3539bf640 1 --2- 172.21.15.1:0/3477876070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb34c106f10 0x7fb34c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:41.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:41.130+0000 7fb3539bf640 1 --2- 172.21.15.1:0/3477876070 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb34c102790 0x7fb34c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:41.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:41.130+0000 7fb3539bf640 1 --2- 172.21.15.1:0/3477876070 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb34c101de0 0x7fb34c1021c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:41.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:41.130+0000 7fb3539bf640 1 -- 172.21.15.1:0/3477876070 >> 172.21.15.1:0/3477876070 conn(0x7fb34c0fd8c0 msgr2=0x7fb34c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:54:41.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:41.130+0000 7fb3539bf640 1 -- 172.21.15.1:0/3477876070 shutdown_connections 2024-09-11T21:54:41.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:41.130+0000 7fb3539bf640 1 -- 172.21.15.1:0/3477876070 wait complete. 2024-09-11T21:54:41.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:41.130+0000 7fb3539bf640 1 Processor -- start 2024-09-11T21:54:41.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:41.131+0000 7fb3539bf640 1 -- start start 2024-09-11T21:54:41.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:41.131+0000 7fb3539bf640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb34c101de0 0x7fb34c198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:54:41.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:41.131+0000 7fb3539bf640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb34c102790 0x7fb34c1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:54:41.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:41.131+0000 7fb3539bf640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb34c106f10 0x7fb34c19cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:54:41.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:41.131+0000 7fb3539bf640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb34c111f90 con 0x7fb34c106f10 2024-09-11T21:54:41.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:41.131+0000 7fb3539bf640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb34c111e10 con 0x7fb34c101de0 2024-09-11T21:54:41.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:41.131+0000 7fb3539bf640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb34c112110 con 0x7fb34c102790 2024-09-11T21:54:41.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:41.131+0000 7fb351f5c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb34c106f10 0x7fb34c19cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:54:41.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:41.131+0000 7fb351f5c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb34c106f10 0x7fb34c19cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33966/0 (socket says 172.21.15.1:33966) 2024-09-11T21:54:41.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:41.131+0000 7fb351f5c640 1 -- 172.21.15.1:0/247148288 learned_addr learned my addr 172.21.15.1:0/247148288 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:54:41.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:41.131+0000 7fb35175b640 1 --2- 172.21.15.1:0/247148288 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb34c101de0 0x7fb34c198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:54:41.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:41.131+0000 7fb350f5a640 1 --2- 172.21.15.1:0/247148288 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb34c102790 0x7fb34c1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:54:41.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:41.131+0000 7fb351f5c640 1 -- 172.21.15.1:0/247148288 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb34c102790 msgr2=0x7fb34c1989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:54:41.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:41.131+0000 7fb351f5c640 1 --2- 172.21.15.1:0/247148288 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb34c102790 0x7fb34c1989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:41.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:41.131+0000 7fb351f5c640 1 -- 172.21.15.1:0/247148288 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb34c101de0 msgr2=0x7fb34c198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:54:41.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:41.131+0000 7fb351f5c640 1 --2- 172.21.15.1:0/247148288 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb34c101de0 0x7fb34c198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:41.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:41.131+0000 7fb351f5c640 1 -- 172.21.15.1:0/247148288 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb34c19d440 con 0x7fb34c106f10 2024-09-11T21:54:41.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:41.131+0000 7fb35175b640 1 --2- 172.21.15.1:0/247148288 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb34c101de0 0x7fb34c198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:54:41.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:41.131+0000 7fb350f5a640 1 --2- 172.21.15.1:0/247148288 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb34c102790 0x7fb34c1989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T21:54:41.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:41.131+0000 7fb351f5c640 1 --2- 172.21.15.1:0/247148288 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb34c106f10 0x7fb34c19cd60 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb34400b970 tx=0x7fb34400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:54:41.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:41.132+0000 7fb3427fc640 1 -- 172.21.15.1:0/247148288 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb344019070 con 0x7fb34c106f10 2024-09-11T21:54:41.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:41.132+0000 7fb3539bf640 1 -- 172.21.15.1:0/247148288 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb34c19d730 con 0x7fb34c106f10 2024-09-11T21:54:41.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:41.132+0000 7fb3427fc640 1 -- 172.21.15.1:0/247148288 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb34400cb40 con 0x7fb34c106f10 2024-09-11T21:54:41.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:41.132+0000 7fb3427fc640 1 -- 172.21.15.1:0/247148288 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb344007610 con 0x7fb34c106f10 2024-09-11T21:54:41.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:41.132+0000 7fb3539bf640 1 -- 172.21.15.1:0/247148288 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb34c1a5010 con 0x7fb34c106f10 2024-09-11T21:54:41.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:41.133+0000 7fb3539bf640 1 -- 172.21.15.1:0/247148288 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb34c1021c0 con 0x7fb34c106f10 2024-09-11T21:54:41.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:41.135+0000 7fb3427fc640 1 -- 172.21.15.1:0/247148288 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fb34401c070 con 0x7fb34c106f10 2024-09-11T21:54:41.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:41.135+0000 7fb3427fc640 1 --2- 172.21.15.1:0/247148288 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fb3380a2b80 0x7fb3380a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:54:41.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:41.135+0000 7fb3427fc640 1 -- 172.21.15.1:0/247148288 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7fb3440ce670 con 0x7fb34c106f10 2024-09-11T21:54:41.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:41.135+0000 7fb35175b640 1 --2- 172.21.15.1:0/247148288 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fb3380a2b80 0x7fb3380a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:54:41.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:41.136+0000 7fb35175b640 1 --2- 172.21.15.1:0/247148288 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fb3380a2b80 0x7fb3380a5040 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb348005ab0 tx=0x7fb348005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:54:41.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:41.139+0000 7fb3427fc640 1 -- 172.21.15.1:0/247148288 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb34409ba70 con 0x7fb34c106f10 2024-09-11T21:54:41.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:41.349+0000 7fb3539bf640 1 -- 172.21.15.1:0/247148288 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb34c02d7c0 con 0x7fb34c106f10 2024-09-11T21:54:41.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:41.349+0000 7fb3427fc640 1 -- 172.21.15.1:0/247148288 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1435 (secure 0 0 0) 0x7fb34409b370 con 0x7fb34c106f10 2024-09-11T21:54:41.350 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:54:41.350 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":2,"btime":"2024-09-11T21:54:38:667041+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/872410024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":872410024},{"type":"v1","addr":"172.21.15.161:6833","nonce":872410024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-11T21:54:41.350 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 2 2024-09-11T21:54:41.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:41.351+0000 7fb3539bf640 1 -- 172.21.15.1:0/247148288 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fb3380a2b80 msgr2=0x7fb3380a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:54:41.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:41.351+0000 7fb3539bf640 1 --2- 172.21.15.1:0/247148288 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fb3380a2b80 0x7fb3380a5040 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb348005ab0 tx=0x7fb348005a20 comp rx=0 tx=0).stop 2024-09-11T21:54:41.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:41.351+0000 7fb3539bf640 1 -- 172.21.15.1:0/247148288 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb34c106f10 msgr2=0x7fb34c19cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:54:41.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:41.351+0000 7fb3539bf640 1 --2- 172.21.15.1:0/247148288 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb34c106f10 0x7fb34c19cd60 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb34400b970 tx=0x7fb34400be30 comp rx=0 tx=0).stop 2024-09-11T21:54:41.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:41.351+0000 7fb3539bf640 1 -- 172.21.15.1:0/247148288 shutdown_connections 2024-09-11T21:54:41.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:41.351+0000 7fb3539bf640 1 --2- 172.21.15.1:0/247148288 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fb3380a2b80 0x7fb3380a5040 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:41.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:41.351+0000 7fb3539bf640 1 --2- 172.21.15.1:0/247148288 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb34c106f10 0x7fb34c19cd60 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:41.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:41.351+0000 7fb3539bf640 1 --2- 172.21.15.1:0/247148288 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb34c102790 0x7fb34c1989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:41.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:41.351+0000 7fb3539bf640 1 --2- 172.21.15.1:0/247148288 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb34c101de0 0x7fb34c198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:41.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:41.351+0000 7fb3539bf640 1 -- 172.21.15.1:0/247148288 >> 172.21.15.1:0/247148288 conn(0x7fb34c0fd8c0 msgr2=0x7fb34c0fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:54:41.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:41.352+0000 7fb3539bf640 1 -- 172.21.15.1:0/247148288 shutdown_connections 2024-09-11T21:54:41.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:41.352+0000 7fb3539bf640 1 -- 172.21.15.1:0/247148288 wait complete. 2024-09-11T21:54:41.399 INFO:journalctl@ceph.mds.k.smithi053.stdout:Sep 11 21:54:41 smithi053 podman[63437]: 2024-09-11 21:54:40.953803658 +0000 UTC m=+0.015908390 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T21:54:41.399 INFO:journalctl@ceph.mds.k.smithi053.stdout:Sep 11 21:54:41 smithi053 podman[63437]: 2024-09-11 21:54:41.07157261 +0000 UTC m=+0.133677372 container create 18b148130eb910259e6b0a67e898f13147ce401da6740458fa48b06bada9a0b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-k, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T21:54:41.399 INFO:journalctl@ceph.mds.k.smithi053.stdout:Sep 11 21:54:41 smithi053 podman[63437]: 2024-09-11 21:54:41.399228344 +0000 UTC m=+0.461333065 container init 18b148130eb910259e6b0a67e898f13147ce401da6740458fa48b06bada9a0b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-k, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T21:54:41.675 INFO:journalctl@ceph.mds.k.smithi053.stdout:Sep 11 21:54:41 smithi053 podman[63437]: 2024-09-11 21:54:41.402345716 +0000 UTC m=+0.464450439 container start 18b148130eb910259e6b0a67e898f13147ce401da6740458fa48b06bada9a0b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-k, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T21:54:41.675 INFO:journalctl@ceph.mds.k.smithi053.stdout:Sep 11 21:54:41 smithi053 ceph-mds[63457]: -- 172.21.15.53:0/2680069147 <== mon.2 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dd2cc15860 con 0x55dd2cbf2800 2024-09-11T21:54:41.675 INFO:journalctl@ceph.mds.k.smithi053.stdout:Sep 11 21:54:41 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-k[63453]: starting mds.k at 2024-09-11T21:54:41.675 INFO:journalctl@ceph.mds.k.smithi053.stdout:Sep 11 21:54:41 smithi053 bash[63437]: 18b148130eb910259e6b0a67e898f13147ce401da6740458fa48b06bada9a0b1 2024-09-11T21:54:41.676 INFO:journalctl@ceph.mds.k.smithi053.stdout:Sep 11 21:54:41 smithi053 systemd[1]: Started Ceph mds.k for 3ffca83c-7087-11ef-bcea-c7b262605968. 2024-09-11T21:54:41.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:41 smithi001 ceph-mon[30935]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:54:41.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:41 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/247148288' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:54:41.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:41 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:41.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:41 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:41.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:41 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:41.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:41 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T21:54:41.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:41 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T21:54:41.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:41 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:54:41.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:41 smithi001 ceph-mon[30935]: Deploying daemon mds.j on smithi001 2024-09-11T21:54:42.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:41 smithi053 ceph-mon[34908]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:54:42.048 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:41 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/247148288' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:54:42.048 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:41 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:42.048 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:41 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:42.048 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:41 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:42.048 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:41 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T21:54:42.048 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:41 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T21:54:42.048 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:41 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:54:42.048 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:41 smithi053 ceph-mon[34908]: Deploying daemon mds.j on smithi001 2024-09-11T21:54:42.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:41 smithi161 ceph-mon[35615]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:54:42.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:41 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/247148288' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:54:42.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:41 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:42.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:41 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:42.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:41 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:42.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:41 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T21:54:42.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:41 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T21:54:42.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:41 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:54:42.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:41 smithi161 ceph-mon[35615]: Deploying daemon mds.j on smithi001 2024-09-11T21:54:42.816 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:42 smithi053 ceph-mon[34908]: mds.? [v2:172.21.15.53:6832/1648944570,v1:172.21.15.53:6833/1648944570] up:boot 2024-09-11T21:54:42.816 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:42 smithi053 ceph-mon[34908]: fsmap 2 up:standby 2024-09-11T21:54:42.817 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:42 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T21:54:42.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:42 smithi001 ceph-mon[30935]: mds.? [v2:172.21.15.53:6832/1648944570,v1:172.21.15.53:6833/1648944570] up:boot 2024-09-11T21:54:42.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:42 smithi001 ceph-mon[30935]: fsmap 2 up:standby 2024-09-11T21:54:42.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:42 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T21:54:43.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:42 smithi161 ceph-mon[35615]: mds.? [v2:172.21.15.53:6832/1648944570,v1:172.21.15.53:6833/1648944570] up:boot 2024-09-11T21:54:43.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:42 smithi161 ceph-mon[35615]: fsmap 2 up:standby 2024-09-11T21:54:43.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:42 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T21:54:43.798 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:43 smithi001 ceph-mon[30935]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:54:44.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:43 smithi053 ceph-mon[34908]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:54:44.091 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:43 smithi161 ceph-mon[35615]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:54:44.193 INFO:journalctl@ceph.mds.j.smithi001.stdout:Sep 11 21:54:44 smithi001 systemd[1]: Starting Ceph mds.j for 3ffca83c-7087-11ef-bcea-c7b262605968... 2024-09-11T21:54:44.201 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T21:54:44.481 INFO:journalctl@ceph.mds.j.smithi001.stdout:Sep 11 21:54:44 smithi001 podman[68113]: 2024-09-11 21:54:44.134755475 +0000 UTC m=+0.016535891 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T21:54:44.481 INFO:journalctl@ceph.mds.j.smithi001.stdout:Sep 11 21:54:44 smithi001 podman[68113]: 2024-09-11 21:54:44.233904433 +0000 UTC m=+0.115684844 container create 7c0ac8cf66ff7cb265a0652f4dd1051db87f60a2eb23d0da76f696e5f7da89b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-j, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-11T21:54:44.481 INFO:journalctl@ceph.mds.j.smithi001.stdout:Sep 11 21:54:44 smithi001 podman[68113]: 2024-09-11 21:54:44.48077511 +0000 UTC m=+0.362555531 container init 7c0ac8cf66ff7cb265a0652f4dd1051db87f60a2eb23d0da76f696e5f7da89b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-j, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-11T21:54:44.656 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:54:44.781 INFO:journalctl@ceph.mds.j.smithi001.stdout:Sep 11 21:54:44 smithi001 podman[68113]: 2024-09-11 21:54:44.484101025 +0000 UTC m=+0.365881446 container start 7c0ac8cf66ff7cb265a0652f4dd1051db87f60a2eb23d0da76f696e5f7da89b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True) 2024-09-11T21:54:44.781 INFO:journalctl@ceph.mds.j.smithi001.stdout:Sep 11 21:54:44 smithi001 ceph-mds[68162]: -- 172.21.15.1:0/3915265300 <== mon.0 v2:172.21.15.1:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558ed0373860 con 0x558ed0351400 2024-09-11T21:54:44.781 INFO:journalctl@ceph.mds.j.smithi001.stdout:Sep 11 21:54:44 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-j[68146]: starting mds.j at 2024-09-11T21:54:44.781 INFO:journalctl@ceph.mds.j.smithi001.stdout:Sep 11 21:54:44 smithi001 bash[68113]: 7c0ac8cf66ff7cb265a0652f4dd1051db87f60a2eb23d0da76f696e5f7da89b7 2024-09-11T21:54:44.781 INFO:journalctl@ceph.mds.j.smithi001.stdout:Sep 11 21:54:44 smithi001 systemd[1]: Started Ceph mds.j for 3ffca83c-7087-11ef-bcea-c7b262605968. 2024-09-11T21:54:45.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:45.184+0000 7f686ef01640 1 -- 172.21.15.1:0/1217137003 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6868102d90 msgr2=0x7f6868103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:54:45.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:45.184+0000 7f686ef01640 1 --2- 172.21.15.1:0/1217137003 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6868102d90 0x7f6868103170 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f685c009930 tx=0x7f685c02a420 comp rx=0 tx=0).stop 2024-09-11T21:54:45.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:45.184+0000 7f686ef01640 1 -- 172.21.15.1:0/1217137003 shutdown_connections 2024-09-11T21:54:45.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:45.184+0000 7f686ef01640 1 --2- 172.21.15.1:0/1217137003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6868108290 0x7f686810ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:45.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:45.184+0000 7f686ef01640 1 --2- 172.21.15.1:0/1217137003 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6868103740 0x7f6868107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:45.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:45.184+0000 7f686ef01640 1 --2- 172.21.15.1:0/1217137003 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6868102d90 0x7f6868103170 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:45.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:45.184+0000 7f686ef01640 1 -- 172.21.15.1:0/1217137003 >> 172.21.15.1:0/1217137003 conn(0x7f68680fc7d0 msgr2=0x7f68680febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:54:45.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:45.184+0000 7f686ef01640 1 -- 172.21.15.1:0/1217137003 shutdown_connections 2024-09-11T21:54:45.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:45.184+0000 7f686ef01640 1 -- 172.21.15.1:0/1217137003 wait complete. 2024-09-11T21:54:45.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:45.185+0000 7f686ef01640 1 Processor -- start 2024-09-11T21:54:45.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:45.185+0000 7f686ef01640 1 -- start start 2024-09-11T21:54:45.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:45.185+0000 7f686ef01640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6868102d90 0x7f6868198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:54:45.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:45.185+0000 7f686ef01640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6868103740 0x7f6868198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:54:45.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:45.185+0000 7f686ef01640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6868108290 0x7f686819cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:54:45.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:45.185+0000 7f686ef01640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6868112090 con 0x7f6868108290 2024-09-11T21:54:45.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:45.185+0000 7f686ef01640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6868111f10 con 0x7f6868102d90 2024-09-11T21:54:45.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:45.185+0000 7f686ef01640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6868112210 con 0x7f6868103740 2024-09-11T21:54:45.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:45.185+0000 7f686d49e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6868108290 0x7f686819cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:54:45.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:45.185+0000 7f686d49e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6868108290 0x7f686819cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:34006/0 (socket says 172.21.15.1:34006) 2024-09-11T21:54:45.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:45.185+0000 7f686d49e640 1 -- 172.21.15.1:0/1613316220 learned_addr learned my addr 172.21.15.1:0/1613316220 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:54:45.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:45.185+0000 7f686cc9d640 1 --2- 172.21.15.1:0/1613316220 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6868102d90 0x7f6868198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:54:45.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:45.185+0000 7f6867fff640 1 --2- 172.21.15.1:0/1613316220 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6868103740 0x7f6868198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:54:45.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:45.185+0000 7f686d49e640 1 -- 172.21.15.1:0/1613316220 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6868103740 msgr2=0x7f6868198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:54:45.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:45.185+0000 7f686d49e640 1 --2- 172.21.15.1:0/1613316220 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6868103740 0x7f6868198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:45.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:45.185+0000 7f686d49e640 1 -- 172.21.15.1:0/1613316220 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6868102d90 msgr2=0x7f6868198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:54:45.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:45.185+0000 7f686d49e640 1 --2- 172.21.15.1:0/1613316220 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6868102d90 0x7f6868198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:45.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:45.185+0000 7f686d49e640 1 -- 172.21.15.1:0/1613316220 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f686819d4c0 con 0x7f6868108290 2024-09-11T21:54:45.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:45.185+0000 7f6867fff640 1 --2- 172.21.15.1:0/1613316220 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6868103740 0x7f6868198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:54:45.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:45.186+0000 7f686d49e640 1 --2- 172.21.15.1:0/1613316220 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6868108290 0x7f686819cde0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f686000cce0 tx=0x7f686000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:54:45.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:45.186+0000 7f6865ffb640 1 -- 172.21.15.1:0/1613316220 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686000f450 con 0x7f6868108290 2024-09-11T21:54:45.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:45.186+0000 7f686ef01640 1 -- 172.21.15.1:0/1613316220 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f686819d7b0 con 0x7f6868108290 2024-09-11T21:54:45.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:45.186+0000 7f6865ffb640 1 -- 172.21.15.1:0/1613316220 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f686000fad0 con 0x7f6868108290 2024-09-11T21:54:45.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:45.186+0000 7f6865ffb640 1 -- 172.21.15.1:0/1613316220 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6860018c10 con 0x7f6868108290 2024-09-11T21:54:45.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:45.186+0000 7f686ef01640 1 -- 172.21.15.1:0/1613316220 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68681a5090 con 0x7f6868108290 2024-09-11T21:54:45.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:45.187+0000 7f686ef01640 1 -- 172.21.15.1:0/1613316220 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f686806a7f0 con 0x7f6868108290 2024-09-11T21:54:45.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:45.188+0000 7f6865ffb640 1 -- 172.21.15.1:0/1613316220 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f686000f5f0 con 0x7f6868108290 2024-09-11T21:54:45.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:45.189+0000 7f6865ffb640 1 --2- 172.21.15.1:0/1613316220 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f68580a2b80 0x7f68580a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:54:45.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:45.189+0000 7f686cc9d640 1 --2- 172.21.15.1:0/1613316220 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f68580a2b80 0x7f68580a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:54:45.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:45.189+0000 7f6865ffb640 1 -- 172.21.15.1:0/1613316220 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f68600cf1d0 con 0x7f6868108290 2024-09-11T21:54:45.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:45.189+0000 7f686cc9d640 1 --2- 172.21.15.1:0/1613316220 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f68580a2b80 0x7f68580a5040 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f685c009900 tx=0x7f685c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:54:45.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:45.193+0000 7f6865ffb640 1 -- 172.21.15.1:0/1613316220 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6860014030 con 0x7f6868108290 2024-09-11T21:54:45.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:45.409+0000 7f686ef01640 1 -- 172.21.15.1:0/1613316220 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f686810a2a0 con 0x7f6868108290 2024-09-11T21:54:45.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:45.410+0000 7f6865ffb640 1 -- 172.21.15.1:0/1613316220 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3079 (secure 0 0 0) 0x7f686009c5d0 con 0x7f6868108290 2024-09-11T21:54:45.411 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:54:45.411 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":4,"btime":"2024-09-11T21:54:44:680881+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/421542832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":421542832},{"type":"v1","addr":"172.21.15.1:6835","nonce":421542832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/872410024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":872410024},{"type":"v1","addr":"172.21.15.161:6833","nonce":872410024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1648944570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1648944570},{"type":"v1","addr":"172.21.15.53:6833","nonce":1648944570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-11T21:54:45.411 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 4 2024-09-11T21:54:45.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:45.412+0000 7f686ef01640 1 -- 172.21.15.1:0/1613316220 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f68580a2b80 msgr2=0x7f68580a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:54:45.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:45.412+0000 7f686ef01640 1 --2- 172.21.15.1:0/1613316220 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f68580a2b80 0x7f68580a5040 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f685c009900 tx=0x7f685c0058d0 comp rx=0 tx=0).stop 2024-09-11T21:54:45.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:45.412+0000 7f686ef01640 1 -- 172.21.15.1:0/1613316220 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6868108290 msgr2=0x7f686819cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:54:45.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:45.412+0000 7f686ef01640 1 --2- 172.21.15.1:0/1613316220 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6868108290 0x7f686819cde0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f686000cce0 tx=0x7f686000a620 comp rx=0 tx=0).stop 2024-09-11T21:54:45.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:45.413+0000 7f686ef01640 1 -- 172.21.15.1:0/1613316220 shutdown_connections 2024-09-11T21:54:45.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:45.413+0000 7f686ef01640 1 --2- 172.21.15.1:0/1613316220 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f68580a2b80 0x7f68580a5040 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:45.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:45.413+0000 7f686ef01640 1 --2- 172.21.15.1:0/1613316220 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6868108290 0x7f686819cde0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:45.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:45.413+0000 7f686ef01640 1 --2- 172.21.15.1:0/1613316220 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6868103740 0x7f6868198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:45.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:45.413+0000 7f686ef01640 1 --2- 172.21.15.1:0/1613316220 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6868102d90 0x7f6868198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:45.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:45.413+0000 7f686ef01640 1 -- 172.21.15.1:0/1613316220 >> 172.21.15.1:0/1613316220 conn(0x7f68680fc7d0 msgr2=0x7f6868105d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:54:45.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:45.413+0000 7f686ef01640 1 -- 172.21.15.1:0/1613316220 shutdown_connections 2024-09-11T21:54:45.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:45.413+0000 7f686ef01640 1 -- 172.21.15.1:0/1613316220 wait complete. 2024-09-11T21:54:45.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:45 smithi001 ceph-mon[30935]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:54:45.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:45 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:45.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:45 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:45.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:45 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:45.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:45 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T21:54:45.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:45 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T21:54:45.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:45 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:54:45.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:45 smithi001 ceph-mon[30935]: Deploying daemon mds.d on smithi001 2024-09-11T21:54:45.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:45 smithi001 ceph-mon[30935]: mds.? [v2:172.21.15.1:6834/421542832,v1:172.21.15.1:6835/421542832] up:boot 2024-09-11T21:54:45.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:45 smithi001 ceph-mon[30935]: fsmap 3 up:standby 2024-09-11T21:54:45.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:45 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-11T21:54:45.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:45 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/1613316220' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:54:46.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:45 smithi053 ceph-mon[34908]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:54:46.048 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:45 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:46.048 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:45 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:46.048 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:45 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:46.048 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:45 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T21:54:46.048 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:45 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T21:54:46.048 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:45 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:54:46.048 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:45 smithi053 ceph-mon[34908]: Deploying daemon mds.d on smithi001 2024-09-11T21:54:46.048 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:45 smithi053 ceph-mon[34908]: mds.? [v2:172.21.15.1:6834/421542832,v1:172.21.15.1:6835/421542832] up:boot 2024-09-11T21:54:46.048 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:45 smithi053 ceph-mon[34908]: fsmap 3 up:standby 2024-09-11T21:54:46.048 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:45 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-11T21:54:46.048 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:45 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/1613316220' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:54:46.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:45 smithi161 ceph-mon[35615]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:54:46.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:45 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:46.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:45 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:46.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:45 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:46.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:45 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T21:54:46.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:45 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T21:54:46.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:45 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:54:46.093 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:45 smithi161 ceph-mon[35615]: Deploying daemon mds.d on smithi001 2024-09-11T21:54:46.093 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:45 smithi161 ceph-mon[35615]: mds.? [v2:172.21.15.1:6834/421542832,v1:172.21.15.1:6835/421542832] up:boot 2024-09-11T21:54:46.093 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:45 smithi161 ceph-mon[35615]: fsmap 3 up:standby 2024-09-11T21:54:46.093 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:45 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-11T21:54:46.093 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:45 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/1613316220' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:54:47.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:47 smithi001 ceph-mon[30935]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:54:47.943 INFO:journalctl@ceph.mds.d.smithi001.stdout:Sep 11 21:54:47 smithi001 systemd[1]: Starting Ceph mds.d for 3ffca83c-7087-11ef-bcea-c7b262605968... 2024-09-11T21:54:48.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:47 smithi053 ceph-mon[34908]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:54:48.091 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:47 smithi161 ceph-mon[35615]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:54:48.193 INFO:journalctl@ceph.mds.d.smithi001.stdout:Sep 11 21:54:47 smithi001 podman[68532]: 2024-09-11 21:54:47.84561866 +0000 UTC m=+0.016587335 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T21:54:48.193 INFO:journalctl@ceph.mds.d.smithi001.stdout:Sep 11 21:54:47 smithi001 podman[68532]: 2024-09-11 21:54:47.976818485 +0000 UTC m=+0.147787194 container create 31b9b50c2867b44155d02bfa2c4a6cff7d99c16bd81796cbb713126450eb852d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-d, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD) 2024-09-11T21:54:48.348 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T21:54:48.621 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:54:48.693 INFO:journalctl@ceph.mds.d.smithi001.stdout:Sep 11 21:54:48 smithi001 podman[68532]: 2024-09-11 21:54:48.237973325 +0000 UTC m=+0.408942009 container init 31b9b50c2867b44155d02bfa2c4a6cff7d99c16bd81796cbb713126450eb852d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-d, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-11T21:54:48.693 INFO:journalctl@ceph.mds.d.smithi001.stdout:Sep 11 21:54:48 smithi001 podman[68532]: 2024-09-11 21:54:48.240809738 +0000 UTC m=+0.411778405 container start 31b9b50c2867b44155d02bfa2c4a6cff7d99c16bd81796cbb713126450eb852d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-d, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-11T21:54:48.693 INFO:journalctl@ceph.mds.d.smithi001.stdout:Sep 11 21:54:48 smithi001 ceph-mds[68551]: -- 172.21.15.1:0/4075729609 <== mon.0 v2:172.21.15.1:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560c0e951860 con 0x560c0e92f000 2024-09-11T21:54:48.694 INFO:journalctl@ceph.mds.d.smithi001.stdout:Sep 11 21:54:48 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-d[68547]: starting mds.d at 2024-09-11T21:54:48.694 INFO:journalctl@ceph.mds.d.smithi001.stdout:Sep 11 21:54:48 smithi001 bash[68532]: 31b9b50c2867b44155d02bfa2c4a6cff7d99c16bd81796cbb713126450eb852d 2024-09-11T21:54:48.694 INFO:journalctl@ceph.mds.d.smithi001.stdout:Sep 11 21:54:48 smithi001 systemd[1]: Started Ceph mds.d for 3ffca83c-7087-11ef-bcea-c7b262605968. 2024-09-11T21:54:49.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:49.117+0000 7f95d2560640 1 -- 172.21.15.1:0/2893258578 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f95cc106f10 msgr2=0x7f95cc10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:54:49.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:49.117+0000 7f95d2560640 1 --2- 172.21.15.1:0/2893258578 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f95cc106f10 0x7f95cc10ea90 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f95c4009ca0 tx=0x7f95c402a8c0 comp rx=0 tx=0).stop 2024-09-11T21:54:49.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:49.117+0000 7f95d2560640 1 -- 172.21.15.1:0/2893258578 shutdown_connections 2024-09-11T21:54:49.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:49.117+0000 7f95d2560640 1 --2- 172.21.15.1:0/2893258578 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f95cc106f10 0x7f95cc10ea90 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:49.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:49.117+0000 7f95d2560640 1 --2- 172.21.15.1:0/2893258578 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95cc102790 0x7f95cc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:49.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:49.117+0000 7f95d2560640 1 --2- 172.21.15.1:0/2893258578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95cc101de0 0x7f95cc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:49.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:49.117+0000 7f95d2560640 1 -- 172.21.15.1:0/2893258578 >> 172.21.15.1:0/2893258578 conn(0x7f95cc0fd8c0 msgr2=0x7f95cc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:54:49.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:49.117+0000 7f95d2560640 1 -- 172.21.15.1:0/2893258578 shutdown_connections 2024-09-11T21:54:49.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:49.117+0000 7f95d2560640 1 -- 172.21.15.1:0/2893258578 wait complete. 2024-09-11T21:54:49.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:49.117+0000 7f95d2560640 1 Processor -- start 2024-09-11T21:54:49.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:49.117+0000 7f95d2560640 1 -- start start 2024-09-11T21:54:49.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:49.118+0000 7f95d2560640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95cc101de0 0x7f95cc198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:54:49.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:49.118+0000 7f95d2560640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f95cc102790 0x7f95cc198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:54:49.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:49.118+0000 7f95d2560640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95cc106f10 0x7f95cc19cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:54:49.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:49.118+0000 7f95d2560640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f95cc111f90 con 0x7f95cc102790 2024-09-11T21:54:49.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:49.118+0000 7f95d2560640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f95cc111e10 con 0x7f95cc101de0 2024-09-11T21:54:49.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:49.118+0000 7f95d2560640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f95cc112110 con 0x7f95cc106f10 2024-09-11T21:54:49.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:49.118+0000 7f95c3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f95cc102790 0x7f95cc198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:54:49.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:49.118+0000 7f95c3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f95cc102790 0x7f95cc198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:34730/0 (socket says 172.21.15.1:34730) 2024-09-11T21:54:49.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:49.118+0000 7f95c3fff640 1 -- 172.21.15.1:0/3723728967 learned_addr learned my addr 172.21.15.1:0/3723728967 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:54:49.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:49.118+0000 7f95d0afd640 1 --2- 172.21.15.1:0/3723728967 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95cc106f10 0x7f95cc19cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:54:49.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:49.118+0000 7f95cbfff640 1 --2- 172.21.15.1:0/3723728967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95cc101de0 0x7f95cc198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:54:49.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:49.118+0000 7f95c3fff640 1 -- 172.21.15.1:0/3723728967 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95cc106f10 msgr2=0x7f95cc19cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:54:49.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:49.118+0000 7f95c3fff640 1 --2- 172.21.15.1:0/3723728967 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95cc106f10 0x7f95cc19cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:49.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:49.118+0000 7f95c3fff640 1 -- 172.21.15.1:0/3723728967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95cc101de0 msgr2=0x7f95cc198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:54:49.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:49.118+0000 7f95c3fff640 1 --2- 172.21.15.1:0/3723728967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95cc101de0 0x7f95cc198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:49.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:49.118+0000 7f95c3fff640 1 -- 172.21.15.1:0/3723728967 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95cc19d3c0 con 0x7f95cc102790 2024-09-11T21:54:49.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:49.118+0000 7f95cbfff640 1 --2- 172.21.15.1:0/3723728967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95cc101de0 0x7f95cc198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:54:49.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:49.118+0000 7f95c3fff640 1 --2- 172.21.15.1:0/3723728967 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f95cc102790 0x7f95cc198950 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f95b400c920 tx=0x7f95b400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:54:49.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:49.118+0000 7f95c9ffb640 1 -- 172.21.15.1:0/3723728967 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b4007c70 con 0x7f95cc102790 2024-09-11T21:54:49.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:49.118+0000 7f95c9ffb640 1 -- 172.21.15.1:0/3723728967 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95b4007e10 con 0x7f95cc102790 2024-09-11T21:54:49.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:49.118+0000 7f95d2560640 1 -- 172.21.15.1:0/3723728967 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95cc19d6b0 con 0x7f95cc102790 2024-09-11T21:54:49.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:49.118+0000 7f95c9ffb640 1 -- 172.21.15.1:0/3723728967 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b4018890 con 0x7f95cc102790 2024-09-11T21:54:49.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:49.118+0000 7f95d2560640 1 -- 172.21.15.1:0/3723728967 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95cc1a4f90 con 0x7f95cc102790 2024-09-11T21:54:49.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:49.119+0000 7f95d2560640 1 -- 172.21.15.1:0/3723728967 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95cc1021c0 con 0x7f95cc102790 2024-09-11T21:54:49.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:49.121+0000 7f95c9ffb640 1 -- 172.21.15.1:0/3723728967 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f95b4018a50 con 0x7f95cc102790 2024-09-11T21:54:49.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:49.122+0000 7f95c9ffb640 1 --2- 172.21.15.1:0/3723728967 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f95b80a2c00 0x7f95b80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:54:49.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:49.122+0000 7f95cbfff640 1 --2- 172.21.15.1:0/3723728967 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f95b80a2c00 0x7f95b80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:54:49.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:49.122+0000 7f95c9ffb640 1 -- 172.21.15.1:0/3723728967 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f95b40cef90 con 0x7f95cc102790 2024-09-11T21:54:49.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:49.122+0000 7f95cbfff640 1 --2- 172.21.15.1:0/3723728967 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f95b80a2c00 0x7f95b80a50c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f95bc009930 tx=0x7f95bc008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:54:49.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:49.125+0000 7f95c9ffb640 1 -- 172.21.15.1:0/3723728967 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f95b4014030 con 0x7f95cc102790 2024-09-11T21:54:49.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:49.338+0000 7f95d2560640 1 -- 172.21.15.1:0/3723728967 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f95cc1a5220 con 0x7f95cc102790 2024-09-11T21:54:49.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:49.339+0000 7f95c9ffb640 1 -- 172.21.15.1:0/3723728967 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3901 (secure 0 0 0) 0x7f95b409c4a0 con 0x7f95cc102790 2024-09-11T21:54:49.339 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:54:49.340 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":5,"btime":"2024-09-11T21:54:48:642321+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/421542832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":421542832},{"type":"v1","addr":"172.21.15.1:6835","nonce":421542832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2836864406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2836864406},{"type":"v1","addr":"172.21.15.1:6837","nonce":2836864406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/872410024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":872410024},{"type":"v1","addr":"172.21.15.161:6833","nonce":872410024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1648944570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1648944570},{"type":"v1","addr":"172.21.15.53:6833","nonce":1648944570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-11T21:54:49.340 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 5 2024-09-11T21:54:49.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:49.340+0000 7f95d2560640 1 -- 172.21.15.1:0/3723728967 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f95b80a2c00 msgr2=0x7f95b80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:54:49.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:49.340+0000 7f95d2560640 1 --2- 172.21.15.1:0/3723728967 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f95b80a2c00 0x7f95b80a50c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f95bc009930 tx=0x7f95bc008040 comp rx=0 tx=0).stop 2024-09-11T21:54:49.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:49.340+0000 7f95d2560640 1 -- 172.21.15.1:0/3723728967 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f95cc102790 msgr2=0x7f95cc198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:54:49.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:49.340+0000 7f95d2560640 1 --2- 172.21.15.1:0/3723728967 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f95cc102790 0x7f95cc198950 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f95b400c920 tx=0x7f95b400cde0 comp rx=0 tx=0).stop 2024-09-11T21:54:49.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:49.341+0000 7f95d2560640 1 -- 172.21.15.1:0/3723728967 shutdown_connections 2024-09-11T21:54:49.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:49.341+0000 7f95d2560640 1 --2- 172.21.15.1:0/3723728967 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f95b80a2c00 0x7f95b80a50c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:49.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:49.341+0000 7f95d2560640 1 --2- 172.21.15.1:0/3723728967 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95cc106f10 0x7f95cc19cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:49.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:49.341+0000 7f95d2560640 1 --2- 172.21.15.1:0/3723728967 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f95cc102790 0x7f95cc198950 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:49.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:49.341+0000 7f95d2560640 1 --2- 172.21.15.1:0/3723728967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95cc101de0 0x7f95cc198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:49.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:49.341+0000 7f95d2560640 1 -- 172.21.15.1:0/3723728967 >> 172.21.15.1:0/3723728967 conn(0x7f95cc0fd8c0 msgr2=0x7f95cc0ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:54:49.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:49.341+0000 7f95d2560640 1 -- 172.21.15.1:0/3723728967 shutdown_connections 2024-09-11T21:54:49.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:49.341+0000 7f95d2560640 1 -- 172.21.15.1:0/3723728967 wait complete. 2024-09-11T21:54:49.682 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:49 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:49.682 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:49 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:49.682 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:49 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:49.682 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:49 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T21:54:49.682 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:49 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T21:54:49.682 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:49 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:54:49.682 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:49 smithi053 ceph-mon[34908]: Deploying daemon mds.e on smithi053 2024-09-11T21:54:49.683 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:49 smithi053 ceph-mon[34908]: mds.? [v2:172.21.15.1:6836/2836864406,v1:172.21.15.1:6837/2836864406] up:boot 2024-09-11T21:54:49.683 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:49 smithi053 ceph-mon[34908]: fsmap 4 up:standby 2024-09-11T21:54:49.683 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:49 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-11T21:54:49.683 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:49 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/3723728967' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:54:49.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:49 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:49.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:49 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:49.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:49 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:49.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:49 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T21:54:49.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:49 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T21:54:49.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:49 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:54:49.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:49 smithi001 ceph-mon[30935]: Deploying daemon mds.e on smithi053 2024-09-11T21:54:49.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:49 smithi001 ceph-mon[30935]: mds.? [v2:172.21.15.1:6836/2836864406,v1:172.21.15.1:6837/2836864406] up:boot 2024-09-11T21:54:49.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:49 smithi001 ceph-mon[30935]: fsmap 4 up:standby 2024-09-11T21:54:49.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:49 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-11T21:54:49.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:49 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/3723728967' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:54:49.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:49 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:49.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:49 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:49.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:49 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:49.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:49 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T21:54:49.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:49 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T21:54:49.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:49 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:54:49.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:49 smithi161 ceph-mon[35615]: Deploying daemon mds.e on smithi053 2024-09-11T21:54:49.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:49 smithi161 ceph-mon[35615]: mds.? [v2:172.21.15.1:6836/2836864406,v1:172.21.15.1:6837/2836864406] up:boot 2024-09-11T21:54:49.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:49 smithi161 ceph-mon[35615]: fsmap 4 up:standby 2024-09-11T21:54:49.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:49 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-11T21:54:49.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:49 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/3723728967' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:54:50.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:50 smithi001 ceph-mon[30935]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:54:50.735 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:50 smithi053 ceph-mon[34908]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:54:50.736 INFO:journalctl@ceph.mds.e.smithi053.stdout:Sep 11 21:54:50 smithi053 systemd[1]: Starting Ceph mds.e for 3ffca83c-7087-11ef-bcea-c7b262605968... 2024-09-11T21:54:50.841 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:50 smithi161 ceph-mon[35615]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:54:50.988 INFO:journalctl@ceph.mds.e.smithi053.stdout:Sep 11 21:54:50 smithi053 podman[63748]: 2024-09-11 21:54:50.73527232 +0000 UTC m=+0.106044374 container create 43bc380734bae6bd44f3327ac5b7a1f2d8df2f9d31d4a7bbaf54341a01bcc6b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-e, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True) 2024-09-11T21:54:50.988 INFO:journalctl@ceph.mds.e.smithi053.stdout:Sep 11 21:54:50 smithi053 podman[63748]: 2024-09-11 21:54:50.645543778 +0000 UTC m=+0.016315849 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T21:54:50.988 INFO:journalctl@ceph.mds.e.smithi053.stdout:Sep 11 21:54:50 smithi053 podman[63748]: 2024-09-11 21:54:50.988004932 +0000 UTC m=+0.358776988 container init 43bc380734bae6bd44f3327ac5b7a1f2d8df2f9d31d4a7bbaf54341a01bcc6b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T21:54:51.297 INFO:journalctl@ceph.mds.e.smithi053.stdout:Sep 11 21:54:50 smithi053 podman[63748]: 2024-09-11 21:54:50.99100246 +0000 UTC m=+0.361774519 container start 43bc380734bae6bd44f3327ac5b7a1f2d8df2f9d31d4a7bbaf54341a01bcc6b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-e, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T21:54:51.298 INFO:journalctl@ceph.mds.e.smithi053.stdout:Sep 11 21:54:51 smithi053 ceph-mds[63767]: -- 172.21.15.53:0/637158706 <== mon.1 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5609ae5ed860 con 0x5609ae5ca800 2024-09-11T21:54:51.298 INFO:journalctl@ceph.mds.e.smithi053.stdout:Sep 11 21:54:51 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-e[63763]: starting mds.e at 2024-09-11T21:54:51.298 INFO:journalctl@ceph.mds.e.smithi053.stdout:Sep 11 21:54:51 smithi053 bash[63748]: 43bc380734bae6bd44f3327ac5b7a1f2d8df2f9d31d4a7bbaf54341a01bcc6b3 2024-09-11T21:54:51.298 INFO:journalctl@ceph.mds.e.smithi053.stdout:Sep 11 21:54:51 smithi053 systemd[1]: Started Ceph mds.e for 3ffca83c-7087-11ef-bcea-c7b262605968. 2024-09-11T21:54:52.158 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T21:54:52.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:52 smithi161 ceph-mon[35615]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:54:52.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:52 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:52.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:52 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:52.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:52 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:52.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:52 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T21:54:52.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:52 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T21:54:52.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:52 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:54:52.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:52 smithi161 ceph-mon[35615]: Deploying daemon mds.f on smithi161 2024-09-11T21:54:52.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:52 smithi161 ceph-mon[35615]: mds.? [v2:172.21.15.53:6834/2126933993,v1:172.21.15.53:6835/2126933993] up:boot 2024-09-11T21:54:52.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:52 smithi161 ceph-mon[35615]: fsmap 5 up:standby 2024-09-11T21:54:52.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:52 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-11T21:54:52.420 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:54:52.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:52 smithi001 ceph-mon[30935]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:54:52.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:52 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:52.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:52 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:52.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:52 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:52.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:52 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T21:54:52.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:52 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T21:54:52.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:52 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:54:52.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:52 smithi001 ceph-mon[30935]: Deploying daemon mds.f on smithi161 2024-09-11T21:54:52.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:52 smithi001 ceph-mon[30935]: mds.? [v2:172.21.15.53:6834/2126933993,v1:172.21.15.53:6835/2126933993] up:boot 2024-09-11T21:54:52.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:52 smithi001 ceph-mon[30935]: fsmap 5 up:standby 2024-09-11T21:54:52.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:52 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-11T21:54:52.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:52 smithi053 ceph-mon[34908]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:54:52.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:52 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:52.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:52 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:52.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:52 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:52.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:52 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T21:54:52.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:52 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T21:54:52.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:52 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:54:52.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:52 smithi053 ceph-mon[34908]: Deploying daemon mds.f on smithi161 2024-09-11T21:54:52.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:52 smithi053 ceph-mon[34908]: mds.? [v2:172.21.15.53:6834/2126933993,v1:172.21.15.53:6835/2126933993] up:boot 2024-09-11T21:54:52.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:52 smithi053 ceph-mon[34908]: fsmap 5 up:standby 2024-09-11T21:54:52.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:52 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-11T21:54:53.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:53.092+0000 7fb0475ce640 1 -- 172.21.15.1:0/3754299366 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb040102d40 msgr2=0x7fb040103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:54:53.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:53.092+0000 7fb0475ce640 1 --2- 172.21.15.1:0/3754299366 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb040102d40 0x7fb040103120 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb03c009900 tx=0x7fb03c02a480 comp rx=0 tx=0).stop 2024-09-11T21:54:53.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:53.092+0000 7fb0475ce640 1 -- 172.21.15.1:0/3754299366 shutdown_connections 2024-09-11T21:54:53.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:53.092+0000 7fb0475ce640 1 --2- 172.21.15.1:0/3754299366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb04010c170 0x7fb04010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:53.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:53.092+0000 7fb0475ce640 1 --2- 172.21.15.1:0/3754299366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb040103660 0x7fb04010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:53.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:53.092+0000 7fb0475ce640 1 --2- 172.21.15.1:0/3754299366 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb040102d40 0x7fb040103120 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:53.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:53.092+0000 7fb0475ce640 1 -- 172.21.15.1:0/3754299366 >> 172.21.15.1:0/3754299366 conn(0x7fb0400fc7a0 msgr2=0x7fb0400febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:54:53.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:53.092+0000 7fb0475ce640 1 -- 172.21.15.1:0/3754299366 shutdown_connections 2024-09-11T21:54:53.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:53.092+0000 7fb0475ce640 1 -- 172.21.15.1:0/3754299366 wait complete. 2024-09-11T21:54:53.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:53.092+0000 7fb0475ce640 1 Processor -- start 2024-09-11T21:54:53.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:53.092+0000 7fb0475ce640 1 -- start start 2024-09-11T21:54:53.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:53.093+0000 7fb0475ce640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb040102d40 0x7fb0401009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:54:53.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:53.093+0000 7fb0475ce640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb040103660 0x7fb040100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:54:53.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:53.093+0000 7fb0475ce640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb04010c170 0x7fb040101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:54:53.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:53.093+0000 7fb0475ce640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb040111f70 con 0x7fb040102d40 2024-09-11T21:54:53.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:53.093+0000 7fb0475ce640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb040111df0 con 0x7fb040103660 2024-09-11T21:54:53.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:53.093+0000 7fb0475ce640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb0401120f0 con 0x7fb04010c170 2024-09-11T21:54:53.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:53.093+0000 7fb04536a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb040102d40 0x7fb0401009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:54:53.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:53.093+0000 7fb04536a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb040102d40 0x7fb0401009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:34748/0 (socket says 172.21.15.1:34748) 2024-09-11T21:54:53.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:53.093+0000 7fb04536a640 1 -- 172.21.15.1:0/1709875900 learned_addr learned my addr 172.21.15.1:0/1709875900 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:54:53.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:53.093+0000 7fb045b6b640 1 --2- 172.21.15.1:0/1709875900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb04010c170 0x7fb040101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:54:53.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:53.093+0000 7fb044b69640 1 --2- 172.21.15.1:0/1709875900 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb040103660 0x7fb040100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:54:53.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:53.093+0000 7fb04536a640 1 -- 172.21.15.1:0/1709875900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb04010c170 msgr2=0x7fb040101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:54:53.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:53.093+0000 7fb04536a640 1 --2- 172.21.15.1:0/1709875900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb04010c170 0x7fb040101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:53.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:53.093+0000 7fb04536a640 1 -- 172.21.15.1:0/1709875900 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb040103660 msgr2=0x7fb040100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:54:53.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:53.093+0000 7fb04536a640 1 --2- 172.21.15.1:0/1709875900 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb040103660 0x7fb040100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:53.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:53.093+0000 7fb04536a640 1 -- 172.21.15.1:0/1709875900 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0401a68f0 con 0x7fb040102d40 2024-09-11T21:54:53.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:53.093+0000 7fb044b69640 1 --2- 172.21.15.1:0/1709875900 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb040103660 0x7fb040100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T21:54:53.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:53.093+0000 7fb04536a640 1 --2- 172.21.15.1:0/1709875900 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb040102d40 0x7fb0401009a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb03c006010 tx=0x7fb03c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:54:53.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:53.093+0000 7fb0367fc640 1 -- 172.21.15.1:0/1709875900 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03c03f030 con 0x7fb040102d40 2024-09-11T21:54:53.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:53.094+0000 7fb0367fc640 1 -- 172.21.15.1:0/1709875900 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb03c003ce0 con 0x7fb040102d40 2024-09-11T21:54:53.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:53.094+0000 7fb0475ce640 1 -- 172.21.15.1:0/1709875900 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0401a6b80 con 0x7fb040102d40 2024-09-11T21:54:53.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:53.094+0000 7fb0367fc640 1 -- 172.21.15.1:0/1709875900 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03c035070 con 0x7fb040102d40 2024-09-11T21:54:53.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:53.094+0000 7fb0475ce640 1 -- 172.21.15.1:0/1709875900 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0401a7060 con 0x7fb040102d40 2024-09-11T21:54:53.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:53.094+0000 7fb0475ce640 1 -- 172.21.15.1:0/1709875900 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb040103120 con 0x7fb040102d40 2024-09-11T21:54:53.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:53.096+0000 7fb0367fc640 1 -- 172.21.15.1:0/1709875900 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fb03c042030 con 0x7fb040102d40 2024-09-11T21:54:53.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:53.096+0000 7fb0367fc640 1 --2- 172.21.15.1:0/1709875900 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fb02c0a2b80 0x7fb02c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:54:53.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:53.097+0000 7fb044b69640 1 --2- 172.21.15.1:0/1709875900 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fb02c0a2b80 0x7fb02c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:54:53.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:53.097+0000 7fb0367fc640 1 -- 172.21.15.1:0/1709875900 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7fb03c0ef400 con 0x7fb040102d40 2024-09-11T21:54:53.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:53.097+0000 7fb044b69640 1 --2- 172.21.15.1:0/1709875900 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fb02c0a2b80 0x7fb02c0a5040 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb040102430 tx=0x7fb0300095d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:54:53.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:53.100+0000 7fb0367fc640 1 -- 172.21.15.1:0/1709875900 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb03c0bc8c0 con 0x7fb040102d40 2024-09-11T21:54:53.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:53.310+0000 7fb0475ce640 1 -- 172.21.15.1:0/1709875900 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb040101ed0 con 0x7fb040102d40 2024-09-11T21:54:53.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:53.311+0000 7fb0367fc640 1 -- 172.21.15.1:0/1709875900 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4726 (secure 0 0 0) 0x7fb040101ed0 con 0x7fb040102d40 2024-09-11T21:54:53.311 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:54:53.311 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":6,"btime":"2024-09-11T21:54:51:398780+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/421542832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":421542832},{"type":"v1","addr":"172.21.15.1:6835","nonce":421542832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2836864406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2836864406},{"type":"v1","addr":"172.21.15.1:6837","nonce":2836864406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/872410024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":872410024},{"type":"v1","addr":"172.21.15.161:6833","nonce":872410024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1648944570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1648944570},{"type":"v1","addr":"172.21.15.53:6833","nonce":1648944570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/2126933993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2126933993},{"type":"v1","addr":"172.21.15.53:6835","nonce":2126933993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-11T21:54:53.312 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 6 2024-09-11T21:54:53.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:53.313+0000 7fb0475ce640 1 -- 172.21.15.1:0/1709875900 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fb02c0a2b80 msgr2=0x7fb02c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:54:53.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:53.313+0000 7fb0475ce640 1 --2- 172.21.15.1:0/1709875900 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fb02c0a2b80 0x7fb02c0a5040 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb040102430 tx=0x7fb0300095d0 comp rx=0 tx=0).stop 2024-09-11T21:54:53.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:53.313+0000 7fb0475ce640 1 -- 172.21.15.1:0/1709875900 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb040102d40 msgr2=0x7fb0401009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:54:53.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:53.313+0000 7fb0475ce640 1 --2- 172.21.15.1:0/1709875900 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb040102d40 0x7fb0401009a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb03c006010 tx=0x7fb03c0042f0 comp rx=0 tx=0).stop 2024-09-11T21:54:53.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:53.313+0000 7fb0475ce640 1 -- 172.21.15.1:0/1709875900 shutdown_connections 2024-09-11T21:54:53.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:53.313+0000 7fb0475ce640 1 --2- 172.21.15.1:0/1709875900 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fb02c0a2b80 0x7fb02c0a5040 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:53.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:53.313+0000 7fb0475ce640 1 --2- 172.21.15.1:0/1709875900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb04010c170 0x7fb040101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:53.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:53.313+0000 7fb0475ce640 1 --2- 172.21.15.1:0/1709875900 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb040103660 0x7fb040100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:53.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:53.313+0000 7fb0475ce640 1 --2- 172.21.15.1:0/1709875900 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb040102d40 0x7fb0401009a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:53.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:53.313+0000 7fb0475ce640 1 -- 172.21.15.1:0/1709875900 >> 172.21.15.1:0/1709875900 conn(0x7fb0400fc7a0 msgr2=0x7fb0400fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:54:53.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:53.313+0000 7fb0475ce640 1 -- 172.21.15.1:0/1709875900 shutdown_connections 2024-09-11T21:54:53.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:53.313+0000 7fb0475ce640 1 -- 172.21.15.1:0/1709875900 wait complete. 2024-09-11T21:54:53.342 INFO:journalctl@ceph.mds.f.smithi161.stdout:Sep 11 21:54:53 smithi161 systemd[1]: Starting Ceph mds.f for 3ffca83c-7087-11ef-bcea-c7b262605968... 2024-09-11T21:54:53.656 INFO:journalctl@ceph.mds.f.smithi161.stdout:Sep 11 21:54:53 smithi161 podman[65346]: 2024-09-11 21:54:53.30879101 +0000 UTC m=+0.016333562 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T21:54:53.656 INFO:journalctl@ceph.mds.f.smithi161.stdout:Sep 11 21:54:53 smithi161 podman[65346]: 2024-09-11 21:54:53.420734847 +0000 UTC m=+0.128277406 container create dc51508f85587e6e1f75facf4de66d2fea202a2c48e31a1af1c69a1676096f87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-f, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T21:54:53.656 INFO:journalctl@ceph.mds.f.smithi161.stdout:Sep 11 21:54:53 smithi161 podman[65346]: 2024-09-11 21:54:53.656603822 +0000 UTC m=+0.364146381 container init dc51508f85587e6e1f75facf4de66d2fea202a2c48e31a1af1c69a1676096f87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-f, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-11T21:54:54.092 INFO:journalctl@ceph.mds.f.smithi161.stdout:Sep 11 21:54:53 smithi161 podman[65346]: 2024-09-11 21:54:53.659384868 +0000 UTC m=+0.366927419 container start dc51508f85587e6e1f75facf4de66d2fea202a2c48e31a1af1c69a1676096f87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-f, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-11T21:54:54.092 INFO:journalctl@ceph.mds.f.smithi161.stdout:Sep 11 21:54:53 smithi161 ceph-mds[65365]: -- 172.21.15.161:0/2690350830 <== mon.1 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564a32857c20 con 0x564a32835400 2024-09-11T21:54:54.092 INFO:journalctl@ceph.mds.f.smithi161.stdout:Sep 11 21:54:53 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-f[65361]: starting mds.f at 2024-09-11T21:54:54.092 INFO:journalctl@ceph.mds.f.smithi161.stdout:Sep 11 21:54:53 smithi161 bash[65346]: dc51508f85587e6e1f75facf4de66d2fea202a2c48e31a1af1c69a1676096f87 2024-09-11T21:54:54.092 INFO:journalctl@ceph.mds.f.smithi161.stdout:Sep 11 21:54:53 smithi161 systemd[1]: Started Ceph mds.f for 3ffca83c-7087-11ef-bcea-c7b262605968. 2024-09-11T21:54:54.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:54 smithi001 ceph-mon[30935]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:54:54.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:54 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/1709875900' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:54:54.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:54 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:54.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:54 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:54.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:54 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:54.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:54 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T21:54:54.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:54 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T21:54:54.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:54 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:54:54.502 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:54 smithi161 ceph-mon[35615]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:54:54.502 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:54 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/1709875900' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:54:54.502 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:54 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:54.502 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:54 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:54.503 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:54 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:54.503 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:54 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T21:54:54.503 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:54 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T21:54:54.503 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:54 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:54:54.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:54 smithi053 ceph-mon[34908]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:54:54.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:54 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/1709875900' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:54:54.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:54 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:54.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:54 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:54.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:54 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:54.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:54 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T21:54:54.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:54 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T21:54:54.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:54 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:54:55.274 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:55 smithi161 ceph-mon[35615]: Deploying daemon mds.c on smithi161 2024-09-11T21:54:55.274 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:55 smithi161 ceph-mon[35615]: mds.? [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] up:boot 2024-09-11T21:54:55.274 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:55 smithi161 ceph-mon[35615]: fsmap 6 up:standby 2024-09-11T21:54:55.274 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:55 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T21:54:55.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:55 smithi001 ceph-mon[30935]: Deploying daemon mds.c on smithi161 2024-09-11T21:54:55.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:55 smithi001 ceph-mon[30935]: mds.? [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] up:boot 2024-09-11T21:54:55.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:55 smithi001 ceph-mon[30935]: fsmap 6 up:standby 2024-09-11T21:54:55.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:55 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T21:54:55.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:55 smithi053 ceph-mon[34908]: Deploying daemon mds.c on smithi161 2024-09-11T21:54:55.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:55 smithi053 ceph-mon[34908]: mds.? [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] up:boot 2024-09-11T21:54:55.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:55 smithi053 ceph-mon[34908]: fsmap 6 up:standby 2024-09-11T21:54:55.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:55 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T21:54:56.002 INFO:journalctl@ceph.mds.c.smithi161.stdout:Sep 11 21:54:55 smithi161 systemd[1]: Starting Ceph mds.c for 3ffca83c-7087-11ef-bcea-c7b262605968... 2024-09-11T21:54:56.114 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T21:54:56.296 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:56 smithi161 ceph-mon[35615]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:54:56.296 INFO:journalctl@ceph.mds.c.smithi161.stdout:Sep 11 21:54:56 smithi161 podman[65648]: 2024-09-11 21:54:55.904758825 +0000 UTC m=+0.016771352 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T21:54:56.296 INFO:journalctl@ceph.mds.c.smithi161.stdout:Sep 11 21:54:56 smithi161 podman[65648]: 2024-09-11 21:54:56.026222053 +0000 UTC m=+0.138234615 container create a2ca22e4e25759fd85e8e5b69fb05c5a85b0a156951178b058779845d4e31e91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-c, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-11T21:54:56.296 INFO:journalctl@ceph.mds.c.smithi161.stdout:Sep 11 21:54:56 smithi161 podman[65648]: 2024-09-11 21:54:56.296112227 +0000 UTC m=+0.408124756 container init a2ca22e4e25759fd85e8e5b69fb05c5a85b0a156951178b058779845d4e31e91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-c, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-09-11T21:54:56.384 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:54:56.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:56 smithi001 ceph-mon[30935]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:54:56.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:56 smithi053 ceph-mon[34908]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:54:56.592 INFO:journalctl@ceph.mds.c.smithi161.stdout:Sep 11 21:54:56 smithi161 podman[65648]: 2024-09-11 21:54:56.30005341 +0000 UTC m=+0.412065931 container start a2ca22e4e25759fd85e8e5b69fb05c5a85b0a156951178b058779845d4e31e91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-c, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-11T21:54:56.592 INFO:journalctl@ceph.mds.c.smithi161.stdout:Sep 11 21:54:56 smithi161 ceph-mds[65667]: -- 172.21.15.161:0/2356422684 <== mon.1 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559a59357860 con 0x559a59335000 2024-09-11T21:54:56.592 INFO:journalctl@ceph.mds.c.smithi161.stdout:Sep 11 21:54:56 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-c[65663]: starting mds.c at 2024-09-11T21:54:56.592 INFO:journalctl@ceph.mds.c.smithi161.stdout:Sep 11 21:54:56 smithi161 bash[65648]: a2ca22e4e25759fd85e8e5b69fb05c5a85b0a156951178b058779845d4e31e91 2024-09-11T21:54:56.592 INFO:journalctl@ceph.mds.c.smithi161.stdout:Sep 11 21:54:56 smithi161 systemd[1]: Started Ceph mds.c for 3ffca83c-7087-11ef-bcea-c7b262605968. 2024-09-11T21:54:56.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:56.875+0000 7fc92ebbb640 1 -- 172.21.15.1:0/2284910744 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc928106f10 msgr2=0x7fc92810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:54:56.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:56.875+0000 7fc92ebbb640 1 --2- 172.21.15.1:0/2284910744 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc928106f10 0x7fc92810ea90 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc920009ca0 tx=0x7fc92002a8c0 comp rx=0 tx=0).stop 2024-09-11T21:54:56.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:56.875+0000 7fc92ebbb640 1 -- 172.21.15.1:0/2284910744 shutdown_connections 2024-09-11T21:54:56.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:56.875+0000 7fc92ebbb640 1 --2- 172.21.15.1:0/2284910744 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc928106f10 0x7fc92810ea90 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:56.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:56.875+0000 7fc92ebbb640 1 --2- 172.21.15.1:0/2284910744 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc928102790 0x7fc9281067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:56.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:56.875+0000 7fc92ebbb640 1 --2- 172.21.15.1:0/2284910744 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc928101de0 0x7fc9281021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:56.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:56.875+0000 7fc92ebbb640 1 -- 172.21.15.1:0/2284910744 >> 172.21.15.1:0/2284910744 conn(0x7fc9280fd8c0 msgr2=0x7fc9280ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:54:56.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:56.875+0000 7fc92ebbb640 1 -- 172.21.15.1:0/2284910744 shutdown_connections 2024-09-11T21:54:56.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:56.875+0000 7fc92ebbb640 1 -- 172.21.15.1:0/2284910744 wait complete. 2024-09-11T21:54:56.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:56.876+0000 7fc92ebbb640 1 Processor -- start 2024-09-11T21:54:56.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:56.876+0000 7fc92ebbb640 1 -- start start 2024-09-11T21:54:56.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:56.876+0000 7fc92ebbb640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc928101de0 0x7fc928198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:54:56.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:56.876+0000 7fc92ebbb640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc928102790 0x7fc928198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:54:56.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:56.876+0000 7fc92ebbb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc928106f10 0x7fc92819cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:54:56.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:56.876+0000 7fc92ebbb640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc928111f70 con 0x7fc928102790 2024-09-11T21:54:56.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:56.876+0000 7fc92ebbb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc928111df0 con 0x7fc928106f10 2024-09-11T21:54:56.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:56.876+0000 7fc92ebbb640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc9281120f0 con 0x7fc928101de0 2024-09-11T21:54:56.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:56.876+0000 7fc91ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc928102790 0x7fc928198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:54:56.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:56.876+0000 7fc91ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc928102790 0x7fc928198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47862/0 (socket says 172.21.15.1:47862) 2024-09-11T21:54:56.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:56.876+0000 7fc91ffff640 1 -- 172.21.15.1:0/1278514150 learned_addr learned my addr 172.21.15.1:0/1278514150 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:54:56.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:56.876+0000 7fc92c957640 1 --2- 172.21.15.1:0/1278514150 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc928101de0 0x7fc928198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:54:56.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:56.876+0000 7fc92d158640 1 --2- 172.21.15.1:0/1278514150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc928106f10 0x7fc92819cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:54:56.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:56.877+0000 7fc91ffff640 1 -- 172.21.15.1:0/1278514150 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc928101de0 msgr2=0x7fc928198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:54:56.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:56.877+0000 7fc91ffff640 1 --2- 172.21.15.1:0/1278514150 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc928101de0 0x7fc928198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:56.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:56.877+0000 7fc91ffff640 1 -- 172.21.15.1:0/1278514150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc928106f10 msgr2=0x7fc92819cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:54:56.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:56.877+0000 7fc91ffff640 1 --2- 172.21.15.1:0/1278514150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc928106f10 0x7fc92819cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:56.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:56.877+0000 7fc91ffff640 1 -- 172.21.15.1:0/1278514150 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc92819d2e0 con 0x7fc928102790 2024-09-11T21:54:56.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:56.877+0000 7fc91ffff640 1 --2- 172.21.15.1:0/1278514150 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc928102790 0x7fc928198870 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc91800d950 tx=0x7fc91800dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:54:56.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:56.877+0000 7fc91dffb640 1 -- 172.21.15.1:0/1278514150 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc918017030 con 0x7fc928102790 2024-09-11T21:54:56.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:56.877+0000 7fc92ebbb640 1 -- 172.21.15.1:0/1278514150 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc92819d5d0 con 0x7fc928102790 2024-09-11T21:54:56.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:56.877+0000 7fc91dffb640 1 -- 172.21.15.1:0/1278514150 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc918004020 con 0x7fc928102790 2024-09-11T21:54:56.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:56.877+0000 7fc91dffb640 1 -- 172.21.15.1:0/1278514150 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc918005650 con 0x7fc928102790 2024-09-11T21:54:56.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:56.877+0000 7fc92ebbb640 1 -- 172.21.15.1:0/1278514150 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9281a4eb0 con 0x7fc928102790 2024-09-11T21:54:56.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:56.878+0000 7fc92ebbb640 1 -- 172.21.15.1:0/1278514150 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9281021c0 con 0x7fc928102790 2024-09-11T21:54:56.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:56.879+0000 7fc91dffb640 1 -- 172.21.15.1:0/1278514150 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fc9180041d0 con 0x7fc928102790 2024-09-11T21:54:56.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:56.880+0000 7fc91dffb640 1 --2- 172.21.15.1:0/1278514150 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fc9100a2b80 0x7fc9100a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:54:56.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:56.880+0000 7fc92c957640 1 --2- 172.21.15.1:0/1278514150 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fc9100a2b80 0x7fc9100a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:54:56.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:56.880+0000 7fc91dffb640 1 -- 172.21.15.1:0/1278514150 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7fc9180ce520 con 0x7fc928102790 2024-09-11T21:54:56.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:56.880+0000 7fc92c957640 1 --2- 172.21.15.1:0/1278514150 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fc9100a2b80 0x7fc9100a5040 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc92400abe0 tx=0x7fc92400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:54:56.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:56.884+0000 7fc91dffb640 1 -- 172.21.15.1:0/1278514150 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc91809b880 con 0x7fc928102790 2024-09-11T21:54:57.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:57.095+0000 7fc92ebbb640 1 -- 172.21.15.1:0/1278514150 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc92810a0f0 con 0x7fc928102790 2024-09-11T21:54:57.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:57.095+0000 7fc91dffb640 1 -- 172.21.15.1:0/1278514150 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5551 (secure 0 0 0) 0x7fc91809b180 con 0x7fc928102790 2024-09-11T21:54:57.096 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:54:57.096 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":7,"btime":"2024-09-11T21:54:54:138610+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/421542832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":421542832},{"type":"v1","addr":"172.21.15.1:6835","nonce":421542832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2836864406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2836864406},{"type":"v1","addr":"172.21.15.1:6837","nonce":2836864406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/872410024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":872410024},{"type":"v1","addr":"172.21.15.161:6833","nonce":872410024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1648944570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1648944570},{"type":"v1","addr":"172.21.15.53:6833","nonce":1648944570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/2126933993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2126933993},{"type":"v1","addr":"172.21.15.53:6835","nonce":2126933993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/828381592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":828381592},{"type":"v1","addr":"172.21.15.161:6835","nonce":828381592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-09-11T21:54:57.096 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 7 2024-09-11T21:54:57.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:57.097+0000 7fc92ebbb640 1 -- 172.21.15.1:0/1278514150 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fc9100a2b80 msgr2=0x7fc9100a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:54:57.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:57.097+0000 7fc92ebbb640 1 --2- 172.21.15.1:0/1278514150 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fc9100a2b80 0x7fc9100a5040 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc92400abe0 tx=0x7fc92400a2d0 comp rx=0 tx=0).stop 2024-09-11T21:54:57.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:57.097+0000 7fc92ebbb640 1 -- 172.21.15.1:0/1278514150 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc928102790 msgr2=0x7fc928198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:54:57.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:57.097+0000 7fc92ebbb640 1 --2- 172.21.15.1:0/1278514150 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc928102790 0x7fc928198870 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc91800d950 tx=0x7fc91800dd50 comp rx=0 tx=0).stop 2024-09-11T21:54:57.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:57.097+0000 7fc92ebbb640 1 -- 172.21.15.1:0/1278514150 shutdown_connections 2024-09-11T21:54:57.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:57.097+0000 7fc92ebbb640 1 --2- 172.21.15.1:0/1278514150 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fc9100a2b80 0x7fc9100a5040 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:57.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:57.097+0000 7fc92ebbb640 1 --2- 172.21.15.1:0/1278514150 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc928106f10 0x7fc92819cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:57.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:57.097+0000 7fc92ebbb640 1 --2- 172.21.15.1:0/1278514150 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc928102790 0x7fc928198870 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:57.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:57.097+0000 7fc92ebbb640 1 --2- 172.21.15.1:0/1278514150 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc928101de0 0x7fc928198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:54:57.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:57.097+0000 7fc92ebbb640 1 -- 172.21.15.1:0/1278514150 >> 172.21.15.1:0/1278514150 conn(0x7fc9280fd8c0 msgr2=0x7fc9280febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:54:57.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:57.098+0000 7fc92ebbb640 1 -- 172.21.15.1:0/1278514150 shutdown_connections 2024-09-11T21:54:57.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:54:57.098+0000 7fc92ebbb640 1 -- 172.21.15.1:0/1278514150 wait complete. 2024-09-11T21:54:57.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:57 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:57.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:57 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:57.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:57 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:57.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:57 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T21:54:57.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:57 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T21:54:57.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:57 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:54:57.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:57 smithi053 ceph-mon[34908]: Deploying daemon mds.h on smithi053 2024-09-11T21:54:57.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:57 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/1278514150' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:54:57.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:57 smithi053 ceph-mon[34908]: mds.? [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] up:boot 2024-09-11T21:54:57.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:57 smithi053 ceph-mon[34908]: fsmap 7 up:standby 2024-09-11T21:54:57.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:57 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-11T21:54:57.783 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:57 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:57.783 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:57 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:57.783 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:57 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:57.783 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:57 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T21:54:57.783 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:57 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T21:54:57.783 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:57 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:54:57.783 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:57 smithi001 ceph-mon[30935]: Deploying daemon mds.h on smithi053 2024-09-11T21:54:57.783 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:57 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/1278514150' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:54:57.783 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:57 smithi001 ceph-mon[30935]: mds.? [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] up:boot 2024-09-11T21:54:57.784 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:57 smithi001 ceph-mon[30935]: fsmap 7 up:standby 2024-09-11T21:54:57.784 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:57 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-11T21:54:57.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:57 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:57.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:57 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:57.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:57 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:54:57.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:57 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T21:54:57.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:57 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T21:54:57.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:57 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:54:57.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:57 smithi161 ceph-mon[35615]: Deploying daemon mds.h on smithi053 2024-09-11T21:54:57.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:57 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/1278514150' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:54:57.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:57 smithi161 ceph-mon[35615]: mds.? [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] up:boot 2024-09-11T21:54:57.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:57 smithi161 ceph-mon[35615]: fsmap 7 up:standby 2024-09-11T21:54:57.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:57 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-11T21:54:58.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:54:58 smithi053 ceph-mon[34908]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:54:58.798 INFO:journalctl@ceph.mds.h.smithi053.stdout:Sep 11 21:54:58 smithi053 systemd[1]: Starting Ceph mds.h for 3ffca83c-7087-11ef-bcea-c7b262605968... 2024-09-11T21:54:58.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:54:58 smithi161 ceph-mon[35615]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:54:58.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:54:58 smithi001 ceph-mon[30935]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:54:59.111 INFO:journalctl@ceph.mds.h.smithi053.stdout:Sep 11 21:54:58 smithi053 podman[64049]: 2024-09-11 21:54:58.704895015 +0000 UTC m=+0.016172654 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T21:54:59.111 INFO:journalctl@ceph.mds.h.smithi053.stdout:Sep 11 21:54:58 smithi053 podman[64049]: 2024-09-11 21:54:58.8415919 +0000 UTC m=+0.152869565 container create 84569b980f2d2d0c3c8221ccebaa3469e7e744a0fd3eab0c353314de2699ce90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-h, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-11T21:54:59.111 INFO:journalctl@ceph.mds.h.smithi053.stdout:Sep 11 21:54:59 smithi053 podman[64049]: 2024-09-11 21:54:59.111174121 +0000 UTC m=+0.422451741 container init 84569b980f2d2d0c3c8221ccebaa3469e7e744a0fd3eab0c353314de2699ce90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-h, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True) 2024-09-11T21:54:59.548 INFO:journalctl@ceph.mds.h.smithi053.stdout:Sep 11 21:54:59 smithi053 podman[64049]: 2024-09-11 21:54:59.114277006 +0000 UTC m=+0.425554632 container start 84569b980f2d2d0c3c8221ccebaa3469e7e744a0fd3eab0c353314de2699ce90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-h, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T21:54:59.548 INFO:journalctl@ceph.mds.h.smithi053.stdout:Sep 11 21:54:59 smithi053 ceph-mds[64067]: -- 172.21.15.53:0/292384378 <== mon.2 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5578ddf25860 con 0x5578ddf03000 2024-09-11T21:54:59.548 INFO:journalctl@ceph.mds.h.smithi053.stdout:Sep 11 21:54:59 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-h[64063]: starting mds.h at 2024-09-11T21:54:59.548 INFO:journalctl@ceph.mds.h.smithi053.stdout:Sep 11 21:54:59 smithi053 bash[64049]: 84569b980f2d2d0c3c8221ccebaa3469e7e744a0fd3eab0c353314de2699ce90 2024-09-11T21:54:59.548 INFO:journalctl@ceph.mds.h.smithi053.stdout:Sep 11 21:54:59 smithi053 systemd[1]: Started Ceph mds.h for 3ffca83c-7087-11ef-bcea-c7b262605968. 2024-09-11T21:54:59.814 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T21:55:00.207 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:55:00.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:00 smithi161 ceph-mon[35615]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:55:00.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:00 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:00.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:00 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:00.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:00 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:00.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:00 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T21:55:00.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:00 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T21:55:00.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:00 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:55:00.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:00 smithi161 ceph-mon[35615]: Deploying daemon mds.i on smithi161 2024-09-11T21:55:00.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:00 smithi161 ceph-mon[35615]: mds.? [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] up:boot 2024-09-11T21:55:00.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:00 smithi161 ceph-mon[35615]: fsmap 8 up:standby 2024-09-11T21:55:00.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:00 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-11T21:55:00.517 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:00 smithi001 ceph-mon[30935]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:55:00.517 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:00 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:00.517 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:00 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:00.517 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:00 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:00.517 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:00 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T21:55:00.518 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:00 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T21:55:00.518 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:00 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:55:00.518 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:00 smithi001 ceph-mon[30935]: Deploying daemon mds.i on smithi161 2024-09-11T21:55:00.518 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:00 smithi001 ceph-mon[30935]: mds.? [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] up:boot 2024-09-11T21:55:00.518 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:00 smithi001 ceph-mon[30935]: fsmap 8 up:standby 2024-09-11T21:55:00.518 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:00 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-11T21:55:00.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:00 smithi053 ceph-mon[34908]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:55:00.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:00 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:00.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:00 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:00.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:00 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:00.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:00 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T21:55:00.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:00 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T21:55:00.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:00 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:55:00.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:00 smithi053 ceph-mon[34908]: Deploying daemon mds.i on smithi161 2024-09-11T21:55:00.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:00 smithi053 ceph-mon[34908]: mds.? [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] up:boot 2024-09-11T21:55:00.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:00 smithi053 ceph-mon[34908]: fsmap 8 up:standby 2024-09-11T21:55:00.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:00 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-11T21:55:00.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:00.692+0000 7f30f10ee640 1 -- 172.21.15.1:0/2023312655 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f30ec101420 msgr2=0x7f30ec10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:00.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:00.692+0000 7f30f10ee640 1 --2- 172.21.15.1:0/2023312655 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f30ec101420 0x7f30ec10d950 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f30e4009cd0 tx=0x7f30e402a8e0 comp rx=0 tx=0).stop 2024-09-11T21:55:00.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:00.692+0000 7f30f10ee640 1 -- 172.21.15.1:0/2023312655 shutdown_connections 2024-09-11T21:55:00.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:00.692+0000 7f30f10ee640 1 --2- 172.21.15.1:0/2023312655 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f30ec101420 0x7f30ec10d950 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:00.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:00.692+0000 7f30f10ee640 1 --2- 172.21.15.1:0/2023312655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30ec100a80 0x7f30ec100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:00.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:00.692+0000 7f30f10ee640 1 --2- 172.21.15.1:0/2023312655 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f30ec106a80 0x7f30ec106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:00.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:00.692+0000 7f30f10ee640 1 -- 172.21.15.1:0/2023312655 >> 172.21.15.1:0/2023312655 conn(0x7f30ec0fc7a0 msgr2=0x7f30ec0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:55:00.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:00.692+0000 7f30f10ee640 1 -- 172.21.15.1:0/2023312655 shutdown_connections 2024-09-11T21:55:00.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:00.692+0000 7f30f10ee640 1 -- 172.21.15.1:0/2023312655 wait complete. 2024-09-11T21:55:00.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:00.693+0000 7f30f10ee640 1 Processor -- start 2024-09-11T21:55:00.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:00.693+0000 7f30f10ee640 1 -- start start 2024-09-11T21:55:00.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:00.693+0000 7f30f10ee640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30ec100a80 0x7f30ec198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:00.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:00.693+0000 7f30f10ee640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f30ec101420 0x7f30ec1988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:00.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:00.693+0000 7f30f10ee640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f30ec106a80 0x7f30ec19cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:00.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:00.693+0000 7f30f10ee640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f30ec111fa0 con 0x7f30ec106a80 2024-09-11T21:55:00.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:00.693+0000 7f30f10ee640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f30ec111e20 con 0x7f30ec100a80 2024-09-11T21:55:00.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:00.693+0000 7f30f10ee640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f30ec112120 con 0x7f30ec101420 2024-09-11T21:55:00.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:00.694+0000 7f30eb59e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f30ec106a80 0x7f30ec19cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:00.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:00.694+0000 7f30eb59e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f30ec106a80 0x7f30ec19cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47880/0 (socket says 172.21.15.1:47880) 2024-09-11T21:55:00.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:00.694+0000 7f30eb59e640 1 -- 172.21.15.1:0/2786916367 learned_addr learned my addr 172.21.15.1:0/2786916367 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:55:00.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:00.694+0000 7f30ea59c640 1 --2- 172.21.15.1:0/2786916367 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f30ec101420 0x7f30ec1988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:00.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:00.694+0000 7f30ead9d640 1 --2- 172.21.15.1:0/2786916367 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30ec100a80 0x7f30ec198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:00.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:00.694+0000 7f30eb59e640 1 -- 172.21.15.1:0/2786916367 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f30ec101420 msgr2=0x7f30ec1988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:00.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:00.694+0000 7f30eb59e640 1 --2- 172.21.15.1:0/2786916367 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f30ec101420 0x7f30ec1988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:00.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:00.694+0000 7f30eb59e640 1 -- 172.21.15.1:0/2786916367 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30ec100a80 msgr2=0x7f30ec198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:00.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:00.694+0000 7f30eb59e640 1 --2- 172.21.15.1:0/2786916367 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30ec100a80 0x7f30ec198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:00.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:00.694+0000 7f30eb59e640 1 -- 172.21.15.1:0/2786916367 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30ec19d340 con 0x7f30ec106a80 2024-09-11T21:55:00.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:00.694+0000 7f30ead9d640 1 --2- 172.21.15.1:0/2786916367 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30ec100a80 0x7f30ec198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:55:00.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:00.694+0000 7f30eb59e640 1 --2- 172.21.15.1:0/2786916367 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f30ec106a80 0x7f30ec19cc60 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f30e4005cc0 tx=0x7f30e4003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:55:00.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:00.694+0000 7f30e3fff640 1 -- 172.21.15.1:0/2786916367 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e4004040 con 0x7f30ec106a80 2024-09-11T21:55:00.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:00.694+0000 7f30f10ee640 1 -- 172.21.15.1:0/2786916367 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30ec19d5d0 con 0x7f30ec106a80 2024-09-11T21:55:00.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:00.694+0000 7f30e3fff640 1 -- 172.21.15.1:0/2786916367 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30e40046c0 con 0x7f30ec106a80 2024-09-11T21:55:00.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:00.694+0000 7f30e3fff640 1 -- 172.21.15.1:0/2786916367 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e4030e70 con 0x7f30ec106a80 2024-09-11T21:55:00.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:00.694+0000 7f30f10ee640 1 -- 172.21.15.1:0/2786916367 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30ec1a4f10 con 0x7f30ec106a80 2024-09-11T21:55:00.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:00.696+0000 7f30f10ee640 1 -- 172.21.15.1:0/2786916367 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30ec100ee0 con 0x7f30ec106a80 2024-09-11T21:55:00.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:00.696+0000 7f30e3fff640 1 -- 172.21.15.1:0/2786916367 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f30e40305a0 con 0x7f30ec106a80 2024-09-11T21:55:00.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:00.697+0000 7f30e3fff640 1 --2- 172.21.15.1:0/2786916367 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f30d80a2b80 0x7f30d80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:00.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:00.697+0000 7f30ead9d640 1 --2- 172.21.15.1:0/2786916367 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f30d80a2b80 0x7f30d80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:00.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:00.697+0000 7f30e3fff640 1 -- 172.21.15.1:0/2786916367 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f30e402e030 con 0x7f30ec106a80 2024-09-11T21:55:00.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:00.698+0000 7f30ead9d640 1 --2- 172.21.15.1:0/2786916367 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f30d80a2b80 0x7f30d80a5040 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f30dc005fd0 tx=0x7f30dc00b5d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:55:00.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:00.701+0000 7f30e3fff640 1 -- 172.21.15.1:0/2786916367 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f30e40098b0 con 0x7f30ec106a80 2024-09-11T21:55:00.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:00.913+0000 7f30f10ee640 1 -- 172.21.15.1:0/2786916367 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f30ec1a5140 con 0x7f30ec106a80 2024-09-11T21:55:00.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:00.914+0000 7f30e3fff640 1 -- 172.21.15.1:0/2786916367 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7204 (secure 0 0 0) 0x7f30e40bd0c0 con 0x7f30ec106a80 2024-09-11T21:55:00.915 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:55:00.915 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":9,"btime":"2024-09-11T21:54:59:491736+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/421542832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":421542832},{"type":"v1","addr":"172.21.15.1:6835","nonce":421542832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2836864406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2836864406},{"type":"v1","addr":"172.21.15.1:6837","nonce":2836864406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/872410024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":872410024},{"type":"v1","addr":"172.21.15.161:6833","nonce":872410024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1648944570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1648944570},{"type":"v1","addr":"172.21.15.53:6833","nonce":1648944570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/2126933993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2126933993},{"type":"v1","addr":"172.21.15.53:6835","nonce":2126933993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/828381592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":828381592},{"type":"v1","addr":"172.21.15.161:6835","nonce":828381592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/1799318656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1799318656},{"type":"v1","addr":"172.21.15.161:6837","nonce":1799318656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24467,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/3457131830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3457131830},{"type":"v1","addr":"172.21.15.53:6837","nonce":3457131830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-11T21:55:00.915 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 9 2024-09-11T21:55:00.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:00.916+0000 7f30f10ee640 1 -- 172.21.15.1:0/2786916367 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f30d80a2b80 msgr2=0x7f30d80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:00.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:00.916+0000 7f30f10ee640 1 --2- 172.21.15.1:0/2786916367 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f30d80a2b80 0x7f30d80a5040 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f30dc005fd0 tx=0x7f30dc00b5d0 comp rx=0 tx=0).stop 2024-09-11T21:55:00.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:00.916+0000 7f30f10ee640 1 -- 172.21.15.1:0/2786916367 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f30ec106a80 msgr2=0x7f30ec19cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:00.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:00.916+0000 7f30f10ee640 1 --2- 172.21.15.1:0/2786916367 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f30ec106a80 0x7f30ec19cc60 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f30e4005cc0 tx=0x7f30e4003970 comp rx=0 tx=0).stop 2024-09-11T21:55:00.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:00.917+0000 7f30f10ee640 1 -- 172.21.15.1:0/2786916367 shutdown_connections 2024-09-11T21:55:00.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:00.917+0000 7f30f10ee640 1 --2- 172.21.15.1:0/2786916367 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f30d80a2b80 0x7f30d80a5040 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:00.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:00.917+0000 7f30f10ee640 1 --2- 172.21.15.1:0/2786916367 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f30ec106a80 0x7f30ec19cc60 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:00.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:00.917+0000 7f30f10ee640 1 --2- 172.21.15.1:0/2786916367 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f30ec101420 0x7f30ec1988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:00.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:00.917+0000 7f30f10ee640 1 --2- 172.21.15.1:0/2786916367 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30ec100a80 0x7f30ec198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:00.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:00.917+0000 7f30f10ee640 1 -- 172.21.15.1:0/2786916367 >> 172.21.15.1:0/2786916367 conn(0x7f30ec0fc7a0 msgr2=0x7f30ec0fceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:55:00.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:00.917+0000 7f30f10ee640 1 -- 172.21.15.1:0/2786916367 shutdown_connections 2024-09-11T21:55:00.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:00.917+0000 7f30f10ee640 1 -- 172.21.15.1:0/2786916367 wait complete. 2024-09-11T21:55:01.268 INFO:journalctl@ceph.mds.i.smithi161.stdout:Sep 11 21:55:01 smithi161 systemd[1]: Starting Ceph mds.i for 3ffca83c-7087-11ef-bcea-c7b262605968... 2024-09-11T21:55:01.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:01 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/2786916367' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:55:01.552 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:01 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/2786916367' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:55:01.552 INFO:journalctl@ceph.mds.i.smithi161.stdout:Sep 11 21:55:01 smithi161 podman[65950]: 2024-09-11 21:55:01.294839716 +0000 UTC m=+0.016558984 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T21:55:01.553 INFO:journalctl@ceph.mds.i.smithi161.stdout:Sep 11 21:55:01 smithi161 podman[65950]: 2024-09-11 21:55:01.423983003 +0000 UTC m=+0.145702261 container create 55301ac4334fa3975f19c228a27daaea647193dc14a470c95c65e01d3544ed70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-i, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T21:55:01.592 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:01 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/2786916367' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:55:02.092 INFO:journalctl@ceph.mds.i.smithi161.stdout:Sep 11 21:55:01 smithi161 podman[65950]: 2024-09-11 21:55:01.67655502 +0000 UTC m=+0.398274283 container init 55301ac4334fa3975f19c228a27daaea647193dc14a470c95c65e01d3544ed70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-i, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-11T21:55:02.092 INFO:journalctl@ceph.mds.i.smithi161.stdout:Sep 11 21:55:01 smithi161 podman[65950]: 2024-09-11 21:55:01.679430961 +0000 UTC m=+0.401150228 container start 55301ac4334fa3975f19c228a27daaea647193dc14a470c95c65e01d3544ed70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-i, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731) 2024-09-11T21:55:02.092 INFO:journalctl@ceph.mds.i.smithi161.stdout:Sep 11 21:55:01 smithi161 ceph-mds[65969]: -- 172.21.15.161:0/3442621019 <== mon.1 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556e1f217860 con 0x556e1f1f5400 2024-09-11T21:55:02.092 INFO:journalctl@ceph.mds.i.smithi161.stdout:Sep 11 21:55:01 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-i[65965]: starting mds.i at 2024-09-11T21:55:02.092 INFO:journalctl@ceph.mds.i.smithi161.stdout:Sep 11 21:55:01 smithi161 bash[65950]: 55301ac4334fa3975f19c228a27daaea647193dc14a470c95c65e01d3544ed70 2024-09-11T21:55:02.092 INFO:journalctl@ceph.mds.i.smithi161.stdout:Sep 11 21:55:01 smithi161 systemd[1]: Started Ceph mds.i for 3ffca83c-7087-11ef-bcea-c7b262605968. 2024-09-11T21:55:02.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:02 smithi053 ceph-mon[34908]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:55:02.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:02 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:02.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:02 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:02.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:02 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:02.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:02 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T21:55:02.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:02 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T21:55:02.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:02 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:55:02.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:02 smithi161 ceph-mon[35615]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:55:02.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:02 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:02.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:02 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:02.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:02 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:02.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:02 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T21:55:02.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:02 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T21:55:02.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:02 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:55:02.604 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:02 smithi001 ceph-mon[30935]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:55:02.604 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:02 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:02.604 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:02 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:02.605 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:02 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:02.605 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:02 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T21:55:02.605 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:02 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T21:55:02.605 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:02 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:55:03.376 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:03 smithi001 ceph-mon[30935]: Deploying daemon mds.g on smithi001 2024-09-11T21:55:03.376 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:03 smithi001 ceph-mon[30935]: mds.? [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] up:boot 2024-09-11T21:55:03.376 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:03 smithi001 ceph-mon[30935]: fsmap 9 up:standby 2024-09-11T21:55:03.376 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:03 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-11T21:55:03.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:03 smithi053 ceph-mon[34908]: Deploying daemon mds.g on smithi001 2024-09-11T21:55:03.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:03 smithi053 ceph-mon[34908]: mds.? [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] up:boot 2024-09-11T21:55:03.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:03 smithi053 ceph-mon[34908]: fsmap 9 up:standby 2024-09-11T21:55:03.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:03 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-11T21:55:03.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:03 smithi161 ceph-mon[35615]: Deploying daemon mds.g on smithi001 2024-09-11T21:55:03.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:03 smithi161 ceph-mon[35615]: mds.? [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] up:boot 2024-09-11T21:55:03.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:03 smithi161 ceph-mon[35615]: fsmap 9 up:standby 2024-09-11T21:55:03.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:03 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-11T21:55:03.623 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T21:55:03.898 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:55:03.943 INFO:journalctl@ceph.mds.g.smithi001.stdout:Sep 11 21:55:03 smithi001 systemd[1]: Starting Ceph mds.g for 3ffca83c-7087-11ef-bcea-c7b262605968... 2024-09-11T21:55:04.273 INFO:journalctl@ceph.mds.g.smithi001.stdout:Sep 11 21:55:04 smithi001 podman[69370]: 2024-09-11 21:55:04.157471123 +0000 UTC m=+0.017353081 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T21:55:04.273 INFO:journalctl@ceph.mds.g.smithi001.stdout:Sep 11 21:55:04 smithi001 podman[69370]: 2024-09-11 21:55:04.272790243 +0000 UTC m=+0.132672193 container create c4b4c7930b9d77ff9418c8cc566a167e5da1135bfdb188a10fbb77e87b4db5c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-g, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS) 2024-09-11T21:55:04.542 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:04 smithi001 ceph-mon[30935]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:55:04.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:04 smithi053 ceph-mon[34908]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:55:04.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:04 smithi161 ceph-mon[35615]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:55:04.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:04.624+0000 7f86c819f640 1 -- 172.21.15.1:0/736688765 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86c0100480 msgr2=0x7f86c0100860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:04.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:04.624+0000 7f86c819f640 1 --2- 172.21.15.1:0/736688765 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86c0100480 0x7f86c0100860 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f86bc00aa10 tx=0x7f86bc02b720 comp rx=0 tx=0).stop 2024-09-11T21:55:04.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:04.624+0000 7f86c819f640 1 -- 172.21.15.1:0/736688765 shutdown_connections 2024-09-11T21:55:04.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:04.624+0000 7f86c819f640 1 --2- 172.21.15.1:0/736688765 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86c01055b0 0x7f86c010d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:04.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:04.624+0000 7f86c819f640 1 --2- 172.21.15.1:0/736688765 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f86c0100e30 0x7f86c0104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:04.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:04.624+0000 7f86c819f640 1 --2- 172.21.15.1:0/736688765 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86c0100480 0x7f86c0100860 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:04.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:04.624+0000 7f86c819f640 1 -- 172.21.15.1:0/736688765 >> 172.21.15.1:0/736688765 conn(0x7f86c0077210 msgr2=0x7f86c00fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:55:04.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:04.625+0000 7f86c819f640 1 -- 172.21.15.1:0/736688765 shutdown_connections 2024-09-11T21:55:04.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:04.625+0000 7f86c819f640 1 -- 172.21.15.1:0/736688765 wait complete. 2024-09-11T21:55:04.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:04.625+0000 7f86c819f640 1 Processor -- start 2024-09-11T21:55:04.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:04.625+0000 7f86c819f640 1 -- start start 2024-09-11T21:55:04.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:04.626+0000 7f86c819f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86c0100480 0x7f86c0196260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:04.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:04.626+0000 7f86c819f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f86c0100e30 0x7f86c01967a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:04.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:04.626+0000 7f86c819f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86c01055b0 0x7f86c019ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:04.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:04.626+0000 7f86c819f640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f86c010fd60 con 0x7f86c0100480 2024-09-11T21:55:04.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:04.626+0000 7f86c819f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f86c010fbe0 con 0x7f86c0100e30 2024-09-11T21:55:04.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:04.626+0000 7f86c819f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f86c010fee0 con 0x7f86c01055b0 2024-09-11T21:55:04.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:04.626+0000 7f86c5f3b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86c0100480 0x7f86c0196260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:04.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:04.626+0000 7f86c5f3b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86c0100480 0x7f86c0196260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47890/0 (socket says 172.21.15.1:47890) 2024-09-11T21:55:04.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:04.626+0000 7f86c5f3b640 1 -- 172.21.15.1:0/2413933712 learned_addr learned my addr 172.21.15.1:0/2413933712 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:55:04.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:04.626+0000 7f86c573a640 1 --2- 172.21.15.1:0/2413933712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f86c0100e30 0x7f86c01967a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:04.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:04.626+0000 7f86c5f3b640 1 -- 172.21.15.1:0/2413933712 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86c01055b0 msgr2=0x7f86c019ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:04.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:04.626+0000 7f86c5f3b640 1 --2- 172.21.15.1:0/2413933712 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86c01055b0 0x7f86c019ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:04.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:04.626+0000 7f86c5f3b640 1 -- 172.21.15.1:0/2413933712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f86c0100e30 msgr2=0x7f86c01967a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:04.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:04.626+0000 7f86c5f3b640 1 --2- 172.21.15.1:0/2413933712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f86c0100e30 0x7f86c01967a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:04.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:04.626+0000 7f86c5f3b640 1 -- 172.21.15.1:0/2413933712 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86c019b210 con 0x7f86c0100480 2024-09-11T21:55:04.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:04.626+0000 7f86c5f3b640 1 --2- 172.21.15.1:0/2413933712 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86c0100480 0x7f86c0196260 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f86bc006010 tx=0x7f86bc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:55:04.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:04.626+0000 7f86b6ffd640 1 -- 172.21.15.1:0/2413933712 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86bc040070 con 0x7f86c0100480 2024-09-11T21:55:04.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:04.626+0000 7f86b6ffd640 1 -- 172.21.15.1:0/2413933712 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86bc03c460 con 0x7f86c0100480 2024-09-11T21:55:04.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:04.626+0000 7f86c819f640 1 -- 172.21.15.1:0/2413933712 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86c019b4a0 con 0x7f86c0100480 2024-09-11T21:55:04.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:04.626+0000 7f86b6ffd640 1 -- 172.21.15.1:0/2413933712 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86bc036030 con 0x7f86c0100480 2024-09-11T21:55:04.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:04.626+0000 7f86c819f640 1 -- 172.21.15.1:0/2413933712 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86c01a2de0 con 0x7f86c0100480 2024-09-11T21:55:04.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:04.628+0000 7f86c819f640 1 -- 172.21.15.1:0/2413933712 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86c0100860 con 0x7f86c0100480 2024-09-11T21:55:04.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:04.629+0000 7f86b6ffd640 1 -- 172.21.15.1:0/2413933712 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f86bc043070 con 0x7f86c0100480 2024-09-11T21:55:04.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:04.629+0000 7f86b6ffd640 1 --2- 172.21.15.1:0/2413933712 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f86ac0a2c00 0x7f86ac0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:04.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:04.629+0000 7f86c573a640 1 --2- 172.21.15.1:0/2413933712 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f86ac0a2c00 0x7f86ac0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:04.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:04.629+0000 7f86b6ffd640 1 -- 172.21.15.1:0/2413933712 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f86bc0f06f0 con 0x7f86c0100480 2024-09-11T21:55:04.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:04.630+0000 7f86c573a640 1 --2- 172.21.15.1:0/2413933712 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f86ac0a2c00 0x7f86ac0a50c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f86c00ff7f0 tx=0x7f86b000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:55:04.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:04.633+0000 7f86b6ffd640 1 -- 172.21.15.1:0/2413933712 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f86bc0bda50 con 0x7f86c0100480 2024-09-11T21:55:04.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:04.846+0000 7f86c819f640 1 -- 172.21.15.1:0/2413933712 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f86c0108200 con 0x7f86c0100480 2024-09-11T21:55:04.848 INFO:journalctl@ceph.mds.g.smithi001.stdout:Sep 11 21:55:04 smithi001 podman[69370]: 2024-09-11 21:55:04.760444796 +0000 UTC m=+0.620326745 container init c4b4c7930b9d77ff9418c8cc566a167e5da1135bfdb188a10fbb77e87b4db5c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-g, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T21:55:04.848 INFO:journalctl@ceph.mds.g.smithi001.stdout:Sep 11 21:55:04 smithi001 podman[69370]: 2024-09-11 21:55:04.763663026 +0000 UTC m=+0.623544969 container start c4b4c7930b9d77ff9418c8cc566a167e5da1135bfdb188a10fbb77e87b4db5c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-g, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T21:55:04.848 INFO:journalctl@ceph.mds.g.smithi001.stdout:Sep 11 21:55:04 smithi001 ceph-mds[69413]: -- 172.21.15.1:0/583296840 <== mon.0 v2:172.21.15.1:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ba85067860 con 0x55ba85045400 2024-09-11T21:55:04.848 INFO:journalctl@ceph.mds.g.smithi001.stdout:Sep 11 21:55:04 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-g[69388]: starting mds.g at 2024-09-11T21:55:04.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:04.847+0000 7f86b6ffd640 1 -- 172.21.15.1:0/2413933712 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8031 (secure 0 0 0) 0x7f86bc0bd350 con 0x7f86c0100480 2024-09-11T21:55:04.849 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:55:04.849 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":10,"btime":"2024-09-11T21:55:02:267519+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/421542832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":421542832},{"type":"v1","addr":"172.21.15.1:6835","nonce":421542832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2836864406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2836864406},{"type":"v1","addr":"172.21.15.1:6837","nonce":2836864406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/872410024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":872410024},{"type":"v1","addr":"172.21.15.161:6833","nonce":872410024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1648944570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1648944570},{"type":"v1","addr":"172.21.15.53:6833","nonce":1648944570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/2126933993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2126933993},{"type":"v1","addr":"172.21.15.53:6835","nonce":2126933993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/828381592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":828381592},{"type":"v1","addr":"172.21.15.161:6835","nonce":828381592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/1799318656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1799318656},{"type":"v1","addr":"172.21.15.161:6837","nonce":1799318656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24467,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/3457131830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3457131830},{"type":"v1","addr":"172.21.15.53:6837","nonce":3457131830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24484,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6839/455365383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":455365383},{"type":"v1","addr":"172.21.15.161:6839","nonce":455365383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-11T21:55:04.849 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 10 2024-09-11T21:55:04.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:04.849+0000 7f86c819f640 1 -- 172.21.15.1:0/2413933712 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f86ac0a2c00 msgr2=0x7f86ac0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:04.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:04.849+0000 7f86c819f640 1 --2- 172.21.15.1:0/2413933712 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f86ac0a2c00 0x7f86ac0a50c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f86c00ff7f0 tx=0x7f86b000b390 comp rx=0 tx=0).stop 2024-09-11T21:55:04.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:04.849+0000 7f86c819f640 1 -- 172.21.15.1:0/2413933712 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86c0100480 msgr2=0x7f86c0196260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:04.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:04.849+0000 7f86c819f640 1 --2- 172.21.15.1:0/2413933712 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86c0100480 0x7f86c0196260 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f86bc006010 tx=0x7f86bc0042f0 comp rx=0 tx=0).stop 2024-09-11T21:55:04.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:04.850+0000 7f86c819f640 1 -- 172.21.15.1:0/2413933712 shutdown_connections 2024-09-11T21:55:04.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:04.850+0000 7f86c819f640 1 --2- 172.21.15.1:0/2413933712 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f86ac0a2c00 0x7f86ac0a50c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:04.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:04.850+0000 7f86c819f640 1 --2- 172.21.15.1:0/2413933712 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86c01055b0 0x7f86c019ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:04.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:04.850+0000 7f86c819f640 1 --2- 172.21.15.1:0/2413933712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f86c0100e30 0x7f86c01967a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:04.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:04.850+0000 7f86c819f640 1 --2- 172.21.15.1:0/2413933712 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86c0100480 0x7f86c0196260 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:04.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:04.850+0000 7f86c819f640 1 -- 172.21.15.1:0/2413933712 >> 172.21.15.1:0/2413933712 conn(0x7f86c0077210 msgr2=0x7f86c0103830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:55:04.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:04.850+0000 7f86c819f640 1 -- 172.21.15.1:0/2413933712 shutdown_connections 2024-09-11T21:55:04.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:04.850+0000 7f86c819f640 1 -- 172.21.15.1:0/2413933712 wait complete. 2024-09-11T21:55:05.193 INFO:journalctl@ceph.mds.g.smithi001.stdout:Sep 11 21:55:04 smithi001 bash[69370]: c4b4c7930b9d77ff9418c8cc566a167e5da1135bfdb188a10fbb77e87b4db5c9 2024-09-11T21:55:05.193 INFO:journalctl@ceph.mds.g.smithi001.stdout:Sep 11 21:55:04 smithi001 systemd[1]: Started Ceph mds.g for 3ffca83c-7087-11ef-bcea-c7b262605968. 2024-09-11T21:55:05.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:05 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/2413933712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:55:05.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:05 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:05.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:05 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:05.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:05 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:05.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:05 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T21:55:05.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:05 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T21:55:05.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:05 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:55:05.549 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:05 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/2413933712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:55:05.549 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:05 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:05.550 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:05 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:05.550 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:05 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:05.550 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:05 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T21:55:05.550 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:05 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T21:55:05.550 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:05 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:55:05.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:05 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/2413933712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:55:05.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:05 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:05.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:05 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:05.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:05 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:05.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:05 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T21:55:05.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:05 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T21:55:05.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:05 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:55:06.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:06 smithi001 ceph-mon[30935]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:55:06.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:06 smithi001 ceph-mon[30935]: Deploying daemon mds.a on smithi001 2024-09-11T21:55:06.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:06 smithi001 ceph-mon[30935]: mds.? [v2:172.21.15.1:6838/407834672,v1:172.21.15.1:6839/407834672] up:boot 2024-09-11T21:55:06.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:06 smithi001 ceph-mon[30935]: fsmap 10 up:standby 2024-09-11T21:55:06.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:06 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-11T21:55:06.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:06 smithi053 ceph-mon[34908]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:55:06.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:06 smithi053 ceph-mon[34908]: Deploying daemon mds.a on smithi001 2024-09-11T21:55:06.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:06 smithi053 ceph-mon[34908]: mds.? [v2:172.21.15.1:6838/407834672,v1:172.21.15.1:6839/407834672] up:boot 2024-09-11T21:55:06.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:06 smithi053 ceph-mon[34908]: fsmap 10 up:standby 2024-09-11T21:55:06.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:06 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-11T21:55:06.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:06 smithi161 ceph-mon[35615]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:55:06.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:06 smithi161 ceph-mon[35615]: Deploying daemon mds.a on smithi001 2024-09-11T21:55:06.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:06 smithi161 ceph-mon[35615]: mds.? [v2:172.21.15.1:6838/407834672,v1:172.21.15.1:6839/407834672] up:boot 2024-09-11T21:55:06.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:06 smithi161 ceph-mon[35615]: fsmap 10 up:standby 2024-09-11T21:55:06.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:06 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-11T21:55:07.443 INFO:journalctl@ceph.mds.a.smithi001.stdout:Sep 11 21:55:07 smithi001 systemd[1]: Starting Ceph mds.a for 3ffca83c-7087-11ef-bcea-c7b262605968... 2024-09-11T21:55:07.580 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T21:55:07.831 INFO:journalctl@ceph.mds.a.smithi001.stdout:Sep 11 21:55:07 smithi001 podman[69714]: 2024-09-11 21:55:07.474076431 +0000 UTC m=+0.016538234 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T21:55:07.831 INFO:journalctl@ceph.mds.a.smithi001.stdout:Sep 11 21:55:07 smithi001 podman[69714]: 2024-09-11 21:55:07.594371006 +0000 UTC m=+0.136832814 container create 78f9f386febc1c5c88a73a3f42496dba74df06ad21f429c7706e26a5fd30a4cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-a, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS) 2024-09-11T21:55:07.831 INFO:journalctl@ceph.mds.a.smithi001.stdout:Sep 11 21:55:07 smithi001 podman[69714]: 2024-09-11 21:55:07.831031436 +0000 UTC m=+0.373493239 container init 78f9f386febc1c5c88a73a3f42496dba74df06ad21f429c7706e26a5fd30a4cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-a, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T21:55:08.012 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:55:08.134 INFO:journalctl@ceph.mds.a.smithi001.stdout:Sep 11 21:55:07 smithi001 podman[69714]: 2024-09-11 21:55:07.835036226 +0000 UTC m=+0.377498031 container start 78f9f386febc1c5c88a73a3f42496dba74df06ad21f429c7706e26a5fd30a4cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-11T21:55:08.135 INFO:journalctl@ceph.mds.a.smithi001.stdout:Sep 11 21:55:07 smithi001 ceph-mds[69763]: -- 172.21.15.1:0/1229306945 <== mon.0 v2:172.21.15.1:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5598b5311860 con 0x5598b52ef400 2024-09-11T21:55:08.135 INFO:journalctl@ceph.mds.a.smithi001.stdout:Sep 11 21:55:07 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-a[69747]: starting mds.a at 2024-09-11T21:55:08.135 INFO:journalctl@ceph.mds.a.smithi001.stdout:Sep 11 21:55:07 smithi001 bash[69714]: 78f9f386febc1c5c88a73a3f42496dba74df06ad21f429c7706e26a5fd30a4cf 2024-09-11T21:55:08.135 INFO:journalctl@ceph.mds.a.smithi001.stdout:Sep 11 21:55:07 smithi001 systemd[1]: Started Ceph mds.a for 3ffca83c-7087-11ef-bcea-c7b262605968. 2024-09-11T21:55:08.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:08 smithi001 ceph-mon[30935]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:55:08.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:08 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:08.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:08 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:08.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:08 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:08.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:08 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T21:55:08.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:08 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T21:55:08.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:08 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:55:08.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:08.509+0000 7f44598bb640 1 -- 172.21.15.1:0/60885419 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4454101420 msgr2=0x7f445410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:08.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:08.509+0000 7f44598bb640 1 --2- 172.21.15.1:0/60885419 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4454101420 0x7f445410d950 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4444009930 tx=0x7f444402a420 comp rx=0 tx=0).stop 2024-09-11T21:55:08.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:08.509+0000 7f44598bb640 1 -- 172.21.15.1:0/60885419 shutdown_connections 2024-09-11T21:55:08.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:08.509+0000 7f44598bb640 1 --2- 172.21.15.1:0/60885419 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4454101420 0x7f445410d950 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:08.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:08.509+0000 7f44598bb640 1 --2- 172.21.15.1:0/60885419 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4454100a80 0x7f4454100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:08.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:08.509+0000 7f44598bb640 1 --2- 172.21.15.1:0/60885419 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4454106a80 0x7f4454106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:08.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:08.509+0000 7f44598bb640 1 -- 172.21.15.1:0/60885419 >> 172.21.15.1:0/60885419 conn(0x7f44540fc7a0 msgr2=0x7f44540febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:55:08.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:08.509+0000 7f44598bb640 1 -- 172.21.15.1:0/60885419 shutdown_connections 2024-09-11T21:55:08.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:08.509+0000 7f44598bb640 1 -- 172.21.15.1:0/60885419 wait complete. 2024-09-11T21:55:08.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:08.510+0000 7f44598bb640 1 Processor -- start 2024-09-11T21:55:08.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:08.510+0000 7f44598bb640 1 -- start start 2024-09-11T21:55:08.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:08.510+0000 7f44598bb640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4454100a80 0x7f44541983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:08.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:08.511+0000 7f44598bb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4454101420 0x7f44541988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:08.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:08.511+0000 7f44598bb640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4454106a80 0x7f445419cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:08.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:08.511+0000 7f44598bb640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4454110e80 con 0x7f4454106a80 2024-09-11T21:55:08.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:08.511+0000 7f44598bb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4454110d00 con 0x7f4454101420 2024-09-11T21:55:08.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:08.511+0000 7f44598bb640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4454111000 con 0x7f4454100a80 2024-09-11T21:55:08.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:08.511+0000 7f44537fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4454106a80 0x7f445419cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:08.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:08.511+0000 7f44537fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4454106a80 0x7f445419cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46212/0 (socket says 172.21.15.1:46212) 2024-09-11T21:55:08.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:08.511+0000 7f44537fe640 1 -- 172.21.15.1:0/1469753413 learned_addr learned my addr 172.21.15.1:0/1469753413 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:55:08.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:08.511+0000 7f4452ffd640 1 --2- 172.21.15.1:0/1469753413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4454100a80 0x7f44541983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:08.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:08.511+0000 7f44527fc640 1 --2- 172.21.15.1:0/1469753413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4454101420 0x7f44541988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:08.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:08.511+0000 7f44537fe640 1 -- 172.21.15.1:0/1469753413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4454100a80 msgr2=0x7f44541983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:08.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:08.511+0000 7f44537fe640 1 --2- 172.21.15.1:0/1469753413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4454100a80 0x7f44541983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:08.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:08.511+0000 7f44537fe640 1 -- 172.21.15.1:0/1469753413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4454101420 msgr2=0x7f44541988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:08.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:08.511+0000 7f44537fe640 1 --2- 172.21.15.1:0/1469753413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4454101420 0x7f44541988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:08.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:08.511+0000 7f44537fe640 1 -- 172.21.15.1:0/1469753413 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f445419d360 con 0x7f4454106a80 2024-09-11T21:55:08.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:08.511+0000 7f44537fe640 1 --2- 172.21.15.1:0/1469753413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4454106a80 0x7f445419cc80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4444005ab0 tx=0x7f44440042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:55:08.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:08.511+0000 7f44588b9640 1 -- 172.21.15.1:0/1469753413 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f444403f030 con 0x7f4454106a80 2024-09-11T21:55:08.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:08.511+0000 7f44588b9640 1 -- 172.21.15.1:0/1469753413 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4444003ce0 con 0x7f4454106a80 2024-09-11T21:55:08.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:08.511+0000 7f44598bb640 1 -- 172.21.15.1:0/1469753413 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f445419d5f0 con 0x7f4454106a80 2024-09-11T21:55:08.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:08.511+0000 7f44588b9640 1 -- 172.21.15.1:0/1469753413 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4444035070 con 0x7f4454106a80 2024-09-11T21:55:08.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:08.512+0000 7f44598bb640 1 -- 172.21.15.1:0/1469753413 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44541a4f30 con 0x7f4454106a80 2024-09-11T21:55:08.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:08.512+0000 7f44598bb640 1 -- 172.21.15.1:0/1469753413 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44541021c0 con 0x7f4454106a80 2024-09-11T21:55:08.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:08.514+0000 7f44588b9640 1 -- 172.21.15.1:0/1469753413 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f4444042030 con 0x7f4454106a80 2024-09-11T21:55:08.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:08.515+0000 7f44588b9640 1 --2- 172.21.15.1:0/1469753413 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f44400a2b80 0x7f44400a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:08.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:08.515+0000 7f4452ffd640 1 --2- 172.21.15.1:0/1469753413 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f44400a2b80 0x7f44400a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:08.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:08.515+0000 7f44588b9640 1 -- 172.21.15.1:0/1469753413 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f44440ef520 con 0x7f4454106a80 2024-09-11T21:55:08.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:08.515+0000 7f4452ffd640 1 --2- 172.21.15.1:0/1469753413 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f44400a2b80 0x7f44400a5040 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f444c005c40 tx=0x7f444c00a590 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:55:08.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:08.518+0000 7f44588b9640 1 -- 172.21.15.1:0/1469753413 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f44440bc920 con 0x7f4454106a80 2024-09-11T21:55:08.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:08 smithi053 ceph-mon[34908]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:55:08.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:08 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:08.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:08 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:08.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:08 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:08.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:08 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T21:55:08.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:08 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T21:55:08.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:08 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:55:08.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:08 smithi161 ceph-mon[35615]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:55:08.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:08 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:08.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:08 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:08.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:08 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:08.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:08 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T21:55:08.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:08 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T21:55:08.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:08 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:55:08.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:08.729+0000 7f44598bb640 1 -- 172.21.15.1:0/1469753413 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f445410a000 con 0x7f4454106a80 2024-09-11T21:55:08.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:08.729+0000 7f44588b9640 1 -- 172.21.15.1:0/1469753413 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9674 (secure 0 0 0) 0x7f44440bc220 con 0x7f4454106a80 2024-09-11T21:55:08.730 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:55:08.731 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":12,"btime":"2024-09-11T21:55:08:287866+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/421542832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":421542832},{"type":"v1","addr":"172.21.15.1:6835","nonce":421542832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2836864406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2836864406},{"type":"v1","addr":"172.21.15.1:6837","nonce":2836864406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/407834672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":407834672},{"type":"v1","addr":"172.21.15.1:6839","nonce":407834672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/1068827678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":1068827678},{"type":"v1","addr":"172.21.15.1:6841","nonce":1068827678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/872410024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":872410024},{"type":"v1","addr":"172.21.15.161:6833","nonce":872410024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1648944570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1648944570},{"type":"v1","addr":"172.21.15.53:6833","nonce":1648944570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/2126933993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2126933993},{"type":"v1","addr":"172.21.15.53:6835","nonce":2126933993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/828381592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":828381592},{"type":"v1","addr":"172.21.15.161:6835","nonce":828381592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/1799318656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1799318656},{"type":"v1","addr":"172.21.15.161:6837","nonce":1799318656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24467,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/3457131830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3457131830},{"type":"v1","addr":"172.21.15.53:6837","nonce":3457131830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24484,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6839/455365383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":455365383},{"type":"v1","addr":"172.21.15.161:6839","nonce":455365383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-11T21:55:08.731 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 12 2024-09-11T21:55:08.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:08.731+0000 7f44598bb640 1 -- 172.21.15.1:0/1469753413 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f44400a2b80 msgr2=0x7f44400a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:08.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:08.731+0000 7f44598bb640 1 --2- 172.21.15.1:0/1469753413 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f44400a2b80 0x7f44400a5040 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f444c005c40 tx=0x7f444c00a590 comp rx=0 tx=0).stop 2024-09-11T21:55:08.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:08.731+0000 7f44598bb640 1 -- 172.21.15.1:0/1469753413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4454106a80 msgr2=0x7f445419cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:08.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:08.731+0000 7f44598bb640 1 --2- 172.21.15.1:0/1469753413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4454106a80 0x7f445419cc80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4444005ab0 tx=0x7f44440042f0 comp rx=0 tx=0).stop 2024-09-11T21:55:08.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:08.731+0000 7f44598bb640 1 -- 172.21.15.1:0/1469753413 shutdown_connections 2024-09-11T21:55:08.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:08.731+0000 7f44598bb640 1 --2- 172.21.15.1:0/1469753413 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f44400a2b80 0x7f44400a5040 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:08.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:08.731+0000 7f44598bb640 1 --2- 172.21.15.1:0/1469753413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4454106a80 0x7f445419cc80 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:08.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:08.731+0000 7f44598bb640 1 --2- 172.21.15.1:0/1469753413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4454101420 0x7f44541988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:08.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:08.731+0000 7f44598bb640 1 --2- 172.21.15.1:0/1469753413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4454100a80 0x7f44541983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:08.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:08.731+0000 7f44598bb640 1 -- 172.21.15.1:0/1469753413 >> 172.21.15.1:0/1469753413 conn(0x7f44540fc7a0 msgr2=0x7f445406a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:55:08.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:08.732+0000 7f44598bb640 1 -- 172.21.15.1:0/1469753413 shutdown_connections 2024-09-11T21:55:08.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:08.732+0000 7f44598bb640 1 -- 172.21.15.1:0/1469753413 wait complete. 2024-09-11T21:55:09.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:09 smithi161 ceph-mon[35615]: Deploying daemon mds.b on smithi053 2024-09-11T21:55:09.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:09 smithi161 ceph-mon[35615]: mds.? [v2:172.21.15.1:6840/1068827678,v1:172.21.15.1:6841/1068827678] up:boot 2024-09-11T21:55:09.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:09 smithi161 ceph-mon[35615]: fsmap 11 up:standby 2024-09-11T21:55:09.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:09 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-11T21:55:09.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:09 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/1469753413' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:55:09.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:09 smithi001 ceph-mon[30935]: Deploying daemon mds.b on smithi053 2024-09-11T21:55:09.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:09 smithi001 ceph-mon[30935]: mds.? [v2:172.21.15.1:6840/1068827678,v1:172.21.15.1:6841/1068827678] up:boot 2024-09-11T21:55:09.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:09 smithi001 ceph-mon[30935]: fsmap 11 up:standby 2024-09-11T21:55:09.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:09 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-11T21:55:09.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:09 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/1469753413' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:55:09.735 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:09 smithi053 ceph-mon[34908]: Deploying daemon mds.b on smithi053 2024-09-11T21:55:09.736 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:09 smithi053 ceph-mon[34908]: mds.? [v2:172.21.15.1:6840/1068827678,v1:172.21.15.1:6841/1068827678] up:boot 2024-09-11T21:55:09.736 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:09 smithi053 ceph-mon[34908]: fsmap 11 up:standby 2024-09-11T21:55:09.736 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:09 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-11T21:55:09.736 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:09 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/1469753413' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:55:10.047 INFO:journalctl@ceph.mds.b.smithi053.stdout:Sep 11 21:55:09 smithi053 systemd[1]: Starting Ceph mds.b for 3ffca83c-7087-11ef-bcea-c7b262605968... 2024-09-11T21:55:10.391 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:10 smithi053 ceph-mon[34908]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:55:10.391 INFO:journalctl@ceph.mds.b.smithi053.stdout:Sep 11 21:55:10 smithi053 podman[64358]: 2024-09-11 21:55:10.040560743 +0000 UTC m=+0.015963041 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T21:55:10.391 INFO:journalctl@ceph.mds.b.smithi053.stdout:Sep 11 21:55:10 smithi053 podman[64358]: 2024-09-11 21:55:10.138688575 +0000 UTC m=+0.114090864 container create 7e08278586c4a359fdb0e1ef579061b2014c783e233d0cd77992014fdda689ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-b, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-11T21:55:10.391 INFO:journalctl@ceph.mds.b.smithi053.stdout:Sep 11 21:55:10 smithi053 podman[64358]: 2024-09-11 21:55:10.390965548 +0000 UTC m=+0.366367837 container init 7e08278586c4a359fdb0e1ef579061b2014c783e233d0cd77992014fdda689ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-b53b6d3) 2024-09-11T21:55:10.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:10 smithi161 ceph-mon[35615]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:55:10.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:10 smithi001 ceph-mon[30935]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:55:10.797 INFO:journalctl@ceph.mds.b.smithi053.stdout:Sep 11 21:55:10 smithi053 podman[64358]: 2024-09-11 21:55:10.394130063 +0000 UTC m=+0.369532357 container start 7e08278586c4a359fdb0e1ef579061b2014c783e233d0cd77992014fdda689ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T21:55:10.797 INFO:journalctl@ceph.mds.b.smithi053.stdout:Sep 11 21:55:10 smithi053 ceph-mds[64377]: -- 172.21.15.53:0/548332022 <== mon.2 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562d9b8bd860 con 0x562d9b89a800 2024-09-11T21:55:10.798 INFO:journalctl@ceph.mds.b.smithi053.stdout:Sep 11 21:55:10 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-b[64373]: starting mds.b at 2024-09-11T21:55:10.798 INFO:journalctl@ceph.mds.b.smithi053.stdout:Sep 11 21:55:10 smithi053 bash[64358]: 7e08278586c4a359fdb0e1ef579061b2014c783e233d0cd77992014fdda689ae 2024-09-11T21:55:10.798 INFO:journalctl@ceph.mds.b.smithi053.stdout:Sep 11 21:55:10 smithi053 systemd[1]: Started Ceph mds.b for 3ffca83c-7087-11ef-bcea-c7b262605968. 2024-09-11T21:55:11.396 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T21:55:11.561 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:11 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:11.561 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:11 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:11.561 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:11 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:11.561 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:11 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:11.561 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:11 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:11.561 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:11 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:55:11.561 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:11 smithi001 ceph-mon[30935]: mds.? [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] up:boot 2024-09-11T21:55:11.562 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:11 smithi001 ceph-mon[30935]: fsmap 12 up:standby 2024-09-11T21:55:11.562 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:11 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-11T21:55:11.589 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:11 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:11.589 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:11 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:11.589 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:11 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:11.589 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:11 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:11.589 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:11 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:11.589 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:11 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:55:11.589 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:11 smithi161 ceph-mon[35615]: mds.? [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] up:boot 2024-09-11T21:55:11.589 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:11 smithi161 ceph-mon[35615]: fsmap 12 up:standby 2024-09-11T21:55:11.589 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:11 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-11T21:55:11.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:11 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:11.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:11 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:11.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:11 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:11.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:11 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:11.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:11 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:11.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:11 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:55:11.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:11 smithi053 ceph-mon[34908]: mds.? [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] up:boot 2024-09-11T21:55:11.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:11 smithi053 ceph-mon[34908]: fsmap 12 up:standby 2024-09-11T21:55:11.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:11 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-11T21:55:11.844 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:55:12.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:12.372+0000 7f62c7208640 1 -- 172.21.15.1:0/549420363 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62c0101420 msgr2=0x7f62c010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:12.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:12.372+0000 7f62c7208640 1 --2- 172.21.15.1:0/549420363 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62c0101420 0x7f62c010d950 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f62b8009cd0 tx=0x7f62b802a8e0 comp rx=0 tx=0).stop 2024-09-11T21:55:12.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:12.372+0000 7f62c7208640 1 -- 172.21.15.1:0/549420363 shutdown_connections 2024-09-11T21:55:12.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:12.372+0000 7f62c7208640 1 --2- 172.21.15.1:0/549420363 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62c0101420 0x7f62c010d950 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:12.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:12.372+0000 7f62c7208640 1 --2- 172.21.15.1:0/549420363 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62c0100a80 0x7f62c0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:12.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:12.372+0000 7f62c7208640 1 --2- 172.21.15.1:0/549420363 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62c0106a80 0x7f62c0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:12.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:12.372+0000 7f62c7208640 1 -- 172.21.15.1:0/549420363 >> 172.21.15.1:0/549420363 conn(0x7f62c00fc7a0 msgr2=0x7f62c00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:55:12.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:12.372+0000 7f62c7208640 1 -- 172.21.15.1:0/549420363 shutdown_connections 2024-09-11T21:55:12.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:12.372+0000 7f62c7208640 1 -- 172.21.15.1:0/549420363 wait complete. 2024-09-11T21:55:12.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:12.372+0000 7f62c7208640 1 Processor -- start 2024-09-11T21:55:12.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:12.373+0000 7f62c7208640 1 -- start start 2024-09-11T21:55:12.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:12.373+0000 7f62c7208640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62c0100a80 0x7f62c00fcd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:12.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:12.373+0000 7f62c7208640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62c0101420 0x7f62c00fd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:12.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:12.373+0000 7f62c7208640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62c0106a80 0x7f62c00fd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:12.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:12.373+0000 7f62c7208640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f62c0110e80 con 0x7f62c0100a80 2024-09-11T21:55:12.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:12.373+0000 7f62c7208640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f62c0110d00 con 0x7f62c0106a80 2024-09-11T21:55:12.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:12.373+0000 7f62c7208640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f62c0111000 con 0x7f62c0101420 2024-09-11T21:55:12.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:12.373+0000 7f62c4fa4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62c0100a80 0x7f62c00fcd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:12.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:12.373+0000 7f62c4fa4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62c0100a80 0x7f62c00fcd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46230/0 (socket says 172.21.15.1:46230) 2024-09-11T21:55:12.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:12.373+0000 7f62c4fa4640 1 -- 172.21.15.1:0/4067154656 learned_addr learned my addr 172.21.15.1:0/4067154656 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:55:12.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:12.373+0000 7f62c57a5640 1 --2- 172.21.15.1:0/4067154656 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62c0106a80 0x7f62c00fd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:12.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:12.373+0000 7f62b7fff640 1 --2- 172.21.15.1:0/4067154656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62c0101420 0x7f62c00fd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:12.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:12.373+0000 7f62c4fa4640 1 -- 172.21.15.1:0/4067154656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62c0101420 msgr2=0x7f62c00fd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:12.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:12.373+0000 7f62c4fa4640 1 --2- 172.21.15.1:0/4067154656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62c0101420 0x7f62c00fd2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:12.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:12.373+0000 7f62c4fa4640 1 -- 172.21.15.1:0/4067154656 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62c0106a80 msgr2=0x7f62c00fd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:12.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:12.373+0000 7f62c4fa4640 1 --2- 172.21.15.1:0/4067154656 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62c0106a80 0x7f62c00fd7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:12.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:12.373+0000 7f62c4fa4640 1 -- 172.21.15.1:0/4067154656 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62c01a6960 con 0x7f62c0100a80 2024-09-11T21:55:12.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:12.373+0000 7f62b7fff640 1 --2- 172.21.15.1:0/4067154656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62c0101420 0x7f62c00fd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:55:12.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:12.373+0000 7f62c4fa4640 1 --2- 172.21.15.1:0/4067154656 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62c0100a80 0x7f62c00fcd60 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f62bc00e9f0 tx=0x7f62bc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:55:12.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:12.373+0000 7f62b5ffb640 1 -- 172.21.15.1:0/4067154656 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62bc00cd80 con 0x7f62c0100a80 2024-09-11T21:55:12.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:12.374+0000 7f62c7208640 1 -- 172.21.15.1:0/4067154656 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62c01a6c50 con 0x7f62c0100a80 2024-09-11T21:55:12.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:12.374+0000 7f62b5ffb640 1 -- 172.21.15.1:0/4067154656 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62bc010910 con 0x7f62c0100a80 2024-09-11T21:55:12.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:12.374+0000 7f62b5ffb640 1 -- 172.21.15.1:0/4067154656 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62bc0188f0 con 0x7f62c0100a80 2024-09-11T21:55:12.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:12.374+0000 7f62c7208640 1 -- 172.21.15.1:0/4067154656 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62c01a7190 con 0x7f62c0100a80 2024-09-11T21:55:12.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:12.375+0000 7f62c7208640 1 -- 172.21.15.1:0/4067154656 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62c0100ee0 con 0x7f62c0100a80 2024-09-11T21:55:12.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:12.376+0000 7f62b5ffb640 1 -- 172.21.15.1:0/4067154656 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f62bc010ac0 con 0x7f62c0100a80 2024-09-11T21:55:12.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:12.377+0000 7f62b5ffb640 1 --2- 172.21.15.1:0/4067154656 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f62a80a2b80 0x7f62a80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:12.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:12.377+0000 7f62b7fff640 1 --2- 172.21.15.1:0/4067154656 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f62a80a2b80 0x7f62a80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:12.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:12.377+0000 7f62b5ffb640 1 -- 172.21.15.1:0/4067154656 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f62bc0cef90 con 0x7f62c0100a80 2024-09-11T21:55:12.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:12.377+0000 7f62b7fff640 1 --2- 172.21.15.1:0/4067154656 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f62a80a2b80 0x7f62a80a5040 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f62c00fe7f0 tx=0x7f62b0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:55:12.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:12.381+0000 7f62b5ffb640 1 -- 172.21.15.1:0/4067154656 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f62bc09c390 con 0x7f62c0100a80 2024-09-11T21:55:12.541 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:12 smithi001 ceph-mon[30935]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:55:12.542 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:12 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:12.542 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:12 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:12.542 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:12 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:12.542 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:12 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:12.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:12.623+0000 7f62c7208640 1 -- 172.21.15.1:0/4067154656 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f62c010a050 con 0x7f62c0100a80 2024-09-11T21:55:12.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:12.624+0000 7f62b5ffb640 1 -- 172.21.15.1:0/4067154656 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10500 (secure 0 0 0) 0x7f62bc0b3920 con 0x7f62c0100a80 2024-09-11T21:55:12.626 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:55:12.626 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":13,"btime":"2024-09-11T21:55:11:299377+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/421542832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":421542832},{"type":"v1","addr":"172.21.15.1:6835","nonce":421542832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2836864406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2836864406},{"type":"v1","addr":"172.21.15.1:6837","nonce":2836864406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14595,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/407834672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":407834672},{"type":"v1","addr":"172.21.15.1:6839","nonce":407834672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/1068827678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":1068827678},{"type":"v1","addr":"172.21.15.1:6841","nonce":1068827678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6833/872410024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":872410024},{"type":"v1","addr":"172.21.15.161:6833","nonce":872410024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/1648944570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1648944570},{"type":"v1","addr":"172.21.15.53:6833","nonce":1648944570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/2126933993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2126933993},{"type":"v1","addr":"172.21.15.53:6835","nonce":2126933993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/828381592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":828381592},{"type":"v1","addr":"172.21.15.161:6835","nonce":828381592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/1799318656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1799318656},{"type":"v1","addr":"172.21.15.161:6837","nonce":1799318656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24467,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/3457131830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3457131830},{"type":"v1","addr":"172.21.15.53:6837","nonce":3457131830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24484,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6839/455365383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":455365383},{"type":"v1","addr":"172.21.15.161:6839","nonce":455365383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24494,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6839/1378584447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1378584447},{"type":"v1","addr":"172.21.15.53:6839","nonce":1378584447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-11T21:55:12.626 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 13 2024-09-11T21:55:12.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:12.627+0000 7f62c7208640 1 -- 172.21.15.1:0/4067154656 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f62a80a2b80 msgr2=0x7f62a80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:12.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:12.627+0000 7f62c7208640 1 --2- 172.21.15.1:0/4067154656 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f62a80a2b80 0x7f62a80a5040 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f62c00fe7f0 tx=0x7f62b0005f90 comp rx=0 tx=0).stop 2024-09-11T21:55:12.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:12.627+0000 7f62c7208640 1 -- 172.21.15.1:0/4067154656 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62c0100a80 msgr2=0x7f62c00fcd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:12.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:12.627+0000 7f62c7208640 1 --2- 172.21.15.1:0/4067154656 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62c0100a80 0x7f62c00fcd60 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f62bc00e9f0 tx=0x7f62bc00eeb0 comp rx=0 tx=0).stop 2024-09-11T21:55:12.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:12.627+0000 7f62c7208640 1 -- 172.21.15.1:0/4067154656 shutdown_connections 2024-09-11T21:55:12.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:12.628+0000 7f62c7208640 1 --2- 172.21.15.1:0/4067154656 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f62a80a2b80 0x7f62a80a5040 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:12.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:12.628+0000 7f62c7208640 1 --2- 172.21.15.1:0/4067154656 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62c0106a80 0x7f62c00fd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:12.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:12.628+0000 7f62c7208640 1 --2- 172.21.15.1:0/4067154656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62c0101420 0x7f62c00fd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:12.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:12.628+0000 7f62c7208640 1 --2- 172.21.15.1:0/4067154656 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62c0100a80 0x7f62c00fcd60 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:12.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:12.628+0000 7f62c7208640 1 -- 172.21.15.1:0/4067154656 >> 172.21.15.1:0/4067154656 conn(0x7f62c00fc7a0 msgr2=0x7f62c006aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:55:12.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:12.628+0000 7f62c7208640 1 -- 172.21.15.1:0/4067154656 shutdown_connections 2024-09-11T21:55:12.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:12.628+0000 7f62c7208640 1 -- 172.21.15.1:0/4067154656 wait complete. 2024-09-11T21:55:12.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:12 smithi053 ceph-mon[34908]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:55:12.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:12 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:12.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:12 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:12.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:12 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:12.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:12 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:12.841 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:12 smithi161 ceph-mon[35615]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:55:12.841 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:12 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:12.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:12 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:12.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:12 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:12.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:12 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:13.283 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-11T21:55:13.660 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:13 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/4067154656' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:55:13.660 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:13 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:13.660 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:13 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:13.660 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:13 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:55:13.660 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:13 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:55:13.660 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:13 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:13.660 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:13 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:55:13.776 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:13 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/4067154656' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:55:13.777 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:13 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:13.777 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:13 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:13.777 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:13 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:55:13.777 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:13 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:55:13.777 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:13 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:13.777 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:13 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:55:13.830 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:13 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/4067154656' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:55:13.830 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:13 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:13.830 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:13 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:13.830 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:13 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:55:13.831 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:13 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:55:13.831 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:13 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:13.831 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:13 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:55:14.221 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:55:14.639 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:14 smithi001 ceph-mon[30935]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:55:14.639 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:14 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:14.639 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:14 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:14.639 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:14 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:14.639 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:14 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:14.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:14.718+0000 7f5962fd9640 1 -- 172.21.15.1:0/2252278079 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f595c101420 msgr2=0x7f595c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:14.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:14.718+0000 7f5962fd9640 1 --2- 172.21.15.1:0/2252278079 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f595c101420 0x7f595c10d950 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5954009cd0 tx=0x7f595402a8e0 comp rx=0 tx=0).stop 2024-09-11T21:55:14.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:14.718+0000 7f5962fd9640 1 -- 172.21.15.1:0/2252278079 shutdown_connections 2024-09-11T21:55:14.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:14.718+0000 7f5962fd9640 1 --2- 172.21.15.1:0/2252278079 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f595c101420 0x7f595c10d950 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:14.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:14.718+0000 7f5962fd9640 1 --2- 172.21.15.1:0/2252278079 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f595c100a80 0x7f595c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:14.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:14.718+0000 7f5962fd9640 1 --2- 172.21.15.1:0/2252278079 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f595c106a80 0x7f595c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:14.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:14.718+0000 7f5962fd9640 1 -- 172.21.15.1:0/2252278079 >> 172.21.15.1:0/2252278079 conn(0x7f595c0fc7a0 msgr2=0x7f595c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:55:14.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:14.718+0000 7f5962fd9640 1 -- 172.21.15.1:0/2252278079 shutdown_connections 2024-09-11T21:55:14.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:14.719+0000 7f5962fd9640 1 -- 172.21.15.1:0/2252278079 wait complete. 2024-09-11T21:55:14.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:14.719+0000 7f5962fd9640 1 Processor -- start 2024-09-11T21:55:14.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:14.719+0000 7f5962fd9640 1 -- start start 2024-09-11T21:55:14.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:14.720+0000 7f5962fd9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f595c100a80 0x7f595c197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:14.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:14.720+0000 7f5962fd9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f595c101420 0x7f595c197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:14.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:14.720+0000 7f5962fd9640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f595c106a80 0x7f595c19bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:14.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:14.720+0000 7f5962fd9640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f595c110e80 con 0x7f595c100a80 2024-09-11T21:55:14.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:14.720+0000 7f5962fd9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f595c110d00 con 0x7f595c101420 2024-09-11T21:55:14.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:14.720+0000 7f5962fd9640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f595c111000 con 0x7f595c106a80 2024-09-11T21:55:14.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:14.720+0000 7f5960d75640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f595c100a80 0x7f595c197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:14.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:14.720+0000 7f5960d75640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f595c100a80 0x7f595c197300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46260/0 (socket says 172.21.15.1:46260) 2024-09-11T21:55:14.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:14.720+0000 7f5960d75640 1 -- 172.21.15.1:0/620170803 learned_addr learned my addr 172.21.15.1:0/620170803 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:55:14.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:14.720+0000 7f5953fff640 1 --2- 172.21.15.1:0/620170803 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f595c101420 0x7f595c197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:14.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:14.720+0000 7f5960d75640 1 -- 172.21.15.1:0/620170803 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f595c106a80 msgr2=0x7f595c19bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:14.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:14.720+0000 7f5960d75640 1 --2- 172.21.15.1:0/620170803 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f595c106a80 0x7f595c19bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:14.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:14.720+0000 7f5960d75640 1 -- 172.21.15.1:0/620170803 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f595c101420 msgr2=0x7f595c197840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:14.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:14.720+0000 7f5960d75640 1 --2- 172.21.15.1:0/620170803 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f595c101420 0x7f595c197840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:14.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:14.720+0000 7f5960d75640 1 -- 172.21.15.1:0/620170803 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f595c19c2b0 con 0x7f595c100a80 2024-09-11T21:55:14.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:14.720+0000 7f5960d75640 1 --2- 172.21.15.1:0/620170803 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f595c100a80 0x7f595c197300 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f595800e9f0 tx=0x7f595800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:55:14.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:14.720+0000 7f5951ffb640 1 -- 172.21.15.1:0/620170803 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595800cd80 con 0x7f595c100a80 2024-09-11T21:55:14.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:14.721+0000 7f5951ffb640 1 -- 172.21.15.1:0/620170803 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5958010910 con 0x7f595c100a80 2024-09-11T21:55:14.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:14.721+0000 7f5962fd9640 1 -- 172.21.15.1:0/620170803 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f595c19c5a0 con 0x7f595c100a80 2024-09-11T21:55:14.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:14.721+0000 7f5951ffb640 1 -- 172.21.15.1:0/620170803 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5958018990 con 0x7f595c100a80 2024-09-11T21:55:14.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:14.721+0000 7f5962fd9640 1 -- 172.21.15.1:0/620170803 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f595c1a3e80 con 0x7f595c100a80 2024-09-11T21:55:14.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:14.722+0000 7f5962fd9640 1 -- 172.21.15.1:0/620170803 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f595c100ee0 con 0x7f595c100a80 2024-09-11T21:55:14.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:14.723+0000 7f5951ffb640 1 -- 172.21.15.1:0/620170803 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f5958010ac0 con 0x7f595c100a80 2024-09-11T21:55:14.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:14.724+0000 7f5951ffb640 1 --2- 172.21.15.1:0/620170803 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f59440a2b80 0x7f59440a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:14.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:14.724+0000 7f5951ffb640 1 -- 172.21.15.1:0/620170803 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f59580cf030 con 0x7f595c100a80 2024-09-11T21:55:14.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:14.724+0000 7f5953fff640 1 --2- 172.21.15.1:0/620170803 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f59440a2b80 0x7f59440a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:14.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:14.724+0000 7f5953fff640 1 --2- 172.21.15.1:0/620170803 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f59440a2b80 0x7f59440a5040 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f595c0fed40 tx=0x7f594c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:55:14.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:14.728+0000 7f5951ffb640 1 -- 172.21.15.1:0/620170803 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f595809c390 con 0x7f595c100a80 2024-09-11T21:55:14.841 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:14 smithi161 ceph-mon[35615]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:55:14.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:14 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:14.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:14 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:14.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:14 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:14.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:14 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:14.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:14.908+0000 7f5962fd9640 1 -- 172.21.15.1:0/620170803 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f595c108a60 con 0x7f595c100a80 2024-09-11T21:55:14.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:14.909+0000 7f5951ffb640 1 -- 172.21.15.1:0/620170803 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v79) ==== 84+0+65 (secure 0 0 0) 0x7f595809bc90 con 0x7f595c100a80 2024-09-11T21:55:14.910 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:55:14.911 INFO:teuthology.orchestra.run.smithi001.stdout:[ 2024-09-11T21:55:14.911 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T21:55:14.911 INFO:teuthology.orchestra.run.smithi001.stdout: "poolnum": 1, 2024-09-11T21:55:14.911 INFO:teuthology.orchestra.run.smithi001.stdout: "poolname": ".mgr" 2024-09-11T21:55:14.911 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T21:55:14.911 INFO:teuthology.orchestra.run.smithi001.stdout:] 2024-09-11T21:55:14.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:14.911+0000 7f5962fd9640 1 -- 172.21.15.1:0/620170803 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f59440a2b80 msgr2=0x7f59440a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:14.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:14.911+0000 7f5962fd9640 1 --2- 172.21.15.1:0/620170803 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f59440a2b80 0x7f59440a5040 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f595c0fed40 tx=0x7f594c005f90 comp rx=0 tx=0).stop 2024-09-11T21:55:14.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:14.911+0000 7f5962fd9640 1 -- 172.21.15.1:0/620170803 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f595c100a80 msgr2=0x7f595c197300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:14.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:14.911+0000 7f5962fd9640 1 --2- 172.21.15.1:0/620170803 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f595c100a80 0x7f595c197300 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f595800e9f0 tx=0x7f595800eeb0 comp rx=0 tx=0).stop 2024-09-11T21:55:14.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:14.911+0000 7f5962fd9640 1 -- 172.21.15.1:0/620170803 shutdown_connections 2024-09-11T21:55:14.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:14.911+0000 7f5962fd9640 1 --2- 172.21.15.1:0/620170803 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f59440a2b80 0x7f59440a5040 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:14.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:14.911+0000 7f5962fd9640 1 --2- 172.21.15.1:0/620170803 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f595c106a80 0x7f595c19bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:14.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:14.911+0000 7f5962fd9640 1 --2- 172.21.15.1:0/620170803 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f595c101420 0x7f595c197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:14.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:14.911+0000 7f5962fd9640 1 --2- 172.21.15.1:0/620170803 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f595c100a80 0x7f595c197300 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:14.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:14.912+0000 7f5962fd9640 1 -- 172.21.15.1:0/620170803 >> 172.21.15.1:0/620170803 conn(0x7f595c0fc7a0 msgr2=0x7f595c0fcec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:55:14.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:14.912+0000 7f5962fd9640 1 -- 172.21.15.1:0/620170803 shutdown_connections 2024-09-11T21:55:14.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:14.912+0000 7f5962fd9640 1 -- 172.21.15.1:0/620170803 wait complete. 2024-09-11T21:55:15.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:14 smithi053 ceph-mon[34908]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:55:15.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:14 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:15.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:14 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:15.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:14 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:15.048 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:14 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:15.576 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-11T21:55:15.577 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-11T21:55:15.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:15 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:15.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:15 smithi001 ceph-mon[30935]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:55:15.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:15 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/620170803' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-11T21:55:15.865 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:55:16.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:15 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:16.048 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:15 smithi053 ceph-mon[34908]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:55:16.048 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:15 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/620170803' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-11T21:55:16.091 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:15 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:16.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:15 smithi161 ceph-mon[35615]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:55:16.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:15 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/620170803' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-11T21:55:16.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:16.341+0000 7f937b0bb640 1 -- 172.21.15.1:0/1213544939 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9374101420 msgr2=0x7f937410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:16.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:16.341+0000 7f937b0bb640 1 --2- 172.21.15.1:0/1213544939 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9374101420 0x7f937410d950 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f936c009cd0 tx=0x7f936c02a8e0 comp rx=0 tx=0).stop 2024-09-11T21:55:16.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:16.341+0000 7f937b0bb640 1 -- 172.21.15.1:0/1213544939 shutdown_connections 2024-09-11T21:55:16.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:16.341+0000 7f937b0bb640 1 --2- 172.21.15.1:0/1213544939 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9374101420 0x7f937410d950 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:16.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:16.341+0000 7f937b0bb640 1 --2- 172.21.15.1:0/1213544939 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9374100a80 0x7f9374100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:16.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:16.341+0000 7f937b0bb640 1 --2- 172.21.15.1:0/1213544939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9374106a80 0x7f9374106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:16.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:16.341+0000 7f937b0bb640 1 -- 172.21.15.1:0/1213544939 >> 172.21.15.1:0/1213544939 conn(0x7f93740fc7a0 msgr2=0x7f93740febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:55:16.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:16.341+0000 7f937b0bb640 1 -- 172.21.15.1:0/1213544939 shutdown_connections 2024-09-11T21:55:16.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:16.341+0000 7f937b0bb640 1 -- 172.21.15.1:0/1213544939 wait complete. 2024-09-11T21:55:16.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:16.342+0000 7f937b0bb640 1 Processor -- start 2024-09-11T21:55:16.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:16.342+0000 7f937b0bb640 1 -- start start 2024-09-11T21:55:16.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:16.342+0000 7f937b0bb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9374100a80 0x7f937419a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:16.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:16.342+0000 7f937b0bb640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9374101420 0x7f937419ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:16.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:16.342+0000 7f937b0bb640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9374106a80 0x7f937419eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:16.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:16.342+0000 7f937b0bb640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9374110ea0 con 0x7f9374101420 2024-09-11T21:55:16.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:16.342+0000 7f937b0bb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9374110d20 con 0x7f9374100a80 2024-09-11T21:55:16.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:16.342+0000 7f937b0bb640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9374111020 con 0x7f9374106a80 2024-09-11T21:55:16.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:16.342+0000 7f9373fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9374101420 0x7f937419ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:16.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:16.342+0000 7f9373fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9374101420 0x7f937419ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46266/0 (socket says 172.21.15.1:46266) 2024-09-11T21:55:16.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:16.342+0000 7f9373fff640 1 -- 172.21.15.1:0/2357110289 learned_addr learned my addr 172.21.15.1:0/2357110289 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:55:16.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:16.342+0000 7f9378e57640 1 --2- 172.21.15.1:0/2357110289 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9374100a80 0x7f937419a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:16.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:16.342+0000 7f9373fff640 1 -- 172.21.15.1:0/2357110289 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9374106a80 msgr2=0x7f937419eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:16.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:16.342+0000 7f9373fff640 1 --2- 172.21.15.1:0/2357110289 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9374106a80 0x7f937419eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:16.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:16.342+0000 7f9373fff640 1 -- 172.21.15.1:0/2357110289 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9374100a80 msgr2=0x7f937419a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:16.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:16.342+0000 7f9373fff640 1 --2- 172.21.15.1:0/2357110289 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9374100a80 0x7f937419a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:16.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:16.342+0000 7f9373fff640 1 -- 172.21.15.1:0/2357110289 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f937419f5c0 con 0x7f9374101420 2024-09-11T21:55:16.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:16.342+0000 7f9378e57640 1 --2- 172.21.15.1:0/2357110289 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9374100a80 0x7f937419a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T21:55:16.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:16.342+0000 7f9373fff640 1 --2- 172.21.15.1:0/2357110289 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9374101420 0x7f937419ab50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f936000c920 tx=0x7f936000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:55:16.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:16.343+0000 7f9371ffb640 1 -- 172.21.15.1:0/2357110289 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9360007c70 con 0x7f9374101420 2024-09-11T21:55:16.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:16.343+0000 7f9371ffb640 1 -- 172.21.15.1:0/2357110289 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9360007e10 con 0x7f9374101420 2024-09-11T21:55:16.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:16.343+0000 7f937b0bb640 1 -- 172.21.15.1:0/2357110289 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f937419f8b0 con 0x7f9374101420 2024-09-11T21:55:16.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:16.343+0000 7f9371ffb640 1 -- 172.21.15.1:0/2357110289 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9360018890 con 0x7f9374101420 2024-09-11T21:55:16.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:16.343+0000 7f937b0bb640 1 -- 172.21.15.1:0/2357110289 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93741a7190 con 0x7f9374101420 2024-09-11T21:55:16.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:16.344+0000 7f937b0bb640 1 -- 172.21.15.1:0/2357110289 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9374100ee0 con 0x7f9374101420 2024-09-11T21:55:16.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:16.345+0000 7f9371ffb640 1 -- 172.21.15.1:0/2357110289 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f9360018a70 con 0x7f9374101420 2024-09-11T21:55:16.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:16.345+0000 7f9371ffb640 1 --2- 172.21.15.1:0/2357110289 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f93640a2c00 0x7f93640a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:16.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:16.346+0000 7f9371ffb640 1 -- 172.21.15.1:0/2357110289 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f93600ced70 con 0x7f9374101420 2024-09-11T21:55:16.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:16.346+0000 7f9378e57640 1 --2- 172.21.15.1:0/2357110289 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f93640a2c00 0x7f93640a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:16.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:16.346+0000 7f9378e57640 1 --2- 172.21.15.1:0/2357110289 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f93640a2c00 0x7f93640a50c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9368006fd0 tx=0x7f9368008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:55:16.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:16.350+0000 7f9371ffb640 1 -- 172.21.15.1:0/2357110289 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9360014030 con 0x7f9374101420 2024-09-11T21:55:16.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:16.527+0000 7f937b0bb640 1 -- 172.21.15.1:0/2357110289 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f93741a74d0 con 0x7f9374101420 2024-09-11T21:55:17.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:17.012+0000 7f9371ffb640 1 -- 172.21.15.1:0/2357110289 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v80) ==== 138+0+0 (secure 0 0 0) 0x7f936009c170 con 0x7f9374101420 2024-09-11T21:55:17.014 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_metadata' created 2024-09-11T21:55:17.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:17.016+0000 7f937b0bb640 1 -- 172.21.15.1:0/2357110289 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f93640a2c00 msgr2=0x7f93640a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:17.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:17.016+0000 7f937b0bb640 1 --2- 172.21.15.1:0/2357110289 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f93640a2c00 0x7f93640a50c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9368006fd0 tx=0x7f9368008040 comp rx=0 tx=0).stop 2024-09-11T21:55:17.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:17.016+0000 7f937b0bb640 1 -- 172.21.15.1:0/2357110289 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9374101420 msgr2=0x7f937419ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:17.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:17.016+0000 7f937b0bb640 1 --2- 172.21.15.1:0/2357110289 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9374101420 0x7f937419ab50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f936000c920 tx=0x7f936000cde0 comp rx=0 tx=0).stop 2024-09-11T21:55:17.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:17.016+0000 7f937b0bb640 1 -- 172.21.15.1:0/2357110289 shutdown_connections 2024-09-11T21:55:17.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:17.016+0000 7f937b0bb640 1 --2- 172.21.15.1:0/2357110289 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f93640a2c00 0x7f93640a50c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:17.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:17.016+0000 7f937b0bb640 1 --2- 172.21.15.1:0/2357110289 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9374106a80 0x7f937419eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:17.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:17.016+0000 7f937b0bb640 1 --2- 172.21.15.1:0/2357110289 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9374101420 0x7f937419ab50 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:17.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:17.016+0000 7f937b0bb640 1 --2- 172.21.15.1:0/2357110289 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9374100a80 0x7f937419a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:17.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:17.016+0000 7f937b0bb640 1 -- 172.21.15.1:0/2357110289 >> 172.21.15.1:0/2357110289 conn(0x7f93740fc7a0 msgr2=0x7f937406a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:55:17.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:17.016+0000 7f937b0bb640 1 -- 172.21.15.1:0/2357110289 shutdown_connections 2024-09-11T21:55:17.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:17.016+0000 7f937b0bb640 1 -- 172.21.15.1:0/2357110289 wait complete. 2024-09-11T21:55:17.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:16 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:17.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:16 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:17.225 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:16 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:55:17.226 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:16 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:55:17.226 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:16 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:17.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:16 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:17.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:16 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:17.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:16 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:55:17.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:16 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:55:17.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:16 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:17.377 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:16 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:17.377 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:16 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:17.377 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:16 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:55:17.377 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:16 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:55:17.377 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:16 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:55:17.698 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-11T21:55:17.953 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:55:18.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:17 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/2357110289' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-11T21:55:18.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:17 smithi161 ceph-mon[35615]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:55:18.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:17 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/2357110289' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-11T21:55:18.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:17 smithi161 ceph-mon[35615]: osdmap e80: 12 total, 12 up, 12 in 2024-09-11T21:55:18.216 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:17 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/2357110289' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-11T21:55:18.216 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:17 smithi001 ceph-mon[30935]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:55:18.216 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:17 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/2357110289' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-11T21:55:18.216 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:17 smithi001 ceph-mon[30935]: osdmap e80: 12 total, 12 up, 12 in 2024-09-11T21:55:18.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:17 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/2357110289' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-11T21:55:18.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:17 smithi053 ceph-mon[34908]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:55:18.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:17 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/2357110289' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-11T21:55:18.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:17 smithi053 ceph-mon[34908]: osdmap e80: 12 total, 12 up, 12 in 2024-09-11T21:55:18.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:18.445+0000 7fa1b8714640 1 -- 172.21.15.1:0/15220357 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1b0102570 msgr2=0x7fa1b01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:18.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:18.445+0000 7fa1b8714640 1 --2- 172.21.15.1:0/15220357 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1b0102570 0x7fa1b01029b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0009900 tx=0x7fa1a002a480 comp rx=0 tx=0).stop 2024-09-11T21:55:18.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:18.445+0000 7fa1b8714640 1 -- 172.21.15.1:0/15220357 shutdown_connections 2024-09-11T21:55:18.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:18.445+0000 7fa1b8714640 1 --2- 172.21.15.1:0/15220357 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1b0102ef0 0x7fa1b010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:18.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:18.445+0000 7fa1b8714640 1 --2- 172.21.15.1:0/15220357 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1b0102570 0x7fa1b01029b0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:18.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:18.445+0000 7fa1b8714640 1 --2- 172.21.15.1:0/15220357 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1b0100e20 0x7fa1b0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:18.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:18.445+0000 7fa1b8714640 1 -- 172.21.15.1:0/15220357 >> 172.21.15.1:0/15220357 conn(0x7fa1b00fc7d0 msgr2=0x7fa1b00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:55:18.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:18.445+0000 7fa1b8714640 1 -- 172.21.15.1:0/15220357 shutdown_connections 2024-09-11T21:55:18.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:18.445+0000 7fa1b8714640 1 -- 172.21.15.1:0/15220357 wait complete. 2024-09-11T21:55:18.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:18.445+0000 7fa1b8714640 1 Processor -- start 2024-09-11T21:55:18.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:18.446+0000 7fa1b8714640 1 -- start start 2024-09-11T21:55:18.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:18.446+0000 7fa1b8714640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1b0100e20 0x7fa1b0198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:18.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:18.446+0000 7fa1b8714640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1b0102570 0x7fa1b0198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:18.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:18.446+0000 7fa1b8714640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1b0102ef0 0x7fa1b019cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:18.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:18.446+0000 7fa1b8714640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b0110f80 con 0x7fa1b0102570 2024-09-11T21:55:18.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:18.446+0000 7fa1b8714640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b0110e00 con 0x7fa1b0100e20 2024-09-11T21:55:18.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:18.446+0000 7fa1b8714640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b0111100 con 0x7fa1b0102ef0 2024-09-11T21:55:18.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:18.446+0000 7fa1b5caf640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1b0102570 0x7fa1b0198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:18.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:18.446+0000 7fa1b5caf640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1b0102570 0x7fa1b0198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43566/0 (socket says 172.21.15.1:43566) 2024-09-11T21:55:18.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:18.446+0000 7fa1b5caf640 1 -- 172.21.15.1:0/2909225204 learned_addr learned my addr 172.21.15.1:0/2909225204 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:55:18.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:18.446+0000 7fa1b64b0640 1 --2- 172.21.15.1:0/2909225204 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1b0100e20 0x7fa1b0198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:18.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:18.446+0000 7fa1b6cb1640 1 --2- 172.21.15.1:0/2909225204 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1b0102ef0 0x7fa1b019cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:18.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:18.446+0000 7fa1b5caf640 1 -- 172.21.15.1:0/2909225204 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1b0102ef0 msgr2=0x7fa1b019cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:18.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:18.446+0000 7fa1b5caf640 1 --2- 172.21.15.1:0/2909225204 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1b0102ef0 0x7fa1b019cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:18.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:18.446+0000 7fa1b5caf640 1 -- 172.21.15.1:0/2909225204 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1b0100e20 msgr2=0x7fa1b0198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:18.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:18.446+0000 7fa1b5caf640 1 --2- 172.21.15.1:0/2909225204 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1b0100e20 0x7fa1b0198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:18.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:18.446+0000 7fa1b5caf640 1 -- 172.21.15.1:0/2909225204 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1b019d4b0 con 0x7fa1b0102570 2024-09-11T21:55:18.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:18.446+0000 7fa1b5caf640 1 --2- 172.21.15.1:0/2909225204 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1b0102570 0x7fa1b0198a40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0006010 tx=0x7fa1a0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:55:18.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:18.447+0000 7fa1a77fe640 1 -- 172.21.15.1:0/2909225204 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a003f070 con 0x7fa1b0102570 2024-09-11T21:55:18.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:18.447+0000 7fa1a77fe640 1 -- 172.21.15.1:0/2909225204 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1a0031c30 con 0x7fa1b0102570 2024-09-11T21:55:18.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:18.447+0000 7fa1b8714640 1 -- 172.21.15.1:0/2909225204 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1b019d740 con 0x7fa1b0102570 2024-09-11T21:55:18.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:18.447+0000 7fa1a77fe640 1 -- 172.21.15.1:0/2909225204 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a0035030 con 0x7fa1b0102570 2024-09-11T21:55:18.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:18.447+0000 7fa1b8714640 1 -- 172.21.15.1:0/2909225204 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1b01a5080 con 0x7fa1b0102570 2024-09-11T21:55:18.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:18.448+0000 7fa1b8714640 1 -- 172.21.15.1:0/2909225204 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1b0101200 con 0x7fa1b0102570 2024-09-11T21:55:18.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:18.449+0000 7fa1a77fe640 1 -- 172.21.15.1:0/2909225204 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fa1a0042070 con 0x7fa1b0102570 2024-09-11T21:55:18.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:18.450+0000 7fa1a77fe640 1 --2- 172.21.15.1:0/2909225204 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fa19c0a2c00 0x7fa19c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:18.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:18.450+0000 7fa1a77fe640 1 -- 172.21.15.1:0/2909225204 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7497+0+0 (secure 0 0 0) 0x7fa1a00efa10 con 0x7fa1b0102570 2024-09-11T21:55:18.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:18.450+0000 7fa1b64b0640 1 --2- 172.21.15.1:0/2909225204 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fa19c0a2c00 0x7fa19c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:18.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:18.450+0000 7fa1b64b0640 1 --2- 172.21.15.1:0/2909225204 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fa19c0a2c00 0x7fa19c0a50c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac00bbe0 tx=0x7fa1ac00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:55:18.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:18.453+0000 7fa1a77fe640 1 -- 172.21.15.1:0/2909225204 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa1a0031de0 con 0x7fa1b0102570 2024-09-11T21:55:18.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:18.633+0000 7fa1b8714640 1 -- 172.21.15.1:0/2909225204 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fa1b010a0a0 con 0x7fa1b0102570 2024-09-11T21:55:19.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:19.070+0000 7fa1a77fe640 1 -- 172.21.15.1:0/2909225204 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v82) ==== 170+0+0 (secure 0 0 0) 0x7fa1a00bc670 con 0x7fa1b0102570 2024-09-11T21:55:19.072 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_data' created 2024-09-11T21:55:19.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:19.073+0000 7fa1b8714640 1 -- 172.21.15.1:0/2909225204 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fa19c0a2c00 msgr2=0x7fa19c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:19.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:19.073+0000 7fa1b8714640 1 --2- 172.21.15.1:0/2909225204 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fa19c0a2c00 0x7fa19c0a50c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac00bbe0 tx=0x7fa1ac00b390 comp rx=0 tx=0).stop 2024-09-11T21:55:19.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:19.073+0000 7fa1b8714640 1 -- 172.21.15.1:0/2909225204 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1b0102570 msgr2=0x7fa1b0198a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:19.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:19.073+0000 7fa1b8714640 1 --2- 172.21.15.1:0/2909225204 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1b0102570 0x7fa1b0198a40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0006010 tx=0x7fa1a0003c10 comp rx=0 tx=0).stop 2024-09-11T21:55:19.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:19.073+0000 7fa1b8714640 1 -- 172.21.15.1:0/2909225204 shutdown_connections 2024-09-11T21:55:19.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:19.073+0000 7fa1b8714640 1 --2- 172.21.15.1:0/2909225204 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fa19c0a2c00 0x7fa19c0a50c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:19.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:19.073+0000 7fa1b8714640 1 --2- 172.21.15.1:0/2909225204 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1b0102ef0 0x7fa1b019cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:19.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:19.073+0000 7fa1b8714640 1 --2- 172.21.15.1:0/2909225204 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1b0102570 0x7fa1b0198a40 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:19.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:19.073+0000 7fa1b8714640 1 --2- 172.21.15.1:0/2909225204 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1b0100e20 0x7fa1b0198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:19.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:19.073+0000 7fa1b8714640 1 -- 172.21.15.1:0/2909225204 >> 172.21.15.1:0/2909225204 conn(0x7fa1b00fc7d0 msgr2=0x7fa1b006a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:55:19.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:19.073+0000 7fa1b8714640 1 -- 172.21.15.1:0/2909225204 shutdown_connections 2024-09-11T21:55:19.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:19.073+0000 7fa1b8714640 1 -- 172.21.15.1:0/2909225204 wait complete. 2024-09-11T21:55:19.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:19 smithi161 ceph-mon[35615]: osdmap e81: 12 total, 12 up, 12 in 2024-09-11T21:55:19.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:19 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/2909225204' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-11T21:55:19.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:19 smithi001 ceph-mon[30935]: osdmap e81: 12 total, 12 up, 12 in 2024-09-11T21:55:19.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:19 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/2909225204' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-11T21:55:19.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:19 smithi053 ceph-mon[34908]: osdmap e81: 12 total, 12 up, 12 in 2024-09-11T21:55:19.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:19 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/2909225204' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-11T21:55:19.786 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-11T21:55:20.045 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:55:20.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:20 smithi161 ceph-mon[35615]: pgmap v278: 33 pgs: 5 creating+peering, 18 unknown, 10 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:55:20.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:20 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/2909225204' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-11T21:55:20.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:20 smithi161 ceph-mon[35615]: osdmap e82: 12 total, 12 up, 12 in 2024-09-11T21:55:20.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:20 smithi161 ceph-mon[35615]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T21:55:20.343 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:20 smithi001 ceph-mon[30935]: pgmap v278: 33 pgs: 5 creating+peering, 18 unknown, 10 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:55:20.343 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:20 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/2909225204' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-11T21:55:20.343 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:20 smithi001 ceph-mon[30935]: osdmap e82: 12 total, 12 up, 12 in 2024-09-11T21:55:20.343 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:20 smithi001 ceph-mon[30935]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T21:55:20.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:20 smithi053 ceph-mon[34908]: pgmap v278: 33 pgs: 5 creating+peering, 18 unknown, 10 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:55:20.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:20 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/2909225204' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-11T21:55:20.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:20 smithi053 ceph-mon[34908]: osdmap e82: 12 total, 12 up, 12 in 2024-09-11T21:55:20.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:20 smithi053 ceph-mon[34908]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T21:55:20.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:20.566+0000 7fed8b636640 1 -- 172.21.15.1:0/43592788 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed84101de0 msgr2=0x7fed841021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:20.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:20.566+0000 7fed8b636640 1 --2- 172.21.15.1:0/43592788 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed84101de0 0x7fed841021c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fed80009900 tx=0x7fed8002a410 comp rx=0 tx=0).stop 2024-09-11T21:55:20.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:20.566+0000 7fed8b636640 1 -- 172.21.15.1:0/43592788 shutdown_connections 2024-09-11T21:55:20.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:20.566+0000 7fed8b636640 1 --2- 172.21.15.1:0/43592788 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed84106f10 0x7fed8410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:20.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:20.566+0000 7fed8b636640 1 --2- 172.21.15.1:0/43592788 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed84102790 0x7fed841067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:20.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:20.566+0000 7fed8b636640 1 --2- 172.21.15.1:0/43592788 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed84101de0 0x7fed841021c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:20.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:20.566+0000 7fed8b636640 1 -- 172.21.15.1:0/43592788 >> 172.21.15.1:0/43592788 conn(0x7fed840fd8c0 msgr2=0x7fed840ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:55:20.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:20.566+0000 7fed8b636640 1 -- 172.21.15.1:0/43592788 shutdown_connections 2024-09-11T21:55:20.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:20.566+0000 7fed8b636640 1 -- 172.21.15.1:0/43592788 wait complete. 2024-09-11T21:55:20.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:20.567+0000 7fed8b636640 1 Processor -- start 2024-09-11T21:55:20.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:20.567+0000 7fed8b636640 1 -- start start 2024-09-11T21:55:20.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:20.567+0000 7fed8b636640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed84101de0 0x7fed84198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:20.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:20.567+0000 7fed8b636640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed84102790 0x7fed84198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:20.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:20.567+0000 7fed8b636640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed84106f10 0x7fed8419cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:20.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:20.567+0000 7fed8b636640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fed84111fe0 con 0x7fed84101de0 2024-09-11T21:55:20.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:20.567+0000 7fed8b636640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fed84111e60 con 0x7fed84102790 2024-09-11T21:55:20.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:20.567+0000 7fed8b636640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fed84112160 con 0x7fed84106f10 2024-09-11T21:55:20.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:20.567+0000 7fed893d2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed84101de0 0x7fed84198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:20.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:20.567+0000 7fed893d2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed84101de0 0x7fed84198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43580/0 (socket says 172.21.15.1:43580) 2024-09-11T21:55:20.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:20.567+0000 7fed893d2640 1 -- 172.21.15.1:0/1716377586 learned_addr learned my addr 172.21.15.1:0/1716377586 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:55:20.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:20.567+0000 7fed89bd3640 1 --2- 172.21.15.1:0/1716377586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed84106f10 0x7fed8419cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:20.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:20.567+0000 7fed88bd1640 1 --2- 172.21.15.1:0/1716377586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed84102790 0x7fed84198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:20.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:20.568+0000 7fed893d2640 1 -- 172.21.15.1:0/1716377586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed84106f10 msgr2=0x7fed8419cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:20.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:20.568+0000 7fed893d2640 1 --2- 172.21.15.1:0/1716377586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed84106f10 0x7fed8419cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:20.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:20.568+0000 7fed893d2640 1 -- 172.21.15.1:0/1716377586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed84102790 msgr2=0x7fed84198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:20.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:20.568+0000 7fed893d2640 1 --2- 172.21.15.1:0/1716377586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed84102790 0x7fed84198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:20.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:20.568+0000 7fed893d2640 1 -- 172.21.15.1:0/1716377586 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed8419d3f0 con 0x7fed84101de0 2024-09-11T21:55:20.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:20.568+0000 7fed88bd1640 1 --2- 172.21.15.1:0/1716377586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed84102790 0x7fed84198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T21:55:20.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:20.568+0000 7fed893d2640 1 --2- 172.21.15.1:0/1716377586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed84101de0 0x7fed84198440 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fed80004770 tx=0x7fed80003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:55:20.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:20.568+0000 7fed7a7fc640 1 -- 172.21.15.1:0/1716377586 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed8003f070 con 0x7fed84101de0 2024-09-11T21:55:20.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:20.568+0000 7fed7a7fc640 1 -- 172.21.15.1:0/1716377586 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed80031ca0 con 0x7fed84101de0 2024-09-11T21:55:20.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:20.568+0000 7fed8b636640 1 -- 172.21.15.1:0/1716377586 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed8419d680 con 0x7fed84101de0 2024-09-11T21:55:20.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:20.568+0000 7fed7a7fc640 1 -- 172.21.15.1:0/1716377586 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed80035030 con 0x7fed84101de0 2024-09-11T21:55:20.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:20.568+0000 7fed8b636640 1 -- 172.21.15.1:0/1716377586 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed841a4fc0 con 0x7fed84101de0 2024-09-11T21:55:20.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:20.569+0000 7fed8b636640 1 -- 172.21.15.1:0/1716377586 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed841032c0 con 0x7fed84101de0 2024-09-11T21:55:20.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:20.570+0000 7fed7a7fc640 1 -- 172.21.15.1:0/1716377586 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fed80042070 con 0x7fed84101de0 2024-09-11T21:55:20.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:20.571+0000 7fed7a7fc640 1 --2- 172.21.15.1:0/1716377586 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fed700a2c00 0x7fed700a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:20.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:20.571+0000 7fed88bd1640 1 --2- 172.21.15.1:0/1716377586 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fed700a2c00 0x7fed700a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:20.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:20.571+0000 7fed7a7fc640 1 -- 172.21.15.1:0/1716377586 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7879+0+0 (secure 0 0 0) 0x7fed800ef6b0 con 0x7fed84101de0 2024-09-11T21:55:20.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:20.571+0000 7fed88bd1640 1 --2- 172.21.15.1:0/1716377586 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fed700a2c00 0x7fed700a50c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fed841088e0 tx=0x7fed74005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:55:20.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:20.576+0000 7fed7a7fc640 1 -- 172.21.15.1:0/1716377586 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fed800bc7c0 con 0x7fed84101de0 2024-09-11T21:55:20.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:20.788+0000 7fed8b636640 1 -- 172.21.15.1:0/1716377586 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fed84109b60 con 0x7fed84101de0 2024-09-11T21:55:21.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:21.087+0000 7fed7a7fc640 1 -- 172.21.15.1:0/1716377586 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fed800bc0c0 con 0x7fed84101de0 2024-09-11T21:55:21.089 INFO:teuthology.orchestra.run.smithi001.stderr:new fs with metadata pool 2 and data pool 3 2024-09-11T21:55:21.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:21.089+0000 7fed8b636640 1 -- 172.21.15.1:0/1716377586 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fed700a2c00 msgr2=0x7fed700a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:21.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:21.089+0000 7fed8b636640 1 --2- 172.21.15.1:0/1716377586 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fed700a2c00 0x7fed700a50c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fed841088e0 tx=0x7fed74005e50 comp rx=0 tx=0).stop 2024-09-11T21:55:21.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:21.089+0000 7fed8b636640 1 -- 172.21.15.1:0/1716377586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed84101de0 msgr2=0x7fed84198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:21.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:21.089+0000 7fed8b636640 1 --2- 172.21.15.1:0/1716377586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed84101de0 0x7fed84198440 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fed80004770 tx=0x7fed80003c10 comp rx=0 tx=0).stop 2024-09-11T21:55:21.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:21.089+0000 7fed8b636640 1 -- 172.21.15.1:0/1716377586 shutdown_connections 2024-09-11T21:55:21.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:21.089+0000 7fed8b636640 1 --2- 172.21.15.1:0/1716377586 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fed700a2c00 0x7fed700a50c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:21.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:21.089+0000 7fed8b636640 1 --2- 172.21.15.1:0/1716377586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fed84106f10 0x7fed8419cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:21.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:21.089+0000 7fed8b636640 1 --2- 172.21.15.1:0/1716377586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fed84102790 0x7fed84198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:21.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:21.089+0000 7fed8b636640 1 --2- 172.21.15.1:0/1716377586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed84101de0 0x7fed84198440 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:21.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:21.089+0000 7fed8b636640 1 -- 172.21.15.1:0/1716377586 >> 172.21.15.1:0/1716377586 conn(0x7fed840fd8c0 msgr2=0x7fed840fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:55:21.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:21.090+0000 7fed8b636640 1 -- 172.21.15.1:0/1716377586 shutdown_connections 2024-09-11T21:55:21.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:21.090+0000 7fed8b636640 1 -- 172.21.15.1:0/1716377586 wait complete. 2024-09-11T21:55:21.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:21 smithi161 ceph-mon[35615]: osdmap e83: 12 total, 12 up, 12 in 2024-09-11T21:55:21.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:21 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/1716377586' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-11T21:55:21.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:21 smithi161 ceph-mon[35615]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-11T21:55:21.389 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:21 smithi001 ceph-mon[30935]: osdmap e83: 12 total, 12 up, 12 in 2024-09-11T21:55:21.389 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:21 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/1716377586' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-11T21:55:21.390 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:21 smithi001 ceph-mon[30935]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-11T21:55:21.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:21 smithi053 ceph-mon[34908]: osdmap e83: 12 total, 12 up, 12 in 2024-09-11T21:55:21.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:21 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/1716377586' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-11T21:55:21.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:21 smithi053 ceph-mon[34908]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-11T21:55:21.766 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T21:55:22.033 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:55:22.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:22 smithi161 ceph-mon[35615]: pgmap v281: 97 pgs: 42 creating+peering, 36 unknown, 19 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:55:22.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:22 smithi161 ceph-mon[35615]: osdmap e84: 12 total, 12 up, 12 in 2024-09-11T21:55:22.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:22 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/1716377586' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-11T21:55:22.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:22 smithi161 ceph-mon[35615]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-11T21:55:22.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:22 smithi161 ceph-mon[35615]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-11T21:55:22.348 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:22 smithi001 ceph-mon[30935]: pgmap v281: 97 pgs: 42 creating+peering, 36 unknown, 19 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:55:22.349 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:22 smithi001 ceph-mon[30935]: osdmap e84: 12 total, 12 up, 12 in 2024-09-11T21:55:22.349 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:22 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/1716377586' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-11T21:55:22.349 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:22 smithi001 ceph-mon[30935]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-11T21:55:22.349 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:22 smithi001 ceph-mon[30935]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-11T21:55:22.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:22 smithi053 ceph-mon[34908]: pgmap v281: 97 pgs: 42 creating+peering, 36 unknown, 19 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:55:22.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:22 smithi053 ceph-mon[34908]: osdmap e84: 12 total, 12 up, 12 in 2024-09-11T21:55:22.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:22 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/1716377586' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-11T21:55:22.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:22 smithi053 ceph-mon[34908]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-11T21:55:22.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:22 smithi053 ceph-mon[34908]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-11T21:55:22.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:22.540+0000 7fc595db0640 1 -- 172.21.15.1:0/1160422301 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc590100a80 msgr2=0x7fc590100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:22.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:22.540+0000 7fc595db0640 1 --2- 172.21.15.1:0/1160422301 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc590100a80 0x7fc590100ee0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc584009930 tx=0x7fc58402a420 comp rx=0 tx=0).stop 2024-09-11T21:55:22.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:22.540+0000 7fc595db0640 1 -- 172.21.15.1:0/1160422301 shutdown_connections 2024-09-11T21:55:22.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:22.540+0000 7fc595db0640 1 --2- 172.21.15.1:0/1160422301 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc590101420 0x7fc59010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:22.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:22.540+0000 7fc595db0640 1 --2- 172.21.15.1:0/1160422301 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc590100a80 0x7fc590100ee0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:22.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:22.540+0000 7fc595db0640 1 --2- 172.21.15.1:0/1160422301 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc590106a80 0x7fc590106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:22.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:22.540+0000 7fc595db0640 1 -- 172.21.15.1:0/1160422301 >> 172.21.15.1:0/1160422301 conn(0x7fc5900fc7a0 msgr2=0x7fc5900febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:55:22.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:22.540+0000 7fc595db0640 1 -- 172.21.15.1:0/1160422301 shutdown_connections 2024-09-11T21:55:22.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:22.540+0000 7fc595db0640 1 -- 172.21.15.1:0/1160422301 wait complete. 2024-09-11T21:55:22.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:22.541+0000 7fc595db0640 1 Processor -- start 2024-09-11T21:55:22.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:22.541+0000 7fc595db0640 1 -- start start 2024-09-11T21:55:22.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:22.542+0000 7fc595db0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc590100a80 0x7fc5901992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:22.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:22.542+0000 7fc595db0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc590101420 0x7fc5901997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:22.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:22.542+0000 7fc595db0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc590106a80 0x7fc5901a4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:22.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:22.542+0000 7fc595db0640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc590110e80 con 0x7fc590100a80 2024-09-11T21:55:22.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:22.542+0000 7fc595db0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc590110d00 con 0x7fc590106a80 2024-09-11T21:55:22.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:22.542+0000 7fc595db0640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc590111000 con 0x7fc590101420 2024-09-11T21:55:22.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:22.542+0000 7fc58f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc590100a80 0x7fc5901992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:22.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:22.542+0000 7fc58f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc590100a80 0x7fc5901992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43588/0 (socket says 172.21.15.1:43588) 2024-09-11T21:55:22.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:22.542+0000 7fc58f7fe640 1 -- 172.21.15.1:0/2305009332 learned_addr learned my addr 172.21.15.1:0/2305009332 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:55:22.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:22.542+0000 7fc58effd640 1 --2- 172.21.15.1:0/2305009332 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc590101420 0x7fc5901997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:22.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:22.542+0000 7fc58f7fe640 1 -- 172.21.15.1:0/2305009332 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc590101420 msgr2=0x7fc5901997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:22.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:22.542+0000 7fc58f7fe640 1 --2- 172.21.15.1:0/2305009332 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc590101420 0x7fc5901997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:22.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:22.542+0000 7fc58f7fe640 1 -- 172.21.15.1:0/2305009332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc590106a80 msgr2=0x7fc5901a4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:22.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:22.542+0000 7fc58ffff640 1 --2- 172.21.15.1:0/2305009332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc590106a80 0x7fc5901a4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:22.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:22.542+0000 7fc58f7fe640 1 --2- 172.21.15.1:0/2305009332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc590106a80 0x7fc5901a4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:22.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:22.542+0000 7fc58f7fe640 1 -- 172.21.15.1:0/2305009332 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5901a4870 con 0x7fc590100a80 2024-09-11T21:55:22.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:22.542+0000 7fc58f7fe640 1 --2- 172.21.15.1:0/2305009332 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc590100a80 0x7fc5901992b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc58000d950 tx=0x7fc58000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:55:22.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:22.542+0000 7fc58cff9640 1 -- 172.21.15.1:0/2305009332 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc580017070 con 0x7fc590100a80 2024-09-11T21:55:22.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:22.542+0000 7fc595db0640 1 -- 172.21.15.1:0/2305009332 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5901a4a40 con 0x7fc590100a80 2024-09-11T21:55:22.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:22.542+0000 7fc58cff9640 1 -- 172.21.15.1:0/2305009332 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc580009c30 con 0x7fc590100a80 2024-09-11T21:55:22.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:22.542+0000 7fc58cff9640 1 -- 172.21.15.1:0/2305009332 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc58000f6b0 con 0x7fc590100a80 2024-09-11T21:55:22.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:22.543+0000 7fc595db0640 1 -- 172.21.15.1:0/2305009332 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5901a4f00 con 0x7fc590100a80 2024-09-11T21:55:22.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:22.544+0000 7fc595db0640 1 -- 172.21.15.1:0/2305009332 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc590100ee0 con 0x7fc590100a80 2024-09-11T21:55:22.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:22.545+0000 7fc58cff9640 1 -- 172.21.15.1:0/2305009332 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fc5800109d0 con 0x7fc590100a80 2024-09-11T21:55:22.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:22.545+0000 7fc58cff9640 1 --2- 172.21.15.1:0/2305009332 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fc5780a2b80 0x7fc5780a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:22.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:22.545+0000 7fc58cff9640 1 -- 172.21.15.1:0/2305009332 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7fc5800ceb40 con 0x7fc590100a80 2024-09-11T21:55:22.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:22.545+0000 7fc58effd640 1 --2- 172.21.15.1:0/2305009332 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fc5780a2b80 0x7fc5780a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:22.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:22.546+0000 7fc58effd640 1 --2- 172.21.15.1:0/2305009332 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fc5780a2b80 0x7fc5780a5040 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc584000c00 tx=0x7fc584005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:55:22.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:22.550+0000 7fc58cff9640 1 -- 172.21.15.1:0/2305009332 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc580009de0 con 0x7fc590100a80 2024-09-11T21:55:22.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:22.726+0000 7fc595db0640 1 -- 172.21.15.1:0/2305009332 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc590109050 con 0x7fc590100a80 2024-09-11T21:55:22.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:22.728+0000 7fc58cff9640 1 -- 172.21.15.1:0/2305009332 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+22198 (secure 0 0 0) 0x7fc58009bbf0 con 0x7fc590100a80 2024-09-11T21:55:22.729 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:55:22.730 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":84,"fsid":"3ffca83c-7087-11ef-bcea-c7b262605968","created":"2024-09-11T21:47:28.868707+0000","modified":"2024-09-11T21:55:20.791073+0000","last_up_change":"2024-09-11T21:54:29.319441+0000","last_in_change":"2024-09-11T21:54:05.532281+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T21:50:41.026490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T21:55:16.961999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T21:55:19.066228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"19e56504-18f1-4234-b2d6-f193ff47d8eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":4012142023},{"type":"v1","addr":"172.21.15.1:6803","nonce":4012142023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":4012142023},{"type":"v1","addr":"172.21.15.1:6805","nonce":4012142023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":4012142023},{"type":"v1","addr":"172.21.15.1:6809","nonce":4012142023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":4012142023},{"type":"v1","addr":"172.21.15.1:6807","nonce":4012142023}]},"public_addr":"172.21.15.1:6803/4012142023","cluster_addr":"172.21.15.1:6805/4012142023","heartbeat_back_addr":"172.21.15.1:6809/4012142023","heartbeat_front_addr":"172.21.15.1:6807/4012142023","state":["exists","up"]},{"osd":1,"uuid":"996c9f0c-c5a4-43b8-b8e0-d712f27c5dfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3039391924},{"type":"v1","addr":"172.21.15.53:6801","nonce":3039391924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3039391924},{"type":"v1","addr":"172.21.15.53:6803","nonce":3039391924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3039391924},{"type":"v1","addr":"172.21.15.53:6807","nonce":3039391924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3039391924},{"type":"v1","addr":"172.21.15.53:6805","nonce":3039391924}]},"public_addr":"172.21.15.53:6801/3039391924","cluster_addr":"172.21.15.53:6803/3039391924","heartbeat_back_addr":"172.21.15.53:6807/3039391924","heartbeat_front_addr":"172.21.15.53:6805/3039391924","state":["exists","up"]},{"osd":2,"uuid":"578d4260-41e1-4645-b9b1-1c5445712c13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1707914885},{"type":"v1","addr":"172.21.15.161:6801","nonce":1707914885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1707914885},{"type":"v1","addr":"172.21.15.161:6803","nonce":1707914885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1707914885},{"type":"v1","addr":"172.21.15.161:6807","nonce":1707914885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1707914885},{"type":"v1","addr":"172.21.15.161:6805","nonce":1707914885}]},"public_addr":"172.21.15.161:6801/1707914885","cluster_addr":"172.21.15.161:6803/1707914885","heartbeat_back_addr":"172.21.15.161:6807/1707914885","heartbeat_front_addr":"172.21.15.161:6805/1707914885","state":["exists","up"]},{"osd":3,"uuid":"73f78766-9662-4aa3-841c-fe2ce654ee9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1541142398},{"type":"v1","addr":"172.21.15.1:6811","nonce":1541142398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1541142398},{"type":"v1","addr":"172.21.15.1:6813","nonce":1541142398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1541142398},{"type":"v1","addr":"172.21.15.1:6817","nonce":1541142398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1541142398},{"type":"v1","addr":"172.21.15.1:6815","nonce":1541142398}]},"public_addr":"172.21.15.1:6811/1541142398","cluster_addr":"172.21.15.1:6813/1541142398","heartbeat_back_addr":"172.21.15.1:6817/1541142398","heartbeat_front_addr":"172.21.15.1:6815/1541142398","state":["exists","up"]},{"osd":4,"uuid":"56c808bb-d4cd-4969-b49e-adef49ecd0f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":3081402681},{"type":"v1","addr":"172.21.15.53:6809","nonce":3081402681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3081402681},{"type":"v1","addr":"172.21.15.53:6811","nonce":3081402681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3081402681},{"type":"v1","addr":"172.21.15.53:6815","nonce":3081402681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3081402681},{"type":"v1","addr":"172.21.15.53:6813","nonce":3081402681}]},"public_addr":"172.21.15.53:6809/3081402681","cluster_addr":"172.21.15.53:6811/3081402681","heartbeat_back_addr":"172.21.15.53:6815/3081402681","heartbeat_front_addr":"172.21.15.53:6813/3081402681","state":["exists","up"]},{"osd":5,"uuid":"7b8266d6-b507-47c9-b4fa-ff16c4a74d35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":1622081327},{"type":"v1","addr":"172.21.15.161:6809","nonce":1622081327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":1622081327},{"type":"v1","addr":"172.21.15.161:6811","nonce":1622081327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":1622081327},{"type":"v1","addr":"172.21.15.161:6815","nonce":1622081327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":1622081327},{"type":"v1","addr":"172.21.15.161:6813","nonce":1622081327}]},"public_addr":"172.21.15.161:6809/1622081327","cluster_addr":"172.21.15.161:6811/1622081327","heartbeat_back_addr":"172.21.15.161:6815/1622081327","heartbeat_front_addr":"172.21.15.161:6813/1622081327","state":["exists","up"]},{"osd":6,"uuid":"3af5925d-a749-4849-a076-80580222cad4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":158434853},{"type":"v1","addr":"172.21.15.1:6819","nonce":158434853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":158434853},{"type":"v1","addr":"172.21.15.1:6821","nonce":158434853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":158434853},{"type":"v1","addr":"172.21.15.1:6825","nonce":158434853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":158434853},{"type":"v1","addr":"172.21.15.1:6823","nonce":158434853}]},"public_addr":"172.21.15.1:6819/158434853","cluster_addr":"172.21.15.1:6821/158434853","heartbeat_back_addr":"172.21.15.1:6825/158434853","heartbeat_front_addr":"172.21.15.1:6823/158434853","state":["exists","up"]},{"osd":7,"uuid":"f8f26c3c-17e9-4e89-8392-289796e0c8a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2132057697},{"type":"v1","addr":"172.21.15.53:6817","nonce":2132057697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2132057697},{"type":"v1","addr":"172.21.15.53:6819","nonce":2132057697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2132057697},{"type":"v1","addr":"172.21.15.53:6823","nonce":2132057697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2132057697},{"type":"v1","addr":"172.21.15.53:6821","nonce":2132057697}]},"public_addr":"172.21.15.53:6817/2132057697","cluster_addr":"172.21.15.53:6819/2132057697","heartbeat_back_addr":"172.21.15.53:6823/2132057697","heartbeat_front_addr":"172.21.15.53:6821/2132057697","state":["exists","up"]},{"osd":8,"uuid":"36ea401b-3c93-4b9d-8880-298d1eb6e30b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1838248669},{"type":"v1","addr":"172.21.15.161:6817","nonce":1838248669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1838248669},{"type":"v1","addr":"172.21.15.161:6819","nonce":1838248669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1838248669},{"type":"v1","addr":"172.21.15.161:6823","nonce":1838248669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1838248669},{"type":"v1","addr":"172.21.15.161:6821","nonce":1838248669}]},"public_addr":"172.21.15.161:6817/1838248669","cluster_addr":"172.21.15.161:6819/1838248669","heartbeat_back_addr":"172.21.15.161:6823/1838248669","heartbeat_front_addr":"172.21.15.161:6821/1838248669","state":["exists","up"]},{"osd":9,"uuid":"4b788d3f-a3bc-4463-bd99-95fdb7a52768","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2806274589},{"type":"v1","addr":"172.21.15.1:6827","nonce":2806274589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2806274589},{"type":"v1","addr":"172.21.15.1:6829","nonce":2806274589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":2806274589},{"type":"v1","addr":"172.21.15.1:6833","nonce":2806274589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2806274589},{"type":"v1","addr":"172.21.15.1:6831","nonce":2806274589}]},"public_addr":"172.21.15.1:6827/2806274589","cluster_addr":"172.21.15.1:6829/2806274589","heartbeat_back_addr":"172.21.15.1:6833/2806274589","heartbeat_front_addr":"172.21.15.1:6831/2806274589","state":["exists","up"]},{"osd":10,"uuid":"70549796-5270-4ae2-ac3f-f61f5de105d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":761923580},{"type":"v1","addr":"172.21.15.53:6825","nonce":761923580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":761923580},{"type":"v1","addr":"172.21.15.53:6827","nonce":761923580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":761923580},{"type":"v1","addr":"172.21.15.53:6831","nonce":761923580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":761923580},{"type":"v1","addr":"172.21.15.53:6829","nonce":761923580}]},"public_addr":"172.21.15.53:6825/761923580","cluster_addr":"172.21.15.53:6827/761923580","heartbeat_back_addr":"172.21.15.53:6831/761923580","heartbeat_front_addr":"172.21.15.53:6829/761923580","state":["exists","up"]},{"osd":11,"uuid":"48f17c4a-370a-493e-956e-bf6ad3dcc594","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1586352190},{"type":"v1","addr":"172.21.15.161:6825","nonce":1586352190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":1586352190},{"type":"v1","addr":"172.21.15.161:6827","nonce":1586352190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":1586352190},{"type":"v1","addr":"172.21.15.161:6831","nonce":1586352190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":1586352190},{"type":"v1","addr":"172.21.15.161:6829","nonce":1586352190}]},"public_addr":"172.21.15.161:6825/1586352190","cluster_addr":"172.21.15.161:6827/1586352190","heartbeat_back_addr":"172.21.15.161:6831/1586352190","heartbeat_front_addr":"172.21.15.161:6829/1586352190","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:49:46.539613+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:50:11.912300+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:50:36.955490+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:51:02.200554+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:51:27.860585+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:51:54.694099+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:52:18.962351+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:52:44.881923+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:53:10.301390+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:53:35.417647+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:54:01.492219+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:54:26.938338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/1749020850":"2024-09-12T21:48:24.524913+0000","172.21.15.1:6801/2821761445":"2024-09-12T21:48:24.524913+0000","172.21.15.1:0/1699984777":"2024-09-12T21:48:24.524913+0000","172.21.15.1:0/276770268":"2024-09-12T21:48:00.674728+0000","172.21.15.1:0/1278877732":"2024-09-12T21:48:24.524913+0000","172.21.15.1:0/2599895977":"2024-09-12T21:48:00.674728+0000","172.21.15.1:6801/2311309446":"2024-09-12T21:48:00.674728+0000","172.21.15.1:6800/2821761445":"2024-09-12T21:48:24.524913+0000","172.21.15.1:6800/2311309446":"2024-09-12T21:48:00.674728+0000","172.21.15.1:0/3202962495":"2024-09-12T21:48:00.674728+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T21:55:22.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:22.729+0000 7fc595db0640 1 -- 172.21.15.1:0/2305009332 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fc5780a2b80 msgr2=0x7fc5780a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:22.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:22.729+0000 7fc595db0640 1 --2- 172.21.15.1:0/2305009332 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fc5780a2b80 0x7fc5780a5040 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc584000c00 tx=0x7fc584005a20 comp rx=0 tx=0).stop 2024-09-11T21:55:22.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:22.729+0000 7fc595db0640 1 -- 172.21.15.1:0/2305009332 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc590100a80 msgr2=0x7fc5901992b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:22.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:22.729+0000 7fc595db0640 1 --2- 172.21.15.1:0/2305009332 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc590100a80 0x7fc5901992b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc58000d950 tx=0x7fc58000de10 comp rx=0 tx=0).stop 2024-09-11T21:55:22.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:22.730+0000 7fc595db0640 1 -- 172.21.15.1:0/2305009332 shutdown_connections 2024-09-11T21:55:22.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:22.730+0000 7fc595db0640 1 --2- 172.21.15.1:0/2305009332 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fc5780a2b80 0x7fc5780a5040 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:22.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:22.730+0000 7fc595db0640 1 --2- 172.21.15.1:0/2305009332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc590106a80 0x7fc5901a4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:22.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:22.730+0000 7fc595db0640 1 --2- 172.21.15.1:0/2305009332 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc590101420 0x7fc5901997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:22.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:22.730+0000 7fc595db0640 1 --2- 172.21.15.1:0/2305009332 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc590100a80 0x7fc5901992b0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:22.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:22.730+0000 7fc595db0640 1 -- 172.21.15.1:0/2305009332 >> 172.21.15.1:0/2305009332 conn(0x7fc5900fc7a0 msgr2=0x7fc5900fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:55:22.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:22.730+0000 7fc595db0640 1 -- 172.21.15.1:0/2305009332 shutdown_connections 2024-09-11T21:55:22.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:22.730+0000 7fc595db0640 1 -- 172.21.15.1:0/2305009332 wait complete. 2024-09-11T21:55:23.357 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:23 smithi001 ceph-mon[30935]: mds.? [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] up:active 2024-09-11T21:55:23.357 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:23 smithi001 ceph-mon[30935]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T21:55:23.357 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:23 smithi001 ceph-mon[30935]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T21:55:23.357 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:23 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/2305009332' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T21:55:23.387 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T21:55:23.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:23 smithi053 ceph-mon[34908]: mds.? [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] up:active 2024-09-11T21:55:23.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:23 smithi053 ceph-mon[34908]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T21:55:23.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:23 smithi053 ceph-mon[34908]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T21:55:23.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:23 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/2305009332' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T21:55:23.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:23 smithi161 ceph-mon[35615]: mds.? [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] up:active 2024-09-11T21:55:23.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:23 smithi161 ceph-mon[35615]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T21:55:23.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:23 smithi161 ceph-mon[35615]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T21:55:23.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:23 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/2305009332' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T21:55:23.648 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:55:24.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:24.143+0000 7f58f8ed6640 1 -- 172.21.15.1:0/1300519130 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58f4100310 msgr2=0x7f58f41006f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:24.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:24.143+0000 7f58f8ed6640 1 --2- 172.21.15.1:0/1300519130 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58f4100310 0x7f58f41006f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f58e4009900 tx=0x7f58e402a410 comp rx=0 tx=0).stop 2024-09-11T21:55:24.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:24.143+0000 7f58f8ed6640 1 -- 172.21.15.1:0/1300519130 shutdown_connections 2024-09-11T21:55:24.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:24.143+0000 7f58f8ed6640 1 --2- 172.21.15.1:0/1300519130 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58f4106f30 0x7f58f410eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:24.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:24.143+0000 7f58f8ed6640 1 --2- 172.21.15.1:0/1300519130 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58f4100cc0 0x7f58f4106800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:24.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:24.143+0000 7f58f8ed6640 1 --2- 172.21.15.1:0/1300519130 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58f4100310 0x7f58f41006f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:24.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:24.143+0000 7f58f8ed6640 1 -- 172.21.15.1:0/1300519130 >> 172.21.15.1:0/1300519130 conn(0x7f58f4077210 msgr2=0x7f58f40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:55:24.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:24.143+0000 7f58f8ed6640 1 -- 172.21.15.1:0/1300519130 shutdown_connections 2024-09-11T21:55:24.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:24.143+0000 7f58f8ed6640 1 -- 172.21.15.1:0/1300519130 wait complete. 2024-09-11T21:55:24.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:24.144+0000 7f58f8ed6640 1 Processor -- start 2024-09-11T21:55:24.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:24.144+0000 7f58f8ed6640 1 -- start start 2024-09-11T21:55:24.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:24.144+0000 7f58f8ed6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58f4100310 0x7f58f4198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:24.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:24.144+0000 7f58f8ed6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58f4100cc0 0x7f58f4198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:24.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:24.144+0000 7f58f8ed6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58f4106f30 0x7f58f419cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:24.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:24.144+0000 7f58f8ed6640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f58f4111f90 con 0x7f58f4106f30 2024-09-11T21:55:24.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:24.144+0000 7f58f8ed6640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f58f4111e10 con 0x7f58f4100310 2024-09-11T21:55:24.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:24.144+0000 7f58f8ed6640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f58f4112110 con 0x7f58f4100cc0 2024-09-11T21:55:24.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:24.144+0000 7f58f2d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58f4106f30 0x7f58f419cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:24.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:24.144+0000 7f58f2d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58f4106f30 0x7f58f419cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43602/0 (socket says 172.21.15.1:43602) 2024-09-11T21:55:24.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:24.144+0000 7f58f2d9d640 1 -- 172.21.15.1:0/4198796651 learned_addr learned my addr 172.21.15.1:0/4198796651 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:55:24.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:24.144+0000 7f58f1d9b640 1 --2- 172.21.15.1:0/4198796651 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58f4100cc0 0x7f58f4198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:24.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:24.144+0000 7f58f259c640 1 --2- 172.21.15.1:0/4198796651 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58f4100310 0x7f58f4198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:24.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:24.145+0000 7f58f2d9d640 1 -- 172.21.15.1:0/4198796651 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58f4100cc0 msgr2=0x7f58f4198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:24.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:24.145+0000 7f58f2d9d640 1 --2- 172.21.15.1:0/4198796651 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58f4100cc0 0x7f58f4198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:24.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:24.145+0000 7f58f2d9d640 1 -- 172.21.15.1:0/4198796651 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58f4100310 msgr2=0x7f58f4198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:24.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:24.145+0000 7f58f2d9d640 1 --2- 172.21.15.1:0/4198796651 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58f4100310 0x7f58f4198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:24.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:24.145+0000 7f58f2d9d640 1 -- 172.21.15.1:0/4198796651 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58f419d4b0 con 0x7f58f4106f30 2024-09-11T21:55:24.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:24.145+0000 7f58f2d9d640 1 --2- 172.21.15.1:0/4198796651 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58f4106f30 0x7f58f419cdd0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f58ec009fd0 tx=0x7f58ec00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:55:24.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:24.145+0000 7f58e37fe640 1 -- 172.21.15.1:0/4198796651 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58ec017070 con 0x7f58f4106f30 2024-09-11T21:55:24.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:24.145+0000 7f58e37fe640 1 -- 172.21.15.1:0/4198796651 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58ec003e50 con 0x7f58f4106f30 2024-09-11T21:55:24.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:24.145+0000 7f58e37fe640 1 -- 172.21.15.1:0/4198796651 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58ec0106c0 con 0x7f58f4106f30 2024-09-11T21:55:24.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:24.145+0000 7f58f8ed6640 1 -- 172.21.15.1:0/4198796651 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58f419d7a0 con 0x7f58f4106f30 2024-09-11T21:55:24.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:24.145+0000 7f58f8ed6640 1 -- 172.21.15.1:0/4198796651 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58f41a5080 con 0x7f58f4106f30 2024-09-11T21:55:24.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:24.146+0000 7f58f8ed6640 1 -- 172.21.15.1:0/4198796651 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58f41006f0 con 0x7f58f4106f30 2024-09-11T21:55:24.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:24.147+0000 7f58e37fe640 1 -- 172.21.15.1:0/4198796651 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f58ec003970 con 0x7f58f4106f30 2024-09-11T21:55:24.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:24.148+0000 7f58e37fe640 1 --2- 172.21.15.1:0/4198796651 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f58dc0a2c00 0x7f58dc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:24.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:24.148+0000 7f58f259c640 1 --2- 172.21.15.1:0/4198796651 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f58dc0a2c00 0x7f58dc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:24.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:24.148+0000 7f58e37fe640 1 -- 172.21.15.1:0/4198796651 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f58ec0ceaa0 con 0x7f58f4106f30 2024-09-11T21:55:24.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:24.149+0000 7f58f259c640 1 --2- 172.21.15.1:0/4198796651 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f58dc0a2c00 0x7f58dc0a50c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f58e4009230 tx=0x7f58e40058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:55:24.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:24.151+0000 7f58e37fe640 1 -- 172.21.15.1:0/4198796651 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f58ec09b580 con 0x7f58f4106f30 2024-09-11T21:55:24.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:24.326+0000 7f58f8ed6640 1 -- 172.21.15.1:0/4198796651 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f58f4199860 con 0x7f58f4106f30 2024-09-11T21:55:24.328 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:24 smithi001 ceph-mon[30935]: pgmap v283: 97 pgs: 37 creating+peering, 6 unknown, 54 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 446 B/s wr, 1 op/s 2024-09-11T21:55:24.328 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:24 smithi001 ceph-mon[30935]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-11T21:55:24.328 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:24 smithi001 ceph-mon[30935]: Cluster is now healthy 2024-09-11T21:55:24.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:24.328+0000 7f58e37fe640 1 -- 172.21.15.1:0/4198796651 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+22198 (secure 0 0 0) 0x7f58f4199860 con 0x7f58f4106f30 2024-09-11T21:55:24.330 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:55:24.330 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":84,"fsid":"3ffca83c-7087-11ef-bcea-c7b262605968","created":"2024-09-11T21:47:28.868707+0000","modified":"2024-09-11T21:55:20.791073+0000","last_up_change":"2024-09-11T21:54:29.319441+0000","last_in_change":"2024-09-11T21:54:05.532281+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T21:50:41.026490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T21:55:16.961999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T21:55:19.066228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"19e56504-18f1-4234-b2d6-f193ff47d8eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":4012142023},{"type":"v1","addr":"172.21.15.1:6803","nonce":4012142023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":4012142023},{"type":"v1","addr":"172.21.15.1:6805","nonce":4012142023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":4012142023},{"type":"v1","addr":"172.21.15.1:6809","nonce":4012142023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":4012142023},{"type":"v1","addr":"172.21.15.1:6807","nonce":4012142023}]},"public_addr":"172.21.15.1:6803/4012142023","cluster_addr":"172.21.15.1:6805/4012142023","heartbeat_back_addr":"172.21.15.1:6809/4012142023","heartbeat_front_addr":"172.21.15.1:6807/4012142023","state":["exists","up"]},{"osd":1,"uuid":"996c9f0c-c5a4-43b8-b8e0-d712f27c5dfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3039391924},{"type":"v1","addr":"172.21.15.53:6801","nonce":3039391924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3039391924},{"type":"v1","addr":"172.21.15.53:6803","nonce":3039391924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3039391924},{"type":"v1","addr":"172.21.15.53:6807","nonce":3039391924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3039391924},{"type":"v1","addr":"172.21.15.53:6805","nonce":3039391924}]},"public_addr":"172.21.15.53:6801/3039391924","cluster_addr":"172.21.15.53:6803/3039391924","heartbeat_back_addr":"172.21.15.53:6807/3039391924","heartbeat_front_addr":"172.21.15.53:6805/3039391924","state":["exists","up"]},{"osd":2,"uuid":"578d4260-41e1-4645-b9b1-1c5445712c13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1707914885},{"type":"v1","addr":"172.21.15.161:6801","nonce":1707914885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1707914885},{"type":"v1","addr":"172.21.15.161:6803","nonce":1707914885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1707914885},{"type":"v1","addr":"172.21.15.161:6807","nonce":1707914885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1707914885},{"type":"v1","addr":"172.21.15.161:6805","nonce":1707914885}]},"public_addr":"172.21.15.161:6801/1707914885","cluster_addr":"172.21.15.161:6803/1707914885","heartbeat_back_addr":"172.21.15.161:6807/1707914885","heartbeat_front_addr":"172.21.15.161:6805/1707914885","state":["exists","up"]},{"osd":3,"uuid":"73f78766-9662-4aa3-841c-fe2ce654ee9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1541142398},{"type":"v1","addr":"172.21.15.1:6811","nonce":1541142398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1541142398},{"type":"v1","addr":"172.21.15.1:6813","nonce":1541142398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1541142398},{"type":"v1","addr":"172.21.15.1:6817","nonce":1541142398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1541142398},{"type":"v1","addr":"172.21.15.1:6815","nonce":1541142398}]},"public_addr":"172.21.15.1:6811/1541142398","cluster_addr":"172.21.15.1:6813/1541142398","heartbeat_back_addr":"172.21.15.1:6817/1541142398","heartbeat_front_addr":"172.21.15.1:6815/1541142398","state":["exists","up"]},{"osd":4,"uuid":"56c808bb-d4cd-4969-b49e-adef49ecd0f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":3081402681},{"type":"v1","addr":"172.21.15.53:6809","nonce":3081402681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3081402681},{"type":"v1","addr":"172.21.15.53:6811","nonce":3081402681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3081402681},{"type":"v1","addr":"172.21.15.53:6815","nonce":3081402681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3081402681},{"type":"v1","addr":"172.21.15.53:6813","nonce":3081402681}]},"public_addr":"172.21.15.53:6809/3081402681","cluster_addr":"172.21.15.53:6811/3081402681","heartbeat_back_addr":"172.21.15.53:6815/3081402681","heartbeat_front_addr":"172.21.15.53:6813/3081402681","state":["exists","up"]},{"osd":5,"uuid":"7b8266d6-b507-47c9-b4fa-ff16c4a74d35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":1622081327},{"type":"v1","addr":"172.21.15.161:6809","nonce":1622081327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":1622081327},{"type":"v1","addr":"172.21.15.161:6811","nonce":1622081327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":1622081327},{"type":"v1","addr":"172.21.15.161:6815","nonce":1622081327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":1622081327},{"type":"v1","addr":"172.21.15.161:6813","nonce":1622081327}]},"public_addr":"172.21.15.161:6809/1622081327","cluster_addr":"172.21.15.161:6811/1622081327","heartbeat_back_addr":"172.21.15.161:6815/1622081327","heartbeat_front_addr":"172.21.15.161:6813/1622081327","state":["exists","up"]},{"osd":6,"uuid":"3af5925d-a749-4849-a076-80580222cad4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":158434853},{"type":"v1","addr":"172.21.15.1:6819","nonce":158434853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":158434853},{"type":"v1","addr":"172.21.15.1:6821","nonce":158434853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":158434853},{"type":"v1","addr":"172.21.15.1:6825","nonce":158434853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":158434853},{"type":"v1","addr":"172.21.15.1:6823","nonce":158434853}]},"public_addr":"172.21.15.1:6819/158434853","cluster_addr":"172.21.15.1:6821/158434853","heartbeat_back_addr":"172.21.15.1:6825/158434853","heartbeat_front_addr":"172.21.15.1:6823/158434853","state":["exists","up"]},{"osd":7,"uuid":"f8f26c3c-17e9-4e89-8392-289796e0c8a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2132057697},{"type":"v1","addr":"172.21.15.53:6817","nonce":2132057697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2132057697},{"type":"v1","addr":"172.21.15.53:6819","nonce":2132057697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2132057697},{"type":"v1","addr":"172.21.15.53:6823","nonce":2132057697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2132057697},{"type":"v1","addr":"172.21.15.53:6821","nonce":2132057697}]},"public_addr":"172.21.15.53:6817/2132057697","cluster_addr":"172.21.15.53:6819/2132057697","heartbeat_back_addr":"172.21.15.53:6823/2132057697","heartbeat_front_addr":"172.21.15.53:6821/2132057697","state":["exists","up"]},{"osd":8,"uuid":"36ea401b-3c93-4b9d-8880-298d1eb6e30b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1838248669},{"type":"v1","addr":"172.21.15.161:6817","nonce":1838248669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1838248669},{"type":"v1","addr":"172.21.15.161:6819","nonce":1838248669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1838248669},{"type":"v1","addr":"172.21.15.161:6823","nonce":1838248669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1838248669},{"type":"v1","addr":"172.21.15.161:6821","nonce":1838248669}]},"public_addr":"172.21.15.161:6817/1838248669","cluster_addr":"172.21.15.161:6819/1838248669","heartbeat_back_addr":"172.21.15.161:6823/1838248669","heartbeat_front_addr":"172.21.15.161:6821/1838248669","state":["exists","up"]},{"osd":9,"uuid":"4b788d3f-a3bc-4463-bd99-95fdb7a52768","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2806274589},{"type":"v1","addr":"172.21.15.1:6827","nonce":2806274589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2806274589},{"type":"v1","addr":"172.21.15.1:6829","nonce":2806274589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":2806274589},{"type":"v1","addr":"172.21.15.1:6833","nonce":2806274589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2806274589},{"type":"v1","addr":"172.21.15.1:6831","nonce":2806274589}]},"public_addr":"172.21.15.1:6827/2806274589","cluster_addr":"172.21.15.1:6829/2806274589","heartbeat_back_addr":"172.21.15.1:6833/2806274589","heartbeat_front_addr":"172.21.15.1:6831/2806274589","state":["exists","up"]},{"osd":10,"uuid":"70549796-5270-4ae2-ac3f-f61f5de105d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":761923580},{"type":"v1","addr":"172.21.15.53:6825","nonce":761923580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":761923580},{"type":"v1","addr":"172.21.15.53:6827","nonce":761923580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":761923580},{"type":"v1","addr":"172.21.15.53:6831","nonce":761923580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":761923580},{"type":"v1","addr":"172.21.15.53:6829","nonce":761923580}]},"public_addr":"172.21.15.53:6825/761923580","cluster_addr":"172.21.15.53:6827/761923580","heartbeat_back_addr":"172.21.15.53:6831/761923580","heartbeat_front_addr":"172.21.15.53:6829/761923580","state":["exists","up"]},{"osd":11,"uuid":"48f17c4a-370a-493e-956e-bf6ad3dcc594","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1586352190},{"type":"v1","addr":"172.21.15.161:6825","nonce":1586352190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":1586352190},{"type":"v1","addr":"172.21.15.161:6827","nonce":1586352190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":1586352190},{"type":"v1","addr":"172.21.15.161:6831","nonce":1586352190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":1586352190},{"type":"v1","addr":"172.21.15.161:6829","nonce":1586352190}]},"public_addr":"172.21.15.161:6825/1586352190","cluster_addr":"172.21.15.161:6827/1586352190","heartbeat_back_addr":"172.21.15.161:6831/1586352190","heartbeat_front_addr":"172.21.15.161:6829/1586352190","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:49:46.539613+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:50:11.912300+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:50:36.955490+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:51:02.200554+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:51:27.860585+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:51:54.694099+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:52:18.962351+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:52:44.881923+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:53:10.301390+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:53:35.417647+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:54:01.492219+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:54:26.938338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/1749020850":"2024-09-12T21:48:24.524913+0000","172.21.15.1:6801/2821761445":"2024-09-12T21:48:24.524913+0000","172.21.15.1:0/1699984777":"2024-09-12T21:48:24.524913+0000","172.21.15.1:0/276770268":"2024-09-12T21:48:00.674728+0000","172.21.15.1:0/1278877732":"2024-09-12T21:48:24.524913+0000","172.21.15.1:0/2599895977":"2024-09-12T21:48:00.674728+0000","172.21.15.1:6801/2311309446":"2024-09-12T21:48:00.674728+0000","172.21.15.1:6800/2821761445":"2024-09-12T21:48:24.524913+0000","172.21.15.1:6800/2311309446":"2024-09-12T21:48:00.674728+0000","172.21.15.1:0/3202962495":"2024-09-12T21:48:00.674728+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T21:55:24.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:24.330+0000 7f58f8ed6640 1 -- 172.21.15.1:0/4198796651 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f58dc0a2c00 msgr2=0x7f58dc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:24.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:24.330+0000 7f58f8ed6640 1 --2- 172.21.15.1:0/4198796651 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f58dc0a2c00 0x7f58dc0a50c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f58e4009230 tx=0x7f58e40058d0 comp rx=0 tx=0).stop 2024-09-11T21:55:24.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:24.330+0000 7f58f8ed6640 1 -- 172.21.15.1:0/4198796651 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58f4106f30 msgr2=0x7f58f419cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:24.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:24.330+0000 7f58f8ed6640 1 --2- 172.21.15.1:0/4198796651 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58f4106f30 0x7f58f419cdd0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f58ec009fd0 tx=0x7f58ec00b260 comp rx=0 tx=0).stop 2024-09-11T21:55:24.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:24.330+0000 7f58f8ed6640 1 -- 172.21.15.1:0/4198796651 shutdown_connections 2024-09-11T21:55:24.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:24.330+0000 7f58f8ed6640 1 --2- 172.21.15.1:0/4198796651 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f58dc0a2c00 0x7f58dc0a50c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:24.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:24.330+0000 7f58f8ed6640 1 --2- 172.21.15.1:0/4198796651 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58f4106f30 0x7f58f419cdd0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:24.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:24.330+0000 7f58f8ed6640 1 --2- 172.21.15.1:0/4198796651 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58f4100cc0 0x7f58f4198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:24.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:24.330+0000 7f58f8ed6640 1 --2- 172.21.15.1:0/4198796651 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f58f4100310 0x7f58f4198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:24.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:24.330+0000 7f58f8ed6640 1 -- 172.21.15.1:0/4198796651 >> 172.21.15.1:0/4198796651 conn(0x7f58f4077210 msgr2=0x7f58f40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:55:24.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:24.330+0000 7f58f8ed6640 1 -- 172.21.15.1:0/4198796651 shutdown_connections 2024-09-11T21:55:24.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:24.330+0000 7f58f8ed6640 1 -- 172.21.15.1:0/4198796651 wait complete. 2024-09-11T21:55:24.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:24 smithi053 ceph-mon[34908]: pgmap v283: 97 pgs: 37 creating+peering, 6 unknown, 54 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 446 B/s wr, 1 op/s 2024-09-11T21:55:24.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:24 smithi053 ceph-mon[34908]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-11T21:55:24.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:24 smithi053 ceph-mon[34908]: Cluster is now healthy 2024-09-11T21:55:24.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:24 smithi161 ceph-mon[35615]: pgmap v283: 97 pgs: 37 creating+peering, 6 unknown, 54 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 446 B/s wr, 1 op/s 2024-09-11T21:55:24.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:24 smithi161 ceph-mon[35615]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-11T21:55:24.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:24 smithi161 ceph-mon[35615]: Cluster is now healthy 2024-09-11T21:55:25.045 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-11T21:55:25.304 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:55:25.425 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:25 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/4198796651' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T21:55:25.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:25 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/4198796651' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T21:55:25.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:25 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/4198796651' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T21:55:25.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:25.760+0000 7f21e0af3640 1 -- 172.21.15.1:0/115436860 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f21dc101420 msgr2=0x7f21dc10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:25.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:25.760+0000 7f21e0af3640 1 --2- 172.21.15.1:0/115436860 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f21dc101420 0x7f21dc10d950 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f21d4009cd0 tx=0x7f21d402a8e0 comp rx=0 tx=0).stop 2024-09-11T21:55:25.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:25.760+0000 7f21e0af3640 1 -- 172.21.15.1:0/115436860 shutdown_connections 2024-09-11T21:55:25.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:25.760+0000 7f21e0af3640 1 --2- 172.21.15.1:0/115436860 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f21dc101420 0x7f21dc10d950 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:25.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:25.760+0000 7f21e0af3640 1 --2- 172.21.15.1:0/115436860 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f21dc100a80 0x7f21dc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:25.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:25.760+0000 7f21e0af3640 1 --2- 172.21.15.1:0/115436860 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f21dc106a80 0x7f21dc106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:25.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:25.760+0000 7f21e0af3640 1 -- 172.21.15.1:0/115436860 >> 172.21.15.1:0/115436860 conn(0x7f21dc0fc7a0 msgr2=0x7f21dc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:55:25.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:25.760+0000 7f21e0af3640 1 -- 172.21.15.1:0/115436860 shutdown_connections 2024-09-11T21:55:25.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:25.760+0000 7f21e0af3640 1 -- 172.21.15.1:0/115436860 wait complete. 2024-09-11T21:55:25.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:25.761+0000 7f21e0af3640 1 Processor -- start 2024-09-11T21:55:25.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:25.761+0000 7f21e0af3640 1 -- start start 2024-09-11T21:55:25.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:25.761+0000 7f21e0af3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f21dc100a80 0x7f21dc198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:25.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:25.761+0000 7f21e0af3640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f21dc101420 0x7f21dc198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:25.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:25.761+0000 7f21e0af3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f21dc106a80 0x7f21dc19ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:25.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:25.761+0000 7f21e0af3640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f21dc111fe0 con 0x7f21dc106a80 2024-09-11T21:55:25.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:25.761+0000 7f21e0af3640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f21dc111e60 con 0x7f21dc100a80 2024-09-11T21:55:25.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:25.761+0000 7f21e0af3640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f21dc112160 con 0x7f21dc101420 2024-09-11T21:55:25.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:25.762+0000 7f21dad9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f21dc106a80 0x7f21dc19ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:25.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:25.762+0000 7f21dad9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f21dc106a80 0x7f21dc19ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43624/0 (socket says 172.21.15.1:43624) 2024-09-11T21:55:25.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:25.762+0000 7f21dad9d640 1 -- 172.21.15.1:0/461727103 learned_addr learned my addr 172.21.15.1:0/461727103 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:55:25.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:25.762+0000 7f21d1d9b640 1 --2- 172.21.15.1:0/461727103 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f21dc101420 0x7f21dc198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:25.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:25.762+0000 7f21da59c640 1 --2- 172.21.15.1:0/461727103 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f21dc100a80 0x7f21dc198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:25.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:25.762+0000 7f21dad9d640 1 -- 172.21.15.1:0/461727103 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f21dc101420 msgr2=0x7f21dc198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:25.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:25.762+0000 7f21dad9d640 1 --2- 172.21.15.1:0/461727103 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f21dc101420 0x7f21dc198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:25.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:25.762+0000 7f21dad9d640 1 -- 172.21.15.1:0/461727103 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f21dc100a80 msgr2=0x7f21dc198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:25.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:25.762+0000 7f21dad9d640 1 --2- 172.21.15.1:0/461727103 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f21dc100a80 0x7f21dc198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:25.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:25.762+0000 7f21dad9d640 1 -- 172.21.15.1:0/461727103 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21dc19d3b0 con 0x7f21dc106a80 2024-09-11T21:55:25.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:25.762+0000 7f21da59c640 1 --2- 172.21.15.1:0/461727103 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f21dc100a80 0x7f21dc198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T21:55:25.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:25.762+0000 7f21dad9d640 1 --2- 172.21.15.1:0/461727103 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f21dc106a80 0x7f21dc19ccd0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f21d4033040 tx=0x7f21d4003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:55:25.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:25.762+0000 7f21d3fff640 1 -- 172.21.15.1:0/461727103 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d4004040 con 0x7f21dc106a80 2024-09-11T21:55:25.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:25.762+0000 7f21d3fff640 1 -- 172.21.15.1:0/461727103 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21d40046c0 con 0x7f21dc106a80 2024-09-11T21:55:25.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:25.762+0000 7f21e0af3640 1 -- 172.21.15.1:0/461727103 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21dc19d640 con 0x7f21dc106a80 2024-09-11T21:55:25.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:25.762+0000 7f21d3fff640 1 -- 172.21.15.1:0/461727103 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d4046480 con 0x7f21dc106a80 2024-09-11T21:55:25.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:25.762+0000 7f21e0af3640 1 -- 172.21.15.1:0/461727103 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21dc1a4f80 con 0x7f21dc106a80 2024-09-11T21:55:25.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:25.763+0000 7f21e0af3640 1 -- 172.21.15.1:0/461727103 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21dc100ee0 con 0x7f21dc106a80 2024-09-11T21:55:25.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:25.764+0000 7f21d3fff640 1 -- 172.21.15.1:0/461727103 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f21d40305a0 con 0x7f21dc106a80 2024-09-11T21:55:25.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:25.765+0000 7f21d3fff640 1 --2- 172.21.15.1:0/461727103 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f21c80a2c00 0x7f21c80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:25.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:25.765+0000 7f21da59c640 1 --2- 172.21.15.1:0/461727103 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f21c80a2c00 0x7f21c80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:25.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:25.765+0000 7f21d3fff640 1 -- 172.21.15.1:0/461727103 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f21d402e030 con 0x7f21dc106a80 2024-09-11T21:55:25.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:25.766+0000 7f21da59c640 1 --2- 172.21.15.1:0/461727103 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f21c80a2c00 0x7f21c80a50c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f21cc005fd0 tx=0x7f21cc0094d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:55:25.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:25.769+0000 7f21d3fff640 1 -- 172.21.15.1:0/461727103 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f21d404b060 con 0x7f21dc106a80 2024-09-11T21:55:25.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:25.982+0000 7f21e0af3640 1 -- 172.21.15.1:0/461727103 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f21dc1a51e0 con 0x7f21dc106a80 2024-09-11T21:55:26.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:26.105+0000 7f21d3fff640 1 -- 172.21.15.1:0/461727103 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f21d40bd6d0 con 0x7f21dc106a80 2024-09-11T21:55:26.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:26.108+0000 7f21e0af3640 1 -- 172.21.15.1:0/461727103 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f21c80a2c00 msgr2=0x7f21c80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:26.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:26.108+0000 7f21e0af3640 1 --2- 172.21.15.1:0/461727103 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f21c80a2c00 0x7f21c80a50c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f21cc005fd0 tx=0x7f21cc0094d0 comp rx=0 tx=0).stop 2024-09-11T21:55:26.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:26.108+0000 7f21e0af3640 1 -- 172.21.15.1:0/461727103 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f21dc106a80 msgr2=0x7f21dc19ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:26.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:26.108+0000 7f21e0af3640 1 --2- 172.21.15.1:0/461727103 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f21dc106a80 0x7f21dc19ccd0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f21d4033040 tx=0x7f21d4003970 comp rx=0 tx=0).stop 2024-09-11T21:55:26.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:26.108+0000 7f21e0af3640 1 -- 172.21.15.1:0/461727103 shutdown_connections 2024-09-11T21:55:26.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:26.108+0000 7f21e0af3640 1 --2- 172.21.15.1:0/461727103 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f21c80a2c00 0x7f21c80a50c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:26.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:26.108+0000 7f21e0af3640 1 --2- 172.21.15.1:0/461727103 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f21dc106a80 0x7f21dc19ccd0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:26.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:26.108+0000 7f21e0af3640 1 --2- 172.21.15.1:0/461727103 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f21dc101420 0x7f21dc198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:26.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:26.108+0000 7f21e0af3640 1 --2- 172.21.15.1:0/461727103 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f21dc100a80 0x7f21dc198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:26.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:26.108+0000 7f21e0af3640 1 -- 172.21.15.1:0/461727103 >> 172.21.15.1:0/461727103 conn(0x7f21dc0fc7a0 msgr2=0x7f21dc0fcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:55:26.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:26.108+0000 7f21e0af3640 1 -- 172.21.15.1:0/461727103 shutdown_connections 2024-09-11T21:55:26.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:26.108+0000 7f21e0af3640 1 -- 172.21.15.1:0/461727103 wait complete. 2024-09-11T21:55:26.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:26 smithi001 ceph-mon[30935]: pgmap v284: 97 pgs: 11 creating+peering, 86 active+clean; 579 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 6 op/s 2024-09-11T21:55:26.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:26 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/461727103' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-11T21:55:26.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:26 smithi053 ceph-mon[34908]: pgmap v284: 97 pgs: 11 creating+peering, 86 active+clean; 579 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 6 op/s 2024-09-11T21:55:26.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:26 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/461727103' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-11T21:55:26.548 INFO:journalctl@ceph.mds.b.smithi053.stdout:Sep 11 21:55:26 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-b[64373]: 2024-09-11T21:55:26.306+0000 7f17b78d0640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-11T21:55:26.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:26 smithi161 ceph-mon[35615]: pgmap v284: 97 pgs: 11 creating+peering, 86 active+clean; 579 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 6 op/s 2024-09-11T21:55:26.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:26 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/461727103' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-11T21:55:26.828 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-09-11T21:55:26.829 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-09-11T21:55:27.085 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:55:27.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:27 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/461727103' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-11T21:55:27.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:27 smithi001 ceph-mon[30935]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T21:55:27.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:27 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/461727103' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-11T21:55:27.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:27 smithi053 ceph-mon[34908]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T21:55:27.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:27.553+0000 7f0c15363640 1 -- 172.21.15.1:0/2548162747 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c10106a80 msgr2=0x7f0c10106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:27.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:27.553+0000 7f0c15363640 1 --2- 172.21.15.1:0/2548162747 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c10106a80 0x7f0c10106e60 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0c00009930 tx=0x7f0c0002a420 comp rx=0 tx=0).stop 2024-09-11T21:55:27.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:27.553+0000 7f0c15363640 1 -- 172.21.15.1:0/2548162747 shutdown_connections 2024-09-11T21:55:27.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:27.553+0000 7f0c15363640 1 --2- 172.21.15.1:0/2548162747 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c10101420 0x7f0c1010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:27.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:27.553+0000 7f0c15363640 1 --2- 172.21.15.1:0/2548162747 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c10100a80 0x7f0c10100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:27.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:27.553+0000 7f0c15363640 1 --2- 172.21.15.1:0/2548162747 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c10106a80 0x7f0c10106e60 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:27.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:27.553+0000 7f0c15363640 1 -- 172.21.15.1:0/2548162747 >> 172.21.15.1:0/2548162747 conn(0x7f0c100fc7a0 msgr2=0x7f0c100febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:55:27.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:27.553+0000 7f0c15363640 1 -- 172.21.15.1:0/2548162747 shutdown_connections 2024-09-11T21:55:27.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:27.553+0000 7f0c15363640 1 -- 172.21.15.1:0/2548162747 wait complete. 2024-09-11T21:55:27.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:27.554+0000 7f0c15363640 1 Processor -- start 2024-09-11T21:55:27.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:27.554+0000 7f0c15363640 1 -- start start 2024-09-11T21:55:27.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:27.554+0000 7f0c15363640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c10100a80 0x7f0c10198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:27.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:27.554+0000 7f0c15363640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c10101420 0x7f0c101989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:27.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:27.554+0000 7f0c15363640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c10106a80 0x7f0c1019cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:27.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:27.554+0000 7f0c15363640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0c10110e80 con 0x7f0c10101420 2024-09-11T21:55:27.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:27.554+0000 7f0c15363640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0c10110d00 con 0x7f0c10106a80 2024-09-11T21:55:27.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:27.554+0000 7f0c15363640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0c10111000 con 0x7f0c10100a80 2024-09-11T21:55:27.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:27.554+0000 7f0c0e7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c10101420 0x7f0c101989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:27.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:27.554+0000 7f0c0e7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c10101420 0x7f0c101989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45748/0 (socket says 172.21.15.1:45748) 2024-09-11T21:55:27.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:27.554+0000 7f0c0e7fc640 1 -- 172.21.15.1:0/1876560186 learned_addr learned my addr 172.21.15.1:0/1876560186 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:55:27.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:27.554+0000 7f0c0effd640 1 --2- 172.21.15.1:0/1876560186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c10100a80 0x7f0c10198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:27.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:27.554+0000 7f0c0f7fe640 1 --2- 172.21.15.1:0/1876560186 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c10106a80 0x7f0c1019cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:27.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:27.555+0000 7f0c0e7fc640 1 -- 172.21.15.1:0/1876560186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c10100a80 msgr2=0x7f0c10198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:27.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:27.555+0000 7f0c0e7fc640 1 --2- 172.21.15.1:0/1876560186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c10100a80 0x7f0c10198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:27.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:27.555+0000 7f0c0e7fc640 1 -- 172.21.15.1:0/1876560186 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c10106a80 msgr2=0x7f0c1019cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:27.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:27.555+0000 7f0c0e7fc640 1 --2- 172.21.15.1:0/1876560186 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c10106a80 0x7f0c1019cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:27.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:27.555+0000 7f0c0e7fc640 1 -- 172.21.15.1:0/1876560186 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c1019d410 con 0x7f0c10101420 2024-09-11T21:55:27.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:27.555+0000 7f0c0e7fc640 1 --2- 172.21.15.1:0/1876560186 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c10101420 0x7f0c101989a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0c0400e9f0 tx=0x7f0c0400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:55:27.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:27.555+0000 7f0bf7fff640 1 -- 172.21.15.1:0/1876560186 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c0400cd80 con 0x7f0c10101420 2024-09-11T21:55:27.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:27.555+0000 7f0c15363640 1 -- 172.21.15.1:0/1876560186 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c1019d700 con 0x7f0c10101420 2024-09-11T21:55:27.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:27.555+0000 7f0bf7fff640 1 -- 172.21.15.1:0/1876560186 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c04010910 con 0x7f0c10101420 2024-09-11T21:55:27.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:27.555+0000 7f0bf7fff640 1 -- 172.21.15.1:0/1876560186 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c040188f0 con 0x7f0c10101420 2024-09-11T21:55:27.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:27.555+0000 7f0c15363640 1 -- 172.21.15.1:0/1876560186 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c101a4fe0 con 0x7f0c10101420 2024-09-11T21:55:27.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:27.556+0000 7f0c15363640 1 -- 172.21.15.1:0/1876560186 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c10100ee0 con 0x7f0c10101420 2024-09-11T21:55:27.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:27.557+0000 7f0bf7fff640 1 -- 172.21.15.1:0/1876560186 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f0c04010ac0 con 0x7f0c10101420 2024-09-11T21:55:27.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:27.558+0000 7f0bf7fff640 1 --2- 172.21.15.1:0/1876560186 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f0bf80a2b80 0x7f0bf80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:27.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:27.558+0000 7f0c0effd640 1 --2- 172.21.15.1:0/1876560186 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f0bf80a2b80 0x7f0bf80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:27.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:27.558+0000 7f0bf7fff640 1 -- 172.21.15.1:0/1876560186 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f0c040cf320 con 0x7f0c10101420 2024-09-11T21:55:27.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:27.558+0000 7f0c0effd640 1 --2- 172.21.15.1:0/1876560186 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f0bf80a2b80 0x7f0bf80a5040 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0c00009900 tx=0x7f0c000058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:55:27.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:27.562+0000 7f0bf7fff640 1 -- 172.21.15.1:0/1876560186 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0c0409bce0 con 0x7f0c10101420 2024-09-11T21:55:27.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:27 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/461727103' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-11T21:55:27.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:27 smithi161 ceph-mon[35615]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T21:55:27.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:27.772+0000 7f0c15363640 1 -- 172.21.15.1:0/1876560186 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f0c1010a030 con 0x7f0c10101420 2024-09-11T21:55:28.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:28.115+0000 7f0bf7fff640 1 -- 172.21.15.1:0/1876560186 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f0c0409ffe0 con 0x7f0c10101420 2024-09-11T21:55:28.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:28.117+0000 7f0c15363640 1 -- 172.21.15.1:0/1876560186 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f0bf80a2b80 msgr2=0x7f0bf80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:28.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:28.117+0000 7f0c15363640 1 --2- 172.21.15.1:0/1876560186 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f0bf80a2b80 0x7f0bf80a5040 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0c00009900 tx=0x7f0c000058d0 comp rx=0 tx=0).stop 2024-09-11T21:55:28.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:28.117+0000 7f0c15363640 1 -- 172.21.15.1:0/1876560186 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c10101420 msgr2=0x7f0c101989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:28.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:28.117+0000 7f0c15363640 1 --2- 172.21.15.1:0/1876560186 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c10101420 0x7f0c101989a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0c0400e9f0 tx=0x7f0c0400eeb0 comp rx=0 tx=0).stop 2024-09-11T21:55:28.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:28.117+0000 7f0c15363640 1 -- 172.21.15.1:0/1876560186 shutdown_connections 2024-09-11T21:55:28.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:28.117+0000 7f0c15363640 1 --2- 172.21.15.1:0/1876560186 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f0bf80a2b80 0x7f0bf80a5040 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:28.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:28.117+0000 7f0c15363640 1 --2- 172.21.15.1:0/1876560186 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c10106a80 0x7f0c1019cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:28.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:28.117+0000 7f0c15363640 1 --2- 172.21.15.1:0/1876560186 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c10101420 0x7f0c101989a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:28.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:28.117+0000 7f0c15363640 1 --2- 172.21.15.1:0/1876560186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c10100a80 0x7f0c10198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:28.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:28.117+0000 7f0c15363640 1 -- 172.21.15.1:0/1876560186 >> 172.21.15.1:0/1876560186 conn(0x7f0c100fc7a0 msgr2=0x7f0c1006aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:55:28.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:28.118+0000 7f0c15363640 1 -- 172.21.15.1:0/1876560186 shutdown_connections 2024-09-11T21:55:28.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:28.118+0000 7f0c15363640 1 -- 172.21.15.1:0/1876560186 wait complete. 2024-09-11T21:55:28.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:28 smithi001 ceph-mon[30935]: pgmap v285: 97 pgs: 97 active+clean; 579 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 5 op/s 2024-09-11T21:55:28.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:28 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/1876560186' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-11T21:55:28.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:28 smithi053 ceph-mon[34908]: pgmap v285: 97 pgs: 97 active+clean; 579 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 5 op/s 2024-09-11T21:55:28.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:28 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/1876560186' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-11T21:55:28.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:28 smithi161 ceph-mon[35615]: pgmap v285: 97 pgs: 97 active+clean; 579 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 5 op/s 2024-09-11T21:55:28.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:28 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/1876560186' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-11T21:55:28.817 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-11T21:55:29.079 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:55:29.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:29 smithi001 ceph-mon[30935]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-11T21:55:29.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:29 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/1876560186' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-11T21:55:29.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:29 smithi001 ceph-mon[30935]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-11T21:55:29.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:29 smithi001 ceph-mon[30935]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T21:55:29.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:29 smithi001 ceph-mon[30935]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-11T21:55:29.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:29 smithi001 ceph-mon[30935]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-11T21:55:29.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:29 smithi053 ceph-mon[34908]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-11T21:55:29.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:29 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/1876560186' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-11T21:55:29.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:29 smithi053 ceph-mon[34908]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-11T21:55:29.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:29 smithi053 ceph-mon[34908]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T21:55:29.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:29 smithi053 ceph-mon[34908]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-11T21:55:29.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:29 smithi053 ceph-mon[34908]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-11T21:55:29.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:29 smithi161 ceph-mon[35615]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-11T21:55:29.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:29 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/1876560186' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-11T21:55:29.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:29 smithi161 ceph-mon[35615]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-11T21:55:29.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:29 smithi161 ceph-mon[35615]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T21:55:29.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:29 smithi161 ceph-mon[35615]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-11T21:55:29.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:29 smithi161 ceph-mon[35615]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-11T21:55:29.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:29.588+0000 7f16f7c93640 1 -- 172.21.15.1:0/104579970 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16f01082e0 msgr2=0x7f16f010ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:29.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:29.588+0000 7f16f7c93640 1 --2- 172.21.15.1:0/104579970 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16f01082e0 0x7f16f010ec50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f16e8009cd0 tx=0x7f16e802a910 comp rx=0 tx=0).stop 2024-09-11T21:55:29.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:29.589+0000 7f16f7c93640 1 -- 172.21.15.1:0/104579970 shutdown_connections 2024-09-11T21:55:29.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:29.589+0000 7f16f7c93640 1 --2- 172.21.15.1:0/104579970 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16f01082e0 0x7f16f010ec50 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:29.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:29.589+0000 7f16f7c93640 1 --2- 172.21.15.1:0/104579970 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16f0103760 0x7f16f0107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:29.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:29.589+0000 7f16f7c93640 1 --2- 172.21.15.1:0/104579970 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f16f0102db0 0x7f16f0103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:29.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:29.589+0000 7f16f7c93640 1 -- 172.21.15.1:0/104579970 >> 172.21.15.1:0/104579970 conn(0x7f16f00fc7d0 msgr2=0x7f16f00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:55:29.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:29.589+0000 7f16f7c93640 1 -- 172.21.15.1:0/104579970 shutdown_connections 2024-09-11T21:55:29.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:29.589+0000 7f16f7c93640 1 -- 172.21.15.1:0/104579970 wait complete. 2024-09-11T21:55:29.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:29.589+0000 7f16f7c93640 1 Processor -- start 2024-09-11T21:55:29.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:29.589+0000 7f16f7c93640 1 -- start start 2024-09-11T21:55:29.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:29.589+0000 7f16f7c93640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f16f0102db0 0x7f16f01986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:29.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:29.590+0000 7f16f7c93640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16f0103760 0x7f16f0198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:29.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:29.590+0000 7f16f7c93640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16f01082e0 0x7f16f019cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:29.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:29.590+0000 7f16f7c93640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f16f01121c0 con 0x7f16f0103760 2024-09-11T21:55:29.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:29.590+0000 7f16f7c93640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f16f0112040 con 0x7f16f0102db0 2024-09-11T21:55:29.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:29.590+0000 7f16f7c93640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f16f0112340 con 0x7f16f01082e0 2024-09-11T21:55:29.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:29.590+0000 7f16f522e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16f0103760 0x7f16f0198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:29.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:29.590+0000 7f16f522e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16f0103760 0x7f16f0198c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45768/0 (socket says 172.21.15.1:45768) 2024-09-11T21:55:29.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:29.590+0000 7f16f522e640 1 -- 172.21.15.1:0/2419245578 learned_addr learned my addr 172.21.15.1:0/2419245578 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:55:29.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:29.590+0000 7f16f5a2f640 1 --2- 172.21.15.1:0/2419245578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f16f0102db0 0x7f16f01986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:29.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:29.590+0000 7f16f6230640 1 --2- 172.21.15.1:0/2419245578 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16f01082e0 0x7f16f019cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:29.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:29.590+0000 7f16f522e640 1 -- 172.21.15.1:0/2419245578 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16f01082e0 msgr2=0x7f16f019cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:29.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:29.590+0000 7f16f522e640 1 --2- 172.21.15.1:0/2419245578 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16f01082e0 0x7f16f019cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:29.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:29.590+0000 7f16f522e640 1 -- 172.21.15.1:0/2419245578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f16f0102db0 msgr2=0x7f16f01986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:29.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:29.590+0000 7f16f522e640 1 --2- 172.21.15.1:0/2419245578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f16f0102db0 0x7f16f01986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:29.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:29.590+0000 7f16f522e640 1 -- 172.21.15.1:0/2419245578 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16f019d690 con 0x7f16f0103760 2024-09-11T21:55:29.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:29.590+0000 7f16f522e640 1 --2- 172.21.15.1:0/2419245578 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16f0103760 0x7f16f0198c20 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f16e4006fd0 tx=0x7f16e400b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:55:29.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:29.590+0000 7f16eeffd640 1 -- 172.21.15.1:0/2419245578 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e4015020 con 0x7f16f0103760 2024-09-11T21:55:29.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:29.590+0000 7f16eeffd640 1 -- 172.21.15.1:0/2419245578 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16e4011d10 con 0x7f16f0103760 2024-09-11T21:55:29.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:29.590+0000 7f16f7c93640 1 -- 172.21.15.1:0/2419245578 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16f019d980 con 0x7f16f0103760 2024-09-11T21:55:29.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:29.590+0000 7f16eeffd640 1 -- 172.21.15.1:0/2419245578 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e400f2e0 con 0x7f16f0103760 2024-09-11T21:55:29.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:29.591+0000 7f16f7c93640 1 -- 172.21.15.1:0/2419245578 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16f01a5260 con 0x7f16f0103760 2024-09-11T21:55:29.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:29.593+0000 7f16f7c93640 1 -- 172.21.15.1:0/2419245578 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16f0103210 con 0x7f16f0103760 2024-09-11T21:55:29.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:29.594+0000 7f16eeffd640 1 -- 172.21.15.1:0/2419245578 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f16e4018020 con 0x7f16f0103760 2024-09-11T21:55:29.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:29.594+0000 7f16eeffd640 1 --2- 172.21.15.1:0/2419245578 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f16d80a2c00 0x7f16d80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:29.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:29.594+0000 7f16eeffd640 1 -- 172.21.15.1:0/2419245578 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f16e40ce780 con 0x7f16f0103760 2024-09-11T21:55:29.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:29.594+0000 7f16f5a2f640 1 --2- 172.21.15.1:0/2419245578 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f16d80a2c00 0x7f16d80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:29.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:29.594+0000 7f16f5a2f640 1 --2- 172.21.15.1:0/2419245578 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f16d80a2c00 0x7f16d80a50c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f16e0009990 tx=0x7f16e0009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:55:29.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:29.598+0000 7f16eeffd640 1 -- 172.21.15.1:0/2419245578 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f16e401f020 con 0x7f16f0103760 2024-09-11T21:55:29.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:29.814+0000 7f16f7c93640 1 -- 172.21.15.1:0/2419245578 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f16f0199a40 con 0x7f16f0103760 2024-09-11T21:55:30.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:30.132+0000 7f16eeffd640 1 -- 172.21.15.1:0/2419245578 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v22) ==== 121+0+0 (secure 0 0 0) 0x7f16e409b830 con 0x7f16f0103760 2024-09-11T21:55:30.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:30.136+0000 7f16f7c93640 1 -- 172.21.15.1:0/2419245578 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f16d80a2c00 msgr2=0x7f16d80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:30.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:30.136+0000 7f16f7c93640 1 --2- 172.21.15.1:0/2419245578 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f16d80a2c00 0x7f16d80a50c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f16e0009990 tx=0x7f16e0009110 comp rx=0 tx=0).stop 2024-09-11T21:55:30.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:30.136+0000 7f16f7c93640 1 -- 172.21.15.1:0/2419245578 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16f0103760 msgr2=0x7f16f0198c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:30.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:30.136+0000 7f16f7c93640 1 --2- 172.21.15.1:0/2419245578 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16f0103760 0x7f16f0198c20 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f16e4006fd0 tx=0x7f16e400b820 comp rx=0 tx=0).stop 2024-09-11T21:55:30.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:30.136+0000 7f16f7c93640 1 -- 172.21.15.1:0/2419245578 shutdown_connections 2024-09-11T21:55:30.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:30.136+0000 7f16f7c93640 1 --2- 172.21.15.1:0/2419245578 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f16d80a2c00 0x7f16d80a50c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:30.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:30.136+0000 7f16f7c93640 1 --2- 172.21.15.1:0/2419245578 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16f01082e0 0x7f16f019cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:30.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:30.136+0000 7f16f7c93640 1 --2- 172.21.15.1:0/2419245578 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16f0103760 0x7f16f0198c20 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:30.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:30.136+0000 7f16f7c93640 1 --2- 172.21.15.1:0/2419245578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f16f0102db0 0x7f16f01986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:30.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:30.136+0000 7f16f7c93640 1 -- 172.21.15.1:0/2419245578 >> 172.21.15.1:0/2419245578 conn(0x7f16f00fc7d0 msgr2=0x7f16f0106390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:55:30.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:30.136+0000 7f16f7c93640 1 -- 172.21.15.1:0/2419245578 shutdown_connections 2024-09-11T21:55:30.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:30.136+0000 7f16f7c93640 1 -- 172.21.15.1:0/2419245578 wait complete. 2024-09-11T21:55:30.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:30 smithi001 ceph-mon[30935]: pgmap v286: 97 pgs: 97 active+clean; 579 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 6 op/s 2024-09-11T21:55:30.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:30 smithi001 ceph-mon[30935]: mds.? [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] up:active 2024-09-11T21:55:30.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:30 smithi001 ceph-mon[30935]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-11T21:55:30.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:30 smithi001 ceph-mon[30935]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-11T21:55:30.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:30 smithi001 ceph-mon[30935]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-11T21:55:30.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:30 smithi001 ceph-mon[30935]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-11T21:55:30.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:30 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/2419245578' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-11T21:55:30.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:30 smithi053 ceph-mon[34908]: pgmap v286: 97 pgs: 97 active+clean; 579 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 6 op/s 2024-09-11T21:55:30.588 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:30 smithi053 ceph-mon[34908]: mds.? [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] up:active 2024-09-11T21:55:30.588 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:30 smithi053 ceph-mon[34908]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-11T21:55:30.588 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:30 smithi053 ceph-mon[34908]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-11T21:55:30.588 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:30 smithi053 ceph-mon[34908]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-11T21:55:30.588 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:30 smithi053 ceph-mon[34908]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-11T21:55:30.588 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:30 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/2419245578' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-11T21:55:30.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:30 smithi161 ceph-mon[35615]: pgmap v286: 97 pgs: 97 active+clean; 579 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 6 op/s 2024-09-11T21:55:30.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:30 smithi161 ceph-mon[35615]: mds.? [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] up:active 2024-09-11T21:55:30.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:30 smithi161 ceph-mon[35615]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-11T21:55:30.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:30 smithi161 ceph-mon[35615]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-11T21:55:30.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:30 smithi161 ceph-mon[35615]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-11T21:55:30.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:30 smithi161 ceph-mon[35615]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-11T21:55:30.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:30 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/2419245578' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-11T21:55:30.825 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-11T21:55:31.082 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:55:31.399 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:31 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/2419245578' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-11T21:55:31.400 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:31 smithi001 ceph-mon[30935]: mds.? [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] up:active 2024-09-11T21:55:31.400 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:31 smithi001 ceph-mon[30935]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-11T21:55:31.400 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:31 smithi001 ceph-mon[30935]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-09-11T21:55:31.400 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:31 smithi001 ceph-mon[30935]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-09-11T21:55:31.400 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:31 smithi001 ceph-mon[30935]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-11T21:55:31.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:31 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/2419245578' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-11T21:55:31.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:31 smithi053 ceph-mon[34908]: mds.? [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] up:active 2024-09-11T21:55:31.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:31 smithi053 ceph-mon[34908]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-11T21:55:31.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:31 smithi053 ceph-mon[34908]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-09-11T21:55:31.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:31 smithi053 ceph-mon[34908]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-09-11T21:55:31.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:31 smithi053 ceph-mon[34908]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-11T21:55:31.548 INFO:journalctl@ceph.mds.b.smithi053.stdout:Sep 11 21:55:31 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-b[64373]: 2024-09-11T21:55:31.307+0000 7f17b78d0640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-11T21:55:31.548 INFO:journalctl@ceph.mds.b.smithi053.stdout:Sep 11 21:55:31 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-b[64373]: 2024-09-11T21:55:31.307+0000 7f17b78d0640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-11T21:55:31.549 INFO:journalctl@ceph.mds.b.smithi053.stdout:Sep 11 21:55:31 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-b[64373]: 2024-09-11T21:55:31.307+0000 7f17b78d0640 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-09-11T21:55:31.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:31 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/2419245578' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-11T21:55:31.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:31 smithi161 ceph-mon[35615]: mds.? [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] up:active 2024-09-11T21:55:31.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:31 smithi161 ceph-mon[35615]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-11T21:55:31.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:31 smithi161 ceph-mon[35615]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-09-11T21:55:31.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:31 smithi161 ceph-mon[35615]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-09-11T21:55:31.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:31 smithi161 ceph-mon[35615]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-11T21:55:31.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:31.591+0000 7f240079a640 1 -- 172.21.15.1:0/2796664879 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f23f81082a0 msgr2=0x7f23f810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:31.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:31.591+0000 7f240079a640 1 --2- 172.21.15.1:0/2796664879 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f23f81082a0 0x7f23f810ec40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f23e8009cd0 tx=0x7f23e802a910 comp rx=0 tx=0).stop 2024-09-11T21:55:31.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:31.591+0000 7f240079a640 1 -- 172.21.15.1:0/2796664879 shutdown_connections 2024-09-11T21:55:31.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:31.591+0000 7f240079a640 1 --2- 172.21.15.1:0/2796664879 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f23f81082a0 0x7f23f810ec40 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:31.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:31.591+0000 7f240079a640 1 --2- 172.21.15.1:0/2796664879 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23f81036e0 0x7f23f8107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:31.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:31.591+0000 7f240079a640 1 --2- 172.21.15.1:0/2796664879 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f23f8102d30 0x7f23f8103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:31.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:31.591+0000 7f240079a640 1 -- 172.21.15.1:0/2796664879 >> 172.21.15.1:0/2796664879 conn(0x7f23f80fc7d0 msgr2=0x7f23f80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:55:31.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:31.591+0000 7f240079a640 1 -- 172.21.15.1:0/2796664879 shutdown_connections 2024-09-11T21:55:31.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:31.591+0000 7f240079a640 1 -- 172.21.15.1:0/2796664879 wait complete. 2024-09-11T21:55:31.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:31.591+0000 7f240079a640 1 Processor -- start 2024-09-11T21:55:31.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:31.591+0000 7f240079a640 1 -- start start 2024-09-11T21:55:31.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:31.592+0000 7f240079a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f23f8102d30 0x7f23f81986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:31.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:31.592+0000 7f240079a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23f81036e0 0x7f23f8198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:31.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:31.592+0000 7f240079a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f23f81082a0 0x7f23f819cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:31.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:31.592+0000 7f240079a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f23f81121b0 con 0x7f23f8102d30 2024-09-11T21:55:31.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:31.592+0000 7f240079a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f23f8112030 con 0x7f23f81082a0 2024-09-11T21:55:31.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:31.592+0000 7f240079a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f23f8112330 con 0x7f23f81036e0 2024-09-11T21:55:31.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:31.592+0000 7f23fe536640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f23f8102d30 0x7f23f81986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:31.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:31.592+0000 7f23fe536640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f23f8102d30 0x7f23f81986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45784/0 (socket says 172.21.15.1:45784) 2024-09-11T21:55:31.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:31.592+0000 7f23fe536640 1 -- 172.21.15.1:0/2733564551 learned_addr learned my addr 172.21.15.1:0/2733564551 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:55:31.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:31.592+0000 7f23fed37640 1 --2- 172.21.15.1:0/2733564551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f23f81082a0 0x7f23f819cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:31.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:31.592+0000 7f23fdd35640 1 --2- 172.21.15.1:0/2733564551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23f81036e0 0x7f23f8198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:31.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:31.592+0000 7f23fe536640 1 -- 172.21.15.1:0/2733564551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23f81036e0 msgr2=0x7f23f8198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:31.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:31.592+0000 7f23fe536640 1 --2- 172.21.15.1:0/2733564551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23f81036e0 0x7f23f8198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:31.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:31.592+0000 7f23fe536640 1 -- 172.21.15.1:0/2733564551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f23f81082a0 msgr2=0x7f23f819cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:31.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:31.592+0000 7f23fe536640 1 --2- 172.21.15.1:0/2733564551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f23f81082a0 0x7f23f819cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:31.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:31.592+0000 7f23fe536640 1 -- 172.21.15.1:0/2733564551 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23f819d680 con 0x7f23f8102d30 2024-09-11T21:55:31.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:31.592+0000 7f23fdd35640 1 --2- 172.21.15.1:0/2733564551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23f81036e0 0x7f23f8198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T21:55:31.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:31.592+0000 7f23fe536640 1 --2- 172.21.15.1:0/2733564551 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f23f8102d30 0x7f23f81986d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f23f000b4a0 tx=0x7f23f000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:55:31.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:31.592+0000 7f23f77fe640 1 -- 172.21.15.1:0/2733564551 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f0014070 con 0x7f23f8102d30 2024-09-11T21:55:31.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:31.592+0000 7f240079a640 1 -- 172.21.15.1:0/2733564551 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23f819d970 con 0x7f23f8102d30 2024-09-11T21:55:31.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:31.593+0000 7f23f77fe640 1 -- 172.21.15.1:0/2733564551 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23f0010af0 con 0x7f23f8102d30 2024-09-11T21:55:31.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:31.593+0000 7f240079a640 1 -- 172.21.15.1:0/2733564551 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23f81a5250 con 0x7f23f8102d30 2024-09-11T21:55:31.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:31.593+0000 7f23f77fe640 1 -- 172.21.15.1:0/2733564551 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f001e060 con 0x7f23f8102d30 2024-09-11T21:55:31.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:31.594+0000 7f240079a640 1 -- 172.21.15.1:0/2733564551 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23f806a850 con 0x7f23f8102d30 2024-09-11T21:55:31.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:31.595+0000 7f23f77fe640 1 -- 172.21.15.1:0/2733564551 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f23f0017070 con 0x7f23f8102d30 2024-09-11T21:55:31.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:31.595+0000 7f23f77fe640 1 --2- 172.21.15.1:0/2733564551 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f23e00a2b80 0x7f23e00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:31.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:31.595+0000 7f23fdd35640 1 --2- 172.21.15.1:0/2733564551 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f23e00a2b80 0x7f23e00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:31.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:31.596+0000 7f23f77fe640 1 -- 172.21.15.1:0/2733564551 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f23f00ce300 con 0x7f23f8102d30 2024-09-11T21:55:31.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:31.596+0000 7f23fdd35640 1 --2- 172.21.15.1:0/2733564551 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f23e00a2b80 0x7f23e00a5040 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f23ec006fd0 tx=0x7f23ec008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:55:31.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:31.600+0000 7f23f77fe640 1 -- 172.21.15.1:0/2733564551 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f23f009b3b0 con 0x7f23f8102d30 2024-09-11T21:55:31.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:31.811+0000 7f240079a640 1 -- 172.21.15.1:0/2733564551 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f23f8103190 con 0x7f23f8102d30 2024-09-11T21:55:32.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:32.148+0000 7f23f77fe640 1 -- 172.21.15.1:0/2733564551 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v26) ==== 115+0+0 (secure 0 0 0) 0x7f23f009acb0 con 0x7f23f8102d30 2024-09-11T21:55:32.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:32.150+0000 7f240079a640 1 -- 172.21.15.1:0/2733564551 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f23e00a2b80 msgr2=0x7f23e00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:32.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:32.150+0000 7f240079a640 1 --2- 172.21.15.1:0/2733564551 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f23e00a2b80 0x7f23e00a5040 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f23ec006fd0 tx=0x7f23ec008040 comp rx=0 tx=0).stop 2024-09-11T21:55:32.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:32.150+0000 7f240079a640 1 -- 172.21.15.1:0/2733564551 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f23f8102d30 msgr2=0x7f23f81986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:32.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:32.150+0000 7f240079a640 1 --2- 172.21.15.1:0/2733564551 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f23f8102d30 0x7f23f81986d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f23f000b4a0 tx=0x7f23f000b960 comp rx=0 tx=0).stop 2024-09-11T21:55:32.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:32.150+0000 7f240079a640 1 -- 172.21.15.1:0/2733564551 shutdown_connections 2024-09-11T21:55:32.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:32.150+0000 7f240079a640 1 --2- 172.21.15.1:0/2733564551 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f23e00a2b80 0x7f23e00a5040 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:32.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:32.150+0000 7f240079a640 1 --2- 172.21.15.1:0/2733564551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f23f81082a0 0x7f23f819cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:32.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:32.150+0000 7f240079a640 1 --2- 172.21.15.1:0/2733564551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23f81036e0 0x7f23f8198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:32.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:32.150+0000 7f240079a640 1 --2- 172.21.15.1:0/2733564551 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f23f8102d30 0x7f23f81986d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:32.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:32.150+0000 7f240079a640 1 -- 172.21.15.1:0/2733564551 >> 172.21.15.1:0/2733564551 conn(0x7f23f80fc7d0 msgr2=0x7f23f8106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:55:32.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:32.151+0000 7f240079a640 1 -- 172.21.15.1:0/2733564551 shutdown_connections 2024-09-11T21:55:32.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:32.151+0000 7f240079a640 1 -- 172.21.15.1:0/2733564551 wait complete. 2024-09-11T21:55:32.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:32 smithi001 ceph-mon[30935]: pgmap v287: 97 pgs: 97 active+clean; 580 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 7 op/s 2024-09-11T21:55:32.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:32 smithi001 ceph-mon[30935]: mds.? [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] up:active 2024-09-11T21:55:32.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:32 smithi001 ceph-mon[30935]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-11T21:55:32.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:32 smithi001 ceph-mon[30935]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-11T21:55:32.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:32 smithi001 ceph-mon[30935]: Cluster is now healthy 2024-09-11T21:55:32.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:32 smithi001 ceph-mon[30935]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2024-09-11T21:55:32.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:32 smithi001 ceph-mon[30935]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-11T21:55:32.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:32 smithi001 ceph-mon[30935]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-09-11T21:55:32.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:32 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/2733564551' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-11T21:55:32.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:32 smithi053 ceph-mon[34908]: pgmap v287: 97 pgs: 97 active+clean; 580 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 7 op/s 2024-09-11T21:55:32.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:32 smithi053 ceph-mon[34908]: mds.? [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] up:active 2024-09-11T21:55:32.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:32 smithi053 ceph-mon[34908]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-11T21:55:32.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:32 smithi053 ceph-mon[34908]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-11T21:55:32.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:32 smithi053 ceph-mon[34908]: Cluster is now healthy 2024-09-11T21:55:32.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:32 smithi053 ceph-mon[34908]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2024-09-11T21:55:32.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:32 smithi053 ceph-mon[34908]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-11T21:55:32.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:32 smithi053 ceph-mon[34908]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-09-11T21:55:32.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:32 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/2733564551' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-11T21:55:32.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:32 smithi161 ceph-mon[35615]: pgmap v287: 97 pgs: 97 active+clean; 580 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 7 op/s 2024-09-11T21:55:32.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:32 smithi161 ceph-mon[35615]: mds.? [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] up:active 2024-09-11T21:55:32.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:32 smithi161 ceph-mon[35615]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-11T21:55:32.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:32 smithi161 ceph-mon[35615]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-11T21:55:32.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:32 smithi161 ceph-mon[35615]: Cluster is now healthy 2024-09-11T21:55:32.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:32 smithi161 ceph-mon[35615]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2024-09-11T21:55:32.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:32 smithi161 ceph-mon[35615]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-11T21:55:32.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:32 smithi161 ceph-mon[35615]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-09-11T21:55:32.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:32 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/2733564551' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-11T21:55:32.868 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2024-09-11T21:55:32.868 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-11T21:55:33.127 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:55:33.402 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:33 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/2733564551' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-11T21:55:33.402 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:33 smithi001 ceph-mon[30935]: mds.? [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] up:active 2024-09-11T21:55:33.402 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:33 smithi001 ceph-mon[30935]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-11T21:55:33.403 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:33 smithi001 ceph-mon[30935]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T21:55:33.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:33 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/2733564551' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-11T21:55:33.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:33 smithi053 ceph-mon[34908]: mds.? [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] up:active 2024-09-11T21:55:33.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:33 smithi053 ceph-mon[34908]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-11T21:55:33.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:33 smithi053 ceph-mon[34908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T21:55:33.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:33 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/2733564551' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-11T21:55:33.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:33 smithi161 ceph-mon[35615]: mds.? [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] up:active 2024-09-11T21:55:33.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:33 smithi161 ceph-mon[35615]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-11T21:55:33.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:33 smithi161 ceph-mon[35615]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T21:55:33.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:33.751+0000 7f4063f03640 1 -- 172.21.15.1:0/3424459220 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f405c102dd0 msgr2=0x7f405c1031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:33.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:33.751+0000 7f4063f03640 1 --2- 172.21.15.1:0/3424459220 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f405c102dd0 0x7f405c1031b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4054009930 tx=0x7f405402a420 comp rx=0 tx=0).stop 2024-09-11T21:55:33.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:33.751+0000 7f4063f03640 1 -- 172.21.15.1:0/3424459220 shutdown_connections 2024-09-11T21:55:33.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:33.751+0000 7f4063f03640 1 --2- 172.21.15.1:0/3424459220 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f405c108300 0x7f405c10ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:33.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:33.751+0000 7f4063f03640 1 --2- 172.21.15.1:0/3424459220 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f405c103780 0x7f405c107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:33.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:33.751+0000 7f4063f03640 1 --2- 172.21.15.1:0/3424459220 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f405c102dd0 0x7f405c1031b0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:33.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:33.751+0000 7f4063f03640 1 -- 172.21.15.1:0/3424459220 >> 172.21.15.1:0/3424459220 conn(0x7f405c0fc7d0 msgr2=0x7f405c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:55:33.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:33.751+0000 7f4063f03640 1 -- 172.21.15.1:0/3424459220 shutdown_connections 2024-09-11T21:55:33.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:33.751+0000 7f4063f03640 1 -- 172.21.15.1:0/3424459220 wait complete. 2024-09-11T21:55:33.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:33.751+0000 7f4063f03640 1 Processor -- start 2024-09-11T21:55:33.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:33.751+0000 7f4063f03640 1 -- start start 2024-09-11T21:55:33.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:33.752+0000 7f4063f03640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f405c102dd0 0x7f405c1985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:33.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:33.752+0000 7f4063f03640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f405c103780 0x7f405c198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:33.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:33.752+0000 7f4063f03640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f405c108300 0x7f405c19cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:33.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:33.752+0000 7f4063f03640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f405c1120b0 con 0x7f405c103780 2024-09-11T21:55:33.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:33.752+0000 7f4063f03640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f405c111f30 con 0x7f405c102dd0 2024-09-11T21:55:33.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:33.752+0000 7f4063f03640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f405c112230 con 0x7f405c108300 2024-09-11T21:55:33.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:33.752+0000 7f406149e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f405c103780 0x7f405c198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:33.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:33.752+0000 7f406149e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f405c103780 0x7f405c198b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45810/0 (socket says 172.21.15.1:45810) 2024-09-11T21:55:33.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:33.752+0000 7f406149e640 1 -- 172.21.15.1:0/598124166 learned_addr learned my addr 172.21.15.1:0/598124166 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:55:33.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:33.752+0000 7f4061c9f640 1 --2- 172.21.15.1:0/598124166 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f405c102dd0 0x7f405c1985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:33.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:33.752+0000 7f40624a0640 1 --2- 172.21.15.1:0/598124166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f405c108300 0x7f405c19cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:33.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:33.752+0000 7f406149e640 1 -- 172.21.15.1:0/598124166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f405c108300 msgr2=0x7f405c19cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:33.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:33.752+0000 7f406149e640 1 --2- 172.21.15.1:0/598124166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f405c108300 0x7f405c19cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:33.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:33.752+0000 7f406149e640 1 -- 172.21.15.1:0/598124166 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f405c102dd0 msgr2=0x7f405c1985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:33.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:33.752+0000 7f406149e640 1 --2- 172.21.15.1:0/598124166 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f405c102dd0 0x7f405c1985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:33.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:33.752+0000 7f406149e640 1 -- 172.21.15.1:0/598124166 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f405c19d5a0 con 0x7f405c103780 2024-09-11T21:55:33.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:33.752+0000 7f40624a0640 1 --2- 172.21.15.1:0/598124166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f405c108300 0x7f405c19cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:55:33.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:33.752+0000 7f406149e640 1 --2- 172.21.15.1:0/598124166 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f405c103780 0x7f405c198b30 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f405000d750 tx=0x7f405000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:55:33.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:33.752+0000 7f405affd640 1 -- 172.21.15.1:0/598124166 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4050017070 con 0x7f405c103780 2024-09-11T21:55:33.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:33.752+0000 7f405affd640 1 -- 172.21.15.1:0/598124166 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4050005500 con 0x7f405c103780 2024-09-11T21:55:33.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:33.752+0000 7f4063f03640 1 -- 172.21.15.1:0/598124166 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f405c19d890 con 0x7f405c103780 2024-09-11T21:55:33.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:33.752+0000 7f405affd640 1 -- 172.21.15.1:0/598124166 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4050004e00 con 0x7f405c103780 2024-09-11T21:55:33.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:33.753+0000 7f4063f03640 1 -- 172.21.15.1:0/598124166 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f405c1a5170 con 0x7f405c103780 2024-09-11T21:55:33.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:33.754+0000 7f4063f03640 1 -- 172.21.15.1:0/598124166 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f405c06a7b0 con 0x7f405c103780 2024-09-11T21:55:33.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:33.755+0000 7f405affd640 1 -- 172.21.15.1:0/598124166 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f4050004fa0 con 0x7f405c103780 2024-09-11T21:55:33.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:33.755+0000 7f405affd640 1 --2- 172.21.15.1:0/598124166 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f40440a2c00 0x7f40440a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:33.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:33.756+0000 7f4061c9f640 1 --2- 172.21.15.1:0/598124166 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f40440a2c00 0x7f40440a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:33.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:33.756+0000 7f405affd640 1 -- 172.21.15.1:0/598124166 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f40500ce830 con 0x7f405c103780 2024-09-11T21:55:33.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:33.756+0000 7f4061c9f640 1 --2- 172.21.15.1:0/598124166 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f40440a2c00 0x7f40440a50c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4054009900 tx=0x7f40540058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:55:33.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:33.760+0000 7f405affd640 1 -- 172.21.15.1:0/598124166 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f405009b8e0 con 0x7f405c103780 2024-09-11T21:55:33.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:33.960+0000 7f4063f03640 1 -- 172.21.15.1:0/598124166 --> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f405c062630 con 0x7f40440a2c00 2024-09-11T21:55:33.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:33.989+0000 7f405affd640 1 -- 172.21.15.1:0/598124166 <== mgr.14150 v2:172.21.15.1:6800/3890350886 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f405c062630 con 0x7f40440a2c00 2024-09-11T21:55:33.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:33.991+0000 7f4063f03640 1 -- 172.21.15.1:0/598124166 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f40440a2c00 msgr2=0x7f40440a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:33.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:33.991+0000 7f4063f03640 1 --2- 172.21.15.1:0/598124166 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f40440a2c00 0x7f40440a50c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4054009900 tx=0x7f40540058d0 comp rx=0 tx=0).stop 2024-09-11T21:55:33.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:33.991+0000 7f4063f03640 1 -- 172.21.15.1:0/598124166 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f405c103780 msgr2=0x7f405c198b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:33.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:33.991+0000 7f4063f03640 1 --2- 172.21.15.1:0/598124166 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f405c103780 0x7f405c198b30 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f405000d750 tx=0x7f405000dc10 comp rx=0 tx=0).stop 2024-09-11T21:55:33.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:33.991+0000 7f4063f03640 1 -- 172.21.15.1:0/598124166 shutdown_connections 2024-09-11T21:55:33.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:33.991+0000 7f4063f03640 1 --2- 172.21.15.1:0/598124166 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f40440a2c00 0x7f40440a50c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:33.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:33.991+0000 7f4063f03640 1 --2- 172.21.15.1:0/598124166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f405c108300 0x7f405c19cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:33.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:33.991+0000 7f4063f03640 1 --2- 172.21.15.1:0/598124166 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f405c103780 0x7f405c198b30 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:33.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:33.991+0000 7f4063f03640 1 --2- 172.21.15.1:0/598124166 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f405c102dd0 0x7f405c1985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:33.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:33.991+0000 7f4063f03640 1 -- 172.21.15.1:0/598124166 >> 172.21.15.1:0/598124166 conn(0x7f405c0fc7d0 msgr2=0x7f405c105e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:55:33.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:33.991+0000 7f4063f03640 1 -- 172.21.15.1:0/598124166 shutdown_connections 2024-09-11T21:55:33.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:33.991+0000 7f4063f03640 1 -- 172.21.15.1:0/598124166 wait complete. 2024-09-11T21:55:34.362 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:34 smithi001 ceph-mon[30935]: pgmap v288: 97 pgs: 97 active+clean; 583 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 621 B/s rd, 4.1 KiB/s wr, 12 op/s 2024-09-11T21:55:34.363 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:55:33 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:55:33.965+0000 7f761261b640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T21:55:34.363 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:55:33 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:55:33.965+0000 7f761261b640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T21:55:34.363 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:55:33 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:55:33.965+0000 7f761261b640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T21:55:34.363 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:55:33 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:55:33.965+0000 7f761261b640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T21:55:34.363 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:55:33 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:55:33.965+0000 7f761261b640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T21:55:34.449 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:34 smithi053 ceph-mon[34908]: pgmap v288: 97 pgs: 97 active+clean; 583 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 621 B/s rd, 4.1 KiB/s wr, 12 op/s 2024-09-11T21:55:34.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:34 smithi161 ceph-mon[35615]: pgmap v288: 97 pgs: 97 active+clean; 583 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 621 B/s rd, 4.1 KiB/s wr, 12 op/s 2024-09-11T21:55:34.720 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-11T21:55:34.979 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:55:35.368 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:35 smithi001 ceph-mon[30935]: from='client.14682 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:55:35.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:35.455+0000 7fda59367640 1 -- 172.21.15.1:0/1991306331 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda54102de0 msgr2=0x7fda541031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:35.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:35.455+0000 7fda59367640 1 --2- 172.21.15.1:0/1991306331 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda54102de0 0x7fda541031c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fda44009900 tx=0x7fda4402a480 comp rx=0 tx=0).stop 2024-09-11T21:55:35.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:35.455+0000 7fda59367640 1 -- 172.21.15.1:0/1991306331 shutdown_connections 2024-09-11T21:55:35.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:35.455+0000 7fda59367640 1 --2- 172.21.15.1:0/1991306331 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda5410c180 0x7fda5410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:35.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:35.455+0000 7fda59367640 1 --2- 172.21.15.1:0/1991306331 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda54103700 0x7fda5410bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:35.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:35.455+0000 7fda59367640 1 --2- 172.21.15.1:0/1991306331 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda54102de0 0x7fda541031c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:35.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:35.455+0000 7fda59367640 1 -- 172.21.15.1:0/1991306331 >> 172.21.15.1:0/1991306331 conn(0x7fda540fc7a0 msgr2=0x7fda540febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:55:35.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:35.455+0000 7fda59367640 1 -- 172.21.15.1:0/1991306331 shutdown_connections 2024-09-11T21:55:35.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:35.455+0000 7fda59367640 1 -- 172.21.15.1:0/1991306331 wait complete. 2024-09-11T21:55:35.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:35.456+0000 7fda59367640 1 Processor -- start 2024-09-11T21:55:35.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:35.456+0000 7fda59367640 1 -- start start 2024-09-11T21:55:35.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:35.456+0000 7fda59367640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda54102de0 0x7fda54198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:35.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:35.456+0000 7fda59367640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda54103700 0x7fda541989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:35.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:35.456+0000 7fda59367640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda5410c180 0x7fda5419cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:35.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:35.456+0000 7fda59367640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fda54111f70 con 0x7fda54102de0 2024-09-11T21:55:35.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:35.456+0000 7fda59367640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fda54111df0 con 0x7fda54103700 2024-09-11T21:55:35.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:35.456+0000 7fda59367640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fda541120f0 con 0x7fda5410c180 2024-09-11T21:55:35.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:35.456+0000 7fda52ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda54102de0 0x7fda54198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:35.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:35.456+0000 7fda52ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda54102de0 0x7fda54198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45854/0 (socket says 172.21.15.1:45854) 2024-09-11T21:55:35.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:35.456+0000 7fda52ffd640 1 -- 172.21.15.1:0/1052663520 learned_addr learned my addr 172.21.15.1:0/1052663520 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:55:35.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:35.456+0000 7fda537fe640 1 --2- 172.21.15.1:0/1052663520 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda5410c180 0x7fda5419cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:35.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:35.456+0000 7fda4bfff640 1 --2- 172.21.15.1:0/1052663520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda54103700 0x7fda541989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:35.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:35.457+0000 7fda52ffd640 1 -- 172.21.15.1:0/1052663520 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda5410c180 msgr2=0x7fda5419cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:35.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:35.457+0000 7fda52ffd640 1 --2- 172.21.15.1:0/1052663520 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda5410c180 0x7fda5419cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:35.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:35.457+0000 7fda52ffd640 1 -- 172.21.15.1:0/1052663520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda54103700 msgr2=0x7fda541989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:35.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:35.457+0000 7fda52ffd640 1 --2- 172.21.15.1:0/1052663520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda54103700 0x7fda541989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:35.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:35.457+0000 7fda52ffd640 1 -- 172.21.15.1:0/1052663520 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda5419d420 con 0x7fda54102de0 2024-09-11T21:55:35.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:35.457+0000 7fda52ffd640 1 --2- 172.21.15.1:0/1052663520 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda54102de0 0x7fda54198470 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fda44006010 tx=0x7fda440042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:55:35.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:35.457+0000 7fda50ff9640 1 -- 172.21.15.1:0/1052663520 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda4403f030 con 0x7fda54102de0 2024-09-11T21:55:35.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:35.457+0000 7fda50ff9640 1 -- 172.21.15.1:0/1052663520 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda44003ce0 con 0x7fda54102de0 2024-09-11T21:55:35.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:35.457+0000 7fda50ff9640 1 -- 172.21.15.1:0/1052663520 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda44035070 con 0x7fda54102de0 2024-09-11T21:55:35.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:35.457+0000 7fda59367640 1 -- 172.21.15.1:0/1052663520 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda5419d6b0 con 0x7fda54102de0 2024-09-11T21:55:35.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:35.457+0000 7fda59367640 1 -- 172.21.15.1:0/1052663520 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda5419db40 con 0x7fda54102de0 2024-09-11T21:55:35.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:35.458+0000 7fda59367640 1 -- 172.21.15.1:0/1052663520 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda541031c0 con 0x7fda54102de0 2024-09-11T21:55:35.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:35.459+0000 7fda50ff9640 1 -- 172.21.15.1:0/1052663520 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fda44042030 con 0x7fda54102de0 2024-09-11T21:55:35.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:35.460+0000 7fda50ff9640 1 --2- 172.21.15.1:0/1052663520 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fda400a2c10 0x7fda400a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:35.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:35.460+0000 7fda4bfff640 1 --2- 172.21.15.1:0/1052663520 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fda400a2c10 0x7fda400a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:35.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:35.460+0000 7fda50ff9640 1 -- 172.21.15.1:0/1052663520 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7fda440ef130 con 0x7fda54102de0 2024-09-11T21:55:35.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:35.460+0000 7fda4bfff640 1 --2- 172.21.15.1:0/1052663520 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fda400a2c10 0x7fda400a50d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fda540feda0 tx=0x7fda3c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:55:35.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:35.464+0000 7fda50ff9640 1 -- 172.21.15.1:0/1052663520 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fda440bbb50 con 0x7fda54102de0 2024-09-11T21:55:35.472 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:35 smithi053 ceph-mon[34908]: from='client.14682 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:55:35.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:35 smithi161 ceph-mon[35615]: from='client.14682 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:55:35.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:35.664+0000 7fda59367640 1 -- 172.21.15.1:0/1052663520 --> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fda54062630 con 0x7fda400a2c10 2024-09-11T21:55:35.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:35.692+0000 7fda50ff9640 1 -- 172.21.15.1:0/1052663520 <== mgr.14150 v2:172.21.15.1:6800/3890350886 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fda54062630 con 0x7fda400a2c10 2024-09-11T21:55:35.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:35.694+0000 7fda59367640 1 -- 172.21.15.1:0/1052663520 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fda400a2c10 msgr2=0x7fda400a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:35.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:35.694+0000 7fda59367640 1 --2- 172.21.15.1:0/1052663520 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fda400a2c10 0x7fda400a50d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fda540feda0 tx=0x7fda3c00b390 comp rx=0 tx=0).stop 2024-09-11T21:55:35.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:35.694+0000 7fda59367640 1 -- 172.21.15.1:0/1052663520 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda54102de0 msgr2=0x7fda54198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:35.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:35.694+0000 7fda59367640 1 --2- 172.21.15.1:0/1052663520 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda54102de0 0x7fda54198470 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fda44006010 tx=0x7fda440042f0 comp rx=0 tx=0).stop 2024-09-11T21:55:35.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:35.694+0000 7fda59367640 1 -- 172.21.15.1:0/1052663520 shutdown_connections 2024-09-11T21:55:35.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:35.694+0000 7fda59367640 1 --2- 172.21.15.1:0/1052663520 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fda400a2c10 0x7fda400a50d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:35.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:35.694+0000 7fda59367640 1 --2- 172.21.15.1:0/1052663520 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda5410c180 0x7fda5419cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:35.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:35.694+0000 7fda59367640 1 --2- 172.21.15.1:0/1052663520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda54103700 0x7fda541989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:35.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:35.694+0000 7fda59367640 1 --2- 172.21.15.1:0/1052663520 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda54102de0 0x7fda54198470 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:35.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:35.694+0000 7fda59367640 1 -- 172.21.15.1:0/1052663520 >> 172.21.15.1:0/1052663520 conn(0x7fda540fc7a0 msgr2=0x7fda540fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:55:35.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:35.694+0000 7fda59367640 1 -- 172.21.15.1:0/1052663520 shutdown_connections 2024-09-11T21:55:35.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:35.694+0000 7fda59367640 1 -- 172.21.15.1:0/1052663520 wait complete. 2024-09-11T21:55:36.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:36 smithi001 ceph-mon[30935]: pgmap v289: 97 pgs: 97 active+clean; 590 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5.4 KiB/s wr, 18 op/s 2024-09-11T21:55:36.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:36 smithi001 ceph-mon[30935]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-11T21:55:36.491 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-11T21:55:36.493 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:36 smithi161 ceph-mon[35615]: pgmap v289: 97 pgs: 97 active+clean; 590 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5.4 KiB/s wr, 18 op/s 2024-09-11T21:55:36.493 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:36 smithi161 ceph-mon[35615]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-11T21:55:36.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:36 smithi053 ceph-mon[34908]: pgmap v289: 97 pgs: 97 active+clean; 590 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5.4 KiB/s wr, 18 op/s 2024-09-11T21:55:36.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:36 smithi053 ceph-mon[34908]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-11T21:55:36.548 INFO:journalctl@ceph.mds.b.smithi053.stdout:Sep 11 21:55:36 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-b[64373]: 2024-09-11T21:55:36.307+0000 7f17b78d0640 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-09-11T21:55:36.756 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:55:37.168 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:37 smithi001 ceph-mon[30935]: from='client.14694 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:55:37.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:37.306+0000 7f55ac71d640 1 -- 172.21.15.1:0/70744683 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f55a4103660 msgr2=0x7f55a410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:37.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:37.306+0000 7f55ac71d640 1 --2- 172.21.15.1:0/70744683 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f55a4103660 0x7f55a410bb50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5594009900 tx=0x7f559402a480 comp rx=0 tx=0).stop 2024-09-11T21:55:37.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:37.306+0000 7f55ac71d640 1 -- 172.21.15.1:0/70744683 shutdown_connections 2024-09-11T21:55:37.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:37.306+0000 7f55ac71d640 1 --2- 172.21.15.1:0/70744683 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55a410c170 0x7f55a410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:37.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:37.306+0000 7f55ac71d640 1 --2- 172.21.15.1:0/70744683 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f55a4103660 0x7f55a410bb50 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:37.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:37.306+0000 7f55ac71d640 1 --2- 172.21.15.1:0/70744683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55a4102d40 0x7f55a4103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:37.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:37.306+0000 7f55ac71d640 1 -- 172.21.15.1:0/70744683 >> 172.21.15.1:0/70744683 conn(0x7f55a40fc7a0 msgr2=0x7f55a40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:55:37.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:37.306+0000 7f55ac71d640 1 -- 172.21.15.1:0/70744683 shutdown_connections 2024-09-11T21:55:37.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:37.306+0000 7f55ac71d640 1 -- 172.21.15.1:0/70744683 wait complete. 2024-09-11T21:55:37.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:37.306+0000 7f55ac71d640 1 Processor -- start 2024-09-11T21:55:37.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:37.306+0000 7f55ac71d640 1 -- start start 2024-09-11T21:55:37.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:37.307+0000 7f55ac71d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55a4102d40 0x7f55a41009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:37.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:37.307+0000 7f55ac71d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55a4103660 0x7f55a4100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:37.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:37.307+0000 7f55ac71d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f55a410c170 0x7f55a4101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:37.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:37.307+0000 7f55ac71d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f55a4111f70 con 0x7f55a410c170 2024-09-11T21:55:37.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:37.307+0000 7f55ac71d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f55a4111df0 con 0x7f55a4103660 2024-09-11T21:55:37.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:37.307+0000 7f55ac71d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f55a41120f0 con 0x7f55a4102d40 2024-09-11T21:55:37.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:37.307+0000 7f55aacba640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f55a410c170 0x7f55a4101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:37.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:37.307+0000 7f55aacba640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f55a410c170 0x7f55a4101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44850/0 (socket says 172.21.15.1:44850) 2024-09-11T21:55:37.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:37.307+0000 7f55aacba640 1 -- 172.21.15.1:0/2606559571 learned_addr learned my addr 172.21.15.1:0/2606559571 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:55:37.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:37.307+0000 7f55a9cb8640 1 --2- 172.21.15.1:0/2606559571 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55a4103660 0x7f55a4100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:37.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:37.307+0000 7f55aa4b9640 1 --2- 172.21.15.1:0/2606559571 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55a4102d40 0x7f55a41009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:37.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:37.307+0000 7f55aacba640 1 -- 172.21.15.1:0/2606559571 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55a4102d40 msgr2=0x7f55a41009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:37.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:37.307+0000 7f55aacba640 1 --2- 172.21.15.1:0/2606559571 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55a4102d40 0x7f55a41009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:37.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:37.307+0000 7f55aacba640 1 -- 172.21.15.1:0/2606559571 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55a4103660 msgr2=0x7f55a4100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:37.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:37.307+0000 7f55aacba640 1 --2- 172.21.15.1:0/2606559571 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55a4103660 0x7f55a4100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:37.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:37.307+0000 7f55aacba640 1 -- 172.21.15.1:0/2606559571 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55a41a6990 con 0x7f55a410c170 2024-09-11T21:55:37.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:37.307+0000 7f55aacba640 1 --2- 172.21.15.1:0/2606559571 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f55a410c170 0x7f55a4101420 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f559c00eb10 tx=0x7f559c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:55:37.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:37.307+0000 7f559b7fe640 1 -- 172.21.15.1:0/2606559571 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559c004b20 con 0x7f55a410c170 2024-09-11T21:55:37.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:37.307+0000 7f559b7fe640 1 -- 172.21.15.1:0/2606559571 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f559c004cc0 con 0x7f55a410c170 2024-09-11T21:55:37.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:37.307+0000 7f55ac71d640 1 -- 172.21.15.1:0/2606559571 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55a41a6c80 con 0x7f55a410c170 2024-09-11T21:55:37.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:37.307+0000 7f559b7fe640 1 -- 172.21.15.1:0/2606559571 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559c010760 con 0x7f55a410c170 2024-09-11T21:55:37.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:37.308+0000 7f55ac71d640 1 -- 172.21.15.1:0/2606559571 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55a41a7190 con 0x7f55a410c170 2024-09-11T21:55:37.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:37.308+0000 7f55ac71d640 1 -- 172.21.15.1:0/2606559571 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55a4103120 con 0x7f55a410c170 2024-09-11T21:55:37.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:37.310+0000 7f559b7fe640 1 -- 172.21.15.1:0/2606559571 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f559c005020 con 0x7f55a410c170 2024-09-11T21:55:37.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:37.311+0000 7f559b7fe640 1 --2- 172.21.15.1:0/2606559571 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f55900a2c90 0x7f55900a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:37.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:37.311+0000 7f55aa4b9640 1 --2- 172.21.15.1:0/2606559571 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f55900a2c90 0x7f55900a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:37.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:37.311+0000 7f559b7fe640 1 -- 172.21.15.1:0/2606559571 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f559c0cf560 con 0x7f55a410c170 2024-09-11T21:55:37.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:37.311+0000 7f55aa4b9640 1 --2- 172.21.15.1:0/2606559571 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f55900a2c90 0x7f55900a5150 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f55a0005fd0 tx=0x7f55a0005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:55:37.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:37.315+0000 7f559b7fe640 1 -- 172.21.15.1:0/2606559571 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f559c09c510 con 0x7f55a410c170 2024-09-11T21:55:37.512 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:37 smithi161 ceph-mon[35615]: from='client.14694 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:55:37.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:37.520+0000 7f55ac71d640 1 -- 172.21.15.1:0/2606559571 --> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f55a4107250 con 0x7f55900a2c90 2024-09-11T21:55:37.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:37.524+0000 7f559b7fe640 1 -- 172.21.15.1:0/2606559571 <== mgr.14150 v2:172.21.15.1:6800/3890350886 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f55a000fb30 con 0x7f55900a2c90 2024-09-11T21:55:37.525 INFO:teuthology.orchestra.run.smithi001.stdout:/volumes/qa/sv_0/bc26220f-1248-4c7c-b84b-a4f1e0005283 2024-09-11T21:55:37.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:37.526+0000 7f55ac71d640 1 -- 172.21.15.1:0/2606559571 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f55900a2c90 msgr2=0x7f55900a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:37.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:37.526+0000 7f55ac71d640 1 --2- 172.21.15.1:0/2606559571 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f55900a2c90 0x7f55900a5150 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f55a0005fd0 tx=0x7f55a0005850 comp rx=0 tx=0).stop 2024-09-11T21:55:37.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:37.526+0000 7f55ac71d640 1 -- 172.21.15.1:0/2606559571 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f55a410c170 msgr2=0x7f55a4101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:37.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:37.526+0000 7f55ac71d640 1 --2- 172.21.15.1:0/2606559571 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f55a410c170 0x7f55a4101420 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f559c00eb10 tx=0x7f559c00c2d0 comp rx=0 tx=0).stop 2024-09-11T21:55:37.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:37.526+0000 7f55ac71d640 1 -- 172.21.15.1:0/2606559571 shutdown_connections 2024-09-11T21:55:37.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:37.526+0000 7f55ac71d640 1 --2- 172.21.15.1:0/2606559571 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f55900a2c90 0x7f55900a5150 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:37.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:37.526+0000 7f55ac71d640 1 --2- 172.21.15.1:0/2606559571 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f55a410c170 0x7f55a4101420 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:37.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:37.526+0000 7f55ac71d640 1 --2- 172.21.15.1:0/2606559571 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55a4103660 0x7f55a4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:37.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:37.526+0000 7f55ac71d640 1 --2- 172.21.15.1:0/2606559571 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55a4102d40 0x7f55a41009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:37.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:37.526+0000 7f55ac71d640 1 -- 172.21.15.1:0/2606559571 >> 172.21.15.1:0/2606559571 conn(0x7f55a40fc7a0 msgr2=0x7f55a40fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:55:37.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:37.526+0000 7f55ac71d640 1 -- 172.21.15.1:0/2606559571 shutdown_connections 2024-09-11T21:55:37.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:37.526+0000 7f55ac71d640 1 -- 172.21.15.1:0/2606559571 wait complete. 2024-09-11T21:55:37.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:37 smithi053 ceph-mon[34908]: from='client.14694 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:55:38.239 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-11T21:55:38.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:38 smithi001 ceph-mon[30935]: pgmap v290: 97 pgs: 97 active+clean; 592 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 5.4 KiB/s wr, 19 op/s 2024-09-11T21:55:38.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:38 smithi001 ceph-mon[30935]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:55:38.497 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:55:38.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:38 smithi053 ceph-mon[34908]: pgmap v290: 97 pgs: 97 active+clean; 592 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 5.4 KiB/s wr, 19 op/s 2024-09-11T21:55:38.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:38 smithi053 ceph-mon[34908]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:55:38.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:38 smithi161 ceph-mon[35615]: pgmap v290: 97 pgs: 97 active+clean; 592 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 5.4 KiB/s wr, 19 op/s 2024-09-11T21:55:38.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:38 smithi161 ceph-mon[35615]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:55:39.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:38.989+0000 7fbcc4c81640 1 -- 172.21.15.1:0/3482313249 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbcc0100d20 msgr2=0x7fbcc0106860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:39.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:38.989+0000 7fbcc4c81640 1 --2- 172.21.15.1:0/3482313249 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbcc0100d20 0x7fbcc0106860 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbcb4009930 tx=0x7fbcb402a420 comp rx=0 tx=0).stop 2024-09-11T21:55:39.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:38.989+0000 7fbcc4c81640 1 -- 172.21.15.1:0/3482313249 shutdown_connections 2024-09-11T21:55:39.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:38.989+0000 7fbcc4c81640 1 --2- 172.21.15.1:0/3482313249 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbcc0106fc0 0x7fbcc010eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:39.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:38.989+0000 7fbcc4c81640 1 --2- 172.21.15.1:0/3482313249 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbcc0100d20 0x7fbcc0106860 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:39.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:38.989+0000 7fbcc4c81640 1 --2- 172.21.15.1:0/3482313249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbcc0100370 0x7fbcc0100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:39.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:38.989+0000 7fbcc4c81640 1 -- 172.21.15.1:0/3482313249 >> 172.21.15.1:0/3482313249 conn(0x7fbcc0077210 msgr2=0x7fbcc00fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:55:39.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:38.989+0000 7fbcc4c81640 1 -- 172.21.15.1:0/3482313249 shutdown_connections 2024-09-11T21:55:39.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:38.989+0000 7fbcc4c81640 1 -- 172.21.15.1:0/3482313249 wait complete. 2024-09-11T21:55:39.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:38.990+0000 7fbcc4c81640 1 Processor -- start 2024-09-11T21:55:39.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:38.990+0000 7fbcc4c81640 1 -- start start 2024-09-11T21:55:39.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:38.990+0000 7fbcc4c81640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbcc0100370 0x7fbcc01012f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:39.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:38.990+0000 7fbcc4c81640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbcc0100d20 0x7fbcc0101830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:39.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:38.991+0000 7fbcc4c81640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbcc0106fc0 0x7fbcc0101d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:39.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:38.991+0000 7fbcc4c81640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc0111fe0 con 0x7fbcc0106fc0 2024-09-11T21:55:39.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:38.991+0000 7fbcc4c81640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc0111e60 con 0x7fbcc0100d20 2024-09-11T21:55:39.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:38.991+0000 7fbcc4c81640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc0112160 con 0x7fbcc0100370 2024-09-11T21:55:39.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:38.991+0000 7fbcbed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbcc0106fc0 0x7fbcc0101d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:39.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:38.991+0000 7fbcbed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbcc0106fc0 0x7fbcc0101d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44872/0 (socket says 172.21.15.1:44872) 2024-09-11T21:55:39.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:38.991+0000 7fbcbed9d640 1 -- 172.21.15.1:0/3113497802 learned_addr learned my addr 172.21.15.1:0/3113497802 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:55:39.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:38.991+0000 7fbcbe59c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbcc0100370 0x7fbcc01012f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:39.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:38.991+0000 7fbcbdd9b640 1 --2- 172.21.15.1:0/3113497802 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbcc0100d20 0x7fbcc0101830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:39.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:38.991+0000 7fbcbed9d640 1 -- 172.21.15.1:0/3113497802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbcc0100370 msgr2=0x7fbcc01012f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:39.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:38.991+0000 7fbcbed9d640 1 --2- 172.21.15.1:0/3113497802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbcc0100370 0x7fbcc01012f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:39.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:38.991+0000 7fbcbed9d640 1 -- 172.21.15.1:0/3113497802 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbcc0100d20 msgr2=0x7fbcc0101830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:39.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:38.991+0000 7fbcbed9d640 1 --2- 172.21.15.1:0/3113497802 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbcc0100d20 0x7fbcc0101830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:39.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:38.991+0000 7fbcbed9d640 1 -- 172.21.15.1:0/3113497802 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcc01a69e0 con 0x7fbcc0106fc0 2024-09-11T21:55:39.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:38.991+0000 7fbcbed9d640 1 --2- 172.21.15.1:0/3113497802 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbcc0106fc0 0x7fbcc0101d70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fbcb800eaa0 tx=0x7fbcb800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:55:39.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:38.991+0000 7fbcaf7fe640 1 -- 172.21.15.1:0/3113497802 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb8004d20 con 0x7fbcc0106fc0 2024-09-11T21:55:39.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:38.991+0000 7fbcaf7fe640 1 -- 172.21.15.1:0/3113497802 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcb8005020 con 0x7fbcc0106fc0 2024-09-11T21:55:39.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:38.991+0000 7fbcc4c81640 1 -- 172.21.15.1:0/3113497802 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcc01a6cd0 con 0x7fbcc0106fc0 2024-09-11T21:55:39.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:38.991+0000 7fbcaf7fe640 1 -- 172.21.15.1:0/3113497802 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb80116d0 con 0x7fbcc0106fc0 2024-09-11T21:55:39.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:38.991+0000 7fbcc4c81640 1 -- 172.21.15.1:0/3113497802 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcc01a71e0 con 0x7fbcc0106fc0 2024-09-11T21:55:39.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:38.992+0000 7fbcc4c81640 1 -- 172.21.15.1:0/3113497802 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcc0100750 con 0x7fbcc0106fc0 2024-09-11T21:55:39.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:38.994+0000 7fbcaf7fe640 1 -- 172.21.15.1:0/3113497802 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fbcb80051d0 con 0x7fbcc0106fc0 2024-09-11T21:55:39.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:38.994+0000 7fbcaf7fe640 1 --2- 172.21.15.1:0/3113497802 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fbca80a2c10 0x7fbca80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:39.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:38.994+0000 7fbcbe59c640 1 --2- 172.21.15.1:0/3113497802 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fbca80a2c10 0x7fbca80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:39.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:38.994+0000 7fbcaf7fe640 1 -- 172.21.15.1:0/3113497802 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7fbcb80cf1a0 con 0x7fbcc0106fc0 2024-09-11T21:55:39.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:38.995+0000 7fbcbe59c640 1 --2- 172.21.15.1:0/3113497802 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fbca80a2c10 0x7fbca80a50d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbcb0009990 tx=0x7fbcb0009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:55:39.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:38.999+0000 7fbcaf7fe640 1 -- 172.21.15.1:0/3113497802 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbcb8016030 con 0x7fbcc0106fc0 2024-09-11T21:55:39.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:39.203+0000 7fbcc4c81640 1 -- 172.21.15.1:0/3113497802 --> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fbcc0062630 con 0x7fbca80a2c10 2024-09-11T21:55:39.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:39.229+0000 7fbcaf7fe640 1 -- 172.21.15.1:0/3113497802 <== mgr.14150 v2:172.21.15.1:6800/3890350886 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fbcc0062630 con 0x7fbca80a2c10 2024-09-11T21:55:39.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:39.231+0000 7fbcc4c81640 1 -- 172.21.15.1:0/3113497802 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fbca80a2c10 msgr2=0x7fbca80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:39.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:39.231+0000 7fbcc4c81640 1 --2- 172.21.15.1:0/3113497802 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fbca80a2c10 0x7fbca80a50d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbcb0009990 tx=0x7fbcb0009110 comp rx=0 tx=0).stop 2024-09-11T21:55:39.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:39.231+0000 7fbcc4c81640 1 -- 172.21.15.1:0/3113497802 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbcc0106fc0 msgr2=0x7fbcc0101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:39.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:39.231+0000 7fbcc4c81640 1 --2- 172.21.15.1:0/3113497802 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbcc0106fc0 0x7fbcc0101d70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fbcb800eaa0 tx=0x7fbcb800eea0 comp rx=0 tx=0).stop 2024-09-11T21:55:39.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:39.231+0000 7fbcc4c81640 1 -- 172.21.15.1:0/3113497802 shutdown_connections 2024-09-11T21:55:39.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:39.231+0000 7fbcc4c81640 1 --2- 172.21.15.1:0/3113497802 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fbca80a2c10 0x7fbca80a50d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:39.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:39.231+0000 7fbcc4c81640 1 --2- 172.21.15.1:0/3113497802 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbcc0106fc0 0x7fbcc0101d70 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:39.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:39.231+0000 7fbcc4c81640 1 --2- 172.21.15.1:0/3113497802 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbcc0100d20 0x7fbcc0101830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:39.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:39.231+0000 7fbcc4c81640 1 --2- 172.21.15.1:0/3113497802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbcc0100370 0x7fbcc01012f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:39.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:39.231+0000 7fbcc4c81640 1 -- 172.21.15.1:0/3113497802 >> 172.21.15.1:0/3113497802 conn(0x7fbcc0077210 msgr2=0x7fbcc00fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:55:39.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:39.231+0000 7fbcc4c81640 1 -- 172.21.15.1:0/3113497802 shutdown_connections 2024-09-11T21:55:39.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:39.231+0000 7fbcc4c81640 1 -- 172.21.15.1:0/3113497802 wait complete. 2024-09-11T21:55:39.993 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-11T21:55:40.254 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:55:40.375 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:40 smithi001 ceph-mon[30935]: pgmap v291: 97 pgs: 97 active+clean; 592 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 5.1 KiB/s wr, 20 op/s 2024-09-11T21:55:40.376 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:40 smithi001 ceph-mon[30935]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:55:40.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:40 smithi053 ceph-mon[34908]: pgmap v291: 97 pgs: 97 active+clean; 592 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 5.1 KiB/s wr, 20 op/s 2024-09-11T21:55:40.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:40 smithi053 ceph-mon[34908]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:55:40.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:40 smithi161 ceph-mon[35615]: pgmap v291: 97 pgs: 97 active+clean; 592 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 5.1 KiB/s wr, 20 op/s 2024-09-11T21:55:40.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:40 smithi161 ceph-mon[35615]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:55:40.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:40.809+0000 7fac497a2640 1 -- 172.21.15.1:0/495037726 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fac441008d0 msgr2=0x7fac44100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:40.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:40.809+0000 7fac497a2640 1 --2- 172.21.15.1:0/495037726 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fac441008d0 0x7fac44100d30 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fac38009930 tx=0x7fac3802a420 comp rx=0 tx=0).stop 2024-09-11T21:55:40.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:40.809+0000 7fac497a2640 1 -- 172.21.15.1:0/495037726 shutdown_connections 2024-09-11T21:55:40.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:40.809+0000 7fac497a2640 1 --2- 172.21.15.1:0/495037726 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac44101270 0x7fac4410d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:40.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:40.809+0000 7fac497a2640 1 --2- 172.21.15.1:0/495037726 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fac441008d0 0x7fac44100d30 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:40.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:40.809+0000 7fac497a2640 1 --2- 172.21.15.1:0/495037726 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac441068d0 0x7fac44106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:40.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:40.809+0000 7fac497a2640 1 -- 172.21.15.1:0/495037726 >> 172.21.15.1:0/495037726 conn(0x7fac440fc610 msgr2=0x7fac440fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:55:40.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:40.809+0000 7fac497a2640 1 -- 172.21.15.1:0/495037726 shutdown_connections 2024-09-11T21:55:40.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:40.809+0000 7fac497a2640 1 -- 172.21.15.1:0/495037726 wait complete. 2024-09-11T21:55:40.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:40.809+0000 7fac497a2640 1 Processor -- start 2024-09-11T21:55:40.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:40.809+0000 7fac497a2640 1 -- start start 2024-09-11T21:55:40.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:40.810+0000 7fac497a2640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac441008d0 0x7fac44198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:40.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:40.810+0000 7fac497a2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fac44101270 0x7fac44198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:40.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:40.810+0000 7fac497a2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac441068d0 0x7fac4419cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:40.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:40.810+0000 7fac497a2640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fac44110c90 con 0x7fac44101270 2024-09-11T21:55:40.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:40.810+0000 7fac497a2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fac44110b10 con 0x7fac441068d0 2024-09-11T21:55:40.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:40.810+0000 7fac497a2640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fac44110e10 con 0x7fac441008d0 2024-09-11T21:55:40.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:40.810+0000 7fac437fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fac44101270 0x7fac44198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:40.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:40.810+0000 7fac437fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fac44101270 0x7fac44198780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44896/0 (socket says 172.21.15.1:44896) 2024-09-11T21:55:40.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:40.810+0000 7fac437fe640 1 -- 172.21.15.1:0/4025355175 learned_addr learned my addr 172.21.15.1:0/4025355175 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:55:40.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:40.810+0000 7fac48fa1640 1 --2- 172.21.15.1:0/4025355175 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac441068d0 0x7fac4419cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:40.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:40.810+0000 7fac43fff640 1 --2- 172.21.15.1:0/4025355175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac441008d0 0x7fac44198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:40.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:40.810+0000 7fac437fe640 1 -- 172.21.15.1:0/4025355175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac441008d0 msgr2=0x7fac44198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:40.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:40.810+0000 7fac437fe640 1 --2- 172.21.15.1:0/4025355175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac441008d0 0x7fac44198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:40.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:40.810+0000 7fac437fe640 1 -- 172.21.15.1:0/4025355175 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac441068d0 msgr2=0x7fac4419cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:40.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:40.810+0000 7fac437fe640 1 --2- 172.21.15.1:0/4025355175 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac441068d0 0x7fac4419cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:40.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:40.810+0000 7fac437fe640 1 -- 172.21.15.1:0/4025355175 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac4419d1f0 con 0x7fac44101270 2024-09-11T21:55:40.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:40.810+0000 7fac48fa1640 1 --2- 172.21.15.1:0/4025355175 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac441068d0 0x7fac4419cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:55:40.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:40.810+0000 7fac43fff640 1 --2- 172.21.15.1:0/4025355175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac441008d0 0x7fac44198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T21:55:40.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:40.810+0000 7fac437fe640 1 --2- 172.21.15.1:0/4025355175 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fac44101270 0x7fac44198780 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fac38009230 tx=0x7fac38004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:55:40.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:40.811+0000 7fac417fa640 1 -- 172.21.15.1:0/4025355175 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3803f030 con 0x7fac44101270 2024-09-11T21:55:40.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:40.811+0000 7fac417fa640 1 -- 172.21.15.1:0/4025355175 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac38003ad0 con 0x7fac44101270 2024-09-11T21:55:40.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:40.811+0000 7fac497a2640 1 -- 172.21.15.1:0/4025355175 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac4419d480 con 0x7fac44101270 2024-09-11T21:55:40.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:40.811+0000 7fac417fa640 1 -- 172.21.15.1:0/4025355175 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac38035070 con 0x7fac44101270 2024-09-11T21:55:40.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:40.811+0000 7fac497a2640 1 -- 172.21.15.1:0/4025355175 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac441a4dc0 con 0x7fac44101270 2024-09-11T21:55:40.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:40.812+0000 7fac497a2640 1 -- 172.21.15.1:0/4025355175 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac44102010 con 0x7fac44101270 2024-09-11T21:55:40.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:40.813+0000 7fac417fa640 1 -- 172.21.15.1:0/4025355175 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fac38042030 con 0x7fac44101270 2024-09-11T21:55:40.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:40.814+0000 7fac417fa640 1 --2- 172.21.15.1:0/4025355175 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fac2c0a2c10 0x7fac2c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:40.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:40.814+0000 7fac43fff640 1 --2- 172.21.15.1:0/4025355175 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fac2c0a2c10 0x7fac2c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:40.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:40.814+0000 7fac417fa640 1 -- 172.21.15.1:0/4025355175 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7fac380ef5c0 con 0x7fac44101270 2024-09-11T21:55:40.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:40.814+0000 7fac43fff640 1 --2- 172.21.15.1:0/4025355175 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fac2c0a2c10 0x7fac2c0a50d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fac3400baa0 tx=0x7fac34005d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:55:40.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:40.818+0000 7fac417fa640 1 -- 172.21.15.1:0/4025355175 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fac380bc570 con 0x7fac44101270 2024-09-11T21:55:41.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:41.019+0000 7fac497a2640 1 -- 172.21.15.1:0/4025355175 --> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fac44062630 con 0x7fac2c0a2c10 2024-09-11T21:55:41.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:41.024+0000 7fac417fa640 1 -- 172.21.15.1:0/4025355175 <== mgr.14150 v2:172.21.15.1:6800/3890350886 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fac44062630 con 0x7fac2c0a2c10 2024-09-11T21:55:41.024 INFO:teuthology.orchestra.run.smithi001.stdout:/volumes/qa/sv_1/799b6bbd-6a94-447f-9bf5-8b4eb83a6f36 2024-09-11T21:55:41.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:41.026+0000 7fac497a2640 1 -- 172.21.15.1:0/4025355175 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fac2c0a2c10 msgr2=0x7fac2c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:41.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:41.026+0000 7fac497a2640 1 --2- 172.21.15.1:0/4025355175 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fac2c0a2c10 0x7fac2c0a50d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fac3400baa0 tx=0x7fac34005d20 comp rx=0 tx=0).stop 2024-09-11T21:55:41.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:41.026+0000 7fac497a2640 1 -- 172.21.15.1:0/4025355175 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fac44101270 msgr2=0x7fac44198780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:41.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:41.026+0000 7fac497a2640 1 --2- 172.21.15.1:0/4025355175 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fac44101270 0x7fac44198780 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fac38009230 tx=0x7fac38004270 comp rx=0 tx=0).stop 2024-09-11T21:55:41.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:41.026+0000 7fac497a2640 1 -- 172.21.15.1:0/4025355175 shutdown_connections 2024-09-11T21:55:41.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:41.026+0000 7fac497a2640 1 --2- 172.21.15.1:0/4025355175 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fac2c0a2c10 0x7fac2c0a50d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:41.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:41.026+0000 7fac497a2640 1 --2- 172.21.15.1:0/4025355175 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fac441068d0 0x7fac4419cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:41.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:41.026+0000 7fac497a2640 1 --2- 172.21.15.1:0/4025355175 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fac44101270 0x7fac44198780 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:41.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:41.026+0000 7fac497a2640 1 --2- 172.21.15.1:0/4025355175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac441008d0 0x7fac44198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:41.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:41.026+0000 7fac497a2640 1 -- 172.21.15.1:0/4025355175 >> 172.21.15.1:0/4025355175 conn(0x7fac440fc610 msgr2=0x7fac4406a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:55:41.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:41.026+0000 7fac497a2640 1 -- 172.21.15.1:0/4025355175 shutdown_connections 2024-09-11T21:55:41.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:41.026+0000 7fac497a2640 1 -- 172.21.15.1:0/4025355175 wait complete. 2024-09-11T21:55:41.719 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T21:55:41.978 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:55:42.376 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:42 smithi001 ceph-mon[30935]: pgmap v292: 97 pgs: 97 active+clean; 624 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.9 KiB/s wr, 23 op/s 2024-09-11T21:55:42.376 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:42 smithi001 ceph-mon[30935]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:55:42.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:42.457+0000 7f8a26df7640 1 -- 172.21.15.1:0/4010155615 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a20101280 msgr2=0x7f8a2010df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:42.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:42.457+0000 7f8a26df7640 1 --2- 172.21.15.1:0/4010155615 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a20101280 0x7f8a2010df40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c009900 tx=0x7f8a0c02a410 comp rx=0 tx=0).stop 2024-09-11T21:55:42.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:42.457+0000 7f8a26df7640 1 -- 172.21.15.1:0/4010155615 shutdown_connections 2024-09-11T21:55:42.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:42.457+0000 7f8a26df7640 1 --2- 172.21.15.1:0/4010155615 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a2010e480 0x7f8a20110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:42.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:42.457+0000 7f8a26df7640 1 --2- 172.21.15.1:0/4010155615 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a20101280 0x7f8a2010df40 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:42.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:42.457+0000 7f8a26df7640 1 --2- 172.21.15.1:0/4010155615 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a201008d0 0x7f8a20100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:42.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:42.457+0000 7f8a26df7640 1 -- 172.21.15.1:0/4010155615 >> 172.21.15.1:0/4010155615 conn(0x7f8a200fc7a0 msgr2=0x7f8a200febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:55:42.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:42.457+0000 7f8a26df7640 1 -- 172.21.15.1:0/4010155615 shutdown_connections 2024-09-11T21:55:42.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:42.457+0000 7f8a26df7640 1 -- 172.21.15.1:0/4010155615 wait complete. 2024-09-11T21:55:42.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:42.458+0000 7f8a26df7640 1 Processor -- start 2024-09-11T21:55:42.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:42.458+0000 7f8a26df7640 1 -- start start 2024-09-11T21:55:42.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:42.458+0000 7f8a26df7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a201008d0 0x7f8a2019a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:42.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:42.458+0000 7f8a26df7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a20101280 0x7f8a2019ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:42.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:42.458+0000 7f8a26df7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a2010e480 0x7f8a2019efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:42.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:42.458+0000 7f8a26df7640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8a20114190 con 0x7f8a201008d0 2024-09-11T21:55:42.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:42.458+0000 7f8a26df7640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8a20114010 con 0x7f8a20101280 2024-09-11T21:55:42.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:42.458+0000 7f8a26df7640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8a20114310 con 0x7f8a2010e480 2024-09-11T21:55:42.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:42.458+0000 7f8a24b93640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a201008d0 0x7f8a2019a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:42.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:42.458+0000 7f8a24b93640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a201008d0 0x7f8a2019a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44908/0 (socket says 172.21.15.1:44908) 2024-09-11T21:55:42.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:42.458+0000 7f8a24b93640 1 -- 172.21.15.1:0/2604120451 learned_addr learned my addr 172.21.15.1:0/2604120451 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:55:42.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:42.458+0000 7f8a1ffff640 1 --2- 172.21.15.1:0/2604120451 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a20101280 0x7f8a2019ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:42.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:42.458+0000 7f8a25394640 1 --2- 172.21.15.1:0/2604120451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a2010e480 0x7f8a2019efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:42.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:42.459+0000 7f8a24b93640 1 -- 172.21.15.1:0/2604120451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a2010e480 msgr2=0x7f8a2019efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:42.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:42.459+0000 7f8a24b93640 1 --2- 172.21.15.1:0/2604120451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a2010e480 0x7f8a2019efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:42.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:42.459+0000 7f8a24b93640 1 -- 172.21.15.1:0/2604120451 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a20101280 msgr2=0x7f8a2019ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:42.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:42.459+0000 7f8a24b93640 1 --2- 172.21.15.1:0/2604120451 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a20101280 0x7f8a2019ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:42.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:42.459+0000 7f8a24b93640 1 -- 172.21.15.1:0/2604120451 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a2019f680 con 0x7f8a201008d0 2024-09-11T21:55:42.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:42.459+0000 7f8a24b93640 1 --2- 172.21.15.1:0/2604120451 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a201008d0 0x7f8a2019a6d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8a1400d950 tx=0x7f8a1400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:55:42.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:42.459+0000 7f8a1dffb640 1 -- 172.21.15.1:0/2604120451 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a14017070 con 0x7f8a201008d0 2024-09-11T21:55:42.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:42.459+0000 7f8a1dffb640 1 -- 172.21.15.1:0/2604120451 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a14009c30 con 0x7f8a201008d0 2024-09-11T21:55:42.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:42.459+0000 7f8a26df7640 1 -- 172.21.15.1:0/2604120451 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a2019f970 con 0x7f8a201008d0 2024-09-11T21:55:42.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:42.459+0000 7f8a1dffb640 1 -- 172.21.15.1:0/2604120451 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1400f6b0 con 0x7f8a201008d0 2024-09-11T21:55:42.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:42.459+0000 7f8a26df7640 1 -- 172.21.15.1:0/2604120451 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a201a7250 con 0x7f8a201008d0 2024-09-11T21:55:42.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:42.461+0000 7f8a26df7640 1 -- 172.21.15.1:0/2604120451 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a20100cb0 con 0x7f8a201008d0 2024-09-11T21:55:42.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:42.462+0000 7f8a1dffb640 1 -- 172.21.15.1:0/2604120451 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f8a140109d0 con 0x7f8a201008d0 2024-09-11T21:55:42.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:42.462+0000 7f8a1dffb640 1 --2- 172.21.15.1:0/2604120451 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f8a100a2c10 0x7f8a100a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:42.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:42.462+0000 7f8a1dffb640 1 -- 172.21.15.1:0/2604120451 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f8a140ced00 con 0x7f8a201008d0 2024-09-11T21:55:42.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:42.462+0000 7f8a1ffff640 1 --2- 172.21.15.1:0/2604120451 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f8a100a2c10 0x7f8a100a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:42.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:42.463+0000 7f8a1ffff640 1 --2- 172.21.15.1:0/2604120451 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f8a100a2c10 0x7f8a100a50d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c009230 tx=0x7f8a0c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:55:42.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:42.466+0000 7f8a1dffb640 1 -- 172.21.15.1:0/2604120451 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8a14009de0 con 0x7f8a201008d0 2024-09-11T21:55:42.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:42 smithi053 ceph-mon[34908]: pgmap v292: 97 pgs: 97 active+clean; 624 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.9 KiB/s wr, 23 op/s 2024-09-11T21:55:42.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:42 smithi053 ceph-mon[34908]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:55:42.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:42 smithi161 ceph-mon[35615]: pgmap v292: 97 pgs: 97 active+clean; 624 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.9 KiB/s wr, 23 op/s 2024-09-11T21:55:42.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:42 smithi161 ceph-mon[35615]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:55:42.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:42.679+0000 7f8a26df7640 1 -- 172.21.15.1:0/2604120451 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8a20109610 con 0x7f8a201008d0 2024-09-11T21:55:42.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:42.680+0000 7f8a1dffb640 1 -- 172.21.15.1:0/2604120451 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11949 (secure 0 0 0) 0x7f8a1409b560 con 0x7f8a201008d0 2024-09-11T21:55:42.681 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:55:42.681 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":27,"btime":"2024-09-11T21:55:32:152241+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/421542832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":421542832},{"type":"v1","addr":"172.21.15.1:6835","nonce":421542832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2836864406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2836864406},{"type":"v1","addr":"172.21.15.1:6837","nonce":2836864406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T21:55:20.791263+0000","modified":"2024-09-11T21:55:32.152227+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24484,"mds_2":24467,"mds_3":24466,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14595":{"gid":14595,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/407834672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":407834672},{"type":"v1","addr":"172.21.15.1:6839","nonce":407834672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/1068827678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":1068827678},{"type":"v1","addr":"172.21.15.1:6841","nonce":1068827678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/872410024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":872410024},{"type":"v1","addr":"172.21.15.161:6833","nonce":872410024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/1648944570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1648944570},{"type":"v1","addr":"172.21.15.53:6833","nonce":1648944570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/2126933993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2126933993},{"type":"v1","addr":"172.21.15.53:6835","nonce":2126933993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.161:6835/828381592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":828381592},{"type":"v1","addr":"172.21.15.161:6835","nonce":828381592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.161:6837/1799318656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1799318656},{"type":"v1","addr":"172.21.15.161:6837","nonce":1799318656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.53:6837/3457131830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3457131830},{"type":"v1","addr":"172.21.15.53:6837","nonce":3457131830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/455365383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":455365383},{"type":"v1","addr":"172.21.15.161:6839","nonce":455365383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.53:6839/1378584447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1378584447},{"type":"v1","addr":"172.21.15.53:6839","nonce":1378584447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24484,24467,24466,24460]},"id":1}]} 2024-09-11T21:55:42.681 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 27 2024-09-11T21:55:42.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:42.682+0000 7f8a26df7640 1 -- 172.21.15.1:0/2604120451 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f8a100a2c10 msgr2=0x7f8a100a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:42.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:42.682+0000 7f8a26df7640 1 --2- 172.21.15.1:0/2604120451 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f8a100a2c10 0x7f8a100a50d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c009230 tx=0x7f8a0c0058d0 comp rx=0 tx=0).stop 2024-09-11T21:55:42.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:42.682+0000 7f8a26df7640 1 -- 172.21.15.1:0/2604120451 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a201008d0 msgr2=0x7f8a2019a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:42.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:42.682+0000 7f8a26df7640 1 --2- 172.21.15.1:0/2604120451 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a201008d0 0x7f8a2019a6d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8a1400d950 tx=0x7f8a1400de10 comp rx=0 tx=0).stop 2024-09-11T21:55:42.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:42.682+0000 7f8a26df7640 1 -- 172.21.15.1:0/2604120451 shutdown_connections 2024-09-11T21:55:42.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:42.682+0000 7f8a26df7640 1 --2- 172.21.15.1:0/2604120451 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f8a100a2c10 0x7f8a100a50d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:42.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:42.682+0000 7f8a26df7640 1 --2- 172.21.15.1:0/2604120451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a2010e480 0x7f8a2019efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:42.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:42.682+0000 7f8a26df7640 1 --2- 172.21.15.1:0/2604120451 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8a20101280 0x7f8a2019ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:42.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:42.682+0000 7f8a26df7640 1 --2- 172.21.15.1:0/2604120451 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a201008d0 0x7f8a2019a6d0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:42.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:42.682+0000 7f8a26df7640 1 -- 172.21.15.1:0/2604120451 >> 172.21.15.1:0/2604120451 conn(0x7f8a200fc7a0 msgr2=0x7f8a200feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:55:42.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:42.682+0000 7f8a26df7640 1 -- 172.21.15.1:0/2604120451 shutdown_connections 2024-09-11T21:55:42.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:42.682+0000 7f8a26df7640 1 -- 172.21.15.1:0/2604120451 wait complete. 2024-09-11T21:55:43.350 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T21:55:43.375 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:43 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/2604120451' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:55:43.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:43 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/2604120451' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:55:43.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:43 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/2604120451' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:55:43.609 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:55:44.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:44.063+0000 7fb71b96b640 1 -- 172.21.15.1:0/425682408 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb7141036e0 msgr2=0x7fb714107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:44.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:44.063+0000 7fb71b96b640 1 --2- 172.21.15.1:0/425682408 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb7141036e0 0x7fb714107b90 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb708009930 tx=0x7fb70802a420 comp rx=0 tx=0).stop 2024-09-11T21:55:44.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:44.063+0000 7fb71b96b640 1 -- 172.21.15.1:0/425682408 shutdown_connections 2024-09-11T21:55:44.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:44.063+0000 7fb71b96b640 1 --2- 172.21.15.1:0/425682408 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7141082a0 0x7fb71410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:44.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:44.063+0000 7fb71b96b640 1 --2- 172.21.15.1:0/425682408 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb7141036e0 0x7fb714107b90 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:44.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:44.063+0000 7fb71b96b640 1 --2- 172.21.15.1:0/425682408 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb714102d30 0x7fb714103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:44.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:44.063+0000 7fb71b96b640 1 -- 172.21.15.1:0/425682408 >> 172.21.15.1:0/425682408 conn(0x7fb7140fc7d0 msgr2=0x7fb7140febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:55:44.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:44.064+0000 7fb71b96b640 1 -- 172.21.15.1:0/425682408 shutdown_connections 2024-09-11T21:55:44.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:44.064+0000 7fb71b96b640 1 -- 172.21.15.1:0/425682408 wait complete. 2024-09-11T21:55:44.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:44.064+0000 7fb71b96b640 1 Processor -- start 2024-09-11T21:55:44.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:44.064+0000 7fb71b96b640 1 -- start start 2024-09-11T21:55:44.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:44.064+0000 7fb71b96b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb714102d30 0x7fb714198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:44.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:44.064+0000 7fb71b96b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb7141036e0 0x7fb714198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:44.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:44.064+0000 7fb71b96b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7141082a0 0x7fb71419cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:44.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:44.064+0000 7fb71b96b640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb7141120a0 con 0x7fb7141036e0 2024-09-11T21:55:44.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:44.064+0000 7fb71b96b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb714111f20 con 0x7fb714102d30 2024-09-11T21:55:44.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:44.064+0000 7fb71b96b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb714112220 con 0x7fb7141082a0 2024-09-11T21:55:44.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:44.064+0000 7fb718f06640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb7141036e0 0x7fb714198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:44.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:44.065+0000 7fb718f06640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb7141036e0 0x7fb714198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44936/0 (socket says 172.21.15.1:44936) 2024-09-11T21:55:44.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:44.065+0000 7fb718f06640 1 -- 172.21.15.1:0/1849592666 learned_addr learned my addr 172.21.15.1:0/1849592666 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:55:44.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:44.065+0000 7fb719f08640 1 --2- 172.21.15.1:0/1849592666 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7141082a0 0x7fb71419cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:44.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:44.065+0000 7fb719707640 1 --2- 172.21.15.1:0/1849592666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb714102d30 0x7fb714198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:44.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:44.065+0000 7fb718f06640 1 -- 172.21.15.1:0/1849592666 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7141082a0 msgr2=0x7fb71419cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:44.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:44.065+0000 7fb718f06640 1 --2- 172.21.15.1:0/1849592666 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7141082a0 0x7fb71419cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:44.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:44.065+0000 7fb718f06640 1 -- 172.21.15.1:0/1849592666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb714102d30 msgr2=0x7fb714198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:44.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:44.065+0000 7fb718f06640 1 --2- 172.21.15.1:0/1849592666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb714102d30 0x7fb714198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:44.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:44.065+0000 7fb718f06640 1 -- 172.21.15.1:0/1849592666 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb71419d4d0 con 0x7fb7141036e0 2024-09-11T21:55:44.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:44.065+0000 7fb718f06640 1 --2- 172.21.15.1:0/1849592666 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb7141036e0 0x7fb714198a60 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb708005940 tx=0x7fb7080042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:55:44.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:44.065+0000 7fb7067fc640 1 -- 172.21.15.1:0/1849592666 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb70803f030 con 0x7fb7141036e0 2024-09-11T21:55:44.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:44.065+0000 7fb7067fc640 1 -- 172.21.15.1:0/1849592666 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb708003ce0 con 0x7fb7141036e0 2024-09-11T21:55:44.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:44.065+0000 7fb71b96b640 1 -- 172.21.15.1:0/1849592666 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb71419d760 con 0x7fb7141036e0 2024-09-11T21:55:44.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:44.065+0000 7fb7067fc640 1 -- 172.21.15.1:0/1849592666 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb708035070 con 0x7fb7141036e0 2024-09-11T21:55:44.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:44.065+0000 7fb71b96b640 1 -- 172.21.15.1:0/1849592666 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7141a50a0 con 0x7fb7141036e0 2024-09-11T21:55:44.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:44.066+0000 7fb71b96b640 1 -- 172.21.15.1:0/1849592666 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb71406a850 con 0x7fb7141036e0 2024-09-11T21:55:44.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:44.067+0000 7fb7067fc640 1 -- 172.21.15.1:0/1849592666 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fb708042030 con 0x7fb7141036e0 2024-09-11T21:55:44.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:44.068+0000 7fb7067fc640 1 --2- 172.21.15.1:0/1849592666 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fb7000a2c10 0x7fb7000a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:44.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:44.068+0000 7fb719707640 1 --2- 172.21.15.1:0/1849592666 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fb7000a2c10 0x7fb7000a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:44.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:44.068+0000 7fb7067fc640 1 -- 172.21.15.1:0/1849592666 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7fb7080ef770 con 0x7fb7141036e0 2024-09-11T21:55:44.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:44.069+0000 7fb719707640 1 --2- 172.21.15.1:0/1849592666 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fb7000a2c10 0x7fb7000a50d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb71000abe0 tx=0x7fb71000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:55:44.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:44.072+0000 7fb7067fc640 1 -- 172.21.15.1:0/1849592666 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb7080bc620 con 0x7fb7141036e0 2024-09-11T21:55:44.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:44.255+0000 7fb71b96b640 1 -- 172.21.15.1:0/1849592666 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb71410a2b0 con 0x7fb7141036e0 2024-09-11T21:55:44.256 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:44 smithi001 ceph-mon[30935]: pgmap v293: 97 pgs: 97 active+clean; 624 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 7.7 KiB/s wr, 22 op/s 2024-09-11T21:55:44.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:44.256+0000 7fb7067fc640 1 -- 172.21.15.1:0/1849592666 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+22198 (secure 0 0 0) 0x7fb7080bbf20 con 0x7fb7141036e0 2024-09-11T21:55:44.257 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:55:44.257 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":84,"fsid":"3ffca83c-7087-11ef-bcea-c7b262605968","created":"2024-09-11T21:47:28.868707+0000","modified":"2024-09-11T21:55:20.791073+0000","last_up_change":"2024-09-11T21:54:29.319441+0000","last_in_change":"2024-09-11T21:54:05.532281+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T21:50:41.026490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T21:55:16.961999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T21:55:19.066228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"19e56504-18f1-4234-b2d6-f193ff47d8eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":4012142023},{"type":"v1","addr":"172.21.15.1:6803","nonce":4012142023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":4012142023},{"type":"v1","addr":"172.21.15.1:6805","nonce":4012142023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":4012142023},{"type":"v1","addr":"172.21.15.1:6809","nonce":4012142023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":4012142023},{"type":"v1","addr":"172.21.15.1:6807","nonce":4012142023}]},"public_addr":"172.21.15.1:6803/4012142023","cluster_addr":"172.21.15.1:6805/4012142023","heartbeat_back_addr":"172.21.15.1:6809/4012142023","heartbeat_front_addr":"172.21.15.1:6807/4012142023","state":["exists","up"]},{"osd":1,"uuid":"996c9f0c-c5a4-43b8-b8e0-d712f27c5dfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3039391924},{"type":"v1","addr":"172.21.15.53:6801","nonce":3039391924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3039391924},{"type":"v1","addr":"172.21.15.53:6803","nonce":3039391924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3039391924},{"type":"v1","addr":"172.21.15.53:6807","nonce":3039391924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3039391924},{"type":"v1","addr":"172.21.15.53:6805","nonce":3039391924}]},"public_addr":"172.21.15.53:6801/3039391924","cluster_addr":"172.21.15.53:6803/3039391924","heartbeat_back_addr":"172.21.15.53:6807/3039391924","heartbeat_front_addr":"172.21.15.53:6805/3039391924","state":["exists","up"]},{"osd":2,"uuid":"578d4260-41e1-4645-b9b1-1c5445712c13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1707914885},{"type":"v1","addr":"172.21.15.161:6801","nonce":1707914885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1707914885},{"type":"v1","addr":"172.21.15.161:6803","nonce":1707914885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1707914885},{"type":"v1","addr":"172.21.15.161:6807","nonce":1707914885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1707914885},{"type":"v1","addr":"172.21.15.161:6805","nonce":1707914885}]},"public_addr":"172.21.15.161:6801/1707914885","cluster_addr":"172.21.15.161:6803/1707914885","heartbeat_back_addr":"172.21.15.161:6807/1707914885","heartbeat_front_addr":"172.21.15.161:6805/1707914885","state":["exists","up"]},{"osd":3,"uuid":"73f78766-9662-4aa3-841c-fe2ce654ee9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1541142398},{"type":"v1","addr":"172.21.15.1:6811","nonce":1541142398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1541142398},{"type":"v1","addr":"172.21.15.1:6813","nonce":1541142398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1541142398},{"type":"v1","addr":"172.21.15.1:6817","nonce":1541142398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1541142398},{"type":"v1","addr":"172.21.15.1:6815","nonce":1541142398}]},"public_addr":"172.21.15.1:6811/1541142398","cluster_addr":"172.21.15.1:6813/1541142398","heartbeat_back_addr":"172.21.15.1:6817/1541142398","heartbeat_front_addr":"172.21.15.1:6815/1541142398","state":["exists","up"]},{"osd":4,"uuid":"56c808bb-d4cd-4969-b49e-adef49ecd0f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":3081402681},{"type":"v1","addr":"172.21.15.53:6809","nonce":3081402681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3081402681},{"type":"v1","addr":"172.21.15.53:6811","nonce":3081402681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3081402681},{"type":"v1","addr":"172.21.15.53:6815","nonce":3081402681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3081402681},{"type":"v1","addr":"172.21.15.53:6813","nonce":3081402681}]},"public_addr":"172.21.15.53:6809/3081402681","cluster_addr":"172.21.15.53:6811/3081402681","heartbeat_back_addr":"172.21.15.53:6815/3081402681","heartbeat_front_addr":"172.21.15.53:6813/3081402681","state":["exists","up"]},{"osd":5,"uuid":"7b8266d6-b507-47c9-b4fa-ff16c4a74d35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":1622081327},{"type":"v1","addr":"172.21.15.161:6809","nonce":1622081327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":1622081327},{"type":"v1","addr":"172.21.15.161:6811","nonce":1622081327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":1622081327},{"type":"v1","addr":"172.21.15.161:6815","nonce":1622081327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":1622081327},{"type":"v1","addr":"172.21.15.161:6813","nonce":1622081327}]},"public_addr":"172.21.15.161:6809/1622081327","cluster_addr":"172.21.15.161:6811/1622081327","heartbeat_back_addr":"172.21.15.161:6815/1622081327","heartbeat_front_addr":"172.21.15.161:6813/1622081327","state":["exists","up"]},{"osd":6,"uuid":"3af5925d-a749-4849-a076-80580222cad4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":158434853},{"type":"v1","addr":"172.21.15.1:6819","nonce":158434853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":158434853},{"type":"v1","addr":"172.21.15.1:6821","nonce":158434853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":158434853},{"type":"v1","addr":"172.21.15.1:6825","nonce":158434853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":158434853},{"type":"v1","addr":"172.21.15.1:6823","nonce":158434853}]},"public_addr":"172.21.15.1:6819/158434853","cluster_addr":"172.21.15.1:6821/158434853","heartbeat_back_addr":"172.21.15.1:6825/158434853","heartbeat_front_addr":"172.21.15.1:6823/158434853","state":["exists","up"]},{"osd":7,"uuid":"f8f26c3c-17e9-4e89-8392-289796e0c8a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2132057697},{"type":"v1","addr":"172.21.15.53:6817","nonce":2132057697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2132057697},{"type":"v1","addr":"172.21.15.53:6819","nonce":2132057697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2132057697},{"type":"v1","addr":"172.21.15.53:6823","nonce":2132057697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2132057697},{"type":"v1","addr":"172.21.15.53:6821","nonce":2132057697}]},"public_addr":"172.21.15.53:6817/2132057697","cluster_addr":"172.21.15.53:6819/2132057697","heartbeat_back_addr":"172.21.15.53:6823/2132057697","heartbeat_front_addr":"172.21.15.53:6821/2132057697","state":["exists","up"]},{"osd":8,"uuid":"36ea401b-3c93-4b9d-8880-298d1eb6e30b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1838248669},{"type":"v1","addr":"172.21.15.161:6817","nonce":1838248669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1838248669},{"type":"v1","addr":"172.21.15.161:6819","nonce":1838248669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1838248669},{"type":"v1","addr":"172.21.15.161:6823","nonce":1838248669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1838248669},{"type":"v1","addr":"172.21.15.161:6821","nonce":1838248669}]},"public_addr":"172.21.15.161:6817/1838248669","cluster_addr":"172.21.15.161:6819/1838248669","heartbeat_back_addr":"172.21.15.161:6823/1838248669","heartbeat_front_addr":"172.21.15.161:6821/1838248669","state":["exists","up"]},{"osd":9,"uuid":"4b788d3f-a3bc-4463-bd99-95fdb7a52768","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2806274589},{"type":"v1","addr":"172.21.15.1:6827","nonce":2806274589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2806274589},{"type":"v1","addr":"172.21.15.1:6829","nonce":2806274589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":2806274589},{"type":"v1","addr":"172.21.15.1:6833","nonce":2806274589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2806274589},{"type":"v1","addr":"172.21.15.1:6831","nonce":2806274589}]},"public_addr":"172.21.15.1:6827/2806274589","cluster_addr":"172.21.15.1:6829/2806274589","heartbeat_back_addr":"172.21.15.1:6833/2806274589","heartbeat_front_addr":"172.21.15.1:6831/2806274589","state":["exists","up"]},{"osd":10,"uuid":"70549796-5270-4ae2-ac3f-f61f5de105d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":761923580},{"type":"v1","addr":"172.21.15.53:6825","nonce":761923580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":761923580},{"type":"v1","addr":"172.21.15.53:6827","nonce":761923580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":761923580},{"type":"v1","addr":"172.21.15.53:6831","nonce":761923580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":761923580},{"type":"v1","addr":"172.21.15.53:6829","nonce":761923580}]},"public_addr":"172.21.15.53:6825/761923580","cluster_addr":"172.21.15.53:6827/761923580","heartbeat_back_addr":"172.21.15.53:6831/761923580","heartbeat_front_addr":"172.21.15.53:6829/761923580","state":["exists","up"]},{"osd":11,"uuid":"48f17c4a-370a-493e-956e-bf6ad3dcc594","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1586352190},{"type":"v1","addr":"172.21.15.161:6825","nonce":1586352190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":1586352190},{"type":"v1","addr":"172.21.15.161:6827","nonce":1586352190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":1586352190},{"type":"v1","addr":"172.21.15.161:6831","nonce":1586352190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":1586352190},{"type":"v1","addr":"172.21.15.161:6829","nonce":1586352190}]},"public_addr":"172.21.15.161:6825/1586352190","cluster_addr":"172.21.15.161:6827/1586352190","heartbeat_back_addr":"172.21.15.161:6831/1586352190","heartbeat_front_addr":"172.21.15.161:6829/1586352190","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:49:46.539613+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:50:11.912300+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:50:36.955490+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:51:02.200554+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:51:27.860585+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:51:54.694099+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:52:18.962351+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:52:44.881923+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:53:10.301390+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:53:35.417647+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:54:01.492219+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:54:26.938338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/1749020850":"2024-09-12T21:48:24.524913+0000","172.21.15.1:6801/2821761445":"2024-09-12T21:48:24.524913+0000","172.21.15.1:0/1699984777":"2024-09-12T21:48:24.524913+0000","172.21.15.1:0/276770268":"2024-09-12T21:48:00.674728+0000","172.21.15.1:0/1278877732":"2024-09-12T21:48:24.524913+0000","172.21.15.1:0/2599895977":"2024-09-12T21:48:00.674728+0000","172.21.15.1:6801/2311309446":"2024-09-12T21:48:00.674728+0000","172.21.15.1:6800/2821761445":"2024-09-12T21:48:24.524913+0000","172.21.15.1:6800/2311309446":"2024-09-12T21:48:00.674728+0000","172.21.15.1:0/3202962495":"2024-09-12T21:48:00.674728+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T21:55:44.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:44.258+0000 7fb71b96b640 1 -- 172.21.15.1:0/1849592666 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fb7000a2c10 msgr2=0x7fb7000a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:44.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:44.258+0000 7fb71b96b640 1 --2- 172.21.15.1:0/1849592666 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fb7000a2c10 0x7fb7000a50d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb71000abe0 tx=0x7fb71000a2d0 comp rx=0 tx=0).stop 2024-09-11T21:55:44.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:44.258+0000 7fb71b96b640 1 -- 172.21.15.1:0/1849592666 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb7141036e0 msgr2=0x7fb714198a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:44.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:44.258+0000 7fb71b96b640 1 --2- 172.21.15.1:0/1849592666 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb7141036e0 0x7fb714198a60 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb708005940 tx=0x7fb7080042f0 comp rx=0 tx=0).stop 2024-09-11T21:55:44.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:44.258+0000 7fb71b96b640 1 -- 172.21.15.1:0/1849592666 shutdown_connections 2024-09-11T21:55:44.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:44.258+0000 7fb71b96b640 1 --2- 172.21.15.1:0/1849592666 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fb7000a2c10 0x7fb7000a50d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:44.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:44.258+0000 7fb71b96b640 1 --2- 172.21.15.1:0/1849592666 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7141082a0 0x7fb71419cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:44.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:44.258+0000 7fb71b96b640 1 --2- 172.21.15.1:0/1849592666 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb7141036e0 0x7fb714198a60 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:44.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:44.258+0000 7fb71b96b640 1 --2- 172.21.15.1:0/1849592666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb714102d30 0x7fb714198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:44.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:44.258+0000 7fb71b96b640 1 -- 172.21.15.1:0/1849592666 >> 172.21.15.1:0/1849592666 conn(0x7fb7140fc7d0 msgr2=0x7fb714105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:55:44.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:44.259+0000 7fb71b96b640 1 -- 172.21.15.1:0/1849592666 shutdown_connections 2024-09-11T21:55:44.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:44.259+0000 7fb71b96b640 1 -- 172.21.15.1:0/1849592666 wait complete. 2024-09-11T21:55:44.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:44 smithi053 ceph-mon[34908]: pgmap v293: 97 pgs: 97 active+clean; 624 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 7.7 KiB/s wr, 22 op/s 2024-09-11T21:55:44.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:44 smithi161 ceph-mon[35615]: pgmap v293: 97 pgs: 97 active+clean; 624 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 7.7 KiB/s wr, 22 op/s 2024-09-11T21:55:44.951 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-11T21:55:44.951 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T21:55:45.250 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:55:45.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:45 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/1849592666' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T21:55:45.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:45 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/1849592666' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T21:55:45.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:45 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/1849592666' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T21:55:45.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:45.704+0000 7fd5322b1640 1 -- 172.21.15.1:0/2312202581 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd52c103720 msgr2=0x7fd52c10bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:45.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:45.704+0000 7fd5322b1640 1 --2- 172.21.15.1:0/2312202581 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd52c103720 0x7fd52c10bc10 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd514009900 tx=0x7fd51402a480 comp rx=0 tx=0).stop 2024-09-11T21:55:45.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:45.704+0000 7fd5322b1640 1 -- 172.21.15.1:0/2312202581 shutdown_connections 2024-09-11T21:55:45.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:45.704+0000 7fd5322b1640 1 --2- 172.21.15.1:0/2312202581 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd52c10c1a0 0x7fd52c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:45.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:45.704+0000 7fd5322b1640 1 --2- 172.21.15.1:0/2312202581 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd52c103720 0x7fd52c10bc10 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:45.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:45.704+0000 7fd5322b1640 1 --2- 172.21.15.1:0/2312202581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd52c102e00 0x7fd52c1031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:45.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:45.704+0000 7fd5322b1640 1 -- 172.21.15.1:0/2312202581 >> 172.21.15.1:0/2312202581 conn(0x7fd52c0fc7a0 msgr2=0x7fd52c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:55:45.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:45.704+0000 7fd5322b1640 1 -- 172.21.15.1:0/2312202581 shutdown_connections 2024-09-11T21:55:45.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:45.704+0000 7fd5322b1640 1 -- 172.21.15.1:0/2312202581 wait complete. 2024-09-11T21:55:45.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:45.704+0000 7fd5322b1640 1 Processor -- start 2024-09-11T21:55:45.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:45.704+0000 7fd5322b1640 1 -- start start 2024-09-11T21:55:45.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:45.705+0000 7fd5322b1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd52c102e00 0x7fd52c198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:45.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:45.705+0000 7fd5322b1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd52c103720 0x7fd52c198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:45.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:45.705+0000 7fd5322b1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd52c10c1a0 0x7fd52c19cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:45.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:45.705+0000 7fd5322b1640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd52c112030 con 0x7fd52c102e00 2024-09-11T21:55:45.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:45.705+0000 7fd5322b1640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd52c111eb0 con 0x7fd52c10c1a0 2024-09-11T21:55:45.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:45.705+0000 7fd5322b1640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd52c1121b0 con 0x7fd52c103720 2024-09-11T21:55:45.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:45.705+0000 7fd52bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd52c102e00 0x7fd52c198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:45.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:45.705+0000 7fd52bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd52c102e00 0x7fd52c198610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44956/0 (socket says 172.21.15.1:44956) 2024-09-11T21:55:45.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:45.705+0000 7fd52bfff640 1 -- 172.21.15.1:0/1375762198 learned_addr learned my addr 172.21.15.1:0/1375762198 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:55:45.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:45.705+0000 7fd53084e640 1 --2- 172.21.15.1:0/1375762198 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd52c10c1a0 0x7fd52c19cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:45.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:45.705+0000 7fd523fff640 1 --2- 172.21.15.1:0/1375762198 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd52c103720 0x7fd52c198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:45.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:45.705+0000 7fd52bfff640 1 -- 172.21.15.1:0/1375762198 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd52c103720 msgr2=0x7fd52c198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:45.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:45.705+0000 7fd52bfff640 1 --2- 172.21.15.1:0/1375762198 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd52c103720 0x7fd52c198b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:45.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:45.705+0000 7fd52bfff640 1 -- 172.21.15.1:0/1375762198 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd52c10c1a0 msgr2=0x7fd52c19cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:45.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:45.705+0000 7fd52bfff640 1 --2- 172.21.15.1:0/1375762198 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd52c10c1a0 0x7fd52c19cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:45.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:45.705+0000 7fd52bfff640 1 -- 172.21.15.1:0/1375762198 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd52c19d5c0 con 0x7fd52c102e00 2024-09-11T21:55:45.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:45.705+0000 7fd53084e640 1 --2- 172.21.15.1:0/1375762198 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd52c10c1a0 0x7fd52c19cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:55:45.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:45.705+0000 7fd52bfff640 1 --2- 172.21.15.1:0/1375762198 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd52c102e00 0x7fd52c198610 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd51c00b4a0 tx=0x7fd51c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:55:45.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:45.705+0000 7fd529ffb640 1 -- 172.21.15.1:0/1375762198 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd51c014070 con 0x7fd52c102e00 2024-09-11T21:55:45.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:45.705+0000 7fd529ffb640 1 -- 172.21.15.1:0/1375762198 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd51c010af0 con 0x7fd52c102e00 2024-09-11T21:55:45.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:45.706+0000 7fd5322b1640 1 -- 172.21.15.1:0/1375762198 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd52c19d8b0 con 0x7fd52c102e00 2024-09-11T21:55:45.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:45.706+0000 7fd529ffb640 1 -- 172.21.15.1:0/1375762198 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd51c017d10 con 0x7fd52c102e00 2024-09-11T21:55:45.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:45.706+0000 7fd5322b1640 1 -- 172.21.15.1:0/1375762198 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd52c1a5190 con 0x7fd52c102e00 2024-09-11T21:55:45.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:45.707+0000 7fd5322b1640 1 -- 172.21.15.1:0/1375762198 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd52c103260 con 0x7fd52c102e00 2024-09-11T21:55:45.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:45.708+0000 7fd529ffb640 1 -- 172.21.15.1:0/1375762198 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fd51c017070 con 0x7fd52c102e00 2024-09-11T21:55:45.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:45.708+0000 7fd529ffb640 1 --2- 172.21.15.1:0/1375762198 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fd5180a2d10 0x7fd5180a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:45.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:45.709+0000 7fd523fff640 1 --2- 172.21.15.1:0/1375762198 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fd5180a2d10 0x7fd5180a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:45.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:45.709+0000 7fd529ffb640 1 -- 172.21.15.1:0/1375762198 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7fd51c0ce3b0 con 0x7fd52c102e00 2024-09-11T21:55:45.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:45.709+0000 7fd523fff640 1 --2- 172.21.15.1:0/1375762198 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fd5180a2d10 0x7fd5180a51d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd5140096c0 tx=0x7fd514000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:55:45.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:45.713+0000 7fd529ffb640 1 -- 172.21.15.1:0/1375762198 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd51c09b370 con 0x7fd52c102e00 2024-09-11T21:55:45.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:45.885+0000 7fd5322b1640 1 -- 172.21.15.1:0/1375762198 --> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd52c107290 con 0x7fd5180a2d10 2024-09-11T21:55:45.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:45.892+0000 7fd529ffb640 1 -- 172.21.15.1:0/1375762198 <== mgr.14150 v2:172.21.15.1:6800/3890350886 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258917 (secure 0 0 0) 0x7fd5140315b0 con 0x7fd5180a2d10 2024-09-11T21:55:45.895 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:55:45.897 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-11T21:55:45.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:45.895+0000 7fd5322b1640 1 -- 172.21.15.1:0/1375762198 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fd5180a2d10 msgr2=0x7fd5180a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:45.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:45.895+0000 7fd5322b1640 1 --2- 172.21.15.1:0/1375762198 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fd5180a2d10 0x7fd5180a51d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd5140096c0 tx=0x7fd514000bc0 comp rx=0 tx=0).stop 2024-09-11T21:55:45.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:45.895+0000 7fd5322b1640 1 -- 172.21.15.1:0/1375762198 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd52c102e00 msgr2=0x7fd52c198610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:45.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:45.895+0000 7fd5322b1640 1 --2- 172.21.15.1:0/1375762198 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd52c102e00 0x7fd52c198610 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd51c00b4a0 tx=0x7fd51c00b960 comp rx=0 tx=0).stop 2024-09-11T21:55:45.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:45.895+0000 7fd5322b1640 1 -- 172.21.15.1:0/1375762198 shutdown_connections 2024-09-11T21:55:45.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:45.895+0000 7fd5322b1640 1 --2- 172.21.15.1:0/1375762198 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fd5180a2d10 0x7fd5180a51d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:45.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:45.895+0000 7fd5322b1640 1 --2- 172.21.15.1:0/1375762198 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd52c10c1a0 0x7fd52c19cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:45.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:45.895+0000 7fd5322b1640 1 --2- 172.21.15.1:0/1375762198 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd52c103720 0x7fd52c198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:45.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:45.895+0000 7fd5322b1640 1 --2- 172.21.15.1:0/1375762198 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd52c102e00 0x7fd52c198610 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:45.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:45.895+0000 7fd5322b1640 1 -- 172.21.15.1:0/1375762198 >> 172.21.15.1:0/1375762198 conn(0x7fd52c0fc7a0 msgr2=0x7fd52c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:55:45.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:45.895+0000 7fd5322b1640 1 -- 172.21.15.1:0/1375762198 shutdown_connections 2024-09-11T21:55:45.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:45.895+0000 7fd5322b1640 1 -- 172.21.15.1:0/1375762198 wait complete. 2024-09-11T21:55:46.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:46 smithi001 ceph-mon[30935]: pgmap v294: 97 pgs: 97 active+clean; 630 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.1 KiB/s wr, 20 op/s 2024-09-11T21:55:46.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:46 smithi053 ceph-mon[34908]: pgmap v294: 97 pgs: 97 active+clean; 630 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.1 KiB/s wr, 20 op/s 2024-09-11T21:55:46.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:46 smithi161 ceph-mon[35615]: pgmap v294: 97 pgs: 97 active+clean; 630 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.1 KiB/s wr, 20 op/s 2024-09-11T21:55:46.721 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2024-09-11T21:55:44.612343+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":645005,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":287,"num_read_kb":261,"num_write":416,"num_write_kb":3058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":340,"ondisk_log_size":340,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":385444,"kb_used_data":11236,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124556380,"statfs":{"total":1151940427776,"available":1151545733120,"internally_reserved":0,"allocated":11505664,"data_stored":7143830,"data_compressed":107988,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19142,"internal_metadata":382973242},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":48223,"num_objects":36,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":143,"num_read_kb":135,"num_write":103,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003057"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.103272+0000","last_change":"2024-09-11T21:55:20.095385+0000","last_active":"2024-09-11T21:55:21.103272+0000","last_peered":"2024-09-11T21:55:21.103272+0000","last_clean":"2024-09-11T21:55:21.103272+0000","last_became_active":"2024-09-11T21:55:20.095077+0000","last_became_peered":"2024-09-11T21:55:20.095077+0000","last_unstale":"2024-09-11T21:55:21.103272+0000","last_undegraded":"2024-09-11T21:55:21.103272+0000","last_fullsized":"2024-09-11T21:55:21.103272+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:53:52.231522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.099859+0000","last_change":"2024-09-11T21:55:20.103540+0000","last_active":"2024-09-11T21:55:21.099859+0000","last_peered":"2024-09-11T21:55:21.099859+0000","last_clean":"2024-09-11T21:55:21.099859+0000","last_became_active":"2024-09-11T21:55:20.103233+0000","last_became_peered":"2024-09-11T21:55:20.103233+0000","last_unstale":"2024-09-11T21:55:21.099859+0000","last_undegraded":"2024-09-11T21:55:21.099859+0000","last_fullsized":"2024-09-11T21:55:21.099859+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:51:29.222202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.100567+0000","last_change":"2024-09-11T21:55:20.098538+0000","last_active":"2024-09-11T21:55:21.100567+0000","last_peered":"2024-09-11T21:55:21.100567+0000","last_clean":"2024-09-11T21:55:21.100567+0000","last_became_active":"2024-09-11T21:55:20.098202+0000","last_became_peered":"2024-09-11T21:55:20.098202+0000","last_unstale":"2024-09-11T21:55:21.100567+0000","last_undegraded":"2024-09-11T21:55:21.100567+0000","last_fullsized":"2024-09-11T21:55:21.100567+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:33:46.636952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.097725+0000","last_change":"2024-09-11T21:55:20.100869+0000","last_active":"2024-09-11T21:55:21.097725+0000","last_peered":"2024-09-11T21:55:21.097725+0000","last_clean":"2024-09-11T21:55:21.097725+0000","last_became_active":"2024-09-11T21:55:20.100573+0000","last_became_peered":"2024-09-11T21:55:20.100573+0000","last_unstale":"2024-09-11T21:55:21.097725+0000","last_undegraded":"2024-09-11T21:55:21.097725+0000","last_fullsized":"2024-09-11T21:55:21.097725+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:58:11.154105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.098166+0000","last_change":"2024-09-11T21:55:20.091648+0000","last_active":"2024-09-11T21:55:21.098166+0000","last_peered":"2024-09-11T21:55:21.098166+0000","last_clean":"2024-09-11T21:55:21.098166+0000","last_became_active":"2024-09-11T21:55:20.091222+0000","last_became_peered":"2024-09-11T21:55:20.091222+0000","last_unstale":"2024-09-11T21:55:21.098166+0000","last_undegraded":"2024-09-11T21:55:21.098166+0000","last_fullsized":"2024-09-11T21:55:21.098166+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:55:03.420754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.086150+0000","last_change":"2024-09-11T21:55:20.100446+0000","last_active":"2024-09-11T21:55:21.086150+0000","last_peered":"2024-09-11T21:55:21.086150+0000","last_clean":"2024-09-11T21:55:21.086150+0000","last_became_active":"2024-09-11T21:55:20.100062+0000","last_became_peered":"2024-09-11T21:55:20.100062+0000","last_unstale":"2024-09-11T21:55:21.086150+0000","last_undegraded":"2024-09-11T21:55:21.086150+0000","last_fullsized":"2024-09-11T21:55:21.086150+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:49:37.109258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.097709+0000","last_change":"2024-09-11T21:55:20.101084+0000","last_active":"2024-09-11T21:55:21.097709+0000","last_peered":"2024-09-11T21:55:21.097709+0000","last_clean":"2024-09-11T21:55:21.097709+0000","last_became_active":"2024-09-11T21:55:20.100739+0000","last_became_peered":"2024-09-11T21:55:20.100739+0000","last_unstale":"2024-09-11T21:55:21.097709+0000","last_undegraded":"2024-09-11T21:55:21.097709+0000","last_fullsized":"2024-09-11T21:55:21.097709+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:46:58.969559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.098694+0000","last_change":"2024-09-11T21:55:20.088715+0000","last_active":"2024-09-11T21:55:21.098694+0000","last_peered":"2024-09-11T21:55:21.098694+0000","last_clean":"2024-09-11T21:55:21.098694+0000","last_became_active":"2024-09-11T21:55:20.088280+0000","last_became_peered":"2024-09-11T21:55:20.088280+0000","last_unstale":"2024-09-11T21:55:21.098694+0000","last_undegraded":"2024-09-11T21:55:21.098694+0000","last_fullsized":"2024-09-11T21:55:21.098694+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:06:24.450444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.101362+0000","last_change":"2024-09-11T21:55:20.101266+0000","last_active":"2024-09-11T21:55:21.101362+0000","last_peered":"2024-09-11T21:55:21.101362+0000","last_clean":"2024-09-11T21:55:21.101362+0000","last_became_active":"2024-09-11T21:55:20.100919+0000","last_became_peered":"2024-09-11T21:55:20.100919+0000","last_unstale":"2024-09-11T21:55:21.101362+0000","last_undegraded":"2024-09-11T21:55:21.101362+0000","last_fullsized":"2024-09-11T21:55:21.101362+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:47:15.466082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.086697+0000","last_change":"2024-09-11T21:55:20.098086+0000","last_active":"2024-09-11T21:55:21.086697+0000","last_peered":"2024-09-11T21:55:21.086697+0000","last_clean":"2024-09-11T21:55:21.086697+0000","last_became_active":"2024-09-11T21:55:20.097754+0000","last_became_peered":"2024-09-11T21:55:20.097754+0000","last_unstale":"2024-09-11T21:55:21.086697+0000","last_undegraded":"2024-09-11T21:55:21.086697+0000","last_fullsized":"2024-09-11T21:55:21.086697+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:37:00.098345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.099716+0000","last_change":"2024-09-11T21:55:20.092671+0000","last_active":"2024-09-11T21:55:21.099716+0000","last_peered":"2024-09-11T21:55:21.099716+0000","last_clean":"2024-09-11T21:55:21.099716+0000","last_became_active":"2024-09-11T21:55:20.092112+0000","last_became_peered":"2024-09-11T21:55:20.092112+0000","last_unstale":"2024-09-11T21:55:21.099716+0000","last_undegraded":"2024-09-11T21:55:21.099716+0000","last_fullsized":"2024-09-11T21:55:21.099716+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:06:28.538838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.097879+0000","last_change":"2024-09-11T21:55:20.099863+0000","last_active":"2024-09-11T21:55:21.097879+0000","last_peered":"2024-09-11T21:55:21.097879+0000","last_clean":"2024-09-11T21:55:21.097879+0000","last_became_active":"2024-09-11T21:55:20.099173+0000","last_became_peered":"2024-09-11T21:55:20.099173+0000","last_unstale":"2024-09-11T21:55:21.097879+0000","last_undegraded":"2024-09-11T21:55:21.097879+0000","last_fullsized":"2024-09-11T21:55:21.097879+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:34:56.669410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.085802+0000","last_change":"2024-09-11T21:55:20.101212+0000","last_active":"2024-09-11T21:55:21.085802+0000","last_peered":"2024-09-11T21:55:21.085802+0000","last_clean":"2024-09-11T21:55:21.085802+0000","last_became_active":"2024-09-11T21:55:20.100876+0000","last_became_peered":"2024-09-11T21:55:20.100876+0000","last_unstale":"2024-09-11T21:55:21.085802+0000","last_undegraded":"2024-09-11T21:55:21.085802+0000","last_fullsized":"2024-09-11T21:55:21.085802+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:14:18.104009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.100238+0000","last_change":"2024-09-11T21:55:20.095394+0000","last_active":"2024-09-11T21:55:21.100238+0000","last_peered":"2024-09-11T21:55:21.100238+0000","last_clean":"2024-09-11T21:55:21.100238+0000","last_became_active":"2024-09-11T21:55:20.095101+0000","last_became_peered":"2024-09-11T21:55:20.095101+0000","last_unstale":"2024-09-11T21:55:21.100238+0000","last_undegraded":"2024-09-11T21:55:21.100238+0000","last_fullsized":"2024-09-11T21:55:21.100238+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:53:32.529359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.100035+0000","last_change":"2024-09-11T21:55:20.099668+0000","last_active":"2024-09-11T21:55:21.100035+0000","last_peered":"2024-09-11T21:55:21.100035+0000","last_clean":"2024-09-11T21:55:21.100035+0000","last_became_active":"2024-09-11T21:55:20.099392+0000","last_became_peered":"2024-09-11T21:55:20.099392+0000","last_unstale":"2024-09-11T21:55:21.100035+0000","last_undegraded":"2024-09-11T21:55:21.100035+0000","last_fullsized":"2024-09-11T21:55:21.100035+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:50:00.479508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.099808+0000","last_change":"2024-09-11T21:55:20.093019+0000","last_active":"2024-09-11T21:55:21.099808+0000","last_peered":"2024-09-11T21:55:21.099808+0000","last_clean":"2024-09-11T21:55:21.099808+0000","last_became_active":"2024-09-11T21:55:20.092691+0000","last_became_peered":"2024-09-11T21:55:20.092691+0000","last_unstale":"2024-09-11T21:55:21.099808+0000","last_undegraded":"2024-09-11T21:55:21.099808+0000","last_fullsized":"2024-09-11T21:55:21.099808+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:04:36.684992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.103502+0000","last_change":"2024-09-11T21:55:20.097333+0000","last_active":"2024-09-11T21:55:21.103502+0000","last_peered":"2024-09-11T21:55:21.103502+0000","last_clean":"2024-09-11T21:55:21.103502+0000","last_became_active":"2024-09-11T21:55:20.096752+0000","last_became_peered":"2024-09-11T21:55:20.096752+0000","last_unstale":"2024-09-11T21:55:21.103502+0000","last_undegraded":"2024-09-11T21:55:21.103502+0000","last_fullsized":"2024-09-11T21:55:21.103502+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:27:20.722174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.097172+0000","last_change":"2024-09-11T21:55:20.093972+0000","last_active":"2024-09-11T21:55:21.097172+0000","last_peered":"2024-09-11T21:55:21.097172+0000","last_clean":"2024-09-11T21:55:21.097172+0000","last_became_active":"2024-09-11T21:55:20.093633+0000","last_became_peered":"2024-09-11T21:55:20.093633+0000","last_unstale":"2024-09-11T21:55:21.097172+0000","last_undegraded":"2024-09-11T21:55:21.097172+0000","last_fullsized":"2024-09-11T21:55:21.097172+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:25:18.973332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.097726+0000","last_change":"2024-09-11T21:55:20.094239+0000","last_active":"2024-09-11T21:55:21.097726+0000","last_peered":"2024-09-11T21:55:21.097726+0000","last_clean":"2024-09-11T21:55:21.097726+0000","last_became_active":"2024-09-11T21:55:20.093924+0000","last_became_peered":"2024-09-11T21:55:20.093924+0000","last_unstale":"2024-09-11T21:55:21.097726+0000","last_undegraded":"2024-09-11T21:55:21.097726+0000","last_fullsized":"2024-09-11T21:55:21.097726+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:39:53.012780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.102943+0000","last_change":"2024-09-11T21:55:20.099335+0000","last_active":"2024-09-11T21:55:21.102943+0000","last_peered":"2024-09-11T21:55:21.102943+0000","last_clean":"2024-09-11T21:55:21.102943+0000","last_became_active":"2024-09-11T21:55:20.098929+0000","last_became_peered":"2024-09-11T21:55:20.098929+0000","last_unstale":"2024-09-11T21:55:21.102943+0000","last_undegraded":"2024-09-11T21:55:21.102943+0000","last_fullsized":"2024-09-11T21:55:21.102943+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:38:43.117829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.085822+0000","last_change":"2024-09-11T21:55:20.094068+0000","last_active":"2024-09-11T21:55:21.085822+0000","last_peered":"2024-09-11T21:55:21.085822+0000","last_clean":"2024-09-11T21:55:21.085822+0000","last_became_active":"2024-09-11T21:55:20.093637+0000","last_became_peered":"2024-09-11T21:55:20.093637+0000","last_unstale":"2024-09-11T21:55:21.085822+0000","last_undegraded":"2024-09-11T21:55:21.085822+0000","last_fullsized":"2024-09-11T21:55:21.085822+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:52:59.289715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.100443+0000","last_change":"2024-09-11T21:55:20.090778+0000","last_active":"2024-09-11T21:55:21.100443+0000","last_peered":"2024-09-11T21:55:21.100443+0000","last_clean":"2024-09-11T21:55:21.100443+0000","last_became_active":"2024-09-11T21:55:20.090443+0000","last_became_peered":"2024-09-11T21:55:20.090443+0000","last_unstale":"2024-09-11T21:55:21.100443+0000","last_undegraded":"2024-09-11T21:55:21.100443+0000","last_fullsized":"2024-09-11T21:55:21.100443+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:46:21.488228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.097160+0000","last_change":"2024-09-11T21:55:20.092238+0000","last_active":"2024-09-11T21:55:21.097160+0000","last_peered":"2024-09-11T21:55:21.097160+0000","last_clean":"2024-09-11T21:55:21.097160+0000","last_became_active":"2024-09-11T21:55:20.091961+0000","last_became_peered":"2024-09-11T21:55:20.091961+0000","last_unstale":"2024-09-11T21:55:21.097160+0000","last_undegraded":"2024-09-11T21:55:21.097160+0000","last_fullsized":"2024-09-11T21:55:21.097160+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:54:40.445441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.086361+0000","last_change":"2024-09-11T21:55:20.102944+0000","last_active":"2024-09-11T21:55:21.086361+0000","last_peered":"2024-09-11T21:55:21.086361+0000","last_clean":"2024-09-11T21:55:21.086361+0000","last_became_active":"2024-09-11T21:55:20.102533+0000","last_became_peered":"2024-09-11T21:55:20.102533+0000","last_unstale":"2024-09-11T21:55:21.086361+0000","last_undegraded":"2024-09-11T21:55:21.086361+0000","last_fullsized":"2024-09-11T21:55:21.086361+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:12:15.832622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.098471+0000","last_change":"2024-09-11T21:55:20.092177+0000","last_active":"2024-09-11T21:55:21.098471+0000","last_peered":"2024-09-11T21:55:21.098471+0000","last_clean":"2024-09-11T21:55:21.098471+0000","last_became_active":"2024-09-11T21:55:20.091897+0000","last_became_peered":"2024-09-11T21:55:20.091897+0000","last_unstale":"2024-09-11T21:55:21.098471+0000","last_undegraded":"2024-09-11T21:55:21.098471+0000","last_fullsized":"2024-09-11T21:55:21.098471+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:38:57.371153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.099223+0000","last_change":"2024-09-11T21:55:20.092166+0000","last_active":"2024-09-11T21:55:21.099223+0000","last_peered":"2024-09-11T21:55:21.099223+0000","last_clean":"2024-09-11T21:55:21.099223+0000","last_became_active":"2024-09-11T21:55:20.091854+0000","last_became_peered":"2024-09-11T21:55:20.091854+0000","last_unstale":"2024-09-11T21:55:21.099223+0000","last_undegraded":"2024-09-11T21:55:21.099223+0000","last_fullsized":"2024-09-11T21:55:21.099223+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:53:32.256008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.103425+0000","last_change":"2024-09-11T21:55:20.103976+0000","last_active":"2024-09-11T21:55:21.103425+0000","last_peered":"2024-09-11T21:55:21.103425+0000","last_clean":"2024-09-11T21:55:21.103425+0000","last_became_active":"2024-09-11T21:55:20.103668+0000","last_became_peered":"2024-09-11T21:55:20.103668+0000","last_unstale":"2024-09-11T21:55:21.103425+0000","last_undegraded":"2024-09-11T21:55:21.103425+0000","last_fullsized":"2024-09-11T21:55:21.103425+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:13:42.663641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.085778+0000","last_change":"2024-09-11T21:55:20.093898+0000","last_active":"2024-09-11T21:55:21.085778+0000","last_peered":"2024-09-11T21:55:21.085778+0000","last_clean":"2024-09-11T21:55:21.085778+0000","last_became_active":"2024-09-11T21:55:20.093348+0000","last_became_peered":"2024-09-11T21:55:20.093348+0000","last_unstale":"2024-09-11T21:55:21.085778+0000","last_undegraded":"2024-09-11T21:55:21.085778+0000","last_fullsized":"2024-09-11T21:55:21.085778+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:00:06.185478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.100902+0000","last_change":"2024-09-11T21:55:20.100575+0000","last_active":"2024-09-11T21:55:21.100902+0000","last_peered":"2024-09-11T21:55:21.100902+0000","last_clean":"2024-09-11T21:55:21.100902+0000","last_became_active":"2024-09-11T21:55:20.100255+0000","last_became_peered":"2024-09-11T21:55:20.100255+0000","last_unstale":"2024-09-11T21:55:21.100902+0000","last_undegraded":"2024-09-11T21:55:21.100902+0000","last_fullsized":"2024-09-11T21:55:21.100902+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:48:57.718298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.097296+0000","last_change":"2024-09-11T21:55:20.097548+0000","last_active":"2024-09-11T21:55:21.097296+0000","last_peered":"2024-09-11T21:55:21.097296+0000","last_clean":"2024-09-11T21:55:21.097296+0000","last_became_active":"2024-09-11T21:55:20.097124+0000","last_became_peered":"2024-09-11T21:55:20.097124+0000","last_unstale":"2024-09-11T21:55:21.097296+0000","last_undegraded":"2024-09-11T21:55:21.097296+0000","last_fullsized":"2024-09-11T21:55:21.097296+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:22:23.904457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.097695+0000","last_change":"2024-09-11T21:55:20.093822+0000","last_active":"2024-09-11T21:55:21.097695+0000","last_peered":"2024-09-11T21:55:21.097695+0000","last_clean":"2024-09-11T21:55:21.097695+0000","last_became_active":"2024-09-11T21:55:20.093361+0000","last_became_peered":"2024-09-11T21:55:20.093361+0000","last_unstale":"2024-09-11T21:55:21.097695+0000","last_undegraded":"2024-09-11T21:55:21.097695+0000","last_fullsized":"2024-09-11T21:55:21.097695+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:37:06.362732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.103360+0000","last_change":"2024-09-11T21:55:20.103990+0000","last_active":"2024-09-11T21:55:21.103360+0000","last_peered":"2024-09-11T21:55:21.103360+0000","last_clean":"2024-09-11T21:55:21.103360+0000","last_became_active":"2024-09-11T21:55:20.103375+0000","last_became_peered":"2024-09-11T21:55:20.103375+0000","last_unstale":"2024-09-11T21:55:21.103360+0000","last_undegraded":"2024-09-11T21:55:21.103360+0000","last_fullsized":"2024-09-11T21:55:21.103360+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:02:06.786888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.099679+0000","last_change":"2024-09-11T21:55:20.101535+0000","last_active":"2024-09-11T21:55:21.099679+0000","last_peered":"2024-09-11T21:55:21.099679+0000","last_clean":"2024-09-11T21:55:21.099679+0000","last_became_active":"2024-09-11T21:55:20.101129+0000","last_became_peered":"2024-09-11T21:55:20.101129+0000","last_unstale":"2024-09-11T21:55:21.099679+0000","last_undegraded":"2024-09-11T21:55:21.099679+0000","last_fullsized":"2024-09-11T21:55:21.099679+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:09:15.673670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"84'6","reported_seq":38,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:41.177259+0000","last_change":"2024-09-11T21:55:18.026947+0000","last_active":"2024-09-11T21:55:41.177259+0000","last_peered":"2024-09-11T21:55:41.177259+0000","last_clean":"2024-09-11T21:55:41.177259+0000","last_became_active":"2024-09-11T21:55:18.026632+0000","last_became_peered":"2024-09-11T21:55:18.026632+0000","last_unstale":"2024-09-11T21:55:41.177259+0000","last_undegraded":"2024-09-11T21:55:41.177259+0000","last_fullsized":"2024-09-11T21:55:41.177259+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:55:16.618074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.086576+0000","last_change":"2024-09-11T21:55:20.100197+0000","last_active":"2024-09-11T21:55:21.086576+0000","last_peered":"2024-09-11T21:55:21.086576+0000","last_clean":"2024-09-11T21:55:21.086576+0000","last_became_active":"2024-09-11T21:55:20.098494+0000","last_became_peered":"2024-09-11T21:55:20.098494+0000","last_unstale":"2024-09-11T21:55:21.086576+0000","last_undegraded":"2024-09-11T21:55:21.086576+0000","last_fullsized":"2024-09-11T21:55:21.086576+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:14.505314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"84'7","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:30.150870+0000","last_change":"2024-09-11T21:55:18.024359+0000","last_active":"2024-09-11T21:55:30.150870+0000","last_peered":"2024-09-11T21:55:30.150870+0000","last_clean":"2024-09-11T21:55:30.150870+0000","last_became_active":"2024-09-11T21:55:18.023624+0000","last_became_peered":"2024-09-11T21:55:18.023624+0000","last_unstale":"2024-09-11T21:55:30.150870+0000","last_undegraded":"2024-09-11T21:55:30.150870+0000","last_fullsized":"2024-09-11T21:55:30.150870+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:13:38.582232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"84'3","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:29.138838+0000","last_change":"2024-09-11T21:55:18.032541+0000","last_active":"2024-09-11T21:55:29.138838+0000","last_peered":"2024-09-11T21:55:29.138838+0000","last_clean":"2024-09-11T21:55:29.138838+0000","last_became_active":"2024-09-11T21:55:18.032218+0000","last_became_peered":"2024-09-11T21:55:18.032218+0000","last_unstale":"2024-09-11T21:55:29.138838+0000","last_undegraded":"2024-09-11T21:55:29.138838+0000","last_fullsized":"2024-09-11T21:55:29.138838+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:38:20.872069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.098851+0000","last_change":"2024-09-11T21:55:20.090030+0000","last_active":"2024-09-11T21:55:21.098851+0000","last_peered":"2024-09-11T21:55:21.098851+0000","last_clean":"2024-09-11T21:55:21.098851+0000","last_became_active":"2024-09-11T21:55:20.089481+0000","last_became_peered":"2024-09-11T21:55:20.089481+0000","last_unstale":"2024-09-11T21:55:21.098851+0000","last_undegraded":"2024-09-11T21:55:21.098851+0000","last_fullsized":"2024-09-11T21:55:21.098851+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:41:57.038519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"84'3","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:31.157367+0000","last_change":"2024-09-11T21:55:18.026656+0000","last_active":"2024-09-11T21:55:31.157367+0000","last_peered":"2024-09-11T21:55:31.157367+0000","last_clean":"2024-09-11T21:55:31.157367+0000","last_became_active":"2024-09-11T21:55:18.026318+0000","last_became_peered":"2024-09-11T21:55:18.026318+0000","last_unstale":"2024-09-11T21:55:31.157367+0000","last_undegraded":"2024-09-11T21:55:31.157367+0000","last_fullsized":"2024-09-11T21:55:31.157367+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:14:41.438334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.096818+0000","last_change":"2024-09-11T21:55:20.090790+0000","last_active":"2024-09-11T21:55:21.096818+0000","last_peered":"2024-09-11T21:55:21.096818+0000","last_clean":"2024-09-11T21:55:21.096818+0000","last_became_active":"2024-09-11T21:55:20.090468+0000","last_became_peered":"2024-09-11T21:55:20.090468+0000","last_unstale":"2024-09-11T21:55:21.096818+0000","last_undegraded":"2024-09-11T21:55:21.096818+0000","last_fullsized":"2024-09-11T21:55:21.096818+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:19.115482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"84'4","reported_seq":48,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:44.179611+0000","last_change":"2024-09-11T21:55:18.026989+0000","last_active":"2024-09-11T21:55:44.179611+0000","last_peered":"2024-09-11T21:55:44.179611+0000","last_clean":"2024-09-11T21:55:44.179611+0000","last_became_active":"2024-09-11T21:55:18.026229+0000","last_became_peered":"2024-09-11T21:55:18.026229+0000","last_unstale":"2024-09-11T21:55:44.179611+0000","last_undegraded":"2024-09-11T21:55:44.179611+0000","last_fullsized":"2024-09-11T21:55:44.179611+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:50:35.745789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.101844+0000","last_change":"2024-09-11T21:55:20.095884+0000","last_active":"2024-09-11T21:55:21.101844+0000","last_peered":"2024-09-11T21:55:21.101844+0000","last_clean":"2024-09-11T21:55:21.101844+0000","last_became_active":"2024-09-11T21:55:20.094887+0000","last_became_peered":"2024-09-11T21:55:20.094887+0000","last_unstale":"2024-09-11T21:55:21.101844+0000","last_undegraded":"2024-09-11T21:55:21.101844+0000","last_fullsized":"2024-09-11T21:55:21.101844+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:45:03.579703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"84'4","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:31.158136+0000","last_change":"2024-09-11T21:55:18.030474+0000","last_active":"2024-09-11T21:55:31.158136+0000","last_peered":"2024-09-11T21:55:31.158136+0000","last_clean":"2024-09-11T21:55:31.158136+0000","last_became_active":"2024-09-11T21:55:18.030104+0000","last_became_peered":"2024-09-11T21:55:18.030104+0000","last_unstale":"2024-09-11T21:55:31.158136+0000","last_undegraded":"2024-09-11T21:55:31.158136+0000","last_fullsized":"2024-09-11T21:55:31.158136+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:21:00.484595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.100730+0000","last_change":"2024-09-11T21:55:20.102807+0000","last_active":"2024-09-11T21:55:21.100730+0000","last_peered":"2024-09-11T21:55:21.100730+0000","last_clean":"2024-09-11T21:55:21.100730+0000","last_became_active":"2024-09-11T21:55:20.102581+0000","last_became_peered":"2024-09-11T21:55:20.102581+0000","last_unstale":"2024-09-11T21:55:21.100730+0000","last_undegraded":"2024-09-11T21:55:21.100730+0000","last_fullsized":"2024-09-11T21:55:21.100730+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:29:23.139788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"84'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:30.150754+0000","last_change":"2024-09-11T21:55:18.117685+0000","last_active":"2024-09-11T21:55:30.150754+0000","last_peered":"2024-09-11T21:55:30.150754+0000","last_clean":"2024-09-11T21:55:30.150754+0000","last_became_active":"2024-09-11T21:55:18.117261+0000","last_became_peered":"2024-09-11T21:55:18.117261+0000","last_unstale":"2024-09-11T21:55:30.150754+0000","last_undegraded":"2024-09-11T21:55:30.150754+0000","last_fullsized":"2024-09-11T21:55:30.150754+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:42:13.180725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.100810+0000","last_change":"2024-09-11T21:55:20.086562+0000","last_active":"2024-09-11T21:55:21.100810+0000","last_peered":"2024-09-11T21:55:21.100810+0000","last_clean":"2024-09-11T21:55:21.100810+0000","last_became_active":"2024-09-11T21:55:20.086197+0000","last_became_peered":"2024-09-11T21:55:20.086197+0000","last_unstale":"2024-09-11T21:55:21.100810+0000","last_undegraded":"2024-09-11T21:55:21.100810+0000","last_fullsized":"2024-09-11T21:55:21.100810+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:23:44.705132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'24","reported_seq":74,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:44.181835+0000","last_change":"2024-09-11T21:55:18.028320+0000","last_active":"2024-09-11T21:55:44.181835+0000","last_peered":"2024-09-11T21:55:44.181835+0000","last_clean":"2024-09-11T21:55:44.181835+0000","last_became_active":"2024-09-11T21:55:18.027982+0000","last_became_peered":"2024-09-11T21:55:18.027982+0000","last_unstale":"2024-09-11T21:55:44.181835+0000","last_undegraded":"2024-09-11T21:55:44.181835+0000","last_fullsized":"2024-09-11T21:55:44.181835+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:08:36.750665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46802,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":61,"num_write":26,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.099277+0000","last_change":"2024-09-11T21:55:20.091695+0000","last_active":"2024-09-11T21:55:21.099277+0000","last_peered":"2024-09-11T21:55:21.099277+0000","last_clean":"2024-09-11T21:55:21.099277+0000","last_became_active":"2024-09-11T21:55:20.091128+0000","last_became_peered":"2024-09-11T21:55:20.091128+0000","last_unstale":"2024-09-11T21:55:21.099277+0000","last_undegraded":"2024-09-11T21:55:21.099277+0000","last_fullsized":"2024-09-11T21:55:21.099277+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:16:20.751952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"84'9","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:32.158923+0000","last_change":"2024-09-11T21:55:18.030827+0000","last_active":"2024-09-11T21:55:32.158923+0000","last_peered":"2024-09-11T21:55:32.158923+0000","last_clean":"2024-09-11T21:55:32.158923+0000","last_became_active":"2024-09-11T21:55:18.030354+0000","last_became_peered":"2024-09-11T21:55:18.030354+0000","last_unstale":"2024-09-11T21:55:32.158923+0000","last_undegraded":"2024-09-11T21:55:32.158923+0000","last_fullsized":"2024-09-11T21:55:32.158923+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:23:19.578820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.085795+0000","last_change":"2024-09-11T21:55:20.090160+0000","last_active":"2024-09-11T21:55:21.085795+0000","last_peered":"2024-09-11T21:55:21.085795+0000","last_clean":"2024-09-11T21:55:21.085795+0000","last_became_active":"2024-09-11T21:55:20.089801+0000","last_became_peered":"2024-09-11T21:55:20.089801+0000","last_unstale":"2024-09-11T21:55:21.085795+0000","last_undegraded":"2024-09-11T21:55:21.085795+0000","last_fullsized":"2024-09-11T21:55:21.085795+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:34:16.093123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"84'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:31.157445+0000","last_change":"2024-09-11T21:55:18.026817+0000","last_active":"2024-09-11T21:55:31.157445+0000","last_peered":"2024-09-11T21:55:31.157445+0000","last_clean":"2024-09-11T21:55:31.157445+0000","last_became_active":"2024-09-11T21:55:18.026188+0000","last_became_peered":"2024-09-11T21:55:18.026188+0000","last_unstale":"2024-09-11T21:55:31.157445+0000","last_undegraded":"2024-09-11T21:55:31.157445+0000","last_fullsized":"2024-09-11T21:55:31.157445+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:55:14.774503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.098605+0000","last_change":"2024-09-11T21:55:20.095613+0000","last_active":"2024-09-11T21:55:21.098605+0000","last_peered":"2024-09-11T21:55:21.098605+0000","last_clean":"2024-09-11T21:55:21.098605+0000","last_became_active":"2024-09-11T21:55:20.095302+0000","last_became_peered":"2024-09-11T21:55:20.095302+0000","last_unstale":"2024-09-11T21:55:21.098605+0000","last_undegraded":"2024-09-11T21:55:21.098605+0000","last_fullsized":"2024-09-11T21:55:21.098605+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:37:27.961213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'5","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:39.174406+0000","last_change":"2024-09-11T21:55:18.032417+0000","last_active":"2024-09-11T21:55:39.174406+0000","last_peered":"2024-09-11T21:55:39.174406+0000","last_clean":"2024-09-11T21:55:39.174406+0000","last_became_active":"2024-09-11T21:55:18.031419+0000","last_became_peered":"2024-09-11T21:55:18.031419+0000","last_unstale":"2024-09-11T21:55:39.174406+0000","last_undegraded":"2024-09-11T21:55:39.174406+0000","last_fullsized":"2024-09-11T21:55:39.174406+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:30:52.285678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":12,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.099572+0000","last_change":"2024-09-11T21:55:20.101965+0000","last_active":"2024-09-11T21:55:21.099572+0000","last_peered":"2024-09-11T21:55:21.099572+0000","last_clean":"2024-09-11T21:55:21.099572+0000","last_became_active":"2024-09-11T21:55:20.101534+0000","last_became_peered":"2024-09-11T21:55:20.101534+0000","last_unstale":"2024-09-11T21:55:21.099572+0000","last_undegraded":"2024-09-11T21:55:21.099572+0000","last_fullsized":"2024-09-11T21:55:21.099572+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:41:30.685121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"84'17","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:31.158074+0000","last_change":"2024-09-11T21:55:18.116059+0000","last_active":"2024-09-11T21:55:31.158074+0000","last_peered":"2024-09-11T21:55:31.158074+0000","last_clean":"2024-09-11T21:55:31.158074+0000","last_became_active":"2024-09-11T21:55:18.115621+0000","last_became_peered":"2024-09-11T21:55:18.115621+0000","last_unstale":"2024-09-11T21:55:31.158074+0000","last_undegraded":"2024-09-11T21:55:31.158074+0000","last_fullsized":"2024-09-11T21:55:31.158074+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:47:17.815695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.086741+0000","last_change":"2024-09-11T21:55:20.100903+0000","last_active":"2024-09-11T21:55:21.086741+0000","last_peered":"2024-09-11T21:55:21.086741+0000","last_clean":"2024-09-11T21:55:21.086741+0000","last_became_active":"2024-09-11T21:55:20.100561+0000","last_became_peered":"2024-09-11T21:55:20.100561+0000","last_unstale":"2024-09-11T21:55:21.086741+0000","last_undegraded":"2024-09-11T21:55:21.086741+0000","last_fullsized":"2024-09-11T21:55:21.086741+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:00:45.472238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'2","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:31.134748+0000","last_change":"2024-09-11T21:55:18.029549+0000","last_active":"2024-09-11T21:55:31.134748+0000","last_peered":"2024-09-11T21:55:31.134748+0000","last_clean":"2024-09-11T21:55:31.134748+0000","last_became_active":"2024-09-11T21:55:18.029178+0000","last_became_peered":"2024-09-11T21:55:18.029178+0000","last_unstale":"2024-09-11T21:55:31.134748+0000","last_undegraded":"2024-09-11T21:55:31.134748+0000","last_fullsized":"2024-09-11T21:55:31.134748+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:37:07.915786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.103401+0000","last_change":"2024-09-11T21:55:20.104202+0000","last_active":"2024-09-11T21:55:21.103401+0000","last_peered":"2024-09-11T21:55:21.103401+0000","last_clean":"2024-09-11T21:55:21.103401+0000","last_became_active":"2024-09-11T21:55:20.103464+0000","last_became_peered":"2024-09-11T21:55:20.103464+0000","last_unstale":"2024-09-11T21:55:21.103401+0000","last_undegraded":"2024-09-11T21:55:21.103401+0000","last_fullsized":"2024-09-11T21:55:21.103401+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:30:16.044143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"84'5","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:31.157256+0000","last_change":"2024-09-11T21:55:18.026183+0000","last_active":"2024-09-11T21:55:31.157256+0000","last_peered":"2024-09-11T21:55:31.157256+0000","last_clean":"2024-09-11T21:55:31.157256+0000","last_became_active":"2024-09-11T21:55:18.025849+0000","last_became_peered":"2024-09-11T21:55:18.025849+0000","last_unstale":"2024-09-11T21:55:31.157256+0000","last_undegraded":"2024-09-11T21:55:31.157256+0000","last_fullsized":"2024-09-11T21:55:31.157256+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:21:12.189158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.098264+0000","last_change":"2024-09-11T21:55:20.090652+0000","last_active":"2024-09-11T21:55:21.098264+0000","last_peered":"2024-09-11T21:55:21.098264+0000","last_clean":"2024-09-11T21:55:21.098264+0000","last_became_active":"2024-09-11T21:55:20.090348+0000","last_became_peered":"2024-09-11T21:55:20.090348+0000","last_unstale":"2024-09-11T21:55:21.098264+0000","last_undegraded":"2024-09-11T21:55:21.098264+0000","last_fullsized":"2024-09-11T21:55:21.098264+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:10:58.733371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'3","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:31.157673+0000","last_change":"2024-09-11T21:55:18.024388+0000","last_active":"2024-09-11T21:55:31.157673+0000","last_peered":"2024-09-11T21:55:31.157673+0000","last_clean":"2024-09-11T21:55:31.157673+0000","last_became_active":"2024-09-11T21:55:18.024100+0000","last_became_peered":"2024-09-11T21:55:18.024100+0000","last_unstale":"2024-09-11T21:55:31.157673+0000","last_undegraded":"2024-09-11T21:55:31.157673+0000","last_fullsized":"2024-09-11T21:55:31.157673+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:21:11.152282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"84'1","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.186238+0000","last_change":"2024-09-11T21:55:20.101007+0000","last_active":"2024-09-11T21:55:21.186238+0000","last_peered":"2024-09-11T21:55:21.186238+0000","last_clean":"2024-09-11T21:55:21.186238+0000","last_became_active":"2024-09-11T21:55:20.100642+0000","last_became_peered":"2024-09-11T21:55:20.100642+0000","last_unstale":"2024-09-11T21:55:21.186238+0000","last_undegraded":"2024-09-11T21:55:21.186238+0000","last_fullsized":"2024-09-11T21:55:21.186238+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:16:05.642184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"84'6","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:38.173290+0000","last_change":"2024-09-11T21:55:18.116216+0000","last_active":"2024-09-11T21:55:38.173290+0000","last_peered":"2024-09-11T21:55:38.173290+0000","last_clean":"2024-09-11T21:55:38.173290+0000","last_became_active":"2024-09-11T21:55:18.115871+0000","last_became_peered":"2024-09-11T21:55:18.115871+0000","last_unstale":"2024-09-11T21:55:38.173290+0000","last_undegraded":"2024-09-11T21:55:38.173290+0000","last_fullsized":"2024-09-11T21:55:38.173290+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:16:56.027636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.099871+0000","last_change":"2024-09-11T21:55:20.092421+0000","last_active":"2024-09-11T21:55:21.099871+0000","last_peered":"2024-09-11T21:55:21.099871+0000","last_clean":"2024-09-11T21:55:21.099871+0000","last_became_active":"2024-09-11T21:55:20.091707+0000","last_became_peered":"2024-09-11T21:55:20.091707+0000","last_unstale":"2024-09-11T21:55:21.099871+0000","last_undegraded":"2024-09-11T21:55:21.099871+0000","last_fullsized":"2024-09-11T21:55:21.099871+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:12:38.157687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":346,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.099897+0000","last_change":"2024-09-11T21:54:07.542362+0000","last_active":"2024-09-11T21:55:21.099897+0000","last_peered":"2024-09-11T21:55:21.099897+0000","last_clean":"2024-09-11T21:55:21.099897+0000","last_became_active":"2024-09-11T21:54:07.541871+0000","last_became_peered":"2024-09-11T21:54:07.541871+0000","last_unstale":"2024-09-11T21:55:21.099897+0000","last_undegraded":"2024-09-11T21:55:21.099897+0000","last_fullsized":"2024-09-11T21:55:21.099897+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:50:41.026895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:50:41.026895+0000","last_clean_scrub_stamp":"2024-09-11T21:50:41.026895+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:05:06.154268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"84'6","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:32.158953+0000","last_change":"2024-09-11T21:55:18.030558+0000","last_active":"2024-09-11T21:55:32.158953+0000","last_peered":"2024-09-11T21:55:32.158953+0000","last_clean":"2024-09-11T21:55:32.158953+0000","last_became_active":"2024-09-11T21:55:18.030216+0000","last_became_peered":"2024-09-11T21:55:18.030216+0000","last_unstale":"2024-09-11T21:55:32.158953+0000","last_undegraded":"2024-09-11T21:55:32.158953+0000","last_fullsized":"2024-09-11T21:55:32.158953+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:33:02.589999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.097756+0000","last_change":"2024-09-11T21:55:20.092938+0000","last_active":"2024-09-11T21:55:21.097756+0000","last_peered":"2024-09-11T21:55:21.097756+0000","last_clean":"2024-09-11T21:55:21.097756+0000","last_became_active":"2024-09-11T21:55:20.092615+0000","last_became_peered":"2024-09-11T21:55:20.092615+0000","last_unstale":"2024-09-11T21:55:21.097756+0000","last_undegraded":"2024-09-11T21:55:21.097756+0000","last_fullsized":"2024-09-11T21:55:21.097756+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:40:06.957836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"84'4","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:29.139061+0000","last_change":"2024-09-11T21:55:18.029127+0000","last_active":"2024-09-11T21:55:29.139061+0000","last_peered":"2024-09-11T21:55:29.139061+0000","last_clean":"2024-09-11T21:55:29.139061+0000","last_became_active":"2024-09-11T21:55:18.028740+0000","last_became_peered":"2024-09-11T21:55:18.028740+0000","last_unstale":"2024-09-11T21:55:29.139061+0000","last_undegraded":"2024-09-11T21:55:29.139061+0000","last_fullsized":"2024-09-11T21:55:29.139061+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:14:01.128486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.085688+0000","last_change":"2024-09-11T21:55:20.092498+0000","last_active":"2024-09-11T21:55:21.085688+0000","last_peered":"2024-09-11T21:55:21.085688+0000","last_clean":"2024-09-11T21:55:21.085688+0000","last_became_active":"2024-09-11T21:55:20.092179+0000","last_became_peered":"2024-09-11T21:55:20.092179+0000","last_unstale":"2024-09-11T21:55:21.085688+0000","last_undegraded":"2024-09-11T21:55:21.085688+0000","last_fullsized":"2024-09-11T21:55:21.085688+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:48:58.536942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"84'5","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:40.175010+0000","last_change":"2024-09-11T21:55:18.115798+0000","last_active":"2024-09-11T21:55:40.175010+0000","last_peered":"2024-09-11T21:55:40.175010+0000","last_clean":"2024-09-11T21:55:40.175010+0000","last_became_active":"2024-09-11T21:55:18.115433+0000","last_became_peered":"2024-09-11T21:55:18.115433+0000","last_unstale":"2024-09-11T21:55:40.175010+0000","last_undegraded":"2024-09-11T21:55:40.175010+0000","last_fullsized":"2024-09-11T21:55:40.175010+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:03:32.097756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.086704+0000","last_change":"2024-09-11T21:55:20.100439+0000","last_active":"2024-09-11T21:55:21.086704+0000","last_peered":"2024-09-11T21:55:21.086704+0000","last_clean":"2024-09-11T21:55:21.086704+0000","last_became_active":"2024-09-11T21:55:20.098717+0000","last_became_peered":"2024-09-11T21:55:20.098717+0000","last_unstale":"2024-09-11T21:55:21.086704+0000","last_undegraded":"2024-09-11T21:55:21.086704+0000","last_fullsized":"2024-09-11T21:55:21.086704+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:15:10.874225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"84'6","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:40.175333+0000","last_change":"2024-09-11T21:55:18.026588+0000","last_active":"2024-09-11T21:55:40.175333+0000","last_peered":"2024-09-11T21:55:40.175333+0000","last_clean":"2024-09-11T21:55:40.175333+0000","last_became_active":"2024-09-11T21:55:18.026205+0000","last_became_peered":"2024-09-11T21:55:18.026205+0000","last_unstale":"2024-09-11T21:55:40.175333+0000","last_undegraded":"2024-09-11T21:55:40.175333+0000","last_fullsized":"2024-09-11T21:55:40.175333+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:45:34.701088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.086341+0000","last_change":"2024-09-11T21:55:20.100677+0000","last_active":"2024-09-11T21:55:21.086341+0000","last_peered":"2024-09-11T21:55:21.086341+0000","last_clean":"2024-09-11T21:55:21.086341+0000","last_became_active":"2024-09-11T21:55:20.100334+0000","last_became_peered":"2024-09-11T21:55:20.100334+0000","last_unstale":"2024-09-11T21:55:21.086341+0000","last_undegraded":"2024-09-11T21:55:21.086341+0000","last_fullsized":"2024-09-11T21:55:21.086341+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:15:16.792831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"84'1","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.096989+0000","last_change":"2024-09-11T21:55:18.116329+0000","last_active":"2024-09-11T21:55:21.096989+0000","last_peered":"2024-09-11T21:55:21.096989+0000","last_clean":"2024-09-11T21:55:21.096989+0000","last_became_active":"2024-09-11T21:55:18.115944+0000","last_became_peered":"2024-09-11T21:55:18.115944+0000","last_unstale":"2024-09-11T21:55:21.096989+0000","last_undegraded":"2024-09-11T21:55:21.096989+0000","last_fullsized":"2024-09-11T21:55:21.096989+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:03:04.164985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"84'1","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.086593+0000","last_change":"2024-09-11T21:55:20.089202+0000","last_active":"2024-09-11T21:55:21.086593+0000","last_peered":"2024-09-11T21:55:21.086593+0000","last_clean":"2024-09-11T21:55:21.086593+0000","last_became_active":"2024-09-11T21:55:20.088853+0000","last_became_peered":"2024-09-11T21:55:20.088853+0000","last_unstale":"2024-09-11T21:55:21.086593+0000","last_undegraded":"2024-09-11T21:55:21.086593+0000","last_fullsized":"2024-09-11T21:55:21.086593+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:39:36.496944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"84'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:31.157241+0000","last_change":"2024-09-11T21:55:18.117917+0000","last_active":"2024-09-11T21:55:31.157241+0000","last_peered":"2024-09-11T21:55:31.157241+0000","last_clean":"2024-09-11T21:55:31.157241+0000","last_became_active":"2024-09-11T21:55:18.117567+0000","last_became_peered":"2024-09-11T21:55:18.117567+0000","last_unstale":"2024-09-11T21:55:31.157241+0000","last_undegraded":"2024-09-11T21:55:31.157241+0000","last_fullsized":"2024-09-11T21:55:31.157241+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:21:32.546620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.085865+0000","last_change":"2024-09-11T21:55:20.092370+0000","last_active":"2024-09-11T21:55:21.085865+0000","last_peered":"2024-09-11T21:55:21.085865+0000","last_clean":"2024-09-11T21:55:21.085865+0000","last_became_active":"2024-09-11T21:55:20.091982+0000","last_became_peered":"2024-09-11T21:55:20.091982+0000","last_unstale":"2024-09-11T21:55:21.085865+0000","last_undegraded":"2024-09-11T21:55:21.085865+0000","last_fullsized":"2024-09-11T21:55:21.085865+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:59:55.214511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"84'12","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:41.098363+0000","last_change":"2024-09-11T21:55:18.023166+0000","last_active":"2024-09-11T21:55:41.098363+0000","last_peered":"2024-09-11T21:55:41.098363+0000","last_clean":"2024-09-11T21:55:41.098363+0000","last_became_active":"2024-09-11T21:55:18.022845+0000","last_became_peered":"2024-09-11T21:55:18.022845+0000","last_unstale":"2024-09-11T21:55:41.098363+0000","last_undegraded":"2024-09-11T21:55:41.098363+0000","last_fullsized":"2024-09-11T21:55:41.098363+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:08:06.770410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.098445+0000","last_change":"2024-09-11T21:55:20.091541+0000","last_active":"2024-09-11T21:55:21.098445+0000","last_peered":"2024-09-11T21:55:21.098445+0000","last_clean":"2024-09-11T21:55:21.098445+0000","last_became_active":"2024-09-11T21:55:20.090971+0000","last_became_peered":"2024-09-11T21:55:20.090971+0000","last_unstale":"2024-09-11T21:55:21.098445+0000","last_undegraded":"2024-09-11T21:55:21.098445+0000","last_fullsized":"2024-09-11T21:55:21.098445+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:22:31.974711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"84'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:31.157079+0000","last_change":"2024-09-11T21:55:18.030694+0000","last_active":"2024-09-11T21:55:31.157079+0000","last_peered":"2024-09-11T21:55:31.157079+0000","last_clean":"2024-09-11T21:55:31.157079+0000","last_became_active":"2024-09-11T21:55:18.029687+0000","last_became_peered":"2024-09-11T21:55:18.029687+0000","last_unstale":"2024-09-11T21:55:31.157079+0000","last_undegraded":"2024-09-11T21:55:31.157079+0000","last_fullsized":"2024-09-11T21:55:31.157079+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:18:15.009543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.100188+0000","last_change":"2024-09-11T21:55:20.100838+0000","last_active":"2024-09-11T21:55:21.100188+0000","last_peered":"2024-09-11T21:55:21.100188+0000","last_clean":"2024-09-11T21:55:21.100188+0000","last_became_active":"2024-09-11T21:55:20.100523+0000","last_became_peered":"2024-09-11T21:55:20.100523+0000","last_unstale":"2024-09-11T21:55:21.100188+0000","last_undegraded":"2024-09-11T21:55:21.100188+0000","last_fullsized":"2024-09-11T21:55:21.100188+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:44:59.052265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"84'2","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:28.131582+0000","last_change":"2024-09-11T21:55:18.115977+0000","last_active":"2024-09-11T21:55:28.131582+0000","last_peered":"2024-09-11T21:55:28.131582+0000","last_clean":"2024-09-11T21:55:28.131582+0000","last_became_active":"2024-09-11T21:55:18.115649+0000","last_became_peered":"2024-09-11T21:55:18.115649+0000","last_unstale":"2024-09-11T21:55:28.131582+0000","last_undegraded":"2024-09-11T21:55:28.131582+0000","last_fullsized":"2024-09-11T21:55:28.131582+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:42:07.621024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.098589+0000","last_change":"2024-09-11T21:55:20.095536+0000","last_active":"2024-09-11T21:55:21.098589+0000","last_peered":"2024-09-11T21:55:21.098589+0000","last_clean":"2024-09-11T21:55:21.098589+0000","last_became_active":"2024-09-11T21:55:20.095117+0000","last_became_peered":"2024-09-11T21:55:20.095117+0000","last_unstale":"2024-09-11T21:55:21.098589+0000","last_undegraded":"2024-09-11T21:55:21.098589+0000","last_fullsized":"2024-09-11T21:55:21.098589+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:46.524922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"84'7","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:31.157374+0000","last_change":"2024-09-11T21:55:18.026592+0000","last_active":"2024-09-11T21:55:31.157374+0000","last_peered":"2024-09-11T21:55:31.157374+0000","last_clean":"2024-09-11T21:55:31.157374+0000","last_became_active":"2024-09-11T21:55:18.026263+0000","last_became_peered":"2024-09-11T21:55:18.026263+0000","last_unstale":"2024-09-11T21:55:31.157374+0000","last_undegraded":"2024-09-11T21:55:31.157374+0000","last_fullsized":"2024-09-11T21:55:31.157374+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:22:29.382964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.103050+0000","last_change":"2024-09-11T21:55:20.093655+0000","last_active":"2024-09-11T21:55:21.103050+0000","last_peered":"2024-09-11T21:55:21.103050+0000","last_clean":"2024-09-11T21:55:21.103050+0000","last_became_active":"2024-09-11T21:55:20.093287+0000","last_became_peered":"2024-09-11T21:55:20.093287+0000","last_unstale":"2024-09-11T21:55:21.103050+0000","last_undegraded":"2024-09-11T21:55:21.103050+0000","last_fullsized":"2024-09-11T21:55:21.103050+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:46:15.912710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"84'3","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:30.150643+0000","last_change":"2024-09-11T21:55:18.028465+0000","last_active":"2024-09-11T21:55:30.150643+0000","last_peered":"2024-09-11T21:55:30.150643+0000","last_clean":"2024-09-11T21:55:30.150643+0000","last_became_active":"2024-09-11T21:55:18.028139+0000","last_became_peered":"2024-09-11T21:55:18.028139+0000","last_unstale":"2024-09-11T21:55:30.150643+0000","last_undegraded":"2024-09-11T21:55:30.150643+0000","last_fullsized":"2024-09-11T21:55:30.150643+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:52:47.808058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.097166+0000","last_change":"2024-09-11T21:55:20.096400+0000","last_active":"2024-09-11T21:55:21.097166+0000","last_peered":"2024-09-11T21:55:21.097166+0000","last_clean":"2024-09-11T21:55:21.097166+0000","last_became_active":"2024-09-11T21:55:20.096023+0000","last_became_peered":"2024-09-11T21:55:20.096023+0000","last_unstale":"2024-09-11T21:55:21.097166+0000","last_undegraded":"2024-09-11T21:55:21.097166+0000","last_fullsized":"2024-09-11T21:55:21.097166+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:19:34.150687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"84'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:28.131668+0000","last_change":"2024-09-11T21:55:18.028759+0000","last_active":"2024-09-11T21:55:28.131668+0000","last_peered":"2024-09-11T21:55:28.131668+0000","last_clean":"2024-09-11T21:55:28.131668+0000","last_became_active":"2024-09-11T21:55:18.028366+0000","last_became_peered":"2024-09-11T21:55:18.028366+0000","last_unstale":"2024-09-11T21:55:28.131668+0000","last_undegraded":"2024-09-11T21:55:28.131668+0000","last_fullsized":"2024-09-11T21:55:28.131668+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:42:05.836114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.099139+0000","last_change":"2024-09-11T21:55:20.093593+0000","last_active":"2024-09-11T21:55:21.099139+0000","last_peered":"2024-09-11T21:55:21.099139+0000","last_clean":"2024-09-11T21:55:21.099139+0000","last_became_active":"2024-09-11T21:55:20.093189+0000","last_became_peered":"2024-09-11T21:55:20.093189+0000","last_unstale":"2024-09-11T21:55:21.099139+0000","last_undegraded":"2024-09-11T21:55:21.099139+0000","last_fullsized":"2024-09-11T21:55:21.099139+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:47:36.802816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.086802+0000","last_change":"2024-09-11T21:55:20.100887+0000","last_active":"2024-09-11T21:55:21.086802+0000","last_peered":"2024-09-11T21:55:21.086802+0000","last_clean":"2024-09-11T21:55:21.086802+0000","last_became_active":"2024-09-11T21:55:20.100233+0000","last_became_peered":"2024-09-11T21:55:20.100233+0000","last_unstale":"2024-09-11T21:55:21.086802+0000","last_undegraded":"2024-09-11T21:55:21.086802+0000","last_fullsized":"2024-09-11T21:55:21.086802+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:30:05.820375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"84'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:31.157480+0000","last_change":"2024-09-11T21:55:18.027365+0000","last_active":"2024-09-11T21:55:31.157480+0000","last_peered":"2024-09-11T21:55:31.157480+0000","last_clean":"2024-09-11T21:55:31.157480+0000","last_became_active":"2024-09-11T21:55:18.027058+0000","last_became_peered":"2024-09-11T21:55:18.027058+0000","last_unstale":"2024-09-11T21:55:31.157480+0000","last_undegraded":"2024-09-11T21:55:31.157480+0000","last_fullsized":"2024-09-11T21:55:31.157480+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:30:50.613246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.100428+0000","last_change":"2024-09-11T21:55:20.095154+0000","last_active":"2024-09-11T21:55:21.100428+0000","last_peered":"2024-09-11T21:55:21.100428+0000","last_clean":"2024-09-11T21:55:21.100428+0000","last_became_active":"2024-09-11T21:55:20.093607+0000","last_became_peered":"2024-09-11T21:55:20.093607+0000","last_unstale":"2024-09-11T21:55:21.100428+0000","last_undegraded":"2024-09-11T21:55:21.100428+0000","last_fullsized":"2024-09-11T21:55:21.100428+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:27:53.612467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"84'12","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:32.152672+0000","last_change":"2024-09-11T21:55:18.029872+0000","last_active":"2024-09-11T21:55:32.152672+0000","last_peered":"2024-09-11T21:55:32.152672+0000","last_clean":"2024-09-11T21:55:32.152672+0000","last_became_active":"2024-09-11T21:55:18.029538+0000","last_became_peered":"2024-09-11T21:55:18.029538+0000","last_unstale":"2024-09-11T21:55:32.152672+0000","last_undegraded":"2024-09-11T21:55:32.152672+0000","last_fullsized":"2024-09-11T21:55:32.152672+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:05:16.824805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.100650+0000","last_change":"2024-09-11T21:55:20.098713+0000","last_active":"2024-09-11T21:55:21.100650+0000","last_peered":"2024-09-11T21:55:21.100650+0000","last_clean":"2024-09-11T21:55:21.100650+0000","last_became_active":"2024-09-11T21:55:20.098219+0000","last_became_peered":"2024-09-11T21:55:20.098219+0000","last_unstale":"2024-09-11T21:55:21.100650+0000","last_undegraded":"2024-09-11T21:55:21.100650+0000","last_fullsized":"2024-09-11T21:55:21.100650+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:07:10.585366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"84'2","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.105133+0000","last_change":"2024-09-11T21:55:18.115868+0000","last_active":"2024-09-11T21:55:21.105133+0000","last_peered":"2024-09-11T21:55:21.105133+0000","last_clean":"2024-09-11T21:55:21.105133+0000","last_became_active":"2024-09-11T21:55:18.115486+0000","last_became_peered":"2024-09-11T21:55:18.115486+0000","last_unstale":"2024-09-11T21:55:21.105133+0000","last_undegraded":"2024-09-11T21:55:21.105133+0000","last_fullsized":"2024-09-11T21:55:21.105133+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:16:57.932041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.099076+0000","last_change":"2024-09-11T21:55:20.096077+0000","last_active":"2024-09-11T21:55:21.099076+0000","last_peered":"2024-09-11T21:55:21.099076+0000","last_clean":"2024-09-11T21:55:21.099076+0000","last_became_active":"2024-09-11T21:55:20.095739+0000","last_became_peered":"2024-09-11T21:55:20.095739+0000","last_unstale":"2024-09-11T21:55:21.099076+0000","last_undegraded":"2024-09-11T21:55:21.099076+0000","last_fullsized":"2024-09-11T21:55:21.099076+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:05:28.089828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"84'13","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:42.176945+0000","last_change":"2024-09-11T21:55:18.025771+0000","last_active":"2024-09-11T21:55:42.176945+0000","last_peered":"2024-09-11T21:55:42.176945+0000","last_clean":"2024-09-11T21:55:42.176945+0000","last_became_active":"2024-09-11T21:55:18.025384+0000","last_became_peered":"2024-09-11T21:55:18.025384+0000","last_unstale":"2024-09-11T21:55:42.176945+0000","last_undegraded":"2024-09-11T21:55:42.176945+0000","last_fullsized":"2024-09-11T21:55:42.176945+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:58:21.070061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":54403,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":151,"num_read_kb":142,"num_write":233,"num_write_kb":118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":520192,"data_stored":157472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":201,"ondisk_log_size":201,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":283045,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449105,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":913408,"data_stored":436131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678038,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36376,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708776,"statfs":{"total":95995035648,"available":95957786624,"internally_reserved":0,"allocated":1056768,"data_stored":1073569,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":66,"seq":283467841564,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36260,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708892,"statfs":{"total":95995035648,"available":95957905408,"internally_reserved":0,"allocated":937984,"data_stored":435668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103201,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":933888,"data_stored":436046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332133,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":987136,"data_stored":1028959,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495658,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":435778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757296,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":1028244,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920820,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":435246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182459,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":435210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411391,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36304,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708848,"statfs":{"total":95995035648,"available":95957860352,"internally_reserved":0,"allocated":983040,"data_stored":478369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574917,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":970752,"data_stored":483493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738442,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":942080,"data_stored":437117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34355,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34355,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34355,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":48947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":43823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":48538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T21:55:46.726 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T21:55:46.986 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:55:47.368 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:47 smithi001 ceph-mon[30935]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T21:55:47.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:47.449+0000 7f5876eba640 1 -- 172.21.15.1:0/3278504644 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f587010c170 msgr2=0x7f587010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:47.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:47.449+0000 7f5876eba640 1 --2- 172.21.15.1:0/3278504644 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f587010c170 0x7f587010e610 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5868009ca0 tx=0x7f586802a950 comp rx=0 tx=0).stop 2024-09-11T21:55:47.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:47.449+0000 7f5876eba640 1 -- 172.21.15.1:0/3278504644 shutdown_connections 2024-09-11T21:55:47.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:47.449+0000 7f5876eba640 1 --2- 172.21.15.1:0/3278504644 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f587010c170 0x7f587010e610 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:47.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:47.449+0000 7f5876eba640 1 --2- 172.21.15.1:0/3278504644 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5870103660 0x7f587010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:47.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:47.449+0000 7f5876eba640 1 --2- 172.21.15.1:0/3278504644 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5870102d40 0x7f5870103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:47.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:47.449+0000 7f5876eba640 1 -- 172.21.15.1:0/3278504644 >> 172.21.15.1:0/3278504644 conn(0x7f58700fc7a0 msgr2=0x7f58700febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:55:47.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:47.449+0000 7f5876eba640 1 -- 172.21.15.1:0/3278504644 shutdown_connections 2024-09-11T21:55:47.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:47.449+0000 7f5876eba640 1 -- 172.21.15.1:0/3278504644 wait complete. 2024-09-11T21:55:47.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:47.450+0000 7f5876eba640 1 Processor -- start 2024-09-11T21:55:47.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:47.450+0000 7f5876eba640 1 -- start start 2024-09-11T21:55:47.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:47.450+0000 7f5876eba640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5870102d40 0x7f5870198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:47.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:47.450+0000 7f5876eba640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5870103660 0x7f58701989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:47.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:47.450+0000 7f5876eba640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f587010c170 0x7f587019cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:47.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:47.450+0000 7f5876eba640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5870111f50 con 0x7f5870102d40 2024-09-11T21:55:47.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:47.450+0000 7f5876eba640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5870111dd0 con 0x7f587010c170 2024-09-11T21:55:47.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:47.450+0000 7f5876eba640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f58701120d0 con 0x7f5870103660 2024-09-11T21:55:47.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:47.450+0000 7f5874c56640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5870102d40 0x7f5870198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:47.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:47.450+0000 7f5874c56640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5870102d40 0x7f5870198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39616/0 (socket says 172.21.15.1:39616) 2024-09-11T21:55:47.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:47.450+0000 7f5874c56640 1 -- 172.21.15.1:0/920294716 learned_addr learned my addr 172.21.15.1:0/920294716 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:55:47.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:47.450+0000 7f586ffff640 1 --2- 172.21.15.1:0/920294716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5870103660 0x7f58701989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:47.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:47.450+0000 7f5875457640 1 --2- 172.21.15.1:0/920294716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f587010c170 0x7f587019cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:47.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:47.451+0000 7f5874c56640 1 -- 172.21.15.1:0/920294716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5870103660 msgr2=0x7f58701989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:47.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:47.451+0000 7f5874c56640 1 --2- 172.21.15.1:0/920294716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5870103660 0x7f58701989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:47.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:47.451+0000 7f5874c56640 1 -- 172.21.15.1:0/920294716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f587010c170 msgr2=0x7f587019cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:47.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:47.451+0000 7f5874c56640 1 --2- 172.21.15.1:0/920294716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f587010c170 0x7f587019cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:47.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:47.451+0000 7f5874c56640 1 -- 172.21.15.1:0/920294716 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f587019d410 con 0x7f5870102d40 2024-09-11T21:55:47.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:47.451+0000 7f586ffff640 1 --2- 172.21.15.1:0/920294716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5870103660 0x7f58701989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:55:47.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:47.451+0000 7f5875457640 1 --2- 172.21.15.1:0/920294716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f587010c170 0x7f587019cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:55:47.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:47.451+0000 7f5874c56640 1 --2- 172.21.15.1:0/920294716 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5870102d40 0x7f5870198460 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f586000c890 tx=0x7f586000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:55:47.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:47.451+0000 7f586dffb640 1 -- 172.21.15.1:0/920294716 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5860004220 con 0x7f5870102d40 2024-09-11T21:55:47.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:47.451+0000 7f586dffb640 1 -- 172.21.15.1:0/920294716 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58600043c0 con 0x7f5870102d40 2024-09-11T21:55:47.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:47.451+0000 7f586dffb640 1 -- 172.21.15.1:0/920294716 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58600049e0 con 0x7f5870102d40 2024-09-11T21:55:47.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:47.451+0000 7f5876eba640 1 -- 172.21.15.1:0/920294716 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f587019d700 con 0x7f5870102d40 2024-09-11T21:55:47.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:47.451+0000 7f5876eba640 1 -- 172.21.15.1:0/920294716 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58701a4fe0 con 0x7f5870102d40 2024-09-11T21:55:47.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:47.452+0000 7f5876eba640 1 -- 172.21.15.1:0/920294716 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5870103120 con 0x7f5870102d40 2024-09-11T21:55:47.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:47.453+0000 7f586dffb640 1 -- 172.21.15.1:0/920294716 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f58600079b0 con 0x7f5870102d40 2024-09-11T21:55:47.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:47.454+0000 7f586dffb640 1 --2- 172.21.15.1:0/920294716 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f58580a2c90 0x7f58580a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:47.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:47.454+0000 7f586ffff640 1 --2- 172.21.15.1:0/920294716 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f58580a2c90 0x7f58580a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:47.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:47.454+0000 7f586dffb640 1 -- 172.21.15.1:0/920294716 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f58600cf0b0 con 0x7f5870102d40 2024-09-11T21:55:47.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:47.455+0000 7f586ffff640 1 --2- 172.21.15.1:0/920294716 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f58580a2c90 0x7f58580a5150 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f58700fedc0 tx=0x7f586400b610 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:55:47.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:47.459+0000 7f586dffb640 1 -- 172.21.15.1:0/920294716 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5860014030 con 0x7f5870102d40 2024-09-11T21:55:47.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:47 smithi053 ceph-mon[34908]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T21:55:47.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:47 smithi161 ceph-mon[35615]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T21:55:47.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:47.630+0000 7f5876eba640 1 -- 172.21.15.1:0/920294716 --> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5870107240 con 0x7f58580a2c90 2024-09-11T21:55:47.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:47.636+0000 7f586dffb640 1 -- 172.21.15.1:0/920294716 <== mgr.14150 v2:172.21.15.1:6800/3890350886 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258912 (secure 0 0 0) 0x7f5870107240 con 0x7f58580a2c90 2024-09-11T21:55:47.639 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:55:47.640 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-11T21:55:47.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:47.639+0000 7f5876eba640 1 -- 172.21.15.1:0/920294716 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f58580a2c90 msgr2=0x7f58580a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:47.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:47.639+0000 7f5876eba640 1 --2- 172.21.15.1:0/920294716 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f58580a2c90 0x7f58580a5150 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f58700fedc0 tx=0x7f586400b610 comp rx=0 tx=0).stop 2024-09-11T21:55:47.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:47.639+0000 7f5876eba640 1 -- 172.21.15.1:0/920294716 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5870102d40 msgr2=0x7f5870198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:47.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:47.639+0000 7f5876eba640 1 --2- 172.21.15.1:0/920294716 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5870102d40 0x7f5870198460 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f586000c890 tx=0x7f586000cd50 comp rx=0 tx=0).stop 2024-09-11T21:55:47.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:47.639+0000 7f5876eba640 1 -- 172.21.15.1:0/920294716 shutdown_connections 2024-09-11T21:55:47.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:47.639+0000 7f5876eba640 1 --2- 172.21.15.1:0/920294716 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f58580a2c90 0x7f58580a5150 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:47.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:47.639+0000 7f5876eba640 1 --2- 172.21.15.1:0/920294716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f587010c170 0x7f587019cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:47.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:47.639+0000 7f5876eba640 1 --2- 172.21.15.1:0/920294716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5870103660 0x7f58701989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:47.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:47.639+0000 7f5876eba640 1 --2- 172.21.15.1:0/920294716 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5870102d40 0x7f5870198460 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:47.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:47.639+0000 7f5876eba640 1 -- 172.21.15.1:0/920294716 >> 172.21.15.1:0/920294716 conn(0x7f58700fc7a0 msgr2=0x7f58700fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:55:47.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:47.639+0000 7f5876eba640 1 -- 172.21.15.1:0/920294716 shutdown_connections 2024-09-11T21:55:47.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:47.639+0000 7f5876eba640 1 -- 172.21.15.1:0/920294716 wait complete. 2024-09-11T21:55:48.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:48 smithi001 ceph-mon[30935]: pgmap v295: 97 pgs: 97 active+clean; 630 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 4.4 KiB/s wr, 14 op/s 2024-09-11T21:55:48.474 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":295,"stamp":"2024-09-11T21:55:46.612852+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":645005,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":307,"num_read_kb":266,"num_write":416,"num_write_kb":3058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":340,"ondisk_log_size":340,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":385452,"kb_used_data":11244,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124556372,"statfs":{"total":1151940427776,"available":1151545724928,"internally_reserved":0,"allocated":11513856,"data_stored":7149567,"data_compressed":107988,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19142,"internal_metadata":382973242},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":40657,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":137,"num_read_kb":111,"num_write":38,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003035"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.103272+0000","last_change":"2024-09-11T21:55:20.095385+0000","last_active":"2024-09-11T21:55:21.103272+0000","last_peered":"2024-09-11T21:55:21.103272+0000","last_clean":"2024-09-11T21:55:21.103272+0000","last_became_active":"2024-09-11T21:55:20.095077+0000","last_became_peered":"2024-09-11T21:55:20.095077+0000","last_unstale":"2024-09-11T21:55:21.103272+0000","last_undegraded":"2024-09-11T21:55:21.103272+0000","last_fullsized":"2024-09-11T21:55:21.103272+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:53:52.231522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.099859+0000","last_change":"2024-09-11T21:55:20.103540+0000","last_active":"2024-09-11T21:55:21.099859+0000","last_peered":"2024-09-11T21:55:21.099859+0000","last_clean":"2024-09-11T21:55:21.099859+0000","last_became_active":"2024-09-11T21:55:20.103233+0000","last_became_peered":"2024-09-11T21:55:20.103233+0000","last_unstale":"2024-09-11T21:55:21.099859+0000","last_undegraded":"2024-09-11T21:55:21.099859+0000","last_fullsized":"2024-09-11T21:55:21.099859+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:51:29.222202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.100567+0000","last_change":"2024-09-11T21:55:20.098538+0000","last_active":"2024-09-11T21:55:21.100567+0000","last_peered":"2024-09-11T21:55:21.100567+0000","last_clean":"2024-09-11T21:55:21.100567+0000","last_became_active":"2024-09-11T21:55:20.098202+0000","last_became_peered":"2024-09-11T21:55:20.098202+0000","last_unstale":"2024-09-11T21:55:21.100567+0000","last_undegraded":"2024-09-11T21:55:21.100567+0000","last_fullsized":"2024-09-11T21:55:21.100567+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:33:46.636952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.097725+0000","last_change":"2024-09-11T21:55:20.100869+0000","last_active":"2024-09-11T21:55:21.097725+0000","last_peered":"2024-09-11T21:55:21.097725+0000","last_clean":"2024-09-11T21:55:21.097725+0000","last_became_active":"2024-09-11T21:55:20.100573+0000","last_became_peered":"2024-09-11T21:55:20.100573+0000","last_unstale":"2024-09-11T21:55:21.097725+0000","last_undegraded":"2024-09-11T21:55:21.097725+0000","last_fullsized":"2024-09-11T21:55:21.097725+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:58:11.154105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.098166+0000","last_change":"2024-09-11T21:55:20.091648+0000","last_active":"2024-09-11T21:55:21.098166+0000","last_peered":"2024-09-11T21:55:21.098166+0000","last_clean":"2024-09-11T21:55:21.098166+0000","last_became_active":"2024-09-11T21:55:20.091222+0000","last_became_peered":"2024-09-11T21:55:20.091222+0000","last_unstale":"2024-09-11T21:55:21.098166+0000","last_undegraded":"2024-09-11T21:55:21.098166+0000","last_fullsized":"2024-09-11T21:55:21.098166+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:55:03.420754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.086150+0000","last_change":"2024-09-11T21:55:20.100446+0000","last_active":"2024-09-11T21:55:21.086150+0000","last_peered":"2024-09-11T21:55:21.086150+0000","last_clean":"2024-09-11T21:55:21.086150+0000","last_became_active":"2024-09-11T21:55:20.100062+0000","last_became_peered":"2024-09-11T21:55:20.100062+0000","last_unstale":"2024-09-11T21:55:21.086150+0000","last_undegraded":"2024-09-11T21:55:21.086150+0000","last_fullsized":"2024-09-11T21:55:21.086150+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:49:37.109258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.097709+0000","last_change":"2024-09-11T21:55:20.101084+0000","last_active":"2024-09-11T21:55:21.097709+0000","last_peered":"2024-09-11T21:55:21.097709+0000","last_clean":"2024-09-11T21:55:21.097709+0000","last_became_active":"2024-09-11T21:55:20.100739+0000","last_became_peered":"2024-09-11T21:55:20.100739+0000","last_unstale":"2024-09-11T21:55:21.097709+0000","last_undegraded":"2024-09-11T21:55:21.097709+0000","last_fullsized":"2024-09-11T21:55:21.097709+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:46:58.969559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.098694+0000","last_change":"2024-09-11T21:55:20.088715+0000","last_active":"2024-09-11T21:55:21.098694+0000","last_peered":"2024-09-11T21:55:21.098694+0000","last_clean":"2024-09-11T21:55:21.098694+0000","last_became_active":"2024-09-11T21:55:20.088280+0000","last_became_peered":"2024-09-11T21:55:20.088280+0000","last_unstale":"2024-09-11T21:55:21.098694+0000","last_undegraded":"2024-09-11T21:55:21.098694+0000","last_fullsized":"2024-09-11T21:55:21.098694+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:06:24.450444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.101362+0000","last_change":"2024-09-11T21:55:20.101266+0000","last_active":"2024-09-11T21:55:21.101362+0000","last_peered":"2024-09-11T21:55:21.101362+0000","last_clean":"2024-09-11T21:55:21.101362+0000","last_became_active":"2024-09-11T21:55:20.100919+0000","last_became_peered":"2024-09-11T21:55:20.100919+0000","last_unstale":"2024-09-11T21:55:21.101362+0000","last_undegraded":"2024-09-11T21:55:21.101362+0000","last_fullsized":"2024-09-11T21:55:21.101362+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:47:15.466082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.086697+0000","last_change":"2024-09-11T21:55:20.098086+0000","last_active":"2024-09-11T21:55:21.086697+0000","last_peered":"2024-09-11T21:55:21.086697+0000","last_clean":"2024-09-11T21:55:21.086697+0000","last_became_active":"2024-09-11T21:55:20.097754+0000","last_became_peered":"2024-09-11T21:55:20.097754+0000","last_unstale":"2024-09-11T21:55:21.086697+0000","last_undegraded":"2024-09-11T21:55:21.086697+0000","last_fullsized":"2024-09-11T21:55:21.086697+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:37:00.098345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.099716+0000","last_change":"2024-09-11T21:55:20.092671+0000","last_active":"2024-09-11T21:55:21.099716+0000","last_peered":"2024-09-11T21:55:21.099716+0000","last_clean":"2024-09-11T21:55:21.099716+0000","last_became_active":"2024-09-11T21:55:20.092112+0000","last_became_peered":"2024-09-11T21:55:20.092112+0000","last_unstale":"2024-09-11T21:55:21.099716+0000","last_undegraded":"2024-09-11T21:55:21.099716+0000","last_fullsized":"2024-09-11T21:55:21.099716+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:06:28.538838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.097879+0000","last_change":"2024-09-11T21:55:20.099863+0000","last_active":"2024-09-11T21:55:21.097879+0000","last_peered":"2024-09-11T21:55:21.097879+0000","last_clean":"2024-09-11T21:55:21.097879+0000","last_became_active":"2024-09-11T21:55:20.099173+0000","last_became_peered":"2024-09-11T21:55:20.099173+0000","last_unstale":"2024-09-11T21:55:21.097879+0000","last_undegraded":"2024-09-11T21:55:21.097879+0000","last_fullsized":"2024-09-11T21:55:21.097879+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:34:56.669410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.085802+0000","last_change":"2024-09-11T21:55:20.101212+0000","last_active":"2024-09-11T21:55:21.085802+0000","last_peered":"2024-09-11T21:55:21.085802+0000","last_clean":"2024-09-11T21:55:21.085802+0000","last_became_active":"2024-09-11T21:55:20.100876+0000","last_became_peered":"2024-09-11T21:55:20.100876+0000","last_unstale":"2024-09-11T21:55:21.085802+0000","last_undegraded":"2024-09-11T21:55:21.085802+0000","last_fullsized":"2024-09-11T21:55:21.085802+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:14:18.104009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.100238+0000","last_change":"2024-09-11T21:55:20.095394+0000","last_active":"2024-09-11T21:55:21.100238+0000","last_peered":"2024-09-11T21:55:21.100238+0000","last_clean":"2024-09-11T21:55:21.100238+0000","last_became_active":"2024-09-11T21:55:20.095101+0000","last_became_peered":"2024-09-11T21:55:20.095101+0000","last_unstale":"2024-09-11T21:55:21.100238+0000","last_undegraded":"2024-09-11T21:55:21.100238+0000","last_fullsized":"2024-09-11T21:55:21.100238+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:53:32.529359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.100035+0000","last_change":"2024-09-11T21:55:20.099668+0000","last_active":"2024-09-11T21:55:21.100035+0000","last_peered":"2024-09-11T21:55:21.100035+0000","last_clean":"2024-09-11T21:55:21.100035+0000","last_became_active":"2024-09-11T21:55:20.099392+0000","last_became_peered":"2024-09-11T21:55:20.099392+0000","last_unstale":"2024-09-11T21:55:21.100035+0000","last_undegraded":"2024-09-11T21:55:21.100035+0000","last_fullsized":"2024-09-11T21:55:21.100035+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:50:00.479508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.099808+0000","last_change":"2024-09-11T21:55:20.093019+0000","last_active":"2024-09-11T21:55:21.099808+0000","last_peered":"2024-09-11T21:55:21.099808+0000","last_clean":"2024-09-11T21:55:21.099808+0000","last_became_active":"2024-09-11T21:55:20.092691+0000","last_became_peered":"2024-09-11T21:55:20.092691+0000","last_unstale":"2024-09-11T21:55:21.099808+0000","last_undegraded":"2024-09-11T21:55:21.099808+0000","last_fullsized":"2024-09-11T21:55:21.099808+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:04:36.684992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.103502+0000","last_change":"2024-09-11T21:55:20.097333+0000","last_active":"2024-09-11T21:55:21.103502+0000","last_peered":"2024-09-11T21:55:21.103502+0000","last_clean":"2024-09-11T21:55:21.103502+0000","last_became_active":"2024-09-11T21:55:20.096752+0000","last_became_peered":"2024-09-11T21:55:20.096752+0000","last_unstale":"2024-09-11T21:55:21.103502+0000","last_undegraded":"2024-09-11T21:55:21.103502+0000","last_fullsized":"2024-09-11T21:55:21.103502+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:27:20.722174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.097172+0000","last_change":"2024-09-11T21:55:20.093972+0000","last_active":"2024-09-11T21:55:21.097172+0000","last_peered":"2024-09-11T21:55:21.097172+0000","last_clean":"2024-09-11T21:55:21.097172+0000","last_became_active":"2024-09-11T21:55:20.093633+0000","last_became_peered":"2024-09-11T21:55:20.093633+0000","last_unstale":"2024-09-11T21:55:21.097172+0000","last_undegraded":"2024-09-11T21:55:21.097172+0000","last_fullsized":"2024-09-11T21:55:21.097172+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:25:18.973332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.097726+0000","last_change":"2024-09-11T21:55:20.094239+0000","last_active":"2024-09-11T21:55:21.097726+0000","last_peered":"2024-09-11T21:55:21.097726+0000","last_clean":"2024-09-11T21:55:21.097726+0000","last_became_active":"2024-09-11T21:55:20.093924+0000","last_became_peered":"2024-09-11T21:55:20.093924+0000","last_unstale":"2024-09-11T21:55:21.097726+0000","last_undegraded":"2024-09-11T21:55:21.097726+0000","last_fullsized":"2024-09-11T21:55:21.097726+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:39:53.012780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.102943+0000","last_change":"2024-09-11T21:55:20.099335+0000","last_active":"2024-09-11T21:55:21.102943+0000","last_peered":"2024-09-11T21:55:21.102943+0000","last_clean":"2024-09-11T21:55:21.102943+0000","last_became_active":"2024-09-11T21:55:20.098929+0000","last_became_peered":"2024-09-11T21:55:20.098929+0000","last_unstale":"2024-09-11T21:55:21.102943+0000","last_undegraded":"2024-09-11T21:55:21.102943+0000","last_fullsized":"2024-09-11T21:55:21.102943+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:38:43.117829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.085822+0000","last_change":"2024-09-11T21:55:20.094068+0000","last_active":"2024-09-11T21:55:21.085822+0000","last_peered":"2024-09-11T21:55:21.085822+0000","last_clean":"2024-09-11T21:55:21.085822+0000","last_became_active":"2024-09-11T21:55:20.093637+0000","last_became_peered":"2024-09-11T21:55:20.093637+0000","last_unstale":"2024-09-11T21:55:21.085822+0000","last_undegraded":"2024-09-11T21:55:21.085822+0000","last_fullsized":"2024-09-11T21:55:21.085822+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:52:59.289715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.100443+0000","last_change":"2024-09-11T21:55:20.090778+0000","last_active":"2024-09-11T21:55:21.100443+0000","last_peered":"2024-09-11T21:55:21.100443+0000","last_clean":"2024-09-11T21:55:21.100443+0000","last_became_active":"2024-09-11T21:55:20.090443+0000","last_became_peered":"2024-09-11T21:55:20.090443+0000","last_unstale":"2024-09-11T21:55:21.100443+0000","last_undegraded":"2024-09-11T21:55:21.100443+0000","last_fullsized":"2024-09-11T21:55:21.100443+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:46:21.488228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.097160+0000","last_change":"2024-09-11T21:55:20.092238+0000","last_active":"2024-09-11T21:55:21.097160+0000","last_peered":"2024-09-11T21:55:21.097160+0000","last_clean":"2024-09-11T21:55:21.097160+0000","last_became_active":"2024-09-11T21:55:20.091961+0000","last_became_peered":"2024-09-11T21:55:20.091961+0000","last_unstale":"2024-09-11T21:55:21.097160+0000","last_undegraded":"2024-09-11T21:55:21.097160+0000","last_fullsized":"2024-09-11T21:55:21.097160+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:54:40.445441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.086361+0000","last_change":"2024-09-11T21:55:20.102944+0000","last_active":"2024-09-11T21:55:21.086361+0000","last_peered":"2024-09-11T21:55:21.086361+0000","last_clean":"2024-09-11T21:55:21.086361+0000","last_became_active":"2024-09-11T21:55:20.102533+0000","last_became_peered":"2024-09-11T21:55:20.102533+0000","last_unstale":"2024-09-11T21:55:21.086361+0000","last_undegraded":"2024-09-11T21:55:21.086361+0000","last_fullsized":"2024-09-11T21:55:21.086361+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:12:15.832622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.098471+0000","last_change":"2024-09-11T21:55:20.092177+0000","last_active":"2024-09-11T21:55:21.098471+0000","last_peered":"2024-09-11T21:55:21.098471+0000","last_clean":"2024-09-11T21:55:21.098471+0000","last_became_active":"2024-09-11T21:55:20.091897+0000","last_became_peered":"2024-09-11T21:55:20.091897+0000","last_unstale":"2024-09-11T21:55:21.098471+0000","last_undegraded":"2024-09-11T21:55:21.098471+0000","last_fullsized":"2024-09-11T21:55:21.098471+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:38:57.371153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.099223+0000","last_change":"2024-09-11T21:55:20.092166+0000","last_active":"2024-09-11T21:55:21.099223+0000","last_peered":"2024-09-11T21:55:21.099223+0000","last_clean":"2024-09-11T21:55:21.099223+0000","last_became_active":"2024-09-11T21:55:20.091854+0000","last_became_peered":"2024-09-11T21:55:20.091854+0000","last_unstale":"2024-09-11T21:55:21.099223+0000","last_undegraded":"2024-09-11T21:55:21.099223+0000","last_fullsized":"2024-09-11T21:55:21.099223+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:53:32.256008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.103425+0000","last_change":"2024-09-11T21:55:20.103976+0000","last_active":"2024-09-11T21:55:21.103425+0000","last_peered":"2024-09-11T21:55:21.103425+0000","last_clean":"2024-09-11T21:55:21.103425+0000","last_became_active":"2024-09-11T21:55:20.103668+0000","last_became_peered":"2024-09-11T21:55:20.103668+0000","last_unstale":"2024-09-11T21:55:21.103425+0000","last_undegraded":"2024-09-11T21:55:21.103425+0000","last_fullsized":"2024-09-11T21:55:21.103425+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:13:42.663641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.085778+0000","last_change":"2024-09-11T21:55:20.093898+0000","last_active":"2024-09-11T21:55:21.085778+0000","last_peered":"2024-09-11T21:55:21.085778+0000","last_clean":"2024-09-11T21:55:21.085778+0000","last_became_active":"2024-09-11T21:55:20.093348+0000","last_became_peered":"2024-09-11T21:55:20.093348+0000","last_unstale":"2024-09-11T21:55:21.085778+0000","last_undegraded":"2024-09-11T21:55:21.085778+0000","last_fullsized":"2024-09-11T21:55:21.085778+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:00:06.185478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.100902+0000","last_change":"2024-09-11T21:55:20.100575+0000","last_active":"2024-09-11T21:55:21.100902+0000","last_peered":"2024-09-11T21:55:21.100902+0000","last_clean":"2024-09-11T21:55:21.100902+0000","last_became_active":"2024-09-11T21:55:20.100255+0000","last_became_peered":"2024-09-11T21:55:20.100255+0000","last_unstale":"2024-09-11T21:55:21.100902+0000","last_undegraded":"2024-09-11T21:55:21.100902+0000","last_fullsized":"2024-09-11T21:55:21.100902+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:48:57.718298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.097296+0000","last_change":"2024-09-11T21:55:20.097548+0000","last_active":"2024-09-11T21:55:21.097296+0000","last_peered":"2024-09-11T21:55:21.097296+0000","last_clean":"2024-09-11T21:55:21.097296+0000","last_became_active":"2024-09-11T21:55:20.097124+0000","last_became_peered":"2024-09-11T21:55:20.097124+0000","last_unstale":"2024-09-11T21:55:21.097296+0000","last_undegraded":"2024-09-11T21:55:21.097296+0000","last_fullsized":"2024-09-11T21:55:21.097296+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:22:23.904457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.097695+0000","last_change":"2024-09-11T21:55:20.093822+0000","last_active":"2024-09-11T21:55:21.097695+0000","last_peered":"2024-09-11T21:55:21.097695+0000","last_clean":"2024-09-11T21:55:21.097695+0000","last_became_active":"2024-09-11T21:55:20.093361+0000","last_became_peered":"2024-09-11T21:55:20.093361+0000","last_unstale":"2024-09-11T21:55:21.097695+0000","last_undegraded":"2024-09-11T21:55:21.097695+0000","last_fullsized":"2024-09-11T21:55:21.097695+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:37:06.362732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.103360+0000","last_change":"2024-09-11T21:55:20.103990+0000","last_active":"2024-09-11T21:55:21.103360+0000","last_peered":"2024-09-11T21:55:21.103360+0000","last_clean":"2024-09-11T21:55:21.103360+0000","last_became_active":"2024-09-11T21:55:20.103375+0000","last_became_peered":"2024-09-11T21:55:20.103375+0000","last_unstale":"2024-09-11T21:55:21.103360+0000","last_undegraded":"2024-09-11T21:55:21.103360+0000","last_fullsized":"2024-09-11T21:55:21.103360+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:02:06.786888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.099679+0000","last_change":"2024-09-11T21:55:20.101535+0000","last_active":"2024-09-11T21:55:21.099679+0000","last_peered":"2024-09-11T21:55:21.099679+0000","last_clean":"2024-09-11T21:55:21.099679+0000","last_became_active":"2024-09-11T21:55:20.101129+0000","last_became_peered":"2024-09-11T21:55:20.101129+0000","last_unstale":"2024-09-11T21:55:21.099679+0000","last_undegraded":"2024-09-11T21:55:21.099679+0000","last_fullsized":"2024-09-11T21:55:21.099679+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:09:15.673670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"84'6","reported_seq":38,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:41.177259+0000","last_change":"2024-09-11T21:55:18.026947+0000","last_active":"2024-09-11T21:55:41.177259+0000","last_peered":"2024-09-11T21:55:41.177259+0000","last_clean":"2024-09-11T21:55:41.177259+0000","last_became_active":"2024-09-11T21:55:18.026632+0000","last_became_peered":"2024-09-11T21:55:18.026632+0000","last_unstale":"2024-09-11T21:55:41.177259+0000","last_undegraded":"2024-09-11T21:55:41.177259+0000","last_fullsized":"2024-09-11T21:55:41.177259+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:55:16.618074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.086576+0000","last_change":"2024-09-11T21:55:20.100197+0000","last_active":"2024-09-11T21:55:21.086576+0000","last_peered":"2024-09-11T21:55:21.086576+0000","last_clean":"2024-09-11T21:55:21.086576+0000","last_became_active":"2024-09-11T21:55:20.098494+0000","last_became_peered":"2024-09-11T21:55:20.098494+0000","last_unstale":"2024-09-11T21:55:21.086576+0000","last_undegraded":"2024-09-11T21:55:21.086576+0000","last_fullsized":"2024-09-11T21:55:21.086576+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:14.505314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"84'7","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:30.150870+0000","last_change":"2024-09-11T21:55:18.024359+0000","last_active":"2024-09-11T21:55:30.150870+0000","last_peered":"2024-09-11T21:55:30.150870+0000","last_clean":"2024-09-11T21:55:30.150870+0000","last_became_active":"2024-09-11T21:55:18.023624+0000","last_became_peered":"2024-09-11T21:55:18.023624+0000","last_unstale":"2024-09-11T21:55:30.150870+0000","last_undegraded":"2024-09-11T21:55:30.150870+0000","last_fullsized":"2024-09-11T21:55:30.150870+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:13:38.582232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"84'3","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:29.138838+0000","last_change":"2024-09-11T21:55:18.032541+0000","last_active":"2024-09-11T21:55:29.138838+0000","last_peered":"2024-09-11T21:55:29.138838+0000","last_clean":"2024-09-11T21:55:29.138838+0000","last_became_active":"2024-09-11T21:55:18.032218+0000","last_became_peered":"2024-09-11T21:55:18.032218+0000","last_unstale":"2024-09-11T21:55:29.138838+0000","last_undegraded":"2024-09-11T21:55:29.138838+0000","last_fullsized":"2024-09-11T21:55:29.138838+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:38:20.872069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.098851+0000","last_change":"2024-09-11T21:55:20.090030+0000","last_active":"2024-09-11T21:55:21.098851+0000","last_peered":"2024-09-11T21:55:21.098851+0000","last_clean":"2024-09-11T21:55:21.098851+0000","last_became_active":"2024-09-11T21:55:20.089481+0000","last_became_peered":"2024-09-11T21:55:20.089481+0000","last_unstale":"2024-09-11T21:55:21.098851+0000","last_undegraded":"2024-09-11T21:55:21.098851+0000","last_fullsized":"2024-09-11T21:55:21.098851+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:41:57.038519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"84'3","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:31.157367+0000","last_change":"2024-09-11T21:55:18.026656+0000","last_active":"2024-09-11T21:55:31.157367+0000","last_peered":"2024-09-11T21:55:31.157367+0000","last_clean":"2024-09-11T21:55:31.157367+0000","last_became_active":"2024-09-11T21:55:18.026318+0000","last_became_peered":"2024-09-11T21:55:18.026318+0000","last_unstale":"2024-09-11T21:55:31.157367+0000","last_undegraded":"2024-09-11T21:55:31.157367+0000","last_fullsized":"2024-09-11T21:55:31.157367+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:14:41.438334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.096818+0000","last_change":"2024-09-11T21:55:20.090790+0000","last_active":"2024-09-11T21:55:21.096818+0000","last_peered":"2024-09-11T21:55:21.096818+0000","last_clean":"2024-09-11T21:55:21.096818+0000","last_became_active":"2024-09-11T21:55:20.090468+0000","last_became_peered":"2024-09-11T21:55:20.090468+0000","last_unstale":"2024-09-11T21:55:21.096818+0000","last_undegraded":"2024-09-11T21:55:21.096818+0000","last_fullsized":"2024-09-11T21:55:21.096818+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:19.115482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"84'4","reported_seq":48,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:44.179611+0000","last_change":"2024-09-11T21:55:18.026989+0000","last_active":"2024-09-11T21:55:44.179611+0000","last_peered":"2024-09-11T21:55:44.179611+0000","last_clean":"2024-09-11T21:55:44.179611+0000","last_became_active":"2024-09-11T21:55:18.026229+0000","last_became_peered":"2024-09-11T21:55:18.026229+0000","last_unstale":"2024-09-11T21:55:44.179611+0000","last_undegraded":"2024-09-11T21:55:44.179611+0000","last_fullsized":"2024-09-11T21:55:44.179611+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:50:35.745789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.101844+0000","last_change":"2024-09-11T21:55:20.095884+0000","last_active":"2024-09-11T21:55:21.101844+0000","last_peered":"2024-09-11T21:55:21.101844+0000","last_clean":"2024-09-11T21:55:21.101844+0000","last_became_active":"2024-09-11T21:55:20.094887+0000","last_became_peered":"2024-09-11T21:55:20.094887+0000","last_unstale":"2024-09-11T21:55:21.101844+0000","last_undegraded":"2024-09-11T21:55:21.101844+0000","last_fullsized":"2024-09-11T21:55:21.101844+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:45:03.579703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"84'4","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:31.158136+0000","last_change":"2024-09-11T21:55:18.030474+0000","last_active":"2024-09-11T21:55:31.158136+0000","last_peered":"2024-09-11T21:55:31.158136+0000","last_clean":"2024-09-11T21:55:31.158136+0000","last_became_active":"2024-09-11T21:55:18.030104+0000","last_became_peered":"2024-09-11T21:55:18.030104+0000","last_unstale":"2024-09-11T21:55:31.158136+0000","last_undegraded":"2024-09-11T21:55:31.158136+0000","last_fullsized":"2024-09-11T21:55:31.158136+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:21:00.484595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.100730+0000","last_change":"2024-09-11T21:55:20.102807+0000","last_active":"2024-09-11T21:55:21.100730+0000","last_peered":"2024-09-11T21:55:21.100730+0000","last_clean":"2024-09-11T21:55:21.100730+0000","last_became_active":"2024-09-11T21:55:20.102581+0000","last_became_peered":"2024-09-11T21:55:20.102581+0000","last_unstale":"2024-09-11T21:55:21.100730+0000","last_undegraded":"2024-09-11T21:55:21.100730+0000","last_fullsized":"2024-09-11T21:55:21.100730+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:29:23.139788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"84'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:30.150754+0000","last_change":"2024-09-11T21:55:18.117685+0000","last_active":"2024-09-11T21:55:30.150754+0000","last_peered":"2024-09-11T21:55:30.150754+0000","last_clean":"2024-09-11T21:55:30.150754+0000","last_became_active":"2024-09-11T21:55:18.117261+0000","last_became_peered":"2024-09-11T21:55:18.117261+0000","last_unstale":"2024-09-11T21:55:30.150754+0000","last_undegraded":"2024-09-11T21:55:30.150754+0000","last_fullsized":"2024-09-11T21:55:30.150754+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:42:13.180725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.100810+0000","last_change":"2024-09-11T21:55:20.086562+0000","last_active":"2024-09-11T21:55:21.100810+0000","last_peered":"2024-09-11T21:55:21.100810+0000","last_clean":"2024-09-11T21:55:21.100810+0000","last_became_active":"2024-09-11T21:55:20.086197+0000","last_became_peered":"2024-09-11T21:55:20.086197+0000","last_unstale":"2024-09-11T21:55:21.100810+0000","last_undegraded":"2024-09-11T21:55:21.100810+0000","last_fullsized":"2024-09-11T21:55:21.100810+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:23:44.705132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'24","reported_seq":74,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:44.181835+0000","last_change":"2024-09-11T21:55:18.028320+0000","last_active":"2024-09-11T21:55:44.181835+0000","last_peered":"2024-09-11T21:55:44.181835+0000","last_clean":"2024-09-11T21:55:44.181835+0000","last_became_active":"2024-09-11T21:55:18.027982+0000","last_became_peered":"2024-09-11T21:55:18.027982+0000","last_unstale":"2024-09-11T21:55:44.181835+0000","last_undegraded":"2024-09-11T21:55:44.181835+0000","last_fullsized":"2024-09-11T21:55:44.181835+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:08:36.750665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46802,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":61,"num_write":26,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.099277+0000","last_change":"2024-09-11T21:55:20.091695+0000","last_active":"2024-09-11T21:55:21.099277+0000","last_peered":"2024-09-11T21:55:21.099277+0000","last_clean":"2024-09-11T21:55:21.099277+0000","last_became_active":"2024-09-11T21:55:20.091128+0000","last_became_peered":"2024-09-11T21:55:20.091128+0000","last_unstale":"2024-09-11T21:55:21.099277+0000","last_undegraded":"2024-09-11T21:55:21.099277+0000","last_fullsized":"2024-09-11T21:55:21.099277+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:16:20.751952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"84'9","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:32.158923+0000","last_change":"2024-09-11T21:55:18.030827+0000","last_active":"2024-09-11T21:55:32.158923+0000","last_peered":"2024-09-11T21:55:32.158923+0000","last_clean":"2024-09-11T21:55:32.158923+0000","last_became_active":"2024-09-11T21:55:18.030354+0000","last_became_peered":"2024-09-11T21:55:18.030354+0000","last_unstale":"2024-09-11T21:55:32.158923+0000","last_undegraded":"2024-09-11T21:55:32.158923+0000","last_fullsized":"2024-09-11T21:55:32.158923+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:23:19.578820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.085795+0000","last_change":"2024-09-11T21:55:20.090160+0000","last_active":"2024-09-11T21:55:21.085795+0000","last_peered":"2024-09-11T21:55:21.085795+0000","last_clean":"2024-09-11T21:55:21.085795+0000","last_became_active":"2024-09-11T21:55:20.089801+0000","last_became_peered":"2024-09-11T21:55:20.089801+0000","last_unstale":"2024-09-11T21:55:21.085795+0000","last_undegraded":"2024-09-11T21:55:21.085795+0000","last_fullsized":"2024-09-11T21:55:21.085795+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:34:16.093123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"84'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:31.157445+0000","last_change":"2024-09-11T21:55:18.026817+0000","last_active":"2024-09-11T21:55:31.157445+0000","last_peered":"2024-09-11T21:55:31.157445+0000","last_clean":"2024-09-11T21:55:31.157445+0000","last_became_active":"2024-09-11T21:55:18.026188+0000","last_became_peered":"2024-09-11T21:55:18.026188+0000","last_unstale":"2024-09-11T21:55:31.157445+0000","last_undegraded":"2024-09-11T21:55:31.157445+0000","last_fullsized":"2024-09-11T21:55:31.157445+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:55:14.774503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.098605+0000","last_change":"2024-09-11T21:55:20.095613+0000","last_active":"2024-09-11T21:55:21.098605+0000","last_peered":"2024-09-11T21:55:21.098605+0000","last_clean":"2024-09-11T21:55:21.098605+0000","last_became_active":"2024-09-11T21:55:20.095302+0000","last_became_peered":"2024-09-11T21:55:20.095302+0000","last_unstale":"2024-09-11T21:55:21.098605+0000","last_undegraded":"2024-09-11T21:55:21.098605+0000","last_fullsized":"2024-09-11T21:55:21.098605+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:37:27.961213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'5","reported_seq":39,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:44.180390+0000","last_change":"2024-09-11T21:55:18.032417+0000","last_active":"2024-09-11T21:55:44.180390+0000","last_peered":"2024-09-11T21:55:44.180390+0000","last_clean":"2024-09-11T21:55:44.180390+0000","last_became_active":"2024-09-11T21:55:18.031419+0000","last_became_peered":"2024-09-11T21:55:18.031419+0000","last_unstale":"2024-09-11T21:55:44.180390+0000","last_undegraded":"2024-09-11T21:55:44.180390+0000","last_fullsized":"2024-09-11T21:55:44.180390+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:30:52.285678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.099572+0000","last_change":"2024-09-11T21:55:20.101965+0000","last_active":"2024-09-11T21:55:21.099572+0000","last_peered":"2024-09-11T21:55:21.099572+0000","last_clean":"2024-09-11T21:55:21.099572+0000","last_became_active":"2024-09-11T21:55:20.101534+0000","last_became_peered":"2024-09-11T21:55:20.101534+0000","last_unstale":"2024-09-11T21:55:21.099572+0000","last_undegraded":"2024-09-11T21:55:21.099572+0000","last_fullsized":"2024-09-11T21:55:21.099572+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:41:30.685121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"84'17","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:31.158074+0000","last_change":"2024-09-11T21:55:18.116059+0000","last_active":"2024-09-11T21:55:31.158074+0000","last_peered":"2024-09-11T21:55:31.158074+0000","last_clean":"2024-09-11T21:55:31.158074+0000","last_became_active":"2024-09-11T21:55:18.115621+0000","last_became_peered":"2024-09-11T21:55:18.115621+0000","last_unstale":"2024-09-11T21:55:31.158074+0000","last_undegraded":"2024-09-11T21:55:31.158074+0000","last_fullsized":"2024-09-11T21:55:31.158074+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:47:17.815695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.086741+0000","last_change":"2024-09-11T21:55:20.100903+0000","last_active":"2024-09-11T21:55:21.086741+0000","last_peered":"2024-09-11T21:55:21.086741+0000","last_clean":"2024-09-11T21:55:21.086741+0000","last_became_active":"2024-09-11T21:55:20.100561+0000","last_became_peered":"2024-09-11T21:55:20.100561+0000","last_unstale":"2024-09-11T21:55:21.086741+0000","last_undegraded":"2024-09-11T21:55:21.086741+0000","last_fullsized":"2024-09-11T21:55:21.086741+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:00:45.472238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'2","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:31.134748+0000","last_change":"2024-09-11T21:55:18.029549+0000","last_active":"2024-09-11T21:55:31.134748+0000","last_peered":"2024-09-11T21:55:31.134748+0000","last_clean":"2024-09-11T21:55:31.134748+0000","last_became_active":"2024-09-11T21:55:18.029178+0000","last_became_peered":"2024-09-11T21:55:18.029178+0000","last_unstale":"2024-09-11T21:55:31.134748+0000","last_undegraded":"2024-09-11T21:55:31.134748+0000","last_fullsized":"2024-09-11T21:55:31.134748+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:37:07.915786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.103401+0000","last_change":"2024-09-11T21:55:20.104202+0000","last_active":"2024-09-11T21:55:21.103401+0000","last_peered":"2024-09-11T21:55:21.103401+0000","last_clean":"2024-09-11T21:55:21.103401+0000","last_became_active":"2024-09-11T21:55:20.103464+0000","last_became_peered":"2024-09-11T21:55:20.103464+0000","last_unstale":"2024-09-11T21:55:21.103401+0000","last_undegraded":"2024-09-11T21:55:21.103401+0000","last_fullsized":"2024-09-11T21:55:21.103401+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:30:16.044143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"84'5","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:31.157256+0000","last_change":"2024-09-11T21:55:18.026183+0000","last_active":"2024-09-11T21:55:31.157256+0000","last_peered":"2024-09-11T21:55:31.157256+0000","last_clean":"2024-09-11T21:55:31.157256+0000","last_became_active":"2024-09-11T21:55:18.025849+0000","last_became_peered":"2024-09-11T21:55:18.025849+0000","last_unstale":"2024-09-11T21:55:31.157256+0000","last_undegraded":"2024-09-11T21:55:31.157256+0000","last_fullsized":"2024-09-11T21:55:31.157256+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:21:12.189158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.098264+0000","last_change":"2024-09-11T21:55:20.090652+0000","last_active":"2024-09-11T21:55:21.098264+0000","last_peered":"2024-09-11T21:55:21.098264+0000","last_clean":"2024-09-11T21:55:21.098264+0000","last_became_active":"2024-09-11T21:55:20.090348+0000","last_became_peered":"2024-09-11T21:55:20.090348+0000","last_unstale":"2024-09-11T21:55:21.098264+0000","last_undegraded":"2024-09-11T21:55:21.098264+0000","last_fullsized":"2024-09-11T21:55:21.098264+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:10:58.733371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'3","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:31.157673+0000","last_change":"2024-09-11T21:55:18.024388+0000","last_active":"2024-09-11T21:55:31.157673+0000","last_peered":"2024-09-11T21:55:31.157673+0000","last_clean":"2024-09-11T21:55:31.157673+0000","last_became_active":"2024-09-11T21:55:18.024100+0000","last_became_peered":"2024-09-11T21:55:18.024100+0000","last_unstale":"2024-09-11T21:55:31.157673+0000","last_undegraded":"2024-09-11T21:55:31.157673+0000","last_fullsized":"2024-09-11T21:55:31.157673+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:21:11.152282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"84'1","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.186238+0000","last_change":"2024-09-11T21:55:20.101007+0000","last_active":"2024-09-11T21:55:21.186238+0000","last_peered":"2024-09-11T21:55:21.186238+0000","last_clean":"2024-09-11T21:55:21.186238+0000","last_became_active":"2024-09-11T21:55:20.100642+0000","last_became_peered":"2024-09-11T21:55:20.100642+0000","last_unstale":"2024-09-11T21:55:21.186238+0000","last_undegraded":"2024-09-11T21:55:21.186238+0000","last_fullsized":"2024-09-11T21:55:21.186238+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:16:05.642184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"84'6","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:43.178949+0000","last_change":"2024-09-11T21:55:18.116216+0000","last_active":"2024-09-11T21:55:43.178949+0000","last_peered":"2024-09-11T21:55:43.178949+0000","last_clean":"2024-09-11T21:55:43.178949+0000","last_became_active":"2024-09-11T21:55:18.115871+0000","last_became_peered":"2024-09-11T21:55:18.115871+0000","last_unstale":"2024-09-11T21:55:43.178949+0000","last_undegraded":"2024-09-11T21:55:43.178949+0000","last_fullsized":"2024-09-11T21:55:43.178949+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:16:56.027636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":16,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.099871+0000","last_change":"2024-09-11T21:55:20.092421+0000","last_active":"2024-09-11T21:55:21.099871+0000","last_peered":"2024-09-11T21:55:21.099871+0000","last_clean":"2024-09-11T21:55:21.099871+0000","last_became_active":"2024-09-11T21:55:20.091707+0000","last_became_peered":"2024-09-11T21:55:20.091707+0000","last_unstale":"2024-09-11T21:55:21.099871+0000","last_undegraded":"2024-09-11T21:55:21.099871+0000","last_fullsized":"2024-09-11T21:55:21.099871+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:12:38.157687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":346,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.099897+0000","last_change":"2024-09-11T21:54:07.542362+0000","last_active":"2024-09-11T21:55:21.099897+0000","last_peered":"2024-09-11T21:55:21.099897+0000","last_clean":"2024-09-11T21:55:21.099897+0000","last_became_active":"2024-09-11T21:54:07.541871+0000","last_became_peered":"2024-09-11T21:54:07.541871+0000","last_unstale":"2024-09-11T21:55:21.099897+0000","last_undegraded":"2024-09-11T21:55:21.099897+0000","last_fullsized":"2024-09-11T21:55:21.099897+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:50:41.026895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:50:41.026895+0000","last_clean_scrub_stamp":"2024-09-11T21:50:41.026895+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:05:06.154268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"84'6","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:32.158953+0000","last_change":"2024-09-11T21:55:18.030558+0000","last_active":"2024-09-11T21:55:32.158953+0000","last_peered":"2024-09-11T21:55:32.158953+0000","last_clean":"2024-09-11T21:55:32.158953+0000","last_became_active":"2024-09-11T21:55:18.030216+0000","last_became_peered":"2024-09-11T21:55:18.030216+0000","last_unstale":"2024-09-11T21:55:32.158953+0000","last_undegraded":"2024-09-11T21:55:32.158953+0000","last_fullsized":"2024-09-11T21:55:32.158953+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:33:02.589999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.097756+0000","last_change":"2024-09-11T21:55:20.092938+0000","last_active":"2024-09-11T21:55:21.097756+0000","last_peered":"2024-09-11T21:55:21.097756+0000","last_clean":"2024-09-11T21:55:21.097756+0000","last_became_active":"2024-09-11T21:55:20.092615+0000","last_became_peered":"2024-09-11T21:55:20.092615+0000","last_unstale":"2024-09-11T21:55:21.097756+0000","last_undegraded":"2024-09-11T21:55:21.097756+0000","last_fullsized":"2024-09-11T21:55:21.097756+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:40:06.957836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"84'4","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:29.139061+0000","last_change":"2024-09-11T21:55:18.029127+0000","last_active":"2024-09-11T21:55:29.139061+0000","last_peered":"2024-09-11T21:55:29.139061+0000","last_clean":"2024-09-11T21:55:29.139061+0000","last_became_active":"2024-09-11T21:55:18.028740+0000","last_became_peered":"2024-09-11T21:55:18.028740+0000","last_unstale":"2024-09-11T21:55:29.139061+0000","last_undegraded":"2024-09-11T21:55:29.139061+0000","last_fullsized":"2024-09-11T21:55:29.139061+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:14:01.128486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.085688+0000","last_change":"2024-09-11T21:55:20.092498+0000","last_active":"2024-09-11T21:55:21.085688+0000","last_peered":"2024-09-11T21:55:21.085688+0000","last_clean":"2024-09-11T21:55:21.085688+0000","last_became_active":"2024-09-11T21:55:20.092179+0000","last_became_peered":"2024-09-11T21:55:20.092179+0000","last_unstale":"2024-09-11T21:55:21.085688+0000","last_undegraded":"2024-09-11T21:55:21.085688+0000","last_fullsized":"2024-09-11T21:55:21.085688+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:48:58.536942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"84'5","reported_seq":39,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:45.181174+0000","last_change":"2024-09-11T21:55:18.115798+0000","last_active":"2024-09-11T21:55:45.181174+0000","last_peered":"2024-09-11T21:55:45.181174+0000","last_clean":"2024-09-11T21:55:45.181174+0000","last_became_active":"2024-09-11T21:55:18.115433+0000","last_became_peered":"2024-09-11T21:55:18.115433+0000","last_unstale":"2024-09-11T21:55:45.181174+0000","last_undegraded":"2024-09-11T21:55:45.181174+0000","last_fullsized":"2024-09-11T21:55:45.181174+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:03:32.097756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.086704+0000","last_change":"2024-09-11T21:55:20.100439+0000","last_active":"2024-09-11T21:55:21.086704+0000","last_peered":"2024-09-11T21:55:21.086704+0000","last_clean":"2024-09-11T21:55:21.086704+0000","last_became_active":"2024-09-11T21:55:20.098717+0000","last_became_peered":"2024-09-11T21:55:20.098717+0000","last_unstale":"2024-09-11T21:55:21.086704+0000","last_undegraded":"2024-09-11T21:55:21.086704+0000","last_fullsized":"2024-09-11T21:55:21.086704+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:15:10.874225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"84'6","reported_seq":41,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:45.180755+0000","last_change":"2024-09-11T21:55:18.026588+0000","last_active":"2024-09-11T21:55:45.180755+0000","last_peered":"2024-09-11T21:55:45.180755+0000","last_clean":"2024-09-11T21:55:45.180755+0000","last_became_active":"2024-09-11T21:55:18.026205+0000","last_became_peered":"2024-09-11T21:55:18.026205+0000","last_unstale":"2024-09-11T21:55:45.180755+0000","last_undegraded":"2024-09-11T21:55:45.180755+0000","last_fullsized":"2024-09-11T21:55:45.180755+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:45:34.701088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.086341+0000","last_change":"2024-09-11T21:55:20.100677+0000","last_active":"2024-09-11T21:55:21.086341+0000","last_peered":"2024-09-11T21:55:21.086341+0000","last_clean":"2024-09-11T21:55:21.086341+0000","last_became_active":"2024-09-11T21:55:20.100334+0000","last_became_peered":"2024-09-11T21:55:20.100334+0000","last_unstale":"2024-09-11T21:55:21.086341+0000","last_undegraded":"2024-09-11T21:55:21.086341+0000","last_fullsized":"2024-09-11T21:55:21.086341+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:15:16.792831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"84'1","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.096989+0000","last_change":"2024-09-11T21:55:18.116329+0000","last_active":"2024-09-11T21:55:21.096989+0000","last_peered":"2024-09-11T21:55:21.096989+0000","last_clean":"2024-09-11T21:55:21.096989+0000","last_became_active":"2024-09-11T21:55:18.115944+0000","last_became_peered":"2024-09-11T21:55:18.115944+0000","last_unstale":"2024-09-11T21:55:21.096989+0000","last_undegraded":"2024-09-11T21:55:21.096989+0000","last_fullsized":"2024-09-11T21:55:21.096989+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:03:04.164985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"84'1","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.086593+0000","last_change":"2024-09-11T21:55:20.089202+0000","last_active":"2024-09-11T21:55:21.086593+0000","last_peered":"2024-09-11T21:55:21.086593+0000","last_clean":"2024-09-11T21:55:21.086593+0000","last_became_active":"2024-09-11T21:55:20.088853+0000","last_became_peered":"2024-09-11T21:55:20.088853+0000","last_unstale":"2024-09-11T21:55:21.086593+0000","last_undegraded":"2024-09-11T21:55:21.086593+0000","last_fullsized":"2024-09-11T21:55:21.086593+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:39:36.496944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"84'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:31.157241+0000","last_change":"2024-09-11T21:55:18.117917+0000","last_active":"2024-09-11T21:55:31.157241+0000","last_peered":"2024-09-11T21:55:31.157241+0000","last_clean":"2024-09-11T21:55:31.157241+0000","last_became_active":"2024-09-11T21:55:18.117567+0000","last_became_peered":"2024-09-11T21:55:18.117567+0000","last_unstale":"2024-09-11T21:55:31.157241+0000","last_undegraded":"2024-09-11T21:55:31.157241+0000","last_fullsized":"2024-09-11T21:55:31.157241+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:21:32.546620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.085865+0000","last_change":"2024-09-11T21:55:20.092370+0000","last_active":"2024-09-11T21:55:21.085865+0000","last_peered":"2024-09-11T21:55:21.085865+0000","last_clean":"2024-09-11T21:55:21.085865+0000","last_became_active":"2024-09-11T21:55:20.091982+0000","last_became_peered":"2024-09-11T21:55:20.091982+0000","last_unstale":"2024-09-11T21:55:21.085865+0000","last_undegraded":"2024-09-11T21:55:21.085865+0000","last_fullsized":"2024-09-11T21:55:21.085865+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:59:55.214511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"84'12","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:41.098363+0000","last_change":"2024-09-11T21:55:18.023166+0000","last_active":"2024-09-11T21:55:41.098363+0000","last_peered":"2024-09-11T21:55:41.098363+0000","last_clean":"2024-09-11T21:55:41.098363+0000","last_became_active":"2024-09-11T21:55:18.022845+0000","last_became_peered":"2024-09-11T21:55:18.022845+0000","last_unstale":"2024-09-11T21:55:41.098363+0000","last_undegraded":"2024-09-11T21:55:41.098363+0000","last_fullsized":"2024-09-11T21:55:41.098363+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:08:06.770410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.098445+0000","last_change":"2024-09-11T21:55:20.091541+0000","last_active":"2024-09-11T21:55:21.098445+0000","last_peered":"2024-09-11T21:55:21.098445+0000","last_clean":"2024-09-11T21:55:21.098445+0000","last_became_active":"2024-09-11T21:55:20.090971+0000","last_became_peered":"2024-09-11T21:55:20.090971+0000","last_unstale":"2024-09-11T21:55:21.098445+0000","last_undegraded":"2024-09-11T21:55:21.098445+0000","last_fullsized":"2024-09-11T21:55:21.098445+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:22:31.974711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"84'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:31.157079+0000","last_change":"2024-09-11T21:55:18.030694+0000","last_active":"2024-09-11T21:55:31.157079+0000","last_peered":"2024-09-11T21:55:31.157079+0000","last_clean":"2024-09-11T21:55:31.157079+0000","last_became_active":"2024-09-11T21:55:18.029687+0000","last_became_peered":"2024-09-11T21:55:18.029687+0000","last_unstale":"2024-09-11T21:55:31.157079+0000","last_undegraded":"2024-09-11T21:55:31.157079+0000","last_fullsized":"2024-09-11T21:55:31.157079+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:18:15.009543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.100188+0000","last_change":"2024-09-11T21:55:20.100838+0000","last_active":"2024-09-11T21:55:21.100188+0000","last_peered":"2024-09-11T21:55:21.100188+0000","last_clean":"2024-09-11T21:55:21.100188+0000","last_became_active":"2024-09-11T21:55:20.100523+0000","last_became_peered":"2024-09-11T21:55:20.100523+0000","last_unstale":"2024-09-11T21:55:21.100188+0000","last_undegraded":"2024-09-11T21:55:21.100188+0000","last_fullsized":"2024-09-11T21:55:21.100188+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:44:59.052265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"84'2","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:28.131582+0000","last_change":"2024-09-11T21:55:18.115977+0000","last_active":"2024-09-11T21:55:28.131582+0000","last_peered":"2024-09-11T21:55:28.131582+0000","last_clean":"2024-09-11T21:55:28.131582+0000","last_became_active":"2024-09-11T21:55:18.115649+0000","last_became_peered":"2024-09-11T21:55:18.115649+0000","last_unstale":"2024-09-11T21:55:28.131582+0000","last_undegraded":"2024-09-11T21:55:28.131582+0000","last_fullsized":"2024-09-11T21:55:28.131582+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:42:07.621024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.098589+0000","last_change":"2024-09-11T21:55:20.095536+0000","last_active":"2024-09-11T21:55:21.098589+0000","last_peered":"2024-09-11T21:55:21.098589+0000","last_clean":"2024-09-11T21:55:21.098589+0000","last_became_active":"2024-09-11T21:55:20.095117+0000","last_became_peered":"2024-09-11T21:55:20.095117+0000","last_unstale":"2024-09-11T21:55:21.098589+0000","last_undegraded":"2024-09-11T21:55:21.098589+0000","last_fullsized":"2024-09-11T21:55:21.098589+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:46.524922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"84'7","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:31.157374+0000","last_change":"2024-09-11T21:55:18.026592+0000","last_active":"2024-09-11T21:55:31.157374+0000","last_peered":"2024-09-11T21:55:31.157374+0000","last_clean":"2024-09-11T21:55:31.157374+0000","last_became_active":"2024-09-11T21:55:18.026263+0000","last_became_peered":"2024-09-11T21:55:18.026263+0000","last_unstale":"2024-09-11T21:55:31.157374+0000","last_undegraded":"2024-09-11T21:55:31.157374+0000","last_fullsized":"2024-09-11T21:55:31.157374+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:22:29.382964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.103050+0000","last_change":"2024-09-11T21:55:20.093655+0000","last_active":"2024-09-11T21:55:21.103050+0000","last_peered":"2024-09-11T21:55:21.103050+0000","last_clean":"2024-09-11T21:55:21.103050+0000","last_became_active":"2024-09-11T21:55:20.093287+0000","last_became_peered":"2024-09-11T21:55:20.093287+0000","last_unstale":"2024-09-11T21:55:21.103050+0000","last_undegraded":"2024-09-11T21:55:21.103050+0000","last_fullsized":"2024-09-11T21:55:21.103050+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:46:15.912710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"84'3","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:30.150643+0000","last_change":"2024-09-11T21:55:18.028465+0000","last_active":"2024-09-11T21:55:30.150643+0000","last_peered":"2024-09-11T21:55:30.150643+0000","last_clean":"2024-09-11T21:55:30.150643+0000","last_became_active":"2024-09-11T21:55:18.028139+0000","last_became_peered":"2024-09-11T21:55:18.028139+0000","last_unstale":"2024-09-11T21:55:30.150643+0000","last_undegraded":"2024-09-11T21:55:30.150643+0000","last_fullsized":"2024-09-11T21:55:30.150643+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:52:47.808058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.097166+0000","last_change":"2024-09-11T21:55:20.096400+0000","last_active":"2024-09-11T21:55:21.097166+0000","last_peered":"2024-09-11T21:55:21.097166+0000","last_clean":"2024-09-11T21:55:21.097166+0000","last_became_active":"2024-09-11T21:55:20.096023+0000","last_became_peered":"2024-09-11T21:55:20.096023+0000","last_unstale":"2024-09-11T21:55:21.097166+0000","last_undegraded":"2024-09-11T21:55:21.097166+0000","last_fullsized":"2024-09-11T21:55:21.097166+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:19:34.150687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"84'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:28.131668+0000","last_change":"2024-09-11T21:55:18.028759+0000","last_active":"2024-09-11T21:55:28.131668+0000","last_peered":"2024-09-11T21:55:28.131668+0000","last_clean":"2024-09-11T21:55:28.131668+0000","last_became_active":"2024-09-11T21:55:18.028366+0000","last_became_peered":"2024-09-11T21:55:18.028366+0000","last_unstale":"2024-09-11T21:55:28.131668+0000","last_undegraded":"2024-09-11T21:55:28.131668+0000","last_fullsized":"2024-09-11T21:55:28.131668+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:42:05.836114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.099139+0000","last_change":"2024-09-11T21:55:20.093593+0000","last_active":"2024-09-11T21:55:21.099139+0000","last_peered":"2024-09-11T21:55:21.099139+0000","last_clean":"2024-09-11T21:55:21.099139+0000","last_became_active":"2024-09-11T21:55:20.093189+0000","last_became_peered":"2024-09-11T21:55:20.093189+0000","last_unstale":"2024-09-11T21:55:21.099139+0000","last_undegraded":"2024-09-11T21:55:21.099139+0000","last_fullsized":"2024-09-11T21:55:21.099139+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:47:36.802816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.086802+0000","last_change":"2024-09-11T21:55:20.100887+0000","last_active":"2024-09-11T21:55:21.086802+0000","last_peered":"2024-09-11T21:55:21.086802+0000","last_clean":"2024-09-11T21:55:21.086802+0000","last_became_active":"2024-09-11T21:55:20.100233+0000","last_became_peered":"2024-09-11T21:55:20.100233+0000","last_unstale":"2024-09-11T21:55:21.086802+0000","last_undegraded":"2024-09-11T21:55:21.086802+0000","last_fullsized":"2024-09-11T21:55:21.086802+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:30:05.820375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"84'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:31.157480+0000","last_change":"2024-09-11T21:55:18.027365+0000","last_active":"2024-09-11T21:55:31.157480+0000","last_peered":"2024-09-11T21:55:31.157480+0000","last_clean":"2024-09-11T21:55:31.157480+0000","last_became_active":"2024-09-11T21:55:18.027058+0000","last_became_peered":"2024-09-11T21:55:18.027058+0000","last_unstale":"2024-09-11T21:55:31.157480+0000","last_undegraded":"2024-09-11T21:55:31.157480+0000","last_fullsized":"2024-09-11T21:55:31.157480+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:30:50.613246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.100428+0000","last_change":"2024-09-11T21:55:20.095154+0000","last_active":"2024-09-11T21:55:21.100428+0000","last_peered":"2024-09-11T21:55:21.100428+0000","last_clean":"2024-09-11T21:55:21.100428+0000","last_became_active":"2024-09-11T21:55:20.093607+0000","last_became_peered":"2024-09-11T21:55:20.093607+0000","last_unstale":"2024-09-11T21:55:21.100428+0000","last_undegraded":"2024-09-11T21:55:21.100428+0000","last_fullsized":"2024-09-11T21:55:21.100428+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:27:53.612467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"84'12","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:32.152672+0000","last_change":"2024-09-11T21:55:18.029872+0000","last_active":"2024-09-11T21:55:32.152672+0000","last_peered":"2024-09-11T21:55:32.152672+0000","last_clean":"2024-09-11T21:55:32.152672+0000","last_became_active":"2024-09-11T21:55:18.029538+0000","last_became_peered":"2024-09-11T21:55:18.029538+0000","last_unstale":"2024-09-11T21:55:32.152672+0000","last_undegraded":"2024-09-11T21:55:32.152672+0000","last_fullsized":"2024-09-11T21:55:32.152672+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:05:16.824805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.100650+0000","last_change":"2024-09-11T21:55:20.098713+0000","last_active":"2024-09-11T21:55:21.100650+0000","last_peered":"2024-09-11T21:55:21.100650+0000","last_clean":"2024-09-11T21:55:21.100650+0000","last_became_active":"2024-09-11T21:55:20.098219+0000","last_became_peered":"2024-09-11T21:55:20.098219+0000","last_unstale":"2024-09-11T21:55:21.100650+0000","last_undegraded":"2024-09-11T21:55:21.100650+0000","last_fullsized":"2024-09-11T21:55:21.100650+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:07:10.585366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"84'2","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.105133+0000","last_change":"2024-09-11T21:55:18.115868+0000","last_active":"2024-09-11T21:55:21.105133+0000","last_peered":"2024-09-11T21:55:21.105133+0000","last_clean":"2024-09-11T21:55:21.105133+0000","last_became_active":"2024-09-11T21:55:18.115486+0000","last_became_peered":"2024-09-11T21:55:18.115486+0000","last_unstale":"2024-09-11T21:55:21.105133+0000","last_undegraded":"2024-09-11T21:55:21.105133+0000","last_fullsized":"2024-09-11T21:55:21.105133+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:16:57.932041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.099076+0000","last_change":"2024-09-11T21:55:20.096077+0000","last_active":"2024-09-11T21:55:21.099076+0000","last_peered":"2024-09-11T21:55:21.099076+0000","last_clean":"2024-09-11T21:55:21.099076+0000","last_became_active":"2024-09-11T21:55:20.095739+0000","last_became_peered":"2024-09-11T21:55:20.095739+0000","last_unstale":"2024-09-11T21:55:21.099076+0000","last_undegraded":"2024-09-11T21:55:21.099076+0000","last_fullsized":"2024-09-11T21:55:21.099076+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:05:28.089828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"84'13","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:42.176945+0000","last_change":"2024-09-11T21:55:18.025771+0000","last_active":"2024-09-11T21:55:42.176945+0000","last_peered":"2024-09-11T21:55:42.176945+0000","last_clean":"2024-09-11T21:55:42.176945+0000","last_became_active":"2024-09-11T21:55:18.025384+0000","last_became_peered":"2024-09-11T21:55:18.025384+0000","last_unstale":"2024-09-11T21:55:42.176945+0000","last_undegraded":"2024-09-11T21:55:42.176945+0000","last_fullsized":"2024-09-11T21:55:42.176945+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:58:21.070061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":54403,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":171,"num_read_kb":147,"num_write":233,"num_write_kb":118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":528384,"data_stored":163209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":201,"ondisk_log_size":201,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":283045,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449106,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":913408,"data_stored":436131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678038,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36376,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708776,"statfs":{"total":95995035648,"available":95957786624,"internally_reserved":0,"allocated":1056768,"data_stored":1073569,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":66,"seq":283467841564,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36260,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708892,"statfs":{"total":95995035648,"available":95957905408,"internally_reserved":0,"allocated":937984,"data_stored":435668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103201,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":933888,"data_stored":436046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332134,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":987136,"data_stored":1028959,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495659,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":435778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757296,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":1028244,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920821,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":435246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182459,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":435210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411392,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":991232,"data_stored":484106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574917,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":970752,"data_stored":483493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738442,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":942080,"data_stored":437117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34355,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34355,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34355,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":48947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":49560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":48538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T21:55:48.478 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-11T21:55:48.479 INFO:tasks.cephadm:Setting up client nodes... 2024-09-11T21:55:48.479 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-11T21:55:48.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:48 smithi053 ceph-mon[34908]: pgmap v295: 97 pgs: 97 active+clean; 630 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 4.4 KiB/s wr, 14 op/s 2024-09-11T21:55:48.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:48 smithi161 ceph-mon[35615]: pgmap v295: 97 pgs: 97 active+clean; 630 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 4.4 KiB/s wr, 14 op/s 2024-09-11T21:55:48.738 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:55:49.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:49.221+0000 7f504b8b7640 1 -- 172.21.15.1:0/273364441 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5044102e00 msgr2=0x7f5044103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:49.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:49.221+0000 7f504b8b7640 1 --2- 172.21.15.1:0/273364441 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5044102e00 0x7f5044103280 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5034009900 tx=0x7f503402a410 comp rx=0 tx=0).stop 2024-09-11T21:55:49.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:49.221+0000 7f504b8b7640 1 -- 172.21.15.1:0/273364441 shutdown_connections 2024-09-11T21:55:49.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:49.221+0000 7f504b8b7640 1 --2- 172.21.15.1:0/273364441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f50441037c0 0x7f504410a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:49.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:49.221+0000 7f504b8b7640 1 --2- 172.21.15.1:0/273364441 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5044102e00 0x7f5044103280 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:49.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:49.221+0000 7f504b8b7640 1 --2- 172.21.15.1:0/273364441 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5044101c00 0x7f5044102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:49.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:49.221+0000 7f504b8b7640 1 -- 172.21.15.1:0/273364441 >> 172.21.15.1:0/273364441 conn(0x7f50440fd390 msgr2=0x7f50440ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:55:49.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:49.221+0000 7f504b8b7640 1 -- 172.21.15.1:0/273364441 shutdown_connections 2024-09-11T21:55:49.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:49.221+0000 7f504b8b7640 1 -- 172.21.15.1:0/273364441 wait complete. 2024-09-11T21:55:49.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:49.222+0000 7f504b8b7640 1 Processor -- start 2024-09-11T21:55:49.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:49.222+0000 7f504b8b7640 1 -- start start 2024-09-11T21:55:49.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:49.222+0000 7f504b8b7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5044101c00 0x7f5044075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:49.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:49.222+0000 7f504b8b7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5044102e00 0x7f50440745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:49.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:49.222+0000 7f504b8b7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f50441037c0 0x7f5044074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:49.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:49.222+0000 7f504b8b7640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f504410d8d0 con 0x7f50441037c0 2024-09-11T21:55:49.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:49.222+0000 7f504b8b7640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f504410d750 con 0x7f5044102e00 2024-09-11T21:55:49.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:49.222+0000 7f504b8b7640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f504410da50 con 0x7f5044101c00 2024-09-11T21:55:49.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:49.222+0000 7f5049e54640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f50441037c0 0x7f5044074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:49.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:49.222+0000 7f5049e54640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f50441037c0 0x7f5044074b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39630/0 (socket says 172.21.15.1:39630) 2024-09-11T21:55:49.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:49.222+0000 7f5049e54640 1 -- 172.21.15.1:0/2114923970 learned_addr learned my addr 172.21.15.1:0/2114923970 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:55:49.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:49.222+0000 7f5049653640 1 --2- 172.21.15.1:0/2114923970 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5044101c00 0x7f5044075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:49.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:49.222+0000 7f5048e52640 1 --2- 172.21.15.1:0/2114923970 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5044102e00 0x7f50440745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:49.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:49.223+0000 7f5049e54640 1 -- 172.21.15.1:0/2114923970 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5044101c00 msgr2=0x7f5044075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:49.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:49.223+0000 7f5049e54640 1 --2- 172.21.15.1:0/2114923970 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5044101c00 0x7f5044075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:49.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:49.223+0000 7f5049e54640 1 -- 172.21.15.1:0/2114923970 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5044102e00 msgr2=0x7f50440745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:49.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:49.223+0000 7f5049e54640 1 --2- 172.21.15.1:0/2114923970 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5044102e00 0x7f50440745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:49.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:49.223+0000 7f5049e54640 1 -- 172.21.15.1:0/2114923970 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50440753e0 con 0x7f50441037c0 2024-09-11T21:55:49.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:49.223+0000 7f5048e52640 1 --2- 172.21.15.1:0/2114923970 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5044102e00 0x7f50440745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T21:55:49.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:49.223+0000 7f5049e54640 1 --2- 172.21.15.1:0/2114923970 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f50441037c0 0x7f5044074b20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f503c00eaa0 tx=0x7f503c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:55:49.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:49.223+0000 7f503a7fc640 1 -- 172.21.15.1:0/2114923970 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f503c00cdb0 con 0x7f50441037c0 2024-09-11T21:55:49.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:49.223+0000 7f503a7fc640 1 -- 172.21.15.1:0/2114923970 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f503c005020 con 0x7f50441037c0 2024-09-11T21:55:49.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:49.223+0000 7f504b8b7640 1 -- 172.21.15.1:0/2114923970 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50441a1ff0 con 0x7f50441037c0 2024-09-11T21:55:49.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:49.223+0000 7f503a7fc640 1 -- 172.21.15.1:0/2114923970 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f503c0116d0 con 0x7f50441037c0 2024-09-11T21:55:49.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:49.223+0000 7f504b8b7640 1 -- 172.21.15.1:0/2114923970 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50441a2550 con 0x7f50441037c0 2024-09-11T21:55:49.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:49.224+0000 7f504b8b7640 1 -- 172.21.15.1:0/2114923970 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5044108fb0 con 0x7f50441037c0 2024-09-11T21:55:49.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:49.225+0000 7f503a7fc640 1 -- 172.21.15.1:0/2114923970 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f503c0054d0 con 0x7f50441037c0 2024-09-11T21:55:49.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:49.226+0000 7f503a7fc640 1 --2- 172.21.15.1:0/2114923970 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f50300a2c90 0x7f50300a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:49.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:49.226+0000 7f5049653640 1 --2- 172.21.15.1:0/2114923970 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f50300a2c90 0x7f50300a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:49.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:49.226+0000 7f503a7fc640 1 -- 172.21.15.1:0/2114923970 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f503c0cf1d0 con 0x7f50441037c0 2024-09-11T21:55:49.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:49.227+0000 7f5049653640 1 --2- 172.21.15.1:0/2114923970 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f50300a2c90 0x7f50300a5150 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5044102c60 tx=0x7f504000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:55:49.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:49.231+0000 7f503a7fc640 1 -- 172.21.15.1:0/2114923970 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f503c016030 con 0x7f50441037c0 2024-09-11T21:55:49.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:49 smithi001 ceph-mon[30935]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T21:55:49.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:49.473+0000 7f504b8b7640 1 -- 172.21.15.1:0/2114923970 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f5044062df0 con 0x7f50441037c0 2024-09-11T21:55:49.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:49.475+0000 7f503a7fc640 1 -- 172.21.15.1:0/2114923970 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f503c09c180 con 0x7f50441037c0 2024-09-11T21:55:49.476 INFO:teuthology.orchestra.run.smithi001.stdout:[client.0] 2024-09-11T21:55:49.476 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQDlEeJmiABKHBAA2A3/XTRk1X34EP+uyrnGoA== 2024-09-11T21:55:49.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:49.477+0000 7f504b8b7640 1 -- 172.21.15.1:0/2114923970 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f50300a2c90 msgr2=0x7f50300a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:49.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:49.477+0000 7f504b8b7640 1 --2- 172.21.15.1:0/2114923970 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f50300a2c90 0x7f50300a5150 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5044102c60 tx=0x7f504000a2d0 comp rx=0 tx=0).stop 2024-09-11T21:55:49.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:49.477+0000 7f504b8b7640 1 -- 172.21.15.1:0/2114923970 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f50441037c0 msgr2=0x7f5044074b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:49.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:49.477+0000 7f504b8b7640 1 --2- 172.21.15.1:0/2114923970 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f50441037c0 0x7f5044074b20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f503c00eaa0 tx=0x7f503c00ef60 comp rx=0 tx=0).stop 2024-09-11T21:55:49.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:49.478+0000 7f504b8b7640 1 -- 172.21.15.1:0/2114923970 shutdown_connections 2024-09-11T21:55:49.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:49.478+0000 7f504b8b7640 1 --2- 172.21.15.1:0/2114923970 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f50300a2c90 0x7f50300a5150 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:49.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:49.478+0000 7f504b8b7640 1 --2- 172.21.15.1:0/2114923970 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f50441037c0 0x7f5044074b20 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:49.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:49.478+0000 7f504b8b7640 1 --2- 172.21.15.1:0/2114923970 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5044102e00 0x7f50440745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:49.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:49.478+0000 7f504b8b7640 1 --2- 172.21.15.1:0/2114923970 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5044101c00 0x7f5044075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:49.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:49.478+0000 7f504b8b7640 1 -- 172.21.15.1:0/2114923970 >> 172.21.15.1:0/2114923970 conn(0x7f50440fd390 msgr2=0x7f5044106180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:55:49.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:49.478+0000 7f504b8b7640 1 -- 172.21.15.1:0/2114923970 shutdown_connections 2024-09-11T21:55:49.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:49.478+0000 7f504b8b7640 1 -- 172.21.15.1:0/2114923970 wait complete. 2024-09-11T21:55:49.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:49 smithi053 ceph-mon[34908]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T21:55:49.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:49 smithi161 ceph-mon[35615]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T21:55:50.160 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-11T21:55:50.160 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-11T21:55:50.160 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-11T21:55:50.201 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-11T21:55:50.201 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-11T21:55:50.201 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph mgr dump --format=json 2024-09-11T21:55:50.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:50 smithi001 ceph-mon[30935]: pgmap v296: 97 pgs: 97 active+clean; 630 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 4.0 KiB/s wr, 11 op/s 2024-09-11T21:55:50.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:50 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/2114923970' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T21:55:50.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:50 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/2114923970' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T21:55:50.499 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:55:50.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:50 smithi053 ceph-mon[34908]: pgmap v296: 97 pgs: 97 active+clean; 630 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 4.0 KiB/s wr, 11 op/s 2024-09-11T21:55:50.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:50 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/2114923970' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T21:55:50.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:50 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/2114923970' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T21:55:50.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:50 smithi161 ceph-mon[35615]: pgmap v296: 97 pgs: 97 active+clean; 630 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 4.0 KiB/s wr, 11 op/s 2024-09-11T21:55:50.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:50 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/2114923970' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T21:55:50.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:50 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/2114923970' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T21:55:50.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:50.970+0000 7f1b028dd640 1 -- 172.21.15.1:0/3287868904 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1afc101420 msgr2=0x7f1afc10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:50.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:50.970+0000 7f1b028dd640 1 --2- 172.21.15.1:0/3287868904 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1afc101420 0x7f1afc10d950 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1af4009cd0 tx=0x7f1af402a8e0 comp rx=0 tx=0).stop 2024-09-11T21:55:50.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:50.970+0000 7f1b028dd640 1 -- 172.21.15.1:0/3287868904 shutdown_connections 2024-09-11T21:55:50.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:50.970+0000 7f1b028dd640 1 --2- 172.21.15.1:0/3287868904 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1afc101420 0x7f1afc10d950 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:50.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:50.970+0000 7f1b028dd640 1 --2- 172.21.15.1:0/3287868904 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1afc100a80 0x7f1afc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:50.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:50.970+0000 7f1b028dd640 1 --2- 172.21.15.1:0/3287868904 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1afc106a80 0x7f1afc106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:50.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:50.970+0000 7f1b028dd640 1 -- 172.21.15.1:0/3287868904 >> 172.21.15.1:0/3287868904 conn(0x7f1afc0fc7a0 msgr2=0x7f1afc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:55:50.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:50.971+0000 7f1b028dd640 1 -- 172.21.15.1:0/3287868904 shutdown_connections 2024-09-11T21:55:50.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:50.971+0000 7f1b028dd640 1 -- 172.21.15.1:0/3287868904 wait complete. 2024-09-11T21:55:50.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:50.971+0000 7f1b028dd640 1 Processor -- start 2024-09-11T21:55:50.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:50.971+0000 7f1b028dd640 1 -- start start 2024-09-11T21:55:50.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:50.971+0000 7f1b028dd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1afc100a80 0x7f1afc198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:50.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:50.971+0000 7f1b028dd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1afc101420 0x7f1afc1988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:50.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:50.971+0000 7f1b028dd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1afc106a80 0x7f1afc19cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:50.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:50.971+0000 7f1b028dd640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1afc111fe0 con 0x7f1afc106a80 2024-09-11T21:55:50.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:50.971+0000 7f1b028dd640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1afc111e60 con 0x7f1afc101420 2024-09-11T21:55:50.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:50.971+0000 7f1b028dd640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1afc112160 con 0x7f1afc100a80 2024-09-11T21:55:50.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:50.971+0000 7f1b00e7a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1afc106a80 0x7f1afc19cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:50.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:50.971+0000 7f1b00e7a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1afc106a80 0x7f1afc19cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39634/0 (socket says 172.21.15.1:39634) 2024-09-11T21:55:50.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:50.972+0000 7f1b00e7a640 1 -- 172.21.15.1:0/3118716664 learned_addr learned my addr 172.21.15.1:0/3118716664 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:55:50.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:50.972+0000 7f1afbfff640 1 --2- 172.21.15.1:0/3118716664 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1afc100a80 0x7f1afc198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:50.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:50.972+0000 7f1af3fff640 1 --2- 172.21.15.1:0/3118716664 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1afc101420 0x7f1afc1988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:50.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:50.972+0000 7f1b00e7a640 1 -- 172.21.15.1:0/3118716664 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1afc100a80 msgr2=0x7f1afc198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:50.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:50.972+0000 7f1b00e7a640 1 --2- 172.21.15.1:0/3118716664 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1afc100a80 0x7f1afc198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:50.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:50.972+0000 7f1b00e7a640 1 -- 172.21.15.1:0/3118716664 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1afc101420 msgr2=0x7f1afc1988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:50.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:50.972+0000 7f1b00e7a640 1 --2- 172.21.15.1:0/3118716664 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1afc101420 0x7f1afc1988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:50.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:50.972+0000 7f1b00e7a640 1 -- 172.21.15.1:0/3118716664 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1afc19d330 con 0x7f1afc106a80 2024-09-11T21:55:50.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:50.972+0000 7f1b00e7a640 1 --2- 172.21.15.1:0/3118716664 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1afc106a80 0x7f1afc19cc50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1af4009ca0 tx=0x7f1af4003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:55:50.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:50.972+0000 7f1af9ffb640 1 -- 172.21.15.1:0/3118716664 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af4004240 con 0x7f1afc106a80 2024-09-11T21:55:50.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:50.972+0000 7f1af9ffb640 1 -- 172.21.15.1:0/3118716664 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1af40305a0 con 0x7f1afc106a80 2024-09-11T21:55:50.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:50.972+0000 7f1b028dd640 1 -- 172.21.15.1:0/3118716664 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1afc19d5c0 con 0x7f1afc106a80 2024-09-11T21:55:50.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:50.972+0000 7f1af9ffb640 1 -- 172.21.15.1:0/3118716664 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af403a660 con 0x7f1afc106a80 2024-09-11T21:55:50.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:50.972+0000 7f1b028dd640 1 -- 172.21.15.1:0/3118716664 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1afc1a4f00 con 0x7f1afc106a80 2024-09-11T21:55:50.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:50.973+0000 7f1b028dd640 1 -- 172.21.15.1:0/3118716664 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1afc100ee0 con 0x7f1afc106a80 2024-09-11T21:55:50.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:50.974+0000 7f1af9ffb640 1 -- 172.21.15.1:0/3118716664 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f1af4030750 con 0x7f1afc106a80 2024-09-11T21:55:50.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:50.975+0000 7f1af9ffb640 1 --2- 172.21.15.1:0/3118716664 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f1ae80a2c10 0x7f1ae80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:50.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:50.975+0000 7f1afbfff640 1 --2- 172.21.15.1:0/3118716664 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f1ae80a2c10 0x7f1ae80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:50.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:50.975+0000 7f1af9ffb640 1 -- 172.21.15.1:0/3118716664 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f1af402e030 con 0x7f1afc106a80 2024-09-11T21:55:50.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:50.975+0000 7f1afbfff640 1 --2- 172.21.15.1:0/3118716664 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f1ae80a2c10 0x7f1ae80a50d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1aec00cac0 tx=0x7f1aec00c4a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:55:50.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:50.979+0000 7f1af9ffb640 1 -- 172.21.15.1:0/3118716664 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1af40098b0 con 0x7f1afc106a80 2024-09-11T21:55:51.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:51.205+0000 7f1b028dd640 1 -- 172.21.15.1:0/3118716664 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f1afc1a50d0 con 0x7f1afc106a80 2024-09-11T21:55:51.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:51.209+0000 7f1af9ffb640 1 -- 172.21.15.1:0/3118716664 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263192 (secure 0 0 0) 0x7f1af40bd2d0 con 0x7f1afc106a80 2024-09-11T21:55:51.213 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:55:51.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:51.213+0000 7f1b028dd640 1 -- 172.21.15.1:0/3118716664 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f1ae80a2c10 msgr2=0x7f1ae80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:51.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:51.213+0000 7f1b028dd640 1 --2- 172.21.15.1:0/3118716664 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f1ae80a2c10 0x7f1ae80a50d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1aec00cac0 tx=0x7f1aec00c4a0 comp rx=0 tx=0).stop 2024-09-11T21:55:51.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:51.213+0000 7f1b028dd640 1 -- 172.21.15.1:0/3118716664 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1afc106a80 msgr2=0x7f1afc19cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:51.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:51.213+0000 7f1b028dd640 1 --2- 172.21.15.1:0/3118716664 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1afc106a80 0x7f1afc19cc50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1af4009ca0 tx=0x7f1af4003ba0 comp rx=0 tx=0).stop 2024-09-11T21:55:51.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:51.213+0000 7f1b028dd640 1 -- 172.21.15.1:0/3118716664 shutdown_connections 2024-09-11T21:55:51.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:51.213+0000 7f1b028dd640 1 --2- 172.21.15.1:0/3118716664 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f1ae80a2c10 0x7f1ae80a50d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:51.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:51.213+0000 7f1b028dd640 1 --2- 172.21.15.1:0/3118716664 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1afc106a80 0x7f1afc19cc50 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:51.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:51.213+0000 7f1b028dd640 1 --2- 172.21.15.1:0/3118716664 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1afc101420 0x7f1afc1988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:51.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:51.213+0000 7f1b028dd640 1 --2- 172.21.15.1:0/3118716664 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1afc100a80 0x7f1afc198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:51.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:51.213+0000 7f1b028dd640 1 -- 172.21.15.1:0/3118716664 >> 172.21.15.1:0/3118716664 conn(0x7f1afc0fc7a0 msgr2=0x7f1afc0fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:55:51.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:51.213+0000 7f1b028dd640 1 -- 172.21.15.1:0/3118716664 shutdown_connections 2024-09-11T21:55:51.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:51.213+0000 7f1b028dd640 1 -- 172.21.15.1:0/3118716664 wait complete. 2024-09-11T21:55:52.042 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":3890350886},{"type":"v1","addr":"172.21.15.1:6801","nonce":3890350886}]},"active_addr":"172.21.15.1:6801/3890350886","active_change":"2024-09-11T21:48:24.525022+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24106,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.1:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.1:0","nonce":4286250488}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.1:0","nonce":1202232270}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.1:0","nonce":2797780151}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.1:0","nonce":2039944931}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.1:0","nonce":1380502135}]}]} 2024-09-11T21:55:52.048 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-11T21:55:52.048 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-11T21:55:52.048 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T21:55:52.316 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:55:52.438 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:52 smithi001 ceph-mon[30935]: pgmap v297: 97 pgs: 97 active+clean; 638 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 4.6 KiB/s wr, 12 op/s 2024-09-11T21:55:52.438 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:52 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/3118716664' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-11T21:55:52.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:52 smithi053 ceph-mon[34908]: pgmap v297: 97 pgs: 97 active+clean; 638 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 4.6 KiB/s wr, 12 op/s 2024-09-11T21:55:52.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:52 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/3118716664' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-11T21:55:52.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:52 smithi161 ceph-mon[35615]: pgmap v297: 97 pgs: 97 active+clean; 638 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 4.6 KiB/s wr, 12 op/s 2024-09-11T21:55:52.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:52 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/3118716664' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-11T21:55:52.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:52.841+0000 7fd549f75640 1 -- 172.21.15.1:0/19163119 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd544102570 msgr2=0x7fd5441029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:52.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:52.841+0000 7fd549f75640 1 --2- 172.21.15.1:0/19163119 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd544102570 0x7fd5441029b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd538009900 tx=0x7fd53802a480 comp rx=0 tx=0).stop 2024-09-11T21:55:52.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:52.841+0000 7fd549f75640 1 -- 172.21.15.1:0/19163119 shutdown_connections 2024-09-11T21:55:52.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:52.841+0000 7fd549f75640 1 --2- 172.21.15.1:0/19163119 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd544102ef0 0x7fd54410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:52.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:52.841+0000 7fd549f75640 1 --2- 172.21.15.1:0/19163119 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd544102570 0x7fd5441029b0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:52.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:52.841+0000 7fd549f75640 1 --2- 172.21.15.1:0/19163119 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd544100e20 0x7fd544101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:52.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:52.841+0000 7fd549f75640 1 -- 172.21.15.1:0/19163119 >> 172.21.15.1:0/19163119 conn(0x7fd5440fc7d0 msgr2=0x7fd5440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:55:52.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:52.841+0000 7fd549f75640 1 -- 172.21.15.1:0/19163119 shutdown_connections 2024-09-11T21:55:52.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:52.841+0000 7fd549f75640 1 -- 172.21.15.1:0/19163119 wait complete. 2024-09-11T21:55:52.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:52.842+0000 7fd549f75640 1 Processor -- start 2024-09-11T21:55:52.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:52.842+0000 7fd549f75640 1 -- start start 2024-09-11T21:55:52.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:52.842+0000 7fd549f75640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd544100e20 0x7fd544199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:52.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:52.842+0000 7fd549f75640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd544102570 0x7fd544199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:52.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:52.842+0000 7fd549f75640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd544102ef0 0x7fd5441a4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:52.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:52.842+0000 7fd549f75640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd544110f80 con 0x7fd544102ef0 2024-09-11T21:55:52.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:52.842+0000 7fd549f75640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd544110e00 con 0x7fd544102570 2024-09-11T21:55:52.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:52.842+0000 7fd549f75640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd544111100 con 0x7fd544100e20 2024-09-11T21:55:52.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:52.842+0000 7fd543fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd544102ef0 0x7fd5441a4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:52.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:52.842+0000 7fd543fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd544102ef0 0x7fd5441a4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39652/0 (socket says 172.21.15.1:39652) 2024-09-11T21:55:52.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:52.842+0000 7fd543fff640 1 -- 172.21.15.1:0/1993359078 learned_addr learned my addr 172.21.15.1:0/1993359078 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:55:52.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:52.842+0000 7fd5437fe640 1 --2- 172.21.15.1:0/1993359078 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd544100e20 0x7fd544199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:52.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:52.843+0000 7fd542ffd640 1 --2- 172.21.15.1:0/1993359078 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd544102570 0x7fd544199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:52.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:52.843+0000 7fd543fff640 1 -- 172.21.15.1:0/1993359078 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd544100e20 msgr2=0x7fd544199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:52.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:52.843+0000 7fd543fff640 1 --2- 172.21.15.1:0/1993359078 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd544100e20 0x7fd544199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:52.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:52.843+0000 7fd543fff640 1 -- 172.21.15.1:0/1993359078 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd544102570 msgr2=0x7fd544199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:52.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:52.843+0000 7fd543fff640 1 --2- 172.21.15.1:0/1993359078 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd544102570 0x7fd544199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:52.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:52.843+0000 7fd543fff640 1 -- 172.21.15.1:0/1993359078 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5441a48d0 con 0x7fd544102ef0 2024-09-11T21:55:52.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:52.843+0000 7fd543fff640 1 --2- 172.21.15.1:0/1993359078 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd544102ef0 0x7fd5441a4390 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd53c00eaa0 tx=0x7fd53c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:55:52.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:52.843+0000 7fd540ff9640 1 -- 172.21.15.1:0/1993359078 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd53c00cdb0 con 0x7fd544102ef0 2024-09-11T21:55:52.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:52.843+0000 7fd540ff9640 1 -- 172.21.15.1:0/1993359078 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd53c005020 con 0x7fd544102ef0 2024-09-11T21:55:52.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:52.843+0000 7fd549f75640 1 -- 172.21.15.1:0/1993359078 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5441a4aa0 con 0x7fd544102ef0 2024-09-11T21:55:52.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:52.843+0000 7fd540ff9640 1 -- 172.21.15.1:0/1993359078 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd53c0116b0 con 0x7fd544102ef0 2024-09-11T21:55:52.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:52.843+0000 7fd549f75640 1 -- 172.21.15.1:0/1993359078 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5441a4fe0 con 0x7fd544102ef0 2024-09-11T21:55:52.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:52.844+0000 7fd549f75640 1 -- 172.21.15.1:0/1993359078 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd544101200 con 0x7fd544102ef0 2024-09-11T21:55:52.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:52.845+0000 7fd540ff9640 1 -- 172.21.15.1:0/1993359078 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fd53c005560 con 0x7fd544102ef0 2024-09-11T21:55:52.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:52.846+0000 7fd540ff9640 1 --2- 172.21.15.1:0/1993359078 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fd52c0a2c10 0x7fd52c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:52.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:52.846+0000 7fd5437fe640 1 --2- 172.21.15.1:0/1993359078 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fd52c0a2c10 0x7fd52c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:52.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:52.846+0000 7fd540ff9640 1 -- 172.21.15.1:0/1993359078 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7fd53c0cf200 con 0x7fd544102ef0 2024-09-11T21:55:52.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:52.847+0000 7fd5437fe640 1 --2- 172.21.15.1:0/1993359078 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fd52c0a2c10 0x7fd52c0a50d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd5340058c0 tx=0x7fd53400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:55:52.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:52.850+0000 7fd540ff9640 1 -- 172.21.15.1:0/1993359078 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd53c016030 con 0x7fd544102ef0 2024-09-11T21:55:53.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:53.025+0000 7fd549f75640 1 -- 172.21.15.1:0/1993359078 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd5441a52a0 con 0x7fd544102ef0 2024-09-11T21:55:53.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:53.027+0000 7fd540ff9640 1 -- 172.21.15.1:0/1993359078 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+22198 (secure 0 0 0) 0x7fd53c09c1b0 con 0x7fd544102ef0 2024-09-11T21:55:53.028 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:55:53.028 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":84,"fsid":"3ffca83c-7087-11ef-bcea-c7b262605968","created":"2024-09-11T21:47:28.868707+0000","modified":"2024-09-11T21:55:20.791073+0000","last_up_change":"2024-09-11T21:54:29.319441+0000","last_in_change":"2024-09-11T21:54:05.532281+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T21:50:41.026490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T21:55:16.961999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T21:55:19.066228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"19e56504-18f1-4234-b2d6-f193ff47d8eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":4012142023},{"type":"v1","addr":"172.21.15.1:6803","nonce":4012142023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":4012142023},{"type":"v1","addr":"172.21.15.1:6805","nonce":4012142023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":4012142023},{"type":"v1","addr":"172.21.15.1:6809","nonce":4012142023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":4012142023},{"type":"v1","addr":"172.21.15.1:6807","nonce":4012142023}]},"public_addr":"172.21.15.1:6803/4012142023","cluster_addr":"172.21.15.1:6805/4012142023","heartbeat_back_addr":"172.21.15.1:6809/4012142023","heartbeat_front_addr":"172.21.15.1:6807/4012142023","state":["exists","up"]},{"osd":1,"uuid":"996c9f0c-c5a4-43b8-b8e0-d712f27c5dfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3039391924},{"type":"v1","addr":"172.21.15.53:6801","nonce":3039391924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3039391924},{"type":"v1","addr":"172.21.15.53:6803","nonce":3039391924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3039391924},{"type":"v1","addr":"172.21.15.53:6807","nonce":3039391924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3039391924},{"type":"v1","addr":"172.21.15.53:6805","nonce":3039391924}]},"public_addr":"172.21.15.53:6801/3039391924","cluster_addr":"172.21.15.53:6803/3039391924","heartbeat_back_addr":"172.21.15.53:6807/3039391924","heartbeat_front_addr":"172.21.15.53:6805/3039391924","state":["exists","up"]},{"osd":2,"uuid":"578d4260-41e1-4645-b9b1-1c5445712c13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1707914885},{"type":"v1","addr":"172.21.15.161:6801","nonce":1707914885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1707914885},{"type":"v1","addr":"172.21.15.161:6803","nonce":1707914885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1707914885},{"type":"v1","addr":"172.21.15.161:6807","nonce":1707914885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1707914885},{"type":"v1","addr":"172.21.15.161:6805","nonce":1707914885}]},"public_addr":"172.21.15.161:6801/1707914885","cluster_addr":"172.21.15.161:6803/1707914885","heartbeat_back_addr":"172.21.15.161:6807/1707914885","heartbeat_front_addr":"172.21.15.161:6805/1707914885","state":["exists","up"]},{"osd":3,"uuid":"73f78766-9662-4aa3-841c-fe2ce654ee9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1541142398},{"type":"v1","addr":"172.21.15.1:6811","nonce":1541142398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1541142398},{"type":"v1","addr":"172.21.15.1:6813","nonce":1541142398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1541142398},{"type":"v1","addr":"172.21.15.1:6817","nonce":1541142398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1541142398},{"type":"v1","addr":"172.21.15.1:6815","nonce":1541142398}]},"public_addr":"172.21.15.1:6811/1541142398","cluster_addr":"172.21.15.1:6813/1541142398","heartbeat_back_addr":"172.21.15.1:6817/1541142398","heartbeat_front_addr":"172.21.15.1:6815/1541142398","state":["exists","up"]},{"osd":4,"uuid":"56c808bb-d4cd-4969-b49e-adef49ecd0f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":3081402681},{"type":"v1","addr":"172.21.15.53:6809","nonce":3081402681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3081402681},{"type":"v1","addr":"172.21.15.53:6811","nonce":3081402681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3081402681},{"type":"v1","addr":"172.21.15.53:6815","nonce":3081402681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3081402681},{"type":"v1","addr":"172.21.15.53:6813","nonce":3081402681}]},"public_addr":"172.21.15.53:6809/3081402681","cluster_addr":"172.21.15.53:6811/3081402681","heartbeat_back_addr":"172.21.15.53:6815/3081402681","heartbeat_front_addr":"172.21.15.53:6813/3081402681","state":["exists","up"]},{"osd":5,"uuid":"7b8266d6-b507-47c9-b4fa-ff16c4a74d35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":1622081327},{"type":"v1","addr":"172.21.15.161:6809","nonce":1622081327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":1622081327},{"type":"v1","addr":"172.21.15.161:6811","nonce":1622081327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":1622081327},{"type":"v1","addr":"172.21.15.161:6815","nonce":1622081327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":1622081327},{"type":"v1","addr":"172.21.15.161:6813","nonce":1622081327}]},"public_addr":"172.21.15.161:6809/1622081327","cluster_addr":"172.21.15.161:6811/1622081327","heartbeat_back_addr":"172.21.15.161:6815/1622081327","heartbeat_front_addr":"172.21.15.161:6813/1622081327","state":["exists","up"]},{"osd":6,"uuid":"3af5925d-a749-4849-a076-80580222cad4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":158434853},{"type":"v1","addr":"172.21.15.1:6819","nonce":158434853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":158434853},{"type":"v1","addr":"172.21.15.1:6821","nonce":158434853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":158434853},{"type":"v1","addr":"172.21.15.1:6825","nonce":158434853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":158434853},{"type":"v1","addr":"172.21.15.1:6823","nonce":158434853}]},"public_addr":"172.21.15.1:6819/158434853","cluster_addr":"172.21.15.1:6821/158434853","heartbeat_back_addr":"172.21.15.1:6825/158434853","heartbeat_front_addr":"172.21.15.1:6823/158434853","state":["exists","up"]},{"osd":7,"uuid":"f8f26c3c-17e9-4e89-8392-289796e0c8a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2132057697},{"type":"v1","addr":"172.21.15.53:6817","nonce":2132057697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2132057697},{"type":"v1","addr":"172.21.15.53:6819","nonce":2132057697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2132057697},{"type":"v1","addr":"172.21.15.53:6823","nonce":2132057697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2132057697},{"type":"v1","addr":"172.21.15.53:6821","nonce":2132057697}]},"public_addr":"172.21.15.53:6817/2132057697","cluster_addr":"172.21.15.53:6819/2132057697","heartbeat_back_addr":"172.21.15.53:6823/2132057697","heartbeat_front_addr":"172.21.15.53:6821/2132057697","state":["exists","up"]},{"osd":8,"uuid":"36ea401b-3c93-4b9d-8880-298d1eb6e30b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1838248669},{"type":"v1","addr":"172.21.15.161:6817","nonce":1838248669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1838248669},{"type":"v1","addr":"172.21.15.161:6819","nonce":1838248669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1838248669},{"type":"v1","addr":"172.21.15.161:6823","nonce":1838248669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1838248669},{"type":"v1","addr":"172.21.15.161:6821","nonce":1838248669}]},"public_addr":"172.21.15.161:6817/1838248669","cluster_addr":"172.21.15.161:6819/1838248669","heartbeat_back_addr":"172.21.15.161:6823/1838248669","heartbeat_front_addr":"172.21.15.161:6821/1838248669","state":["exists","up"]},{"osd":9,"uuid":"4b788d3f-a3bc-4463-bd99-95fdb7a52768","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2806274589},{"type":"v1","addr":"172.21.15.1:6827","nonce":2806274589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2806274589},{"type":"v1","addr":"172.21.15.1:6829","nonce":2806274589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":2806274589},{"type":"v1","addr":"172.21.15.1:6833","nonce":2806274589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2806274589},{"type":"v1","addr":"172.21.15.1:6831","nonce":2806274589}]},"public_addr":"172.21.15.1:6827/2806274589","cluster_addr":"172.21.15.1:6829/2806274589","heartbeat_back_addr":"172.21.15.1:6833/2806274589","heartbeat_front_addr":"172.21.15.1:6831/2806274589","state":["exists","up"]},{"osd":10,"uuid":"70549796-5270-4ae2-ac3f-f61f5de105d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":761923580},{"type":"v1","addr":"172.21.15.53:6825","nonce":761923580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":761923580},{"type":"v1","addr":"172.21.15.53:6827","nonce":761923580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":761923580},{"type":"v1","addr":"172.21.15.53:6831","nonce":761923580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":761923580},{"type":"v1","addr":"172.21.15.53:6829","nonce":761923580}]},"public_addr":"172.21.15.53:6825/761923580","cluster_addr":"172.21.15.53:6827/761923580","heartbeat_back_addr":"172.21.15.53:6831/761923580","heartbeat_front_addr":"172.21.15.53:6829/761923580","state":["exists","up"]},{"osd":11,"uuid":"48f17c4a-370a-493e-956e-bf6ad3dcc594","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1586352190},{"type":"v1","addr":"172.21.15.161:6825","nonce":1586352190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":1586352190},{"type":"v1","addr":"172.21.15.161:6827","nonce":1586352190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":1586352190},{"type":"v1","addr":"172.21.15.161:6831","nonce":1586352190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":1586352190},{"type":"v1","addr":"172.21.15.161:6829","nonce":1586352190}]},"public_addr":"172.21.15.161:6825/1586352190","cluster_addr":"172.21.15.161:6827/1586352190","heartbeat_back_addr":"172.21.15.161:6831/1586352190","heartbeat_front_addr":"172.21.15.161:6829/1586352190","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:49:46.539613+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:50:11.912300+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:50:36.955490+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:51:02.200554+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:51:27.860585+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:51:54.694099+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:52:18.962351+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:52:44.881923+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:53:10.301390+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:53:35.417647+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:54:01.492219+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:54:26.938338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/1749020850":"2024-09-12T21:48:24.524913+0000","172.21.15.1:6801/2821761445":"2024-09-12T21:48:24.524913+0000","172.21.15.1:0/1699984777":"2024-09-12T21:48:24.524913+0000","172.21.15.1:0/276770268":"2024-09-12T21:48:00.674728+0000","172.21.15.1:0/1278877732":"2024-09-12T21:48:24.524913+0000","172.21.15.1:0/2599895977":"2024-09-12T21:48:00.674728+0000","172.21.15.1:6801/2311309446":"2024-09-12T21:48:00.674728+0000","172.21.15.1:6800/2821761445":"2024-09-12T21:48:24.524913+0000","172.21.15.1:6800/2311309446":"2024-09-12T21:48:00.674728+0000","172.21.15.1:0/3202962495":"2024-09-12T21:48:00.674728+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T21:55:53.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:53.029+0000 7fd549f75640 1 -- 172.21.15.1:0/1993359078 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fd52c0a2c10 msgr2=0x7fd52c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:53.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:53.029+0000 7fd549f75640 1 --2- 172.21.15.1:0/1993359078 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fd52c0a2c10 0x7fd52c0a50d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd5340058c0 tx=0x7fd53400a2d0 comp rx=0 tx=0).stop 2024-09-11T21:55:53.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:53.029+0000 7fd549f75640 1 -- 172.21.15.1:0/1993359078 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd544102ef0 msgr2=0x7fd5441a4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:53.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:53.029+0000 7fd549f75640 1 --2- 172.21.15.1:0/1993359078 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd544102ef0 0x7fd5441a4390 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd53c00eaa0 tx=0x7fd53c00ef60 comp rx=0 tx=0).stop 2024-09-11T21:55:53.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:53.029+0000 7fd549f75640 1 -- 172.21.15.1:0/1993359078 shutdown_connections 2024-09-11T21:55:53.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:53.029+0000 7fd549f75640 1 --2- 172.21.15.1:0/1993359078 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fd52c0a2c10 0x7fd52c0a50d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:53.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:53.029+0000 7fd549f75640 1 --2- 172.21.15.1:0/1993359078 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd544102ef0 0x7fd5441a4390 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:53.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:53.029+0000 7fd549f75640 1 --2- 172.21.15.1:0/1993359078 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd544102570 0x7fd544199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:53.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:53.029+0000 7fd549f75640 1 --2- 172.21.15.1:0/1993359078 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd544100e20 0x7fd544199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:53.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:53.029+0000 7fd549f75640 1 -- 172.21.15.1:0/1993359078 >> 172.21.15.1:0/1993359078 conn(0x7fd5440fc7d0 msgr2=0x7fd5441068a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:55:53.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:53.029+0000 7fd549f75640 1 -- 172.21.15.1:0/1993359078 shutdown_connections 2024-09-11T21:55:53.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:53.029+0000 7fd549f75640 1 -- 172.21.15.1:0/1993359078 wait complete. 2024-09-11T21:55:53.397 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:53 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/1993359078' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T21:55:53.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:53 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/1993359078' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T21:55:53.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:53 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/1993359078' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T21:55:53.735 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-11T21:55:53.735 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T21:55:54.000 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:55:54.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:54 smithi001 ceph-mon[30935]: pgmap v298: 97 pgs: 97 active+clean; 638 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-09-11T21:55:54.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:54.510+0000 7f2008ea7640 1 -- 172.21.15.1:0/1036092301 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2004102790 msgr2=0x7f20041067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:54.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:54.510+0000 7f2008ea7640 1 --2- 172.21.15.1:0/1036092301 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2004102790 0x7f20041067e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1fec009900 tx=0x7f1fec02a410 comp rx=0 tx=0).stop 2024-09-11T21:55:54.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:54.510+0000 7f2008ea7640 1 -- 172.21.15.1:0/1036092301 shutdown_connections 2024-09-11T21:55:54.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:54.510+0000 7f2008ea7640 1 --2- 172.21.15.1:0/1036092301 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2004106f10 0x7f200410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:54.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:54.510+0000 7f2008ea7640 1 --2- 172.21.15.1:0/1036092301 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2004102790 0x7f20041067e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:54.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:54.510+0000 7f2008ea7640 1 --2- 172.21.15.1:0/1036092301 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2004101de0 0x7f20041021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:54.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:54.510+0000 7f2008ea7640 1 -- 172.21.15.1:0/1036092301 >> 172.21.15.1:0/1036092301 conn(0x7f20040fd8c0 msgr2=0x7f20040ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:55:54.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:54.510+0000 7f2008ea7640 1 -- 172.21.15.1:0/1036092301 shutdown_connections 2024-09-11T21:55:54.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:54.510+0000 7f2008ea7640 1 -- 172.21.15.1:0/1036092301 wait complete. 2024-09-11T21:55:54.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:54.511+0000 7f2008ea7640 1 Processor -- start 2024-09-11T21:55:54.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:54.511+0000 7f2008ea7640 1 -- start start 2024-09-11T21:55:54.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:54.511+0000 7f2008ea7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2004101de0 0x7f20041984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:54.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:54.511+0000 7f2008ea7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2004102790 0x7f2004198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:54.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:54.511+0000 7f2008ea7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2004106f10 0x7f200419cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:54.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:54.511+0000 7f2008ea7640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2004111f70 con 0x7f2004102790 2024-09-11T21:55:54.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:54.511+0000 7f2008ea7640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2004111df0 con 0x7f2004101de0 2024-09-11T21:55:54.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:54.511+0000 7f2008ea7640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f20041120f0 con 0x7f2004106f10 2024-09-11T21:55:54.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:54.511+0000 7f1ff9d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2004102790 0x7f2004198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:54.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:54.511+0000 7f1ff9d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2004102790 0x7f2004198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39670/0 (socket says 172.21.15.1:39670) 2024-09-11T21:55:54.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:54.511+0000 7f1ff9d9b640 1 -- 172.21.15.1:0/783498950 learned_addr learned my addr 172.21.15.1:0/783498950 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:55:54.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:54.511+0000 7f200259c640 1 --2- 172.21.15.1:0/783498950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2004101de0 0x7f20041984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:54.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:54.511+0000 7f2002d9d640 1 --2- 172.21.15.1:0/783498950 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2004106f10 0x7f200419cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:54.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:54.512+0000 7f1ff9d9b640 1 -- 172.21.15.1:0/783498950 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2004106f10 msgr2=0x7f200419cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:54.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:54.512+0000 7f1ff9d9b640 1 --2- 172.21.15.1:0/783498950 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2004106f10 0x7f200419cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:54.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:54.512+0000 7f1ff9d9b640 1 -- 172.21.15.1:0/783498950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2004101de0 msgr2=0x7f20041984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:54.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:54.512+0000 7f1ff9d9b640 1 --2- 172.21.15.1:0/783498950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2004101de0 0x7f20041984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:54.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:54.512+0000 7f1ff9d9b640 1 -- 172.21.15.1:0/783498950 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f200419d480 con 0x7f2004102790 2024-09-11T21:55:54.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:54.512+0000 7f2002d9d640 1 --2- 172.21.15.1:0/783498950 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2004106f10 0x7f200419cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T21:55:54.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:54.512+0000 7f1ff9d9b640 1 --2- 172.21.15.1:0/783498950 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2004102790 0x7f2004198a10 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1fec006010 tx=0x7f1fec0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:55:54.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:54.512+0000 7f1ffbfff640 1 -- 172.21.15.1:0/783498950 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fec03f030 con 0x7f2004102790 2024-09-11T21:55:54.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:54.512+0000 7f2008ea7640 1 -- 172.21.15.1:0/783498950 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f200419d710 con 0x7f2004102790 2024-09-11T21:55:54.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:54.512+0000 7f1ffbfff640 1 -- 172.21.15.1:0/783498950 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fec003ce0 con 0x7f2004102790 2024-09-11T21:55:54.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:54.512+0000 7f1ffbfff640 1 -- 172.21.15.1:0/783498950 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fec035070 con 0x7f2004102790 2024-09-11T21:55:54.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:54.512+0000 7f2008ea7640 1 -- 172.21.15.1:0/783498950 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20041a5050 con 0x7f2004102790 2024-09-11T21:55:54.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:54.513+0000 7f2008ea7640 1 -- 172.21.15.1:0/783498950 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20041021c0 con 0x7f2004102790 2024-09-11T21:55:54.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:54.514+0000 7f1ffbfff640 1 -- 172.21.15.1:0/783498950 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f1fec042030 con 0x7f2004102790 2024-09-11T21:55:54.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:54.515+0000 7f1ffbfff640 1 --2- 172.21.15.1:0/783498950 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f1ff00a2c10 0x7f1ff00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:54.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:54.515+0000 7f1ffbfff640 1 -- 172.21.15.1:0/783498950 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f1fec0ef980 con 0x7f2004102790 2024-09-11T21:55:54.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:54.515+0000 7f200259c640 1 --2- 172.21.15.1:0/783498950 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f1ff00a2c10 0x7f1ff00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:54.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:54.516+0000 7f200259c640 1 --2- 172.21.15.1:0/783498950 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f1ff00a2c10 0x7f1ff00a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1ff400bbe0 tx=0x7f1ff400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:55:54.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:54.518+0000 7f1ffbfff640 1 -- 172.21.15.1:0/783498950 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1fec0bc9f0 con 0x7f2004102790 2024-09-11T21:55:54.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:54 smithi053 ceph-mon[34908]: pgmap v298: 97 pgs: 97 active+clean; 638 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-09-11T21:55:54.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:54 smithi161 ceph-mon[35615]: pgmap v298: 97 pgs: 97 active+clean; 638 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-09-11T21:55:54.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:54.695+0000 7f2008ea7640 1 -- 172.21.15.1:0/783498950 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f200410a150 con 0x7f2004102790 2024-09-11T21:55:54.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:54.696+0000 7f1ffbfff640 1 -- 172.21.15.1:0/783498950 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+22198 (secure 0 0 0) 0x7f1fec0bc2f0 con 0x7f2004102790 2024-09-11T21:55:54.697 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:55:54.697 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":84,"fsid":"3ffca83c-7087-11ef-bcea-c7b262605968","created":"2024-09-11T21:47:28.868707+0000","modified":"2024-09-11T21:55:20.791073+0000","last_up_change":"2024-09-11T21:54:29.319441+0000","last_in_change":"2024-09-11T21:54:05.532281+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T21:50:41.026490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T21:55:16.961999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T21:55:19.066228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"19e56504-18f1-4234-b2d6-f193ff47d8eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":4012142023},{"type":"v1","addr":"172.21.15.1:6803","nonce":4012142023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":4012142023},{"type":"v1","addr":"172.21.15.1:6805","nonce":4012142023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":4012142023},{"type":"v1","addr":"172.21.15.1:6809","nonce":4012142023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":4012142023},{"type":"v1","addr":"172.21.15.1:6807","nonce":4012142023}]},"public_addr":"172.21.15.1:6803/4012142023","cluster_addr":"172.21.15.1:6805/4012142023","heartbeat_back_addr":"172.21.15.1:6809/4012142023","heartbeat_front_addr":"172.21.15.1:6807/4012142023","state":["exists","up"]},{"osd":1,"uuid":"996c9f0c-c5a4-43b8-b8e0-d712f27c5dfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3039391924},{"type":"v1","addr":"172.21.15.53:6801","nonce":3039391924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3039391924},{"type":"v1","addr":"172.21.15.53:6803","nonce":3039391924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3039391924},{"type":"v1","addr":"172.21.15.53:6807","nonce":3039391924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3039391924},{"type":"v1","addr":"172.21.15.53:6805","nonce":3039391924}]},"public_addr":"172.21.15.53:6801/3039391924","cluster_addr":"172.21.15.53:6803/3039391924","heartbeat_back_addr":"172.21.15.53:6807/3039391924","heartbeat_front_addr":"172.21.15.53:6805/3039391924","state":["exists","up"]},{"osd":2,"uuid":"578d4260-41e1-4645-b9b1-1c5445712c13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1707914885},{"type":"v1","addr":"172.21.15.161:6801","nonce":1707914885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1707914885},{"type":"v1","addr":"172.21.15.161:6803","nonce":1707914885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1707914885},{"type":"v1","addr":"172.21.15.161:6807","nonce":1707914885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1707914885},{"type":"v1","addr":"172.21.15.161:6805","nonce":1707914885}]},"public_addr":"172.21.15.161:6801/1707914885","cluster_addr":"172.21.15.161:6803/1707914885","heartbeat_back_addr":"172.21.15.161:6807/1707914885","heartbeat_front_addr":"172.21.15.161:6805/1707914885","state":["exists","up"]},{"osd":3,"uuid":"73f78766-9662-4aa3-841c-fe2ce654ee9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1541142398},{"type":"v1","addr":"172.21.15.1:6811","nonce":1541142398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1541142398},{"type":"v1","addr":"172.21.15.1:6813","nonce":1541142398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1541142398},{"type":"v1","addr":"172.21.15.1:6817","nonce":1541142398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1541142398},{"type":"v1","addr":"172.21.15.1:6815","nonce":1541142398}]},"public_addr":"172.21.15.1:6811/1541142398","cluster_addr":"172.21.15.1:6813/1541142398","heartbeat_back_addr":"172.21.15.1:6817/1541142398","heartbeat_front_addr":"172.21.15.1:6815/1541142398","state":["exists","up"]},{"osd":4,"uuid":"56c808bb-d4cd-4969-b49e-adef49ecd0f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":3081402681},{"type":"v1","addr":"172.21.15.53:6809","nonce":3081402681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3081402681},{"type":"v1","addr":"172.21.15.53:6811","nonce":3081402681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3081402681},{"type":"v1","addr":"172.21.15.53:6815","nonce":3081402681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3081402681},{"type":"v1","addr":"172.21.15.53:6813","nonce":3081402681}]},"public_addr":"172.21.15.53:6809/3081402681","cluster_addr":"172.21.15.53:6811/3081402681","heartbeat_back_addr":"172.21.15.53:6815/3081402681","heartbeat_front_addr":"172.21.15.53:6813/3081402681","state":["exists","up"]},{"osd":5,"uuid":"7b8266d6-b507-47c9-b4fa-ff16c4a74d35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":1622081327},{"type":"v1","addr":"172.21.15.161:6809","nonce":1622081327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":1622081327},{"type":"v1","addr":"172.21.15.161:6811","nonce":1622081327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":1622081327},{"type":"v1","addr":"172.21.15.161:6815","nonce":1622081327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":1622081327},{"type":"v1","addr":"172.21.15.161:6813","nonce":1622081327}]},"public_addr":"172.21.15.161:6809/1622081327","cluster_addr":"172.21.15.161:6811/1622081327","heartbeat_back_addr":"172.21.15.161:6815/1622081327","heartbeat_front_addr":"172.21.15.161:6813/1622081327","state":["exists","up"]},{"osd":6,"uuid":"3af5925d-a749-4849-a076-80580222cad4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":158434853},{"type":"v1","addr":"172.21.15.1:6819","nonce":158434853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":158434853},{"type":"v1","addr":"172.21.15.1:6821","nonce":158434853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":158434853},{"type":"v1","addr":"172.21.15.1:6825","nonce":158434853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":158434853},{"type":"v1","addr":"172.21.15.1:6823","nonce":158434853}]},"public_addr":"172.21.15.1:6819/158434853","cluster_addr":"172.21.15.1:6821/158434853","heartbeat_back_addr":"172.21.15.1:6825/158434853","heartbeat_front_addr":"172.21.15.1:6823/158434853","state":["exists","up"]},{"osd":7,"uuid":"f8f26c3c-17e9-4e89-8392-289796e0c8a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2132057697},{"type":"v1","addr":"172.21.15.53:6817","nonce":2132057697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2132057697},{"type":"v1","addr":"172.21.15.53:6819","nonce":2132057697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2132057697},{"type":"v1","addr":"172.21.15.53:6823","nonce":2132057697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2132057697},{"type":"v1","addr":"172.21.15.53:6821","nonce":2132057697}]},"public_addr":"172.21.15.53:6817/2132057697","cluster_addr":"172.21.15.53:6819/2132057697","heartbeat_back_addr":"172.21.15.53:6823/2132057697","heartbeat_front_addr":"172.21.15.53:6821/2132057697","state":["exists","up"]},{"osd":8,"uuid":"36ea401b-3c93-4b9d-8880-298d1eb6e30b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1838248669},{"type":"v1","addr":"172.21.15.161:6817","nonce":1838248669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1838248669},{"type":"v1","addr":"172.21.15.161:6819","nonce":1838248669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1838248669},{"type":"v1","addr":"172.21.15.161:6823","nonce":1838248669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1838248669},{"type":"v1","addr":"172.21.15.161:6821","nonce":1838248669}]},"public_addr":"172.21.15.161:6817/1838248669","cluster_addr":"172.21.15.161:6819/1838248669","heartbeat_back_addr":"172.21.15.161:6823/1838248669","heartbeat_front_addr":"172.21.15.161:6821/1838248669","state":["exists","up"]},{"osd":9,"uuid":"4b788d3f-a3bc-4463-bd99-95fdb7a52768","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2806274589},{"type":"v1","addr":"172.21.15.1:6827","nonce":2806274589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2806274589},{"type":"v1","addr":"172.21.15.1:6829","nonce":2806274589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":2806274589},{"type":"v1","addr":"172.21.15.1:6833","nonce":2806274589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2806274589},{"type":"v1","addr":"172.21.15.1:6831","nonce":2806274589}]},"public_addr":"172.21.15.1:6827/2806274589","cluster_addr":"172.21.15.1:6829/2806274589","heartbeat_back_addr":"172.21.15.1:6833/2806274589","heartbeat_front_addr":"172.21.15.1:6831/2806274589","state":["exists","up"]},{"osd":10,"uuid":"70549796-5270-4ae2-ac3f-f61f5de105d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":761923580},{"type":"v1","addr":"172.21.15.53:6825","nonce":761923580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":761923580},{"type":"v1","addr":"172.21.15.53:6827","nonce":761923580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":761923580},{"type":"v1","addr":"172.21.15.53:6831","nonce":761923580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":761923580},{"type":"v1","addr":"172.21.15.53:6829","nonce":761923580}]},"public_addr":"172.21.15.53:6825/761923580","cluster_addr":"172.21.15.53:6827/761923580","heartbeat_back_addr":"172.21.15.53:6831/761923580","heartbeat_front_addr":"172.21.15.53:6829/761923580","state":["exists","up"]},{"osd":11,"uuid":"48f17c4a-370a-493e-956e-bf6ad3dcc594","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1586352190},{"type":"v1","addr":"172.21.15.161:6825","nonce":1586352190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":1586352190},{"type":"v1","addr":"172.21.15.161:6827","nonce":1586352190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":1586352190},{"type":"v1","addr":"172.21.15.161:6831","nonce":1586352190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":1586352190},{"type":"v1","addr":"172.21.15.161:6829","nonce":1586352190}]},"public_addr":"172.21.15.161:6825/1586352190","cluster_addr":"172.21.15.161:6827/1586352190","heartbeat_back_addr":"172.21.15.161:6831/1586352190","heartbeat_front_addr":"172.21.15.161:6829/1586352190","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:49:46.539613+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:50:11.912300+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:50:36.955490+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:51:02.200554+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:51:27.860585+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:51:54.694099+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:52:18.962351+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:52:44.881923+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:53:10.301390+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:53:35.417647+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:54:01.492219+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:54:26.938338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/1749020850":"2024-09-12T21:48:24.524913+0000","172.21.15.1:6801/2821761445":"2024-09-12T21:48:24.524913+0000","172.21.15.1:0/1699984777":"2024-09-12T21:48:24.524913+0000","172.21.15.1:0/276770268":"2024-09-12T21:48:00.674728+0000","172.21.15.1:0/1278877732":"2024-09-12T21:48:24.524913+0000","172.21.15.1:0/2599895977":"2024-09-12T21:48:00.674728+0000","172.21.15.1:6801/2311309446":"2024-09-12T21:48:00.674728+0000","172.21.15.1:6800/2821761445":"2024-09-12T21:48:24.524913+0000","172.21.15.1:6800/2311309446":"2024-09-12T21:48:00.674728+0000","172.21.15.1:0/3202962495":"2024-09-12T21:48:00.674728+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T21:55:54.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:54.699+0000 7f2008ea7640 1 -- 172.21.15.1:0/783498950 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f1ff00a2c10 msgr2=0x7f1ff00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:54.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:54.699+0000 7f2008ea7640 1 --2- 172.21.15.1:0/783498950 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f1ff00a2c10 0x7f1ff00a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1ff400bbe0 tx=0x7f1ff400b390 comp rx=0 tx=0).stop 2024-09-11T21:55:54.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:54.699+0000 7f2008ea7640 1 -- 172.21.15.1:0/783498950 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2004102790 msgr2=0x7f2004198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:54.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:54.699+0000 7f2008ea7640 1 --2- 172.21.15.1:0/783498950 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2004102790 0x7f2004198a10 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1fec006010 tx=0x7f1fec0042f0 comp rx=0 tx=0).stop 2024-09-11T21:55:54.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:54.699+0000 7f2008ea7640 1 -- 172.21.15.1:0/783498950 shutdown_connections 2024-09-11T21:55:54.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:54.699+0000 7f2008ea7640 1 --2- 172.21.15.1:0/783498950 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f1ff00a2c10 0x7f1ff00a50d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:54.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:54.699+0000 7f2008ea7640 1 --2- 172.21.15.1:0/783498950 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2004106f10 0x7f200419cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:54.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:54.699+0000 7f2008ea7640 1 --2- 172.21.15.1:0/783498950 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2004102790 0x7f2004198a10 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:54.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:54.699+0000 7f2008ea7640 1 --2- 172.21.15.1:0/783498950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2004101de0 0x7f20041984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:54.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:54.699+0000 7f2008ea7640 1 -- 172.21.15.1:0/783498950 >> 172.21.15.1:0/783498950 conn(0x7f20040fd8c0 msgr2=0x7f20040fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:55:54.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:54.699+0000 7f2008ea7640 1 -- 172.21.15.1:0/783498950 shutdown_connections 2024-09-11T21:55:54.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:54.699+0000 7f2008ea7640 1 -- 172.21.15.1:0/783498950 wait complete. 2024-09-11T21:55:55.436 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:55 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/783498950' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T21:55:55.438 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-11T21:55:55.439 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-11T21:55:55.439 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-11T21:55:55.440 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-11T21:55:55.440 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-11T21:55:55.440 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-11T21:55:55.441 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-11T21:55:55.441 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-11T21:55:55.442 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-11T21:55:55.442 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-11T21:55:55.442 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-11T21:55:55.443 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-11T21:55:55.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:55 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/783498950' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T21:55:55.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:55 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/783498950' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T21:55:56.150 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:55:56.279 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:55:56.279 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:55:56.280 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:55:56.284 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:55:56.287 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:55:56.287 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:55:56.287 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:55:56.290 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:55:56.341 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:55:56.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:56 smithi053 ceph-mon[34908]: pgmap v299: 97 pgs: 97 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-09-11T21:55:56.585 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:56 smithi001 ceph-mon[30935]: pgmap v299: 97 pgs: 97 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-09-11T21:55:56.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:56 smithi161 ceph-mon[35615]: pgmap v299: 97 pgs: 97 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-09-11T21:55:57.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.518+0000 7f0ca93db640 1 -- 172.21.15.1:0/111435194 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ca4102790 msgr2=0x7f0ca41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:57.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.518+0000 7f0ca93db640 1 --2- 172.21.15.1:0/111435194 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ca4102790 0x7f0ca41067e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0c98009900 tx=0x7f0c9802a410 comp rx=0 tx=0).stop 2024-09-11T21:55:57.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.519+0000 7f0ca93db640 1 -- 172.21.15.1:0/111435194 shutdown_connections 2024-09-11T21:55:57.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.519+0000 7f0ca93db640 1 --2- 172.21.15.1:0/111435194 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ca4106f10 0x7f0ca410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:57.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.519+0000 7f0ca93db640 1 --2- 172.21.15.1:0/111435194 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ca4102790 0x7f0ca41067e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:57.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.519+0000 7f0ca93db640 1 --2- 172.21.15.1:0/111435194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ca4101de0 0x7f0ca41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:57.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.519+0000 7f0ca93db640 1 -- 172.21.15.1:0/111435194 >> 172.21.15.1:0/111435194 conn(0x7f0ca40fd8c0 msgr2=0x7f0ca40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:55:57.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.519+0000 7f0ca93db640 1 -- 172.21.15.1:0/111435194 shutdown_connections 2024-09-11T21:55:57.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.519+0000 7f0ca93db640 1 -- 172.21.15.1:0/111435194 wait complete. 2024-09-11T21:55:57.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.519+0000 7f0ca93db640 1 Processor -- start 2024-09-11T21:55:57.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.519+0000 7f0ca93db640 1 -- start start 2024-09-11T21:55:57.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.520+0000 7f0ca93db640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ca4101de0 0x7f0ca4198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:57.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.520+0000 7f0ca93db640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ca4102790 0x7f0ca41989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:57.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.520+0000 7f0ca93db640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ca4106f10 0x7f0ca419cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:57.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.520+0000 7f0ca93db640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca4111f70 con 0x7f0ca4101de0 2024-09-11T21:55:57.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.520+0000 7f0ca93db640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca4111df0 con 0x7f0ca4102790 2024-09-11T21:55:57.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.520+0000 7f0ca93db640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca41120f0 con 0x7f0ca4106f10 2024-09-11T21:55:57.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.520+0000 7f0ca2ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ca4101de0 0x7f0ca4198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:57.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.520+0000 7f0ca2ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ca4101de0 0x7f0ca4198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56730/0 (socket says 172.21.15.1:56730) 2024-09-11T21:55:57.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.520+0000 7f0ca2ffd640 1 -- 172.21.15.1:0/1907415213 learned_addr learned my addr 172.21.15.1:0/1907415213 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:55:57.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.520+0000 7f0ca27fc640 1 --2- 172.21.15.1:0/1907415213 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ca4102790 0x7f0ca41989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:57.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.520+0000 7f0ca37fe640 1 --2- 172.21.15.1:0/1907415213 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ca4106f10 0x7f0ca419cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:57.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.520+0000 7f0ca2ffd640 1 -- 172.21.15.1:0/1907415213 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ca4106f10 msgr2=0x7f0ca419cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:57.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.520+0000 7f0ca2ffd640 1 --2- 172.21.15.1:0/1907415213 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ca4106f10 0x7f0ca419cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:57.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.520+0000 7f0ca2ffd640 1 -- 172.21.15.1:0/1907415213 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ca4102790 msgr2=0x7f0ca41989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:57.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.520+0000 7f0ca2ffd640 1 --2- 172.21.15.1:0/1907415213 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0ca4102790 0x7f0ca41989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:57.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.520+0000 7f0ca2ffd640 1 -- 172.21.15.1:0/1907415213 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ca419d440 con 0x7f0ca4101de0 2024-09-11T21:55:57.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.520+0000 7f0ca37fe640 1 --2- 172.21.15.1:0/1907415213 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0ca4106f10 0x7f0ca419cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T21:55:57.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.520+0000 7f0ca2ffd640 1 --2- 172.21.15.1:0/1907415213 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ca4101de0 0x7f0ca4198490 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0c9400d770 tx=0x7f0c9400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:55:57.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.521+0000 7f0c8bfff640 1 -- 172.21.15.1:0/1907415213 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c94017070 con 0x7f0ca4101de0 2024-09-11T21:55:57.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.521+0000 7f0ca93db640 1 -- 172.21.15.1:0/1907415213 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ca419d730 con 0x7f0ca4101de0 2024-09-11T21:55:57.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.521+0000 7f0c8bfff640 1 -- 172.21.15.1:0/1907415213 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c94005500 con 0x7f0ca4101de0 2024-09-11T21:55:57.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.521+0000 7f0c8bfff640 1 -- 172.21.15.1:0/1907415213 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c94004e00 con 0x7f0ca4101de0 2024-09-11T21:55:57.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.521+0000 7f0ca93db640 1 -- 172.21.15.1:0/1907415213 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ca41a5010 con 0x7f0ca4101de0 2024-09-11T21:55:57.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.521+0000 7f0ca93db640 1 -- 172.21.15.1:0/1907415213 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0ca4062980 con 0x7f0ca4101de0 2024-09-11T21:55:57.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.523+0000 7f0c8bfff640 1 -- 172.21.15.1:0/1907415213 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f0c94004fa0 con 0x7f0ca4101de0 2024-09-11T21:55:57.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.523+0000 7f0c8bfff640 1 --2- 172.21.15.1:0/1907415213 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f0c8c0a2c10 0x7f0c8c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:57.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.523+0000 7f0ca27fc640 1 --2- 172.21.15.1:0/1907415213 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f0c8c0a2c10 0x7f0c8c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:57.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.524+0000 7f0c8bfff640 1 -- 172.21.15.1:0/1907415213 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f0c940cee90 con 0x7f0ca4101de0 2024-09-11T21:55:57.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.524+0000 7f0c8bfff640 1 --2- 172.21.15.1:0/1907415213 >> [v2:172.21.15.53:6800/3039391924,v1:172.21.15.53:6801/3039391924] conn(0x7f0c8c0ac460 0x7f0c8c0ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:57.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.524+0000 7f0c8bfff640 1 -- 172.21.15.1:0/1907415213 --> [v2:172.21.15.53:6800/3039391924,v1:172.21.15.53:6801/3039391924] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0c94001d30 con 0x7f0c8c0ac460 2024-09-11T21:55:57.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.524+0000 7f0c8bfff640 1 -- 172.21.15.1:0/1907415213 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f0c940cf370 con 0x7f0ca4101de0 2024-09-11T21:55:57.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.524+0000 7f0ca27fc640 1 --2- 172.21.15.1:0/1907415213 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f0c8c0a2c10 0x7f0c8c0a50d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0c98009230 tx=0x7f0c980058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:55:57.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.524+0000 7f0ca37fe640 1 --2- 172.21.15.1:0/1907415213 >> [v2:172.21.15.53:6800/3039391924,v1:172.21.15.53:6801/3039391924] conn(0x7f0c8c0ac460 0x7f0c8c0ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:57.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.525+0000 7f0ca37fe640 1 --2- 172.21.15.1:0/1907415213 >> [v2:172.21.15.53:6800/3039391924,v1:172.21.15.53:6801/3039391924] conn(0x7f0c8c0ac460 0x7f0c8c0ae880 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:55:57.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.525+0000 7f0c8bfff640 1 -- 172.21.15.1:0/1907415213 <== osd.1 v2:172.21.15.53:6800/3039391924 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0c94001d30 con 0x7f0c8c0ac460 2024-09-11T21:55:57.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.540+0000 7f0ca93db640 1 -- 172.21.15.1:0/1907415213 --> [v2:172.21.15.53:6800/3039391924,v1:172.21.15.53:6801/3039391924] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0ca4062e00 con 0x7f0c8c0ac460 2024-09-11T21:55:57.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.541+0000 7f0c8bfff640 1 -- 172.21.15.1:0/1907415213 <== osd.1 v2:172.21.15.53:6800/3039391924 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0ca4062e00 con 0x7f0c8c0ac460 2024-09-11T21:55:57.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.541+0000 7f0ca93db640 1 -- 172.21.15.1:0/1907415213 >> [v2:172.21.15.53:6800/3039391924,v1:172.21.15.53:6801/3039391924] conn(0x7f0c8c0ac460 msgr2=0x7f0c8c0ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:57.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.541+0000 7f0ca93db640 1 --2- 172.21.15.1:0/1907415213 >> [v2:172.21.15.53:6800/3039391924,v1:172.21.15.53:6801/3039391924] conn(0x7f0c8c0ac460 0x7f0c8c0ae880 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:57.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.541+0000 7f0ca93db640 1 -- 172.21.15.1:0/1907415213 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f0c8c0a2c10 msgr2=0x7f0c8c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:57.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.541+0000 7f0ca93db640 1 --2- 172.21.15.1:0/1907415213 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f0c8c0a2c10 0x7f0c8c0a50d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0c98009230 tx=0x7f0c980058d0 comp rx=0 tx=0).stop 2024-09-11T21:55:57.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.542+0000 7f0ca93db640 1 -- 172.21.15.1:0/1907415213 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ca4101de0 msgr2=0x7f0ca4198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:57.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.542+0000 7f0ca93db640 1 --2- 172.21.15.1:0/1907415213 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ca4101de0 0x7f0ca4198490 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0c9400d770 tx=0x7f0c9400dc30 comp rx=0 tx=0).stop 2024-09-11T21:55:57.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.542+0000 7f0ca37fe640 1 -- 172.21.15.1:0/1907415213 reap_dead start 2024-09-11T21:55:57.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.542+0000 7f0ca93db640 1 -- 172.21.15.1:0/1907415213 shutdown_connections 2024-09-11T21:55:57.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.542+0000 7f0ca93db640 1 -- 172.21.15.1:0/1907415213 >> 172.21.15.1:0/1907415213 conn(0x7f0ca40fd8c0 msgr2=0x7f0ca40fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:55:57.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.542+0000 7f0ca93db640 1 -- 172.21.15.1:0/1907415213 shutdown_connections 2024-09-11T21:55:57.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.542+0000 7f0ca93db640 1 -- 172.21.15.1:0/1907415213 wait complete. 2024-09-11T21:55:57.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.971+0000 7f49055dc640 1 -- 172.21.15.1:0/3044637252 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4900102ef0 msgr2=0x7f490010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:57.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.971+0000 7f49055dc640 1 --2- 172.21.15.1:0/3044637252 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4900102ef0 0x7f490010da40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f48f8009ca0 tx=0x7f48f802a950 comp rx=0 tx=0).stop 2024-09-11T21:55:57.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.971+0000 7f49055dc640 1 -- 172.21.15.1:0/3044637252 shutdown_connections 2024-09-11T21:55:57.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.971+0000 7f49055dc640 1 --2- 172.21.15.1:0/3044637252 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4900102ef0 0x7f490010da40 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:57.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.971+0000 7f49055dc640 1 --2- 172.21.15.1:0/3044637252 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4900102570 0x7f49001029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:57.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.971+0000 7f49055dc640 1 --2- 172.21.15.1:0/3044637252 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4900100e20 0x7f4900101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:57.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.971+0000 7f49055dc640 1 -- 172.21.15.1:0/3044637252 >> 172.21.15.1:0/3044637252 conn(0x7f49000fc7d0 msgr2=0x7f49000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:55:57.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.971+0000 7f49055dc640 1 -- 172.21.15.1:0/3044637252 shutdown_connections 2024-09-11T21:55:57.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.971+0000 7f49055dc640 1 -- 172.21.15.1:0/3044637252 wait complete. 2024-09-11T21:55:57.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.971+0000 7f49055dc640 1 Processor -- start 2024-09-11T21:55:57.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.972+0000 7f49055dc640 1 -- start start 2024-09-11T21:55:57.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.972+0000 7f49055dc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4900100e20 0x7f49001962c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:57.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.972+0000 7f49055dc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4900102570 0x7f4900196800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:57.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.972+0000 7f49055dc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4900102ef0 0x7f490019ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:57.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.972+0000 7f49055dc640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f490010fec0 con 0x7f4900100e20 2024-09-11T21:55:57.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.972+0000 7f49055dc640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f490010fd40 con 0x7f4900102570 2024-09-11T21:55:57.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.972+0000 7f49055dc640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4900110040 con 0x7f4900102ef0 2024-09-11T21:55:57.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.972+0000 7f48feffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4900100e20 0x7f49001962c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:57.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.972+0000 7f48feffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4900100e20 0x7f49001962c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56750/0 (socket says 172.21.15.1:56750) 2024-09-11T21:55:57.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.972+0000 7f48feffd640 1 -- 172.21.15.1:0/447551380 learned_addr learned my addr 172.21.15.1:0/447551380 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:55:57.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.972+0000 7f48fe7fc640 1 --2- 172.21.15.1:0/447551380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4900102570 0x7f4900196800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:57.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.972+0000 7f48feffd640 1 -- 172.21.15.1:0/447551380 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4900102ef0 msgr2=0x7f490019ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:57.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.972+0000 7f48feffd640 1 --2- 172.21.15.1:0/447551380 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4900102ef0 0x7f490019ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:57.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.972+0000 7f48feffd640 1 -- 172.21.15.1:0/447551380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4900102570 msgr2=0x7f4900196800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:57.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.972+0000 7f48feffd640 1 --2- 172.21.15.1:0/447551380 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4900102570 0x7f4900196800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:57.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.972+0000 7f48feffd640 1 -- 172.21.15.1:0/447551380 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f490019b270 con 0x7f4900100e20 2024-09-11T21:55:57.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.972+0000 7f48feffd640 1 --2- 172.21.15.1:0/447551380 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4900100e20 0x7f49001962c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f48e800c890 tx=0x7f48e800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:55:57.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.973+0000 7f48f7fff640 1 -- 172.21.15.1:0/447551380 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e8004220 con 0x7f4900100e20 2024-09-11T21:55:57.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.973+0000 7f49055dc640 1 -- 172.21.15.1:0/447551380 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f490019b560 con 0x7f4900100e20 2024-09-11T21:55:57.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.973+0000 7f48f7fff640 1 -- 172.21.15.1:0/447551380 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48e80043c0 con 0x7f4900100e20 2024-09-11T21:55:57.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.973+0000 7f48f7fff640 1 -- 172.21.15.1:0/447551380 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e8004a00 con 0x7f4900100e20 2024-09-11T21:55:57.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.973+0000 7f49055dc640 1 -- 172.21.15.1:0/447551380 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49001a2e40 con 0x7f4900100e20 2024-09-11T21:55:57.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.973+0000 7f49055dc640 1 -- 172.21.15.1:0/447551380 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4900107230 con 0x7f4900100e20 2024-09-11T21:55:57.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.975+0000 7f48f7fff640 1 -- 172.21.15.1:0/447551380 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f48e80079b0 con 0x7f4900100e20 2024-09-11T21:55:57.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.976+0000 7f48f7fff640 1 --2- 172.21.15.1:0/447551380 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f48ec0a4e20 0x7f48ec0a72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:57.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.976+0000 7f48fe7fc640 1 --2- 172.21.15.1:0/447551380 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f48ec0a4e20 0x7f48ec0a72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:57.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.976+0000 7f48f7fff640 1 -- 172.21.15.1:0/447551380 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f48e80cf120 con 0x7f4900100e20 2024-09-11T21:55:57.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.976+0000 7f48f7fff640 1 --2- 172.21.15.1:0/447551380 >> [v2:172.21.15.1:6802/4012142023,v1:172.21.15.1:6803/4012142023] conn(0x7f48ec0ae670 0x7f48ec0b0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:55:57.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.976+0000 7f48f7fff640 1 -- 172.21.15.1:0/447551380 --> [v2:172.21.15.1:6802/4012142023,v1:172.21.15.1:6803/4012142023] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f48e8008e50 con 0x7f48ec0ae670 2024-09-11T21:55:57.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.976+0000 7f48f7fff640 1 -- 172.21.15.1:0/447551380 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f48e80cf600 con 0x7f4900100e20 2024-09-11T21:55:57.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.976+0000 7f48ff7fe640 1 --2- 172.21.15.1:0/447551380 >> [v2:172.21.15.1:6802/4012142023,v1:172.21.15.1:6803/4012142023] conn(0x7f48ec0ae670 0x7f48ec0b0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:55:57.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.976+0000 7f48fe7fc640 1 --2- 172.21.15.1:0/447551380 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f48ec0a4e20 0x7f48ec0a72e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f49000fec10 tx=0x7f48f000a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:55:57.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.976+0000 7f48ff7fe640 1 --2- 172.21.15.1:0/447551380 >> [v2:172.21.15.1:6802/4012142023,v1:172.21.15.1:6803/4012142023] conn(0x7f48ec0ae670 0x7f48ec0b0a90 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:55:57.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.977+0000 7f48f7fff640 1 -- 172.21.15.1:0/447551380 <== osd.0 v2:172.21.15.1:6802/4012142023 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f48e8008e50 con 0x7f48ec0ae670 2024-09-11T21:55:57.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.991+0000 7f49055dc640 1 -- 172.21.15.1:0/447551380 --> [v2:172.21.15.1:6802/4012142023,v1:172.21.15.1:6803/4012142023] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f49001021b0 con 0x7f48ec0ae670 2024-09-11T21:55:57.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.991+0000 7f48f7fff640 1 -- 172.21.15.1:0/447551380 <== osd.0 v2:172.21.15.1:6802/4012142023 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f49001021b0 con 0x7f48ec0ae670 2024-09-11T21:55:57.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.992+0000 7f49055dc640 1 -- 172.21.15.1:0/447551380 >> [v2:172.21.15.1:6802/4012142023,v1:172.21.15.1:6803/4012142023] conn(0x7f48ec0ae670 msgr2=0x7f48ec0b0a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:57.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.992+0000 7f49055dc640 1 --2- 172.21.15.1:0/447551380 >> [v2:172.21.15.1:6802/4012142023,v1:172.21.15.1:6803/4012142023] conn(0x7f48ec0ae670 0x7f48ec0b0a90 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:55:57.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.992+0000 7f49055dc640 1 -- 172.21.15.1:0/447551380 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f48ec0a4e20 msgr2=0x7f48ec0a72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:57.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.992+0000 7f49055dc640 1 --2- 172.21.15.1:0/447551380 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f48ec0a4e20 0x7f48ec0a72e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f49000fec10 tx=0x7f48f000a250 comp rx=0 tx=0).stop 2024-09-11T21:55:57.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.992+0000 7f49055dc640 1 -- 172.21.15.1:0/447551380 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4900100e20 msgr2=0x7f49001962c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:55:57.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.992+0000 7f49055dc640 1 --2- 172.21.15.1:0/447551380 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4900100e20 0x7f49001962c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f48e800c890 tx=0x7f48e800cd50 comp rx=0 tx=0).stop 2024-09-11T21:55:57.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.992+0000 7f48ff7fe640 1 -- 172.21.15.1:0/447551380 reap_dead start 2024-09-11T21:55:57.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.992+0000 7f49055dc640 1 -- 172.21.15.1:0/447551380 shutdown_connections 2024-09-11T21:55:57.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.992+0000 7f49055dc640 1 -- 172.21.15.1:0/447551380 >> 172.21.15.1:0/447551380 conn(0x7f49000fc7d0 msgr2=0x7f4900106810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:55:57.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.992+0000 7f49055dc640 1 -- 172.21.15.1:0/447551380 shutdown_connections 2024-09-11T21:55:57.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:55:57.992+0000 7f49055dc640 1 -- 172.21.15.1:0/447551380 wait complete. 2024-09-11T21:55:58.225 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:55:58.226 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:55:58.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:55:58 smithi053 ceph-mon[34908]: pgmap v300: 97 pgs: 97 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-11T21:55:58.581 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:55:58 smithi001 ceph-mon[30935]: pgmap v300: 97 pgs: 97 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-11T21:55:58.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:55:58 smithi161 ceph-mon[35615]: pgmap v300: 97 pgs: 97 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-11T21:55:59.011 INFO:teuthology.orchestra.run.smithi001.stdout:34359738445 2024-09-11T21:55:59.011 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-11T21:55:59.552 INFO:teuthology.orchestra.run.smithi001.stdout:55834574920 2024-09-11T21:55:59.552 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-11T21:56:00.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.278+0000 7f2be3cc6640 1 -- 172.21.15.1:0/1645964667 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2bdc101420 msgr2=0x7f2bdc10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:00.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.278+0000 7f2be3cc6640 1 --2- 172.21.15.1:0/1645964667 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2bdc101420 0x7f2bdc10d950 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4009cd0 tx=0x7f2bd402a8e0 comp rx=0 tx=0).stop 2024-09-11T21:56:00.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.278+0000 7f2be3cc6640 1 -- 172.21.15.1:0/1645964667 shutdown_connections 2024-09-11T21:56:00.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.278+0000 7f2be3cc6640 1 --2- 172.21.15.1:0/1645964667 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2bdc101420 0x7f2bdc10d950 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:00.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.278+0000 7f2be3cc6640 1 --2- 172.21.15.1:0/1645964667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2bdc100a80 0x7f2bdc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:00.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.278+0000 7f2be3cc6640 1 --2- 172.21.15.1:0/1645964667 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2bdc106a80 0x7f2bdc106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:00.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.278+0000 7f2be3cc6640 1 -- 172.21.15.1:0/1645964667 >> 172.21.15.1:0/1645964667 conn(0x7f2bdc0fc7a0 msgr2=0x7f2bdc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:00.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.278+0000 7f2be3cc6640 1 -- 172.21.15.1:0/1645964667 shutdown_connections 2024-09-11T21:56:00.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.278+0000 7f2be3cc6640 1 -- 172.21.15.1:0/1645964667 wait complete. 2024-09-11T21:56:00.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.279+0000 7f2be3cc6640 1 Processor -- start 2024-09-11T21:56:00.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.279+0000 7f2be3cc6640 1 -- start start 2024-09-11T21:56:00.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.279+0000 7f2be3cc6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2bdc100a80 0x7f2bdc199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:00.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.279+0000 7f2be3cc6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2bdc101420 0x7f2bdc1997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:00.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.279+0000 7f2be3cc6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2bdc106a80 0x7f2bdc1a42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:00.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.279+0000 7f2be3cc6640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2bdc110e80 con 0x7f2bdc100a80 2024-09-11T21:56:00.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.279+0000 7f2be3cc6640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2bdc110d00 con 0x7f2bdc101420 2024-09-11T21:56:00.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.279+0000 7f2be3cc6640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2bdc111000 con 0x7f2bdc106a80 2024-09-11T21:56:00.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.279+0000 7f2be1a62640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2bdc100a80 0x7f2bdc199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:00.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.279+0000 7f2be1a62640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2bdc100a80 0x7f2bdc199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56770/0 (socket says 172.21.15.1:56770) 2024-09-11T21:56:00.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.279+0000 7f2be1a62640 1 -- 172.21.15.1:0/565560466 learned_addr learned my addr 172.21.15.1:0/565560466 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:56:00.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.279+0000 7f2be1261640 1 --2- 172.21.15.1:0/565560466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2bdc101420 0x7f2bdc1997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:00.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.279+0000 7f2be2263640 1 --2- 172.21.15.1:0/565560466 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2bdc106a80 0x7f2bdc1a42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:00.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.279+0000 7f2be1a62640 1 -- 172.21.15.1:0/565560466 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2bdc106a80 msgr2=0x7f2bdc1a42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:00.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.279+0000 7f2be1a62640 1 --2- 172.21.15.1:0/565560466 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2bdc106a80 0x7f2bdc1a42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:00.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.279+0000 7f2be1a62640 1 -- 172.21.15.1:0/565560466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2bdc101420 msgr2=0x7f2bdc1997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:00.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.279+0000 7f2be1a62640 1 --2- 172.21.15.1:0/565560466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2bdc101420 0x7f2bdc1997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:00.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.279+0000 7f2be1a62640 1 -- 172.21.15.1:0/565560466 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bdc1a4820 con 0x7f2bdc100a80 2024-09-11T21:56:00.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.280+0000 7f2be1a62640 1 --2- 172.21.15.1:0/565560466 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2bdc100a80 0x7f2bdc199260 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2bd800c890 tx=0x7f2bd800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:00.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.280+0000 7f2bd2ffd640 1 -- 172.21.15.1:0/565560466 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd8004220 con 0x7f2bdc100a80 2024-09-11T21:56:00.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.280+0000 7f2bd2ffd640 1 -- 172.21.15.1:0/565560466 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bd80043c0 con 0x7f2bdc100a80 2024-09-11T21:56:00.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.280+0000 7f2be3cc6640 1 -- 172.21.15.1:0/565560466 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bdc1a49f0 con 0x7f2bdc100a80 2024-09-11T21:56:00.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.280+0000 7f2bd2ffd640 1 -- 172.21.15.1:0/565560466 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd80049e0 con 0x7f2bdc100a80 2024-09-11T21:56:00.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.280+0000 7f2be3cc6640 1 -- 172.21.15.1:0/565560466 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bdc1a4f00 con 0x7f2bdc100a80 2024-09-11T21:56:00.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.281+0000 7f2be3cc6640 1 -- 172.21.15.1:0/565560466 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2bdc062900 con 0x7f2bdc100a80 2024-09-11T21:56:00.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.282+0000 7f2bd2ffd640 1 -- 172.21.15.1:0/565560466 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f2bd80079b0 con 0x7f2bdc100a80 2024-09-11T21:56:00.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.283+0000 7f2bd2ffd640 1 --2- 172.21.15.1:0/565560466 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f2bc80a2c10 0x7f2bc80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:00.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.283+0000 7f2be1261640 1 --2- 172.21.15.1:0/565560466 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f2bc80a2c10 0x7f2bc80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:00.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.283+0000 7f2bd2ffd640 1 -- 172.21.15.1:0/565560466 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f2bd80cf260 con 0x7f2bdc100a80 2024-09-11T21:56:00.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.283+0000 7f2bd2ffd640 1 --2- 172.21.15.1:0/565560466 >> [v2:172.21.15.161:6824/1586352190,v1:172.21.15.161:6825/1586352190] conn(0x7f2bc80ac460 0x7f2bc80ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:00.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.283+0000 7f2bd2ffd640 1 -- 172.21.15.1:0/565560466 --> [v2:172.21.15.161:6824/1586352190,v1:172.21.15.161:6825/1586352190] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2bd8005110 con 0x7f2bc80ac460 2024-09-11T21:56:00.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.283+0000 7f2bd2ffd640 1 -- 172.21.15.1:0/565560466 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f2bd80cf740 con 0x7f2bdc100a80 2024-09-11T21:56:00.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.283+0000 7f2be1261640 1 --2- 172.21.15.1:0/565560466 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f2bc80a2c10 0x7f2bc80a50d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc06b2e0 tx=0x7f2bcc00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:00.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.283+0000 7f2be2263640 1 --2- 172.21.15.1:0/565560466 >> [v2:172.21.15.161:6824/1586352190,v1:172.21.15.161:6825/1586352190] conn(0x7f2bc80ac460 0x7f2bc80ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:00.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.284+0000 7f2be2263640 1 --2- 172.21.15.1:0/565560466 >> [v2:172.21.15.161:6824/1586352190,v1:172.21.15.161:6825/1586352190] conn(0x7f2bc80ac460 0x7f2bc80ae880 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:00.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.284+0000 7f2bd2ffd640 1 -- 172.21.15.1:0/565560466 <== osd.11 v2:172.21.15.161:6824/1586352190 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2bd8005110 con 0x7f2bc80ac460 2024-09-11T21:56:00.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.298+0000 7f2be3cc6640 1 -- 172.21.15.1:0/565560466 --> [v2:172.21.15.161:6824/1586352190,v1:172.21.15.161:6825/1586352190] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2bdc062d80 con 0x7f2bc80ac460 2024-09-11T21:56:00.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.299+0000 7f2bd2ffd640 1 -- 172.21.15.1:0/565560466 <== osd.11 v2:172.21.15.161:6824/1586352190 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2bdc062d80 con 0x7f2bc80ac460 2024-09-11T21:56:00.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.299+0000 7f2be3cc6640 1 -- 172.21.15.1:0/565560466 >> [v2:172.21.15.161:6824/1586352190,v1:172.21.15.161:6825/1586352190] conn(0x7f2bc80ac460 msgr2=0x7f2bc80ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:00.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.299+0000 7f2be3cc6640 1 --2- 172.21.15.1:0/565560466 >> [v2:172.21.15.161:6824/1586352190,v1:172.21.15.161:6825/1586352190] conn(0x7f2bc80ac460 0x7f2bc80ae880 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:00.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.299+0000 7f2be3cc6640 1 -- 172.21.15.1:0/565560466 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f2bc80a2c10 msgr2=0x7f2bc80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:00.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.299+0000 7f2be3cc6640 1 --2- 172.21.15.1:0/565560466 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f2bc80a2c10 0x7f2bc80a50d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc06b2e0 tx=0x7f2bcc00b390 comp rx=0 tx=0).stop 2024-09-11T21:56:00.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.299+0000 7f2be3cc6640 1 -- 172.21.15.1:0/565560466 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2bdc100a80 msgr2=0x7f2bdc199260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:00.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.299+0000 7f2be3cc6640 1 --2- 172.21.15.1:0/565560466 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2bdc100a80 0x7f2bdc199260 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2bd800c890 tx=0x7f2bd800cd50 comp rx=0 tx=0).stop 2024-09-11T21:56:00.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.299+0000 7f2be2263640 1 -- 172.21.15.1:0/565560466 reap_dead start 2024-09-11T21:56:00.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.299+0000 7f2be3cc6640 1 -- 172.21.15.1:0/565560466 shutdown_connections 2024-09-11T21:56:00.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.299+0000 7f2be3cc6640 1 -- 172.21.15.1:0/565560466 >> 172.21.15.1:0/565560466 conn(0x7f2bdc0fc7a0 msgr2=0x7f2bdc0fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:00.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.299+0000 7f2be3cc6640 1 -- 172.21.15.1:0/565560466 shutdown_connections 2024-09-11T21:56:00.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.299+0000 7f2be3cc6640 1 -- 172.21.15.1:0/565560466 wait complete. 2024-09-11T21:56:00.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.154+0000 7f6ab1407640 1 -- 172.21.15.1:0/3592381226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6aac102570 msgr2=0x7f6aac1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:00.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.154+0000 7f6ab1407640 1 --2- 172.21.15.1:0/3592381226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6aac102570 0x7f6aac1029b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0009900 tx=0x7f6aa002a480 comp rx=0 tx=0).stop 2024-09-11T21:56:00.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.155+0000 7f6ab1407640 1 -- 172.21.15.1:0/3592381226 shutdown_connections 2024-09-11T21:56:00.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.155+0000 7f6ab1407640 1 --2- 172.21.15.1:0/3592381226 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6aac102ef0 0x7f6aac10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:00.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.155+0000 7f6ab1407640 1 --2- 172.21.15.1:0/3592381226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6aac102570 0x7f6aac1029b0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:00.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.155+0000 7f6ab1407640 1 --2- 172.21.15.1:0/3592381226 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6aac100e20 0x7f6aac101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:00.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.155+0000 7f6ab1407640 1 -- 172.21.15.1:0/3592381226 >> 172.21.15.1:0/3592381226 conn(0x7f6aac0fc7d0 msgr2=0x7f6aac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:00.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.155+0000 7f6ab1407640 1 -- 172.21.15.1:0/3592381226 shutdown_connections 2024-09-11T21:56:00.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.155+0000 7f6ab1407640 1 -- 172.21.15.1:0/3592381226 wait complete. 2024-09-11T21:56:00.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.155+0000 7f6ab1407640 1 Processor -- start 2024-09-11T21:56:00.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.155+0000 7f6ab1407640 1 -- start start 2024-09-11T21:56:00.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.156+0000 7f6ab1407640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6aac100e20 0x7f6aac199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:00.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.156+0000 7f6ab1407640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6aac102570 0x7f6aac199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:00.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.156+0000 7f6ab1407640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6aac102ef0 0x7f6aac1a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:00.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.156+0000 7f6ab1407640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6aac110f80 con 0x7f6aac102570 2024-09-11T21:56:00.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.156+0000 7f6ab1407640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6aac110e00 con 0x7f6aac100e20 2024-09-11T21:56:00.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.156+0000 7f6ab1407640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6aac111100 con 0x7f6aac102ef0 2024-09-11T21:56:00.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.156+0000 7f6aaa7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6aac102570 0x7f6aac199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:00.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.156+0000 7f6aaa7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6aac102570 0x7f6aac199870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56764/0 (socket says 172.21.15.1:56764) 2024-09-11T21:56:00.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.156+0000 7f6aaa7fc640 1 -- 172.21.15.1:0/826864192 learned_addr learned my addr 172.21.15.1:0/826864192 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:56:00.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.156+0000 7f6aaaffd640 1 --2- 172.21.15.1:0/826864192 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6aac100e20 0x7f6aac199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:00.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.156+0000 7f6aab7fe640 1 --2- 172.21.15.1:0/826864192 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6aac102ef0 0x7f6aac1a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:00.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.156+0000 7f6aaa7fc640 1 -- 172.21.15.1:0/826864192 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6aac102ef0 msgr2=0x7f6aac1a43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:00.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.156+0000 7f6aaa7fc640 1 --2- 172.21.15.1:0/826864192 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6aac102ef0 0x7f6aac1a43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:00.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.156+0000 7f6aaa7fc640 1 -- 172.21.15.1:0/826864192 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6aac100e20 msgr2=0x7f6aac199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:00.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.156+0000 7f6aaa7fc640 1 --2- 172.21.15.1:0/826864192 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6aac100e20 0x7f6aac199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:00.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.156+0000 7f6aaa7fc640 1 -- 172.21.15.1:0/826864192 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6aac1a48f0 con 0x7f6aac102570 2024-09-11T21:56:00.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.156+0000 7f6aab7fe640 1 --2- 172.21.15.1:0/826864192 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6aac102ef0 0x7f6aac1a43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:56:00.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.156+0000 7f6aaa7fc640 1 --2- 172.21.15.1:0/826864192 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6aac102570 0x7f6aac199870 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0006010 tx=0x7f6aa0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:00.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.156+0000 7f6a93fff640 1 -- 172.21.15.1:0/826864192 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa003f070 con 0x7f6aac102570 2024-09-11T21:56:00.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.156+0000 7f6ab1407640 1 -- 172.21.15.1:0/826864192 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6aac1a4ac0 con 0x7f6aac102570 2024-09-11T21:56:00.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.156+0000 7f6a93fff640 1 -- 172.21.15.1:0/826864192 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6aa0031c30 con 0x7f6aac102570 2024-09-11T21:56:00.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.156+0000 7f6a93fff640 1 -- 172.21.15.1:0/826864192 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa0035030 con 0x7f6aac102570 2024-09-11T21:56:00.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.157+0000 7f6ab1407640 1 -- 172.21.15.1:0/826864192 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6aac1a4f70 con 0x7f6aac102570 2024-09-11T21:56:00.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.157+0000 7f6ab1407640 1 -- 172.21.15.1:0/826864192 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6aac107300 con 0x7f6aac102570 2024-09-11T21:56:00.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.159+0000 7f6a93fff640 1 -- 172.21.15.1:0/826864192 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f6aa0042070 con 0x7f6aac102570 2024-09-11T21:56:00.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.159+0000 7f6a93fff640 1 --2- 172.21.15.1:0/826864192 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f6a940a2c10 0x7f6a940a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:00.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.159+0000 7f6aaaffd640 1 --2- 172.21.15.1:0/826864192 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f6a940a2c10 0x7f6a940a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:00.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.159+0000 7f6a93fff640 1 -- 172.21.15.1:0/826864192 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f6aa00f01d0 con 0x7f6aac102570 2024-09-11T21:56:00.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.160+0000 7f6a93fff640 1 --2- 172.21.15.1:0/826864192 >> [v2:172.21.15.1:6810/1541142398,v1:172.21.15.1:6811/1541142398] conn(0x7f6a940ac460 0x7f6a940ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:00.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.160+0000 7f6a93fff640 1 -- 172.21.15.1:0/826864192 --> [v2:172.21.15.1:6810/1541142398,v1:172.21.15.1:6811/1541142398] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6a940aef30 con 0x7f6a940ac460 2024-09-11T21:56:00.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.160+0000 7f6a93fff640 1 -- 172.21.15.1:0/826864192 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f6aa00f0580 con 0x7f6aac102570 2024-09-11T21:56:00.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.160+0000 7f6aab7fe640 1 --2- 172.21.15.1:0/826864192 >> [v2:172.21.15.1:6810/1541142398,v1:172.21.15.1:6811/1541142398] conn(0x7f6a940ac460 0x7f6a940ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:00.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.160+0000 7f6aaaffd640 1 --2- 172.21.15.1:0/826864192 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f6a940a2c10 0x7f6a940a50d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c00bbe0 tx=0x7f6a9c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:00.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.160+0000 7f6aab7fe640 1 --2- 172.21.15.1:0/826864192 >> [v2:172.21.15.1:6810/1541142398,v1:172.21.15.1:6811/1541142398] conn(0x7f6a940ac460 0x7f6a940ae880 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:00.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.161+0000 7f6a93fff640 1 -- 172.21.15.1:0/826864192 <== osd.3 v2:172.21.15.1:6810/1541142398 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6a940aef30 con 0x7f6a940ac460 2024-09-11T21:56:00.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.174+0000 7f6ab1407640 1 -- 172.21.15.1:0/826864192 --> [v2:172.21.15.1:6810/1541142398,v1:172.21.15.1:6811/1541142398] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6aac1021b0 con 0x7f6a940ac460 2024-09-11T21:56:00.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.174+0000 7f6a93fff640 1 -- 172.21.15.1:0/826864192 <== osd.3 v2:172.21.15.1:6810/1541142398 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6aac1021b0 con 0x7f6a940ac460 2024-09-11T21:56:00.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.175+0000 7f6ab1407640 1 -- 172.21.15.1:0/826864192 >> [v2:172.21.15.1:6810/1541142398,v1:172.21.15.1:6811/1541142398] conn(0x7f6a940ac460 msgr2=0x7f6a940ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:00.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.175+0000 7f6ab1407640 1 --2- 172.21.15.1:0/826864192 >> [v2:172.21.15.1:6810/1541142398,v1:172.21.15.1:6811/1541142398] conn(0x7f6a940ac460 0x7f6a940ae880 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:00.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.175+0000 7f6ab1407640 1 -- 172.21.15.1:0/826864192 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f6a940a2c10 msgr2=0x7f6a940a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:00.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.175+0000 7f6ab1407640 1 --2- 172.21.15.1:0/826864192 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f6a940a2c10 0x7f6a940a50d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c00bbe0 tx=0x7f6a9c00b390 comp rx=0 tx=0).stop 2024-09-11T21:56:00.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.175+0000 7f6ab1407640 1 -- 172.21.15.1:0/826864192 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6aac102570 msgr2=0x7f6aac199870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:00.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.175+0000 7f6ab1407640 1 --2- 172.21.15.1:0/826864192 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6aac102570 0x7f6aac199870 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0006010 tx=0x7f6aa0003c10 comp rx=0 tx=0).stop 2024-09-11T21:56:00.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.175+0000 7f6aab7fe640 1 -- 172.21.15.1:0/826864192 reap_dead start 2024-09-11T21:56:00.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.175+0000 7f6ab1407640 1 -- 172.21.15.1:0/826864192 shutdown_connections 2024-09-11T21:56:00.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.175+0000 7f6ab1407640 1 -- 172.21.15.1:0/826864192 >> 172.21.15.1:0/826864192 conn(0x7f6aac0fc7d0 msgr2=0x7f6aac0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:00.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.175+0000 7f6ab1407640 1 -- 172.21.15.1:0/826864192 shutdown_connections 2024-09-11T21:56:00.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:00.175+0000 7f6ab1407640 1 -- 172.21.15.1:0/826864192 wait complete. 2024-09-11T21:56:00.402 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:00 smithi001 ceph-mon[30935]: pgmap v301: 97 pgs: 97 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-11T21:56:00.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:00 smithi053 ceph-mon[34908]: pgmap v301: 97 pgs: 97 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-11T21:56:00.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:00 smithi161 ceph-mon[35615]: pgmap v301: 97 pgs: 97 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-11T21:56:02.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.101+0000 7fafe4a0e640 1 -- 172.21.15.1:0/1083926030 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fafe0102ef0 msgr2=0x7fafe010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:02.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.101+0000 7fafe4a0e640 1 --2- 172.21.15.1:0/1083926030 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fafe0102ef0 0x7fafe010da40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fafd8009ca0 tx=0x7fafd802a950 comp rx=0 tx=0).stop 2024-09-11T21:56:02.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.101+0000 7fafe4a0e640 1 -- 172.21.15.1:0/1083926030 shutdown_connections 2024-09-11T21:56:02.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.101+0000 7fafe4a0e640 1 --2- 172.21.15.1:0/1083926030 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fafe0102ef0 0x7fafe010da40 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:02.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.101+0000 7fafe4a0e640 1 --2- 172.21.15.1:0/1083926030 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fafe0102570 0x7fafe01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:02.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.101+0000 7fafe4a0e640 1 --2- 172.21.15.1:0/1083926030 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafe0100e20 0x7fafe0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:02.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.101+0000 7fafe4a0e640 1 -- 172.21.15.1:0/1083926030 >> 172.21.15.1:0/1083926030 conn(0x7fafe00fc7d0 msgr2=0x7fafe00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:02.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.102+0000 7fafe4a0e640 1 -- 172.21.15.1:0/1083926030 shutdown_connections 2024-09-11T21:56:02.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.102+0000 7fafe4a0e640 1 -- 172.21.15.1:0/1083926030 wait complete. 2024-09-11T21:56:02.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.102+0000 7fafe4a0e640 1 Processor -- start 2024-09-11T21:56:02.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.102+0000 7fafe4a0e640 1 -- start start 2024-09-11T21:56:02.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.103+0000 7fafe4a0e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fafe0100e20 0x7fafe00fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:02.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.103+0000 7fafe4a0e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafe0102570 0x7fafe00fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:02.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.103+0000 7fafe4a0e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fafe0102ef0 0x7fafe00fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:02.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.103+0000 7fafe4a0e640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fafe0110f80 con 0x7fafe0102ef0 2024-09-11T21:56:02.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.103+0000 7fafe4a0e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fafe0110e00 con 0x7fafe0102570 2024-09-11T21:56:02.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.103+0000 7fafe4a0e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fafe0111100 con 0x7fafe0100e20 2024-09-11T21:56:02.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.103+0000 7fafded9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fafe0102ef0 0x7fafe00fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:02.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.103+0000 7fafded9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fafe0102ef0 0x7fafe00fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56848/0 (socket says 172.21.15.1:56848) 2024-09-11T21:56:02.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.103+0000 7fafded9d640 1 -- 172.21.15.1:0/1717242646 learned_addr learned my addr 172.21.15.1:0/1717242646 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:56:02.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.103+0000 7fafd5d9b640 1 --2- 172.21.15.1:0/1717242646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafe0102570 0x7fafe00fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:02.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.103+0000 7fafde59c640 1 --2- 172.21.15.1:0/1717242646 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fafe0100e20 0x7fafe00fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:02.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.103+0000 7fafded9d640 1 -- 172.21.15.1:0/1717242646 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fafe0100e20 msgr2=0x7fafe00fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:02.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.103+0000 7fafded9d640 1 --2- 172.21.15.1:0/1717242646 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fafe0100e20 0x7fafe00fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:02.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.103+0000 7fafded9d640 1 -- 172.21.15.1:0/1717242646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafe0102570 msgr2=0x7fafe00fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:02.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.103+0000 7fafded9d640 1 --2- 172.21.15.1:0/1717242646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fafe0102570 0x7fafe00fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:02.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.103+0000 7fafded9d640 1 -- 172.21.15.1:0/1717242646 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafe01a6910 con 0x7fafe0102ef0 2024-09-11T21:56:02.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.103+0000 7fafded9d640 1 --2- 172.21.15.1:0/1717242646 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fafe0102ef0 0x7fafe00fd700 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fafd802ae70 tx=0x7fafd802f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:02.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.104+0000 7fafd7fff640 1 -- 172.21.15.1:0/1717242646 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd8034040 con 0x7fafe0102ef0 2024-09-11T21:56:02.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.104+0000 7fafe4a0e640 1 -- 172.21.15.1:0/1717242646 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafe01a6ba0 con 0x7fafe0102ef0 2024-09-11T21:56:02.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.104+0000 7fafd7fff640 1 -- 172.21.15.1:0/1717242646 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafd8030920 con 0x7fafe0102ef0 2024-09-11T21:56:02.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.104+0000 7fafd7fff640 1 -- 172.21.15.1:0/1717242646 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd8045480 con 0x7fafe0102ef0 2024-09-11T21:56:02.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.104+0000 7fafe4a0e640 1 -- 172.21.15.1:0/1717242646 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafe01a7080 con 0x7fafe0102ef0 2024-09-11T21:56:02.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.104+0000 7fafe4a0e640 1 -- 172.21.15.1:0/1717242646 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fafe0062940 con 0x7fafe0102ef0 2024-09-11T21:56:02.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.106+0000 7fafd7fff640 1 -- 172.21.15.1:0/1717242646 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fafd8037020 con 0x7fafe0102ef0 2024-09-11T21:56:02.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.107+0000 7fafd7fff640 1 --2- 172.21.15.1:0/1717242646 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fafcc0a2c10 0x7fafcc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:02.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.107+0000 7fafde59c640 1 --2- 172.21.15.1:0/1717242646 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fafcc0a2c10 0x7fafcc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:02.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.107+0000 7fafd7fff640 1 -- 172.21.15.1:0/1717242646 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7fafd80f0300 con 0x7fafe0102ef0 2024-09-11T21:56:02.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.107+0000 7fafd7fff640 1 --2- 172.21.15.1:0/1717242646 >> [v2:172.21.15.53:6824/761923580,v1:172.21.15.53:6825/761923580] conn(0x7fafcc0ac460 0x7fafcc0ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:02.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.107+0000 7fafd7fff640 1 -- 172.21.15.1:0/1717242646 --> [v2:172.21.15.53:6824/761923580,v1:172.21.15.53:6825/761923580] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fafcc0aef30 con 0x7fafcc0ac460 2024-09-11T21:56:02.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.107+0000 7fafd7fff640 1 -- 172.21.15.1:0/1717242646 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7fafd80f06b0 con 0x7fafe0102ef0 2024-09-11T21:56:02.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.107+0000 7fafde59c640 1 --2- 172.21.15.1:0/1717242646 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fafcc0a2c10 0x7fafcc0a50d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fafd00058c0 tx=0x7fafd000a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:02.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.107+0000 7fafd5d9b640 1 --2- 172.21.15.1:0/1717242646 >> [v2:172.21.15.53:6824/761923580,v1:172.21.15.53:6825/761923580] conn(0x7fafcc0ac460 0x7fafcc0ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:02.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.108+0000 7fafd5d9b640 1 --2- 172.21.15.1:0/1717242646 >> [v2:172.21.15.53:6824/761923580,v1:172.21.15.53:6825/761923580] conn(0x7fafcc0ac460 0x7fafcc0ae880 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:02.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.108+0000 7fafd7fff640 1 -- 172.21.15.1:0/1717242646 <== osd.10 v2:172.21.15.53:6824/761923580 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fafcc0aef30 con 0x7fafcc0ac460 2024-09-11T21:56:02.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.122+0000 7fafe4a0e640 1 -- 172.21.15.1:0/1717242646 --> [v2:172.21.15.53:6824/761923580,v1:172.21.15.53:6825/761923580] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fafe0062dc0 con 0x7fafcc0ac460 2024-09-11T21:56:02.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.123+0000 7fafd7fff640 1 -- 172.21.15.1:0/1717242646 <== osd.10 v2:172.21.15.53:6824/761923580 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fafe0062dc0 con 0x7fafcc0ac460 2024-09-11T21:56:02.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.123+0000 7fafe4a0e640 1 -- 172.21.15.1:0/1717242646 >> [v2:172.21.15.53:6824/761923580,v1:172.21.15.53:6825/761923580] conn(0x7fafcc0ac460 msgr2=0x7fafcc0ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:02.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.123+0000 7fafe4a0e640 1 --2- 172.21.15.1:0/1717242646 >> [v2:172.21.15.53:6824/761923580,v1:172.21.15.53:6825/761923580] conn(0x7fafcc0ac460 0x7fafcc0ae880 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:02.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.123+0000 7fafe4a0e640 1 -- 172.21.15.1:0/1717242646 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fafcc0a2c10 msgr2=0x7fafcc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:02.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.123+0000 7fafe4a0e640 1 --2- 172.21.15.1:0/1717242646 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fafcc0a2c10 0x7fafcc0a50d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fafd00058c0 tx=0x7fafd000a4c0 comp rx=0 tx=0).stop 2024-09-11T21:56:02.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.123+0000 7fafe4a0e640 1 -- 172.21.15.1:0/1717242646 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fafe0102ef0 msgr2=0x7fafe00fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:02.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.123+0000 7fafe4a0e640 1 --2- 172.21.15.1:0/1717242646 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fafe0102ef0 0x7fafe00fd700 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fafd802ae70 tx=0x7fafd802f5d0 comp rx=0 tx=0).stop 2024-09-11T21:56:02.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.123+0000 7fafded9d640 1 -- 172.21.15.1:0/1717242646 reap_dead start 2024-09-11T21:56:02.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.123+0000 7fafe4a0e640 1 -- 172.21.15.1:0/1717242646 shutdown_connections 2024-09-11T21:56:02.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.123+0000 7fafe4a0e640 1 -- 172.21.15.1:0/1717242646 >> 172.21.15.1:0/1717242646 conn(0x7fafe00fc7d0 msgr2=0x7fafe006a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:02.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.123+0000 7fafe4a0e640 1 -- 172.21.15.1:0/1717242646 shutdown_connections 2024-09-11T21:56:02.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.123+0000 7fafe4a0e640 1 -- 172.21.15.1:0/1717242646 wait complete. 2024-09-11T21:56:02.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.875+0000 7efc48feb640 1 -- 172.21.15.1:0/318335699 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc44102790 msgr2=0x7efc441067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:02.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.875+0000 7efc48feb640 1 --2- 172.21.15.1:0/318335699 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc44102790 0x7efc441067e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7efc38009900 tx=0x7efc3802a410 comp rx=0 tx=0).stop 2024-09-11T21:56:02.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.875+0000 7efc48feb640 1 -- 172.21.15.1:0/318335699 shutdown_connections 2024-09-11T21:56:02.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.875+0000 7efc48feb640 1 --2- 172.21.15.1:0/318335699 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc44106f10 0x7efc4410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:02.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.875+0000 7efc48feb640 1 --2- 172.21.15.1:0/318335699 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc44102790 0x7efc441067e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:02.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.875+0000 7efc48feb640 1 --2- 172.21.15.1:0/318335699 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc44101de0 0x7efc441021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:02.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.875+0000 7efc48feb640 1 -- 172.21.15.1:0/318335699 >> 172.21.15.1:0/318335699 conn(0x7efc440fd8c0 msgr2=0x7efc440ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:02.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.875+0000 7efc48feb640 1 -- 172.21.15.1:0/318335699 shutdown_connections 2024-09-11T21:56:02.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.875+0000 7efc48feb640 1 -- 172.21.15.1:0/318335699 wait complete. 2024-09-11T21:56:02.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.876+0000 7efc48feb640 1 Processor -- start 2024-09-11T21:56:02.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.876+0000 7efc48feb640 1 -- start start 2024-09-11T21:56:02.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.876+0000 7efc48feb640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc44101de0 0x7efc4419a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:02.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.876+0000 7efc48feb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc44102790 0x7efc4419ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:02.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.876+0000 7efc48feb640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc44106f10 0x7efc4419ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:02.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.876+0000 7efc48feb640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7efc44111f70 con 0x7efc44106f10 2024-09-11T21:56:02.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.876+0000 7efc48feb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7efc44111df0 con 0x7efc44102790 2024-09-11T21:56:02.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.876+0000 7efc48feb640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7efc441120f0 con 0x7efc44101de0 2024-09-11T21:56:02.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.876+0000 7efc42d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc44106f10 0x7efc4419ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:02.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.876+0000 7efc4259c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc44101de0 0x7efc4419a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:02.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.876+0000 7efc4259c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc44101de0 0x7efc4419a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:40814/0 (socket says 172.21.15.1:40814) 2024-09-11T21:56:02.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.876+0000 7efc42d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc44106f10 0x7efc4419ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56788/0 (socket says 172.21.15.1:56788) 2024-09-11T21:56:02.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.876+0000 7efc4259c640 1 -- 172.21.15.1:0/1258869798 learned_addr learned my addr 172.21.15.1:0/1258869798 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:56:02.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.876+0000 7efc41d9b640 1 --2- 172.21.15.1:0/1258869798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc44102790 0x7efc4419ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:02.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.877+0000 7efc42d9d640 1 -- 172.21.15.1:0/1258869798 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc44101de0 msgr2=0x7efc4419a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:02.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.877+0000 7efc42d9d640 1 --2- 172.21.15.1:0/1258869798 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc44101de0 0x7efc4419a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:02.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.877+0000 7efc42d9d640 1 -- 172.21.15.1:0/1258869798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc44102790 msgr2=0x7efc4419ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:02.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.877+0000 7efc42d9d640 1 --2- 172.21.15.1:0/1258869798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc44102790 0x7efc4419ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:02.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.877+0000 7efc42d9d640 1 -- 172.21.15.1:0/1258869798 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc4419f600 con 0x7efc44106f10 2024-09-11T21:56:02.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.877+0000 7efc4259c640 1 --2- 172.21.15.1:0/1258869798 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc44101de0 0x7efc4419a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T21:56:02.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.877+0000 7efc42d9d640 1 --2- 172.21.15.1:0/1258869798 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc44106f10 0x7efc4419ef20 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7efc3c00e940 tx=0x7efc3c00ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:02.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.877+0000 7efc337fe640 1 -- 172.21.15.1:0/1258869798 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc3c004d20 con 0x7efc44106f10 2024-09-11T21:56:02.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.877+0000 7efc48feb640 1 -- 172.21.15.1:0/1258869798 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc4419f8f0 con 0x7efc44106f10 2024-09-11T21:56:02.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.877+0000 7efc337fe640 1 -- 172.21.15.1:0/1258869798 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc3c005020 con 0x7efc44106f10 2024-09-11T21:56:02.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.877+0000 7efc337fe640 1 -- 172.21.15.1:0/1258869798 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc3c0116d0 con 0x7efc44106f10 2024-09-11T21:56:02.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.877+0000 7efc48feb640 1 -- 172.21.15.1:0/1258869798 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc441a71d0 con 0x7efc44106f10 2024-09-11T21:56:02.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.878+0000 7efc48feb640 1 -- 172.21.15.1:0/1258869798 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7efc44075780 con 0x7efc44106f10 2024-09-11T21:56:02.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.879+0000 7efc337fe640 1 -- 172.21.15.1:0/1258869798 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7efc3c0054d0 con 0x7efc44106f10 2024-09-11T21:56:02.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.880+0000 7efc337fe640 1 --2- 172.21.15.1:0/1258869798 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7efc2c0a2c10 0x7efc2c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:02.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.880+0000 7efc4259c640 1 --2- 172.21.15.1:0/1258869798 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7efc2c0a2c10 0x7efc2c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:02.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.880+0000 7efc337fe640 1 -- 172.21.15.1:0/1258869798 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7efc3c0cf470 con 0x7efc44106f10 2024-09-11T21:56:02.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.880+0000 7efc337fe640 1 --2- 172.21.15.1:0/1258869798 >> [v2:172.21.15.161:6800/1707914885,v1:172.21.15.161:6801/1707914885] conn(0x7efc2c0ac460 0x7efc2c0ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:02.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.880+0000 7efc337fe640 1 -- 172.21.15.1:0/1258869798 --> [v2:172.21.15.161:6800/1707914885,v1:172.21.15.161:6801/1707914885] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efc3c014ec0 con 0x7efc2c0ac460 2024-09-11T21:56:02.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.880+0000 7efc337fe640 1 -- 172.21.15.1:0/1258869798 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7efc3c0d6050 con 0x7efc44106f10 2024-09-11T21:56:02.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.880+0000 7efc4259c640 1 --2- 172.21.15.1:0/1258869798 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7efc2c0a2c10 0x7efc2c0a50d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7efc34005fd0 tx=0x7efc3400b4a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:02.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.881+0000 7efc41d9b640 1 --2- 172.21.15.1:0/1258869798 >> [v2:172.21.15.161:6800/1707914885,v1:172.21.15.161:6801/1707914885] conn(0x7efc2c0ac460 0x7efc2c0ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:02.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.881+0000 7efc41d9b640 1 --2- 172.21.15.1:0/1258869798 >> [v2:172.21.15.161:6800/1707914885,v1:172.21.15.161:6801/1707914885] conn(0x7efc2c0ac460 0x7efc2c0ae880 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:02.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.882+0000 7efc337fe640 1 -- 172.21.15.1:0/1258869798 <== osd.2 v2:172.21.15.161:6800/1707914885 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efc3c014ec0 con 0x7efc2c0ac460 2024-09-11T21:56:02.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.896+0000 7efc48feb640 1 -- 172.21.15.1:0/1258869798 --> [v2:172.21.15.161:6800/1707914885,v1:172.21.15.161:6801/1707914885] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7efc441032c0 con 0x7efc2c0ac460 2024-09-11T21:56:02.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.896+0000 7efc337fe640 1 -- 172.21.15.1:0/1258869798 <== osd.2 v2:172.21.15.161:6800/1707914885 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7efc441032c0 con 0x7efc2c0ac460 2024-09-11T21:56:02.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.897+0000 7efc48feb640 1 -- 172.21.15.1:0/1258869798 >> [v2:172.21.15.161:6800/1707914885,v1:172.21.15.161:6801/1707914885] conn(0x7efc2c0ac460 msgr2=0x7efc2c0ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:02.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.897+0000 7efc48feb640 1 --2- 172.21.15.1:0/1258869798 >> [v2:172.21.15.161:6800/1707914885,v1:172.21.15.161:6801/1707914885] conn(0x7efc2c0ac460 0x7efc2c0ae880 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:02.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.897+0000 7efc48feb640 1 -- 172.21.15.1:0/1258869798 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7efc2c0a2c10 msgr2=0x7efc2c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:02.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.897+0000 7efc48feb640 1 --2- 172.21.15.1:0/1258869798 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7efc2c0a2c10 0x7efc2c0a50d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7efc34005fd0 tx=0x7efc3400b4a0 comp rx=0 tx=0).stop 2024-09-11T21:56:02.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.897+0000 7efc48feb640 1 -- 172.21.15.1:0/1258869798 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc44106f10 msgr2=0x7efc4419ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:02.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.897+0000 7efc48feb640 1 --2- 172.21.15.1:0/1258869798 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc44106f10 0x7efc4419ef20 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7efc3c00e940 tx=0x7efc3c00ee00 comp rx=0 tx=0).stop 2024-09-11T21:56:02.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.897+0000 7efc42d9d640 1 -- 172.21.15.1:0/1258869798 reap_dead start 2024-09-11T21:56:02.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.897+0000 7efc48feb640 1 -- 172.21.15.1:0/1258869798 shutdown_connections 2024-09-11T21:56:02.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.897+0000 7efc48feb640 1 -- 172.21.15.1:0/1258869798 >> 172.21.15.1:0/1258869798 conn(0x7efc440fd8c0 msgr2=0x7efc440fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:02.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.897+0000 7efc48feb640 1 -- 172.21.15.1:0/1258869798 shutdown_connections 2024-09-11T21:56:02.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.897+0000 7efc48feb640 1 -- 172.21.15.1:0/1258869798 wait complete. 2024-09-11T21:56:02.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:02 smithi001 ceph-mon[30935]: pgmap v302: 97 pgs: 97 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-11T21:56:02.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:02 smithi053 ceph-mon[34908]: pgmap v302: 97 pgs: 97 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-11T21:56:02.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:02 smithi161 ceph-mon[35615]: pgmap v302: 97 pgs: 97 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-11T21:56:02.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.668+0000 7f60ec079640 1 -- 172.21.15.1:0/909498797 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60e4106a80 msgr2=0x7f60e4106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:02.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.668+0000 7f60ec079640 1 --2- 172.21.15.1:0/909498797 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60e4106a80 0x7f60e4106e60 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f60e0009930 tx=0x7f60e002a420 comp rx=0 tx=0).stop 2024-09-11T21:56:02.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.668+0000 7f60ec079640 1 -- 172.21.15.1:0/909498797 shutdown_connections 2024-09-11T21:56:02.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.668+0000 7f60ec079640 1 --2- 172.21.15.1:0/909498797 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60e4101420 0x7f60e410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:02.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.668+0000 7f60ec079640 1 --2- 172.21.15.1:0/909498797 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60e4100a80 0x7f60e4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:02.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.668+0000 7f60ec079640 1 --2- 172.21.15.1:0/909498797 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60e4106a80 0x7f60e4106e60 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:02.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.668+0000 7f60ec079640 1 -- 172.21.15.1:0/909498797 >> 172.21.15.1:0/909498797 conn(0x7f60e40fc7a0 msgr2=0x7f60e40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:02.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.668+0000 7f60ec079640 1 -- 172.21.15.1:0/909498797 shutdown_connections 2024-09-11T21:56:02.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.668+0000 7f60ec079640 1 -- 172.21.15.1:0/909498797 wait complete. 2024-09-11T21:56:02.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.669+0000 7f60ec079640 1 Processor -- start 2024-09-11T21:56:02.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.669+0000 7f60ec079640 1 -- start start 2024-09-11T21:56:02.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.669+0000 7f60ec079640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60e4100a80 0x7f60e4197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:02.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.669+0000 7f60ec079640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60e4101420 0x7f60e4197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:02.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.669+0000 7f60ec079640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60e4106a80 0x7f60e419bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:02.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.669+0000 7f60ec079640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f60e4110e80 con 0x7f60e4106a80 2024-09-11T21:56:02.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.669+0000 7f60ec079640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f60e4110d00 con 0x7f60e4100a80 2024-09-11T21:56:02.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.669+0000 7f60ec079640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f60e4111000 con 0x7f60e4101420 2024-09-11T21:56:02.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.669+0000 7f60ea616640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60e4106a80 0x7f60e419bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:02.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.669+0000 7f60ea616640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60e4106a80 0x7f60e419bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56874/0 (socket says 172.21.15.1:56874) 2024-09-11T21:56:02.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.669+0000 7f60ea616640 1 -- 172.21.15.1:0/3742488947 learned_addr learned my addr 172.21.15.1:0/3742488947 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:56:02.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.669+0000 7f60e9e15640 1 --2- 172.21.15.1:0/3742488947 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60e4100a80 0x7f60e4197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:02.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.669+0000 7f60e9614640 1 --2- 172.21.15.1:0/3742488947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60e4101420 0x7f60e4197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:02.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.669+0000 7f60ea616640 1 -- 172.21.15.1:0/3742488947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60e4101420 msgr2=0x7f60e4197880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:02.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.669+0000 7f60ea616640 1 --2- 172.21.15.1:0/3742488947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60e4101420 0x7f60e4197880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:02.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.669+0000 7f60ea616640 1 -- 172.21.15.1:0/3742488947 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60e4100a80 msgr2=0x7f60e4197340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:02.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.669+0000 7f60ea616640 1 --2- 172.21.15.1:0/3742488947 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60e4100a80 0x7f60e4197340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:02.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.669+0000 7f60ea616640 1 -- 172.21.15.1:0/3742488947 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60e419c2f0 con 0x7f60e4106a80 2024-09-11T21:56:02.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.670+0000 7f60ea616640 1 --2- 172.21.15.1:0/3742488947 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60e4106a80 0x7f60e419bc10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f60dc00eaa0 tx=0x7f60dc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:02.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.670+0000 7f60daffd640 1 -- 172.21.15.1:0/3742488947 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60dc00cdb0 con 0x7f60e4106a80 2024-09-11T21:56:02.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.670+0000 7f60ec079640 1 -- 172.21.15.1:0/3742488947 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60e419c5e0 con 0x7f60e4106a80 2024-09-11T21:56:02.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.670+0000 7f60daffd640 1 -- 172.21.15.1:0/3742488947 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60dc005020 con 0x7f60e4106a80 2024-09-11T21:56:02.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.670+0000 7f60daffd640 1 -- 172.21.15.1:0/3742488947 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60dc0116d0 con 0x7f60e4106a80 2024-09-11T21:56:02.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.670+0000 7f60ec079640 1 -- 172.21.15.1:0/3742488947 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60e41a3ec0 con 0x7f60e4106a80 2024-09-11T21:56:02.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.671+0000 7f60ec079640 1 -- 172.21.15.1:0/3742488947 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f60e4062980 con 0x7f60e4106a80 2024-09-11T21:56:02.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.672+0000 7f60daffd640 1 -- 172.21.15.1:0/3742488947 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f60dc0054d0 con 0x7f60e4106a80 2024-09-11T21:56:02.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.673+0000 7f60daffd640 1 --2- 172.21.15.1:0/3742488947 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f60d00a2c10 0x7f60d00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:02.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.673+0000 7f60e9e15640 1 --2- 172.21.15.1:0/3742488947 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f60d00a2c10 0x7f60d00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:02.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.673+0000 7f60daffd640 1 -- 172.21.15.1:0/3742488947 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f60dc0cf760 con 0x7f60e4106a80 2024-09-11T21:56:02.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.673+0000 7f60daffd640 1 --2- 172.21.15.1:0/3742488947 >> [v2:172.21.15.161:6808/1622081327,v1:172.21.15.161:6809/1622081327] conn(0x7f60d00ac460 0x7f60d00ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:02.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.673+0000 7f60daffd640 1 -- 172.21.15.1:0/3742488947 --> [v2:172.21.15.161:6808/1622081327,v1:172.21.15.161:6809/1622081327] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f60dc014ec0 con 0x7f60d00ac460 2024-09-11T21:56:02.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.673+0000 7f60daffd640 1 -- 172.21.15.1:0/3742488947 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f60dc0cfb10 con 0x7f60e4106a80 2024-09-11T21:56:02.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.673+0000 7f60e9e15640 1 --2- 172.21.15.1:0/3742488947 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f60d00a2c10 0x7f60d00a50d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f60e0000c00 tx=0x7f60e0005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:02.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.674+0000 7f60e9614640 1 --2- 172.21.15.1:0/3742488947 >> [v2:172.21.15.161:6808/1622081327,v1:172.21.15.161:6809/1622081327] conn(0x7f60d00ac460 0x7f60d00ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:02.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.676+0000 7f60e9614640 1 --2- 172.21.15.1:0/3742488947 >> [v2:172.21.15.161:6808/1622081327,v1:172.21.15.161:6809/1622081327] conn(0x7f60d00ac460 0x7f60d00ae880 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:02.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.677+0000 7f60daffd640 1 -- 172.21.15.1:0/3742488947 <== osd.5 v2:172.21.15.161:6808/1622081327 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f60dc014ec0 con 0x7f60d00ac460 2024-09-11T21:56:02.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.691+0000 7f60ec079640 1 -- 172.21.15.1:0/3742488947 --> [v2:172.21.15.161:6808/1622081327,v1:172.21.15.161:6809/1622081327] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f60e4062d80 con 0x7f60d00ac460 2024-09-11T21:56:02.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.691+0000 7f60daffd640 1 -- 172.21.15.1:0/3742488947 <== osd.5 v2:172.21.15.161:6808/1622081327 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f60e4062d80 con 0x7f60d00ac460 2024-09-11T21:56:02.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.692+0000 7f60ec079640 1 -- 172.21.15.1:0/3742488947 >> [v2:172.21.15.161:6808/1622081327,v1:172.21.15.161:6809/1622081327] conn(0x7f60d00ac460 msgr2=0x7f60d00ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:02.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.692+0000 7f60ec079640 1 --2- 172.21.15.1:0/3742488947 >> [v2:172.21.15.161:6808/1622081327,v1:172.21.15.161:6809/1622081327] conn(0x7f60d00ac460 0x7f60d00ae880 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:02.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.692+0000 7f60ec079640 1 -- 172.21.15.1:0/3742488947 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f60d00a2c10 msgr2=0x7f60d00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:02.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.692+0000 7f60ec079640 1 --2- 172.21.15.1:0/3742488947 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f60d00a2c10 0x7f60d00a50d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f60e0000c00 tx=0x7f60e0005a20 comp rx=0 tx=0).stop 2024-09-11T21:56:02.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.692+0000 7f60ec079640 1 -- 172.21.15.1:0/3742488947 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60e4106a80 msgr2=0x7f60e419bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:02.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.692+0000 7f60ec079640 1 --2- 172.21.15.1:0/3742488947 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60e4106a80 0x7f60e419bc10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f60dc00eaa0 tx=0x7f60dc00ef60 comp rx=0 tx=0).stop 2024-09-11T21:56:02.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.692+0000 7f60ea616640 1 -- 172.21.15.1:0/3742488947 reap_dead start 2024-09-11T21:56:02.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.692+0000 7f60ec079640 1 -- 172.21.15.1:0/3742488947 shutdown_connections 2024-09-11T21:56:02.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.692+0000 7f60ec079640 1 -- 172.21.15.1:0/3742488947 >> 172.21.15.1:0/3742488947 conn(0x7f60e40fc7a0 msgr2=0x7f60e40fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:02.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.692+0000 7f60ec079640 1 -- 172.21.15.1:0/3742488947 shutdown_connections 2024-09-11T21:56:02.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.692+0000 7f60ec079640 1 -- 172.21.15.1:0/3742488947 wait complete. 2024-09-11T21:56:03.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.988+0000 7fdb9369c640 1 -- 172.21.15.1:0/3887495322 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb8c101de0 msgr2=0x7fdb8c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:03.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.988+0000 7fdb9369c640 1 --2- 172.21.15.1:0/3887495322 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb8c101de0 0x7fdb8c1021c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fdb88009900 tx=0x7fdb8802a410 comp rx=0 tx=0).stop 2024-09-11T21:56:03.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.988+0000 7fdb9369c640 1 -- 172.21.15.1:0/3887495322 shutdown_connections 2024-09-11T21:56:03.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.988+0000 7fdb9369c640 1 --2- 172.21.15.1:0/3887495322 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb8c106f10 0x7fdb8c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:03.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.988+0000 7fdb9369c640 1 --2- 172.21.15.1:0/3887495322 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdb8c102790 0x7fdb8c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:03.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.988+0000 7fdb9369c640 1 --2- 172.21.15.1:0/3887495322 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb8c101de0 0x7fdb8c1021c0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:03.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.988+0000 7fdb9369c640 1 -- 172.21.15.1:0/3887495322 >> 172.21.15.1:0/3887495322 conn(0x7fdb8c0fd8c0 msgr2=0x7fdb8c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:03.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.989+0000 7fdb9369c640 1 -- 172.21.15.1:0/3887495322 shutdown_connections 2024-09-11T21:56:03.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.989+0000 7fdb9369c640 1 -- 172.21.15.1:0/3887495322 wait complete. 2024-09-11T21:56:03.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.989+0000 7fdb9369c640 1 Processor -- start 2024-09-11T21:56:03.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.989+0000 7fdb9369c640 1 -- start start 2024-09-11T21:56:03.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.989+0000 7fdb9369c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb8c101de0 0x7fdb8c198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:03.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.989+0000 7fdb9369c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdb8c102790 0x7fdb8c198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:03.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.989+0000 7fdb9369c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb8c106f10 0x7fdb8c19ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:03.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.989+0000 7fdb9369c640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdb8c111f90 con 0x7fdb8c101de0 2024-09-11T21:56:03.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.989+0000 7fdb9369c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fdb8c111e10 con 0x7fdb8c106f10 2024-09-11T21:56:03.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.989+0000 7fdb9369c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fdb8c112110 con 0x7fdb8c102790 2024-09-11T21:56:03.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.990+0000 7fdb91438640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb8c101de0 0x7fdb8c198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:03.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.990+0000 7fdb91438640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb8c101de0 0x7fdb8c198530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56818/0 (socket says 172.21.15.1:56818) 2024-09-11T21:56:03.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.990+0000 7fdb91438640 1 -- 172.21.15.1:0/2768263523 learned_addr learned my addr 172.21.15.1:0/2768263523 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:56:03.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.990+0000 7fdb90c37640 1 --2- 172.21.15.1:0/2768263523 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdb8c102790 0x7fdb8c198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:03.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.990+0000 7fdb91438640 1 -- 172.21.15.1:0/2768263523 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdb8c102790 msgr2=0x7fdb8c198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:03.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.990+0000 7fdb91c39640 1 --2- 172.21.15.1:0/2768263523 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb8c106f10 0x7fdb8c19ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:03.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.990+0000 7fdb91438640 1 --2- 172.21.15.1:0/2768263523 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdb8c102790 0x7fdb8c198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:03.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.990+0000 7fdb91438640 1 -- 172.21.15.1:0/2768263523 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb8c106f10 msgr2=0x7fdb8c19ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:03.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.990+0000 7fdb91438640 1 --2- 172.21.15.1:0/2768263523 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb8c106f10 0x7fdb8c19ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:03.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.990+0000 7fdb91438640 1 -- 172.21.15.1:0/2768263523 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb8c19d4e0 con 0x7fdb8c101de0 2024-09-11T21:56:03.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.990+0000 7fdb91438640 1 --2- 172.21.15.1:0/2768263523 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb8c101de0 0x7fdb8c198530 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fdb88006010 tx=0x7fdb880042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:03.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.990+0000 7fdb827fc640 1 -- 172.21.15.1:0/2768263523 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8803f030 con 0x7fdb8c101de0 2024-09-11T21:56:03.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.990+0000 7fdb827fc640 1 -- 172.21.15.1:0/2768263523 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb88003ce0 con 0x7fdb8c101de0 2024-09-11T21:56:03.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.990+0000 7fdb9369c640 1 -- 172.21.15.1:0/2768263523 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb8c19d770 con 0x7fdb8c101de0 2024-09-11T21:56:03.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.990+0000 7fdb827fc640 1 -- 172.21.15.1:0/2768263523 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb88035070 con 0x7fdb8c101de0 2024-09-11T21:56:03.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.990+0000 7fdb9369c640 1 -- 172.21.15.1:0/2768263523 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb8c1a50b0 con 0x7fdb8c101de0 2024-09-11T21:56:03.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.991+0000 7fdb9369c640 1 -- 172.21.15.1:0/2768263523 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdb8c062980 con 0x7fdb8c101de0 2024-09-11T21:56:03.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.993+0000 7fdb827fc640 1 -- 172.21.15.1:0/2768263523 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fdb88042030 con 0x7fdb8c101de0 2024-09-11T21:56:03.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.993+0000 7fdb827fc640 1 --2- 172.21.15.1:0/2768263523 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fdb780a2c90 0x7fdb780a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:03.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.993+0000 7fdb90c37640 1 --2- 172.21.15.1:0/2768263523 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fdb780a2c90 0x7fdb780a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:03.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.993+0000 7fdb827fc640 1 -- 172.21.15.1:0/2768263523 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7fdb880efc70 con 0x7fdb8c101de0 2024-09-11T21:56:03.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.993+0000 7fdb827fc640 1 --2- 172.21.15.1:0/2768263523 >> [v2:172.21.15.1:6826/2806274589,v1:172.21.15.1:6827/2806274589] conn(0x7fdb780ac4e0 0x7fdb780ae900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:03.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.993+0000 7fdb827fc640 1 -- 172.21.15.1:0/2768263523 --> [v2:172.21.15.1:6826/2806274589,v1:172.21.15.1:6827/2806274589] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdb88001d30 con 0x7fdb780ac4e0 2024-09-11T21:56:03.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.993+0000 7fdb827fc640 1 -- 172.21.15.1:0/2768263523 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7fdb880f0150 con 0x7fdb8c101de0 2024-09-11T21:56:03.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.994+0000 7fdb91c39640 1 --2- 172.21.15.1:0/2768263523 >> [v2:172.21.15.1:6826/2806274589,v1:172.21.15.1:6827/2806274589] conn(0x7fdb780ac4e0 0x7fdb780ae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:03.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.994+0000 7fdb90c37640 1 --2- 172.21.15.1:0/2768263523 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fdb780a2c90 0x7fdb780a5150 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c0fdce0 tx=0x7fdb7c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:03.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.994+0000 7fdb91c39640 1 --2- 172.21.15.1:0/2768263523 >> [v2:172.21.15.1:6826/2806274589,v1:172.21.15.1:6827/2806274589] conn(0x7fdb780ac4e0 0x7fdb780ae900 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:03.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:01.995+0000 7fdb827fc640 1 -- 172.21.15.1:0/2768263523 <== osd.9 v2:172.21.15.1:6826/2806274589 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdb88001d30 con 0x7fdb780ac4e0 2024-09-11T21:56:03.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.008+0000 7fdb9369c640 1 -- 172.21.15.1:0/2768263523 --> [v2:172.21.15.1:6826/2806274589,v1:172.21.15.1:6827/2806274589] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdb8c062d80 con 0x7fdb780ac4e0 2024-09-11T21:56:03.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.008+0000 7fdb827fc640 1 -- 172.21.15.1:0/2768263523 <== osd.9 v2:172.21.15.1:6826/2806274589 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fdb8c062d80 con 0x7fdb780ac4e0 2024-09-11T21:56:03.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.009+0000 7fdb9369c640 1 -- 172.21.15.1:0/2768263523 >> [v2:172.21.15.1:6826/2806274589,v1:172.21.15.1:6827/2806274589] conn(0x7fdb780ac4e0 msgr2=0x7fdb780ae900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:03.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.009+0000 7fdb9369c640 1 --2- 172.21.15.1:0/2768263523 >> [v2:172.21.15.1:6826/2806274589,v1:172.21.15.1:6827/2806274589] conn(0x7fdb780ac4e0 0x7fdb780ae900 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:03.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.009+0000 7fdb9369c640 1 -- 172.21.15.1:0/2768263523 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fdb780a2c90 msgr2=0x7fdb780a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:03.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.009+0000 7fdb9369c640 1 --2- 172.21.15.1:0/2768263523 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fdb780a2c90 0x7fdb780a5150 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c0fdce0 tx=0x7fdb7c008040 comp rx=0 tx=0).stop 2024-09-11T21:56:03.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.009+0000 7fdb9369c640 1 -- 172.21.15.1:0/2768263523 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb8c101de0 msgr2=0x7fdb8c198530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:03.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.009+0000 7fdb9369c640 1 --2- 172.21.15.1:0/2768263523 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb8c101de0 0x7fdb8c198530 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fdb88006010 tx=0x7fdb880042f0 comp rx=0 tx=0).stop 2024-09-11T21:56:03.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.009+0000 7fdb91c39640 1 -- 172.21.15.1:0/2768263523 reap_dead start 2024-09-11T21:56:03.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.009+0000 7fdb9369c640 1 -- 172.21.15.1:0/2768263523 shutdown_connections 2024-09-11T21:56:03.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.009+0000 7fdb9369c640 1 -- 172.21.15.1:0/2768263523 >> 172.21.15.1:0/2768263523 conn(0x7fdb8c0fd8c0 msgr2=0x7fdb8c0fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:03.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.009+0000 7fdb9369c640 1 -- 172.21.15.1:0/2768263523 shutdown_connections 2024-09-11T21:56:03.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:02.009+0000 7fdb9369c640 1 -- 172.21.15.1:0/2768263523 wait complete. 2024-09-11T21:56:03.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.915+0000 7f6aa78fb640 1 -- 172.21.15.1:0/1230836109 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6aa0102d40 msgr2=0x7f6aa0103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:03.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.915+0000 7f6aa78fb640 1 --2- 172.21.15.1:0/1230836109 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6aa0102d40 0x7f6aa0103120 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c009900 tx=0x7f6a9c02a480 comp rx=0 tx=0).stop 2024-09-11T21:56:03.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.915+0000 7f6aa78fb640 1 -- 172.21.15.1:0/1230836109 shutdown_connections 2024-09-11T21:56:03.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.915+0000 7f6aa78fb640 1 --2- 172.21.15.1:0/1230836109 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6aa010c170 0x7f6aa010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:03.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.915+0000 7f6aa78fb640 1 --2- 172.21.15.1:0/1230836109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6aa0103660 0x7f6aa010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:03.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.915+0000 7f6aa78fb640 1 --2- 172.21.15.1:0/1230836109 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6aa0102d40 0x7f6aa0103120 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:03.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.915+0000 7f6aa78fb640 1 -- 172.21.15.1:0/1230836109 >> 172.21.15.1:0/1230836109 conn(0x7f6aa00fc7a0 msgr2=0x7f6aa00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:03.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.915+0000 7f6aa78fb640 1 -- 172.21.15.1:0/1230836109 shutdown_connections 2024-09-11T21:56:03.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.916+0000 7f6aa78fb640 1 -- 172.21.15.1:0/1230836109 wait complete. 2024-09-11T21:56:03.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.916+0000 7f6aa78fb640 1 Processor -- start 2024-09-11T21:56:03.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.916+0000 7f6aa78fb640 1 -- start start 2024-09-11T21:56:03.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.916+0000 7f6aa78fb640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6aa0102d40 0x7f6aa01009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:03.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.916+0000 7f6aa78fb640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6aa0103660 0x7f6aa0100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:03.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.916+0000 7f6aa78fb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6aa010c170 0x7f6aa0101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:03.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.916+0000 7f6aa78fb640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa0111f70 con 0x7f6aa0102d40 2024-09-11T21:56:03.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.916+0000 7f6aa78fb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa0111df0 con 0x7f6aa010c170 2024-09-11T21:56:03.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.916+0000 7f6aa78fb640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa01120f0 con 0x7f6aa0103660 2024-09-11T21:56:03.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.916+0000 7f6aa5697640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6aa0102d40 0x7f6aa01009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:03.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.917+0000 7f6aa5697640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6aa0102d40 0x7f6aa01009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56908/0 (socket says 172.21.15.1:56908) 2024-09-11T21:56:03.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.917+0000 7f6aa5697640 1 -- 172.21.15.1:0/2797033394 learned_addr learned my addr 172.21.15.1:0/2797033394 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:56:03.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.917+0000 7f6aa5e98640 1 --2- 172.21.15.1:0/2797033394 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6aa010c170 0x7f6aa0101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:03.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.917+0000 7f6aa4e96640 1 --2- 172.21.15.1:0/2797033394 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6aa0103660 0x7f6aa0100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:03.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.917+0000 7f6aa5697640 1 -- 172.21.15.1:0/2797033394 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6aa0103660 msgr2=0x7f6aa0100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:03.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.917+0000 7f6aa5697640 1 --2- 172.21.15.1:0/2797033394 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6aa0103660 0x7f6aa0100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:03.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.917+0000 7f6aa5697640 1 -- 172.21.15.1:0/2797033394 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6aa010c170 msgr2=0x7f6aa0101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:03.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.917+0000 7f6aa5697640 1 --2- 172.21.15.1:0/2797033394 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6aa010c170 0x7f6aa0101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:03.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.917+0000 7f6aa5697640 1 -- 172.21.15.1:0/2797033394 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6aa01a69b0 con 0x7f6aa0102d40 2024-09-11T21:56:03.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.917+0000 7f6aa5e98640 1 --2- 172.21.15.1:0/2797033394 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6aa010c170 0x7f6aa0101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:56:03.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.917+0000 7f6aa5697640 1 --2- 172.21.15.1:0/2797033394 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6aa0102d40 0x7f6aa01009a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c006010 tx=0x7f6a9c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:03.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.917+0000 7f6a967fc640 1 -- 172.21.15.1:0/2797033394 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c03f030 con 0x7f6aa0102d40 2024-09-11T21:56:03.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.917+0000 7f6a967fc640 1 -- 172.21.15.1:0/2797033394 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a9c003ce0 con 0x7f6aa0102d40 2024-09-11T21:56:03.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.917+0000 7f6aa78fb640 1 -- 172.21.15.1:0/2797033394 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6aa01a6c40 con 0x7f6aa0102d40 2024-09-11T21:56:03.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.917+0000 7f6a967fc640 1 -- 172.21.15.1:0/2797033394 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c035070 con 0x7f6aa0102d40 2024-09-11T21:56:03.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.917+0000 7f6aa78fb640 1 -- 172.21.15.1:0/2797033394 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6aa01a7170 con 0x7f6aa0102d40 2024-09-11T21:56:03.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.918+0000 7f6aa78fb640 1 -- 172.21.15.1:0/2797033394 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6aa00628c0 con 0x7f6aa0102d40 2024-09-11T21:56:03.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.920+0000 7f6a967fc640 1 -- 172.21.15.1:0/2797033394 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f6a9c042030 con 0x7f6aa0102d40 2024-09-11T21:56:03.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.920+0000 7f6a967fc640 1 --2- 172.21.15.1:0/2797033394 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f6a8c0a2c90 0x7f6a8c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:03.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.920+0000 7f6aa4e96640 1 --2- 172.21.15.1:0/2797033394 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f6a8c0a2c90 0x7f6a8c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:03.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.920+0000 7f6a967fc640 1 -- 172.21.15.1:0/2797033394 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f6a9c0efc10 con 0x7f6aa0102d40 2024-09-11T21:56:03.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.920+0000 7f6a967fc640 1 --2- 172.21.15.1:0/2797033394 >> [v2:172.21.15.1:6818/158434853,v1:172.21.15.1:6819/158434853] conn(0x7f6a8c0ac4e0 0x7f6a8c0ae900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:03.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.920+0000 7f6a967fc640 1 -- 172.21.15.1:0/2797033394 --> [v2:172.21.15.1:6818/158434853,v1:172.21.15.1:6819/158434853] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6a9c001d30 con 0x7f6a8c0ac4e0 2024-09-11T21:56:03.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.921+0000 7f6a967fc640 1 -- 172.21.15.1:0/2797033394 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f6a9c0f00f0 con 0x7f6aa0102d40 2024-09-11T21:56:03.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.921+0000 7f6aa5e98640 1 --2- 172.21.15.1:0/2797033394 >> [v2:172.21.15.1:6818/158434853,v1:172.21.15.1:6819/158434853] conn(0x7f6a8c0ac4e0 0x7f6a8c0ae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:03.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.921+0000 7f6aa4e96640 1 --2- 172.21.15.1:0/2797033394 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f6a8c0a2c90 0x7f6a8c0a5150 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6a90004630 tx=0x7f6a90005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:03.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.921+0000 7f6aa5e98640 1 --2- 172.21.15.1:0/2797033394 >> [v2:172.21.15.1:6818/158434853,v1:172.21.15.1:6819/158434853] conn(0x7f6a8c0ac4e0 0x7f6a8c0ae900 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:03.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.922+0000 7f6a967fc640 1 -- 172.21.15.1:0/2797033394 <== osd.6 v2:172.21.15.1:6818/158434853 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6a9c001d30 con 0x7f6a8c0ac4e0 2024-09-11T21:56:03.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.935+0000 7f6aa78fb640 1 -- 172.21.15.1:0/2797033394 --> [v2:172.21.15.1:6818/158434853,v1:172.21.15.1:6819/158434853] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6aa0062cc0 con 0x7f6a8c0ac4e0 2024-09-11T21:56:03.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.935+0000 7f6a967fc640 1 -- 172.21.15.1:0/2797033394 <== osd.6 v2:172.21.15.1:6818/158434853 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6aa0062cc0 con 0x7f6a8c0ac4e0 2024-09-11T21:56:03.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.936+0000 7f6aa78fb640 1 -- 172.21.15.1:0/2797033394 >> [v2:172.21.15.1:6818/158434853,v1:172.21.15.1:6819/158434853] conn(0x7f6a8c0ac4e0 msgr2=0x7f6a8c0ae900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:03.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.936+0000 7f6aa78fb640 1 --2- 172.21.15.1:0/2797033394 >> [v2:172.21.15.1:6818/158434853,v1:172.21.15.1:6819/158434853] conn(0x7f6a8c0ac4e0 0x7f6a8c0ae900 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:03.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.936+0000 7f6aa78fb640 1 -- 172.21.15.1:0/2797033394 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f6a8c0a2c90 msgr2=0x7f6a8c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:03.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.936+0000 7f6aa78fb640 1 --2- 172.21.15.1:0/2797033394 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f6a8c0a2c90 0x7f6a8c0a5150 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6a90004630 tx=0x7f6a90005c50 comp rx=0 tx=0).stop 2024-09-11T21:56:03.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.936+0000 7f6aa78fb640 1 -- 172.21.15.1:0/2797033394 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6aa0102d40 msgr2=0x7f6aa01009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:03.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.936+0000 7f6aa78fb640 1 --2- 172.21.15.1:0/2797033394 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6aa0102d40 0x7f6aa01009a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c006010 tx=0x7f6a9c0042f0 comp rx=0 tx=0).stop 2024-09-11T21:56:03.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.936+0000 7f6aa5e98640 1 -- 172.21.15.1:0/2797033394 reap_dead start 2024-09-11T21:56:03.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.936+0000 7f6aa78fb640 1 -- 172.21.15.1:0/2797033394 shutdown_connections 2024-09-11T21:56:03.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.936+0000 7f6aa78fb640 1 -- 172.21.15.1:0/2797033394 >> 172.21.15.1:0/2797033394 conn(0x7f6aa00fc7a0 msgr2=0x7f6aa00fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:03.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.936+0000 7f6aa78fb640 1 -- 172.21.15.1:0/2797033394 shutdown_connections 2024-09-11T21:56:03.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.936+0000 7f6aa78fb640 1 -- 172.21.15.1:0/2797033394 wait complete. 2024-09-11T21:56:04.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.120+0000 7f51db043640 1 -- 172.21.15.1:0/1849792590 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f51d4101410 msgr2=0x7f51d410d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:04.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.120+0000 7f51db043640 1 --2- 172.21.15.1:0/1849792590 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f51d4101410 0x7f51d410d940 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f51cc009ca0 tx=0x7f51cc02a8c0 comp rx=0 tx=0).stop 2024-09-11T21:56:04.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.120+0000 7f51db043640 1 -- 172.21.15.1:0/1849792590 shutdown_connections 2024-09-11T21:56:04.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.120+0000 7f51db043640 1 --2- 172.21.15.1:0/1849792590 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f51d4101410 0x7f51d410d940 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:04.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.120+0000 7f51db043640 1 --2- 172.21.15.1:0/1849792590 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51d4100a70 0x7f51d4100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:04.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.120+0000 7f51db043640 1 --2- 172.21.15.1:0/1849792590 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f51d4106a70 0x7f51d4106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:04.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.120+0000 7f51db043640 1 -- 172.21.15.1:0/1849792590 >> 172.21.15.1:0/1849792590 conn(0x7f51d40fc790 msgr2=0x7f51d40febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:04.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.120+0000 7f51db043640 1 -- 172.21.15.1:0/1849792590 shutdown_connections 2024-09-11T21:56:04.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.120+0000 7f51db043640 1 -- 172.21.15.1:0/1849792590 wait complete. 2024-09-11T21:56:04.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.121+0000 7f51db043640 1 Processor -- start 2024-09-11T21:56:04.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.121+0000 7f51db043640 1 -- start start 2024-09-11T21:56:04.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.121+0000 7f51db043640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f51d4100a70 0x7f51d41985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:04.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.121+0000 7f51db043640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f51d4101410 0x7f51d4198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:04.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.121+0000 7f51db043640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51d4106a70 0x7f51d419ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:04.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.121+0000 7f51db043640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f51d4112010 con 0x7f51d4101410 2024-09-11T21:56:04.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.121+0000 7f51db043640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f51d4111e90 con 0x7f51d4106a70 2024-09-11T21:56:04.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.121+0000 7f51db043640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f51d4112190 con 0x7f51d4100a70 2024-09-11T21:56:04.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.121+0000 7f51d3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f51d4101410 0x7f51d4198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:04.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.121+0000 7f51d3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f51d4101410 0x7f51d4198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56900/0 (socket says 172.21.15.1:56900) 2024-09-11T21:56:04.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.121+0000 7f51d3fff640 1 -- 172.21.15.1:0/3493644871 learned_addr learned my addr 172.21.15.1:0/3493644871 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:56:04.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.121+0000 7f51d8ddf640 1 --2- 172.21.15.1:0/3493644871 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f51d4100a70 0x7f51d41985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:04.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.121+0000 7f51d95e0640 1 --2- 172.21.15.1:0/3493644871 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51d4106a70 0x7f51d419ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:04.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.121+0000 7f51d3fff640 1 -- 172.21.15.1:0/3493644871 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f51d4100a70 msgr2=0x7f51d41985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:04.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.121+0000 7f51d3fff640 1 --2- 172.21.15.1:0/3493644871 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f51d4100a70 0x7f51d41985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:04.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.121+0000 7f51d3fff640 1 -- 172.21.15.1:0/3493644871 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51d4106a70 msgr2=0x7f51d419ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:04.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.121+0000 7f51d3fff640 1 --2- 172.21.15.1:0/3493644871 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51d4106a70 0x7f51d419ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:04.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.121+0000 7f51d3fff640 1 -- 172.21.15.1:0/3493644871 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51d419d560 con 0x7f51d4101410 2024-09-11T21:56:04.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.121+0000 7f51d95e0640 1 --2- 172.21.15.1:0/3493644871 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51d4106a70 0x7f51d419ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:56:04.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.121+0000 7f51d8ddf640 1 --2- 172.21.15.1:0/3493644871 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f51d4100a70 0x7f51d41985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:56:04.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.122+0000 7f51d3fff640 1 --2- 172.21.15.1:0/3493644871 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f51d4101410 0x7f51d4198af0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f51bc00b4a0 tx=0x7f51bc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:04.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.122+0000 7f51d1ffb640 1 -- 172.21.15.1:0/3493644871 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51bc014070 con 0x7f51d4101410 2024-09-11T21:56:04.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.122+0000 7f51d1ffb640 1 -- 172.21.15.1:0/3493644871 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51bc010af0 con 0x7f51d4101410 2024-09-11T21:56:04.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.122+0000 7f51db043640 1 -- 172.21.15.1:0/3493644871 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51d419d850 con 0x7f51d4101410 2024-09-11T21:56:04.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.122+0000 7f51d1ffb640 1 -- 172.21.15.1:0/3493644871 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51bc017d10 con 0x7f51d4101410 2024-09-11T21:56:04.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.122+0000 7f51db043640 1 -- 172.21.15.1:0/3493644871 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51d41a5130 con 0x7f51d4101410 2024-09-11T21:56:04.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.123+0000 7f51db043640 1 -- 172.21.15.1:0/3493644871 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f51d4062980 con 0x7f51d4101410 2024-09-11T21:56:04.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.124+0000 7f51d1ffb640 1 -- 172.21.15.1:0/3493644871 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f51bc017070 con 0x7f51d4101410 2024-09-11T21:56:04.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.125+0000 7f51d1ffb640 1 --2- 172.21.15.1:0/3493644871 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f51c00a2c90 0x7f51c00a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:04.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.125+0000 7f51d8ddf640 1 --2- 172.21.15.1:0/3493644871 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f51c00a2c90 0x7f51c00a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:04.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.125+0000 7f51d1ffb640 1 -- 172.21.15.1:0/3493644871 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f51bc0ce2b0 con 0x7f51d4101410 2024-09-11T21:56:04.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.125+0000 7f51d1ffb640 1 --2- 172.21.15.1:0/3493644871 >> [v2:172.21.15.53:6816/2132057697,v1:172.21.15.53:6817/2132057697] conn(0x7f51c00ac4e0 0x7f51c00ae900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:04.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.125+0000 7f51d1ffb640 1 -- 172.21.15.1:0/3493644871 --> [v2:172.21.15.53:6816/2132057697,v1:172.21.15.53:6817/2132057697] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f51c00aefb0 con 0x7f51c00ac4e0 2024-09-11T21:56:04.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.125+0000 7f51d1ffb640 1 -- 172.21.15.1:0/3493644871 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f51bc0d5050 con 0x7f51d4101410 2024-09-11T21:56:04.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.125+0000 7f51d8ddf640 1 --2- 172.21.15.1:0/3493644871 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f51c00a2c90 0x7f51c00a5150 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f51c4009930 tx=0x7f51c4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:04.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.125+0000 7f51d95e0640 1 --2- 172.21.15.1:0/3493644871 >> [v2:172.21.15.53:6816/2132057697,v1:172.21.15.53:6817/2132057697] conn(0x7f51c00ac4e0 0x7f51c00ae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:04.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.126+0000 7f51d95e0640 1 --2- 172.21.15.1:0/3493644871 >> [v2:172.21.15.53:6816/2132057697,v1:172.21.15.53:6817/2132057697] conn(0x7f51c00ac4e0 0x7f51c00ae900 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:04.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.127+0000 7f51d1ffb640 1 -- 172.21.15.1:0/3493644871 <== osd.7 v2:172.21.15.53:6816/2132057697 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f51c00aefb0 con 0x7f51c00ac4e0 2024-09-11T21:56:04.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.140+0000 7f51db043640 1 -- 172.21.15.1:0/3493644871 --> [v2:172.21.15.53:6816/2132057697,v1:172.21.15.53:6817/2132057697] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f51d4062db0 con 0x7f51c00ac4e0 2024-09-11T21:56:04.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.141+0000 7f51d1ffb640 1 -- 172.21.15.1:0/3493644871 <== osd.7 v2:172.21.15.53:6816/2132057697 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f51d4062db0 con 0x7f51c00ac4e0 2024-09-11T21:56:04.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.141+0000 7f51db043640 1 -- 172.21.15.1:0/3493644871 >> [v2:172.21.15.53:6816/2132057697,v1:172.21.15.53:6817/2132057697] conn(0x7f51c00ac4e0 msgr2=0x7f51c00ae900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:04.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.141+0000 7f51db043640 1 --2- 172.21.15.1:0/3493644871 >> [v2:172.21.15.53:6816/2132057697,v1:172.21.15.53:6817/2132057697] conn(0x7f51c00ac4e0 0x7f51c00ae900 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:04.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.141+0000 7f51db043640 1 -- 172.21.15.1:0/3493644871 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f51c00a2c90 msgr2=0x7f51c00a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:04.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.141+0000 7f51db043640 1 --2- 172.21.15.1:0/3493644871 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f51c00a2c90 0x7f51c00a5150 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f51c4009930 tx=0x7f51c4008040 comp rx=0 tx=0).stop 2024-09-11T21:56:04.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.141+0000 7f51db043640 1 -- 172.21.15.1:0/3493644871 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f51d4101410 msgr2=0x7f51d4198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:04.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.141+0000 7f51db043640 1 --2- 172.21.15.1:0/3493644871 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f51d4101410 0x7f51d4198af0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f51bc00b4a0 tx=0x7f51bc00b960 comp rx=0 tx=0).stop 2024-09-11T21:56:04.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.141+0000 7f51d95e0640 1 -- 172.21.15.1:0/3493644871 reap_dead start 2024-09-11T21:56:04.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.142+0000 7f51db043640 1 -- 172.21.15.1:0/3493644871 shutdown_connections 2024-09-11T21:56:04.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.142+0000 7f51db043640 1 -- 172.21.15.1:0/3493644871 >> 172.21.15.1:0/3493644871 conn(0x7f51d40fc790 msgr2=0x7f51d40fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:04.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.142+0000 7f51db043640 1 -- 172.21.15.1:0/3493644871 shutdown_connections 2024-09-11T21:56:04.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:03.142+0000 7f51db043640 1 -- 172.21.15.1:0/3493644871 wait complete. 2024-09-11T21:56:04.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:04 smithi053 ceph-mon[34908]: pgmap v303: 97 pgs: 97 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-11T21:56:04.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:04 smithi161 ceph-mon[35615]: pgmap v303: 97 pgs: 97 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-11T21:56:04.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:04 smithi001 ceph-mon[30935]: pgmap v303: 97 pgs: 97 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-11T21:56:05.197 INFO:teuthology.orchestra.run.smithi001.stdout:77309411396 2024-09-11T21:56:05.197 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-11T21:56:05.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:05.365+0000 7f95a2c90640 1 -- 172.21.15.1:0/2017884835 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f959c102ef0 msgr2=0x7f959c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:05.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:05.365+0000 7f95a2c90640 1 --2- 172.21.15.1:0/2017884835 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f959c102ef0 0x7f959c10da40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9594009ca0 tx=0x7f959402a950 comp rx=0 tx=0).stop 2024-09-11T21:56:05.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:05.365+0000 7f95a2c90640 1 -- 172.21.15.1:0/2017884835 shutdown_connections 2024-09-11T21:56:05.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:05.365+0000 7f95a2c90640 1 --2- 172.21.15.1:0/2017884835 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f959c102ef0 0x7f959c10da40 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:05.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:05.365+0000 7f95a2c90640 1 --2- 172.21.15.1:0/2017884835 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f959c102570 0x7f959c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:05.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:05.365+0000 7f95a2c90640 1 --2- 172.21.15.1:0/2017884835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f959c100e20 0x7f959c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:05.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:05.365+0000 7f95a2c90640 1 -- 172.21.15.1:0/2017884835 >> 172.21.15.1:0/2017884835 conn(0x7f959c0fc7d0 msgr2=0x7f959c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:05.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:05.365+0000 7f95a2c90640 1 -- 172.21.15.1:0/2017884835 shutdown_connections 2024-09-11T21:56:05.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:05.365+0000 7f95a2c90640 1 -- 172.21.15.1:0/2017884835 wait complete. 2024-09-11T21:56:05.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:05.366+0000 7f95a2c90640 1 Processor -- start 2024-09-11T21:56:05.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:05.366+0000 7f95a2c90640 1 -- start start 2024-09-11T21:56:05.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:05.366+0000 7f95a2c90640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f959c100e20 0x7f959c197390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:05.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:05.366+0000 7f95a2c90640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f959c102570 0x7f959c1978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:05.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:05.366+0000 7f95a2c90640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f959c102ef0 0x7f959c19bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:05.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:05.366+0000 7f95a2c90640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f959c110fd0 con 0x7f959c100e20 2024-09-11T21:56:05.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:05.366+0000 7f95a2c90640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f959c110e50 con 0x7f959c102570 2024-09-11T21:56:05.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:05.366+0000 7f95a2c90640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f959c111150 con 0x7f959c102ef0 2024-09-11T21:56:05.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:05.366+0000 7f95a0a2c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f959c100e20 0x7f959c197390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:05.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:05.366+0000 7f95a0a2c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f959c100e20 0x7f959c197390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56926/0 (socket says 172.21.15.1:56926) 2024-09-11T21:56:05.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:05.366+0000 7f95a0a2c640 1 -- 172.21.15.1:0/3221482293 learned_addr learned my addr 172.21.15.1:0/3221482293 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:56:05.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:05.367+0000 7f959bfff640 1 --2- 172.21.15.1:0/3221482293 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f959c102570 0x7f959c1978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:05.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:05.367+0000 7f95a122d640 1 --2- 172.21.15.1:0/3221482293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f959c102ef0 0x7f959c19bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:05.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:05.367+0000 7f95a0a2c640 1 -- 172.21.15.1:0/3221482293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f959c102ef0 msgr2=0x7f959c19bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:05.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:05.367+0000 7f95a0a2c640 1 --2- 172.21.15.1:0/3221482293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f959c102ef0 0x7f959c19bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:05.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:05.367+0000 7f95a0a2c640 1 -- 172.21.15.1:0/3221482293 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f959c102570 msgr2=0x7f959c1978d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:05.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:05.367+0000 7f95a0a2c640 1 --2- 172.21.15.1:0/3221482293 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f959c102570 0x7f959c1978d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:05.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:05.367+0000 7f95a0a2c640 1 -- 172.21.15.1:0/3221482293 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f959c19c340 con 0x7f959c100e20 2024-09-11T21:56:05.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:05.367+0000 7f959bfff640 1 --2- 172.21.15.1:0/3221482293 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f959c102570 0x7f959c1978d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T21:56:05.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:05.367+0000 7f95a122d640 1 --2- 172.21.15.1:0/3221482293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f959c102ef0 0x7f959c19bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T21:56:05.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:05.367+0000 7f95a0a2c640 1 --2- 172.21.15.1:0/3221482293 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f959c100e20 0x7f959c197390 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f959000d950 tx=0x7f959000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:05.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:05.367+0000 7f9599ffb640 1 -- 172.21.15.1:0/3221482293 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9590017070 con 0x7f959c100e20 2024-09-11T21:56:05.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:05.367+0000 7f9599ffb640 1 -- 172.21.15.1:0/3221482293 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9590009c30 con 0x7f959c100e20 2024-09-11T21:56:05.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:05.367+0000 7f95a2c90640 1 -- 172.21.15.1:0/3221482293 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f959c19c630 con 0x7f959c100e20 2024-09-11T21:56:05.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:05.367+0000 7f9599ffb640 1 -- 172.21.15.1:0/3221482293 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959000f6b0 con 0x7f959c100e20 2024-09-11T21:56:05.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:05.367+0000 7f95a2c90640 1 -- 172.21.15.1:0/3221482293 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f959c1a3f10 con 0x7f959c100e20 2024-09-11T21:56:05.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:05.368+0000 7f95a2c90640 1 -- 172.21.15.1:0/3221482293 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f959c107290 con 0x7f959c100e20 2024-09-11T21:56:05.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:05.369+0000 7f9599ffb640 1 -- 172.21.15.1:0/3221482293 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f95900109d0 con 0x7f959c100e20 2024-09-11T21:56:05.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:05.370+0000 7f9599ffb640 1 --2- 172.21.15.1:0/3221482293 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f958c0a2c10 0x7f958c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:05.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:05.370+0000 7f959bfff640 1 --2- 172.21.15.1:0/3221482293 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f958c0a2c10 0x7f958c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:05.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:05.371+0000 7f9599ffb640 1 -- 172.21.15.1:0/3221482293 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f95900cf1e0 con 0x7f959c100e20 2024-09-11T21:56:05.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:05.371+0000 7f9599ffb640 1 --2- 172.21.15.1:0/3221482293 >> [v2:172.21.15.161:6816/1838248669,v1:172.21.15.161:6817/1838248669] conn(0x7f958c0ac460 0x7f958c0ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:05.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:05.371+0000 7f9599ffb640 1 -- 172.21.15.1:0/3221482293 --> [v2:172.21.15.161:6816/1838248669,v1:172.21.15.161:6817/1838248669] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9590016ec0 con 0x7f958c0ac460 2024-09-11T21:56:05.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:05.371+0000 7f9599ffb640 1 -- 172.21.15.1:0/3221482293 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f95900cf590 con 0x7f959c100e20 2024-09-11T21:56:05.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:05.371+0000 7f959bfff640 1 --2- 172.21.15.1:0/3221482293 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f958c0a2c10 0x7f958c0a50d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f959c0fec10 tx=0x7f9588005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:05.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:05.371+0000 7f95a122d640 1 --2- 172.21.15.1:0/3221482293 >> [v2:172.21.15.161:6816/1838248669,v1:172.21.15.161:6817/1838248669] conn(0x7f958c0ac460 0x7f958c0ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:05.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:05.372+0000 7f95a122d640 1 --2- 172.21.15.1:0/3221482293 >> [v2:172.21.15.161:6816/1838248669,v1:172.21.15.161:6817/1838248669] conn(0x7f958c0ac460 0x7f958c0ae880 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:05.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:05.372+0000 7f9599ffb640 1 -- 172.21.15.1:0/3221482293 <== osd.8 v2:172.21.15.161:6816/1838248669 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9590016ec0 con 0x7f958c0ac460 2024-09-11T21:56:05.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:05.386+0000 7f95a2c90640 1 -- 172.21.15.1:0/3221482293 --> [v2:172.21.15.161:6816/1838248669,v1:172.21.15.161:6817/1838248669] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f959c1021b0 con 0x7f958c0ac460 2024-09-11T21:56:05.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:05.387+0000 7f9599ffb640 1 -- 172.21.15.1:0/3221482293 <== osd.8 v2:172.21.15.161:6816/1838248669 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f959c1021b0 con 0x7f958c0ac460 2024-09-11T21:56:05.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:05.387+0000 7f95a2c90640 1 -- 172.21.15.1:0/3221482293 >> [v2:172.21.15.161:6816/1838248669,v1:172.21.15.161:6817/1838248669] conn(0x7f958c0ac460 msgr2=0x7f958c0ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:05.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:05.387+0000 7f95a2c90640 1 --2- 172.21.15.1:0/3221482293 >> [v2:172.21.15.161:6816/1838248669,v1:172.21.15.161:6817/1838248669] conn(0x7f958c0ac460 0x7f958c0ae880 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:05.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:05.387+0000 7f95a2c90640 1 -- 172.21.15.1:0/3221482293 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f958c0a2c10 msgr2=0x7f958c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:05.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:05.387+0000 7f95a2c90640 1 --2- 172.21.15.1:0/3221482293 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f958c0a2c10 0x7f958c0a50d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f959c0fec10 tx=0x7f9588005e50 comp rx=0 tx=0).stop 2024-09-11T21:56:05.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:05.387+0000 7f95a2c90640 1 -- 172.21.15.1:0/3221482293 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f959c100e20 msgr2=0x7f959c197390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:05.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:05.387+0000 7f95a2c90640 1 --2- 172.21.15.1:0/3221482293 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f959c100e20 0x7f959c197390 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f959000d950 tx=0x7f959000de10 comp rx=0 tx=0).stop 2024-09-11T21:56:05.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:05.387+0000 7f95a122d640 1 -- 172.21.15.1:0/3221482293 reap_dead start 2024-09-11T21:56:05.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:05.387+0000 7f95a2c90640 1 -- 172.21.15.1:0/3221482293 shutdown_connections 2024-09-11T21:56:05.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:05.387+0000 7f95a2c90640 1 -- 172.21.15.1:0/3221482293 >> 172.21.15.1:0/3221482293 conn(0x7f959c0fc7d0 msgr2=0x7f959c106830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:05.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:05.387+0000 7f95a2c90640 1 -- 172.21.15.1:0/3221482293 shutdown_connections 2024-09-11T21:56:05.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:05.387+0000 7f95a2c90640 1 -- 172.21.15.1:0/3221482293 wait complete. 2024-09-11T21:56:05.972 INFO:teuthology.orchestra.run.smithi001.stdout:163208757301 2024-09-11T21:56:05.972 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-11T21:56:06.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:06 smithi053 ceph-mon[34908]: pgmap v304: 97 pgs: 97 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-11T21:56:06.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:06 smithi161 ceph-mon[35615]: pgmap v304: 97 pgs: 97 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-11T21:56:06.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:06 smithi001 ceph-mon[30935]: pgmap v304: 97 pgs: 97 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-11T21:56:06.668 INFO:teuthology.orchestra.run.smithi001.stdout:107374182463 2024-09-11T21:56:06.668 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-11T21:56:07.225 INFO:teuthology.orchestra.run.smithi001.stdout:197568495663 2024-09-11T21:56:07.225 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-11T21:56:07.378 INFO:teuthology.orchestra.run.smithi001.stdout:219043332138 2024-09-11T21:56:07.379 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-11T21:56:07.772 INFO:teuthology.orchestra.run.smithi001.stdout:283467841568 2024-09-11T21:56:07.773 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-11T21:56:08.404 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:08 smithi001 ceph-mon[30935]: pgmap v305: 97 pgs: 97 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T21:56:08.435 INFO:teuthology.orchestra.run.smithi001.stdout:304942678042 2024-09-11T21:56:08.435 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-11T21:56:08.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:08 smithi053 ceph-mon[34908]: pgmap v305: 97 pgs: 97 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T21:56:08.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:08 smithi161 ceph-mon[35615]: pgmap v305: 97 pgs: 97 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T21:56:09.079 INFO:teuthology.orchestra.run.smithi001.stdout:249108103206 2024-09-11T21:56:09.079 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-11T21:56:09.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:09.271+0000 7f249857d640 1 -- 172.21.15.1:0/2410104975 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2490106a90 msgr2=0x7f2490106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:09.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:09.271+0000 7f249857d640 1 --2- 172.21.15.1:0/2410104975 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2490106a90 0x7f2490106e70 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f248c009930 tx=0x7f248c02a420 comp rx=0 tx=0).stop 2024-09-11T21:56:09.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:09.271+0000 7f249857d640 1 -- 172.21.15.1:0/2410104975 shutdown_connections 2024-09-11T21:56:09.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:09.271+0000 7f249857d640 1 --2- 172.21.15.1:0/2410104975 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2490101430 0x7f249010d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:09.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:09.271+0000 7f249857d640 1 --2- 172.21.15.1:0/2410104975 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2490100a90 0x7f2490100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:09.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:09.271+0000 7f249857d640 1 --2- 172.21.15.1:0/2410104975 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2490106a90 0x7f2490106e70 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:09.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:09.272+0000 7f249857d640 1 -- 172.21.15.1:0/2410104975 >> 172.21.15.1:0/2410104975 conn(0x7f24900fc7d0 msgr2=0x7f24900febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:09.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:09.272+0000 7f249857d640 1 -- 172.21.15.1:0/2410104975 shutdown_connections 2024-09-11T21:56:09.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:09.272+0000 7f249857d640 1 -- 172.21.15.1:0/2410104975 wait complete. 2024-09-11T21:56:09.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:09.272+0000 7f249857d640 1 Processor -- start 2024-09-11T21:56:09.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:09.272+0000 7f249857d640 1 -- start start 2024-09-11T21:56:09.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:09.273+0000 7f249857d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2490100a90 0x7f24901974c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:09.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:09.273+0000 7f249857d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2490101430 0x7f2490197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:09.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:09.273+0000 7f249857d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2490106a90 0x7f249019bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:09.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:09.273+0000 7f249857d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2490110f30 con 0x7f2490106a90 2024-09-11T21:56:09.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:09.273+0000 7f249857d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2490110db0 con 0x7f2490101430 2024-09-11T21:56:09.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:09.273+0000 7f249857d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f24901110b0 con 0x7f2490100a90 2024-09-11T21:56:09.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:09.273+0000 7f2496b1a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2490106a90 0x7f249019bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:09.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:09.273+0000 7f2496b1a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2490106a90 0x7f249019bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42954/0 (socket says 172.21.15.1:42954) 2024-09-11T21:56:09.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:09.273+0000 7f2496b1a640 1 -- 172.21.15.1:0/1770917388 learned_addr learned my addr 172.21.15.1:0/1770917388 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:56:09.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:09.273+0000 7f2495b18640 1 --2- 172.21.15.1:0/1770917388 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2490101430 0x7f2490197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:09.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:09.273+0000 7f2496319640 1 --2- 172.21.15.1:0/1770917388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2490100a90 0x7f24901974c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:09.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:09.273+0000 7f2496b1a640 1 -- 172.21.15.1:0/1770917388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2490100a90 msgr2=0x7f24901974c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:09.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:09.273+0000 7f2496b1a640 1 --2- 172.21.15.1:0/1770917388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2490100a90 0x7f24901974c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:09.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:09.273+0000 7f2496b1a640 1 -- 172.21.15.1:0/1770917388 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2490101430 msgr2=0x7f2490197a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:09.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:09.273+0000 7f2496b1a640 1 --2- 172.21.15.1:0/1770917388 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2490101430 0x7f2490197a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:09.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:09.273+0000 7f2496b1a640 1 -- 172.21.15.1:0/1770917388 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f249019c490 con 0x7f2490106a90 2024-09-11T21:56:09.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:09.273+0000 7f2495b18640 1 --2- 172.21.15.1:0/1770917388 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2490101430 0x7f2490197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T21:56:09.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:09.274+0000 7f2496b1a640 1 --2- 172.21.15.1:0/1770917388 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2490106a90 0x7f249019bd90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f248800b970 tx=0x7f248800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:09.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:09.274+0000 7f24877fe640 1 -- 172.21.15.1:0/1770917388 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2488019070 con 0x7f2490106a90 2024-09-11T21:56:09.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:09.274+0000 7f249857d640 1 -- 172.21.15.1:0/1770917388 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f249019c780 con 0x7f2490106a90 2024-09-11T21:56:09.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:09.274+0000 7f24877fe640 1 -- 172.21.15.1:0/1770917388 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2488014440 con 0x7f2490106a90 2024-09-11T21:56:09.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:09.274+0000 7f24877fe640 1 -- 172.21.15.1:0/1770917388 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24880075d0 con 0x7f2490106a90 2024-09-11T21:56:09.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:09.274+0000 7f249857d640 1 -- 172.21.15.1:0/1770917388 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f249006a790 con 0x7f2490106a90 2024-09-11T21:56:09.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:09.275+0000 7f249857d640 1 -- 172.21.15.1:0/1770917388 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f249006ada0 con 0x7f2490106a90 2024-09-11T21:56:09.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:09.276+0000 7f24877fe640 1 -- 172.21.15.1:0/1770917388 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f248801c070 con 0x7f2490106a90 2024-09-11T21:56:09.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:09.277+0000 7f24877fe640 1 --2- 172.21.15.1:0/1770917388 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f247c0a2c10 0x7f247c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:09.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:09.277+0000 7f2496319640 1 --2- 172.21.15.1:0/1770917388 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f247c0a2c10 0x7f247c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:09.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:09.277+0000 7f24877fe640 1 -- 172.21.15.1:0/1770917388 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f24880cec80 con 0x7f2490106a90 2024-09-11T21:56:09.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:09.277+0000 7f24877fe640 1 --2- 172.21.15.1:0/1770917388 >> [v2:172.21.15.53:6808/3081402681,v1:172.21.15.53:6809/3081402681] conn(0x7f247c0ac460 0x7f247c0ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:09.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:09.277+0000 7f24877fe640 1 -- 172.21.15.1:0/1770917388 --> [v2:172.21.15.53:6808/3081402681,v1:172.21.15.53:6809/3081402681] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f247c0aef30 con 0x7f247c0ac460 2024-09-11T21:56:09.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:09.277+0000 7f24877fe640 1 -- 172.21.15.1:0/1770917388 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f24880cf160 con 0x7f2490106a90 2024-09-11T21:56:09.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:09.277+0000 7f2496319640 1 --2- 172.21.15.1:0/1770917388 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f247c0a2c10 0x7f247c0a50d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f248c009900 tx=0x7f248c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:09.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:09.277+0000 7f2495b18640 1 --2- 172.21.15.1:0/1770917388 >> [v2:172.21.15.53:6808/3081402681,v1:172.21.15.53:6809/3081402681] conn(0x7f247c0ac460 0x7f247c0ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:09.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:09.278+0000 7f2495b18640 1 --2- 172.21.15.1:0/1770917388 >> [v2:172.21.15.53:6808/3081402681,v1:172.21.15.53:6809/3081402681] conn(0x7f247c0ac460 0x7f247c0ae880 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:09.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:09.279+0000 7f24877fe640 1 -- 172.21.15.1:0/1770917388 <== osd.4 v2:172.21.15.53:6808/3081402681 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f247c0aef30 con 0x7f247c0ac460 2024-09-11T21:56:09.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:09.293+0000 7f249857d640 1 -- 172.21.15.1:0/1770917388 --> [v2:172.21.15.53:6808/3081402681,v1:172.21.15.53:6809/3081402681] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f249006b220 con 0x7f247c0ac460 2024-09-11T21:56:09.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:09.294+0000 7f24877fe640 1 -- 172.21.15.1:0/1770917388 <== osd.4 v2:172.21.15.53:6808/3081402681 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f249006b220 con 0x7f247c0ac460 2024-09-11T21:56:09.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:09.294+0000 7f249857d640 1 -- 172.21.15.1:0/1770917388 >> [v2:172.21.15.53:6808/3081402681,v1:172.21.15.53:6809/3081402681] conn(0x7f247c0ac460 msgr2=0x7f247c0ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:09.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:09.294+0000 7f249857d640 1 --2- 172.21.15.1:0/1770917388 >> [v2:172.21.15.53:6808/3081402681,v1:172.21.15.53:6809/3081402681] conn(0x7f247c0ac460 0x7f247c0ae880 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:09.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:09.294+0000 7f249857d640 1 -- 172.21.15.1:0/1770917388 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f247c0a2c10 msgr2=0x7f247c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:09.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:09.294+0000 7f249857d640 1 --2- 172.21.15.1:0/1770917388 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f247c0a2c10 0x7f247c0a50d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f248c009900 tx=0x7f248c0058d0 comp rx=0 tx=0).stop 2024-09-11T21:56:09.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:09.294+0000 7f249857d640 1 -- 172.21.15.1:0/1770917388 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2490106a90 msgr2=0x7f249019bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:09.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:09.294+0000 7f249857d640 1 --2- 172.21.15.1:0/1770917388 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2490106a90 0x7f249019bd90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f248800b970 tx=0x7f248800bd70 comp rx=0 tx=0).stop 2024-09-11T21:56:09.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:09.294+0000 7f2496b1a640 1 -- 172.21.15.1:0/1770917388 reap_dead start 2024-09-11T21:56:09.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:09.294+0000 7f249857d640 1 -- 172.21.15.1:0/1770917388 shutdown_connections 2024-09-11T21:56:09.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:09.294+0000 7f249857d640 1 -- 172.21.15.1:0/1770917388 >> 172.21.15.1:0/1770917388 conn(0x7f24900fc7d0 msgr2=0x7f24900fd150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:09.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:09.295+0000 7f249857d640 1 -- 172.21.15.1:0/1770917388 shutdown_connections 2024-09-11T21:56:09.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:09.295+0000 7f249857d640 1 -- 172.21.15.1:0/1770917388 wait complete. 2024-09-11T21:56:10.001 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:56:10.007 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:56:10.007 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:56:10.012 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:56:10.045 INFO:teuthology.orchestra.run.smithi001.stdout:335007449110 2024-09-11T21:56:10.045 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-11T21:56:10.523 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:10 smithi001 ceph-mon[30935]: pgmap v306: 97 pgs: 97 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T21:56:10.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:10 smithi053 ceph-mon[34908]: pgmap v306: 97 pgs: 97 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T21:56:10.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:10 smithi161 ceph-mon[35615]: pgmap v306: 97 pgs: 97 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T21:56:10.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:10.915+0000 7f5fea36b640 1 -- 172.21.15.1:0/2449265604 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5fe4106a80 msgr2=0x7f5fe4106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:10.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:10.915+0000 7f5fea36b640 1 --2- 172.21.15.1:0/2449265604 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5fe4106a80 0x7f5fe4106e60 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4009930 tx=0x7f5fd402a420 comp rx=0 tx=0).stop 2024-09-11T21:56:10.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:10.915+0000 7f5fea36b640 1 -- 172.21.15.1:0/2449265604 shutdown_connections 2024-09-11T21:56:10.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:10.915+0000 7f5fea36b640 1 --2- 172.21.15.1:0/2449265604 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fe4101420 0x7f5fe410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:10.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:10.915+0000 7f5fea36b640 1 --2- 172.21.15.1:0/2449265604 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5fe4100a80 0x7f5fe4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:10.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:10.915+0000 7f5fea36b640 1 --2- 172.21.15.1:0/2449265604 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5fe4106a80 0x7f5fe4106e60 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:10.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:10.915+0000 7f5fea36b640 1 -- 172.21.15.1:0/2449265604 >> 172.21.15.1:0/2449265604 conn(0x7f5fe40fc7a0 msgr2=0x7f5fe40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:10.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:10.915+0000 7f5fea36b640 1 -- 172.21.15.1:0/2449265604 shutdown_connections 2024-09-11T21:56:10.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:10.915+0000 7f5fea36b640 1 -- 172.21.15.1:0/2449265604 wait complete. 2024-09-11T21:56:10.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:10.916+0000 7f5fea36b640 1 Processor -- start 2024-09-11T21:56:10.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:10.916+0000 7f5fea36b640 1 -- start start 2024-09-11T21:56:10.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:10.916+0000 7f5fea36b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5fe4100a80 0x7f5fe4198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:10.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:10.916+0000 7f5fea36b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5fe4101420 0x7f5fe41989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:10.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:10.916+0000 7f5fea36b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fe4106a80 0x7f5fe419cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:10.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:10.916+0000 7f5fea36b640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe4110e80 con 0x7f5fe4100a80 2024-09-11T21:56:10.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:10.916+0000 7f5fea36b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe4110d00 con 0x7f5fe4106a80 2024-09-11T21:56:10.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:10.916+0000 7f5fea36b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe4111000 con 0x7f5fe4101420 2024-09-11T21:56:10.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:10.916+0000 7f5fe3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5fe4100a80 0x7f5fe4198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:10.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:10.916+0000 7f5fe3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5fe4100a80 0x7f5fe4198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42980/0 (socket says 172.21.15.1:42980) 2024-09-11T21:56:10.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:10.916+0000 7f5fe3fff640 1 -- 172.21.15.1:0/2004517791 learned_addr learned my addr 172.21.15.1:0/2004517791 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:56:10.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:10.917+0000 7f5fe37fe640 1 --2- 172.21.15.1:0/2004517791 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5fe4101420 0x7f5fe41989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:10.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:10.917+0000 7f5fe8908640 1 --2- 172.21.15.1:0/2004517791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fe4106a80 0x7f5fe419cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:10.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:10.917+0000 7f5fe3fff640 1 -- 172.21.15.1:0/2004517791 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5fe4101420 msgr2=0x7f5fe41989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:10.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:10.917+0000 7f5fe3fff640 1 --2- 172.21.15.1:0/2004517791 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5fe4101420 0x7f5fe41989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:10.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:10.917+0000 7f5fe3fff640 1 -- 172.21.15.1:0/2004517791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fe4106a80 msgr2=0x7f5fe419cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:10.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:10.917+0000 7f5fe3fff640 1 --2- 172.21.15.1:0/2004517791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fe4106a80 0x7f5fe419cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:10.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:10.917+0000 7f5fe3fff640 1 -- 172.21.15.1:0/2004517791 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fe419d410 con 0x7f5fe4100a80 2024-09-11T21:56:10.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:10.917+0000 7f5fe3fff640 1 --2- 172.21.15.1:0/2004517791 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5fe4100a80 0x7f5fe4198460 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4009a50 tx=0x7f5fd4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:10.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:10.917+0000 7f5fe17fa640 1 -- 172.21.15.1:0/2004517791 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd403f030 con 0x7f5fe4100a80 2024-09-11T21:56:10.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:10.917+0000 7f5fe17fa640 1 -- 172.21.15.1:0/2004517791 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fd4003ce0 con 0x7f5fe4100a80 2024-09-11T21:56:10.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:10.917+0000 7f5fe17fa640 1 -- 172.21.15.1:0/2004517791 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd4035070 con 0x7f5fe4100a80 2024-09-11T21:56:10.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:10.917+0000 7f5fea36b640 1 -- 172.21.15.1:0/2004517791 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fe419d6a0 con 0x7f5fe4100a80 2024-09-11T21:56:10.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:10.917+0000 7f5fea36b640 1 -- 172.21.15.1:0/2004517791 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fe41a4fe0 con 0x7f5fe4100a80 2024-09-11T21:56:10.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:10.918+0000 7f5fea36b640 1 -- 172.21.15.1:0/2004517791 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fe4100ee0 con 0x7f5fe4100a80 2024-09-11T21:56:10.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:10.919+0000 7f5fe17fa640 1 -- 172.21.15.1:0/2004517791 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f5fd4042030 con 0x7f5fe4100a80 2024-09-11T21:56:10.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:10.920+0000 7f5fe17fa640 1 --2- 172.21.15.1:0/2004517791 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f5fcc0a2c90 0x7f5fcc0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:10.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:10.920+0000 7f5fe37fe640 1 --2- 172.21.15.1:0/2004517791 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f5fcc0a2c90 0x7f5fcc0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:10.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:10.920+0000 7f5fe17fa640 1 -- 172.21.15.1:0/2004517791 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f5fd40ef6f0 con 0x7f5fe4100a80 2024-09-11T21:56:10.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:10.921+0000 7f5fe37fe640 1 --2- 172.21.15.1:0/2004517791 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f5fcc0a2c90 0x7f5fcc0a5150 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8005fd0 tx=0x7f5fd8005e20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:10.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:10.925+0000 7f5fe17fa640 1 -- 172.21.15.1:0/2004517791 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5fd40bc5a0 con 0x7f5fe4100a80 2024-09-11T21:56:11.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.102+0000 7f5fea36b640 1 -- 172.21.15.1:0/2004517791 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f5fe410a000 con 0x7f5fe4100a80 2024-09-11T21:56:11.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.102+0000 7f5fe17fa640 1 -- 172.21.15.1:0/2004517791 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5fd40bbea0 con 0x7f5fe4100a80 2024-09-11T21:56:11.103 INFO:teuthology.orchestra.run.smithi001.stdout:34359738447 2024-09-11T21:56:11.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.105+0000 7f5fea36b640 1 -- 172.21.15.1:0/2004517791 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f5fcc0a2c90 msgr2=0x7f5fcc0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:11.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.105+0000 7f5fea36b640 1 --2- 172.21.15.1:0/2004517791 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f5fcc0a2c90 0x7f5fcc0a5150 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8005fd0 tx=0x7f5fd8005e20 comp rx=0 tx=0).stop 2024-09-11T21:56:11.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.105+0000 7f5fea36b640 1 -- 172.21.15.1:0/2004517791 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5fe4100a80 msgr2=0x7f5fe4198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:11.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.105+0000 7f5fea36b640 1 --2- 172.21.15.1:0/2004517791 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5fe4100a80 0x7f5fe4198460 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4009a50 tx=0x7f5fd4004270 comp rx=0 tx=0).stop 2024-09-11T21:56:11.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.105+0000 7f5fea36b640 1 -- 172.21.15.1:0/2004517791 shutdown_connections 2024-09-11T21:56:11.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.105+0000 7f5fea36b640 1 --2- 172.21.15.1:0/2004517791 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f5fcc0a2c90 0x7f5fcc0a5150 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:11.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.105+0000 7f5fea36b640 1 --2- 172.21.15.1:0/2004517791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fe4106a80 0x7f5fe419cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:11.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.105+0000 7f5fea36b640 1 --2- 172.21.15.1:0/2004517791 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5fe4101420 0x7f5fe41989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:11.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.105+0000 7f5fea36b640 1 --2- 172.21.15.1:0/2004517791 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5fe4100a80 0x7f5fe4198460 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:11.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.105+0000 7f5fea36b640 1 -- 172.21.15.1:0/2004517791 >> 172.21.15.1:0/2004517791 conn(0x7f5fe40fc7a0 msgr2=0x7f5fe406aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:11.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.105+0000 7f5fea36b640 1 -- 172.21.15.1:0/2004517791 shutdown_connections 2024-09-11T21:56:11.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.105+0000 7f5fea36b640 1 -- 172.21.15.1:0/2004517791 wait complete. 2024-09-11T21:56:11.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.310+0000 7fb7c62b2640 1 -- 172.21.15.1:0/2415302027 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb7c01036e0 msgr2=0x7fb7c0107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:11.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.310+0000 7fb7c62b2640 1 --2- 172.21.15.1:0/2415302027 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb7c01036e0 0x7fb7c0107b90 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb7b4009930 tx=0x7fb7b402a420 comp rx=0 tx=0).stop 2024-09-11T21:56:11.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.310+0000 7fb7c62b2640 1 -- 172.21.15.1:0/2415302027 shutdown_connections 2024-09-11T21:56:11.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.310+0000 7fb7c62b2640 1 --2- 172.21.15.1:0/2415302027 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7c01082a0 0x7fb7c010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:11.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.310+0000 7fb7c62b2640 1 --2- 172.21.15.1:0/2415302027 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb7c01036e0 0x7fb7c0107b90 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:11.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.310+0000 7fb7c62b2640 1 --2- 172.21.15.1:0/2415302027 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7c0102d30 0x7fb7c0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:11.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.310+0000 7fb7c62b2640 1 -- 172.21.15.1:0/2415302027 >> 172.21.15.1:0/2415302027 conn(0x7fb7c00fc7d0 msgr2=0x7fb7c00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:11.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.310+0000 7fb7c62b2640 1 -- 172.21.15.1:0/2415302027 shutdown_connections 2024-09-11T21:56:11.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.310+0000 7fb7c62b2640 1 -- 172.21.15.1:0/2415302027 wait complete. 2024-09-11T21:56:11.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.311+0000 7fb7c62b2640 1 Processor -- start 2024-09-11T21:56:11.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.311+0000 7fb7c62b2640 1 -- start start 2024-09-11T21:56:11.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.311+0000 7fb7c62b2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb7c0102d30 0x7fb7c0198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:11.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.311+0000 7fb7c62b2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7c01036e0 0x7fb7c0198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:11.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.311+0000 7fb7c62b2640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7c01082a0 0x7fb7c019cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:11.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.311+0000 7fb7c62b2640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c01120c0 con 0x7fb7c0102d30 2024-09-11T21:56:11.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.311+0000 7fb7c62b2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c0111f40 con 0x7fb7c01036e0 2024-09-11T21:56:11.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.311+0000 7fb7c62b2640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c0112240 con 0x7fb7c01082a0 2024-09-11T21:56:11.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.311+0000 7fb7bffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb7c0102d30 0x7fb7c0198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:11.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.311+0000 7fb7bffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb7c0102d30 0x7fb7c0198680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43006/0 (socket says 172.21.15.1:43006) 2024-09-11T21:56:11.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.311+0000 7fb7bffff640 1 -- 172.21.15.1:0/2239472993 learned_addr learned my addr 172.21.15.1:0/2239472993 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:56:11.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.312+0000 7fb7bf7fe640 1 --2- 172.21.15.1:0/2239472993 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7c01036e0 0x7fb7c0198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:11.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.312+0000 7fb7bffff640 1 -- 172.21.15.1:0/2239472993 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7c01082a0 msgr2=0x7fb7c019cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:11.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.312+0000 7fb7bffff640 1 --2- 172.21.15.1:0/2239472993 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7c01082a0 0x7fb7c019cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:11.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.312+0000 7fb7bffff640 1 -- 172.21.15.1:0/2239472993 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7c01036e0 msgr2=0x7fb7c0198bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:11.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.312+0000 7fb7bffff640 1 --2- 172.21.15.1:0/2239472993 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7c01036e0 0x7fb7c0198bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:11.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.312+0000 7fb7bffff640 1 -- 172.21.15.1:0/2239472993 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7c019d630 con 0x7fb7c0102d30 2024-09-11T21:56:11.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.312+0000 7fb7bf7fe640 1 --2- 172.21.15.1:0/2239472993 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7c01036e0 0x7fb7c0198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T21:56:11.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.312+0000 7fb7bffff640 1 --2- 172.21.15.1:0/2239472993 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb7c0102d30 0x7fb7c0198680 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb7b000b4a0 tx=0x7fb7b000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:11.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.312+0000 7fb7bd7fa640 1 -- 172.21.15.1:0/2239472993 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b0014070 con 0x7fb7c0102d30 2024-09-11T21:56:11.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.312+0000 7fb7c62b2640 1 -- 172.21.15.1:0/2239472993 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7c019d920 con 0x7fb7c0102d30 2024-09-11T21:56:11.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.312+0000 7fb7bd7fa640 1 -- 172.21.15.1:0/2239472993 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7b0010af0 con 0x7fb7c0102d30 2024-09-11T21:56:11.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.312+0000 7fb7bd7fa640 1 -- 172.21.15.1:0/2239472993 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b0017d10 con 0x7fb7c0102d30 2024-09-11T21:56:11.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.312+0000 7fb7c62b2640 1 -- 172.21.15.1:0/2239472993 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7c01a5200 con 0x7fb7c0102d30 2024-09-11T21:56:11.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.313+0000 7fb7c62b2640 1 -- 172.21.15.1:0/2239472993 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7c006a850 con 0x7fb7c0102d30 2024-09-11T21:56:11.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.314+0000 7fb7bd7fa640 1 -- 172.21.15.1:0/2239472993 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fb7b0017070 con 0x7fb7c0102d30 2024-09-11T21:56:11.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.315+0000 7fb7bd7fa640 1 --2- 172.21.15.1:0/2239472993 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fb7a80a2c10 0x7fb7a80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:11.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.315+0000 7fb7bd7fa640 1 -- 172.21.15.1:0/2239472993 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7fb7b00ce390 con 0x7fb7c0102d30 2024-09-11T21:56:11.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.315+0000 7fb7bf7fe640 1 --2- 172.21.15.1:0/2239472993 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fb7a80a2c10 0x7fb7a80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:11.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.316+0000 7fb7bf7fe640 1 --2- 172.21.15.1:0/2239472993 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fb7a80a2c10 0x7fb7a80a50d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb7b402a940 tx=0x7fb7b4032c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:11.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.319+0000 7fb7bd7fa640 1 -- 172.21.15.1:0/2239472993 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb7b009b450 con 0x7fb7c0102d30 2024-09-11T21:56:11.405 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:11 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/2004517791' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-11T21:56:11.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.189+0000 7faf5b12f640 1 -- 172.21.15.1:0/1441793768 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf54102570 msgr2=0x7faf541029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:11.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.189+0000 7faf5b12f640 1 --2- 172.21.15.1:0/1441793768 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf54102570 0x7faf541029b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7faf3c009900 tx=0x7faf3c02a480 comp rx=0 tx=0).stop 2024-09-11T21:56:11.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.189+0000 7faf5b12f640 1 -- 172.21.15.1:0/1441793768 shutdown_connections 2024-09-11T21:56:11.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.189+0000 7faf5b12f640 1 --2- 172.21.15.1:0/1441793768 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faf54102ef0 0x7faf5410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:11.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.189+0000 7faf5b12f640 1 --2- 172.21.15.1:0/1441793768 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf54102570 0x7faf541029b0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:11.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.189+0000 7faf5b12f640 1 --2- 172.21.15.1:0/1441793768 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf54100e20 0x7faf54101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:11.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.190+0000 7faf5b12f640 1 -- 172.21.15.1:0/1441793768 >> 172.21.15.1:0/1441793768 conn(0x7faf540fc7d0 msgr2=0x7faf540febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:11.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.190+0000 7faf5b12f640 1 -- 172.21.15.1:0/1441793768 shutdown_connections 2024-09-11T21:56:11.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.190+0000 7faf5b12f640 1 -- 172.21.15.1:0/1441793768 wait complete. 2024-09-11T21:56:11.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.190+0000 7faf5b12f640 1 Processor -- start 2024-09-11T21:56:11.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.190+0000 7faf5b12f640 1 -- start start 2024-09-11T21:56:11.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.190+0000 7faf5b12f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf54100e20 0x7faf540fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:11.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.190+0000 7faf5b12f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faf54102570 0x7faf540fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:11.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.190+0000 7faf5b12f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf54102ef0 0x7faf540fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:11.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.190+0000 7faf5b12f640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7faf54110f50 con 0x7faf54100e20 2024-09-11T21:56:11.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.190+0000 7faf5b12f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7faf54110dd0 con 0x7faf54102ef0 2024-09-11T21:56:11.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.190+0000 7faf5b12f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7faf541110d0 con 0x7faf54102570 2024-09-11T21:56:11.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.191+0000 7faf58ecb640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf54100e20 0x7faf540fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:11.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.191+0000 7faf58ecb640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf54100e20 0x7faf540fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42988/0 (socket says 172.21.15.1:42988) 2024-09-11T21:56:11.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.191+0000 7faf58ecb640 1 -- 172.21.15.1:0/208779098 learned_addr learned my addr 172.21.15.1:0/208779098 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:56:11.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.191+0000 7faf53fff640 1 --2- 172.21.15.1:0/208779098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faf54102570 0x7faf540fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:11.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.191+0000 7faf596cc640 1 --2- 172.21.15.1:0/208779098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf54102ef0 0x7faf540fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:11.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.191+0000 7faf58ecb640 1 -- 172.21.15.1:0/208779098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faf54102570 msgr2=0x7faf540fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:11.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.191+0000 7faf58ecb640 1 --2- 172.21.15.1:0/208779098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faf54102570 0x7faf540fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:11.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.191+0000 7faf58ecb640 1 -- 172.21.15.1:0/208779098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf54102ef0 msgr2=0x7faf540fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:11.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.191+0000 7faf58ecb640 1 --2- 172.21.15.1:0/208779098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf54102ef0 0x7faf540fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:11.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.191+0000 7faf58ecb640 1 -- 172.21.15.1:0/208779098 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf541a6880 con 0x7faf54100e20 2024-09-11T21:56:11.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.191+0000 7faf58ecb640 1 --2- 172.21.15.1:0/208779098 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf54100e20 0x7faf540fcc80 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7faf4c00c920 tx=0x7faf4c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:11.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.191+0000 7faf51ffb640 1 -- 172.21.15.1:0/208779098 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf4c007c70 con 0x7faf54100e20 2024-09-11T21:56:11.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.191+0000 7faf51ffb640 1 -- 172.21.15.1:0/208779098 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf4c007e10 con 0x7faf54100e20 2024-09-11T21:56:11.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.191+0000 7faf5b12f640 1 -- 172.21.15.1:0/208779098 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf541a6b70 con 0x7faf54100e20 2024-09-11T21:56:11.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.191+0000 7faf51ffb640 1 -- 172.21.15.1:0/208779098 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf4c018890 con 0x7faf54100e20 2024-09-11T21:56:11.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.191+0000 7faf5b12f640 1 -- 172.21.15.1:0/208779098 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf541a7080 con 0x7faf54100e20 2024-09-11T21:56:11.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.192+0000 7faf5b12f640 1 -- 172.21.15.1:0/208779098 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf54101200 con 0x7faf54100e20 2024-09-11T21:56:11.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.194+0000 7faf51ffb640 1 -- 172.21.15.1:0/208779098 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7faf4c018a50 con 0x7faf54100e20 2024-09-11T21:56:11.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.194+0000 7faf51ffb640 1 --2- 172.21.15.1:0/208779098 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7faf400a2c10 0x7faf400a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:11.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.194+0000 7faf53fff640 1 --2- 172.21.15.1:0/208779098 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7faf400a2c10 0x7faf400a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:11.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.194+0000 7faf51ffb640 1 -- 172.21.15.1:0/208779098 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7faf4c0cf400 con 0x7faf54100e20 2024-09-11T21:56:11.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.195+0000 7faf53fff640 1 --2- 172.21.15.1:0/208779098 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7faf400a2c10 0x7faf400a50d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7faf3c000c00 tx=0x7faf3c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:11.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.197+0000 7faf51ffb640 1 -- 172.21.15.1:0/208779098 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faf4c014030 con 0x7faf54100e20 2024-09-11T21:56:11.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.404+0000 7faf5b12f640 1 -- 172.21.15.1:0/208779098 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7faf540fe1b0 con 0x7faf54100e20 2024-09-11T21:56:11.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.405+0000 7faf51ffb640 1 -- 172.21.15.1:0/208779098 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7faf4c09c440 con 0x7faf54100e20 2024-09-11T21:56:11.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.408+0000 7faf5b12f640 1 -- 172.21.15.1:0/208779098 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7faf400a2c10 msgr2=0x7faf400a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:11.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.408+0000 7faf5b12f640 1 --2- 172.21.15.1:0/208779098 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7faf400a2c10 0x7faf400a50d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7faf3c000c00 tx=0x7faf3c0058d0 comp rx=0 tx=0).stop 2024-09-11T21:56:11.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.408+0000 7faf5b12f640 1 -- 172.21.15.1:0/208779098 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf54100e20 msgr2=0x7faf540fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:11.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.408+0000 7faf5b12f640 1 --2- 172.21.15.1:0/208779098 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf54100e20 0x7faf540fcc80 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7faf4c00c920 tx=0x7faf4c00cde0 comp rx=0 tx=0).stop 2024-09-11T21:56:11.503 INFO:teuthology.orchestra.run.smithi001.stdout:55834574923 2024-09-11T21:56:11.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.408+0000 7faf5b12f640 1 -- 172.21.15.1:0/208779098 shutdown_connections 2024-09-11T21:56:11.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.408+0000 7faf5b12f640 1 --2- 172.21.15.1:0/208779098 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7faf400a2c10 0x7faf400a50d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:11.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.408+0000 7faf5b12f640 1 --2- 172.21.15.1:0/208779098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf54102ef0 0x7faf540fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:11.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.408+0000 7faf5b12f640 1 --2- 172.21.15.1:0/208779098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faf54102570 0x7faf540fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:11.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.408+0000 7faf5b12f640 1 --2- 172.21.15.1:0/208779098 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf54100e20 0x7faf540fcc80 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:11.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.408+0000 7faf5b12f640 1 -- 172.21.15.1:0/208779098 >> 172.21.15.1:0/208779098 conn(0x7faf540fc7d0 msgr2=0x7faf5406a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:11.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.408+0000 7faf5b12f640 1 -- 172.21.15.1:0/208779098 shutdown_connections 2024-09-11T21:56:11.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.408+0000 7faf5b12f640 1 -- 172.21.15.1:0/208779098 wait complete. 2024-09-11T21:56:11.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.539+0000 7fb7c62b2640 1 -- 172.21.15.1:0/2239472993 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fb7c010a2b0 con 0x7fb7c0102d30 2024-09-11T21:56:11.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.540+0000 7fb7bd7fa640 1 -- 172.21.15.1:0/2239472993 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb7b009ad50 con 0x7fb7c0102d30 2024-09-11T21:56:11.541 INFO:teuthology.orchestra.run.smithi001.stdout:163208757302 2024-09-11T21:56:11.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.542+0000 7fb7c62b2640 1 -- 172.21.15.1:0/2239472993 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fb7a80a2c10 msgr2=0x7fb7a80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:11.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.542+0000 7fb7c62b2640 1 --2- 172.21.15.1:0/2239472993 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fb7a80a2c10 0x7fb7a80a50d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb7b402a940 tx=0x7fb7b4032c70 comp rx=0 tx=0).stop 2024-09-11T21:56:11.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.542+0000 7fb7c62b2640 1 -- 172.21.15.1:0/2239472993 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb7c0102d30 msgr2=0x7fb7c0198680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:11.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.542+0000 7fb7c62b2640 1 --2- 172.21.15.1:0/2239472993 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb7c0102d30 0x7fb7c0198680 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb7b000b4a0 tx=0x7fb7b000b960 comp rx=0 tx=0).stop 2024-09-11T21:56:11.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.542+0000 7fb7c62b2640 1 -- 172.21.15.1:0/2239472993 shutdown_connections 2024-09-11T21:56:11.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.542+0000 7fb7c62b2640 1 --2- 172.21.15.1:0/2239472993 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fb7a80a2c10 0x7fb7a80a50d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:11.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.542+0000 7fb7c62b2640 1 --2- 172.21.15.1:0/2239472993 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7c01082a0 0x7fb7c019cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:11.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.542+0000 7fb7c62b2640 1 --2- 172.21.15.1:0/2239472993 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7c01036e0 0x7fb7c0198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:11.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.542+0000 7fb7c62b2640 1 --2- 172.21.15.1:0/2239472993 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb7c0102d30 0x7fb7c0198680 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:11.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.542+0000 7fb7c62b2640 1 -- 172.21.15.1:0/2239472993 >> 172.21.15.1:0/2239472993 conn(0x7fb7c00fc7d0 msgr2=0x7fb7c0105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:11.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.542+0000 7fb7c62b2640 1 -- 172.21.15.1:0/2239472993 shutdown_connections 2024-09-11T21:56:11.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:11.542+0000 7fb7c62b2640 1 -- 172.21.15.1:0/2239472993 wait complete. 2024-09-11T21:56:11.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:11 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/2004517791' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-11T21:56:11.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:11 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/2004517791' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-11T21:56:12.529 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:12 smithi001 ceph-mon[30935]: pgmap v307: 97 pgs: 97 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-11T21:56:12.529 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:12 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/208779098' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-11T21:56:12.529 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:12 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/2239472993' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-11T21:56:12.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:12 smithi053 ceph-mon[34908]: pgmap v307: 97 pgs: 97 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-11T21:56:12.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:12 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/208779098' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-11T21:56:12.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:12 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/2239472993' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-11T21:56:12.565 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738445 got 34359738447 for osd.0 2024-09-11T21:56:12.565 DEBUG:teuthology.parallel:result is None 2024-09-11T21:56:12.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:12 smithi161 ceph-mon[35615]: pgmap v307: 97 pgs: 97 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-11T21:56:12.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:12 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/208779098' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-11T21:56:12.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:12 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/2239472993' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-11T21:56:13.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:12.986+0000 7ffa62144640 1 -- 172.21.15.1:0/231369312 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffa5c10c160 msgr2=0x7ffa5c10e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:13.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:12.986+0000 7ffa62144640 1 --2- 172.21.15.1:0/231369312 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffa5c10c160 0x7ffa5c10e600 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ffa54009ca0 tx=0x7ffa5402a950 comp rx=0 tx=0).stop 2024-09-11T21:56:13.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:12.986+0000 7ffa62144640 1 -- 172.21.15.1:0/231369312 shutdown_connections 2024-09-11T21:56:13.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:12.986+0000 7ffa62144640 1 --2- 172.21.15.1:0/231369312 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffa5c10c160 0x7ffa5c10e600 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:13.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:12.986+0000 7ffa62144640 1 --2- 172.21.15.1:0/231369312 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa5c103650 0x7ffa5c10bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:13.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:12.986+0000 7ffa62144640 1 --2- 172.21.15.1:0/231369312 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffa5c102d30 0x7ffa5c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:13.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:12.986+0000 7ffa62144640 1 -- 172.21.15.1:0/231369312 >> 172.21.15.1:0/231369312 conn(0x7ffa5c0fc7d0 msgr2=0x7ffa5c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:13.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:12.986+0000 7ffa62144640 1 -- 172.21.15.1:0/231369312 shutdown_connections 2024-09-11T21:56:13.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:12.986+0000 7ffa62144640 1 -- 172.21.15.1:0/231369312 wait complete. 2024-09-11T21:56:13.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:12.987+0000 7ffa62144640 1 Processor -- start 2024-09-11T21:56:13.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:12.987+0000 7ffa62144640 1 -- start start 2024-09-11T21:56:13.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:12.987+0000 7ffa62144640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa5c102d30 0x7ffa5c100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:13.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:12.987+0000 7ffa62144640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffa5c103650 0x7ffa5c100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:13.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:12.987+0000 7ffa62144640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffa5c10c160 0x7ffa5c101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:13.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:12.987+0000 7ffa62144640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5c111f20 con 0x7ffa5c10c160 2024-09-11T21:56:13.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:12.987+0000 7ffa62144640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5c111da0 con 0x7ffa5c103650 2024-09-11T21:56:13.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:12.987+0000 7ffa62144640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5c1120a0 con 0x7ffa5c102d30 2024-09-11T21:56:13.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:12.987+0000 7ffa5bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffa5c10c160 0x7ffa5c101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:13.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:12.987+0000 7ffa5bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffa5c10c160 0x7ffa5c101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43026/0 (socket says 172.21.15.1:43026) 2024-09-11T21:56:13.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:12.987+0000 7ffa5bfff640 1 -- 172.21.15.1:0/4211921229 learned_addr learned my addr 172.21.15.1:0/4211921229 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:56:13.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:12.987+0000 7ffa5b7fe640 1 --2- 172.21.15.1:0/4211921229 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa5c102d30 0x7ffa5c100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:13.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:12.987+0000 7ffa5affd640 1 --2- 172.21.15.1:0/4211921229 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffa5c103650 0x7ffa5c100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:13.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:12.988+0000 7ffa5bfff640 1 -- 172.21.15.1:0/4211921229 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa5c102d30 msgr2=0x7ffa5c100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:13.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:12.988+0000 7ffa5bfff640 1 --2- 172.21.15.1:0/4211921229 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa5c102d30 0x7ffa5c100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:13.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:12.988+0000 7ffa5bfff640 1 -- 172.21.15.1:0/4211921229 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffa5c103650 msgr2=0x7ffa5c100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:13.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:12.988+0000 7ffa5bfff640 1 --2- 172.21.15.1:0/4211921229 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffa5c103650 0x7ffa5c100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:13.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:12.988+0000 7ffa5bfff640 1 -- 172.21.15.1:0/4211921229 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa5c1a68a0 con 0x7ffa5c10c160 2024-09-11T21:56:13.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:12.988+0000 7ffa5bfff640 1 --2- 172.21.15.1:0/4211921229 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffa5c10c160 0x7ffa5c101410 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ffa5402ae70 tx=0x7ffa5402f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:13.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:12.988+0000 7ffa58ff9640 1 -- 172.21.15.1:0/4211921229 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa54034040 con 0x7ffa5c10c160 2024-09-11T21:56:13.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:12.988+0000 7ffa58ff9640 1 -- 172.21.15.1:0/4211921229 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa54030920 con 0x7ffa5c10c160 2024-09-11T21:56:13.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:12.988+0000 7ffa62144640 1 -- 172.21.15.1:0/4211921229 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa5c1a6b30 con 0x7ffa5c10c160 2024-09-11T21:56:13.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:12.988+0000 7ffa58ff9640 1 -- 172.21.15.1:0/4211921229 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa54045480 con 0x7ffa5c10c160 2024-09-11T21:56:13.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:12.988+0000 7ffa62144640 1 -- 172.21.15.1:0/4211921229 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa5c1a7060 con 0x7ffa5c10c160 2024-09-11T21:56:13.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:12.989+0000 7ffa62144640 1 -- 172.21.15.1:0/4211921229 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa5c103110 con 0x7ffa5c10c160 2024-09-11T21:56:13.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:12.990+0000 7ffa58ff9640 1 -- 172.21.15.1:0/4211921229 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7ffa54037020 con 0x7ffa5c10c160 2024-09-11T21:56:13.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:12.991+0000 7ffa58ff9640 1 --2- 172.21.15.1:0/4211921229 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7ffa440a2c90 0x7ffa440a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:13.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:12.991+0000 7ffa5b7fe640 1 --2- 172.21.15.1:0/4211921229 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7ffa440a2c90 0x7ffa440a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:13.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:12.991+0000 7ffa58ff9640 1 -- 172.21.15.1:0/4211921229 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7ffa540efef0 con 0x7ffa5c10c160 2024-09-11T21:56:13.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:12.992+0000 7ffa5b7fe640 1 --2- 172.21.15.1:0/4211921229 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7ffa440a2c90 0x7ffa440a5150 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c005d00 tx=0x7ffa4c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:13.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:12.994+0000 7ffa58ff9640 1 -- 172.21.15.1:0/4211921229 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffa540811b0 con 0x7ffa5c10c160 2024-09-11T21:56:13.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:13.171+0000 7ffa62144640 1 -- 172.21.15.1:0/4211921229 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7ffa5c02d7c0 con 0x7ffa5c10c160 2024-09-11T21:56:13.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:13.171+0000 7ffa58ff9640 1 -- 172.21.15.1:0/4211921229 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ffa540bcfb0 con 0x7ffa5c10c160 2024-09-11T21:56:13.172 INFO:teuthology.orchestra.run.smithi001.stdout:77309411398 2024-09-11T21:56:13.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:13.174+0000 7ffa62144640 1 -- 172.21.15.1:0/4211921229 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7ffa440a2c90 msgr2=0x7ffa440a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:13.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:13.174+0000 7ffa62144640 1 --2- 172.21.15.1:0/4211921229 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7ffa440a2c90 0x7ffa440a5150 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c005d00 tx=0x7ffa4c00b410 comp rx=0 tx=0).stop 2024-09-11T21:56:13.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:13.174+0000 7ffa62144640 1 -- 172.21.15.1:0/4211921229 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffa5c10c160 msgr2=0x7ffa5c101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:13.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:13.174+0000 7ffa62144640 1 --2- 172.21.15.1:0/4211921229 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffa5c10c160 0x7ffa5c101410 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ffa5402ae70 tx=0x7ffa5402f5b0 comp rx=0 tx=0).stop 2024-09-11T21:56:13.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:13.174+0000 7ffa62144640 1 -- 172.21.15.1:0/4211921229 shutdown_connections 2024-09-11T21:56:13.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:13.174+0000 7ffa62144640 1 --2- 172.21.15.1:0/4211921229 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7ffa440a2c90 0x7ffa440a5150 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:13.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:13.174+0000 7ffa62144640 1 --2- 172.21.15.1:0/4211921229 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffa5c10c160 0x7ffa5c101410 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:13.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:13.174+0000 7ffa62144640 1 --2- 172.21.15.1:0/4211921229 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffa5c103650 0x7ffa5c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:13.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:13.174+0000 7ffa62144640 1 --2- 172.21.15.1:0/4211921229 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa5c102d30 0x7ffa5c100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:13.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:13.174+0000 7ffa62144640 1 -- 172.21.15.1:0/4211921229 >> 172.21.15.1:0/4211921229 conn(0x7ffa5c0fc7d0 msgr2=0x7ffa5c0fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:13.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:13.174+0000 7ffa62144640 1 -- 172.21.15.1:0/4211921229 shutdown_connections 2024-09-11T21:56:13.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:13.174+0000 7ffa62144640 1 -- 172.21.15.1:0/4211921229 wait complete. 2024-09-11T21:56:13.422 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:13 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/4211921229' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-11T21:56:13.452 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574920 got 55834574923 for osd.1 2024-09-11T21:56:13.452 DEBUG:teuthology.parallel:result is None 2024-09-11T21:56:13.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:13 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/4211921229' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-11T21:56:13.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:13 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/4211921229' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-11T21:56:13.866 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757301 got 163208757302 for osd.5 2024-09-11T21:56:13.867 DEBUG:teuthology.parallel:result is None 2024-09-11T21:56:14.278 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:56:14.279 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:56:14.284 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:56:14.286 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:56:14.287 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:56:14.287 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:56:14.327 INFO:teuthology.orchestra.run.smithi001.stdout:141733920826 2024-09-11T21:56:14.327 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-11T21:56:14.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:14 smithi053 ceph-mon[34908]: pgmap v308: 97 pgs: 97 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T21:56:14.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:14 smithi161 ceph-mon[35615]: pgmap v308: 97 pgs: 97 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T21:56:14.598 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:14 smithi001 ceph-mon[30935]: pgmap v308: 97 pgs: 97 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T21:56:14.934 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411396 got 77309411398 for osd.2 2024-09-11T21:56:14.935 DEBUG:teuthology.parallel:result is None 2024-09-11T21:56:14.953 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:56:15.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:15.624+0000 7f5318e83640 1 -- 172.21.15.1:0/1790080020 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5314100a80 msgr2=0x7f5314100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:15.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:15.624+0000 7f5318e83640 1 --2- 172.21.15.1:0/1790080020 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5314100a80 0x7f5314100ee0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5308009930 tx=0x7f530802a420 comp rx=0 tx=0).stop 2024-09-11T21:56:15.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:15.624+0000 7f5318e83640 1 -- 172.21.15.1:0/1790080020 shutdown_connections 2024-09-11T21:56:15.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:15.624+0000 7f5318e83640 1 --2- 172.21.15.1:0/1790080020 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5314101420 0x7f531410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:15.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:15.624+0000 7f5318e83640 1 --2- 172.21.15.1:0/1790080020 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5314100a80 0x7f5314100ee0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:15.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:15.624+0000 7f5318e83640 1 --2- 172.21.15.1:0/1790080020 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5314106a80 0x7f5314106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:15.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:15.624+0000 7f5318e83640 1 -- 172.21.15.1:0/1790080020 >> 172.21.15.1:0/1790080020 conn(0x7f53140fc7a0 msgr2=0x7f53140febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:15.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:15.624+0000 7f5318e83640 1 -- 172.21.15.1:0/1790080020 shutdown_connections 2024-09-11T21:56:15.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:15.624+0000 7f5318e83640 1 -- 172.21.15.1:0/1790080020 wait complete. 2024-09-11T21:56:15.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:15.625+0000 7f5318e83640 1 Processor -- start 2024-09-11T21:56:15.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:15.625+0000 7f5318e83640 1 -- start start 2024-09-11T21:56:15.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:15.626+0000 7f5318e83640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5314100a80 0x7f5314199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:15.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:15.626+0000 7f5318e83640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5314101420 0x7f5314199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:15.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:15.626+0000 7f5318e83640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5314106a80 0x7f53141a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:15.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:15.626+0000 7f5318e83640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5314110ea0 con 0x7f5314101420 2024-09-11T21:56:15.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:15.626+0000 7f5318e83640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5314110d20 con 0x7f5314100a80 2024-09-11T21:56:15.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:15.626+0000 7f5318e83640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5314111020 con 0x7f5314106a80 2024-09-11T21:56:15.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:15.626+0000 7f5311d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5314101420 0x7f5314199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:15.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:15.626+0000 7f5311d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5314101420 0x7f5314199870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43040/0 (socket says 172.21.15.1:43040) 2024-09-11T21:56:15.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:15.626+0000 7f5311d9b640 1 -- 172.21.15.1:0/1166898178 learned_addr learned my addr 172.21.15.1:0/1166898178 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:56:15.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:15.626+0000 7f5312d9d640 1 --2- 172.21.15.1:0/1166898178 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5314106a80 0x7f53141a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:15.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:15.626+0000 7f531259c640 1 --2- 172.21.15.1:0/1166898178 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5314100a80 0x7f5314199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:15.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:15.626+0000 7f5311d9b640 1 -- 172.21.15.1:0/1166898178 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5314106a80 msgr2=0x7f53141a43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:15.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:15.626+0000 7f5311d9b640 1 --2- 172.21.15.1:0/1166898178 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5314106a80 0x7f53141a43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:15.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:15.626+0000 7f5311d9b640 1 -- 172.21.15.1:0/1166898178 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5314100a80 msgr2=0x7f5314199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:15.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:15.626+0000 7f5311d9b640 1 --2- 172.21.15.1:0/1166898178 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5314100a80 0x7f5314199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:15.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:15.626+0000 7f5311d9b640 1 -- 172.21.15.1:0/1166898178 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53141a48f0 con 0x7f5314101420 2024-09-11T21:56:15.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:15.626+0000 7f531259c640 1 --2- 172.21.15.1:0/1166898178 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5314100a80 0x7f5314199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T21:56:15.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:15.626+0000 7f5311d9b640 1 --2- 172.21.15.1:0/1166898178 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5314101420 0x7f5314199870 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5308009230 tx=0x7f5308004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:15.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:15.626+0000 7f53037fe640 1 -- 172.21.15.1:0/1166898178 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f530803f030 con 0x7f5314101420 2024-09-11T21:56:15.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:15.626+0000 7f53037fe640 1 -- 172.21.15.1:0/1166898178 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5308003ad0 con 0x7f5314101420 2024-09-11T21:56:15.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:15.626+0000 7f5318e83640 1 -- 172.21.15.1:0/1166898178 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53141a4ac0 con 0x7f5314101420 2024-09-11T21:56:15.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:15.626+0000 7f53037fe640 1 -- 172.21.15.1:0/1166898178 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5308035070 con 0x7f5314101420 2024-09-11T21:56:15.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:15.627+0000 7f5318e83640 1 -- 172.21.15.1:0/1166898178 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53141a4f70 con 0x7f5314101420 2024-09-11T21:56:15.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:15.628+0000 7f5318e83640 1 -- 172.21.15.1:0/1166898178 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5314100ee0 con 0x7f5314101420 2024-09-11T21:56:15.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:15.629+0000 7f53037fe640 1 -- 172.21.15.1:0/1166898178 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f5308042030 con 0x7f5314101420 2024-09-11T21:56:15.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:15.630+0000 7f53037fe640 1 --2- 172.21.15.1:0/1166898178 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f52fc0a2c10 0x7f52fc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:15.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:15.630+0000 7f531259c640 1 --2- 172.21.15.1:0/1166898178 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f52fc0a2c10 0x7f52fc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:15.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:15.630+0000 7f53037fe640 1 -- 172.21.15.1:0/1166898178 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f53080eef10 con 0x7f5314101420 2024-09-11T21:56:15.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:15.630+0000 7f531259c640 1 --2- 172.21.15.1:0/1166898178 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f52fc0a2c10 0x7f52fc0a50d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5304007eb0 tx=0x7f53040073c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:15.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:15.634+0000 7f53037fe640 1 -- 172.21.15.1:0/1166898178 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f53080bbec0 con 0x7f5314101420 2024-09-11T21:56:15.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:15.812+0000 7f5318e83640 1 -- 172.21.15.1:0/1166898178 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f531402d7c0 con 0x7f5314101420 2024-09-11T21:56:15.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:15.813+0000 7f53037fe640 1 -- 172.21.15.1:0/1166898178 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f53080bb7c0 con 0x7f5314101420 2024-09-11T21:56:15.814 INFO:teuthology.orchestra.run.smithi001.stdout:304942678045 2024-09-11T21:56:15.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:15.815+0000 7f5318e83640 1 -- 172.21.15.1:0/1166898178 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f52fc0a2c10 msgr2=0x7f52fc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:15.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:15.815+0000 7f5318e83640 1 --2- 172.21.15.1:0/1166898178 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f52fc0a2c10 0x7f52fc0a50d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5304007eb0 tx=0x7f53040073c0 comp rx=0 tx=0).stop 2024-09-11T21:56:15.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:15.815+0000 7f5318e83640 1 -- 172.21.15.1:0/1166898178 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5314101420 msgr2=0x7f5314199870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:15.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:15.815+0000 7f5318e83640 1 --2- 172.21.15.1:0/1166898178 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5314101420 0x7f5314199870 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5308009230 tx=0x7f5308004270 comp rx=0 tx=0).stop 2024-09-11T21:56:15.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:15.815+0000 7f5318e83640 1 -- 172.21.15.1:0/1166898178 shutdown_connections 2024-09-11T21:56:15.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:15.815+0000 7f5318e83640 1 --2- 172.21.15.1:0/1166898178 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f52fc0a2c10 0x7f52fc0a50d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:15.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:15.815+0000 7f5318e83640 1 --2- 172.21.15.1:0/1166898178 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5314106a80 0x7f53141a43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:15.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:15.815+0000 7f5318e83640 1 --2- 172.21.15.1:0/1166898178 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5314101420 0x7f5314199870 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:15.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:15.815+0000 7f5318e83640 1 --2- 172.21.15.1:0/1166898178 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5314100a80 0x7f5314199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:15.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:15.815+0000 7f5318e83640 1 -- 172.21.15.1:0/1166898178 >> 172.21.15.1:0/1166898178 conn(0x7f53140fc7a0 msgr2=0x7f53140fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:15.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:15.815+0000 7f5318e83640 1 -- 172.21.15.1:0/1166898178 shutdown_connections 2024-09-11T21:56:15.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:15.815+0000 7f5318e83640 1 -- 172.21.15.1:0/1166898178 wait complete. 2024-09-11T21:56:16.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:16 smithi053 ceph-mon[34908]: pgmap v309: 97 pgs: 97 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2024-09-11T21:56:16.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:16 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/1166898178' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-11T21:56:16.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:16 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:56:16.563 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:16 smithi161 ceph-mon[35615]: pgmap v309: 97 pgs: 97 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2024-09-11T21:56:16.564 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:16 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/1166898178' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-11T21:56:16.564 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:16 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:56:16.593 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:16 smithi001 ceph-mon[30935]: pgmap v309: 97 pgs: 97 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2024-09-11T21:56:16.593 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:16 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/1166898178' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-11T21:56:16.593 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:16 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:56:16.632 INFO:tasks.cephadm.ceph_manager.ceph:need seq 304942678042 got 304942678045 for osd.10 2024-09-11T21:56:16.632 DEBUG:teuthology.parallel:result is None 2024-09-11T21:56:17.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:16.957+0000 7ff79f239640 1 -- 172.21.15.1:0/2158810723 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff798102d30 msgr2=0x7ff798103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:17.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:16.957+0000 7ff79f239640 1 --2- 172.21.15.1:0/2158810723 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff798102d30 0x7ff798103110 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff790009930 tx=0x7ff79002a420 comp rx=0 tx=0).stop 2024-09-11T21:56:17.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:16.957+0000 7ff79f239640 1 -- 172.21.15.1:0/2158810723 shutdown_connections 2024-09-11T21:56:17.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:16.957+0000 7ff79f239640 1 --2- 172.21.15.1:0/2158810723 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff7981082a0 0x7ff79810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:17.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:16.957+0000 7ff79f239640 1 --2- 172.21.15.1:0/2158810723 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7981036e0 0x7ff798107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:17.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:16.957+0000 7ff79f239640 1 --2- 172.21.15.1:0/2158810723 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff798102d30 0x7ff798103110 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:17.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:16.957+0000 7ff79f239640 1 -- 172.21.15.1:0/2158810723 >> 172.21.15.1:0/2158810723 conn(0x7ff7980fc7d0 msgr2=0x7ff7980febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:17.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:16.957+0000 7ff79f239640 1 -- 172.21.15.1:0/2158810723 shutdown_connections 2024-09-11T21:56:17.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:16.957+0000 7ff79f239640 1 -- 172.21.15.1:0/2158810723 wait complete. 2024-09-11T21:56:17.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:16.957+0000 7ff79f239640 1 Processor -- start 2024-09-11T21:56:17.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:16.957+0000 7ff79f239640 1 -- start start 2024-09-11T21:56:17.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:16.958+0000 7ff79f239640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff798102d30 0x7ff798100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:17.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:16.958+0000 7ff79f239640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff7981036e0 0x7ff798100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:17.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:16.958+0000 7ff79f239640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff7981082a0 0x7ff798101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:17.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:16.958+0000 7ff79f239640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff7981120e0 con 0x7ff7981036e0 2024-09-11T21:56:17.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:16.958+0000 7ff79f239640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff798111f60 con 0x7ff798102d30 2024-09-11T21:56:17.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:16.958+0000 7ff79f239640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff798112260 con 0x7ff7981082a0 2024-09-11T21:56:17.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:16.958+0000 7ff797fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff7981036e0 0x7ff798100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:17.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:16.958+0000 7ff797fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff7981036e0 0x7ff798100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55674/0 (socket says 172.21.15.1:55674) 2024-09-11T21:56:17.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:16.958+0000 7ff797fff640 1 -- 172.21.15.1:0/3481651620 learned_addr learned my addr 172.21.15.1:0/3481651620 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:56:17.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:16.958+0000 7ff79cfd5640 1 --2- 172.21.15.1:0/3481651620 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff798102d30 0x7ff798100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:17.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:16.958+0000 7ff79d7d6640 1 --2- 172.21.15.1:0/3481651620 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff7981082a0 0x7ff798101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:17.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:16.958+0000 7ff797fff640 1 -- 172.21.15.1:0/3481651620 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff7981082a0 msgr2=0x7ff798101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:17.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:16.958+0000 7ff797fff640 1 --2- 172.21.15.1:0/3481651620 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff7981082a0 0x7ff798101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:17.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:16.958+0000 7ff797fff640 1 -- 172.21.15.1:0/3481651620 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff798102d30 msgr2=0x7ff798100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:17.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:16.958+0000 7ff797fff640 1 --2- 172.21.15.1:0/3481651620 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff798102d30 0x7ff798100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:17.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:16.958+0000 7ff797fff640 1 -- 172.21.15.1:0/3481651620 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7981a6a80 con 0x7ff7981036e0 2024-09-11T21:56:17.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:16.958+0000 7ff797fff640 1 --2- 172.21.15.1:0/3481651620 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff7981036e0 0x7ff798100ed0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff78000e9f0 tx=0x7ff78000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:17.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:16.958+0000 7ff795ffb640 1 -- 172.21.15.1:0/3481651620 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff78000ccc0 con 0x7ff7981036e0 2024-09-11T21:56:17.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:16.958+0000 7ff795ffb640 1 -- 172.21.15.1:0/3481651620 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff780010910 con 0x7ff7981036e0 2024-09-11T21:56:17.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:16.958+0000 7ff795ffb640 1 -- 172.21.15.1:0/3481651620 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff780018920 con 0x7ff7981036e0 2024-09-11T21:56:17.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:16.958+0000 7ff79f239640 1 -- 172.21.15.1:0/3481651620 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7981a6d70 con 0x7ff7981036e0 2024-09-11T21:56:17.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:16.959+0000 7ff79f239640 1 -- 172.21.15.1:0/3481651620 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7981a7200 con 0x7ff7981036e0 2024-09-11T21:56:17.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:16.960+0000 7ff79f239640 1 -- 172.21.15.1:0/3481651620 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff798103110 con 0x7ff7981036e0 2024-09-11T21:56:17.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:16.961+0000 7ff795ffb640 1 -- 172.21.15.1:0/3481651620 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7ff780010ac0 con 0x7ff7981036e0 2024-09-11T21:56:17.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:16.962+0000 7ff795ffb640 1 --2- 172.21.15.1:0/3481651620 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7ff7840a2c90 0x7ff7840a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:17.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:16.962+0000 7ff79cfd5640 1 --2- 172.21.15.1:0/3481651620 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7ff7840a2c90 0x7ff7840a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:17.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:16.962+0000 7ff795ffb640 1 -- 172.21.15.1:0/3481651620 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7ff7800cf280 con 0x7ff7981036e0 2024-09-11T21:56:17.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:16.962+0000 7ff79cfd5640 1 --2- 172.21.15.1:0/3481651620 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7ff7840a2c90 0x7ff7840a5150 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff790006010 tx=0x7ff790000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:17.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:16.966+0000 7ff795ffb640 1 -- 172.21.15.1:0/3481651620 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff78009bbe0 con 0x7ff7981036e0 2024-09-11T21:56:17.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.150+0000 7ff79f239640 1 -- 172.21.15.1:0/3481651620 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7ff79810a2b0 con 0x7ff7981036e0 2024-09-11T21:56:17.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.150+0000 7ff795ffb640 1 -- 172.21.15.1:0/3481651620 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7ff78009fee0 con 0x7ff7981036e0 2024-09-11T21:56:17.151 INFO:teuthology.orchestra.run.smithi001.stdout:335007449112 2024-09-11T21:56:17.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.152+0000 7ff79f239640 1 -- 172.21.15.1:0/3481651620 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7ff7840a2c90 msgr2=0x7ff7840a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:17.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.152+0000 7ff79f239640 1 --2- 172.21.15.1:0/3481651620 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7ff7840a2c90 0x7ff7840a5150 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff790006010 tx=0x7ff790000bc0 comp rx=0 tx=0).stop 2024-09-11T21:56:17.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.152+0000 7ff79f239640 1 -- 172.21.15.1:0/3481651620 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff7981036e0 msgr2=0x7ff798100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:17.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.152+0000 7ff79f239640 1 --2- 172.21.15.1:0/3481651620 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff7981036e0 0x7ff798100ed0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff78000e9f0 tx=0x7ff78000eeb0 comp rx=0 tx=0).stop 2024-09-11T21:56:17.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.152+0000 7ff79f239640 1 -- 172.21.15.1:0/3481651620 shutdown_connections 2024-09-11T21:56:17.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.152+0000 7ff79f239640 1 --2- 172.21.15.1:0/3481651620 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7ff7840a2c90 0x7ff7840a5150 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:17.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.152+0000 7ff79f239640 1 --2- 172.21.15.1:0/3481651620 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff7981082a0 0x7ff798101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:17.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.152+0000 7ff79f239640 1 --2- 172.21.15.1:0/3481651620 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff7981036e0 0x7ff798100ed0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:17.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.152+0000 7ff79f239640 1 --2- 172.21.15.1:0/3481651620 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff798102d30 0x7ff798100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:17.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.152+0000 7ff79f239640 1 -- 172.21.15.1:0/3481651620 >> 172.21.15.1:0/3481651620 conn(0x7ff7980fc7d0 msgr2=0x7ff798105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:17.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.153+0000 7ff79f239640 1 -- 172.21.15.1:0/3481651620 shutdown_connections 2024-09-11T21:56:17.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.153+0000 7ff79f239640 1 -- 172.21.15.1:0/3481651620 wait complete. 2024-09-11T21:56:17.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.062+0000 7fb03d75b640 1 -- 172.21.15.1:0/92997102 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb038106a90 msgr2=0x7fb038106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:17.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.062+0000 7fb03d75b640 1 --2- 172.21.15.1:0/92997102 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb038106a90 0x7fb038106e70 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb028009930 tx=0x7fb02802a420 comp rx=0 tx=0).stop 2024-09-11T21:56:17.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.062+0000 7fb03d75b640 1 -- 172.21.15.1:0/92997102 shutdown_connections 2024-09-11T21:56:17.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.062+0000 7fb03d75b640 1 --2- 172.21.15.1:0/92997102 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb038101430 0x7fb03810d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:17.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.062+0000 7fb03d75b640 1 --2- 172.21.15.1:0/92997102 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb038100a90 0x7fb038100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:17.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.062+0000 7fb03d75b640 1 --2- 172.21.15.1:0/92997102 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb038106a90 0x7fb038106e70 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:17.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.062+0000 7fb03d75b640 1 -- 172.21.15.1:0/92997102 >> 172.21.15.1:0/92997102 conn(0x7fb0380fc7d0 msgr2=0x7fb0380febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:17.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.062+0000 7fb03d75b640 1 -- 172.21.15.1:0/92997102 shutdown_connections 2024-09-11T21:56:17.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.062+0000 7fb03d75b640 1 -- 172.21.15.1:0/92997102 wait complete. 2024-09-11T21:56:17.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.063+0000 7fb03d75b640 1 Processor -- start 2024-09-11T21:56:17.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.063+0000 7fb03d75b640 1 -- start start 2024-09-11T21:56:17.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.063+0000 7fb03d75b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb038100a90 0x7fb038197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:17.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.063+0000 7fb03d75b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb038101430 0x7fb038197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:17.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.063+0000 7fb03d75b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb038106a90 0x7fb03819bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:17.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.063+0000 7fb03d75b640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb038110e90 con 0x7fb038101430 2024-09-11T21:56:17.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.063+0000 7fb03d75b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb038110d10 con 0x7fb038100a90 2024-09-11T21:56:17.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.063+0000 7fb03d75b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb038111010 con 0x7fb038106a90 2024-09-11T21:56:17.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.063+0000 7fb0367fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb038101430 0x7fb038197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:17.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.064+0000 7fb036ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb038100a90 0x7fb038197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:17.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.063+0000 7fb0367fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb038101430 0x7fb038197860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55692/0 (socket says 172.21.15.1:55692) 2024-09-11T21:56:17.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.064+0000 7fb0367fc640 1 -- 172.21.15.1:0/3048281040 learned_addr learned my addr 172.21.15.1:0/3048281040 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:56:17.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.064+0000 7fb0377fe640 1 --2- 172.21.15.1:0/3048281040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb038106a90 0x7fb03819bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:17.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.064+0000 7fb0367fc640 1 -- 172.21.15.1:0/3048281040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb038106a90 msgr2=0x7fb03819bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:17.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.064+0000 7fb0367fc640 1 --2- 172.21.15.1:0/3048281040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb038106a90 0x7fb03819bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:17.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.064+0000 7fb0367fc640 1 -- 172.21.15.1:0/3048281040 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb038100a90 msgr2=0x7fb038197320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:17.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.064+0000 7fb0367fc640 1 --2- 172.21.15.1:0/3048281040 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb038100a90 0x7fb038197320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:17.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.064+0000 7fb0367fc640 1 -- 172.21.15.1:0/3048281040 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb03819c2d0 con 0x7fb038101430 2024-09-11T21:56:17.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.064+0000 7fb036ffd640 1 --2- 172.21.15.1:0/3048281040 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb038100a90 0x7fb038197320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T21:56:17.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.064+0000 7fb0367fc640 1 --2- 172.21.15.1:0/3048281040 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb038101430 0x7fb038197860 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb02c00e9f0 tx=0x7fb02c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:17.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.064+0000 7fb01ffff640 1 -- 172.21.15.1:0/3048281040 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb02c00cd80 con 0x7fb038101430 2024-09-11T21:56:17.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.064+0000 7fb03d75b640 1 -- 172.21.15.1:0/3048281040 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb03819c5c0 con 0x7fb038101430 2024-09-11T21:56:17.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.064+0000 7fb01ffff640 1 -- 172.21.15.1:0/3048281040 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb02c010910 con 0x7fb038101430 2024-09-11T21:56:17.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.064+0000 7fb01ffff640 1 -- 172.21.15.1:0/3048281040 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb02c0188f0 con 0x7fb038101430 2024-09-11T21:56:17.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.064+0000 7fb03d75b640 1 -- 172.21.15.1:0/3048281040 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0381a3ea0 con 0x7fb038101430 2024-09-11T21:56:17.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.065+0000 7fb03d75b640 1 -- 172.21.15.1:0/3048281040 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb038100ef0 con 0x7fb038101430 2024-09-11T21:56:17.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.067+0000 7fb01ffff640 1 -- 172.21.15.1:0/3048281040 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fb02c010ac0 con 0x7fb038101430 2024-09-11T21:56:17.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.068+0000 7fb01ffff640 1 --2- 172.21.15.1:0/3048281040 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fb0200a2c10 0x7fb0200a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:17.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.068+0000 7fb036ffd640 1 --2- 172.21.15.1:0/3048281040 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fb0200a2c10 0x7fb0200a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:17.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.068+0000 7fb01ffff640 1 -- 172.21.15.1:0/3048281040 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7fb02c0cf840 con 0x7fb038101430 2024-09-11T21:56:17.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.068+0000 7fb036ffd640 1 --2- 172.21.15.1:0/3048281040 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fb0200a2c10 0x7fb0200a50d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb028009900 tx=0x7fb0280058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:17.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.071+0000 7fb01ffff640 1 -- 172.21.15.1:0/3048281040 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb02c09c0e0 con 0x7fb038101430 2024-09-11T21:56:17.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.259+0000 7fb03d75b640 1 -- 172.21.15.1:0/3048281040 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fb038198680 con 0x7fb038101430 2024-09-11T21:56:17.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.259+0000 7fb01ffff640 1 -- 172.21.15.1:0/3048281040 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb038198680 con 0x7fb038101430 2024-09-11T21:56:17.260 INFO:teuthology.orchestra.run.smithi001.stdout:219043332141 2024-09-11T21:56:17.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.262+0000 7fb03d75b640 1 -- 172.21.15.1:0/3048281040 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fb0200a2c10 msgr2=0x7fb0200a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:17.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.262+0000 7fb03d75b640 1 --2- 172.21.15.1:0/3048281040 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fb0200a2c10 0x7fb0200a50d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb028009900 tx=0x7fb0280058d0 comp rx=0 tx=0).stop 2024-09-11T21:56:17.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.262+0000 7fb03d75b640 1 -- 172.21.15.1:0/3048281040 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb038101430 msgr2=0x7fb038197860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:17.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.262+0000 7fb03d75b640 1 --2- 172.21.15.1:0/3048281040 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb038101430 0x7fb038197860 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb02c00e9f0 tx=0x7fb02c00eeb0 comp rx=0 tx=0).stop 2024-09-11T21:56:17.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.262+0000 7fb03d75b640 1 -- 172.21.15.1:0/3048281040 shutdown_connections 2024-09-11T21:56:17.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.262+0000 7fb03d75b640 1 --2- 172.21.15.1:0/3048281040 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fb0200a2c10 0x7fb0200a50d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:17.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.262+0000 7fb03d75b640 1 --2- 172.21.15.1:0/3048281040 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb038106a90 0x7fb03819bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:17.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.262+0000 7fb03d75b640 1 --2- 172.21.15.1:0/3048281040 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb038101430 0x7fb038197860 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:17.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.262+0000 7fb03d75b640 1 --2- 172.21.15.1:0/3048281040 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb038100a90 0x7fb038197320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:17.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.262+0000 7fb03d75b640 1 -- 172.21.15.1:0/3048281040 >> 172.21.15.1:0/3048281040 conn(0x7fb0380fc7d0 msgr2=0x7fb0380fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:17.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.262+0000 7fb03d75b640 1 -- 172.21.15.1:0/3048281040 shutdown_connections 2024-09-11T21:56:17.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.262+0000 7fb03d75b640 1 -- 172.21.15.1:0/3048281040 wait complete. 2024-09-11T21:56:17.407 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:17 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/3481651620' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-11T21:56:17.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.406+0000 7f126124f640 1 -- 172.21.15.1:0/815293211 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f125c102570 msgr2=0x7f125c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:17.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.406+0000 7f126124f640 1 --2- 172.21.15.1:0/815293211 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f125c102570 0x7f125c1029b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1244009900 tx=0x7f124402a480 comp rx=0 tx=0).stop 2024-09-11T21:56:17.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.406+0000 7f126124f640 1 -- 172.21.15.1:0/815293211 shutdown_connections 2024-09-11T21:56:17.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.406+0000 7f126124f640 1 --2- 172.21.15.1:0/815293211 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f125c102ef0 0x7f125c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:17.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.406+0000 7f126124f640 1 --2- 172.21.15.1:0/815293211 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f125c102570 0x7f125c1029b0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:17.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.407+0000 7f126124f640 1 --2- 172.21.15.1:0/815293211 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f125c100e20 0x7f125c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:17.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.407+0000 7f126124f640 1 -- 172.21.15.1:0/815293211 >> 172.21.15.1:0/815293211 conn(0x7f125c0fc7d0 msgr2=0x7f125c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:17.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.407+0000 7f126124f640 1 -- 172.21.15.1:0/815293211 shutdown_connections 2024-09-11T21:56:17.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.407+0000 7f126124f640 1 -- 172.21.15.1:0/815293211 wait complete. 2024-09-11T21:56:17.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.407+0000 7f126124f640 1 Processor -- start 2024-09-11T21:56:17.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.408+0000 7f126124f640 1 -- start start 2024-09-11T21:56:17.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.408+0000 7f126124f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f125c100e20 0x7f125c198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:17.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.408+0000 7f126124f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f125c102570 0x7f125c198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:17.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.408+0000 7f126124f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f125c102ef0 0x7f125c19cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:17.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.408+0000 7f126124f640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f125c1120a0 con 0x7f125c102570 2024-09-11T21:56:17.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.408+0000 7f126124f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f125c111f20 con 0x7f125c100e20 2024-09-11T21:56:17.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.408+0000 7f126124f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f125c112220 con 0x7f125c102ef0 2024-09-11T21:56:17.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.408+0000 7f125a59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f125c102570 0x7f125c198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:17.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.408+0000 7f125ad9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f125c100e20 0x7f125c198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:17.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.408+0000 7f125ad9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f125c100e20 0x7f125c198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:35176/0 (socket says 172.21.15.1:35176) 2024-09-11T21:56:17.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.408+0000 7f125a59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f125c102570 0x7f125c198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55714/0 (socket says 172.21.15.1:55714) 2024-09-11T21:56:17.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.408+0000 7f125ad9d640 1 -- 172.21.15.1:0/1295043457 learned_addr learned my addr 172.21.15.1:0/1295043457 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:56:17.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.408+0000 7f125b59e640 1 --2- 172.21.15.1:0/1295043457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f125c102ef0 0x7f125c19cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:17.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.408+0000 7f125ad9d640 1 -- 172.21.15.1:0/1295043457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f125c102ef0 msgr2=0x7f125c19cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:17.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.408+0000 7f125ad9d640 1 --2- 172.21.15.1:0/1295043457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f125c102ef0 0x7f125c19cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:17.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.408+0000 7f125ad9d640 1 -- 172.21.15.1:0/1295043457 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f125c102570 msgr2=0x7f125c198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:17.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.408+0000 7f125ad9d640 1 --2- 172.21.15.1:0/1295043457 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f125c102570 0x7f125c198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:17.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.408+0000 7f125ad9d640 1 -- 172.21.15.1:0/1295043457 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f125c19d4b0 con 0x7f125c100e20 2024-09-11T21:56:17.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.409+0000 7f125ad9d640 1 --2- 172.21.15.1:0/1295043457 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f125c100e20 0x7f125c198500 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f124c00e9f0 tx=0x7f124c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:17.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.409+0000 7f1253fff640 1 -- 172.21.15.1:0/1295043457 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f124c00cd80 con 0x7f125c100e20 2024-09-11T21:56:17.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.409+0000 7f126124f640 1 -- 172.21.15.1:0/1295043457 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f125c19d7a0 con 0x7f125c100e20 2024-09-11T21:56:17.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.409+0000 7f1253fff640 1 -- 172.21.15.1:0/1295043457 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f124c010910 con 0x7f125c100e20 2024-09-11T21:56:17.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.409+0000 7f1253fff640 1 -- 172.21.15.1:0/1295043457 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f124c0188f0 con 0x7f125c100e20 2024-09-11T21:56:17.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.409+0000 7f126124f640 1 -- 172.21.15.1:0/1295043457 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f125c1a5080 con 0x7f125c100e20 2024-09-11T21:56:17.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.410+0000 7f126124f640 1 -- 172.21.15.1:0/1295043457 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f125c101200 con 0x7f125c100e20 2024-09-11T21:56:17.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.411+0000 7f1253fff640 1 -- 172.21.15.1:0/1295043457 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f124c010ac0 con 0x7f125c100e20 2024-09-11T21:56:17.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.412+0000 7f1253fff640 1 --2- 172.21.15.1:0/1295043457 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f12480a2c10 0x7f12480a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:17.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.412+0000 7f125a59c640 1 --2- 172.21.15.1:0/1295043457 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f12480a2c10 0x7f12480a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:17.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.412+0000 7f1253fff640 1 -- 172.21.15.1:0/1295043457 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f124c0cf9e0 con 0x7f125c100e20 2024-09-11T21:56:17.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.412+0000 7f125a59c640 1 --2- 172.21.15.1:0/1295043457 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f12480a2c10 0x7f12480a50d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1244009230 tx=0x7f12440058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:17.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.415+0000 7f1253fff640 1 -- 172.21.15.1:0/1295043457 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f124c09c240 con 0x7f125c100e20 2024-09-11T21:56:17.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:17 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/3481651620' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-11T21:56:17.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:17 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/3481651620' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-11T21:56:17.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.595+0000 7f126124f640 1 -- 172.21.15.1:0/1295043457 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f125c109090 con 0x7f125c100e20 2024-09-11T21:56:17.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.595+0000 7f1253fff640 1 -- 172.21.15.1:0/1295043457 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f124c0a0540 con 0x7f125c100e20 2024-09-11T21:56:17.596 INFO:teuthology.orchestra.run.smithi001.stdout:283467841571 2024-09-11T21:56:17.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.597+0000 7f126124f640 1 -- 172.21.15.1:0/1295043457 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f12480a2c10 msgr2=0x7f12480a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:17.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.597+0000 7f126124f640 1 --2- 172.21.15.1:0/1295043457 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f12480a2c10 0x7f12480a50d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1244009230 tx=0x7f12440058d0 comp rx=0 tx=0).stop 2024-09-11T21:56:17.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.597+0000 7f126124f640 1 -- 172.21.15.1:0/1295043457 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f125c100e20 msgr2=0x7f125c198500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:17.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.597+0000 7f126124f640 1 --2- 172.21.15.1:0/1295043457 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f125c100e20 0x7f125c198500 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f124c00e9f0 tx=0x7f124c00eeb0 comp rx=0 tx=0).stop 2024-09-11T21:56:17.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.597+0000 7f126124f640 1 -- 172.21.15.1:0/1295043457 shutdown_connections 2024-09-11T21:56:17.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.597+0000 7f126124f640 1 --2- 172.21.15.1:0/1295043457 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f12480a2c10 0x7f12480a50d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:17.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.597+0000 7f126124f640 1 --2- 172.21.15.1:0/1295043457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f125c102ef0 0x7f125c19cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:17.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.597+0000 7f126124f640 1 --2- 172.21.15.1:0/1295043457 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f125c102570 0x7f125c198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:17.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.597+0000 7f126124f640 1 --2- 172.21.15.1:0/1295043457 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f125c100e20 0x7f125c198500 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:17.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.597+0000 7f126124f640 1 -- 172.21.15.1:0/1295043457 >> 172.21.15.1:0/1295043457 conn(0x7f125c0fc7d0 msgr2=0x7f125c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:17.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.597+0000 7f126124f640 1 -- 172.21.15.1:0/1295043457 shutdown_connections 2024-09-11T21:56:17.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:17.597+0000 7f126124f640 1 -- 172.21.15.1:0/1295043457 wait complete. 2024-09-11T21:56:18.298 INFO:tasks.cephadm.ceph_manager.ceph:need seq 283467841568 got 283467841571 for osd.9 2024-09-11T21:56:18.298 DEBUG:teuthology.parallel:result is None 2024-09-11T21:56:18.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:18 smithi001 ceph-mon[30935]: pgmap v310: 97 pgs: 97 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-09-11T21:56:18.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:18 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/3048281040' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-11T21:56:18.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:18 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:56:18.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:18 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:56:18.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:18 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:56:18.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:18 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:56:18.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:18 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/1295043457' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-11T21:56:18.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:18 smithi053 ceph-mon[34908]: pgmap v310: 97 pgs: 97 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-09-11T21:56:18.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:18 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/3048281040' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-11T21:56:18.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:18 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:56:18.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:18 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:56:18.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:18 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:56:18.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:18 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:56:18.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:18 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/1295043457' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-11T21:56:18.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:18 smithi161 ceph-mon[35615]: pgmap v310: 97 pgs: 97 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-09-11T21:56:18.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:18 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/3048281040' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-11T21:56:18.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:18 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:56:18.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:18 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:56:18.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:18 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:56:18.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:18 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:56:18.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:18 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/1295043457' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-11T21:56:18.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:18.687+0000 7f179aca0640 1 -- 172.21.15.1:0/1532755937 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17941027a0 msgr2=0x7f17941067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:18.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:18.687+0000 7f179aca0640 1 --2- 172.21.15.1:0/1532755937 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17941027a0 0x7f17941067f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1780009900 tx=0x7f178002a410 comp rx=0 tx=0).stop 2024-09-11T21:56:18.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:18.687+0000 7f179aca0640 1 -- 172.21.15.1:0/1532755937 shutdown_connections 2024-09-11T21:56:18.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:18.687+0000 7f179aca0640 1 --2- 172.21.15.1:0/1532755937 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1794106f20 0x7f179410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:18.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:18.687+0000 7f179aca0640 1 --2- 172.21.15.1:0/1532755937 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17941027a0 0x7f17941067f0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:18.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:18.687+0000 7f179aca0640 1 --2- 172.21.15.1:0/1532755937 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1794101df0 0x7f17941021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:18.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:18.687+0000 7f179aca0640 1 -- 172.21.15.1:0/1532755937 >> 172.21.15.1:0/1532755937 conn(0x7f17940fd8b0 msgr2=0x7f17940ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:18.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:18.687+0000 7f179aca0640 1 -- 172.21.15.1:0/1532755937 shutdown_connections 2024-09-11T21:56:18.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:18.687+0000 7f179aca0640 1 -- 172.21.15.1:0/1532755937 wait complete. 2024-09-11T21:56:18.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:18.688+0000 7f179aca0640 1 Processor -- start 2024-09-11T21:56:18.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:18.688+0000 7f179aca0640 1 -- start start 2024-09-11T21:56:18.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:18.688+0000 7f179aca0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1794101df0 0x7f179419a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:18.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:18.688+0000 7f179aca0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17941027a0 0x7f179419ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:18.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:18.688+0000 7f179aca0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1794106f20 0x7f179419eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:18.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:18.688+0000 7f179aca0640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1794111f80 con 0x7f1794101df0 2024-09-11T21:56:18.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:18.688+0000 7f179aca0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1794111e00 con 0x7f17941027a0 2024-09-11T21:56:18.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:18.688+0000 7f179aca0640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1794112100 con 0x7f1794106f20 2024-09-11T21:56:18.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:18.688+0000 7f1798a3c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1794101df0 0x7f179419a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:18.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:18.688+0000 7f1798a3c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1794101df0 0x7f179419a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55728/0 (socket says 172.21.15.1:55728) 2024-09-11T21:56:18.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:18.688+0000 7f1798a3c640 1 -- 172.21.15.1:0/4010704313 learned_addr learned my addr 172.21.15.1:0/4010704313 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:56:18.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:18.688+0000 7f1793fff640 1 --2- 172.21.15.1:0/4010704313 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17941027a0 0x7f179419ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:18.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:18.688+0000 7f179923d640 1 --2- 172.21.15.1:0/4010704313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1794106f20 0x7f179419eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:18.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:18.689+0000 7f1798a3c640 1 -- 172.21.15.1:0/4010704313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1794106f20 msgr2=0x7f179419eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:18.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:18.689+0000 7f1798a3c640 1 --2- 172.21.15.1:0/4010704313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1794106f20 0x7f179419eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:18.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:18.689+0000 7f1798a3c640 1 -- 172.21.15.1:0/4010704313 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17941027a0 msgr2=0x7f179419ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:18.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:18.689+0000 7f1798a3c640 1 --2- 172.21.15.1:0/4010704313 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17941027a0 0x7f179419ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:18.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:18.689+0000 7f1798a3c640 1 -- 172.21.15.1:0/4010704313 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f179419f5b0 con 0x7f1794101df0 2024-09-11T21:56:18.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:18.689+0000 7f1798a3c640 1 --2- 172.21.15.1:0/4010704313 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1794101df0 0x7f179419a600 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f178800d770 tx=0x7f178800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:18.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:18.689+0000 7f1791ffb640 1 -- 172.21.15.1:0/4010704313 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1788017070 con 0x7f1794101df0 2024-09-11T21:56:18.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:18.689+0000 7f1791ffb640 1 -- 172.21.15.1:0/4010704313 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1788005500 con 0x7f1794101df0 2024-09-11T21:56:18.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:18.689+0000 7f179aca0640 1 -- 172.21.15.1:0/4010704313 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f179419f8a0 con 0x7f1794101df0 2024-09-11T21:56:18.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:18.689+0000 7f1791ffb640 1 -- 172.21.15.1:0/4010704313 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1788004e00 con 0x7f1794101df0 2024-09-11T21:56:18.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:18.689+0000 7f179aca0640 1 -- 172.21.15.1:0/4010704313 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17941a7180 con 0x7f1794101df0 2024-09-11T21:56:18.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:18.690+0000 7f179aca0640 1 -- 172.21.15.1:0/4010704313 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17941021d0 con 0x7f1794101df0 2024-09-11T21:56:18.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:18.692+0000 7f1791ffb640 1 -- 172.21.15.1:0/4010704313 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f1788004fa0 con 0x7f1794101df0 2024-09-11T21:56:18.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:18.693+0000 7f1791ffb640 1 --2- 172.21.15.1:0/4010704313 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f17840a2c90 0x7f17840a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:18.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:18.693+0000 7f1793fff640 1 --2- 172.21.15.1:0/4010704313 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f17840a2c90 0x7f17840a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:18.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:18.693+0000 7f1791ffb640 1 -- 172.21.15.1:0/4010704313 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f17880ce9f0 con 0x7f1794101df0 2024-09-11T21:56:18.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:18.693+0000 7f1793fff640 1 --2- 172.21.15.1:0/4010704313 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f17840a2c90 0x7f17840a5150 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1780009230 tx=0x7f17800058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:18.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:18.697+0000 7f1791ffb640 1 -- 172.21.15.1:0/4010704313 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f178809b9a0 con 0x7f1794101df0 2024-09-11T21:56:18.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:18.875+0000 7f179aca0640 1 -- 172.21.15.1:0/4010704313 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f17941a73b0 con 0x7f1794101df0 2024-09-11T21:56:18.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:18.876+0000 7f1791ffb640 1 -- 172.21.15.1:0/4010704313 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f178809b2a0 con 0x7f1794101df0 2024-09-11T21:56:18.877 INFO:teuthology.orchestra.run.smithi001.stdout:107374182466 2024-09-11T21:56:18.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:18.878+0000 7f179aca0640 1 -- 172.21.15.1:0/4010704313 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f17840a2c90 msgr2=0x7f17840a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:18.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:18.878+0000 7f179aca0640 1 --2- 172.21.15.1:0/4010704313 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f17840a2c90 0x7f17840a5150 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1780009230 tx=0x7f17800058d0 comp rx=0 tx=0).stop 2024-09-11T21:56:18.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:18.878+0000 7f179aca0640 1 -- 172.21.15.1:0/4010704313 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1794101df0 msgr2=0x7f179419a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:18.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:18.878+0000 7f179aca0640 1 --2- 172.21.15.1:0/4010704313 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1794101df0 0x7f179419a600 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f178800d770 tx=0x7f178800dc30 comp rx=0 tx=0).stop 2024-09-11T21:56:18.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:18.878+0000 7f179aca0640 1 -- 172.21.15.1:0/4010704313 shutdown_connections 2024-09-11T21:56:18.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:18.878+0000 7f179aca0640 1 --2- 172.21.15.1:0/4010704313 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f17840a2c90 0x7f17840a5150 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:18.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:18.878+0000 7f179aca0640 1 --2- 172.21.15.1:0/4010704313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1794106f20 0x7f179419eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:18.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:18.878+0000 7f179aca0640 1 --2- 172.21.15.1:0/4010704313 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17941027a0 0x7f179419ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:18.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:18.878+0000 7f179aca0640 1 --2- 172.21.15.1:0/4010704313 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1794101df0 0x7f179419a600 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:18.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:18.878+0000 7f179aca0640 1 -- 172.21.15.1:0/4010704313 >> 172.21.15.1:0/4010704313 conn(0x7f17940fd8b0 msgr2=0x7f17940fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:18.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:18.878+0000 7f179aca0640 1 -- 172.21.15.1:0/4010704313 shutdown_connections 2024-09-11T21:56:18.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:18.878+0000 7f179aca0640 1 -- 172.21.15.1:0/4010704313 wait complete. 2024-09-11T21:56:19.541 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:19 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/4010704313' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-11T21:56:19.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:19 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/4010704313' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-11T21:56:19.579 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332138 got 219043332141 for osd.7 2024-09-11T21:56:19.579 DEBUG:teuthology.parallel:result is None 2024-09-11T21:56:19.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:19 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/4010704313' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-11T21:56:20.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.347+0000 7f01fe38d640 1 -- 172.21.15.1:0/1509584023 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f01f8102790 msgr2=0x7f01f81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:20.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.347+0000 7f01fe38d640 1 --2- 172.21.15.1:0/1509584023 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f01f8102790 0x7f01f81067e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f01e0009900 tx=0x7f01e002a410 comp rx=0 tx=0).stop 2024-09-11T21:56:20.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.347+0000 7f01fe38d640 1 -- 172.21.15.1:0/1509584023 shutdown_connections 2024-09-11T21:56:20.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.347+0000 7f01fe38d640 1 --2- 172.21.15.1:0/1509584023 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01f8106f10 0x7f01f810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:20.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.347+0000 7f01fe38d640 1 --2- 172.21.15.1:0/1509584023 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f01f8102790 0x7f01f81067e0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:20.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.347+0000 7f01fe38d640 1 --2- 172.21.15.1:0/1509584023 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01f8101de0 0x7f01f81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:20.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.347+0000 7f01fe38d640 1 -- 172.21.15.1:0/1509584023 >> 172.21.15.1:0/1509584023 conn(0x7f01f80fd8c0 msgr2=0x7f01f80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:20.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.347+0000 7f01fe38d640 1 -- 172.21.15.1:0/1509584023 shutdown_connections 2024-09-11T21:56:20.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.347+0000 7f01fe38d640 1 -- 172.21.15.1:0/1509584023 wait complete. 2024-09-11T21:56:20.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.348+0000 7f01fe38d640 1 Processor -- start 2024-09-11T21:56:20.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.348+0000 7f01fe38d640 1 -- start start 2024-09-11T21:56:20.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.348+0000 7f01fe38d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01f8101de0 0x7f01f80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:20.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.348+0000 7f01fe38d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01f8102790 0x7f01f8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:20.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.348+0000 7f01fe38d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f01f8106f10 0x7f01f8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:20.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.348+0000 7f01fe38d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f01f8111f70 con 0x7f01f8106f10 2024-09-11T21:56:20.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.348+0000 7f01fe38d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f01f8111df0 con 0x7f01f8102790 2024-09-11T21:56:20.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.348+0000 7f01fe38d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f01f81120f0 con 0x7f01f8101de0 2024-09-11T21:56:20.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.348+0000 7f01fc92a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f01f8106f10 0x7f01f8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:20.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.348+0000 7f01fc92a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f01f8106f10 0x7f01f8074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55766/0 (socket says 172.21.15.1:55766) 2024-09-11T21:56:20.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.348+0000 7f01fc92a640 1 -- 172.21.15.1:0/250218777 learned_addr learned my addr 172.21.15.1:0/250218777 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:56:20.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.348+0000 7f01f7fff640 1 --2- 172.21.15.1:0/250218777 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01f8101de0 0x7f01f80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:20.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.348+0000 7f01effff640 1 --2- 172.21.15.1:0/250218777 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01f8102790 0x7f01f8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:20.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.348+0000 7f01fc92a640 1 -- 172.21.15.1:0/250218777 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01f8101de0 msgr2=0x7f01f80744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:20.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.348+0000 7f01fc92a640 1 --2- 172.21.15.1:0/250218777 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01f8101de0 0x7f01f80744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:20.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.348+0000 7f01fc92a640 1 -- 172.21.15.1:0/250218777 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01f8102790 msgr2=0x7f01f8074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:20.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.348+0000 7f01fc92a640 1 --2- 172.21.15.1:0/250218777 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01f8102790 0x7f01f8074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:20.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.348+0000 7f01fc92a640 1 -- 172.21.15.1:0/250218777 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01f81a6850 con 0x7f01f8106f10 2024-09-11T21:56:20.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.348+0000 7f01effff640 1 --2- 172.21.15.1:0/250218777 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01f8102790 0x7f01f8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:56:20.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.349+0000 7f01fc92a640 1 --2- 172.21.15.1:0/250218777 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f01f8106f10 0x7f01f8074f70 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f01f000eaa0 tx=0x7f01f000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:20.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.349+0000 7f01f5ffb640 1 -- 172.21.15.1:0/250218777 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f0004d20 con 0x7f01f8106f10 2024-09-11T21:56:20.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.349+0000 7f01fe38d640 1 -- 172.21.15.1:0/250218777 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01f81a6b40 con 0x7f01f8106f10 2024-09-11T21:56:20.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.349+0000 7f01f5ffb640 1 -- 172.21.15.1:0/250218777 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01f0005020 con 0x7f01f8106f10 2024-09-11T21:56:20.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.349+0000 7f01f5ffb640 1 -- 172.21.15.1:0/250218777 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f00116d0 con 0x7f01f8106f10 2024-09-11T21:56:20.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.349+0000 7f01fe38d640 1 -- 172.21.15.1:0/250218777 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01f81a7080 con 0x7f01f8106f10 2024-09-11T21:56:20.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.350+0000 7f01fe38d640 1 -- 172.21.15.1:0/250218777 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01f81021c0 con 0x7f01f8106f10 2024-09-11T21:56:20.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.351+0000 7f01f5ffb640 1 -- 172.21.15.1:0/250218777 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f01f00051d0 con 0x7f01f8106f10 2024-09-11T21:56:20.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.352+0000 7f01f5ffb640 1 --2- 172.21.15.1:0/250218777 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f01e40a2c10 0x7f01e40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:20.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.352+0000 7f01f7fff640 1 --2- 172.21.15.1:0/250218777 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f01e40a2c10 0x7f01e40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:20.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.352+0000 7f01f5ffb640 1 -- 172.21.15.1:0/250218777 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f01f00cf630 con 0x7f01f8106f10 2024-09-11T21:56:20.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.353+0000 7f01f7fff640 1 --2- 172.21.15.1:0/250218777 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f01e40a2c10 0x7f01e40a50d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f01e800abe0 tx=0x7f01e800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:20.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.355+0000 7f01f5ffb640 1 -- 172.21.15.1:0/250218777 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f01f0016030 con 0x7f01f8106f10 2024-09-11T21:56:20.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:20 smithi001 ceph-mon[30935]: pgmap v311: 97 pgs: 97 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T21:56:20.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.253+0000 7f5a34387640 1 -- 172.21.15.1:0/342356087 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a2c103660 msgr2=0x7f5a2c10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:20.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.253+0000 7f5a34387640 1 --2- 172.21.15.1:0/342356087 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a2c103660 0x7f5a2c10bb50 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c009900 tx=0x7f5a1c02a480 comp rx=0 tx=0).stop 2024-09-11T21:56:20.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.254+0000 7f5a34387640 1 -- 172.21.15.1:0/342356087 shutdown_connections 2024-09-11T21:56:20.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.254+0000 7f5a34387640 1 --2- 172.21.15.1:0/342356087 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a2c10c170 0x7f5a2c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:20.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.254+0000 7f5a34387640 1 --2- 172.21.15.1:0/342356087 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a2c103660 0x7f5a2c10bb50 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:20.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.254+0000 7f5a34387640 1 --2- 172.21.15.1:0/342356087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a2c102d40 0x7f5a2c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:20.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.254+0000 7f5a34387640 1 -- 172.21.15.1:0/342356087 >> 172.21.15.1:0/342356087 conn(0x7f5a2c0fc7a0 msgr2=0x7f5a2c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:20.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.254+0000 7f5a34387640 1 -- 172.21.15.1:0/342356087 shutdown_connections 2024-09-11T21:56:20.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.254+0000 7f5a34387640 1 -- 172.21.15.1:0/342356087 wait complete. 2024-09-11T21:56:20.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.254+0000 7f5a34387640 1 Processor -- start 2024-09-11T21:56:20.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.254+0000 7f5a34387640 1 -- start start 2024-09-11T21:56:20.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.254+0000 7f5a34387640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a2c102d40 0x7f5a2c1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:20.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.254+0000 7f5a34387640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a2c103660 0x7f5a2c100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:20.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.254+0000 7f5a34387640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a2c10c170 0x7f5a2c101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:20.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.254+0000 7f5a34387640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5a2c111f70 con 0x7f5a2c103660 2024-09-11T21:56:20.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.254+0000 7f5a34387640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5a2c111df0 con 0x7f5a2c102d40 2024-09-11T21:56:20.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.254+0000 7f5a34387640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5a2c1120f0 con 0x7f5a2c10c170 2024-09-11T21:56:20.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.255+0000 7f5a31922640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a2c103660 0x7f5a2c100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:20.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.255+0000 7f5a31922640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a2c103660 0x7f5a2c100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55756/0 (socket says 172.21.15.1:55756) 2024-09-11T21:56:20.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.255+0000 7f5a31922640 1 -- 172.21.15.1:0/2468453057 learned_addr learned my addr 172.21.15.1:0/2468453057 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:56:20.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.255+0000 7f5a32123640 1 --2- 172.21.15.1:0/2468453057 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a2c102d40 0x7f5a2c1009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:20.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.255+0000 7f5a31922640 1 -- 172.21.15.1:0/2468453057 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a2c10c170 msgr2=0x7f5a2c101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:20.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.255+0000 7f5a31922640 1 --2- 172.21.15.1:0/2468453057 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a2c10c170 0x7f5a2c101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:20.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.255+0000 7f5a31922640 1 -- 172.21.15.1:0/2468453057 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a2c102d40 msgr2=0x7f5a2c1009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:20.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.255+0000 7f5a31922640 1 --2- 172.21.15.1:0/2468453057 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a2c102d40 0x7f5a2c1009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:20.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.255+0000 7f5a31922640 1 -- 172.21.15.1:0/2468453057 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a2c1a6930 con 0x7f5a2c103660 2024-09-11T21:56:20.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.255+0000 7f5a31922640 1 --2- 172.21.15.1:0/2468453057 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a2c103660 0x7f5a2c100ee0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c006010 tx=0x7f5a1c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:20.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.255+0000 7f5a237fe640 1 -- 172.21.15.1:0/2468453057 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c03f030 con 0x7f5a2c103660 2024-09-11T21:56:20.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.255+0000 7f5a237fe640 1 -- 172.21.15.1:0/2468453057 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a1c003ce0 con 0x7f5a2c103660 2024-09-11T21:56:20.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.255+0000 7f5a34387640 1 -- 172.21.15.1:0/2468453057 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a2c1a6bc0 con 0x7f5a2c103660 2024-09-11T21:56:20.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.255+0000 7f5a237fe640 1 -- 172.21.15.1:0/2468453057 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c035070 con 0x7f5a2c103660 2024-09-11T21:56:20.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.255+0000 7f5a34387640 1 -- 172.21.15.1:0/2468453057 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a2c1a70a0 con 0x7f5a2c103660 2024-09-11T21:56:20.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.256+0000 7f5a34387640 1 -- 172.21.15.1:0/2468453057 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a2c103120 con 0x7f5a2c103660 2024-09-11T21:56:20.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.257+0000 7f5a237fe640 1 -- 172.21.15.1:0/2468453057 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f5a1c042030 con 0x7f5a2c103660 2024-09-11T21:56:20.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.258+0000 7f5a237fe640 1 --2- 172.21.15.1:0/2468453057 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f5a180a2c10 0x7f5a180a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:20.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.258+0000 7f5a32123640 1 --2- 172.21.15.1:0/2468453057 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f5a180a2c10 0x7f5a180a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:20.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.258+0000 7f5a237fe640 1 -- 172.21.15.1:0/2468453057 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f5a1c0ef160 con 0x7f5a2c103660 2024-09-11T21:56:20.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.259+0000 7f5a32123640 1 --2- 172.21.15.1:0/2468453057 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f5a180a2c10 0x7f5a180a50d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5a280058c0 tx=0x7f5a2800a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:20.521 INFO:teuthology.orchestra.run.smithi001.stdout:197568495666 2024-09-11T21:56:20.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.262+0000 7f5a237fe640 1 -- 172.21.15.1:0/2468453057 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5a1c0bc1d0 con 0x7f5a2c103660 2024-09-11T21:56:20.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.444+0000 7f5a34387640 1 -- 172.21.15.1:0/2468453057 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f5a2c106c50 con 0x7f5a2c103660 2024-09-11T21:56:20.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.445+0000 7f5a237fe640 1 -- 172.21.15.1:0/2468453057 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f5a1c0bbad0 con 0x7f5a2c103660 2024-09-11T21:56:20.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.447+0000 7f5a34387640 1 -- 172.21.15.1:0/2468453057 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f5a180a2c10 msgr2=0x7f5a180a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:20.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.447+0000 7f5a34387640 1 --2- 172.21.15.1:0/2468453057 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f5a180a2c10 0x7f5a180a50d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5a280058c0 tx=0x7f5a2800a4c0 comp rx=0 tx=0).stop 2024-09-11T21:56:20.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.447+0000 7f5a34387640 1 -- 172.21.15.1:0/2468453057 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a2c103660 msgr2=0x7f5a2c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:20.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.447+0000 7f5a34387640 1 --2- 172.21.15.1:0/2468453057 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a2c103660 0x7f5a2c100ee0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c006010 tx=0x7f5a1c0042f0 comp rx=0 tx=0).stop 2024-09-11T21:56:20.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.447+0000 7f5a34387640 1 -- 172.21.15.1:0/2468453057 shutdown_connections 2024-09-11T21:56:20.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.447+0000 7f5a34387640 1 --2- 172.21.15.1:0/2468453057 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f5a180a2c10 0x7f5a180a50d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:20.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.447+0000 7f5a34387640 1 --2- 172.21.15.1:0/2468453057 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a2c10c170 0x7f5a2c101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:20.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.447+0000 7f5a34387640 1 --2- 172.21.15.1:0/2468453057 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a2c103660 0x7f5a2c100ee0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:20.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.447+0000 7f5a34387640 1 --2- 172.21.15.1:0/2468453057 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a2c102d40 0x7f5a2c1009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:20.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.447+0000 7f5a34387640 1 -- 172.21.15.1:0/2468453057 >> 172.21.15.1:0/2468453057 conn(0x7f5a2c0fc7a0 msgr2=0x7f5a2c0fd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:20.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.447+0000 7f5a34387640 1 -- 172.21.15.1:0/2468453057 shutdown_connections 2024-09-11T21:56:20.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.447+0000 7f5a34387640 1 -- 172.21.15.1:0/2468453057 wait complete. 2024-09-11T21:56:20.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.545+0000 7f01fe38d640 1 -- 172.21.15.1:0/250218777 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f01f8075a20 con 0x7f01f8106f10 2024-09-11T21:56:20.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.545+0000 7f01f5ffb640 1 -- 172.21.15.1:0/250218777 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f01f8075a20 con 0x7f01f8106f10 2024-09-11T21:56:20.546 INFO:teuthology.orchestra.run.smithi001.stdout:249108103209 2024-09-11T21:56:20.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:20 smithi053 ceph-mon[34908]: pgmap v311: 97 pgs: 97 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T21:56:20.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.547+0000 7f01fe38d640 1 -- 172.21.15.1:0/250218777 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f01e40a2c10 msgr2=0x7f01e40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:20.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.547+0000 7f01fe38d640 1 --2- 172.21.15.1:0/250218777 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f01e40a2c10 0x7f01e40a50d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f01e800abe0 tx=0x7f01e800a2d0 comp rx=0 tx=0).stop 2024-09-11T21:56:20.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.547+0000 7f01fe38d640 1 -- 172.21.15.1:0/250218777 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f01f8106f10 msgr2=0x7f01f8074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:20.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.547+0000 7f01fe38d640 1 --2- 172.21.15.1:0/250218777 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f01f8106f10 0x7f01f8074f70 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f01f000eaa0 tx=0x7f01f000eea0 comp rx=0 tx=0).stop 2024-09-11T21:56:20.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.547+0000 7f01fe38d640 1 -- 172.21.15.1:0/250218777 shutdown_connections 2024-09-11T21:56:20.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.547+0000 7f01fe38d640 1 --2- 172.21.15.1:0/250218777 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f01e40a2c10 0x7f01e40a50d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:20.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.547+0000 7f01fe38d640 1 --2- 172.21.15.1:0/250218777 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f01f8106f10 0x7f01f8074f70 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:20.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.547+0000 7f01fe38d640 1 --2- 172.21.15.1:0/250218777 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01f8102790 0x7f01f8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:20.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.547+0000 7f01fe38d640 1 --2- 172.21.15.1:0/250218777 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01f8101de0 0x7f01f80744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:20.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.547+0000 7f01fe38d640 1 -- 172.21.15.1:0/250218777 >> 172.21.15.1:0/250218777 conn(0x7f01f80fd8c0 msgr2=0x7f01f80feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:20.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.547+0000 7f01fe38d640 1 -- 172.21.15.1:0/250218777 shutdown_connections 2024-09-11T21:56:20.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:20.547+0000 7f01fe38d640 1 -- 172.21.15.1:0/250218777 wait complete. 2024-09-11T21:56:20.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:20 smithi161 ceph-mon[35615]: pgmap v311: 97 pgs: 97 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T21:56:21.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:21 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/2468453057' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-11T21:56:21.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:21 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/250218777' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-11T21:56:21.486 INFO:tasks.cephadm.ceph_manager.ceph:need seq 249108103206 got 249108103209 for osd.8 2024-09-11T21:56:21.486 DEBUG:teuthology.parallel:result is None 2024-09-11T21:56:21.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:21 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/2468453057' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-11T21:56:21.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:21 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/250218777' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-11T21:56:21.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:21 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/2468453057' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-11T21:56:21.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:21 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/250218777' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-11T21:56:21.957 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182463 got 107374182466 for osd.3 2024-09-11T21:56:21.957 DEBUG:teuthology.parallel:result is None 2024-09-11T21:56:22.359 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:22 smithi001 ceph-mon[30935]: pgmap v312: 97 pgs: 97 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-11T21:56:22.397 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495663 got 197568495666 for osd.6 2024-09-11T21:56:22.397 DEBUG:teuthology.parallel:result is None 2024-09-11T21:56:22.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:22 smithi053 ceph-mon[34908]: pgmap v312: 97 pgs: 97 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-11T21:56:22.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:22 smithi161 ceph-mon[35615]: pgmap v312: 97 pgs: 97 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-11T21:56:22.813 INFO:tasks.cephadm.ceph_manager.ceph:need seq 335007449110 got 335007449112 for osd.11 2024-09-11T21:56:22.813 DEBUG:teuthology.parallel:result is None 2024-09-11T21:56:22.836 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:56:23.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:23.335+0000 7fd5b1930640 1 -- 172.21.15.1:0/1662057679 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5ac105ad0 msgr2=0x7fd5ac105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:23.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:23.335+0000 7fd5b1930640 1 --2- 172.21.15.1:0/1662057679 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5ac105ad0 0x7fd5ac105eb0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd59c009900 tx=0x7fd59c02a410 comp rx=0 tx=0).stop 2024-09-11T21:56:23.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:23.335+0000 7fd5b1930640 1 -- 172.21.15.1:0/1662057679 shutdown_connections 2024-09-11T21:56:23.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:23.335+0000 7fd5b1930640 1 --2- 172.21.15.1:0/1662057679 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5ac101ba0 0x7fd5ac10d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:23.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:23.335+0000 7fd5b1930640 1 --2- 172.21.15.1:0/1662057679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5ac101200 0x7fd5ac101660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:23.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:23.335+0000 7fd5b1930640 1 --2- 172.21.15.1:0/1662057679 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5ac105ad0 0x7fd5ac105eb0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:23.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:23.335+0000 7fd5b1930640 1 -- 172.21.15.1:0/1662057679 >> 172.21.15.1:0/1662057679 conn(0x7fd5ac077210 msgr2=0x7fd5ac0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:23.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:23.336+0000 7fd5b1930640 1 -- 172.21.15.1:0/1662057679 shutdown_connections 2024-09-11T21:56:23.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:23.336+0000 7fd5b1930640 1 -- 172.21.15.1:0/1662057679 wait complete. 2024-09-11T21:56:23.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:23.336+0000 7fd5b1930640 1 Processor -- start 2024-09-11T21:56:23.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:23.336+0000 7fd5b1930640 1 -- start start 2024-09-11T21:56:23.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:23.336+0000 7fd5b1930640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5ac101200 0x7fd5ac1a0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:23.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:23.336+0000 7fd5b1930640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5ac101ba0 0x7fd5ac1a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:23.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:23.337+0000 7fd5b1930640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5ac105ad0 0x7fd5ac19a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:23.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:23.337+0000 7fd5b1930640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd5ac110f70 con 0x7fd5ac101200 2024-09-11T21:56:23.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:23.337+0000 7fd5b1930640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd5ac110df0 con 0x7fd5ac101ba0 2024-09-11T21:56:23.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:23.337+0000 7fd5b1930640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd5ac1110f0 con 0x7fd5ac105ad0 2024-09-11T21:56:23.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:23.337+0000 7fd5aaffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5ac101200 0x7fd5ac1a0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:23.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:23.337+0000 7fd5aaffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5ac101200 0x7fd5ac1a0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55782/0 (socket says 172.21.15.1:55782) 2024-09-11T21:56:23.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:23.337+0000 7fd5aaffd640 1 -- 172.21.15.1:0/3202909355 learned_addr learned my addr 172.21.15.1:0/3202909355 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:56:23.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:23.337+0000 7fd5aa7fc640 1 --2- 172.21.15.1:0/3202909355 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5ac101ba0 0x7fd5ac1a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:23.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:23.337+0000 7fd5ab7fe640 1 --2- 172.21.15.1:0/3202909355 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5ac105ad0 0x7fd5ac19a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:23.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:23.337+0000 7fd5aaffd640 1 -- 172.21.15.1:0/3202909355 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5ac105ad0 msgr2=0x7fd5ac19a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:23.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:23.337+0000 7fd5aaffd640 1 --2- 172.21.15.1:0/3202909355 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5ac105ad0 0x7fd5ac19a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:23.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:23.337+0000 7fd5aaffd640 1 -- 172.21.15.1:0/3202909355 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5ac101ba0 msgr2=0x7fd5ac1a09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:23.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:23.337+0000 7fd5aaffd640 1 --2- 172.21.15.1:0/3202909355 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5ac101ba0 0x7fd5ac1a09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:23.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:23.337+0000 7fd5aaffd640 1 -- 172.21.15.1:0/3202909355 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5ac19ad50 con 0x7fd5ac101200 2024-09-11T21:56:23.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:23.337+0000 7fd5aaffd640 1 --2- 172.21.15.1:0/3202909355 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5ac101200 0x7fd5ac1a0460 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd59c006010 tx=0x7fd59c003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:23.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:23.337+0000 7fd5b092e640 1 -- 172.21.15.1:0/3202909355 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59c03f070 con 0x7fd5ac101200 2024-09-11T21:56:23.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:23.337+0000 7fd5b092e640 1 -- 172.21.15.1:0/3202909355 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd59c031e30 con 0x7fd5ac101200 2024-09-11T21:56:23.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:23.337+0000 7fd5b1930640 1 -- 172.21.15.1:0/3202909355 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5ac19afe0 con 0x7fd5ac101200 2024-09-11T21:56:23.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:23.337+0000 7fd5b092e640 1 -- 172.21.15.1:0/3202909355 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59c035030 con 0x7fd5ac101200 2024-09-11T21:56:23.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:23.338+0000 7fd5b1930640 1 -- 172.21.15.1:0/3202909355 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5ac1a7330 con 0x7fd5ac101200 2024-09-11T21:56:23.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:23.338+0000 7fd5b1930640 1 -- 172.21.15.1:0/3202909355 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5ac1016e0 con 0x7fd5ac101200 2024-09-11T21:56:23.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:23.340+0000 7fd5b092e640 1 -- 172.21.15.1:0/3202909355 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fd59c042070 con 0x7fd5ac101200 2024-09-11T21:56:23.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:23.340+0000 7fd5b092e640 1 --2- 172.21.15.1:0/3202909355 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fd5940a2c90 0x7fd5940a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:23.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:23.340+0000 7fd5aa7fc640 1 --2- 172.21.15.1:0/3202909355 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fd5940a2c90 0x7fd5940a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:23.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:23.340+0000 7fd5b092e640 1 -- 172.21.15.1:0/3202909355 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7fd59c0ef9a0 con 0x7fd5ac101200 2024-09-11T21:56:23.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:23.341+0000 7fd5aa7fc640 1 --2- 172.21.15.1:0/3202909355 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fd5940a2c90 0x7fd5940a5150 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac10ddb0 tx=0x7fd5a0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:23.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:23.344+0000 7fd5b092e640 1 -- 172.21.15.1:0/3202909355 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd59c0bc950 con 0x7fd5ac101200 2024-09-11T21:56:23.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:23.521+0000 7fd5b1930640 1 -- 172.21.15.1:0/3202909355 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fd5ac1a7590 con 0x7fd5ac101200 2024-09-11T21:56:23.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:23.522+0000 7fd5b092e640 1 -- 172.21.15.1:0/3202909355 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd59c0bc250 con 0x7fd5ac101200 2024-09-11T21:56:23.522 INFO:teuthology.orchestra.run.smithi001.stdout:141733920829 2024-09-11T21:56:23.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:23.524+0000 7fd5b1930640 1 -- 172.21.15.1:0/3202909355 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fd5940a2c90 msgr2=0x7fd5940a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:23.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:23.524+0000 7fd5b1930640 1 --2- 172.21.15.1:0/3202909355 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fd5940a2c90 0x7fd5940a5150 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac10ddb0 tx=0x7fd5a0008040 comp rx=0 tx=0).stop 2024-09-11T21:56:23.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:23.524+0000 7fd5b1930640 1 -- 172.21.15.1:0/3202909355 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5ac101200 msgr2=0x7fd5ac1a0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:23.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:23.524+0000 7fd5b1930640 1 --2- 172.21.15.1:0/3202909355 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5ac101200 0x7fd5ac1a0460 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd59c006010 tx=0x7fd59c003c80 comp rx=0 tx=0).stop 2024-09-11T21:56:23.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:23.524+0000 7fd5b1930640 1 -- 172.21.15.1:0/3202909355 shutdown_connections 2024-09-11T21:56:23.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:23.524+0000 7fd5b1930640 1 --2- 172.21.15.1:0/3202909355 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fd5940a2c90 0x7fd5940a5150 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:23.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:23.524+0000 7fd5b1930640 1 --2- 172.21.15.1:0/3202909355 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5ac105ad0 0x7fd5ac19a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:23.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:23.524+0000 7fd5b1930640 1 --2- 172.21.15.1:0/3202909355 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5ac101ba0 0x7fd5ac1a09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:23.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:23.524+0000 7fd5b1930640 1 --2- 172.21.15.1:0/3202909355 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5ac101200 0x7fd5ac1a0460 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:23.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:23.524+0000 7fd5b1930640 1 -- 172.21.15.1:0/3202909355 >> 172.21.15.1:0/3202909355 conn(0x7fd5ac077210 msgr2=0x7fd5ac104d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:23.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:23.524+0000 7fd5b1930640 1 -- 172.21.15.1:0/3202909355 shutdown_connections 2024-09-11T21:56:23.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:23.524+0000 7fd5b1930640 1 -- 172.21.15.1:0/3202909355 wait complete. 2024-09-11T21:56:24.318 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:24 smithi001 ceph-mon[30935]: pgmap v313: 97 pgs: 97 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T21:56:24.318 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:24 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/3202909355' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-11T21:56:24.347 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920826 got 141733920829 for osd.4 2024-09-11T21:56:24.347 DEBUG:teuthology.parallel:result is None 2024-09-11T21:56:24.347 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-11T21:56:24.348 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T21:56:24.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:24 smithi053 ceph-mon[34908]: pgmap v313: 97 pgs: 97 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T21:56:24.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:24 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/3202909355' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-11T21:56:24.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:24 smithi161 ceph-mon[35615]: pgmap v313: 97 pgs: 97 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T21:56:24.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:24 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/3202909355' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-11T21:56:24.711 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:56:25.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:25.184+0000 7f1fc0a22640 1 -- 172.21.15.1:0/3397098287 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1fbc102d40 msgr2=0x7f1fbc103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:25.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:25.184+0000 7f1fc0a22640 1 --2- 172.21.15.1:0/3397098287 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1fbc102d40 0x7f1fbc103120 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1fac009900 tx=0x7f1fac02a480 comp rx=0 tx=0).stop 2024-09-11T21:56:25.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:25.184+0000 7f1fc0a22640 1 -- 172.21.15.1:0/3397098287 shutdown_connections 2024-09-11T21:56:25.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:25.184+0000 7f1fc0a22640 1 --2- 172.21.15.1:0/3397098287 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fbc10c170 0x7f1fbc10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:25.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:25.184+0000 7f1fc0a22640 1 --2- 172.21.15.1:0/3397098287 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fbc103660 0x7f1fbc10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:25.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:25.184+0000 7f1fc0a22640 1 --2- 172.21.15.1:0/3397098287 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1fbc102d40 0x7f1fbc103120 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:25.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:25.184+0000 7f1fc0a22640 1 -- 172.21.15.1:0/3397098287 >> 172.21.15.1:0/3397098287 conn(0x7f1fbc0fc7a0 msgr2=0x7f1fbc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:25.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:25.185+0000 7f1fc0a22640 1 -- 172.21.15.1:0/3397098287 shutdown_connections 2024-09-11T21:56:25.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:25.185+0000 7f1fc0a22640 1 -- 172.21.15.1:0/3397098287 wait complete. 2024-09-11T21:56:25.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:25.185+0000 7f1fc0a22640 1 Processor -- start 2024-09-11T21:56:25.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:25.185+0000 7f1fc0a22640 1 -- start start 2024-09-11T21:56:25.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:25.185+0000 7f1fc0a22640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fbc102d40 0x7f1fbc19a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:25.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:25.185+0000 7f1fc0a22640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fbc103660 0x7f1fbc19ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:25.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:25.185+0000 7f1fc0a22640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1fbc10c170 0x7f1fbc19eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:25.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:25.185+0000 7f1fc0a22640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1fbc111f70 con 0x7f1fbc10c170 2024-09-11T21:56:25.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:25.185+0000 7f1fc0a22640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1fbc111df0 con 0x7f1fbc102d40 2024-09-11T21:56:25.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:25.186+0000 7f1fc0a22640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1fbc1120f0 con 0x7f1fbc103660 2024-09-11T21:56:25.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:25.186+0000 7f1fbad9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1fbc10c170 0x7f1fbc19eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:25.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:25.186+0000 7f1fbad9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1fbc10c170 0x7f1fbc19eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55802/0 (socket says 172.21.15.1:55802) 2024-09-11T21:56:25.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:25.186+0000 7f1fbad9d640 1 -- 172.21.15.1:0/2835675874 learned_addr learned my addr 172.21.15.1:0/2835675874 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:56:25.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:25.186+0000 7f1fba59c640 1 --2- 172.21.15.1:0/2835675874 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fbc102d40 0x7f1fbc19a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:25.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:25.186+0000 7f1fb9d9b640 1 --2- 172.21.15.1:0/2835675874 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fbc103660 0x7f1fbc19ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:25.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:25.186+0000 7f1fbad9d640 1 -- 172.21.15.1:0/2835675874 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fbc103660 msgr2=0x7f1fbc19ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:25.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:25.186+0000 7f1fbad9d640 1 --2- 172.21.15.1:0/2835675874 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fbc103660 0x7f1fbc19ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:25.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:25.186+0000 7f1fbad9d640 1 -- 172.21.15.1:0/2835675874 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fbc102d40 msgr2=0x7f1fbc19a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:25.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:25.186+0000 7f1fbad9d640 1 --2- 172.21.15.1:0/2835675874 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fbc102d40 0x7f1fbc19a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:25.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:25.186+0000 7f1fbad9d640 1 -- 172.21.15.1:0/2835675874 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fbc19f5c0 con 0x7f1fbc10c170 2024-09-11T21:56:25.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:25.186+0000 7f1fbad9d640 1 --2- 172.21.15.1:0/2835675874 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1fbc10c170 0x7f1fbc19eee0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1fb400eaa0 tx=0x7f1fb400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:25.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:25.186+0000 7f1fab7fe640 1 -- 172.21.15.1:0/2835675874 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb400cdb0 con 0x7f1fbc10c170 2024-09-11T21:56:25.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:25.186+0000 7f1fc0a22640 1 -- 172.21.15.1:0/2835675874 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fbc19f8b0 con 0x7f1fbc10c170 2024-09-11T21:56:25.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:25.186+0000 7f1fab7fe640 1 -- 172.21.15.1:0/2835675874 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fb4005020 con 0x7f1fbc10c170 2024-09-11T21:56:25.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:25.186+0000 7f1fab7fe640 1 -- 172.21.15.1:0/2835675874 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb40116b0 con 0x7f1fbc10c170 2024-09-11T21:56:25.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:25.186+0000 7f1fc0a22640 1 -- 172.21.15.1:0/2835675874 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fbc1a7190 con 0x7f1fbc10c170 2024-09-11T21:56:25.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:25.187+0000 7f1fc0a22640 1 -- 172.21.15.1:0/2835675874 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fbc103120 con 0x7f1fbc10c170 2024-09-11T21:56:25.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:25.189+0000 7f1fab7fe640 1 -- 172.21.15.1:0/2835675874 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f1fb4005560 con 0x7f1fbc10c170 2024-09-11T21:56:25.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:25.190+0000 7f1fab7fe640 1 --2- 172.21.15.1:0/2835675874 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f1fa40a2c10 0x7f1fa40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:25.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:25.190+0000 7f1fba59c640 1 --2- 172.21.15.1:0/2835675874 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f1fa40a2c10 0x7f1fa40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:25.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:25.190+0000 7f1fab7fe640 1 -- 172.21.15.1:0/2835675874 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f1fb40cf6e0 con 0x7f1fbc10c170 2024-09-11T21:56:25.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:25.190+0000 7f1fba59c640 1 --2- 172.21.15.1:0/2835675874 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f1fa40a2c10 0x7f1fa40a50d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1fac009230 tx=0x7f1fac0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:25.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:25.193+0000 7f1fab7fe640 1 -- 172.21.15.1:0/2835675874 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1fb4016030 con 0x7f1fbc10c170 2024-09-11T21:56:25.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:25.390+0000 7f1fc0a22640 1 -- 172.21.15.1:0/2835675874 --> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1fbc102100 con 0x7f1fa40a2c10 2024-09-11T21:56:25.391 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:25 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-09-11T21:56:25.391 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:25 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]: dispatch 2024-09-11T21:56:25.391 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:25 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]: dispatch 2024-09-11T21:56:25.391 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:25 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2024-09-11T21:56:25.391 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:25 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]: dispatch 2024-09-11T21:56:25.391 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:25 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-09-11T21:56:25.391 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:25 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.26", "id": [1, 9]}]: dispatch 2024-09-11T21:56:25.391 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:25 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-09-11T21:56:25.391 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:25 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]: dispatch 2024-09-11T21:56:25.391 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:25 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 7]}]: dispatch 2024-09-11T21:56:25.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:25.399+0000 7f1fab7fe640 1 -- 172.21.15.1:0/2835675874 <== mgr.14150 v2:172.21.15.1:6800/3890350886 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258912 (secure 0 0 0) 0x7f1fbc102100 con 0x7f1fa40a2c10 2024-09-11T21:56:25.402 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:56:25.403 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-11T21:56:25.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:25.402+0000 7f1fc0a22640 1 -- 172.21.15.1:0/2835675874 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f1fa40a2c10 msgr2=0x7f1fa40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:25.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:25.402+0000 7f1fc0a22640 1 --2- 172.21.15.1:0/2835675874 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f1fa40a2c10 0x7f1fa40a50d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1fac009230 tx=0x7f1fac0058d0 comp rx=0 tx=0).stop 2024-09-11T21:56:25.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:25.402+0000 7f1fc0a22640 1 -- 172.21.15.1:0/2835675874 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1fbc10c170 msgr2=0x7f1fbc19eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:25.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:25.402+0000 7f1fc0a22640 1 --2- 172.21.15.1:0/2835675874 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1fbc10c170 0x7f1fbc19eee0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1fb400eaa0 tx=0x7f1fb400ef60 comp rx=0 tx=0).stop 2024-09-11T21:56:25.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:25.402+0000 7f1fc0a22640 1 -- 172.21.15.1:0/2835675874 shutdown_connections 2024-09-11T21:56:25.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:25.402+0000 7f1fc0a22640 1 --2- 172.21.15.1:0/2835675874 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f1fa40a2c10 0x7f1fa40a50d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:25.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:25.402+0000 7f1fc0a22640 1 --2- 172.21.15.1:0/2835675874 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1fbc10c170 0x7f1fbc19eee0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:25.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:25.402+0000 7f1fc0a22640 1 --2- 172.21.15.1:0/2835675874 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fbc103660 0x7f1fbc19ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:25.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:25.402+0000 7f1fc0a22640 1 --2- 172.21.15.1:0/2835675874 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fbc102d40 0x7f1fbc19a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:25.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:25.402+0000 7f1fc0a22640 1 -- 172.21.15.1:0/2835675874 >> 172.21.15.1:0/2835675874 conn(0x7f1fbc0fc7a0 msgr2=0x7f1fbc10d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:25.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:25.402+0000 7f1fc0a22640 1 -- 172.21.15.1:0/2835675874 shutdown_connections 2024-09-11T21:56:25.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:25.402+0000 7f1fc0a22640 1 -- 172.21.15.1:0/2835675874 wait complete. 2024-09-11T21:56:25.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:25 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-09-11T21:56:25.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:25 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]: dispatch 2024-09-11T21:56:25.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:25 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]: dispatch 2024-09-11T21:56:25.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:25 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2024-09-11T21:56:25.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:25 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]: dispatch 2024-09-11T21:56:25.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:25 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-09-11T21:56:25.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:25 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.26", "id": [1, 9]}]: dispatch 2024-09-11T21:56:25.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:25 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-09-11T21:56:25.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:25 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]: dispatch 2024-09-11T21:56:25.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:25 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 7]}]: dispatch 2024-09-11T21:56:25.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:25 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-09-11T21:56:25.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:25 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]: dispatch 2024-09-11T21:56:25.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:25 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]: dispatch 2024-09-11T21:56:25.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:25 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2024-09-11T21:56:25.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:25 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]: dispatch 2024-09-11T21:56:25.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:25 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-09-11T21:56:25.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:25 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.26", "id": [1, 9]}]: dispatch 2024-09-11T21:56:25.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:25 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-09-11T21:56:25.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:25 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]: dispatch 2024-09-11T21:56:25.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:25 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 7]}]: dispatch 2024-09-11T21:56:26.154 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":315,"stamp":"2024-09-11T21:56:25.279249+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657414,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":692,"num_read_kb":474,"num_write":424,"num_write_kb":3073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":346,"ondisk_log_size":346,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":385488,"kb_used_data":11280,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124556336,"statfs":{"total":1151940427776,"available":1151545688064,"internally_reserved":0,"allocated":11550720,"data_stored":7186794,"data_compressed":107988,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19142,"internal_metadata":382973242},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":105,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.659584"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.103272+0000","last_change":"2024-09-11T21:55:20.095385+0000","last_active":"2024-09-11T21:55:21.103272+0000","last_peered":"2024-09-11T21:55:21.103272+0000","last_clean":"2024-09-11T21:55:21.103272+0000","last_became_active":"2024-09-11T21:55:20.095077+0000","last_became_peered":"2024-09-11T21:55:20.095077+0000","last_unstale":"2024-09-11T21:55:21.103272+0000","last_undegraded":"2024-09-11T21:55:21.103272+0000","last_fullsized":"2024-09-11T21:55:21.103272+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:53:52.231522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.099859+0000","last_change":"2024-09-11T21:55:20.103540+0000","last_active":"2024-09-11T21:55:21.099859+0000","last_peered":"2024-09-11T21:55:21.099859+0000","last_clean":"2024-09-11T21:55:21.099859+0000","last_became_active":"2024-09-11T21:55:20.103233+0000","last_became_peered":"2024-09-11T21:55:20.103233+0000","last_unstale":"2024-09-11T21:55:21.099859+0000","last_undegraded":"2024-09-11T21:55:21.099859+0000","last_fullsized":"2024-09-11T21:55:21.099859+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:51:29.222202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.100567+0000","last_change":"2024-09-11T21:55:20.098538+0000","last_active":"2024-09-11T21:55:21.100567+0000","last_peered":"2024-09-11T21:55:21.100567+0000","last_clean":"2024-09-11T21:55:21.100567+0000","last_became_active":"2024-09-11T21:55:20.098202+0000","last_became_peered":"2024-09-11T21:55:20.098202+0000","last_unstale":"2024-09-11T21:55:21.100567+0000","last_undegraded":"2024-09-11T21:55:21.100567+0000","last_fullsized":"2024-09-11T21:55:21.100567+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:33:46.636952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.097725+0000","last_change":"2024-09-11T21:55:20.100869+0000","last_active":"2024-09-11T21:55:21.097725+0000","last_peered":"2024-09-11T21:55:21.097725+0000","last_clean":"2024-09-11T21:55:21.097725+0000","last_became_active":"2024-09-11T21:55:20.100573+0000","last_became_peered":"2024-09-11T21:55:20.100573+0000","last_unstale":"2024-09-11T21:55:21.097725+0000","last_undegraded":"2024-09-11T21:55:21.097725+0000","last_fullsized":"2024-09-11T21:55:21.097725+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:58:11.154105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.098166+0000","last_change":"2024-09-11T21:55:20.091648+0000","last_active":"2024-09-11T21:55:21.098166+0000","last_peered":"2024-09-11T21:55:21.098166+0000","last_clean":"2024-09-11T21:55:21.098166+0000","last_became_active":"2024-09-11T21:55:20.091222+0000","last_became_peered":"2024-09-11T21:55:20.091222+0000","last_unstale":"2024-09-11T21:55:21.098166+0000","last_undegraded":"2024-09-11T21:55:21.098166+0000","last_fullsized":"2024-09-11T21:55:21.098166+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:55:03.420754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.086150+0000","last_change":"2024-09-11T21:55:20.100446+0000","last_active":"2024-09-11T21:55:21.086150+0000","last_peered":"2024-09-11T21:55:21.086150+0000","last_clean":"2024-09-11T21:55:21.086150+0000","last_became_active":"2024-09-11T21:55:20.100062+0000","last_became_peered":"2024-09-11T21:55:20.100062+0000","last_unstale":"2024-09-11T21:55:21.086150+0000","last_undegraded":"2024-09-11T21:55:21.086150+0000","last_fullsized":"2024-09-11T21:55:21.086150+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:49:37.109258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.097709+0000","last_change":"2024-09-11T21:55:20.101084+0000","last_active":"2024-09-11T21:55:21.097709+0000","last_peered":"2024-09-11T21:55:21.097709+0000","last_clean":"2024-09-11T21:55:21.097709+0000","last_became_active":"2024-09-11T21:55:20.100739+0000","last_became_peered":"2024-09-11T21:55:20.100739+0000","last_unstale":"2024-09-11T21:55:21.097709+0000","last_undegraded":"2024-09-11T21:55:21.097709+0000","last_fullsized":"2024-09-11T21:55:21.097709+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:46:58.969559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.098694+0000","last_change":"2024-09-11T21:55:20.088715+0000","last_active":"2024-09-11T21:55:21.098694+0000","last_peered":"2024-09-11T21:55:21.098694+0000","last_clean":"2024-09-11T21:55:21.098694+0000","last_became_active":"2024-09-11T21:55:20.088280+0000","last_became_peered":"2024-09-11T21:55:20.088280+0000","last_unstale":"2024-09-11T21:55:21.098694+0000","last_undegraded":"2024-09-11T21:55:21.098694+0000","last_fullsized":"2024-09-11T21:55:21.098694+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:06:24.450444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.101362+0000","last_change":"2024-09-11T21:55:20.101266+0000","last_active":"2024-09-11T21:55:21.101362+0000","last_peered":"2024-09-11T21:55:21.101362+0000","last_clean":"2024-09-11T21:55:21.101362+0000","last_became_active":"2024-09-11T21:55:20.100919+0000","last_became_peered":"2024-09-11T21:55:20.100919+0000","last_unstale":"2024-09-11T21:55:21.101362+0000","last_undegraded":"2024-09-11T21:55:21.101362+0000","last_fullsized":"2024-09-11T21:55:21.101362+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:47:15.466082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.086697+0000","last_change":"2024-09-11T21:55:20.098086+0000","last_active":"2024-09-11T21:55:21.086697+0000","last_peered":"2024-09-11T21:55:21.086697+0000","last_clean":"2024-09-11T21:55:21.086697+0000","last_became_active":"2024-09-11T21:55:20.097754+0000","last_became_peered":"2024-09-11T21:55:20.097754+0000","last_unstale":"2024-09-11T21:55:21.086697+0000","last_undegraded":"2024-09-11T21:55:21.086697+0000","last_fullsized":"2024-09-11T21:55:21.086697+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:37:00.098345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.099716+0000","last_change":"2024-09-11T21:55:20.092671+0000","last_active":"2024-09-11T21:55:21.099716+0000","last_peered":"2024-09-11T21:55:21.099716+0000","last_clean":"2024-09-11T21:55:21.099716+0000","last_became_active":"2024-09-11T21:55:20.092112+0000","last_became_peered":"2024-09-11T21:55:20.092112+0000","last_unstale":"2024-09-11T21:55:21.099716+0000","last_undegraded":"2024-09-11T21:55:21.099716+0000","last_fullsized":"2024-09-11T21:55:21.099716+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:06:28.538838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.097879+0000","last_change":"2024-09-11T21:55:20.099863+0000","last_active":"2024-09-11T21:55:21.097879+0000","last_peered":"2024-09-11T21:55:21.097879+0000","last_clean":"2024-09-11T21:55:21.097879+0000","last_became_active":"2024-09-11T21:55:20.099173+0000","last_became_peered":"2024-09-11T21:55:20.099173+0000","last_unstale":"2024-09-11T21:55:21.097879+0000","last_undegraded":"2024-09-11T21:55:21.097879+0000","last_fullsized":"2024-09-11T21:55:21.097879+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:34:56.669410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.085802+0000","last_change":"2024-09-11T21:55:20.101212+0000","last_active":"2024-09-11T21:55:21.085802+0000","last_peered":"2024-09-11T21:55:21.085802+0000","last_clean":"2024-09-11T21:55:21.085802+0000","last_became_active":"2024-09-11T21:55:20.100876+0000","last_became_peered":"2024-09-11T21:55:20.100876+0000","last_unstale":"2024-09-11T21:55:21.085802+0000","last_undegraded":"2024-09-11T21:55:21.085802+0000","last_fullsized":"2024-09-11T21:55:21.085802+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:14:18.104009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.100238+0000","last_change":"2024-09-11T21:55:20.095394+0000","last_active":"2024-09-11T21:55:21.100238+0000","last_peered":"2024-09-11T21:55:21.100238+0000","last_clean":"2024-09-11T21:55:21.100238+0000","last_became_active":"2024-09-11T21:55:20.095101+0000","last_became_peered":"2024-09-11T21:55:20.095101+0000","last_unstale":"2024-09-11T21:55:21.100238+0000","last_undegraded":"2024-09-11T21:55:21.100238+0000","last_fullsized":"2024-09-11T21:55:21.100238+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:53:32.529359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.100035+0000","last_change":"2024-09-11T21:55:20.099668+0000","last_active":"2024-09-11T21:55:21.100035+0000","last_peered":"2024-09-11T21:55:21.100035+0000","last_clean":"2024-09-11T21:55:21.100035+0000","last_became_active":"2024-09-11T21:55:20.099392+0000","last_became_peered":"2024-09-11T21:55:20.099392+0000","last_unstale":"2024-09-11T21:55:21.100035+0000","last_undegraded":"2024-09-11T21:55:21.100035+0000","last_fullsized":"2024-09-11T21:55:21.100035+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:50:00.479508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.099808+0000","last_change":"2024-09-11T21:55:20.093019+0000","last_active":"2024-09-11T21:55:21.099808+0000","last_peered":"2024-09-11T21:55:21.099808+0000","last_clean":"2024-09-11T21:55:21.099808+0000","last_became_active":"2024-09-11T21:55:20.092691+0000","last_became_peered":"2024-09-11T21:55:20.092691+0000","last_unstale":"2024-09-11T21:55:21.099808+0000","last_undegraded":"2024-09-11T21:55:21.099808+0000","last_fullsized":"2024-09-11T21:55:21.099808+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:04:36.684992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.103502+0000","last_change":"2024-09-11T21:55:20.097333+0000","last_active":"2024-09-11T21:55:21.103502+0000","last_peered":"2024-09-11T21:55:21.103502+0000","last_clean":"2024-09-11T21:55:21.103502+0000","last_became_active":"2024-09-11T21:55:20.096752+0000","last_became_peered":"2024-09-11T21:55:20.096752+0000","last_unstale":"2024-09-11T21:55:21.103502+0000","last_undegraded":"2024-09-11T21:55:21.103502+0000","last_fullsized":"2024-09-11T21:55:21.103502+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:27:20.722174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.097172+0000","last_change":"2024-09-11T21:55:20.093972+0000","last_active":"2024-09-11T21:55:21.097172+0000","last_peered":"2024-09-11T21:55:21.097172+0000","last_clean":"2024-09-11T21:55:21.097172+0000","last_became_active":"2024-09-11T21:55:20.093633+0000","last_became_peered":"2024-09-11T21:55:20.093633+0000","last_unstale":"2024-09-11T21:55:21.097172+0000","last_undegraded":"2024-09-11T21:55:21.097172+0000","last_fullsized":"2024-09-11T21:55:21.097172+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:25:18.973332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.097726+0000","last_change":"2024-09-11T21:55:20.094239+0000","last_active":"2024-09-11T21:55:21.097726+0000","last_peered":"2024-09-11T21:55:21.097726+0000","last_clean":"2024-09-11T21:55:21.097726+0000","last_became_active":"2024-09-11T21:55:20.093924+0000","last_became_peered":"2024-09-11T21:55:20.093924+0000","last_unstale":"2024-09-11T21:55:21.097726+0000","last_undegraded":"2024-09-11T21:55:21.097726+0000","last_fullsized":"2024-09-11T21:55:21.097726+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:39:53.012780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.102943+0000","last_change":"2024-09-11T21:55:20.099335+0000","last_active":"2024-09-11T21:55:21.102943+0000","last_peered":"2024-09-11T21:55:21.102943+0000","last_clean":"2024-09-11T21:55:21.102943+0000","last_became_active":"2024-09-11T21:55:20.098929+0000","last_became_peered":"2024-09-11T21:55:20.098929+0000","last_unstale":"2024-09-11T21:55:21.102943+0000","last_undegraded":"2024-09-11T21:55:21.102943+0000","last_fullsized":"2024-09-11T21:55:21.102943+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:38:43.117829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.085822+0000","last_change":"2024-09-11T21:55:20.094068+0000","last_active":"2024-09-11T21:55:21.085822+0000","last_peered":"2024-09-11T21:55:21.085822+0000","last_clean":"2024-09-11T21:55:21.085822+0000","last_became_active":"2024-09-11T21:55:20.093637+0000","last_became_peered":"2024-09-11T21:55:20.093637+0000","last_unstale":"2024-09-11T21:55:21.085822+0000","last_undegraded":"2024-09-11T21:55:21.085822+0000","last_fullsized":"2024-09-11T21:55:21.085822+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:52:59.289715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.100443+0000","last_change":"2024-09-11T21:55:20.090778+0000","last_active":"2024-09-11T21:55:21.100443+0000","last_peered":"2024-09-11T21:55:21.100443+0000","last_clean":"2024-09-11T21:55:21.100443+0000","last_became_active":"2024-09-11T21:55:20.090443+0000","last_became_peered":"2024-09-11T21:55:20.090443+0000","last_unstale":"2024-09-11T21:55:21.100443+0000","last_undegraded":"2024-09-11T21:55:21.100443+0000","last_fullsized":"2024-09-11T21:55:21.100443+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:46:21.488228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.097160+0000","last_change":"2024-09-11T21:55:20.092238+0000","last_active":"2024-09-11T21:55:21.097160+0000","last_peered":"2024-09-11T21:55:21.097160+0000","last_clean":"2024-09-11T21:55:21.097160+0000","last_became_active":"2024-09-11T21:55:20.091961+0000","last_became_peered":"2024-09-11T21:55:20.091961+0000","last_unstale":"2024-09-11T21:55:21.097160+0000","last_undegraded":"2024-09-11T21:55:21.097160+0000","last_fullsized":"2024-09-11T21:55:21.097160+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:54:40.445441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.086361+0000","last_change":"2024-09-11T21:55:20.102944+0000","last_active":"2024-09-11T21:55:21.086361+0000","last_peered":"2024-09-11T21:55:21.086361+0000","last_clean":"2024-09-11T21:55:21.086361+0000","last_became_active":"2024-09-11T21:55:20.102533+0000","last_became_peered":"2024-09-11T21:55:20.102533+0000","last_unstale":"2024-09-11T21:55:21.086361+0000","last_undegraded":"2024-09-11T21:55:21.086361+0000","last_fullsized":"2024-09-11T21:55:21.086361+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:12:15.832622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.098471+0000","last_change":"2024-09-11T21:55:20.092177+0000","last_active":"2024-09-11T21:55:21.098471+0000","last_peered":"2024-09-11T21:55:21.098471+0000","last_clean":"2024-09-11T21:55:21.098471+0000","last_became_active":"2024-09-11T21:55:20.091897+0000","last_became_peered":"2024-09-11T21:55:20.091897+0000","last_unstale":"2024-09-11T21:55:21.098471+0000","last_undegraded":"2024-09-11T21:55:21.098471+0000","last_fullsized":"2024-09-11T21:55:21.098471+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:38:57.371153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.099223+0000","last_change":"2024-09-11T21:55:20.092166+0000","last_active":"2024-09-11T21:55:21.099223+0000","last_peered":"2024-09-11T21:55:21.099223+0000","last_clean":"2024-09-11T21:55:21.099223+0000","last_became_active":"2024-09-11T21:55:20.091854+0000","last_became_peered":"2024-09-11T21:55:20.091854+0000","last_unstale":"2024-09-11T21:55:21.099223+0000","last_undegraded":"2024-09-11T21:55:21.099223+0000","last_fullsized":"2024-09-11T21:55:21.099223+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:53:32.256008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.103425+0000","last_change":"2024-09-11T21:55:20.103976+0000","last_active":"2024-09-11T21:55:21.103425+0000","last_peered":"2024-09-11T21:55:21.103425+0000","last_clean":"2024-09-11T21:55:21.103425+0000","last_became_active":"2024-09-11T21:55:20.103668+0000","last_became_peered":"2024-09-11T21:55:20.103668+0000","last_unstale":"2024-09-11T21:55:21.103425+0000","last_undegraded":"2024-09-11T21:55:21.103425+0000","last_fullsized":"2024-09-11T21:55:21.103425+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:13:42.663641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.085778+0000","last_change":"2024-09-11T21:55:20.093898+0000","last_active":"2024-09-11T21:55:21.085778+0000","last_peered":"2024-09-11T21:55:21.085778+0000","last_clean":"2024-09-11T21:55:21.085778+0000","last_became_active":"2024-09-11T21:55:20.093348+0000","last_became_peered":"2024-09-11T21:55:20.093348+0000","last_unstale":"2024-09-11T21:55:21.085778+0000","last_undegraded":"2024-09-11T21:55:21.085778+0000","last_fullsized":"2024-09-11T21:55:21.085778+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:00:06.185478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.100902+0000","last_change":"2024-09-11T21:55:20.100575+0000","last_active":"2024-09-11T21:55:21.100902+0000","last_peered":"2024-09-11T21:55:21.100902+0000","last_clean":"2024-09-11T21:55:21.100902+0000","last_became_active":"2024-09-11T21:55:20.100255+0000","last_became_peered":"2024-09-11T21:55:20.100255+0000","last_unstale":"2024-09-11T21:55:21.100902+0000","last_undegraded":"2024-09-11T21:55:21.100902+0000","last_fullsized":"2024-09-11T21:55:21.100902+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:48:57.718298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.097296+0000","last_change":"2024-09-11T21:55:20.097548+0000","last_active":"2024-09-11T21:55:21.097296+0000","last_peered":"2024-09-11T21:55:21.097296+0000","last_clean":"2024-09-11T21:55:21.097296+0000","last_became_active":"2024-09-11T21:55:20.097124+0000","last_became_peered":"2024-09-11T21:55:20.097124+0000","last_unstale":"2024-09-11T21:55:21.097296+0000","last_undegraded":"2024-09-11T21:55:21.097296+0000","last_fullsized":"2024-09-11T21:55:21.097296+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:22:23.904457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.097695+0000","last_change":"2024-09-11T21:55:20.093822+0000","last_active":"2024-09-11T21:55:21.097695+0000","last_peered":"2024-09-11T21:55:21.097695+0000","last_clean":"2024-09-11T21:55:21.097695+0000","last_became_active":"2024-09-11T21:55:20.093361+0000","last_became_peered":"2024-09-11T21:55:20.093361+0000","last_unstale":"2024-09-11T21:55:21.097695+0000","last_undegraded":"2024-09-11T21:55:21.097695+0000","last_fullsized":"2024-09-11T21:55:21.097695+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:37:06.362732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.103360+0000","last_change":"2024-09-11T21:55:20.103990+0000","last_active":"2024-09-11T21:55:21.103360+0000","last_peered":"2024-09-11T21:55:21.103360+0000","last_clean":"2024-09-11T21:55:21.103360+0000","last_became_active":"2024-09-11T21:55:20.103375+0000","last_became_peered":"2024-09-11T21:55:20.103375+0000","last_unstale":"2024-09-11T21:55:21.103360+0000","last_undegraded":"2024-09-11T21:55:21.103360+0000","last_fullsized":"2024-09-11T21:55:21.103360+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:02:06.786888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.099679+0000","last_change":"2024-09-11T21:55:20.101535+0000","last_active":"2024-09-11T21:55:21.099679+0000","last_peered":"2024-09-11T21:55:21.099679+0000","last_clean":"2024-09-11T21:55:21.099679+0000","last_became_active":"2024-09-11T21:55:20.101129+0000","last_became_peered":"2024-09-11T21:55:20.101129+0000","last_unstale":"2024-09-11T21:55:21.099679+0000","last_undegraded":"2024-09-11T21:55:21.099679+0000","last_fullsized":"2024-09-11T21:55:21.099679+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:09:15.673670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"84'6","reported_seq":77,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:56:20.229052+0000","last_change":"2024-09-11T21:55:18.026947+0000","last_active":"2024-09-11T21:56:20.229052+0000","last_peered":"2024-09-11T21:56:20.229052+0000","last_clean":"2024-09-11T21:56:20.229052+0000","last_became_active":"2024-09-11T21:55:18.026632+0000","last_became_peered":"2024-09-11T21:55:18.026632+0000","last_unstale":"2024-09-11T21:56:20.229052+0000","last_undegraded":"2024-09-11T21:56:20.229052+0000","last_fullsized":"2024-09-11T21:56:20.229052+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:55:16.618074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.086576+0000","last_change":"2024-09-11T21:55:20.100197+0000","last_active":"2024-09-11T21:55:21.086576+0000","last_peered":"2024-09-11T21:55:21.086576+0000","last_clean":"2024-09-11T21:55:21.086576+0000","last_became_active":"2024-09-11T21:55:20.098494+0000","last_became_peered":"2024-09-11T21:55:20.098494+0000","last_unstale":"2024-09-11T21:55:21.086576+0000","last_undegraded":"2024-09-11T21:55:21.086576+0000","last_fullsized":"2024-09-11T21:55:21.086576+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:14.505314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"84'7","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:30.150870+0000","last_change":"2024-09-11T21:55:18.024359+0000","last_active":"2024-09-11T21:55:30.150870+0000","last_peered":"2024-09-11T21:55:30.150870+0000","last_clean":"2024-09-11T21:55:30.150870+0000","last_became_active":"2024-09-11T21:55:18.023624+0000","last_became_peered":"2024-09-11T21:55:18.023624+0000","last_unstale":"2024-09-11T21:55:30.150870+0000","last_undegraded":"2024-09-11T21:55:30.150870+0000","last_fullsized":"2024-09-11T21:55:30.150870+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:13:38.582232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"84'3","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:29.138838+0000","last_change":"2024-09-11T21:55:18.032541+0000","last_active":"2024-09-11T21:55:29.138838+0000","last_peered":"2024-09-11T21:55:29.138838+0000","last_clean":"2024-09-11T21:55:29.138838+0000","last_became_active":"2024-09-11T21:55:18.032218+0000","last_became_peered":"2024-09-11T21:55:18.032218+0000","last_unstale":"2024-09-11T21:55:29.138838+0000","last_undegraded":"2024-09-11T21:55:29.138838+0000","last_fullsized":"2024-09-11T21:55:29.138838+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:38:20.872069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.098851+0000","last_change":"2024-09-11T21:55:20.090030+0000","last_active":"2024-09-11T21:55:21.098851+0000","last_peered":"2024-09-11T21:55:21.098851+0000","last_clean":"2024-09-11T21:55:21.098851+0000","last_became_active":"2024-09-11T21:55:20.089481+0000","last_became_peered":"2024-09-11T21:55:20.089481+0000","last_unstale":"2024-09-11T21:55:21.098851+0000","last_undegraded":"2024-09-11T21:55:21.098851+0000","last_fullsized":"2024-09-11T21:55:21.098851+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:41:57.038519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"84'3","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:31.157367+0000","last_change":"2024-09-11T21:55:18.026656+0000","last_active":"2024-09-11T21:55:31.157367+0000","last_peered":"2024-09-11T21:55:31.157367+0000","last_clean":"2024-09-11T21:55:31.157367+0000","last_became_active":"2024-09-11T21:55:18.026318+0000","last_became_peered":"2024-09-11T21:55:18.026318+0000","last_unstale":"2024-09-11T21:55:31.157367+0000","last_undegraded":"2024-09-11T21:55:31.157367+0000","last_fullsized":"2024-09-11T21:55:31.157367+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:14:41.438334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.096818+0000","last_change":"2024-09-11T21:55:20.090790+0000","last_active":"2024-09-11T21:55:21.096818+0000","last_peered":"2024-09-11T21:55:21.096818+0000","last_clean":"2024-09-11T21:55:21.096818+0000","last_became_active":"2024-09-11T21:55:20.090468+0000","last_became_peered":"2024-09-11T21:55:20.090468+0000","last_unstale":"2024-09-11T21:55:21.096818+0000","last_undegraded":"2024-09-11T21:55:21.096818+0000","last_fullsized":"2024-09-11T21:55:21.096818+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:19.115482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"84'4","reported_seq":128,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:56:24.233855+0000","last_change":"2024-09-11T21:55:18.026989+0000","last_active":"2024-09-11T21:56:24.233855+0000","last_peered":"2024-09-11T21:56:24.233855+0000","last_clean":"2024-09-11T21:56:24.233855+0000","last_became_active":"2024-09-11T21:55:18.026229+0000","last_became_peered":"2024-09-11T21:55:18.026229+0000","last_unstale":"2024-09-11T21:56:24.233855+0000","last_undegraded":"2024-09-11T21:56:24.233855+0000","last_fullsized":"2024-09-11T21:56:24.233855+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:50:35.745789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":106,"num_read_kb":106,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.101844+0000","last_change":"2024-09-11T21:55:20.095884+0000","last_active":"2024-09-11T21:55:21.101844+0000","last_peered":"2024-09-11T21:55:21.101844+0000","last_clean":"2024-09-11T21:55:21.101844+0000","last_became_active":"2024-09-11T21:55:20.094887+0000","last_became_peered":"2024-09-11T21:55:20.094887+0000","last_unstale":"2024-09-11T21:55:21.101844+0000","last_undegraded":"2024-09-11T21:55:21.101844+0000","last_fullsized":"2024-09-11T21:55:21.101844+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:45:03.579703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"84'4","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:31.158136+0000","last_change":"2024-09-11T21:55:18.030474+0000","last_active":"2024-09-11T21:55:31.158136+0000","last_peered":"2024-09-11T21:55:31.158136+0000","last_clean":"2024-09-11T21:55:31.158136+0000","last_became_active":"2024-09-11T21:55:18.030104+0000","last_became_peered":"2024-09-11T21:55:18.030104+0000","last_unstale":"2024-09-11T21:55:31.158136+0000","last_undegraded":"2024-09-11T21:55:31.158136+0000","last_fullsized":"2024-09-11T21:55:31.158136+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:21:00.484595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.100730+0000","last_change":"2024-09-11T21:55:20.102807+0000","last_active":"2024-09-11T21:55:21.100730+0000","last_peered":"2024-09-11T21:55:21.100730+0000","last_clean":"2024-09-11T21:55:21.100730+0000","last_became_active":"2024-09-11T21:55:20.102581+0000","last_became_peered":"2024-09-11T21:55:20.102581+0000","last_unstale":"2024-09-11T21:55:21.100730+0000","last_undegraded":"2024-09-11T21:55:21.100730+0000","last_fullsized":"2024-09-11T21:55:21.100730+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:29:23.139788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"84'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:30.150754+0000","last_change":"2024-09-11T21:55:18.117685+0000","last_active":"2024-09-11T21:55:30.150754+0000","last_peered":"2024-09-11T21:55:30.150754+0000","last_clean":"2024-09-11T21:55:30.150754+0000","last_became_active":"2024-09-11T21:55:18.117261+0000","last_became_peered":"2024-09-11T21:55:18.117261+0000","last_unstale":"2024-09-11T21:55:30.150754+0000","last_undegraded":"2024-09-11T21:55:30.150754+0000","last_fullsized":"2024-09-11T21:55:30.150754+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:42:13.180725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.100810+0000","last_change":"2024-09-11T21:55:20.086562+0000","last_active":"2024-09-11T21:55:21.100810+0000","last_peered":"2024-09-11T21:55:21.100810+0000","last_clean":"2024-09-11T21:55:21.100810+0000","last_became_active":"2024-09-11T21:55:20.086197+0000","last_became_peered":"2024-09-11T21:55:20.086197+0000","last_unstale":"2024-09-11T21:55:21.100810+0000","last_undegraded":"2024-09-11T21:55:21.100810+0000","last_fullsized":"2024-09-11T21:55:21.100810+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:23:44.705132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'28","reported_seq":160,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:56:24.233919+0000","last_change":"2024-09-11T21:55:18.028320+0000","last_active":"2024-09-11T21:56:24.233919+0000","last_peered":"2024-09-11T21:56:24.233919+0000","last_clean":"2024-09-11T21:56:24.233919+0000","last_became_active":"2024-09-11T21:55:18.027982+0000","last_became_peered":"2024-09-11T21:55:18.027982+0000","last_unstale":"2024-09-11T21:56:24.233919+0000","last_undegraded":"2024-09-11T21:56:24.233919+0000","last_fullsized":"2024-09-11T21:56:24.233919+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:08:36.750665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59211,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":114,"num_read_kb":114,"num_write":30,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.099277+0000","last_change":"2024-09-11T21:55:20.091695+0000","last_active":"2024-09-11T21:55:21.099277+0000","last_peered":"2024-09-11T21:55:21.099277+0000","last_clean":"2024-09-11T21:55:21.099277+0000","last_became_active":"2024-09-11T21:55:20.091128+0000","last_became_peered":"2024-09-11T21:55:20.091128+0000","last_unstale":"2024-09-11T21:55:21.099277+0000","last_undegraded":"2024-09-11T21:55:21.099277+0000","last_fullsized":"2024-09-11T21:55:21.099277+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:16:20.751952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"84'9","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:32.158923+0000","last_change":"2024-09-11T21:55:18.030827+0000","last_active":"2024-09-11T21:55:32.158923+0000","last_peered":"2024-09-11T21:55:32.158923+0000","last_clean":"2024-09-11T21:55:32.158923+0000","last_became_active":"2024-09-11T21:55:18.030354+0000","last_became_peered":"2024-09-11T21:55:18.030354+0000","last_unstale":"2024-09-11T21:55:32.158923+0000","last_undegraded":"2024-09-11T21:55:32.158923+0000","last_fullsized":"2024-09-11T21:55:32.158923+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:23:19.578820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.085795+0000","last_change":"2024-09-11T21:55:20.090160+0000","last_active":"2024-09-11T21:55:21.085795+0000","last_peered":"2024-09-11T21:55:21.085795+0000","last_clean":"2024-09-11T21:55:21.085795+0000","last_became_active":"2024-09-11T21:55:20.089801+0000","last_became_peered":"2024-09-11T21:55:20.089801+0000","last_unstale":"2024-09-11T21:55:21.085795+0000","last_undegraded":"2024-09-11T21:55:21.085795+0000","last_fullsized":"2024-09-11T21:55:21.085795+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:34:16.093123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"84'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:31.157445+0000","last_change":"2024-09-11T21:55:18.026817+0000","last_active":"2024-09-11T21:55:31.157445+0000","last_peered":"2024-09-11T21:55:31.157445+0000","last_clean":"2024-09-11T21:55:31.157445+0000","last_became_active":"2024-09-11T21:55:18.026188+0000","last_became_peered":"2024-09-11T21:55:18.026188+0000","last_unstale":"2024-09-11T21:55:31.157445+0000","last_undegraded":"2024-09-11T21:55:31.157445+0000","last_fullsized":"2024-09-11T21:55:31.157445+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:55:14.774503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.098605+0000","last_change":"2024-09-11T21:55:20.095613+0000","last_active":"2024-09-11T21:55:21.098605+0000","last_peered":"2024-09-11T21:55:21.098605+0000","last_clean":"2024-09-11T21:55:21.098605+0000","last_became_active":"2024-09-11T21:55:20.095302+0000","last_became_peered":"2024-09-11T21:55:20.095302+0000","last_unstale":"2024-09-11T21:55:21.098605+0000","last_undegraded":"2024-09-11T21:55:21.098605+0000","last_fullsized":"2024-09-11T21:55:21.098605+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:37:27.961213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'5","reported_seq":74,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:56:19.227405+0000","last_change":"2024-09-11T21:55:18.032417+0000","last_active":"2024-09-11T21:56:19.227405+0000","last_peered":"2024-09-11T21:56:19.227405+0000","last_clean":"2024-09-11T21:56:19.227405+0000","last_became_active":"2024-09-11T21:55:18.031419+0000","last_became_peered":"2024-09-11T21:55:18.031419+0000","last_unstale":"2024-09-11T21:56:19.227405+0000","last_undegraded":"2024-09-11T21:56:19.227405+0000","last_fullsized":"2024-09-11T21:56:19.227405+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:30:52.285678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":52,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.099572+0000","last_change":"2024-09-11T21:55:20.101965+0000","last_active":"2024-09-11T21:55:21.099572+0000","last_peered":"2024-09-11T21:55:21.099572+0000","last_clean":"2024-09-11T21:55:21.099572+0000","last_became_active":"2024-09-11T21:55:20.101534+0000","last_became_peered":"2024-09-11T21:55:20.101534+0000","last_unstale":"2024-09-11T21:55:21.099572+0000","last_undegraded":"2024-09-11T21:55:21.099572+0000","last_fullsized":"2024-09-11T21:55:21.099572+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:41:30.685121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"84'17","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:31.158074+0000","last_change":"2024-09-11T21:55:18.116059+0000","last_active":"2024-09-11T21:55:31.158074+0000","last_peered":"2024-09-11T21:55:31.158074+0000","last_clean":"2024-09-11T21:55:31.158074+0000","last_became_active":"2024-09-11T21:55:18.115621+0000","last_became_peered":"2024-09-11T21:55:18.115621+0000","last_unstale":"2024-09-11T21:55:31.158074+0000","last_undegraded":"2024-09-11T21:55:31.158074+0000","last_fullsized":"2024-09-11T21:55:31.158074+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:47:17.815695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.086741+0000","last_change":"2024-09-11T21:55:20.100903+0000","last_active":"2024-09-11T21:55:21.086741+0000","last_peered":"2024-09-11T21:55:21.086741+0000","last_clean":"2024-09-11T21:55:21.086741+0000","last_became_active":"2024-09-11T21:55:20.100561+0000","last_became_peered":"2024-09-11T21:55:20.100561+0000","last_unstale":"2024-09-11T21:55:21.086741+0000","last_undegraded":"2024-09-11T21:55:21.086741+0000","last_fullsized":"2024-09-11T21:55:21.086741+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:00:45.472238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'2","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:31.134748+0000","last_change":"2024-09-11T21:55:18.029549+0000","last_active":"2024-09-11T21:55:31.134748+0000","last_peered":"2024-09-11T21:55:31.134748+0000","last_clean":"2024-09-11T21:55:31.134748+0000","last_became_active":"2024-09-11T21:55:18.029178+0000","last_became_peered":"2024-09-11T21:55:18.029178+0000","last_unstale":"2024-09-11T21:55:31.134748+0000","last_undegraded":"2024-09-11T21:55:31.134748+0000","last_fullsized":"2024-09-11T21:55:31.134748+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:37:07.915786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.103401+0000","last_change":"2024-09-11T21:55:20.104202+0000","last_active":"2024-09-11T21:55:21.103401+0000","last_peered":"2024-09-11T21:55:21.103401+0000","last_clean":"2024-09-11T21:55:21.103401+0000","last_became_active":"2024-09-11T21:55:20.103464+0000","last_became_peered":"2024-09-11T21:55:20.103464+0000","last_unstale":"2024-09-11T21:55:21.103401+0000","last_undegraded":"2024-09-11T21:55:21.103401+0000","last_fullsized":"2024-09-11T21:55:21.103401+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:30:16.044143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"84'5","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:31.157256+0000","last_change":"2024-09-11T21:55:18.026183+0000","last_active":"2024-09-11T21:55:31.157256+0000","last_peered":"2024-09-11T21:55:31.157256+0000","last_clean":"2024-09-11T21:55:31.157256+0000","last_became_active":"2024-09-11T21:55:18.025849+0000","last_became_peered":"2024-09-11T21:55:18.025849+0000","last_unstale":"2024-09-11T21:55:31.157256+0000","last_undegraded":"2024-09-11T21:55:31.157256+0000","last_fullsized":"2024-09-11T21:55:31.157256+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:21:12.189158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.098264+0000","last_change":"2024-09-11T21:55:20.090652+0000","last_active":"2024-09-11T21:55:21.098264+0000","last_peered":"2024-09-11T21:55:21.098264+0000","last_clean":"2024-09-11T21:55:21.098264+0000","last_became_active":"2024-09-11T21:55:20.090348+0000","last_became_peered":"2024-09-11T21:55:20.090348+0000","last_unstale":"2024-09-11T21:55:21.098264+0000","last_undegraded":"2024-09-11T21:55:21.098264+0000","last_fullsized":"2024-09-11T21:55:21.098264+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:10:58.733371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'3","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:31.157673+0000","last_change":"2024-09-11T21:55:18.024388+0000","last_active":"2024-09-11T21:55:31.157673+0000","last_peered":"2024-09-11T21:55:31.157673+0000","last_clean":"2024-09-11T21:55:31.157673+0000","last_became_active":"2024-09-11T21:55:18.024100+0000","last_became_peered":"2024-09-11T21:55:18.024100+0000","last_unstale":"2024-09-11T21:55:31.157673+0000","last_undegraded":"2024-09-11T21:55:31.157673+0000","last_fullsized":"2024-09-11T21:55:31.157673+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:21:11.152282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"84'1","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.186238+0000","last_change":"2024-09-11T21:55:20.101007+0000","last_active":"2024-09-11T21:55:21.186238+0000","last_peered":"2024-09-11T21:55:21.186238+0000","last_clean":"2024-09-11T21:55:21.186238+0000","last_became_active":"2024-09-11T21:55:20.100642+0000","last_became_peered":"2024-09-11T21:55:20.100642+0000","last_unstale":"2024-09-11T21:55:21.186238+0000","last_undegraded":"2024-09-11T21:55:21.186238+0000","last_fullsized":"2024-09-11T21:55:21.186238+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:16:05.642184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"84'6","reported_seq":80,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:56:23.244362+0000","last_change":"2024-09-11T21:55:18.116216+0000","last_active":"2024-09-11T21:56:23.244362+0000","last_peered":"2024-09-11T21:56:23.244362+0000","last_clean":"2024-09-11T21:56:23.244362+0000","last_became_active":"2024-09-11T21:55:18.115871+0000","last_became_peered":"2024-09-11T21:55:18.115871+0000","last_unstale":"2024-09-11T21:56:23.244362+0000","last_undegraded":"2024-09-11T21:56:23.244362+0000","last_fullsized":"2024-09-11T21:56:23.244362+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:16:56.027636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":56,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.099871+0000","last_change":"2024-09-11T21:55:20.092421+0000","last_active":"2024-09-11T21:55:21.099871+0000","last_peered":"2024-09-11T21:55:21.099871+0000","last_clean":"2024-09-11T21:55:21.099871+0000","last_became_active":"2024-09-11T21:55:20.091707+0000","last_became_peered":"2024-09-11T21:55:20.091707+0000","last_unstale":"2024-09-11T21:55:21.099871+0000","last_undegraded":"2024-09-11T21:55:21.099871+0000","last_fullsized":"2024-09-11T21:55:21.099871+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:12:38.157687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":346,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.099897+0000","last_change":"2024-09-11T21:54:07.542362+0000","last_active":"2024-09-11T21:55:21.099897+0000","last_peered":"2024-09-11T21:55:21.099897+0000","last_clean":"2024-09-11T21:55:21.099897+0000","last_became_active":"2024-09-11T21:54:07.541871+0000","last_became_peered":"2024-09-11T21:54:07.541871+0000","last_unstale":"2024-09-11T21:55:21.099897+0000","last_undegraded":"2024-09-11T21:55:21.099897+0000","last_fullsized":"2024-09-11T21:55:21.099897+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:50:41.026895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:50:41.026895+0000","last_clean_scrub_stamp":"2024-09-11T21:50:41.026895+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:05:06.154268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"84'6","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:32.158953+0000","last_change":"2024-09-11T21:55:18.030558+0000","last_active":"2024-09-11T21:55:32.158953+0000","last_peered":"2024-09-11T21:55:32.158953+0000","last_clean":"2024-09-11T21:55:32.158953+0000","last_became_active":"2024-09-11T21:55:18.030216+0000","last_became_peered":"2024-09-11T21:55:18.030216+0000","last_unstale":"2024-09-11T21:55:32.158953+0000","last_undegraded":"2024-09-11T21:55:32.158953+0000","last_fullsized":"2024-09-11T21:55:32.158953+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:33:02.589999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.097756+0000","last_change":"2024-09-11T21:55:20.092938+0000","last_active":"2024-09-11T21:55:21.097756+0000","last_peered":"2024-09-11T21:55:21.097756+0000","last_clean":"2024-09-11T21:55:21.097756+0000","last_became_active":"2024-09-11T21:55:20.092615+0000","last_became_peered":"2024-09-11T21:55:20.092615+0000","last_unstale":"2024-09-11T21:55:21.097756+0000","last_undegraded":"2024-09-11T21:55:21.097756+0000","last_fullsized":"2024-09-11T21:55:21.097756+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:40:06.957836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"84'4","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:29.139061+0000","last_change":"2024-09-11T21:55:18.029127+0000","last_active":"2024-09-11T21:55:29.139061+0000","last_peered":"2024-09-11T21:55:29.139061+0000","last_clean":"2024-09-11T21:55:29.139061+0000","last_became_active":"2024-09-11T21:55:18.028740+0000","last_became_peered":"2024-09-11T21:55:18.028740+0000","last_unstale":"2024-09-11T21:55:29.139061+0000","last_undegraded":"2024-09-11T21:55:29.139061+0000","last_fullsized":"2024-09-11T21:55:29.139061+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:14:01.128486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.085688+0000","last_change":"2024-09-11T21:55:20.092498+0000","last_active":"2024-09-11T21:55:21.085688+0000","last_peered":"2024-09-11T21:55:21.085688+0000","last_clean":"2024-09-11T21:55:21.085688+0000","last_became_active":"2024-09-11T21:55:20.092179+0000","last_became_peered":"2024-09-11T21:55:20.092179+0000","last_unstale":"2024-09-11T21:55:21.085688+0000","last_undegraded":"2024-09-11T21:55:21.085688+0000","last_fullsized":"2024-09-11T21:55:21.085688+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:48:58.536942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"84'5","reported_seq":74,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:56:20.228249+0000","last_change":"2024-09-11T21:55:18.115798+0000","last_active":"2024-09-11T21:56:20.228249+0000","last_peered":"2024-09-11T21:56:20.228249+0000","last_clean":"2024-09-11T21:56:20.228249+0000","last_became_active":"2024-09-11T21:55:18.115433+0000","last_became_peered":"2024-09-11T21:55:18.115433+0000","last_unstale":"2024-09-11T21:56:20.228249+0000","last_undegraded":"2024-09-11T21:56:20.228249+0000","last_fullsized":"2024-09-11T21:56:20.228249+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:03:32.097756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":51,"num_read_kb":51,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.086704+0000","last_change":"2024-09-11T21:55:20.100439+0000","last_active":"2024-09-11T21:55:21.086704+0000","last_peered":"2024-09-11T21:55:21.086704+0000","last_clean":"2024-09-11T21:55:21.086704+0000","last_became_active":"2024-09-11T21:55:20.098717+0000","last_became_peered":"2024-09-11T21:55:20.098717+0000","last_unstale":"2024-09-11T21:55:21.086704+0000","last_undegraded":"2024-09-11T21:55:21.086704+0000","last_fullsized":"2024-09-11T21:55:21.086704+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:15:10.874225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"84'6","reported_seq":75,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:56:19.227055+0000","last_change":"2024-09-11T21:55:18.026588+0000","last_active":"2024-09-11T21:56:19.227055+0000","last_peered":"2024-09-11T21:56:19.227055+0000","last_clean":"2024-09-11T21:56:19.227055+0000","last_became_active":"2024-09-11T21:55:18.026205+0000","last_became_peered":"2024-09-11T21:55:18.026205+0000","last_unstale":"2024-09-11T21:56:19.227055+0000","last_undegraded":"2024-09-11T21:56:19.227055+0000","last_fullsized":"2024-09-11T21:56:19.227055+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:45:34.701088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":51,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.086341+0000","last_change":"2024-09-11T21:55:20.100677+0000","last_active":"2024-09-11T21:55:21.086341+0000","last_peered":"2024-09-11T21:55:21.086341+0000","last_clean":"2024-09-11T21:55:21.086341+0000","last_became_active":"2024-09-11T21:55:20.100334+0000","last_became_peered":"2024-09-11T21:55:20.100334+0000","last_unstale":"2024-09-11T21:55:21.086341+0000","last_undegraded":"2024-09-11T21:55:21.086341+0000","last_fullsized":"2024-09-11T21:55:21.086341+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:15:16.792831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"84'1","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.096989+0000","last_change":"2024-09-11T21:55:18.116329+0000","last_active":"2024-09-11T21:55:21.096989+0000","last_peered":"2024-09-11T21:55:21.096989+0000","last_clean":"2024-09-11T21:55:21.096989+0000","last_became_active":"2024-09-11T21:55:18.115944+0000","last_became_peered":"2024-09-11T21:55:18.115944+0000","last_unstale":"2024-09-11T21:55:21.096989+0000","last_undegraded":"2024-09-11T21:55:21.096989+0000","last_fullsized":"2024-09-11T21:55:21.096989+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:03:04.164985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"84'1","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.086593+0000","last_change":"2024-09-11T21:55:20.089202+0000","last_active":"2024-09-11T21:55:21.086593+0000","last_peered":"2024-09-11T21:55:21.086593+0000","last_clean":"2024-09-11T21:55:21.086593+0000","last_became_active":"2024-09-11T21:55:20.088853+0000","last_became_peered":"2024-09-11T21:55:20.088853+0000","last_unstale":"2024-09-11T21:55:21.086593+0000","last_undegraded":"2024-09-11T21:55:21.086593+0000","last_fullsized":"2024-09-11T21:55:21.086593+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:39:36.496944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"84'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:31.157241+0000","last_change":"2024-09-11T21:55:18.117917+0000","last_active":"2024-09-11T21:55:31.157241+0000","last_peered":"2024-09-11T21:55:31.157241+0000","last_clean":"2024-09-11T21:55:31.157241+0000","last_became_active":"2024-09-11T21:55:18.117567+0000","last_became_peered":"2024-09-11T21:55:18.117567+0000","last_unstale":"2024-09-11T21:55:31.157241+0000","last_undegraded":"2024-09-11T21:55:31.157241+0000","last_fullsized":"2024-09-11T21:55:31.157241+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:21:32.546620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.085865+0000","last_change":"2024-09-11T21:55:20.092370+0000","last_active":"2024-09-11T21:55:21.085865+0000","last_peered":"2024-09-11T21:55:21.085865+0000","last_clean":"2024-09-11T21:55:21.085865+0000","last_became_active":"2024-09-11T21:55:20.091982+0000","last_became_peered":"2024-09-11T21:55:20.091982+0000","last_unstale":"2024-09-11T21:55:21.085865+0000","last_undegraded":"2024-09-11T21:55:21.085865+0000","last_fullsized":"2024-09-11T21:55:21.085865+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:59:55.214511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"84'14","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:45.098695+0000","last_change":"2024-09-11T21:55:18.023166+0000","last_active":"2024-09-11T21:55:45.098695+0000","last_peered":"2024-09-11T21:55:45.098695+0000","last_clean":"2024-09-11T21:55:45.098695+0000","last_became_active":"2024-09-11T21:55:18.022845+0000","last_became_peered":"2024-09-11T21:55:18.022845+0000","last_unstale":"2024-09-11T21:55:45.098695+0000","last_undegraded":"2024-09-11T21:55:45.098695+0000","last_fullsized":"2024-09-11T21:55:45.098695+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:08:06.770410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.098445+0000","last_change":"2024-09-11T21:55:20.091541+0000","last_active":"2024-09-11T21:55:21.098445+0000","last_peered":"2024-09-11T21:55:21.098445+0000","last_clean":"2024-09-11T21:55:21.098445+0000","last_became_active":"2024-09-11T21:55:20.090971+0000","last_became_peered":"2024-09-11T21:55:20.090971+0000","last_unstale":"2024-09-11T21:55:21.098445+0000","last_undegraded":"2024-09-11T21:55:21.098445+0000","last_fullsized":"2024-09-11T21:55:21.098445+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:22:31.974711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"84'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:31.157079+0000","last_change":"2024-09-11T21:55:18.030694+0000","last_active":"2024-09-11T21:55:31.157079+0000","last_peered":"2024-09-11T21:55:31.157079+0000","last_clean":"2024-09-11T21:55:31.157079+0000","last_became_active":"2024-09-11T21:55:18.029687+0000","last_became_peered":"2024-09-11T21:55:18.029687+0000","last_unstale":"2024-09-11T21:55:31.157079+0000","last_undegraded":"2024-09-11T21:55:31.157079+0000","last_fullsized":"2024-09-11T21:55:31.157079+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:18:15.009543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.100188+0000","last_change":"2024-09-11T21:55:20.100838+0000","last_active":"2024-09-11T21:55:21.100188+0000","last_peered":"2024-09-11T21:55:21.100188+0000","last_clean":"2024-09-11T21:55:21.100188+0000","last_became_active":"2024-09-11T21:55:20.100523+0000","last_became_peered":"2024-09-11T21:55:20.100523+0000","last_unstale":"2024-09-11T21:55:21.100188+0000","last_undegraded":"2024-09-11T21:55:21.100188+0000","last_fullsized":"2024-09-11T21:55:21.100188+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:44:59.052265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"84'2","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:28.131582+0000","last_change":"2024-09-11T21:55:18.115977+0000","last_active":"2024-09-11T21:55:28.131582+0000","last_peered":"2024-09-11T21:55:28.131582+0000","last_clean":"2024-09-11T21:55:28.131582+0000","last_became_active":"2024-09-11T21:55:18.115649+0000","last_became_peered":"2024-09-11T21:55:18.115649+0000","last_unstale":"2024-09-11T21:55:28.131582+0000","last_undegraded":"2024-09-11T21:55:28.131582+0000","last_fullsized":"2024-09-11T21:55:28.131582+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:42:07.621024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.098589+0000","last_change":"2024-09-11T21:55:20.095536+0000","last_active":"2024-09-11T21:55:21.098589+0000","last_peered":"2024-09-11T21:55:21.098589+0000","last_clean":"2024-09-11T21:55:21.098589+0000","last_became_active":"2024-09-11T21:55:20.095117+0000","last_became_peered":"2024-09-11T21:55:20.095117+0000","last_unstale":"2024-09-11T21:55:21.098589+0000","last_undegraded":"2024-09-11T21:55:21.098589+0000","last_fullsized":"2024-09-11T21:55:21.098589+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:46.524922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"84'7","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:31.157374+0000","last_change":"2024-09-11T21:55:18.026592+0000","last_active":"2024-09-11T21:55:31.157374+0000","last_peered":"2024-09-11T21:55:31.157374+0000","last_clean":"2024-09-11T21:55:31.157374+0000","last_became_active":"2024-09-11T21:55:18.026263+0000","last_became_peered":"2024-09-11T21:55:18.026263+0000","last_unstale":"2024-09-11T21:55:31.157374+0000","last_undegraded":"2024-09-11T21:55:31.157374+0000","last_fullsized":"2024-09-11T21:55:31.157374+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:22:29.382964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.103050+0000","last_change":"2024-09-11T21:55:20.093655+0000","last_active":"2024-09-11T21:55:21.103050+0000","last_peered":"2024-09-11T21:55:21.103050+0000","last_clean":"2024-09-11T21:55:21.103050+0000","last_became_active":"2024-09-11T21:55:20.093287+0000","last_became_peered":"2024-09-11T21:55:20.093287+0000","last_unstale":"2024-09-11T21:55:21.103050+0000","last_undegraded":"2024-09-11T21:55:21.103050+0000","last_fullsized":"2024-09-11T21:55:21.103050+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:46:15.912710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"84'3","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:30.150643+0000","last_change":"2024-09-11T21:55:18.028465+0000","last_active":"2024-09-11T21:55:30.150643+0000","last_peered":"2024-09-11T21:55:30.150643+0000","last_clean":"2024-09-11T21:55:30.150643+0000","last_became_active":"2024-09-11T21:55:18.028139+0000","last_became_peered":"2024-09-11T21:55:18.028139+0000","last_unstale":"2024-09-11T21:55:30.150643+0000","last_undegraded":"2024-09-11T21:55:30.150643+0000","last_fullsized":"2024-09-11T21:55:30.150643+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:52:47.808058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.097166+0000","last_change":"2024-09-11T21:55:20.096400+0000","last_active":"2024-09-11T21:55:21.097166+0000","last_peered":"2024-09-11T21:55:21.097166+0000","last_clean":"2024-09-11T21:55:21.097166+0000","last_became_active":"2024-09-11T21:55:20.096023+0000","last_became_peered":"2024-09-11T21:55:20.096023+0000","last_unstale":"2024-09-11T21:55:21.097166+0000","last_undegraded":"2024-09-11T21:55:21.097166+0000","last_fullsized":"2024-09-11T21:55:21.097166+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:19:34.150687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"84'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:28.131668+0000","last_change":"2024-09-11T21:55:18.028759+0000","last_active":"2024-09-11T21:55:28.131668+0000","last_peered":"2024-09-11T21:55:28.131668+0000","last_clean":"2024-09-11T21:55:28.131668+0000","last_became_active":"2024-09-11T21:55:18.028366+0000","last_became_peered":"2024-09-11T21:55:18.028366+0000","last_unstale":"2024-09-11T21:55:28.131668+0000","last_undegraded":"2024-09-11T21:55:28.131668+0000","last_fullsized":"2024-09-11T21:55:28.131668+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:42:05.836114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.099139+0000","last_change":"2024-09-11T21:55:20.093593+0000","last_active":"2024-09-11T21:55:21.099139+0000","last_peered":"2024-09-11T21:55:21.099139+0000","last_clean":"2024-09-11T21:55:21.099139+0000","last_became_active":"2024-09-11T21:55:20.093189+0000","last_became_peered":"2024-09-11T21:55:20.093189+0000","last_unstale":"2024-09-11T21:55:21.099139+0000","last_undegraded":"2024-09-11T21:55:21.099139+0000","last_fullsized":"2024-09-11T21:55:21.099139+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:47:36.802816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.086802+0000","last_change":"2024-09-11T21:55:20.100887+0000","last_active":"2024-09-11T21:55:21.086802+0000","last_peered":"2024-09-11T21:55:21.086802+0000","last_clean":"2024-09-11T21:55:21.086802+0000","last_became_active":"2024-09-11T21:55:20.100233+0000","last_became_peered":"2024-09-11T21:55:20.100233+0000","last_unstale":"2024-09-11T21:55:21.086802+0000","last_undegraded":"2024-09-11T21:55:21.086802+0000","last_fullsized":"2024-09-11T21:55:21.086802+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:30:05.820375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"84'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:31.157480+0000","last_change":"2024-09-11T21:55:18.027365+0000","last_active":"2024-09-11T21:55:31.157480+0000","last_peered":"2024-09-11T21:55:31.157480+0000","last_clean":"2024-09-11T21:55:31.157480+0000","last_became_active":"2024-09-11T21:55:18.027058+0000","last_became_peered":"2024-09-11T21:55:18.027058+0000","last_unstale":"2024-09-11T21:55:31.157480+0000","last_undegraded":"2024-09-11T21:55:31.157480+0000","last_fullsized":"2024-09-11T21:55:31.157480+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:30:50.613246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.100428+0000","last_change":"2024-09-11T21:55:20.095154+0000","last_active":"2024-09-11T21:55:21.100428+0000","last_peered":"2024-09-11T21:55:21.100428+0000","last_clean":"2024-09-11T21:55:21.100428+0000","last_became_active":"2024-09-11T21:55:20.093607+0000","last_became_peered":"2024-09-11T21:55:20.093607+0000","last_unstale":"2024-09-11T21:55:21.100428+0000","last_undegraded":"2024-09-11T21:55:21.100428+0000","last_fullsized":"2024-09-11T21:55:21.100428+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:27:53.612467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"84'12","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:32.152672+0000","last_change":"2024-09-11T21:55:18.029872+0000","last_active":"2024-09-11T21:55:32.152672+0000","last_peered":"2024-09-11T21:55:32.152672+0000","last_clean":"2024-09-11T21:55:32.152672+0000","last_became_active":"2024-09-11T21:55:18.029538+0000","last_became_peered":"2024-09-11T21:55:18.029538+0000","last_unstale":"2024-09-11T21:55:32.152672+0000","last_undegraded":"2024-09-11T21:55:32.152672+0000","last_fullsized":"2024-09-11T21:55:32.152672+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:05:16.824805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.100650+0000","last_change":"2024-09-11T21:55:20.098713+0000","last_active":"2024-09-11T21:55:21.100650+0000","last_peered":"2024-09-11T21:55:21.100650+0000","last_clean":"2024-09-11T21:55:21.100650+0000","last_became_active":"2024-09-11T21:55:20.098219+0000","last_became_peered":"2024-09-11T21:55:20.098219+0000","last_unstale":"2024-09-11T21:55:21.100650+0000","last_undegraded":"2024-09-11T21:55:21.100650+0000","last_fullsized":"2024-09-11T21:55:21.100650+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:07:10.585366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"84'2","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.105133+0000","last_change":"2024-09-11T21:55:18.115868+0000","last_active":"2024-09-11T21:55:21.105133+0000","last_peered":"2024-09-11T21:55:21.105133+0000","last_clean":"2024-09-11T21:55:21.105133+0000","last_became_active":"2024-09-11T21:55:18.115486+0000","last_became_peered":"2024-09-11T21:55:18.115486+0000","last_unstale":"2024-09-11T21:55:21.105133+0000","last_undegraded":"2024-09-11T21:55:21.105133+0000","last_fullsized":"2024-09-11T21:55:21.105133+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:16:57.932041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.099076+0000","last_change":"2024-09-11T21:55:20.096077+0000","last_active":"2024-09-11T21:55:21.099076+0000","last_peered":"2024-09-11T21:55:21.099076+0000","last_clean":"2024-09-11T21:55:21.099076+0000","last_became_active":"2024-09-11T21:55:20.095739+0000","last_became_peered":"2024-09-11T21:55:20.095739+0000","last_unstale":"2024-09-11T21:55:21.099076+0000","last_undegraded":"2024-09-11T21:55:21.099076+0000","last_fullsized":"2024-09-11T21:55:21.099076+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:05:28.089828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"84'13","reported_seq":84,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:56:22.230740+0000","last_change":"2024-09-11T21:55:18.025771+0000","last_active":"2024-09-11T21:56:22.230740+0000","last_peered":"2024-09-11T21:56:22.230740+0000","last_clean":"2024-09-11T21:56:22.230740+0000","last_became_active":"2024-09-11T21:55:18.025384+0000","last_became_peered":"2024-09-11T21:55:18.025384+0000","last_unstale":"2024-09-11T21:56:22.230740+0000","last_undegraded":"2024-09-11T21:56:22.230740+0000","last_fullsized":"2024-09-11T21:56:22.230740+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:58:21.070061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":53,"num_read_kb":53,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66812,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":556,"num_read_kb":355,"num_write":241,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":200436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":207,"ondisk_log_size":207,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":283045,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449115,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":913408,"data_stored":436131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678047,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1069056,"data_stored":1085978,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":66,"seq":283467841573,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36260,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708892,"statfs":{"total":95995035648,"available":95957905408,"internally_reserved":0,"allocated":937984,"data_stored":435668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103210,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":933888,"data_stored":436046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332142,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":987136,"data_stored":1028959,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495667,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":435778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757305,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":1028244,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920829,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":435246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182468,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":435210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411401,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1003520,"data_stored":496515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36304,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708848,"statfs":{"total":95995035648,"available":95957860352,"internally_reserved":0,"allocated":983040,"data_stored":495902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738451,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":942080,"data_stored":437117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34355,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34355,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34355,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":60947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T21:56:26.159 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T21:56:26.432 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:56:26.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:26 smithi001 ceph-mon[30935]: pgmap v314: 97 pgs: 97 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-11T21:56:26.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:26 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-09-11T21:56:26.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:26 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]': finished 2024-09-11T21:56:26.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:26 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2024-09-11T21:56:26.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:26 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-09-11T21:56:26.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:26 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]': finished 2024-09-11T21:56:26.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:26 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-09-11T21:56:26.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:26 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.26", "id": [1, 9]}]': finished 2024-09-11T21:56:26.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:26 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-09-11T21:56:26.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:26 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]': finished 2024-09-11T21:56:26.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:26 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 7]}]': finished 2024-09-11T21:56:26.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:26 smithi001 ceph-mon[30935]: osdmap e85: 12 total, 12 up, 12 in 2024-09-11T21:56:26.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:26 smithi001 ceph-mon[30935]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T21:56:26.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:26 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:56:26.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:26 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:56:26.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:26 smithi053 ceph-mon[34908]: pgmap v314: 97 pgs: 97 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-11T21:56:26.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:26 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-09-11T21:56:26.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:26 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]': finished 2024-09-11T21:56:26.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:26 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2024-09-11T21:56:26.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:26 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-09-11T21:56:26.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:26 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]': finished 2024-09-11T21:56:26.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:26 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-09-11T21:56:26.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:26 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.26", "id": [1, 9]}]': finished 2024-09-11T21:56:26.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:26 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-09-11T21:56:26.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:26 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]': finished 2024-09-11T21:56:26.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:26 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 7]}]': finished 2024-09-11T21:56:26.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:26 smithi053 ceph-mon[34908]: osdmap e85: 12 total, 12 up, 12 in 2024-09-11T21:56:26.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:26 smithi053 ceph-mon[34908]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T21:56:26.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:26 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:56:26.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:26 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:56:26.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:26 smithi161 ceph-mon[35615]: pgmap v314: 97 pgs: 97 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-11T21:56:26.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:26 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-09-11T21:56:26.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:26 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]': finished 2024-09-11T21:56:26.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:26 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2024-09-11T21:56:26.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:26 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-09-11T21:56:26.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:26 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]': finished 2024-09-11T21:56:26.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:26 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-09-11T21:56:26.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:26 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.26", "id": [1, 9]}]': finished 2024-09-11T21:56:26.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:26 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-09-11T21:56:26.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:26 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 6]}]': finished 2024-09-11T21:56:26.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:26 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 7]}]': finished 2024-09-11T21:56:26.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:26 smithi161 ceph-mon[35615]: osdmap e85: 12 total, 12 up, 12 in 2024-09-11T21:56:26.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:26 smithi161 ceph-mon[35615]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T21:56:26.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:26 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:56:26.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:26 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:56:26.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:26.910+0000 7f017bb9e640 1 -- 172.21.15.1:0/3029708825 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0174106f10 msgr2=0x7f017410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:26.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:26.910+0000 7f017bb9e640 1 --2- 172.21.15.1:0/3029708825 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0174106f10 0x7f017410ea90 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f016c009ca0 tx=0x7f016c02a8c0 comp rx=0 tx=0).stop 2024-09-11T21:56:26.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:26.910+0000 7f017bb9e640 1 -- 172.21.15.1:0/3029708825 shutdown_connections 2024-09-11T21:56:26.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:26.910+0000 7f017bb9e640 1 --2- 172.21.15.1:0/3029708825 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0174106f10 0x7f017410ea90 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:26.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:26.910+0000 7f017bb9e640 1 --2- 172.21.15.1:0/3029708825 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0174102790 0x7f01741067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:26.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:26.910+0000 7f017bb9e640 1 --2- 172.21.15.1:0/3029708825 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0174101de0 0x7f01741021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:26.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:26.910+0000 7f017bb9e640 1 -- 172.21.15.1:0/3029708825 >> 172.21.15.1:0/3029708825 conn(0x7f01740fd8c0 msgr2=0x7f01740ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:26.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:26.910+0000 7f017bb9e640 1 -- 172.21.15.1:0/3029708825 shutdown_connections 2024-09-11T21:56:26.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:26.910+0000 7f017bb9e640 1 -- 172.21.15.1:0/3029708825 wait complete. 2024-09-11T21:56:26.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:26.911+0000 7f017bb9e640 1 Processor -- start 2024-09-11T21:56:26.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:26.911+0000 7f017bb9e640 1 -- start start 2024-09-11T21:56:26.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:26.911+0000 7f017bb9e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0174101de0 0x7f0174198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:26.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:26.911+0000 7f017bb9e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0174102790 0x7f0174198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:26.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:26.911+0000 7f017bb9e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0174106f10 0x7f017419cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:26.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:26.911+0000 7f017bb9e640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0174111f90 con 0x7f0174106f10 2024-09-11T21:56:26.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:26.911+0000 7f017bb9e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0174111e10 con 0x7f0174101de0 2024-09-11T21:56:26.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:26.911+0000 7f017bb9e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0174112110 con 0x7f0174102790 2024-09-11T21:56:26.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:26.911+0000 7f017a13b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0174106f10 0x7f017419cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:26.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:26.911+0000 7f017a13b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0174106f10 0x7f017419cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45200/0 (socket says 172.21.15.1:45200) 2024-09-11T21:56:26.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:26.911+0000 7f017a13b640 1 -- 172.21.15.1:0/1139984988 learned_addr learned my addr 172.21.15.1:0/1139984988 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:56:26.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:26.911+0000 7f017993a640 1 --2- 172.21.15.1:0/1139984988 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0174101de0 0x7f0174198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:26.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:26.911+0000 7f0179139640 1 --2- 172.21.15.1:0/1139984988 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0174102790 0x7f0174198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:26.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:26.911+0000 7f017a13b640 1 -- 172.21.15.1:0/1139984988 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0174102790 msgr2=0x7f0174198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:26.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:26.911+0000 7f017a13b640 1 --2- 172.21.15.1:0/1139984988 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0174102790 0x7f0174198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:26.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:26.911+0000 7f017a13b640 1 -- 172.21.15.1:0/1139984988 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0174101de0 msgr2=0x7f0174198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:26.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:26.911+0000 7f017a13b640 1 --2- 172.21.15.1:0/1139984988 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0174101de0 0x7f0174198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:26.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:26.911+0000 7f017a13b640 1 -- 172.21.15.1:0/1139984988 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f017419d3c0 con 0x7f0174106f10 2024-09-11T21:56:26.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:26.912+0000 7f017a13b640 1 --2- 172.21.15.1:0/1139984988 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0174106f10 0x7f017419cce0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f016c031a60 tx=0x7f016c031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:26.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:26.912+0000 7f016affd640 1 -- 172.21.15.1:0/1139984988 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f016c003c30 con 0x7f0174106f10 2024-09-11T21:56:26.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:26.912+0000 7f017bb9e640 1 -- 172.21.15.1:0/1139984988 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f017419d650 con 0x7f0174106f10 2024-09-11T21:56:26.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:26.912+0000 7f016affd640 1 -- 172.21.15.1:0/1139984988 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f016c003dd0 con 0x7f0174106f10 2024-09-11T21:56:26.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:26.912+0000 7f016affd640 1 -- 172.21.15.1:0/1139984988 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f016c0047e0 con 0x7f0174106f10 2024-09-11T21:56:26.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:26.912+0000 7f017bb9e640 1 -- 172.21.15.1:0/1139984988 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01741a4f90 con 0x7f0174106f10 2024-09-11T21:56:26.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:26.913+0000 7f017bb9e640 1 -- 172.21.15.1:0/1139984988 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01741021c0 con 0x7f0174106f10 2024-09-11T21:56:26.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:26.914+0000 7f016affd640 1 -- 172.21.15.1:0/1139984988 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f016c004b40 con 0x7f0174106f10 2024-09-11T21:56:26.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:26.915+0000 7f016affd640 1 --2- 172.21.15.1:0/1139984988 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f01600a2c10 0x7f01600a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:26.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:26.915+0000 7f016affd640 1 -- 172.21.15.1:0/1139984988 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f016c0f06f0 con 0x7f0174106f10 2024-09-11T21:56:26.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:26.915+0000 7f017993a640 1 --2- 172.21.15.1:0/1139984988 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f01600a2c10 0x7f01600a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:26.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:26.916+0000 7f017993a640 1 --2- 172.21.15.1:0/1139984988 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f01600a2c10 0x7f01600a50d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0170009890 tx=0x7f0170008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:26.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:26.918+0000 7f016affd640 1 -- 172.21.15.1:0/1139984988 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f016c0bd5e0 con 0x7f0174106f10 2024-09-11T21:56:27.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:27.102+0000 7f017bb9e640 1 -- 172.21.15.1:0/1139984988 --> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f01740fec20 con 0x7f01600a2c10 2024-09-11T21:56:27.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:27.109+0000 7f016affd640 1 -- 172.21.15.1:0/1139984988 <== mgr.14150 v2:172.21.15.1:6800/3890350886 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258952 (secure 0 0 0) 0x7f01740fec20 con 0x7f01600a2c10 2024-09-11T21:56:27.112 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:56:27.113 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-11T21:56:27.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:27.112+0000 7f017bb9e640 1 -- 172.21.15.1:0/1139984988 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f01600a2c10 msgr2=0x7f01600a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:27.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:27.112+0000 7f017bb9e640 1 --2- 172.21.15.1:0/1139984988 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f01600a2c10 0x7f01600a50d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0170009890 tx=0x7f0170008040 comp rx=0 tx=0).stop 2024-09-11T21:56:27.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:27.112+0000 7f017bb9e640 1 -- 172.21.15.1:0/1139984988 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0174106f10 msgr2=0x7f017419cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:27.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:27.112+0000 7f017bb9e640 1 --2- 172.21.15.1:0/1139984988 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0174106f10 0x7f017419cce0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f016c031a60 tx=0x7f016c031a90 comp rx=0 tx=0).stop 2024-09-11T21:56:27.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:27.112+0000 7f017bb9e640 1 -- 172.21.15.1:0/1139984988 shutdown_connections 2024-09-11T21:56:27.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:27.112+0000 7f017bb9e640 1 --2- 172.21.15.1:0/1139984988 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f01600a2c10 0x7f01600a50d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:27.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:27.112+0000 7f017bb9e640 1 --2- 172.21.15.1:0/1139984988 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0174106f10 0x7f017419cce0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:27.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:27.112+0000 7f017bb9e640 1 --2- 172.21.15.1:0/1139984988 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0174102790 0x7f0174198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:27.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:27.112+0000 7f017bb9e640 1 --2- 172.21.15.1:0/1139984988 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0174101de0 0x7f0174198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:27.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:27.112+0000 7f017bb9e640 1 -- 172.21.15.1:0/1139984988 >> 172.21.15.1:0/1139984988 conn(0x7f01740fd8c0 msgr2=0x7f01740ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:27.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:27.112+0000 7f017bb9e640 1 -- 172.21.15.1:0/1139984988 shutdown_connections 2024-09-11T21:56:27.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:27.112+0000 7f017bb9e640 1 -- 172.21.15.1:0/1139984988 wait complete. 2024-09-11T21:56:27.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:27 smithi001 ceph-mon[30935]: osdmap e86: 12 total, 12 up, 12 in 2024-09-11T21:56:27.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:27 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:56:27.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:27 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:56:27.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:27 smithi001 ceph-mon[30935]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:56:27.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:27 smithi053 ceph-mon[34908]: osdmap e86: 12 total, 12 up, 12 in 2024-09-11T21:56:27.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:27 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:56:27.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:27 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:56:27.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:27 smithi053 ceph-mon[34908]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:56:27.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:27 smithi161 ceph-mon[35615]: osdmap e86: 12 total, 12 up, 12 in 2024-09-11T21:56:27.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:27 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:56:27.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:27 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:56:27.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:27 smithi161 ceph-mon[35615]: from='mgr.14150 172.21.15.1:0/706499385' entity='mgr.x' 2024-09-11T21:56:27.797 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":317,"stamp":"2024-09-11T21:56:26.622765+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657414,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":707,"num_read_kb":479,"num_write":424,"num_write_kb":3073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":346,"ondisk_log_size":346,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":385536,"kb_used_data":11316,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124556288,"statfs":{"total":1151940427776,"available":1151545638912,"internally_reserved":0,"allocated":11587584,"data_stored":7213071,"data_compressed":116634,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19142,"internal_metadata":382973242},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":45,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002179"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.103272+0000","last_change":"2024-09-11T21:55:20.095385+0000","last_active":"2024-09-11T21:55:21.103272+0000","last_peered":"2024-09-11T21:55:21.103272+0000","last_clean":"2024-09-11T21:55:21.103272+0000","last_became_active":"2024-09-11T21:55:20.095077+0000","last_became_peered":"2024-09-11T21:55:20.095077+0000","last_unstale":"2024-09-11T21:55:21.103272+0000","last_undegraded":"2024-09-11T21:55:21.103272+0000","last_fullsized":"2024-09-11T21:55:21.103272+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:53:52.231522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.099859+0000","last_change":"2024-09-11T21:55:20.103540+0000","last_active":"2024-09-11T21:55:21.099859+0000","last_peered":"2024-09-11T21:55:21.099859+0000","last_clean":"2024-09-11T21:55:21.099859+0000","last_became_active":"2024-09-11T21:55:20.103233+0000","last_became_peered":"2024-09-11T21:55:20.103233+0000","last_unstale":"2024-09-11T21:55:21.099859+0000","last_undegraded":"2024-09-11T21:55:21.099859+0000","last_fullsized":"2024-09-11T21:55:21.099859+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:51:29.222202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.100567+0000","last_change":"2024-09-11T21:55:20.098538+0000","last_active":"2024-09-11T21:55:21.100567+0000","last_peered":"2024-09-11T21:55:21.100567+0000","last_clean":"2024-09-11T21:55:21.100567+0000","last_became_active":"2024-09-11T21:55:20.098202+0000","last_became_peered":"2024-09-11T21:55:20.098202+0000","last_unstale":"2024-09-11T21:55:21.100567+0000","last_undegraded":"2024-09-11T21:55:21.100567+0000","last_fullsized":"2024-09-11T21:55:21.100567+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:33:46.636952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T21:56:25.286828+0000","last_change":"2024-09-11T21:55:20.100869+0000","last_active":"2024-09-11T21:56:25.286828+0000","last_peered":"2024-09-11T21:56:25.286828+0000","last_clean":"2024-09-11T21:56:25.286828+0000","last_became_active":"2024-09-11T21:55:20.100573+0000","last_became_peered":"2024-09-11T21:55:20.100573+0000","last_unstale":"2024-09-11T21:56:25.286828+0000","last_undegraded":"2024-09-11T21:56:25.286828+0000","last_fullsized":"2024-09-11T21:56:25.286828+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:58:11.154105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.098166+0000","last_change":"2024-09-11T21:55:20.091648+0000","last_active":"2024-09-11T21:55:21.098166+0000","last_peered":"2024-09-11T21:55:21.098166+0000","last_clean":"2024-09-11T21:55:21.098166+0000","last_became_active":"2024-09-11T21:55:20.091222+0000","last_became_peered":"2024-09-11T21:55:20.091222+0000","last_unstale":"2024-09-11T21:55:21.098166+0000","last_undegraded":"2024-09-11T21:55:21.098166+0000","last_fullsized":"2024-09-11T21:55:21.098166+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:55:03.420754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.086150+0000","last_change":"2024-09-11T21:55:20.100446+0000","last_active":"2024-09-11T21:55:21.086150+0000","last_peered":"2024-09-11T21:55:21.086150+0000","last_clean":"2024-09-11T21:55:21.086150+0000","last_became_active":"2024-09-11T21:55:20.100062+0000","last_became_peered":"2024-09-11T21:55:20.100062+0000","last_unstale":"2024-09-11T21:55:21.086150+0000","last_undegraded":"2024-09-11T21:55:21.086150+0000","last_fullsized":"2024-09-11T21:55:21.086150+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:49:37.109258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T21:56:25.285976+0000","last_change":"2024-09-11T21:55:20.101084+0000","last_active":"2024-09-11T21:56:25.285976+0000","last_peered":"2024-09-11T21:56:25.285976+0000","last_clean":"2024-09-11T21:56:25.285976+0000","last_became_active":"2024-09-11T21:55:20.100739+0000","last_became_peered":"2024-09-11T21:55:20.100739+0000","last_unstale":"2024-09-11T21:56:25.285976+0000","last_undegraded":"2024-09-11T21:56:25.285976+0000","last_fullsized":"2024-09-11T21:56:25.285976+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:46:58.969559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.098694+0000","last_change":"2024-09-11T21:55:20.088715+0000","last_active":"2024-09-11T21:55:21.098694+0000","last_peered":"2024-09-11T21:55:21.098694+0000","last_clean":"2024-09-11T21:55:21.098694+0000","last_became_active":"2024-09-11T21:55:20.088280+0000","last_became_peered":"2024-09-11T21:55:20.088280+0000","last_unstale":"2024-09-11T21:55:21.098694+0000","last_undegraded":"2024-09-11T21:55:21.098694+0000","last_fullsized":"2024-09-11T21:55:21.098694+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:06:24.450444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.101362+0000","last_change":"2024-09-11T21:55:20.101266+0000","last_active":"2024-09-11T21:55:21.101362+0000","last_peered":"2024-09-11T21:55:21.101362+0000","last_clean":"2024-09-11T21:55:21.101362+0000","last_became_active":"2024-09-11T21:55:20.100919+0000","last_became_peered":"2024-09-11T21:55:20.100919+0000","last_unstale":"2024-09-11T21:55:21.101362+0000","last_undegraded":"2024-09-11T21:55:21.101362+0000","last_fullsized":"2024-09-11T21:55:21.101362+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:47:15.466082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.086697+0000","last_change":"2024-09-11T21:55:20.098086+0000","last_active":"2024-09-11T21:55:21.086697+0000","last_peered":"2024-09-11T21:55:21.086697+0000","last_clean":"2024-09-11T21:55:21.086697+0000","last_became_active":"2024-09-11T21:55:20.097754+0000","last_became_peered":"2024-09-11T21:55:20.097754+0000","last_unstale":"2024-09-11T21:55:21.086697+0000","last_undegraded":"2024-09-11T21:55:21.086697+0000","last_fullsized":"2024-09-11T21:55:21.086697+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:37:00.098345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T21:56:25.286545+0000","last_change":"2024-09-11T21:55:20.092671+0000","last_active":"2024-09-11T21:56:25.286545+0000","last_peered":"2024-09-11T21:56:25.286545+0000","last_clean":"2024-09-11T21:56:25.286545+0000","last_became_active":"2024-09-11T21:55:20.092112+0000","last_became_peered":"2024-09-11T21:55:20.092112+0000","last_unstale":"2024-09-11T21:56:25.286545+0000","last_undegraded":"2024-09-11T21:56:25.286545+0000","last_fullsized":"2024-09-11T21:56:25.286545+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:06:28.538838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T21:56:25.286871+0000","last_change":"2024-09-11T21:55:20.099863+0000","last_active":"2024-09-11T21:56:25.286871+0000","last_peered":"2024-09-11T21:56:25.286871+0000","last_clean":"2024-09-11T21:56:25.286871+0000","last_became_active":"2024-09-11T21:55:20.099173+0000","last_became_peered":"2024-09-11T21:55:20.099173+0000","last_unstale":"2024-09-11T21:56:25.286871+0000","last_undegraded":"2024-09-11T21:56:25.286871+0000","last_fullsized":"2024-09-11T21:56:25.286871+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:34:56.669410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T21:56:26.374755+0000","last_change":"2024-09-11T21:55:20.101212+0000","last_active":"2024-09-11T21:56:26.374755+0000","last_peered":"2024-09-11T21:56:26.374755+0000","last_clean":"2024-09-11T21:56:26.374755+0000","last_became_active":"2024-09-11T21:55:20.100876+0000","last_became_peered":"2024-09-11T21:55:20.100876+0000","last_unstale":"2024-09-11T21:56:26.374755+0000","last_undegraded":"2024-09-11T21:56:26.374755+0000","last_fullsized":"2024-09-11T21:56:26.374755+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:14:18.104009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.100238+0000","last_change":"2024-09-11T21:55:20.095394+0000","last_active":"2024-09-11T21:55:21.100238+0000","last_peered":"2024-09-11T21:55:21.100238+0000","last_clean":"2024-09-11T21:55:21.100238+0000","last_became_active":"2024-09-11T21:55:20.095101+0000","last_became_peered":"2024-09-11T21:55:20.095101+0000","last_unstale":"2024-09-11T21:55:21.100238+0000","last_undegraded":"2024-09-11T21:55:21.100238+0000","last_fullsized":"2024-09-11T21:55:21.100238+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:53:32.529359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.100035+0000","last_change":"2024-09-11T21:55:20.099668+0000","last_active":"2024-09-11T21:55:21.100035+0000","last_peered":"2024-09-11T21:55:21.100035+0000","last_clean":"2024-09-11T21:55:21.100035+0000","last_became_active":"2024-09-11T21:55:20.099392+0000","last_became_peered":"2024-09-11T21:55:20.099392+0000","last_unstale":"2024-09-11T21:55:21.100035+0000","last_undegraded":"2024-09-11T21:55:21.100035+0000","last_fullsized":"2024-09-11T21:55:21.100035+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:50:00.479508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.099808+0000","last_change":"2024-09-11T21:55:20.093019+0000","last_active":"2024-09-11T21:55:21.099808+0000","last_peered":"2024-09-11T21:55:21.099808+0000","last_clean":"2024-09-11T21:55:21.099808+0000","last_became_active":"2024-09-11T21:55:20.092691+0000","last_became_peered":"2024-09-11T21:55:20.092691+0000","last_unstale":"2024-09-11T21:55:21.099808+0000","last_undegraded":"2024-09-11T21:55:21.099808+0000","last_fullsized":"2024-09-11T21:55:21.099808+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:04:36.684992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.103502+0000","last_change":"2024-09-11T21:55:20.097333+0000","last_active":"2024-09-11T21:55:21.103502+0000","last_peered":"2024-09-11T21:55:21.103502+0000","last_clean":"2024-09-11T21:55:21.103502+0000","last_became_active":"2024-09-11T21:55:20.096752+0000","last_became_peered":"2024-09-11T21:55:20.096752+0000","last_unstale":"2024-09-11T21:55:21.103502+0000","last_undegraded":"2024-09-11T21:55:21.103502+0000","last_fullsized":"2024-09-11T21:55:21.103502+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:27:20.722174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.097172+0000","last_change":"2024-09-11T21:55:20.093972+0000","last_active":"2024-09-11T21:55:21.097172+0000","last_peered":"2024-09-11T21:55:21.097172+0000","last_clean":"2024-09-11T21:55:21.097172+0000","last_became_active":"2024-09-11T21:55:20.093633+0000","last_became_peered":"2024-09-11T21:55:20.093633+0000","last_unstale":"2024-09-11T21:55:21.097172+0000","last_undegraded":"2024-09-11T21:55:21.097172+0000","last_fullsized":"2024-09-11T21:55:21.097172+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:25:18.973332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.097726+0000","last_change":"2024-09-11T21:55:20.094239+0000","last_active":"2024-09-11T21:55:21.097726+0000","last_peered":"2024-09-11T21:55:21.097726+0000","last_clean":"2024-09-11T21:55:21.097726+0000","last_became_active":"2024-09-11T21:55:20.093924+0000","last_became_peered":"2024-09-11T21:55:20.093924+0000","last_unstale":"2024-09-11T21:55:21.097726+0000","last_undegraded":"2024-09-11T21:55:21.097726+0000","last_fullsized":"2024-09-11T21:55:21.097726+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:39:53.012780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.102943+0000","last_change":"2024-09-11T21:55:20.099335+0000","last_active":"2024-09-11T21:55:21.102943+0000","last_peered":"2024-09-11T21:55:21.102943+0000","last_clean":"2024-09-11T21:55:21.102943+0000","last_became_active":"2024-09-11T21:55:20.098929+0000","last_became_peered":"2024-09-11T21:55:20.098929+0000","last_unstale":"2024-09-11T21:55:21.102943+0000","last_undegraded":"2024-09-11T21:55:21.102943+0000","last_fullsized":"2024-09-11T21:55:21.102943+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:38:43.117829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T21:56:26.374865+0000","last_change":"2024-09-11T21:55:20.094068+0000","last_active":"2024-09-11T21:56:26.374865+0000","last_peered":"2024-09-11T21:56:26.374865+0000","last_clean":"2024-09-11T21:56:26.374865+0000","last_became_active":"2024-09-11T21:55:20.093637+0000","last_became_peered":"2024-09-11T21:55:20.093637+0000","last_unstale":"2024-09-11T21:56:26.374865+0000","last_undegraded":"2024-09-11T21:56:26.374865+0000","last_fullsized":"2024-09-11T21:56:26.374865+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:52:59.289715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.100443+0000","last_change":"2024-09-11T21:55:20.090778+0000","last_active":"2024-09-11T21:55:21.100443+0000","last_peered":"2024-09-11T21:55:21.100443+0000","last_clean":"2024-09-11T21:55:21.100443+0000","last_became_active":"2024-09-11T21:55:20.090443+0000","last_became_peered":"2024-09-11T21:55:20.090443+0000","last_unstale":"2024-09-11T21:55:21.100443+0000","last_undegraded":"2024-09-11T21:55:21.100443+0000","last_fullsized":"2024-09-11T21:55:21.100443+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:46:21.488228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.097160+0000","last_change":"2024-09-11T21:55:20.092238+0000","last_active":"2024-09-11T21:55:21.097160+0000","last_peered":"2024-09-11T21:55:21.097160+0000","last_clean":"2024-09-11T21:55:21.097160+0000","last_became_active":"2024-09-11T21:55:20.091961+0000","last_became_peered":"2024-09-11T21:55:20.091961+0000","last_unstale":"2024-09-11T21:55:21.097160+0000","last_undegraded":"2024-09-11T21:55:21.097160+0000","last_fullsized":"2024-09-11T21:55:21.097160+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:54:40.445441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.086361+0000","last_change":"2024-09-11T21:55:20.102944+0000","last_active":"2024-09-11T21:55:21.086361+0000","last_peered":"2024-09-11T21:55:21.086361+0000","last_clean":"2024-09-11T21:55:21.086361+0000","last_became_active":"2024-09-11T21:55:20.102533+0000","last_became_peered":"2024-09-11T21:55:20.102533+0000","last_unstale":"2024-09-11T21:55:21.086361+0000","last_undegraded":"2024-09-11T21:55:21.086361+0000","last_fullsized":"2024-09-11T21:55:21.086361+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:12:15.832622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.098471+0000","last_change":"2024-09-11T21:55:20.092177+0000","last_active":"2024-09-11T21:55:21.098471+0000","last_peered":"2024-09-11T21:55:21.098471+0000","last_clean":"2024-09-11T21:55:21.098471+0000","last_became_active":"2024-09-11T21:55:20.091897+0000","last_became_peered":"2024-09-11T21:55:20.091897+0000","last_unstale":"2024-09-11T21:55:21.098471+0000","last_undegraded":"2024-09-11T21:55:21.098471+0000","last_fullsized":"2024-09-11T21:55:21.098471+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:38:57.371153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.099223+0000","last_change":"2024-09-11T21:55:20.092166+0000","last_active":"2024-09-11T21:55:21.099223+0000","last_peered":"2024-09-11T21:55:21.099223+0000","last_clean":"2024-09-11T21:55:21.099223+0000","last_became_active":"2024-09-11T21:55:20.091854+0000","last_became_peered":"2024-09-11T21:55:20.091854+0000","last_unstale":"2024-09-11T21:55:21.099223+0000","last_undegraded":"2024-09-11T21:55:21.099223+0000","last_fullsized":"2024-09-11T21:55:21.099223+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:53:32.256008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.103425+0000","last_change":"2024-09-11T21:55:20.103976+0000","last_active":"2024-09-11T21:55:21.103425+0000","last_peered":"2024-09-11T21:55:21.103425+0000","last_clean":"2024-09-11T21:55:21.103425+0000","last_became_active":"2024-09-11T21:55:20.103668+0000","last_became_peered":"2024-09-11T21:55:20.103668+0000","last_unstale":"2024-09-11T21:55:21.103425+0000","last_undegraded":"2024-09-11T21:55:21.103425+0000","last_fullsized":"2024-09-11T21:55:21.103425+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:13:42.663641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T21:56:26.375052+0000","last_change":"2024-09-11T21:55:20.093898+0000","last_active":"2024-09-11T21:56:26.375052+0000","last_peered":"2024-09-11T21:56:26.375052+0000","last_clean":"2024-09-11T21:56:26.375052+0000","last_became_active":"2024-09-11T21:55:20.093348+0000","last_became_peered":"2024-09-11T21:55:20.093348+0000","last_unstale":"2024-09-11T21:56:26.375052+0000","last_undegraded":"2024-09-11T21:56:26.375052+0000","last_fullsized":"2024-09-11T21:56:26.375052+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:00:06.185478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.100902+0000","last_change":"2024-09-11T21:55:20.100575+0000","last_active":"2024-09-11T21:55:21.100902+0000","last_peered":"2024-09-11T21:55:21.100902+0000","last_clean":"2024-09-11T21:55:21.100902+0000","last_became_active":"2024-09-11T21:55:20.100255+0000","last_became_peered":"2024-09-11T21:55:20.100255+0000","last_unstale":"2024-09-11T21:55:21.100902+0000","last_undegraded":"2024-09-11T21:55:21.100902+0000","last_fullsized":"2024-09-11T21:55:21.100902+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:48:57.718298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.097296+0000","last_change":"2024-09-11T21:55:20.097548+0000","last_active":"2024-09-11T21:55:21.097296+0000","last_peered":"2024-09-11T21:55:21.097296+0000","last_clean":"2024-09-11T21:55:21.097296+0000","last_became_active":"2024-09-11T21:55:20.097124+0000","last_became_peered":"2024-09-11T21:55:20.097124+0000","last_unstale":"2024-09-11T21:55:21.097296+0000","last_undegraded":"2024-09-11T21:55:21.097296+0000","last_fullsized":"2024-09-11T21:55:21.097296+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:22:23.904457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.097695+0000","last_change":"2024-09-11T21:55:20.093822+0000","last_active":"2024-09-11T21:55:21.097695+0000","last_peered":"2024-09-11T21:55:21.097695+0000","last_clean":"2024-09-11T21:55:21.097695+0000","last_became_active":"2024-09-11T21:55:20.093361+0000","last_became_peered":"2024-09-11T21:55:20.093361+0000","last_unstale":"2024-09-11T21:55:21.097695+0000","last_undegraded":"2024-09-11T21:55:21.097695+0000","last_fullsized":"2024-09-11T21:55:21.097695+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:37:06.362732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.103360+0000","last_change":"2024-09-11T21:55:20.103990+0000","last_active":"2024-09-11T21:55:21.103360+0000","last_peered":"2024-09-11T21:55:21.103360+0000","last_clean":"2024-09-11T21:55:21.103360+0000","last_became_active":"2024-09-11T21:55:20.103375+0000","last_became_peered":"2024-09-11T21:55:20.103375+0000","last_unstale":"2024-09-11T21:55:21.103360+0000","last_undegraded":"2024-09-11T21:55:21.103360+0000","last_fullsized":"2024-09-11T21:55:21.103360+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:02:06.786888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.099679+0000","last_change":"2024-09-11T21:55:20.101535+0000","last_active":"2024-09-11T21:55:21.099679+0000","last_peered":"2024-09-11T21:55:21.099679+0000","last_clean":"2024-09-11T21:55:21.099679+0000","last_became_active":"2024-09-11T21:55:20.101129+0000","last_became_peered":"2024-09-11T21:55:20.101129+0000","last_unstale":"2024-09-11T21:55:21.099679+0000","last_undegraded":"2024-09-11T21:55:21.099679+0000","last_fullsized":"2024-09-11T21:55:21.099679+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:09:15.673670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"84'6","reported_seq":77,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:56:20.229052+0000","last_change":"2024-09-11T21:55:18.026947+0000","last_active":"2024-09-11T21:56:20.229052+0000","last_peered":"2024-09-11T21:56:20.229052+0000","last_clean":"2024-09-11T21:56:20.229052+0000","last_became_active":"2024-09-11T21:55:18.026632+0000","last_became_peered":"2024-09-11T21:55:18.026632+0000","last_unstale":"2024-09-11T21:56:20.229052+0000","last_undegraded":"2024-09-11T21:56:20.229052+0000","last_fullsized":"2024-09-11T21:56:20.229052+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:55:16.618074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.086576+0000","last_change":"2024-09-11T21:55:20.100197+0000","last_active":"2024-09-11T21:55:21.086576+0000","last_peered":"2024-09-11T21:55:21.086576+0000","last_clean":"2024-09-11T21:55:21.086576+0000","last_became_active":"2024-09-11T21:55:20.098494+0000","last_became_peered":"2024-09-11T21:55:20.098494+0000","last_unstale":"2024-09-11T21:55:21.086576+0000","last_undegraded":"2024-09-11T21:55:21.086576+0000","last_fullsized":"2024-09-11T21:55:21.086576+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:14.505314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"84'7","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:30.150870+0000","last_change":"2024-09-11T21:55:18.024359+0000","last_active":"2024-09-11T21:55:30.150870+0000","last_peered":"2024-09-11T21:55:30.150870+0000","last_clean":"2024-09-11T21:55:30.150870+0000","last_became_active":"2024-09-11T21:55:18.023624+0000","last_became_peered":"2024-09-11T21:55:18.023624+0000","last_unstale":"2024-09-11T21:55:30.150870+0000","last_undegraded":"2024-09-11T21:55:30.150870+0000","last_fullsized":"2024-09-11T21:55:30.150870+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:13:38.582232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"84'3","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T21:56:26.374505+0000","last_change":"2024-09-11T21:55:18.032541+0000","last_active":"2024-09-11T21:56:26.374505+0000","last_peered":"2024-09-11T21:56:26.374505+0000","last_clean":"2024-09-11T21:56:26.374505+0000","last_became_active":"2024-09-11T21:55:18.032218+0000","last_became_peered":"2024-09-11T21:55:18.032218+0000","last_unstale":"2024-09-11T21:56:26.374505+0000","last_undegraded":"2024-09-11T21:56:26.374505+0000","last_fullsized":"2024-09-11T21:56:26.374505+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:38:20.872069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.098851+0000","last_change":"2024-09-11T21:55:20.090030+0000","last_active":"2024-09-11T21:55:21.098851+0000","last_peered":"2024-09-11T21:55:21.098851+0000","last_clean":"2024-09-11T21:55:21.098851+0000","last_became_active":"2024-09-11T21:55:20.089481+0000","last_became_peered":"2024-09-11T21:55:20.089481+0000","last_unstale":"2024-09-11T21:55:21.098851+0000","last_undegraded":"2024-09-11T21:55:21.098851+0000","last_fullsized":"2024-09-11T21:55:21.098851+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:41:57.038519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"84'3","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:31.157367+0000","last_change":"2024-09-11T21:55:18.026656+0000","last_active":"2024-09-11T21:55:31.157367+0000","last_peered":"2024-09-11T21:55:31.157367+0000","last_clean":"2024-09-11T21:55:31.157367+0000","last_became_active":"2024-09-11T21:55:18.026318+0000","last_became_peered":"2024-09-11T21:55:18.026318+0000","last_unstale":"2024-09-11T21:55:31.157367+0000","last_undegraded":"2024-09-11T21:55:31.157367+0000","last_fullsized":"2024-09-11T21:55:31.157367+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:14:41.438334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.096818+0000","last_change":"2024-09-11T21:55:20.090790+0000","last_active":"2024-09-11T21:55:21.096818+0000","last_peered":"2024-09-11T21:55:21.096818+0000","last_clean":"2024-09-11T21:55:21.096818+0000","last_became_active":"2024-09-11T21:55:20.090468+0000","last_became_peered":"2024-09-11T21:55:20.090468+0000","last_unstale":"2024-09-11T21:55:21.096818+0000","last_undegraded":"2024-09-11T21:55:21.096818+0000","last_fullsized":"2024-09-11T21:55:21.096818+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:19.115482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"84'4","reported_seq":128,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:56:24.233855+0000","last_change":"2024-09-11T21:55:18.026989+0000","last_active":"2024-09-11T21:56:24.233855+0000","last_peered":"2024-09-11T21:56:24.233855+0000","last_clean":"2024-09-11T21:56:24.233855+0000","last_became_active":"2024-09-11T21:55:18.026229+0000","last_became_peered":"2024-09-11T21:55:18.026229+0000","last_unstale":"2024-09-11T21:56:24.233855+0000","last_undegraded":"2024-09-11T21:56:24.233855+0000","last_fullsized":"2024-09-11T21:56:24.233855+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:50:35.745789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":106,"num_read_kb":106,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.101844+0000","last_change":"2024-09-11T21:55:20.095884+0000","last_active":"2024-09-11T21:55:21.101844+0000","last_peered":"2024-09-11T21:55:21.101844+0000","last_clean":"2024-09-11T21:55:21.101844+0000","last_became_active":"2024-09-11T21:55:20.094887+0000","last_became_peered":"2024-09-11T21:55:20.094887+0000","last_unstale":"2024-09-11T21:55:21.101844+0000","last_undegraded":"2024-09-11T21:55:21.101844+0000","last_fullsized":"2024-09-11T21:55:21.101844+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:45:03.579703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"84'4","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T21:56:25.286426+0000","last_change":"2024-09-11T21:55:18.030474+0000","last_active":"2024-09-11T21:56:25.286426+0000","last_peered":"2024-09-11T21:56:25.286426+0000","last_clean":"2024-09-11T21:56:25.286426+0000","last_became_active":"2024-09-11T21:55:18.030104+0000","last_became_peered":"2024-09-11T21:55:18.030104+0000","last_unstale":"2024-09-11T21:56:25.286426+0000","last_undegraded":"2024-09-11T21:56:25.286426+0000","last_fullsized":"2024-09-11T21:56:25.286426+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:21:00.484595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.100730+0000","last_change":"2024-09-11T21:55:20.102807+0000","last_active":"2024-09-11T21:55:21.100730+0000","last_peered":"2024-09-11T21:55:21.100730+0000","last_clean":"2024-09-11T21:55:21.100730+0000","last_became_active":"2024-09-11T21:55:20.102581+0000","last_became_peered":"2024-09-11T21:55:20.102581+0000","last_unstale":"2024-09-11T21:55:21.100730+0000","last_undegraded":"2024-09-11T21:55:21.100730+0000","last_fullsized":"2024-09-11T21:55:21.100730+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:29:23.139788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"84'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:30.150754+0000","last_change":"2024-09-11T21:55:18.117685+0000","last_active":"2024-09-11T21:55:30.150754+0000","last_peered":"2024-09-11T21:55:30.150754+0000","last_clean":"2024-09-11T21:55:30.150754+0000","last_became_active":"2024-09-11T21:55:18.117261+0000","last_became_peered":"2024-09-11T21:55:18.117261+0000","last_unstale":"2024-09-11T21:55:30.150754+0000","last_undegraded":"2024-09-11T21:55:30.150754+0000","last_fullsized":"2024-09-11T21:55:30.150754+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:42:13.180725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.100810+0000","last_change":"2024-09-11T21:55:20.086562+0000","last_active":"2024-09-11T21:55:21.100810+0000","last_peered":"2024-09-11T21:55:21.100810+0000","last_clean":"2024-09-11T21:55:21.100810+0000","last_became_active":"2024-09-11T21:55:20.086197+0000","last_became_peered":"2024-09-11T21:55:20.086197+0000","last_unstale":"2024-09-11T21:55:21.100810+0000","last_undegraded":"2024-09-11T21:55:21.100810+0000","last_fullsized":"2024-09-11T21:55:21.100810+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:23:44.705132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'28","reported_seq":160,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:56:24.233919+0000","last_change":"2024-09-11T21:55:18.028320+0000","last_active":"2024-09-11T21:56:24.233919+0000","last_peered":"2024-09-11T21:56:24.233919+0000","last_clean":"2024-09-11T21:56:24.233919+0000","last_became_active":"2024-09-11T21:55:18.027982+0000","last_became_peered":"2024-09-11T21:55:18.027982+0000","last_unstale":"2024-09-11T21:56:24.233919+0000","last_undegraded":"2024-09-11T21:56:24.233919+0000","last_fullsized":"2024-09-11T21:56:24.233919+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:08:36.750665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59211,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":114,"num_read_kb":114,"num_write":30,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.099277+0000","last_change":"2024-09-11T21:55:20.091695+0000","last_active":"2024-09-11T21:55:21.099277+0000","last_peered":"2024-09-11T21:55:21.099277+0000","last_clean":"2024-09-11T21:55:21.099277+0000","last_became_active":"2024-09-11T21:55:20.091128+0000","last_became_peered":"2024-09-11T21:55:20.091128+0000","last_unstale":"2024-09-11T21:55:21.099277+0000","last_undegraded":"2024-09-11T21:55:21.099277+0000","last_fullsized":"2024-09-11T21:55:21.099277+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:16:20.751952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"84'9","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:32.158923+0000","last_change":"2024-09-11T21:55:18.030827+0000","last_active":"2024-09-11T21:55:32.158923+0000","last_peered":"2024-09-11T21:55:32.158923+0000","last_clean":"2024-09-11T21:55:32.158923+0000","last_became_active":"2024-09-11T21:55:18.030354+0000","last_became_peered":"2024-09-11T21:55:18.030354+0000","last_unstale":"2024-09-11T21:55:32.158923+0000","last_undegraded":"2024-09-11T21:55:32.158923+0000","last_fullsized":"2024-09-11T21:55:32.158923+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:23:19.578820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T21:56:26.374775+0000","last_change":"2024-09-11T21:55:20.090160+0000","last_active":"2024-09-11T21:56:26.374775+0000","last_peered":"2024-09-11T21:56:26.374775+0000","last_clean":"2024-09-11T21:56:26.374775+0000","last_became_active":"2024-09-11T21:55:20.089801+0000","last_became_peered":"2024-09-11T21:55:20.089801+0000","last_unstale":"2024-09-11T21:56:26.374775+0000","last_undegraded":"2024-09-11T21:56:26.374775+0000","last_fullsized":"2024-09-11T21:56:26.374775+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:34:16.093123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"84'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:31.157445+0000","last_change":"2024-09-11T21:55:18.026817+0000","last_active":"2024-09-11T21:55:31.157445+0000","last_peered":"2024-09-11T21:55:31.157445+0000","last_clean":"2024-09-11T21:55:31.157445+0000","last_became_active":"2024-09-11T21:55:18.026188+0000","last_became_peered":"2024-09-11T21:55:18.026188+0000","last_unstale":"2024-09-11T21:55:31.157445+0000","last_undegraded":"2024-09-11T21:55:31.157445+0000","last_fullsized":"2024-09-11T21:55:31.157445+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:55:14.774503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T21:56:25.286390+0000","last_change":"2024-09-11T21:55:20.095613+0000","last_active":"2024-09-11T21:56:25.286390+0000","last_peered":"2024-09-11T21:56:25.286390+0000","last_clean":"2024-09-11T21:56:25.286390+0000","last_became_active":"2024-09-11T21:55:20.095302+0000","last_became_peered":"2024-09-11T21:55:20.095302+0000","last_unstale":"2024-09-11T21:56:25.286390+0000","last_undegraded":"2024-09-11T21:56:25.286390+0000","last_fullsized":"2024-09-11T21:56:25.286390+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:37:27.961213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'5","reported_seq":81,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T21:56:25.286763+0000","last_change":"2024-09-11T21:55:18.032417+0000","last_active":"2024-09-11T21:56:25.286763+0000","last_peered":"2024-09-11T21:56:25.286763+0000","last_clean":"2024-09-11T21:56:25.286763+0000","last_became_active":"2024-09-11T21:55:18.031419+0000","last_became_peered":"2024-09-11T21:55:18.031419+0000","last_unstale":"2024-09-11T21:56:25.286763+0000","last_undegraded":"2024-09-11T21:56:25.286763+0000","last_fullsized":"2024-09-11T21:56:25.286763+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:30:52.285678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":57,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T21:56:25.285877+0000","last_change":"2024-09-11T21:55:20.101965+0000","last_active":"2024-09-11T21:56:25.285877+0000","last_peered":"2024-09-11T21:56:25.285877+0000","last_clean":"2024-09-11T21:56:25.285877+0000","last_became_active":"2024-09-11T21:55:20.101534+0000","last_became_peered":"2024-09-11T21:55:20.101534+0000","last_unstale":"2024-09-11T21:56:25.285877+0000","last_undegraded":"2024-09-11T21:56:25.285877+0000","last_fullsized":"2024-09-11T21:56:25.285877+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:41:30.685121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"84'17","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:31.158074+0000","last_change":"2024-09-11T21:55:18.116059+0000","last_active":"2024-09-11T21:55:31.158074+0000","last_peered":"2024-09-11T21:55:31.158074+0000","last_clean":"2024-09-11T21:55:31.158074+0000","last_became_active":"2024-09-11T21:55:18.115621+0000","last_became_peered":"2024-09-11T21:55:18.115621+0000","last_unstale":"2024-09-11T21:55:31.158074+0000","last_undegraded":"2024-09-11T21:55:31.158074+0000","last_fullsized":"2024-09-11T21:55:31.158074+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:47:17.815695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.086741+0000","last_change":"2024-09-11T21:55:20.100903+0000","last_active":"2024-09-11T21:55:21.086741+0000","last_peered":"2024-09-11T21:55:21.086741+0000","last_clean":"2024-09-11T21:55:21.086741+0000","last_became_active":"2024-09-11T21:55:20.100561+0000","last_became_peered":"2024-09-11T21:55:20.100561+0000","last_unstale":"2024-09-11T21:55:21.086741+0000","last_undegraded":"2024-09-11T21:55:21.086741+0000","last_fullsized":"2024-09-11T21:55:21.086741+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:00:45.472238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'2","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:31.134748+0000","last_change":"2024-09-11T21:55:18.029549+0000","last_active":"2024-09-11T21:55:31.134748+0000","last_peered":"2024-09-11T21:55:31.134748+0000","last_clean":"2024-09-11T21:55:31.134748+0000","last_became_active":"2024-09-11T21:55:18.029178+0000","last_became_peered":"2024-09-11T21:55:18.029178+0000","last_unstale":"2024-09-11T21:55:31.134748+0000","last_undegraded":"2024-09-11T21:55:31.134748+0000","last_fullsized":"2024-09-11T21:55:31.134748+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:37:07.915786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.103401+0000","last_change":"2024-09-11T21:55:20.104202+0000","last_active":"2024-09-11T21:55:21.103401+0000","last_peered":"2024-09-11T21:55:21.103401+0000","last_clean":"2024-09-11T21:55:21.103401+0000","last_became_active":"2024-09-11T21:55:20.103464+0000","last_became_peered":"2024-09-11T21:55:20.103464+0000","last_unstale":"2024-09-11T21:55:21.103401+0000","last_undegraded":"2024-09-11T21:55:21.103401+0000","last_fullsized":"2024-09-11T21:55:21.103401+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:30:16.044143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"84'5","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T21:56:26.374843+0000","last_change":"2024-09-11T21:55:18.026183+0000","last_active":"2024-09-11T21:56:26.374843+0000","last_peered":"2024-09-11T21:56:26.374843+0000","last_clean":"2024-09-11T21:56:26.374843+0000","last_became_active":"2024-09-11T21:55:18.025849+0000","last_became_peered":"2024-09-11T21:55:18.025849+0000","last_unstale":"2024-09-11T21:56:26.374843+0000","last_undegraded":"2024-09-11T21:56:26.374843+0000","last_fullsized":"2024-09-11T21:56:26.374843+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:21:12.189158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T21:56:25.286627+0000","last_change":"2024-09-11T21:55:20.090652+0000","last_active":"2024-09-11T21:56:25.286627+0000","last_peered":"2024-09-11T21:56:25.286627+0000","last_clean":"2024-09-11T21:56:25.286627+0000","last_became_active":"2024-09-11T21:55:20.090348+0000","last_became_peered":"2024-09-11T21:55:20.090348+0000","last_unstale":"2024-09-11T21:56:25.286627+0000","last_undegraded":"2024-09-11T21:56:25.286627+0000","last_fullsized":"2024-09-11T21:56:25.286627+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:10:58.733371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'3","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:31.157673+0000","last_change":"2024-09-11T21:55:18.024388+0000","last_active":"2024-09-11T21:55:31.157673+0000","last_peered":"2024-09-11T21:55:31.157673+0000","last_clean":"2024-09-11T21:55:31.157673+0000","last_became_active":"2024-09-11T21:55:18.024100+0000","last_became_peered":"2024-09-11T21:55:18.024100+0000","last_unstale":"2024-09-11T21:55:31.157673+0000","last_undegraded":"2024-09-11T21:55:31.157673+0000","last_fullsized":"2024-09-11T21:55:31.157673+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:21:11.152282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T21:56:25.286316+0000","last_change":"2024-09-11T21:55:20.101007+0000","last_active":"2024-09-11T21:56:25.286316+0000","last_peered":"2024-09-11T21:56:25.286316+0000","last_clean":"2024-09-11T21:56:25.286316+0000","last_became_active":"2024-09-11T21:55:20.100642+0000","last_became_peered":"2024-09-11T21:55:20.100642+0000","last_unstale":"2024-09-11T21:56:25.286316+0000","last_undegraded":"2024-09-11T21:56:25.286316+0000","last_fullsized":"2024-09-11T21:56:25.286316+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:16:05.642184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"84'6","reported_seq":80,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:56:23.244362+0000","last_change":"2024-09-11T21:55:18.116216+0000","last_active":"2024-09-11T21:56:23.244362+0000","last_peered":"2024-09-11T21:56:23.244362+0000","last_clean":"2024-09-11T21:56:23.244362+0000","last_became_active":"2024-09-11T21:55:18.115871+0000","last_became_peered":"2024-09-11T21:55:18.115871+0000","last_unstale":"2024-09-11T21:56:23.244362+0000","last_undegraded":"2024-09-11T21:56:23.244362+0000","last_fullsized":"2024-09-11T21:56:23.244362+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:16:56.027636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":56,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T21:56:25.286672+0000","last_change":"2024-09-11T21:55:20.092421+0000","last_active":"2024-09-11T21:56:25.286672+0000","last_peered":"2024-09-11T21:56:25.286672+0000","last_clean":"2024-09-11T21:56:25.286672+0000","last_became_active":"2024-09-11T21:55:20.091707+0000","last_became_peered":"2024-09-11T21:55:20.091707+0000","last_unstale":"2024-09-11T21:56:25.286672+0000","last_undegraded":"2024-09-11T21:56:25.286672+0000","last_fullsized":"2024-09-11T21:56:25.286672+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:12:38.157687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":348,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T21:56:25.286163+0000","last_change":"2024-09-11T21:54:07.542362+0000","last_active":"2024-09-11T21:56:25.286163+0000","last_peered":"2024-09-11T21:56:25.286163+0000","last_clean":"2024-09-11T21:56:25.286163+0000","last_became_active":"2024-09-11T21:54:07.541871+0000","last_became_peered":"2024-09-11T21:54:07.541871+0000","last_unstale":"2024-09-11T21:56:25.286163+0000","last_undegraded":"2024-09-11T21:56:25.286163+0000","last_fullsized":"2024-09-11T21:56:25.286163+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:50:41.026895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:50:41.026895+0000","last_clean_scrub_stamp":"2024-09-11T21:50:41.026895+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:05:06.154268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"84'6","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T21:56:25.285854+0000","last_change":"2024-09-11T21:55:18.030558+0000","last_active":"2024-09-11T21:56:25.285854+0000","last_peered":"2024-09-11T21:56:25.285854+0000","last_clean":"2024-09-11T21:56:25.285854+0000","last_became_active":"2024-09-11T21:55:18.030216+0000","last_became_peered":"2024-09-11T21:55:18.030216+0000","last_unstale":"2024-09-11T21:56:25.285854+0000","last_undegraded":"2024-09-11T21:56:25.285854+0000","last_fullsized":"2024-09-11T21:56:25.285854+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:33:02.589999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.097756+0000","last_change":"2024-09-11T21:55:20.092938+0000","last_active":"2024-09-11T21:55:21.097756+0000","last_peered":"2024-09-11T21:55:21.097756+0000","last_clean":"2024-09-11T21:55:21.097756+0000","last_became_active":"2024-09-11T21:55:20.092615+0000","last_became_peered":"2024-09-11T21:55:20.092615+0000","last_unstale":"2024-09-11T21:55:21.097756+0000","last_undegraded":"2024-09-11T21:55:21.097756+0000","last_fullsized":"2024-09-11T21:55:21.097756+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:40:06.957836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"84'4","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T21:56:25.286063+0000","last_change":"2024-09-11T21:55:18.029127+0000","last_active":"2024-09-11T21:56:25.286063+0000","last_peered":"2024-09-11T21:56:25.286063+0000","last_clean":"2024-09-11T21:56:25.286063+0000","last_became_active":"2024-09-11T21:55:18.028740+0000","last_became_peered":"2024-09-11T21:55:18.028740+0000","last_unstale":"2024-09-11T21:56:25.286063+0000","last_undegraded":"2024-09-11T21:56:25.286063+0000","last_fullsized":"2024-09-11T21:56:25.286063+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:14:01.128486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T21:56:26.374460+0000","last_change":"2024-09-11T21:55:20.092498+0000","last_active":"2024-09-11T21:56:26.374460+0000","last_peered":"2024-09-11T21:56:26.374460+0000","last_clean":"2024-09-11T21:56:26.374460+0000","last_became_active":"2024-09-11T21:55:20.092179+0000","last_became_peered":"2024-09-11T21:55:20.092179+0000","last_unstale":"2024-09-11T21:56:26.374460+0000","last_undegraded":"2024-09-11T21:56:26.374460+0000","last_fullsized":"2024-09-11T21:56:26.374460+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:48:58.536942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"84'5","reported_seq":83,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T21:56:26.374567+0000","last_change":"2024-09-11T21:55:18.115798+0000","last_active":"2024-09-11T21:56:26.374567+0000","last_peered":"2024-09-11T21:56:26.374567+0000","last_clean":"2024-09-11T21:56:26.374567+0000","last_became_active":"2024-09-11T21:55:18.115433+0000","last_became_peered":"2024-09-11T21:55:18.115433+0000","last_unstale":"2024-09-11T21:56:26.374567+0000","last_undegraded":"2024-09-11T21:56:26.374567+0000","last_fullsized":"2024-09-11T21:56:26.374567+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:03:32.097756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":56,"num_read_kb":56,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.086704+0000","last_change":"2024-09-11T21:55:20.100439+0000","last_active":"2024-09-11T21:55:21.086704+0000","last_peered":"2024-09-11T21:55:21.086704+0000","last_clean":"2024-09-11T21:55:21.086704+0000","last_became_active":"2024-09-11T21:55:20.098717+0000","last_became_peered":"2024-09-11T21:55:20.098717+0000","last_unstale":"2024-09-11T21:55:21.086704+0000","last_undegraded":"2024-09-11T21:55:21.086704+0000","last_fullsized":"2024-09-11T21:55:21.086704+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:15:10.874225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"84'6","reported_seq":82,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T21:56:25.287039+0000","last_change":"2024-09-11T21:55:18.026588+0000","last_active":"2024-09-11T21:56:25.287039+0000","last_peered":"2024-09-11T21:56:25.287039+0000","last_clean":"2024-09-11T21:56:25.287039+0000","last_became_active":"2024-09-11T21:55:18.026205+0000","last_became_peered":"2024-09-11T21:55:18.026205+0000","last_unstale":"2024-09-11T21:56:25.287039+0000","last_undegraded":"2024-09-11T21:56:25.287039+0000","last_fullsized":"2024-09-11T21:56:25.287039+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:45:34.701088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.086341+0000","last_change":"2024-09-11T21:55:20.100677+0000","last_active":"2024-09-11T21:55:21.086341+0000","last_peered":"2024-09-11T21:55:21.086341+0000","last_clean":"2024-09-11T21:55:21.086341+0000","last_became_active":"2024-09-11T21:55:20.100334+0000","last_became_peered":"2024-09-11T21:55:20.100334+0000","last_unstale":"2024-09-11T21:55:21.086341+0000","last_undegraded":"2024-09-11T21:55:21.086341+0000","last_fullsized":"2024-09-11T21:55:21.086341+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:15:16.792831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"84'1","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.096989+0000","last_change":"2024-09-11T21:55:18.116329+0000","last_active":"2024-09-11T21:55:21.096989+0000","last_peered":"2024-09-11T21:55:21.096989+0000","last_clean":"2024-09-11T21:55:21.096989+0000","last_became_active":"2024-09-11T21:55:18.115944+0000","last_became_peered":"2024-09-11T21:55:18.115944+0000","last_unstale":"2024-09-11T21:55:21.096989+0000","last_undegraded":"2024-09-11T21:55:21.096989+0000","last_fullsized":"2024-09-11T21:55:21.096989+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:03:04.164985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"84'1","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.086593+0000","last_change":"2024-09-11T21:55:20.089202+0000","last_active":"2024-09-11T21:55:21.086593+0000","last_peered":"2024-09-11T21:55:21.086593+0000","last_clean":"2024-09-11T21:55:21.086593+0000","last_became_active":"2024-09-11T21:55:20.088853+0000","last_became_peered":"2024-09-11T21:55:20.088853+0000","last_unstale":"2024-09-11T21:55:21.086593+0000","last_undegraded":"2024-09-11T21:55:21.086593+0000","last_fullsized":"2024-09-11T21:55:21.086593+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:39:36.496944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"84'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:31.157241+0000","last_change":"2024-09-11T21:55:18.117917+0000","last_active":"2024-09-11T21:55:31.157241+0000","last_peered":"2024-09-11T21:55:31.157241+0000","last_clean":"2024-09-11T21:55:31.157241+0000","last_became_active":"2024-09-11T21:55:18.117567+0000","last_became_peered":"2024-09-11T21:55:18.117567+0000","last_unstale":"2024-09-11T21:55:31.157241+0000","last_undegraded":"2024-09-11T21:55:31.157241+0000","last_fullsized":"2024-09-11T21:55:31.157241+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:21:32.546620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-11T21:56:26.375145+0000","last_change":"2024-09-11T21:55:20.092370+0000","last_active":"2024-09-11T21:56:26.375145+0000","last_peered":"2024-09-11T21:56:26.375145+0000","last_clean":"2024-09-11T21:56:26.375145+0000","last_became_active":"2024-09-11T21:55:20.091982+0000","last_became_peered":"2024-09-11T21:55:20.091982+0000","last_unstale":"2024-09-11T21:56:26.375145+0000","last_undegraded":"2024-09-11T21:56:26.375145+0000","last_fullsized":"2024-09-11T21:56:26.375145+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:59:55.214511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"84'14","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:45.098695+0000","last_change":"2024-09-11T21:55:18.023166+0000","last_active":"2024-09-11T21:55:45.098695+0000","last_peered":"2024-09-11T21:55:45.098695+0000","last_clean":"2024-09-11T21:55:45.098695+0000","last_became_active":"2024-09-11T21:55:18.022845+0000","last_became_peered":"2024-09-11T21:55:18.022845+0000","last_unstale":"2024-09-11T21:55:45.098695+0000","last_undegraded":"2024-09-11T21:55:45.098695+0000","last_fullsized":"2024-09-11T21:55:45.098695+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:08:06.770410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.098445+0000","last_change":"2024-09-11T21:55:20.091541+0000","last_active":"2024-09-11T21:55:21.098445+0000","last_peered":"2024-09-11T21:55:21.098445+0000","last_clean":"2024-09-11T21:55:21.098445+0000","last_became_active":"2024-09-11T21:55:20.090971+0000","last_became_peered":"2024-09-11T21:55:20.090971+0000","last_unstale":"2024-09-11T21:55:21.098445+0000","last_undegraded":"2024-09-11T21:55:21.098445+0000","last_fullsized":"2024-09-11T21:55:21.098445+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:22:31.974711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"84'7","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T21:56:25.286984+0000","last_change":"2024-09-11T21:55:18.030694+0000","last_active":"2024-09-11T21:56:25.286984+0000","last_peered":"2024-09-11T21:56:25.286984+0000","last_clean":"2024-09-11T21:56:25.286984+0000","last_became_active":"2024-09-11T21:55:18.029687+0000","last_became_peered":"2024-09-11T21:55:18.029687+0000","last_unstale":"2024-09-11T21:56:25.286984+0000","last_undegraded":"2024-09-11T21:56:25.286984+0000","last_fullsized":"2024-09-11T21:56:25.286984+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:18:15.009543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.100188+0000","last_change":"2024-09-11T21:55:20.100838+0000","last_active":"2024-09-11T21:55:21.100188+0000","last_peered":"2024-09-11T21:55:21.100188+0000","last_clean":"2024-09-11T21:55:21.100188+0000","last_became_active":"2024-09-11T21:55:20.100523+0000","last_became_peered":"2024-09-11T21:55:20.100523+0000","last_unstale":"2024-09-11T21:55:21.100188+0000","last_undegraded":"2024-09-11T21:55:21.100188+0000","last_fullsized":"2024-09-11T21:55:21.100188+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:44:59.052265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"84'2","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:28.131582+0000","last_change":"2024-09-11T21:55:18.115977+0000","last_active":"2024-09-11T21:55:28.131582+0000","last_peered":"2024-09-11T21:55:28.131582+0000","last_clean":"2024-09-11T21:55:28.131582+0000","last_became_active":"2024-09-11T21:55:18.115649+0000","last_became_peered":"2024-09-11T21:55:18.115649+0000","last_unstale":"2024-09-11T21:55:28.131582+0000","last_undegraded":"2024-09-11T21:55:28.131582+0000","last_fullsized":"2024-09-11T21:55:28.131582+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:42:07.621024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.098589+0000","last_change":"2024-09-11T21:55:20.095536+0000","last_active":"2024-09-11T21:55:21.098589+0000","last_peered":"2024-09-11T21:55:21.098589+0000","last_clean":"2024-09-11T21:55:21.098589+0000","last_became_active":"2024-09-11T21:55:20.095117+0000","last_became_peered":"2024-09-11T21:55:20.095117+0000","last_unstale":"2024-09-11T21:55:21.098589+0000","last_undegraded":"2024-09-11T21:55:21.098589+0000","last_fullsized":"2024-09-11T21:55:21.098589+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:46.524922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"84'7","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:31.157374+0000","last_change":"2024-09-11T21:55:18.026592+0000","last_active":"2024-09-11T21:55:31.157374+0000","last_peered":"2024-09-11T21:55:31.157374+0000","last_clean":"2024-09-11T21:55:31.157374+0000","last_became_active":"2024-09-11T21:55:18.026263+0000","last_became_peered":"2024-09-11T21:55:18.026263+0000","last_unstale":"2024-09-11T21:55:31.157374+0000","last_undegraded":"2024-09-11T21:55:31.157374+0000","last_fullsized":"2024-09-11T21:55:31.157374+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:22:29.382964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.103050+0000","last_change":"2024-09-11T21:55:20.093655+0000","last_active":"2024-09-11T21:55:21.103050+0000","last_peered":"2024-09-11T21:55:21.103050+0000","last_clean":"2024-09-11T21:55:21.103050+0000","last_became_active":"2024-09-11T21:55:20.093287+0000","last_became_peered":"2024-09-11T21:55:20.093287+0000","last_unstale":"2024-09-11T21:55:21.103050+0000","last_undegraded":"2024-09-11T21:55:21.103050+0000","last_fullsized":"2024-09-11T21:55:21.103050+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:46:15.912710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"84'3","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:30.150643+0000","last_change":"2024-09-11T21:55:18.028465+0000","last_active":"2024-09-11T21:55:30.150643+0000","last_peered":"2024-09-11T21:55:30.150643+0000","last_clean":"2024-09-11T21:55:30.150643+0000","last_became_active":"2024-09-11T21:55:18.028139+0000","last_became_peered":"2024-09-11T21:55:18.028139+0000","last_unstale":"2024-09-11T21:55:30.150643+0000","last_undegraded":"2024-09-11T21:55:30.150643+0000","last_fullsized":"2024-09-11T21:55:30.150643+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:52:47.808058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.097166+0000","last_change":"2024-09-11T21:55:20.096400+0000","last_active":"2024-09-11T21:55:21.097166+0000","last_peered":"2024-09-11T21:55:21.097166+0000","last_clean":"2024-09-11T21:55:21.097166+0000","last_became_active":"2024-09-11T21:55:20.096023+0000","last_became_peered":"2024-09-11T21:55:20.096023+0000","last_unstale":"2024-09-11T21:55:21.097166+0000","last_undegraded":"2024-09-11T21:55:21.097166+0000","last_fullsized":"2024-09-11T21:55:21.097166+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:19:34.150687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"84'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:28.131668+0000","last_change":"2024-09-11T21:55:18.028759+0000","last_active":"2024-09-11T21:55:28.131668+0000","last_peered":"2024-09-11T21:55:28.131668+0000","last_clean":"2024-09-11T21:55:28.131668+0000","last_became_active":"2024-09-11T21:55:18.028366+0000","last_became_peered":"2024-09-11T21:55:18.028366+0000","last_unstale":"2024-09-11T21:55:28.131668+0000","last_undegraded":"2024-09-11T21:55:28.131668+0000","last_fullsized":"2024-09-11T21:55:28.131668+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:42:05.836114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-11T21:56:25.286949+0000","last_change":"2024-09-11T21:55:20.093593+0000","last_active":"2024-09-11T21:56:25.286949+0000","last_peered":"2024-09-11T21:56:25.286949+0000","last_clean":"2024-09-11T21:56:25.286949+0000","last_became_active":"2024-09-11T21:55:20.093189+0000","last_became_peered":"2024-09-11T21:55:20.093189+0000","last_unstale":"2024-09-11T21:56:25.286949+0000","last_undegraded":"2024-09-11T21:56:25.286949+0000","last_fullsized":"2024-09-11T21:56:25.286949+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:47:36.802816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.086802+0000","last_change":"2024-09-11T21:55:20.100887+0000","last_active":"2024-09-11T21:55:21.086802+0000","last_peered":"2024-09-11T21:55:21.086802+0000","last_clean":"2024-09-11T21:55:21.086802+0000","last_became_active":"2024-09-11T21:55:20.100233+0000","last_became_peered":"2024-09-11T21:55:20.100233+0000","last_unstale":"2024-09-11T21:55:21.086802+0000","last_undegraded":"2024-09-11T21:55:21.086802+0000","last_fullsized":"2024-09-11T21:55:21.086802+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:30:05.820375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"84'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:31.157480+0000","last_change":"2024-09-11T21:55:18.027365+0000","last_active":"2024-09-11T21:55:31.157480+0000","last_peered":"2024-09-11T21:55:31.157480+0000","last_clean":"2024-09-11T21:55:31.157480+0000","last_became_active":"2024-09-11T21:55:18.027058+0000","last_became_peered":"2024-09-11T21:55:18.027058+0000","last_unstale":"2024-09-11T21:55:31.157480+0000","last_undegraded":"2024-09-11T21:55:31.157480+0000","last_fullsized":"2024-09-11T21:55:31.157480+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:30:50.613246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.100428+0000","last_change":"2024-09-11T21:55:20.095154+0000","last_active":"2024-09-11T21:55:21.100428+0000","last_peered":"2024-09-11T21:55:21.100428+0000","last_clean":"2024-09-11T21:55:21.100428+0000","last_became_active":"2024-09-11T21:55:20.093607+0000","last_became_peered":"2024-09-11T21:55:20.093607+0000","last_unstale":"2024-09-11T21:55:21.100428+0000","last_undegraded":"2024-09-11T21:55:21.100428+0000","last_fullsized":"2024-09-11T21:55:21.100428+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:27:53.612467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"84'12","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:32.152672+0000","last_change":"2024-09-11T21:55:18.029872+0000","last_active":"2024-09-11T21:55:32.152672+0000","last_peered":"2024-09-11T21:55:32.152672+0000","last_clean":"2024-09-11T21:55:32.152672+0000","last_became_active":"2024-09-11T21:55:18.029538+0000","last_became_peered":"2024-09-11T21:55:18.029538+0000","last_unstale":"2024-09-11T21:55:32.152672+0000","last_undegraded":"2024-09-11T21:55:32.152672+0000","last_fullsized":"2024-09-11T21:55:32.152672+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:05:16.824805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.100650+0000","last_change":"2024-09-11T21:55:20.098713+0000","last_active":"2024-09-11T21:55:21.100650+0000","last_peered":"2024-09-11T21:55:21.100650+0000","last_clean":"2024-09-11T21:55:21.100650+0000","last_became_active":"2024-09-11T21:55:20.098219+0000","last_became_peered":"2024-09-11T21:55:20.098219+0000","last_unstale":"2024-09-11T21:55:21.100650+0000","last_undegraded":"2024-09-11T21:55:21.100650+0000","last_fullsized":"2024-09-11T21:55:21.100650+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:07:10.585366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"84'2","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.105133+0000","last_change":"2024-09-11T21:55:18.115868+0000","last_active":"2024-09-11T21:55:21.105133+0000","last_peered":"2024-09-11T21:55:21.105133+0000","last_clean":"2024-09-11T21:55:21.105133+0000","last_became_active":"2024-09-11T21:55:18.115486+0000","last_became_peered":"2024-09-11T21:55:18.115486+0000","last_unstale":"2024-09-11T21:55:21.105133+0000","last_undegraded":"2024-09-11T21:55:21.105133+0000","last_fullsized":"2024-09-11T21:55:21.105133+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:16:57.932041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:55:21.099076+0000","last_change":"2024-09-11T21:55:20.096077+0000","last_active":"2024-09-11T21:55:21.099076+0000","last_peered":"2024-09-11T21:55:21.099076+0000","last_clean":"2024-09-11T21:55:21.099076+0000","last_became_active":"2024-09-11T21:55:20.095739+0000","last_became_peered":"2024-09-11T21:55:20.095739+0000","last_unstale":"2024-09-11T21:55:21.099076+0000","last_undegraded":"2024-09-11T21:55:21.099076+0000","last_fullsized":"2024-09-11T21:55:21.099076+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:05:28.089828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"84'13","reported_seq":84,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T21:56:22.230740+0000","last_change":"2024-09-11T21:55:18.025771+0000","last_active":"2024-09-11T21:56:22.230740+0000","last_peered":"2024-09-11T21:56:22.230740+0000","last_clean":"2024-09-11T21:56:22.230740+0000","last_became_active":"2024-09-11T21:55:18.025384+0000","last_became_peered":"2024-09-11T21:55:18.025384+0000","last_unstale":"2024-09-11T21:56:22.230740+0000","last_undegraded":"2024-09-11T21:56:22.230740+0000","last_fullsized":"2024-09-11T21:56:22.230740+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:58:21.070061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":53,"num_read_kb":53,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66812,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":571,"num_read_kb":360,"num_write":241,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":200436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":207,"ondisk_log_size":207,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":283045,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449115,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":913408,"data_stored":436131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678047,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1069056,"data_stored":1085978,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":66,"seq":283467841573,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36260,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708892,"statfs":{"total":95995035648,"available":95957905408,"internally_reserved":0,"allocated":937984,"data_stored":435668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103210,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":933888,"data_stored":436046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332143,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":999424,"data_stored":1037718,"data_compressed":38878,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495668,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":933888,"data_stored":444537,"data_compressed":2882,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757305,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":1028244,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920830,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":933888,"data_stored":444005,"data_compressed":2882,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182468,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":435210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411401,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1003520,"data_stored":496515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36304,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708848,"statfs":{"total":95995035648,"available":95957860352,"internally_reserved":0,"allocated":983040,"data_stored":495902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738451,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":942080,"data_stored":437117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34355,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34355,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34355,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":60947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T21:56:27.802 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-11T21:56:27.802 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-11T21:56:27.802 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-11T21:56:27.802 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph health --format=json 2024-09-11T21:56:28.059 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:56:28.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:28 smithi001 ceph-mon[30935]: pgmap v317: 97 pgs: 97 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-09-11T21:56:28.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:28 smithi001 ceph-mon[30935]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T21:56:28.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:28.562+0000 7f982950b640 1 -- 172.21.15.1:0/3392952446 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9824101420 msgr2=0x7f982410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:28.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:28.562+0000 7f982950b640 1 --2- 172.21.15.1:0/3392952446 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9824101420 0x7f982410d950 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f981c009cd0 tx=0x7f981c02a8e0 comp rx=0 tx=0).stop 2024-09-11T21:56:28.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:28.562+0000 7f982950b640 1 -- 172.21.15.1:0/3392952446 shutdown_connections 2024-09-11T21:56:28.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:28.562+0000 7f982950b640 1 --2- 172.21.15.1:0/3392952446 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9824101420 0x7f982410d950 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:28.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:28.562+0000 7f982950b640 1 --2- 172.21.15.1:0/3392952446 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9824100a80 0x7f9824100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:28.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:28.562+0000 7f982950b640 1 --2- 172.21.15.1:0/3392952446 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9824106a80 0x7f9824106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:28.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:28.562+0000 7f982950b640 1 -- 172.21.15.1:0/3392952446 >> 172.21.15.1:0/3392952446 conn(0x7f98240fc7a0 msgr2=0x7f98240febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:28.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:28.562+0000 7f982950b640 1 -- 172.21.15.1:0/3392952446 shutdown_connections 2024-09-11T21:56:28.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:28.562+0000 7f982950b640 1 -- 172.21.15.1:0/3392952446 wait complete. 2024-09-11T21:56:28.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:28.562+0000 7f982950b640 1 Processor -- start 2024-09-11T21:56:28.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:28.563+0000 7f982950b640 1 -- start start 2024-09-11T21:56:28.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:28.563+0000 7f982950b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9824100a80 0x7f9824198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:28.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:28.563+0000 7f982950b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9824101420 0x7f9824198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:28.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:28.563+0000 7f982950b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9824106a80 0x7f982419ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:28.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:28.563+0000 7f982950b640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9824111fa0 con 0x7f9824106a80 2024-09-11T21:56:28.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:28.563+0000 7f982950b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9824111e20 con 0x7f9824101420 2024-09-11T21:56:28.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:28.563+0000 7f982950b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9824112120 con 0x7f9824100a80 2024-09-11T21:56:28.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:28.563+0000 7f98237fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9824106a80 0x7f982419ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:28.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:28.563+0000 7f98237fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9824106a80 0x7f982419ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45220/0 (socket says 172.21.15.1:45220) 2024-09-11T21:56:28.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:28.563+0000 7f98237fe640 1 -- 172.21.15.1:0/3379670193 learned_addr learned my addr 172.21.15.1:0/3379670193 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:56:28.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:28.563+0000 7f981a7fc640 1 --2- 172.21.15.1:0/3379670193 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9824101420 0x7f9824198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:28.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:28.563+0000 7f9822ffd640 1 --2- 172.21.15.1:0/3379670193 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9824100a80 0x7f9824198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:28.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:28.563+0000 7f98237fe640 1 -- 172.21.15.1:0/3379670193 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9824100a80 msgr2=0x7f9824198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:28.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:28.563+0000 7f98237fe640 1 --2- 172.21.15.1:0/3379670193 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9824100a80 0x7f9824198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:28.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:28.563+0000 7f98237fe640 1 -- 172.21.15.1:0/3379670193 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9824101420 msgr2=0x7f9824198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:28.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:28.563+0000 7f98237fe640 1 --2- 172.21.15.1:0/3379670193 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9824101420 0x7f9824198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:28.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:28.563+0000 7f98237fe640 1 -- 172.21.15.1:0/3379670193 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f982419d3b0 con 0x7f9824106a80 2024-09-11T21:56:28.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:28.563+0000 7f981a7fc640 1 --2- 172.21.15.1:0/3379670193 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9824101420 0x7f9824198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T21:56:28.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:28.563+0000 7f98237fe640 1 --2- 172.21.15.1:0/3379670193 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9824106a80 0x7f982419ccd0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f981c006010 tx=0x7f981c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:28.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:28.564+0000 7f9820ff9640 1 -- 172.21.15.1:0/3379670193 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f981c004450 con 0x7f9824106a80 2024-09-11T21:56:28.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:28.564+0000 7f9820ff9640 1 -- 172.21.15.1:0/3379670193 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f981c003e10 con 0x7f9824106a80 2024-09-11T21:56:28.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:28.564+0000 7f982950b640 1 -- 172.21.15.1:0/3379670193 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f982419d640 con 0x7f9824106a80 2024-09-11T21:56:28.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:28.564+0000 7f9820ff9640 1 -- 172.21.15.1:0/3379670193 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f981c0466e0 con 0x7f9824106a80 2024-09-11T21:56:28.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:28.564+0000 7f982950b640 1 -- 172.21.15.1:0/3379670193 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98241a4f80 con 0x7f9824106a80 2024-09-11T21:56:28.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:28.565+0000 7f982950b640 1 -- 172.21.15.1:0/3379670193 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9824100ee0 con 0x7f9824106a80 2024-09-11T21:56:28.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:28.566+0000 7f9820ff9640 1 -- 172.21.15.1:0/3379670193 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f981c0305a0 con 0x7f9824106a80 2024-09-11T21:56:28.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:28.566+0000 7f9820ff9640 1 --2- 172.21.15.1:0/3379670193 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f98100a2c90 0x7f98100a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:28.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:28.567+0000 7f9820ff9640 1 -- 172.21.15.1:0/3379670193 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f981c02e030 con 0x7f9824106a80 2024-09-11T21:56:28.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:28.567+0000 7f9822ffd640 1 --2- 172.21.15.1:0/3379670193 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f98100a2c90 0x7f98100a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:28.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:28.567+0000 7f9822ffd640 1 --2- 172.21.15.1:0/3379670193 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f98100a2c90 0x7f98100a5150 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f981400a810 tx=0x7f9814008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:28.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:28.570+0000 7f9820ff9640 1 -- 172.21.15.1:0/3379670193 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f981c04b060 con 0x7f9824106a80 2024-09-11T21:56:28.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:28 smithi161 ceph-mon[35615]: pgmap v317: 97 pgs: 97 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-09-11T21:56:28.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:28 smithi161 ceph-mon[35615]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T21:56:28.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:28.785+0000 7f982950b640 1 -- 172.21.15.1:0/3379670193 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f98241a51b0 con 0x7f9824106a80 2024-09-11T21:56:28.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:28.785+0000 7f9820ff9640 1 -- 172.21.15.1:0/3379670193 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f981c0bd730 con 0x7f9824106a80 2024-09-11T21:56:28.786 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:56:28.786 INFO:teuthology.orchestra.run.smithi001.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-11T21:56:28.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:28.787+0000 7f982950b640 1 -- 172.21.15.1:0/3379670193 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f98100a2c90 msgr2=0x7f98100a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:28.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:28.787+0000 7f982950b640 1 --2- 172.21.15.1:0/3379670193 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f98100a2c90 0x7f98100a5150 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f981400a810 tx=0x7f9814008040 comp rx=0 tx=0).stop 2024-09-11T21:56:28.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:28.787+0000 7f982950b640 1 -- 172.21.15.1:0/3379670193 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9824106a80 msgr2=0x7f982419ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:28.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:28.787+0000 7f982950b640 1 --2- 172.21.15.1:0/3379670193 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9824106a80 0x7f982419ccd0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f981c006010 tx=0x7f981c003c20 comp rx=0 tx=0).stop 2024-09-11T21:56:28.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:28.788+0000 7f982950b640 1 -- 172.21.15.1:0/3379670193 shutdown_connections 2024-09-11T21:56:28.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:28.788+0000 7f982950b640 1 --2- 172.21.15.1:0/3379670193 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f98100a2c90 0x7f98100a5150 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:28.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:28.788+0000 7f982950b640 1 --2- 172.21.15.1:0/3379670193 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9824106a80 0x7f982419ccd0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:28.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:28.788+0000 7f982950b640 1 --2- 172.21.15.1:0/3379670193 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9824101420 0x7f9824198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:28.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:28.788+0000 7f982950b640 1 --2- 172.21.15.1:0/3379670193 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9824100a80 0x7f9824198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:28.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:28.788+0000 7f982950b640 1 -- 172.21.15.1:0/3379670193 >> 172.21.15.1:0/3379670193 conn(0x7f98240fc7a0 msgr2=0x7f98240fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:28.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:28.788+0000 7f982950b640 1 -- 172.21.15.1:0/3379670193 shutdown_connections 2024-09-11T21:56:28.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:28.788+0000 7f982950b640 1 -- 172.21.15.1:0/3379670193 wait complete. 2024-09-11T21:56:28.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:28 smithi053 ceph-mon[34908]: pgmap v317: 97 pgs: 97 active+clean; 642 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-09-11T21:56:28.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:28 smithi053 ceph-mon[34908]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T21:56:29.395 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:29 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/3379670193' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-11T21:56:29.427 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-11T21:56:29.427 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T21:56:29.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:29 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/3379670193' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-11T21:56:29.689 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:56:29.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:29 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/3379670193' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-11T21:56:30.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:30.147+0000 7feabbf8a640 1 -- 172.21.15.1:0/3399714416 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feab4101280 msgr2=0x7feab410df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:30.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:30.147+0000 7feabbf8a640 1 --2- 172.21.15.1:0/3399714416 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feab4101280 0x7feab410df40 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7feaa8009900 tx=0x7feaa802a410 comp rx=0 tx=0).stop 2024-09-11T21:56:30.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:30.147+0000 7feabbf8a640 1 -- 172.21.15.1:0/3399714416 shutdown_connections 2024-09-11T21:56:30.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:30.147+0000 7feabbf8a640 1 --2- 172.21.15.1:0/3399714416 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feab410e480 0x7feab4110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:30.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:30.147+0000 7feabbf8a640 1 --2- 172.21.15.1:0/3399714416 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feab4101280 0x7feab410df40 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:30.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:30.147+0000 7feabbf8a640 1 --2- 172.21.15.1:0/3399714416 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feab41008d0 0x7feab4100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:30.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:30.147+0000 7feabbf8a640 1 -- 172.21.15.1:0/3399714416 >> 172.21.15.1:0/3399714416 conn(0x7feab40fc7a0 msgr2=0x7feab40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:30.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:30.147+0000 7feabbf8a640 1 -- 172.21.15.1:0/3399714416 shutdown_connections 2024-09-11T21:56:30.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:30.147+0000 7feabbf8a640 1 -- 172.21.15.1:0/3399714416 wait complete. 2024-09-11T21:56:30.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:30.148+0000 7feabbf8a640 1 Processor -- start 2024-09-11T21:56:30.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:30.148+0000 7feabbf8a640 1 -- start start 2024-09-11T21:56:30.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:30.148+0000 7feabbf8a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feab41008d0 0x7feab41a05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:30.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:30.148+0000 7feabbf8a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feab4101280 0x7feab41a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:30.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:30.148+0000 7feabbf8a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feab410e480 0x7feab419a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:30.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:30.148+0000 7feabbf8a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7feab4114260 con 0x7feab410e480 2024-09-11T21:56:30.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:30.148+0000 7feabbf8a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7feab41140e0 con 0x7feab4101280 2024-09-11T21:56:30.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:30.148+0000 7feabbf8a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7feab41143e0 con 0x7feab41008d0 2024-09-11T21:56:30.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:30.148+0000 7feaba527640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feab410e480 0x7feab419a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:30.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:30.148+0000 7feaba527640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feab410e480 0x7feab419a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45242/0 (socket says 172.21.15.1:45242) 2024-09-11T21:56:30.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:30.148+0000 7feaba527640 1 -- 172.21.15.1:0/3447817983 learned_addr learned my addr 172.21.15.1:0/3447817983 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:56:30.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:30.149+0000 7feab9d26640 1 --2- 172.21.15.1:0/3447817983 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feab41008d0 0x7feab41a05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:30.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:30.149+0000 7feab9525640 1 --2- 172.21.15.1:0/3447817983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feab4101280 0x7feab41a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:30.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:30.149+0000 7feaba527640 1 -- 172.21.15.1:0/3447817983 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feab41008d0 msgr2=0x7feab41a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:30.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:30.149+0000 7feaba527640 1 --2- 172.21.15.1:0/3447817983 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feab41008d0 0x7feab41a05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:30.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:30.149+0000 7feaba527640 1 -- 172.21.15.1:0/3447817983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feab4101280 msgr2=0x7feab41a0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:30.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:30.149+0000 7feaba527640 1 --2- 172.21.15.1:0/3447817983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feab4101280 0x7feab41a0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:30.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:30.149+0000 7feaba527640 1 -- 172.21.15.1:0/3447817983 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feab419aee0 con 0x7feab410e480 2024-09-11T21:56:30.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:30.149+0000 7feab9d26640 1 --2- 172.21.15.1:0/3447817983 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feab41008d0 0x7feab41a05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T21:56:30.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:30.149+0000 7feab9525640 1 --2- 172.21.15.1:0/3447817983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feab4101280 0x7feab41a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T21:56:30.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:30.149+0000 7feaba527640 1 --2- 172.21.15.1:0/3447817983 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feab410e480 0x7feab419a670 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7feaac00b3c0 tx=0x7feaac00b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:30.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:30.149+0000 7feab2ffd640 1 -- 172.21.15.1:0/3447817983 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaac011830 con 0x7feab410e480 2024-09-11T21:56:30.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:30.149+0000 7feab2ffd640 1 -- 172.21.15.1:0/3447817983 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feaac016070 con 0x7feab410e480 2024-09-11T21:56:30.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:30.149+0000 7feabbf8a640 1 -- 172.21.15.1:0/3447817983 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feab419b1d0 con 0x7feab410e480 2024-09-11T21:56:30.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:30.149+0000 7feab2ffd640 1 -- 172.21.15.1:0/3447817983 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaac00f2e0 con 0x7feab410e480 2024-09-11T21:56:30.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:30.149+0000 7feabbf8a640 1 -- 172.21.15.1:0/3447817983 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feab41a7350 con 0x7feab410e480 2024-09-11T21:56:30.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:30.150+0000 7feabbf8a640 1 -- 172.21.15.1:0/3447817983 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feab406b070 con 0x7feab410e480 2024-09-11T21:56:30.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:30.151+0000 7feab2ffd640 1 -- 172.21.15.1:0/3447817983 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7feaac0119d0 con 0x7feab410e480 2024-09-11T21:56:30.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:30.152+0000 7feab2ffd640 1 --2- 172.21.15.1:0/3447817983 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fea9c0a2c10 0x7fea9c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:30.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:30.152+0000 7feab9d26640 1 --2- 172.21.15.1:0/3447817983 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fea9c0a2c10 0x7fea9c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:30.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:30.152+0000 7feab2ffd640 1 -- 172.21.15.1:0/3447817983 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7feaac0cee90 con 0x7feab410e480 2024-09-11T21:56:30.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:30.153+0000 7feab9d26640 1 --2- 172.21.15.1:0/3447817983 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fea9c0a2c10 0x7fea9c0a50d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7feaa4007eb0 tx=0x7feaa40073c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:30.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:30.155+0000 7feab2ffd640 1 -- 172.21.15.1:0/3447817983 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7feaac09bd20 con 0x7feab410e480 2024-09-11T21:56:30.369 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:30 smithi001 ceph-mon[30935]: pgmap v318: 97 pgs: 97 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-09-11T21:56:30.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:30.367+0000 7feabbf8a640 1 -- 172.21.15.1:0/3447817983 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feab402d7c0 con 0x7feab410e480 2024-09-11T21:56:30.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:30.367+0000 7feab2ffd640 1 -- 172.21.15.1:0/3447817983 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11949 (secure 0 0 0) 0x7feaac09b620 con 0x7feab410e480 2024-09-11T21:56:30.370 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:56:30.370 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":27,"btime":"2024-09-11T21:55:32:152241+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/421542832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":421542832},{"type":"v1","addr":"172.21.15.1:6835","nonce":421542832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2836864406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2836864406},{"type":"v1","addr":"172.21.15.1:6837","nonce":2836864406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T21:55:20.791263+0000","modified":"2024-09-11T21:55:32.152227+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24484,"mds_2":24467,"mds_3":24466,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14595":{"gid":14595,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/407834672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":407834672},{"type":"v1","addr":"172.21.15.1:6839","nonce":407834672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/1068827678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":1068827678},{"type":"v1","addr":"172.21.15.1:6841","nonce":1068827678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/872410024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":872410024},{"type":"v1","addr":"172.21.15.161:6833","nonce":872410024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/1648944570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1648944570},{"type":"v1","addr":"172.21.15.53:6833","nonce":1648944570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/2126933993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2126933993},{"type":"v1","addr":"172.21.15.53:6835","nonce":2126933993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.161:6835/828381592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":828381592},{"type":"v1","addr":"172.21.15.161:6835","nonce":828381592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.161:6837/1799318656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1799318656},{"type":"v1","addr":"172.21.15.161:6837","nonce":1799318656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.53:6837/3457131830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3457131830},{"type":"v1","addr":"172.21.15.53:6837","nonce":3457131830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/455365383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":455365383},{"type":"v1","addr":"172.21.15.161:6839","nonce":455365383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.53:6839/1378584447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1378584447},{"type":"v1","addr":"172.21.15.53:6839","nonce":1378584447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24484,24467,24466,24460]},"id":1}]} 2024-09-11T21:56:30.370 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 27 2024-09-11T21:56:30.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:30.370+0000 7feabbf8a640 1 -- 172.21.15.1:0/3447817983 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fea9c0a2c10 msgr2=0x7fea9c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:30.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:30.370+0000 7feabbf8a640 1 --2- 172.21.15.1:0/3447817983 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fea9c0a2c10 0x7fea9c0a50d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7feaa4007eb0 tx=0x7feaa40073c0 comp rx=0 tx=0).stop 2024-09-11T21:56:30.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:30.370+0000 7feabbf8a640 1 -- 172.21.15.1:0/3447817983 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feab410e480 msgr2=0x7feab419a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:30.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:30.370+0000 7feabbf8a640 1 --2- 172.21.15.1:0/3447817983 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feab410e480 0x7feab419a670 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7feaac00b3c0 tx=0x7feaac00b880 comp rx=0 tx=0).stop 2024-09-11T21:56:30.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:30.370+0000 7feabbf8a640 1 -- 172.21.15.1:0/3447817983 shutdown_connections 2024-09-11T21:56:30.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:30.370+0000 7feabbf8a640 1 --2- 172.21.15.1:0/3447817983 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fea9c0a2c10 0x7fea9c0a50d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:30.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:30.370+0000 7feabbf8a640 1 --2- 172.21.15.1:0/3447817983 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feab410e480 0x7feab419a670 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:30.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:30.370+0000 7feabbf8a640 1 --2- 172.21.15.1:0/3447817983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feab4101280 0x7feab41a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:30.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:30.370+0000 7feabbf8a640 1 --2- 172.21.15.1:0/3447817983 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feab41008d0 0x7feab41a05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:30.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:30.370+0000 7feabbf8a640 1 -- 172.21.15.1:0/3447817983 >> 172.21.15.1:0/3447817983 conn(0x7feab40fc7a0 msgr2=0x7feab40feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:30.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:30.370+0000 7feabbf8a640 1 -- 172.21.15.1:0/3447817983 shutdown_connections 2024-09-11T21:56:30.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:30.370+0000 7feabbf8a640 1 -- 172.21.15.1:0/3447817983 wait complete. 2024-09-11T21:56:30.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:30 smithi161 ceph-mon[35615]: pgmap v318: 97 pgs: 97 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-09-11T21:56:30.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:30 smithi053 ceph-mon[34908]: pgmap v318: 97 pgs: 97 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-09-11T21:56:31.020 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T21:56:31.278 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:56:31.569 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:31 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/3447817983' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:56:31.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:31 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/3447817983' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:56:31.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:31.747+0000 7ffbb6313640 1 -- 172.21.15.1:0/2885525030 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffbb0102570 msgr2=0x7ffbb01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:31.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:31.747+0000 7ffbb6313640 1 --2- 172.21.15.1:0/2885525030 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffbb0102570 0x7ffbb01029b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ffb98009980 tx=0x7ffb9802c6c0 comp rx=0 tx=0).stop 2024-09-11T21:56:31.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:31.747+0000 7ffbb6313640 1 -- 172.21.15.1:0/2885525030 shutdown_connections 2024-09-11T21:56:31.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:31.747+0000 7ffbb6313640 1 --2- 172.21.15.1:0/2885525030 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffbb0102ef0 0x7ffbb010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:31.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:31.747+0000 7ffbb6313640 1 --2- 172.21.15.1:0/2885525030 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffbb0102570 0x7ffbb01029b0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:31.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:31.747+0000 7ffbb6313640 1 --2- 172.21.15.1:0/2885525030 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffbb0100e20 0x7ffbb0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:31.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:31.747+0000 7ffbb6313640 1 -- 172.21.15.1:0/2885525030 >> 172.21.15.1:0/2885525030 conn(0x7ffbb00fc7d0 msgr2=0x7ffbb00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:31.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:31.747+0000 7ffbb6313640 1 -- 172.21.15.1:0/2885525030 shutdown_connections 2024-09-11T21:56:31.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:31.747+0000 7ffbb6313640 1 -- 172.21.15.1:0/2885525030 wait complete. 2024-09-11T21:56:31.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:31.747+0000 7ffbb6313640 1 Processor -- start 2024-09-11T21:56:31.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:31.747+0000 7ffbb6313640 1 -- start start 2024-09-11T21:56:31.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:31.747+0000 7ffbb6313640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffbb0100e20 0x7ffbb0196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:31.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:31.747+0000 7ffbb6313640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffbb0102570 0x7ffbb01967c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:31.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:31.747+0000 7ffbb6313640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffbb0102ef0 0x7ffbb019ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:31.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:31.747+0000 7ffbb6313640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb010fec0 con 0x7ffbb0102570 2024-09-11T21:56:31.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:31.747+0000 7ffbb6313640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb010fd40 con 0x7ffbb0100e20 2024-09-11T21:56:31.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:31.747+0000 7ffbb6313640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb0110040 con 0x7ffbb0102ef0 2024-09-11T21:56:31.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:31.748+0000 7ffba7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffbb0102570 0x7ffbb01967c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:31.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:31.748+0000 7ffba7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffbb0102570 0x7ffbb01967c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45274/0 (socket says 172.21.15.1:45274) 2024-09-11T21:56:31.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:31.748+0000 7ffba7fff640 1 -- 172.21.15.1:0/3711210273 learned_addr learned my addr 172.21.15.1:0/3711210273 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:56:31.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:31.748+0000 7ffbb48b0640 1 --2- 172.21.15.1:0/3711210273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffbb0102ef0 0x7ffbb019ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:31.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:31.748+0000 7ffbaffff640 1 --2- 172.21.15.1:0/3711210273 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffbb0100e20 0x7ffbb0196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:31.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:31.748+0000 7ffba7fff640 1 -- 172.21.15.1:0/3711210273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffbb0102ef0 msgr2=0x7ffbb019ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:31.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:31.748+0000 7ffba7fff640 1 --2- 172.21.15.1:0/3711210273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffbb0102ef0 0x7ffbb019ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:31.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:31.748+0000 7ffba7fff640 1 -- 172.21.15.1:0/3711210273 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffbb0100e20 msgr2=0x7ffbb0196280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:31.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:31.748+0000 7ffba7fff640 1 --2- 172.21.15.1:0/3711210273 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffbb0100e20 0x7ffbb0196280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:31.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:31.748+0000 7ffba7fff640 1 -- 172.21.15.1:0/3711210273 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbb019b230 con 0x7ffbb0102570 2024-09-11T21:56:31.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:31.748+0000 7ffbaffff640 1 --2- 172.21.15.1:0/3711210273 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffbb0100e20 0x7ffbb0196280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T21:56:31.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:31.748+0000 7ffba7fff640 1 --2- 172.21.15.1:0/3711210273 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffbb0102570 0x7ffbb01967c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ffb9802ccb0 tx=0x7ffb98003b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:31.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:31.748+0000 7ffbadffb640 1 -- 172.21.15.1:0/3711210273 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb98041070 con 0x7ffbb0102570 2024-09-11T21:56:31.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:31.748+0000 7ffbadffb640 1 -- 172.21.15.1:0/3711210273 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb98033c20 con 0x7ffbb0102570 2024-09-11T21:56:31.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:31.748+0000 7ffbb6313640 1 -- 172.21.15.1:0/3711210273 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbb019b4c0 con 0x7ffbb0102570 2024-09-11T21:56:31.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:31.748+0000 7ffbadffb640 1 -- 172.21.15.1:0/3711210273 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb98037030 con 0x7ffbb0102570 2024-09-11T21:56:31.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:31.748+0000 7ffbb6313640 1 -- 172.21.15.1:0/3711210273 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbb01a2e00 con 0x7ffbb0102570 2024-09-11T21:56:31.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:31.749+0000 7ffbb6313640 1 -- 172.21.15.1:0/3711210273 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbb0101200 con 0x7ffbb0102570 2024-09-11T21:56:31.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:31.750+0000 7ffbadffb640 1 -- 172.21.15.1:0/3711210273 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7ffb98044070 con 0x7ffbb0102570 2024-09-11T21:56:31.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:31.751+0000 7ffbadffb640 1 --2- 172.21.15.1:0/3711210273 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7ffb9c0a2c90 0x7ffb9c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:31.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:31.751+0000 7ffbaffff640 1 --2- 172.21.15.1:0/3711210273 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7ffb9c0a2c90 0x7ffb9c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:31.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:31.751+0000 7ffbadffb640 1 -- 172.21.15.1:0/3711210273 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7ffb980f18d0 con 0x7ffbb0102570 2024-09-11T21:56:31.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:31.752+0000 7ffbaffff640 1 --2- 172.21.15.1:0/3711210273 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7ffb9c0a2c90 0x7ffb9c0a5150 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ffba0009b40 tx=0x7ffba0005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:31.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:31.755+0000 7ffbadffb640 1 -- 172.21.15.1:0/3711210273 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffb980be6b0 con 0x7ffbb0102570 2024-09-11T21:56:31.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:31 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/3447817983' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:56:31.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:31.967+0000 7ffbb6313640 1 -- 172.21.15.1:0/3711210273 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffbb01a2fd0 con 0x7ffbb0102570 2024-09-11T21:56:31.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:31.968+0000 7ffbadffb640 1 -- 172.21.15.1:0/3711210273 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11949 (secure 0 0 0) 0x7ffb980bdfb0 con 0x7ffbb0102570 2024-09-11T21:56:31.970 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:56:31.970 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":27,"btime":"2024-09-11T21:55:32:152241+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/421542832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":421542832},{"type":"v1","addr":"172.21.15.1:6835","nonce":421542832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2836864406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2836864406},{"type":"v1","addr":"172.21.15.1:6837","nonce":2836864406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T21:55:20.791263+0000","modified":"2024-09-11T21:55:32.152227+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24484,"mds_2":24467,"mds_3":24466,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14595":{"gid":14595,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/407834672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":407834672},{"type":"v1","addr":"172.21.15.1:6839","nonce":407834672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/1068827678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":1068827678},{"type":"v1","addr":"172.21.15.1:6841","nonce":1068827678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/872410024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":872410024},{"type":"v1","addr":"172.21.15.161:6833","nonce":872410024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/1648944570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1648944570},{"type":"v1","addr":"172.21.15.53:6833","nonce":1648944570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/2126933993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2126933993},{"type":"v1","addr":"172.21.15.53:6835","nonce":2126933993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.161:6835/828381592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":828381592},{"type":"v1","addr":"172.21.15.161:6835","nonce":828381592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.161:6837/1799318656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1799318656},{"type":"v1","addr":"172.21.15.161:6837","nonce":1799318656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.53:6837/3457131830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3457131830},{"type":"v1","addr":"172.21.15.53:6837","nonce":3457131830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/455365383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":455365383},{"type":"v1","addr":"172.21.15.161:6839","nonce":455365383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.53:6839/1378584447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1378584447},{"type":"v1","addr":"172.21.15.53:6839","nonce":1378584447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24484,24467,24466,24460]},"id":1}]} 2024-09-11T21:56:31.970 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 27 2024-09-11T21:56:31.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:31.970+0000 7ffbb6313640 1 -- 172.21.15.1:0/3711210273 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7ffb9c0a2c90 msgr2=0x7ffb9c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:31.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:31.970+0000 7ffbb6313640 1 --2- 172.21.15.1:0/3711210273 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7ffb9c0a2c90 0x7ffb9c0a5150 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ffba0009b40 tx=0x7ffba0005e50 comp rx=0 tx=0).stop 2024-09-11T21:56:31.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:31.970+0000 7ffbb6313640 1 -- 172.21.15.1:0/3711210273 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffbb0102570 msgr2=0x7ffbb01967c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:31.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:31.970+0000 7ffbb6313640 1 --2- 172.21.15.1:0/3711210273 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffbb0102570 0x7ffbb01967c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ffb9802ccb0 tx=0x7ffb98003b60 comp rx=0 tx=0).stop 2024-09-11T21:56:31.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:31.970+0000 7ffbb6313640 1 -- 172.21.15.1:0/3711210273 shutdown_connections 2024-09-11T21:56:31.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:31.970+0000 7ffbb6313640 1 --2- 172.21.15.1:0/3711210273 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7ffb9c0a2c90 0x7ffb9c0a5150 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:31.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:31.970+0000 7ffbb6313640 1 --2- 172.21.15.1:0/3711210273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffbb0102ef0 0x7ffbb019ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:31.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:31.970+0000 7ffbb6313640 1 --2- 172.21.15.1:0/3711210273 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffbb0102570 0x7ffbb01967c0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:31.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:31.970+0000 7ffbb6313640 1 --2- 172.21.15.1:0/3711210273 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffbb0100e20 0x7ffbb0196280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:31.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:31.970+0000 7ffbb6313640 1 -- 172.21.15.1:0/3711210273 >> 172.21.15.1:0/3711210273 conn(0x7ffbb00fc7d0 msgr2=0x7ffbb0106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:31.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:31.970+0000 7ffbb6313640 1 -- 172.21.15.1:0/3711210273 shutdown_connections 2024-09-11T21:56:31.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:31.970+0000 7ffbb6313640 1 -- 172.21.15.1:0/3711210273 wait complete. 2024-09-11T21:56:32.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:32 smithi161 ceph-mon[35615]: pgmap v319: 97 pgs: 97 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 8 op/s; 7.2 KiB/s, 0 objects/s recovering 2024-09-11T21:56:32.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:32 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/3711210273' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:56:32.614 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:32 smithi001 ceph-mon[30935]: pgmap v319: 97 pgs: 97 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 8 op/s; 7.2 KiB/s, 0 objects/s recovering 2024-09-11T21:56:32.614 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:32 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/3711210273' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:56:32.646 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T21:56:32.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:32 smithi053 ceph-mon[34908]: pgmap v319: 97 pgs: 97 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 8 op/s; 7.2 KiB/s, 0 objects/s recovering 2024-09-11T21:56:32.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:32 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/3711210273' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:56:32.904 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:56:33.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:33.374+0000 7f8c1b7dd640 1 -- 172.21.15.1:0/3042323609 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8c14100a80 msgr2=0x7f8c14100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:33.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:33.374+0000 7f8c1b7dd640 1 --2- 172.21.15.1:0/3042323609 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8c14100a80 0x7f8c14100ee0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8c08009930 tx=0x7f8c0802a420 comp rx=0 tx=0).stop 2024-09-11T21:56:33.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:33.374+0000 7f8c1b7dd640 1 -- 172.21.15.1:0/3042323609 shutdown_connections 2024-09-11T21:56:33.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:33.374+0000 7f8c1b7dd640 1 --2- 172.21.15.1:0/3042323609 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c14101420 0x7f8c1410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:33.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:33.374+0000 7f8c1b7dd640 1 --2- 172.21.15.1:0/3042323609 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8c14100a80 0x7f8c14100ee0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:33.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:33.374+0000 7f8c1b7dd640 1 --2- 172.21.15.1:0/3042323609 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c14106a80 0x7f8c14106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:33.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:33.374+0000 7f8c1b7dd640 1 -- 172.21.15.1:0/3042323609 >> 172.21.15.1:0/3042323609 conn(0x7f8c140fc7a0 msgr2=0x7f8c140febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:33.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:33.374+0000 7f8c1b7dd640 1 -- 172.21.15.1:0/3042323609 shutdown_connections 2024-09-11T21:56:33.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:33.374+0000 7f8c1b7dd640 1 -- 172.21.15.1:0/3042323609 wait complete. 2024-09-11T21:56:33.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:33.375+0000 7f8c1b7dd640 1 Processor -- start 2024-09-11T21:56:33.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:33.375+0000 7f8c1b7dd640 1 -- start start 2024-09-11T21:56:33.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:33.375+0000 7f8c1b7dd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c14100a80 0x7f8c14102650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:33.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:33.375+0000 7f8c1b7dd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8c14101420 0x7f8c14102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:33.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:33.375+0000 7f8c1b7dd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c14106a80 0x7f8c141030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:33.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:33.375+0000 7f8c1b7dd640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8c14110ed0 con 0x7f8c14101420 2024-09-11T21:56:33.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:33.375+0000 7f8c1b7dd640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8c14110d50 con 0x7f8c14100a80 2024-09-11T21:56:33.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:33.375+0000 7f8c1b7dd640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8c14111050 con 0x7f8c14106a80 2024-09-11T21:56:33.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:33.375+0000 7f8c18d78640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8c14101420 0x7f8c14102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:33.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:33.375+0000 7f8c18d78640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8c14101420 0x7f8c14102b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45290/0 (socket says 172.21.15.1:45290) 2024-09-11T21:56:33.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:33.375+0000 7f8c18d78640 1 -- 172.21.15.1:0/1895044313 learned_addr learned my addr 172.21.15.1:0/1895044313 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:56:33.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:33.375+0000 7f8c19d7a640 1 --2- 172.21.15.1:0/1895044313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c14106a80 0x7f8c141030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:33.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:33.376+0000 7f8c19579640 1 --2- 172.21.15.1:0/1895044313 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c14100a80 0x7f8c14102650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:33.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:33.376+0000 7f8c18d78640 1 -- 172.21.15.1:0/1895044313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c14106a80 msgr2=0x7f8c141030d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:33.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:33.376+0000 7f8c18d78640 1 --2- 172.21.15.1:0/1895044313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c14106a80 0x7f8c141030d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:33.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:33.376+0000 7f8c18d78640 1 -- 172.21.15.1:0/1895044313 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c14100a80 msgr2=0x7f8c14102650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:33.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:33.376+0000 7f8c18d78640 1 --2- 172.21.15.1:0/1895044313 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c14100a80 0x7f8c14102650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:33.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:33.376+0000 7f8c18d78640 1 -- 172.21.15.1:0/1895044313 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c141a68b0 con 0x7f8c14101420 2024-09-11T21:56:33.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:33.376+0000 7f8c18d78640 1 --2- 172.21.15.1:0/1895044313 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8c14101420 0x7f8c14102b90 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f8c0802a940 tx=0x7f8c08003c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:33.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:33.376+0000 7f8c127fc640 1 -- 172.21.15.1:0/1895044313 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c0803f070 con 0x7f8c14101420 2024-09-11T21:56:33.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:33.376+0000 7f8c127fc640 1 -- 172.21.15.1:0/1895044313 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c0803b460 con 0x7f8c14101420 2024-09-11T21:56:33.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:33.376+0000 7f8c1b7dd640 1 -- 172.21.15.1:0/1895044313 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c141a6b40 con 0x7f8c14101420 2024-09-11T21:56:33.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:33.376+0000 7f8c127fc640 1 -- 172.21.15.1:0/1895044313 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c08035030 con 0x7f8c14101420 2024-09-11T21:56:33.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:33.376+0000 7f8c1b7dd640 1 -- 172.21.15.1:0/1895044313 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c141a7070 con 0x7f8c14101420 2024-09-11T21:56:33.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:33.377+0000 7f8c1b7dd640 1 -- 172.21.15.1:0/1895044313 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c14100ee0 con 0x7f8c14101420 2024-09-11T21:56:33.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:33.378+0000 7f8c127fc640 1 -- 172.21.15.1:0/1895044313 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f8c08042070 con 0x7f8c14101420 2024-09-11T21:56:33.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:33.379+0000 7f8c127fc640 1 --2- 172.21.15.1:0/1895044313 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f8bfc0a2c90 0x7f8bfc0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:33.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:33.379+0000 7f8c19579640 1 --2- 172.21.15.1:0/1895044313 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f8bfc0a2c90 0x7f8bfc0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:33.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:33.380+0000 7f8c127fc640 1 -- 172.21.15.1:0/1895044313 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f8c080efe40 con 0x7f8c14101420 2024-09-11T21:56:33.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:33.380+0000 7f8c19579640 1 --2- 172.21.15.1:0/1895044313 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f8bfc0a2c90 0x7f8bfc0a5150 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8c0400a300 tx=0x7f8c0400a290 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:33.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:33.383+0000 7f8c127fc640 1 -- 172.21.15.1:0/1895044313 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8c080bcbc0 con 0x7f8c14101420 2024-09-11T21:56:33.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:33.559+0000 7f8c1b7dd640 1 -- 172.21.15.1:0/1895044313 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8c14103b80 con 0x7f8c14101420 2024-09-11T21:56:33.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:33.560+0000 7f8c127fc640 1 -- 172.21.15.1:0/1895044313 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22669 (secure 0 0 0) 0x7f8c14103b80 con 0x7f8c14101420 2024-09-11T21:56:33.562 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:56:33.562 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":86,"fsid":"3ffca83c-7087-11ef-bcea-c7b262605968","created":"2024-09-11T21:47:28.868707+0000","modified":"2024-09-11T21:56:26.278614+0000","last_up_change":"2024-09-11T21:54:29.319441+0000","last_in_change":"2024-09-11T21:54:05.532281+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T21:50:41.026490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T21:55:16.961999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T21:55:19.066228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"19e56504-18f1-4234-b2d6-f193ff47d8eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":4012142023},{"type":"v1","addr":"172.21.15.1:6803","nonce":4012142023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":4012142023},{"type":"v1","addr":"172.21.15.1:6805","nonce":4012142023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":4012142023},{"type":"v1","addr":"172.21.15.1:6809","nonce":4012142023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":4012142023},{"type":"v1","addr":"172.21.15.1:6807","nonce":4012142023}]},"public_addr":"172.21.15.1:6803/4012142023","cluster_addr":"172.21.15.1:6805/4012142023","heartbeat_back_addr":"172.21.15.1:6809/4012142023","heartbeat_front_addr":"172.21.15.1:6807/4012142023","state":["exists","up"]},{"osd":1,"uuid":"996c9f0c-c5a4-43b8-b8e0-d712f27c5dfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3039391924},{"type":"v1","addr":"172.21.15.53:6801","nonce":3039391924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3039391924},{"type":"v1","addr":"172.21.15.53:6803","nonce":3039391924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3039391924},{"type":"v1","addr":"172.21.15.53:6807","nonce":3039391924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3039391924},{"type":"v1","addr":"172.21.15.53:6805","nonce":3039391924}]},"public_addr":"172.21.15.53:6801/3039391924","cluster_addr":"172.21.15.53:6803/3039391924","heartbeat_back_addr":"172.21.15.53:6807/3039391924","heartbeat_front_addr":"172.21.15.53:6805/3039391924","state":["exists","up"]},{"osd":2,"uuid":"578d4260-41e1-4645-b9b1-1c5445712c13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1707914885},{"type":"v1","addr":"172.21.15.161:6801","nonce":1707914885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1707914885},{"type":"v1","addr":"172.21.15.161:6803","nonce":1707914885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1707914885},{"type":"v1","addr":"172.21.15.161:6807","nonce":1707914885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1707914885},{"type":"v1","addr":"172.21.15.161:6805","nonce":1707914885}]},"public_addr":"172.21.15.161:6801/1707914885","cluster_addr":"172.21.15.161:6803/1707914885","heartbeat_back_addr":"172.21.15.161:6807/1707914885","heartbeat_front_addr":"172.21.15.161:6805/1707914885","state":["exists","up"]},{"osd":3,"uuid":"73f78766-9662-4aa3-841c-fe2ce654ee9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1541142398},{"type":"v1","addr":"172.21.15.1:6811","nonce":1541142398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1541142398},{"type":"v1","addr":"172.21.15.1:6813","nonce":1541142398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1541142398},{"type":"v1","addr":"172.21.15.1:6817","nonce":1541142398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1541142398},{"type":"v1","addr":"172.21.15.1:6815","nonce":1541142398}]},"public_addr":"172.21.15.1:6811/1541142398","cluster_addr":"172.21.15.1:6813/1541142398","heartbeat_back_addr":"172.21.15.1:6817/1541142398","heartbeat_front_addr":"172.21.15.1:6815/1541142398","state":["exists","up"]},{"osd":4,"uuid":"56c808bb-d4cd-4969-b49e-adef49ecd0f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":3081402681},{"type":"v1","addr":"172.21.15.53:6809","nonce":3081402681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3081402681},{"type":"v1","addr":"172.21.15.53:6811","nonce":3081402681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3081402681},{"type":"v1","addr":"172.21.15.53:6815","nonce":3081402681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3081402681},{"type":"v1","addr":"172.21.15.53:6813","nonce":3081402681}]},"public_addr":"172.21.15.53:6809/3081402681","cluster_addr":"172.21.15.53:6811/3081402681","heartbeat_back_addr":"172.21.15.53:6815/3081402681","heartbeat_front_addr":"172.21.15.53:6813/3081402681","state":["exists","up"]},{"osd":5,"uuid":"7b8266d6-b507-47c9-b4fa-ff16c4a74d35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":1622081327},{"type":"v1","addr":"172.21.15.161:6809","nonce":1622081327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":1622081327},{"type":"v1","addr":"172.21.15.161:6811","nonce":1622081327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":1622081327},{"type":"v1","addr":"172.21.15.161:6815","nonce":1622081327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":1622081327},{"type":"v1","addr":"172.21.15.161:6813","nonce":1622081327}]},"public_addr":"172.21.15.161:6809/1622081327","cluster_addr":"172.21.15.161:6811/1622081327","heartbeat_back_addr":"172.21.15.161:6815/1622081327","heartbeat_front_addr":"172.21.15.161:6813/1622081327","state":["exists","up"]},{"osd":6,"uuid":"3af5925d-a749-4849-a076-80580222cad4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":158434853},{"type":"v1","addr":"172.21.15.1:6819","nonce":158434853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":158434853},{"type":"v1","addr":"172.21.15.1:6821","nonce":158434853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":158434853},{"type":"v1","addr":"172.21.15.1:6825","nonce":158434853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":158434853},{"type":"v1","addr":"172.21.15.1:6823","nonce":158434853}]},"public_addr":"172.21.15.1:6819/158434853","cluster_addr":"172.21.15.1:6821/158434853","heartbeat_back_addr":"172.21.15.1:6825/158434853","heartbeat_front_addr":"172.21.15.1:6823/158434853","state":["exists","up"]},{"osd":7,"uuid":"f8f26c3c-17e9-4e89-8392-289796e0c8a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2132057697},{"type":"v1","addr":"172.21.15.53:6817","nonce":2132057697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2132057697},{"type":"v1","addr":"172.21.15.53:6819","nonce":2132057697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2132057697},{"type":"v1","addr":"172.21.15.53:6823","nonce":2132057697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2132057697},{"type":"v1","addr":"172.21.15.53:6821","nonce":2132057697}]},"public_addr":"172.21.15.53:6817/2132057697","cluster_addr":"172.21.15.53:6819/2132057697","heartbeat_back_addr":"172.21.15.53:6823/2132057697","heartbeat_front_addr":"172.21.15.53:6821/2132057697","state":["exists","up"]},{"osd":8,"uuid":"36ea401b-3c93-4b9d-8880-298d1eb6e30b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1838248669},{"type":"v1","addr":"172.21.15.161:6817","nonce":1838248669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1838248669},{"type":"v1","addr":"172.21.15.161:6819","nonce":1838248669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1838248669},{"type":"v1","addr":"172.21.15.161:6823","nonce":1838248669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1838248669},{"type":"v1","addr":"172.21.15.161:6821","nonce":1838248669}]},"public_addr":"172.21.15.161:6817/1838248669","cluster_addr":"172.21.15.161:6819/1838248669","heartbeat_back_addr":"172.21.15.161:6823/1838248669","heartbeat_front_addr":"172.21.15.161:6821/1838248669","state":["exists","up"]},{"osd":9,"uuid":"4b788d3f-a3bc-4463-bd99-95fdb7a52768","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2806274589},{"type":"v1","addr":"172.21.15.1:6827","nonce":2806274589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2806274589},{"type":"v1","addr":"172.21.15.1:6829","nonce":2806274589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":2806274589},{"type":"v1","addr":"172.21.15.1:6833","nonce":2806274589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2806274589},{"type":"v1","addr":"172.21.15.1:6831","nonce":2806274589}]},"public_addr":"172.21.15.1:6827/2806274589","cluster_addr":"172.21.15.1:6829/2806274589","heartbeat_back_addr":"172.21.15.1:6833/2806274589","heartbeat_front_addr":"172.21.15.1:6831/2806274589","state":["exists","up"]},{"osd":10,"uuid":"70549796-5270-4ae2-ac3f-f61f5de105d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":761923580},{"type":"v1","addr":"172.21.15.53:6825","nonce":761923580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":761923580},{"type":"v1","addr":"172.21.15.53:6827","nonce":761923580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":761923580},{"type":"v1","addr":"172.21.15.53:6831","nonce":761923580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":761923580},{"type":"v1","addr":"172.21.15.53:6829","nonce":761923580}]},"public_addr":"172.21.15.53:6825/761923580","cluster_addr":"172.21.15.53:6827/761923580","heartbeat_back_addr":"172.21.15.53:6831/761923580","heartbeat_front_addr":"172.21.15.53:6829/761923580","state":["exists","up"]},{"osd":11,"uuid":"48f17c4a-370a-493e-956e-bf6ad3dcc594","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1586352190},{"type":"v1","addr":"172.21.15.161:6825","nonce":1586352190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":1586352190},{"type":"v1","addr":"172.21.15.161:6827","nonce":1586352190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":1586352190},{"type":"v1","addr":"172.21.15.161:6831","nonce":1586352190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":1586352190},{"type":"v1","addr":"172.21.15.161:6829","nonce":1586352190}]},"public_addr":"172.21.15.161:6825/1586352190","cluster_addr":"172.21.15.161:6827/1586352190","heartbeat_back_addr":"172.21.15.161:6831/1586352190","heartbeat_front_addr":"172.21.15.161:6829/1586352190","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:49:46.539613+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:50:11.912300+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:50:36.955490+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:51:02.200554+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:51:27.860585+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:51:54.694099+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:52:18.962351+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:52:44.881923+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:53:10.301390+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:53:35.417647+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:54:01.492219+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:54:26.938338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.26","mappings":[{"from":1,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/1749020850":"2024-09-12T21:48:24.524913+0000","172.21.15.1:6801/2821761445":"2024-09-12T21:48:24.524913+0000","172.21.15.1:0/1699984777":"2024-09-12T21:48:24.524913+0000","172.21.15.1:0/276770268":"2024-09-12T21:48:00.674728+0000","172.21.15.1:0/1278877732":"2024-09-12T21:48:24.524913+0000","172.21.15.1:0/2599895977":"2024-09-12T21:48:00.674728+0000","172.21.15.1:6801/2311309446":"2024-09-12T21:48:00.674728+0000","172.21.15.1:6800/2821761445":"2024-09-12T21:48:24.524913+0000","172.21.15.1:6800/2311309446":"2024-09-12T21:48:00.674728+0000","172.21.15.1:0/3202962495":"2024-09-12T21:48:00.674728+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T21:56:33.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:33.563+0000 7f8c1b7dd640 1 -- 172.21.15.1:0/1895044313 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f8bfc0a2c90 msgr2=0x7f8bfc0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:33.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:33.563+0000 7f8c1b7dd640 1 --2- 172.21.15.1:0/1895044313 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f8bfc0a2c90 0x7f8bfc0a5150 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8c0400a300 tx=0x7f8c0400a290 comp rx=0 tx=0).stop 2024-09-11T21:56:33.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:33.563+0000 7f8c1b7dd640 1 -- 172.21.15.1:0/1895044313 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8c14101420 msgr2=0x7f8c14102b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:33.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:33.563+0000 7f8c1b7dd640 1 --2- 172.21.15.1:0/1895044313 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8c14101420 0x7f8c14102b90 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f8c0802a940 tx=0x7f8c08003c00 comp rx=0 tx=0).stop 2024-09-11T21:56:33.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:33.563+0000 7f8c1b7dd640 1 -- 172.21.15.1:0/1895044313 shutdown_connections 2024-09-11T21:56:33.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:33.563+0000 7f8c1b7dd640 1 --2- 172.21.15.1:0/1895044313 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f8bfc0a2c90 0x7f8bfc0a5150 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:33.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:33.563+0000 7f8c1b7dd640 1 --2- 172.21.15.1:0/1895044313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c14106a80 0x7f8c141030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:33.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:33.563+0000 7f8c1b7dd640 1 --2- 172.21.15.1:0/1895044313 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8c14101420 0x7f8c14102b90 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:33.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:33.563+0000 7f8c1b7dd640 1 --2- 172.21.15.1:0/1895044313 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8c14100a80 0x7f8c14102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:33.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:33.563+0000 7f8c1b7dd640 1 -- 172.21.15.1:0/1895044313 >> 172.21.15.1:0/1895044313 conn(0x7f8c140fc7a0 msgr2=0x7f8c141047f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:33.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:33.563+0000 7f8c1b7dd640 1 -- 172.21.15.1:0/1895044313 shutdown_connections 2024-09-11T21:56:33.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:33.563+0000 7f8c1b7dd640 1 -- 172.21.15.1:0/1895044313 wait complete. 2024-09-11T21:56:34.351 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T21:56:34.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:34 smithi161 ceph-mon[35615]: pgmap v320: 97 pgs: 97 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 8 op/s; 7.2 KiB/s, 0 objects/s recovering 2024-09-11T21:56:34.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:34 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/1895044313' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T21:56:34.600 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:34 smithi053 ceph-mon[34908]: pgmap v320: 97 pgs: 97 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 8 op/s; 7.2 KiB/s, 0 objects/s recovering 2024-09-11T21:56:34.600 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:34 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/1895044313' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T21:56:34.610 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:56:34.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:34 smithi001 ceph-mon[30935]: pgmap v320: 97 pgs: 97 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 8 op/s; 7.2 KiB/s, 0 objects/s recovering 2024-09-11T21:56:34.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:34 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/1895044313' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T21:56:35.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:35.057+0000 7efd8275c640 1 -- 172.21.15.1:0/2184351327 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efd7c100d20 msgr2=0x7efd7c106860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:35.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:35.057+0000 7efd8275c640 1 --2- 172.21.15.1:0/2184351327 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efd7c100d20 0x7efd7c106860 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7efd64009930 tx=0x7efd6402a420 comp rx=0 tx=0).stop 2024-09-11T21:56:35.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:35.057+0000 7efd8275c640 1 -- 172.21.15.1:0/2184351327 shutdown_connections 2024-09-11T21:56:35.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:35.057+0000 7efd8275c640 1 --2- 172.21.15.1:0/2184351327 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd7c106fc0 0x7efd7c10eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:35.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:35.057+0000 7efd8275c640 1 --2- 172.21.15.1:0/2184351327 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efd7c100d20 0x7efd7c106860 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:35.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:35.057+0000 7efd8275c640 1 --2- 172.21.15.1:0/2184351327 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd7c100370 0x7efd7c100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:35.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:35.057+0000 7efd8275c640 1 -- 172.21.15.1:0/2184351327 >> 172.21.15.1:0/2184351327 conn(0x7efd7c077210 msgr2=0x7efd7c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:35.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:35.057+0000 7efd8275c640 1 -- 172.21.15.1:0/2184351327 shutdown_connections 2024-09-11T21:56:35.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:35.057+0000 7efd8275c640 1 -- 172.21.15.1:0/2184351327 wait complete. 2024-09-11T21:56:35.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:35.058+0000 7efd8275c640 1 Processor -- start 2024-09-11T21:56:35.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:35.058+0000 7efd8275c640 1 -- start start 2024-09-11T21:56:35.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:35.058+0000 7efd8275c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd7c100370 0x7efd7c1012f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:35.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:35.058+0000 7efd8275c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd7c100d20 0x7efd7c101830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:35.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:35.058+0000 7efd8275c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efd7c106fc0 0x7efd7c101d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:35.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:35.058+0000 7efd8275c640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7efd7c111fa0 con 0x7efd7c106fc0 2024-09-11T21:56:35.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:35.058+0000 7efd8275c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7efd7c111e20 con 0x7efd7c100370 2024-09-11T21:56:35.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:35.058+0000 7efd8275c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7efd7c112120 con 0x7efd7c100d20 2024-09-11T21:56:35.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:35.058+0000 7efd80cf9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efd7c106fc0 0x7efd7c101d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:35.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:35.058+0000 7efd80cf9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efd7c106fc0 0x7efd7c101d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45304/0 (socket says 172.21.15.1:45304) 2024-09-11T21:56:35.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:35.059+0000 7efd80cf9640 1 -- 172.21.15.1:0/4205132520 learned_addr learned my addr 172.21.15.1:0/4205132520 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:56:35.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:35.059+0000 7efd73fff640 1 --2- 172.21.15.1:0/4205132520 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd7c100d20 0x7efd7c101830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:35.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:35.059+0000 7efd7bfff640 1 --2- 172.21.15.1:0/4205132520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd7c100370 0x7efd7c1012f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:35.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:35.059+0000 7efd80cf9640 1 -- 172.21.15.1:0/4205132520 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd7c100d20 msgr2=0x7efd7c101830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:35.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:35.059+0000 7efd80cf9640 1 --2- 172.21.15.1:0/4205132520 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd7c100d20 0x7efd7c101830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:35.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:35.059+0000 7efd80cf9640 1 -- 172.21.15.1:0/4205132520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd7c100370 msgr2=0x7efd7c1012f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:35.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:35.059+0000 7efd80cf9640 1 --2- 172.21.15.1:0/4205132520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd7c100370 0x7efd7c1012f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:35.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:35.059+0000 7efd80cf9640 1 -- 172.21.15.1:0/4205132520 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd7c1a67f0 con 0x7efd7c106fc0 2024-09-11T21:56:35.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:35.059+0000 7efd73fff640 1 --2- 172.21.15.1:0/4205132520 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd7c100d20 0x7efd7c101830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:56:35.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:35.059+0000 7efd80cf9640 1 --2- 172.21.15.1:0/4205132520 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efd7c106fc0 0x7efd7c101d70 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7efd7400cce0 tx=0x7efd7400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:35.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:35.059+0000 7efd79ffb640 1 -- 172.21.15.1:0/4205132520 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd7400f450 con 0x7efd7c106fc0 2024-09-11T21:56:35.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:35.059+0000 7efd79ffb640 1 -- 172.21.15.1:0/4205132520 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd7400fad0 con 0x7efd7c106fc0 2024-09-11T21:56:35.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:35.059+0000 7efd79ffb640 1 -- 172.21.15.1:0/4205132520 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd74018c10 con 0x7efd7c106fc0 2024-09-11T21:56:35.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:35.059+0000 7efd8275c640 1 -- 172.21.15.1:0/4205132520 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd7c1a6ae0 con 0x7efd7c106fc0 2024-09-11T21:56:35.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:35.059+0000 7efd8275c640 1 -- 172.21.15.1:0/4205132520 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd7c1a6fa0 con 0x7efd7c106fc0 2024-09-11T21:56:35.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:35.060+0000 7efd8275c640 1 -- 172.21.15.1:0/4205132520 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd7c103430 con 0x7efd7c106fc0 2024-09-11T21:56:35.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:35.062+0000 7efd79ffb640 1 -- 172.21.15.1:0/4205132520 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7efd7400f5f0 con 0x7efd7c106fc0 2024-09-11T21:56:35.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:35.063+0000 7efd79ffb640 1 --2- 172.21.15.1:0/4205132520 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7efd680a2c90 0x7efd680a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:35.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:35.063+0000 7efd7bfff640 1 --2- 172.21.15.1:0/4205132520 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7efd680a2c90 0x7efd680a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:35.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:35.063+0000 7efd79ffb640 1 -- 172.21.15.1:0/4205132520 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7efd740cf920 con 0x7efd7c106fc0 2024-09-11T21:56:35.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:35.064+0000 7efd7bfff640 1 --2- 172.21.15.1:0/4205132520 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7efd680a2c90 0x7efd680a5150 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7efd6c00ba40 tx=0x7efd6c005cc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:35.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:35.066+0000 7efd79ffb640 1 -- 172.21.15.1:0/4205132520 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efd74014030 con 0x7efd7c106fc0 2024-09-11T21:56:35.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:35.279+0000 7efd8275c640 1 -- 172.21.15.1:0/4205132520 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efd7c100750 con 0x7efd7c106fc0 2024-09-11T21:56:35.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:35.279+0000 7efd79ffb640 1 -- 172.21.15.1:0/4205132520 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11949 (secure 0 0 0) 0x7efd7c100750 con 0x7efd7c106fc0 2024-09-11T21:56:35.281 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:56:35.281 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":27,"btime":"2024-09-11T21:55:32:152241+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/421542832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":421542832},{"type":"v1","addr":"172.21.15.1:6835","nonce":421542832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2836864406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2836864406},{"type":"v1","addr":"172.21.15.1:6837","nonce":2836864406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T21:55:20.791263+0000","modified":"2024-09-11T21:55:32.152227+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24484,"mds_2":24467,"mds_3":24466,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14595":{"gid":14595,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/407834672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":407834672},{"type":"v1","addr":"172.21.15.1:6839","nonce":407834672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/1068827678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":1068827678},{"type":"v1","addr":"172.21.15.1:6841","nonce":1068827678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/872410024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":872410024},{"type":"v1","addr":"172.21.15.161:6833","nonce":872410024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/1648944570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1648944570},{"type":"v1","addr":"172.21.15.53:6833","nonce":1648944570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/2126933993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2126933993},{"type":"v1","addr":"172.21.15.53:6835","nonce":2126933993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.161:6835/828381592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":828381592},{"type":"v1","addr":"172.21.15.161:6835","nonce":828381592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.161:6837/1799318656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1799318656},{"type":"v1","addr":"172.21.15.161:6837","nonce":1799318656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.53:6837/3457131830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3457131830},{"type":"v1","addr":"172.21.15.53:6837","nonce":3457131830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/455365383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":455365383},{"type":"v1","addr":"172.21.15.161:6839","nonce":455365383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.53:6839/1378584447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1378584447},{"type":"v1","addr":"172.21.15.53:6839","nonce":1378584447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24484,24467,24466,24460]},"id":1}]} 2024-09-11T21:56:35.282 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 27 2024-09-11T21:56:35.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:35.282+0000 7efd8275c640 1 -- 172.21.15.1:0/4205132520 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7efd680a2c90 msgr2=0x7efd680a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:35.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:35.282+0000 7efd8275c640 1 --2- 172.21.15.1:0/4205132520 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7efd680a2c90 0x7efd680a5150 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7efd6c00ba40 tx=0x7efd6c005cc0 comp rx=0 tx=0).stop 2024-09-11T21:56:35.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:35.282+0000 7efd8275c640 1 -- 172.21.15.1:0/4205132520 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efd7c106fc0 msgr2=0x7efd7c101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:35.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:35.282+0000 7efd8275c640 1 --2- 172.21.15.1:0/4205132520 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efd7c106fc0 0x7efd7c101d70 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7efd7400cce0 tx=0x7efd7400a620 comp rx=0 tx=0).stop 2024-09-11T21:56:35.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:35.282+0000 7efd8275c640 1 -- 172.21.15.1:0/4205132520 shutdown_connections 2024-09-11T21:56:35.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:35.282+0000 7efd8275c640 1 --2- 172.21.15.1:0/4205132520 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7efd680a2c90 0x7efd680a5150 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:35.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:35.282+0000 7efd8275c640 1 --2- 172.21.15.1:0/4205132520 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efd7c106fc0 0x7efd7c101d70 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:35.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:35.282+0000 7efd8275c640 1 --2- 172.21.15.1:0/4205132520 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd7c100d20 0x7efd7c101830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:35.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:35.282+0000 7efd8275c640 1 --2- 172.21.15.1:0/4205132520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efd7c100370 0x7efd7c1012f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:35.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:35.282+0000 7efd8275c640 1 -- 172.21.15.1:0/4205132520 >> 172.21.15.1:0/4205132520 conn(0x7efd7c077210 msgr2=0x7efd7c105730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:35.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:35.282+0000 7efd8275c640 1 -- 172.21.15.1:0/4205132520 shutdown_connections 2024-09-11T21:56:35.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:35.282+0000 7efd8275c640 1 -- 172.21.15.1:0/4205132520 wait complete. 2024-09-11T21:56:35.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:35 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/4205132520' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:56:35.621 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:35 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/4205132520' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:56:35.649 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:35 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/4205132520' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:56:35.984 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-11T21:55:20.791263+0000', 'modified': '2024-09-11T21:55:32.152227+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24494, 'mds_1': 24484, 'mds_2': 24467, 'mds_3': 24466, 'mds_4': 24460}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14595': {'gid': 14595, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.1:6839/407834672', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6838', 'nonce': 407834672}, {'type': 'v1', 'addr': '172.21.15.1:6839', 'nonce': 407834672}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14601': {'gid': 14601, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.1:6841/1068827678', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6840', 'nonce': 1068827678}, {'type': 'v1', 'addr': '172.21.15.1:6841', 'nonce': 1068827678}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24430': {'gid': 24430, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.161:6833/872410024', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6832', 'nonce': 872410024}, {'type': 'v1', 'addr': '172.21.15.161:6833', 'nonce': 872410024}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24431': {'gid': 24431, 'name': 'k', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.53:6833/1648944570', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6832', 'nonce': 1648944570}, {'type': 'v1', 'addr': '172.21.15.53:6833', 'nonce': 1648944570}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24449': {'gid': 24449, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.53:6835/2126933993', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6834', 'nonce': 2126933993}, {'type': 'v1', 'addr': '172.21.15.53:6835', 'nonce': 2126933993}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24460': {'gid': 24460, 'name': 'f', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.161:6835/828381592', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6834', 'nonce': 828381592}, {'type': 'v1', 'addr': '172.21.15.161:6835', 'nonce': 828381592}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24466': {'gid': 24466, 'name': 'c', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.161:6837/1799318656', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6836', 'nonce': 1799318656}, {'type': 'v1', 'addr': '172.21.15.161:6837', 'nonce': 1799318656}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24467': {'gid': 24467, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.53:6837/3457131830', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6836', 'nonce': 3457131830}, {'type': 'v1', 'addr': '172.21.15.53:6837', 'nonce': 3457131830}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24484': {'gid': 24484, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.161:6839/455365383', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6838', 'nonce': 455365383}, {'type': 'v1', 'addr': '172.21.15.161:6839', 'nonce': 455365383}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24494': {'gid': 24494, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.53:6839/1378584447', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6838', 'nonce': 1378584447}, {'type': 'v1', 'addr': '172.21.15.53:6839', 'nonce': 1378584447}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24494, 'qdb_cluster': [24494, 24484, 24467, 24466, 24460]} 2024-09-11T21:56:35.984 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-11T21:56:35.985 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.f status 2024-09-11T21:56:36.243 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:56:36.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:36 smithi161 ceph-mon[35615]: pgmap v321: 97 pgs: 97 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 6.2 KiB/s, 0 objects/s recovering 2024-09-11T21:56:36.679 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:36 smithi001 ceph-mon[30935]: pgmap v321: 97 pgs: 97 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 6.2 KiB/s, 0 objects/s recovering 2024-09-11T21:56:36.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:36 smithi053 ceph-mon[34908]: pgmap v321: 97 pgs: 97 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 6.2 KiB/s, 0 objects/s recovering 2024-09-11T21:56:36.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.895+0000 7f0d280a8640 1 -- 172.21.15.1:0/3183010413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d20102ef0 msgr2=0x7f0d2010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:36.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.895+0000 7f0d280a8640 1 --2- 172.21.15.1:0/3183010413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d20102ef0 0x7f0d2010da40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0d18009ca0 tx=0x7f0d1802a950 comp rx=0 tx=0).stop 2024-09-11T21:56:36.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.895+0000 7f0d280a8640 1 -- 172.21.15.1:0/3183010413 shutdown_connections 2024-09-11T21:56:36.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.895+0000 7f0d280a8640 1 --2- 172.21.15.1:0/3183010413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d20102ef0 0x7f0d2010da40 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:36.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.895+0000 7f0d280a8640 1 --2- 172.21.15.1:0/3183010413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d20102570 0x7f0d201029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:36.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.895+0000 7f0d280a8640 1 --2- 172.21.15.1:0/3183010413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d20100e20 0x7f0d20101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:36.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.895+0000 7f0d280a8640 1 -- 172.21.15.1:0/3183010413 >> 172.21.15.1:0/3183010413 conn(0x7f0d200fc7d0 msgr2=0x7f0d200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:36.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.895+0000 7f0d280a8640 1 -- 172.21.15.1:0/3183010413 shutdown_connections 2024-09-11T21:56:36.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.895+0000 7f0d280a8640 1 -- 172.21.15.1:0/3183010413 wait complete. 2024-09-11T21:56:36.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.896+0000 7f0d280a8640 1 Processor -- start 2024-09-11T21:56:36.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.896+0000 7f0d280a8640 1 -- start start 2024-09-11T21:56:36.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.896+0000 7f0d280a8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d20100e20 0x7f0d201973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:36.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.896+0000 7f0d280a8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d20102570 0x7f0d20197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:36.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.896+0000 7f0d280a8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d20102ef0 0x7f0d2019bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:36.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.896+0000 7f0d280a8640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0d20110fa0 con 0x7f0d20102ef0 2024-09-11T21:56:36.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.896+0000 7f0d280a8640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0d20110e20 con 0x7f0d20100e20 2024-09-11T21:56:36.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.896+0000 7f0d280a8640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0d20111120 con 0x7f0d20102570 2024-09-11T21:56:36.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.896+0000 7f0d26645640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d20102ef0 0x7f0d2019bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:36.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.896+0000 7f0d25e44640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d20100e20 0x7f0d201973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:36.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.896+0000 7f0d26645640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d20102ef0 0x7f0d2019bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41712/0 (socket says 172.21.15.1:41712) 2024-09-11T21:56:36.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.896+0000 7f0d25e44640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d20100e20 0x7f0d201973e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:43310/0 (socket says 172.21.15.1:43310) 2024-09-11T21:56:36.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.897+0000 7f0d26645640 1 -- 172.21.15.1:0/1283651731 learned_addr learned my addr 172.21.15.1:0/1283651731 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:56:36.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.897+0000 7f0d25643640 1 --2- 172.21.15.1:0/1283651731 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d20102570 0x7f0d20197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:36.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.897+0000 7f0d26645640 1 -- 172.21.15.1:0/1283651731 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d20102570 msgr2=0x7f0d20197920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:36.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.897+0000 7f0d26645640 1 --2- 172.21.15.1:0/1283651731 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d20102570 0x7f0d20197920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:36.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.897+0000 7f0d26645640 1 -- 172.21.15.1:0/1283651731 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d20100e20 msgr2=0x7f0d201973e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:36.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.897+0000 7f0d26645640 1 --2- 172.21.15.1:0/1283651731 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d20100e20 0x7f0d201973e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:36.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.897+0000 7f0d26645640 1 -- 172.21.15.1:0/1283651731 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d2019c390 con 0x7f0d20102ef0 2024-09-11T21:56:36.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.897+0000 7f0d25e44640 1 --2- 172.21.15.1:0/1283651731 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d20100e20 0x7f0d201973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T21:56:36.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.897+0000 7f0d26645640 1 --2- 172.21.15.1:0/1283651731 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d20102ef0 0x7f0d2019bcb0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0d18004af0 tx=0x7f0d1802f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:36.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.897+0000 7f0d16ffd640 1 -- 172.21.15.1:0/1283651731 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d18034040 con 0x7f0d20102ef0 2024-09-11T21:56:36.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.897+0000 7f0d16ffd640 1 -- 172.21.15.1:0/1283651731 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d18030920 con 0x7f0d20102ef0 2024-09-11T21:56:36.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.897+0000 7f0d280a8640 1 -- 172.21.15.1:0/1283651731 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d2019c620 con 0x7f0d20102ef0 2024-09-11T21:56:36.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.897+0000 7f0d16ffd640 1 -- 172.21.15.1:0/1283651731 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d18004740 con 0x7f0d20102ef0 2024-09-11T21:56:36.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.897+0000 7f0d280a8640 1 -- 172.21.15.1:0/1283651731 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d201a3f60 con 0x7f0d20102ef0 2024-09-11T21:56:36.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.900+0000 7f0d16ffd640 1 -- 172.21.15.1:0/1283651731 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f0d18037020 con 0x7f0d20102ef0 2024-09-11T21:56:36.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.900+0000 7f0d2a481740 1 Processor -- start 2024-09-11T21:56:36.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.900+0000 7f0d2a481740 1 -- start start 2024-09-11T21:56:36.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.900+0000 7f0d2a481740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56009fdc38b0 0x56009fdc3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:36.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.900+0000 7f0d2a481740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56009fdc41d0 0x56009fdd0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:36.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.900+0000 7f0d2a481740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56009fdd0dc0 0x56009fdd31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:36.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.900+0000 7f0d2a481740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56009fd2a090 con 0x56009fdc38b0 2024-09-11T21:56:36.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.900+0000 7f0d2a481740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56009fcd1a70 con 0x56009fdc41d0 2024-09-11T21:56:36.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.900+0000 7f0d2a481740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56009fd26050 con 0x56009fdd0dc0 2024-09-11T21:56:36.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.900+0000 7f0d25643640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56009fdc38b0 0x56009fdc3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:36.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.901+0000 7f0d25643640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56009fdc38b0 0x56009fdc3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41726/0 (socket says 172.21.15.1:41726) 2024-09-11T21:56:36.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.901+0000 7f0d25643640 1 -- 172.21.15.1:0/1691929510 learned_addr learned my addr 172.21.15.1:0/1691929510 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:56:36.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.901+0000 7f0d16ffd640 1 --2- 172.21.15.1:0/1283651731 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f0d0c0a2c90 0x7f0d0c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:36.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.901+0000 7f0d25e44640 1 --2- 172.21.15.1:0/1691929510 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56009fdc41d0 0x56009fdd0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:36.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.901+0000 7f0d25643640 1 -- 172.21.15.1:0/1691929510 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56009fdd0dc0 msgr2=0x56009fdd31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:56:36.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.901+0000 7f0d25643640 1 --2- 172.21.15.1:0/1691929510 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56009fdd0dc0 0x56009fdd31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:36.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.901+0000 7f0d25643640 1 -- 172.21.15.1:0/1691929510 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56009fdc41d0 msgr2=0x56009fdd0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:56:36.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.901+0000 7f0d25643640 1 --2- 172.21.15.1:0/1691929510 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56009fdc41d0 0x56009fdd0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:36.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.901+0000 7f0d25643640 1 -- 172.21.15.1:0/1691929510 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56009fcde130 con 0x56009fdc38b0 2024-09-11T21:56:36.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.901+0000 7f0d25e44640 1 --2- 172.21.15.1:0/1691929510 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56009fdc41d0 0x56009fdd0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:56:36.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.901+0000 7f0d26645640 1 --2- 172.21.15.1:0/1283651731 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f0d0c0a2c90 0x7f0d0c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:36.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.901+0000 7f0d16ffd640 1 -- 172.21.15.1:0/1283651731 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f0d180efe30 con 0x7f0d20102ef0 2024-09-11T21:56:36.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.901+0000 7f0d25643640 1 --2- 172.21.15.1:0/1691929510 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56009fdc38b0 0x56009fdc3c90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0d10006aa0 tx=0x7f0d10009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbbcc8a5ff425375 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:36.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.901+0000 7f0d26645640 1 --2- 172.21.15.1:0/1283651731 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f0d0c0a2c90 0x7f0d0c0a5150 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0d18048000 tx=0x7f0d1804e000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:36.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.901+0000 7f0d0bfff640 1 -- 172.21.15.1:0/1691929510 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d10018020 con 0x56009fdc38b0 2024-09-11T21:56:36.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.901+0000 7f0d0bfff640 1 -- 172.21.15.1:0/1691929510 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d10010dc0 con 0x56009fdc38b0 2024-09-11T21:56:36.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.901+0000 7f0d0bfff640 1 -- 172.21.15.1:0/1691929510 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d100137c0 con 0x56009fdc38b0 2024-09-11T21:56:36.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.902+0000 7f0d2a481740 1 -- 172.21.15.1:0/1691929510 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56009fdc38b0 msgr2=0x56009fdc3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:36.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.902+0000 7f0d2a481740 1 --2- 172.21.15.1:0/1691929510 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56009fdc38b0 0x56009fdc3c90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0d10006aa0 tx=0x7f0d10009f70 comp rx=0 tx=0).stop 2024-09-11T21:56:36.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.902+0000 7f0d2a481740 1 -- 172.21.15.1:0/1691929510 shutdown_connections 2024-09-11T21:56:36.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.902+0000 7f0d2a481740 1 --2- 172.21.15.1:0/1691929510 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56009fdd0dc0 0x56009fdd31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:36.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.902+0000 7f0d2a481740 1 --2- 172.21.15.1:0/1691929510 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56009fdc41d0 0x56009fdd0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:36.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.902+0000 7f0d2a481740 1 --2- 172.21.15.1:0/1691929510 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56009fdc38b0 0x56009fdc3c90 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:36.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.902+0000 7f0d2a481740 1 -- 172.21.15.1:0/1691929510 >> 172.21.15.1:0/1691929510 conn(0x56009fdbff60 msgr2=0x56009fdc0340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:36.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.902+0000 7f0d2a481740 1 -- 172.21.15.1:0/1691929510 shutdown_connections 2024-09-11T21:56:36.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.902+0000 7f0d2a481740 1 -- 172.21.15.1:0/1691929510 wait complete. 2024-09-11T21:56:36.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.903+0000 7f0d2a481740 1 -- start start 2024-09-11T21:56:36.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.903+0000 7f0d2a481740 1 Processor -- start 2024-09-11T21:56:36.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.903+0000 7f0d2a481740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56009fdc38b0 0x56009fdd0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:36.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.903+0000 7f0d2a481740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56009fdc41d0 0x56009fdcd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:36.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.903+0000 7f0d2a481740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56009fdd0dc0 0x56009fdcde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:36.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.903+0000 7f0d2a481740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56009fdd3990 con 0x56009fdc41d0 2024-09-11T21:56:36.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.903+0000 7f0d2a481740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56009fce4f20 con 0x56009fdd0dc0 2024-09-11T21:56:36.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.903+0000 7f0d2a481740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56009fdd3b10 con 0x56009fdc38b0 2024-09-11T21:56:36.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.903+0000 7f0d2a481740 10 client.0 _ll_register_callbacks cb 0x56009fcae030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:56:36.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.903+0000 7f0d25e44640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56009fdc41d0 0x56009fdcd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:36.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.903+0000 7f0d25e44640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56009fdc41d0 0x56009fdcd930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41738/0 (socket says 172.21.15.1:41738) 2024-09-11T21:56:36.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.903+0000 7f0d25e44640 1 -- 172.21.15.1:0/2582911 learned_addr learned my addr 172.21.15.1:0/2582911 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:56:36.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.904+0000 7f0d25643640 1 --2- 172.21.15.1:0/2582911 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56009fdc38b0 0x56009fdd0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:36.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.904+0000 7f0d25643640 1 --2- 172.21.15.1:0/2582911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56009fdd0dc0 0x56009fdcde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:36.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.904+0000 7f0d25e44640 1 -- 172.21.15.1:0/2582911 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56009fdc38b0 msgr2=0x56009fdd0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:56:36.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.904+0000 7f0d25e44640 1 --2- 172.21.15.1:0/2582911 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56009fdc38b0 0x56009fdd0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:36.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.904+0000 7f0d25e44640 1 -- 172.21.15.1:0/2582911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56009fdd0dc0 msgr2=0x56009fdcde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:56:36.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.904+0000 7f0d25e44640 1 --2- 172.21.15.1:0/2582911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56009fdd0dc0 0x56009fdcde70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:36.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.904+0000 7f0d25e44640 1 -- 172.21.15.1:0/2582911 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56009fa9b2b0 con 0x56009fdc41d0 2024-09-11T21:56:36.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.904+0000 7f0d25643640 1 --2- 172.21.15.1:0/2582911 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56009fdc38b0 0x56009fdd0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:56:36.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.904+0000 7f0d25e44640 1 --2- 172.21.15.1:0/2582911 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56009fdc41d0 0x56009fdcd930 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c012d20 tx=0x7f0d1c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=922a99400cbe6df9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:36.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.904+0000 7f0d097fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T21:56:36.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.904+0000 7f0d097fa640 1 -- 172.21.15.1:0/2582911 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d1c0120d0 con 0x56009fdc41d0 2024-09-11T21:56:36.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.904+0000 7f0d097fa640 1 -- 172.21.15.1:0/2582911 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f0d1c010430 con 0x56009fdc41d0 2024-09-11T21:56:36.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.904+0000 7f0d2a481740 1 -- 172.21.15.1:0/2582911 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d10010dc0 con 0x56009fdc41d0 2024-09-11T21:56:36.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.904+0000 7f0d097fa640 1 -- 172.21.15.1:0/2582911 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0d1c012270 con 0x56009fdc41d0 2024-09-11T21:56:36.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.904+0000 7f0d2a481740 10 client.14958 fetch_fsmap learned FSMap version 27 2024-09-11T21:56:36.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.904+0000 7f0d2a481740 1 -- 172.21.15.1:0/2582911 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56009fddf2b0 con 0x56009fdc41d0 2024-09-11T21:56:36.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.905+0000 7f0d097fa640 1 -- 172.21.15.1:0/2582911 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d10010dc0 con 0x56009fdc41d0 2024-09-11T21:56:36.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.905+0000 7f0d097fa640 1 -- 172.21.15.1:0/2582911 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f0d1c026800 con 0x56009fdc41d0 2024-09-11T21:56:36.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.905+0000 7f0d2a481740 10 client.14958 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T21:56:36.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.905+0000 7f0d2a481740 10 client.14958 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-11T21:56:36.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.905+0000 7f0d2a481740 20 client.14958 populate_metadata read hostname 'smithi001' 2024-09-11T21:56:36.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.905+0000 7f0d2a481740 1 --2- 172.21.15.1:0/2582911 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x56009fde18e0 0x56009fde2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:36.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.905+0000 7f0d2a481740 4 client.14958 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T21:56:36.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.905+0000 7f0d2a481740 1 -- 172.21.15.1:0/2582911 --> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56009fd37600 con 0x56009fde18e0 2024-09-11T21:56:36.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.905+0000 7f0d25643640 1 --2- 172.21.15.1:0/2582911 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x56009fde18e0 0x56009fde2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:36.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.906+0000 7f0d097fa640 10 client.14958 ms_handle_connect on v2:172.21.15.161:6834/828381592 2024-09-11T21:56:36.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.906+0000 7f0d25643640 1 --2- 172.21.15.1:0/2582911 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x56009fde18e0 0x56009fde2d50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b607ee74ad5394bb server_cookie=61e92af41e62f335 in_seq=0 out_seq=0 2024-09-11T21:56:36.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.906+0000 7f0d097fa640 1 -- 172.21.15.1:0/2582911 <== mds.4 v2:172.21.15.161:6834/828381592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56009fd37600 con 0x56009fde18e0 2024-09-11T21:56:36.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.906+0000 7f0d097fa640 10 client.14958 handle_command_reply: tid=0 2024-09-11T21:56:36.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.906+0000 7f0d2a481740 1 client.14958 shutdown 2024-09-11T21:56:36.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.907+0000 7f0d2a481740 1 -- 172.21.15.1:0/2582911 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56009fdc41d0 msgr2=0x56009fdcd930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:36.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.907+0000 7f0d2a481740 1 --2- 172.21.15.1:0/2582911 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56009fdc41d0 0x56009fdcd930 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c012d20 tx=0x7f0d1c00f790 comp rx=0 tx=0).stop 2024-09-11T21:56:36.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.907+0000 7f0d2a481740 1 -- 172.21.15.1:0/2582911 shutdown_connections 2024-09-11T21:56:36.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.907+0000 7f0d2a481740 1 --2- 172.21.15.1:0/2582911 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x56009fde18e0 0x56009fde2d50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:36.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.907+0000 7f0d2a481740 1 --2- 172.21.15.1:0/2582911 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56009fdd0dc0 0x56009fdcde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:36.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.907+0000 7f0d2a481740 1 --2- 172.21.15.1:0/2582911 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56009fdc41d0 0x56009fdcd930 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:36.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.907+0000 7f0d097fa640 0 client.14958 ms_handle_reset on v2:172.21.15.161:6834/828381592 2024-09-11T21:56:36.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.907+0000 7f0d2a481740 1 --2- 172.21.15.1:0/2582911 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56009fdc38b0 0x56009fdd0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:36.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.907+0000 7f0d2a481740 1 -- 172.21.15.1:0/2582911 >> 172.21.15.1:0/2582911 conn(0x56009fdbff60 msgr2=0x56009fdc5cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:36.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.907+0000 7f0d2a481740 1 -- 172.21.15.1:0/2582911 shutdown_connections 2024-09-11T21:56:36.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.907+0000 7f0d2a481740 1 -- 172.21.15.1:0/2582911 wait complete. 2024-09-11T21:56:36.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.907+0000 7f0d2a481740 20 client.14958 trim_cache size 0 max 16384 2024-09-11T21:56:36.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.925+0000 7f0d2a481740 1 Processor -- start 2024-09-11T21:56:36.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.925+0000 7f0d2a481740 1 -- start start 2024-09-11T21:56:36.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.925+0000 7f0d2a481740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56009fdc38b0 0x56009fde2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:36.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.925+0000 7f0d2a481740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56009fdc41d0 0x56009fdeb420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:36.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.925+0000 7f0d2a481740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56009fdd0dc0 0x56009fdbe7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:36.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.925+0000 7f0d2a481740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56009fde1090 con 0x56009fdc38b0 2024-09-11T21:56:36.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.925+0000 7f0d2a481740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56009fde1760 con 0x56009fdc41d0 2024-09-11T21:56:36.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.925+0000 7f0d2a481740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56009fde1e30 con 0x56009fdd0dc0 2024-09-11T21:56:36.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.925+0000 7f0d25643640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56009fdc38b0 0x56009fde2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:36.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.925+0000 7f0d25643640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56009fdc38b0 0x56009fde2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41752/0 (socket says 172.21.15.1:41752) 2024-09-11T21:56:36.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.925+0000 7f0d25643640 1 -- 172.21.15.1:0/1117399065 learned_addr learned my addr 172.21.15.1:0/1117399065 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:56:36.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.925+0000 7f0d25643640 1 -- 172.21.15.1:0/1117399065 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56009fdd0dc0 msgr2=0x56009fdbe7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:56:36.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.925+0000 7f0d25643640 1 --2- 172.21.15.1:0/1117399065 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56009fdd0dc0 0x56009fdbe7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:36.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.925+0000 7f0d25643640 1 -- 172.21.15.1:0/1117399065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56009fdc41d0 msgr2=0x56009fdeb420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:56:36.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.925+0000 7f0d25643640 1 --2- 172.21.15.1:0/1117399065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56009fdc41d0 0x56009fdeb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:36.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.925+0000 7f0d25643640 1 -- 172.21.15.1:0/1117399065 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56009fde22e0 con 0x56009fdc38b0 2024-09-11T21:56:36.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.925+0000 7f0d25643640 1 --2- 172.21.15.1:0/1117399065 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56009fdc38b0 0x56009fde2d50 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0d10010c80 tx=0x7f0d10014f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31f6d4323fefe4a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:36.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.926+0000 7f0d0b7fe640 1 -- 172.21.15.1:0/1117399065 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d10018040 con 0x56009fdc38b0 2024-09-11T21:56:36.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.926+0000 7f0d0b7fe640 1 -- 172.21.15.1:0/1117399065 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d10014ba0 con 0x56009fdc38b0 2024-09-11T21:56:36.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.926+0000 7f0d0b7fe640 1 -- 172.21.15.1:0/1117399065 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d10013da0 con 0x56009fdc38b0 2024-09-11T21:56:36.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.926+0000 7f0d2a481740 1 -- 172.21.15.1:0/1117399065 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56009fdc38b0 msgr2=0x56009fde2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:36.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.926+0000 7f0d2a481740 1 --2- 172.21.15.1:0/1117399065 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56009fdc38b0 0x56009fde2d50 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0d10010c80 tx=0x7f0d10014f50 comp rx=0 tx=0).stop 2024-09-11T21:56:36.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.926+0000 7f0d2a481740 1 -- 172.21.15.1:0/1117399065 shutdown_connections 2024-09-11T21:56:36.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.926+0000 7f0d2a481740 1 --2- 172.21.15.1:0/1117399065 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56009fdd0dc0 0x56009fdbe7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:36.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.926+0000 7f0d2a481740 1 --2- 172.21.15.1:0/1117399065 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56009fdc41d0 0x56009fdeb420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:36.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.926+0000 7f0d2a481740 1 --2- 172.21.15.1:0/1117399065 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56009fdc38b0 0x56009fde2d50 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:36.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.926+0000 7f0d2a481740 1 -- 172.21.15.1:0/1117399065 >> 172.21.15.1:0/1117399065 conn(0x56009fdbff60 msgr2=0x56009fdbace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:36.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.926+0000 7f0d2a481740 1 -- 172.21.15.1:0/1117399065 shutdown_connections 2024-09-11T21:56:36.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.926+0000 7f0d2a481740 1 -- 172.21.15.1:0/1117399065 wait complete. 2024-09-11T21:56:36.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.927+0000 7f0d2a481740 1 -- start start 2024-09-11T21:56:36.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.927+0000 7f0d2a481740 1 Processor -- start 2024-09-11T21:56:36.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.927+0000 7f0d2a481740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56009fdc38b0 0x56009fde5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:36.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.927+0000 7f0d2a481740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56009fdc41d0 0x56009fde3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:36.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.927+0000 7f0d2a481740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56009fdd0dc0 0x56009fde4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:36.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.927+0000 7f0d2a481740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56009fde2bd0 con 0x56009fdc41d0 2024-09-11T21:56:36.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.927+0000 7f0d2a481740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56009fde2500 con 0x56009fdc38b0 2024-09-11T21:56:36.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.927+0000 7f0d2a481740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56009fde32a0 con 0x56009fdd0dc0 2024-09-11T21:56:36.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.927+0000 7f0d2a481740 10 client.0 _ll_register_callbacks cb 0x7f0d1c009c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:56:36.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.927+0000 7f0d25e44640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56009fdc41d0 0x56009fde3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:36.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.927+0000 7f0d25643640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56009fdc38b0 0x56009fde5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:36.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.927+0000 7f0d25e44640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56009fdc41d0 0x56009fde3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41768/0 (socket says 172.21.15.1:41768) 2024-09-11T21:56:36.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.927+0000 7f0d25e44640 1 -- 172.21.15.1:0/3758928509 learned_addr learned my addr 172.21.15.1:0/3758928509 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:56:36.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.927+0000 7f0d25643640 1 --2- 172.21.15.1:0/3758928509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56009fdd0dc0 0x56009fde4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:36.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.928+0000 7f0d25e44640 1 -- 172.21.15.1:0/3758928509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56009fdd0dc0 msgr2=0x56009fde4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:56:36.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.928+0000 7f0d25e44640 1 --2- 172.21.15.1:0/3758928509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56009fdd0dc0 0x56009fde4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:36.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.928+0000 7f0d25e44640 1 -- 172.21.15.1:0/3758928509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56009fdc38b0 msgr2=0x56009fde5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:56:36.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.928+0000 7f0d25e44640 1 --2- 172.21.15.1:0/3758928509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56009fdc38b0 0x56009fde5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:36.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.928+0000 7f0d25e44640 1 -- 172.21.15.1:0/3758928509 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56009fde4b30 con 0x56009fdc41d0 2024-09-11T21:56:36.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.928+0000 7f0d25643640 1 --2- 172.21.15.1:0/3758928509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56009fdd0dc0 0x56009fde4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:56:36.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.928+0000 7f0d25e44640 1 --2- 172.21.15.1:0/3758928509 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56009fdc41d0 0x56009fde3f50 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c026e10 tx=0x7f0d1c027f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f431b0f004039b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:36.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.928+0000 7f0d097fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T21:56:36.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.928+0000 7f0d097fa640 1 -- 172.21.15.1:0/3758928509 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d1c00fc90 con 0x56009fdc41d0 2024-09-11T21:56:36.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.928+0000 7f0d097fa640 1 -- 172.21.15.1:0/3758928509 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f0d1c018040 con 0x56009fdc41d0 2024-09-11T21:56:36.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.928+0000 7f0d2a481740 1 -- 172.21.15.1:0/3758928509 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d10014ba0 con 0x56009fdc41d0 2024-09-11T21:56:36.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.928+0000 7f0d097fa640 1 -- 172.21.15.1:0/3758928509 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0d1c0127f0 con 0x56009fdc41d0 2024-09-11T21:56:36.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.928+0000 7f0d2a481740 10 client.14964 fetch_fsmap learned FSMap version 27 2024-09-11T21:56:36.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.928+0000 7f0d2a481740 1 -- 172.21.15.1:0/3758928509 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56009fe4deb0 con 0x56009fdc41d0 2024-09-11T21:56:36.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.929+0000 7f0d097fa640 1 -- 172.21.15.1:0/3758928509 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d10014ba0 con 0x56009fdc41d0 2024-09-11T21:56:36.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.929+0000 7f0d097fa640 1 -- 172.21.15.1:0/3758928509 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f0d1c01ca90 con 0x56009fdc41d0 2024-09-11T21:56:36.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.929+0000 7f0d2a481740 10 client.14964 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T21:56:36.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.929+0000 7f0d2a481740 10 client.14964 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-11T21:56:36.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.929+0000 7f0d2a481740 20 client.14964 populate_metadata read hostname 'smithi001' 2024-09-11T21:56:36.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.929+0000 7f0d2a481740 1 --2- 172.21.15.1:0/3758928509 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x56009fe52a50 0x56009fe51760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:36.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.929+0000 7f0d2a481740 4 client.14964 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T21:56:36.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.929+0000 7f0d2a481740 1 -- 172.21.15.1:0/3758928509 --> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] -- command(tid 0: {"prefix": "status"}) -- 0x56009fd31b10 con 0x56009fe52a50 2024-09-11T21:56:36.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.929+0000 7f0d25643640 1 --2- 172.21.15.1:0/3758928509 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x56009fe52a50 0x56009fe51760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:36.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.929+0000 7f0d25643640 1 --2- 172.21.15.1:0/3758928509 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x56009fe52a50 0x56009fe51760 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c15918a37b602094 server_cookie=97b3e8b9ea1adaeb in_seq=0 out_seq=0 2024-09-11T21:56:36.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.929+0000 7f0d097fa640 10 client.14964 ms_handle_connect on v2:172.21.15.161:6834/828381592 2024-09-11T21:56:36.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.930+0000 7f0d097fa640 1 -- 172.21.15.1:0/3758928509 <== mds.4 v2:172.21.15.161:6834/828381592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x56009fd31b10 con 0x56009fe52a50 2024-09-11T21:56:36.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.930+0000 7f0d097fa640 10 client.14964 handle_command_reply: tid=0 2024-09-11T21:56:36.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.930+0000 7f0d2a481740 1 client.14964 shutdown 2024-09-11T21:56:36.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.930+0000 7f0d2a481740 1 -- 172.21.15.1:0/3758928509 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56009fdc41d0 msgr2=0x56009fde3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:36.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.930+0000 7f0d2a481740 1 --2- 172.21.15.1:0/3758928509 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56009fdc41d0 0x56009fde3f50 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c026e10 tx=0x7f0d1c027f20 comp rx=0 tx=0).stop 2024-09-11T21:56:36.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.930+0000 7f0d2a481740 1 -- 172.21.15.1:0/3758928509 shutdown_connections 2024-09-11T21:56:36.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.930+0000 7f0d2a481740 1 --2- 172.21.15.1:0/3758928509 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56009fdd0dc0 0x56009fde4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:36.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.930+0000 7f0d2a481740 1 --2- 172.21.15.1:0/3758928509 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56009fdc41d0 0x56009fde3f50 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:36.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.930+0000 7f0d2a481740 1 --2- 172.21.15.1:0/3758928509 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x56009fe52a50 0x56009fe51760 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:36.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.930+0000 7f0d2a481740 1 --2- 172.21.15.1:0/3758928509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56009fdc38b0 0x56009fde5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:36.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.930+0000 7f0d097fa640 0 client.14964 ms_handle_reset on v2:172.21.15.161:6834/828381592 2024-09-11T21:56:36.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.930+0000 7f0d2a481740 1 -- 172.21.15.1:0/3758928509 >> 172.21.15.1:0/3758928509 conn(0x56009fdbff60 msgr2=0x56009fdbe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:36.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.930+0000 7f0d2a481740 1 -- 172.21.15.1:0/3758928509 shutdown_connections 2024-09-11T21:56:36.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.930+0000 7f0d2a481740 1 -- 172.21.15.1:0/3758928509 wait complete. 2024-09-11T21:56:36.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.930+0000 7f0d2a481740 20 client.14964 trim_cache size 0 max 16384 2024-09-11T21:56:36.939 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-11T21:56:36.939 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "3ffca83c-7087-11ef-bcea-c7b262605968", 2024-09-11T21:56:36.939 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 4, 2024-09-11T21:56:36.939 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 24460, 2024-09-11T21:56:36.939 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-09-11T21:56:36.939 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-09-11T21:56:36.939 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-09-11T21:56:36.939 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 65.783369378000003, 2024-09-11T21:56:36.939 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 27, 2024-09-11T21:56:36.939 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 86, 2024-09-11T21:56:36.939 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 84, 2024-09-11T21:56:36.940 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 103.24635337700001 2024-09-11T21:56:36.940 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-11T21:56:36.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.931+0000 7f0d280a8640 1 -- 172.21.15.1:0/1283651731 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f0d0c0a2c90 msgr2=0x7f0d0c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:36.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.931+0000 7f0d280a8640 1 --2- 172.21.15.1:0/1283651731 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f0d0c0a2c90 0x7f0d0c0a5150 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0d18048000 tx=0x7f0d1804e000 comp rx=0 tx=0).stop 2024-09-11T21:56:36.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.931+0000 7f0d280a8640 1 -- 172.21.15.1:0/1283651731 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d20102ef0 msgr2=0x7f0d2019bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:36.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.931+0000 7f0d280a8640 1 --2- 172.21.15.1:0/1283651731 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d20102ef0 0x7f0d2019bcb0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0d18004af0 tx=0x7f0d1802f5b0 comp rx=0 tx=0).stop 2024-09-11T21:56:36.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.931+0000 7f0d280a8640 1 -- 172.21.15.1:0/1283651731 shutdown_connections 2024-09-11T21:56:36.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.931+0000 7f0d280a8640 1 --2- 172.21.15.1:0/1283651731 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f0d0c0a2c90 0x7f0d0c0a5150 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:36.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.931+0000 7f0d280a8640 1 --2- 172.21.15.1:0/1283651731 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d20102ef0 0x7f0d2019bcb0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:36.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.931+0000 7f0d280a8640 1 --2- 172.21.15.1:0/1283651731 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d20102570 0x7f0d20197920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:36.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.931+0000 7f0d280a8640 1 --2- 172.21.15.1:0/1283651731 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0d20100e20 0x7f0d201973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:36.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.931+0000 7f0d280a8640 1 -- 172.21.15.1:0/1283651731 >> 172.21.15.1:0/1283651731 conn(0x7f0d200fc7d0 msgr2=0x7f0d20106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:36.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.931+0000 7f0d280a8640 1 -- 172.21.15.1:0/1283651731 shutdown_connections 2024-09-11T21:56:36.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:36.931+0000 7f0d280a8640 1 -- 172.21.15.1:0/1283651731 wait complete. 2024-09-11T21:56:37.584 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.c status 2024-09-11T21:56:37.844 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:56:38.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.315+0000 7effda765640 1 -- 172.21.15.1:0/3070631986 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7effd4102d40 msgr2=0x7effd4103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:38.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.315+0000 7effda765640 1 --2- 172.21.15.1:0/3070631986 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7effd4102d40 0x7effd4103120 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7effc4009900 tx=0x7effc402a480 comp rx=0 tx=0).stop 2024-09-11T21:56:38.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.315+0000 7effda765640 1 -- 172.21.15.1:0/3070631986 shutdown_connections 2024-09-11T21:56:38.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.315+0000 7effda765640 1 --2- 172.21.15.1:0/3070631986 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7effd410c170 0x7effd410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:38.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.315+0000 7effda765640 1 --2- 172.21.15.1:0/3070631986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7effd4103660 0x7effd410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:38.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.315+0000 7effda765640 1 --2- 172.21.15.1:0/3070631986 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7effd4102d40 0x7effd4103120 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:38.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.315+0000 7effda765640 1 -- 172.21.15.1:0/3070631986 >> 172.21.15.1:0/3070631986 conn(0x7effd40fc7a0 msgr2=0x7effd40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:38.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.315+0000 7effda765640 1 -- 172.21.15.1:0/3070631986 shutdown_connections 2024-09-11T21:56:38.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.315+0000 7effda765640 1 -- 172.21.15.1:0/3070631986 wait complete. 2024-09-11T21:56:38.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.315+0000 7effda765640 1 Processor -- start 2024-09-11T21:56:38.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.315+0000 7effda765640 1 -- start start 2024-09-11T21:56:38.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.316+0000 7effda765640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7effd4102d40 0x7effd4198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:38.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.316+0000 7effda765640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7effd4103660 0x7effd41989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:38.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.316+0000 7effda765640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7effd410c170 0x7effd419cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:38.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.316+0000 7effda765640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7effd4111f70 con 0x7effd4103660 2024-09-11T21:56:38.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.316+0000 7effda765640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7effd4111df0 con 0x7effd4102d40 2024-09-11T21:56:38.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.316+0000 7effda765640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7effd41120f0 con 0x7effd410c170 2024-09-11T21:56:38.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.316+0000 7effcbfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7effd4103660 0x7effd41989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:38.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.316+0000 7effcbfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7effd4103660 0x7effd41989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41786/0 (socket says 172.21.15.1:41786) 2024-09-11T21:56:38.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.316+0000 7effcbfff640 1 -- 172.21.15.1:0/3891769111 learned_addr learned my addr 172.21.15.1:0/3891769111 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:56:38.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.316+0000 7effd8d02640 1 --2- 172.21.15.1:0/3891769111 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7effd410c170 0x7effd419cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:38.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.316+0000 7effd3fff640 1 --2- 172.21.15.1:0/3891769111 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7effd4102d40 0x7effd4198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:38.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.316+0000 7effcbfff640 1 -- 172.21.15.1:0/3891769111 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7effd410c170 msgr2=0x7effd419cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:38.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.316+0000 7effcbfff640 1 --2- 172.21.15.1:0/3891769111 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7effd410c170 0x7effd419cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:38.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.316+0000 7effcbfff640 1 -- 172.21.15.1:0/3891769111 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7effd4102d40 msgr2=0x7effd4198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:38.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.316+0000 7effcbfff640 1 --2- 172.21.15.1:0/3891769111 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7effd4102d40 0x7effd4198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:38.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.316+0000 7effcbfff640 1 -- 172.21.15.1:0/3891769111 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effd419d420 con 0x7effd4103660 2024-09-11T21:56:38.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.316+0000 7effcbfff640 1 --2- 172.21.15.1:0/3891769111 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7effd4103660 0x7effd41989b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7effbc00e9f0 tx=0x7effbc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:38.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.316+0000 7effd1ffb640 1 -- 172.21.15.1:0/3891769111 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effbc00ccc0 con 0x7effd4103660 2024-09-11T21:56:38.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.317+0000 7effda765640 1 -- 172.21.15.1:0/3891769111 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effd419d710 con 0x7effd4103660 2024-09-11T21:56:38.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.317+0000 7effd1ffb640 1 -- 172.21.15.1:0/3891769111 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effbc00ce60 con 0x7effd4103660 2024-09-11T21:56:38.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.317+0000 7effd1ffb640 1 -- 172.21.15.1:0/3891769111 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effbc010690 con 0x7effd4103660 2024-09-11T21:56:38.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.317+0000 7effda765640 1 -- 172.21.15.1:0/3891769111 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effd41a4ff0 con 0x7effd4103660 2024-09-11T21:56:38.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.319+0000 7effd1ffb640 1 -- 172.21.15.1:0/3891769111 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7effbc0046c0 con 0x7effd4103660 2024-09-11T21:56:38.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.319+0000 7effdcb3e740 1 Processor -- start 2024-09-11T21:56:38.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.319+0000 7effdcb3e740 1 -- start start 2024-09-11T21:56:38.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.320+0000 7effdcb3e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55caf194f9d0 0x55caf194fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:38.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.320+0000 7effdcb3e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55caf19502f0 0x55caf195a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:38.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.320+0000 7effdcb3e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55caf195abb0 0x55caf195cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:38.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.320+0000 7effdcb3e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55caf18b6090 con 0x55caf194f9d0 2024-09-11T21:56:38.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.320+0000 7effdcb3e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55caf185da70 con 0x55caf19502f0 2024-09-11T21:56:38.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.320+0000 7effdcb3e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55caf18b2050 con 0x55caf195abb0 2024-09-11T21:56:38.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.320+0000 7effd8d02640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55caf194f9d0 0x55caf194fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:38.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.320+0000 7effd8d02640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55caf194f9d0 0x55caf194fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41802/0 (socket says 172.21.15.1:41802) 2024-09-11T21:56:38.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.320+0000 7effd8d02640 1 -- 172.21.15.1:0/505150097 learned_addr learned my addr 172.21.15.1:0/505150097 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:56:38.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.320+0000 7effcbfff640 1 --2- 172.21.15.1:0/505150097 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55caf195abb0 0x55caf195cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:38.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.320+0000 7effd3fff640 1 --2- 172.21.15.1:0/505150097 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55caf19502f0 0x55caf195a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:38.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.320+0000 7effd8d02640 1 -- 172.21.15.1:0/505150097 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55caf195abb0 msgr2=0x55caf195cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:56:38.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.320+0000 7effd1ffb640 1 --2- 172.21.15.1:0/3891769111 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7effc00a2c10 0x7effc00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:38.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.320+0000 7effd8d02640 1 --2- 172.21.15.1:0/505150097 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55caf195abb0 0x55caf195cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:38.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.320+0000 7effd8d02640 1 -- 172.21.15.1:0/505150097 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55caf19502f0 msgr2=0x55caf195a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:56:38.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.320+0000 7effd8d02640 1 --2- 172.21.15.1:0/505150097 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55caf19502f0 0x55caf195a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:38.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.320+0000 7effd8d02640 1 -- 172.21.15.1:0/505150097 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55caf186a130 con 0x55caf194f9d0 2024-09-11T21:56:38.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.320+0000 7effd1ffb640 1 -- 172.21.15.1:0/3891769111 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7effbc0cf660 con 0x7effd4103660 2024-09-11T21:56:38.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.320+0000 7effd8d02640 1 --2- 172.21.15.1:0/3891769111 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7effc00a2c10 0x7effc00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:38.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.320+0000 7effd8d02640 1 --2- 172.21.15.1:0/505150097 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55caf194f9d0 0x55caf194fdb0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7effcc009f90 tx=0x7effcc010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae8678e942c0ee8f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:38.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.321+0000 7effca7fc640 1 -- 172.21.15.1:0/505150097 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effcc022070 con 0x55caf194f9d0 2024-09-11T21:56:38.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.321+0000 7effca7fc640 1 -- 172.21.15.1:0/505150097 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effcc01b3e0 con 0x55caf194f9d0 2024-09-11T21:56:38.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.321+0000 7effca7fc640 1 -- 172.21.15.1:0/505150097 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effcc01a5a0 con 0x55caf194f9d0 2024-09-11T21:56:38.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.321+0000 7effdcb3e740 1 -- 172.21.15.1:0/505150097 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55caf194f9d0 msgr2=0x55caf194fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:38.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.321+0000 7effd8d02640 1 --2- 172.21.15.1:0/3891769111 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7effc00a2c10 0x7effc00a50d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7effcc01f000 tx=0x7effcc020000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:38.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.321+0000 7effdcb3e740 1 --2- 172.21.15.1:0/505150097 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55caf194f9d0 0x55caf194fdb0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7effcc009f90 tx=0x7effcc010860 comp rx=0 tx=0).stop 2024-09-11T21:56:38.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.322+0000 7effdcb3e740 1 -- 172.21.15.1:0/505150097 shutdown_connections 2024-09-11T21:56:38.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.322+0000 7effdcb3e740 1 --2- 172.21.15.1:0/505150097 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55caf195abb0 0x55caf195cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:38.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.322+0000 7effdcb3e740 1 --2- 172.21.15.1:0/505150097 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55caf19502f0 0x55caf195a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:38.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.322+0000 7effdcb3e740 1 --2- 172.21.15.1:0/505150097 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55caf194f9d0 0x55caf194fdb0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:38.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.322+0000 7effdcb3e740 1 -- 172.21.15.1:0/505150097 >> 172.21.15.1:0/505150097 conn(0x55caf194bf60 msgr2=0x55caf194c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:38.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.322+0000 7effdcb3e740 1 -- 172.21.15.1:0/505150097 shutdown_connections 2024-09-11T21:56:38.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.322+0000 7effdcb3e740 1 -- 172.21.15.1:0/505150097 wait complete. 2024-09-11T21:56:38.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.322+0000 7effdcb3e740 1 -- start start 2024-09-11T21:56:38.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.322+0000 7effdcb3e740 1 Processor -- start 2024-09-11T21:56:38.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.323+0000 7effdcb3e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55caf194f9d0 0x55caf195a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:38.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.323+0000 7effdcb3e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55caf19502f0 0x55caf1957760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:38.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.323+0000 7effdcb3e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55caf195abb0 0x55caf1957ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:38.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.323+0000 7effdcb3e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55caf195d780 con 0x55caf194f9d0 2024-09-11T21:56:38.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.323+0000 7effdcb3e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55caf1870f20 con 0x55caf19502f0 2024-09-11T21:56:38.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.323+0000 7effdcb3e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55caf195d900 con 0x55caf195abb0 2024-09-11T21:56:38.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.323+0000 7effdcb3e740 10 client.0 _ll_register_callbacks cb 0x55caf183a030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:56:38.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.323+0000 7effd3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55caf194f9d0 0x55caf195a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:38.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.323+0000 7effd3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55caf194f9d0 0x55caf195a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41818/0 (socket says 172.21.15.1:41818) 2024-09-11T21:56:38.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.323+0000 7effd3fff640 1 -- 172.21.15.1:0/2147675616 learned_addr learned my addr 172.21.15.1:0/2147675616 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:56:38.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.323+0000 7effd8d02640 1 --2- 172.21.15.1:0/2147675616 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55caf195abb0 0x55caf1957ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:38.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.323+0000 7effcbfff640 1 --2- 172.21.15.1:0/2147675616 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55caf19502f0 0x55caf1957760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:38.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.323+0000 7effd3fff640 1 -- 172.21.15.1:0/2147675616 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55caf195abb0 msgr2=0x55caf1957ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:56:38.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.323+0000 7effd3fff640 1 --2- 172.21.15.1:0/2147675616 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55caf195abb0 0x55caf1957ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:38.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.323+0000 7effd3fff640 1 -- 172.21.15.1:0/2147675616 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55caf19502f0 msgr2=0x55caf1957760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:56:38.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.323+0000 7effd3fff640 1 --2- 172.21.15.1:0/2147675616 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55caf19502f0 0x55caf1957760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:38.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.323+0000 7effd3fff640 1 -- 172.21.15.1:0/2147675616 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55caf16272b0 con 0x55caf194f9d0 2024-09-11T21:56:38.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.323+0000 7effd8d02640 1 --2- 172.21.15.1:0/2147675616 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55caf195abb0 0x55caf1957ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T21:56:38.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.323+0000 7effd3fff640 1 --2- 172.21.15.1:0/2147675616 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55caf194f9d0 0x55caf195a410 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7effc4009900 tx=0x7effc40317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c9049c45775a1b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:38.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.323+0000 7effb37fe640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T21:56:38.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.323+0000 7effb37fe640 1 -- 172.21.15.1:0/2147675616 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc4035040 con 0x55caf194f9d0 2024-09-11T21:56:38.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.323+0000 7effb37fe640 1 -- 172.21.15.1:0/2147675616 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7effc4004240 con 0x55caf194f9d0 2024-09-11T21:56:38.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.323+0000 7effdcb3e740 1 -- 172.21.15.1:0/2147675616 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effcc01b3e0 con 0x55caf194f9d0 2024-09-11T21:56:38.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.324+0000 7effb37fe640 1 -- 172.21.15.1:0/2147675616 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7effc4004440 con 0x55caf194f9d0 2024-09-11T21:56:38.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.324+0000 7effdcb3e740 10 client.14976 fetch_fsmap learned FSMap version 27 2024-09-11T21:56:38.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.324+0000 7effdcb3e740 1 -- 172.21.15.1:0/2147675616 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55caf1969070 con 0x55caf194f9d0 2024-09-11T21:56:38.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.324+0000 7effb37fe640 1 -- 172.21.15.1:0/2147675616 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effcc01b3e0 con 0x55caf194f9d0 2024-09-11T21:56:38.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.324+0000 7effb37fe640 1 -- 172.21.15.1:0/2147675616 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7effc4004a80 con 0x55caf194f9d0 2024-09-11T21:56:38.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.324+0000 7effdcb3e740 10 client.14976 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T21:56:38.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.324+0000 7effdcb3e740 10 client.14976 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-11T21:56:38.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.324+0000 7effdcb3e740 20 client.14976 populate_metadata read hostname 'smithi001' 2024-09-11T21:56:38.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.324+0000 7effdcb3e740 1 --2- 172.21.15.1:0/2147675616 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x55caf196b6c0 0x55caf196cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:38.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.324+0000 7effdcb3e740 4 client.14976 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T21:56:38.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.324+0000 7effdcb3e740 1 -- 172.21.15.1:0/2147675616 --> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55caf18c3600 con 0x55caf196b6c0 2024-09-11T21:56:38.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.324+0000 7effcbfff640 1 --2- 172.21.15.1:0/2147675616 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x55caf196b6c0 0x55caf196cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:38.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.325+0000 7effcbfff640 1 --2- 172.21.15.1:0/2147675616 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x55caf196b6c0 0x55caf196cb30 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fb44a934c475ae6 server_cookie=d7b2851db3bc426a in_seq=0 out_seq=0 2024-09-11T21:56:38.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.325+0000 7effb37fe640 10 client.14976 ms_handle_connect on v2:172.21.15.161:6836/1799318656 2024-09-11T21:56:38.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.325+0000 7effb37fe640 1 -- 172.21.15.1:0/2147675616 <== mds.3 v2:172.21.15.161:6836/1799318656 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55caf18c3600 con 0x55caf196b6c0 2024-09-11T21:56:38.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.325+0000 7effb37fe640 10 client.14976 handle_command_reply: tid=0 2024-09-11T21:56:38.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.326+0000 7effdcb3e740 1 client.14976 shutdown 2024-09-11T21:56:38.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.326+0000 7effdcb3e740 1 -- 172.21.15.1:0/2147675616 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55caf194f9d0 msgr2=0x55caf195a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:38.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.326+0000 7effdcb3e740 1 --2- 172.21.15.1:0/2147675616 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55caf194f9d0 0x55caf195a410 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7effc4009900 tx=0x7effc40317e0 comp rx=0 tx=0).stop 2024-09-11T21:56:38.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.326+0000 7effdcb3e740 1 -- 172.21.15.1:0/2147675616 shutdown_connections 2024-09-11T21:56:38.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.326+0000 7effdcb3e740 1 --2- 172.21.15.1:0/2147675616 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x55caf196b6c0 0x55caf196cb30 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:38.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.326+0000 7effdcb3e740 1 --2- 172.21.15.1:0/2147675616 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55caf195abb0 0x55caf1957ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:38.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.326+0000 7effdcb3e740 1 --2- 172.21.15.1:0/2147675616 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55caf19502f0 0x55caf1957760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:38.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.326+0000 7effdcb3e740 1 --2- 172.21.15.1:0/2147675616 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55caf194f9d0 0x55caf195a410 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:38.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.326+0000 7effb37fe640 0 client.14976 ms_handle_reset on v2:172.21.15.161:6836/1799318656 2024-09-11T21:56:38.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.326+0000 7effdcb3e740 1 -- 172.21.15.1:0/2147675616 >> 172.21.15.1:0/2147675616 conn(0x55caf194bf60 msgr2=0x55caf1951c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:38.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.326+0000 7effdcb3e740 1 -- 172.21.15.1:0/2147675616 shutdown_connections 2024-09-11T21:56:38.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.326+0000 7effdcb3e740 1 -- 172.21.15.1:0/2147675616 wait complete. 2024-09-11T21:56:38.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.326+0000 7effdcb3e740 20 client.14976 trim_cache size 0 max 16384 2024-09-11T21:56:38.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.344+0000 7effdcb3e740 1 Processor -- start 2024-09-11T21:56:38.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.344+0000 7effdcb3e740 1 -- start start 2024-09-11T21:56:38.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.344+0000 7effdcb3e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55caf194f9d0 0x55caf196cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:38.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.344+0000 7effdcb3e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55caf19502f0 0x55caf196a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:38.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.344+0000 7effdcb3e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55caf195abb0 0x55caf1974e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:38.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.344+0000 7effdcb3e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55caf196ae70 con 0x55caf19502f0 2024-09-11T21:56:38.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.344+0000 7effdcb3e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55caf196b540 con 0x55caf194f9d0 2024-09-11T21:56:38.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.344+0000 7effdcb3e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55caf196bc10 con 0x55caf195abb0 2024-09-11T21:56:38.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.345+0000 7effcbfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55caf19502f0 0x55caf196a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:38.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.345+0000 7effcbfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55caf19502f0 0x55caf196a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41832/0 (socket says 172.21.15.1:41832) 2024-09-11T21:56:38.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.345+0000 7effcbfff640 1 -- 172.21.15.1:0/3148482482 learned_addr learned my addr 172.21.15.1:0/3148482482 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:56:38.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.345+0000 7effd8d02640 1 --2- 172.21.15.1:0/3148482482 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55caf195abb0 0x55caf1974e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:38.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.345+0000 7effd3fff640 1 --2- 172.21.15.1:0/3148482482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55caf194f9d0 0x55caf196cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:38.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.345+0000 7effcbfff640 1 -- 172.21.15.1:0/3148482482 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55caf195abb0 msgr2=0x55caf1974e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:56:38.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.345+0000 7effcbfff640 1 --2- 172.21.15.1:0/3148482482 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55caf195abb0 0x55caf1974e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:38.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.345+0000 7effcbfff640 1 -- 172.21.15.1:0/3148482482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55caf194f9d0 msgr2=0x55caf196cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:56:38.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.345+0000 7effcbfff640 1 --2- 172.21.15.1:0/3148482482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55caf194f9d0 0x55caf196cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:38.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.345+0000 7effcbfff640 1 -- 172.21.15.1:0/3148482482 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55caf19500e0 con 0x55caf19502f0 2024-09-11T21:56:38.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.345+0000 7effcbfff640 1 --2- 172.21.15.1:0/3148482482 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55caf19502f0 0x55caf196a530 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7effbc0986b0 tx=0x7effbc0d1ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e48e6eb373d596f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:38.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.345+0000 7effc9ffb640 1 -- 172.21.15.1:0/3148482482 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effbc09c3e0 con 0x55caf19502f0 2024-09-11T21:56:38.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.345+0000 7effc9ffb640 1 -- 172.21.15.1:0/3148482482 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effbc09bce0 con 0x55caf19502f0 2024-09-11T21:56:38.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.345+0000 7effc9ffb640 1 -- 172.21.15.1:0/3148482482 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effbc0a44c0 con 0x55caf19502f0 2024-09-11T21:56:38.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.346+0000 7effdcb3e740 1 -- 172.21.15.1:0/3148482482 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55caf19502f0 msgr2=0x55caf196a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:38.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.346+0000 7effdcb3e740 1 --2- 172.21.15.1:0/3148482482 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55caf19502f0 0x55caf196a530 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7effbc0986b0 tx=0x7effbc0d1ab0 comp rx=0 tx=0).stop 2024-09-11T21:56:38.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.346+0000 7effdcb3e740 1 -- 172.21.15.1:0/3148482482 shutdown_connections 2024-09-11T21:56:38.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.346+0000 7effdcb3e740 1 --2- 172.21.15.1:0/3148482482 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55caf195abb0 0x55caf1974e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:38.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.346+0000 7effdcb3e740 1 --2- 172.21.15.1:0/3148482482 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55caf19502f0 0x55caf196a530 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:38.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.346+0000 7effdcb3e740 1 --2- 172.21.15.1:0/3148482482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55caf194f9d0 0x55caf196cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:38.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.346+0000 7effdcb3e740 1 -- 172.21.15.1:0/3148482482 >> 172.21.15.1:0/3148482482 conn(0x55caf194bf60 msgr2=0x55caf194b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:38.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.346+0000 7effdcb3e740 1 -- 172.21.15.1:0/3148482482 shutdown_connections 2024-09-11T21:56:38.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.346+0000 7effdcb3e740 1 -- 172.21.15.1:0/3148482482 wait complete. 2024-09-11T21:56:38.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.346+0000 7effdcb3e740 1 -- start start 2024-09-11T21:56:38.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.347+0000 7effdcb3e740 1 Processor -- start 2024-09-11T21:56:38.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.347+0000 7effdcb3e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55caf194f9d0 0x55caf1972370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:38.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.347+0000 7effdcb3e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55caf19502f0 0x55caf19728b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:38.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.347+0000 7effdcb3e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55caf195abb0 0x55caf19d3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:38.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.347+0000 7effdcb3e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55caf196c9b0 con 0x55caf194f9d0 2024-09-11T21:56:38.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.347+0000 7effdcb3e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55caf196c2e0 con 0x55caf195abb0 2024-09-11T21:56:38.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.347+0000 7effdcb3e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55caf196d080 con 0x55caf19502f0 2024-09-11T21:56:38.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.347+0000 7effdcb3e740 10 client.0 _ll_register_callbacks cb 0x7effc4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:56:38.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.347+0000 7effd3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55caf194f9d0 0x55caf1972370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:38.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.347+0000 7effd3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55caf194f9d0 0x55caf1972370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41836/0 (socket says 172.21.15.1:41836) 2024-09-11T21:56:38.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.347+0000 7effd3fff640 1 -- 172.21.15.1:0/412714849 learned_addr learned my addr 172.21.15.1:0/412714849 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:56:38.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.347+0000 7effcbfff640 1 --2- 172.21.15.1:0/412714849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55caf19502f0 0x55caf19728b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:38.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.347+0000 7effd8d02640 1 --2- 172.21.15.1:0/412714849 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55caf195abb0 0x55caf19d3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:38.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.347+0000 7effd3fff640 1 -- 172.21.15.1:0/412714849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55caf19502f0 msgr2=0x55caf19728b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:56:38.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.347+0000 7effd3fff640 1 --2- 172.21.15.1:0/412714849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55caf19502f0 0x55caf19728b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:38.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.347+0000 7effd3fff640 1 -- 172.21.15.1:0/412714849 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55caf195abb0 msgr2=0x55caf19d3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:56:38.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.347+0000 7effd3fff640 1 --2- 172.21.15.1:0/412714849 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55caf195abb0 0x55caf19d3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:38.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.347+0000 7effd3fff640 1 -- 172.21.15.1:0/412714849 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55caf19d4410 con 0x55caf194f9d0 2024-09-11T21:56:38.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.347+0000 7effd3fff640 1 --2- 172.21.15.1:0/412714849 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55caf194f9d0 0x55caf1972370 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7effc4006010 tx=0x7effc4004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=822f05e0b19be579 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:38.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.348+0000 7effb37fe640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T21:56:38.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.348+0000 7effb37fe640 1 -- 172.21.15.1:0/412714849 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc4035070 con 0x55caf194f9d0 2024-09-11T21:56:38.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.348+0000 7effb37fe640 1 -- 172.21.15.1:0/412714849 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7effc4005200 con 0x55caf194f9d0 2024-09-11T21:56:38.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.348+0000 7effdcb3e740 1 -- 172.21.15.1:0/412714849 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effbc09bce0 con 0x55caf194f9d0 2024-09-11T21:56:38.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.348+0000 7effb37fe640 1 -- 172.21.15.1:0/412714849 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7effc4005d50 con 0x55caf194f9d0 2024-09-11T21:56:38.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.348+0000 7effdcb3e740 10 client.14982 fetch_fsmap learned FSMap version 27 2024-09-11T21:56:38.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.348+0000 7effdcb3e740 1 -- 172.21.15.1:0/412714849 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55caf19d56f0 con 0x55caf194f9d0 2024-09-11T21:56:38.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.348+0000 7effb37fe640 1 -- 172.21.15.1:0/412714849 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effbc09bce0 con 0x55caf194f9d0 2024-09-11T21:56:38.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.348+0000 7effb37fe640 1 -- 172.21.15.1:0/412714849 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7effc4003fb0 con 0x55caf194f9d0 2024-09-11T21:56:38.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.348+0000 7effdcb3e740 10 client.14982 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T21:56:38.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.348+0000 7effdcb3e740 10 client.14982 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-11T21:56:38.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.348+0000 7effdcb3e740 20 client.14982 populate_metadata read hostname 'smithi001' 2024-09-11T21:56:38.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.348+0000 7effdcb3e740 1 --2- 172.21.15.1:0/412714849 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x55caf19da320 0x55caf19d9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:38.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.348+0000 7effdcb3e740 4 client.14982 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T21:56:38.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.348+0000 7effdcb3e740 1 -- 172.21.15.1:0/412714849 --> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] -- command(tid 0: {"prefix": "status"}) -- 0x55caf18bdb10 con 0x55caf19da320 2024-09-11T21:56:38.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.349+0000 7effcbfff640 1 --2- 172.21.15.1:0/412714849 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x55caf19da320 0x55caf19d9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:38.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.349+0000 7effcbfff640 1 --2- 172.21.15.1:0/412714849 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x55caf19da320 0x55caf19d9030 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dff1700ec8258789 server_cookie=76c6c7030d4dd1f2 in_seq=0 out_seq=0 2024-09-11T21:56:38.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.349+0000 7effb37fe640 10 client.14982 ms_handle_connect on v2:172.21.15.161:6836/1799318656 2024-09-11T21:56:38.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.349+0000 7effb37fe640 1 -- 172.21.15.1:0/412714849 <== mds.3 v2:172.21.15.161:6836/1799318656 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55caf18bdb10 con 0x55caf19da320 2024-09-11T21:56:38.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.349+0000 7effb37fe640 10 client.14982 handle_command_reply: tid=0 2024-09-11T21:56:38.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.349+0000 7effdcb3e740 1 client.14982 shutdown 2024-09-11T21:56:38.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.349+0000 7effdcb3e740 1 -- 172.21.15.1:0/412714849 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55caf194f9d0 msgr2=0x55caf1972370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:38.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.349+0000 7effdcb3e740 1 --2- 172.21.15.1:0/412714849 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55caf194f9d0 0x55caf1972370 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7effc4006010 tx=0x7effc4004970 comp rx=0 tx=0).stop 2024-09-11T21:56:38.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.349+0000 7effdcb3e740 1 -- 172.21.15.1:0/412714849 shutdown_connections 2024-09-11T21:56:38.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.349+0000 7effdcb3e740 1 --2- 172.21.15.1:0/412714849 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x55caf19da320 0x55caf19d9030 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:38.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.349+0000 7effdcb3e740 1 --2- 172.21.15.1:0/412714849 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55caf195abb0 0x55caf19d3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:38.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.349+0000 7effdcb3e740 1 --2- 172.21.15.1:0/412714849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55caf19502f0 0x55caf19728b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:38.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.349+0000 7effdcb3e740 1 --2- 172.21.15.1:0/412714849 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55caf194f9d0 0x55caf1972370 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:38.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.349+0000 7effdcb3e740 1 -- 172.21.15.1:0/412714849 >> 172.21.15.1:0/412714849 conn(0x55caf194bf60 msgr2=0x55caf1958db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:38.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.349+0000 7effb37fe640 0 client.14982 ms_handle_reset on v2:172.21.15.161:6836/1799318656 2024-09-11T21:56:38.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.350+0000 7effdcb3e740 1 -- 172.21.15.1:0/412714849 shutdown_connections 2024-09-11T21:56:38.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.350+0000 7effdcb3e740 1 -- 172.21.15.1:0/412714849 wait complete. 2024-09-11T21:56:38.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.350+0000 7effdcb3e740 20 client.14982 trim_cache size 0 max 16384 2024-09-11T21:56:38.358 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-11T21:56:38.358 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "3ffca83c-7087-11ef-bcea-c7b262605968", 2024-09-11T21:56:38.358 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 3, 2024-09-11T21:56:38.358 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 24466, 2024-09-11T21:56:38.358 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-09-11T21:56:38.359 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-09-11T21:56:38.359 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-09-11T21:56:38.359 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 68.209246442999998, 2024-09-11T21:56:38.359 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 27, 2024-09-11T21:56:38.359 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 86, 2024-09-11T21:56:38.359 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 84, 2024-09-11T21:56:38.359 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 102.02565816800001 2024-09-11T21:56:38.359 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-11T21:56:38.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.350+0000 7effda765640 1 -- 172.21.15.1:0/3891769111 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7effc00a2c10 msgr2=0x7effc00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:38.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.350+0000 7effda765640 1 --2- 172.21.15.1:0/3891769111 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7effc00a2c10 0x7effc00a50d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7effcc01f000 tx=0x7effcc020000 comp rx=0 tx=0).stop 2024-09-11T21:56:38.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.350+0000 7effda765640 1 -- 172.21.15.1:0/3891769111 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7effd4103660 msgr2=0x7effd41989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:38.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.350+0000 7effda765640 1 --2- 172.21.15.1:0/3891769111 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7effd4103660 0x7effd41989b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7effbc00e9f0 tx=0x7effbc00eeb0 comp rx=0 tx=0).stop 2024-09-11T21:56:38.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.350+0000 7effda765640 1 -- 172.21.15.1:0/3891769111 shutdown_connections 2024-09-11T21:56:38.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.350+0000 7effda765640 1 --2- 172.21.15.1:0/3891769111 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7effc00a2c10 0x7effc00a50d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:38.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.350+0000 7effda765640 1 --2- 172.21.15.1:0/3891769111 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7effd410c170 0x7effd419cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:38.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.350+0000 7effda765640 1 --2- 172.21.15.1:0/3891769111 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7effd4103660 0x7effd41989b0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:38.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.350+0000 7effda765640 1 --2- 172.21.15.1:0/3891769111 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7effd4102d40 0x7effd4198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:38.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.350+0000 7effda765640 1 -- 172.21.15.1:0/3891769111 >> 172.21.15.1:0/3891769111 conn(0x7effd40fc7a0 msgr2=0x7effd40fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:38.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.350+0000 7effda765640 1 -- 172.21.15.1:0/3891769111 shutdown_connections 2024-09-11T21:56:38.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:38.350+0000 7effda765640 1 -- 172.21.15.1:0/3891769111 wait complete. 2024-09-11T21:56:38.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:38 smithi161 ceph-mon[35615]: pgmap v322: 97 pgs: 97 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 5.6 KiB/s, 0 objects/s recovering 2024-09-11T21:56:38.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:38 smithi001 ceph-mon[30935]: pgmap v322: 97 pgs: 97 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 5.6 KiB/s, 0 objects/s recovering 2024-09-11T21:56:38.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:38 smithi053 ceph-mon[34908]: pgmap v322: 97 pgs: 97 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 5.6 KiB/s, 0 objects/s recovering 2024-09-11T21:56:39.144 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.h status 2024-09-11T21:56:39.402 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:56:39.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.865+0000 7f1aa30f2640 1 -- 172.21.15.1:0/1110560555 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a9c101420 msgr2=0x7f1a9c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:39.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.865+0000 7f1aa30f2640 1 --2- 172.21.15.1:0/1110560555 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a9c101420 0x7f1a9c10d950 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f1a94009cd0 tx=0x7f1a9402a8e0 comp rx=0 tx=0).stop 2024-09-11T21:56:39.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.865+0000 7f1aa30f2640 1 -- 172.21.15.1:0/1110560555 shutdown_connections 2024-09-11T21:56:39.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.865+0000 7f1aa30f2640 1 --2- 172.21.15.1:0/1110560555 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a9c101420 0x7f1a9c10d950 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:39.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.865+0000 7f1aa30f2640 1 --2- 172.21.15.1:0/1110560555 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a9c100a80 0x7f1a9c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:39.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.865+0000 7f1aa30f2640 1 --2- 172.21.15.1:0/1110560555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a9c106a80 0x7f1a9c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:39.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.865+0000 7f1aa30f2640 1 -- 172.21.15.1:0/1110560555 >> 172.21.15.1:0/1110560555 conn(0x7f1a9c0fc7a0 msgr2=0x7f1a9c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:39.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.865+0000 7f1aa30f2640 1 -- 172.21.15.1:0/1110560555 shutdown_connections 2024-09-11T21:56:39.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.865+0000 7f1aa30f2640 1 -- 172.21.15.1:0/1110560555 wait complete. 2024-09-11T21:56:39.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.866+0000 7f1aa30f2640 1 Processor -- start 2024-09-11T21:56:39.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.866+0000 7f1aa30f2640 1 -- start start 2024-09-11T21:56:39.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.866+0000 7f1aa30f2640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a9c100a80 0x7f1a9c197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:39.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.866+0000 7f1aa30f2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a9c101420 0x7f1a9c1978c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:39.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.866+0000 7f1aa30f2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a9c106a80 0x7f1a9c19bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:39.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.866+0000 7f1aa30f2640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1a9c110e80 con 0x7f1a9c101420 2024-09-11T21:56:39.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.866+0000 7f1aa30f2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1a9c110d00 con 0x7f1a9c106a80 2024-09-11T21:56:39.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.866+0000 7f1aa30f2640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1a9c111000 con 0x7f1a9c100a80 2024-09-11T21:56:39.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.866+0000 7f1a9bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a9c101420 0x7f1a9c1978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:39.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.866+0000 7f1a9bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a9c101420 0x7f1a9c1978c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41856/0 (socket says 172.21.15.1:41856) 2024-09-11T21:56:39.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.866+0000 7f1a9bfff640 1 -- 172.21.15.1:0/2582218586 learned_addr learned my addr 172.21.15.1:0/2582218586 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:56:39.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.866+0000 7f1aa0e8e640 1 --2- 172.21.15.1:0/2582218586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a9c100a80 0x7f1a9c197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:39.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.866+0000 7f1aa168f640 1 --2- 172.21.15.1:0/2582218586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a9c106a80 0x7f1a9c19bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:39.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.866+0000 7f1a9bfff640 1 -- 172.21.15.1:0/2582218586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a9c100a80 msgr2=0x7f1a9c197380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:39.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.866+0000 7f1a9bfff640 1 --2- 172.21.15.1:0/2582218586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a9c100a80 0x7f1a9c197380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:39.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.866+0000 7f1a9bfff640 1 -- 172.21.15.1:0/2582218586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a9c106a80 msgr2=0x7f1a9c19bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:39.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.866+0000 7f1a9bfff640 1 --2- 172.21.15.1:0/2582218586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a9c106a80 0x7f1a9c19bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:39.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.866+0000 7f1a9bfff640 1 -- 172.21.15.1:0/2582218586 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a9c19c330 con 0x7f1a9c101420 2024-09-11T21:56:39.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.867+0000 7f1a9bfff640 1 --2- 172.21.15.1:0/2582218586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a9c101420 0x7f1a9c1978c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f1a8400d950 tx=0x7f1a8400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:39.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.867+0000 7f1a99ffb640 1 -- 172.21.15.1:0/2582218586 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a84017070 con 0x7f1a9c101420 2024-09-11T21:56:39.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.867+0000 7f1a99ffb640 1 -- 172.21.15.1:0/2582218586 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a84009c30 con 0x7f1a9c101420 2024-09-11T21:56:39.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.867+0000 7f1a99ffb640 1 -- 172.21.15.1:0/2582218586 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a8400f6b0 con 0x7f1a9c101420 2024-09-11T21:56:39.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.867+0000 7f1aa30f2640 1 -- 172.21.15.1:0/2582218586 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a9c19c620 con 0x7f1a9c101420 2024-09-11T21:56:39.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.867+0000 7f1aa30f2640 1 -- 172.21.15.1:0/2582218586 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a9c1a3f00 con 0x7f1a9c101420 2024-09-11T21:56:39.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.869+0000 7f1a99ffb640 1 -- 172.21.15.1:0/2582218586 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f1a840109d0 con 0x7f1a9c101420 2024-09-11T21:56:39.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.870+0000 7f1aa54cb740 1 Processor -- start 2024-09-11T21:56:39.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.870+0000 7f1aa54cb740 1 -- start start 2024-09-11T21:56:39.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.870+0000 7f1a99ffb640 1 --2- 172.21.15.1:0/2582218586 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f1a880a2c90 0x7f1a880a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:39.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.870+0000 7f1aa54cb740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c13282890 0x561c13282c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:39.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.870+0000 7f1aa54cb740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561c132831b0 0x561c13291c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:39.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.870+0000 7f1a99ffb640 1 -- 172.21.15.1:0/2582218586 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f1a840cee40 con 0x7f1a9c101420 2024-09-11T21:56:39.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.870+0000 7f1aa54cb740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561c132921b0 0x561c132945a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:39.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.870+0000 7f1aa54cb740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x561c131e9090 con 0x561c132921b0 2024-09-11T21:56:39.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.870+0000 7f1aa54cb740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x561c13190a70 con 0x561c132831b0 2024-09-11T21:56:39.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.870+0000 7f1aa54cb740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x561c131e5050 con 0x561c13282890 2024-09-11T21:56:39.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.870+0000 7f1aa168f640 1 --2- 172.21.15.1:0/2582218586 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f1a880a2c90 0x7f1a880a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:39.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.870+0000 7f1aa168f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561c132921b0 0x561c132945a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:39.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.870+0000 7f1aa168f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561c132921b0 0x561c132945a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41870/0 (socket says 172.21.15.1:41870) 2024-09-11T21:56:39.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.870+0000 7f1aa168f640 1 -- 172.21.15.1:0/3067716523 learned_addr learned my addr 172.21.15.1:0/3067716523 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:56:39.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.870+0000 7f1aa0e8e640 1 --2- 172.21.15.1:0/3067716523 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c13282890 0x561c13282c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:39.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.870+0000 7f1a9bfff640 1 --2- 172.21.15.1:0/3067716523 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561c132831b0 0x561c13291c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:39.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.870+0000 7f1aa0e8e640 1 -- 172.21.15.1:0/3067716523 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561c132831b0 msgr2=0x561c13291c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:56:39.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.870+0000 7f1aa0e8e640 1 --2- 172.21.15.1:0/3067716523 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561c132831b0 0x561c13291c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:39.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.870+0000 7f1aa168f640 1 --2- 172.21.15.1:0/2582218586 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f1a880a2c90 0x7f1a880a5150 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1a94009cd0 tx=0x7f1a940316c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:39.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.870+0000 7f1aa0e8e640 1 -- 172.21.15.1:0/3067716523 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561c132921b0 msgr2=0x561c132945a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:56:39.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.870+0000 7f1aa0e8e640 1 --2- 172.21.15.1:0/3067716523 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561c132921b0 0x561c132945a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:39.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.871+0000 7f1aa0e8e640 1 -- 172.21.15.1:0/3067716523 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c1319d130 con 0x561c13282890 2024-09-11T21:56:39.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.871+0000 7f1aa168f640 1 --2- 172.21.15.1:0/3067716523 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561c132921b0 0x561c132945a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T21:56:39.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.871+0000 7f1aa0e8e640 1 --2- 172.21.15.1:0/3067716523 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c13282890 0x561c13282c70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c00bb70 tx=0x7f1a8c0107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e74fdfbabcc0ea server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:39.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.871+0000 7f1a92ffd640 1 -- 172.21.15.1:0/3067716523 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c00be60 con 0x561c13282890 2024-09-11T21:56:39.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.871+0000 7f1a92ffd640 1 -- 172.21.15.1:0/3067716523 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a8c0035f0 con 0x561c13282890 2024-09-11T21:56:39.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.871+0000 7f1a92ffd640 1 -- 172.21.15.1:0/3067716523 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c0116a0 con 0x561c13282890 2024-09-11T21:56:39.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.872+0000 7f1aa54cb740 1 -- 172.21.15.1:0/3067716523 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c13282890 msgr2=0x561c13282c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:39.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.872+0000 7f1aa54cb740 1 --2- 172.21.15.1:0/3067716523 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c13282890 0x561c13282c70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c00bb70 tx=0x7f1a8c0107a0 comp rx=0 tx=0).stop 2024-09-11T21:56:39.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.872+0000 7f1aa54cb740 1 -- 172.21.15.1:0/3067716523 shutdown_connections 2024-09-11T21:56:39.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.872+0000 7f1aa54cb740 1 --2- 172.21.15.1:0/3067716523 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561c132921b0 0x561c132945a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:39.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.872+0000 7f1aa54cb740 1 --2- 172.21.15.1:0/3067716523 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561c132831b0 0x561c13291c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:39.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.872+0000 7f1aa54cb740 1 --2- 172.21.15.1:0/3067716523 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c13282890 0x561c13282c70 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:39.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.872+0000 7f1aa54cb740 1 -- 172.21.15.1:0/3067716523 >> 172.21.15.1:0/3067716523 conn(0x561c1327ef60 msgr2=0x561c1327f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:39.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.872+0000 7f1aa54cb740 1 -- 172.21.15.1:0/3067716523 shutdown_connections 2024-09-11T21:56:39.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.872+0000 7f1aa54cb740 1 -- 172.21.15.1:0/3067716523 wait complete. 2024-09-11T21:56:39.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.872+0000 7f1aa54cb740 1 -- start start 2024-09-11T21:56:39.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.872+0000 7f1aa54cb740 1 Processor -- start 2024-09-11T21:56:39.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.873+0000 7f1aa54cb740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c13282890 0x561c13291a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:39.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.873+0000 7f1aa54cb740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561c132831b0 0x561c1328edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:39.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.873+0000 7f1aa54cb740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561c132921b0 0x561c1328f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:39.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.873+0000 7f1aa54cb740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x561c13294ba0 con 0x561c132831b0 2024-09-11T21:56:39.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.873+0000 7f1aa54cb740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x561c131a3f20 con 0x561c132921b0 2024-09-11T21:56:39.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.873+0000 7f1aa54cb740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x561c13294d20 con 0x561c13282890 2024-09-11T21:56:39.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.873+0000 7f1aa54cb740 10 client.0 _ll_register_callbacks cb 0x561c1316d030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:56:39.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.873+0000 7f1a9bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561c132831b0 0x561c1328edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:39.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.873+0000 7f1a9bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561c132831b0 0x561c1328edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41874/0 (socket says 172.21.15.1:41874) 2024-09-11T21:56:39.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.873+0000 7f1a9bfff640 1 -- 172.21.15.1:0/399362277 learned_addr learned my addr 172.21.15.1:0/399362277 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:56:39.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.873+0000 7f1aa0e8e640 1 --2- 172.21.15.1:0/399362277 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c13282890 0x561c13291a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:39.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.873+0000 7f1aa168f640 1 --2- 172.21.15.1:0/399362277 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561c132921b0 0x561c1328f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:39.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.873+0000 7f1a9bfff640 1 -- 172.21.15.1:0/399362277 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c13282890 msgr2=0x561c13291a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:56:39.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.873+0000 7f1a9bfff640 1 --2- 172.21.15.1:0/399362277 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c13282890 0x561c13291a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:39.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.873+0000 7f1a9bfff640 1 -- 172.21.15.1:0/399362277 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561c132921b0 msgr2=0x561c1328f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:56:39.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.873+0000 7f1a9bfff640 1 --2- 172.21.15.1:0/399362277 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561c132921b0 0x561c1328f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:39.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.873+0000 7f1a9bfff640 1 -- 172.21.15.1:0/399362277 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c12f5a2b0 con 0x561c132831b0 2024-09-11T21:56:39.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.873+0000 7f1aa168f640 1 --2- 172.21.15.1:0/399362277 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561c132921b0 0x561c1328f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:56:39.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.873+0000 7f1a9bfff640 1 --2- 172.21.15.1:0/399362277 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561c132831b0 0x561c1328edc0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1a84009610 tx=0x7f1a84005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fc6d7cd8fe93e35 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:39.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.873+0000 7f1a7bfff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T21:56:39.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.874+0000 7f1a7bfff640 1 -- 172.21.15.1:0/399362277 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a840b4060 con 0x561c132831b0 2024-09-11T21:56:39.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.874+0000 7f1a7bfff640 1 -- 172.21.15.1:0/399362277 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f1a840b4200 con 0x561c132831b0 2024-09-11T21:56:39.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.874+0000 7f1aa54cb740 1 -- 172.21.15.1:0/399362277 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a8c0035f0 con 0x561c132831b0 2024-09-11T21:56:39.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.874+0000 7f1a7bfff640 1 -- 172.21.15.1:0/399362277 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1a84020040 con 0x561c132831b0 2024-09-11T21:56:39.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.874+0000 7f1aa54cb740 10 client.14994 fetch_fsmap learned FSMap version 27 2024-09-11T21:56:39.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.874+0000 7f1aa54cb740 1 -- 172.21.15.1:0/399362277 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561c132a0530 con 0x561c132831b0 2024-09-11T21:56:39.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.874+0000 7f1a7bfff640 1 -- 172.21.15.1:0/399362277 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a8c0035f0 con 0x561c132831b0 2024-09-11T21:56:39.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.874+0000 7f1a7bfff640 1 -- 172.21.15.1:0/399362277 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f1a8409a9a0 con 0x561c132831b0 2024-09-11T21:56:39.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.874+0000 7f1aa54cb740 10 client.14994 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T21:56:39.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.874+0000 7f1aa54cb740 10 client.14994 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-11T21:56:39.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.874+0000 7f1aa54cb740 20 client.14994 populate_metadata read hostname 'smithi001' 2024-09-11T21:56:39.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.874+0000 7f1aa54cb740 1 --2- 172.21.15.1:0/399362277 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x561c132a2ad0 0x561c132a3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:39.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.874+0000 7f1aa54cb740 4 client.14994 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T21:56:39.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.874+0000 7f1aa54cb740 1 -- 172.21.15.1:0/399362277 --> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c131f6600 con 0x561c132a2ad0 2024-09-11T21:56:39.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.875+0000 7f1aa0e8e640 1 --2- 172.21.15.1:0/399362277 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x561c132a2ad0 0x561c132a3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:39.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.875+0000 7f1aa0e8e640 1 --2- 172.21.15.1:0/399362277 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x561c132a2ad0 0x561c132a3f40 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2f58e6daadd53e86 server_cookie=51a8f39b8f8057cf in_seq=0 out_seq=0 2024-09-11T21:56:39.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.875+0000 7f1a7bfff640 10 client.14994 ms_handle_connect on v2:172.21.15.53:6836/3457131830 2024-09-11T21:56:39.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.876+0000 7f1a7bfff640 1 -- 172.21.15.1:0/399362277 <== mds.2 v2:172.21.15.53:6836/3457131830 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561c131f6600 con 0x561c132a2ad0 2024-09-11T21:56:39.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.876+0000 7f1a7bfff640 10 client.14994 handle_command_reply: tid=0 2024-09-11T21:56:39.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.876+0000 7f1aa54cb740 1 client.14994 shutdown 2024-09-11T21:56:39.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.876+0000 7f1aa54cb740 1 -- 172.21.15.1:0/399362277 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561c132831b0 msgr2=0x561c1328edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:39.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.876+0000 7f1aa54cb740 1 --2- 172.21.15.1:0/399362277 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561c132831b0 0x561c1328edc0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1a84009610 tx=0x7f1a84005f90 comp rx=0 tx=0).stop 2024-09-11T21:56:39.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.876+0000 7f1aa54cb740 1 -- 172.21.15.1:0/399362277 shutdown_connections 2024-09-11T21:56:39.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.876+0000 7f1aa54cb740 1 --2- 172.21.15.1:0/399362277 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x561c132a2ad0 0x561c132a3f40 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:39.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.876+0000 7f1aa54cb740 1 --2- 172.21.15.1:0/399362277 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561c132921b0 0x561c1328f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:39.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.876+0000 7f1aa54cb740 1 --2- 172.21.15.1:0/399362277 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561c132831b0 0x561c1328edc0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:39.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.876+0000 7f1a7bfff640 0 client.14994 ms_handle_reset on v2:172.21.15.53:6836/3457131830 2024-09-11T21:56:39.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.876+0000 7f1aa54cb740 1 --2- 172.21.15.1:0/399362277 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c13282890 0x561c13291a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:39.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.876+0000 7f1aa54cb740 1 -- 172.21.15.1:0/399362277 >> 172.21.15.1:0/399362277 conn(0x561c1327ef60 msgr2=0x561c13284cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:39.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.876+0000 7f1aa54cb740 1 -- 172.21.15.1:0/399362277 shutdown_connections 2024-09-11T21:56:39.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.876+0000 7f1aa54cb740 1 -- 172.21.15.1:0/399362277 wait complete. 2024-09-11T21:56:39.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.876+0000 7f1aa54cb740 20 client.14994 trim_cache size 0 max 16384 2024-09-11T21:56:39.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.894+0000 7f1aa54cb740 1 Processor -- start 2024-09-11T21:56:39.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.894+0000 7f1aa54cb740 1 -- start start 2024-09-11T21:56:39.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.894+0000 7f1aa54cb740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561c13282890 0x561c1328e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:39.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.894+0000 7f1aa54cb740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561c132831b0 0x561c13290510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:39.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.894+0000 7f1aa54cb740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c132921b0 0x561c13290a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:39.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.894+0000 7f1aa54cb740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x561c132a2280 con 0x561c132831b0 2024-09-11T21:56:39.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.894+0000 7f1aa54cb740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x561c132a2950 con 0x561c13282890 2024-09-11T21:56:39.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.894+0000 7f1aa54cb740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x561c132a3020 con 0x561c132921b0 2024-09-11T21:56:39.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.894+0000 7f1a9bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561c132831b0 0x561c13290510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:39.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.894+0000 7f1a9bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561c132831b0 0x561c13290510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41884/0 (socket says 172.21.15.1:41884) 2024-09-11T21:56:39.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.894+0000 7f1a9bfff640 1 -- 172.21.15.1:0/1494412091 learned_addr learned my addr 172.21.15.1:0/1494412091 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:56:39.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.895+0000 7f1aa0e8e640 1 --2- 172.21.15.1:0/1494412091 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561c13282890 0x561c1328e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:39.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.895+0000 7f1a9bfff640 1 -- 172.21.15.1:0/1494412091 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c132921b0 msgr2=0x561c13290a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:56:39.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.895+0000 7f1a9bfff640 1 --2- 172.21.15.1:0/1494412091 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c132921b0 0x561c13290a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:39.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.895+0000 7f1a9bfff640 1 -- 172.21.15.1:0/1494412091 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561c13282890 msgr2=0x561c1328e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:56:39.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.895+0000 7f1a9bfff640 1 --2- 172.21.15.1:0/1494412091 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561c13282890 0x561c1328e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:39.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.895+0000 7f1a9bfff640 1 -- 172.21.15.1:0/1494412091 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c1327fec0 con 0x561c132831b0 2024-09-11T21:56:39.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.895+0000 7f1a9bfff640 1 --2- 172.21.15.1:0/1494412091 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561c132831b0 0x561c13290510 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1a840a3c00 tx=0x7f1a8409b640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d66b807dfdd70ada server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:39.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.895+0000 7f1a927fc640 1 -- 172.21.15.1:0/1494412091 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a840b4040 con 0x561c132831b0 2024-09-11T21:56:39.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.895+0000 7f1a927fc640 1 -- 172.21.15.1:0/1494412091 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a8409bbc0 con 0x561c132831b0 2024-09-11T21:56:39.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.895+0000 7f1a927fc640 1 -- 172.21.15.1:0/1494412091 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a840b41e0 con 0x561c132831b0 2024-09-11T21:56:39.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.896+0000 7f1aa54cb740 1 -- 172.21.15.1:0/1494412091 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561c132831b0 msgr2=0x561c13290510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:39.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.896+0000 7f1aa54cb740 1 --2- 172.21.15.1:0/1494412091 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561c132831b0 0x561c13290510 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1a840a3c00 tx=0x7f1a8409b640 comp rx=0 tx=0).stop 2024-09-11T21:56:39.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.896+0000 7f1aa54cb740 1 -- 172.21.15.1:0/1494412091 shutdown_connections 2024-09-11T21:56:39.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.896+0000 7f1aa54cb740 1 --2- 172.21.15.1:0/1494412091 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c132921b0 0x561c13290a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:39.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.896+0000 7f1aa54cb740 1 --2- 172.21.15.1:0/1494412091 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561c132831b0 0x561c13290510 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:39.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.896+0000 7f1aa54cb740 1 --2- 172.21.15.1:0/1494412091 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561c13282890 0x561c1328e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:39.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.896+0000 7f1aa54cb740 1 -- 172.21.15.1:0/1494412091 >> 172.21.15.1:0/1494412091 conn(0x561c1327ef60 msgr2=0x561c1328d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:39.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.896+0000 7f1aa54cb740 1 -- 172.21.15.1:0/1494412091 shutdown_connections 2024-09-11T21:56:39.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.896+0000 7f1aa54cb740 1 -- 172.21.15.1:0/1494412091 wait complete. 2024-09-11T21:56:39.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.896+0000 7f1aa54cb740 1 -- start start 2024-09-11T21:56:39.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.896+0000 7f1aa54cb740 1 Processor -- start 2024-09-11T21:56:39.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.896+0000 7f1aa54cb740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561c13282890 0x561c132868b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:39.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.896+0000 7f1aa54cb740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c132831b0 0x561c13286df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:39.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.896+0000 7f1aa54cb740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561c132921b0 0x561c132ac190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:39.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.896+0000 7f1aa54cb740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x561c132a3dc0 con 0x561c132921b0 2024-09-11T21:56:39.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.897+0000 7f1aa54cb740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x561c132a36f0 con 0x561c13282890 2024-09-11T21:56:39.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.897+0000 7f1aa54cb740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x561c132a4490 con 0x561c132831b0 2024-09-11T21:56:39.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.897+0000 7f1aa54cb740 10 client.0 _ll_register_callbacks cb 0x7f1a8409b000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:56:39.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.897+0000 7f1aa168f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561c132921b0 0x561c132ac190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:39.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.897+0000 7f1aa168f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561c132921b0 0x561c132ac190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41900/0 (socket says 172.21.15.1:41900) 2024-09-11T21:56:39.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.897+0000 7f1aa168f640 1 -- 172.21.15.1:0/3627828164 learned_addr learned my addr 172.21.15.1:0/3627828164 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:56:39.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.897+0000 7f1aa0e8e640 1 --2- 172.21.15.1:0/3627828164 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561c13282890 0x561c132868b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:39.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.897+0000 7f1a9bfff640 1 --2- 172.21.15.1:0/3627828164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c132831b0 0x561c13286df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:39.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.897+0000 7f1aa168f640 1 -- 172.21.15.1:0/3627828164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c132831b0 msgr2=0x561c13286df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:56:39.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.897+0000 7f1aa168f640 1 --2- 172.21.15.1:0/3627828164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c132831b0 0x561c13286df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:39.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.897+0000 7f1aa168f640 1 -- 172.21.15.1:0/3627828164 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561c13282890 msgr2=0x561c132868b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:56:39.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.897+0000 7f1aa168f640 1 --2- 172.21.15.1:0/3627828164 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561c13282890 0x561c132868b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:39.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.897+0000 7f1aa168f640 1 -- 172.21.15.1:0/3627828164 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c132ac9c0 con 0x561c132921b0 2024-09-11T21:56:39.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.897+0000 7f1a9bfff640 1 --2- 172.21.15.1:0/3627828164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c132831b0 0x561c13286df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T21:56:39.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.897+0000 7f1aa0e8e640 1 --2- 172.21.15.1:0/3627828164 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561c13282890 0x561c132868b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:56:39.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.897+0000 7f1aa168f640 1 --2- 172.21.15.1:0/3627828164 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561c132921b0 0x561c132ac190 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1a94005fe0 tx=0x7f1a940095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=112c7256b55ba3c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:39.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.897+0000 7f1a7bfff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T21:56:39.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.897+0000 7f1a7bfff640 1 -- 172.21.15.1:0/3627828164 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a940305a0 con 0x561c132921b0 2024-09-11T21:56:39.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.897+0000 7f1a7bfff640 1 -- 172.21.15.1:0/3627828164 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f1a94030740 con 0x561c132921b0 2024-09-11T21:56:39.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.897+0000 7f1aa54cb740 1 -- 172.21.15.1:0/3627828164 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a8409bbc0 con 0x561c132921b0 2024-09-11T21:56:39.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.897+0000 7f1a7bfff640 1 -- 172.21.15.1:0/3627828164 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1a94030940 con 0x561c132921b0 2024-09-11T21:56:39.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.898+0000 7f1aa54cb740 10 client.15000 fetch_fsmap learned FSMap version 27 2024-09-11T21:56:39.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.898+0000 7f1aa54cb740 1 -- 172.21.15.1:0/3627828164 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561c1327e480 con 0x561c132921b0 2024-09-11T21:56:39.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.898+0000 7f1a7bfff640 1 -- 172.21.15.1:0/3627828164 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a8409bbc0 con 0x561c132921b0 2024-09-11T21:56:39.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.898+0000 7f1a7bfff640 1 -- 172.21.15.1:0/3627828164 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f1a94051070 con 0x561c132921b0 2024-09-11T21:56:39.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.898+0000 7f1aa54cb740 10 client.15000 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T21:56:39.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.898+0000 7f1aa54cb740 10 client.15000 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-11T21:56:39.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.898+0000 7f1aa54cb740 20 client.15000 populate_metadata read hostname 'smithi001' 2024-09-11T21:56:39.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.898+0000 7f1aa54cb740 1 --2- 172.21.15.1:0/3627828164 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x561c133119a0 0x561c133106b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:39.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.898+0000 7f1aa54cb740 4 client.15000 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T21:56:39.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.898+0000 7f1aa54cb740 1 -- 172.21.15.1:0/3627828164 --> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] -- command(tid 0: {"prefix": "status"}) -- 0x561c131f0b10 con 0x561c133119a0 2024-09-11T21:56:39.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.898+0000 7f1aa0e8e640 1 --2- 172.21.15.1:0/3627828164 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x561c133119a0 0x561c133106b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:39.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.899+0000 7f1aa0e8e640 1 --2- 172.21.15.1:0/3627828164 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x561c133119a0 0x561c133106b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d82648379208b4bf server_cookie=ef2eb7028b2af5b3 in_seq=0 out_seq=0 2024-09-11T21:56:39.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.899+0000 7f1a7bfff640 10 client.15000 ms_handle_connect on v2:172.21.15.53:6836/3457131830 2024-09-11T21:56:39.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.899+0000 7f1a7bfff640 1 -- 172.21.15.1:0/3627828164 <== mds.2 v2:172.21.15.53:6836/3457131830 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x561c131f0b10 con 0x561c133119a0 2024-09-11T21:56:39.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.899+0000 7f1a7bfff640 10 client.15000 handle_command_reply: tid=0 2024-09-11T21:56:39.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.899+0000 7f1aa54cb740 1 client.15000 shutdown 2024-09-11T21:56:39.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.899+0000 7f1aa54cb740 1 -- 172.21.15.1:0/3627828164 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561c132921b0 msgr2=0x561c132ac190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:39.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.899+0000 7f1aa54cb740 1 --2- 172.21.15.1:0/3627828164 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561c132921b0 0x561c132ac190 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1a94005fe0 tx=0x7f1a940095c0 comp rx=0 tx=0).stop 2024-09-11T21:56:39.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.899+0000 7f1aa54cb740 1 -- 172.21.15.1:0/3627828164 shutdown_connections 2024-09-11T21:56:39.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.899+0000 7f1aa54cb740 1 --2- 172.21.15.1:0/3627828164 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x561c133119a0 0x561c133106b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:39.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.899+0000 7f1aa54cb740 1 --2- 172.21.15.1:0/3627828164 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561c132921b0 0x561c132ac190 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:39.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.899+0000 7f1aa54cb740 1 --2- 172.21.15.1:0/3627828164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561c132831b0 0x561c13286df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:39.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.899+0000 7f1a7bfff640 0 client.15000 ms_handle_reset on v2:172.21.15.53:6836/3457131830 2024-09-11T21:56:39.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.899+0000 7f1aa54cb740 1 --2- 172.21.15.1:0/3627828164 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561c13282890 0x561c132868b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:39.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.899+0000 7f1aa54cb740 1 -- 172.21.15.1:0/3627828164 >> 172.21.15.1:0/3627828164 conn(0x561c1327ef60 msgr2=0x561c13293ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:39.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.899+0000 7f1aa54cb740 1 -- 172.21.15.1:0/3627828164 shutdown_connections 2024-09-11T21:56:39.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.899+0000 7f1aa54cb740 1 -- 172.21.15.1:0/3627828164 wait complete. 2024-09-11T21:56:39.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.899+0000 7f1aa54cb740 20 client.15000 trim_cache size 0 max 16384 2024-09-11T21:56:39.929 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-11T21:56:39.929 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "3ffca83c-7087-11ef-bcea-c7b262605968", 2024-09-11T21:56:39.929 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 2, 2024-09-11T21:56:39.929 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 24467, 2024-09-11T21:56:39.930 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-09-11T21:56:39.930 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-09-11T21:56:39.930 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-09-11T21:56:39.930 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 70.771569588000006, 2024-09-11T21:56:39.930 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 27, 2024-09-11T21:56:39.930 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 86, 2024-09-11T21:56:39.930 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 84, 2024-09-11T21:56:39.931 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 100.76098121 2024-09-11T21:56:39.931 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-11T21:56:39.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.900+0000 7f1aa30f2640 1 -- 172.21.15.1:0/2582218586 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f1a880a2c90 msgr2=0x7f1a880a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:39.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.900+0000 7f1aa30f2640 1 --2- 172.21.15.1:0/2582218586 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f1a880a2c90 0x7f1a880a5150 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1a94009cd0 tx=0x7f1a940316c0 comp rx=0 tx=0).stop 2024-09-11T21:56:39.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.900+0000 7f1aa30f2640 1 -- 172.21.15.1:0/2582218586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a9c101420 msgr2=0x7f1a9c1978c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:39.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.900+0000 7f1aa30f2640 1 --2- 172.21.15.1:0/2582218586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a9c101420 0x7f1a9c1978c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f1a8400d950 tx=0x7f1a8400de10 comp rx=0 tx=0).stop 2024-09-11T21:56:39.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.900+0000 7f1aa30f2640 1 -- 172.21.15.1:0/2582218586 shutdown_connections 2024-09-11T21:56:39.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.900+0000 7f1aa30f2640 1 --2- 172.21.15.1:0/2582218586 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f1a880a2c90 0x7f1a880a5150 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:39.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.900+0000 7f1aa30f2640 1 --2- 172.21.15.1:0/2582218586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a9c106a80 0x7f1a9c19bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:39.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.900+0000 7f1aa30f2640 1 --2- 172.21.15.1:0/2582218586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a9c101420 0x7f1a9c1978c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:39.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.900+0000 7f1aa30f2640 1 --2- 172.21.15.1:0/2582218586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a9c100a80 0x7f1a9c197380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:39.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.900+0000 7f1aa30f2640 1 -- 172.21.15.1:0/2582218586 >> 172.21.15.1:0/2582218586 conn(0x7f1a9c0fc7a0 msgr2=0x7f1a9c0fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:39.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.900+0000 7f1aa30f2640 1 -- 172.21.15.1:0/2582218586 shutdown_connections 2024-09-11T21:56:39.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:39.900+0000 7f1aa30f2640 1 -- 172.21.15.1:0/2582218586 wait complete. 2024-09-11T21:56:40.572 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:40 smithi001 ceph-mon[30935]: pgmap v323: 97 pgs: 97 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s; 4.8 KiB/s, 0 objects/s recovering 2024-09-11T21:56:40.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:40 smithi161 ceph-mon[35615]: pgmap v323: 97 pgs: 97 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s; 4.8 KiB/s, 0 objects/s recovering 2024-09-11T21:56:40.602 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-11T21:56:40.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:40 smithi053 ceph-mon[34908]: pgmap v323: 97 pgs: 97 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s; 4.8 KiB/s, 0 objects/s recovering 2024-09-11T21:56:40.860 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:56:41.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.299+0000 7f929d42f640 1 -- 172.21.15.1:0/3354572552 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9298102790 msgr2=0x7f92981067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:41.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.299+0000 7f929d42f640 1 --2- 172.21.15.1:0/3354572552 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9298102790 0x7f92981067e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9280009900 tx=0x7f928002a410 comp rx=0 tx=0).stop 2024-09-11T21:56:41.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.299+0000 7f929d42f640 1 -- 172.21.15.1:0/3354572552 shutdown_connections 2024-09-11T21:56:41.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.299+0000 7f929d42f640 1 --2- 172.21.15.1:0/3354572552 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9298106f10 0x7f929810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:41.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.299+0000 7f929d42f640 1 --2- 172.21.15.1:0/3354572552 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9298102790 0x7f92981067e0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:41.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.299+0000 7f929d42f640 1 --2- 172.21.15.1:0/3354572552 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9298101de0 0x7f92981021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:41.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.299+0000 7f929d42f640 1 -- 172.21.15.1:0/3354572552 >> 172.21.15.1:0/3354572552 conn(0x7f92980fd8c0 msgr2=0x7f92980ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:41.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.300+0000 7f929d42f640 1 -- 172.21.15.1:0/3354572552 shutdown_connections 2024-09-11T21:56:41.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.300+0000 7f929d42f640 1 -- 172.21.15.1:0/3354572552 wait complete. 2024-09-11T21:56:41.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.300+0000 7f929d42f640 1 Processor -- start 2024-09-11T21:56:41.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.300+0000 7f929d42f640 1 -- start start 2024-09-11T21:56:41.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.301+0000 7f929d42f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9298101de0 0x7f92981985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:41.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.301+0000 7f929d42f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9298102790 0x7f9298198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:41.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.301+0000 7f929d42f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9298106f10 0x7f929819ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:41.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.301+0000 7f929d42f640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9298112010 con 0x7f9298102790 2024-09-11T21:56:41.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.301+0000 7f929d42f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9298111e90 con 0x7f9298101de0 2024-09-11T21:56:41.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.301+0000 7f929d42f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9298112190 con 0x7f9298106f10 2024-09-11T21:56:41.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.301+0000 7f928ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9298102790 0x7f9298198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:41.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.301+0000 7f928ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9298102790 0x7f9298198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41916/0 (socket says 172.21.15.1:41916) 2024-09-11T21:56:41.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.301+0000 7f928ffff640 1 -- 172.21.15.1:0/1532785555 learned_addr learned my addr 172.21.15.1:0/1532785555 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:56:41.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.301+0000 7f9296ffd640 1 --2- 172.21.15.1:0/1532785555 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9298101de0 0x7f92981985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:41.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.301+0000 7f92977fe640 1 --2- 172.21.15.1:0/1532785555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9298106f10 0x7f929819ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:41.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.301+0000 7f928ffff640 1 -- 172.21.15.1:0/1532785555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9298106f10 msgr2=0x7f929819ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:41.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.301+0000 7f928ffff640 1 --2- 172.21.15.1:0/1532785555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9298106f10 0x7f929819ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:41.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.301+0000 7f928ffff640 1 -- 172.21.15.1:0/1532785555 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9298101de0 msgr2=0x7f92981985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:41.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.301+0000 7f928ffff640 1 --2- 172.21.15.1:0/1532785555 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9298101de0 0x7f92981985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:41.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.301+0000 7f928ffff640 1 -- 172.21.15.1:0/1532785555 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f929819d560 con 0x7f9298102790 2024-09-11T21:56:41.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.301+0000 7f92977fe640 1 --2- 172.21.15.1:0/1532785555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9298106f10 0x7f929819ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T21:56:41.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.301+0000 7f928ffff640 1 --2- 172.21.15.1:0/1532785555 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9298102790 0x7f9298198af0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f9280005ab0 tx=0x7f92800042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:41.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.302+0000 7f9294ff9640 1 -- 172.21.15.1:0/1532785555 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f928003f030 con 0x7f9298102790 2024-09-11T21:56:41.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.302+0000 7f929d42f640 1 -- 172.21.15.1:0/1532785555 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f929819d7f0 con 0x7f9298102790 2024-09-11T21:56:41.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.302+0000 7f9294ff9640 1 -- 172.21.15.1:0/1532785555 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9280003ce0 con 0x7f9298102790 2024-09-11T21:56:41.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.302+0000 7f929d42f640 1 -- 172.21.15.1:0/1532785555 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92981a5130 con 0x7f9298102790 2024-09-11T21:56:41.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.302+0000 7f9294ff9640 1 -- 172.21.15.1:0/1532785555 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9280035070 con 0x7f9298102790 2024-09-11T21:56:41.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.304+0000 7f9294ff9640 1 -- 172.21.15.1:0/1532785555 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f9280042030 con 0x7f9298102790 2024-09-11T21:56:41.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.304+0000 7f929f808740 1 Processor -- start 2024-09-11T21:56:41.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.304+0000 7f929f808740 1 -- start start 2024-09-11T21:56:41.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.305+0000 7f929f808740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae90fdb9d0 0x55ae90fdbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:41.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.305+0000 7f929f808740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae90fdc2f0 0x55ae90fe6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:41.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.305+0000 7f9294ff9640 1 --2- 172.21.15.1:0/1532785555 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f92840a2c90 0x7f92840a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:41.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.305+0000 7f929f808740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae90fe6bb0 0x55ae90fe8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:41.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.305+0000 7f929f808740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55ae90f42090 con 0x55ae90fdc2f0 2024-09-11T21:56:41.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.305+0000 7f929f808740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55ae90ee9a70 con 0x55ae90fe6bb0 2024-09-11T21:56:41.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.305+0000 7f929f808740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ae90f3e050 con 0x55ae90fdb9d0 2024-09-11T21:56:41.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.305+0000 7f9296ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae90fdc2f0 0x55ae90fe6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:41.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.305+0000 7f9296ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae90fdc2f0 0x55ae90fe6670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41928/0 (socket says 172.21.15.1:41928) 2024-09-11T21:56:41.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.305+0000 7f9296ffd640 1 -- 172.21.15.1:0/3457591753 learned_addr learned my addr 172.21.15.1:0/3457591753 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:56:41.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.305+0000 7f928ffff640 1 --2- 172.21.15.1:0/1532785555 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f92840a2c90 0x7f92840a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:41.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.305+0000 7f92977fe640 1 --2- 172.21.15.1:0/3457591753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae90fdb9d0 0x55ae90fdbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:41.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.305+0000 7f9294ff9640 1 -- 172.21.15.1:0/1532785555 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f92800efad0 con 0x7f9298102790 2024-09-11T21:56:41.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.305+0000 7f92977fe640 1 --2- 172.21.15.1:0/3457591753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae90fe6bb0 0x55ae90fe8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:41.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.305+0000 7f9296ffd640 1 -- 172.21.15.1:0/3457591753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae90fdb9d0 msgr2=0x55ae90fdbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:56:41.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.305+0000 7f9296ffd640 1 --2- 172.21.15.1:0/3457591753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae90fdb9d0 0x55ae90fdbdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:41.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.305+0000 7f9296ffd640 1 -- 172.21.15.1:0/3457591753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae90fe6bb0 msgr2=0x55ae90fe8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:56:41.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.305+0000 7f9296ffd640 1 --2- 172.21.15.1:0/3457591753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae90fe6bb0 0x55ae90fe8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:41.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.305+0000 7f9296ffd640 1 -- 172.21.15.1:0/3457591753 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae90ef6130 con 0x55ae90fdc2f0 2024-09-11T21:56:41.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.305+0000 7f92977fe640 1 --2- 172.21.15.1:0/3457591753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae90fe6bb0 0x55ae90fe8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:56:41.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.305+0000 7f928ffff640 1 --2- 172.21.15.1:0/1532785555 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f92840a2c90 0x7f92840a5150 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9280005940 tx=0x7f928003b790 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:41.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.305+0000 7f9296ffd640 1 --2- 172.21.15.1:0/3457591753 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae90fdc2f0 0x55ae90fe6670 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9288009a70 tx=0x7f9288009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b2bc4081ff91c9c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:41.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.305+0000 7f928d7fa640 1 -- 172.21.15.1:0/3457591753 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9288017070 con 0x55ae90fdc2f0 2024-09-11T21:56:41.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.306+0000 7f928d7fa640 1 -- 172.21.15.1:0/3457591753 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9288013470 con 0x55ae90fdc2f0 2024-09-11T21:56:41.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.306+0000 7f928d7fa640 1 -- 172.21.15.1:0/3457591753 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9288012630 con 0x55ae90fdc2f0 2024-09-11T21:56:41.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.306+0000 7f929f808740 1 -- 172.21.15.1:0/3457591753 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae90fdc2f0 msgr2=0x55ae90fe6670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:41.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.306+0000 7f929f808740 1 --2- 172.21.15.1:0/3457591753 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae90fdc2f0 0x55ae90fe6670 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9288009a70 tx=0x7f9288009f30 comp rx=0 tx=0).stop 2024-09-11T21:56:41.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.306+0000 7f929f808740 1 -- 172.21.15.1:0/3457591753 shutdown_connections 2024-09-11T21:56:41.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.306+0000 7f929f808740 1 --2- 172.21.15.1:0/3457591753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae90fe6bb0 0x55ae90fe8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:41.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.306+0000 7f929f808740 1 --2- 172.21.15.1:0/3457591753 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae90fdc2f0 0x55ae90fe6670 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:41.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.306+0000 7f929f808740 1 --2- 172.21.15.1:0/3457591753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae90fdb9d0 0x55ae90fdbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:41.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.306+0000 7f929f808740 1 -- 172.21.15.1:0/3457591753 >> 172.21.15.1:0/3457591753 conn(0x55ae90fd7f60 msgr2=0x55ae90fd8340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:41.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.306+0000 7f929f808740 1 -- 172.21.15.1:0/3457591753 shutdown_connections 2024-09-11T21:56:41.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.306+0000 7f929f808740 1 -- 172.21.15.1:0/3457591753 wait complete. 2024-09-11T21:56:41.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.307+0000 7f929f808740 1 -- start start 2024-09-11T21:56:41.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.307+0000 7f929f808740 1 Processor -- start 2024-09-11T21:56:41.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.307+0000 7f929f808740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae90fdb9d0 0x55ae90fe63f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:41.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.307+0000 7f929f808740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae90fdc2f0 0x55ae90fe3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:41.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.307+0000 7f929f808740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae90fe6bb0 0x55ae90fe3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:41.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.307+0000 7f929f808740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55ae90fe9780 con 0x55ae90fdc2f0 2024-09-11T21:56:41.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.307+0000 7f929f808740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55ae90efcf20 con 0x55ae90fe6bb0 2024-09-11T21:56:41.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.307+0000 7f929f808740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ae90fe9900 con 0x55ae90fdb9d0 2024-09-11T21:56:41.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.307+0000 7f9296ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae90fdc2f0 0x55ae90fe3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:41.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.307+0000 7f929f808740 10 client.0 _ll_register_callbacks cb 0x55ae90ec6030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:56:41.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.307+0000 7f9296ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae90fdc2f0 0x55ae90fe3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41930/0 (socket says 172.21.15.1:41930) 2024-09-11T21:56:41.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.307+0000 7f9296ffd640 1 -- 172.21.15.1:0/2599260331 learned_addr learned my addr 172.21.15.1:0/2599260331 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:56:41.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.307+0000 7f92977fe640 1 --2- 172.21.15.1:0/2599260331 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae90fdb9d0 0x55ae90fe63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:41.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.307+0000 7f9296ffd640 1 -- 172.21.15.1:0/2599260331 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae90fdb9d0 msgr2=0x55ae90fe63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:56:41.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.307+0000 7f9296ffd640 1 --2- 172.21.15.1:0/2599260331 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae90fdb9d0 0x55ae90fe63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:41.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.307+0000 7f9296ffd640 1 -- 172.21.15.1:0/2599260331 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae90fe6bb0 msgr2=0x55ae90fe3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:56:41.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.307+0000 7f92977fe640 1 --2- 172.21.15.1:0/2599260331 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae90fe6bb0 0x55ae90fe3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:41.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.308+0000 7f9296ffd640 1 --2- 172.21.15.1:0/2599260331 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae90fe6bb0 0x55ae90fe3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:41.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.308+0000 7f9296ffd640 1 -- 172.21.15.1:0/2599260331 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae90cb32b0 con 0x55ae90fdc2f0 2024-09-11T21:56:41.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.308+0000 7f9296ffd640 1 --2- 172.21.15.1:0/2599260331 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae90fdc2f0 0x55ae90fe3740 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f9288000c00 tx=0x7f9288006e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77e0fac8475edaab server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:41.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.308+0000 7f927e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T21:56:41.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.308+0000 7f927e7fc640 1 -- 172.21.15.1:0/2599260331 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9288017050 con 0x55ae90fdc2f0 2024-09-11T21:56:41.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.308+0000 7f927e7fc640 1 -- 172.21.15.1:0/2599260331 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f928801c650 con 0x55ae90fdc2f0 2024-09-11T21:56:41.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.308+0000 7f929f808740 1 -- 172.21.15.1:0/2599260331 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9288013470 con 0x55ae90fdc2f0 2024-09-11T21:56:41.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.308+0000 7f927e7fc640 1 -- 172.21.15.1:0/2599260331 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9288013970 con 0x55ae90fdc2f0 2024-09-11T21:56:41.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.308+0000 7f929f808740 10 client.15012 fetch_fsmap learned FSMap version 27 2024-09-11T21:56:41.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.308+0000 7f929f808740 1 -- 172.21.15.1:0/2599260331 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ae90ff5040 con 0x55ae90fdc2f0 2024-09-11T21:56:41.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.308+0000 7f927e7fc640 1 -- 172.21.15.1:0/2599260331 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9288013470 con 0x55ae90fdc2f0 2024-09-11T21:56:41.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.308+0000 7f927e7fc640 1 -- 172.21.15.1:0/2599260331 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f9288031ac0 con 0x55ae90fdc2f0 2024-09-11T21:56:41.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.308+0000 7f929f808740 10 client.15012 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T21:56:41.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.309+0000 7f929f808740 10 client.15012 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-11T21:56:41.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.309+0000 7f929f808740 20 client.15012 populate_metadata read hostname 'smithi001' 2024-09-11T21:56:41.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.309+0000 7f929f808740 1 --2- 172.21.15.1:0/2599260331 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x55ae90ff7350 0x55ae90ff87c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:41.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.309+0000 7f929f808740 4 client.15012 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T21:56:41.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.309+0000 7f929f808740 1 -- 172.21.15.1:0/2599260331 --> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae90f4f600 con 0x55ae90ff7350 2024-09-11T21:56:41.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.309+0000 7f92977fe640 1 --2- 172.21.15.1:0/2599260331 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x55ae90ff7350 0x55ae90ff87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:41.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.309+0000 7f92977fe640 1 --2- 172.21.15.1:0/2599260331 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x55ae90ff7350 0x55ae90ff87c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9251ca9fa656923c server_cookie=438657a1c3ae450b in_seq=0 out_seq=0 2024-09-11T21:56:41.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.309+0000 7f927e7fc640 10 client.15012 ms_handle_connect on v2:172.21.15.161:6838/455365383 2024-09-11T21:56:41.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.310+0000 7f927e7fc640 1 -- 172.21.15.1:0/2599260331 <== mds.1 v2:172.21.15.161:6838/455365383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ae90f4f600 con 0x55ae90ff7350 2024-09-11T21:56:41.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.310+0000 7f927e7fc640 10 client.15012 handle_command_reply: tid=0 2024-09-11T21:56:41.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.310+0000 7f929f808740 1 client.15012 shutdown 2024-09-11T21:56:41.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.310+0000 7f929f808740 1 -- 172.21.15.1:0/2599260331 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae90fdc2f0 msgr2=0x55ae90fe3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:41.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.310+0000 7f929f808740 1 --2- 172.21.15.1:0/2599260331 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae90fdc2f0 0x55ae90fe3740 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f9288000c00 tx=0x7f9288006e40 comp rx=0 tx=0).stop 2024-09-11T21:56:41.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.310+0000 7f929f808740 1 -- 172.21.15.1:0/2599260331 shutdown_connections 2024-09-11T21:56:41.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.310+0000 7f929f808740 1 --2- 172.21.15.1:0/2599260331 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x55ae90ff7350 0x55ae90ff87c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:41.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.310+0000 7f929f808740 1 --2- 172.21.15.1:0/2599260331 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae90fe6bb0 0x55ae90fe3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:41.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.310+0000 7f929f808740 1 --2- 172.21.15.1:0/2599260331 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae90fdc2f0 0x55ae90fe3740 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:41.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.310+0000 7f927e7fc640 0 client.15012 ms_handle_reset on v2:172.21.15.161:6838/455365383 2024-09-11T21:56:41.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.310+0000 7f929f808740 1 --2- 172.21.15.1:0/2599260331 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae90fdb9d0 0x55ae90fe63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:41.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.310+0000 7f929f808740 1 -- 172.21.15.1:0/2599260331 >> 172.21.15.1:0/2599260331 conn(0x55ae90fd7f60 msgr2=0x55ae90fddc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:41.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.310+0000 7f929f808740 1 -- 172.21.15.1:0/2599260331 shutdown_connections 2024-09-11T21:56:41.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.310+0000 7f929f808740 1 -- 172.21.15.1:0/2599260331 wait complete. 2024-09-11T21:56:41.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.310+0000 7f929f808740 20 client.15012 trim_cache size 0 max 16384 2024-09-11T21:56:41.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.328+0000 7f929f808740 1 Processor -- start 2024-09-11T21:56:41.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.328+0000 7f929f808740 1 -- start start 2024-09-11T21:56:41.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.329+0000 7f929f808740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae90fdb9d0 0x55ae90ff87c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:41.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.329+0000 7f929f808740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae90fdc2f0 0x55ae90fd7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:41.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.329+0000 7f929f808740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae90fe6bb0 0x55ae90fe09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:41.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.329+0000 7f929f808740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55ae90ff6b00 con 0x55ae90fe6bb0 2024-09-11T21:56:41.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.329+0000 7f929f808740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55ae90ff71d0 con 0x55ae90fdb9d0 2024-09-11T21:56:41.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.329+0000 7f929f808740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ae90ff78a0 con 0x55ae90fdc2f0 2024-09-11T21:56:41.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.329+0000 7f92977fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae90fe6bb0 0x55ae90fe09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:41.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.329+0000 7f92977fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae90fe6bb0 0x55ae90fe09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41932/0 (socket says 172.21.15.1:41932) 2024-09-11T21:56:41.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.329+0000 7f92977fe640 1 -- 172.21.15.1:0/1303968710 learned_addr learned my addr 172.21.15.1:0/1303968710 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:56:41.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.329+0000 7f92977fe640 1 --2- 172.21.15.1:0/1303968710 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae90fdb9d0 0x55ae90ff87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:41.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.329+0000 7f9296ffd640 1 --2- 172.21.15.1:0/1303968710 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae90fdc2f0 0x55ae90fd7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:41.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.329+0000 7f92977fe640 1 -- 172.21.15.1:0/1303968710 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae90fdc2f0 msgr2=0x55ae90fd7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:56:41.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.329+0000 7f92977fe640 1 --2- 172.21.15.1:0/1303968710 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae90fdc2f0 0x55ae90fd7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:41.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.329+0000 7f92977fe640 1 -- 172.21.15.1:0/1303968710 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae90fdb9d0 msgr2=0x55ae90ff87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:56:41.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.329+0000 7f92977fe640 1 --2- 172.21.15.1:0/1303968710 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae90fdb9d0 0x55ae90ff87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:41.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.329+0000 7f92977fe640 1 -- 172.21.15.1:0/1303968710 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae90fe6010 con 0x55ae90fe6bb0 2024-09-11T21:56:41.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.329+0000 7f9296ffd640 1 --2- 172.21.15.1:0/1303968710 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae90fdc2f0 0x55ae90fd7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T21:56:41.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.329+0000 7f92977fe640 1 --2- 172.21.15.1:0/1303968710 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae90fe6bb0 0x55ae90fe09a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x55ae90fe4bd0 tx=0x7f9290011760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6003cd25b34664a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:41.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.330+0000 7f928cff9640 1 -- 172.21.15.1:0/1303968710 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f929000ab20 con 0x55ae90fe6bb0 2024-09-11T21:56:41.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.330+0000 7f928cff9640 1 -- 172.21.15.1:0/1303968710 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9290010550 con 0x55ae90fe6bb0 2024-09-11T21:56:41.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.330+0000 7f928cff9640 1 -- 172.21.15.1:0/1303968710 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f929000faa0 con 0x55ae90fe6bb0 2024-09-11T21:56:41.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.330+0000 7f929f808740 1 -- 172.21.15.1:0/1303968710 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae90fe6bb0 msgr2=0x55ae90fe09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:41.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.330+0000 7f929f808740 1 --2- 172.21.15.1:0/1303968710 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae90fe6bb0 0x55ae90fe09a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x55ae90fe4bd0 tx=0x7f9290011760 comp rx=0 tx=0).stop 2024-09-11T21:56:41.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.330+0000 7f929f808740 1 -- 172.21.15.1:0/1303968710 shutdown_connections 2024-09-11T21:56:41.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.330+0000 7f929f808740 1 --2- 172.21.15.1:0/1303968710 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae90fe6bb0 0x55ae90fe09a0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:41.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.330+0000 7f929f808740 1 --2- 172.21.15.1:0/1303968710 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae90fdc2f0 0x55ae90fd7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:41.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.330+0000 7f929f808740 1 --2- 172.21.15.1:0/1303968710 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae90fdb9d0 0x55ae90ff87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:41.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.331+0000 7f929f808740 1 -- 172.21.15.1:0/1303968710 >> 172.21.15.1:0/1303968710 conn(0x55ae90fd7f60 msgr2=0x55ae90fd8340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:41.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.331+0000 7f929f808740 1 -- 172.21.15.1:0/1303968710 shutdown_connections 2024-09-11T21:56:41.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.331+0000 7f929f808740 1 -- 172.21.15.1:0/1303968710 wait complete. 2024-09-11T21:56:41.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.331+0000 7f929f808740 1 -- start start 2024-09-11T21:56:41.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.331+0000 7f929f808740 1 Processor -- start 2024-09-11T21:56:41.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.331+0000 7f929f808740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae90fdb9d0 0x55ae90ffc750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:41.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.331+0000 7f929f808740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae90fdc2f0 0x55ae90ffcc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:41.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.331+0000 7f929f808740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae90fe6bb0 0x55ae90ffd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:41.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.331+0000 7f929f808740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55ae90ff8640 con 0x55ae90fdb9d0 2024-09-11T21:56:41.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.331+0000 7f929f808740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55ae90ff7f70 con 0x55ae90fe6bb0 2024-09-11T21:56:41.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.331+0000 7f929f808740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ae90ff8d10 con 0x55ae90fdc2f0 2024-09-11T21:56:41.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.332+0000 7f929f808740 10 client.0 _ll_register_callbacks cb 0x7f9290011ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:56:41.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.332+0000 7f92977fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae90fdb9d0 0x55ae90ffc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:41.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.332+0000 7f92977fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae90fdb9d0 0x55ae90ffc750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41944/0 (socket says 172.21.15.1:41944) 2024-09-11T21:56:41.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.332+0000 7f92977fe640 1 -- 172.21.15.1:0/3510650120 learned_addr learned my addr 172.21.15.1:0/3510650120 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:56:41.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.332+0000 7f9296ffd640 1 --2- 172.21.15.1:0/3510650120 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae90fdc2f0 0x55ae90ffcc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:41.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.332+0000 7f92977fe640 1 --2- 172.21.15.1:0/3510650120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae90fe6bb0 0x55ae90ffd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:41.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.332+0000 7f92977fe640 1 -- 172.21.15.1:0/3510650120 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae90fdc2f0 msgr2=0x55ae90ffcc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:56:41.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.332+0000 7f92977fe640 1 --2- 172.21.15.1:0/3510650120 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae90fdc2f0 0x55ae90ffcc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:41.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.332+0000 7f92977fe640 1 -- 172.21.15.1:0/3510650120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae90fe6bb0 msgr2=0x55ae90ffd1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:56:41.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.332+0000 7f92977fe640 1 --2- 172.21.15.1:0/3510650120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae90fe6bb0 0x55ae90ffd1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:41.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.332+0000 7f92977fe640 1 -- 172.21.15.1:0/3510650120 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae91060a60 con 0x55ae90fdb9d0 2024-09-11T21:56:41.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.332+0000 7f92977fe640 1 --2- 172.21.15.1:0/3510650120 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae90fdb9d0 0x55ae90ffc750 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9290004a60 tx=0x7f929000ab20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=806345b3e95f4f9f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:41.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.332+0000 7f927e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T21:56:41.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.334+0000 7f927e7fc640 1 -- 172.21.15.1:0/3510650120 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f929000add0 con 0x55ae90fdb9d0 2024-09-11T21:56:41.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.334+0000 7f927e7fc640 1 -- 172.21.15.1:0/3510650120 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f929001a040 con 0x55ae90fdb9d0 2024-09-11T21:56:41.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.334+0000 7f929f808740 1 -- 172.21.15.1:0/3510650120 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9290010550 con 0x55ae90fdb9d0 2024-09-11T21:56:41.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.335+0000 7f927e7fc640 1 -- 172.21.15.1:0/3510650120 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f929001e430 con 0x55ae90fdb9d0 2024-09-11T21:56:41.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.335+0000 7f929f808740 10 client.15018 fetch_fsmap learned FSMap version 27 2024-09-11T21:56:41.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.335+0000 7f929f808740 1 -- 172.21.15.1:0/3510650120 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ae91061c30 con 0x55ae90fdb9d0 2024-09-11T21:56:41.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.335+0000 7f927e7fc640 1 -- 172.21.15.1:0/3510650120 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f929002c900 con 0x55ae90fdb9d0 2024-09-11T21:56:41.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.335+0000 7f927e7fc640 1 -- 172.21.15.1:0/3510650120 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f92900207c0 con 0x55ae90fdb9d0 2024-09-11T21:56:41.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.335+0000 7f929f808740 10 client.15018 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T21:56:41.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.335+0000 7f929f808740 10 client.15018 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-11T21:56:41.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.335+0000 7f929f808740 20 client.15018 populate_metadata read hostname 'smithi001' 2024-09-11T21:56:41.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.335+0000 7f929f808740 1 --2- 172.21.15.1:0/3510650120 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x55ae91066290 0x55ae91064fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:41.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.335+0000 7f929f808740 4 client.15018 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T21:56:41.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.335+0000 7f929f808740 1 -- 172.21.15.1:0/3510650120 --> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] -- command(tid 0: {"prefix": "status"}) -- 0x7f9290010550 con 0x55ae91066290 2024-09-11T21:56:41.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.335+0000 7f9296ffd640 1 --2- 172.21.15.1:0/3510650120 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x55ae91066290 0x55ae91064fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:41.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.336+0000 7f9296ffd640 1 --2- 172.21.15.1:0/3510650120 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x55ae91066290 0x55ae91064fa0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=de8af5946fd96f30 server_cookie=28a84a915eceaf90 in_seq=0 out_seq=0 2024-09-11T21:56:41.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.336+0000 7f927e7fc640 10 client.15018 ms_handle_connect on v2:172.21.15.161:6838/455365383 2024-09-11T21:56:41.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.336+0000 7f927e7fc640 1 -- 172.21.15.1:0/3510650120 <== mds.1 v2:172.21.15.161:6838/455365383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x7f9290010550 con 0x55ae91066290 2024-09-11T21:56:41.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.336+0000 7f927e7fc640 10 client.15018 handle_command_reply: tid=0 2024-09-11T21:56:41.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.336+0000 7f929f808740 1 client.15018 shutdown 2024-09-11T21:56:41.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.336+0000 7f929f808740 1 -- 172.21.15.1:0/3510650120 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae90fdb9d0 msgr2=0x55ae90ffc750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:41.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.336+0000 7f929f808740 1 --2- 172.21.15.1:0/3510650120 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae90fdb9d0 0x55ae90ffc750 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9290004a60 tx=0x7f929000ab20 comp rx=0 tx=0).stop 2024-09-11T21:56:41.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.336+0000 7f929f808740 1 -- 172.21.15.1:0/3510650120 shutdown_connections 2024-09-11T21:56:41.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.336+0000 7f929f808740 1 --2- 172.21.15.1:0/3510650120 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x55ae91066290 0x55ae91064fa0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:41.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.336+0000 7f929f808740 1 --2- 172.21.15.1:0/3510650120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae90fe6bb0 0x55ae90ffd1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:41.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.336+0000 7f929f808740 1 --2- 172.21.15.1:0/3510650120 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ae90fdc2f0 0x55ae90ffcc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:41.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.336+0000 7f929f808740 1 --2- 172.21.15.1:0/3510650120 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae90fdb9d0 0x55ae90ffc750 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:41.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.336+0000 7f927e7fc640 0 client.15018 ms_handle_reset on v2:172.21.15.161:6838/455365383 2024-09-11T21:56:41.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.336+0000 7f929f808740 1 -- 172.21.15.1:0/3510650120 >> 172.21.15.1:0/3510650120 conn(0x55ae90fd7f60 msgr2=0x55ae90fe55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:41.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.336+0000 7f929f808740 1 -- 172.21.15.1:0/3510650120 shutdown_connections 2024-09-11T21:56:41.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.336+0000 7f929f808740 1 -- 172.21.15.1:0/3510650120 wait complete. 2024-09-11T21:56:41.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.337+0000 7f929f808740 20 client.15018 trim_cache size 0 max 16384 2024-09-11T21:56:41.348 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-11T21:56:41.348 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "3ffca83c-7087-11ef-bcea-c7b262605968", 2024-09-11T21:56:41.348 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 1, 2024-09-11T21:56:41.348 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 24484, 2024-09-11T21:56:41.349 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-09-11T21:56:41.349 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-09-11T21:56:41.349 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-09-11T21:56:41.349 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 73.215362881999994, 2024-09-11T21:56:41.349 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 27, 2024-09-11T21:56:41.349 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 86, 2024-09-11T21:56:41.349 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 84, 2024-09-11T21:56:41.349 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 99.632552892999996 2024-09-11T21:56:41.349 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-11T21:56:41.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.337+0000 7f929d42f640 1 -- 172.21.15.1:0/1532785555 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f92840a2c90 msgr2=0x7f92840a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:41.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.337+0000 7f929d42f640 1 --2- 172.21.15.1:0/1532785555 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f92840a2c90 0x7f92840a5150 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9280005940 tx=0x7f928003b790 comp rx=0 tx=0).stop 2024-09-11T21:56:41.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.337+0000 7f929d42f640 1 -- 172.21.15.1:0/1532785555 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9298102790 msgr2=0x7f9298198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:41.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.337+0000 7f929d42f640 1 --2- 172.21.15.1:0/1532785555 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9298102790 0x7f9298198af0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f9280005ab0 tx=0x7f92800042f0 comp rx=0 tx=0).stop 2024-09-11T21:56:41.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.337+0000 7f929d42f640 1 -- 172.21.15.1:0/1532785555 shutdown_connections 2024-09-11T21:56:41.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.337+0000 7f929d42f640 1 --2- 172.21.15.1:0/1532785555 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f92840a2c90 0x7f92840a5150 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:41.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.337+0000 7f929d42f640 1 --2- 172.21.15.1:0/1532785555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9298106f10 0x7f929819ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:41.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.337+0000 7f929d42f640 1 --2- 172.21.15.1:0/1532785555 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9298102790 0x7f9298198af0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:41.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.337+0000 7f929d42f640 1 --2- 172.21.15.1:0/1532785555 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9298101de0 0x7f92981985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:41.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.337+0000 7f929d42f640 1 -- 172.21.15.1:0/1532785555 >> 172.21.15.1:0/1532785555 conn(0x7f92980fd8c0 msgr2=0x7f92980fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:41.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.337+0000 7f929d42f640 1 -- 172.21.15.1:0/1532785555 shutdown_connections 2024-09-11T21:56:41.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:41.337+0000 7f929d42f640 1 -- 172.21.15.1:0/1532785555 wait complete. 2024-09-11T21:56:41.955 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-11T21:56:42.217 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:56:42.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:42 smithi161 ceph-mon[35615]: pgmap v324: 97 pgs: 97 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s; 4.8 KiB/s, 0 objects/s recovering 2024-09-11T21:56:42.660 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:42 smithi001 ceph-mon[30935]: pgmap v324: 97 pgs: 97 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s; 4.8 KiB/s, 0 objects/s recovering 2024-09-11T21:56:42.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.657+0000 7fd7e355d640 1 -- 172.21.15.1:0/611588292 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7dc10c2f0 msgr2=0x7fd7dc10e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:42.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.657+0000 7fd7e355d640 1 --2- 172.21.15.1:0/611588292 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7dc10c2f0 0x7fd7dc10e790 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd7d4009900 tx=0x7fd7d402a410 comp rx=0 tx=0).stop 2024-09-11T21:56:42.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.658+0000 7fd7e355d640 1 -- 172.21.15.1:0/611588292 shutdown_connections 2024-09-11T21:56:42.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.658+0000 7fd7e355d640 1 --2- 172.21.15.1:0/611588292 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7dc10c2f0 0x7fd7dc10e790 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:42.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.658+0000 7fd7e355d640 1 --2- 172.21.15.1:0/611588292 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7dc1037a0 0x7fd7dc10bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:42.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.658+0000 7fd7e355d640 1 --2- 172.21.15.1:0/611588292 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7dc102df0 0x7fd7dc1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:42.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.658+0000 7fd7e355d640 1 -- 172.21.15.1:0/611588292 >> 172.21.15.1:0/611588292 conn(0x7fd7dc0fc7d0 msgr2=0x7fd7dc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:42.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.658+0000 7fd7e355d640 1 -- 172.21.15.1:0/611588292 shutdown_connections 2024-09-11T21:56:42.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.658+0000 7fd7e355d640 1 -- 172.21.15.1:0/611588292 wait complete. 2024-09-11T21:56:42.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.658+0000 7fd7e355d640 1 Processor -- start 2024-09-11T21:56:42.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.658+0000 7fd7e355d640 1 -- start start 2024-09-11T21:56:42.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.659+0000 7fd7e355d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7dc102df0 0x7fd7dc1a0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:42.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.659+0000 7fd7e355d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7dc1037a0 0x7fd7dc1a0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:42.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.659+0000 7fd7e355d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7dc10c2f0 0x7fd7dc19a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:42.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.659+0000 7fd7e355d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd7dc112110 con 0x7fd7dc10c2f0 2024-09-11T21:56:42.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.659+0000 7fd7e355d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd7dc111f90 con 0x7fd7dc102df0 2024-09-11T21:56:42.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.659+0000 7fd7e355d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd7dc112290 con 0x7fd7dc1037a0 2024-09-11T21:56:42.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.659+0000 7fd7e1afa640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7dc10c2f0 0x7fd7dc19a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:42.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.659+0000 7fd7e1afa640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7dc10c2f0 0x7fd7dc19a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41962/0 (socket says 172.21.15.1:41962) 2024-09-11T21:56:42.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.659+0000 7fd7e1afa640 1 -- 172.21.15.1:0/1394454453 learned_addr learned my addr 172.21.15.1:0/1394454453 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:56:42.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.659+0000 7fd7e12f9640 1 --2- 172.21.15.1:0/1394454453 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7dc102df0 0x7fd7dc1a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:42.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.659+0000 7fd7e0af8640 1 --2- 172.21.15.1:0/1394454453 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7dc1037a0 0x7fd7dc1a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:42.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.659+0000 7fd7e1afa640 1 -- 172.21.15.1:0/1394454453 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7dc1037a0 msgr2=0x7fd7dc1a0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:42.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.659+0000 7fd7e1afa640 1 --2- 172.21.15.1:0/1394454453 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7dc1037a0 0x7fd7dc1a0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:42.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.659+0000 7fd7e1afa640 1 -- 172.21.15.1:0/1394454453 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7dc102df0 msgr2=0x7fd7dc1a0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:42.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.659+0000 7fd7e1afa640 1 --2- 172.21.15.1:0/1394454453 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7dc102df0 0x7fd7dc1a0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:42.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.659+0000 7fd7e1afa640 1 -- 172.21.15.1:0/1394454453 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7dc19ad20 con 0x7fd7dc10c2f0 2024-09-11T21:56:42.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.659+0000 7fd7e1afa640 1 --2- 172.21.15.1:0/1394454453 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7dc10c2f0 0x7fd7dc19a5f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd7d4005ab0 tx=0x7fd7d40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:42.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.660+0000 7fd7da7fc640 1 -- 172.21.15.1:0/1394454453 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d403f030 con 0x7fd7dc10c2f0 2024-09-11T21:56:42.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.660+0000 7fd7da7fc640 1 -- 172.21.15.1:0/1394454453 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7d4003ce0 con 0x7fd7dc10c2f0 2024-09-11T21:56:42.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.660+0000 7fd7e355d640 1 -- 172.21.15.1:0/1394454453 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7dc19afb0 con 0x7fd7dc10c2f0 2024-09-11T21:56:42.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.660+0000 7fd7da7fc640 1 -- 172.21.15.1:0/1394454453 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d4035070 con 0x7fd7dc10c2f0 2024-09-11T21:56:42.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.660+0000 7fd7e355d640 1 -- 172.21.15.1:0/1394454453 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7dc1a7300 con 0x7fd7dc10c2f0 2024-09-11T21:56:42.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.662+0000 7fd7da7fc640 1 -- 172.21.15.1:0/1394454453 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fd7d4042030 con 0x7fd7dc10c2f0 2024-09-11T21:56:42.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.662+0000 7fd7e5936740 1 Processor -- start 2024-09-11T21:56:42.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.663+0000 7fd7e5936740 1 -- start start 2024-09-11T21:56:42.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.663+0000 7fd7e5936740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5607f90199d0 0x5607f9019db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:42.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.663+0000 7fd7e5936740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5607f901a2f0 0x5607f9024670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:42.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.663+0000 7fd7e5936740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5607f9024bb0 0x5607f9026fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:42.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.663+0000 7fd7e5936740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5607f8f80090 con 0x5607f901a2f0 2024-09-11T21:56:42.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.663+0000 7fd7e5936740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5607f8f27a70 con 0x5607f9024bb0 2024-09-11T21:56:42.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.663+0000 7fd7e5936740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5607f8f7c050 con 0x5607f90199d0 2024-09-11T21:56:42.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.663+0000 7fd7da7fc640 1 --2- 172.21.15.1:0/1394454453 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fd7c40a2c10 0x7fd7c40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:42.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.663+0000 7fd7e12f9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5607f901a2f0 0x5607f9024670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:42.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.663+0000 7fd7e0af8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5607f9024bb0 0x5607f9026fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:42.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.663+0000 7fd7e12f9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5607f901a2f0 0x5607f9024670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41972/0 (socket says 172.21.15.1:41972) 2024-09-11T21:56:42.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.663+0000 7fd7e1afa640 1 --2- 172.21.15.1:0/1394454453 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fd7c40a2c10 0x7fd7c40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:42.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.663+0000 7fd7e12f9640 1 -- 172.21.15.1:0/472328005 learned_addr learned my addr 172.21.15.1:0/472328005 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:56:42.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.663+0000 7fd7e0af8640 1 --2- 172.21.15.1:0/472328005 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5607f90199d0 0x5607f9019db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:42.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.663+0000 7fd7da7fc640 1 -- 172.21.15.1:0/1394454453 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fd7d40ef950 con 0x7fd7dc10c2f0 2024-09-11T21:56:42.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.663+0000 7fd7e12f9640 1 -- 172.21.15.1:0/472328005 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5607f90199d0 msgr2=0x5607f9019db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:56:42.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.663+0000 7fd7e12f9640 1 --2- 172.21.15.1:0/472328005 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5607f90199d0 0x5607f9019db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:42.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.663+0000 7fd7e12f9640 1 -- 172.21.15.1:0/472328005 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5607f9024bb0 msgr2=0x5607f9026fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:56:42.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.663+0000 7fd7e12f9640 1 --2- 172.21.15.1:0/472328005 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5607f9024bb0 0x5607f9026fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:42.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.663+0000 7fd7e12f9640 1 -- 172.21.15.1:0/472328005 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607f8f34130 con 0x5607f901a2f0 2024-09-11T21:56:42.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.663+0000 7fd7e0af8640 1 --2- 172.21.15.1:0/472328005 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5607f9024bb0 0x5607f9026fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T21:56:42.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.664+0000 7fd7e12f9640 1 --2- 172.21.15.1:0/472328005 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5607f901a2f0 0x5607f9024670 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc00bd10 tx=0x7fd7cc0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3183d326c55f11e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:42.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.664+0000 7fd7e1afa640 1 --2- 172.21.15.1:0/1394454453 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fd7c40a2c10 0x7fd7c40a50d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd7d40bbde0 tx=0x7fd7d408e0f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:42.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.664+0000 7fd7c37fe640 1 -- 172.21.15.1:0/472328005 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc004030 con 0x5607f901a2f0 2024-09-11T21:56:42.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.664+0000 7fd7c37fe640 1 -- 172.21.15.1:0/472328005 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7cc0041d0 con 0x5607f901a2f0 2024-09-11T21:56:42.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.664+0000 7fd7c37fe640 1 -- 172.21.15.1:0/472328005 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc0117e0 con 0x5607f901a2f0 2024-09-11T21:56:42.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.664+0000 7fd7e5936740 1 -- 172.21.15.1:0/472328005 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5607f901a2f0 msgr2=0x5607f9024670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:42.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.664+0000 7fd7e5936740 1 --2- 172.21.15.1:0/472328005 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5607f901a2f0 0x5607f9024670 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc00bd10 tx=0x7fd7cc0107a0 comp rx=0 tx=0).stop 2024-09-11T21:56:42.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.665+0000 7fd7e5936740 1 -- 172.21.15.1:0/472328005 shutdown_connections 2024-09-11T21:56:42.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.665+0000 7fd7e5936740 1 --2- 172.21.15.1:0/472328005 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5607f9024bb0 0x5607f9026fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:42.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.665+0000 7fd7e5936740 1 --2- 172.21.15.1:0/472328005 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5607f901a2f0 0x5607f9024670 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:42.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.665+0000 7fd7e5936740 1 --2- 172.21.15.1:0/472328005 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5607f90199d0 0x5607f9019db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:42.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.665+0000 7fd7e5936740 1 -- 172.21.15.1:0/472328005 >> 172.21.15.1:0/472328005 conn(0x5607f9015f60 msgr2=0x5607f9016340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:42.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.665+0000 7fd7e5936740 1 -- 172.21.15.1:0/472328005 shutdown_connections 2024-09-11T21:56:42.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.665+0000 7fd7e5936740 1 -- 172.21.15.1:0/472328005 wait complete. 2024-09-11T21:56:42.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.665+0000 7fd7e5936740 1 -- start start 2024-09-11T21:56:42.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.665+0000 7fd7e5936740 1 Processor -- start 2024-09-11T21:56:42.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.665+0000 7fd7e5936740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5607f90199d0 0x5607f9024450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:42.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.665+0000 7fd7e5936740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5607f901a2f0 0x5607f90217a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:42.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.665+0000 7fd7e5936740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5607f9024bb0 0x5607f9021ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:42.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.665+0000 7fd7e5936740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5607f9027780 con 0x5607f9024bb0 2024-09-11T21:56:42.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.665+0000 7fd7e5936740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5607f8f3af20 con 0x5607f90199d0 2024-09-11T21:56:42.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.665+0000 7fd7e5936740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5607f9027900 con 0x5607f901a2f0 2024-09-11T21:56:42.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.666+0000 7fd7e5936740 10 client.0 _ll_register_callbacks cb 0x5607f8f04030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:56:42.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.666+0000 7fd7e0af8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5607f9024bb0 0x5607f9021ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:42.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.666+0000 7fd7e0af8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5607f9024bb0 0x5607f9021ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41980/0 (socket says 172.21.15.1:41980) 2024-09-11T21:56:42.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.666+0000 7fd7e0af8640 1 -- 172.21.15.1:0/3911185677 learned_addr learned my addr 172.21.15.1:0/3911185677 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:56:42.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.666+0000 7fd7e12f9640 1 --2- 172.21.15.1:0/3911185677 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5607f901a2f0 0x5607f90217a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:42.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.666+0000 7fd7e0af8640 1 --2- 172.21.15.1:0/3911185677 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5607f90199d0 0x5607f9024450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:42.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.666+0000 7fd7e0af8640 1 -- 172.21.15.1:0/3911185677 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5607f901a2f0 msgr2=0x5607f90217a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:56:42.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.666+0000 7fd7e0af8640 1 --2- 172.21.15.1:0/3911185677 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5607f901a2f0 0x5607f90217a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:42.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.666+0000 7fd7e0af8640 1 -- 172.21.15.1:0/3911185677 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5607f90199d0 msgr2=0x5607f9024450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:56:42.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.666+0000 7fd7e0af8640 1 --2- 172.21.15.1:0/3911185677 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5607f90199d0 0x5607f9024450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:42.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.666+0000 7fd7e0af8640 1 -- 172.21.15.1:0/3911185677 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607f8cf12b0 con 0x5607f9024bb0 2024-09-11T21:56:42.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.666+0000 7fd7e0af8640 1 --2- 172.21.15.1:0/3911185677 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5607f9024bb0 0x5607f9021ce0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc0697d0 tx=0x7fd7d0014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f056feb9fd9343a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:42.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.666+0000 7fd7c0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T21:56:42.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.666+0000 7fd7c0ff9640 1 -- 172.21.15.1:0/3911185677 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d0004b20 con 0x5607f9024bb0 2024-09-11T21:56:42.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.666+0000 7fd7c0ff9640 1 -- 172.21.15.1:0/3911185677 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fd7d0004cc0 con 0x5607f9024bb0 2024-09-11T21:56:42.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.666+0000 7fd7e5936740 1 -- 172.21.15.1:0/3911185677 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7cc0041d0 con 0x5607f9024bb0 2024-09-11T21:56:42.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.666+0000 7fd7c0ff9640 1 -- 172.21.15.1:0/3911185677 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd7d0003970 con 0x5607f9024bb0 2024-09-11T21:56:42.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.666+0000 7fd7e5936740 10 client.15030 fetch_fsmap learned FSMap version 27 2024-09-11T21:56:42.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.666+0000 7fd7e5936740 1 -- 172.21.15.1:0/3911185677 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5607f9033100 con 0x5607f9024bb0 2024-09-11T21:56:42.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.667+0000 7fd7c0ff9640 1 -- 172.21.15.1:0/3911185677 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7cc0041d0 con 0x5607f9024bb0 2024-09-11T21:56:42.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.667+0000 7fd7c0ff9640 1 -- 172.21.15.1:0/3911185677 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd7d000f040 con 0x5607f9024bb0 2024-09-11T21:56:42.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.667+0000 7fd7e5936740 10 client.15030 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T21:56:42.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.667+0000 7fd7e5936740 10 client.15030 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T21:56:42.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.667+0000 7fd7e5936740 20 client.15030 populate_metadata read hostname 'smithi001' 2024-09-11T21:56:42.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.667+0000 7fd7e5936740 1 --2- 172.21.15.1:0/3911185677 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5607f90356a0 0x5607f9036b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:42.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.667+0000 7fd7e5936740 4 client.15030 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T21:56:42.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.667+0000 7fd7e5936740 1 -- 172.21.15.1:0/3911185677 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607f8f8d600 con 0x5607f90356a0 2024-09-11T21:56:42.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.667+0000 7fd7e12f9640 1 --2- 172.21.15.1:0/3911185677 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5607f90356a0 0x5607f9036b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:42.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.667+0000 7fd7c0ff9640 10 client.15030 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T21:56:42.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.667+0000 7fd7e12f9640 1 --2- 172.21.15.1:0/3911185677 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5607f90356a0 0x5607f9036b10 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c28211ecc8cdee8 server_cookie=2210f03083d23b4 in_seq=0 out_seq=0 2024-09-11T21:56:42.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.668+0000 7fd7c0ff9640 1 -- 172.21.15.1:0/3911185677 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5607f8f8d600 con 0x5607f90356a0 2024-09-11T21:56:42.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.668+0000 7fd7c0ff9640 10 client.15030 handle_command_reply: tid=0 2024-09-11T21:56:42.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.668+0000 7fd7e5936740 1 client.15030 shutdown 2024-09-11T21:56:42.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.668+0000 7fd7e5936740 1 -- 172.21.15.1:0/3911185677 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5607f9024bb0 msgr2=0x5607f9021ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:42.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.668+0000 7fd7e5936740 1 --2- 172.21.15.1:0/3911185677 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5607f9024bb0 0x5607f9021ce0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc0697d0 tx=0x7fd7d0014040 comp rx=0 tx=0).stop 2024-09-11T21:56:42.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.669+0000 7fd7e5936740 1 -- 172.21.15.1:0/3911185677 shutdown_connections 2024-09-11T21:56:42.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.669+0000 7fd7e5936740 1 --2- 172.21.15.1:0/3911185677 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5607f90356a0 0x5607f9036b10 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:42.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.669+0000 7fd7e5936740 1 --2- 172.21.15.1:0/3911185677 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5607f9024bb0 0x5607f9021ce0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:42.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.669+0000 7fd7e5936740 1 --2- 172.21.15.1:0/3911185677 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5607f901a2f0 0x5607f90217a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:42.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.669+0000 7fd7e5936740 1 --2- 172.21.15.1:0/3911185677 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5607f90199d0 0x5607f9024450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:42.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.669+0000 7fd7e5936740 1 -- 172.21.15.1:0/3911185677 >> 172.21.15.1:0/3911185677 conn(0x5607f9015f60 msgr2=0x5607f901bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:42.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.669+0000 7fd7c0ff9640 0 client.15030 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T21:56:42.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.669+0000 7fd7e5936740 1 -- 172.21.15.1:0/3911185677 shutdown_connections 2024-09-11T21:56:42.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.669+0000 7fd7e5936740 1 -- 172.21.15.1:0/3911185677 wait complete. 2024-09-11T21:56:42.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.669+0000 7fd7e5936740 20 client.15030 trim_cache size 0 max 16384 2024-09-11T21:56:42.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.688+0000 7fd7e5936740 1 Processor -- start 2024-09-11T21:56:42.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.688+0000 7fd7e5936740 1 -- start start 2024-09-11T21:56:42.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.688+0000 7fd7e5936740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5607f90199d0 0x5607f9034fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:42.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.688+0000 7fd7e5936740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5607f901a2f0 0x5607f9036440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:42.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.688+0000 7fd7e5936740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5607f9024bb0 0x5607f9021760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:42.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.688+0000 7fd7e5936740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5607f9034e50 con 0x5607f9024bb0 2024-09-11T21:56:42.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.688+0000 7fd7e5936740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5607f9035520 con 0x5607f90199d0 2024-09-11T21:56:42.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.688+0000 7fd7e5936740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5607f9035bf0 con 0x5607f901a2f0 2024-09-11T21:56:42.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.688+0000 7fd7e0af8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5607f9024bb0 0x5607f9021760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:42.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.688+0000 7fd7e0af8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5607f9024bb0 0x5607f9021760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41988/0 (socket says 172.21.15.1:41988) 2024-09-11T21:56:42.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.688+0000 7fd7e0af8640 1 -- 172.21.15.1:0/4035544619 learned_addr learned my addr 172.21.15.1:0/4035544619 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:56:42.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.688+0000 7fd7e12f9640 1 --2- 172.21.15.1:0/4035544619 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5607f901a2f0 0x5607f9036440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:42.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.688+0000 7fd7e0af8640 1 -- 172.21.15.1:0/4035544619 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5607f901a2f0 msgr2=0x5607f9036440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:56:42.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.688+0000 7fd7e0af8640 1 --2- 172.21.15.1:0/4035544619 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5607f901a2f0 0x5607f9036440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:42.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.688+0000 7fd7e0af8640 1 -- 172.21.15.1:0/4035544619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5607f90199d0 msgr2=0x5607f9034fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:56:42.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.688+0000 7fd7e0af8640 1 --2- 172.21.15.1:0/4035544619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5607f90199d0 0x5607f9034fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:42.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.688+0000 7fd7e0af8640 1 -- 172.21.15.1:0/4035544619 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607f9019800 con 0x5607f9024bb0 2024-09-11T21:56:42.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.688+0000 7fd7e12f9640 1 --2- 172.21.15.1:0/4035544619 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5607f901a2f0 0x5607f9036440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T21:56:42.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.688+0000 7fd7e0af8640 1 --2- 172.21.15.1:0/4035544619 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5607f9024bb0 0x5607f9021760 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd7d0002230 tx=0x7fd7d0010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c86e1c53c85648b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:42.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.689+0000 7fd7c2ffd640 1 -- 172.21.15.1:0/4035544619 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d0012ab0 con 0x5607f9024bb0 2024-09-11T21:56:42.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.689+0000 7fd7c2ffd640 1 -- 172.21.15.1:0/4035544619 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7d00120c0 con 0x5607f9024bb0 2024-09-11T21:56:42.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.689+0000 7fd7c2ffd640 1 -- 172.21.15.1:0/4035544619 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d002b8a0 con 0x5607f9024bb0 2024-09-11T21:56:42.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.689+0000 7fd7e5936740 1 -- 172.21.15.1:0/4035544619 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5607f9024bb0 msgr2=0x5607f9021760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:42.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.689+0000 7fd7e5936740 1 --2- 172.21.15.1:0/4035544619 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5607f9024bb0 0x5607f9021760 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd7d0002230 tx=0x7fd7d0010d50 comp rx=0 tx=0).stop 2024-09-11T21:56:42.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.689+0000 7fd7e5936740 1 -- 172.21.15.1:0/4035544619 shutdown_connections 2024-09-11T21:56:42.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.689+0000 7fd7e5936740 1 --2- 172.21.15.1:0/4035544619 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5607f9024bb0 0x5607f9021760 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:42.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.689+0000 7fd7e5936740 1 --2- 172.21.15.1:0/4035544619 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5607f901a2f0 0x5607f9036440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:42.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.689+0000 7fd7e5936740 1 --2- 172.21.15.1:0/4035544619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5607f90199d0 0x5607f9034fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:42.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.689+0000 7fd7e5936740 1 -- 172.21.15.1:0/4035544619 >> 172.21.15.1:0/4035544619 conn(0x5607f9015f60 msgr2=0x5607f9016340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:42.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.689+0000 7fd7e5936740 1 -- 172.21.15.1:0/4035544619 shutdown_connections 2024-09-11T21:56:42.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.689+0000 7fd7e5936740 1 -- 172.21.15.1:0/4035544619 wait complete. 2024-09-11T21:56:42.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.690+0000 7fd7e5936740 1 -- start start 2024-09-11T21:56:42.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.690+0000 7fd7e5936740 1 Processor -- start 2024-09-11T21:56:42.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.690+0000 7fd7e5936740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5607f90199d0 0x5607f9039ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:42.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.690+0000 7fd7e5936740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5607f901a2f0 0x5607f903a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:42.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.690+0000 7fd7e5936740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5607f9024bb0 0x5607f903a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:42.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.690+0000 7fd7e5936740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5607f9036990 con 0x5607f9024bb0 2024-09-11T21:56:42.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.690+0000 7fd7e5936740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5607f90362c0 con 0x5607f90199d0 2024-09-11T21:56:42.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.690+0000 7fd7e5936740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5607f9037060 con 0x5607f901a2f0 2024-09-11T21:56:42.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.690+0000 7fd7e5936740 10 client.0 _ll_register_callbacks cb 0x7fd7d0009d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:56:42.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.690+0000 7fd7e0af8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5607f9024bb0 0x5607f903a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:42.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.690+0000 7fd7e0af8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5607f9024bb0 0x5607f903a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41994/0 (socket says 172.21.15.1:41994) 2024-09-11T21:56:42.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.690+0000 7fd7e0af8640 1 -- 172.21.15.1:0/3876447736 learned_addr learned my addr 172.21.15.1:0/3876447736 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:56:42.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.690+0000 7fd7e0af8640 1 --2- 172.21.15.1:0/3876447736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5607f90199d0 0x5607f9039ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:42.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.690+0000 7fd7e12f9640 1 --2- 172.21.15.1:0/3876447736 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5607f901a2f0 0x5607f903a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:42.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.691+0000 7fd7e0af8640 1 -- 172.21.15.1:0/3876447736 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5607f901a2f0 msgr2=0x5607f903a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:56:42.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.691+0000 7fd7e0af8640 1 --2- 172.21.15.1:0/3876447736 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5607f901a2f0 0x5607f903a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:42.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.691+0000 7fd7e0af8640 1 -- 172.21.15.1:0/3876447736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5607f90199d0 msgr2=0x5607f9039ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:56:42.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.691+0000 7fd7e0af8640 1 --2- 172.21.15.1:0/3876447736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5607f90199d0 0x5607f9039ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:42.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.691+0000 7fd7e0af8640 1 -- 172.21.15.1:0/3876447736 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607f909df40 con 0x5607f9024bb0 2024-09-11T21:56:42.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.691+0000 7fd7e0af8640 1 --2- 172.21.15.1:0/3876447736 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5607f9024bb0 0x5607f903a720 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd7d0005fd0 tx=0x7fd7d002b420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49191ac7f9f76764 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:42.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.691+0000 7fd7c0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T21:56:42.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.691+0000 7fd7c0ff9640 1 -- 172.21.15.1:0/3876447736 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d003b060 con 0x5607f9024bb0 2024-09-11T21:56:42.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.691+0000 7fd7c0ff9640 1 -- 172.21.15.1:0/3876447736 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fd7d0004cb0 con 0x5607f9024bb0 2024-09-11T21:56:42.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.691+0000 7fd7e5936740 1 -- 172.21.15.1:0/3876447736 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7d00120c0 con 0x5607f9024bb0 2024-09-11T21:56:42.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.691+0000 7fd7c0ff9640 1 -- 172.21.15.1:0/3876447736 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd7d002ba80 con 0x5607f9024bb0 2024-09-11T21:56:42.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.691+0000 7fd7e5936740 10 client.15036 fetch_fsmap learned FSMap version 27 2024-09-11T21:56:42.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.691+0000 7fd7e5936740 1 -- 172.21.15.1:0/3876447736 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5607f909f150 con 0x5607f9024bb0 2024-09-11T21:56:42.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.691+0000 7fd7c0ff9640 1 -- 172.21.15.1:0/3876447736 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7d00120c0 con 0x5607f9024bb0 2024-09-11T21:56:42.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.691+0000 7fd7c0ff9640 1 -- 172.21.15.1:0/3876447736 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd7d00052d0 con 0x5607f9024bb0 2024-09-11T21:56:42.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.692+0000 7fd7e5936740 10 client.15036 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T21:56:42.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.692+0000 7fd7e5936740 10 client.15036 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T21:56:42.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.692+0000 7fd7e5936740 20 client.15036 populate_metadata read hostname 'smithi001' 2024-09-11T21:56:42.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.692+0000 7fd7e5936740 1 --2- 172.21.15.1:0/3876447736 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5607f90a3a40 0x5607f90a2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:42.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.692+0000 7fd7e5936740 4 client.15036 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T21:56:42.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.692+0000 7fd7e5936740 1 -- 172.21.15.1:0/3876447736 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "status"}) -- 0x5607f8f87b10 con 0x5607f90a3a40 2024-09-11T21:56:42.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.692+0000 7fd7e12f9640 1 --2- 172.21.15.1:0/3876447736 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5607f90a3a40 0x5607f90a2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:42.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.692+0000 7fd7e12f9640 1 --2- 172.21.15.1:0/3876447736 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5607f90a3a40 0x5607f90a2750 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43273388f8b54c15 server_cookie=11386d5cda468733 in_seq=0 out_seq=0 2024-09-11T21:56:42.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.692+0000 7fd7c0ff9640 10 client.15036 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T21:56:42.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.693+0000 7fd7c0ff9640 1 -- 172.21.15.1:0/3876447736 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x5607f8f87b10 con 0x5607f90a3a40 2024-09-11T21:56:42.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.693+0000 7fd7c0ff9640 10 client.15036 handle_command_reply: tid=0 2024-09-11T21:56:42.732 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-11T21:56:42.732 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "3ffca83c-7087-11ef-bcea-c7b262605968", 2024-09-11T21:56:42.732 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 0, 2024-09-11T21:56:42.732 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 24494, 2024-09-11T21:56:42.732 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-09-11T21:56:42.732 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-09-11T21:56:42.732 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-09-11T21:56:42.732 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 81.604534684000001, 2024-09-11T21:56:42.732 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 27, 2024-09-11T21:56:42.732 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 86, 2024-09-11T21:56:42.732 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 84, 2024-09-11T21:56:42.733 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 92.274207038 2024-09-11T21:56:42.733 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-11T21:56:42.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.693+0000 7fd7e5936740 1 client.15036 shutdown 2024-09-11T21:56:42.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.693+0000 7fd7e5936740 1 -- 172.21.15.1:0/3876447736 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5607f9024bb0 msgr2=0x5607f903a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:42.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.693+0000 7fd7e5936740 1 --2- 172.21.15.1:0/3876447736 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5607f9024bb0 0x5607f903a720 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd7d0005fd0 tx=0x7fd7d002b420 comp rx=0 tx=0).stop 2024-09-11T21:56:42.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.693+0000 7fd7e5936740 1 -- 172.21.15.1:0/3876447736 shutdown_connections 2024-09-11T21:56:42.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.693+0000 7fd7e5936740 1 --2- 172.21.15.1:0/3876447736 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5607f90a3a40 0x5607f90a2750 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:42.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.693+0000 7fd7e5936740 1 --2- 172.21.15.1:0/3876447736 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5607f9024bb0 0x5607f903a720 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:42.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.693+0000 7fd7e5936740 1 --2- 172.21.15.1:0/3876447736 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5607f901a2f0 0x5607f903a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:42.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.693+0000 7fd7c0ff9640 0 client.15036 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T21:56:42.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.693+0000 7fd7e5936740 1 --2- 172.21.15.1:0/3876447736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5607f90199d0 0x5607f9039ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:42.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.693+0000 7fd7e5936740 1 -- 172.21.15.1:0/3876447736 >> 172.21.15.1:0/3876447736 conn(0x5607f9015f60 msgr2=0x5607f9021760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:42.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.693+0000 7fd7e5936740 1 -- 172.21.15.1:0/3876447736 shutdown_connections 2024-09-11T21:56:42.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.693+0000 7fd7e5936740 1 -- 172.21.15.1:0/3876447736 wait complete. 2024-09-11T21:56:42.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.693+0000 7fd7e5936740 20 client.15036 trim_cache size 0 max 16384 2024-09-11T21:56:42.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.693+0000 7fd7e355d640 1 -- 172.21.15.1:0/1394454453 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fd7c40a2c10 msgr2=0x7fd7c40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:42.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.693+0000 7fd7e355d640 1 --2- 172.21.15.1:0/1394454453 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fd7c40a2c10 0x7fd7c40a50d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd7d40bbde0 tx=0x7fd7d408e0f0 comp rx=0 tx=0).stop 2024-09-11T21:56:42.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.693+0000 7fd7e355d640 1 -- 172.21.15.1:0/1394454453 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7dc10c2f0 msgr2=0x7fd7dc19a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:42.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.693+0000 7fd7e355d640 1 --2- 172.21.15.1:0/1394454453 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7dc10c2f0 0x7fd7dc19a5f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd7d4005ab0 tx=0x7fd7d40042f0 comp rx=0 tx=0).stop 2024-09-11T21:56:42.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.694+0000 7fd7e355d640 1 -- 172.21.15.1:0/1394454453 shutdown_connections 2024-09-11T21:56:42.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.694+0000 7fd7e355d640 1 --2- 172.21.15.1:0/1394454453 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fd7c40a2c10 0x7fd7c40a50d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:42.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.694+0000 7fd7e355d640 1 --2- 172.21.15.1:0/1394454453 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7dc10c2f0 0x7fd7dc19a5f0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:42.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.694+0000 7fd7e355d640 1 --2- 172.21.15.1:0/1394454453 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7dc1037a0 0x7fd7dc1a0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:42.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.694+0000 7fd7e355d640 1 --2- 172.21.15.1:0/1394454453 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7dc102df0 0x7fd7dc1a0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:42.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.694+0000 7fd7e355d640 1 -- 172.21.15.1:0/1394454453 >> 172.21.15.1:0/1394454453 conn(0x7fd7dc0fc7d0 msgr2=0x7fd7dc10d560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:42.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.694+0000 7fd7e355d640 1 -- 172.21.15.1:0/1394454453 shutdown_connections 2024-09-11T21:56:42.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:42.694+0000 7fd7e355d640 1 -- 172.21.15.1:0/1394454453 wait complete. 2024-09-11T21:56:42.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:42 smithi053 ceph-mon[34908]: pgmap v324: 97 pgs: 97 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s; 4.8 KiB/s, 0 objects/s recovering 2024-09-11T21:56:43.379 INFO:tasks.cephadm:Setup complete, yielding 2024-09-11T21:56:43.380 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-11T21:56:43.388 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi001.front.sepia.ceph.com 2024-09-11T21:56:43.389 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- bash -c 'ceph orch status' 2024-09-11T21:56:43.645 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:56:44.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:44.118+0000 7f48f4fd5640 1 -- 172.21.15.1:0/3342598301 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f48f0102690 msgr2=0x7f48f0108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:44.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:44.118+0000 7f48f4fd5640 1 --2- 172.21.15.1:0/3342598301 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f48f0102690 0x7f48f0108f20 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f48e8009ca0 tx=0x7f48e802a8c0 comp rx=0 tx=0).stop 2024-09-11T21:56:44.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:44.118+0000 7f48f4fd5640 1 -- 172.21.15.1:0/3342598301 shutdown_connections 2024-09-11T21:56:44.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:44.118+0000 7f48f4fd5640 1 --2- 172.21.15.1:0/3342598301 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f48f0102690 0x7f48f0108f20 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:44.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:44.118+0000 7f48f4fd5640 1 --2- 172.21.15.1:0/3342598301 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48f0101cd0 0x7f48f0102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:44.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:44.118+0000 7f48f4fd5640 1 --2- 172.21.15.1:0/3342598301 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48f0100ad0 0x7f48f0100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:44.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:44.118+0000 7f48f4fd5640 1 -- 172.21.15.1:0/3342598301 >> 172.21.15.1:0/3342598301 conn(0x7f48f00fc280 msgr2=0x7f48f00fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:44.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:44.118+0000 7f48f4fd5640 1 -- 172.21.15.1:0/3342598301 shutdown_connections 2024-09-11T21:56:44.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:44.118+0000 7f48f4fd5640 1 -- 172.21.15.1:0/3342598301 wait complete. 2024-09-11T21:56:44.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:44.119+0000 7f48f4fd5640 1 Processor -- start 2024-09-11T21:56:44.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:44.119+0000 7f48f4fd5640 1 -- start start 2024-09-11T21:56:44.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:44.119+0000 7f48f4fd5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48f0100ad0 0x7f48f006aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:44.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:44.119+0000 7f48f4fd5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f48f0101cd0 0x7f48f019cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:44.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:44.119+0000 7f48f4fd5640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48f0102690 0x7f48f019f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:44.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:44.119+0000 7f48f4fd5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f48f010c800 con 0x7f48f0101cd0 2024-09-11T21:56:44.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:44.119+0000 7f48f4fd5640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f48f010c680 con 0x7f48f0100ad0 2024-09-11T21:56:44.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:44.119+0000 7f48f4fd5640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f48f010c980 con 0x7f48f0102690 2024-09-11T21:56:44.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:44.119+0000 7f48e7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f48f0101cd0 0x7f48f019cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:44.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:44.119+0000 7f48e7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f48f0101cd0 0x7f48f019cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42018/0 (socket says 172.21.15.1:42018) 2024-09-11T21:56:44.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:44.119+0000 7f48ee59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48f0100ad0 0x7f48f006aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:44.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:44.119+0000 7f48e7fff640 1 -- 172.21.15.1:0/730665366 learned_addr learned my addr 172.21.15.1:0/730665366 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:56:44.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:44.119+0000 7f48eed9d640 1 --2- 172.21.15.1:0/730665366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48f0102690 0x7f48f019f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:44.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:44.119+0000 7f48e7fff640 1 -- 172.21.15.1:0/730665366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48f0102690 msgr2=0x7f48f019f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:44.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:44.119+0000 7f48e7fff640 1 --2- 172.21.15.1:0/730665366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48f0102690 0x7f48f019f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:44.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:44.119+0000 7f48e7fff640 1 -- 172.21.15.1:0/730665366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48f0100ad0 msgr2=0x7f48f006aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:44.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:44.119+0000 7f48e7fff640 1 --2- 172.21.15.1:0/730665366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48f0100ad0 0x7f48f006aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:44.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:44.119+0000 7f48e7fff640 1 -- 172.21.15.1:0/730665366 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48f019fb50 con 0x7f48f0101cd0 2024-09-11T21:56:44.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:44.119+0000 7f48e7fff640 1 --2- 172.21.15.1:0/730665366 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f48f0101cd0 0x7f48f019cc20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f48d800b4a0 tx=0x7f48d800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:44.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:44.120+0000 7f48e77fe640 1 -- 172.21.15.1:0/730665366 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d8014070 con 0x7f48f0101cd0 2024-09-11T21:56:44.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:44.120+0000 7f48f4fd5640 1 -- 172.21.15.1:0/730665366 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48f019fde0 con 0x7f48f0101cd0 2024-09-11T21:56:44.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:44.120+0000 7f48e77fe640 1 -- 172.21.15.1:0/730665366 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48d8010af0 con 0x7f48f0101cd0 2024-09-11T21:56:44.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:44.120+0000 7f48e77fe640 1 -- 172.21.15.1:0/730665366 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d8017d10 con 0x7f48f0101cd0 2024-09-11T21:56:44.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:44.120+0000 7f48f4fd5640 1 -- 172.21.15.1:0/730665366 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48f01a0310 con 0x7f48f0101cd0 2024-09-11T21:56:44.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:44.121+0000 7f48f4fd5640 1 -- 172.21.15.1:0/730665366 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48f0107da0 con 0x7f48f0101cd0 2024-09-11T21:56:44.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:44.122+0000 7f48e77fe640 1 -- 172.21.15.1:0/730665366 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f48d8017070 con 0x7f48f0101cd0 2024-09-11T21:56:44.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:44.123+0000 7f48e77fe640 1 --2- 172.21.15.1:0/730665366 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f48dc0a2c10 0x7f48dc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:44.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:44.123+0000 7f48ee59c640 1 --2- 172.21.15.1:0/730665366 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f48dc0a2c10 0x7f48dc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:44.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:44.123+0000 7f48e77fe640 1 -- 172.21.15.1:0/730665366 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f48d80ce880 con 0x7f48f0101cd0 2024-09-11T21:56:44.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:44.123+0000 7f48ee59c640 1 --2- 172.21.15.1:0/730665366 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f48dc0a2c10 0x7f48dc0a50d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f48f0101b30 tx=0x7f48e0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:44.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:44.126+0000 7f48e77fe640 1 -- 172.21.15.1:0/730665366 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f48d809b710 con 0x7f48f0101cd0 2024-09-11T21:56:44.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:44.320+0000 7f48f4fd5640 1 -- 172.21.15.1:0/730665366 --> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f48f0062630 con 0x7f48dc0a2c10 2024-09-11T21:56:44.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:44.322+0000 7f48e77fe640 1 -- 172.21.15.1:0/730665366 <== mgr.14150 v2:172.21.15.1:6800/3890350886 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f48f0062630 con 0x7f48dc0a2c10 2024-09-11T21:56:44.323 INFO:teuthology.orchestra.run.smithi001.stdout:Backend: cephadm 2024-09-11T21:56:44.323 INFO:teuthology.orchestra.run.smithi001.stdout:Available: Yes 2024-09-11T21:56:44.323 INFO:teuthology.orchestra.run.smithi001.stdout:Paused: No 2024-09-11T21:56:44.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:44.324+0000 7f48f4fd5640 1 -- 172.21.15.1:0/730665366 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f48dc0a2c10 msgr2=0x7f48dc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:44.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:44.324+0000 7f48f4fd5640 1 --2- 172.21.15.1:0/730665366 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f48dc0a2c10 0x7f48dc0a50d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f48f0101b30 tx=0x7f48e0005f90 comp rx=0 tx=0).stop 2024-09-11T21:56:44.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:44.324+0000 7f48f4fd5640 1 -- 172.21.15.1:0/730665366 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f48f0101cd0 msgr2=0x7f48f019cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:44.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:44.324+0000 7f48f4fd5640 1 --2- 172.21.15.1:0/730665366 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f48f0101cd0 0x7f48f019cc20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f48d800b4a0 tx=0x7f48d800b960 comp rx=0 tx=0).stop 2024-09-11T21:56:44.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:44.324+0000 7f48f4fd5640 1 -- 172.21.15.1:0/730665366 shutdown_connections 2024-09-11T21:56:44.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:44.324+0000 7f48f4fd5640 1 --2- 172.21.15.1:0/730665366 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f48dc0a2c10 0x7f48dc0a50d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:44.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:44.324+0000 7f48f4fd5640 1 --2- 172.21.15.1:0/730665366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48f0102690 0x7f48f019f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:44.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:44.324+0000 7f48f4fd5640 1 --2- 172.21.15.1:0/730665366 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f48f0101cd0 0x7f48f019cc20 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:44.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:44.324+0000 7f48f4fd5640 1 --2- 172.21.15.1:0/730665366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f48f0100ad0 0x7f48f006aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:44.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:44.324+0000 7f48f4fd5640 1 -- 172.21.15.1:0/730665366 >> 172.21.15.1:0/730665366 conn(0x7f48f00fc280 msgr2=0x7f48f0104f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:44.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:44.324+0000 7f48f4fd5640 1 -- 172.21.15.1:0/730665366 shutdown_connections 2024-09-11T21:56:44.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:44.324+0000 7f48f4fd5640 1 -- 172.21.15.1:0/730665366 wait complete. 2024-09-11T21:56:44.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:44 smithi161 ceph-mon[35615]: pgmap v325: 97 pgs: 97 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T21:56:44.624 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:44 smithi001 ceph-mon[30935]: pgmap v325: 97 pgs: 97 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T21:56:44.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:44 smithi053 ceph-mon[34908]: pgmap v325: 97 pgs: 97 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T21:56:44.931 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-11T21:56:45.190 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:56:45.596 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:45 smithi001 ceph-mon[30935]: from='client.15042 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:56:45.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:45.677+0000 7f0a9a8bc640 1 -- 172.21.15.1:0/2773429562 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a94101c00 msgr2=0x7f0a94102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:45.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:45.677+0000 7f0a9a8bc640 1 --2- 172.21.15.1:0/2773429562 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a94101c00 0x7f0a94102000 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0a84009900 tx=0x7f0a8402a410 comp rx=0 tx=0).stop 2024-09-11T21:56:45.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:45.677+0000 7f0a9a8bc640 1 -- 172.21.15.1:0/2773429562 shutdown_connections 2024-09-11T21:56:45.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:45.677+0000 7f0a9a8bc640 1 --2- 172.21.15.1:0/2773429562 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0a941037c0 0x7f0a9410a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:45.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:45.677+0000 7f0a9a8bc640 1 --2- 172.21.15.1:0/2773429562 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a94102e00 0x7f0a94103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:45.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:45.677+0000 7f0a9a8bc640 1 --2- 172.21.15.1:0/2773429562 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a94101c00 0x7f0a94102000 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:45.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:45.677+0000 7f0a9a8bc640 1 -- 172.21.15.1:0/2773429562 >> 172.21.15.1:0/2773429562 conn(0x7f0a940fd390 msgr2=0x7f0a940ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:45.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:45.677+0000 7f0a9a8bc640 1 -- 172.21.15.1:0/2773429562 shutdown_connections 2024-09-11T21:56:45.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:45.677+0000 7f0a9a8bc640 1 -- 172.21.15.1:0/2773429562 wait complete. 2024-09-11T21:56:45.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:45.678+0000 7f0a9a8bc640 1 Processor -- start 2024-09-11T21:56:45.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:45.678+0000 7f0a9a8bc640 1 -- start start 2024-09-11T21:56:45.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:45.678+0000 7f0a9a8bc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0a94101c00 0x7f0a94198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:45.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:45.678+0000 7f0a9a8bc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a94102e00 0x7f0a94198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:45.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:45.678+0000 7f0a9a8bc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a941037c0 0x7f0a9419f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:45.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:45.678+0000 7f0a9a8bc640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9410d960 con 0x7f0a94102e00 2024-09-11T21:56:45.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:45.678+0000 7f0a9a8bc640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9410d7e0 con 0x7f0a94101c00 2024-09-11T21:56:45.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:45.678+0000 7f0a9a8bc640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9410dae0 con 0x7f0a941037c0 2024-09-11T21:56:45.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:45.678+0000 7f0a937fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a94102e00 0x7f0a94198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:45.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:45.678+0000 7f0a937fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a94102e00 0x7f0a94198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42026/0 (socket says 172.21.15.1:42026) 2024-09-11T21:56:45.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:45.678+0000 7f0a937fe640 1 -- 172.21.15.1:0/2803928136 learned_addr learned my addr 172.21.15.1:0/2803928136 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:56:45.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:45.678+0000 7f0a93fff640 1 --2- 172.21.15.1:0/2803928136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0a94101c00 0x7f0a94198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:45.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:45.678+0000 7f0a98e59640 1 --2- 172.21.15.1:0/2803928136 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a941037c0 0x7f0a9419f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:45.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:45.679+0000 7f0a937fe640 1 -- 172.21.15.1:0/2803928136 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a941037c0 msgr2=0x7f0a9419f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:45.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:45.679+0000 7f0a937fe640 1 --2- 172.21.15.1:0/2803928136 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a941037c0 0x7f0a9419f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:45.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:45.679+0000 7f0a937fe640 1 -- 172.21.15.1:0/2803928136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0a94101c00 msgr2=0x7f0a94198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:45.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:45.679+0000 7f0a937fe640 1 --2- 172.21.15.1:0/2803928136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0a94101c00 0x7f0a94198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:45.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:45.679+0000 7f0a937fe640 1 -- 172.21.15.1:0/2803928136 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a9419fc80 con 0x7f0a94102e00 2024-09-11T21:56:45.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:45.679+0000 7f0a937fe640 1 --2- 172.21.15.1:0/2803928136 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a94102e00 0x7f0a94198590 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0a8800b4a0 tx=0x7f0a8800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:45.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:45.679+0000 7f0a917fa640 1 -- 172.21.15.1:0/2803928136 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a88014070 con 0x7f0a94102e00 2024-09-11T21:56:45.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:45.679+0000 7f0a917fa640 1 -- 172.21.15.1:0/2803928136 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a88010af0 con 0x7f0a94102e00 2024-09-11T21:56:45.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:45.679+0000 7f0a9a8bc640 1 -- 172.21.15.1:0/2803928136 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a9419ff70 con 0x7f0a94102e00 2024-09-11T21:56:45.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:45.679+0000 7f0a917fa640 1 -- 172.21.15.1:0/2803928136 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a88017d10 con 0x7f0a94102e00 2024-09-11T21:56:45.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:45.679+0000 7f0a9a8bc640 1 -- 172.21.15.1:0/2803928136 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a941a04f0 con 0x7f0a94102e00 2024-09-11T21:56:45.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:45.680+0000 7f0a9a8bc640 1 -- 172.21.15.1:0/2803928136 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a941090a0 con 0x7f0a94102e00 2024-09-11T21:56:45.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:45.681+0000 7f0a917fa640 1 -- 172.21.15.1:0/2803928136 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f0a88017070 con 0x7f0a94102e00 2024-09-11T21:56:45.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:45.682+0000 7f0a917fa640 1 --2- 172.21.15.1:0/2803928136 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f0a7c0a2cc0 0x7f0a7c0a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:45.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:45.682+0000 7f0a93fff640 1 --2- 172.21.15.1:0/2803928136 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f0a7c0a2cc0 0x7f0a7c0a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:45.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:45.682+0000 7f0a917fa640 1 -- 172.21.15.1:0/2803928136 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f0a880ce730 con 0x7f0a94102e00 2024-09-11T21:56:45.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:45.682+0000 7f0a93fff640 1 --2- 172.21.15.1:0/2803928136 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f0a7c0a2cc0 0x7f0a7c0a5180 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0a84005ab0 tx=0x7f0a84000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:45.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:45.685+0000 7f0a917fa640 1 -- 172.21.15.1:0/2803928136 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0a8809b620 con 0x7f0a94102e00 2024-09-11T21:56:45.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:45 smithi053 ceph-mon[34908]: from='client.15042 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:56:45.841 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:45 smithi161 ceph-mon[35615]: from='client.15042 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:56:45.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:45.871+0000 7f0a9a8bc640 1 -- 172.21.15.1:0/2803928136 --> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f0a94062630 con 0x7f0a7c0a2cc0 2024-09-11T21:56:45.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:45.884+0000 7f0a917fa640 1 -- 172.21.15.1:0/2803928136 <== mgr.14150 v2:172.21.15.1:6800/3890350886 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4154 (secure 0 0 0) 0x7f0a94062630 con 0x7f0a7c0a2cc0 2024-09-11T21:56:45.888 INFO:teuthology.orchestra.run.smithi001.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-11T21:56:45.888 INFO:teuthology.orchestra.run.smithi001.stdout:mds.a smithi001 running (98s) 20s ago 97s 16.1M - 19.1.1-298-gb53b6d3e 54467abc55f8 78f9f386febc 2024-09-11T21:56:45.889 INFO:teuthology.orchestra.run.smithi001.stdout:mds.b smithi053 running (95s) 28s ago 95s 25.9M - 19.1.1-298-gb53b6d3e 54467abc55f8 7e08278586c4 2024-09-11T21:56:45.889 INFO:teuthology.orchestra.run.smithi001.stdout:mds.c smithi161 running (109s) 28s ago 109s 18.1M - 19.1.1-298-gb53b6d3e 54467abc55f8 a2ca22e4e257 2024-09-11T21:56:45.889 INFO:teuthology.orchestra.run.smithi001.stdout:mds.d smithi001 running (118s) 20s ago 117s 14.4M - 19.1.1-298-gb53b6d3e 54467abc55f8 31b9b50c2867 2024-09-11T21:56:45.889 INFO:teuthology.orchestra.run.smithi001.stdout:mds.e smithi053 running (115s) 28s ago 114s 20.1M - 19.1.1-298-gb53b6d3e 54467abc55f8 43bc380734ba 2024-09-11T21:56:45.889 INFO:teuthology.orchestra.run.smithi001.stdout:mds.f smithi161 running (112s) 28s ago 112s 18.0M - 19.1.1-298-gb53b6d3e 54467abc55f8 dc51508f8558 2024-09-11T21:56:45.889 INFO:teuthology.orchestra.run.smithi001.stdout:mds.g smithi001 running (101s) 20s ago 100s 16.9M - 19.1.1-298-gb53b6d3e 54467abc55f8 c4b4c7930b9d 2024-09-11T21:56:45.889 INFO:teuthology.orchestra.run.smithi001.stdout:mds.h smithi053 running (107s) 28s ago 106s 16.8M - 19.1.1-298-gb53b6d3e 54467abc55f8 84569b980f2d 2024-09-11T21:56:45.889 INFO:teuthology.orchestra.run.smithi001.stdout:mds.i smithi161 running (104s) 28s ago 104s 20.7M - 19.1.1-298-gb53b6d3e 54467abc55f8 55301ac4334f 2024-09-11T21:56:45.889 INFO:teuthology.orchestra.run.smithi001.stdout:mds.j smithi001 running (2m) 20s ago 2m 14.4M - 19.1.1-298-gb53b6d3e 54467abc55f8 7c0ac8cf66ff 2024-09-11T21:56:45.889 INFO:teuthology.orchestra.run.smithi001.stdout:mds.k smithi053 running (2m) 28s ago 2m 19.3M - 19.1.1-298-gb53b6d3e 54467abc55f8 18b148130eb9 2024-09-11T21:56:45.889 INFO:teuthology.orchestra.run.smithi001.stdout:mds.l smithi161 running (2m) 28s ago 2m 19.2M - 19.1.1-298-gb53b6d3e 54467abc55f8 c8172226b478 2024-09-11T21:56:45.889 INFO:teuthology.orchestra.run.smithi001.stdout:mgr.x smithi001 *:9283,8765 running (9m) 20s ago 8m 510M - 19.1.1-298-gb53b6d3e 54467abc55f8 4bf5fcbd71e5 2024-09-11T21:56:45.890 INFO:teuthology.orchestra.run.smithi001.stdout:mgr.y smithi053 *:8443,8765 running (7m) 28s ago 7m 442M - 19.1.1-298-gb53b6d3e 54467abc55f8 a86754c804b9 2024-09-11T21:56:45.890 INFO:teuthology.orchestra.run.smithi001.stdout:mgr.z smithi161 *:8443,8765 running (7m) 28s ago 7m 442M - 19.1.1-298-gb53b6d3e 54467abc55f8 57051efff982 2024-09-11T21:56:45.890 INFO:teuthology.orchestra.run.smithi001.stdout:mon.a smithi001 running (9m) 20s ago 9m 53.5M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 f1d1cf7d2034 2024-09-11T21:56:45.890 INFO:teuthology.orchestra.run.smithi001.stdout:mon.b smithi053 running (7m) 28s ago 7m 42.3M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 90807c5d7261 2024-09-11T21:56:45.890 INFO:teuthology.orchestra.run.smithi001.stdout:mon.c smithi161 running (7m) 28s ago 7m 40.2M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 04ee519b67d4 2024-09-11T21:56:45.890 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0 smithi001 running (7m) 20s ago 7m 48.9M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 975edddfaf6e 2024-09-11T21:56:45.890 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1 smithi053 running (6m) 28s ago 6m 59.8M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 6e03d87d492f 2024-09-11T21:56:45.890 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2 smithi161 running (6m) 28s ago 6m 56.7M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 681a2fb4e57d 2024-09-11T21:56:45.891 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3 smithi001 running (5m) 20s ago 5m 45.7M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 d5ace9b3ae60 2024-09-11T21:56:45.891 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4 smithi053 running (5m) 28s ago 5m 45.2M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 bff20c95cd68 2024-09-11T21:56:45.891 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5 smithi161 running (4m) 28s ago 4m 46.5M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 3c0b50492934 2024-09-11T21:56:45.891 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6 smithi001 running (4m) 20s ago 4m 42.2M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 03adbad21ad7 2024-09-11T21:56:45.891 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7 smithi053 running (4m) 28s ago 4m 56.0M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 74733ddfb279 2024-09-11T21:56:45.891 INFO:teuthology.orchestra.run.smithi001.stdout:osd.8 smithi161 running (3m) 28s ago 3m 55.6M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 603b13a6fa1c 2024-09-11T21:56:45.891 INFO:teuthology.orchestra.run.smithi001.stdout:osd.9 smithi001 running (3m) 20s ago 3m 52.4M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 f668411bc676 2024-09-11T21:56:45.891 INFO:teuthology.orchestra.run.smithi001.stdout:osd.10 smithi053 running (2m) 28s ago 2m 53.6M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 e5d06c9fd99c 2024-09-11T21:56:45.891 INFO:teuthology.orchestra.run.smithi001.stdout:osd.11 smithi161 running (2m) 28s ago 2m 50.0M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 8ab1b5a399c9 2024-09-11T21:56:45.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:45.887+0000 7f0a9a8bc640 1 -- 172.21.15.1:0/2803928136 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f0a7c0a2cc0 msgr2=0x7f0a7c0a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:45.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:45.887+0000 7f0a9a8bc640 1 --2- 172.21.15.1:0/2803928136 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f0a7c0a2cc0 0x7f0a7c0a5180 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0a84005ab0 tx=0x7f0a84000bc0 comp rx=0 tx=0).stop 2024-09-11T21:56:45.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:45.887+0000 7f0a9a8bc640 1 -- 172.21.15.1:0/2803928136 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a94102e00 msgr2=0x7f0a94198590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:45.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:45.887+0000 7f0a9a8bc640 1 --2- 172.21.15.1:0/2803928136 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a94102e00 0x7f0a94198590 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0a8800b4a0 tx=0x7f0a8800b960 comp rx=0 tx=0).stop 2024-09-11T21:56:45.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:45.887+0000 7f0a9a8bc640 1 -- 172.21.15.1:0/2803928136 shutdown_connections 2024-09-11T21:56:45.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:45.887+0000 7f0a9a8bc640 1 --2- 172.21.15.1:0/2803928136 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f0a7c0a2cc0 0x7f0a7c0a5180 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:45.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:45.887+0000 7f0a9a8bc640 1 --2- 172.21.15.1:0/2803928136 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a941037c0 0x7f0a9419f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:45.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:45.887+0000 7f0a9a8bc640 1 --2- 172.21.15.1:0/2803928136 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a94102e00 0x7f0a94198590 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:45.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:45.887+0000 7f0a9a8bc640 1 --2- 172.21.15.1:0/2803928136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0a94101c00 0x7f0a94198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:45.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:45.887+0000 7f0a9a8bc640 1 -- 172.21.15.1:0/2803928136 >> 172.21.15.1:0/2803928136 conn(0x7f0a940fd390 msgr2=0x7f0a94106270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:45.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:45.887+0000 7f0a9a8bc640 1 -- 172.21.15.1:0/2803928136 shutdown_connections 2024-09-11T21:56:45.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:45.887+0000 7f0a9a8bc640 1 -- 172.21.15.1:0/2803928136 wait complete. 2024-09-11T21:56:46.560 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:46 smithi001 ceph-mon[30935]: pgmap v326: 97 pgs: 97 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-11T21:56:46.589 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-11T21:56:46.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:46 smithi053 ceph-mon[34908]: pgmap v326: 97 pgs: 97 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-11T21:56:46.841 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:46 smithi161 ceph-mon[35615]: pgmap v326: 97 pgs: 97 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-11T21:56:46.846 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:56:47.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:47.327+0000 7fcfbf82e640 1 -- 172.21.15.1:0/330860467 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcfb81037c0 msgr2=0x7fcfb810a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:47.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:47.327+0000 7fcfbf82e640 1 --2- 172.21.15.1:0/330860467 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcfb81037c0 0x7fcfb810a050 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fcfb0009ca0 tx=0x7fcfb002a8c0 comp rx=0 tx=0).stop 2024-09-11T21:56:47.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:47.327+0000 7fcfbf82e640 1 -- 172.21.15.1:0/330860467 shutdown_connections 2024-09-11T21:56:47.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:47.327+0000 7fcfbf82e640 1 --2- 172.21.15.1:0/330860467 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcfb81037c0 0x7fcfb810a050 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:47.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:47.327+0000 7fcfbf82e640 1 --2- 172.21.15.1:0/330860467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfb8102e00 0x7fcfb8103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:47.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:47.327+0000 7fcfbf82e640 1 --2- 172.21.15.1:0/330860467 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfb8101c00 0x7fcfb8102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:47.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:47.327+0000 7fcfbf82e640 1 -- 172.21.15.1:0/330860467 >> 172.21.15.1:0/330860467 conn(0x7fcfb80fd390 msgr2=0x7fcfb80ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:47.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:47.328+0000 7fcfbf82e640 1 -- 172.21.15.1:0/330860467 shutdown_connections 2024-09-11T21:56:47.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:47.328+0000 7fcfbf82e640 1 -- 172.21.15.1:0/330860467 wait complete. 2024-09-11T21:56:47.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:47.328+0000 7fcfbf82e640 1 Processor -- start 2024-09-11T21:56:47.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:47.328+0000 7fcfbf82e640 1 -- start start 2024-09-11T21:56:47.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:47.328+0000 7fcfbf82e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfb8101c00 0x7fcfb8197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:47.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:47.328+0000 7fcfbf82e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcfb8102e00 0x7fcfb81983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:47.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:47.328+0000 7fcfbf82e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfb81037c0 0x7fcfb819f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:47.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:47.328+0000 7fcfbf82e640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb810d8e0 con 0x7fcfb8102e00 2024-09-11T21:56:47.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:47.328+0000 7fcfbf82e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb810d760 con 0x7fcfb8101c00 2024-09-11T21:56:47.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:47.328+0000 7fcfbf82e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb810da60 con 0x7fcfb81037c0 2024-09-11T21:56:47.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:47.328+0000 7fcfbcdc9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcfb8102e00 0x7fcfb81983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:47.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:47.329+0000 7fcfbcdc9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcfb8102e00 0x7fcfb81983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57876/0 (socket says 172.21.15.1:57876) 2024-09-11T21:56:47.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:47.329+0000 7fcfbcdc9640 1 -- 172.21.15.1:0/409438490 learned_addr learned my addr 172.21.15.1:0/409438490 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:56:47.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:47.329+0000 7fcfbddcb640 1 --2- 172.21.15.1:0/409438490 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfb81037c0 0x7fcfb819f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:47.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:47.329+0000 7fcfbd5ca640 1 --2- 172.21.15.1:0/409438490 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfb8101c00 0x7fcfb8197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:47.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:47.329+0000 7fcfbcdc9640 1 -- 172.21.15.1:0/409438490 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfb81037c0 msgr2=0x7fcfb819f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:47.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:47.329+0000 7fcfbcdc9640 1 --2- 172.21.15.1:0/409438490 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfb81037c0 0x7fcfb819f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:47.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:47.329+0000 7fcfbcdc9640 1 -- 172.21.15.1:0/409438490 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfb8101c00 msgr2=0x7fcfb8197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:47.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:47.329+0000 7fcfbcdc9640 1 --2- 172.21.15.1:0/409438490 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfb8101c00 0x7fcfb8197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:47.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:47.329+0000 7fcfbcdc9640 1 -- 172.21.15.1:0/409438490 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfb819fad0 con 0x7fcfb8102e00 2024-09-11T21:56:47.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:47.329+0000 7fcfbd5ca640 1 --2- 172.21.15.1:0/409438490 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfb8101c00 0x7fcfb8197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T21:56:47.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:47.329+0000 7fcfbcdc9640 1 --2- 172.21.15.1:0/409438490 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcfb8102e00 0x7fcfb81983e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fcfb400b4a0 tx=0x7fcfb400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:47.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:47.329+0000 7fcfae7fc640 1 -- 172.21.15.1:0/409438490 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb4014070 con 0x7fcfb8102e00 2024-09-11T21:56:47.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:47.329+0000 7fcfae7fc640 1 -- 172.21.15.1:0/409438490 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfb4010af0 con 0x7fcfb8102e00 2024-09-11T21:56:47.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:47.329+0000 7fcfbf82e640 1 -- 172.21.15.1:0/409438490 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfb819fdc0 con 0x7fcfb8102e00 2024-09-11T21:56:47.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:47.329+0000 7fcfae7fc640 1 -- 172.21.15.1:0/409438490 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb4017d10 con 0x7fcfb8102e00 2024-09-11T21:56:47.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:47.329+0000 7fcfbf82e640 1 -- 172.21.15.1:0/409438490 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfb81a0300 con 0x7fcfb8102e00 2024-09-11T21:56:47.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:47.330+0000 7fcfbf82e640 1 -- 172.21.15.1:0/409438490 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfb8108fb0 con 0x7fcfb8102e00 2024-09-11T21:56:47.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:47.331+0000 7fcfae7fc640 1 -- 172.21.15.1:0/409438490 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fcfb4017070 con 0x7fcfb8102e00 2024-09-11T21:56:47.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:47.332+0000 7fcfae7fc640 1 --2- 172.21.15.1:0/409438490 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fcfa40a2cc0 0x7fcfa40a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:47.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:47.332+0000 7fcfbd5ca640 1 --2- 172.21.15.1:0/409438490 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fcfa40a2cc0 0x7fcfa40a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:47.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:47.332+0000 7fcfae7fc640 1 -- 172.21.15.1:0/409438490 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fcfb40ce440 con 0x7fcfb8102e00 2024-09-11T21:56:47.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:47.333+0000 7fcfbd5ca640 1 --2- 172.21.15.1:0/409438490 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fcfa40a2cc0 0x7fcfa40a5180 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8102c60 tx=0x7fcfa8005db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:47.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:47.336+0000 7fcfae7fc640 1 -- 172.21.15.1:0/409438490 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcfb409b330 con 0x7fcfb8102e00 2024-09-11T21:56:47.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:47.526+0000 7fcfbf82e640 1 -- 172.21.15.1:0/409438490 --> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fcfb8062630 con 0x7fcfa40a2cc0 2024-09-11T21:56:47.528 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:47 smithi001 ceph-mon[30935]: from='client.15048 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:56:47.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:47.533+0000 7fcfae7fc640 1 -- 172.21.15.1:0/409438490 <== mgr.14150 v2:172.21.15.1:6800/3890350886 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fcfb8062630 con 0x7fcfa40a2cc0 2024-09-11T21:56:47.534 INFO:teuthology.orchestra.run.smithi001.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-11T21:56:47.534 INFO:teuthology.orchestra.run.smithi001.stdout:mds.all 12/12 30s ago 2m smithi001=a;smithi001=d;smithi001=g;smithi001=j;smithi053=b;smithi053=e;smithi053=h;smithi053=k;smithi161=c;smithi161=f;smithi161=i;smithi161=l;count:12 2024-09-11T21:56:47.534 INFO:teuthology.orchestra.run.smithi001.stdout:mgr 3/3 30s ago 7m smithi001=x;smithi053=y;smithi161=z;count:3 2024-09-11T21:56:47.534 INFO:teuthology.orchestra.run.smithi001.stdout:mon 3/3 30s ago 7m smithi001:172.21.15.1=a;smithi053:172.21.15.53=b;smithi161:172.21.15.161=c;count:3 2024-09-11T21:56:47.534 INFO:teuthology.orchestra.run.smithi001.stdout:osd 12 30s ago - 2024-09-11T21:56:47.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:47.536+0000 7fcfbf82e640 1 -- 172.21.15.1:0/409438490 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fcfa40a2cc0 msgr2=0x7fcfa40a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:47.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:47.536+0000 7fcfbf82e640 1 --2- 172.21.15.1:0/409438490 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fcfa40a2cc0 0x7fcfa40a5180 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8102c60 tx=0x7fcfa8005db0 comp rx=0 tx=0).stop 2024-09-11T21:56:47.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:47.536+0000 7fcfbf82e640 1 -- 172.21.15.1:0/409438490 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcfb8102e00 msgr2=0x7fcfb81983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:47.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:47.536+0000 7fcfbf82e640 1 --2- 172.21.15.1:0/409438490 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcfb8102e00 0x7fcfb81983e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fcfb400b4a0 tx=0x7fcfb400b960 comp rx=0 tx=0).stop 2024-09-11T21:56:47.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:47.536+0000 7fcfbf82e640 1 -- 172.21.15.1:0/409438490 shutdown_connections 2024-09-11T21:56:47.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:47.536+0000 7fcfbf82e640 1 --2- 172.21.15.1:0/409438490 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fcfa40a2cc0 0x7fcfa40a5180 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:47.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:47.536+0000 7fcfbf82e640 1 --2- 172.21.15.1:0/409438490 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfb81037c0 0x7fcfb819f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:47.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:47.536+0000 7fcfbf82e640 1 --2- 172.21.15.1:0/409438490 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcfb8102e00 0x7fcfb81983e0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:47.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:47.536+0000 7fcfbf82e640 1 --2- 172.21.15.1:0/409438490 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfb8101c00 0x7fcfb8197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:47.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:47.536+0000 7fcfbf82e640 1 -- 172.21.15.1:0/409438490 >> 172.21.15.1:0/409438490 conn(0x7fcfb80fd390 msgr2=0x7fcfb8106180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:47.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:47.536+0000 7fcfbf82e640 1 -- 172.21.15.1:0/409438490 shutdown_connections 2024-09-11T21:56:47.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:47.536+0000 7fcfbf82e640 1 -- 172.21.15.1:0/409438490 wait complete. 2024-09-11T21:56:47.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:47 smithi053 ceph-mon[34908]: from='client.15048 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:56:47.841 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:47 smithi161 ceph-mon[35615]: from='client.15048 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:56:48.253 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-11T21:56:48.511 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:56:48.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:48 smithi161 ceph-mon[35615]: pgmap v327: 97 pgs: 97 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T21:56:48.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:48 smithi161 ceph-mon[35615]: from='client.15054 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:56:48.632 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:48 smithi001 ceph-mon[30935]: pgmap v327: 97 pgs: 97 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T21:56:48.632 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:48 smithi001 ceph-mon[30935]: from='client.15054 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:56:48.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:48 smithi053 ceph-mon[34908]: pgmap v327: 97 pgs: 97 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T21:56:48.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:48 smithi053 ceph-mon[34908]: from='client.15054 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:56:48.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:48.972+0000 7f5fc4eb7640 1 -- 172.21.15.1:0/990111589 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5fc0101c00 msgr2=0x7f5fc0102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:48.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:48.972+0000 7f5fc4eb7640 1 --2- 172.21.15.1:0/990111589 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5fc0101c00 0x7f5fc0102000 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5fb0009900 tx=0x7f5fb002a3f0 comp rx=0 tx=0).stop 2024-09-11T21:56:48.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:48.973+0000 7f5fc4eb7640 1 -- 172.21.15.1:0/990111589 shutdown_connections 2024-09-11T21:56:48.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:48.973+0000 7f5fc4eb7640 1 --2- 172.21.15.1:0/990111589 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fc01037c0 0x7f5fc010a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:48.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:48.973+0000 7f5fc4eb7640 1 --2- 172.21.15.1:0/990111589 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5fc0102e00 0x7f5fc0103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:48.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:48.973+0000 7f5fc4eb7640 1 --2- 172.21.15.1:0/990111589 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5fc0101c00 0x7f5fc0102000 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:48.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:48.973+0000 7f5fc4eb7640 1 -- 172.21.15.1:0/990111589 >> 172.21.15.1:0/990111589 conn(0x7f5fc00fd390 msgr2=0x7f5fc00ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:48.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:48.973+0000 7f5fc4eb7640 1 -- 172.21.15.1:0/990111589 shutdown_connections 2024-09-11T21:56:48.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:48.973+0000 7f5fc4eb7640 1 -- 172.21.15.1:0/990111589 wait complete. 2024-09-11T21:56:48.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:48.973+0000 7f5fc4eb7640 1 Processor -- start 2024-09-11T21:56:48.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:48.973+0000 7f5fc4eb7640 1 -- start start 2024-09-11T21:56:48.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:48.974+0000 7f5fc4eb7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5fc0101c00 0x7f5fc0197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:48.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:48.974+0000 7f5fc4eb7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fc0102e00 0x7f5fc01983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:48.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:48.974+0000 7f5fc4eb7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5fc01037c0 0x7f5fc019f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:48.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:48.974+0000 7f5fc4eb7640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc010d860 con 0x7f5fc01037c0 2024-09-11T21:56:48.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:48.974+0000 7f5fc4eb7640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc010d6e0 con 0x7f5fc0102e00 2024-09-11T21:56:48.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:48.974+0000 7f5fc4eb7640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc010d9e0 con 0x7f5fc0101c00 2024-09-11T21:56:48.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:48.974+0000 7f5fbed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5fc01037c0 0x7f5fc019f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:48.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:48.974+0000 7f5fbed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5fc01037c0 0x7f5fc019f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57902/0 (socket says 172.21.15.1:57902) 2024-09-11T21:56:48.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:48.974+0000 7f5fbed9d640 1 -- 172.21.15.1:0/1367746960 learned_addr learned my addr 172.21.15.1:0/1367746960 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:56:48.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:48.974+0000 7f5fbe59c640 1 --2- 172.21.15.1:0/1367746960 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5fc0101c00 0x7f5fc0197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:48.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:48.974+0000 7f5fbdd9b640 1 --2- 172.21.15.1:0/1367746960 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fc0102e00 0x7f5fc01983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:48.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:48.974+0000 7f5fbed9d640 1 -- 172.21.15.1:0/1367746960 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5fc0101c00 msgr2=0x7f5fc0197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:48.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:48.974+0000 7f5fbed9d640 1 --2- 172.21.15.1:0/1367746960 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5fc0101c00 0x7f5fc0197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:48.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:48.974+0000 7f5fbed9d640 1 -- 172.21.15.1:0/1367746960 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fc0102e00 msgr2=0x7f5fc01983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:48.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:48.974+0000 7f5fbed9d640 1 --2- 172.21.15.1:0/1367746960 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fc0102e00 0x7f5fc01983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:48.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:48.974+0000 7f5fbed9d640 1 -- 172.21.15.1:0/1367746960 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fc019fab0 con 0x7f5fc01037c0 2024-09-11T21:56:48.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:48.974+0000 7f5fbdd9b640 1 --2- 172.21.15.1:0/1367746960 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fc0102e00 0x7f5fc01983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T21:56:48.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:48.974+0000 7f5fbed9d640 1 --2- 172.21.15.1:0/1367746960 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5fc01037c0 0x7f5fc019f440 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5fb800eaa0 tx=0x7f5fb800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:48.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:48.974+0000 7f5faf7fe640 1 -- 172.21.15.1:0/1367746960 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb800cdb0 con 0x7f5fc01037c0 2024-09-11T21:56:48.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:48.974+0000 7f5fc4eb7640 1 -- 172.21.15.1:0/1367746960 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fc019fda0 con 0x7f5fc01037c0 2024-09-11T21:56:48.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:48.974+0000 7f5fc4eb7640 1 -- 172.21.15.1:0/1367746960 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fc01a0250 con 0x7f5fc01037c0 2024-09-11T21:56:48.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:48.974+0000 7f5faf7fe640 1 -- 172.21.15.1:0/1367746960 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fb8005020 con 0x7f5fc01037c0 2024-09-11T21:56:48.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:48.975+0000 7f5faf7fe640 1 -- 172.21.15.1:0/1367746960 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb801f060 con 0x7f5fc01037c0 2024-09-11T21:56:48.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:48.976+0000 7f5fc4eb7640 1 -- 172.21.15.1:0/1367746960 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fc0109000 con 0x7f5fc01037c0 2024-09-11T21:56:48.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:48.977+0000 7f5faf7fe640 1 -- 172.21.15.1:0/1367746960 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f5fb80051d0 con 0x7f5fc01037c0 2024-09-11T21:56:48.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:48.977+0000 7f5faf7fe640 1 --2- 172.21.15.1:0/1367746960 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f5fa80a2c40 0x7f5fa80a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:48.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:48.978+0000 7f5fbe59c640 1 --2- 172.21.15.1:0/1367746960 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f5fa80a2c40 0x7f5fa80a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:48.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:48.978+0000 7f5faf7fe640 1 -- 172.21.15.1:0/1367746960 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f5fb80cf3b0 con 0x7f5fc01037c0 2024-09-11T21:56:48.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:48.978+0000 7f5fbe59c640 1 --2- 172.21.15.1:0/1367746960 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f5fa80a2c40 0x7f5fa80a5100 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5fb0005ab0 tx=0x7f5fb0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:48.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:48.982+0000 7f5faf7fe640 1 -- 172.21.15.1:0/1367746960 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5fb8016030 con 0x7f5fc01037c0 2024-09-11T21:56:49.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:49.163+0000 7f5fc4eb7640 1 -- 172.21.15.1:0/1367746960 --> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f5fc0062630 con 0x7f5fa80a2c40 2024-09-11T21:56:49.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:49.165+0000 7f5faf7fe640 1 -- 172.21.15.1:0/1367746960 <== mgr.14150 v2:172.21.15.1:6800/3890350886 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f5fc0062630 con 0x7f5fa80a2c40 2024-09-11T21:56:49.165 INFO:teuthology.orchestra.run.smithi001.stdout:HOST ADDR LABELS STATUS 2024-09-11T21:56:49.165 INFO:teuthology.orchestra.run.smithi001.stdout:smithi001 172.21.15.1 2024-09-11T21:56:49.166 INFO:teuthology.orchestra.run.smithi001.stdout:smithi053 172.21.15.53 2024-09-11T21:56:49.166 INFO:teuthology.orchestra.run.smithi001.stdout:smithi161 172.21.15.161 2024-09-11T21:56:49.166 INFO:teuthology.orchestra.run.smithi001.stdout:3 hosts in cluster 2024-09-11T21:56:49.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:49.167+0000 7f5fc4eb7640 1 -- 172.21.15.1:0/1367746960 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f5fa80a2c40 msgr2=0x7f5fa80a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:49.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:49.167+0000 7f5fc4eb7640 1 --2- 172.21.15.1:0/1367746960 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f5fa80a2c40 0x7f5fa80a5100 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5fb0005ab0 tx=0x7f5fb0000bc0 comp rx=0 tx=0).stop 2024-09-11T21:56:49.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:49.167+0000 7f5fc4eb7640 1 -- 172.21.15.1:0/1367746960 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5fc01037c0 msgr2=0x7f5fc019f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:49.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:49.167+0000 7f5fc4eb7640 1 --2- 172.21.15.1:0/1367746960 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5fc01037c0 0x7f5fc019f440 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5fb800eaa0 tx=0x7f5fb800ef60 comp rx=0 tx=0).stop 2024-09-11T21:56:49.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:49.167+0000 7f5fc4eb7640 1 -- 172.21.15.1:0/1367746960 shutdown_connections 2024-09-11T21:56:49.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:49.167+0000 7f5fc4eb7640 1 --2- 172.21.15.1:0/1367746960 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f5fa80a2c40 0x7f5fa80a5100 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:49.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:49.167+0000 7f5fc4eb7640 1 --2- 172.21.15.1:0/1367746960 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5fc01037c0 0x7f5fc019f440 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:49.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:49.167+0000 7f5fc4eb7640 1 --2- 172.21.15.1:0/1367746960 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fc0102e00 0x7f5fc01983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:49.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:49.167+0000 7f5fc4eb7640 1 --2- 172.21.15.1:0/1367746960 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5fc0101c00 0x7f5fc0197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:49.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:49.167+0000 7f5fc4eb7640 1 -- 172.21.15.1:0/1367746960 >> 172.21.15.1:0/1367746960 conn(0x7f5fc00fd390 msgr2=0x7f5fc01061d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:49.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:49.167+0000 7f5fc4eb7640 1 -- 172.21.15.1:0/1367746960 shutdown_connections 2024-09-11T21:56:49.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:49.167+0000 7f5fc4eb7640 1 -- 172.21.15.1:0/1367746960 wait complete. 2024-09-11T21:56:49.832 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-11T21:56:50.088 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:56:50.482 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:50 smithi001 ceph-mon[30935]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T21:56:50.482 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:50 smithi001 ceph-mon[30935]: from='client.15060 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:56:50.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:50.566+0000 7faef4c1d640 1 -- 172.21.15.1:0/3759374538 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faef0102690 msgr2=0x7faef0108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:50.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:50.566+0000 7faef4c1d640 1 --2- 172.21.15.1:0/3759374538 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faef0102690 0x7faef0108f20 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7faee8009ca0 tx=0x7faee802a8c0 comp rx=0 tx=0).stop 2024-09-11T21:56:50.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:50.566+0000 7faef4c1d640 1 -- 172.21.15.1:0/3759374538 shutdown_connections 2024-09-11T21:56:50.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:50.566+0000 7faef4c1d640 1 --2- 172.21.15.1:0/3759374538 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faef0102690 0x7faef0108f20 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:50.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:50.566+0000 7faef4c1d640 1 --2- 172.21.15.1:0/3759374538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faef0101cd0 0x7faef0102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:50.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:50.566+0000 7faef4c1d640 1 --2- 172.21.15.1:0/3759374538 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faef0100ad0 0x7faef0100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:50.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:50.566+0000 7faef4c1d640 1 -- 172.21.15.1:0/3759374538 >> 172.21.15.1:0/3759374538 conn(0x7faef00fc280 msgr2=0x7faef00fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:50.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:50.566+0000 7faef4c1d640 1 -- 172.21.15.1:0/3759374538 shutdown_connections 2024-09-11T21:56:50.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:50.566+0000 7faef4c1d640 1 -- 172.21.15.1:0/3759374538 wait complete. 2024-09-11T21:56:50.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:50.566+0000 7faef4c1d640 1 Processor -- start 2024-09-11T21:56:50.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:50.567+0000 7faef4c1d640 1 -- start start 2024-09-11T21:56:50.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:50.567+0000 7faef4c1d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faef0100ad0 0x7faef0197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:50.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:50.567+0000 7faef4c1d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faef0101cd0 0x7faef0198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:50.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:50.567+0000 7faef4c1d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faef0102690 0x7faef019f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:50.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:50.567+0000 7faef4c1d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7faef010d8e0 con 0x7faef0102690 2024-09-11T21:56:50.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:50.567+0000 7faef4c1d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7faef010d760 con 0x7faef0101cd0 2024-09-11T21:56:50.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:50.567+0000 7faef4c1d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7faef010da60 con 0x7faef0100ad0 2024-09-11T21:56:50.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:50.567+0000 7faeeed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faef0102690 0x7faef019f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:50.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:50.567+0000 7faeeed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faef0102690 0x7faef019f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57922/0 (socket says 172.21.15.1:57922) 2024-09-11T21:56:50.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:50.567+0000 7faeeed9d640 1 -- 172.21.15.1:0/985307759 learned_addr learned my addr 172.21.15.1:0/985307759 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:56:50.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:50.567+0000 7faeedd9b640 1 --2- 172.21.15.1:0/985307759 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faef0101cd0 0x7faef0198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:50.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:50.567+0000 7faeee59c640 1 --2- 172.21.15.1:0/985307759 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faef0100ad0 0x7faef0197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:50.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:50.567+0000 7faeeed9d640 1 -- 172.21.15.1:0/985307759 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faef0100ad0 msgr2=0x7faef0197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:50.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:50.567+0000 7faeeed9d640 1 --2- 172.21.15.1:0/985307759 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faef0100ad0 0x7faef0197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:50.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:50.567+0000 7faeeed9d640 1 -- 172.21.15.1:0/985307759 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faef0101cd0 msgr2=0x7faef0198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:50.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:50.567+0000 7faeeed9d640 1 --2- 172.21.15.1:0/985307759 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faef0101cd0 0x7faef0198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:50.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:50.567+0000 7faeeed9d640 1 -- 172.21.15.1:0/985307759 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faef019fb60 con 0x7faef0102690 2024-09-11T21:56:50.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:50.567+0000 7faeee59c640 1 --2- 172.21.15.1:0/985307759 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faef0100ad0 0x7faef0197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T21:56:50.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:50.567+0000 7faeedd9b640 1 --2- 172.21.15.1:0/985307759 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faef0101cd0 0x7faef0198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:56:50.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:50.567+0000 7faeeed9d640 1 --2- 172.21.15.1:0/985307759 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faef0102690 0x7faef019f4f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7faee80098d0 tx=0x7faee802f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:50.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:50.568+0000 7faedf7fe640 1 -- 172.21.15.1:0/985307759 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee802fc30 con 0x7faef0102690 2024-09-11T21:56:50.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:50.568+0000 7faef4c1d640 1 -- 172.21.15.1:0/985307759 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faef019fdf0 con 0x7faef0102690 2024-09-11T21:56:50.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:50.568+0000 7faedf7fe640 1 -- 172.21.15.1:0/985307759 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faee8003e50 con 0x7faef0102690 2024-09-11T21:56:50.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:50.568+0000 7faedf7fe640 1 -- 172.21.15.1:0/985307759 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee80305c0 con 0x7faef0102690 2024-09-11T21:56:50.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:50.568+0000 7faef4c1d640 1 -- 172.21.15.1:0/985307759 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faef01a0320 con 0x7faef0102690 2024-09-11T21:56:50.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:50.569+0000 7faef4c1d640 1 -- 172.21.15.1:0/985307759 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faef0107ef0 con 0x7faef0102690 2024-09-11T21:56:50.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:50.570+0000 7faedf7fe640 1 -- 172.21.15.1:0/985307759 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7faee8004b40 con 0x7faef0102690 2024-09-11T21:56:50.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:50.571+0000 7faedf7fe640 1 --2- 172.21.15.1:0/985307759 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7faed80a2c40 0x7faed80a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:50.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:50.571+0000 7faeee59c640 1 --2- 172.21.15.1:0/985307759 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7faed80a2c40 0x7faed80a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:50.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:50.571+0000 7faedf7fe640 1 -- 172.21.15.1:0/985307759 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7faee80f06e0 con 0x7faef0102690 2024-09-11T21:56:50.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:50.571+0000 7faeee59c640 1 --2- 172.21.15.1:0/985307759 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7faed80a2c40 0x7faed80a5100 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7faee0006fd0 tx=0x7faee0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:50.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:50.574+0000 7faedf7fe640 1 -- 172.21.15.1:0/985307759 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faee80433d0 con 0x7faef0102690 2024-09-11T21:56:50.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:50.756+0000 7faef4c1d640 1 -- 172.21.15.1:0/985307759 --> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7faef0062630 con 0x7faed80a2c40 2024-09-11T21:56:50.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:50.758+0000 7faedf7fe640 1 -- 172.21.15.1:0/985307759 <== mgr.14150 v2:172.21.15.1:6800/3890350886 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7faef0062630 con 0x7faed80a2c40 2024-09-11T21:56:50.759 INFO:teuthology.orchestra.run.smithi001.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-11T21:56:50.759 INFO:teuthology.orchestra.run.smithi001.stdout:smithi001 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7513000D480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-09-11T21:56:50.759 INFO:teuthology.orchestra.run.smithi001.stdout:smithi053 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB751300G8480DGN 447G No 2m ago Has a FileSystem, LVM detected 2024-09-11T21:56:50.759 INFO:teuthology.orchestra.run.smithi001.stdout:smithi161 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6233001X400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-11T21:56:50.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:50.760+0000 7faef4c1d640 1 -- 172.21.15.1:0/985307759 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7faed80a2c40 msgr2=0x7faed80a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:50.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:50.760+0000 7faef4c1d640 1 --2- 172.21.15.1:0/985307759 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7faed80a2c40 0x7faed80a5100 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7faee0006fd0 tx=0x7faee0008040 comp rx=0 tx=0).stop 2024-09-11T21:56:50.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:50.760+0000 7faef4c1d640 1 -- 172.21.15.1:0/985307759 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faef0102690 msgr2=0x7faef019f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:50.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:50.760+0000 7faef4c1d640 1 --2- 172.21.15.1:0/985307759 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faef0102690 0x7faef019f4f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7faee80098d0 tx=0x7faee802f840 comp rx=0 tx=0).stop 2024-09-11T21:56:50.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:50.760+0000 7faef4c1d640 1 -- 172.21.15.1:0/985307759 shutdown_connections 2024-09-11T21:56:50.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:50.760+0000 7faef4c1d640 1 --2- 172.21.15.1:0/985307759 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7faed80a2c40 0x7faed80a5100 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:50.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:50.760+0000 7faef4c1d640 1 --2- 172.21.15.1:0/985307759 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faef0102690 0x7faef019f4f0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:50.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:50.760+0000 7faef4c1d640 1 --2- 172.21.15.1:0/985307759 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faef0101cd0 0x7faef0198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:50.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:50.760+0000 7faef4c1d640 1 --2- 172.21.15.1:0/985307759 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faef0100ad0 0x7faef0197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:50.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:50.760+0000 7faef4c1d640 1 -- 172.21.15.1:0/985307759 >> 172.21.15.1:0/985307759 conn(0x7faef00fc280 msgr2=0x7faef01050c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:50.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:50.760+0000 7faef4c1d640 1 -- 172.21.15.1:0/985307759 shutdown_connections 2024-09-11T21:56:50.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:50.760+0000 7faef4c1d640 1 -- 172.21.15.1:0/985307759 wait complete. 2024-09-11T21:56:50.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:50 smithi053 ceph-mon[34908]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T21:56:50.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:50 smithi053 ceph-mon[34908]: from='client.15060 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:56:50.841 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:50 smithi161 ceph-mon[35615]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T21:56:50.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:50 smithi161 ceph-mon[35615]: from='client.15060 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:56:51.401 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-11T21:56:51.411 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi001.front.sepia.ceph.com 2024-09-11T21:56:51.411 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-11T21:56:51.676 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:56:52.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:52.175+0000 7f7efb7dd640 1 -- 172.21.15.1:0/2262040891 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ef4101cd0 msgr2=0x7f7ef4102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:52.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:52.175+0000 7f7efb7dd640 1 --2- 172.21.15.1:0/2262040891 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ef4101cd0 0x7f7ef4102150 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8009900 tx=0x7f7ee802a410 comp rx=0 tx=0).stop 2024-09-11T21:56:52.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:52.175+0000 7f7efb7dd640 1 -- 172.21.15.1:0/2262040891 shutdown_connections 2024-09-11T21:56:52.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:52.175+0000 7f7efb7dd640 1 --2- 172.21.15.1:0/2262040891 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ef4102690 0x7f7ef4108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:52.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:52.175+0000 7f7efb7dd640 1 --2- 172.21.15.1:0/2262040891 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ef4101cd0 0x7f7ef4102150 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:52.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:52.175+0000 7f7efb7dd640 1 --2- 172.21.15.1:0/2262040891 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ef4100ad0 0x7f7ef4100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:52.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:52.175+0000 7f7efb7dd640 1 -- 172.21.15.1:0/2262040891 >> 172.21.15.1:0/2262040891 conn(0x7f7ef40fc280 msgr2=0x7f7ef40fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:52.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:52.175+0000 7f7efb7dd640 1 -- 172.21.15.1:0/2262040891 shutdown_connections 2024-09-11T21:56:52.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:52.175+0000 7f7efb7dd640 1 -- 172.21.15.1:0/2262040891 wait complete. 2024-09-11T21:56:52.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:52.176+0000 7f7efb7dd640 1 Processor -- start 2024-09-11T21:56:52.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:52.176+0000 7f7efb7dd640 1 -- start start 2024-09-11T21:56:52.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:52.176+0000 7f7efb7dd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ef4100ad0 0x7f7ef4197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:52.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:52.176+0000 7f7efb7dd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ef4101cd0 0x7f7ef4198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:52.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:52.176+0000 7f7efb7dd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ef4102690 0x7f7ef419f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:52.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:52.176+0000 7f7efb7dd640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef410d880 con 0x7f7ef4102690 2024-09-11T21:56:52.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:52.176+0000 7f7efb7dd640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef410d700 con 0x7f7ef4101cd0 2024-09-11T21:56:52.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:52.176+0000 7f7efb7dd640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef410da00 con 0x7f7ef4100ad0 2024-09-11T21:56:52.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:52.176+0000 7f7ef9d7a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ef4102690 0x7f7ef419f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:52.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:52.176+0000 7f7ef9d7a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ef4102690 0x7f7ef419f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57932/0 (socket says 172.21.15.1:57932) 2024-09-11T21:56:52.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:52.177+0000 7f7ef9d7a640 1 -- 172.21.15.1:0/232192551 learned_addr learned my addr 172.21.15.1:0/232192551 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:56:52.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:52.177+0000 7f7ef9579640 1 --2- 172.21.15.1:0/232192551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ef4100ad0 0x7f7ef4197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:52.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:52.177+0000 7f7ef8d78640 1 --2- 172.21.15.1:0/232192551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ef4101cd0 0x7f7ef4198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:52.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:52.177+0000 7f7ef9d7a640 1 -- 172.21.15.1:0/232192551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ef4100ad0 msgr2=0x7f7ef4197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:52.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:52.177+0000 7f7ef9d7a640 1 --2- 172.21.15.1:0/232192551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ef4100ad0 0x7f7ef4197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:52.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:52.177+0000 7f7ef9d7a640 1 -- 172.21.15.1:0/232192551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ef4101cd0 msgr2=0x7f7ef4198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:52.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:52.177+0000 7f7ef9d7a640 1 --2- 172.21.15.1:0/232192551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ef4101cd0 0x7f7ef4198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:52.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:52.177+0000 7f7ef9d7a640 1 -- 172.21.15.1:0/232192551 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ef419fb60 con 0x7f7ef4102690 2024-09-11T21:56:52.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:52.177+0000 7f7ef9579640 1 --2- 172.21.15.1:0/232192551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ef4100ad0 0x7f7ef4197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:56:52.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:52.177+0000 7f7ef9d7a640 1 --2- 172.21.15.1:0/232192551 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ef4102690 0x7f7ef419f4f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f7eec00c970 tx=0x7f7eec00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:52.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:52.177+0000 7f7ef27fc640 1 -- 172.21.15.1:0/232192551 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eec007bb0 con 0x7f7ef4102690 2024-09-11T21:56:52.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:52.177+0000 7f7ef27fc640 1 -- 172.21.15.1:0/232192551 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7eec007d50 con 0x7f7ef4102690 2024-09-11T21:56:52.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:52.177+0000 7f7efb7dd640 1 -- 172.21.15.1:0/232192551 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ef419fe50 con 0x7f7ef4102690 2024-09-11T21:56:52.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:52.177+0000 7f7ef27fc640 1 -- 172.21.15.1:0/232192551 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eec00f450 con 0x7f7ef4102690 2024-09-11T21:56:52.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:52.177+0000 7f7efb7dd640 1 -- 172.21.15.1:0/232192551 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ef41a02b0 con 0x7f7ef4102690 2024-09-11T21:56:52.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:52.178+0000 7f7efb7dd640 1 -- 172.21.15.1:0/232192551 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ef4107f20 con 0x7f7ef4102690 2024-09-11T21:56:52.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:52.180+0000 7f7ef27fc640 1 -- 172.21.15.1:0/232192551 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f7eec004f20 con 0x7f7ef4102690 2024-09-11T21:56:52.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:52.180+0000 7f7ef27fc640 1 --2- 172.21.15.1:0/232192551 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f7edc0a2c40 0x7f7edc0a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:52.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:52.181+0000 7f7ef9579640 1 --2- 172.21.15.1:0/232192551 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f7edc0a2c40 0x7f7edc0a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:52.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:52.181+0000 7f7ef27fc640 1 -- 172.21.15.1:0/232192551 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f7eec0cef20 con 0x7f7ef4102690 2024-09-11T21:56:52.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:52.181+0000 7f7ef9579640 1 --2- 172.21.15.1:0/232192551 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f7edc0a2c40 0x7f7edc0a5100 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4101b30 tx=0x7f7ee4009540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:52.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:52.185+0000 7f7ef27fc640 1 -- 172.21.15.1:0/232192551 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7eec014030 con 0x7f7ef4102690 2024-09-11T21:56:52.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:52.417+0000 7f7efb7dd640 1 -- 172.21.15.1:0/232192551 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f7ef402d7c0 con 0x7f7ef4102690 2024-09-11T21:56:52.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:52.418+0000 7f7ef27fc640 1 -- 172.21.15.1:0/232192551 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+3112 (secure 0 0 0) 0x7f7eec09bd00 con 0x7f7ef4102690 2024-09-11T21:56:52.419 INFO:teuthology.orchestra.run.smithi001.stdout:e27 2024-09-11T21:56:52.419 INFO:teuthology.orchestra.run.smithi001.stdout:btime 2024-09-11T21:55:32:152241+0000 2024-09-11T21:56:52.419 INFO:teuthology.orchestra.run.smithi001.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-11T21:56:52.419 INFO:teuthology.orchestra.run.smithi001.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-11T21:56:52.419 INFO:teuthology.orchestra.run.smithi001.stdout:legacy client fscid: 1 2024-09-11T21:56:52.419 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:56:52.419 INFO:teuthology.orchestra.run.smithi001.stdout:Filesystem 'cephfs' (1) 2024-09-11T21:56:52.419 INFO:teuthology.orchestra.run.smithi001.stdout:fs_name cephfs 2024-09-11T21:56:52.419 INFO:teuthology.orchestra.run.smithi001.stdout:epoch 27 2024-09-11T21:56:52.420 INFO:teuthology.orchestra.run.smithi001.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-11T21:56:52.420 INFO:teuthology.orchestra.run.smithi001.stdout:created 2024-09-11T21:55:20.791263+0000 2024-09-11T21:56:52.420 INFO:teuthology.orchestra.run.smithi001.stdout:modified 2024-09-11T21:55:32.152227+0000 2024-09-11T21:56:52.420 INFO:teuthology.orchestra.run.smithi001.stdout:tableserver 0 2024-09-11T21:56:52.420 INFO:teuthology.orchestra.run.smithi001.stdout:root 0 2024-09-11T21:56:52.420 INFO:teuthology.orchestra.run.smithi001.stdout:session_timeout 300 2024-09-11T21:56:52.420 INFO:teuthology.orchestra.run.smithi001.stdout:session_autoclose 300 2024-09-11T21:56:52.420 INFO:teuthology.orchestra.run.smithi001.stdout:max_file_size 1099511627776 2024-09-11T21:56:52.420 INFO:teuthology.orchestra.run.smithi001.stdout:max_xattr_size 65536 2024-09-11T21:56:52.420 INFO:teuthology.orchestra.run.smithi001.stdout:required_client_features {} 2024-09-11T21:56:52.420 INFO:teuthology.orchestra.run.smithi001.stdout:last_failure 0 2024-09-11T21:56:52.420 INFO:teuthology.orchestra.run.smithi001.stdout:last_failure_osd_epoch 0 2024-09-11T21:56:52.421 INFO:teuthology.orchestra.run.smithi001.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-11T21:56:52.421 INFO:teuthology.orchestra.run.smithi001.stdout:max_mds 5 2024-09-11T21:56:52.421 INFO:teuthology.orchestra.run.smithi001.stdout:in 0,1,2,3,4 2024-09-11T21:56:52.421 INFO:teuthology.orchestra.run.smithi001.stdout:up {0=24494,1=24484,2=24467,3=24466,4=24460} 2024-09-11T21:56:52.421 INFO:teuthology.orchestra.run.smithi001.stdout:failed 2024-09-11T21:56:52.421 INFO:teuthology.orchestra.run.smithi001.stdout:damaged 2024-09-11T21:56:52.421 INFO:teuthology.orchestra.run.smithi001.stdout:stopped 2024-09-11T21:56:52.421 INFO:teuthology.orchestra.run.smithi001.stdout:data_pools [3] 2024-09-11T21:56:52.421 INFO:teuthology.orchestra.run.smithi001.stdout:metadata_pool 2 2024-09-11T21:56:52.421 INFO:teuthology.orchestra.run.smithi001.stdout:inline_data disabled 2024-09-11T21:56:52.421 INFO:teuthology.orchestra.run.smithi001.stdout:balancer 2024-09-11T21:56:52.421 INFO:teuthology.orchestra.run.smithi001.stdout:bal_rank_mask -1 2024-09-11T21:56:52.421 INFO:teuthology.orchestra.run.smithi001.stdout:standby_count_wanted 0 2024-09-11T21:56:52.421 INFO:teuthology.orchestra.run.smithi001.stdout:qdb_cluster leader: 24494 members: 24494,24484,24467,24466,24460 2024-09-11T21:56:52.422 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.b{0:24494} state up:active seq 4 addr [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T21:56:52.422 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.e{0:24449} state up:standby-replay seq 1 addr [v2:172.21.15.53:6834/2126933993,v1:172.21.15.53:6835/2126933993] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T21:56:52.422 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.i{1:24484} state up:active seq 8 addr [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T21:56:52.422 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.k{1:24431} state up:standby-replay seq 1 addr [v2:172.21.15.53:6832/1648944570,v1:172.21.15.53:6833/1648944570] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T21:56:52.422 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.h{2:24467} state up:active seq 9 addr [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T21:56:52.422 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.l{2:24430} state up:standby-replay seq 1 addr [v2:172.21.15.161:6832/872410024,v1:172.21.15.161:6833/872410024] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T21:56:52.422 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.c{3:24466} state up:active seq 10 addr [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T21:56:52.422 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.a{3:14601} state up:standby-replay seq 1 addr [v2:172.21.15.1:6840/1068827678,v1:172.21.15.1:6841/1068827678] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T21:56:52.422 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.f{4:24460} state up:active seq 11 addr [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T21:56:52.422 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.g{4:14595} state up:standby-replay seq 1 addr [v2:172.21.15.1:6838/407834672,v1:172.21.15.1:6839/407834672] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T21:56:52.422 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:56:52.422 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:56:52.422 INFO:teuthology.orchestra.run.smithi001.stdout:Standby daemons: 2024-09-11T21:56:52.422 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:56:52.422 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.j{-1:14526} state up:standby seq 1 addr [v2:172.21.15.1:6834/421542832,v1:172.21.15.1:6835/421542832] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T21:56:52.423 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.d{-1:14538} state up:standby seq 1 addr [v2:172.21.15.1:6836/2836864406,v1:172.21.15.1:6837/2836864406] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T21:56:52.423 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 27 2024-09-11T21:56:52.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:52.420+0000 7f7efb7dd640 1 -- 172.21.15.1:0/232192551 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f7edc0a2c40 msgr2=0x7f7edc0a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:52.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:52.420+0000 7f7efb7dd640 1 --2- 172.21.15.1:0/232192551 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f7edc0a2c40 0x7f7edc0a5100 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4101b30 tx=0x7f7ee4009540 comp rx=0 tx=0).stop 2024-09-11T21:56:52.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:52.420+0000 7f7efb7dd640 1 -- 172.21.15.1:0/232192551 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ef4102690 msgr2=0x7f7ef419f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:52.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:52.420+0000 7f7efb7dd640 1 --2- 172.21.15.1:0/232192551 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ef4102690 0x7f7ef419f4f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f7eec00c970 tx=0x7f7eec00ce30 comp rx=0 tx=0).stop 2024-09-11T21:56:52.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:52.420+0000 7f7efb7dd640 1 -- 172.21.15.1:0/232192551 shutdown_connections 2024-09-11T21:56:52.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:52.420+0000 7f7efb7dd640 1 --2- 172.21.15.1:0/232192551 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f7edc0a2c40 0x7f7edc0a5100 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:52.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:52.420+0000 7f7efb7dd640 1 --2- 172.21.15.1:0/232192551 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ef4102690 0x7f7ef419f4f0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:52.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:52.420+0000 7f7efb7dd640 1 --2- 172.21.15.1:0/232192551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ef4101cd0 0x7f7ef4198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:52.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:52.420+0000 7f7efb7dd640 1 --2- 172.21.15.1:0/232192551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ef4100ad0 0x7f7ef4197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:52.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:52.420+0000 7f7efb7dd640 1 -- 172.21.15.1:0/232192551 >> 172.21.15.1:0/232192551 conn(0x7f7ef40fc280 msgr2=0x7f7ef41050f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:52.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:52.420+0000 7f7efb7dd640 1 -- 172.21.15.1:0/232192551 shutdown_connections 2024-09-11T21:56:52.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:52.421+0000 7f7efb7dd640 1 -- 172.21.15.1:0/232192551 wait complete. 2024-09-11T21:56:52.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:52 smithi001 ceph-mon[30935]: pgmap v329: 97 pgs: 97 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-11T21:56:52.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:52 smithi001 ceph-mon[30935]: from='client.15066 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:56:52.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:52 smithi053 ceph-mon[34908]: pgmap v329: 97 pgs: 97 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-11T21:56:52.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:52 smithi053 ceph-mon[34908]: from='client.15066 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:56:52.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:52 smithi161 ceph-mon[35615]: pgmap v329: 97 pgs: 97 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-11T21:56:52.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:52 smithi161 ceph-mon[35615]: from='client.15066 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:56:53.151 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-11T21:56:53.411 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:56:53.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:53 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/232192551' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-11T21:56:53.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:53 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/232192551' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-11T21:56:53.841 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:53 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/232192551' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-11T21:56:53.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:53.910+0000 7f44ede12640 1 -- 172.21.15.1:0/505472491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f44e8102660 msgr2=0x7f44e8104a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:53.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:53.910+0000 7f44ede12640 1 --2- 172.21.15.1:0/505472491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f44e8102660 0x7f44e8104a50 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f44d8009ca0 tx=0x7f44d802a8c0 comp rx=0 tx=0).stop 2024-09-11T21:56:53.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:53.910+0000 7f44ede12640 1 -- 172.21.15.1:0/505472491 shutdown_connections 2024-09-11T21:56:53.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:53.910+0000 7f44ede12640 1 --2- 172.21.15.1:0/505472491 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f44e81078c0 0x7f44e8109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:53.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:53.910+0000 7f44ede12640 1 --2- 172.21.15.1:0/505472491 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44e8104f90 0x7f44e8107380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:53.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:53.910+0000 7f44ede12640 1 --2- 172.21.15.1:0/505472491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f44e8102660 0x7f44e8104a50 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:53.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:53.910+0000 7f44ede12640 1 -- 172.21.15.1:0/505472491 >> 172.21.15.1:0/505472491 conn(0x7f44e80fc280 msgr2=0x7f44e80fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:53.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:53.910+0000 7f44ede12640 1 -- 172.21.15.1:0/505472491 shutdown_connections 2024-09-11T21:56:53.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:53.910+0000 7f44ede12640 1 -- 172.21.15.1:0/505472491 wait complete. 2024-09-11T21:56:53.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:53.910+0000 7f44ede12640 1 Processor -- start 2024-09-11T21:56:53.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:53.911+0000 7f44ede12640 1 -- start start 2024-09-11T21:56:53.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:53.911+0000 7f44ede12640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f44e8102660 0x7f44e8197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:53.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:53.911+0000 7f44ede12640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44e8104f90 0x7f44e8198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:53.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:53.911+0000 7f44ede12640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f44e81078c0 0x7f44e819f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:53.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:53.911+0000 7f44ede12640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f44e810d890 con 0x7f44e81078c0 2024-09-11T21:56:53.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:53.911+0000 7f44ede12640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f44e810d710 con 0x7f44e8104f90 2024-09-11T21:56:53.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:53.911+0000 7f44ede12640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f44e810da10 con 0x7f44e8102660 2024-09-11T21:56:53.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:53.911+0000 7f44e7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f44e81078c0 0x7f44e819f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:53.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:53.911+0000 7f44e7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f44e81078c0 0x7f44e819f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57956/0 (socket says 172.21.15.1:57956) 2024-09-11T21:56:53.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:53.911+0000 7f44e7fff640 1 -- 172.21.15.1:0/1909433158 learned_addr learned my addr 172.21.15.1:0/1909433158 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:56:53.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:53.911+0000 7f44e6ffd640 1 --2- 172.21.15.1:0/1909433158 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44e8104f90 0x7f44e8198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:53.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:53.911+0000 7f44e77fe640 1 --2- 172.21.15.1:0/1909433158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f44e8102660 0x7f44e8197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:53.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:53.911+0000 7f44e7fff640 1 -- 172.21.15.1:0/1909433158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f44e8102660 msgr2=0x7f44e8197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:53.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:53.911+0000 7f44e7fff640 1 --2- 172.21.15.1:0/1909433158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f44e8102660 0x7f44e8197ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:53.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:53.911+0000 7f44e7fff640 1 -- 172.21.15.1:0/1909433158 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44e8104f90 msgr2=0x7f44e8198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:53.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:53.911+0000 7f44e7fff640 1 --2- 172.21.15.1:0/1909433158 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44e8104f90 0x7f44e8198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:53.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:53.911+0000 7f44e7fff640 1 -- 172.21.15.1:0/1909433158 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44e819faf0 con 0x7f44e81078c0 2024-09-11T21:56:53.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:53.911+0000 7f44e77fe640 1 --2- 172.21.15.1:0/1909433158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f44e8102660 0x7f44e8197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T21:56:53.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:53.911+0000 7f44e6ffd640 1 --2- 172.21.15.1:0/1909433158 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44e8104f90 0x7f44e8198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:56:53.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:53.912+0000 7f44e7fff640 1 --2- 172.21.15.1:0/1909433158 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f44e81078c0 0x7f44e819f480 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f44e000c970 tx=0x7f44e000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:53.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:53.912+0000 7f44e4ff9640 1 -- 172.21.15.1:0/1909433158 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e0007bb0 con 0x7f44e81078c0 2024-09-11T21:56:53.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:53.912+0000 7f44e4ff9640 1 -- 172.21.15.1:0/1909433158 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44e0007d50 con 0x7f44e81078c0 2024-09-11T21:56:53.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:53.912+0000 7f44ede12640 1 -- 172.21.15.1:0/1909433158 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44e819fde0 con 0x7f44e81078c0 2024-09-11T21:56:53.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:53.912+0000 7f44e4ff9640 1 -- 172.21.15.1:0/1909433158 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e000f450 con 0x7f44e81078c0 2024-09-11T21:56:53.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:53.912+0000 7f44ede12640 1 -- 172.21.15.1:0/1909433158 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44e81a02e0 con 0x7f44e81078c0 2024-09-11T21:56:53.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:53.913+0000 7f44ede12640 1 -- 172.21.15.1:0/1909433158 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44e80fdd70 con 0x7f44e81078c0 2024-09-11T21:56:53.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:53.914+0000 7f44e4ff9640 1 -- 172.21.15.1:0/1909433158 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f44e0004ee0 con 0x7f44e81078c0 2024-09-11T21:56:53.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:53.915+0000 7f44e4ff9640 1 --2- 172.21.15.1:0/1909433158 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f44d00a2cc0 0x7f44d00a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:53.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:53.915+0000 7f44e77fe640 1 --2- 172.21.15.1:0/1909433158 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f44d00a2cc0 0x7f44d00a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:53.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:53.915+0000 7f44e4ff9640 1 -- 172.21.15.1:0/1909433158 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f44e00ceed0 con 0x7f44e81078c0 2024-09-11T21:56:53.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:53.915+0000 7f44e77fe640 1 --2- 172.21.15.1:0/1909433158 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f44d00a2cc0 0x7f44d00a5180 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f44d8006010 tx=0x7f44d8000f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:53.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:53.919+0000 7f44e4ff9640 1 -- 172.21.15.1:0/1909433158 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f44e0014030 con 0x7f44e81078c0 2024-09-11T21:56:54.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:54.097+0000 7f44ede12640 1 -- 172.21.15.1:0/1909433158 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f44e802da10 con 0x7f44e81078c0 2024-09-11T21:56:54.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:54.098+0000 7f44e4ff9640 1 -- 172.21.15.1:0/1909433158 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v86) ==== 56+0+5298 (secure 0 0 0) 0x7f44e009bc50 con 0x7f44e81078c0 2024-09-11T21:56:54.099 INFO:teuthology.orchestra.run.smithi001.stdout:epoch 86 2024-09-11T21:56:54.099 INFO:teuthology.orchestra.run.smithi001.stdout:fsid 3ffca83c-7087-11ef-bcea-c7b262605968 2024-09-11T21:56:54.099 INFO:teuthology.orchestra.run.smithi001.stdout:created 2024-09-11T21:47:28.868707+0000 2024-09-11T21:56:54.099 INFO:teuthology.orchestra.run.smithi001.stdout:modified 2024-09-11T21:56:26.278614+0000 2024-09-11T21:56:54.100 INFO:teuthology.orchestra.run.smithi001.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-11T21:56:54.100 INFO:teuthology.orchestra.run.smithi001.stdout:crush_version 26 2024-09-11T21:56:54.100 INFO:teuthology.orchestra.run.smithi001.stdout:full_ratio 0.95 2024-09-11T21:56:54.100 INFO:teuthology.orchestra.run.smithi001.stdout:backfillfull_ratio 0.9 2024-09-11T21:56:54.100 INFO:teuthology.orchestra.run.smithi001.stdout:nearfull_ratio 0.85 2024-09-11T21:56:54.100 INFO:teuthology.orchestra.run.smithi001.stdout:require_min_compat_client luminous 2024-09-11T21:56:54.100 INFO:teuthology.orchestra.run.smithi001.stdout:min_compat_client luminous 2024-09-11T21:56:54.100 INFO:teuthology.orchestra.run.smithi001.stdout:require_osd_release squid 2024-09-11T21:56:54.100 INFO:teuthology.orchestra.run.smithi001.stdout:stretch_mode_enabled false 2024-09-11T21:56:54.100 INFO:teuthology.orchestra.run.smithi001.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-11T21:56:54.100 INFO:teuthology.orchestra.run.smithi001.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-09-11T21:56:54.101 INFO:teuthology.orchestra.run.smithi001.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-09-11T21:56:54.101 INFO:teuthology.orchestra.run.smithi001.stdout:max_osd 12 2024-09-11T21:56:54.101 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0 up in weight 1 up_from 8 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.1:6802/4012142023,v1:172.21.15.1:6803/4012142023] [v2:172.21.15.1:6804/4012142023,v1:172.21.15.1:6805/4012142023] exists,up 19e56504-18f1-4234-b2d6-f193ff47d8eb 2024-09-11T21:56:54.101 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1 up in weight 1 up_from 13 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.53:6800/3039391924,v1:172.21.15.53:6801/3039391924] [v2:172.21.15.53:6802/3039391924,v1:172.21.15.53:6803/3039391924] exists,up 996c9f0c-c5a4-43b8-b8e0-d712f27c5dfa 2024-09-11T21:56:54.101 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2 up in weight 1 up_from 18 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.161:6800/1707914885,v1:172.21.15.161:6801/1707914885] [v2:172.21.15.161:6802/1707914885,v1:172.21.15.161:6803/1707914885] exists,up 578d4260-41e1-4645-b9b1-1c5445712c13 2024-09-11T21:56:54.101 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3 up in weight 1 up_from 25 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.1:6810/1541142398,v1:172.21.15.1:6811/1541142398] [v2:172.21.15.1:6812/1541142398,v1:172.21.15.1:6813/1541142398] exists,up 73f78766-9662-4aa3-841c-fe2ce654ee9c 2024-09-11T21:56:54.101 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4 up in weight 1 up_from 33 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.53:6808/3081402681,v1:172.21.15.53:6809/3081402681] [v2:172.21.15.53:6810/3081402681,v1:172.21.15.53:6811/3081402681] exists,up 56c808bb-d4cd-4969-b49e-adef49ecd0f5 2024-09-11T21:56:54.101 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5 up in weight 1 up_from 38 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.161:6808/1622081327,v1:172.21.15.161:6809/1622081327] [v2:172.21.15.161:6810/1622081327,v1:172.21.15.161:6811/1622081327] exists,up 7b8266d6-b507-47c9-b4fa-ff16c4a74d35 2024-09-11T21:56:54.101 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6 up in weight 1 up_from 46 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.1:6818/158434853,v1:172.21.15.1:6819/158434853] [v2:172.21.15.1:6820/158434853,v1:172.21.15.1:6821/158434853] exists,up 3af5925d-a749-4849-a076-80580222cad4 2024-09-11T21:56:54.101 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7 up in weight 1 up_from 51 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.53:6816/2132057697,v1:172.21.15.53:6817/2132057697] [v2:172.21.15.53:6818/2132057697,v1:172.21.15.53:6819/2132057697] exists,up f8f26c3c-17e9-4e89-8392-289796e0c8a6 2024-09-11T21:56:54.101 INFO:teuthology.orchestra.run.smithi001.stdout:osd.8 up in weight 1 up_from 58 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.161:6816/1838248669,v1:172.21.15.161:6817/1838248669] [v2:172.21.15.161:6818/1838248669,v1:172.21.15.161:6819/1838248669] exists,up 36ea401b-3c93-4b9d-8880-298d1eb6e30b 2024-09-11T21:56:54.101 INFO:teuthology.orchestra.run.smithi001.stdout:osd.9 up in weight 1 up_from 66 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.1:6826/2806274589,v1:172.21.15.1:6827/2806274589] [v2:172.21.15.1:6828/2806274589,v1:172.21.15.1:6829/2806274589] exists,up 4b788d3f-a3bc-4463-bd99-95fdb7a52768 2024-09-11T21:56:54.102 INFO:teuthology.orchestra.run.smithi001.stdout:osd.10 up in weight 1 up_from 71 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.53:6824/761923580,v1:172.21.15.53:6825/761923580] [v2:172.21.15.53:6826/761923580,v1:172.21.15.53:6827/761923580] exists,up 70549796-5270-4ae2-ac3f-f61f5de105d4 2024-09-11T21:56:54.102 INFO:teuthology.orchestra.run.smithi001.stdout:osd.11 up in weight 1 up_from 78 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.161:6824/1586352190,v1:172.21.15.161:6825/1586352190] [v2:172.21.15.161:6826/1586352190,v1:172.21.15.161:6827/1586352190] exists,up 48f17c4a-370a-493e-956e-bf6ad3dcc594 2024-09-11T21:56:54.102 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 2.14 [2,4] 2024-09-11T21:56:54.102 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 3.6 [3,7] 2024-09-11T21:56:54.102 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 3.1a [5,6] 2024-09-11T21:56:54.102 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 3.1d [4,8] 2024-09-11T21:56:54.102 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 3.22 [3,9] 2024-09-11T21:56:54.103 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 3.25 [5,9] 2024-09-11T21:56:54.103 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 3.26 [1,9] 2024-09-11T21:56:54.103 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 3.27 [4,8] 2024-09-11T21:56:54.103 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 3.31 [4,6] 2024-09-11T21:56:54.103 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 3.3a [4,7] 2024-09-11T21:56:54.103 INFO:teuthology.orchestra.run.smithi001.stdout:blocklist 172.21.15.1:0/1749020850 expires 2024-09-12T21:48:24.524913+0000 2024-09-11T21:56:54.103 INFO:teuthology.orchestra.run.smithi001.stdout:blocklist 172.21.15.1:6801/2821761445 expires 2024-09-12T21:48:24.524913+0000 2024-09-11T21:56:54.103 INFO:teuthology.orchestra.run.smithi001.stdout:blocklist 172.21.15.1:0/1699984777 expires 2024-09-12T21:48:24.524913+0000 2024-09-11T21:56:54.103 INFO:teuthology.orchestra.run.smithi001.stdout:blocklist 172.21.15.1:0/276770268 expires 2024-09-12T21:48:00.674728+0000 2024-09-11T21:56:54.103 INFO:teuthology.orchestra.run.smithi001.stdout:blocklist 172.21.15.1:0/1278877732 expires 2024-09-12T21:48:24.524913+0000 2024-09-11T21:56:54.103 INFO:teuthology.orchestra.run.smithi001.stdout:blocklist 172.21.15.1:0/2599895977 expires 2024-09-12T21:48:00.674728+0000 2024-09-11T21:56:54.103 INFO:teuthology.orchestra.run.smithi001.stdout:blocklist 172.21.15.1:6801/2311309446 expires 2024-09-12T21:48:00.674728+0000 2024-09-11T21:56:54.104 INFO:teuthology.orchestra.run.smithi001.stdout:blocklist 172.21.15.1:6800/2821761445 expires 2024-09-12T21:48:24.524913+0000 2024-09-11T21:56:54.104 INFO:teuthology.orchestra.run.smithi001.stdout:blocklist 172.21.15.1:6800/2311309446 expires 2024-09-12T21:48:00.674728+0000 2024-09-11T21:56:54.104 INFO:teuthology.orchestra.run.smithi001.stdout:blocklist 172.21.15.1:0/3202962495 expires 2024-09-12T21:48:00.674728+0000 2024-09-11T21:56:54.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:54.101+0000 7f44ede12640 1 -- 172.21.15.1:0/1909433158 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f44d00a2cc0 msgr2=0x7f44d00a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:54.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:54.101+0000 7f44ede12640 1 --2- 172.21.15.1:0/1909433158 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f44d00a2cc0 0x7f44d00a5180 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f44d8006010 tx=0x7f44d8000f40 comp rx=0 tx=0).stop 2024-09-11T21:56:54.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:54.101+0000 7f44ede12640 1 -- 172.21.15.1:0/1909433158 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f44e81078c0 msgr2=0x7f44e819f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:54.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:54.101+0000 7f44ede12640 1 --2- 172.21.15.1:0/1909433158 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f44e81078c0 0x7f44e819f480 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f44e000c970 tx=0x7f44e000ce30 comp rx=0 tx=0).stop 2024-09-11T21:56:54.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:54.101+0000 7f44ede12640 1 -- 172.21.15.1:0/1909433158 shutdown_connections 2024-09-11T21:56:54.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:54.101+0000 7f44ede12640 1 --2- 172.21.15.1:0/1909433158 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f44d00a2cc0 0x7f44d00a5180 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:54.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:54.101+0000 7f44ede12640 1 --2- 172.21.15.1:0/1909433158 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f44e81078c0 0x7f44e819f480 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:54.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:54.101+0000 7f44ede12640 1 --2- 172.21.15.1:0/1909433158 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f44e8104f90 0x7f44e8198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:54.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:54.101+0000 7f44ede12640 1 --2- 172.21.15.1:0/1909433158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f44e8102660 0x7f44e8197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:54.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:54.101+0000 7f44ede12640 1 -- 172.21.15.1:0/1909433158 >> 172.21.15.1:0/1909433158 conn(0x7f44e80fc280 msgr2=0x7f44e8102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:54.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:54.101+0000 7f44ede12640 1 -- 172.21.15.1:0/1909433158 shutdown_connections 2024-09-11T21:56:54.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:54.101+0000 7f44ede12640 1 -- 172.21.15.1:0/1909433158 wait complete. 2024-09-11T21:56:54.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:54 smithi001 ceph-mon[30935]: pgmap v330: 97 pgs: 97 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T21:56:54.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:54 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/1909433158' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-11T21:56:54.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:54 smithi053 ceph-mon[34908]: pgmap v330: 97 pgs: 97 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T21:56:54.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:54 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/1909433158' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-11T21:56:54.826 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-11T21:56:54.837 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T21:56:54.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:54 smithi161 ceph-mon[35615]: pgmap v330: 97 pgs: 97 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T21:56:54.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:54 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/1909433158' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-11T21:56:55.095 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:56:55.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:55.567+0000 7fa86eb39640 1 -- 172.21.15.1:0/1112563864 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa868101280 msgr2=0x7fa86810df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:55.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:55.567+0000 7fa86eb39640 1 --2- 172.21.15.1:0/1112563864 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa868101280 0x7fa86810df40 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa858009900 tx=0x7fa85802a410 comp rx=0 tx=0).stop 2024-09-11T21:56:55.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:55.567+0000 7fa86eb39640 1 -- 172.21.15.1:0/1112563864 shutdown_connections 2024-09-11T21:56:55.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:55.567+0000 7fa86eb39640 1 --2- 172.21.15.1:0/1112563864 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa86810e480 0x7fa868110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:55.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:55.567+0000 7fa86eb39640 1 --2- 172.21.15.1:0/1112563864 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa868101280 0x7fa86810df40 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:55.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:55.567+0000 7fa86eb39640 1 --2- 172.21.15.1:0/1112563864 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa8681008d0 0x7fa868100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:55.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:55.567+0000 7fa86eb39640 1 -- 172.21.15.1:0/1112563864 >> 172.21.15.1:0/1112563864 conn(0x7fa8680fc7a0 msgr2=0x7fa8680febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:55.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:55.567+0000 7fa86eb39640 1 -- 172.21.15.1:0/1112563864 shutdown_connections 2024-09-11T21:56:55.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:55.567+0000 7fa86eb39640 1 -- 172.21.15.1:0/1112563864 wait complete. 2024-09-11T21:56:55.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:55.568+0000 7fa86eb39640 1 Processor -- start 2024-09-11T21:56:55.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:55.568+0000 7fa86eb39640 1 -- start start 2024-09-11T21:56:55.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:55.568+0000 7fa86eb39640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8681008d0 0x7fa86819a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:55.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:55.568+0000 7fa86eb39640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa868101280 0x7fa86819abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:55.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:55.568+0000 7fa86eb39640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa86810e480 0x7fa86819ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:55.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:55.568+0000 7fa86eb39640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa868114250 con 0x7fa868101280 2024-09-11T21:56:55.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:55.568+0000 7fa86eb39640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa8681140d0 con 0x7fa8681008d0 2024-09-11T21:56:55.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:55.568+0000 7fa86eb39640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa8681143d0 con 0x7fa86810e480 2024-09-11T21:56:55.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:55.568+0000 7fa85ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa868101280 0x7fa86819abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:55.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:55.568+0000 7fa85ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa868101280 0x7fa86819abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57970/0 (socket says 172.21.15.1:57970) 2024-09-11T21:56:55.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:55.568+0000 7fa85ffff640 1 -- 172.21.15.1:0/2977785004 learned_addr learned my addr 172.21.15.1:0/2977785004 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:56:55.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:55.568+0000 7fa86d0d6640 1 --2- 172.21.15.1:0/2977785004 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa86810e480 0x7fa86819ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:55.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:55.568+0000 7fa86c8d5640 1 --2- 172.21.15.1:0/2977785004 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8681008d0 0x7fa86819a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:55.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:55.568+0000 7fa85ffff640 1 -- 172.21.15.1:0/2977785004 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa86810e480 msgr2=0x7fa86819ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:55.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:55.568+0000 7fa85ffff640 1 --2- 172.21.15.1:0/2977785004 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa86810e480 0x7fa86819ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:55.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:55.568+0000 7fa85ffff640 1 -- 172.21.15.1:0/2977785004 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8681008d0 msgr2=0x7fa86819a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:55.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:55.568+0000 7fa85ffff640 1 --2- 172.21.15.1:0/2977785004 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8681008d0 0x7fa86819a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:55.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:55.568+0000 7fa85ffff640 1 -- 172.21.15.1:0/2977785004 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa86819f640 con 0x7fa868101280 2024-09-11T21:56:55.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:55.569+0000 7fa85ffff640 1 --2- 172.21.15.1:0/2977785004 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa868101280 0x7fa86819abd0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa858006010 tx=0x7fa858003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:55.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:55.569+0000 7fa85dffb640 1 -- 172.21.15.1:0/2977785004 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa85803f070 con 0x7fa868101280 2024-09-11T21:56:55.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:55.569+0000 7fa85dffb640 1 -- 172.21.15.1:0/2977785004 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa858031c90 con 0x7fa868101280 2024-09-11T21:56:55.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:55.569+0000 7fa86eb39640 1 -- 172.21.15.1:0/2977785004 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa86819f8d0 con 0x7fa868101280 2024-09-11T21:56:55.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:55.569+0000 7fa85dffb640 1 -- 172.21.15.1:0/2977785004 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa858035030 con 0x7fa868101280 2024-09-11T21:56:55.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:55.569+0000 7fa86eb39640 1 -- 172.21.15.1:0/2977785004 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8681a7210 con 0x7fa868101280 2024-09-11T21:56:55.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:55.570+0000 7fa86eb39640 1 -- 172.21.15.1:0/2977785004 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa86806b070 con 0x7fa868101280 2024-09-11T21:56:55.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:55.571+0000 7fa85dffb640 1 -- 172.21.15.1:0/2977785004 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fa858042070 con 0x7fa868101280 2024-09-11T21:56:55.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:55.572+0000 7fa85dffb640 1 --2- 172.21.15.1:0/2977785004 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fa8500a2b80 0x7fa8500a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:55.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:55.572+0000 7fa86c8d5640 1 --2- 172.21.15.1:0/2977785004 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fa8500a2b80 0x7fa8500a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:55.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:55.572+0000 7fa85dffb640 1 -- 172.21.15.1:0/2977785004 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fa8580efa10 con 0x7fa868101280 2024-09-11T21:56:55.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:55.572+0000 7fa86c8d5640 1 --2- 172.21.15.1:0/2977785004 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fa8500a2b80 0x7fa8500a5040 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa86400a810 tx=0x7fa864008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:55.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:55.576+0000 7fa85dffb640 1 -- 172.21.15.1:0/2977785004 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa8580bc950 con 0x7fa868101280 2024-09-11T21:56:55.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:55.786+0000 7fa86eb39640 1 -- 172.21.15.1:0/2977785004 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa8681a73e0 con 0x7fa868101280 2024-09-11T21:56:55.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:55.787+0000 7fa85dffb640 1 -- 172.21.15.1:0/2977785004 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11949 (secure 0 0 0) 0x7fa8580bc250 con 0x7fa868101280 2024-09-11T21:56:55.788 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:56:55.788 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":27,"btime":"2024-09-11T21:55:32:152241+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/421542832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":421542832},{"type":"v1","addr":"172.21.15.1:6835","nonce":421542832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2836864406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2836864406},{"type":"v1","addr":"172.21.15.1:6837","nonce":2836864406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T21:55:20.791263+0000","modified":"2024-09-11T21:55:32.152227+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24484,"mds_2":24467,"mds_3":24466,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14595":{"gid":14595,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/407834672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":407834672},{"type":"v1","addr":"172.21.15.1:6839","nonce":407834672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/1068827678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":1068827678},{"type":"v1","addr":"172.21.15.1:6841","nonce":1068827678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/872410024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":872410024},{"type":"v1","addr":"172.21.15.161:6833","nonce":872410024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/1648944570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1648944570},{"type":"v1","addr":"172.21.15.53:6833","nonce":1648944570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/2126933993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2126933993},{"type":"v1","addr":"172.21.15.53:6835","nonce":2126933993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.161:6835/828381592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":828381592},{"type":"v1","addr":"172.21.15.161:6835","nonce":828381592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.161:6837/1799318656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1799318656},{"type":"v1","addr":"172.21.15.161:6837","nonce":1799318656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.53:6837/3457131830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3457131830},{"type":"v1","addr":"172.21.15.53:6837","nonce":3457131830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/455365383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":455365383},{"type":"v1","addr":"172.21.15.161:6839","nonce":455365383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.53:6839/1378584447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1378584447},{"type":"v1","addr":"172.21.15.53:6839","nonce":1378584447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24484,24467,24466,24460]},"id":1}]} 2024-09-11T21:56:55.789 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 27 2024-09-11T21:56:55.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:55.789+0000 7fa86eb39640 1 -- 172.21.15.1:0/2977785004 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fa8500a2b80 msgr2=0x7fa8500a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:55.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:55.789+0000 7fa86eb39640 1 --2- 172.21.15.1:0/2977785004 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fa8500a2b80 0x7fa8500a5040 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa86400a810 tx=0x7fa864008040 comp rx=0 tx=0).stop 2024-09-11T21:56:55.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:55.789+0000 7fa86eb39640 1 -- 172.21.15.1:0/2977785004 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa868101280 msgr2=0x7fa86819abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:55.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:55.790+0000 7fa86eb39640 1 --2- 172.21.15.1:0/2977785004 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa868101280 0x7fa86819abd0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa858006010 tx=0x7fa858003ba0 comp rx=0 tx=0).stop 2024-09-11T21:56:55.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:55.790+0000 7fa86eb39640 1 -- 172.21.15.1:0/2977785004 shutdown_connections 2024-09-11T21:56:55.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:55.790+0000 7fa86eb39640 1 --2- 172.21.15.1:0/2977785004 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fa8500a2b80 0x7fa8500a5040 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:55.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:55.790+0000 7fa86eb39640 1 --2- 172.21.15.1:0/2977785004 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa86810e480 0x7fa86819ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:55.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:55.790+0000 7fa86eb39640 1 --2- 172.21.15.1:0/2977785004 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa868101280 0x7fa86819abd0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:55.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:55.790+0000 7fa86eb39640 1 --2- 172.21.15.1:0/2977785004 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8681008d0 0x7fa86819a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:55.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:55.790+0000 7fa86eb39640 1 -- 172.21.15.1:0/2977785004 >> 172.21.15.1:0/2977785004 conn(0x7fa8680fc7a0 msgr2=0x7fa8680feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:55.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:55.790+0000 7fa86eb39640 1 -- 172.21.15.1:0/2977785004 shutdown_connections 2024-09-11T21:56:55.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:55.790+0000 7fa86eb39640 1 -- 172.21.15.1:0/2977785004 wait complete. 2024-09-11T21:56:56.550 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T21:56:56.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:56 smithi001 ceph-mon[30935]: pgmap v331: 97 pgs: 97 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-11T21:56:56.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:56 smithi001 ceph-mon[30935]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-11T21:56:56.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:56 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/2977785004' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:56:56.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:56 smithi053 ceph-mon[34908]: pgmap v331: 97 pgs: 97 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-11T21:56:56.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:56 smithi053 ceph-mon[34908]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-11T21:56:56.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:56 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/2977785004' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:56:56.811 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:56:56.841 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:56 smithi161 ceph-mon[35615]: pgmap v331: 97 pgs: 97 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-11T21:56:56.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:56 smithi161 ceph-mon[35615]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-11T21:56:56.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:56 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/2977785004' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:56:57.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:57.312+0000 7fb3c2fba640 1 -- 172.21.15.1:0/3362598832 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3bc102790 msgr2=0x7fb3bc1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:57.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:57.312+0000 7fb3c2fba640 1 --2- 172.21.15.1:0/3362598832 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3bc102790 0x7fb3bc1067e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac009900 tx=0x7fb3ac02a410 comp rx=0 tx=0).stop 2024-09-11T21:56:57.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:57.312+0000 7fb3c2fba640 1 -- 172.21.15.1:0/3362598832 shutdown_connections 2024-09-11T21:56:57.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:57.312+0000 7fb3c2fba640 1 --2- 172.21.15.1:0/3362598832 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb3bc106f10 0x7fb3bc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:57.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:57.312+0000 7fb3c2fba640 1 --2- 172.21.15.1:0/3362598832 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3bc102790 0x7fb3bc1067e0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:57.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:57.312+0000 7fb3c2fba640 1 --2- 172.21.15.1:0/3362598832 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3bc101de0 0x7fb3bc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:57.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:57.312+0000 7fb3c2fba640 1 -- 172.21.15.1:0/3362598832 >> 172.21.15.1:0/3362598832 conn(0x7fb3bc0fd8c0 msgr2=0x7fb3bc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:57.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:57.312+0000 7fb3c2fba640 1 -- 172.21.15.1:0/3362598832 shutdown_connections 2024-09-11T21:56:57.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:57.312+0000 7fb3c2fba640 1 -- 172.21.15.1:0/3362598832 wait complete. 2024-09-11T21:56:57.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:57.313+0000 7fb3c2fba640 1 Processor -- start 2024-09-11T21:56:57.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:57.313+0000 7fb3c2fba640 1 -- start start 2024-09-11T21:56:57.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:57.313+0000 7fb3c2fba640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb3bc101de0 0x7fb3bc198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:57.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:57.313+0000 7fb3c2fba640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3bc102790 0x7fb3bc198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:57.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:57.313+0000 7fb3c2fba640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3bc106f10 0x7fb3bc19ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:57.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:57.313+0000 7fb3c2fba640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb3bc111f90 con 0x7fb3bc102790 2024-09-11T21:56:57.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:57.313+0000 7fb3c2fba640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb3bc111e10 con 0x7fb3bc106f10 2024-09-11T21:56:57.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:57.313+0000 7fb3c2fba640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb3bc112110 con 0x7fb3bc101de0 2024-09-11T21:56:57.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:57.313+0000 7fb3b3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3bc102790 0x7fb3bc198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:57.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:57.313+0000 7fb3b3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3bc102790 0x7fb3bc198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54842/0 (socket says 172.21.15.1:54842) 2024-09-11T21:56:57.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:57.313+0000 7fb3b3fff640 1 -- 172.21.15.1:0/1238593730 learned_addr learned my addr 172.21.15.1:0/1238593730 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:56:57.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:57.313+0000 7fb3c0d56640 1 --2- 172.21.15.1:0/1238593730 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb3bc101de0 0x7fb3bc198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:57.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:57.313+0000 7fb3b3fff640 1 -- 172.21.15.1:0/1238593730 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb3bc101de0 msgr2=0x7fb3bc198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:57.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:57.313+0000 7fb3c1557640 1 --2- 172.21.15.1:0/1238593730 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3bc106f10 0x7fb3bc19ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:57.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:57.313+0000 7fb3b3fff640 1 --2- 172.21.15.1:0/1238593730 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb3bc101de0 0x7fb3bc198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:57.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:57.313+0000 7fb3b3fff640 1 -- 172.21.15.1:0/1238593730 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3bc106f10 msgr2=0x7fb3bc19ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:57.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:57.313+0000 7fb3b3fff640 1 --2- 172.21.15.1:0/1238593730 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3bc106f10 0x7fb3bc19ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:57.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:57.313+0000 7fb3b3fff640 1 -- 172.21.15.1:0/1238593730 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3bc19d520 con 0x7fb3bc102790 2024-09-11T21:56:57.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:57.313+0000 7fb3c0d56640 1 --2- 172.21.15.1:0/1238593730 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb3bc101de0 0x7fb3bc198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T21:56:57.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:57.314+0000 7fb3b3fff640 1 --2- 172.21.15.1:0/1238593730 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3bc102790 0x7fb3bc198ab0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac006010 tx=0x7fb3ac0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:57.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:57.314+0000 7fb3b1ffb640 1 -- 172.21.15.1:0/1238593730 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac03f030 con 0x7fb3bc102790 2024-09-11T21:56:57.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:57.314+0000 7fb3c2fba640 1 -- 172.21.15.1:0/1238593730 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3bc19d7b0 con 0x7fb3bc102790 2024-09-11T21:56:57.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:57.314+0000 7fb3b1ffb640 1 -- 172.21.15.1:0/1238593730 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3ac003ce0 con 0x7fb3bc102790 2024-09-11T21:56:57.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:57.314+0000 7fb3b1ffb640 1 -- 172.21.15.1:0/1238593730 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac035070 con 0x7fb3bc102790 2024-09-11T21:56:57.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:57.314+0000 7fb3c2fba640 1 -- 172.21.15.1:0/1238593730 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3bc1a50f0 con 0x7fb3bc102790 2024-09-11T21:56:57.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:57.315+0000 7fb3c2fba640 1 -- 172.21.15.1:0/1238593730 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3bc1021c0 con 0x7fb3bc102790 2024-09-11T21:56:57.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:57.316+0000 7fb3b1ffb640 1 -- 172.21.15.1:0/1238593730 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fb3ac042030 con 0x7fb3bc102790 2024-09-11T21:56:57.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:57.317+0000 7fb3b1ffb640 1 --2- 172.21.15.1:0/1238593730 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fb3a40a2b80 0x7fb3a40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:57.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:57.317+0000 7fb3b1ffb640 1 -- 172.21.15.1:0/1238593730 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fb3ac0efb00 con 0x7fb3bc102790 2024-09-11T21:56:57.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:57.317+0000 7fb3c0d56640 1 --2- 172.21.15.1:0/1238593730 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fb3a40a2b80 0x7fb3a40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:57.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:57.318+0000 7fb3c0d56640 1 --2- 172.21.15.1:0/1238593730 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fb3a40a2b80 0x7fb3a40a5040 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8009b40 tx=0x7fb3b8005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:57.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:57.320+0000 7fb3b1ffb640 1 -- 172.21.15.1:0/1238593730 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb3ac0bcaa0 con 0x7fb3bc102790 2024-09-11T21:56:57.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:57.531+0000 7fb3c2fba640 1 -- 172.21.15.1:0/1238593730 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb3bc10a150 con 0x7fb3bc102790 2024-09-11T21:56:57.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:57.532+0000 7fb3b1ffb640 1 -- 172.21.15.1:0/1238593730 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11949 (secure 0 0 0) 0x7fb3ac0bc3a0 con 0x7fb3bc102790 2024-09-11T21:56:57.533 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:56:57.533 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":27,"btime":"2024-09-11T21:55:32:152241+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/421542832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":421542832},{"type":"v1","addr":"172.21.15.1:6835","nonce":421542832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2836864406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2836864406},{"type":"v1","addr":"172.21.15.1:6837","nonce":2836864406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T21:55:20.791263+0000","modified":"2024-09-11T21:55:32.152227+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24484,"mds_2":24467,"mds_3":24466,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14595":{"gid":14595,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/407834672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":407834672},{"type":"v1","addr":"172.21.15.1:6839","nonce":407834672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/1068827678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":1068827678},{"type":"v1","addr":"172.21.15.1:6841","nonce":1068827678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/872410024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":872410024},{"type":"v1","addr":"172.21.15.161:6833","nonce":872410024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/1648944570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1648944570},{"type":"v1","addr":"172.21.15.53:6833","nonce":1648944570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/2126933993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2126933993},{"type":"v1","addr":"172.21.15.53:6835","nonce":2126933993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.161:6835/828381592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":828381592},{"type":"v1","addr":"172.21.15.161:6835","nonce":828381592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.161:6837/1799318656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1799318656},{"type":"v1","addr":"172.21.15.161:6837","nonce":1799318656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.53:6837/3457131830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3457131830},{"type":"v1","addr":"172.21.15.53:6837","nonce":3457131830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/455365383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":455365383},{"type":"v1","addr":"172.21.15.161:6839","nonce":455365383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.53:6839/1378584447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1378584447},{"type":"v1","addr":"172.21.15.53:6839","nonce":1378584447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24484,24467,24466,24460]},"id":1}]} 2024-09-11T21:56:57.533 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 27 2024-09-11T21:56:57.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:57.535+0000 7fb3c2fba640 1 -- 172.21.15.1:0/1238593730 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fb3a40a2b80 msgr2=0x7fb3a40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:57.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:57.535+0000 7fb3c2fba640 1 --2- 172.21.15.1:0/1238593730 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fb3a40a2b80 0x7fb3a40a5040 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8009b40 tx=0x7fb3b8005e50 comp rx=0 tx=0).stop 2024-09-11T21:56:57.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:57.535+0000 7fb3c2fba640 1 -- 172.21.15.1:0/1238593730 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3bc102790 msgr2=0x7fb3bc198ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:57.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:57.535+0000 7fb3c2fba640 1 --2- 172.21.15.1:0/1238593730 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3bc102790 0x7fb3bc198ab0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac006010 tx=0x7fb3ac0042f0 comp rx=0 tx=0).stop 2024-09-11T21:56:57.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:57.535+0000 7fb3c2fba640 1 -- 172.21.15.1:0/1238593730 shutdown_connections 2024-09-11T21:56:57.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:57.535+0000 7fb3c2fba640 1 --2- 172.21.15.1:0/1238593730 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fb3a40a2b80 0x7fb3a40a5040 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:57.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:57.535+0000 7fb3c2fba640 1 --2- 172.21.15.1:0/1238593730 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3bc106f10 0x7fb3bc19ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:57.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:57.535+0000 7fb3c2fba640 1 --2- 172.21.15.1:0/1238593730 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3bc102790 0x7fb3bc198ab0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:57.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:57.535+0000 7fb3c2fba640 1 --2- 172.21.15.1:0/1238593730 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb3bc101de0 0x7fb3bc198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:57.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:57.535+0000 7fb3c2fba640 1 -- 172.21.15.1:0/1238593730 >> 172.21.15.1:0/1238593730 conn(0x7fb3bc0fd8c0 msgr2=0x7fb3bc0fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:57.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:57.535+0000 7fb3c2fba640 1 -- 172.21.15.1:0/1238593730 shutdown_connections 2024-09-11T21:56:57.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:57.535+0000 7fb3c2fba640 1 -- 172.21.15.1:0/1238593730 wait complete. 2024-09-11T21:56:58.182 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T21:56:58.443 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:56:58.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:58 smithi001 ceph-mon[30935]: pgmap v332: 97 pgs: 97 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-11T21:56:58.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:58 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/1238593730' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:56:58.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:58 smithi053 ceph-mon[34908]: pgmap v332: 97 pgs: 97 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-11T21:56:58.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:58 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/1238593730' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:56:58.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:58 smithi161 ceph-mon[35615]: pgmap v332: 97 pgs: 97 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-11T21:56:58.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:58 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/1238593730' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:56:58.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:58.903+0000 7f1c5679f640 1 -- 172.21.15.1:0/187174173 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c50107af0 msgr2=0x7f1c50107ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:58.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:58.903+0000 7f1c5679f640 1 --2- 172.21.15.1:0/187174173 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c50107af0 0x7f1c50107ed0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f1c40009cd0 tx=0x7f1c4002a8e0 comp rx=0 tx=0).stop 2024-09-11T21:56:58.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:58.904+0000 7f1c5679f640 1 -- 172.21.15.1:0/187174173 shutdown_connections 2024-09-11T21:56:58.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:58.904+0000 7f1c5679f640 1 --2- 172.21.15.1:0/187174173 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c50102380 0x7f1c5010e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:58.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:58.904+0000 7f1c5679f640 1 --2- 172.21.15.1:0/187174173 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c501019e0 0x7f1c50101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:58.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:58.904+0000 7f1c5679f640 1 --2- 172.21.15.1:0/187174173 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c50107af0 0x7f1c50107ed0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:58.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:58.904+0000 7f1c5679f640 1 -- 172.21.15.1:0/187174173 >> 172.21.15.1:0/187174173 conn(0x7f1c500fd700 msgr2=0x7f1c500ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:58.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:58.904+0000 7f1c5679f640 1 -- 172.21.15.1:0/187174173 shutdown_connections 2024-09-11T21:56:58.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:58.904+0000 7f1c5679f640 1 -- 172.21.15.1:0/187174173 wait complete. 2024-09-11T21:56:58.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:58.904+0000 7f1c5679f640 1 Processor -- start 2024-09-11T21:56:58.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:58.904+0000 7f1c5679f640 1 -- start start 2024-09-11T21:56:58.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:58.905+0000 7f1c5679f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c501019e0 0x7f1c501984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:58.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:58.905+0000 7f1c5679f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c50102380 0x7f1c50198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:58.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:58.905+0000 7f1c5679f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c50107af0 0x7f1c5019cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:58.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:58.905+0000 7f1c5679f640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1c50111f90 con 0x7f1c50102380 2024-09-11T21:56:58.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:58.905+0000 7f1c5679f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1c50111e10 con 0x7f1c50107af0 2024-09-11T21:56:58.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:58.905+0000 7f1c5679f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1c50112110 con 0x7f1c501019e0 2024-09-11T21:56:58.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:58.905+0000 7f1c54f9c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c50102380 0x7f1c50198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:58.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:58.905+0000 7f1c54f9c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c50102380 0x7f1c50198a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54860/0 (socket says 172.21.15.1:54860) 2024-09-11T21:56:58.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:58.905+0000 7f1c54f9c640 1 -- 172.21.15.1:0/1581503725 learned_addr learned my addr 172.21.15.1:0/1581503725 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:56:58.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:58.905+0000 7f1c55f9e640 1 --2- 172.21.15.1:0/1581503725 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c50107af0 0x7f1c5019cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:58.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:58.905+0000 7f1c5579d640 1 --2- 172.21.15.1:0/1581503725 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c501019e0 0x7f1c501984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:58.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:58.905+0000 7f1c54f9c640 1 -- 172.21.15.1:0/1581503725 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c501019e0 msgr2=0x7f1c501984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:58.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:58.905+0000 7f1c54f9c640 1 --2- 172.21.15.1:0/1581503725 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c501019e0 0x7f1c501984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:58.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:58.905+0000 7f1c54f9c640 1 -- 172.21.15.1:0/1581503725 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c50107af0 msgr2=0x7f1c5019cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:58.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:58.905+0000 7f1c54f9c640 1 --2- 172.21.15.1:0/1581503725 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c50107af0 0x7f1c5019cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:58.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:58.905+0000 7f1c54f9c640 1 -- 172.21.15.1:0/1581503725 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c5019d4a0 con 0x7f1c50102380 2024-09-11T21:56:58.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:58.905+0000 7f1c54f9c640 1 --2- 172.21.15.1:0/1581503725 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c50102380 0x7f1c50198a30 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1c4400b4a0 tx=0x7f1c4400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:58.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:58.905+0000 7f1c4e7fc640 1 -- 172.21.15.1:0/1581503725 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c44014070 con 0x7f1c50102380 2024-09-11T21:56:58.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:58.905+0000 7f1c5679f640 1 -- 172.21.15.1:0/1581503725 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c5019d790 con 0x7f1c50102380 2024-09-11T21:56:58.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:58.905+0000 7f1c4e7fc640 1 -- 172.21.15.1:0/1581503725 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c44010af0 con 0x7f1c50102380 2024-09-11T21:56:58.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:58.905+0000 7f1c4e7fc640 1 -- 172.21.15.1:0/1581503725 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c44017d10 con 0x7f1c50102380 2024-09-11T21:56:58.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:58.906+0000 7f1c5679f640 1 -- 172.21.15.1:0/1581503725 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c501a5070 con 0x7f1c50102380 2024-09-11T21:56:58.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:58.906+0000 7f1c5679f640 1 -- 172.21.15.1:0/1581503725 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c50101ec0 con 0x7f1c50102380 2024-09-11T21:56:58.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:58.908+0000 7f1c4e7fc640 1 -- 172.21.15.1:0/1581503725 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f1c44017070 con 0x7f1c50102380 2024-09-11T21:56:58.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:58.908+0000 7f1c4e7fc640 1 --2- 172.21.15.1:0/1581503725 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f1c380a2b80 0x7f1c380a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:56:58.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:58.908+0000 7f1c4e7fc640 1 -- 172.21.15.1:0/1581503725 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f1c440ce7c0 con 0x7f1c50102380 2024-09-11T21:56:58.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:58.908+0000 7f1c5579d640 1 --2- 172.21.15.1:0/1581503725 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f1c380a2b80 0x7f1c380a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:56:58.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:58.909+0000 7f1c5579d640 1 --2- 172.21.15.1:0/1581503725 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f1c380a2b80 0x7f1c380a5040 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1c40006010 tx=0x7f1c40009800 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:56:58.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:58.912+0000 7f1c4e7fc640 1 -- 172.21.15.1:0/1581503725 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1c4409b700 con 0x7f1c50102380 2024-09-11T21:56:59.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:59.087+0000 7f1c5679f640 1 -- 172.21.15.1:0/1581503725 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1c50199850 con 0x7f1c50102380 2024-09-11T21:56:59.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:59.089+0000 7f1c4e7fc640 1 -- 172.21.15.1:0/1581503725 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22669 (secure 0 0 0) 0x7f1c50199850 con 0x7f1c50102380 2024-09-11T21:56:59.090 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:56:59.090 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":86,"fsid":"3ffca83c-7087-11ef-bcea-c7b262605968","created":"2024-09-11T21:47:28.868707+0000","modified":"2024-09-11T21:56:26.278614+0000","last_up_change":"2024-09-11T21:54:29.319441+0000","last_in_change":"2024-09-11T21:54:05.532281+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T21:50:41.026490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T21:55:16.961999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T21:55:19.066228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"19e56504-18f1-4234-b2d6-f193ff47d8eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":4012142023},{"type":"v1","addr":"172.21.15.1:6803","nonce":4012142023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":4012142023},{"type":"v1","addr":"172.21.15.1:6805","nonce":4012142023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":4012142023},{"type":"v1","addr":"172.21.15.1:6809","nonce":4012142023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":4012142023},{"type":"v1","addr":"172.21.15.1:6807","nonce":4012142023}]},"public_addr":"172.21.15.1:6803/4012142023","cluster_addr":"172.21.15.1:6805/4012142023","heartbeat_back_addr":"172.21.15.1:6809/4012142023","heartbeat_front_addr":"172.21.15.1:6807/4012142023","state":["exists","up"]},{"osd":1,"uuid":"996c9f0c-c5a4-43b8-b8e0-d712f27c5dfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3039391924},{"type":"v1","addr":"172.21.15.53:6801","nonce":3039391924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3039391924},{"type":"v1","addr":"172.21.15.53:6803","nonce":3039391924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3039391924},{"type":"v1","addr":"172.21.15.53:6807","nonce":3039391924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3039391924},{"type":"v1","addr":"172.21.15.53:6805","nonce":3039391924}]},"public_addr":"172.21.15.53:6801/3039391924","cluster_addr":"172.21.15.53:6803/3039391924","heartbeat_back_addr":"172.21.15.53:6807/3039391924","heartbeat_front_addr":"172.21.15.53:6805/3039391924","state":["exists","up"]},{"osd":2,"uuid":"578d4260-41e1-4645-b9b1-1c5445712c13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1707914885},{"type":"v1","addr":"172.21.15.161:6801","nonce":1707914885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1707914885},{"type":"v1","addr":"172.21.15.161:6803","nonce":1707914885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1707914885},{"type":"v1","addr":"172.21.15.161:6807","nonce":1707914885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1707914885},{"type":"v1","addr":"172.21.15.161:6805","nonce":1707914885}]},"public_addr":"172.21.15.161:6801/1707914885","cluster_addr":"172.21.15.161:6803/1707914885","heartbeat_back_addr":"172.21.15.161:6807/1707914885","heartbeat_front_addr":"172.21.15.161:6805/1707914885","state":["exists","up"]},{"osd":3,"uuid":"73f78766-9662-4aa3-841c-fe2ce654ee9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1541142398},{"type":"v1","addr":"172.21.15.1:6811","nonce":1541142398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1541142398},{"type":"v1","addr":"172.21.15.1:6813","nonce":1541142398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1541142398},{"type":"v1","addr":"172.21.15.1:6817","nonce":1541142398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1541142398},{"type":"v1","addr":"172.21.15.1:6815","nonce":1541142398}]},"public_addr":"172.21.15.1:6811/1541142398","cluster_addr":"172.21.15.1:6813/1541142398","heartbeat_back_addr":"172.21.15.1:6817/1541142398","heartbeat_front_addr":"172.21.15.1:6815/1541142398","state":["exists","up"]},{"osd":4,"uuid":"56c808bb-d4cd-4969-b49e-adef49ecd0f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":3081402681},{"type":"v1","addr":"172.21.15.53:6809","nonce":3081402681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3081402681},{"type":"v1","addr":"172.21.15.53:6811","nonce":3081402681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3081402681},{"type":"v1","addr":"172.21.15.53:6815","nonce":3081402681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3081402681},{"type":"v1","addr":"172.21.15.53:6813","nonce":3081402681}]},"public_addr":"172.21.15.53:6809/3081402681","cluster_addr":"172.21.15.53:6811/3081402681","heartbeat_back_addr":"172.21.15.53:6815/3081402681","heartbeat_front_addr":"172.21.15.53:6813/3081402681","state":["exists","up"]},{"osd":5,"uuid":"7b8266d6-b507-47c9-b4fa-ff16c4a74d35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":1622081327},{"type":"v1","addr":"172.21.15.161:6809","nonce":1622081327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":1622081327},{"type":"v1","addr":"172.21.15.161:6811","nonce":1622081327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":1622081327},{"type":"v1","addr":"172.21.15.161:6815","nonce":1622081327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":1622081327},{"type":"v1","addr":"172.21.15.161:6813","nonce":1622081327}]},"public_addr":"172.21.15.161:6809/1622081327","cluster_addr":"172.21.15.161:6811/1622081327","heartbeat_back_addr":"172.21.15.161:6815/1622081327","heartbeat_front_addr":"172.21.15.161:6813/1622081327","state":["exists","up"]},{"osd":6,"uuid":"3af5925d-a749-4849-a076-80580222cad4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":158434853},{"type":"v1","addr":"172.21.15.1:6819","nonce":158434853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":158434853},{"type":"v1","addr":"172.21.15.1:6821","nonce":158434853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":158434853},{"type":"v1","addr":"172.21.15.1:6825","nonce":158434853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":158434853},{"type":"v1","addr":"172.21.15.1:6823","nonce":158434853}]},"public_addr":"172.21.15.1:6819/158434853","cluster_addr":"172.21.15.1:6821/158434853","heartbeat_back_addr":"172.21.15.1:6825/158434853","heartbeat_front_addr":"172.21.15.1:6823/158434853","state":["exists","up"]},{"osd":7,"uuid":"f8f26c3c-17e9-4e89-8392-289796e0c8a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2132057697},{"type":"v1","addr":"172.21.15.53:6817","nonce":2132057697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2132057697},{"type":"v1","addr":"172.21.15.53:6819","nonce":2132057697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2132057697},{"type":"v1","addr":"172.21.15.53:6823","nonce":2132057697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2132057697},{"type":"v1","addr":"172.21.15.53:6821","nonce":2132057697}]},"public_addr":"172.21.15.53:6817/2132057697","cluster_addr":"172.21.15.53:6819/2132057697","heartbeat_back_addr":"172.21.15.53:6823/2132057697","heartbeat_front_addr":"172.21.15.53:6821/2132057697","state":["exists","up"]},{"osd":8,"uuid":"36ea401b-3c93-4b9d-8880-298d1eb6e30b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1838248669},{"type":"v1","addr":"172.21.15.161:6817","nonce":1838248669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1838248669},{"type":"v1","addr":"172.21.15.161:6819","nonce":1838248669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1838248669},{"type":"v1","addr":"172.21.15.161:6823","nonce":1838248669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1838248669},{"type":"v1","addr":"172.21.15.161:6821","nonce":1838248669}]},"public_addr":"172.21.15.161:6817/1838248669","cluster_addr":"172.21.15.161:6819/1838248669","heartbeat_back_addr":"172.21.15.161:6823/1838248669","heartbeat_front_addr":"172.21.15.161:6821/1838248669","state":["exists","up"]},{"osd":9,"uuid":"4b788d3f-a3bc-4463-bd99-95fdb7a52768","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2806274589},{"type":"v1","addr":"172.21.15.1:6827","nonce":2806274589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2806274589},{"type":"v1","addr":"172.21.15.1:6829","nonce":2806274589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":2806274589},{"type":"v1","addr":"172.21.15.1:6833","nonce":2806274589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2806274589},{"type":"v1","addr":"172.21.15.1:6831","nonce":2806274589}]},"public_addr":"172.21.15.1:6827/2806274589","cluster_addr":"172.21.15.1:6829/2806274589","heartbeat_back_addr":"172.21.15.1:6833/2806274589","heartbeat_front_addr":"172.21.15.1:6831/2806274589","state":["exists","up"]},{"osd":10,"uuid":"70549796-5270-4ae2-ac3f-f61f5de105d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":761923580},{"type":"v1","addr":"172.21.15.53:6825","nonce":761923580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":761923580},{"type":"v1","addr":"172.21.15.53:6827","nonce":761923580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":761923580},{"type":"v1","addr":"172.21.15.53:6831","nonce":761923580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":761923580},{"type":"v1","addr":"172.21.15.53:6829","nonce":761923580}]},"public_addr":"172.21.15.53:6825/761923580","cluster_addr":"172.21.15.53:6827/761923580","heartbeat_back_addr":"172.21.15.53:6831/761923580","heartbeat_front_addr":"172.21.15.53:6829/761923580","state":["exists","up"]},{"osd":11,"uuid":"48f17c4a-370a-493e-956e-bf6ad3dcc594","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1586352190},{"type":"v1","addr":"172.21.15.161:6825","nonce":1586352190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":1586352190},{"type":"v1","addr":"172.21.15.161:6827","nonce":1586352190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":1586352190},{"type":"v1","addr":"172.21.15.161:6831","nonce":1586352190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":1586352190},{"type":"v1","addr":"172.21.15.161:6829","nonce":1586352190}]},"public_addr":"172.21.15.161:6825/1586352190","cluster_addr":"172.21.15.161:6827/1586352190","heartbeat_back_addr":"172.21.15.161:6831/1586352190","heartbeat_front_addr":"172.21.15.161:6829/1586352190","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:49:46.539613+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:50:11.912300+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:50:36.955490+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:51:02.200554+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:51:27.860585+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:51:54.694099+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:52:18.962351+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:52:44.881923+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:53:10.301390+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:53:35.417647+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:54:01.492219+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:54:26.938338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.26","mappings":[{"from":1,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/1749020850":"2024-09-12T21:48:24.524913+0000","172.21.15.1:6801/2821761445":"2024-09-12T21:48:24.524913+0000","172.21.15.1:0/1699984777":"2024-09-12T21:48:24.524913+0000","172.21.15.1:0/276770268":"2024-09-12T21:48:00.674728+0000","172.21.15.1:0/1278877732":"2024-09-12T21:48:24.524913+0000","172.21.15.1:0/2599895977":"2024-09-12T21:48:00.674728+0000","172.21.15.1:6801/2311309446":"2024-09-12T21:48:00.674728+0000","172.21.15.1:6800/2821761445":"2024-09-12T21:48:24.524913+0000","172.21.15.1:6800/2311309446":"2024-09-12T21:48:00.674728+0000","172.21.15.1:0/3202962495":"2024-09-12T21:48:00.674728+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T21:56:59.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:59.091+0000 7f1c5679f640 1 -- 172.21.15.1:0/1581503725 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f1c380a2b80 msgr2=0x7f1c380a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:59.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:59.091+0000 7f1c5679f640 1 --2- 172.21.15.1:0/1581503725 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f1c380a2b80 0x7f1c380a5040 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1c40006010 tx=0x7f1c40009800 comp rx=0 tx=0).stop 2024-09-11T21:56:59.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:59.091+0000 7f1c5679f640 1 -- 172.21.15.1:0/1581503725 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c50102380 msgr2=0x7f1c50198a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:56:59.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:59.091+0000 7f1c5679f640 1 --2- 172.21.15.1:0/1581503725 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c50102380 0x7f1c50198a30 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1c4400b4a0 tx=0x7f1c4400b960 comp rx=0 tx=0).stop 2024-09-11T21:56:59.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:59.091+0000 7f1c5679f640 1 -- 172.21.15.1:0/1581503725 shutdown_connections 2024-09-11T21:56:59.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:59.091+0000 7f1c5679f640 1 --2- 172.21.15.1:0/1581503725 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f1c380a2b80 0x7f1c380a5040 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:59.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:59.091+0000 7f1c5679f640 1 --2- 172.21.15.1:0/1581503725 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c50107af0 0x7f1c5019cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:59.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:59.091+0000 7f1c5679f640 1 --2- 172.21.15.1:0/1581503725 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c50102380 0x7f1c50198a30 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:59.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:59.091+0000 7f1c5679f640 1 --2- 172.21.15.1:0/1581503725 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c501019e0 0x7f1c501984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:56:59.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:59.091+0000 7f1c5679f640 1 -- 172.21.15.1:0/1581503725 >> 172.21.15.1:0/1581503725 conn(0x7f1c500fd700 msgr2=0x7f1c500fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:56:59.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:59.091+0000 7f1c5679f640 1 -- 172.21.15.1:0/1581503725 shutdown_connections 2024-09-11T21:56:59.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:56:59.091+0000 7f1c5679f640 1 -- 172.21.15.1:0/1581503725 wait complete. 2024-09-11T21:56:59.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:56:59 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/1581503725' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T21:56:59.773 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-11T21:55:20.791263+0000', 'modified': '2024-09-11T21:55:32.152227+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24494, 'mds_1': 24484, 'mds_2': 24467, 'mds_3': 24466, 'mds_4': 24460}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14595': {'gid': 14595, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.1:6839/407834672', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6838', 'nonce': 407834672}, {'type': 'v1', 'addr': '172.21.15.1:6839', 'nonce': 407834672}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14601': {'gid': 14601, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.1:6841/1068827678', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6840', 'nonce': 1068827678}, {'type': 'v1', 'addr': '172.21.15.1:6841', 'nonce': 1068827678}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24430': {'gid': 24430, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.161:6833/872410024', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6832', 'nonce': 872410024}, {'type': 'v1', 'addr': '172.21.15.161:6833', 'nonce': 872410024}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24431': {'gid': 24431, 'name': 'k', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.53:6833/1648944570', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6832', 'nonce': 1648944570}, {'type': 'v1', 'addr': '172.21.15.53:6833', 'nonce': 1648944570}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24449': {'gid': 24449, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.53:6835/2126933993', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6834', 'nonce': 2126933993}, {'type': 'v1', 'addr': '172.21.15.53:6835', 'nonce': 2126933993}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24460': {'gid': 24460, 'name': 'f', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.161:6835/828381592', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6834', 'nonce': 828381592}, {'type': 'v1', 'addr': '172.21.15.161:6835', 'nonce': 828381592}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24466': {'gid': 24466, 'name': 'c', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.161:6837/1799318656', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6836', 'nonce': 1799318656}, {'type': 'v1', 'addr': '172.21.15.161:6837', 'nonce': 1799318656}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24467': {'gid': 24467, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.53:6837/3457131830', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6836', 'nonce': 3457131830}, {'type': 'v1', 'addr': '172.21.15.53:6837', 'nonce': 3457131830}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24484': {'gid': 24484, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.161:6839/455365383', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6838', 'nonce': 455365383}, {'type': 'v1', 'addr': '172.21.15.161:6839', 'nonce': 455365383}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24494': {'gid': 24494, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.53:6839/1378584447', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6838', 'nonce': 1378584447}, {'type': 'v1', 'addr': '172.21.15.53:6839', 'nonce': 1378584447}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24494, 'qdb_cluster': [24494, 24484, 24467, 24466, 24460]} 2024-09-11T21:56:59.773 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-11T21:56:59.773 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.f status 2024-09-11T21:56:59.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:56:59 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/1581503725' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T21:56:59.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:56:59 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/1581503725' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T21:57:00.038 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:57:00.423 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:00 smithi001 ceph-mon[30935]: pgmap v333: 97 pgs: 97 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 9 op/s 2024-09-11T21:57:00.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.505+0000 7f72c9495640 1 -- 172.21.15.1:0/1465017155 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f72c4074e30 msgr2=0x7f72c410df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:00.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.505+0000 7f72c9495640 1 --2- 172.21.15.1:0/1465017155 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f72c4074e30 0x7f72c410df00 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f72b8009900 tx=0x7f72b802a480 comp rx=0 tx=0).stop 2024-09-11T21:57:00.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.505+0000 7f72c9495640 1 -- 172.21.15.1:0/1465017155 shutdown_connections 2024-09-11T21:57:00.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.505+0000 7f72c9495640 1 --2- 172.21.15.1:0/1465017155 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72c410e440 0x7f72c4110830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:00.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.505+0000 7f72c9495640 1 --2- 172.21.15.1:0/1465017155 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f72c4074e30 0x7f72c410df00 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:00.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.505+0000 7f72c9495640 1 --2- 172.21.15.1:0/1465017155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72c4074510 0x7f72c40748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:00.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.505+0000 7f72c9495640 1 -- 172.21.15.1:0/1465017155 >> 172.21.15.1:0/1465017155 conn(0x7f72c40fd8c0 msgr2=0x7f72c40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:00.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.505+0000 7f72c9495640 1 -- 172.21.15.1:0/1465017155 shutdown_connections 2024-09-11T21:57:00.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.505+0000 7f72c9495640 1 -- 172.21.15.1:0/1465017155 wait complete. 2024-09-11T21:57:00.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.505+0000 7f72c9495640 1 Processor -- start 2024-09-11T21:57:00.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.506+0000 7f72c9495640 1 -- start start 2024-09-11T21:57:00.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.506+0000 7f72c9495640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72c4074510 0x7f72c419a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:00.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.506+0000 7f72c9495640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f72c4074e30 0x7f72c419abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:00.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.506+0000 7f72c9495640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72c410e440 0x7f72c419ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:00.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.506+0000 7f72c9495640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f72c4114140 con 0x7f72c4074e30 2024-09-11T21:57:00.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.506+0000 7f72c9495640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f72c4113fc0 con 0x7f72c4074510 2024-09-11T21:57:00.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.506+0000 7f72c9495640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f72c41142c0 con 0x7f72c410e440 2024-09-11T21:57:00.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.506+0000 7f72c27fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f72c4074e30 0x7f72c419abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:00.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.506+0000 7f72c27fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f72c4074e30 0x7f72c419abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54878/0 (socket says 172.21.15.1:54878) 2024-09-11T21:57:00.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.506+0000 7f72c27fc640 1 -- 172.21.15.1:0/4109887041 learned_addr learned my addr 172.21.15.1:0/4109887041 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:00.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.506+0000 7f72c2ffd640 1 --2- 172.21.15.1:0/4109887041 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72c4074510 0x7f72c419a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:00.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.506+0000 7f72c37fe640 1 --2- 172.21.15.1:0/4109887041 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72c410e440 0x7f72c419ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:00.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.506+0000 7f72c27fc640 1 -- 172.21.15.1:0/4109887041 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72c410e440 msgr2=0x7f72c419ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:00.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.506+0000 7f72c27fc640 1 --2- 172.21.15.1:0/4109887041 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72c410e440 0x7f72c419ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:00.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.506+0000 7f72c27fc640 1 -- 172.21.15.1:0/4109887041 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72c4074510 msgr2=0x7f72c419a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:00.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.506+0000 7f72c27fc640 1 --2- 172.21.15.1:0/4109887041 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72c4074510 0x7f72c419a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:00.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.506+0000 7f72c27fc640 1 -- 172.21.15.1:0/4109887041 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72c419f650 con 0x7f72c4074e30 2024-09-11T21:57:00.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.507+0000 7f72c27fc640 1 --2- 172.21.15.1:0/4109887041 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f72c4074e30 0x7f72c419abe0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f72b8006010 tx=0x7f72b80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:00.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.507+0000 7f72abfff640 1 -- 172.21.15.1:0/4109887041 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b803f030 con 0x7f72c4074e30 2024-09-11T21:57:00.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.507+0000 7f72abfff640 1 -- 172.21.15.1:0/4109887041 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72b8003ce0 con 0x7f72c4074e30 2024-09-11T21:57:00.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.507+0000 7f72c9495640 1 -- 172.21.15.1:0/4109887041 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72c419f8e0 con 0x7f72c4074e30 2024-09-11T21:57:00.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.507+0000 7f72abfff640 1 -- 172.21.15.1:0/4109887041 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b8035070 con 0x7f72c4074e30 2024-09-11T21:57:00.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.507+0000 7f72c9495640 1 -- 172.21.15.1:0/4109887041 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72c41a7220 con 0x7f72c4074e30 2024-09-11T21:57:00.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.509+0000 7f72abfff640 1 -- 172.21.15.1:0/4109887041 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f72b8042030 con 0x7f72c4074e30 2024-09-11T21:57:00.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.509+0000 7f72cb86e740 1 Processor -- start 2024-09-11T21:57:00.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.509+0000 7f72cb86e740 1 -- start start 2024-09-11T21:57:00.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.510+0000 7f72cb86e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c562c29d0 0x559c562c2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:00.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.510+0000 7f72cb86e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559c562c32f0 0x559c562cd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:00.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.510+0000 7f72cb86e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559c562cdbb0 0x559c562cffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:00.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.510+0000 7f72cb86e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x559c56229090 con 0x559c562c32f0 2024-09-11T21:57:00.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.510+0000 7f72cb86e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x559c561d0a70 con 0x559c562cdbb0 2024-09-11T21:57:00.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.510+0000 7f72cb86e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x559c56225050 con 0x559c562c29d0 2024-09-11T21:57:00.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.510+0000 7f72c2ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559c562c32f0 0x559c562cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:00.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.510+0000 7f72c2ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559c562c32f0 0x559c562cd670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54890/0 (socket says 172.21.15.1:54890) 2024-09-11T21:57:00.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.510+0000 7f72c2ffd640 1 -- 172.21.15.1:0/3987633610 learned_addr learned my addr 172.21.15.1:0/3987633610 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:00.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.510+0000 7f72c27fc640 1 --2- 172.21.15.1:0/3987633610 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559c562cdbb0 0x559c562cffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:00.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.510+0000 7f72abfff640 1 --2- 172.21.15.1:0/4109887041 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f72ac0a2c00 0x7f72ac0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:00.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.510+0000 7f72c37fe640 1 --2- 172.21.15.1:0/3987633610 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c562c29d0 0x559c562c2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:00.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.510+0000 7f72c2ffd640 1 -- 172.21.15.1:0/3987633610 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c562c29d0 msgr2=0x559c562c2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:00.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.510+0000 7f72c2ffd640 1 --2- 172.21.15.1:0/3987633610 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c562c29d0 0x559c562c2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:00.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.510+0000 7f72c2ffd640 1 -- 172.21.15.1:0/3987633610 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559c562cdbb0 msgr2=0x559c562cffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:00.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.510+0000 7f72c2ffd640 1 --2- 172.21.15.1:0/3987633610 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559c562cdbb0 0x559c562cffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:00.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.510+0000 7f72c2ffd640 1 -- 172.21.15.1:0/3987633610 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c561dd130 con 0x559c562c32f0 2024-09-11T21:57:00.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.510+0000 7f72c37fe640 1 --2- 172.21.15.1:0/3987633610 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c562c29d0 0x559c562c2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T21:57:00.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.510+0000 7f72c37fe640 1 --2- 172.21.15.1:0/4109887041 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f72ac0a2c00 0x7f72ac0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:00.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.510+0000 7f72abfff640 1 -- 172.21.15.1:0/4109887041 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f72b80ef420 con 0x7f72c4074e30 2024-09-11T21:57:00.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.510+0000 7f72c2ffd640 1 --2- 172.21.15.1:0/3987633610 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559c562c32f0 0x559c562cd670 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f72b400ad00 tx=0x7f72b400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67142cb99eb98c19 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:00.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.511+0000 7f72a97fa640 1 -- 172.21.15.1:0/3987633610 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b401d070 con 0x559c562c32f0 2024-09-11T21:57:00.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.511+0000 7f72a97fa640 1 -- 172.21.15.1:0/3987633610 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72b4018b20 con 0x559c562c32f0 2024-09-11T21:57:00.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.511+0000 7f72a97fa640 1 -- 172.21.15.1:0/3987633610 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b4010bc0 con 0x559c562c32f0 2024-09-11T21:57:00.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.511+0000 7f72cb86e740 1 -- 172.21.15.1:0/3987633610 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559c562c32f0 msgr2=0x559c562cd670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:00.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.511+0000 7f72c37fe640 1 --2- 172.21.15.1:0/4109887041 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f72ac0a2c00 0x7f72ac0a50c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f72c40754f0 tx=0x7f72bc004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:00.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.511+0000 7f72cb86e740 1 --2- 172.21.15.1:0/3987633610 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559c562c32f0 0x559c562cd670 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f72b400ad00 tx=0x7f72b400f790 comp rx=0 tx=0).stop 2024-09-11T21:57:00.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.511+0000 7f72cb86e740 1 -- 172.21.15.1:0/3987633610 shutdown_connections 2024-09-11T21:57:00.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.511+0000 7f72cb86e740 1 --2- 172.21.15.1:0/3987633610 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559c562cdbb0 0x559c562cffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:00.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.511+0000 7f72cb86e740 1 --2- 172.21.15.1:0/3987633610 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559c562c32f0 0x559c562cd670 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:00.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.511+0000 7f72cb86e740 1 --2- 172.21.15.1:0/3987633610 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c562c29d0 0x559c562c2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:00.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.511+0000 7f72cb86e740 1 -- 172.21.15.1:0/3987633610 >> 172.21.15.1:0/3987633610 conn(0x559c562bef60 msgr2=0x559c562bf340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:00.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.511+0000 7f72cb86e740 1 -- 172.21.15.1:0/3987633610 shutdown_connections 2024-09-11T21:57:00.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.512+0000 7f72cb86e740 1 -- 172.21.15.1:0/3987633610 wait complete. 2024-09-11T21:57:00.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.512+0000 7f72cb86e740 1 -- start start 2024-09-11T21:57:00.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.512+0000 7f72cb86e740 1 Processor -- start 2024-09-11T21:57:00.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.512+0000 7f72cb86e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559c562c29d0 0x559c562cd440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:00.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.512+0000 7f72cb86e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c562c32f0 0x559c562ca790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:00.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.512+0000 7f72cb86e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559c562cdbb0 0x559c562cacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:00.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.512+0000 7f72cb86e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x559c562d0780 con 0x559c562cdbb0 2024-09-11T21:57:00.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.512+0000 7f72cb86e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x559c561e3f20 con 0x559c562c29d0 2024-09-11T21:57:00.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.512+0000 7f72cb86e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x559c562d0900 con 0x559c562c32f0 2024-09-11T21:57:00.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.512+0000 7f72cb86e740 10 client.0 _ll_register_callbacks cb 0x559c561ad030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:57:00.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.512+0000 7f72c37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559c562cdbb0 0x559c562cacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:00.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.513+0000 7f72c37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559c562cdbb0 0x559c562cacd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54892/0 (socket says 172.21.15.1:54892) 2024-09-11T21:57:00.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.513+0000 7f72c37fe640 1 -- 172.21.15.1:0/742414392 learned_addr learned my addr 172.21.15.1:0/742414392 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:00.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.513+0000 7f72c27fc640 1 --2- 172.21.15.1:0/742414392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c562c32f0 0x559c562ca790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:00.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.513+0000 7f72c2ffd640 1 --2- 172.21.15.1:0/742414392 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559c562c29d0 0x559c562cd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:00.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.513+0000 7f72c37fe640 1 -- 172.21.15.1:0/742414392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c562c32f0 msgr2=0x559c562ca790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:00.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.513+0000 7f72c37fe640 1 --2- 172.21.15.1:0/742414392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c562c32f0 0x559c562ca790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:00.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.513+0000 7f72c37fe640 1 -- 172.21.15.1:0/742414392 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559c562c29d0 msgr2=0x559c562cd440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:00.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.513+0000 7f72c37fe640 1 --2- 172.21.15.1:0/742414392 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559c562c29d0 0x559c562cd440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:00.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.513+0000 7f72c37fe640 1 -- 172.21.15.1:0/742414392 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c55f9a2b0 con 0x559c562cdbb0 2024-09-11T21:57:00.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.513+0000 7f72c2ffd640 1 --2- 172.21.15.1:0/742414392 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559c562c29d0 0x559c562cd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T21:57:00.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.513+0000 7f72c37fe640 1 --2- 172.21.15.1:0/742414392 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559c562cdbb0 0x559c562cacd0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f72bc011c00 tx=0x7f72bc009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e29992db968e8085 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:00.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.513+0000 7f72a6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T21:57:00.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.513+0000 7f72a6ff5640 1 -- 172.21.15.1:0/742414392 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72bc022070 con 0x559c562cdbb0 2024-09-11T21:57:00.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.513+0000 7f72a6ff5640 1 -- 172.21.15.1:0/742414392 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f72bc00ed50 con 0x559c562cdbb0 2024-09-11T21:57:00.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.513+0000 7f72cb86e740 1 -- 172.21.15.1:0/742414392 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72b4018b20 con 0x559c562cdbb0 2024-09-11T21:57:00.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.513+0000 7f72a6ff5640 1 -- 172.21.15.1:0/742414392 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f72bc02e050 con 0x559c562cdbb0 2024-09-11T21:57:00.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.513+0000 7f72cb86e740 10 client.15108 fetch_fsmap learned FSMap version 27 2024-09-11T21:57:00.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.513+0000 7f72cb86e740 1 -- 172.21.15.1:0/742414392 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559c562dc060 con 0x559c562cdbb0 2024-09-11T21:57:00.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.514+0000 7f72a6ff5640 1 -- 172.21.15.1:0/742414392 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72b4018b20 con 0x559c562cdbb0 2024-09-11T21:57:00.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.514+0000 7f72a6ff5640 1 -- 172.21.15.1:0/742414392 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f72bc026590 con 0x559c562cdbb0 2024-09-11T21:57:00.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.514+0000 7f72cb86e740 10 client.15108 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T21:57:00.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.514+0000 7f72cb86e740 10 client.15108 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-11T21:57:00.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.514+0000 7f72cb86e740 20 client.15108 populate_metadata read hostname 'smithi001' 2024-09-11T21:57:00.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.514+0000 7f72cb86e740 1 --2- 172.21.15.1:0/742414392 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x559c562de6c0 0x559c562dfb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:00.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.514+0000 7f72cb86e740 4 client.15108 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T21:57:00.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.514+0000 7f72cb86e740 1 -- 172.21.15.1:0/742414392 --> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c56236600 con 0x559c562de6c0 2024-09-11T21:57:00.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.514+0000 7f72c2ffd640 1 --2- 172.21.15.1:0/742414392 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x559c562de6c0 0x559c562dfb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:00.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.515+0000 7f72c2ffd640 1 --2- 172.21.15.1:0/742414392 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x559c562de6c0 0x559c562dfb30 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=766f46efad79131b server_cookie=5a10b1519391590a in_seq=0 out_seq=0 2024-09-11T21:57:00.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.515+0000 7f72a6ff5640 10 client.15108 ms_handle_connect on v2:172.21.15.161:6834/828381592 2024-09-11T21:57:00.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.515+0000 7f72a6ff5640 1 -- 172.21.15.1:0/742414392 <== mds.4 v2:172.21.15.161:6834/828381592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559c56236600 con 0x559c562de6c0 2024-09-11T21:57:00.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.515+0000 7f72a6ff5640 10 client.15108 handle_command_reply: tid=0 2024-09-11T21:57:00.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.515+0000 7f72cb86e740 1 client.15108 shutdown 2024-09-11T21:57:00.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.516+0000 7f72cb86e740 1 -- 172.21.15.1:0/742414392 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559c562cdbb0 msgr2=0x559c562cacd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:00.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.516+0000 7f72cb86e740 1 --2- 172.21.15.1:0/742414392 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559c562cdbb0 0x559c562cacd0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f72bc011c00 tx=0x7f72bc009240 comp rx=0 tx=0).stop 2024-09-11T21:57:00.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.516+0000 7f72cb86e740 1 -- 172.21.15.1:0/742414392 shutdown_connections 2024-09-11T21:57:00.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.516+0000 7f72cb86e740 1 --2- 172.21.15.1:0/742414392 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559c562cdbb0 0x559c562cacd0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:00.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.516+0000 7f72cb86e740 1 --2- 172.21.15.1:0/742414392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c562c32f0 0x559c562ca790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:00.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.516+0000 7f72cb86e740 1 --2- 172.21.15.1:0/742414392 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x559c562de6c0 0x559c562dfb30 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:00.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.516+0000 7f72cb86e740 1 --2- 172.21.15.1:0/742414392 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559c562c29d0 0x559c562cd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:00.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.516+0000 7f72cb86e740 1 -- 172.21.15.1:0/742414392 >> 172.21.15.1:0/742414392 conn(0x559c562bef60 msgr2=0x559c562c4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:00.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.516+0000 7f72a6ff5640 0 client.15108 ms_handle_reset on v2:172.21.15.161:6834/828381592 2024-09-11T21:57:00.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.516+0000 7f72cb86e740 1 -- 172.21.15.1:0/742414392 shutdown_connections 2024-09-11T21:57:00.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.516+0000 7f72cb86e740 1 -- 172.21.15.1:0/742414392 wait complete. 2024-09-11T21:57:00.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.516+0000 7f72cb86e740 20 client.15108 trim_cache size 0 max 16384 2024-09-11T21:57:00.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.534+0000 7f72cb86e740 1 Processor -- start 2024-09-11T21:57:00.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.534+0000 7f72cb86e740 1 -- start start 2024-09-11T21:57:00.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.534+0000 7f72cb86e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c562c29d0 0x559c562ded90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:00.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.534+0000 7f72cb86e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559c562c32f0 0x559c562ddff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:00.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.534+0000 7f72cb86e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559c562cdbb0 0x559c562e7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:00.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.534+0000 7f72cb86e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x559c562dde70 con 0x559c562c32f0 2024-09-11T21:57:00.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.534+0000 7f72cb86e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x559c562de540 con 0x559c562cdbb0 2024-09-11T21:57:00.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.534+0000 7f72cb86e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x559c562dec10 con 0x559c562c29d0 2024-09-11T21:57:00.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.534+0000 7f72c27fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559c562c32f0 0x559c562ddff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:00.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.534+0000 7f72c27fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559c562c32f0 0x559c562ddff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54896/0 (socket says 172.21.15.1:54896) 2024-09-11T21:57:00.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.534+0000 7f72c27fc640 1 -- 172.21.15.1:0/721034313 learned_addr learned my addr 172.21.15.1:0/721034313 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:00.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.534+0000 7f72c27fc640 1 -- 172.21.15.1:0/721034313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c562c29d0 msgr2=0x559c562ded90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:00.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.534+0000 7f72c2ffd640 1 --2- 172.21.15.1:0/721034313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c562c29d0 0x559c562ded90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:00.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.534+0000 7f72c37fe640 1 --2- 172.21.15.1:0/721034313 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559c562cdbb0 0x559c562e7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:00.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.534+0000 7f72c27fc640 1 --2- 172.21.15.1:0/721034313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c562c29d0 0x559c562ded90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:00.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.534+0000 7f72c27fc640 1 -- 172.21.15.1:0/721034313 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559c562cdbb0 msgr2=0x559c562e7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:00.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.534+0000 7f72c27fc640 1 --2- 172.21.15.1:0/721034313 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559c562cdbb0 0x559c562e7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:00.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.534+0000 7f72c27fc640 1 -- 172.21.15.1:0/721034313 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c562cd060 con 0x559c562c32f0 2024-09-11T21:57:00.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.535+0000 7f72c37fe640 1 --2- 172.21.15.1:0/721034313 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559c562cdbb0 0x559c562e7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T21:57:00.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.535+0000 7f72c27fc640 1 --2- 172.21.15.1:0/721034313 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559c562c32f0 0x559c562ddff0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f72b8000c00 tx=0x7f72b8005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caeeb9c75215e92b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:00.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.535+0000 7f72a8ff9640 1 -- 172.21.15.1:0/721034313 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b8049040 con 0x559c562c32f0 2024-09-11T21:57:00.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.535+0000 7f72a8ff9640 1 -- 172.21.15.1:0/721034313 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72b803b510 con 0x559c562c32f0 2024-09-11T21:57:00.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.535+0000 7f72a8ff9640 1 -- 172.21.15.1:0/721034313 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b80c48d0 con 0x559c562c32f0 2024-09-11T21:57:00.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.535+0000 7f72cb86e740 1 -- 172.21.15.1:0/721034313 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559c562c32f0 msgr2=0x559c562ddff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:00.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.535+0000 7f72cb86e740 1 --2- 172.21.15.1:0/721034313 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559c562c32f0 0x559c562ddff0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f72b8000c00 tx=0x7f72b8005c90 comp rx=0 tx=0).stop 2024-09-11T21:57:00.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.536+0000 7f72cb86e740 1 -- 172.21.15.1:0/721034313 shutdown_connections 2024-09-11T21:57:00.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.536+0000 7f72cb86e740 1 --2- 172.21.15.1:0/721034313 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559c562cdbb0 0x559c562e7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:00.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.536+0000 7f72cb86e740 1 --2- 172.21.15.1:0/721034313 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559c562c32f0 0x559c562ddff0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:00.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.536+0000 7f72cb86e740 1 --2- 172.21.15.1:0/721034313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c562c29d0 0x559c562ded90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:00.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.536+0000 7f72cb86e740 1 -- 172.21.15.1:0/721034313 >> 172.21.15.1:0/721034313 conn(0x559c562bef60 msgr2=0x559c562c9060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:00.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.536+0000 7f72cb86e740 1 -- 172.21.15.1:0/721034313 shutdown_connections 2024-09-11T21:57:00.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.536+0000 7f72cb86e740 1 -- 172.21.15.1:0/721034313 wait complete. 2024-09-11T21:57:00.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.536+0000 7f72cb86e740 1 -- start start 2024-09-11T21:57:00.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.536+0000 7f72cb86e740 1 Processor -- start 2024-09-11T21:57:00.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.536+0000 7f72cb86e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c562c29d0 0x559c562e0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:00.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.536+0000 7f72cb86e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559c562c32f0 0x559c562e1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:00.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.536+0000 7f72cb86e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559c562cdbb0 0x559c562e1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:00.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.536+0000 7f72cb86e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x559c562df9b0 con 0x559c562cdbb0 2024-09-11T21:57:00.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.536+0000 7f72cb86e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x559c562df2e0 con 0x559c562c32f0 2024-09-11T21:57:00.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.536+0000 7f72cb86e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x559c562e0080 con 0x559c562c29d0 2024-09-11T21:57:00.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.536+0000 7f72cb86e740 10 client.0 _ll_register_callbacks cb 0x7f72bc009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:57:00.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.536+0000 7f72c37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559c562cdbb0 0x559c562e1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:00.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.536+0000 7f72c37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559c562cdbb0 0x559c562e1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54902/0 (socket says 172.21.15.1:54902) 2024-09-11T21:57:00.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.536+0000 7f72c37fe640 1 -- 172.21.15.1:0/1882975496 learned_addr learned my addr 172.21.15.1:0/1882975496 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:00.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.537+0000 7f72c27fc640 1 --2- 172.21.15.1:0/1882975496 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559c562c32f0 0x559c562e1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:00.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.537+0000 7f72c2ffd640 1 --2- 172.21.15.1:0/1882975496 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c562c29d0 0x559c562e0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:00.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.537+0000 7f72c37fe640 1 -- 172.21.15.1:0/1882975496 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c562c29d0 msgr2=0x559c562e0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:00.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.537+0000 7f72c37fe640 1 --2- 172.21.15.1:0/1882975496 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c562c29d0 0x559c562e0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:00.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.537+0000 7f72c37fe640 1 -- 172.21.15.1:0/1882975496 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559c562c32f0 msgr2=0x559c562e1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:00.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.537+0000 7f72c37fe640 1 --2- 172.21.15.1:0/1882975496 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559c562c32f0 0x559c562e1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:00.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.537+0000 7f72c37fe640 1 -- 172.21.15.1:0/1882975496 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c56346dc0 con 0x559c562cdbb0 2024-09-11T21:57:00.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.537+0000 7f72c27fc640 1 --2- 172.21.15.1:0/1882975496 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559c562c32f0 0x559c562e1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:57:00.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.537+0000 7f72c37fe640 1 --2- 172.21.15.1:0/1882975496 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559c562cdbb0 0x559c562e1560 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f72bc000f80 tx=0x7f72bc026e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4da34b5d00c5b61e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:00.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.537+0000 7f72a6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T21:57:00.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.537+0000 7f72a6ff5640 1 -- 172.21.15.1:0/1882975496 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72bc036070 con 0x559c562cdbb0 2024-09-11T21:57:00.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.537+0000 7f72a6ff5640 1 -- 172.21.15.1:0/1882975496 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f72bc004ab0 con 0x559c562cdbb0 2024-09-11T21:57:00.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.537+0000 7f72cb86e740 1 -- 172.21.15.1:0/1882975496 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559c56230b10 con 0x559c562cdbb0 2024-09-11T21:57:00.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.537+0000 7f72a6ff5640 1 -- 172.21.15.1:0/1882975496 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f72bc042050 con 0x559c562cdbb0 2024-09-11T21:57:00.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.537+0000 7f72cb86e740 10 client.15114 fetch_fsmap learned FSMap version 27 2024-09-11T21:57:00.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.537+0000 7f72cb86e740 1 -- 172.21.15.1:0/1882975496 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559c56347fa0 con 0x559c562cdbb0 2024-09-11T21:57:00.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.538+0000 7f72a6ff5640 1 -- 172.21.15.1:0/1882975496 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x559c56230b10 con 0x559c562cdbb0 2024-09-11T21:57:00.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.538+0000 7f72a6ff5640 1 -- 172.21.15.1:0/1882975496 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f72bc0051d0 con 0x559c562cdbb0 2024-09-11T21:57:00.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.538+0000 7f72cb86e740 10 client.15114 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T21:57:00.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.538+0000 7f72cb86e740 10 client.15114 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-11T21:57:00.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.538+0000 7f72cb86e740 20 client.15114 populate_metadata read hostname 'smithi001' 2024-09-11T21:57:00.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.538+0000 7f72cb86e740 1 --2- 172.21.15.1:0/1882975496 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x559c5634cb90 0x559c5634b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:00.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.538+0000 7f72cb86e740 4 client.15114 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T21:57:00.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.538+0000 7f72cb86e740 1 -- 172.21.15.1:0/1882975496 --> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] -- command(tid 0: {"prefix": "status"}) -- 0x559c561ad0e0 con 0x559c5634cb90 2024-09-11T21:57:00.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.538+0000 7f72c2ffd640 1 --2- 172.21.15.1:0/1882975496 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x559c5634cb90 0x559c5634b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:00.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.539+0000 7f72a6ff5640 10 client.15114 ms_handle_connect on v2:172.21.15.161:6834/828381592 2024-09-11T21:57:00.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.539+0000 7f72c2ffd640 1 --2- 172.21.15.1:0/1882975496 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x559c5634cb90 0x559c5634b8a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7e8d3b6fda3f486e server_cookie=6608ac3feea83b68 in_seq=0 out_seq=0 2024-09-11T21:57:00.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.539+0000 7f72a6ff5640 1 -- 172.21.15.1:0/1882975496 <== mds.4 v2:172.21.15.161:6834/828381592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x559c561ad0e0 con 0x559c5634cb90 2024-09-11T21:57:00.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.539+0000 7f72a6ff5640 10 client.15114 handle_command_reply: tid=0 2024-09-11T21:57:00.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.539+0000 7f72cb86e740 1 client.15114 shutdown 2024-09-11T21:57:00.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.539+0000 7f72cb86e740 1 -- 172.21.15.1:0/1882975496 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559c562cdbb0 msgr2=0x559c562e1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:00.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.539+0000 7f72cb86e740 1 --2- 172.21.15.1:0/1882975496 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559c562cdbb0 0x559c562e1560 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f72bc000f80 tx=0x7f72bc026e00 comp rx=0 tx=0).stop 2024-09-11T21:57:00.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.539+0000 7f72cb86e740 1 -- 172.21.15.1:0/1882975496 shutdown_connections 2024-09-11T21:57:00.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.539+0000 7f72cb86e740 1 --2- 172.21.15.1:0/1882975496 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559c562cdbb0 0x559c562e1560 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:00.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.539+0000 7f72cb86e740 1 --2- 172.21.15.1:0/1882975496 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x559c5634cb90 0x559c5634b8a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:00.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.539+0000 7f72a6ff5640 0 client.15114 ms_handle_reset on v2:172.21.15.161:6834/828381592 2024-09-11T21:57:00.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.539+0000 7f72cb86e740 1 --2- 172.21.15.1:0/1882975496 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559c562c32f0 0x559c562e1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:00.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.539+0000 7f72cb86e740 1 --2- 172.21.15.1:0/1882975496 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559c562c29d0 0x559c562e0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:00.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.539+0000 7f72cb86e740 1 -- 172.21.15.1:0/1882975496 >> 172.21.15.1:0/1882975496 conn(0x559c562bef60 msgr2=0x559c562c4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:00.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.539+0000 7f72cb86e740 1 -- 172.21.15.1:0/1882975496 shutdown_connections 2024-09-11T21:57:00.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.539+0000 7f72cb86e740 1 -- 172.21.15.1:0/1882975496 wait complete. 2024-09-11T21:57:00.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.539+0000 7f72cb86e740 20 client.15114 trim_cache size 0 max 16384 2024-09-11T21:57:00.575 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-11T21:57:00.575 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "3ffca83c-7087-11ef-bcea-c7b262605968", 2024-09-11T21:57:00.575 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 4, 2024-09-11T21:57:00.575 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 24460, 2024-09-11T21:57:00.576 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-09-11T21:57:00.576 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-09-11T21:57:00.576 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-09-11T21:57:00.576 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 89.391504510000004, 2024-09-11T21:57:00.576 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 27, 2024-09-11T21:57:00.577 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 86, 2024-09-11T21:57:00.577 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 84, 2024-09-11T21:57:00.577 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 126.854485493 2024-09-11T21:57:00.577 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-11T21:57:00.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.540+0000 7f72c9495640 1 -- 172.21.15.1:0/4109887041 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f72ac0a2c00 msgr2=0x7f72ac0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:00.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.540+0000 7f72c9495640 1 --2- 172.21.15.1:0/4109887041 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f72ac0a2c00 0x7f72ac0a50c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f72c40754f0 tx=0x7f72bc004a40 comp rx=0 tx=0).stop 2024-09-11T21:57:00.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.540+0000 7f72c9495640 1 -- 172.21.15.1:0/4109887041 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f72c4074e30 msgr2=0x7f72c419abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:00.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.540+0000 7f72c9495640 1 --2- 172.21.15.1:0/4109887041 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f72c4074e30 0x7f72c419abe0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f72b8006010 tx=0x7f72b80042f0 comp rx=0 tx=0).stop 2024-09-11T21:57:00.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.540+0000 7f72c9495640 1 -- 172.21.15.1:0/4109887041 shutdown_connections 2024-09-11T21:57:00.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.540+0000 7f72c9495640 1 --2- 172.21.15.1:0/4109887041 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f72ac0a2c00 0x7f72ac0a50c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:00.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.540+0000 7f72c9495640 1 --2- 172.21.15.1:0/4109887041 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72c410e440 0x7f72c419ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:00.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.540+0000 7f72c9495640 1 --2- 172.21.15.1:0/4109887041 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f72c4074e30 0x7f72c419abe0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:00.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.540+0000 7f72c9495640 1 --2- 172.21.15.1:0/4109887041 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f72c4074510 0x7f72c419a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:00.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.540+0000 7f72c9495640 1 -- 172.21.15.1:0/4109887041 >> 172.21.15.1:0/4109887041 conn(0x7f72c40fd8c0 msgr2=0x7f72c40fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:00.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.540+0000 7f72c9495640 1 -- 172.21.15.1:0/4109887041 shutdown_connections 2024-09-11T21:57:00.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:00.540+0000 7f72c9495640 1 -- 172.21.15.1:0/4109887041 wait complete. 2024-09-11T21:57:00.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:00 smithi053 ceph-mon[34908]: pgmap v333: 97 pgs: 97 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 9 op/s 2024-09-11T21:57:00.845 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:00 smithi161 ceph-mon[35615]: pgmap v333: 97 pgs: 97 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 9 op/s 2024-09-11T21:57:01.248 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.c status 2024-09-11T21:57:01.505 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:57:02.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.015+0000 7f37bd263640 1 -- 172.21.15.1:0/2197051980 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37b8102790 msgr2=0x7f37b81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:02.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.015+0000 7f37bd263640 1 --2- 172.21.15.1:0/2197051980 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37b8102790 0x7f37b81067e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f37ac009900 tx=0x7f37ac02a410 comp rx=0 tx=0).stop 2024-09-11T21:57:02.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.015+0000 7f37bd263640 1 -- 172.21.15.1:0/2197051980 shutdown_connections 2024-09-11T21:57:02.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.015+0000 7f37bd263640 1 --2- 172.21.15.1:0/2197051980 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37b8106f10 0x7f37b810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:02.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.015+0000 7f37bd263640 1 --2- 172.21.15.1:0/2197051980 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37b8102790 0x7f37b81067e0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:02.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.015+0000 7f37bd263640 1 --2- 172.21.15.1:0/2197051980 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f37b8101de0 0x7f37b81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:02.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.015+0000 7f37bd263640 1 -- 172.21.15.1:0/2197051980 >> 172.21.15.1:0/2197051980 conn(0x7f37b80fd8c0 msgr2=0x7f37b80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:02.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.015+0000 7f37bd263640 1 -- 172.21.15.1:0/2197051980 shutdown_connections 2024-09-11T21:57:02.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.015+0000 7f37bd263640 1 -- 172.21.15.1:0/2197051980 wait complete. 2024-09-11T21:57:02.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.016+0000 7f37bd263640 1 Processor -- start 2024-09-11T21:57:02.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.016+0000 7f37bd263640 1 -- start start 2024-09-11T21:57:02.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.016+0000 7f37bd263640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37b8101de0 0x7f37b80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:02.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.017+0000 7f37bd263640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f37b8102790 0x7f37b8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:02.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.017+0000 7f37bd263640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37b8106f10 0x7f37b8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:02.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.017+0000 7f37bd263640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f37b8111f90 con 0x7f37b8101de0 2024-09-11T21:57:02.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.017+0000 7f37bd263640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f37b8111e10 con 0x7f37b8106f10 2024-09-11T21:57:02.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.017+0000 7f37bd263640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f37b8112110 con 0x7f37b8102790 2024-09-11T21:57:02.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.017+0000 7f37b6ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37b8101de0 0x7f37b80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:02.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.017+0000 7f37b6ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37b8101de0 0x7f37b80744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54930/0 (socket says 172.21.15.1:54930) 2024-09-11T21:57:02.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.017+0000 7f37b67fc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f37b8102790 0x7f37b8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:02.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.017+0000 7f37b6ffd640 1 -- 172.21.15.1:0/3540472954 learned_addr learned my addr 172.21.15.1:0/3540472954 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:02.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.017+0000 7f37b77fe640 1 --2- 172.21.15.1:0/3540472954 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37b8106f10 0x7f37b8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:02.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.017+0000 7f37b6ffd640 1 -- 172.21.15.1:0/3540472954 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f37b8102790 msgr2=0x7f37b8074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:02.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.017+0000 7f37b6ffd640 1 --2- 172.21.15.1:0/3540472954 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f37b8102790 0x7f37b8074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:02.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.017+0000 7f37b6ffd640 1 -- 172.21.15.1:0/3540472954 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37b8106f10 msgr2=0x7f37b8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:02.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.017+0000 7f37b6ffd640 1 --2- 172.21.15.1:0/3540472954 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37b8106f10 0x7f37b8074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:02.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.017+0000 7f37b6ffd640 1 -- 172.21.15.1:0/3540472954 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37b81a68f0 con 0x7f37b8101de0 2024-09-11T21:57:02.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.017+0000 7f37b6ffd640 1 --2- 172.21.15.1:0/3540472954 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37b8101de0 0x7f37b80744f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f37a800b4a0 tx=0x7f37a800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:02.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.017+0000 7f379ffff640 1 -- 172.21.15.1:0/3540472954 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a8014070 con 0x7f37b8101de0 2024-09-11T21:57:02.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.017+0000 7f37bd263640 1 -- 172.21.15.1:0/3540472954 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37b81a6be0 con 0x7f37b8101de0 2024-09-11T21:57:02.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.017+0000 7f379ffff640 1 -- 172.21.15.1:0/3540472954 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37a8010af0 con 0x7f37b8101de0 2024-09-11T21:57:02.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.018+0000 7f379ffff640 1 -- 172.21.15.1:0/3540472954 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a8017d10 con 0x7f37b8101de0 2024-09-11T21:57:02.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.018+0000 7f37bd263640 1 -- 172.21.15.1:0/3540472954 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37b81a70f0 con 0x7f37b8101de0 2024-09-11T21:57:02.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.020+0000 7f37bf63c740 1 Processor -- start 2024-09-11T21:57:02.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.020+0000 7f37bf63c740 1 -- start start 2024-09-11T21:57:02.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.020+0000 7f37bf63c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56122dd149d0 0x56122dd14db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:02.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.020+0000 7f37bf63c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56122dd152f0 0x56122dd1f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:02.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.020+0000 7f37bf63c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56122dd1fbb0 0x56122dd21fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:02.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.020+0000 7f37bf63c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56122dc7b090 con 0x56122dd1fbb0 2024-09-11T21:57:02.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.020+0000 7f37bf63c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56122dc22a70 con 0x56122dd149d0 2024-09-11T21:57:02.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.020+0000 7f37bf63c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56122dc77050 con 0x56122dd152f0 2024-09-11T21:57:02.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.020+0000 7f37b67fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56122dd1fbb0 0x56122dd21fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:02.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.020+0000 7f37b67fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56122dd1fbb0 0x56122dd21fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54946/0 (socket says 172.21.15.1:54946) 2024-09-11T21:57:02.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.020+0000 7f37b67fc640 1 -- 172.21.15.1:0/1260497464 learned_addr learned my addr 172.21.15.1:0/1260497464 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:02.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.021+0000 7f379ffff640 1 -- 172.21.15.1:0/3540472954 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f37a8017070 con 0x7f37b8101de0 2024-09-11T21:57:02.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.021+0000 7f37b77fe640 1 --2- 172.21.15.1:0/1260497464 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56122dd149d0 0x56122dd14db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:02.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.021+0000 7f37b67fc640 1 -- 172.21.15.1:0/1260497464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56122dd152f0 msgr2=0x56122dd1f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:02.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.021+0000 7f37b67fc640 1 --2- 172.21.15.1:0/1260497464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56122dd152f0 0x56122dd1f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:02.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.021+0000 7f37b67fc640 1 -- 172.21.15.1:0/1260497464 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56122dd149d0 msgr2=0x56122dd14db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:02.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.021+0000 7f37b67fc640 1 --2- 172.21.15.1:0/1260497464 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56122dd149d0 0x56122dd14db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:02.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.021+0000 7f37b67fc640 1 -- 172.21.15.1:0/1260497464 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56122dc2f130 con 0x56122dd1fbb0 2024-09-11T21:57:02.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.021+0000 7f37b77fe640 1 --2- 172.21.15.1:0/1260497464 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56122dd149d0 0x56122dd14db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T21:57:02.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.021+0000 7f37b67fc640 1 --2- 172.21.15.1:0/1260497464 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56122dd1fbb0 0x56122dd21fa0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f37ac009900 tx=0x7f37ac0316e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51ecb95c514944df server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:02.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.021+0000 7f379d7fa640 1 -- 172.21.15.1:0/1260497464 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ac040070 con 0x56122dd1fbb0 2024-09-11T21:57:02.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.021+0000 7f379d7fa640 1 -- 172.21.15.1:0/1260497464 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37ac005020 con 0x56122dd1fbb0 2024-09-11T21:57:02.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.021+0000 7f379d7fa640 1 -- 172.21.15.1:0/1260497464 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ac035040 con 0x56122dd1fbb0 2024-09-11T21:57:02.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.021+0000 7f379ffff640 1 --2- 172.21.15.1:0/3540472954 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f37a00a2b80 0x7f37a00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:02.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.022+0000 7f37b77fe640 1 --2- 172.21.15.1:0/3540472954 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f37a00a2b80 0x7f37a00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:02.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.022+0000 7f379ffff640 1 -- 172.21.15.1:0/3540472954 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f37a80ce430 con 0x7f37b8101de0 2024-09-11T21:57:02.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.022+0000 7f37bf63c740 1 -- 172.21.15.1:0/1260497464 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56122dd1fbb0 msgr2=0x56122dd21fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:02.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.022+0000 7f37bf63c740 1 --2- 172.21.15.1:0/1260497464 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56122dd1fbb0 0x56122dd21fa0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f37ac009900 tx=0x7f37ac0316e0 comp rx=0 tx=0).stop 2024-09-11T21:57:02.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.022+0000 7f37b77fe640 1 --2- 172.21.15.1:0/3540472954 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f37a00a2b80 0x7f37a00a5040 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f37b000ae90 tx=0x7f37b000a440 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:02.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.022+0000 7f37bf63c740 1 -- 172.21.15.1:0/1260497464 shutdown_connections 2024-09-11T21:57:02.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.022+0000 7f37bf63c740 1 --2- 172.21.15.1:0/1260497464 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56122dd1fbb0 0x56122dd21fa0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:02.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.022+0000 7f37bf63c740 1 --2- 172.21.15.1:0/1260497464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56122dd152f0 0x56122dd1f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:02.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.022+0000 7f37bf63c740 1 --2- 172.21.15.1:0/1260497464 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56122dd149d0 0x56122dd14db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:02.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.022+0000 7f37bf63c740 1 -- 172.21.15.1:0/1260497464 >> 172.21.15.1:0/1260497464 conn(0x56122dd10f60 msgr2=0x56122dd11340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:02.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.022+0000 7f37bf63c740 1 -- 172.21.15.1:0/1260497464 shutdown_connections 2024-09-11T21:57:02.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.022+0000 7f37bf63c740 1 -- 172.21.15.1:0/1260497464 wait complete. 2024-09-11T21:57:02.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.023+0000 7f37bf63c740 1 -- start start 2024-09-11T21:57:02.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.023+0000 7f37bf63c740 1 Processor -- start 2024-09-11T21:57:02.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.023+0000 7f37bf63c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56122dd149d0 0x56122dd1f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:02.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.023+0000 7f37bf63c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56122dd152f0 0x56122dd1c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:02.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.023+0000 7f37bf63c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56122dd1fbb0 0x56122dd1ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:02.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.023+0000 7f37bf63c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56122dd22780 con 0x56122dd152f0 2024-09-11T21:57:02.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.023+0000 7f37bf63c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56122dc35f20 con 0x56122dd149d0 2024-09-11T21:57:02.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.023+0000 7f37bf63c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56122dd22900 con 0x56122dd1fbb0 2024-09-11T21:57:02.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.023+0000 7f37bf63c740 10 client.0 _ll_register_callbacks cb 0x56122dbff030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:57:02.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.023+0000 7f37b67fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56122dd152f0 0x56122dd1c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:02.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.023+0000 7f37b67fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56122dd152f0 0x56122dd1c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54960/0 (socket says 172.21.15.1:54960) 2024-09-11T21:57:02.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.023+0000 7f37b67fc640 1 -- 172.21.15.1:0/263077574 learned_addr learned my addr 172.21.15.1:0/263077574 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:02.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.023+0000 7f37b77fe640 1 --2- 172.21.15.1:0/263077574 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56122dd1fbb0 0x56122dd1ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:02.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.024+0000 7f37b6ffd640 1 --2- 172.21.15.1:0/263077574 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56122dd149d0 0x56122dd1f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:02.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.024+0000 7f37b67fc640 1 -- 172.21.15.1:0/263077574 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56122dd1fbb0 msgr2=0x56122dd1ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:02.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.024+0000 7f37b67fc640 1 --2- 172.21.15.1:0/263077574 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56122dd1fbb0 0x56122dd1ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:02.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.024+0000 7f37b67fc640 1 -- 172.21.15.1:0/263077574 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56122dd149d0 msgr2=0x56122dd1f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:02.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.024+0000 7f37b67fc640 1 --2- 172.21.15.1:0/263077574 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56122dd149d0 0x56122dd1f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:02.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.024+0000 7f37b67fc640 1 -- 172.21.15.1:0/263077574 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56122d9ec2b0 con 0x56122dd152f0 2024-09-11T21:57:02.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.024+0000 7f37b6ffd640 1 --2- 172.21.15.1:0/263077574 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56122dd149d0 0x56122dd1f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T21:57:02.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.024+0000 7f37b67fc640 1 --2- 172.21.15.1:0/263077574 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56122dd152f0 0x56122dd1c770 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f37ac009670 tx=0x7f37ac0051d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56f2767db4ba1a35 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:02.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.024+0000 7f379aff5640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T21:57:02.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.024+0000 7f379aff5640 1 -- 172.21.15.1:0/263077574 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ac04c020 con 0x56122dd152f0 2024-09-11T21:57:02.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.024+0000 7f379aff5640 1 -- 172.21.15.1:0/263077574 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f37ac004af0 con 0x56122dd152f0 2024-09-11T21:57:02.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.024+0000 7f37bf63c740 1 -- 172.21.15.1:0/263077574 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37ac005020 con 0x56122dd152f0 2024-09-11T21:57:02.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.024+0000 7f379aff5640 1 -- 172.21.15.1:0/263077574 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f37ac057050 con 0x56122dd152f0 2024-09-11T21:57:02.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.024+0000 7f37bf63c740 10 client.15126 fetch_fsmap learned FSMap version 27 2024-09-11T21:57:02.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.024+0000 7f37bf63c740 1 -- 172.21.15.1:0/263077574 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56122dd2e010 con 0x56122dd152f0 2024-09-11T21:57:02.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.025+0000 7f379aff5640 1 -- 172.21.15.1:0/263077574 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37ac004cf0 con 0x56122dd152f0 2024-09-11T21:57:02.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.025+0000 7f379aff5640 1 -- 172.21.15.1:0/263077574 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f37ac044520 con 0x56122dd152f0 2024-09-11T21:57:02.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.025+0000 7f37bf63c740 10 client.15126 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T21:57:02.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.025+0000 7f37bf63c740 10 client.15126 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-11T21:57:02.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.025+0000 7f37bf63c740 20 client.15126 populate_metadata read hostname 'smithi001' 2024-09-11T21:57:02.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.025+0000 7f37bf63c740 1 --2- 172.21.15.1:0/263077574 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x56122dd30150 0x56122dd315c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:02.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.025+0000 7f37bf63c740 4 client.15126 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T21:57:02.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.025+0000 7f37bf63c740 1 -- 172.21.15.1:0/263077574 --> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f37ac005020 con 0x56122dd30150 2024-09-11T21:57:02.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.025+0000 7f37b6ffd640 1 --2- 172.21.15.1:0/263077574 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x56122dd30150 0x56122dd315c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:02.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.026+0000 7f37b6ffd640 1 --2- 172.21.15.1:0/263077574 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x56122dd30150 0x56122dd315c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cb519e5d89a50092 server_cookie=73fb4a3d9da2ad7d in_seq=0 out_seq=0 2024-09-11T21:57:02.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.026+0000 7f379aff5640 10 client.15126 ms_handle_connect on v2:172.21.15.161:6836/1799318656 2024-09-11T21:57:02.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.027+0000 7f379aff5640 1 -- 172.21.15.1:0/263077574 <== mds.3 v2:172.21.15.161:6836/1799318656 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f37ac005020 con 0x56122dd30150 2024-09-11T21:57:02.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.027+0000 7f379aff5640 10 client.15126 handle_command_reply: tid=0 2024-09-11T21:57:02.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.027+0000 7f37bf63c740 1 client.15126 shutdown 2024-09-11T21:57:02.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.027+0000 7f37bf63c740 1 -- 172.21.15.1:0/263077574 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56122dd152f0 msgr2=0x56122dd1c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:02.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.027+0000 7f37bf63c740 1 --2- 172.21.15.1:0/263077574 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56122dd152f0 0x56122dd1c770 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f37ac009670 tx=0x7f37ac0051d0 comp rx=0 tx=0).stop 2024-09-11T21:57:02.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.027+0000 7f37bf63c740 1 -- 172.21.15.1:0/263077574 shutdown_connections 2024-09-11T21:57:02.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.027+0000 7f37bf63c740 1 --2- 172.21.15.1:0/263077574 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x56122dd30150 0x56122dd315c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:02.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.027+0000 7f37bf63c740 1 --2- 172.21.15.1:0/263077574 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56122dd1fbb0 0x56122dd1ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:02.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.027+0000 7f37bf63c740 1 --2- 172.21.15.1:0/263077574 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56122dd152f0 0x56122dd1c770 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:02.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.027+0000 7f37bf63c740 1 --2- 172.21.15.1:0/263077574 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56122dd149d0 0x56122dd1f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:02.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.027+0000 7f379aff5640 0 client.15126 ms_handle_reset on v2:172.21.15.161:6836/1799318656 2024-09-11T21:57:02.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.027+0000 7f37bf63c740 1 -- 172.21.15.1:0/263077574 >> 172.21.15.1:0/263077574 conn(0x56122dd10f60 msgr2=0x56122dd16c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:02.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.027+0000 7f37bf63c740 1 -- 172.21.15.1:0/263077574 shutdown_connections 2024-09-11T21:57:02.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.027+0000 7f37bf63c740 1 -- 172.21.15.1:0/263077574 wait complete. 2024-09-11T21:57:02.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.028+0000 7f37bf63c740 20 client.15126 trim_cache size 0 max 16384 2024-09-11T21:57:02.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.046+0000 7f37bf63c740 1 Processor -- start 2024-09-11T21:57:02.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.046+0000 7f37bf63c740 1 -- start start 2024-09-11T21:57:02.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.046+0000 7f37bf63c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56122dd149d0 0x56122dd315c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:02.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.046+0000 7f37bf63c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56122dd152f0 0x56122dd17c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:02.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.046+0000 7f37bf63c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56122dd1fbb0 0x56122dd199d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:02.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.046+0000 7f37bf63c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56122dd2f900 con 0x56122dd149d0 2024-09-11T21:57:02.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.046+0000 7f37bf63c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56122dd2ffd0 con 0x56122dd1fbb0 2024-09-11T21:57:02.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.046+0000 7f37bf63c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56122dd306a0 con 0x56122dd152f0 2024-09-11T21:57:02.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.047+0000 7f37b6ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56122dd149d0 0x56122dd315c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:02.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.047+0000 7f37b6ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56122dd149d0 0x56122dd315c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54970/0 (socket says 172.21.15.1:54970) 2024-09-11T21:57:02.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.047+0000 7f37b6ffd640 1 -- 172.21.15.1:0/1188303751 learned_addr learned my addr 172.21.15.1:0/1188303751 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:02.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.047+0000 7f37b67fc640 1 --2- 172.21.15.1:0/1188303751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56122dd152f0 0x56122dd17c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:02.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.047+0000 7f37b77fe640 1 --2- 172.21.15.1:0/1188303751 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56122dd1fbb0 0x56122dd199d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:02.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.047+0000 7f37b6ffd640 1 -- 172.21.15.1:0/1188303751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56122dd152f0 msgr2=0x56122dd17c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:02.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.047+0000 7f37b6ffd640 1 --2- 172.21.15.1:0/1188303751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56122dd152f0 0x56122dd17c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:02.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.047+0000 7f37b6ffd640 1 -- 172.21.15.1:0/1188303751 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56122dd1fbb0 msgr2=0x56122dd199d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:02.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.047+0000 7f37b6ffd640 1 --2- 172.21.15.1:0/1188303751 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56122dd1fbb0 0x56122dd199d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:02.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.047+0000 7f37b6ffd640 1 -- 172.21.15.1:0/1188303751 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56122dd1acc0 con 0x56122dd149d0 2024-09-11T21:57:02.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.047+0000 7f37b67fc640 1 --2- 172.21.15.1:0/1188303751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56122dd152f0 0x56122dd17c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:57:02.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.047+0000 7f37b6ffd640 1 --2- 172.21.15.1:0/1188303751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56122dd149d0 0x56122dd315c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f37a8009420 tx=0x7f37a80047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28afc289b35b4547 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:02.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.047+0000 7f379cff9640 1 -- 172.21.15.1:0/1188303751 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a801e040 con 0x56122dd149d0 2024-09-11T21:57:02.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.047+0000 7f379cff9640 1 -- 172.21.15.1:0/1188303751 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37a80a38e0 con 0x56122dd149d0 2024-09-11T21:57:02.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.047+0000 7f379cff9640 1 -- 172.21.15.1:0/1188303751 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a8010ca0 con 0x56122dd149d0 2024-09-11T21:57:02.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.048+0000 7f37bf63c740 1 -- 172.21.15.1:0/1188303751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56122dd149d0 msgr2=0x56122dd315c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:02.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.048+0000 7f37bf63c740 1 --2- 172.21.15.1:0/1188303751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56122dd149d0 0x56122dd315c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f37a8009420 tx=0x7f37a80047d0 comp rx=0 tx=0).stop 2024-09-11T21:57:02.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.048+0000 7f37bf63c740 1 -- 172.21.15.1:0/1188303751 shutdown_connections 2024-09-11T21:57:02.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.048+0000 7f37bf63c740 1 --2- 172.21.15.1:0/1188303751 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56122dd1fbb0 0x56122dd199d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:02.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.048+0000 7f37bf63c740 1 --2- 172.21.15.1:0/1188303751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56122dd152f0 0x56122dd17c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:02.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.048+0000 7f37bf63c740 1 --2- 172.21.15.1:0/1188303751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56122dd149d0 0x56122dd315c0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:02.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.048+0000 7f37bf63c740 1 -- 172.21.15.1:0/1188303751 >> 172.21.15.1:0/1188303751 conn(0x56122dd10f60 msgr2=0x56122dd17560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:02.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.048+0000 7f37bf63c740 1 -- 172.21.15.1:0/1188303751 shutdown_connections 2024-09-11T21:57:02.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.048+0000 7f37bf63c740 1 -- 172.21.15.1:0/1188303751 wait complete. 2024-09-11T21:57:02.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.048+0000 7f37bf63c740 1 -- start start 2024-09-11T21:57:02.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.049+0000 7f37bf63c740 1 Processor -- start 2024-09-11T21:57:02.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.049+0000 7f37bf63c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56122dd149d0 0x56122dd3a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:02.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.049+0000 7f37bf63c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56122dd152f0 0x56122dd33560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:02.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.049+0000 7f37bf63c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56122dd1fbb0 0x56122dd33cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:02.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.049+0000 7f37bf63c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56122dd31440 con 0x56122dd1fbb0 2024-09-11T21:57:02.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.049+0000 7f37bf63c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56122dd30d70 con 0x56122dd149d0 2024-09-11T21:57:02.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.049+0000 7f37bf63c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56122dd31b10 con 0x56122dd152f0 2024-09-11T21:57:02.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.049+0000 7f37bf63c740 10 client.0 _ll_register_callbacks cb 0x7f37ac0315b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:57:02.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.049+0000 7f37b77fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56122dd1fbb0 0x56122dd33cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:02.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.049+0000 7f37b6ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56122dd149d0 0x56122dd3a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:02.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.049+0000 7f37b6ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56122dd149d0 0x56122dd3a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:43954/0 (socket says 172.21.15.1:43954) 2024-09-11T21:57:02.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.049+0000 7f37b77fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56122dd1fbb0 0x56122dd33cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54976/0 (socket says 172.21.15.1:54976) 2024-09-11T21:57:02.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.049+0000 7f37b6ffd640 1 -- 172.21.15.1:0/228255197 learned_addr learned my addr 172.21.15.1:0/228255197 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:02.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.049+0000 7f37b67fc640 1 --2- 172.21.15.1:0/228255197 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56122dd152f0 0x56122dd33560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:02.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.049+0000 7f37b77fe640 1 -- 172.21.15.1:0/228255197 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56122dd152f0 msgr2=0x56122dd33560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:02.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.049+0000 7f37b77fe640 1 --2- 172.21.15.1:0/228255197 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56122dd152f0 0x56122dd33560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:02.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.049+0000 7f37b77fe640 1 -- 172.21.15.1:0/228255197 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56122dd149d0 msgr2=0x56122dd3a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:02.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.049+0000 7f37b77fe640 1 --2- 172.21.15.1:0/228255197 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56122dd149d0 0x56122dd3a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:02.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.049+0000 7f37b77fe640 1 -- 172.21.15.1:0/228255197 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56122dd37cb0 con 0x56122dd1fbb0 2024-09-11T21:57:02.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.050+0000 7f37b77fe640 1 --2- 172.21.15.1:0/228255197 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56122dd1fbb0 0x56122dd33cf0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f37b000a680 tx=0x7f37b0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ace9b3991277f3c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:02.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.050+0000 7f379aff5640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T21:57:02.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.050+0000 7f379aff5640 1 -- 172.21.15.1:0/228255197 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b002d060 con 0x56122dd1fbb0 2024-09-11T21:57:02.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.050+0000 7f379aff5640 1 -- 172.21.15.1:0/228255197 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f37b0011de0 con 0x56122dd1fbb0 2024-09-11T21:57:02.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.050+0000 7f37bf63c740 1 -- 172.21.15.1:0/228255197 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37a80a38e0 con 0x56122dd1fbb0 2024-09-11T21:57:02.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.050+0000 7f379aff5640 1 -- 172.21.15.1:0/228255197 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f37b0033400 con 0x56122dd1fbb0 2024-09-11T21:57:02.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.050+0000 7f37bf63c740 10 client.15132 fetch_fsmap learned FSMap version 27 2024-09-11T21:57:02.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.050+0000 7f37bf63c740 1 -- 172.21.15.1:0/228255197 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56122dd9ab30 con 0x56122dd1fbb0 2024-09-11T21:57:02.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.050+0000 7f379aff5640 1 -- 172.21.15.1:0/228255197 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37a80a38e0 con 0x56122dd1fbb0 2024-09-11T21:57:02.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.050+0000 7f379aff5640 1 -- 172.21.15.1:0/228255197 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f37b0027030 con 0x56122dd1fbb0 2024-09-11T21:57:02.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.050+0000 7f37bf63c740 10 client.15132 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T21:57:02.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.050+0000 7f37bf63c740 10 client.15132 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-11T21:57:02.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.050+0000 7f37bf63c740 20 client.15132 populate_metadata read hostname 'smithi001' 2024-09-11T21:57:02.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.050+0000 7f37bf63c740 1 --2- 172.21.15.1:0/228255197 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x56122dd9f6e0 0x56122dd9e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:02.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.050+0000 7f37bf63c740 4 client.15132 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T21:57:02.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.050+0000 7f37bf63c740 1 -- 172.21.15.1:0/228255197 --> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] -- command(tid 0: {"prefix": "status"}) -- 0x56122dc88600 con 0x56122dd9f6e0 2024-09-11T21:57:02.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.051+0000 7f37b6ffd640 1 --2- 172.21.15.1:0/228255197 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x56122dd9f6e0 0x56122dd9e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:02.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.051+0000 7f37b6ffd640 1 --2- 172.21.15.1:0/228255197 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x56122dd9f6e0 0x56122dd9e3f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f3218d82f320bc65 server_cookie=3bba006a71fb5ed9 in_seq=0 out_seq=0 2024-09-11T21:57:02.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.051+0000 7f379aff5640 10 client.15132 ms_handle_connect on v2:172.21.15.161:6836/1799318656 2024-09-11T21:57:02.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.051+0000 7f379aff5640 1 -- 172.21.15.1:0/228255197 <== mds.3 v2:172.21.15.161:6836/1799318656 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x56122dc88600 con 0x56122dd9f6e0 2024-09-11T21:57:02.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.051+0000 7f379aff5640 10 client.15132 handle_command_reply: tid=0 2024-09-11T21:57:02.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.051+0000 7f37bf63c740 1 client.15132 shutdown 2024-09-11T21:57:02.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.051+0000 7f37bf63c740 1 -- 172.21.15.1:0/228255197 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56122dd1fbb0 msgr2=0x56122dd33cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:02.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.051+0000 7f37bf63c740 1 --2- 172.21.15.1:0/228255197 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56122dd1fbb0 0x56122dd33cf0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f37b000a680 tx=0x7f37b0004ab0 comp rx=0 tx=0).stop 2024-09-11T21:57:02.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.051+0000 7f37bf63c740 1 -- 172.21.15.1:0/228255197 shutdown_connections 2024-09-11T21:57:02.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.051+0000 7f37bf63c740 1 --2- 172.21.15.1:0/228255197 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x56122dd9f6e0 0x56122dd9e3f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:02.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.051+0000 7f37bf63c740 1 --2- 172.21.15.1:0/228255197 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56122dd1fbb0 0x56122dd33cf0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:02.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.051+0000 7f37bf63c740 1 --2- 172.21.15.1:0/228255197 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56122dd152f0 0x56122dd33560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:02.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.051+0000 7f37bf63c740 1 --2- 172.21.15.1:0/228255197 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56122dd149d0 0x56122dd3a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:02.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.051+0000 7f37bf63c740 1 -- 172.21.15.1:0/228255197 >> 172.21.15.1:0/228255197 conn(0x56122dd10f60 msgr2=0x56122dd1c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:02.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.051+0000 7f379aff5640 0 client.15132 ms_handle_reset on v2:172.21.15.161:6836/1799318656 2024-09-11T21:57:02.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.052+0000 7f37bf63c740 1 -- 172.21.15.1:0/228255197 shutdown_connections 2024-09-11T21:57:02.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.052+0000 7f37bf63c740 1 -- 172.21.15.1:0/228255197 wait complete. 2024-09-11T21:57:02.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.052+0000 7f37bf63c740 20 client.15132 trim_cache size 0 max 16384 2024-09-11T21:57:02.059 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-11T21:57:02.059 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "3ffca83c-7087-11ef-bcea-c7b262605968", 2024-09-11T21:57:02.059 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 3, 2024-09-11T21:57:02.059 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 24466, 2024-09-11T21:57:02.059 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-09-11T21:57:02.059 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-09-11T21:57:02.059 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-09-11T21:57:02.059 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 91.910351148000004, 2024-09-11T21:57:02.059 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 27, 2024-09-11T21:57:02.060 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 86, 2024-09-11T21:57:02.060 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 84, 2024-09-11T21:57:02.060 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 125.726763862 2024-09-11T21:57:02.060 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-11T21:57:02.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.052+0000 7f37bd263640 1 -- 172.21.15.1:0/3540472954 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f37a00a2b80 msgr2=0x7f37a00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:02.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.052+0000 7f37bd263640 1 --2- 172.21.15.1:0/3540472954 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f37a00a2b80 0x7f37a00a5040 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f37b000ae90 tx=0x7f37b000a440 comp rx=0 tx=0).stop 2024-09-11T21:57:02.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.052+0000 7f37bd263640 1 -- 172.21.15.1:0/3540472954 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37b8101de0 msgr2=0x7f37b80744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:02.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.052+0000 7f37bd263640 1 --2- 172.21.15.1:0/3540472954 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37b8101de0 0x7f37b80744f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f37a800b4a0 tx=0x7f37a800b960 comp rx=0 tx=0).stop 2024-09-11T21:57:02.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.052+0000 7f37bd263640 1 -- 172.21.15.1:0/3540472954 shutdown_connections 2024-09-11T21:57:02.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.052+0000 7f37bd263640 1 --2- 172.21.15.1:0/3540472954 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f37a00a2b80 0x7f37a00a5040 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:02.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.052+0000 7f37bd263640 1 --2- 172.21.15.1:0/3540472954 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37b8106f10 0x7f37b8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:02.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.052+0000 7f37bd263640 1 --2- 172.21.15.1:0/3540472954 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f37b8102790 0x7f37b8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:02.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.052+0000 7f37bd263640 1 --2- 172.21.15.1:0/3540472954 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37b8101de0 0x7f37b80744f0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:02.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.052+0000 7f37bd263640 1 -- 172.21.15.1:0/3540472954 >> 172.21.15.1:0/3540472954 conn(0x7f37b80fd8c0 msgr2=0x7f37b80fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:02.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.052+0000 7f37bd263640 1 -- 172.21.15.1:0/3540472954 shutdown_connections 2024-09-11T21:57:02.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:02.052+0000 7f37bd263640 1 -- 172.21.15.1:0/3540472954 wait complete. 2024-09-11T21:57:02.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:02 smithi001 ceph-mon[30935]: pgmap v334: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-11T21:57:02.735 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.h status 2024-09-11T21:57:02.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:02 smithi053 ceph-mon[34908]: pgmap v334: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-11T21:57:02.841 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:02 smithi161 ceph-mon[35615]: pgmap v334: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-11T21:57:02.991 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:57:03.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.465+0000 7fe77c377640 1 -- 172.21.15.1:0/2658319009 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe774106a80 msgr2=0x7fe774106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:03.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.465+0000 7fe77c377640 1 --2- 172.21.15.1:0/2658319009 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe774106a80 0x7fe774106e60 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe770009930 tx=0x7fe77002a420 comp rx=0 tx=0).stop 2024-09-11T21:57:03.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.465+0000 7fe77c377640 1 -- 172.21.15.1:0/2658319009 shutdown_connections 2024-09-11T21:57:03.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.465+0000 7fe77c377640 1 --2- 172.21.15.1:0/2658319009 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe774101420 0x7fe77410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:03.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.465+0000 7fe77c377640 1 --2- 172.21.15.1:0/2658319009 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe774100a80 0x7fe774100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:03.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.465+0000 7fe77c377640 1 --2- 172.21.15.1:0/2658319009 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe774106a80 0x7fe774106e60 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:03.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.465+0000 7fe77c377640 1 -- 172.21.15.1:0/2658319009 >> 172.21.15.1:0/2658319009 conn(0x7fe7740fc7a0 msgr2=0x7fe7740febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:03.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.465+0000 7fe77c377640 1 -- 172.21.15.1:0/2658319009 shutdown_connections 2024-09-11T21:57:03.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.466+0000 7fe77c377640 1 -- 172.21.15.1:0/2658319009 wait complete. 2024-09-11T21:57:03.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.466+0000 7fe77c377640 1 Processor -- start 2024-09-11T21:57:03.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.466+0000 7fe77c377640 1 -- start start 2024-09-11T21:57:03.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.466+0000 7fe77c377640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe774100a80 0x7fe774198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:03.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.466+0000 7fe77c377640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe774101420 0x7fe774198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:03.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.466+0000 7fe77c377640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe774106a80 0x7fe77419cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:03.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.466+0000 7fe77c377640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe774110ea0 con 0x7fe774106a80 2024-09-11T21:57:03.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.466+0000 7fe77c377640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe774110d20 con 0x7fe774100a80 2024-09-11T21:57:03.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.466+0000 7fe77c377640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe774111020 con 0x7fe774101420 2024-09-11T21:57:03.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.466+0000 7fe77a914640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe774106a80 0x7fe77419cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:03.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.467+0000 7fe77a914640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe774106a80 0x7fe77419cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54994/0 (socket says 172.21.15.1:54994) 2024-09-11T21:57:03.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.467+0000 7fe77a914640 1 -- 172.21.15.1:0/1678077638 learned_addr learned my addr 172.21.15.1:0/1678077638 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:03.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.467+0000 7fe779912640 1 --2- 172.21.15.1:0/1678077638 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe774101420 0x7fe774198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:03.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.467+0000 7fe77a113640 1 --2- 172.21.15.1:0/1678077638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe774100a80 0x7fe774198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:03.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.467+0000 7fe77a914640 1 -- 172.21.15.1:0/1678077638 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe774101420 msgr2=0x7fe774198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:03.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.467+0000 7fe77a914640 1 --2- 172.21.15.1:0/1678077638 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe774101420 0x7fe774198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:03.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.467+0000 7fe77a914640 1 -- 172.21.15.1:0/1678077638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe774100a80 msgr2=0x7fe774198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:03.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.467+0000 7fe77a914640 1 --2- 172.21.15.1:0/1678077638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe774100a80 0x7fe774198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:03.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.467+0000 7fe77a914640 1 -- 172.21.15.1:0/1678077638 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe77419d4d0 con 0x7fe774106a80 2024-09-11T21:57:03.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.467+0000 7fe77a113640 1 --2- 172.21.15.1:0/1678077638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe774100a80 0x7fe774198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:57:03.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.467+0000 7fe77a914640 1 --2- 172.21.15.1:0/1678077638 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe774106a80 0x7fe77419cdf0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe76c00cad0 tx=0x7fe76c00ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:03.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.467+0000 7fe76b7fe640 1 -- 172.21.15.1:0/1678077638 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe76c00ad60 con 0x7fe774106a80 2024-09-11T21:57:03.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.467+0000 7fe76b7fe640 1 -- 172.21.15.1:0/1678077638 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe76c005020 con 0x7fe774106a80 2024-09-11T21:57:03.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.467+0000 7fe76b7fe640 1 -- 172.21.15.1:0/1678077638 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe76c00f6b0 con 0x7fe774106a80 2024-09-11T21:57:03.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.467+0000 7fe77c377640 1 -- 172.21.15.1:0/1678077638 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe77419d7c0 con 0x7fe774106a80 2024-09-11T21:57:03.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.467+0000 7fe77c377640 1 -- 172.21.15.1:0/1678077638 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7741a50a0 con 0x7fe774106a80 2024-09-11T21:57:03.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.470+0000 7fe76b7fe640 1 -- 172.21.15.1:0/1678077638 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fe76c005560 con 0x7fe774106a80 2024-09-11T21:57:03.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.470+0000 7fe77e750740 1 Processor -- start 2024-09-11T21:57:03.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.471+0000 7fe77e750740 1 -- start start 2024-09-11T21:57:03.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.471+0000 7fe77e750740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644419e29d0 0x5644419e2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:03.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.471+0000 7fe77e750740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644419e32f0 0x5644419ed670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:03.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.471+0000 7fe77e750740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644419edbb0 0x5644419effa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:03.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.471+0000 7fe77e750740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x564441949090 con 0x5644419e29d0 2024-09-11T21:57:03.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.471+0000 7fe77e750740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5644418f0a70 con 0x5644419e32f0 2024-09-11T21:57:03.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.471+0000 7fe77e750740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x564441945050 con 0x5644419edbb0 2024-09-11T21:57:03.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.471+0000 7fe779912640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644419e29d0 0x5644419e2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:03.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.471+0000 7fe779912640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644419e29d0 0x5644419e2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55010/0 (socket says 172.21.15.1:55010) 2024-09-11T21:57:03.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.471+0000 7fe779912640 1 -- 172.21.15.1:0/2667258588 learned_addr learned my addr 172.21.15.1:0/2667258588 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:03.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.471+0000 7fe77a113640 1 --2- 172.21.15.1:0/2667258588 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644419e32f0 0x5644419ed670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:03.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.471+0000 7fe779912640 1 -- 172.21.15.1:0/2667258588 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644419edbb0 msgr2=0x5644419effa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:03.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.471+0000 7fe779912640 1 --2- 172.21.15.1:0/2667258588 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644419edbb0 0x5644419effa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:03.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.471+0000 7fe779912640 1 -- 172.21.15.1:0/2667258588 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644419e32f0 msgr2=0x5644419ed670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:03.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.471+0000 7fe779912640 1 --2- 172.21.15.1:0/2667258588 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644419e32f0 0x5644419ed670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:03.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.471+0000 7fe76b7fe640 1 --2- 172.21.15.1:0/1678077638 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fe7600a2c00 0x7fe7600a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:03.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.471+0000 7fe779912640 1 -- 172.21.15.1:0/2667258588 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644418fd130 con 0x5644419e29d0 2024-09-11T21:57:03.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.471+0000 7fe77a113640 1 --2- 172.21.15.1:0/2667258588 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644419e32f0 0x5644419ed670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:57:03.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.471+0000 7fe77a914640 1 --2- 172.21.15.1:0/1678077638 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fe7600a2c00 0x7fe7600a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:03.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.471+0000 7fe779912640 1 --2- 172.21.15.1:0/2667258588 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644419e29d0 0x5644419e2db0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe764006aa0 tx=0x7fe764009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7579ed1359b9441 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:03.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.471+0000 7fe76b7fe640 1 -- 172.21.15.1:0/1678077638 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fe76c0cf010 con 0x7fe774106a80 2024-09-11T21:57:03.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.472+0000 7fe768dea640 1 -- 172.21.15.1:0/2667258588 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe764018020 con 0x5644419e29d0 2024-09-11T21:57:03.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.472+0000 7fe768dea640 1 -- 172.21.15.1:0/2667258588 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe764010dc0 con 0x5644419e29d0 2024-09-11T21:57:03.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.472+0000 7fe768dea640 1 -- 172.21.15.1:0/2667258588 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe764013840 con 0x5644419e29d0 2024-09-11T21:57:03.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.472+0000 7fe77e750740 1 -- 172.21.15.1:0/2667258588 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644419e29d0 msgr2=0x5644419e2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:03.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.472+0000 7fe77e750740 1 --2- 172.21.15.1:0/2667258588 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644419e29d0 0x5644419e2db0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe764006aa0 tx=0x7fe764009f70 comp rx=0 tx=0).stop 2024-09-11T21:57:03.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.472+0000 7fe77a914640 1 --2- 172.21.15.1:0/1678077638 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fe7600a2c00 0x7fe7600a50c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe76c06e980 tx=0x7fe76c06e9b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:03.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.472+0000 7fe77e750740 1 -- 172.21.15.1:0/2667258588 shutdown_connections 2024-09-11T21:57:03.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.472+0000 7fe77e750740 1 --2- 172.21.15.1:0/2667258588 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644419edbb0 0x5644419effa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:03.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.472+0000 7fe77e750740 1 --2- 172.21.15.1:0/2667258588 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644419e32f0 0x5644419ed670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:03.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.472+0000 7fe77e750740 1 --2- 172.21.15.1:0/2667258588 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644419e29d0 0x5644419e2db0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:03.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.472+0000 7fe77e750740 1 -- 172.21.15.1:0/2667258588 >> 172.21.15.1:0/2667258588 conn(0x5644419def60 msgr2=0x5644419df340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:03.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.472+0000 7fe77e750740 1 -- 172.21.15.1:0/2667258588 shutdown_connections 2024-09-11T21:57:03.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.473+0000 7fe77e750740 1 -- 172.21.15.1:0/2667258588 wait complete. 2024-09-11T21:57:03.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.473+0000 7fe77e750740 1 -- start start 2024-09-11T21:57:03.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.473+0000 7fe77e750740 1 Processor -- start 2024-09-11T21:57:03.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.473+0000 7fe77e750740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644419e29d0 0x5644419ed410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:03.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.473+0000 7fe77e750740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644419e32f0 0x5644419ea760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:03.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.473+0000 7fe77e750740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644419edbb0 0x5644419eaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:03.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.473+0000 7fe77e750740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5644419f0780 con 0x5644419edbb0 2024-09-11T21:57:03.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.473+0000 7fe77e750740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x564441903f20 con 0x5644419e32f0 2024-09-11T21:57:03.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.473+0000 7fe77e750740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5644419f0900 con 0x5644419e29d0 2024-09-11T21:57:03.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.473+0000 7fe77e750740 10 client.0 _ll_register_callbacks cb 0x5644418cd030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:57:03.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.473+0000 7fe779912640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644419edbb0 0x5644419eaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:03.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.473+0000 7fe779912640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644419edbb0 0x5644419eaca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55020/0 (socket says 172.21.15.1:55020) 2024-09-11T21:57:03.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.473+0000 7fe779912640 1 -- 172.21.15.1:0/493013835 learned_addr learned my addr 172.21.15.1:0/493013835 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:03.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.473+0000 7fe77a113640 1 --2- 172.21.15.1:0/493013835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644419e32f0 0x5644419ea760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:03.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.474+0000 7fe779912640 1 -- 172.21.15.1:0/493013835 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644419e29d0 msgr2=0x5644419ed410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:03.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.474+0000 7fe779912640 1 --2- 172.21.15.1:0/493013835 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644419e29d0 0x5644419ed410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:03.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.474+0000 7fe779912640 1 -- 172.21.15.1:0/493013835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644419e32f0 msgr2=0x5644419ea760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:03.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.474+0000 7fe779912640 1 --2- 172.21.15.1:0/493013835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644419e32f0 0x5644419ea760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:03.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.474+0000 7fe779912640 1 -- 172.21.15.1:0/493013835 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644416ba2b0 con 0x5644419edbb0 2024-09-11T21:57:03.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.474+0000 7fe77a113640 1 --2- 172.21.15.1:0/493013835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644419e32f0 0x5644419ea760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T21:57:03.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.474+0000 7fe779912640 1 --2- 172.21.15.1:0/493013835 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644419edbb0 0x5644419eaca0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe7640139e0 tx=0x7fe764006ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3579942cac2ad95 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:03.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.474+0000 7fe75dffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T21:57:03.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.474+0000 7fe75dffb640 1 -- 172.21.15.1:0/493013835 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe764018030 con 0x5644419edbb0 2024-09-11T21:57:03.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.474+0000 7fe75dffb640 1 -- 172.21.15.1:0/493013835 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fe764014440 con 0x5644419edbb0 2024-09-11T21:57:03.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.474+0000 7fe77e750740 1 -- 172.21.15.1:0/493013835 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe764010dc0 con 0x5644419edbb0 2024-09-11T21:57:03.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.474+0000 7fe75dffb640 1 -- 172.21.15.1:0/493013835 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe764013ea0 con 0x5644419edbb0 2024-09-11T21:57:03.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.474+0000 7fe77e750740 10 client.15144 fetch_fsmap learned FSMap version 27 2024-09-11T21:57:03.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.474+0000 7fe77e750740 1 -- 172.21.15.1:0/493013835 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5644419fc070 con 0x5644419edbb0 2024-09-11T21:57:03.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.474+0000 7fe75dffb640 1 -- 172.21.15.1:0/493013835 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe764010dc0 con 0x5644419edbb0 2024-09-11T21:57:03.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.474+0000 7fe75dffb640 1 -- 172.21.15.1:0/493013835 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe76401da60 con 0x5644419edbb0 2024-09-11T21:57:03.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.475+0000 7fe77e750740 10 client.15144 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T21:57:03.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.475+0000 7fe77e750740 10 client.15144 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-11T21:57:03.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.475+0000 7fe77e750740 20 client.15144 populate_metadata read hostname 'smithi001' 2024-09-11T21:57:03.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.475+0000 7fe77e750740 1 --2- 172.21.15.1:0/493013835 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x5644419fe200 0x5644419ff670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:03.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.475+0000 7fe77e750740 4 client.15144 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T21:57:03.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.475+0000 7fe77e750740 1 -- 172.21.15.1:0/493013835 --> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564441956600 con 0x5644419fe200 2024-09-11T21:57:03.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.475+0000 7fe77a113640 1 --2- 172.21.15.1:0/493013835 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x5644419fe200 0x5644419ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:03.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.476+0000 7fe77a113640 1 --2- 172.21.15.1:0/493013835 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x5644419fe200 0x5644419ff670 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cf0eb8be62ce4772 server_cookie=ad04c6cff520a6ca in_seq=0 out_seq=0 2024-09-11T21:57:03.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.476+0000 7fe75dffb640 10 client.15144 ms_handle_connect on v2:172.21.15.53:6836/3457131830 2024-09-11T21:57:03.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.476+0000 7fe75dffb640 1 -- 172.21.15.1:0/493013835 <== mds.2 v2:172.21.15.53:6836/3457131830 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564441956600 con 0x5644419fe200 2024-09-11T21:57:03.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.476+0000 7fe75dffb640 10 client.15144 handle_command_reply: tid=0 2024-09-11T21:57:03.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.476+0000 7fe77e750740 1 client.15144 shutdown 2024-09-11T21:57:03.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.476+0000 7fe77e750740 1 -- 172.21.15.1:0/493013835 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644419edbb0 msgr2=0x5644419eaca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:03.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.476+0000 7fe77e750740 1 --2- 172.21.15.1:0/493013835 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644419edbb0 0x5644419eaca0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe7640139e0 tx=0x7fe764006ec0 comp rx=0 tx=0).stop 2024-09-11T21:57:03.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.476+0000 7fe77e750740 1 -- 172.21.15.1:0/493013835 shutdown_connections 2024-09-11T21:57:03.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.476+0000 7fe77e750740 1 --2- 172.21.15.1:0/493013835 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x5644419fe200 0x5644419ff670 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:03.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.476+0000 7fe77e750740 1 --2- 172.21.15.1:0/493013835 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644419edbb0 0x5644419eaca0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:03.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.476+0000 7fe77e750740 1 --2- 172.21.15.1:0/493013835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644419e32f0 0x5644419ea760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:03.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.476+0000 7fe75dffb640 0 client.15144 ms_handle_reset on v2:172.21.15.53:6836/3457131830 2024-09-11T21:57:03.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.476+0000 7fe77e750740 1 --2- 172.21.15.1:0/493013835 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644419e29d0 0x5644419ed410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:03.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.476+0000 7fe77e750740 1 -- 172.21.15.1:0/493013835 >> 172.21.15.1:0/493013835 conn(0x5644419def60 msgr2=0x5644419e4c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:03.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.477+0000 7fe77e750740 1 -- 172.21.15.1:0/493013835 shutdown_connections 2024-09-11T21:57:03.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.477+0000 7fe77e750740 1 -- 172.21.15.1:0/493013835 wait complete. 2024-09-11T21:57:03.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.477+0000 7fe77e750740 20 client.15144 trim_cache size 0 max 16384 2024-09-11T21:57:03.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.495+0000 7fe77e750740 1 Processor -- start 2024-09-11T21:57:03.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.495+0000 7fe77e750740 1 -- start start 2024-09-11T21:57:03.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.495+0000 7fe77e750740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644419e29d0 0x5644419ff670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:03.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.495+0000 7fe77e750740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644419e32f0 0x564441a07d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:03.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.495+0000 7fe77e750740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644419edbb0 0x5644419e9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:03.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.495+0000 7fe77e750740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5644419fd9b0 con 0x5644419edbb0 2024-09-11T21:57:03.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.495+0000 7fe77e750740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5644419fe080 con 0x5644419e32f0 2024-09-11T21:57:03.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.495+0000 7fe77e750740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5644419fe750 con 0x5644419e29d0 2024-09-11T21:57:03.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.495+0000 7fe779912640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644419edbb0 0x5644419e9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:03.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.495+0000 7fe779912640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644419edbb0 0x5644419e9030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55036/0 (socket says 172.21.15.1:55036) 2024-09-11T21:57:03.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.495+0000 7fe779912640 1 -- 172.21.15.1:0/1806755410 learned_addr learned my addr 172.21.15.1:0/1806755410 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:03.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.495+0000 7fe779912640 1 --2- 172.21.15.1:0/1806755410 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644419e29d0 0x5644419ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:03.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.495+0000 7fe77a113640 1 --2- 172.21.15.1:0/1806755410 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644419e32f0 0x564441a07d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:03.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.495+0000 7fe779912640 1 -- 172.21.15.1:0/1806755410 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644419e29d0 msgr2=0x5644419ff670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:03.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.495+0000 7fe779912640 1 --2- 172.21.15.1:0/1806755410 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644419e29d0 0x5644419ff670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:03.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.495+0000 7fe779912640 1 -- 172.21.15.1:0/1806755410 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644419e32f0 msgr2=0x564441a07d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:03.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.495+0000 7fe779912640 1 --2- 172.21.15.1:0/1806755410 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644419e32f0 0x564441a07d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:03.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.495+0000 7fe779912640 1 -- 172.21.15.1:0/1806755410 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644419de9e0 con 0x5644419edbb0 2024-09-11T21:57:03.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.495+0000 7fe779912640 1 --2- 172.21.15.1:0/1806755410 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644419edbb0 0x5644419e9030 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fe764001eb0 tx=0x7fe764016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4969425f15651e5c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:03.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.496+0000 7fe75ffff640 1 -- 172.21.15.1:0/1806755410 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe764018040 con 0x5644419edbb0 2024-09-11T21:57:03.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.496+0000 7fe75ffff640 1 -- 172.21.15.1:0/1806755410 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe764033020 con 0x5644419edbb0 2024-09-11T21:57:03.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.496+0000 7fe75ffff640 1 -- 172.21.15.1:0/1806755410 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe76401d7a0 con 0x5644419edbb0 2024-09-11T21:57:03.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.496+0000 7fe77e750740 1 -- 172.21.15.1:0/1806755410 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644419edbb0 msgr2=0x5644419e9030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:03.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.496+0000 7fe77e750740 1 --2- 172.21.15.1:0/1806755410 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644419edbb0 0x5644419e9030 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fe764001eb0 tx=0x7fe764016040 comp rx=0 tx=0).stop 2024-09-11T21:57:03.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.496+0000 7fe77e750740 1 -- 172.21.15.1:0/1806755410 shutdown_connections 2024-09-11T21:57:03.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.496+0000 7fe77e750740 1 --2- 172.21.15.1:0/1806755410 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644419edbb0 0x5644419e9030 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:03.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.496+0000 7fe77e750740 1 --2- 172.21.15.1:0/1806755410 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644419e32f0 0x564441a07d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:03.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.496+0000 7fe77e750740 1 --2- 172.21.15.1:0/1806755410 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644419e29d0 0x5644419ff670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:03.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.496+0000 7fe77e750740 1 -- 172.21.15.1:0/1806755410 >> 172.21.15.1:0/1806755410 conn(0x5644419def60 msgr2=0x5644419df340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:03.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.496+0000 7fe77e750740 1 -- 172.21.15.1:0/1806755410 shutdown_connections 2024-09-11T21:57:03.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.496+0000 7fe77e750740 1 -- 172.21.15.1:0/1806755410 wait complete. 2024-09-11T21:57:03.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.496+0000 7fe77e750740 1 -- start start 2024-09-11T21:57:03.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.497+0000 7fe77e750740 1 Processor -- start 2024-09-11T21:57:03.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.497+0000 7fe77e750740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644419e29d0 0x5644419e7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:03.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.497+0000 7fe77e750740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644419e32f0 0x5644419e7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:03.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.497+0000 7fe77e750740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644419edbb0 0x564441a639e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:03.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.497+0000 7fe77e750740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5644419ff4f0 con 0x5644419e32f0 2024-09-11T21:57:03.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.497+0000 7fe77e750740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5644419fee20 con 0x5644419edbb0 2024-09-11T21:57:03.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.497+0000 7fe77e750740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5644419ffbc0 con 0x5644419e29d0 2024-09-11T21:57:03.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.497+0000 7fe77e750740 10 client.0 _ll_register_callbacks cb 0x7fe7640099e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:57:03.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.497+0000 7fe77a113640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644419e32f0 0x5644419e7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:03.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.497+0000 7fe77a113640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644419e32f0 0x5644419e7a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55038/0 (socket says 172.21.15.1:55038) 2024-09-11T21:57:03.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.497+0000 7fe77a113640 1 -- 172.21.15.1:0/406859492 learned_addr learned my addr 172.21.15.1:0/406859492 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:03.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.497+0000 7fe779912640 1 --2- 172.21.15.1:0/406859492 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644419e29d0 0x5644419e7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:03.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.497+0000 7fe779912640 1 --2- 172.21.15.1:0/406859492 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644419edbb0 0x564441a639e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:03.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.497+0000 7fe77a113640 1 -- 172.21.15.1:0/406859492 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644419e29d0 msgr2=0x5644419e7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:03.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.497+0000 7fe77a113640 1 --2- 172.21.15.1:0/406859492 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644419e29d0 0x5644419e7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:03.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.497+0000 7fe77a113640 1 -- 172.21.15.1:0/406859492 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644419edbb0 msgr2=0x564441a639e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:03.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.497+0000 7fe77a113640 1 --2- 172.21.15.1:0/406859492 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644419edbb0 0x564441a639e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:03.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.497+0000 7fe77a113640 1 -- 172.21.15.1:0/406859492 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564441a64080 con 0x5644419e32f0 2024-09-11T21:57:03.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.497+0000 7fe779912640 1 --2- 172.21.15.1:0/406859492 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644419e29d0 0x5644419e7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:57:03.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.497+0000 7fe77a113640 1 --2- 172.21.15.1:0/406859492 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644419e32f0 0x5644419e7a60 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe77002a940 tx=0x7fe7700314f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0c222baefe67d8e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:03.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.497+0000 7fe75dffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T21:57:03.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.498+0000 7fe75dffb640 1 -- 172.21.15.1:0/406859492 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe770048070 con 0x5644419e32f0 2024-09-11T21:57:03.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.498+0000 7fe75dffb640 1 -- 172.21.15.1:0/406859492 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fe770004450 con 0x5644419e32f0 2024-09-11T21:57:03.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.498+0000 7fe77e750740 1 -- 172.21.15.1:0/406859492 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe764033020 con 0x5644419e32f0 2024-09-11T21:57:03.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.498+0000 7fe75dffb640 1 -- 172.21.15.1:0/406859492 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe770053050 con 0x5644419e32f0 2024-09-11T21:57:03.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.498+0000 7fe77e750740 10 client.15150 fetch_fsmap learned FSMap version 27 2024-09-11T21:57:03.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.498+0000 7fe77e750740 1 -- 172.21.15.1:0/406859492 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564441a6c650 con 0x5644419e32f0 2024-09-11T21:57:03.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.498+0000 7fe75dffb640 1 -- 172.21.15.1:0/406859492 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe764033020 con 0x5644419e32f0 2024-09-11T21:57:03.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.498+0000 7fe75dffb640 1 -- 172.21.15.1:0/406859492 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe770044490 con 0x5644419e32f0 2024-09-11T21:57:03.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.498+0000 7fe77e750740 10 client.15150 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T21:57:03.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.498+0000 7fe77e750740 10 client.15150 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-11T21:57:03.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.498+0000 7fe77e750740 20 client.15150 populate_metadata read hostname 'smithi001' 2024-09-11T21:57:03.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.498+0000 7fe77e750740 1 --2- 172.21.15.1:0/406859492 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x564441a71220 0x564441a6ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:03.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.498+0000 7fe77e750740 4 client.15150 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T21:57:03.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.498+0000 7fe77e750740 1 -- 172.21.15.1:0/406859492 --> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] -- command(tid 0: {"prefix": "status"}) -- 0x564441950b10 con 0x564441a71220 2024-09-11T21:57:03.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.499+0000 7fe779912640 1 --2- 172.21.15.1:0/406859492 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x564441a71220 0x564441a6ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:03.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.499+0000 7fe75dffb640 10 client.15150 ms_handle_connect on v2:172.21.15.53:6836/3457131830 2024-09-11T21:57:03.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.499+0000 7fe779912640 1 --2- 172.21.15.1:0/406859492 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x564441a71220 0x564441a6ff30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3da1b99f12bd0920 server_cookie=2ead54b83078ef8 in_seq=0 out_seq=0 2024-09-11T21:57:03.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.499+0000 7fe75dffb640 1 -- 172.21.15.1:0/406859492 <== mds.2 v2:172.21.15.53:6836/3457131830 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x564441950b10 con 0x564441a71220 2024-09-11T21:57:03.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.499+0000 7fe75dffb640 10 client.15150 handle_command_reply: tid=0 2024-09-11T21:57:03.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.499+0000 7fe77e750740 1 client.15150 shutdown 2024-09-11T21:57:03.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.499+0000 7fe77e750740 1 -- 172.21.15.1:0/406859492 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644419e32f0 msgr2=0x5644419e7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:03.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.499+0000 7fe77e750740 1 --2- 172.21.15.1:0/406859492 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644419e32f0 0x5644419e7a60 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe77002a940 tx=0x7fe7700314f0 comp rx=0 tx=0).stop 2024-09-11T21:57:03.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.499+0000 7fe77e750740 1 -- 172.21.15.1:0/406859492 shutdown_connections 2024-09-11T21:57:03.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.499+0000 7fe77e750740 1 --2- 172.21.15.1:0/406859492 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x564441a71220 0x564441a6ff30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:03.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.499+0000 7fe77e750740 1 --2- 172.21.15.1:0/406859492 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5644419edbb0 0x564441a639e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:03.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.499+0000 7fe77e750740 1 --2- 172.21.15.1:0/406859492 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5644419e32f0 0x5644419e7a60 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:03.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.499+0000 7fe77e750740 1 --2- 172.21.15.1:0/406859492 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5644419e29d0 0x5644419e7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:03.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.499+0000 7fe77e750740 1 -- 172.21.15.1:0/406859492 >> 172.21.15.1:0/406859492 conn(0x5644419def60 msgr2=0x5644419e8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:03.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.499+0000 7fe75dffb640 0 client.15150 ms_handle_reset on v2:172.21.15.53:6836/3457131830 2024-09-11T21:57:03.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.500+0000 7fe77e750740 1 -- 172.21.15.1:0/406859492 shutdown_connections 2024-09-11T21:57:03.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.500+0000 7fe77e750740 1 -- 172.21.15.1:0/406859492 wait complete. 2024-09-11T21:57:03.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.500+0000 7fe77e750740 20 client.15150 trim_cache size 0 max 16384 2024-09-11T21:57:03.515 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-11T21:57:03.515 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "3ffca83c-7087-11ef-bcea-c7b262605968", 2024-09-11T21:57:03.515 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 2, 2024-09-11T21:57:03.515 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 24467, 2024-09-11T21:57:03.515 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-09-11T21:57:03.515 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-09-11T21:57:03.515 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-09-11T21:57:03.515 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 94.370847050999998, 2024-09-11T21:57:03.515 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 27, 2024-09-11T21:57:03.515 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 86, 2024-09-11T21:57:03.515 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 84, 2024-09-11T21:57:03.515 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 124.36024914799999 2024-09-11T21:57:03.516 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-11T21:57:03.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.500+0000 7fe77c377640 1 -- 172.21.15.1:0/1678077638 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fe7600a2c00 msgr2=0x7fe7600a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:03.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.500+0000 7fe77c377640 1 --2- 172.21.15.1:0/1678077638 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fe7600a2c00 0x7fe7600a50c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe76c06e980 tx=0x7fe76c06e9b0 comp rx=0 tx=0).stop 2024-09-11T21:57:03.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.500+0000 7fe77c377640 1 -- 172.21.15.1:0/1678077638 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe774106a80 msgr2=0x7fe77419cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:03.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.500+0000 7fe77c377640 1 --2- 172.21.15.1:0/1678077638 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe774106a80 0x7fe77419cdf0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe76c00cad0 tx=0x7fe76c00ced0 comp rx=0 tx=0).stop 2024-09-11T21:57:03.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.500+0000 7fe77c377640 1 -- 172.21.15.1:0/1678077638 shutdown_connections 2024-09-11T21:57:03.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.500+0000 7fe77c377640 1 --2- 172.21.15.1:0/1678077638 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fe7600a2c00 0x7fe7600a50c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:03.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.500+0000 7fe77c377640 1 --2- 172.21.15.1:0/1678077638 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe774106a80 0x7fe77419cdf0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:03.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.500+0000 7fe77c377640 1 --2- 172.21.15.1:0/1678077638 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe774101420 0x7fe774198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:03.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.500+0000 7fe77c377640 1 --2- 172.21.15.1:0/1678077638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe774100a80 0x7fe774198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:03.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.500+0000 7fe77c377640 1 -- 172.21.15.1:0/1678077638 >> 172.21.15.1:0/1678077638 conn(0x7fe7740fc7a0 msgr2=0x7fe77406aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:03.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.500+0000 7fe77c377640 1 -- 172.21.15.1:0/1678077638 shutdown_connections 2024-09-11T21:57:03.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:03.500+0000 7fe77c377640 1 -- 172.21.15.1:0/1678077638 wait complete. 2024-09-11T21:57:04.346 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-11T21:57:04.607 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:57:04.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:04 smithi001 ceph-mon[30935]: pgmap v335: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-11T21:57:04.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:04 smithi053 ceph-mon[34908]: pgmap v335: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-11T21:57:04.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:04 smithi161 ceph-mon[35615]: pgmap v335: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-11T21:57:05.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.090+0000 7f46c35a2640 1 -- 172.21.15.1:0/2736671855 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46bc1008d0 msgr2=0x7f46bc100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:05.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.090+0000 7f46c35a2640 1 --2- 172.21.15.1:0/2736671855 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46bc1008d0 0x7f46bc100cb0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f46b8009900 tx=0x7f46b802a410 comp rx=0 tx=0).stop 2024-09-11T21:57:05.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.090+0000 7f46c35a2640 1 -- 172.21.15.1:0/2736671855 shutdown_connections 2024-09-11T21:57:05.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.090+0000 7f46c35a2640 1 --2- 172.21.15.1:0/2736671855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f46bc10e480 0x7f46bc110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:05.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.090+0000 7f46c35a2640 1 --2- 172.21.15.1:0/2736671855 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46bc101280 0x7f46bc10df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:05.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.090+0000 7f46c35a2640 1 --2- 172.21.15.1:0/2736671855 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46bc1008d0 0x7f46bc100cb0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:05.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.090+0000 7f46c35a2640 1 -- 172.21.15.1:0/2736671855 >> 172.21.15.1:0/2736671855 conn(0x7f46bc0fc7a0 msgr2=0x7f46bc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:05.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.090+0000 7f46c35a2640 1 -- 172.21.15.1:0/2736671855 shutdown_connections 2024-09-11T21:57:05.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.090+0000 7f46c35a2640 1 -- 172.21.15.1:0/2736671855 wait complete. 2024-09-11T21:57:05.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.091+0000 7f46c35a2640 1 Processor -- start 2024-09-11T21:57:05.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.091+0000 7f46c35a2640 1 -- start start 2024-09-11T21:57:05.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.091+0000 7f46c35a2640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46bc1008d0 0x7f46bc1a0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:05.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.091+0000 7f46c35a2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46bc101280 0x7f46bc1a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:05.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.091+0000 7f46c35a2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f46bc10e480 0x7f46bc19a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:05.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.091+0000 7f46c35a2640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f46bc114260 con 0x7f46bc101280 2024-09-11T21:57:05.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.091+0000 7f46c35a2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f46bc1140e0 con 0x7f46bc10e480 2024-09-11T21:57:05.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.091+0000 7f46c35a2640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f46bc1143e0 con 0x7f46bc1008d0 2024-09-11T21:57:05.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.091+0000 7f46c0b3d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46bc101280 0x7f46bc1a0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:05.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.091+0000 7f46c0b3d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46bc101280 0x7f46bc1a0ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55052/0 (socket says 172.21.15.1:55052) 2024-09-11T21:57:05.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.091+0000 7f46c0b3d640 1 -- 172.21.15.1:0/2778167025 learned_addr learned my addr 172.21.15.1:0/2778167025 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:05.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.092+0000 7f46c133e640 1 --2- 172.21.15.1:0/2778167025 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46bc1008d0 0x7f46bc1a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:05.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.092+0000 7f46c1b3f640 1 --2- 172.21.15.1:0/2778167025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f46bc10e480 0x7f46bc19a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:05.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.092+0000 7f46c0b3d640 1 -- 172.21.15.1:0/2778167025 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46bc1008d0 msgr2=0x7f46bc1a0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:05.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.092+0000 7f46c0b3d640 1 --2- 172.21.15.1:0/2778167025 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46bc1008d0 0x7f46bc1a0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:05.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.092+0000 7f46c0b3d640 1 -- 172.21.15.1:0/2778167025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f46bc10e480 msgr2=0x7f46bc19a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:05.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.092+0000 7f46c0b3d640 1 --2- 172.21.15.1:0/2778167025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f46bc10e480 0x7f46bc19a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:05.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.092+0000 7f46c0b3d640 1 -- 172.21.15.1:0/2778167025 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46bc19aec0 con 0x7f46bc101280 2024-09-11T21:57:05.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.092+0000 7f46c0b3d640 1 --2- 172.21.15.1:0/2778167025 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46bc101280 0x7f46bc1a0ac0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f46ac00b4a0 tx=0x7f46ac00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:05.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.092+0000 7f46b27fc640 1 -- 172.21.15.1:0/2778167025 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ac014070 con 0x7f46bc101280 2024-09-11T21:57:05.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.092+0000 7f46c35a2640 1 -- 172.21.15.1:0/2778167025 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46bc19b1b0 con 0x7f46bc101280 2024-09-11T21:57:05.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.092+0000 7f46b27fc640 1 -- 172.21.15.1:0/2778167025 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46ac010af0 con 0x7f46bc101280 2024-09-11T21:57:05.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.092+0000 7f46b27fc640 1 -- 172.21.15.1:0/2778167025 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ac017d10 con 0x7f46bc101280 2024-09-11T21:57:05.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.092+0000 7f46c35a2640 1 -- 172.21.15.1:0/2778167025 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46bc1a7500 con 0x7f46bc101280 2024-09-11T21:57:05.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.094+0000 7f46b27fc640 1 -- 172.21.15.1:0/2778167025 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f46ac017070 con 0x7f46bc101280 2024-09-11T21:57:05.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.095+0000 7f46c597b740 1 Processor -- start 2024-09-11T21:57:05.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.095+0000 7f46c597b740 1 -- start start 2024-09-11T21:57:05.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.095+0000 7f46c597b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628896809d0 0x562889680db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:05.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.095+0000 7f46c597b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628896812f0 0x56288968b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:05.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.095+0000 7f46c597b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56288968bbb0 0x56288968dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:05.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.095+0000 7f46c597b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5628895e7090 con 0x56288968bbb0 2024-09-11T21:57:05.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.095+0000 7f46c597b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56288958ea70 con 0x5628896809d0 2024-09-11T21:57:05.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.095+0000 7f46c597b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5628895e3050 con 0x5628896812f0 2024-09-11T21:57:05.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.095+0000 7f46c0b3d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56288968bbb0 0x56288968dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:05.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.095+0000 7f46c1b3f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628896809d0 0x562889680db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:05.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.095+0000 7f46c0b3d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56288968bbb0 0x56288968dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55054/0 (socket says 172.21.15.1:55054) 2024-09-11T21:57:05.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.095+0000 7f46b27fc640 1 --2- 172.21.15.1:0/2778167025 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f46a80a2b80 0x7f46a80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:05.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.095+0000 7f46c0b3d640 1 -- 172.21.15.1:0/1648905473 learned_addr learned my addr 172.21.15.1:0/1648905473 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:05.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.095+0000 7f46c1b3f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628896809d0 0x562889680db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:44046/0 (socket says 172.21.15.1:44046) 2024-09-11T21:57:05.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.095+0000 7f46c133e640 1 --2- 172.21.15.1:0/1648905473 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628896812f0 0x56288968b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:05.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.096+0000 7f46c1b3f640 1 --2- 172.21.15.1:0/2778167025 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f46a80a2b80 0x7f46a80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:05.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.096+0000 7f46b27fc640 1 -- 172.21.15.1:0/2778167025 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f46ac0ce4e0 con 0x7f46bc101280 2024-09-11T21:57:05.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.096+0000 7f46c0b3d640 1 -- 172.21.15.1:0/1648905473 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628896812f0 msgr2=0x56288968b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:05.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.096+0000 7f46c0b3d640 1 --2- 172.21.15.1:0/1648905473 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628896812f0 0x56288968b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:05.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.096+0000 7f46c0b3d640 1 -- 172.21.15.1:0/1648905473 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628896809d0 msgr2=0x562889680db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:05.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.096+0000 7f46c0b3d640 1 --2- 172.21.15.1:0/1648905473 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628896809d0 0x562889680db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:05.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.096+0000 7f46c0b3d640 1 -- 172.21.15.1:0/1648905473 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56288959b130 con 0x56288968bbb0 2024-09-11T21:57:05.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.096+0000 7f46c1b3f640 1 --2- 172.21.15.1:0/1648905473 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628896809d0 0x562889680db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T21:57:05.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.096+0000 7f46c0b3d640 1 --2- 172.21.15.1:0/1648905473 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56288968bbb0 0x56288968dfa0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f46ac097180 tx=0x7f46ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4c701465730dca0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:05.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.096+0000 7f46a77fe640 1 -- 172.21.15.1:0/1648905473 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ac01e070 con 0x56288968bbb0 2024-09-11T21:57:05.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.096+0000 7f46c1b3f640 1 --2- 172.21.15.1:0/2778167025 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f46a80a2b80 0x7f46a80a5040 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f46b4009fd0 tx=0x7f46b4004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:05.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.096+0000 7f46a77fe640 1 -- 172.21.15.1:0/1648905473 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46ac09b420 con 0x56288968bbb0 2024-09-11T21:57:05.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.096+0000 7f46a77fe640 1 -- 172.21.15.1:0/1648905473 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ac097390 con 0x56288968bbb0 2024-09-11T21:57:05.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.097+0000 7f46c597b740 1 -- 172.21.15.1:0/1648905473 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56288968bbb0 msgr2=0x56288968dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:05.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.097+0000 7f46c597b740 1 --2- 172.21.15.1:0/1648905473 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56288968bbb0 0x56288968dfa0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f46ac097180 tx=0x7f46ac0046c0 comp rx=0 tx=0).stop 2024-09-11T21:57:05.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.097+0000 7f46c597b740 1 -- 172.21.15.1:0/1648905473 shutdown_connections 2024-09-11T21:57:05.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.097+0000 7f46c597b740 1 --2- 172.21.15.1:0/1648905473 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56288968bbb0 0x56288968dfa0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:05.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.097+0000 7f46c597b740 1 --2- 172.21.15.1:0/1648905473 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628896812f0 0x56288968b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:05.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.097+0000 7f46c597b740 1 --2- 172.21.15.1:0/1648905473 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628896809d0 0x562889680db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:05.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.097+0000 7f46c597b740 1 -- 172.21.15.1:0/1648905473 >> 172.21.15.1:0/1648905473 conn(0x56288967cf60 msgr2=0x56288967d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:05.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.097+0000 7f46c597b740 1 -- 172.21.15.1:0/1648905473 shutdown_connections 2024-09-11T21:57:05.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.097+0000 7f46c597b740 1 -- 172.21.15.1:0/1648905473 wait complete. 2024-09-11T21:57:05.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.097+0000 7f46c597b740 1 -- start start 2024-09-11T21:57:05.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.098+0000 7f46c597b740 1 Processor -- start 2024-09-11T21:57:05.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.098+0000 7f46c597b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628896809d0 0x56288968b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:05.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.098+0000 7f46c597b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628896812f0 0x562889688780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:05.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.098+0000 7f46c597b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56288968bbb0 0x562889688cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:05.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.098+0000 7f46c597b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56288968e720 con 0x56288968bbb0 2024-09-11T21:57:05.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.098+0000 7f46c597b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5628895a1f20 con 0x5628896809d0 2024-09-11T21:57:05.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.098+0000 7f46c597b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56288968e8a0 con 0x5628896812f0 2024-09-11T21:57:05.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.098+0000 7f46c597b740 10 client.0 _ll_register_callbacks cb 0x56288956b030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:57:05.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.098+0000 7f46c1b3f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56288968bbb0 0x562889688cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:05.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.098+0000 7f46c1b3f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56288968bbb0 0x562889688cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55060/0 (socket says 172.21.15.1:55060) 2024-09-11T21:57:05.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.098+0000 7f46c1b3f640 1 -- 172.21.15.1:0/1878286967 learned_addr learned my addr 172.21.15.1:0/1878286967 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:05.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.098+0000 7f46c133e640 1 --2- 172.21.15.1:0/1878286967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628896809d0 0x56288968b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:05.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.098+0000 7f46c0b3d640 1 --2- 172.21.15.1:0/1878286967 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628896812f0 0x562889688780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:05.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.098+0000 7f46c1b3f640 1 -- 172.21.15.1:0/1878286967 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628896812f0 msgr2=0x562889688780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:05.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.098+0000 7f46c1b3f640 1 --2- 172.21.15.1:0/1878286967 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628896812f0 0x562889688780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:05.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.098+0000 7f46c1b3f640 1 -- 172.21.15.1:0/1878286967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628896809d0 msgr2=0x56288968b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:05.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.098+0000 7f46c1b3f640 1 --2- 172.21.15.1:0/1878286967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628896809d0 0x56288968b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:05.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.098+0000 7f46c1b3f640 1 -- 172.21.15.1:0/1878286967 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628893582b0 con 0x56288968bbb0 2024-09-11T21:57:05.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.099+0000 7f46c1b3f640 1 --2- 172.21.15.1:0/1878286967 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56288968bbb0 0x562889688cc0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f46b4007af0 tx=0x7f46b4004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea6dc68c70590cb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:05.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.099+0000 7f46a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T21:57:05.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.099+0000 7f46a4ff9640 1 -- 172.21.15.1:0/1878286967 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b4020070 con 0x56288968bbb0 2024-09-11T21:57:05.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.099+0000 7f46a4ff9640 1 -- 172.21.15.1:0/1878286967 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f46b4025460 con 0x56288968bbb0 2024-09-11T21:57:05.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.099+0000 7f46c597b740 1 -- 172.21.15.1:0/1878286967 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46ac09b420 con 0x56288968bbb0 2024-09-11T21:57:05.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.099+0000 7f46a4ff9640 1 -- 172.21.15.1:0/1878286967 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f46b4015050 con 0x56288968bbb0 2024-09-11T21:57:05.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.099+0000 7f46c597b740 10 client.15162 fetch_fsmap learned FSMap version 27 2024-09-11T21:57:05.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.099+0000 7f46c597b740 1 -- 172.21.15.1:0/1878286967 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56288969a090 con 0x56288968bbb0 2024-09-11T21:57:05.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.099+0000 7f46a4ff9640 1 -- 172.21.15.1:0/1878286967 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46ac09b420 con 0x56288968bbb0 2024-09-11T21:57:05.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.099+0000 7f46a4ff9640 1 -- 172.21.15.1:0/1878286967 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f46b401a390 con 0x56288968bbb0 2024-09-11T21:57:05.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.099+0000 7f46c597b740 10 client.15162 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T21:57:05.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.099+0000 7f46c597b740 10 client.15162 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-11T21:57:05.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.099+0000 7f46c597b740 20 client.15162 populate_metadata read hostname 'smithi001' 2024-09-11T21:57:05.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.099+0000 7f46c597b740 1 --2- 172.21.15.1:0/1878286967 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x56288969c6d0 0x56288969db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:05.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.099+0000 7f46c597b740 4 client.15162 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T21:57:05.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.099+0000 7f46c597b740 1 -- 172.21.15.1:0/1878286967 --> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5628895f4600 con 0x56288969c6d0 2024-09-11T21:57:05.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.100+0000 7f46c133e640 1 --2- 172.21.15.1:0/1878286967 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x56288969c6d0 0x56288969db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:05.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.100+0000 7f46c133e640 1 --2- 172.21.15.1:0/1878286967 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x56288969c6d0 0x56288969db40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4c29fc04801a7b86 server_cookie=949b5740728f9e1f in_seq=0 out_seq=0 2024-09-11T21:57:05.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.100+0000 7f46a4ff9640 10 client.15162 ms_handle_connect on v2:172.21.15.161:6838/455365383 2024-09-11T21:57:05.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.101+0000 7f46a4ff9640 1 -- 172.21.15.1:0/1878286967 <== mds.1 v2:172.21.15.161:6838/455365383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5628895f4600 con 0x56288969c6d0 2024-09-11T21:57:05.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.101+0000 7f46a4ff9640 10 client.15162 handle_command_reply: tid=0 2024-09-11T21:57:05.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.101+0000 7f46c597b740 1 client.15162 shutdown 2024-09-11T21:57:05.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.101+0000 7f46c597b740 1 -- 172.21.15.1:0/1878286967 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56288968bbb0 msgr2=0x562889688cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:05.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.101+0000 7f46c597b740 1 --2- 172.21.15.1:0/1878286967 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56288968bbb0 0x562889688cc0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f46b4007af0 tx=0x7f46b4004b50 comp rx=0 tx=0).stop 2024-09-11T21:57:05.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.101+0000 7f46c597b740 1 -- 172.21.15.1:0/1878286967 shutdown_connections 2024-09-11T21:57:05.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.101+0000 7f46c597b740 1 --2- 172.21.15.1:0/1878286967 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x56288969c6d0 0x56288969db40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:05.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.101+0000 7f46c597b740 1 --2- 172.21.15.1:0/1878286967 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56288968bbb0 0x562889688cc0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:05.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.101+0000 7f46c597b740 1 --2- 172.21.15.1:0/1878286967 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628896812f0 0x562889688780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:05.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.101+0000 7f46c597b740 1 --2- 172.21.15.1:0/1878286967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628896809d0 0x56288968b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:05.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.101+0000 7f46c597b740 1 -- 172.21.15.1:0/1878286967 >> 172.21.15.1:0/1878286967 conn(0x56288967cf60 msgr2=0x562889682c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:05.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.101+0000 7f46a4ff9640 0 client.15162 ms_handle_reset on v2:172.21.15.161:6838/455365383 2024-09-11T21:57:05.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.101+0000 7f46c597b740 1 -- 172.21.15.1:0/1878286967 shutdown_connections 2024-09-11T21:57:05.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.101+0000 7f46c597b740 1 -- 172.21.15.1:0/1878286967 wait complete. 2024-09-11T21:57:05.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.101+0000 7f46c597b740 20 client.15162 trim_cache size 0 max 16384 2024-09-11T21:57:05.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.119+0000 7f46c597b740 1 Processor -- start 2024-09-11T21:57:05.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.119+0000 7f46c597b740 1 -- start start 2024-09-11T21:57:05.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.120+0000 7f46c597b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628896809d0 0x562889687050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:05.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.120+0000 7f46c597b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628896812f0 0x562889687590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:05.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.120+0000 7f46c597b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56288968bbb0 0x5628896a5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:05.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.120+0000 7f46c597b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56288969be80 con 0x56288968bbb0 2024-09-11T21:57:05.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.120+0000 7f46c597b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56288969c550 con 0x5628896812f0 2024-09-11T21:57:05.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.120+0000 7f46c597b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56288969cc20 con 0x5628896809d0 2024-09-11T21:57:05.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.120+0000 7f46c1b3f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56288968bbb0 0x5628896a5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:05.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.120+0000 7f46c1b3f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56288968bbb0 0x5628896a5e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55066/0 (socket says 172.21.15.1:55066) 2024-09-11T21:57:05.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.120+0000 7f46c1b3f640 1 -- 172.21.15.1:0/684432075 learned_addr learned my addr 172.21.15.1:0/684432075 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:05.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.120+0000 7f46c133e640 1 --2- 172.21.15.1:0/684432075 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628896809d0 0x562889687050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:05.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.120+0000 7f46c0b3d640 1 --2- 172.21.15.1:0/684432075 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628896812f0 0x562889687590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:05.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.120+0000 7f46c1b3f640 1 -- 172.21.15.1:0/684432075 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628896809d0 msgr2=0x562889687050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:05.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.120+0000 7f46c1b3f640 1 --2- 172.21.15.1:0/684432075 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628896809d0 0x562889687050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:05.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.120+0000 7f46c1b3f640 1 -- 172.21.15.1:0/684432075 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628896812f0 msgr2=0x562889687590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:05.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.120+0000 7f46c1b3f640 1 --2- 172.21.15.1:0/684432075 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628896812f0 0x562889687590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:05.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.120+0000 7f46c1b3f640 1 -- 172.21.15.1:0/684432075 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562889689540 con 0x56288968bbb0 2024-09-11T21:57:05.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.120+0000 7f46c1b3f640 1 --2- 172.21.15.1:0/684432075 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56288968bbb0 0x5628896a5e20 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f46b400f040 tx=0x7f46b4009c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=339c075542db9774 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:05.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.121+0000 7f46a6ffd640 1 -- 172.21.15.1:0/684432075 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b4020040 con 0x56288968bbb0 2024-09-11T21:57:05.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.121+0000 7f46a6ffd640 1 -- 172.21.15.1:0/684432075 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46b4034070 con 0x56288968bbb0 2024-09-11T21:57:05.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.121+0000 7f46a6ffd640 1 -- 172.21.15.1:0/684432075 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b4039890 con 0x56288968bbb0 2024-09-11T21:57:05.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.121+0000 7f46c597b740 1 -- 172.21.15.1:0/684432075 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56288968bbb0 msgr2=0x5628896a5e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:05.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.121+0000 7f46c597b740 1 --2- 172.21.15.1:0/684432075 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56288968bbb0 0x5628896a5e20 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f46b400f040 tx=0x7f46b4009c70 comp rx=0 tx=0).stop 2024-09-11T21:57:05.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.121+0000 7f46c597b740 1 -- 172.21.15.1:0/684432075 shutdown_connections 2024-09-11T21:57:05.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.121+0000 7f46c597b740 1 --2- 172.21.15.1:0/684432075 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56288968bbb0 0x5628896a5e20 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:05.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.121+0000 7f46c597b740 1 --2- 172.21.15.1:0/684432075 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628896812f0 0x562889687590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:05.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.121+0000 7f46c597b740 1 --2- 172.21.15.1:0/684432075 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628896809d0 0x562889687050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:05.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.121+0000 7f46c597b740 1 -- 172.21.15.1:0/684432075 >> 172.21.15.1:0/684432075 conn(0x56288967cf60 msgr2=0x562889687cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:05.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.121+0000 7f46c597b740 1 -- 172.21.15.1:0/684432075 shutdown_connections 2024-09-11T21:57:05.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.121+0000 7f46c597b740 1 -- 172.21.15.1:0/684432075 wait complete. 2024-09-11T21:57:05.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.122+0000 7f46c597b740 1 -- start start 2024-09-11T21:57:05.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.122+0000 7f46c597b740 1 Processor -- start 2024-09-11T21:57:05.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.122+0000 7f46c597b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628896809d0 0x5628896a6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:05.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.122+0000 7f46c597b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628896812f0 0x56288969f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:05.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.122+0000 7f46c597b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56288968bbb0 0x56288969fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:05.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.122+0000 7f46c597b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56288969d9c0 con 0x56288968bbb0 2024-09-11T21:57:05.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.122+0000 7f46c597b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56288969d2f0 con 0x5628896809d0 2024-09-11T21:57:05.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.122+0000 7f46c597b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56288969e090 con 0x5628896812f0 2024-09-11T21:57:05.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.122+0000 7f46c597b740 10 client.0 _ll_register_callbacks cb 0x7f46b400cea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:57:05.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.122+0000 7f46c1b3f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56288968bbb0 0x56288969fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:05.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.122+0000 7f46c1b3f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56288968bbb0 0x56288969fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55076/0 (socket says 172.21.15.1:55076) 2024-09-11T21:57:05.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.122+0000 7f46c1b3f640 1 -- 172.21.15.1:0/561054536 learned_addr learned my addr 172.21.15.1:0/561054536 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:05.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.122+0000 7f46c0b3d640 1 --2- 172.21.15.1:0/561054536 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628896812f0 0x56288969f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:05.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.122+0000 7f46c133e640 1 --2- 172.21.15.1:0/561054536 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628896809d0 0x5628896a6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:05.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.123+0000 7f46c1b3f640 1 -- 172.21.15.1:0/561054536 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628896812f0 msgr2=0x56288969f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:05.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.123+0000 7f46c1b3f640 1 --2- 172.21.15.1:0/561054536 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628896812f0 0x56288969f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:05.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.123+0000 7f46c1b3f640 1 -- 172.21.15.1:0/561054536 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628896809d0 msgr2=0x5628896a6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:05.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.123+0000 7f46c1b3f640 1 --2- 172.21.15.1:0/561054536 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628896809d0 0x5628896a6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:05.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.123+0000 7f46c1b3f640 1 -- 172.21.15.1:0/561054536 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628896a0600 con 0x56288968bbb0 2024-09-11T21:57:05.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.123+0000 7f46c133e640 1 --2- 172.21.15.1:0/561054536 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628896809d0 0x5628896a6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:57:05.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.123+0000 7f46c1b3f640 1 --2- 172.21.15.1:0/561054536 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56288968bbb0 0x56288969fdd0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f46b4007af0 tx=0x7f46b401abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=337b006c52c65cf8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:05.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.123+0000 7f46a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T21:57:05.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.123+0000 7f46a4ff9640 1 -- 172.21.15.1:0/561054536 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b403e020 con 0x56288968bbb0 2024-09-11T21:57:05.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.123+0000 7f46a4ff9640 1 -- 172.21.15.1:0/561054536 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f46b4039710 con 0x56288968bbb0 2024-09-11T21:57:05.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.123+0000 7f46c597b740 1 -- 172.21.15.1:0/561054536 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46b4034070 con 0x56288968bbb0 2024-09-11T21:57:05.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.123+0000 7f46a4ff9640 1 -- 172.21.15.1:0/561054536 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f46b4020040 con 0x56288968bbb0 2024-09-11T21:57:05.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.123+0000 7f46c597b740 10 client.15168 fetch_fsmap learned FSMap version 27 2024-09-11T21:57:05.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.123+0000 7f46c597b740 1 -- 172.21.15.1:0/561054536 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562889706e40 con 0x56288968bbb0 2024-09-11T21:57:05.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.123+0000 7f46a4ff9640 1 -- 172.21.15.1:0/561054536 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46b4034070 con 0x56288968bbb0 2024-09-11T21:57:05.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.124+0000 7f46a4ff9640 1 -- 172.21.15.1:0/561054536 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f46b4025760 con 0x56288968bbb0 2024-09-11T21:57:05.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.124+0000 7f46c597b740 10 client.15168 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T21:57:05.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.124+0000 7f46c597b740 10 client.15168 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-11T21:57:05.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.124+0000 7f46c597b740 20 client.15168 populate_metadata read hostname 'smithi001' 2024-09-11T21:57:05.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.124+0000 7f46c597b740 1 --2- 172.21.15.1:0/561054536 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x56288970b4a0 0x56288970a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:05.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.124+0000 7f46c597b740 4 client.15168 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T21:57:05.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.124+0000 7f46c597b740 1 -- 172.21.15.1:0/561054536 --> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] -- command(tid 0: {"prefix": "status"}) -- 0x56288967db10 con 0x56288970b4a0 2024-09-11T21:57:05.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.124+0000 7f46c133e640 1 --2- 172.21.15.1:0/561054536 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x56288970b4a0 0x56288970a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:05.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.124+0000 7f46c133e640 1 --2- 172.21.15.1:0/561054536 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x56288970b4a0 0x56288970a1b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=703eb15886e14876 server_cookie=a83468d5f0ce8996 in_seq=0 out_seq=0 2024-09-11T21:57:05.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.124+0000 7f46a4ff9640 10 client.15168 ms_handle_connect on v2:172.21.15.161:6838/455365383 2024-09-11T21:57:05.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.125+0000 7f46a4ff9640 1 -- 172.21.15.1:0/561054536 <== mds.1 v2:172.21.15.161:6838/455365383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x56288967db10 con 0x56288970b4a0 2024-09-11T21:57:05.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.125+0000 7f46a4ff9640 10 client.15168 handle_command_reply: tid=0 2024-09-11T21:57:05.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.125+0000 7f46c597b740 1 client.15168 shutdown 2024-09-11T21:57:05.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.125+0000 7f46c597b740 1 -- 172.21.15.1:0/561054536 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56288968bbb0 msgr2=0x56288969fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:05.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.125+0000 7f46c597b740 1 --2- 172.21.15.1:0/561054536 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56288968bbb0 0x56288969fdd0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f46b4007af0 tx=0x7f46b401abd0 comp rx=0 tx=0).stop 2024-09-11T21:57:05.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.125+0000 7f46c597b740 1 -- 172.21.15.1:0/561054536 shutdown_connections 2024-09-11T21:57:05.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.125+0000 7f46c597b740 1 --2- 172.21.15.1:0/561054536 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x56288970b4a0 0x56288970a1b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:05.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.125+0000 7f46c597b740 1 --2- 172.21.15.1:0/561054536 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56288968bbb0 0x56288969fdd0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:05.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.125+0000 7f46c597b740 1 --2- 172.21.15.1:0/561054536 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5628896812f0 0x56288969f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:05.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.125+0000 7f46c597b740 1 --2- 172.21.15.1:0/561054536 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5628896809d0 0x5628896a6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:05.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.125+0000 7f46c597b740 1 -- 172.21.15.1:0/561054536 >> 172.21.15.1:0/561054536 conn(0x56288967cf60 msgr2=0x562889686fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:05.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.125+0000 7f46a4ff9640 0 client.15168 ms_handle_reset on v2:172.21.15.161:6838/455365383 2024-09-11T21:57:05.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.125+0000 7f46c597b740 1 -- 172.21.15.1:0/561054536 shutdown_connections 2024-09-11T21:57:05.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.125+0000 7f46c597b740 1 -- 172.21.15.1:0/561054536 wait complete. 2024-09-11T21:57:05.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.125+0000 7f46c597b740 20 client.15168 trim_cache size 0 max 16384 2024-09-11T21:57:05.136 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-11T21:57:05.136 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "3ffca83c-7087-11ef-bcea-c7b262605968", 2024-09-11T21:57:05.136 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 1, 2024-09-11T21:57:05.136 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 24484, 2024-09-11T21:57:05.136 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-09-11T21:57:05.136 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-09-11T21:57:05.136 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-09-11T21:57:05.137 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 97.002893474999993, 2024-09-11T21:57:05.137 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 27, 2024-09-11T21:57:05.137 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 86, 2024-09-11T21:57:05.137 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 84, 2024-09-11T21:57:05.137 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 123.420096234 2024-09-11T21:57:05.137 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-11T21:57:05.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.126+0000 7f46c35a2640 1 -- 172.21.15.1:0/2778167025 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f46a80a2b80 msgr2=0x7f46a80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:05.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.126+0000 7f46c35a2640 1 --2- 172.21.15.1:0/2778167025 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f46a80a2b80 0x7f46a80a5040 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f46b4009fd0 tx=0x7f46b4004a40 comp rx=0 tx=0).stop 2024-09-11T21:57:05.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.126+0000 7f46c35a2640 1 -- 172.21.15.1:0/2778167025 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46bc101280 msgr2=0x7f46bc1a0ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:05.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.126+0000 7f46c35a2640 1 --2- 172.21.15.1:0/2778167025 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46bc101280 0x7f46bc1a0ac0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f46ac00b4a0 tx=0x7f46ac00b960 comp rx=0 tx=0).stop 2024-09-11T21:57:05.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.126+0000 7f46c35a2640 1 -- 172.21.15.1:0/2778167025 shutdown_connections 2024-09-11T21:57:05.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.126+0000 7f46c35a2640 1 --2- 172.21.15.1:0/2778167025 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f46a80a2b80 0x7f46a80a5040 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:05.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.126+0000 7f46c35a2640 1 --2- 172.21.15.1:0/2778167025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f46bc10e480 0x7f46bc19a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:05.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.126+0000 7f46c35a2640 1 --2- 172.21.15.1:0/2778167025 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46bc101280 0x7f46bc1a0ac0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:05.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.126+0000 7f46c35a2640 1 --2- 172.21.15.1:0/2778167025 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46bc1008d0 0x7f46bc1a0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:05.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.126+0000 7f46c35a2640 1 -- 172.21.15.1:0/2778167025 >> 172.21.15.1:0/2778167025 conn(0x7f46bc0fc7a0 msgr2=0x7f46bc0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:05.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.126+0000 7f46c35a2640 1 -- 172.21.15.1:0/2778167025 shutdown_connections 2024-09-11T21:57:05.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:05.126+0000 7f46c35a2640 1 -- 172.21.15.1:0/2778167025 wait complete. 2024-09-11T21:57:06.059 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-11T21:57:06.322 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:57:06.643 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:06 smithi001 ceph-mon[30935]: pgmap v336: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-11T21:57:06.644 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:06 smithi161 ceph-mon[35615]: pgmap v336: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-11T21:57:06.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:06 smithi053 ceph-mon[34908]: pgmap v336: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-11T21:57:06.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.828+0000 7f1fbf59e640 1 -- 172.21.15.1:0/4043420819 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1fc01068d0 msgr2=0x7f1fc0106cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:06.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.828+0000 7f1fbf59e640 1 --2- 172.21.15.1:0/4043420819 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1fc01068d0 0x7f1fc0106cb0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f1fb0009930 tx=0x7f1fb002a420 comp rx=0 tx=0).stop 2024-09-11T21:57:06.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.828+0000 7f1fbf59e640 1 -- 172.21.15.1:0/4043420819 shutdown_connections 2024-09-11T21:57:06.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.828+0000 7f1fbf59e640 1 --2- 172.21.15.1:0/4043420819 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fc0101270 0x7f1fc010d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:06.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.828+0000 7f1fbf59e640 1 --2- 172.21.15.1:0/4043420819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fc01008d0 0x7f1fc0100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:06.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.828+0000 7f1fbf59e640 1 --2- 172.21.15.1:0/4043420819 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1fc01068d0 0x7f1fc0106cb0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:06.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.828+0000 7f1fbf59e640 1 -- 172.21.15.1:0/4043420819 >> 172.21.15.1:0/4043420819 conn(0x7f1fc00fc610 msgr2=0x7f1fc00fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:06.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.828+0000 7f1fbf59e640 1 -- 172.21.15.1:0/4043420819 shutdown_connections 2024-09-11T21:57:06.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.828+0000 7f1fbf59e640 1 -- 172.21.15.1:0/4043420819 wait complete. 2024-09-11T21:57:06.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.828+0000 7f1fbf59e640 1 Processor -- start 2024-09-11T21:57:06.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.828+0000 7f1fbf59e640 1 -- start start 2024-09-11T21:57:06.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.829+0000 7f1fbf59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fc01008d0 0x7f1fc0197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:06.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.829+0000 7f1fbf59e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1fc0101270 0x7f1fc0197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:06.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.829+0000 7f1fbf59e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fc01068d0 0x7f1fc019bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:06.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.829+0000 7f1fbf59e640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc0110cf0 con 0x7f1fc0101270 2024-09-11T21:57:06.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.829+0000 7f1fbf59e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc0110b70 con 0x7f1fc01008d0 2024-09-11T21:57:06.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.829+0000 7f1fbf59e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc0110e70 con 0x7f1fc01068d0 2024-09-11T21:57:06.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.829+0000 7f1fb7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1fc0101270 0x7f1fc0197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:06.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.829+0000 7f1fb7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1fc0101270 0x7f1fc0197740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46544/0 (socket says 172.21.15.1:46544) 2024-09-11T21:57:06.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.829+0000 7f1fb7fff640 1 -- 172.21.15.1:0/808571190 learned_addr learned my addr 172.21.15.1:0/808571190 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:06.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.829+0000 7f1fbe59c640 1 --2- 172.21.15.1:0/808571190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fc01008d0 0x7f1fc0197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:06.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.829+0000 7f1fbed9d640 1 --2- 172.21.15.1:0/808571190 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fc01068d0 0x7f1fc019bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:06.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.829+0000 7f1fb7fff640 1 -- 172.21.15.1:0/808571190 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fc01068d0 msgr2=0x7f1fc019bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:06.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.829+0000 7f1fb7fff640 1 --2- 172.21.15.1:0/808571190 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fc01068d0 0x7f1fc019bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:06.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.829+0000 7f1fb7fff640 1 -- 172.21.15.1:0/808571190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fc01008d0 msgr2=0x7f1fc0197200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:06.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.829+0000 7f1fb7fff640 1 --2- 172.21.15.1:0/808571190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fc01008d0 0x7f1fc0197200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:06.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.829+0000 7f1fb7fff640 1 -- 172.21.15.1:0/808571190 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fc019c1b0 con 0x7f1fc0101270 2024-09-11T21:57:06.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.829+0000 7f1fb7fff640 1 --2- 172.21.15.1:0/808571190 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1fc0101270 0x7f1fc0197740 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f1fa800d950 tx=0x7f1fa800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:06.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.829+0000 7f1fb77fe640 1 -- 172.21.15.1:0/808571190 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa8017070 con 0x7f1fc0101270 2024-09-11T21:57:06.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.830+0000 7f1fbf59e640 1 -- 172.21.15.1:0/808571190 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fc019c4a0 con 0x7f1fc0101270 2024-09-11T21:57:06.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.830+0000 7f1fb77fe640 1 -- 172.21.15.1:0/808571190 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fa8009c30 con 0x7f1fc0101270 2024-09-11T21:57:06.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.830+0000 7f1fb77fe640 1 -- 172.21.15.1:0/808571190 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa800f6b0 con 0x7f1fc0101270 2024-09-11T21:57:06.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.830+0000 7f1fbf59e640 1 -- 172.21.15.1:0/808571190 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fc01a3d80 con 0x7f1fc0101270 2024-09-11T21:57:06.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.833+0000 7f1fb77fe640 1 -- 172.21.15.1:0/808571190 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f1fa80109d0 con 0x7f1fc0101270 2024-09-11T21:57:06.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.833+0000 7f1fc6fda740 1 Processor -- start 2024-09-11T21:57:06.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.833+0000 7f1fc6fda740 1 -- start start 2024-09-11T21:57:06.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.833+0000 7f1fc6fda740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ab9fe1a10 0x560ab9fe1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:06.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.833+0000 7f1fc6fda740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ab9fe2330 0x560ab9feea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:06.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.833+0000 7f1fc6fda740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ab9feefc0 0x560ab9ff13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:06.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.833+0000 7f1fc6fda740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x560ab9f48090 con 0x560ab9fe1a10 2024-09-11T21:57:06.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.833+0000 7f1fc6fda740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x560ab9eefa70 con 0x560ab9feefc0 2024-09-11T21:57:06.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.833+0000 7f1fc6fda740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x560ab9f44050 con 0x560ab9fe2330 2024-09-11T21:57:06.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.833+0000 7f1fbed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ab9fe1a10 0x560ab9fe1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:06.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.833+0000 7f1fbed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ab9fe1a10 0x560ab9fe1df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46554/0 (socket says 172.21.15.1:46554) 2024-09-11T21:57:06.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.833+0000 7f1fbed9d640 1 -- 172.21.15.1:0/3561434594 learned_addr learned my addr 172.21.15.1:0/3561434594 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:06.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.833+0000 7f1fbe59c640 1 --2- 172.21.15.1:0/3561434594 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ab9fe2330 0x560ab9feea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:06.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.834+0000 7f1fb7fff640 1 --2- 172.21.15.1:0/3561434594 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ab9feefc0 0x560ab9ff13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:06.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.834+0000 7f1fbed9d640 1 -- 172.21.15.1:0/3561434594 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ab9fe2330 msgr2=0x560ab9feea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:06.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.834+0000 7f1fbed9d640 1 --2- 172.21.15.1:0/3561434594 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ab9fe2330 0x560ab9feea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:06.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.834+0000 7f1fbed9d640 1 -- 172.21.15.1:0/3561434594 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ab9feefc0 msgr2=0x560ab9ff13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:06.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.834+0000 7f1fbed9d640 1 --2- 172.21.15.1:0/3561434594 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ab9feefc0 0x560ab9ff13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:06.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.834+0000 7f1fbed9d640 1 -- 172.21.15.1:0/3561434594 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ab9efc130 con 0x560ab9fe1a10 2024-09-11T21:57:06.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.834+0000 7f1fb7fff640 1 --2- 172.21.15.1:0/3561434594 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ab9feefc0 0x560ab9ff13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T21:57:06.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.834+0000 7f1fb77fe640 1 --2- 172.21.15.1:0/808571190 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f1fac0a2b80 0x7f1fac0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:06.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.834+0000 7f1fbed9d640 1 --2- 172.21.15.1:0/3561434594 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ab9fe1a10 0x560ab9fe1df0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8009f20 tx=0x7f1fb800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab67563f6d0429dd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:06.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.834+0000 7f1fbe59c640 1 --2- 172.21.15.1:0/808571190 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f1fac0a2b80 0x7f1fac0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:06.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.834+0000 7f1fb77fe640 1 -- 172.21.15.1:0/808571190 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f1fa80cecd0 con 0x7f1fc0101270 2024-09-11T21:57:06.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.834+0000 7f1fb4ff9640 1 -- 172.21.15.1:0/3561434594 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb801a070 con 0x560ab9fe1a10 2024-09-11T21:57:06.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.834+0000 7f1fb4ff9640 1 -- 172.21.15.1:0/3561434594 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fb80158f0 con 0x560ab9fe1a10 2024-09-11T21:57:06.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.834+0000 7f1fb4ff9640 1 -- 172.21.15.1:0/3561434594 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb800a8c0 con 0x560ab9fe1a10 2024-09-11T21:57:06.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.835+0000 7f1fc6fda740 1 -- 172.21.15.1:0/3561434594 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ab9fe1a10 msgr2=0x560ab9fe1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:06.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.835+0000 7f1fbe59c640 1 --2- 172.21.15.1:0/808571190 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f1fac0a2b80 0x7f1fac0a5040 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1fb0009900 tx=0x7f1fb00058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:06.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.835+0000 7f1fc6fda740 1 --2- 172.21.15.1:0/3561434594 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ab9fe1a10 0x560ab9fe1df0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8009f20 tx=0x7f1fb800c7f0 comp rx=0 tx=0).stop 2024-09-11T21:57:06.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.835+0000 7f1fc6fda740 1 -- 172.21.15.1:0/3561434594 shutdown_connections 2024-09-11T21:57:06.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.835+0000 7f1fc6fda740 1 --2- 172.21.15.1:0/3561434594 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ab9feefc0 0x560ab9ff13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:06.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.835+0000 7f1fc6fda740 1 --2- 172.21.15.1:0/3561434594 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ab9fe2330 0x560ab9feea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:06.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.835+0000 7f1fc6fda740 1 --2- 172.21.15.1:0/3561434594 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ab9fe1a10 0x560ab9fe1df0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:06.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.835+0000 7f1fc6fda740 1 -- 172.21.15.1:0/3561434594 >> 172.21.15.1:0/3561434594 conn(0x560ab9fde160 msgr2=0x560ab9fde540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:06.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.835+0000 7f1fc6fda740 1 -- 172.21.15.1:0/3561434594 shutdown_connections 2024-09-11T21:57:06.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.835+0000 7f1fc6fda740 1 -- 172.21.15.1:0/3561434594 wait complete. 2024-09-11T21:57:06.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.836+0000 7f1fc6fda740 1 -- start start 2024-09-11T21:57:06.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.836+0000 7f1fc6fda740 1 Processor -- start 2024-09-11T21:57:06.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.836+0000 7f1fc6fda740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ab9fe1a10 0x560ab9fee850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:06.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.836+0000 7f1fc6fda740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ab9fe2330 0x560ab9febba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:06.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.836+0000 7f1fc6fda740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ab9feefc0 0x560ab9fec0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:06.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.836+0000 7f1fc6fda740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x560ab9ff1b90 con 0x560ab9feefc0 2024-09-11T21:57:06.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.836+0000 7f1fc6fda740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x560ab9f02f20 con 0x560ab9fe1a10 2024-09-11T21:57:06.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.836+0000 7f1fc6fda740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x560ab9ff1d10 con 0x560ab9fe2330 2024-09-11T21:57:06.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.836+0000 7f1fc6fda740 10 client.0 _ll_register_callbacks cb 0x560ab9ecc030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:57:06.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.836+0000 7f1fbed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ab9feefc0 0x560ab9fec0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:06.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.836+0000 7f1fbed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ab9feefc0 0x560ab9fec0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46566/0 (socket says 172.21.15.1:46566) 2024-09-11T21:57:06.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.836+0000 7f1fbed9d640 1 -- 172.21.15.1:0/3407348085 learned_addr learned my addr 172.21.15.1:0/3407348085 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:06.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.836+0000 7f1fb7fff640 1 --2- 172.21.15.1:0/3407348085 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ab9fe2330 0x560ab9febba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:06.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.836+0000 7f1fbe59c640 1 --2- 172.21.15.1:0/3407348085 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ab9fe1a10 0x560ab9fee850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:06.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.837+0000 7f1fbed9d640 1 -- 172.21.15.1:0/3407348085 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ab9fe2330 msgr2=0x560ab9febba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:06.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.837+0000 7f1fbed9d640 1 --2- 172.21.15.1:0/3407348085 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ab9fe2330 0x560ab9febba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:06.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.837+0000 7f1fbed9d640 1 -- 172.21.15.1:0/3407348085 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ab9fe1a10 msgr2=0x560ab9fee850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:06.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.837+0000 7f1fbed9d640 1 --2- 172.21.15.1:0/3407348085 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ab9fe1a10 0x560ab9fee850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:06.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.837+0000 7f1fbed9d640 1 -- 172.21.15.1:0/3407348085 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ab9cb92b0 con 0x560ab9feefc0 2024-09-11T21:57:06.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.837+0000 7f1fbed9d640 1 --2- 172.21.15.1:0/3407348085 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ab9feefc0 0x560ab9fec0e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8018040 tx=0x7f1fb800c350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f61b4a731b167c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:06.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.837+0000 7f1fa5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T21:57:06.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.837+0000 7f1fa5ffb640 1 -- 172.21.15.1:0/3407348085 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb801a040 con 0x560ab9feefc0 2024-09-11T21:57:06.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.837+0000 7f1fc6fda740 1 -- 172.21.15.1:0/3407348085 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fb80158f0 con 0x560ab9feefc0 2024-09-11T21:57:06.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.837+0000 7f1fa5ffb640 1 -- 172.21.15.1:0/3407348085 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f1fb800a640 con 0x560ab9feefc0 2024-09-11T21:57:06.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.837+0000 7f1fa5ffb640 1 -- 172.21.15.1:0/3407348085 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1fb8015e50 con 0x560ab9feefc0 2024-09-11T21:57:06.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.837+0000 7f1fc6fda740 10 client.15180 fetch_fsmap learned FSMap version 27 2024-09-11T21:57:06.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.837+0000 7f1fc6fda740 1 -- 172.21.15.1:0/3407348085 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560ab9ffd450 con 0x560ab9feefc0 2024-09-11T21:57:06.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.837+0000 7f1fa5ffb640 1 -- 172.21.15.1:0/3407348085 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fb80158f0 con 0x560ab9feefc0 2024-09-11T21:57:06.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.838+0000 7f1fa5ffb640 1 -- 172.21.15.1:0/3407348085 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f1fb8020720 con 0x560ab9feefc0 2024-09-11T21:57:06.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.838+0000 7f1fc6fda740 10 client.15180 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T21:57:06.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.838+0000 7f1fc6fda740 10 client.15180 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T21:57:06.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.838+0000 7f1fc6fda740 20 client.15180 populate_metadata read hostname 'smithi001' 2024-09-11T21:57:06.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.838+0000 7f1fc6fda740 1 --2- 172.21.15.1:0/3407348085 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x560ab9fff6d0 0x560aba000b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:06.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.838+0000 7f1fc6fda740 4 client.15180 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T21:57:06.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.838+0000 7f1fc6fda740 1 -- 172.21.15.1:0/3407348085 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ab9f55600 con 0x560ab9fff6d0 2024-09-11T21:57:06.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.838+0000 7f1fbe59c640 1 --2- 172.21.15.1:0/3407348085 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x560ab9fff6d0 0x560aba000b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:06.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.839+0000 7f1fa5ffb640 10 client.15180 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T21:57:06.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.839+0000 7f1fbe59c640 1 --2- 172.21.15.1:0/3407348085 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x560ab9fff6d0 0x560aba000b40 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74e7cde4695f31b6 server_cookie=bef1a4e6593e6f2e in_seq=0 out_seq=0 2024-09-11T21:57:06.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.839+0000 7f1fa5ffb640 1 -- 172.21.15.1:0/3407348085 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560ab9f55600 con 0x560ab9fff6d0 2024-09-11T21:57:06.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.839+0000 7f1fa5ffb640 10 client.15180 handle_command_reply: tid=0 2024-09-11T21:57:06.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.839+0000 7f1fc6fda740 1 client.15180 shutdown 2024-09-11T21:57:06.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.839+0000 7f1fc6fda740 1 -- 172.21.15.1:0/3407348085 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ab9feefc0 msgr2=0x560ab9fec0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:06.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.839+0000 7f1fc6fda740 1 --2- 172.21.15.1:0/3407348085 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ab9feefc0 0x560ab9fec0e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8018040 tx=0x7f1fb800c350 comp rx=0 tx=0).stop 2024-09-11T21:57:06.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.839+0000 7f1fc6fda740 1 -- 172.21.15.1:0/3407348085 shutdown_connections 2024-09-11T21:57:06.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.839+0000 7f1fc6fda740 1 --2- 172.21.15.1:0/3407348085 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x560ab9fff6d0 0x560aba000b40 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:06.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.839+0000 7f1fc6fda740 1 --2- 172.21.15.1:0/3407348085 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ab9feefc0 0x560ab9fec0e0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:06.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.839+0000 7f1fc6fda740 1 --2- 172.21.15.1:0/3407348085 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ab9fe2330 0x560ab9febba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:06.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.839+0000 7f1fc6fda740 1 --2- 172.21.15.1:0/3407348085 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ab9fe1a10 0x560ab9fee850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:06.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.839+0000 7f1fc6fda740 1 -- 172.21.15.1:0/3407348085 >> 172.21.15.1:0/3407348085 conn(0x560ab9fde160 msgr2=0x560ab9fe3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:06.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.840+0000 7f1fa5ffb640 0 client.15180 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T21:57:06.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.840+0000 7f1fc6fda740 1 -- 172.21.15.1:0/3407348085 shutdown_connections 2024-09-11T21:57:06.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.840+0000 7f1fc6fda740 1 -- 172.21.15.1:0/3407348085 wait complete. 2024-09-11T21:57:06.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.840+0000 7f1fc6fda740 20 client.15180 trim_cache size 0 max 16384 2024-09-11T21:57:06.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.857+0000 7f1fc6fda740 1 Processor -- start 2024-09-11T21:57:06.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.858+0000 7f1fc6fda740 1 -- start start 2024-09-11T21:57:06.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.858+0000 7f1fc6fda740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ab9fe1a10 0x560ab9fff6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:06.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.858+0000 7f1fc6fda740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ab9fe2330 0x560ab9fe3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:06.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.858+0000 7f1fc6fda740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ab9feefc0 0x560ab9fe8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:06.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.858+0000 7f1fc6fda740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x560ab9ffee80 con 0x560ab9fe1a10 2024-09-11T21:57:06.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.858+0000 7f1fc6fda740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x560ab9fff550 con 0x560ab9fe2330 2024-09-11T21:57:06.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.858+0000 7f1fc6fda740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x560ab9fffc20 con 0x560ab9feefc0 2024-09-11T21:57:06.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.858+0000 7f1fbe59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ab9fe1a10 0x560ab9fff6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:06.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.858+0000 7f1fbe59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ab9fe1a10 0x560ab9fff6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46582/0 (socket says 172.21.15.1:46582) 2024-09-11T21:57:06.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.858+0000 7f1fbe59c640 1 -- 172.21.15.1:0/2856665420 learned_addr learned my addr 172.21.15.1:0/2856665420 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:06.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.858+0000 7f1fb7fff640 1 --2- 172.21.15.1:0/2856665420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ab9fe2330 0x560ab9fe3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:06.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.858+0000 7f1fbe59c640 1 -- 172.21.15.1:0/2856665420 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ab9feefc0 msgr2=0x560ab9fe8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:06.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.858+0000 7f1fbe59c640 1 --2- 172.21.15.1:0/2856665420 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ab9feefc0 0x560ab9fe8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:06.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.858+0000 7f1fbe59c640 1 -- 172.21.15.1:0/2856665420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ab9fe2330 msgr2=0x560ab9fe3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:06.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.858+0000 7f1fbe59c640 1 --2- 172.21.15.1:0/2856665420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ab9fe2330 0x560ab9fe3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:06.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.858+0000 7f1fbe59c640 1 -- 172.21.15.1:0/2856665420 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ab9feebc0 con 0x560ab9fe1a10 2024-09-11T21:57:06.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.858+0000 7f1fbe59c640 1 --2- 172.21.15.1:0/2856665420 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ab9fe1a10 0x560ab9fff6d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f1fb00096c0 tx=0x7f1fb003f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b728a2eabd37366 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:06.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.859+0000 7f1fa7fff640 1 -- 172.21.15.1:0/2856665420 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb0058070 con 0x560ab9fe1a10 2024-09-11T21:57:06.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.859+0000 7f1fa7fff640 1 -- 172.21.15.1:0/2856665420 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fb0003f20 con 0x560ab9fe1a10 2024-09-11T21:57:06.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.859+0000 7f1fa7fff640 1 -- 172.21.15.1:0/2856665420 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb0035040 con 0x560ab9fe1a10 2024-09-11T21:57:06.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.859+0000 7f1fc6fda740 1 -- 172.21.15.1:0/2856665420 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ab9fe1a10 msgr2=0x560ab9fff6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:06.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.859+0000 7f1fc6fda740 1 --2- 172.21.15.1:0/2856665420 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ab9fe1a10 0x560ab9fff6d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f1fb00096c0 tx=0x7f1fb003f000 comp rx=0 tx=0).stop 2024-09-11T21:57:06.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.859+0000 7f1fc6fda740 1 -- 172.21.15.1:0/2856665420 shutdown_connections 2024-09-11T21:57:06.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.859+0000 7f1fc6fda740 1 --2- 172.21.15.1:0/2856665420 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ab9feefc0 0x560ab9fe8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:06.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.859+0000 7f1fc6fda740 1 --2- 172.21.15.1:0/2856665420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ab9fe2330 0x560ab9fe3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:06.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.859+0000 7f1fc6fda740 1 --2- 172.21.15.1:0/2856665420 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ab9fe1a10 0x560ab9fff6d0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:06.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.859+0000 7f1fc6fda740 1 -- 172.21.15.1:0/2856665420 >> 172.21.15.1:0/2856665420 conn(0x560ab9fde160 msgr2=0x560ab9fe35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:06.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.859+0000 7f1fc6fda740 1 -- 172.21.15.1:0/2856665420 shutdown_connections 2024-09-11T21:57:06.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.859+0000 7f1fc6fda740 1 -- 172.21.15.1:0/2856665420 wait complete. 2024-09-11T21:57:06.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.860+0000 7f1fc6fda740 1 -- start start 2024-09-11T21:57:06.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.860+0000 7f1fc6fda740 1 Processor -- start 2024-09-11T21:57:06.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.860+0000 7f1fc6fda740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ab9fe1a10 0x560aba009220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:06.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.860+0000 7f1fc6fda740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ab9fe2330 0x560ab9fe5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:06.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.860+0000 7f1fc6fda740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ab9feefc0 0x560ab9fe5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:06.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.860+0000 7f1fc6fda740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x560aba0009c0 con 0x560ab9fe2330 2024-09-11T21:57:06.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.860+0000 7f1fc6fda740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x560aba0002f0 con 0x560ab9fe1a10 2024-09-11T21:57:06.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.860+0000 7f1fc6fda740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x560aba001090 con 0x560ab9feefc0 2024-09-11T21:57:06.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.860+0000 7f1fc6fda740 10 client.0 _ll_register_callbacks cb 0x7f1fb8018070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:57:06.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.860+0000 7f1fb7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ab9fe2330 0x560ab9fe5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:06.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.860+0000 7f1fb7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ab9fe2330 0x560ab9fe5930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46598/0 (socket says 172.21.15.1:46598) 2024-09-11T21:57:06.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.860+0000 7f1fb7fff640 1 -- 172.21.15.1:0/3374339501 learned_addr learned my addr 172.21.15.1:0/3374339501 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:06.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.860+0000 7f1fbe59c640 1 --2- 172.21.15.1:0/3374339501 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ab9fe1a10 0x560aba009220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:06.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.860+0000 7f1fbed9d640 1 --2- 172.21.15.1:0/3374339501 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ab9feefc0 0x560ab9fe5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:06.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.860+0000 7f1fb7fff640 1 -- 172.21.15.1:0/3374339501 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ab9feefc0 msgr2=0x560ab9fe5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:06.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.860+0000 7f1fb7fff640 1 --2- 172.21.15.1:0/3374339501 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ab9feefc0 0x560ab9fe5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:06.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.860+0000 7f1fb7fff640 1 -- 172.21.15.1:0/3374339501 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ab9fe1a10 msgr2=0x560aba009220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:06.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.860+0000 7f1fb7fff640 1 --2- 172.21.15.1:0/3374339501 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ab9fe1a10 0x560aba009220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:06.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.860+0000 7f1fb7fff640 1 -- 172.21.15.1:0/3374339501 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560aba009810 con 0x560ab9fe2330 2024-09-11T21:57:06.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.860+0000 7f1fbed9d640 1 --2- 172.21.15.1:0/3374339501 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ab9feefc0 0x560ab9fe5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T21:57:06.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.861+0000 7f1fb7fff640 1 --2- 172.21.15.1:0/3374339501 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ab9fe2330 0x560ab9fe5930 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1fa809b4a0 tx=0x7f1fa809b110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=585979e1d33d428d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:06.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.861+0000 7f1fa5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T21:57:06.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.861+0000 7f1fa5ffb640 1 -- 172.21.15.1:0/3374339501 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa80b4060 con 0x560ab9fe2330 2024-09-11T21:57:06.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.861+0000 7f1fa5ffb640 1 -- 172.21.15.1:0/3374339501 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f1fa80b4200 con 0x560ab9fe2330 2024-09-11T21:57:06.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.861+0000 7f1fc6fda740 1 -- 172.21.15.1:0/3374339501 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fb0003f20 con 0x560ab9fe2330 2024-09-11T21:57:06.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.861+0000 7f1fa5ffb640 1 -- 172.21.15.1:0/3374339501 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1fa8020040 con 0x560ab9fe2330 2024-09-11T21:57:06.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.861+0000 7f1fc6fda740 10 client.15186 fetch_fsmap learned FSMap version 27 2024-09-11T21:57:06.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.861+0000 7f1fc6fda740 1 -- 172.21.15.1:0/3374339501 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560aba06b770 con 0x560ab9fe2330 2024-09-11T21:57:06.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.861+0000 7f1fa5ffb640 1 -- 172.21.15.1:0/3374339501 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fb0003f20 con 0x560ab9fe2330 2024-09-11T21:57:06.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.861+0000 7f1fa5ffb640 1 -- 172.21.15.1:0/3374339501 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f1fa809a960 con 0x560ab9fe2330 2024-09-11T21:57:06.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.861+0000 7f1fc6fda740 10 client.15186 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T21:57:06.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.861+0000 7f1fc6fda740 10 client.15186 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T21:57:06.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.861+0000 7f1fc6fda740 20 client.15186 populate_metadata read hostname 'smithi001' 2024-09-11T21:57:06.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.861+0000 7f1fc6fda740 1 --2- 172.21.15.1:0/3374339501 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x560aba070370 0x560aba06f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:06.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.861+0000 7f1fc6fda740 4 client.15186 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T21:57:06.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.861+0000 7f1fc6fda740 1 -- 172.21.15.1:0/3374339501 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "status"}) -- 0x560ab9f4fb10 con 0x560aba070370 2024-09-11T21:57:06.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.862+0000 7f1fbe59c640 1 --2- 172.21.15.1:0/3374339501 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x560aba070370 0x560aba06f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:06.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.862+0000 7f1fa5ffb640 10 client.15186 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T21:57:06.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.862+0000 7f1fbe59c640 1 --2- 172.21.15.1:0/3374339501 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x560aba070370 0x560aba06f080 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e6e7f923739ed40 server_cookie=998b13c50d53eefc in_seq=0 out_seq=0 2024-09-11T21:57:06.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.862+0000 7f1fa5ffb640 1 -- 172.21.15.1:0/3374339501 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x560ab9f4fb10 con 0x560aba070370 2024-09-11T21:57:06.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.862+0000 7f1fa5ffb640 10 client.15186 handle_command_reply: tid=0 2024-09-11T21:57:06.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.862+0000 7f1fc6fda740 1 client.15186 shutdown 2024-09-11T21:57:06.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.863+0000 7f1fc6fda740 1 -- 172.21.15.1:0/3374339501 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ab9fe2330 msgr2=0x560ab9fe5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:06.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.863+0000 7f1fc6fda740 1 --2- 172.21.15.1:0/3374339501 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ab9fe2330 0x560ab9fe5930 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1fa809b4a0 tx=0x7f1fa809b110 comp rx=0 tx=0).stop 2024-09-11T21:57:06.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.863+0000 7f1fc6fda740 1 -- 172.21.15.1:0/3374339501 shutdown_connections 2024-09-11T21:57:06.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.863+0000 7f1fc6fda740 1 --2- 172.21.15.1:0/3374339501 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x560aba070370 0x560aba06f080 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:06.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.863+0000 7f1fc6fda740 1 --2- 172.21.15.1:0/3374339501 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ab9feefc0 0x560ab9fe5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:06.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.863+0000 7f1fc6fda740 1 --2- 172.21.15.1:0/3374339501 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ab9fe2330 0x560ab9fe5930 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:06.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.863+0000 7f1fc6fda740 1 --2- 172.21.15.1:0/3374339501 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ab9fe1a10 0x560aba009220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:06.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.863+0000 7f1fa5ffb640 0 client.15186 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T21:57:06.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.863+0000 7f1fc6fda740 1 -- 172.21.15.1:0/3374339501 >> 172.21.15.1:0/3374339501 conn(0x560ab9fde160 msgr2=0x560ab9fe3dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:06.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.863+0000 7f1fc6fda740 1 -- 172.21.15.1:0/3374339501 shutdown_connections 2024-09-11T21:57:06.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.863+0000 7f1fc6fda740 1 -- 172.21.15.1:0/3374339501 wait complete. 2024-09-11T21:57:06.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.863+0000 7f1fc6fda740 20 client.15186 trim_cache size 0 max 16384 2024-09-11T21:57:06.869 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-11T21:57:06.869 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "3ffca83c-7087-11ef-bcea-c7b262605968", 2024-09-11T21:57:06.869 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 0, 2024-09-11T21:57:06.869 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 24494, 2024-09-11T21:57:06.869 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-09-11T21:57:06.869 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-09-11T21:57:06.869 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-09-11T21:57:06.869 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 105.773410309, 2024-09-11T21:57:06.869 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 27, 2024-09-11T21:57:06.869 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 86, 2024-09-11T21:57:06.869 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 84, 2024-09-11T21:57:06.870 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 116.443083159 2024-09-11T21:57:06.870 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-11T21:57:06.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.863+0000 7f1fbf59e640 1 -- 172.21.15.1:0/808571190 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f1fac0a2b80 msgr2=0x7f1fac0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:06.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.863+0000 7f1fbf59e640 1 --2- 172.21.15.1:0/808571190 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f1fac0a2b80 0x7f1fac0a5040 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1fb0009900 tx=0x7f1fb00058d0 comp rx=0 tx=0).stop 2024-09-11T21:57:06.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.863+0000 7f1fbf59e640 1 -- 172.21.15.1:0/808571190 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1fc0101270 msgr2=0x7f1fc0197740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:06.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.863+0000 7f1fbf59e640 1 --2- 172.21.15.1:0/808571190 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1fc0101270 0x7f1fc0197740 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f1fa800d950 tx=0x7f1fa800de10 comp rx=0 tx=0).stop 2024-09-11T21:57:06.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.863+0000 7f1fbf59e640 1 -- 172.21.15.1:0/808571190 shutdown_connections 2024-09-11T21:57:06.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.863+0000 7f1fbf59e640 1 --2- 172.21.15.1:0/808571190 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f1fac0a2b80 0x7f1fac0a5040 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:06.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.863+0000 7f1fbf59e640 1 --2- 172.21.15.1:0/808571190 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fc01068d0 0x7f1fc019bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:06.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.863+0000 7f1fbf59e640 1 --2- 172.21.15.1:0/808571190 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1fc0101270 0x7f1fc0197740 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:06.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.863+0000 7f1fbf59e640 1 --2- 172.21.15.1:0/808571190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fc01008d0 0x7f1fc0197200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:06.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.863+0000 7f1fbf59e640 1 -- 172.21.15.1:0/808571190 >> 172.21.15.1:0/808571190 conn(0x7f1fc00fc610 msgr2=0x7f1fc00fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:06.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.864+0000 7f1fbf59e640 1 -- 172.21.15.1:0/808571190 shutdown_connections 2024-09-11T21:57:06.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:06.864+0000 7f1fbf59e640 1 -- 172.21.15.1:0/808571190 wait complete. 2024-09-11T21:57:07.835 INFO:teuthology.run_tasks:Running task sequential... 2024-09-11T21:57:07.851 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-11T21:57:07.852 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-11T21:57:07.852 INFO:teuthology.task.print:Enabling mgr modules 2024-09-11T21:57:07.852 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-09-11T21:57:07.853 INFO:teuthology.task.exec:Executing custom commands... 2024-09-11T21:57:07.853 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi001.front.sepia.ceph.com 2024-09-11T21:57:07.853 DEBUG:teuthology.orchestra.run.smithi001:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-09-11T21:57:07.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:07.964+0000 7f1975077640 1 -- 172.21.15.1:0/3926385258 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1970102620 msgr2=0x7f1970108eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:07.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:07.964+0000 7f1975077640 1 --2- 172.21.15.1:0/3926385258 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1970102620 0x7f1970108eb0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1968009ca0 tx=0x7f196802a8c0 comp rx=0 tx=0).stop 2024-09-11T21:57:07.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:07.964+0000 7f1975077640 1 -- 172.21.15.1:0/3926385258 shutdown_connections 2024-09-11T21:57:07.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:07.964+0000 7f1975077640 1 --2- 172.21.15.1:0/3926385258 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1970102620 0x7f1970108eb0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:07.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:07.964+0000 7f1975077640 1 --2- 172.21.15.1:0/3926385258 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1970101c60 0x7f19701020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:07.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:07.964+0000 7f1975077640 1 --2- 172.21.15.1:0/3926385258 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1970100a70 0x7f1970100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:07.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:07.964+0000 7f1975077640 1 -- 172.21.15.1:0/3926385258 >> 172.21.15.1:0/3926385258 conn(0x7f19700fc1f0 msgr2=0x7f19700fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:07.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:07.964+0000 7f1975077640 1 -- 172.21.15.1:0/3926385258 shutdown_connections 2024-09-11T21:57:07.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:07.964+0000 7f1975077640 1 -- 172.21.15.1:0/3926385258 wait complete. 2024-09-11T21:57:07.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:07.964+0000 7f1975077640 1 Processor -- start 2024-09-11T21:57:07.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:07.965+0000 7f1975077640 1 -- start start 2024-09-11T21:57:07.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:07.965+0000 7f1975077640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1970100a70 0x7f1970196f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:07.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:07.965+0000 7f1975077640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1970101c60 0x7f1970197440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:07.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:07.965+0000 7f1975077640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1970102620 0x7f197019e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:07.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:07.965+0000 7f1975077640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f197010c7f0 con 0x7f1970102620 2024-09-11T21:57:07.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:07.965+0000 7f1975077640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f197010c670 con 0x7f1970101c60 2024-09-11T21:57:07.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:07.965+0000 7f1975077640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f197010c970 con 0x7f1970100a70 2024-09-11T21:57:07.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:07.965+0000 7f196f59e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1970102620 0x7f197019e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:07.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:07.965+0000 7f196f59e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1970102620 0x7f197019e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46620/0 (socket says 172.21.15.1:46620) 2024-09-11T21:57:07.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:07.965+0000 7f196f59e640 1 -- 172.21.15.1:0/905611082 learned_addr learned my addr 172.21.15.1:0/905611082 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:07.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:07.965+0000 7f196e59c640 1 --2- 172.21.15.1:0/905611082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1970101c60 0x7f1970197440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:07.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:07.965+0000 7f196ed9d640 1 --2- 172.21.15.1:0/905611082 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1970100a70 0x7f1970196f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:07.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:07.965+0000 7f196f59e640 1 -- 172.21.15.1:0/905611082 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1970100a70 msgr2=0x7f1970196f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:07.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:07.965+0000 7f196f59e640 1 --2- 172.21.15.1:0/905611082 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1970100a70 0x7f1970196f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:07.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:07.965+0000 7f196f59e640 1 -- 172.21.15.1:0/905611082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1970101c60 msgr2=0x7f1970197440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:07.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:07.965+0000 7f196f59e640 1 --2- 172.21.15.1:0/905611082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1970101c60 0x7f1970197440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:07.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:07.965+0000 7f196f59e640 1 -- 172.21.15.1:0/905611082 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f197019eb20 con 0x7f1970102620 2024-09-11T21:57:07.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:07.966+0000 7f196f59e640 1 --2- 172.21.15.1:0/905611082 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1970102620 0x7f197019e4b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1968031ec0 tx=0x7f1968031ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:07.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:07.966+0000 7f1957fff640 1 -- 172.21.15.1:0/905611082 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1968003c30 con 0x7f1970102620 2024-09-11T21:57:07.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:07.966+0000 7f1957fff640 1 -- 172.21.15.1:0/905611082 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1968003dd0 con 0x7f1970102620 2024-09-11T21:57:07.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:07.966+0000 7f1957fff640 1 -- 172.21.15.1:0/905611082 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1968004770 con 0x7f1970102620 2024-09-11T21:57:07.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:07.966+0000 7f1975077640 1 -- 172.21.15.1:0/905611082 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f197019edb0 con 0x7f1970102620 2024-09-11T21:57:07.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:07.966+0000 7f1975077640 1 -- 172.21.15.1:0/905611082 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f197019f2f0 con 0x7f1970102620 2024-09-11T21:57:07.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:07.967+0000 7f1975077640 1 -- 172.21.15.1:0/905611082 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1970107ea0 con 0x7f1970102620 2024-09-11T21:57:07.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:07.968+0000 7f1957fff640 1 -- 172.21.15.1:0/905611082 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f1968004b40 con 0x7f1970102620 2024-09-11T21:57:07.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:07.969+0000 7f1957fff640 1 --2- 172.21.15.1:0/905611082 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f19580a2b80 0x7f19580a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:07.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:07.969+0000 7f196ed9d640 1 --2- 172.21.15.1:0/905611082 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f19580a2b80 0x7f19580a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:07.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:07.969+0000 7f1957fff640 1 -- 172.21.15.1:0/905611082 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f19680f0680 con 0x7f1970102620 2024-09-11T21:57:07.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:07.969+0000 7f196ed9d640 1 --2- 172.21.15.1:0/905611082 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f19580a2b80 0x7f19580a5040 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1970101ac0 tx=0x7f1960008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:07.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:07.972+0000 7f1957fff640 1 -- 172.21.15.1:0/905611082 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f19680bd5c0 con 0x7f1970102620 2024-09-11T21:57:08.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:08.196+0000 7f1975077640 1 -- 172.21.15.1:0/905611082 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f1970062df0 con 0x7f1970102620 2024-09-11T21:57:08.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:08.391+0000 7f1957fff640 1 -- 172.21.15.1:0/905611082 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f19680bcec0 con 0x7f1970102620 2024-09-11T21:57:08.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:08.394+0000 7f1975077640 1 -- 172.21.15.1:0/905611082 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f19580a2b80 msgr2=0x7f19580a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:08.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:08.395+0000 7f1975077640 1 --2- 172.21.15.1:0/905611082 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f19580a2b80 0x7f19580a5040 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1970101ac0 tx=0x7f1960008040 comp rx=0 tx=0).stop 2024-09-11T21:57:08.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:08.395+0000 7f1975077640 1 -- 172.21.15.1:0/905611082 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1970102620 msgr2=0x7f197019e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:08.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:08.395+0000 7f1975077640 1 --2- 172.21.15.1:0/905611082 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1970102620 0x7f197019e4b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1968031ec0 tx=0x7f1968031ef0 comp rx=0 tx=0).stop 2024-09-11T21:57:08.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:08.395+0000 7f1975077640 1 -- 172.21.15.1:0/905611082 shutdown_connections 2024-09-11T21:57:08.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:08.395+0000 7f1975077640 1 --2- 172.21.15.1:0/905611082 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f19580a2b80 0x7f19580a5040 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:08.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:08.395+0000 7f1975077640 1 --2- 172.21.15.1:0/905611082 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1970102620 0x7f197019e4b0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:08.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:08.395+0000 7f1975077640 1 --2- 172.21.15.1:0/905611082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1970101c60 0x7f1970197440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:08.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:08.395+0000 7f1975077640 1 --2- 172.21.15.1:0/905611082 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1970100a70 0x7f1970196f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:08.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:08.395+0000 7f1975077640 1 -- 172.21.15.1:0/905611082 >> 172.21.15.1:0/905611082 conn(0x7f19700fc1f0 msgr2=0x7f1970105070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:08.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:08.395+0000 7f1975077640 1 -- 172.21.15.1:0/905611082 shutdown_connections 2024-09-11T21:57:08.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:08.395+0000 7f1975077640 1 -- 172.21.15.1:0/905611082 wait complete. 2024-09-11T21:57:08.412 INFO:teuthology.run_tasks:Running task kclient... 2024-09-11T21:57:08.441 INFO:tasks.kclient:Mounting kernel clients... 2024-09-11T21:57:08.441 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-11T21:57:08.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T21:57:08.442 DEBUG:teuthology.orchestra.run.smithi001:> ip netns list 2024-09-11T21:57:08.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T21:57:08.460 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link delete ceph-brx 2024-09-11T21:57:08.529 INFO:teuthology.orchestra.run.smithi001.stderr:Cannot find device "ceph-brx" 2024-09-11T21:57:08.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T21:57:08.531 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-09-11T21:57:08.531 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-11T21:57:08.531 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-11T21:57:08.531 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph fs ls 2024-09-11T21:57:08.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:08 smithi001 ceph-mon[30935]: pgmap v337: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-11T21:57:08.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:08 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/905611082' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-11T21:57:08.694 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:57:08 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: ignoring --setuser ceph since I am not root 2024-09-11T21:57:08.694 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:57:08 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: ignoring --setgroup ceph since I am not root 2024-09-11T21:57:08.694 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:57:08 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:57:08.549+0000 7f5faf926040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T21:57:08.694 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:57:08 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:57:08.611+0000 7f5faf926040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T21:57:08.754 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:08 smithi161 ceph-mon[35615]: pgmap v337: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-11T21:57:08.754 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:08 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/905611082' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-11T21:57:08.755 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 11 21:57:08 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-z[37416]: ignoring --setuser ceph since I am not root 2024-09-11T21:57:08.755 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 11 21:57:08 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-z[37416]: ignoring --setgroup ceph since I am not root 2024-09-11T21:57:08.755 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 11 21:57:08 smithi161 ceph-mgr[37420]: -- 172.21.15.161:0/2002806077 <== mon.1 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5632803250e0 con 0x563280303000 2024-09-11T21:57:08.755 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 11 21:57:08 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-z[37416]: 2024-09-11T21:57:08.545+0000 7fcfe6d89040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T21:57:08.755 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 11 21:57:08 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-z[37416]: 2024-09-11T21:57:08.603+0000 7fcfe6d89040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T21:57:08.755 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 11 21:57:08 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-z[37416]: 2024-09-11T21:57:08.752+0000 7fcfe6d89040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-11T21:57:08.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:08 smithi053 ceph-mon[34908]: pgmap v337: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-11T21:57:08.762 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:08 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/905611082' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-11T21:57:08.762 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 11 21:57:08 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-y[36280]: ignoring --setuser ceph since I am not root 2024-09-11T21:57:08.762 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 11 21:57:08 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-y[36280]: ignoring --setgroup ceph since I am not root 2024-09-11T21:57:08.762 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 11 21:57:08 smithi053 ceph-mgr[36284]: -- 172.21.15.53:0/3423769860 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b52576a680 con 0x55b52573f400 2024-09-11T21:57:08.762 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 11 21:57:08 smithi053 ceph-mgr[36284]: -- 172.21.15.53:0/3423769860 <== mon.2 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b5257610e0 con 0x55b52573f400 2024-09-11T21:57:08.763 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 11 21:57:08 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-y[36280]: 2024-09-11T21:57:08.548+0000 7ff2421f9040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T21:57:08.763 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 11 21:57:08 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-y[36280]: 2024-09-11T21:57:08.608+0000 7ff2421f9040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T21:57:08.763 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 11 21:57:08 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-y[36280]: 2024-09-11T21:57:08.761+0000 7ff2421f9040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-11T21:57:08.836 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:57:09.047 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 11 21:57:08 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-y[36280]: 2024-09-11T21:57:08.830+0000 7ff2421f9040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T21:57:09.047 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 11 21:57:08 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-y[36280]: 2024-09-11T21:57:08.982+0000 7ff2421f9040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T21:57:09.078 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:57:08 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:57:08.771+0000 7f5faf926040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-11T21:57:09.079 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:57:08 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:57:08.849+0000 7f5faf926040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T21:57:09.079 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:57:09 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:57:09.005+0000 7f5faf926040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T21:57:09.092 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 11 21:57:08 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-z[37416]: 2024-09-11T21:57:08.819+0000 7fcfe6d89040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T21:57:09.092 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 11 21:57:08 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-z[37416]: 2024-09-11T21:57:08.967+0000 7fcfe6d89040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T21:57:09.356 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:57:09 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:57:09.159+0000 7f5faf926040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T21:57:09.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:09.355+0000 7ffafff4f640 1 -- 172.21.15.1:0/620358092 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffaf8105ec0 msgr2=0x7ffaf8106340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:09.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:09.355+0000 7ffafff4f640 1 --2- 172.21.15.1:0/620358092 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffaf8105ec0 0x7ffaf8106340 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ffaf0009ca0 tx=0x7ffaf002a8c0 comp rx=0 tx=0).stop 2024-09-11T21:57:09.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:09.355+0000 7ffafff4f640 1 -- 172.21.15.1:0/620358092 shutdown_connections 2024-09-11T21:57:09.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:09.355+0000 7ffafff4f640 1 --2- 172.21.15.1:0/620358092 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffaf8105ec0 0x7ffaf8106340 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:09.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:09.355+0000 7ffafff4f640 1 --2- 172.21.15.1:0/620358092 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffaf81011e0 0x7ffaf8105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:09.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:09.355+0000 7ffafff4f640 1 --2- 172.21.15.1:0/620358092 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffaf81008c0 0x7ffaf8100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:09.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:09.355+0000 7ffafff4f640 1 -- 172.21.15.1:0/620358092 >> 172.21.15.1:0/620358092 conn(0x7ffaf80fc7d0 msgr2=0x7ffaf80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:09.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:09.355+0000 7ffafff4f640 1 -- 172.21.15.1:0/620358092 shutdown_connections 2024-09-11T21:57:09.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:09.355+0000 7ffafff4f640 1 -- 172.21.15.1:0/620358092 wait complete. 2024-09-11T21:57:09.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:09.356+0000 7ffafff4f640 1 Processor -- start 2024-09-11T21:57:09.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:09.356+0000 7ffafff4f640 1 -- start start 2024-09-11T21:57:09.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:09.356+0000 7ffafff4f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffaf81008c0 0x7ffaf81a47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:09.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:09.356+0000 7ffafff4f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffaf81011e0 0x7ffaf81a4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:09.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:09.356+0000 7ffafff4f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffaf8105ec0 0x7ffaf819e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:09.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:09.356+0000 7ffafff4f640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ffaf8118630 con 0x7ffaf81011e0 2024-09-11T21:57:09.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:09.356+0000 7ffafff4f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ffaf81184b0 con 0x7ffaf81008c0 2024-09-11T21:57:09.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:09.356+0000 7ffafff4f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ffaf81187b0 con 0x7ffaf8105ec0 2024-09-11T21:57:09.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:09.356+0000 7ffafd4ea640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffaf81011e0 0x7ffaf81a4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:09.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:09.357+0000 7ffafd4ea640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffaf81011e0 0x7ffaf81a4ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46644/0 (socket says 172.21.15.1:46644) 2024-09-11T21:57:09.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:09.357+0000 7ffafd4ea640 1 -- 172.21.15.1:0/914619396 learned_addr learned my addr 172.21.15.1:0/914619396 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:09.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:09.357+0000 7ffafe4ec640 1 --2- 172.21.15.1:0/914619396 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffaf8105ec0 0x7ffaf819e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:09.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:09.357+0000 7ffafdceb640 1 --2- 172.21.15.1:0/914619396 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffaf81008c0 0x7ffaf81a47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:09.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:09.357+0000 7ffafd4ea640 1 -- 172.21.15.1:0/914619396 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffaf8105ec0 msgr2=0x7ffaf819e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:09.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:09.357+0000 7ffafd4ea640 1 --2- 172.21.15.1:0/914619396 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffaf8105ec0 0x7ffaf819e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:09.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:09.357+0000 7ffafd4ea640 1 -- 172.21.15.1:0/914619396 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffaf81008c0 msgr2=0x7ffaf81a47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:09.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:09.357+0000 7ffafd4ea640 1 --2- 172.21.15.1:0/914619396 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffaf81008c0 0x7ffaf81a47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:09.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:09.357+0000 7ffafd4ea640 1 -- 172.21.15.1:0/914619396 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaf819f090 con 0x7ffaf81011e0 2024-09-11T21:57:09.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:09.357+0000 7ffafdceb640 1 --2- 172.21.15.1:0/914619396 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffaf81008c0 0x7ffaf81a47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:57:09.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:09.357+0000 7ffafd4ea640 1 --2- 172.21.15.1:0/914619396 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffaf81011e0 0x7ffaf81a4ce0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ffae800e9f0 tx=0x7ffae800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:09.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:09.357+0000 7ffaeeffd640 1 -- 172.21.15.1:0/914619396 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae800cd80 con 0x7ffaf81011e0 2024-09-11T21:57:09.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:09.357+0000 7ffafff4f640 1 -- 172.21.15.1:0/914619396 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffaf819f380 con 0x7ffaf81011e0 2024-09-11T21:57:09.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:09.357+0000 7ffafff4f640 1 -- 172.21.15.1:0/914619396 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffaf81ab670 con 0x7ffaf81011e0 2024-09-11T21:57:09.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:09.357+0000 7ffaeeffd640 1 -- 172.21.15.1:0/914619396 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffae8005500 con 0x7ffaf81011e0 2024-09-11T21:57:09.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:09.357+0000 7ffaeeffd640 1 -- 172.21.15.1:0/914619396 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae8010690 con 0x7ffaf81011e0 2024-09-11T21:57:09.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:09.359+0000 7ffafff4f640 1 -- 172.21.15.1:0/914619396 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffaf8105980 con 0x7ffaf81011e0 2024-09-11T21:57:09.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:09.360+0000 7ffaeeffd640 1 -- 172.21.15.1:0/914619396 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 19) ==== 137849+0+0 (secure 0 0 0) 0x7ffae80046c0 con 0x7ffaf81011e0 2024-09-11T21:57:09.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:09.360+0000 7ffaeeffd640 1 --2- 172.21.15.1:0/914619396 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7ffae40a2bd0 0x7ffae40a5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:09.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:09.360+0000 7ffafdceb640 1 -- 172.21.15.1:0/914619396 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7ffae40a2bd0 msgr2=0x7ffae40a5090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/3890350886 2024-09-11T21:57:09.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:09.360+0000 7ffaeeffd640 1 -- 172.21.15.1:0/914619396 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7ffae80cf550 con 0x7ffaf81011e0 2024-09-11T21:57:09.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:09.360+0000 7ffafdceb640 1 --2- 172.21.15.1:0/914619396 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7ffae40a2bd0 0x7ffae40a5090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T21:57:09.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:09.365+0000 7ffaeeffd640 1 -- 172.21.15.1:0/914619396 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffae809c430 con 0x7ffaf81011e0 2024-09-11T21:57:09.393 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 11 21:57:09 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-y[36280]: 2024-09-11T21:57:09.129+0000 7ff2421f9040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T21:57:09.393 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 11 21:57:09 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-z[37416]: 2024-09-11T21:57:09.109+0000 7fcfe6d89040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T21:57:09.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:09.561+0000 7ffafdceb640 1 -- 172.21.15.1:0/914619396 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7ffae40a2bd0 msgr2=0x7ffae40a5090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/3890350886 2024-09-11T21:57:09.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:09.561+0000 7ffafdceb640 1 --2- 172.21.15.1:0/914619396 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7ffae40a2bd0 0x7ffae40a5090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-11T21:57:09.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:09.581+0000 7ffafff4f640 1 -- 172.21.15.1:0/914619396 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ffaf81ab900 con 0x7ffaf81011e0 2024-09-11T21:57:09.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:09.581+0000 7ffaeeffd640 1 -- 172.21.15.1:0/914619396 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) ==== 53+0+73 (secure 0 0 0) 0x7ffae809bd30 con 0x7ffaf81011e0 2024-09-11T21:57:09.582 INFO:teuthology.orchestra.run.smithi001.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-11T21:57:09.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:09.584+0000 7ffafff4f640 1 -- 172.21.15.1:0/914619396 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7ffae40a2bd0 msgr2=0x7ffae40a5090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T21:57:09.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:09.584+0000 7ffafff4f640 1 --2- 172.21.15.1:0/914619396 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7ffae40a2bd0 0x7ffae40a5090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:09.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:09.584+0000 7ffafff4f640 1 -- 172.21.15.1:0/914619396 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffaf81011e0 msgr2=0x7ffaf81a4ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:09.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:09.584+0000 7ffafff4f640 1 --2- 172.21.15.1:0/914619396 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffaf81011e0 0x7ffaf81a4ce0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ffae800e9f0 tx=0x7ffae800edf0 comp rx=0 tx=0).stop 2024-09-11T21:57:09.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:09.584+0000 7ffafff4f640 1 -- 172.21.15.1:0/914619396 shutdown_connections 2024-09-11T21:57:09.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:09.584+0000 7ffafff4f640 1 --2- 172.21.15.1:0/914619396 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7ffae40a2bd0 0x7ffae40a5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:09.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:09.584+0000 7ffafff4f640 1 --2- 172.21.15.1:0/914619396 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffaf8105ec0 0x7ffaf819e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:09.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:09.584+0000 7ffafff4f640 1 --2- 172.21.15.1:0/914619396 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffaf81011e0 0x7ffaf81a4ce0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:09.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:09.584+0000 7ffafff4f640 1 --2- 172.21.15.1:0/914619396 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffaf81008c0 0x7ffaf81a47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:09.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:09.584+0000 7ffafff4f640 1 -- 172.21.15.1:0/914619396 >> 172.21.15.1:0/914619396 conn(0x7ffaf80fc7d0 msgr2=0x7ffaf8103950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:09.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:09.584+0000 7ffafff4f640 1 -- 172.21.15.1:0/914619396 shutdown_connections 2024-09-11T21:57:09.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:09.584+0000 7ffafff4f640 1 -- 172.21.15.1:0/914619396 wait complete. 2024-09-11T21:57:09.673 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:09 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/905611082' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-11T21:57:09.673 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:09 smithi161 ceph-mon[35615]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-11T21:57:09.673 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 11 21:57:09 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-z[37416]: 2024-09-11T21:57:09.425+0000 7fcfe6d89040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T21:57:09.673 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 11 21:57:09 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-z[37416]: 2024-09-11T21:57:09.486+0000 7fcfe6d89040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T21:57:09.673 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 11 21:57:09 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-z[37416]: 2024-09-11T21:57:09.671+0000 7fcfe6d89040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T21:57:09.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:09 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/905611082' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-11T21:57:09.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:09 smithi001 ceph-mon[30935]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-11T21:57:09.693 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:57:09 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:57:09.498+0000 7f5faf926040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T21:57:09.693 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:57:09 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:57:09.562+0000 7f5faf926040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T21:57:09.712 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:09 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/905611082' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-11T21:57:09.712 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:09 smithi053 ceph-mon[34908]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-11T21:57:09.713 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 11 21:57:09 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-y[36280]: 2024-09-11T21:57:09.456+0000 7ff2421f9040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T21:57:09.713 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 11 21:57:09 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-y[36280]: 2024-09-11T21:57:09.519+0000 7ff2421f9040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T21:57:09.713 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 11 21:57:09 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-y[36280]: 2024-09-11T21:57:09.711+0000 7ff2421f9040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T21:57:10.193 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:57:09 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:57:09.760+0000 7f5faf926040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T21:57:10.343 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-11T21:57:10.343 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-11T21:57:10.343 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi001.front.sepia.ceph.com 2024-09-11T21:57:10.343 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-11T21:57:10.343 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-11T21:57:10.343 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-11T21:57:10.344 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-11T21:57:10.344 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-11T21:57:10.344 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-11T21:57:10.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T21:57:10.344 DEBUG:teuthology.orchestra.run.smithi001:> ip addr 2024-09-11T21:57:10.361 INFO:teuthology.orchestra.run.smithi001.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-11T21:57:10.361 INFO:teuthology.orchestra.run.smithi001.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-11T21:57:10.362 INFO:teuthology.orchestra.run.smithi001.stdout: inet 127.0.0.1/8 scope host lo 2024-09-11T21:57:10.362 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-09-11T21:57:10.362 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 ::1/128 scope host 2024-09-11T21:57:10.362 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-09-11T21:57:10.362 INFO:teuthology.orchestra.run.smithi001.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-11T21:57:10.362 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:6b:f6:7a brd ff:ff:ff:ff:ff:ff 2024-09-11T21:57:10.362 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp7s0f0 2024-09-11T21:57:10.362 INFO:teuthology.orchestra.run.smithi001.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-11T21:57:10.362 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:6b:f6:7b brd ff:ff:ff:ff:ff:ff 2024-09-11T21:57:10.362 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp7s0f1 2024-09-11T21:57:10.363 INFO:teuthology.orchestra.run.smithi001.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-11T21:57:10.363 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:bd:15:e8 brd ff:ff:ff:ff:ff:ff 2024-09-11T21:57:10.363 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp1s0f0 2024-09-11T21:57:10.363 INFO:teuthology.orchestra.run.smithi001.stdout: inet 172.21.15.1/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-11T21:57:10.363 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft 42172sec preferred_lft 42172sec 2024-09-11T21:57:10.363 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::ec4:7aff:febd:15e8/64 scope link 2024-09-11T21:57:10.363 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-09-11T21:57:10.363 INFO:teuthology.orchestra.run.smithi001.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-11T21:57:10.363 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:bd:15:e9 brd ff:ff:ff:ff:ff:ff 2024-09-11T21:57:10.363 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp1s0f1 2024-09-11T21:57:10.364 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-11T21:57:10.364 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T21:57:10.364 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-09-11T21:57:10.364 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link add name ceph-brx type bridge 2024-09-11T21:57:10.364 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip addr flush dev ceph-brx 2024-09-11T21:57:10.364 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link set ceph-brx up 2024-09-11T21:57:10.364 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-11T21:57:10.364 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-09-11T21:57:10.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T21:57:10.563 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:10 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/914619396' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-11T21:57:10.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T21:57:10.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T21:57:10.582 DEBUG:teuthology.orchestra.run.smithi001:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-11T21:57:10.607 INFO:teuthology.orchestra.run.smithi001.stdout:1 2024-09-11T21:57:10.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T21:57:10.610 DEBUG:teuthology.orchestra.run.smithi001:> route 2024-09-11T21:57:10.681 INFO:teuthology.orchestra.run.smithi001.stdout:Kernel IP routing table 2024-09-11T21:57:10.681 INFO:teuthology.orchestra.run.smithi001.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-11T21:57:10.681 INFO:teuthology.orchestra.run.smithi001.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-11T21:57:10.681 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-11T21:57:10.681 INFO:teuthology.orchestra.run.smithi001.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-11T21:57:10.682 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T21:57:10.682 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-09-11T21:57:10.682 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-11T21:57:10.682 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-11T21:57:10.682 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-11T21:57:10.682 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-09-11T21:57:10.700 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:10 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/914619396' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-11T21:57:10.700 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 11 21:57:10 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-y[36280]: 2024-09-11T21:57:10.629+0000 7ff2421f9040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T21:57:10.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T21:57:10.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:10 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/914619396' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-11T21:57:10.842 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 11 21:57:10 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-z[37416]: 2024-09-11T21:57:10.563+0000 7fcfe6d89040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T21:57:10.842 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 11 21:57:10 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-z[37416]: 2024-09-11T21:57:10.630+0000 7fcfe6d89040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T21:57:10.892 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:57:10 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:57:10.728+0000 7f5faf926040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T21:57:10.892 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:57:10 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:57:10.801+0000 7f5faf926040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T21:57:10.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T21:57:10.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T21:57:10.993 DEBUG:teuthology.orchestra.run.smithi001:> ip netns list 2024-09-11T21:57:11.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T21:57:11.010 DEBUG:teuthology.orchestra.run.smithi001:> ip netns list-id 2024-09-11T21:57:11.047 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 11 21:57:10 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-y[36280]: 2024-09-11T21:57:10.699+0000 7ff2421f9040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T21:57:11.066 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T21:57:11.066 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-09-11T21:57:11.066 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-11T21:57:11.066 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-11T21:57:11.066 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-09-11T21:57:11.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T21:57:11.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T21:57:11.189 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-11T21:57:11.189 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T21:57:11.189 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-09-11T21:57:11.189 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-11T21:57:11.189 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-11T21:57:11.189 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-11T21:57:11.189 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-11T21:57:11.189 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-11T21:57:11.190 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-09-11T21:57:11.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T21:57:11.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T21:57:11.336 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T21:57:11.336 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-09-11T21:57:11.336 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link set brx.0 up 2024-09-11T21:57:11.336 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link set dev brx.0 master ceph-brx 2024-09-11T21:57:11.336 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-09-11T21:57:11.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T21:57:11.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T21:57:11.417 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T21:57:11.417 DEBUG:teuthology.orchestra.run.smithi001:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-11T21:57:11.441 INFO:teuthology.orchestra.run.smithi001.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-11T21:57:11.441 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T21:57:11.441 DEBUG:teuthology.orchestra.run.smithi001:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-11T21:57:11.497 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-11T21:57:11.497 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-11T21:57:11.497 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/799b6bbd-6a94-447f-9bf5-8b4eb83a6f36 2024-09-11T21:57:11.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T21:57:11.497 DEBUG:teuthology.orchestra.run.smithi001:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/799b6bbd-6a94-447f-9bf5-8b4eb83a6f36 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-09-11T21:57:11.598 INFO:teuthology.orchestra.run.smithi001.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-09-11T21:57:11.598 INFO:teuthology.orchestra.run.smithi001.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync". 2024-09-11T21:57:11.614 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 11 21:57:11 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-z[37416]: 2024-09-11T21:57:11.350+0000 7fcfe6d89040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T21:57:11.614 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 11 21:57:11 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-z[37416]: 2024-09-11T21:57:11.469+0000 7fcfe6d89040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T21:57:11.711 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 11 21:57:11 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-y[36280]: 2024-09-11T21:57:11.442+0000 7ff2421f9040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T21:57:11.712 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 11 21:57:11 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-y[36280]: 2024-09-11T21:57:11.562+0000 7ff2421f9040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T21:57:11.741 INFO:teuthology.orchestra.run.smithi001.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-09-11T21:57:11.741 INFO:teuthology.orchestra.run.smithi001.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync". 2024-09-11T21:57:11.741 INFO:teuthology.orchestra.run.smithi001.stdout:mount.ceph: resolved to: "172.21.15.1:3300,172.21.15.53:3300,172.21.15.161:3300" 2024-09-11T21:57:11.741 INFO:teuthology.orchestra.run.smithi001.stdout:mount.ceph: trying mount with new device syntax: 0@3ffca83c-7087-11ef-bcea-c7b262605968.cephfs=/volumes/qa/sv_1/799b6bbd-6a94-447f-9bf5-8b4eb83a6f36 2024-09-11T21:57:11.741 INFO:teuthology.orchestra.run.smithi001.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync,name=0,key=0,mon_addr=172.21.15.1:3300/172.21.15.53:3300/172.21.15.161:3300" will pass to kernel 2024-09-11T21:57:11.741 INFO:teuthology.orchestra.run.smithi001.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-11T21:57:11.742 INFO:teuthology.orchestra.run.smithi001.stdout: You just mounted a file system that supports labels which does not 2024-09-11T21:57:11.742 INFO:teuthology.orchestra.run.smithi001.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-11T21:57:11.742 INFO:teuthology.orchestra.run.smithi001.stdout: applications will generate AVC messages and not be allowed access to 2024-09-11T21:57:11.742 INFO:teuthology.orchestra.run.smithi001.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-11T21:57:11.746 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-11T21:57:11.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T21:57:11.746 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-11T21:57:11.773 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-11T21:57:11.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:11 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T21:57:11.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T21:57:11.871 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:57:11 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:57:11.588+0000 7f5faf926040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T21:57:11.871 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:57:11 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:57:11.713+0000 7f5faf926040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T21:57:11.872 INFO:teuthology.orchestra.run.smithi001.stdout:client24881 2024-09-11T21:57:11.873 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-11T21:57:11.873 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T21:57:11.890 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-11T21:57:11.892 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 11 21:57:11 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-z[37416]: 2024-09-11T21:57:11.613+0000 7fcfe6d89040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T21:57:11.893 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 11 21:57:11 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-z[37416]: 2024-09-11T21:57:11.674+0000 7fcfe6d89040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T21:57:11.893 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 11 21:57:11 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-z[37416]: 2024-09-11T21:57:11.822+0000 7fcfe6d89040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T21:57:11.893 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 11 21:57:11 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-z[37416]: 2024-09-11T21:57:11.891+0000 7fcfe6d89040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T21:57:11.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:11 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T21:57:11.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T21:57:11.990 INFO:teuthology.orchestra.run.smithi001.stdout:client24881 2024-09-11T21:57:11.990 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-11T21:57:12.003 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 11 21:57:11 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-y[36280]: 2024-09-11T21:57:11.710+0000 7ff2421f9040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T21:57:12.004 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 11 21:57:11 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-y[36280]: 2024-09-11T21:57:11.776+0000 7ff2421f9040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T21:57:12.004 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 11 21:57:11 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-y[36280]: 2024-09-11T21:57:11.932+0000 7ff2421f9040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T21:57:12.004 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 11 21:57:12 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-y[36280]: 2024-09-11T21:57:12.003+0000 7ff2421f9040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T21:57:12.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T21:57:12.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T21:57:12.049 INFO:teuthology.orchestra.run.smithi001.stdout:client24881 2024-09-11T21:57:12.049 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/3ffca83c-7087-11ef-bcea-c7b262605968.client24881/status') 2024-09-11T21:57:12.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T21:57:12.112 INFO:teuthology.orchestra.run.smithi001.stdout:instance: client.24881 (3)172.21.15.1:0/1150385348 2024-09-11T21:57:12.113 INFO:teuthology.orchestra.run.smithi001.stdout:blocklisted: false 2024-09-11T21:57:12.113 INFO:teuthology.orchestra.run.smithi001.stderr:0+1 records in 2024-09-11T21:57:12.113 INFO:teuthology.orchestra.run.smithi001.stderr:0+1 records out 2024-09-11T21:57:12.113 INFO:teuthology.orchestra.run.smithi001.stderr:70 bytes copied, 2.9964e-05 s, 2.3 MB/s 2024-09-11T21:57:12.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T21:57:12.120 INFO:teuthology.run_tasks:Running task exec... 2024-09-11T21:57:12.129 INFO:teuthology.task.exec:Executing custom commands... 2024-09-11T21:57:12.129 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi001.front.sepia.ceph.com 2024-09-11T21:57:12.129 DEBUG:teuthology.orchestra.run.smithi001:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-09-11T21:57:12.151 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:57:11 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:57:11.870+0000 7f5faf926040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T21:57:12.151 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:57:11 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:57:11.935+0000 7f5faf926040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T21:57:12.152 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:57:12 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:57:12.100+0000 7f5faf926040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T21:57:12.182 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 11 21:57:11 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-z[37416]: 2024-09-11T21:57:11.951+0000 7fcfe6d89040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T21:57:12.182 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 11 21:57:12 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-z[37416]: 2024-09-11T21:57:12.113+0000 7fcfe6d89040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T21:57:12.182 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 11 21:57:12 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-z[37416]: 2024-09-11T21:57:12.181+0000 7fcfe6d89040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T21:57:12.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.235+0000 7f769401e640 1 -- 172.21.15.1:0/2234969431 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f768c101c30 msgr2=0x7f768c1020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:12.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.235+0000 7f769401e640 1 --2- 172.21.15.1:0/2234969431 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f768c101c30 0x7f768c1020b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f767c009900 tx=0x7f767c02a410 comp rx=0 tx=0).stop 2024-09-11T21:57:12.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.235+0000 7f769401e640 1 -- 172.21.15.1:0/2234969431 shutdown_connections 2024-09-11T21:57:12.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.235+0000 7f769401e640 1 --2- 172.21.15.1:0/2234969431 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f768c1025f0 0x7f768c108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:12.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.235+0000 7f769401e640 1 --2- 172.21.15.1:0/2234969431 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f768c101c30 0x7f768c1020b0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:12.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.235+0000 7f769401e640 1 --2- 172.21.15.1:0/2234969431 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f768c100a40 0x7f768c100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:12.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.235+0000 7f769401e640 1 -- 172.21.15.1:0/2234969431 >> 172.21.15.1:0/2234969431 conn(0x7f768c0fc1c0 msgr2=0x7f768c0fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:12.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.235+0000 7f769401e640 1 -- 172.21.15.1:0/2234969431 shutdown_connections 2024-09-11T21:57:12.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.235+0000 7f769401e640 1 -- 172.21.15.1:0/2234969431 wait complete. 2024-09-11T21:57:12.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.236+0000 7f769401e640 1 Processor -- start 2024-09-11T21:57:12.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.236+0000 7f769401e640 1 -- start start 2024-09-11T21:57:12.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.236+0000 7f769401e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f768c100a40 0x7f768c197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:12.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.236+0000 7f769401e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f768c101c30 0x7f768c1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:12.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.236+0000 7f769401e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f768c1025f0 0x7f768c19f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:12.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.236+0000 7f769401e640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f768c10c6c0 con 0x7f768c1025f0 2024-09-11T21:57:12.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.236+0000 7f769401e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f768c10c540 con 0x7f768c101c30 2024-09-11T21:57:12.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.236+0000 7f769401e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f768c10c840 con 0x7f768c100a40 2024-09-11T21:57:12.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.236+0000 7f76925bb640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f768c1025f0 0x7f768c19f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:12.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.236+0000 7f76925bb640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f768c1025f0 0x7f768c19f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46672/0 (socket says 172.21.15.1:46672) 2024-09-11T21:57:12.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.236+0000 7f76925bb640 1 -- 172.21.15.1:0/4261431762 learned_addr learned my addr 172.21.15.1:0/4261431762 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:12.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.236+0000 7f76915b9640 1 --2- 172.21.15.1:0/4261431762 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f768c101c30 0x7f768c1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:12.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.236+0000 7f7691dba640 1 --2- 172.21.15.1:0/4261431762 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f768c100a40 0x7f768c197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:12.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.236+0000 7f76925bb640 1 -- 172.21.15.1:0/4261431762 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f768c100a40 msgr2=0x7f768c197e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:12.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.236+0000 7f76925bb640 1 --2- 172.21.15.1:0/4261431762 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f768c100a40 0x7f768c197e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:12.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.236+0000 7f76925bb640 1 -- 172.21.15.1:0/4261431762 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f768c101c30 msgr2=0x7f768c1983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:12.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.236+0000 7f76925bb640 1 --2- 172.21.15.1:0/4261431762 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f768c101c30 0x7f768c1983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:12.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.236+0000 7f76925bb640 1 -- 172.21.15.1:0/4261431762 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f768c19fab0 con 0x7f768c1025f0 2024-09-11T21:57:12.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.236+0000 7f76915b9640 1 --2- 172.21.15.1:0/4261431762 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f768c101c30 0x7f768c1983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T21:57:12.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.237+0000 7f76925bb640 1 --2- 172.21.15.1:0/4261431762 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f768c1025f0 0x7f768c19f440 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f768400eb10 tx=0x7f768400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:12.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.237+0000 7f7682ffd640 1 -- 172.21.15.1:0/4261431762 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7684004b20 con 0x7f768c1025f0 2024-09-11T21:57:12.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.237+0000 7f769401e640 1 -- 172.21.15.1:0/4261431762 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f768c19fda0 con 0x7f768c1025f0 2024-09-11T21:57:12.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.237+0000 7f7682ffd640 1 -- 172.21.15.1:0/4261431762 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7684004cc0 con 0x7f768c1025f0 2024-09-11T21:57:12.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.237+0000 7f7682ffd640 1 -- 172.21.15.1:0/4261431762 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7684010760 con 0x7f768c1025f0 2024-09-11T21:57:12.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.237+0000 7f769401e640 1 -- 172.21.15.1:0/4261431762 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f768c1a0270 con 0x7f768c1025f0 2024-09-11T21:57:12.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.238+0000 7f769401e640 1 -- 172.21.15.1:0/4261431762 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f768c1a0500 con 0x7f768c1025f0 2024-09-11T21:57:12.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.239+0000 7f7682ffd640 1 -- 172.21.15.1:0/4261431762 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 19) ==== 137849+0+0 (secure 0 0 0) 0x7f7684005020 con 0x7f768c1025f0 2024-09-11T21:57:12.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.240+0000 7f7682ffd640 1 --2- 172.21.15.1:0/4261431762 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f76780a2bd0 0x7f76780a5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:12.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.240+0000 7f7691dba640 1 -- 172.21.15.1:0/4261431762 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f76780a2bd0 msgr2=0x7f76780a5090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/3890350886 2024-09-11T21:57:12.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.240+0000 7f7682ffd640 1 -- 172.21.15.1:0/4261431762 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f76840cf760 con 0x7f768c1025f0 2024-09-11T21:57:12.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.240+0000 7f7691dba640 1 --2- 172.21.15.1:0/4261431762 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f76780a2bd0 0x7f76780a5090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T21:57:12.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.244+0000 7f7682ffd640 1 -- 172.21.15.1:0/4261431762 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f768409c6a0 con 0x7f768c1025f0 2024-09-11T21:57:12.297 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 11 21:57:12 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-y[36280]: 2024-09-11T21:57:12.065+0000 7ff2421f9040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T21:57:12.297 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 11 21:57:12 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-y[36280]: 2024-09-11T21:57:12.233+0000 7ff2421f9040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T21:57:12.412 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:57:12 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:57:12.176+0000 7f5faf926040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T21:57:12.413 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:57:12 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:57:12.242+0000 7f5faf926040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T21:57:12.413 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:57:12 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:57:12.412+0000 7f5faf926040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T21:57:12.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.441+0000 7f7691dba640 1 -- 172.21.15.1:0/4261431762 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f76780a2bd0 msgr2=0x7f76780a5090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/3890350886 2024-09-11T21:57:12.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.441+0000 7f7691dba640 1 --2- 172.21.15.1:0/4261431762 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f76780a2bd0 0x7f76780a5090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-11T21:57:12.452 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 11 21:57:12 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-z[37416]: 2024-09-11T21:57:12.248+0000 7fcfe6d89040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T21:57:12.453 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 11 21:57:12 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-z[37416]: 2024-09-11T21:57:12.332+0000 7fcfe6d89040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T21:57:12.453 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 11 21:57:12 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-z[37416]: 2024-09-11T21:57:12.392+0000 7fcfe6d89040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T21:57:12.453 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 11 21:57:12 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-z[37416]: 2024-09-11T21:57:12.451+0000 7fcfe6d89040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T21:57:12.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.461+0000 7f769401e640 1 -- 172.21.15.1:0/4261431762 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f768c1a08f0 con 0x7f768c1025f0 2024-09-11T21:57:12.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.513+0000 7f7682ffd640 1 -- 172.21.15.1:0/4261431762 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v28) ==== 117+0+0 (secure 0 0 0) 0x7f768409bfa0 con 0x7f768c1025f0 2024-09-11T21:57:12.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.516+0000 7f769401e640 1 -- 172.21.15.1:0/4261431762 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f76780a2bd0 msgr2=0x7f76780a5090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T21:57:12.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.516+0000 7f769401e640 1 --2- 172.21.15.1:0/4261431762 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f76780a2bd0 0x7f76780a5090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:12.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.516+0000 7f769401e640 1 -- 172.21.15.1:0/4261431762 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f768c1025f0 msgr2=0x7f768c19f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:12.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.516+0000 7f769401e640 1 --2- 172.21.15.1:0/4261431762 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f768c1025f0 0x7f768c19f440 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f768400eb10 tx=0x7f768400c2d0 comp rx=0 tx=0).stop 2024-09-11T21:57:12.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.516+0000 7f769401e640 1 -- 172.21.15.1:0/4261431762 shutdown_connections 2024-09-11T21:57:12.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.516+0000 7f769401e640 1 --2- 172.21.15.1:0/4261431762 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f76780a2bd0 0x7f76780a5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:12.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.516+0000 7f769401e640 1 --2- 172.21.15.1:0/4261431762 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f768c1025f0 0x7f768c19f440 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:12.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.516+0000 7f769401e640 1 --2- 172.21.15.1:0/4261431762 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f768c101c30 0x7f768c1983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:12.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.516+0000 7f769401e640 1 --2- 172.21.15.1:0/4261431762 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f768c100a40 0x7f768c197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:12.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.516+0000 7f769401e640 1 -- 172.21.15.1:0/4261431762 >> 172.21.15.1:0/4261431762 conn(0x7f768c0fc1c0 msgr2=0x7f768c069a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:12.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.516+0000 7f769401e640 1 -- 172.21.15.1:0/4261431762 shutdown_connections 2024-09-11T21:57:12.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.516+0000 7f769401e640 1 -- 172.21.15.1:0/4261431762 wait complete. 2024-09-11T21:57:12.528 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-11T21:57:12.538 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-09-11T21:57:12.548 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T21:57:12.581 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:12 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/4261431762' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-11T21:57:12.581 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 11 21:57:12 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-y[36280]: 2024-09-11T21:57:12.301+0000 7ff2421f9040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T21:57:12.581 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 11 21:57:12 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-y[36280]: 2024-09-11T21:57:12.370+0000 7ff2421f9040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T21:57:12.581 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 11 21:57:12 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-y[36280]: 2024-09-11T21:57:12.456+0000 7ff2421f9040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T21:57:12.581 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 11 21:57:12 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-y[36280]: 2024-09-11T21:57:12.518+0000 7ff2421f9040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T21:57:12.581 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 11 21:57:12 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-y[36280]: 2024-09-11T21:57:12.580+0000 7ff2421f9040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T21:57:12.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.673+0000 7fd2649fd640 1 -- 172.21.15.1:0/2820776259 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd260101c40 msgr2=0x7fd2601020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:12.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.673+0000 7fd2649fd640 1 --2- 172.21.15.1:0/2820776259 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd260101c40 0x7fd2601020c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd254009860 tx=0x7fd25402a340 comp rx=0 tx=0).stop 2024-09-11T21:57:12.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.673+0000 7fd2649fd640 1 -- 172.21.15.1:0/2820776259 shutdown_connections 2024-09-11T21:57:12.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.673+0000 7fd2649fd640 1 --2- 172.21.15.1:0/2820776259 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd260102600 0x7fd260108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:12.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.673+0000 7fd2649fd640 1 --2- 172.21.15.1:0/2820776259 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd260101c40 0x7fd2601020c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:12.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.673+0000 7fd2649fd640 1 --2- 172.21.15.1:0/2820776259 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd260100a50 0x7fd260100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:12.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.674+0000 7fd2649fd640 1 -- 172.21.15.1:0/2820776259 >> 172.21.15.1:0/2820776259 conn(0x7fd2600fc1f0 msgr2=0x7fd2600fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:12.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.674+0000 7fd2649fd640 1 -- 172.21.15.1:0/2820776259 shutdown_connections 2024-09-11T21:57:12.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.674+0000 7fd2649fd640 1 -- 172.21.15.1:0/2820776259 wait complete. 2024-09-11T21:57:12.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.674+0000 7fd2649fd640 1 Processor -- start 2024-09-11T21:57:12.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.674+0000 7fd2649fd640 1 -- start start 2024-09-11T21:57:12.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.674+0000 7fd2649fd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd260100a50 0x7fd260197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:12.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.674+0000 7fd2649fd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd260101c40 0x7fd2601983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:12.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.674+0000 7fd2649fd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd260102600 0x7fd26019f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:12.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.674+0000 7fd2649fd640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd26010c6d0 con 0x7fd260100a50 2024-09-11T21:57:12.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.674+0000 7fd2649fd640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd26010c550 con 0x7fd260101c40 2024-09-11T21:57:12.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.675+0000 7fd2649fd640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd26010c850 con 0x7fd260102600 2024-09-11T21:57:12.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.675+0000 7fd25e59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd260100a50 0x7fd260197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:12.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.675+0000 7fd25dd9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd260101c40 0x7fd2601983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:12.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.675+0000 7fd25e59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd260100a50 0x7fd260197ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46694/0 (socket says 172.21.15.1:46694) 2024-09-11T21:57:12.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.675+0000 7fd25e59c640 1 -- 172.21.15.1:0/3624341793 learned_addr learned my addr 172.21.15.1:0/3624341793 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:12.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.675+0000 7fd25ed9d640 1 --2- 172.21.15.1:0/3624341793 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd260102600 0x7fd26019f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:12.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.675+0000 7fd25dd9b640 1 -- 172.21.15.1:0/3624341793 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd260102600 msgr2=0x7fd26019f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:12.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.675+0000 7fd25dd9b640 1 --2- 172.21.15.1:0/3624341793 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd260102600 0x7fd26019f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:12.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.675+0000 7fd25dd9b640 1 -- 172.21.15.1:0/3624341793 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd260100a50 msgr2=0x7fd260197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:12.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.675+0000 7fd25dd9b640 1 --2- 172.21.15.1:0/3624341793 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd260100a50 0x7fd260197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:12.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.675+0000 7fd25dd9b640 1 -- 172.21.15.1:0/3624341793 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd26019fac0 con 0x7fd260101c40 2024-09-11T21:57:12.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.675+0000 7fd25dd9b640 1 --2- 172.21.15.1:0/3624341793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd260101c40 0x7fd2601983e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd254035040 tx=0x7fd254033d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:12.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.675+0000 7fd24f7fe640 1 -- 172.21.15.1:0/3624341793 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd254037040 con 0x7fd260101c40 2024-09-11T21:57:12.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.675+0000 7fd2649fd640 1 -- 172.21.15.1:0/3624341793 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd26019fd50 con 0x7fd260101c40 2024-09-11T21:57:12.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.675+0000 7fd24f7fe640 1 -- 172.21.15.1:0/3624341793 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd254004040 con 0x7fd260101c40 2024-09-11T21:57:12.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.675+0000 7fd24f7fe640 1 -- 172.21.15.1:0/3624341793 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2540055c0 con 0x7fd260101c40 2024-09-11T21:57:12.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.675+0000 7fd2649fd640 1 -- 172.21.15.1:0/3624341793 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2601a01e0 con 0x7fd260101c40 2024-09-11T21:57:12.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.676+0000 7fd2649fd640 1 -- 172.21.15.1:0/3624341793 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2601a0410 con 0x7fd260101c40 2024-09-11T21:57:12.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.678+0000 7fd24f7fe640 1 -- 172.21.15.1:0/3624341793 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) ==== 137849+0+0 (secure 0 0 0) 0x7fd254042020 con 0x7fd260101c40 2024-09-11T21:57:12.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.678+0000 7fd24f7fe640 1 --2- 172.21.15.1:0/3624341793 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fd2480a2be0 0x7fd2480a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:12.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.678+0000 7fd25e59c640 1 -- 172.21.15.1:0/3624341793 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fd2480a2be0 msgr2=0x7fd2480a50a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/3890350886 2024-09-11T21:57:12.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.678+0000 7fd24f7fe640 1 -- 172.21.15.1:0/3624341793 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fd2540eebb0 con 0x7fd260101c40 2024-09-11T21:57:12.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.678+0000 7fd25e59c640 1 --2- 172.21.15.1:0/3624341793 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fd2480a2be0 0x7fd2480a50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T21:57:12.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.681+0000 7fd24f7fe640 1 -- 172.21.15.1:0/3624341793 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd254049020 con 0x7fd260101c40 2024-09-11T21:57:12.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:12 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/4261431762' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-11T21:57:12.693 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:57:12 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:57:12.481+0000 7f5faf926040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T21:57:12.693 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:57:12 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:57:12.553+0000 7f5faf926040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T21:57:12.693 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:57:12 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:57:12.643+0000 7f5faf926040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T21:57:12.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:12 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/4261431762' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-11T21:57:12.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.858+0000 7fd2649fd640 1 -- 172.21.15.1:0/3624341793 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd2601a07a0 con 0x7fd260101c40 2024-09-11T21:57:12.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.860+0000 7fd24f7fe640 1 -- 172.21.15.1:0/3624341793 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22669 (secure 0 0 0) 0x7fd2540bbaf0 con 0x7fd260101c40 2024-09-11T21:57:12.861 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:57:12.861 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":86,"fsid":"3ffca83c-7087-11ef-bcea-c7b262605968","created":"2024-09-11T21:47:28.868707+0000","modified":"2024-09-11T21:56:26.278614+0000","last_up_change":"2024-09-11T21:54:29.319441+0000","last_in_change":"2024-09-11T21:54:05.532281+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T21:50:41.026490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T21:55:16.961999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T21:55:19.066228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"19e56504-18f1-4234-b2d6-f193ff47d8eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":4012142023},{"type":"v1","addr":"172.21.15.1:6803","nonce":4012142023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":4012142023},{"type":"v1","addr":"172.21.15.1:6805","nonce":4012142023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":4012142023},{"type":"v1","addr":"172.21.15.1:6809","nonce":4012142023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":4012142023},{"type":"v1","addr":"172.21.15.1:6807","nonce":4012142023}]},"public_addr":"172.21.15.1:6803/4012142023","cluster_addr":"172.21.15.1:6805/4012142023","heartbeat_back_addr":"172.21.15.1:6809/4012142023","heartbeat_front_addr":"172.21.15.1:6807/4012142023","state":["exists","up"]},{"osd":1,"uuid":"996c9f0c-c5a4-43b8-b8e0-d712f27c5dfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3039391924},{"type":"v1","addr":"172.21.15.53:6801","nonce":3039391924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3039391924},{"type":"v1","addr":"172.21.15.53:6803","nonce":3039391924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3039391924},{"type":"v1","addr":"172.21.15.53:6807","nonce":3039391924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3039391924},{"type":"v1","addr":"172.21.15.53:6805","nonce":3039391924}]},"public_addr":"172.21.15.53:6801/3039391924","cluster_addr":"172.21.15.53:6803/3039391924","heartbeat_back_addr":"172.21.15.53:6807/3039391924","heartbeat_front_addr":"172.21.15.53:6805/3039391924","state":["exists","up"]},{"osd":2,"uuid":"578d4260-41e1-4645-b9b1-1c5445712c13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1707914885},{"type":"v1","addr":"172.21.15.161:6801","nonce":1707914885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1707914885},{"type":"v1","addr":"172.21.15.161:6803","nonce":1707914885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1707914885},{"type":"v1","addr":"172.21.15.161:6807","nonce":1707914885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1707914885},{"type":"v1","addr":"172.21.15.161:6805","nonce":1707914885}]},"public_addr":"172.21.15.161:6801/1707914885","cluster_addr":"172.21.15.161:6803/1707914885","heartbeat_back_addr":"172.21.15.161:6807/1707914885","heartbeat_front_addr":"172.21.15.161:6805/1707914885","state":["exists","up"]},{"osd":3,"uuid":"73f78766-9662-4aa3-841c-fe2ce654ee9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1541142398},{"type":"v1","addr":"172.21.15.1:6811","nonce":1541142398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1541142398},{"type":"v1","addr":"172.21.15.1:6813","nonce":1541142398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1541142398},{"type":"v1","addr":"172.21.15.1:6817","nonce":1541142398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1541142398},{"type":"v1","addr":"172.21.15.1:6815","nonce":1541142398}]},"public_addr":"172.21.15.1:6811/1541142398","cluster_addr":"172.21.15.1:6813/1541142398","heartbeat_back_addr":"172.21.15.1:6817/1541142398","heartbeat_front_addr":"172.21.15.1:6815/1541142398","state":["exists","up"]},{"osd":4,"uuid":"56c808bb-d4cd-4969-b49e-adef49ecd0f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":3081402681},{"type":"v1","addr":"172.21.15.53:6809","nonce":3081402681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3081402681},{"type":"v1","addr":"172.21.15.53:6811","nonce":3081402681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3081402681},{"type":"v1","addr":"172.21.15.53:6815","nonce":3081402681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3081402681},{"type":"v1","addr":"172.21.15.53:6813","nonce":3081402681}]},"public_addr":"172.21.15.53:6809/3081402681","cluster_addr":"172.21.15.53:6811/3081402681","heartbeat_back_addr":"172.21.15.53:6815/3081402681","heartbeat_front_addr":"172.21.15.53:6813/3081402681","state":["exists","up"]},{"osd":5,"uuid":"7b8266d6-b507-47c9-b4fa-ff16c4a74d35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":1622081327},{"type":"v1","addr":"172.21.15.161:6809","nonce":1622081327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":1622081327},{"type":"v1","addr":"172.21.15.161:6811","nonce":1622081327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":1622081327},{"type":"v1","addr":"172.21.15.161:6815","nonce":1622081327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":1622081327},{"type":"v1","addr":"172.21.15.161:6813","nonce":1622081327}]},"public_addr":"172.21.15.161:6809/1622081327","cluster_addr":"172.21.15.161:6811/1622081327","heartbeat_back_addr":"172.21.15.161:6815/1622081327","heartbeat_front_addr":"172.21.15.161:6813/1622081327","state":["exists","up"]},{"osd":6,"uuid":"3af5925d-a749-4849-a076-80580222cad4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":158434853},{"type":"v1","addr":"172.21.15.1:6819","nonce":158434853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":158434853},{"type":"v1","addr":"172.21.15.1:6821","nonce":158434853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":158434853},{"type":"v1","addr":"172.21.15.1:6825","nonce":158434853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":158434853},{"type":"v1","addr":"172.21.15.1:6823","nonce":158434853}]},"public_addr":"172.21.15.1:6819/158434853","cluster_addr":"172.21.15.1:6821/158434853","heartbeat_back_addr":"172.21.15.1:6825/158434853","heartbeat_front_addr":"172.21.15.1:6823/158434853","state":["exists","up"]},{"osd":7,"uuid":"f8f26c3c-17e9-4e89-8392-289796e0c8a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2132057697},{"type":"v1","addr":"172.21.15.53:6817","nonce":2132057697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2132057697},{"type":"v1","addr":"172.21.15.53:6819","nonce":2132057697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2132057697},{"type":"v1","addr":"172.21.15.53:6823","nonce":2132057697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2132057697},{"type":"v1","addr":"172.21.15.53:6821","nonce":2132057697}]},"public_addr":"172.21.15.53:6817/2132057697","cluster_addr":"172.21.15.53:6819/2132057697","heartbeat_back_addr":"172.21.15.53:6823/2132057697","heartbeat_front_addr":"172.21.15.53:6821/2132057697","state":["exists","up"]},{"osd":8,"uuid":"36ea401b-3c93-4b9d-8880-298d1eb6e30b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1838248669},{"type":"v1","addr":"172.21.15.161:6817","nonce":1838248669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1838248669},{"type":"v1","addr":"172.21.15.161:6819","nonce":1838248669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1838248669},{"type":"v1","addr":"172.21.15.161:6823","nonce":1838248669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1838248669},{"type":"v1","addr":"172.21.15.161:6821","nonce":1838248669}]},"public_addr":"172.21.15.161:6817/1838248669","cluster_addr":"172.21.15.161:6819/1838248669","heartbeat_back_addr":"172.21.15.161:6823/1838248669","heartbeat_front_addr":"172.21.15.161:6821/1838248669","state":["exists","up"]},{"osd":9,"uuid":"4b788d3f-a3bc-4463-bd99-95fdb7a52768","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2806274589},{"type":"v1","addr":"172.21.15.1:6827","nonce":2806274589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2806274589},{"type":"v1","addr":"172.21.15.1:6829","nonce":2806274589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":2806274589},{"type":"v1","addr":"172.21.15.1:6833","nonce":2806274589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2806274589},{"type":"v1","addr":"172.21.15.1:6831","nonce":2806274589}]},"public_addr":"172.21.15.1:6827/2806274589","cluster_addr":"172.21.15.1:6829/2806274589","heartbeat_back_addr":"172.21.15.1:6833/2806274589","heartbeat_front_addr":"172.21.15.1:6831/2806274589","state":["exists","up"]},{"osd":10,"uuid":"70549796-5270-4ae2-ac3f-f61f5de105d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":761923580},{"type":"v1","addr":"172.21.15.53:6825","nonce":761923580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":761923580},{"type":"v1","addr":"172.21.15.53:6827","nonce":761923580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":761923580},{"type":"v1","addr":"172.21.15.53:6831","nonce":761923580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":761923580},{"type":"v1","addr":"172.21.15.53:6829","nonce":761923580}]},"public_addr":"172.21.15.53:6825/761923580","cluster_addr":"172.21.15.53:6827/761923580","heartbeat_back_addr":"172.21.15.53:6831/761923580","heartbeat_front_addr":"172.21.15.53:6829/761923580","state":["exists","up"]},{"osd":11,"uuid":"48f17c4a-370a-493e-956e-bf6ad3dcc594","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1586352190},{"type":"v1","addr":"172.21.15.161:6825","nonce":1586352190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":1586352190},{"type":"v1","addr":"172.21.15.161:6827","nonce":1586352190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":1586352190},{"type":"v1","addr":"172.21.15.161:6831","nonce":1586352190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":1586352190},{"type":"v1","addr":"172.21.15.161:6829","nonce":1586352190}]},"public_addr":"172.21.15.161:6825/1586352190","cluster_addr":"172.21.15.161:6827/1586352190","heartbeat_back_addr":"172.21.15.161:6831/1586352190","heartbeat_front_addr":"172.21.15.161:6829/1586352190","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:49:46.539613+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:50:11.912300+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:50:36.955490+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:51:02.200554+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:51:27.860585+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:51:54.694099+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:52:18.962351+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:52:44.881923+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:53:10.301390+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:53:35.417647+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:54:01.492219+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:54:26.938338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.26","mappings":[{"from":1,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/1749020850":"2024-09-12T21:48:24.524913+0000","172.21.15.1:6801/2821761445":"2024-09-12T21:48:24.524913+0000","172.21.15.1:0/1699984777":"2024-09-12T21:48:24.524913+0000","172.21.15.1:0/276770268":"2024-09-12T21:48:00.674728+0000","172.21.15.1:0/1278877732":"2024-09-12T21:48:24.524913+0000","172.21.15.1:0/2599895977":"2024-09-12T21:48:00.674728+0000","172.21.15.1:6801/2311309446":"2024-09-12T21:48:00.674728+0000","172.21.15.1:6800/2821761445":"2024-09-12T21:48:24.524913+0000","172.21.15.1:6800/2311309446":"2024-09-12T21:48:00.674728+0000","172.21.15.1:0/3202962495":"2024-09-12T21:48:00.674728+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T21:57:12.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.863+0000 7fd2649fd640 1 -- 172.21.15.1:0/3624341793 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fd2480a2be0 msgr2=0x7fd2480a50a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T21:57:12.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.863+0000 7fd2649fd640 1 --2- 172.21.15.1:0/3624341793 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fd2480a2be0 0x7fd2480a50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:12.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.863+0000 7fd2649fd640 1 -- 172.21.15.1:0/3624341793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd260101c40 msgr2=0x7fd2601983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:12.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.863+0000 7fd2649fd640 1 --2- 172.21.15.1:0/3624341793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd260101c40 0x7fd2601983e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd254035040 tx=0x7fd254033d90 comp rx=0 tx=0).stop 2024-09-11T21:57:12.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.863+0000 7fd2649fd640 1 -- 172.21.15.1:0/3624341793 shutdown_connections 2024-09-11T21:57:12.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.863+0000 7fd2649fd640 1 --2- 172.21.15.1:0/3624341793 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fd2480a2be0 0x7fd2480a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:12.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.863+0000 7fd2649fd640 1 --2- 172.21.15.1:0/3624341793 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd260102600 0x7fd26019f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:12.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.863+0000 7fd2649fd640 1 --2- 172.21.15.1:0/3624341793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd260101c40 0x7fd2601983e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:12.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.863+0000 7fd2649fd640 1 --2- 172.21.15.1:0/3624341793 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd260100a50 0x7fd260197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:12.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.863+0000 7fd2649fd640 1 -- 172.21.15.1:0/3624341793 >> 172.21.15.1:0/3624341793 conn(0x7fd2600fc1f0 msgr2=0x7fd260069520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:12.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.863+0000 7fd2649fd640 1 -- 172.21.15.1:0/3624341793 shutdown_connections 2024-09-11T21:57:12.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.863+0000 7fd2649fd640 1 -- 172.21.15.1:0/3624341793 wait complete. 2024-09-11T21:57:12.879 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T21:50:41.026490+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-11T21:55:16.961999+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-11T21:55:19.066228+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-11T21:57:12.880 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-11T21:57:12.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.987+0000 7f824c931640 1 -- 172.21.15.1:0/2449765518 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82481025e0 msgr2=0x7f8248108e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:12.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.987+0000 7f824c931640 1 --2- 172.21.15.1:0/2449765518 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82481025e0 0x7f8248108e70 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f8240009ca0 tx=0x7f824002a8c0 comp rx=0 tx=0).stop 2024-09-11T21:57:12.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.988+0000 7f824c931640 1 -- 172.21.15.1:0/2449765518 shutdown_connections 2024-09-11T21:57:12.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.988+0000 7f824c931640 1 --2- 172.21.15.1:0/2449765518 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82481025e0 0x7f8248108e70 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:12.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.988+0000 7f824c931640 1 --2- 172.21.15.1:0/2449765518 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8248101c20 0x7f82481020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:12.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.988+0000 7f824c931640 1 --2- 172.21.15.1:0/2449765518 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8248100a30 0x7f8248100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:12.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.988+0000 7f824c931640 1 -- 172.21.15.1:0/2449765518 >> 172.21.15.1:0/2449765518 conn(0x7f82480fc190 msgr2=0x7f82480fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:12.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.988+0000 7f824c931640 1 -- 172.21.15.1:0/2449765518 shutdown_connections 2024-09-11T21:57:12.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.988+0000 7f824c931640 1 -- 172.21.15.1:0/2449765518 wait complete. 2024-09-11T21:57:12.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.988+0000 7f824c931640 1 Processor -- start 2024-09-11T21:57:12.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.988+0000 7f824c931640 1 -- start start 2024-09-11T21:57:12.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.988+0000 7f824c931640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8248100a30 0x7f8248197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:12.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.988+0000 7f824c931640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8248101c20 0x7f8248198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:12.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.988+0000 7f824c931640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82481025e0 0x7f824819f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:12.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.988+0000 7f824c931640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f824810d800 con 0x7f8248100a30 2024-09-11T21:57:12.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.988+0000 7f824c931640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f824810d680 con 0x7f82481025e0 2024-09-11T21:57:12.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.989+0000 7f824c931640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f824810d980 con 0x7f8248101c20 2024-09-11T21:57:12.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.989+0000 7f824659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8248100a30 0x7f8248197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:12.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.989+0000 7f824659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8248100a30 0x7f8248197e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46718/0 (socket says 172.21.15.1:46718) 2024-09-11T21:57:12.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.989+0000 7f824659c640 1 -- 172.21.15.1:0/3724058766 learned_addr learned my addr 172.21.15.1:0/3724058766 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:12.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.989+0000 7f8246d9d640 1 --2- 172.21.15.1:0/3724058766 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82481025e0 0x7f824819f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:12.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.989+0000 7f8245d9b640 1 --2- 172.21.15.1:0/3724058766 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8248101c20 0x7f8248198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:12.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.989+0000 7f8246d9d640 1 -- 172.21.15.1:0/3724058766 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8248101c20 msgr2=0x7f8248198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:12.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.989+0000 7f8246d9d640 1 --2- 172.21.15.1:0/3724058766 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8248101c20 0x7f8248198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:12.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.989+0000 7f8246d9d640 1 -- 172.21.15.1:0/3724058766 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8248100a30 msgr2=0x7f8248197e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:12.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.989+0000 7f8246d9d640 1 --2- 172.21.15.1:0/3724058766 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8248100a30 0x7f8248197e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:12.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.989+0000 7f8246d9d640 1 -- 172.21.15.1:0/3724058766 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f824819fa40 con 0x7f82481025e0 2024-09-11T21:57:12.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.989+0000 7f824659c640 1 --2- 172.21.15.1:0/3724058766 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8248100a30 0x7f8248197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T21:57:12.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.989+0000 7f8246d9d640 1 --2- 172.21.15.1:0/3724058766 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82481025e0 0x7f824819f3d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8240031ae0 tx=0x7f8240004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:12.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.989+0000 7f82377fe640 1 -- 172.21.15.1:0/3724058766 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8240003b70 con 0x7f82481025e0 2024-09-11T21:57:12.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.989+0000 7f82377fe640 1 -- 172.21.15.1:0/3724058766 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8240003d10 con 0x7f82481025e0 2024-09-11T21:57:12.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.989+0000 7f824c931640 1 -- 172.21.15.1:0/3724058766 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f824819fcd0 con 0x7f82481025e0 2024-09-11T21:57:12.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.989+0000 7f82377fe640 1 -- 172.21.15.1:0/3724058766 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824002fc40 con 0x7f82481025e0 2024-09-11T21:57:12.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.990+0000 7f824c931640 1 -- 172.21.15.1:0/3724058766 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82481a01a0 con 0x7f82481025e0 2024-09-11T21:57:12.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.990+0000 7f824c931640 1 -- 172.21.15.1:0/3724058766 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8248107e70 con 0x7f82481025e0 2024-09-11T21:57:12.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.992+0000 7f82377fe640 1 -- 172.21.15.1:0/3724058766 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) ==== 137849+0+0 (secure 0 0 0) 0x7f8240070b40 con 0x7f82481025e0 2024-09-11T21:57:12.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.992+0000 7f82377fe640 1 --2- 172.21.15.1:0/3724058766 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f82300a2be0 0x7f82300a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:12.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.992+0000 7f824659c640 1 -- 172.21.15.1:0/3724058766 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f82300a2be0 msgr2=0x7f82300a50a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/3890350886 2024-09-11T21:57:12.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.992+0000 7f82377fe640 1 -- 172.21.15.1:0/3724058766 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f82400f02c0 con 0x7f82481025e0 2024-09-11T21:57:12.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.992+0000 7f824659c640 1 --2- 172.21.15.1:0/3724058766 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f82300a2be0 0x7f82300a50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T21:57:12.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:12.996+0000 7f82377fe640 1 -- 172.21.15.1:0/3724058766 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f82400bd1a0 con 0x7f82481025e0 2024-09-11T21:57:13.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.172+0000 7f824c931640 1 -- 172.21.15.1:0/3724058766 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f824802d7c0 con 0x7f82481025e0 2024-09-11T21:57:13.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.173+0000 7f82377fe640 1 -- 172.21.15.1:0/3724058766 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) ==== 93+0+10 (secure 0 0 0) 0x7f82400bcaa0 con 0x7f82481025e0 2024-09-11T21:57:13.173 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-09-11T21:57:13.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.175+0000 7f824c931640 1 -- 172.21.15.1:0/3724058766 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f82300a2be0 msgr2=0x7f82300a50a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T21:57:13.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.175+0000 7f824c931640 1 --2- 172.21.15.1:0/3724058766 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f82300a2be0 0x7f82300a50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:13.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.175+0000 7f824c931640 1 -- 172.21.15.1:0/3724058766 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82481025e0 msgr2=0x7f824819f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:13.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.175+0000 7f824c931640 1 --2- 172.21.15.1:0/3724058766 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82481025e0 0x7f824819f3d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8240031ae0 tx=0x7f8240004ab0 comp rx=0 tx=0).stop 2024-09-11T21:57:13.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.175+0000 7f824c931640 1 -- 172.21.15.1:0/3724058766 shutdown_connections 2024-09-11T21:57:13.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.175+0000 7f824c931640 1 --2- 172.21.15.1:0/3724058766 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f82300a2be0 0x7f82300a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:13.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.175+0000 7f824c931640 1 --2- 172.21.15.1:0/3724058766 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f82481025e0 0x7f824819f3d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:13.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.175+0000 7f824c931640 1 --2- 172.21.15.1:0/3724058766 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8248101c20 0x7f8248198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:13.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.175+0000 7f824c931640 1 --2- 172.21.15.1:0/3724058766 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8248100a30 0x7f8248197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:13.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.175+0000 7f824c931640 1 -- 172.21.15.1:0/3724058766 >> 172.21.15.1:0/3724058766 conn(0x7f82480fc190 msgr2=0x7f8248105040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:13.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.175+0000 7f824c931640 1 -- 172.21.15.1:0/3724058766 shutdown_connections 2024-09-11T21:57:13.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.175+0000 7f824c931640 1 -- 172.21.15.1:0/3724058766 wait complete. 2024-09-11T21:57:13.186 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:57:12 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:57:12.708+0000 7f5faf926040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T21:57:13.186 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:57:12 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:57:12.771+0000 7f5faf926040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T21:57:13.187 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-11T21:57:13.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.295+0000 7fa5cb427640 1 -- 172.21.15.1:0/2718175435 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa5c4100a30 msgr2=0x7fa5c4100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:13.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.295+0000 7fa5cb427640 1 --2- 172.21.15.1:0/2718175435 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa5c4100a30 0x7fa5c4100e30 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0009900 tx=0x7fa5c002a410 comp rx=0 tx=0).stop 2024-09-11T21:57:13.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.295+0000 7fa5cb427640 1 -- 172.21.15.1:0/2718175435 shutdown_connections 2024-09-11T21:57:13.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.295+0000 7fa5cb427640 1 --2- 172.21.15.1:0/2718175435 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5c41025e0 0x7fa5c4108e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:13.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.295+0000 7fa5cb427640 1 --2- 172.21.15.1:0/2718175435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5c4101c20 0x7fa5c41020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:13.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.295+0000 7fa5cb427640 1 --2- 172.21.15.1:0/2718175435 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa5c4100a30 0x7fa5c4100e30 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:13.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.295+0000 7fa5cb427640 1 -- 172.21.15.1:0/2718175435 >> 172.21.15.1:0/2718175435 conn(0x7fa5c40fc190 msgr2=0x7fa5c40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:13.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.295+0000 7fa5cb427640 1 -- 172.21.15.1:0/2718175435 shutdown_connections 2024-09-11T21:57:13.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.295+0000 7fa5cb427640 1 -- 172.21.15.1:0/2718175435 wait complete. 2024-09-11T21:57:13.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.296+0000 7fa5cb427640 1 Processor -- start 2024-09-11T21:57:13.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.296+0000 7fa5cb427640 1 -- start start 2024-09-11T21:57:13.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.296+0000 7fa5cb427640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5c4100a30 0x7fa5c410b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:13.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.296+0000 7fa5cb427640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5c4101c20 0x7fa5c41099f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:13.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.296+0000 7fa5cb427640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa5c41025e0 0x7fa5c4109f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:13.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.296+0000 7fa5cb427640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c410d800 con 0x7fa5c41025e0 2024-09-11T21:57:13.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.296+0000 7fa5c91c3640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5c4100a30 0x7fa5c410b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:13.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.296+0000 7fa5cb427640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c410d680 con 0x7fa5c4101c20 2024-09-11T21:57:13.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.296+0000 7fa5c99c4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa5c41025e0 0x7fa5c4109f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:13.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.296+0000 7fa5cb427640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c410d980 con 0x7fa5c4100a30 2024-09-11T21:57:13.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.296+0000 7fa5c99c4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa5c41025e0 0x7fa5c4109f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46750/0 (socket says 172.21.15.1:46750) 2024-09-11T21:57:13.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.296+0000 7fa5c99c4640 1 -- 172.21.15.1:0/1744116788 learned_addr learned my addr 172.21.15.1:0/1744116788 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:13.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.296+0000 7fa5c89c2640 1 --2- 172.21.15.1:0/1744116788 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5c4101c20 0x7fa5c41099f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:13.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.297+0000 7fa5c89c2640 1 -- 172.21.15.1:0/1744116788 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5c4100a30 msgr2=0x7fa5c410b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:13.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.297+0000 7fa5c89c2640 1 --2- 172.21.15.1:0/1744116788 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5c4100a30 0x7fa5c410b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:13.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.297+0000 7fa5c89c2640 1 -- 172.21.15.1:0/1744116788 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa5c41025e0 msgr2=0x7fa5c4109f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:13.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.297+0000 7fa5c89c2640 1 --2- 172.21.15.1:0/1744116788 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa5c41025e0 0x7fa5c4109f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:13.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.297+0000 7fa5c89c2640 1 -- 172.21.15.1:0/1744116788 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5c410a7c0 con 0x7fa5c4101c20 2024-09-11T21:57:13.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.297+0000 7fa5c99c4640 1 --2- 172.21.15.1:0/1744116788 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa5c41025e0 0x7fa5c4109f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T21:57:13.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.297+0000 7fa5c91c3640 1 --2- 172.21.15.1:0/1744116788 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5c4100a30 0x7fa5c410b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T21:57:13.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.297+0000 7fa5c89c2640 1 --2- 172.21.15.1:0/1744116788 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5c4101c20 0x7fa5c41099f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa5b400e9f0 tx=0x7fa5b400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:13.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.297+0000 7fa5ba7fc640 1 -- 172.21.15.1:0/1744116788 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b400cd80 con 0x7fa5c4101c20 2024-09-11T21:57:13.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.297+0000 7fa5cb427640 1 -- 172.21.15.1:0/1744116788 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5c41a1fa0 con 0x7fa5c4101c20 2024-09-11T21:57:13.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.297+0000 7fa5ba7fc640 1 -- 172.21.15.1:0/1744116788 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5b40035f0 con 0x7fa5c4101c20 2024-09-11T21:57:13.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.297+0000 7fa5ba7fc640 1 -- 172.21.15.1:0/1744116788 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b4010670 con 0x7fa5c4101c20 2024-09-11T21:57:13.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.297+0000 7fa5cb427640 1 -- 172.21.15.1:0/1744116788 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5c41a2410 con 0x7fa5c4101c20 2024-09-11T21:57:13.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.299+0000 7fa5cb427640 1 -- 172.21.15.1:0/1744116788 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5c4107e10 con 0x7fa5c4101c20 2024-09-11T21:57:13.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.300+0000 7fa5ba7fc640 1 -- 172.21.15.1:0/1744116788 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 19) ==== 137849+0+0 (secure 0 0 0) 0x7fa5b40037a0 con 0x7fa5c4101c20 2024-09-11T21:57:13.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.300+0000 7fa5ba7fc640 1 --2- 172.21.15.1:0/1744116788 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fa5b00a2be0 0x7fa5b00a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:13.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.300+0000 7fa5c91c3640 1 -- 172.21.15.1:0/1744116788 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fa5b00a2be0 msgr2=0x7fa5b00a50a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/3890350886 2024-09-11T21:57:13.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.300+0000 7fa5ba7fc640 1 -- 172.21.15.1:0/1744116788 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7fa5b40cf400 con 0x7fa5c4101c20 2024-09-11T21:57:13.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.300+0000 7fa5c91c3640 1 --2- 172.21.15.1:0/1744116788 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fa5b00a2be0 0x7fa5b00a50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T21:57:13.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.304+0000 7fa5ba7fc640 1 -- 172.21.15.1:0/1744116788 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa5b409c2e0 con 0x7fa5c4101c20 2024-09-11T21:57:13.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.501+0000 7fa5c91c3640 1 -- 172.21.15.1:0/1744116788 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fa5b00a2be0 msgr2=0x7fa5b00a50a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/3890350886 2024-09-11T21:57:13.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.501+0000 7fa5c91c3640 1 --2- 172.21.15.1:0/1744116788 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fa5b00a2be0 0x7fa5b00a50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-11T21:57:13.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.508+0000 7fa5cb427640 1 -- 172.21.15.1:0/1744116788 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa5c4062df0 con 0x7fa5c4101c20 2024-09-11T21:57:13.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.508+0000 7fa5ba7fc640 1 -- 172.21.15.1:0/1744116788 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v86) ==== 104+0+11 (secure 0 0 0) 0x7fa5b409bbe0 con 0x7fa5c4101c20 2024-09-11T21:57:13.509 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-09-11T21:57:13.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.510+0000 7fa5cb427640 1 -- 172.21.15.1:0/1744116788 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fa5b00a2be0 msgr2=0x7fa5b00a50a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T21:57:13.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.510+0000 7fa5cb427640 1 --2- 172.21.15.1:0/1744116788 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fa5b00a2be0 0x7fa5b00a50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:13.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.510+0000 7fa5cb427640 1 -- 172.21.15.1:0/1744116788 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5c4101c20 msgr2=0x7fa5c41099f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:13.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.510+0000 7fa5cb427640 1 --2- 172.21.15.1:0/1744116788 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5c4101c20 0x7fa5c41099f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa5b400e9f0 tx=0x7fa5b400eeb0 comp rx=0 tx=0).stop 2024-09-11T21:57:13.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.511+0000 7fa5cb427640 1 -- 172.21.15.1:0/1744116788 shutdown_connections 2024-09-11T21:57:13.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.511+0000 7fa5cb427640 1 --2- 172.21.15.1:0/1744116788 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7fa5b00a2be0 0x7fa5b00a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:13.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.511+0000 7fa5cb427640 1 --2- 172.21.15.1:0/1744116788 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa5c41025e0 0x7fa5c4109f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:13.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.511+0000 7fa5cb427640 1 --2- 172.21.15.1:0/1744116788 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa5c4101c20 0x7fa5c41099f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:13.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.511+0000 7fa5cb427640 1 --2- 172.21.15.1:0/1744116788 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa5c4100a30 0x7fa5c410b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:13.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.511+0000 7fa5cb427640 1 -- 172.21.15.1:0/1744116788 >> 172.21.15.1:0/1744116788 conn(0x7fa5c40fc190 msgr2=0x7fa5c4104fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:13.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.511+0000 7fa5cb427640 1 -- 172.21.15.1:0/1744116788 shutdown_connections 2024-09-11T21:57:13.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.511+0000 7fa5cb427640 1 -- 172.21.15.1:0/1744116788 wait complete. 2024-09-11T21:57:13.520 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 11 21:57:13 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-z[37416]: 2024-09-11T21:57:13.246+0000 7fcfe6d89040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T21:57:13.522 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-11T21:57:13.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.633+0000 7f1d202e4640 1 -- 172.21.15.1:0/3643184214 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d18101c20 msgr2=0x7f1d181020a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:13.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.633+0000 7f1d202e4640 1 --2- 172.21.15.1:0/3643184214 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d18101c20 0x7f1d181020a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f1d08009900 tx=0x7f1d0802a410 comp rx=0 tx=0).stop 2024-09-11T21:57:13.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.633+0000 7f1d202e4640 1 -- 172.21.15.1:0/3643184214 shutdown_connections 2024-09-11T21:57:13.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.633+0000 7f1d202e4640 1 --2- 172.21.15.1:0/3643184214 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d181025e0 0x7f1d18108e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:13.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.633+0000 7f1d202e4640 1 --2- 172.21.15.1:0/3643184214 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d18101c20 0x7f1d181020a0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:13.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.633+0000 7f1d202e4640 1 --2- 172.21.15.1:0/3643184214 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d18100a30 0x7f1d18100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:13.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.633+0000 7f1d202e4640 1 -- 172.21.15.1:0/3643184214 >> 172.21.15.1:0/3643184214 conn(0x7f1d180fc190 msgr2=0x7f1d180fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:13.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.633+0000 7f1d202e4640 1 -- 172.21.15.1:0/3643184214 shutdown_connections 2024-09-11T21:57:13.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.633+0000 7f1d202e4640 1 -- 172.21.15.1:0/3643184214 wait complete. 2024-09-11T21:57:13.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.634+0000 7f1d202e4640 1 Processor -- start 2024-09-11T21:57:13.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.634+0000 7f1d202e4640 1 -- start start 2024-09-11T21:57:13.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.634+0000 7f1d202e4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d18100a30 0x7f1d18197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:13.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.634+0000 7f1d202e4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d18101c20 0x7f1d18198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:13.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.634+0000 7f1d202e4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d181025e0 0x7f1d1819f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:13.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.634+0000 7f1d202e4640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1d1810d800 con 0x7f1d181025e0 2024-09-11T21:57:13.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.634+0000 7f1d202e4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1d1810d680 con 0x7f1d18100a30 2024-09-11T21:57:13.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.634+0000 7f1d202e4640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1d1810d980 con 0x7f1d18101c20 2024-09-11T21:57:13.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.634+0000 7f1d1e881640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d181025e0 0x7f1d1819f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:13.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.634+0000 7f1d1e080640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d18100a30 0x7f1d18197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:13.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.634+0000 7f1d1e881640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d181025e0 0x7f1d1819f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46756/0 (socket says 172.21.15.1:46756) 2024-09-11T21:57:13.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.634+0000 7f1d1e881640 1 -- 172.21.15.1:0/745250816 learned_addr learned my addr 172.21.15.1:0/745250816 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:13.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.634+0000 7f1d1d87f640 1 --2- 172.21.15.1:0/745250816 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d18101c20 0x7f1d18198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:13.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.635+0000 7f1d1e881640 1 -- 172.21.15.1:0/745250816 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d18101c20 msgr2=0x7f1d18198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:13.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.635+0000 7f1d1e881640 1 --2- 172.21.15.1:0/745250816 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d18101c20 0x7f1d18198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:13.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.635+0000 7f1d1e881640 1 -- 172.21.15.1:0/745250816 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d18100a30 msgr2=0x7f1d18197e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:13.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.635+0000 7f1d1e881640 1 --2- 172.21.15.1:0/745250816 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d18100a30 0x7f1d18197e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:13.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.635+0000 7f1d1e881640 1 -- 172.21.15.1:0/745250816 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d1819fa20 con 0x7f1d181025e0 2024-09-11T21:57:13.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.635+0000 7f1d1e080640 1 --2- 172.21.15.1:0/745250816 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d18100a30 0x7f1d18197e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T21:57:13.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.635+0000 7f1d1e881640 1 --2- 172.21.15.1:0/745250816 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d181025e0 0x7f1d1819f3b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f1d10009fd0 tx=0x7f1d1000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:13.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.635+0000 7f1d0f7fe640 1 -- 172.21.15.1:0/745250816 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d10017070 con 0x7f1d181025e0 2024-09-11T21:57:13.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.635+0000 7f1d0f7fe640 1 -- 172.21.15.1:0/745250816 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d10003e50 con 0x7f1d181025e0 2024-09-11T21:57:13.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.635+0000 7f1d202e4640 1 -- 172.21.15.1:0/745250816 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d1819fd10 con 0x7f1d181025e0 2024-09-11T21:57:13.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.635+0000 7f1d0f7fe640 1 -- 172.21.15.1:0/745250816 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d100106c0 con 0x7f1d181025e0 2024-09-11T21:57:13.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.635+0000 7f1d202e4640 1 -- 172.21.15.1:0/745250816 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d181a01d0 con 0x7f1d181025e0 2024-09-11T21:57:13.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.636+0000 7f1d202e4640 1 -- 172.21.15.1:0/745250816 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d18107e70 con 0x7f1d181025e0 2024-09-11T21:57:13.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.637+0000 7f1d0f7fe640 1 -- 172.21.15.1:0/745250816 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 20) ==== 137849+0+0 (secure 0 0 0) 0x7f1d10003970 con 0x7f1d181025e0 2024-09-11T21:57:13.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.638+0000 7f1d0f7fe640 1 --2- 172.21.15.1:0/745250816 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f1d040a2c60 0x7f1d040a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:13.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.638+0000 7f1d1e080640 1 -- 172.21.15.1:0/745250816 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f1d040a2c60 msgr2=0x7f1d040a5120 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/3890350886 2024-09-11T21:57:13.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.638+0000 7f1d0f7fe640 1 -- 172.21.15.1:0/745250816 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8269+0+0 (secure 0 0 0) 0x7f1d100cede0 con 0x7f1d181025e0 2024-09-11T21:57:13.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.638+0000 7f1d1e080640 1 --2- 172.21.15.1:0/745250816 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f1d040a2c60 0x7f1d040a5120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T21:57:13.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.646+0000 7f1d0f7fe640 1 -- 172.21.15.1:0/745250816 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1d1009bda0 con 0x7f1d181025e0 2024-09-11T21:57:13.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.676+0000 7f1d0f7fe640 1 -- 172.21.15.1:0/745250816 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mgrmap(e 21) ==== 137518+0+0 (secure 0 0 0) 0x7f1d10097d10 con 0x7f1d181025e0 2024-09-11T21:57:13.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.677+0000 7f1d0f7fe640 1 -- 172.21.15.1:0/745250816 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f1d040a2c60 msgr2=0x7f1d040a5120 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T21:57:13.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.677+0000 7f1d0f7fe640 1 --2- 172.21.15.1:0/745250816 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f1d040a2c60 0x7f1d040a5120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:13.797 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 11 21:57:13 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-y[36280]: 2024-09-11T21:57:13.407+0000 7ff2421f9040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T21:57:13.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:13 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/4261431762' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-11T21:57:13.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:13 smithi053 ceph-mon[34908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T21:57:13.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:13 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/3624341793' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T21:57:13.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:13 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/3724058766' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T21:57:13.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:13 smithi053 ceph-mon[34908]: Standby manager daemon z restarted 2024-09-11T21:57:13.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:13 smithi053 ceph-mon[34908]: Standby manager daemon z started 2024-09-11T21:57:13.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:13 smithi053 ceph-mon[34908]: from='mgr.? 172.21.15.161:0/1180476717' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-11T21:57:13.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:13 smithi053 ceph-mon[34908]: from='mgr.? 172.21.15.161:0/1180476717' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T21:57:13.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:13 smithi053 ceph-mon[34908]: from='mgr.? 172.21.15.161:0/1180476717' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-11T21:57:13.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:13 smithi053 ceph-mon[34908]: from='mgr.? 172.21.15.161:0/1180476717' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T21:57:13.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:13 smithi053 ceph-mon[34908]: Standby manager daemon y restarted 2024-09-11T21:57:13.799 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:13 smithi053 ceph-mon[34908]: Standby manager daemon y started 2024-09-11T21:57:13.799 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:13 smithi053 ceph-mon[34908]: from='mgr.? 172.21.15.53:0/2119157840' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-11T21:57:13.799 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:13 smithi053 ceph-mon[34908]: from='mgr.? 172.21.15.53:0/2119157840' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T21:57:13.799 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:13 smithi053 ceph-mon[34908]: from='mgr.? 172.21.15.53:0/2119157840' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-11T21:57:13.799 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:13 smithi053 ceph-mon[34908]: from='mgr.? 172.21.15.53:0/2119157840' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T21:57:13.799 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:13 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/1744116788' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-11T21:57:13.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:13 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/4261431762' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-11T21:57:13.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:13 smithi161 ceph-mon[35615]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T21:57:13.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:13 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/3624341793' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T21:57:13.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:13 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/3724058766' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T21:57:13.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:13 smithi161 ceph-mon[35615]: Standby manager daemon z restarted 2024-09-11T21:57:13.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:13 smithi161 ceph-mon[35615]: Standby manager daemon z started 2024-09-11T21:57:13.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:13 smithi161 ceph-mon[35615]: from='mgr.? 172.21.15.161:0/1180476717' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-11T21:57:13.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:13 smithi161 ceph-mon[35615]: from='mgr.? 172.21.15.161:0/1180476717' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T21:57:13.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:13 smithi161 ceph-mon[35615]: from='mgr.? 172.21.15.161:0/1180476717' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-11T21:57:13.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:13 smithi161 ceph-mon[35615]: from='mgr.? 172.21.15.161:0/1180476717' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T21:57:13.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:13 smithi161 ceph-mon[35615]: Standby manager daemon y restarted 2024-09-11T21:57:13.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:13 smithi161 ceph-mon[35615]: Standby manager daemon y started 2024-09-11T21:57:13.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:13 smithi161 ceph-mon[35615]: from='mgr.? 172.21.15.53:0/2119157840' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-11T21:57:13.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:13 smithi161 ceph-mon[35615]: from='mgr.? 172.21.15.53:0/2119157840' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T21:57:13.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:13 smithi161 ceph-mon[35615]: from='mgr.? 172.21.15.53:0/2119157840' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-11T21:57:13.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:13 smithi161 ceph-mon[35615]: from='mgr.? 172.21.15.53:0/2119157840' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T21:57:13.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:13 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/1744116788' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-11T21:57:13.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.912+0000 7f1d202e4640 1 -- 172.21.15.1:0/745250816 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1d1802d7c0 con 0x7f1d181025e0 2024-09-11T21:57:13.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.912+0000 7f1d0f7fe640 1 -- 172.21.15.1:0/745250816 <== mon.0 v2:172.21.15.1:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7f1d100cab40 con 0x7f1d181025e0 2024-09-11T21:57:13.913 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-09-11T21:57:13.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.914+0000 7f1d202e4640 1 -- 172.21.15.1:0/745250816 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d181025e0 msgr2=0x7f1d1819f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:13.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.914+0000 7f1d202e4640 1 --2- 172.21.15.1:0/745250816 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d181025e0 0x7f1d1819f3b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f1d10009fd0 tx=0x7f1d1000b260 comp rx=0 tx=0).stop 2024-09-11T21:57:13.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.914+0000 7f1d202e4640 1 -- 172.21.15.1:0/745250816 shutdown_connections 2024-09-11T21:57:13.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.914+0000 7f1d202e4640 1 --2- 172.21.15.1:0/745250816 >> [v2:172.21.15.1:6800/3890350886,v1:172.21.15.1:6801/3890350886] conn(0x7f1d040a2c60 0x7f1d040a5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:13.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.914+0000 7f1d202e4640 1 --2- 172.21.15.1:0/745250816 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d181025e0 0x7f1d1819f3b0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:13.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.914+0000 7f1d202e4640 1 --2- 172.21.15.1:0/745250816 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d18101c20 0x7f1d18198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:13.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.914+0000 7f1d202e4640 1 --2- 172.21.15.1:0/745250816 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d18100a30 0x7f1d18197e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:13.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.914+0000 7f1d202e4640 1 -- 172.21.15.1:0/745250816 >> 172.21.15.1:0/745250816 conn(0x7f1d180fc190 msgr2=0x7f1d18105040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:13.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.915+0000 7f1d202e4640 1 -- 172.21.15.1:0/745250816 shutdown_connections 2024-09-11T21:57:13.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:13.915+0000 7f1d202e4640 1 -- 172.21.15.1:0/745250816 wait complete. 2024-09-11T21:57:13.923 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:57:13 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:57:13.662+0000 7f5faf926040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T21:57:13.924 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:13 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/4261431762' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-11T21:57:13.924 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:13 smithi001 ceph-mon[30935]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T21:57:13.924 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:13 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/3624341793' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T21:57:13.924 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:13 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/3724058766' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T21:57:13.924 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:13 smithi001 ceph-mon[30935]: Standby manager daemon z restarted 2024-09-11T21:57:13.924 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:13 smithi001 ceph-mon[30935]: Standby manager daemon z started 2024-09-11T21:57:13.924 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:13 smithi001 ceph-mon[30935]: from='mgr.? 172.21.15.161:0/1180476717' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-11T21:57:13.924 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:13 smithi001 ceph-mon[30935]: from='mgr.? 172.21.15.161:0/1180476717' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T21:57:13.924 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:13 smithi001 ceph-mon[30935]: from='mgr.? 172.21.15.161:0/1180476717' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-11T21:57:13.924 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:13 smithi001 ceph-mon[30935]: from='mgr.? 172.21.15.161:0/1180476717' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T21:57:13.924 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:13 smithi001 ceph-mon[30935]: Standby manager daemon y restarted 2024-09-11T21:57:13.925 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:13 smithi001 ceph-mon[30935]: Standby manager daemon y started 2024-09-11T21:57:13.925 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:13 smithi001 ceph-mon[30935]: from='mgr.? 172.21.15.53:0/2119157840' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-11T21:57:13.925 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:13 smithi001 ceph-mon[30935]: from='mgr.? 172.21.15.53:0/2119157840' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T21:57:13.925 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:13 smithi001 ceph-mon[30935]: from='mgr.? 172.21.15.53:0/2119157840' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-11T21:57:13.925 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:13 smithi001 ceph-mon[30935]: from='mgr.? 172.21.15.53:0/2119157840' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T21:57:13.925 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:13 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/1744116788' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-11T21:57:13.929 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-09-11T21:57:13.930 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T21:57:14.193 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:57:13 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:57:13.922+0000 7f5f17a34640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T21:57:14.194 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:57:13 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:57:13.922+0000 7f5f17a34640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T21:57:14.194 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:57:13 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:57:13.922+0000 7f5f17a34640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T21:57:14.194 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:57:13 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:57:13.922+0000 7f5f17a34640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T21:57:14.194 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:57:13 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:57:13.922+0000 7f5f17a34640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T21:57:14.194 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:57:13 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:57:13.925+0000 7f5f1aaba640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T21:57:14.194 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:57:13 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:57:13.925+0000 7f5f1aaba640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T21:57:14.194 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:57:13 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:57:13.925+0000 7f5f1aaba640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T21:57:14.194 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:57:13 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:57:13.925+0000 7f5f1aaba640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T21:57:14.194 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:57:13 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:57:13.925+0000 7f5f1aaba640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T21:57:14.251 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:57:14.565 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:14 smithi001 ceph-mon[30935]: mgrmap e20: x(active, since 8m), standbys: y, z 2024-09-11T21:57:14.566 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:14 smithi001 ceph-mon[30935]: Active manager daemon x restarted 2024-09-11T21:57:14.566 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:14 smithi001 ceph-mon[30935]: Activating manager daemon x 2024-09-11T21:57:14.566 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:14 smithi001 ceph-mon[30935]: osdmap e87: 12 total, 12 up, 12 in 2024-09-11T21:57:14.566 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:14 smithi001 ceph-mon[30935]: mgrmap e21: x(active, starting, since 0.0134737s), standbys: y, z 2024-09-11T21:57:14.566 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:14 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T21:57:14.566 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:14 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T21:57:14.566 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:14 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T21:57:14.566 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:14 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-11T21:57:14.566 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:14 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-11T21:57:14.566 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:14 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-11T21:57:14.567 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:14 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-11T21:57:14.567 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:14 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-11T21:57:14.567 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:14 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T21:57:14.567 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:14 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-11T21:57:14.567 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:14 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T21:57:14.567 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:14 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-11T21:57:14.567 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:14 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-11T21:57:14.567 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:14 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-11T21:57:14.567 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:14 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-11T21:57:14.567 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:14 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T21:57:14.567 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:14 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T21:57:14.567 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:14 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-11T21:57:14.567 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:14 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T21:57:14.567 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:14 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T21:57:14.568 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:14 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T21:57:14.568 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:14 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T21:57:14.568 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:14 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T21:57:14.568 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:14 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T21:57:14.568 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:14 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T21:57:14.568 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:14 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T21:57:14.568 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:14 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T21:57:14.568 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:14 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T21:57:14.568 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:14 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T21:57:14.568 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:14 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T21:57:14.568 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:14 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T21:57:14.568 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:14 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T21:57:14.568 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:14 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T21:57:14.569 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:14 smithi001 ceph-mon[30935]: Manager daemon x is now available 2024-09-11T21:57:14.569 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:14 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:57:14.569 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:14 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-11T21:57:14.569 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:14 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-11T21:57:14.569 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:14 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-11T21:57:14.569 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:14 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-11T21:57:14.569 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:14 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/745250816' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-11T21:57:14.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:14.771+0000 7f211b473640 1 -- 172.21.15.1:0/3376811834 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2114106f10 msgr2=0x7f211410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:14.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:14.771+0000 7f211b473640 1 --2- 172.21.15.1:0/3376811834 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2114106f10 0x7f211410ea90 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f210c009ca0 tx=0x7f210c02a8c0 comp rx=0 tx=0).stop 2024-09-11T21:57:14.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:14.771+0000 7f211b473640 1 -- 172.21.15.1:0/3376811834 shutdown_connections 2024-09-11T21:57:14.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:14.771+0000 7f211b473640 1 --2- 172.21.15.1:0/3376811834 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2114106f10 0x7f211410ea90 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:14.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:14.771+0000 7f211b473640 1 --2- 172.21.15.1:0/3376811834 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2114102790 0x7f21141067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:14.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:14.771+0000 7f211b473640 1 --2- 172.21.15.1:0/3376811834 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2114101de0 0x7f21141021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:14.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:14.771+0000 7f211b473640 1 -- 172.21.15.1:0/3376811834 >> 172.21.15.1:0/3376811834 conn(0x7f21140fd8e0 msgr2=0x7f21140ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:14.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:14.771+0000 7f211b473640 1 -- 172.21.15.1:0/3376811834 shutdown_connections 2024-09-11T21:57:14.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:14.771+0000 7f211b473640 1 -- 172.21.15.1:0/3376811834 wait complete. 2024-09-11T21:57:14.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:14.772+0000 7f211b473640 1 Processor -- start 2024-09-11T21:57:14.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:14.772+0000 7f211b473640 1 -- start start 2024-09-11T21:57:14.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:14.772+0000 7f211b473640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2114101de0 0x7f21141983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:14.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:14.772+0000 7f211b473640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2114102790 0x7f2114198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:14.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:14.772+0000 7f211b473640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2114106f10 0x7f211419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:14.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:14.772+0000 7f211b473640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2114111f70 con 0x7f2114102790 2024-09-11T21:57:14.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:14.772+0000 7f211b473640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2114111df0 con 0x7f2114106f10 2024-09-11T21:57:14.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:14.772+0000 7f211b473640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f21141120f0 con 0x7f2114101de0 2024-09-11T21:57:14.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:14.772+0000 7f2118a0e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2114102790 0x7f2114198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:14.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:14.773+0000 7f2118a0e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2114102790 0x7f2114198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46884/0 (socket says 172.21.15.1:46884) 2024-09-11T21:57:14.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:14.773+0000 7f2118a0e640 1 -- 172.21.15.1:0/1046879434 learned_addr learned my addr 172.21.15.1:0/1046879434 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:14.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:14.773+0000 7f2119a10640 1 --2- 172.21.15.1:0/1046879434 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2114106f10 0x7f211419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:14.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:14.773+0000 7f2118a0e640 1 -- 172.21.15.1:0/1046879434 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2114101de0 msgr2=0x7f21141983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:14.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:14.773+0000 7f211920f640 1 --2- 172.21.15.1:0/1046879434 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2114101de0 0x7f21141983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:14.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:14.773+0000 7f2118a0e640 1 --2- 172.21.15.1:0/1046879434 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2114101de0 0x7f21141983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:14.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:14.773+0000 7f2118a0e640 1 -- 172.21.15.1:0/1046879434 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2114106f10 msgr2=0x7f211419cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:14.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:14.773+0000 7f2118a0e640 1 --2- 172.21.15.1:0/1046879434 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2114106f10 0x7f211419cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:14.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:14.773+0000 7f2118a0e640 1 -- 172.21.15.1:0/1046879434 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f211419d380 con 0x7f2114102790 2024-09-11T21:57:14.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:14.773+0000 7f2119a10640 1 --2- 172.21.15.1:0/1046879434 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2114106f10 0x7f211419cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:57:14.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:14.773+0000 7f211920f640 1 --2- 172.21.15.1:0/1046879434 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2114101de0 0x7f21141983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T21:57:14.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:14.773+0000 7f2118a0e640 1 --2- 172.21.15.1:0/1046879434 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2114102790 0x7f2114198910 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f211000e9f0 tx=0x7f211000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:14.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:14.773+0000 7f21067fc640 1 -- 172.21.15.1:0/1046879434 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f211000cd80 con 0x7f2114102790 2024-09-11T21:57:14.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:14.773+0000 7f21067fc640 1 -- 172.21.15.1:0/1046879434 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2110005500 con 0x7f2114102790 2024-09-11T21:57:14.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:14.773+0000 7f211b473640 1 -- 172.21.15.1:0/1046879434 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f211419d670 con 0x7f2114102790 2024-09-11T21:57:14.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:14.773+0000 7f21067fc640 1 -- 172.21.15.1:0/1046879434 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2110010690 con 0x7f2114102790 2024-09-11T21:57:14.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:14.773+0000 7f211b473640 1 -- 172.21.15.1:0/1046879434 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21141a4f50 con 0x7f2114102790 2024-09-11T21:57:14.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:14.775+0000 7f211b473640 1 -- 172.21.15.1:0/1046879434 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21141021c0 con 0x7f2114102790 2024-09-11T21:57:14.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:14.776+0000 7f21067fc640 1 -- 172.21.15.1:0/1046879434 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 22) ==== 137645+0+0 (secure 0 0 0) 0x7f21100046c0 con 0x7f2114102790 2024-09-11T21:57:14.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:14.776+0000 7f21067fc640 1 --2- 172.21.15.1:0/1046879434 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f21000a29f0 0x7f21000a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:14.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:14.776+0000 7f211920f640 1 --2- 172.21.15.1:0/1046879434 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f21000a29f0 0x7f21000a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:14.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:14.776+0000 7f21067fc640 1 -- 172.21.15.1:0/1046879434 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7f21100cf0e0 con 0x7f2114102790 2024-09-11T21:57:14.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:14.777+0000 7f211920f640 1 --2- 172.21.15.1:0/1046879434 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f21000a29f0 0x7f21000a4eb0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2108009b50 tx=0x7f2108005f90 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:14.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:14.781+0000 7f21067fc640 1 -- 172.21.15.1:0/1046879434 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f211009c2e0 con 0x7f2114102790 2024-09-11T21:57:14.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:14 smithi053 ceph-mon[34908]: mgrmap e20: x(active, since 8m), standbys: y, z 2024-09-11T21:57:14.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:14 smithi053 ceph-mon[34908]: Active manager daemon x restarted 2024-09-11T21:57:14.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:14 smithi053 ceph-mon[34908]: Activating manager daemon x 2024-09-11T21:57:14.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:14 smithi053 ceph-mon[34908]: osdmap e87: 12 total, 12 up, 12 in 2024-09-11T21:57:14.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:14 smithi053 ceph-mon[34908]: mgrmap e21: x(active, starting, since 0.0134737s), standbys: y, z 2024-09-11T21:57:14.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:14 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T21:57:14.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:14 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T21:57:14.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:14 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T21:57:14.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:14 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-11T21:57:14.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:14 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-11T21:57:14.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:14 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-11T21:57:14.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:14 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-11T21:57:14.799 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:14 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-11T21:57:14.799 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:14 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T21:57:14.799 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:14 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-11T21:57:14.799 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:14 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T21:57:14.799 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:14 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-11T21:57:14.799 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:14 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-11T21:57:14.799 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:14 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-11T21:57:14.799 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:14 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-11T21:57:14.799 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:14 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T21:57:14.799 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:14 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T21:57:14.799 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:14 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-11T21:57:14.799 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:14 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T21:57:14.800 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:14 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T21:57:14.800 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:14 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T21:57:14.800 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:14 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T21:57:14.800 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:14 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T21:57:14.800 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:14 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T21:57:14.800 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:14 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T21:57:14.800 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:14 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T21:57:14.800 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:14 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T21:57:14.800 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:14 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T21:57:14.800 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:14 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T21:57:14.800 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:14 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T21:57:14.800 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:14 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T21:57:14.800 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:14 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T21:57:14.800 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:14 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T21:57:14.801 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:14 smithi053 ceph-mon[34908]: Manager daemon x is now available 2024-09-11T21:57:14.801 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:14 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:57:14.801 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:14 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-11T21:57:14.801 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:14 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-11T21:57:14.801 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:14 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-11T21:57:14.801 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:14 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-11T21:57:14.801 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:14 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/745250816' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-11T21:57:14.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:14 smithi161 ceph-mon[35615]: mgrmap e20: x(active, since 8m), standbys: y, z 2024-09-11T21:57:14.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:14 smithi161 ceph-mon[35615]: Active manager daemon x restarted 2024-09-11T21:57:14.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:14 smithi161 ceph-mon[35615]: Activating manager daemon x 2024-09-11T21:57:14.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:14 smithi161 ceph-mon[35615]: osdmap e87: 12 total, 12 up, 12 in 2024-09-11T21:57:14.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:14 smithi161 ceph-mon[35615]: mgrmap e21: x(active, starting, since 0.0134737s), standbys: y, z 2024-09-11T21:57:14.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:14 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T21:57:14.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:14 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T21:57:14.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:14 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T21:57:14.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:14 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-11T21:57:14.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:14 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-11T21:57:14.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:14 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-11T21:57:14.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:14 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-11T21:57:14.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:14 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-11T21:57:14.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:14 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T21:57:14.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:14 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-11T21:57:14.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:14 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T21:57:14.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:14 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-11T21:57:14.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:14 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-11T21:57:14.844 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:14 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-11T21:57:14.844 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:14 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-11T21:57:14.844 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:14 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T21:57:14.844 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:14 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T21:57:14.844 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:14 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-11T21:57:14.844 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:14 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T21:57:14.844 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:14 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T21:57:14.844 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:14 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T21:57:14.844 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:14 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T21:57:14.844 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:14 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T21:57:14.844 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:14 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T21:57:14.844 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:14 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T21:57:14.845 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:14 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T21:57:14.845 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:14 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T21:57:14.845 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:14 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T21:57:14.845 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:14 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T21:57:14.845 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:14 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T21:57:14.845 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:14 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T21:57:14.845 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:14 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T21:57:14.845 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:14 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T21:57:14.845 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:14 smithi161 ceph-mon[35615]: Manager daemon x is now available 2024-09-11T21:57:14.845 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:14 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:57:14.845 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:14 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-11T21:57:14.845 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:14 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-11T21:57:14.845 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:14 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-11T21:57:14.845 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:14 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-11T21:57:14.846 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:14 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/745250816' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-11T21:57:15.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:15.006+0000 7f211b473640 1 -- 172.21.15.1:0/1046879434 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f211402d7c0 con 0x7f2114102790 2024-09-11T21:57:15.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:15.006+0000 7f21067fc640 1 -- 172.21.15.1:0/1046879434 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11949 (secure 0 0 0) 0x7f211009bbe0 con 0x7f2114102790 2024-09-11T21:57:15.008 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:57:15.008 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":28,"btime":"2024-09-11T21:57:12:512925+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/421542832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":421542832},{"type":"v1","addr":"172.21.15.1:6835","nonce":421542832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2836864406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2836864406},{"type":"v1","addr":"172.21.15.1:6837","nonce":2836864406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T21:55:20.791263+0000","modified":"2024-09-11T21:57:12.462736+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24484,"mds_2":24467,"mds_3":24466,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14595":{"gid":14595,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/407834672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":407834672},{"type":"v1","addr":"172.21.15.1:6839","nonce":407834672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/1068827678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":1068827678},{"type":"v1","addr":"172.21.15.1:6841","nonce":1068827678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/872410024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":872410024},{"type":"v1","addr":"172.21.15.161:6833","nonce":872410024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/1648944570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1648944570},{"type":"v1","addr":"172.21.15.53:6833","nonce":1648944570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/2126933993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2126933993},{"type":"v1","addr":"172.21.15.53:6835","nonce":2126933993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.161:6835/828381592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":828381592},{"type":"v1","addr":"172.21.15.161:6835","nonce":828381592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.161:6837/1799318656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1799318656},{"type":"v1","addr":"172.21.15.161:6837","nonce":1799318656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.53:6837/3457131830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3457131830},{"type":"v1","addr":"172.21.15.53:6837","nonce":3457131830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/455365383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":455365383},{"type":"v1","addr":"172.21.15.161:6839","nonce":455365383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.53:6839/1378584447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1378584447},{"type":"v1","addr":"172.21.15.53:6839","nonce":1378584447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24460,24466,24467,24484,24494]},"id":1}]} 2024-09-11T21:57:15.008 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 28 2024-09-11T21:57:15.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:15.009+0000 7f211b473640 1 -- 172.21.15.1:0/1046879434 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f21000a29f0 msgr2=0x7f21000a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:15.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:15.009+0000 7f211b473640 1 --2- 172.21.15.1:0/1046879434 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f21000a29f0 0x7f21000a4eb0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2108009b50 tx=0x7f2108005f90 comp rx=0 tx=0).stop 2024-09-11T21:57:15.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:15.009+0000 7f211b473640 1 -- 172.21.15.1:0/1046879434 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2114102790 msgr2=0x7f2114198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:15.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:15.009+0000 7f211b473640 1 --2- 172.21.15.1:0/1046879434 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2114102790 0x7f2114198910 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f211000e9f0 tx=0x7f211000edf0 comp rx=0 tx=0).stop 2024-09-11T21:57:15.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:15.009+0000 7f211b473640 1 -- 172.21.15.1:0/1046879434 shutdown_connections 2024-09-11T21:57:15.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:15.009+0000 7f211b473640 1 --2- 172.21.15.1:0/1046879434 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f21000a29f0 0x7f21000a4eb0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:15.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:15.009+0000 7f211b473640 1 --2- 172.21.15.1:0/1046879434 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2114106f10 0x7f211419cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:15.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:15.009+0000 7f211b473640 1 --2- 172.21.15.1:0/1046879434 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2114102790 0x7f2114198910 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:15.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:15.009+0000 7f211b473640 1 --2- 172.21.15.1:0/1046879434 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2114101de0 0x7f21141983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:15.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:15.009+0000 7f211b473640 1 -- 172.21.15.1:0/1046879434 >> 172.21.15.1:0/1046879434 conn(0x7f21140fd8e0 msgr2=0x7f21140fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:15.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:15.009+0000 7f211b473640 1 -- 172.21.15.1:0/1046879434 shutdown_connections 2024-09-11T21:57:15.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:15.009+0000 7f211b473640 1 -- 172.21.15.1:0/1046879434 wait complete. 2024-09-11T21:57:15.894 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:15 smithi001 ceph-mon[30935]: mgrmap e22: x(active, since 1.01705s), standbys: y, z 2024-09-11T21:57:15.895 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:15 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/1046879434' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:57:15.895 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:15 smithi001 ceph-mon[30935]: [11/Sep/2024:21:57:15] ENGINE Bus STARTING 2024-09-11T21:57:15.895 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:15 smithi001 ceph-mon[30935]: [11/Sep/2024:21:57:15] ENGINE Serving on https://172.21.15.1:7150 2024-09-11T21:57:15.895 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:57:15 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:57:15.453+0000 7f5f03191640 -1 log_channel(cephadm) log [ERR] : [11/Sep/2024:21:57:15] ENGINE Error in HTTPServer.serve 2024-09-11T21:57:15.895 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:57:15 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: Traceback (most recent call last): 2024-09-11T21:57:15.895 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:57:15 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T21:57:15.895 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:57:15 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: self._connections.run(self.expiration_interval) 2024-09-11T21:57:15.895 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:57:15 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T21:57:15.895 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:57:15 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: self._run(expiration_interval) 2024-09-11T21:57:15.895 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:57:15 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T21:57:15.896 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:57:15 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T21:57:15.896 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:57:15 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T21:57:15.896 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:57:15 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T21:57:15.896 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:57:15 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T21:57:15.896 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:57:15 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: s = self.context.wrap_socket( 2024-09-11T21:57:15.896 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:57:15 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T21:57:15.896 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:57:15 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: return self.sslsocket_class._create( 2024-09-11T21:57:15.896 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:57:15 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T21:57:15.896 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:57:15 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: self.do_handshake() 2024-09-11T21:57:15.896 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:57:15 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T21:57:15.896 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:57:15 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: self._sslobj.do_handshake() 2024-09-11T21:57:15.896 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:57:15 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T21:57:15.897 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:57:15 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:57:15.921 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-09-11T21:57:15.921 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-09-11T21:57:15.921 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T21:57:16.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.027+0000 7f15b00c9640 1 -- 172.21.15.1:0/1168674548 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15a810e110 msgr2=0x7f15a8110500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:16.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.027+0000 7f15b00c9640 1 --2- 172.21.15.1:0/1168674548 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15a810e110 0x7f15a8110500 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f15a0009ca0 tx=0x7f15a002a8c0 comp rx=0 tx=0).stop 2024-09-11T21:57:16.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.027+0000 7f15b00c9640 1 -- 172.21.15.1:0/1168674548 shutdown_connections 2024-09-11T21:57:16.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.027+0000 7f15b00c9640 1 --2- 172.21.15.1:0/1168674548 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15a810e110 0x7f15a8110500 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:16.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.027+0000 7f15b00c9640 1 --2- 172.21.15.1:0/1168674548 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15a8074c50 0x7f15a80750b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:16.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.027+0000 7f15b00c9640 1 --2- 172.21.15.1:0/1168674548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15a8076230 0x7f15a8074680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:16.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.027+0000 7f15b00c9640 1 -- 172.21.15.1:0/1168674548 >> 172.21.15.1:0/1168674548 conn(0x7f15a80fd2a0 msgr2=0x7f15a80ff6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:16.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.028+0000 7f15b00c9640 1 -- 172.21.15.1:0/1168674548 shutdown_connections 2024-09-11T21:57:16.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.028+0000 7f15b00c9640 1 -- 172.21.15.1:0/1168674548 wait complete. 2024-09-11T21:57:16.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.028+0000 7f15b00c9640 1 Processor -- start 2024-09-11T21:57:16.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.028+0000 7f15b00c9640 1 -- start start 2024-09-11T21:57:16.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.028+0000 7f15b00c9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15a8074c50 0x7f15a819e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:16.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.028+0000 7f15b00c9640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15a8076230 0x7f15a819e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:16.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.028+0000 7f15b00c9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15a810e110 0x7f15a81a59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:16.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.028+0000 7f15b00c9640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f15a8113dd0 con 0x7f15a8074c50 2024-09-11T21:57:16.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.028+0000 7f15b00c9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f15a8113c50 con 0x7f15a810e110 2024-09-11T21:57:16.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.028+0000 7f15b00c9640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f15a8113f50 con 0x7f15a8076230 2024-09-11T21:57:16.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.028+0000 7f15ade65640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15a8074c50 0x7f15a819e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:16.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.029+0000 7f15ade65640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15a8074c50 0x7f15a819e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46912/0 (socket says 172.21.15.1:46912) 2024-09-11T21:57:16.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.029+0000 7f15ad664640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15a8076230 0x7f15a819e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:16.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.029+0000 7f15ade65640 1 -- 172.21.15.1:0/560462594 learned_addr learned my addr 172.21.15.1:0/560462594 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:16.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.029+0000 7f15ae666640 1 --2- 172.21.15.1:0/560462594 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15a810e110 0x7f15a81a59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:16.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.029+0000 7f15ade65640 1 -- 172.21.15.1:0/560462594 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15a8076230 msgr2=0x7f15a819e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:16.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.029+0000 7f15ade65640 1 --2- 172.21.15.1:0/560462594 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15a8076230 0x7f15a819e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:16.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.029+0000 7f15ade65640 1 -- 172.21.15.1:0/560462594 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15a810e110 msgr2=0x7f15a81a59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:16.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.029+0000 7f15ade65640 1 --2- 172.21.15.1:0/560462594 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15a810e110 0x7f15a81a59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:16.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.029+0000 7f15ade65640 1 -- 172.21.15.1:0/560462594 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15a81a6020 con 0x7f15a8074c50 2024-09-11T21:57:16.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.029+0000 7f15ade65640 1 --2- 172.21.15.1:0/560462594 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15a8074c50 0x7f15a819e400 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f15a400e9f0 tx=0x7f15a400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:16.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.029+0000 7f159effd640 1 -- 172.21.15.1:0/560462594 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a400cd80 con 0x7f15a8074c50 2024-09-11T21:57:16.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.029+0000 7f15b00c9640 1 -- 172.21.15.1:0/560462594 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15a81a6310 con 0x7f15a8074c50 2024-09-11T21:57:16.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.029+0000 7f159effd640 1 -- 172.21.15.1:0/560462594 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15a4010910 con 0x7f15a8074c50 2024-09-11T21:57:16.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.029+0000 7f159effd640 1 -- 172.21.15.1:0/560462594 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a4018990 con 0x7f15a8074c50 2024-09-11T21:57:16.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.029+0000 7f15b00c9640 1 -- 172.21.15.1:0/560462594 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15a81a6860 con 0x7f15a8074c50 2024-09-11T21:57:16.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.030+0000 7f15b00c9640 1 -- 172.21.15.1:0/560462594 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15a81042e0 con 0x7f15a8074c50 2024-09-11T21:57:16.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.031+0000 7f159effd640 1 -- 172.21.15.1:0/560462594 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 23) ==== 137952+0+0 (secure 0 0 0) 0x7f15a4010ac0 con 0x7f15a8074c50 2024-09-11T21:57:16.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.032+0000 7f159effd640 1 --2- 172.21.15.1:0/560462594 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f15940a2d00 0x7f15940a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:16.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.032+0000 7f15ad664640 1 --2- 172.21.15.1:0/560462594 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f15940a2d00 0x7f15940a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:16.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.032+0000 7f159effd640 1 -- 172.21.15.1:0/560462594 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7f15a40cf730 con 0x7f15a8074c50 2024-09-11T21:57:16.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.032+0000 7f15ad664640 1 --2- 172.21.15.1:0/560462594 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f15940a2d00 0x7f15940a51c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f15a819f910 tx=0x7f1598005f90 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:16.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.036+0000 7f159effd640 1 -- 172.21.15.1:0/560462594 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f15a409bd00 con 0x7f15a8074c50 2024-09-11T21:57:16.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:15 smithi053 ceph-mon[34908]: mgrmap e22: x(active, since 1.01705s), standbys: y, z 2024-09-11T21:57:16.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:15 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/1046879434' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:57:16.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:15 smithi053 ceph-mon[34908]: [11/Sep/2024:21:57:15] ENGINE Bus STARTING 2024-09-11T21:57:16.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:15 smithi053 ceph-mon[34908]: [11/Sep/2024:21:57:15] ENGINE Serving on https://172.21.15.1:7150 2024-09-11T21:57:16.091 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:15 smithi161 ceph-mon[35615]: mgrmap e22: x(active, since 1.01705s), standbys: y, z 2024-09-11T21:57:16.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:15 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/1046879434' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:57:16.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:15 smithi161 ceph-mon[35615]: [11/Sep/2024:21:57:15] ENGINE Bus STARTING 2024-09-11T21:57:16.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:15 smithi161 ceph-mon[35615]: [11/Sep/2024:21:57:15] ENGINE Serving on https://172.21.15.1:7150 2024-09-11T21:57:16.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.241+0000 7f15b00c9640 1 -- 172.21.15.1:0/560462594 --> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f15a8062630 con 0x7f15940a2d00 2024-09-11T21:57:16.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.248+0000 7f159effd640 1 -- 172.21.15.1:0/560462594 <== mgr.15204 v2:172.21.15.1:6800/3543150772 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258070 (secure 0 0 0) 0x7f15a8062630 con 0x7f15940a2d00 2024-09-11T21:57:16.251 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:57:16.253 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-11T21:57:16.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.251+0000 7f15b00c9640 1 -- 172.21.15.1:0/560462594 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f15940a2d00 msgr2=0x7f15940a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:16.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.251+0000 7f15b00c9640 1 --2- 172.21.15.1:0/560462594 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f15940a2d00 0x7f15940a51c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f15a819f910 tx=0x7f1598005f90 comp rx=0 tx=0).stop 2024-09-11T21:57:16.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.251+0000 7f15b00c9640 1 -- 172.21.15.1:0/560462594 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15a8074c50 msgr2=0x7f15a819e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:16.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.251+0000 7f15b00c9640 1 --2- 172.21.15.1:0/560462594 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15a8074c50 0x7f15a819e400 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f15a400e9f0 tx=0x7f15a400eeb0 comp rx=0 tx=0).stop 2024-09-11T21:57:16.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.251+0000 7f15b00c9640 1 -- 172.21.15.1:0/560462594 shutdown_connections 2024-09-11T21:57:16.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.251+0000 7f15b00c9640 1 --2- 172.21.15.1:0/560462594 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f15940a2d00 0x7f15940a51c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:16.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.251+0000 7f15b00c9640 1 --2- 172.21.15.1:0/560462594 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15a810e110 0x7f15a81a59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:16.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.251+0000 7f15b00c9640 1 --2- 172.21.15.1:0/560462594 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15a8076230 0x7f15a819e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:16.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.251+0000 7f15b00c9640 1 --2- 172.21.15.1:0/560462594 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15a8074c50 0x7f15a819e400 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:16.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.251+0000 7f15b00c9640 1 -- 172.21.15.1:0/560462594 >> 172.21.15.1:0/560462594 conn(0x7f15a80fd2a0 msgr2=0x7f15a81014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:16.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.251+0000 7f15b00c9640 1 -- 172.21.15.1:0/560462594 shutdown_connections 2024-09-11T21:57:16.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.251+0000 7f15b00c9640 1 -- 172.21.15.1:0/560462594 wait complete. 2024-09-11T21:57:16.262 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":4,"stamp":"2024-09-11T21:57:15.682853+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1191642,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1302,"num_read_kb":820,"num_write":563,"num_write_kb":4410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1239947,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":945,"ondisk_log_size":945,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":390172,"kb_used_data":11772,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124551652,"statfs":{"total":1151940427776,"available":1151540891648,"internally_reserved":0,"allocated":12054528,"data_stored":8695560,"data_compressed":275530,"data_compressed_allocated":364544,"data_compressed_original":3268608,"omap_allocated":19142,"internal_metadata":387233082},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.990279"},"pg_stats":[{"pgid":"3.3f","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788313+0000","last_change":"2024-09-11T21:55:20.095385+0000","last_active":"2024-09-11T21:57:13.788313+0000","last_peered":"2024-09-11T21:57:13.788313+0000","last_clean":"2024-09-11T21:57:13.788313+0000","last_became_active":"2024-09-11T21:55:20.095077+0000","last_became_peered":"2024-09-11T21:55:20.095077+0000","last_unstale":"2024-09-11T21:57:13.788313+0000","last_undegraded":"2024-09-11T21:57:13.788313+0000","last_fullsized":"2024-09-11T21:57:13.788313+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:53:52.231522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"86'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.669584+0000","last_change":"2024-09-11T21:55:20.103540+0000","last_active":"2024-09-11T21:57:13.669584+0000","last_peered":"2024-09-11T21:57:13.669584+0000","last_clean":"2024-09-11T21:57:13.669584+0000","last_became_active":"2024-09-11T21:55:20.103233+0000","last_became_peered":"2024-09-11T21:55:20.103233+0000","last_unstale":"2024-09-11T21:57:13.669584+0000","last_undegraded":"2024-09-11T21:57:13.669584+0000","last_fullsized":"2024-09-11T21:57:13.669584+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:51:29.222202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.669576+0000","last_change":"2024-09-11T21:55:20.098538+0000","last_active":"2024-09-11T21:57:13.669576+0000","last_peered":"2024-09-11T21:57:13.669576+0000","last_clean":"2024-09-11T21:57:13.669576+0000","last_became_active":"2024-09-11T21:55:20.098202+0000","last_became_peered":"2024-09-11T21:55:20.098202+0000","last_unstale":"2024-09-11T21:57:13.669576+0000","last_undegraded":"2024-09-11T21:57:13.669576+0000","last_fullsized":"2024-09-11T21:57:13.669576+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:33:46.636952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788491+0000","last_change":"2024-09-11T21:55:20.100869+0000","last_active":"2024-09-11T21:57:13.788491+0000","last_peered":"2024-09-11T21:57:13.788491+0000","last_clean":"2024-09-11T21:57:13.788491+0000","last_became_active":"2024-09-11T21:55:20.100573+0000","last_became_peered":"2024-09-11T21:55:20.100573+0000","last_unstale":"2024-09-11T21:57:13.788491+0000","last_undegraded":"2024-09-11T21:57:13.788491+0000","last_fullsized":"2024-09-11T21:57:13.788491+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:58:11.154105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"86'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671033+0000","last_change":"2024-09-11T21:55:20.091648+0000","last_active":"2024-09-11T21:57:13.671033+0000","last_peered":"2024-09-11T21:57:13.671033+0000","last_clean":"2024-09-11T21:57:13.671033+0000","last_became_active":"2024-09-11T21:55:20.091222+0000","last_became_peered":"2024-09-11T21:55:20.091222+0000","last_unstale":"2024-09-11T21:57:13.671033+0000","last_undegraded":"2024-09-11T21:57:13.671033+0000","last_fullsized":"2024-09-11T21:57:13.671033+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:55:03.420754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"86'6","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.764975+0000","last_change":"2024-09-11T21:56:26.376595+0000","last_active":"2024-09-11T21:57:13.764975+0000","last_peered":"2024-09-11T21:57:13.764975+0000","last_clean":"2024-09-11T21:57:13.764975+0000","last_became_active":"2024-09-11T21:56:26.375784+0000","last_became_peered":"2024-09-11T21:56:26.375784+0000","last_unstale":"2024-09-11T21:57:13.764975+0000","last_undegraded":"2024-09-11T21:57:13.764975+0000","last_fullsized":"2024-09-11T21:57:13.764975+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:19:38.371463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,7],"acting":[10,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"86'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.670787+0000","last_change":"2024-09-11T21:55:20.101084+0000","last_active":"2024-09-11T21:57:13.670787+0000","last_peered":"2024-09-11T21:57:13.670787+0000","last_clean":"2024-09-11T21:57:13.670787+0000","last_became_active":"2024-09-11T21:55:20.100739+0000","last_became_peered":"2024-09-11T21:55:20.100739+0000","last_unstale":"2024-09-11T21:57:13.670787+0000","last_undegraded":"2024-09-11T21:57:13.670787+0000","last_fullsized":"2024-09-11T21:57:13.670787+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:46:58.969559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.753160+0000","last_change":"2024-09-11T21:55:20.088715+0000","last_active":"2024-09-11T21:57:13.753160+0000","last_peered":"2024-09-11T21:57:13.753160+0000","last_clean":"2024-09-11T21:57:13.753160+0000","last_became_active":"2024-09-11T21:55:20.088280+0000","last_became_peered":"2024-09-11T21:55:20.088280+0000","last_unstale":"2024-09-11T21:57:13.753160+0000","last_undegraded":"2024-09-11T21:57:13.753160+0000","last_fullsized":"2024-09-11T21:57:13.753160+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:06:24.450444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"86'11","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788671+0000","last_change":"2024-09-11T21:55:20.101266+0000","last_active":"2024-09-11T21:57:13.788671+0000","last_peered":"2024-09-11T21:57:13.788671+0000","last_clean":"2024-09-11T21:57:13.788671+0000","last_became_active":"2024-09-11T21:55:20.100919+0000","last_became_peered":"2024-09-11T21:55:20.100919+0000","last_unstale":"2024-09-11T21:57:13.788671+0000","last_undegraded":"2024-09-11T21:57:13.788671+0000","last_fullsized":"2024-09-11T21:57:13.788671+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:47:15.466082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"86'16","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.765361+0000","last_change":"2024-09-11T21:55:20.098086+0000","last_active":"2024-09-11T21:57:13.765361+0000","last_peered":"2024-09-11T21:57:13.765361+0000","last_clean":"2024-09-11T21:57:13.765361+0000","last_became_active":"2024-09-11T21:55:20.097754+0000","last_became_peered":"2024-09-11T21:55:20.097754+0000","last_unstale":"2024-09-11T21:57:13.765361+0000","last_undegraded":"2024-09-11T21:57:13.765361+0000","last_fullsized":"2024-09-11T21:57:13.765361+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:37:00.098345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"86'6","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671036+0000","last_change":"2024-09-11T21:55:20.092671+0000","last_active":"2024-09-11T21:57:13.671036+0000","last_peered":"2024-09-11T21:57:13.671036+0000","last_clean":"2024-09-11T21:57:13.671036+0000","last_became_active":"2024-09-11T21:55:20.092112+0000","last_became_peered":"2024-09-11T21:55:20.092112+0000","last_unstale":"2024-09-11T21:57:13.671036+0000","last_undegraded":"2024-09-11T21:57:13.671036+0000","last_fullsized":"2024-09-11T21:57:13.671036+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:06:28.538838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788646+0000","last_change":"2024-09-11T21:55:20.099863+0000","last_active":"2024-09-11T21:57:13.788646+0000","last_peered":"2024-09-11T21:57:13.788646+0000","last_clean":"2024-09-11T21:57:13.788646+0000","last_became_active":"2024-09-11T21:55:20.099173+0000","last_became_peered":"2024-09-11T21:55:20.099173+0000","last_unstale":"2024-09-11T21:57:13.788646+0000","last_undegraded":"2024-09-11T21:57:13.788646+0000","last_fullsized":"2024-09-11T21:57:13.788646+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:34:56.669410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"86'10","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788507+0000","last_change":"2024-09-11T21:55:20.101212+0000","last_active":"2024-09-11T21:57:13.788507+0000","last_peered":"2024-09-11T21:57:13.788507+0000","last_clean":"2024-09-11T21:57:13.788507+0000","last_became_active":"2024-09-11T21:55:20.100876+0000","last_became_peered":"2024-09-11T21:55:20.100876+0000","last_unstale":"2024-09-11T21:57:13.788507+0000","last_undegraded":"2024-09-11T21:57:13.788507+0000","last_fullsized":"2024-09-11T21:57:13.788507+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:14:18.104009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.669316+0000","last_change":"2024-09-11T21:55:20.095394+0000","last_active":"2024-09-11T21:57:13.669316+0000","last_peered":"2024-09-11T21:57:13.669316+0000","last_clean":"2024-09-11T21:57:13.669316+0000","last_became_active":"2024-09-11T21:55:20.095101+0000","last_became_peered":"2024-09-11T21:55:20.095101+0000","last_unstale":"2024-09-11T21:57:13.669316+0000","last_undegraded":"2024-09-11T21:57:13.669316+0000","last_fullsized":"2024-09-11T21:57:13.669316+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:53:32.529359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"86'7","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.669266+0000","last_change":"2024-09-11T21:56:26.376206+0000","last_active":"2024-09-11T21:57:13.669266+0000","last_peered":"2024-09-11T21:57:13.669266+0000","last_clean":"2024-09-11T21:57:13.669266+0000","last_became_active":"2024-09-11T21:56:26.375829+0000","last_became_peered":"2024-09-11T21:56:26.375829+0000","last_unstale":"2024-09-11T21:57:13.669266+0000","last_undegraded":"2024-09-11T21:57:13.669266+0000","last_fullsized":"2024-09-11T21:57:13.669266+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:21:59.484414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.787934+0000","last_change":"2024-09-11T21:55:20.093019+0000","last_active":"2024-09-11T21:57:13.787934+0000","last_peered":"2024-09-11T21:57:13.787934+0000","last_clean":"2024-09-11T21:57:13.787934+0000","last_became_active":"2024-09-11T21:55:20.092691+0000","last_became_peered":"2024-09-11T21:55:20.092691+0000","last_unstale":"2024-09-11T21:57:13.787934+0000","last_undegraded":"2024-09-11T21:57:13.787934+0000","last_fullsized":"2024-09-11T21:57:13.787934+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:04:36.684992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788379+0000","last_change":"2024-09-11T21:55:20.097333+0000","last_active":"2024-09-11T21:57:13.788379+0000","last_peered":"2024-09-11T21:57:13.788379+0000","last_clean":"2024-09-11T21:57:13.788379+0000","last_became_active":"2024-09-11T21:55:20.096752+0000","last_became_peered":"2024-09-11T21:55:20.096752+0000","last_unstale":"2024-09-11T21:57:13.788379+0000","last_undegraded":"2024-09-11T21:57:13.788379+0000","last_fullsized":"2024-09-11T21:57:13.788379+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:27:20.722174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"86'10","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.751322+0000","last_change":"2024-09-11T21:55:20.093972+0000","last_active":"2024-09-11T21:57:13.751322+0000","last_peered":"2024-09-11T21:57:13.751322+0000","last_clean":"2024-09-11T21:57:13.751322+0000","last_became_active":"2024-09-11T21:55:20.093633+0000","last_became_peered":"2024-09-11T21:55:20.093633+0000","last_unstale":"2024-09-11T21:57:13.751322+0000","last_undegraded":"2024-09-11T21:57:13.751322+0000","last_fullsized":"2024-09-11T21:57:13.751322+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:25:18.973332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671334+0000","last_change":"2024-09-11T21:55:20.094239+0000","last_active":"2024-09-11T21:57:13.671334+0000","last_peered":"2024-09-11T21:57:13.671334+0000","last_clean":"2024-09-11T21:57:13.671334+0000","last_became_active":"2024-09-11T21:55:20.093924+0000","last_became_peered":"2024-09-11T21:55:20.093924+0000","last_unstale":"2024-09-11T21:57:13.671334+0000","last_undegraded":"2024-09-11T21:57:13.671334+0000","last_fullsized":"2024-09-11T21:57:13.671334+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:39:53.012780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788075+0000","last_change":"2024-09-11T21:55:20.099335+0000","last_active":"2024-09-11T21:57:13.788075+0000","last_peered":"2024-09-11T21:57:13.788075+0000","last_clean":"2024-09-11T21:57:13.788075+0000","last_became_active":"2024-09-11T21:55:20.098929+0000","last_became_peered":"2024-09-11T21:55:20.098929+0000","last_unstale":"2024-09-11T21:57:13.788075+0000","last_undegraded":"2024-09-11T21:57:13.788075+0000","last_fullsized":"2024-09-11T21:57:13.788075+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:38:43.117829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"86'11","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788526+0000","last_change":"2024-09-11T21:55:20.094068+0000","last_active":"2024-09-11T21:57:13.788526+0000","last_peered":"2024-09-11T21:57:13.788526+0000","last_clean":"2024-09-11T21:57:13.788526+0000","last_became_active":"2024-09-11T21:55:20.093637+0000","last_became_peered":"2024-09-11T21:55:20.093637+0000","last_unstale":"2024-09-11T21:57:13.788526+0000","last_undegraded":"2024-09-11T21:57:13.788526+0000","last_fullsized":"2024-09-11T21:57:13.788526+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:52:59.289715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"86'10","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788122+0000","last_change":"2024-09-11T21:55:20.090778+0000","last_active":"2024-09-11T21:57:13.788122+0000","last_peered":"2024-09-11T21:57:13.788122+0000","last_clean":"2024-09-11T21:57:13.788122+0000","last_became_active":"2024-09-11T21:55:20.090443+0000","last_became_peered":"2024-09-11T21:55:20.090443+0000","last_unstale":"2024-09-11T21:57:13.788122+0000","last_undegraded":"2024-09-11T21:57:13.788122+0000","last_fullsized":"2024-09-11T21:57:13.788122+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:46:21.488228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.753203+0000","last_change":"2024-09-11T21:55:20.092238+0000","last_active":"2024-09-11T21:57:13.753203+0000","last_peered":"2024-09-11T21:57:13.753203+0000","last_clean":"2024-09-11T21:57:13.753203+0000","last_became_active":"2024-09-11T21:55:20.091961+0000","last_became_peered":"2024-09-11T21:55:20.091961+0000","last_unstale":"2024-09-11T21:57:13.753203+0000","last_undegraded":"2024-09-11T21:57:13.753203+0000","last_fullsized":"2024-09-11T21:57:13.753203+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:54:40.445441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"86'6","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.761984+0000","last_change":"2024-09-11T21:55:20.102944+0000","last_active":"2024-09-11T21:57:13.761984+0000","last_peered":"2024-09-11T21:57:13.761984+0000","last_clean":"2024-09-11T21:57:13.761984+0000","last_became_active":"2024-09-11T21:55:20.102533+0000","last_became_peered":"2024-09-11T21:55:20.102533+0000","last_unstale":"2024-09-11T21:57:13.761984+0000","last_undegraded":"2024-09-11T21:57:13.761984+0000","last_fullsized":"2024-09-11T21:57:13.761984+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:12:15.832622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"86'2","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.753729+0000","last_change":"2024-09-11T21:56:26.292272+0000","last_active":"2024-09-11T21:57:13.753729+0000","last_peered":"2024-09-11T21:57:13.753729+0000","last_clean":"2024-09-11T21:57:13.753729+0000","last_became_active":"2024-09-11T21:56:26.291920+0000","last_became_peered":"2024-09-11T21:56:26.291920+0000","last_unstale":"2024-09-11T21:57:13.753729+0000","last_undegraded":"2024-09-11T21:57:13.753729+0000","last_fullsized":"2024-09-11T21:57:13.753729+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:13:13.579907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"86'15","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671489+0000","last_change":"2024-09-11T21:56:26.376364+0000","last_active":"2024-09-11T21:57:13.671489+0000","last_peered":"2024-09-11T21:57:13.671489+0000","last_clean":"2024-09-11T21:57:13.671489+0000","last_became_active":"2024-09-11T21:56:26.375927+0000","last_became_peered":"2024-09-11T21:56:26.375927+0000","last_unstale":"2024-09-11T21:57:13.671489+0000","last_undegraded":"2024-09-11T21:57:13.671489+0000","last_fullsized":"2024-09-11T21:57:13.671489+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:23:03.664207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,9],"acting":[2,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"86'6","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788153+0000","last_change":"2024-09-11T21:56:26.376702+0000","last_active":"2024-09-11T21:57:13.788153+0000","last_peered":"2024-09-11T21:57:13.788153+0000","last_clean":"2024-09-11T21:57:13.788153+0000","last_became_active":"2024-09-11T21:56:26.376179+0000","last_became_peered":"2024-09-11T21:56:26.376179+0000","last_unstale":"2024-09-11T21:57:13.788153+0000","last_undegraded":"2024-09-11T21:57:13.788153+0000","last_fullsized":"2024-09-11T21:57:13.788153+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:27:34.666962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"86'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788960+0000","last_change":"2024-09-11T21:55:20.093898+0000","last_active":"2024-09-11T21:57:13.788960+0000","last_peered":"2024-09-11T21:57:13.788960+0000","last_clean":"2024-09-11T21:57:13.788960+0000","last_became_active":"2024-09-11T21:55:20.093348+0000","last_became_peered":"2024-09-11T21:55:20.093348+0000","last_unstale":"2024-09-11T21:57:13.788960+0000","last_undegraded":"2024-09-11T21:57:13.788960+0000","last_fullsized":"2024-09-11T21:57:13.788960+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:00:06.185478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.669457+0000","last_change":"2024-09-11T21:55:20.100575+0000","last_active":"2024-09-11T21:57:13.669457+0000","last_peered":"2024-09-11T21:57:13.669457+0000","last_clean":"2024-09-11T21:57:13.669457+0000","last_became_active":"2024-09-11T21:55:20.100255+0000","last_became_peered":"2024-09-11T21:55:20.100255+0000","last_unstale":"2024-09-11T21:57:13.669457+0000","last_undegraded":"2024-09-11T21:57:13.669457+0000","last_fullsized":"2024-09-11T21:57:13.669457+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:48:57.718298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"86'8","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.670722+0000","last_change":"2024-09-11T21:56:26.377183+0000","last_active":"2024-09-11T21:57:13.670722+0000","last_peered":"2024-09-11T21:57:13.670722+0000","last_clean":"2024-09-11T21:57:13.670722+0000","last_became_active":"2024-09-11T21:56:26.375906+0000","last_became_peered":"2024-09-11T21:56:26.375906+0000","last_unstale":"2024-09-11T21:57:13.670722+0000","last_undegraded":"2024-09-11T21:57:13.670722+0000","last_fullsized":"2024-09-11T21:57:13.670722+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:00:29.033961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"86'12","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.670784+0000","last_change":"2024-09-11T21:55:20.093822+0000","last_active":"2024-09-11T21:57:13.670784+0000","last_peered":"2024-09-11T21:57:13.670784+0000","last_clean":"2024-09-11T21:57:13.670784+0000","last_became_active":"2024-09-11T21:55:20.093361+0000","last_became_peered":"2024-09-11T21:55:20.093361+0000","last_unstale":"2024-09-11T21:57:13.670784+0000","last_undegraded":"2024-09-11T21:57:13.670784+0000","last_fullsized":"2024-09-11T21:57:13.670784+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:37:06.362732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.787998+0000","last_change":"2024-09-11T21:55:20.103990+0000","last_active":"2024-09-11T21:57:13.787998+0000","last_peered":"2024-09-11T21:57:13.787998+0000","last_clean":"2024-09-11T21:57:13.787998+0000","last_became_active":"2024-09-11T21:55:20.103375+0000","last_became_peered":"2024-09-11T21:55:20.103375+0000","last_unstale":"2024-09-11T21:57:13.787998+0000","last_undegraded":"2024-09-11T21:57:13.787998+0000","last_fullsized":"2024-09-11T21:57:13.787998+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:02:06.786888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"86'7","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671024+0000","last_change":"2024-09-11T21:56:26.376261+0000","last_active":"2024-09-11T21:57:13.671024+0000","last_peered":"2024-09-11T21:57:13.671024+0000","last_clean":"2024-09-11T21:57:13.671024+0000","last_became_active":"2024-09-11T21:56:26.375863+0000","last_became_peered":"2024-09-11T21:56:26.375863+0000","last_unstale":"2024-09-11T21:57:13.671024+0000","last_undegraded":"2024-09-11T21:57:13.671024+0000","last_fullsized":"2024-09-11T21:57:13.671024+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:58:21.359836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"87'8","reported_seq":140,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:14.299174+0000","last_change":"2024-09-11T21:55:18.026947+0000","last_active":"2024-09-11T21:57:14.299174+0000","last_peered":"2024-09-11T21:57:14.299174+0000","last_clean":"2024-09-11T21:57:14.299174+0000","last_became_active":"2024-09-11T21:55:18.026632+0000","last_became_peered":"2024-09-11T21:55:18.026632+0000","last_unstale":"2024-09-11T21:57:14.299174+0000","last_undegraded":"2024-09-11T21:57:14.299174+0000","last_fullsized":"2024-09-11T21:57:14.299174+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:55:16.618074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1549,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":108,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.762157+0000","last_change":"2024-09-11T21:55:20.100197+0000","last_active":"2024-09-11T21:57:13.762157+0000","last_peered":"2024-09-11T21:57:13.762157+0000","last_clean":"2024-09-11T21:57:13.762157+0000","last_became_active":"2024-09-11T21:55:20.098494+0000","last_became_peered":"2024-09-11T21:55:20.098494+0000","last_unstale":"2024-09-11T21:57:13.762157+0000","last_undegraded":"2024-09-11T21:57:13.762157+0000","last_fullsized":"2024-09-11T21:57:13.762157+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:14.505314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"84'7","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.765023+0000","last_change":"2024-09-11T21:55:18.024359+0000","last_active":"2024-09-11T21:57:13.765023+0000","last_peered":"2024-09-11T21:57:13.765023+0000","last_clean":"2024-09-11T21:57:13.765023+0000","last_became_active":"2024-09-11T21:55:18.023624+0000","last_became_peered":"2024-09-11T21:55:18.023624+0000","last_unstale":"2024-09-11T21:57:13.765023+0000","last_undegraded":"2024-09-11T21:57:13.765023+0000","last_fullsized":"2024-09-11T21:57:13.765023+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:13:38.582232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"84'3","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788356+0000","last_change":"2024-09-11T21:55:18.032541+0000","last_active":"2024-09-11T21:57:13.788356+0000","last_peered":"2024-09-11T21:57:13.788356+0000","last_clean":"2024-09-11T21:57:13.788356+0000","last_became_active":"2024-09-11T21:55:18.032218+0000","last_became_peered":"2024-09-11T21:55:18.032218+0000","last_unstale":"2024-09-11T21:57:13.788356+0000","last_undegraded":"2024-09-11T21:57:13.788356+0000","last_fullsized":"2024-09-11T21:57:13.788356+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:38:20.872069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.753229+0000","last_change":"2024-09-11T21:55:20.090030+0000","last_active":"2024-09-11T21:57:13.753229+0000","last_peered":"2024-09-11T21:57:13.753229+0000","last_clean":"2024-09-11T21:57:13.753229+0000","last_became_active":"2024-09-11T21:55:20.089481+0000","last_became_peered":"2024-09-11T21:55:20.089481+0000","last_unstale":"2024-09-11T21:57:13.753229+0000","last_undegraded":"2024-09-11T21:57:13.753229+0000","last_fullsized":"2024-09-11T21:57:13.753229+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:41:57.038519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"84'3","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.762394+0000","last_change":"2024-09-11T21:55:18.026656+0000","last_active":"2024-09-11T21:57:13.762394+0000","last_peered":"2024-09-11T21:57:13.762394+0000","last_clean":"2024-09-11T21:57:13.762394+0000","last_became_active":"2024-09-11T21:55:18.026318+0000","last_became_peered":"2024-09-11T21:55:18.026318+0000","last_unstale":"2024-09-11T21:57:13.762394+0000","last_undegraded":"2024-09-11T21:57:13.762394+0000","last_fullsized":"2024-09-11T21:57:13.762394+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:14:41.438334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"86'11","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.750970+0000","last_change":"2024-09-11T21:55:20.090790+0000","last_active":"2024-09-11T21:57:13.750970+0000","last_peered":"2024-09-11T21:57:13.750970+0000","last_clean":"2024-09-11T21:57:13.750970+0000","last_became_active":"2024-09-11T21:55:20.090468+0000","last_became_peered":"2024-09-11T21:55:20.090468+0000","last_unstale":"2024-09-11T21:57:13.750970+0000","last_undegraded":"2024-09-11T21:57:13.750970+0000","last_fullsized":"2024-09-11T21:57:13.750970+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:19.115482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'8","reported_seq":238,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:14.298769+0000","last_change":"2024-09-11T21:55:18.026989+0000","last_active":"2024-09-11T21:57:14.298769+0000","last_peered":"2024-09-11T21:57:14.298769+0000","last_clean":"2024-09-11T21:57:14.298769+0000","last_became_active":"2024-09-11T21:55:18.026229+0000","last_became_peered":"2024-09-11T21:55:18.026229+0000","last_unstale":"2024-09-11T21:57:14.298769+0000","last_undegraded":"2024-09-11T21:57:14.298769+0000","last_fullsized":"2024-09-11T21:57:14.298769+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:50:35.745789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":206,"num_read_kb":206,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"86'10","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788388+0000","last_change":"2024-09-11T21:55:20.095884+0000","last_active":"2024-09-11T21:57:13.788388+0000","last_peered":"2024-09-11T21:57:13.788388+0000","last_clean":"2024-09-11T21:57:13.788388+0000","last_became_active":"2024-09-11T21:55:20.094887+0000","last_became_peered":"2024-09-11T21:55:20.094887+0000","last_unstale":"2024-09-11T21:57:13.788388+0000","last_undegraded":"2024-09-11T21:57:13.788388+0000","last_fullsized":"2024-09-11T21:57:13.788388+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:45:03.579703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"84'4","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671167+0000","last_change":"2024-09-11T21:55:18.030474+0000","last_active":"2024-09-11T21:57:13.671167+0000","last_peered":"2024-09-11T21:57:13.671167+0000","last_clean":"2024-09-11T21:57:13.671167+0000","last_became_active":"2024-09-11T21:55:18.030104+0000","last_became_peered":"2024-09-11T21:55:18.030104+0000","last_unstale":"2024-09-11T21:57:13.671167+0000","last_undegraded":"2024-09-11T21:57:13.671167+0000","last_fullsized":"2024-09-11T21:57:13.671167+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:21:00.484595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.669795+0000","last_change":"2024-09-11T21:55:20.102807+0000","last_active":"2024-09-11T21:57:13.669795+0000","last_peered":"2024-09-11T21:57:13.669795+0000","last_clean":"2024-09-11T21:57:13.669795+0000","last_became_active":"2024-09-11T21:55:20.102581+0000","last_became_peered":"2024-09-11T21:55:20.102581+0000","last_unstale":"2024-09-11T21:57:13.669795+0000","last_undegraded":"2024-09-11T21:57:13.669795+0000","last_fullsized":"2024-09-11T21:57:13.669795+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:29:23.139788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"84'5","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.762200+0000","last_change":"2024-09-11T21:55:18.117685+0000","last_active":"2024-09-11T21:57:13.762200+0000","last_peered":"2024-09-11T21:57:13.762200+0000","last_clean":"2024-09-11T21:57:13.762200+0000","last_became_active":"2024-09-11T21:55:18.117261+0000","last_became_peered":"2024-09-11T21:55:18.117261+0000","last_unstale":"2024-09-11T21:57:13.762200+0000","last_undegraded":"2024-09-11T21:57:13.762200+0000","last_fullsized":"2024-09-11T21:57:13.762200+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:42:13.180725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"86'11","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788081+0000","last_change":"2024-09-11T21:55:20.086562+0000","last_active":"2024-09-11T21:57:13.788081+0000","last_peered":"2024-09-11T21:57:13.788081+0000","last_clean":"2024-09-11T21:57:13.788081+0000","last_became_active":"2024-09-11T21:55:20.086197+0000","last_became_peered":"2024-09-11T21:55:20.086197+0000","last_unstale":"2024-09-11T21:57:13.788081+0000","last_undegraded":"2024-09-11T21:57:13.788081+0000","last_fullsized":"2024-09-11T21:57:13.788081+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:23:44.705132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'35","reported_seq":296,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:14.358002+0000","last_change":"2024-09-11T21:56:26.299386+0000","last_active":"2024-09-11T21:57:14.358002+0000","last_peered":"2024-09-11T21:57:14.358002+0000","last_clean":"2024-09-11T21:57:14.358002+0000","last_became_active":"2024-09-11T21:56:26.291951+0000","last_became_peered":"2024-09-11T21:56:26.291951+0000","last_unstale":"2024-09-11T21:57:14.358002+0000","last_undegraded":"2024-09-11T21:57:14.358002+0000","last_fullsized":"2024-09-11T21:57:14.358002+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:54:47.346798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65927,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":209,"num_read_kb":169,"num_write":37,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":59211,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"86'6","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671198+0000","last_change":"2024-09-11T21:55:20.091695+0000","last_active":"2024-09-11T21:57:13.671198+0000","last_peered":"2024-09-11T21:57:13.671198+0000","last_clean":"2024-09-11T21:57:13.671198+0000","last_became_active":"2024-09-11T21:55:20.091128+0000","last_became_peered":"2024-09-11T21:55:20.091128+0000","last_unstale":"2024-09-11T21:57:13.671198+0000","last_undegraded":"2024-09-11T21:57:13.671198+0000","last_fullsized":"2024-09-11T21:57:13.671198+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:16:20.751952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"84'9","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671024+0000","last_change":"2024-09-11T21:55:18.030827+0000","last_active":"2024-09-11T21:57:13.671024+0000","last_peered":"2024-09-11T21:57:13.671024+0000","last_clean":"2024-09-11T21:57:13.671024+0000","last_became_active":"2024-09-11T21:55:18.030354+0000","last_became_peered":"2024-09-11T21:55:18.030354+0000","last_unstale":"2024-09-11T21:57:13.671024+0000","last_undegraded":"2024-09-11T21:57:13.671024+0000","last_fullsized":"2024-09-11T21:57:13.671024+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:23:19.578820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788550+0000","last_change":"2024-09-11T21:55:20.090160+0000","last_active":"2024-09-11T21:57:13.788550+0000","last_peered":"2024-09-11T21:57:13.788550+0000","last_clean":"2024-09-11T21:57:13.788550+0000","last_became_active":"2024-09-11T21:55:20.089801+0000","last_became_peered":"2024-09-11T21:55:20.089801+0000","last_unstale":"2024-09-11T21:57:13.788550+0000","last_undegraded":"2024-09-11T21:57:13.788550+0000","last_fullsized":"2024-09-11T21:57:13.788550+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:34:16.093123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"84'5","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.761998+0000","last_change":"2024-09-11T21:55:18.026817+0000","last_active":"2024-09-11T21:57:13.761998+0000","last_peered":"2024-09-11T21:57:13.761998+0000","last_clean":"2024-09-11T21:57:13.761998+0000","last_became_active":"2024-09-11T21:55:18.026188+0000","last_became_peered":"2024-09-11T21:55:18.026188+0000","last_unstale":"2024-09-11T21:57:13.761998+0000","last_undegraded":"2024-09-11T21:57:13.761998+0000","last_fullsized":"2024-09-11T21:57:13.761998+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:55:14.774503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"86'15","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.670991+0000","last_change":"2024-09-11T21:55:20.095613+0000","last_active":"2024-09-11T21:57:13.670991+0000","last_peered":"2024-09-11T21:57:13.670991+0000","last_clean":"2024-09-11T21:57:13.670991+0000","last_became_active":"2024-09-11T21:55:20.095302+0000","last_became_peered":"2024-09-11T21:55:20.095302+0000","last_unstale":"2024-09-11T21:57:13.670991+0000","last_undegraded":"2024-09-11T21:57:13.670991+0000","last_fullsized":"2024-09-11T21:57:13.670991+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:37:27.961213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'7","reported_seq":138,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:14.300345+0000","last_change":"2024-09-11T21:55:18.032417+0000","last_active":"2024-09-11T21:57:14.300345+0000","last_peered":"2024-09-11T21:57:14.300345+0000","last_clean":"2024-09-11T21:57:14.300345+0000","last_became_active":"2024-09-11T21:55:18.031419+0000","last_became_peered":"2024-09-11T21:55:18.031419+0000","last_unstale":"2024-09-11T21:57:14.300345+0000","last_undegraded":"2024-09-11T21:57:14.300345+0000","last_fullsized":"2024-09-11T21:57:14.300345+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:30:52.285678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1289,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":108,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"86'6","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.670730+0000","last_change":"2024-09-11T21:55:20.101965+0000","last_active":"2024-09-11T21:57:13.670730+0000","last_peered":"2024-09-11T21:57:13.670730+0000","last_clean":"2024-09-11T21:57:13.670730+0000","last_became_active":"2024-09-11T21:55:20.101534+0000","last_became_peered":"2024-09-11T21:55:20.101534+0000","last_unstale":"2024-09-11T21:57:13.670730+0000","last_undegraded":"2024-09-11T21:57:13.670730+0000","last_fullsized":"2024-09-11T21:57:13.670730+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:41:30.685121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"84'17","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.670811+0000","last_change":"2024-09-11T21:55:18.116059+0000","last_active":"2024-09-11T21:57:13.670811+0000","last_peered":"2024-09-11T21:57:13.670811+0000","last_clean":"2024-09-11T21:57:13.670811+0000","last_became_active":"2024-09-11T21:55:18.115621+0000","last_became_peered":"2024-09-11T21:55:18.115621+0000","last_unstale":"2024-09-11T21:57:13.670811+0000","last_undegraded":"2024-09-11T21:57:13.670811+0000","last_fullsized":"2024-09-11T21:57:13.670811+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:47:17.815695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"86'6","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.762037+0000","last_change":"2024-09-11T21:55:20.100903+0000","last_active":"2024-09-11T21:57:13.762037+0000","last_peered":"2024-09-11T21:57:13.762037+0000","last_clean":"2024-09-11T21:57:13.762037+0000","last_became_active":"2024-09-11T21:55:20.100561+0000","last_became_peered":"2024-09-11T21:55:20.100561+0000","last_unstale":"2024-09-11T21:57:13.762037+0000","last_undegraded":"2024-09-11T21:57:13.762037+0000","last_fullsized":"2024-09-11T21:57:13.762037+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:00:45.472238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'2","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671632+0000","last_change":"2024-09-11T21:55:18.029549+0000","last_active":"2024-09-11T21:57:13.671632+0000","last_peered":"2024-09-11T21:57:13.671632+0000","last_clean":"2024-09-11T21:57:13.671632+0000","last_became_active":"2024-09-11T21:55:18.029178+0000","last_became_peered":"2024-09-11T21:55:18.029178+0000","last_unstale":"2024-09-11T21:57:13.671632+0000","last_undegraded":"2024-09-11T21:57:13.671632+0000","last_fullsized":"2024-09-11T21:57:13.671632+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:37:07.915786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"86'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788144+0000","last_change":"2024-09-11T21:55:20.104202+0000","last_active":"2024-09-11T21:57:13.788144+0000","last_peered":"2024-09-11T21:57:13.788144+0000","last_clean":"2024-09-11T21:57:13.788144+0000","last_became_active":"2024-09-11T21:55:20.103464+0000","last_became_peered":"2024-09-11T21:55:20.103464+0000","last_unstale":"2024-09-11T21:57:13.788144+0000","last_undegraded":"2024-09-11T21:57:13.788144+0000","last_fullsized":"2024-09-11T21:57:13.788144+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:30:16.044143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"84'5","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788728+0000","last_change":"2024-09-11T21:55:18.026183+0000","last_active":"2024-09-11T21:57:13.788728+0000","last_peered":"2024-09-11T21:57:13.788728+0000","last_clean":"2024-09-11T21:57:13.788728+0000","last_became_active":"2024-09-11T21:55:18.025849+0000","last_became_peered":"2024-09-11T21:55:18.025849+0000","last_unstale":"2024-09-11T21:57:13.788728+0000","last_undegraded":"2024-09-11T21:57:13.788728+0000","last_fullsized":"2024-09-11T21:57:13.788728+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:21:12.189158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671103+0000","last_change":"2024-09-11T21:55:20.090652+0000","last_active":"2024-09-11T21:57:13.671103+0000","last_peered":"2024-09-11T21:57:13.671103+0000","last_clean":"2024-09-11T21:57:13.671103+0000","last_became_active":"2024-09-11T21:55:20.090348+0000","last_became_peered":"2024-09-11T21:55:20.090348+0000","last_unstale":"2024-09-11T21:57:13.671103+0000","last_undegraded":"2024-09-11T21:57:13.671103+0000","last_fullsized":"2024-09-11T21:57:13.671103+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:10:58.733371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'41","reported_seq":73,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.873064+0000","last_change":"2024-09-11T21:55:18.024388+0000","last_active":"2024-09-11T21:57:13.873064+0000","last_peered":"2024-09-11T21:57:13.873064+0000","last_clean":"2024-09-11T21:57:13.873064+0000","last_became_active":"2024-09-11T21:55:18.024100+0000","last_became_peered":"2024-09-11T21:55:18.024100+0000","last_unstale":"2024-09-11T21:57:13.873064+0000","last_undegraded":"2024-09-11T21:57:13.873064+0000","last_fullsized":"2024-09-11T21:57:13.873064+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:21:11.152282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.670936+0000","last_change":"2024-09-11T21:55:20.101007+0000","last_active":"2024-09-11T21:57:13.670936+0000","last_peered":"2024-09-11T21:57:13.670936+0000","last_clean":"2024-09-11T21:57:13.670936+0000","last_became_active":"2024-09-11T21:55:20.100642+0000","last_became_peered":"2024-09-11T21:55:20.100642+0000","last_unstale":"2024-09-11T21:57:13.670936+0000","last_undegraded":"2024-09-11T21:57:13.670936+0000","last_fullsized":"2024-09-11T21:57:13.670936+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:16:05.642184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"87'8","reported_seq":140,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:14.325956+0000","last_change":"2024-09-11T21:55:18.116216+0000","last_active":"2024-09-11T21:57:14.325956+0000","last_peered":"2024-09-11T21:57:14.325956+0000","last_clean":"2024-09-11T21:57:14.325956+0000","last_became_active":"2024-09-11T21:55:18.115871+0000","last_became_peered":"2024-09-11T21:55:18.115871+0000","last_unstale":"2024-09-11T21:57:14.325956+0000","last_undegraded":"2024-09-11T21:57:14.325956+0000","last_fullsized":"2024-09-11T21:57:14.325956+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:16:56.027636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":108,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671175+0000","last_change":"2024-09-11T21:55:20.092421+0000","last_active":"2024-09-11T21:57:13.671175+0000","last_peered":"2024-09-11T21:57:13.671175+0000","last_clean":"2024-09-11T21:57:13.671175+0000","last_became_active":"2024-09-11T21:55:20.091707+0000","last_became_peered":"2024-09-11T21:55:20.091707+0000","last_unstale":"2024-09-11T21:57:13.671175+0000","last_undegraded":"2024-09-11T21:57:13.671175+0000","last_fullsized":"2024-09-11T21:57:13.671175+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:12:38.157687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":353,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.670940+0000","last_change":"2024-09-11T21:54:07.542362+0000","last_active":"2024-09-11T21:57:13.670940+0000","last_peered":"2024-09-11T21:57:13.670940+0000","last_clean":"2024-09-11T21:57:13.670940+0000","last_became_active":"2024-09-11T21:54:07.541871+0000","last_became_peered":"2024-09-11T21:54:07.541871+0000","last_unstale":"2024-09-11T21:57:13.670940+0000","last_undegraded":"2024-09-11T21:57:13.670940+0000","last_fullsized":"2024-09-11T21:57:13.670940+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:50:41.026895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:50:41.026895+0000","last_clean_scrub_stamp":"2024-09-11T21:50:41.026895+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:05:06.154268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"84'6","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788499+0000","last_change":"2024-09-11T21:55:18.030558+0000","last_active":"2024-09-11T21:57:13.788499+0000","last_peered":"2024-09-11T21:57:13.788499+0000","last_clean":"2024-09-11T21:57:13.788499+0000","last_became_active":"2024-09-11T21:55:18.030216+0000","last_became_peered":"2024-09-11T21:55:18.030216+0000","last_unstale":"2024-09-11T21:57:13.788499+0000","last_undegraded":"2024-09-11T21:57:13.788499+0000","last_fullsized":"2024-09-11T21:57:13.788499+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:33:02.589999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.753426+0000","last_change":"2024-09-11T21:55:20.092938+0000","last_active":"2024-09-11T21:57:13.753426+0000","last_peered":"2024-09-11T21:57:13.753426+0000","last_clean":"2024-09-11T21:57:13.753426+0000","last_became_active":"2024-09-11T21:55:20.092615+0000","last_became_peered":"2024-09-11T21:55:20.092615+0000","last_unstale":"2024-09-11T21:57:13.753426+0000","last_undegraded":"2024-09-11T21:57:13.753426+0000","last_fullsized":"2024-09-11T21:57:13.753426+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:40:06.957836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"84'4","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.670838+0000","last_change":"2024-09-11T21:55:18.029127+0000","last_active":"2024-09-11T21:57:13.670838+0000","last_peered":"2024-09-11T21:57:13.670838+0000","last_clean":"2024-09-11T21:57:13.670838+0000","last_became_active":"2024-09-11T21:55:18.028740+0000","last_became_peered":"2024-09-11T21:55:18.028740+0000","last_unstale":"2024-09-11T21:57:13.670838+0000","last_undegraded":"2024-09-11T21:57:13.670838+0000","last_fullsized":"2024-09-11T21:57:13.670838+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:14:01.128486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"86'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788303+0000","last_change":"2024-09-11T21:55:20.092498+0000","last_active":"2024-09-11T21:57:13.788303+0000","last_peered":"2024-09-11T21:57:13.788303+0000","last_clean":"2024-09-11T21:57:13.788303+0000","last_became_active":"2024-09-11T21:55:20.092179+0000","last_became_peered":"2024-09-11T21:55:20.092179+0000","last_unstale":"2024-09-11T21:57:13.788303+0000","last_undegraded":"2024-09-11T21:57:13.788303+0000","last_fullsized":"2024-09-11T21:57:13.788303+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:48:58.536942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"86'6","reported_seq":134,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788554+0000","last_change":"2024-09-11T21:55:18.115798+0000","last_active":"2024-09-11T21:57:13.788554+0000","last_peered":"2024-09-11T21:57:13.788554+0000","last_clean":"2024-09-11T21:57:13.788554+0000","last_became_active":"2024-09-11T21:55:18.115433+0000","last_became_peered":"2024-09-11T21:55:18.115433+0000","last_unstale":"2024-09-11T21:57:13.788554+0000","last_undegraded":"2024-09-11T21:57:13.788554+0000","last_fullsized":"2024-09-11T21:57:13.788554+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:03:32.097756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":104,"num_read_kb":104,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.762043+0000","last_change":"2024-09-11T21:55:20.100439+0000","last_active":"2024-09-11T21:57:13.762043+0000","last_peered":"2024-09-11T21:57:13.762043+0000","last_clean":"2024-09-11T21:57:13.762043+0000","last_became_active":"2024-09-11T21:55:20.098717+0000","last_became_peered":"2024-09-11T21:55:20.098717+0000","last_unstale":"2024-09-11T21:57:13.762043+0000","last_undegraded":"2024-09-11T21:57:13.762043+0000","last_fullsized":"2024-09-11T21:57:13.762043+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:15:10.874225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"87'9","reported_seq":140,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:14.298986+0000","last_change":"2024-09-11T21:55:18.026588+0000","last_active":"2024-09-11T21:57:14.298986+0000","last_peered":"2024-09-11T21:57:14.298986+0000","last_clean":"2024-09-11T21:57:14.298986+0000","last_became_active":"2024-09-11T21:55:18.026205+0000","last_became_peered":"2024-09-11T21:55:18.026205+0000","last_unstale":"2024-09-11T21:57:14.298986+0000","last_undegraded":"2024-09-11T21:57:14.298986+0000","last_fullsized":"2024-09-11T21:57:14.298986+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:45:34.701088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1908,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":107,"num_read_kb":4,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.765120+0000","last_change":"2024-09-11T21:55:20.100677+0000","last_active":"2024-09-11T21:57:13.765120+0000","last_peered":"2024-09-11T21:57:13.765120+0000","last_clean":"2024-09-11T21:57:13.765120+0000","last_became_active":"2024-09-11T21:55:20.100334+0000","last_became_peered":"2024-09-11T21:55:20.100334+0000","last_unstale":"2024-09-11T21:57:13.765120+0000","last_undegraded":"2024-09-11T21:57:13.765120+0000","last_fullsized":"2024-09-11T21:57:13.765120+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:15:16.792831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"84'1","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.751199+0000","last_change":"2024-09-11T21:55:18.116329+0000","last_active":"2024-09-11T21:57:13.751199+0000","last_peered":"2024-09-11T21:57:13.751199+0000","last_clean":"2024-09-11T21:57:13.751199+0000","last_became_active":"2024-09-11T21:55:18.115944+0000","last_became_peered":"2024-09-11T21:55:18.115944+0000","last_unstale":"2024-09-11T21:57:13.751199+0000","last_undegraded":"2024-09-11T21:57:13.751199+0000","last_fullsized":"2024-09-11T21:57:13.751199+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:03:04.164985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.765132+0000","last_change":"2024-09-11T21:55:20.089202+0000","last_active":"2024-09-11T21:57:13.765132+0000","last_peered":"2024-09-11T21:57:13.765132+0000","last_clean":"2024-09-11T21:57:13.765132+0000","last_became_active":"2024-09-11T21:55:20.088853+0000","last_became_peered":"2024-09-11T21:55:20.088853+0000","last_unstale":"2024-09-11T21:57:13.765132+0000","last_undegraded":"2024-09-11T21:57:13.765132+0000","last_fullsized":"2024-09-11T21:57:13.765132+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:39:36.496944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"84'4","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671429+0000","last_change":"2024-09-11T21:55:18.117917+0000","last_active":"2024-09-11T21:57:13.671429+0000","last_peered":"2024-09-11T21:57:13.671429+0000","last_clean":"2024-09-11T21:57:13.671429+0000","last_became_active":"2024-09-11T21:55:18.117567+0000","last_became_peered":"2024-09-11T21:55:18.117567+0000","last_unstale":"2024-09-11T21:57:13.671429+0000","last_undegraded":"2024-09-11T21:57:13.671429+0000","last_fullsized":"2024-09-11T21:57:13.671429+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:21:32.546620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.789090+0000","last_change":"2024-09-11T21:55:20.092370+0000","last_active":"2024-09-11T21:57:13.789090+0000","last_peered":"2024-09-11T21:57:13.789090+0000","last_clean":"2024-09-11T21:57:13.789090+0000","last_became_active":"2024-09-11T21:55:20.091982+0000","last_became_peered":"2024-09-11T21:55:20.091982+0000","last_unstale":"2024-09-11T21:57:13.789090+0000","last_undegraded":"2024-09-11T21:57:13.789090+0000","last_fullsized":"2024-09-11T21:57:13.789090+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:59:55.214511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'51","reported_seq":98,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.879119+0000","last_change":"2024-09-11T21:55:18.023166+0000","last_active":"2024-09-11T21:57:13.879119+0000","last_peered":"2024-09-11T21:57:13.879119+0000","last_clean":"2024-09-11T21:57:13.879119+0000","last_became_active":"2024-09-11T21:55:18.022845+0000","last_became_peered":"2024-09-11T21:55:18.022845+0000","last_unstale":"2024-09-11T21:57:13.879119+0000","last_undegraded":"2024-09-11T21:57:13.879119+0000","last_fullsized":"2024-09-11T21:57:13.879119+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:08:06.770410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":91,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"86'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.753738+0000","last_change":"2024-09-11T21:55:20.091541+0000","last_active":"2024-09-11T21:57:13.753738+0000","last_peered":"2024-09-11T21:57:13.753738+0000","last_clean":"2024-09-11T21:57:13.753738+0000","last_became_active":"2024-09-11T21:55:20.090971+0000","last_became_peered":"2024-09-11T21:55:20.090971+0000","last_unstale":"2024-09-11T21:57:13.753738+0000","last_undegraded":"2024-09-11T21:57:13.753738+0000","last_fullsized":"2024-09-11T21:57:13.753738+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:22:31.974711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"84'7","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788398+0000","last_change":"2024-09-11T21:55:18.030694+0000","last_active":"2024-09-11T21:57:13.788398+0000","last_peered":"2024-09-11T21:57:13.788398+0000","last_clean":"2024-09-11T21:57:13.788398+0000","last_became_active":"2024-09-11T21:55:18.029687+0000","last_became_peered":"2024-09-11T21:55:18.029687+0000","last_unstale":"2024-09-11T21:57:13.788398+0000","last_undegraded":"2024-09-11T21:57:13.788398+0000","last_fullsized":"2024-09-11T21:57:13.788398+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:18:15.009543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"86'7","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.669625+0000","last_change":"2024-09-11T21:56:26.376104+0000","last_active":"2024-09-11T21:57:13.669625+0000","last_peered":"2024-09-11T21:57:13.669625+0000","last_clean":"2024-09-11T21:57:13.669625+0000","last_became_active":"2024-09-11T21:56:26.375665+0000","last_became_peered":"2024-09-11T21:56:26.375665+0000","last_unstale":"2024-09-11T21:57:13.669625+0000","last_undegraded":"2024-09-11T21:57:13.669625+0000","last_fullsized":"2024-09-11T21:57:13.669625+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:19:46.616205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"84'2","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788029+0000","last_change":"2024-09-11T21:55:18.115977+0000","last_active":"2024-09-11T21:57:13.788029+0000","last_peered":"2024-09-11T21:57:13.788029+0000","last_clean":"2024-09-11T21:57:13.788029+0000","last_became_active":"2024-09-11T21:55:18.115649+0000","last_became_peered":"2024-09-11T21:55:18.115649+0000","last_unstale":"2024-09-11T21:57:13.788029+0000","last_undegraded":"2024-09-11T21:57:13.788029+0000","last_fullsized":"2024-09-11T21:57:13.788029+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:42:07.621024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"86'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.670884+0000","last_change":"2024-09-11T21:55:20.095536+0000","last_active":"2024-09-11T21:57:13.670884+0000","last_peered":"2024-09-11T21:57:13.670884+0000","last_clean":"2024-09-11T21:57:13.670884+0000","last_became_active":"2024-09-11T21:55:20.095117+0000","last_became_peered":"2024-09-11T21:55:20.095117+0000","last_unstale":"2024-09-11T21:57:13.670884+0000","last_undegraded":"2024-09-11T21:57:13.670884+0000","last_fullsized":"2024-09-11T21:57:13.670884+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:46.524922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"84'7","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.753348+0000","last_change":"2024-09-11T21:55:18.026592+0000","last_active":"2024-09-11T21:57:13.753348+0000","last_peered":"2024-09-11T21:57:13.753348+0000","last_clean":"2024-09-11T21:57:13.753348+0000","last_became_active":"2024-09-11T21:55:18.026263+0000","last_became_peered":"2024-09-11T21:55:18.026263+0000","last_unstale":"2024-09-11T21:57:13.753348+0000","last_undegraded":"2024-09-11T21:57:13.753348+0000","last_fullsized":"2024-09-11T21:57:13.753348+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:22:29.382964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788053+0000","last_change":"2024-09-11T21:55:20.093655+0000","last_active":"2024-09-11T21:57:13.788053+0000","last_peered":"2024-09-11T21:57:13.788053+0000","last_clean":"2024-09-11T21:57:13.788053+0000","last_became_active":"2024-09-11T21:55:20.093287+0000","last_became_peered":"2024-09-11T21:55:20.093287+0000","last_unstale":"2024-09-11T21:57:13.788053+0000","last_undegraded":"2024-09-11T21:57:13.788053+0000","last_fullsized":"2024-09-11T21:57:13.788053+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:46:15.912710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"84'3","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.670879+0000","last_change":"2024-09-11T21:55:18.028465+0000","last_active":"2024-09-11T21:57:13.670879+0000","last_peered":"2024-09-11T21:57:13.670879+0000","last_clean":"2024-09-11T21:57:13.670879+0000","last_became_active":"2024-09-11T21:55:18.028139+0000","last_became_peered":"2024-09-11T21:55:18.028139+0000","last_unstale":"2024-09-11T21:57:13.670879+0000","last_undegraded":"2024-09-11T21:57:13.670879+0000","last_fullsized":"2024-09-11T21:57:13.670879+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:52:47.808058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.753473+0000","last_change":"2024-09-11T21:55:20.096400+0000","last_active":"2024-09-11T21:57:13.753473+0000","last_peered":"2024-09-11T21:57:13.753473+0000","last_clean":"2024-09-11T21:57:13.753473+0000","last_became_active":"2024-09-11T21:55:20.096023+0000","last_became_peered":"2024-09-11T21:55:20.096023+0000","last_unstale":"2024-09-11T21:57:13.753473+0000","last_undegraded":"2024-09-11T21:57:13.753473+0000","last_fullsized":"2024-09-11T21:57:13.753473+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:19:34.150687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"84'4","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671181+0000","last_change":"2024-09-11T21:55:18.028759+0000","last_active":"2024-09-11T21:57:13.671181+0000","last_peered":"2024-09-11T21:57:13.671181+0000","last_clean":"2024-09-11T21:57:13.671181+0000","last_became_active":"2024-09-11T21:55:18.028366+0000","last_became_peered":"2024-09-11T21:55:18.028366+0000","last_unstale":"2024-09-11T21:57:13.671181+0000","last_undegraded":"2024-09-11T21:57:13.671181+0000","last_fullsized":"2024-09-11T21:57:13.671181+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:42:05.836114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"86'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671002+0000","last_change":"2024-09-11T21:55:20.093593+0000","last_active":"2024-09-11T21:57:13.671002+0000","last_peered":"2024-09-11T21:57:13.671002+0000","last_clean":"2024-09-11T21:57:13.671002+0000","last_became_active":"2024-09-11T21:55:20.093189+0000","last_became_peered":"2024-09-11T21:55:20.093189+0000","last_unstale":"2024-09-11T21:57:13.671002+0000","last_undegraded":"2024-09-11T21:57:13.671002+0000","last_fullsized":"2024-09-11T21:57:13.671002+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:47:36.802816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"86'15","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.762163+0000","last_change":"2024-09-11T21:56:26.292793+0000","last_active":"2024-09-11T21:57:13.762163+0000","last_peered":"2024-09-11T21:57:13.762163+0000","last_clean":"2024-09-11T21:57:13.762163+0000","last_became_active":"2024-09-11T21:56:26.292437+0000","last_became_peered":"2024-09-11T21:56:26.292437+0000","last_unstale":"2024-09-11T21:57:13.762163+0000","last_undegraded":"2024-09-11T21:57:13.762163+0000","last_fullsized":"2024-09-11T21:57:13.762163+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:49:08.867435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"84'5","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671366+0000","last_change":"2024-09-11T21:55:18.027365+0000","last_active":"2024-09-11T21:57:13.671366+0000","last_peered":"2024-09-11T21:57:13.671366+0000","last_clean":"2024-09-11T21:57:13.671366+0000","last_became_active":"2024-09-11T21:55:18.027058+0000","last_became_peered":"2024-09-11T21:55:18.027058+0000","last_unstale":"2024-09-11T21:57:13.671366+0000","last_undegraded":"2024-09-11T21:57:13.671366+0000","last_fullsized":"2024-09-11T21:57:13.671366+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:30:50.613246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788034+0000","last_change":"2024-09-11T21:55:20.095154+0000","last_active":"2024-09-11T21:57:13.788034+0000","last_peered":"2024-09-11T21:57:13.788034+0000","last_clean":"2024-09-11T21:57:13.788034+0000","last_became_active":"2024-09-11T21:55:20.093607+0000","last_became_peered":"2024-09-11T21:55:20.093607+0000","last_unstale":"2024-09-11T21:57:13.788034+0000","last_undegraded":"2024-09-11T21:57:13.788034+0000","last_fullsized":"2024-09-11T21:57:13.788034+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:27:53.612467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"87'13","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671385+0000","last_change":"2024-09-11T21:55:18.029872+0000","last_active":"2024-09-11T21:57:13.671385+0000","last_peered":"2024-09-11T21:57:13.671385+0000","last_clean":"2024-09-11T21:57:13.671385+0000","last_became_active":"2024-09-11T21:55:18.029538+0000","last_became_peered":"2024-09-11T21:55:18.029538+0000","last_unstale":"2024-09-11T21:57:13.671385+0000","last_undegraded":"2024-09-11T21:57:13.671385+0000","last_fullsized":"2024-09-11T21:57:13.671385+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:05:16.824805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"86'6","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.669783+0000","last_change":"2024-09-11T21:55:20.098713+0000","last_active":"2024-09-11T21:57:13.669783+0000","last_peered":"2024-09-11T21:57:13.669783+0000","last_clean":"2024-09-11T21:57:13.669783+0000","last_became_active":"2024-09-11T21:55:20.098219+0000","last_became_peered":"2024-09-11T21:55:20.098219+0000","last_unstale":"2024-09-11T21:57:13.669783+0000","last_undegraded":"2024-09-11T21:57:13.669783+0000","last_fullsized":"2024-09-11T21:57:13.669783+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:07:10.585366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"84'2","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.753854+0000","last_change":"2024-09-11T21:55:18.115868+0000","last_active":"2024-09-11T21:57:13.753854+0000","last_peered":"2024-09-11T21:57:13.753854+0000","last_clean":"2024-09-11T21:57:13.753854+0000","last_became_active":"2024-09-11T21:55:18.115486+0000","last_became_peered":"2024-09-11T21:55:18.115486+0000","last_unstale":"2024-09-11T21:57:13.753854+0000","last_undegraded":"2024-09-11T21:57:13.753854+0000","last_fullsized":"2024-09-11T21:57:13.753854+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:16:57.932041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"86'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.753686+0000","last_change":"2024-09-11T21:55:20.096077+0000","last_active":"2024-09-11T21:57:13.753686+0000","last_peered":"2024-09-11T21:57:13.753686+0000","last_clean":"2024-09-11T21:57:13.753686+0000","last_became_active":"2024-09-11T21:55:20.095739+0000","last_became_peered":"2024-09-11T21:55:20.095739+0000","last_unstale":"2024-09-11T21:57:13.753686+0000","last_undegraded":"2024-09-11T21:57:13.753686+0000","last_fullsized":"2024-09-11T21:57:13.753686+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:05:28.089828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"87'15","reported_seq":143,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.931982+0000","last_change":"2024-09-11T21:55:18.025771+0000","last_active":"2024-09-11T21:57:13.931982+0000","last_peered":"2024-09-11T21:57:13.931982+0000","last_clean":"2024-09-11T21:57:13.931982+0000","last_became_active":"2024-09-11T21:55:18.025384+0000","last_became_peered":"2024-09-11T21:55:18.025384+0000","last_unstale":"2024-09-11T21:57:13.931982+0000","last_undegraded":"2024-09-11T21:57:13.931982+0000","last_fullsized":"2024-09-11T21:57:13.931982+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:58:21.070061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":104,"num_read_kb":104,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":601040,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1166,"num_read_kb":701,"num_write":380,"num_write_kb":1470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":59211,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":667648,"data_stored":1803120,"data_compressed":85492,"data_compressed_allocated":110592,"data_compressed_original":1605632,"omap_allocated":0,"internal_metadata":0},"log_size":306,"ondisk_log_size":306,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107988,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449124,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":937984,"data_stored":453556,"data_compressed":5771,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678056,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36448,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708704,"statfs":{"total":95995035648,"available":95957712896,"internally_reserved":0,"allocated":1130496,"data_stored":1251264,"data_compressed":50827,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":66,"seq":283467841582,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":974848,"data_stored":461937,"data_compressed":8675,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103220,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":995328,"data_stored":855531,"data_compressed":27821,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332152,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1024000,"data_stored":1057390,"data_compressed":44671,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495677,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":958464,"data_stored":462974,"data_compressed":8675,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757315,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":1186647,"data_compressed":50827,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920839,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":983040,"data_stored":521591,"data_compressed":18261,"data_compressed_allocated":24576,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182477,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":986295,"data_compressed":33977,"data_compressed_allocated":45056,"data_compressed_original":540672,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":970752,"data_stored":464641,"data_compressed":8675,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1024000,"data_stored":529814,"data_compressed":8675,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":978944,"data_stored":463920,"data_compressed":8675,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":68999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":67328,"data_compressed":9586,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":135347,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":6090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":394716,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":199964,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T21:57:16.267 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T21:57:16.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.405+0000 7fcf57f9f640 1 -- 172.21.15.1:0/2873212982 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcf50104f00 msgr2=0x7fcf501072f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:16.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.405+0000 7fcf57f9f640 1 --2- 172.21.15.1:0/2873212982 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcf50104f00 0x7fcf501072f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fcf40009900 tx=0x7fcf4002a410 comp rx=0 tx=0).stop 2024-09-11T21:57:16.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.405+0000 7fcf57f9f640 1 -- 172.21.15.1:0/2873212982 shutdown_connections 2024-09-11T21:57:16.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.405+0000 7fcf57f9f640 1 --2- 172.21.15.1:0/2873212982 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf50107830 0x7fcf50109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:16.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.405+0000 7fcf57f9f640 1 --2- 172.21.15.1:0/2873212982 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcf50104f00 0x7fcf501072f0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:16.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.405+0000 7fcf57f9f640 1 --2- 172.21.15.1:0/2873212982 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf501025d0 0x7fcf501049c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:16.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.405+0000 7fcf57f9f640 1 -- 172.21.15.1:0/2873212982 >> 172.21.15.1:0/2873212982 conn(0x7fcf500fc190 msgr2=0x7fcf500fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:16.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.405+0000 7fcf57f9f640 1 -- 172.21.15.1:0/2873212982 shutdown_connections 2024-09-11T21:57:16.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.406+0000 7fcf57f9f640 1 -- 172.21.15.1:0/2873212982 wait complete. 2024-09-11T21:57:16.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.406+0000 7fcf57f9f640 1 Processor -- start 2024-09-11T21:57:16.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.406+0000 7fcf57f9f640 1 -- start start 2024-09-11T21:57:16.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.406+0000 7fcf57f9f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf501025d0 0x7fcf50101f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:16.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.406+0000 7fcf57f9f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf50104f00 0x7fcf50100580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:16.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.406+0000 7fcf57f9f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcf50107830 0x7fcf50100ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:16.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.406+0000 7fcf57f9f640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcf5010d800 con 0x7fcf50107830 2024-09-11T21:57:16.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.406+0000 7fcf57f9f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcf5010d680 con 0x7fcf501025d0 2024-09-11T21:57:16.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.406+0000 7fcf57f9f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcf5010d980 con 0x7fcf50104f00 2024-09-11T21:57:16.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.406+0000 7fcf5653c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcf50107830 0x7fcf50100ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:16.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.406+0000 7fcf55d3b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf501025d0 0x7fcf50101f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:16.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.406+0000 7fcf5653c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcf50107830 0x7fcf50100ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46930/0 (socket says 172.21.15.1:46930) 2024-09-11T21:57:16.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.407+0000 7fcf55d3b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf501025d0 0x7fcf50101f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:54574/0 (socket says 172.21.15.1:54574) 2024-09-11T21:57:16.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.407+0000 7fcf5653c640 1 -- 172.21.15.1:0/1476188121 learned_addr learned my addr 172.21.15.1:0/1476188121 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:16.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.407+0000 7fcf5553a640 1 --2- 172.21.15.1:0/1476188121 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf50104f00 0x7fcf50100580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:16.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.407+0000 7fcf5653c640 1 -- 172.21.15.1:0/1476188121 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf50104f00 msgr2=0x7fcf50100580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:16.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.407+0000 7fcf5653c640 1 --2- 172.21.15.1:0/1476188121 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf50104f00 0x7fcf50100580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:16.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.407+0000 7fcf5653c640 1 -- 172.21.15.1:0/1476188121 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf501025d0 msgr2=0x7fcf50101f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:16.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.407+0000 7fcf5653c640 1 --2- 172.21.15.1:0/1476188121 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf501025d0 0x7fcf50101f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:16.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.407+0000 7fcf5653c640 1 -- 172.21.15.1:0/1476188121 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf50101350 con 0x7fcf50107830 2024-09-11T21:57:16.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.407+0000 7fcf5653c640 1 --2- 172.21.15.1:0/1476188121 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcf50107830 0x7fcf50100ac0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fcf4800e970 tx=0x7fcf4800ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:16.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.407+0000 7fcf46ffd640 1 -- 172.21.15.1:0/1476188121 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf4800ccc0 con 0x7fcf50107830 2024-09-11T21:57:16.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.407+0000 7fcf46ffd640 1 -- 172.21.15.1:0/1476188121 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf48005500 con 0x7fcf50107830 2024-09-11T21:57:16.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.407+0000 7fcf57f9f640 1 -- 172.21.15.1:0/1476188121 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf501a1fe0 con 0x7fcf50107830 2024-09-11T21:57:16.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.407+0000 7fcf46ffd640 1 -- 172.21.15.1:0/1476188121 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf48010690 con 0x7fcf50107830 2024-09-11T21:57:16.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.407+0000 7fcf57f9f640 1 -- 172.21.15.1:0/1476188121 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf501a2450 con 0x7fcf50107830 2024-09-11T21:57:16.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.408+0000 7fcf57f9f640 1 -- 172.21.15.1:0/1476188121 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf500fdc80 con 0x7fcf50107830 2024-09-11T21:57:16.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.409+0000 7fcf46ffd640 1 -- 172.21.15.1:0/1476188121 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 23) ==== 137952+0+0 (secure 0 0 0) 0x7fcf480046c0 con 0x7fcf50107830 2024-09-11T21:57:16.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.410+0000 7fcf46ffd640 1 --2- 172.21.15.1:0/1476188121 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fcf3c0a2d00 0x7fcf3c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:16.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.410+0000 7fcf55d3b640 1 --2- 172.21.15.1:0/1476188121 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fcf3c0a2d00 0x7fcf3c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:16.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.410+0000 7fcf46ffd640 1 -- 172.21.15.1:0/1476188121 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7fcf480cfb00 con 0x7fcf50107830 2024-09-11T21:57:16.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.410+0000 7fcf55d3b640 1 --2- 172.21.15.1:0/1476188121 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fcf3c0a2d00 0x7fcf3c0a51c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c005fd0 tx=0x7fcf4c009500 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:16.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.413+0000 7fcf46ffd640 1 -- 172.21.15.1:0/1476188121 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcf480541b0 con 0x7fcf50107830 2024-09-11T21:57:16.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.603+0000 7fcf57f9f640 1 -- 172.21.15.1:0/1476188121 --> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcf50062630 con 0x7fcf3c0a2d00 2024-09-11T21:57:16.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.610+0000 7fcf46ffd640 1 -- 172.21.15.1:0/1476188121 <== mgr.15204 v2:172.21.15.1:6800/3543150772 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258070 (secure 0 0 0) 0x7fcf50062630 con 0x7fcf3c0a2d00 2024-09-11T21:57:16.613 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:57:16.614 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-11T21:57:16.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.613+0000 7fcf57f9f640 1 -- 172.21.15.1:0/1476188121 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fcf3c0a2d00 msgr2=0x7fcf3c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:16.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.613+0000 7fcf57f9f640 1 --2- 172.21.15.1:0/1476188121 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fcf3c0a2d00 0x7fcf3c0a51c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c005fd0 tx=0x7fcf4c009500 comp rx=0 tx=0).stop 2024-09-11T21:57:16.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.613+0000 7fcf57f9f640 1 -- 172.21.15.1:0/1476188121 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcf50107830 msgr2=0x7fcf50100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:16.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.613+0000 7fcf57f9f640 1 --2- 172.21.15.1:0/1476188121 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcf50107830 0x7fcf50100ac0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fcf4800e970 tx=0x7fcf4800ee30 comp rx=0 tx=0).stop 2024-09-11T21:57:16.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.614+0000 7fcf57f9f640 1 -- 172.21.15.1:0/1476188121 shutdown_connections 2024-09-11T21:57:16.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.614+0000 7fcf57f9f640 1 --2- 172.21.15.1:0/1476188121 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fcf3c0a2d00 0x7fcf3c0a51c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:16.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.614+0000 7fcf57f9f640 1 --2- 172.21.15.1:0/1476188121 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcf50107830 0x7fcf50100ac0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:16.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.614+0000 7fcf57f9f640 1 --2- 172.21.15.1:0/1476188121 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf50104f00 0x7fcf50100580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:16.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.614+0000 7fcf57f9f640 1 --2- 172.21.15.1:0/1476188121 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf501025d0 0x7fcf50101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:16.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.614+0000 7fcf57f9f640 1 -- 172.21.15.1:0/1476188121 >> 172.21.15.1:0/1476188121 conn(0x7fcf500fc190 msgr2=0x7fcf50103130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:16.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.614+0000 7fcf57f9f640 1 -- 172.21.15.1:0/1476188121 shutdown_connections 2024-09-11T21:57:16.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.614+0000 7fcf57f9f640 1 -- 172.21.15.1:0/1476188121 wait complete. 2024-09-11T21:57:16.626 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":4,"stamp":"2024-09-11T21:57:15.682853+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1191642,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1302,"num_read_kb":820,"num_write":563,"num_write_kb":4410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1239947,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":945,"ondisk_log_size":945,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":390172,"kb_used_data":11772,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124551652,"statfs":{"total":1151940427776,"available":1151540891648,"internally_reserved":0,"allocated":12054528,"data_stored":8695560,"data_compressed":275530,"data_compressed_allocated":364544,"data_compressed_original":3268608,"omap_allocated":19142,"internal_metadata":387233082},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.990279"},"pg_stats":[{"pgid":"3.3f","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788313+0000","last_change":"2024-09-11T21:55:20.095385+0000","last_active":"2024-09-11T21:57:13.788313+0000","last_peered":"2024-09-11T21:57:13.788313+0000","last_clean":"2024-09-11T21:57:13.788313+0000","last_became_active":"2024-09-11T21:55:20.095077+0000","last_became_peered":"2024-09-11T21:55:20.095077+0000","last_unstale":"2024-09-11T21:57:13.788313+0000","last_undegraded":"2024-09-11T21:57:13.788313+0000","last_fullsized":"2024-09-11T21:57:13.788313+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:53:52.231522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"86'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.669584+0000","last_change":"2024-09-11T21:55:20.103540+0000","last_active":"2024-09-11T21:57:13.669584+0000","last_peered":"2024-09-11T21:57:13.669584+0000","last_clean":"2024-09-11T21:57:13.669584+0000","last_became_active":"2024-09-11T21:55:20.103233+0000","last_became_peered":"2024-09-11T21:55:20.103233+0000","last_unstale":"2024-09-11T21:57:13.669584+0000","last_undegraded":"2024-09-11T21:57:13.669584+0000","last_fullsized":"2024-09-11T21:57:13.669584+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:51:29.222202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.669576+0000","last_change":"2024-09-11T21:55:20.098538+0000","last_active":"2024-09-11T21:57:13.669576+0000","last_peered":"2024-09-11T21:57:13.669576+0000","last_clean":"2024-09-11T21:57:13.669576+0000","last_became_active":"2024-09-11T21:55:20.098202+0000","last_became_peered":"2024-09-11T21:55:20.098202+0000","last_unstale":"2024-09-11T21:57:13.669576+0000","last_undegraded":"2024-09-11T21:57:13.669576+0000","last_fullsized":"2024-09-11T21:57:13.669576+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:33:46.636952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788491+0000","last_change":"2024-09-11T21:55:20.100869+0000","last_active":"2024-09-11T21:57:13.788491+0000","last_peered":"2024-09-11T21:57:13.788491+0000","last_clean":"2024-09-11T21:57:13.788491+0000","last_became_active":"2024-09-11T21:55:20.100573+0000","last_became_peered":"2024-09-11T21:55:20.100573+0000","last_unstale":"2024-09-11T21:57:13.788491+0000","last_undegraded":"2024-09-11T21:57:13.788491+0000","last_fullsized":"2024-09-11T21:57:13.788491+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:58:11.154105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"86'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671033+0000","last_change":"2024-09-11T21:55:20.091648+0000","last_active":"2024-09-11T21:57:13.671033+0000","last_peered":"2024-09-11T21:57:13.671033+0000","last_clean":"2024-09-11T21:57:13.671033+0000","last_became_active":"2024-09-11T21:55:20.091222+0000","last_became_peered":"2024-09-11T21:55:20.091222+0000","last_unstale":"2024-09-11T21:57:13.671033+0000","last_undegraded":"2024-09-11T21:57:13.671033+0000","last_fullsized":"2024-09-11T21:57:13.671033+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:55:03.420754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"86'6","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.764975+0000","last_change":"2024-09-11T21:56:26.376595+0000","last_active":"2024-09-11T21:57:13.764975+0000","last_peered":"2024-09-11T21:57:13.764975+0000","last_clean":"2024-09-11T21:57:13.764975+0000","last_became_active":"2024-09-11T21:56:26.375784+0000","last_became_peered":"2024-09-11T21:56:26.375784+0000","last_unstale":"2024-09-11T21:57:13.764975+0000","last_undegraded":"2024-09-11T21:57:13.764975+0000","last_fullsized":"2024-09-11T21:57:13.764975+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:19:38.371463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,7],"acting":[10,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"86'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.670787+0000","last_change":"2024-09-11T21:55:20.101084+0000","last_active":"2024-09-11T21:57:13.670787+0000","last_peered":"2024-09-11T21:57:13.670787+0000","last_clean":"2024-09-11T21:57:13.670787+0000","last_became_active":"2024-09-11T21:55:20.100739+0000","last_became_peered":"2024-09-11T21:55:20.100739+0000","last_unstale":"2024-09-11T21:57:13.670787+0000","last_undegraded":"2024-09-11T21:57:13.670787+0000","last_fullsized":"2024-09-11T21:57:13.670787+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:46:58.969559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.753160+0000","last_change":"2024-09-11T21:55:20.088715+0000","last_active":"2024-09-11T21:57:13.753160+0000","last_peered":"2024-09-11T21:57:13.753160+0000","last_clean":"2024-09-11T21:57:13.753160+0000","last_became_active":"2024-09-11T21:55:20.088280+0000","last_became_peered":"2024-09-11T21:55:20.088280+0000","last_unstale":"2024-09-11T21:57:13.753160+0000","last_undegraded":"2024-09-11T21:57:13.753160+0000","last_fullsized":"2024-09-11T21:57:13.753160+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:06:24.450444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"86'11","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788671+0000","last_change":"2024-09-11T21:55:20.101266+0000","last_active":"2024-09-11T21:57:13.788671+0000","last_peered":"2024-09-11T21:57:13.788671+0000","last_clean":"2024-09-11T21:57:13.788671+0000","last_became_active":"2024-09-11T21:55:20.100919+0000","last_became_peered":"2024-09-11T21:55:20.100919+0000","last_unstale":"2024-09-11T21:57:13.788671+0000","last_undegraded":"2024-09-11T21:57:13.788671+0000","last_fullsized":"2024-09-11T21:57:13.788671+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:47:15.466082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"86'16","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.765361+0000","last_change":"2024-09-11T21:55:20.098086+0000","last_active":"2024-09-11T21:57:13.765361+0000","last_peered":"2024-09-11T21:57:13.765361+0000","last_clean":"2024-09-11T21:57:13.765361+0000","last_became_active":"2024-09-11T21:55:20.097754+0000","last_became_peered":"2024-09-11T21:55:20.097754+0000","last_unstale":"2024-09-11T21:57:13.765361+0000","last_undegraded":"2024-09-11T21:57:13.765361+0000","last_fullsized":"2024-09-11T21:57:13.765361+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:37:00.098345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"86'6","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671036+0000","last_change":"2024-09-11T21:55:20.092671+0000","last_active":"2024-09-11T21:57:13.671036+0000","last_peered":"2024-09-11T21:57:13.671036+0000","last_clean":"2024-09-11T21:57:13.671036+0000","last_became_active":"2024-09-11T21:55:20.092112+0000","last_became_peered":"2024-09-11T21:55:20.092112+0000","last_unstale":"2024-09-11T21:57:13.671036+0000","last_undegraded":"2024-09-11T21:57:13.671036+0000","last_fullsized":"2024-09-11T21:57:13.671036+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:06:28.538838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788646+0000","last_change":"2024-09-11T21:55:20.099863+0000","last_active":"2024-09-11T21:57:13.788646+0000","last_peered":"2024-09-11T21:57:13.788646+0000","last_clean":"2024-09-11T21:57:13.788646+0000","last_became_active":"2024-09-11T21:55:20.099173+0000","last_became_peered":"2024-09-11T21:55:20.099173+0000","last_unstale":"2024-09-11T21:57:13.788646+0000","last_undegraded":"2024-09-11T21:57:13.788646+0000","last_fullsized":"2024-09-11T21:57:13.788646+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:34:56.669410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"86'10","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788507+0000","last_change":"2024-09-11T21:55:20.101212+0000","last_active":"2024-09-11T21:57:13.788507+0000","last_peered":"2024-09-11T21:57:13.788507+0000","last_clean":"2024-09-11T21:57:13.788507+0000","last_became_active":"2024-09-11T21:55:20.100876+0000","last_became_peered":"2024-09-11T21:55:20.100876+0000","last_unstale":"2024-09-11T21:57:13.788507+0000","last_undegraded":"2024-09-11T21:57:13.788507+0000","last_fullsized":"2024-09-11T21:57:13.788507+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:14:18.104009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.669316+0000","last_change":"2024-09-11T21:55:20.095394+0000","last_active":"2024-09-11T21:57:13.669316+0000","last_peered":"2024-09-11T21:57:13.669316+0000","last_clean":"2024-09-11T21:57:13.669316+0000","last_became_active":"2024-09-11T21:55:20.095101+0000","last_became_peered":"2024-09-11T21:55:20.095101+0000","last_unstale":"2024-09-11T21:57:13.669316+0000","last_undegraded":"2024-09-11T21:57:13.669316+0000","last_fullsized":"2024-09-11T21:57:13.669316+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:53:32.529359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"86'7","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.669266+0000","last_change":"2024-09-11T21:56:26.376206+0000","last_active":"2024-09-11T21:57:13.669266+0000","last_peered":"2024-09-11T21:57:13.669266+0000","last_clean":"2024-09-11T21:57:13.669266+0000","last_became_active":"2024-09-11T21:56:26.375829+0000","last_became_peered":"2024-09-11T21:56:26.375829+0000","last_unstale":"2024-09-11T21:57:13.669266+0000","last_undegraded":"2024-09-11T21:57:13.669266+0000","last_fullsized":"2024-09-11T21:57:13.669266+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:21:59.484414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.787934+0000","last_change":"2024-09-11T21:55:20.093019+0000","last_active":"2024-09-11T21:57:13.787934+0000","last_peered":"2024-09-11T21:57:13.787934+0000","last_clean":"2024-09-11T21:57:13.787934+0000","last_became_active":"2024-09-11T21:55:20.092691+0000","last_became_peered":"2024-09-11T21:55:20.092691+0000","last_unstale":"2024-09-11T21:57:13.787934+0000","last_undegraded":"2024-09-11T21:57:13.787934+0000","last_fullsized":"2024-09-11T21:57:13.787934+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:04:36.684992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788379+0000","last_change":"2024-09-11T21:55:20.097333+0000","last_active":"2024-09-11T21:57:13.788379+0000","last_peered":"2024-09-11T21:57:13.788379+0000","last_clean":"2024-09-11T21:57:13.788379+0000","last_became_active":"2024-09-11T21:55:20.096752+0000","last_became_peered":"2024-09-11T21:55:20.096752+0000","last_unstale":"2024-09-11T21:57:13.788379+0000","last_undegraded":"2024-09-11T21:57:13.788379+0000","last_fullsized":"2024-09-11T21:57:13.788379+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:27:20.722174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"86'10","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.751322+0000","last_change":"2024-09-11T21:55:20.093972+0000","last_active":"2024-09-11T21:57:13.751322+0000","last_peered":"2024-09-11T21:57:13.751322+0000","last_clean":"2024-09-11T21:57:13.751322+0000","last_became_active":"2024-09-11T21:55:20.093633+0000","last_became_peered":"2024-09-11T21:55:20.093633+0000","last_unstale":"2024-09-11T21:57:13.751322+0000","last_undegraded":"2024-09-11T21:57:13.751322+0000","last_fullsized":"2024-09-11T21:57:13.751322+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:25:18.973332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671334+0000","last_change":"2024-09-11T21:55:20.094239+0000","last_active":"2024-09-11T21:57:13.671334+0000","last_peered":"2024-09-11T21:57:13.671334+0000","last_clean":"2024-09-11T21:57:13.671334+0000","last_became_active":"2024-09-11T21:55:20.093924+0000","last_became_peered":"2024-09-11T21:55:20.093924+0000","last_unstale":"2024-09-11T21:57:13.671334+0000","last_undegraded":"2024-09-11T21:57:13.671334+0000","last_fullsized":"2024-09-11T21:57:13.671334+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:39:53.012780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788075+0000","last_change":"2024-09-11T21:55:20.099335+0000","last_active":"2024-09-11T21:57:13.788075+0000","last_peered":"2024-09-11T21:57:13.788075+0000","last_clean":"2024-09-11T21:57:13.788075+0000","last_became_active":"2024-09-11T21:55:20.098929+0000","last_became_peered":"2024-09-11T21:55:20.098929+0000","last_unstale":"2024-09-11T21:57:13.788075+0000","last_undegraded":"2024-09-11T21:57:13.788075+0000","last_fullsized":"2024-09-11T21:57:13.788075+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:38:43.117829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"86'11","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788526+0000","last_change":"2024-09-11T21:55:20.094068+0000","last_active":"2024-09-11T21:57:13.788526+0000","last_peered":"2024-09-11T21:57:13.788526+0000","last_clean":"2024-09-11T21:57:13.788526+0000","last_became_active":"2024-09-11T21:55:20.093637+0000","last_became_peered":"2024-09-11T21:55:20.093637+0000","last_unstale":"2024-09-11T21:57:13.788526+0000","last_undegraded":"2024-09-11T21:57:13.788526+0000","last_fullsized":"2024-09-11T21:57:13.788526+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:52:59.289715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"86'10","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788122+0000","last_change":"2024-09-11T21:55:20.090778+0000","last_active":"2024-09-11T21:57:13.788122+0000","last_peered":"2024-09-11T21:57:13.788122+0000","last_clean":"2024-09-11T21:57:13.788122+0000","last_became_active":"2024-09-11T21:55:20.090443+0000","last_became_peered":"2024-09-11T21:55:20.090443+0000","last_unstale":"2024-09-11T21:57:13.788122+0000","last_undegraded":"2024-09-11T21:57:13.788122+0000","last_fullsized":"2024-09-11T21:57:13.788122+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:46:21.488228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.753203+0000","last_change":"2024-09-11T21:55:20.092238+0000","last_active":"2024-09-11T21:57:13.753203+0000","last_peered":"2024-09-11T21:57:13.753203+0000","last_clean":"2024-09-11T21:57:13.753203+0000","last_became_active":"2024-09-11T21:55:20.091961+0000","last_became_peered":"2024-09-11T21:55:20.091961+0000","last_unstale":"2024-09-11T21:57:13.753203+0000","last_undegraded":"2024-09-11T21:57:13.753203+0000","last_fullsized":"2024-09-11T21:57:13.753203+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:54:40.445441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"86'6","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.761984+0000","last_change":"2024-09-11T21:55:20.102944+0000","last_active":"2024-09-11T21:57:13.761984+0000","last_peered":"2024-09-11T21:57:13.761984+0000","last_clean":"2024-09-11T21:57:13.761984+0000","last_became_active":"2024-09-11T21:55:20.102533+0000","last_became_peered":"2024-09-11T21:55:20.102533+0000","last_unstale":"2024-09-11T21:57:13.761984+0000","last_undegraded":"2024-09-11T21:57:13.761984+0000","last_fullsized":"2024-09-11T21:57:13.761984+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:12:15.832622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"86'2","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.753729+0000","last_change":"2024-09-11T21:56:26.292272+0000","last_active":"2024-09-11T21:57:13.753729+0000","last_peered":"2024-09-11T21:57:13.753729+0000","last_clean":"2024-09-11T21:57:13.753729+0000","last_became_active":"2024-09-11T21:56:26.291920+0000","last_became_peered":"2024-09-11T21:56:26.291920+0000","last_unstale":"2024-09-11T21:57:13.753729+0000","last_undegraded":"2024-09-11T21:57:13.753729+0000","last_fullsized":"2024-09-11T21:57:13.753729+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:13:13.579907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"86'15","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671489+0000","last_change":"2024-09-11T21:56:26.376364+0000","last_active":"2024-09-11T21:57:13.671489+0000","last_peered":"2024-09-11T21:57:13.671489+0000","last_clean":"2024-09-11T21:57:13.671489+0000","last_became_active":"2024-09-11T21:56:26.375927+0000","last_became_peered":"2024-09-11T21:56:26.375927+0000","last_unstale":"2024-09-11T21:57:13.671489+0000","last_undegraded":"2024-09-11T21:57:13.671489+0000","last_fullsized":"2024-09-11T21:57:13.671489+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:23:03.664207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,9],"acting":[2,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"86'6","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788153+0000","last_change":"2024-09-11T21:56:26.376702+0000","last_active":"2024-09-11T21:57:13.788153+0000","last_peered":"2024-09-11T21:57:13.788153+0000","last_clean":"2024-09-11T21:57:13.788153+0000","last_became_active":"2024-09-11T21:56:26.376179+0000","last_became_peered":"2024-09-11T21:56:26.376179+0000","last_unstale":"2024-09-11T21:57:13.788153+0000","last_undegraded":"2024-09-11T21:57:13.788153+0000","last_fullsized":"2024-09-11T21:57:13.788153+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:27:34.666962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"86'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788960+0000","last_change":"2024-09-11T21:55:20.093898+0000","last_active":"2024-09-11T21:57:13.788960+0000","last_peered":"2024-09-11T21:57:13.788960+0000","last_clean":"2024-09-11T21:57:13.788960+0000","last_became_active":"2024-09-11T21:55:20.093348+0000","last_became_peered":"2024-09-11T21:55:20.093348+0000","last_unstale":"2024-09-11T21:57:13.788960+0000","last_undegraded":"2024-09-11T21:57:13.788960+0000","last_fullsized":"2024-09-11T21:57:13.788960+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:00:06.185478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.669457+0000","last_change":"2024-09-11T21:55:20.100575+0000","last_active":"2024-09-11T21:57:13.669457+0000","last_peered":"2024-09-11T21:57:13.669457+0000","last_clean":"2024-09-11T21:57:13.669457+0000","last_became_active":"2024-09-11T21:55:20.100255+0000","last_became_peered":"2024-09-11T21:55:20.100255+0000","last_unstale":"2024-09-11T21:57:13.669457+0000","last_undegraded":"2024-09-11T21:57:13.669457+0000","last_fullsized":"2024-09-11T21:57:13.669457+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:48:57.718298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"86'8","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.670722+0000","last_change":"2024-09-11T21:56:26.377183+0000","last_active":"2024-09-11T21:57:13.670722+0000","last_peered":"2024-09-11T21:57:13.670722+0000","last_clean":"2024-09-11T21:57:13.670722+0000","last_became_active":"2024-09-11T21:56:26.375906+0000","last_became_peered":"2024-09-11T21:56:26.375906+0000","last_unstale":"2024-09-11T21:57:13.670722+0000","last_undegraded":"2024-09-11T21:57:13.670722+0000","last_fullsized":"2024-09-11T21:57:13.670722+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:00:29.033961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"86'12","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.670784+0000","last_change":"2024-09-11T21:55:20.093822+0000","last_active":"2024-09-11T21:57:13.670784+0000","last_peered":"2024-09-11T21:57:13.670784+0000","last_clean":"2024-09-11T21:57:13.670784+0000","last_became_active":"2024-09-11T21:55:20.093361+0000","last_became_peered":"2024-09-11T21:55:20.093361+0000","last_unstale":"2024-09-11T21:57:13.670784+0000","last_undegraded":"2024-09-11T21:57:13.670784+0000","last_fullsized":"2024-09-11T21:57:13.670784+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:37:06.362732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.787998+0000","last_change":"2024-09-11T21:55:20.103990+0000","last_active":"2024-09-11T21:57:13.787998+0000","last_peered":"2024-09-11T21:57:13.787998+0000","last_clean":"2024-09-11T21:57:13.787998+0000","last_became_active":"2024-09-11T21:55:20.103375+0000","last_became_peered":"2024-09-11T21:55:20.103375+0000","last_unstale":"2024-09-11T21:57:13.787998+0000","last_undegraded":"2024-09-11T21:57:13.787998+0000","last_fullsized":"2024-09-11T21:57:13.787998+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:02:06.786888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"86'7","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671024+0000","last_change":"2024-09-11T21:56:26.376261+0000","last_active":"2024-09-11T21:57:13.671024+0000","last_peered":"2024-09-11T21:57:13.671024+0000","last_clean":"2024-09-11T21:57:13.671024+0000","last_became_active":"2024-09-11T21:56:26.375863+0000","last_became_peered":"2024-09-11T21:56:26.375863+0000","last_unstale":"2024-09-11T21:57:13.671024+0000","last_undegraded":"2024-09-11T21:57:13.671024+0000","last_fullsized":"2024-09-11T21:57:13.671024+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:58:21.359836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"87'8","reported_seq":140,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:14.299174+0000","last_change":"2024-09-11T21:55:18.026947+0000","last_active":"2024-09-11T21:57:14.299174+0000","last_peered":"2024-09-11T21:57:14.299174+0000","last_clean":"2024-09-11T21:57:14.299174+0000","last_became_active":"2024-09-11T21:55:18.026632+0000","last_became_peered":"2024-09-11T21:55:18.026632+0000","last_unstale":"2024-09-11T21:57:14.299174+0000","last_undegraded":"2024-09-11T21:57:14.299174+0000","last_fullsized":"2024-09-11T21:57:14.299174+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:55:16.618074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1549,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":108,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.762157+0000","last_change":"2024-09-11T21:55:20.100197+0000","last_active":"2024-09-11T21:57:13.762157+0000","last_peered":"2024-09-11T21:57:13.762157+0000","last_clean":"2024-09-11T21:57:13.762157+0000","last_became_active":"2024-09-11T21:55:20.098494+0000","last_became_peered":"2024-09-11T21:55:20.098494+0000","last_unstale":"2024-09-11T21:57:13.762157+0000","last_undegraded":"2024-09-11T21:57:13.762157+0000","last_fullsized":"2024-09-11T21:57:13.762157+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:14.505314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"84'7","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.765023+0000","last_change":"2024-09-11T21:55:18.024359+0000","last_active":"2024-09-11T21:57:13.765023+0000","last_peered":"2024-09-11T21:57:13.765023+0000","last_clean":"2024-09-11T21:57:13.765023+0000","last_became_active":"2024-09-11T21:55:18.023624+0000","last_became_peered":"2024-09-11T21:55:18.023624+0000","last_unstale":"2024-09-11T21:57:13.765023+0000","last_undegraded":"2024-09-11T21:57:13.765023+0000","last_fullsized":"2024-09-11T21:57:13.765023+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:13:38.582232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"84'3","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788356+0000","last_change":"2024-09-11T21:55:18.032541+0000","last_active":"2024-09-11T21:57:13.788356+0000","last_peered":"2024-09-11T21:57:13.788356+0000","last_clean":"2024-09-11T21:57:13.788356+0000","last_became_active":"2024-09-11T21:55:18.032218+0000","last_became_peered":"2024-09-11T21:55:18.032218+0000","last_unstale":"2024-09-11T21:57:13.788356+0000","last_undegraded":"2024-09-11T21:57:13.788356+0000","last_fullsized":"2024-09-11T21:57:13.788356+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:38:20.872069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.753229+0000","last_change":"2024-09-11T21:55:20.090030+0000","last_active":"2024-09-11T21:57:13.753229+0000","last_peered":"2024-09-11T21:57:13.753229+0000","last_clean":"2024-09-11T21:57:13.753229+0000","last_became_active":"2024-09-11T21:55:20.089481+0000","last_became_peered":"2024-09-11T21:55:20.089481+0000","last_unstale":"2024-09-11T21:57:13.753229+0000","last_undegraded":"2024-09-11T21:57:13.753229+0000","last_fullsized":"2024-09-11T21:57:13.753229+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:41:57.038519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"84'3","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.762394+0000","last_change":"2024-09-11T21:55:18.026656+0000","last_active":"2024-09-11T21:57:13.762394+0000","last_peered":"2024-09-11T21:57:13.762394+0000","last_clean":"2024-09-11T21:57:13.762394+0000","last_became_active":"2024-09-11T21:55:18.026318+0000","last_became_peered":"2024-09-11T21:55:18.026318+0000","last_unstale":"2024-09-11T21:57:13.762394+0000","last_undegraded":"2024-09-11T21:57:13.762394+0000","last_fullsized":"2024-09-11T21:57:13.762394+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:14:41.438334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"86'11","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.750970+0000","last_change":"2024-09-11T21:55:20.090790+0000","last_active":"2024-09-11T21:57:13.750970+0000","last_peered":"2024-09-11T21:57:13.750970+0000","last_clean":"2024-09-11T21:57:13.750970+0000","last_became_active":"2024-09-11T21:55:20.090468+0000","last_became_peered":"2024-09-11T21:55:20.090468+0000","last_unstale":"2024-09-11T21:57:13.750970+0000","last_undegraded":"2024-09-11T21:57:13.750970+0000","last_fullsized":"2024-09-11T21:57:13.750970+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:19.115482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'8","reported_seq":238,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:14.298769+0000","last_change":"2024-09-11T21:55:18.026989+0000","last_active":"2024-09-11T21:57:14.298769+0000","last_peered":"2024-09-11T21:57:14.298769+0000","last_clean":"2024-09-11T21:57:14.298769+0000","last_became_active":"2024-09-11T21:55:18.026229+0000","last_became_peered":"2024-09-11T21:55:18.026229+0000","last_unstale":"2024-09-11T21:57:14.298769+0000","last_undegraded":"2024-09-11T21:57:14.298769+0000","last_fullsized":"2024-09-11T21:57:14.298769+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:50:35.745789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":206,"num_read_kb":206,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"86'10","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788388+0000","last_change":"2024-09-11T21:55:20.095884+0000","last_active":"2024-09-11T21:57:13.788388+0000","last_peered":"2024-09-11T21:57:13.788388+0000","last_clean":"2024-09-11T21:57:13.788388+0000","last_became_active":"2024-09-11T21:55:20.094887+0000","last_became_peered":"2024-09-11T21:55:20.094887+0000","last_unstale":"2024-09-11T21:57:13.788388+0000","last_undegraded":"2024-09-11T21:57:13.788388+0000","last_fullsized":"2024-09-11T21:57:13.788388+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:45:03.579703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"84'4","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671167+0000","last_change":"2024-09-11T21:55:18.030474+0000","last_active":"2024-09-11T21:57:13.671167+0000","last_peered":"2024-09-11T21:57:13.671167+0000","last_clean":"2024-09-11T21:57:13.671167+0000","last_became_active":"2024-09-11T21:55:18.030104+0000","last_became_peered":"2024-09-11T21:55:18.030104+0000","last_unstale":"2024-09-11T21:57:13.671167+0000","last_undegraded":"2024-09-11T21:57:13.671167+0000","last_fullsized":"2024-09-11T21:57:13.671167+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:21:00.484595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.669795+0000","last_change":"2024-09-11T21:55:20.102807+0000","last_active":"2024-09-11T21:57:13.669795+0000","last_peered":"2024-09-11T21:57:13.669795+0000","last_clean":"2024-09-11T21:57:13.669795+0000","last_became_active":"2024-09-11T21:55:20.102581+0000","last_became_peered":"2024-09-11T21:55:20.102581+0000","last_unstale":"2024-09-11T21:57:13.669795+0000","last_undegraded":"2024-09-11T21:57:13.669795+0000","last_fullsized":"2024-09-11T21:57:13.669795+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:29:23.139788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"84'5","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.762200+0000","last_change":"2024-09-11T21:55:18.117685+0000","last_active":"2024-09-11T21:57:13.762200+0000","last_peered":"2024-09-11T21:57:13.762200+0000","last_clean":"2024-09-11T21:57:13.762200+0000","last_became_active":"2024-09-11T21:55:18.117261+0000","last_became_peered":"2024-09-11T21:55:18.117261+0000","last_unstale":"2024-09-11T21:57:13.762200+0000","last_undegraded":"2024-09-11T21:57:13.762200+0000","last_fullsized":"2024-09-11T21:57:13.762200+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:42:13.180725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"86'11","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788081+0000","last_change":"2024-09-11T21:55:20.086562+0000","last_active":"2024-09-11T21:57:13.788081+0000","last_peered":"2024-09-11T21:57:13.788081+0000","last_clean":"2024-09-11T21:57:13.788081+0000","last_became_active":"2024-09-11T21:55:20.086197+0000","last_became_peered":"2024-09-11T21:55:20.086197+0000","last_unstale":"2024-09-11T21:57:13.788081+0000","last_undegraded":"2024-09-11T21:57:13.788081+0000","last_fullsized":"2024-09-11T21:57:13.788081+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:23:44.705132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'35","reported_seq":296,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:14.358002+0000","last_change":"2024-09-11T21:56:26.299386+0000","last_active":"2024-09-11T21:57:14.358002+0000","last_peered":"2024-09-11T21:57:14.358002+0000","last_clean":"2024-09-11T21:57:14.358002+0000","last_became_active":"2024-09-11T21:56:26.291951+0000","last_became_peered":"2024-09-11T21:56:26.291951+0000","last_unstale":"2024-09-11T21:57:14.358002+0000","last_undegraded":"2024-09-11T21:57:14.358002+0000","last_fullsized":"2024-09-11T21:57:14.358002+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:54:47.346798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65927,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":209,"num_read_kb":169,"num_write":37,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":59211,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"86'6","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671198+0000","last_change":"2024-09-11T21:55:20.091695+0000","last_active":"2024-09-11T21:57:13.671198+0000","last_peered":"2024-09-11T21:57:13.671198+0000","last_clean":"2024-09-11T21:57:13.671198+0000","last_became_active":"2024-09-11T21:55:20.091128+0000","last_became_peered":"2024-09-11T21:55:20.091128+0000","last_unstale":"2024-09-11T21:57:13.671198+0000","last_undegraded":"2024-09-11T21:57:13.671198+0000","last_fullsized":"2024-09-11T21:57:13.671198+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:16:20.751952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"84'9","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671024+0000","last_change":"2024-09-11T21:55:18.030827+0000","last_active":"2024-09-11T21:57:13.671024+0000","last_peered":"2024-09-11T21:57:13.671024+0000","last_clean":"2024-09-11T21:57:13.671024+0000","last_became_active":"2024-09-11T21:55:18.030354+0000","last_became_peered":"2024-09-11T21:55:18.030354+0000","last_unstale":"2024-09-11T21:57:13.671024+0000","last_undegraded":"2024-09-11T21:57:13.671024+0000","last_fullsized":"2024-09-11T21:57:13.671024+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:23:19.578820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788550+0000","last_change":"2024-09-11T21:55:20.090160+0000","last_active":"2024-09-11T21:57:13.788550+0000","last_peered":"2024-09-11T21:57:13.788550+0000","last_clean":"2024-09-11T21:57:13.788550+0000","last_became_active":"2024-09-11T21:55:20.089801+0000","last_became_peered":"2024-09-11T21:55:20.089801+0000","last_unstale":"2024-09-11T21:57:13.788550+0000","last_undegraded":"2024-09-11T21:57:13.788550+0000","last_fullsized":"2024-09-11T21:57:13.788550+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:34:16.093123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"84'5","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.761998+0000","last_change":"2024-09-11T21:55:18.026817+0000","last_active":"2024-09-11T21:57:13.761998+0000","last_peered":"2024-09-11T21:57:13.761998+0000","last_clean":"2024-09-11T21:57:13.761998+0000","last_became_active":"2024-09-11T21:55:18.026188+0000","last_became_peered":"2024-09-11T21:55:18.026188+0000","last_unstale":"2024-09-11T21:57:13.761998+0000","last_undegraded":"2024-09-11T21:57:13.761998+0000","last_fullsized":"2024-09-11T21:57:13.761998+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:55:14.774503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"86'15","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.670991+0000","last_change":"2024-09-11T21:55:20.095613+0000","last_active":"2024-09-11T21:57:13.670991+0000","last_peered":"2024-09-11T21:57:13.670991+0000","last_clean":"2024-09-11T21:57:13.670991+0000","last_became_active":"2024-09-11T21:55:20.095302+0000","last_became_peered":"2024-09-11T21:55:20.095302+0000","last_unstale":"2024-09-11T21:57:13.670991+0000","last_undegraded":"2024-09-11T21:57:13.670991+0000","last_fullsized":"2024-09-11T21:57:13.670991+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:37:27.961213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'7","reported_seq":138,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:14.300345+0000","last_change":"2024-09-11T21:55:18.032417+0000","last_active":"2024-09-11T21:57:14.300345+0000","last_peered":"2024-09-11T21:57:14.300345+0000","last_clean":"2024-09-11T21:57:14.300345+0000","last_became_active":"2024-09-11T21:55:18.031419+0000","last_became_peered":"2024-09-11T21:55:18.031419+0000","last_unstale":"2024-09-11T21:57:14.300345+0000","last_undegraded":"2024-09-11T21:57:14.300345+0000","last_fullsized":"2024-09-11T21:57:14.300345+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:30:52.285678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1289,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":108,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"86'6","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.670730+0000","last_change":"2024-09-11T21:55:20.101965+0000","last_active":"2024-09-11T21:57:13.670730+0000","last_peered":"2024-09-11T21:57:13.670730+0000","last_clean":"2024-09-11T21:57:13.670730+0000","last_became_active":"2024-09-11T21:55:20.101534+0000","last_became_peered":"2024-09-11T21:55:20.101534+0000","last_unstale":"2024-09-11T21:57:13.670730+0000","last_undegraded":"2024-09-11T21:57:13.670730+0000","last_fullsized":"2024-09-11T21:57:13.670730+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:41:30.685121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"84'17","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.670811+0000","last_change":"2024-09-11T21:55:18.116059+0000","last_active":"2024-09-11T21:57:13.670811+0000","last_peered":"2024-09-11T21:57:13.670811+0000","last_clean":"2024-09-11T21:57:13.670811+0000","last_became_active":"2024-09-11T21:55:18.115621+0000","last_became_peered":"2024-09-11T21:55:18.115621+0000","last_unstale":"2024-09-11T21:57:13.670811+0000","last_undegraded":"2024-09-11T21:57:13.670811+0000","last_fullsized":"2024-09-11T21:57:13.670811+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:47:17.815695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"86'6","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.762037+0000","last_change":"2024-09-11T21:55:20.100903+0000","last_active":"2024-09-11T21:57:13.762037+0000","last_peered":"2024-09-11T21:57:13.762037+0000","last_clean":"2024-09-11T21:57:13.762037+0000","last_became_active":"2024-09-11T21:55:20.100561+0000","last_became_peered":"2024-09-11T21:55:20.100561+0000","last_unstale":"2024-09-11T21:57:13.762037+0000","last_undegraded":"2024-09-11T21:57:13.762037+0000","last_fullsized":"2024-09-11T21:57:13.762037+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:00:45.472238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'2","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671632+0000","last_change":"2024-09-11T21:55:18.029549+0000","last_active":"2024-09-11T21:57:13.671632+0000","last_peered":"2024-09-11T21:57:13.671632+0000","last_clean":"2024-09-11T21:57:13.671632+0000","last_became_active":"2024-09-11T21:55:18.029178+0000","last_became_peered":"2024-09-11T21:55:18.029178+0000","last_unstale":"2024-09-11T21:57:13.671632+0000","last_undegraded":"2024-09-11T21:57:13.671632+0000","last_fullsized":"2024-09-11T21:57:13.671632+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:37:07.915786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"86'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788144+0000","last_change":"2024-09-11T21:55:20.104202+0000","last_active":"2024-09-11T21:57:13.788144+0000","last_peered":"2024-09-11T21:57:13.788144+0000","last_clean":"2024-09-11T21:57:13.788144+0000","last_became_active":"2024-09-11T21:55:20.103464+0000","last_became_peered":"2024-09-11T21:55:20.103464+0000","last_unstale":"2024-09-11T21:57:13.788144+0000","last_undegraded":"2024-09-11T21:57:13.788144+0000","last_fullsized":"2024-09-11T21:57:13.788144+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:30:16.044143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"84'5","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788728+0000","last_change":"2024-09-11T21:55:18.026183+0000","last_active":"2024-09-11T21:57:13.788728+0000","last_peered":"2024-09-11T21:57:13.788728+0000","last_clean":"2024-09-11T21:57:13.788728+0000","last_became_active":"2024-09-11T21:55:18.025849+0000","last_became_peered":"2024-09-11T21:55:18.025849+0000","last_unstale":"2024-09-11T21:57:13.788728+0000","last_undegraded":"2024-09-11T21:57:13.788728+0000","last_fullsized":"2024-09-11T21:57:13.788728+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:21:12.189158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671103+0000","last_change":"2024-09-11T21:55:20.090652+0000","last_active":"2024-09-11T21:57:13.671103+0000","last_peered":"2024-09-11T21:57:13.671103+0000","last_clean":"2024-09-11T21:57:13.671103+0000","last_became_active":"2024-09-11T21:55:20.090348+0000","last_became_peered":"2024-09-11T21:55:20.090348+0000","last_unstale":"2024-09-11T21:57:13.671103+0000","last_undegraded":"2024-09-11T21:57:13.671103+0000","last_fullsized":"2024-09-11T21:57:13.671103+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:10:58.733371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'41","reported_seq":73,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.873064+0000","last_change":"2024-09-11T21:55:18.024388+0000","last_active":"2024-09-11T21:57:13.873064+0000","last_peered":"2024-09-11T21:57:13.873064+0000","last_clean":"2024-09-11T21:57:13.873064+0000","last_became_active":"2024-09-11T21:55:18.024100+0000","last_became_peered":"2024-09-11T21:55:18.024100+0000","last_unstale":"2024-09-11T21:57:13.873064+0000","last_undegraded":"2024-09-11T21:57:13.873064+0000","last_fullsized":"2024-09-11T21:57:13.873064+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:21:11.152282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.670936+0000","last_change":"2024-09-11T21:55:20.101007+0000","last_active":"2024-09-11T21:57:13.670936+0000","last_peered":"2024-09-11T21:57:13.670936+0000","last_clean":"2024-09-11T21:57:13.670936+0000","last_became_active":"2024-09-11T21:55:20.100642+0000","last_became_peered":"2024-09-11T21:55:20.100642+0000","last_unstale":"2024-09-11T21:57:13.670936+0000","last_undegraded":"2024-09-11T21:57:13.670936+0000","last_fullsized":"2024-09-11T21:57:13.670936+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:16:05.642184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"87'8","reported_seq":140,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:14.325956+0000","last_change":"2024-09-11T21:55:18.116216+0000","last_active":"2024-09-11T21:57:14.325956+0000","last_peered":"2024-09-11T21:57:14.325956+0000","last_clean":"2024-09-11T21:57:14.325956+0000","last_became_active":"2024-09-11T21:55:18.115871+0000","last_became_peered":"2024-09-11T21:55:18.115871+0000","last_unstale":"2024-09-11T21:57:14.325956+0000","last_undegraded":"2024-09-11T21:57:14.325956+0000","last_fullsized":"2024-09-11T21:57:14.325956+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:16:56.027636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":108,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671175+0000","last_change":"2024-09-11T21:55:20.092421+0000","last_active":"2024-09-11T21:57:13.671175+0000","last_peered":"2024-09-11T21:57:13.671175+0000","last_clean":"2024-09-11T21:57:13.671175+0000","last_became_active":"2024-09-11T21:55:20.091707+0000","last_became_peered":"2024-09-11T21:55:20.091707+0000","last_unstale":"2024-09-11T21:57:13.671175+0000","last_undegraded":"2024-09-11T21:57:13.671175+0000","last_fullsized":"2024-09-11T21:57:13.671175+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:12:38.157687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":353,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.670940+0000","last_change":"2024-09-11T21:54:07.542362+0000","last_active":"2024-09-11T21:57:13.670940+0000","last_peered":"2024-09-11T21:57:13.670940+0000","last_clean":"2024-09-11T21:57:13.670940+0000","last_became_active":"2024-09-11T21:54:07.541871+0000","last_became_peered":"2024-09-11T21:54:07.541871+0000","last_unstale":"2024-09-11T21:57:13.670940+0000","last_undegraded":"2024-09-11T21:57:13.670940+0000","last_fullsized":"2024-09-11T21:57:13.670940+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:50:41.026895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:50:41.026895+0000","last_clean_scrub_stamp":"2024-09-11T21:50:41.026895+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:05:06.154268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"84'6","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788499+0000","last_change":"2024-09-11T21:55:18.030558+0000","last_active":"2024-09-11T21:57:13.788499+0000","last_peered":"2024-09-11T21:57:13.788499+0000","last_clean":"2024-09-11T21:57:13.788499+0000","last_became_active":"2024-09-11T21:55:18.030216+0000","last_became_peered":"2024-09-11T21:55:18.030216+0000","last_unstale":"2024-09-11T21:57:13.788499+0000","last_undegraded":"2024-09-11T21:57:13.788499+0000","last_fullsized":"2024-09-11T21:57:13.788499+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:33:02.589999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.753426+0000","last_change":"2024-09-11T21:55:20.092938+0000","last_active":"2024-09-11T21:57:13.753426+0000","last_peered":"2024-09-11T21:57:13.753426+0000","last_clean":"2024-09-11T21:57:13.753426+0000","last_became_active":"2024-09-11T21:55:20.092615+0000","last_became_peered":"2024-09-11T21:55:20.092615+0000","last_unstale":"2024-09-11T21:57:13.753426+0000","last_undegraded":"2024-09-11T21:57:13.753426+0000","last_fullsized":"2024-09-11T21:57:13.753426+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:40:06.957836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"84'4","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.670838+0000","last_change":"2024-09-11T21:55:18.029127+0000","last_active":"2024-09-11T21:57:13.670838+0000","last_peered":"2024-09-11T21:57:13.670838+0000","last_clean":"2024-09-11T21:57:13.670838+0000","last_became_active":"2024-09-11T21:55:18.028740+0000","last_became_peered":"2024-09-11T21:55:18.028740+0000","last_unstale":"2024-09-11T21:57:13.670838+0000","last_undegraded":"2024-09-11T21:57:13.670838+0000","last_fullsized":"2024-09-11T21:57:13.670838+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:14:01.128486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"86'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788303+0000","last_change":"2024-09-11T21:55:20.092498+0000","last_active":"2024-09-11T21:57:13.788303+0000","last_peered":"2024-09-11T21:57:13.788303+0000","last_clean":"2024-09-11T21:57:13.788303+0000","last_became_active":"2024-09-11T21:55:20.092179+0000","last_became_peered":"2024-09-11T21:55:20.092179+0000","last_unstale":"2024-09-11T21:57:13.788303+0000","last_undegraded":"2024-09-11T21:57:13.788303+0000","last_fullsized":"2024-09-11T21:57:13.788303+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:48:58.536942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"86'6","reported_seq":134,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788554+0000","last_change":"2024-09-11T21:55:18.115798+0000","last_active":"2024-09-11T21:57:13.788554+0000","last_peered":"2024-09-11T21:57:13.788554+0000","last_clean":"2024-09-11T21:57:13.788554+0000","last_became_active":"2024-09-11T21:55:18.115433+0000","last_became_peered":"2024-09-11T21:55:18.115433+0000","last_unstale":"2024-09-11T21:57:13.788554+0000","last_undegraded":"2024-09-11T21:57:13.788554+0000","last_fullsized":"2024-09-11T21:57:13.788554+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:03:32.097756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":104,"num_read_kb":104,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.762043+0000","last_change":"2024-09-11T21:55:20.100439+0000","last_active":"2024-09-11T21:57:13.762043+0000","last_peered":"2024-09-11T21:57:13.762043+0000","last_clean":"2024-09-11T21:57:13.762043+0000","last_became_active":"2024-09-11T21:55:20.098717+0000","last_became_peered":"2024-09-11T21:55:20.098717+0000","last_unstale":"2024-09-11T21:57:13.762043+0000","last_undegraded":"2024-09-11T21:57:13.762043+0000","last_fullsized":"2024-09-11T21:57:13.762043+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:15:10.874225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"87'9","reported_seq":140,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:14.298986+0000","last_change":"2024-09-11T21:55:18.026588+0000","last_active":"2024-09-11T21:57:14.298986+0000","last_peered":"2024-09-11T21:57:14.298986+0000","last_clean":"2024-09-11T21:57:14.298986+0000","last_became_active":"2024-09-11T21:55:18.026205+0000","last_became_peered":"2024-09-11T21:55:18.026205+0000","last_unstale":"2024-09-11T21:57:14.298986+0000","last_undegraded":"2024-09-11T21:57:14.298986+0000","last_fullsized":"2024-09-11T21:57:14.298986+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:45:34.701088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1908,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":107,"num_read_kb":4,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.765120+0000","last_change":"2024-09-11T21:55:20.100677+0000","last_active":"2024-09-11T21:57:13.765120+0000","last_peered":"2024-09-11T21:57:13.765120+0000","last_clean":"2024-09-11T21:57:13.765120+0000","last_became_active":"2024-09-11T21:55:20.100334+0000","last_became_peered":"2024-09-11T21:55:20.100334+0000","last_unstale":"2024-09-11T21:57:13.765120+0000","last_undegraded":"2024-09-11T21:57:13.765120+0000","last_fullsized":"2024-09-11T21:57:13.765120+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:15:16.792831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"84'1","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.751199+0000","last_change":"2024-09-11T21:55:18.116329+0000","last_active":"2024-09-11T21:57:13.751199+0000","last_peered":"2024-09-11T21:57:13.751199+0000","last_clean":"2024-09-11T21:57:13.751199+0000","last_became_active":"2024-09-11T21:55:18.115944+0000","last_became_peered":"2024-09-11T21:55:18.115944+0000","last_unstale":"2024-09-11T21:57:13.751199+0000","last_undegraded":"2024-09-11T21:57:13.751199+0000","last_fullsized":"2024-09-11T21:57:13.751199+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:03:04.164985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.765132+0000","last_change":"2024-09-11T21:55:20.089202+0000","last_active":"2024-09-11T21:57:13.765132+0000","last_peered":"2024-09-11T21:57:13.765132+0000","last_clean":"2024-09-11T21:57:13.765132+0000","last_became_active":"2024-09-11T21:55:20.088853+0000","last_became_peered":"2024-09-11T21:55:20.088853+0000","last_unstale":"2024-09-11T21:57:13.765132+0000","last_undegraded":"2024-09-11T21:57:13.765132+0000","last_fullsized":"2024-09-11T21:57:13.765132+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:39:36.496944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"84'4","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671429+0000","last_change":"2024-09-11T21:55:18.117917+0000","last_active":"2024-09-11T21:57:13.671429+0000","last_peered":"2024-09-11T21:57:13.671429+0000","last_clean":"2024-09-11T21:57:13.671429+0000","last_became_active":"2024-09-11T21:55:18.117567+0000","last_became_peered":"2024-09-11T21:55:18.117567+0000","last_unstale":"2024-09-11T21:57:13.671429+0000","last_undegraded":"2024-09-11T21:57:13.671429+0000","last_fullsized":"2024-09-11T21:57:13.671429+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:21:32.546620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.789090+0000","last_change":"2024-09-11T21:55:20.092370+0000","last_active":"2024-09-11T21:57:13.789090+0000","last_peered":"2024-09-11T21:57:13.789090+0000","last_clean":"2024-09-11T21:57:13.789090+0000","last_became_active":"2024-09-11T21:55:20.091982+0000","last_became_peered":"2024-09-11T21:55:20.091982+0000","last_unstale":"2024-09-11T21:57:13.789090+0000","last_undegraded":"2024-09-11T21:57:13.789090+0000","last_fullsized":"2024-09-11T21:57:13.789090+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:59:55.214511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'51","reported_seq":98,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.879119+0000","last_change":"2024-09-11T21:55:18.023166+0000","last_active":"2024-09-11T21:57:13.879119+0000","last_peered":"2024-09-11T21:57:13.879119+0000","last_clean":"2024-09-11T21:57:13.879119+0000","last_became_active":"2024-09-11T21:55:18.022845+0000","last_became_peered":"2024-09-11T21:55:18.022845+0000","last_unstale":"2024-09-11T21:57:13.879119+0000","last_undegraded":"2024-09-11T21:57:13.879119+0000","last_fullsized":"2024-09-11T21:57:13.879119+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:08:06.770410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":91,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"86'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.753738+0000","last_change":"2024-09-11T21:55:20.091541+0000","last_active":"2024-09-11T21:57:13.753738+0000","last_peered":"2024-09-11T21:57:13.753738+0000","last_clean":"2024-09-11T21:57:13.753738+0000","last_became_active":"2024-09-11T21:55:20.090971+0000","last_became_peered":"2024-09-11T21:55:20.090971+0000","last_unstale":"2024-09-11T21:57:13.753738+0000","last_undegraded":"2024-09-11T21:57:13.753738+0000","last_fullsized":"2024-09-11T21:57:13.753738+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:22:31.974711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"84'7","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788398+0000","last_change":"2024-09-11T21:55:18.030694+0000","last_active":"2024-09-11T21:57:13.788398+0000","last_peered":"2024-09-11T21:57:13.788398+0000","last_clean":"2024-09-11T21:57:13.788398+0000","last_became_active":"2024-09-11T21:55:18.029687+0000","last_became_peered":"2024-09-11T21:55:18.029687+0000","last_unstale":"2024-09-11T21:57:13.788398+0000","last_undegraded":"2024-09-11T21:57:13.788398+0000","last_fullsized":"2024-09-11T21:57:13.788398+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:18:15.009543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"86'7","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.669625+0000","last_change":"2024-09-11T21:56:26.376104+0000","last_active":"2024-09-11T21:57:13.669625+0000","last_peered":"2024-09-11T21:57:13.669625+0000","last_clean":"2024-09-11T21:57:13.669625+0000","last_became_active":"2024-09-11T21:56:26.375665+0000","last_became_peered":"2024-09-11T21:56:26.375665+0000","last_unstale":"2024-09-11T21:57:13.669625+0000","last_undegraded":"2024-09-11T21:57:13.669625+0000","last_fullsized":"2024-09-11T21:57:13.669625+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:19:46.616205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"84'2","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788029+0000","last_change":"2024-09-11T21:55:18.115977+0000","last_active":"2024-09-11T21:57:13.788029+0000","last_peered":"2024-09-11T21:57:13.788029+0000","last_clean":"2024-09-11T21:57:13.788029+0000","last_became_active":"2024-09-11T21:55:18.115649+0000","last_became_peered":"2024-09-11T21:55:18.115649+0000","last_unstale":"2024-09-11T21:57:13.788029+0000","last_undegraded":"2024-09-11T21:57:13.788029+0000","last_fullsized":"2024-09-11T21:57:13.788029+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:42:07.621024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"86'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.670884+0000","last_change":"2024-09-11T21:55:20.095536+0000","last_active":"2024-09-11T21:57:13.670884+0000","last_peered":"2024-09-11T21:57:13.670884+0000","last_clean":"2024-09-11T21:57:13.670884+0000","last_became_active":"2024-09-11T21:55:20.095117+0000","last_became_peered":"2024-09-11T21:55:20.095117+0000","last_unstale":"2024-09-11T21:57:13.670884+0000","last_undegraded":"2024-09-11T21:57:13.670884+0000","last_fullsized":"2024-09-11T21:57:13.670884+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:46.524922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"84'7","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.753348+0000","last_change":"2024-09-11T21:55:18.026592+0000","last_active":"2024-09-11T21:57:13.753348+0000","last_peered":"2024-09-11T21:57:13.753348+0000","last_clean":"2024-09-11T21:57:13.753348+0000","last_became_active":"2024-09-11T21:55:18.026263+0000","last_became_peered":"2024-09-11T21:55:18.026263+0000","last_unstale":"2024-09-11T21:57:13.753348+0000","last_undegraded":"2024-09-11T21:57:13.753348+0000","last_fullsized":"2024-09-11T21:57:13.753348+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:22:29.382964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788053+0000","last_change":"2024-09-11T21:55:20.093655+0000","last_active":"2024-09-11T21:57:13.788053+0000","last_peered":"2024-09-11T21:57:13.788053+0000","last_clean":"2024-09-11T21:57:13.788053+0000","last_became_active":"2024-09-11T21:55:20.093287+0000","last_became_peered":"2024-09-11T21:55:20.093287+0000","last_unstale":"2024-09-11T21:57:13.788053+0000","last_undegraded":"2024-09-11T21:57:13.788053+0000","last_fullsized":"2024-09-11T21:57:13.788053+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:46:15.912710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"84'3","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.670879+0000","last_change":"2024-09-11T21:55:18.028465+0000","last_active":"2024-09-11T21:57:13.670879+0000","last_peered":"2024-09-11T21:57:13.670879+0000","last_clean":"2024-09-11T21:57:13.670879+0000","last_became_active":"2024-09-11T21:55:18.028139+0000","last_became_peered":"2024-09-11T21:55:18.028139+0000","last_unstale":"2024-09-11T21:57:13.670879+0000","last_undegraded":"2024-09-11T21:57:13.670879+0000","last_fullsized":"2024-09-11T21:57:13.670879+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:52:47.808058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.753473+0000","last_change":"2024-09-11T21:55:20.096400+0000","last_active":"2024-09-11T21:57:13.753473+0000","last_peered":"2024-09-11T21:57:13.753473+0000","last_clean":"2024-09-11T21:57:13.753473+0000","last_became_active":"2024-09-11T21:55:20.096023+0000","last_became_peered":"2024-09-11T21:55:20.096023+0000","last_unstale":"2024-09-11T21:57:13.753473+0000","last_undegraded":"2024-09-11T21:57:13.753473+0000","last_fullsized":"2024-09-11T21:57:13.753473+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:19:34.150687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"84'4","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671181+0000","last_change":"2024-09-11T21:55:18.028759+0000","last_active":"2024-09-11T21:57:13.671181+0000","last_peered":"2024-09-11T21:57:13.671181+0000","last_clean":"2024-09-11T21:57:13.671181+0000","last_became_active":"2024-09-11T21:55:18.028366+0000","last_became_peered":"2024-09-11T21:55:18.028366+0000","last_unstale":"2024-09-11T21:57:13.671181+0000","last_undegraded":"2024-09-11T21:57:13.671181+0000","last_fullsized":"2024-09-11T21:57:13.671181+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:42:05.836114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"86'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671002+0000","last_change":"2024-09-11T21:55:20.093593+0000","last_active":"2024-09-11T21:57:13.671002+0000","last_peered":"2024-09-11T21:57:13.671002+0000","last_clean":"2024-09-11T21:57:13.671002+0000","last_became_active":"2024-09-11T21:55:20.093189+0000","last_became_peered":"2024-09-11T21:55:20.093189+0000","last_unstale":"2024-09-11T21:57:13.671002+0000","last_undegraded":"2024-09-11T21:57:13.671002+0000","last_fullsized":"2024-09-11T21:57:13.671002+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:47:36.802816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"86'15","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.762163+0000","last_change":"2024-09-11T21:56:26.292793+0000","last_active":"2024-09-11T21:57:13.762163+0000","last_peered":"2024-09-11T21:57:13.762163+0000","last_clean":"2024-09-11T21:57:13.762163+0000","last_became_active":"2024-09-11T21:56:26.292437+0000","last_became_peered":"2024-09-11T21:56:26.292437+0000","last_unstale":"2024-09-11T21:57:13.762163+0000","last_undegraded":"2024-09-11T21:57:13.762163+0000","last_fullsized":"2024-09-11T21:57:13.762163+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:49:08.867435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"84'5","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671366+0000","last_change":"2024-09-11T21:55:18.027365+0000","last_active":"2024-09-11T21:57:13.671366+0000","last_peered":"2024-09-11T21:57:13.671366+0000","last_clean":"2024-09-11T21:57:13.671366+0000","last_became_active":"2024-09-11T21:55:18.027058+0000","last_became_peered":"2024-09-11T21:55:18.027058+0000","last_unstale":"2024-09-11T21:57:13.671366+0000","last_undegraded":"2024-09-11T21:57:13.671366+0000","last_fullsized":"2024-09-11T21:57:13.671366+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:30:50.613246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788034+0000","last_change":"2024-09-11T21:55:20.095154+0000","last_active":"2024-09-11T21:57:13.788034+0000","last_peered":"2024-09-11T21:57:13.788034+0000","last_clean":"2024-09-11T21:57:13.788034+0000","last_became_active":"2024-09-11T21:55:20.093607+0000","last_became_peered":"2024-09-11T21:55:20.093607+0000","last_unstale":"2024-09-11T21:57:13.788034+0000","last_undegraded":"2024-09-11T21:57:13.788034+0000","last_fullsized":"2024-09-11T21:57:13.788034+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:27:53.612467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"87'13","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671385+0000","last_change":"2024-09-11T21:55:18.029872+0000","last_active":"2024-09-11T21:57:13.671385+0000","last_peered":"2024-09-11T21:57:13.671385+0000","last_clean":"2024-09-11T21:57:13.671385+0000","last_became_active":"2024-09-11T21:55:18.029538+0000","last_became_peered":"2024-09-11T21:55:18.029538+0000","last_unstale":"2024-09-11T21:57:13.671385+0000","last_undegraded":"2024-09-11T21:57:13.671385+0000","last_fullsized":"2024-09-11T21:57:13.671385+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:05:16.824805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"86'6","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.669783+0000","last_change":"2024-09-11T21:55:20.098713+0000","last_active":"2024-09-11T21:57:13.669783+0000","last_peered":"2024-09-11T21:57:13.669783+0000","last_clean":"2024-09-11T21:57:13.669783+0000","last_became_active":"2024-09-11T21:55:20.098219+0000","last_became_peered":"2024-09-11T21:55:20.098219+0000","last_unstale":"2024-09-11T21:57:13.669783+0000","last_undegraded":"2024-09-11T21:57:13.669783+0000","last_fullsized":"2024-09-11T21:57:13.669783+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:07:10.585366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"84'2","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.753854+0000","last_change":"2024-09-11T21:55:18.115868+0000","last_active":"2024-09-11T21:57:13.753854+0000","last_peered":"2024-09-11T21:57:13.753854+0000","last_clean":"2024-09-11T21:57:13.753854+0000","last_became_active":"2024-09-11T21:55:18.115486+0000","last_became_peered":"2024-09-11T21:55:18.115486+0000","last_unstale":"2024-09-11T21:57:13.753854+0000","last_undegraded":"2024-09-11T21:57:13.753854+0000","last_fullsized":"2024-09-11T21:57:13.753854+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:16:57.932041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"86'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.753686+0000","last_change":"2024-09-11T21:55:20.096077+0000","last_active":"2024-09-11T21:57:13.753686+0000","last_peered":"2024-09-11T21:57:13.753686+0000","last_clean":"2024-09-11T21:57:13.753686+0000","last_became_active":"2024-09-11T21:55:20.095739+0000","last_became_peered":"2024-09-11T21:55:20.095739+0000","last_unstale":"2024-09-11T21:57:13.753686+0000","last_undegraded":"2024-09-11T21:57:13.753686+0000","last_fullsized":"2024-09-11T21:57:13.753686+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:05:28.089828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"87'15","reported_seq":143,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.931982+0000","last_change":"2024-09-11T21:55:18.025771+0000","last_active":"2024-09-11T21:57:13.931982+0000","last_peered":"2024-09-11T21:57:13.931982+0000","last_clean":"2024-09-11T21:57:13.931982+0000","last_became_active":"2024-09-11T21:55:18.025384+0000","last_became_peered":"2024-09-11T21:55:18.025384+0000","last_unstale":"2024-09-11T21:57:13.931982+0000","last_undegraded":"2024-09-11T21:57:13.931982+0000","last_fullsized":"2024-09-11T21:57:13.931982+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:58:21.070061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":104,"num_read_kb":104,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":601040,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1166,"num_read_kb":701,"num_write":380,"num_write_kb":1470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":59211,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":667648,"data_stored":1803120,"data_compressed":85492,"data_compressed_allocated":110592,"data_compressed_original":1605632,"omap_allocated":0,"internal_metadata":0},"log_size":306,"ondisk_log_size":306,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107988,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449124,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":937984,"data_stored":453556,"data_compressed":5771,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678056,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36448,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708704,"statfs":{"total":95995035648,"available":95957712896,"internally_reserved":0,"allocated":1130496,"data_stored":1251264,"data_compressed":50827,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":66,"seq":283467841582,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":974848,"data_stored":461937,"data_compressed":8675,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103220,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":995328,"data_stored":855531,"data_compressed":27821,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332152,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1024000,"data_stored":1057390,"data_compressed":44671,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495677,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":958464,"data_stored":462974,"data_compressed":8675,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757315,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":1186647,"data_compressed":50827,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920839,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":983040,"data_stored":521591,"data_compressed":18261,"data_compressed_allocated":24576,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182477,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":986295,"data_compressed":33977,"data_compressed_allocated":45056,"data_compressed_original":540672,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":970752,"data_stored":464641,"data_compressed":8675,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1024000,"data_stored":529814,"data_compressed":8675,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":978944,"data_stored":463920,"data_compressed":8675,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":68999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":67328,"data_compressed":9586,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":135347,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":6090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":394716,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":199964,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T21:57:16.630 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-09-11T21:57:16.631 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T21:57:16.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.777+0000 7f20208a5640 1 -- 172.21.15.1:0/2768352063 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f201c100a50 msgr2=0x7f201c100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:16.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.777+0000 7f20208a5640 1 --2- 172.21.15.1:0/2768352063 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f201c100a50 0x7f201c100e50 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f200c009900 tx=0x7f200c02a410 comp rx=0 tx=0).stop 2024-09-11T21:57:16.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.777+0000 7f20208a5640 1 -- 172.21.15.1:0/2768352063 shutdown_connections 2024-09-11T21:57:16.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.777+0000 7f20208a5640 1 --2- 172.21.15.1:0/2768352063 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f201c102600 0x7f201c108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:16.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.777+0000 7f20208a5640 1 --2- 172.21.15.1:0/2768352063 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f201c101c40 0x7f201c1020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:16.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.777+0000 7f20208a5640 1 --2- 172.21.15.1:0/2768352063 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f201c100a50 0x7f201c100e50 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:16.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.777+0000 7f20208a5640 1 -- 172.21.15.1:0/2768352063 >> 172.21.15.1:0/2768352063 conn(0x7f201c0fc1f0 msgr2=0x7f201c0fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:16.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.777+0000 7f20208a5640 1 -- 172.21.15.1:0/2768352063 shutdown_connections 2024-09-11T21:57:16.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.777+0000 7f20208a5640 1 -- 172.21.15.1:0/2768352063 wait complete. 2024-09-11T21:57:16.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.778+0000 7f20208a5640 1 Processor -- start 2024-09-11T21:57:16.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.778+0000 7f20208a5640 1 -- start start 2024-09-11T21:57:16.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.778+0000 7f20208a5640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f201c100a50 0x7f201c196db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:16.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.778+0000 7f20208a5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f201c101c40 0x7f201c1972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:16.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.778+0000 7f20208a5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f201c102600 0x7f201c19e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:16.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.778+0000 7f20208a5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f201c10c700 con 0x7f201c101c40 2024-09-11T21:57:16.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.778+0000 7f20208a5640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f201c10c580 con 0x7f201c102600 2024-09-11T21:57:16.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.778+0000 7f20208a5640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f201c10c880 con 0x7f201c100a50 2024-09-11T21:57:16.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.778+0000 7f2019d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f201c101c40 0x7f201c1972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:16.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.778+0000 7f2019d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f201c101c40 0x7f201c1972f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38632/0 (socket says 172.21.15.1:38632) 2024-09-11T21:57:16.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.778+0000 7f2019d9b640 1 -- 172.21.15.1:0/479238188 learned_addr learned my addr 172.21.15.1:0/479238188 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:16.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.778+0000 7f201a59c640 1 --2- 172.21.15.1:0/479238188 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f201c100a50 0x7f201c196db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:16.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.778+0000 7f201ad9d640 1 --2- 172.21.15.1:0/479238188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f201c102600 0x7f201c19e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:16.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.779+0000 7f2019d9b640 1 -- 172.21.15.1:0/479238188 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f201c100a50 msgr2=0x7f201c196db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:16.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.779+0000 7f2019d9b640 1 --2- 172.21.15.1:0/479238188 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f201c100a50 0x7f201c196db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:16.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.779+0000 7f2019d9b640 1 -- 172.21.15.1:0/479238188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f201c102600 msgr2=0x7f201c19e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:16.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.779+0000 7f2019d9b640 1 --2- 172.21.15.1:0/479238188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f201c102600 0x7f201c19e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:16.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.779+0000 7f2019d9b640 1 -- 172.21.15.1:0/479238188 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f201c19e9d0 con 0x7f201c101c40 2024-09-11T21:57:16.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.779+0000 7f201ad9d640 1 --2- 172.21.15.1:0/479238188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f201c102600 0x7f201c19e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:57:16.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.779+0000 7f2019d9b640 1 --2- 172.21.15.1:0/479238188 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f201c101c40 0x7f201c1972f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f201000d950 tx=0x7f201000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:16.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.779+0000 7f200b7fe640 1 -- 172.21.15.1:0/479238188 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2010017070 con 0x7f201c101c40 2024-09-11T21:57:16.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.779+0000 7f20208a5640 1 -- 172.21.15.1:0/479238188 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f201c19ecc0 con 0x7f201c101c40 2024-09-11T21:57:16.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.779+0000 7f200b7fe640 1 -- 172.21.15.1:0/479238188 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2010009c30 con 0x7f201c101c40 2024-09-11T21:57:16.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.779+0000 7f200b7fe640 1 -- 172.21.15.1:0/479238188 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201000f6b0 con 0x7f201c101c40 2024-09-11T21:57:16.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.779+0000 7f20208a5640 1 -- 172.21.15.1:0/479238188 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f201c19f200 con 0x7f201c101c40 2024-09-11T21:57:16.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.780+0000 7f20208a5640 1 -- 172.21.15.1:0/479238188 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f201c107ee0 con 0x7f201c101c40 2024-09-11T21:57:16.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.782+0000 7f200b7fe640 1 -- 172.21.15.1:0/479238188 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 23) ==== 137952+0+0 (secure 0 0 0) 0x7f20100109d0 con 0x7f201c101c40 2024-09-11T21:57:16.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.783+0000 7f200b7fe640 1 --2- 172.21.15.1:0/479238188 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f20040a2d00 0x7f20040a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:16.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.783+0000 7f200b7fe640 1 -- 172.21.15.1:0/479238188 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7f20100cf020 con 0x7f201c101c40 2024-09-11T21:57:16.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.783+0000 7f201a59c640 1 --2- 172.21.15.1:0/479238188 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f20040a2d00 0x7f20040a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:16.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.784+0000 7f201a59c640 1 --2- 172.21.15.1:0/479238188 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f20040a2d00 0x7f20040a51c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f200c02aa00 tx=0x7f200c0315b0 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:16.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.786+0000 7f200b7fe640 1 -- 172.21.15.1:0/479238188 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2010009de0 con 0x7f201c101c40 2024-09-11T21:57:16.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:16 smithi001 ceph-mon[30935]: [11/Sep/2024:21:57:15] ENGINE Error in HTTPServer.serve 2024-09-11T21:57:16.944 INFO:journalctl@ceph.mon.a.smithi001.stdout: Traceback (most recent call last): 2024-09-11T21:57:16.944 INFO:journalctl@ceph.mon.a.smithi001.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T21:57:16.944 INFO:journalctl@ceph.mon.a.smithi001.stdout: self._connections.run(self.expiration_interval) 2024-09-11T21:57:16.944 INFO:journalctl@ceph.mon.a.smithi001.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T21:57:16.944 INFO:journalctl@ceph.mon.a.smithi001.stdout: self._run(expiration_interval) 2024-09-11T21:57:16.944 INFO:journalctl@ceph.mon.a.smithi001.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T21:57:16.944 INFO:journalctl@ceph.mon.a.smithi001.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T21:57:16.944 INFO:journalctl@ceph.mon.a.smithi001.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T21:57:16.945 INFO:journalctl@ceph.mon.a.smithi001.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T21:57:16.945 INFO:journalctl@ceph.mon.a.smithi001.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T21:57:16.945 INFO:journalctl@ceph.mon.a.smithi001.stdout: s = self.context.wrap_socket( 2024-09-11T21:57:16.945 INFO:journalctl@ceph.mon.a.smithi001.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T21:57:16.945 INFO:journalctl@ceph.mon.a.smithi001.stdout: return self.sslsocket_class._create( 2024-09-11T21:57:16.945 INFO:journalctl@ceph.mon.a.smithi001.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T21:57:16.945 INFO:journalctl@ceph.mon.a.smithi001.stdout: self.do_handshake() 2024-09-11T21:57:16.945 INFO:journalctl@ceph.mon.a.smithi001.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T21:57:16.945 INFO:journalctl@ceph.mon.a.smithi001.stdout: self._sslobj.do_handshake() 2024-09-11T21:57:16.945 INFO:journalctl@ceph.mon.a.smithi001.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T21:57:16.945 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:16 smithi001 ceph-mon[30935]: [11/Sep/2024:21:57:15] ENGINE Serving on http://172.21.15.1:8765 2024-09-11T21:57:16.945 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:16 smithi001 ceph-mon[30935]: [11/Sep/2024:21:57:15] ENGINE Bus STARTED 2024-09-11T21:57:16.946 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:16 smithi001 ceph-mon[30935]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:57:16.946 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:16 smithi001 ceph-mon[30935]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-09-11T21:57:16.946 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:16 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:57:16.946 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:16 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:57:16.946 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:16 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:57:16.946 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:16 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:57:16.946 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:16 smithi001 ceph-mon[30935]: from='client.15297 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T21:57:16.946 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:16 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:57:16.946 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:16 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:57:16.946 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:16 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:16.946 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:16 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:16.946 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:16 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:16.947 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:16 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:16.947 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:16 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:16.947 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:16 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:16.947 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:16 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:16.947 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:16 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:16.947 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:16 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:57:16.947 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:16 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:57:16.947 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:16 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:16.947 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:16 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:16.947 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:16 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:16.947 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:16 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:16.948 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:16 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:16.948 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:16 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:16.948 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:16 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:16.948 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:16 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:16.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.988+0000 7f20208a5640 1 -- 172.21.15.1:0/479238188 --> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f201c062630 con 0x7f20040a2d00 2024-09-11T21:57:16.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.995+0000 7f200b7fe640 1 -- 172.21.15.1:0/479238188 <== mgr.15204 v2:172.21.15.1:6800/3543150772 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258070 (secure 0 0 0) 0x7f201c062630 con 0x7f20040a2d00 2024-09-11T21:57:16.998 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:57:16.999 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-11T21:57:17.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.998+0000 7f20208a5640 1 -- 172.21.15.1:0/479238188 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f20040a2d00 msgr2=0x7f20040a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:17.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.998+0000 7f20208a5640 1 --2- 172.21.15.1:0/479238188 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f20040a2d00 0x7f20040a51c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f200c02aa00 tx=0x7f200c0315b0 comp rx=0 tx=0).stop 2024-09-11T21:57:17.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.998+0000 7f20208a5640 1 -- 172.21.15.1:0/479238188 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f201c101c40 msgr2=0x7f201c1972f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:17.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.998+0000 7f20208a5640 1 --2- 172.21.15.1:0/479238188 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f201c101c40 0x7f201c1972f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f201000d950 tx=0x7f201000de10 comp rx=0 tx=0).stop 2024-09-11T21:57:17.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.998+0000 7f20208a5640 1 -- 172.21.15.1:0/479238188 shutdown_connections 2024-09-11T21:57:17.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.998+0000 7f20208a5640 1 --2- 172.21.15.1:0/479238188 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f20040a2d00 0x7f20040a51c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:17.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.998+0000 7f20208a5640 1 --2- 172.21.15.1:0/479238188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f201c102600 0x7f201c19e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:17.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.998+0000 7f20208a5640 1 --2- 172.21.15.1:0/479238188 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f201c101c40 0x7f201c1972f0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:17.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.998+0000 7f20208a5640 1 --2- 172.21.15.1:0/479238188 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f201c100a50 0x7f201c196db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:17.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.998+0000 7f20208a5640 1 -- 172.21.15.1:0/479238188 >> 172.21.15.1:0/479238188 conn(0x7f201c0fc1f0 msgr2=0x7f201c1050b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:17.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.998+0000 7f20208a5640 1 -- 172.21.15.1:0/479238188 shutdown_connections 2024-09-11T21:57:17.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:16.998+0000 7f20208a5640 1 -- 172.21.15.1:0/479238188 wait complete. 2024-09-11T21:57:17.009 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":4,"stamp":"2024-09-11T21:57:15.682853+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1191642,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1302,"num_read_kb":820,"num_write":563,"num_write_kb":4410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1239947,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":945,"ondisk_log_size":945,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":390172,"kb_used_data":11772,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124551652,"statfs":{"total":1151940427776,"available":1151540891648,"internally_reserved":0,"allocated":12054528,"data_stored":8695560,"data_compressed":275530,"data_compressed_allocated":364544,"data_compressed_original":3268608,"omap_allocated":19142,"internal_metadata":387233082},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.990279"},"pg_stats":[{"pgid":"3.3f","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788313+0000","last_change":"2024-09-11T21:55:20.095385+0000","last_active":"2024-09-11T21:57:13.788313+0000","last_peered":"2024-09-11T21:57:13.788313+0000","last_clean":"2024-09-11T21:57:13.788313+0000","last_became_active":"2024-09-11T21:55:20.095077+0000","last_became_peered":"2024-09-11T21:55:20.095077+0000","last_unstale":"2024-09-11T21:57:13.788313+0000","last_undegraded":"2024-09-11T21:57:13.788313+0000","last_fullsized":"2024-09-11T21:57:13.788313+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:53:52.231522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"86'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.669584+0000","last_change":"2024-09-11T21:55:20.103540+0000","last_active":"2024-09-11T21:57:13.669584+0000","last_peered":"2024-09-11T21:57:13.669584+0000","last_clean":"2024-09-11T21:57:13.669584+0000","last_became_active":"2024-09-11T21:55:20.103233+0000","last_became_peered":"2024-09-11T21:55:20.103233+0000","last_unstale":"2024-09-11T21:57:13.669584+0000","last_undegraded":"2024-09-11T21:57:13.669584+0000","last_fullsized":"2024-09-11T21:57:13.669584+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:51:29.222202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.669576+0000","last_change":"2024-09-11T21:55:20.098538+0000","last_active":"2024-09-11T21:57:13.669576+0000","last_peered":"2024-09-11T21:57:13.669576+0000","last_clean":"2024-09-11T21:57:13.669576+0000","last_became_active":"2024-09-11T21:55:20.098202+0000","last_became_peered":"2024-09-11T21:55:20.098202+0000","last_unstale":"2024-09-11T21:57:13.669576+0000","last_undegraded":"2024-09-11T21:57:13.669576+0000","last_fullsized":"2024-09-11T21:57:13.669576+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:33:46.636952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788491+0000","last_change":"2024-09-11T21:55:20.100869+0000","last_active":"2024-09-11T21:57:13.788491+0000","last_peered":"2024-09-11T21:57:13.788491+0000","last_clean":"2024-09-11T21:57:13.788491+0000","last_became_active":"2024-09-11T21:55:20.100573+0000","last_became_peered":"2024-09-11T21:55:20.100573+0000","last_unstale":"2024-09-11T21:57:13.788491+0000","last_undegraded":"2024-09-11T21:57:13.788491+0000","last_fullsized":"2024-09-11T21:57:13.788491+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:58:11.154105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"86'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671033+0000","last_change":"2024-09-11T21:55:20.091648+0000","last_active":"2024-09-11T21:57:13.671033+0000","last_peered":"2024-09-11T21:57:13.671033+0000","last_clean":"2024-09-11T21:57:13.671033+0000","last_became_active":"2024-09-11T21:55:20.091222+0000","last_became_peered":"2024-09-11T21:55:20.091222+0000","last_unstale":"2024-09-11T21:57:13.671033+0000","last_undegraded":"2024-09-11T21:57:13.671033+0000","last_fullsized":"2024-09-11T21:57:13.671033+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:55:03.420754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"86'6","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.764975+0000","last_change":"2024-09-11T21:56:26.376595+0000","last_active":"2024-09-11T21:57:13.764975+0000","last_peered":"2024-09-11T21:57:13.764975+0000","last_clean":"2024-09-11T21:57:13.764975+0000","last_became_active":"2024-09-11T21:56:26.375784+0000","last_became_peered":"2024-09-11T21:56:26.375784+0000","last_unstale":"2024-09-11T21:57:13.764975+0000","last_undegraded":"2024-09-11T21:57:13.764975+0000","last_fullsized":"2024-09-11T21:57:13.764975+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:19:38.371463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,7],"acting":[10,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"86'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.670787+0000","last_change":"2024-09-11T21:55:20.101084+0000","last_active":"2024-09-11T21:57:13.670787+0000","last_peered":"2024-09-11T21:57:13.670787+0000","last_clean":"2024-09-11T21:57:13.670787+0000","last_became_active":"2024-09-11T21:55:20.100739+0000","last_became_peered":"2024-09-11T21:55:20.100739+0000","last_unstale":"2024-09-11T21:57:13.670787+0000","last_undegraded":"2024-09-11T21:57:13.670787+0000","last_fullsized":"2024-09-11T21:57:13.670787+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:46:58.969559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.753160+0000","last_change":"2024-09-11T21:55:20.088715+0000","last_active":"2024-09-11T21:57:13.753160+0000","last_peered":"2024-09-11T21:57:13.753160+0000","last_clean":"2024-09-11T21:57:13.753160+0000","last_became_active":"2024-09-11T21:55:20.088280+0000","last_became_peered":"2024-09-11T21:55:20.088280+0000","last_unstale":"2024-09-11T21:57:13.753160+0000","last_undegraded":"2024-09-11T21:57:13.753160+0000","last_fullsized":"2024-09-11T21:57:13.753160+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:06:24.450444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"86'11","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788671+0000","last_change":"2024-09-11T21:55:20.101266+0000","last_active":"2024-09-11T21:57:13.788671+0000","last_peered":"2024-09-11T21:57:13.788671+0000","last_clean":"2024-09-11T21:57:13.788671+0000","last_became_active":"2024-09-11T21:55:20.100919+0000","last_became_peered":"2024-09-11T21:55:20.100919+0000","last_unstale":"2024-09-11T21:57:13.788671+0000","last_undegraded":"2024-09-11T21:57:13.788671+0000","last_fullsized":"2024-09-11T21:57:13.788671+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:47:15.466082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"86'16","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.765361+0000","last_change":"2024-09-11T21:55:20.098086+0000","last_active":"2024-09-11T21:57:13.765361+0000","last_peered":"2024-09-11T21:57:13.765361+0000","last_clean":"2024-09-11T21:57:13.765361+0000","last_became_active":"2024-09-11T21:55:20.097754+0000","last_became_peered":"2024-09-11T21:55:20.097754+0000","last_unstale":"2024-09-11T21:57:13.765361+0000","last_undegraded":"2024-09-11T21:57:13.765361+0000","last_fullsized":"2024-09-11T21:57:13.765361+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:37:00.098345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"86'6","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671036+0000","last_change":"2024-09-11T21:55:20.092671+0000","last_active":"2024-09-11T21:57:13.671036+0000","last_peered":"2024-09-11T21:57:13.671036+0000","last_clean":"2024-09-11T21:57:13.671036+0000","last_became_active":"2024-09-11T21:55:20.092112+0000","last_became_peered":"2024-09-11T21:55:20.092112+0000","last_unstale":"2024-09-11T21:57:13.671036+0000","last_undegraded":"2024-09-11T21:57:13.671036+0000","last_fullsized":"2024-09-11T21:57:13.671036+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:06:28.538838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788646+0000","last_change":"2024-09-11T21:55:20.099863+0000","last_active":"2024-09-11T21:57:13.788646+0000","last_peered":"2024-09-11T21:57:13.788646+0000","last_clean":"2024-09-11T21:57:13.788646+0000","last_became_active":"2024-09-11T21:55:20.099173+0000","last_became_peered":"2024-09-11T21:55:20.099173+0000","last_unstale":"2024-09-11T21:57:13.788646+0000","last_undegraded":"2024-09-11T21:57:13.788646+0000","last_fullsized":"2024-09-11T21:57:13.788646+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:34:56.669410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"86'10","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788507+0000","last_change":"2024-09-11T21:55:20.101212+0000","last_active":"2024-09-11T21:57:13.788507+0000","last_peered":"2024-09-11T21:57:13.788507+0000","last_clean":"2024-09-11T21:57:13.788507+0000","last_became_active":"2024-09-11T21:55:20.100876+0000","last_became_peered":"2024-09-11T21:55:20.100876+0000","last_unstale":"2024-09-11T21:57:13.788507+0000","last_undegraded":"2024-09-11T21:57:13.788507+0000","last_fullsized":"2024-09-11T21:57:13.788507+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:14:18.104009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.669316+0000","last_change":"2024-09-11T21:55:20.095394+0000","last_active":"2024-09-11T21:57:13.669316+0000","last_peered":"2024-09-11T21:57:13.669316+0000","last_clean":"2024-09-11T21:57:13.669316+0000","last_became_active":"2024-09-11T21:55:20.095101+0000","last_became_peered":"2024-09-11T21:55:20.095101+0000","last_unstale":"2024-09-11T21:57:13.669316+0000","last_undegraded":"2024-09-11T21:57:13.669316+0000","last_fullsized":"2024-09-11T21:57:13.669316+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:53:32.529359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"86'7","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.669266+0000","last_change":"2024-09-11T21:56:26.376206+0000","last_active":"2024-09-11T21:57:13.669266+0000","last_peered":"2024-09-11T21:57:13.669266+0000","last_clean":"2024-09-11T21:57:13.669266+0000","last_became_active":"2024-09-11T21:56:26.375829+0000","last_became_peered":"2024-09-11T21:56:26.375829+0000","last_unstale":"2024-09-11T21:57:13.669266+0000","last_undegraded":"2024-09-11T21:57:13.669266+0000","last_fullsized":"2024-09-11T21:57:13.669266+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:21:59.484414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.787934+0000","last_change":"2024-09-11T21:55:20.093019+0000","last_active":"2024-09-11T21:57:13.787934+0000","last_peered":"2024-09-11T21:57:13.787934+0000","last_clean":"2024-09-11T21:57:13.787934+0000","last_became_active":"2024-09-11T21:55:20.092691+0000","last_became_peered":"2024-09-11T21:55:20.092691+0000","last_unstale":"2024-09-11T21:57:13.787934+0000","last_undegraded":"2024-09-11T21:57:13.787934+0000","last_fullsized":"2024-09-11T21:57:13.787934+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:04:36.684992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788379+0000","last_change":"2024-09-11T21:55:20.097333+0000","last_active":"2024-09-11T21:57:13.788379+0000","last_peered":"2024-09-11T21:57:13.788379+0000","last_clean":"2024-09-11T21:57:13.788379+0000","last_became_active":"2024-09-11T21:55:20.096752+0000","last_became_peered":"2024-09-11T21:55:20.096752+0000","last_unstale":"2024-09-11T21:57:13.788379+0000","last_undegraded":"2024-09-11T21:57:13.788379+0000","last_fullsized":"2024-09-11T21:57:13.788379+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:27:20.722174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"86'10","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.751322+0000","last_change":"2024-09-11T21:55:20.093972+0000","last_active":"2024-09-11T21:57:13.751322+0000","last_peered":"2024-09-11T21:57:13.751322+0000","last_clean":"2024-09-11T21:57:13.751322+0000","last_became_active":"2024-09-11T21:55:20.093633+0000","last_became_peered":"2024-09-11T21:55:20.093633+0000","last_unstale":"2024-09-11T21:57:13.751322+0000","last_undegraded":"2024-09-11T21:57:13.751322+0000","last_fullsized":"2024-09-11T21:57:13.751322+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:25:18.973332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671334+0000","last_change":"2024-09-11T21:55:20.094239+0000","last_active":"2024-09-11T21:57:13.671334+0000","last_peered":"2024-09-11T21:57:13.671334+0000","last_clean":"2024-09-11T21:57:13.671334+0000","last_became_active":"2024-09-11T21:55:20.093924+0000","last_became_peered":"2024-09-11T21:55:20.093924+0000","last_unstale":"2024-09-11T21:57:13.671334+0000","last_undegraded":"2024-09-11T21:57:13.671334+0000","last_fullsized":"2024-09-11T21:57:13.671334+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:39:53.012780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788075+0000","last_change":"2024-09-11T21:55:20.099335+0000","last_active":"2024-09-11T21:57:13.788075+0000","last_peered":"2024-09-11T21:57:13.788075+0000","last_clean":"2024-09-11T21:57:13.788075+0000","last_became_active":"2024-09-11T21:55:20.098929+0000","last_became_peered":"2024-09-11T21:55:20.098929+0000","last_unstale":"2024-09-11T21:57:13.788075+0000","last_undegraded":"2024-09-11T21:57:13.788075+0000","last_fullsized":"2024-09-11T21:57:13.788075+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:38:43.117829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"86'11","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788526+0000","last_change":"2024-09-11T21:55:20.094068+0000","last_active":"2024-09-11T21:57:13.788526+0000","last_peered":"2024-09-11T21:57:13.788526+0000","last_clean":"2024-09-11T21:57:13.788526+0000","last_became_active":"2024-09-11T21:55:20.093637+0000","last_became_peered":"2024-09-11T21:55:20.093637+0000","last_unstale":"2024-09-11T21:57:13.788526+0000","last_undegraded":"2024-09-11T21:57:13.788526+0000","last_fullsized":"2024-09-11T21:57:13.788526+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:52:59.289715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"86'10","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788122+0000","last_change":"2024-09-11T21:55:20.090778+0000","last_active":"2024-09-11T21:57:13.788122+0000","last_peered":"2024-09-11T21:57:13.788122+0000","last_clean":"2024-09-11T21:57:13.788122+0000","last_became_active":"2024-09-11T21:55:20.090443+0000","last_became_peered":"2024-09-11T21:55:20.090443+0000","last_unstale":"2024-09-11T21:57:13.788122+0000","last_undegraded":"2024-09-11T21:57:13.788122+0000","last_fullsized":"2024-09-11T21:57:13.788122+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:46:21.488228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.753203+0000","last_change":"2024-09-11T21:55:20.092238+0000","last_active":"2024-09-11T21:57:13.753203+0000","last_peered":"2024-09-11T21:57:13.753203+0000","last_clean":"2024-09-11T21:57:13.753203+0000","last_became_active":"2024-09-11T21:55:20.091961+0000","last_became_peered":"2024-09-11T21:55:20.091961+0000","last_unstale":"2024-09-11T21:57:13.753203+0000","last_undegraded":"2024-09-11T21:57:13.753203+0000","last_fullsized":"2024-09-11T21:57:13.753203+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:54:40.445441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"86'6","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.761984+0000","last_change":"2024-09-11T21:55:20.102944+0000","last_active":"2024-09-11T21:57:13.761984+0000","last_peered":"2024-09-11T21:57:13.761984+0000","last_clean":"2024-09-11T21:57:13.761984+0000","last_became_active":"2024-09-11T21:55:20.102533+0000","last_became_peered":"2024-09-11T21:55:20.102533+0000","last_unstale":"2024-09-11T21:57:13.761984+0000","last_undegraded":"2024-09-11T21:57:13.761984+0000","last_fullsized":"2024-09-11T21:57:13.761984+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:12:15.832622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"86'2","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.753729+0000","last_change":"2024-09-11T21:56:26.292272+0000","last_active":"2024-09-11T21:57:13.753729+0000","last_peered":"2024-09-11T21:57:13.753729+0000","last_clean":"2024-09-11T21:57:13.753729+0000","last_became_active":"2024-09-11T21:56:26.291920+0000","last_became_peered":"2024-09-11T21:56:26.291920+0000","last_unstale":"2024-09-11T21:57:13.753729+0000","last_undegraded":"2024-09-11T21:57:13.753729+0000","last_fullsized":"2024-09-11T21:57:13.753729+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:13:13.579907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"86'15","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671489+0000","last_change":"2024-09-11T21:56:26.376364+0000","last_active":"2024-09-11T21:57:13.671489+0000","last_peered":"2024-09-11T21:57:13.671489+0000","last_clean":"2024-09-11T21:57:13.671489+0000","last_became_active":"2024-09-11T21:56:26.375927+0000","last_became_peered":"2024-09-11T21:56:26.375927+0000","last_unstale":"2024-09-11T21:57:13.671489+0000","last_undegraded":"2024-09-11T21:57:13.671489+0000","last_fullsized":"2024-09-11T21:57:13.671489+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:23:03.664207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,9],"acting":[2,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"86'6","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788153+0000","last_change":"2024-09-11T21:56:26.376702+0000","last_active":"2024-09-11T21:57:13.788153+0000","last_peered":"2024-09-11T21:57:13.788153+0000","last_clean":"2024-09-11T21:57:13.788153+0000","last_became_active":"2024-09-11T21:56:26.376179+0000","last_became_peered":"2024-09-11T21:56:26.376179+0000","last_unstale":"2024-09-11T21:57:13.788153+0000","last_undegraded":"2024-09-11T21:57:13.788153+0000","last_fullsized":"2024-09-11T21:57:13.788153+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:27:34.666962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"86'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788960+0000","last_change":"2024-09-11T21:55:20.093898+0000","last_active":"2024-09-11T21:57:13.788960+0000","last_peered":"2024-09-11T21:57:13.788960+0000","last_clean":"2024-09-11T21:57:13.788960+0000","last_became_active":"2024-09-11T21:55:20.093348+0000","last_became_peered":"2024-09-11T21:55:20.093348+0000","last_unstale":"2024-09-11T21:57:13.788960+0000","last_undegraded":"2024-09-11T21:57:13.788960+0000","last_fullsized":"2024-09-11T21:57:13.788960+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:00:06.185478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.669457+0000","last_change":"2024-09-11T21:55:20.100575+0000","last_active":"2024-09-11T21:57:13.669457+0000","last_peered":"2024-09-11T21:57:13.669457+0000","last_clean":"2024-09-11T21:57:13.669457+0000","last_became_active":"2024-09-11T21:55:20.100255+0000","last_became_peered":"2024-09-11T21:55:20.100255+0000","last_unstale":"2024-09-11T21:57:13.669457+0000","last_undegraded":"2024-09-11T21:57:13.669457+0000","last_fullsized":"2024-09-11T21:57:13.669457+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:48:57.718298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"86'8","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.670722+0000","last_change":"2024-09-11T21:56:26.377183+0000","last_active":"2024-09-11T21:57:13.670722+0000","last_peered":"2024-09-11T21:57:13.670722+0000","last_clean":"2024-09-11T21:57:13.670722+0000","last_became_active":"2024-09-11T21:56:26.375906+0000","last_became_peered":"2024-09-11T21:56:26.375906+0000","last_unstale":"2024-09-11T21:57:13.670722+0000","last_undegraded":"2024-09-11T21:57:13.670722+0000","last_fullsized":"2024-09-11T21:57:13.670722+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:00:29.033961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"86'12","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.670784+0000","last_change":"2024-09-11T21:55:20.093822+0000","last_active":"2024-09-11T21:57:13.670784+0000","last_peered":"2024-09-11T21:57:13.670784+0000","last_clean":"2024-09-11T21:57:13.670784+0000","last_became_active":"2024-09-11T21:55:20.093361+0000","last_became_peered":"2024-09-11T21:55:20.093361+0000","last_unstale":"2024-09-11T21:57:13.670784+0000","last_undegraded":"2024-09-11T21:57:13.670784+0000","last_fullsized":"2024-09-11T21:57:13.670784+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:37:06.362732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.787998+0000","last_change":"2024-09-11T21:55:20.103990+0000","last_active":"2024-09-11T21:57:13.787998+0000","last_peered":"2024-09-11T21:57:13.787998+0000","last_clean":"2024-09-11T21:57:13.787998+0000","last_became_active":"2024-09-11T21:55:20.103375+0000","last_became_peered":"2024-09-11T21:55:20.103375+0000","last_unstale":"2024-09-11T21:57:13.787998+0000","last_undegraded":"2024-09-11T21:57:13.787998+0000","last_fullsized":"2024-09-11T21:57:13.787998+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:02:06.786888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"86'7","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671024+0000","last_change":"2024-09-11T21:56:26.376261+0000","last_active":"2024-09-11T21:57:13.671024+0000","last_peered":"2024-09-11T21:57:13.671024+0000","last_clean":"2024-09-11T21:57:13.671024+0000","last_became_active":"2024-09-11T21:56:26.375863+0000","last_became_peered":"2024-09-11T21:56:26.375863+0000","last_unstale":"2024-09-11T21:57:13.671024+0000","last_undegraded":"2024-09-11T21:57:13.671024+0000","last_fullsized":"2024-09-11T21:57:13.671024+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:58:21.359836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"87'8","reported_seq":140,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:14.299174+0000","last_change":"2024-09-11T21:55:18.026947+0000","last_active":"2024-09-11T21:57:14.299174+0000","last_peered":"2024-09-11T21:57:14.299174+0000","last_clean":"2024-09-11T21:57:14.299174+0000","last_became_active":"2024-09-11T21:55:18.026632+0000","last_became_peered":"2024-09-11T21:55:18.026632+0000","last_unstale":"2024-09-11T21:57:14.299174+0000","last_undegraded":"2024-09-11T21:57:14.299174+0000","last_fullsized":"2024-09-11T21:57:14.299174+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:55:16.618074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1549,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":108,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.762157+0000","last_change":"2024-09-11T21:55:20.100197+0000","last_active":"2024-09-11T21:57:13.762157+0000","last_peered":"2024-09-11T21:57:13.762157+0000","last_clean":"2024-09-11T21:57:13.762157+0000","last_became_active":"2024-09-11T21:55:20.098494+0000","last_became_peered":"2024-09-11T21:55:20.098494+0000","last_unstale":"2024-09-11T21:57:13.762157+0000","last_undegraded":"2024-09-11T21:57:13.762157+0000","last_fullsized":"2024-09-11T21:57:13.762157+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:14.505314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"84'7","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.765023+0000","last_change":"2024-09-11T21:55:18.024359+0000","last_active":"2024-09-11T21:57:13.765023+0000","last_peered":"2024-09-11T21:57:13.765023+0000","last_clean":"2024-09-11T21:57:13.765023+0000","last_became_active":"2024-09-11T21:55:18.023624+0000","last_became_peered":"2024-09-11T21:55:18.023624+0000","last_unstale":"2024-09-11T21:57:13.765023+0000","last_undegraded":"2024-09-11T21:57:13.765023+0000","last_fullsized":"2024-09-11T21:57:13.765023+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:13:38.582232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"84'3","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788356+0000","last_change":"2024-09-11T21:55:18.032541+0000","last_active":"2024-09-11T21:57:13.788356+0000","last_peered":"2024-09-11T21:57:13.788356+0000","last_clean":"2024-09-11T21:57:13.788356+0000","last_became_active":"2024-09-11T21:55:18.032218+0000","last_became_peered":"2024-09-11T21:55:18.032218+0000","last_unstale":"2024-09-11T21:57:13.788356+0000","last_undegraded":"2024-09-11T21:57:13.788356+0000","last_fullsized":"2024-09-11T21:57:13.788356+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:38:20.872069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.753229+0000","last_change":"2024-09-11T21:55:20.090030+0000","last_active":"2024-09-11T21:57:13.753229+0000","last_peered":"2024-09-11T21:57:13.753229+0000","last_clean":"2024-09-11T21:57:13.753229+0000","last_became_active":"2024-09-11T21:55:20.089481+0000","last_became_peered":"2024-09-11T21:55:20.089481+0000","last_unstale":"2024-09-11T21:57:13.753229+0000","last_undegraded":"2024-09-11T21:57:13.753229+0000","last_fullsized":"2024-09-11T21:57:13.753229+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:41:57.038519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"84'3","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.762394+0000","last_change":"2024-09-11T21:55:18.026656+0000","last_active":"2024-09-11T21:57:13.762394+0000","last_peered":"2024-09-11T21:57:13.762394+0000","last_clean":"2024-09-11T21:57:13.762394+0000","last_became_active":"2024-09-11T21:55:18.026318+0000","last_became_peered":"2024-09-11T21:55:18.026318+0000","last_unstale":"2024-09-11T21:57:13.762394+0000","last_undegraded":"2024-09-11T21:57:13.762394+0000","last_fullsized":"2024-09-11T21:57:13.762394+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:14:41.438334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"86'11","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.750970+0000","last_change":"2024-09-11T21:55:20.090790+0000","last_active":"2024-09-11T21:57:13.750970+0000","last_peered":"2024-09-11T21:57:13.750970+0000","last_clean":"2024-09-11T21:57:13.750970+0000","last_became_active":"2024-09-11T21:55:20.090468+0000","last_became_peered":"2024-09-11T21:55:20.090468+0000","last_unstale":"2024-09-11T21:57:13.750970+0000","last_undegraded":"2024-09-11T21:57:13.750970+0000","last_fullsized":"2024-09-11T21:57:13.750970+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:19.115482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'8","reported_seq":238,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:14.298769+0000","last_change":"2024-09-11T21:55:18.026989+0000","last_active":"2024-09-11T21:57:14.298769+0000","last_peered":"2024-09-11T21:57:14.298769+0000","last_clean":"2024-09-11T21:57:14.298769+0000","last_became_active":"2024-09-11T21:55:18.026229+0000","last_became_peered":"2024-09-11T21:55:18.026229+0000","last_unstale":"2024-09-11T21:57:14.298769+0000","last_undegraded":"2024-09-11T21:57:14.298769+0000","last_fullsized":"2024-09-11T21:57:14.298769+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:50:35.745789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":206,"num_read_kb":206,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"86'10","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788388+0000","last_change":"2024-09-11T21:55:20.095884+0000","last_active":"2024-09-11T21:57:13.788388+0000","last_peered":"2024-09-11T21:57:13.788388+0000","last_clean":"2024-09-11T21:57:13.788388+0000","last_became_active":"2024-09-11T21:55:20.094887+0000","last_became_peered":"2024-09-11T21:55:20.094887+0000","last_unstale":"2024-09-11T21:57:13.788388+0000","last_undegraded":"2024-09-11T21:57:13.788388+0000","last_fullsized":"2024-09-11T21:57:13.788388+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:45:03.579703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"84'4","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671167+0000","last_change":"2024-09-11T21:55:18.030474+0000","last_active":"2024-09-11T21:57:13.671167+0000","last_peered":"2024-09-11T21:57:13.671167+0000","last_clean":"2024-09-11T21:57:13.671167+0000","last_became_active":"2024-09-11T21:55:18.030104+0000","last_became_peered":"2024-09-11T21:55:18.030104+0000","last_unstale":"2024-09-11T21:57:13.671167+0000","last_undegraded":"2024-09-11T21:57:13.671167+0000","last_fullsized":"2024-09-11T21:57:13.671167+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:21:00.484595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.669795+0000","last_change":"2024-09-11T21:55:20.102807+0000","last_active":"2024-09-11T21:57:13.669795+0000","last_peered":"2024-09-11T21:57:13.669795+0000","last_clean":"2024-09-11T21:57:13.669795+0000","last_became_active":"2024-09-11T21:55:20.102581+0000","last_became_peered":"2024-09-11T21:55:20.102581+0000","last_unstale":"2024-09-11T21:57:13.669795+0000","last_undegraded":"2024-09-11T21:57:13.669795+0000","last_fullsized":"2024-09-11T21:57:13.669795+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:29:23.139788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"84'5","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.762200+0000","last_change":"2024-09-11T21:55:18.117685+0000","last_active":"2024-09-11T21:57:13.762200+0000","last_peered":"2024-09-11T21:57:13.762200+0000","last_clean":"2024-09-11T21:57:13.762200+0000","last_became_active":"2024-09-11T21:55:18.117261+0000","last_became_peered":"2024-09-11T21:55:18.117261+0000","last_unstale":"2024-09-11T21:57:13.762200+0000","last_undegraded":"2024-09-11T21:57:13.762200+0000","last_fullsized":"2024-09-11T21:57:13.762200+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:42:13.180725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"86'11","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788081+0000","last_change":"2024-09-11T21:55:20.086562+0000","last_active":"2024-09-11T21:57:13.788081+0000","last_peered":"2024-09-11T21:57:13.788081+0000","last_clean":"2024-09-11T21:57:13.788081+0000","last_became_active":"2024-09-11T21:55:20.086197+0000","last_became_peered":"2024-09-11T21:55:20.086197+0000","last_unstale":"2024-09-11T21:57:13.788081+0000","last_undegraded":"2024-09-11T21:57:13.788081+0000","last_fullsized":"2024-09-11T21:57:13.788081+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:23:44.705132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'35","reported_seq":296,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:14.358002+0000","last_change":"2024-09-11T21:56:26.299386+0000","last_active":"2024-09-11T21:57:14.358002+0000","last_peered":"2024-09-11T21:57:14.358002+0000","last_clean":"2024-09-11T21:57:14.358002+0000","last_became_active":"2024-09-11T21:56:26.291951+0000","last_became_peered":"2024-09-11T21:56:26.291951+0000","last_unstale":"2024-09-11T21:57:14.358002+0000","last_undegraded":"2024-09-11T21:57:14.358002+0000","last_fullsized":"2024-09-11T21:57:14.358002+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:54:47.346798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65927,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":209,"num_read_kb":169,"num_write":37,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":59211,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"86'6","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671198+0000","last_change":"2024-09-11T21:55:20.091695+0000","last_active":"2024-09-11T21:57:13.671198+0000","last_peered":"2024-09-11T21:57:13.671198+0000","last_clean":"2024-09-11T21:57:13.671198+0000","last_became_active":"2024-09-11T21:55:20.091128+0000","last_became_peered":"2024-09-11T21:55:20.091128+0000","last_unstale":"2024-09-11T21:57:13.671198+0000","last_undegraded":"2024-09-11T21:57:13.671198+0000","last_fullsized":"2024-09-11T21:57:13.671198+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:16:20.751952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"84'9","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671024+0000","last_change":"2024-09-11T21:55:18.030827+0000","last_active":"2024-09-11T21:57:13.671024+0000","last_peered":"2024-09-11T21:57:13.671024+0000","last_clean":"2024-09-11T21:57:13.671024+0000","last_became_active":"2024-09-11T21:55:18.030354+0000","last_became_peered":"2024-09-11T21:55:18.030354+0000","last_unstale":"2024-09-11T21:57:13.671024+0000","last_undegraded":"2024-09-11T21:57:13.671024+0000","last_fullsized":"2024-09-11T21:57:13.671024+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:23:19.578820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788550+0000","last_change":"2024-09-11T21:55:20.090160+0000","last_active":"2024-09-11T21:57:13.788550+0000","last_peered":"2024-09-11T21:57:13.788550+0000","last_clean":"2024-09-11T21:57:13.788550+0000","last_became_active":"2024-09-11T21:55:20.089801+0000","last_became_peered":"2024-09-11T21:55:20.089801+0000","last_unstale":"2024-09-11T21:57:13.788550+0000","last_undegraded":"2024-09-11T21:57:13.788550+0000","last_fullsized":"2024-09-11T21:57:13.788550+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:34:16.093123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"84'5","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.761998+0000","last_change":"2024-09-11T21:55:18.026817+0000","last_active":"2024-09-11T21:57:13.761998+0000","last_peered":"2024-09-11T21:57:13.761998+0000","last_clean":"2024-09-11T21:57:13.761998+0000","last_became_active":"2024-09-11T21:55:18.026188+0000","last_became_peered":"2024-09-11T21:55:18.026188+0000","last_unstale":"2024-09-11T21:57:13.761998+0000","last_undegraded":"2024-09-11T21:57:13.761998+0000","last_fullsized":"2024-09-11T21:57:13.761998+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:55:14.774503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"86'15","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.670991+0000","last_change":"2024-09-11T21:55:20.095613+0000","last_active":"2024-09-11T21:57:13.670991+0000","last_peered":"2024-09-11T21:57:13.670991+0000","last_clean":"2024-09-11T21:57:13.670991+0000","last_became_active":"2024-09-11T21:55:20.095302+0000","last_became_peered":"2024-09-11T21:55:20.095302+0000","last_unstale":"2024-09-11T21:57:13.670991+0000","last_undegraded":"2024-09-11T21:57:13.670991+0000","last_fullsized":"2024-09-11T21:57:13.670991+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:37:27.961213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'7","reported_seq":138,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:14.300345+0000","last_change":"2024-09-11T21:55:18.032417+0000","last_active":"2024-09-11T21:57:14.300345+0000","last_peered":"2024-09-11T21:57:14.300345+0000","last_clean":"2024-09-11T21:57:14.300345+0000","last_became_active":"2024-09-11T21:55:18.031419+0000","last_became_peered":"2024-09-11T21:55:18.031419+0000","last_unstale":"2024-09-11T21:57:14.300345+0000","last_undegraded":"2024-09-11T21:57:14.300345+0000","last_fullsized":"2024-09-11T21:57:14.300345+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:30:52.285678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1289,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":108,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"86'6","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.670730+0000","last_change":"2024-09-11T21:55:20.101965+0000","last_active":"2024-09-11T21:57:13.670730+0000","last_peered":"2024-09-11T21:57:13.670730+0000","last_clean":"2024-09-11T21:57:13.670730+0000","last_became_active":"2024-09-11T21:55:20.101534+0000","last_became_peered":"2024-09-11T21:55:20.101534+0000","last_unstale":"2024-09-11T21:57:13.670730+0000","last_undegraded":"2024-09-11T21:57:13.670730+0000","last_fullsized":"2024-09-11T21:57:13.670730+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:41:30.685121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"84'17","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.670811+0000","last_change":"2024-09-11T21:55:18.116059+0000","last_active":"2024-09-11T21:57:13.670811+0000","last_peered":"2024-09-11T21:57:13.670811+0000","last_clean":"2024-09-11T21:57:13.670811+0000","last_became_active":"2024-09-11T21:55:18.115621+0000","last_became_peered":"2024-09-11T21:55:18.115621+0000","last_unstale":"2024-09-11T21:57:13.670811+0000","last_undegraded":"2024-09-11T21:57:13.670811+0000","last_fullsized":"2024-09-11T21:57:13.670811+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:47:17.815695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"86'6","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.762037+0000","last_change":"2024-09-11T21:55:20.100903+0000","last_active":"2024-09-11T21:57:13.762037+0000","last_peered":"2024-09-11T21:57:13.762037+0000","last_clean":"2024-09-11T21:57:13.762037+0000","last_became_active":"2024-09-11T21:55:20.100561+0000","last_became_peered":"2024-09-11T21:55:20.100561+0000","last_unstale":"2024-09-11T21:57:13.762037+0000","last_undegraded":"2024-09-11T21:57:13.762037+0000","last_fullsized":"2024-09-11T21:57:13.762037+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:00:45.472238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'2","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671632+0000","last_change":"2024-09-11T21:55:18.029549+0000","last_active":"2024-09-11T21:57:13.671632+0000","last_peered":"2024-09-11T21:57:13.671632+0000","last_clean":"2024-09-11T21:57:13.671632+0000","last_became_active":"2024-09-11T21:55:18.029178+0000","last_became_peered":"2024-09-11T21:55:18.029178+0000","last_unstale":"2024-09-11T21:57:13.671632+0000","last_undegraded":"2024-09-11T21:57:13.671632+0000","last_fullsized":"2024-09-11T21:57:13.671632+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:37:07.915786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"86'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788144+0000","last_change":"2024-09-11T21:55:20.104202+0000","last_active":"2024-09-11T21:57:13.788144+0000","last_peered":"2024-09-11T21:57:13.788144+0000","last_clean":"2024-09-11T21:57:13.788144+0000","last_became_active":"2024-09-11T21:55:20.103464+0000","last_became_peered":"2024-09-11T21:55:20.103464+0000","last_unstale":"2024-09-11T21:57:13.788144+0000","last_undegraded":"2024-09-11T21:57:13.788144+0000","last_fullsized":"2024-09-11T21:57:13.788144+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:30:16.044143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"84'5","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788728+0000","last_change":"2024-09-11T21:55:18.026183+0000","last_active":"2024-09-11T21:57:13.788728+0000","last_peered":"2024-09-11T21:57:13.788728+0000","last_clean":"2024-09-11T21:57:13.788728+0000","last_became_active":"2024-09-11T21:55:18.025849+0000","last_became_peered":"2024-09-11T21:55:18.025849+0000","last_unstale":"2024-09-11T21:57:13.788728+0000","last_undegraded":"2024-09-11T21:57:13.788728+0000","last_fullsized":"2024-09-11T21:57:13.788728+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:21:12.189158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671103+0000","last_change":"2024-09-11T21:55:20.090652+0000","last_active":"2024-09-11T21:57:13.671103+0000","last_peered":"2024-09-11T21:57:13.671103+0000","last_clean":"2024-09-11T21:57:13.671103+0000","last_became_active":"2024-09-11T21:55:20.090348+0000","last_became_peered":"2024-09-11T21:55:20.090348+0000","last_unstale":"2024-09-11T21:57:13.671103+0000","last_undegraded":"2024-09-11T21:57:13.671103+0000","last_fullsized":"2024-09-11T21:57:13.671103+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:10:58.733371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'41","reported_seq":73,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.873064+0000","last_change":"2024-09-11T21:55:18.024388+0000","last_active":"2024-09-11T21:57:13.873064+0000","last_peered":"2024-09-11T21:57:13.873064+0000","last_clean":"2024-09-11T21:57:13.873064+0000","last_became_active":"2024-09-11T21:55:18.024100+0000","last_became_peered":"2024-09-11T21:55:18.024100+0000","last_unstale":"2024-09-11T21:57:13.873064+0000","last_undegraded":"2024-09-11T21:57:13.873064+0000","last_fullsized":"2024-09-11T21:57:13.873064+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:21:11.152282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.670936+0000","last_change":"2024-09-11T21:55:20.101007+0000","last_active":"2024-09-11T21:57:13.670936+0000","last_peered":"2024-09-11T21:57:13.670936+0000","last_clean":"2024-09-11T21:57:13.670936+0000","last_became_active":"2024-09-11T21:55:20.100642+0000","last_became_peered":"2024-09-11T21:55:20.100642+0000","last_unstale":"2024-09-11T21:57:13.670936+0000","last_undegraded":"2024-09-11T21:57:13.670936+0000","last_fullsized":"2024-09-11T21:57:13.670936+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:16:05.642184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"87'8","reported_seq":140,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:14.325956+0000","last_change":"2024-09-11T21:55:18.116216+0000","last_active":"2024-09-11T21:57:14.325956+0000","last_peered":"2024-09-11T21:57:14.325956+0000","last_clean":"2024-09-11T21:57:14.325956+0000","last_became_active":"2024-09-11T21:55:18.115871+0000","last_became_peered":"2024-09-11T21:55:18.115871+0000","last_unstale":"2024-09-11T21:57:14.325956+0000","last_undegraded":"2024-09-11T21:57:14.325956+0000","last_fullsized":"2024-09-11T21:57:14.325956+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:16:56.027636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":108,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671175+0000","last_change":"2024-09-11T21:55:20.092421+0000","last_active":"2024-09-11T21:57:13.671175+0000","last_peered":"2024-09-11T21:57:13.671175+0000","last_clean":"2024-09-11T21:57:13.671175+0000","last_became_active":"2024-09-11T21:55:20.091707+0000","last_became_peered":"2024-09-11T21:55:20.091707+0000","last_unstale":"2024-09-11T21:57:13.671175+0000","last_undegraded":"2024-09-11T21:57:13.671175+0000","last_fullsized":"2024-09-11T21:57:13.671175+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:12:38.157687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":353,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.670940+0000","last_change":"2024-09-11T21:54:07.542362+0000","last_active":"2024-09-11T21:57:13.670940+0000","last_peered":"2024-09-11T21:57:13.670940+0000","last_clean":"2024-09-11T21:57:13.670940+0000","last_became_active":"2024-09-11T21:54:07.541871+0000","last_became_peered":"2024-09-11T21:54:07.541871+0000","last_unstale":"2024-09-11T21:57:13.670940+0000","last_undegraded":"2024-09-11T21:57:13.670940+0000","last_fullsized":"2024-09-11T21:57:13.670940+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:50:41.026895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:50:41.026895+0000","last_clean_scrub_stamp":"2024-09-11T21:50:41.026895+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:05:06.154268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"84'6","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788499+0000","last_change":"2024-09-11T21:55:18.030558+0000","last_active":"2024-09-11T21:57:13.788499+0000","last_peered":"2024-09-11T21:57:13.788499+0000","last_clean":"2024-09-11T21:57:13.788499+0000","last_became_active":"2024-09-11T21:55:18.030216+0000","last_became_peered":"2024-09-11T21:55:18.030216+0000","last_unstale":"2024-09-11T21:57:13.788499+0000","last_undegraded":"2024-09-11T21:57:13.788499+0000","last_fullsized":"2024-09-11T21:57:13.788499+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:33:02.589999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.753426+0000","last_change":"2024-09-11T21:55:20.092938+0000","last_active":"2024-09-11T21:57:13.753426+0000","last_peered":"2024-09-11T21:57:13.753426+0000","last_clean":"2024-09-11T21:57:13.753426+0000","last_became_active":"2024-09-11T21:55:20.092615+0000","last_became_peered":"2024-09-11T21:55:20.092615+0000","last_unstale":"2024-09-11T21:57:13.753426+0000","last_undegraded":"2024-09-11T21:57:13.753426+0000","last_fullsized":"2024-09-11T21:57:13.753426+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:40:06.957836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"84'4","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.670838+0000","last_change":"2024-09-11T21:55:18.029127+0000","last_active":"2024-09-11T21:57:13.670838+0000","last_peered":"2024-09-11T21:57:13.670838+0000","last_clean":"2024-09-11T21:57:13.670838+0000","last_became_active":"2024-09-11T21:55:18.028740+0000","last_became_peered":"2024-09-11T21:55:18.028740+0000","last_unstale":"2024-09-11T21:57:13.670838+0000","last_undegraded":"2024-09-11T21:57:13.670838+0000","last_fullsized":"2024-09-11T21:57:13.670838+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:14:01.128486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"86'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788303+0000","last_change":"2024-09-11T21:55:20.092498+0000","last_active":"2024-09-11T21:57:13.788303+0000","last_peered":"2024-09-11T21:57:13.788303+0000","last_clean":"2024-09-11T21:57:13.788303+0000","last_became_active":"2024-09-11T21:55:20.092179+0000","last_became_peered":"2024-09-11T21:55:20.092179+0000","last_unstale":"2024-09-11T21:57:13.788303+0000","last_undegraded":"2024-09-11T21:57:13.788303+0000","last_fullsized":"2024-09-11T21:57:13.788303+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:48:58.536942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"86'6","reported_seq":134,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788554+0000","last_change":"2024-09-11T21:55:18.115798+0000","last_active":"2024-09-11T21:57:13.788554+0000","last_peered":"2024-09-11T21:57:13.788554+0000","last_clean":"2024-09-11T21:57:13.788554+0000","last_became_active":"2024-09-11T21:55:18.115433+0000","last_became_peered":"2024-09-11T21:55:18.115433+0000","last_unstale":"2024-09-11T21:57:13.788554+0000","last_undegraded":"2024-09-11T21:57:13.788554+0000","last_fullsized":"2024-09-11T21:57:13.788554+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:03:32.097756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":104,"num_read_kb":104,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.762043+0000","last_change":"2024-09-11T21:55:20.100439+0000","last_active":"2024-09-11T21:57:13.762043+0000","last_peered":"2024-09-11T21:57:13.762043+0000","last_clean":"2024-09-11T21:57:13.762043+0000","last_became_active":"2024-09-11T21:55:20.098717+0000","last_became_peered":"2024-09-11T21:55:20.098717+0000","last_unstale":"2024-09-11T21:57:13.762043+0000","last_undegraded":"2024-09-11T21:57:13.762043+0000","last_fullsized":"2024-09-11T21:57:13.762043+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:15:10.874225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"87'9","reported_seq":140,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:14.298986+0000","last_change":"2024-09-11T21:55:18.026588+0000","last_active":"2024-09-11T21:57:14.298986+0000","last_peered":"2024-09-11T21:57:14.298986+0000","last_clean":"2024-09-11T21:57:14.298986+0000","last_became_active":"2024-09-11T21:55:18.026205+0000","last_became_peered":"2024-09-11T21:55:18.026205+0000","last_unstale":"2024-09-11T21:57:14.298986+0000","last_undegraded":"2024-09-11T21:57:14.298986+0000","last_fullsized":"2024-09-11T21:57:14.298986+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:45:34.701088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1908,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":107,"num_read_kb":4,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.765120+0000","last_change":"2024-09-11T21:55:20.100677+0000","last_active":"2024-09-11T21:57:13.765120+0000","last_peered":"2024-09-11T21:57:13.765120+0000","last_clean":"2024-09-11T21:57:13.765120+0000","last_became_active":"2024-09-11T21:55:20.100334+0000","last_became_peered":"2024-09-11T21:55:20.100334+0000","last_unstale":"2024-09-11T21:57:13.765120+0000","last_undegraded":"2024-09-11T21:57:13.765120+0000","last_fullsized":"2024-09-11T21:57:13.765120+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:15:16.792831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"84'1","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.751199+0000","last_change":"2024-09-11T21:55:18.116329+0000","last_active":"2024-09-11T21:57:13.751199+0000","last_peered":"2024-09-11T21:57:13.751199+0000","last_clean":"2024-09-11T21:57:13.751199+0000","last_became_active":"2024-09-11T21:55:18.115944+0000","last_became_peered":"2024-09-11T21:55:18.115944+0000","last_unstale":"2024-09-11T21:57:13.751199+0000","last_undegraded":"2024-09-11T21:57:13.751199+0000","last_fullsized":"2024-09-11T21:57:13.751199+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:03:04.164985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.765132+0000","last_change":"2024-09-11T21:55:20.089202+0000","last_active":"2024-09-11T21:57:13.765132+0000","last_peered":"2024-09-11T21:57:13.765132+0000","last_clean":"2024-09-11T21:57:13.765132+0000","last_became_active":"2024-09-11T21:55:20.088853+0000","last_became_peered":"2024-09-11T21:55:20.088853+0000","last_unstale":"2024-09-11T21:57:13.765132+0000","last_undegraded":"2024-09-11T21:57:13.765132+0000","last_fullsized":"2024-09-11T21:57:13.765132+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:39:36.496944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"84'4","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671429+0000","last_change":"2024-09-11T21:55:18.117917+0000","last_active":"2024-09-11T21:57:13.671429+0000","last_peered":"2024-09-11T21:57:13.671429+0000","last_clean":"2024-09-11T21:57:13.671429+0000","last_became_active":"2024-09-11T21:55:18.117567+0000","last_became_peered":"2024-09-11T21:55:18.117567+0000","last_unstale":"2024-09-11T21:57:13.671429+0000","last_undegraded":"2024-09-11T21:57:13.671429+0000","last_fullsized":"2024-09-11T21:57:13.671429+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:21:32.546620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.789090+0000","last_change":"2024-09-11T21:55:20.092370+0000","last_active":"2024-09-11T21:57:13.789090+0000","last_peered":"2024-09-11T21:57:13.789090+0000","last_clean":"2024-09-11T21:57:13.789090+0000","last_became_active":"2024-09-11T21:55:20.091982+0000","last_became_peered":"2024-09-11T21:55:20.091982+0000","last_unstale":"2024-09-11T21:57:13.789090+0000","last_undegraded":"2024-09-11T21:57:13.789090+0000","last_fullsized":"2024-09-11T21:57:13.789090+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:59:55.214511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'51","reported_seq":98,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.879119+0000","last_change":"2024-09-11T21:55:18.023166+0000","last_active":"2024-09-11T21:57:13.879119+0000","last_peered":"2024-09-11T21:57:13.879119+0000","last_clean":"2024-09-11T21:57:13.879119+0000","last_became_active":"2024-09-11T21:55:18.022845+0000","last_became_peered":"2024-09-11T21:55:18.022845+0000","last_unstale":"2024-09-11T21:57:13.879119+0000","last_undegraded":"2024-09-11T21:57:13.879119+0000","last_fullsized":"2024-09-11T21:57:13.879119+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:08:06.770410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":91,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"86'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.753738+0000","last_change":"2024-09-11T21:55:20.091541+0000","last_active":"2024-09-11T21:57:13.753738+0000","last_peered":"2024-09-11T21:57:13.753738+0000","last_clean":"2024-09-11T21:57:13.753738+0000","last_became_active":"2024-09-11T21:55:20.090971+0000","last_became_peered":"2024-09-11T21:55:20.090971+0000","last_unstale":"2024-09-11T21:57:13.753738+0000","last_undegraded":"2024-09-11T21:57:13.753738+0000","last_fullsized":"2024-09-11T21:57:13.753738+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:22:31.974711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"84'7","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788398+0000","last_change":"2024-09-11T21:55:18.030694+0000","last_active":"2024-09-11T21:57:13.788398+0000","last_peered":"2024-09-11T21:57:13.788398+0000","last_clean":"2024-09-11T21:57:13.788398+0000","last_became_active":"2024-09-11T21:55:18.029687+0000","last_became_peered":"2024-09-11T21:55:18.029687+0000","last_unstale":"2024-09-11T21:57:13.788398+0000","last_undegraded":"2024-09-11T21:57:13.788398+0000","last_fullsized":"2024-09-11T21:57:13.788398+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:18:15.009543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"86'7","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.669625+0000","last_change":"2024-09-11T21:56:26.376104+0000","last_active":"2024-09-11T21:57:13.669625+0000","last_peered":"2024-09-11T21:57:13.669625+0000","last_clean":"2024-09-11T21:57:13.669625+0000","last_became_active":"2024-09-11T21:56:26.375665+0000","last_became_peered":"2024-09-11T21:56:26.375665+0000","last_unstale":"2024-09-11T21:57:13.669625+0000","last_undegraded":"2024-09-11T21:57:13.669625+0000","last_fullsized":"2024-09-11T21:57:13.669625+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:19:46.616205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"84'2","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788029+0000","last_change":"2024-09-11T21:55:18.115977+0000","last_active":"2024-09-11T21:57:13.788029+0000","last_peered":"2024-09-11T21:57:13.788029+0000","last_clean":"2024-09-11T21:57:13.788029+0000","last_became_active":"2024-09-11T21:55:18.115649+0000","last_became_peered":"2024-09-11T21:55:18.115649+0000","last_unstale":"2024-09-11T21:57:13.788029+0000","last_undegraded":"2024-09-11T21:57:13.788029+0000","last_fullsized":"2024-09-11T21:57:13.788029+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:42:07.621024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"86'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.670884+0000","last_change":"2024-09-11T21:55:20.095536+0000","last_active":"2024-09-11T21:57:13.670884+0000","last_peered":"2024-09-11T21:57:13.670884+0000","last_clean":"2024-09-11T21:57:13.670884+0000","last_became_active":"2024-09-11T21:55:20.095117+0000","last_became_peered":"2024-09-11T21:55:20.095117+0000","last_unstale":"2024-09-11T21:57:13.670884+0000","last_undegraded":"2024-09-11T21:57:13.670884+0000","last_fullsized":"2024-09-11T21:57:13.670884+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:46.524922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"84'7","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.753348+0000","last_change":"2024-09-11T21:55:18.026592+0000","last_active":"2024-09-11T21:57:13.753348+0000","last_peered":"2024-09-11T21:57:13.753348+0000","last_clean":"2024-09-11T21:57:13.753348+0000","last_became_active":"2024-09-11T21:55:18.026263+0000","last_became_peered":"2024-09-11T21:55:18.026263+0000","last_unstale":"2024-09-11T21:57:13.753348+0000","last_undegraded":"2024-09-11T21:57:13.753348+0000","last_fullsized":"2024-09-11T21:57:13.753348+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:22:29.382964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788053+0000","last_change":"2024-09-11T21:55:20.093655+0000","last_active":"2024-09-11T21:57:13.788053+0000","last_peered":"2024-09-11T21:57:13.788053+0000","last_clean":"2024-09-11T21:57:13.788053+0000","last_became_active":"2024-09-11T21:55:20.093287+0000","last_became_peered":"2024-09-11T21:55:20.093287+0000","last_unstale":"2024-09-11T21:57:13.788053+0000","last_undegraded":"2024-09-11T21:57:13.788053+0000","last_fullsized":"2024-09-11T21:57:13.788053+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:46:15.912710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"84'3","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.670879+0000","last_change":"2024-09-11T21:55:18.028465+0000","last_active":"2024-09-11T21:57:13.670879+0000","last_peered":"2024-09-11T21:57:13.670879+0000","last_clean":"2024-09-11T21:57:13.670879+0000","last_became_active":"2024-09-11T21:55:18.028139+0000","last_became_peered":"2024-09-11T21:55:18.028139+0000","last_unstale":"2024-09-11T21:57:13.670879+0000","last_undegraded":"2024-09-11T21:57:13.670879+0000","last_fullsized":"2024-09-11T21:57:13.670879+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:52:47.808058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.753473+0000","last_change":"2024-09-11T21:55:20.096400+0000","last_active":"2024-09-11T21:57:13.753473+0000","last_peered":"2024-09-11T21:57:13.753473+0000","last_clean":"2024-09-11T21:57:13.753473+0000","last_became_active":"2024-09-11T21:55:20.096023+0000","last_became_peered":"2024-09-11T21:55:20.096023+0000","last_unstale":"2024-09-11T21:57:13.753473+0000","last_undegraded":"2024-09-11T21:57:13.753473+0000","last_fullsized":"2024-09-11T21:57:13.753473+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:19:34.150687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"84'4","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671181+0000","last_change":"2024-09-11T21:55:18.028759+0000","last_active":"2024-09-11T21:57:13.671181+0000","last_peered":"2024-09-11T21:57:13.671181+0000","last_clean":"2024-09-11T21:57:13.671181+0000","last_became_active":"2024-09-11T21:55:18.028366+0000","last_became_peered":"2024-09-11T21:55:18.028366+0000","last_unstale":"2024-09-11T21:57:13.671181+0000","last_undegraded":"2024-09-11T21:57:13.671181+0000","last_fullsized":"2024-09-11T21:57:13.671181+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:42:05.836114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"86'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671002+0000","last_change":"2024-09-11T21:55:20.093593+0000","last_active":"2024-09-11T21:57:13.671002+0000","last_peered":"2024-09-11T21:57:13.671002+0000","last_clean":"2024-09-11T21:57:13.671002+0000","last_became_active":"2024-09-11T21:55:20.093189+0000","last_became_peered":"2024-09-11T21:55:20.093189+0000","last_unstale":"2024-09-11T21:57:13.671002+0000","last_undegraded":"2024-09-11T21:57:13.671002+0000","last_fullsized":"2024-09-11T21:57:13.671002+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:47:36.802816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"86'15","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.762163+0000","last_change":"2024-09-11T21:56:26.292793+0000","last_active":"2024-09-11T21:57:13.762163+0000","last_peered":"2024-09-11T21:57:13.762163+0000","last_clean":"2024-09-11T21:57:13.762163+0000","last_became_active":"2024-09-11T21:56:26.292437+0000","last_became_peered":"2024-09-11T21:56:26.292437+0000","last_unstale":"2024-09-11T21:57:13.762163+0000","last_undegraded":"2024-09-11T21:57:13.762163+0000","last_fullsized":"2024-09-11T21:57:13.762163+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:49:08.867435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"84'5","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671366+0000","last_change":"2024-09-11T21:55:18.027365+0000","last_active":"2024-09-11T21:57:13.671366+0000","last_peered":"2024-09-11T21:57:13.671366+0000","last_clean":"2024-09-11T21:57:13.671366+0000","last_became_active":"2024-09-11T21:55:18.027058+0000","last_became_peered":"2024-09-11T21:55:18.027058+0000","last_unstale":"2024-09-11T21:57:13.671366+0000","last_undegraded":"2024-09-11T21:57:13.671366+0000","last_fullsized":"2024-09-11T21:57:13.671366+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:30:50.613246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788034+0000","last_change":"2024-09-11T21:55:20.095154+0000","last_active":"2024-09-11T21:57:13.788034+0000","last_peered":"2024-09-11T21:57:13.788034+0000","last_clean":"2024-09-11T21:57:13.788034+0000","last_became_active":"2024-09-11T21:55:20.093607+0000","last_became_peered":"2024-09-11T21:55:20.093607+0000","last_unstale":"2024-09-11T21:57:13.788034+0000","last_undegraded":"2024-09-11T21:57:13.788034+0000","last_fullsized":"2024-09-11T21:57:13.788034+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:27:53.612467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"87'13","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671385+0000","last_change":"2024-09-11T21:55:18.029872+0000","last_active":"2024-09-11T21:57:13.671385+0000","last_peered":"2024-09-11T21:57:13.671385+0000","last_clean":"2024-09-11T21:57:13.671385+0000","last_became_active":"2024-09-11T21:55:18.029538+0000","last_became_peered":"2024-09-11T21:55:18.029538+0000","last_unstale":"2024-09-11T21:57:13.671385+0000","last_undegraded":"2024-09-11T21:57:13.671385+0000","last_fullsized":"2024-09-11T21:57:13.671385+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:05:16.824805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"86'6","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.669783+0000","last_change":"2024-09-11T21:55:20.098713+0000","last_active":"2024-09-11T21:57:13.669783+0000","last_peered":"2024-09-11T21:57:13.669783+0000","last_clean":"2024-09-11T21:57:13.669783+0000","last_became_active":"2024-09-11T21:55:20.098219+0000","last_became_peered":"2024-09-11T21:55:20.098219+0000","last_unstale":"2024-09-11T21:57:13.669783+0000","last_undegraded":"2024-09-11T21:57:13.669783+0000","last_fullsized":"2024-09-11T21:57:13.669783+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:07:10.585366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"84'2","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.753854+0000","last_change":"2024-09-11T21:55:18.115868+0000","last_active":"2024-09-11T21:57:13.753854+0000","last_peered":"2024-09-11T21:57:13.753854+0000","last_clean":"2024-09-11T21:57:13.753854+0000","last_became_active":"2024-09-11T21:55:18.115486+0000","last_became_peered":"2024-09-11T21:55:18.115486+0000","last_unstale":"2024-09-11T21:57:13.753854+0000","last_undegraded":"2024-09-11T21:57:13.753854+0000","last_fullsized":"2024-09-11T21:57:13.753854+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:16:57.932041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"86'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.753686+0000","last_change":"2024-09-11T21:55:20.096077+0000","last_active":"2024-09-11T21:57:13.753686+0000","last_peered":"2024-09-11T21:57:13.753686+0000","last_clean":"2024-09-11T21:57:13.753686+0000","last_became_active":"2024-09-11T21:55:20.095739+0000","last_became_peered":"2024-09-11T21:55:20.095739+0000","last_unstale":"2024-09-11T21:57:13.753686+0000","last_undegraded":"2024-09-11T21:57:13.753686+0000","last_fullsized":"2024-09-11T21:57:13.753686+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:05:28.089828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"87'15","reported_seq":143,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.931982+0000","last_change":"2024-09-11T21:55:18.025771+0000","last_active":"2024-09-11T21:57:13.931982+0000","last_peered":"2024-09-11T21:57:13.931982+0000","last_clean":"2024-09-11T21:57:13.931982+0000","last_became_active":"2024-09-11T21:55:18.025384+0000","last_became_peered":"2024-09-11T21:55:18.025384+0000","last_unstale":"2024-09-11T21:57:13.931982+0000","last_undegraded":"2024-09-11T21:57:13.931982+0000","last_fullsized":"2024-09-11T21:57:13.931982+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:58:21.070061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":104,"num_read_kb":104,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":601040,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1166,"num_read_kb":701,"num_write":380,"num_write_kb":1470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":59211,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":667648,"data_stored":1803120,"data_compressed":85492,"data_compressed_allocated":110592,"data_compressed_original":1605632,"omap_allocated":0,"internal_metadata":0},"log_size":306,"ondisk_log_size":306,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107988,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449124,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":937984,"data_stored":453556,"data_compressed":5771,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678056,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36448,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708704,"statfs":{"total":95995035648,"available":95957712896,"internally_reserved":0,"allocated":1130496,"data_stored":1251264,"data_compressed":50827,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":66,"seq":283467841582,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":974848,"data_stored":461937,"data_compressed":8675,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103220,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":995328,"data_stored":855531,"data_compressed":27821,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332152,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1024000,"data_stored":1057390,"data_compressed":44671,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495677,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":958464,"data_stored":462974,"data_compressed":8675,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757315,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":1186647,"data_compressed":50827,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920839,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":983040,"data_stored":521591,"data_compressed":18261,"data_compressed_allocated":24576,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182477,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":986295,"data_compressed":33977,"data_compressed_allocated":45056,"data_compressed_original":540672,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":970752,"data_stored":464641,"data_compressed":8675,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1024000,"data_stored":529814,"data_compressed":8675,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":978944,"data_stored":463920,"data_compressed":8675,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":68999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":67328,"data_compressed":9586,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":135347,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":6090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":394716,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":199964,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T21:57:17.014 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T21:57:17.048 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:16 smithi053 ceph-mon[34908]: [11/Sep/2024:21:57:15] ENGINE Error in HTTPServer.serve 2024-09-11T21:57:17.048 INFO:journalctl@ceph.mon.b.smithi053.stdout: Traceback (most recent call last): 2024-09-11T21:57:17.048 INFO:journalctl@ceph.mon.b.smithi053.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T21:57:17.048 INFO:journalctl@ceph.mon.b.smithi053.stdout: self._connections.run(self.expiration_interval) 2024-09-11T21:57:17.048 INFO:journalctl@ceph.mon.b.smithi053.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T21:57:17.048 INFO:journalctl@ceph.mon.b.smithi053.stdout: self._run(expiration_interval) 2024-09-11T21:57:17.048 INFO:journalctl@ceph.mon.b.smithi053.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T21:57:17.048 INFO:journalctl@ceph.mon.b.smithi053.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T21:57:17.049 INFO:journalctl@ceph.mon.b.smithi053.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T21:57:17.049 INFO:journalctl@ceph.mon.b.smithi053.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T21:57:17.049 INFO:journalctl@ceph.mon.b.smithi053.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T21:57:17.049 INFO:journalctl@ceph.mon.b.smithi053.stdout: s = self.context.wrap_socket( 2024-09-11T21:57:17.049 INFO:journalctl@ceph.mon.b.smithi053.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T21:57:17.049 INFO:journalctl@ceph.mon.b.smithi053.stdout: return self.sslsocket_class._create( 2024-09-11T21:57:17.049 INFO:journalctl@ceph.mon.b.smithi053.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T21:57:17.049 INFO:journalctl@ceph.mon.b.smithi053.stdout: self.do_handshake() 2024-09-11T21:57:17.049 INFO:journalctl@ceph.mon.b.smithi053.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T21:57:17.049 INFO:journalctl@ceph.mon.b.smithi053.stdout: self._sslobj.do_handshake() 2024-09-11T21:57:17.049 INFO:journalctl@ceph.mon.b.smithi053.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T21:57:17.049 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:16 smithi053 ceph-mon[34908]: [11/Sep/2024:21:57:15] ENGINE Serving on http://172.21.15.1:8765 2024-09-11T21:57:17.049 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:16 smithi053 ceph-mon[34908]: [11/Sep/2024:21:57:15] ENGINE Bus STARTED 2024-09-11T21:57:17.050 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:16 smithi053 ceph-mon[34908]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:57:17.050 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:16 smithi053 ceph-mon[34908]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-09-11T21:57:17.050 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:16 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:57:17.050 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:16 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:57:17.050 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:16 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:57:17.050 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:16 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:57:17.050 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:16 smithi053 ceph-mon[34908]: from='client.15297 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T21:57:17.050 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:16 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:57:17.050 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:16 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:57:17.050 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:16 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:17.050 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:16 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:17.050 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:16 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:17.050 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:16 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:17.051 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:16 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:17.051 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:16 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:17.051 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:16 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:17.051 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:16 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:17.051 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:16 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:57:17.051 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:16 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:57:17.051 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:16 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:17.051 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:16 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:17.051 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:16 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:17.051 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:16 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:17.051 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:16 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:17.051 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:16 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:17.051 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:16 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:17.052 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:16 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:17.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:16 smithi161 ceph-mon[35615]: [11/Sep/2024:21:57:15] ENGINE Error in HTTPServer.serve 2024-09-11T21:57:17.092 INFO:journalctl@ceph.mon.c.smithi161.stdout: Traceback (most recent call last): 2024-09-11T21:57:17.092 INFO:journalctl@ceph.mon.c.smithi161.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T21:57:17.092 INFO:journalctl@ceph.mon.c.smithi161.stdout: self._connections.run(self.expiration_interval) 2024-09-11T21:57:17.092 INFO:journalctl@ceph.mon.c.smithi161.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T21:57:17.092 INFO:journalctl@ceph.mon.c.smithi161.stdout: self._run(expiration_interval) 2024-09-11T21:57:17.092 INFO:journalctl@ceph.mon.c.smithi161.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T21:57:17.093 INFO:journalctl@ceph.mon.c.smithi161.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T21:57:17.093 INFO:journalctl@ceph.mon.c.smithi161.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T21:57:17.093 INFO:journalctl@ceph.mon.c.smithi161.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T21:57:17.093 INFO:journalctl@ceph.mon.c.smithi161.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T21:57:17.093 INFO:journalctl@ceph.mon.c.smithi161.stdout: s = self.context.wrap_socket( 2024-09-11T21:57:17.093 INFO:journalctl@ceph.mon.c.smithi161.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T21:57:17.093 INFO:journalctl@ceph.mon.c.smithi161.stdout: return self.sslsocket_class._create( 2024-09-11T21:57:17.093 INFO:journalctl@ceph.mon.c.smithi161.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T21:57:17.093 INFO:journalctl@ceph.mon.c.smithi161.stdout: self.do_handshake() 2024-09-11T21:57:17.093 INFO:journalctl@ceph.mon.c.smithi161.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T21:57:17.094 INFO:journalctl@ceph.mon.c.smithi161.stdout: self._sslobj.do_handshake() 2024-09-11T21:57:17.094 INFO:journalctl@ceph.mon.c.smithi161.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T21:57:17.094 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:16 smithi161 ceph-mon[35615]: [11/Sep/2024:21:57:15] ENGINE Serving on http://172.21.15.1:8765 2024-09-11T21:57:17.094 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:16 smithi161 ceph-mon[35615]: [11/Sep/2024:21:57:15] ENGINE Bus STARTED 2024-09-11T21:57:17.094 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:16 smithi161 ceph-mon[35615]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:57:17.094 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:16 smithi161 ceph-mon[35615]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-09-11T21:57:17.094 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:16 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:57:17.094 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:16 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:57:17.094 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:16 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:57:17.094 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:16 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:57:17.094 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:16 smithi161 ceph-mon[35615]: from='client.15297 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T21:57:17.094 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:16 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:57:17.094 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:16 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:57:17.095 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:16 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:17.095 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:16 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:17.095 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:16 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:17.095 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:16 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:17.095 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:16 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:17.095 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:16 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:17.095 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:16 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:17.095 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:16 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:17.095 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:16 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:57:17.095 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:16 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:57:17.095 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:16 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:17.095 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:16 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:17.095 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:16 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:17.095 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:16 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:17.096 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:16 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:17.096 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:16 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:17.096 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:16 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:17.096 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:16 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:17.317 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:57:17.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:17.827+0000 7f4f5a472640 1 -- 172.21.15.1:0/2492435469 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f54101de0 msgr2=0x7f4f541021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:17.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:17.827+0000 7f4f5a472640 1 --2- 172.21.15.1:0/2492435469 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f54101de0 0x7f4f541021c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f4f44009900 tx=0x7f4f4402a410 comp rx=0 tx=0).stop 2024-09-11T21:57:17.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:17.827+0000 7f4f5a472640 1 -- 172.21.15.1:0/2492435469 shutdown_connections 2024-09-11T21:57:17.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:17.827+0000 7f4f5a472640 1 --2- 172.21.15.1:0/2492435469 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f54106f10 0x7f4f5410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:17.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:17.827+0000 7f4f5a472640 1 --2- 172.21.15.1:0/2492435469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f54102790 0x7f4f541067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:17.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:17.827+0000 7f4f5a472640 1 --2- 172.21.15.1:0/2492435469 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f54101de0 0x7f4f541021c0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:17.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:17.827+0000 7f4f5a472640 1 -- 172.21.15.1:0/2492435469 >> 172.21.15.1:0/2492435469 conn(0x7f4f540fd8e0 msgr2=0x7f4f540ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:17.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:17.827+0000 7f4f5a472640 1 -- 172.21.15.1:0/2492435469 shutdown_connections 2024-09-11T21:57:17.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:17.827+0000 7f4f5a472640 1 -- 172.21.15.1:0/2492435469 wait complete. 2024-09-11T21:57:17.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:17.828+0000 7f4f5a472640 1 Processor -- start 2024-09-11T21:57:17.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:17.828+0000 7f4f5a472640 1 -- start start 2024-09-11T21:57:17.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:17.828+0000 7f4f5a472640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f54101de0 0x7f4f540744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:17.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:17.828+0000 7f4f5a472640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f54102790 0x7f4f54074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:17.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:17.828+0000 7f4f5a472640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f54106f10 0x7f4f54074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:17.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:17.828+0000 7f4f5a472640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4f54111f70 con 0x7f4f54102790 2024-09-11T21:57:17.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:17.828+0000 7f4f5a472640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4f54111df0 con 0x7f4f54106f10 2024-09-11T21:57:17.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:17.828+0000 7f4f5a472640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4f541120f0 con 0x7f4f54101de0 2024-09-11T21:57:17.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:17.829+0000 7f4f537fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f54102790 0x7f4f54074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:17.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:17.829+0000 7f4f537fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f54102790 0x7f4f54074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38646/0 (socket says 172.21.15.1:38646) 2024-09-11T21:57:17.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:17.829+0000 7f4f537fe640 1 -- 172.21.15.1:0/2298756310 learned_addr learned my addr 172.21.15.1:0/2298756310 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:17.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:17.829+0000 7f4f53fff640 1 --2- 172.21.15.1:0/2298756310 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f54101de0 0x7f4f540744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:17.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:17.829+0000 7f4f58a0f640 1 --2- 172.21.15.1:0/2298756310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f54106f10 0x7f4f54074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:17.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:17.829+0000 7f4f537fe640 1 -- 172.21.15.1:0/2298756310 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f54101de0 msgr2=0x7f4f540744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:17.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:17.829+0000 7f4f537fe640 1 --2- 172.21.15.1:0/2298756310 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f54101de0 0x7f4f540744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:17.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:17.829+0000 7f4f537fe640 1 -- 172.21.15.1:0/2298756310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f54106f10 msgr2=0x7f4f54074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:17.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:17.829+0000 7f4f537fe640 1 --2- 172.21.15.1:0/2298756310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f54106f10 0x7f4f54074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:17.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:17.829+0000 7f4f537fe640 1 -- 172.21.15.1:0/2298756310 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f541a6970 con 0x7f4f54102790 2024-09-11T21:57:17.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:17.829+0000 7f4f537fe640 1 --2- 172.21.15.1:0/2298756310 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f54102790 0x7f4f54074a30 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f4f4800e9f0 tx=0x7f4f4800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:17.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:17.829+0000 7f4f517fa640 1 -- 172.21.15.1:0/2298756310 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4800cd80 con 0x7f4f54102790 2024-09-11T21:57:17.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:17.829+0000 7f4f5a472640 1 -- 172.21.15.1:0/2298756310 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f541a6c60 con 0x7f4f54102790 2024-09-11T21:57:17.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:17.830+0000 7f4f517fa640 1 -- 172.21.15.1:0/2298756310 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f48010910 con 0x7f4f54102790 2024-09-11T21:57:17.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:17.830+0000 7f4f517fa640 1 -- 172.21.15.1:0/2298756310 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f480188f0 con 0x7f4f54102790 2024-09-11T21:57:17.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:17.830+0000 7f4f5a472640 1 -- 172.21.15.1:0/2298756310 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f541a71a0 con 0x7f4f54102790 2024-09-11T21:57:17.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:17.831+0000 7f4f5a472640 1 -- 172.21.15.1:0/2298756310 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f541021c0 con 0x7f4f54102790 2024-09-11T21:57:17.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:17.833+0000 7f4f517fa640 1 -- 172.21.15.1:0/2298756310 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 23) ==== 137952+0+0 (secure 0 0 0) 0x7f4f48010ac0 con 0x7f4f54102790 2024-09-11T21:57:17.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:17.834+0000 7f4f517fa640 1 --2- 172.21.15.1:0/2298756310 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f4f3c0a2cf0 0x7f4f3c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:17.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:17.834+0000 7f4f53fff640 1 --2- 172.21.15.1:0/2298756310 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f4f3c0a2cf0 0x7f4f3c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:17.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:17.834+0000 7f4f517fa640 1 -- 172.21.15.1:0/2298756310 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7f4f480cf7d0 con 0x7f4f54102790 2024-09-11T21:57:17.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:17.834+0000 7f4f53fff640 1 --2- 172.21.15.1:0/2298756310 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f4f3c0a2cf0 0x7f4f3c0a51b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4f44009230 tx=0x7f4f440058d0 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:17.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:17.838+0000 7f4f517fa640 1 -- 172.21.15.1:0/2298756310 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4f4809bda0 con 0x7f4f54102790 2024-09-11T21:57:18.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:18.060+0000 7f4f5a472640 1 -- 172.21.15.1:0/2298756310 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4f5410a0f0 con 0x7f4f54102790 2024-09-11T21:57:18.061 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:17 smithi001 ceph-mon[30935]: Adjusting osd_memory_target on smithi161 to 176.0M 2024-09-11T21:57:18.061 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:17 smithi001 ceph-mon[30935]: Unable to set osd_memory_target on smithi161 to 184591769: error parsing value: Value '184591769' is below minimum 939524096 2024-09-11T21:57:18.061 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:17 smithi001 ceph-mon[30935]: Adjusting osd_memory_target on smithi053 to 177.0M 2024-09-11T21:57:18.062 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:17 smithi001 ceph-mon[30935]: Unable to set osd_memory_target on smithi053 to 185699942: error parsing value: Value '185699942' is below minimum 939524096 2024-09-11T21:57:18.062 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:17 smithi001 ceph-mon[30935]: from='client.15303 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T21:57:18.062 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:17 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:57:18.062 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:17 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:57:18.062 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:17 smithi001 ceph-mon[30935]: from='client.15309 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T21:57:18.062 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:17 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:57:18.062 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:17 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:57:18.062 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:17 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:18.062 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:17 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:18.062 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:17 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:18.062 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:17 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:18.062 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:17 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:18.063 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:17 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:18.063 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:17 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:18.063 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:17 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:18.063 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:17 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:57:18.063 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:17 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:57:18.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:18.061+0000 7f4f517fa640 1 -- 172.21.15.1:0/2298756310 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11949 (secure 0 0 0) 0x7f4f480a00a0 con 0x7f4f54102790 2024-09-11T21:57:18.064 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:57:18.064 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 28 2024-09-11T21:57:18.064 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":28,"btime":"2024-09-11T21:57:12:512925+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/421542832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":421542832},{"type":"v1","addr":"172.21.15.1:6835","nonce":421542832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2836864406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2836864406},{"type":"v1","addr":"172.21.15.1:6837","nonce":2836864406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T21:55:20.791263+0000","modified":"2024-09-11T21:57:12.462736+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24484,"mds_2":24467,"mds_3":24466,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14595":{"gid":14595,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/407834672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":407834672},{"type":"v1","addr":"172.21.15.1:6839","nonce":407834672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/1068827678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":1068827678},{"type":"v1","addr":"172.21.15.1:6841","nonce":1068827678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/872410024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":872410024},{"type":"v1","addr":"172.21.15.161:6833","nonce":872410024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/1648944570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1648944570},{"type":"v1","addr":"172.21.15.53:6833","nonce":1648944570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/2126933993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2126933993},{"type":"v1","addr":"172.21.15.53:6835","nonce":2126933993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.161:6835/828381592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":828381592},{"type":"v1","addr":"172.21.15.161:6835","nonce":828381592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.161:6837/1799318656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1799318656},{"type":"v1","addr":"172.21.15.161:6837","nonce":1799318656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.53:6837/3457131830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3457131830},{"type":"v1","addr":"172.21.15.53:6837","nonce":3457131830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/455365383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":455365383},{"type":"v1","addr":"172.21.15.161:6839","nonce":455365383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.53:6839/1378584447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1378584447},{"type":"v1","addr":"172.21.15.53:6839","nonce":1378584447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24460,24466,24467,24484,24494]},"id":1}]} 2024-09-11T21:57:18.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:18.064+0000 7f4f5a472640 1 -- 172.21.15.1:0/2298756310 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f4f3c0a2cf0 msgr2=0x7f4f3c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:18.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:18.064+0000 7f4f5a472640 1 --2- 172.21.15.1:0/2298756310 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f4f3c0a2cf0 0x7f4f3c0a51b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4f44009230 tx=0x7f4f440058d0 comp rx=0 tx=0).stop 2024-09-11T21:57:18.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:18.064+0000 7f4f5a472640 1 -- 172.21.15.1:0/2298756310 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f54102790 msgr2=0x7f4f54074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:18.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:18.064+0000 7f4f5a472640 1 --2- 172.21.15.1:0/2298756310 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f54102790 0x7f4f54074a30 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f4f4800e9f0 tx=0x7f4f4800eeb0 comp rx=0 tx=0).stop 2024-09-11T21:57:18.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:18.064+0000 7f4f5a472640 1 -- 172.21.15.1:0/2298756310 shutdown_connections 2024-09-11T21:57:18.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:18.064+0000 7f4f5a472640 1 --2- 172.21.15.1:0/2298756310 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f4f3c0a2cf0 0x7f4f3c0a51b0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:18.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:18.064+0000 7f4f5a472640 1 --2- 172.21.15.1:0/2298756310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f54106f10 0x7f4f54074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:18.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:18.064+0000 7f4f5a472640 1 --2- 172.21.15.1:0/2298756310 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f54102790 0x7f4f54074a30 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:18.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:18.064+0000 7f4f5a472640 1 --2- 172.21.15.1:0/2298756310 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f54101de0 0x7f4f540744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:18.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:18.064+0000 7f4f5a472640 1 -- 172.21.15.1:0/2298756310 >> 172.21.15.1:0/2298756310 conn(0x7f4f540fd8e0 msgr2=0x7f4f540fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:18.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:18.064+0000 7f4f5a472640 1 -- 172.21.15.1:0/2298756310 shutdown_connections 2024-09-11T21:57:18.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:18.064+0000 7f4f5a472640 1 -- 172.21.15.1:0/2298756310 wait complete. 2024-09-11T21:57:18.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:17 smithi053 ceph-mon[34908]: Adjusting osd_memory_target on smithi161 to 176.0M 2024-09-11T21:57:18.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:17 smithi053 ceph-mon[34908]: Unable to set osd_memory_target on smithi161 to 184591769: error parsing value: Value '184591769' is below minimum 939524096 2024-09-11T21:57:18.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:17 smithi053 ceph-mon[34908]: Adjusting osd_memory_target on smithi053 to 177.0M 2024-09-11T21:57:18.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:17 smithi053 ceph-mon[34908]: Unable to set osd_memory_target on smithi053 to 185699942: error parsing value: Value '185699942' is below minimum 939524096 2024-09-11T21:57:18.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:17 smithi053 ceph-mon[34908]: from='client.15303 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T21:57:18.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:17 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:57:18.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:17 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:57:18.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:17 smithi053 ceph-mon[34908]: from='client.15309 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T21:57:18.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:17 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:57:18.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:17 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:57:18.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:17 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:18.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:17 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:18.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:17 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:18.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:17 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:18.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:17 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:18.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:17 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:18.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:17 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:18.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:17 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:18.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:17 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:57:18.300 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:17 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:57:18.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:17 smithi161 ceph-mon[35615]: Adjusting osd_memory_target on smithi161 to 176.0M 2024-09-11T21:57:18.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:17 smithi161 ceph-mon[35615]: Unable to set osd_memory_target on smithi161 to 184591769: error parsing value: Value '184591769' is below minimum 939524096 2024-09-11T21:57:18.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:17 smithi161 ceph-mon[35615]: Adjusting osd_memory_target on smithi053 to 177.0M 2024-09-11T21:57:18.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:17 smithi161 ceph-mon[35615]: Unable to set osd_memory_target on smithi053 to 185699942: error parsing value: Value '185699942' is below minimum 939524096 2024-09-11T21:57:18.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:17 smithi161 ceph-mon[35615]: from='client.15303 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T21:57:18.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:17 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:57:18.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:17 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:57:18.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:17 smithi161 ceph-mon[35615]: from='client.15309 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T21:57:18.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:17 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:57:18.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:17 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:57:18.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:17 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:18.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:17 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:18.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:17 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:18.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:17 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:18.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:17 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:18.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:17 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:18.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:17 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:18.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:17 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T21:57:18.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:17 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:57:18.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:17 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:57:18.912 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T21:57:19.169 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:57:19.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:18 smithi001 ceph-mon[30935]: Adjusting osd_memory_target on smithi001 to 177.0M 2024-09-11T21:57:19.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:18 smithi001 ceph-mon[30935]: Unable to set osd_memory_target on smithi001 to 185689190: error parsing value: Value '185689190' is below minimum 939524096 2024-09-11T21:57:19.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:18 smithi001 ceph-mon[30935]: Updating smithi001:/etc/ceph/ceph.conf 2024-09-11T21:57:19.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:18 smithi001 ceph-mon[30935]: Updating smithi053:/etc/ceph/ceph.conf 2024-09-11T21:57:19.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:18 smithi001 ceph-mon[30935]: Updating smithi161:/etc/ceph/ceph.conf 2024-09-11T21:57:19.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:18 smithi001 ceph-mon[30935]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:57:19.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:18 smithi001 ceph-mon[30935]: Updating smithi053:/var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T21:57:19.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:18 smithi001 ceph-mon[30935]: Updating smithi001:/var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T21:57:19.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:18 smithi001 ceph-mon[30935]: Updating smithi161:/var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T21:57:19.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:18 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/2298756310' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:57:19.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:18 smithi001 ceph-mon[30935]: Updating smithi053:/etc/ceph/ceph.client.admin.keyring 2024-09-11T21:57:19.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:18 smithi001 ceph-mon[30935]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2024-09-11T21:57:19.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:18 smithi001 ceph-mon[30935]: Updating smithi001:/etc/ceph/ceph.client.admin.keyring 2024-09-11T21:57:19.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:18 smithi001 ceph-mon[30935]: Updating smithi053:/var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T21:57:19.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:18 smithi001 ceph-mon[30935]: Updating smithi161:/var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T21:57:19.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:18 smithi001 ceph-mon[30935]: Updating smithi001:/var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T21:57:19.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:18 smithi001 ceph-mon[30935]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-09-11T21:57:19.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:18 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:57:19.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:18 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:57:19.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:18 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:57:19.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:18 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:57:19.195 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:18 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:57:19.195 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:18 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:57:19.195 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:18 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:57:19.195 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:18 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:57:19.195 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:18 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:57:19.195 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:18 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:57:19.195 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:18 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:57:19.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:18 smithi053 ceph-mon[34908]: Adjusting osd_memory_target on smithi001 to 177.0M 2024-09-11T21:57:19.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:18 smithi053 ceph-mon[34908]: Unable to set osd_memory_target on smithi001 to 185689190: error parsing value: Value '185689190' is below minimum 939524096 2024-09-11T21:57:19.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:18 smithi053 ceph-mon[34908]: Updating smithi001:/etc/ceph/ceph.conf 2024-09-11T21:57:19.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:18 smithi053 ceph-mon[34908]: Updating smithi053:/etc/ceph/ceph.conf 2024-09-11T21:57:19.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:18 smithi053 ceph-mon[34908]: Updating smithi161:/etc/ceph/ceph.conf 2024-09-11T21:57:19.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:18 smithi053 ceph-mon[34908]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:57:19.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:18 smithi053 ceph-mon[34908]: Updating smithi053:/var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T21:57:19.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:18 smithi053 ceph-mon[34908]: Updating smithi001:/var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T21:57:19.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:18 smithi053 ceph-mon[34908]: Updating smithi161:/var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T21:57:19.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:18 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/2298756310' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:57:19.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:18 smithi053 ceph-mon[34908]: Updating smithi053:/etc/ceph/ceph.client.admin.keyring 2024-09-11T21:57:19.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:18 smithi053 ceph-mon[34908]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2024-09-11T21:57:19.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:18 smithi053 ceph-mon[34908]: Updating smithi001:/etc/ceph/ceph.client.admin.keyring 2024-09-11T21:57:19.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:18 smithi053 ceph-mon[34908]: Updating smithi053:/var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T21:57:19.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:18 smithi053 ceph-mon[34908]: Updating smithi161:/var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T21:57:19.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:18 smithi053 ceph-mon[34908]: Updating smithi001:/var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T21:57:19.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:18 smithi053 ceph-mon[34908]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-09-11T21:57:19.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:18 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:57:19.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:18 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:57:19.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:18 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:57:19.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:18 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:57:19.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:18 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:57:19.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:18 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:57:19.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:18 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:57:19.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:18 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:57:19.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:18 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:57:19.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:18 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:57:19.300 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:18 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:57:19.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:18 smithi161 ceph-mon[35615]: Adjusting osd_memory_target on smithi001 to 177.0M 2024-09-11T21:57:19.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:18 smithi161 ceph-mon[35615]: Unable to set osd_memory_target on smithi001 to 185689190: error parsing value: Value '185689190' is below minimum 939524096 2024-09-11T21:57:19.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:18 smithi161 ceph-mon[35615]: Updating smithi001:/etc/ceph/ceph.conf 2024-09-11T21:57:19.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:18 smithi161 ceph-mon[35615]: Updating smithi053:/etc/ceph/ceph.conf 2024-09-11T21:57:19.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:18 smithi161 ceph-mon[35615]: Updating smithi161:/etc/ceph/ceph.conf 2024-09-11T21:57:19.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:18 smithi161 ceph-mon[35615]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:57:19.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:18 smithi161 ceph-mon[35615]: Updating smithi053:/var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T21:57:19.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:18 smithi161 ceph-mon[35615]: Updating smithi001:/var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T21:57:19.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:18 smithi161 ceph-mon[35615]: Updating smithi161:/var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T21:57:19.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:18 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/2298756310' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:57:19.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:18 smithi161 ceph-mon[35615]: Updating smithi053:/etc/ceph/ceph.client.admin.keyring 2024-09-11T21:57:19.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:18 smithi161 ceph-mon[35615]: Updating smithi161:/etc/ceph/ceph.client.admin.keyring 2024-09-11T21:57:19.344 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:18 smithi161 ceph-mon[35615]: Updating smithi001:/etc/ceph/ceph.client.admin.keyring 2024-09-11T21:57:19.344 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:18 smithi161 ceph-mon[35615]: Updating smithi053:/var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T21:57:19.344 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:18 smithi161 ceph-mon[35615]: Updating smithi161:/var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T21:57:19.344 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:18 smithi161 ceph-mon[35615]: Updating smithi001:/var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T21:57:19.344 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:18 smithi161 ceph-mon[35615]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-09-11T21:57:19.344 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:18 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:57:19.344 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:18 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:57:19.344 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:18 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:57:19.344 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:18 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:57:19.345 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:18 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:57:19.345 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:18 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:57:19.345 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:18 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:57:19.345 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:18 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:57:19.345 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:18 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:57:19.345 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:18 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:57:19.345 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:18 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:57:19.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:19.610+0000 7fca6566f640 1 -- 172.21.15.1:0/1194506256 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca60102570 msgr2=0x7fca601029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:19.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:19.610+0000 7fca6566f640 1 --2- 172.21.15.1:0/1194506256 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca60102570 0x7fca601029b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fca48009900 tx=0x7fca4802a480 comp rx=0 tx=0).stop 2024-09-11T21:57:19.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:19.610+0000 7fca6566f640 1 -- 172.21.15.1:0/1194506256 shutdown_connections 2024-09-11T21:57:19.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:19.610+0000 7fca6566f640 1 --2- 172.21.15.1:0/1194506256 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca60102ef0 0x7fca6010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:19.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:19.610+0000 7fca6566f640 1 --2- 172.21.15.1:0/1194506256 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca60102570 0x7fca601029b0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:19.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:19.610+0000 7fca6566f640 1 --2- 172.21.15.1:0/1194506256 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca60100e20 0x7fca60101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:19.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:19.610+0000 7fca6566f640 1 -- 172.21.15.1:0/1194506256 >> 172.21.15.1:0/1194506256 conn(0x7fca600fc7d0 msgr2=0x7fca600febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:19.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:19.611+0000 7fca6566f640 1 -- 172.21.15.1:0/1194506256 shutdown_connections 2024-09-11T21:57:19.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:19.611+0000 7fca6566f640 1 -- 172.21.15.1:0/1194506256 wait complete. 2024-09-11T21:57:19.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:19.611+0000 7fca6566f640 1 Processor -- start 2024-09-11T21:57:19.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:19.611+0000 7fca6566f640 1 -- start start 2024-09-11T21:57:19.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:19.612+0000 7fca6566f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca60100e20 0x7fca6010e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:19.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:19.612+0000 7fca6566f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca60102570 0x7fca6010ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:19.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:19.612+0000 7fca6566f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca60102ef0 0x7fca6010ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:19.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:19.612+0000 7fca6566f640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fca601120a0 con 0x7fca60102ef0 2024-09-11T21:57:19.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:19.612+0000 7fca6566f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fca60111f20 con 0x7fca60102570 2024-09-11T21:57:19.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:19.612+0000 7fca6566f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fca60112220 con 0x7fca60100e20 2024-09-11T21:57:19.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:19.612+0000 7fca5f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca60102ef0 0x7fca6010ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:19.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:19.612+0000 7fca5f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca60102ef0 0x7fca6010ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38656/0 (socket says 172.21.15.1:38656) 2024-09-11T21:57:19.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:19.612+0000 7fca5effd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca60100e20 0x7fca6010e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:19.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:19.612+0000 7fca5f7fe640 1 -- 172.21.15.1:0/1922493739 learned_addr learned my addr 172.21.15.1:0/1922493739 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:19.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:19.612+0000 7fca5e7fc640 1 --2- 172.21.15.1:0/1922493739 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca60102570 0x7fca6010ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:19.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:19.612+0000 7fca5f7fe640 1 -- 172.21.15.1:0/1922493739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca60100e20 msgr2=0x7fca6010e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:19.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:19.612+0000 7fca5f7fe640 1 --2- 172.21.15.1:0/1922493739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca60100e20 0x7fca6010e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:19.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:19.612+0000 7fca5f7fe640 1 -- 172.21.15.1:0/1922493739 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca60102570 msgr2=0x7fca6010ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:19.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:19.612+0000 7fca5f7fe640 1 --2- 172.21.15.1:0/1922493739 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca60102570 0x7fca6010ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:19.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:19.612+0000 7fca5f7fe640 1 -- 172.21.15.1:0/1922493739 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca601a6a30 con 0x7fca60102ef0 2024-09-11T21:57:19.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:19.612+0000 7fca5effd640 1 --2- 172.21.15.1:0/1922493739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca60100e20 0x7fca6010e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T21:57:19.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:19.612+0000 7fca5e7fc640 1 --2- 172.21.15.1:0/1922493739 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca60102570 0x7fca6010ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:57:19.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:19.612+0000 7fca5f7fe640 1 --2- 172.21.15.1:0/1922493739 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca60102ef0 0x7fca6010ef50 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fca5000eaa0 tx=0x7fca5000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:19.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:19.613+0000 7fca57fff640 1 -- 172.21.15.1:0/1922493739 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5000cdb0 con 0x7fca60102ef0 2024-09-11T21:57:19.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:19.613+0000 7fca6566f640 1 -- 172.21.15.1:0/1922493739 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca601a6d20 con 0x7fca60102ef0 2024-09-11T21:57:19.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:19.613+0000 7fca57fff640 1 -- 172.21.15.1:0/1922493739 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca50005020 con 0x7fca60102ef0 2024-09-11T21:57:19.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:19.613+0000 7fca6566f640 1 -- 172.21.15.1:0/1922493739 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca601a7260 con 0x7fca60102ef0 2024-09-11T21:57:19.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:19.613+0000 7fca57fff640 1 -- 172.21.15.1:0/1922493739 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca500116b0 con 0x7fca60102ef0 2024-09-11T21:57:19.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:19.614+0000 7fca6566f640 1 -- 172.21.15.1:0/1922493739 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca60101200 con 0x7fca60102ef0 2024-09-11T21:57:19.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:19.616+0000 7fca57fff640 1 -- 172.21.15.1:0/1922493739 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 24) ==== 138008+0+0 (secure 0 0 0) 0x7fca50005560 con 0x7fca60102ef0 2024-09-11T21:57:19.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:19.617+0000 7fca57fff640 1 --2- 172.21.15.1:0/1922493739 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fca4c0a2d80 0x7fca4c0a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:19.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:19.617+0000 7fca5effd640 1 --2- 172.21.15.1:0/1922493739 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fca4c0a2d80 0x7fca4c0a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:19.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:19.617+0000 7fca57fff640 1 -- 172.21.15.1:0/1922493739 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7fca500cfb90 con 0x7fca60102ef0 2024-09-11T21:57:19.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:19.617+0000 7fca5effd640 1 --2- 172.21.15.1:0/1922493739 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fca4c0a2d80 0x7fca4c0a5240 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fca5800bfd0 tx=0x7fca5800b390 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:19.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:19.620+0000 7fca57fff640 1 -- 172.21.15.1:0/1922493739 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fca50016030 con 0x7fca60102ef0 2024-09-11T21:57:19.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:19.797+0000 7fca6566f640 1 -- 172.21.15.1:0/1922493739 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fca60109090 con 0x7fca60102ef0 2024-09-11T21:57:19.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:19.798+0000 7fca57fff640 1 -- 172.21.15.1:0/1922493739 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+23041 (secure 0 0 0) 0x7fca5009c640 con 0x7fca60102ef0 2024-09-11T21:57:19.799 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:57:19.799 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":87,"fsid":"3ffca83c-7087-11ef-bcea-c7b262605968","created":"2024-09-11T21:47:28.868707+0000","modified":"2024-09-11T21:57:13.664420+0000","last_up_change":"2024-09-11T21:54:29.319441+0000","last_in_change":"2024-09-11T21:54:05.532281+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T21:50:41.026490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T21:55:16.961999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T21:55:19.066228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"19e56504-18f1-4234-b2d6-f193ff47d8eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":4012142023},{"type":"v1","addr":"172.21.15.1:6803","nonce":4012142023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":4012142023},{"type":"v1","addr":"172.21.15.1:6805","nonce":4012142023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":4012142023},{"type":"v1","addr":"172.21.15.1:6809","nonce":4012142023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":4012142023},{"type":"v1","addr":"172.21.15.1:6807","nonce":4012142023}]},"public_addr":"172.21.15.1:6803/4012142023","cluster_addr":"172.21.15.1:6805/4012142023","heartbeat_back_addr":"172.21.15.1:6809/4012142023","heartbeat_front_addr":"172.21.15.1:6807/4012142023","state":["exists","up"]},{"osd":1,"uuid":"996c9f0c-c5a4-43b8-b8e0-d712f27c5dfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3039391924},{"type":"v1","addr":"172.21.15.53:6801","nonce":3039391924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3039391924},{"type":"v1","addr":"172.21.15.53:6803","nonce":3039391924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3039391924},{"type":"v1","addr":"172.21.15.53:6807","nonce":3039391924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3039391924},{"type":"v1","addr":"172.21.15.53:6805","nonce":3039391924}]},"public_addr":"172.21.15.53:6801/3039391924","cluster_addr":"172.21.15.53:6803/3039391924","heartbeat_back_addr":"172.21.15.53:6807/3039391924","heartbeat_front_addr":"172.21.15.53:6805/3039391924","state":["exists","up"]},{"osd":2,"uuid":"578d4260-41e1-4645-b9b1-1c5445712c13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1707914885},{"type":"v1","addr":"172.21.15.161:6801","nonce":1707914885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1707914885},{"type":"v1","addr":"172.21.15.161:6803","nonce":1707914885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1707914885},{"type":"v1","addr":"172.21.15.161:6807","nonce":1707914885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1707914885},{"type":"v1","addr":"172.21.15.161:6805","nonce":1707914885}]},"public_addr":"172.21.15.161:6801/1707914885","cluster_addr":"172.21.15.161:6803/1707914885","heartbeat_back_addr":"172.21.15.161:6807/1707914885","heartbeat_front_addr":"172.21.15.161:6805/1707914885","state":["exists","up"]},{"osd":3,"uuid":"73f78766-9662-4aa3-841c-fe2ce654ee9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1541142398},{"type":"v1","addr":"172.21.15.1:6811","nonce":1541142398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1541142398},{"type":"v1","addr":"172.21.15.1:6813","nonce":1541142398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1541142398},{"type":"v1","addr":"172.21.15.1:6817","nonce":1541142398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1541142398},{"type":"v1","addr":"172.21.15.1:6815","nonce":1541142398}]},"public_addr":"172.21.15.1:6811/1541142398","cluster_addr":"172.21.15.1:6813/1541142398","heartbeat_back_addr":"172.21.15.1:6817/1541142398","heartbeat_front_addr":"172.21.15.1:6815/1541142398","state":["exists","up"]},{"osd":4,"uuid":"56c808bb-d4cd-4969-b49e-adef49ecd0f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":3081402681},{"type":"v1","addr":"172.21.15.53:6809","nonce":3081402681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3081402681},{"type":"v1","addr":"172.21.15.53:6811","nonce":3081402681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3081402681},{"type":"v1","addr":"172.21.15.53:6815","nonce":3081402681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3081402681},{"type":"v1","addr":"172.21.15.53:6813","nonce":3081402681}]},"public_addr":"172.21.15.53:6809/3081402681","cluster_addr":"172.21.15.53:6811/3081402681","heartbeat_back_addr":"172.21.15.53:6815/3081402681","heartbeat_front_addr":"172.21.15.53:6813/3081402681","state":["exists","up"]},{"osd":5,"uuid":"7b8266d6-b507-47c9-b4fa-ff16c4a74d35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":1622081327},{"type":"v1","addr":"172.21.15.161:6809","nonce":1622081327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":1622081327},{"type":"v1","addr":"172.21.15.161:6811","nonce":1622081327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":1622081327},{"type":"v1","addr":"172.21.15.161:6815","nonce":1622081327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":1622081327},{"type":"v1","addr":"172.21.15.161:6813","nonce":1622081327}]},"public_addr":"172.21.15.161:6809/1622081327","cluster_addr":"172.21.15.161:6811/1622081327","heartbeat_back_addr":"172.21.15.161:6815/1622081327","heartbeat_front_addr":"172.21.15.161:6813/1622081327","state":["exists","up"]},{"osd":6,"uuid":"3af5925d-a749-4849-a076-80580222cad4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":158434853},{"type":"v1","addr":"172.21.15.1:6819","nonce":158434853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":158434853},{"type":"v1","addr":"172.21.15.1:6821","nonce":158434853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":158434853},{"type":"v1","addr":"172.21.15.1:6825","nonce":158434853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":158434853},{"type":"v1","addr":"172.21.15.1:6823","nonce":158434853}]},"public_addr":"172.21.15.1:6819/158434853","cluster_addr":"172.21.15.1:6821/158434853","heartbeat_back_addr":"172.21.15.1:6825/158434853","heartbeat_front_addr":"172.21.15.1:6823/158434853","state":["exists","up"]},{"osd":7,"uuid":"f8f26c3c-17e9-4e89-8392-289796e0c8a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2132057697},{"type":"v1","addr":"172.21.15.53:6817","nonce":2132057697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2132057697},{"type":"v1","addr":"172.21.15.53:6819","nonce":2132057697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2132057697},{"type":"v1","addr":"172.21.15.53:6823","nonce":2132057697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2132057697},{"type":"v1","addr":"172.21.15.53:6821","nonce":2132057697}]},"public_addr":"172.21.15.53:6817/2132057697","cluster_addr":"172.21.15.53:6819/2132057697","heartbeat_back_addr":"172.21.15.53:6823/2132057697","heartbeat_front_addr":"172.21.15.53:6821/2132057697","state":["exists","up"]},{"osd":8,"uuid":"36ea401b-3c93-4b9d-8880-298d1eb6e30b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1838248669},{"type":"v1","addr":"172.21.15.161:6817","nonce":1838248669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1838248669},{"type":"v1","addr":"172.21.15.161:6819","nonce":1838248669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1838248669},{"type":"v1","addr":"172.21.15.161:6823","nonce":1838248669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1838248669},{"type":"v1","addr":"172.21.15.161:6821","nonce":1838248669}]},"public_addr":"172.21.15.161:6817/1838248669","cluster_addr":"172.21.15.161:6819/1838248669","heartbeat_back_addr":"172.21.15.161:6823/1838248669","heartbeat_front_addr":"172.21.15.161:6821/1838248669","state":["exists","up"]},{"osd":9,"uuid":"4b788d3f-a3bc-4463-bd99-95fdb7a52768","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2806274589},{"type":"v1","addr":"172.21.15.1:6827","nonce":2806274589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2806274589},{"type":"v1","addr":"172.21.15.1:6829","nonce":2806274589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":2806274589},{"type":"v1","addr":"172.21.15.1:6833","nonce":2806274589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2806274589},{"type":"v1","addr":"172.21.15.1:6831","nonce":2806274589}]},"public_addr":"172.21.15.1:6827/2806274589","cluster_addr":"172.21.15.1:6829/2806274589","heartbeat_back_addr":"172.21.15.1:6833/2806274589","heartbeat_front_addr":"172.21.15.1:6831/2806274589","state":["exists","up"]},{"osd":10,"uuid":"70549796-5270-4ae2-ac3f-f61f5de105d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":761923580},{"type":"v1","addr":"172.21.15.53:6825","nonce":761923580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":761923580},{"type":"v1","addr":"172.21.15.53:6827","nonce":761923580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":761923580},{"type":"v1","addr":"172.21.15.53:6831","nonce":761923580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":761923580},{"type":"v1","addr":"172.21.15.53:6829","nonce":761923580}]},"public_addr":"172.21.15.53:6825/761923580","cluster_addr":"172.21.15.53:6827/761923580","heartbeat_back_addr":"172.21.15.53:6831/761923580","heartbeat_front_addr":"172.21.15.53:6829/761923580","state":["exists","up"]},{"osd":11,"uuid":"48f17c4a-370a-493e-956e-bf6ad3dcc594","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1586352190},{"type":"v1","addr":"172.21.15.161:6825","nonce":1586352190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":1586352190},{"type":"v1","addr":"172.21.15.161:6827","nonce":1586352190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":1586352190},{"type":"v1","addr":"172.21.15.161:6831","nonce":1586352190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":1586352190},{"type":"v1","addr":"172.21.15.161:6829","nonce":1586352190}]},"public_addr":"172.21.15.161:6825/1586352190","cluster_addr":"172.21.15.161:6827/1586352190","heartbeat_back_addr":"172.21.15.161:6831/1586352190","heartbeat_front_addr":"172.21.15.161:6829/1586352190","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:49:46.539613+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:50:11.912300+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:50:36.955490+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:51:02.200554+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:51:27.860585+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:51:54.694099+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:52:18.962351+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:52:44.881923+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:53:10.301390+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:53:35.417647+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:54:01.492219+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:54:26.938338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.26","mappings":[{"from":1,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/4286250488":"2024-09-12T21:57:13.664392+0000","172.21.15.1:0/2039944931":"2024-09-12T21:57:13.664392+0000","172.21.15.1:0/3202962495":"2024-09-12T21:48:00.674728+0000","172.21.15.1:6800/2311309446":"2024-09-12T21:48:00.674728+0000","172.21.15.1:6801/3890350886":"2024-09-12T21:57:13.664392+0000","172.21.15.1:6801/2311309446":"2024-09-12T21:48:00.674728+0000","172.21.15.1:0/2599895977":"2024-09-12T21:48:00.674728+0000","172.21.15.1:0/1278877732":"2024-09-12T21:48:24.524913+0000","172.21.15.1:0/276770268":"2024-09-12T21:48:00.674728+0000","172.21.15.1:0/1699984777":"2024-09-12T21:48:24.524913+0000","172.21.15.1:6800/2821761445":"2024-09-12T21:48:24.524913+0000","172.21.15.1:0/2797780151":"2024-09-12T21:57:13.664392+0000","172.21.15.1:6801/2821761445":"2024-09-12T21:48:24.524913+0000","172.21.15.1:0/1749020850":"2024-09-12T21:48:24.524913+0000","172.21.15.1:0/1202232270":"2024-09-12T21:57:13.664392+0000","172.21.15.1:6800/3890350886":"2024-09-12T21:57:13.664392+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T21:57:19.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:19.800+0000 7fca6566f640 1 -- 172.21.15.1:0/1922493739 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fca4c0a2d80 msgr2=0x7fca4c0a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:19.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:19.800+0000 7fca6566f640 1 --2- 172.21.15.1:0/1922493739 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fca4c0a2d80 0x7fca4c0a5240 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fca5800bfd0 tx=0x7fca5800b390 comp rx=0 tx=0).stop 2024-09-11T21:57:19.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:19.800+0000 7fca6566f640 1 -- 172.21.15.1:0/1922493739 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca60102ef0 msgr2=0x7fca6010ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:19.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:19.800+0000 7fca6566f640 1 --2- 172.21.15.1:0/1922493739 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca60102ef0 0x7fca6010ef50 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fca5000eaa0 tx=0x7fca5000ef60 comp rx=0 tx=0).stop 2024-09-11T21:57:19.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:19.801+0000 7fca6566f640 1 -- 172.21.15.1:0/1922493739 shutdown_connections 2024-09-11T21:57:19.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:19.801+0000 7fca6566f640 1 --2- 172.21.15.1:0/1922493739 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fca4c0a2d80 0x7fca4c0a5240 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:19.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:19.801+0000 7fca6566f640 1 --2- 172.21.15.1:0/1922493739 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca60102ef0 0x7fca6010ef50 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:19.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:19.801+0000 7fca6566f640 1 --2- 172.21.15.1:0/1922493739 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca60102570 0x7fca6010ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:19.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:19.801+0000 7fca6566f640 1 --2- 172.21.15.1:0/1922493739 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca60100e20 0x7fca6010e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:19.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:19.801+0000 7fca6566f640 1 -- 172.21.15.1:0/1922493739 >> 172.21.15.1:0/1922493739 conn(0x7fca600fc7d0 msgr2=0x7fca600febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:19.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:19.801+0000 7fca6566f640 1 -- 172.21.15.1:0/1922493739 shutdown_connections 2024-09-11T21:57:19.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:19.801+0000 7fca6566f640 1 -- 172.21.15.1:0/1922493739 wait complete. 2024-09-11T21:57:20.151 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:19 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/1922493739' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T21:57:20.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:19 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/1922493739' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T21:57:20.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:19 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/1922493739' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T21:57:20.604 DEBUG:tasks.fwd_scrub:Yielding 2024-09-11T21:57:20.604 INFO:teuthology.run_tasks:Running task exec... 2024-09-11T21:57:20.613 INFO:teuthology.task.exec:Executing custom commands... 2024-09-11T21:57:20.613 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi001.front.sepia.ceph.com 2024-09-11T21:57:20.613 DEBUG:teuthology.orchestra.run.smithi001:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/log_level debug' 2024-09-11T21:57:20.614 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-09-11T21:57:20.614 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T21:57:20.614 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T21:57:20.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:20.728+0000 7f8f5443c640 1 -- 172.21.15.1:0/595399443 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f4c101340 msgr2=0x7f8f4c10d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:20.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:20.728+0000 7f8f5443c640 1 --2- 172.21.15.1:0/595399443 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f4c101340 0x7f8f4c10d870 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f8f44009cd0 tx=0x7f8f4402a8e0 comp rx=0 tx=0).stop 2024-09-11T21:57:20.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:20.728+0000 7f8f5443c640 1 -- 172.21.15.1:0/595399443 shutdown_connections 2024-09-11T21:57:20.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:20.728+0000 7f8f5443c640 1 --2- 172.21.15.1:0/595399443 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f4c101340 0x7f8f4c10d870 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:20.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:20.728+0000 7f8f5443c640 1 --2- 172.21.15.1:0/595399443 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f4c1009a0 0x7f8f4c100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:20.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:20.728+0000 7f8f5443c640 1 --2- 172.21.15.1:0/595399443 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f4c1069b0 0x7f8f4c106d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:20.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:20.728+0000 7f8f5443c640 1 -- 172.21.15.1:0/595399443 >> 172.21.15.1:0/595399443 conn(0x7f8f4c0779a0 msgr2=0x7f8f4c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:20.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:20.728+0000 7f8f5443c640 1 -- 172.21.15.1:0/595399443 shutdown_connections 2024-09-11T21:57:20.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:20.728+0000 7f8f5443c640 1 -- 172.21.15.1:0/595399443 wait complete. 2024-09-11T21:57:20.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:20.729+0000 7f8f5443c640 1 Processor -- start 2024-09-11T21:57:20.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:20.729+0000 7f8f5443c640 1 -- start start 2024-09-11T21:57:20.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:20.729+0000 7f8f5443c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f4c1009a0 0x7f8f4c199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:20.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:20.729+0000 7f8f5443c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f4c101340 0x7f8f4c1996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:20.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:20.729+0000 7f8f5443c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f4c1069b0 0x7f8f4c1a4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:20.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:20.729+0000 7f8f5443c640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8f4c110e40 con 0x7f8f4c1069b0 2024-09-11T21:57:20.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:20.729+0000 7f8f5443c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8f4c110cc0 con 0x7f8f4c1009a0 2024-09-11T21:57:20.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:20.729+0000 7f8f5443c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8f4c110fc0 con 0x7f8f4c101340 2024-09-11T21:57:20.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:20.729+0000 7f8f529d9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f4c1069b0 0x7f8f4c1a4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:20.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:20.729+0000 7f8f529d9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f4c1069b0 0x7f8f4c1a4200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38672/0 (socket says 172.21.15.1:38672) 2024-09-11T21:57:20.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:20.729+0000 7f8f529d9640 1 -- 172.21.15.1:0/3020876212 learned_addr learned my addr 172.21.15.1:0/3020876212 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:20.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:20.729+0000 7f8f521d8640 1 --2- 172.21.15.1:0/3020876212 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f4c1009a0 0x7f8f4c199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:20.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:20.729+0000 7f8f519d7640 1 --2- 172.21.15.1:0/3020876212 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f4c101340 0x7f8f4c1996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:20.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:20.729+0000 7f8f529d9640 1 -- 172.21.15.1:0/3020876212 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f4c101340 msgr2=0x7f8f4c1996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:20.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:20.730+0000 7f8f529d9640 1 --2- 172.21.15.1:0/3020876212 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f4c101340 0x7f8f4c1996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:20.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:20.730+0000 7f8f529d9640 1 -- 172.21.15.1:0/3020876212 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f4c1009a0 msgr2=0x7f8f4c199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:20.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:20.730+0000 7f8f529d9640 1 --2- 172.21.15.1:0/3020876212 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f4c1009a0 0x7f8f4c199180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:20.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:20.730+0000 7f8f529d9640 1 -- 172.21.15.1:0/3020876212 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f4c1a4740 con 0x7f8f4c1069b0 2024-09-11T21:57:20.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:20.730+0000 7f8f529d9640 1 --2- 172.21.15.1:0/3020876212 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f4c1069b0 0x7f8f4c1a4200 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f8f44009ca0 tx=0x7f8f44003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:20.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:20.730+0000 7f8f3f7fe640 1 -- 172.21.15.1:0/3020876212 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f44004240 con 0x7f8f4c1069b0 2024-09-11T21:57:20.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:20.730+0000 7f8f3f7fe640 1 -- 172.21.15.1:0/3020876212 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f440305a0 con 0x7f8f4c1069b0 2024-09-11T21:57:20.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:20.730+0000 7f8f5443c640 1 -- 172.21.15.1:0/3020876212 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f4c1a4910 con 0x7f8f4c1069b0 2024-09-11T21:57:20.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:20.730+0000 7f8f3f7fe640 1 -- 172.21.15.1:0/3020876212 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f4403a660 con 0x7f8f4c1069b0 2024-09-11T21:57:20.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:20.730+0000 7f8f5443c640 1 -- 172.21.15.1:0/3020876212 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f4c1a4e20 con 0x7f8f4c1069b0 2024-09-11T21:57:20.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:20.731+0000 7f8f5443c640 1 -- 172.21.15.1:0/3020876212 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f4c100e00 con 0x7f8f4c1069b0 2024-09-11T21:57:20.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:20.732+0000 7f8f3f7fe640 1 -- 172.21.15.1:0/3020876212 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 24) ==== 138008+0+0 (secure 0 0 0) 0x7f8f44030750 con 0x7f8f4c1069b0 2024-09-11T21:57:20.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:20.733+0000 7f8f3f7fe640 1 --2- 172.21.15.1:0/3020876212 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f8f380a2e00 0x7f8f380a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:20.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:20.733+0000 7f8f521d8640 1 --2- 172.21.15.1:0/3020876212 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f8f380a2e00 0x7f8f380a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:20.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:20.733+0000 7f8f3f7fe640 1 -- 172.21.15.1:0/3020876212 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7f8f4402e030 con 0x7f8f4c1069b0 2024-09-11T21:57:20.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:20.734+0000 7f8f521d8640 1 --2- 172.21.15.1:0/3020876212 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f8f380a2e00 0x7f8f380a52c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8f4000ab60 tx=0x7f8f4000a2d0 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:20.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:20.736+0000 7f8f3f7fe640 1 -- 172.21.15.1:0/3020876212 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8f440bd2a0 con 0x7f8f4c1069b0 2024-09-11T21:57:20.891 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:57:20.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:20.944+0000 7f8f5443c640 1 -- 172.21.15.1:0/3020876212 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7f8f4c1a5120 con 0x7f8f4c1069b0 2024-09-11T21:57:20.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:20.948+0000 7f8f3f7fe640 1 -- 172.21.15.1:0/3020876212 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v30)=0 v30) ==== 129+0+0 (secure 0 0 0) 0x7f8f440bcba0 con 0x7f8f4c1069b0 2024-09-11T21:57:20.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:20.950+0000 7f8f5443c640 1 -- 172.21.15.1:0/3020876212 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f8f380a2e00 msgr2=0x7f8f380a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:20.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:20.950+0000 7f8f5443c640 1 --2- 172.21.15.1:0/3020876212 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f8f380a2e00 0x7f8f380a52c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8f4000ab60 tx=0x7f8f4000a2d0 comp rx=0 tx=0).stop 2024-09-11T21:57:20.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:20.950+0000 7f8f5443c640 1 -- 172.21.15.1:0/3020876212 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f4c1069b0 msgr2=0x7f8f4c1a4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:20.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:20.950+0000 7f8f5443c640 1 --2- 172.21.15.1:0/3020876212 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f4c1069b0 0x7f8f4c1a4200 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f8f44009ca0 tx=0x7f8f44003ba0 comp rx=0 tx=0).stop 2024-09-11T21:57:20.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:20.951+0000 7f8f5443c640 1 -- 172.21.15.1:0/3020876212 shutdown_connections 2024-09-11T21:57:20.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:20.951+0000 7f8f5443c640 1 --2- 172.21.15.1:0/3020876212 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f8f380a2e00 0x7f8f380a52c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:20.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:20.951+0000 7f8f5443c640 1 --2- 172.21.15.1:0/3020876212 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f4c1069b0 0x7f8f4c1a4200 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:20.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:20.951+0000 7f8f5443c640 1 --2- 172.21.15.1:0/3020876212 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f4c101340 0x7f8f4c1996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:20.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:20.951+0000 7f8f5443c640 1 --2- 172.21.15.1:0/3020876212 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f4c1009a0 0x7f8f4c199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:20.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:20.951+0000 7f8f5443c640 1 -- 172.21.15.1:0/3020876212 >> 172.21.15.1:0/3020876212 conn(0x7f8f4c0779a0 msgr2=0x7f8f4c0fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:20.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:20.951+0000 7f8f5443c640 1 -- 172.21.15.1:0/3020876212 shutdown_connections 2024-09-11T21:57:20.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:20.951+0000 7f8f5443c640 1 -- 172.21.15.1:0/3020876212 wait complete. 2024-09-11T21:57:20.963 DEBUG:teuthology.orchestra.run.smithi001:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-09-11T21:57:21.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.072+0000 7f98266ae640 1 -- 172.21.15.1:0/3724974707 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f98201069b0 msgr2=0x7f9820106d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:21.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.072+0000 7f98266ae640 1 --2- 172.21.15.1:0/3724974707 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f98201069b0 0x7f9820106d90 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f9810009930 tx=0x7f981002a420 comp rx=0 tx=0).stop 2024-09-11T21:57:21.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.072+0000 7f98266ae640 1 -- 172.21.15.1:0/3724974707 shutdown_connections 2024-09-11T21:57:21.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.072+0000 7f98266ae640 1 --2- 172.21.15.1:0/3724974707 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9820101340 0x7f982010d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:21.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.072+0000 7f98266ae640 1 --2- 172.21.15.1:0/3724974707 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98201009a0 0x7f9820100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:21.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.072+0000 7f98266ae640 1 --2- 172.21.15.1:0/3724974707 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f98201069b0 0x7f9820106d90 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:21.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.072+0000 7f98266ae640 1 -- 172.21.15.1:0/3724974707 >> 172.21.15.1:0/3724974707 conn(0x7f98200779a0 msgr2=0x7f98200fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:21.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.072+0000 7f98266ae640 1 -- 172.21.15.1:0/3724974707 shutdown_connections 2024-09-11T21:57:21.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.072+0000 7f98266ae640 1 -- 172.21.15.1:0/3724974707 wait complete. 2024-09-11T21:57:21.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.073+0000 7f98266ae640 1 Processor -- start 2024-09-11T21:57:21.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.073+0000 7f98266ae640 1 -- start start 2024-09-11T21:57:21.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.073+0000 7f98266ae640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98201009a0 0x7f98200fcde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:21.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.073+0000 7f98266ae640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9820101340 0x7f98200fd320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:21.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.073+0000 7f98266ae640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f98201069b0 0x7f98200fd860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:21.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.073+0000 7f98266ae640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9820110df0 con 0x7f98201069b0 2024-09-11T21:57:21.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.073+0000 7f98266ae640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9820110c70 con 0x7f9820101340 2024-09-11T21:57:21.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.073+0000 7f98266ae640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9820110f70 con 0x7f98201009a0 2024-09-11T21:57:21.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.074+0000 7f9824c4b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f98201069b0 0x7f98200fd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:21.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.074+0000 7f981ffff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98201009a0 0x7f98200fcde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:21.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.074+0000 7f9824c4b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f98201069b0 0x7f98200fd860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38696/0 (socket says 172.21.15.1:38696) 2024-09-11T21:57:21.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.074+0000 7f9824c4b640 1 -- 172.21.15.1:0/2034795584 learned_addr learned my addr 172.21.15.1:0/2034795584 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:21.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.074+0000 7f981f7fe640 1 --2- 172.21.15.1:0/2034795584 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9820101340 0x7f98200fd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:21.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.074+0000 7f9824c4b640 1 -- 172.21.15.1:0/2034795584 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98201009a0 msgr2=0x7f98200fcde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:21.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.074+0000 7f9824c4b640 1 --2- 172.21.15.1:0/2034795584 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98201009a0 0x7f98200fcde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:21.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.074+0000 7f9824c4b640 1 -- 172.21.15.1:0/2034795584 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9820101340 msgr2=0x7f98200fd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:21.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.074+0000 7f9824c4b640 1 --2- 172.21.15.1:0/2034795584 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9820101340 0x7f98200fd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:21.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.074+0000 7f9824c4b640 1 -- 172.21.15.1:0/2034795584 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98201a6930 con 0x7f98201069b0 2024-09-11T21:57:21.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.074+0000 7f981f7fe640 1 --2- 172.21.15.1:0/2034795584 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9820101340 0x7f98200fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:57:21.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.074+0000 7f981ffff640 1 --2- 172.21.15.1:0/2034795584 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98201009a0 0x7f98200fcde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T21:57:21.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.074+0000 7f9824c4b640 1 --2- 172.21.15.1:0/2034795584 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f98201069b0 0x7f98200fd860 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f981800eb10 tx=0x7f981800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:21.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.074+0000 7f981d7fa640 1 -- 172.21.15.1:0/2034795584 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9818004b20 con 0x7f98201069b0 2024-09-11T21:57:21.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.074+0000 7f98266ae640 1 -- 172.21.15.1:0/2034795584 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98201a6c20 con 0x7f98201069b0 2024-09-11T21:57:21.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.074+0000 7f981d7fa640 1 -- 172.21.15.1:0/2034795584 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9818004cc0 con 0x7f98201069b0 2024-09-11T21:57:21.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.074+0000 7f981d7fa640 1 -- 172.21.15.1:0/2034795584 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9818010760 con 0x7f98201069b0 2024-09-11T21:57:21.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.074+0000 7f98266ae640 1 -- 172.21.15.1:0/2034795584 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98201a7120 con 0x7f98201069b0 2024-09-11T21:57:21.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.075+0000 7f98266ae640 1 -- 172.21.15.1:0/2034795584 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98201020e0 con 0x7f98201069b0 2024-09-11T21:57:21.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.077+0000 7f981d7fa640 1 -- 172.21.15.1:0/2034795584 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 24) ==== 138008+0+0 (secure 0 0 0) 0x7f9818010900 con 0x7f98201069b0 2024-09-11T21:57:21.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.077+0000 7f981d7fa640 1 --2- 172.21.15.1:0/2034795584 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f98080a2d80 0x7f98080a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:21.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.078+0000 7f981ffff640 1 --2- 172.21.15.1:0/2034795584 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f98080a2d80 0x7f98080a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:21.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.078+0000 7f981d7fa640 1 -- 172.21.15.1:0/2034795584 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7f98180cfdc0 con 0x7f98201069b0 2024-09-11T21:57:21.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.078+0000 7f981ffff640 1 --2- 172.21.15.1:0/2034795584 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f98080a2d80 0x7f98080a5240 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9810009900 tx=0x7f98100058d0 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:21.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.085+0000 7f981d7fa640 1 -- 172.21.15.1:0/2034795584 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f98180541b0 con 0x7f98201069b0 2024-09-11T21:57:21.144 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:20 smithi001 ceph-mon[30935]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:57:21.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.271+0000 7f98266ae640 1 -- 172.21.15.1:0/2034795584 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f9820100e00 con 0x7f98201069b0 2024-09-11T21:57:21.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.274+0000 7f981d7fa640 1 -- 172.21.15.1:0/2034795584 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v31)=0 v31) ==== 138+0+0 (secure 0 0 0) 0x7f981809c870 con 0x7f98201069b0 2024-09-11T21:57:21.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.277+0000 7f98266ae640 1 -- 172.21.15.1:0/2034795584 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f98080a2d80 msgr2=0x7f98080a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:21.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.277+0000 7f98266ae640 1 --2- 172.21.15.1:0/2034795584 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f98080a2d80 0x7f98080a5240 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9810009900 tx=0x7f98100058d0 comp rx=0 tx=0).stop 2024-09-11T21:57:21.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.277+0000 7f98266ae640 1 -- 172.21.15.1:0/2034795584 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f98201069b0 msgr2=0x7f98200fd860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:21.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.277+0000 7f98266ae640 1 --2- 172.21.15.1:0/2034795584 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f98201069b0 0x7f98200fd860 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f981800eb10 tx=0x7f981800c2d0 comp rx=0 tx=0).stop 2024-09-11T21:57:21.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.277+0000 7f98266ae640 1 -- 172.21.15.1:0/2034795584 shutdown_connections 2024-09-11T21:57:21.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.277+0000 7f98266ae640 1 --2- 172.21.15.1:0/2034795584 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f98080a2d80 0x7f98080a5240 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:21.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.277+0000 7f98266ae640 1 --2- 172.21.15.1:0/2034795584 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f98201069b0 0x7f98200fd860 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:21.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.277+0000 7f98266ae640 1 --2- 172.21.15.1:0/2034795584 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9820101340 0x7f98200fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:21.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.277+0000 7f98266ae640 1 --2- 172.21.15.1:0/2034795584 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98201009a0 0x7f98200fcde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:21.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.277+0000 7f98266ae640 1 -- 172.21.15.1:0/2034795584 >> 172.21.15.1:0/2034795584 conn(0x7f98200779a0 msgr2=0x7f9820069ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:21.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.278+0000 7f98266ae640 1 -- 172.21.15.1:0/2034795584 shutdown_connections 2024-09-11T21:57:21.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.278+0000 7f98266ae640 1 -- 172.21.15.1:0/2034795584 wait complete. 2024-09-11T21:57:21.292 DEBUG:teuthology.orchestra.run.smithi001:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-09-11T21:57:21.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:20 smithi053 ceph-mon[34908]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:57:21.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:20 smithi161 ceph-mon[35615]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T21:57:21.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.385+0000 7f061e5fd640 1 -- 172.21.15.1:0/2885845435 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0618102df0 msgr2=0x7f06181031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:21.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.385+0000 7f061e5fd640 1 --2- 172.21.15.1:0/2885845435 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0618102df0 0x7f06181031d0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f0608009900 tx=0x7f060802a410 comp rx=0 tx=0).stop 2024-09-11T21:57:21.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.385+0000 7f061e5fd640 1 -- 172.21.15.1:0/2885845435 shutdown_connections 2024-09-11T21:57:21.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.385+0000 7f061e5fd640 1 --2- 172.21.15.1:0/2885845435 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f061810c2f0 0x7f061810e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:21.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.385+0000 7f061e5fd640 1 --2- 172.21.15.1:0/2885845435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f06181037a0 0x7f061810bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:21.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.385+0000 7f061e5fd640 1 --2- 172.21.15.1:0/2885845435 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0618102df0 0x7f06181031d0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:21.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.385+0000 7f061e5fd640 1 -- 172.21.15.1:0/2885845435 >> 172.21.15.1:0/2885845435 conn(0x7f06180fc7d0 msgr2=0x7f06180febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:21.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.385+0000 7f061e5fd640 1 -- 172.21.15.1:0/2885845435 shutdown_connections 2024-09-11T21:57:21.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.385+0000 7f061e5fd640 1 -- 172.21.15.1:0/2885845435 wait complete. 2024-09-11T21:57:21.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.386+0000 7f061e5fd640 1 Processor -- start 2024-09-11T21:57:21.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.386+0000 7f061e5fd640 1 -- start start 2024-09-11T21:57:21.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.387+0000 7f061e5fd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0618102df0 0x7f0618100a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:21.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.387+0000 7f061e5fd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f06181037a0 0x7f0618100f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:21.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.387+0000 7f061e5fd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f061810c2f0 0x7f06181014d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:21.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.387+0000 7f061e5fd640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0618112110 con 0x7f06181037a0 2024-09-11T21:57:21.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.387+0000 7f061e5fd640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0618111f90 con 0x7f061810c2f0 2024-09-11T21:57:21.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.387+0000 7f061e5fd640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0618112290 con 0x7f0618102df0 2024-09-11T21:57:21.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.387+0000 7f06177fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f06181037a0 0x7f0618100f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:21.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.387+0000 7f06177fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f06181037a0 0x7f0618100f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38722/0 (socket says 172.21.15.1:38722) 2024-09-11T21:57:21.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.387+0000 7f06177fe640 1 -- 172.21.15.1:0/3641744710 learned_addr learned my addr 172.21.15.1:0/3641744710 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:21.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.387+0000 7f0617fff640 1 --2- 172.21.15.1:0/3641744710 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0618102df0 0x7f0618100a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:21.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.387+0000 7f061cb9a640 1 --2- 172.21.15.1:0/3641744710 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f061810c2f0 0x7f06181014d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:21.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.387+0000 7f06177fe640 1 -- 172.21.15.1:0/3641744710 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0618102df0 msgr2=0x7f0618100a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:21.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.387+0000 7f06177fe640 1 --2- 172.21.15.1:0/3641744710 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0618102df0 0x7f0618100a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:21.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.387+0000 7f06177fe640 1 -- 172.21.15.1:0/3641744710 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f061810c2f0 msgr2=0x7f06181014d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:21.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.387+0000 7f06177fe640 1 --2- 172.21.15.1:0/3641744710 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f061810c2f0 0x7f06181014d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:21.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.387+0000 7f06177fe640 1 -- 172.21.15.1:0/3641744710 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06181a6ad0 con 0x7f06181037a0 2024-09-11T21:57:21.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.387+0000 7f061cb9a640 1 --2- 172.21.15.1:0/3641744710 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f061810c2f0 0x7f06181014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:57:21.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.387+0000 7f06177fe640 1 --2- 172.21.15.1:0/3641744710 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f06181037a0 0x7f0618100f90 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f060c00d950 tx=0x7f060c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:21.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.388+0000 7f06157fa640 1 -- 172.21.15.1:0/3641744710 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f060c017070 con 0x7f06181037a0 2024-09-11T21:57:21.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.388+0000 7f06157fa640 1 -- 172.21.15.1:0/3641744710 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f060c009c30 con 0x7f06181037a0 2024-09-11T21:57:21.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.388+0000 7f061e5fd640 1 -- 172.21.15.1:0/3641744710 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06181a6dc0 con 0x7f06181037a0 2024-09-11T21:57:21.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.388+0000 7f06157fa640 1 -- 172.21.15.1:0/3641744710 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f060c0047a0 con 0x7f06181037a0 2024-09-11T21:57:21.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.388+0000 7f061e5fd640 1 -- 172.21.15.1:0/3641744710 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06181a7270 con 0x7f06181037a0 2024-09-11T21:57:21.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.390+0000 7f06209d6740 1 Processor -- start 2024-09-11T21:57:21.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.391+0000 7f06157fa640 1 -- 172.21.15.1:0/3641744710 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 24) ==== 138008+0+0 (secure 0 0 0) 0x7f060c004020 con 0x7f06181037a0 2024-09-11T21:57:21.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.391+0000 7f06209d6740 1 -- start start 2024-09-11T21:57:21.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.392+0000 7f06209d6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5563a1de1b60 0x5563a1de1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:21.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.392+0000 7f06209d6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5563a1de2480 0x5563a1dec800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:21.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.392+0000 7f06209d6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5563a1decd40 0x5563a1def130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:21.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.392+0000 7f06209d6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5563a1d47280 con 0x5563a1decd40 2024-09-11T21:57:21.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.392+0000 7f06209d6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5563a1ceeb10 con 0x5563a1de2480 2024-09-11T21:57:21.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.392+0000 7f06209d6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5563a1d43240 con 0x5563a1de1b60 2024-09-11T21:57:21.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.392+0000 7f06177fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5563a1decd40 0x5563a1def130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:21.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.392+0000 7f06177fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5563a1decd40 0x5563a1def130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38724/0 (socket says 172.21.15.1:38724) 2024-09-11T21:57:21.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.392+0000 7f06177fe640 1 -- 172.21.15.1:0/969836861 learned_addr learned my addr 172.21.15.1:0/969836861 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:21.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.392+0000 7f061cb9a640 1 --2- 172.21.15.1:0/969836861 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5563a1de1b60 0x5563a1de1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:21.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.392+0000 7f0617fff640 1 --2- 172.21.15.1:0/969836861 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5563a1de2480 0x5563a1dec800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:21.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.392+0000 7f06157fa640 1 --2- 172.21.15.1:0/3641744710 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f06000a2e00 0x7f06000a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:21.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.392+0000 7f061cb9a640 1 -- 172.21.15.1:0/969836861 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5563a1de2480 msgr2=0x5563a1dec800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:21.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.392+0000 7f061cb9a640 1 --2- 172.21.15.1:0/969836861 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5563a1de2480 0x5563a1dec800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:21.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.392+0000 7f061cb9a640 1 -- 172.21.15.1:0/969836861 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5563a1decd40 msgr2=0x5563a1def130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:21.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.392+0000 7f061cb9a640 1 --2- 172.21.15.1:0/969836861 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5563a1decd40 0x5563a1def130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:21.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.392+0000 7f061cb9a640 1 -- 172.21.15.1:0/969836861 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563a1cfb1f0 con 0x5563a1de1b60 2024-09-11T21:57:21.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.392+0000 7f06177fe640 1 --2- 172.21.15.1:0/969836861 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5563a1decd40 0x5563a1def130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T21:57:21.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.393+0000 7f06157fa640 1 -- 172.21.15.1:0/3641744710 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7f060c0cefe0 con 0x7f06181037a0 2024-09-11T21:57:21.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.393+0000 7f061cb9a640 1 --2- 172.21.15.1:0/3641744710 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f06000a2e00 0x7f06000a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:21.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.393+0000 7f061cb9a640 1 --2- 172.21.15.1:0/969836861 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5563a1de1b60 0x5563a1de1f40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f06100104a0 tx=0x7f0610010960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f26bfbd0d5168837 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:21.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.393+0000 7f061cb9a640 1 --2- 172.21.15.1:0/3641744710 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f06000a2e00 0x7f06000a52c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0610010fd0 tx=0x7f0610015000 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:21.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.394+0000 7f05fe7fc640 1 -- 172.21.15.1:0/969836861 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0610021070 con 0x5563a1de1b60 2024-09-11T21:57:21.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.394+0000 7f05fe7fc640 1 -- 172.21.15.1:0/969836861 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f061000ed60 con 0x5563a1de1b60 2024-09-11T21:57:21.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.394+0000 7f05fe7fc640 1 -- 172.21.15.1:0/969836861 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0610019070 con 0x5563a1de1b60 2024-09-11T21:57:21.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.394+0000 7f06209d6740 1 -- 172.21.15.1:0/969836861 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5563a1de1b60 msgr2=0x5563a1de1f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:21.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.394+0000 7f06209d6740 1 --2- 172.21.15.1:0/969836861 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5563a1de1b60 0x5563a1de1f40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f06100104a0 tx=0x7f0610010960 comp rx=0 tx=0).stop 2024-09-11T21:57:21.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.394+0000 7f06209d6740 1 -- 172.21.15.1:0/969836861 shutdown_connections 2024-09-11T21:57:21.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.394+0000 7f06209d6740 1 --2- 172.21.15.1:0/969836861 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5563a1decd40 0x5563a1def130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:21.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.395+0000 7f06209d6740 1 --2- 172.21.15.1:0/969836861 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5563a1de2480 0x5563a1dec800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:21.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.395+0000 7f06209d6740 1 --2- 172.21.15.1:0/969836861 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5563a1de1b60 0x5563a1de1f40 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:21.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.395+0000 7f06209d6740 1 -- 172.21.15.1:0/969836861 >> 172.21.15.1:0/969836861 conn(0x5563a1dde0f0 msgr2=0x5563a1dde4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:21.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.395+0000 7f06209d6740 1 -- 172.21.15.1:0/969836861 shutdown_connections 2024-09-11T21:57:21.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.395+0000 7f06209d6740 1 -- 172.21.15.1:0/969836861 wait complete. 2024-09-11T21:57:21.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.395+0000 7f06209d6740 1 -- start start 2024-09-11T21:57:21.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.395+0000 7f06209d6740 1 Processor -- start 2024-09-11T21:57:21.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.396+0000 7f06209d6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5563a1de1b60 0x5563a1de9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:21.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.396+0000 7f06209d6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5563a1de2480 0x5563a1dea1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:21.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.396+0000 7f06209d6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5563a1decd40 0x5563a1dea6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:21.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.396+0000 7f06209d6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5563a1def910 con 0x5563a1de2480 2024-09-11T21:57:21.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.396+0000 7f06209d6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5563a1d01f10 con 0x5563a1de1b60 2024-09-11T21:57:21.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.396+0000 7f06209d6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5563a1defa90 con 0x5563a1decd40 2024-09-11T21:57:21.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.396+0000 7f06209d6740 10 client.0 _ll_register_callbacks cb 0x5563a1ccaa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:57:21.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.396+0000 7f0617fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5563a1de1b60 0x5563a1de9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:21.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.396+0000 7f0617fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5563a1de1b60 0x5563a1de9c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:33764/0 (socket says 172.21.15.1:33764) 2024-09-11T21:57:21.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.396+0000 7f0617fff640 1 -- 172.21.15.1:0/3314184464 learned_addr learned my addr 172.21.15.1:0/3314184464 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:21.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.396+0000 7f0617fff640 1 -- 172.21.15.1:0/3314184464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5563a1decd40 msgr2=0x5563a1dea6e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T21:57:21.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.396+0000 7f0617fff640 1 --2- 172.21.15.1:0/3314184464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5563a1decd40 0x5563a1dea6e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:21.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.396+0000 7f0617fff640 1 -- 172.21.15.1:0/3314184464 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5563a1de2480 msgr2=0x5563a1dea1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:21.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.396+0000 7f0617fff640 1 --2- 172.21.15.1:0/3314184464 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5563a1de2480 0x5563a1dea1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:21.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.396+0000 7f0617fff640 1 -- 172.21.15.1:0/3314184464 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563a1ab86a0 con 0x5563a1de1b60 2024-09-11T21:57:21.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.397+0000 7f0617fff640 1 --2- 172.21.15.1:0/3314184464 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5563a1de1b60 0x5563a1de9c60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f060802a930 tx=0x7f0608031850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfc721abf5ce2556 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:21.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.397+0000 7f05f77fe640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-11T21:57:21.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.397+0000 7f05f77fe640 1 -- 172.21.15.1:0/3314184464 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0608035040 con 0x5563a1de1b60 2024-09-11T21:57:21.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.397+0000 7f06209d6740 1 -- 172.21.15.1:0/3314184464 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f061000ed60 con 0x5563a1de1b60 2024-09-11T21:57:21.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.397+0000 7f05f77fe640 1 -- 172.21.15.1:0/3314184464 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7f0608004240 con 0x5563a1de1b60 2024-09-11T21:57:21.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.397+0000 7f05f77fe640 1 -- 172.21.15.1:0/3314184464 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0608004440 con 0x5563a1de1b60 2024-09-11T21:57:21.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.397+0000 7f06209d6740 10 client.25001 fetch_fsmap learned FSMap version 28 2024-09-11T21:57:21.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.397+0000 7f06209d6740 1 -- 172.21.15.1:0/3314184464 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5563a1e03db0 con 0x5563a1de1b60 2024-09-11T21:57:21.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.397+0000 7f05f77fe640 1 -- 172.21.15.1:0/3314184464 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f061000ed60 con 0x5563a1de1b60 2024-09-11T21:57:21.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.398+0000 7f05f77fe640 1 -- 172.21.15.1:0/3314184464 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f060803d6c0 con 0x5563a1de1b60 2024-09-11T21:57:21.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.398+0000 7f06209d6740 10 client.25001 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T21:57:21.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.398+0000 7f06209d6740 10 client.25001 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T21:57:21.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.398+0000 7f06209d6740 20 client.25001 populate_metadata read hostname 'smithi001' 2024-09-11T21:57:21.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.398+0000 7f06209d6740 1 --2- 172.21.15.1:0/3314184464 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5563a1e05230 0x5563a1e076f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:21.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.398+0000 7f06209d6740 4 client.25001 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T21:57:21.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.398+0000 7f06209d6740 1 -- 172.21.15.1:0/3314184464 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563a1d547f0 con 0x5563a1e05230 2024-09-11T21:57:21.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.398+0000 7f06177fe640 1 --2- 172.21.15.1:0/3314184464 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5563a1e05230 0x5563a1e076f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:21.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.398+0000 7f05f77fe640 10 client.25001 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T21:57:21.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.398+0000 7f06177fe640 1 --2- 172.21.15.1:0/3314184464 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5563a1e05230 0x5563a1e076f0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6e13feec0d6e2ee server_cookie=8f2d5408151cc0b8 in_seq=0 out_seq=0 2024-09-11T21:57:21.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.399+0000 7f05f77fe640 1 -- 172.21.15.1:0/3314184464 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5563a1d547f0 con 0x5563a1e05230 2024-09-11T21:57:21.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.399+0000 7f05f77fe640 10 client.25001 handle_command_reply: tid=0 2024-09-11T21:57:21.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.399+0000 7f06209d6740 1 client.25001 shutdown 2024-09-11T21:57:21.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.400+0000 7f06209d6740 1 -- 172.21.15.1:0/3314184464 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5563a1de1b60 msgr2=0x5563a1de9c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:21.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.400+0000 7f06209d6740 1 --2- 172.21.15.1:0/3314184464 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5563a1de1b60 0x5563a1de9c60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f060802a930 tx=0x7f0608031850 comp rx=0 tx=0).stop 2024-09-11T21:57:21.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.400+0000 7f06209d6740 1 -- 172.21.15.1:0/3314184464 shutdown_connections 2024-09-11T21:57:21.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.400+0000 7f06209d6740 1 --2- 172.21.15.1:0/3314184464 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5563a1e05230 0x5563a1e076f0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:21.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.400+0000 7f06209d6740 1 --2- 172.21.15.1:0/3314184464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5563a1decd40 0x5563a1dea6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:21.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.400+0000 7f06209d6740 1 --2- 172.21.15.1:0/3314184464 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5563a1de2480 0x5563a1dea1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:21.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.400+0000 7f06209d6740 1 --2- 172.21.15.1:0/3314184464 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5563a1de1b60 0x5563a1de9c60 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:21.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.400+0000 7f06209d6740 1 -- 172.21.15.1:0/3314184464 >> 172.21.15.1:0/3314184464 conn(0x5563a1dde0f0 msgr2=0x5563a1de3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:21.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.400+0000 7f05f77fe640 0 client.25001 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T21:57:21.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.400+0000 7f06209d6740 1 -- 172.21.15.1:0/3314184464 shutdown_connections 2024-09-11T21:57:21.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.400+0000 7f06209d6740 1 -- 172.21.15.1:0/3314184464 wait complete. 2024-09-11T21:57:21.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.400+0000 7f06209d6740 20 client.25001 trim_cache size 0 max 16384 2024-09-11T21:57:21.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.410+0000 7f0fbbfff640 1 -- 172.21.15.1:0/539678448 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0fb4096590 msgr2=0x7f0fb4006190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:21.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.410+0000 7f0fbbfff640 1 --2- 172.21.15.1:0/539678448 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0fb4096590 0x7f0fb4006190 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f0fac009900 tx=0x7f0fac02a410 comp rx=0 tx=0).stop 2024-09-11T21:57:21.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.410+0000 7f0fbbfff640 1 -- 172.21.15.1:0/539678448 shutdown_connections 2024-09-11T21:57:21.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.410+0000 7f0fbbfff640 1 --2- 172.21.15.1:0/539678448 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0fb40a2040 0x7f0fb40a4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:21.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.410+0000 7f0fbbfff640 1 --2- 172.21.15.1:0/539678448 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0fb40066d0 0x7f0fb4006b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:21.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.410+0000 7f0fbbfff640 1 --2- 172.21.15.1:0/539678448 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0fb4096590 0x7f0fb4006190 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:21.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.410+0000 7f0fbbfff640 1 -- 172.21.15.1:0/539678448 >> 172.21.15.1:0/539678448 conn(0x7f0fb400a360 msgr2=0x7f0fb400a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:21.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.410+0000 7f0fbbfff640 1 -- 172.21.15.1:0/539678448 shutdown_connections 2024-09-11T21:57:21.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.410+0000 7f0fbbfff640 1 -- 172.21.15.1:0/539678448 wait complete. 2024-09-11T21:57:21.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.411+0000 7f0fbbfff640 1 Processor -- start 2024-09-11T21:57:21.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.411+0000 7f0fbbfff640 1 -- start start 2024-09-11T21:57:21.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.411+0000 7f0fbbfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0fb40066d0 0x7f0fb412e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:21.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.411+0000 7f0fbbfff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0fb4096590 0x7f0fb412e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:21.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.411+0000 7f0fbbfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0fb40a2040 0x7f0fb4132ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:21.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.411+0000 7f0fbbfff640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb40a7c40 con 0x7f0fb40066d0 2024-09-11T21:57:21.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.411+0000 7f0fbbfff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb40a7ac0 con 0x7f0fb40a2040 2024-09-11T21:57:21.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.411+0000 7f0fbbfff640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb40a7dc0 con 0x7f0fb4096590 2024-09-11T21:57:21.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.411+0000 7f0fbb7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0fb40a2040 0x7f0fb4132ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:21.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.411+0000 7f0fba7fc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0fb4096590 0x7f0fb412e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:21.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.411+0000 7f0fbaffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0fb40066d0 0x7f0fb412e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:21.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.411+0000 7f0fbb7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0fb40a2040 0x7f0fb4132ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:33782/0 (socket says 172.21.15.1:33782) 2024-09-11T21:57:21.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.411+0000 7f0fbb7fe640 1 -- 172.21.15.1:0/3827476397 learned_addr learned my addr 172.21.15.1:0/3827476397 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:21.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.412+0000 7f0fbaffd640 1 -- 172.21.15.1:0/3827476397 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0fb4096590 msgr2=0x7f0fb412e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:21.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.412+0000 7f0fbaffd640 1 --2- 172.21.15.1:0/3827476397 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0fb4096590 0x7f0fb412e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:21.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.412+0000 7f0fbaffd640 1 -- 172.21.15.1:0/3827476397 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0fb40a2040 msgr2=0x7f0fb4132ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:21.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.412+0000 7f0fbaffd640 1 --2- 172.21.15.1:0/3827476397 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0fb40a2040 0x7f0fb4132ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:21.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.412+0000 7f0fbaffd640 1 -- 172.21.15.1:0/3827476397 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fb41331e0 con 0x7f0fb40066d0 2024-09-11T21:57:21.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.412+0000 7f0fbb7fe640 1 --2- 172.21.15.1:0/3827476397 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0fb40a2040 0x7f0fb4132ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T21:57:21.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.412+0000 7f0fbaffd640 1 --2- 172.21.15.1:0/3827476397 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0fb40066d0 0x7f0fb412e1f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f0fac005940 tx=0x7f0fac004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:21.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.412+0000 7f0fa3fff640 1 -- 172.21.15.1:0/3827476397 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fac03f030 con 0x7f0fb40066d0 2024-09-11T21:57:21.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.412+0000 7f0fa3fff640 1 -- 172.21.15.1:0/3827476397 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0fac003ad0 con 0x7f0fb40066d0 2024-09-11T21:57:21.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.412+0000 7f0fbbfff640 1 -- 172.21.15.1:0/3827476397 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fb4133470 con 0x7f0fb40066d0 2024-09-11T21:57:21.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.412+0000 7f0fa3fff640 1 -- 172.21.15.1:0/3827476397 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fac035070 con 0x7f0fb40066d0 2024-09-11T21:57:21.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.412+0000 7f0fbbfff640 1 -- 172.21.15.1:0/3827476397 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fb413ad60 con 0x7f0fb40066d0 2024-09-11T21:57:21.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.413+0000 7f0fbbfff640 1 -- 172.21.15.1:0/3827476397 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fb4006b10 con 0x7f0fb40066d0 2024-09-11T21:57:21.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.415+0000 7f0fa3fff640 1 -- 172.21.15.1:0/3827476397 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 24) ==== 138008+0+0 (secure 0 0 0) 0x7f0fac042030 con 0x7f0fb40066d0 2024-09-11T21:57:21.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.415+0000 7f0fa3fff640 1 --2- 172.21.15.1:0/3827476397 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f0fa40a2e10 0x7f0fa40a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:21.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.415+0000 7f0fa3fff640 1 -- 172.21.15.1:0/3827476397 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7f0fac0efde0 con 0x7f0fb40066d0 2024-09-11T21:57:21.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.416+0000 7f0fba7fc640 1 --2- 172.21.15.1:0/3827476397 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f0fa40a2e10 0x7f0fa40a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:21.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.416+0000 7f0fba7fc640 1 --2- 172.21.15.1:0/3827476397 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f0fa40a2e10 0x7f0fa40a52d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0fb4093fe0 tx=0x7f0fb0005f90 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:21.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.418+0000 7f0fa3fff640 1 -- 172.21.15.1:0/3827476397 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0fac0bc1e0 con 0x7f0fb40066d0 2024-09-11T21:57:21.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.420+0000 7f06209d6740 1 Processor -- start 2024-09-11T21:57:21.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.420+0000 7f06209d6740 1 -- start start 2024-09-11T21:57:21.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.420+0000 7f06209d6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5563a1de1b60 0x5563a1e0d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:21.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.420+0000 7f06209d6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5563a1de2480 0x5563a1e0d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:21.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.420+0000 7f06209d6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5563a1decd40 0x5563a1de6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:21.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.421+0000 7f06209d6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f06181ab770 con 0x5563a1decd40 2024-09-11T21:57:21.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.421+0000 7f061cb9a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5563a1decd40 0x5563a1de6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:21.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.421+0000 7f06177fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5563a1de2480 0x5563a1e0d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:21.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.421+0000 7f061cb9a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5563a1decd40 0x5563a1de6ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38778/0 (socket says 172.21.15.1:38778) 2024-09-11T21:57:21.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.421+0000 7f0617fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5563a1de1b60 0x5563a1e0d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:21.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.421+0000 7f061cb9a640 1 -- 172.21.15.1:0/3055514805 learned_addr learned my addr 172.21.15.1:0/3055514805 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:21.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.421+0000 7f06209d6740 1 -- 172.21.15.1:0/3055514805 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f06181ad2b0 con 0x5563a1de2480 2024-09-11T21:57:21.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.421+0000 7f06209d6740 1 -- 172.21.15.1:0/3055514805 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f06181abe40 con 0x5563a1de1b60 2024-09-11T21:57:21.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.421+0000 7f061cb9a640 1 -- 172.21.15.1:0/3055514805 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5563a1de1b60 msgr2=0x5563a1e0d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:21.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.421+0000 7f061cb9a640 1 --2- 172.21.15.1:0/3055514805 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5563a1de1b60 0x5563a1e0d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:21.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.421+0000 7f061cb9a640 1 -- 172.21.15.1:0/3055514805 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5563a1de2480 msgr2=0x5563a1e0d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:21.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.421+0000 7f061cb9a640 1 --2- 172.21.15.1:0/3055514805 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5563a1de2480 0x5563a1e0d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:21.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.421+0000 7f061cb9a640 1 -- 172.21.15.1:0/3055514805 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563a1ddf260 con 0x5563a1decd40 2024-09-11T21:57:21.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.421+0000 7f0617fff640 1 --2- 172.21.15.1:0/3055514805 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5563a1de1b60 0x5563a1e0d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T21:57:21.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.421+0000 7f061cb9a640 1 --2- 172.21.15.1:0/3055514805 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5563a1decd40 0x5563a1de6ee0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f0610013690 tx=0x7f06100136c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd713e9ed5cada2f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:21.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.422+0000 7f05fdffb640 1 -- 172.21.15.1:0/3055514805 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0610029070 con 0x5563a1decd40 2024-09-11T21:57:21.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.422+0000 7f05fdffb640 1 -- 172.21.15.1:0/3055514805 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f061000e880 con 0x5563a1decd40 2024-09-11T21:57:21.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.422+0000 7f05fdffb640 1 -- 172.21.15.1:0/3055514805 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0610019030 con 0x5563a1decd40 2024-09-11T21:57:21.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.423+0000 7f06209d6740 1 -- 172.21.15.1:0/3055514805 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5563a1decd40 msgr2=0x5563a1de6ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:21.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.423+0000 7f06209d6740 1 --2- 172.21.15.1:0/3055514805 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5563a1decd40 0x5563a1de6ee0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f0610013690 tx=0x7f06100136c0 comp rx=0 tx=0).stop 2024-09-11T21:57:21.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.423+0000 7f06209d6740 1 -- 172.21.15.1:0/3055514805 shutdown_connections 2024-09-11T21:57:21.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.423+0000 7f06209d6740 1 --2- 172.21.15.1:0/3055514805 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5563a1decd40 0x5563a1de6ee0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:21.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.423+0000 7f06209d6740 1 --2- 172.21.15.1:0/3055514805 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5563a1de2480 0x5563a1e0d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:21.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.423+0000 7f06209d6740 1 --2- 172.21.15.1:0/3055514805 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5563a1de1b60 0x5563a1e0d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:21.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.423+0000 7f06209d6740 1 -- 172.21.15.1:0/3055514805 >> 172.21.15.1:0/3055514805 conn(0x5563a1dde0f0 msgr2=0x5563a1de3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:21.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.424+0000 7f06209d6740 1 -- 172.21.15.1:0/3055514805 shutdown_connections 2024-09-11T21:57:21.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.424+0000 7f06209d6740 1 -- 172.21.15.1:0/3055514805 wait complete. 2024-09-11T21:57:21.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.424+0000 7f06209d6740 1 -- start start 2024-09-11T21:57:21.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.425+0000 7f06209d6740 1 Processor -- start 2024-09-11T21:57:21.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.425+0000 7f06209d6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5563a1de1b60 0x5563a1dfd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:21.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.425+0000 7f06209d6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5563a1de2480 0x5563a1dfdab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:21.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.425+0000 7f0617fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5563a1de1b60 0x5563a1dfd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:21.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.425+0000 7f06209d6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5563a1decd40 0x5563a1dfa540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:21.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.425+0000 7f0617fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5563a1de1b60 0x5563a1dfd570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:33802/0 (socket says 172.21.15.1:33802) 2024-09-11T21:57:21.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.426+0000 7f06177fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5563a1de2480 0x5563a1dfdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:21.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.426+0000 7f06177fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5563a1de2480 0x5563a1dfdab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38790/0 (socket says 172.21.15.1:38790) 2024-09-11T21:57:21.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.426+0000 7f0617fff640 1 -- 172.21.15.1:0/365259473 learned_addr learned my addr 172.21.15.1:0/365259473 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:21.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.426+0000 7f06209d6740 1 -- 172.21.15.1:0/365259473 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f06181acbe0 con 0x5563a1de2480 2024-09-11T21:57:21.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.426+0000 7f06209d6740 1 -- 172.21.15.1:0/365259473 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f06181ac510 con 0x5563a1de1b60 2024-09-11T21:57:21.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.426+0000 7f06209d6740 1 -- 172.21.15.1:0/365259473 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f06181a9770 con 0x5563a1decd40 2024-09-11T21:57:21.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.426+0000 7f0617fff640 1 -- 172.21.15.1:0/365259473 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5563a1decd40 msgr2=0x5563a1dfa540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:21.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.426+0000 7f061cb9a640 1 --2- 172.21.15.1:0/365259473 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5563a1decd40 0x5563a1dfa540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:21.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.426+0000 7f0617fff640 1 --2- 172.21.15.1:0/365259473 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5563a1decd40 0x5563a1dfa540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:21.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.426+0000 7f0617fff640 1 -- 172.21.15.1:0/365259473 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5563a1de2480 msgr2=0x5563a1dfdab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:21.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.426+0000 7f0617fff640 1 --2- 172.21.15.1:0/365259473 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5563a1de2480 0x5563a1dfdab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:21.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.426+0000 7f0617fff640 1 -- 172.21.15.1:0/365259473 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563a1dfe1f0 con 0x5563a1de1b60 2024-09-11T21:57:21.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.426+0000 7f06209d6740 10 client.0 _ll_register_callbacks cb 0x7f0608005de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:57:21.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.426+0000 7f06209d6740 1 -- 172.21.15.1:0/365259473 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f061000e880 con 0x5563a1de1b60 2024-09-11T21:57:21.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.426+0000 7f0617fff640 1 --2- 172.21.15.1:0/365259473 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5563a1de1b60 0x5563a1dfd570 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0608009850 tx=0x7f060803df80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39e7bc1ed240fca8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:21.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.427+0000 7f05f77fe640 10 client.25013 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-11T21:57:21.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.427+0000 7f05f77fe640 1 -- 172.21.15.1:0/365259473 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f060803d860 con 0x5563a1de1b60 2024-09-11T21:57:21.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.427+0000 7f05f77fe640 1 -- 172.21.15.1:0/365259473 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7f0608005030 con 0x5563a1de1b60 2024-09-11T21:57:21.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.427+0000 7f05f77fe640 1 -- 172.21.15.1:0/365259473 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0608005270 con 0x5563a1de1b60 2024-09-11T21:57:21.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.427+0000 7f06209d6740 10 client.25013 fetch_fsmap learned FSMap version 28 2024-09-11T21:57:21.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.427+0000 7f06209d6740 1 -- 172.21.15.1:0/365259473 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({fsmap=28}) -- 0x5563a1e70560 con 0x5563a1de1b60 2024-09-11T21:57:21.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.427+0000 7f05f77fe640 1 -- 172.21.15.1:0/365259473 <== mon.2 v2:172.21.15.53:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f0608005410 con 0x5563a1de1b60 2024-09-11T21:57:21.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.427+0000 7f06209d6740 10 client.25013 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T21:57:21.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.427+0000 7f06209d6740 10 client.25013 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T21:57:21.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.427+0000 7f06209d6740 20 client.25013 populate_metadata read hostname 'smithi001' 2024-09-11T21:57:21.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.428+0000 7f06209d6740 1 --2- 172.21.15.1:0/365259473 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5563a1e70b70 0x5563a1e72f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:21.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.428+0000 7f06209d6740 4 client.25013 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T21:57:21.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.428+0000 7f06209d6740 1 -- 172.21.15.1:0/365259473 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5563a1d4ed00 con 0x5563a1e70b70 2024-09-11T21:57:21.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.428+0000 7f06177fe640 1 --2- 172.21.15.1:0/365259473 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5563a1e70b70 0x5563a1e72f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:21.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.429+0000 7f05f77fe640 10 client.25013 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T21:57:21.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.429+0000 7f06177fe640 1 --2- 172.21.15.1:0/365259473 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5563a1e70b70 0x5563a1e72f90 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7db03db38486798e server_cookie=a24beae9c32a17a9 in_seq=0 out_seq=0 2024-09-11T21:57:21.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.430+0000 7f05f77fe640 1 -- 172.21.15.1:0/365259473 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5563a1d4ed00 con 0x5563a1e70b70 2024-09-11T21:57:21.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.430+0000 7f05f77fe640 10 client.25013 handle_command_reply: tid=0 2024-09-11T21:57:21.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.430+0000 7f06209d6740 1 client.25013 shutdown 2024-09-11T21:57:21.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.430+0000 7f06209d6740 1 -- 172.21.15.1:0/365259473 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5563a1de1b60 msgr2=0x5563a1dfd570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:21.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.430+0000 7f06209d6740 1 --2- 172.21.15.1:0/365259473 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5563a1de1b60 0x5563a1dfd570 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0608009850 tx=0x7f060803df80 comp rx=0 tx=0).stop 2024-09-11T21:57:21.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.430+0000 7f06209d6740 1 -- 172.21.15.1:0/365259473 shutdown_connections 2024-09-11T21:57:21.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.430+0000 7f06209d6740 1 --2- 172.21.15.1:0/365259473 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5563a1e70b70 0x5563a1e72f90 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:21.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.430+0000 7f06209d6740 1 --2- 172.21.15.1:0/365259473 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5563a1decd40 0x5563a1dfa540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:21.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.430+0000 7f06209d6740 1 --2- 172.21.15.1:0/365259473 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5563a1de2480 0x5563a1dfdab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:21.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.430+0000 7f05f77fe640 0 client.25013 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T21:57:21.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.430+0000 7f06209d6740 1 --2- 172.21.15.1:0/365259473 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5563a1de1b60 0x5563a1dfd570 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:21.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.430+0000 7f06209d6740 1 -- 172.21.15.1:0/365259473 >> 172.21.15.1:0/365259473 conn(0x5563a1dde0f0 msgr2=0x5563a1de3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:21.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.431+0000 7f06209d6740 1 -- 172.21.15.1:0/365259473 shutdown_connections 2024-09-11T21:57:21.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.431+0000 7f06209d6740 1 -- 172.21.15.1:0/365259473 wait complete. 2024-09-11T21:57:21.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.431+0000 7f06209d6740 20 client.25013 trim_cache size 0 max 16384 2024-09-11T21:57:21.433 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-11T21:57:21.433 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-09-11T21:57:21.433 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "02307e43-eeeb-4387-84d1-e1da69e7046b", 2024-09-11T21:57:21.433 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-09-11T21:57:21.433 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-11T21:57:21.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.432+0000 7f061e5fd640 1 -- 172.21.15.1:0/3641744710 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f06000a2e00 msgr2=0x7f06000a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:21.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.432+0000 7f061e5fd640 1 --2- 172.21.15.1:0/3641744710 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f06000a2e00 0x7f06000a52c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0610010fd0 tx=0x7f0610015000 comp rx=0 tx=0).stop 2024-09-11T21:57:21.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.432+0000 7f061e5fd640 1 -- 172.21.15.1:0/3641744710 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f06181037a0 msgr2=0x7f0618100f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:21.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.432+0000 7f061e5fd640 1 --2- 172.21.15.1:0/3641744710 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f06181037a0 0x7f0618100f90 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f060c00d950 tx=0x7f060c00de10 comp rx=0 tx=0).stop 2024-09-11T21:57:21.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.432+0000 7f061e5fd640 1 -- 172.21.15.1:0/3641744710 shutdown_connections 2024-09-11T21:57:21.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.432+0000 7f061e5fd640 1 --2- 172.21.15.1:0/3641744710 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f06000a2e00 0x7f06000a52c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:21.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.432+0000 7f061e5fd640 1 --2- 172.21.15.1:0/3641744710 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f061810c2f0 0x7f06181014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:21.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.432+0000 7f061e5fd640 1 --2- 172.21.15.1:0/3641744710 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f06181037a0 0x7f0618100f90 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:21.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.432+0000 7f061e5fd640 1 --2- 172.21.15.1:0/3641744710 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0618102df0 0x7f0618100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:21.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.433+0000 7f061e5fd640 1 -- 172.21.15.1:0/3641744710 >> 172.21.15.1:0/3641744710 conn(0x7f06180fc7d0 msgr2=0x7f06180fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:21.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.433+0000 7f061e5fd640 1 -- 172.21.15.1:0/3641744710 shutdown_connections 2024-09-11T21:57:21.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.433+0000 7f061e5fd640 1 -- 172.21.15.1:0/3641744710 wait complete. 2024-09-11T21:57:21.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.603+0000 7f0fbbfff640 1 -- 172.21.15.1:0/3827476397 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f0fb413af30 con 0x7f0fb40066d0 2024-09-11T21:57:21.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.607+0000 7f0fa3fff640 1 -- 172.21.15.1:0/3827476397 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v32)=0 v32) ==== 133+0+0 (secure 0 0 0) 0x7f0fac0c04e0 con 0x7f0fb40066d0 2024-09-11T21:57:21.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.610+0000 7f0fbbfff640 1 -- 172.21.15.1:0/3827476397 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f0fa40a2e10 msgr2=0x7f0fa40a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:21.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.610+0000 7f0fbbfff640 1 --2- 172.21.15.1:0/3827476397 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f0fa40a2e10 0x7f0fa40a52d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0fb4093fe0 tx=0x7f0fb0005f90 comp rx=0 tx=0).stop 2024-09-11T21:57:21.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.610+0000 7f0fbbfff640 1 -- 172.21.15.1:0/3827476397 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0fb40066d0 msgr2=0x7f0fb412e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:21.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.610+0000 7f0fbbfff640 1 --2- 172.21.15.1:0/3827476397 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0fb40066d0 0x7f0fb412e1f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f0fac005940 tx=0x7f0fac004270 comp rx=0 tx=0).stop 2024-09-11T21:57:21.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.610+0000 7f0fbbfff640 1 -- 172.21.15.1:0/3827476397 shutdown_connections 2024-09-11T21:57:21.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.610+0000 7f0fbbfff640 1 --2- 172.21.15.1:0/3827476397 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f0fa40a2e10 0x7f0fa40a52d0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:21.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.610+0000 7f0fbbfff640 1 --2- 172.21.15.1:0/3827476397 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0fb40a2040 0x7f0fb4132ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:21.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.610+0000 7f0fbbfff640 1 --2- 172.21.15.1:0/3827476397 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0fb4096590 0x7f0fb412e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:21.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.610+0000 7f0fbbfff640 1 --2- 172.21.15.1:0/3827476397 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0fb40066d0 0x7f0fb412e1f0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:21.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.610+0000 7f0fbbfff640 1 -- 172.21.15.1:0/3827476397 >> 172.21.15.1:0/3827476397 conn(0x7f0fb400a360 msgr2=0x7f0fb40a2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:21.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.610+0000 7f0fbbfff640 1 -- 172.21.15.1:0/3827476397 shutdown_connections 2024-09-11T21:57:21.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.610+0000 7f0fbbfff640 1 -- 172.21.15.1:0/3827476397 wait complete. 2024-09-11T21:57:21.622 DEBUG:teuthology.orchestra.run.smithi001:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-09-11T21:57:21.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.739+0000 7fd66ddc6640 1 -- 172.21.15.1:0/3736996260 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd668101340 msgr2=0x7fd66810d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:21.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.739+0000 7fd66ddc6640 1 --2- 172.21.15.1:0/3736996260 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd668101340 0x7fd66810d870 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd660009cd0 tx=0x7fd66002a8e0 comp rx=0 tx=0).stop 2024-09-11T21:57:21.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.739+0000 7fd66ddc6640 1 -- 172.21.15.1:0/3736996260 shutdown_connections 2024-09-11T21:57:21.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.739+0000 7fd66ddc6640 1 --2- 172.21.15.1:0/3736996260 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd668101340 0x7fd66810d870 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:21.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.739+0000 7fd66ddc6640 1 --2- 172.21.15.1:0/3736996260 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6681009a0 0x7fd668100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:21.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.739+0000 7fd66ddc6640 1 --2- 172.21.15.1:0/3736996260 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6681069b0 0x7fd668106d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:21.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.739+0000 7fd66ddc6640 1 -- 172.21.15.1:0/3736996260 >> 172.21.15.1:0/3736996260 conn(0x7fd6680779a0 msgr2=0x7fd6680fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:21.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.739+0000 7fd66ddc6640 1 -- 172.21.15.1:0/3736996260 shutdown_connections 2024-09-11T21:57:21.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.739+0000 7fd66ddc6640 1 -- 172.21.15.1:0/3736996260 wait complete. 2024-09-11T21:57:21.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.739+0000 7fd66ddc6640 1 Processor -- start 2024-09-11T21:57:21.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.740+0000 7fd66ddc6640 1 -- start start 2024-09-11T21:57:21.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.740+0000 7fd66ddc6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd6681009a0 0x7fd668199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:21.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.740+0000 7fd66ddc6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd668101340 0x7fd6681997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:21.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.740+0000 7fd66ddc6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6681069b0 0x7fd6681a4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:21.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.740+0000 7fd66ddc6640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd668110df0 con 0x7fd6681009a0 2024-09-11T21:57:21.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.740+0000 7fd66ddc6640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd668110c70 con 0x7fd6681069b0 2024-09-11T21:57:21.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.740+0000 7fd66ddc6640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd668110f70 con 0x7fd668101340 2024-09-11T21:57:21.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.740+0000 7fd6677fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd6681009a0 0x7fd668199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:21.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.740+0000 7fd6677fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd6681009a0 0x7fd668199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38802/0 (socket says 172.21.15.1:38802) 2024-09-11T21:57:21.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.740+0000 7fd6677fe640 1 -- 172.21.15.1:0/3747046138 learned_addr learned my addr 172.21.15.1:0/3747046138 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:21.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.740+0000 7fd65effd640 1 --2- 172.21.15.1:0/3747046138 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd668101340 0x7fd6681997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:21.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.740+0000 7fd667fff640 1 --2- 172.21.15.1:0/3747046138 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6681069b0 0x7fd6681a4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:21.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.740+0000 7fd6677fe640 1 -- 172.21.15.1:0/3747046138 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd668101340 msgr2=0x7fd6681997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:21.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.740+0000 7fd6677fe640 1 --2- 172.21.15.1:0/3747046138 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd668101340 0x7fd6681997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:21.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.740+0000 7fd6677fe640 1 -- 172.21.15.1:0/3747046138 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6681069b0 msgr2=0x7fd6681a4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:21.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.740+0000 7fd6677fe640 1 --2- 172.21.15.1:0/3747046138 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6681069b0 0x7fd6681a4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:21.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.740+0000 7fd6677fe640 1 -- 172.21.15.1:0/3747046138 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6681a4840 con 0x7fd6681009a0 2024-09-11T21:57:21.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.741+0000 7fd6677fe640 1 --2- 172.21.15.1:0/3747046138 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd6681009a0 0x7fd668199280 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd65800e9f0 tx=0x7fd65800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:21.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.741+0000 7fd6657fa640 1 -- 172.21.15.1:0/3747046138 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd65800cd80 con 0x7fd6681009a0 2024-09-11T21:57:21.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.741+0000 7fd6657fa640 1 -- 172.21.15.1:0/3747046138 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd658005500 con 0x7fd6681009a0 2024-09-11T21:57:21.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.741+0000 7fd66ddc6640 1 -- 172.21.15.1:0/3747046138 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6681a4a10 con 0x7fd6681009a0 2024-09-11T21:57:21.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.741+0000 7fd6657fa640 1 -- 172.21.15.1:0/3747046138 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd658010690 con 0x7fd6681009a0 2024-09-11T21:57:21.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.741+0000 7fd66ddc6640 1 -- 172.21.15.1:0/3747046138 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6681a4f30 con 0x7fd6681009a0 2024-09-11T21:57:21.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.742+0000 7fd66ddc6640 1 -- 172.21.15.1:0/3747046138 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6681020e0 con 0x7fd6681009a0 2024-09-11T21:57:21.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.744+0000 7fd6657fa640 1 -- 172.21.15.1:0/3747046138 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 24) ==== 138008+0+0 (secure 0 0 0) 0x7fd6580046c0 con 0x7fd6681009a0 2024-09-11T21:57:21.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.745+0000 7fd6657fa640 1 --2- 172.21.15.1:0/3747046138 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fd6540a2e00 0x7fd6540a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:21.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.745+0000 7fd65effd640 1 --2- 172.21.15.1:0/3747046138 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fd6540a2e00 0x7fd6540a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:21.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.745+0000 7fd6657fa640 1 -- 172.21.15.1:0/3747046138 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7fd6580cfb10 con 0x7fd6681009a0 2024-09-11T21:57:21.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.745+0000 7fd65effd640 1 --2- 172.21.15.1:0/3747046138 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fd6540a2e00 0x7fd6540a52c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd66806a560 tx=0x7fd65000b390 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:21.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.748+0000 7fd6657fa640 1 -- 172.21.15.1:0/3747046138 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd6580541b0 con 0x7fd6681009a0 2024-09-11T21:57:21.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:21.948+0000 7fd66ddc6640 1 -- 172.21.15.1:0/3747046138 --> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7fd668062630 con 0x7fd6540a2e00 2024-09-11T21:57:22.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.012+0000 7fd6657fa640 1 -- 172.21.15.1:0/3747046138 <== mgr.15204 v2:172.21.15.1:6800/3543150772 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fd668062630 con 0x7fd6540a2e00 2024-09-11T21:57:22.013 INFO:teuthology.orchestra.run.smithi001.stdout:Schedule set for path / 2024-09-11T21:57:22.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.014+0000 7fd66ddc6640 1 -- 172.21.15.1:0/3747046138 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fd6540a2e00 msgr2=0x7fd6540a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:22.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.014+0000 7fd66ddc6640 1 --2- 172.21.15.1:0/3747046138 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fd6540a2e00 0x7fd6540a52c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd66806a560 tx=0x7fd65000b390 comp rx=0 tx=0).stop 2024-09-11T21:57:22.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.014+0000 7fd66ddc6640 1 -- 172.21.15.1:0/3747046138 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd6681009a0 msgr2=0x7fd668199280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:22.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.014+0000 7fd66ddc6640 1 --2- 172.21.15.1:0/3747046138 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd6681009a0 0x7fd668199280 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd65800e9f0 tx=0x7fd65800edf0 comp rx=0 tx=0).stop 2024-09-11T21:57:22.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.015+0000 7fd66ddc6640 1 -- 172.21.15.1:0/3747046138 shutdown_connections 2024-09-11T21:57:22.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.015+0000 7fd66ddc6640 1 --2- 172.21.15.1:0/3747046138 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fd6540a2e00 0x7fd6540a52c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:22.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.015+0000 7fd66ddc6640 1 --2- 172.21.15.1:0/3747046138 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd6681069b0 0x7fd6681a4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:22.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.015+0000 7fd66ddc6640 1 --2- 172.21.15.1:0/3747046138 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd668101340 0x7fd6681997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:22.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.015+0000 7fd66ddc6640 1 --2- 172.21.15.1:0/3747046138 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd6681009a0 0x7fd668199280 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:22.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.015+0000 7fd66ddc6640 1 -- 172.21.15.1:0/3747046138 >> 172.21.15.1:0/3747046138 conn(0x7fd6680779a0 msgr2=0x7fd668103970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:22.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.015+0000 7fd66ddc6640 1 -- 172.21.15.1:0/3747046138 shutdown_connections 2024-09-11T21:57:22.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.015+0000 7fd66ddc6640 1 -- 172.21.15.1:0/3747046138 wait complete. 2024-09-11T21:57:22.026 DEBUG:teuthology.orchestra.run.smithi001:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-09-11T21:57:22.093 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T21:57:22.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.136+0000 7fb2ebd6d640 1 -- 172.21.15.1:0/2316038086 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb2e4102d00 msgr2=0x7fb2e41030e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:22.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.136+0000 7fb2ebd6d640 1 --2- 172.21.15.1:0/2316038086 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb2e4102d00 0x7fb2e41030e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0009900 tx=0x7fb2e002a480 comp rx=0 tx=0).stop 2024-09-11T21:57:22.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.136+0000 7fb2ebd6d640 1 -- 172.21.15.1:0/2316038086 shutdown_connections 2024-09-11T21:57:22.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.136+0000 7fb2ebd6d640 1 --2- 172.21.15.1:0/2316038086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2e410c0d0 0x7fb2e410e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:22.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.136+0000 7fb2ebd6d640 1 --2- 172.21.15.1:0/2316038086 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2e4103620 0x7fb2e410bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:22.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.136+0000 7fb2ebd6d640 1 --2- 172.21.15.1:0/2316038086 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb2e4102d00 0x7fb2e41030e0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:22.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.136+0000 7fb2ebd6d640 1 -- 172.21.15.1:0/2316038086 >> 172.21.15.1:0/2316038086 conn(0x7fb2e40779a0 msgr2=0x7fb2e40fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:22.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.136+0000 7fb2ebd6d640 1 -- 172.21.15.1:0/2316038086 shutdown_connections 2024-09-11T21:57:22.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.136+0000 7fb2ebd6d640 1 -- 172.21.15.1:0/2316038086 wait complete. 2024-09-11T21:57:22.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.137+0000 7fb2ebd6d640 1 Processor -- start 2024-09-11T21:57:22.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.137+0000 7fb2ebd6d640 1 -- start start 2024-09-11T21:57:22.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.137+0000 7fb2ebd6d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2e4102d00 0x7fb2e41983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:22.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.137+0000 7fb2ebd6d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb2e4103620 0x7fb2e4198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:22.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.137+0000 7fb2ebd6d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2e410c0d0 0x7fb2e419cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:22.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.137+0000 7fb2ebd6d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e4111ef0 con 0x7fb2e4103620 2024-09-11T21:57:22.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.137+0000 7fb2ebd6d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e4111d70 con 0x7fb2e4102d00 2024-09-11T21:57:22.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.137+0000 7fb2ebd6d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e4112070 con 0x7fb2e410c0d0 2024-09-11T21:57:22.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.137+0000 7fb2e9308640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb2e4103620 0x7fb2e4198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:22.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.137+0000 7fb2e9308640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb2e4103620 0x7fb2e4198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38822/0 (socket says 172.21.15.1:38822) 2024-09-11T21:57:22.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.137+0000 7fb2e9308640 1 -- 172.21.15.1:0/3456425538 learned_addr learned my addr 172.21.15.1:0/3456425538 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:22.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.137+0000 7fb2e9b09640 1 --2- 172.21.15.1:0/3456425538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2e4102d00 0x7fb2e41983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:22.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.137+0000 7fb2ea30a640 1 --2- 172.21.15.1:0/3456425538 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2e410c0d0 0x7fb2e419cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:22.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.137+0000 7fb2e9308640 1 -- 172.21.15.1:0/3456425538 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2e410c0d0 msgr2=0x7fb2e419cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:22.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.137+0000 7fb2e9308640 1 --2- 172.21.15.1:0/3456425538 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2e410c0d0 0x7fb2e419cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:22.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.137+0000 7fb2e9308640 1 -- 172.21.15.1:0/3456425538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2e4102d00 msgr2=0x7fb2e41983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:22.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.137+0000 7fb2e9308640 1 --2- 172.21.15.1:0/3456425538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2e4102d00 0x7fb2e41983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:22.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.137+0000 7fb2e9308640 1 -- 172.21.15.1:0/3456425538 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2e419d3b0 con 0x7fb2e4103620 2024-09-11T21:57:22.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.138+0000 7fb2e9308640 1 --2- 172.21.15.1:0/3456425538 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb2e4103620 0x7fb2e4198900 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb2d400d950 tx=0x7fb2d400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:22.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.138+0000 7fb2daffd640 1 -- 172.21.15.1:0/3456425538 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d4017030 con 0x7fb2e4103620 2024-09-11T21:57:22.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.138+0000 7fb2daffd640 1 -- 172.21.15.1:0/3456425538 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2d4004020 con 0x7fb2e4103620 2024-09-11T21:57:22.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.138+0000 7fb2ebd6d640 1 -- 172.21.15.1:0/3456425538 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2e419d6a0 con 0x7fb2e4103620 2024-09-11T21:57:22.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.138+0000 7fb2daffd640 1 -- 172.21.15.1:0/3456425538 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d40047a0 con 0x7fb2e4103620 2024-09-11T21:57:22.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.138+0000 7fb2ebd6d640 1 -- 172.21.15.1:0/3456425538 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2e41a4f30 con 0x7fb2e4103620 2024-09-11T21:57:22.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.139+0000 7fb2ebd6d640 1 -- 172.21.15.1:0/3456425538 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2e406a260 con 0x7fb2e4103620 2024-09-11T21:57:22.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.141+0000 7fb2daffd640 1 -- 172.21.15.1:0/3456425538 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 24) ==== 138008+0+0 (secure 0 0 0) 0x7fb2d40041d0 con 0x7fb2e4103620 2024-09-11T21:57:22.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.142+0000 7fb2daffd640 1 --2- 172.21.15.1:0/3456425538 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fb2d00a2e00 0x7fb2d00a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:22.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.143+0000 7fb2e9b09640 1 --2- 172.21.15.1:0/3456425538 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fb2d00a2e00 0x7fb2d00a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:22.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.143+0000 7fb2daffd640 1 -- 172.21.15.1:0/3456425538 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7fb2d40cf5a0 con 0x7fb2e4103620 2024-09-11T21:57:22.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.143+0000 7fb2e9b09640 1 --2- 172.21.15.1:0/3456425538 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fb2d00a2e00 0x7fb2d00a52c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0000c00 tx=0x7fb2e0005b50 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:22.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.145+0000 7fb2daffd640 1 -- 172.21.15.1:0/3456425538 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb2d409c0d0 con 0x7fb2e4103620 2024-09-11T21:57:22.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:21 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/3020876212' entity='client.admin' 2024-09-11T21:57:22.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:21 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:57:22.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:21 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:57:22.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:21 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:57:22.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:21 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:57:22.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:21 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/2034795584' entity='client.admin' 2024-09-11T21:57:22.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:21 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:57:22.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:21 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:57:22.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:21 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:57:22.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:21 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:57:22.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:21 smithi053 ceph-mon[34908]: scrub queued for path: / 2024-09-11T21:57:22.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:21 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T21:57:22.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:21 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T21:57:22.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:21 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T21:57:22.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:21 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/3827476397' entity='client.admin' 2024-09-11T21:57:22.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:21 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:57:22.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:21 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:57:22.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:21 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:57:22.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:21 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:57:22.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:21 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/3020876212' entity='client.admin' 2024-09-11T21:57:22.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:21 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:57:22.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:21 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:57:22.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:21 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:57:22.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:21 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:57:22.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:21 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/2034795584' entity='client.admin' 2024-09-11T21:57:22.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:21 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:57:22.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:21 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:57:22.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:21 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:57:22.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:21 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:57:22.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:21 smithi161 ceph-mon[35615]: scrub queued for path: / 2024-09-11T21:57:22.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:21 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T21:57:22.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:21 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T21:57:22.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:21 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T21:57:22.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:21 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/3827476397' entity='client.admin' 2024-09-11T21:57:22.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:21 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:57:22.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:21 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:57:22.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:21 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:57:22.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:21 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:57:22.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.369+0000 7fb2ebd6d640 1 -- 172.21.15.1:0/3456425538 --> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7fb2e4062630 con 0x7fb2d00a2e00 2024-09-11T21:57:22.374 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:21 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/3020876212' entity='client.admin' 2024-09-11T21:57:22.374 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:21 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:57:22.374 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:21 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:57:22.374 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:21 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:57:22.374 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:21 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:57:22.374 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:21 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/2034795584' entity='client.admin' 2024-09-11T21:57:22.374 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:21 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:57:22.375 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:21 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:57:22.375 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:21 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:57:22.375 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:21 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:57:22.375 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:21 smithi001 ceph-mon[30935]: scrub queued for path: / 2024-09-11T21:57:22.375 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:21 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T21:57:22.375 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:21 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T21:57:22.375 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:21 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T21:57:22.375 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:21 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/3827476397' entity='client.admin' 2024-09-11T21:57:22.375 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:21 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:57:22.375 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:21 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:57:22.375 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:21 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:57:22.375 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:21 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:57:22.376 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:57:22.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.432+0000 7fb2daffd640 1 -- 172.21.15.1:0/3456425538 <== mgr.15204 v2:172.21.15.1:6800/3543150772 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7fb2e4062630 con 0x7fb2d00a2e00 2024-09-11T21:57:22.433 INFO:teuthology.orchestra.run.smithi001.stdout:Retention added to path / 2024-09-11T21:57:22.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.434+0000 7fb2ebd6d640 1 -- 172.21.15.1:0/3456425538 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fb2d00a2e00 msgr2=0x7fb2d00a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:22.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.434+0000 7fb2ebd6d640 1 --2- 172.21.15.1:0/3456425538 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fb2d00a2e00 0x7fb2d00a52c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0000c00 tx=0x7fb2e0005b50 comp rx=0 tx=0).stop 2024-09-11T21:57:22.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.434+0000 7fb2ebd6d640 1 -- 172.21.15.1:0/3456425538 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb2e4103620 msgr2=0x7fb2e4198900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:22.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.434+0000 7fb2ebd6d640 1 --2- 172.21.15.1:0/3456425538 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb2e4103620 0x7fb2e4198900 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb2d400d950 tx=0x7fb2d400de10 comp rx=0 tx=0).stop 2024-09-11T21:57:22.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.434+0000 7fb2ebd6d640 1 -- 172.21.15.1:0/3456425538 shutdown_connections 2024-09-11T21:57:22.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.434+0000 7fb2ebd6d640 1 --2- 172.21.15.1:0/3456425538 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fb2d00a2e00 0x7fb2d00a52c0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:22.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.434+0000 7fb2ebd6d640 1 --2- 172.21.15.1:0/3456425538 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2e410c0d0 0x7fb2e419cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:22.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.434+0000 7fb2ebd6d640 1 --2- 172.21.15.1:0/3456425538 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb2e4103620 0x7fb2e4198900 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:22.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.434+0000 7fb2ebd6d640 1 --2- 172.21.15.1:0/3456425538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb2e4102d00 0x7fb2e41983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:22.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.434+0000 7fb2ebd6d640 1 -- 172.21.15.1:0/3456425538 >> 172.21.15.1:0/3456425538 conn(0x7fb2e40779a0 msgr2=0x7fb2e40fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:22.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.434+0000 7fb2ebd6d640 1 -- 172.21.15.1:0/3456425538 shutdown_connections 2024-09-11T21:57:22.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.434+0000 7fb2ebd6d640 1 -- 172.21.15.1:0/3456425538 wait complete. 2024-09-11T21:57:22.446 DEBUG:teuthology.orchestra.run.smithi001:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-11T21:57:22.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.590+0000 7fa92691a640 1 -- 172.21.15.1:0/1849229972 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa9201035c0 msgr2=0x7fa92010baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:22.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.590+0000 7fa92691a640 1 --2- 172.21.15.1:0/1849229972 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa9201035c0 0x7fa92010baa0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa914009900 tx=0x7fa91402a480 comp rx=0 tx=0).stop 2024-09-11T21:57:22.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.590+0000 7fa92691a640 1 -- 172.21.15.1:0/1849229972 shutdown_connections 2024-09-11T21:57:22.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.590+0000 7fa92691a640 1 --2- 172.21.15.1:0/1849229972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa92010c0f0 0x7fa92010e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:22.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.590+0000 7fa92691a640 1 --2- 172.21.15.1:0/1849229972 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa9201035c0 0x7fa92010baa0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:22.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.590+0000 7fa92691a640 1 --2- 172.21.15.1:0/1849229972 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa920102ca0 0x7fa920103080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:22.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.590+0000 7fa92691a640 1 -- 172.21.15.1:0/1849229972 >> 172.21.15.1:0/1849229972 conn(0x7fa9200fc6b0 msgr2=0x7fa9200fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:22.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.590+0000 7fa92691a640 1 -- 172.21.15.1:0/1849229972 shutdown_connections 2024-09-11T21:57:22.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.590+0000 7fa92691a640 1 -- 172.21.15.1:0/1849229972 wait complete. 2024-09-11T21:57:22.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.591+0000 7fa92691a640 1 Processor -- start 2024-09-11T21:57:22.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.591+0000 7fa92691a640 1 -- start start 2024-09-11T21:57:22.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.591+0000 7fa92691a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa920102ca0 0x7fa9201983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:22.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.591+0000 7fa92691a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa9201035c0 0x7fa920198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:22.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.591+0000 7fa92691a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa92010c0f0 0x7fa92019cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:22.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.591+0000 7fa92691a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa920111ed0 con 0x7fa9201035c0 2024-09-11T21:57:22.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.591+0000 7fa92691a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa920111d50 con 0x7fa920102ca0 2024-09-11T21:57:22.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.591+0000 7fa92691a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa920112050 con 0x7fa92010c0f0 2024-09-11T21:57:22.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.592+0000 7fa91f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa9201035c0 0x7fa920198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:22.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.592+0000 7fa91f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa9201035c0 0x7fa920198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38838/0 (socket says 172.21.15.1:38838) 2024-09-11T21:57:22.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.592+0000 7fa91f7fe640 1 -- 172.21.15.1:0/1061101156 learned_addr learned my addr 172.21.15.1:0/1061101156 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:22.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.592+0000 7fa91ffff640 1 --2- 172.21.15.1:0/1061101156 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa920102ca0 0x7fa9201983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:22.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.592+0000 7fa924eb7640 1 --2- 172.21.15.1:0/1061101156 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa92010c0f0 0x7fa92019cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:22.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.592+0000 7fa91f7fe640 1 -- 172.21.15.1:0/1061101156 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa92010c0f0 msgr2=0x7fa92019cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:22.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.592+0000 7fa91f7fe640 1 --2- 172.21.15.1:0/1061101156 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa92010c0f0 0x7fa92019cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:22.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.592+0000 7fa91f7fe640 1 -- 172.21.15.1:0/1061101156 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa920102ca0 msgr2=0x7fa9201983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:22.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.592+0000 7fa91f7fe640 1 --2- 172.21.15.1:0/1061101156 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa920102ca0 0x7fa9201983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:22.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.592+0000 7fa91f7fe640 1 -- 172.21.15.1:0/1061101156 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa92019d3b0 con 0x7fa9201035c0 2024-09-11T21:57:22.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.592+0000 7fa91f7fe640 1 --2- 172.21.15.1:0/1061101156 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa9201035c0 0x7fa920198900 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa914006010 tx=0x7fa9140042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:22.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.592+0000 7fa91d7fa640 1 -- 172.21.15.1:0/1061101156 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91403f030 con 0x7fa9201035c0 2024-09-11T21:57:22.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.592+0000 7fa91d7fa640 1 -- 172.21.15.1:0/1061101156 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa914003ce0 con 0x7fa9201035c0 2024-09-11T21:57:22.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.592+0000 7fa92691a640 1 -- 172.21.15.1:0/1061101156 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa92019d640 con 0x7fa9201035c0 2024-09-11T21:57:22.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.592+0000 7fa91d7fa640 1 -- 172.21.15.1:0/1061101156 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa914035070 con 0x7fa9201035c0 2024-09-11T21:57:22.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.592+0000 7fa92691a640 1 -- 172.21.15.1:0/1061101156 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9201a4f30 con 0x7fa9201035c0 2024-09-11T21:57:22.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.594+0000 7fa92691a640 1 -- 172.21.15.1:0/1061101156 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa92006ac00 con 0x7fa9201035c0 2024-09-11T21:57:22.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.595+0000 7fa91d7fa640 1 -- 172.21.15.1:0/1061101156 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 24) ==== 138008+0+0 (secure 0 0 0) 0x7fa914042030 con 0x7fa9201035c0 2024-09-11T21:57:22.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.596+0000 7fa91d7fa640 1 --2- 172.21.15.1:0/1061101156 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fa9080a2d80 0x7fa9080a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:22.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.596+0000 7fa91ffff640 1 --2- 172.21.15.1:0/1061101156 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fa9080a2d80 0x7fa9080a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:22.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.596+0000 7fa91d7fa640 1 -- 172.21.15.1:0/1061101156 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7fa9140efb20 con 0x7fa9201035c0 2024-09-11T21:57:22.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.596+0000 7fa91ffff640 1 --2- 172.21.15.1:0/1061101156 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fa9080a2d80 0x7fa9080a5240 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa91000bbe0 tx=0x7fa91000b390 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:22.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.602+0000 7fa91d7fa640 1 -- 172.21.15.1:0/1061101156 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa9140bc040 con 0x7fa9201035c0 2024-09-11T21:57:22.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.801+0000 7fa92691a640 1 -- 172.21.15.1:0/1061101156 --> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fa920062630 con 0x7fa9080a2d80 2024-09-11T21:57:22.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.808+0000 7fa91d7fa640 1 -- 172.21.15.1:0/1061101156 <== mgr.15204 v2:172.21.15.1:6800/3543150772 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7fa920062630 con 0x7fa9080a2d80 2024-09-11T21:57:22.809 INFO:teuthology.orchestra.run.smithi001.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-11T00:00:00", "created": "2024-09-11T21:57:21", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-09-11T21:57:22.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.810+0000 7fa92691a640 1 -- 172.21.15.1:0/1061101156 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fa9080a2d80 msgr2=0x7fa9080a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:22.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.810+0000 7fa92691a640 1 --2- 172.21.15.1:0/1061101156 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fa9080a2d80 0x7fa9080a5240 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa91000bbe0 tx=0x7fa91000b390 comp rx=0 tx=0).stop 2024-09-11T21:57:22.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.810+0000 7fa92691a640 1 -- 172.21.15.1:0/1061101156 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa9201035c0 msgr2=0x7fa920198900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:22.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.810+0000 7fa92691a640 1 --2- 172.21.15.1:0/1061101156 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa9201035c0 0x7fa920198900 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa914006010 tx=0x7fa9140042f0 comp rx=0 tx=0).stop 2024-09-11T21:57:22.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.810+0000 7fa92691a640 1 -- 172.21.15.1:0/1061101156 shutdown_connections 2024-09-11T21:57:22.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.810+0000 7fa92691a640 1 --2- 172.21.15.1:0/1061101156 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fa9080a2d80 0x7fa9080a5240 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:22.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.810+0000 7fa92691a640 1 --2- 172.21.15.1:0/1061101156 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa92010c0f0 0x7fa92019cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:22.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.810+0000 7fa92691a640 1 --2- 172.21.15.1:0/1061101156 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa9201035c0 0x7fa920198900 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:22.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.810+0000 7fa92691a640 1 --2- 172.21.15.1:0/1061101156 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa920102ca0 0x7fa9201983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:22.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.810+0000 7fa92691a640 1 -- 172.21.15.1:0/1061101156 >> 172.21.15.1:0/1061101156 conn(0x7fa9200fc6b0 msgr2=0x7fa9200fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:22.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.811+0000 7fa92691a640 1 -- 172.21.15.1:0/1061101156 shutdown_connections 2024-09-11T21:57:22.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.811+0000 7fa92691a640 1 -- 172.21.15.1:0/1061101156 wait complete. 2024-09-11T21:57:22.823 DEBUG:teuthology.orchestra.run.smithi001:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-11T21:57:22.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.864+0000 7f5c1756e640 1 -- 172.21.15.1:0/1771238725 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c10100e20 msgr2=0x7f5c10101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:22.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.864+0000 7f5c1756e640 1 --2- 172.21.15.1:0/1771238725 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c10100e20 0x7f5c10101200 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c009900 tx=0x7f5c0c02a480 comp rx=0 tx=0).stop 2024-09-11T21:57:22.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.864+0000 7f5c1756e640 1 -- 172.21.15.1:0/1771238725 shutdown_connections 2024-09-11T21:57:22.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.864+0000 7f5c1756e640 1 --2- 172.21.15.1:0/1771238725 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c10102ef0 0x7f5c1010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:22.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.864+0000 7f5c1756e640 1 --2- 172.21.15.1:0/1771238725 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c10102570 0x7f5c101029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:22.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.864+0000 7f5c1756e640 1 --2- 172.21.15.1:0/1771238725 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c10100e20 0x7f5c10101200 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:22.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.864+0000 7f5c1756e640 1 -- 172.21.15.1:0/1771238725 >> 172.21.15.1:0/1771238725 conn(0x7f5c100fc7d0 msgr2=0x7f5c100febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:22.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.864+0000 7f5c1756e640 1 -- 172.21.15.1:0/1771238725 shutdown_connections 2024-09-11T21:57:22.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.864+0000 7f5c1756e640 1 -- 172.21.15.1:0/1771238725 wait complete. 2024-09-11T21:57:22.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.865+0000 7f5c1756e640 1 Processor -- start 2024-09-11T21:57:22.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.865+0000 7f5c1756e640 1 -- start start 2024-09-11T21:57:22.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.865+0000 7f5c1756e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c10100e20 0x7f5c101984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:22.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.865+0000 7f5c1756e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c10102570 0x7f5c10198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:22.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.865+0000 7f5c1756e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c10102ef0 0x7f5c1019cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:22.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.865+0000 7f5c1756e640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5c10110f80 con 0x7f5c10102ef0 2024-09-11T21:57:22.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.865+0000 7f5c1756e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5c10110e00 con 0x7f5c10100e20 2024-09-11T21:57:22.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.865+0000 7f5c1756e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5c10111100 con 0x7f5c10102570 2024-09-11T21:57:22.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.865+0000 7f5c15b0b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c10102ef0 0x7f5c1019cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:22.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.865+0000 7f5c15b0b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c10102ef0 0x7f5c1019cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38846/0 (socket says 172.21.15.1:38846) 2024-09-11T21:57:22.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.865+0000 7f5c15b0b640 1 -- 172.21.15.1:0/2526055627 learned_addr learned my addr 172.21.15.1:0/2526055627 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:22.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.865+0000 7f5c1530a640 1 --2- 172.21.15.1:0/2526055627 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c10100e20 0x7f5c101984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:22.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.865+0000 7f5c14b09640 1 --2- 172.21.15.1:0/2526055627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c10102570 0x7f5c10198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:22.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.866+0000 7f5c15b0b640 1 -- 172.21.15.1:0/2526055627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c10102570 msgr2=0x7f5c10198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:22.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.866+0000 7f5c15b0b640 1 --2- 172.21.15.1:0/2526055627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c10102570 0x7f5c10198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:22.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.866+0000 7f5c15b0b640 1 -- 172.21.15.1:0/2526055627 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c10100e20 msgr2=0x7f5c101984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:22.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.866+0000 7f5c15b0b640 1 --2- 172.21.15.1:0/2526055627 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c10100e20 0x7f5c101984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:22.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.866+0000 7f5c15b0b640 1 -- 172.21.15.1:0/2526055627 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c1019d490 con 0x7f5c10102ef0 2024-09-11T21:57:22.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.866+0000 7f5c14b09640 1 --2- 172.21.15.1:0/2526055627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c10102570 0x7f5c10198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:57:22.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.866+0000 7f5c15b0b640 1 --2- 172.21.15.1:0/2526055627 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c10102ef0 0x7f5c1019cdb0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5c0800eaa0 tx=0x7f5c0800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:22.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.866+0000 7f5c067fc640 1 -- 172.21.15.1:0/2526055627 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c0800cdb0 con 0x7f5c10102ef0 2024-09-11T21:57:22.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.866+0000 7f5c1756e640 1 -- 172.21.15.1:0/2526055627 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c1019d780 con 0x7f5c10102ef0 2024-09-11T21:57:22.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.866+0000 7f5c067fc640 1 -- 172.21.15.1:0/2526055627 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c08005020 con 0x7f5c10102ef0 2024-09-11T21:57:22.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.866+0000 7f5c067fc640 1 -- 172.21.15.1:0/2526055627 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c080116b0 con 0x7f5c10102ef0 2024-09-11T21:57:22.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.866+0000 7f5c1756e640 1 -- 172.21.15.1:0/2526055627 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c101a5060 con 0x7f5c10102ef0 2024-09-11T21:57:22.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.869+0000 7f5c067fc640 1 -- 172.21.15.1:0/2526055627 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 24) ==== 138008+0+0 (secure 0 0 0) 0x7f5c08005560 con 0x7f5c10102ef0 2024-09-11T21:57:22.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.869+0000 7f5c19947740 1 Processor -- start 2024-09-11T21:57:22.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.869+0000 7f5c19947740 1 -- start start 2024-09-11T21:57:22.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.870+0000 7f5c19947740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559bbc4f64a0 0x559bbc4f6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:22.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.870+0000 7f5c19947740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559bbc4f6dc0 0x559bbc501140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:22.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.870+0000 7f5c19947740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bbc501680 0x559bbc503a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:22.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.870+0000 7f5c19947740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x559bbc45c2e0 con 0x559bbc4f6dc0 2024-09-11T21:57:22.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.870+0000 7f5c19947740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x559bbc403ce0 con 0x559bbc4f64a0 2024-09-11T21:57:22.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.870+0000 7f5c19947740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x559bbc4582a0 con 0x559bbc501680 2024-09-11T21:57:22.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.870+0000 7f5c1530a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559bbc4f6dc0 0x559bbc501140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:22.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.870+0000 7f5c1530a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559bbc4f6dc0 0x559bbc501140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38854/0 (socket says 172.21.15.1:38854) 2024-09-11T21:57:22.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.870+0000 7f5c1530a640 1 -- 172.21.15.1:0/777250881 learned_addr learned my addr 172.21.15.1:0/777250881 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:22.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.870+0000 7f5c14b09640 1 --2- 172.21.15.1:0/777250881 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559bbc4f64a0 0x559bbc4f6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:22.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.870+0000 7f5c14b09640 1 --2- 172.21.15.1:0/777250881 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bbc501680 0x559bbc503a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:22.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.870+0000 7f5c067fc640 1 --2- 172.21.15.1:0/2526055627 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f5bfc0a2d80 0x7f5bfc0a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:22.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.870+0000 7f5c1530a640 1 -- 172.21.15.1:0/777250881 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bbc501680 msgr2=0x559bbc503a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:22.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.870+0000 7f5c1530a640 1 --2- 172.21.15.1:0/777250881 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bbc501680 0x559bbc503a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:22.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.870+0000 7f5c1530a640 1 -- 172.21.15.1:0/777250881 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559bbc4f64a0 msgr2=0x559bbc4f6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:22.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.870+0000 7f5c1530a640 1 --2- 172.21.15.1:0/777250881 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559bbc4f64a0 0x559bbc4f6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:22.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.870+0000 7f5c1530a640 1 -- 172.21.15.1:0/777250881 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bbc3e8440 con 0x559bbc4f6dc0 2024-09-11T21:57:22.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.870+0000 7f5c14b09640 1 --2- 172.21.15.1:0/777250881 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bbc501680 0x559bbc503a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:57:22.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.870+0000 7f5c15b0b640 1 --2- 172.21.15.1:0/2526055627 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f5bfc0a2d80 0x7f5bfc0a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:22.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.870+0000 7f5c067fc640 1 -- 172.21.15.1:0/2526055627 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7f5c080cf8d0 con 0x7f5c10102ef0 2024-09-11T21:57:22.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.871+0000 7f5c1530a640 1 --2- 172.21.15.1:0/777250881 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559bbc4f6dc0 0x559bbc501140 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c02a9a0 tx=0x7f5c0c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bc1788cc69ec6f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:22.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.871+0000 7f5bfb7fe640 1 -- 172.21.15.1:0/777250881 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c0c035030 con 0x559bbc4f6dc0 2024-09-11T21:57:22.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.871+0000 7f5c15b0b640 1 --2- 172.21.15.1:0/2526055627 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f5bfc0a2d80 0x7f5bfc0a5240 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5c080ccf60 tx=0x7f5c080054d0 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:22.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.871+0000 7f5bfb7fe640 1 -- 172.21.15.1:0/777250881 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c0c040070 con 0x559bbc4f6dc0 2024-09-11T21:57:22.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.871+0000 7f5bfb7fe640 1 -- 172.21.15.1:0/777250881 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c0c0319d0 con 0x559bbc4f6dc0 2024-09-11T21:57:22.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.872+0000 7f5c19947740 1 -- 172.21.15.1:0/777250881 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559bbc4f6dc0 msgr2=0x559bbc501140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:22.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.872+0000 7f5c19947740 1 --2- 172.21.15.1:0/777250881 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559bbc4f6dc0 0x559bbc501140 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c02a9a0 tx=0x7f5c0c0040c0 comp rx=0 tx=0).stop 2024-09-11T21:57:22.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.872+0000 7f5c19947740 1 -- 172.21.15.1:0/777250881 shutdown_connections 2024-09-11T21:57:22.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.872+0000 7f5c19947740 1 --2- 172.21.15.1:0/777250881 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bbc501680 0x559bbc503a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:22.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.872+0000 7f5c19947740 1 --2- 172.21.15.1:0/777250881 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559bbc4f6dc0 0x559bbc501140 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:22.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.872+0000 7f5c19947740 1 --2- 172.21.15.1:0/777250881 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559bbc4f64a0 0x559bbc4f6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:22.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.872+0000 7f5c19947740 1 -- 172.21.15.1:0/777250881 >> 172.21.15.1:0/777250881 conn(0x559bbc4f2a30 msgr2=0x559bbc4f2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:22.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.872+0000 7f5c19947740 1 -- 172.21.15.1:0/777250881 shutdown_connections 2024-09-11T21:57:22.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.872+0000 7f5c19947740 1 -- 172.21.15.1:0/777250881 wait complete. 2024-09-11T21:57:22.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.872+0000 7f5c19947740 1 -- start start 2024-09-11T21:57:22.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.872+0000 7f5c19947740 1 Processor -- start 2024-09-11T21:57:22.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.873+0000 7f5c19947740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559bbc4f64a0 0x559bbc500f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:22.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.873+0000 7f5c19947740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bbc4f6dc0 0x559bbc4fe220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:22.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.873+0000 7f5c19947740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559bbc501680 0x559bbc4fe760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:22.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.873+0000 7f5c19947740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x559bbc504250 con 0x559bbc501680 2024-09-11T21:57:22.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.873+0000 7f5c19947740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x559bbc417050 con 0x559bbc4f64a0 2024-09-11T21:57:22.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.873+0000 7f5c19947740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x559bbc5043d0 con 0x559bbc4f6dc0 2024-09-11T21:57:22.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.873+0000 7f5c19947740 10 client.0 _ll_register_callbacks cb 0x559bbc457160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:57:22.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.873+0000 7f5c14b09640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559bbc501680 0x559bbc4fe760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:22.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.873+0000 7f5c14b09640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559bbc501680 0x559bbc4fe760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38860/0 (socket says 172.21.15.1:38860) 2024-09-11T21:57:22.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.873+0000 7f5c14b09640 1 -- 172.21.15.1:0/1211031606 learned_addr learned my addr 172.21.15.1:0/1211031606 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:22.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.873+0000 7f5c14b09640 1 --2- 172.21.15.1:0/1211031606 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559bbc4f64a0 0x559bbc500f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:22.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.873+0000 7f5c1530a640 1 --2- 172.21.15.1:0/1211031606 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bbc4f6dc0 0x559bbc4fe220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:22.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.873+0000 7f5c14b09640 1 -- 172.21.15.1:0/1211031606 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bbc4f6dc0 msgr2=0x559bbc4fe220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:22.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.873+0000 7f5c14b09640 1 --2- 172.21.15.1:0/1211031606 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bbc4f6dc0 0x559bbc4fe220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:22.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.873+0000 7f5c14b09640 1 -- 172.21.15.1:0/1211031606 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559bbc4f64a0 msgr2=0x559bbc500f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:22.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.873+0000 7f5c14b09640 1 --2- 172.21.15.1:0/1211031606 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559bbc4f64a0 0x559bbc500f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:22.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.873+0000 7f5c14b09640 1 -- 172.21.15.1:0/1211031606 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bbc1cd3c0 con 0x559bbc501680 2024-09-11T21:57:22.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.873+0000 7f5c14b09640 1 --2- 172.21.15.1:0/1211031606 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559bbc501680 0x559bbc4fe760 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f5c00014e00 tx=0x7f5c00004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89cf89498940a4a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:22.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.873+0000 7f5bf8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T21:57:22.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.873+0000 7f5bf8ff9640 1 -- 172.21.15.1:0/1211031606 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c000140d0 con 0x559bbc501680 2024-09-11T21:57:22.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.874+0000 7f5bf8ff9640 1 -- 172.21.15.1:0/1211031606 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7f5c00014270 con 0x559bbc501680 2024-09-11T21:57:22.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.874+0000 7f5c19947740 1 -- 172.21.15.1:0/1211031606 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c0c040070 con 0x559bbc501680 2024-09-11T21:57:22.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.874+0000 7f5bf8ff9640 1 -- 172.21.15.1:0/1211031606 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5c00005020 con 0x559bbc501680 2024-09-11T21:57:22.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.874+0000 7f5c19947740 10 client.15384 fetch_fsmap learned FSMap version 28 2024-09-11T21:57:22.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.874+0000 7f5c19947740 1 -- 172.21.15.1:0/1211031606 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559bbc50fbc0 con 0x559bbc501680 2024-09-11T21:57:22.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.874+0000 7f5bf8ff9640 1 -- 172.21.15.1:0/1211031606 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c0c040070 con 0x559bbc501680 2024-09-11T21:57:22.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.874+0000 7f5bf8ff9640 1 -- 172.21.15.1:0/1211031606 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5c00004c60 con 0x559bbc501680 2024-09-11T21:57:22.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.874+0000 7f5c19947740 10 client.15384 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T21:57:22.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.874+0000 7f5c19947740 10 client.15384 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T21:57:22.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.874+0000 7f5c19947740 20 client.15384 populate_metadata read hostname 'smithi001' 2024-09-11T21:57:22.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.874+0000 7f5c19947740 1 --2- 172.21.15.1:0/1211031606 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x559bbc510fd0 0x559bbc5133c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:22.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.874+0000 7f5c19947740 4 client.15384 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T21:57:22.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.874+0000 7f5c19947740 1 -- 172.21.15.1:0/1211031606 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559bbc469850 con 0x559bbc510fd0 2024-09-11T21:57:22.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.875+0000 7f5c1530a640 1 --2- 172.21.15.1:0/1211031606 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x559bbc510fd0 0x559bbc5133c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:22.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.876+0000 7f5c1530a640 1 --2- 172.21.15.1:0/1211031606 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x559bbc510fd0 0x559bbc5133c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c016cbc1b0de3ff5 server_cookie=e9a09370b3e1f87 in_seq=0 out_seq=0 2024-09-11T21:57:22.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.876+0000 7f5bf8ff9640 10 client.15384 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T21:57:22.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.876+0000 7f5bf8ff9640 1 -- 172.21.15.1:0/1211031606 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559bbc469850 con 0x559bbc510fd0 2024-09-11T21:57:22.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.876+0000 7f5bf8ff9640 10 client.15384 handle_command_reply: tid=0 2024-09-11T21:57:22.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.876+0000 7f5c19947740 1 client.15384 shutdown 2024-09-11T21:57:22.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.877+0000 7f5c19947740 1 -- 172.21.15.1:0/1211031606 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559bbc501680 msgr2=0x559bbc4fe760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:22.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.877+0000 7f5c19947740 1 --2- 172.21.15.1:0/1211031606 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559bbc501680 0x559bbc4fe760 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f5c00014e00 tx=0x7f5c00004730 comp rx=0 tx=0).stop 2024-09-11T21:57:22.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.877+0000 7f5c19947740 1 -- 172.21.15.1:0/1211031606 shutdown_connections 2024-09-11T21:57:22.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.877+0000 7f5c19947740 1 --2- 172.21.15.1:0/1211031606 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x559bbc510fd0 0x559bbc5133c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:22.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.877+0000 7f5bf8ff9640 0 client.15384 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T21:57:22.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.877+0000 7f5c19947740 1 --2- 172.21.15.1:0/1211031606 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559bbc501680 0x559bbc4fe760 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:22.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.877+0000 7f5c19947740 1 --2- 172.21.15.1:0/1211031606 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bbc4f6dc0 0x559bbc4fe220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:22.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.877+0000 7f5c19947740 1 --2- 172.21.15.1:0/1211031606 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559bbc4f64a0 0x559bbc500f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:22.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.877+0000 7f5c19947740 1 -- 172.21.15.1:0/1211031606 >> 172.21.15.1:0/1211031606 conn(0x559bbc4f2a30 msgr2=0x559bbc4f1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:22.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.877+0000 7f5c19947740 1 -- 172.21.15.1:0/1211031606 shutdown_connections 2024-09-11T21:57:22.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.877+0000 7f5c19947740 1 -- 172.21.15.1:0/1211031606 wait complete. 2024-09-11T21:57:22.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.877+0000 7f5c19947740 20 client.15384 trim_cache size 0 max 16384 2024-09-11T21:57:22.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.894+0000 7f5c19947740 1 Processor -- start 2024-09-11T21:57:22.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.895+0000 7f5c19947740 1 -- start start 2024-09-11T21:57:22.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.895+0000 7f5c19947740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559bbc4f2a30 0x559bbc50d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:22.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.895+0000 7f5c19947740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bbc4f64a0 0x559bbc518c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:22.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.895+0000 7f5c19947740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559bbc4f6dc0 0x559bbc5191b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:22.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.895+0000 7f5c19947740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5c101ad8f0 con 0x559bbc4f6dc0 2024-09-11T21:57:22.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.895+0000 7f5c19947740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5c101af430 con 0x559bbc4f2a30 2024-09-11T21:57:22.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.895+0000 7f5c19947740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5c101adfc0 con 0x559bbc4f64a0 2024-09-11T21:57:22.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.895+0000 7f5c14b09640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559bbc4f6dc0 0x559bbc5191b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:22.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.895+0000 7f5c14b09640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559bbc4f6dc0 0x559bbc5191b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38868/0 (socket says 172.21.15.1:38868) 2024-09-11T21:57:22.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.895+0000 7f5c14b09640 1 -- 172.21.15.1:0/334865750 learned_addr learned my addr 172.21.15.1:0/334865750 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:22.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.895+0000 7f5c14b09640 1 --2- 172.21.15.1:0/334865750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559bbc4f2a30 0x559bbc50d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:22.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.895+0000 7f5c1530a640 1 --2- 172.21.15.1:0/334865750 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bbc4f64a0 0x559bbc518c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:22.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.895+0000 7f5c14b09640 1 -- 172.21.15.1:0/334865750 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bbc4f64a0 msgr2=0x559bbc518c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:22.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.895+0000 7f5c14b09640 1 --2- 172.21.15.1:0/334865750 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bbc4f64a0 0x559bbc518c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:22.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.895+0000 7f5c14b09640 1 -- 172.21.15.1:0/334865750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559bbc4f2a30 msgr2=0x559bbc50d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:22.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.895+0000 7f5c14b09640 1 --2- 172.21.15.1:0/334865750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559bbc4f2a30 0x559bbc50d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:22.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.895+0000 7f5c14b09640 1 -- 172.21.15.1:0/334865750 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bbc5012e0 con 0x559bbc4f6dc0 2024-09-11T21:57:22.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.896+0000 7f5c14b09640 1 --2- 172.21.15.1:0/334865750 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559bbc4f6dc0 0x559bbc5191b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f5c00001eb0 tx=0x7f5c000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b3a5544f880bb97 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:22.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.896+0000 7f5bfaffd640 1 -- 172.21.15.1:0/334865750 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c000140d0 con 0x559bbc4f6dc0 2024-09-11T21:57:22.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.896+0000 7f5bfaffd640 1 -- 172.21.15.1:0/334865750 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c00030070 con 0x559bbc4f6dc0 2024-09-11T21:57:22.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.896+0000 7f5bfaffd640 1 -- 172.21.15.1:0/334865750 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c000047a0 con 0x559bbc4f6dc0 2024-09-11T21:57:22.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.897+0000 7f5c19947740 1 -- 172.21.15.1:0/334865750 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559bbc4f6dc0 msgr2=0x559bbc5191b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:22.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.897+0000 7f5c19947740 1 --2- 172.21.15.1:0/334865750 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559bbc4f6dc0 0x559bbc5191b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f5c00001eb0 tx=0x7f5c000046c0 comp rx=0 tx=0).stop 2024-09-11T21:57:22.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.897+0000 7f5c19947740 1 -- 172.21.15.1:0/334865750 shutdown_connections 2024-09-11T21:57:22.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.897+0000 7f5c19947740 1 --2- 172.21.15.1:0/334865750 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559bbc4f6dc0 0x559bbc5191b0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:22.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.897+0000 7f5c19947740 1 --2- 172.21.15.1:0/334865750 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bbc4f64a0 0x559bbc518c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:22.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.897+0000 7f5c19947740 1 --2- 172.21.15.1:0/334865750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559bbc4f2a30 0x559bbc50d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:22.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.897+0000 7f5c19947740 1 -- 172.21.15.1:0/334865750 >> 172.21.15.1:0/334865750 conn(0x559bbc4fe8c0 msgr2=0x559bbc4fcaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:22.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.897+0000 7f5c19947740 1 -- 172.21.15.1:0/334865750 shutdown_connections 2024-09-11T21:57:22.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.897+0000 7f5c19947740 1 -- 172.21.15.1:0/334865750 wait complete. 2024-09-11T21:57:22.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.897+0000 7f5c19947740 1 -- start start 2024-09-11T21:57:22.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.897+0000 7f5c19947740 1 Processor -- start 2024-09-11T21:57:22.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.898+0000 7f5c19947740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bbc4f2a30 0x559bbc509600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:22.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.898+0000 7f5c19947740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559bbc4f64a0 0x559bbc509b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:22.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.898+0000 7f5c19947740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559bbc4f6dc0 0x559bbc574b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:22.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.898+0000 7f5c19947740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5c101aed60 con 0x559bbc4f6dc0 2024-09-11T21:57:22.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.898+0000 7f5c19947740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5c101ae690 con 0x559bbc4f64a0 2024-09-11T21:57:22.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.898+0000 7f5c19947740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5c101ab8d0 con 0x559bbc4f2a30 2024-09-11T21:57:22.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.898+0000 7f5c19947740 10 client.0 _ll_register_callbacks cb 0x7f5c0000c8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:57:22.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.898+0000 7f5c14b09640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559bbc4f6dc0 0x559bbc574b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:22.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.898+0000 7f5c14b09640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559bbc4f6dc0 0x559bbc574b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38874/0 (socket says 172.21.15.1:38874) 2024-09-11T21:57:22.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.898+0000 7f5c14b09640 1 -- 172.21.15.1:0/2454154714 learned_addr learned my addr 172.21.15.1:0/2454154714 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:22.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.898+0000 7f5c1530a640 1 --2- 172.21.15.1:0/2454154714 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559bbc4f64a0 0x559bbc509b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:22.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.898+0000 7f5c14b09640 1 --2- 172.21.15.1:0/2454154714 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bbc4f2a30 0x559bbc509600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:22.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.898+0000 7f5c14b09640 1 -- 172.21.15.1:0/2454154714 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bbc4f2a30 msgr2=0x559bbc509600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:22.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.898+0000 7f5c14b09640 1 --2- 172.21.15.1:0/2454154714 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bbc4f2a30 0x559bbc509600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:22.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.898+0000 7f5c14b09640 1 -- 172.21.15.1:0/2454154714 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559bbc4f64a0 msgr2=0x559bbc509b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:22.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.898+0000 7f5c14b09640 1 --2- 172.21.15.1:0/2454154714 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559bbc4f64a0 0x559bbc509b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:22.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.898+0000 7f5c14b09640 1 -- 172.21.15.1:0/2454154714 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bbc577830 con 0x559bbc4f6dc0 2024-09-11T21:57:22.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.899+0000 7f5c1530a640 1 --2- 172.21.15.1:0/2454154714 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559bbc4f64a0 0x559bbc509b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T21:57:22.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.899+0000 7f5c14b09640 1 --2- 172.21.15.1:0/2454154714 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559bbc4f6dc0 0x559bbc574b00 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f5c00012010 tx=0x7f5c00014410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ed9dedce806eac6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:22.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.899+0000 7f5bf8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T21:57:22.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.899+0000 7f5bf8ff9640 1 -- 172.21.15.1:0/2454154714 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c0003d060 con 0x559bbc4f6dc0 2024-09-11T21:57:22.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.899+0000 7f5bf8ff9640 1 -- 172.21.15.1:0/2454154714 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7f5c000140d0 con 0x559bbc4f6dc0 2024-09-11T21:57:22.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.899+0000 7f5c19947740 1 -- 172.21.15.1:0/2454154714 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c00030070 con 0x559bbc4f6dc0 2024-09-11T21:57:22.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.899+0000 7f5bf8ff9640 1 -- 172.21.15.1:0/2454154714 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5c000051c0 con 0x559bbc4f6dc0 2024-09-11T21:57:22.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.899+0000 7f5c19947740 10 client.15390 fetch_fsmap learned FSMap version 28 2024-09-11T21:57:22.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.899+0000 7f5c19947740 1 -- 172.21.15.1:0/2454154714 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559bbc5804c0 con 0x559bbc4f6dc0 2024-09-11T21:57:22.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.899+0000 7f5bf8ff9640 1 -- 172.21.15.1:0/2454154714 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c00030070 con 0x559bbc4f6dc0 2024-09-11T21:57:22.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.900+0000 7f5bf8ff9640 1 -- 172.21.15.1:0/2454154714 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5c00041460 con 0x559bbc4f6dc0 2024-09-11T21:57:22.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.900+0000 7f5c19947740 10 client.15390 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T21:57:22.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.900+0000 7f5c19947740 10 client.15390 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T21:57:22.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.900+0000 7f5c19947740 20 client.15390 populate_metadata read hostname 'smithi001' 2024-09-11T21:57:22.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.900+0000 7f5c19947740 1 --2- 172.21.15.1:0/2454154714 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x559bbc501680 0x559bbc582c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:22.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.900+0000 7f5c19947740 4 client.15390 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T21:57:22.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.900+0000 7f5c19947740 1 -- 172.21.15.1:0/2454154714 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559bbc463d60 con 0x559bbc501680 2024-09-11T21:57:22.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.900+0000 7f5c1530a640 1 --2- 172.21.15.1:0/2454154714 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x559bbc501680 0x559bbc582c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:22.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.901+0000 7f5bf8ff9640 10 client.15390 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T21:57:22.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.901+0000 7f5c1530a640 1 --2- 172.21.15.1:0/2454154714 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x559bbc501680 0x559bbc582c30 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1c07cc3c17ece1e server_cookie=507814e3b567cd66 in_seq=0 out_seq=0 2024-09-11T21:57:22.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.901+0000 7f5bf8ff9640 1 -- 172.21.15.1:0/2454154714 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x559bbc463d60 con 0x559bbc501680 2024-09-11T21:57:22.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.901+0000 7f5bf8ff9640 10 client.15390 handle_command_reply: tid=0 2024-09-11T21:57:22.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.901+0000 7f5c19947740 1 client.15390 shutdown 2024-09-11T21:57:22.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.901+0000 7f5c19947740 1 -- 172.21.15.1:0/2454154714 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559bbc4f6dc0 msgr2=0x559bbc574b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:22.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.901+0000 7f5c19947740 1 --2- 172.21.15.1:0/2454154714 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559bbc4f6dc0 0x559bbc574b00 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f5c00012010 tx=0x7f5c00014410 comp rx=0 tx=0).stop 2024-09-11T21:57:22.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.901+0000 7f5c19947740 1 -- 172.21.15.1:0/2454154714 shutdown_connections 2024-09-11T21:57:22.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.901+0000 7f5c19947740 1 --2- 172.21.15.1:0/2454154714 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x559bbc501680 0x559bbc582c30 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:22.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.901+0000 7f5c19947740 1 --2- 172.21.15.1:0/2454154714 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559bbc4f6dc0 0x559bbc574b00 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:22.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.901+0000 7f5c19947740 1 --2- 172.21.15.1:0/2454154714 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559bbc4f64a0 0x559bbc509b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:22.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.901+0000 7f5c19947740 1 --2- 172.21.15.1:0/2454154714 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bbc4f2a30 0x559bbc509600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:22.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.901+0000 7f5c19947740 1 -- 172.21.15.1:0/2454154714 >> 172.21.15.1:0/2454154714 conn(0x559bbc4fe8c0 msgr2=0x559bbc4f1350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:22.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.901+0000 7f5bf8ff9640 0 client.15390 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T21:57:22.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.901+0000 7f5c19947740 1 -- 172.21.15.1:0/2454154714 shutdown_connections 2024-09-11T21:57:22.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.901+0000 7f5c19947740 1 -- 172.21.15.1:0/2454154714 wait complete. 2024-09-11T21:57:22.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.901+0000 7f5c19947740 20 client.15390 trim_cache size 0 max 16384 2024-09-11T21:57:22.907 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-11T21:57:22.908 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T21:57:22.908 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-09-11T21:57:22.908 INFO:teuthology.orchestra.run.smithi001.stdout: "02307e43-eeeb-4387-84d1-e1da69e7046b": { 2024-09-11T21:57:22.908 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2024-09-11T21:57:22.908 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "02307e43-eeeb-4387-84d1-e1da69e7046b", 2024-09-11T21:57:22.908 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2024-09-11T21:57:22.908 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T21:57:22.908 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T21:57:22.908 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-11T21:57:22.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.902+0000 7f5c1756e640 1 -- 172.21.15.1:0/2526055627 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f5bfc0a2d80 msgr2=0x7f5bfc0a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:22.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.902+0000 7f5c1756e640 1 --2- 172.21.15.1:0/2526055627 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f5bfc0a2d80 0x7f5bfc0a5240 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5c080ccf60 tx=0x7f5c080054d0 comp rx=0 tx=0).stop 2024-09-11T21:57:22.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.902+0000 7f5c1756e640 1 -- 172.21.15.1:0/2526055627 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c10102ef0 msgr2=0x7f5c1019cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:22.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.902+0000 7f5c1756e640 1 --2- 172.21.15.1:0/2526055627 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c10102ef0 0x7f5c1019cdb0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5c0800eaa0 tx=0x7f5c0800ef60 comp rx=0 tx=0).stop 2024-09-11T21:57:22.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.902+0000 7f5c1756e640 1 -- 172.21.15.1:0/2526055627 shutdown_connections 2024-09-11T21:57:22.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.902+0000 7f5c1756e640 1 --2- 172.21.15.1:0/2526055627 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f5bfc0a2d80 0x7f5bfc0a5240 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:22.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.902+0000 7f5c1756e640 1 --2- 172.21.15.1:0/2526055627 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c10102ef0 0x7f5c1019cdb0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:22.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.902+0000 7f5c1756e640 1 --2- 172.21.15.1:0/2526055627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c10102570 0x7f5c10198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:22.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.902+0000 7f5c1756e640 1 --2- 172.21.15.1:0/2526055627 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c10100e20 0x7f5c101984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:22.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.902+0000 7f5c1756e640 1 -- 172.21.15.1:0/2526055627 >> 172.21.15.1:0/2526055627 conn(0x7f5c100fc7d0 msgr2=0x7f5c1006a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:22.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.902+0000 7f5c1756e640 1 -- 172.21.15.1:0/2526055627 shutdown_connections 2024-09-11T21:57:22.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.902+0000 7f5c1756e640 1 -- 172.21.15.1:0/2526055627 wait complete. 2024-09-11T21:57:22.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.937+0000 7f9b92ee4640 1 -- 172.21.15.1:0/611566480 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9b8c101920 msgr2=0x7f9b8c101d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:22.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.937+0000 7f9b92ee4640 1 --2- 172.21.15.1:0/611566480 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9b8c101920 0x7f9b8c101d00 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f9b80009900 tx=0x7f9b8002a410 comp rx=0 tx=0).stop 2024-09-11T21:57:22.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.937+0000 7f9b92ee4640 1 -- 172.21.15.1:0/611566480 shutdown_connections 2024-09-11T21:57:22.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.937+0000 7f9b92ee4640 1 --2- 172.21.15.1:0/611566480 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b8c106e60 0x7f9b8c10d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:22.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.937+0000 7f9b92ee4640 1 --2- 172.21.15.1:0/611566480 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b8c1022d0 0x7f9b8c106700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:22.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.937+0000 7f9b92ee4640 1 --2- 172.21.15.1:0/611566480 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9b8c101920 0x7f9b8c101d00 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:22.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.937+0000 7f9b92ee4640 1 -- 172.21.15.1:0/611566480 >> 172.21.15.1:0/611566480 conn(0x7f9b8c075eb0 msgr2=0x7f9b8c0762c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:22.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.937+0000 7f9b92ee4640 1 -- 172.21.15.1:0/611566480 shutdown_connections 2024-09-11T21:57:22.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.937+0000 7f9b92ee4640 1 -- 172.21.15.1:0/611566480 wait complete. 2024-09-11T21:57:22.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.938+0000 7f9b92ee4640 1 Processor -- start 2024-09-11T21:57:22.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.938+0000 7f9b92ee4640 1 -- start start 2024-09-11T21:57:22.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.938+0000 7f9b92ee4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b8c101920 0x7f9b8c198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:22.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.938+0000 7f9b92ee4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9b8c106e60 0x7f9b8c198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:22.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.938+0000 7f9b92ee4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b8c19cce0 0x7f9b8c19d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:22.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.938+0000 7f9b92ee4640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8c111ed0 con 0x7f9b8c106e60 2024-09-11T21:57:22.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.938+0000 7f9b92ee4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8c111d50 con 0x7f9b8c101920 2024-09-11T21:57:22.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.938+0000 7f9b92ee4640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8c112050 con 0x7f9b8c19cce0 2024-09-11T21:57:22.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.938+0000 7f9b8bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9b8c106e60 0x7f9b8c198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:22.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.938+0000 7f9b8bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9b8c106e60 0x7f9b8c198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38894/0 (socket says 172.21.15.1:38894) 2024-09-11T21:57:22.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.938+0000 7f9b8bfff640 1 -- 172.21.15.1:0/3501365598 learned_addr learned my addr 172.21.15.1:0/3501365598 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:22.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.938+0000 7f9b91481640 1 --2- 172.21.15.1:0/3501365598 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b8c19cce0 0x7f9b8c19d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:22.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.938+0000 7f9b90c80640 1 --2- 172.21.15.1:0/3501365598 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b8c101920 0x7f9b8c198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:22.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.939+0000 7f9b8bfff640 1 -- 172.21.15.1:0/3501365598 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b8c19cce0 msgr2=0x7f9b8c19d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:22.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.939+0000 7f9b8bfff640 1 --2- 172.21.15.1:0/3501365598 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b8c19cce0 0x7f9b8c19d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:22.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.939+0000 7f9b8bfff640 1 -- 172.21.15.1:0/3501365598 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b8c101920 msgr2=0x7f9b8c198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:22.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.939+0000 7f9b8bfff640 1 --2- 172.21.15.1:0/3501365598 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b8c101920 0x7f9b8c198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:22.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.939+0000 7f9b8bfff640 1 -- 172.21.15.1:0/3501365598 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b8c19d7d0 con 0x7f9b8c106e60 2024-09-11T21:57:22.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.939+0000 7f9b90c80640 1 --2- 172.21.15.1:0/3501365598 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b8c101920 0x7f9b8c198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:57:22.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.939+0000 7f9b91481640 1 --2- 172.21.15.1:0/3501365598 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b8c19cce0 0x7f9b8c19d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:57:22.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.939+0000 7f9b8bfff640 1 --2- 172.21.15.1:0/3501365598 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9b8c106e60 0x7f9b8c198960 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f9b7800d740 tx=0x7f9b7800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:22.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.939+0000 7f9b89ffb640 1 -- 172.21.15.1:0/3501365598 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b78017070 con 0x7f9b8c106e60 2024-09-11T21:57:22.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.939+0000 7f9b89ffb640 1 -- 172.21.15.1:0/3501365598 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b7800de50 con 0x7f9b8c106e60 2024-09-11T21:57:22.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.939+0000 7f9b92ee4640 1 -- 172.21.15.1:0/3501365598 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b8c19dac0 con 0x7f9b8c106e60 2024-09-11T21:57:22.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.939+0000 7f9b89ffb640 1 -- 172.21.15.1:0/3501365598 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b78004e00 con 0x7f9b8c106e60 2024-09-11T21:57:22.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.939+0000 7f9b92ee4640 1 -- 172.21.15.1:0/3501365598 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b8c1a5250 con 0x7f9b8c106e60 2024-09-11T21:57:22.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.940+0000 7f9b92ee4640 1 -- 172.21.15.1:0/3501365598 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b8c069450 con 0x7f9b8c106e60 2024-09-11T21:57:22.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.941+0000 7f9b89ffb640 1 -- 172.21.15.1:0/3501365598 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 24) ==== 138008+0+0 (secure 0 0 0) 0x7f9b78004fa0 con 0x7f9b8c106e60 2024-09-11T21:57:22.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.942+0000 7f9b89ffb640 1 --2- 172.21.15.1:0/3501365598 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f9b7c0a2e00 0x7f9b7c0a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:22.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.942+0000 7f9b90c80640 1 --2- 172.21.15.1:0/3501365598 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f9b7c0a2e00 0x7f9b7c0a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:22.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.942+0000 7f9b89ffb640 1 -- 172.21.15.1:0/3501365598 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7f9b780cf040 con 0x7f9b8c106e60 2024-09-11T21:57:22.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.943+0000 7f9b90c80640 1 --2- 172.21.15.1:0/3501365598 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f9b7c0a2e00 0x7f9b7c0a52c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9b80005ab0 tx=0x7f9b80005a20 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:22.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:22.946+0000 7f9b89ffb640 1 -- 172.21.15.1:0/3501365598 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9b7809ba90 con 0x7f9b8c106e60 2024-09-11T21:57:23.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:23.148+0000 7f9b92ee4640 1 -- 172.21.15.1:0/3501365598 --> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f9b8c1022d0 con 0x7f9b7c0a2e00 2024-09-11T21:57:23.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:23.154+0000 7f9b89ffb640 1 -- 172.21.15.1:0/3501365598 <== mgr.15204 v2:172.21.15.1:6800/3543150772 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f9b8c1022d0 con 0x7f9b7c0a2e00 2024-09-11T21:57:23.155 INFO:teuthology.orchestra.run.smithi001.stdout:/ 1m 6m3h 2024-09-11T21:57:23.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:23.156+0000 7f9b92ee4640 1 -- 172.21.15.1:0/3501365598 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f9b7c0a2e00 msgr2=0x7f9b7c0a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:23.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:23.156+0000 7f9b92ee4640 1 --2- 172.21.15.1:0/3501365598 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f9b7c0a2e00 0x7f9b7c0a52c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9b80005ab0 tx=0x7f9b80005a20 comp rx=0 tx=0).stop 2024-09-11T21:57:23.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:23.156+0000 7f9b92ee4640 1 -- 172.21.15.1:0/3501365598 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9b8c106e60 msgr2=0x7f9b8c198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:23.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:23.156+0000 7f9b92ee4640 1 --2- 172.21.15.1:0/3501365598 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9b8c106e60 0x7f9b8c198960 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f9b7800d740 tx=0x7f9b7800dc00 comp rx=0 tx=0).stop 2024-09-11T21:57:23.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:23.156+0000 7f9b92ee4640 1 -- 172.21.15.1:0/3501365598 shutdown_connections 2024-09-11T21:57:23.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:23.156+0000 7f9b92ee4640 1 --2- 172.21.15.1:0/3501365598 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f9b7c0a2e00 0x7f9b7c0a52c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:23.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:23.156+0000 7f9b92ee4640 1 --2- 172.21.15.1:0/3501365598 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b8c19cce0 0x7f9b8c19d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:23.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:23.156+0000 7f9b92ee4640 1 --2- 172.21.15.1:0/3501365598 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9b8c106e60 0x7f9b8c198960 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:23.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:23.156+0000 7f9b92ee4640 1 --2- 172.21.15.1:0/3501365598 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b8c101920 0x7f9b8c198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:23.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:23.156+0000 7f9b92ee4640 1 -- 172.21.15.1:0/3501365598 >> 172.21.15.1:0/3501365598 conn(0x7f9b8c075eb0 msgr2=0x7f9b8c104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:23.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:23.157+0000 7f9b92ee4640 1 -- 172.21.15.1:0/3501365598 shutdown_connections 2024-09-11T21:57:23.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:23.157+0000 7f9b92ee4640 1 -- 172.21.15.1:0/3501365598 wait complete. 2024-09-11T21:57:23.167 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:22 smithi001 ceph-mon[30935]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 0 B/s wr, 24 op/s 2024-09-11T21:57:23.167 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:22 smithi001 ceph-mon[30935]: from='client.15360 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:57:23.167 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:22 smithi001 ceph-mon[30935]: from='client.15366 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:57:23.168 DEBUG:teuthology.orchestra.run.smithi001:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-09-11T21:57:23.196 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-09-11T21:57:23.224 INFO:teuthology.run_tasks:Running task background_exec... 2024-09-11T21:57:23.246 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi001.front.sepia.ceph.com 2024-09-11T21:57:23.247 DEBUG:teuthology.orchestra.run.smithi001:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-09-11T21:57:23.249 INFO:teuthology.run_tasks:Running task quiescer... 2024-09-11T21:57:23.263 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-11T21:57:23.263 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T21:57:23.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:22 smithi053 ceph-mon[34908]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 0 B/s wr, 24 op/s 2024-09-11T21:57:23.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:22 smithi053 ceph-mon[34908]: from='client.15360 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:57:23.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:22 smithi053 ceph-mon[34908]: from='client.15366 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:57:23.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:22 smithi161 ceph-mon[35615]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 0 B/s wr, 24 op/s 2024-09-11T21:57:23.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:22 smithi161 ceph-mon[35615]: from='client.15360 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:57:23.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:22 smithi161 ceph-mon[35615]: from='client.15366 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:57:23.604 INFO:tasks.cephfs.filesystem:scrub status for tag:02307e43-eeeb-4387-84d1-e1da69e7046b - {'path': '/', 'tag': '02307e43-eeeb-4387-84d1-e1da69e7046b', 'options': 'recursive,force'} 2024-09-11T21:57:23.649 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:57:24.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:24.099+0000 7f2317d23640 1 -- 172.21.15.1:0/1259707546 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2310106f10 msgr2=0x7f231010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:24.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:24.099+0000 7f2317d23640 1 --2- 172.21.15.1:0/1259707546 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2310106f10 0x7f231010ea90 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f2308009ca0 tx=0x7f230802a8c0 comp rx=0 tx=0).stop 2024-09-11T21:57:24.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:24.099+0000 7f2317d23640 1 -- 172.21.15.1:0/1259707546 shutdown_connections 2024-09-11T21:57:24.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:24.099+0000 7f2317d23640 1 --2- 172.21.15.1:0/1259707546 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2310106f10 0x7f231010ea90 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:24.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:24.099+0000 7f2317d23640 1 --2- 172.21.15.1:0/1259707546 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2310102790 0x7f23101067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:24.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:24.099+0000 7f2317d23640 1 --2- 172.21.15.1:0/1259707546 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2310101de0 0x7f23101021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:24.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:24.099+0000 7f2317d23640 1 -- 172.21.15.1:0/1259707546 >> 172.21.15.1:0/1259707546 conn(0x7f23100fd8c0 msgr2=0x7f23100ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:24.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:24.099+0000 7f2317d23640 1 -- 172.21.15.1:0/1259707546 shutdown_connections 2024-09-11T21:57:24.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:24.099+0000 7f2317d23640 1 -- 172.21.15.1:0/1259707546 wait complete. 2024-09-11T21:57:24.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:24.100+0000 7f2317d23640 1 Processor -- start 2024-09-11T21:57:24.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:24.100+0000 7f2317d23640 1 -- start start 2024-09-11T21:57:24.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:24.100+0000 7f2317d23640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2310101de0 0x7f23101983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:24.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:24.100+0000 7f2317d23640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2310102790 0x7f2310198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:24.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:24.100+0000 7f2317d23640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2310106f10 0x7f231019ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:24.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:24.100+0000 7f2317d23640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2310111f70 con 0x7f2310106f10 2024-09-11T21:57:24.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:24.100+0000 7f2317d23640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2310111df0 con 0x7f2310101de0 2024-09-11T21:57:24.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:24.100+0000 7f2317d23640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f23101120f0 con 0x7f2310102790 2024-09-11T21:57:24.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:24.100+0000 7f23162c0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2310106f10 0x7f231019ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:24.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:24.100+0000 7f23162c0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2310106f10 0x7f231019ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38912/0 (socket says 172.21.15.1:38912) 2024-09-11T21:57:24.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:24.100+0000 7f23152be640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2310102790 0x7f2310198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:24.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:24.100+0000 7f23162c0640 1 -- 172.21.15.1:0/3935832468 learned_addr learned my addr 172.21.15.1:0/3935832468 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:24.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:24.101+0000 7f2315abf640 1 --2- 172.21.15.1:0/3935832468 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2310101de0 0x7f23101983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:24.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:24.101+0000 7f23162c0640 1 -- 172.21.15.1:0/3935832468 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2310102790 msgr2=0x7f2310198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:24.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:24.101+0000 7f23162c0640 1 --2- 172.21.15.1:0/3935832468 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2310102790 0x7f2310198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:24.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:24.101+0000 7f23162c0640 1 -- 172.21.15.1:0/3935832468 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2310101de0 msgr2=0x7f23101983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:24.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:24.101+0000 7f23162c0640 1 --2- 172.21.15.1:0/3935832468 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2310101de0 0x7f23101983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:24.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:24.101+0000 7f23162c0640 1 -- 172.21.15.1:0/3935832468 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f231019d3a0 con 0x7f2310106f10 2024-09-11T21:57:24.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:24.101+0000 7f2315abf640 1 --2- 172.21.15.1:0/3935832468 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2310101de0 0x7f23101983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:57:24.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:24.101+0000 7f23162c0640 1 --2- 172.21.15.1:0/3935832468 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2310106f10 0x7f231019ccc0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f2308031a20 tx=0x7f2308031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:24.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:24.101+0000 7f2306ffd640 1 -- 172.21.15.1:0/3935832468 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2308003c30 con 0x7f2310106f10 2024-09-11T21:57:24.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:24.101+0000 7f2306ffd640 1 -- 172.21.15.1:0/3935832468 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2308003dd0 con 0x7f2310106f10 2024-09-11T21:57:24.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:24.101+0000 7f2306ffd640 1 -- 172.21.15.1:0/3935832468 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f230802fcf0 con 0x7f2310106f10 2024-09-11T21:57:24.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:24.101+0000 7f2317d23640 1 -- 172.21.15.1:0/3935832468 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f231019d630 con 0x7f2310106f10 2024-09-11T21:57:24.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:24.101+0000 7f2317d23640 1 -- 172.21.15.1:0/3935832468 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23101a4f70 con 0x7f2310106f10 2024-09-11T21:57:24.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:24.102+0000 7f2317d23640 1 -- 172.21.15.1:0/3935832468 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23101032c0 con 0x7f2310106f10 2024-09-11T21:57:24.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:24.104+0000 7f2306ffd640 1 -- 172.21.15.1:0/3935832468 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 24) ==== 138008+0+0 (secure 0 0 0) 0x7f2308004050 con 0x7f2310106f10 2024-09-11T21:57:24.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:24.104+0000 7f2306ffd640 1 --2- 172.21.15.1:0/3935832468 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f22fc0a2e00 0x7f22fc0a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:24.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:24.104+0000 7f2315abf640 1 --2- 172.21.15.1:0/3935832468 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f22fc0a2e00 0x7f22fc0a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:24.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:24.104+0000 7f2306ffd640 1 -- 172.21.15.1:0/3935832468 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7f23080f09b0 con 0x7f2310106f10 2024-09-11T21:57:24.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:24.105+0000 7f2315abf640 1 --2- 172.21.15.1:0/3935832468 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f22fc0a2e00 0x7f22fc0a52c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f230c0058c0 tx=0x7f230c005850 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:24.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:24.109+0000 7f2306ffd640 1 -- 172.21.15.1:0/3935832468 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f23080bd600 con 0x7f2310106f10 2024-09-11T21:57:24.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:23 smithi001 ceph-mon[30935]: from='client.15372 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:57:24.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:23 smithi001 ceph-mon[30935]: from='client.15396 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:57:24.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:24.279+0000 7f2317d23640 1 -- 172.21.15.1:0/3935832468 --> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f231010a160 con 0x7f22fc0a2e00 2024-09-11T21:57:24.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:24.286+0000 7f2306ffd640 1 -- 172.21.15.1:0/3935832468 <== mgr.15204 v2:172.21.15.1:6800/3543150772 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258079 (secure 0 0 0) 0x7f230c00bca0 con 0x7f22fc0a2e00 2024-09-11T21:57:24.289 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:57:24.290 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-11T21:57:24.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:24.289+0000 7f2317d23640 1 -- 172.21.15.1:0/3935832468 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f22fc0a2e00 msgr2=0x7f22fc0a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:24.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:24.289+0000 7f2317d23640 1 --2- 172.21.15.1:0/3935832468 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f22fc0a2e00 0x7f22fc0a52c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f230c0058c0 tx=0x7f230c005850 comp rx=0 tx=0).stop 2024-09-11T21:57:24.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:24.289+0000 7f2317d23640 1 -- 172.21.15.1:0/3935832468 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2310106f10 msgr2=0x7f231019ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:24.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:24.289+0000 7f2317d23640 1 --2- 172.21.15.1:0/3935832468 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2310106f10 0x7f231019ccc0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f2308031a20 tx=0x7f2308031a50 comp rx=0 tx=0).stop 2024-09-11T21:57:24.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:24.289+0000 7f2317d23640 1 -- 172.21.15.1:0/3935832468 shutdown_connections 2024-09-11T21:57:24.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:24.289+0000 7f2317d23640 1 --2- 172.21.15.1:0/3935832468 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f22fc0a2e00 0x7f22fc0a52c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:24.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:24.289+0000 7f2317d23640 1 --2- 172.21.15.1:0/3935832468 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2310106f10 0x7f231019ccc0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:24.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:24.289+0000 7f2317d23640 1 --2- 172.21.15.1:0/3935832468 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2310102790 0x7f2310198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:24.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:24.289+0000 7f2317d23640 1 --2- 172.21.15.1:0/3935832468 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2310101de0 0x7f23101983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:24.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:24.289+0000 7f2317d23640 1 -- 172.21.15.1:0/3935832468 >> 172.21.15.1:0/3935832468 conn(0x7f23100fd8c0 msgr2=0x7f23100fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:24.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:24.289+0000 7f2317d23640 1 -- 172.21.15.1:0/3935832468 shutdown_connections 2024-09-11T21:57:24.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:24.289+0000 7f2317d23640 1 -- 172.21.15.1:0/3935832468 wait complete. 2024-09-11T21:57:24.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:23 smithi053 ceph-mon[34908]: from='client.15372 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:57:24.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:23 smithi053 ceph-mon[34908]: from='client.15396 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:57:24.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:23 smithi161 ceph-mon[35615]: from='client.15372 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:57:24.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:23 smithi161 ceph-mon[35615]: from='client.15396 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T21:57:24.998 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":8,"stamp":"2024-09-11T21:57:23.684435+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1191642,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1453,"num_read_kb":1056,"num_write":583,"num_write_kb":4410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1239947,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":956,"ondisk_log_size":956,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":390216,"kb_used_data":11812,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124551608,"statfs":{"total":1151940427776,"available":1151540846592,"internally_reserved":0,"allocated":12095488,"data_stored":9104706,"data_compressed":297580,"data_compressed_allocated":393216,"data_compressed_original":3670016,"omap_allocated":19142,"internal_metadata":387233082},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":151,"num_read_kb":236,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.991861"},"pg_stats":[{"pgid":"3.3f","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788313+0000","last_change":"2024-09-11T21:55:20.095385+0000","last_active":"2024-09-11T21:57:13.788313+0000","last_peered":"2024-09-11T21:57:13.788313+0000","last_clean":"2024-09-11T21:57:13.788313+0000","last_became_active":"2024-09-11T21:55:20.095077+0000","last_became_peered":"2024-09-11T21:55:20.095077+0000","last_unstale":"2024-09-11T21:57:13.788313+0000","last_undegraded":"2024-09-11T21:57:13.788313+0000","last_fullsized":"2024-09-11T21:57:13.788313+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:53:52.231522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"86'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.669584+0000","last_change":"2024-09-11T21:55:20.103540+0000","last_active":"2024-09-11T21:57:13.669584+0000","last_peered":"2024-09-11T21:57:13.669584+0000","last_clean":"2024-09-11T21:57:13.669584+0000","last_became_active":"2024-09-11T21:55:20.103233+0000","last_became_peered":"2024-09-11T21:55:20.103233+0000","last_unstale":"2024-09-11T21:57:13.669584+0000","last_undegraded":"2024-09-11T21:57:13.669584+0000","last_fullsized":"2024-09-11T21:57:13.669584+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:51:29.222202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.669576+0000","last_change":"2024-09-11T21:55:20.098538+0000","last_active":"2024-09-11T21:57:13.669576+0000","last_peered":"2024-09-11T21:57:13.669576+0000","last_clean":"2024-09-11T21:57:13.669576+0000","last_became_active":"2024-09-11T21:55:20.098202+0000","last_became_peered":"2024-09-11T21:55:20.098202+0000","last_unstale":"2024-09-11T21:57:13.669576+0000","last_undegraded":"2024-09-11T21:57:13.669576+0000","last_fullsized":"2024-09-11T21:57:13.669576+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:33:46.636952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788491+0000","last_change":"2024-09-11T21:55:20.100869+0000","last_active":"2024-09-11T21:57:13.788491+0000","last_peered":"2024-09-11T21:57:13.788491+0000","last_clean":"2024-09-11T21:57:13.788491+0000","last_became_active":"2024-09-11T21:55:20.100573+0000","last_became_peered":"2024-09-11T21:55:20.100573+0000","last_unstale":"2024-09-11T21:57:13.788491+0000","last_undegraded":"2024-09-11T21:57:13.788491+0000","last_fullsized":"2024-09-11T21:57:13.788491+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:58:11.154105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"86'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671033+0000","last_change":"2024-09-11T21:55:20.091648+0000","last_active":"2024-09-11T21:57:13.671033+0000","last_peered":"2024-09-11T21:57:13.671033+0000","last_clean":"2024-09-11T21:57:13.671033+0000","last_became_active":"2024-09-11T21:55:20.091222+0000","last_became_peered":"2024-09-11T21:55:20.091222+0000","last_unstale":"2024-09-11T21:57:13.671033+0000","last_undegraded":"2024-09-11T21:57:13.671033+0000","last_fullsized":"2024-09-11T21:57:13.671033+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:55:03.420754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"86'6","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.764975+0000","last_change":"2024-09-11T21:56:26.376595+0000","last_active":"2024-09-11T21:57:13.764975+0000","last_peered":"2024-09-11T21:57:13.764975+0000","last_clean":"2024-09-11T21:57:13.764975+0000","last_became_active":"2024-09-11T21:56:26.375784+0000","last_became_peered":"2024-09-11T21:56:26.375784+0000","last_unstale":"2024-09-11T21:57:13.764975+0000","last_undegraded":"2024-09-11T21:57:13.764975+0000","last_fullsized":"2024-09-11T21:57:13.764975+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:19:38.371463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,7],"acting":[10,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"86'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.670787+0000","last_change":"2024-09-11T21:55:20.101084+0000","last_active":"2024-09-11T21:57:13.670787+0000","last_peered":"2024-09-11T21:57:13.670787+0000","last_clean":"2024-09-11T21:57:13.670787+0000","last_became_active":"2024-09-11T21:55:20.100739+0000","last_became_peered":"2024-09-11T21:55:20.100739+0000","last_unstale":"2024-09-11T21:57:13.670787+0000","last_undegraded":"2024-09-11T21:57:13.670787+0000","last_fullsized":"2024-09-11T21:57:13.670787+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:46:58.969559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.753160+0000","last_change":"2024-09-11T21:55:20.088715+0000","last_active":"2024-09-11T21:57:13.753160+0000","last_peered":"2024-09-11T21:57:13.753160+0000","last_clean":"2024-09-11T21:57:13.753160+0000","last_became_active":"2024-09-11T21:55:20.088280+0000","last_became_peered":"2024-09-11T21:55:20.088280+0000","last_unstale":"2024-09-11T21:57:13.753160+0000","last_undegraded":"2024-09-11T21:57:13.753160+0000","last_fullsized":"2024-09-11T21:57:13.753160+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:06:24.450444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"86'11","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788671+0000","last_change":"2024-09-11T21:55:20.101266+0000","last_active":"2024-09-11T21:57:13.788671+0000","last_peered":"2024-09-11T21:57:13.788671+0000","last_clean":"2024-09-11T21:57:13.788671+0000","last_became_active":"2024-09-11T21:55:20.100919+0000","last_became_peered":"2024-09-11T21:55:20.100919+0000","last_unstale":"2024-09-11T21:57:13.788671+0000","last_undegraded":"2024-09-11T21:57:13.788671+0000","last_fullsized":"2024-09-11T21:57:13.788671+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:47:15.466082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"86'16","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.765361+0000","last_change":"2024-09-11T21:55:20.098086+0000","last_active":"2024-09-11T21:57:13.765361+0000","last_peered":"2024-09-11T21:57:13.765361+0000","last_clean":"2024-09-11T21:57:13.765361+0000","last_became_active":"2024-09-11T21:55:20.097754+0000","last_became_peered":"2024-09-11T21:55:20.097754+0000","last_unstale":"2024-09-11T21:57:13.765361+0000","last_undegraded":"2024-09-11T21:57:13.765361+0000","last_fullsized":"2024-09-11T21:57:13.765361+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:37:00.098345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"86'6","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671036+0000","last_change":"2024-09-11T21:55:20.092671+0000","last_active":"2024-09-11T21:57:13.671036+0000","last_peered":"2024-09-11T21:57:13.671036+0000","last_clean":"2024-09-11T21:57:13.671036+0000","last_became_active":"2024-09-11T21:55:20.092112+0000","last_became_peered":"2024-09-11T21:55:20.092112+0000","last_unstale":"2024-09-11T21:57:13.671036+0000","last_undegraded":"2024-09-11T21:57:13.671036+0000","last_fullsized":"2024-09-11T21:57:13.671036+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:06:28.538838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788646+0000","last_change":"2024-09-11T21:55:20.099863+0000","last_active":"2024-09-11T21:57:13.788646+0000","last_peered":"2024-09-11T21:57:13.788646+0000","last_clean":"2024-09-11T21:57:13.788646+0000","last_became_active":"2024-09-11T21:55:20.099173+0000","last_became_peered":"2024-09-11T21:55:20.099173+0000","last_unstale":"2024-09-11T21:57:13.788646+0000","last_undegraded":"2024-09-11T21:57:13.788646+0000","last_fullsized":"2024-09-11T21:57:13.788646+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:34:56.669410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"86'10","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788507+0000","last_change":"2024-09-11T21:55:20.101212+0000","last_active":"2024-09-11T21:57:13.788507+0000","last_peered":"2024-09-11T21:57:13.788507+0000","last_clean":"2024-09-11T21:57:13.788507+0000","last_became_active":"2024-09-11T21:55:20.100876+0000","last_became_peered":"2024-09-11T21:55:20.100876+0000","last_unstale":"2024-09-11T21:57:13.788507+0000","last_undegraded":"2024-09-11T21:57:13.788507+0000","last_fullsized":"2024-09-11T21:57:13.788507+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:14:18.104009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.669316+0000","last_change":"2024-09-11T21:55:20.095394+0000","last_active":"2024-09-11T21:57:13.669316+0000","last_peered":"2024-09-11T21:57:13.669316+0000","last_clean":"2024-09-11T21:57:13.669316+0000","last_became_active":"2024-09-11T21:55:20.095101+0000","last_became_peered":"2024-09-11T21:55:20.095101+0000","last_unstale":"2024-09-11T21:57:13.669316+0000","last_undegraded":"2024-09-11T21:57:13.669316+0000","last_fullsized":"2024-09-11T21:57:13.669316+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:53:32.529359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"86'7","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.669266+0000","last_change":"2024-09-11T21:56:26.376206+0000","last_active":"2024-09-11T21:57:13.669266+0000","last_peered":"2024-09-11T21:57:13.669266+0000","last_clean":"2024-09-11T21:57:13.669266+0000","last_became_active":"2024-09-11T21:56:26.375829+0000","last_became_peered":"2024-09-11T21:56:26.375829+0000","last_unstale":"2024-09-11T21:57:13.669266+0000","last_undegraded":"2024-09-11T21:57:13.669266+0000","last_fullsized":"2024-09-11T21:57:13.669266+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:21:59.484414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.787934+0000","last_change":"2024-09-11T21:55:20.093019+0000","last_active":"2024-09-11T21:57:13.787934+0000","last_peered":"2024-09-11T21:57:13.787934+0000","last_clean":"2024-09-11T21:57:13.787934+0000","last_became_active":"2024-09-11T21:55:20.092691+0000","last_became_peered":"2024-09-11T21:55:20.092691+0000","last_unstale":"2024-09-11T21:57:13.787934+0000","last_undegraded":"2024-09-11T21:57:13.787934+0000","last_fullsized":"2024-09-11T21:57:13.787934+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:04:36.684992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788379+0000","last_change":"2024-09-11T21:55:20.097333+0000","last_active":"2024-09-11T21:57:13.788379+0000","last_peered":"2024-09-11T21:57:13.788379+0000","last_clean":"2024-09-11T21:57:13.788379+0000","last_became_active":"2024-09-11T21:55:20.096752+0000","last_became_peered":"2024-09-11T21:55:20.096752+0000","last_unstale":"2024-09-11T21:57:13.788379+0000","last_undegraded":"2024-09-11T21:57:13.788379+0000","last_fullsized":"2024-09-11T21:57:13.788379+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:27:20.722174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"86'10","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.751322+0000","last_change":"2024-09-11T21:55:20.093972+0000","last_active":"2024-09-11T21:57:13.751322+0000","last_peered":"2024-09-11T21:57:13.751322+0000","last_clean":"2024-09-11T21:57:13.751322+0000","last_became_active":"2024-09-11T21:55:20.093633+0000","last_became_peered":"2024-09-11T21:55:20.093633+0000","last_unstale":"2024-09-11T21:57:13.751322+0000","last_undegraded":"2024-09-11T21:57:13.751322+0000","last_fullsized":"2024-09-11T21:57:13.751322+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:25:18.973332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671334+0000","last_change":"2024-09-11T21:55:20.094239+0000","last_active":"2024-09-11T21:57:13.671334+0000","last_peered":"2024-09-11T21:57:13.671334+0000","last_clean":"2024-09-11T21:57:13.671334+0000","last_became_active":"2024-09-11T21:55:20.093924+0000","last_became_peered":"2024-09-11T21:55:20.093924+0000","last_unstale":"2024-09-11T21:57:13.671334+0000","last_undegraded":"2024-09-11T21:57:13.671334+0000","last_fullsized":"2024-09-11T21:57:13.671334+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:39:53.012780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788075+0000","last_change":"2024-09-11T21:55:20.099335+0000","last_active":"2024-09-11T21:57:13.788075+0000","last_peered":"2024-09-11T21:57:13.788075+0000","last_clean":"2024-09-11T21:57:13.788075+0000","last_became_active":"2024-09-11T21:55:20.098929+0000","last_became_peered":"2024-09-11T21:55:20.098929+0000","last_unstale":"2024-09-11T21:57:13.788075+0000","last_undegraded":"2024-09-11T21:57:13.788075+0000","last_fullsized":"2024-09-11T21:57:13.788075+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:38:43.117829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"86'11","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788526+0000","last_change":"2024-09-11T21:55:20.094068+0000","last_active":"2024-09-11T21:57:13.788526+0000","last_peered":"2024-09-11T21:57:13.788526+0000","last_clean":"2024-09-11T21:57:13.788526+0000","last_became_active":"2024-09-11T21:55:20.093637+0000","last_became_peered":"2024-09-11T21:55:20.093637+0000","last_unstale":"2024-09-11T21:57:13.788526+0000","last_undegraded":"2024-09-11T21:57:13.788526+0000","last_fullsized":"2024-09-11T21:57:13.788526+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:52:59.289715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"86'10","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788122+0000","last_change":"2024-09-11T21:55:20.090778+0000","last_active":"2024-09-11T21:57:13.788122+0000","last_peered":"2024-09-11T21:57:13.788122+0000","last_clean":"2024-09-11T21:57:13.788122+0000","last_became_active":"2024-09-11T21:55:20.090443+0000","last_became_peered":"2024-09-11T21:55:20.090443+0000","last_unstale":"2024-09-11T21:57:13.788122+0000","last_undegraded":"2024-09-11T21:57:13.788122+0000","last_fullsized":"2024-09-11T21:57:13.788122+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:46:21.488228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.753203+0000","last_change":"2024-09-11T21:55:20.092238+0000","last_active":"2024-09-11T21:57:13.753203+0000","last_peered":"2024-09-11T21:57:13.753203+0000","last_clean":"2024-09-11T21:57:13.753203+0000","last_became_active":"2024-09-11T21:55:20.091961+0000","last_became_peered":"2024-09-11T21:55:20.091961+0000","last_unstale":"2024-09-11T21:57:13.753203+0000","last_undegraded":"2024-09-11T21:57:13.753203+0000","last_fullsized":"2024-09-11T21:57:13.753203+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:54:40.445441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"86'6","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.761984+0000","last_change":"2024-09-11T21:55:20.102944+0000","last_active":"2024-09-11T21:57:13.761984+0000","last_peered":"2024-09-11T21:57:13.761984+0000","last_clean":"2024-09-11T21:57:13.761984+0000","last_became_active":"2024-09-11T21:55:20.102533+0000","last_became_peered":"2024-09-11T21:55:20.102533+0000","last_unstale":"2024-09-11T21:57:13.761984+0000","last_undegraded":"2024-09-11T21:57:13.761984+0000","last_fullsized":"2024-09-11T21:57:13.761984+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:12:15.832622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"86'2","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.753729+0000","last_change":"2024-09-11T21:56:26.292272+0000","last_active":"2024-09-11T21:57:13.753729+0000","last_peered":"2024-09-11T21:57:13.753729+0000","last_clean":"2024-09-11T21:57:13.753729+0000","last_became_active":"2024-09-11T21:56:26.291920+0000","last_became_peered":"2024-09-11T21:56:26.291920+0000","last_unstale":"2024-09-11T21:57:13.753729+0000","last_undegraded":"2024-09-11T21:57:13.753729+0000","last_fullsized":"2024-09-11T21:57:13.753729+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:13:13.579907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"86'15","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671489+0000","last_change":"2024-09-11T21:56:26.376364+0000","last_active":"2024-09-11T21:57:13.671489+0000","last_peered":"2024-09-11T21:57:13.671489+0000","last_clean":"2024-09-11T21:57:13.671489+0000","last_became_active":"2024-09-11T21:56:26.375927+0000","last_became_peered":"2024-09-11T21:56:26.375927+0000","last_unstale":"2024-09-11T21:57:13.671489+0000","last_undegraded":"2024-09-11T21:57:13.671489+0000","last_fullsized":"2024-09-11T21:57:13.671489+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:23:03.664207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,9],"acting":[2,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"86'6","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788153+0000","last_change":"2024-09-11T21:56:26.376702+0000","last_active":"2024-09-11T21:57:13.788153+0000","last_peered":"2024-09-11T21:57:13.788153+0000","last_clean":"2024-09-11T21:57:13.788153+0000","last_became_active":"2024-09-11T21:56:26.376179+0000","last_became_peered":"2024-09-11T21:56:26.376179+0000","last_unstale":"2024-09-11T21:57:13.788153+0000","last_undegraded":"2024-09-11T21:57:13.788153+0000","last_fullsized":"2024-09-11T21:57:13.788153+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:27:34.666962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"86'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788960+0000","last_change":"2024-09-11T21:55:20.093898+0000","last_active":"2024-09-11T21:57:13.788960+0000","last_peered":"2024-09-11T21:57:13.788960+0000","last_clean":"2024-09-11T21:57:13.788960+0000","last_became_active":"2024-09-11T21:55:20.093348+0000","last_became_peered":"2024-09-11T21:55:20.093348+0000","last_unstale":"2024-09-11T21:57:13.788960+0000","last_undegraded":"2024-09-11T21:57:13.788960+0000","last_fullsized":"2024-09-11T21:57:13.788960+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:00:06.185478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.669457+0000","last_change":"2024-09-11T21:55:20.100575+0000","last_active":"2024-09-11T21:57:13.669457+0000","last_peered":"2024-09-11T21:57:13.669457+0000","last_clean":"2024-09-11T21:57:13.669457+0000","last_became_active":"2024-09-11T21:55:20.100255+0000","last_became_peered":"2024-09-11T21:55:20.100255+0000","last_unstale":"2024-09-11T21:57:13.669457+0000","last_undegraded":"2024-09-11T21:57:13.669457+0000","last_fullsized":"2024-09-11T21:57:13.669457+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:48:57.718298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"86'8","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.670722+0000","last_change":"2024-09-11T21:56:26.377183+0000","last_active":"2024-09-11T21:57:13.670722+0000","last_peered":"2024-09-11T21:57:13.670722+0000","last_clean":"2024-09-11T21:57:13.670722+0000","last_became_active":"2024-09-11T21:56:26.375906+0000","last_became_peered":"2024-09-11T21:56:26.375906+0000","last_unstale":"2024-09-11T21:57:13.670722+0000","last_undegraded":"2024-09-11T21:57:13.670722+0000","last_fullsized":"2024-09-11T21:57:13.670722+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:00:29.033961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"86'12","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.670784+0000","last_change":"2024-09-11T21:55:20.093822+0000","last_active":"2024-09-11T21:57:13.670784+0000","last_peered":"2024-09-11T21:57:13.670784+0000","last_clean":"2024-09-11T21:57:13.670784+0000","last_became_active":"2024-09-11T21:55:20.093361+0000","last_became_peered":"2024-09-11T21:55:20.093361+0000","last_unstale":"2024-09-11T21:57:13.670784+0000","last_undegraded":"2024-09-11T21:57:13.670784+0000","last_fullsized":"2024-09-11T21:57:13.670784+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:37:06.362732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.787998+0000","last_change":"2024-09-11T21:55:20.103990+0000","last_active":"2024-09-11T21:57:13.787998+0000","last_peered":"2024-09-11T21:57:13.787998+0000","last_clean":"2024-09-11T21:57:13.787998+0000","last_became_active":"2024-09-11T21:55:20.103375+0000","last_became_peered":"2024-09-11T21:55:20.103375+0000","last_unstale":"2024-09-11T21:57:13.787998+0000","last_undegraded":"2024-09-11T21:57:13.787998+0000","last_fullsized":"2024-09-11T21:57:13.787998+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:02:06.786888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"86'7","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671024+0000","last_change":"2024-09-11T21:56:26.376261+0000","last_active":"2024-09-11T21:57:13.671024+0000","last_peered":"2024-09-11T21:57:13.671024+0000","last_clean":"2024-09-11T21:57:13.671024+0000","last_became_active":"2024-09-11T21:56:26.375863+0000","last_became_peered":"2024-09-11T21:56:26.375863+0000","last_unstale":"2024-09-11T21:57:13.671024+0000","last_undegraded":"2024-09-11T21:57:13.671024+0000","last_fullsized":"2024-09-11T21:57:13.671024+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:58:21.359836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"87'8","reported_seq":145,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:18.304167+0000","last_change":"2024-09-11T21:55:18.026947+0000","last_active":"2024-09-11T21:57:18.304167+0000","last_peered":"2024-09-11T21:57:18.304167+0000","last_clean":"2024-09-11T21:57:18.304167+0000","last_became_active":"2024-09-11T21:55:18.026632+0000","last_became_peered":"2024-09-11T21:55:18.026632+0000","last_unstale":"2024-09-11T21:57:18.304167+0000","last_undegraded":"2024-09-11T21:57:18.304167+0000","last_fullsized":"2024-09-11T21:57:18.304167+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:55:16.618074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1549,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":113,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.762157+0000","last_change":"2024-09-11T21:55:20.100197+0000","last_active":"2024-09-11T21:57:13.762157+0000","last_peered":"2024-09-11T21:57:13.762157+0000","last_clean":"2024-09-11T21:57:13.762157+0000","last_became_active":"2024-09-11T21:55:20.098494+0000","last_became_peered":"2024-09-11T21:55:20.098494+0000","last_unstale":"2024-09-11T21:57:13.762157+0000","last_undegraded":"2024-09-11T21:57:13.762157+0000","last_fullsized":"2024-09-11T21:57:13.762157+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:14.505314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"84'7","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.765023+0000","last_change":"2024-09-11T21:55:18.024359+0000","last_active":"2024-09-11T21:57:13.765023+0000","last_peered":"2024-09-11T21:57:13.765023+0000","last_clean":"2024-09-11T21:57:13.765023+0000","last_became_active":"2024-09-11T21:55:18.023624+0000","last_became_peered":"2024-09-11T21:55:18.023624+0000","last_unstale":"2024-09-11T21:57:13.765023+0000","last_undegraded":"2024-09-11T21:57:13.765023+0000","last_fullsized":"2024-09-11T21:57:13.765023+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:13:38.582232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"84'3","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788356+0000","last_change":"2024-09-11T21:55:18.032541+0000","last_active":"2024-09-11T21:57:13.788356+0000","last_peered":"2024-09-11T21:57:13.788356+0000","last_clean":"2024-09-11T21:57:13.788356+0000","last_became_active":"2024-09-11T21:55:18.032218+0000","last_became_peered":"2024-09-11T21:55:18.032218+0000","last_unstale":"2024-09-11T21:57:13.788356+0000","last_undegraded":"2024-09-11T21:57:13.788356+0000","last_fullsized":"2024-09-11T21:57:13.788356+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:38:20.872069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.753229+0000","last_change":"2024-09-11T21:55:20.090030+0000","last_active":"2024-09-11T21:57:13.753229+0000","last_peered":"2024-09-11T21:57:13.753229+0000","last_clean":"2024-09-11T21:57:13.753229+0000","last_became_active":"2024-09-11T21:55:20.089481+0000","last_became_peered":"2024-09-11T21:55:20.089481+0000","last_unstale":"2024-09-11T21:57:13.753229+0000","last_undegraded":"2024-09-11T21:57:13.753229+0000","last_fullsized":"2024-09-11T21:57:13.753229+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:41:57.038519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"84'3","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.762394+0000","last_change":"2024-09-11T21:55:18.026656+0000","last_active":"2024-09-11T21:57:13.762394+0000","last_peered":"2024-09-11T21:57:13.762394+0000","last_clean":"2024-09-11T21:57:13.762394+0000","last_became_active":"2024-09-11T21:55:18.026318+0000","last_became_peered":"2024-09-11T21:55:18.026318+0000","last_unstale":"2024-09-11T21:57:13.762394+0000","last_undegraded":"2024-09-11T21:57:13.762394+0000","last_fullsized":"2024-09-11T21:57:13.762394+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:14:41.438334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"86'11","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.750970+0000","last_change":"2024-09-11T21:55:20.090790+0000","last_active":"2024-09-11T21:57:13.750970+0000","last_peered":"2024-09-11T21:57:13.750970+0000","last_clean":"2024-09-11T21:57:13.750970+0000","last_became_active":"2024-09-11T21:55:20.090468+0000","last_became_peered":"2024-09-11T21:55:20.090468+0000","last_unstale":"2024-09-11T21:57:13.750970+0000","last_undegraded":"2024-09-11T21:57:13.750970+0000","last_fullsized":"2024-09-11T21:57:13.750970+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:19.115482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'8","reported_seq":248,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:19.304944+0000","last_change":"2024-09-11T21:55:18.026989+0000","last_active":"2024-09-11T21:57:19.304944+0000","last_peered":"2024-09-11T21:57:19.304944+0000","last_clean":"2024-09-11T21:57:19.304944+0000","last_became_active":"2024-09-11T21:55:18.026229+0000","last_became_peered":"2024-09-11T21:55:18.026229+0000","last_unstale":"2024-09-11T21:57:19.304944+0000","last_undegraded":"2024-09-11T21:57:19.304944+0000","last_fullsized":"2024-09-11T21:57:19.304944+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:50:35.745789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":216,"num_read_kb":216,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"86'10","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788388+0000","last_change":"2024-09-11T21:55:20.095884+0000","last_active":"2024-09-11T21:57:13.788388+0000","last_peered":"2024-09-11T21:57:13.788388+0000","last_clean":"2024-09-11T21:57:13.788388+0000","last_became_active":"2024-09-11T21:55:20.094887+0000","last_became_peered":"2024-09-11T21:55:20.094887+0000","last_unstale":"2024-09-11T21:57:13.788388+0000","last_undegraded":"2024-09-11T21:57:13.788388+0000","last_fullsized":"2024-09-11T21:57:13.788388+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:45:03.579703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"84'4","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671167+0000","last_change":"2024-09-11T21:55:18.030474+0000","last_active":"2024-09-11T21:57:13.671167+0000","last_peered":"2024-09-11T21:57:13.671167+0000","last_clean":"2024-09-11T21:57:13.671167+0000","last_became_active":"2024-09-11T21:55:18.030104+0000","last_became_peered":"2024-09-11T21:55:18.030104+0000","last_unstale":"2024-09-11T21:57:13.671167+0000","last_undegraded":"2024-09-11T21:57:13.671167+0000","last_fullsized":"2024-09-11T21:57:13.671167+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:21:00.484595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.669795+0000","last_change":"2024-09-11T21:55:20.102807+0000","last_active":"2024-09-11T21:57:13.669795+0000","last_peered":"2024-09-11T21:57:13.669795+0000","last_clean":"2024-09-11T21:57:13.669795+0000","last_became_active":"2024-09-11T21:55:20.102581+0000","last_became_peered":"2024-09-11T21:55:20.102581+0000","last_unstale":"2024-09-11T21:57:13.669795+0000","last_undegraded":"2024-09-11T21:57:13.669795+0000","last_fullsized":"2024-09-11T21:57:13.669795+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:29:23.139788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"84'5","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.762200+0000","last_change":"2024-09-11T21:55:18.117685+0000","last_active":"2024-09-11T21:57:13.762200+0000","last_peered":"2024-09-11T21:57:13.762200+0000","last_clean":"2024-09-11T21:57:13.762200+0000","last_became_active":"2024-09-11T21:55:18.117261+0000","last_became_peered":"2024-09-11T21:55:18.117261+0000","last_unstale":"2024-09-11T21:57:13.762200+0000","last_undegraded":"2024-09-11T21:57:13.762200+0000","last_fullsized":"2024-09-11T21:57:13.762200+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:42:13.180725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"86'11","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788081+0000","last_change":"2024-09-11T21:55:20.086562+0000","last_active":"2024-09-11T21:57:13.788081+0000","last_peered":"2024-09-11T21:57:13.788081+0000","last_clean":"2024-09-11T21:57:13.788081+0000","last_became_active":"2024-09-11T21:55:20.086197+0000","last_became_peered":"2024-09-11T21:55:20.086197+0000","last_unstale":"2024-09-11T21:57:13.788081+0000","last_undegraded":"2024-09-11T21:57:13.788081+0000","last_fullsized":"2024-09-11T21:57:13.788081+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:23:44.705132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'35","reported_seq":306,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:19.363897+0000","last_change":"2024-09-11T21:56:26.299386+0000","last_active":"2024-09-11T21:57:19.363897+0000","last_peered":"2024-09-11T21:57:19.363897+0000","last_clean":"2024-09-11T21:57:19.363897+0000","last_became_active":"2024-09-11T21:56:26.291951+0000","last_became_peered":"2024-09-11T21:56:26.291951+0000","last_unstale":"2024-09-11T21:57:19.363897+0000","last_undegraded":"2024-09-11T21:57:19.363897+0000","last_fullsized":"2024-09-11T21:57:19.363897+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:54:47.346798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65927,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":219,"num_read_kb":174,"num_write":37,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":59211,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"86'6","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671198+0000","last_change":"2024-09-11T21:55:20.091695+0000","last_active":"2024-09-11T21:57:13.671198+0000","last_peered":"2024-09-11T21:57:13.671198+0000","last_clean":"2024-09-11T21:57:13.671198+0000","last_became_active":"2024-09-11T21:55:20.091128+0000","last_became_peered":"2024-09-11T21:55:20.091128+0000","last_unstale":"2024-09-11T21:57:13.671198+0000","last_undegraded":"2024-09-11T21:57:13.671198+0000","last_fullsized":"2024-09-11T21:57:13.671198+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:16:20.751952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"84'9","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671024+0000","last_change":"2024-09-11T21:55:18.030827+0000","last_active":"2024-09-11T21:57:13.671024+0000","last_peered":"2024-09-11T21:57:13.671024+0000","last_clean":"2024-09-11T21:57:13.671024+0000","last_became_active":"2024-09-11T21:55:18.030354+0000","last_became_peered":"2024-09-11T21:55:18.030354+0000","last_unstale":"2024-09-11T21:57:13.671024+0000","last_undegraded":"2024-09-11T21:57:13.671024+0000","last_fullsized":"2024-09-11T21:57:13.671024+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:23:19.578820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788550+0000","last_change":"2024-09-11T21:55:20.090160+0000","last_active":"2024-09-11T21:57:13.788550+0000","last_peered":"2024-09-11T21:57:13.788550+0000","last_clean":"2024-09-11T21:57:13.788550+0000","last_became_active":"2024-09-11T21:55:20.089801+0000","last_became_peered":"2024-09-11T21:55:20.089801+0000","last_unstale":"2024-09-11T21:57:13.788550+0000","last_undegraded":"2024-09-11T21:57:13.788550+0000","last_fullsized":"2024-09-11T21:57:13.788550+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:34:16.093123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"84'5","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.761998+0000","last_change":"2024-09-11T21:55:18.026817+0000","last_active":"2024-09-11T21:57:13.761998+0000","last_peered":"2024-09-11T21:57:13.761998+0000","last_clean":"2024-09-11T21:57:13.761998+0000","last_became_active":"2024-09-11T21:55:18.026188+0000","last_became_peered":"2024-09-11T21:55:18.026188+0000","last_unstale":"2024-09-11T21:57:13.761998+0000","last_undegraded":"2024-09-11T21:57:13.761998+0000","last_fullsized":"2024-09-11T21:57:13.761998+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:55:14.774503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"86'15","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.670991+0000","last_change":"2024-09-11T21:55:20.095613+0000","last_active":"2024-09-11T21:57:13.670991+0000","last_peered":"2024-09-11T21:57:13.670991+0000","last_clean":"2024-09-11T21:57:13.670991+0000","last_became_active":"2024-09-11T21:55:20.095302+0000","last_became_peered":"2024-09-11T21:55:20.095302+0000","last_unstale":"2024-09-11T21:57:13.670991+0000","last_undegraded":"2024-09-11T21:57:13.670991+0000","last_fullsized":"2024-09-11T21:57:13.670991+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:37:27.961213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'7","reported_seq":143,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:18.305422+0000","last_change":"2024-09-11T21:55:18.032417+0000","last_active":"2024-09-11T21:57:18.305422+0000","last_peered":"2024-09-11T21:57:18.305422+0000","last_clean":"2024-09-11T21:57:18.305422+0000","last_became_active":"2024-09-11T21:55:18.031419+0000","last_became_peered":"2024-09-11T21:55:18.031419+0000","last_unstale":"2024-09-11T21:57:18.305422+0000","last_undegraded":"2024-09-11T21:57:18.305422+0000","last_fullsized":"2024-09-11T21:57:18.305422+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:30:52.285678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1289,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":113,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"86'6","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.670730+0000","last_change":"2024-09-11T21:55:20.101965+0000","last_active":"2024-09-11T21:57:13.670730+0000","last_peered":"2024-09-11T21:57:13.670730+0000","last_clean":"2024-09-11T21:57:13.670730+0000","last_became_active":"2024-09-11T21:55:20.101534+0000","last_became_peered":"2024-09-11T21:55:20.101534+0000","last_unstale":"2024-09-11T21:57:13.670730+0000","last_undegraded":"2024-09-11T21:57:13.670730+0000","last_fullsized":"2024-09-11T21:57:13.670730+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:41:30.685121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"84'17","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.670811+0000","last_change":"2024-09-11T21:55:18.116059+0000","last_active":"2024-09-11T21:57:13.670811+0000","last_peered":"2024-09-11T21:57:13.670811+0000","last_clean":"2024-09-11T21:57:13.670811+0000","last_became_active":"2024-09-11T21:55:18.115621+0000","last_became_peered":"2024-09-11T21:55:18.115621+0000","last_unstale":"2024-09-11T21:57:13.670811+0000","last_undegraded":"2024-09-11T21:57:13.670811+0000","last_fullsized":"2024-09-11T21:57:13.670811+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:47:17.815695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"86'6","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.762037+0000","last_change":"2024-09-11T21:55:20.100903+0000","last_active":"2024-09-11T21:57:13.762037+0000","last_peered":"2024-09-11T21:57:13.762037+0000","last_clean":"2024-09-11T21:57:13.762037+0000","last_became_active":"2024-09-11T21:55:20.100561+0000","last_became_peered":"2024-09-11T21:55:20.100561+0000","last_unstale":"2024-09-11T21:57:13.762037+0000","last_undegraded":"2024-09-11T21:57:13.762037+0000","last_fullsized":"2024-09-11T21:57:13.762037+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:00:45.472238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'2","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671632+0000","last_change":"2024-09-11T21:55:18.029549+0000","last_active":"2024-09-11T21:57:13.671632+0000","last_peered":"2024-09-11T21:57:13.671632+0000","last_clean":"2024-09-11T21:57:13.671632+0000","last_became_active":"2024-09-11T21:55:18.029178+0000","last_became_peered":"2024-09-11T21:55:18.029178+0000","last_unstale":"2024-09-11T21:57:13.671632+0000","last_undegraded":"2024-09-11T21:57:13.671632+0000","last_fullsized":"2024-09-11T21:57:13.671632+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:37:07.915786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"86'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788144+0000","last_change":"2024-09-11T21:55:20.104202+0000","last_active":"2024-09-11T21:57:13.788144+0000","last_peered":"2024-09-11T21:57:13.788144+0000","last_clean":"2024-09-11T21:57:13.788144+0000","last_became_active":"2024-09-11T21:55:20.103464+0000","last_became_peered":"2024-09-11T21:55:20.103464+0000","last_unstale":"2024-09-11T21:57:13.788144+0000","last_undegraded":"2024-09-11T21:57:13.788144+0000","last_fullsized":"2024-09-11T21:57:13.788144+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:30:16.044143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"84'5","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788728+0000","last_change":"2024-09-11T21:55:18.026183+0000","last_active":"2024-09-11T21:57:13.788728+0000","last_peered":"2024-09-11T21:57:13.788728+0000","last_clean":"2024-09-11T21:57:13.788728+0000","last_became_active":"2024-09-11T21:55:18.025849+0000","last_became_peered":"2024-09-11T21:55:18.025849+0000","last_unstale":"2024-09-11T21:57:13.788728+0000","last_undegraded":"2024-09-11T21:57:13.788728+0000","last_fullsized":"2024-09-11T21:57:13.788728+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:21:12.189158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671103+0000","last_change":"2024-09-11T21:55:20.090652+0000","last_active":"2024-09-11T21:57:13.671103+0000","last_peered":"2024-09-11T21:57:13.671103+0000","last_clean":"2024-09-11T21:57:13.671103+0000","last_became_active":"2024-09-11T21:55:20.090348+0000","last_became_peered":"2024-09-11T21:55:20.090348+0000","last_unstale":"2024-09-11T21:57:13.671103+0000","last_undegraded":"2024-09-11T21:57:13.671103+0000","last_fullsized":"2024-09-11T21:57:13.671103+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:10:58.733371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'41","reported_seq":73,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.873064+0000","last_change":"2024-09-11T21:55:18.024388+0000","last_active":"2024-09-11T21:57:13.873064+0000","last_peered":"2024-09-11T21:57:13.873064+0000","last_clean":"2024-09-11T21:57:13.873064+0000","last_became_active":"2024-09-11T21:55:18.024100+0000","last_became_peered":"2024-09-11T21:55:18.024100+0000","last_unstale":"2024-09-11T21:57:13.873064+0000","last_undegraded":"2024-09-11T21:57:13.873064+0000","last_fullsized":"2024-09-11T21:57:13.873064+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:21:11.152282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.670936+0000","last_change":"2024-09-11T21:55:20.101007+0000","last_active":"2024-09-11T21:57:13.670936+0000","last_peered":"2024-09-11T21:57:13.670936+0000","last_clean":"2024-09-11T21:57:13.670936+0000","last_became_active":"2024-09-11T21:55:20.100642+0000","last_became_peered":"2024-09-11T21:55:20.100642+0000","last_unstale":"2024-09-11T21:57:13.670936+0000","last_undegraded":"2024-09-11T21:57:13.670936+0000","last_fullsized":"2024-09-11T21:57:13.670936+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:16:05.642184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"87'8","reported_seq":145,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:18.331840+0000","last_change":"2024-09-11T21:55:18.116216+0000","last_active":"2024-09-11T21:57:18.331840+0000","last_peered":"2024-09-11T21:57:18.331840+0000","last_clean":"2024-09-11T21:57:18.331840+0000","last_became_active":"2024-09-11T21:55:18.115871+0000","last_became_peered":"2024-09-11T21:55:18.115871+0000","last_unstale":"2024-09-11T21:57:18.331840+0000","last_undegraded":"2024-09-11T21:57:18.331840+0000","last_fullsized":"2024-09-11T21:57:18.331840+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:16:56.027636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":113,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671175+0000","last_change":"2024-09-11T21:55:20.092421+0000","last_active":"2024-09-11T21:57:13.671175+0000","last_peered":"2024-09-11T21:57:13.671175+0000","last_clean":"2024-09-11T21:57:13.671175+0000","last_became_active":"2024-09-11T21:55:20.091707+0000","last_became_peered":"2024-09-11T21:55:20.091707+0000","last_unstale":"2024-09-11T21:57:13.671175+0000","last_undegraded":"2024-09-11T21:57:13.671175+0000","last_fullsized":"2024-09-11T21:57:13.671175+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:12:38.157687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"87'148","reported_seq":397,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:15.768029+0000","last_change":"2024-09-11T21:54:07.542362+0000","last_active":"2024-09-11T21:57:15.768029+0000","last_peered":"2024-09-11T21:57:15.768029+0000","last_clean":"2024-09-11T21:57:15.768029+0000","last_became_active":"2024-09-11T21:54:07.541871+0000","last_became_peered":"2024-09-11T21:54:07.541871+0000","last_unstale":"2024-09-11T21:57:15.768029+0000","last_undegraded":"2024-09-11T21:57:15.768029+0000","last_fullsized":"2024-09-11T21:57:15.768029+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:50:41.026895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:50:41.026895+0000","last_clean_scrub_stamp":"2024-09-11T21:50:41.026895+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:05:06.154268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"84'6","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788499+0000","last_change":"2024-09-11T21:55:18.030558+0000","last_active":"2024-09-11T21:57:13.788499+0000","last_peered":"2024-09-11T21:57:13.788499+0000","last_clean":"2024-09-11T21:57:13.788499+0000","last_became_active":"2024-09-11T21:55:18.030216+0000","last_became_peered":"2024-09-11T21:55:18.030216+0000","last_unstale":"2024-09-11T21:57:13.788499+0000","last_undegraded":"2024-09-11T21:57:13.788499+0000","last_fullsized":"2024-09-11T21:57:13.788499+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:33:02.589999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.753426+0000","last_change":"2024-09-11T21:55:20.092938+0000","last_active":"2024-09-11T21:57:13.753426+0000","last_peered":"2024-09-11T21:57:13.753426+0000","last_clean":"2024-09-11T21:57:13.753426+0000","last_became_active":"2024-09-11T21:55:20.092615+0000","last_became_peered":"2024-09-11T21:55:20.092615+0000","last_unstale":"2024-09-11T21:57:13.753426+0000","last_undegraded":"2024-09-11T21:57:13.753426+0000","last_fullsized":"2024-09-11T21:57:13.753426+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:40:06.957836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"84'4","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.670838+0000","last_change":"2024-09-11T21:55:18.029127+0000","last_active":"2024-09-11T21:57:13.670838+0000","last_peered":"2024-09-11T21:57:13.670838+0000","last_clean":"2024-09-11T21:57:13.670838+0000","last_became_active":"2024-09-11T21:55:18.028740+0000","last_became_peered":"2024-09-11T21:55:18.028740+0000","last_unstale":"2024-09-11T21:57:13.670838+0000","last_undegraded":"2024-09-11T21:57:13.670838+0000","last_fullsized":"2024-09-11T21:57:13.670838+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:14:01.128486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"86'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788303+0000","last_change":"2024-09-11T21:55:20.092498+0000","last_active":"2024-09-11T21:57:13.788303+0000","last_peered":"2024-09-11T21:57:13.788303+0000","last_clean":"2024-09-11T21:57:13.788303+0000","last_became_active":"2024-09-11T21:55:20.092179+0000","last_became_peered":"2024-09-11T21:55:20.092179+0000","last_unstale":"2024-09-11T21:57:13.788303+0000","last_undegraded":"2024-09-11T21:57:13.788303+0000","last_fullsized":"2024-09-11T21:57:13.788303+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:48:58.536942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"86'6","reported_seq":139,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:18.305052+0000","last_change":"2024-09-11T21:55:18.115798+0000","last_active":"2024-09-11T21:57:18.305052+0000","last_peered":"2024-09-11T21:57:18.305052+0000","last_clean":"2024-09-11T21:57:18.305052+0000","last_became_active":"2024-09-11T21:55:18.115433+0000","last_became_peered":"2024-09-11T21:55:18.115433+0000","last_unstale":"2024-09-11T21:57:18.305052+0000","last_undegraded":"2024-09-11T21:57:18.305052+0000","last_fullsized":"2024-09-11T21:57:18.305052+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:03:32.097756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":109,"num_read_kb":109,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.762043+0000","last_change":"2024-09-11T21:55:20.100439+0000","last_active":"2024-09-11T21:57:13.762043+0000","last_peered":"2024-09-11T21:57:13.762043+0000","last_clean":"2024-09-11T21:57:13.762043+0000","last_became_active":"2024-09-11T21:55:20.098717+0000","last_became_peered":"2024-09-11T21:55:20.098717+0000","last_unstale":"2024-09-11T21:57:13.762043+0000","last_undegraded":"2024-09-11T21:57:13.762043+0000","last_fullsized":"2024-09-11T21:57:13.762043+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:15:10.874225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"87'9","reported_seq":145,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:18.303752+0000","last_change":"2024-09-11T21:55:18.026588+0000","last_active":"2024-09-11T21:57:18.303752+0000","last_peered":"2024-09-11T21:57:18.303752+0000","last_clean":"2024-09-11T21:57:18.303752+0000","last_became_active":"2024-09-11T21:55:18.026205+0000","last_became_peered":"2024-09-11T21:55:18.026205+0000","last_unstale":"2024-09-11T21:57:18.303752+0000","last_undegraded":"2024-09-11T21:57:18.303752+0000","last_fullsized":"2024-09-11T21:57:18.303752+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:45:34.701088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1908,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":112,"num_read_kb":4,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.765120+0000","last_change":"2024-09-11T21:55:20.100677+0000","last_active":"2024-09-11T21:57:13.765120+0000","last_peered":"2024-09-11T21:57:13.765120+0000","last_clean":"2024-09-11T21:57:13.765120+0000","last_became_active":"2024-09-11T21:55:20.100334+0000","last_became_peered":"2024-09-11T21:55:20.100334+0000","last_unstale":"2024-09-11T21:57:13.765120+0000","last_undegraded":"2024-09-11T21:57:13.765120+0000","last_fullsized":"2024-09-11T21:57:13.765120+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:15:16.792831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"84'1","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.751199+0000","last_change":"2024-09-11T21:55:18.116329+0000","last_active":"2024-09-11T21:57:13.751199+0000","last_peered":"2024-09-11T21:57:13.751199+0000","last_clean":"2024-09-11T21:57:13.751199+0000","last_became_active":"2024-09-11T21:55:18.115944+0000","last_became_peered":"2024-09-11T21:55:18.115944+0000","last_unstale":"2024-09-11T21:57:13.751199+0000","last_undegraded":"2024-09-11T21:57:13.751199+0000","last_fullsized":"2024-09-11T21:57:13.751199+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:03:04.164985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.765132+0000","last_change":"2024-09-11T21:55:20.089202+0000","last_active":"2024-09-11T21:57:13.765132+0000","last_peered":"2024-09-11T21:57:13.765132+0000","last_clean":"2024-09-11T21:57:13.765132+0000","last_became_active":"2024-09-11T21:55:20.088853+0000","last_became_peered":"2024-09-11T21:55:20.088853+0000","last_unstale":"2024-09-11T21:57:13.765132+0000","last_undegraded":"2024-09-11T21:57:13.765132+0000","last_fullsized":"2024-09-11T21:57:13.765132+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:39:36.496944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"84'4","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671429+0000","last_change":"2024-09-11T21:55:18.117917+0000","last_active":"2024-09-11T21:57:13.671429+0000","last_peered":"2024-09-11T21:57:13.671429+0000","last_clean":"2024-09-11T21:57:13.671429+0000","last_became_active":"2024-09-11T21:55:18.117567+0000","last_became_peered":"2024-09-11T21:55:18.117567+0000","last_unstale":"2024-09-11T21:57:13.671429+0000","last_undegraded":"2024-09-11T21:57:13.671429+0000","last_fullsized":"2024-09-11T21:57:13.671429+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:21:32.546620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.789090+0000","last_change":"2024-09-11T21:55:20.092370+0000","last_active":"2024-09-11T21:57:13.789090+0000","last_peered":"2024-09-11T21:57:13.789090+0000","last_clean":"2024-09-11T21:57:13.789090+0000","last_became_active":"2024-09-11T21:55:20.091982+0000","last_became_peered":"2024-09-11T21:55:20.091982+0000","last_unstale":"2024-09-11T21:57:13.789090+0000","last_undegraded":"2024-09-11T21:57:13.789090+0000","last_fullsized":"2024-09-11T21:57:13.789090+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:59:55.214511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'51","reported_seq":98,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.879119+0000","last_change":"2024-09-11T21:55:18.023166+0000","last_active":"2024-09-11T21:57:13.879119+0000","last_peered":"2024-09-11T21:57:13.879119+0000","last_clean":"2024-09-11T21:57:13.879119+0000","last_became_active":"2024-09-11T21:55:18.022845+0000","last_became_peered":"2024-09-11T21:55:18.022845+0000","last_unstale":"2024-09-11T21:57:13.879119+0000","last_undegraded":"2024-09-11T21:57:13.879119+0000","last_fullsized":"2024-09-11T21:57:13.879119+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:08:06.770410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":91,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"86'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.753738+0000","last_change":"2024-09-11T21:55:20.091541+0000","last_active":"2024-09-11T21:57:13.753738+0000","last_peered":"2024-09-11T21:57:13.753738+0000","last_clean":"2024-09-11T21:57:13.753738+0000","last_became_active":"2024-09-11T21:55:20.090971+0000","last_became_peered":"2024-09-11T21:55:20.090971+0000","last_unstale":"2024-09-11T21:57:13.753738+0000","last_undegraded":"2024-09-11T21:57:13.753738+0000","last_fullsized":"2024-09-11T21:57:13.753738+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:22:31.974711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"84'7","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788398+0000","last_change":"2024-09-11T21:55:18.030694+0000","last_active":"2024-09-11T21:57:13.788398+0000","last_peered":"2024-09-11T21:57:13.788398+0000","last_clean":"2024-09-11T21:57:13.788398+0000","last_became_active":"2024-09-11T21:55:18.029687+0000","last_became_peered":"2024-09-11T21:55:18.029687+0000","last_unstale":"2024-09-11T21:57:13.788398+0000","last_undegraded":"2024-09-11T21:57:13.788398+0000","last_fullsized":"2024-09-11T21:57:13.788398+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:18:15.009543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"86'7","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.669625+0000","last_change":"2024-09-11T21:56:26.376104+0000","last_active":"2024-09-11T21:57:13.669625+0000","last_peered":"2024-09-11T21:57:13.669625+0000","last_clean":"2024-09-11T21:57:13.669625+0000","last_became_active":"2024-09-11T21:56:26.375665+0000","last_became_peered":"2024-09-11T21:56:26.375665+0000","last_unstale":"2024-09-11T21:57:13.669625+0000","last_undegraded":"2024-09-11T21:57:13.669625+0000","last_fullsized":"2024-09-11T21:57:13.669625+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:19:46.616205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"84'2","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788029+0000","last_change":"2024-09-11T21:55:18.115977+0000","last_active":"2024-09-11T21:57:13.788029+0000","last_peered":"2024-09-11T21:57:13.788029+0000","last_clean":"2024-09-11T21:57:13.788029+0000","last_became_active":"2024-09-11T21:55:18.115649+0000","last_became_peered":"2024-09-11T21:55:18.115649+0000","last_unstale":"2024-09-11T21:57:13.788029+0000","last_undegraded":"2024-09-11T21:57:13.788029+0000","last_fullsized":"2024-09-11T21:57:13.788029+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:42:07.621024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"86'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.670884+0000","last_change":"2024-09-11T21:55:20.095536+0000","last_active":"2024-09-11T21:57:13.670884+0000","last_peered":"2024-09-11T21:57:13.670884+0000","last_clean":"2024-09-11T21:57:13.670884+0000","last_became_active":"2024-09-11T21:55:20.095117+0000","last_became_peered":"2024-09-11T21:55:20.095117+0000","last_unstale":"2024-09-11T21:57:13.670884+0000","last_undegraded":"2024-09-11T21:57:13.670884+0000","last_fullsized":"2024-09-11T21:57:13.670884+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:46.524922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"84'7","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.753348+0000","last_change":"2024-09-11T21:55:18.026592+0000","last_active":"2024-09-11T21:57:13.753348+0000","last_peered":"2024-09-11T21:57:13.753348+0000","last_clean":"2024-09-11T21:57:13.753348+0000","last_became_active":"2024-09-11T21:55:18.026263+0000","last_became_peered":"2024-09-11T21:55:18.026263+0000","last_unstale":"2024-09-11T21:57:13.753348+0000","last_undegraded":"2024-09-11T21:57:13.753348+0000","last_fullsized":"2024-09-11T21:57:13.753348+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:22:29.382964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788053+0000","last_change":"2024-09-11T21:55:20.093655+0000","last_active":"2024-09-11T21:57:13.788053+0000","last_peered":"2024-09-11T21:57:13.788053+0000","last_clean":"2024-09-11T21:57:13.788053+0000","last_became_active":"2024-09-11T21:55:20.093287+0000","last_became_peered":"2024-09-11T21:55:20.093287+0000","last_unstale":"2024-09-11T21:57:13.788053+0000","last_undegraded":"2024-09-11T21:57:13.788053+0000","last_fullsized":"2024-09-11T21:57:13.788053+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:46:15.912710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"84'3","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.670879+0000","last_change":"2024-09-11T21:55:18.028465+0000","last_active":"2024-09-11T21:57:13.670879+0000","last_peered":"2024-09-11T21:57:13.670879+0000","last_clean":"2024-09-11T21:57:13.670879+0000","last_became_active":"2024-09-11T21:55:18.028139+0000","last_became_peered":"2024-09-11T21:55:18.028139+0000","last_unstale":"2024-09-11T21:57:13.670879+0000","last_undegraded":"2024-09-11T21:57:13.670879+0000","last_fullsized":"2024-09-11T21:57:13.670879+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:52:47.808058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.753473+0000","last_change":"2024-09-11T21:55:20.096400+0000","last_active":"2024-09-11T21:57:13.753473+0000","last_peered":"2024-09-11T21:57:13.753473+0000","last_clean":"2024-09-11T21:57:13.753473+0000","last_became_active":"2024-09-11T21:55:20.096023+0000","last_became_peered":"2024-09-11T21:55:20.096023+0000","last_unstale":"2024-09-11T21:57:13.753473+0000","last_undegraded":"2024-09-11T21:57:13.753473+0000","last_fullsized":"2024-09-11T21:57:13.753473+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:19:34.150687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"84'4","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671181+0000","last_change":"2024-09-11T21:55:18.028759+0000","last_active":"2024-09-11T21:57:13.671181+0000","last_peered":"2024-09-11T21:57:13.671181+0000","last_clean":"2024-09-11T21:57:13.671181+0000","last_became_active":"2024-09-11T21:55:18.028366+0000","last_became_peered":"2024-09-11T21:55:18.028366+0000","last_unstale":"2024-09-11T21:57:13.671181+0000","last_undegraded":"2024-09-11T21:57:13.671181+0000","last_fullsized":"2024-09-11T21:57:13.671181+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:42:05.836114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"86'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671002+0000","last_change":"2024-09-11T21:55:20.093593+0000","last_active":"2024-09-11T21:57:13.671002+0000","last_peered":"2024-09-11T21:57:13.671002+0000","last_clean":"2024-09-11T21:57:13.671002+0000","last_became_active":"2024-09-11T21:55:20.093189+0000","last_became_peered":"2024-09-11T21:55:20.093189+0000","last_unstale":"2024-09-11T21:57:13.671002+0000","last_undegraded":"2024-09-11T21:57:13.671002+0000","last_fullsized":"2024-09-11T21:57:13.671002+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:47:36.802816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"86'15","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.762163+0000","last_change":"2024-09-11T21:56:26.292793+0000","last_active":"2024-09-11T21:57:13.762163+0000","last_peered":"2024-09-11T21:57:13.762163+0000","last_clean":"2024-09-11T21:57:13.762163+0000","last_became_active":"2024-09-11T21:56:26.292437+0000","last_became_peered":"2024-09-11T21:56:26.292437+0000","last_unstale":"2024-09-11T21:57:13.762163+0000","last_undegraded":"2024-09-11T21:57:13.762163+0000","last_fullsized":"2024-09-11T21:57:13.762163+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:49:08.867435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"84'5","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671366+0000","last_change":"2024-09-11T21:55:18.027365+0000","last_active":"2024-09-11T21:57:13.671366+0000","last_peered":"2024-09-11T21:57:13.671366+0000","last_clean":"2024-09-11T21:57:13.671366+0000","last_became_active":"2024-09-11T21:55:18.027058+0000","last_became_peered":"2024-09-11T21:55:18.027058+0000","last_unstale":"2024-09-11T21:57:13.671366+0000","last_undegraded":"2024-09-11T21:57:13.671366+0000","last_fullsized":"2024-09-11T21:57:13.671366+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:30:50.613246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788034+0000","last_change":"2024-09-11T21:55:20.095154+0000","last_active":"2024-09-11T21:57:13.788034+0000","last_peered":"2024-09-11T21:57:13.788034+0000","last_clean":"2024-09-11T21:57:13.788034+0000","last_became_active":"2024-09-11T21:55:20.093607+0000","last_became_peered":"2024-09-11T21:55:20.093607+0000","last_unstale":"2024-09-11T21:57:13.788034+0000","last_undegraded":"2024-09-11T21:57:13.788034+0000","last_fullsized":"2024-09-11T21:57:13.788034+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:27:53.612467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"87'13","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671385+0000","last_change":"2024-09-11T21:55:18.029872+0000","last_active":"2024-09-11T21:57:13.671385+0000","last_peered":"2024-09-11T21:57:13.671385+0000","last_clean":"2024-09-11T21:57:13.671385+0000","last_became_active":"2024-09-11T21:55:18.029538+0000","last_became_peered":"2024-09-11T21:55:18.029538+0000","last_unstale":"2024-09-11T21:57:13.671385+0000","last_undegraded":"2024-09-11T21:57:13.671385+0000","last_fullsized":"2024-09-11T21:57:13.671385+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:05:16.824805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"86'6","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.669783+0000","last_change":"2024-09-11T21:55:20.098713+0000","last_active":"2024-09-11T21:57:13.669783+0000","last_peered":"2024-09-11T21:57:13.669783+0000","last_clean":"2024-09-11T21:57:13.669783+0000","last_became_active":"2024-09-11T21:55:20.098219+0000","last_became_peered":"2024-09-11T21:55:20.098219+0000","last_unstale":"2024-09-11T21:57:13.669783+0000","last_undegraded":"2024-09-11T21:57:13.669783+0000","last_fullsized":"2024-09-11T21:57:13.669783+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:07:10.585366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"84'2","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.753877+0000","last_change":"2024-09-11T21:55:18.115868+0000","last_active":"2024-09-11T21:57:13.753877+0000","last_peered":"2024-09-11T21:57:13.753877+0000","last_clean":"2024-09-11T21:57:13.753877+0000","last_became_active":"2024-09-11T21:55:18.115486+0000","last_became_peered":"2024-09-11T21:55:18.115486+0000","last_unstale":"2024-09-11T21:57:13.753877+0000","last_undegraded":"2024-09-11T21:57:13.753877+0000","last_fullsized":"2024-09-11T21:57:13.753877+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:16:57.932041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"86'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.753721+0000","last_change":"2024-09-11T21:55:20.096077+0000","last_active":"2024-09-11T21:57:13.753721+0000","last_peered":"2024-09-11T21:57:13.753721+0000","last_clean":"2024-09-11T21:57:13.753721+0000","last_became_active":"2024-09-11T21:55:20.095739+0000","last_became_peered":"2024-09-11T21:55:20.095739+0000","last_unstale":"2024-09-11T21:57:13.753721+0000","last_undegraded":"2024-09-11T21:57:13.753721+0000","last_fullsized":"2024-09-11T21:57:13.753721+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:05:28.089828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"87'15","reported_seq":149,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:19.304465+0000","last_change":"2024-09-11T21:55:18.025771+0000","last_active":"2024-09-11T21:57:19.304465+0000","last_peered":"2024-09-11T21:57:19.304465+0000","last_clean":"2024-09-11T21:57:19.304465+0000","last_became_active":"2024-09-11T21:55:18.025384+0000","last_became_peered":"2024-09-11T21:55:18.025384+0000","last_unstale":"2024-09-11T21:57:19.304465+0000","last_undegraded":"2024-09-11T21:57:19.304465+0000","last_fullsized":"2024-09-11T21:57:19.304465+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:58:21.070061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":109,"num_read_kb":109,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":601040,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1216,"num_read_kb":726,"num_write":380,"num_write_kb":1470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":59211,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":667648,"data_stored":1803120,"data_compressed":85492,"data_compressed_allocated":110592,"data_compressed_original":1605632,"omap_allocated":0,"internal_metadata":0},"log_size":306,"ondisk_log_size":306,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107988,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449125,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":950272,"data_stored":462934,"data_compressed":8675,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678057,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36448,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708704,"statfs":{"total":95995035648,"available":95957712896,"internally_reserved":0,"allocated":1130496,"data_stored":1251264,"data_compressed":50827,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":66,"seq":283467841583,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":999424,"data_stored":855153,"data_compressed":27821,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103221,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":995328,"data_stored":855531,"data_compressed":27821,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332153,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1024000,"data_stored":1057390,"data_compressed":44671,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495678,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":958464,"data_stored":462974,"data_compressed":8675,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757316,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":1186647,"data_compressed":50827,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920840,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":987136,"data_stored":528143,"data_compressed":18261,"data_compressed_allocated":24576,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182478,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":986295,"data_compressed":33977,"data_compressed_allocated":45056,"data_compressed_original":540672,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":970752,"data_stored":464641,"data_compressed":8675,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1024000,"data_stored":529814,"data_compressed":8675,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":978944,"data_stored":463920,"data_compressed":8675,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":68999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":67328,"data_compressed":9586,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":135347,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":6090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":394716,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":199964,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T21:57:25.003 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T21:57:25.264 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:57:25.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:24 smithi053 ceph-mon[34908]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 0 B/s wr, 19 op/s 2024-09-11T21:57:25.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:24 smithi053 ceph-mon[34908]: from='client.15402 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T21:57:25.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:24 smithi161 ceph-mon[35615]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 0 B/s wr, 19 op/s 2024-09-11T21:57:25.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:24 smithi161 ceph-mon[35615]: from='client.15402 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T21:57:25.385 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:24 smithi001 ceph-mon[30935]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 0 B/s wr, 19 op/s 2024-09-11T21:57:25.386 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:24 smithi001 ceph-mon[30935]: from='client.15402 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T21:57:25.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:25.744+0000 7f984ec33640 1 -- 172.21.15.1:0/327384272 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9848101de0 msgr2=0x7f98481021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:25.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:25.744+0000 7f984ec33640 1 --2- 172.21.15.1:0/327384272 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9848101de0 0x7f98481021c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f9838009900 tx=0x7f983802a410 comp rx=0 tx=0).stop 2024-09-11T21:57:25.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:25.745+0000 7f984ec33640 1 -- 172.21.15.1:0/327384272 shutdown_connections 2024-09-11T21:57:25.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:25.745+0000 7f984ec33640 1 --2- 172.21.15.1:0/327384272 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9848106f10 0x7f984810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:25.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:25.745+0000 7f984ec33640 1 --2- 172.21.15.1:0/327384272 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9848102790 0x7f98481067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:25.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:25.745+0000 7f984ec33640 1 --2- 172.21.15.1:0/327384272 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9848101de0 0x7f98481021c0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:25.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:25.745+0000 7f984ec33640 1 -- 172.21.15.1:0/327384272 >> 172.21.15.1:0/327384272 conn(0x7f98480fd8c0 msgr2=0x7f98480ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:25.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:25.745+0000 7f984ec33640 1 -- 172.21.15.1:0/327384272 shutdown_connections 2024-09-11T21:57:25.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:25.745+0000 7f984ec33640 1 -- 172.21.15.1:0/327384272 wait complete. 2024-09-11T21:57:25.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:25.746+0000 7f984ec33640 1 Processor -- start 2024-09-11T21:57:25.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:25.746+0000 7f984ec33640 1 -- start start 2024-09-11T21:57:25.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:25.746+0000 7f984ec33640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9848101de0 0x7f9848198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:25.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:25.746+0000 7f984ec33640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9848102790 0x7f98481989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:25.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:25.746+0000 7f984ec33640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9848106f10 0x7f984819cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:25.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:25.746+0000 7f984ec33640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9848111f80 con 0x7f9848102790 2024-09-11T21:57:25.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:25.746+0000 7f984ec33640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9848111e00 con 0x7f9848101de0 2024-09-11T21:57:25.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:25.746+0000 7f984ec33640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9848112100 con 0x7f9848106f10 2024-09-11T21:57:25.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:25.746+0000 7f9847fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9848102790 0x7f98481989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:25.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:25.746+0000 7f9847fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9848102790 0x7f98481989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38922/0 (socket says 172.21.15.1:38922) 2024-09-11T21:57:25.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:25.746+0000 7f9847fff640 1 -- 172.21.15.1:0/3801246155 learned_addr learned my addr 172.21.15.1:0/3801246155 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:25.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:25.746+0000 7f984c9cf640 1 --2- 172.21.15.1:0/3801246155 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9848101de0 0x7f9848198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:25.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:25.746+0000 7f984d1d0640 1 --2- 172.21.15.1:0/3801246155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9848106f10 0x7f984819cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:25.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:25.747+0000 7f9847fff640 1 -- 172.21.15.1:0/3801246155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9848106f10 msgr2=0x7f984819cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:25.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:25.747+0000 7f9847fff640 1 --2- 172.21.15.1:0/3801246155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9848106f10 0x7f984819cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:25.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:25.747+0000 7f9847fff640 1 -- 172.21.15.1:0/3801246155 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9848101de0 msgr2=0x7f9848198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:25.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:25.747+0000 7f9847fff640 1 --2- 172.21.15.1:0/3801246155 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9848101de0 0x7f9848198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:25.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:25.747+0000 7f9847fff640 1 -- 172.21.15.1:0/3801246155 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f984819d410 con 0x7f9848102790 2024-09-11T21:57:25.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:25.747+0000 7f9847fff640 1 --2- 172.21.15.1:0/3801246155 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9848102790 0x7f98481989a0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f983c00d950 tx=0x7f983c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:25.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:25.747+0000 7f9845ffb640 1 -- 172.21.15.1:0/3801246155 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983c017070 con 0x7f9848102790 2024-09-11T21:57:25.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:25.747+0000 7f984ec33640 1 -- 172.21.15.1:0/3801246155 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f984819d700 con 0x7f9848102790 2024-09-11T21:57:25.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:25.747+0000 7f9845ffb640 1 -- 172.21.15.1:0/3801246155 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f983c009c30 con 0x7f9848102790 2024-09-11T21:57:25.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:25.747+0000 7f9845ffb640 1 -- 172.21.15.1:0/3801246155 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983c00f6b0 con 0x7f9848102790 2024-09-11T21:57:25.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:25.747+0000 7f984ec33640 1 -- 172.21.15.1:0/3801246155 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98481a4fe0 con 0x7f9848102790 2024-09-11T21:57:25.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:25.748+0000 7f984ec33640 1 -- 172.21.15.1:0/3801246155 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98481021c0 con 0x7f9848102790 2024-09-11T21:57:25.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:25.749+0000 7f9845ffb640 1 -- 172.21.15.1:0/3801246155 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 24) ==== 138008+0+0 (secure 0 0 0) 0x7f983c0109d0 con 0x7f9848102790 2024-09-11T21:57:25.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:25.750+0000 7f9845ffb640 1 --2- 172.21.15.1:0/3801246155 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f98300a2d80 0x7f98300a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:25.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:25.750+0000 7f984c9cf640 1 --2- 172.21.15.1:0/3801246155 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f98300a2d80 0x7f98300a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:25.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:25.750+0000 7f9845ffb640 1 -- 172.21.15.1:0/3801246155 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7f983c0cf1d0 con 0x7f9848102790 2024-09-11T21:57:25.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:25.751+0000 7f984c9cf640 1 --2- 172.21.15.1:0/3801246155 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f98300a2d80 0x7f98300a5240 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9838000c00 tx=0x7f9838005b50 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:25.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:25.755+0000 7f9845ffb640 1 -- 172.21.15.1:0/3801246155 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f983c009de0 con 0x7f9848102790 2024-09-11T21:57:25.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:25.925+0000 7f984ec33640 1 -- 172.21.15.1:0/3801246155 --> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9848062630 con 0x7f98300a2d80 2024-09-11T21:57:25.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:25.932+0000 7f9845ffb640 1 -- 172.21.15.1:0/3801246155 <== mgr.15204 v2:172.21.15.1:6800/3543150772 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258106 (secure 0 0 0) 0x7f9848062630 con 0x7f98300a2d80 2024-09-11T21:57:25.934 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:57:25.935 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-11T21:57:25.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:25.935+0000 7f984ec33640 1 -- 172.21.15.1:0/3801246155 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f98300a2d80 msgr2=0x7f98300a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:25.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:25.935+0000 7f984ec33640 1 --2- 172.21.15.1:0/3801246155 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f98300a2d80 0x7f98300a5240 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9838000c00 tx=0x7f9838005b50 comp rx=0 tx=0).stop 2024-09-11T21:57:25.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:25.935+0000 7f984ec33640 1 -- 172.21.15.1:0/3801246155 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9848102790 msgr2=0x7f98481989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:25.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:25.935+0000 7f984ec33640 1 --2- 172.21.15.1:0/3801246155 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9848102790 0x7f98481989a0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f983c00d950 tx=0x7f983c00de10 comp rx=0 tx=0).stop 2024-09-11T21:57:25.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:25.935+0000 7f984ec33640 1 -- 172.21.15.1:0/3801246155 shutdown_connections 2024-09-11T21:57:25.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:25.935+0000 7f984ec33640 1 --2- 172.21.15.1:0/3801246155 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f98300a2d80 0x7f98300a5240 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:25.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:25.935+0000 7f984ec33640 1 --2- 172.21.15.1:0/3801246155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9848106f10 0x7f984819cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:25.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:25.935+0000 7f984ec33640 1 --2- 172.21.15.1:0/3801246155 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9848102790 0x7f98481989a0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:25.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:25.935+0000 7f984ec33640 1 --2- 172.21.15.1:0/3801246155 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9848101de0 0x7f9848198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:25.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:25.935+0000 7f984ec33640 1 -- 172.21.15.1:0/3801246155 >> 172.21.15.1:0/3801246155 conn(0x7f98480fd8c0 msgr2=0x7f98480fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:25.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:25.935+0000 7f984ec33640 1 -- 172.21.15.1:0/3801246155 shutdown_connections 2024-09-11T21:57:25.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:25.935+0000 7f984ec33640 1 -- 172.21.15.1:0/3801246155 wait complete. 2024-09-11T21:57:26.597 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":9,"stamp":"2024-09-11T21:57:25.684928+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1389412,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1759,"num_read_kb":1315,"num_write":710,"num_write_kb":5326,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1239947,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1036,"ondisk_log_size":1036,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":390308,"kb_used_data":11896,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124551516,"statfs":{"total":1151940427776,"available":1151540752384,"internally_reserved":0,"allocated":12181504,"data_stored":9698016,"data_compressed":325147,"data_compressed_allocated":430080,"data_compressed_original":4222976,"omap_allocated":19142,"internal_metadata":387233082},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":197770,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":457,"num_read_kb":495,"num_write":147,"num_write_kb":916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.992354"},"pg_stats":[{"pgid":"3.3f","version":"86'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788356+0000","last_change":"2024-09-11T21:55:20.095385+0000","last_active":"2024-09-11T21:57:13.788356+0000","last_peered":"2024-09-11T21:57:13.788356+0000","last_clean":"2024-09-11T21:57:13.788356+0000","last_became_active":"2024-09-11T21:55:20.095077+0000","last_became_peered":"2024-09-11T21:55:20.095077+0000","last_unstale":"2024-09-11T21:57:13.788356+0000","last_undegraded":"2024-09-11T21:57:13.788356+0000","last_fullsized":"2024-09-11T21:57:13.788356+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:53:52.231522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"86'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.669632+0000","last_change":"2024-09-11T21:55:20.103540+0000","last_active":"2024-09-11T21:57:13.669632+0000","last_peered":"2024-09-11T21:57:13.669632+0000","last_clean":"2024-09-11T21:57:13.669632+0000","last_became_active":"2024-09-11T21:55:20.103233+0000","last_became_peered":"2024-09-11T21:55:20.103233+0000","last_unstale":"2024-09-11T21:57:13.669632+0000","last_undegraded":"2024-09-11T21:57:13.669632+0000","last_fullsized":"2024-09-11T21:57:13.669632+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:51:29.222202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"86'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.669623+0000","last_change":"2024-09-11T21:55:20.098538+0000","last_active":"2024-09-11T21:57:13.669623+0000","last_peered":"2024-09-11T21:57:13.669623+0000","last_clean":"2024-09-11T21:57:13.669623+0000","last_became_active":"2024-09-11T21:55:20.098202+0000","last_became_peered":"2024-09-11T21:55:20.098202+0000","last_unstale":"2024-09-11T21:57:13.669623+0000","last_undegraded":"2024-09-11T21:57:13.669623+0000","last_fullsized":"2024-09-11T21:57:13.669623+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:33:46.636952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"86'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788540+0000","last_change":"2024-09-11T21:55:20.100869+0000","last_active":"2024-09-11T21:57:13.788540+0000","last_peered":"2024-09-11T21:57:13.788540+0000","last_clean":"2024-09-11T21:57:13.788540+0000","last_became_active":"2024-09-11T21:55:20.100573+0000","last_became_peered":"2024-09-11T21:55:20.100573+0000","last_unstale":"2024-09-11T21:57:13.788540+0000","last_undegraded":"2024-09-11T21:57:13.788540+0000","last_fullsized":"2024-09-11T21:57:13.788540+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:58:11.154105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"86'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671033+0000","last_change":"2024-09-11T21:55:20.091648+0000","last_active":"2024-09-11T21:57:13.671033+0000","last_peered":"2024-09-11T21:57:13.671033+0000","last_clean":"2024-09-11T21:57:13.671033+0000","last_became_active":"2024-09-11T21:55:20.091222+0000","last_became_peered":"2024-09-11T21:55:20.091222+0000","last_unstale":"2024-09-11T21:57:13.671033+0000","last_undegraded":"2024-09-11T21:57:13.671033+0000","last_fullsized":"2024-09-11T21:57:13.671033+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:55:03.420754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"86'6","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.764975+0000","last_change":"2024-09-11T21:56:26.376595+0000","last_active":"2024-09-11T21:57:13.764975+0000","last_peered":"2024-09-11T21:57:13.764975+0000","last_clean":"2024-09-11T21:57:13.764975+0000","last_became_active":"2024-09-11T21:56:26.375784+0000","last_became_peered":"2024-09-11T21:56:26.375784+0000","last_unstale":"2024-09-11T21:57:13.764975+0000","last_undegraded":"2024-09-11T21:57:13.764975+0000","last_fullsized":"2024-09-11T21:57:13.764975+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:19:38.371463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,7],"acting":[10,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"86'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.670787+0000","last_change":"2024-09-11T21:55:20.101084+0000","last_active":"2024-09-11T21:57:13.670787+0000","last_peered":"2024-09-11T21:57:13.670787+0000","last_clean":"2024-09-11T21:57:13.670787+0000","last_became_active":"2024-09-11T21:55:20.100739+0000","last_became_peered":"2024-09-11T21:55:20.100739+0000","last_unstale":"2024-09-11T21:57:13.670787+0000","last_undegraded":"2024-09-11T21:57:13.670787+0000","last_fullsized":"2024-09-11T21:57:13.670787+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:46:58.969559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.753160+0000","last_change":"2024-09-11T21:55:20.088715+0000","last_active":"2024-09-11T21:57:13.753160+0000","last_peered":"2024-09-11T21:57:13.753160+0000","last_clean":"2024-09-11T21:57:13.753160+0000","last_became_active":"2024-09-11T21:55:20.088280+0000","last_became_peered":"2024-09-11T21:55:20.088280+0000","last_unstale":"2024-09-11T21:57:13.753160+0000","last_undegraded":"2024-09-11T21:57:13.753160+0000","last_fullsized":"2024-09-11T21:57:13.753160+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:06:24.450444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"86'11","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788671+0000","last_change":"2024-09-11T21:55:20.101266+0000","last_active":"2024-09-11T21:57:13.788671+0000","last_peered":"2024-09-11T21:57:13.788671+0000","last_clean":"2024-09-11T21:57:13.788671+0000","last_became_active":"2024-09-11T21:55:20.100919+0000","last_became_peered":"2024-09-11T21:55:20.100919+0000","last_unstale":"2024-09-11T21:57:13.788671+0000","last_undegraded":"2024-09-11T21:57:13.788671+0000","last_fullsized":"2024-09-11T21:57:13.788671+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:47:15.466082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"86'16","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.765361+0000","last_change":"2024-09-11T21:55:20.098086+0000","last_active":"2024-09-11T21:57:13.765361+0000","last_peered":"2024-09-11T21:57:13.765361+0000","last_clean":"2024-09-11T21:57:13.765361+0000","last_became_active":"2024-09-11T21:55:20.097754+0000","last_became_peered":"2024-09-11T21:55:20.097754+0000","last_unstale":"2024-09-11T21:57:13.765361+0000","last_undegraded":"2024-09-11T21:57:13.765361+0000","last_fullsized":"2024-09-11T21:57:13.765361+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:37:00.098345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"86'6","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671036+0000","last_change":"2024-09-11T21:55:20.092671+0000","last_active":"2024-09-11T21:57:13.671036+0000","last_peered":"2024-09-11T21:57:13.671036+0000","last_clean":"2024-09-11T21:57:13.671036+0000","last_became_active":"2024-09-11T21:55:20.092112+0000","last_became_peered":"2024-09-11T21:55:20.092112+0000","last_unstale":"2024-09-11T21:57:13.671036+0000","last_undegraded":"2024-09-11T21:57:13.671036+0000","last_fullsized":"2024-09-11T21:57:13.671036+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:06:28.538838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788646+0000","last_change":"2024-09-11T21:55:20.099863+0000","last_active":"2024-09-11T21:57:13.788646+0000","last_peered":"2024-09-11T21:57:13.788646+0000","last_clean":"2024-09-11T21:57:13.788646+0000","last_became_active":"2024-09-11T21:55:20.099173+0000","last_became_peered":"2024-09-11T21:55:20.099173+0000","last_unstale":"2024-09-11T21:57:13.788646+0000","last_undegraded":"2024-09-11T21:57:13.788646+0000","last_fullsized":"2024-09-11T21:57:13.788646+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:34:56.669410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"86'10","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788507+0000","last_change":"2024-09-11T21:55:20.101212+0000","last_active":"2024-09-11T21:57:13.788507+0000","last_peered":"2024-09-11T21:57:13.788507+0000","last_clean":"2024-09-11T21:57:13.788507+0000","last_became_active":"2024-09-11T21:55:20.100876+0000","last_became_peered":"2024-09-11T21:55:20.100876+0000","last_unstale":"2024-09-11T21:57:13.788507+0000","last_undegraded":"2024-09-11T21:57:13.788507+0000","last_fullsized":"2024-09-11T21:57:13.788507+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:14:18.104009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.669316+0000","last_change":"2024-09-11T21:55:20.095394+0000","last_active":"2024-09-11T21:57:13.669316+0000","last_peered":"2024-09-11T21:57:13.669316+0000","last_clean":"2024-09-11T21:57:13.669316+0000","last_became_active":"2024-09-11T21:55:20.095101+0000","last_became_peered":"2024-09-11T21:55:20.095101+0000","last_unstale":"2024-09-11T21:57:13.669316+0000","last_undegraded":"2024-09-11T21:57:13.669316+0000","last_fullsized":"2024-09-11T21:57:13.669316+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:53:32.529359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"86'7","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.669266+0000","last_change":"2024-09-11T21:56:26.376206+0000","last_active":"2024-09-11T21:57:13.669266+0000","last_peered":"2024-09-11T21:57:13.669266+0000","last_clean":"2024-09-11T21:57:13.669266+0000","last_became_active":"2024-09-11T21:56:26.375829+0000","last_became_peered":"2024-09-11T21:56:26.375829+0000","last_unstale":"2024-09-11T21:57:13.669266+0000","last_undegraded":"2024-09-11T21:57:13.669266+0000","last_fullsized":"2024-09-11T21:57:13.669266+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:21:59.484414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.787934+0000","last_change":"2024-09-11T21:55:20.093019+0000","last_active":"2024-09-11T21:57:13.787934+0000","last_peered":"2024-09-11T21:57:13.787934+0000","last_clean":"2024-09-11T21:57:13.787934+0000","last_became_active":"2024-09-11T21:55:20.092691+0000","last_became_peered":"2024-09-11T21:55:20.092691+0000","last_unstale":"2024-09-11T21:57:13.787934+0000","last_undegraded":"2024-09-11T21:57:13.787934+0000","last_fullsized":"2024-09-11T21:57:13.787934+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:04:36.684992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788379+0000","last_change":"2024-09-11T21:55:20.097333+0000","last_active":"2024-09-11T21:57:13.788379+0000","last_peered":"2024-09-11T21:57:13.788379+0000","last_clean":"2024-09-11T21:57:13.788379+0000","last_became_active":"2024-09-11T21:55:20.096752+0000","last_became_peered":"2024-09-11T21:55:20.096752+0000","last_unstale":"2024-09-11T21:57:13.788379+0000","last_undegraded":"2024-09-11T21:57:13.788379+0000","last_fullsized":"2024-09-11T21:57:13.788379+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:27:20.722174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"86'10","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.751322+0000","last_change":"2024-09-11T21:55:20.093972+0000","last_active":"2024-09-11T21:57:13.751322+0000","last_peered":"2024-09-11T21:57:13.751322+0000","last_clean":"2024-09-11T21:57:13.751322+0000","last_became_active":"2024-09-11T21:55:20.093633+0000","last_became_peered":"2024-09-11T21:55:20.093633+0000","last_unstale":"2024-09-11T21:57:13.751322+0000","last_undegraded":"2024-09-11T21:57:13.751322+0000","last_fullsized":"2024-09-11T21:57:13.751322+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:25:18.973332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671334+0000","last_change":"2024-09-11T21:55:20.094239+0000","last_active":"2024-09-11T21:57:13.671334+0000","last_peered":"2024-09-11T21:57:13.671334+0000","last_clean":"2024-09-11T21:57:13.671334+0000","last_became_active":"2024-09-11T21:55:20.093924+0000","last_became_peered":"2024-09-11T21:55:20.093924+0000","last_unstale":"2024-09-11T21:57:13.671334+0000","last_undegraded":"2024-09-11T21:57:13.671334+0000","last_fullsized":"2024-09-11T21:57:13.671334+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:39:53.012780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788075+0000","last_change":"2024-09-11T21:55:20.099335+0000","last_active":"2024-09-11T21:57:13.788075+0000","last_peered":"2024-09-11T21:57:13.788075+0000","last_clean":"2024-09-11T21:57:13.788075+0000","last_became_active":"2024-09-11T21:55:20.098929+0000","last_became_peered":"2024-09-11T21:55:20.098929+0000","last_unstale":"2024-09-11T21:57:13.788075+0000","last_undegraded":"2024-09-11T21:57:13.788075+0000","last_fullsized":"2024-09-11T21:57:13.788075+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:38:43.117829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"86'11","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788526+0000","last_change":"2024-09-11T21:55:20.094068+0000","last_active":"2024-09-11T21:57:13.788526+0000","last_peered":"2024-09-11T21:57:13.788526+0000","last_clean":"2024-09-11T21:57:13.788526+0000","last_became_active":"2024-09-11T21:55:20.093637+0000","last_became_peered":"2024-09-11T21:55:20.093637+0000","last_unstale":"2024-09-11T21:57:13.788526+0000","last_undegraded":"2024-09-11T21:57:13.788526+0000","last_fullsized":"2024-09-11T21:57:13.788526+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:52:59.289715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"86'10","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788122+0000","last_change":"2024-09-11T21:55:20.090778+0000","last_active":"2024-09-11T21:57:13.788122+0000","last_peered":"2024-09-11T21:57:13.788122+0000","last_clean":"2024-09-11T21:57:13.788122+0000","last_became_active":"2024-09-11T21:55:20.090443+0000","last_became_peered":"2024-09-11T21:55:20.090443+0000","last_unstale":"2024-09-11T21:57:13.788122+0000","last_undegraded":"2024-09-11T21:57:13.788122+0000","last_fullsized":"2024-09-11T21:57:13.788122+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:46:21.488228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.753203+0000","last_change":"2024-09-11T21:55:20.092238+0000","last_active":"2024-09-11T21:57:13.753203+0000","last_peered":"2024-09-11T21:57:13.753203+0000","last_clean":"2024-09-11T21:57:13.753203+0000","last_became_active":"2024-09-11T21:55:20.091961+0000","last_became_peered":"2024-09-11T21:55:20.091961+0000","last_unstale":"2024-09-11T21:57:13.753203+0000","last_undegraded":"2024-09-11T21:57:13.753203+0000","last_fullsized":"2024-09-11T21:57:13.753203+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:54:40.445441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"86'6","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.761984+0000","last_change":"2024-09-11T21:55:20.102944+0000","last_active":"2024-09-11T21:57:13.761984+0000","last_peered":"2024-09-11T21:57:13.761984+0000","last_clean":"2024-09-11T21:57:13.761984+0000","last_became_active":"2024-09-11T21:55:20.102533+0000","last_became_peered":"2024-09-11T21:55:20.102533+0000","last_unstale":"2024-09-11T21:57:13.761984+0000","last_undegraded":"2024-09-11T21:57:13.761984+0000","last_fullsized":"2024-09-11T21:57:13.761984+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:12:15.832622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"86'2","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.753729+0000","last_change":"2024-09-11T21:56:26.292272+0000","last_active":"2024-09-11T21:57:13.753729+0000","last_peered":"2024-09-11T21:57:13.753729+0000","last_clean":"2024-09-11T21:57:13.753729+0000","last_became_active":"2024-09-11T21:56:26.291920+0000","last_became_peered":"2024-09-11T21:56:26.291920+0000","last_unstale":"2024-09-11T21:57:13.753729+0000","last_undegraded":"2024-09-11T21:57:13.753729+0000","last_fullsized":"2024-09-11T21:57:13.753729+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:13:13.579907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"86'15","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671489+0000","last_change":"2024-09-11T21:56:26.376364+0000","last_active":"2024-09-11T21:57:13.671489+0000","last_peered":"2024-09-11T21:57:13.671489+0000","last_clean":"2024-09-11T21:57:13.671489+0000","last_became_active":"2024-09-11T21:56:26.375927+0000","last_became_peered":"2024-09-11T21:56:26.375927+0000","last_unstale":"2024-09-11T21:57:13.671489+0000","last_undegraded":"2024-09-11T21:57:13.671489+0000","last_fullsized":"2024-09-11T21:57:13.671489+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:23:03.664207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,9],"acting":[2,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"86'6","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788153+0000","last_change":"2024-09-11T21:56:26.376702+0000","last_active":"2024-09-11T21:57:13.788153+0000","last_peered":"2024-09-11T21:57:13.788153+0000","last_clean":"2024-09-11T21:57:13.788153+0000","last_became_active":"2024-09-11T21:56:26.376179+0000","last_became_peered":"2024-09-11T21:56:26.376179+0000","last_unstale":"2024-09-11T21:57:13.788153+0000","last_undegraded":"2024-09-11T21:57:13.788153+0000","last_fullsized":"2024-09-11T21:57:13.788153+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:27:34.666962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"86'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788960+0000","last_change":"2024-09-11T21:55:20.093898+0000","last_active":"2024-09-11T21:57:13.788960+0000","last_peered":"2024-09-11T21:57:13.788960+0000","last_clean":"2024-09-11T21:57:13.788960+0000","last_became_active":"2024-09-11T21:55:20.093348+0000","last_became_peered":"2024-09-11T21:55:20.093348+0000","last_unstale":"2024-09-11T21:57:13.788960+0000","last_undegraded":"2024-09-11T21:57:13.788960+0000","last_fullsized":"2024-09-11T21:57:13.788960+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:00:06.185478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"86'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.669510+0000","last_change":"2024-09-11T21:55:20.100575+0000","last_active":"2024-09-11T21:57:13.669510+0000","last_peered":"2024-09-11T21:57:13.669510+0000","last_clean":"2024-09-11T21:57:13.669510+0000","last_became_active":"2024-09-11T21:55:20.100255+0000","last_became_peered":"2024-09-11T21:55:20.100255+0000","last_unstale":"2024-09-11T21:57:13.669510+0000","last_undegraded":"2024-09-11T21:57:13.669510+0000","last_fullsized":"2024-09-11T21:57:13.669510+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:48:57.718298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"86'8","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.670780+0000","last_change":"2024-09-11T21:56:26.377183+0000","last_active":"2024-09-11T21:57:13.670780+0000","last_peered":"2024-09-11T21:57:13.670780+0000","last_clean":"2024-09-11T21:57:13.670780+0000","last_became_active":"2024-09-11T21:56:26.375906+0000","last_became_peered":"2024-09-11T21:56:26.375906+0000","last_unstale":"2024-09-11T21:57:13.670780+0000","last_undegraded":"2024-09-11T21:57:13.670780+0000","last_fullsized":"2024-09-11T21:57:13.670780+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:00:29.033961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"86'12","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.670836+0000","last_change":"2024-09-11T21:55:20.093822+0000","last_active":"2024-09-11T21:57:13.670836+0000","last_peered":"2024-09-11T21:57:13.670836+0000","last_clean":"2024-09-11T21:57:13.670836+0000","last_became_active":"2024-09-11T21:55:20.093361+0000","last_became_peered":"2024-09-11T21:55:20.093361+0000","last_unstale":"2024-09-11T21:57:13.670836+0000","last_undegraded":"2024-09-11T21:57:13.670836+0000","last_fullsized":"2024-09-11T21:57:13.670836+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:37:06.362732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"86'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788121+0000","last_change":"2024-09-11T21:55:20.103990+0000","last_active":"2024-09-11T21:57:13.788121+0000","last_peered":"2024-09-11T21:57:13.788121+0000","last_clean":"2024-09-11T21:57:13.788121+0000","last_became_active":"2024-09-11T21:55:20.103375+0000","last_became_peered":"2024-09-11T21:55:20.103375+0000","last_unstale":"2024-09-11T21:57:13.788121+0000","last_undegraded":"2024-09-11T21:57:13.788121+0000","last_fullsized":"2024-09-11T21:57:13.788121+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:02:06.786888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"86'7","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671024+0000","last_change":"2024-09-11T21:56:26.376261+0000","last_active":"2024-09-11T21:57:13.671024+0000","last_peered":"2024-09-11T21:57:13.671024+0000","last_clean":"2024-09-11T21:57:13.671024+0000","last_became_active":"2024-09-11T21:56:26.375863+0000","last_became_peered":"2024-09-11T21:56:26.375863+0000","last_unstale":"2024-09-11T21:57:13.671024+0000","last_undegraded":"2024-09-11T21:57:13.671024+0000","last_fullsized":"2024-09-11T21:57:13.671024+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:58:21.359836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"87'8","reported_seq":150,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:23.310305+0000","last_change":"2024-09-11T21:55:18.026947+0000","last_active":"2024-09-11T21:57:23.310305+0000","last_peered":"2024-09-11T21:57:23.310305+0000","last_clean":"2024-09-11T21:57:23.310305+0000","last_became_active":"2024-09-11T21:55:18.026632+0000","last_became_peered":"2024-09-11T21:55:18.026632+0000","last_unstale":"2024-09-11T21:57:23.310305+0000","last_undegraded":"2024-09-11T21:57:23.310305+0000","last_fullsized":"2024-09-11T21:57:23.310305+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:55:16.618074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1549,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.762157+0000","last_change":"2024-09-11T21:55:20.100197+0000","last_active":"2024-09-11T21:57:13.762157+0000","last_peered":"2024-09-11T21:57:13.762157+0000","last_clean":"2024-09-11T21:57:13.762157+0000","last_became_active":"2024-09-11T21:55:20.098494+0000","last_became_peered":"2024-09-11T21:55:20.098494+0000","last_unstale":"2024-09-11T21:57:13.762157+0000","last_undegraded":"2024-09-11T21:57:13.762157+0000","last_fullsized":"2024-09-11T21:57:13.762157+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:14.505314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"84'7","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.765023+0000","last_change":"2024-09-11T21:55:18.024359+0000","last_active":"2024-09-11T21:57:13.765023+0000","last_peered":"2024-09-11T21:57:13.765023+0000","last_clean":"2024-09-11T21:57:13.765023+0000","last_became_active":"2024-09-11T21:55:18.023624+0000","last_became_peered":"2024-09-11T21:55:18.023624+0000","last_unstale":"2024-09-11T21:57:13.765023+0000","last_undegraded":"2024-09-11T21:57:13.765023+0000","last_fullsized":"2024-09-11T21:57:13.765023+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:13:38.582232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"84'3","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788356+0000","last_change":"2024-09-11T21:55:18.032541+0000","last_active":"2024-09-11T21:57:13.788356+0000","last_peered":"2024-09-11T21:57:13.788356+0000","last_clean":"2024-09-11T21:57:13.788356+0000","last_became_active":"2024-09-11T21:55:18.032218+0000","last_became_peered":"2024-09-11T21:55:18.032218+0000","last_unstale":"2024-09-11T21:57:13.788356+0000","last_undegraded":"2024-09-11T21:57:13.788356+0000","last_fullsized":"2024-09-11T21:57:13.788356+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:38:20.872069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.753229+0000","last_change":"2024-09-11T21:55:20.090030+0000","last_active":"2024-09-11T21:57:13.753229+0000","last_peered":"2024-09-11T21:57:13.753229+0000","last_clean":"2024-09-11T21:57:13.753229+0000","last_became_active":"2024-09-11T21:55:20.089481+0000","last_became_peered":"2024-09-11T21:55:20.089481+0000","last_unstale":"2024-09-11T21:57:13.753229+0000","last_undegraded":"2024-09-11T21:57:13.753229+0000","last_fullsized":"2024-09-11T21:57:13.753229+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:41:57.038519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"84'3","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.762394+0000","last_change":"2024-09-11T21:55:18.026656+0000","last_active":"2024-09-11T21:57:13.762394+0000","last_peered":"2024-09-11T21:57:13.762394+0000","last_clean":"2024-09-11T21:57:13.762394+0000","last_became_active":"2024-09-11T21:55:18.026318+0000","last_became_peered":"2024-09-11T21:55:18.026318+0000","last_unstale":"2024-09-11T21:57:13.762394+0000","last_undegraded":"2024-09-11T21:57:13.762394+0000","last_fullsized":"2024-09-11T21:57:13.762394+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:14:41.438334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"86'11","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.750970+0000","last_change":"2024-09-11T21:55:20.090790+0000","last_active":"2024-09-11T21:57:13.750970+0000","last_peered":"2024-09-11T21:57:13.750970+0000","last_clean":"2024-09-11T21:57:13.750970+0000","last_became_active":"2024-09-11T21:55:20.090468+0000","last_became_peered":"2024-09-11T21:55:20.090468+0000","last_unstale":"2024-09-11T21:57:13.750970+0000","last_undegraded":"2024-09-11T21:57:13.750970+0000","last_fullsized":"2024-09-11T21:57:13.750970+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:19.115482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'8","reported_seq":258,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:24.311247+0000","last_change":"2024-09-11T21:55:18.026989+0000","last_active":"2024-09-11T21:57:24.311247+0000","last_peered":"2024-09-11T21:57:24.311247+0000","last_clean":"2024-09-11T21:57:24.311247+0000","last_became_active":"2024-09-11T21:55:18.026229+0000","last_became_peered":"2024-09-11T21:55:18.026229+0000","last_unstale":"2024-09-11T21:57:24.311247+0000","last_undegraded":"2024-09-11T21:57:24.311247+0000","last_fullsized":"2024-09-11T21:57:24.311247+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:50:35.745789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":226,"num_read_kb":226,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"86'10","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788388+0000","last_change":"2024-09-11T21:55:20.095884+0000","last_active":"2024-09-11T21:57:13.788388+0000","last_peered":"2024-09-11T21:57:13.788388+0000","last_clean":"2024-09-11T21:57:13.788388+0000","last_became_active":"2024-09-11T21:55:20.094887+0000","last_became_peered":"2024-09-11T21:55:20.094887+0000","last_unstale":"2024-09-11T21:57:13.788388+0000","last_undegraded":"2024-09-11T21:57:13.788388+0000","last_fullsized":"2024-09-11T21:57:13.788388+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:45:03.579703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"84'4","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671167+0000","last_change":"2024-09-11T21:55:18.030474+0000","last_active":"2024-09-11T21:57:13.671167+0000","last_peered":"2024-09-11T21:57:13.671167+0000","last_clean":"2024-09-11T21:57:13.671167+0000","last_became_active":"2024-09-11T21:55:18.030104+0000","last_became_peered":"2024-09-11T21:55:18.030104+0000","last_unstale":"2024-09-11T21:57:13.671167+0000","last_undegraded":"2024-09-11T21:57:13.671167+0000","last_fullsized":"2024-09-11T21:57:13.671167+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:21:00.484595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.669795+0000","last_change":"2024-09-11T21:55:20.102807+0000","last_active":"2024-09-11T21:57:13.669795+0000","last_peered":"2024-09-11T21:57:13.669795+0000","last_clean":"2024-09-11T21:57:13.669795+0000","last_became_active":"2024-09-11T21:55:20.102581+0000","last_became_peered":"2024-09-11T21:55:20.102581+0000","last_unstale":"2024-09-11T21:57:13.669795+0000","last_undegraded":"2024-09-11T21:57:13.669795+0000","last_fullsized":"2024-09-11T21:57:13.669795+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:29:23.139788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"84'5","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.762200+0000","last_change":"2024-09-11T21:55:18.117685+0000","last_active":"2024-09-11T21:57:13.762200+0000","last_peered":"2024-09-11T21:57:13.762200+0000","last_clean":"2024-09-11T21:57:13.762200+0000","last_became_active":"2024-09-11T21:55:18.117261+0000","last_became_peered":"2024-09-11T21:55:18.117261+0000","last_unstale":"2024-09-11T21:57:13.762200+0000","last_undegraded":"2024-09-11T21:57:13.762200+0000","last_fullsized":"2024-09-11T21:57:13.762200+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:42:13.180725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"86'11","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788081+0000","last_change":"2024-09-11T21:55:20.086562+0000","last_active":"2024-09-11T21:57:13.788081+0000","last_peered":"2024-09-11T21:57:13.788081+0000","last_clean":"2024-09-11T21:57:13.788081+0000","last_became_active":"2024-09-11T21:55:20.086197+0000","last_became_peered":"2024-09-11T21:55:20.086197+0000","last_unstale":"2024-09-11T21:57:13.788081+0000","last_undegraded":"2024-09-11T21:57:13.788081+0000","last_fullsized":"2024-09-11T21:57:13.788081+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:23:44.705132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'36","reported_seq":319,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:24.370948+0000","last_change":"2024-09-11T21:56:26.299386+0000","last_active":"2024-09-11T21:57:24.370948+0000","last_peered":"2024-09-11T21:57:24.370948+0000","last_clean":"2024-09-11T21:57:24.370948+0000","last_became_active":"2024-09-11T21:56:26.291951+0000","last_became_peered":"2024-09-11T21:56:26.291951+0000","last_unstale":"2024-09-11T21:57:24.370948+0000","last_undegraded":"2024-09-11T21:57:24.370948+0000","last_fullsized":"2024-09-11T21:57:24.370948+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:54:47.346798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67065,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":231,"num_read_kb":182,"num_write":38,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":59211,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"86'6","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671198+0000","last_change":"2024-09-11T21:55:20.091695+0000","last_active":"2024-09-11T21:57:13.671198+0000","last_peered":"2024-09-11T21:57:13.671198+0000","last_clean":"2024-09-11T21:57:13.671198+0000","last_became_active":"2024-09-11T21:55:20.091128+0000","last_became_peered":"2024-09-11T21:55:20.091128+0000","last_unstale":"2024-09-11T21:57:13.671198+0000","last_undegraded":"2024-09-11T21:57:13.671198+0000","last_fullsized":"2024-09-11T21:57:13.671198+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:16:20.751952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"84'9","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671024+0000","last_change":"2024-09-11T21:55:18.030827+0000","last_active":"2024-09-11T21:57:13.671024+0000","last_peered":"2024-09-11T21:57:13.671024+0000","last_clean":"2024-09-11T21:57:13.671024+0000","last_became_active":"2024-09-11T21:55:18.030354+0000","last_became_peered":"2024-09-11T21:55:18.030354+0000","last_unstale":"2024-09-11T21:57:13.671024+0000","last_undegraded":"2024-09-11T21:57:13.671024+0000","last_fullsized":"2024-09-11T21:57:13.671024+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:23:19.578820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788550+0000","last_change":"2024-09-11T21:55:20.090160+0000","last_active":"2024-09-11T21:57:13.788550+0000","last_peered":"2024-09-11T21:57:13.788550+0000","last_clean":"2024-09-11T21:57:13.788550+0000","last_became_active":"2024-09-11T21:55:20.089801+0000","last_became_peered":"2024-09-11T21:55:20.089801+0000","last_unstale":"2024-09-11T21:57:13.788550+0000","last_undegraded":"2024-09-11T21:57:13.788550+0000","last_fullsized":"2024-09-11T21:57:13.788550+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:34:16.093123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"84'5","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.761998+0000","last_change":"2024-09-11T21:55:18.026817+0000","last_active":"2024-09-11T21:57:13.761998+0000","last_peered":"2024-09-11T21:57:13.761998+0000","last_clean":"2024-09-11T21:57:13.761998+0000","last_became_active":"2024-09-11T21:55:18.026188+0000","last_became_peered":"2024-09-11T21:55:18.026188+0000","last_unstale":"2024-09-11T21:57:13.761998+0000","last_undegraded":"2024-09-11T21:57:13.761998+0000","last_fullsized":"2024-09-11T21:57:13.761998+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:55:14.774503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"86'15","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.670991+0000","last_change":"2024-09-11T21:55:20.095613+0000","last_active":"2024-09-11T21:57:13.670991+0000","last_peered":"2024-09-11T21:57:13.670991+0000","last_clean":"2024-09-11T21:57:13.670991+0000","last_became_active":"2024-09-11T21:55:20.095302+0000","last_became_peered":"2024-09-11T21:55:20.095302+0000","last_unstale":"2024-09-11T21:57:13.670991+0000","last_undegraded":"2024-09-11T21:57:13.670991+0000","last_fullsized":"2024-09-11T21:57:13.670991+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:37:27.961213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'7","reported_seq":148,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:23.311297+0000","last_change":"2024-09-11T21:55:18.032417+0000","last_active":"2024-09-11T21:57:23.311297+0000","last_peered":"2024-09-11T21:57:23.311297+0000","last_clean":"2024-09-11T21:57:23.311297+0000","last_became_active":"2024-09-11T21:55:18.031419+0000","last_became_peered":"2024-09-11T21:55:18.031419+0000","last_unstale":"2024-09-11T21:57:23.311297+0000","last_undegraded":"2024-09-11T21:57:23.311297+0000","last_fullsized":"2024-09-11T21:57:23.311297+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:30:52.285678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1289,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":118,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"86'6","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.670730+0000","last_change":"2024-09-11T21:55:20.101965+0000","last_active":"2024-09-11T21:57:13.670730+0000","last_peered":"2024-09-11T21:57:13.670730+0000","last_clean":"2024-09-11T21:57:13.670730+0000","last_became_active":"2024-09-11T21:55:20.101534+0000","last_became_peered":"2024-09-11T21:55:20.101534+0000","last_unstale":"2024-09-11T21:57:13.670730+0000","last_undegraded":"2024-09-11T21:57:13.670730+0000","last_fullsized":"2024-09-11T21:57:13.670730+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:41:30.685121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"84'17","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.670811+0000","last_change":"2024-09-11T21:55:18.116059+0000","last_active":"2024-09-11T21:57:13.670811+0000","last_peered":"2024-09-11T21:57:13.670811+0000","last_clean":"2024-09-11T21:57:13.670811+0000","last_became_active":"2024-09-11T21:55:18.115621+0000","last_became_peered":"2024-09-11T21:55:18.115621+0000","last_unstale":"2024-09-11T21:57:13.670811+0000","last_undegraded":"2024-09-11T21:57:13.670811+0000","last_fullsized":"2024-09-11T21:57:13.670811+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:47:17.815695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"86'6","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.762037+0000","last_change":"2024-09-11T21:55:20.100903+0000","last_active":"2024-09-11T21:57:13.762037+0000","last_peered":"2024-09-11T21:57:13.762037+0000","last_clean":"2024-09-11T21:57:13.762037+0000","last_became_active":"2024-09-11T21:55:20.100561+0000","last_became_peered":"2024-09-11T21:55:20.100561+0000","last_unstale":"2024-09-11T21:57:13.762037+0000","last_undegraded":"2024-09-11T21:57:13.762037+0000","last_fullsized":"2024-09-11T21:57:13.762037+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:00:45.472238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'2","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671632+0000","last_change":"2024-09-11T21:55:18.029549+0000","last_active":"2024-09-11T21:57:13.671632+0000","last_peered":"2024-09-11T21:57:13.671632+0000","last_clean":"2024-09-11T21:57:13.671632+0000","last_became_active":"2024-09-11T21:55:18.029178+0000","last_became_peered":"2024-09-11T21:55:18.029178+0000","last_unstale":"2024-09-11T21:57:13.671632+0000","last_undegraded":"2024-09-11T21:57:13.671632+0000","last_fullsized":"2024-09-11T21:57:13.671632+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:37:07.915786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"86'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788144+0000","last_change":"2024-09-11T21:55:20.104202+0000","last_active":"2024-09-11T21:57:13.788144+0000","last_peered":"2024-09-11T21:57:13.788144+0000","last_clean":"2024-09-11T21:57:13.788144+0000","last_became_active":"2024-09-11T21:55:20.103464+0000","last_became_peered":"2024-09-11T21:55:20.103464+0000","last_unstale":"2024-09-11T21:57:13.788144+0000","last_undegraded":"2024-09-11T21:57:13.788144+0000","last_fullsized":"2024-09-11T21:57:13.788144+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:30:16.044143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"84'5","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788728+0000","last_change":"2024-09-11T21:55:18.026183+0000","last_active":"2024-09-11T21:57:13.788728+0000","last_peered":"2024-09-11T21:57:13.788728+0000","last_clean":"2024-09-11T21:57:13.788728+0000","last_became_active":"2024-09-11T21:55:18.025849+0000","last_became_peered":"2024-09-11T21:55:18.025849+0000","last_unstale":"2024-09-11T21:57:13.788728+0000","last_undegraded":"2024-09-11T21:57:13.788728+0000","last_fullsized":"2024-09-11T21:57:13.788728+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:21:12.189158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671103+0000","last_change":"2024-09-11T21:55:20.090652+0000","last_active":"2024-09-11T21:57:13.671103+0000","last_peered":"2024-09-11T21:57:13.671103+0000","last_clean":"2024-09-11T21:57:13.671103+0000","last_became_active":"2024-09-11T21:55:20.090348+0000","last_became_peered":"2024-09-11T21:55:20.090348+0000","last_unstale":"2024-09-11T21:57:13.671103+0000","last_undegraded":"2024-09-11T21:57:13.671103+0000","last_fullsized":"2024-09-11T21:57:13.671103+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:10:58.733371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'67","reported_seq":122,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:22.806643+0000","last_change":"2024-09-11T21:55:18.024388+0000","last_active":"2024-09-11T21:57:22.806643+0000","last_peered":"2024-09-11T21:57:22.806643+0000","last_clean":"2024-09-11T21:57:22.806643+0000","last_became_active":"2024-09-11T21:55:18.024100+0000","last_became_peered":"2024-09-11T21:55:18.024100+0000","last_unstale":"2024-09-11T21:57:22.806643+0000","last_undegraded":"2024-09-11T21:57:22.806643+0000","last_fullsized":"2024-09-11T21:57:22.806643+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:21:11.152282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.670936+0000","last_change":"2024-09-11T21:55:20.101007+0000","last_active":"2024-09-11T21:57:13.670936+0000","last_peered":"2024-09-11T21:57:13.670936+0000","last_clean":"2024-09-11T21:57:13.670936+0000","last_became_active":"2024-09-11T21:55:20.100642+0000","last_became_peered":"2024-09-11T21:55:20.100642+0000","last_unstale":"2024-09-11T21:57:13.670936+0000","last_undegraded":"2024-09-11T21:57:13.670936+0000","last_fullsized":"2024-09-11T21:57:13.670936+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:16:05.642184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"87'8","reported_seq":150,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:23.339239+0000","last_change":"2024-09-11T21:55:18.116216+0000","last_active":"2024-09-11T21:57:23.339239+0000","last_peered":"2024-09-11T21:57:23.339239+0000","last_clean":"2024-09-11T21:57:23.339239+0000","last_became_active":"2024-09-11T21:55:18.115871+0000","last_became_peered":"2024-09-11T21:55:18.115871+0000","last_unstale":"2024-09-11T21:57:23.339239+0000","last_undegraded":"2024-09-11T21:57:23.339239+0000","last_fullsized":"2024-09-11T21:57:23.339239+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:16:56.027636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":118,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671175+0000","last_change":"2024-09-11T21:55:20.092421+0000","last_active":"2024-09-11T21:57:13.671175+0000","last_peered":"2024-09-11T21:57:13.671175+0000","last_clean":"2024-09-11T21:57:13.671175+0000","last_became_active":"2024-09-11T21:55:20.091707+0000","last_became_peered":"2024-09-11T21:55:20.091707+0000","last_unstale":"2024-09-11T21:57:13.671175+0000","last_undegraded":"2024-09-11T21:57:13.671175+0000","last_fullsized":"2024-09-11T21:57:13.671175+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:12:38.157687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"87'148","reported_seq":397,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:15.768029+0000","last_change":"2024-09-11T21:54:07.542362+0000","last_active":"2024-09-11T21:57:15.768029+0000","last_peered":"2024-09-11T21:57:15.768029+0000","last_clean":"2024-09-11T21:57:15.768029+0000","last_became_active":"2024-09-11T21:54:07.541871+0000","last_became_peered":"2024-09-11T21:54:07.541871+0000","last_unstale":"2024-09-11T21:57:15.768029+0000","last_undegraded":"2024-09-11T21:57:15.768029+0000","last_fullsized":"2024-09-11T21:57:15.768029+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:50:41.026895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:50:41.026895+0000","last_clean_scrub_stamp":"2024-09-11T21:50:41.026895+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:05:06.154268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"84'6","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788543+0000","last_change":"2024-09-11T21:55:18.030558+0000","last_active":"2024-09-11T21:57:13.788543+0000","last_peered":"2024-09-11T21:57:13.788543+0000","last_clean":"2024-09-11T21:57:13.788543+0000","last_became_active":"2024-09-11T21:55:18.030216+0000","last_became_peered":"2024-09-11T21:55:18.030216+0000","last_unstale":"2024-09-11T21:57:13.788543+0000","last_undegraded":"2024-09-11T21:57:13.788543+0000","last_fullsized":"2024-09-11T21:57:13.788543+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:33:02.589999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"86'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.753511+0000","last_change":"2024-09-11T21:55:20.092938+0000","last_active":"2024-09-11T21:57:13.753511+0000","last_peered":"2024-09-11T21:57:13.753511+0000","last_clean":"2024-09-11T21:57:13.753511+0000","last_became_active":"2024-09-11T21:55:20.092615+0000","last_became_peered":"2024-09-11T21:55:20.092615+0000","last_unstale":"2024-09-11T21:57:13.753511+0000","last_undegraded":"2024-09-11T21:57:13.753511+0000","last_fullsized":"2024-09-11T21:57:13.753511+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:40:06.957836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"84'4","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.670890+0000","last_change":"2024-09-11T21:55:18.029127+0000","last_active":"2024-09-11T21:57:13.670890+0000","last_peered":"2024-09-11T21:57:13.670890+0000","last_clean":"2024-09-11T21:57:13.670890+0000","last_became_active":"2024-09-11T21:55:18.028740+0000","last_became_peered":"2024-09-11T21:55:18.028740+0000","last_unstale":"2024-09-11T21:57:13.670890+0000","last_undegraded":"2024-09-11T21:57:13.670890+0000","last_fullsized":"2024-09-11T21:57:13.670890+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:14:01.128486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"86'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788358+0000","last_change":"2024-09-11T21:55:20.092498+0000","last_active":"2024-09-11T21:57:13.788358+0000","last_peered":"2024-09-11T21:57:13.788358+0000","last_clean":"2024-09-11T21:57:13.788358+0000","last_became_active":"2024-09-11T21:55:20.092179+0000","last_became_peered":"2024-09-11T21:55:20.092179+0000","last_unstale":"2024-09-11T21:57:13.788358+0000","last_undegraded":"2024-09-11T21:57:13.788358+0000","last_fullsized":"2024-09-11T21:57:13.788358+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:48:58.536942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"86'6","reported_seq":145,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:24.311968+0000","last_change":"2024-09-11T21:55:18.115798+0000","last_active":"2024-09-11T21:57:24.311968+0000","last_peered":"2024-09-11T21:57:24.311968+0000","last_clean":"2024-09-11T21:57:24.311968+0000","last_became_active":"2024-09-11T21:55:18.115433+0000","last_became_peered":"2024-09-11T21:55:18.115433+0000","last_unstale":"2024-09-11T21:57:24.311968+0000","last_undegraded":"2024-09-11T21:57:24.311968+0000","last_fullsized":"2024-09-11T21:57:24.311968+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:03:32.097756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":114,"num_read_kb":114,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"86'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.762091+0000","last_change":"2024-09-11T21:55:20.100439+0000","last_active":"2024-09-11T21:57:13.762091+0000","last_peered":"2024-09-11T21:57:13.762091+0000","last_clean":"2024-09-11T21:57:13.762091+0000","last_became_active":"2024-09-11T21:55:20.098717+0000","last_became_peered":"2024-09-11T21:55:20.098717+0000","last_unstale":"2024-09-11T21:57:13.762091+0000","last_undegraded":"2024-09-11T21:57:13.762091+0000","last_fullsized":"2024-09-11T21:57:13.762091+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:15:10.874225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"87'9","reported_seq":151,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:24.311197+0000","last_change":"2024-09-11T21:55:18.026588+0000","last_active":"2024-09-11T21:57:24.311197+0000","last_peered":"2024-09-11T21:57:24.311197+0000","last_clean":"2024-09-11T21:57:24.311197+0000","last_became_active":"2024-09-11T21:55:18.026205+0000","last_became_peered":"2024-09-11T21:55:18.026205+0000","last_unstale":"2024-09-11T21:57:24.311197+0000","last_undegraded":"2024-09-11T21:57:24.311197+0000","last_fullsized":"2024-09-11T21:57:24.311197+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:45:34.701088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1908,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":4,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"86'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.765165+0000","last_change":"2024-09-11T21:55:20.100677+0000","last_active":"2024-09-11T21:57:13.765165+0000","last_peered":"2024-09-11T21:57:13.765165+0000","last_clean":"2024-09-11T21:57:13.765165+0000","last_became_active":"2024-09-11T21:55:20.100334+0000","last_became_peered":"2024-09-11T21:55:20.100334+0000","last_unstale":"2024-09-11T21:57:13.765165+0000","last_undegraded":"2024-09-11T21:57:13.765165+0000","last_fullsized":"2024-09-11T21:57:13.765165+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:15:16.792831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"84'1","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.751258+0000","last_change":"2024-09-11T21:55:18.116329+0000","last_active":"2024-09-11T21:57:13.751258+0000","last_peered":"2024-09-11T21:57:13.751258+0000","last_clean":"2024-09-11T21:57:13.751258+0000","last_became_active":"2024-09-11T21:55:18.115944+0000","last_became_peered":"2024-09-11T21:55:18.115944+0000","last_unstale":"2024-09-11T21:57:13.751258+0000","last_undegraded":"2024-09-11T21:57:13.751258+0000","last_fullsized":"2024-09-11T21:57:13.751258+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:03:04.164985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.765187+0000","last_change":"2024-09-11T21:55:20.089202+0000","last_active":"2024-09-11T21:57:13.765187+0000","last_peered":"2024-09-11T21:57:13.765187+0000","last_clean":"2024-09-11T21:57:13.765187+0000","last_became_active":"2024-09-11T21:55:20.088853+0000","last_became_peered":"2024-09-11T21:55:20.088853+0000","last_unstale":"2024-09-11T21:57:13.765187+0000","last_undegraded":"2024-09-11T21:57:13.765187+0000","last_fullsized":"2024-09-11T21:57:13.765187+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:39:36.496944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"84'4","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671474+0000","last_change":"2024-09-11T21:55:18.117917+0000","last_active":"2024-09-11T21:57:13.671474+0000","last_peered":"2024-09-11T21:57:13.671474+0000","last_clean":"2024-09-11T21:57:13.671474+0000","last_became_active":"2024-09-11T21:55:18.117567+0000","last_became_peered":"2024-09-11T21:55:18.117567+0000","last_unstale":"2024-09-11T21:57:13.671474+0000","last_undegraded":"2024-09-11T21:57:13.671474+0000","last_fullsized":"2024-09-11T21:57:13.671474+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:21:32.546620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.789090+0000","last_change":"2024-09-11T21:55:20.092370+0000","last_active":"2024-09-11T21:57:13.789090+0000","last_peered":"2024-09-11T21:57:13.789090+0000","last_clean":"2024-09-11T21:57:13.789090+0000","last_became_active":"2024-09-11T21:55:20.091982+0000","last_became_peered":"2024-09-11T21:55:20.091982+0000","last_unstale":"2024-09-11T21:57:13.789090+0000","last_undegraded":"2024-09-11T21:57:13.789090+0000","last_fullsized":"2024-09-11T21:57:13.789090+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:59:55.214511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'104","reported_seq":221,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:23.155263+0000","last_change":"2024-09-11T21:55:18.023166+0000","last_active":"2024-09-11T21:57:23.155263+0000","last_peered":"2024-09-11T21:57:23.155263+0000","last_clean":"2024-09-11T21:57:23.155263+0000","last_became_active":"2024-09-11T21:55:18.022845+0000","last_became_peered":"2024-09-11T21:55:18.022845+0000","last_unstale":"2024-09-11T21:57:23.155263+0000","last_undegraded":"2024-09-11T21:57:23.155263+0000","last_fullsized":"2024-09-11T21:57:23.155263+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:08:06.770410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":190,"num_write_kb":1286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"86'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.753738+0000","last_change":"2024-09-11T21:55:20.091541+0000","last_active":"2024-09-11T21:57:13.753738+0000","last_peered":"2024-09-11T21:57:13.753738+0000","last_clean":"2024-09-11T21:57:13.753738+0000","last_became_active":"2024-09-11T21:55:20.090971+0000","last_became_peered":"2024-09-11T21:55:20.090971+0000","last_unstale":"2024-09-11T21:57:13.753738+0000","last_undegraded":"2024-09-11T21:57:13.753738+0000","last_fullsized":"2024-09-11T21:57:13.753738+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:22:31.974711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"84'7","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788398+0000","last_change":"2024-09-11T21:55:18.030694+0000","last_active":"2024-09-11T21:57:13.788398+0000","last_peered":"2024-09-11T21:57:13.788398+0000","last_clean":"2024-09-11T21:57:13.788398+0000","last_became_active":"2024-09-11T21:55:18.029687+0000","last_became_peered":"2024-09-11T21:55:18.029687+0000","last_unstale":"2024-09-11T21:57:13.788398+0000","last_undegraded":"2024-09-11T21:57:13.788398+0000","last_fullsized":"2024-09-11T21:57:13.788398+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:18:15.009543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"86'7","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.669625+0000","last_change":"2024-09-11T21:56:26.376104+0000","last_active":"2024-09-11T21:57:13.669625+0000","last_peered":"2024-09-11T21:57:13.669625+0000","last_clean":"2024-09-11T21:57:13.669625+0000","last_became_active":"2024-09-11T21:56:26.375665+0000","last_became_peered":"2024-09-11T21:56:26.375665+0000","last_unstale":"2024-09-11T21:57:13.669625+0000","last_undegraded":"2024-09-11T21:57:13.669625+0000","last_fullsized":"2024-09-11T21:57:13.669625+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:19:46.616205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"84'2","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788029+0000","last_change":"2024-09-11T21:55:18.115977+0000","last_active":"2024-09-11T21:57:13.788029+0000","last_peered":"2024-09-11T21:57:13.788029+0000","last_clean":"2024-09-11T21:57:13.788029+0000","last_became_active":"2024-09-11T21:55:18.115649+0000","last_became_peered":"2024-09-11T21:55:18.115649+0000","last_unstale":"2024-09-11T21:57:13.788029+0000","last_undegraded":"2024-09-11T21:57:13.788029+0000","last_fullsized":"2024-09-11T21:57:13.788029+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:42:07.621024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"86'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.670884+0000","last_change":"2024-09-11T21:55:20.095536+0000","last_active":"2024-09-11T21:57:13.670884+0000","last_peered":"2024-09-11T21:57:13.670884+0000","last_clean":"2024-09-11T21:57:13.670884+0000","last_became_active":"2024-09-11T21:55:20.095117+0000","last_became_peered":"2024-09-11T21:55:20.095117+0000","last_unstale":"2024-09-11T21:57:13.670884+0000","last_undegraded":"2024-09-11T21:57:13.670884+0000","last_fullsized":"2024-09-11T21:57:13.670884+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:46.524922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"84'7","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.753348+0000","last_change":"2024-09-11T21:55:18.026592+0000","last_active":"2024-09-11T21:57:13.753348+0000","last_peered":"2024-09-11T21:57:13.753348+0000","last_clean":"2024-09-11T21:57:13.753348+0000","last_became_active":"2024-09-11T21:55:18.026263+0000","last_became_peered":"2024-09-11T21:55:18.026263+0000","last_unstale":"2024-09-11T21:57:13.753348+0000","last_undegraded":"2024-09-11T21:57:13.753348+0000","last_fullsized":"2024-09-11T21:57:13.753348+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:22:29.382964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788053+0000","last_change":"2024-09-11T21:55:20.093655+0000","last_active":"2024-09-11T21:57:13.788053+0000","last_peered":"2024-09-11T21:57:13.788053+0000","last_clean":"2024-09-11T21:57:13.788053+0000","last_became_active":"2024-09-11T21:55:20.093287+0000","last_became_peered":"2024-09-11T21:55:20.093287+0000","last_unstale":"2024-09-11T21:57:13.788053+0000","last_undegraded":"2024-09-11T21:57:13.788053+0000","last_fullsized":"2024-09-11T21:57:13.788053+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:46:15.912710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"84'3","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.670879+0000","last_change":"2024-09-11T21:55:18.028465+0000","last_active":"2024-09-11T21:57:13.670879+0000","last_peered":"2024-09-11T21:57:13.670879+0000","last_clean":"2024-09-11T21:57:13.670879+0000","last_became_active":"2024-09-11T21:55:18.028139+0000","last_became_peered":"2024-09-11T21:55:18.028139+0000","last_unstale":"2024-09-11T21:57:13.670879+0000","last_undegraded":"2024-09-11T21:57:13.670879+0000","last_fullsized":"2024-09-11T21:57:13.670879+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:52:47.808058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.753473+0000","last_change":"2024-09-11T21:55:20.096400+0000","last_active":"2024-09-11T21:57:13.753473+0000","last_peered":"2024-09-11T21:57:13.753473+0000","last_clean":"2024-09-11T21:57:13.753473+0000","last_became_active":"2024-09-11T21:55:20.096023+0000","last_became_peered":"2024-09-11T21:55:20.096023+0000","last_unstale":"2024-09-11T21:57:13.753473+0000","last_undegraded":"2024-09-11T21:57:13.753473+0000","last_fullsized":"2024-09-11T21:57:13.753473+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:19:34.150687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"84'4","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671181+0000","last_change":"2024-09-11T21:55:18.028759+0000","last_active":"2024-09-11T21:57:13.671181+0000","last_peered":"2024-09-11T21:57:13.671181+0000","last_clean":"2024-09-11T21:57:13.671181+0000","last_became_active":"2024-09-11T21:55:18.028366+0000","last_became_peered":"2024-09-11T21:55:18.028366+0000","last_unstale":"2024-09-11T21:57:13.671181+0000","last_undegraded":"2024-09-11T21:57:13.671181+0000","last_fullsized":"2024-09-11T21:57:13.671181+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:42:05.836114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"86'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671002+0000","last_change":"2024-09-11T21:55:20.093593+0000","last_active":"2024-09-11T21:57:13.671002+0000","last_peered":"2024-09-11T21:57:13.671002+0000","last_clean":"2024-09-11T21:57:13.671002+0000","last_became_active":"2024-09-11T21:55:20.093189+0000","last_became_peered":"2024-09-11T21:55:20.093189+0000","last_unstale":"2024-09-11T21:57:13.671002+0000","last_undegraded":"2024-09-11T21:57:13.671002+0000","last_fullsized":"2024-09-11T21:57:13.671002+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:47:36.802816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"86'15","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.762163+0000","last_change":"2024-09-11T21:56:26.292793+0000","last_active":"2024-09-11T21:57:13.762163+0000","last_peered":"2024-09-11T21:57:13.762163+0000","last_clean":"2024-09-11T21:57:13.762163+0000","last_became_active":"2024-09-11T21:56:26.292437+0000","last_became_peered":"2024-09-11T21:56:26.292437+0000","last_unstale":"2024-09-11T21:57:13.762163+0000","last_undegraded":"2024-09-11T21:57:13.762163+0000","last_fullsized":"2024-09-11T21:57:13.762163+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:49:08.867435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"84'5","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671366+0000","last_change":"2024-09-11T21:55:18.027365+0000","last_active":"2024-09-11T21:57:13.671366+0000","last_peered":"2024-09-11T21:57:13.671366+0000","last_clean":"2024-09-11T21:57:13.671366+0000","last_became_active":"2024-09-11T21:55:18.027058+0000","last_became_peered":"2024-09-11T21:55:18.027058+0000","last_unstale":"2024-09-11T21:57:13.671366+0000","last_undegraded":"2024-09-11T21:57:13.671366+0000","last_fullsized":"2024-09-11T21:57:13.671366+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:30:50.613246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788034+0000","last_change":"2024-09-11T21:55:20.095154+0000","last_active":"2024-09-11T21:57:13.788034+0000","last_peered":"2024-09-11T21:57:13.788034+0000","last_clean":"2024-09-11T21:57:13.788034+0000","last_became_active":"2024-09-11T21:55:20.093607+0000","last_became_peered":"2024-09-11T21:55:20.093607+0000","last_unstale":"2024-09-11T21:57:13.788034+0000","last_undegraded":"2024-09-11T21:57:13.788034+0000","last_fullsized":"2024-09-11T21:57:13.788034+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:27:53.612467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"87'13","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671385+0000","last_change":"2024-09-11T21:55:18.029872+0000","last_active":"2024-09-11T21:57:13.671385+0000","last_peered":"2024-09-11T21:57:13.671385+0000","last_clean":"2024-09-11T21:57:13.671385+0000","last_became_active":"2024-09-11T21:55:18.029538+0000","last_became_peered":"2024-09-11T21:55:18.029538+0000","last_unstale":"2024-09-11T21:57:13.671385+0000","last_undegraded":"2024-09-11T21:57:13.671385+0000","last_fullsized":"2024-09-11T21:57:13.671385+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:05:16.824805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"86'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.669810+0000","last_change":"2024-09-11T21:55:20.098713+0000","last_active":"2024-09-11T21:57:13.669810+0000","last_peered":"2024-09-11T21:57:13.669810+0000","last_clean":"2024-09-11T21:57:13.669810+0000","last_became_active":"2024-09-11T21:55:20.098219+0000","last_became_peered":"2024-09-11T21:55:20.098219+0000","last_unstale":"2024-09-11T21:57:13.669810+0000","last_undegraded":"2024-09-11T21:57:13.669810+0000","last_fullsized":"2024-09-11T21:57:13.669810+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:07:10.585366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"84'2","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.753877+0000","last_change":"2024-09-11T21:55:18.115868+0000","last_active":"2024-09-11T21:57:13.753877+0000","last_peered":"2024-09-11T21:57:13.753877+0000","last_clean":"2024-09-11T21:57:13.753877+0000","last_became_active":"2024-09-11T21:55:18.115486+0000","last_became_peered":"2024-09-11T21:55:18.115486+0000","last_unstale":"2024-09-11T21:57:13.753877+0000","last_undegraded":"2024-09-11T21:57:13.753877+0000","last_fullsized":"2024-09-11T21:57:13.753877+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:16:57.932041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"86'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.753721+0000","last_change":"2024-09-11T21:55:20.096077+0000","last_active":"2024-09-11T21:57:13.753721+0000","last_peered":"2024-09-11T21:57:13.753721+0000","last_clean":"2024-09-11T21:57:13.753721+0000","last_became_active":"2024-09-11T21:55:20.095739+0000","last_became_peered":"2024-09-11T21:55:20.095739+0000","last_unstale":"2024-09-11T21:57:13.753721+0000","last_undegraded":"2024-09-11T21:57:13.753721+0000","last_fullsized":"2024-09-11T21:57:13.753721+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:05:28.089828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"87'15","reported_seq":155,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:23.309436+0000","last_change":"2024-09-11T21:55:18.025771+0000","last_active":"2024-09-11T21:57:23.309436+0000","last_peered":"2024-09-11T21:57:23.309436+0000","last_clean":"2024-09-11T21:57:23.309436+0000","last_became_active":"2024-09-11T21:55:18.025384+0000","last_became_peered":"2024-09-11T21:55:18.025384+0000","last_unstale":"2024-09-11T21:57:23.309436+0000","last_undegraded":"2024-09-11T21:57:23.309436+0000","last_fullsized":"2024-09-11T21:57:23.309436+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:58:21.070061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":798810,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1522,"num_read_kb":985,"num_write":507,"num_write_kb":2386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":59211,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":753664,"data_stored":2396430,"data_compressed":113059,"data_compressed_allocated":147456,"data_compressed_original":2158592,"omap_allocated":0,"internal_metadata":0},"log_size":386,"ondisk_log_size":386,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107988,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449126,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":950272,"data_stored":462934,"data_compressed":8675,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678058,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36484,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708668,"statfs":{"total":95995035648,"available":95957676032,"internally_reserved":0,"allocated":1159168,"data_stored":1449034,"data_compressed":59873,"data_compressed_allocated":77824,"data_compressed_original":921600,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":66,"seq":283467841584,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":999424,"data_stored":855153,"data_compressed":27964,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103222,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36316,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708836,"statfs":{"total":95995035648,"available":95957848064,"internally_reserved":0,"allocated":995328,"data_stored":855531,"data_compressed":27964,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332154,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1024000,"data_stored":1057390,"data_compressed":44671,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495679,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":958464,"data_stored":462974,"data_compressed":8675,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757317,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1081344,"data_stored":1383279,"data_compressed":59873,"data_compressed_allocated":77824,"data_compressed_original":921600,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920841,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":529281,"data_compressed":18261,"data_compressed_allocated":24576,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182479,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1044480,"data_stored":1182927,"data_compressed":43166,"data_compressed_allocated":57344,"data_compressed_original":724992,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":970752,"data_stored":464641,"data_compressed":8675,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1028096,"data_stored":530952,"data_compressed":8675,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":978944,"data_stored":463920,"data_compressed":8675,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":70137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":721878,"data_compressed":34491,"data_compressed_allocated":45056,"data_compressed_original":700416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":68466,"data_compressed":9586,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":331979,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":6090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":394716,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":167936,"data_stored":397734,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T21:57:26.604 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-11T21:57:26.604 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T21:57:26.862 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:57:27.238 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:26 smithi001 ceph-mon[30935]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 83 KiB/s wr, 54 op/s 2024-09-11T21:57:27.238 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:26 smithi001 ceph-mon[30935]: from='client.15408 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T21:57:27.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:26 smithi053 ceph-mon[34908]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 83 KiB/s wr, 54 op/s 2024-09-11T21:57:27.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:26 smithi053 ceph-mon[34908]: from='client.15408 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T21:57:27.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:27.320+0000 7f565dd81640 1 -- 172.21.15.1:0/374043006 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5658100a60 msgr2=0x7f5658100ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:27.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:27.320+0000 7f565dd81640 1 --2- 172.21.15.1:0/374043006 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5658100a60 0x7f5658100ec0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f5640009930 tx=0x7f564002a420 comp rx=0 tx=0).stop 2024-09-11T21:57:27.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:27.320+0000 7f565dd81640 1 -- 172.21.15.1:0/374043006 shutdown_connections 2024-09-11T21:57:27.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:27.320+0000 7f565dd81640 1 --2- 172.21.15.1:0/374043006 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5658101400 0x7f565810d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:27.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:27.320+0000 7f565dd81640 1 --2- 172.21.15.1:0/374043006 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5658100a60 0x7f5658100ec0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:27.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:27.320+0000 7f565dd81640 1 --2- 172.21.15.1:0/374043006 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5658106a60 0x7f5658106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:27.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:27.320+0000 7f565dd81640 1 -- 172.21.15.1:0/374043006 >> 172.21.15.1:0/374043006 conn(0x7f56580fc780 msgr2=0x7f56580feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:27.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:27.320+0000 7f565dd81640 1 -- 172.21.15.1:0/374043006 shutdown_connections 2024-09-11T21:57:27.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:27.320+0000 7f565dd81640 1 -- 172.21.15.1:0/374043006 wait complete. 2024-09-11T21:57:27.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:27.321+0000 7f565dd81640 1 Processor -- start 2024-09-11T21:57:27.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:27.321+0000 7f565dd81640 1 -- start start 2024-09-11T21:57:27.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:27.321+0000 7f565dd81640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5658100a60 0x7f5658102630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:27.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:27.321+0000 7f565dd81640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5658101400 0x7f5658102b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:27.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:27.321+0000 7f565dd81640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5658106a60 0x7f56581030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:27.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:27.321+0000 7f565dd81640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5658110e60 con 0x7f5658100a60 2024-09-11T21:57:27.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:27.321+0000 7f565dd81640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5658110ce0 con 0x7f5658101400 2024-09-11T21:57:27.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:27.321+0000 7f565dd81640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5658110fe0 con 0x7f5658106a60 2024-09-11T21:57:27.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:27.321+0000 7f56577fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5658100a60 0x7f5658102630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:27.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:27.321+0000 7f56577fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5658100a60 0x7f5658102630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56772/0 (socket says 172.21.15.1:56772) 2024-09-11T21:57:27.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:27.321+0000 7f56577fe640 1 -- 172.21.15.1:0/516597718 learned_addr learned my addr 172.21.15.1:0/516597718 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:27.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:27.321+0000 7f564effd640 1 --2- 172.21.15.1:0/516597718 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5658101400 0x7f5658102b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:27.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:27.321+0000 7f5657fff640 1 --2- 172.21.15.1:0/516597718 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5658106a60 0x7f56581030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:27.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:27.322+0000 7f56577fe640 1 -- 172.21.15.1:0/516597718 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5658106a60 msgr2=0x7f56581030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:27.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:27.322+0000 7f56577fe640 1 --2- 172.21.15.1:0/516597718 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5658106a60 0x7f56581030b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:27.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:27.322+0000 7f56577fe640 1 -- 172.21.15.1:0/516597718 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5658101400 msgr2=0x7f5658102b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:27.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:27.322+0000 7f56577fe640 1 --2- 172.21.15.1:0/516597718 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5658101400 0x7f5658102b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:27.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:27.322+0000 7f56577fe640 1 -- 172.21.15.1:0/516597718 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56581a68e0 con 0x7f5658100a60 2024-09-11T21:57:27.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:27.322+0000 7f564effd640 1 --2- 172.21.15.1:0/516597718 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5658101400 0x7f5658102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T21:57:27.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:27.322+0000 7f5657fff640 1 --2- 172.21.15.1:0/516597718 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5658106a60 0x7f56581030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T21:57:27.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:27.322+0000 7f56577fe640 1 --2- 172.21.15.1:0/516597718 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5658100a60 0x7f5658102630 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f564800c920 tx=0x7f564800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:27.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:27.322+0000 7f56557fa640 1 -- 172.21.15.1:0/516597718 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5648007c70 con 0x7f5658100a60 2024-09-11T21:57:27.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:27.322+0000 7f56557fa640 1 -- 172.21.15.1:0/516597718 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5648007e10 con 0x7f5658100a60 2024-09-11T21:57:27.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:27.322+0000 7f565dd81640 1 -- 172.21.15.1:0/516597718 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56581a6bd0 con 0x7f5658100a60 2024-09-11T21:57:27.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:27.322+0000 7f56557fa640 1 -- 172.21.15.1:0/516597718 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5648018890 con 0x7f5658100a60 2024-09-11T21:57:27.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:27.322+0000 7f565dd81640 1 -- 172.21.15.1:0/516597718 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56581a7110 con 0x7f5658100a60 2024-09-11T21:57:27.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:27.323+0000 7f565dd81640 1 -- 172.21.15.1:0/516597718 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5658100ec0 con 0x7f5658100a60 2024-09-11T21:57:27.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:27.324+0000 7f56557fa640 1 -- 172.21.15.1:0/516597718 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 24) ==== 138008+0+0 (secure 0 0 0) 0x7f5648018a50 con 0x7f5658100a60 2024-09-11T21:57:27.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:27.325+0000 7f56557fa640 1 --2- 172.21.15.1:0/516597718 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f56440a2d80 0x7f56440a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:27.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:27.325+0000 7f564effd640 1 --2- 172.21.15.1:0/516597718 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f56440a2d80 0x7f56440a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:27.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:27.325+0000 7f56557fa640 1 -- 172.21.15.1:0/516597718 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7f56480cf9e0 con 0x7f5658100a60 2024-09-11T21:57:27.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:27.326+0000 7f564effd640 1 --2- 172.21.15.1:0/516597718 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f56440a2d80 0x7f56440a5240 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5640000c00 tx=0x7f5640032c70 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:27.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:27.328+0000 7f56557fa640 1 -- 172.21.15.1:0/516597718 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5648014030 con 0x7f5658100a60 2024-09-11T21:57:27.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:26 smithi161 ceph-mon[35615]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 83 KiB/s wr, 54 op/s 2024-09-11T21:57:27.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:26 smithi161 ceph-mon[35615]: from='client.15408 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T21:57:27.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:27.499+0000 7f565dd81640 1 -- 172.21.15.1:0/516597718 --> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f565806a790 con 0x7f56440a2d80 2024-09-11T21:57:27.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:27.505+0000 7f56557fa640 1 -- 172.21.15.1:0/516597718 <== mgr.15204 v2:172.21.15.1:6800/3543150772 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258106 (secure 0 0 0) 0x7f565806a790 con 0x7f56440a2d80 2024-09-11T21:57:27.508 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:57:27.509 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-09-11T21:57:27.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:27.508+0000 7f565dd81640 1 -- 172.21.15.1:0/516597718 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f56440a2d80 msgr2=0x7f56440a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:27.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:27.508+0000 7f565dd81640 1 --2- 172.21.15.1:0/516597718 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f56440a2d80 0x7f56440a5240 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5640000c00 tx=0x7f5640032c70 comp rx=0 tx=0).stop 2024-09-11T21:57:27.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:27.508+0000 7f565dd81640 1 -- 172.21.15.1:0/516597718 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5658100a60 msgr2=0x7f5658102630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:27.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:27.508+0000 7f565dd81640 1 --2- 172.21.15.1:0/516597718 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5658100a60 0x7f5658102630 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f564800c920 tx=0x7f564800cde0 comp rx=0 tx=0).stop 2024-09-11T21:57:27.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:27.508+0000 7f565dd81640 1 -- 172.21.15.1:0/516597718 shutdown_connections 2024-09-11T21:57:27.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:27.508+0000 7f565dd81640 1 --2- 172.21.15.1:0/516597718 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f56440a2d80 0x7f56440a5240 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:27.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:27.508+0000 7f565dd81640 1 --2- 172.21.15.1:0/516597718 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5658106a60 0x7f56581030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:27.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:27.508+0000 7f565dd81640 1 --2- 172.21.15.1:0/516597718 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5658101400 0x7f5658102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:27.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:27.508+0000 7f565dd81640 1 --2- 172.21.15.1:0/516597718 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5658100a60 0x7f5658102630 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:27.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:27.508+0000 7f565dd81640 1 -- 172.21.15.1:0/516597718 >> 172.21.15.1:0/516597718 conn(0x7f56580fc780 msgr2=0x7f5658104870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:27.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:27.508+0000 7f565dd81640 1 -- 172.21.15.1:0/516597718 shutdown_connections 2024-09-11T21:57:27.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:27.508+0000 7f565dd81640 1 -- 172.21.15.1:0/516597718 wait complete. 2024-09-11T21:57:28.189 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":9,"stamp":"2024-09-11T21:57:25.684928+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1389412,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1759,"num_read_kb":1315,"num_write":710,"num_write_kb":5326,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1239947,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1036,"ondisk_log_size":1036,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":390308,"kb_used_data":11896,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124551516,"statfs":{"total":1151940427776,"available":1151540752384,"internally_reserved":0,"allocated":12181504,"data_stored":9698016,"data_compressed":325147,"data_compressed_allocated":430080,"data_compressed_original":4222976,"omap_allocated":19142,"internal_metadata":387233082},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":197770,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":457,"num_read_kb":495,"num_write":147,"num_write_kb":916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.992354"},"pg_stats":[{"pgid":"3.3f","version":"86'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788356+0000","last_change":"2024-09-11T21:55:20.095385+0000","last_active":"2024-09-11T21:57:13.788356+0000","last_peered":"2024-09-11T21:57:13.788356+0000","last_clean":"2024-09-11T21:57:13.788356+0000","last_became_active":"2024-09-11T21:55:20.095077+0000","last_became_peered":"2024-09-11T21:55:20.095077+0000","last_unstale":"2024-09-11T21:57:13.788356+0000","last_undegraded":"2024-09-11T21:57:13.788356+0000","last_fullsized":"2024-09-11T21:57:13.788356+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:53:52.231522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"86'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.669632+0000","last_change":"2024-09-11T21:55:20.103540+0000","last_active":"2024-09-11T21:57:13.669632+0000","last_peered":"2024-09-11T21:57:13.669632+0000","last_clean":"2024-09-11T21:57:13.669632+0000","last_became_active":"2024-09-11T21:55:20.103233+0000","last_became_peered":"2024-09-11T21:55:20.103233+0000","last_unstale":"2024-09-11T21:57:13.669632+0000","last_undegraded":"2024-09-11T21:57:13.669632+0000","last_fullsized":"2024-09-11T21:57:13.669632+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:51:29.222202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"86'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.669623+0000","last_change":"2024-09-11T21:55:20.098538+0000","last_active":"2024-09-11T21:57:13.669623+0000","last_peered":"2024-09-11T21:57:13.669623+0000","last_clean":"2024-09-11T21:57:13.669623+0000","last_became_active":"2024-09-11T21:55:20.098202+0000","last_became_peered":"2024-09-11T21:55:20.098202+0000","last_unstale":"2024-09-11T21:57:13.669623+0000","last_undegraded":"2024-09-11T21:57:13.669623+0000","last_fullsized":"2024-09-11T21:57:13.669623+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:33:46.636952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"86'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788540+0000","last_change":"2024-09-11T21:55:20.100869+0000","last_active":"2024-09-11T21:57:13.788540+0000","last_peered":"2024-09-11T21:57:13.788540+0000","last_clean":"2024-09-11T21:57:13.788540+0000","last_became_active":"2024-09-11T21:55:20.100573+0000","last_became_peered":"2024-09-11T21:55:20.100573+0000","last_unstale":"2024-09-11T21:57:13.788540+0000","last_undegraded":"2024-09-11T21:57:13.788540+0000","last_fullsized":"2024-09-11T21:57:13.788540+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:58:11.154105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"86'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671033+0000","last_change":"2024-09-11T21:55:20.091648+0000","last_active":"2024-09-11T21:57:13.671033+0000","last_peered":"2024-09-11T21:57:13.671033+0000","last_clean":"2024-09-11T21:57:13.671033+0000","last_became_active":"2024-09-11T21:55:20.091222+0000","last_became_peered":"2024-09-11T21:55:20.091222+0000","last_unstale":"2024-09-11T21:57:13.671033+0000","last_undegraded":"2024-09-11T21:57:13.671033+0000","last_fullsized":"2024-09-11T21:57:13.671033+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:55:03.420754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"86'6","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.764975+0000","last_change":"2024-09-11T21:56:26.376595+0000","last_active":"2024-09-11T21:57:13.764975+0000","last_peered":"2024-09-11T21:57:13.764975+0000","last_clean":"2024-09-11T21:57:13.764975+0000","last_became_active":"2024-09-11T21:56:26.375784+0000","last_became_peered":"2024-09-11T21:56:26.375784+0000","last_unstale":"2024-09-11T21:57:13.764975+0000","last_undegraded":"2024-09-11T21:57:13.764975+0000","last_fullsized":"2024-09-11T21:57:13.764975+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:19:38.371463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,7],"acting":[10,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"86'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.670787+0000","last_change":"2024-09-11T21:55:20.101084+0000","last_active":"2024-09-11T21:57:13.670787+0000","last_peered":"2024-09-11T21:57:13.670787+0000","last_clean":"2024-09-11T21:57:13.670787+0000","last_became_active":"2024-09-11T21:55:20.100739+0000","last_became_peered":"2024-09-11T21:55:20.100739+0000","last_unstale":"2024-09-11T21:57:13.670787+0000","last_undegraded":"2024-09-11T21:57:13.670787+0000","last_fullsized":"2024-09-11T21:57:13.670787+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:46:58.969559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.753160+0000","last_change":"2024-09-11T21:55:20.088715+0000","last_active":"2024-09-11T21:57:13.753160+0000","last_peered":"2024-09-11T21:57:13.753160+0000","last_clean":"2024-09-11T21:57:13.753160+0000","last_became_active":"2024-09-11T21:55:20.088280+0000","last_became_peered":"2024-09-11T21:55:20.088280+0000","last_unstale":"2024-09-11T21:57:13.753160+0000","last_undegraded":"2024-09-11T21:57:13.753160+0000","last_fullsized":"2024-09-11T21:57:13.753160+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:06:24.450444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"86'11","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788671+0000","last_change":"2024-09-11T21:55:20.101266+0000","last_active":"2024-09-11T21:57:13.788671+0000","last_peered":"2024-09-11T21:57:13.788671+0000","last_clean":"2024-09-11T21:57:13.788671+0000","last_became_active":"2024-09-11T21:55:20.100919+0000","last_became_peered":"2024-09-11T21:55:20.100919+0000","last_unstale":"2024-09-11T21:57:13.788671+0000","last_undegraded":"2024-09-11T21:57:13.788671+0000","last_fullsized":"2024-09-11T21:57:13.788671+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:47:15.466082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"86'16","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.765361+0000","last_change":"2024-09-11T21:55:20.098086+0000","last_active":"2024-09-11T21:57:13.765361+0000","last_peered":"2024-09-11T21:57:13.765361+0000","last_clean":"2024-09-11T21:57:13.765361+0000","last_became_active":"2024-09-11T21:55:20.097754+0000","last_became_peered":"2024-09-11T21:55:20.097754+0000","last_unstale":"2024-09-11T21:57:13.765361+0000","last_undegraded":"2024-09-11T21:57:13.765361+0000","last_fullsized":"2024-09-11T21:57:13.765361+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:37:00.098345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"86'6","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671036+0000","last_change":"2024-09-11T21:55:20.092671+0000","last_active":"2024-09-11T21:57:13.671036+0000","last_peered":"2024-09-11T21:57:13.671036+0000","last_clean":"2024-09-11T21:57:13.671036+0000","last_became_active":"2024-09-11T21:55:20.092112+0000","last_became_peered":"2024-09-11T21:55:20.092112+0000","last_unstale":"2024-09-11T21:57:13.671036+0000","last_undegraded":"2024-09-11T21:57:13.671036+0000","last_fullsized":"2024-09-11T21:57:13.671036+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:06:28.538838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788646+0000","last_change":"2024-09-11T21:55:20.099863+0000","last_active":"2024-09-11T21:57:13.788646+0000","last_peered":"2024-09-11T21:57:13.788646+0000","last_clean":"2024-09-11T21:57:13.788646+0000","last_became_active":"2024-09-11T21:55:20.099173+0000","last_became_peered":"2024-09-11T21:55:20.099173+0000","last_unstale":"2024-09-11T21:57:13.788646+0000","last_undegraded":"2024-09-11T21:57:13.788646+0000","last_fullsized":"2024-09-11T21:57:13.788646+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:34:56.669410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"86'10","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788507+0000","last_change":"2024-09-11T21:55:20.101212+0000","last_active":"2024-09-11T21:57:13.788507+0000","last_peered":"2024-09-11T21:57:13.788507+0000","last_clean":"2024-09-11T21:57:13.788507+0000","last_became_active":"2024-09-11T21:55:20.100876+0000","last_became_peered":"2024-09-11T21:55:20.100876+0000","last_unstale":"2024-09-11T21:57:13.788507+0000","last_undegraded":"2024-09-11T21:57:13.788507+0000","last_fullsized":"2024-09-11T21:57:13.788507+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:14:18.104009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.669316+0000","last_change":"2024-09-11T21:55:20.095394+0000","last_active":"2024-09-11T21:57:13.669316+0000","last_peered":"2024-09-11T21:57:13.669316+0000","last_clean":"2024-09-11T21:57:13.669316+0000","last_became_active":"2024-09-11T21:55:20.095101+0000","last_became_peered":"2024-09-11T21:55:20.095101+0000","last_unstale":"2024-09-11T21:57:13.669316+0000","last_undegraded":"2024-09-11T21:57:13.669316+0000","last_fullsized":"2024-09-11T21:57:13.669316+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:53:32.529359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"86'7","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.669266+0000","last_change":"2024-09-11T21:56:26.376206+0000","last_active":"2024-09-11T21:57:13.669266+0000","last_peered":"2024-09-11T21:57:13.669266+0000","last_clean":"2024-09-11T21:57:13.669266+0000","last_became_active":"2024-09-11T21:56:26.375829+0000","last_became_peered":"2024-09-11T21:56:26.375829+0000","last_unstale":"2024-09-11T21:57:13.669266+0000","last_undegraded":"2024-09-11T21:57:13.669266+0000","last_fullsized":"2024-09-11T21:57:13.669266+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:21:59.484414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.787934+0000","last_change":"2024-09-11T21:55:20.093019+0000","last_active":"2024-09-11T21:57:13.787934+0000","last_peered":"2024-09-11T21:57:13.787934+0000","last_clean":"2024-09-11T21:57:13.787934+0000","last_became_active":"2024-09-11T21:55:20.092691+0000","last_became_peered":"2024-09-11T21:55:20.092691+0000","last_unstale":"2024-09-11T21:57:13.787934+0000","last_undegraded":"2024-09-11T21:57:13.787934+0000","last_fullsized":"2024-09-11T21:57:13.787934+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:04:36.684992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788379+0000","last_change":"2024-09-11T21:55:20.097333+0000","last_active":"2024-09-11T21:57:13.788379+0000","last_peered":"2024-09-11T21:57:13.788379+0000","last_clean":"2024-09-11T21:57:13.788379+0000","last_became_active":"2024-09-11T21:55:20.096752+0000","last_became_peered":"2024-09-11T21:55:20.096752+0000","last_unstale":"2024-09-11T21:57:13.788379+0000","last_undegraded":"2024-09-11T21:57:13.788379+0000","last_fullsized":"2024-09-11T21:57:13.788379+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:27:20.722174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"86'10","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.751322+0000","last_change":"2024-09-11T21:55:20.093972+0000","last_active":"2024-09-11T21:57:13.751322+0000","last_peered":"2024-09-11T21:57:13.751322+0000","last_clean":"2024-09-11T21:57:13.751322+0000","last_became_active":"2024-09-11T21:55:20.093633+0000","last_became_peered":"2024-09-11T21:55:20.093633+0000","last_unstale":"2024-09-11T21:57:13.751322+0000","last_undegraded":"2024-09-11T21:57:13.751322+0000","last_fullsized":"2024-09-11T21:57:13.751322+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:25:18.973332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671334+0000","last_change":"2024-09-11T21:55:20.094239+0000","last_active":"2024-09-11T21:57:13.671334+0000","last_peered":"2024-09-11T21:57:13.671334+0000","last_clean":"2024-09-11T21:57:13.671334+0000","last_became_active":"2024-09-11T21:55:20.093924+0000","last_became_peered":"2024-09-11T21:55:20.093924+0000","last_unstale":"2024-09-11T21:57:13.671334+0000","last_undegraded":"2024-09-11T21:57:13.671334+0000","last_fullsized":"2024-09-11T21:57:13.671334+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:39:53.012780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788075+0000","last_change":"2024-09-11T21:55:20.099335+0000","last_active":"2024-09-11T21:57:13.788075+0000","last_peered":"2024-09-11T21:57:13.788075+0000","last_clean":"2024-09-11T21:57:13.788075+0000","last_became_active":"2024-09-11T21:55:20.098929+0000","last_became_peered":"2024-09-11T21:55:20.098929+0000","last_unstale":"2024-09-11T21:57:13.788075+0000","last_undegraded":"2024-09-11T21:57:13.788075+0000","last_fullsized":"2024-09-11T21:57:13.788075+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:38:43.117829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"86'11","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788526+0000","last_change":"2024-09-11T21:55:20.094068+0000","last_active":"2024-09-11T21:57:13.788526+0000","last_peered":"2024-09-11T21:57:13.788526+0000","last_clean":"2024-09-11T21:57:13.788526+0000","last_became_active":"2024-09-11T21:55:20.093637+0000","last_became_peered":"2024-09-11T21:55:20.093637+0000","last_unstale":"2024-09-11T21:57:13.788526+0000","last_undegraded":"2024-09-11T21:57:13.788526+0000","last_fullsized":"2024-09-11T21:57:13.788526+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:52:59.289715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"86'10","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788122+0000","last_change":"2024-09-11T21:55:20.090778+0000","last_active":"2024-09-11T21:57:13.788122+0000","last_peered":"2024-09-11T21:57:13.788122+0000","last_clean":"2024-09-11T21:57:13.788122+0000","last_became_active":"2024-09-11T21:55:20.090443+0000","last_became_peered":"2024-09-11T21:55:20.090443+0000","last_unstale":"2024-09-11T21:57:13.788122+0000","last_undegraded":"2024-09-11T21:57:13.788122+0000","last_fullsized":"2024-09-11T21:57:13.788122+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:46:21.488228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.753203+0000","last_change":"2024-09-11T21:55:20.092238+0000","last_active":"2024-09-11T21:57:13.753203+0000","last_peered":"2024-09-11T21:57:13.753203+0000","last_clean":"2024-09-11T21:57:13.753203+0000","last_became_active":"2024-09-11T21:55:20.091961+0000","last_became_peered":"2024-09-11T21:55:20.091961+0000","last_unstale":"2024-09-11T21:57:13.753203+0000","last_undegraded":"2024-09-11T21:57:13.753203+0000","last_fullsized":"2024-09-11T21:57:13.753203+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:54:40.445441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"86'6","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.761984+0000","last_change":"2024-09-11T21:55:20.102944+0000","last_active":"2024-09-11T21:57:13.761984+0000","last_peered":"2024-09-11T21:57:13.761984+0000","last_clean":"2024-09-11T21:57:13.761984+0000","last_became_active":"2024-09-11T21:55:20.102533+0000","last_became_peered":"2024-09-11T21:55:20.102533+0000","last_unstale":"2024-09-11T21:57:13.761984+0000","last_undegraded":"2024-09-11T21:57:13.761984+0000","last_fullsized":"2024-09-11T21:57:13.761984+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:12:15.832622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"86'2","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.753729+0000","last_change":"2024-09-11T21:56:26.292272+0000","last_active":"2024-09-11T21:57:13.753729+0000","last_peered":"2024-09-11T21:57:13.753729+0000","last_clean":"2024-09-11T21:57:13.753729+0000","last_became_active":"2024-09-11T21:56:26.291920+0000","last_became_peered":"2024-09-11T21:56:26.291920+0000","last_unstale":"2024-09-11T21:57:13.753729+0000","last_undegraded":"2024-09-11T21:57:13.753729+0000","last_fullsized":"2024-09-11T21:57:13.753729+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:13:13.579907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"86'15","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671489+0000","last_change":"2024-09-11T21:56:26.376364+0000","last_active":"2024-09-11T21:57:13.671489+0000","last_peered":"2024-09-11T21:57:13.671489+0000","last_clean":"2024-09-11T21:57:13.671489+0000","last_became_active":"2024-09-11T21:56:26.375927+0000","last_became_peered":"2024-09-11T21:56:26.375927+0000","last_unstale":"2024-09-11T21:57:13.671489+0000","last_undegraded":"2024-09-11T21:57:13.671489+0000","last_fullsized":"2024-09-11T21:57:13.671489+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:23:03.664207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,9],"acting":[2,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"86'6","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788153+0000","last_change":"2024-09-11T21:56:26.376702+0000","last_active":"2024-09-11T21:57:13.788153+0000","last_peered":"2024-09-11T21:57:13.788153+0000","last_clean":"2024-09-11T21:57:13.788153+0000","last_became_active":"2024-09-11T21:56:26.376179+0000","last_became_peered":"2024-09-11T21:56:26.376179+0000","last_unstale":"2024-09-11T21:57:13.788153+0000","last_undegraded":"2024-09-11T21:57:13.788153+0000","last_fullsized":"2024-09-11T21:57:13.788153+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:27:34.666962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"86'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788960+0000","last_change":"2024-09-11T21:55:20.093898+0000","last_active":"2024-09-11T21:57:13.788960+0000","last_peered":"2024-09-11T21:57:13.788960+0000","last_clean":"2024-09-11T21:57:13.788960+0000","last_became_active":"2024-09-11T21:55:20.093348+0000","last_became_peered":"2024-09-11T21:55:20.093348+0000","last_unstale":"2024-09-11T21:57:13.788960+0000","last_undegraded":"2024-09-11T21:57:13.788960+0000","last_fullsized":"2024-09-11T21:57:13.788960+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:00:06.185478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"86'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.669510+0000","last_change":"2024-09-11T21:55:20.100575+0000","last_active":"2024-09-11T21:57:13.669510+0000","last_peered":"2024-09-11T21:57:13.669510+0000","last_clean":"2024-09-11T21:57:13.669510+0000","last_became_active":"2024-09-11T21:55:20.100255+0000","last_became_peered":"2024-09-11T21:55:20.100255+0000","last_unstale":"2024-09-11T21:57:13.669510+0000","last_undegraded":"2024-09-11T21:57:13.669510+0000","last_fullsized":"2024-09-11T21:57:13.669510+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:48:57.718298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"86'8","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.670780+0000","last_change":"2024-09-11T21:56:26.377183+0000","last_active":"2024-09-11T21:57:13.670780+0000","last_peered":"2024-09-11T21:57:13.670780+0000","last_clean":"2024-09-11T21:57:13.670780+0000","last_became_active":"2024-09-11T21:56:26.375906+0000","last_became_peered":"2024-09-11T21:56:26.375906+0000","last_unstale":"2024-09-11T21:57:13.670780+0000","last_undegraded":"2024-09-11T21:57:13.670780+0000","last_fullsized":"2024-09-11T21:57:13.670780+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:00:29.033961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"86'12","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.670836+0000","last_change":"2024-09-11T21:55:20.093822+0000","last_active":"2024-09-11T21:57:13.670836+0000","last_peered":"2024-09-11T21:57:13.670836+0000","last_clean":"2024-09-11T21:57:13.670836+0000","last_became_active":"2024-09-11T21:55:20.093361+0000","last_became_peered":"2024-09-11T21:55:20.093361+0000","last_unstale":"2024-09-11T21:57:13.670836+0000","last_undegraded":"2024-09-11T21:57:13.670836+0000","last_fullsized":"2024-09-11T21:57:13.670836+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:37:06.362732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"86'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788121+0000","last_change":"2024-09-11T21:55:20.103990+0000","last_active":"2024-09-11T21:57:13.788121+0000","last_peered":"2024-09-11T21:57:13.788121+0000","last_clean":"2024-09-11T21:57:13.788121+0000","last_became_active":"2024-09-11T21:55:20.103375+0000","last_became_peered":"2024-09-11T21:55:20.103375+0000","last_unstale":"2024-09-11T21:57:13.788121+0000","last_undegraded":"2024-09-11T21:57:13.788121+0000","last_fullsized":"2024-09-11T21:57:13.788121+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:02:06.786888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"86'7","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671024+0000","last_change":"2024-09-11T21:56:26.376261+0000","last_active":"2024-09-11T21:57:13.671024+0000","last_peered":"2024-09-11T21:57:13.671024+0000","last_clean":"2024-09-11T21:57:13.671024+0000","last_became_active":"2024-09-11T21:56:26.375863+0000","last_became_peered":"2024-09-11T21:56:26.375863+0000","last_unstale":"2024-09-11T21:57:13.671024+0000","last_undegraded":"2024-09-11T21:57:13.671024+0000","last_fullsized":"2024-09-11T21:57:13.671024+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:58:21.359836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"87'8","reported_seq":150,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:23.310305+0000","last_change":"2024-09-11T21:55:18.026947+0000","last_active":"2024-09-11T21:57:23.310305+0000","last_peered":"2024-09-11T21:57:23.310305+0000","last_clean":"2024-09-11T21:57:23.310305+0000","last_became_active":"2024-09-11T21:55:18.026632+0000","last_became_peered":"2024-09-11T21:55:18.026632+0000","last_unstale":"2024-09-11T21:57:23.310305+0000","last_undegraded":"2024-09-11T21:57:23.310305+0000","last_fullsized":"2024-09-11T21:57:23.310305+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:55:16.618074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1549,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.762157+0000","last_change":"2024-09-11T21:55:20.100197+0000","last_active":"2024-09-11T21:57:13.762157+0000","last_peered":"2024-09-11T21:57:13.762157+0000","last_clean":"2024-09-11T21:57:13.762157+0000","last_became_active":"2024-09-11T21:55:20.098494+0000","last_became_peered":"2024-09-11T21:55:20.098494+0000","last_unstale":"2024-09-11T21:57:13.762157+0000","last_undegraded":"2024-09-11T21:57:13.762157+0000","last_fullsized":"2024-09-11T21:57:13.762157+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:14.505314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"84'7","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.765023+0000","last_change":"2024-09-11T21:55:18.024359+0000","last_active":"2024-09-11T21:57:13.765023+0000","last_peered":"2024-09-11T21:57:13.765023+0000","last_clean":"2024-09-11T21:57:13.765023+0000","last_became_active":"2024-09-11T21:55:18.023624+0000","last_became_peered":"2024-09-11T21:55:18.023624+0000","last_unstale":"2024-09-11T21:57:13.765023+0000","last_undegraded":"2024-09-11T21:57:13.765023+0000","last_fullsized":"2024-09-11T21:57:13.765023+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:13:38.582232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"84'3","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788356+0000","last_change":"2024-09-11T21:55:18.032541+0000","last_active":"2024-09-11T21:57:13.788356+0000","last_peered":"2024-09-11T21:57:13.788356+0000","last_clean":"2024-09-11T21:57:13.788356+0000","last_became_active":"2024-09-11T21:55:18.032218+0000","last_became_peered":"2024-09-11T21:55:18.032218+0000","last_unstale":"2024-09-11T21:57:13.788356+0000","last_undegraded":"2024-09-11T21:57:13.788356+0000","last_fullsized":"2024-09-11T21:57:13.788356+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:38:20.872069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.753229+0000","last_change":"2024-09-11T21:55:20.090030+0000","last_active":"2024-09-11T21:57:13.753229+0000","last_peered":"2024-09-11T21:57:13.753229+0000","last_clean":"2024-09-11T21:57:13.753229+0000","last_became_active":"2024-09-11T21:55:20.089481+0000","last_became_peered":"2024-09-11T21:55:20.089481+0000","last_unstale":"2024-09-11T21:57:13.753229+0000","last_undegraded":"2024-09-11T21:57:13.753229+0000","last_fullsized":"2024-09-11T21:57:13.753229+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:41:57.038519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"84'3","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.762394+0000","last_change":"2024-09-11T21:55:18.026656+0000","last_active":"2024-09-11T21:57:13.762394+0000","last_peered":"2024-09-11T21:57:13.762394+0000","last_clean":"2024-09-11T21:57:13.762394+0000","last_became_active":"2024-09-11T21:55:18.026318+0000","last_became_peered":"2024-09-11T21:55:18.026318+0000","last_unstale":"2024-09-11T21:57:13.762394+0000","last_undegraded":"2024-09-11T21:57:13.762394+0000","last_fullsized":"2024-09-11T21:57:13.762394+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:14:41.438334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"86'11","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.750970+0000","last_change":"2024-09-11T21:55:20.090790+0000","last_active":"2024-09-11T21:57:13.750970+0000","last_peered":"2024-09-11T21:57:13.750970+0000","last_clean":"2024-09-11T21:57:13.750970+0000","last_became_active":"2024-09-11T21:55:20.090468+0000","last_became_peered":"2024-09-11T21:55:20.090468+0000","last_unstale":"2024-09-11T21:57:13.750970+0000","last_undegraded":"2024-09-11T21:57:13.750970+0000","last_fullsized":"2024-09-11T21:57:13.750970+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:19.115482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'8","reported_seq":258,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:24.311247+0000","last_change":"2024-09-11T21:55:18.026989+0000","last_active":"2024-09-11T21:57:24.311247+0000","last_peered":"2024-09-11T21:57:24.311247+0000","last_clean":"2024-09-11T21:57:24.311247+0000","last_became_active":"2024-09-11T21:55:18.026229+0000","last_became_peered":"2024-09-11T21:55:18.026229+0000","last_unstale":"2024-09-11T21:57:24.311247+0000","last_undegraded":"2024-09-11T21:57:24.311247+0000","last_fullsized":"2024-09-11T21:57:24.311247+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:50:35.745789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":226,"num_read_kb":226,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"86'10","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788388+0000","last_change":"2024-09-11T21:55:20.095884+0000","last_active":"2024-09-11T21:57:13.788388+0000","last_peered":"2024-09-11T21:57:13.788388+0000","last_clean":"2024-09-11T21:57:13.788388+0000","last_became_active":"2024-09-11T21:55:20.094887+0000","last_became_peered":"2024-09-11T21:55:20.094887+0000","last_unstale":"2024-09-11T21:57:13.788388+0000","last_undegraded":"2024-09-11T21:57:13.788388+0000","last_fullsized":"2024-09-11T21:57:13.788388+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:45:03.579703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"84'4","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671167+0000","last_change":"2024-09-11T21:55:18.030474+0000","last_active":"2024-09-11T21:57:13.671167+0000","last_peered":"2024-09-11T21:57:13.671167+0000","last_clean":"2024-09-11T21:57:13.671167+0000","last_became_active":"2024-09-11T21:55:18.030104+0000","last_became_peered":"2024-09-11T21:55:18.030104+0000","last_unstale":"2024-09-11T21:57:13.671167+0000","last_undegraded":"2024-09-11T21:57:13.671167+0000","last_fullsized":"2024-09-11T21:57:13.671167+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:21:00.484595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.669795+0000","last_change":"2024-09-11T21:55:20.102807+0000","last_active":"2024-09-11T21:57:13.669795+0000","last_peered":"2024-09-11T21:57:13.669795+0000","last_clean":"2024-09-11T21:57:13.669795+0000","last_became_active":"2024-09-11T21:55:20.102581+0000","last_became_peered":"2024-09-11T21:55:20.102581+0000","last_unstale":"2024-09-11T21:57:13.669795+0000","last_undegraded":"2024-09-11T21:57:13.669795+0000","last_fullsized":"2024-09-11T21:57:13.669795+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:29:23.139788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"84'5","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.762200+0000","last_change":"2024-09-11T21:55:18.117685+0000","last_active":"2024-09-11T21:57:13.762200+0000","last_peered":"2024-09-11T21:57:13.762200+0000","last_clean":"2024-09-11T21:57:13.762200+0000","last_became_active":"2024-09-11T21:55:18.117261+0000","last_became_peered":"2024-09-11T21:55:18.117261+0000","last_unstale":"2024-09-11T21:57:13.762200+0000","last_undegraded":"2024-09-11T21:57:13.762200+0000","last_fullsized":"2024-09-11T21:57:13.762200+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:42:13.180725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"86'11","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788081+0000","last_change":"2024-09-11T21:55:20.086562+0000","last_active":"2024-09-11T21:57:13.788081+0000","last_peered":"2024-09-11T21:57:13.788081+0000","last_clean":"2024-09-11T21:57:13.788081+0000","last_became_active":"2024-09-11T21:55:20.086197+0000","last_became_peered":"2024-09-11T21:55:20.086197+0000","last_unstale":"2024-09-11T21:57:13.788081+0000","last_undegraded":"2024-09-11T21:57:13.788081+0000","last_fullsized":"2024-09-11T21:57:13.788081+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:23:44.705132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'36","reported_seq":319,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:24.370948+0000","last_change":"2024-09-11T21:56:26.299386+0000","last_active":"2024-09-11T21:57:24.370948+0000","last_peered":"2024-09-11T21:57:24.370948+0000","last_clean":"2024-09-11T21:57:24.370948+0000","last_became_active":"2024-09-11T21:56:26.291951+0000","last_became_peered":"2024-09-11T21:56:26.291951+0000","last_unstale":"2024-09-11T21:57:24.370948+0000","last_undegraded":"2024-09-11T21:57:24.370948+0000","last_fullsized":"2024-09-11T21:57:24.370948+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:54:47.346798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67065,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":231,"num_read_kb":182,"num_write":38,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":59211,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"86'6","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671198+0000","last_change":"2024-09-11T21:55:20.091695+0000","last_active":"2024-09-11T21:57:13.671198+0000","last_peered":"2024-09-11T21:57:13.671198+0000","last_clean":"2024-09-11T21:57:13.671198+0000","last_became_active":"2024-09-11T21:55:20.091128+0000","last_became_peered":"2024-09-11T21:55:20.091128+0000","last_unstale":"2024-09-11T21:57:13.671198+0000","last_undegraded":"2024-09-11T21:57:13.671198+0000","last_fullsized":"2024-09-11T21:57:13.671198+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:16:20.751952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"84'9","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671024+0000","last_change":"2024-09-11T21:55:18.030827+0000","last_active":"2024-09-11T21:57:13.671024+0000","last_peered":"2024-09-11T21:57:13.671024+0000","last_clean":"2024-09-11T21:57:13.671024+0000","last_became_active":"2024-09-11T21:55:18.030354+0000","last_became_peered":"2024-09-11T21:55:18.030354+0000","last_unstale":"2024-09-11T21:57:13.671024+0000","last_undegraded":"2024-09-11T21:57:13.671024+0000","last_fullsized":"2024-09-11T21:57:13.671024+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:23:19.578820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788550+0000","last_change":"2024-09-11T21:55:20.090160+0000","last_active":"2024-09-11T21:57:13.788550+0000","last_peered":"2024-09-11T21:57:13.788550+0000","last_clean":"2024-09-11T21:57:13.788550+0000","last_became_active":"2024-09-11T21:55:20.089801+0000","last_became_peered":"2024-09-11T21:55:20.089801+0000","last_unstale":"2024-09-11T21:57:13.788550+0000","last_undegraded":"2024-09-11T21:57:13.788550+0000","last_fullsized":"2024-09-11T21:57:13.788550+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:34:16.093123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"84'5","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.761998+0000","last_change":"2024-09-11T21:55:18.026817+0000","last_active":"2024-09-11T21:57:13.761998+0000","last_peered":"2024-09-11T21:57:13.761998+0000","last_clean":"2024-09-11T21:57:13.761998+0000","last_became_active":"2024-09-11T21:55:18.026188+0000","last_became_peered":"2024-09-11T21:55:18.026188+0000","last_unstale":"2024-09-11T21:57:13.761998+0000","last_undegraded":"2024-09-11T21:57:13.761998+0000","last_fullsized":"2024-09-11T21:57:13.761998+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:55:14.774503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"86'15","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.670991+0000","last_change":"2024-09-11T21:55:20.095613+0000","last_active":"2024-09-11T21:57:13.670991+0000","last_peered":"2024-09-11T21:57:13.670991+0000","last_clean":"2024-09-11T21:57:13.670991+0000","last_became_active":"2024-09-11T21:55:20.095302+0000","last_became_peered":"2024-09-11T21:55:20.095302+0000","last_unstale":"2024-09-11T21:57:13.670991+0000","last_undegraded":"2024-09-11T21:57:13.670991+0000","last_fullsized":"2024-09-11T21:57:13.670991+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:37:27.961213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'7","reported_seq":148,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:23.311297+0000","last_change":"2024-09-11T21:55:18.032417+0000","last_active":"2024-09-11T21:57:23.311297+0000","last_peered":"2024-09-11T21:57:23.311297+0000","last_clean":"2024-09-11T21:57:23.311297+0000","last_became_active":"2024-09-11T21:55:18.031419+0000","last_became_peered":"2024-09-11T21:55:18.031419+0000","last_unstale":"2024-09-11T21:57:23.311297+0000","last_undegraded":"2024-09-11T21:57:23.311297+0000","last_fullsized":"2024-09-11T21:57:23.311297+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:30:52.285678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1289,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":118,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"86'6","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.670730+0000","last_change":"2024-09-11T21:55:20.101965+0000","last_active":"2024-09-11T21:57:13.670730+0000","last_peered":"2024-09-11T21:57:13.670730+0000","last_clean":"2024-09-11T21:57:13.670730+0000","last_became_active":"2024-09-11T21:55:20.101534+0000","last_became_peered":"2024-09-11T21:55:20.101534+0000","last_unstale":"2024-09-11T21:57:13.670730+0000","last_undegraded":"2024-09-11T21:57:13.670730+0000","last_fullsized":"2024-09-11T21:57:13.670730+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:41:30.685121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"84'17","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.670811+0000","last_change":"2024-09-11T21:55:18.116059+0000","last_active":"2024-09-11T21:57:13.670811+0000","last_peered":"2024-09-11T21:57:13.670811+0000","last_clean":"2024-09-11T21:57:13.670811+0000","last_became_active":"2024-09-11T21:55:18.115621+0000","last_became_peered":"2024-09-11T21:55:18.115621+0000","last_unstale":"2024-09-11T21:57:13.670811+0000","last_undegraded":"2024-09-11T21:57:13.670811+0000","last_fullsized":"2024-09-11T21:57:13.670811+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:47:17.815695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"86'6","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.762037+0000","last_change":"2024-09-11T21:55:20.100903+0000","last_active":"2024-09-11T21:57:13.762037+0000","last_peered":"2024-09-11T21:57:13.762037+0000","last_clean":"2024-09-11T21:57:13.762037+0000","last_became_active":"2024-09-11T21:55:20.100561+0000","last_became_peered":"2024-09-11T21:55:20.100561+0000","last_unstale":"2024-09-11T21:57:13.762037+0000","last_undegraded":"2024-09-11T21:57:13.762037+0000","last_fullsized":"2024-09-11T21:57:13.762037+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:00:45.472238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'2","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671632+0000","last_change":"2024-09-11T21:55:18.029549+0000","last_active":"2024-09-11T21:57:13.671632+0000","last_peered":"2024-09-11T21:57:13.671632+0000","last_clean":"2024-09-11T21:57:13.671632+0000","last_became_active":"2024-09-11T21:55:18.029178+0000","last_became_peered":"2024-09-11T21:55:18.029178+0000","last_unstale":"2024-09-11T21:57:13.671632+0000","last_undegraded":"2024-09-11T21:57:13.671632+0000","last_fullsized":"2024-09-11T21:57:13.671632+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:37:07.915786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"86'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788144+0000","last_change":"2024-09-11T21:55:20.104202+0000","last_active":"2024-09-11T21:57:13.788144+0000","last_peered":"2024-09-11T21:57:13.788144+0000","last_clean":"2024-09-11T21:57:13.788144+0000","last_became_active":"2024-09-11T21:55:20.103464+0000","last_became_peered":"2024-09-11T21:55:20.103464+0000","last_unstale":"2024-09-11T21:57:13.788144+0000","last_undegraded":"2024-09-11T21:57:13.788144+0000","last_fullsized":"2024-09-11T21:57:13.788144+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:30:16.044143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"84'5","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788728+0000","last_change":"2024-09-11T21:55:18.026183+0000","last_active":"2024-09-11T21:57:13.788728+0000","last_peered":"2024-09-11T21:57:13.788728+0000","last_clean":"2024-09-11T21:57:13.788728+0000","last_became_active":"2024-09-11T21:55:18.025849+0000","last_became_peered":"2024-09-11T21:55:18.025849+0000","last_unstale":"2024-09-11T21:57:13.788728+0000","last_undegraded":"2024-09-11T21:57:13.788728+0000","last_fullsized":"2024-09-11T21:57:13.788728+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:21:12.189158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671103+0000","last_change":"2024-09-11T21:55:20.090652+0000","last_active":"2024-09-11T21:57:13.671103+0000","last_peered":"2024-09-11T21:57:13.671103+0000","last_clean":"2024-09-11T21:57:13.671103+0000","last_became_active":"2024-09-11T21:55:20.090348+0000","last_became_peered":"2024-09-11T21:55:20.090348+0000","last_unstale":"2024-09-11T21:57:13.671103+0000","last_undegraded":"2024-09-11T21:57:13.671103+0000","last_fullsized":"2024-09-11T21:57:13.671103+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:10:58.733371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'67","reported_seq":122,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:22.806643+0000","last_change":"2024-09-11T21:55:18.024388+0000","last_active":"2024-09-11T21:57:22.806643+0000","last_peered":"2024-09-11T21:57:22.806643+0000","last_clean":"2024-09-11T21:57:22.806643+0000","last_became_active":"2024-09-11T21:55:18.024100+0000","last_became_peered":"2024-09-11T21:55:18.024100+0000","last_unstale":"2024-09-11T21:57:22.806643+0000","last_undegraded":"2024-09-11T21:57:22.806643+0000","last_fullsized":"2024-09-11T21:57:22.806643+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:21:11.152282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.670936+0000","last_change":"2024-09-11T21:55:20.101007+0000","last_active":"2024-09-11T21:57:13.670936+0000","last_peered":"2024-09-11T21:57:13.670936+0000","last_clean":"2024-09-11T21:57:13.670936+0000","last_became_active":"2024-09-11T21:55:20.100642+0000","last_became_peered":"2024-09-11T21:55:20.100642+0000","last_unstale":"2024-09-11T21:57:13.670936+0000","last_undegraded":"2024-09-11T21:57:13.670936+0000","last_fullsized":"2024-09-11T21:57:13.670936+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:16:05.642184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"87'8","reported_seq":150,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:23.339239+0000","last_change":"2024-09-11T21:55:18.116216+0000","last_active":"2024-09-11T21:57:23.339239+0000","last_peered":"2024-09-11T21:57:23.339239+0000","last_clean":"2024-09-11T21:57:23.339239+0000","last_became_active":"2024-09-11T21:55:18.115871+0000","last_became_peered":"2024-09-11T21:55:18.115871+0000","last_unstale":"2024-09-11T21:57:23.339239+0000","last_undegraded":"2024-09-11T21:57:23.339239+0000","last_fullsized":"2024-09-11T21:57:23.339239+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:16:56.027636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":118,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671175+0000","last_change":"2024-09-11T21:55:20.092421+0000","last_active":"2024-09-11T21:57:13.671175+0000","last_peered":"2024-09-11T21:57:13.671175+0000","last_clean":"2024-09-11T21:57:13.671175+0000","last_became_active":"2024-09-11T21:55:20.091707+0000","last_became_peered":"2024-09-11T21:55:20.091707+0000","last_unstale":"2024-09-11T21:57:13.671175+0000","last_undegraded":"2024-09-11T21:57:13.671175+0000","last_fullsized":"2024-09-11T21:57:13.671175+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:12:38.157687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"87'148","reported_seq":397,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:15.768029+0000","last_change":"2024-09-11T21:54:07.542362+0000","last_active":"2024-09-11T21:57:15.768029+0000","last_peered":"2024-09-11T21:57:15.768029+0000","last_clean":"2024-09-11T21:57:15.768029+0000","last_became_active":"2024-09-11T21:54:07.541871+0000","last_became_peered":"2024-09-11T21:54:07.541871+0000","last_unstale":"2024-09-11T21:57:15.768029+0000","last_undegraded":"2024-09-11T21:57:15.768029+0000","last_fullsized":"2024-09-11T21:57:15.768029+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:50:41.026895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:50:41.026895+0000","last_clean_scrub_stamp":"2024-09-11T21:50:41.026895+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:05:06.154268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"84'6","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788543+0000","last_change":"2024-09-11T21:55:18.030558+0000","last_active":"2024-09-11T21:57:13.788543+0000","last_peered":"2024-09-11T21:57:13.788543+0000","last_clean":"2024-09-11T21:57:13.788543+0000","last_became_active":"2024-09-11T21:55:18.030216+0000","last_became_peered":"2024-09-11T21:55:18.030216+0000","last_unstale":"2024-09-11T21:57:13.788543+0000","last_undegraded":"2024-09-11T21:57:13.788543+0000","last_fullsized":"2024-09-11T21:57:13.788543+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:33:02.589999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"86'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.753511+0000","last_change":"2024-09-11T21:55:20.092938+0000","last_active":"2024-09-11T21:57:13.753511+0000","last_peered":"2024-09-11T21:57:13.753511+0000","last_clean":"2024-09-11T21:57:13.753511+0000","last_became_active":"2024-09-11T21:55:20.092615+0000","last_became_peered":"2024-09-11T21:55:20.092615+0000","last_unstale":"2024-09-11T21:57:13.753511+0000","last_undegraded":"2024-09-11T21:57:13.753511+0000","last_fullsized":"2024-09-11T21:57:13.753511+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:40:06.957836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"84'4","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.670890+0000","last_change":"2024-09-11T21:55:18.029127+0000","last_active":"2024-09-11T21:57:13.670890+0000","last_peered":"2024-09-11T21:57:13.670890+0000","last_clean":"2024-09-11T21:57:13.670890+0000","last_became_active":"2024-09-11T21:55:18.028740+0000","last_became_peered":"2024-09-11T21:55:18.028740+0000","last_unstale":"2024-09-11T21:57:13.670890+0000","last_undegraded":"2024-09-11T21:57:13.670890+0000","last_fullsized":"2024-09-11T21:57:13.670890+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:14:01.128486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"86'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788358+0000","last_change":"2024-09-11T21:55:20.092498+0000","last_active":"2024-09-11T21:57:13.788358+0000","last_peered":"2024-09-11T21:57:13.788358+0000","last_clean":"2024-09-11T21:57:13.788358+0000","last_became_active":"2024-09-11T21:55:20.092179+0000","last_became_peered":"2024-09-11T21:55:20.092179+0000","last_unstale":"2024-09-11T21:57:13.788358+0000","last_undegraded":"2024-09-11T21:57:13.788358+0000","last_fullsized":"2024-09-11T21:57:13.788358+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:48:58.536942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"86'6","reported_seq":145,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:24.311968+0000","last_change":"2024-09-11T21:55:18.115798+0000","last_active":"2024-09-11T21:57:24.311968+0000","last_peered":"2024-09-11T21:57:24.311968+0000","last_clean":"2024-09-11T21:57:24.311968+0000","last_became_active":"2024-09-11T21:55:18.115433+0000","last_became_peered":"2024-09-11T21:55:18.115433+0000","last_unstale":"2024-09-11T21:57:24.311968+0000","last_undegraded":"2024-09-11T21:57:24.311968+0000","last_fullsized":"2024-09-11T21:57:24.311968+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:03:32.097756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":114,"num_read_kb":114,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"86'8","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.762091+0000","last_change":"2024-09-11T21:55:20.100439+0000","last_active":"2024-09-11T21:57:13.762091+0000","last_peered":"2024-09-11T21:57:13.762091+0000","last_clean":"2024-09-11T21:57:13.762091+0000","last_became_active":"2024-09-11T21:55:20.098717+0000","last_became_peered":"2024-09-11T21:55:20.098717+0000","last_unstale":"2024-09-11T21:57:13.762091+0000","last_undegraded":"2024-09-11T21:57:13.762091+0000","last_fullsized":"2024-09-11T21:57:13.762091+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:15:10.874225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"87'9","reported_seq":151,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:24.311197+0000","last_change":"2024-09-11T21:55:18.026588+0000","last_active":"2024-09-11T21:57:24.311197+0000","last_peered":"2024-09-11T21:57:24.311197+0000","last_clean":"2024-09-11T21:57:24.311197+0000","last_became_active":"2024-09-11T21:55:18.026205+0000","last_became_peered":"2024-09-11T21:55:18.026205+0000","last_unstale":"2024-09-11T21:57:24.311197+0000","last_undegraded":"2024-09-11T21:57:24.311197+0000","last_fullsized":"2024-09-11T21:57:24.311197+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:45:34.701088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1908,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":4,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"86'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.765165+0000","last_change":"2024-09-11T21:55:20.100677+0000","last_active":"2024-09-11T21:57:13.765165+0000","last_peered":"2024-09-11T21:57:13.765165+0000","last_clean":"2024-09-11T21:57:13.765165+0000","last_became_active":"2024-09-11T21:55:20.100334+0000","last_became_peered":"2024-09-11T21:55:20.100334+0000","last_unstale":"2024-09-11T21:57:13.765165+0000","last_undegraded":"2024-09-11T21:57:13.765165+0000","last_fullsized":"2024-09-11T21:57:13.765165+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:15:16.792831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"84'1","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.751258+0000","last_change":"2024-09-11T21:55:18.116329+0000","last_active":"2024-09-11T21:57:13.751258+0000","last_peered":"2024-09-11T21:57:13.751258+0000","last_clean":"2024-09-11T21:57:13.751258+0000","last_became_active":"2024-09-11T21:55:18.115944+0000","last_became_peered":"2024-09-11T21:55:18.115944+0000","last_unstale":"2024-09-11T21:57:13.751258+0000","last_undegraded":"2024-09-11T21:57:13.751258+0000","last_fullsized":"2024-09-11T21:57:13.751258+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:03:04.164985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.765187+0000","last_change":"2024-09-11T21:55:20.089202+0000","last_active":"2024-09-11T21:57:13.765187+0000","last_peered":"2024-09-11T21:57:13.765187+0000","last_clean":"2024-09-11T21:57:13.765187+0000","last_became_active":"2024-09-11T21:55:20.088853+0000","last_became_peered":"2024-09-11T21:55:20.088853+0000","last_unstale":"2024-09-11T21:57:13.765187+0000","last_undegraded":"2024-09-11T21:57:13.765187+0000","last_fullsized":"2024-09-11T21:57:13.765187+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:39:36.496944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"84'4","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671474+0000","last_change":"2024-09-11T21:55:18.117917+0000","last_active":"2024-09-11T21:57:13.671474+0000","last_peered":"2024-09-11T21:57:13.671474+0000","last_clean":"2024-09-11T21:57:13.671474+0000","last_became_active":"2024-09-11T21:55:18.117567+0000","last_became_peered":"2024-09-11T21:55:18.117567+0000","last_unstale":"2024-09-11T21:57:13.671474+0000","last_undegraded":"2024-09-11T21:57:13.671474+0000","last_fullsized":"2024-09-11T21:57:13.671474+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:21:32.546620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.789090+0000","last_change":"2024-09-11T21:55:20.092370+0000","last_active":"2024-09-11T21:57:13.789090+0000","last_peered":"2024-09-11T21:57:13.789090+0000","last_clean":"2024-09-11T21:57:13.789090+0000","last_became_active":"2024-09-11T21:55:20.091982+0000","last_became_peered":"2024-09-11T21:55:20.091982+0000","last_unstale":"2024-09-11T21:57:13.789090+0000","last_undegraded":"2024-09-11T21:57:13.789090+0000","last_fullsized":"2024-09-11T21:57:13.789090+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:59:55.214511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'104","reported_seq":221,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:23.155263+0000","last_change":"2024-09-11T21:55:18.023166+0000","last_active":"2024-09-11T21:57:23.155263+0000","last_peered":"2024-09-11T21:57:23.155263+0000","last_clean":"2024-09-11T21:57:23.155263+0000","last_became_active":"2024-09-11T21:55:18.022845+0000","last_became_peered":"2024-09-11T21:55:18.022845+0000","last_unstale":"2024-09-11T21:57:23.155263+0000","last_undegraded":"2024-09-11T21:57:23.155263+0000","last_fullsized":"2024-09-11T21:57:23.155263+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:08:06.770410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":190,"num_write_kb":1286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"86'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.753738+0000","last_change":"2024-09-11T21:55:20.091541+0000","last_active":"2024-09-11T21:57:13.753738+0000","last_peered":"2024-09-11T21:57:13.753738+0000","last_clean":"2024-09-11T21:57:13.753738+0000","last_became_active":"2024-09-11T21:55:20.090971+0000","last_became_peered":"2024-09-11T21:55:20.090971+0000","last_unstale":"2024-09-11T21:57:13.753738+0000","last_undegraded":"2024-09-11T21:57:13.753738+0000","last_fullsized":"2024-09-11T21:57:13.753738+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:22:31.974711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"84'7","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788398+0000","last_change":"2024-09-11T21:55:18.030694+0000","last_active":"2024-09-11T21:57:13.788398+0000","last_peered":"2024-09-11T21:57:13.788398+0000","last_clean":"2024-09-11T21:57:13.788398+0000","last_became_active":"2024-09-11T21:55:18.029687+0000","last_became_peered":"2024-09-11T21:55:18.029687+0000","last_unstale":"2024-09-11T21:57:13.788398+0000","last_undegraded":"2024-09-11T21:57:13.788398+0000","last_fullsized":"2024-09-11T21:57:13.788398+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:18:15.009543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"86'7","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.669625+0000","last_change":"2024-09-11T21:56:26.376104+0000","last_active":"2024-09-11T21:57:13.669625+0000","last_peered":"2024-09-11T21:57:13.669625+0000","last_clean":"2024-09-11T21:57:13.669625+0000","last_became_active":"2024-09-11T21:56:26.375665+0000","last_became_peered":"2024-09-11T21:56:26.375665+0000","last_unstale":"2024-09-11T21:57:13.669625+0000","last_undegraded":"2024-09-11T21:57:13.669625+0000","last_fullsized":"2024-09-11T21:57:13.669625+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:19:46.616205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"84'2","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788029+0000","last_change":"2024-09-11T21:55:18.115977+0000","last_active":"2024-09-11T21:57:13.788029+0000","last_peered":"2024-09-11T21:57:13.788029+0000","last_clean":"2024-09-11T21:57:13.788029+0000","last_became_active":"2024-09-11T21:55:18.115649+0000","last_became_peered":"2024-09-11T21:55:18.115649+0000","last_unstale":"2024-09-11T21:57:13.788029+0000","last_undegraded":"2024-09-11T21:57:13.788029+0000","last_fullsized":"2024-09-11T21:57:13.788029+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:42:07.621024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"86'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.670884+0000","last_change":"2024-09-11T21:55:20.095536+0000","last_active":"2024-09-11T21:57:13.670884+0000","last_peered":"2024-09-11T21:57:13.670884+0000","last_clean":"2024-09-11T21:57:13.670884+0000","last_became_active":"2024-09-11T21:55:20.095117+0000","last_became_peered":"2024-09-11T21:55:20.095117+0000","last_unstale":"2024-09-11T21:57:13.670884+0000","last_undegraded":"2024-09-11T21:57:13.670884+0000","last_fullsized":"2024-09-11T21:57:13.670884+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:46.524922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"84'7","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.753348+0000","last_change":"2024-09-11T21:55:18.026592+0000","last_active":"2024-09-11T21:57:13.753348+0000","last_peered":"2024-09-11T21:57:13.753348+0000","last_clean":"2024-09-11T21:57:13.753348+0000","last_became_active":"2024-09-11T21:55:18.026263+0000","last_became_peered":"2024-09-11T21:55:18.026263+0000","last_unstale":"2024-09-11T21:57:13.753348+0000","last_undegraded":"2024-09-11T21:57:13.753348+0000","last_fullsized":"2024-09-11T21:57:13.753348+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:22:29.382964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788053+0000","last_change":"2024-09-11T21:55:20.093655+0000","last_active":"2024-09-11T21:57:13.788053+0000","last_peered":"2024-09-11T21:57:13.788053+0000","last_clean":"2024-09-11T21:57:13.788053+0000","last_became_active":"2024-09-11T21:55:20.093287+0000","last_became_peered":"2024-09-11T21:55:20.093287+0000","last_unstale":"2024-09-11T21:57:13.788053+0000","last_undegraded":"2024-09-11T21:57:13.788053+0000","last_fullsized":"2024-09-11T21:57:13.788053+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:46:15.912710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"84'3","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.670879+0000","last_change":"2024-09-11T21:55:18.028465+0000","last_active":"2024-09-11T21:57:13.670879+0000","last_peered":"2024-09-11T21:57:13.670879+0000","last_clean":"2024-09-11T21:57:13.670879+0000","last_became_active":"2024-09-11T21:55:18.028139+0000","last_became_peered":"2024-09-11T21:55:18.028139+0000","last_unstale":"2024-09-11T21:57:13.670879+0000","last_undegraded":"2024-09-11T21:57:13.670879+0000","last_fullsized":"2024-09-11T21:57:13.670879+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:52:47.808058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.753473+0000","last_change":"2024-09-11T21:55:20.096400+0000","last_active":"2024-09-11T21:57:13.753473+0000","last_peered":"2024-09-11T21:57:13.753473+0000","last_clean":"2024-09-11T21:57:13.753473+0000","last_became_active":"2024-09-11T21:55:20.096023+0000","last_became_peered":"2024-09-11T21:55:20.096023+0000","last_unstale":"2024-09-11T21:57:13.753473+0000","last_undegraded":"2024-09-11T21:57:13.753473+0000","last_fullsized":"2024-09-11T21:57:13.753473+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:19:34.150687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"84'4","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671181+0000","last_change":"2024-09-11T21:55:18.028759+0000","last_active":"2024-09-11T21:57:13.671181+0000","last_peered":"2024-09-11T21:57:13.671181+0000","last_clean":"2024-09-11T21:57:13.671181+0000","last_became_active":"2024-09-11T21:55:18.028366+0000","last_became_peered":"2024-09-11T21:55:18.028366+0000","last_unstale":"2024-09-11T21:57:13.671181+0000","last_undegraded":"2024-09-11T21:57:13.671181+0000","last_fullsized":"2024-09-11T21:57:13.671181+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:42:05.836114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"86'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671002+0000","last_change":"2024-09-11T21:55:20.093593+0000","last_active":"2024-09-11T21:57:13.671002+0000","last_peered":"2024-09-11T21:57:13.671002+0000","last_clean":"2024-09-11T21:57:13.671002+0000","last_became_active":"2024-09-11T21:55:20.093189+0000","last_became_peered":"2024-09-11T21:55:20.093189+0000","last_unstale":"2024-09-11T21:57:13.671002+0000","last_undegraded":"2024-09-11T21:57:13.671002+0000","last_fullsized":"2024-09-11T21:57:13.671002+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:47:36.802816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"86'15","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.762163+0000","last_change":"2024-09-11T21:56:26.292793+0000","last_active":"2024-09-11T21:57:13.762163+0000","last_peered":"2024-09-11T21:57:13.762163+0000","last_clean":"2024-09-11T21:57:13.762163+0000","last_became_active":"2024-09-11T21:56:26.292437+0000","last_became_peered":"2024-09-11T21:56:26.292437+0000","last_unstale":"2024-09-11T21:57:13.762163+0000","last_undegraded":"2024-09-11T21:57:13.762163+0000","last_fullsized":"2024-09-11T21:57:13.762163+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:49:08.867435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"84'5","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671366+0000","last_change":"2024-09-11T21:55:18.027365+0000","last_active":"2024-09-11T21:57:13.671366+0000","last_peered":"2024-09-11T21:57:13.671366+0000","last_clean":"2024-09-11T21:57:13.671366+0000","last_became_active":"2024-09-11T21:55:18.027058+0000","last_became_peered":"2024-09-11T21:55:18.027058+0000","last_unstale":"2024-09-11T21:57:13.671366+0000","last_undegraded":"2024-09-11T21:57:13.671366+0000","last_fullsized":"2024-09-11T21:57:13.671366+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:30:50.613246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.788034+0000","last_change":"2024-09-11T21:55:20.095154+0000","last_active":"2024-09-11T21:57:13.788034+0000","last_peered":"2024-09-11T21:57:13.788034+0000","last_clean":"2024-09-11T21:57:13.788034+0000","last_became_active":"2024-09-11T21:55:20.093607+0000","last_became_peered":"2024-09-11T21:55:20.093607+0000","last_unstale":"2024-09-11T21:57:13.788034+0000","last_undegraded":"2024-09-11T21:57:13.788034+0000","last_fullsized":"2024-09-11T21:57:13.788034+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:27:53.612467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"87'13","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.671385+0000","last_change":"2024-09-11T21:55:18.029872+0000","last_active":"2024-09-11T21:57:13.671385+0000","last_peered":"2024-09-11T21:57:13.671385+0000","last_clean":"2024-09-11T21:57:13.671385+0000","last_became_active":"2024-09-11T21:55:18.029538+0000","last_became_peered":"2024-09-11T21:55:18.029538+0000","last_unstale":"2024-09-11T21:57:13.671385+0000","last_undegraded":"2024-09-11T21:57:13.671385+0000","last_fullsized":"2024-09-11T21:57:13.671385+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:05:16.824805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"86'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.669810+0000","last_change":"2024-09-11T21:55:20.098713+0000","last_active":"2024-09-11T21:57:13.669810+0000","last_peered":"2024-09-11T21:57:13.669810+0000","last_clean":"2024-09-11T21:57:13.669810+0000","last_became_active":"2024-09-11T21:55:20.098219+0000","last_became_peered":"2024-09-11T21:55:20.098219+0000","last_unstale":"2024-09-11T21:57:13.669810+0000","last_undegraded":"2024-09-11T21:57:13.669810+0000","last_fullsized":"2024-09-11T21:57:13.669810+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:07:10.585366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"84'2","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.753877+0000","last_change":"2024-09-11T21:55:18.115868+0000","last_active":"2024-09-11T21:57:13.753877+0000","last_peered":"2024-09-11T21:57:13.753877+0000","last_clean":"2024-09-11T21:57:13.753877+0000","last_became_active":"2024-09-11T21:55:18.115486+0000","last_became_peered":"2024-09-11T21:55:18.115486+0000","last_unstale":"2024-09-11T21:57:13.753877+0000","last_undegraded":"2024-09-11T21:57:13.753877+0000","last_fullsized":"2024-09-11T21:57:13.753877+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:16:57.932041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"86'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:13.753721+0000","last_change":"2024-09-11T21:55:20.096077+0000","last_active":"2024-09-11T21:57:13.753721+0000","last_peered":"2024-09-11T21:57:13.753721+0000","last_clean":"2024-09-11T21:57:13.753721+0000","last_became_active":"2024-09-11T21:55:20.095739+0000","last_became_peered":"2024-09-11T21:55:20.095739+0000","last_unstale":"2024-09-11T21:57:13.753721+0000","last_undegraded":"2024-09-11T21:57:13.753721+0000","last_fullsized":"2024-09-11T21:57:13.753721+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:19.066674+0000","last_clean_scrub_stamp":"2024-09-11T21:55:19.066674+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:05:28.089828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"87'15","reported_seq":155,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T21:57:23.309436+0000","last_change":"2024-09-11T21:55:18.025771+0000","last_active":"2024-09-11T21:57:23.309436+0000","last_peered":"2024-09-11T21:57:23.309436+0000","last_clean":"2024-09-11T21:57:23.309436+0000","last_became_active":"2024-09-11T21:55:18.025384+0000","last_became_peered":"2024-09-11T21:55:18.025384+0000","last_unstale":"2024-09-11T21:57:23.309436+0000","last_undegraded":"2024-09-11T21:57:23.309436+0000","last_fullsized":"2024-09-11T21:57:23.309436+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T21:55:17.012201+0000","last_clean_scrub_stamp":"2024-09-11T21:55:17.012201+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:58:21.070061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":798810,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1522,"num_read_kb":985,"num_write":507,"num_write_kb":2386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":59211,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":753664,"data_stored":2396430,"data_compressed":113059,"data_compressed_allocated":147456,"data_compressed_original":2158592,"omap_allocated":0,"internal_metadata":0},"log_size":386,"ondisk_log_size":386,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107988,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449126,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":950272,"data_stored":462934,"data_compressed":8675,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678058,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36484,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708668,"statfs":{"total":95995035648,"available":95957676032,"internally_reserved":0,"allocated":1159168,"data_stored":1449034,"data_compressed":59873,"data_compressed_allocated":77824,"data_compressed_original":921600,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":66,"seq":283467841584,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":999424,"data_stored":855153,"data_compressed":27964,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103222,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36316,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708836,"statfs":{"total":95995035648,"available":95957848064,"internally_reserved":0,"allocated":995328,"data_stored":855531,"data_compressed":27964,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332154,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1024000,"data_stored":1057390,"data_compressed":44671,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495679,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":958464,"data_stored":462974,"data_compressed":8675,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757317,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1081344,"data_stored":1383279,"data_compressed":59873,"data_compressed_allocated":77824,"data_compressed_original":921600,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920841,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":529281,"data_compressed":18261,"data_compressed_allocated":24576,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182479,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1044480,"data_stored":1182927,"data_compressed":43166,"data_compressed_allocated":57344,"data_compressed_original":724992,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":970752,"data_stored":464641,"data_compressed":8675,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1028096,"data_stored":530952,"data_compressed":8675,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":978944,"data_stored":463920,"data_compressed":8675,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35996,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":70137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":721878,"data_compressed":34491,"data_compressed_allocated":45056,"data_compressed_original":700416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":68466,"data_compressed":9586,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":331979,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":6090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":394716,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":167936,"data_stored":397734,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T21:57:28.252 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T21:57:28.511 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:57:28.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:28.970+0000 7fcba13a5640 1 -- 172.21.15.1:0/842325089 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb9c1036e0 msgr2=0x7fcb9c107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:28.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:28.970+0000 7fcba13a5640 1 --2- 172.21.15.1:0/842325089 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb9c1036e0 0x7fcb9c107b90 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fcb90009930 tx=0x7fcb9002a420 comp rx=0 tx=0).stop 2024-09-11T21:57:28.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:28.970+0000 7fcba13a5640 1 -- 172.21.15.1:0/842325089 shutdown_connections 2024-09-11T21:57:28.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:28.970+0000 7fcba13a5640 1 --2- 172.21.15.1:0/842325089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb9c1082a0 0x7fcb9c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:28.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:28.970+0000 7fcba13a5640 1 --2- 172.21.15.1:0/842325089 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb9c1036e0 0x7fcb9c107b90 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:28.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:28.970+0000 7fcba13a5640 1 --2- 172.21.15.1:0/842325089 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb9c102d30 0x7fcb9c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:28.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:28.970+0000 7fcba13a5640 1 -- 172.21.15.1:0/842325089 >> 172.21.15.1:0/842325089 conn(0x7fcb9c0fc7d0 msgr2=0x7fcb9c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:28.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:28.970+0000 7fcba13a5640 1 -- 172.21.15.1:0/842325089 shutdown_connections 2024-09-11T21:57:28.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:28.970+0000 7fcba13a5640 1 -- 172.21.15.1:0/842325089 wait complete. 2024-09-11T21:57:28.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:28.970+0000 7fcba13a5640 1 Processor -- start 2024-09-11T21:57:28.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:28.971+0000 7fcba13a5640 1 -- start start 2024-09-11T21:57:28.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:28.971+0000 7fcba13a5640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb9c102d30 0x7fcb9c100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:28.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:28.971+0000 7fcba13a5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb9c1036e0 0x7fcb9c100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:28.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:28.971+0000 7fcba13a5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb9c1082a0 0x7fcb9c101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:28.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:28.971+0000 7fcba13a5640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9c112060 con 0x7fcb9c1082a0 2024-09-11T21:57:28.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:28.971+0000 7fcba13a5640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9c111ee0 con 0x7fcb9c1036e0 2024-09-11T21:57:28.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:28.971+0000 7fcba13a5640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9c1121e0 con 0x7fcb9c102d30 2024-09-11T21:57:28.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:28.971+0000 7fcb9b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb9c1082a0 0x7fcb9c101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:28.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:28.971+0000 7fcb9b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb9c1082a0 0x7fcb9c101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56792/0 (socket says 172.21.15.1:56792) 2024-09-11T21:57:28.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:28.971+0000 7fcb9b7fe640 1 -- 172.21.15.1:0/3625662214 learned_addr learned my addr 172.21.15.1:0/3625662214 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:28.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:28.971+0000 7fcb9a7fc640 1 --2- 172.21.15.1:0/3625662214 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb9c1036e0 0x7fcb9c100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:28.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:28.971+0000 7fcb9affd640 1 --2- 172.21.15.1:0/3625662214 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb9c102d30 0x7fcb9c100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:28.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:28.971+0000 7fcb9b7fe640 1 -- 172.21.15.1:0/3625662214 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb9c102d30 msgr2=0x7fcb9c100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:28.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:28.971+0000 7fcb9b7fe640 1 --2- 172.21.15.1:0/3625662214 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb9c102d30 0x7fcb9c100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:28.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:28.971+0000 7fcb9b7fe640 1 -- 172.21.15.1:0/3625662214 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb9c1036e0 msgr2=0x7fcb9c100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:28.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:28.971+0000 7fcb9b7fe640 1 --2- 172.21.15.1:0/3625662214 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb9c1036e0 0x7fcb9c100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:28.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:28.971+0000 7fcb9b7fe640 1 -- 172.21.15.1:0/3625662214 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb9c1a6a60 con 0x7fcb9c1082a0 2024-09-11T21:57:29.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:28.971+0000 7fcb9a7fc640 1 --2- 172.21.15.1:0/3625662214 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb9c1036e0 0x7fcb9c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T21:57:29.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:28.972+0000 7fcb9b7fe640 1 --2- 172.21.15.1:0/3625662214 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb9c1082a0 0x7fcb9c101410 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fcb9400e970 tx=0x7fcb9400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:29.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:28.974+0000 7fcb83fff640 1 -- 172.21.15.1:0/3625662214 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb9400ccc0 con 0x7fcb9c1082a0 2024-09-11T21:57:29.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:28.974+0000 7fcb83fff640 1 -- 172.21.15.1:0/3625662214 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb9400ce60 con 0x7fcb9c1082a0 2024-09-11T21:57:29.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:28.974+0000 7fcba13a5640 1 -- 172.21.15.1:0/3625662214 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb9c1a6d50 con 0x7fcb9c1082a0 2024-09-11T21:57:29.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:28.974+0000 7fcb83fff640 1 -- 172.21.15.1:0/3625662214 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb94010490 con 0x7fcb9c1082a0 2024-09-11T21:57:29.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:28.974+0000 7fcba13a5640 1 -- 172.21.15.1:0/3625662214 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb9c1a7260 con 0x7fcb9c1082a0 2024-09-11T21:57:29.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:28.975+0000 7fcba13a5640 1 -- 172.21.15.1:0/3625662214 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb9c103110 con 0x7fcb9c1082a0 2024-09-11T21:57:29.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:28.976+0000 7fcb83fff640 1 -- 172.21.15.1:0/3625662214 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 24) ==== 138008+0+0 (secure 0 0 0) 0x7fcb94005020 con 0x7fcb9c1082a0 2024-09-11T21:57:29.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:28.977+0000 7fcb83fff640 1 --2- 172.21.15.1:0/3625662214 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fcb840a2e00 0x7fcb840a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:29.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:28.977+0000 7fcb9affd640 1 --2- 172.21.15.1:0/3625662214 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fcb840a2e00 0x7fcb840a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:29.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:28.977+0000 7fcb83fff640 1 -- 172.21.15.1:0/3625662214 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7fcb940cf990 con 0x7fcb9c1082a0 2024-09-11T21:57:29.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:28.977+0000 7fcb9affd640 1 --2- 172.21.15.1:0/3625662214 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fcb840a2e00 0x7fcb840a52c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c005fd0 tx=0x7fcb8c009760 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:29.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:28.981+0000 7fcb83fff640 1 -- 172.21.15.1:0/3625662214 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcb9409c440 con 0x7fcb9c1082a0 2024-09-11T21:57:29.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:29.190+0000 7fcba13a5640 1 -- 172.21.15.1:0/3625662214 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcb9c10a2b0 con 0x7fcb9c1082a0 2024-09-11T21:57:29.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:28 smithi001 ceph-mon[30935]: from='client.15414 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T21:57:29.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:28 smithi001 ceph-mon[30935]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 76 KiB/s wr, 50 op/s 2024-09-11T21:57:29.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:29.191+0000 7fcb83fff640 1 -- 172.21.15.1:0/3625662214 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11949 (secure 0 0 0) 0x7fcb9409bd40 con 0x7fcb9c1082a0 2024-09-11T21:57:29.192 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:57:29.192 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":28,"btime":"2024-09-11T21:57:12:512925+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/421542832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":421542832},{"type":"v1","addr":"172.21.15.1:6835","nonce":421542832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2836864406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2836864406},{"type":"v1","addr":"172.21.15.1:6837","nonce":2836864406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T21:55:20.791263+0000","modified":"2024-09-11T21:57:12.462736+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24484,"mds_2":24467,"mds_3":24466,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14595":{"gid":14595,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/407834672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":407834672},{"type":"v1","addr":"172.21.15.1:6839","nonce":407834672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/1068827678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":1068827678},{"type":"v1","addr":"172.21.15.1:6841","nonce":1068827678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/872410024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":872410024},{"type":"v1","addr":"172.21.15.161:6833","nonce":872410024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/1648944570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1648944570},{"type":"v1","addr":"172.21.15.53:6833","nonce":1648944570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/2126933993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2126933993},{"type":"v1","addr":"172.21.15.53:6835","nonce":2126933993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.161:6835/828381592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":828381592},{"type":"v1","addr":"172.21.15.161:6835","nonce":828381592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.161:6837/1799318656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1799318656},{"type":"v1","addr":"172.21.15.161:6837","nonce":1799318656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.53:6837/3457131830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3457131830},{"type":"v1","addr":"172.21.15.53:6837","nonce":3457131830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/455365383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":455365383},{"type":"v1","addr":"172.21.15.161:6839","nonce":455365383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.53:6839/1378584447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1378584447},{"type":"v1","addr":"172.21.15.53:6839","nonce":1378584447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24460,24466,24467,24484,24494]},"id":1}]} 2024-09-11T21:57:29.192 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 28 2024-09-11T21:57:29.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:29.193+0000 7fcba13a5640 1 -- 172.21.15.1:0/3625662214 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fcb840a2e00 msgr2=0x7fcb840a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:29.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:29.193+0000 7fcba13a5640 1 --2- 172.21.15.1:0/3625662214 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fcb840a2e00 0x7fcb840a52c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c005fd0 tx=0x7fcb8c009760 comp rx=0 tx=0).stop 2024-09-11T21:57:29.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:29.193+0000 7fcba13a5640 1 -- 172.21.15.1:0/3625662214 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb9c1082a0 msgr2=0x7fcb9c101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:29.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:29.193+0000 7fcba13a5640 1 --2- 172.21.15.1:0/3625662214 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb9c1082a0 0x7fcb9c101410 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fcb9400e970 tx=0x7fcb9400ed70 comp rx=0 tx=0).stop 2024-09-11T21:57:29.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:29.193+0000 7fcba13a5640 1 -- 172.21.15.1:0/3625662214 shutdown_connections 2024-09-11T21:57:29.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:29.193+0000 7fcba13a5640 1 --2- 172.21.15.1:0/3625662214 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fcb840a2e00 0x7fcb840a52c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:29.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:29.193+0000 7fcba13a5640 1 --2- 172.21.15.1:0/3625662214 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb9c1082a0 0x7fcb9c101410 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:29.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:29.193+0000 7fcba13a5640 1 --2- 172.21.15.1:0/3625662214 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcb9c1036e0 0x7fcb9c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:29.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:29.193+0000 7fcba13a5640 1 --2- 172.21.15.1:0/3625662214 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb9c102d30 0x7fcb9c100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:29.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:29.193+0000 7fcba13a5640 1 -- 172.21.15.1:0/3625662214 >> 172.21.15.1:0/3625662214 conn(0x7fcb9c0fc7d0 msgr2=0x7fcb9c105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:29.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:29.193+0000 7fcba13a5640 1 -- 172.21.15.1:0/3625662214 shutdown_connections 2024-09-11T21:57:29.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:29.193+0000 7fcba13a5640 1 -- 172.21.15.1:0/3625662214 wait complete. 2024-09-11T21:57:29.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:28 smithi053 ceph-mon[34908]: from='client.15414 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T21:57:29.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:28 smithi053 ceph-mon[34908]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 76 KiB/s wr, 50 op/s 2024-09-11T21:57:29.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:28 smithi161 ceph-mon[35615]: from='client.15414 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T21:57:29.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:28 smithi161 ceph-mon[35615]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 76 KiB/s wr, 50 op/s 2024-09-11T21:57:29.883 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T21:57:30.144 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:57:30.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:29 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/3625662214' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:57:30.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:29 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/3625662214' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:57:30.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:29 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/3625662214' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:57:30.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:30.631+0000 7f007d97f640 1 -- 172.21.15.1:0/1960291320 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0078100a80 msgr2=0x7f0078100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:30.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:30.631+0000 7f007d97f640 1 --2- 172.21.15.1:0/1960291320 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0078100a80 0x7f0078100ee0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0060009930 tx=0x7f006002a420 comp rx=0 tx=0).stop 2024-09-11T21:57:30.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:30.631+0000 7f007d97f640 1 -- 172.21.15.1:0/1960291320 shutdown_connections 2024-09-11T21:57:30.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:30.631+0000 7f007d97f640 1 --2- 172.21.15.1:0/1960291320 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0078101420 0x7f007810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:30.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:30.631+0000 7f007d97f640 1 --2- 172.21.15.1:0/1960291320 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0078100a80 0x7f0078100ee0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:30.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:30.631+0000 7f007d97f640 1 --2- 172.21.15.1:0/1960291320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0078106a80 0x7f0078106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:30.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:30.631+0000 7f007d97f640 1 -- 172.21.15.1:0/1960291320 >> 172.21.15.1:0/1960291320 conn(0x7f00780fc7a0 msgr2=0x7f00780febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:30.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:30.631+0000 7f007d97f640 1 -- 172.21.15.1:0/1960291320 shutdown_connections 2024-09-11T21:57:30.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:30.631+0000 7f007d97f640 1 -- 172.21.15.1:0/1960291320 wait complete. 2024-09-11T21:57:30.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:30.632+0000 7f007d97f640 1 Processor -- start 2024-09-11T21:57:30.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:30.632+0000 7f007d97f640 1 -- start start 2024-09-11T21:57:30.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:30.632+0000 7f007d97f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0078100a80 0x7f0078199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:30.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:30.632+0000 7f007d97f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0078101420 0x7f0078199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:30.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:30.632+0000 7f007d97f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0078106a80 0x7f00781a4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:30.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:30.632+0000 7f007d97f640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0078110e80 con 0x7f0078100a80 2024-09-11T21:57:30.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:30.632+0000 7f007d97f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0078110d00 con 0x7f0078101420 2024-09-11T21:57:30.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:30.632+0000 7f007d97f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0078111000 con 0x7f0078106a80 2024-09-11T21:57:30.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:30.633+0000 7f0076ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0078100a80 0x7f0078199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:30.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:30.633+0000 7f0076ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0078100a80 0x7f0078199310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56818/0 (socket says 172.21.15.1:56818) 2024-09-11T21:57:30.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:30.633+0000 7f0076ffd640 1 -- 172.21.15.1:0/1956561722 learned_addr learned my addr 172.21.15.1:0/1956561722 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:30.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:30.633+0000 7f00777fe640 1 --2- 172.21.15.1:0/1956561722 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0078106a80 0x7f00781a4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:30.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:30.633+0000 7f006ffff640 1 --2- 172.21.15.1:0/1956561722 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0078101420 0x7f0078199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:30.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:30.633+0000 7f0076ffd640 1 -- 172.21.15.1:0/1956561722 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0078106a80 msgr2=0x7f00781a4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:30.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:30.633+0000 7f0076ffd640 1 --2- 172.21.15.1:0/1956561722 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0078106a80 0x7f00781a4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:30.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:30.633+0000 7f0076ffd640 1 -- 172.21.15.1:0/1956561722 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0078101420 msgr2=0x7f0078199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:30.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:30.633+0000 7f0076ffd640 1 --2- 172.21.15.1:0/1956561722 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0078101420 0x7f0078199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:30.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:30.633+0000 7f0076ffd640 1 -- 172.21.15.1:0/1956561722 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00781a48d0 con 0x7f0078100a80 2024-09-11T21:57:30.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:30.633+0000 7f0076ffd640 1 --2- 172.21.15.1:0/1956561722 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0078100a80 0x7f0078199310 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f006800d950 tx=0x7f006800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:30.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:30.633+0000 7f007c97d640 1 -- 172.21.15.1:0/1956561722 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0068017070 con 0x7f0078100a80 2024-09-11T21:57:30.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:30.633+0000 7f007c97d640 1 -- 172.21.15.1:0/1956561722 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0068009c30 con 0x7f0078100a80 2024-09-11T21:57:30.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:30.633+0000 7f007d97f640 1 -- 172.21.15.1:0/1956561722 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00781a4aa0 con 0x7f0078100a80 2024-09-11T21:57:30.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:30.633+0000 7f007c97d640 1 -- 172.21.15.1:0/1956561722 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f006800f6b0 con 0x7f0078100a80 2024-09-11T21:57:30.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:30.633+0000 7f007d97f640 1 -- 172.21.15.1:0/1956561722 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00781a4fb0 con 0x7f0078100a80 2024-09-11T21:57:30.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:30.634+0000 7f007d97f640 1 -- 172.21.15.1:0/1956561722 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0078100ee0 con 0x7f0078100a80 2024-09-11T21:57:30.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:30.636+0000 7f007c97d640 1 -- 172.21.15.1:0/1956561722 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 24) ==== 138008+0+0 (secure 0 0 0) 0x7f00680109d0 con 0x7f0078100a80 2024-09-11T21:57:30.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:30.637+0000 7f007c97d640 1 --2- 172.21.15.1:0/1956561722 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f00640a2e00 0x7f00640a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:30.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:30.637+0000 7f006ffff640 1 --2- 172.21.15.1:0/1956561722 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f00640a2e00 0x7f00640a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:30.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:30.637+0000 7f007c97d640 1 -- 172.21.15.1:0/1956561722 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7f00680cf140 con 0x7f0078100a80 2024-09-11T21:57:30.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:30.638+0000 7f006ffff640 1 --2- 172.21.15.1:0/1956561722 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f00640a2e00 0x7f00640a52c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0060000c00 tx=0x7f0060005a20 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:30.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:30.640+0000 7f007c97d640 1 -- 172.21.15.1:0/1956561722 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0068009de0 con 0x7f0078100a80 2024-09-11T21:57:30.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:30.855+0000 7f007d97f640 1 -- 172.21.15.1:0/1956561722 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f007806ad80 con 0x7f0078100a80 2024-09-11T21:57:30.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:30.855+0000 7f007c97d640 1 -- 172.21.15.1:0/1956561722 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11949 (secure 0 0 0) 0x7f006809bc10 con 0x7f0078100a80 2024-09-11T21:57:30.856 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:57:30.856 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":28,"btime":"2024-09-11T21:57:12:512925+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/421542832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":421542832},{"type":"v1","addr":"172.21.15.1:6835","nonce":421542832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2836864406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2836864406},{"type":"v1","addr":"172.21.15.1:6837","nonce":2836864406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T21:55:20.791263+0000","modified":"2024-09-11T21:57:12.462736+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24484,"mds_2":24467,"mds_3":24466,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14595":{"gid":14595,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/407834672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":407834672},{"type":"v1","addr":"172.21.15.1:6839","nonce":407834672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/1068827678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":1068827678},{"type":"v1","addr":"172.21.15.1:6841","nonce":1068827678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/872410024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":872410024},{"type":"v1","addr":"172.21.15.161:6833","nonce":872410024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/1648944570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1648944570},{"type":"v1","addr":"172.21.15.53:6833","nonce":1648944570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/2126933993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2126933993},{"type":"v1","addr":"172.21.15.53:6835","nonce":2126933993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.161:6835/828381592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":828381592},{"type":"v1","addr":"172.21.15.161:6835","nonce":828381592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.161:6837/1799318656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1799318656},{"type":"v1","addr":"172.21.15.161:6837","nonce":1799318656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.53:6837/3457131830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3457131830},{"type":"v1","addr":"172.21.15.53:6837","nonce":3457131830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/455365383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":455365383},{"type":"v1","addr":"172.21.15.161:6839","nonce":455365383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.53:6839/1378584447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1378584447},{"type":"v1","addr":"172.21.15.53:6839","nonce":1378584447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24460,24466,24467,24484,24494]},"id":1}]} 2024-09-11T21:57:30.857 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 28 2024-09-11T21:57:30.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:30.858+0000 7f007d97f640 1 -- 172.21.15.1:0/1956561722 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f00640a2e00 msgr2=0x7f00640a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:30.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:30.858+0000 7f007d97f640 1 --2- 172.21.15.1:0/1956561722 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f00640a2e00 0x7f00640a52c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0060000c00 tx=0x7f0060005a20 comp rx=0 tx=0).stop 2024-09-11T21:57:30.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:30.858+0000 7f007d97f640 1 -- 172.21.15.1:0/1956561722 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0078100a80 msgr2=0x7f0078199310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:30.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:30.858+0000 7f007d97f640 1 --2- 172.21.15.1:0/1956561722 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0078100a80 0x7f0078199310 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f006800d950 tx=0x7f006800de10 comp rx=0 tx=0).stop 2024-09-11T21:57:30.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:30.858+0000 7f007d97f640 1 -- 172.21.15.1:0/1956561722 shutdown_connections 2024-09-11T21:57:30.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:30.858+0000 7f007d97f640 1 --2- 172.21.15.1:0/1956561722 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f00640a2e00 0x7f00640a52c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:30.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:30.858+0000 7f007d97f640 1 --2- 172.21.15.1:0/1956561722 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0078106a80 0x7f00781a4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:30.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:30.858+0000 7f007d97f640 1 --2- 172.21.15.1:0/1956561722 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0078101420 0x7f0078199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:30.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:30.858+0000 7f007d97f640 1 --2- 172.21.15.1:0/1956561722 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0078100a80 0x7f0078199310 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:30.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:30.858+0000 7f007d97f640 1 -- 172.21.15.1:0/1956561722 >> 172.21.15.1:0/1956561722 conn(0x7f00780fc7a0 msgr2=0x7f00780fd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:30.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:30.858+0000 7f007d97f640 1 -- 172.21.15.1:0/1956561722 shutdown_connections 2024-09-11T21:57:30.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:30.858+0000 7f007d97f640 1 -- 172.21.15.1:0/1956561722 wait complete. 2024-09-11T21:57:31.169 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:30 smithi001 ceph-mon[30935]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 76 KiB/s wr, 51 op/s 2024-09-11T21:57:31.169 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:30 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/1956561722' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:57:31.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:30 smithi053 ceph-mon[34908]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 76 KiB/s wr, 51 op/s 2024-09-11T21:57:31.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:30 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/1956561722' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:57:31.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:30 smithi161 ceph-mon[35615]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 76 KiB/s wr, 51 op/s 2024-09-11T21:57:31.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:30 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/1956561722' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:57:31.493 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T21:57:31.751 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:57:32.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:32.197+0000 7f4024840640 1 -- 172.21.15.1:0/2613753073 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4020108290 msgr2=0x7f402010ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:32.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:32.197+0000 7f4024840640 1 --2- 172.21.15.1:0/2613753073 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4020108290 0x7f402010ec30 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f4018009cd0 tx=0x7f401802a8e0 comp rx=0 tx=0).stop 2024-09-11T21:57:32.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:32.197+0000 7f4024840640 1 -- 172.21.15.1:0/2613753073 shutdown_connections 2024-09-11T21:57:32.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:32.197+0000 7f4024840640 1 --2- 172.21.15.1:0/2613753073 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4020108290 0x7f402010ec30 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:32.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:32.197+0000 7f4024840640 1 --2- 172.21.15.1:0/2613753073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4020103740 0x7f4020107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:32.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:32.197+0000 7f4024840640 1 --2- 172.21.15.1:0/2613753073 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4020102d90 0x7f4020103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:32.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:32.197+0000 7f4024840640 1 -- 172.21.15.1:0/2613753073 >> 172.21.15.1:0/2613753073 conn(0x7f40200fc7d0 msgr2=0x7f40200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:32.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:32.197+0000 7f4024840640 1 -- 172.21.15.1:0/2613753073 shutdown_connections 2024-09-11T21:57:32.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:32.197+0000 7f4024840640 1 -- 172.21.15.1:0/2613753073 wait complete. 2024-09-11T21:57:32.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:32.198+0000 7f4024840640 1 Processor -- start 2024-09-11T21:57:32.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:32.198+0000 7f4024840640 1 -- start start 2024-09-11T21:57:32.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:32.198+0000 7f4024840640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4020102d90 0x7f4020198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:32.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:32.198+0000 7f4024840640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4020103740 0x7f4020198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:32.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:32.198+0000 7f4024840640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4020108290 0x7f402019ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:32.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:32.198+0000 7f4024840640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4020112090 con 0x7f4020102d90 2024-09-11T21:57:32.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:32.198+0000 7f4024840640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4020111f10 con 0x7f4020103740 2024-09-11T21:57:32.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:32.198+0000 7f4024840640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4020112210 con 0x7f4020108290 2024-09-11T21:57:32.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:32.198+0000 7f401e59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4020102d90 0x7f4020198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:32.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:32.198+0000 7f401e59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4020102d90 0x7f4020198550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56834/0 (socket says 172.21.15.1:56834) 2024-09-11T21:57:32.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:32.198+0000 7f401e59c640 1 -- 172.21.15.1:0/537745382 learned_addr learned my addr 172.21.15.1:0/537745382 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:32.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:32.199+0000 7f401ed9d640 1 --2- 172.21.15.1:0/537745382 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4020108290 0x7f402019ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:32.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:32.199+0000 7f401e59c640 1 -- 172.21.15.1:0/537745382 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4020108290 msgr2=0x7f402019ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:32.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:32.199+0000 7f401e59c640 1 --2- 172.21.15.1:0/537745382 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4020108290 0x7f402019ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:32.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:32.199+0000 7f401e59c640 1 -- 172.21.15.1:0/537745382 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4020103740 msgr2=0x7f4020198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:32.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:32.199+0000 7f401dd9b640 1 --2- 172.21.15.1:0/537745382 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4020103740 0x7f4020198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:32.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:32.199+0000 7f401e59c640 1 --2- 172.21.15.1:0/537745382 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4020103740 0x7f4020198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:32.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:32.199+0000 7f401e59c640 1 -- 172.21.15.1:0/537745382 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f402019d500 con 0x7f4020102d90 2024-09-11T21:57:32.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:32.199+0000 7f401ed9d640 1 --2- 172.21.15.1:0/537745382 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4020108290 0x7f402019ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:57:32.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:32.199+0000 7f401dd9b640 1 --2- 172.21.15.1:0/537745382 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4020103740 0x7f4020198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T21:57:32.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:32.199+0000 7f401e59c640 1 --2- 172.21.15.1:0/537745382 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4020102d90 0x7f4020198550 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f401000c890 tx=0x7f401000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:32.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:32.199+0000 7f400f7fe640 1 -- 172.21.15.1:0/537745382 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4010004220 con 0x7f4020102d90 2024-09-11T21:57:32.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:32.199+0000 7f400f7fe640 1 -- 172.21.15.1:0/537745382 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40100043c0 con 0x7f4020102d90 2024-09-11T21:57:32.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:32.199+0000 7f400f7fe640 1 -- 172.21.15.1:0/537745382 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40100049e0 con 0x7f4020102d90 2024-09-11T21:57:32.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:32.199+0000 7f4024840640 1 -- 172.21.15.1:0/537745382 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f402019d7f0 con 0x7f4020102d90 2024-09-11T21:57:32.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:32.199+0000 7f4024840640 1 -- 172.21.15.1:0/537745382 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40201a50d0 con 0x7f4020102d90 2024-09-11T21:57:32.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:32.200+0000 7f4024840640 1 -- 172.21.15.1:0/537745382 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4020103170 con 0x7f4020102d90 2024-09-11T21:57:32.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:32.201+0000 7f400f7fe640 1 -- 172.21.15.1:0/537745382 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 24) ==== 138008+0+0 (secure 0 0 0) 0x7f40100079b0 con 0x7f4020102d90 2024-09-11T21:57:32.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:32.202+0000 7f400f7fe640 1 --2- 172.21.15.1:0/537745382 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f40080a2d80 0x7f40080a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:32.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:32.202+0000 7f401dd9b640 1 --2- 172.21.15.1:0/537745382 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f40080a2d80 0x7f40080a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:32.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:32.202+0000 7f400f7fe640 1 -- 172.21.15.1:0/537745382 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7f40100cf760 con 0x7f4020102d90 2024-09-11T21:57:32.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:32.203+0000 7f401dd9b640 1 --2- 172.21.15.1:0/537745382 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f40080a2d80 0x7f40080a5240 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f40200feda0 tx=0x7f4014005cb0 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:32.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:32.206+0000 7f400f7fe640 1 -- 172.21.15.1:0/537745382 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4010014030 con 0x7f4020102d90 2024-09-11T21:57:32.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:32.383+0000 7f4024840640 1 -- 172.21.15.1:0/537745382 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f40201998b0 con 0x7f4020102d90 2024-09-11T21:57:32.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:32.384+0000 7f400f7fe640 1 -- 172.21.15.1:0/537745382 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+23041 (secure 0 0 0) 0x7f401009c1b0 con 0x7f4020102d90 2024-09-11T21:57:32.385 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:57:32.385 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":87,"fsid":"3ffca83c-7087-11ef-bcea-c7b262605968","created":"2024-09-11T21:47:28.868707+0000","modified":"2024-09-11T21:57:13.664420+0000","last_up_change":"2024-09-11T21:54:29.319441+0000","last_in_change":"2024-09-11T21:54:05.532281+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T21:50:41.026490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T21:55:16.961999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T21:55:19.066228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"19e56504-18f1-4234-b2d6-f193ff47d8eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":4012142023},{"type":"v1","addr":"172.21.15.1:6803","nonce":4012142023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":4012142023},{"type":"v1","addr":"172.21.15.1:6805","nonce":4012142023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":4012142023},{"type":"v1","addr":"172.21.15.1:6809","nonce":4012142023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":4012142023},{"type":"v1","addr":"172.21.15.1:6807","nonce":4012142023}]},"public_addr":"172.21.15.1:6803/4012142023","cluster_addr":"172.21.15.1:6805/4012142023","heartbeat_back_addr":"172.21.15.1:6809/4012142023","heartbeat_front_addr":"172.21.15.1:6807/4012142023","state":["exists","up"]},{"osd":1,"uuid":"996c9f0c-c5a4-43b8-b8e0-d712f27c5dfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3039391924},{"type":"v1","addr":"172.21.15.53:6801","nonce":3039391924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3039391924},{"type":"v1","addr":"172.21.15.53:6803","nonce":3039391924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3039391924},{"type":"v1","addr":"172.21.15.53:6807","nonce":3039391924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3039391924},{"type":"v1","addr":"172.21.15.53:6805","nonce":3039391924}]},"public_addr":"172.21.15.53:6801/3039391924","cluster_addr":"172.21.15.53:6803/3039391924","heartbeat_back_addr":"172.21.15.53:6807/3039391924","heartbeat_front_addr":"172.21.15.53:6805/3039391924","state":["exists","up"]},{"osd":2,"uuid":"578d4260-41e1-4645-b9b1-1c5445712c13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1707914885},{"type":"v1","addr":"172.21.15.161:6801","nonce":1707914885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1707914885},{"type":"v1","addr":"172.21.15.161:6803","nonce":1707914885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1707914885},{"type":"v1","addr":"172.21.15.161:6807","nonce":1707914885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1707914885},{"type":"v1","addr":"172.21.15.161:6805","nonce":1707914885}]},"public_addr":"172.21.15.161:6801/1707914885","cluster_addr":"172.21.15.161:6803/1707914885","heartbeat_back_addr":"172.21.15.161:6807/1707914885","heartbeat_front_addr":"172.21.15.161:6805/1707914885","state":["exists","up"]},{"osd":3,"uuid":"73f78766-9662-4aa3-841c-fe2ce654ee9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1541142398},{"type":"v1","addr":"172.21.15.1:6811","nonce":1541142398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1541142398},{"type":"v1","addr":"172.21.15.1:6813","nonce":1541142398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1541142398},{"type":"v1","addr":"172.21.15.1:6817","nonce":1541142398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1541142398},{"type":"v1","addr":"172.21.15.1:6815","nonce":1541142398}]},"public_addr":"172.21.15.1:6811/1541142398","cluster_addr":"172.21.15.1:6813/1541142398","heartbeat_back_addr":"172.21.15.1:6817/1541142398","heartbeat_front_addr":"172.21.15.1:6815/1541142398","state":["exists","up"]},{"osd":4,"uuid":"56c808bb-d4cd-4969-b49e-adef49ecd0f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":3081402681},{"type":"v1","addr":"172.21.15.53:6809","nonce":3081402681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3081402681},{"type":"v1","addr":"172.21.15.53:6811","nonce":3081402681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3081402681},{"type":"v1","addr":"172.21.15.53:6815","nonce":3081402681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3081402681},{"type":"v1","addr":"172.21.15.53:6813","nonce":3081402681}]},"public_addr":"172.21.15.53:6809/3081402681","cluster_addr":"172.21.15.53:6811/3081402681","heartbeat_back_addr":"172.21.15.53:6815/3081402681","heartbeat_front_addr":"172.21.15.53:6813/3081402681","state":["exists","up"]},{"osd":5,"uuid":"7b8266d6-b507-47c9-b4fa-ff16c4a74d35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":1622081327},{"type":"v1","addr":"172.21.15.161:6809","nonce":1622081327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":1622081327},{"type":"v1","addr":"172.21.15.161:6811","nonce":1622081327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":1622081327},{"type":"v1","addr":"172.21.15.161:6815","nonce":1622081327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":1622081327},{"type":"v1","addr":"172.21.15.161:6813","nonce":1622081327}]},"public_addr":"172.21.15.161:6809/1622081327","cluster_addr":"172.21.15.161:6811/1622081327","heartbeat_back_addr":"172.21.15.161:6815/1622081327","heartbeat_front_addr":"172.21.15.161:6813/1622081327","state":["exists","up"]},{"osd":6,"uuid":"3af5925d-a749-4849-a076-80580222cad4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":158434853},{"type":"v1","addr":"172.21.15.1:6819","nonce":158434853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":158434853},{"type":"v1","addr":"172.21.15.1:6821","nonce":158434853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":158434853},{"type":"v1","addr":"172.21.15.1:6825","nonce":158434853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":158434853},{"type":"v1","addr":"172.21.15.1:6823","nonce":158434853}]},"public_addr":"172.21.15.1:6819/158434853","cluster_addr":"172.21.15.1:6821/158434853","heartbeat_back_addr":"172.21.15.1:6825/158434853","heartbeat_front_addr":"172.21.15.1:6823/158434853","state":["exists","up"]},{"osd":7,"uuid":"f8f26c3c-17e9-4e89-8392-289796e0c8a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2132057697},{"type":"v1","addr":"172.21.15.53:6817","nonce":2132057697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2132057697},{"type":"v1","addr":"172.21.15.53:6819","nonce":2132057697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2132057697},{"type":"v1","addr":"172.21.15.53:6823","nonce":2132057697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2132057697},{"type":"v1","addr":"172.21.15.53:6821","nonce":2132057697}]},"public_addr":"172.21.15.53:6817/2132057697","cluster_addr":"172.21.15.53:6819/2132057697","heartbeat_back_addr":"172.21.15.53:6823/2132057697","heartbeat_front_addr":"172.21.15.53:6821/2132057697","state":["exists","up"]},{"osd":8,"uuid":"36ea401b-3c93-4b9d-8880-298d1eb6e30b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1838248669},{"type":"v1","addr":"172.21.15.161:6817","nonce":1838248669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1838248669},{"type":"v1","addr":"172.21.15.161:6819","nonce":1838248669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1838248669},{"type":"v1","addr":"172.21.15.161:6823","nonce":1838248669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1838248669},{"type":"v1","addr":"172.21.15.161:6821","nonce":1838248669}]},"public_addr":"172.21.15.161:6817/1838248669","cluster_addr":"172.21.15.161:6819/1838248669","heartbeat_back_addr":"172.21.15.161:6823/1838248669","heartbeat_front_addr":"172.21.15.161:6821/1838248669","state":["exists","up"]},{"osd":9,"uuid":"4b788d3f-a3bc-4463-bd99-95fdb7a52768","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2806274589},{"type":"v1","addr":"172.21.15.1:6827","nonce":2806274589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2806274589},{"type":"v1","addr":"172.21.15.1:6829","nonce":2806274589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":2806274589},{"type":"v1","addr":"172.21.15.1:6833","nonce":2806274589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2806274589},{"type":"v1","addr":"172.21.15.1:6831","nonce":2806274589}]},"public_addr":"172.21.15.1:6827/2806274589","cluster_addr":"172.21.15.1:6829/2806274589","heartbeat_back_addr":"172.21.15.1:6833/2806274589","heartbeat_front_addr":"172.21.15.1:6831/2806274589","state":["exists","up"]},{"osd":10,"uuid":"70549796-5270-4ae2-ac3f-f61f5de105d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":761923580},{"type":"v1","addr":"172.21.15.53:6825","nonce":761923580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":761923580},{"type":"v1","addr":"172.21.15.53:6827","nonce":761923580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":761923580},{"type":"v1","addr":"172.21.15.53:6831","nonce":761923580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":761923580},{"type":"v1","addr":"172.21.15.53:6829","nonce":761923580}]},"public_addr":"172.21.15.53:6825/761923580","cluster_addr":"172.21.15.53:6827/761923580","heartbeat_back_addr":"172.21.15.53:6831/761923580","heartbeat_front_addr":"172.21.15.53:6829/761923580","state":["exists","up"]},{"osd":11,"uuid":"48f17c4a-370a-493e-956e-bf6ad3dcc594","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1586352190},{"type":"v1","addr":"172.21.15.161:6825","nonce":1586352190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":1586352190},{"type":"v1","addr":"172.21.15.161:6827","nonce":1586352190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":1586352190},{"type":"v1","addr":"172.21.15.161:6831","nonce":1586352190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":1586352190},{"type":"v1","addr":"172.21.15.161:6829","nonce":1586352190}]},"public_addr":"172.21.15.161:6825/1586352190","cluster_addr":"172.21.15.161:6827/1586352190","heartbeat_back_addr":"172.21.15.161:6831/1586352190","heartbeat_front_addr":"172.21.15.161:6829/1586352190","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:49:46.539613+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:50:11.912300+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:50:36.955490+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:51:02.200554+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:51:27.860585+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:51:54.694099+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:52:18.962351+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:52:44.881923+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:53:10.301390+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:53:35.417647+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:54:01.492219+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:54:26.938338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.26","mappings":[{"from":1,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/4286250488":"2024-09-12T21:57:13.664392+0000","172.21.15.1:0/2039944931":"2024-09-12T21:57:13.664392+0000","172.21.15.1:0/3202962495":"2024-09-12T21:48:00.674728+0000","172.21.15.1:6800/2311309446":"2024-09-12T21:48:00.674728+0000","172.21.15.1:6801/3890350886":"2024-09-12T21:57:13.664392+0000","172.21.15.1:6801/2311309446":"2024-09-12T21:48:00.674728+0000","172.21.15.1:0/2599895977":"2024-09-12T21:48:00.674728+0000","172.21.15.1:0/1278877732":"2024-09-12T21:48:24.524913+0000","172.21.15.1:0/276770268":"2024-09-12T21:48:00.674728+0000","172.21.15.1:0/1699984777":"2024-09-12T21:48:24.524913+0000","172.21.15.1:6800/2821761445":"2024-09-12T21:48:24.524913+0000","172.21.15.1:0/2797780151":"2024-09-12T21:57:13.664392+0000","172.21.15.1:6801/2821761445":"2024-09-12T21:48:24.524913+0000","172.21.15.1:0/1749020850":"2024-09-12T21:48:24.524913+0000","172.21.15.1:0/1202232270":"2024-09-12T21:57:13.664392+0000","172.21.15.1:6800/3890350886":"2024-09-12T21:57:13.664392+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T21:57:32.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:32.386+0000 7f4024840640 1 -- 172.21.15.1:0/537745382 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f40080a2d80 msgr2=0x7f40080a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:32.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:32.386+0000 7f4024840640 1 --2- 172.21.15.1:0/537745382 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f40080a2d80 0x7f40080a5240 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f40200feda0 tx=0x7f4014005cb0 comp rx=0 tx=0).stop 2024-09-11T21:57:32.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:32.386+0000 7f4024840640 1 -- 172.21.15.1:0/537745382 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4020102d90 msgr2=0x7f4020198550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:32.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:32.386+0000 7f4024840640 1 --2- 172.21.15.1:0/537745382 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4020102d90 0x7f4020198550 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f401000c890 tx=0x7f401000cd50 comp rx=0 tx=0).stop 2024-09-11T21:57:32.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:32.386+0000 7f4024840640 1 -- 172.21.15.1:0/537745382 shutdown_connections 2024-09-11T21:57:32.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:32.386+0000 7f4024840640 1 --2- 172.21.15.1:0/537745382 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f40080a2d80 0x7f40080a5240 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:32.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:32.386+0000 7f4024840640 1 --2- 172.21.15.1:0/537745382 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4020108290 0x7f402019ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:32.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:32.386+0000 7f4024840640 1 --2- 172.21.15.1:0/537745382 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4020103740 0x7f4020198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:32.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:32.386+0000 7f4024840640 1 --2- 172.21.15.1:0/537745382 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4020102d90 0x7f4020198550 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:32.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:32.386+0000 7f4024840640 1 -- 172.21.15.1:0/537745382 >> 172.21.15.1:0/537745382 conn(0x7f40200fc7d0 msgr2=0x7f4020106350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:32.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:32.387+0000 7f4024840640 1 -- 172.21.15.1:0/537745382 shutdown_connections 2024-09-11T21:57:32.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:32.387+0000 7f4024840640 1 -- 172.21.15.1:0/537745382 wait complete. 2024-09-11T21:57:33.032 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-09-11T21:57:33.297 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:57:33.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:32 smithi053 ceph-mon[34908]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 76 KiB/s wr, 54 op/s 2024-09-11T21:57:33.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:32 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/537745382' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T21:57:33.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:32 smithi161 ceph-mon[35615]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 76 KiB/s wr, 54 op/s 2024-09-11T21:57:33.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:32 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/537745382' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T21:57:33.419 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:32 smithi001 ceph-mon[30935]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 76 KiB/s wr, 54 op/s 2024-09-11T21:57:33.419 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:32 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/537745382' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T21:57:34.501 INFO:teuthology.orchestra.run.smithi001.stdout:{"run_dir":"/var/run/ceph"} 2024-09-11T21:57:34.502 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-09-11T21:57:34.502 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 481 2024-09-11T21:57:34.502 DEBUG:tasks.quiescer:Yielding 2024-09-11T21:57:34.502 INFO:teuthology.run_tasks:Running task workunit... 2024-09-11T21:57:34.516 INFO:tasks.workunit:Pulling workunits from ref 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 2024-09-11T21:57:34.516 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-11T21:57:34.517 INFO:tasks.workunit:timeout=3h 2024-09-11T21:57:34.517 INFO:tasks.workunit:cleanup=True 2024-09-11T21:57:34.517 DEBUG:teuthology.orchestra.run.smithi001:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-11T21:57:34.517 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T21:57:34.534 INFO:teuthology.orchestra.run.smithi001.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-11T21:57:34.535 INFO:teuthology.orchestra.run.smithi001.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-09-11T21:57:34.535 INFO:teuthology.orchestra.run.smithi001.stdout:Device: 78h/120d Inode: 1099511628283 Links: 2 2024-09-11T21:57:34.535 INFO:teuthology.orchestra.run.smithi001.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T21:57:34.535 INFO:teuthology.orchestra.run.smithi001.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-11T21:57:34.535 INFO:teuthology.orchestra.run.smithi001.stdout:Access: 2024-09-11 21:55:39.209114692 +0000 2024-09-11T21:57:34.535 INFO:teuthology.orchestra.run.smithi001.stdout:Modify: 2024-09-11 21:55:39.209114692 +0000 2024-09-11T21:57:34.535 INFO:teuthology.orchestra.run.smithi001.stdout:Change: 2024-09-11 21:57:11.770466201 +0000 2024-09-11T21:57:34.535 INFO:teuthology.orchestra.run.smithi001.stdout: Birth: 2024-09-11 21:55:39.209114692 +0000 2024-09-11T21:57:34.535 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-11T21:57:34.535 DEBUG:teuthology.orchestra.run.smithi001:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-11T21:57:34.611 DEBUG:teuthology.orchestra.run.smithi001:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 2024-09-11T21:57:34.667 INFO:tasks.workunit.client.0.smithi001.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-11T21:57:34.780 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:57:35.155 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:34 smithi001 ceph-mon[30935]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 76 KiB/s wr, 40 op/s 2024-09-11T21:57:35.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:35.234+0000 7fd1700f8640 1 -- 172.21.15.1:0/3073599025 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd16810c2f0 msgr2=0x7fd16810e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:35.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:35.234+0000 7fd1700f8640 1 --2- 172.21.15.1:0/3073599025 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd16810c2f0 0x7fd16810e790 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fd160009900 tx=0x7fd16002a410 comp rx=0 tx=0).stop 2024-09-11T21:57:35.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:35.234+0000 7fd1700f8640 1 -- 172.21.15.1:0/3073599025 shutdown_connections 2024-09-11T21:57:35.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:35.234+0000 7fd1700f8640 1 --2- 172.21.15.1:0/3073599025 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd16810c2f0 0x7fd16810e790 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:35.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:35.234+0000 7fd1700f8640 1 --2- 172.21.15.1:0/3073599025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1681037a0 0x7fd16810bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:35.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:35.234+0000 7fd1700f8640 1 --2- 172.21.15.1:0/3073599025 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd168102df0 0x7fd1681031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:35.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:35.234+0000 7fd1700f8640 1 -- 172.21.15.1:0/3073599025 >> 172.21.15.1:0/3073599025 conn(0x7fd1680fc7d0 msgr2=0x7fd1680febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:35.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:35.234+0000 7fd1700f8640 1 -- 172.21.15.1:0/3073599025 shutdown_connections 2024-09-11T21:57:35.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:35.234+0000 7fd1700f8640 1 -- 172.21.15.1:0/3073599025 wait complete. 2024-09-11T21:57:35.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:35.235+0000 7fd1700f8640 1 Processor -- start 2024-09-11T21:57:35.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:35.235+0000 7fd1700f8640 1 -- start start 2024-09-11T21:57:35.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:35.235+0000 7fd1700f8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd168102df0 0x7fd168100a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:35.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:35.235+0000 7fd1700f8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1681037a0 0x7fd168100f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:35.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:35.235+0000 7fd1700f8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd16810c2f0 0x7fd1681014d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:35.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:35.235+0000 7fd1700f8640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd168112110 con 0x7fd16810c2f0 2024-09-11T21:57:35.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:35.235+0000 7fd1700f8640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd168111f90 con 0x7fd168102df0 2024-09-11T21:57:35.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:35.235+0000 7fd1700f8640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd168112290 con 0x7fd1681037a0 2024-09-11T21:57:35.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:35.235+0000 7fd16e695640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd16810c2f0 0x7fd1681014d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:35.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:35.235+0000 7fd16e695640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd16810c2f0 0x7fd1681014d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56858/0 (socket says 172.21.15.1:56858) 2024-09-11T21:57:35.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:35.235+0000 7fd16de94640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd168102df0 0x7fd168100a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:35.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:35.235+0000 7fd16e695640 1 -- 172.21.15.1:0/2731183550 learned_addr learned my addr 172.21.15.1:0/2731183550 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:35.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:35.235+0000 7fd16d693640 1 --2- 172.21.15.1:0/2731183550 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1681037a0 0x7fd168100f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:35.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:35.236+0000 7fd16de94640 1 -- 172.21.15.1:0/2731183550 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1681037a0 msgr2=0x7fd168100f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:35.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:35.236+0000 7fd16de94640 1 --2- 172.21.15.1:0/2731183550 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1681037a0 0x7fd168100f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:35.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:35.236+0000 7fd16de94640 1 -- 172.21.15.1:0/2731183550 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd16810c2f0 msgr2=0x7fd1681014d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:35.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:35.236+0000 7fd16de94640 1 --2- 172.21.15.1:0/2731183550 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd16810c2f0 0x7fd1681014d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:35.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:35.236+0000 7fd16de94640 1 -- 172.21.15.1:0/2731183550 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1681a6a70 con 0x7fd168102df0 2024-09-11T21:57:35.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:35.236+0000 7fd16e695640 1 --2- 172.21.15.1:0/2731183550 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd16810c2f0 0x7fd1681014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T21:57:35.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:35.236+0000 7fd16de94640 1 --2- 172.21.15.1:0/2731183550 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd168102df0 0x7fd168100a50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd158009fd0 tx=0x7fd15800ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:35.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:35.236+0000 7fd166ffd640 1 -- 172.21.15.1:0/2731183550 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd158004b20 con 0x7fd168102df0 2024-09-11T21:57:35.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:35.236+0000 7fd166ffd640 1 -- 172.21.15.1:0/2731183550 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd158004cc0 con 0x7fd168102df0 2024-09-11T21:57:35.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:35.236+0000 7fd1700f8640 1 -- 172.21.15.1:0/2731183550 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1681a6d60 con 0x7fd168102df0 2024-09-11T21:57:35.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:35.236+0000 7fd166ffd640 1 -- 172.21.15.1:0/2731183550 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1580107b0 con 0x7fd168102df0 2024-09-11T21:57:35.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:35.236+0000 7fd1700f8640 1 -- 172.21.15.1:0/2731183550 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1681a7240 con 0x7fd168102df0 2024-09-11T21:57:35.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:35.238+0000 7fd1700f8640 1 -- 172.21.15.1:0/2731183550 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1681031d0 con 0x7fd168102df0 2024-09-11T21:57:35.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:35.239+0000 7fd166ffd640 1 -- 172.21.15.1:0/2731183550 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 24) ==== 138008+0+0 (secure 0 0 0) 0x7fd158005020 con 0x7fd168102df0 2024-09-11T21:57:35.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:35.239+0000 7fd166ffd640 1 --2- 172.21.15.1:0/2731183550 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fd1500a2e00 0x7fd1500a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:35.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:35.239+0000 7fd166ffd640 1 -- 172.21.15.1:0/2731183550 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7fd1580cfcd0 con 0x7fd168102df0 2024-09-11T21:57:35.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:35.239+0000 7fd16d693640 1 --2- 172.21.15.1:0/2731183550 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fd1500a2e00 0x7fd1500a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:35.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:35.240+0000 7fd16d693640 1 --2- 172.21.15.1:0/2731183550 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fd1500a2e00 0x7fd1500a52c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd1681024e0 tx=0x7fd15c005f90 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:35.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:35.244+0000 7fd166ffd640 1 -- 172.21.15.1:0/2731183550 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd15809c780 con 0x7fd168102df0 2024-09-11T21:57:35.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:34 smithi053 ceph-mon[34908]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 76 KiB/s wr, 40 op/s 2024-09-11T21:57:35.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:34 smithi161 ceph-mon[35615]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 76 KiB/s wr, 40 op/s 2024-09-11T21:57:35.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:35.457+0000 7fd1700f8640 1 -- 172.21.15.1:0/2731183550 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd1681a74d0 con 0x7fd168102df0 2024-09-11T21:57:35.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:35.458+0000 7fd166ffd640 1 -- 172.21.15.1:0/2731183550 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11949 (secure 0 0 0) 0x7fd15809c080 con 0x7fd168102df0 2024-09-11T21:57:35.459 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:57:35.459 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":28,"btime":"2024-09-11T21:57:12:512925+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/421542832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":421542832},{"type":"v1","addr":"172.21.15.1:6835","nonce":421542832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2836864406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2836864406},{"type":"v1","addr":"172.21.15.1:6837","nonce":2836864406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T21:55:20.791263+0000","modified":"2024-09-11T21:57:12.462736+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24484,"mds_2":24467,"mds_3":24466,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14595":{"gid":14595,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/407834672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":407834672},{"type":"v1","addr":"172.21.15.1:6839","nonce":407834672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/1068827678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":1068827678},{"type":"v1","addr":"172.21.15.1:6841","nonce":1068827678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/872410024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":872410024},{"type":"v1","addr":"172.21.15.161:6833","nonce":872410024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/1648944570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1648944570},{"type":"v1","addr":"172.21.15.53:6833","nonce":1648944570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/2126933993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2126933993},{"type":"v1","addr":"172.21.15.53:6835","nonce":2126933993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.161:6835/828381592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":828381592},{"type":"v1","addr":"172.21.15.161:6835","nonce":828381592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.161:6837/1799318656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1799318656},{"type":"v1","addr":"172.21.15.161:6837","nonce":1799318656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.53:6837/3457131830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3457131830},{"type":"v1","addr":"172.21.15.53:6837","nonce":3457131830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/455365383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":455365383},{"type":"v1","addr":"172.21.15.161:6839","nonce":455365383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.53:6839/1378584447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1378584447},{"type":"v1","addr":"172.21.15.53:6839","nonce":1378584447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24460,24466,24467,24484,24494]},"id":1}]} 2024-09-11T21:57:35.459 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 28 2024-09-11T21:57:35.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:35.460+0000 7fd1700f8640 1 -- 172.21.15.1:0/2731183550 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fd1500a2e00 msgr2=0x7fd1500a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:35.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:35.460+0000 7fd1700f8640 1 --2- 172.21.15.1:0/2731183550 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fd1500a2e00 0x7fd1500a52c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd1681024e0 tx=0x7fd15c005f90 comp rx=0 tx=0).stop 2024-09-11T21:57:35.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:35.460+0000 7fd1700f8640 1 -- 172.21.15.1:0/2731183550 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd168102df0 msgr2=0x7fd168100a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:35.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:35.460+0000 7fd1700f8640 1 --2- 172.21.15.1:0/2731183550 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd168102df0 0x7fd168100a50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd158009fd0 tx=0x7fd15800ef90 comp rx=0 tx=0).stop 2024-09-11T21:57:35.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:35.460+0000 7fd1700f8640 1 -- 172.21.15.1:0/2731183550 shutdown_connections 2024-09-11T21:57:35.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:35.460+0000 7fd1700f8640 1 --2- 172.21.15.1:0/2731183550 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fd1500a2e00 0x7fd1500a52c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:35.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:35.460+0000 7fd1700f8640 1 --2- 172.21.15.1:0/2731183550 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd16810c2f0 0x7fd1681014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:35.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:35.460+0000 7fd1700f8640 1 --2- 172.21.15.1:0/2731183550 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1681037a0 0x7fd168100f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:35.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:35.460+0000 7fd1700f8640 1 --2- 172.21.15.1:0/2731183550 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd168102df0 0x7fd168100a50 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:35.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:35.460+0000 7fd1700f8640 1 -- 172.21.15.1:0/2731183550 >> 172.21.15.1:0/2731183550 conn(0x7fd1680fc7d0 msgr2=0x7fd1680fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:35.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:35.461+0000 7fd1700f8640 1 -- 172.21.15.1:0/2731183550 shutdown_connections 2024-09-11T21:57:35.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:35.461+0000 7fd1700f8640 1 -- 172.21.15.1:0/2731183550 wait complete. 2024-09-11T21:57:36.141 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:35 smithi001 ceph-mon[30935]: scrub summary: idle 2024-09-11T21:57:36.141 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:35 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/2731183550' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:57:36.174 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 306, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': True}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-11T21:55:20.791263+0000', 'modified': '2024-09-11T21:57:12.462736+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24494, 'mds_1': 24484, 'mds_2': 24467, 'mds_3': 24466, 'mds_4': 24460}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14595': {'gid': 14595, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.1:6839/407834672', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6838', 'nonce': 407834672}, {'type': 'v1', 'addr': '172.21.15.1:6839', 'nonce': 407834672}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14601': {'gid': 14601, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.1:6841/1068827678', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6840', 'nonce': 1068827678}, {'type': 'v1', 'addr': '172.21.15.1:6841', 'nonce': 1068827678}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24430': {'gid': 24430, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.161:6833/872410024', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6832', 'nonce': 872410024}, {'type': 'v1', 'addr': '172.21.15.161:6833', 'nonce': 872410024}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24431': {'gid': 24431, 'name': 'k', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.53:6833/1648944570', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6832', 'nonce': 1648944570}, {'type': 'v1', 'addr': '172.21.15.53:6833', 'nonce': 1648944570}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24449': {'gid': 24449, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.53:6835/2126933993', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6834', 'nonce': 2126933993}, {'type': 'v1', 'addr': '172.21.15.53:6835', 'nonce': 2126933993}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24460': {'gid': 24460, 'name': 'f', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.161:6835/828381592', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6834', 'nonce': 828381592}, {'type': 'v1', 'addr': '172.21.15.161:6835', 'nonce': 828381592}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24466': {'gid': 24466, 'name': 'c', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.161:6837/1799318656', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6836', 'nonce': 1799318656}, {'type': 'v1', 'addr': '172.21.15.161:6837', 'nonce': 1799318656}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24467': {'gid': 24467, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.53:6837/3457131830', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6836', 'nonce': 3457131830}, {'type': 'v1', 'addr': '172.21.15.53:6837', 'nonce': 3457131830}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24484': {'gid': 24484, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.161:6839/455365383', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6838', 'nonce': 455365383}, {'type': 'v1', 'addr': '172.21.15.161:6839', 'nonce': 455365383}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24494': {'gid': 24494, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.53:6839/1378584447', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6838', 'nonce': 1378584447}, {'type': 'v1', 'addr': '172.21.15.53:6839', 'nonce': 1378584447}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24494, 'qdb_cluster': [24460, 24466, 24467, 24484, 24494]} 2024-09-11T21:57:36.175 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-11T21:57:36.175 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.f status 2024-09-11T21:57:36.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:35 smithi161 ceph-mon[35615]: scrub summary: idle 2024-09-11T21:57:36.243 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:35 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/2731183550' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:57:36.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:35 smithi053 ceph-mon[34908]: scrub summary: idle 2024-09-11T21:57:36.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:35 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/2731183550' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:57:36.386 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.383+0000 7fc5da6a6640 1 -- 172.21.15.1:0/1262734520 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5d4101340 msgr2=0x7fc5d410d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:36.386 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.383+0000 7fc5da6a6640 1 --2- 172.21.15.1:0/1262734520 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5d4101340 0x7fc5d410d870 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc009cd0 tx=0x7fc5cc02a8e0 comp rx=0 tx=0).stop 2024-09-11T21:57:36.386 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.383+0000 7fc5da6a6640 1 -- 172.21.15.1:0/1262734520 shutdown_connections 2024-09-11T21:57:36.386 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.383+0000 7fc5da6a6640 1 --2- 172.21.15.1:0/1262734520 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5d4101340 0x7fc5d410d870 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:36.386 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.383+0000 7fc5da6a6640 1 --2- 172.21.15.1:0/1262734520 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc5d41009a0 0x7fc5d4100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:36.386 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.383+0000 7fc5da6a6640 1 --2- 172.21.15.1:0/1262734520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc5d41069b0 0x7fc5d4106d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:36.386 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.383+0000 7fc5da6a6640 1 -- 172.21.15.1:0/1262734520 >> 172.21.15.1:0/1262734520 conn(0x7fc5d40779a0 msgr2=0x7fc5d40fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:36.386 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.383+0000 7fc5da6a6640 1 -- 172.21.15.1:0/1262734520 shutdown_connections 2024-09-11T21:57:36.386 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.383+0000 7fc5da6a6640 1 -- 172.21.15.1:0/1262734520 wait complete. 2024-09-11T21:57:36.386 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.383+0000 7fc5da6a6640 1 Processor -- start 2024-09-11T21:57:36.387 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.383+0000 7fc5da6a6640 1 -- start start 2024-09-11T21:57:36.387 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.384+0000 7fc5da6a6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5d41009a0 0x7fc5d41983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:36.387 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.384+0000 7fc5da6a6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc5d4101340 0x7fc5d41988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:36.387 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.384+0000 7fc5da6a6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc5d41069b0 0x7fc5d419cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:36.387 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.384+0000 7fc5da6a6640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d4110df0 con 0x7fc5d41009a0 2024-09-11T21:57:36.387 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.384+0000 7fc5da6a6640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d4110c70 con 0x7fc5d4101340 2024-09-11T21:57:36.387 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.384+0000 7fc5da6a6640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d4110f70 con 0x7fc5d41069b0 2024-09-11T21:57:36.387 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.384+0000 7fc5d3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5d41009a0 0x7fc5d41983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:36.387 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.384+0000 7fc5d37fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc5d4101340 0x7fc5d41988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:36.387 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.384+0000 7fc5d3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5d41009a0 0x7fc5d41983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56878/0 (socket says 172.21.15.1:56878) 2024-09-11T21:57:36.387 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.384+0000 7fc5d3fff640 1 -- 172.21.15.1:0/1164265505 learned_addr learned my addr 172.21.15.1:0/1164265505 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:36.387 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.384+0000 7fc5d8c43640 1 --2- 172.21.15.1:0/1164265505 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc5d41069b0 0x7fc5d419cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:36.387 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.384+0000 7fc5d37fe640 1 -- 172.21.15.1:0/1164265505 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc5d41069b0 msgr2=0x7fc5d419cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:36.387 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.384+0000 7fc5d37fe640 1 --2- 172.21.15.1:0/1164265505 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc5d41069b0 0x7fc5d419cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:36.388 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.384+0000 7fc5d37fe640 1 -- 172.21.15.1:0/1164265505 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5d41009a0 msgr2=0x7fc5d41983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:36.388 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.384+0000 7fc5d37fe640 1 --2- 172.21.15.1:0/1164265505 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5d41009a0 0x7fc5d41983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:36.388 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.384+0000 7fc5d37fe640 1 -- 172.21.15.1:0/1164265505 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5d419d390 con 0x7fc5d4101340 2024-09-11T21:57:36.388 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.384+0000 7fc5d3fff640 1 --2- 172.21.15.1:0/1164265505 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5d41009a0 0x7fc5d41983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T21:57:36.388 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.385+0000 7fc5d37fe640 1 --2- 172.21.15.1:0/1164265505 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc5d4101340 0x7fc5d41988e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc5c800e9f0 tx=0x7fc5c800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:36.388 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.385+0000 7fc5d17fa640 1 -- 172.21.15.1:0/1164265505 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c800ccc0 con 0x7fc5d4101340 2024-09-11T21:57:36.388 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.385+0000 7fc5d17fa640 1 -- 172.21.15.1:0/1164265505 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5c8005020 con 0x7fc5d4101340 2024-09-11T21:57:36.388 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.385+0000 7fc5da6a6640 1 -- 172.21.15.1:0/1164265505 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5d419d680 con 0x7fc5d4101340 2024-09-11T21:57:36.388 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.385+0000 7fc5d17fa640 1 -- 172.21.15.1:0/1164265505 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c80049e0 con 0x7fc5d4101340 2024-09-11T21:57:36.388 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.385+0000 7fc5da6a6640 1 -- 172.21.15.1:0/1164265505 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5d41a4f10 con 0x7fc5d4101340 2024-09-11T21:57:36.393 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.387+0000 7fc5d17fa640 1 -- 172.21.15.1:0/1164265505 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 24) ==== 138008+0+0 (secure 0 0 0) 0x7fc5c8005560 con 0x7fc5d4101340 2024-09-11T21:57:36.393 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.387+0000 7fc5dd43a740 1 Processor -- start 2024-09-11T21:57:36.393 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.387+0000 7fc5dd43a740 1 -- start start 2024-09-11T21:57:36.393 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.388+0000 7fc5dd43a740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556df7f41af0 0x556df7f41ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:36.393 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.388+0000 7fc5dd43a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556df7f42410 0x556df7f4c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:36.393 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.388+0000 7fc5dd43a740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556df7f4ccb0 0x556df7f4f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:36.393 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.388+0000 7fc5dd43a740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x556df7ea62f0 con 0x556df7f4ccb0 2024-09-11T21:57:36.393 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.388+0000 7fc5dd43a740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x556df7e4e270 con 0x556df7f41af0 2024-09-11T21:57:36.393 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.388+0000 7fc5dd43a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556df7ea22b0 con 0x556df7f42410 2024-09-11T21:57:36.393 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.388+0000 7fc5d37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556df7f4ccb0 0x556df7f4f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.388+0000 7fc5d8c43640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556df7f41af0 0x556df7f41ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.388+0000 7fc5d37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556df7f4ccb0 0x556df7f4f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56888/0 (socket says 172.21.15.1:56888) 2024-09-11T21:57:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.388+0000 7fc5d37fe640 1 -- 172.21.15.1:0/2720458276 learned_addr learned my addr 172.21.15.1:0/2720458276 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.388+0000 7fc5d17fa640 1 --2- 172.21.15.1:0/1164265505 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fc5bc0a2d80 0x7fc5bc0a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.388+0000 7fc5d8c43640 1 --2- 172.21.15.1:0/1164265505 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fc5bc0a2d80 0x7fc5bc0a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.388+0000 7fc5d8c43640 1 -- 172.21.15.1:0/2720458276 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556df7f42410 msgr2=0x556df7f4c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.388+0000 7fc5d8c43640 1 --2- 172.21.15.1:0/2720458276 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556df7f42410 0x556df7f4c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.388+0000 7fc5d8c43640 1 -- 172.21.15.1:0/2720458276 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556df7f4ccb0 msgr2=0x556df7f4f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.388+0000 7fc5d8c43640 1 --2- 172.21.15.1:0/2720458276 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556df7f4ccb0 0x556df7f4f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.388+0000 7fc5d17fa640 1 -- 172.21.15.1:0/1164265505 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7fc5c80cf5a0 con 0x7fc5d4101340 2024-09-11T21:57:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.388+0000 7fc5d8c43640 1 -- 172.21.15.1:0/2720458276 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556df7e4d340 con 0x556df7f41af0 2024-09-11T21:57:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.389+0000 7fc5d8c43640 1 --2- 172.21.15.1:0/2720458276 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556df7f41af0 0x556df7f41ed0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc009ca0 tx=0x7fc5cc0319c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3232a2586bec867 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.389+0000 7fc5d8c43640 1 --2- 172.21.15.1:0/1164265505 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fc5bc0a2d80 0x7fc5bc0a5240 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc047000 tx=0x7fc5cc04a000 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.389+0000 7fc5ba7fc640 1 -- 172.21.15.1:0/2720458276 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc003d40 con 0x556df7f41af0 2024-09-11T21:57:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.389+0000 7fc5ba7fc640 1 -- 172.21.15.1:0/2720458276 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5cc036070 con 0x556df7f41af0 2024-09-11T21:57:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.389+0000 7fc5ba7fc640 1 -- 172.21.15.1:0/2720458276 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc03b400 con 0x556df7f41af0 2024-09-11T21:57:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.389+0000 7fc5dd43a740 1 -- 172.21.15.1:0/2720458276 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556df7f41af0 msgr2=0x556df7f41ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.389+0000 7fc5dd43a740 1 --2- 172.21.15.1:0/2720458276 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556df7f41af0 0x556df7f41ed0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc009ca0 tx=0x7fc5cc0319c0 comp rx=0 tx=0).stop 2024-09-11T21:57:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.389+0000 7fc5dd43a740 1 -- 172.21.15.1:0/2720458276 shutdown_connections 2024-09-11T21:57:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.389+0000 7fc5dd43a740 1 --2- 172.21.15.1:0/2720458276 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556df7f4ccb0 0x556df7f4f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.389+0000 7fc5dd43a740 1 --2- 172.21.15.1:0/2720458276 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556df7f42410 0x556df7f4c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.389+0000 7fc5dd43a740 1 --2- 172.21.15.1:0/2720458276 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556df7f41af0 0x556df7f41ed0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.389+0000 7fc5dd43a740 1 -- 172.21.15.1:0/2720458276 >> 172.21.15.1:0/2720458276 conn(0x556df7f3e060 msgr2=0x556df7f3e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.389+0000 7fc5dd43a740 1 -- 172.21.15.1:0/2720458276 shutdown_connections 2024-09-11T21:57:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.389+0000 7fc5dd43a740 1 -- 172.21.15.1:0/2720458276 wait complete. 2024-09-11T21:57:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.390+0000 7fc5dd43a740 1 -- start start 2024-09-11T21:57:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.390+0000 7fc5dd43a740 1 Processor -- start 2024-09-11T21:57:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.390+0000 7fc5dd43a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556df7f41af0 0x556df7f49c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.390+0000 7fc5dd43a740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556df7f42410 0x556df7f4a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.390+0000 7fc5dd43a740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556df7f4ccb0 0x556df7f4a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.390+0000 7fc5dd43a740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x556df7f4f820 con 0x556df7f42410 2024-09-11T21:57:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.390+0000 7fc5dd43a740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x556df7e2f290 con 0x556df7f4ccb0 2024-09-11T21:57:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.390+0000 7fc5dd43a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556df7f4f9a0 con 0x556df7f41af0 2024-09-11T21:57:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.390+0000 7fc5dd43a740 10 client.0 _ll_register_callbacks cb 0x556df7e60140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:57:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.390+0000 7fc5d37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556df7f42410 0x556df7f4a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.391+0000 7fc5d3fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556df7f41af0 0x556df7f49c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.391+0000 7fc5d37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556df7f42410 0x556df7f4a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56902/0 (socket says 172.21.15.1:56902) 2024-09-11T21:57:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.391+0000 7fc5d37fe640 1 -- 172.21.15.1:0/1907628337 learned_addr learned my addr 172.21.15.1:0/1907628337 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.391+0000 7fc5d3fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556df7f41af0 0x556df7f49c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:49176/0 (socket says 172.21.15.1:49176) 2024-09-11T21:57:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.391+0000 7fc5d8c43640 1 --2- 172.21.15.1:0/1907628337 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556df7f4ccb0 0x556df7f4a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.391+0000 7fc5d3fff640 1 -- 172.21.15.1:0/1907628337 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556df7f4ccb0 msgr2=0x556df7f4a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.391+0000 7fc5d3fff640 1 --2- 172.21.15.1:0/1907628337 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556df7f4ccb0 0x556df7f4a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.391+0000 7fc5d3fff640 1 -- 172.21.15.1:0/1907628337 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556df7f42410 msgr2=0x556df7f4a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.391+0000 7fc5d3fff640 1 --2- 172.21.15.1:0/1907628337 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556df7f42410 0x556df7f4a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.391+0000 7fc5d3fff640 1 -- 172.21.15.1:0/1907628337 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556df7bb9210 con 0x556df7f41af0 2024-09-11T21:57:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.391+0000 7fc5d37fe640 1 --2- 172.21.15.1:0/1907628337 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556df7f42410 0x556df7f4a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T21:57:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.391+0000 7fc5d3fff640 1 --2- 172.21.15.1:0/1907628337 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556df7f41af0 0x556df7f49c60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x556df7f3eec0 tx=0x7fc5c40039c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9232a8a3aef2cb44 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.391+0000 7fc5b37fe640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-11T21:57:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.391+0000 7fc5b37fe640 1 -- 172.21.15.1:0/1907628337 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c4011850 con 0x556df7f41af0 2024-09-11T21:57:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.391+0000 7fc5b37fe640 1 -- 172.21.15.1:0/1907628337 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7fc5c40119f0 con 0x556df7f41af0 2024-09-11T21:57:36.398 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.391+0000 7fc5dd43a740 1 -- 172.21.15.1:0/1907628337 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5cc036070 con 0x556df7f41af0 2024-09-11T21:57:36.398 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.391+0000 7fc5b37fe640 1 -- 172.21.15.1:0/1907628337 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc5c40044d0 con 0x556df7f41af0 2024-09-11T21:57:36.398 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.391+0000 7fc5dd43a740 10 client.25078 fetch_fsmap learned FSMap version 28 2024-09-11T21:57:36.398 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.391+0000 7fc5dd43a740 1 -- 172.21.15.1:0/1907628337 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556df7c85620 con 0x556df7f41af0 2024-09-11T21:57:36.398 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.392+0000 7fc5b37fe640 1 -- 172.21.15.1:0/1907628337 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5cc036070 con 0x556df7f41af0 2024-09-11T21:57:36.398 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.392+0000 7fc5b37fe640 1 -- 172.21.15.1:0/1907628337 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc5c401a470 con 0x556df7f41af0 2024-09-11T21:57:36.398 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.392+0000 7fc5dd43a740 10 client.25078 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T21:57:36.398 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.392+0000 7fc5dd43a740 10 client.25078 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T21:57:36.398 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.392+0000 7fc5dd43a740 20 client.25078 populate_metadata read hostname 'smithi001' 2024-09-11T21:57:36.398 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.392+0000 7fc5dd43a740 1 --2- 172.21.15.1:0/1907628337 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x556df7f5cc60 0x556df7f5f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:36.398 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.392+0000 7fc5dd43a740 4 client.25078 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T21:57:36.398 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.392+0000 7fc5dd43a740 1 -- 172.21.15.1:0/1907628337 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556df7eb3a20 con 0x556df7f5cc60 2024-09-11T21:57:36.398 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.392+0000 7fc5d37fe640 1 --2- 172.21.15.1:0/1907628337 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x556df7f5cc60 0x556df7f5f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:36.399 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.392+0000 7fc5d37fe640 1 --2- 172.21.15.1:0/1907628337 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x556df7f5cc60 0x556df7f5f050 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fcb1899be4c5d31 server_cookie=60aff08afaf412f4 in_seq=0 out_seq=0 2024-09-11T21:57:36.399 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.392+0000 7fc5b37fe640 10 client.25078 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T21:57:36.400 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.393+0000 7fc5b37fe640 1 -- 172.21.15.1:0/1907628337 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556df7eb3a20 con 0x556df7f5cc60 2024-09-11T21:57:36.400 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.393+0000 7fc5b37fe640 10 client.25078 handle_command_reply: tid=0 2024-09-11T21:57:36.400 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.393+0000 7fc5dd43a740 1 client.25078 shutdown 2024-09-11T21:57:36.400 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.393+0000 7fc5dd43a740 1 -- 172.21.15.1:0/1907628337 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556df7f41af0 msgr2=0x556df7f49c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:36.400 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.393+0000 7fc5dd43a740 1 --2- 172.21.15.1:0/1907628337 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556df7f41af0 0x556df7f49c60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x556df7f3eec0 tx=0x7fc5c40039c0 comp rx=0 tx=0).stop 2024-09-11T21:57:36.400 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.393+0000 7fc5dd43a740 1 -- 172.21.15.1:0/1907628337 shutdown_connections 2024-09-11T21:57:36.400 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.393+0000 7fc5dd43a740 1 --2- 172.21.15.1:0/1907628337 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x556df7f5cc60 0x556df7f5f050 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:36.400 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.393+0000 7fc5dd43a740 1 --2- 172.21.15.1:0/1907628337 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556df7f4ccb0 0x556df7f4a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:36.401 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.393+0000 7fc5b37fe640 0 client.25078 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T21:57:36.401 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.393+0000 7fc5dd43a740 1 --2- 172.21.15.1:0/1907628337 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556df7f42410 0x556df7f4a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:36.401 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.393+0000 7fc5dd43a740 1 --2- 172.21.15.1:0/1907628337 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556df7f41af0 0x556df7f49c60 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:36.401 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.393+0000 7fc5dd43a740 1 -- 172.21.15.1:0/1907628337 >> 172.21.15.1:0/1907628337 conn(0x556df7f3e060 msgr2=0x556df7f43dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:36.401 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.393+0000 7fc5dd43a740 1 -- 172.21.15.1:0/1907628337 shutdown_connections 2024-09-11T21:57:36.401 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.393+0000 7fc5dd43a740 1 -- 172.21.15.1:0/1907628337 wait complete. 2024-09-11T21:57:36.401 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.393+0000 7fc5dd43a740 20 client.25078 trim_cache size 0 max 16384 2024-09-11T21:57:36.410 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.408+0000 7fc5dd43a740 1 Processor -- start 2024-09-11T21:57:36.410 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.408+0000 7fc5dd43a740 1 -- start start 2024-09-11T21:57:36.410 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.408+0000 7fc5dd43a740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556df7f41af0 0x556df7f48c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:36.410 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.408+0000 7fc5dd43a740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556df7f42410 0x556df7f4a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:36.410 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.408+0000 7fc5dd43a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556df7f4ccb0 0x556df7f64920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:36.410 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.408+0000 7fc5dd43a740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d41ad8f0 con 0x556df7f41af0 2024-09-11T21:57:36.410 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.408+0000 7fc5dd43a740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d41af430 con 0x556df7f42410 2024-09-11T21:57:36.410 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.408+0000 7fc5dd43a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d41adfc0 con 0x556df7f4ccb0 2024-09-11T21:57:36.410 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.409+0000 7fc5d8c43640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556df7f4ccb0 0x556df7f64920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:36.411 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.409+0000 7fc5d3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556df7f41af0 0x556df7f48c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:36.411 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.409+0000 7fc5d3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556df7f41af0 0x556df7f48c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56908/0 (socket says 172.21.15.1:56908) 2024-09-11T21:57:36.411 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.409+0000 7fc5d3fff640 1 -- 172.21.15.1:0/1923290404 learned_addr learned my addr 172.21.15.1:0/1923290404 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:36.411 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.409+0000 7fc5d37fe640 1 --2- 172.21.15.1:0/1923290404 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556df7f42410 0x556df7f4a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:36.411 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.409+0000 7fc5d8c43640 1 --2- 172.21.15.1:0/1923290404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556df7f4ccb0 0x556df7f64920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:49182/0 (socket says 172.21.15.1:49182) 2024-09-11T21:57:36.411 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.409+0000 7fc5d37fe640 1 -- 172.21.15.1:0/1923290404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556df7f4ccb0 msgr2=0x556df7f64920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:36.411 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.409+0000 7fc5d37fe640 1 --2- 172.21.15.1:0/1923290404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556df7f4ccb0 0x556df7f64920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:36.411 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.409+0000 7fc5d37fe640 1 -- 172.21.15.1:0/1923290404 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556df7f41af0 msgr2=0x556df7f48c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:36.411 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.409+0000 7fc5d37fe640 1 --2- 172.21.15.1:0/1923290404 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556df7f41af0 0x556df7f48c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:36.411 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.409+0000 7fc5d37fe640 1 -- 172.21.15.1:0/1923290404 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556df7f3ecf0 con 0x556df7f42410 2024-09-11T21:57:36.411 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.409+0000 7fc5d3fff640 1 --2- 172.21.15.1:0/1923290404 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556df7f41af0 0x556df7f48c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T21:57:36.411 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.409+0000 7fc5d37fe640 1 --2- 172.21.15.1:0/1923290404 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556df7f42410 0x556df7f4a550 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc5c80982c0 tx=0x7fc5c80d19f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db4c64af7ba0264 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:36.411 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.409+0000 7fc5b9ffb640 1 -- 172.21.15.1:0/1923290404 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c801c040 con 0x556df7f42410 2024-09-11T21:57:36.412 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.409+0000 7fc5b9ffb640 1 -- 172.21.15.1:0/1923290404 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5c809b8f0 con 0x556df7f42410 2024-09-11T21:57:36.412 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.409+0000 7fc5b9ffb640 1 -- 172.21.15.1:0/1923290404 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c80a40d0 con 0x556df7f42410 2024-09-11T21:57:36.415 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.410+0000 7fc5dd43a740 1 -- 172.21.15.1:0/1923290404 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556df7f42410 msgr2=0x556df7f4a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:36.415 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.410+0000 7fc5dd43a740 1 --2- 172.21.15.1:0/1923290404 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556df7f42410 0x556df7f4a550 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc5c80982c0 tx=0x7fc5c80d19f0 comp rx=0 tx=0).stop 2024-09-11T21:57:36.415 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.410+0000 7fc5dd43a740 1 -- 172.21.15.1:0/1923290404 shutdown_connections 2024-09-11T21:57:36.415 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.410+0000 7fc5dd43a740 1 --2- 172.21.15.1:0/1923290404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556df7f4ccb0 0x556df7f64920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:36.415 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.410+0000 7fc5dd43a740 1 --2- 172.21.15.1:0/1923290404 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556df7f42410 0x556df7f4a550 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:36.415 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.410+0000 7fc5dd43a740 1 --2- 172.21.15.1:0/1923290404 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556df7f41af0 0x556df7f48c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:36.415 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.410+0000 7fc5dd43a740 1 -- 172.21.15.1:0/1923290404 >> 172.21.15.1:0/1923290404 conn(0x556df7f3e060 msgr2=0x556df7f4bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:36.415 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.410+0000 7fc5dd43a740 1 -- 172.21.15.1:0/1923290404 shutdown_connections 2024-09-11T21:57:36.416 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.410+0000 7fc5dd43a740 1 -- 172.21.15.1:0/1923290404 wait complete. 2024-09-11T21:57:36.416 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.410+0000 7fc5dd43a740 1 -- start start 2024-09-11T21:57:36.416 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.411+0000 7fc5dd43a740 1 Processor -- start 2024-09-11T21:57:36.416 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.411+0000 7fc5dd43a740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556df7f41af0 0x556df7f551a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:36.416 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.411+0000 7fc5dd43a740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556df7f42410 0x556df7f46810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:36.416 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.411+0000 7fc5dd43a740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556df7f4ccb0 0x556df7f46d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:36.416 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.411+0000 7fc5dd43a740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d41aed60 con 0x556df7f42410 2024-09-11T21:57:36.416 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.411+0000 7fc5dd43a740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d41ae690 con 0x556df7f4ccb0 2024-09-11T21:57:36.416 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.411+0000 7fc5dd43a740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d41ab8d0 con 0x556df7f41af0 2024-09-11T21:57:36.416 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.411+0000 7fc5dd43a740 10 client.0 _ll_register_callbacks cb 0x7fc5c4005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:57:36.416 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.411+0000 7fc5d37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556df7f42410 0x556df7f46810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:36.416 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.411+0000 7fc5d3fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556df7f41af0 0x556df7f551a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.411+0000 7fc5d37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556df7f42410 0x556df7f46810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56916/0 (socket says 172.21.15.1:56916) 2024-09-11T21:57:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.411+0000 7fc5d37fe640 1 -- 172.21.15.1:0/166166011 learned_addr learned my addr 172.21.15.1:0/166166011 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.411+0000 7fc5d37fe640 1 -- 172.21.15.1:0/166166011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556df7f41af0 msgr2=0x556df7f551a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.411+0000 7fc5d8c43640 1 --2- 172.21.15.1:0/166166011 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556df7f4ccb0 0x556df7f46d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.411+0000 7fc5d37fe640 1 --2- 172.21.15.1:0/166166011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556df7f41af0 0x556df7f551a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.411+0000 7fc5d37fe640 1 -- 172.21.15.1:0/166166011 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556df7f4ccb0 msgr2=0x556df7f46d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.411+0000 7fc5d37fe640 1 --2- 172.21.15.1:0/166166011 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556df7f4ccb0 0x556df7f46d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.411+0000 7fc5d37fe640 1 -- 172.21.15.1:0/166166011 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556df7f47390 con 0x556df7f42410 2024-09-11T21:57:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.412+0000 7fc5d37fe640 1 --2- 172.21.15.1:0/166166011 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556df7f42410 0x556df7f46810 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc5c800c6a0 tx=0x7fc5c80cf460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e026731cf80893cb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.412+0000 7fc5b37fe640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T21:57:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.412+0000 7fc5b37fe640 1 -- 172.21.15.1:0/166166011 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c80b3070 con 0x556df7f42410 2024-09-11T21:57:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.412+0000 7fc5b37fe640 1 -- 172.21.15.1:0/166166011 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7fc5c809bd50 con 0x556df7f42410 2024-09-11T21:57:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.412+0000 7fc5dd43a740 1 -- 172.21.15.1:0/166166011 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5c809b8f0 con 0x556df7f42410 2024-09-11T21:57:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.412+0000 7fc5b37fe640 1 -- 172.21.15.1:0/166166011 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc5c801c050 con 0x556df7f42410 2024-09-11T21:57:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.412+0000 7fc5dd43a740 10 client.15456 fetch_fsmap learned FSMap version 28 2024-09-11T21:57:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.412+0000 7fc5dd43a740 1 -- 172.21.15.1:0/166166011 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556df7fc7730 con 0x556df7f42410 2024-09-11T21:57:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.413+0000 7fc5b37fe640 1 -- 172.21.15.1:0/166166011 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5c809b8f0 con 0x556df7f42410 2024-09-11T21:57:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.413+0000 7fc5b37fe640 1 -- 172.21.15.1:0/166166011 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc5c80cce80 con 0x556df7f42410 2024-09-11T21:57:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.413+0000 7fc5dd43a740 10 client.15456 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T21:57:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.413+0000 7fc5dd43a740 10 client.15456 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T21:57:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.413+0000 7fc5dd43a740 20 client.15456 populate_metadata read hostname 'smithi001' 2024-09-11T21:57:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.413+0000 7fc5dd43a740 1 --2- 172.21.15.1:0/166166011 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x556df7fc7d70 0x556df7fca190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.413+0000 7fc5dd43a740 4 client.15456 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T21:57:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.413+0000 7fc5dd43a740 1 -- 172.21.15.1:0/166166011 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556df7eadd70 con 0x556df7fc7d70 2024-09-11T21:57:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.413+0000 7fc5d3fff640 1 --2- 172.21.15.1:0/166166011 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x556df7fc7d70 0x556df7fca190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.413+0000 7fc5b37fe640 10 client.15456 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T21:57:36.419 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.413+0000 7fc5d3fff640 1 --2- 172.21.15.1:0/166166011 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x556df7fc7d70 0x556df7fca190 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e40d982660cd290b server_cookie=489b5606f9edc419 in_seq=0 out_seq=0 2024-09-11T21:57:36.424 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.423+0000 7fc5b37fe640 1 -- 172.21.15.1:0/166166011 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556df7eadd70 con 0x556df7fc7d70 2024-09-11T21:57:36.425 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.423+0000 7fc5b37fe640 10 client.15456 handle_command_reply: tid=0 2024-09-11T21:57:36.425 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.423+0000 7fc5dd43a740 1 client.15456 shutdown 2024-09-11T21:57:36.425 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.424+0000 7fc5dd43a740 1 -- 172.21.15.1:0/166166011 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556df7f42410 msgr2=0x556df7f46810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:36.425 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.424+0000 7fc5dd43a740 1 --2- 172.21.15.1:0/166166011 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556df7f42410 0x556df7f46810 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc5c800c6a0 tx=0x7fc5c80cf460 comp rx=0 tx=0).stop 2024-09-11T21:57:36.425 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.424+0000 7fc5dd43a740 1 -- 172.21.15.1:0/166166011 shutdown_connections 2024-09-11T21:57:36.425 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.424+0000 7fc5dd43a740 1 --2- 172.21.15.1:0/166166011 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x556df7fc7d70 0x556df7fca190 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:36.425 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.424+0000 7fc5dd43a740 1 --2- 172.21.15.1:0/166166011 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556df7f4ccb0 0x556df7f46d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:36.425 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.424+0000 7fc5b37fe640 0 client.15456 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T21:57:36.425 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.424+0000 7fc5dd43a740 1 --2- 172.21.15.1:0/166166011 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556df7f42410 0x556df7f46810 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:36.425 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.424+0000 7fc5dd43a740 1 --2- 172.21.15.1:0/166166011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556df7f41af0 0x556df7f551a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:36.425 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.424+0000 7fc5dd43a740 1 -- 172.21.15.1:0/166166011 >> 172.21.15.1:0/166166011 conn(0x556df7f3e060 msgr2=0x556df7f3c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:36.425 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.424+0000 7fc5dd43a740 1 -- 172.21.15.1:0/166166011 shutdown_connections 2024-09-11T21:57:36.425 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.424+0000 7fc5dd43a740 1 -- 172.21.15.1:0/166166011 wait complete. 2024-09-11T21:57:36.426 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.424+0000 7fc5dd43a740 20 client.15456 trim_cache size 0 max 16384 2024-09-11T21:57:36.426 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-09-11T21:57:36.426 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-09-11T21:57:36.426 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-09-11T21:57:36.426 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-09-11T21:57:36.427 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.424+0000 7fc5da6a6640 1 -- 172.21.15.1:0/1164265505 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fc5bc0a2d80 msgr2=0x7fc5bc0a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:36.427 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.424+0000 7fc5da6a6640 1 --2- 172.21.15.1:0/1164265505 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fc5bc0a2d80 0x7fc5bc0a5240 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc047000 tx=0x7fc5cc04a000 comp rx=0 tx=0).stop 2024-09-11T21:57:36.427 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.424+0000 7fc5da6a6640 1 -- 172.21.15.1:0/1164265505 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc5d4101340 msgr2=0x7fc5d41988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:36.427 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.424+0000 7fc5da6a6640 1 --2- 172.21.15.1:0/1164265505 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc5d4101340 0x7fc5d41988e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc5c800e9f0 tx=0x7fc5c800eeb0 comp rx=0 tx=0).stop 2024-09-11T21:57:36.427 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.425+0000 7fc5da6a6640 1 -- 172.21.15.1:0/1164265505 shutdown_connections 2024-09-11T21:57:36.427 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.425+0000 7fc5da6a6640 1 --2- 172.21.15.1:0/1164265505 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fc5bc0a2d80 0x7fc5bc0a5240 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:36.427 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.425+0000 7fc5da6a6640 1 --2- 172.21.15.1:0/1164265505 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc5d41069b0 0x7fc5d419cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:36.427 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.425+0000 7fc5da6a6640 1 --2- 172.21.15.1:0/1164265505 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc5d4101340 0x7fc5d41988e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:36.428 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.425+0000 7fc5da6a6640 1 --2- 172.21.15.1:0/1164265505 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5d41009a0 0x7fc5d41983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:36.428 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.425+0000 7fc5da6a6640 1 -- 172.21.15.1:0/1164265505 >> 172.21.15.1:0/1164265505 conn(0x7fc5d40779a0 msgr2=0x7fc5d4069ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:36.428 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.425+0000 7fc5da6a6640 1 -- 172.21.15.1:0/1164265505 shutdown_connections 2024-09-11T21:57:36.428 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:36.425+0000 7fc5da6a6640 1 -- 172.21.15.1:0/1164265505 wait complete. 2024-09-11T21:57:36.483 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:57:36.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.945+0000 7ffa44327640 1 -- 172.21.15.1:0/2240232044 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffa3c100d00 msgr2=0x7ffa3c106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:36.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.945+0000 7ffa44327640 1 --2- 172.21.15.1:0/2240232044 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffa3c100d00 0x7ffa3c106840 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c009930 tx=0x7ffa2c02a420 comp rx=0 tx=0).stop 2024-09-11T21:57:36.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.946+0000 7ffa44327640 1 -- 172.21.15.1:0/2240232044 shutdown_connections 2024-09-11T21:57:36.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.946+0000 7ffa44327640 1 --2- 172.21.15.1:0/2240232044 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa3c106f50 0x7ffa3c10eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:36.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.946+0000 7ffa44327640 1 --2- 172.21.15.1:0/2240232044 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffa3c100d00 0x7ffa3c106840 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:36.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.946+0000 7ffa44327640 1 --2- 172.21.15.1:0/2240232044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffa3c100350 0x7ffa3c100730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:36.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.946+0000 7ffa44327640 1 -- 172.21.15.1:0/2240232044 >> 172.21.15.1:0/2240232044 conn(0x7ffa3c077210 msgr2=0x7ffa3c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:36.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.946+0000 7ffa44327640 1 -- 172.21.15.1:0/2240232044 shutdown_connections 2024-09-11T21:57:36.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.946+0000 7ffa44327640 1 -- 172.21.15.1:0/2240232044 wait complete. 2024-09-11T21:57:36.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.946+0000 7ffa44327640 1 Processor -- start 2024-09-11T21:57:36.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.946+0000 7ffa44327640 1 -- start start 2024-09-11T21:57:36.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.946+0000 7ffa44327640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffa3c100350 0x7ffa3c198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:36.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.946+0000 7ffa44327640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffa3c100d00 0x7ffa3c198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:36.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.946+0000 7ffa44327640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa3c106f50 0x7ffa3c19cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:36.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.946+0000 7ffa44327640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3c111fa0 con 0x7ffa3c100350 2024-09-11T21:57:36.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.946+0000 7ffa44327640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3c111e20 con 0x7ffa3c100d00 2024-09-11T21:57:36.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.946+0000 7ffa44327640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3c112120 con 0x7ffa3c106f50 2024-09-11T21:57:36.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.947+0000 7ffa420c3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffa3c100350 0x7ffa3c198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:36.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.947+0000 7ffa420c3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffa3c100350 0x7ffa3c198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33276/0 (socket says 172.21.15.1:33276) 2024-09-11T21:57:36.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.947+0000 7ffa420c3640 1 -- 172.21.15.1:0/1808620674 learned_addr learned my addr 172.21.15.1:0/1808620674 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:36.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.947+0000 7ffa428c4640 1 --2- 172.21.15.1:0/1808620674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa3c106f50 0x7ffa3c19cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:36.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.947+0000 7ffa418c2640 1 --2- 172.21.15.1:0/1808620674 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffa3c100d00 0x7ffa3c198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:36.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.947+0000 7ffa420c3640 1 -- 172.21.15.1:0/1808620674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa3c106f50 msgr2=0x7ffa3c19cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:36.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.947+0000 7ffa420c3640 1 --2- 172.21.15.1:0/1808620674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa3c106f50 0x7ffa3c19cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:36.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.947+0000 7ffa420c3640 1 -- 172.21.15.1:0/1808620674 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffa3c100d00 msgr2=0x7ffa3c198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:36.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.947+0000 7ffa420c3640 1 --2- 172.21.15.1:0/1808620674 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffa3c100d00 0x7ffa3c198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:36.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.947+0000 7ffa420c3640 1 -- 172.21.15.1:0/1808620674 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa3c19d4b0 con 0x7ffa3c100350 2024-09-11T21:57:36.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.947+0000 7ffa428c4640 1 --2- 172.21.15.1:0/1808620674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa3c106f50 0x7ffa3c19cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T21:57:36.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.947+0000 7ffa420c3640 1 --2- 172.21.15.1:0/1808620674 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffa3c100350 0x7ffa3c198500 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ffa3800ebc0 tx=0x7ffa3800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:36.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.947+0000 7ffa337fe640 1 -- 172.21.15.1:0/1808620674 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa38010430 con 0x7ffa3c100350 2024-09-11T21:57:36.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.947+0000 7ffa337fe640 1 -- 172.21.15.1:0/1808620674 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa38010ab0 con 0x7ffa3c100350 2024-09-11T21:57:36.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.947+0000 7ffa44327640 1 -- 172.21.15.1:0/1808620674 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa3c19d7a0 con 0x7ffa3c100350 2024-09-11T21:57:36.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.947+0000 7ffa337fe640 1 -- 172.21.15.1:0/1808620674 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa38018b30 con 0x7ffa3c100350 2024-09-11T21:57:36.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.947+0000 7ffa44327640 1 -- 172.21.15.1:0/1808620674 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa3c1a5080 con 0x7ffa3c100350 2024-09-11T21:57:36.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.949+0000 7ffa337fe640 1 -- 172.21.15.1:0/1808620674 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 24) ==== 138008+0+0 (secure 0 0 0) 0x7ffa38018cd0 con 0x7ffa3c100350 2024-09-11T21:57:36.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.950+0000 7ffa46700740 1 Processor -- start 2024-09-11T21:57:36.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.950+0000 7ffa46700740 1 -- start start 2024-09-11T21:57:36.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.950+0000 7ffa46700740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a3a6ef9d0 0x555a3a6efdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:36.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.950+0000 7ffa46700740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555a3a6f02f0 0x555a3a6fa670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:36.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.950+0000 7ffa46700740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a3a6fabb0 0x555a3a6fcfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:36.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.950+0000 7ffa46700740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x555a3a656090 con 0x555a3a6f02f0 2024-09-11T21:57:36.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.950+0000 7ffa46700740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x555a3a5fda70 con 0x555a3a6fabb0 2024-09-11T21:57:36.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.950+0000 7ffa46700740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x555a3a652050 con 0x555a3a6ef9d0 2024-09-11T21:57:36.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.950+0000 7ffa420c3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555a3a6f02f0 0x555a3a6fa670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:36.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.950+0000 7ffa428c4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a3a6ef9d0 0x555a3a6efdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:36.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.950+0000 7ffa420c3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555a3a6f02f0 0x555a3a6fa670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33290/0 (socket says 172.21.15.1:33290) 2024-09-11T21:57:36.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.950+0000 7ffa337fe640 1 --2- 172.21.15.1:0/1808620674 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7ffa280a2d80 0x7ffa280a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:36.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.950+0000 7ffa420c3640 1 -- 172.21.15.1:0/3962181904 learned_addr learned my addr 172.21.15.1:0/3962181904 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:36.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.950+0000 7ffa428c4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a3a6ef9d0 0x555a3a6efdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:50770/0 (socket says 172.21.15.1:50770) 2024-09-11T21:57:36.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.951+0000 7ffa418c2640 1 --2- 172.21.15.1:0/3962181904 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a3a6fabb0 0x555a3a6fcfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:36.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.951+0000 7ffa428c4640 1 --2- 172.21.15.1:0/1808620674 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7ffa280a2d80 0x7ffa280a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:36.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.951+0000 7ffa337fe640 1 -- 172.21.15.1:0/1808620674 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7ffa380cfd70 con 0x7ffa3c100350 2024-09-11T21:57:36.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.951+0000 7ffa428c4640 1 -- 172.21.15.1:0/3962181904 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a3a6fabb0 msgr2=0x555a3a6fcfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:36.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.951+0000 7ffa428c4640 1 --2- 172.21.15.1:0/3962181904 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a3a6fabb0 0x555a3a6fcfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:36.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.951+0000 7ffa428c4640 1 -- 172.21.15.1:0/3962181904 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555a3a6f02f0 msgr2=0x555a3a6fa670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:36.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.951+0000 7ffa428c4640 1 --2- 172.21.15.1:0/3962181904 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555a3a6f02f0 0x555a3a6fa670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:36.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.951+0000 7ffa428c4640 1 -- 172.21.15.1:0/3962181904 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a3a60a130 con 0x555a3a6ef9d0 2024-09-11T21:57:36.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.951+0000 7ffa420c3640 1 --2- 172.21.15.1:0/3962181904 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555a3a6f02f0 0x555a3a6fa670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T21:57:36.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.951+0000 7ffa428c4640 1 --2- 172.21.15.1:0/3962181904 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a3a6ef9d0 0x555a3a6efdb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ffa34013a30 tx=0x7ffa34013ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2f1c72e52d2d1a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:36.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.951+0000 7ffa428c4640 1 --2- 172.21.15.1:0/1808620674 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7ffa280a2d80 0x7ffa280a5240 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ffa34013000 tx=0x7ffa3401f000 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:36.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.951+0000 7ffa30dea640 1 -- 172.21.15.1:0/3962181904 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa340120d0 con 0x555a3a6ef9d0 2024-09-11T21:57:36.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.951+0000 7ffa30dea640 1 -- 172.21.15.1:0/3962181904 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa34022030 con 0x555a3a6ef9d0 2024-09-11T21:57:36.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.951+0000 7ffa30dea640 1 -- 172.21.15.1:0/3962181904 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa34010770 con 0x555a3a6ef9d0 2024-09-11T21:57:36.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.952+0000 7ffa46700740 1 -- 172.21.15.1:0/3962181904 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a3a6ef9d0 msgr2=0x555a3a6efdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:36.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.952+0000 7ffa46700740 1 --2- 172.21.15.1:0/3962181904 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a3a6ef9d0 0x555a3a6efdb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ffa34013a30 tx=0x7ffa34013ef0 comp rx=0 tx=0).stop 2024-09-11T21:57:36.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.952+0000 7ffa46700740 1 -- 172.21.15.1:0/3962181904 shutdown_connections 2024-09-11T21:57:36.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.952+0000 7ffa46700740 1 --2- 172.21.15.1:0/3962181904 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a3a6fabb0 0x555a3a6fcfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:36.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.952+0000 7ffa46700740 1 --2- 172.21.15.1:0/3962181904 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555a3a6f02f0 0x555a3a6fa670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:36.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.952+0000 7ffa46700740 1 --2- 172.21.15.1:0/3962181904 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a3a6ef9d0 0x555a3a6efdb0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:36.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.952+0000 7ffa46700740 1 -- 172.21.15.1:0/3962181904 >> 172.21.15.1:0/3962181904 conn(0x555a3a6ebf60 msgr2=0x555a3a6ec340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:36.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.952+0000 7ffa46700740 1 -- 172.21.15.1:0/3962181904 shutdown_connections 2024-09-11T21:57:36.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.952+0000 7ffa46700740 1 -- 172.21.15.1:0/3962181904 wait complete. 2024-09-11T21:57:36.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.953+0000 7ffa46700740 1 -- start start 2024-09-11T21:57:36.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.953+0000 7ffa46700740 1 Processor -- start 2024-09-11T21:57:36.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.953+0000 7ffa46700740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a3a6ef9d0 0x555a3a6fa4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:36.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.953+0000 7ffa46700740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a3a6f02f0 0x555a3a6f7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:36.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.953+0000 7ffa46700740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555a3a6fabb0 0x555a3a6f7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:36.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.953+0000 7ffa46700740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x555a3a6fd780 con 0x555a3a6fabb0 2024-09-11T21:57:36.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.953+0000 7ffa46700740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x555a3a610f20 con 0x555a3a6f02f0 2024-09-11T21:57:36.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.953+0000 7ffa46700740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x555a3a6fd900 con 0x555a3a6ef9d0 2024-09-11T21:57:36.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.953+0000 7ffa46700740 10 client.0 _ll_register_callbacks cb 0x555a3a5da030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:57:36.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.953+0000 7ffa428c4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555a3a6fabb0 0x555a3a6f7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:36.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.953+0000 7ffa428c4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555a3a6fabb0 0x555a3a6f7d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33306/0 (socket says 172.21.15.1:33306) 2024-09-11T21:57:36.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.953+0000 7ffa428c4640 1 -- 172.21.15.1:0/311017757 learned_addr learned my addr 172.21.15.1:0/311017757 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:36.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.953+0000 7ffa420c3640 1 --2- 172.21.15.1:0/311017757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a3a6ef9d0 0x555a3a6fa4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:36.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.953+0000 7ffa418c2640 1 --2- 172.21.15.1:0/311017757 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a3a6f02f0 0x555a3a6f7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:36.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.953+0000 7ffa428c4640 1 -- 172.21.15.1:0/311017757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a3a6ef9d0 msgr2=0x555a3a6fa4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:36.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.953+0000 7ffa428c4640 1 --2- 172.21.15.1:0/311017757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a3a6ef9d0 0x555a3a6fa4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:36.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.953+0000 7ffa428c4640 1 -- 172.21.15.1:0/311017757 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a3a6f02f0 msgr2=0x555a3a6f7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:36.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.953+0000 7ffa428c4640 1 --2- 172.21.15.1:0/311017757 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a3a6f02f0 0x555a3a6f7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:36.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.953+0000 7ffa428c4640 1 -- 172.21.15.1:0/311017757 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a3a3c72b0 con 0x555a3a6fabb0 2024-09-11T21:57:36.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.953+0000 7ffa418c2640 1 --2- 172.21.15.1:0/311017757 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a3a6f02f0 0x555a3a6f7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:57:36.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.954+0000 7ffa428c4640 1 --2- 172.21.15.1:0/311017757 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555a3a6fabb0 0x555a3a6f7d40 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ffa34005bd0 tx=0x7ffa34013660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=775dd07782cd2bba server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:36.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.954+0000 7ffa25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T21:57:36.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.954+0000 7ffa25ffb640 1 -- 172.21.15.1:0/311017757 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3400fe20 con 0x555a3a6fabb0 2024-09-11T21:57:36.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.954+0000 7ffa25ffb640 1 -- 172.21.15.1:0/311017757 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7ffa34012820 con 0x555a3a6fabb0 2024-09-11T21:57:36.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.954+0000 7ffa46700740 1 -- 172.21.15.1:0/311017757 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa34022030 con 0x555a3a6fabb0 2024-09-11T21:57:36.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.954+0000 7ffa25ffb640 1 -- 172.21.15.1:0/311017757 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ffa34012a20 con 0x555a3a6fabb0 2024-09-11T21:57:36.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.954+0000 7ffa46700740 10 client.15468 fetch_fsmap learned FSMap version 28 2024-09-11T21:57:36.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.954+0000 7ffa46700740 1 -- 172.21.15.1:0/311017757 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555a3a7090f0 con 0x555a3a6fabb0 2024-09-11T21:57:36.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.954+0000 7ffa25ffb640 1 -- 172.21.15.1:0/311017757 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa34022030 con 0x555a3a6fabb0 2024-09-11T21:57:36.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.954+0000 7ffa25ffb640 1 -- 172.21.15.1:0/311017757 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ffa34003740 con 0x555a3a6fabb0 2024-09-11T21:57:36.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.954+0000 7ffa46700740 10 client.15468 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T21:57:36.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.954+0000 7ffa46700740 10 client.15468 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-11T21:57:36.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.954+0000 7ffa46700740 20 client.15468 populate_metadata read hostname 'smithi001' 2024-09-11T21:57:36.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.954+0000 7ffa46700740 1 --2- 172.21.15.1:0/311017757 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x555a3a70b490 0x555a3a70c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:36.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.955+0000 7ffa46700740 4 client.15468 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T21:57:36.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.955+0000 7ffa46700740 1 -- 172.21.15.1:0/311017757 --> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a3a663600 con 0x555a3a70b490 2024-09-11T21:57:36.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.955+0000 7ffa420c3640 1 --2- 172.21.15.1:0/311017757 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x555a3a70b490 0x555a3a70c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:36.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.955+0000 7ffa420c3640 1 --2- 172.21.15.1:0/311017757 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x555a3a70b490 0x555a3a70c900 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=56eb4c70882309c6 server_cookie=fbb7888b2df0af29 in_seq=0 out_seq=0 2024-09-11T21:57:36.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.955+0000 7ffa25ffb640 10 client.15468 ms_handle_connect on v2:172.21.15.161:6834/828381592 2024-09-11T21:57:36.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.956+0000 7ffa25ffb640 1 -- 172.21.15.1:0/311017757 <== mds.4 v2:172.21.15.161:6834/828381592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555a3a663600 con 0x555a3a70b490 2024-09-11T21:57:36.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.956+0000 7ffa25ffb640 10 client.15468 handle_command_reply: tid=0 2024-09-11T21:57:36.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.956+0000 7ffa46700740 1 client.15468 shutdown 2024-09-11T21:57:36.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.956+0000 7ffa46700740 1 -- 172.21.15.1:0/311017757 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555a3a6fabb0 msgr2=0x555a3a6f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:36.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.956+0000 7ffa46700740 1 --2- 172.21.15.1:0/311017757 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555a3a6fabb0 0x555a3a6f7d40 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ffa34005bd0 tx=0x7ffa34013660 comp rx=0 tx=0).stop 2024-09-11T21:57:36.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.956+0000 7ffa46700740 1 -- 172.21.15.1:0/311017757 shutdown_connections 2024-09-11T21:57:36.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.956+0000 7ffa46700740 1 --2- 172.21.15.1:0/311017757 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555a3a6fabb0 0x555a3a6f7d40 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:36.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.956+0000 7ffa46700740 1 --2- 172.21.15.1:0/311017757 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x555a3a70b490 0x555a3a70c900 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:36.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.956+0000 7ffa46700740 1 --2- 172.21.15.1:0/311017757 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a3a6f02f0 0x555a3a6f7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:36.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.956+0000 7ffa46700740 1 --2- 172.21.15.1:0/311017757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a3a6ef9d0 0x555a3a6fa4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:36.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.956+0000 7ffa25ffb640 0 client.15468 ms_handle_reset on v2:172.21.15.161:6834/828381592 2024-09-11T21:57:36.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.956+0000 7ffa46700740 1 -- 172.21.15.1:0/311017757 >> 172.21.15.1:0/311017757 conn(0x555a3a6ebf60 msgr2=0x555a3a6f1c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:36.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.956+0000 7ffa46700740 1 -- 172.21.15.1:0/311017757 shutdown_connections 2024-09-11T21:57:36.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.956+0000 7ffa46700740 1 -- 172.21.15.1:0/311017757 wait complete. 2024-09-11T21:57:36.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.956+0000 7ffa46700740 20 client.15468 trim_cache size 0 max 16384 2024-09-11T21:57:36.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.974+0000 7ffa46700740 1 Processor -- start 2024-09-11T21:57:36.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.974+0000 7ffa46700740 1 -- start start 2024-09-11T21:57:36.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.974+0000 7ffa46700740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a3a6ef9d0 0x555a3a714fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:36.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.974+0000 7ffa46700740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a3a6f02f0 0x555a3a6ea7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:36.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.974+0000 7ffa46700740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555a3a6fabb0 0x555a3a6ead20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:36.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.974+0000 7ffa46700740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x555a3a70ac40 con 0x555a3a6fabb0 2024-09-11T21:57:36.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.974+0000 7ffa46700740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x555a3a70b310 con 0x555a3a6ef9d0 2024-09-11T21:57:36.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.974+0000 7ffa46700740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x555a3a70b9e0 con 0x555a3a6f02f0 2024-09-11T21:57:36.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.974+0000 7ffa428c4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555a3a6fabb0 0x555a3a6ead20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:36.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.974+0000 7ffa428c4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555a3a6fabb0 0x555a3a6ead20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33310/0 (socket says 172.21.15.1:33310) 2024-09-11T21:57:36.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.974+0000 7ffa428c4640 1 -- 172.21.15.1:0/1949672586 learned_addr learned my addr 172.21.15.1:0/1949672586 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:36.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.975+0000 7ffa418c2640 1 --2- 172.21.15.1:0/1949672586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a3a6f02f0 0x555a3a6ea7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:36.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.975+0000 7ffa420c3640 1 --2- 172.21.15.1:0/1949672586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a3a6ef9d0 0x555a3a714fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:36.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.975+0000 7ffa428c4640 1 -- 172.21.15.1:0/1949672586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a3a6f02f0 msgr2=0x555a3a6ea7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:36.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.975+0000 7ffa428c4640 1 --2- 172.21.15.1:0/1949672586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a3a6f02f0 0x555a3a6ea7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:36.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.975+0000 7ffa428c4640 1 -- 172.21.15.1:0/1949672586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a3a6ef9d0 msgr2=0x555a3a714fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:36.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.975+0000 7ffa428c4640 1 --2- 172.21.15.1:0/1949672586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a3a6ef9d0 0x555a3a714fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:36.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.975+0000 7ffa428c4640 1 -- 172.21.15.1:0/1949672586 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a3a6f8340 con 0x555a3a6fabb0 2024-09-11T21:57:36.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.975+0000 7ffa428c4640 1 --2- 172.21.15.1:0/1949672586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555a3a6fabb0 0x555a3a6ead20 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ffa34005c20 tx=0x7ffa340136d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d233c8b342af467e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:36.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.975+0000 7ffa27fff640 1 -- 172.21.15.1:0/1949672586 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa34012820 con 0x555a3a6fabb0 2024-09-11T21:57:36.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.975+0000 7ffa27fff640 1 -- 172.21.15.1:0/1949672586 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa34022040 con 0x555a3a6fabb0 2024-09-11T21:57:36.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.975+0000 7ffa27fff640 1 -- 172.21.15.1:0/1949672586 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa340035f0 con 0x555a3a6fabb0 2024-09-11T21:57:36.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.976+0000 7ffa46700740 1 -- 172.21.15.1:0/1949672586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555a3a6fabb0 msgr2=0x555a3a6ead20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:36.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.976+0000 7ffa46700740 1 --2- 172.21.15.1:0/1949672586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555a3a6fabb0 0x555a3a6ead20 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ffa34005c20 tx=0x7ffa340136d0 comp rx=0 tx=0).stop 2024-09-11T21:57:36.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.976+0000 7ffa46700740 1 -- 172.21.15.1:0/1949672586 shutdown_connections 2024-09-11T21:57:36.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.976+0000 7ffa46700740 1 --2- 172.21.15.1:0/1949672586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555a3a6fabb0 0x555a3a6ead20 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:36.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.976+0000 7ffa46700740 1 --2- 172.21.15.1:0/1949672586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a3a6f02f0 0x555a3a6ea7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:36.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.976+0000 7ffa46700740 1 --2- 172.21.15.1:0/1949672586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a3a6ef9d0 0x555a3a714fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:36.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.976+0000 7ffa46700740 1 -- 172.21.15.1:0/1949672586 >> 172.21.15.1:0/1949672586 conn(0x555a3a6ebf60 msgr2=0x555a3a6f2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:36.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.976+0000 7ffa46700740 1 -- 172.21.15.1:0/1949672586 shutdown_connections 2024-09-11T21:57:36.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.976+0000 7ffa46700740 1 -- 172.21.15.1:0/1949672586 wait complete. 2024-09-11T21:57:36.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.976+0000 7ffa46700740 1 -- start start 2024-09-11T21:57:36.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.976+0000 7ffa46700740 1 Processor -- start 2024-09-11T21:57:36.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.977+0000 7ffa46700740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a3a6ef9d0 0x555a3a6f3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:36.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.977+0000 7ffa46700740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a3a6f02f0 0x555a3a6f3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:36.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.977+0000 7ffa46700740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555a3a6fabb0 0x555a3a6f44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:36.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.977+0000 7ffa46700740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x555a3a70c780 con 0x555a3a6fabb0 2024-09-11T21:57:36.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.977+0000 7ffa46700740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x555a3a70c0b0 con 0x555a3a6f02f0 2024-09-11T21:57:36.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.977+0000 7ffa46700740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x555a3a70ce50 con 0x555a3a6ef9d0 2024-09-11T21:57:36.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.977+0000 7ffa46700740 10 client.0 _ll_register_callbacks cb 0x7ffa380cd290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:57:36.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.977+0000 7ffa428c4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555a3a6fabb0 0x555a3a6f44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:36.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.977+0000 7ffa420c3640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a3a6ef9d0 0x555a3a6f3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:36.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.977+0000 7ffa428c4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555a3a6fabb0 0x555a3a6f44f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33320/0 (socket says 172.21.15.1:33320) 2024-09-11T21:57:36.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.977+0000 7ffa428c4640 1 -- 172.21.15.1:0/2020522522 learned_addr learned my addr 172.21.15.1:0/2020522522 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:36.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.977+0000 7ffa418c2640 1 --2- 172.21.15.1:0/2020522522 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a3a6f02f0 0x555a3a6f3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:36.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.977+0000 7ffa428c4640 1 -- 172.21.15.1:0/2020522522 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a3a6ef9d0 msgr2=0x555a3a6f3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:36.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.977+0000 7ffa428c4640 1 --2- 172.21.15.1:0/2020522522 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a3a6ef9d0 0x555a3a6f3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:36.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.977+0000 7ffa428c4640 1 -- 172.21.15.1:0/2020522522 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a3a6f02f0 msgr2=0x555a3a6f3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:36.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.977+0000 7ffa428c4640 1 --2- 172.21.15.1:0/2020522522 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a3a6f02f0 0x555a3a6f3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:36.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.977+0000 7ffa428c4640 1 -- 172.21.15.1:0/2020522522 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a3a6f4c30 con 0x555a3a6fabb0 2024-09-11T21:57:36.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.977+0000 7ffa420c3640 1 --2- 172.21.15.1:0/2020522522 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a3a6ef9d0 0x555a3a6f3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T21:57:36.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.977+0000 7ffa428c4640 1 --2- 172.21.15.1:0/2020522522 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555a3a6fabb0 0x555a3a6f44f0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ffa34003b10 tx=0x7ffa3400fd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1bb82bdb41d765d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:36.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.977+0000 7ffa25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T21:57:36.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.978+0000 7ffa25ffb640 1 -- 172.21.15.1:0/2020522522 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa34012e40 con 0x555a3a6fabb0 2024-09-11T21:57:36.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.978+0000 7ffa25ffb640 1 -- 172.21.15.1:0/2020522522 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7ffa340035f0 con 0x555a3a6fabb0 2024-09-11T21:57:36.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.978+0000 7ffa46700740 1 -- 172.21.15.1:0/2020522522 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa34022040 con 0x555a3a6fabb0 2024-09-11T21:57:36.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.978+0000 7ffa25ffb640 1 -- 172.21.15.1:0/2020522522 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ffa340037f0 con 0x555a3a6fabb0 2024-09-11T21:57:36.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.978+0000 7ffa46700740 10 client.15474 fetch_fsmap learned FSMap version 28 2024-09-11T21:57:36.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.978+0000 7ffa46700740 1 -- 172.21.15.1:0/2020522522 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555a3a77a0a0 con 0x555a3a6fabb0 2024-09-11T21:57:36.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.978+0000 7ffa25ffb640 1 -- 172.21.15.1:0/2020522522 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa34022040 con 0x555a3a6fabb0 2024-09-11T21:57:36.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.978+0000 7ffa25ffb640 1 -- 172.21.15.1:0/2020522522 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ffa34010770 con 0x555a3a6fabb0 2024-09-11T21:57:36.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.978+0000 7ffa46700740 10 client.15474 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T21:57:36.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.978+0000 7ffa46700740 10 client.15474 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-11T21:57:36.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.978+0000 7ffa46700740 20 client.15474 populate_metadata read hostname 'smithi001' 2024-09-11T21:57:36.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.978+0000 7ffa46700740 1 --2- 172.21.15.1:0/2020522522 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x555a3a77eb60 0x555a3a77d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:36.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.978+0000 7ffa46700740 4 client.15474 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T21:57:36.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.978+0000 7ffa46700740 1 -- 172.21.15.1:0/2020522522 --> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] -- command(tid 0: {"prefix": "status"}) -- 0x555a3a65db10 con 0x555a3a77eb60 2024-09-11T21:57:36.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.979+0000 7ffa420c3640 1 --2- 172.21.15.1:0/2020522522 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x555a3a77eb60 0x555a3a77d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:36.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.979+0000 7ffa420c3640 1 --2- 172.21.15.1:0/2020522522 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x555a3a77eb60 0x555a3a77d870 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1793ff378c618661 server_cookie=ec2e099e543ea7b7 in_seq=0 out_seq=0 2024-09-11T21:57:36.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.979+0000 7ffa25ffb640 10 client.15474 ms_handle_connect on v2:172.21.15.161:6834/828381592 2024-09-11T21:57:36.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.979+0000 7ffa25ffb640 1 -- 172.21.15.1:0/2020522522 <== mds.4 v2:172.21.15.161:6834/828381592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x555a3a65db10 con 0x555a3a77eb60 2024-09-11T21:57:36.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.979+0000 7ffa25ffb640 10 client.15474 handle_command_reply: tid=0 2024-09-11T21:57:36.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.979+0000 7ffa46700740 1 client.15474 shutdown 2024-09-11T21:57:36.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.979+0000 7ffa46700740 1 -- 172.21.15.1:0/2020522522 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555a3a6fabb0 msgr2=0x555a3a6f44f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:36.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.979+0000 7ffa46700740 1 --2- 172.21.15.1:0/2020522522 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555a3a6fabb0 0x555a3a6f44f0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ffa34003b10 tx=0x7ffa3400fd20 comp rx=0 tx=0).stop 2024-09-11T21:57:36.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.979+0000 7ffa46700740 1 -- 172.21.15.1:0/2020522522 shutdown_connections 2024-09-11T21:57:36.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.979+0000 7ffa46700740 1 --2- 172.21.15.1:0/2020522522 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555a3a6fabb0 0x555a3a6f44f0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:36.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.979+0000 7ffa46700740 1 --2- 172.21.15.1:0/2020522522 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x555a3a77eb60 0x555a3a77d870 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:36.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.979+0000 7ffa46700740 1 --2- 172.21.15.1:0/2020522522 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555a3a6f02f0 0x555a3a6f3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:36.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.979+0000 7ffa46700740 1 --2- 172.21.15.1:0/2020522522 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555a3a6ef9d0 0x555a3a6f3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:36.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.979+0000 7ffa25ffb640 0 client.15474 ms_handle_reset on v2:172.21.15.161:6834/828381592 2024-09-11T21:57:36.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.979+0000 7ffa46700740 1 -- 172.21.15.1:0/2020522522 >> 172.21.15.1:0/2020522522 conn(0x555a3a6ebf60 msgr2=0x555a3a6ea7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:36.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.979+0000 7ffa46700740 1 -- 172.21.15.1:0/2020522522 shutdown_connections 2024-09-11T21:57:36.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.979+0000 7ffa46700740 1 -- 172.21.15.1:0/2020522522 wait complete. 2024-09-11T21:57:36.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.980+0000 7ffa46700740 20 client.15474 trim_cache size 0 max 16384 2024-09-11T21:57:36.992 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-11T21:57:36.993 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "3ffca83c-7087-11ef-bcea-c7b262605968", 2024-09-11T21:57:36.993 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 4, 2024-09-11T21:57:36.993 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 24460, 2024-09-11T21:57:36.993 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-09-11T21:57:36.993 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-09-11T21:57:36.993 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-09-11T21:57:36.993 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 125.831845118, 2024-09-11T21:57:36.993 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 28, 2024-09-11T21:57:36.993 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 87, 2024-09-11T21:57:36.993 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 86, 2024-09-11T21:57:36.993 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 163.294826605 2024-09-11T21:57:36.993 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-11T21:57:36.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.980+0000 7ffa44327640 1 -- 172.21.15.1:0/1808620674 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7ffa280a2d80 msgr2=0x7ffa280a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:36.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.980+0000 7ffa44327640 1 --2- 172.21.15.1:0/1808620674 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7ffa280a2d80 0x7ffa280a5240 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ffa34013000 tx=0x7ffa3401f000 comp rx=0 tx=0).stop 2024-09-11T21:57:36.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.980+0000 7ffa44327640 1 -- 172.21.15.1:0/1808620674 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffa3c100350 msgr2=0x7ffa3c198500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:36.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.980+0000 7ffa44327640 1 --2- 172.21.15.1:0/1808620674 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffa3c100350 0x7ffa3c198500 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ffa3800ebc0 tx=0x7ffa3800c2d0 comp rx=0 tx=0).stop 2024-09-11T21:57:36.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.980+0000 7ffa44327640 1 -- 172.21.15.1:0/1808620674 shutdown_connections 2024-09-11T21:57:36.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.980+0000 7ffa44327640 1 --2- 172.21.15.1:0/1808620674 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7ffa280a2d80 0x7ffa280a5240 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:36.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.980+0000 7ffa44327640 1 --2- 172.21.15.1:0/1808620674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa3c106f50 0x7ffa3c19cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:36.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.980+0000 7ffa44327640 1 --2- 172.21.15.1:0/1808620674 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffa3c100d00 0x7ffa3c198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:36.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.980+0000 7ffa44327640 1 --2- 172.21.15.1:0/1808620674 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffa3c100350 0x7ffa3c198500 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:36.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.980+0000 7ffa44327640 1 -- 172.21.15.1:0/1808620674 >> 172.21.15.1:0/1808620674 conn(0x7ffa3c077210 msgr2=0x7ffa3c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:36.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.980+0000 7ffa44327640 1 -- 172.21.15.1:0/1808620674 shutdown_connections 2024-09-11T21:57:36.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:36.980+0000 7ffa44327640 1 -- 172.21.15.1:0/1808620674 wait complete. 2024-09-11T21:57:37.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:36 smithi001 ceph-mon[30935]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 77 KiB/s wr, 44 op/s 2024-09-11T21:57:37.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:36 smithi053 ceph-mon[34908]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 77 KiB/s wr, 44 op/s 2024-09-11T21:57:37.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:36 smithi161 ceph-mon[35615]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 77 KiB/s wr, 44 op/s 2024-09-11T21:57:37.707 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.c status 2024-09-11T21:57:37.966 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:57:38.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.511+0000 7f7ecc96c640 1 -- 172.21.15.1:0/2550557035 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ec8102da0 msgr2=0x7f7ec8103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:38.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.511+0000 7f7ecc96c640 1 --2- 172.21.15.1:0/2550557035 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ec8102da0 0x7f7ec8103180 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8009900 tx=0x7f7eb802a480 comp rx=0 tx=0).stop 2024-09-11T21:57:38.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.511+0000 7f7ecc96c640 1 -- 172.21.15.1:0/2550557035 shutdown_connections 2024-09-11T21:57:38.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.511+0000 7f7ecc96c640 1 --2- 172.21.15.1:0/2550557035 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ec810c180 0x7f7ec810e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:38.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.511+0000 7f7ecc96c640 1 --2- 172.21.15.1:0/2550557035 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ec81036c0 0x7f7ec810bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:38.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.511+0000 7f7ecc96c640 1 --2- 172.21.15.1:0/2550557035 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ec8102da0 0x7f7ec8103180 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:38.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.511+0000 7f7ecc96c640 1 -- 172.21.15.1:0/2550557035 >> 172.21.15.1:0/2550557035 conn(0x7f7ec80fc7a0 msgr2=0x7f7ec80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:38.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.511+0000 7f7ecc96c640 1 -- 172.21.15.1:0/2550557035 shutdown_connections 2024-09-11T21:57:38.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.511+0000 7f7ecc96c640 1 -- 172.21.15.1:0/2550557035 wait complete. 2024-09-11T21:57:38.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.512+0000 7f7ecc96c640 1 Processor -- start 2024-09-11T21:57:38.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.512+0000 7f7ecc96c640 1 -- start start 2024-09-11T21:57:38.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.512+0000 7f7ecc96c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ec8102da0 0x7f7ec81983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:38.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.512+0000 7f7ecc96c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ec81036c0 0x7f7ec8198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:38.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.512+0000 7f7ecc96c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ec810c180 0x7f7ec819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:38.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.512+0000 7f7ecc96c640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec8111fb0 con 0x7f7ec81036c0 2024-09-11T21:57:38.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.512+0000 7f7ecc96c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec8111e30 con 0x7f7ec810c180 2024-09-11T21:57:38.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.512+0000 7f7ecc96c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec8112130 con 0x7f7ec8102da0 2024-09-11T21:57:38.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.512+0000 7f7ebffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ec81036c0 0x7f7ec8198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:38.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.512+0000 7f7ebffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ec81036c0 0x7f7ec8198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33338/0 (socket says 172.21.15.1:33338) 2024-09-11T21:57:38.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.512+0000 7f7ebffff640 1 -- 172.21.15.1:0/4168625845 learned_addr learned my addr 172.21.15.1:0/4168625845 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:38.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.513+0000 7f7ec6d9d640 1 --2- 172.21.15.1:0/4168625845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ec810c180 0x7f7ec819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:38.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.513+0000 7f7ec659c640 1 --2- 172.21.15.1:0/4168625845 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ec8102da0 0x7f7ec81983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:38.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.513+0000 7f7ebffff640 1 -- 172.21.15.1:0/4168625845 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ec8102da0 msgr2=0x7f7ec81983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:38.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.513+0000 7f7ebffff640 1 --2- 172.21.15.1:0/4168625845 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ec8102da0 0x7f7ec81983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:38.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.513+0000 7f7ebffff640 1 -- 172.21.15.1:0/4168625845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ec810c180 msgr2=0x7f7ec819cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:38.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.513+0000 7f7ebffff640 1 --2- 172.21.15.1:0/4168625845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ec810c180 0x7f7ec819cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:38.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.513+0000 7f7ebffff640 1 -- 172.21.15.1:0/4168625845 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ec819d380 con 0x7f7ec81036c0 2024-09-11T21:57:38.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.513+0000 7f7ebffff640 1 --2- 172.21.15.1:0/4168625845 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ec81036c0 0x7f7ec8198910 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f7eb000d950 tx=0x7f7eb000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:38.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.513+0000 7f7ebf7fe640 1 -- 172.21.15.1:0/4168625845 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb0017030 con 0x7f7ec81036c0 2024-09-11T21:57:38.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.513+0000 7f7ebf7fe640 1 -- 172.21.15.1:0/4168625845 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7eb0004020 con 0x7f7ec81036c0 2024-09-11T21:57:38.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.513+0000 7f7ecc96c640 1 -- 172.21.15.1:0/4168625845 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ec819d670 con 0x7f7ec81036c0 2024-09-11T21:57:38.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.513+0000 7f7ebf7fe640 1 -- 172.21.15.1:0/4168625845 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb00047a0 con 0x7f7ec81036c0 2024-09-11T21:57:38.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.513+0000 7f7ecc96c640 1 -- 172.21.15.1:0/4168625845 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ec81a4f50 con 0x7f7ec81036c0 2024-09-11T21:57:38.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.515+0000 7f7ebf7fe640 1 -- 172.21.15.1:0/4168625845 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 24) ==== 138008+0+0 (secure 0 0 0) 0x7f7eb00041d0 con 0x7f7ec81036c0 2024-09-11T21:57:38.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.516+0000 7f7eced45740 1 Processor -- start 2024-09-11T21:57:38.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.516+0000 7f7eced45740 1 -- start start 2024-09-11T21:57:38.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.516+0000 7f7eced45740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d755cb9d0 0x562d755cbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:38.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.516+0000 7f7eced45740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d755cc2f0 0x562d755d6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:38.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.516+0000 7f7eced45740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d755d6bb0 0x562d755d8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:38.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.516+0000 7f7eced45740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x562d75532090 con 0x562d755d6bb0 2024-09-11T21:57:38.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.516+0000 7f7eced45740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x562d754d9a70 con 0x562d755cb9d0 2024-09-11T21:57:38.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.516+0000 7f7eced45740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x562d7552e050 con 0x562d755cc2f0 2024-09-11T21:57:38.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.516+0000 7f7ebf7fe640 1 --2- 172.21.15.1:0/4168625845 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f7eb40a2d80 0x7f7eb40a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:38.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.516+0000 7f7ebffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d755d6bb0 0x562d755d8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:38.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.517+0000 7f7ec6d9d640 1 --2- 172.21.15.1:0/4168625845 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f7eb40a2d80 0x7f7eb40a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:38.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.517+0000 7f7ec6d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d755cb9d0 0x562d755cbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:38.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.517+0000 7f7ebffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d755d6bb0 0x562d755d8fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33346/0 (socket says 172.21.15.1:33346) 2024-09-11T21:57:38.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.517+0000 7f7ebffff640 1 -- 172.21.15.1:0/1723582558 learned_addr learned my addr 172.21.15.1:0/1723582558 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:38.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.517+0000 7f7ebf7fe640 1 -- 172.21.15.1:0/4168625845 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7f7eb00cefd0 con 0x7f7ec81036c0 2024-09-11T21:57:38.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.517+0000 7f7ec6d9d640 1 -- 172.21.15.1:0/1723582558 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d755cc2f0 msgr2=0x562d755d6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:38.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.517+0000 7f7ec6d9d640 1 --2- 172.21.15.1:0/1723582558 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d755cc2f0 0x562d755d6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:38.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.517+0000 7f7ec6d9d640 1 -- 172.21.15.1:0/1723582558 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d755d6bb0 msgr2=0x562d755d8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:38.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.517+0000 7f7ec6d9d640 1 --2- 172.21.15.1:0/1723582558 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d755d6bb0 0x562d755d8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:38.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.517+0000 7f7ec6d9d640 1 -- 172.21.15.1:0/1723582558 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d754e6130 con 0x562d755cb9d0 2024-09-11T21:57:38.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.517+0000 7f7ebffff640 1 --2- 172.21.15.1:0/1723582558 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d755d6bb0 0x562d755d8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T21:57:38.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.517+0000 7f7ec6d9d640 1 --2- 172.21.15.1:0/4168625845 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f7eb40a2d80 0x7f7eb40a5240 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0007fd0 tx=0x7f7ec000c000 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:38.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.517+0000 7f7ec6d9d640 1 --2- 172.21.15.1:0/1723582558 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d755cb9d0 0x562d755cbdb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0014dd0 tx=0x7f7ec00117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e87fe3e6a6be7b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:38.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.517+0000 7f7ebcff9640 1 -- 172.21.15.1:0/1723582558 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec000ee20 con 0x562d755cb9d0 2024-09-11T21:57:38.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.518+0000 7f7ebcff9640 1 -- 172.21.15.1:0/1723582558 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ec0007ba0 con 0x562d755cb9d0 2024-09-11T21:57:38.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.518+0000 7f7ebcff9640 1 -- 172.21.15.1:0/1723582558 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec0004490 con 0x562d755cb9d0 2024-09-11T21:57:38.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.518+0000 7f7eced45740 1 -- 172.21.15.1:0/1723582558 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d755cb9d0 msgr2=0x562d755cbdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:38.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.518+0000 7f7eced45740 1 --2- 172.21.15.1:0/1723582558 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d755cb9d0 0x562d755cbdb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0014dd0 tx=0x7f7ec00117b0 comp rx=0 tx=0).stop 2024-09-11T21:57:38.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.518+0000 7f7eced45740 1 -- 172.21.15.1:0/1723582558 shutdown_connections 2024-09-11T21:57:38.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.518+0000 7f7eced45740 1 --2- 172.21.15.1:0/1723582558 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d755d6bb0 0x562d755d8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:38.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.518+0000 7f7eced45740 1 --2- 172.21.15.1:0/1723582558 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d755cc2f0 0x562d755d6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:38.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.518+0000 7f7eced45740 1 --2- 172.21.15.1:0/1723582558 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d755cb9d0 0x562d755cbdb0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:38.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.518+0000 7f7eced45740 1 -- 172.21.15.1:0/1723582558 >> 172.21.15.1:0/1723582558 conn(0x562d755c7f60 msgr2=0x562d755c8340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:38.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.518+0000 7f7eced45740 1 -- 172.21.15.1:0/1723582558 shutdown_connections 2024-09-11T21:57:38.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.518+0000 7f7eced45740 1 -- 172.21.15.1:0/1723582558 wait complete. 2024-09-11T21:57:38.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.519+0000 7f7eced45740 1 -- start start 2024-09-11T21:57:38.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.519+0000 7f7eced45740 1 Processor -- start 2024-09-11T21:57:38.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.519+0000 7f7eced45740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d755cb9d0 0x562d755d64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:38.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.519+0000 7f7eced45740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d755cc2f0 0x562d755d3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:38.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.519+0000 7f7eced45740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d755d6bb0 0x562d755d3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:38.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.519+0000 7f7eced45740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x562d755d9720 con 0x562d755cc2f0 2024-09-11T21:57:38.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.519+0000 7f7eced45740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x562d754ecf20 con 0x562d755cb9d0 2024-09-11T21:57:38.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.519+0000 7f7eced45740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x562d755d98a0 con 0x562d755d6bb0 2024-09-11T21:57:38.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.519+0000 7f7eced45740 10 client.0 _ll_register_callbacks cb 0x562d754b6030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:57:38.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.519+0000 7f7ebffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d755cc2f0 0x562d755d3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:38.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.519+0000 7f7ebffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d755cc2f0 0x562d755d3830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33358/0 (socket says 172.21.15.1:33358) 2024-09-11T21:57:38.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.519+0000 7f7ebffff640 1 -- 172.21.15.1:0/3722125419 learned_addr learned my addr 172.21.15.1:0/3722125419 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:38.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.519+0000 7f7ec659c640 1 --2- 172.21.15.1:0/3722125419 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d755cb9d0 0x562d755d64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:38.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.520+0000 7f7ebffff640 1 -- 172.21.15.1:0/3722125419 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d755d6bb0 msgr2=0x562d755d3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:38.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.520+0000 7f7ebffff640 1 --2- 172.21.15.1:0/3722125419 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d755d6bb0 0x562d755d3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:38.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.520+0000 7f7ebffff640 1 -- 172.21.15.1:0/3722125419 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d755cb9d0 msgr2=0x562d755d64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:38.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.520+0000 7f7ebffff640 1 --2- 172.21.15.1:0/3722125419 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d755cb9d0 0x562d755d64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:38.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.520+0000 7f7ebffff640 1 -- 172.21.15.1:0/3722125419 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d752a32b0 con 0x562d755cc2f0 2024-09-11T21:57:38.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.520+0000 7f7ec659c640 1 --2- 172.21.15.1:0/3722125419 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d755cb9d0 0x562d755d64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:57:38.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.520+0000 7f7ebffff640 1 --2- 172.21.15.1:0/3722125419 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d755cc2f0 0x562d755d3830 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0097790 tx=0x7f7eb006a2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1f93a707af6405c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:38.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.520+0000 7f7eadffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T21:57:38.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.520+0000 7f7eadffb640 1 -- 172.21.15.1:0/3722125419 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb0099b30 con 0x562d755cc2f0 2024-09-11T21:57:38.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.520+0000 7f7eadffb640 1 -- 172.21.15.1:0/3722125419 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7f7eb00cc910 con 0x562d755cc2f0 2024-09-11T21:57:38.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.520+0000 7f7eced45740 1 -- 172.21.15.1:0/3722125419 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ec0007ba0 con 0x562d755cc2f0 2024-09-11T21:57:38.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.520+0000 7f7eadffb640 1 -- 172.21.15.1:0/3722125419 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7eb00a2cf0 con 0x562d755cc2f0 2024-09-11T21:57:38.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.520+0000 7f7eced45740 10 client.15486 fetch_fsmap learned FSMap version 28 2024-09-11T21:57:38.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.520+0000 7f7eced45740 1 -- 172.21.15.1:0/3722125419 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562d755e50d0 con 0x562d755cc2f0 2024-09-11T21:57:38.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.520+0000 7f7eadffb640 1 -- 172.21.15.1:0/3722125419 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ec0007ba0 con 0x562d755cc2f0 2024-09-11T21:57:38.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.520+0000 7f7eadffb640 1 -- 172.21.15.1:0/3722125419 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7eb006d020 con 0x562d755cc2f0 2024-09-11T21:57:38.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.520+0000 7f7eced45740 10 client.15486 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T21:57:38.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.520+0000 7f7eced45740 10 client.15486 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-11T21:57:38.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.521+0000 7f7eced45740 20 client.15486 populate_metadata read hostname 'smithi001' 2024-09-11T21:57:38.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.521+0000 7f7eced45740 1 --2- 172.21.15.1:0/3722125419 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x562d755e7780 0x562d755e8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:38.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.521+0000 7f7eced45740 4 client.15486 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T21:57:38.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.521+0000 7f7eced45740 1 -- 172.21.15.1:0/3722125419 --> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d7553f600 con 0x562d755e7780 2024-09-11T21:57:38.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.521+0000 7f7ec659c640 1 --2- 172.21.15.1:0/3722125419 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x562d755e7780 0x562d755e8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:38.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.521+0000 7f7eadffb640 10 client.15486 ms_handle_connect on v2:172.21.15.161:6836/1799318656 2024-09-11T21:57:38.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.521+0000 7f7ec659c640 1 --2- 172.21.15.1:0/3722125419 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x562d755e7780 0x562d755e8bf0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fc15131526df596d server_cookie=61b77f34a5244f27 in_seq=0 out_seq=0 2024-09-11T21:57:38.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.522+0000 7f7eadffb640 1 -- 172.21.15.1:0/3722125419 <== mds.3 v2:172.21.15.161:6836/1799318656 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562d7553f600 con 0x562d755e7780 2024-09-11T21:57:38.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.522+0000 7f7eadffb640 10 client.15486 handle_command_reply: tid=0 2024-09-11T21:57:38.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.522+0000 7f7eced45740 1 client.15486 shutdown 2024-09-11T21:57:38.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.522+0000 7f7eced45740 1 -- 172.21.15.1:0/3722125419 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d755cc2f0 msgr2=0x562d755d3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:38.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.522+0000 7f7eced45740 1 --2- 172.21.15.1:0/3722125419 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d755cc2f0 0x562d755d3830 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0097790 tx=0x7f7eb006a2b0 comp rx=0 tx=0).stop 2024-09-11T21:57:38.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.522+0000 7f7eced45740 1 -- 172.21.15.1:0/3722125419 shutdown_connections 2024-09-11T21:57:38.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.522+0000 7f7eced45740 1 --2- 172.21.15.1:0/3722125419 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x562d755e7780 0x562d755e8bf0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:38.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.522+0000 7f7eced45740 1 --2- 172.21.15.1:0/3722125419 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d755d6bb0 0x562d755d3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:38.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.522+0000 7f7eced45740 1 --2- 172.21.15.1:0/3722125419 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d755cc2f0 0x562d755d3830 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:38.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.522+0000 7f7eced45740 1 --2- 172.21.15.1:0/3722125419 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d755cb9d0 0x562d755d64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:38.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.522+0000 7f7eced45740 1 -- 172.21.15.1:0/3722125419 >> 172.21.15.1:0/3722125419 conn(0x562d755c7f60 msgr2=0x562d755cdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:38.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.522+0000 7f7eadffb640 0 client.15486 ms_handle_reset on v2:172.21.15.161:6836/1799318656 2024-09-11T21:57:38.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.522+0000 7f7eced45740 1 -- 172.21.15.1:0/3722125419 shutdown_connections 2024-09-11T21:57:38.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.522+0000 7f7eced45740 1 -- 172.21.15.1:0/3722125419 wait complete. 2024-09-11T21:57:38.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.523+0000 7f7eced45740 20 client.15486 trim_cache size 0 max 16384 2024-09-11T21:57:38.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.540+0000 7f7eced45740 1 Processor -- start 2024-09-11T21:57:38.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.540+0000 7f7eced45740 1 -- start start 2024-09-11T21:57:38.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.540+0000 7f7eced45740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d755cb9d0 0x562d755e7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:38.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.540+0000 7f7eced45740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d755cc2f0 0x562d755d37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:38.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.541+0000 7f7eced45740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d755d6bb0 0x562d755d3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:38.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.541+0000 7f7eced45740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x562d755e6f30 con 0x562d755d6bb0 2024-09-11T21:57:38.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.541+0000 7f7eced45740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x562d755e7600 con 0x562d755cc2f0 2024-09-11T21:57:38.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.541+0000 7f7eced45740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x562d755e7cd0 con 0x562d755cb9d0 2024-09-11T21:57:38.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.541+0000 7f7ec6d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d755d6bb0 0x562d755d3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:38.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.541+0000 7f7ec6d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d755d6bb0 0x562d755d3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33364/0 (socket says 172.21.15.1:33364) 2024-09-11T21:57:38.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.541+0000 7f7ec6d9d640 1 -- 172.21.15.1:0/113542143 learned_addr learned my addr 172.21.15.1:0/113542143 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:38.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.541+0000 7f7ebffff640 1 --2- 172.21.15.1:0/113542143 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d755cc2f0 0x562d755d37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:38.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.541+0000 7f7ec659c640 1 --2- 172.21.15.1:0/113542143 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d755cb9d0 0x562d755e7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:38.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.541+0000 7f7ec6d9d640 1 -- 172.21.15.1:0/113542143 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d755cb9d0 msgr2=0x562d755e7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:38.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.541+0000 7f7ec6d9d640 1 --2- 172.21.15.1:0/113542143 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d755cb9d0 0x562d755e7e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:38.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.541+0000 7f7ec6d9d640 1 -- 172.21.15.1:0/113542143 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d755cc2f0 msgr2=0x562d755d37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:38.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.541+0000 7f7ec6d9d640 1 --2- 172.21.15.1:0/113542143 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d755cc2f0 0x562d755d37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:38.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.541+0000 7f7ec6d9d640 1 -- 172.21.15.1:0/113542143 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d755d6850 con 0x562d755d6bb0 2024-09-11T21:57:38.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.541+0000 7f7ec659c640 1 --2- 172.21.15.1:0/113542143 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d755cb9d0 0x562d755e7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T21:57:38.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.541+0000 7f7ec6d9d640 1 --2- 172.21.15.1:0/113542143 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d755d6bb0 0x562d755d3d30 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0011cb0 tx=0x7f7ec00071c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b35613c86304169a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:38.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.541+0000 7f7eaffff640 1 -- 172.21.15.1:0/113542143 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec002e070 con 0x562d755d6bb0 2024-09-11T21:57:38.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.541+0000 7f7eaffff640 1 -- 172.21.15.1:0/113542143 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ec0003e50 con 0x562d755d6bb0 2024-09-11T21:57:38.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.541+0000 7f7eaffff640 1 -- 172.21.15.1:0/113542143 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec00054f0 con 0x562d755d6bb0 2024-09-11T21:57:38.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.542+0000 7f7eced45740 1 -- 172.21.15.1:0/113542143 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d755d6bb0 msgr2=0x562d755d3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:38.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.542+0000 7f7eced45740 1 --2- 172.21.15.1:0/113542143 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d755d6bb0 0x562d755d3d30 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0011cb0 tx=0x7f7ec00071c0 comp rx=0 tx=0).stop 2024-09-11T21:57:38.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.542+0000 7f7eced45740 1 -- 172.21.15.1:0/113542143 shutdown_connections 2024-09-11T21:57:38.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.542+0000 7f7eced45740 1 --2- 172.21.15.1:0/113542143 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d755d6bb0 0x562d755d3d30 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:38.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.542+0000 7f7eced45740 1 --2- 172.21.15.1:0/113542143 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d755cc2f0 0x562d755d37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:38.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.542+0000 7f7eced45740 1 --2- 172.21.15.1:0/113542143 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d755cb9d0 0x562d755e7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:38.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.542+0000 7f7eced45740 1 -- 172.21.15.1:0/113542143 >> 172.21.15.1:0/113542143 conn(0x562d755c7f60 msgr2=0x562d755c2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:38.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.542+0000 7f7eced45740 1 -- 172.21.15.1:0/113542143 shutdown_connections 2024-09-11T21:57:38.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.542+0000 7f7eced45740 1 -- 172.21.15.1:0/113542143 wait complete. 2024-09-11T21:57:38.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.542+0000 7f7eced45740 1 -- start start 2024-09-11T21:57:38.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.542+0000 7f7eced45740 1 Processor -- start 2024-09-11T21:57:38.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.543+0000 7f7eced45740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d755cb9d0 0x562d755ec4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:38.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.543+0000 7f7eced45740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d755cc2f0 0x562d755eca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:38.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.543+0000 7f7eced45740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d755d6bb0 0x562d7564fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:38.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.543+0000 7f7eced45740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x562d755e8a70 con 0x562d755cb9d0 2024-09-11T21:57:38.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.543+0000 7f7eced45740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x562d755e83a0 con 0x562d755d6bb0 2024-09-11T21:57:38.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.543+0000 7f7eced45740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x562d755e9140 con 0x562d755cc2f0 2024-09-11T21:57:38.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.543+0000 7f7ec659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d755cb9d0 0x562d755ec4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:38.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.543+0000 7f7eced45740 10 client.0 _ll_register_callbacks cb 0x7f7eb009b210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:57:38.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.543+0000 7f7ec659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d755cb9d0 0x562d755ec4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33376/0 (socket says 172.21.15.1:33376) 2024-09-11T21:57:38.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.543+0000 7f7ec659c640 1 -- 172.21.15.1:0/4221426784 learned_addr learned my addr 172.21.15.1:0/4221426784 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:38.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.543+0000 7f7ec6d9d640 1 --2- 172.21.15.1:0/4221426784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d755d6bb0 0x562d7564fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:38.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.543+0000 7f7ec659c640 1 -- 172.21.15.1:0/4221426784 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d755cc2f0 msgr2=0x562d755eca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:38.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.543+0000 7f7ec659c640 1 --2- 172.21.15.1:0/4221426784 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d755cc2f0 0x562d755eca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:38.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.543+0000 7f7ec659c640 1 -- 172.21.15.1:0/4221426784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d755d6bb0 msgr2=0x562d7564fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:38.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.543+0000 7f7ec659c640 1 --2- 172.21.15.1:0/4221426784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d755d6bb0 0x562d7564fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:38.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.543+0000 7f7ec659c640 1 -- 172.21.15.1:0/4221426784 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d75650470 con 0x562d755cb9d0 2024-09-11T21:57:38.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.543+0000 7f7ec6d9d640 1 --2- 172.21.15.1:0/4221426784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d755d6bb0 0x562d7564fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T21:57:38.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.543+0000 7f7ec659c640 1 --2- 172.21.15.1:0/4221426784 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d755cb9d0 0x562d755ec4d0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8009230 tx=0x7f7eb80044b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c090071add47f32b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:38.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.543+0000 7f7eadffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T21:57:38.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.544+0000 7f7eadffb640 1 -- 172.21.15.1:0/4221426784 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb8035040 con 0x562d755cb9d0 2024-09-11T21:57:38.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.544+0000 7f7eadffb640 1 -- 172.21.15.1:0/4221426784 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7f7eb8005250 con 0x562d755cb9d0 2024-09-11T21:57:38.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.544+0000 7f7eced45740 1 -- 172.21.15.1:0/4221426784 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ec0003e50 con 0x562d755cb9d0 2024-09-11T21:57:38.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.544+0000 7f7eadffb640 1 -- 172.21.15.1:0/4221426784 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7eb8051050 con 0x562d755cb9d0 2024-09-11T21:57:38.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.544+0000 7f7eced45740 10 client.15492 fetch_fsmap learned FSMap version 28 2024-09-11T21:57:38.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.544+0000 7f7eced45740 1 -- 172.21.15.1:0/4221426784 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562d75651720 con 0x562d755cb9d0 2024-09-11T21:57:38.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.544+0000 7f7eadffb640 1 -- 172.21.15.1:0/4221426784 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ec0003e50 con 0x562d755cb9d0 2024-09-11T21:57:38.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.544+0000 7f7eadffb640 1 -- 172.21.15.1:0/4221426784 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7eb8031740 con 0x562d755cb9d0 2024-09-11T21:57:38.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.544+0000 7f7eced45740 10 client.15492 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T21:57:38.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.544+0000 7f7eced45740 10 client.15492 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-11T21:57:38.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.544+0000 7f7eced45740 20 client.15492 populate_metadata read hostname 'smithi001' 2024-09-11T21:57:38.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.544+0000 7f7eced45740 1 --2- 172.21.15.1:0/4221426784 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x562d756563a0 0x562d756550b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:38.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.544+0000 7f7eced45740 4 client.15492 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T21:57:38.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.544+0000 7f7eced45740 1 -- 172.21.15.1:0/4221426784 --> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] -- command(tid 0: {"prefix": "status"}) -- 0x562d75539b10 con 0x562d756563a0 2024-09-11T21:57:38.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.545+0000 7f7ebffff640 1 --2- 172.21.15.1:0/4221426784 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x562d756563a0 0x562d756550b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:38.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.545+0000 7f7ebffff640 1 --2- 172.21.15.1:0/4221426784 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x562d756563a0 0x562d756550b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=423da23d2fd7672d server_cookie=b82aa14f8ce11994 in_seq=0 out_seq=0 2024-09-11T21:57:38.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.545+0000 7f7eadffb640 10 client.15492 ms_handle_connect on v2:172.21.15.161:6836/1799318656 2024-09-11T21:57:38.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.545+0000 7f7eadffb640 1 -- 172.21.15.1:0/4221426784 <== mds.3 v2:172.21.15.161:6836/1799318656 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x562d75539b10 con 0x562d756563a0 2024-09-11T21:57:38.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.545+0000 7f7eadffb640 10 client.15492 handle_command_reply: tid=0 2024-09-11T21:57:38.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.545+0000 7f7eced45740 1 client.15492 shutdown 2024-09-11T21:57:38.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.545+0000 7f7eced45740 1 -- 172.21.15.1:0/4221426784 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d755cb9d0 msgr2=0x562d755ec4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:38.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.545+0000 7f7eced45740 1 --2- 172.21.15.1:0/4221426784 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d755cb9d0 0x562d755ec4d0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8009230 tx=0x7f7eb80044b0 comp rx=0 tx=0).stop 2024-09-11T21:57:38.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.545+0000 7f7eced45740 1 -- 172.21.15.1:0/4221426784 shutdown_connections 2024-09-11T21:57:38.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.545+0000 7f7eced45740 1 --2- 172.21.15.1:0/4221426784 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x562d756563a0 0x562d756550b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:38.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.545+0000 7f7eadffb640 0 client.15492 ms_handle_reset on v2:172.21.15.161:6836/1799318656 2024-09-11T21:57:38.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.545+0000 7f7eced45740 1 --2- 172.21.15.1:0/4221426784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d755d6bb0 0x562d7564fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:38.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.545+0000 7f7eced45740 1 --2- 172.21.15.1:0/4221426784 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d755cc2f0 0x562d755eca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:38.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.545+0000 7f7eced45740 1 --2- 172.21.15.1:0/4221426784 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d755cb9d0 0x562d755ec4d0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:38.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.545+0000 7f7eced45740 1 -- 172.21.15.1:0/4221426784 >> 172.21.15.1:0/4221426784 conn(0x562d755c7f60 msgr2=0x562d755d37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:38.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.545+0000 7f7eced45740 1 -- 172.21.15.1:0/4221426784 shutdown_connections 2024-09-11T21:57:38.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.545+0000 7f7eced45740 1 -- 172.21.15.1:0/4221426784 wait complete. 2024-09-11T21:57:38.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.546+0000 7f7eced45740 20 client.15492 trim_cache size 0 max 16384 2024-09-11T21:57:38.553 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-11T21:57:38.553 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "3ffca83c-7087-11ef-bcea-c7b262605968", 2024-09-11T21:57:38.553 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 3, 2024-09-11T21:57:38.553 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 24466, 2024-09-11T21:57:38.553 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-09-11T21:57:38.554 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-09-11T21:57:38.554 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-09-11T21:57:38.554 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 128.40427925899999, 2024-09-11T21:57:38.554 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 28, 2024-09-11T21:57:38.554 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 87, 2024-09-11T21:57:38.554 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 86, 2024-09-11T21:57:38.554 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 162.22068335099999 2024-09-11T21:57:38.554 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-11T21:57:38.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.546+0000 7f7ecc96c640 1 -- 172.21.15.1:0/4168625845 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f7eb40a2d80 msgr2=0x7f7eb40a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:38.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.546+0000 7f7ecc96c640 1 --2- 172.21.15.1:0/4168625845 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f7eb40a2d80 0x7f7eb40a5240 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0007fd0 tx=0x7f7ec000c000 comp rx=0 tx=0).stop 2024-09-11T21:57:38.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.546+0000 7f7ecc96c640 1 -- 172.21.15.1:0/4168625845 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ec81036c0 msgr2=0x7f7ec8198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:38.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.546+0000 7f7ecc96c640 1 --2- 172.21.15.1:0/4168625845 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ec81036c0 0x7f7ec8198910 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f7eb000d950 tx=0x7f7eb000de10 comp rx=0 tx=0).stop 2024-09-11T21:57:38.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.546+0000 7f7ecc96c640 1 -- 172.21.15.1:0/4168625845 shutdown_connections 2024-09-11T21:57:38.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.546+0000 7f7ecc96c640 1 --2- 172.21.15.1:0/4168625845 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f7eb40a2d80 0x7f7eb40a5240 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:38.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.546+0000 7f7ecc96c640 1 --2- 172.21.15.1:0/4168625845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ec810c180 0x7f7ec819cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:38.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.546+0000 7f7ecc96c640 1 --2- 172.21.15.1:0/4168625845 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ec81036c0 0x7f7ec8198910 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:38.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.546+0000 7f7ecc96c640 1 --2- 172.21.15.1:0/4168625845 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ec8102da0 0x7f7ec81983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:38.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.546+0000 7f7ecc96c640 1 -- 172.21.15.1:0/4168625845 >> 172.21.15.1:0/4168625845 conn(0x7f7ec80fc7a0 msgr2=0x7f7ec80fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:38.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.546+0000 7f7ecc96c640 1 -- 172.21.15.1:0/4168625845 shutdown_connections 2024-09-11T21:57:38.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:38.546+0000 7f7ecc96c640 1 -- 172.21.15.1:0/4168625845 wait complete. 2024-09-11T21:57:39.185 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:38 smithi001 ceph-mon[30935]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 938 B/s wr, 8 op/s 2024-09-11T21:57:39.214 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.h status 2024-09-11T21:57:39.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:38 smithi053 ceph-mon[34908]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 938 B/s wr, 8 op/s 2024-09-11T21:57:39.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:38 smithi161 ceph-mon[35615]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 938 B/s wr, 8 op/s 2024-09-11T21:57:39.477 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:57:39.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.979+0000 7fa1fcb82640 1 -- 172.21.15.1:0/4078821575 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1f8103660 msgr2=0x7fa1f810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:39.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.979+0000 7fa1fcb82640 1 --2- 172.21.15.1:0/4078821575 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1f8103660 0x7fa1f810bb50 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec009900 tx=0x7fa1ec02a480 comp rx=0 tx=0).stop 2024-09-11T21:57:39.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.979+0000 7fa1fcb82640 1 -- 172.21.15.1:0/4078821575 shutdown_connections 2024-09-11T21:57:39.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.979+0000 7fa1fcb82640 1 --2- 172.21.15.1:0/4078821575 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1f810c170 0x7fa1f810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:39.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.979+0000 7fa1fcb82640 1 --2- 172.21.15.1:0/4078821575 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1f8103660 0x7fa1f810bb50 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:39.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.979+0000 7fa1fcb82640 1 --2- 172.21.15.1:0/4078821575 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1f8102d40 0x7fa1f8103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:39.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.979+0000 7fa1fcb82640 1 -- 172.21.15.1:0/4078821575 >> 172.21.15.1:0/4078821575 conn(0x7fa1f80fc7a0 msgr2=0x7fa1f80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:39.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.980+0000 7fa1fcb82640 1 -- 172.21.15.1:0/4078821575 shutdown_connections 2024-09-11T21:57:39.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.980+0000 7fa1fcb82640 1 -- 172.21.15.1:0/4078821575 wait complete. 2024-09-11T21:57:39.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.980+0000 7fa1fcb82640 1 Processor -- start 2024-09-11T21:57:39.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.980+0000 7fa1fcb82640 1 -- start start 2024-09-11T21:57:39.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.980+0000 7fa1fcb82640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1f8102d40 0x7fa1f819a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:39.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.980+0000 7fa1fcb82640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1f8103660 0x7fa1f819abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:39.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.981+0000 7fa1fcb82640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1f810c170 0x7fa1f819ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:39.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.981+0000 7fa1fcb82640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f8111fc0 con 0x7fa1f810c170 2024-09-11T21:57:39.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.981+0000 7fa1fcb82640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f8111e40 con 0x7fa1f8103660 2024-09-11T21:57:40.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.981+0000 7fa1fcb82640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f8112140 con 0x7fa1f8102d40 2024-09-11T21:57:40.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.981+0000 7fa1f6d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1f810c170 0x7fa1f819ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:40.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.981+0000 7fa1f6d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1f810c170 0x7fa1f819ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33394/0 (socket says 172.21.15.1:33394) 2024-09-11T21:57:40.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.981+0000 7fa1f6d9d640 1 -- 172.21.15.1:0/138922070 learned_addr learned my addr 172.21.15.1:0/138922070 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:40.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.981+0000 7fa1f659c640 1 --2- 172.21.15.1:0/138922070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1f8102d40 0x7fa1f819a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:40.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.981+0000 7fa1f5d9b640 1 --2- 172.21.15.1:0/138922070 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1f8103660 0x7fa1f819abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:40.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.981+0000 7fa1f6d9d640 1 -- 172.21.15.1:0/138922070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1f8102d40 msgr2=0x7fa1f819a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:40.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.981+0000 7fa1f6d9d640 1 --2- 172.21.15.1:0/138922070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1f8102d40 0x7fa1f819a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:40.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.981+0000 7fa1f6d9d640 1 -- 172.21.15.1:0/138922070 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1f8103660 msgr2=0x7fa1f819abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:40.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.981+0000 7fa1f6d9d640 1 --2- 172.21.15.1:0/138922070 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1f8103660 0x7fa1f819abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:40.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.981+0000 7fa1f6d9d640 1 -- 172.21.15.1:0/138922070 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1f819f630 con 0x7fa1f810c170 2024-09-11T21:57:40.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.981+0000 7fa1f5d9b640 1 --2- 172.21.15.1:0/138922070 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1f8103660 0x7fa1f819abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:57:40.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.981+0000 7fa1f6d9d640 1 --2- 172.21.15.1:0/138922070 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1f810c170 0x7fa1f819ef50 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fa1f000c970 tx=0x7fa1f000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:40.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.981+0000 7fa1e77fe640 1 -- 172.21.15.1:0/138922070 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f0007bb0 con 0x7fa1f810c170 2024-09-11T21:57:40.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.981+0000 7fa1e77fe640 1 -- 172.21.15.1:0/138922070 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1f0007d50 con 0x7fa1f810c170 2024-09-11T21:57:40.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.981+0000 7fa1fcb82640 1 -- 172.21.15.1:0/138922070 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1f819f920 con 0x7fa1f810c170 2024-09-11T21:57:40.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.981+0000 7fa1e77fe640 1 -- 172.21.15.1:0/138922070 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f000f450 con 0x7fa1f810c170 2024-09-11T21:57:40.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.982+0000 7fa1fcb82640 1 -- 172.21.15.1:0/138922070 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1f81a7200 con 0x7fa1f810c170 2024-09-11T21:57:40.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.984+0000 7fa1e77fe640 1 -- 172.21.15.1:0/138922070 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 24) ==== 138008+0+0 (secure 0 0 0) 0x7fa1f0004f20 con 0x7fa1f810c170 2024-09-11T21:57:40.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.984+0000 7fa1fef5b740 1 Processor -- start 2024-09-11T21:57:40.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.984+0000 7fa1fef5b740 1 -- start start 2024-09-11T21:57:40.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.984+0000 7fa1e77fe640 1 --2- 172.21.15.1:0/138922070 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fa1e00a2d80 0x7fa1e00a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:40.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.985+0000 7fa1fef5b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c33a019d0 0x555c33a01db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:40.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.985+0000 7fa1fef5b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c33a022f0 0x555c33a0c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:40.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.985+0000 7fa1fef5b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c33a0cbb0 0x555c33a0efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:40.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.985+0000 7fa1fef5b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x555c33968090 con 0x555c33a0cbb0 2024-09-11T21:57:40.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.985+0000 7fa1fef5b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x555c3390fa70 con 0x555c33a022f0 2024-09-11T21:57:40.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.985+0000 7fa1fef5b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x555c33964050 con 0x555c33a019d0 2024-09-11T21:57:40.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.985+0000 7fa1f5d9b640 1 --2- 172.21.15.1:0/138922070 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fa1e00a2d80 0x7fa1e00a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:40.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.985+0000 7fa1e77fe640 1 -- 172.21.15.1:0/138922070 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7fa1f00cf2b0 con 0x7fa1f810c170 2024-09-11T21:57:40.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.985+0000 7fa1f6d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c33a0cbb0 0x555c33a0efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:40.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.985+0000 7fa1f6d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c33a0cbb0 0x555c33a0efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33396/0 (socket says 172.21.15.1:33396) 2024-09-11T21:57:40.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.985+0000 7fa1f6d9d640 1 -- 172.21.15.1:0/3608695528 learned_addr learned my addr 172.21.15.1:0/3608695528 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:40.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.985+0000 7fa1f5d9b640 1 --2- 172.21.15.1:0/3608695528 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c33a022f0 0x555c33a0c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:40.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.985+0000 7fa1f659c640 1 --2- 172.21.15.1:0/3608695528 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c33a019d0 0x555c33a01db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:40.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.985+0000 7fa1f6d9d640 1 -- 172.21.15.1:0/3608695528 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c33a019d0 msgr2=0x555c33a01db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:40.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.985+0000 7fa1f6d9d640 1 --2- 172.21.15.1:0/3608695528 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c33a019d0 0x555c33a01db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:40.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.985+0000 7fa1f6d9d640 1 -- 172.21.15.1:0/3608695528 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c33a022f0 msgr2=0x555c33a0c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:40.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.985+0000 7fa1f6d9d640 1 --2- 172.21.15.1:0/3608695528 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c33a022f0 0x555c33a0c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:40.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.985+0000 7fa1f6d9d640 1 -- 172.21.15.1:0/3608695528 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c3391c130 con 0x555c33a0cbb0 2024-09-11T21:57:40.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.985+0000 7fa1f5d9b640 1 --2- 172.21.15.1:0/138922070 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fa1e00a2d80 0x7fa1e00a5240 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec000c00 tx=0x7fa1ec0041d0 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:40.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.985+0000 7fa1f6d9d640 1 --2- 172.21.15.1:0/3608695528 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c33a0cbb0 0x555c33a0efa0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0097ac0 tx=0x7fa1f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=687fd2291d7f6e49 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:40.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.986+0000 7fa1e4ff9640 1 -- 172.21.15.1:0/3608695528 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f001d040 con 0x555c33a0cbb0 2024-09-11T21:57:40.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.986+0000 7fa1e4ff9640 1 -- 172.21.15.1:0/3608695528 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1f009bd60 con 0x555c33a0cbb0 2024-09-11T21:57:40.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.986+0000 7fa1e4ff9640 1 -- 172.21.15.1:0/3608695528 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f0097cd0 con 0x555c33a0cbb0 2024-09-11T21:57:40.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.986+0000 7fa1fef5b740 1 -- 172.21.15.1:0/3608695528 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c33a0cbb0 msgr2=0x555c33a0efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:40.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.986+0000 7fa1fef5b740 1 --2- 172.21.15.1:0/3608695528 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c33a0cbb0 0x555c33a0efa0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0097ac0 tx=0x7fa1f00046c0 comp rx=0 tx=0).stop 2024-09-11T21:57:40.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.986+0000 7fa1fef5b740 1 -- 172.21.15.1:0/3608695528 shutdown_connections 2024-09-11T21:57:40.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.986+0000 7fa1fef5b740 1 --2- 172.21.15.1:0/3608695528 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c33a0cbb0 0x555c33a0efa0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:40.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.986+0000 7fa1fef5b740 1 --2- 172.21.15.1:0/3608695528 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c33a022f0 0x555c33a0c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:40.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.986+0000 7fa1fef5b740 1 --2- 172.21.15.1:0/3608695528 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c33a019d0 0x555c33a01db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:40.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.986+0000 7fa1fef5b740 1 -- 172.21.15.1:0/3608695528 >> 172.21.15.1:0/3608695528 conn(0x555c339fdf60 msgr2=0x555c339fe340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:40.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.987+0000 7fa1fef5b740 1 -- 172.21.15.1:0/3608695528 shutdown_connections 2024-09-11T21:57:40.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.987+0000 7fa1fef5b740 1 -- 172.21.15.1:0/3608695528 wait complete. 2024-09-11T21:57:40.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.987+0000 7fa1fef5b740 1 -- start start 2024-09-11T21:57:40.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.987+0000 7fa1fef5b740 1 Processor -- start 2024-09-11T21:57:40.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.987+0000 7fa1fef5b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c33a019d0 0x555c33a0c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:40.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.987+0000 7fa1fef5b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c33a022f0 0x555c33a09750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:40.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.987+0000 7fa1fef5b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c33a0cbb0 0x555c33a09c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:40.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.987+0000 7fa1fef5b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x555c33a0f780 con 0x555c33a0cbb0 2024-09-11T21:57:40.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.987+0000 7fa1fef5b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x555c33922f20 con 0x555c33a019d0 2024-09-11T21:57:40.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.987+0000 7fa1fef5b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x555c33a0f900 con 0x555c33a022f0 2024-09-11T21:57:40.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.987+0000 7fa1fef5b740 10 client.0 _ll_register_callbacks cb 0x555c338ec030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:57:40.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.987+0000 7fa1f6d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c33a0cbb0 0x555c33a09c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:40.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.987+0000 7fa1f6d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c33a0cbb0 0x555c33a09c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33404/0 (socket says 172.21.15.1:33404) 2024-09-11T21:57:40.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.987+0000 7fa1f6d9d640 1 -- 172.21.15.1:0/3874351954 learned_addr learned my addr 172.21.15.1:0/3874351954 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:40.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.987+0000 7fa1f659c640 1 --2- 172.21.15.1:0/3874351954 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c33a019d0 0x555c33a0c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:40.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.988+0000 7fa1f5d9b640 1 --2- 172.21.15.1:0/3874351954 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c33a022f0 0x555c33a09750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:40.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.988+0000 7fa1f6d9d640 1 -- 172.21.15.1:0/3874351954 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c33a022f0 msgr2=0x555c33a09750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:40.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.988+0000 7fa1f6d9d640 1 --2- 172.21.15.1:0/3874351954 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c33a022f0 0x555c33a09750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:40.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.988+0000 7fa1f6d9d640 1 -- 172.21.15.1:0/3874351954 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c33a019d0 msgr2=0x555c33a0c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:40.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.988+0000 7fa1f6d9d640 1 --2- 172.21.15.1:0/3874351954 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c33a019d0 0x555c33a0c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:40.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.988+0000 7fa1f6d9d640 1 -- 172.21.15.1:0/3874351954 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c336d92b0 con 0x555c33a0cbb0 2024-09-11T21:57:40.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.988+0000 7fa1f659c640 1 --2- 172.21.15.1:0/3874351954 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c33a019d0 0x555c33a0c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T21:57:40.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.988+0000 7fa1f5d9b640 1 --2- 172.21.15.1:0/3874351954 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c33a022f0 0x555c33a09750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:57:40.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.988+0000 7fa1f6d9d640 1 --2- 172.21.15.1:0/3874351954 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c33a0cbb0 0x555c33a09c90 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fa1f00cca20 tx=0x7fa1f00860d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2bbcb8a6373cc20 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:40.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.988+0000 7fa1ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T21:57:40.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.988+0000 7fa1ddffb640 1 -- 172.21.15.1:0/3874351954 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f00b4060 con 0x555c33a0cbb0 2024-09-11T21:57:40.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.988+0000 7fa1fef5b740 1 -- 172.21.15.1:0/3874351954 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1f009bd60 con 0x555c33a0cbb0 2024-09-11T21:57:40.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.988+0000 7fa1ddffb640 1 -- 172.21.15.1:0/3874351954 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7fa1f00ccbf0 con 0x555c33a0cbb0 2024-09-11T21:57:40.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.988+0000 7fa1ddffb640 1 -- 172.21.15.1:0/3874351954 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa1f001d040 con 0x555c33a0cbb0 2024-09-11T21:57:40.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.988+0000 7fa1fef5b740 10 client.15504 fetch_fsmap learned FSMap version 28 2024-09-11T21:57:40.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.988+0000 7fa1fef5b740 1 -- 172.21.15.1:0/3874351954 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555c33a0f780 con 0x555c33a0cbb0 2024-09-11T21:57:40.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.988+0000 7fa1ddffb640 1 -- 172.21.15.1:0/3874351954 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1f009bd60 con 0x555c33a0cbb0 2024-09-11T21:57:40.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.989+0000 7fa1ddffb640 1 -- 172.21.15.1:0/3874351954 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa1f0004050 con 0x555c33a0cbb0 2024-09-11T21:57:40.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.989+0000 7fa1fef5b740 10 client.15504 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T21:57:40.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.989+0000 7fa1fef5b740 10 client.15504 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-11T21:57:40.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.989+0000 7fa1fef5b740 20 client.15504 populate_metadata read hostname 'smithi001' 2024-09-11T21:57:40.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.989+0000 7fa1fef5b740 1 --2- 172.21.15.1:0/3874351954 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x555c33a1d160 0x555c33a1e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:40.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.989+0000 7fa1fef5b740 4 client.15504 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T21:57:40.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.989+0000 7fa1fef5b740 1 -- 172.21.15.1:0/3874351954 --> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c33975600 con 0x555c33a1d160 2024-09-11T21:57:40.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.989+0000 7fa1f659c640 1 --2- 172.21.15.1:0/3874351954 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x555c33a1d160 0x555c33a1e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:40.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.989+0000 7fa1ddffb640 10 client.15504 ms_handle_connect on v2:172.21.15.53:6836/3457131830 2024-09-11T21:57:40.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.989+0000 7fa1f659c640 1 --2- 172.21.15.1:0/3874351954 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x555c33a1d160 0x555c33a1e5d0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7273cf75d8a517c1 server_cookie=ae6f6ce570f1aaf5 in_seq=0 out_seq=0 2024-09-11T21:57:40.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.990+0000 7fa1ddffb640 1 -- 172.21.15.1:0/3874351954 <== mds.2 v2:172.21.15.53:6836/3457131830 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555c33975600 con 0x555c33a1d160 2024-09-11T21:57:40.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.990+0000 7fa1ddffb640 10 client.15504 handle_command_reply: tid=0 2024-09-11T21:57:40.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.990+0000 7fa1fef5b740 1 client.15504 shutdown 2024-09-11T21:57:40.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.990+0000 7fa1fef5b740 1 -- 172.21.15.1:0/3874351954 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c33a0cbb0 msgr2=0x555c33a09c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:40.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.990+0000 7fa1fef5b740 1 --2- 172.21.15.1:0/3874351954 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c33a0cbb0 0x555c33a09c90 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fa1f00cca20 tx=0x7fa1f00860d0 comp rx=0 tx=0).stop 2024-09-11T21:57:40.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.990+0000 7fa1fef5b740 1 -- 172.21.15.1:0/3874351954 shutdown_connections 2024-09-11T21:57:40.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.990+0000 7fa1fef5b740 1 --2- 172.21.15.1:0/3874351954 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x555c33a1d160 0x555c33a1e5d0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:40.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.990+0000 7fa1ddffb640 0 client.15504 ms_handle_reset on v2:172.21.15.53:6836/3457131830 2024-09-11T21:57:40.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.990+0000 7fa1fef5b740 1 --2- 172.21.15.1:0/3874351954 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c33a0cbb0 0x555c33a09c90 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:40.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.990+0000 7fa1fef5b740 1 --2- 172.21.15.1:0/3874351954 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c33a022f0 0x555c33a09750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:40.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.990+0000 7fa1fef5b740 1 --2- 172.21.15.1:0/3874351954 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c33a019d0 0x555c33a0c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:40.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.990+0000 7fa1fef5b740 1 -- 172.21.15.1:0/3874351954 >> 172.21.15.1:0/3874351954 conn(0x555c339fdf60 msgr2=0x555c33a03c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:40.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.990+0000 7fa1fef5b740 1 -- 172.21.15.1:0/3874351954 shutdown_connections 2024-09-11T21:57:40.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.990+0000 7fa1fef5b740 1 -- 172.21.15.1:0/3874351954 wait complete. 2024-09-11T21:57:40.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:39.990+0000 7fa1fef5b740 20 client.15504 trim_cache size 0 max 16384 2024-09-11T21:57:40.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.009+0000 7fa1fef5b740 1 Processor -- start 2024-09-11T21:57:40.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.009+0000 7fa1fef5b740 1 -- start start 2024-09-11T21:57:40.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.009+0000 7fa1fef5b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c33a019d0 0x555c33a1e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:40.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.009+0000 7fa1fef5b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c33a022f0 0x555c33a04cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:40.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.009+0000 7fa1fef5b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c33a0cbb0 0x555c33a069b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:40.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.009+0000 7fa1fef5b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x555c33a1c910 con 0x555c33a0cbb0 2024-09-11T21:57:40.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.009+0000 7fa1fef5b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x555c33a1cfe0 con 0x555c33a022f0 2024-09-11T21:57:40.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.009+0000 7fa1fef5b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x555c33a1d6b0 con 0x555c33a019d0 2024-09-11T21:57:40.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.009+0000 7fa1f6d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c33a0cbb0 0x555c33a069b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:40.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.009+0000 7fa1f6d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c33a0cbb0 0x555c33a069b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33420/0 (socket says 172.21.15.1:33420) 2024-09-11T21:57:40.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.009+0000 7fa1f6d9d640 1 -- 172.21.15.1:0/498883199 learned_addr learned my addr 172.21.15.1:0/498883199 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:40.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.009+0000 7fa1f659c640 1 --2- 172.21.15.1:0/498883199 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c33a019d0 0x555c33a1e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:40.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.009+0000 7fa1f5d9b640 1 --2- 172.21.15.1:0/498883199 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c33a022f0 0x555c33a04cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:40.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.009+0000 7fa1f6d9d640 1 -- 172.21.15.1:0/498883199 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c33a019d0 msgr2=0x555c33a1e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:40.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.009+0000 7fa1f6d9d640 1 --2- 172.21.15.1:0/498883199 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c33a019d0 0x555c33a1e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:40.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.009+0000 7fa1f6d9d640 1 -- 172.21.15.1:0/498883199 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c33a022f0 msgr2=0x555c33a04cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:40.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.009+0000 7fa1f6d9d640 1 --2- 172.21.15.1:0/498883199 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c33a022f0 0x555c33a04cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:40.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.009+0000 7fa1f6d9d640 1 -- 172.21.15.1:0/498883199 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c339fdd90 con 0x555c33a0cbb0 2024-09-11T21:57:40.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.010+0000 7fa1f6d9d640 1 --2- 172.21.15.1:0/498883199 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c33a0cbb0 0x555c33a069b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0097fe0 tx=0x7fa1f0099b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75999ed19aaeddee server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:40.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.010+0000 7fa1dffff640 1 -- 172.21.15.1:0/498883199 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f00b4050 con 0x555c33a0cbb0 2024-09-11T21:57:40.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.010+0000 7fa1dffff640 1 -- 172.21.15.1:0/498883199 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1f009b660 con 0x555c33a0cbb0 2024-09-11T21:57:40.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.010+0000 7fa1dffff640 1 -- 172.21.15.1:0/498883199 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f0060dc0 con 0x555c33a0cbb0 2024-09-11T21:57:40.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.010+0000 7fa1fef5b740 1 -- 172.21.15.1:0/498883199 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c33a0cbb0 msgr2=0x555c33a069b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:40.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.010+0000 7fa1fef5b740 1 --2- 172.21.15.1:0/498883199 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c33a0cbb0 0x555c33a069b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0097fe0 tx=0x7fa1f0099b70 comp rx=0 tx=0).stop 2024-09-11T21:57:40.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.011+0000 7fa1fef5b740 1 -- 172.21.15.1:0/498883199 shutdown_connections 2024-09-11T21:57:40.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.011+0000 7fa1fef5b740 1 --2- 172.21.15.1:0/498883199 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c33a0cbb0 0x555c33a069b0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:40.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.011+0000 7fa1fef5b740 1 --2- 172.21.15.1:0/498883199 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c33a022f0 0x555c33a04cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:40.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.011+0000 7fa1fef5b740 1 --2- 172.21.15.1:0/498883199 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c33a019d0 0x555c33a1e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:40.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.011+0000 7fa1fef5b740 1 -- 172.21.15.1:0/498883199 >> 172.21.15.1:0/498883199 conn(0x555c339fdf60 msgr2=0x555c33a045c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:40.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.011+0000 7fa1fef5b740 1 -- 172.21.15.1:0/498883199 shutdown_connections 2024-09-11T21:57:40.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.011+0000 7fa1fef5b740 1 -- 172.21.15.1:0/498883199 wait complete. 2024-09-11T21:57:40.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.011+0000 7fa1fef5b740 1 -- start start 2024-09-11T21:57:40.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.011+0000 7fa1fef5b740 1 Processor -- start 2024-09-11T21:57:40.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.011+0000 7fa1fef5b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c33a019d0 0x555c33a20780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:40.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.011+0000 7fa1fef5b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c33a022f0 0x555c33a20cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:40.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.011+0000 7fa1fef5b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c33a0cbb0 0x555c33a21200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:40.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.011+0000 7fa1fef5b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x555c33a1e450 con 0x555c33a022f0 2024-09-11T21:57:40.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.011+0000 7fa1fef5b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x555c33a1dd80 con 0x555c33a019d0 2024-09-11T21:57:40.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.011+0000 7fa1fef5b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x555c33a1eb20 con 0x555c33a0cbb0 2024-09-11T21:57:40.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.011+0000 7fa1fef5b740 10 client.0 _ll_register_callbacks cb 0x7fa1f009f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:57:40.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.011+0000 7fa1f5d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c33a022f0 0x555c33a20cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:40.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.011+0000 7fa1f5d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c33a022f0 0x555c33a20cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33428/0 (socket says 172.21.15.1:33428) 2024-09-11T21:57:40.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.011+0000 7fa1f5d9b640 1 -- 172.21.15.1:0/3735782681 learned_addr learned my addr 172.21.15.1:0/3735782681 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:40.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.012+0000 7fa1f6d9d640 1 --2- 172.21.15.1:0/3735782681 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c33a0cbb0 0x555c33a21200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:40.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.012+0000 7fa1f659c640 1 --2- 172.21.15.1:0/3735782681 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c33a019d0 0x555c33a20780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:40.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.012+0000 7fa1f5d9b640 1 -- 172.21.15.1:0/3735782681 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c33a0cbb0 msgr2=0x555c33a21200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:40.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.012+0000 7fa1f5d9b640 1 --2- 172.21.15.1:0/3735782681 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c33a0cbb0 0x555c33a21200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:40.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.012+0000 7fa1f5d9b640 1 -- 172.21.15.1:0/3735782681 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c33a019d0 msgr2=0x555c33a20780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:40.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.012+0000 7fa1f5d9b640 1 --2- 172.21.15.1:0/3735782681 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c33a019d0 0x555c33a20780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:40.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.012+0000 7fa1f5d9b640 1 -- 172.21.15.1:0/3735782681 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c339ff0f0 con 0x555c33a022f0 2024-09-11T21:57:40.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.012+0000 7fa1f5d9b640 1 --2- 172.21.15.1:0/3735782681 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c33a022f0 0x555c33a20cc0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec009230 tx=0x7fa1ec005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc37bb558cfc492e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:40.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.012+0000 7fa1ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T21:57:40.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.012+0000 7fa1ddffb640 1 -- 172.21.15.1:0/3735782681 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec035040 con 0x555c33a022f0 2024-09-11T21:57:40.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.012+0000 7fa1ddffb640 1 -- 172.21.15.1:0/3735782681 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7fa1ec031b00 con 0x555c33a022f0 2024-09-11T21:57:40.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.012+0000 7fa1fef5b740 1 -- 172.21.15.1:0/3735782681 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1f009b660 con 0x555c33a022f0 2024-09-11T21:57:40.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.012+0000 7fa1ddffb640 1 -- 172.21.15.1:0/3735782681 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa1ec031d00 con 0x555c33a022f0 2024-09-11T21:57:40.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.012+0000 7fa1fef5b740 10 client.15510 fetch_fsmap learned FSMap version 28 2024-09-11T21:57:40.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.012+0000 7fa1fef5b740 1 -- 172.21.15.1:0/3735782681 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555c33a87b70 con 0x555c33a022f0 2024-09-11T21:57:40.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.013+0000 7fa1ddffb640 1 -- 172.21.15.1:0/3735782681 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1f009b660 con 0x555c33a022f0 2024-09-11T21:57:40.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.013+0000 7fa1ddffb640 1 -- 172.21.15.1:0/3735782681 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa1ec03a040 con 0x555c33a022f0 2024-09-11T21:57:40.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.013+0000 7fa1fef5b740 10 client.15510 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T21:57:40.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.013+0000 7fa1fef5b740 10 client.15510 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-11T21:57:40.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.013+0000 7fa1fef5b740 20 client.15510 populate_metadata read hostname 'smithi001' 2024-09-11T21:57:40.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.013+0000 7fa1fef5b740 1 --2- 172.21.15.1:0/3735782681 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x555c33a1d160 0x555c33a8b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:40.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.013+0000 7fa1fef5b740 4 client.15510 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T21:57:40.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.013+0000 7fa1fef5b740 1 -- 172.21.15.1:0/3735782681 --> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] -- command(tid 0: {"prefix": "status"}) -- 0x555c3396fb10 con 0x555c33a1d160 2024-09-11T21:57:40.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.013+0000 7fa1f659c640 1 --2- 172.21.15.1:0/3735782681 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x555c33a1d160 0x555c33a8b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:40.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.013+0000 7fa1f659c640 1 --2- 172.21.15.1:0/3735782681 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x555c33a1d160 0x555c33a8b4e0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=292e6adea81b6d5b server_cookie=e85894717d94bdc6 in_seq=0 out_seq=0 2024-09-11T21:57:40.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.013+0000 7fa1ddffb640 10 client.15510 ms_handle_connect on v2:172.21.15.53:6836/3457131830 2024-09-11T21:57:40.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.014+0000 7fa1ddffb640 1 -- 172.21.15.1:0/3735782681 <== mds.2 v2:172.21.15.53:6836/3457131830 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x555c3396fb10 con 0x555c33a1d160 2024-09-11T21:57:40.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.014+0000 7fa1ddffb640 10 client.15510 handle_command_reply: tid=0 2024-09-11T21:57:40.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.014+0000 7fa1fef5b740 1 client.15510 shutdown 2024-09-11T21:57:40.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.014+0000 7fa1fef5b740 1 -- 172.21.15.1:0/3735782681 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c33a022f0 msgr2=0x555c33a20cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:40.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.014+0000 7fa1fef5b740 1 --2- 172.21.15.1:0/3735782681 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c33a022f0 0x555c33a20cc0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec009230 tx=0x7fa1ec005a20 comp rx=0 tx=0).stop 2024-09-11T21:57:40.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.014+0000 7fa1fef5b740 1 -- 172.21.15.1:0/3735782681 shutdown_connections 2024-09-11T21:57:40.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.014+0000 7fa1fef5b740 1 --2- 172.21.15.1:0/3735782681 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x555c33a1d160 0x555c33a8b4e0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:40.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.014+0000 7fa1fef5b740 1 --2- 172.21.15.1:0/3735782681 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555c33a0cbb0 0x555c33a21200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:40.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.014+0000 7fa1fef5b740 1 --2- 172.21.15.1:0/3735782681 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555c33a022f0 0x555c33a20cc0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:40.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.014+0000 7fa1ddffb640 0 client.15510 ms_handle_reset on v2:172.21.15.53:6836/3457131830 2024-09-11T21:57:40.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.014+0000 7fa1fef5b740 1 --2- 172.21.15.1:0/3735782681 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555c33a019d0 0x555c33a20780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:40.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.014+0000 7fa1fef5b740 1 -- 172.21.15.1:0/3735782681 >> 172.21.15.1:0/3735782681 conn(0x555c339fdf60 msgr2=0x555c33a04de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:40.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.014+0000 7fa1fef5b740 1 -- 172.21.15.1:0/3735782681 shutdown_connections 2024-09-11T21:57:40.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.014+0000 7fa1fef5b740 1 -- 172.21.15.1:0/3735782681 wait complete. 2024-09-11T21:57:40.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.014+0000 7fa1fef5b740 20 client.15510 trim_cache size 0 max 16384 2024-09-11T21:57:40.025 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-11T21:57:40.025 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "3ffca83c-7087-11ef-bcea-c7b262605968", 2024-09-11T21:57:40.025 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 2, 2024-09-11T21:57:40.025 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 24467, 2024-09-11T21:57:40.025 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-09-11T21:57:40.025 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-09-11T21:57:40.025 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-09-11T21:57:40.025 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 130.88529665199999, 2024-09-11T21:57:40.025 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 28, 2024-09-11T21:57:40.026 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 87, 2024-09-11T21:57:40.026 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 86, 2024-09-11T21:57:40.026 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 160.874699058 2024-09-11T21:57:40.026 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-11T21:57:40.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.014+0000 7fa1fcb82640 1 -- 172.21.15.1:0/138922070 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fa1e00a2d80 msgr2=0x7fa1e00a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:40.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.014+0000 7fa1fcb82640 1 --2- 172.21.15.1:0/138922070 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fa1e00a2d80 0x7fa1e00a5240 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec000c00 tx=0x7fa1ec0041d0 comp rx=0 tx=0).stop 2024-09-11T21:57:40.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.014+0000 7fa1fcb82640 1 -- 172.21.15.1:0/138922070 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1f810c170 msgr2=0x7fa1f819ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:40.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.015+0000 7fa1fcb82640 1 --2- 172.21.15.1:0/138922070 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1f810c170 0x7fa1f819ef50 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fa1f000c970 tx=0x7fa1f000ce30 comp rx=0 tx=0).stop 2024-09-11T21:57:40.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.015+0000 7fa1fcb82640 1 -- 172.21.15.1:0/138922070 shutdown_connections 2024-09-11T21:57:40.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.015+0000 7fa1fcb82640 1 --2- 172.21.15.1:0/138922070 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fa1e00a2d80 0x7fa1e00a5240 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:40.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.015+0000 7fa1fcb82640 1 --2- 172.21.15.1:0/138922070 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1f810c170 0x7fa1f819ef50 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:40.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.015+0000 7fa1fcb82640 1 --2- 172.21.15.1:0/138922070 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1f8103660 0x7fa1f819abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:40.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.015+0000 7fa1fcb82640 1 --2- 172.21.15.1:0/138922070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1f8102d40 0x7fa1f819a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:40.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.015+0000 7fa1fcb82640 1 -- 172.21.15.1:0/138922070 >> 172.21.15.1:0/138922070 conn(0x7fa1f80fc7a0 msgr2=0x7fa1f80fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:40.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.015+0000 7fa1fcb82640 1 -- 172.21.15.1:0/138922070 shutdown_connections 2024-09-11T21:57:40.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:40.015+0000 7fa1fcb82640 1 -- 172.21.15.1:0/138922070 wait complete. 2024-09-11T21:57:40.710 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-11T21:57:40.971 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:57:41.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:40 smithi053 ceph-mon[34908]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-09-11T21:57:41.331 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:40 smithi001 ceph-mon[30935]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-09-11T21:57:41.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:40 smithi161 ceph-mon[35615]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-09-11T21:57:41.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.415+0000 7f96441ce640 1 -- 172.21.15.1:0/1476647432 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f963c100a80 msgr2=0x7f963c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:41.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.415+0000 7f96441ce640 1 --2- 172.21.15.1:0/1476647432 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f963c100a80 0x7f963c100ee0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f962c009930 tx=0x7f962c02a420 comp rx=0 tx=0).stop 2024-09-11T21:57:41.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.415+0000 7f96441ce640 1 -- 172.21.15.1:0/1476647432 shutdown_connections 2024-09-11T21:57:41.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.415+0000 7f96441ce640 1 --2- 172.21.15.1:0/1476647432 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f963c101420 0x7f963c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:41.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.415+0000 7f96441ce640 1 --2- 172.21.15.1:0/1476647432 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f963c100a80 0x7f963c100ee0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:41.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.415+0000 7f96441ce640 1 --2- 172.21.15.1:0/1476647432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f963c106a80 0x7f963c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:41.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.415+0000 7f96441ce640 1 -- 172.21.15.1:0/1476647432 >> 172.21.15.1:0/1476647432 conn(0x7f963c0fc7a0 msgr2=0x7f963c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:41.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.416+0000 7f96441ce640 1 -- 172.21.15.1:0/1476647432 shutdown_connections 2024-09-11T21:57:41.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.416+0000 7f96441ce640 1 -- 172.21.15.1:0/1476647432 wait complete. 2024-09-11T21:57:41.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.416+0000 7f96441ce640 1 Processor -- start 2024-09-11T21:57:41.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.416+0000 7f96441ce640 1 -- start start 2024-09-11T21:57:41.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.416+0000 7f96441ce640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f963c100a80 0x7f963c1972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:41.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.416+0000 7f96441ce640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f963c101420 0x7f963c197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:41.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.416+0000 7f96441ce640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f963c106a80 0x7f963c19bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:41.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.416+0000 7f96441ce640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f963c110e90 con 0x7f963c100a80 2024-09-11T21:57:41.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.416+0000 7f96441ce640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f963c110d10 con 0x7f963c101420 2024-09-11T21:57:41.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.416+0000 7f96441ce640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f963c111010 con 0x7f963c106a80 2024-09-11T21:57:41.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.417+0000 7f9641f6a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f963c100a80 0x7f963c1972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:41.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.417+0000 7f9641f6a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f963c100a80 0x7f963c1972e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33446/0 (socket says 172.21.15.1:33446) 2024-09-11T21:57:41.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.417+0000 7f9641f6a640 1 -- 172.21.15.1:0/3263472235 learned_addr learned my addr 172.21.15.1:0/3263472235 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:41.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.417+0000 7f9641769640 1 --2- 172.21.15.1:0/3263472235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f963c101420 0x7f963c197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:41.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.417+0000 7f964276b640 1 --2- 172.21.15.1:0/3263472235 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f963c106a80 0x7f963c19bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:41.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.417+0000 7f9641f6a640 1 -- 172.21.15.1:0/3263472235 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f963c106a80 msgr2=0x7f963c19bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:41.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.417+0000 7f9641f6a640 1 --2- 172.21.15.1:0/3263472235 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f963c106a80 0x7f963c19bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:41.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.417+0000 7f9641f6a640 1 -- 172.21.15.1:0/3263472235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f963c101420 msgr2=0x7f963c197820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:41.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.417+0000 7f9641f6a640 1 --2- 172.21.15.1:0/3263472235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f963c101420 0x7f963c197820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:41.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.417+0000 7f9641f6a640 1 -- 172.21.15.1:0/3263472235 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f963c19c290 con 0x7f963c100a80 2024-09-11T21:57:41.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.417+0000 7f9641f6a640 1 --2- 172.21.15.1:0/3263472235 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f963c100a80 0x7f963c1972e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f963800c920 tx=0x7f963800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:41.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.417+0000 7f9632ffd640 1 -- 172.21.15.1:0/3263472235 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9638007c70 con 0x7f963c100a80 2024-09-11T21:57:41.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.417+0000 7f9632ffd640 1 -- 172.21.15.1:0/3263472235 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9638007e10 con 0x7f963c100a80 2024-09-11T21:57:41.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.417+0000 7f96441ce640 1 -- 172.21.15.1:0/3263472235 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f963c19c580 con 0x7f963c100a80 2024-09-11T21:57:41.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.417+0000 7f9632ffd640 1 -- 172.21.15.1:0/3263472235 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9638018890 con 0x7f963c100a80 2024-09-11T21:57:41.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.417+0000 7f96441ce640 1 -- 172.21.15.1:0/3263472235 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f963c1a3e60 con 0x7f963c100a80 2024-09-11T21:57:41.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.420+0000 7f9632ffd640 1 -- 172.21.15.1:0/3263472235 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 24) ==== 138008+0+0 (secure 0 0 0) 0x7f9638018a70 con 0x7f963c100a80 2024-09-11T21:57:41.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.420+0000 7f96465a7740 1 Processor -- start 2024-09-11T21:57:41.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.420+0000 7f96465a7740 1 -- start start 2024-09-11T21:57:41.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.420+0000 7f96465a7740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d64f289d0 0x560d64f28db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:41.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.420+0000 7f96465a7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d64f292f0 0x560d64f33670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:41.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.421+0000 7f96465a7740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d64f33bb0 0x560d64f35fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:41.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.421+0000 7f9632ffd640 1 --2- 172.21.15.1:0/3263472235 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f96280a2d80 0x7f96280a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:41.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.421+0000 7f96465a7740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x560d64e8f090 con 0x560d64f289d0 2024-09-11T21:57:41.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.421+0000 7f96465a7740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x560d64e36a70 con 0x560d64f33bb0 2024-09-11T21:57:41.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.421+0000 7f96465a7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x560d64e8b050 con 0x560d64f292f0 2024-09-11T21:57:41.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.421+0000 7f964276b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d64f289d0 0x560d64f28db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:41.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.421+0000 7f964276b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d64f289d0 0x560d64f28db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33452/0 (socket says 172.21.15.1:33452) 2024-09-11T21:57:41.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.421+0000 7f964276b640 1 -- 172.21.15.1:0/1572419695 learned_addr learned my addr 172.21.15.1:0/1572419695 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:41.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.421+0000 7f964276b640 1 --2- 172.21.15.1:0/3263472235 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f96280a2d80 0x7f96280a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:41.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.421+0000 7f9632ffd640 1 -- 172.21.15.1:0/3263472235 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7f96380cf640 con 0x7f963c100a80 2024-09-11T21:57:41.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.421+0000 7f9641f6a640 1 --2- 172.21.15.1:0/1572419695 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d64f292f0 0x560d64f33670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:41.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.421+0000 7f9641769640 1 --2- 172.21.15.1:0/1572419695 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d64f33bb0 0x560d64f35fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:41.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.421+0000 7f964276b640 1 -- 172.21.15.1:0/1572419695 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d64f292f0 msgr2=0x560d64f33670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:41.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.421+0000 7f964276b640 1 --2- 172.21.15.1:0/1572419695 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d64f292f0 0x560d64f33670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:41.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.421+0000 7f964276b640 1 -- 172.21.15.1:0/1572419695 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d64f33bb0 msgr2=0x560d64f35fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:41.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.421+0000 7f964276b640 1 --2- 172.21.15.1:0/1572419695 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d64f33bb0 0x560d64f35fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:41.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.421+0000 7f964276b640 1 -- 172.21.15.1:0/1572419695 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d64e43130 con 0x560d64f289d0 2024-09-11T21:57:41.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.421+0000 7f964276b640 1 --2- 172.21.15.1:0/1572419695 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d64f289d0 0x560d64f28db0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f9634012f20 tx=0x7f963400e780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfbd93836f4bc061 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:41.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.422+0000 7f964276b640 1 --2- 172.21.15.1:0/3263472235 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f96280a2d80 0x7f96280a5240 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f963400ec80 tx=0x7f9634019000 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:41.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.422+0000 7f9627fff640 1 -- 172.21.15.1:0/1572419695 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9634024070 con 0x560d64f289d0 2024-09-11T21:57:41.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.422+0000 7f9627fff640 1 -- 172.21.15.1:0/1572419695 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9634009da0 con 0x560d64f289d0 2024-09-11T21:57:41.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.422+0000 7f9627fff640 1 -- 172.21.15.1:0/1572419695 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9634005650 con 0x560d64f289d0 2024-09-11T21:57:41.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.422+0000 7f96465a7740 1 -- 172.21.15.1:0/1572419695 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d64f289d0 msgr2=0x560d64f28db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:41.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.422+0000 7f96465a7740 1 --2- 172.21.15.1:0/1572419695 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d64f289d0 0x560d64f28db0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f9634012f20 tx=0x7f963400e780 comp rx=0 tx=0).stop 2024-09-11T21:57:41.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.422+0000 7f96465a7740 1 -- 172.21.15.1:0/1572419695 shutdown_connections 2024-09-11T21:57:41.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.422+0000 7f96465a7740 1 --2- 172.21.15.1:0/1572419695 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d64f33bb0 0x560d64f35fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:41.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.422+0000 7f96465a7740 1 --2- 172.21.15.1:0/1572419695 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d64f292f0 0x560d64f33670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:41.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.422+0000 7f96465a7740 1 --2- 172.21.15.1:0/1572419695 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d64f289d0 0x560d64f28db0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:41.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.422+0000 7f96465a7740 1 -- 172.21.15.1:0/1572419695 >> 172.21.15.1:0/1572419695 conn(0x560d64f24f60 msgr2=0x560d64f25340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:41.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.423+0000 7f96465a7740 1 -- 172.21.15.1:0/1572419695 shutdown_connections 2024-09-11T21:57:41.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.423+0000 7f96465a7740 1 -- 172.21.15.1:0/1572419695 wait complete. 2024-09-11T21:57:41.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.423+0000 7f96465a7740 1 -- start start 2024-09-11T21:57:41.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.423+0000 7f96465a7740 1 Processor -- start 2024-09-11T21:57:41.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.423+0000 7f96465a7740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d64f289d0 0x560d64f334c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:41.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.423+0000 7f96465a7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d64f292f0 0x560d64f30810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:41.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.423+0000 7f96465a7740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d64f33bb0 0x560d64f30d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:41.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.423+0000 7f96465a7740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x560d64f36780 con 0x560d64f33bb0 2024-09-11T21:57:41.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.423+0000 7f96465a7740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x560d64e49f20 con 0x560d64f289d0 2024-09-11T21:57:41.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.423+0000 7f96465a7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x560d64f36900 con 0x560d64f292f0 2024-09-11T21:57:41.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.424+0000 7f96465a7740 10 client.0 _ll_register_callbacks cb 0x560d64e13030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:57:41.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.424+0000 7f964276b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d64f33bb0 0x560d64f30d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:41.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.424+0000 7f964276b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d64f33bb0 0x560d64f30d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33462/0 (socket says 172.21.15.1:33462) 2024-09-11T21:57:41.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.424+0000 7f964276b640 1 -- 172.21.15.1:0/1525223753 learned_addr learned my addr 172.21.15.1:0/1525223753 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:41.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.424+0000 7f9641769640 1 --2- 172.21.15.1:0/1525223753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d64f292f0 0x560d64f30810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:41.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.424+0000 7f9641f6a640 1 --2- 172.21.15.1:0/1525223753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d64f289d0 0x560d64f334c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:41.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.424+0000 7f964276b640 1 -- 172.21.15.1:0/1525223753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d64f292f0 msgr2=0x560d64f30810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:41.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.424+0000 7f964276b640 1 --2- 172.21.15.1:0/1525223753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d64f292f0 0x560d64f30810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:41.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.424+0000 7f964276b640 1 -- 172.21.15.1:0/1525223753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d64f289d0 msgr2=0x560d64f334c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:41.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.424+0000 7f964276b640 1 --2- 172.21.15.1:0/1525223753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d64f289d0 0x560d64f334c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:41.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.424+0000 7f964276b640 1 -- 172.21.15.1:0/1525223753 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d64c002b0 con 0x560d64f33bb0 2024-09-11T21:57:41.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.424+0000 7f9641f6a640 1 --2- 172.21.15.1:0/1525223753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d64f289d0 0x560d64f334c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T21:57:41.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.424+0000 7f964276b640 1 --2- 172.21.15.1:0/1525223753 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d64f33bb0 0x560d64f30d50 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f963400ef90 tx=0x7f9634004ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ce561bddb532c46 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:41.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.424+0000 7f96257fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T21:57:41.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.424+0000 7f96257fa640 1 -- 172.21.15.1:0/1525223753 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9634024070 con 0x560d64f33bb0 2024-09-11T21:57:41.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.424+0000 7f96257fa640 1 -- 172.21.15.1:0/1525223753 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7f963400cd50 con 0x560d64f33bb0 2024-09-11T21:57:41.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.424+0000 7f96465a7740 1 -- 172.21.15.1:0/1525223753 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9634009da0 con 0x560d64f33bb0 2024-09-11T21:57:41.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.425+0000 7f96257fa640 1 -- 172.21.15.1:0/1525223753 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f963403e050 con 0x560d64f33bb0 2024-09-11T21:57:41.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.425+0000 7f96465a7740 10 client.15522 fetch_fsmap learned FSMap version 28 2024-09-11T21:57:41.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.425+0000 7f96465a7740 1 -- 172.21.15.1:0/1525223753 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560d64f42100 con 0x560d64f33bb0 2024-09-11T21:57:41.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.425+0000 7f96257fa640 1 -- 172.21.15.1:0/1525223753 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9634009da0 con 0x560d64f33bb0 2024-09-11T21:57:41.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.425+0000 7f96257fa640 1 -- 172.21.15.1:0/1525223753 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f963400f040 con 0x560d64f33bb0 2024-09-11T21:57:41.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.425+0000 7f96465a7740 10 client.15522 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T21:57:41.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.425+0000 7f96465a7740 10 client.15522 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-11T21:57:41.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.425+0000 7f96465a7740 20 client.15522 populate_metadata read hostname 'smithi001' 2024-09-11T21:57:41.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.425+0000 7f96465a7740 1 --2- 172.21.15.1:0/1525223753 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x560d64f44540 0x560d64f459b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:41.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.425+0000 7f96465a7740 4 client.15522 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T21:57:41.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.425+0000 7f96465a7740 1 -- 172.21.15.1:0/1525223753 --> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d64e9c600 con 0x560d64f44540 2024-09-11T21:57:41.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.425+0000 7f9641f6a640 1 --2- 172.21.15.1:0/1525223753 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x560d64f44540 0x560d64f459b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:41.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.426+0000 7f9641f6a640 1 --2- 172.21.15.1:0/1525223753 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x560d64f44540 0x560d64f459b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=61e2c33355727ab server_cookie=92e1dce8895736ea in_seq=0 out_seq=0 2024-09-11T21:57:41.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.426+0000 7f96257fa640 10 client.15522 ms_handle_connect on v2:172.21.15.161:6838/455365383 2024-09-11T21:57:41.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.427+0000 7f96257fa640 1 -- 172.21.15.1:0/1525223753 <== mds.1 v2:172.21.15.161:6838/455365383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560d64e9c600 con 0x560d64f44540 2024-09-11T21:57:41.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.427+0000 7f96257fa640 10 client.15522 handle_command_reply: tid=0 2024-09-11T21:57:41.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.427+0000 7f96465a7740 1 client.15522 shutdown 2024-09-11T21:57:41.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.427+0000 7f96465a7740 1 -- 172.21.15.1:0/1525223753 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d64f33bb0 msgr2=0x560d64f30d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:41.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.427+0000 7f96465a7740 1 --2- 172.21.15.1:0/1525223753 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d64f33bb0 0x560d64f30d50 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f963400ef90 tx=0x7f9634004ec0 comp rx=0 tx=0).stop 2024-09-11T21:57:41.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.427+0000 7f96465a7740 1 -- 172.21.15.1:0/1525223753 shutdown_connections 2024-09-11T21:57:41.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.427+0000 7f96465a7740 1 --2- 172.21.15.1:0/1525223753 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x560d64f44540 0x560d64f459b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:41.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.427+0000 7f96465a7740 1 --2- 172.21.15.1:0/1525223753 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d64f33bb0 0x560d64f30d50 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:41.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.427+0000 7f96465a7740 1 --2- 172.21.15.1:0/1525223753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d64f292f0 0x560d64f30810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:41.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.427+0000 7f96257fa640 0 client.15522 ms_handle_reset on v2:172.21.15.161:6838/455365383 2024-09-11T21:57:41.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.427+0000 7f96465a7740 1 --2- 172.21.15.1:0/1525223753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d64f289d0 0x560d64f334c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:41.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.427+0000 7f96465a7740 1 -- 172.21.15.1:0/1525223753 >> 172.21.15.1:0/1525223753 conn(0x560d64f24f60 msgr2=0x560d64f2ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:41.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.427+0000 7f96465a7740 1 -- 172.21.15.1:0/1525223753 shutdown_connections 2024-09-11T21:57:41.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.427+0000 7f96465a7740 1 -- 172.21.15.1:0/1525223753 wait complete. 2024-09-11T21:57:41.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.427+0000 7f96465a7740 20 client.15522 trim_cache size 0 max 16384 2024-09-11T21:57:41.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.445+0000 7f96465a7740 1 Processor -- start 2024-09-11T21:57:41.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.445+0000 7f96465a7740 1 -- start start 2024-09-11T21:57:41.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.445+0000 7f96465a7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d64f289d0 0x560d64f2dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:41.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.445+0000 7f96465a7740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d64f292f0 0x560d64f2dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:41.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.445+0000 7f96465a7740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d64f33bb0 0x560d64f4dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:41.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.445+0000 7f96465a7740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x560d64f43cf0 con 0x560d64f33bb0 2024-09-11T21:57:41.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.445+0000 7f96465a7740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x560d64f443c0 con 0x560d64f292f0 2024-09-11T21:57:41.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.445+0000 7f96465a7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x560d64f44a90 con 0x560d64f289d0 2024-09-11T21:57:41.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.445+0000 7f964276b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d64f33bb0 0x560d64f4dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:41.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.445+0000 7f964276b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d64f33bb0 0x560d64f4dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33468/0 (socket says 172.21.15.1:33468) 2024-09-11T21:57:41.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.445+0000 7f964276b640 1 -- 172.21.15.1:0/1704827425 learned_addr learned my addr 172.21.15.1:0/1704827425 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:41.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.445+0000 7f9641769640 1 --2- 172.21.15.1:0/1704827425 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d64f292f0 0x560d64f2dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:41.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.445+0000 7f9641f6a640 1 --2- 172.21.15.1:0/1704827425 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d64f289d0 0x560d64f2dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:41.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.446+0000 7f964276b640 1 -- 172.21.15.1:0/1704827425 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d64f289d0 msgr2=0x560d64f2dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:41.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.446+0000 7f964276b640 1 --2- 172.21.15.1:0/1704827425 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d64f289d0 0x560d64f2dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:41.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.446+0000 7f964276b640 1 -- 172.21.15.1:0/1704827425 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d64f292f0 msgr2=0x560d64f2dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:41.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.446+0000 7f964276b640 1 --2- 172.21.15.1:0/1704827425 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d64f292f0 0x560d64f2dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:41.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.446+0000 7f964276b640 1 -- 172.21.15.1:0/1704827425 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d64f25bf0 con 0x560d64f33bb0 2024-09-11T21:57:41.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.446+0000 7f9641f6a640 1 --2- 172.21.15.1:0/1704827425 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d64f289d0 0x560d64f2dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T21:57:41.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.446+0000 7f964276b640 1 --2- 172.21.15.1:0/1704827425 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d64f33bb0 0x560d64f4dc90 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f963400e7c0 tx=0x7f9634012880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60bf98140235dd44 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:41.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.446+0000 7f96277fe640 1 -- 172.21.15.1:0/1704827425 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9634047070 con 0x560d64f33bb0 2024-09-11T21:57:41.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.446+0000 7f96277fe640 1 -- 172.21.15.1:0/1704827425 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9634035070 con 0x560d64f33bb0 2024-09-11T21:57:41.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.446+0000 7f96277fe640 1 -- 172.21.15.1:0/1704827425 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9634024050 con 0x560d64f33bb0 2024-09-11T21:57:41.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.447+0000 7f96465a7740 1 -- 172.21.15.1:0/1704827425 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d64f33bb0 msgr2=0x560d64f4dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:41.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.447+0000 7f96465a7740 1 --2- 172.21.15.1:0/1704827425 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d64f33bb0 0x560d64f4dc90 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f963400e7c0 tx=0x7f9634012880 comp rx=0 tx=0).stop 2024-09-11T21:57:41.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.447+0000 7f96465a7740 1 -- 172.21.15.1:0/1704827425 shutdown_connections 2024-09-11T21:57:41.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.447+0000 7f96465a7740 1 --2- 172.21.15.1:0/1704827425 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d64f33bb0 0x560d64f4dc90 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:41.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.447+0000 7f96465a7740 1 --2- 172.21.15.1:0/1704827425 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d64f292f0 0x560d64f2dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:41.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.447+0000 7f96465a7740 1 --2- 172.21.15.1:0/1704827425 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d64f289d0 0x560d64f2dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:41.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.447+0000 7f96465a7740 1 -- 172.21.15.1:0/1704827425 >> 172.21.15.1:0/1704827425 conn(0x560d64f24f60 msgr2=0x560d64f1fce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:41.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.447+0000 7f96465a7740 1 -- 172.21.15.1:0/1704827425 shutdown_connections 2024-09-11T21:57:41.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.447+0000 7f96465a7740 1 -- 172.21.15.1:0/1704827425 wait complete. 2024-09-11T21:57:41.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.447+0000 7f96465a7740 1 -- start start 2024-09-11T21:57:41.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.447+0000 7f96465a7740 1 Processor -- start 2024-09-11T21:57:41.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.447+0000 7f96465a7740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d64f289d0 0x560d64fa6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:41.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.447+0000 7f96465a7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d64f292f0 0x560d64fa7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:41.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.447+0000 7f96465a7740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d64f33bb0 0x560d64f4d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:41.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.448+0000 7f96465a7740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x560d64f45830 con 0x560d64f289d0 2024-09-11T21:57:41.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.448+0000 7f96465a7740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x560d64f45160 con 0x560d64f33bb0 2024-09-11T21:57:41.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.448+0000 7f96465a7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x560d64f45f00 con 0x560d64f292f0 2024-09-11T21:57:41.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.448+0000 7f9641f6a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d64f289d0 0x560d64fa6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:41.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.448+0000 7f96465a7740 10 client.0 _ll_register_callbacks cb 0x7f96380ccb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:57:41.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.448+0000 7f9641f6a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d64f289d0 0x560d64fa6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33470/0 (socket says 172.21.15.1:33470) 2024-09-11T21:57:41.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.448+0000 7f9641f6a640 1 -- 172.21.15.1:0/4044678953 learned_addr learned my addr 172.21.15.1:0/4044678953 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:41.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.448+0000 7f9641769640 1 --2- 172.21.15.1:0/4044678953 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d64f292f0 0x560d64fa7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:41.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.448+0000 7f964276b640 1 --2- 172.21.15.1:0/4044678953 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d64f33bb0 0x560d64f4d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:41.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.448+0000 7f9641f6a640 1 -- 172.21.15.1:0/4044678953 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d64f292f0 msgr2=0x560d64fa7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:41.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.448+0000 7f9641f6a640 1 --2- 172.21.15.1:0/4044678953 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d64f292f0 0x560d64fa7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:41.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.448+0000 7f9641f6a640 1 -- 172.21.15.1:0/4044678953 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d64f33bb0 msgr2=0x560d64f4d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:41.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.448+0000 7f9641f6a640 1 --2- 172.21.15.1:0/4044678953 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d64f33bb0 0x560d64f4d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:41.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.448+0000 7f9641f6a640 1 -- 172.21.15.1:0/4044678953 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d64f36400 con 0x560d64f289d0 2024-09-11T21:57:41.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.448+0000 7f964276b640 1 --2- 172.21.15.1:0/4044678953 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d64f33bb0 0x560d64f4d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T21:57:41.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.448+0000 7f9641f6a640 1 --2- 172.21.15.1:0/4044678953 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d64f289d0 0x560d64fa6df0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f9638097e50 tx=0x7f9638086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65c250c35980d1fd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:41.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.448+0000 7f96257fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T21:57:41.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.448+0000 7f96257fa640 1 -- 172.21.15.1:0/4044678953 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f963809b9f0 con 0x560d64f289d0 2024-09-11T21:57:41.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.448+0000 7f96257fa640 1 -- 172.21.15.1:0/4044678953 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7f963800fb30 con 0x560d64f289d0 2024-09-11T21:57:41.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.448+0000 7f96465a7740 1 -- 172.21.15.1:0/4044678953 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9634035070 con 0x560d64f289d0 2024-09-11T21:57:41.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.449+0000 7f96257fa640 1 -- 172.21.15.1:0/4044678953 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f963809fda0 con 0x560d64f289d0 2024-09-11T21:57:41.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.449+0000 7f96465a7740 10 client.15528 fetch_fsmap learned FSMap version 28 2024-09-11T21:57:41.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.449+0000 7f96465a7740 1 -- 172.21.15.1:0/4044678953 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560d64fae450 con 0x560d64f289d0 2024-09-11T21:57:41.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.449+0000 7f96257fa640 1 -- 172.21.15.1:0/4044678953 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9634035070 con 0x560d64f289d0 2024-09-11T21:57:41.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.449+0000 7f96257fa640 1 -- 172.21.15.1:0/4044678953 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9638098060 con 0x560d64f289d0 2024-09-11T21:57:41.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.449+0000 7f96465a7740 10 client.15528 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T21:57:41.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.449+0000 7f96465a7740 10 client.15528 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-11T21:57:41.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.449+0000 7f96465a7740 20 client.15528 populate_metadata read hostname 'smithi001' 2024-09-11T21:57:41.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.449+0000 7f96465a7740 1 --2- 172.21.15.1:0/4044678953 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x560d64fb2fe0 0x560d64fb1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:41.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.449+0000 7f96465a7740 4 client.15528 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T21:57:41.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.449+0000 7f96465a7740 1 -- 172.21.15.1:0/4044678953 --> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] -- command(tid 0: {"prefix": "status"}) -- 0x560d64e96b10 con 0x560d64fb2fe0 2024-09-11T21:57:41.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.449+0000 7f9641769640 1 --2- 172.21.15.1:0/4044678953 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x560d64fb2fe0 0x560d64fb1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:41.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.450+0000 7f9641769640 1 --2- 172.21.15.1:0/4044678953 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x560d64fb2fe0 0x560d64fb1cf0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a1c7e90510099d95 server_cookie=930bee3c1c5df3b8 in_seq=0 out_seq=0 2024-09-11T21:57:41.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.450+0000 7f96257fa640 10 client.15528 ms_handle_connect on v2:172.21.15.161:6838/455365383 2024-09-11T21:57:41.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.450+0000 7f96257fa640 1 -- 172.21.15.1:0/4044678953 <== mds.1 v2:172.21.15.161:6838/455365383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x560d64e96b10 con 0x560d64fb2fe0 2024-09-11T21:57:41.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.450+0000 7f96257fa640 10 client.15528 handle_command_reply: tid=0 2024-09-11T21:57:41.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.450+0000 7f96465a7740 1 client.15528 shutdown 2024-09-11T21:57:41.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.450+0000 7f96465a7740 1 -- 172.21.15.1:0/4044678953 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d64f289d0 msgr2=0x560d64fa6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:41.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.450+0000 7f96465a7740 1 --2- 172.21.15.1:0/4044678953 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d64f289d0 0x560d64fa6df0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f9638097e50 tx=0x7f9638086040 comp rx=0 tx=0).stop 2024-09-11T21:57:41.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.450+0000 7f96465a7740 1 -- 172.21.15.1:0/4044678953 shutdown_connections 2024-09-11T21:57:41.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.450+0000 7f96465a7740 1 --2- 172.21.15.1:0/4044678953 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x560d64fb2fe0 0x560d64fb1cf0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:41.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.450+0000 7f96465a7740 1 --2- 172.21.15.1:0/4044678953 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d64f33bb0 0x560d64f4d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:41.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.450+0000 7f96465a7740 1 --2- 172.21.15.1:0/4044678953 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d64f292f0 0x560d64fa7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:41.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.450+0000 7f96465a7740 1 --2- 172.21.15.1:0/4044678953 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d64f289d0 0x560d64fa6df0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:41.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.450+0000 7f96257fa640 0 client.15528 ms_handle_reset on v2:172.21.15.161:6838/455365383 2024-09-11T21:57:41.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.450+0000 7f96465a7740 1 -- 172.21.15.1:0/4044678953 >> 172.21.15.1:0/4044678953 conn(0x560d64f24f60 msgr2=0x560d64f31cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:41.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.450+0000 7f96465a7740 1 -- 172.21.15.1:0/4044678953 shutdown_connections 2024-09-11T21:57:41.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.450+0000 7f96465a7740 1 -- 172.21.15.1:0/4044678953 wait complete. 2024-09-11T21:57:41.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.451+0000 7f96465a7740 20 client.15528 trim_cache size 0 max 16384 2024-09-11T21:57:41.462 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-11T21:57:41.463 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "3ffca83c-7087-11ef-bcea-c7b262605968", 2024-09-11T21:57:41.463 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 1, 2024-09-11T21:57:41.463 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 24484, 2024-09-11T21:57:41.463 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-09-11T21:57:41.463 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-09-11T21:57:41.463 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-09-11T21:57:41.463 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 133.32849007600001, 2024-09-11T21:57:41.463 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 28, 2024-09-11T21:57:41.463 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 87, 2024-09-11T21:57:41.463 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 86, 2024-09-11T21:57:41.463 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 159.74567827600001 2024-09-11T21:57:41.463 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-11T21:57:41.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.451+0000 7f96441ce640 1 -- 172.21.15.1:0/3263472235 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f96280a2d80 msgr2=0x7f96280a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:41.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.451+0000 7f96441ce640 1 --2- 172.21.15.1:0/3263472235 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f96280a2d80 0x7f96280a5240 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f963400ec80 tx=0x7f9634019000 comp rx=0 tx=0).stop 2024-09-11T21:57:41.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.451+0000 7f96441ce640 1 -- 172.21.15.1:0/3263472235 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f963c100a80 msgr2=0x7f963c1972e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:41.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.451+0000 7f96441ce640 1 --2- 172.21.15.1:0/3263472235 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f963c100a80 0x7f963c1972e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f963800c920 tx=0x7f963800cde0 comp rx=0 tx=0).stop 2024-09-11T21:57:41.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.451+0000 7f96441ce640 1 -- 172.21.15.1:0/3263472235 shutdown_connections 2024-09-11T21:57:41.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.451+0000 7f96441ce640 1 --2- 172.21.15.1:0/3263472235 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f96280a2d80 0x7f96280a5240 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:41.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.451+0000 7f96441ce640 1 --2- 172.21.15.1:0/3263472235 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f963c106a80 0x7f963c19bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:41.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.451+0000 7f96441ce640 1 --2- 172.21.15.1:0/3263472235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f963c101420 0x7f963c197820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:41.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.451+0000 7f96441ce640 1 --2- 172.21.15.1:0/3263472235 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f963c100a80 0x7f963c1972e0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:41.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.451+0000 7f96441ce640 1 -- 172.21.15.1:0/3263472235 >> 172.21.15.1:0/3263472235 conn(0x7f963c0fc7a0 msgr2=0x7f963c0fce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:41.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.451+0000 7f96441ce640 1 -- 172.21.15.1:0/3263472235 shutdown_connections 2024-09-11T21:57:41.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:41.451+0000 7f96441ce640 1 -- 172.21.15.1:0/3263472235 wait complete. 2024-09-11T21:57:42.157 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-11T21:57:42.414 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:57:42.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.924+0000 7fe27f1ec640 1 -- 172.21.15.1:0/251219405 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe278106f00 msgr2=0x7fe27810ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:42.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.924+0000 7fe27f1ec640 1 --2- 172.21.15.1:0/251219405 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe278106f00 0x7fe27810ea80 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fe270009ca0 tx=0x7fe27002a8c0 comp rx=0 tx=0).stop 2024-09-11T21:57:42.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.924+0000 7fe27f1ec640 1 -- 172.21.15.1:0/251219405 shutdown_connections 2024-09-11T21:57:42.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.924+0000 7fe27f1ec640 1 --2- 172.21.15.1:0/251219405 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe278106f00 0x7fe27810ea80 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:42.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.924+0000 7fe27f1ec640 1 --2- 172.21.15.1:0/251219405 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe278102780 0x7fe2781067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:42.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.924+0000 7fe27f1ec640 1 --2- 172.21.15.1:0/251219405 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe278101dd0 0x7fe2781021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:42.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.924+0000 7fe27f1ec640 1 -- 172.21.15.1:0/251219405 >> 172.21.15.1:0/251219405 conn(0x7fe2780fd8b0 msgr2=0x7fe2780ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:42.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.924+0000 7fe27f1ec640 1 -- 172.21.15.1:0/251219405 shutdown_connections 2024-09-11T21:57:42.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.924+0000 7fe27f1ec640 1 -- 172.21.15.1:0/251219405 wait complete. 2024-09-11T21:57:42.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.925+0000 7fe27f1ec640 1 Processor -- start 2024-09-11T21:57:42.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.925+0000 7fe27f1ec640 1 -- start start 2024-09-11T21:57:42.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.925+0000 7fe27f1ec640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe278101dd0 0x7fe278198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:42.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.925+0000 7fe27f1ec640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe278102780 0x7fe278198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:42.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.925+0000 7fe27f1ec640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe278106f00 0x7fe27819ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:42.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.925+0000 7fe27f1ec640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe278111f60 con 0x7fe278102780 2024-09-11T21:57:42.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.925+0000 7fe27f1ec640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe278111de0 con 0x7fe278101dd0 2024-09-11T21:57:42.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.925+0000 7fe27f1ec640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe2781120e0 con 0x7fe278106f00 2024-09-11T21:57:42.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.925+0000 7fe26ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe278102780 0x7fe278198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:42.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.925+0000 7fe26ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe278102780 0x7fe278198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33488/0 (socket says 172.21.15.1:33488) 2024-09-11T21:57:42.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.925+0000 7fe26ffff640 1 -- 172.21.15.1:0/1457374597 learned_addr learned my addr 172.21.15.1:0/1457374597 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:42.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.926+0000 7fe27d789640 1 --2- 172.21.15.1:0/1457374597 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe278106f00 0x7fe27819ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:42.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.926+0000 7fe27cf88640 1 --2- 172.21.15.1:0/1457374597 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe278101dd0 0x7fe278198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:42.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.926+0000 7fe26ffff640 1 -- 172.21.15.1:0/1457374597 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe278106f00 msgr2=0x7fe27819ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:42.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.926+0000 7fe26ffff640 1 --2- 172.21.15.1:0/1457374597 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe278106f00 0x7fe27819ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:42.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.926+0000 7fe26ffff640 1 -- 172.21.15.1:0/1457374597 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe278101dd0 msgr2=0x7fe278198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:42.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.926+0000 7fe26ffff640 1 --2- 172.21.15.1:0/1457374597 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe278101dd0 0x7fe278198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:42.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.926+0000 7fe26ffff640 1 -- 172.21.15.1:0/1457374597 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe27819d3d0 con 0x7fe278102780 2024-09-11T21:57:42.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.926+0000 7fe26ffff640 1 --2- 172.21.15.1:0/1457374597 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe278102780 0x7fe278198960 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fe26800d950 tx=0x7fe26800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:42.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.926+0000 7fe26dffb640 1 -- 172.21.15.1:0/1457374597 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe268017030 con 0x7fe278102780 2024-09-11T21:57:42.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.926+0000 7fe26dffb640 1 -- 172.21.15.1:0/1457374597 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe268004020 con 0x7fe278102780 2024-09-11T21:57:42.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.926+0000 7fe27f1ec640 1 -- 172.21.15.1:0/1457374597 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe27819d6c0 con 0x7fe278102780 2024-09-11T21:57:42.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.926+0000 7fe26dffb640 1 -- 172.21.15.1:0/1457374597 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2680047a0 con 0x7fe278102780 2024-09-11T21:57:42.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.926+0000 7fe27f1ec640 1 -- 172.21.15.1:0/1457374597 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2781a4fa0 con 0x7fe278102780 2024-09-11T21:57:42.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.929+0000 7fe26dffb640 1 -- 172.21.15.1:0/1457374597 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 24) ==== 138008+0+0 (secure 0 0 0) 0x7fe2680041d0 con 0x7fe278102780 2024-09-11T21:57:42.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.930+0000 7fe2815c5740 1 Processor -- start 2024-09-11T21:57:42.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.930+0000 7fe2815c5740 1 -- start start 2024-09-11T21:57:42.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.930+0000 7fe2815c5740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fc93ea8b0 0x558fc93eac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:42.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.930+0000 7fe2815c5740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558fc93eb1d0 0x558fc93f9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:42.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.930+0000 7fe2815c5740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558fc93f9fc0 0x558fc93fc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:42.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.930+0000 7fe2815c5740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x558fc9351090 con 0x558fc93ea8b0 2024-09-11T21:57:42.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.930+0000 7fe2815c5740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x558fc92f8a70 con 0x558fc93eb1d0 2024-09-11T21:57:42.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.930+0000 7fe2815c5740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x558fc934d050 con 0x558fc93f9fc0 2024-09-11T21:57:42.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.930+0000 7fe27d789640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fc93ea8b0 0x558fc93eac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:42.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.930+0000 7fe27d789640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fc93ea8b0 0x558fc93eac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33504/0 (socket says 172.21.15.1:33504) 2024-09-11T21:57:42.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.930+0000 7fe27d789640 1 -- 172.21.15.1:0/1709263607 learned_addr learned my addr 172.21.15.1:0/1709263607 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:42.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.930+0000 7fe26dffb640 1 --2- 172.21.15.1:0/1457374597 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fe2600a2d80 0x7fe2600a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:42.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.930+0000 7fe27cf88640 1 --2- 172.21.15.1:0/1709263607 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558fc93eb1d0 0x558fc93f9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:42.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.930+0000 7fe26ffff640 1 --2- 172.21.15.1:0/1709263607 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558fc93f9fc0 0x558fc93fc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:42.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.930+0000 7fe27d789640 1 -- 172.21.15.1:0/1709263607 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558fc93f9fc0 msgr2=0x558fc93fc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:42.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.930+0000 7fe27d789640 1 --2- 172.21.15.1:0/1709263607 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558fc93f9fc0 0x558fc93fc3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:42.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.930+0000 7fe27d789640 1 -- 172.21.15.1:0/1709263607 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558fc93eb1d0 msgr2=0x558fc93f9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:42.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.930+0000 7fe27d789640 1 --2- 172.21.15.1:0/1709263607 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558fc93eb1d0 0x558fc93f9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:42.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.930+0000 7fe27d789640 1 -- 172.21.15.1:0/1709263607 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fc9305130 con 0x558fc93ea8b0 2024-09-11T21:57:42.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.930+0000 7fe26ffff640 1 --2- 172.21.15.1:0/1709263607 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558fc93f9fc0 0x558fc93fc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T21:57:42.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.930+0000 7fe26dffb640 1 -- 172.21.15.1:0/1457374597 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7fe2680cf080 con 0x7fe278102780 2024-09-11T21:57:42.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.930+0000 7fe27d789640 1 --2- 172.21.15.1:0/1457374597 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fe2600a2d80 0x7fe2600a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:42.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.931+0000 7fe27d789640 1 --2- 172.21.15.1:0/1709263607 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fc93ea8b0 0x558fc93eac90 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fe270005cc0 tx=0x7fe270009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c979b5ab852863a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:42.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.931+0000 7fe25effd640 1 -- 172.21.15.1:0/1709263607 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe270041040 con 0x558fc93ea8b0 2024-09-11T21:57:42.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.931+0000 7fe25effd640 1 -- 172.21.15.1:0/1709263607 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe270036070 con 0x558fc93ea8b0 2024-09-11T21:57:42.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.931+0000 7fe25effd640 1 -- 172.21.15.1:0/1709263607 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe27002f8c0 con 0x558fc93ea8b0 2024-09-11T21:57:42.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.932+0000 7fe2815c5740 1 -- 172.21.15.1:0/1709263607 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fc93ea8b0 msgr2=0x558fc93eac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:42.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.932+0000 7fe2815c5740 1 --2- 172.21.15.1:0/1709263607 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fc93ea8b0 0x558fc93eac90 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fe270005cc0 tx=0x7fe270009880 comp rx=0 tx=0).stop 2024-09-11T21:57:42.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.932+0000 7fe27d789640 1 --2- 172.21.15.1:0/1457374597 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fe2600a2d80 0x7fe2600a5240 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe27004c000 tx=0x7fe27004d000 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:42.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.932+0000 7fe2815c5740 1 -- 172.21.15.1:0/1709263607 shutdown_connections 2024-09-11T21:57:42.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.932+0000 7fe2815c5740 1 --2- 172.21.15.1:0/1709263607 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558fc93f9fc0 0x558fc93fc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:42.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.932+0000 7fe2815c5740 1 --2- 172.21.15.1:0/1709263607 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558fc93eb1d0 0x558fc93f9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:42.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.932+0000 7fe2815c5740 1 --2- 172.21.15.1:0/1709263607 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fc93ea8b0 0x558fc93eac90 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:42.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.932+0000 7fe2815c5740 1 -- 172.21.15.1:0/1709263607 >> 172.21.15.1:0/1709263607 conn(0x558fc93e6f60 msgr2=0x558fc93e7340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:42.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.932+0000 7fe2815c5740 1 -- 172.21.15.1:0/1709263607 shutdown_connections 2024-09-11T21:57:42.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.932+0000 7fe2815c5740 1 -- 172.21.15.1:0/1709263607 wait complete. 2024-09-11T21:57:42.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.932+0000 7fe2815c5740 1 -- start start 2024-09-11T21:57:42.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.932+0000 7fe2815c5740 1 Processor -- start 2024-09-11T21:57:42.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.933+0000 7fe2815c5740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fc93ea8b0 0x558fc93f97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:42.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.933+0000 7fe2815c5740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558fc93eb1d0 0x558fc93f6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:42.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.933+0000 7fe2815c5740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558fc93f9fc0 0x558fc93f70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:42.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.933+0000 7fe2815c5740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x558fc93fcb90 con 0x558fc93ea8b0 2024-09-11T21:57:42.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.933+0000 7fe2815c5740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x558fc930bf20 con 0x558fc93eb1d0 2024-09-11T21:57:42.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.933+0000 7fe2815c5740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x558fc93fcd10 con 0x558fc93f9fc0 2024-09-11T21:57:42.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.933+0000 7fe2815c5740 10 client.0 _ll_register_callbacks cb 0x558fc92d5030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:57:42.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.933+0000 7fe27cf88640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fc93ea8b0 0x558fc93f97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:42.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.933+0000 7fe27cf88640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fc93ea8b0 0x558fc93f97c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33514/0 (socket says 172.21.15.1:33514) 2024-09-11T21:57:42.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.933+0000 7fe27cf88640 1 -- 172.21.15.1:0/4139116750 learned_addr learned my addr 172.21.15.1:0/4139116750 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:42.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.933+0000 7fe26ffff640 1 --2- 172.21.15.1:0/4139116750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558fc93eb1d0 0x558fc93f6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:42.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.933+0000 7fe27d789640 1 --2- 172.21.15.1:0/4139116750 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558fc93f9fc0 0x558fc93f70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:42.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.933+0000 7fe27cf88640 1 -- 172.21.15.1:0/4139116750 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558fc93f9fc0 msgr2=0x558fc93f70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:42.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.933+0000 7fe27cf88640 1 --2- 172.21.15.1:0/4139116750 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558fc93f9fc0 0x558fc93f70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:42.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.933+0000 7fe27cf88640 1 -- 172.21.15.1:0/4139116750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558fc93eb1d0 msgr2=0x558fc93f6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:42.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.933+0000 7fe27cf88640 1 --2- 172.21.15.1:0/4139116750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558fc93eb1d0 0x558fc93f6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:42.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.933+0000 7fe27cf88640 1 -- 172.21.15.1:0/4139116750 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fc90c22b0 con 0x558fc93ea8b0 2024-09-11T21:57:42.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.933+0000 7fe26ffff640 1 --2- 172.21.15.1:0/4139116750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558fc93eb1d0 0x558fc93f6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T21:57:42.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.933+0000 7fe27cf88640 1 --2- 172.21.15.1:0/4139116750 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fc93ea8b0 0x558fc93f97c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe27400da10 tx=0x7fe27400ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0684fe19f0ae1f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:42.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.933+0000 7fe257fff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T21:57:42.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.934+0000 7fe257fff640 1 -- 172.21.15.1:0/4139116750 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe27401b640 con 0x558fc93ea8b0 2024-09-11T21:57:42.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.934+0000 7fe257fff640 1 -- 172.21.15.1:0/4139116750 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7fe274011440 con 0x558fc93ea8b0 2024-09-11T21:57:42.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.934+0000 7fe2815c5740 1 -- 172.21.15.1:0/4139116750 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe270036070 con 0x558fc93ea8b0 2024-09-11T21:57:42.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.934+0000 7fe257fff640 1 -- 172.21.15.1:0/4139116750 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe27401b7e0 con 0x558fc93ea8b0 2024-09-11T21:57:42.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.934+0000 7fe2815c5740 10 client.15540 fetch_fsmap learned FSMap version 28 2024-09-11T21:57:42.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.934+0000 7fe2815c5740 1 -- 172.21.15.1:0/4139116750 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558fc9408460 con 0x558fc93ea8b0 2024-09-11T21:57:42.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.934+0000 7fe257fff640 1 -- 172.21.15.1:0/4139116750 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe270036070 con 0x558fc93ea8b0 2024-09-11T21:57:42.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.934+0000 7fe257fff640 1 -- 172.21.15.1:0/4139116750 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe274019020 con 0x558fc93ea8b0 2024-09-11T21:57:42.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.934+0000 7fe2815c5740 10 client.15540 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T21:57:42.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.934+0000 7fe2815c5740 10 client.15540 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T21:57:42.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.934+0000 7fe2815c5740 20 client.15540 populate_metadata read hostname 'smithi001' 2024-09-11T21:57:42.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.934+0000 7fe2815c5740 1 --2- 172.21.15.1:0/4139116750 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x558fc940aa40 0x558fc940beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:42.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.934+0000 7fe2815c5740 4 client.15540 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T21:57:42.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.934+0000 7fe2815c5740 1 -- 172.21.15.1:0/4139116750 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558fc935e600 con 0x558fc940aa40 2024-09-11T21:57:42.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.935+0000 7fe26ffff640 1 --2- 172.21.15.1:0/4139116750 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x558fc940aa40 0x558fc940beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:42.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.935+0000 7fe26ffff640 1 --2- 172.21.15.1:0/4139116750 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x558fc940aa40 0x558fc940beb0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2361a8e233a71eeb server_cookie=40c5aa46df4694ea in_seq=0 out_seq=0 2024-09-11T21:57:42.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.935+0000 7fe257fff640 10 client.15540 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T21:57:42.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.936+0000 7fe257fff640 1 -- 172.21.15.1:0/4139116750 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558fc935e600 con 0x558fc940aa40 2024-09-11T21:57:42.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.936+0000 7fe257fff640 10 client.15540 handle_command_reply: tid=0 2024-09-11T21:57:42.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.936+0000 7fe2815c5740 1 client.15540 shutdown 2024-09-11T21:57:42.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.936+0000 7fe2815c5740 1 -- 172.21.15.1:0/4139116750 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fc93ea8b0 msgr2=0x558fc93f97c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:42.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.936+0000 7fe2815c5740 1 --2- 172.21.15.1:0/4139116750 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fc93ea8b0 0x558fc93f97c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe27400da10 tx=0x7fe27400ded0 comp rx=0 tx=0).stop 2024-09-11T21:57:42.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.936+0000 7fe2815c5740 1 -- 172.21.15.1:0/4139116750 shutdown_connections 2024-09-11T21:57:42.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.936+0000 7fe2815c5740 1 --2- 172.21.15.1:0/4139116750 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x558fc940aa40 0x558fc940beb0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:42.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.936+0000 7fe2815c5740 1 --2- 172.21.15.1:0/4139116750 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558fc93f9fc0 0x558fc93f70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:42.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.936+0000 7fe2815c5740 1 --2- 172.21.15.1:0/4139116750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558fc93eb1d0 0x558fc93f6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:42.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.936+0000 7fe257fff640 0 client.15540 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T21:57:42.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.936+0000 7fe2815c5740 1 --2- 172.21.15.1:0/4139116750 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fc93ea8b0 0x558fc93f97c0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:42.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.936+0000 7fe2815c5740 1 -- 172.21.15.1:0/4139116750 >> 172.21.15.1:0/4139116750 conn(0x558fc93e6f60 msgr2=0x558fc93eccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:42.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.936+0000 7fe2815c5740 1 -- 172.21.15.1:0/4139116750 shutdown_connections 2024-09-11T21:57:42.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.936+0000 7fe2815c5740 1 -- 172.21.15.1:0/4139116750 wait complete. 2024-09-11T21:57:42.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.936+0000 7fe2815c5740 20 client.15540 trim_cache size 0 max 16384 2024-09-11T21:57:42.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.954+0000 7fe2815c5740 1 Processor -- start 2024-09-11T21:57:42.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.954+0000 7fe2815c5740 1 -- start start 2024-09-11T21:57:42.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.954+0000 7fe2815c5740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558fc93ea8b0 0x558fc940a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:42.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.954+0000 7fe2815c5740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fc93eb1d0 0x558fc940b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:42.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.954+0000 7fe2815c5740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558fc93f9fc0 0x558fc93f1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:42.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.954+0000 7fe2815c5740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x558fc940a1f0 con 0x558fc93eb1d0 2024-09-11T21:57:42.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.954+0000 7fe2815c5740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x558fc940a8c0 con 0x558fc93ea8b0 2024-09-11T21:57:42.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.954+0000 7fe2815c5740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x558fc940af90 con 0x558fc93f9fc0 2024-09-11T21:57:42.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.954+0000 7fe26ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fc93eb1d0 0x558fc940b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:42.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.954+0000 7fe26ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fc93eb1d0 0x558fc940b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33528/0 (socket says 172.21.15.1:33528) 2024-09-11T21:57:42.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.954+0000 7fe26ffff640 1 -- 172.21.15.1:0/900536397 learned_addr learned my addr 172.21.15.1:0/900536397 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:42.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.954+0000 7fe27cf88640 1 --2- 172.21.15.1:0/900536397 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558fc93ea8b0 0x558fc940a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:42.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.954+0000 7fe26ffff640 1 -- 172.21.15.1:0/900536397 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558fc93f9fc0 msgr2=0x558fc93f1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:42.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.954+0000 7fe26ffff640 1 --2- 172.21.15.1:0/900536397 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558fc93f9fc0 0x558fc93f1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:42.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.954+0000 7fe26ffff640 1 -- 172.21.15.1:0/900536397 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558fc93ea8b0 msgr2=0x558fc940a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:42.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.954+0000 7fe26ffff640 1 --2- 172.21.15.1:0/900536397 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558fc93ea8b0 0x558fc940a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:42.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.954+0000 7fe26ffff640 1 -- 172.21.15.1:0/900536397 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fc93f50b0 con 0x558fc93eb1d0 2024-09-11T21:57:42.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.954+0000 7fe27cf88640 1 --2- 172.21.15.1:0/900536397 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558fc93ea8b0 0x558fc940a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T21:57:42.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.955+0000 7fe26ffff640 1 --2- 172.21.15.1:0/900536397 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fc93eb1d0 0x558fc940b110 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fe268097830 tx=0x7fe26806d390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6e97928587f8805 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:42.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.955+0000 7fe25e7fc640 1 -- 172.21.15.1:0/900536397 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe26809ad30 con 0x558fc93eb1d0 2024-09-11T21:57:42.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.955+0000 7fe25e7fc640 1 -- 172.21.15.1:0/900536397 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe268066460 con 0x558fc93eb1d0 2024-09-11T21:57:42.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.955+0000 7fe25e7fc640 1 -- 172.21.15.1:0/900536397 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe26809b630 con 0x558fc93eb1d0 2024-09-11T21:57:42.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.955+0000 7fe2815c5740 1 -- 172.21.15.1:0/900536397 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fc93eb1d0 msgr2=0x558fc940b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:42.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.955+0000 7fe2815c5740 1 --2- 172.21.15.1:0/900536397 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fc93eb1d0 0x558fc940b110 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fe268097830 tx=0x7fe26806d390 comp rx=0 tx=0).stop 2024-09-11T21:57:42.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.955+0000 7fe2815c5740 1 -- 172.21.15.1:0/900536397 shutdown_connections 2024-09-11T21:57:42.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.955+0000 7fe2815c5740 1 --2- 172.21.15.1:0/900536397 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558fc93f9fc0 0x558fc93f1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:42.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.955+0000 7fe2815c5740 1 --2- 172.21.15.1:0/900536397 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fc93eb1d0 0x558fc940b110 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:42.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.955+0000 7fe2815c5740 1 --2- 172.21.15.1:0/900536397 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558fc93ea8b0 0x558fc940a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:42.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.955+0000 7fe2815c5740 1 -- 172.21.15.1:0/900536397 >> 172.21.15.1:0/900536397 conn(0x558fc93e6f60 msgr2=0x558fc93e1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:42.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.955+0000 7fe2815c5740 1 -- 172.21.15.1:0/900536397 shutdown_connections 2024-09-11T21:57:42.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.956+0000 7fe2815c5740 1 -- 172.21.15.1:0/900536397 wait complete. 2024-09-11T21:57:42.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.956+0000 7fe2815c5740 1 -- start start 2024-09-11T21:57:42.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.956+0000 7fe2815c5740 1 Processor -- start 2024-09-11T21:57:42.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.956+0000 7fe2815c5740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558fc93ea8b0 0x558fc9414310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:42.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.956+0000 7fe2815c5740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fc93eb1d0 0x558fc9414850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:42.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.956+0000 7fe2815c5740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558fc93f9fc0 0x558fc940d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:42.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.956+0000 7fe2815c5740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x558fc940bd30 con 0x558fc93eb1d0 2024-09-11T21:57:42.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.956+0000 7fe2815c5740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x558fc940b660 con 0x558fc93f9fc0 2024-09-11T21:57:42.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.956+0000 7fe2815c5740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x558fc940c400 con 0x558fc93ea8b0 2024-09-11T21:57:42.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.956+0000 7fe2815c5740 10 client.0 _ll_register_callbacks cb 0x7fe274004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:57:42.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.956+0000 7fe26ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fc93eb1d0 0x558fc9414850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:42.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.957+0000 7fe26ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fc93eb1d0 0x558fc9414850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33538/0 (socket says 172.21.15.1:33538) 2024-09-11T21:57:42.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.957+0000 7fe26ffff640 1 -- 172.21.15.1:0/149163195 learned_addr learned my addr 172.21.15.1:0/149163195 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:42.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.957+0000 7fe27cf88640 1 --2- 172.21.15.1:0/149163195 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558fc93ea8b0 0x558fc9414310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:42.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.957+0000 7fe27d789640 1 --2- 172.21.15.1:0/149163195 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558fc93f9fc0 0x558fc940d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:42.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.957+0000 7fe26ffff640 1 -- 172.21.15.1:0/149163195 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558fc93ea8b0 msgr2=0x558fc9414310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:42.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.957+0000 7fe26ffff640 1 --2- 172.21.15.1:0/149163195 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558fc93ea8b0 0x558fc9414310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:42.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.957+0000 7fe26ffff640 1 -- 172.21.15.1:0/149163195 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558fc93f9fc0 msgr2=0x558fc940d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:42.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.957+0000 7fe26ffff640 1 --2- 172.21.15.1:0/149163195 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558fc93f9fc0 0x558fc940d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:42.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.957+0000 7fe26ffff640 1 -- 172.21.15.1:0/149163195 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fc940de50 con 0x558fc93eb1d0 2024-09-11T21:57:42.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.957+0000 7fe27d789640 1 --2- 172.21.15.1:0/149163195 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558fc93f9fc0 0x558fc940d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T21:57:42.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.957+0000 7fe26ffff640 1 --2- 172.21.15.1:0/149163195 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fc93eb1d0 0x558fc9414850 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe268009fd0 tx=0x7fe268099730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4ae6dd43ca2e793 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:42.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.957+0000 7fe257fff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T21:57:42.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.957+0000 7fe257fff640 1 -- 172.21.15.1:0/149163195 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2680a3db0 con 0x558fc93eb1d0 2024-09-11T21:57:42.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.957+0000 7fe257fff640 1 -- 172.21.15.1:0/149163195 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7fe2680cc960 con 0x558fc93eb1d0 2024-09-11T21:57:42.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.957+0000 7fe2815c5740 1 -- 172.21.15.1:0/149163195 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe268066460 con 0x558fc93eb1d0 2024-09-11T21:57:42.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.957+0000 7fe257fff640 1 -- 172.21.15.1:0/149163195 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe26809bad0 con 0x558fc93eb1d0 2024-09-11T21:57:42.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.957+0000 7fe2815c5740 10 client.15546 fetch_fsmap learned FSMap version 28 2024-09-11T21:57:42.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.957+0000 7fe2815c5740 1 -- 172.21.15.1:0/149163195 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558fc9412800 con 0x558fc93eb1d0 2024-09-11T21:57:42.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.958+0000 7fe257fff640 1 -- 172.21.15.1:0/149163195 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe268066460 con 0x558fc93eb1d0 2024-09-11T21:57:42.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.958+0000 7fe257fff640 1 -- 172.21.15.1:0/149163195 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe26809a820 con 0x558fc93eb1d0 2024-09-11T21:57:42.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.958+0000 7fe2815c5740 10 client.15546 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T21:57:42.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.958+0000 7fe2815c5740 10 client.15546 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T21:57:42.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.958+0000 7fe2815c5740 20 client.15546 populate_metadata read hostname 'smithi001' 2024-09-11T21:57:42.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.958+0000 7fe2815c5740 1 --2- 172.21.15.1:0/149163195 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x558fc9479190 0x558fc9477ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:42.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.958+0000 7fe2815c5740 4 client.15546 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T21:57:42.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.958+0000 7fe2815c5740 1 -- 172.21.15.1:0/149163195 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "status"}) -- 0x558fc9358b10 con 0x558fc9479190 2024-09-11T21:57:42.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.958+0000 7fe27cf88640 1 --2- 172.21.15.1:0/149163195 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x558fc9479190 0x558fc9477ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:42.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.959+0000 7fe27cf88640 1 --2- 172.21.15.1:0/149163195 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x558fc9479190 0x558fc9477ea0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a891afa4c700ad26 server_cookie=d0babf3565d70c8f in_seq=0 out_seq=0 2024-09-11T21:57:42.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.959+0000 7fe257fff640 10 client.15546 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T21:57:42.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.959+0000 7fe257fff640 1 -- 172.21.15.1:0/149163195 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x558fc9358b10 con 0x558fc9479190 2024-09-11T21:57:42.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.959+0000 7fe257fff640 10 client.15546 handle_command_reply: tid=0 2024-09-11T21:57:42.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.959+0000 7fe2815c5740 1 client.15546 shutdown 2024-09-11T21:57:42.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.959+0000 7fe2815c5740 1 -- 172.21.15.1:0/149163195 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fc93eb1d0 msgr2=0x558fc9414850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:42.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.959+0000 7fe2815c5740 1 --2- 172.21.15.1:0/149163195 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fc93eb1d0 0x558fc9414850 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe268009fd0 tx=0x7fe268099730 comp rx=0 tx=0).stop 2024-09-11T21:57:42.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.959+0000 7fe2815c5740 1 -- 172.21.15.1:0/149163195 shutdown_connections 2024-09-11T21:57:42.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.959+0000 7fe2815c5740 1 --2- 172.21.15.1:0/149163195 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x558fc9479190 0x558fc9477ea0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:42.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.959+0000 7fe2815c5740 1 --2- 172.21.15.1:0/149163195 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558fc93f9fc0 0x558fc940d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:42.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.959+0000 7fe2815c5740 1 --2- 172.21.15.1:0/149163195 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fc93eb1d0 0x558fc9414850 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:42.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.959+0000 7fe2815c5740 1 --2- 172.21.15.1:0/149163195 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558fc93ea8b0 0x558fc9414310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:42.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.959+0000 7fe2815c5740 1 -- 172.21.15.1:0/149163195 >> 172.21.15.1:0/149163195 conn(0x558fc93e6f60 msgr2=0x558fc93f1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:42.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.959+0000 7fe257fff640 0 client.15546 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T21:57:42.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.959+0000 7fe2815c5740 1 -- 172.21.15.1:0/149163195 shutdown_connections 2024-09-11T21:57:42.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.959+0000 7fe2815c5740 1 -- 172.21.15.1:0/149163195 wait complete. 2024-09-11T21:57:42.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.959+0000 7fe2815c5740 20 client.15546 trim_cache size 0 max 16384 2024-09-11T21:57:42.966 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-11T21:57:42.966 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "3ffca83c-7087-11ef-bcea-c7b262605968", 2024-09-11T21:57:42.966 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 0, 2024-09-11T21:57:42.966 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 24494, 2024-09-11T21:57:42.966 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-09-11T21:57:42.966 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-09-11T21:57:42.966 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-09-11T21:57:42.966 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 141.86990791400001, 2024-09-11T21:57:42.966 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 28, 2024-09-11T21:57:42.966 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 87, 2024-09-11T21:57:42.967 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 86, 2024-09-11T21:57:42.967 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 152.539579828 2024-09-11T21:57:42.967 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-11T21:57:42.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.960+0000 7fe27f1ec640 1 -- 172.21.15.1:0/1457374597 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fe2600a2d80 msgr2=0x7fe2600a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:42.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.960+0000 7fe27f1ec640 1 --2- 172.21.15.1:0/1457374597 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fe2600a2d80 0x7fe2600a5240 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe27004c000 tx=0x7fe27004d000 comp rx=0 tx=0).stop 2024-09-11T21:57:42.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.960+0000 7fe27f1ec640 1 -- 172.21.15.1:0/1457374597 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe278102780 msgr2=0x7fe278198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:42.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.960+0000 7fe27f1ec640 1 --2- 172.21.15.1:0/1457374597 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe278102780 0x7fe278198960 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fe26800d950 tx=0x7fe26800de10 comp rx=0 tx=0).stop 2024-09-11T21:57:42.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.960+0000 7fe27f1ec640 1 -- 172.21.15.1:0/1457374597 shutdown_connections 2024-09-11T21:57:42.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.960+0000 7fe27f1ec640 1 --2- 172.21.15.1:0/1457374597 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fe2600a2d80 0x7fe2600a5240 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:42.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.960+0000 7fe27f1ec640 1 --2- 172.21.15.1:0/1457374597 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe278106f00 0x7fe27819ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:42.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.960+0000 7fe27f1ec640 1 --2- 172.21.15.1:0/1457374597 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe278102780 0x7fe278198960 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:42.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.960+0000 7fe27f1ec640 1 --2- 172.21.15.1:0/1457374597 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe278101dd0 0x7fe278198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:42.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.960+0000 7fe27f1ec640 1 -- 172.21.15.1:0/1457374597 >> 172.21.15.1:0/1457374597 conn(0x7fe2780fd8b0 msgr2=0x7fe2780febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:42.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.960+0000 7fe27f1ec640 1 -- 172.21.15.1:0/1457374597 shutdown_connections 2024-09-11T21:57:42.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:42.960+0000 7fe27f1ec640 1 -- 172.21.15.1:0/1457374597 wait complete. 2024-09-11T21:57:43.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:43 smithi001 ceph-mon[30935]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.7 KiB/s wr, 17 op/s 2024-09-11T21:57:43.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:43 smithi053 ceph-mon[34908]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.7 KiB/s wr, 17 op/s 2024-09-11T21:57:43.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:43 smithi161 ceph-mon[35615]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.7 KiB/s wr, 17 op/s 2024-09-11T21:57:43.677 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-09-11T21:57:45.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:45 smithi053 ceph-mon[34908]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.7 KiB/s wr, 13 op/s 2024-09-11T21:57:45.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:45 smithi161 ceph-mon[35615]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.7 KiB/s wr, 13 op/s 2024-09-11T21:57:45.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:45 smithi001 ceph-mon[30935]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.7 KiB/s wr, 13 op/s 2024-09-11T21:57:47.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:47 smithi053 ceph-mon[34908]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.7 KiB/s wr, 17 op/s 2024-09-11T21:57:47.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:47 smithi161 ceph-mon[35615]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.7 KiB/s wr, 17 op/s 2024-09-11T21:57:47.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:47 smithi001 ceph-mon[30935]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.7 KiB/s wr, 17 op/s 2024-09-11T21:57:49.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:49 smithi053 ceph-mon[34908]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.8 KiB/s wr, 13 op/s 2024-09-11T21:57:49.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:49 smithi161 ceph-mon[35615]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.8 KiB/s wr, 13 op/s 2024-09-11T21:57:49.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:49 smithi001 ceph-mon[30935]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.8 KiB/s wr, 13 op/s 2024-09-11T21:57:49.530 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.526+0000 7fb16ead0640 1 -- 172.21.15.1:0/1214436950 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb168101ac0 msgr2=0x7fb168101f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:49.530 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.526+0000 7fb16ead0640 1 --2- 172.21.15.1:0/1214436950 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb168101ac0 0x7fb168101f20 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fb154009930 tx=0x7fb15402a420 comp rx=0 tx=0).stop 2024-09-11T21:57:49.530 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.526+0000 7fb16ead0640 1 -- 172.21.15.1:0/1214436950 shutdown_connections 2024-09-11T21:57:49.530 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.526+0000 7fb16ead0640 1 --2- 172.21.15.1:0/1214436950 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb168102460 0x7fb16810e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:49.531 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.526+0000 7fb16ead0640 1 --2- 172.21.15.1:0/1214436950 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb168101ac0 0x7fb168101f20 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:49.531 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.526+0000 7fb16ead0640 1 --2- 172.21.15.1:0/1214436950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb168107ad0 0x7fb168107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:49.531 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.526+0000 7fb16ead0640 1 -- 172.21.15.1:0/1214436950 >> 172.21.15.1:0/1214436950 conn(0x7fb1680fd7a0 msgr2=0x7fb1680ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:49.531 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.526+0000 7fb16ead0640 1 -- 172.21.15.1:0/1214436950 shutdown_connections 2024-09-11T21:57:49.531 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.526+0000 7fb16ead0640 1 -- 172.21.15.1:0/1214436950 wait complete. 2024-09-11T21:57:49.532 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.527+0000 7fb16ead0640 1 Processor -- start 2024-09-11T21:57:49.532 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.527+0000 7fb16ead0640 1 -- start start 2024-09-11T21:57:49.532 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.527+0000 7fb16ead0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb168101ac0 0x7fb1680744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:49.532 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.527+0000 7fb16ead0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb168102460 0x7fb168074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:49.532 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.527+0000 7fb16ead0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb168107ad0 0x7fb168074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:49.533 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.527+0000 7fb16ead0640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb168111f10 con 0x7fb168101ac0 2024-09-11T21:57:49.533 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.527+0000 7fb16ead0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb168111d90 con 0x7fb168102460 2024-09-11T21:57:49.533 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.527+0000 7fb16ead0640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb168112090 con 0x7fb168107ad0 2024-09-11T21:57:49.533 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.527+0000 7fb16c86c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb168101ac0 0x7fb1680744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:49.533 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.527+0000 7fb16c86c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb168101ac0 0x7fb1680744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39384/0 (socket says 172.21.15.1:39384) 2024-09-11T21:57:49.533 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.527+0000 7fb16c86c640 1 -- 172.21.15.1:0/814095623 learned_addr learned my addr 172.21.15.1:0/814095623 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:49.533 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.527+0000 7fb167fff640 1 --2- 172.21.15.1:0/814095623 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb168102460 0x7fb168074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:49.533 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.527+0000 7fb16d06d640 1 --2- 172.21.15.1:0/814095623 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb168107ad0 0x7fb168074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:49.533 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.527+0000 7fb16c86c640 1 -- 172.21.15.1:0/814095623 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb168107ad0 msgr2=0x7fb168074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:49.533 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.527+0000 7fb16c86c640 1 --2- 172.21.15.1:0/814095623 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb168107ad0 0x7fb168074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:49.533 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.527+0000 7fb16c86c640 1 -- 172.21.15.1:0/814095623 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb168102460 msgr2=0x7fb168074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:49.533 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.527+0000 7fb16c86c640 1 --2- 172.21.15.1:0/814095623 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb168102460 0x7fb168074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:49.533 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.527+0000 7fb16c86c640 1 -- 172.21.15.1:0/814095623 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1681a6900 con 0x7fb168101ac0 2024-09-11T21:57:49.534 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.527+0000 7fb16d06d640 1 --2- 172.21.15.1:0/814095623 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb168107ad0 0x7fb168074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T21:57:49.534 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.528+0000 7fb16c86c640 1 --2- 172.21.15.1:0/814095623 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb168101ac0 0x7fb1680744f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb15c00e9f0 tx=0x7fb15c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:49.534 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.528+0000 7fb165ffb640 1 -- 172.21.15.1:0/814095623 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15c00cd80 con 0x7fb168101ac0 2024-09-11T21:57:49.534 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.528+0000 7fb16ead0640 1 -- 172.21.15.1:0/814095623 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1681a6bf0 con 0x7fb168101ac0 2024-09-11T21:57:49.534 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.528+0000 7fb16ead0640 1 -- 172.21.15.1:0/814095623 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1681a7160 con 0x7fb168101ac0 2024-09-11T21:57:49.534 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.528+0000 7fb165ffb640 1 -- 172.21.15.1:0/814095623 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb15c010910 con 0x7fb168101ac0 2024-09-11T21:57:49.534 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.528+0000 7fb165ffb640 1 -- 172.21.15.1:0/814095623 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15c018980 con 0x7fb168101ac0 2024-09-11T21:57:49.542 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.530+0000 7fb165ffb640 1 -- 172.21.15.1:0/814095623 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 24) ==== 138008+0+0 (secure 0 0 0) 0x7fb15c010ac0 con 0x7fb168101ac0 2024-09-11T21:57:49.542 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.532+0000 7fb165ffb640 1 --2- 172.21.15.1:0/814095623 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fb1580a2d80 0x7fb1580a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:49.542 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.532+0000 7fb17186c740 1 Processor -- start 2024-09-11T21:57:49.542 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.532+0000 7fb17186c740 1 -- start start 2024-09-11T21:57:49.542 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.532+0000 7fb167fff640 1 --2- 172.21.15.1:0/814095623 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fb1580a2d80 0x7fb1580a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:49.542 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.532+0000 7fb165ffb640 1 -- 172.21.15.1:0/814095623 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7fb15c0cf870 con 0x7fb168101ac0 2024-09-11T21:57:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.532+0000 7fb17186c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d93e5caf0 0x562d93e5ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.532+0000 7fb17186c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d93e5d410 0x562d93e67770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.532+0000 7fb17186c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d93e67cb0 0x562d93e6a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.532+0000 7fb17186c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x562d93dc12f0 con 0x562d93e5caf0 2024-09-11T21:57:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.532+0000 7fb17186c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x562d93d69270 con 0x562d93e67cb0 2024-09-11T21:57:49.543 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.532+0000 7fb17186c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x562d93dbd2b0 con 0x562d93e5d410 2024-09-11T21:57:49.544 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.532+0000 7fb16c86c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d93e5caf0 0x562d93e5ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:49.544 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.532+0000 7fb16c86c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d93e5caf0 0x562d93e5ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39386/0 (socket says 172.21.15.1:39386) 2024-09-11T21:57:49.544 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.532+0000 7fb16c86c640 1 -- 172.21.15.1:0/4118251179 learned_addr learned my addr 172.21.15.1:0/4118251179 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:49.544 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.532+0000 7fb167fff640 1 --2- 172.21.15.1:0/814095623 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fb1580a2d80 0x7fb1580a5240 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb154009900 tx=0x7fb1540058d0 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:49.544 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.532+0000 7fb16d06d640 1 --2- 172.21.15.1:0/4118251179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d93e67cb0 0x562d93e6a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:49.545 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.532+0000 7fb167fff640 1 --2- 172.21.15.1:0/4118251179 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d93e5d410 0x562d93e67770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:49.545 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.533+0000 7fb16c86c640 1 -- 172.21.15.1:0/4118251179 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d93e5d410 msgr2=0x562d93e67770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:49.545 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.533+0000 7fb16c86c640 1 --2- 172.21.15.1:0/4118251179 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d93e5d410 0x562d93e67770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:49.545 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.533+0000 7fb16c86c640 1 -- 172.21.15.1:0/4118251179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d93e67cb0 msgr2=0x562d93e6a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:49.545 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.533+0000 7fb16c86c640 1 --2- 172.21.15.1:0/4118251179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d93e67cb0 0x562d93e6a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:49.546 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.533+0000 7fb16c86c640 1 -- 172.21.15.1:0/4118251179 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d93d68340 con 0x562d93e5caf0 2024-09-11T21:57:49.546 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.533+0000 7fb16d06d640 1 --2- 172.21.15.1:0/4118251179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d93e67cb0 0x562d93e6a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T21:57:49.546 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.533+0000 7fb167fff640 1 --2- 172.21.15.1:0/4118251179 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d93e5d410 0x562d93e67770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T21:57:49.546 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.533+0000 7fb16c86c640 1 --2- 172.21.15.1:0/4118251179 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d93e5caf0 0x562d93e5ced0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fb15c000c00 tx=0x7fb15c09bb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3145f065883c23ae server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:49.546 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.533+0000 7fb152ffd640 1 -- 172.21.15.1:0/4118251179 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15c09bc20 con 0x562d93e5caf0 2024-09-11T21:57:49.547 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.533+0000 7fb152ffd640 1 -- 172.21.15.1:0/4118251179 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb15c09ffd0 con 0x562d93e5caf0 2024-09-11T21:57:49.547 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.533+0000 7fb152ffd640 1 -- 172.21.15.1:0/4118251179 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15c09b2d0 con 0x562d93e5caf0 2024-09-11T21:57:49.547 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.534+0000 7fb17186c740 1 -- 172.21.15.1:0/4118251179 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d93e5caf0 msgr2=0x562d93e5ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:49.547 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.534+0000 7fb17186c740 1 --2- 172.21.15.1:0/4118251179 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d93e5caf0 0x562d93e5ced0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fb15c000c00 tx=0x7fb15c09bb20 comp rx=0 tx=0).stop 2024-09-11T21:57:49.547 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.534+0000 7fb17186c740 1 -- 172.21.15.1:0/4118251179 shutdown_connections 2024-09-11T21:57:49.548 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.534+0000 7fb17186c740 1 --2- 172.21.15.1:0/4118251179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d93e67cb0 0x562d93e6a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:49.548 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.534+0000 7fb17186c740 1 --2- 172.21.15.1:0/4118251179 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d93e5d410 0x562d93e67770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:49.548 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.534+0000 7fb17186c740 1 --2- 172.21.15.1:0/4118251179 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d93e5caf0 0x562d93e5ced0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:49.548 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.534+0000 7fb17186c740 1 -- 172.21.15.1:0/4118251179 >> 172.21.15.1:0/4118251179 conn(0x562d93e59060 msgr2=0x562d93e59440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:49.548 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.534+0000 7fb17186c740 1 -- 172.21.15.1:0/4118251179 shutdown_connections 2024-09-11T21:57:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.534+0000 7fb17186c740 1 -- 172.21.15.1:0/4118251179 wait complete. 2024-09-11T21:57:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.534+0000 7fb17186c740 1 -- start start 2024-09-11T21:57:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.535+0000 7fb17186c740 1 Processor -- start 2024-09-11T21:57:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.535+0000 7fb17186c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d93e5caf0 0x562d93e64b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.535+0000 7fb17186c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d93e5d410 0x562d93e65090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.535+0000 7fb17186c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d93e67cb0 0x562d93e655d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.535+0000 7fb17186c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x562d93e6a880 con 0x562d93e5d410 2024-09-11T21:57:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.535+0000 7fb17186c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x562d93d4a290 con 0x562d93e5caf0 2024-09-11T21:57:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.535+0000 7fb17186c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x562d93e6aa00 con 0x562d93e67cb0 2024-09-11T21:57:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.535+0000 7fb17186c740 10 client.0 _ll_register_callbacks cb 0x562d93d7b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:57:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.535+0000 7fb167fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d93e5d410 0x562d93e65090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.535+0000 7fb167fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d93e5d410 0x562d93e65090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39388/0 (socket says 172.21.15.1:39388) 2024-09-11T21:57:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.535+0000 7fb167fff640 1 -- 172.21.15.1:0/4173694352 learned_addr learned my addr 172.21.15.1:0/4173694352 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.535+0000 7fb16d06d640 1 --2- 172.21.15.1:0/4173694352 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d93e67cb0 0x562d93e655d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.535+0000 7fb16c86c640 1 --2- 172.21.15.1:0/4173694352 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d93e5caf0 0x562d93e64b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.536+0000 7fb167fff640 1 -- 172.21.15.1:0/4173694352 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d93e67cb0 msgr2=0x562d93e655d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.536+0000 7fb167fff640 1 --2- 172.21.15.1:0/4173694352 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d93e67cb0 0x562d93e655d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.536+0000 7fb167fff640 1 -- 172.21.15.1:0/4173694352 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d93e5caf0 msgr2=0x562d93e64b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.536+0000 7fb167fff640 1 --2- 172.21.15.1:0/4173694352 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d93e5caf0 0x562d93e64b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.536+0000 7fb167fff640 1 -- 172.21.15.1:0/4173694352 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d93ad4210 con 0x562d93e5d410 2024-09-11T21:57:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.536+0000 7fb16c86c640 1 --2- 172.21.15.1:0/4173694352 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d93e5caf0 0x562d93e64b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T21:57:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.536+0000 7fb167fff640 1 --2- 172.21.15.1:0/4173694352 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d93e5d410 0x562d93e65090 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fb15402a940 tx=0x7fb154009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e55b6ee2162f70ba server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.536+0000 7fb147fff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T21:57:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.536+0000 7fb147fff640 1 -- 172.21.15.1:0/4173694352 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb154035040 con 0x562d93e5d410 2024-09-11T21:57:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.536+0000 7fb147fff640 1 -- 172.21.15.1:0/4173694352 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7fb154003ac0 con 0x562d93e5d410 2024-09-11T21:57:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.536+0000 7fb17186c740 1 -- 172.21.15.1:0/4173694352 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb15c09ffd0 con 0x562d93e5d410 2024-09-11T21:57:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.536+0000 7fb147fff640 1 -- 172.21.15.1:0/4173694352 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb154003cc0 con 0x562d93e5d410 2024-09-11T21:57:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.536+0000 7fb17186c740 10 client.15558 fetch_fsmap learned FSMap version 28 2024-09-11T21:57:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.536+0000 7fb17186c740 1 -- 172.21.15.1:0/4173694352 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562d93ba0620 con 0x562d93e5d410 2024-09-11T21:57:49.552 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.536+0000 7fb147fff640 1 -- 172.21.15.1:0/4173694352 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb15c09ffd0 con 0x562d93e5d410 2024-09-11T21:57:49.552 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.537+0000 7fb147fff640 1 -- 172.21.15.1:0/4173694352 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb15403c4d0 con 0x562d93e5d410 2024-09-11T21:57:49.552 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.537+0000 7fb17186c740 10 client.15558 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T21:57:49.552 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.537+0000 7fb17186c740 10 client.15558 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T21:57:49.552 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.537+0000 7fb17186c740 20 client.15558 populate_metadata read hostname 'smithi001' 2024-09-11T21:57:49.552 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.537+0000 7fb17186c740 1 --2- 172.21.15.1:0/4173694352 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x562d93e77b70 0x562d93e79fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.537+0000 7fb17186c740 4 client.15558 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T21:57:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.537+0000 7fb17186c740 1 -- 172.21.15.1:0/4173694352 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d93dcea20 con 0x562d93e77b70 2024-09-11T21:57:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.537+0000 7fb16c86c640 1 --2- 172.21.15.1:0/4173694352 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x562d93e77b70 0x562d93e79fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.537+0000 7fb16c86c640 1 --2- 172.21.15.1:0/4173694352 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x562d93e77b70 0x562d93e79fd0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b08c0924b12b1455 server_cookie=e7507ad68c2304e9 in_seq=0 out_seq=0 2024-09-11T21:57:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.537+0000 7fb147fff640 10 client.15558 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T21:57:49.554 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.538+0000 7fb147fff640 1 -- 172.21.15.1:0/4173694352 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562d93dcea20 con 0x562d93e77b70 2024-09-11T21:57:49.554 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.538+0000 7fb147fff640 10 client.15558 handle_command_reply: tid=0 2024-09-11T21:57:49.554 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.538+0000 7fb17186c740 1 client.15558 shutdown 2024-09-11T21:57:49.554 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.538+0000 7fb17186c740 1 -- 172.21.15.1:0/4173694352 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d93e5d410 msgr2=0x562d93e65090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:49.554 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.538+0000 7fb17186c740 1 --2- 172.21.15.1:0/4173694352 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d93e5d410 0x562d93e65090 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fb15402a940 tx=0x7fb154009260 comp rx=0 tx=0).stop 2024-09-11T21:57:49.554 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.538+0000 7fb17186c740 1 -- 172.21.15.1:0/4173694352 shutdown_connections 2024-09-11T21:57:49.554 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.538+0000 7fb17186c740 1 --2- 172.21.15.1:0/4173694352 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x562d93e77b70 0x562d93e79fd0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:49.554 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.538+0000 7fb17186c740 1 --2- 172.21.15.1:0/4173694352 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d93e67cb0 0x562d93e655d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:49.554 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.538+0000 7fb17186c740 1 --2- 172.21.15.1:0/4173694352 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d93e5d410 0x562d93e65090 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:49.554 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.538+0000 7fb17186c740 1 --2- 172.21.15.1:0/4173694352 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d93e5caf0 0x562d93e64b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:49.554 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.538+0000 7fb17186c740 1 -- 172.21.15.1:0/4173694352 >> 172.21.15.1:0/4173694352 conn(0x562d93e59060 msgr2=0x562d93e5ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:49.554 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.538+0000 7fb147fff640 0 client.15558 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T21:57:49.554 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.539+0000 7fb17186c740 1 -- 172.21.15.1:0/4173694352 shutdown_connections 2024-09-11T21:57:49.555 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.539+0000 7fb17186c740 1 -- 172.21.15.1:0/4173694352 wait complete. 2024-09-11T21:57:49.555 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.539+0000 7fb17186c740 20 client.15558 trim_cache size 0 max 16384 2024-09-11T21:57:49.561 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.553+0000 7fb17186c740 1 Processor -- start 2024-09-11T21:57:49.562 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.553+0000 7fb17186c740 1 -- start start 2024-09-11T21:57:49.562 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.554+0000 7fb17186c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d93e5caf0 0x562d93e65400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:49.562 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.554+0000 7fb17186c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d93e5d410 0x562d93e66ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:49.562 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.554+0000 7fb17186c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d93e67cb0 0x562d93e7f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:49.562 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.554+0000 7fb17186c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb14c00e7d0 con 0x562d93e5caf0 2024-09-11T21:57:49.562 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.554+0000 7fb17186c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb14c010310 con 0x562d93e5d410 2024-09-11T21:57:49.562 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.554+0000 7fb17186c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb14c00eea0 con 0x562d93e67cb0 2024-09-11T21:57:49.562 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.554+0000 7fb16c86c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d93e5caf0 0x562d93e65400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:49.562 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.554+0000 7fb16c86c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d93e5caf0 0x562d93e65400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39394/0 (socket says 172.21.15.1:39394) 2024-09-11T21:57:49.562 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.554+0000 7fb16c86c640 1 -- 172.21.15.1:0/415619646 learned_addr learned my addr 172.21.15.1:0/415619646 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:49.562 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.554+0000 7fb16d06d640 1 --2- 172.21.15.1:0/415619646 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d93e67cb0 0x562d93e7f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:49.562 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.554+0000 7fb16c86c640 1 -- 172.21.15.1:0/415619646 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d93e67cb0 msgr2=0x562d93e7f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:49.562 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.554+0000 7fb16c86c640 1 --2- 172.21.15.1:0/415619646 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d93e67cb0 0x562d93e7f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:49.563 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.554+0000 7fb16c86c640 1 -- 172.21.15.1:0/415619646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d93e5d410 msgr2=0x562d93e66ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:49.563 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.554+0000 7fb167fff640 1 --2- 172.21.15.1:0/415619646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d93e5d410 0x562d93e66ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:49.563 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.554+0000 7fb16c86c640 1 --2- 172.21.15.1:0/415619646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d93e5d410 0x562d93e66ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:49.563 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.554+0000 7fb16c86c640 1 -- 172.21.15.1:0/415619646 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d93e58e90 con 0x562d93e5caf0 2024-09-11T21:57:49.563 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.554+0000 7fb167fff640 1 --2- 172.21.15.1:0/415619646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d93e5d410 0x562d93e66ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T21:57:49.563 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.554+0000 7fb16c86c640 1 --2- 172.21.15.1:0/415619646 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d93e5caf0 0x562d93e65400 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fb15c09b920 tx=0x7fb15c097ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cf6180a6d7103f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:49.563 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.554+0000 7fb1527fc640 1 -- 172.21.15.1:0/415619646 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15c09ffd0 con 0x562d93e5caf0 2024-09-11T21:57:49.563 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.554+0000 7fb1527fc640 1 -- 172.21.15.1:0/415619646 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb15c09bc20 con 0x562d93e5caf0 2024-09-11T21:57:49.563 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.554+0000 7fb1527fc640 1 -- 172.21.15.1:0/415619646 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15c097970 con 0x562d93e5caf0 2024-09-11T21:57:49.563 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.555+0000 7fb17186c740 1 -- 172.21.15.1:0/415619646 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d93e5caf0 msgr2=0x562d93e65400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:49.563 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.555+0000 7fb17186c740 1 --2- 172.21.15.1:0/415619646 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d93e5caf0 0x562d93e65400 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fb15c09b920 tx=0x7fb15c097ff0 comp rx=0 tx=0).stop 2024-09-11T21:57:49.563 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.555+0000 7fb17186c740 1 -- 172.21.15.1:0/415619646 shutdown_connections 2024-09-11T21:57:49.563 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.555+0000 7fb17186c740 1 --2- 172.21.15.1:0/415619646 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d93e67cb0 0x562d93e7f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:49.564 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.555+0000 7fb17186c740 1 --2- 172.21.15.1:0/415619646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d93e5d410 0x562d93e66ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:49.564 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.555+0000 7fb17186c740 1 --2- 172.21.15.1:0/415619646 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d93e5caf0 0x562d93e65400 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:49.564 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.555+0000 7fb17186c740 1 -- 172.21.15.1:0/415619646 >> 172.21.15.1:0/415619646 conn(0x562d93e59060 msgr2=0x562d93e663b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:49.564 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.555+0000 7fb17186c740 1 -- 172.21.15.1:0/415619646 shutdown_connections 2024-09-11T21:57:49.564 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.555+0000 7fb17186c740 1 -- 172.21.15.1:0/415619646 wait complete. 2024-09-11T21:57:49.564 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.555+0000 7fb17186c740 1 -- start start 2024-09-11T21:57:49.564 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.555+0000 7fb17186c740 1 Processor -- start 2024-09-11T21:57:49.564 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.556+0000 7fb17186c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d93e5caf0 0x562d93e6ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:49.564 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.556+0000 7fb17186c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d93e5d410 0x562d93e70490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:49.564 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.556+0000 7fb17186c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d93e67cb0 0x562d93e613e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:49.564 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.556+0000 7fb17186c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb14c00fc40 con 0x562d93e67cb0 2024-09-11T21:57:49.564 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.556+0000 7fb17186c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb14c00f570 con 0x562d93e5caf0 2024-09-11T21:57:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.556+0000 7fb17186c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb14c00c7b0 con 0x562d93e5d410 2024-09-11T21:57:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.556+0000 7fb17186c740 10 client.0 _ll_register_callbacks cb 0x7fb154040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:57:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.556+0000 7fb16d06d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d93e67cb0 0x562d93e613e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.556+0000 7fb16d06d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d93e67cb0 0x562d93e613e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39400/0 (socket says 172.21.15.1:39400) 2024-09-11T21:57:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.556+0000 7fb16d06d640 1 -- 172.21.15.1:0/1510854536 learned_addr learned my addr 172.21.15.1:0/1510854536 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.556+0000 7fb16c86c640 1 --2- 172.21.15.1:0/1510854536 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d93e5caf0 0x562d93e6ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.556+0000 7fb167fff640 1 --2- 172.21.15.1:0/1510854536 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d93e5d410 0x562d93e70490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.556+0000 7fb16d06d640 1 -- 172.21.15.1:0/1510854536 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d93e5d410 msgr2=0x562d93e70490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.556+0000 7fb16d06d640 1 --2- 172.21.15.1:0/1510854536 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d93e5d410 0x562d93e70490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.556+0000 7fb16d06d640 1 -- 172.21.15.1:0/1510854536 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d93e5caf0 msgr2=0x562d93e6ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.556+0000 7fb16d06d640 1 --2- 172.21.15.1:0/1510854536 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d93e5caf0 0x562d93e6ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.556+0000 7fb16d06d640 1 -- 172.21.15.1:0/1510854536 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d93e61c10 con 0x562d93e67cb0 2024-09-11T21:57:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.556+0000 7fb167fff640 1 --2- 172.21.15.1:0/1510854536 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d93e5d410 0x562d93e70490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:57:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.557+0000 7fb16d06d640 1 --2- 172.21.15.1:0/1510854536 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d93e67cb0 0x562d93e613e0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fb16000fbc0 tx=0x7fb16000fcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95ceb343937dc138 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.557+0000 7fb147fff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T21:57:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.557+0000 7fb147fff640 1 -- 172.21.15.1:0/1510854536 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16001e060 con 0x562d93e67cb0 2024-09-11T21:57:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.557+0000 7fb147fff640 1 -- 172.21.15.1:0/1510854536 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7fb16000ed00 con 0x562d93e67cb0 2024-09-11T21:57:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.557+0000 7fb17186c740 1 -- 172.21.15.1:0/1510854536 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb15c09bc20 con 0x562d93e67cb0 2024-09-11T21:57:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.557+0000 7fb147fff640 1 -- 172.21.15.1:0/1510854536 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb160022410 con 0x562d93e67cb0 2024-09-11T21:57:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.557+0000 7fb17186c740 10 client.15564 fetch_fsmap learned FSMap version 28 2024-09-11T21:57:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.557+0000 7fb17186c740 1 -- 172.21.15.1:0/1510854536 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562d93ee23d0 con 0x562d93e67cb0 2024-09-11T21:57:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.557+0000 7fb147fff640 1 -- 172.21.15.1:0/1510854536 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb15c09bc20 con 0x562d93e67cb0 2024-09-11T21:57:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.557+0000 7fb147fff640 1 -- 172.21.15.1:0/1510854536 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb160018070 con 0x562d93e67cb0 2024-09-11T21:57:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.557+0000 7fb17186c740 10 client.15564 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T21:57:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.557+0000 7fb17186c740 10 client.15564 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T21:57:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.557+0000 7fb17186c740 20 client.15564 populate_metadata read hostname 'smithi001' 2024-09-11T21:57:49.567 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.557+0000 7fb17186c740 1 --2- 172.21.15.1:0/1510854536 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x562d93ee2b50 0x562d93ee4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:49.567 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.557+0000 7fb17186c740 4 client.15564 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T21:57:49.567 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.557+0000 7fb17186c740 1 -- 172.21.15.1:0/1510854536 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562d93dc8d70 con 0x562d93ee2b50 2024-09-11T21:57:49.567 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.558+0000 7fb16c86c640 1 --2- 172.21.15.1:0/1510854536 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x562d93ee2b50 0x562d93ee4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:49.567 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.558+0000 7fb16c86c640 1 --2- 172.21.15.1:0/1510854536 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x562d93ee2b50 0x562d93ee4f70 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd9d1a0ad3c08ba server_cookie=67d03e0b11f342e9 in_seq=0 out_seq=0 2024-09-11T21:57:49.567 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.558+0000 7fb147fff640 10 client.15564 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T21:57:49.568 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.564+0000 7fb147fff640 1 -- 172.21.15.1:0/1510854536 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562d93dc8d70 con 0x562d93ee2b50 2024-09-11T21:57:49.568 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.564+0000 7fb147fff640 10 client.15564 handle_command_reply: tid=0 2024-09-11T21:57:49.568 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.564+0000 7fb17186c740 1 client.15564 shutdown 2024-09-11T21:57:49.568 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.565+0000 7fb17186c740 1 -- 172.21.15.1:0/1510854536 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d93e67cb0 msgr2=0x562d93e613e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:49.569 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.565+0000 7fb17186c740 1 --2- 172.21.15.1:0/1510854536 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d93e67cb0 0x562d93e613e0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fb16000fbc0 tx=0x7fb16000fcb0 comp rx=0 tx=0).stop 2024-09-11T21:57:49.569 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.565+0000 7fb17186c740 1 -- 172.21.15.1:0/1510854536 shutdown_connections 2024-09-11T21:57:49.569 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.565+0000 7fb17186c740 1 --2- 172.21.15.1:0/1510854536 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x562d93ee2b50 0x562d93ee4f70 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:49.569 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.565+0000 7fb17186c740 1 --2- 172.21.15.1:0/1510854536 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562d93e67cb0 0x562d93e613e0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:49.569 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.565+0000 7fb17186c740 1 --2- 172.21.15.1:0/1510854536 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562d93e5d410 0x562d93e70490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:49.569 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.565+0000 7fb17186c740 1 --2- 172.21.15.1:0/1510854536 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562d93e5caf0 0x562d93e6ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:49.569 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.565+0000 7fb147fff640 0 client.15564 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T21:57:49.569 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.565+0000 7fb17186c740 1 -- 172.21.15.1:0/1510854536 >> 172.21.15.1:0/1510854536 conn(0x562d93e59060 msgr2=0x562d93e581c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:49.569 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.565+0000 7fb17186c740 1 -- 172.21.15.1:0/1510854536 shutdown_connections 2024-09-11T21:57:49.569 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.565+0000 7fb17186c740 1 -- 172.21.15.1:0/1510854536 wait complete. 2024-09-11T21:57:49.569 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.565+0000 7fb17186c740 20 client.15564 trim_cache size 0 max 16384 2024-09-11T21:57:49.570 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-09-11T21:57:49.570 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-09-11T21:57:49.570 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-09-11T21:57:49.570 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-09-11T21:57:49.571 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.566+0000 7fb16ead0640 1 -- 172.21.15.1:0/814095623 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fb1580a2d80 msgr2=0x7fb1580a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:49.571 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.566+0000 7fb16ead0640 1 --2- 172.21.15.1:0/814095623 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fb1580a2d80 0x7fb1580a5240 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb154009900 tx=0x7fb1540058d0 comp rx=0 tx=0).stop 2024-09-11T21:57:49.571 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.566+0000 7fb16ead0640 1 -- 172.21.15.1:0/814095623 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb168101ac0 msgr2=0x7fb1680744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:49.571 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.566+0000 7fb16ead0640 1 --2- 172.21.15.1:0/814095623 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb168101ac0 0x7fb1680744f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb15c00e9f0 tx=0x7fb15c00eeb0 comp rx=0 tx=0).stop 2024-09-11T21:57:49.571 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.566+0000 7fb16ead0640 1 -- 172.21.15.1:0/814095623 shutdown_connections 2024-09-11T21:57:49.571 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.566+0000 7fb16ead0640 1 --2- 172.21.15.1:0/814095623 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fb1580a2d80 0x7fb1580a5240 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:49.571 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.566+0000 7fb16ead0640 1 --2- 172.21.15.1:0/814095623 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb168107ad0 0x7fb168074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:49.571 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.566+0000 7fb16ead0640 1 --2- 172.21.15.1:0/814095623 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb168102460 0x7fb168074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:49.571 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.566+0000 7fb16ead0640 1 --2- 172.21.15.1:0/814095623 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb168101ac0 0x7fb1680744f0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:49.571 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.566+0000 7fb16ead0640 1 -- 172.21.15.1:0/814095623 >> 172.21.15.1:0/814095623 conn(0x7fb1680fd7a0 msgr2=0x7fb1680fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:49.572 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.566+0000 7fb16ead0640 1 -- 172.21.15.1:0/814095623 shutdown_connections 2024-09-11T21:57:49.572 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:57:49.566+0000 7fb16ead0640 1 -- 172.21.15.1:0/814095623 wait complete. 2024-09-11T21:57:51.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:51 smithi053 ceph-mon[34908]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.8 KiB/s wr, 14 op/s 2024-09-11T21:57:51.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:51 smithi161 ceph-mon[35615]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.8 KiB/s wr, 14 op/s 2024-09-11T21:57:51.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:51 smithi001 ceph-mon[30935]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.8 KiB/s wr, 14 op/s 2024-09-11T21:57:53.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:53 smithi053 ceph-mon[34908]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.1 KiB/s wr, 16 op/s 2024-09-11T21:57:53.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:53 smithi161 ceph-mon[35615]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.1 KiB/s wr, 16 op/s 2024-09-11T21:57:53.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:53 smithi001 ceph-mon[30935]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.1 KiB/s wr, 16 op/s 2024-09-11T21:57:53.606 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T21:57:53.880 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:57:54.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.393+0000 7f526cd0f640 1 -- 172.21.15.1:0/2377908438 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5268101de0 msgr2=0x7f52681021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:54.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.393+0000 7f526cd0f640 1 --2- 172.21.15.1:0/2377908438 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5268101de0 0x7f52681021c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f5258009900 tx=0x7f525802a410 comp rx=0 tx=0).stop 2024-09-11T21:57:54.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.393+0000 7f526cd0f640 1 -- 172.21.15.1:0/2377908438 shutdown_connections 2024-09-11T21:57:54.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.393+0000 7f526cd0f640 1 --2- 172.21.15.1:0/2377908438 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5268106f10 0x7f526810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:54.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.393+0000 7f526cd0f640 1 --2- 172.21.15.1:0/2377908438 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5268102790 0x7f52681067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:54.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.393+0000 7f526cd0f640 1 --2- 172.21.15.1:0/2377908438 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5268101de0 0x7f52681021c0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:54.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.393+0000 7f526cd0f640 1 -- 172.21.15.1:0/2377908438 >> 172.21.15.1:0/2377908438 conn(0x7f52680fd8e0 msgr2=0x7f52680ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:54.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.393+0000 7f526cd0f640 1 -- 172.21.15.1:0/2377908438 shutdown_connections 2024-09-11T21:57:54.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.393+0000 7f526cd0f640 1 -- 172.21.15.1:0/2377908438 wait complete. 2024-09-11T21:57:54.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.394+0000 7f526cd0f640 1 Processor -- start 2024-09-11T21:57:54.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.394+0000 7f526cd0f640 1 -- start start 2024-09-11T21:57:54.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.394+0000 7f526cd0f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5268101de0 0x7f5268198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:54.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.394+0000 7f526cd0f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5268102790 0x7f5268198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:54.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.394+0000 7f526cd0f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5268106f10 0x7f526819cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:54.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.394+0000 7f526cd0f640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5268111f30 con 0x7f5268101de0 2024-09-11T21:57:54.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.394+0000 7f526cd0f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5268111db0 con 0x7f5268102790 2024-09-11T21:57:54.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.394+0000 7f526cd0f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f52681120b0 con 0x7f5268106f10 2024-09-11T21:57:54.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.394+0000 7f526659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5268101de0 0x7f5268198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:54.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.394+0000 7f526659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5268101de0 0x7f5268198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39422/0 (socket says 172.21.15.1:39422) 2024-09-11T21:57:54.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.394+0000 7f526659c640 1 -- 172.21.15.1:0/238796673 learned_addr learned my addr 172.21.15.1:0/238796673 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:54.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.395+0000 7f5266d9d640 1 --2- 172.21.15.1:0/238796673 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5268106f10 0x7f526819cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:54.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.395+0000 7f5265d9b640 1 --2- 172.21.15.1:0/238796673 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5268102790 0x7f5268198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:54.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.395+0000 7f526659c640 1 -- 172.21.15.1:0/238796673 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5268106f10 msgr2=0x7f526819cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:54.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.395+0000 7f526659c640 1 --2- 172.21.15.1:0/238796673 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5268106f10 0x7f526819cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:54.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.395+0000 7f526659c640 1 -- 172.21.15.1:0/238796673 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5268102790 msgr2=0x7f5268198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:54.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.395+0000 7f526659c640 1 --2- 172.21.15.1:0/238796673 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5268102790 0x7f5268198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:54.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.395+0000 7f526659c640 1 -- 172.21.15.1:0/238796673 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f526819d3c0 con 0x7f5268101de0 2024-09-11T21:57:54.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.395+0000 7f526659c640 1 --2- 172.21.15.1:0/238796673 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5268101de0 0x7f5268198410 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f5258006010 tx=0x7f52580042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:54.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.395+0000 7f52577fe640 1 -- 172.21.15.1:0/238796673 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f525803f030 con 0x7f5268101de0 2024-09-11T21:57:54.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.395+0000 7f526cd0f640 1 -- 172.21.15.1:0/238796673 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f526819d650 con 0x7f5268101de0 2024-09-11T21:57:54.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.395+0000 7f52577fe640 1 -- 172.21.15.1:0/238796673 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5258003ce0 con 0x7f5268101de0 2024-09-11T21:57:54.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.395+0000 7f52577fe640 1 -- 172.21.15.1:0/238796673 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5258035070 con 0x7f5268101de0 2024-09-11T21:57:54.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.395+0000 7f526cd0f640 1 -- 172.21.15.1:0/238796673 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52681a4f90 con 0x7f5268101de0 2024-09-11T21:57:54.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.398+0000 7f526f0e8740 1 Processor -- start 2024-09-11T21:57:54.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.398+0000 7f526f0e8740 1 -- start start 2024-09-11T21:57:54.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.399+0000 7f526f0e8740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef96b5d4a0 0x55ef96b5d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:54.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.399+0000 7f526f0e8740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef96b5ddc0 0x55ef96b68140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:54.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.399+0000 7f526f0e8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef96b68680 0x55ef96b6aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:54.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.399+0000 7f526f0e8740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55ef96ac32e0 con 0x55ef96b5ddc0 2024-09-11T21:57:54.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.399+0000 7f526f0e8740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55ef96a6ace0 con 0x55ef96b5d4a0 2024-09-11T21:57:54.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.399+0000 7f526f0e8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ef96abf2a0 con 0x55ef96b68680 2024-09-11T21:57:54.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.399+0000 7f52577fe640 1 -- 172.21.15.1:0/238796673 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 24) ==== 138008+0+0 (secure 0 0 0) 0x7f5258042030 con 0x7f5268101de0 2024-09-11T21:57:54.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.399+0000 7f5266d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef96b5d4a0 0x55ef96b5d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:54.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.399+0000 7f5266d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef96b5d4a0 0x55ef96b5d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:50004/0 (socket says 172.21.15.1:50004) 2024-09-11T21:57:54.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.399+0000 7f5266d9d640 1 -- 172.21.15.1:0/3912753788 learned_addr learned my addr 172.21.15.1:0/3912753788 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:54.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.399+0000 7f526659c640 1 --2- 172.21.15.1:0/3912753788 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef96b5ddc0 0x55ef96b68140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:54.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.399+0000 7f5265d9b640 1 --2- 172.21.15.1:0/3912753788 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef96b68680 0x55ef96b6aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:54.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.399+0000 7f5266d9d640 1 -- 172.21.15.1:0/3912753788 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef96b68680 msgr2=0x55ef96b6aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:54.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.399+0000 7f5266d9d640 1 --2- 172.21.15.1:0/3912753788 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef96b68680 0x55ef96b6aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:54.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.399+0000 7f5266d9d640 1 -- 172.21.15.1:0/3912753788 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef96b5ddc0 msgr2=0x55ef96b68140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:54.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.399+0000 7f5266d9d640 1 --2- 172.21.15.1:0/3912753788 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef96b5ddc0 0x55ef96b68140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:54.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.399+0000 7f5266d9d640 1 -- 172.21.15.1:0/3912753788 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef96a4f440 con 0x55ef96b5d4a0 2024-09-11T21:57:54.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.400+0000 7f5266d9d640 1 --2- 172.21.15.1:0/3912753788 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef96b5d4a0 0x55ef96b5d880 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5260009f70 tx=0x7f5260010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34985bb3aaf815a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:54.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.400+0000 7f5254ff9640 1 -- 172.21.15.1:0/3912753788 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5260016070 con 0x55ef96b5d4a0 2024-09-11T21:57:54.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.400+0000 7f5254ff9640 1 -- 172.21.15.1:0/3912753788 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5260004d20 con 0x55ef96b5d4a0 2024-09-11T21:57:54.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.400+0000 7f5254ff9640 1 -- 172.21.15.1:0/3912753788 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5260012910 con 0x55ef96b5d4a0 2024-09-11T21:57:54.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.400+0000 7f52577fe640 1 --2- 172.21.15.1:0/238796673 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f52500a2d80 0x7f52500a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:54.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.400+0000 7f526659c640 1 --2- 172.21.15.1:0/238796673 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f52500a2d80 0x7f52500a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:54.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.400+0000 7f52577fe640 1 -- 172.21.15.1:0/238796673 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7f52580efcb0 con 0x7f5268101de0 2024-09-11T21:57:54.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.401+0000 7f526f0e8740 1 -- 172.21.15.1:0/3912753788 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef96b5d4a0 msgr2=0x55ef96b5d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:54.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.401+0000 7f526f0e8740 1 --2- 172.21.15.1:0/3912753788 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef96b5d4a0 0x55ef96b5d880 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5260009f70 tx=0x7f5260010800 comp rx=0 tx=0).stop 2024-09-11T21:57:54.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.401+0000 7f526659c640 1 --2- 172.21.15.1:0/238796673 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f52500a2d80 0x7f52500a5240 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f52580ed3a0 tx=0x7f52580f3730 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:54.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.401+0000 7f526f0e8740 1 -- 172.21.15.1:0/3912753788 shutdown_connections 2024-09-11T21:57:54.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.401+0000 7f526f0e8740 1 --2- 172.21.15.1:0/3912753788 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef96b68680 0x55ef96b6aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:54.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.401+0000 7f526f0e8740 1 --2- 172.21.15.1:0/3912753788 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef96b5ddc0 0x55ef96b68140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:54.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.401+0000 7f526f0e8740 1 --2- 172.21.15.1:0/3912753788 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef96b5d4a0 0x55ef96b5d880 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:54.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.401+0000 7f526f0e8740 1 -- 172.21.15.1:0/3912753788 >> 172.21.15.1:0/3912753788 conn(0x55ef96b59a30 msgr2=0x55ef96b59e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:54.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.401+0000 7f526f0e8740 1 -- 172.21.15.1:0/3912753788 shutdown_connections 2024-09-11T21:57:54.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.401+0000 7f526f0e8740 1 -- 172.21.15.1:0/3912753788 wait complete. 2024-09-11T21:57:54.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.401+0000 7f526f0e8740 1 -- start start 2024-09-11T21:57:54.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.401+0000 7f526f0e8740 1 Processor -- start 2024-09-11T21:57:54.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.402+0000 7f526f0e8740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef96b5d4a0 0x55ef96b67fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:54.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.402+0000 7f526f0e8740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef96b5ddc0 0x55ef96b65260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:54.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.402+0000 7f526f0e8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef96b68680 0x55ef96b657a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:54.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.402+0000 7f526f0e8740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55ef96b6b250 con 0x55ef96b5ddc0 2024-09-11T21:57:54.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.402+0000 7f526f0e8740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55ef96a7e050 con 0x55ef96b5d4a0 2024-09-11T21:57:54.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.402+0000 7f526f0e8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ef96b6b3d0 con 0x55ef96b68680 2024-09-11T21:57:54.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.402+0000 7f526f0e8740 10 client.0 _ll_register_callbacks cb 0x55ef96abe160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:57:54.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.402+0000 7f5265d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef96b5ddc0 0x55ef96b65260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:54.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.402+0000 7f5265d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef96b5ddc0 0x55ef96b65260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39440/0 (socket says 172.21.15.1:39440) 2024-09-11T21:57:54.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.402+0000 7f5265d9b640 1 -- 172.21.15.1:0/2925601767 learned_addr learned my addr 172.21.15.1:0/2925601767 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:54.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.402+0000 7f5266d9d640 1 --2- 172.21.15.1:0/2925601767 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef96b5d4a0 0x55ef96b67fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:54.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.402+0000 7f5266d9d640 1 --2- 172.21.15.1:0/2925601767 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef96b68680 0x55ef96b657a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:54.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.402+0000 7f5265d9b640 1 -- 172.21.15.1:0/2925601767 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef96b68680 msgr2=0x55ef96b657a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:54.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.402+0000 7f5265d9b640 1 --2- 172.21.15.1:0/2925601767 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef96b68680 0x55ef96b657a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:54.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.402+0000 7f5265d9b640 1 -- 172.21.15.1:0/2925601767 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef96b5d4a0 msgr2=0x55ef96b67fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:54.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.402+0000 7f5265d9b640 1 --2- 172.21.15.1:0/2925601767 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef96b5d4a0 0x55ef96b67fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:54.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.402+0000 7f5265d9b640 1 -- 172.21.15.1:0/2925601767 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef968343c0 con 0x55ef96b5ddc0 2024-09-11T21:57:54.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.402+0000 7f5266d9d640 1 --2- 172.21.15.1:0/2925601767 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef96b5d4a0 0x55ef96b67fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T21:57:54.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.402+0000 7f5265d9b640 1 --2- 172.21.15.1:0/2925601767 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef96b5ddc0 0x55ef96b65260 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f525c011740 tx=0x7f525c011c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fdf739a2ec0f383 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:54.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.402+0000 7f524dffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T21:57:54.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.403+0000 7f524dffb640 1 -- 172.21.15.1:0/2925601767 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f525c01e060 con 0x55ef96b5ddc0 2024-09-11T21:57:54.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.403+0000 7f524dffb640 1 -- 172.21.15.1:0/2925601767 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7f525c00fdb0 con 0x55ef96b5ddc0 2024-09-11T21:57:54.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.403+0000 7f526f0e8740 1 -- 172.21.15.1:0/2925601767 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5260004d20 con 0x55ef96b5ddc0 2024-09-11T21:57:54.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.403+0000 7f524dffb640 1 -- 172.21.15.1:0/2925601767 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f525c022410 con 0x55ef96b5ddc0 2024-09-11T21:57:54.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.403+0000 7f526f0e8740 10 client.15576 fetch_fsmap learned FSMap version 28 2024-09-11T21:57:54.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.403+0000 7f526f0e8740 1 -- 172.21.15.1:0/2925601767 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ef96b76c60 con 0x55ef96b5ddc0 2024-09-11T21:57:54.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.403+0000 7f524dffb640 1 -- 172.21.15.1:0/2925601767 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5260004d20 con 0x55ef96b5ddc0 2024-09-11T21:57:54.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.403+0000 7f524dffb640 1 -- 172.21.15.1:0/2925601767 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f525c018030 con 0x55ef96b5ddc0 2024-09-11T21:57:54.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.403+0000 7f526f0e8740 10 client.15576 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T21:57:54.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.403+0000 7f526f0e8740 10 client.15576 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T21:57:54.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.403+0000 7f526f0e8740 20 client.15576 populate_metadata read hostname 'smithi001' 2024-09-11T21:57:54.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.403+0000 7f526f0e8740 1 --2- 172.21.15.1:0/2925601767 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55ef96b78170 0x55ef96b7a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:54.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.404+0000 7f526f0e8740 4 client.15576 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T21:57:54.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.404+0000 7f526f0e8740 1 -- 172.21.15.1:0/2925601767 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef96ad0850 con 0x55ef96b78170 2024-09-11T21:57:54.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.404+0000 7f5266d9d640 1 --2- 172.21.15.1:0/2925601767 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55ef96b78170 0x55ef96b7a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:54.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.404+0000 7f524dffb640 10 client.15576 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T21:57:54.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.404+0000 7f5266d9d640 1 --2- 172.21.15.1:0/2925601767 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55ef96b78170 0x55ef96b7a5b0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d98f4fdcb94e9c89 server_cookie=e34d51ddc541bdd1 in_seq=0 out_seq=0 2024-09-11T21:57:54.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.405+0000 7f524dffb640 1 -- 172.21.15.1:0/2925601767 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ef96ad0850 con 0x55ef96b78170 2024-09-11T21:57:54.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.405+0000 7f524dffb640 10 client.15576 handle_command_reply: tid=0 2024-09-11T21:57:54.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.405+0000 7f526f0e8740 1 client.15576 shutdown 2024-09-11T21:57:54.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.405+0000 7f526f0e8740 1 -- 172.21.15.1:0/2925601767 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef96b5ddc0 msgr2=0x55ef96b65260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:54.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.405+0000 7f526f0e8740 1 --2- 172.21.15.1:0/2925601767 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef96b5ddc0 0x55ef96b65260 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f525c011740 tx=0x7f525c011c00 comp rx=0 tx=0).stop 2024-09-11T21:57:54.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.405+0000 7f526f0e8740 1 -- 172.21.15.1:0/2925601767 shutdown_connections 2024-09-11T21:57:54.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.405+0000 7f526f0e8740 1 --2- 172.21.15.1:0/2925601767 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55ef96b78170 0x55ef96b7a5b0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:54.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.405+0000 7f526f0e8740 1 --2- 172.21.15.1:0/2925601767 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef96b68680 0x55ef96b657a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:54.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.405+0000 7f526f0e8740 1 --2- 172.21.15.1:0/2925601767 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef96b5ddc0 0x55ef96b65260 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:54.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.405+0000 7f526f0e8740 1 --2- 172.21.15.1:0/2925601767 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef96b5d4a0 0x55ef96b67fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:54.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.405+0000 7f526f0e8740 1 -- 172.21.15.1:0/2925601767 >> 172.21.15.1:0/2925601767 conn(0x55ef96b59a30 msgr2=0x55ef96b58b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:54.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.405+0000 7f524dffb640 0 client.15576 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T21:57:54.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.405+0000 7f526f0e8740 1 -- 172.21.15.1:0/2925601767 shutdown_connections 2024-09-11T21:57:54.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.405+0000 7f526f0e8740 1 -- 172.21.15.1:0/2925601767 wait complete. 2024-09-11T21:57:54.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.405+0000 7f526f0e8740 20 client.15576 trim_cache size 0 max 16384 2024-09-11T21:57:54.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.423+0000 7f526f0e8740 1 Processor -- start 2024-09-11T21:57:54.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.423+0000 7f526f0e8740 1 -- start start 2024-09-11T21:57:54.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.424+0000 7f526f0e8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef96b59a30 0x55ef96b744b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:54.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.424+0000 7f526f0e8740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef96b5d4a0 0x55ef96b7fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:54.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.424+0000 7f526f0e8740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef96b5ddc0 0x55ef96b803c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:54.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.424+0000 7f526f0e8740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f52681ad920 con 0x55ef96b5d4a0 2024-09-11T21:57:54.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.424+0000 7f526f0e8740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f52681af460 con 0x55ef96b5ddc0 2024-09-11T21:57:54.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.424+0000 7f526f0e8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f52681adff0 con 0x55ef96b59a30 2024-09-11T21:57:54.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.424+0000 7f5265d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef96b5d4a0 0x55ef96b7fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:54.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.424+0000 7f5265d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef96b5d4a0 0x55ef96b7fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39442/0 (socket says 172.21.15.1:39442) 2024-09-11T21:57:54.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.424+0000 7f5265d9b640 1 -- 172.21.15.1:0/2555331645 learned_addr learned my addr 172.21.15.1:0/2555331645 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:54.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.424+0000 7f5266d9d640 1 --2- 172.21.15.1:0/2555331645 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef96b59a30 0x55ef96b744b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:54.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.424+0000 7f5266d9d640 1 --2- 172.21.15.1:0/2555331645 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef96b5ddc0 0x55ef96b803c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:54.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.424+0000 7f5265d9b640 1 -- 172.21.15.1:0/2555331645 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef96b59a30 msgr2=0x55ef96b744b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:54.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.424+0000 7f5265d9b640 1 --2- 172.21.15.1:0/2555331645 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef96b59a30 0x55ef96b744b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:54.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.424+0000 7f5265d9b640 1 -- 172.21.15.1:0/2555331645 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef96b5ddc0 msgr2=0x55ef96b803c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:54.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.424+0000 7f5265d9b640 1 --2- 172.21.15.1:0/2555331645 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef96b5ddc0 0x55ef96b803c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:54.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.424+0000 7f5265d9b640 1 -- 172.21.15.1:0/2555331645 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef96b5e330 con 0x55ef96b5d4a0 2024-09-11T21:57:54.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.424+0000 7f5265d9b640 1 --2- 172.21.15.1:0/2555331645 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef96b5d4a0 0x55ef96b7fe80 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f525c00e690 tx=0x7f525c005a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24b605f441aa3d7c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:54.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.425+0000 7f524ffff640 1 -- 172.21.15.1:0/2555331645 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f525c011240 con 0x55ef96b5d4a0 2024-09-11T21:57:54.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.425+0000 7f524ffff640 1 -- 172.21.15.1:0/2555331645 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f525c011e50 con 0x55ef96b5d4a0 2024-09-11T21:57:54.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.425+0000 7f524ffff640 1 -- 172.21.15.1:0/2555331645 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f525c00eb20 con 0x55ef96b5d4a0 2024-09-11T21:57:54.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.425+0000 7f526f0e8740 1 -- 172.21.15.1:0/2555331645 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef96b5d4a0 msgr2=0x55ef96b7fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:54.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.425+0000 7f526f0e8740 1 --2- 172.21.15.1:0/2555331645 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef96b5d4a0 0x55ef96b7fe80 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f525c00e690 tx=0x7f525c005a90 comp rx=0 tx=0).stop 2024-09-11T21:57:54.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.426+0000 7f526f0e8740 1 -- 172.21.15.1:0/2555331645 shutdown_connections 2024-09-11T21:57:54.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.426+0000 7f526f0e8740 1 --2- 172.21.15.1:0/2555331645 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef96b5ddc0 0x55ef96b803c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:54.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.426+0000 7f526f0e8740 1 --2- 172.21.15.1:0/2555331645 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef96b5d4a0 0x55ef96b7fe80 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:54.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.426+0000 7f526f0e8740 1 --2- 172.21.15.1:0/2555331645 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef96b59a30 0x55ef96b744b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:54.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.426+0000 7f526f0e8740 1 -- 172.21.15.1:0/2555331645 >> 172.21.15.1:0/2555331645 conn(0x55ef96b65900 msgr2=0x55ef96b64350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:54.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.426+0000 7f526f0e8740 1 -- 172.21.15.1:0/2555331645 shutdown_connections 2024-09-11T21:57:54.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.426+0000 7f526f0e8740 1 -- 172.21.15.1:0/2555331645 wait complete. 2024-09-11T21:57:54.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.426+0000 7f526f0e8740 1 -- start start 2024-09-11T21:57:54.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.426+0000 7f526f0e8740 1 Processor -- start 2024-09-11T21:57:54.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.426+0000 7f526f0e8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef96b59a30 0x55ef96b6de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:54.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.426+0000 7f526f0e8740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef96b5d4a0 0x55ef96b6e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:54.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.426+0000 7f526f0e8740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef96b5ddc0 0x55ef96bd9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:54.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.426+0000 7f526f0e8740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f52681aed90 con 0x55ef96b5d4a0 2024-09-11T21:57:54.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.426+0000 7f526f0e8740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f52681ae6c0 con 0x55ef96b5ddc0 2024-09-11T21:57:54.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.426+0000 7f526f0e8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f52681ab900 con 0x55ef96b59a30 2024-09-11T21:57:54.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.426+0000 7f526f0e8740 10 client.0 _ll_register_callbacks cb 0x7f525c00c690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:57:54.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.427+0000 7f5265d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef96b5d4a0 0x55ef96b6e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:54.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.427+0000 7f5265d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef96b5d4a0 0x55ef96b6e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39450/0 (socket says 172.21.15.1:39450) 2024-09-11T21:57:54.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.427+0000 7f5265d9b640 1 -- 172.21.15.1:0/766468609 learned_addr learned my addr 172.21.15.1:0/766468609 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:54.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.427+0000 7f5266d9d640 1 --2- 172.21.15.1:0/766468609 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef96b59a30 0x55ef96b6de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:54.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.427+0000 7f5266d9d640 1 --2- 172.21.15.1:0/766468609 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef96b5ddc0 0x55ef96bd9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:54.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.427+0000 7f5265d9b640 1 -- 172.21.15.1:0/766468609 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef96b59a30 msgr2=0x55ef96b6de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:54.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.427+0000 7f5265d9b640 1 --2- 172.21.15.1:0/766468609 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef96b59a30 0x55ef96b6de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:54.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.427+0000 7f5265d9b640 1 -- 172.21.15.1:0/766468609 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef96b5ddc0 msgr2=0x55ef96bd9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:54.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.427+0000 7f5265d9b640 1 --2- 172.21.15.1:0/766468609 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef96b5ddc0 0x55ef96bd9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:54.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.427+0000 7f5265d9b640 1 -- 172.21.15.1:0/766468609 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef96bda4b0 con 0x55ef96b5d4a0 2024-09-11T21:57:54.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.427+0000 7f5266d9d640 1 --2- 172.21.15.1:0/766468609 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef96b5ddc0 0x55ef96bd9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T21:57:54.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.427+0000 7f5265d9b640 1 --2- 172.21.15.1:0/766468609 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef96b5d4a0 0x55ef96b6e3a0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f525c00e7b0 tx=0x7f525c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=670f11fa008fd239 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:54.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.427+0000 7f524dffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T21:57:54.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.427+0000 7f524dffb640 1 -- 172.21.15.1:0/766468609 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f525c004430 con 0x55ef96b5d4a0 2024-09-11T21:57:54.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.427+0000 7f524dffb640 1 -- 172.21.15.1:0/766468609 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7f525c00e8b0 con 0x55ef96b5d4a0 2024-09-11T21:57:54.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.427+0000 7f526f0e8740 1 -- 172.21.15.1:0/766468609 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f525c011e50 con 0x55ef96b5d4a0 2024-09-11T21:57:54.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.428+0000 7f524dffb640 1 -- 172.21.15.1:0/766468609 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f525c042050 con 0x55ef96b5d4a0 2024-09-11T21:57:54.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.428+0000 7f526f0e8740 10 client.15582 fetch_fsmap learned FSMap version 28 2024-09-11T21:57:54.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.428+0000 7f526f0e8740 1 -- 172.21.15.1:0/766468609 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ef96be2a30 con 0x55ef96b5d4a0 2024-09-11T21:57:54.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.428+0000 7f524dffb640 1 -- 172.21.15.1:0/766468609 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f525c011e50 con 0x55ef96b5d4a0 2024-09-11T21:57:54.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.428+0000 7f524dffb640 1 -- 172.21.15.1:0/766468609 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f525c018070 con 0x55ef96b5d4a0 2024-09-11T21:57:54.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.428+0000 7f526f0e8740 10 client.15582 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T21:57:54.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.428+0000 7f526f0e8740 10 client.15582 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T21:57:54.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.428+0000 7f526f0e8740 20 client.15582 populate_metadata read hostname 'smithi001' 2024-09-11T21:57:54.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.428+0000 7f526f0e8740 1 --2- 172.21.15.1:0/766468609 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55ef96b68680 0x55ef96be5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:54.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.428+0000 7f526f0e8740 4 client.15582 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T21:57:54.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.428+0000 7f526f0e8740 1 -- 172.21.15.1:0/766468609 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ef96acad60 con 0x55ef96b68680 2024-09-11T21:57:54.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.429+0000 7f5266d9d640 1 --2- 172.21.15.1:0/766468609 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55ef96b68680 0x55ef96be5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:54.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.429+0000 7f5266d9d640 1 --2- 172.21.15.1:0/766468609 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55ef96b68680 0x55ef96be5100 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acb85ca4c4b5b368 server_cookie=349416517a3ae14a in_seq=0 out_seq=0 2024-09-11T21:57:54.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.429+0000 7f524dffb640 10 client.15582 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T21:57:54.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.429+0000 7f524dffb640 1 -- 172.21.15.1:0/766468609 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ef96acad60 con 0x55ef96b68680 2024-09-11T21:57:54.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.429+0000 7f524dffb640 10 client.15582 handle_command_reply: tid=0 2024-09-11T21:57:54.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.429+0000 7f526f0e8740 1 client.15582 shutdown 2024-09-11T21:57:54.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.429+0000 7f526f0e8740 1 -- 172.21.15.1:0/766468609 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef96b5d4a0 msgr2=0x55ef96b6e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:54.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.429+0000 7f526f0e8740 1 --2- 172.21.15.1:0/766468609 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef96b5d4a0 0x55ef96b6e3a0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f525c00e7b0 tx=0x7f525c0058a0 comp rx=0 tx=0).stop 2024-09-11T21:57:54.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.429+0000 7f526f0e8740 1 -- 172.21.15.1:0/766468609 shutdown_connections 2024-09-11T21:57:54.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.429+0000 7f526f0e8740 1 --2- 172.21.15.1:0/766468609 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55ef96b68680 0x55ef96be5100 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:54.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.429+0000 7f526f0e8740 1 --2- 172.21.15.1:0/766468609 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ef96b5ddc0 0x55ef96bd9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:54.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.429+0000 7f526f0e8740 1 --2- 172.21.15.1:0/766468609 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef96b5d4a0 0x55ef96b6e3a0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:54.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.429+0000 7f524dffb640 0 client.15582 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T21:57:54.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.429+0000 7f526f0e8740 1 --2- 172.21.15.1:0/766468609 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ef96b59a30 0x55ef96b6de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:54.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.429+0000 7f526f0e8740 1 -- 172.21.15.1:0/766468609 >> 172.21.15.1:0/766468609 conn(0x55ef96b65900 msgr2=0x55ef96b58350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:54.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.430+0000 7f526f0e8740 1 -- 172.21.15.1:0/766468609 shutdown_connections 2024-09-11T21:57:54.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.430+0000 7f526f0e8740 1 -- 172.21.15.1:0/766468609 wait complete. 2024-09-11T21:57:54.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.430+0000 7f526f0e8740 20 client.15582 trim_cache size 0 max 16384 2024-09-11T21:57:54.441 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-11T21:57:54.441 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-09-11T21:57:54.441 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-09-11T21:57:54.441 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-11T21:57:54.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.430+0000 7f526cd0f640 1 -- 172.21.15.1:0/238796673 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f52500a2d80 msgr2=0x7f52500a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:54.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.430+0000 7f526cd0f640 1 --2- 172.21.15.1:0/238796673 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f52500a2d80 0x7f52500a5240 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f52580ed3a0 tx=0x7f52580f3730 comp rx=0 tx=0).stop 2024-09-11T21:57:54.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.430+0000 7f526cd0f640 1 -- 172.21.15.1:0/238796673 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5268101de0 msgr2=0x7f5268198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:54.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.430+0000 7f526cd0f640 1 --2- 172.21.15.1:0/238796673 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5268101de0 0x7f5268198410 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f5258006010 tx=0x7f52580042f0 comp rx=0 tx=0).stop 2024-09-11T21:57:54.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.430+0000 7f526cd0f640 1 -- 172.21.15.1:0/238796673 shutdown_connections 2024-09-11T21:57:54.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.431+0000 7f526cd0f640 1 --2- 172.21.15.1:0/238796673 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f52500a2d80 0x7f52500a5240 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:54.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.431+0000 7f526cd0f640 1 --2- 172.21.15.1:0/238796673 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5268106f10 0x7f526819cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:54.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.431+0000 7f526cd0f640 1 --2- 172.21.15.1:0/238796673 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5268102790 0x7f5268198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:54.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.431+0000 7f526cd0f640 1 --2- 172.21.15.1:0/238796673 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5268101de0 0x7f5268198410 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:54.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.431+0000 7f526cd0f640 1 -- 172.21.15.1:0/238796673 >> 172.21.15.1:0/238796673 conn(0x7f52680fd8e0 msgr2=0x7f52680fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:54.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.431+0000 7f526cd0f640 1 -- 172.21.15.1:0/238796673 shutdown_connections 2024-09-11T21:57:54.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:54.431+0000 7f526cd0f640 1 -- 172.21.15.1:0/238796673 wait complete. 2024-09-11T21:57:55.171 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T21:57:55.172 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T21:57:55.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:55 smithi053 ceph-mon[34908]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-11T21:57:55.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:55 smithi161 ceph-mon[35615]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-11T21:57:55.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:55 smithi001 ceph-mon[30935]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-11T21:57:55.452 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:57:55.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:55.952+0000 7fcf0f654640 1 -- 172.21.15.1:0/4077171972 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcf08102d30 msgr2=0x7fcf08103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:55.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:55.952+0000 7fcf0f654640 1 --2- 172.21.15.1:0/4077171972 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcf08102d30 0x7fcf08103110 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fcf04009930 tx=0x7fcf0402a420 comp rx=0 tx=0).stop 2024-09-11T21:57:55.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:55.952+0000 7fcf0f654640 1 -- 172.21.15.1:0/4077171972 shutdown_connections 2024-09-11T21:57:55.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:55.952+0000 7fcf0f654640 1 --2- 172.21.15.1:0/4077171972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf081082a0 0x7fcf0810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:55.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:55.952+0000 7fcf0f654640 1 --2- 172.21.15.1:0/4077171972 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf081036e0 0x7fcf08107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:55.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:55.952+0000 7fcf0f654640 1 --2- 172.21.15.1:0/4077171972 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcf08102d30 0x7fcf08103110 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:55.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:55.952+0000 7fcf0f654640 1 -- 172.21.15.1:0/4077171972 >> 172.21.15.1:0/4077171972 conn(0x7fcf080fc7d0 msgr2=0x7fcf080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:55.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:55.952+0000 7fcf0f654640 1 -- 172.21.15.1:0/4077171972 shutdown_connections 2024-09-11T21:57:55.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:55.952+0000 7fcf0f654640 1 -- 172.21.15.1:0/4077171972 wait complete. 2024-09-11T21:57:55.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:55.953+0000 7fcf0f654640 1 Processor -- start 2024-09-11T21:57:55.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:55.953+0000 7fcf0f654640 1 -- start start 2024-09-11T21:57:55.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:55.953+0000 7fcf0f654640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf08102d30 0x7fcf081985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:55.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:55.953+0000 7fcf0f654640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf081036e0 0x7fcf08198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:55.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:55.953+0000 7fcf0f654640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcf081082a0 0x7fcf0819ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:55.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:55.953+0000 7fcf0f654640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcf081120a0 con 0x7fcf081082a0 2024-09-11T21:57:55.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:55.953+0000 7fcf0f654640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcf08111f20 con 0x7fcf08102d30 2024-09-11T21:57:55.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:55.953+0000 7fcf0f654640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcf08112220 con 0x7fcf081036e0 2024-09-11T21:57:55.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:55.953+0000 7fcf0dbf1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcf081082a0 0x7fcf0819ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:55.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:55.953+0000 7fcf0d3f0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf08102d30 0x7fcf081985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:55.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:55.953+0000 7fcf0dbf1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcf081082a0 0x7fcf0819ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39478/0 (socket says 172.21.15.1:39478) 2024-09-11T21:57:55.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:55.953+0000 7fcf0dbf1640 1 -- 172.21.15.1:0/1079704133 learned_addr learned my addr 172.21.15.1:0/1079704133 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:55.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:55.953+0000 7fcf0d3f0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf08102d30 0x7fcf081985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:50066/0 (socket says 172.21.15.1:50066) 2024-09-11T21:57:55.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:55.953+0000 7fcf0cbef640 1 --2- 172.21.15.1:0/1079704133 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf081036e0 0x7fcf08198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:55.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:55.953+0000 7fcf0dbf1640 1 -- 172.21.15.1:0/1079704133 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf081036e0 msgr2=0x7fcf08198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:55.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:55.953+0000 7fcf0dbf1640 1 --2- 172.21.15.1:0/1079704133 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf081036e0 0x7fcf08198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:55.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:55.954+0000 7fcf0dbf1640 1 -- 172.21.15.1:0/1079704133 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf08102d30 msgr2=0x7fcf081985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:55.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:55.954+0000 7fcf0dbf1640 1 --2- 172.21.15.1:0/1079704133 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf08102d30 0x7fcf081985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:55.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:55.954+0000 7fcf0dbf1640 1 -- 172.21.15.1:0/1079704133 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf0819d550 con 0x7fcf081082a0 2024-09-11T21:57:55.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:55.954+0000 7fcf0dbf1640 1 --2- 172.21.15.1:0/1079704133 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcf081082a0 0x7fcf0819ce70 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fcf0000eaa0 tx=0x7fcf0000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:55.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:55.954+0000 7fcefe7fc640 1 -- 172.21.15.1:0/1079704133 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0000cdb0 con 0x7fcf081082a0 2024-09-11T21:57:55.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:55.954+0000 7fcf0f654640 1 -- 172.21.15.1:0/1079704133 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf0819d840 con 0x7fcf081082a0 2024-09-11T21:57:55.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:55.954+0000 7fcefe7fc640 1 -- 172.21.15.1:0/1079704133 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf00005020 con 0x7fcf081082a0 2024-09-11T21:57:55.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:55.954+0000 7fcefe7fc640 1 -- 172.21.15.1:0/1079704133 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf000116b0 con 0x7fcf081082a0 2024-09-11T21:57:55.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:55.954+0000 7fcf0f654640 1 -- 172.21.15.1:0/1079704133 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf081a5120 con 0x7fcf081082a0 2024-09-11T21:57:55.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:55.955+0000 7fcf0f654640 1 -- 172.21.15.1:0/1079704133 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf0806a850 con 0x7fcf081082a0 2024-09-11T21:57:55.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:55.956+0000 7fcefe7fc640 1 -- 172.21.15.1:0/1079704133 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 24) ==== 138008+0+0 (secure 0 0 0) 0x7fcf00005560 con 0x7fcf081082a0 2024-09-11T21:57:55.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:55.957+0000 7fcefe7fc640 1 --2- 172.21.15.1:0/1079704133 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fcef40a2e00 0x7fcef40a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:55.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:55.957+0000 7fcf0d3f0640 1 --2- 172.21.15.1:0/1079704133 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fcef40a2e00 0x7fcef40a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:55.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:55.957+0000 7fcefe7fc640 1 -- 172.21.15.1:0/1079704133 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7fcf000cfa60 con 0x7fcf081082a0 2024-09-11T21:57:55.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:55.958+0000 7fcf0d3f0640 1 --2- 172.21.15.1:0/1079704133 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fcef40a2e00 0x7fcef40a52c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fcf04009900 tx=0x7fcf040058d0 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:55.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:55.960+0000 7fcefe7fc640 1 -- 172.21.15.1:0/1079704133 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcf00016030 con 0x7fcf081082a0 2024-09-11T21:57:56.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:56.211+0000 7fcf0f654640 1 -- 172.21.15.1:0/1079704133 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcf0810a2b0 con 0x7fcf081082a0 2024-09-11T21:57:56.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:56.212+0000 7fcefe7fc640 1 -- 172.21.15.1:0/1079704133 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11949 (secure 0 0 0) 0x7fcf0009bf40 con 0x7fcf081082a0 2024-09-11T21:57:56.213 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:57:56.214 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":28,"btime":"2024-09-11T21:57:12:512925+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/421542832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":421542832},{"type":"v1","addr":"172.21.15.1:6835","nonce":421542832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2836864406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2836864406},{"type":"v1","addr":"172.21.15.1:6837","nonce":2836864406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T21:55:20.791263+0000","modified":"2024-09-11T21:57:12.462736+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24484,"mds_2":24467,"mds_3":24466,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14595":{"gid":14595,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/407834672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":407834672},{"type":"v1","addr":"172.21.15.1:6839","nonce":407834672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/1068827678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":1068827678},{"type":"v1","addr":"172.21.15.1:6841","nonce":1068827678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/872410024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":872410024},{"type":"v1","addr":"172.21.15.161:6833","nonce":872410024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/1648944570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1648944570},{"type":"v1","addr":"172.21.15.53:6833","nonce":1648944570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/2126933993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2126933993},{"type":"v1","addr":"172.21.15.53:6835","nonce":2126933993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.161:6835/828381592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":828381592},{"type":"v1","addr":"172.21.15.161:6835","nonce":828381592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.161:6837/1799318656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1799318656},{"type":"v1","addr":"172.21.15.161:6837","nonce":1799318656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.53:6837/3457131830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3457131830},{"type":"v1","addr":"172.21.15.53:6837","nonce":3457131830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/455365383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":455365383},{"type":"v1","addr":"172.21.15.161:6839","nonce":455365383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.53:6839/1378584447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1378584447},{"type":"v1","addr":"172.21.15.53:6839","nonce":1378584447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24460,24466,24467,24484,24494]},"id":1}]} 2024-09-11T21:57:56.214 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 28 2024-09-11T21:57:56.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:56.214+0000 7fcf0f654640 1 -- 172.21.15.1:0/1079704133 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fcef40a2e00 msgr2=0x7fcef40a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:56.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:56.214+0000 7fcf0f654640 1 --2- 172.21.15.1:0/1079704133 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fcef40a2e00 0x7fcef40a52c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fcf04009900 tx=0x7fcf040058d0 comp rx=0 tx=0).stop 2024-09-11T21:57:56.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:56.214+0000 7fcf0f654640 1 -- 172.21.15.1:0/1079704133 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcf081082a0 msgr2=0x7fcf0819ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:56.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:56.214+0000 7fcf0f654640 1 --2- 172.21.15.1:0/1079704133 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcf081082a0 0x7fcf0819ce70 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fcf0000eaa0 tx=0x7fcf0000ef60 comp rx=0 tx=0).stop 2024-09-11T21:57:56.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:56.214+0000 7fcf0f654640 1 -- 172.21.15.1:0/1079704133 shutdown_connections 2024-09-11T21:57:56.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:56.214+0000 7fcf0f654640 1 --2- 172.21.15.1:0/1079704133 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fcef40a2e00 0x7fcef40a52c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:56.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:56.214+0000 7fcf0f654640 1 --2- 172.21.15.1:0/1079704133 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcf081082a0 0x7fcf0819ce70 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:56.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:56.214+0000 7fcf0f654640 1 --2- 172.21.15.1:0/1079704133 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf081036e0 0x7fcf08198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:56.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:56.214+0000 7fcf0f654640 1 --2- 172.21.15.1:0/1079704133 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf08102d30 0x7fcf081985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:56.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:56.214+0000 7fcf0f654640 1 -- 172.21.15.1:0/1079704133 >> 172.21.15.1:0/1079704133 conn(0x7fcf080fc7d0 msgr2=0x7fcf08105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:56.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:56.214+0000 7fcf0f654640 1 -- 172.21.15.1:0/1079704133 shutdown_connections 2024-09-11T21:57:56.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:56.214+0000 7fcf0f654640 1 -- 172.21.15.1:0/1079704133 wait complete. 2024-09-11T21:57:56.932 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T21:57:57.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:57 smithi001 ceph-mon[30935]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 426 B/s wr, 13 op/s 2024-09-11T21:57:57.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:57 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/1079704133' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:57:57.202 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:57:57.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:57 smithi053 ceph-mon[34908]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 426 B/s wr, 13 op/s 2024-09-11T21:57:57.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:57 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/1079704133' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:57:57.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:57 smithi161 ceph-mon[35615]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 426 B/s wr, 13 op/s 2024-09-11T21:57:57.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:57 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/1079704133' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:57:57.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.680+0000 7f2088305640 1 -- 172.21.15.1:0/961648607 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2080100e20 msgr2=0x7f2080101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:57.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.680+0000 7f2088305640 1 --2- 172.21.15.1:0/961648607 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2080100e20 0x7f2080101200 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f207c009900 tx=0x7f207c02a480 comp rx=0 tx=0).stop 2024-09-11T21:57:57.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.680+0000 7f2088305640 1 -- 172.21.15.1:0/961648607 shutdown_connections 2024-09-11T21:57:57.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.680+0000 7f2088305640 1 --2- 172.21.15.1:0/961648607 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2080102ef0 0x7f208010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:57.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.680+0000 7f2088305640 1 --2- 172.21.15.1:0/961648607 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2080102570 0x7f20801029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:57.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.680+0000 7f2088305640 1 --2- 172.21.15.1:0/961648607 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2080100e20 0x7f2080101200 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:57.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.680+0000 7f2088305640 1 -- 172.21.15.1:0/961648607 >> 172.21.15.1:0/961648607 conn(0x7f20800fc7d0 msgr2=0x7f20800febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:57.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.680+0000 7f2088305640 1 -- 172.21.15.1:0/961648607 shutdown_connections 2024-09-11T21:57:57.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.680+0000 7f2088305640 1 -- 172.21.15.1:0/961648607 wait complete. 2024-09-11T21:57:57.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.681+0000 7f2088305640 1 Processor -- start 2024-09-11T21:57:57.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.681+0000 7f2088305640 1 -- start start 2024-09-11T21:57:57.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.681+0000 7f2088305640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2080100e20 0x7f2080197410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:57.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.681+0000 7f2088305640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2080102570 0x7f2080197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:57.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.681+0000 7f2088305640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2080102ef0 0x7f208019bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:57.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.681+0000 7f2088305640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2080110fd0 con 0x7f2080102570 2024-09-11T21:57:57.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.681+0000 7f2088305640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2080110e50 con 0x7f2080100e20 2024-09-11T21:57:57.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.681+0000 7f2088305640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2080111150 con 0x7f2080102ef0 2024-09-11T21:57:57.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.682+0000 7f20858a0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2080102570 0x7f2080197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:57.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.682+0000 7f20858a0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2080102570 0x7f2080197950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60162/0 (socket says 172.21.15.1:60162) 2024-09-11T21:57:57.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.682+0000 7f20858a0640 1 -- 172.21.15.1:0/1616224701 learned_addr learned my addr 172.21.15.1:0/1616224701 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:57.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.682+0000 7f20868a2640 1 --2- 172.21.15.1:0/1616224701 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2080102ef0 0x7f208019bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:57.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.682+0000 7f20860a1640 1 --2- 172.21.15.1:0/1616224701 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2080100e20 0x7f2080197410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:57.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.682+0000 7f20858a0640 1 -- 172.21.15.1:0/1616224701 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2080102ef0 msgr2=0x7f208019bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:57.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.682+0000 7f20858a0640 1 --2- 172.21.15.1:0/1616224701 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2080102ef0 0x7f208019bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:57.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.682+0000 7f20858a0640 1 -- 172.21.15.1:0/1616224701 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2080100e20 msgr2=0x7f2080197410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:57.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.682+0000 7f20858a0640 1 --2- 172.21.15.1:0/1616224701 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2080100e20 0x7f2080197410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:57.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.682+0000 7f20858a0640 1 -- 172.21.15.1:0/1616224701 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f208019c3c0 con 0x7f2080102570 2024-09-11T21:57:57.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.682+0000 7f20858a0640 1 --2- 172.21.15.1:0/1616224701 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2080102570 0x7f2080197950 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f207000d770 tx=0x7f207000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:57.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.682+0000 7f20777fe640 1 -- 172.21.15.1:0/1616224701 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2070017070 con 0x7f2080102570 2024-09-11T21:57:57.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.682+0000 7f2088305640 1 -- 172.21.15.1:0/1616224701 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f208019c6b0 con 0x7f2080102570 2024-09-11T21:57:57.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.682+0000 7f20777fe640 1 -- 172.21.15.1:0/1616224701 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2070005500 con 0x7f2080102570 2024-09-11T21:57:57.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.682+0000 7f20777fe640 1 -- 172.21.15.1:0/1616224701 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2070004e00 con 0x7f2080102570 2024-09-11T21:57:57.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.683+0000 7f2088305640 1 -- 172.21.15.1:0/1616224701 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2080069f70 con 0x7f2080102570 2024-09-11T21:57:57.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.685+0000 7f208a6de740 1 Processor -- start 2024-09-11T21:57:57.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.685+0000 7f208a6de740 1 -- start start 2024-09-11T21:57:57.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.686+0000 7f208a6de740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b980b66b40 0x55b980b65090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:57.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.686+0000 7f208a6de740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b980b655d0 0x55b980b65a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:57.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.686+0000 7f208a6de740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b980b70370 0x55b980b72760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:57.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.686+0000 7f208a6de740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55b980acb2f0 con 0x55b980b70370 2024-09-11T21:57:57.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.686+0000 7f208a6de740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55b980a72cf0 con 0x55b980b66b40 2024-09-11T21:57:57.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.686+0000 7f208a6de740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b980ac72b0 con 0x55b980b655d0 2024-09-11T21:57:57.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.686+0000 7f20777fe640 1 -- 172.21.15.1:0/1616224701 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 24) ==== 138008+0+0 (secure 0 0 0) 0x7f2070004fa0 con 0x7f2080102570 2024-09-11T21:57:57.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.686+0000 7f20868a2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b980b66b40 0x55b980b65090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:57.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.686+0000 7f20860a1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b980b655d0 0x55b980b65a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:57.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.686+0000 7f20868a2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b980b66b40 0x55b980b65090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:39840/0 (socket says 172.21.15.1:39840) 2024-09-11T21:57:57.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.686+0000 7f20868a2640 1 -- 172.21.15.1:0/2482288066 learned_addr learned my addr 172.21.15.1:0/2482288066 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:57.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.686+0000 7f20860a1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b980b655d0 0x55b980b65a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:37860/0 (socket says 172.21.15.1:37860) 2024-09-11T21:57:57.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.686+0000 7f20858a0640 1 --2- 172.21.15.1:0/2482288066 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b980b70370 0x55b980b72760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:57.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.686+0000 7f20868a2640 1 -- 172.21.15.1:0/2482288066 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b980b655d0 msgr2=0x55b980b65a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:57.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.686+0000 7f20868a2640 1 --2- 172.21.15.1:0/2482288066 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b980b655d0 0x55b980b65a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:57.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.686+0000 7f20868a2640 1 -- 172.21.15.1:0/2482288066 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b980b70370 msgr2=0x55b980b72760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:57.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.686+0000 7f20868a2640 1 --2- 172.21.15.1:0/2482288066 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b980b70370 0x55b980b72760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:57.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.686+0000 7f20868a2640 1 -- 172.21.15.1:0/2482288066 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b980a573a0 con 0x55b980b66b40 2024-09-11T21:57:57.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.687+0000 7f20868a2640 1 --2- 172.21.15.1:0/2482288066 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b980b66b40 0x55b980b65090 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2078009fd0 tx=0x7f20780118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28426aac9fc9c7f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:57.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.687+0000 7f20777fe640 1 --2- 172.21.15.1:0/1616224701 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f206c0a4f90 0x7f206c0a7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:57.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.687+0000 7f2074dea640 1 -- 172.21.15.1:0/2482288066 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2078004d20 con 0x55b980b66b40 2024-09-11T21:57:57.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.687+0000 7f20860a1640 1 --2- 172.21.15.1:0/1616224701 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f206c0a4f90 0x7f206c0a7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:57.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.687+0000 7f2074dea640 1 -- 172.21.15.1:0/2482288066 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2078004050 con 0x55b980b66b40 2024-09-11T21:57:57.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.687+0000 7f20777fe640 1 -- 172.21.15.1:0/1616224701 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7f20700cefc0 con 0x7f2080102570 2024-09-11T21:57:57.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.687+0000 7f2074dea640 1 -- 172.21.15.1:0/2482288066 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2078005240 con 0x55b980b66b40 2024-09-11T21:57:57.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.688+0000 7f208a6de740 1 -- 172.21.15.1:0/2482288066 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b980b66b40 msgr2=0x55b980b65090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:57.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.688+0000 7f20860a1640 1 --2- 172.21.15.1:0/1616224701 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f206c0a4f90 0x7f206c0a7450 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f207c000c00 tx=0x7f207c005b50 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:57.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.688+0000 7f208a6de740 1 --2- 172.21.15.1:0/2482288066 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b980b66b40 0x55b980b65090 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2078009fd0 tx=0x7f20780118d0 comp rx=0 tx=0).stop 2024-09-11T21:57:57.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.688+0000 7f208a6de740 1 -- 172.21.15.1:0/2482288066 shutdown_connections 2024-09-11T21:57:57.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.688+0000 7f208a6de740 1 --2- 172.21.15.1:0/2482288066 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b980b70370 0x55b980b72760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:57.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.688+0000 7f208a6de740 1 --2- 172.21.15.1:0/2482288066 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b980b655d0 0x55b980b65a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:57.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.688+0000 7f208a6de740 1 --2- 172.21.15.1:0/2482288066 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b980b66b40 0x55b980b65090 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:57.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.688+0000 7f208a6de740 1 -- 172.21.15.1:0/2482288066 >> 172.21.15.1:0/2482288066 conn(0x55b980b2e310 msgr2=0x55b980b61c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:57.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.688+0000 7f208a6de740 1 -- 172.21.15.1:0/2482288066 shutdown_connections 2024-09-11T21:57:57.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.688+0000 7f208a6de740 1 -- 172.21.15.1:0/2482288066 wait complete. 2024-09-11T21:57:57.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.688+0000 7f208a6de740 1 -- start start 2024-09-11T21:57:57.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.689+0000 7f208a6de740 1 Processor -- start 2024-09-11T21:57:57.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.689+0000 7f208a6de740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b980b655d0 0x55b980b6fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:57.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.689+0000 7f208a6de740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b980b66b40 0x55b980b6d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:57.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.689+0000 7f208a6de740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b980b70370 0x55b980b6d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:57.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.689+0000 7f208a6de740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55b980b73070 con 0x55b980b66b40 2024-09-11T21:57:57.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.689+0000 7f208a6de740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55b980a86060 con 0x55b980b70370 2024-09-11T21:57:57.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.689+0000 7f208a6de740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b980b731f0 con 0x55b980b655d0 2024-09-11T21:57:57.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.689+0000 7f208a6de740 10 client.0 _ll_register_callbacks cb 0x55b9807d98c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:57:57.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.689+0000 7f20858a0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b980b66b40 0x55b980b6d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:57.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.689+0000 7f20860a1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b980b655d0 0x55b980b6fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:57.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.689+0000 7f20858a0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b980b66b40 0x55b980b6d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60178/0 (socket says 172.21.15.1:60178) 2024-09-11T21:57:57.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.689+0000 7f20858a0640 1 -- 172.21.15.1:0/2549949905 learned_addr learned my addr 172.21.15.1:0/2549949905 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:57.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.689+0000 7f20860a1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b980b655d0 0x55b980b6fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:37864/0 (socket says 172.21.15.1:37864) 2024-09-11T21:57:57.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.689+0000 7f20868a2640 1 --2- 172.21.15.1:0/2549949905 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b980b70370 0x55b980b6d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:57.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.689+0000 7f20858a0640 1 -- 172.21.15.1:0/2549949905 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b980b655d0 msgr2=0x55b980b6fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:57.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.689+0000 7f20858a0640 1 --2- 172.21.15.1:0/2549949905 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b980b655d0 0x55b980b6fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:57.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.689+0000 7f20858a0640 1 -- 172.21.15.1:0/2549949905 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b980b70370 msgr2=0x55b980b6d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:57.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.689+0000 7f20858a0640 1 --2- 172.21.15.1:0/2549949905 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b980b70370 0x55b980b6d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:57.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.689+0000 7f20858a0640 1 -- 172.21.15.1:0/2549949905 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b98083c3c0 con 0x55b980b66b40 2024-09-11T21:57:57.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.690+0000 7f20858a0640 1 --2- 172.21.15.1:0/2549949905 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b980b66b40 0x55b980b6d000 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f20700cc6e0 tx=0x7f207000df70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e6084b7ebd24f10 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:57.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.690+0000 7f2069ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T21:57:57.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.690+0000 7f2069ffb640 1 -- 172.21.15.1:0/2549949905 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20700cce60 con 0x55b980b66b40 2024-09-11T21:57:57.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.690+0000 7f2069ffb640 1 -- 172.21.15.1:0/2549949905 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7f2070097a90 con 0x55b980b66b40 2024-09-11T21:57:57.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.690+0000 7f208a6de740 1 -- 172.21.15.1:0/2549949905 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2078004050 con 0x55b980b66b40 2024-09-11T21:57:57.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.690+0000 7f2069ffb640 1 -- 172.21.15.1:0/2549949905 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2070020050 con 0x55b980b66b40 2024-09-11T21:57:57.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.690+0000 7f208a6de740 10 client.15600 fetch_fsmap learned FSMap version 28 2024-09-11T21:57:57.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.690+0000 7f208a6de740 1 -- 172.21.15.1:0/2549949905 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b980b7eaf0 con 0x55b980b66b40 2024-09-11T21:57:57.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.690+0000 7f2069ffb640 1 -- 172.21.15.1:0/2549949905 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2078004050 con 0x55b980b66b40 2024-09-11T21:57:57.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.691+0000 7f2069ffb640 1 -- 172.21.15.1:0/2549949905 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f20700b41a0 con 0x55b980b66b40 2024-09-11T21:57:57.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.691+0000 7f208a6de740 10 client.15600 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T21:57:57.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.691+0000 7f208a6de740 10 client.15600 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-11T21:57:57.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.691+0000 7f208a6de740 20 client.15600 populate_metadata read hostname 'smithi001' 2024-09-11T21:57:57.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.691+0000 7f208a6de740 1 --2- 172.21.15.1:0/2549949905 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x55b980b7ff00 0x55b980b822f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:57.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.691+0000 7f208a6de740 4 client.15600 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T21:57:57.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.691+0000 7f208a6de740 1 -- 172.21.15.1:0/2549949905 --> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b980ad8860 con 0x55b980b7ff00 2024-09-11T21:57:57.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.691+0000 7f20860a1640 1 --2- 172.21.15.1:0/2549949905 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x55b980b7ff00 0x55b980b822f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:57.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.692+0000 7f20860a1640 1 --2- 172.21.15.1:0/2549949905 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x55b980b7ff00 0x55b980b822f0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=733be133738e31d5 server_cookie=7e7d529db18285aa in_seq=0 out_seq=0 2024-09-11T21:57:57.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.692+0000 7f2069ffb640 10 client.15600 ms_handle_connect on v2:172.21.15.161:6834/828381592 2024-09-11T21:57:57.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.692+0000 7f2069ffb640 1 -- 172.21.15.1:0/2549949905 <== mds.4 v2:172.21.15.161:6834/828381592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b980ad8860 con 0x55b980b7ff00 2024-09-11T21:57:57.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.692+0000 7f2069ffb640 10 client.15600 handle_command_reply: tid=0 2024-09-11T21:57:57.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.692+0000 7f208a6de740 1 client.15600 shutdown 2024-09-11T21:57:57.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.692+0000 7f208a6de740 1 -- 172.21.15.1:0/2549949905 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b980b66b40 msgr2=0x55b980b6d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:57.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.692+0000 7f208a6de740 1 --2- 172.21.15.1:0/2549949905 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b980b66b40 0x55b980b6d000 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f20700cc6e0 tx=0x7f207000df70 comp rx=0 tx=0).stop 2024-09-11T21:57:57.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.692+0000 7f208a6de740 1 -- 172.21.15.1:0/2549949905 shutdown_connections 2024-09-11T21:57:57.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.693+0000 7f208a6de740 1 --2- 172.21.15.1:0/2549949905 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x55b980b7ff00 0x55b980b822f0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:57.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.693+0000 7f208a6de740 1 --2- 172.21.15.1:0/2549949905 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b980b70370 0x55b980b6d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:57.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.693+0000 7f208a6de740 1 --2- 172.21.15.1:0/2549949905 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b980b66b40 0x55b980b6d000 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:57.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.693+0000 7f208a6de740 1 --2- 172.21.15.1:0/2549949905 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b980b655d0 0x55b980b6fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:57.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.693+0000 7f2069ffb640 0 client.15600 ms_handle_reset on v2:172.21.15.161:6834/828381592 2024-09-11T21:57:57.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.693+0000 7f208a6de740 1 -- 172.21.15.1:0/2549949905 >> 172.21.15.1:0/2549949905 conn(0x55b980b2e310 msgr2=0x55b980b67960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:57.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.693+0000 7f208a6de740 1 -- 172.21.15.1:0/2549949905 shutdown_connections 2024-09-11T21:57:57.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.693+0000 7f208a6de740 1 -- 172.21.15.1:0/2549949905 wait complete. 2024-09-11T21:57:57.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.693+0000 7f208a6de740 20 client.15600 trim_cache size 0 max 16384 2024-09-11T21:57:57.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.718+0000 7f208a6de740 1 Processor -- start 2024-09-11T21:57:57.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.718+0000 7f208a6de740 1 -- start start 2024-09-11T21:57:57.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.718+0000 7f208a6de740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b980b655d0 0x55b980b7c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:57.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.718+0000 7f208a6de740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b980b66b40 0x55b980b6cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:57.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.718+0000 7f208a6de740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b980b70370 0x55b980b87bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:57.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.718+0000 7f208a6de740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f20801ab680 con 0x55b980b66b40 2024-09-11T21:57:57.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.718+0000 7f208a6de740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f20801ad1c0 con 0x55b980b655d0 2024-09-11T21:57:57.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.718+0000 7f208a6de740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f20801abd50 con 0x55b980b70370 2024-09-11T21:57:57.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.718+0000 7f20858a0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b980b66b40 0x55b980b6cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:57.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.718+0000 7f20858a0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b980b66b40 0x55b980b6cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60194/0 (socket says 172.21.15.1:60194) 2024-09-11T21:57:57.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.718+0000 7f20858a0640 1 -- 172.21.15.1:0/2228033034 learned_addr learned my addr 172.21.15.1:0/2228033034 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:57.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.718+0000 7f20868a2640 1 --2- 172.21.15.1:0/2228033034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b980b70370 0x55b980b87bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:57.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.718+0000 7f20860a1640 1 --2- 172.21.15.1:0/2228033034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b980b655d0 0x55b980b7c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:57.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.719+0000 7f20858a0640 1 -- 172.21.15.1:0/2228033034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b980b70370 msgr2=0x55b980b87bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:57.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.719+0000 7f20858a0640 1 --2- 172.21.15.1:0/2228033034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b980b70370 0x55b980b87bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:57.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.719+0000 7f20858a0640 1 -- 172.21.15.1:0/2228033034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b980b655d0 msgr2=0x55b980b7c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:57.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.719+0000 7f20858a0640 1 --2- 172.21.15.1:0/2228033034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b980b655d0 0x55b980b7c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:57.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.719+0000 7f20858a0640 1 -- 172.21.15.1:0/2228033034 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b980b82660 con 0x55b980b66b40 2024-09-11T21:57:57.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.719+0000 7f20858a0640 1 --2- 172.21.15.1:0/2228033034 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b980b66b40 0x55b980b6cfc0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f2070012040 tx=0x7f2070009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a796440a56655ba server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:57.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.719+0000 7f206bfff640 1 -- 172.21.15.1:0/2228033034 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20700cce60 con 0x55b980b66b40 2024-09-11T21:57:57.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.719+0000 7f206bfff640 1 -- 172.21.15.1:0/2228033034 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20700046c0 con 0x55b980b66b40 2024-09-11T21:57:57.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.719+0000 7f206bfff640 1 -- 172.21.15.1:0/2228033034 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2070097a90 con 0x55b980b66b40 2024-09-11T21:57:57.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.720+0000 7f208a6de740 1 -- 172.21.15.1:0/2228033034 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b980b66b40 msgr2=0x55b980b6cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:57.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.720+0000 7f208a6de740 1 --2- 172.21.15.1:0/2228033034 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b980b66b40 0x55b980b6cfc0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f2070012040 tx=0x7f2070009c30 comp rx=0 tx=0).stop 2024-09-11T21:57:57.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.720+0000 7f208a6de740 1 -- 172.21.15.1:0/2228033034 shutdown_connections 2024-09-11T21:57:57.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.720+0000 7f208a6de740 1 --2- 172.21.15.1:0/2228033034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b980b70370 0x55b980b87bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:57.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.720+0000 7f208a6de740 1 --2- 172.21.15.1:0/2228033034 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b980b66b40 0x55b980b6cfc0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:57.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.720+0000 7f208a6de740 1 --2- 172.21.15.1:0/2228033034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b980b655d0 0x55b980b7c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:57.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.720+0000 7f208a6de740 1 -- 172.21.15.1:0/2228033034 >> 172.21.15.1:0/2228033034 conn(0x55b980b2e310 msgr2=0x55b980b6c0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:57.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.720+0000 7f208a6de740 1 -- 172.21.15.1:0/2228033034 shutdown_connections 2024-09-11T21:57:57.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.720+0000 7f208a6de740 1 -- 172.21.15.1:0/2228033034 wait complete. 2024-09-11T21:57:57.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.720+0000 7f208a6de740 1 -- start start 2024-09-11T21:57:57.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.720+0000 7f208a6de740 1 Processor -- start 2024-09-11T21:57:57.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.721+0000 7f208a6de740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b980b655d0 0x55b980b78480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:57.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.721+0000 7f208a6de740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b980b66b40 0x55b980b789c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:57.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.721+0000 7f208a6de740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b980b70370 0x55b980b75450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:57.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.721+0000 7f208a6de740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f20801acaf0 con 0x55b980b70370 2024-09-11T21:57:57.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.721+0000 7f208a6de740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f20801ac420 con 0x55b980b655d0 2024-09-11T21:57:57.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.721+0000 7f208a6de740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f20801a9660 con 0x55b980b66b40 2024-09-11T21:57:57.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.721+0000 7f208a6de740 10 client.0 _ll_register_callbacks cb 0x7f20700cc4e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:57:57.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.721+0000 7f20868a2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b980b70370 0x55b980b75450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:57.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.721+0000 7f20868a2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b980b70370 0x55b980b75450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60204/0 (socket says 172.21.15.1:60204) 2024-09-11T21:57:57.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.721+0000 7f20868a2640 1 -- 172.21.15.1:0/4162409381 learned_addr learned my addr 172.21.15.1:0/4162409381 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:57:57.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.721+0000 7f20858a0640 1 --2- 172.21.15.1:0/4162409381 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b980b66b40 0x55b980b789c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:57.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.721+0000 7f20860a1640 1 --2- 172.21.15.1:0/4162409381 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b980b655d0 0x55b980b78480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:57.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.721+0000 7f20868a2640 1 -- 172.21.15.1:0/4162409381 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b980b66b40 msgr2=0x55b980b789c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:57.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.721+0000 7f20868a2640 1 --2- 172.21.15.1:0/4162409381 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b980b66b40 0x55b980b789c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:57.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.721+0000 7f20868a2640 1 -- 172.21.15.1:0/4162409381 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b980b655d0 msgr2=0x55b980b78480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:57:57.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.721+0000 7f20868a2640 1 --2- 172.21.15.1:0/4162409381 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b980b655d0 0x55b980b78480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:57.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.721+0000 7f20868a2640 1 -- 172.21.15.1:0/4162409381 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b980b75990 con 0x55b980b70370 2024-09-11T21:57:57.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.722+0000 7f20868a2640 1 --2- 172.21.15.1:0/4162409381 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b980b70370 0x55b980b75450 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f2078009f70 tx=0x7f2078004c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a19ac342e8c07b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:57:57.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.722+0000 7f2069ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T21:57:57.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.722+0000 7f2069ffb640 1 -- 172.21.15.1:0/4162409381 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2078009c60 con 0x55b980b70370 2024-09-11T21:57:57.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.722+0000 7f208a6de740 1 -- 172.21.15.1:0/4162409381 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b980ad2d70 con 0x55b980b70370 2024-09-11T21:57:57.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.722+0000 7f2069ffb640 1 -- 172.21.15.1:0/4162409381 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7f2078004050 con 0x55b980b70370 2024-09-11T21:57:57.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.722+0000 7f2069ffb640 1 -- 172.21.15.1:0/4162409381 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2078004250 con 0x55b980b70370 2024-09-11T21:57:57.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.722+0000 7f208a6de740 10 client.15606 fetch_fsmap learned FSMap version 28 2024-09-11T21:57:57.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.722+0000 7f208a6de740 1 -- 172.21.15.1:0/4162409381 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b980beb1a0 con 0x55b980b70370 2024-09-11T21:57:57.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.722+0000 7f2069ffb640 1 -- 172.21.15.1:0/4162409381 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55b980ad2d70 con 0x55b980b70370 2024-09-11T21:57:57.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.722+0000 7f2069ffb640 1 -- 172.21.15.1:0/4162409381 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f207800f4e0 con 0x55b980b70370 2024-09-11T21:57:57.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.722+0000 7f208a6de740 10 client.15606 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T21:57:57.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.722+0000 7f208a6de740 10 client.15606 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-11T21:57:57.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.722+0000 7f208a6de740 20 client.15606 populate_metadata read hostname 'smithi001' 2024-09-11T21:57:57.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.722+0000 7f208a6de740 1 --2- 172.21.15.1:0/4162409381 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x55b980beb680 0x55b980beda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:57:57.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.723+0000 7f208a6de740 4 client.15606 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T21:57:57.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.723+0000 7f208a6de740 1 -- 172.21.15.1:0/4162409381 --> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b980a4f170 con 0x55b980beb680 2024-09-11T21:57:57.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.723+0000 7f20860a1640 1 --2- 172.21.15.1:0/4162409381 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x55b980beb680 0x55b980beda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:57:57.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.723+0000 7f20860a1640 1 --2- 172.21.15.1:0/4162409381 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x55b980beb680 0x55b980beda70 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=961b0becbaee1f06 server_cookie=8c0ddffc15a30e0c in_seq=0 out_seq=0 2024-09-11T21:57:57.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.723+0000 7f2069ffb640 10 client.15606 ms_handle_connect on v2:172.21.15.161:6834/828381592 2024-09-11T21:57:57.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.723+0000 7f2069ffb640 1 -- 172.21.15.1:0/4162409381 <== mds.4 v2:172.21.15.161:6834/828381592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b980a4f170 con 0x55b980beb680 2024-09-11T21:57:57.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.723+0000 7f2069ffb640 10 client.15606 handle_command_reply: tid=0 2024-09-11T21:57:57.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.723+0000 7f208a6de740 1 client.15606 shutdown 2024-09-11T21:57:57.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.724+0000 7f208a6de740 1 -- 172.21.15.1:0/4162409381 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b980b70370 msgr2=0x55b980b75450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:57.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.724+0000 7f208a6de740 1 --2- 172.21.15.1:0/4162409381 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b980b70370 0x55b980b75450 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f2078009f70 tx=0x7f2078004c30 comp rx=0 tx=0).stop 2024-09-11T21:57:57.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.724+0000 7f208a6de740 1 -- 172.21.15.1:0/4162409381 shutdown_connections 2024-09-11T21:57:57.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.724+0000 7f208a6de740 1 --2- 172.21.15.1:0/4162409381 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b980b70370 0x55b980b75450 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:57.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.724+0000 7f208a6de740 1 --2- 172.21.15.1:0/4162409381 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b980b66b40 0x55b980b789c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:57.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.724+0000 7f208a6de740 1 --2- 172.21.15.1:0/4162409381 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x55b980beb680 0x55b980beda70 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:57.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.724+0000 7f208a6de740 1 --2- 172.21.15.1:0/4162409381 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b980b655d0 0x55b980b78480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:57.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.724+0000 7f2069ffb640 0 client.15606 ms_handle_reset on v2:172.21.15.161:6834/828381592 2024-09-11T21:57:57.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.724+0000 7f208a6de740 1 -- 172.21.15.1:0/4162409381 >> 172.21.15.1:0/4162409381 conn(0x55b980b2e310 msgr2=0x55b980b604e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:57.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.724+0000 7f208a6de740 1 -- 172.21.15.1:0/4162409381 shutdown_connections 2024-09-11T21:57:57.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.724+0000 7f208a6de740 1 -- 172.21.15.1:0/4162409381 wait complete. 2024-09-11T21:57:57.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.724+0000 7f208a6de740 20 client.15606 trim_cache size 0 max 16384 2024-09-11T21:57:57.731 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-11T21:57:57.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.724+0000 7f2088305640 1 -- 172.21.15.1:0/1616224701 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f206c0a4f90 msgr2=0x7f206c0a7450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:57.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.724+0000 7f2088305640 1 --2- 172.21.15.1:0/1616224701 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f206c0a4f90 0x7f206c0a7450 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f207c000c00 tx=0x7f207c005b50 comp rx=0 tx=0).stop 2024-09-11T21:57:57.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.724+0000 7f2088305640 1 -- 172.21.15.1:0/1616224701 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2080102570 msgr2=0x7f2080197950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:57:57.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.724+0000 7f2088305640 1 --2- 172.21.15.1:0/1616224701 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2080102570 0x7f2080197950 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f207000d770 tx=0x7f207000dc30 comp rx=0 tx=0).stop 2024-09-11T21:57:57.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.725+0000 7f2088305640 1 -- 172.21.15.1:0/1616224701 shutdown_connections 2024-09-11T21:57:57.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.725+0000 7f2088305640 1 --2- 172.21.15.1:0/1616224701 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f206c0a4f90 0x7f206c0a7450 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:57.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.725+0000 7f2088305640 1 --2- 172.21.15.1:0/1616224701 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2080102ef0 0x7f208019bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:57.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.725+0000 7f2088305640 1 --2- 172.21.15.1:0/1616224701 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2080102570 0x7f2080197950 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:57.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.725+0000 7f2088305640 1 --2- 172.21.15.1:0/1616224701 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2080100e20 0x7f2080197410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:57:57.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.725+0000 7f2088305640 1 -- 172.21.15.1:0/1616224701 >> 172.21.15.1:0/1616224701 conn(0x7f20800fc7d0 msgr2=0x7f20800febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:57:57.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.725+0000 7f2088305640 1 -- 172.21.15.1:0/1616224701 shutdown_connections 2024-09-11T21:57:57.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:57:57.725+0000 7f2088305640 1 -- 172.21.15.1:0/1616224701 wait complete. 2024-09-11T21:57:58.449 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T21:57:58.677 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.5 2024-09-11T21:57:58.678 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T21:57:58.728 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:57:59.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:57:59 smithi053 ceph-mon[34908]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-11T21:57:59.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:57:59 smithi161 ceph-mon[35615]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-11T21:57:59.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:57:59 smithi001 ceph-mon[30935]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-11T21:58:00.768 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:58:00 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:58:00.441+0000 7f5ef37f2640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T21:58:00.769 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:58:00 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:58:00.441+0000 7f5ef37f2640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T21:58:00.769 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:58:00 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:58:00.441+0000 7f5ef37f2640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T21:58:00.769 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:58:00 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:58:00.441+0000 7f5ef37f2640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T21:58:00.769 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 21:58:00 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T21:58:00.441+0000 7f5ef37f2640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T21:58:01.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:58:01 smithi053 ceph-mon[34908]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-11T21:58:01.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:58:01 smithi161 ceph-mon[35615]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-11T21:58:01.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:58:01 smithi001 ceph-mon[30935]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-11T21:58:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.677+0000 7f6942a5c640 1 -- 172.21.15.1:0/1554842240 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f693c101ac0 msgr2=0x7f693c101f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.677+0000 7f6942a5c640 1 --2- 172.21.15.1:0/1554842240 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f693c101ac0 0x7f693c101f20 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f6930009930 tx=0x7f693002a420 comp rx=0 tx=0).stop 2024-09-11T21:58:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.677+0000 7f6942a5c640 1 -- 172.21.15.1:0/1554842240 shutdown_connections 2024-09-11T21:58:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.677+0000 7f6942a5c640 1 --2- 172.21.15.1:0/1554842240 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f693c102460 0x7f693c10e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.677+0000 7f6942a5c640 1 --2- 172.21.15.1:0/1554842240 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f693c101ac0 0x7f693c101f20 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.677+0000 7f6942a5c640 1 --2- 172.21.15.1:0/1554842240 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f693c107ad0 0x7f693c107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.677+0000 7f6942a5c640 1 -- 172.21.15.1:0/1554842240 >> 172.21.15.1:0/1554842240 conn(0x7f693c0fd7a0 msgr2=0x7f693c0ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.677+0000 7f6942a5c640 1 -- 172.21.15.1:0/1554842240 shutdown_connections 2024-09-11T21:58:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.677+0000 7f6942a5c640 1 -- 172.21.15.1:0/1554842240 wait complete. 2024-09-11T21:58:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.677+0000 7f6942a5c640 1 Processor -- start 2024-09-11T21:58:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.678+0000 7f6942a5c640 1 -- start start 2024-09-11T21:58:02.681 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.678+0000 7f6942a5c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f693c101ac0 0x7f693c19a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.678+0000 7f6942a5c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f693c102460 0x7f693c19ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.678+0000 7f6942a5c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f693c107ad0 0x7f693c19ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.678+0000 7f6942a5c640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f693c111f10 con 0x7f693c101ac0 2024-09-11T21:58:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.678+0000 7f6942a5c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f693c111d90 con 0x7f693c107ad0 2024-09-11T21:58:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.678+0000 7f6942a5c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f693c112090 con 0x7f693c102460 2024-09-11T21:58:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.678+0000 7f693bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f693c101ac0 0x7f693c19a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.678+0000 7f693bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f693c101ac0 0x7f693c19a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60276/0 (socket says 172.21.15.1:60276) 2024-09-11T21:58:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.678+0000 7f693bfff640 1 -- 172.21.15.1:0/298871045 learned_addr learned my addr 172.21.15.1:0/298871045 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:58:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.678+0000 7f6940ff9640 1 --2- 172.21.15.1:0/298871045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f693c107ad0 0x7f693c19ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.678+0000 7f693b7fe640 1 --2- 172.21.15.1:0/298871045 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f693c102460 0x7f693c19ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.678+0000 7f693bfff640 1 -- 172.21.15.1:0/298871045 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f693c102460 msgr2=0x7f693c19ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.678+0000 7f693bfff640 1 --2- 172.21.15.1:0/298871045 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f693c102460 0x7f693c19ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.678+0000 7f693bfff640 1 -- 172.21.15.1:0/298871045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f693c107ad0 msgr2=0x7f693c19ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.679+0000 7f693bfff640 1 --2- 172.21.15.1:0/298871045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f693c107ad0 0x7f693c19ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.679+0000 7f693bfff640 1 -- 172.21.15.1:0/298871045 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f693c19f630 con 0x7f693c101ac0 2024-09-11T21:58:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.679+0000 7f693b7fe640 1 --2- 172.21.15.1:0/298871045 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f693c102460 0x7f693c19ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:58:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.679+0000 7f693bfff640 1 --2- 172.21.15.1:0/298871045 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f693c101ac0 0x7f693c19a640 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f692c00d950 tx=0x7f692c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.679+0000 7f69397fa640 1 -- 172.21.15.1:0/298871045 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f692c017070 con 0x7f693c101ac0 2024-09-11T21:58:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.679+0000 7f69397fa640 1 -- 172.21.15.1:0/298871045 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f692c009c30 con 0x7f693c101ac0 2024-09-11T21:58:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.679+0000 7f6942a5c640 1 -- 172.21.15.1:0/298871045 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f693c19f920 con 0x7f693c101ac0 2024-09-11T21:58:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.679+0000 7f69397fa640 1 -- 172.21.15.1:0/298871045 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f692c00f6b0 con 0x7f693c101ac0 2024-09-11T21:58:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.679+0000 7f6942a5c640 1 -- 172.21.15.1:0/298871045 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f693c1a71b0 con 0x7f693c101ac0 2024-09-11T21:58:02.684 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.682+0000 7f69457ed740 1 Processor -- start 2024-09-11T21:58:02.684 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.682+0000 7f69397fa640 1 -- 172.21.15.1:0/298871045 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) ==== 138065+0+0 (secure 0 0 0) 0x7f692c0109d0 con 0x7f693c101ac0 2024-09-11T21:58:02.684 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.682+0000 7f69457ed740 1 -- start start 2024-09-11T21:58:02.686 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.683+0000 7f69457ed740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6739eaaf0 0x55a6739eaed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:02.686 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.683+0000 7f69457ed740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6739eb410 0x55a6739f5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:02.686 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.683+0000 7f69457ed740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6739f5cb0 0x55a6739f80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:02.686 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.683+0000 7f69457ed740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55a67394f2f0 con 0x55a6739f5cb0 2024-09-11T21:58:02.686 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.683+0000 7f69457ed740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55a6738f7270 con 0x55a6739eb410 2024-09-11T21:58:02.686 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.683+0000 7f69457ed740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a67394b2b0 con 0x55a6739eaaf0 2024-09-11T21:58:02.686 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.683+0000 7f693b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6739f5cb0 0x55a6739f80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:02.686 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.683+0000 7f693b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6739f5cb0 0x55a6739f80a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60284/0 (socket says 172.21.15.1:60284) 2024-09-11T21:58:02.686 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.683+0000 7f693b7fe640 1 -- 172.21.15.1:0/2886691055 learned_addr learned my addr 172.21.15.1:0/2886691055 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:58:02.686 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.683+0000 7f693bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6739eb410 0x55a6739f5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:02.686 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.683+0000 7f6940ff9640 1 --2- 172.21.15.1:0/2886691055 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6739eaaf0 0x55a6739eaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.683+0000 7f69397fa640 1 --2- 172.21.15.1:0/298871045 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f69240a2e10 0x7f69240a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.683+0000 7f693b7fe640 1 -- 172.21.15.1:0/2886691055 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6739eaaf0 msgr2=0x55a6739eaed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.683+0000 7f693b7fe640 1 --2- 172.21.15.1:0/2886691055 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6739eaaf0 0x55a6739eaed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.683+0000 7f693b7fe640 1 -- 172.21.15.1:0/2886691055 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6739eb410 msgr2=0x55a6739f5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.683+0000 7f693b7fe640 1 --2- 172.21.15.1:0/2886691055 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6739eb410 0x55a6739f5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.683+0000 7f693b7fe640 1 -- 172.21.15.1:0/2886691055 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6738f6340 con 0x55a6739f5cb0 2024-09-11T21:58:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.683+0000 7f69397fa640 1 -- 172.21.15.1:0/298871045 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7f692c0cf2b0 con 0x7f693c101ac0 2024-09-11T21:58:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.683+0000 7f6940ff9640 1 --2- 172.21.15.1:0/298871045 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f69240a2e10 0x7f69240a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.684+0000 7f693b7fe640 1 --2- 172.21.15.1:0/2886691055 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6739f5cb0 0x55a6739f80a0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f6930009930 tx=0x7f6930032cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=324780581f58d84f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.684+0000 7f69227fc640 1 -- 172.21.15.1:0/2886691055 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f693003f070 con 0x55a6739f5cb0 2024-09-11T21:58:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.684+0000 7f69227fc640 1 -- 172.21.15.1:0/2886691055 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69300317b0 con 0x55a6739f5cb0 2024-09-11T21:58:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.684+0000 7f69227fc640 1 -- 172.21.15.1:0/2886691055 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6930035070 con 0x55a6739f5cb0 2024-09-11T21:58:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.684+0000 7f6940ff9640 1 --2- 172.21.15.1:0/298871045 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f69240a2e10 0x7f69240a52d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6934009fd0 tx=0x7f6934004ab0 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.685+0000 7f69457ed740 1 -- 172.21.15.1:0/2886691055 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6739f5cb0 msgr2=0x55a6739f80a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.685+0000 7f69457ed740 1 --2- 172.21.15.1:0/2886691055 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6739f5cb0 0x55a6739f80a0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f6930009930 tx=0x7f6930032cb0 comp rx=0 tx=0).stop 2024-09-11T21:58:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.685+0000 7f69457ed740 1 -- 172.21.15.1:0/2886691055 shutdown_connections 2024-09-11T21:58:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.685+0000 7f69457ed740 1 --2- 172.21.15.1:0/2886691055 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6739f5cb0 0x55a6739f80a0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.685+0000 7f69457ed740 1 --2- 172.21.15.1:0/2886691055 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6739eb410 0x55a6739f5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.685+0000 7f69457ed740 1 --2- 172.21.15.1:0/2886691055 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6739eaaf0 0x55a6739eaed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.685+0000 7f69457ed740 1 -- 172.21.15.1:0/2886691055 >> 172.21.15.1:0/2886691055 conn(0x55a6739e7060 msgr2=0x55a6739e7440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.685+0000 7f69457ed740 1 -- 172.21.15.1:0/2886691055 shutdown_connections 2024-09-11T21:58:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.685+0000 7f69457ed740 1 -- 172.21.15.1:0/2886691055 wait complete. 2024-09-11T21:58:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.685+0000 7f69457ed740 1 -- start start 2024-09-11T21:58:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.686+0000 7f69457ed740 1 Processor -- start 2024-09-11T21:58:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.686+0000 7f69457ed740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6739eaaf0 0x55a6739f2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.686+0000 7f69457ed740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6739eb410 0x55a6739f30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.686+0000 7f69457ed740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6739f5cb0 0x55a6739f3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.686+0000 7f69457ed740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55a6739f8880 con 0x55a6739eaaf0 2024-09-11T21:58:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.686+0000 7f69457ed740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55a6738d8290 con 0x55a6739f5cb0 2024-09-11T21:58:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.686+0000 7f69457ed740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a6739f8a00 con 0x55a6739eb410 2024-09-11T21:58:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.686+0000 7f69457ed740 10 client.0 _ll_register_callbacks cb 0x55a673909140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:58:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.686+0000 7f693bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6739eaaf0 0x55a6739f2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.686+0000 7f693bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6739eaaf0 0x55a6739f2bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60286/0 (socket says 172.21.15.1:60286) 2024-09-11T21:58:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.686+0000 7f693bfff640 1 -- 172.21.15.1:0/862339521 learned_addr learned my addr 172.21.15.1:0/862339521 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:58:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.686+0000 7f693b7fe640 1 --2- 172.21.15.1:0/862339521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6739eb410 0x55a6739f30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.686+0000 7f6940ff9640 1 --2- 172.21.15.1:0/862339521 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6739f5cb0 0x55a6739f3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:02.694 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.686+0000 7f693bfff640 1 -- 172.21.15.1:0/862339521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6739eb410 msgr2=0x55a6739f30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:02.694 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.686+0000 7f693bfff640 1 --2- 172.21.15.1:0/862339521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6739eb410 0x55a6739f30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:02.694 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.686+0000 7f693bfff640 1 -- 172.21.15.1:0/862339521 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6739f5cb0 msgr2=0x55a6739f3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:02.694 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.686+0000 7f693bfff640 1 --2- 172.21.15.1:0/862339521 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6739f5cb0 0x55a6739f3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:02.694 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.686+0000 7f693bfff640 1 -- 172.21.15.1:0/862339521 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a673662210 con 0x55a6739eaaf0 2024-09-11T21:58:02.694 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.686+0000 7f6940ff9640 1 --2- 172.21.15.1:0/862339521 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6739f5cb0 0x55a6739f3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:58:02.694 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.686+0000 7f693b7fe640 1 --2- 172.21.15.1:0/862339521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6739eb410 0x55a6739f30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:58:02.694 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.687+0000 7f693bfff640 1 --2- 172.21.15.1:0/862339521 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6739eaaf0 0x55a6739f2bb0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f692c09b510 tx=0x7f692c097220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbcc892df354ff1f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:02.694 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.687+0000 7f691b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T21:58:02.694 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.687+0000 7f691b7fe640 1 -- 172.21.15.1:0/862339521 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f692c020070 con 0x55a6739eaaf0 2024-09-11T21:58:02.694 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.687+0000 7f691b7fe640 1 -- 172.21.15.1:0/862339521 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7f692c0ccb60 con 0x55a6739eaaf0 2024-09-11T21:58:02.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.687+0000 7f69457ed740 1 -- 172.21.15.1:0/862339521 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69300317b0 con 0x55a6739eaaf0 2024-09-11T21:58:02.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.687+0000 7f691b7fe640 1 -- 172.21.15.1:0/862339521 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f692c09b0c0 con 0x55a6739eaaf0 2024-09-11T21:58:02.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.687+0000 7f69457ed740 10 client.15642 fetch_fsmap learned FSMap version 28 2024-09-11T21:58:02.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.687+0000 7f69457ed740 1 -- 172.21.15.1:0/862339521 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a67372e620 con 0x55a6739eaaf0 2024-09-11T21:58:02.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.687+0000 7f691b7fe640 1 -- 172.21.15.1:0/862339521 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69300317b0 con 0x55a6739eaaf0 2024-09-11T21:58:02.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.687+0000 7f691b7fe640 1 -- 172.21.15.1:0/862339521 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f692c09a9d0 con 0x55a6739eaaf0 2024-09-11T21:58:02.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.687+0000 7f69457ed740 10 client.15642 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T21:58:02.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.687+0000 7f69457ed740 10 client.15642 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T21:58:02.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.687+0000 7f69457ed740 20 client.15642 populate_metadata read hostname 'smithi001' 2024-09-11T21:58:02.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.687+0000 7f69457ed740 1 --2- 172.21.15.1:0/862339521 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55a673a05c50 0x55a673a080a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:02.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.687+0000 7f69457ed740 4 client.15642 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T21:58:02.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.687+0000 7f69457ed740 1 -- 172.21.15.1:0/862339521 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a67395ca20 con 0x55a673a05c50 2024-09-11T21:58:02.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.688+0000 7f693b7fe640 1 --2- 172.21.15.1:0/862339521 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55a673a05c50 0x55a673a080a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:02.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.688+0000 7f691b7fe640 10 client.15642 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T21:58:02.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.688+0000 7f693b7fe640 1 --2- 172.21.15.1:0/862339521 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55a673a05c50 0x55a673a080a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee745dcb79a6060e server_cookie=5465f8c113c607f5 in_seq=0 out_seq=0 2024-09-11T21:58:02.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.689+0000 7f691b7fe640 1 -- 172.21.15.1:0/862339521 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a67395ca20 con 0x55a673a05c50 2024-09-11T21:58:02.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.689+0000 7f691b7fe640 10 client.15642 handle_command_reply: tid=0 2024-09-11T21:58:02.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.689+0000 7f69457ed740 1 client.15642 shutdown 2024-09-11T21:58:02.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.689+0000 7f69457ed740 1 -- 172.21.15.1:0/862339521 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6739eaaf0 msgr2=0x55a6739f2bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:02.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.689+0000 7f69457ed740 1 --2- 172.21.15.1:0/862339521 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6739eaaf0 0x55a6739f2bb0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f692c09b510 tx=0x7f692c097220 comp rx=0 tx=0).stop 2024-09-11T21:58:02.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.689+0000 7f69457ed740 1 -- 172.21.15.1:0/862339521 shutdown_connections 2024-09-11T21:58:02.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.689+0000 7f69457ed740 1 --2- 172.21.15.1:0/862339521 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55a673a05c50 0x55a673a080a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:02.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.689+0000 7f69457ed740 1 --2- 172.21.15.1:0/862339521 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6739f5cb0 0x55a6739f3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:02.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.689+0000 7f69457ed740 1 --2- 172.21.15.1:0/862339521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6739eb410 0x55a6739f30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:02.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.689+0000 7f691b7fe640 0 client.15642 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T21:58:02.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.689+0000 7f69457ed740 1 --2- 172.21.15.1:0/862339521 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6739eaaf0 0x55a6739f2bb0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:02.697 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.689+0000 7f69457ed740 1 -- 172.21.15.1:0/862339521 >> 172.21.15.1:0/862339521 conn(0x55a6739e7060 msgr2=0x55a6739ecd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:02.697 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.689+0000 7f69457ed740 1 -- 172.21.15.1:0/862339521 shutdown_connections 2024-09-11T21:58:02.697 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.689+0000 7f69457ed740 1 -- 172.21.15.1:0/862339521 wait complete. 2024-09-11T21:58:02.697 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.690+0000 7f69457ed740 20 client.15642 trim_cache size 0 max 16384 2024-09-11T21:58:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.705+0000 7f69457ed740 1 Processor -- start 2024-09-11T21:58:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.705+0000 7f69457ed740 1 -- start start 2024-09-11T21:58:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.705+0000 7f69457ed740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6739eaaf0 0x55a6739f3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.705+0000 7f69457ed740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6739eb410 0x55a6739f4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.705+0000 7f69457ed740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6739f5cb0 0x55a673a0d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.705+0000 7f69457ed740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f693c1afa80 con 0x55a6739eb410 2024-09-11T21:58:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.705+0000 7f69457ed740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f693c1b15c0 con 0x55a6739f5cb0 2024-09-11T21:58:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.705+0000 7f69457ed740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f693c1b0150 con 0x55a6739eaaf0 2024-09-11T21:58:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.706+0000 7f693b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6739eb410 0x55a6739f4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.706+0000 7f693b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6739eb410 0x55a6739f4b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60300/0 (socket says 172.21.15.1:60300) 2024-09-11T21:58:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.706+0000 7f693b7fe640 1 -- 172.21.15.1:0/2256280333 learned_addr learned my addr 172.21.15.1:0/2256280333 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:58:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.706+0000 7f6940ff9640 1 --2- 172.21.15.1:0/2256280333 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6739f5cb0 0x55a673a0d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.706+0000 7f693bfff640 1 --2- 172.21.15.1:0/2256280333 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6739eaaf0 0x55a6739f3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.706+0000 7f693b7fe640 1 -- 172.21.15.1:0/2256280333 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6739eaaf0 msgr2=0x55a6739f3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.706+0000 7f693b7fe640 1 --2- 172.21.15.1:0/2256280333 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6739eaaf0 0x55a6739f3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.706+0000 7f693b7fe640 1 -- 172.21.15.1:0/2256280333 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6739f5cb0 msgr2=0x55a673a0d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.706+0000 7f693b7fe640 1 --2- 172.21.15.1:0/2256280333 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6739f5cb0 0x55a673a0d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.706+0000 7f693b7fe640 1 -- 172.21.15.1:0/2256280333 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6739e6b00 con 0x55a6739eb410 2024-09-11T21:58:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.706+0000 7f6940ff9640 1 --2- 172.21.15.1:0/2256280333 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6739f5cb0 0x55a673a0d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:58:02.710 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.706+0000 7f693b7fe640 1 --2- 172.21.15.1:0/2256280333 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6739eb410 0x55a6739f4b20 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f69300052d0 tx=0x7f6930000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32974c338bbb1894 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:02.710 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.706+0000 7f6921ffb640 1 -- 172.21.15.1:0/2256280333 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6930035040 con 0x55a6739eb410 2024-09-11T21:58:02.710 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.706+0000 7f6921ffb640 1 -- 172.21.15.1:0/2256280333 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6930004030 con 0x55a6739eb410 2024-09-11T21:58:02.710 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.706+0000 7f6921ffb640 1 -- 172.21.15.1:0/2256280333 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f693003f040 con 0x55a6739eb410 2024-09-11T21:58:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.707+0000 7f69457ed740 1 -- 172.21.15.1:0/2256280333 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6739eb410 msgr2=0x55a6739f4b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.707+0000 7f69457ed740 1 --2- 172.21.15.1:0/2256280333 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6739eb410 0x55a6739f4b20 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f69300052d0 tx=0x7f6930000bc0 comp rx=0 tx=0).stop 2024-09-11T21:58:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.707+0000 7f69457ed740 1 -- 172.21.15.1:0/2256280333 shutdown_connections 2024-09-11T21:58:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.707+0000 7f69457ed740 1 --2- 172.21.15.1:0/2256280333 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6739f5cb0 0x55a673a0d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.707+0000 7f69457ed740 1 --2- 172.21.15.1:0/2256280333 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6739eb410 0x55a6739f4b20 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.707+0000 7f69457ed740 1 --2- 172.21.15.1:0/2256280333 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6739eaaf0 0x55a6739f3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.707+0000 7f69457ed740 1 -- 172.21.15.1:0/2256280333 >> 172.21.15.1:0/2256280333 conn(0x55a6739e7060 msgr2=0x55a6739f4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.707+0000 7f69457ed740 1 -- 172.21.15.1:0/2256280333 shutdown_connections 2024-09-11T21:58:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.707+0000 7f69457ed740 1 -- 172.21.15.1:0/2256280333 wait complete. 2024-09-11T21:58:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.707+0000 7f69457ed740 1 -- start start 2024-09-11T21:58:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.708+0000 7f69457ed740 1 Processor -- start 2024-09-11T21:58:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.708+0000 7f69457ed740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6739eaaf0 0x55a6739fdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.708+0000 7f69457ed740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6739eb410 0x55a6739fe3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.708+0000 7f69457ed740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6739f5cb0 0x55a6739ef3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.708+0000 7f69457ed740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f693c1b0ef0 con 0x55a6739eaaf0 2024-09-11T21:58:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.708+0000 7f69457ed740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f693c1b0820 con 0x55a6739eb410 2024-09-11T21:58:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.708+0000 7f69457ed740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f693c1ada60 con 0x55a6739f5cb0 2024-09-11T21:58:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.708+0000 7f69457ed740 10 client.0 _ll_register_callbacks cb 0x7f692c0d2f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:58:02.714 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.708+0000 7f693bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6739eaaf0 0x55a6739fdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:02.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.708+0000 7f693bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6739eaaf0 0x55a6739fdeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60306/0 (socket says 172.21.15.1:60306) 2024-09-11T21:58:02.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.708+0000 7f693bfff640 1 -- 172.21.15.1:0/1432636414 learned_addr learned my addr 172.21.15.1:0/1432636414 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:58:02.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.708+0000 7f6940ff9640 1 --2- 172.21.15.1:0/1432636414 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6739f5cb0 0x55a6739ef3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:02.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.708+0000 7f693b7fe640 1 --2- 172.21.15.1:0/1432636414 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6739eb410 0x55a6739fe3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:02.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.709+0000 7f693bfff640 1 -- 172.21.15.1:0/1432636414 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6739f5cb0 msgr2=0x55a6739ef3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:02.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.709+0000 7f693bfff640 1 --2- 172.21.15.1:0/1432636414 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6739f5cb0 0x55a6739ef3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:02.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.709+0000 7f693bfff640 1 -- 172.21.15.1:0/1432636414 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6739eb410 msgr2=0x55a6739fe3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:02.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.709+0000 7f693bfff640 1 --2- 172.21.15.1:0/1432636414 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6739eb410 0x55a6739fe3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:02.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.709+0000 7f693bfff640 1 -- 172.21.15.1:0/1432636414 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6739efbe0 con 0x55a6739eaaf0 2024-09-11T21:58:02.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.709+0000 7f693bfff640 1 --2- 172.21.15.1:0/1432636414 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6739eaaf0 0x55a6739fdeb0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f692c097f50 tx=0x7f692c005a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cea8fa5a35039c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:02.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.709+0000 7f691b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T21:58:02.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.709+0000 7f691b7fe640 1 -- 172.21.15.1:0/1432636414 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f692c014030 con 0x55a6739eaaf0 2024-09-11T21:58:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.709+0000 7f691b7fe640 1 -- 172.21.15.1:0/1432636414 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7f692c0d2a50 con 0x55a6739eaaf0 2024-09-11T21:58:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.709+0000 7f69457ed740 1 -- 172.21.15.1:0/1432636414 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6930004030 con 0x55a6739eaaf0 2024-09-11T21:58:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.709+0000 7f691b7fe640 1 -- 172.21.15.1:0/1432636414 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f692c0b9050 con 0x55a6739eaaf0 2024-09-11T21:58:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.709+0000 7f69457ed740 10 client.15648 fetch_fsmap learned FSMap version 28 2024-09-11T21:58:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.709+0000 7f69457ed740 1 -- 172.21.15.1:0/1432636414 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a673a703f0 con 0x55a6739eaaf0 2024-09-11T21:58:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.709+0000 7f691b7fe640 1 -- 172.21.15.1:0/1432636414 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6930004030 con 0x55a6739eaaf0 2024-09-11T21:58:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.709+0000 7f691b7fe640 1 -- 172.21.15.1:0/1432636414 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f692c084ac0 con 0x55a6739eaaf0 2024-09-11T21:58:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.710+0000 7f69457ed740 10 client.15648 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T21:58:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.710+0000 7f69457ed740 10 client.15648 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T21:58:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.710+0000 7f69457ed740 20 client.15648 populate_metadata read hostname 'smithi001' 2024-09-11T21:58:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.710+0000 7f69457ed740 1 --2- 172.21.15.1:0/1432636414 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55a673a70920 0x55a673a72d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:02.716 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.710+0000 7f69457ed740 4 client.15648 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T21:58:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.710+0000 7f69457ed740 1 -- 172.21.15.1:0/1432636414 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a673956d70 con 0x55a673a70920 2024-09-11T21:58:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.710+0000 7f693b7fe640 1 --2- 172.21.15.1:0/1432636414 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55a673a70920 0x55a673a72d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.710+0000 7f693b7fe640 1 --2- 172.21.15.1:0/1432636414 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55a673a70920 0x55a673a72d10 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d93470411e9d66b9 server_cookie=56e5ac33a583c2c4 in_seq=0 out_seq=0 2024-09-11T21:58:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:02.710+0000 7f691b7fe640 10 client.15648 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T21:58:02.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.057+0000 7feb42dd6640 1 -- 172.21.15.1:0/954383406 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feb3c102570 msgr2=0x7feb3c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:02.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.057+0000 7feb42dd6640 1 --2- 172.21.15.1:0/954383406 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feb3c102570 0x7feb3c1029b0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7feb28009900 tx=0x7feb2802a480 comp rx=0 tx=0).stop 2024-09-11T21:58:02.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.057+0000 7feb42dd6640 1 -- 172.21.15.1:0/954383406 shutdown_connections 2024-09-11T21:58:02.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.057+0000 7feb42dd6640 1 --2- 172.21.15.1:0/954383406 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb3c102ef0 0x7feb3c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:02.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.057+0000 7feb42dd6640 1 --2- 172.21.15.1:0/954383406 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feb3c102570 0x7feb3c1029b0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:02.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.057+0000 7feb42dd6640 1 --2- 172.21.15.1:0/954383406 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb3c100e20 0x7feb3c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:02.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.057+0000 7feb42dd6640 1 -- 172.21.15.1:0/954383406 >> 172.21.15.1:0/954383406 conn(0x7feb3c0fc7d0 msgr2=0x7feb3c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:02.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.057+0000 7feb42dd6640 1 -- 172.21.15.1:0/954383406 shutdown_connections 2024-09-11T21:58:02.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.057+0000 7feb42dd6640 1 -- 172.21.15.1:0/954383406 wait complete. 2024-09-11T21:58:02.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.057+0000 7feb42dd6640 1 Processor -- start 2024-09-11T21:58:02.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.057+0000 7feb42dd6640 1 -- start start 2024-09-11T21:58:02.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.058+0000 7feb42dd6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb3c100e20 0x7feb3c1984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:02.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.058+0000 7feb42dd6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb3c102570 0x7feb3c198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:02.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.058+0000 7feb42dd6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feb3c102ef0 0x7feb3c19cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:02.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.058+0000 7feb42dd6640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7feb3c110f80 con 0x7feb3c102ef0 2024-09-11T21:58:02.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.058+0000 7feb42dd6640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7feb3c110e00 con 0x7feb3c100e20 2024-09-11T21:58:02.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.058+0000 7feb42dd6640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7feb3c111100 con 0x7feb3c102570 2024-09-11T21:58:02.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.058+0000 7feb41373640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feb3c102ef0 0x7feb3c19cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:02.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.058+0000 7feb41373640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feb3c102ef0 0x7feb3c19cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60238/0 (socket says 172.21.15.1:60238) 2024-09-11T21:58:02.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.058+0000 7feb41373640 1 -- 172.21.15.1:0/3123710978 learned_addr learned my addr 172.21.15.1:0/3123710978 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:58:02.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.058+0000 7feb40b72640 1 --2- 172.21.15.1:0/3123710978 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb3c100e20 0x7feb3c1984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:02.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.058+0000 7feb3bfff640 1 --2- 172.21.15.1:0/3123710978 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb3c102570 0x7feb3c198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:02.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.058+0000 7feb41373640 1 -- 172.21.15.1:0/3123710978 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb3c102570 msgr2=0x7feb3c198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:02.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.058+0000 7feb41373640 1 --2- 172.21.15.1:0/3123710978 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb3c102570 0x7feb3c198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:02.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.058+0000 7feb41373640 1 -- 172.21.15.1:0/3123710978 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb3c100e20 msgr2=0x7feb3c1984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:02.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.058+0000 7feb41373640 1 --2- 172.21.15.1:0/3123710978 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb3c100e20 0x7feb3c1984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:02.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.058+0000 7feb41373640 1 -- 172.21.15.1:0/3123710978 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb3c19d480 con 0x7feb3c102ef0 2024-09-11T21:58:02.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.058+0000 7feb3bfff640 1 --2- 172.21.15.1:0/3123710978 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb3c102570 0x7feb3c198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T21:58:02.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.059+0000 7feb41373640 1 --2- 172.21.15.1:0/3123710978 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feb3c102ef0 0x7feb3c19cda0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7feb3400eaa0 tx=0x7feb3400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:02.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.059+0000 7feb39ffb640 1 -- 172.21.15.1:0/3123710978 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb3400cdb0 con 0x7feb3c102ef0 2024-09-11T21:58:02.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.059+0000 7feb39ffb640 1 -- 172.21.15.1:0/3123710978 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb34005020 con 0x7feb3c102ef0 2024-09-11T21:58:02.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.059+0000 7feb39ffb640 1 -- 172.21.15.1:0/3123710978 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb340116b0 con 0x7feb3c102ef0 2024-09-11T21:58:02.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.059+0000 7feb42dd6640 1 -- 172.21.15.1:0/3123710978 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb3c19d770 con 0x7feb3c102ef0 2024-09-11T21:58:02.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.059+0000 7feb42dd6640 1 -- 172.21.15.1:0/3123710978 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb3c1a5050 con 0x7feb3c102ef0 2024-09-11T21:58:02.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.062+0000 7feb451af740 1 Processor -- start 2024-09-11T21:58:02.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.062+0000 7feb39ffb640 1 -- 172.21.15.1:0/3123710978 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) ==== 138065+0+0 (secure 0 0 0) 0x7feb34005560 con 0x7feb3c102ef0 2024-09-11T21:58:02.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.062+0000 7feb451af740 1 -- start start 2024-09-11T21:58:02.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.063+0000 7feb451af740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a42b9c7b40 0x55a42b9c6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:02.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.063+0000 7feb451af740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a42b9c65d0 0x55a42b9c6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:02.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.063+0000 7feb451af740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a42b9d5780 0x55a42b9d7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:02.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.063+0000 7feb451af740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55a42b92c2f0 con 0x55a42b9c65d0 2024-09-11T21:58:02.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.063+0000 7feb451af740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55a42b8d3cf0 con 0x55a42b9d5780 2024-09-11T21:58:02.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.063+0000 7feb451af740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a42b9282b0 con 0x55a42b9c7b40 2024-09-11T21:58:02.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.063+0000 7feb40b72640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a42b9c65d0 0x55a42b9c6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:02.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.063+0000 7feb3bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a42b9d5780 0x55a42b9d7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:02.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.063+0000 7feb40b72640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a42b9c65d0 0x55a42b9c6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60240/0 (socket says 172.21.15.1:60240) 2024-09-11T21:58:02.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.063+0000 7feb40b72640 1 -- 172.21.15.1:0/3940682376 learned_addr learned my addr 172.21.15.1:0/3940682376 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:58:02.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.063+0000 7feb3bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a42b9d5780 0x55a42b9d7b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:39900/0 (socket says 172.21.15.1:39900) 2024-09-11T21:58:02.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.063+0000 7feb3bfff640 1 --2- 172.21.15.1:0/3940682376 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a42b9c7b40 0x55a42b9c6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:02.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.063+0000 7feb39ffb640 1 --2- 172.21.15.1:0/3123710978 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7feb2c0a2e90 0x7feb2c0a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:02.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.063+0000 7feb40b72640 1 -- 172.21.15.1:0/3940682376 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a42b9c7b40 msgr2=0x55a42b9c6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:02.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.063+0000 7feb40b72640 1 --2- 172.21.15.1:0/3940682376 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a42b9c7b40 0x55a42b9c6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:02.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.063+0000 7feb40b72640 1 -- 172.21.15.1:0/3940682376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a42b9d5780 msgr2=0x55a42b9d7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:02.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.063+0000 7feb40b72640 1 --2- 172.21.15.1:0/3940682376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a42b9d5780 0x55a42b9d7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:02.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.063+0000 7feb40b72640 1 -- 172.21.15.1:0/3940682376 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a42b8b83a0 con 0x55a42b9c65d0 2024-09-11T21:58:02.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.063+0000 7feb39ffb640 1 -- 172.21.15.1:0/3123710978 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7feb340cf840 con 0x7feb3c102ef0 2024-09-11T21:58:02.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.063+0000 7feb41373640 1 --2- 172.21.15.1:0/3123710978 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7feb2c0a2e90 0x7feb2c0a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:02.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.064+0000 7feb40b72640 1 --2- 172.21.15.1:0/3940682376 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a42b9c65d0 0x55a42b9c6a10 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7feb3000bd50 tx=0x7feb300107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6408d2ff6ebcc22 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:02.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.064+0000 7feb26ffd640 1 -- 172.21.15.1:0/3940682376 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb30005020 con 0x55a42b9c65d0 2024-09-11T21:58:02.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.064+0000 7feb26ffd640 1 -- 172.21.15.1:0/3940682376 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb300047a0 con 0x55a42b9c65d0 2024-09-11T21:58:02.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.064+0000 7feb26ffd640 1 -- 172.21.15.1:0/3940682376 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb30011800 con 0x55a42b9c65d0 2024-09-11T21:58:02.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.065+0000 7feb451af740 1 -- 172.21.15.1:0/3940682376 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a42b9c65d0 msgr2=0x55a42b9c6a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:02.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.065+0000 7feb41373640 1 --2- 172.21.15.1:0/3123710978 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7feb2c0a2e90 0x7feb2c0a5350 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7feb340cce70 tx=0x7feb340cfb40 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:02.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.065+0000 7feb451af740 1 --2- 172.21.15.1:0/3940682376 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a42b9c65d0 0x55a42b9c6a10 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7feb3000bd50 tx=0x7feb300107a0 comp rx=0 tx=0).stop 2024-09-11T21:58:02.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.065+0000 7feb451af740 1 -- 172.21.15.1:0/3940682376 shutdown_connections 2024-09-11T21:58:02.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.065+0000 7feb451af740 1 --2- 172.21.15.1:0/3940682376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a42b9d5780 0x55a42b9d7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:02.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.065+0000 7feb451af740 1 --2- 172.21.15.1:0/3940682376 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a42b9c65d0 0x55a42b9c6a10 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:02.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.065+0000 7feb451af740 1 --2- 172.21.15.1:0/3940682376 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a42b9c7b40 0x55a42b9c6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:02.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.065+0000 7feb451af740 1 -- 172.21.15.1:0/3940682376 >> 172.21.15.1:0/3940682376 conn(0x55a42b98f310 msgr2=0x55a42b9c2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:02.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.065+0000 7feb451af740 1 -- 172.21.15.1:0/3940682376 shutdown_connections 2024-09-11T21:58:02.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.065+0000 7feb451af740 1 -- 172.21.15.1:0/3940682376 wait complete. 2024-09-11T21:58:02.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.065+0000 7feb451af740 1 -- start start 2024-09-11T21:58:02.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.066+0000 7feb451af740 1 Processor -- start 2024-09-11T21:58:02.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.066+0000 7feb451af740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a42b9c65d0 0x55a42b9d5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:02.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.066+0000 7feb451af740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a42b9c7b40 0x55a42b9d2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:02.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.066+0000 7feb451af740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a42b9d5780 0x55a42b9d2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:02.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.066+0000 7feb451af740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55a42b9d8480 con 0x55a42b9c65d0 2024-09-11T21:58:02.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.066+0000 7feb451af740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55a42b8e7060 con 0x55a42b9c7b40 2024-09-11T21:58:02.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.066+0000 7feb451af740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a42b9d8600 con 0x55a42b9d5780 2024-09-11T21:58:02.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.066+0000 7feb451af740 10 client.0 _ll_register_callbacks cb 0x55a42b63a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:58:02.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.066+0000 7feb3bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a42b9c65d0 0x55a42b9d5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:02.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.066+0000 7feb3bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a42b9c65d0 0x55a42b9d5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60252/0 (socket says 172.21.15.1:60252) 2024-09-11T21:58:02.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.066+0000 7feb3bfff640 1 -- 172.21.15.1:0/4270989643 learned_addr learned my addr 172.21.15.1:0/4270989643 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:58:02.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.066+0000 7feb40b72640 1 --2- 172.21.15.1:0/4270989643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a42b9c7b40 0x55a42b9d2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:02.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.067+0000 7feb3bfff640 1 -- 172.21.15.1:0/4270989643 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a42b9d5780 msgr2=0x55a42b9d2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:02.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.067+0000 7feb3bfff640 1 --2- 172.21.15.1:0/4270989643 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a42b9d5780 0x55a42b9d2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:02.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.067+0000 7feb3bfff640 1 -- 172.21.15.1:0/4270989643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a42b9c7b40 msgr2=0x55a42b9d2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:02.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.067+0000 7feb3bfff640 1 --2- 172.21.15.1:0/4270989643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a42b9c7b40 0x55a42b9d2440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:02.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.067+0000 7feb3bfff640 1 -- 172.21.15.1:0/4270989643 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a42b69d3c0 con 0x55a42b9c65d0 2024-09-11T21:58:02.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.067+0000 7feb40b72640 1 --2- 172.21.15.1:0/4270989643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a42b9c7b40 0x55a42b9d2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T21:58:02.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.067+0000 7feb3bfff640 1 --2- 172.21.15.1:0/4270989643 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a42b9c65d0 0x55a42b9d5190 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7feb28009900 tx=0x7feb28031440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5428e2a8bfcf5cc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:02.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.067+0000 7feb1bfff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T21:58:02.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.067+0000 7feb1bfff640 1 -- 172.21.15.1:0/4270989643 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb28035040 con 0x55a42b9c65d0 2024-09-11T21:58:02.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.067+0000 7feb1bfff640 1 -- 172.21.15.1:0/4270989643 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7feb28004ad0 con 0x55a42b9c65d0 2024-09-11T21:58:02.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.067+0000 7feb451af740 1 -- 172.21.15.1:0/4270989643 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb300047a0 con 0x55a42b9c65d0 2024-09-11T21:58:02.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.067+0000 7feb1bfff640 1 -- 172.21.15.1:0/4270989643 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7feb28004cd0 con 0x55a42b9c65d0 2024-09-11T21:58:02.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.067+0000 7feb451af740 10 client.15624 fetch_fsmap learned FSMap version 28 2024-09-11T21:58:02.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.067+0000 7feb451af740 1 -- 172.21.15.1:0/4270989643 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a42b9e3fa0 con 0x55a42b9c65d0 2024-09-11T21:58:02.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.067+0000 7feb1bfff640 1 -- 172.21.15.1:0/4270989643 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb300047a0 con 0x55a42b9c65d0 2024-09-11T21:58:02.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.068+0000 7feb1bfff640 1 -- 172.21.15.1:0/4270989643 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7feb28046070 con 0x55a42b9c65d0 2024-09-11T21:58:02.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.068+0000 7feb451af740 10 client.15624 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T21:58:02.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.068+0000 7feb451af740 10 client.15624 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-11T21:58:02.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.068+0000 7feb451af740 20 client.15624 populate_metadata read hostname 'smithi001' 2024-09-11T21:58:02.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.068+0000 7feb451af740 1 --2- 172.21.15.1:0/4270989643 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x55a42b9e53b0 0x55a42b9e7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:02.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.068+0000 7feb451af740 4 client.15624 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T21:58:02.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.068+0000 7feb451af740 1 -- 172.21.15.1:0/4270989643 --> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a42b939860 con 0x55a42b9e53b0 2024-09-11T21:58:02.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.068+0000 7feb40b72640 1 --2- 172.21.15.1:0/4270989643 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x55a42b9e53b0 0x55a42b9e7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:02.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.069+0000 7feb1bfff640 10 client.15624 ms_handle_connect on v2:172.21.15.161:6836/1799318656 2024-09-11T21:58:02.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.069+0000 7feb40b72640 1 --2- 172.21.15.1:0/4270989643 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x55a42b9e53b0 0x55a42b9e7820 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2e5e2970966d3ee1 server_cookie=5a54d32684931fe2 in_seq=0 out_seq=0 2024-09-11T21:58:02.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.069+0000 7feb1bfff640 1 -- 172.21.15.1:0/4270989643 <== mds.3 v2:172.21.15.161:6836/1799318656 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a42b939860 con 0x55a42b9e53b0 2024-09-11T21:58:02.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.069+0000 7feb1bfff640 10 client.15624 handle_command_reply: tid=0 2024-09-11T21:58:02.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.070+0000 7feb451af740 1 client.15624 shutdown 2024-09-11T21:58:02.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.070+0000 7feb451af740 1 -- 172.21.15.1:0/4270989643 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a42b9c65d0 msgr2=0x55a42b9d5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:02.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.070+0000 7feb451af740 1 --2- 172.21.15.1:0/4270989643 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a42b9c65d0 0x55a42b9d5190 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7feb28009900 tx=0x7feb28031440 comp rx=0 tx=0).stop 2024-09-11T21:58:02.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.070+0000 7feb451af740 1 -- 172.21.15.1:0/4270989643 shutdown_connections 2024-09-11T21:58:02.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.070+0000 7feb451af740 1 --2- 172.21.15.1:0/4270989643 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x55a42b9e53b0 0x55a42b9e7820 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:02.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.070+0000 7feb1bfff640 0 client.15624 ms_handle_reset on v2:172.21.15.161:6836/1799318656 2024-09-11T21:58:02.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.070+0000 7feb451af740 1 --2- 172.21.15.1:0/4270989643 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a42b9d5780 0x55a42b9d2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:02.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.070+0000 7feb451af740 1 --2- 172.21.15.1:0/4270989643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a42b9c7b40 0x55a42b9d2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:02.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.070+0000 7feb451af740 1 --2- 172.21.15.1:0/4270989643 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a42b9c65d0 0x55a42b9d5190 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:02.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.070+0000 7feb451af740 1 -- 172.21.15.1:0/4270989643 >> 172.21.15.1:0/4270989643 conn(0x55a42b98f310 msgr2=0x55a42b9c8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:02.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.070+0000 7feb451af740 1 -- 172.21.15.1:0/4270989643 shutdown_connections 2024-09-11T21:58:02.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.070+0000 7feb451af740 1 -- 172.21.15.1:0/4270989643 wait complete. 2024-09-11T21:58:02.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.070+0000 7feb451af740 20 client.15624 trim_cache size 0 max 16384 2024-09-11T21:58:02.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.083+0000 7feb451af740 1 Processor -- start 2024-09-11T21:58:02.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.083+0000 7feb451af740 1 -- start start 2024-09-11T21:58:02.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.083+0000 7feb451af740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a42b9c65d0 0x55a42b9c2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:02.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.083+0000 7feb451af740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a42b9c7b40 0x55a42b9e1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:02.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.083+0000 7feb451af740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a42b9d5780 0x55a42b9ed0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:02.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.083+0000 7feb451af740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7feb3c1ab7e0 con 0x55a42b9c7b40 2024-09-11T21:58:02.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.083+0000 7feb451af740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7feb3c1ad320 con 0x55a42b9c65d0 2024-09-11T21:58:02.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.083+0000 7feb451af740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7feb3c1abeb0 con 0x55a42b9d5780 2024-09-11T21:58:02.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.083+0000 7feb40b72640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a42b9c7b40 0x55a42b9e1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:02.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.083+0000 7feb40b72640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a42b9c7b40 0x55a42b9e1770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60258/0 (socket says 172.21.15.1:60258) 2024-09-11T21:58:02.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.083+0000 7feb40b72640 1 -- 172.21.15.1:0/2723532506 learned_addr learned my addr 172.21.15.1:0/2723532506 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:58:02.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.084+0000 7feb40b72640 1 -- 172.21.15.1:0/2723532506 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a42b9d5780 msgr2=0x55a42b9ed0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:02.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.084+0000 7feb40b72640 1 --2- 172.21.15.1:0/2723532506 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a42b9d5780 0x55a42b9ed0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:02.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.084+0000 7feb40b72640 1 -- 172.21.15.1:0/2723532506 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a42b9c65d0 msgr2=0x55a42b9c2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:02.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.084+0000 7feb40b72640 1 --2- 172.21.15.1:0/2723532506 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a42b9c65d0 0x55a42b9c2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:02.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.084+0000 7feb40b72640 1 -- 172.21.15.1:0/2723532506 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a42b9d8360 con 0x55a42b9c7b40 2024-09-11T21:58:02.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.084+0000 7feb40b72640 1 --2- 172.21.15.1:0/2723532506 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a42b9c7b40 0x55a42b9e1770 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7feb3000bd50 tx=0x7feb3000b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63e3d806f769b5bb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:02.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.084+0000 7feb267fc640 1 -- 172.21.15.1:0/2723532506 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb3002c070 con 0x55a42b9c7b40 2024-09-11T21:58:02.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.084+0000 7feb267fc640 1 -- 172.21.15.1:0/2723532506 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb30005520 con 0x55a42b9c7b40 2024-09-11T21:58:02.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.084+0000 7feb267fc640 1 -- 172.21.15.1:0/2723532506 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb30004cc0 con 0x55a42b9c7b40 2024-09-11T21:58:02.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.084+0000 7feb451af740 1 -- 172.21.15.1:0/2723532506 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a42b9c7b40 msgr2=0x55a42b9e1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:02.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.084+0000 7feb451af740 1 --2- 172.21.15.1:0/2723532506 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a42b9c7b40 0x55a42b9e1770 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7feb3000bd50 tx=0x7feb3000b410 comp rx=0 tx=0).stop 2024-09-11T21:58:02.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.085+0000 7feb451af740 1 -- 172.21.15.1:0/2723532506 shutdown_connections 2024-09-11T21:58:02.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.085+0000 7feb451af740 1 --2- 172.21.15.1:0/2723532506 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a42b9d5780 0x55a42b9ed0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:02.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.085+0000 7feb451af740 1 --2- 172.21.15.1:0/2723532506 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a42b9c7b40 0x55a42b9e1770 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:02.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.085+0000 7feb451af740 1 --2- 172.21.15.1:0/2723532506 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a42b9c65d0 0x55a42b9c2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:02.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.085+0000 7feb451af740 1 -- 172.21.15.1:0/2723532506 >> 172.21.15.1:0/2723532506 conn(0x55a42b98f310 msgr2=0x55a42b9c2700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:02.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.085+0000 7feb451af740 1 -- 172.21.15.1:0/2723532506 shutdown_connections 2024-09-11T21:58:02.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.085+0000 7feb451af740 1 -- 172.21.15.1:0/2723532506 wait complete. 2024-09-11T21:58:02.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.085+0000 7feb451af740 1 -- start start 2024-09-11T21:58:02.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.085+0000 7feb451af740 1 Processor -- start 2024-09-11T21:58:02.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.085+0000 7feb451af740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a42b9c65d0 0x55a42b9dd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:02.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.085+0000 7feb451af740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a42b9c7b40 0x55a42b9dde20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:02.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.085+0000 7feb451af740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a42b9d5780 0x55a42b9ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:02.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.085+0000 7feb451af740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7feb3c1acc50 con 0x55a42b9d5780 2024-09-11T21:58:02.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.085+0000 7feb451af740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7feb3c1ac580 con 0x55a42b9c7b40 2024-09-11T21:58:02.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.085+0000 7feb451af740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7feb3c1a97c0 con 0x55a42b9c65d0 2024-09-11T21:58:02.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.085+0000 7feb451af740 10 client.0 _ll_register_callbacks cb 0x7feb28005420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:58:02.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.086+0000 7feb3bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a42b9d5780 0x55a42b9ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:02.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.086+0000 7feb3bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a42b9d5780 0x55a42b9ceab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60268/0 (socket says 172.21.15.1:60268) 2024-09-11T21:58:02.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.086+0000 7feb3bfff640 1 -- 172.21.15.1:0/2515013310 learned_addr learned my addr 172.21.15.1:0/2515013310 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:58:02.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.086+0000 7feb3bfff640 1 --2- 172.21.15.1:0/2515013310 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a42b9c65d0 0x55a42b9dd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:02.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.086+0000 7feb40b72640 1 --2- 172.21.15.1:0/2515013310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a42b9c7b40 0x55a42b9dde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:02.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.086+0000 7feb3bfff640 1 -- 172.21.15.1:0/2515013310 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a42b9c65d0 msgr2=0x55a42b9dd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:02.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.086+0000 7feb3bfff640 1 --2- 172.21.15.1:0/2515013310 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a42b9c65d0 0x55a42b9dd8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:02.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.086+0000 7feb3bfff640 1 -- 172.21.15.1:0/2515013310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a42b9c7b40 msgr2=0x55a42b9dde20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:02.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.086+0000 7feb3bfff640 1 --2- 172.21.15.1:0/2515013310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a42b9c7b40 0x55a42b9dde20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:02.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.086+0000 7feb3bfff640 1 -- 172.21.15.1:0/2515013310 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a42b9cf070 con 0x55a42b9d5780 2024-09-11T21:58:02.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.086+0000 7feb3bfff640 1 --2- 172.21.15.1:0/2515013310 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a42b9d5780 0x55a42b9ceab0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7feb28003a70 tx=0x7feb28005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=525a091de0b257be server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:02.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.086+0000 7feb1bfff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T21:58:02.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.086+0000 7feb1bfff640 1 -- 172.21.15.1:0/2515013310 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb28035070 con 0x55a42b9d5780 2024-09-11T21:58:02.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.086+0000 7feb1bfff640 1 -- 172.21.15.1:0/2515013310 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7feb28004ad0 con 0x55a42b9d5780 2024-09-11T21:58:02.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.086+0000 7feb451af740 1 -- 172.21.15.1:0/2515013310 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb30005520 con 0x55a42b9d5780 2024-09-11T21:58:02.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.086+0000 7feb1bfff640 1 -- 172.21.15.1:0/2515013310 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7feb28031540 con 0x55a42b9d5780 2024-09-11T21:58:02.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.086+0000 7feb451af740 10 client.15630 fetch_fsmap learned FSMap version 28 2024-09-11T21:58:02.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.086+0000 7feb451af740 1 -- 172.21.15.1:0/2515013310 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a42ba4f960 con 0x55a42b9d5780 2024-09-11T21:58:02.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.087+0000 7feb1bfff640 1 -- 172.21.15.1:0/2515013310 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb30005520 con 0x55a42b9d5780 2024-09-11T21:58:02.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.087+0000 7feb1bfff640 1 -- 172.21.15.1:0/2515013310 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7feb28046040 con 0x55a42b9d5780 2024-09-11T21:58:02.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.087+0000 7feb451af740 10 client.15630 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T21:58:02.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.087+0000 7feb451af740 10 client.15630 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-11T21:58:02.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.087+0000 7feb451af740 20 client.15630 populate_metadata read hostname 'smithi001' 2024-09-11T21:58:02.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.087+0000 7feb451af740 1 --2- 172.21.15.1:0/2515013310 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x55a42b9e53b0 0x55a42ba51fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:02.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.087+0000 7feb451af740 4 client.15630 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T21:58:02.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.087+0000 7feb451af740 1 -- 172.21.15.1:0/2515013310 --> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a42b933d70 con 0x55a42b9e53b0 2024-09-11T21:58:02.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.087+0000 7feb40b72640 1 --2- 172.21.15.1:0/2515013310 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x55a42b9e53b0 0x55a42ba51fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:02.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.088+0000 7feb40b72640 1 --2- 172.21.15.1:0/2515013310 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x55a42b9e53b0 0x55a42ba51fc0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8b8d3ab31bdeeb6e server_cookie=75faf0f103d6b9ed in_seq=0 out_seq=0 2024-09-11T21:58:02.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.088+0000 7feb1bfff640 10 client.15630 ms_handle_connect on v2:172.21.15.161:6836/1799318656 2024-09-11T21:58:02.881 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-11T21:58:02.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.088+0000 7feb1bfff640 1 -- 172.21.15.1:0/2515013310 <== mds.3 v2:172.21.15.161:6836/1799318656 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a42b933d70 con 0x55a42b9e53b0 2024-09-11T21:58:02.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.088+0000 7feb1bfff640 10 client.15630 handle_command_reply: tid=0 2024-09-11T21:58:02.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.088+0000 7feb451af740 1 client.15630 shutdown 2024-09-11T21:58:02.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.088+0000 7feb451af740 1 -- 172.21.15.1:0/2515013310 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a42b9d5780 msgr2=0x55a42b9ceab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:02.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.088+0000 7feb451af740 1 --2- 172.21.15.1:0/2515013310 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a42b9d5780 0x55a42b9ceab0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7feb28003a70 tx=0x7feb28005fd0 comp rx=0 tx=0).stop 2024-09-11T21:58:02.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.088+0000 7feb451af740 1 -- 172.21.15.1:0/2515013310 shutdown_connections 2024-09-11T21:58:02.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.088+0000 7feb451af740 1 --2- 172.21.15.1:0/2515013310 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x55a42b9e53b0 0x55a42ba51fc0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:02.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.088+0000 7feb451af740 1 --2- 172.21.15.1:0/2515013310 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a42b9d5780 0x55a42b9ceab0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:02.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.088+0000 7feb451af740 1 --2- 172.21.15.1:0/2515013310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a42b9c7b40 0x55a42b9dde20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:02.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.088+0000 7feb1bfff640 0 client.15630 ms_handle_reset on v2:172.21.15.161:6836/1799318656 2024-09-11T21:58:02.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.088+0000 7feb451af740 1 --2- 172.21.15.1:0/2515013310 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a42b9c65d0 0x55a42b9dd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:02.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.088+0000 7feb451af740 1 -- 172.21.15.1:0/2515013310 >> 172.21.15.1:0/2515013310 conn(0x55a42b98f310 msgr2=0x55a42b9c8110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:02.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.088+0000 7feb451af740 1 -- 172.21.15.1:0/2515013310 shutdown_connections 2024-09-11T21:58:02.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.088+0000 7feb451af740 1 -- 172.21.15.1:0/2515013310 wait complete. 2024-09-11T21:58:02.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.088+0000 7feb451af740 20 client.15630 trim_cache size 0 max 16384 2024-09-11T21:58:02.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.089+0000 7feb42dd6640 1 -- 172.21.15.1:0/3123710978 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7feb2c0a2e90 msgr2=0x7feb2c0a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:02.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.089+0000 7feb42dd6640 1 --2- 172.21.15.1:0/3123710978 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7feb2c0a2e90 0x7feb2c0a5350 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7feb340cce70 tx=0x7feb340cfb40 comp rx=0 tx=0).stop 2024-09-11T21:58:02.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.089+0000 7feb42dd6640 1 -- 172.21.15.1:0/3123710978 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feb3c102ef0 msgr2=0x7feb3c19cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:02.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.089+0000 7feb42dd6640 1 --2- 172.21.15.1:0/3123710978 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feb3c102ef0 0x7feb3c19cda0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7feb3400eaa0 tx=0x7feb3400ef60 comp rx=0 tx=0).stop 2024-09-11T21:58:02.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.089+0000 7feb42dd6640 1 -- 172.21.15.1:0/3123710978 shutdown_connections 2024-09-11T21:58:02.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.089+0000 7feb42dd6640 1 --2- 172.21.15.1:0/3123710978 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7feb2c0a2e90 0x7feb2c0a5350 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:02.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.089+0000 7feb42dd6640 1 --2- 172.21.15.1:0/3123710978 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feb3c102ef0 0x7feb3c19cda0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:02.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.089+0000 7feb42dd6640 1 --2- 172.21.15.1:0/3123710978 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feb3c102570 0x7feb3c198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:02.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.089+0000 7feb42dd6640 1 --2- 172.21.15.1:0/3123710978 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb3c100e20 0x7feb3c1984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:02.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.089+0000 7feb42dd6640 1 -- 172.21.15.1:0/3123710978 >> 172.21.15.1:0/3123710978 conn(0x7feb3c0fc7d0 msgr2=0x7feb3c06a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:02.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.089+0000 7feb42dd6640 1 -- 172.21.15.1:0/3123710978 shutdown_connections 2024-09-11T21:58:02.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:02.089+0000 7feb42dd6640 1 -- 172.21.15.1:0/3123710978 wait complete. 2024-09-11T21:58:03.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:58:03 smithi001 ceph-mon[30935]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 426 B/s wr, 12 op/s 2024-09-11T21:58:03.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:58:03 smithi001 ceph-mon[30935]: mgrmap e25: x(active, since 48s), standbys: y, z 2024-09-11T21:58:03.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:58:03 smithi053 ceph-mon[34908]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 426 B/s wr, 12 op/s 2024-09-11T21:58:03.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:58:03 smithi053 ceph-mon[34908]: mgrmap e25: x(active, since 48s), standbys: y, z 2024-09-11T21:58:03.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:58:03 smithi161 ceph-mon[35615]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 426 B/s wr, 12 op/s 2024-09-11T21:58:03.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:58:03 smithi161 ceph-mon[35615]: mgrmap e25: x(active, since 48s), standbys: y, z 2024-09-11T21:58:03.710 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T21:58:03.719 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:58:04.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.259+0000 7fe94df19640 1 -- 172.21.15.1:0/3428708088 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe948100860 msgr2=0x7fe9481045a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:04.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.259+0000 7fe94df19640 1 --2- 172.21.15.1:0/3428708088 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe948100860 0x7fe9481045a0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fe940009ca0 tx=0x7fe94002a8c0 comp rx=0 tx=0).stop 2024-09-11T21:58:04.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.259+0000 7fe94df19640 1 -- 172.21.15.1:0/3428708088 shutdown_connections 2024-09-11T21:58:04.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.259+0000 7fe94df19640 1 --2- 172.21.15.1:0/3428708088 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe948100860 0x7fe9481045a0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:04.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.259+0000 7fe94df19640 1 --2- 172.21.15.1:0/3428708088 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe948107440 0x7fe948100100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:04.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.259+0000 7fe94df19640 1 --2- 172.21.15.1:0/3428708088 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe948106a90 0x7fe948106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:04.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.259+0000 7fe94df19640 1 -- 172.21.15.1:0/3428708088 >> 172.21.15.1:0/3428708088 conn(0x7fe948077230 msgr2=0x7fe9480fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:04.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.260+0000 7fe94df19640 1 -- 172.21.15.1:0/3428708088 shutdown_connections 2024-09-11T21:58:04.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.260+0000 7fe94df19640 1 -- 172.21.15.1:0/3428708088 wait complete. 2024-09-11T21:58:04.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.260+0000 7fe94df19640 1 Processor -- start 2024-09-11T21:58:04.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.261+0000 7fe94df19640 1 -- start start 2024-09-11T21:58:04.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.261+0000 7fe94df19640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe948100860 0x7fe9481a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:04.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.261+0000 7fe94df19640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe948106a90 0x7fe9481a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:04.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.261+0000 7fe94df19640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe94819e890 0x7fe94819ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:04.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.261+0000 7fe94df19640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe948118600 con 0x7fe948100860 2024-09-11T21:58:04.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.261+0000 7fe94df19640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe948118480 con 0x7fe948106a90 2024-09-11T21:58:04.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.261+0000 7fe94df19640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe948118780 con 0x7fe94819e890 2024-09-11T21:58:04.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.261+0000 7fe9477fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe948100860 0x7fe9481a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:04.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.261+0000 7fe9477fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe948100860 0x7fe9481a46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60326/0 (socket says 172.21.15.1:60326) 2024-09-11T21:58:04.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.262+0000 7fe9477fe640 1 -- 172.21.15.1:0/3032773054 learned_addr learned my addr 172.21.15.1:0/3032773054 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:58:04.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.262+0000 7fe947fff640 1 --2- 172.21.15.1:0/3032773054 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe94819e890 0x7fe94819ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:04.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.262+0000 7fe946ffd640 1 --2- 172.21.15.1:0/3032773054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe948106a90 0x7fe9481a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:04.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.262+0000 7fe9477fe640 1 -- 172.21.15.1:0/3032773054 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe94819e890 msgr2=0x7fe94819ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:04.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.262+0000 7fe9477fe640 1 --2- 172.21.15.1:0/3032773054 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe94819e890 0x7fe94819ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:04.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.262+0000 7fe9477fe640 1 -- 172.21.15.1:0/3032773054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe948106a90 msgr2=0x7fe9481a4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:04.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.262+0000 7fe9477fe640 1 --2- 172.21.15.1:0/3032773054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe948106a90 0x7fe9481a4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:04.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.262+0000 7fe9477fe640 1 -- 172.21.15.1:0/3032773054 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe94819f3b0 con 0x7fe948100860 2024-09-11T21:58:04.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.262+0000 7fe9477fe640 1 --2- 172.21.15.1:0/3032773054 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe948100860 0x7fe9481a46d0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fe93800d950 tx=0x7fe93800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:04.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.262+0000 7fe944ff9640 1 -- 172.21.15.1:0/3032773054 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe938017070 con 0x7fe948100860 2024-09-11T21:58:04.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.263+0000 7fe944ff9640 1 -- 172.21.15.1:0/3032773054 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe938009c30 con 0x7fe948100860 2024-09-11T21:58:04.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.263+0000 7fe94df19640 1 -- 172.21.15.1:0/3032773054 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe94819f6a0 con 0x7fe948100860 2024-09-11T21:58:04.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.263+0000 7fe944ff9640 1 -- 172.21.15.1:0/3032773054 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93800f6b0 con 0x7fe948100860 2024-09-11T21:58:04.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.263+0000 7fe94df19640 1 -- 172.21.15.1:0/3032773054 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9481ab870 con 0x7fe948100860 2024-09-11T21:58:04.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.264+0000 7fe94df19640 1 -- 172.21.15.1:0/3032773054 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe948106e70 con 0x7fe948100860 2024-09-11T21:58:04.267 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:58:04.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.267+0000 7fe944ff9640 1 -- 172.21.15.1:0/3032773054 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) ==== 138065+0+0 (secure 0 0 0) 0x7fe9380109d0 con 0x7fe948100860 2024-09-11T21:58:04.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.268+0000 7fe944ff9640 1 --2- 172.21.15.1:0/3032773054 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fe9300a2ec0 0x7fe9300a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:04.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.268+0000 7fe944ff9640 1 -- 172.21.15.1:0/3032773054 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7fe9380cefc0 con 0x7fe948100860 2024-09-11T21:58:04.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.268+0000 7fe946ffd640 1 --2- 172.21.15.1:0/3032773054 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fe9300a2ec0 0x7fe9300a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:04.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.268+0000 7fe946ffd640 1 --2- 172.21.15.1:0/3032773054 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fe9300a2ec0 0x7fe9300a5380 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe948103f90 tx=0x7fe93c008040 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:04.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.276+0000 7fe944ff9640 1 -- 172.21.15.1:0/3032773054 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe93809b990 con 0x7fe948100860 2024-09-11T21:58:04.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.582+0000 7fe94df19640 1 -- 172.21.15.1:0/3032773054 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe94819fdc0 con 0x7fe948100860 2024-09-11T21:58:04.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.583+0000 7fe944ff9640 1 -- 172.21.15.1:0/3032773054 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11949 (secure 0 0 0) 0x7fe93809b290 con 0x7fe948100860 2024-09-11T21:58:04.586 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:58:04.586 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":28,"btime":"2024-09-11T21:57:12:512925+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/421542832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":421542832},{"type":"v1","addr":"172.21.15.1:6835","nonce":421542832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2836864406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2836864406},{"type":"v1","addr":"172.21.15.1:6837","nonce":2836864406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T21:55:20.791263+0000","modified":"2024-09-11T21:57:12.462736+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24484,"mds_2":24467,"mds_3":24466,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14595":{"gid":14595,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/407834672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":407834672},{"type":"v1","addr":"172.21.15.1:6839","nonce":407834672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/1068827678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":1068827678},{"type":"v1","addr":"172.21.15.1:6841","nonce":1068827678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/872410024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":872410024},{"type":"v1","addr":"172.21.15.161:6833","nonce":872410024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/1648944570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1648944570},{"type":"v1","addr":"172.21.15.53:6833","nonce":1648944570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/2126933993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2126933993},{"type":"v1","addr":"172.21.15.53:6835","nonce":2126933993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.161:6835/828381592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":828381592},{"type":"v1","addr":"172.21.15.161:6835","nonce":828381592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.161:6837/1799318656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1799318656},{"type":"v1","addr":"172.21.15.161:6837","nonce":1799318656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.53:6837/3457131830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3457131830},{"type":"v1","addr":"172.21.15.53:6837","nonce":3457131830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/455365383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":455365383},{"type":"v1","addr":"172.21.15.161:6839","nonce":455365383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.53:6839/1378584447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1378584447},{"type":"v1","addr":"172.21.15.53:6839","nonce":1378584447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24460,24466,24467,24484,24494]},"id":1}]} 2024-09-11T21:58:04.587 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 28 2024-09-11T21:58:04.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.586+0000 7fe94df19640 1 -- 172.21.15.1:0/3032773054 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fe9300a2ec0 msgr2=0x7fe9300a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:04.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.586+0000 7fe94df19640 1 --2- 172.21.15.1:0/3032773054 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fe9300a2ec0 0x7fe9300a5380 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe948103f90 tx=0x7fe93c008040 comp rx=0 tx=0).stop 2024-09-11T21:58:04.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.587+0000 7fe94df19640 1 -- 172.21.15.1:0/3032773054 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe948100860 msgr2=0x7fe9481a46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:04.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.587+0000 7fe94df19640 1 --2- 172.21.15.1:0/3032773054 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe948100860 0x7fe9481a46d0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fe93800d950 tx=0x7fe93800de10 comp rx=0 tx=0).stop 2024-09-11T21:58:04.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.587+0000 7fe94df19640 1 -- 172.21.15.1:0/3032773054 shutdown_connections 2024-09-11T21:58:04.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.587+0000 7fe94df19640 1 --2- 172.21.15.1:0/3032773054 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fe9300a2ec0 0x7fe9300a5380 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:04.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.587+0000 7fe94df19640 1 --2- 172.21.15.1:0/3032773054 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe94819e890 0x7fe94819ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:04.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.587+0000 7fe94df19640 1 --2- 172.21.15.1:0/3032773054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe948106a90 0x7fe9481a4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:04.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.587+0000 7fe94df19640 1 --2- 172.21.15.1:0/3032773054 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe948100860 0x7fe9481a46d0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:04.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.587+0000 7fe94df19640 1 -- 172.21.15.1:0/3032773054 >> 172.21.15.1:0/3032773054 conn(0x7fe948077230 msgr2=0x7fe948102660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:04.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.587+0000 7fe94df19640 1 -- 172.21.15.1:0/3032773054 shutdown_connections 2024-09-11T21:58:04.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.587+0000 7fe94df19640 1 -- 172.21.15.1:0/3032773054 wait complete. 2024-09-11T21:58:04.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.793+0000 7f8bca944640 1 -- 172.21.15.1:0/1638015797 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bc4102ef0 msgr2=0x7f8bc410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:04.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.793+0000 7f8bca944640 1 --2- 172.21.15.1:0/1638015797 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bc4102ef0 0x7f8bc410da40 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc009ca0 tx=0x7f8bbc02a950 comp rx=0 tx=0).stop 2024-09-11T21:58:04.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.793+0000 7f8bca944640 1 -- 172.21.15.1:0/1638015797 shutdown_connections 2024-09-11T21:58:04.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.793+0000 7f8bca944640 1 --2- 172.21.15.1:0/1638015797 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bc4102ef0 0x7f8bc410da40 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:04.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.793+0000 7f8bca944640 1 --2- 172.21.15.1:0/1638015797 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8bc4102570 0x7f8bc41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:04.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.793+0000 7f8bca944640 1 --2- 172.21.15.1:0/1638015797 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8bc4100e20 0x7f8bc4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:04.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.793+0000 7f8bca944640 1 -- 172.21.15.1:0/1638015797 >> 172.21.15.1:0/1638015797 conn(0x7f8bc40fc7d0 msgr2=0x7f8bc40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:04.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.794+0000 7f8bca944640 1 -- 172.21.15.1:0/1638015797 shutdown_connections 2024-09-11T21:58:04.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.794+0000 7f8bca944640 1 -- 172.21.15.1:0/1638015797 wait complete. 2024-09-11T21:58:04.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.794+0000 7f8bca944640 1 Processor -- start 2024-09-11T21:58:04.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.794+0000 7f8bca944640 1 -- start start 2024-09-11T21:58:04.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.795+0000 7f8bca944640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8bc4100e20 0x7f8bc419a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:04.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.795+0000 7f8bca944640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bc4102570 0x7f8bc419abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:04.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.795+0000 7f8bca944640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8bc4102ef0 0x7f8bc419ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:04.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.795+0000 7f8bc37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bc4102570 0x7f8bc419abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:04.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.795+0000 7f8bc37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bc4102570 0x7f8bc419abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60346/0 (socket says 172.21.15.1:60346) 2024-09-11T21:58:04.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.795+0000 7f8bc37fe640 1 -- 172.21.15.1:0/2739785250 learned_addr learned my addr 172.21.15.1:0/2739785250 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:58:04.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.795+0000 7f8bca944640 1 -- 172.21.15.1:0/2739785250 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc41120a0 con 0x7f8bc4102570 2024-09-11T21:58:04.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.795+0000 7f8bca944640 1 -- 172.21.15.1:0/2739785250 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc4111f20 con 0x7f8bc4100e20 2024-09-11T21:58:04.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.795+0000 7f8bca944640 1 -- 172.21.15.1:0/2739785250 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc4112220 con 0x7f8bc4102ef0 2024-09-11T21:58:04.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.796+0000 7f8bc3fff640 1 --2- 172.21.15.1:0/2739785250 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8bc4100e20 0x7f8bc419a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:04.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.796+0000 7f8bc8ee1640 1 --2- 172.21.15.1:0/2739785250 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8bc4102ef0 0x7f8bc419ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:04.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.796+0000 7f8bc3fff640 1 -- 172.21.15.1:0/2739785250 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8bc4102ef0 msgr2=0x7f8bc419ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:04.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.796+0000 7f8bc3fff640 1 --2- 172.21.15.1:0/2739785250 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8bc4102ef0 0x7f8bc419ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:04.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.796+0000 7f8bc3fff640 1 -- 172.21.15.1:0/2739785250 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bc4102570 msgr2=0x7f8bc419abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:04.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.796+0000 7f8bc3fff640 1 --2- 172.21.15.1:0/2739785250 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bc4102570 0x7f8bc419abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:04.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.796+0000 7f8bc3fff640 1 -- 172.21.15.1:0/2739785250 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bc419f620 con 0x7f8bc4100e20 2024-09-11T21:58:04.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.796+0000 7f8bc3fff640 1 --2- 172.21.15.1:0/2739785250 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8bc4100e20 0x7f8bc419a670 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8bb4005fd0 tx=0x7f8bb400dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:04.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.797+0000 7f8bc17fa640 1 -- 172.21.15.1:0/2739785250 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb4009c30 con 0x7f8bc4100e20 2024-09-11T21:58:04.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.797+0000 7f8bc17fa640 1 -- 172.21.15.1:0/2739785250 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bb4009dd0 con 0x7f8bc4100e20 2024-09-11T21:58:04.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.797+0000 7f8bc17fa640 1 -- 172.21.15.1:0/2739785250 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb4010460 con 0x7f8bc4100e20 2024-09-11T21:58:04.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.797+0000 7f8bca944640 1 -- 172.21.15.1:0/2739785250 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bc419f910 con 0x7f8bc4100e20 2024-09-11T21:58:04.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.797+0000 7f8bca944640 1 -- 172.21.15.1:0/2739785250 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bc41a71f0 con 0x7f8bc4100e20 2024-09-11T21:58:04.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.801+0000 7f8bc17fa640 1 -- 172.21.15.1:0/2739785250 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 25) ==== 138065+0+0 (secure 0 0 0) 0x7f8bb4003f50 con 0x7f8bc4100e20 2024-09-11T21:58:04.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.801+0000 7f8bccd1d740 1 Processor -- start 2024-09-11T21:58:04.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.801+0000 7f8bccd1d740 1 -- start start 2024-09-11T21:58:04.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.801+0000 7f8bccd1d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646dad2db40 0x5646dad2c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:04.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.801+0000 7f8bccd1d740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5646dad2c5d0 0x5646dad2ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:04.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.801+0000 7f8bccd1d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646dad3fb80 0x5646dad41f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:04.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.801+0000 7f8bccd1d740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5646dac922f0 con 0x5646dad2c5d0 2024-09-11T21:58:04.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.801+0000 7f8bccd1d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5646dac39cf0 con 0x5646dad2db40 2024-09-11T21:58:04.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.801+0000 7f8bccd1d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5646dac8e2b0 con 0x5646dad3fb80 2024-09-11T21:58:04.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.802+0000 7f8bc3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5646dad2c5d0 0x5646dad2ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:04.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.802+0000 7f8bc3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5646dad2c5d0 0x5646dad2ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60348/0 (socket says 172.21.15.1:60348) 2024-09-11T21:58:04.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.802+0000 7f8bc8ee1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646dad2db40 0x5646dad2c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:04.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.802+0000 7f8bc3fff640 1 -- 172.21.15.1:0/3901993226 learned_addr learned my addr 172.21.15.1:0/3901993226 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:58:04.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.802+0000 7f8bc37fe640 1 --2- 172.21.15.1:0/3901993226 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646dad3fb80 0x5646dad41f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:04.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.802+0000 7f8bc17fa640 1 --2- 172.21.15.1:0/2739785250 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f8bac0a2f10 0x7f8bac0a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:04.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.802+0000 7f8bc8ee1640 1 -- 172.21.15.1:0/3901993226 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646dad3fb80 msgr2=0x5646dad41f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:04.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.802+0000 7f8bc8ee1640 1 --2- 172.21.15.1:0/3901993226 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646dad3fb80 0x5646dad41f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:04.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.802+0000 7f8bc8ee1640 1 -- 172.21.15.1:0/3901993226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5646dad2c5d0 msgr2=0x5646dad2ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:04.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.802+0000 7f8bc8ee1640 1 --2- 172.21.15.1:0/3901993226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5646dad2c5d0 0x5646dad2ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:04.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.802+0000 7f8bc17fa640 1 -- 172.21.15.1:0/2739785250 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7f8bb40cf680 con 0x7f8bc4100e20 2024-09-11T21:58:04.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.802+0000 7f8bc8ee1640 1 -- 172.21.15.1:0/3901993226 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646dac1e3a0 con 0x5646dad2db40 2024-09-11T21:58:04.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.803+0000 7f8bc8ee1640 1 --2- 172.21.15.1:0/2739785250 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f8bac0a2f10 0x7f8bac0a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:04.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.803+0000 7f8bc8ee1640 1 --2- 172.21.15.1:0/3901993226 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646dad2db40 0x5646dad2c090 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc005cc0 tx=0x7f8bbc0098f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1159396bac56fd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:04.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.803+0000 7f8baa7fc640 1 -- 172.21.15.1:0/3901993226 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc034040 con 0x5646dad2db40 2024-09-11T21:58:04.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.803+0000 7f8baa7fc640 1 -- 172.21.15.1:0/3901993226 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bbc003c30 con 0x5646dad2db40 2024-09-11T21:58:04.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.803+0000 7f8baa7fc640 1 -- 172.21.15.1:0/3901993226 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc057680 con 0x5646dad2db40 2024-09-11T21:58:04.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.803+0000 7f8bc8ee1640 1 --2- 172.21.15.1:0/2739785250 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f8bac0a2f10 0x7f8bac0a53d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x5646dad2e170 tx=0x7f8bbc05b000 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:04.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.804+0000 7f8bccd1d740 1 -- 172.21.15.1:0/3901993226 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646dad2db40 msgr2=0x5646dad2c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:04.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.804+0000 7f8bccd1d740 1 --2- 172.21.15.1:0/3901993226 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646dad2db40 0x5646dad2c090 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc005cc0 tx=0x7f8bbc0098f0 comp rx=0 tx=0).stop 2024-09-11T21:58:04.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.804+0000 7f8bccd1d740 1 -- 172.21.15.1:0/3901993226 shutdown_connections 2024-09-11T21:58:04.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.804+0000 7f8bccd1d740 1 --2- 172.21.15.1:0/3901993226 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646dad3fb80 0x5646dad41f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:04.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.804+0000 7f8bccd1d740 1 --2- 172.21.15.1:0/3901993226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5646dad2c5d0 0x5646dad2ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:04.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.804+0000 7f8bccd1d740 1 --2- 172.21.15.1:0/3901993226 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646dad2db40 0x5646dad2c090 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:04.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.804+0000 7f8bccd1d740 1 -- 172.21.15.1:0/3901993226 >> 172.21.15.1:0/3901993226 conn(0x5646dacf5310 msgr2=0x5646dad28c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:04.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.804+0000 7f8bccd1d740 1 -- 172.21.15.1:0/3901993226 shutdown_connections 2024-09-11T21:58:04.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.804+0000 7f8bccd1d740 1 -- 172.21.15.1:0/3901993226 wait complete. 2024-09-11T21:58:04.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.805+0000 7f8bccd1d740 1 -- start start 2024-09-11T21:58:04.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.805+0000 7f8bccd1d740 1 Processor -- start 2024-09-11T21:58:04.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.805+0000 7f8bccd1d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646dad2c5d0 0x5646dad39330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:04.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.805+0000 7f8bccd1d740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5646dad2db40 0x5646dad39870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:04.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.806+0000 7f8bccd1d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646dad3fb80 0x5646dad3f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:04.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.806+0000 7f8bccd1d740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5646dad2f550 con 0x5646dad2db40 2024-09-11T21:58:04.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.806+0000 7f8bccd1d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5646dac4d060 con 0x5646dad3fb80 2024-09-11T21:58:04.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.806+0000 7f8bc37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5646dad2db40 0x5646dad39870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:04.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.806+0000 7f8bc37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5646dad2db40 0x5646dad39870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60352/0 (socket says 172.21.15.1:60352) 2024-09-11T21:58:04.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.806+0000 7f8bc3fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646dad2c5d0 0x5646dad39330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:04.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.806+0000 7f8bc37fe640 1 -- 172.21.15.1:0/1481172154 learned_addr learned my addr 172.21.15.1:0/1481172154 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:58:04.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.806+0000 7f8bccd1d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5646dad2f6d0 con 0x5646dad2c5d0 2024-09-11T21:58:04.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.806+0000 7f8bccd1d740 10 client.0 _ll_register_callbacks cb 0x5646da9a08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:58:04.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.806+0000 7f8bc8ee1640 1 --2- 172.21.15.1:0/1481172154 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646dad3fb80 0x5646dad3f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:04.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.806+0000 7f8bc3fff640 1 -- 172.21.15.1:0/1481172154 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646dad3fb80 msgr2=0x5646dad3f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:04.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.806+0000 7f8bc3fff640 1 --2- 172.21.15.1:0/1481172154 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646dad3fb80 0x5646dad3f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:04.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.806+0000 7f8bc3fff640 1 -- 172.21.15.1:0/1481172154 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5646dad2db40 msgr2=0x5646dad39870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:04.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.806+0000 7f8bc3fff640 1 --2- 172.21.15.1:0/1481172154 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5646dad2db40 0x5646dad39870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:04.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.806+0000 7f8bc3fff640 1 -- 172.21.15.1:0/1481172154 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646daa033c0 con 0x5646dad2c5d0 2024-09-11T21:58:04.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.806+0000 7f8bc3fff640 1 --2- 172.21.15.1:0/1481172154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646dad2c5d0 0x5646dad39330 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x5646dad29ac0 tx=0x7f8bb40d2c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc5eb8282ba167be server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:04.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.806+0000 7f8ba37fe640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-11T21:58:04.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.806+0000 7f8ba37fe640 1 -- 172.21.15.1:0/1481172154 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb4020070 con 0x5646dad2c5d0 2024-09-11T21:58:04.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.806+0000 7f8bccd1d740 1 -- 172.21.15.1:0/1481172154 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bbc003c30 con 0x5646dad2c5d0 2024-09-11T21:58:04.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.806+0000 7f8ba37fe640 1 -- 172.21.15.1:0/1481172154 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7f8bb409bc00 con 0x5646dad2c5d0 2024-09-11T21:58:04.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.807+0000 7f8ba37fe640 1 -- 172.21.15.1:0/1481172154 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8bb409b300 con 0x5646dad2c5d0 2024-09-11T21:58:04.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.807+0000 7f8bccd1d740 10 client.25246 fetch_fsmap learned FSMap version 28 2024-09-11T21:58:04.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.807+0000 7f8bccd1d740 1 -- 172.21.15.1:0/1481172154 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5646dad45ca0 con 0x5646dad2c5d0 2024-09-11T21:58:04.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.807+0000 7f8ba37fe640 1 -- 172.21.15.1:0/1481172154 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bbc003c30 con 0x5646dad2c5d0 2024-09-11T21:58:04.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.807+0000 7f8ba37fe640 1 -- 172.21.15.1:0/1481172154 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8bb40a40d0 con 0x5646dad2c5d0 2024-09-11T21:58:04.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.807+0000 7f8bccd1d740 10 client.25246 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T21:58:04.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.807+0000 7f8bccd1d740 10 client.25246 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-11T21:58:04.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.807+0000 7f8bccd1d740 20 client.25246 populate_metadata read hostname 'smithi001' 2024-09-11T21:58:04.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.807+0000 7f8bccd1d740 1 --2- 172.21.15.1:0/1481172154 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x5646dad46ff0 0x5646dad493e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:04.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.807+0000 7f8bccd1d740 4 client.25246 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T21:58:04.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.807+0000 7f8bccd1d740 1 -- 172.21.15.1:0/1481172154 --> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646dac9f860 con 0x5646dad46ff0 2024-09-11T21:58:04.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.808+0000 7f8bc37fe640 1 --2- 172.21.15.1:0/1481172154 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x5646dad46ff0 0x5646dad493e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:04.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.808+0000 7f8bc37fe640 1 --2- 172.21.15.1:0/1481172154 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x5646dad46ff0 0x5646dad493e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ee89a76164750d80 server_cookie=cb49c169814e4602 in_seq=0 out_seq=0 2024-09-11T21:58:04.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.808+0000 7f8ba37fe640 10 client.25246 ms_handle_connect on v2:172.21.15.53:6836/3457131830 2024-09-11T21:58:04.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.809+0000 7f8ba37fe640 1 -- 172.21.15.1:0/1481172154 <== mds.2 v2:172.21.15.53:6836/3457131830 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5646dac9f860 con 0x5646dad46ff0 2024-09-11T21:58:04.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.809+0000 7f8ba37fe640 10 client.25246 handle_command_reply: tid=0 2024-09-11T21:58:04.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.809+0000 7f8bccd1d740 1 client.25246 shutdown 2024-09-11T21:58:04.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.809+0000 7f8bccd1d740 1 -- 172.21.15.1:0/1481172154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646dad2c5d0 msgr2=0x5646dad39330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:04.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.809+0000 7f8bccd1d740 1 --2- 172.21.15.1:0/1481172154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646dad2c5d0 0x5646dad39330 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x5646dad29ac0 tx=0x7f8bb40d2c60 comp rx=0 tx=0).stop 2024-09-11T21:58:04.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.809+0000 7f8bccd1d740 1 -- 172.21.15.1:0/1481172154 shutdown_connections 2024-09-11T21:58:04.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.809+0000 7f8bccd1d740 1 --2- 172.21.15.1:0/1481172154 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x5646dad46ff0 0x5646dad493e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:04.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.809+0000 7f8ba37fe640 0 client.25246 ms_handle_reset on v2:172.21.15.53:6836/3457131830 2024-09-11T21:58:04.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.809+0000 7f8bccd1d740 1 --2- 172.21.15.1:0/1481172154 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646dad3fb80 0x5646dad3f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:04.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.809+0000 7f8bccd1d740 1 --2- 172.21.15.1:0/1481172154 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5646dad2db40 0x5646dad39870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:04.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.809+0000 7f8bccd1d740 1 --2- 172.21.15.1:0/1481172154 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646dad2c5d0 0x5646dad39330 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:04.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.809+0000 7f8bccd1d740 1 -- 172.21.15.1:0/1481172154 >> 172.21.15.1:0/1481172154 conn(0x5646dacf5310 msgr2=0x5646dad2e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:04.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.810+0000 7f8bccd1d740 1 -- 172.21.15.1:0/1481172154 shutdown_connections 2024-09-11T21:58:04.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.810+0000 7f8bccd1d740 1 -- 172.21.15.1:0/1481172154 wait complete. 2024-09-11T21:58:04.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.810+0000 7f8bccd1d740 20 client.25246 trim_cache size 0 max 16384 2024-09-11T21:58:04.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.836+0000 7f8bccd1d740 1 Processor -- start 2024-09-11T21:58:04.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.836+0000 7f8bccd1d740 1 -- start start 2024-09-11T21:58:04.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.836+0000 7f8bccd1d740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5646dad2c5d0 0x5646dad3f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:04.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.836+0000 7f8bccd1d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646dad2db40 0x5646dad2e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:04.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.836+0000 7f8bccd1d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646dad3fb80 0x5646dad4ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:04.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.836+0000 7f8bccd1d740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc41ab780 con 0x5646dad2c5d0 2024-09-11T21:58:04.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.836+0000 7f8bccd1d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc41ad2c0 con 0x5646dad3fb80 2024-09-11T21:58:04.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.836+0000 7f8bccd1d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc41abe50 con 0x5646dad2db40 2024-09-11T21:58:04.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.836+0000 7f8bc3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5646dad2c5d0 0x5646dad3f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:04.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.836+0000 7f8bc3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5646dad2c5d0 0x5646dad3f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60356/0 (socket says 172.21.15.1:60356) 2024-09-11T21:58:04.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.836+0000 7f8bc3fff640 1 -- 172.21.15.1:0/273424817 learned_addr learned my addr 172.21.15.1:0/273424817 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:58:04.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.836+0000 7f8bc37fe640 1 --2- 172.21.15.1:0/273424817 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646dad2db40 0x5646dad2e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:04.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.836+0000 7f8bc8ee1640 1 --2- 172.21.15.1:0/273424817 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646dad3fb80 0x5646dad4ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:04.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.837+0000 7f8bc3fff640 1 -- 172.21.15.1:0/273424817 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646dad2db40 msgr2=0x5646dad2e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:04.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.837+0000 7f8bc3fff640 1 --2- 172.21.15.1:0/273424817 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646dad2db40 0x5646dad2e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:04.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.837+0000 7f8bc3fff640 1 -- 172.21.15.1:0/273424817 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646dad3fb80 msgr2=0x5646dad4ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:04.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.837+0000 7f8bc3fff640 1 --2- 172.21.15.1:0/273424817 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646dad3fb80 0x5646dad4ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:04.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.837+0000 7f8bc3fff640 1 -- 172.21.15.1:0/273424817 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646dad32c10 con 0x5646dad2c5d0 2024-09-11T21:58:04.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.837+0000 7f8bc3fff640 1 --2- 172.21.15.1:0/273424817 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5646dad2c5d0 0x5646dad3f020 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f8bb40cced0 tx=0x7f8bb4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce2827ea240bab98 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:04.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.837+0000 7f8ba9ffb640 1 -- 172.21.15.1:0/273424817 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb4020070 con 0x5646dad2c5d0 2024-09-11T21:58:04.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.837+0000 7f8ba9ffb640 1 -- 172.21.15.1:0/273424817 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bb409fca0 con 0x5646dad2c5d0 2024-09-11T21:58:04.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.837+0000 7f8ba9ffb640 1 -- 172.21.15.1:0/273424817 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb409bc00 con 0x5646dad2c5d0 2024-09-11T21:58:04.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.838+0000 7f8bccd1d740 1 -- 172.21.15.1:0/273424817 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5646dad2c5d0 msgr2=0x5646dad3f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:04.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.838+0000 7f8bccd1d740 1 --2- 172.21.15.1:0/273424817 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5646dad2c5d0 0x5646dad3f020 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f8bb40cced0 tx=0x7f8bb4000bc0 comp rx=0 tx=0).stop 2024-09-11T21:58:04.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.838+0000 7f8bccd1d740 1 -- 172.21.15.1:0/273424817 shutdown_connections 2024-09-11T21:58:04.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.838+0000 7f8bccd1d740 1 --2- 172.21.15.1:0/273424817 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646dad3fb80 0x5646dad4ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:04.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.838+0000 7f8bccd1d740 1 --2- 172.21.15.1:0/273424817 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646dad2db40 0x5646dad2e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:04.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.838+0000 7f8bccd1d740 1 --2- 172.21.15.1:0/273424817 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5646dad2c5d0 0x5646dad3f020 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:04.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.838+0000 7f8bccd1d740 1 -- 172.21.15.1:0/273424817 >> 172.21.15.1:0/273424817 conn(0x5646dacf5310 msgr2=0x5646dad2e1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:04.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.838+0000 7f8bccd1d740 1 -- 172.21.15.1:0/273424817 shutdown_connections 2024-09-11T21:58:04.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.838+0000 7f8bccd1d740 1 -- 172.21.15.1:0/273424817 wait complete. 2024-09-11T21:58:04.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.839+0000 7f8bccd1d740 1 -- start start 2024-09-11T21:58:04.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.839+0000 7f8bccd1d740 1 Processor -- start 2024-09-11T21:58:04.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.840+0000 7f8bccd1d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646dad2c5d0 0x5646dad56f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:04.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.840+0000 7f8bccd1d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646dad2db40 0x5646dad57440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:04.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.840+0000 7f8bccd1d740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5646dad3fb80 0x5646dadaaa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:04.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.840+0000 7f8bc3fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646dad2c5d0 0x5646dad56f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:04.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.840+0000 7f8bc3fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646dad2c5d0 0x5646dad56f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:38044/0 (socket says 172.21.15.1:38044) 2024-09-11T21:58:04.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.840+0000 7f8bc37fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646dad2db40 0x5646dad57440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:04.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.840+0000 7f8bc37fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646dad2db40 0x5646dad57440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:40054/0 (socket says 172.21.15.1:40054) 2024-09-11T21:58:04.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.840+0000 7f8bc3fff640 1 -- 172.21.15.1:0/2935456588 learned_addr learned my addr 172.21.15.1:0/2935456588 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:58:04.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.840+0000 7f8bccd1d740 1 -- 172.21.15.1:0/2935456588 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc41acbf0 con 0x5646dad3fb80 2024-09-11T21:58:04.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.840+0000 7f8bccd1d740 1 -- 172.21.15.1:0/2935456588 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc41ac520 con 0x5646dad2db40 2024-09-11T21:58:04.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.840+0000 7f8bccd1d740 1 -- 172.21.15.1:0/2935456588 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc41a9760 con 0x5646dad2c5d0 2024-09-11T21:58:04.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.841+0000 7f8bccd1d740 10 client.0 _ll_register_callbacks cb 0x7f8bb400df10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:58:04.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.841+0000 7f8bc3fff640 1 -- 172.21.15.1:0/2935456588 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646dad2db40 msgr2=0x5646dad57440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:04.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.841+0000 7f8bc3fff640 1 --2- 172.21.15.1:0/2935456588 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646dad2db40 0x5646dad57440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:04.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.841+0000 7f8bc3fff640 1 -- 172.21.15.1:0/2935456588 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5646dad3fb80 msgr2=0x5646dadaaa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:04.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.841+0000 7f8bc3fff640 1 --2- 172.21.15.1:0/2935456588 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5646dad3fb80 0x5646dadaaa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:04.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.841+0000 7f8bc3fff640 1 -- 172.21.15.1:0/2935456588 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646dadab0a0 con 0x5646dad2c5d0 2024-09-11T21:58:04.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.841+0000 7f8bccd1d740 1 -- 172.21.15.1:0/2935456588 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bb409fca0 con 0x5646dad2c5d0 2024-09-11T21:58:04.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.841+0000 7f8bc3fff640 1 --2- 172.21.15.1:0/2935456588 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646dad2c5d0 0x5646dad56f00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8bb409e050 tx=0x7f8bb40d5110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83fde4e2938663e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:04.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.841+0000 7f8ba37fe640 10 client.25252 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-11T21:58:04.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.841+0000 7f8ba37fe640 1 -- 172.21.15.1:0/2935456588 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb4020070 con 0x5646dad2c5d0 2024-09-11T21:58:04.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.841+0000 7f8ba37fe640 1 -- 172.21.15.1:0/2935456588 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7f8bb40862e0 con 0x5646dad2c5d0 2024-09-11T21:58:04.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.842+0000 7f8ba37fe640 1 -- 172.21.15.1:0/2935456588 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8bb409b770 con 0x5646dad2c5d0 2024-09-11T21:58:04.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.842+0000 7f8bccd1d740 10 client.25252 fetch_fsmap learned FSMap version 28 2024-09-11T21:58:04.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.842+0000 7f8bccd1d740 1 -- 172.21.15.1:0/2935456588 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({fsmap=28}) -- 0x5646dadb2260 con 0x5646dad2c5d0 2024-09-11T21:58:04.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.842+0000 7f8ba37fe640 1 -- 172.21.15.1:0/2935456588 <== mon.1 v2:172.21.15.161:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8bb409adf0 con 0x5646dad2c5d0 2024-09-11T21:58:04.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.842+0000 7f8bccd1d740 10 client.25252 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T21:58:04.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.842+0000 7f8bccd1d740 10 client.25252 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-11T21:58:04.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.842+0000 7f8bccd1d740 20 client.25252 populate_metadata read hostname 'smithi001' 2024-09-11T21:58:04.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.842+0000 7f8bccd1d740 1 --2- 172.21.15.1:0/2935456588 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x5646dad46ff0 0x5646dadb45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:04.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.842+0000 7f8bccd1d740 4 client.25252 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T21:58:04.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.842+0000 7f8bccd1d740 1 -- 172.21.15.1:0/2935456588 --> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5646dac99d70 con 0x5646dad46ff0 2024-09-11T21:58:04.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.843+0000 7f8bc37fe640 1 --2- 172.21.15.1:0/2935456588 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x5646dad46ff0 0x5646dadb45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:04.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.843+0000 7f8bc37fe640 1 --2- 172.21.15.1:0/2935456588 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x5646dad46ff0 0x5646dadb45a0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7b59612a64909c8e server_cookie=2f13bbb3b4c1cff9 in_seq=0 out_seq=0 2024-09-11T21:58:04.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.843+0000 7f8ba37fe640 10 client.25252 ms_handle_connect on v2:172.21.15.53:6836/3457131830 2024-09-11T21:58:04.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.843+0000 7f8ba37fe640 1 -- 172.21.15.1:0/2935456588 <== mds.2 v2:172.21.15.53:6836/3457131830 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5646dac99d70 con 0x5646dad46ff0 2024-09-11T21:58:04.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.843+0000 7f8ba37fe640 10 client.25252 handle_command_reply: tid=0 2024-09-11T21:58:04.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.843+0000 7f8bccd1d740 1 client.25252 shutdown 2024-09-11T21:58:04.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.844+0000 7f8bccd1d740 1 -- 172.21.15.1:0/2935456588 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646dad2c5d0 msgr2=0x5646dad56f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:04.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.844+0000 7f8bccd1d740 1 --2- 172.21.15.1:0/2935456588 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646dad2c5d0 0x5646dad56f00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8bb409e050 tx=0x7f8bb40d5110 comp rx=0 tx=0).stop 2024-09-11T21:58:04.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.844+0000 7f8bccd1d740 1 -- 172.21.15.1:0/2935456588 shutdown_connections 2024-09-11T21:58:04.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.844+0000 7f8bccd1d740 1 --2- 172.21.15.1:0/2935456588 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x5646dad46ff0 0x5646dadb45a0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:04.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.844+0000 7f8bccd1d740 1 --2- 172.21.15.1:0/2935456588 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5646dad3fb80 0x5646dadaaa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:04.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.844+0000 7f8bccd1d740 1 --2- 172.21.15.1:0/2935456588 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5646dad2db40 0x5646dad57440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:04.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.844+0000 7f8bccd1d740 1 --2- 172.21.15.1:0/2935456588 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5646dad2c5d0 0x5646dad56f00 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:04.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.844+0000 7f8bccd1d740 1 -- 172.21.15.1:0/2935456588 >> 172.21.15.1:0/2935456588 conn(0x5646dacf5310 msgr2=0x5646dad2e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:04.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.844+0000 7f8ba37fe640 0 client.25252 ms_handle_reset on v2:172.21.15.53:6836/3457131830 2024-09-11T21:58:04.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.844+0000 7f8bccd1d740 1 -- 172.21.15.1:0/2935456588 shutdown_connections 2024-09-11T21:58:04.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.844+0000 7f8bccd1d740 1 -- 172.21.15.1:0/2935456588 wait complete. 2024-09-11T21:58:04.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.844+0000 7f8bccd1d740 20 client.25252 trim_cache size 0 max 16384 2024-09-11T21:58:04.852 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-11T21:58:04.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.845+0000 7f8bca944640 1 -- 172.21.15.1:0/2739785250 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f8bac0a2f10 msgr2=0x7f8bac0a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:04.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.845+0000 7f8bca944640 1 --2- 172.21.15.1:0/2739785250 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f8bac0a2f10 0x7f8bac0a53d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x5646dad2e170 tx=0x7f8bbc05b000 comp rx=0 tx=0).stop 2024-09-11T21:58:04.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.845+0000 7f8bca944640 1 -- 172.21.15.1:0/2739785250 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8bc4100e20 msgr2=0x7f8bc419a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:04.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.845+0000 7f8bca944640 1 --2- 172.21.15.1:0/2739785250 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8bc4100e20 0x7f8bc419a670 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8bb4005fd0 tx=0x7f8bb400dde0 comp rx=0 tx=0).stop 2024-09-11T21:58:04.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.845+0000 7f8bca944640 1 -- 172.21.15.1:0/2739785250 shutdown_connections 2024-09-11T21:58:04.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.845+0000 7f8bca944640 1 --2- 172.21.15.1:0/2739785250 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f8bac0a2f10 0x7f8bac0a53d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:04.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.845+0000 7f8bca944640 1 --2- 172.21.15.1:0/2739785250 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8bc4102ef0 0x7f8bc419ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:04.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.845+0000 7f8bca944640 1 --2- 172.21.15.1:0/2739785250 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bc4102570 0x7f8bc419abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:04.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.845+0000 7f8bca944640 1 --2- 172.21.15.1:0/2739785250 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8bc4100e20 0x7f8bc419a670 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:04.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.845+0000 7f8bca944640 1 -- 172.21.15.1:0/2739785250 >> 172.21.15.1:0/2739785250 conn(0x7f8bc40fc7d0 msgr2=0x7f8bc4106df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:04.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.845+0000 7f8bca944640 1 -- 172.21.15.1:0/2739785250 shutdown_connections 2024-09-11T21:58:04.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:04.845+0000 7f8bca944640 1 -- 172.21.15.1:0/2739785250 wait complete. 2024-09-11T21:58:05.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:58:05 smithi001 ceph-mon[30935]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T21:58:05.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:58:05 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/3032773054' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:58:05.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:58:05 smithi053 ceph-mon[34908]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T21:58:05.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:58:05 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/3032773054' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:58:05.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:58:05 smithi161 ceph-mon[35615]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T21:58:05.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:58:05 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/3032773054' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:58:05.434 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:05.432+0000 7f691b7fe640 1 -- 172.21.15.1:0/1432636414 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a673956d70 con 0x55a673a70920 2024-09-11T21:58:05.435 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:05.432+0000 7f691b7fe640 10 client.15648 handle_command_reply: tid=0 2024-09-11T21:58:05.435 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:05.432+0000 7f69457ed740 1 client.15648 shutdown 2024-09-11T21:58:05.435 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:05.432+0000 7f69457ed740 1 -- 172.21.15.1:0/1432636414 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6739eaaf0 msgr2=0x55a6739fdeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:05.435 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:05.432+0000 7f69457ed740 1 --2- 172.21.15.1:0/1432636414 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6739eaaf0 0x55a6739fdeb0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f692c097f50 tx=0x7f692c005a90 comp rx=0 tx=0).stop 2024-09-11T21:58:05.435 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:05.432+0000 7f69457ed740 1 -- 172.21.15.1:0/1432636414 shutdown_connections 2024-09-11T21:58:05.435 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:05.432+0000 7f69457ed740 1 --2- 172.21.15.1:0/1432636414 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55a673a70920 0x55a673a72d10 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:05.435 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:05.432+0000 7f69457ed740 1 --2- 172.21.15.1:0/1432636414 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6739f5cb0 0x55a6739ef3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:05.435 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:05.432+0000 7f69457ed740 1 --2- 172.21.15.1:0/1432636414 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6739eb410 0x55a6739fe3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:05.435 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:05.432+0000 7f69457ed740 1 --2- 172.21.15.1:0/1432636414 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6739eaaf0 0x55a6739fdeb0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:05.435 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:05.432+0000 7f69457ed740 1 -- 172.21.15.1:0/1432636414 >> 172.21.15.1:0/1432636414 conn(0x55a6739e7060 msgr2=0x55a6739e59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:05.435 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:05.433+0000 7f691b7fe640 0 client.15648 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T21:58:05.436 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:05.433+0000 7f69457ed740 1 -- 172.21.15.1:0/1432636414 shutdown_connections 2024-09-11T21:58:05.436 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:05.433+0000 7f69457ed740 1 -- 172.21.15.1:0/1432636414 wait complete. 2024-09-11T21:58:05.436 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:05.433+0000 7f69457ed740 20 client.15648 trim_cache size 0 max 16384 2024-09-11T21:58:05.436 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-09-11T21:58:05.436 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-09-11T21:58:05.436 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-09-11T21:58:05.436 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-09-11T21:58:05.437 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:05.434+0000 7f6942a5c640 1 -- 172.21.15.1:0/298871045 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f69240a2e10 msgr2=0x7f69240a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:05.437 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:05.434+0000 7f6942a5c640 1 --2- 172.21.15.1:0/298871045 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f69240a2e10 0x7f69240a52d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6934009fd0 tx=0x7f6934004ab0 comp rx=0 tx=0).stop 2024-09-11T21:58:05.437 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:05.434+0000 7f6942a5c640 1 -- 172.21.15.1:0/298871045 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f693c101ac0 msgr2=0x7f693c19a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:05.437 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:05.434+0000 7f6942a5c640 1 --2- 172.21.15.1:0/298871045 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f693c101ac0 0x7f693c19a640 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f692c00d950 tx=0x7f692c00de10 comp rx=0 tx=0).stop 2024-09-11T21:58:05.437 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:05.434+0000 7f6942a5c640 1 -- 172.21.15.1:0/298871045 shutdown_connections 2024-09-11T21:58:05.438 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:05.434+0000 7f6942a5c640 1 --2- 172.21.15.1:0/298871045 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f69240a2e10 0x7f69240a52d0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:05.438 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:05.434+0000 7f6942a5c640 1 --2- 172.21.15.1:0/298871045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f693c107ad0 0x7f693c19ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:05.438 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:05.434+0000 7f6942a5c640 1 --2- 172.21.15.1:0/298871045 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f693c102460 0x7f693c19ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:05.438 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:05.434+0000 7f6942a5c640 1 --2- 172.21.15.1:0/298871045 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f693c101ac0 0x7f693c19a640 secure :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f692c00d950 tx=0x7f692c00de10 comp rx=0 tx=0).stop 2024-09-11T21:58:05.438 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:05.434+0000 7f6942a5c640 1 -- 172.21.15.1:0/298871045 >> 172.21.15.1:0/298871045 conn(0x7f693c0fd7a0 msgr2=0x7f693c10d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:05.438 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:05.435+0000 7f6942a5c640 1 -- 172.21.15.1:0/298871045 shutdown_connections 2024-09-11T21:58:05.438 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:05.435+0000 7f6942a5c640 1 -- 172.21.15.1:0/298871045 wait complete. 2024-09-11T21:58:05.476 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db / --timeout 90 --expiration 144.5 --await-for 45.0' 2024-09-11T21:58:05.476 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db / --timeout 90 --expiration 144.5 --await-for 45.0 2024-09-11T21:58:05.816 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:58:06.253 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T21:58:06.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.803+0000 7f5833862640 1 -- 172.21.15.1:0/2748193179 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f582c112640 msgr2=0x7f582c112a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:06.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.803+0000 7f5833862640 1 --2- 172.21.15.1:0/2748193179 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f582c112640 0x7f582c112a20 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f5828009900 tx=0x7f582802a3f0 comp rx=0 tx=0).stop 2024-09-11T21:58:06.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.803+0000 7f5833862640 1 -- 172.21.15.1:0/2748193179 shutdown_connections 2024-09-11T21:58:06.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.803+0000 7f5833862640 1 --2- 172.21.15.1:0/2748193179 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f582c10cf30 0x7f582c119410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:06.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.803+0000 7f5833862640 1 --2- 172.21.15.1:0/2748193179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f582c10c590 0x7f582c10c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:06.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.803+0000 7f5833862640 1 --2- 172.21.15.1:0/2748193179 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f582c112640 0x7f582c112a20 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:06.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.803+0000 7f5833862640 1 -- 172.21.15.1:0/2748193179 >> 172.21.15.1:0/2748193179 conn(0x7f582c06cdc0 msgr2=0x7f582c06d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:06.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.803+0000 7f5833862640 1 -- 172.21.15.1:0/2748193179 shutdown_connections 2024-09-11T21:58:06.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.803+0000 7f5833862640 1 -- 172.21.15.1:0/2748193179 wait complete. 2024-09-11T21:58:06.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.804+0000 7f5833862640 1 Processor -- start 2024-09-11T21:58:06.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.804+0000 7f5833862640 1 -- start start 2024-09-11T21:58:06.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.805+0000 7f5833862640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f582c10c590 0x7f582c1a8880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:06.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.805+0000 7f5833862640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f582c10cf30 0x7f582c1a8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:06.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.805+0000 7f5833862640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f582c112640 0x7f582c1a2a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:06.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.805+0000 7f5833862640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f582c11c9b0 con 0x7f582c112640 2024-09-11T21:58:06.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.805+0000 7f5833862640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f582c11c830 con 0x7f582c10c590 2024-09-11T21:58:06.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.805+0000 7f5833862640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f582c11cb30 con 0x7f582c10cf30 2024-09-11T21:58:06.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.805+0000 7f58315fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f582c10c590 0x7f582c1a8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:06.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.805+0000 7f58315fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f582c10c590 0x7f582c1a8880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:55392/0 (socket says 172.21.15.1:55392) 2024-09-11T21:58:06.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.805+0000 7f58315fe640 1 -- 172.21.15.1:0/3113993753 learned_addr learned my addr 172.21.15.1:0/3113993753 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:58:06.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.805+0000 7f5831dff640 1 --2- 172.21.15.1:0/3113993753 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f582c112640 0x7f582c1a2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:06.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.805+0000 7f5830dfd640 1 --2- 172.21.15.1:0/3113993753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f582c10cf30 0x7f582c1a8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:06.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.805+0000 7f58315fe640 1 -- 172.21.15.1:0/3113993753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f582c10cf30 msgr2=0x7f582c1a8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:06.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.805+0000 7f58315fe640 1 --2- 172.21.15.1:0/3113993753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f582c10cf30 0x7f582c1a8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:06.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.805+0000 7f58315fe640 1 -- 172.21.15.1:0/3113993753 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f582c112640 msgr2=0x7f582c1a2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:06.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.805+0000 7f58315fe640 1 --2- 172.21.15.1:0/3113993753 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f582c112640 0x7f582c1a2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:06.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.805+0000 7f58315fe640 1 -- 172.21.15.1:0/3113993753 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f582c1a3170 con 0x7f582c10c590 2024-09-11T21:58:06.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.805+0000 7f5830dfd640 1 --2- 172.21.15.1:0/3113993753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f582c10cf30 0x7f582c1a8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T21:58:06.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.806+0000 7f58315fe640 1 --2- 172.21.15.1:0/3113993753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f582c10c590 0x7f582c1a8880 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5828009850 tx=0x7f5828030640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:06.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.806+0000 7f58227fc640 1 -- 172.21.15.1:0/3113993753 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582803f070 con 0x7f582c10c590 2024-09-11T21:58:06.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.806+0000 7f5833862640 1 -- 172.21.15.1:0/3113993753 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f582c1a3400 con 0x7f582c10c590 2024-09-11T21:58:06.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.806+0000 7f58227fc640 1 -- 172.21.15.1:0/3113993753 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5828004460 con 0x7f582c10c590 2024-09-11T21:58:06.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.806+0000 7f5833862640 1 -- 172.21.15.1:0/3113993753 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f582c1af750 con 0x7f582c10c590 2024-09-11T21:58:06.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.806+0000 7f58227fc640 1 -- 172.21.15.1:0/3113993753 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5828035030 con 0x7f582c10c590 2024-09-11T21:58:06.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.810+0000 7f5835c3b740 1 Processor -- start 2024-09-11T21:58:06.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.810+0000 7f5835c3b740 1 -- start start 2024-09-11T21:58:06.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.810+0000 7f5835c3b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56177cb3e760 0x56177cb3eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:06.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.810+0000 7f5835c3b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56177cb3f080 0x56177cb49400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:06.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.810+0000 7f5835c3b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56177cb49940 0x56177cb4bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:06.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.810+0000 7f5835c3b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56177caa42a0 con 0x56177cb3f080 2024-09-11T21:58:06.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.810+0000 7f5835c3b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56177ca4bb10 con 0x56177cb3e760 2024-09-11T21:58:06.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.810+0000 7f5835c3b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56177caa0260 con 0x56177cb49940 2024-09-11T21:58:06.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.810+0000 7f58227fc640 1 -- 172.21.15.1:0/3113993753 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 25) ==== 138065+0+0 (secure 0 0 0) 0x7f5828042070 con 0x7f582c10c590 2024-09-11T21:58:06.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.810+0000 7f58315fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56177cb3f080 0x56177cb49400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:06.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.811+0000 7f58315fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56177cb3f080 0x56177cb49400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42636/0 (socket says 172.21.15.1:42636) 2024-09-11T21:58:06.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.811+0000 7f58315fe640 1 -- 172.21.15.1:0/1026456296 learned_addr learned my addr 172.21.15.1:0/1026456296 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:58:06.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.811+0000 7f5831dff640 1 --2- 172.21.15.1:0/1026456296 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56177cb3e760 0x56177cb3eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:06.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.811+0000 7f5830dfd640 1 --2- 172.21.15.1:0/1026456296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56177cb49940 0x56177cb4bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:06.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.811+0000 7f58315fe640 1 -- 172.21.15.1:0/1026456296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56177cb49940 msgr2=0x56177cb4bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:06.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.811+0000 7f58315fe640 1 --2- 172.21.15.1:0/1026456296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56177cb49940 0x56177cb4bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:06.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.811+0000 7f58315fe640 1 -- 172.21.15.1:0/1026456296 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56177cb3e760 msgr2=0x56177cb3eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:06.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.811+0000 7f58315fe640 1 --2- 172.21.15.1:0/1026456296 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56177cb3e760 0x56177cb3eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:06.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.811+0000 7f58315fe640 1 -- 172.21.15.1:0/1026456296 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56177c815b90 con 0x56177cb3f080 2024-09-11T21:58:06.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.812+0000 7f58315fe640 1 --2- 172.21.15.1:0/1026456296 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56177cb3f080 0x56177cb49400 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f58280ed5a0 tx=0x7f58280f82a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44315dfff960788c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:06.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.812+0000 7f58227fc640 1 --2- 172.21.15.1:0/3113993753 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f58180a2f50 0x7f58180a5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:06.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.812+0000 7f58177fe640 1 -- 172.21.15.1:0/1026456296 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58280bb820 con 0x56177cb3f080 2024-09-11T21:58:06.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.812+0000 7f58227fc640 1 -- 172.21.15.1:0/3113993753 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7f58280f0090 con 0x7f582c10c590 2024-09-11T21:58:06.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.813+0000 7f58177fe640 1 -- 172.21.15.1:0/1026456296 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58280fd030 con 0x56177cb3f080 2024-09-11T21:58:06.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.813+0000 7f5831dff640 1 --2- 172.21.15.1:0/3113993753 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f58180a2f50 0x7f58180a5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:06.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.813+0000 7f58177fe640 1 -- 172.21.15.1:0/1026456296 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58280bbb50 con 0x56177cb3f080 2024-09-11T21:58:06.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.813+0000 7f5835c3b740 1 -- 172.21.15.1:0/1026456296 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56177cb3f080 msgr2=0x56177cb49400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:06.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.813+0000 7f5835c3b740 1 --2- 172.21.15.1:0/1026456296 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56177cb3f080 0x56177cb49400 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f58280ed5a0 tx=0x7f58280f82a0 comp rx=0 tx=0).stop 2024-09-11T21:58:06.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.814+0000 7f5831dff640 1 --2- 172.21.15.1:0/3113993753 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f58180a2f50 0x7f58180a5410 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5824005fd0 tx=0x7f58240096a0 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:06.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.814+0000 7f5835c3b740 1 -- 172.21.15.1:0/1026456296 shutdown_connections 2024-09-11T21:58:06.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.814+0000 7f5835c3b740 1 --2- 172.21.15.1:0/1026456296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56177cb49940 0x56177cb4bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:06.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.814+0000 7f5835c3b740 1 --2- 172.21.15.1:0/1026456296 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56177cb3f080 0x56177cb49400 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:06.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.814+0000 7f5835c3b740 1 --2- 172.21.15.1:0/1026456296 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56177cb3e760 0x56177cb3eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:06.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.814+0000 7f5835c3b740 1 -- 172.21.15.1:0/1026456296 >> 172.21.15.1:0/1026456296 conn(0x56177cb3acf0 msgr2=0x56177cb3b0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:06.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.814+0000 7f5835c3b740 1 -- 172.21.15.1:0/1026456296 shutdown_connections 2024-09-11T21:58:06.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.814+0000 7f5835c3b740 1 -- 172.21.15.1:0/1026456296 wait complete. 2024-09-11T21:58:06.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.815+0000 7f5835c3b740 1 -- start start 2024-09-11T21:58:06.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.815+0000 7f5835c3b740 1 Processor -- start 2024-09-11T21:58:06.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.815+0000 7f5835c3b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56177cb3e760 0x56177cb467b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:06.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.816+0000 7f5835c3b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56177cb3f080 0x56177cb46cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:06.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.816+0000 7f5835c3b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56177cb49940 0x56177cb47230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:06.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.816+0000 7f5835c3b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56177cb4c2d0 con 0x56177cb3e760 2024-09-11T21:58:06.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.816+0000 7f5835c3b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56177ca2c6e0 con 0x56177cb3f080 2024-09-11T21:58:06.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.816+0000 7f5835c3b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56177cb4c450 con 0x56177cb49940 2024-09-11T21:58:06.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.816+0000 7f5835c3b740 10 client.0 _ll_register_callbacks cb 0x56177ca50a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:58:06.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.816+0000 7f58315fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56177cb3e760 0x56177cb467b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:06.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.816+0000 7f5831dff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56177cb49940 0x56177cb47230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:06.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.816+0000 7f58315fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56177cb3e760 0x56177cb467b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42650/0 (socket says 172.21.15.1:42650) 2024-09-11T21:58:06.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.816+0000 7f58315fe640 1 -- 172.21.15.1:0/4132017452 learned_addr learned my addr 172.21.15.1:0/4132017452 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:58:06.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.816+0000 7f5830dfd640 1 --2- 172.21.15.1:0/4132017452 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56177cb3f080 0x56177cb46cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:06.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.816+0000 7f58315fe640 1 -- 172.21.15.1:0/4132017452 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56177cb49940 msgr2=0x56177cb47230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:06.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.816+0000 7f58315fe640 1 --2- 172.21.15.1:0/4132017452 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56177cb49940 0x56177cb47230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:06.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.816+0000 7f58315fe640 1 -- 172.21.15.1:0/4132017452 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56177cb3f080 msgr2=0x56177cb46cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:06.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.816+0000 7f58315fe640 1 --2- 172.21.15.1:0/4132017452 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56177cb3f080 0x56177cb46cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:06.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.817+0000 7f58315fe640 1 -- 172.21.15.1:0/4132017452 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56177cb55700 con 0x56177cb3e760 2024-09-11T21:58:06.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.817+0000 7f5830dfd640 1 --2- 172.21.15.1:0/4132017452 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56177cb3f080 0x56177cb46cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T21:58:06.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.817+0000 7f58315fe640 1 --2- 172.21.15.1:0/4132017452 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56177cb3e760 0x56177cb467b0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f58280306b0 tx=0x7f582802a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b9998fcf3b10447 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:06.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.817+0000 7f5814ff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T21:58:06.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.817+0000 7f5814ff9640 1 -- 172.21.15.1:0/4132017452 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58280bbd30 con 0x56177cb3e760 2024-09-11T21:58:06.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.817+0000 7f5835c3b740 1 -- 172.21.15.1:0/4132017452 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58280fd030 con 0x56177cb3e760 2024-09-11T21:58:06.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.817+0000 7f5814ff9640 1 -- 172.21.15.1:0/4132017452 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7f58280035f0 con 0x56177cb3e760 2024-09-11T21:58:06.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.817+0000 7f5814ff9640 1 -- 172.21.15.1:0/4132017452 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5828049020 con 0x56177cb3e760 2024-09-11T21:58:06.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.818+0000 7f5835c3b740 10 client.15681 fetch_fsmap learned FSMap version 28 2024-09-11T21:58:06.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.818+0000 7f5835c3b740 1 -- 172.21.15.1:0/4132017452 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56177cb4c2d0 con 0x56177cb3e760 2024-09-11T21:58:06.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.818+0000 7f5814ff9640 1 -- 172.21.15.1:0/4132017452 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58280fd030 con 0x56177cb3e760 2024-09-11T21:58:06.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.818+0000 7f5814ff9640 1 -- 172.21.15.1:0/4132017452 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f58280b8a40 con 0x56177cb3e760 2024-09-11T21:58:06.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.818+0000 7f5835c3b740 10 client.15681 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T21:58:06.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.818+0000 7f5835c3b740 10 client.15681 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-11T21:58:06.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.818+0000 7f5835c3b740 20 client.15681 populate_metadata read hostname 'smithi001' 2024-09-11T21:58:06.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.818+0000 7f5835c3b740 1 --2- 172.21.15.1:0/4132017452 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x56177cb59300 0x56177cb5b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:06.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.818+0000 7f5835c3b740 4 client.15681 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T21:58:06.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.818+0000 7f5835c3b740 1 -- 172.21.15.1:0/4132017452 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56177cab19d0 con 0x56177cb59300 2024-09-11T21:58:06.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.819+0000 7f5830dfd640 1 --2- 172.21.15.1:0/4132017452 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x56177cb59300 0x56177cb5b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:06.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.819+0000 7f5830dfd640 1 --2- 172.21.15.1:0/4132017452 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x56177cb59300 0x56177cb5b7c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c56e29d87156eee server_cookie=cb5dd1634b4790b5 in_seq=0 out_seq=0 2024-09-11T21:58:06.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.819+0000 7f5814ff9640 10 client.15681 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T21:58:06.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.820+0000 7f5814ff9640 1 -- 172.21.15.1:0/4132017452 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56177cab19d0 con 0x56177cb59300 2024-09-11T21:58:06.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.820+0000 7f5814ff9640 10 client.15681 handle_command_reply: tid=0 2024-09-11T21:58:06.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.820+0000 7f5835c3b740 1 client.15681 shutdown 2024-09-11T21:58:06.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.820+0000 7f5835c3b740 1 -- 172.21.15.1:0/4132017452 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56177cb3e760 msgr2=0x56177cb467b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:06.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.820+0000 7f5835c3b740 1 --2- 172.21.15.1:0/4132017452 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56177cb3e760 0x56177cb467b0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f58280306b0 tx=0x7f582802a3f0 comp rx=0 tx=0).stop 2024-09-11T21:58:06.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.820+0000 7f5835c3b740 1 -- 172.21.15.1:0/4132017452 shutdown_connections 2024-09-11T21:58:06.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.820+0000 7f5835c3b740 1 --2- 172.21.15.1:0/4132017452 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x56177cb59300 0x56177cb5b7c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:06.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.820+0000 7f5835c3b740 1 --2- 172.21.15.1:0/4132017452 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56177cb49940 0x56177cb47230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:06.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.820+0000 7f5835c3b740 1 --2- 172.21.15.1:0/4132017452 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56177cb3f080 0x56177cb46cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:06.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.820+0000 7f5835c3b740 1 --2- 172.21.15.1:0/4132017452 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56177cb3e760 0x56177cb467b0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:06.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.820+0000 7f5835c3b740 1 -- 172.21.15.1:0/4132017452 >> 172.21.15.1:0/4132017452 conn(0x56177cb3acf0 msgr2=0x56177cb40a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:06.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.820+0000 7f5814ff9640 0 client.15681 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T21:58:06.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.821+0000 7f5835c3b740 1 -- 172.21.15.1:0/4132017452 shutdown_connections 2024-09-11T21:58:06.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.821+0000 7f5835c3b740 1 -- 172.21.15.1:0/4132017452 wait complete. 2024-09-11T21:58:06.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.821+0000 7f5835c3b740 20 client.15681 trim_cache size 0 max 16384 2024-09-11T21:58:06.831 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:58:06.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.846+0000 7f5835c3b740 1 Processor -- start 2024-09-11T21:58:06.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.846+0000 7f5835c3b740 1 -- start start 2024-09-11T21:58:06.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.846+0000 7f5835c3b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56177cb3e760 0x56177cb39d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:06.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.846+0000 7f5835c3b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56177cb3f080 0x56177cb61090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:06.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.846+0000 7f5835c3b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56177cb49940 0x56177cb615d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:06.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.846+0000 7f5835c3b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f582c1b8350 con 0x56177cb3f080 2024-09-11T21:58:06.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.846+0000 7f5835c3b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f582c1b9e90 con 0x56177cb49940 2024-09-11T21:58:06.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.846+0000 7f5835c3b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f582c1b8a20 con 0x56177cb3e760 2024-09-11T21:58:06.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.846+0000 7f5831dff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56177cb49940 0x56177cb615d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:06.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.847+0000 7f5830dfd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56177cb3f080 0x56177cb61090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:06.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.847+0000 7f5831dff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56177cb49940 0x56177cb615d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:55418/0 (socket says 172.21.15.1:55418) 2024-09-11T21:58:06.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.847+0000 7f58315fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56177cb3e760 0x56177cb39d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:06.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.847+0000 7f5831dff640 1 -- 172.21.15.1:0/191378742 learned_addr learned my addr 172.21.15.1:0/191378742 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:58:06.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.847+0000 7f5831dff640 1 -- 172.21.15.1:0/191378742 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56177cb3e760 msgr2=0x56177cb39d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:06.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.847+0000 7f5831dff640 1 --2- 172.21.15.1:0/191378742 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56177cb3e760 0x56177cb39d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:06.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.847+0000 7f5831dff640 1 -- 172.21.15.1:0/191378742 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56177cb3f080 msgr2=0x56177cb61090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:06.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.847+0000 7f5831dff640 1 --2- 172.21.15.1:0/191378742 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56177cb3f080 0x56177cb61090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:06.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.847+0000 7f5831dff640 1 -- 172.21.15.1:0/191378742 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56177cb3ab20 con 0x56177cb49940 2024-09-11T21:58:06.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.847+0000 7f5831dff640 1 --2- 172.21.15.1:0/191378742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56177cb49940 0x56177cb615d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5824009ad0 tx=0x7f5824015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6cfb67ac26ecb54 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:06.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.847+0000 7f5816ffd640 1 -- 172.21.15.1:0/191378742 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5824012e20 con 0x56177cb49940 2024-09-11T21:58:06.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.848+0000 7f5816ffd640 1 -- 172.21.15.1:0/191378742 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f582401b930 con 0x56177cb49940 2024-09-11T21:58:06.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.848+0000 7f5816ffd640 1 -- 172.21.15.1:0/191378742 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5824010b00 con 0x56177cb49940 2024-09-11T21:58:06.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.849+0000 7f5835c3b740 1 -- 172.21.15.1:0/191378742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56177cb49940 msgr2=0x56177cb615d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:06.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.849+0000 7f5835c3b740 1 --2- 172.21.15.1:0/191378742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56177cb49940 0x56177cb615d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5824009ad0 tx=0x7f5824015000 comp rx=0 tx=0).stop 2024-09-11T21:58:06.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.849+0000 7f5835c3b740 1 -- 172.21.15.1:0/191378742 shutdown_connections 2024-09-11T21:58:06.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.849+0000 7f5835c3b740 1 --2- 172.21.15.1:0/191378742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56177cb49940 0x56177cb615d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:06.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.849+0000 7f5835c3b740 1 --2- 172.21.15.1:0/191378742 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56177cb3f080 0x56177cb61090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:06.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.849+0000 7f5835c3b740 1 --2- 172.21.15.1:0/191378742 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56177cb3e760 0x56177cb39d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:06.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.849+0000 7f5835c3b740 1 -- 172.21.15.1:0/191378742 >> 172.21.15.1:0/191378742 conn(0x56177cb3acf0 msgr2=0x56177cb39650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:06.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.849+0000 7f5835c3b740 1 -- 172.21.15.1:0/191378742 shutdown_connections 2024-09-11T21:58:06.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.849+0000 7f5835c3b740 1 -- 172.21.15.1:0/191378742 wait complete. 2024-09-11T21:58:06.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.850+0000 7f5835c3b740 1 -- start start 2024-09-11T21:58:06.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.850+0000 7f5835c3b740 1 Processor -- start 2024-09-11T21:58:06.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.851+0000 7f5835c3b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56177cb3e760 0x56177cb4dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:06.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.851+0000 7f5835c3b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56177cb3f080 0x56177cb4e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:06.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.851+0000 7f5835c3b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56177cb49940 0x56177cbc3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:06.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.851+0000 7f5830dfd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56177cb3f080 0x56177cb4e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:06.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.851+0000 7f5830dfd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56177cb3f080 0x56177cb4e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42680/0 (socket says 172.21.15.1:42680) 2024-09-11T21:58:06.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.851+0000 7f58315fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56177cb3e760 0x56177cb4dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:06.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.851+0000 7f58315fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56177cb3e760 0x56177cb4dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:41438/0 (socket says 172.21.15.1:41438) 2024-09-11T21:58:06.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.851+0000 7f5831dff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56177cb49940 0x56177cbc3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:06.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.851+0000 7f5831dff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56177cb49940 0x56177cbc3060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:55420/0 (socket says 172.21.15.1:55420) 2024-09-11T21:58:06.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.851+0000 7f5835c3b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f582c1b97c0 con 0x56177cb3f080 2024-09-11T21:58:06.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.851+0000 7f5830dfd640 1 -- 172.21.15.1:0/3074585900 learned_addr learned my addr 172.21.15.1:0/3074585900 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:58:06.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.851+0000 7f5835c3b740 1 -- 172.21.15.1:0/3074585900 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f582c1b90f0 con 0x56177cb49940 2024-09-11T21:58:06.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.852+0000 7f5835c3b740 1 -- 172.21.15.1:0/3074585900 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f582c1b6330 con 0x56177cb3e760 2024-09-11T21:58:06.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.852+0000 7f5835c3b740 10 client.0 _ll_register_callbacks cb 0x7f5828003880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:58:06.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.852+0000 7f5830dfd640 1 -- 172.21.15.1:0/3074585900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56177cb3e760 msgr2=0x56177cb4dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:06.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.852+0000 7f5830dfd640 1 --2- 172.21.15.1:0/3074585900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56177cb3e760 0x56177cb4dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:06.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.852+0000 7f5830dfd640 1 -- 172.21.15.1:0/3074585900 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56177cb49940 msgr2=0x56177cbc3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:06.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.852+0000 7f5830dfd640 1 --2- 172.21.15.1:0/3074585900 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56177cb49940 0x56177cbc3060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:06.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.852+0000 7f5830dfd640 1 -- 172.21.15.1:0/3074585900 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56177cbc3650 con 0x56177cb3f080 2024-09-11T21:58:06.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.852+0000 7f58315fe640 1 --2- 172.21.15.1:0/3074585900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56177cb3e760 0x56177cb4dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:58:06.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.853+0000 7f5830dfd640 1 --2- 172.21.15.1:0/3074585900 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56177cb3f080 0x56177cb4e110 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f582c10bea0 tx=0x7f581c013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=549a81a185007a90 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:06.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.853+0000 7f5814ff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T21:58:06.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.853+0000 7f5814ff9640 1 -- 172.21.15.1:0/3074585900 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f581c005220 con 0x56177cb3f080 2024-09-11T21:58:06.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.853+0000 7f5814ff9640 1 -- 172.21.15.1:0/3074585900 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7f581c0053c0 con 0x56177cb3f080 2024-09-11T21:58:06.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.853+0000 7f5835c3b740 1 -- 172.21.15.1:0/3074585900 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f582401b930 con 0x56177cb3f080 2024-09-11T21:58:06.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.853+0000 7f5814ff9640 1 -- 172.21.15.1:0/3074585900 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f581c0055c0 con 0x56177cb3f080 2024-09-11T21:58:06.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.853+0000 7f5835c3b740 10 client.15687 fetch_fsmap learned FSMap version 28 2024-09-11T21:58:06.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.853+0000 7f5835c3b740 1 -- 172.21.15.1:0/3074585900 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56177cbc4960 con 0x56177cb3f080 2024-09-11T21:58:06.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.854+0000 7f5814ff9640 1 -- 172.21.15.1:0/3074585900 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f582401b930 con 0x56177cb3f080 2024-09-11T21:58:06.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.854+0000 7f5814ff9640 1 -- 172.21.15.1:0/3074585900 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f581c021070 con 0x56177cb3f080 2024-09-11T21:58:06.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.854+0000 7f5835c3b740 10 client.15687 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T21:58:06.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.854+0000 7f5835c3b740 10 client.15687 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-11T21:58:06.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.854+0000 7f5835c3b740 20 client.15687 populate_metadata read hostname 'smithi001' 2024-09-11T21:58:06.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.854+0000 7f5835c3b740 1 --2- 172.21.15.1:0/3074585900 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x56177cbc5130 0x56177cbc7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:06.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.854+0000 7f5835c3b740 4 client.15687 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.5, "await_for": 45.0}] 2024-09-11T21:58:06.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.854+0000 7f5835c3b740 1 -- 172.21.15.1:0/3074585900 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.5, "await_for": 45.0}) -- 0x56177caabd20 con 0x56177cbc5130 2024-09-11T21:58:06.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.854+0000 7f58315fe640 1 --2- 172.21.15.1:0/3074585900 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x56177cbc5130 0x56177cbc7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:06.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.855+0000 7f58315fe640 1 --2- 172.21.15.1:0/3074585900 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x56177cbc5130 0x56177cbc7550 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6d05315d2bebd03 server_cookie=b63a90f2251c8410 in_seq=0 out_seq=0 2024-09-11T21:58:06.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.855+0000 7f5814ff9640 10 client.15687 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T21:58:06.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.867+0000 7f5814ff9640 1 -- 172.21.15.1:0/3074585900 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x56177caabd20 con 0x56177cbc5130 2024-09-11T21:58:06.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.867+0000 7f5814ff9640 10 client.15687 handle_command_reply: tid=0 2024-09-11T21:58:06.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.867+0000 7f5835c3b740 1 client.15687 shutdown 2024-09-11T21:58:06.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.868+0000 7f5835c3b740 1 -- 172.21.15.1:0/3074585900 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56177cb3f080 msgr2=0x56177cb4e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:06.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.868+0000 7f5835c3b740 1 --2- 172.21.15.1:0/3074585900 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56177cb3f080 0x56177cb4e110 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f582c10bea0 tx=0x7f581c013830 comp rx=0 tx=0).stop 2024-09-11T21:58:06.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.868+0000 7f5835c3b740 1 -- 172.21.15.1:0/3074585900 shutdown_connections 2024-09-11T21:58:06.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.868+0000 7f5835c3b740 1 --2- 172.21.15.1:0/3074585900 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x56177cbc5130 0x56177cbc7550 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:06.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.868+0000 7f5835c3b740 1 --2- 172.21.15.1:0/3074585900 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56177cb49940 0x56177cbc3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:06.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.868+0000 7f5835c3b740 1 --2- 172.21.15.1:0/3074585900 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56177cb3f080 0x56177cb4e110 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:06.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.868+0000 7f5835c3b740 1 --2- 172.21.15.1:0/3074585900 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56177cb3e760 0x56177cb4dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:06.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.868+0000 7f5835c3b740 1 -- 172.21.15.1:0/3074585900 >> 172.21.15.1:0/3074585900 conn(0x56177cb3acf0 msgr2=0x56177cb46f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:06.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.868+0000 7f5814ff9640 0 client.15687 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T21:58:06.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.868+0000 7f5835c3b740 1 -- 172.21.15.1:0/3074585900 shutdown_connections 2024-09-11T21:58:06.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.868+0000 7f5835c3b740 1 -- 172.21.15.1:0/3074585900 wait complete. 2024-09-11T21:58:06.871 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-11T21:58:06.871 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 28, 2024-09-11T21:58:06.871 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24494, 2024-09-11T21:58:06.871 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 2, 2024-09-11T21:58:06.871 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-09-11T21:58:06.871 INFO:teuthology.orchestra.run.smithi001.stdout: "f7fcf15b": { 2024-09-11T21:58:06.871 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 2, 2024-09-11T21:58:06.871 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-09-11T21:58:06.871 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-11T21:58:06.872 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-09-11T21:58:06.872 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-11T21:58:06.872 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T21:58:06.872 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 90.0, 2024-09-11T21:58:06.872 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 144.5, 2024-09-11T21:58:06.872 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-09-11T21:58:06.872 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-09-11T21:58:06.872 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-09-11T21:58:06.872 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-11T21:58:06.872 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-09-11T21:58:06.872 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-11T21:58:06.872 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T21:58:06.872 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T21:58:06.873 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T21:58:06.873 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T21:58:06.873 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T21:58:06.873 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-11T21:58:06.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.868+0000 7f5835c3b740 20 client.15687 trim_cache size 0 max 16384 2024-09-11T21:58:06.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.869+0000 7f5833862640 1 -- 172.21.15.1:0/3113993753 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f58180a2f50 msgr2=0x7f58180a5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:06.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.869+0000 7f5833862640 1 --2- 172.21.15.1:0/3113993753 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f58180a2f50 0x7f58180a5410 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5824005fd0 tx=0x7f58240096a0 comp rx=0 tx=0).stop 2024-09-11T21:58:06.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.869+0000 7f5833862640 1 -- 172.21.15.1:0/3113993753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f582c10c590 msgr2=0x7f582c1a8880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:06.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.869+0000 7f5833862640 1 --2- 172.21.15.1:0/3113993753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f582c10c590 0x7f582c1a8880 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5828009850 tx=0x7f5828030640 comp rx=0 tx=0).stop 2024-09-11T21:58:06.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.869+0000 7f5833862640 1 -- 172.21.15.1:0/3113993753 shutdown_connections 2024-09-11T21:58:06.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.869+0000 7f5833862640 1 --2- 172.21.15.1:0/3113993753 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f58180a2f50 0x7f58180a5410 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:06.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.869+0000 7f5833862640 1 --2- 172.21.15.1:0/3113993753 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f582c112640 0x7f582c1a2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:06.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.869+0000 7f5833862640 1 --2- 172.21.15.1:0/3113993753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f582c10cf30 0x7f582c1a8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:06.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.869+0000 7f5833862640 1 --2- 172.21.15.1:0/3113993753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f582c10c590 0x7f582c1a8880 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:06.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.869+0000 7f5833862640 1 -- 172.21.15.1:0/3113993753 >> 172.21.15.1:0/3113993753 conn(0x7f582c06cdc0 msgr2=0x7f582c10ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:06.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.869+0000 7f5833862640 1 -- 172.21.15.1:0/3113993753 shutdown_connections 2024-09-11T21:58:06.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:06.869+0000 7f5833862640 1 -- 172.21.15.1:0/3113993753 wait complete. 2024-09-11T21:58:07.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:58:07 smithi053 ceph-mon[34908]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-09-11T21:58:07.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:58:07 smithi161 ceph-mon[35615]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-09-11T21:58:07.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.356+0000 7fcceec9c640 1 -- 172.21.15.1:0/646801692 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcce8102780 msgr2=0x7fcce81067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:07.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.356+0000 7fcceec9c640 1 --2- 172.21.15.1:0/646801692 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcce8102780 0x7fcce81067d0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fccd4009900 tx=0x7fccd402a410 comp rx=0 tx=0).stop 2024-09-11T21:58:07.357 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:58:07 smithi001 ceph-mon[30935]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-09-11T21:58:07.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.356+0000 7fcceec9c640 1 -- 172.21.15.1:0/646801692 shutdown_connections 2024-09-11T21:58:07.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.356+0000 7fcceec9c640 1 --2- 172.21.15.1:0/646801692 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcce8106f00 0x7fcce810ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:07.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.356+0000 7fcceec9c640 1 --2- 172.21.15.1:0/646801692 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcce8102780 0x7fcce81067d0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:07.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.356+0000 7fcceec9c640 1 --2- 172.21.15.1:0/646801692 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcce8101dd0 0x7fcce81021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:07.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.356+0000 7fcceec9c640 1 -- 172.21.15.1:0/646801692 >> 172.21.15.1:0/646801692 conn(0x7fcce80fd8b0 msgr2=0x7fcce80ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:07.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.356+0000 7fcceec9c640 1 -- 172.21.15.1:0/646801692 shutdown_connections 2024-09-11T21:58:07.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.356+0000 7fcceec9c640 1 -- 172.21.15.1:0/646801692 wait complete. 2024-09-11T21:58:07.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.356+0000 7fcceec9c640 1 Processor -- start 2024-09-11T21:58:07.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.358+0000 7fcceec9c640 1 -- start start 2024-09-11T21:58:07.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.358+0000 7fcceec9c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcce8101dd0 0x7fcce8198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:07.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.358+0000 7fcceec9c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcce8102780 0x7fcce8198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:07.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.358+0000 7fcceec9c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcce8106f00 0x7fcce819cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:07.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.358+0000 7fcceec9c640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcce8111f60 con 0x7fcce8102780 2024-09-11T21:58:07.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.358+0000 7fcceec9c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcce8111de0 con 0x7fcce8101dd0 2024-09-11T21:58:07.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.358+0000 7fcceec9c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcce81120e0 con 0x7fcce8106f00 2024-09-11T21:58:07.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.358+0000 7fcceca38640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcce8101dd0 0x7fcce8198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:07.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.358+0000 7fcce7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcce8102780 0x7fcce8198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:07.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.358+0000 7fcceca38640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcce8101dd0 0x7fcce8198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:55424/0 (socket says 172.21.15.1:55424) 2024-09-11T21:58:07.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.358+0000 7fcceca38640 1 -- 172.21.15.1:0/1316700996 learned_addr learned my addr 172.21.15.1:0/1316700996 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:58:07.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.358+0000 7fcced239640 1 --2- 172.21.15.1:0/1316700996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcce8106f00 0x7fcce819cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:07.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.358+0000 7fcceca38640 1 -- 172.21.15.1:0/1316700996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcce8106f00 msgr2=0x7fcce819cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:07.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.358+0000 7fcceca38640 1 --2- 172.21.15.1:0/1316700996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcce8106f00 0x7fcce819cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:07.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.358+0000 7fcceca38640 1 -- 172.21.15.1:0/1316700996 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcce8102780 msgr2=0x7fcce8198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:07.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.358+0000 7fcceca38640 1 --2- 172.21.15.1:0/1316700996 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcce8102780 0x7fcce8198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:07.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.358+0000 7fcceca38640 1 -- 172.21.15.1:0/1316700996 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcce819d3f0 con 0x7fcce8101dd0 2024-09-11T21:58:07.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.358+0000 7fcce7fff640 1 --2- 172.21.15.1:0/1316700996 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcce8102780 0x7fcce8198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T21:58:07.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.358+0000 7fcceca38640 1 --2- 172.21.15.1:0/1316700996 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcce8101dd0 0x7fcce8198440 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fccdc00e9f0 tx=0x7fccdc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:07.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.358+0000 7fcce5ffb640 1 -- 172.21.15.1:0/1316700996 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccdc00cd80 con 0x7fcce8101dd0 2024-09-11T21:58:07.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.358+0000 7fcceec9c640 1 -- 172.21.15.1:0/1316700996 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcce819d6e0 con 0x7fcce8101dd0 2024-09-11T21:58:07.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.358+0000 7fcce5ffb640 1 -- 172.21.15.1:0/1316700996 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccdc010910 con 0x7fcce8101dd0 2024-09-11T21:58:07.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.358+0000 7fcce5ffb640 1 -- 172.21.15.1:0/1316700996 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccdc0188f0 con 0x7fcce8101dd0 2024-09-11T21:58:07.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.360+0000 7fcceec9c640 1 -- 172.21.15.1:0/1316700996 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcce8074420 con 0x7fcce8101dd0 2024-09-11T21:58:07.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.361+0000 7fccf1075740 1 Processor -- start 2024-09-11T21:58:07.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.361+0000 7fccf1075740 1 -- start start 2024-09-11T21:58:07.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.362+0000 7fccf1075740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562654d25b40 0x562654d24090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:07.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.362+0000 7fccf1075740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562654d245d0 0x562654d24a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:07.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.362+0000 7fccf1075740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562654d2f370 0x562654d31760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:07.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.362+0000 7fccf1075740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x562654c8a2f0 con 0x562654d2f370 2024-09-11T21:58:07.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.362+0000 7fccf1075740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x562654c31cf0 con 0x562654d245d0 2024-09-11T21:58:07.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.362+0000 7fccf1075740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x562654c862b0 con 0x562654d25b40 2024-09-11T21:58:07.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.362+0000 7fcce7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562654d2f370 0x562654d31760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:07.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.362+0000 7fcce7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562654d2f370 0x562654d31760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42714/0 (socket says 172.21.15.1:42714) 2024-09-11T21:58:07.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.362+0000 7fcce7fff640 1 -- 172.21.15.1:0/641305604 learned_addr learned my addr 172.21.15.1:0/641305604 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:58:07.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.362+0000 7fcce5ffb640 1 -- 172.21.15.1:0/1316700996 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 25) ==== 138065+0+0 (secure 0 0 0) 0x7fccdc010ac0 con 0x7fcce8101dd0 2024-09-11T21:58:07.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.362+0000 7fcced239640 1 --2- 172.21.15.1:0/641305604 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562654d25b40 0x562654d24090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:07.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.362+0000 7fcce7fff640 1 -- 172.21.15.1:0/641305604 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562654d25b40 msgr2=0x562654d24090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:07.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.362+0000 7fcce7fff640 1 --2- 172.21.15.1:0/641305604 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562654d25b40 0x562654d24090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:07.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.362+0000 7fcce7fff640 1 -- 172.21.15.1:0/641305604 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562654d245d0 msgr2=0x562654d24a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:07.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.362+0000 7fcce7fff640 1 --2- 172.21.15.1:0/641305604 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562654d245d0 0x562654d24a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:07.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.363+0000 7fcce7fff640 1 -- 172.21.15.1:0/641305604 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562654c163a0 con 0x562654d2f370 2024-09-11T21:58:07.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.363+0000 7fcce7fff640 1 --2- 172.21.15.1:0/641305604 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562654d2f370 0x562654d31760 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fccd4009900 tx=0x7fccd4005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b417045355e0c08e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:07.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.363+0000 7fccd2ffd640 1 -- 172.21.15.1:0/641305604 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd4035040 con 0x562654d2f370 2024-09-11T21:58:07.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.363+0000 7fccd2ffd640 1 -- 172.21.15.1:0/641305604 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccd4004030 con 0x562654d2f370 2024-09-11T21:58:07.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.363+0000 7fccd2ffd640 1 -- 172.21.15.1:0/641305604 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd4005590 con 0x562654d2f370 2024-09-11T21:58:07.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.364+0000 7fcce5ffb640 1 --2- 172.21.15.1:0/1316700996 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fccd80a2e10 0x7fccd80a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:07.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.364+0000 7fcced239640 1 --2- 172.21.15.1:0/1316700996 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fccd80a2e10 0x7fccd80a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:07.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.364+0000 7fcce5ffb640 1 -- 172.21.15.1:0/1316700996 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7fccdc0cfaa0 con 0x7fcce8101dd0 2024-09-11T21:58:07.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.364+0000 7fccf1075740 1 -- 172.21.15.1:0/641305604 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562654d2f370 msgr2=0x562654d31760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:07.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.364+0000 7fccf1075740 1 --2- 172.21.15.1:0/641305604 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562654d2f370 0x562654d31760 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fccd4009900 tx=0x7fccd4005be0 comp rx=0 tx=0).stop 2024-09-11T21:58:07.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.364+0000 7fccf1075740 1 -- 172.21.15.1:0/641305604 shutdown_connections 2024-09-11T21:58:07.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.364+0000 7fccf1075740 1 --2- 172.21.15.1:0/641305604 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562654d2f370 0x562654d31760 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:07.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.364+0000 7fccf1075740 1 --2- 172.21.15.1:0/641305604 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562654d245d0 0x562654d24a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:07.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.364+0000 7fccf1075740 1 --2- 172.21.15.1:0/641305604 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562654d25b40 0x562654d24090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:07.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.364+0000 7fccf1075740 1 -- 172.21.15.1:0/641305604 >> 172.21.15.1:0/641305604 conn(0x562654ced310 msgr2=0x562654d20c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:07.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.364+0000 7fcced239640 1 --2- 172.21.15.1:0/1316700996 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fccd80a2e10 0x7fccd80a52d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fcce0005fd0 tx=0x7fcce0009500 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:07.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.364+0000 7fccf1075740 1 -- 172.21.15.1:0/641305604 shutdown_connections 2024-09-11T21:58:07.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.366+0000 7fccf1075740 1 -- 172.21.15.1:0/641305604 wait complete. 2024-09-11T21:58:07.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.366+0000 7fccf1075740 1 -- start start 2024-09-11T21:58:07.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.366+0000 7fccf1075740 1 Processor -- start 2024-09-11T21:58:07.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.366+0000 7fccf1075740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562654d245d0 0x562654d2ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:07.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.366+0000 7fccf1075740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562654d25b40 0x562654d2c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:07.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.366+0000 7fcce7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562654d25b40 0x562654d2c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:07.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.366+0000 7fcce7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562654d25b40 0x562654d2c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42730/0 (socket says 172.21.15.1:42730) 2024-09-11T21:58:07.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.366+0000 7fcce7fff640 1 -- 172.21.15.1:0/4871483 learned_addr learned my addr 172.21.15.1:0/4871483 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:58:07.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.366+0000 7fccf1075740 1 --2- 172.21.15.1:0/4871483 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562654d2f370 0x562654d2c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:07.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.366+0000 7fccf1075740 1 -- 172.21.15.1:0/4871483 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x562654d31fb0 con 0x562654d25b40 2024-09-11T21:58:07.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.366+0000 7fccf1075740 1 -- 172.21.15.1:0/4871483 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x562654c45060 con 0x562654d245d0 2024-09-11T21:58:07.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.366+0000 7fccf1075740 1 -- 172.21.15.1:0/4871483 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x562654d32130 con 0x562654d2f370 2024-09-11T21:58:07.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.366+0000 7fcceca38640 1 --2- 172.21.15.1:0/4871483 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562654d245d0 0x562654d2ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:07.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.366+0000 7fccf1075740 10 client.0 _ll_register_callbacks cb 0x5626549988c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:58:07.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.366+0000 7fcceca38640 1 -- 172.21.15.1:0/4871483 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562654d2f370 msgr2=0x562654d2c7d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T21:58:07.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.366+0000 7fcceca38640 1 --2- 172.21.15.1:0/4871483 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562654d2f370 0x562654d2c7d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:07.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.366+0000 7fcceca38640 1 -- 172.21.15.1:0/4871483 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562654d25b40 msgr2=0x562654d2c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:07.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.366+0000 7fcceca38640 1 --2- 172.21.15.1:0/4871483 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562654d25b40 0x562654d2c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:07.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.366+0000 7fcceca38640 1 -- 172.21.15.1:0/4871483 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626549fb3c0 con 0x562654d245d0 2024-09-11T21:58:07.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.366+0000 7fcce7fff640 1 --2- 172.21.15.1:0/4871483 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562654d25b40 0x562654d2c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T21:58:07.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.368+0000 7fcceca38640 1 --2- 172.21.15.1:0/4871483 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562654d245d0 0x562654d2ede0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fccdc0cd070 tx=0x7fccdc09be20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=726561807234ae3b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:07.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.368+0000 7fccc7fff640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-11T21:58:07.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.368+0000 7fccc7fff640 1 -- 172.21.15.1:0/4871483 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccdc09b270 con 0x562654d245d0 2024-09-11T21:58:07.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.368+0000 7fccc7fff640 1 -- 172.21.15.1:0/4871483 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7fccdc09b410 con 0x562654d245d0 2024-09-11T21:58:07.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.368+0000 7fccf1075740 1 -- 172.21.15.1:0/4871483 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fccd4004030 con 0x562654d245d0 2024-09-11T21:58:07.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.368+0000 7fccc7fff640 1 -- 172.21.15.1:0/4871483 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fccdc010430 con 0x562654d245d0 2024-09-11T21:58:07.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.368+0000 7fccf1075740 10 client.25298 fetch_fsmap learned FSMap version 28 2024-09-11T21:58:07.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.368+0000 7fccf1075740 1 -- 172.21.15.1:0/4871483 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562654d3dae0 con 0x562654d245d0 2024-09-11T21:58:07.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.368+0000 7fccc7fff640 1 -- 172.21.15.1:0/4871483 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccd4004030 con 0x562654d245d0 2024-09-11T21:58:07.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.368+0000 7fccc7fff640 1 -- 172.21.15.1:0/4871483 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fccdc018a90 con 0x562654d245d0 2024-09-11T21:58:07.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.370+0000 7fccf1075740 10 client.25298 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T21:58:07.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.370+0000 7fccf1075740 10 client.25298 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T21:58:07.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.370+0000 7fccf1075740 20 client.25298 populate_metadata read hostname 'smithi001' 2024-09-11T21:58:07.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.370+0000 7fccf1075740 1 --2- 172.21.15.1:0/4871483 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x562654d3efa0 0x562654d41390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:07.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.370+0000 7fccf1075740 4 client.25298 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T21:58:07.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.370+0000 7fccf1075740 1 -- 172.21.15.1:0/4871483 --> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562654c97860 con 0x562654d3efa0 2024-09-11T21:58:07.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.370+0000 7fcce7fff640 1 --2- 172.21.15.1:0/4871483 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x562654d3efa0 0x562654d41390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:07.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.370+0000 7fccc7fff640 10 client.25298 ms_handle_connect on v2:172.21.15.161:6838/455365383 2024-09-11T21:58:07.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.370+0000 7fcce7fff640 1 --2- 172.21.15.1:0/4871483 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x562654d3efa0 0x562654d41390 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9f29952f495edd09 server_cookie=77b01220322da1e9 in_seq=0 out_seq=0 2024-09-11T21:58:07.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.371+0000 7fccc7fff640 1 -- 172.21.15.1:0/4871483 <== mds.1 v2:172.21.15.161:6838/455365383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562654c97860 con 0x562654d3efa0 2024-09-11T21:58:07.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.371+0000 7fccc7fff640 10 client.25298 handle_command_reply: tid=0 2024-09-11T21:58:07.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.371+0000 7fccf1075740 1 client.25298 shutdown 2024-09-11T21:58:07.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.371+0000 7fccf1075740 1 -- 172.21.15.1:0/4871483 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562654d245d0 msgr2=0x562654d2ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:07.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.371+0000 7fccf1075740 1 --2- 172.21.15.1:0/4871483 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562654d245d0 0x562654d2ede0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fccdc0cd070 tx=0x7fccdc09be20 comp rx=0 tx=0).stop 2024-09-11T21:58:07.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.371+0000 7fccf1075740 1 -- 172.21.15.1:0/4871483 shutdown_connections 2024-09-11T21:58:07.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.371+0000 7fccf1075740 1 --2- 172.21.15.1:0/4871483 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x562654d3efa0 0x562654d41390 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:07.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.371+0000 7fccf1075740 1 --2- 172.21.15.1:0/4871483 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562654d2f370 0x562654d2c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:07.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.371+0000 7fccf1075740 1 --2- 172.21.15.1:0/4871483 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562654d25b40 0x562654d2c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:07.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.371+0000 7fccf1075740 1 --2- 172.21.15.1:0/4871483 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562654d245d0 0x562654d2ede0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:07.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.371+0000 7fccf1075740 1 -- 172.21.15.1:0/4871483 >> 172.21.15.1:0/4871483 conn(0x562654ced310 msgr2=0x562654d26960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:07.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.371+0000 7fccc7fff640 0 client.25298 ms_handle_reset on v2:172.21.15.161:6838/455365383 2024-09-11T21:58:07.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.371+0000 7fccf1075740 1 -- 172.21.15.1:0/4871483 shutdown_connections 2024-09-11T21:58:07.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.371+0000 7fccf1075740 1 -- 172.21.15.1:0/4871483 wait complete. 2024-09-11T21:58:07.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.371+0000 7fccf1075740 20 client.25298 trim_cache size 0 max 16384 2024-09-11T21:58:07.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.385+0000 7fccf1075740 1 Processor -- start 2024-09-11T21:58:07.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.385+0000 7fccf1075740 1 -- start start 2024-09-11T21:58:07.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.385+0000 7fccf1075740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562654d245d0 0x562654d3b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:07.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.385+0000 7fccf1075740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562654d25b40 0x562654d27120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:07.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.385+0000 7fccf1075740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562654d2f370 0x562654d27660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:07.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.385+0000 7fccf1075740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcce81ab8a0 con 0x562654d2f370 2024-09-11T21:58:07.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.385+0000 7fccf1075740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcce81ad3e0 con 0x562654d245d0 2024-09-11T21:58:07.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.385+0000 7fccf1075740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcce81abf70 con 0x562654d25b40 2024-09-11T21:58:07.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.386+0000 7fcce7fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562654d25b40 0x562654d27120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:07.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.386+0000 7fcce7fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562654d25b40 0x562654d27120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:41464/0 (socket says 172.21.15.1:41464) 2024-09-11T21:58:07.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.386+0000 7fcce7fff640 1 -- 172.21.15.1:0/1987988762 learned_addr learned my addr 172.21.15.1:0/1987988762 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:58:07.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.386+0000 7fcced239640 1 --2- 172.21.15.1:0/1987988762 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562654d2f370 0x562654d27660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:07.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.386+0000 7fcceca38640 1 --2- 172.21.15.1:0/1987988762 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562654d245d0 0x562654d3b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:07.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.386+0000 7fcce7fff640 1 -- 172.21.15.1:0/1987988762 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562654d245d0 msgr2=0x562654d3b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:07.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.386+0000 7fcce7fff640 1 --2- 172.21.15.1:0/1987988762 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562654d245d0 0x562654d3b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:07.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.386+0000 7fcce7fff640 1 -- 172.21.15.1:0/1987988762 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562654d2f370 msgr2=0x562654d27660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:07.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.386+0000 7fcce7fff640 1 --2- 172.21.15.1:0/1987988762 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562654d2f370 0x562654d27660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:07.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.386+0000 7fcce7fff640 1 -- 172.21.15.1:0/1987988762 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562654d3b790 con 0x562654d25b40 2024-09-11T21:58:07.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.386+0000 7fcced239640 1 --2- 172.21.15.1:0/1987988762 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562654d2f370 0x562654d27660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T21:58:07.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.386+0000 7fcce7fff640 1 --2- 172.21.15.1:0/1987988762 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562654d25b40 0x562654d27120 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fccd4005240 tx=0x7fccd4005130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15266a2ac56f08c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:07.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.387+0000 7fccd27fc640 1 -- 172.21.15.1:0/1987988762 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd4035040 con 0x562654d25b40 2024-09-11T21:58:07.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.387+0000 7fccd27fc640 1 -- 172.21.15.1:0/1987988762 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccd4056070 con 0x562654d25b40 2024-09-11T21:58:07.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.387+0000 7fccd27fc640 1 -- 172.21.15.1:0/1987988762 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd4004ad0 con 0x562654d25b40 2024-09-11T21:58:07.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.387+0000 7fccf1075740 1 -- 172.21.15.1:0/1987988762 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562654d25b40 msgr2=0x562654d27120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:07.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.387+0000 7fccf1075740 1 --2- 172.21.15.1:0/1987988762 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562654d25b40 0x562654d27120 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fccd4005240 tx=0x7fccd4005130 comp rx=0 tx=0).stop 2024-09-11T21:58:07.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.387+0000 7fccf1075740 1 -- 172.21.15.1:0/1987988762 shutdown_connections 2024-09-11T21:58:07.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.387+0000 7fccf1075740 1 --2- 172.21.15.1:0/1987988762 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562654d2f370 0x562654d27660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:07.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.387+0000 7fccf1075740 1 --2- 172.21.15.1:0/1987988762 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562654d25b40 0x562654d27120 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:07.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.387+0000 7fccf1075740 1 --2- 172.21.15.1:0/1987988762 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562654d245d0 0x562654d3b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:07.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.387+0000 7fccf1075740 1 -- 172.21.15.1:0/1987988762 >> 172.21.15.1:0/1987988762 conn(0x562654ced310 msgr2=0x562654d2b180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:07.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.388+0000 7fccf1075740 1 -- 172.21.15.1:0/1987988762 shutdown_connections 2024-09-11T21:58:07.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.388+0000 7fccf1075740 1 -- 172.21.15.1:0/1987988762 wait complete. 2024-09-11T21:58:07.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.388+0000 7fccf1075740 1 -- start start 2024-09-11T21:58:07.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.388+0000 7fccf1075740 1 Processor -- start 2024-09-11T21:58:07.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.389+0000 7fccf1075740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562654d245d0 0x562654d32920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:07.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.389+0000 7fccf1075740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562654d25b40 0x562654d32e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:07.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.389+0000 7fccf1075740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562654d2f370 0x562654da7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:07.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.389+0000 7fccf1075740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcce81acd10 con 0x562654d25b40 2024-09-11T21:58:07.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.389+0000 7fccf1075740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcce81ac640 con 0x562654d2f370 2024-09-11T21:58:07.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.389+0000 7fcceca38640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562654d245d0 0x562654d32920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:07.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.389+0000 7fccf1075740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcce81a9880 con 0x562654d245d0 2024-09-11T21:58:07.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.389+0000 7fcce7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562654d25b40 0x562654d32e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:07.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.389+0000 7fcceca38640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562654d245d0 0x562654d32920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:41474/0 (socket says 172.21.15.1:41474) 2024-09-11T21:58:07.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.389+0000 7fcceca38640 1 -- 172.21.15.1:0/442097201 learned_addr learned my addr 172.21.15.1:0/442097201 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:58:07.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.389+0000 7fccf1075740 10 client.0 _ll_register_callbacks cb 0x7fccdc00cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:58:07.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.390+0000 7fcce7fff640 1 -- 172.21.15.1:0/442097201 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562654d245d0 msgr2=0x562654d32920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:07.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.390+0000 7fcce7fff640 1 --2- 172.21.15.1:0/442097201 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562654d245d0 0x562654d32920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:07.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.390+0000 7fcce7fff640 1 -- 172.21.15.1:0/442097201 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562654d2f370 msgr2=0x562654da7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:07.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.390+0000 7fcce7fff640 1 --2- 172.21.15.1:0/442097201 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562654d2f370 0x562654da7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:07.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.390+0000 7fcce7fff640 1 -- 172.21.15.1:0/442097201 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562654da8210 con 0x562654d25b40 2024-09-11T21:58:07.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.390+0000 7fcce7fff640 1 --2- 172.21.15.1:0/442097201 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562654d25b40 0x562654d32e60 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x562654d2d270 tx=0x7fccd402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=801286248001c2f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:07.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.390+0000 7fccc7fff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T21:58:07.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.390+0000 7fccc7fff640 1 -- 172.21.15.1:0/442097201 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd4035030 con 0x562654d25b40 2024-09-11T21:58:07.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.390+0000 7fccc7fff640 1 -- 172.21.15.1:0/442097201 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7fccd40317c0 con 0x562654d25b40 2024-09-11T21:58:07.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.390+0000 7fccf1075740 1 -- 172.21.15.1:0/442097201 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fccd4056070 con 0x562654d25b40 2024-09-11T21:58:07.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.390+0000 7fccc7fff640 1 -- 172.21.15.1:0/442097201 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fccd4009520 con 0x562654d25b40 2024-09-11T21:58:07.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.390+0000 7fccf1075740 10 client.15705 fetch_fsmap learned FSMap version 28 2024-09-11T21:58:07.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.390+0000 7fccf1075740 1 -- 172.21.15.1:0/442097201 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562654da94f0 con 0x562654d25b40 2024-09-11T21:58:07.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.390+0000 7fccc7fff640 1 -- 172.21.15.1:0/442097201 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccd4004850 con 0x562654d25b40 2024-09-11T21:58:07.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.390+0000 7fccc7fff640 1 -- 172.21.15.1:0/442097201 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fccd4043070 con 0x562654d25b40 2024-09-11T21:58:07.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.390+0000 7fccf1075740 10 client.15705 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T21:58:07.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.390+0000 7fccf1075740 10 client.15705 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T21:58:07.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.390+0000 7fccf1075740 20 client.15705 populate_metadata read hostname 'smithi001' 2024-09-11T21:58:07.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.390+0000 7fccf1075740 1 --2- 172.21.15.1:0/442097201 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x562654da9770 0x562654dabb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:07.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.392+0000 7fccf1075740 4 client.15705 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T21:58:07.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.392+0000 7fccf1075740 1 -- 172.21.15.1:0/442097201 --> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fccd4056070 con 0x562654da9770 2024-09-11T21:58:07.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.392+0000 7fcceca38640 1 --2- 172.21.15.1:0/442097201 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x562654da9770 0x562654dabb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:07.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.392+0000 7fcceca38640 1 --2- 172.21.15.1:0/442097201 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x562654da9770 0x562654dabb60 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2a5c09f357b329a3 server_cookie=a8d74161f4b0c6b5 in_seq=0 out_seq=0 2024-09-11T21:58:07.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.392+0000 7fccc7fff640 10 client.15705 ms_handle_connect on v2:172.21.15.161:6838/455365383 2024-09-11T21:58:07.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.392+0000 7fccc7fff640 1 -- 172.21.15.1:0/442097201 <== mds.1 v2:172.21.15.161:6838/455365383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fccd4056070 con 0x562654da9770 2024-09-11T21:58:07.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.392+0000 7fccc7fff640 10 client.15705 handle_command_reply: tid=0 2024-09-11T21:58:07.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.392+0000 7fccf1075740 1 client.15705 shutdown 2024-09-11T21:58:07.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.392+0000 7fccf1075740 1 -- 172.21.15.1:0/442097201 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562654d25b40 msgr2=0x562654d32e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:07.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.392+0000 7fccf1075740 1 --2- 172.21.15.1:0/442097201 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562654d25b40 0x562654d32e60 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x562654d2d270 tx=0x7fccd402a410 comp rx=0 tx=0).stop 2024-09-11T21:58:07.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.392+0000 7fccf1075740 1 -- 172.21.15.1:0/442097201 shutdown_connections 2024-09-11T21:58:07.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.392+0000 7fccf1075740 1 --2- 172.21.15.1:0/442097201 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x562654da9770 0x562654dabb60 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:07.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.392+0000 7fccf1075740 1 --2- 172.21.15.1:0/442097201 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562654d2f370 0x562654da7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:07.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.392+0000 7fccc7fff640 0 client.15705 ms_handle_reset on v2:172.21.15.161:6838/455365383 2024-09-11T21:58:07.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.392+0000 7fccf1075740 1 --2- 172.21.15.1:0/442097201 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562654d25b40 0x562654d32e60 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:07.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.392+0000 7fccf1075740 1 --2- 172.21.15.1:0/442097201 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562654d245d0 0x562654d32920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:07.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.392+0000 7fccf1075740 1 -- 172.21.15.1:0/442097201 >> 172.21.15.1:0/442097201 conn(0x562654ced310 msgr2=0x562654d1f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:07.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.394+0000 7fccf1075740 1 -- 172.21.15.1:0/442097201 shutdown_connections 2024-09-11T21:58:07.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.394+0000 7fccf1075740 1 -- 172.21.15.1:0/442097201 wait complete. 2024-09-11T21:58:07.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.394+0000 7fccf1075740 20 client.15705 trim_cache size 0 max 16384 2024-09-11T21:58:07.399 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-11T21:58:07.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.394+0000 7fcceec9c640 1 -- 172.21.15.1:0/1316700996 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fccd80a2e10 msgr2=0x7fccd80a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:07.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.394+0000 7fcceec9c640 1 --2- 172.21.15.1:0/1316700996 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fccd80a2e10 0x7fccd80a52d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fcce0005fd0 tx=0x7fcce0009500 comp rx=0 tx=0).stop 2024-09-11T21:58:07.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.395+0000 7fcceec9c640 1 -- 172.21.15.1:0/1316700996 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcce8101dd0 msgr2=0x7fcce8198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:07.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.395+0000 7fcceec9c640 1 --2- 172.21.15.1:0/1316700996 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcce8101dd0 0x7fcce8198440 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fccdc00e9f0 tx=0x7fccdc00eeb0 comp rx=0 tx=0).stop 2024-09-11T21:58:07.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.395+0000 7fcceec9c640 1 -- 172.21.15.1:0/1316700996 shutdown_connections 2024-09-11T21:58:07.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.395+0000 7fcceec9c640 1 --2- 172.21.15.1:0/1316700996 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fccd80a2e10 0x7fccd80a52d0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:07.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.395+0000 7fcceec9c640 1 --2- 172.21.15.1:0/1316700996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcce8106f00 0x7fcce819cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:07.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.395+0000 7fcceec9c640 1 --2- 172.21.15.1:0/1316700996 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcce8102780 0x7fcce8198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:07.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.395+0000 7fcceec9c640 1 --2- 172.21.15.1:0/1316700996 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcce8101dd0 0x7fcce8198440 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:07.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.395+0000 7fcceec9c640 1 -- 172.21.15.1:0/1316700996 >> 172.21.15.1:0/1316700996 conn(0x7fcce80fd8b0 msgr2=0x7fcce80fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:07.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.395+0000 7fcceec9c640 1 -- 172.21.15.1:0/1316700996 shutdown_connections 2024-09-11T21:58:07.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:07.395+0000 7fcceec9c640 1 -- 172.21.15.1:0/1316700996 wait complete. 2024-09-11T21:58:08.096 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T21:58:08.865 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'f7fcf15b', quiesce took 10.2 seconds. Will release after: 14.3 2024-09-11T21:58:08.938 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:58:09.078 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:58:09 smithi001 ceph-mon[30935]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-11T21:58:09.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:58:09 smithi053 ceph-mon[34908]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-11T21:58:09.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:58:09 smithi161 ceph-mon[35615]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-11T21:58:09.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.469+0000 7f583398e640 1 -- 172.21.15.1:0/1076519611 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f582c102ef0 msgr2=0x7f582c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:09.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.469+0000 7f583398e640 1 --2- 172.21.15.1:0/1076519611 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f582c102ef0 0x7f582c10da40 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f5824009ca0 tx=0x7f582402a950 comp rx=0 tx=0).stop 2024-09-11T21:58:09.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.469+0000 7f583398e640 1 -- 172.21.15.1:0/1076519611 shutdown_connections 2024-09-11T21:58:09.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.469+0000 7f583398e640 1 --2- 172.21.15.1:0/1076519611 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f582c102ef0 0x7f582c10da40 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:09.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.469+0000 7f583398e640 1 --2- 172.21.15.1:0/1076519611 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f582c102570 0x7f582c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:09.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.469+0000 7f583398e640 1 --2- 172.21.15.1:0/1076519611 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f582c100e20 0x7f582c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:09.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.469+0000 7f583398e640 1 -- 172.21.15.1:0/1076519611 >> 172.21.15.1:0/1076519611 conn(0x7f582c0fc7d0 msgr2=0x7f582c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:09.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.470+0000 7f583398e640 1 -- 172.21.15.1:0/1076519611 shutdown_connections 2024-09-11T21:58:09.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.470+0000 7f583398e640 1 -- 172.21.15.1:0/1076519611 wait complete. 2024-09-11T21:58:09.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.471+0000 7f583398e640 1 Processor -- start 2024-09-11T21:58:09.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.471+0000 7f583398e640 1 -- start start 2024-09-11T21:58:09.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.471+0000 7f583398e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f582c100e20 0x7f582c199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:09.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.471+0000 7f583398e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f582c102570 0x7f582c1998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:09.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.471+0000 7f583398e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f582c102ef0 0x7f582c1a43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:09.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.471+0000 7f583398e640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f582c110f80 con 0x7f582c100e20 2024-09-11T21:58:09.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.471+0000 7f583172a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f582c100e20 0x7f582c199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:09.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.471+0000 7f583172a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f582c100e20 0x7f582c199370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42758/0 (socket says 172.21.15.1:42758) 2024-09-11T21:58:09.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.471+0000 7f583398e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f582c110e00 con 0x7f582c102570 2024-09-11T21:58:09.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.471+0000 7f583172a640 1 -- 172.21.15.1:0/3786704119 learned_addr learned my addr 172.21.15.1:0/3786704119 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:58:09.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.471+0000 7f583398e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f582c111100 con 0x7f582c102ef0 2024-09-11T21:58:09.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.472+0000 7f5831f2b640 1 --2- 172.21.15.1:0/3786704119 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f582c102ef0 0x7f582c1a43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:09.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.472+0000 7f5830f29640 1 --2- 172.21.15.1:0/3786704119 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f582c102570 0x7f582c1998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:09.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.472+0000 7f583172a640 1 -- 172.21.15.1:0/3786704119 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f582c102ef0 msgr2=0x7f582c1a43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:09.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.472+0000 7f583172a640 1 --2- 172.21.15.1:0/3786704119 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f582c102ef0 0x7f582c1a43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:09.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.472+0000 7f583172a640 1 -- 172.21.15.1:0/3786704119 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f582c102570 msgr2=0x7f582c1998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:09.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.472+0000 7f583172a640 1 --2- 172.21.15.1:0/3786704119 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f582c102570 0x7f582c1998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:09.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.472+0000 7f583172a640 1 -- 172.21.15.1:0/3786704119 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f582c1a4930 con 0x7f582c100e20 2024-09-11T21:58:09.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.472+0000 7f583172a640 1 --2- 172.21.15.1:0/3786704119 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f582c100e20 0x7f582c199370 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f5828009f40 tx=0x7f582800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:09.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.472+0000 7f58227fc640 1 -- 172.21.15.1:0/3786704119 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582800cd80 con 0x7f582c100e20 2024-09-11T21:58:09.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.472+0000 7f58227fc640 1 -- 172.21.15.1:0/3786704119 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5828005500 con 0x7f582c100e20 2024-09-11T21:58:09.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.472+0000 7f58227fc640 1 -- 172.21.15.1:0/3786704119 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5828004a00 con 0x7f582c100e20 2024-09-11T21:58:09.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.473+0000 7f583398e640 1 -- 172.21.15.1:0/3786704119 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f582c197160 con 0x7f582c100e20 2024-09-11T21:58:09.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.473+0000 7f583398e640 1 -- 172.21.15.1:0/3786704119 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f582c197590 con 0x7f582c100e20 2024-09-11T21:58:09.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.476+0000 7f5835d67740 1 Processor -- start 2024-09-11T21:58:09.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.476+0000 7f5835d67740 1 -- start start 2024-09-11T21:58:09.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.476+0000 7f58227fc640 1 -- 172.21.15.1:0/3786704119 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) ==== 138065+0+0 (secure 0 0 0) 0x7f5828004ba0 con 0x7f582c100e20 2024-09-11T21:58:09.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.477+0000 7f5835d67740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fac13b0b40 0x55fac13af090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:09.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.477+0000 7f5835d67740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fac13af5d0 0x55fac13afa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:09.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.477+0000 7f5835d67740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fac13c2b80 0x55fac13c4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:09.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.477+0000 7f5835d67740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55fac13152f0 con 0x55fac13c2b80 2024-09-11T21:58:09.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.477+0000 7f5835d67740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55fac12bccf0 con 0x55fac13af5d0 2024-09-11T21:58:09.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.477+0000 7f5835d67740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55fac13112b0 con 0x55fac13b0b40 2024-09-11T21:58:09.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.477+0000 7f5830f29640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fac13c2b80 0x55fac13c4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:09.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.477+0000 7f583172a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fac13af5d0 0x55fac13afa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:09.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.477+0000 7f5830f29640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fac13c2b80 0x55fac13c4f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42774/0 (socket says 172.21.15.1:42774) 2024-09-11T21:58:09.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.477+0000 7f5830f29640 1 -- 172.21.15.1:0/2767762617 learned_addr learned my addr 172.21.15.1:0/2767762617 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:58:09.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.478+0000 7f5831f2b640 1 --2- 172.21.15.1:0/2767762617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fac13b0b40 0x55fac13af090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:09.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.478+0000 7f58227fc640 1 --2- 172.21.15.1:0/3786704119 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f58180a2f10 0x7f58180a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:09.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.478+0000 7f58227fc640 1 -- 172.21.15.1:0/3786704119 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7f58280d39c0 con 0x7f582c100e20 2024-09-11T21:58:09.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.478+0000 7f583172a640 1 -- 172.21.15.1:0/2767762617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fac13b0b40 msgr2=0x55fac13af090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:09.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.478+0000 7f583172a640 1 --2- 172.21.15.1:0/2767762617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fac13b0b40 0x55fac13af090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:09.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.478+0000 7f583172a640 1 -- 172.21.15.1:0/2767762617 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fac13c2b80 msgr2=0x55fac13c4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:09.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.478+0000 7f583172a640 1 --2- 172.21.15.1:0/2767762617 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fac13c2b80 0x55fac13c4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:09.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.478+0000 7f583172a640 1 -- 172.21.15.1:0/2767762617 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fac12a13a0 con 0x55fac13af5d0 2024-09-11T21:58:09.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.478+0000 7f5830f29640 1 --2- 172.21.15.1:0/2767762617 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fac13c2b80 0x55fac13c4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T21:58:09.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.479+0000 7f5831f2b640 1 --2- 172.21.15.1:0/3786704119 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f58180a2f10 0x7f58180a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:09.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.479+0000 7f583172a640 1 --2- 172.21.15.1:0/2767762617 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fac13af5d0 0x55fac13afa10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f58280d1040 tx=0x7f5828009720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9d8ed93fa4ad8ba server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:09.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.479+0000 7f58177fe640 1 -- 172.21.15.1:0/2767762617 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582809f6a0 con 0x55fac13af5d0 2024-09-11T21:58:09.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.479+0000 7f58177fe640 1 -- 172.21.15.1:0/2767762617 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58280a0440 con 0x55fac13af5d0 2024-09-11T21:58:09.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.479+0000 7f58177fe640 1 -- 172.21.15.1:0/2767762617 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58280056b0 con 0x55fac13af5d0 2024-09-11T21:58:09.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.480+0000 7f5835d67740 1 -- 172.21.15.1:0/2767762617 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fac13af5d0 msgr2=0x55fac13afa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:09.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.480+0000 7f5835d67740 1 --2- 172.21.15.1:0/2767762617 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fac13af5d0 0x55fac13afa10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f58280d1040 tx=0x7f5828009720 comp rx=0 tx=0).stop 2024-09-11T21:58:09.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.480+0000 7f5831f2b640 1 --2- 172.21.15.1:0/3786704119 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f58180a2f10 0x7f58180a53d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5824005fe0 tx=0x7f582402a950 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:09.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.480+0000 7f5835d67740 1 -- 172.21.15.1:0/2767762617 shutdown_connections 2024-09-11T21:58:09.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.480+0000 7f5835d67740 1 --2- 172.21.15.1:0/2767762617 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fac13c2b80 0x55fac13c4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:09.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.480+0000 7f5835d67740 1 --2- 172.21.15.1:0/2767762617 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fac13af5d0 0x55fac13afa10 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:09.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.480+0000 7f5835d67740 1 --2- 172.21.15.1:0/2767762617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fac13b0b40 0x55fac13af090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:09.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.480+0000 7f5835d67740 1 -- 172.21.15.1:0/2767762617 >> 172.21.15.1:0/2767762617 conn(0x55fac1378310 msgr2=0x55fac13abc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:09.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.480+0000 7f5835d67740 1 -- 172.21.15.1:0/2767762617 shutdown_connections 2024-09-11T21:58:09.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.480+0000 7f5835d67740 1 -- 172.21.15.1:0/2767762617 wait complete. 2024-09-11T21:58:09.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.481+0000 7f5835d67740 1 -- start start 2024-09-11T21:58:09.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.481+0000 7f5835d67740 1 Processor -- start 2024-09-11T21:58:09.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.481+0000 7f5835d67740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fac13af5d0 0x55fac13c25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:09.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.482+0000 7f5835d67740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fac13b0b40 0x55fac13bf860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:09.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.482+0000 7f5835d67740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fac13c2b80 0x55fac13bfff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:09.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.482+0000 7f5835d67740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55fac13c5820 con 0x55fac13b0b40 2024-09-11T21:58:09.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.482+0000 7f5835d67740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55fac12d0060 con 0x55fac13c2b80 2024-09-11T21:58:09.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.482+0000 7f5835d67740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55fac13c59a0 con 0x55fac13af5d0 2024-09-11T21:58:09.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.482+0000 7f5835d67740 10 client.0 _ll_register_callbacks cb 0x55fac10238c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:58:09.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.482+0000 7f583172a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fac13af5d0 0x55fac13c25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:09.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.482+0000 7f583172a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fac13af5d0 0x55fac13c25b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:41514/0 (socket says 172.21.15.1:41514) 2024-09-11T21:58:09.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.482+0000 7f583172a640 1 -- 172.21.15.1:0/1164784669 learned_addr learned my addr 172.21.15.1:0/1164784669 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:58:09.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.482+0000 7f5831f2b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fac13c2b80 0x55fac13bfff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:09.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.482+0000 7f5830f29640 1 --2- 172.21.15.1:0/1164784669 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fac13b0b40 0x55fac13bf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:09.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.482+0000 7f5830f29640 1 -- 172.21.15.1:0/1164784669 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fac13af5d0 msgr2=0x55fac13c25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:09.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.482+0000 7f5830f29640 1 --2- 172.21.15.1:0/1164784669 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fac13af5d0 0x55fac13c25b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:09.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.482+0000 7f5830f29640 1 -- 172.21.15.1:0/1164784669 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fac13c2b80 msgr2=0x55fac13bfff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:09.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.482+0000 7f5830f29640 1 --2- 172.21.15.1:0/1164784669 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fac13c2b80 0x55fac13bfff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:09.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.482+0000 7f5830f29640 1 -- 172.21.15.1:0/1164784669 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fac10863c0 con 0x55fac13b0b40 2024-09-11T21:58:09.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.482+0000 7f5830f29640 1 --2- 172.21.15.1:0/1164784669 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fac13b0b40 0x55fac13bf860 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f581c005f60 tx=0x7f581c012b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2592247ba4ea6809 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:09.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.482+0000 7f5814ff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T21:58:09.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.483+0000 7f5814ff9640 1 -- 172.21.15.1:0/1164784669 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f581c011c70 con 0x55fac13b0b40 2024-09-11T21:58:09.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.483+0000 7f5814ff9640 1 -- 172.21.15.1:0/1164784669 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7f581c011e10 con 0x55fac13b0b40 2024-09-11T21:58:09.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.483+0000 7f5835d67740 1 -- 172.21.15.1:0/1164784669 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58280a0440 con 0x55fac13b0b40 2024-09-11T21:58:09.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.483+0000 7f5814ff9640 1 -- 172.21.15.1:0/1164784669 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f581c004020 con 0x55fac13b0b40 2024-09-11T21:58:09.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.483+0000 7f5835d67740 10 client.15717 fetch_fsmap learned FSMap version 28 2024-09-11T21:58:09.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.483+0000 7f5835d67740 1 -- 172.21.15.1:0/1164784669 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fac13d1360 con 0x55fac13b0b40 2024-09-11T21:58:09.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.484+0000 7f5814ff9640 1 -- 172.21.15.1:0/1164784669 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58280a0440 con 0x55fac13b0b40 2024-09-11T21:58:09.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.484+0000 7f5814ff9640 1 -- 172.21.15.1:0/1164784669 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f581c025480 con 0x55fac13b0b40 2024-09-11T21:58:09.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.484+0000 7f5835d67740 10 client.15717 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T21:58:09.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.484+0000 7f5835d67740 10 client.15717 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T21:58:09.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.484+0000 7f5835d67740 20 client.15717 populate_metadata read hostname 'smithi001' 2024-09-11T21:58:09.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.484+0000 7f5835d67740 1 --2- 172.21.15.1:0/1164784669 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55fac13d2770 0x55fac13d4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:09.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.484+0000 7f5835d67740 4 client.15717 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T21:58:09.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.484+0000 7f5835d67740 1 -- 172.21.15.1:0/1164784669 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fac1322860 con 0x55fac13d2770 2024-09-11T21:58:09.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.484+0000 7f583172a640 1 --2- 172.21.15.1:0/1164784669 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55fac13d2770 0x55fac13d4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:09.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.485+0000 7f5814ff9640 10 client.15717 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T21:58:09.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.485+0000 7f583172a640 1 --2- 172.21.15.1:0/1164784669 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55fac13d2770 0x55fac13d4b90 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29170805a9c3cf89 server_cookie=df895b94e7952dda in_seq=0 out_seq=0 2024-09-11T21:58:09.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.485+0000 7f5814ff9640 1 -- 172.21.15.1:0/1164784669 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fac1322860 con 0x55fac13d2770 2024-09-11T21:58:09.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.485+0000 7f5814ff9640 10 client.15717 handle_command_reply: tid=0 2024-09-11T21:58:09.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.485+0000 7f5835d67740 1 client.15717 shutdown 2024-09-11T21:58:09.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.486+0000 7f5835d67740 1 -- 172.21.15.1:0/1164784669 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fac13b0b40 msgr2=0x55fac13bf860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:09.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.486+0000 7f5835d67740 1 --2- 172.21.15.1:0/1164784669 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fac13b0b40 0x55fac13bf860 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f581c005f60 tx=0x7f581c012b60 comp rx=0 tx=0).stop 2024-09-11T21:58:09.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.486+0000 7f5835d67740 1 -- 172.21.15.1:0/1164784669 shutdown_connections 2024-09-11T21:58:09.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.486+0000 7f5835d67740 1 --2- 172.21.15.1:0/1164784669 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55fac13d2770 0x55fac13d4b90 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:09.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.486+0000 7f5814ff9640 0 client.15717 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T21:58:09.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.486+0000 7f5835d67740 1 --2- 172.21.15.1:0/1164784669 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fac13c2b80 0x55fac13bfff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:09.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.486+0000 7f5835d67740 1 --2- 172.21.15.1:0/1164784669 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fac13b0b40 0x55fac13bf860 secure :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f581c005f60 tx=0x7f581c012b60 comp rx=0 tx=0).stop 2024-09-11T21:58:09.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.486+0000 7f5835d67740 1 --2- 172.21.15.1:0/1164784669 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fac13af5d0 0x55fac13c25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:09.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.486+0000 7f5835d67740 1 -- 172.21.15.1:0/1164784669 >> 172.21.15.1:0/1164784669 conn(0x55fac1378310 msgr2=0x55fac13b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:09.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.486+0000 7f5835d67740 1 -- 172.21.15.1:0/1164784669 shutdown_connections 2024-09-11T21:58:09.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.486+0000 7f5835d67740 1 -- 172.21.15.1:0/1164784669 wait complete. 2024-09-11T21:58:09.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.486+0000 7f5835d67740 20 client.15717 trim_cache size 0 max 16384 2024-09-11T21:58:09.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.499+0000 7f5835d67740 1 Processor -- start 2024-09-11T21:58:09.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.499+0000 7f5835d67740 1 -- start start 2024-09-11T21:58:09.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.500+0000 7f5835d67740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fac13af5d0 0x55fac13ab4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:09.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.500+0000 7f5835d67740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fac13c2b80 0x55fac13aba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:09.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.500+0000 7f5835d67740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fac13be5d0 0x55fac13da440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:09.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.500+0000 7f5835d67740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f582c1ab780 con 0x55fac13af5d0 2024-09-11T21:58:09.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.500+0000 7f5835d67740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f582c1ad2c0 con 0x55fac13c2b80 2024-09-11T21:58:09.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.500+0000 7f5835d67740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f582c1abe50 con 0x55fac13be5d0 2024-09-11T21:58:09.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.500+0000 7f583172a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fac13af5d0 0x55fac13ab4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:09.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.500+0000 7f583172a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fac13af5d0 0x55fac13ab4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42782/0 (socket says 172.21.15.1:42782) 2024-09-11T21:58:09.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.500+0000 7f583172a640 1 -- 172.21.15.1:0/1280693726 learned_addr learned my addr 172.21.15.1:0/1280693726 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:58:09.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.500+0000 7f5830f29640 1 --2- 172.21.15.1:0/1280693726 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fac13c2b80 0x55fac13aba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:09.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.500+0000 7f5831f2b640 1 --2- 172.21.15.1:0/1280693726 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fac13be5d0 0x55fac13da440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:09.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.500+0000 7f583172a640 1 -- 172.21.15.1:0/1280693726 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fac13be5d0 msgr2=0x55fac13da440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:09.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.500+0000 7f583172a640 1 --2- 172.21.15.1:0/1280693726 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fac13be5d0 0x55fac13da440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:09.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.500+0000 7f583172a640 1 -- 172.21.15.1:0/1280693726 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fac13c2b80 msgr2=0x55fac13aba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:09.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.500+0000 7f583172a640 1 --2- 172.21.15.1:0/1280693726 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fac13c2b80 0x55fac13aba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:09.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.500+0000 7f583172a640 1 -- 172.21.15.1:0/1280693726 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fac13cefa0 con 0x55fac13af5d0 2024-09-11T21:58:09.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.501+0000 7f5831f2b640 1 --2- 172.21.15.1:0/1280693726 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fac13be5d0 0x55fac13da440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T21:58:09.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.501+0000 7f583172a640 1 --2- 172.21.15.1:0/1280693726 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fac13af5d0 0x55fac13ab4f0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f582809fcd0 tx=0x7f582809c110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5fa0b1dedfb4eb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:09.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.501+0000 7f5816ffd640 1 -- 172.21.15.1:0/1280693726 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582809f880 con 0x55fac13af5d0 2024-09-11T21:58:09.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.501+0000 7f5816ffd640 1 -- 172.21.15.1:0/1280693726 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58280a0440 con 0x55fac13af5d0 2024-09-11T21:58:09.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.501+0000 7f5816ffd640 1 -- 172.21.15.1:0/1280693726 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58280056b0 con 0x55fac13af5d0 2024-09-11T21:58:09.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.502+0000 7f5835d67740 1 -- 172.21.15.1:0/1280693726 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fac13af5d0 msgr2=0x55fac13ab4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:09.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.502+0000 7f5835d67740 1 --2- 172.21.15.1:0/1280693726 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fac13af5d0 0x55fac13ab4f0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f582809fcd0 tx=0x7f582809c110 comp rx=0 tx=0).stop 2024-09-11T21:58:09.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.502+0000 7f5835d67740 1 -- 172.21.15.1:0/1280693726 shutdown_connections 2024-09-11T21:58:09.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.502+0000 7f5835d67740 1 --2- 172.21.15.1:0/1280693726 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fac13be5d0 0x55fac13da440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:09.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.502+0000 7f5835d67740 1 --2- 172.21.15.1:0/1280693726 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fac13c2b80 0x55fac13aba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:09.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.502+0000 7f5835d67740 1 --2- 172.21.15.1:0/1280693726 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fac13af5d0 0x55fac13ab4f0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:09.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.502+0000 7f5835d67740 1 -- 172.21.15.1:0/1280693726 >> 172.21.15.1:0/1280693726 conn(0x55fac1378310 msgr2=0x55fac13c18e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:09.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.502+0000 7f5835d67740 1 -- 172.21.15.1:0/1280693726 shutdown_connections 2024-09-11T21:58:09.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.502+0000 7f5835d67740 1 -- 172.21.15.1:0/1280693726 wait complete. 2024-09-11T21:58:09.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.503+0000 7f5835d67740 1 -- start start 2024-09-11T21:58:09.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.503+0000 7f5835d67740 1 Processor -- start 2024-09-11T21:58:09.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.503+0000 7f5835d67740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fac13be5d0 0x55fac13cb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:09.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.503+0000 7f5835d67740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fac13c2b80 0x55fac13c7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:09.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.503+0000 7f5835d67740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fac13c8190 0x55fac13c85f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:09.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.503+0000 7f5835d67740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f582c1acbf0 con 0x55fac13be5d0 2024-09-11T21:58:09.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.503+0000 7f5835d67740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f582c1ac520 con 0x55fac13c2b80 2024-09-11T21:58:09.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.503+0000 7f5835d67740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f582c1a9760 con 0x55fac13c8190 2024-09-11T21:58:09.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.503+0000 7f5835d67740 10 client.0 _ll_register_callbacks cb 0x7f581c009d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:58:09.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.503+0000 7f583172a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fac13be5d0 0x55fac13cb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:09.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.503+0000 7f583172a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fac13be5d0 0x55fac13cb4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42796/0 (socket says 172.21.15.1:42796) 2024-09-11T21:58:09.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.503+0000 7f583172a640 1 -- 172.21.15.1:0/4152708680 learned_addr learned my addr 172.21.15.1:0/4152708680 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:58:09.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.504+0000 7f5831f2b640 1 --2- 172.21.15.1:0/4152708680 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fac13c8190 0x55fac13c85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:09.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.504+0000 7f5830f29640 1 --2- 172.21.15.1:0/4152708680 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fac13c2b80 0x55fac13c7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:09.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.504+0000 7f583172a640 1 -- 172.21.15.1:0/4152708680 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fac13c8190 msgr2=0x55fac13c85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:09.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.504+0000 7f583172a640 1 --2- 172.21.15.1:0/4152708680 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fac13c8190 0x55fac13c85f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:09.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.504+0000 7f583172a640 1 -- 172.21.15.1:0/4152708680 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fac13c2b80 msgr2=0x55fac13c7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:09.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.504+0000 7f583172a640 1 --2- 172.21.15.1:0/4152708680 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fac13c2b80 0x55fac13c7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:09.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.504+0000 7f583172a640 1 -- 172.21.15.1:0/4152708680 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fac143d650 con 0x55fac13be5d0 2024-09-11T21:58:09.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.504+0000 7f583172a640 1 --2- 172.21.15.1:0/4152708680 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fac13be5d0 0x55fac13cb4c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f582809dff0 tx=0x7f582809b9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7004f1f66940cfe7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:09.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.504+0000 7f5814ff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T21:58:09.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.504+0000 7f5814ff9640 1 -- 172.21.15.1:0/4152708680 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582809f880 con 0x55fac13be5d0 2024-09-11T21:58:09.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.504+0000 7f5814ff9640 1 -- 172.21.15.1:0/4152708680 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7f58280a8590 con 0x55fac13be5d0 2024-09-11T21:58:09.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.505+0000 7f5835d67740 1 -- 172.21.15.1:0/4152708680 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58280a0440 con 0x55fac13be5d0 2024-09-11T21:58:09.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.505+0000 7f5814ff9640 1 -- 172.21.15.1:0/4152708680 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f58280a40f0 con 0x55fac13be5d0 2024-09-11T21:58:09.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.505+0000 7f5835d67740 10 client.15723 fetch_fsmap learned FSMap version 28 2024-09-11T21:58:09.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.505+0000 7f5835d67740 1 -- 172.21.15.1:0/4152708680 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fac143e930 con 0x55fac13be5d0 2024-09-11T21:58:09.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.505+0000 7f5814ff9640 1 -- 172.21.15.1:0/4152708680 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58280a0440 con 0x55fac13be5d0 2024-09-11T21:58:09.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.505+0000 7f5814ff9640 1 -- 172.21.15.1:0/4152708680 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f58280d1210 con 0x55fac13be5d0 2024-09-11T21:58:09.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.505+0000 7f5835d67740 10 client.15723 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T21:58:09.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.505+0000 7f5835d67740 10 client.15723 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T21:58:09.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.505+0000 7f5835d67740 20 client.15723 populate_metadata read hostname 'smithi001' 2024-09-11T21:58:09.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.505+0000 7f5835d67740 1 --2- 172.21.15.1:0/4152708680 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55fac143eb60 0x55fac1440f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:09.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.506+0000 7f5835d67740 4 client.15723 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T21:58:09.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.506+0000 7f5835d67740 1 -- 172.21.15.1:0/4152708680 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fac131cd70 con 0x55fac143eb60 2024-09-11T21:58:09.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.506+0000 7f5830f29640 1 --2- 172.21.15.1:0/4152708680 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55fac143eb60 0x55fac1440f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:09.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.506+0000 7f5814ff9640 10 client.15723 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T21:58:09.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.506+0000 7f5830f29640 1 --2- 172.21.15.1:0/4152708680 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55fac143eb60 0x55fac1440f50 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fef8ce9905c451e server_cookie=a8f7e6763299ba9e in_seq=0 out_seq=0 2024-09-11T21:58:09.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.507+0000 7f5814ff9640 1 -- 172.21.15.1:0/4152708680 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fac131cd70 con 0x55fac143eb60 2024-09-11T21:58:09.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.507+0000 7f5814ff9640 10 client.15723 handle_command_reply: tid=0 2024-09-11T21:58:09.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.507+0000 7f5835d67740 1 client.15723 shutdown 2024-09-11T21:58:09.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.507+0000 7f5835d67740 1 -- 172.21.15.1:0/4152708680 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fac13be5d0 msgr2=0x55fac13cb4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:09.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.507+0000 7f5835d67740 1 --2- 172.21.15.1:0/4152708680 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fac13be5d0 0x55fac13cb4c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f582809dff0 tx=0x7f582809b9e0 comp rx=0 tx=0).stop 2024-09-11T21:58:09.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.507+0000 7f5835d67740 1 -- 172.21.15.1:0/4152708680 shutdown_connections 2024-09-11T21:58:09.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.507+0000 7f5835d67740 1 --2- 172.21.15.1:0/4152708680 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55fac143eb60 0x55fac1440f50 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:09.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.507+0000 7f5835d67740 1 --2- 172.21.15.1:0/4152708680 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fac13c8190 0x55fac13c85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:09.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.507+0000 7f5814ff9640 0 client.15723 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T21:58:09.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.507+0000 7f5835d67740 1 --2- 172.21.15.1:0/4152708680 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fac13c2b80 0x55fac13c7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:09.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.507+0000 7f5835d67740 1 --2- 172.21.15.1:0/4152708680 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fac13be5d0 0x55fac13cb4c0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:09.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.507+0000 7f5835d67740 1 -- 172.21.15.1:0/4152708680 >> 172.21.15.1:0/4152708680 conn(0x55fac1378310 msgr2=0x55fac13b10e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:09.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.507+0000 7f5835d67740 1 -- 172.21.15.1:0/4152708680 shutdown_connections 2024-09-11T21:58:09.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.507+0000 7f5835d67740 1 -- 172.21.15.1:0/4152708680 wait complete. 2024-09-11T21:58:09.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.508+0000 7f5835d67740 20 client.15723 trim_cache size 0 max 16384 2024-09-11T21:58:09.513 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-11T21:58:09.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.508+0000 7f583398e640 1 -- 172.21.15.1:0/3786704119 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f58180a2f10 msgr2=0x7f58180a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:09.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.508+0000 7f583398e640 1 --2- 172.21.15.1:0/3786704119 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f58180a2f10 0x7f58180a53d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5824005fe0 tx=0x7f582402a950 comp rx=0 tx=0).stop 2024-09-11T21:58:09.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.508+0000 7f583398e640 1 -- 172.21.15.1:0/3786704119 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f582c100e20 msgr2=0x7f582c199370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:09.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.508+0000 7f583398e640 1 --2- 172.21.15.1:0/3786704119 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f582c100e20 0x7f582c199370 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f5828009f40 tx=0x7f582800ee80 comp rx=0 tx=0).stop 2024-09-11T21:58:09.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.509+0000 7f583398e640 1 -- 172.21.15.1:0/3786704119 shutdown_connections 2024-09-11T21:58:09.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.509+0000 7f583398e640 1 --2- 172.21.15.1:0/3786704119 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f58180a2f10 0x7f58180a53d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:09.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.509+0000 7f583398e640 1 --2- 172.21.15.1:0/3786704119 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f582c102ef0 0x7f582c1a43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:09.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.509+0000 7f583398e640 1 --2- 172.21.15.1:0/3786704119 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f582c102570 0x7f582c1998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:09.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.509+0000 7f583398e640 1 --2- 172.21.15.1:0/3786704119 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f582c100e20 0x7f582c199370 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:09.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.509+0000 7f583398e640 1 -- 172.21.15.1:0/3786704119 >> 172.21.15.1:0/3786704119 conn(0x7f582c0fc7d0 msgr2=0x7f582c106880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:09.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.509+0000 7f583398e640 1 -- 172.21.15.1:0/3786704119 shutdown_connections 2024-09-11T21:58:09.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:09.509+0000 7f583398e640 1 -- 172.21.15.1:0/3786704119 wait complete. 2024-09-11T21:58:11.199 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T21:58:11.200 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T21:58:11.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:58:11 smithi053 ceph-mon[34908]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-09-11T21:58:11.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:58:11 smithi161 ceph-mon[35615]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-09-11T21:58:11.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:58:11 smithi001 ceph-mon[30935]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-09-11T21:58:11.474 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:58:11.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.971+0000 7f2b55188640 1 -- 172.21.15.1:0/2913031683 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b50100b20 msgr2=0x7f2b50100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:11.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.971+0000 7f2b55188640 1 --2- 172.21.15.1:0/2913031683 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b50100b20 0x7f2b50100f00 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f2b40009900 tx=0x7f2b4002a410 comp rx=0 tx=0).stop 2024-09-11T21:58:11.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.971+0000 7f2b55188640 1 -- 172.21.15.1:0/2913031683 shutdown_connections 2024-09-11T21:58:11.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.971+0000 7f2b55188640 1 --2- 172.21.15.1:0/2913031683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b5010e500 0x7f2b50110910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:11.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.971+0000 7f2b55188640 1 --2- 172.21.15.1:0/2913031683 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b50101440 0x7f2b5010df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:11.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.971+0000 7f2b55188640 1 --2- 172.21.15.1:0/2913031683 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b50100b20 0x7f2b50100f00 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:11.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.971+0000 7f2b55188640 1 -- 172.21.15.1:0/2913031683 >> 172.21.15.1:0/2913031683 conn(0x7f2b500fc7d0 msgr2=0x7f2b500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:11.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.971+0000 7f2b55188640 1 -- 172.21.15.1:0/2913031683 shutdown_connections 2024-09-11T21:58:11.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.971+0000 7f2b55188640 1 -- 172.21.15.1:0/2913031683 wait complete. 2024-09-11T21:58:11.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.972+0000 7f2b55188640 1 Processor -- start 2024-09-11T21:58:11.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.972+0000 7f2b55188640 1 -- start start 2024-09-11T21:58:11.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.973+0000 7f2b55188640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b50101440 0x7f2b501a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:11.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.973+0000 7f2b55188640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b5010e500 0x7f2b501a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:11.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.973+0000 7f2b55188640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b5019a850 0x7f2b5019ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:11.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.973+0000 7f2b55188640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2b50114260 con 0x7f2b5019a850 2024-09-11T21:58:11.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.973+0000 7f2b55188640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2b501140e0 con 0x7f2b50101440 2024-09-11T21:58:11.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.973+0000 7f2b55188640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2b501143e0 con 0x7f2b5010e500 2024-09-11T21:58:11.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.973+0000 7f2b4ed9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b50101440 0x7f2b501a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:11.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.973+0000 7f2b4ed9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b50101440 0x7f2b501a06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:55516/0 (socket says 172.21.15.1:55516) 2024-09-11T21:58:11.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.973+0000 7f2b4ed9d640 1 -- 172.21.15.1:0/106146791 learned_addr learned my addr 172.21.15.1:0/106146791 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:58:11.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.973+0000 7f2b4e59c640 1 --2- 172.21.15.1:0/106146791 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b5010e500 0x7f2b501a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:11.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.973+0000 7f2b4f59e640 1 --2- 172.21.15.1:0/106146791 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b5019a850 0x7f2b5019ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:11.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.974+0000 7f2b4e59c640 1 -- 172.21.15.1:0/106146791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b50101440 msgr2=0x7f2b501a06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:11.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.974+0000 7f2b4e59c640 1 --2- 172.21.15.1:0/106146791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b50101440 0x7f2b501a06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:11.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.974+0000 7f2b4e59c640 1 -- 172.21.15.1:0/106146791 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b5019a850 msgr2=0x7f2b5019ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:11.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.974+0000 7f2b4e59c640 1 --2- 172.21.15.1:0/106146791 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b5019a850 0x7f2b5019ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:11.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.974+0000 7f2b4e59c640 1 -- 172.21.15.1:0/106146791 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b5019b430 con 0x7f2b5010e500 2024-09-11T21:58:11.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.974+0000 7f2b4f59e640 1 --2- 172.21.15.1:0/106146791 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b5019a850 0x7f2b5019ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T21:58:11.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.974+0000 7f2b4e59c640 1 --2- 172.21.15.1:0/106146791 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b5010e500 0x7f2b501a0c10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2b5006a140 tx=0x7f2b4400dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:11.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.974+0000 7f2b37fff640 1 -- 172.21.15.1:0/106146791 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b44017070 con 0x7f2b5010e500 2024-09-11T21:58:11.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.974+0000 7f2b55188640 1 -- 172.21.15.1:0/106146791 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b5019b6c0 con 0x7f2b5010e500 2024-09-11T21:58:11.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.975+0000 7f2b37fff640 1 -- 172.21.15.1:0/106146791 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b44009c30 con 0x7f2b5010e500 2024-09-11T21:58:11.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.975+0000 7f2b55188640 1 -- 172.21.15.1:0/106146791 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b501a7830 con 0x7f2b5010e500 2024-09-11T21:58:11.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.975+0000 7f2b37fff640 1 -- 172.21.15.1:0/106146791 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b44021060 con 0x7f2b5010e500 2024-09-11T21:58:11.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.977+0000 7f2b37fff640 1 -- 172.21.15.1:0/106146791 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 25) ==== 138065+0+0 (secure 0 0 0) 0x7f2b4401a070 con 0x7f2b5010e500 2024-09-11T21:58:11.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.978+0000 7f2b37fff640 1 --2- 172.21.15.1:0/106146791 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f2b380a2e90 0x7f2b380a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:11.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.978+0000 7f2b57561740 1 Processor -- start 2024-09-11T21:58:11.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.978+0000 7f2b4ed9d640 1 --2- 172.21.15.1:0/106146791 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f2b380a2e90 0x7f2b380a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:11.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.978+0000 7f2b57561740 1 -- start start 2024-09-11T21:58:11.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.978+0000 7f2b37fff640 1 -- 172.21.15.1:0/106146791 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7f2b440cefe0 con 0x7f2b5010e500 2024-09-11T21:58:11.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.979+0000 7f2b4ed9d640 1 --2- 172.21.15.1:0/106146791 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f2b380a2e90 0x7f2b380a5350 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2b400096c0 tx=0x7f2b400058d0 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:11.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.979+0000 7f2b57561740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f423dcdb60 0x55f423dcdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:11.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.979+0000 7f2b57561740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f423dce480 0x55f423dd8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:11.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.979+0000 7f2b57561740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f423dd8d40 0x55f423ddb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:11.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.979+0000 7f2b57561740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55f423d33280 con 0x55f423dcdb60 2024-09-11T21:58:11.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.979+0000 7f2b57561740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55f423cdab10 con 0x55f423dce480 2024-09-11T21:58:11.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.979+0000 7f2b57561740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f423d2f240 con 0x55f423dd8d40 2024-09-11T21:58:11.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.979+0000 7f2b4e59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f423dce480 0x55f423dd8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:11.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.979+0000 7f2b4e59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f423dce480 0x55f423dd8800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:55530/0 (socket says 172.21.15.1:55530) 2024-09-11T21:58:11.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.979+0000 7f2b4e59c640 1 -- 172.21.15.1:0/4262276157 learned_addr learned my addr 172.21.15.1:0/4262276157 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:58:11.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.979+0000 7f2b4ed9d640 1 --2- 172.21.15.1:0/4262276157 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f423dcdb60 0x55f423dcdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:11.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.980+0000 7f2b4f59e640 1 --2- 172.21.15.1:0/4262276157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f423dd8d40 0x55f423ddb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:11.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.980+0000 7f2b4e59c640 1 -- 172.21.15.1:0/4262276157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f423dd8d40 msgr2=0x55f423ddb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:11.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.980+0000 7f2b4e59c640 1 --2- 172.21.15.1:0/4262276157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f423dd8d40 0x55f423ddb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:11.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.980+0000 7f2b4e59c640 1 -- 172.21.15.1:0/4262276157 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f423dcdb60 msgr2=0x55f423dcdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:11.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.980+0000 7f2b4e59c640 1 --2- 172.21.15.1:0/4262276157 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f423dcdb60 0x55f423dcdf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:11.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.980+0000 7f2b4e59c640 1 -- 172.21.15.1:0/4262276157 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f423ce71f0 con 0x55f423dce480 2024-09-11T21:58:11.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.980+0000 7f2b4ed9d640 1 --2- 172.21.15.1:0/4262276157 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f423dcdb60 0x55f423dcdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T21:58:11.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.980+0000 7f2b4e59c640 1 --2- 172.21.15.1:0/4262276157 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f423dce480 0x55f423dd8800 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2b4409af50 tx=0x7f2b44009f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b2b31aad4e936ba server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:11.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.981+0000 7f2b357fa640 1 -- 172.21.15.1:0/4262276157 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b440a3ec0 con 0x55f423dce480 2024-09-11T21:58:11.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.981+0000 7f2b357fa640 1 -- 172.21.15.1:0/4262276157 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b440978c0 con 0x55f423dce480 2024-09-11T21:58:11.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.981+0000 7f2b357fa640 1 -- 172.21.15.1:0/4262276157 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b4409f120 con 0x55f423dce480 2024-09-11T21:58:11.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.982+0000 7f2b57561740 1 -- 172.21.15.1:0/4262276157 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f423dce480 msgr2=0x55f423dd8800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:11.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.982+0000 7f2b57561740 1 --2- 172.21.15.1:0/4262276157 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f423dce480 0x55f423dd8800 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2b4409af50 tx=0x7f2b44009f20 comp rx=0 tx=0).stop 2024-09-11T21:58:11.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.982+0000 7f2b57561740 1 -- 172.21.15.1:0/4262276157 shutdown_connections 2024-09-11T21:58:11.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.982+0000 7f2b57561740 1 --2- 172.21.15.1:0/4262276157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f423dd8d40 0x55f423ddb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:11.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.982+0000 7f2b57561740 1 --2- 172.21.15.1:0/4262276157 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f423dce480 0x55f423dd8800 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:11.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.982+0000 7f2b57561740 1 --2- 172.21.15.1:0/4262276157 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f423dcdb60 0x55f423dcdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:11.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.982+0000 7f2b57561740 1 -- 172.21.15.1:0/4262276157 >> 172.21.15.1:0/4262276157 conn(0x55f423dca0f0 msgr2=0x55f423dca4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:11.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.982+0000 7f2b57561740 1 -- 172.21.15.1:0/4262276157 shutdown_connections 2024-09-11T21:58:11.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.982+0000 7f2b57561740 1 -- 172.21.15.1:0/4262276157 wait complete. 2024-09-11T21:58:11.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.982+0000 7f2b57561740 1 -- start start 2024-09-11T21:58:11.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.983+0000 7f2b57561740 1 Processor -- start 2024-09-11T21:58:11.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.983+0000 7f2b57561740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f423dcdb60 0x55f423dd5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:11.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.983+0000 7f2b57561740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f423dce480 0x55f423dd6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:11.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.983+0000 7f2b57561740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f423dd8d40 0x55f423dd66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:11.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.983+0000 7f2b57561740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55f423ddb910 con 0x55f423dce480 2024-09-11T21:58:11.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.983+0000 7f2b57561740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55f423cedf10 con 0x55f423dd8d40 2024-09-11T21:58:11.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.983+0000 7f2b57561740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f423ddba90 con 0x55f423dcdb60 2024-09-11T21:58:11.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.983+0000 7f2b57561740 10 client.0 _ll_register_callbacks cb 0x55f423cb6a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:58:11.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.983+0000 7f2b4ed9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f423dcdb60 0x55f423dd5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:11.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.983+0000 7f2b4ed9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f423dcdb60 0x55f423dd5c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:41578/0 (socket says 172.21.15.1:41578) 2024-09-11T21:58:11.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.983+0000 7f2b4ed9d640 1 -- 172.21.15.1:0/4019599208 learned_addr learned my addr 172.21.15.1:0/4019599208 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:58:11.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.983+0000 7f2b4f59e640 1 --2- 172.21.15.1:0/4019599208 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f423dd8d40 0x55f423dd66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:11.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.983+0000 7f2b4e59c640 1 --2- 172.21.15.1:0/4019599208 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f423dce480 0x55f423dd6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:11.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.984+0000 7f2b4ed9d640 1 -- 172.21.15.1:0/4019599208 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f423dd8d40 msgr2=0x55f423dd66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:11.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.984+0000 7f2b4ed9d640 1 --2- 172.21.15.1:0/4019599208 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f423dd8d40 0x55f423dd66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:11.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.984+0000 7f2b4ed9d640 1 -- 172.21.15.1:0/4019599208 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f423dce480 msgr2=0x55f423dd6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:11.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.984+0000 7f2b4ed9d640 1 --2- 172.21.15.1:0/4019599208 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f423dce480 0x55f423dd6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:11.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.984+0000 7f2b4ed9d640 1 -- 172.21.15.1:0/4019599208 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f423aa46a0 con 0x55f423dcdb60 2024-09-11T21:58:11.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.984+0000 7f2b4e59c640 1 --2- 172.21.15.1:0/4019599208 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f423dce480 0x55f423dd6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:58:11.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.984+0000 7f2b4ed9d640 1 --2- 172.21.15.1:0/4019599208 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f423dcdb60 0x55f423dd5c20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2b40033000 tx=0x7f2b40009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c567c0f3a7bdbbdd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:11.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.984+0000 7f2b32ff5640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-11T21:58:11.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.984+0000 7f2b32ff5640 1 -- 172.21.15.1:0/4019599208 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b40035040 con 0x55f423dcdb60 2024-09-11T21:58:11.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.985+0000 7f2b32ff5640 1 -- 172.21.15.1:0/4019599208 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7f2b400035f0 con 0x55f423dcdb60 2024-09-11T21:58:11.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.985+0000 7f2b57561740 1 -- 172.21.15.1:0/4019599208 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b440978c0 con 0x55f423dcdb60 2024-09-11T21:58:11.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.985+0000 7f2b32ff5640 1 -- 172.21.15.1:0/4019599208 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2b400037f0 con 0x55f423dcdb60 2024-09-11T21:58:11.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.985+0000 7f2b57561740 10 client.25309 fetch_fsmap learned FSMap version 28 2024-09-11T21:58:11.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.985+0000 7f2b57561740 1 -- 172.21.15.1:0/4019599208 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f423de7570 con 0x55f423dcdb60 2024-09-11T21:58:11.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.985+0000 7f2b32ff5640 1 -- 172.21.15.1:0/4019599208 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b4005b480 con 0x55f423dcdb60 2024-09-11T21:58:11.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.986+0000 7f2b32ff5640 1 -- 172.21.15.1:0/4019599208 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2b4003c440 con 0x55f423dcdb60 2024-09-11T21:58:11.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.986+0000 7f2b57561740 10 client.25309 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T21:58:11.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.986+0000 7f2b57561740 10 client.25309 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T21:58:11.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.986+0000 7f2b57561740 20 client.25309 populate_metadata read hostname 'smithi001' 2024-09-11T21:58:11.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.986+0000 7f2b57561740 1 --2- 172.21.15.1:0/4019599208 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55f423de8980 0x55f423deadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:11.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.986+0000 7f2b57561740 4 client.25309 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T21:58:11.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.986+0000 7f2b57561740 1 -- 172.21.15.1:0/4019599208 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f2b440978c0 con 0x55f423de8980 2024-09-11T21:58:11.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.987+0000 7f2b4e59c640 1 --2- 172.21.15.1:0/4019599208 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55f423de8980 0x55f423deadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:11.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.987+0000 7f2b4e59c640 1 --2- 172.21.15.1:0/4019599208 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55f423de8980 0x55f423deadd0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59e3f3da21aeaf77 server_cookie=89deb8d952fad2b0 in_seq=0 out_seq=0 2024-09-11T21:58:11.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.987+0000 7f2b32ff5640 10 client.25309 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T21:58:11.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.988+0000 7f2b32ff5640 1 -- 172.21.15.1:0/4019599208 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f2b440978c0 con 0x55f423de8980 2024-09-11T21:58:11.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.988+0000 7f2b32ff5640 10 client.25309 handle_command_reply: tid=0 2024-09-11T21:58:11.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.988+0000 7f2b57561740 1 client.25309 shutdown 2024-09-11T21:58:11.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.988+0000 7f2b57561740 1 -- 172.21.15.1:0/4019599208 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f423dcdb60 msgr2=0x55f423dd5c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:11.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.989+0000 7f2b57561740 1 --2- 172.21.15.1:0/4019599208 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f423dcdb60 0x55f423dd5c20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2b40033000 tx=0x7f2b40009260 comp rx=0 tx=0).stop 2024-09-11T21:58:11.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.989+0000 7f2b57561740 1 -- 172.21.15.1:0/4019599208 shutdown_connections 2024-09-11T21:58:11.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.989+0000 7f2b57561740 1 --2- 172.21.15.1:0/4019599208 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55f423de8980 0x55f423deadd0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:11.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.989+0000 7f2b57561740 1 --2- 172.21.15.1:0/4019599208 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f423dd8d40 0x55f423dd66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:11.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.989+0000 7f2b57561740 1 --2- 172.21.15.1:0/4019599208 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f423dce480 0x55f423dd6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:11.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.989+0000 7f2b57561740 1 --2- 172.21.15.1:0/4019599208 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f423dcdb60 0x55f423dd5c20 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:11.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.989+0000 7f2b57561740 1 -- 172.21.15.1:0/4019599208 >> 172.21.15.1:0/4019599208 conn(0x55f423dca0f0 msgr2=0x55f423dcfe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:11.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.989+0000 7f2b32ff5640 0 client.25309 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T21:58:11.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.989+0000 7f2b57561740 1 -- 172.21.15.1:0/4019599208 shutdown_connections 2024-09-11T21:58:11.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.989+0000 7f2b57561740 1 -- 172.21.15.1:0/4019599208 wait complete. 2024-09-11T21:58:11.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:11.989+0000 7f2b57561740 20 client.25309 trim_cache size 0 max 16384 2024-09-11T21:58:12.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.023+0000 7f2b57561740 1 Processor -- start 2024-09-11T21:58:12.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.023+0000 7f2b57561740 1 -- start start 2024-09-11T21:58:12.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.023+0000 7f2b57561740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f423dcdb60 0x55f423dd4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:12.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.023+0000 7f2b57561740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f423dce480 0x55f423df06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:12.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.023+0000 7f2b57561740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f423dd8d40 0x55f423df0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:12.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.023+0000 7f2b57561740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2b3c00e7d0 con 0x55f423dd8d40 2024-09-11T21:58:12.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.023+0000 7f2b57561740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2b3c010310 con 0x55f423dcdb60 2024-09-11T21:58:12.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.023+0000 7f2b57561740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2b3c00eea0 con 0x55f423dce480 2024-09-11T21:58:12.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.024+0000 7f2b4f59e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f423dd8d40 0x55f423df0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:12.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.024+0000 7f2b4f59e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f423dd8d40 0x55f423df0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42846/0 (socket says 172.21.15.1:42846) 2024-09-11T21:58:12.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.024+0000 7f2b4f59e640 1 -- 172.21.15.1:0/2934543124 learned_addr learned my addr 172.21.15.1:0/2934543124 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:58:12.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.024+0000 7f2b4e59c640 1 --2- 172.21.15.1:0/2934543124 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f423dce480 0x55f423df06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:12.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.024+0000 7f2b4ed9d640 1 --2- 172.21.15.1:0/2934543124 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f423dcdb60 0x55f423dd4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:12.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.024+0000 7f2b4f59e640 1 -- 172.21.15.1:0/2934543124 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f423dce480 msgr2=0x55f423df06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:12.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.024+0000 7f2b4f59e640 1 --2- 172.21.15.1:0/2934543124 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f423dce480 0x55f423df06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:12.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.024+0000 7f2b4f59e640 1 -- 172.21.15.1:0/2934543124 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f423dcdb60 msgr2=0x55f423dd4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:12.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.024+0000 7f2b4f59e640 1 --2- 172.21.15.1:0/2934543124 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f423dcdb60 0x55f423dd4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:12.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.024+0000 7f2b4f59e640 1 -- 172.21.15.1:0/2934543124 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f423dcead0 con 0x55f423dd8d40 2024-09-11T21:58:12.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.024+0000 7f2b4f59e640 1 --2- 172.21.15.1:0/2934543124 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f423dd8d40 0x55f423df0be0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f2b48013490 tx=0x7f2b48013950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=536410784f5bc878 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:12.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.025+0000 7f2b34ff9640 1 -- 172.21.15.1:0/2934543124 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b480117c0 con 0x55f423dd8d40 2024-09-11T21:58:12.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.025+0000 7f2b34ff9640 1 -- 172.21.15.1:0/2934543124 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b48011e40 con 0x55f423dd8d40 2024-09-11T21:58:12.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.025+0000 7f2b34ff9640 1 -- 172.21.15.1:0/2934543124 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b4801c3a0 con 0x55f423dd8d40 2024-09-11T21:58:12.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.026+0000 7f2b57561740 1 -- 172.21.15.1:0/2934543124 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f423dd8d40 msgr2=0x55f423df0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:12.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.026+0000 7f2b57561740 1 --2- 172.21.15.1:0/2934543124 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f423dd8d40 0x55f423df0be0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f2b48013490 tx=0x7f2b48013950 comp rx=0 tx=0).stop 2024-09-11T21:58:12.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.026+0000 7f2b57561740 1 -- 172.21.15.1:0/2934543124 shutdown_connections 2024-09-11T21:58:12.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.026+0000 7f2b57561740 1 --2- 172.21.15.1:0/2934543124 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f423dd8d40 0x55f423df0be0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:12.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.026+0000 7f2b57561740 1 --2- 172.21.15.1:0/2934543124 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f423dce480 0x55f423df06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:12.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.026+0000 7f2b57561740 1 --2- 172.21.15.1:0/2934543124 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f423dcdb60 0x55f423dd4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:12.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.026+0000 7f2b57561740 1 -- 172.21.15.1:0/2934543124 >> 172.21.15.1:0/2934543124 conn(0x55f423dca0f0 msgr2=0x55f423dd43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:12.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.026+0000 7f2b57561740 1 -- 172.21.15.1:0/2934543124 shutdown_connections 2024-09-11T21:58:12.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.026+0000 7f2b57561740 1 -- 172.21.15.1:0/2934543124 wait complete. 2024-09-11T21:58:12.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.027+0000 7f2b57561740 1 -- start start 2024-09-11T21:58:12.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.027+0000 7f2b57561740 1 Processor -- start 2024-09-11T21:58:12.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.027+0000 7f2b57561740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f423dcdb60 0x55f423de15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:12.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.027+0000 7f2b57561740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f423dce480 0x55f423ddc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:12.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.028+0000 7f2b57561740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f423dd8d40 0x55f423ddcf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:12.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.028+0000 7f2b4e59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f423dce480 0x55f423ddc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:12.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.028+0000 7f2b4e59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f423dce480 0x55f423ddc900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42858/0 (socket says 172.21.15.1:42858) 2024-09-11T21:58:12.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.028+0000 7f2b4e59c640 1 -- 172.21.15.1:0/1097334260 learned_addr learned my addr 172.21.15.1:0/1097334260 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:58:12.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.028+0000 7f2b57561740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2b3c00fc40 con 0x55f423dce480 2024-09-11T21:58:12.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.028+0000 7f2b57561740 1 -- 172.21.15.1:0/1097334260 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2b3c00f570 con 0x55f423dcdb60 2024-09-11T21:58:12.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.028+0000 7f2b4f59e640 1 --2- 172.21.15.1:0/1097334260 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f423dd8d40 0x55f423ddcf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:12.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.028+0000 7f2b57561740 1 -- 172.21.15.1:0/1097334260 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2b3c00c7b0 con 0x55f423dd8d40 2024-09-11T21:58:12.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.028+0000 7f2b4ed9d640 1 --2- 172.21.15.1:0/1097334260 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f423dcdb60 0x55f423de15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:12.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.028+0000 7f2b57561740 10 client.0 _ll_register_callbacks cb 0x7f2b400317b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:58:12.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.028+0000 7f2b4e59c640 1 -- 172.21.15.1:0/1097334260 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f423dd8d40 msgr2=0x55f423ddcf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:12.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.028+0000 7f2b4e59c640 1 --2- 172.21.15.1:0/1097334260 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f423dd8d40 0x55f423ddcf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:12.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.028+0000 7f2b4e59c640 1 -- 172.21.15.1:0/1097334260 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f423dcdb60 msgr2=0x55f423de15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:12.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.028+0000 7f2b4e59c640 1 --2- 172.21.15.1:0/1097334260 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f423dcdb60 0x55f423de15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:12.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.028+0000 7f2b4e59c640 1 -- 172.21.15.1:0/1097334260 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f423ddd740 con 0x55f423dce480 2024-09-11T21:58:12.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.029+0000 7f2b4e59c640 1 --2- 172.21.15.1:0/1097334260 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f423dce480 0x55f423ddc900 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f2b44009610 tx=0x7f2b4409aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99f6f075de73a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:12.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.029+0000 7f2b57561740 1 -- 172.21.15.1:0/1097334260 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b48011e40 con 0x55f423dce480 2024-09-11T21:58:12.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.029+0000 7f2b32ff5640 10 client.15738 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T21:58:12.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.029+0000 7f2b32ff5640 1 -- 172.21.15.1:0/1097334260 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b440978c0 con 0x55f423dce480 2024-09-11T21:58:12.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.029+0000 7f2b32ff5640 1 -- 172.21.15.1:0/1097334260 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7f2b440bb0b0 con 0x55f423dce480 2024-09-11T21:58:12.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.029+0000 7f2b32ff5640 1 -- 172.21.15.1:0/1097334260 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2b440bb2b0 con 0x55f423dce480 2024-09-11T21:58:12.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.029+0000 7f2b57561740 10 client.15738 fetch_fsmap learned FSMap version 28 2024-09-11T21:58:12.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.029+0000 7f2b57561740 1 -- 172.21.15.1:0/1097334260 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({fsmap=28}) -- 0x55f423e5bf10 con 0x55f423dce480 2024-09-11T21:58:12.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.030+0000 7f2b32ff5640 1 -- 172.21.15.1:0/1097334260 <== mon.0 v2:172.21.15.1:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2b440cc830 con 0x55f423dce480 2024-09-11T21:58:12.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.030+0000 7f2b57561740 10 client.15738 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T21:58:12.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.030+0000 7f2b57561740 10 client.15738 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T21:58:12.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.030+0000 7f2b57561740 20 client.15738 populate_metadata read hostname 'smithi001' 2024-09-11T21:58:12.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.030+0000 7f2b57561740 1 --2- 172.21.15.1:0/1097334260 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55f423e5c140 0x55f423e5e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:12.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.030+0000 7f2b57561740 4 client.15738 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T21:58:12.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.030+0000 7f2b57561740 1 -- 172.21.15.1:0/1097334260 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f423d407f0 con 0x55f423e5c140 2024-09-11T21:58:12.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.030+0000 7f2b4ed9d640 1 --2- 172.21.15.1:0/1097334260 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55f423e5c140 0x55f423e5e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:12.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.030+0000 7f2b4ed9d640 1 --2- 172.21.15.1:0/1097334260 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55f423e5c140 0x55f423e5e530 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1661e47a9039a41 server_cookie=bfcb3a226f814765 in_seq=0 out_seq=0 2024-09-11T21:58:12.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.030+0000 7f2b32ff5640 10 client.15738 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T21:58:12.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.032+0000 7f2b32ff5640 1 -- 172.21.15.1:0/1097334260 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f423d407f0 con 0x55f423e5c140 2024-09-11T21:58:12.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.032+0000 7f2b32ff5640 10 client.15738 handle_command_reply: tid=0 2024-09-11T21:58:12.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.032+0000 7f2b57561740 1 client.15738 shutdown 2024-09-11T21:58:12.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.032+0000 7f2b57561740 1 -- 172.21.15.1:0/1097334260 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f423dce480 msgr2=0x55f423ddc900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:12.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.032+0000 7f2b57561740 1 --2- 172.21.15.1:0/1097334260 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f423dce480 0x55f423ddc900 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f2b44009610 tx=0x7f2b4409aca0 comp rx=0 tx=0).stop 2024-09-11T21:58:12.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.032+0000 7f2b57561740 1 -- 172.21.15.1:0/1097334260 shutdown_connections 2024-09-11T21:58:12.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.032+0000 7f2b57561740 1 --2- 172.21.15.1:0/1097334260 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55f423e5c140 0x55f423e5e530 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:12.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.032+0000 7f2b57561740 1 --2- 172.21.15.1:0/1097334260 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f423dd8d40 0x55f423ddcf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:12.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.032+0000 7f2b57561740 1 --2- 172.21.15.1:0/1097334260 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f423dce480 0x55f423ddc900 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:12.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.032+0000 7f2b32ff5640 0 client.15738 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T21:58:12.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.032+0000 7f2b57561740 1 --2- 172.21.15.1:0/1097334260 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f423dcdb60 0x55f423de15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:12.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.032+0000 7f2b57561740 1 -- 172.21.15.1:0/1097334260 >> 172.21.15.1:0/1097334260 conn(0x55f423dca0f0 msgr2=0x55f423dc9270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:12.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.032+0000 7f2b57561740 1 -- 172.21.15.1:0/1097334260 shutdown_connections 2024-09-11T21:58:12.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.032+0000 7f2b57561740 1 -- 172.21.15.1:0/1097334260 wait complete. 2024-09-11T21:58:12.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.033+0000 7f2b57561740 20 client.15738 trim_cache size 0 max 16384 2024-09-11T21:58:12.038 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-11T21:58:12.038 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-09-11T21:58:12.038 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "672dce12-47f2-4540-97eb-f68a653b4be6", 2024-09-11T21:58:12.038 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-09-11T21:58:12.038 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-11T21:58:12.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.033+0000 7f2b55188640 1 -- 172.21.15.1:0/106146791 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f2b380a2e90 msgr2=0x7f2b380a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:12.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.033+0000 7f2b55188640 1 --2- 172.21.15.1:0/106146791 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f2b380a2e90 0x7f2b380a5350 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2b400096c0 tx=0x7f2b400058d0 comp rx=0 tx=0).stop 2024-09-11T21:58:12.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.033+0000 7f2b55188640 1 -- 172.21.15.1:0/106146791 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b5010e500 msgr2=0x7f2b501a0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:12.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.033+0000 7f2b55188640 1 --2- 172.21.15.1:0/106146791 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b5010e500 0x7f2b501a0c10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2b5006a140 tx=0x7f2b4400dde0 comp rx=0 tx=0).stop 2024-09-11T21:58:12.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.033+0000 7f2b55188640 1 -- 172.21.15.1:0/106146791 shutdown_connections 2024-09-11T21:58:12.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.033+0000 7f2b55188640 1 --2- 172.21.15.1:0/106146791 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f2b380a2e90 0x7f2b380a5350 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:12.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.033+0000 7f2b55188640 1 --2- 172.21.15.1:0/106146791 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b5019a850 0x7f2b5019ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:12.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.033+0000 7f2b55188640 1 --2- 172.21.15.1:0/106146791 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b5010e500 0x7f2b501a0c10 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:12.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.034+0000 7f2b55188640 1 --2- 172.21.15.1:0/106146791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2b50101440 0x7f2b501a06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:12.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.034+0000 7f2b55188640 1 -- 172.21.15.1:0/106146791 >> 172.21.15.1:0/106146791 conn(0x7f2b500fc7d0 msgr2=0x7f2b500febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:12.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.034+0000 7f2b55188640 1 -- 172.21.15.1:0/106146791 shutdown_connections 2024-09-11T21:58:12.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:12.034+0000 7f2b55188640 1 -- 172.21.15.1:0/106146791 wait complete. 2024-09-11T21:58:12.788 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T21:58:13.138 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:58:13.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:58:13 smithi053 ceph-mon[34908]: pgmap v32: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 29 op/s 2024-09-11T21:58:13.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:58:13 smithi053 ceph-mon[34908]: scrub queued for path: / 2024-09-11T21:58:13.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:58:13 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T21:58:13.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:58:13 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T21:58:13.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:58:13 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T21:58:13.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:58:13 smithi161 ceph-mon[35615]: pgmap v32: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 29 op/s 2024-09-11T21:58:13.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:58:13 smithi161 ceph-mon[35615]: scrub queued for path: / 2024-09-11T21:58:13.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:58:13 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T21:58:13.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:58:13 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T21:58:13.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:58:13 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T21:58:13.410 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:58:13 smithi001 ceph-mon[30935]: pgmap v32: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 29 op/s 2024-09-11T21:58:13.410 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:58:13 smithi001 ceph-mon[30935]: scrub queued for path: / 2024-09-11T21:58:13.410 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:58:13 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T21:58:13.410 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:58:13 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T21:58:13.410 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:58:13 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T21:58:13.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.672+0000 7fb36ba77640 1 -- 172.21.15.1:0/4085534312 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb364102570 msgr2=0x7fb3641029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:13.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.672+0000 7fb36ba77640 1 --2- 172.21.15.1:0/4085534312 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb364102570 0x7fb3641029b0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fb354009900 tx=0x7fb35402a480 comp rx=0 tx=0).stop 2024-09-11T21:58:13.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.672+0000 7fb36ba77640 1 -- 172.21.15.1:0/4085534312 shutdown_connections 2024-09-11T21:58:13.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.672+0000 7fb36ba77640 1 --2- 172.21.15.1:0/4085534312 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb364102ef0 0x7fb36410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:13.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.672+0000 7fb36ba77640 1 --2- 172.21.15.1:0/4085534312 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb364102570 0x7fb3641029b0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:13.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.672+0000 7fb36ba77640 1 --2- 172.21.15.1:0/4085534312 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb364100e20 0x7fb364101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:13.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.673+0000 7fb36ba77640 1 -- 172.21.15.1:0/4085534312 >> 172.21.15.1:0/4085534312 conn(0x7fb3640fc7d0 msgr2=0x7fb3640febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:13.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.673+0000 7fb36ba77640 1 -- 172.21.15.1:0/4085534312 shutdown_connections 2024-09-11T21:58:13.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.673+0000 7fb36ba77640 1 -- 172.21.15.1:0/4085534312 wait complete. 2024-09-11T21:58:13.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.675+0000 7fb36ba77640 1 Processor -- start 2024-09-11T21:58:13.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.675+0000 7fb36ba77640 1 -- start start 2024-09-11T21:58:13.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.675+0000 7fb36ba77640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb364100e20 0x7fb364196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:13.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.676+0000 7fb36ba77640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb364102570 0x7fb3641967c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:13.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.676+0000 7fb36ba77640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb364102ef0 0x7fb36419ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:13.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.676+0000 7fb36a014640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb364102ef0 0x7fb36419ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:13.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.676+0000 7fb36a014640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb364102ef0 0x7fb36419ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42876/0 (socket says 172.21.15.1:42876) 2024-09-11T21:58:13.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.676+0000 7fb369012640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb364102570 0x7fb3641967c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:13.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.676+0000 7fb369012640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb364102570 0x7fb3641967c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:55570/0 (socket says 172.21.15.1:55570) 2024-09-11T21:58:13.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.676+0000 7fb36a014640 1 -- 172.21.15.1:0/4060478071 learned_addr learned my addr 172.21.15.1:0/4060478071 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:58:13.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.676+0000 7fb36ba77640 1 -- 172.21.15.1:0/4060478071 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb36410fe80 con 0x7fb364102ef0 2024-09-11T21:58:13.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.676+0000 7fb36ba77640 1 -- 172.21.15.1:0/4060478071 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb36410fd00 con 0x7fb364102570 2024-09-11T21:58:13.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.676+0000 7fb36ba77640 1 -- 172.21.15.1:0/4060478071 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb364110000 con 0x7fb364100e20 2024-09-11T21:58:13.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.676+0000 7fb369813640 1 --2- 172.21.15.1:0/4060478071 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb364100e20 0x7fb364196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:13.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.677+0000 7fb36a014640 1 -- 172.21.15.1:0/4060478071 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb364100e20 msgr2=0x7fb364196280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:13.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.677+0000 7fb36a014640 1 --2- 172.21.15.1:0/4060478071 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb364100e20 0x7fb364196280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:13.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.677+0000 7fb36a014640 1 -- 172.21.15.1:0/4060478071 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb364102570 msgr2=0x7fb3641967c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:13.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.677+0000 7fb36a014640 1 --2- 172.21.15.1:0/4060478071 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb364102570 0x7fb3641967c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:13.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.677+0000 7fb36a014640 1 -- 172.21.15.1:0/4060478071 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb36419b230 con 0x7fb364102ef0 2024-09-11T21:58:13.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.677+0000 7fb369813640 1 --2- 172.21.15.1:0/4060478071 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb364100e20 0x7fb364196280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T21:58:13.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.677+0000 7fb36a014640 1 --2- 172.21.15.1:0/4060478071 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb364102ef0 0x7fb36419ab50 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fb35c00eaa0 tx=0x7fb35c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:13.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.677+0000 7fb35affd640 1 -- 172.21.15.1:0/4060478071 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb35c00cdb0 con 0x7fb364102ef0 2024-09-11T21:58:13.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.678+0000 7fb35affd640 1 -- 172.21.15.1:0/4060478071 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb35c005020 con 0x7fb364102ef0 2024-09-11T21:58:13.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.678+0000 7fb35affd640 1 -- 172.21.15.1:0/4060478071 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb35c0116b0 con 0x7fb364102ef0 2024-09-11T21:58:13.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.678+0000 7fb36ba77640 1 -- 172.21.15.1:0/4060478071 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb36419b520 con 0x7fb364102ef0 2024-09-11T21:58:13.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.678+0000 7fb36ba77640 1 -- 172.21.15.1:0/4060478071 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3641a2e00 con 0x7fb364102ef0 2024-09-11T21:58:13.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.681+0000 7fb36de50740 1 Processor -- start 2024-09-11T21:58:13.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.682+0000 7fb35affd640 1 -- 172.21.15.1:0/4060478071 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) ==== 138065+0+0 (secure 0 0 0) 0x7fb35c0051d0 con 0x7fb364102ef0 2024-09-11T21:58:13.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.683+0000 7fb36de50740 1 -- start start 2024-09-11T21:58:13.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.683+0000 7fb36de50740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559bbef764a0 0x559bbef76880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:13.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.683+0000 7fb36de50740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559bbef76dc0 0x559bbef81140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:13.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.684+0000 7fb369813640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559bbef76dc0 0x559bbef81140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:13.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.684+0000 7fb369813640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559bbef76dc0 0x559bbef81140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42892/0 (socket says 172.21.15.1:42892) 2024-09-11T21:58:13.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.684+0000 7fb369813640 1 -- 172.21.15.1:0/3660542071 learned_addr learned my addr 172.21.15.1:0/3660542071 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:58:13.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.684+0000 7fb369012640 1 --2- 172.21.15.1:0/3660542071 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559bbef764a0 0x559bbef76880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:13.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.684+0000 7fb35affd640 1 --2- 172.21.15.1:0/4060478071 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fb3500a2e90 0x7fb3500a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:13.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.684+0000 7fb36a014640 1 --2- 172.21.15.1:0/4060478071 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fb3500a2e90 0x7fb3500a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:13.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.684+0000 7fb36de50740 1 --2- 172.21.15.1:0/3660542071 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bbef81680 0x559bbef83a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:13.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.684+0000 7fb36de50740 1 -- 172.21.15.1:0/3660542071 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x559bbeedc2e0 con 0x559bbef76dc0 2024-09-11T21:58:13.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.684+0000 7fb36de50740 1 -- 172.21.15.1:0/3660542071 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x559bbee83ce0 con 0x559bbef764a0 2024-09-11T21:58:13.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.684+0000 7fb36de50740 1 -- 172.21.15.1:0/3660542071 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x559bbeed82a0 con 0x559bbef81680 2024-09-11T21:58:13.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.684+0000 7fb35affd640 1 -- 172.21.15.1:0/4060478071 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7fb35c0cfa00 con 0x7fb364102ef0 2024-09-11T21:58:13.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.684+0000 7fb369012640 1 --2- 172.21.15.1:0/3660542071 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bbef81680 0x559bbef83a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:13.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.685+0000 7fb369012640 1 -- 172.21.15.1:0/3660542071 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bbef81680 msgr2=0x559bbef83a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:13.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.685+0000 7fb369012640 1 --2- 172.21.15.1:0/3660542071 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bbef81680 0x559bbef83a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:13.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.685+0000 7fb369012640 1 -- 172.21.15.1:0/3660542071 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559bbef76dc0 msgr2=0x559bbef81140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:13.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.685+0000 7fb369012640 1 --2- 172.21.15.1:0/3660542071 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559bbef76dc0 0x559bbef81140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:13.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.685+0000 7fb369012640 1 -- 172.21.15.1:0/3660542071 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bbee68440 con 0x559bbef764a0 2024-09-11T21:58:13.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.685+0000 7fb369012640 1 --2- 172.21.15.1:0/3660542071 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559bbef764a0 0x559bbef76880 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb354005940 tx=0x7fb354009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f2d55f9d61a7bf5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:13.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.685+0000 7fb36a014640 1 --2- 172.21.15.1:0/4060478071 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fb3500a2e90 0x7fb3500a5350 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb35c0cd030 tx=0x7fb35c06dd10 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:13.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.685+0000 7fb34ffff640 1 -- 172.21.15.1:0/3660542071 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb354035030 con 0x559bbef764a0 2024-09-11T21:58:13.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.686+0000 7fb34ffff640 1 -- 172.21.15.1:0/3660542071 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb354003ad0 con 0x559bbef764a0 2024-09-11T21:58:13.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.686+0000 7fb34ffff640 1 -- 172.21.15.1:0/3660542071 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb354031c20 con 0x559bbef764a0 2024-09-11T21:58:13.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.687+0000 7fb36de50740 1 -- 172.21.15.1:0/3660542071 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559bbef764a0 msgr2=0x559bbef76880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:13.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.687+0000 7fb36de50740 1 --2- 172.21.15.1:0/3660542071 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559bbef764a0 0x559bbef76880 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb354005940 tx=0x7fb354009560 comp rx=0 tx=0).stop 2024-09-11T21:58:13.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.687+0000 7fb36de50740 1 -- 172.21.15.1:0/3660542071 shutdown_connections 2024-09-11T21:58:13.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.687+0000 7fb36de50740 1 --2- 172.21.15.1:0/3660542071 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bbef81680 0x559bbef83a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:13.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.687+0000 7fb36de50740 1 --2- 172.21.15.1:0/3660542071 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559bbef76dc0 0x559bbef81140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:13.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.687+0000 7fb36de50740 1 --2- 172.21.15.1:0/3660542071 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559bbef764a0 0x559bbef76880 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:13.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.687+0000 7fb36de50740 1 -- 172.21.15.1:0/3660542071 >> 172.21.15.1:0/3660542071 conn(0x559bbef72a30 msgr2=0x559bbef72e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:13.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.687+0000 7fb36de50740 1 -- 172.21.15.1:0/3660542071 shutdown_connections 2024-09-11T21:58:13.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.687+0000 7fb36de50740 1 -- 172.21.15.1:0/3660542071 wait complete. 2024-09-11T21:58:13.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.688+0000 7fb36de50740 1 -- start start 2024-09-11T21:58:13.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.688+0000 7fb36de50740 1 Processor -- start 2024-09-11T21:58:13.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.689+0000 7fb36de50740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bbef764a0 0x559bbef81030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:13.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.689+0000 7fb36de50740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559bbef76dc0 0x559bbef7e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:13.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.689+0000 7fb36de50740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559bbef81680 0x559bbef7e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:13.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.689+0000 7fb369012640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bbef764a0 0x559bbef81030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:13.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.690+0000 7fb369813640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559bbef76dc0 0x559bbef7e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:13.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.690+0000 7fb369813640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559bbef76dc0 0x559bbef7e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42902/0 (socket says 172.21.15.1:42902) 2024-09-11T21:58:13.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.690+0000 7fb369012640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bbef764a0 0x559bbef81030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:41624/0 (socket says 172.21.15.1:41624) 2024-09-11T21:58:13.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.690+0000 7fb369813640 1 -- 172.21.15.1:0/1882793491 learned_addr learned my addr 172.21.15.1:0/1882793491 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:58:13.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.690+0000 7fb36de50740 1 -- 172.21.15.1:0/1882793491 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x559bbef88660 con 0x559bbef76dc0 2024-09-11T21:58:13.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.690+0000 7fb36de50740 1 -- 172.21.15.1:0/1882793491 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x559bbee97050 con 0x559bbef81680 2024-09-11T21:58:13.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.690+0000 7fb36de50740 1 -- 172.21.15.1:0/1882793491 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x559bbef887e0 con 0x559bbef764a0 2024-09-11T21:58:13.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.690+0000 7fb36de50740 10 client.0 _ll_register_callbacks cb 0x559bbeed7160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:58:13.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.690+0000 7fb369012640 1 --2- 172.21.15.1:0/1882793491 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559bbef81680 0x559bbef7e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:13.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.690+0000 7fb369813640 1 -- 172.21.15.1:0/1882793491 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bbef764a0 msgr2=0x559bbef81030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:13.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.690+0000 7fb369813640 1 --2- 172.21.15.1:0/1882793491 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bbef764a0 0x559bbef81030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:13.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.690+0000 7fb369813640 1 -- 172.21.15.1:0/1882793491 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559bbef81680 msgr2=0x559bbef7e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:13.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.690+0000 7fb369813640 1 --2- 172.21.15.1:0/1882793491 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559bbef81680 0x559bbef7e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:13.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.690+0000 7fb369813640 1 -- 172.21.15.1:0/1882793491 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bbec4d3c0 con 0x559bbef76dc0 2024-09-11T21:58:13.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.690+0000 7fb369012640 1 --2- 172.21.15.1:0/1882793491 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559bbef81680 0x559bbef7e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T21:58:13.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.691+0000 7fb369813640 1 --2- 172.21.15.1:0/1882793491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559bbef76dc0 0x559bbef7e2e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fb360005fd0 tx=0x7fb360013600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf4c4c24f70856d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:13.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.691+0000 7fb34d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T21:58:13.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.691+0000 7fb34d7fa640 1 -- 172.21.15.1:0/1882793491 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36001d030 con 0x559bbef76dc0 2024-09-11T21:58:13.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.691+0000 7fb34d7fa640 1 -- 172.21.15.1:0/1882793491 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7fb360004020 con 0x559bbef76dc0 2024-09-11T21:58:13.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.691+0000 7fb36de50740 1 -- 172.21.15.1:0/1882793491 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb354003ad0 con 0x559bbef76dc0 2024-09-11T21:58:13.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.691+0000 7fb34d7fa640 1 -- 172.21.15.1:0/1882793491 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb360027050 con 0x559bbef76dc0 2024-09-11T21:58:13.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.691+0000 7fb36de50740 10 client.15750 fetch_fsmap learned FSMap version 28 2024-09-11T21:58:13.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.691+0000 7fb36de50740 1 -- 172.21.15.1:0/1882793491 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559bbef94090 con 0x559bbef76dc0 2024-09-11T21:58:13.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.692+0000 7fb34d7fa640 1 -- 172.21.15.1:0/1882793491 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb354003ad0 con 0x559bbef76dc0 2024-09-11T21:58:13.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.693+0000 7fb34d7fa640 1 -- 172.21.15.1:0/1882793491 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb3600194f0 con 0x559bbef76dc0 2024-09-11T21:58:13.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.693+0000 7fb36de50740 10 client.15750 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T21:58:13.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.693+0000 7fb36de50740 10 client.15750 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T21:58:13.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.693+0000 7fb36de50740 20 client.15750 populate_metadata read hostname 'smithi001' 2024-09-11T21:58:13.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.693+0000 7fb36de50740 1 --2- 172.21.15.1:0/1882793491 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x559bbef95550 0x559bbef979a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:13.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.693+0000 7fb36de50740 4 client.15750 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T21:58:13.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.693+0000 7fb36de50740 1 -- 172.21.15.1:0/1882793491 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559bbeee9850 con 0x559bbef95550 2024-09-11T21:58:13.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.693+0000 7fb369012640 1 --2- 172.21.15.1:0/1882793491 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x559bbef95550 0x559bbef979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:13.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.693+0000 7fb34d7fa640 10 client.15750 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T21:58:13.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.693+0000 7fb369012640 1 --2- 172.21.15.1:0/1882793491 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x559bbef95550 0x559bbef979a0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78d95ba628bc75d6 server_cookie=ed5202ae9117d782 in_seq=0 out_seq=0 2024-09-11T21:58:13.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.694+0000 7fb34d7fa640 1 -- 172.21.15.1:0/1882793491 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559bbeee9850 con 0x559bbef95550 2024-09-11T21:58:13.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.694+0000 7fb34d7fa640 10 client.15750 handle_command_reply: tid=0 2024-09-11T21:58:13.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.694+0000 7fb36de50740 1 client.15750 shutdown 2024-09-11T21:58:13.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.694+0000 7fb36de50740 1 -- 172.21.15.1:0/1882793491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559bbef76dc0 msgr2=0x559bbef7e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:13.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.694+0000 7fb36de50740 1 --2- 172.21.15.1:0/1882793491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559bbef76dc0 0x559bbef7e2e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fb360005fd0 tx=0x7fb360013600 comp rx=0 tx=0).stop 2024-09-11T21:58:13.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.695+0000 7fb36de50740 1 -- 172.21.15.1:0/1882793491 shutdown_connections 2024-09-11T21:58:13.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.695+0000 7fb36de50740 1 --2- 172.21.15.1:0/1882793491 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x559bbef95550 0x559bbef979a0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:13.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.695+0000 7fb36de50740 1 --2- 172.21.15.1:0/1882793491 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559bbef81680 0x559bbef7e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:13.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.695+0000 7fb36de50740 1 --2- 172.21.15.1:0/1882793491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559bbef76dc0 0x559bbef7e2e0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:13.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.695+0000 7fb36de50740 1 --2- 172.21.15.1:0/1882793491 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bbef764a0 0x559bbef81030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:13.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.695+0000 7fb34d7fa640 0 client.15750 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T21:58:13.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.695+0000 7fb36de50740 1 -- 172.21.15.1:0/1882793491 >> 172.21.15.1:0/1882793491 conn(0x559bbef72a30 msgr2=0x559bbef71b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:13.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.695+0000 7fb36de50740 1 -- 172.21.15.1:0/1882793491 shutdown_connections 2024-09-11T21:58:13.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.695+0000 7fb36de50740 1 -- 172.21.15.1:0/1882793491 wait complete. 2024-09-11T21:58:13.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.695+0000 7fb36de50740 20 client.15750 trim_cache size 0 max 16384 2024-09-11T21:58:13.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.728+0000 7fb36de50740 1 Processor -- start 2024-09-11T21:58:13.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.728+0000 7fb36de50740 1 -- start start 2024-09-11T21:58:13.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.728+0000 7fb36de50740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bbef764a0 0x559bbef918c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:13.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.728+0000 7fb36de50740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559bbef76dc0 0x559bbef9d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:13.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.728+0000 7fb36de50740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559bbef81680 0x559bbef9d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:13.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.728+0000 7fb36de50740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb3641ab6e0 con 0x559bbef76dc0 2024-09-11T21:58:13.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.728+0000 7fb36de50740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb3641ad220 con 0x559bbef81680 2024-09-11T21:58:13.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.728+0000 7fb36de50740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb3641abdb0 con 0x559bbef764a0 2024-09-11T21:58:13.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.728+0000 7fb369813640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559bbef76dc0 0x559bbef9d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:13.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.729+0000 7fb369813640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559bbef76dc0 0x559bbef9d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42914/0 (socket says 172.21.15.1:42914) 2024-09-11T21:58:13.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.729+0000 7fb369813640 1 -- 172.21.15.1:0/4042757692 learned_addr learned my addr 172.21.15.1:0/4042757692 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:58:13.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.729+0000 7fb369012640 1 --2- 172.21.15.1:0/4042757692 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bbef764a0 0x559bbef918c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:13.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.729+0000 7fb369813640 1 -- 172.21.15.1:0/4042757692 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bbef764a0 msgr2=0x559bbef918c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:13.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.729+0000 7fb369813640 1 --2- 172.21.15.1:0/4042757692 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bbef764a0 0x559bbef918c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:13.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.729+0000 7fb369813640 1 -- 172.21.15.1:0/4042757692 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559bbef81680 msgr2=0x559bbef9d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:13.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.729+0000 7fb369813640 1 --2- 172.21.15.1:0/4042757692 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559bbef81680 0x559bbef9d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:13.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.729+0000 7fb369813640 1 -- 172.21.15.1:0/4042757692 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bbef7ebb0 con 0x559bbef76dc0 2024-09-11T21:58:13.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.729+0000 7fb369813640 1 --2- 172.21.15.1:0/4042757692 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559bbef76dc0 0x559bbef9d270 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fb360013b00 tx=0x7fb360019d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a33acd7046257ad server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:13.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.729+0000 7fb369012640 1 --2- 172.21.15.1:0/4042757692 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bbef764a0 0x559bbef918c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T21:58:13.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.729+0000 7fb34f7fe640 1 -- 172.21.15.1:0/4042757692 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36001d070 con 0x559bbef76dc0 2024-09-11T21:58:13.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.729+0000 7fb34f7fe640 1 -- 172.21.15.1:0/4042757692 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb360019660 con 0x559bbef76dc0 2024-09-11T21:58:13.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.730+0000 7fb34f7fe640 1 -- 172.21.15.1:0/4042757692 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb360004a20 con 0x559bbef76dc0 2024-09-11T21:58:13.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.731+0000 7fb36de50740 1 -- 172.21.15.1:0/4042757692 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559bbef76dc0 msgr2=0x559bbef9d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:13.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.731+0000 7fb36de50740 1 --2- 172.21.15.1:0/4042757692 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559bbef76dc0 0x559bbef9d270 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fb360013b00 tx=0x7fb360019d60 comp rx=0 tx=0).stop 2024-09-11T21:58:13.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.731+0000 7fb36de50740 1 -- 172.21.15.1:0/4042757692 shutdown_connections 2024-09-11T21:58:13.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.731+0000 7fb36de50740 1 --2- 172.21.15.1:0/4042757692 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559bbef81680 0x559bbef9d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:13.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.731+0000 7fb36de50740 1 --2- 172.21.15.1:0/4042757692 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559bbef76dc0 0x559bbef9d270 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:13.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.731+0000 7fb36de50740 1 --2- 172.21.15.1:0/4042757692 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bbef764a0 0x559bbef918c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:13.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.731+0000 7fb36de50740 1 -- 172.21.15.1:0/4042757692 >> 172.21.15.1:0/4042757692 conn(0x559bbef72a30 msgr2=0x559bbef7d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:13.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.731+0000 7fb36de50740 1 -- 172.21.15.1:0/4042757692 shutdown_connections 2024-09-11T21:58:13.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.731+0000 7fb36de50740 1 -- 172.21.15.1:0/4042757692 wait complete. 2024-09-11T21:58:13.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.731+0000 7fb36de50740 1 -- start start 2024-09-11T21:58:13.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.732+0000 7fb36de50740 1 Processor -- start 2024-09-11T21:58:13.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.732+0000 7fb36de50740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559bbef764a0 0x559bbef8ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:13.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.732+0000 7fb369012640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559bbef764a0 0x559bbef8ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:13.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.733+0000 7fb369012640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559bbef764a0 0x559bbef8ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42920/0 (socket says 172.21.15.1:42920) 2024-09-11T21:58:13.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.732+0000 7fb36de50740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bbef76dc0 0x559bbef8e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:13.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.733+0000 7fb369012640 1 -- 172.21.15.1:0/805878308 learned_addr learned my addr 172.21.15.1:0/805878308 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:58:13.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.733+0000 7fb36de50740 1 --2- 172.21.15.1:0/805878308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559bbef81680 0x559bbef8b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:13.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.733+0000 7fb36de50740 1 -- 172.21.15.1:0/805878308 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb3641acb50 con 0x559bbef764a0 2024-09-11T21:58:13.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.733+0000 7fb36de50740 1 -- 172.21.15.1:0/805878308 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb3641ac480 con 0x559bbef81680 2024-09-11T21:58:13.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.733+0000 7fb36de50740 1 -- 172.21.15.1:0/805878308 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb3641a96c0 con 0x559bbef76dc0 2024-09-11T21:58:13.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.733+0000 7fb369813640 1 --2- 172.21.15.1:0/805878308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bbef76dc0 0x559bbef8e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:13.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.733+0000 7fb369012640 1 -- 172.21.15.1:0/805878308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bbef76dc0 msgr2=0x559bbef8e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:13.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.733+0000 7fb369012640 1 --2- 172.21.15.1:0/805878308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bbef76dc0 0x559bbef8e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:13.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.733+0000 7fb369012640 1 -- 172.21.15.1:0/805878308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559bbef81680 msgr2=0x559bbef8b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:13.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.733+0000 7fb369012640 1 --2- 172.21.15.1:0/805878308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559bbef81680 0x559bbef8b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:13.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.733+0000 7fb369012640 1 -- 172.21.15.1:0/805878308 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bbef8bbe0 con 0x559bbef764a0 2024-09-11T21:58:13.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.733+0000 7fb369813640 1 --2- 172.21.15.1:0/805878308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bbef76dc0 0x559bbef8e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:58:13.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.734+0000 7fb369012640 1 --2- 172.21.15.1:0/805878308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559bbef764a0 0x559bbef8ddc0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fb3540314e0 tx=0x7fb354005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe498acca34cdd3b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:13.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.734+0000 7fb34d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T21:58:13.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.734+0000 7fb34d7fa640 1 -- 172.21.15.1:0/805878308 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb354035040 con 0x559bbef764a0 2024-09-11T21:58:13.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.734+0000 7fb34d7fa640 1 -- 172.21.15.1:0/805878308 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8527+0+0 (secure 0 0 0) 0x7fb354031b30 con 0x559bbef764a0 2024-09-11T21:58:13.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.734+0000 7fb36de50740 10 client.0 _ll_register_callbacks cb 0x7fb36000bec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:58:13.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.734+0000 7fb36de50740 1 -- 172.21.15.1:0/805878308 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb360019660 con 0x559bbef764a0 2024-09-11T21:58:13.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.734+0000 7fb34d7fa640 1 -- 172.21.15.1:0/805878308 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb354060050 con 0x559bbef764a0 2024-09-11T21:58:13.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.734+0000 7fb36de50740 10 client.15756 fetch_fsmap learned FSMap version 28 2024-09-11T21:58:13.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.734+0000 7fb36de50740 1 -- 172.21.15.1:0/805878308 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({fsmap=28}) -- 0x559bbefffc90 con 0x559bbef764a0 2024-09-11T21:58:13.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.735+0000 7fb34d7fa640 1 -- 172.21.15.1:0/805878308 <== mon.0 v2:172.21.15.1:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb3540035f0 con 0x559bbef764a0 2024-09-11T21:58:13.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.735+0000 7fb36de50740 10 client.15756 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T21:58:13.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.735+0000 7fb36de50740 10 client.15756 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T21:58:13.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.735+0000 7fb36de50740 20 client.15756 populate_metadata read hostname 'smithi001' 2024-09-11T21:58:13.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.735+0000 7fb36de50740 1 --2- 172.21.15.1:0/805878308 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x559bbefffeb0 0x559bbf0022a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:13.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.735+0000 7fb36de50740 4 client.15756 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T21:58:13.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.735+0000 7fb36de50740 1 -- 172.21.15.1:0/805878308 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fb360019660 con 0x559bbefffeb0 2024-09-11T21:58:13.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.735+0000 7fb369813640 1 --2- 172.21.15.1:0/805878308 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x559bbefffeb0 0x559bbf0022a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:13.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.736+0000 7fb34d7fa640 10 client.15756 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T21:58:13.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.736+0000 7fb369813640 1 --2- 172.21.15.1:0/805878308 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x559bbefffeb0 0x559bbf0022a0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fc7278d18d0403f server_cookie=a653484fb2164d47 in_seq=0 out_seq=0 2024-09-11T21:58:13.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.736+0000 7fb34d7fa640 1 -- 172.21.15.1:0/805878308 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7fb360019660 con 0x559bbefffeb0 2024-09-11T21:58:13.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.736+0000 7fb34d7fa640 10 client.15756 handle_command_reply: tid=0 2024-09-11T21:58:13.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.736+0000 7fb36de50740 1 client.15756 shutdown 2024-09-11T21:58:13.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.736+0000 7fb36de50740 1 -- 172.21.15.1:0/805878308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559bbef764a0 msgr2=0x559bbef8ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:13.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.736+0000 7fb36de50740 1 --2- 172.21.15.1:0/805878308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559bbef764a0 0x559bbef8ddc0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fb3540314e0 tx=0x7fb354005760 comp rx=0 tx=0).stop 2024-09-11T21:58:13.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.737+0000 7fb36de50740 1 -- 172.21.15.1:0/805878308 shutdown_connections 2024-09-11T21:58:13.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.737+0000 7fb36de50740 1 --2- 172.21.15.1:0/805878308 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x559bbefffeb0 0x559bbf0022a0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:13.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.737+0000 7fb36de50740 1 --2- 172.21.15.1:0/805878308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559bbef81680 0x559bbef8b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:13.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.737+0000 7fb36de50740 1 --2- 172.21.15.1:0/805878308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559bbef76dc0 0x559bbef8e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:13.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.737+0000 7fb36de50740 1 --2- 172.21.15.1:0/805878308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559bbef764a0 0x559bbef8ddc0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:13.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.737+0000 7fb36de50740 1 -- 172.21.15.1:0/805878308 >> 172.21.15.1:0/805878308 conn(0x559bbef72a30 msgr2=0x559bbef7fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:13.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.737+0000 7fb34d7fa640 0 client.15756 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T21:58:13.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.737+0000 7fb36de50740 1 -- 172.21.15.1:0/805878308 shutdown_connections 2024-09-11T21:58:13.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.737+0000 7fb36de50740 1 -- 172.21.15.1:0/805878308 wait complete. 2024-09-11T21:58:13.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.737+0000 7fb36de50740 20 client.15756 trim_cache size 0 max 16384 2024-09-11T21:58:13.742 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-11T21:58:13.742 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T21:58:13.743 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-09-11T21:58:13.743 INFO:teuthology.orchestra.run.smithi001.stdout: "672dce12-47f2-4540-97eb-f68a653b4be6": { 2024-09-11T21:58:13.743 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2024-09-11T21:58:13.743 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "672dce12-47f2-4540-97eb-f68a653b4be6", 2024-09-11T21:58:13.743 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2024-09-11T21:58:13.743 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T21:58:13.743 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T21:58:13.743 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-11T21:58:13.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.738+0000 7fb36ba77640 1 -- 172.21.15.1:0/4060478071 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fb3500a2e90 msgr2=0x7fb3500a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:13.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.738+0000 7fb36ba77640 1 --2- 172.21.15.1:0/4060478071 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fb3500a2e90 0x7fb3500a5350 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb35c0cd030 tx=0x7fb35c06dd10 comp rx=0 tx=0).stop 2024-09-11T21:58:13.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.738+0000 7fb36ba77640 1 -- 172.21.15.1:0/4060478071 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb364102ef0 msgr2=0x7fb36419ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:13.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.738+0000 7fb36ba77640 1 --2- 172.21.15.1:0/4060478071 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb364102ef0 0x7fb36419ab50 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fb35c00eaa0 tx=0x7fb35c00ef60 comp rx=0 tx=0).stop 2024-09-11T21:58:13.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.738+0000 7fb36ba77640 1 -- 172.21.15.1:0/4060478071 shutdown_connections 2024-09-11T21:58:13.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.738+0000 7fb36ba77640 1 --2- 172.21.15.1:0/4060478071 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fb3500a2e90 0x7fb3500a5350 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:13.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.738+0000 7fb36ba77640 1 --2- 172.21.15.1:0/4060478071 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb364102ef0 0x7fb36419ab50 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:13.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.738+0000 7fb36ba77640 1 --2- 172.21.15.1:0/4060478071 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb364102570 0x7fb3641967c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:13.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.738+0000 7fb36ba77640 1 --2- 172.21.15.1:0/4060478071 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb364100e20 0x7fb364196280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:13.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.738+0000 7fb36ba77640 1 -- 172.21.15.1:0/4060478071 >> 172.21.15.1:0/4060478071 conn(0x7fb3640fc7d0 msgr2=0x7fb3640febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:13.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.738+0000 7fb36ba77640 1 -- 172.21.15.1:0/4060478071 shutdown_connections 2024-09-11T21:58:13.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:13.738+0000 7fb36ba77640 1 -- 172.21.15.1:0/4060478071 wait complete. 2024-09-11T21:58:14.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:58:14 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-09-11T21:58:14.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:58:14 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-09-11T21:58:14.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:58:14 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-09-11T21:58:14.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:58:14 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-09-11T21:58:14.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:58:14 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-09-11T21:58:14.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:58:14 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-09-11T21:58:14.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:58:14 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-09-11T21:58:14.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:58:14 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-09-11T21:58:14.365 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:58:14 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-09-11T21:58:14.365 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:58:14 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-09-11T21:58:14.365 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:58:14 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-09-11T21:58:14.365 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:58:14 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-09-11T21:58:14.405 INFO:tasks.cephfs.filesystem:scrub status for tag:672dce12-47f2-4540-97eb-f68a653b4be6 - {'path': '/', 'tag': '672dce12-47f2-4540-97eb-f68a653b4be6', 'options': 'recursive,force'} 2024-09-11T21:58:15.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:58:15 smithi161 ceph-mon[35615]: pgmap v33: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 26 op/s 2024-09-11T21:58:15.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:58:15 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-11T21:58:15.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:58:15 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-09-11T21:58:15.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:58:15 smithi161 ceph-mon[35615]: osdmap e88: 12 total, 12 up, 12 in 2024-09-11T21:58:15.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:58:15 smithi001 ceph-mon[30935]: pgmap v33: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 26 op/s 2024-09-11T21:58:15.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:58:15 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-11T21:58:15.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:58:15 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-09-11T21:58:15.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:58:15 smithi001 ceph-mon[30935]: osdmap e88: 12 total, 12 up, 12 in 2024-09-11T21:58:15.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:58:15 smithi053 ceph-mon[34908]: pgmap v33: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 26 op/s 2024-09-11T21:58:15.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:58:15 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-11T21:58:15.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:58:15 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-09-11T21:58:15.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:58:15 smithi053 ceph-mon[34908]: osdmap e88: 12 total, 12 up, 12 in 2024-09-11T21:58:16.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:58:16 smithi161 ceph-mon[35615]: osdmap e89: 12 total, 12 up, 12 in 2024-09-11T21:58:16.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:58:16 smithi161 ceph-mon[35615]: scrub summary: idle 2024-09-11T21:58:16.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:58:16 smithi001 ceph-mon[30935]: osdmap e89: 12 total, 12 up, 12 in 2024-09-11T21:58:16.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:58:16 smithi001 ceph-mon[30935]: scrub summary: idle 2024-09-11T21:58:16.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:58:16 smithi053 ceph-mon[34908]: osdmap e89: 12 total, 12 up, 12 in 2024-09-11T21:58:16.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:58:16 smithi053 ceph-mon[34908]: scrub summary: idle 2024-09-11T21:58:17.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:58:17 smithi161 ceph-mon[35615]: pgmap v36: 97 pgs: 2 peering, 95 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 3.1 KiB/s wr, 17 op/s 2024-09-11T21:58:17.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:58:17 smithi001 ceph-mon[30935]: pgmap v36: 97 pgs: 2 peering, 95 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 3.1 KiB/s wr, 17 op/s 2024-09-11T21:58:17.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:58:17 smithi053 ceph-mon[34908]: pgmap v36: 97 pgs: 2 peering, 95 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 3.1 KiB/s wr, 17 op/s 2024-09-11T21:58:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.573+0000 7fa7bd8f1640 1 -- 172.21.15.1:0/874558016 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7b81009a0 msgr2=0x7fa7b8100e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.573+0000 7fa7bd8f1640 1 --2- 172.21.15.1:0/874558016 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7b81009a0 0x7fa7b8100e00 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0009930 tx=0x7fa7a002a420 comp rx=0 tx=0).stop 2024-09-11T21:58:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.573+0000 7fa7bd8f1640 1 -- 172.21.15.1:0/874558016 shutdown_connections 2024-09-11T21:58:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.573+0000 7fa7bd8f1640 1 --2- 172.21.15.1:0/874558016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7b8101340 0x7fa7b810d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.573+0000 7fa7bd8f1640 1 --2- 172.21.15.1:0/874558016 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7b81009a0 0x7fa7b8100e00 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.573+0000 7fa7bd8f1640 1 --2- 172.21.15.1:0/874558016 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7b81069b0 0x7fa7b8106d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.573+0000 7fa7bd8f1640 1 -- 172.21.15.1:0/874558016 >> 172.21.15.1:0/874558016 conn(0x7fa7b80779a0 msgr2=0x7fa7b80fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.573+0000 7fa7bd8f1640 1 -- 172.21.15.1:0/874558016 shutdown_connections 2024-09-11T21:58:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.574+0000 7fa7bd8f1640 1 -- 172.21.15.1:0/874558016 wait complete. 2024-09-11T21:58:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.574+0000 7fa7bd8f1640 1 Processor -- start 2024-09-11T21:58:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.574+0000 7fa7bd8f1640 1 -- start start 2024-09-11T21:58:18.577 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.575+0000 7fa7bd8f1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7b81009a0 0x7fa7b81993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:18.577 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.575+0000 7fa7bd8f1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7b8101340 0x7fa7b81998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:18.577 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.575+0000 7fa7bd8f1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7b81069b0 0x7fa7b8199e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:18.577 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.575+0000 7fa7bd8f1640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b8110e40 con 0x7fa7b81069b0 2024-09-11T21:58:18.578 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.575+0000 7fa7bd8f1640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b8110cc0 con 0x7fa7b8101340 2024-09-11T21:58:18.578 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.575+0000 7fa7bd8f1640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b8110fc0 con 0x7fa7b81009a0 2024-09-11T21:58:18.578 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.575+0000 7fa7b6ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7b81009a0 0x7fa7b81993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:18.578 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.575+0000 7fa7b77fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7b81069b0 0x7fa7b8199e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:18.578 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.575+0000 7fa7b6ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7b81009a0 0x7fa7b81993b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:45270/0 (socket says 172.21.15.1:45270) 2024-09-11T21:58:18.578 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.575+0000 7fa7b6ffd640 1 -- 172.21.15.1:0/1508228674 learned_addr learned my addr 172.21.15.1:0/1508228674 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:58:18.578 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.575+0000 7fa7b77fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7b81069b0 0x7fa7b8199e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53200/0 (socket says 172.21.15.1:53200) 2024-09-11T21:58:18.578 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.575+0000 7fa7affff640 1 --2- 172.21.15.1:0/1508228674 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7b8101340 0x7fa7b81998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:18.578 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.575+0000 7fa7b6ffd640 1 -- 172.21.15.1:0/1508228674 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7b8101340 msgr2=0x7fa7b81998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:18.578 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.575+0000 7fa7b6ffd640 1 --2- 172.21.15.1:0/1508228674 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7b8101340 0x7fa7b81998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:18.578 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.575+0000 7fa7b6ffd640 1 -- 172.21.15.1:0/1508228674 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7b81069b0 msgr2=0x7fa7b8199e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:18.578 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.575+0000 7fa7b6ffd640 1 --2- 172.21.15.1:0/1508228674 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7b81069b0 0x7fa7b8199e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:18.579 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.575+0000 7fa7b6ffd640 1 -- 172.21.15.1:0/1508228674 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7b81a47b0 con 0x7fa7b81009a0 2024-09-11T21:58:18.579 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.575+0000 7fa7b77fe640 1 --2- 172.21.15.1:0/1508228674 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7b81069b0 0x7fa7b8199e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T21:58:18.579 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.576+0000 7fa7b6ffd640 1 --2- 172.21.15.1:0/1508228674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7b81009a0 0x7fa7b81993b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8009f40 tx=0x7fa7a800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:18.579 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.576+0000 7fa7bc8ef640 1 -- 172.21.15.1:0/1508228674 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a800cd80 con 0x7fa7b81009a0 2024-09-11T21:58:18.579 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.576+0000 7fa7bd8f1640 1 -- 172.21.15.1:0/1508228674 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7b8197200 con 0x7fa7b81009a0 2024-09-11T21:58:18.579 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.576+0000 7fa7bc8ef640 1 -- 172.21.15.1:0/1508228674 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7a8005500 con 0x7fa7b81009a0 2024-09-11T21:58:18.579 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.576+0000 7fa7bc8ef640 1 -- 172.21.15.1:0/1508228674 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a8004a00 con 0x7fa7b81009a0 2024-09-11T21:58:18.579 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.577+0000 7fa7bd8f1640 1 -- 172.21.15.1:0/1508228674 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7b81976d0 con 0x7fa7b81009a0 2024-09-11T21:58:18.581 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.580+0000 7fa7c068b740 1 Processor -- start 2024-09-11T21:58:18.581 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.580+0000 7fa7bc8ef640 1 -- 172.21.15.1:0/1508228674 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 25) ==== 138065+0+0 (secure 0 0 0) 0x7fa7a8004ba0 con 0x7fa7b81009a0 2024-09-11T21:58:18.582 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.580+0000 7fa7c068b740 1 -- start start 2024-09-11T21:58:18.582 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.581+0000 7fa7c068b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d780857af0 0x55d780857ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:18.582 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.581+0000 7fa7c068b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d780858410 0x55d780862770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:18.582 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.581+0000 7fa7c068b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d780862cb0 0x55d7808650a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:18.582 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.581+0000 7fa7c068b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55d7807bc2f0 con 0x55d780858410 2024-09-11T21:58:18.583 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.581+0000 7fa7c068b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55d780764270 con 0x55d780862cb0 2024-09-11T21:58:18.583 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.581+0000 7fa7c068b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d7807b82b0 con 0x55d780857af0 2024-09-11T21:58:18.583 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.581+0000 7fa7b6ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d780858410 0x55d780862770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:18.583 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.581+0000 7fa7b6ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d780858410 0x55d780862770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53206/0 (socket says 172.21.15.1:53206) 2024-09-11T21:58:18.583 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.581+0000 7fa7b6ffd640 1 -- 172.21.15.1:0/1870797663 learned_addr learned my addr 172.21.15.1:0/1870797663 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:58:18.583 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.581+0000 7fa7affff640 1 --2- 172.21.15.1:0/1870797663 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d780862cb0 0x55d7808650a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:18.583 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.581+0000 7fa7b77fe640 1 --2- 172.21.15.1:0/1870797663 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d780857af0 0x55d780857ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:18.583 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.581+0000 7fa7b6ffd640 1 -- 172.21.15.1:0/1870797663 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d780857af0 msgr2=0x55d780857ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:18.583 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.581+0000 7fa7b6ffd640 1 --2- 172.21.15.1:0/1870797663 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d780857af0 0x55d780857ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:18.583 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.581+0000 7fa7b6ffd640 1 -- 172.21.15.1:0/1870797663 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d780862cb0 msgr2=0x55d7808650a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:18.583 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.581+0000 7fa7b6ffd640 1 --2- 172.21.15.1:0/1870797663 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d780862cb0 0x55d7808650a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:18.583 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.581+0000 7fa7b6ffd640 1 -- 172.21.15.1:0/1870797663 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d780763340 con 0x55d780858410 2024-09-11T21:58:18.584 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.581+0000 7fa7affff640 1 --2- 172.21.15.1:0/1870797663 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d780862cb0 0x55d7808650a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T21:58:18.584 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.581+0000 7fa7bc8ef640 1 --2- 172.21.15.1:0/1508228674 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fa7a40a2e10 0x7fa7a40a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:18.584 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.582+0000 7fa7b6ffd640 1 --2- 172.21.15.1:0/1870797663 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d780858410 0x55d780862770 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa7a80d0f30 tx=0x7fa7a80d9b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50dd7221b16b7bba server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:18.584 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.582+0000 7fa7b77fe640 1 --2- 172.21.15.1:0/1508228674 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fa7a40a2e10 0x7fa7a40a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:18.584 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.582+0000 7fa7bc8ef640 1 -- 172.21.15.1:0/1508228674 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7fa7a80d3950 con 0x7fa7b81009a0 2024-09-11T21:58:18.586 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.582+0000 7fa7adffb640 1 -- 172.21.15.1:0/1870797663 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a809bfe0 con 0x55d780858410 2024-09-11T21:58:18.587 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.582+0000 7fa7adffb640 1 -- 172.21.15.1:0/1870797663 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7a809fd40 con 0x55d780858410 2024-09-11T21:58:18.587 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.582+0000 7fa7adffb640 1 -- 172.21.15.1:0/1870797663 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a80056b0 con 0x55d780858410 2024-09-11T21:58:18.587 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.583+0000 7fa7c068b740 1 -- 172.21.15.1:0/1870797663 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d780858410 msgr2=0x55d780862770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:18.587 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.583+0000 7fa7b77fe640 1 --2- 172.21.15.1:0/1508228674 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fa7a40a2e10 0x7fa7a40a52d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa7b0005fd0 tx=0x7fa7b0009500 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:18.587 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.583+0000 7fa7c068b740 1 --2- 172.21.15.1:0/1870797663 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d780858410 0x55d780862770 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa7a80d0f30 tx=0x7fa7a80d9b40 comp rx=0 tx=0).stop 2024-09-11T21:58:18.587 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.583+0000 7fa7c068b740 1 -- 172.21.15.1:0/1870797663 shutdown_connections 2024-09-11T21:58:18.587 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.583+0000 7fa7c068b740 1 --2- 172.21.15.1:0/1870797663 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d780862cb0 0x55d7808650a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:18.587 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.583+0000 7fa7c068b740 1 --2- 172.21.15.1:0/1870797663 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d780858410 0x55d780862770 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:18.587 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.583+0000 7fa7c068b740 1 --2- 172.21.15.1:0/1870797663 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d780857af0 0x55d780857ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:18.588 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.583+0000 7fa7c068b740 1 -- 172.21.15.1:0/1870797663 >> 172.21.15.1:0/1870797663 conn(0x55d780854060 msgr2=0x55d780854440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:18.588 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.583+0000 7fa7c068b740 1 -- 172.21.15.1:0/1870797663 shutdown_connections 2024-09-11T21:58:18.588 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.583+0000 7fa7c068b740 1 -- 172.21.15.1:0/1870797663 wait complete. 2024-09-11T21:58:18.588 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.584+0000 7fa7c068b740 1 -- start start 2024-09-11T21:58:18.588 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.584+0000 7fa7c068b740 1 Processor -- start 2024-09-11T21:58:18.588 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.584+0000 7fa7c068b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d780857af0 0x55d78085fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:18.588 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.584+0000 7fa7c068b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d780858410 0x55d7808601d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:18.588 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.584+0000 7fa7c068b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d780862cb0 0x55d780860710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:18.588 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.584+0000 7fa7affff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d780858410 0x55d7808601d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:18.588 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.585+0000 7fa7affff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d780858410 0x55d7808601d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53220/0 (socket says 172.21.15.1:53220) 2024-09-11T21:58:18.588 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.585+0000 7fa7b6ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d780857af0 0x55d78085fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:18.588 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.585+0000 7fa7b6ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d780857af0 0x55d78085fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:45294/0 (socket says 172.21.15.1:45294) 2024-09-11T21:58:18.589 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.585+0000 7fa7affff640 1 -- 172.21.15.1:0/1291006296 learned_addr learned my addr 172.21.15.1:0/1291006296 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:58:18.589 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.585+0000 7fa7c068b740 1 -- 172.21.15.1:0/1291006296 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55d780865880 con 0x55d780858410 2024-09-11T21:58:18.589 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.585+0000 7fa7c068b740 1 -- 172.21.15.1:0/1291006296 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55d780745290 con 0x55d780862cb0 2024-09-11T21:58:18.589 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.585+0000 7fa7c068b740 1 -- 172.21.15.1:0/1291006296 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d780865a00 con 0x55d780857af0 2024-09-11T21:58:18.589 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.585+0000 7fa7affff640 1 -- 172.21.15.1:0/1291006296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d780857af0 msgr2=0x55d78085fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:18.589 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.585+0000 7fa7affff640 1 --2- 172.21.15.1:0/1291006296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d780857af0 0x55d78085fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:18.589 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.585+0000 7fa7affff640 1 -- 172.21.15.1:0/1291006296 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d780862cb0 msgr2=0x55d780860710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:18.589 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.585+0000 7fa7affff640 1 --2- 172.21.15.1:0/1291006296 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d780862cb0 0x55d780860710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:18.589 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.585+0000 7fa7affff640 1 -- 172.21.15.1:0/1291006296 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7804cf210 con 0x55d780858410 2024-09-11T21:58:18.589 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.585+0000 7fa7c068b740 10 client.0 _ll_register_callbacks cb 0x55d780776140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:58:18.589 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.586+0000 7fa7c068b740 1 -- 172.21.15.1:0/1291006296 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7a809fd40 con 0x55d780858410 2024-09-11T21:58:18.589 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.586+0000 7fa7affff640 1 --2- 172.21.15.1:0/1291006296 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d780858410 0x55d7808601d0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fa7a002aa10 tx=0x7fa7a00043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69afda0d096b6369 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:18.590 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.586+0000 7fa796ffd640 10 client.15768 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T21:58:18.590 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.586+0000 7fa796ffd640 1 -- 172.21.15.1:0/1291006296 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a0035040 con 0x55d780858410 2024-09-11T21:58:18.590 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.586+0000 7fa796ffd640 1 -- 172.21.15.1:0/1291006296 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7fa7a0005450 con 0x55d780858410 2024-09-11T21:58:18.590 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.586+0000 7fa796ffd640 1 -- 172.21.15.1:0/1291006296 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa7a00046c0 con 0x55d780858410 2024-09-11T21:58:18.590 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.586+0000 7fa7c068b740 10 client.15768 fetch_fsmap learned FSMap version 28 2024-09-11T21:58:18.590 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.586+0000 7fa7c068b740 1 -- 172.21.15.1:0/1291006296 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({fsmap=28}) -- 0x55d78059b620 con 0x55d780858410 2024-09-11T21:58:18.592 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.587+0000 7fa796ffd640 1 -- 172.21.15.1:0/1291006296 <== mon.0 v2:172.21.15.1:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa7a0004860 con 0x55d780858410 2024-09-11T21:58:18.592 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.587+0000 7fa7c068b740 10 client.15768 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T21:58:18.592 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.587+0000 7fa7c068b740 10 client.15768 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T21:58:18.592 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.587+0000 7fa7c068b740 20 client.15768 populate_metadata read hostname 'smithi001' 2024-09-11T21:58:18.592 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.587+0000 7fa7c068b740 1 --2- 172.21.15.1:0/1291006296 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55d780872ad0 0x55d780874f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:18.592 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.587+0000 7fa7c068b740 4 client.15768 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T21:58:18.592 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.587+0000 7fa7c068b740 1 -- 172.21.15.1:0/1291006296 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7807c9a20 con 0x55d780872ad0 2024-09-11T21:58:18.592 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.587+0000 7fa7b6ffd640 1 --2- 172.21.15.1:0/1291006296 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55d780872ad0 0x55d780874f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:18.592 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.588+0000 7fa796ffd640 10 client.15768 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T21:58:18.592 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.588+0000 7fa7b6ffd640 1 --2- 172.21.15.1:0/1291006296 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55d780872ad0 0x55d780874f60 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=826179e3118eb5e6 server_cookie=1d96c3a8b3b469db in_seq=0 out_seq=0 2024-09-11T21:58:18.593 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.588+0000 7fa796ffd640 1 -- 172.21.15.1:0/1291006296 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d7807c9a20 con 0x55d780872ad0 2024-09-11T21:58:18.593 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.588+0000 7fa796ffd640 10 client.15768 handle_command_reply: tid=0 2024-09-11T21:58:18.593 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.588+0000 7fa7c068b740 1 client.15768 shutdown 2024-09-11T21:58:18.593 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.589+0000 7fa7c068b740 1 -- 172.21.15.1:0/1291006296 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d780858410 msgr2=0x55d7808601d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:18.593 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.589+0000 7fa7c068b740 1 --2- 172.21.15.1:0/1291006296 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d780858410 0x55d7808601d0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fa7a002aa10 tx=0x7fa7a00043b0 comp rx=0 tx=0).stop 2024-09-11T21:58:18.593 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.589+0000 7fa7c068b740 1 -- 172.21.15.1:0/1291006296 shutdown_connections 2024-09-11T21:58:18.593 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.589+0000 7fa7c068b740 1 --2- 172.21.15.1:0/1291006296 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55d780872ad0 0x55d780874f60 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:18.593 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.589+0000 7fa7c068b740 1 --2- 172.21.15.1:0/1291006296 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d780862cb0 0x55d780860710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:18.593 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.589+0000 7fa7c068b740 1 --2- 172.21.15.1:0/1291006296 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d780858410 0x55d7808601d0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:18.593 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.589+0000 7fa7c068b740 1 --2- 172.21.15.1:0/1291006296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d780857af0 0x55d78085fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:18.593 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.589+0000 7fa7c068b740 1 -- 172.21.15.1:0/1291006296 >> 172.21.15.1:0/1291006296 conn(0x55d780854060 msgr2=0x55d780859d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:18.593 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.589+0000 7fa796ffd640 0 client.15768 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T21:58:18.594 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.589+0000 7fa7c068b740 1 -- 172.21.15.1:0/1291006296 shutdown_connections 2024-09-11T21:58:18.594 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.589+0000 7fa7c068b740 1 -- 172.21.15.1:0/1291006296 wait complete. 2024-09-11T21:58:18.594 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.589+0000 7fa7c068b740 20 client.15768 trim_cache size 0 max 16384 2024-09-11T21:58:18.619 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.618+0000 7fa7c068b740 1 Processor -- start 2024-09-11T21:58:18.619 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.618+0000 7fa7c068b740 1 -- start start 2024-09-11T21:58:18.619 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.618+0000 7fa7c068b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d780857af0 0x55d780860540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:18.620 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.618+0000 7fa7c068b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d780858410 0x55d780861c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:18.620 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.618+0000 7fa7c068b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d780862cb0 0x55d78087a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:18.620 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.619+0000 7fa7c068b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b81ab780 con 0x55d780862cb0 2024-09-11T21:58:18.620 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.619+0000 7fa7b77fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d780862cb0 0x55d78087a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:18.620 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.619+0000 7fa7b77fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d780862cb0 0x55d78087a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53222/0 (socket says 172.21.15.1:53222) 2024-09-11T21:58:18.620 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.619+0000 7fa7b77fe640 1 -- 172.21.15.1:0/2883846973 learned_addr learned my addr 172.21.15.1:0/2883846973 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:58:18.620 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.619+0000 7fa7b6ffd640 1 --2- 172.21.15.1:0/2883846973 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d780857af0 0x55d780860540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:18.620 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.619+0000 7fa7c068b740 1 -- 172.21.15.1:0/2883846973 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b81ad2c0 con 0x55d780857af0 2024-09-11T21:58:18.620 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.619+0000 7fa7affff640 1 --2- 172.21.15.1:0/2883846973 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d780858410 0x55d780861c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:18.620 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.619+0000 7fa7c068b740 1 -- 172.21.15.1:0/2883846973 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b81abe50 con 0x55d780858410 2024-09-11T21:58:18.622 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.619+0000 7fa7affff640 1 -- 172.21.15.1:0/2883846973 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d780857af0 msgr2=0x55d780860540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:18.622 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.619+0000 7fa7affff640 1 --2- 172.21.15.1:0/2883846973 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d780857af0 0x55d780860540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:18.622 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.619+0000 7fa7affff640 1 -- 172.21.15.1:0/2883846973 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d780862cb0 msgr2=0x55d78087a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:18.622 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.619+0000 7fa7affff640 1 --2- 172.21.15.1:0/2883846973 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d780862cb0 0x55d78087a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:18.622 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.619+0000 7fa7affff640 1 -- 172.21.15.1:0/2883846973 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d780860a80 con 0x55d780858410 2024-09-11T21:58:18.622 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.619+0000 7fa7b77fe640 1 --2- 172.21.15.1:0/2883846973 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d780862cb0 0x55d78087a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T21:58:18.622 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.619+0000 7fa7b6ffd640 1 --2- 172.21.15.1:0/2883846973 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d780857af0 0x55d780860540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:58:18.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.620+0000 7fa7affff640 1 --2- 172.21.15.1:0/2883846973 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d780858410 0x55d780861c00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0006010 tx=0x7fa7a003f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97c1aa3b781a0ee8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:18.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.620+0000 7fa7ad7fa640 1 -- 172.21.15.1:0/2883846973 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a0035070 con 0x55d780858410 2024-09-11T21:58:18.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.620+0000 7fa7ad7fa640 1 -- 172.21.15.1:0/2883846973 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7a809fd40 con 0x55d780858410 2024-09-11T21:58:18.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.620+0000 7fa7ad7fa640 1 -- 172.21.15.1:0/2883846973 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a0000e60 con 0x55d780858410 2024-09-11T21:58:18.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.621+0000 7fa7c068b740 1 -- 172.21.15.1:0/2883846973 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d780858410 msgr2=0x55d780861c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:18.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.621+0000 7fa7c068b740 1 --2- 172.21.15.1:0/2883846973 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d780858410 0x55d780861c00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0006010 tx=0x7fa7a003f000 comp rx=0 tx=0).stop 2024-09-11T21:58:18.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.621+0000 7fa7c068b740 1 -- 172.21.15.1:0/2883846973 shutdown_connections 2024-09-11T21:58:18.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.621+0000 7fa7c068b740 1 --2- 172.21.15.1:0/2883846973 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d780862cb0 0x55d78087a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:18.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.621+0000 7fa7c068b740 1 --2- 172.21.15.1:0/2883846973 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d780858410 0x55d780861c00 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:18.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.621+0000 7fa7c068b740 1 --2- 172.21.15.1:0/2883846973 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d780857af0 0x55d780860540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:18.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.621+0000 7fa7c068b740 1 -- 172.21.15.1:0/2883846973 >> 172.21.15.1:0/2883846973 conn(0x55d780854060 msgr2=0x55d7808614f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:18.624 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.621+0000 7fa7c068b740 1 -- 172.21.15.1:0/2883846973 shutdown_connections 2024-09-11T21:58:18.624 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.621+0000 7fa7c068b740 1 -- 172.21.15.1:0/2883846973 wait complete. 2024-09-11T21:58:18.624 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.621+0000 7fa7c068b740 1 -- start start 2024-09-11T21:58:18.624 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.622+0000 7fa7c068b740 1 Processor -- start 2024-09-11T21:58:18.626 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.622+0000 7fa7c068b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d780857af0 0x55d78086b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:18.626 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.622+0000 7fa7c068b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d780858410 0x55d780866720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:18.626 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.622+0000 7fa7c068b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d780862cb0 0x55d780866c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:18.626 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.622+0000 7fa7c068b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b81acbf0 con 0x55d780862cb0 2024-09-11T21:58:18.626 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.622+0000 7fa7c068b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b81ac520 con 0x55d780858410 2024-09-11T21:58:18.626 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.622+0000 7fa7c068b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b81a9760 con 0x55d780857af0 2024-09-11T21:58:18.627 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.622+0000 7fa7b77fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d780862cb0 0x55d780866c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:18.627 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.622+0000 7fa7b77fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d780862cb0 0x55d780866c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53236/0 (socket says 172.21.15.1:53236) 2024-09-11T21:58:18.627 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.622+0000 7fa7b6ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d780857af0 0x55d78086b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:18.627 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.622+0000 7fa7b77fe640 1 -- 172.21.15.1:0/2373293995 learned_addr learned my addr 172.21.15.1:0/2373293995 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:58:18.628 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.623+0000 7fa7affff640 1 --2- 172.21.15.1:0/2373293995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d780858410 0x55d780866720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:18.628 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.623+0000 7fa7c068b740 10 client.0 _ll_register_callbacks cb 0x7fa7a0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:58:18.628 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.623+0000 7fa7b77fe640 1 -- 172.21.15.1:0/2373293995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d780857af0 msgr2=0x55d78086b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:18.628 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.623+0000 7fa7b77fe640 1 --2- 172.21.15.1:0/2373293995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d780857af0 0x55d78086b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:18.628 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.623+0000 7fa7b77fe640 1 -- 172.21.15.1:0/2373293995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d780858410 msgr2=0x55d780866720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:18.628 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.623+0000 7fa7b77fe640 1 --2- 172.21.15.1:0/2373293995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d780858410 0x55d780866720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:18.628 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.623+0000 7fa7b77fe640 1 -- 172.21.15.1:0/2373293995 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7808674b0 con 0x55d780862cb0 2024-09-11T21:58:18.628 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.623+0000 7fa7affff640 1 --2- 172.21.15.1:0/2373293995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d780858410 0x55d780866720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T21:58:18.628 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.623+0000 7fa7b6ffd640 1 --2- 172.21.15.1:0/2373293995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d780857af0 0x55d78086b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T21:58:18.628 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.623+0000 7fa7b77fe640 1 --2- 172.21.15.1:0/2373293995 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d780862cb0 0x55d780866c60 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fa7b000fab0 tx=0x7fa7b0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a29384f8fbb28eee server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:18.628 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.623+0000 7fa796ffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T21:58:18.628 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.623+0000 7fa796ffd640 1 -- 172.21.15.1:0/2373293995 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b0004810 con 0x55d780862cb0 2024-09-11T21:58:18.629 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.623+0000 7fa796ffd640 1 -- 172.21.15.1:0/2373293995 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7fa7b001f430 con 0x55d780862cb0 2024-09-11T21:58:18.629 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.623+0000 7fa7c068b740 1 -- 172.21.15.1:0/2373293995 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7a809fd40 con 0x55d780862cb0 2024-09-11T21:58:18.629 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.624+0000 7fa796ffd640 1 -- 172.21.15.1:0/2373293995 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa7b001f630 con 0x55d780862cb0 2024-09-11T21:58:18.629 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.624+0000 7fa7c068b740 10 client.15774 fetch_fsmap learned FSMap version 28 2024-09-11T21:58:18.629 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.624+0000 7fa7c068b740 1 -- 172.21.15.1:0/2373293995 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d7808dd8f0 con 0x55d780862cb0 2024-09-11T21:58:18.629 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.624+0000 7fa796ffd640 1 -- 172.21.15.1:0/2373293995 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7a809fd40 con 0x55d780862cb0 2024-09-11T21:58:18.629 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.624+0000 7fa796ffd640 1 -- 172.21.15.1:0/2373293995 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa7b0030030 con 0x55d780862cb0 2024-09-11T21:58:18.629 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.624+0000 7fa7c068b740 10 client.15774 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T21:58:18.629 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.624+0000 7fa7c068b740 10 client.15774 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T21:58:18.629 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.624+0000 7fa7c068b740 20 client.15774 populate_metadata read hostname 'smithi001' 2024-09-11T21:58:18.629 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.624+0000 7fa7c068b740 1 --2- 172.21.15.1:0/2373293995 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55d7808de060 0x55d7808e0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:18.629 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.624+0000 7fa7c068b740 4 client.15774 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T21:58:18.629 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.624+0000 7fa7c068b740 1 -- 172.21.15.1:0/2373293995 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d7807c3d70 con 0x55d7808de060 2024-09-11T21:58:18.630 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.624+0000 7fa7b6ffd640 1 --2- 172.21.15.1:0/2373293995 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55d7808de060 0x55d7808e0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:18.630 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.625+0000 7fa796ffd640 10 client.15774 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T21:58:18.630 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.625+0000 7fa7b6ffd640 1 --2- 172.21.15.1:0/2373293995 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55d7808de060 0x55d7808e0490 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cb357a711b47933 server_cookie=e4c7c4e65b84594a in_seq=0 out_seq=0 2024-09-11T21:58:18.634 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.633+0000 7fa796ffd640 1 -- 172.21.15.1:0/2373293995 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d7807c3d70 con 0x55d7808de060 2024-09-11T21:58:18.634 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.633+0000 7fa796ffd640 10 client.15774 handle_command_reply: tid=0 2024-09-11T21:58:18.634 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.633+0000 7fa7c068b740 1 client.15774 shutdown 2024-09-11T21:58:18.634 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.633+0000 7fa7c068b740 1 -- 172.21.15.1:0/2373293995 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d780862cb0 msgr2=0x55d780866c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:18.634 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.633+0000 7fa7c068b740 1 --2- 172.21.15.1:0/2373293995 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d780862cb0 0x55d780866c60 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fa7b000fab0 tx=0x7fa7b0004ab0 comp rx=0 tx=0).stop 2024-09-11T21:58:18.634 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.633+0000 7fa7c068b740 1 -- 172.21.15.1:0/2373293995 shutdown_connections 2024-09-11T21:58:18.634 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.633+0000 7fa7c068b740 1 --2- 172.21.15.1:0/2373293995 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55d7808de060 0x55d7808e0490 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:18.634 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.633+0000 7fa7c068b740 1 --2- 172.21.15.1:0/2373293995 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d780862cb0 0x55d780866c60 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:18.635 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.633+0000 7fa7c068b740 1 --2- 172.21.15.1:0/2373293995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d780858410 0x55d780866720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:18.635 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.633+0000 7fa7c068b740 1 --2- 172.21.15.1:0/2373293995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d780857af0 0x55d78086b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:18.635 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.633+0000 7fa7c068b740 1 -- 172.21.15.1:0/2373293995 >> 172.21.15.1:0/2373293995 conn(0x55d780854060 msgr2=0x55d7808531c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:18.635 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.633+0000 7fa796ffd640 0 client.15774 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T21:58:18.635 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.633+0000 7fa7c068b740 1 -- 172.21.15.1:0/2373293995 shutdown_connections 2024-09-11T21:58:18.635 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.633+0000 7fa7c068b740 1 -- 172.21.15.1:0/2373293995 wait complete. 2024-09-11T21:58:18.635 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.633+0000 7fa7c068b740 20 client.15774 trim_cache size 0 max 16384 2024-09-11T21:58:18.635 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-09-11T21:58:18.635 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-09-11T21:58:18.636 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-09-11T21:58:18.636 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-09-11T21:58:18.636 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.634+0000 7fa7bd8f1640 1 -- 172.21.15.1:0/1508228674 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fa7a40a2e10 msgr2=0x7fa7a40a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:18.637 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.634+0000 7fa7bd8f1640 1 --2- 172.21.15.1:0/1508228674 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fa7a40a2e10 0x7fa7a40a52d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa7b0005fd0 tx=0x7fa7b0009500 comp rx=0 tx=0).stop 2024-09-11T21:58:18.637 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.634+0000 7fa7bd8f1640 1 -- 172.21.15.1:0/1508228674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7b81009a0 msgr2=0x7fa7b81993b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:18.637 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.634+0000 7fa7bd8f1640 1 --2- 172.21.15.1:0/1508228674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7b81009a0 0x7fa7b81993b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8009f40 tx=0x7fa7a800ee80 comp rx=0 tx=0).stop 2024-09-11T21:58:18.637 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.634+0000 7fa7bd8f1640 1 -- 172.21.15.1:0/1508228674 shutdown_connections 2024-09-11T21:58:18.637 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.634+0000 7fa7bd8f1640 1 --2- 172.21.15.1:0/1508228674 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fa7a40a2e10 0x7fa7a40a52d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:18.637 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.634+0000 7fa7bd8f1640 1 --2- 172.21.15.1:0/1508228674 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7b81069b0 0x7fa7b8199e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:18.637 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.634+0000 7fa7bd8f1640 1 --2- 172.21.15.1:0/1508228674 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa7b8101340 0x7fa7b81998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:18.637 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.634+0000 7fa7bd8f1640 1 --2- 172.21.15.1:0/1508228674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7b81009a0 0x7fa7b81993b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:18.637 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.634+0000 7fa7bd8f1640 1 -- 172.21.15.1:0/1508228674 >> 172.21.15.1:0/1508228674 conn(0x7fa7b80779a0 msgr2=0x7fa7b8103920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:18.637 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.634+0000 7fa7bd8f1640 1 -- 172.21.15.1:0/1508228674 shutdown_connections 2024-09-11T21:58:18.637 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:18.634+0000 7fa7bd8f1640 1 -- 172.21.15.1:0/1508228674 wait complete. 2024-09-11T21:58:19.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:58:19 smithi161 ceph-mon[35615]: pgmap v37: 97 pgs: 2 peering, 95 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 3.0 KiB/s wr, 15 op/s 2024-09-11T21:58:19.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:58:19 smithi001 ceph-mon[30935]: pgmap v37: 97 pgs: 2 peering, 95 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 3.0 KiB/s wr, 15 op/s 2024-09-11T21:58:19.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:58:19 smithi053 ceph-mon[34908]: pgmap v37: 97 pgs: 2 peering, 95 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 3.0 KiB/s wr, 15 op/s 2024-09-11T21:58:21.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:58:21 smithi161 ceph-mon[35615]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 1.6 KiB/s wr, 16 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-11T21:58:21.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:58:21 smithi001 ceph-mon[30935]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 1.6 KiB/s wr, 16 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-11T21:58:21.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:58:21 smithi053 ceph-mon[34908]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 1.6 KiB/s wr, 16 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-11T21:58:22.079 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:58:22 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:58:22.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:58:22 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:58:22.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:58:22 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:58:23.165 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T21:58:23.437 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:58:23 smithi001 ceph-mon[30935]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 1.9 KiB/s wr, 17 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-11T21:58:23.437 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:58:23 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:58:23.437 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:58:23 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:58:23.437 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:58:23 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:58:23.437 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:58:23 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:58:23.437 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:58:23 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:58:23.522 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:58:23.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:58:23 smithi053 ceph-mon[34908]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 1.9 KiB/s wr, 17 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-11T21:58:23.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:58:23 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:58:23.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:58:23 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:58:23.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:58:23 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:58:23.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:58:23 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:58:23.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:58:23 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:58:23.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:58:23 smithi161 ceph-mon[35615]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 1.9 KiB/s wr, 17 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-11T21:58:23.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:58:23 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:58:23.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:58:23 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:58:23.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:58:23 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:58:23.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:58:23 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:58:23.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:58:23 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:58:24.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:24.122+0000 7f62acb31640 1 -- 172.21.15.1:0/1111172784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62a8101dd0 msgr2=0x7f62a81021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:24.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:24.122+0000 7f62acb31640 1 --2- 172.21.15.1:0/1111172784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62a8101dd0 0x7f62a81021b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6298009900 tx=0x7f629802a3f0 comp rx=0 tx=0).stop 2024-09-11T21:58:24.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:24.122+0000 7f62acb31640 1 -- 172.21.15.1:0/1111172784 shutdown_connections 2024-09-11T21:58:24.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:24.122+0000 7f62acb31640 1 --2- 172.21.15.1:0/1111172784 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62a8106f00 0x7f62a810ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:24.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:24.122+0000 7f62acb31640 1 --2- 172.21.15.1:0/1111172784 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62a8102780 0x7f62a81067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:24.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:24.122+0000 7f62acb31640 1 --2- 172.21.15.1:0/1111172784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62a8101dd0 0x7f62a81021b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:24.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:24.122+0000 7f62acb31640 1 -- 172.21.15.1:0/1111172784 >> 172.21.15.1:0/1111172784 conn(0x7f62a80fd8b0 msgr2=0x7f62a80ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:24.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:24.122+0000 7f62acb31640 1 -- 172.21.15.1:0/1111172784 shutdown_connections 2024-09-11T21:58:24.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:24.122+0000 7f62acb31640 1 -- 172.21.15.1:0/1111172784 wait complete. 2024-09-11T21:58:24.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:24.123+0000 7f62acb31640 1 Processor -- start 2024-09-11T21:58:24.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:24.123+0000 7f62acb31640 1 -- start start 2024-09-11T21:58:24.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:24.124+0000 7f62acb31640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62a8101dd0 0x7f62a80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:24.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:24.124+0000 7f62acb31640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62a8102780 0x7f62a8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:24.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:24.124+0000 7f62acb31640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62a8106f00 0x7f62a8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:24.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:24.124+0000 7f629ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62a8102780 0x7f62a8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:24.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:24.124+0000 7f629ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62a8102780 0x7f62a8074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53246/0 (socket says 172.21.15.1:53246) 2024-09-11T21:58:24.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:24.124+0000 7f629ffff640 1 -- 172.21.15.1:0/3066092265 learned_addr learned my addr 172.21.15.1:0/3066092265 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:58:24.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:24.124+0000 7f62acb31640 1 -- 172.21.15.1:0/3066092265 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f62a8111fb0 con 0x7f62a8102780 2024-09-11T21:58:24.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:24.124+0000 7f62acb31640 1 -- 172.21.15.1:0/3066092265 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f62a8111e30 con 0x7f62a8101dd0 2024-09-11T21:58:24.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:24.124+0000 7f62acb31640 1 -- 172.21.15.1:0/3066092265 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f62a8112130 con 0x7f62a8106f00 2024-09-11T21:58:24.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:24.124+0000 7f62a6d9d640 1 --2- 172.21.15.1:0/3066092265 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62a8106f00 0x7f62a8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:24.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:24.124+0000 7f62a659c640 1 --2- 172.21.15.1:0/3066092265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62a8101dd0 0x7f62a80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:24.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:24.125+0000 7f62a6d9d640 1 -- 172.21.15.1:0/3066092265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62a8101dd0 msgr2=0x7f62a80744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:24.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:24.125+0000 7f62a6d9d640 1 --2- 172.21.15.1:0/3066092265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62a8101dd0 0x7f62a80744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:24.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:24.125+0000 7f62a6d9d640 1 -- 172.21.15.1:0/3066092265 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62a8102780 msgr2=0x7f62a8074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:24.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:24.125+0000 7f62a6d9d640 1 --2- 172.21.15.1:0/3066092265 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62a8102780 0x7f62a8074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:24.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:24.125+0000 7f62a6d9d640 1 -- 172.21.15.1:0/3066092265 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62a81a6830 con 0x7f62a8106f00 2024-09-11T21:58:24.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:24.125+0000 7f62a659c640 1 --2- 172.21.15.1:0/3066092265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62a8101dd0 0x7f62a80744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:58:24.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:24.125+0000 7f62a6d9d640 1 --2- 172.21.15.1:0/3066092265 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62a8106f00 0x7f62a8074f70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f62a0009fd0 tx=0x7f62a000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:24.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:24.125+0000 7f629f7fe640 1 -- 172.21.15.1:0/3066092265 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a000ce20 con 0x7f62a8106f00 2024-09-11T21:58:24.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:24.126+0000 7f629f7fe640 1 -- 172.21.15.1:0/3066092265 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62a0005020 con 0x7f62a8106f00 2024-09-11T21:58:24.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:24.126+0000 7f629f7fe640 1 -- 172.21.15.1:0/3066092265 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a0010720 con 0x7f62a8106f00 2024-09-11T21:58:24.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:24.126+0000 7f62acb31640 1 -- 172.21.15.1:0/3066092265 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62a81a6b20 con 0x7f62a8106f00 2024-09-11T21:58:24.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:24.126+0000 7f62acb31640 1 -- 172.21.15.1:0/3066092265 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62a81a7010 con 0x7f62a8106f00 2024-09-11T21:58:24.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:24.127+0000 7f62acb31640 1 -- 172.21.15.1:0/3066092265 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62a81021b0 con 0x7f62a8106f00 2024-09-11T21:58:24.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:24.129+0000 7f629f7fe640 1 -- 172.21.15.1:0/3066092265 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 25) ==== 138065+0+0 (secure 0 0 0) 0x7f62a0005540 con 0x7f62a8106f00 2024-09-11T21:58:24.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:24.130+0000 7f629f7fe640 1 --2- 172.21.15.1:0/3066092265 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f62940a2e90 0x7f62940a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:24.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:24.130+0000 7f62a659c640 1 --2- 172.21.15.1:0/3066092265 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f62940a2e90 0x7f62940a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:24.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:24.131+0000 7f629f7fe640 1 -- 172.21.15.1:0/3066092265 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f62a00cfe10 con 0x7f62a8106f00 2024-09-11T21:58:24.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:24.131+0000 7f62a659c640 1 --2- 172.21.15.1:0/3066092265 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f62940a2e90 0x7f62940a5350 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6298000c00 tx=0x7f6298005a40 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:24.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:24.132+0000 7f629f7fe640 1 -- 172.21.15.1:0/3066092265 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f62a009c740 con 0x7f62a8106f00 2024-09-11T21:58:24.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:24.451+0000 7f62acb31640 1 -- 172.21.15.1:0/3066092265 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f62a8075a20 con 0x7f62a8106f00 2024-09-11T21:58:24.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:24.452+0000 7f629f7fe640 1 -- 172.21.15.1:0/3066092265 <== mon.1 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11949 (secure 0 0 0) 0x7f62a8075a20 con 0x7f62a8106f00 2024-09-11T21:58:24.453 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:58:24.453 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":28,"btime":"2024-09-11T21:57:12:512925+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/421542832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":421542832},{"type":"v1","addr":"172.21.15.1:6835","nonce":421542832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2836864406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2836864406},{"type":"v1","addr":"172.21.15.1:6837","nonce":2836864406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T21:55:20.791263+0000","modified":"2024-09-11T21:57:12.462736+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24484,"mds_2":24467,"mds_3":24466,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14595":{"gid":14595,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/407834672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":407834672},{"type":"v1","addr":"172.21.15.1:6839","nonce":407834672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/1068827678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":1068827678},{"type":"v1","addr":"172.21.15.1:6841","nonce":1068827678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/872410024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":872410024},{"type":"v1","addr":"172.21.15.161:6833","nonce":872410024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/1648944570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1648944570},{"type":"v1","addr":"172.21.15.53:6833","nonce":1648944570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/2126933993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2126933993},{"type":"v1","addr":"172.21.15.53:6835","nonce":2126933993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.161:6835/828381592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":828381592},{"type":"v1","addr":"172.21.15.161:6835","nonce":828381592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.161:6837/1799318656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1799318656},{"type":"v1","addr":"172.21.15.161:6837","nonce":1799318656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.53:6837/3457131830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3457131830},{"type":"v1","addr":"172.21.15.53:6837","nonce":3457131830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/455365383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":455365383},{"type":"v1","addr":"172.21.15.161:6839","nonce":455365383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.53:6839/1378584447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1378584447},{"type":"v1","addr":"172.21.15.53:6839","nonce":1378584447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24460,24466,24467,24484,24494]},"id":1}]} 2024-09-11T21:58:24.454 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 28 2024-09-11T21:58:24.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:24.455+0000 7f62acb31640 1 -- 172.21.15.1:0/3066092265 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f62940a2e90 msgr2=0x7f62940a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:24.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:24.455+0000 7f62acb31640 1 --2- 172.21.15.1:0/3066092265 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f62940a2e90 0x7f62940a5350 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6298000c00 tx=0x7f6298005a40 comp rx=0 tx=0).stop 2024-09-11T21:58:24.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:24.455+0000 7f62acb31640 1 -- 172.21.15.1:0/3066092265 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62a8106f00 msgr2=0x7f62a8074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:24.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:24.455+0000 7f62acb31640 1 --2- 172.21.15.1:0/3066092265 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62a8106f00 0x7f62a8074f70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f62a0009fd0 tx=0x7f62a000ef10 comp rx=0 tx=0).stop 2024-09-11T21:58:24.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:24.456+0000 7f62acb31640 1 -- 172.21.15.1:0/3066092265 shutdown_connections 2024-09-11T21:58:24.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:24.456+0000 7f62acb31640 1 --2- 172.21.15.1:0/3066092265 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f62940a2e90 0x7f62940a5350 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:24.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:24.456+0000 7f62acb31640 1 --2- 172.21.15.1:0/3066092265 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f62a8106f00 0x7f62a8074f70 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:24.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:24.456+0000 7f62acb31640 1 --2- 172.21.15.1:0/3066092265 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62a8102780 0x7f62a8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:24.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:24.456+0000 7f62acb31640 1 --2- 172.21.15.1:0/3066092265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62a8101dd0 0x7f62a80744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:24.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:24.456+0000 7f62acb31640 1 -- 172.21.15.1:0/3066092265 >> 172.21.15.1:0/3066092265 conn(0x7f62a80fd8b0 msgr2=0x7f62a80febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:24.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:24.456+0000 7f62acb31640 1 -- 172.21.15.1:0/3066092265 shutdown_connections 2024-09-11T21:58:24.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:24.456+0000 7f62acb31640 1 -- 172.21.15.1:0/3066092265 wait complete. 2024-09-11T21:58:25.290 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db --set-id f7fcf15b --release --await' 2024-09-11T21:58:25.290 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db --set-id f7fcf15b --release --await 2024-09-11T21:58:25.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:58:25 smithi001 ceph-mon[30935]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.6 KiB/s wr, 14 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-11T21:58:25.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:58:25 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/3066092265' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:58:25.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:58:25 smithi053 ceph-mon[34908]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.6 KiB/s wr, 14 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-11T21:58:25.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:58:25 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/3066092265' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:58:25.580 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:58:25.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:58:25 smithi161 ceph-mon[35615]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.6 KiB/s wr, 14 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-11T21:58:25.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:58:25 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/3066092265' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:58:26.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.115+0000 7fe6a8304640 1 -- 172.21.15.1:0/4130002974 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6a0102570 msgr2=0x7fe6a01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:26.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.115+0000 7fe6a8304640 1 --2- 172.21.15.1:0/4130002974 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6a0102570 0x7fe6a01029b0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fe690009900 tx=0x7fe69002a480 comp rx=0 tx=0).stop 2024-09-11T21:58:26.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.115+0000 7fe6a8304640 1 -- 172.21.15.1:0/4130002974 shutdown_connections 2024-09-11T21:58:26.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.115+0000 7fe6a8304640 1 --2- 172.21.15.1:0/4130002974 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6a0102ef0 0x7fe6a010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:26.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.115+0000 7fe6a8304640 1 --2- 172.21.15.1:0/4130002974 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6a0102570 0x7fe6a01029b0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:26.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.115+0000 7fe6a8304640 1 --2- 172.21.15.1:0/4130002974 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6a0100e20 0x7fe6a0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:26.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.115+0000 7fe6a8304640 1 -- 172.21.15.1:0/4130002974 >> 172.21.15.1:0/4130002974 conn(0x7fe6a00fc7d0 msgr2=0x7fe6a00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:26.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.115+0000 7fe6a8304640 1 -- 172.21.15.1:0/4130002974 shutdown_connections 2024-09-11T21:58:26.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.115+0000 7fe6a8304640 1 -- 172.21.15.1:0/4130002974 wait complete. 2024-09-11T21:58:26.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.116+0000 7fe6a8304640 1 Processor -- start 2024-09-11T21:58:26.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.116+0000 7fe6a8304640 1 -- start start 2024-09-11T21:58:26.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.117+0000 7fe6a8304640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6a0100e20 0x7fe6a0198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:26.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.117+0000 7fe6a8304640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6a0102570 0x7fe6a0198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:26.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.117+0000 7fe6a60a0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6a0100e20 0x7fe6a0198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:26.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.117+0000 7fe6a60a0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6a0100e20 0x7fe6a0198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53270/0 (socket says 172.21.15.1:53270) 2024-09-11T21:58:26.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.117+0000 7fe6a589f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6a0102570 0x7fe6a0198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:26.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.117+0000 7fe6a8304640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6a0102ef0 0x7fe6a019cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:26.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.117+0000 7fe6a8304640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a0110f80 con 0x7fe6a0100e20 2024-09-11T21:58:26.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.117+0000 7fe6a60a0640 1 -- 172.21.15.1:0/1125403619 learned_addr learned my addr 172.21.15.1:0/1125403619 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:58:26.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.117+0000 7fe6a8304640 1 -- 172.21.15.1:0/1125403619 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a0110e00 con 0x7fe6a0102ef0 2024-09-11T21:58:26.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.117+0000 7fe6a8304640 1 -- 172.21.15.1:0/1125403619 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a0111100 con 0x7fe6a0102570 2024-09-11T21:58:26.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.117+0000 7fe6a68a1640 1 --2- 172.21.15.1:0/1125403619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6a0102ef0 0x7fe6a019cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:26.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.118+0000 7fe6a60a0640 1 -- 172.21.15.1:0/1125403619 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6a0102570 msgr2=0x7fe6a0198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:26.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.118+0000 7fe6a60a0640 1 --2- 172.21.15.1:0/1125403619 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6a0102570 0x7fe6a0198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:26.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.118+0000 7fe6a60a0640 1 -- 172.21.15.1:0/1125403619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6a0102ef0 msgr2=0x7fe6a019cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:26.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.118+0000 7fe6a60a0640 1 --2- 172.21.15.1:0/1125403619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6a0102ef0 0x7fe6a019cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:26.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.118+0000 7fe6a60a0640 1 -- 172.21.15.1:0/1125403619 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6a019d3c0 con 0x7fe6a0100e20 2024-09-11T21:58:26.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.118+0000 7fe6a60a0640 1 --2- 172.21.15.1:0/1125403619 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6a0100e20 0x7fe6a0198410 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fe69c00ae20 tx=0x7fe69c007590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:26.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.118+0000 7fe6977fe640 1 -- 172.21.15.1:0/1125403619 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe69c004220 con 0x7fe6a0100e20 2024-09-11T21:58:26.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.118+0000 7fe6977fe640 1 -- 172.21.15.1:0/1125403619 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe69c0043c0 con 0x7fe6a0100e20 2024-09-11T21:58:26.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.118+0000 7fe6977fe640 1 -- 172.21.15.1:0/1125403619 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe69c0035f0 con 0x7fe6a0100e20 2024-09-11T21:58:26.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.118+0000 7fe6a8304640 1 -- 172.21.15.1:0/1125403619 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6a019d6b0 con 0x7fe6a0100e20 2024-09-11T21:58:26.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.118+0000 7fe6a8304640 1 -- 172.21.15.1:0/1125403619 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6a0103ad0 con 0x7fe6a0100e20 2024-09-11T21:58:26.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.122+0000 7fe6977fe640 1 -- 172.21.15.1:0/1125403619 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) ==== 138065+0+0 (secure 0 0 0) 0x7fe69c010050 con 0x7fe6a0100e20 2024-09-11T21:58:26.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.122+0000 7fe6aa6dd740 1 Processor -- start 2024-09-11T21:58:26.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.122+0000 7fe6aa6dd740 1 -- start start 2024-09-11T21:58:26.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.123+0000 7fe6aa6dd740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b834a3f060 0x55b834a3f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:26.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.123+0000 7fe6aa6dd740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b834a3f980 0x55b834a49d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:26.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.123+0000 7fe6aa6dd740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b834a4a240 0x55b834a4c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:26.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.123+0000 7fe6aa6dd740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55b8349a4620 con 0x55b834a4a240 2024-09-11T21:58:26.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.123+0000 7fe6aa6dd740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55b83494be50 con 0x55b834a3f060 2024-09-11T21:58:26.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.123+0000 7fe6aa6dd740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b8349a05e0 con 0x55b834a3f980 2024-09-11T21:58:26.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.123+0000 7fe6a68a1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b834a3f060 0x55b834a3f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:26.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.123+0000 7fe6a68a1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b834a3f060 0x55b834a3f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:59716/0 (socket says 172.21.15.1:59716) 2024-09-11T21:58:26.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.123+0000 7fe6a68a1640 1 -- 172.21.15.1:0/2605695059 learned_addr learned my addr 172.21.15.1:0/2605695059 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:58:26.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.123+0000 7fe6977fe640 1 --2- 172.21.15.1:0/1125403619 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fe68c0a2e90 0x7fe68c0a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:26.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.124+0000 7fe6977fe640 1 -- 172.21.15.1:0/1125403619 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7fe69c0d3250 con 0x7fe6a0100e20 2024-09-11T21:58:26.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.124+0000 7fe6a68a1640 1 -- 172.21.15.1:0/2605695059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b834a3f980 msgr2=0x55b834a49d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:26.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.124+0000 7fe6a68a1640 1 --2- 172.21.15.1:0/2605695059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b834a3f980 0x55b834a49d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:26.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.124+0000 7fe6a68a1640 1 -- 172.21.15.1:0/2605695059 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b834a4a240 msgr2=0x55b834a4c630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T21:58:26.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.124+0000 7fe6a68a1640 1 --2- 172.21.15.1:0/2605695059 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b834a4a240 0x55b834a4c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:26.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.124+0000 7fe6a68a1640 1 -- 172.21.15.1:0/2605695059 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b83493cd40 con 0x55b834a3f060 2024-09-11T21:58:26.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.124+0000 7fe6a68a1640 1 --2- 172.21.15.1:0/1125403619 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fe68c0a2e90 0x7fe68c0a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:26.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.125+0000 7fe6a68a1640 1 --2- 172.21.15.1:0/1125403619 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fe68c0a2e90 0x7fe68c0a5350 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe698010e60 tx=0x7fe698010f90 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:26.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.125+0000 7fe6a68a1640 1 --2- 172.21.15.1:0/2605695059 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b834a3f060 0x55b834a3f440 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe698010510 tx=0x7fe698009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2bb0d6619ed28ccc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:26.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.125+0000 7fe694dea640 1 -- 172.21.15.1:0/2605695059 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe69800e850 con 0x55b834a3f060 2024-09-11T21:58:26.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.125+0000 7fe694dea640 1 -- 172.21.15.1:0/2605695059 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe69801d430 con 0x55b834a3f060 2024-09-11T21:58:26.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.125+0000 7fe694dea640 1 -- 172.21.15.1:0/2605695059 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe698011600 con 0x55b834a3f060 2024-09-11T21:58:26.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.126+0000 7fe6aa6dd740 1 -- 172.21.15.1:0/2605695059 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b834a3f060 msgr2=0x55b834a3f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:26.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.126+0000 7fe6aa6dd740 1 --2- 172.21.15.1:0/2605695059 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b834a3f060 0x55b834a3f440 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe698010510 tx=0x7fe698009f90 comp rx=0 tx=0).stop 2024-09-11T21:58:26.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.126+0000 7fe6aa6dd740 1 -- 172.21.15.1:0/2605695059 shutdown_connections 2024-09-11T21:58:26.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.126+0000 7fe6aa6dd740 1 --2- 172.21.15.1:0/2605695059 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b834a4a240 0x55b834a4c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:26.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.126+0000 7fe6aa6dd740 1 --2- 172.21.15.1:0/2605695059 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b834a3f980 0x55b834a49d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:26.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.126+0000 7fe6aa6dd740 1 --2- 172.21.15.1:0/2605695059 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b834a3f060 0x55b834a3f440 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:26.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.126+0000 7fe6aa6dd740 1 -- 172.21.15.1:0/2605695059 >> 172.21.15.1:0/2605695059 conn(0x55b834a3b5f0 msgr2=0x55b834a3b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:26.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.126+0000 7fe6aa6dd740 1 -- 172.21.15.1:0/2605695059 shutdown_connections 2024-09-11T21:58:26.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.127+0000 7fe6aa6dd740 1 -- 172.21.15.1:0/2605695059 wait complete. 2024-09-11T21:58:26.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.127+0000 7fe6aa6dd740 1 -- start start 2024-09-11T21:58:26.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.127+0000 7fe6aa6dd740 1 Processor -- start 2024-09-11T21:58:26.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.127+0000 7fe6aa6dd740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b834a3f060 0x55b834a47160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:26.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.127+0000 7fe6aa6dd740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b834a3f980 0x55b834a476a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:26.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.127+0000 7fe6aa6dd740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b834a4a240 0x55b834a47be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:26.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.127+0000 7fe6aa6dd740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55b834a4ce10 con 0x55b834a4a240 2024-09-11T21:58:26.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.127+0000 7fe6aa6dd740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55b83492cf10 con 0x55b834a3f060 2024-09-11T21:58:26.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.127+0000 7fe6aa6dd740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b834a4cf90 con 0x55b834a3f980 2024-09-11T21:58:26.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.127+0000 7fe6aa6dd740 10 client.0 _ll_register_callbacks cb 0x55b834950db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:58:26.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.127+0000 7fe6a68a1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b834a4a240 0x55b834a47be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:26.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.128+0000 7fe6a68a1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b834a4a240 0x55b834a47be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53294/0 (socket says 172.21.15.1:53294) 2024-09-11T21:58:26.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.128+0000 7fe6a68a1640 1 -- 172.21.15.1:0/614271756 learned_addr learned my addr 172.21.15.1:0/614271756 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:58:26.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.128+0000 7fe6a60a0640 1 --2- 172.21.15.1:0/614271756 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b834a3f060 0x55b834a47160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:26.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.128+0000 7fe6a589f640 1 --2- 172.21.15.1:0/614271756 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b834a3f980 0x55b834a476a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:26.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.128+0000 7fe6a60a0640 1 -- 172.21.15.1:0/614271756 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b834a3f980 msgr2=0x55b834a476a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:26.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.128+0000 7fe6a60a0640 1 --2- 172.21.15.1:0/614271756 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b834a3f980 0x55b834a476a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:26.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.128+0000 7fe6a60a0640 1 -- 172.21.15.1:0/614271756 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b834a4a240 msgr2=0x55b834a47be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:26.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.128+0000 7fe6a60a0640 1 --2- 172.21.15.1:0/614271756 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b834a4a240 0x55b834a47be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:26.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.128+0000 7fe6a60a0640 1 -- 172.21.15.1:0/614271756 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8346b3580 con 0x55b834a3f060 2024-09-11T21:58:26.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.128+0000 7fe6a60a0640 1 --2- 172.21.15.1:0/614271756 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b834a3f060 0x55b834a47160 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe69c0d07b0 tx=0x7fe69c09b780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ffa108ad4c09db1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:26.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.128+0000 7fe689ffb640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-11T21:58:26.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.128+0000 7fe689ffb640 1 -- 172.21.15.1:0/614271756 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe69c09ee40 con 0x55b834a3f060 2024-09-11T21:58:26.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.128+0000 7fe689ffb640 1 -- 172.21.15.1:0/614271756 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7fe69c0d0cc0 con 0x55b834a3f060 2024-09-11T21:58:26.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.128+0000 7fe6aa6dd740 1 -- 172.21.15.1:0/614271756 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe69801d430 con 0x55b834a3f060 2024-09-11T21:58:26.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.129+0000 7fe689ffb640 1 -- 172.21.15.1:0/614271756 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe69c09fbe0 con 0x55b834a3f060 2024-09-11T21:58:26.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.129+0000 7fe6aa6dd740 10 client.25376 fetch_fsmap learned FSMap version 28 2024-09-11T21:58:26.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.129+0000 7fe6aa6dd740 1 -- 172.21.15.1:0/614271756 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b83477d960 con 0x55b834a3f060 2024-09-11T21:58:26.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.129+0000 7fe689ffb640 1 -- 172.21.15.1:0/614271756 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe69801d430 con 0x55b834a3f060 2024-09-11T21:58:26.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.129+0000 7fe689ffb640 1 -- 172.21.15.1:0/614271756 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe69c09eac0 con 0x55b834a3f060 2024-09-11T21:58:26.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.129+0000 7fe6aa6dd740 10 client.25376 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T21:58:26.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.129+0000 7fe6aa6dd740 10 client.25376 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-11T21:58:26.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.129+0000 7fe6aa6dd740 20 client.25376 populate_metadata read hostname 'smithi001' 2024-09-11T21:58:26.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.129+0000 7fe6aa6dd740 1 --2- 172.21.15.1:0/614271756 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55b834a59c70 0x55b834a5c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:26.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.129+0000 7fe6aa6dd740 4 client.25376 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T21:58:26.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.129+0000 7fe6aa6dd740 1 -- 172.21.15.1:0/614271756 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b8349b1b90 con 0x55b834a59c70 2024-09-11T21:58:26.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.130+0000 7fe6a589f640 1 --2- 172.21.15.1:0/614271756 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55b834a59c70 0x55b834a5c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:26.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.130+0000 7fe6a589f640 1 --2- 172.21.15.1:0/614271756 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55b834a59c70 0x55b834a5c060 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=320af0d390668e44 server_cookie=66dac3da2d76dcc3 in_seq=0 out_seq=0 2024-09-11T21:58:26.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.130+0000 7fe689ffb640 10 client.25376 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T21:58:26.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.131+0000 7fe689ffb640 1 -- 172.21.15.1:0/614271756 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b8349b1b90 con 0x55b834a59c70 2024-09-11T21:58:26.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.131+0000 7fe689ffb640 10 client.25376 handle_command_reply: tid=0 2024-09-11T21:58:26.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.131+0000 7fe6aa6dd740 1 client.25376 shutdown 2024-09-11T21:58:26.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.131+0000 7fe6aa6dd740 1 -- 172.21.15.1:0/614271756 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b834a3f060 msgr2=0x55b834a47160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:26.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.131+0000 7fe6aa6dd740 1 --2- 172.21.15.1:0/614271756 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b834a3f060 0x55b834a47160 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe69c0d07b0 tx=0x7fe69c09b780 comp rx=0 tx=0).stop 2024-09-11T21:58:26.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.131+0000 7fe6aa6dd740 1 -- 172.21.15.1:0/614271756 shutdown_connections 2024-09-11T21:58:26.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.131+0000 7fe6aa6dd740 1 --2- 172.21.15.1:0/614271756 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55b834a59c70 0x55b834a5c060 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:26.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.131+0000 7fe6aa6dd740 1 --2- 172.21.15.1:0/614271756 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b834a4a240 0x55b834a47be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:26.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.131+0000 7fe6aa6dd740 1 --2- 172.21.15.1:0/614271756 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b834a3f980 0x55b834a476a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:26.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.131+0000 7fe689ffb640 0 client.25376 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T21:58:26.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.131+0000 7fe6aa6dd740 1 --2- 172.21.15.1:0/614271756 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b834a3f060 0x55b834a47160 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:26.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.131+0000 7fe6aa6dd740 1 -- 172.21.15.1:0/614271756 >> 172.21.15.1:0/614271756 conn(0x55b834a3b5f0 msgr2=0x55b834a41320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:26.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.131+0000 7fe6aa6dd740 1 -- 172.21.15.1:0/614271756 shutdown_connections 2024-09-11T21:58:26.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.131+0000 7fe6aa6dd740 1 -- 172.21.15.1:0/614271756 wait complete. 2024-09-11T21:58:26.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.131+0000 7fe6aa6dd740 20 client.25376 trim_cache size 0 max 16384 2024-09-11T21:58:26.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.164+0000 7fe6aa6dd740 1 Processor -- start 2024-09-11T21:58:26.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.164+0000 7fe6aa6dd740 1 -- start start 2024-09-11T21:58:26.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.164+0000 7fe6aa6dd740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b834a3f060 0x55b834a41b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:26.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.164+0000 7fe6aa6dd740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b834a3f980 0x55b834a49030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:26.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.165+0000 7fe6aa6dd740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b834a4a240 0x55b834a61870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:26.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.165+0000 7fe6aa6dd740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a0072430 con 0x55b834a4a240 2024-09-11T21:58:26.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.165+0000 7fe6aa6dd740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a0073f70 con 0x55b834a3f060 2024-09-11T21:58:26.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.165+0000 7fe6aa6dd740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a0072b00 con 0x55b834a3f980 2024-09-11T21:58:26.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.165+0000 7fe6a68a1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b834a4a240 0x55b834a61870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:26.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.165+0000 7fe6a68a1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b834a4a240 0x55b834a61870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53304/0 (socket says 172.21.15.1:53304) 2024-09-11T21:58:26.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.165+0000 7fe6a68a1640 1 -- 172.21.15.1:0/1136479109 learned_addr learned my addr 172.21.15.1:0/1136479109 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:58:26.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.165+0000 7fe6a589f640 1 --2- 172.21.15.1:0/1136479109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b834a3f980 0x55b834a49030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:26.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.165+0000 7fe6a60a0640 1 --2- 172.21.15.1:0/1136479109 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b834a3f060 0x55b834a41b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:26.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.165+0000 7fe6a68a1640 1 -- 172.21.15.1:0/1136479109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b834a3f980 msgr2=0x55b834a49030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:26.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.165+0000 7fe6a68a1640 1 --2- 172.21.15.1:0/1136479109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b834a3f980 0x55b834a49030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:26.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.165+0000 7fe6a68a1640 1 -- 172.21.15.1:0/1136479109 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b834a3f060 msgr2=0x55b834a41b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:26.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.165+0000 7fe6a68a1640 1 --2- 172.21.15.1:0/1136479109 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b834a3f060 0x55b834a41b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:26.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.165+0000 7fe6a68a1640 1 -- 172.21.15.1:0/1136479109 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b834a49ad0 con 0x55b834a4a240 2024-09-11T21:58:26.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.166+0000 7fe6a68a1640 1 --2- 172.21.15.1:0/1136479109 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b834a4a240 0x55b834a61870 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fe69801dea0 tx=0x7fe698009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ad18329f6aec54d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:26.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.166+0000 7fe68bfff640 1 -- 172.21.15.1:0/1136479109 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe698043020 con 0x55b834a4a240 2024-09-11T21:58:26.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.166+0000 7fe68bfff640 1 -- 172.21.15.1:0/1136479109 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe69803b040 con 0x55b834a4a240 2024-09-11T21:58:26.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.166+0000 7fe68bfff640 1 -- 172.21.15.1:0/1136479109 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe69803a050 con 0x55b834a4a240 2024-09-11T21:58:26.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.167+0000 7fe6aa6dd740 1 -- 172.21.15.1:0/1136479109 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b834a4a240 msgr2=0x55b834a61870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:26.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.167+0000 7fe6aa6dd740 1 --2- 172.21.15.1:0/1136479109 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b834a4a240 0x55b834a61870 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fe69801dea0 tx=0x7fe698009850 comp rx=0 tx=0).stop 2024-09-11T21:58:26.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.167+0000 7fe6aa6dd740 1 -- 172.21.15.1:0/1136479109 shutdown_connections 2024-09-11T21:58:26.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.167+0000 7fe6aa6dd740 1 --2- 172.21.15.1:0/1136479109 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b834a4a240 0x55b834a61870 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:26.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.167+0000 7fe6aa6dd740 1 --2- 172.21.15.1:0/1136479109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b834a3f980 0x55b834a49030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:26.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.167+0000 7fe6aa6dd740 1 --2- 172.21.15.1:0/1136479109 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b834a3f060 0x55b834a41b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:26.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.167+0000 7fe6aa6dd740 1 -- 172.21.15.1:0/1136479109 >> 172.21.15.1:0/1136479109 conn(0x55b834a3b5f0 msgr2=0x55b834a3a750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:26.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.167+0000 7fe6aa6dd740 1 -- 172.21.15.1:0/1136479109 shutdown_connections 2024-09-11T21:58:26.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.167+0000 7fe6aa6dd740 1 -- 172.21.15.1:0/1136479109 wait complete. 2024-09-11T21:58:26.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.168+0000 7fe6aa6dd740 1 -- start start 2024-09-11T21:58:26.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.168+0000 7fe6aa6dd740 1 Processor -- start 2024-09-11T21:58:26.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.168+0000 7fe6aa6dd740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b834a3f060 0x55b834a52640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:26.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.168+0000 7fe6aa6dd740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b834a3f980 0x55b834a4f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:26.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.168+0000 7fe6aa6dd740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b834a4a240 0x55b834a4f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:26.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.168+0000 7fe6aa6dd740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a00738a0 con 0x55b834a3f980 2024-09-11T21:58:26.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.168+0000 7fe6aa6dd740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a00731d0 con 0x55b834a4a240 2024-09-11T21:58:26.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.168+0000 7fe6aa6dd740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a0070410 con 0x55b834a3f060 2024-09-11T21:58:26.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.168+0000 7fe6aa6dd740 10 client.0 _ll_register_callbacks cb 0x7fe69c09b0c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:58:26.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.169+0000 7fe6a60a0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b834a3f060 0x55b834a52640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:26.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.169+0000 7fe6a60a0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b834a3f060 0x55b834a52640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:45378/0 (socket says 172.21.15.1:45378) 2024-09-11T21:58:26.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.169+0000 7fe6a60a0640 1 -- 172.21.15.1:0/3892654798 learned_addr learned my addr 172.21.15.1:0/3892654798 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:58:26.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.169+0000 7fe6a68a1640 1 --2- 172.21.15.1:0/3892654798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b834a4a240 0x55b834a4f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:26.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.169+0000 7fe6a589f640 1 --2- 172.21.15.1:0/3892654798 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b834a3f980 0x55b834a4f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:26.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.169+0000 7fe6a60a0640 1 -- 172.21.15.1:0/3892654798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b834a4a240 msgr2=0x55b834a4f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:26.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.169+0000 7fe6a60a0640 1 --2- 172.21.15.1:0/3892654798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b834a4a240 0x55b834a4f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:26.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.169+0000 7fe6a60a0640 1 -- 172.21.15.1:0/3892654798 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b834a3f980 msgr2=0x55b834a4f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:26.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.169+0000 7fe6a60a0640 1 --2- 172.21.15.1:0/3892654798 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b834a3f980 0x55b834a4f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:26.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.169+0000 7fe6a60a0640 1 -- 172.21.15.1:0/3892654798 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b834a4fc30 con 0x55b834a3f060 2024-09-11T21:58:26.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.169+0000 7fe6a68a1640 1 --2- 172.21.15.1:0/3892654798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b834a4a240 0x55b834a4f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T21:58:26.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.169+0000 7fe6a589f640 1 --2- 172.21.15.1:0/3892654798 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b834a3f980 0x55b834a4f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:58:26.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.170+0000 7fe689ffb640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-11T21:58:26.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.169+0000 7fe6a60a0640 1 --2- 172.21.15.1:0/3892654798 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b834a3f060 0x55b834a52640 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe69c09b940 tx=0x7fe69c09e690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b3d6ab4ed5d2d91 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:26.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.170+0000 7fe689ffb640 1 -- 172.21.15.1:0/3892654798 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe69c09ee40 con 0x55b834a3f060 2024-09-11T21:58:26.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.170+0000 7fe689ffb640 1 -- 172.21.15.1:0/3892654798 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7fe69c09b440 con 0x55b834a3f060 2024-09-11T21:58:26.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.170+0000 7fe6aa6dd740 1 -- 172.21.15.1:0/3892654798 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe69803b040 con 0x55b834a3f060 2024-09-11T21:58:26.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.170+0000 7fe689ffb640 1 -- 172.21.15.1:0/3892654798 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe69c086050 con 0x55b834a3f060 2024-09-11T21:58:26.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.170+0000 7fe6aa6dd740 10 client.25357 fetch_fsmap learned FSMap version 28 2024-09-11T21:58:26.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.170+0000 7fe6aa6dd740 1 -- 172.21.15.1:0/3892654798 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b834ac55a0 con 0x55b834a3f060 2024-09-11T21:58:26.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.170+0000 7fe689ffb640 1 -- 172.21.15.1:0/3892654798 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe69803b040 con 0x55b834a3f060 2024-09-11T21:58:26.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.171+0000 7fe689ffb640 1 -- 172.21.15.1:0/3892654798 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe69c0b80d0 con 0x55b834a3f060 2024-09-11T21:58:26.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.171+0000 7fe6aa6dd740 10 client.25357 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T21:58:26.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.171+0000 7fe6aa6dd740 10 client.25357 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-11T21:58:26.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.171+0000 7fe6aa6dd740 20 client.25357 populate_metadata read hostname 'smithi001' 2024-09-11T21:58:26.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.171+0000 7fe6aa6dd740 1 --2- 172.21.15.1:0/3892654798 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55b834a59c70 0x55b834ac7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:26.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.171+0000 7fe6aa6dd740 4 client.25357 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "f7fcf15b", "release": true, "await": true}] 2024-09-11T21:58:26.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.171+0000 7fe6aa6dd740 1 -- 172.21.15.1:0/3892654798 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "quiesce db", "set_id": "f7fcf15b", "release": true, "await": true}) -- 0x55b8349ac0a0 con 0x55b834a59c70 2024-09-11T21:58:26.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.172+0000 7fe6a589f640 1 --2- 172.21.15.1:0/3892654798 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55b834a59c70 0x55b834ac7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:26.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.173+0000 7fe6a589f640 1 --2- 172.21.15.1:0/3892654798 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55b834a59c70 0x55b834ac7d00 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c5241d8877a99d9 server_cookie=231ac0b9d65b496d in_seq=0 out_seq=0 2024-09-11T21:58:26.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.173+0000 7fe689ffb640 10 client.25357 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T21:58:26.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.175+0000 7fe689ffb640 1 -- 172.21.15.1:0/3892654798 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55b8349ac0a0 con 0x55b834a59c70 2024-09-11T21:58:26.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.175+0000 7fe689ffb640 10 client.25357 handle_command_reply: tid=0 2024-09-11T21:58:26.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.175+0000 7fe6aa6dd740 1 client.25357 shutdown 2024-09-11T21:58:26.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.176+0000 7fe6aa6dd740 1 -- 172.21.15.1:0/3892654798 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b834a3f060 msgr2=0x55b834a52640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:26.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.176+0000 7fe6aa6dd740 1 --2- 172.21.15.1:0/3892654798 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b834a3f060 0x55b834a52640 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe69c09b940 tx=0x7fe69c09e690 comp rx=0 tx=0).stop 2024-09-11T21:58:26.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.176+0000 7fe6aa6dd740 1 -- 172.21.15.1:0/3892654798 shutdown_connections 2024-09-11T21:58:26.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.176+0000 7fe6aa6dd740 1 --2- 172.21.15.1:0/3892654798 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55b834a59c70 0x55b834ac7d00 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:26.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.176+0000 7fe6aa6dd740 1 --2- 172.21.15.1:0/3892654798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b834a4a240 0x55b834a4f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:26.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.176+0000 7fe6aa6dd740 1 --2- 172.21.15.1:0/3892654798 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b834a3f980 0x55b834a4f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:26.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.176+0000 7fe689ffb640 0 client.25357 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T21:58:26.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.176+0000 7fe6aa6dd740 1 --2- 172.21.15.1:0/3892654798 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b834a3f060 0x55b834a52640 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:26.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.176+0000 7fe6aa6dd740 1 -- 172.21.15.1:0/3892654798 >> 172.21.15.1:0/3892654798 conn(0x55b834a3b5f0 msgr2=0x55b834a39f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:26.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.176+0000 7fe6aa6dd740 1 -- 172.21.15.1:0/3892654798 shutdown_connections 2024-09-11T21:58:26.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.176+0000 7fe6aa6dd740 1 -- 172.21.15.1:0/3892654798 wait complete. 2024-09-11T21:58:26.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.176+0000 7fe6aa6dd740 20 client.25357 trim_cache size 0 max 16384 2024-09-11T21:58:26.179 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-11T21:58:26.179 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 28, 2024-09-11T21:58:26.179 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24494, 2024-09-11T21:58:26.179 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 4, 2024-09-11T21:58:26.179 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-09-11T21:58:26.179 INFO:teuthology.orchestra.run.smithi001.stdout: "f7fcf15b": { 2024-09-11T21:58:26.179 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 4, 2024-09-11T21:58:26.179 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-09-11T21:58:26.179 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-11T21:58:26.179 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RELEASED", 2024-09-11T21:58:26.180 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-11T21:58:26.180 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T21:58:26.180 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 90.0, 2024-09-11T21:58:26.180 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 144.5, 2024-09-11T21:58:26.180 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-09-11T21:58:26.180 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-09-11T21:58:26.180 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-09-11T21:58:26.180 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-11T21:58:26.180 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RELEASED", 2024-09-11T21:58:26.180 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-11T21:58:26.180 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T21:58:26.180 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T21:58:26.180 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T21:58:26.181 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T21:58:26.181 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T21:58:26.181 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-11T21:58:26.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.177+0000 7fe6a8304640 1 -- 172.21.15.1:0/1125403619 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fe68c0a2e90 msgr2=0x7fe68c0a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:26.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.177+0000 7fe6a8304640 1 --2- 172.21.15.1:0/1125403619 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fe68c0a2e90 0x7fe68c0a5350 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe698010e60 tx=0x7fe698010f90 comp rx=0 tx=0).stop 2024-09-11T21:58:26.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.177+0000 7fe6a8304640 1 -- 172.21.15.1:0/1125403619 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6a0100e20 msgr2=0x7fe6a0198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:26.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.177+0000 7fe6a8304640 1 --2- 172.21.15.1:0/1125403619 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6a0100e20 0x7fe6a0198410 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fe69c00ae20 tx=0x7fe69c007590 comp rx=0 tx=0).stop 2024-09-11T21:58:26.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.178+0000 7fe6a8304640 1 -- 172.21.15.1:0/1125403619 shutdown_connections 2024-09-11T21:58:26.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.178+0000 7fe6a8304640 1 --2- 172.21.15.1:0/1125403619 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fe68c0a2e90 0x7fe68c0a5350 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:26.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.178+0000 7fe6a8304640 1 --2- 172.21.15.1:0/1125403619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6a0102ef0 0x7fe6a019cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:26.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.178+0000 7fe6a8304640 1 --2- 172.21.15.1:0/1125403619 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6a0102570 0x7fe6a0198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:26.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.178+0000 7fe6a8304640 1 --2- 172.21.15.1:0/1125403619 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6a0100e20 0x7fe6a0198410 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:26.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.178+0000 7fe6a8304640 1 -- 172.21.15.1:0/1125403619 >> 172.21.15.1:0/1125403619 conn(0x7fe6a00fc7d0 msgr2=0x7fe6a0105d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:26.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.178+0000 7fe6a8304640 1 -- 172.21.15.1:0/1125403619 shutdown_connections 2024-09-11T21:58:26.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:26.178+0000 7fe6a8304640 1 -- 172.21.15.1:0/1125403619 wait complete. 2024-09-11T21:58:26.992 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'f7fcf15b', total seconds elapsed: 28.3 2024-09-11T21:58:27.113 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:58:27 smithi001 ceph-mon[30935]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.3 KiB/s wr, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-11T21:58:27.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:58:27 smithi053 ceph-mon[34908]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.3 KiB/s wr, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-11T21:58:27.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:58:27 smithi161 ceph-mon[35615]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.3 KiB/s wr, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-11T21:58:29.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:58:29 smithi001 ceph-mon[30935]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-11T21:58:29.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:58:29 smithi053 ceph-mon[34908]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-11T21:58:29.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:58:29 smithi161 ceph-mon[35615]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-11T21:58:31.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:58:31 smithi001 ceph-mon[30935]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 14 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-11T21:58:31.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:58:31 smithi053 ceph-mon[34908]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 14 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-11T21:58:31.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:58:31 smithi161 ceph-mon[35615]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 14 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-11T21:58:31.731 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.730+0000 7f5dc16ae640 1 -- 172.21.15.1:0/1210566876 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5dbc102470 msgr2=0x7f5dbc10e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:31.732 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.730+0000 7f5dc16ae640 1 --2- 172.21.15.1:0/1210566876 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5dbc102470 0x7f5dbc10e9a0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f5db4009cd0 tx=0x7f5db402a8e0 comp rx=0 tx=0).stop 2024-09-11T21:58:31.732 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.730+0000 7f5dc16ae640 1 -- 172.21.15.1:0/1210566876 shutdown_connections 2024-09-11T21:58:31.732 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.730+0000 7f5dc16ae640 1 --2- 172.21.15.1:0/1210566876 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5dbc102470 0x7f5dbc10e9a0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:31.732 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.730+0000 7f5dc16ae640 1 --2- 172.21.15.1:0/1210566876 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5dbc101ad0 0x7f5dbc101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:31.732 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.730+0000 7f5dc16ae640 1 --2- 172.21.15.1:0/1210566876 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5dbc107b70 0x7f5dbc107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:31.732 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.730+0000 7f5dc16ae640 1 -- 172.21.15.1:0/1210566876 >> 172.21.15.1:0/1210566876 conn(0x7f5dbc0fd7d0 msgr2=0x7f5dbc0ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:31.732 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.731+0000 7f5dc16ae640 1 -- 172.21.15.1:0/1210566876 shutdown_connections 2024-09-11T21:58:31.732 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.731+0000 7f5dc16ae640 1 -- 172.21.15.1:0/1210566876 wait complete. 2024-09-11T21:58:31.732 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.731+0000 7f5dc16ae640 1 Processor -- start 2024-09-11T21:58:31.732 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.731+0000 7f5dc16ae640 1 -- start start 2024-09-11T21:58:31.734 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.732+0000 7f5dc16ae640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5dbc101ad0 0x7f5dbc198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:31.734 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.732+0000 7f5dc16ae640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5dbc102470 0x7f5dbc1988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:31.734 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.732+0000 7f5dc16ae640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5dbc107b70 0x7f5dbc19cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:31.734 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.732+0000 7f5dc16ae640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5dbc111ee0 con 0x7f5dbc102470 2024-09-11T21:58:31.734 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.732+0000 7f5dc16ae640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5dbc111d60 con 0x7f5dbc101ad0 2024-09-11T21:58:31.734 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.732+0000 7f5dc16ae640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5dbc112060 con 0x7f5dbc107b70 2024-09-11T21:58:31.734 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.732+0000 7f5dba7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5dbc102470 0x7f5dbc1988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:31.734 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.732+0000 7f5dba7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5dbc102470 0x7f5dbc1988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35254/0 (socket says 172.21.15.1:35254) 2024-09-11T21:58:31.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.732+0000 7f5dba7fc640 1 -- 172.21.15.1:0/3962070621 learned_addr learned my addr 172.21.15.1:0/3962070621 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:58:31.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.732+0000 7f5dbaffd640 1 --2- 172.21.15.1:0/3962070621 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5dbc101ad0 0x7f5dbc198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:31.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.732+0000 7f5dbb7fe640 1 --2- 172.21.15.1:0/3962070621 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5dbc107b70 0x7f5dbc19cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:31.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.732+0000 7f5dba7fc640 1 -- 172.21.15.1:0/3962070621 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5dbc107b70 msgr2=0x7f5dbc19cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:31.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.732+0000 7f5dba7fc640 1 --2- 172.21.15.1:0/3962070621 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5dbc107b70 0x7f5dbc19cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:31.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.732+0000 7f5dba7fc640 1 -- 172.21.15.1:0/3962070621 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5dbc101ad0 msgr2=0x7f5dbc198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:31.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.732+0000 7f5dba7fc640 1 --2- 172.21.15.1:0/3962070621 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5dbc101ad0 0x7f5dbc198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:31.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.732+0000 7f5dba7fc640 1 -- 172.21.15.1:0/3962070621 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dbc19d360 con 0x7f5dbc102470 2024-09-11T21:58:31.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.732+0000 7f5dbb7fe640 1 --2- 172.21.15.1:0/3962070621 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5dbc107b70 0x7f5dbc19cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:58:31.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.732+0000 7f5dba7fc640 1 --2- 172.21.15.1:0/3962070621 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5dbc102470 0x7f5dbc1988b0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f5db000d950 tx=0x7f5db000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:31.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.732+0000 7f5da3fff640 1 -- 172.21.15.1:0/3962070621 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db0017030 con 0x7f5dbc102470 2024-09-11T21:58:31.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.732+0000 7f5da3fff640 1 -- 172.21.15.1:0/3962070621 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5db0004020 con 0x7f5dbc102470 2024-09-11T21:58:31.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.733+0000 7f5dc16ae640 1 -- 172.21.15.1:0/3962070621 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dbc19d650 con 0x7f5dbc102470 2024-09-11T21:58:31.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.733+0000 7f5da3fff640 1 -- 172.21.15.1:0/3962070621 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db00047a0 con 0x7f5dbc102470 2024-09-11T21:58:31.736 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.733+0000 7f5dc16ae640 1 -- 172.21.15.1:0/3962070621 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dbc1a4ee0 con 0x7f5dbc102470 2024-09-11T21:58:31.741 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.735+0000 7f5da3fff640 1 -- 172.21.15.1:0/3962070621 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) ==== 138065+0+0 (secure 0 0 0) 0x7f5db00041d0 con 0x7f5dbc102470 2024-09-11T21:58:31.741 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.735+0000 7f5dc4448740 1 Processor -- start 2024-09-11T21:58:31.741 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.735+0000 7f5dc4448740 1 -- start start 2024-09-11T21:58:31.741 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.735+0000 7f5dc4448740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc429abaf0 0x55fc429abed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:31.741 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.735+0000 7f5dc4448740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fc429ac410 0x55fc429b6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:31.741 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.735+0000 7f5dc4448740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc429b6cb0 0x55fc429b90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:31.741 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.735+0000 7f5dc4448740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55fc429102f0 con 0x55fc429ac410 2024-09-11T21:58:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.735+0000 7f5dc4448740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55fc428b8270 con 0x55fc429abaf0 2024-09-11T21:58:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.735+0000 7f5dc4448740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55fc4290c2b0 con 0x55fc429b6cb0 2024-09-11T21:58:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.736+0000 7f5dbaffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fc429ac410 0x55fc429b6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.736+0000 7f5dbaffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fc429ac410 0x55fc429b6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35268/0 (socket says 172.21.15.1:35268) 2024-09-11T21:58:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.736+0000 7f5dbaffd640 1 -- 172.21.15.1:0/2356188191 learned_addr learned my addr 172.21.15.1:0/2356188191 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:58:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.736+0000 7f5dba7fc640 1 --2- 172.21.15.1:0/2356188191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc429b6cb0 0x55fc429b90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.736+0000 7f5da3fff640 1 --2- 172.21.15.1:0/3962070621 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f5da40a2e90 0x7f5da40a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.736+0000 7f5dbb7fe640 1 --2- 172.21.15.1:0/2356188191 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc429abaf0 0x55fc429abed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.736+0000 7f5dbaffd640 1 -- 172.21.15.1:0/2356188191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc429b6cb0 msgr2=0x55fc429b90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.736+0000 7f5dbaffd640 1 --2- 172.21.15.1:0/2356188191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc429b6cb0 0x55fc429b90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.736+0000 7f5dbaffd640 1 -- 172.21.15.1:0/2356188191 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc429abaf0 msgr2=0x55fc429abed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.736+0000 7f5dbaffd640 1 --2- 172.21.15.1:0/2356188191 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc429abaf0 0x55fc429abed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.736+0000 7f5dbaffd640 1 -- 172.21.15.1:0/2356188191 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc428b7340 con 0x55fc429ac410 2024-09-11T21:58:31.743 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.736+0000 7f5dbb7fe640 1 --2- 172.21.15.1:0/3962070621 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f5da40a2e90 0x7f5da40a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:31.743 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.736+0000 7f5da3fff640 1 -- 172.21.15.1:0/3962070621 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f5db00cf110 con 0x7f5dbc102470 2024-09-11T21:58:31.743 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.736+0000 7f5dbaffd640 1 --2- 172.21.15.1:0/2356188191 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fc429ac410 0x55fc429b6770 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f5dac00bc90 tx=0x7f5dac0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=481d299d44baaef3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:31.743 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.736+0000 7f5da17fa640 1 -- 172.21.15.1:0/2356188191 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dac01c480 con 0x55fc429ac410 2024-09-11T21:58:31.743 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.736+0000 7f5da17fa640 1 -- 172.21.15.1:0/2356188191 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5dac01cb00 con 0x55fc429ac410 2024-09-11T21:58:31.743 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.736+0000 7f5da17fa640 1 -- 172.21.15.1:0/2356188191 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dac011bb0 con 0x55fc429ac410 2024-09-11T21:58:31.743 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.737+0000 7f5dc4448740 1 -- 172.21.15.1:0/2356188191 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fc429ac410 msgr2=0x55fc429b6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:31.743 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.737+0000 7f5dbb7fe640 1 --2- 172.21.15.1:0/3962070621 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f5da40a2e90 0x7f5da40a5350 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5db4009a60 tx=0x7f5db403e040 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:31.743 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.737+0000 7f5dc4448740 1 --2- 172.21.15.1:0/2356188191 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fc429ac410 0x55fc429b6770 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f5dac00bc90 tx=0x7f5dac0107a0 comp rx=0 tx=0).stop 2024-09-11T21:58:31.743 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.737+0000 7f5dc4448740 1 -- 172.21.15.1:0/2356188191 shutdown_connections 2024-09-11T21:58:31.743 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.737+0000 7f5dc4448740 1 --2- 172.21.15.1:0/2356188191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc429b6cb0 0x55fc429b90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:31.743 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.737+0000 7f5dc4448740 1 --2- 172.21.15.1:0/2356188191 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fc429ac410 0x55fc429b6770 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:31.743 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.737+0000 7f5dc4448740 1 --2- 172.21.15.1:0/2356188191 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc429abaf0 0x55fc429abed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:31.743 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.737+0000 7f5dc4448740 1 -- 172.21.15.1:0/2356188191 >> 172.21.15.1:0/2356188191 conn(0x55fc429a8060 msgr2=0x55fc429a8440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.737+0000 7f5dc4448740 1 -- 172.21.15.1:0/2356188191 shutdown_connections 2024-09-11T21:58:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.737+0000 7f5dc4448740 1 -- 172.21.15.1:0/2356188191 wait complete. 2024-09-11T21:58:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.737+0000 7f5dc4448740 1 -- start start 2024-09-11T21:58:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.737+0000 7f5dc4448740 1 Processor -- start 2024-09-11T21:58:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.738+0000 7f5dc4448740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc429abaf0 0x55fc429b3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.738+0000 7f5dc4448740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fc429ac410 0x55fc429b4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.738+0000 7f5dc4448740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc429b6cb0 0x55fc429b45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.738+0000 7f5dc4448740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55fc429b9880 con 0x55fc429ac410 2024-09-11T21:58:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.738+0000 7f5dc4448740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55fc42899290 con 0x55fc429abaf0 2024-09-11T21:58:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.738+0000 7f5dc4448740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55fc429b9a00 con 0x55fc429b6cb0 2024-09-11T21:58:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.738+0000 7f5dc4448740 10 client.0 _ll_register_callbacks cb 0x55fc428ca140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:58:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.738+0000 7f5dba7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fc429ac410 0x55fc429b4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.738+0000 7f5dba7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fc429ac410 0x55fc429b4060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35280/0 (socket says 172.21.15.1:35280) 2024-09-11T21:58:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.738+0000 7f5dba7fc640 1 -- 172.21.15.1:0/1506410058 learned_addr learned my addr 172.21.15.1:0/1506410058 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:58:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.738+0000 7f5dbb7fe640 1 --2- 172.21.15.1:0/1506410058 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc429b6cb0 0x55fc429b45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.738+0000 7f5dbaffd640 1 --2- 172.21.15.1:0/1506410058 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc429abaf0 0x55fc429b3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.738+0000 7f5dba7fc640 1 -- 172.21.15.1:0/1506410058 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc429b6cb0 msgr2=0x55fc429b45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.738+0000 7f5dba7fc640 1 --2- 172.21.15.1:0/1506410058 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc429b6cb0 0x55fc429b45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.738+0000 7f5dba7fc640 1 -- 172.21.15.1:0/1506410058 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc429abaf0 msgr2=0x55fc429b3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.738+0000 7f5dba7fc640 1 --2- 172.21.15.1:0/1506410058 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc429abaf0 0x55fc429b3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.738+0000 7f5dba7fc640 1 -- 172.21.15.1:0/1506410058 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc42623210 con 0x55fc429ac410 2024-09-11T21:58:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.738+0000 7f5dba7fc640 1 --2- 172.21.15.1:0/1506410058 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fc429ac410 0x55fc429b4060 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f5db009b040 tx=0x7f5db009af40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac523dc265631fac server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.738+0000 7f5d9eff5640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T21:58:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.738+0000 7f5d9eff5640 1 -- 172.21.15.1:0/1506410058 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db009b340 con 0x55fc429ac410 2024-09-11T21:58:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.738+0000 7f5d9eff5640 1 -- 172.21.15.1:0/1506410058 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f5db00d1070 con 0x55fc429ac410 2024-09-11T21:58:31.746 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.738+0000 7f5dc4448740 1 -- 172.21.15.1:0/1506410058 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5dac01cb00 con 0x55fc429ac410 2024-09-11T21:58:31.746 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.739+0000 7f5d9eff5640 1 -- 172.21.15.1:0/1506410058 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5db009ba40 con 0x55fc429ac410 2024-09-11T21:58:31.746 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.739+0000 7f5dc4448740 10 client.15807 fetch_fsmap learned FSMap version 28 2024-09-11T21:58:31.746 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.739+0000 7f5dc4448740 1 -- 172.21.15.1:0/1506410058 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fc426ef620 con 0x55fc429ac410 2024-09-11T21:58:31.746 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.739+0000 7f5d9eff5640 1 -- 172.21.15.1:0/1506410058 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5dac01cb00 con 0x55fc429ac410 2024-09-11T21:58:31.746 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.739+0000 7f5d9eff5640 1 -- 172.21.15.1:0/1506410058 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5db009a790 con 0x55fc429ac410 2024-09-11T21:58:31.746 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.739+0000 7f5dc4448740 10 client.15807 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T21:58:31.746 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.739+0000 7f5dc4448740 10 client.15807 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T21:58:31.746 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.739+0000 7f5dc4448740 20 client.15807 populate_metadata read hostname 'smithi001' 2024-09-11T21:58:31.746 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.739+0000 7f5dc4448740 1 --2- 172.21.15.1:0/1506410058 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55fc429c6b80 0x55fc429c8f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:31.746 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.739+0000 7f5dc4448740 4 client.15807 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T21:58:31.746 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.739+0000 7f5dc4448740 1 -- 172.21.15.1:0/1506410058 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc4291da20 con 0x55fc429c6b80 2024-09-11T21:58:31.746 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.739+0000 7f5dbaffd640 1 --2- 172.21.15.1:0/1506410058 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55fc429c6b80 0x55fc429c8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:31.746 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.740+0000 7f5dbaffd640 1 --2- 172.21.15.1:0/1506410058 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55fc429c6b80 0x55fc429c8f90 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd57610cf1f25cf server_cookie=8b029c4504bad514 in_seq=0 out_seq=0 2024-09-11T21:58:31.747 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.740+0000 7f5d9eff5640 10 client.15807 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T21:58:31.747 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.740+0000 7f5d9eff5640 1 -- 172.21.15.1:0/1506410058 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fc4291da20 con 0x55fc429c6b80 2024-09-11T21:58:31.747 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.740+0000 7f5d9eff5640 10 client.15807 handle_command_reply: tid=0 2024-09-11T21:58:31.747 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.740+0000 7f5dc4448740 1 client.15807 shutdown 2024-09-11T21:58:31.747 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.740+0000 7f5dc4448740 1 -- 172.21.15.1:0/1506410058 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fc429ac410 msgr2=0x55fc429b4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:31.747 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.740+0000 7f5dc4448740 1 --2- 172.21.15.1:0/1506410058 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fc429ac410 0x55fc429b4060 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f5db009b040 tx=0x7f5db009af40 comp rx=0 tx=0).stop 2024-09-11T21:58:31.747 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.740+0000 7f5dc4448740 1 -- 172.21.15.1:0/1506410058 shutdown_connections 2024-09-11T21:58:31.747 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.740+0000 7f5dc4448740 1 --2- 172.21.15.1:0/1506410058 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55fc429c6b80 0x55fc429c8f90 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:31.747 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.740+0000 7f5dc4448740 1 --2- 172.21.15.1:0/1506410058 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc429b6cb0 0x55fc429b45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:31.747 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.740+0000 7f5dc4448740 1 --2- 172.21.15.1:0/1506410058 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fc429ac410 0x55fc429b4060 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:31.747 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.740+0000 7f5dc4448740 1 --2- 172.21.15.1:0/1506410058 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc429abaf0 0x55fc429b3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:31.747 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.740+0000 7f5d9eff5640 0 client.15807 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T21:58:31.747 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.740+0000 7f5dc4448740 1 -- 172.21.15.1:0/1506410058 >> 172.21.15.1:0/1506410058 conn(0x55fc429a8060 msgr2=0x55fc429addc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:31.748 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.741+0000 7f5dc4448740 1 -- 172.21.15.1:0/1506410058 shutdown_connections 2024-09-11T21:58:31.748 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.741+0000 7f5dc4448740 1 -- 172.21.15.1:0/1506410058 wait complete. 2024-09-11T21:58:31.748 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.741+0000 7f5dc4448740 20 client.15807 trim_cache size 0 max 16384 2024-09-11T21:58:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.755+0000 7f5dc4448740 1 Processor -- start 2024-09-11T21:58:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.755+0000 7f5dc4448740 1 -- start start 2024-09-11T21:58:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.755+0000 7f5dc4448740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc429abaf0 0x55fc429b2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.755+0000 7f5dc4448740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc429ac410 0x55fc429af250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.755+0000 7f5dc4448740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fc429b6cb0 0x55fc429b43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.755+0000 7f5dc4448740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5dbc1ada60 con 0x55fc429b6cb0 2024-09-11T21:58:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.755+0000 7f5dc4448740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5dbc1af5a0 con 0x55fc429ac410 2024-09-11T21:58:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.755+0000 7f5dc4448740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5dbc1ae130 con 0x55fc429abaf0 2024-09-11T21:58:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.756+0000 7f5dbb7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fc429b6cb0 0x55fc429b43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.756+0000 7f5dbb7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fc429b6cb0 0x55fc429b43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35282/0 (socket says 172.21.15.1:35282) 2024-09-11T21:58:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.756+0000 7f5dbb7fe640 1 -- 172.21.15.1:0/3649212010 learned_addr learned my addr 172.21.15.1:0/3649212010 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:58:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.756+0000 7f5dbaffd640 1 --2- 172.21.15.1:0/3649212010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc429abaf0 0x55fc429b2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.756+0000 7f5dba7fc640 1 --2- 172.21.15.1:0/3649212010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc429ac410 0x55fc429af250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.756+0000 7f5dbb7fe640 1 -- 172.21.15.1:0/3649212010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc429abaf0 msgr2=0x55fc429b2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.756+0000 7f5dbb7fe640 1 --2- 172.21.15.1:0/3649212010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc429abaf0 0x55fc429b2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.756+0000 7f5dbb7fe640 1 -- 172.21.15.1:0/3649212010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc429ac410 msgr2=0x55fc429af250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.756+0000 7f5dbb7fe640 1 --2- 172.21.15.1:0/3649212010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc429ac410 0x55fc429af250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.756+0000 7f5dbb7fe640 1 -- 172.21.15.1:0/3649212010 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc429ac240 con 0x55fc429b6cb0 2024-09-11T21:58:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.756+0000 7f5dbb7fe640 1 --2- 172.21.15.1:0/3649212010 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fc429b6cb0 0x55fc429b43d0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f5db4035000 tx=0x7f5db4040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fa68be1dfbb84a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.756+0000 7f5da0ff9640 1 -- 172.21.15.1:0/3649212010 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db4003b70 con 0x55fc429b6cb0 2024-09-11T21:58:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.756+0000 7f5da0ff9640 1 -- 172.21.15.1:0/3649212010 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5db4036040 con 0x55fc429b6cb0 2024-09-11T21:58:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.757+0000 7f5da0ff9640 1 -- 172.21.15.1:0/3649212010 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db4030800 con 0x55fc429b6cb0 2024-09-11T21:58:31.761 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.757+0000 7f5dc4448740 1 -- 172.21.15.1:0/3649212010 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fc429b6cb0 msgr2=0x55fc429b43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:31.761 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.757+0000 7f5dc4448740 1 --2- 172.21.15.1:0/3649212010 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fc429b6cb0 0x55fc429b43d0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f5db4035000 tx=0x7f5db4040000 comp rx=0 tx=0).stop 2024-09-11T21:58:31.762 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.757+0000 7f5dc4448740 1 -- 172.21.15.1:0/3649212010 shutdown_connections 2024-09-11T21:58:31.762 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.757+0000 7f5dc4448740 1 --2- 172.21.15.1:0/3649212010 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fc429b6cb0 0x55fc429b43d0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:31.762 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.757+0000 7f5dc4448740 1 --2- 172.21.15.1:0/3649212010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc429ac410 0x55fc429af250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:31.762 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.757+0000 7f5dc4448740 1 --2- 172.21.15.1:0/3649212010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc429abaf0 0x55fc429b2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:31.762 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.757+0000 7f5dc4448740 1 -- 172.21.15.1:0/3649212010 >> 172.21.15.1:0/3649212010 conn(0x55fc429a8060 msgr2=0x55fc429b5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:31.762 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.757+0000 7f5dc4448740 1 -- 172.21.15.1:0/3649212010 shutdown_connections 2024-09-11T21:58:31.762 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.757+0000 7f5dc4448740 1 -- 172.21.15.1:0/3649212010 wait complete. 2024-09-11T21:58:31.762 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.758+0000 7f5dc4448740 1 -- start start 2024-09-11T21:58:31.762 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.758+0000 7f5dc4448740 1 Processor -- start 2024-09-11T21:58:31.762 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.758+0000 7f5dc4448740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc429abaf0 0x55fc429bf180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:31.762 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.758+0000 7f5dc4448740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fc429ac410 0x55fc429ba4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:31.762 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.758+0000 7f5dc4448740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc429b6cb0 0x55fc429baa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:31.762 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.758+0000 7f5dc4448740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5dbc1aeed0 con 0x55fc429ac410 2024-09-11T21:58:31.763 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.758+0000 7f5dc4448740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5dbc1ae800 con 0x55fc429abaf0 2024-09-11T21:58:31.763 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.758+0000 7f5dc4448740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5dbc1aba40 con 0x55fc429b6cb0 2024-09-11T21:58:31.763 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.758+0000 7f5dc4448740 10 client.0 _ll_register_callbacks cb 0x7f5db0005d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:58:31.763 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.758+0000 7f5dba7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fc429ac410 0x55fc429ba4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:31.763 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.758+0000 7f5dba7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fc429ac410 0x55fc429ba4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35284/0 (socket says 172.21.15.1:35284) 2024-09-11T21:58:31.763 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.758+0000 7f5dba7fc640 1 -- 172.21.15.1:0/563448757 learned_addr learned my addr 172.21.15.1:0/563448757 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:58:31.763 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.758+0000 7f5dbaffd640 1 --2- 172.21.15.1:0/563448757 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc429abaf0 0x55fc429bf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:31.763 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.758+0000 7f5dbb7fe640 1 --2- 172.21.15.1:0/563448757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc429b6cb0 0x55fc429baa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:31.763 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.758+0000 7f5dba7fc640 1 -- 172.21.15.1:0/563448757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc429b6cb0 msgr2=0x55fc429baa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:31.763 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.758+0000 7f5dba7fc640 1 --2- 172.21.15.1:0/563448757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc429b6cb0 0x55fc429baa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:31.763 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.758+0000 7f5dba7fc640 1 -- 172.21.15.1:0/563448757 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc429abaf0 msgr2=0x55fc429bf180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:31.763 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.758+0000 7f5dba7fc640 1 --2- 172.21.15.1:0/563448757 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc429abaf0 0x55fc429bf180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:31.763 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.758+0000 7f5dba7fc640 1 -- 172.21.15.1:0/563448757 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc429baff0 con 0x55fc429ac410 2024-09-11T21:58:31.763 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.758+0000 7f5dbb7fe640 1 --2- 172.21.15.1:0/563448757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc429b6cb0 0x55fc429baa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:58:31.764 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.759+0000 7f5dba7fc640 1 --2- 172.21.15.1:0/563448757 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fc429ac410 0x55fc429ba4f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f5db009ab90 tx=0x7f5db0097be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=752771100c439f0a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:31.764 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.759+0000 7f5d9eff5640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T21:58:31.764 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.759+0000 7f5d9eff5640 1 -- 172.21.15.1:0/563448757 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db009a9a0 con 0x55fc429ac410 2024-09-11T21:58:31.764 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.759+0000 7f5d9eff5640 1 -- 172.21.15.1:0/563448757 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f5db0097290 con 0x55fc429ac410 2024-09-11T21:58:31.764 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.759+0000 7f5dc4448740 1 -- 172.21.15.1:0/563448757 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5db4036040 con 0x55fc429ac410 2024-09-11T21:58:31.764 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.759+0000 7f5d9eff5640 1 -- 172.21.15.1:0/563448757 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5db009b340 con 0x55fc429ac410 2024-09-11T21:58:31.764 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.759+0000 7f5dc4448740 10 client.15813 fetch_fsmap learned FSMap version 28 2024-09-11T21:58:31.764 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.759+0000 7f5dc4448740 1 -- 172.21.15.1:0/563448757 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fc42a315e0 con 0x55fc429ac410 2024-09-11T21:58:31.764 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.759+0000 7f5d9eff5640 1 -- 172.21.15.1:0/563448757 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5db4036040 con 0x55fc429ac410 2024-09-11T21:58:31.764 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.759+0000 7f5d9eff5640 1 -- 172.21.15.1:0/563448757 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5db009f130 con 0x55fc429ac410 2024-09-11T21:58:31.764 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.759+0000 7f5dc4448740 10 client.15813 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T21:58:31.764 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.759+0000 7f5dc4448740 10 client.15813 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T21:58:31.764 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.759+0000 7f5dc4448740 20 client.15813 populate_metadata read hostname 'smithi001' 2024-09-11T21:58:31.765 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.759+0000 7f5dc4448740 1 --2- 172.21.15.1:0/563448757 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55fc42a31d60 0x55fc42a34180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:31.765 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.759+0000 7f5dc4448740 4 client.15813 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T21:58:31.765 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.759+0000 7f5dc4448740 1 -- 172.21.15.1:0/563448757 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fc42917d70 con 0x55fc42a31d60 2024-09-11T21:58:31.765 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.760+0000 7f5dbaffd640 1 --2- 172.21.15.1:0/563448757 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55fc42a31d60 0x55fc42a34180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:31.765 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.760+0000 7f5dbaffd640 1 --2- 172.21.15.1:0/563448757 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55fc42a31d60 0x55fc42a34180 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13e91d7e8b7a9824 server_cookie=aafb119d66996c9a in_seq=0 out_seq=0 2024-09-11T21:58:31.765 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.760+0000 7f5d9eff5640 10 client.15813 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T21:58:31.766 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.763+0000 7f5d9eff5640 1 -- 172.21.15.1:0/563448757 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fc42917d70 con 0x55fc42a31d60 2024-09-11T21:58:31.766 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.763+0000 7f5d9eff5640 10 client.15813 handle_command_reply: tid=0 2024-09-11T21:58:31.766 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.763+0000 7f5dc4448740 1 client.15813 shutdown 2024-09-11T21:58:31.766 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.764+0000 7f5dc4448740 1 -- 172.21.15.1:0/563448757 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fc429ac410 msgr2=0x55fc429ba4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:31.766 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.764+0000 7f5dc4448740 1 --2- 172.21.15.1:0/563448757 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fc429ac410 0x55fc429ba4f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f5db009ab90 tx=0x7f5db0097be0 comp rx=0 tx=0).stop 2024-09-11T21:58:31.767 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.764+0000 7f5dc4448740 1 -- 172.21.15.1:0/563448757 shutdown_connections 2024-09-11T21:58:31.767 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.764+0000 7f5dc4448740 1 --2- 172.21.15.1:0/563448757 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55fc42a31d60 0x55fc42a34180 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:31.767 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.764+0000 7f5dc4448740 1 --2- 172.21.15.1:0/563448757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc429b6cb0 0x55fc429baa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:31.767 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.764+0000 7f5dc4448740 1 --2- 172.21.15.1:0/563448757 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fc429ac410 0x55fc429ba4f0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:31.767 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.764+0000 7f5d9eff5640 0 client.15813 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T21:58:31.767 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.764+0000 7f5dc4448740 1 --2- 172.21.15.1:0/563448757 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc429abaf0 0x55fc429bf180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:31.767 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.764+0000 7f5dc4448740 1 -- 172.21.15.1:0/563448757 >> 172.21.15.1:0/563448757 conn(0x55fc429a8060 msgr2=0x55fc429a69a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:31.767 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.764+0000 7f5dc4448740 1 -- 172.21.15.1:0/563448757 shutdown_connections 2024-09-11T21:58:31.767 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.764+0000 7f5dc4448740 1 -- 172.21.15.1:0/563448757 wait complete. 2024-09-11T21:58:31.767 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.764+0000 7f5dc4448740 20 client.15813 trim_cache size 0 max 16384 2024-09-11T21:58:31.767 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-09-11T21:58:31.768 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-09-11T21:58:31.768 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-09-11T21:58:31.768 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-09-11T21:58:31.768 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.764+0000 7f5dc16ae640 1 -- 172.21.15.1:0/3962070621 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f5da40a2e90 msgr2=0x7f5da40a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:31.769 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.764+0000 7f5dc16ae640 1 --2- 172.21.15.1:0/3962070621 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f5da40a2e90 0x7f5da40a5350 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5db4009a60 tx=0x7f5db403e040 comp rx=0 tx=0).stop 2024-09-11T21:58:31.769 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.764+0000 7f5dc16ae640 1 -- 172.21.15.1:0/3962070621 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5dbc102470 msgr2=0x7f5dbc1988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:31.769 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.764+0000 7f5dc16ae640 1 --2- 172.21.15.1:0/3962070621 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5dbc102470 0x7f5dbc1988b0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f5db000d950 tx=0x7f5db000de10 comp rx=0 tx=0).stop 2024-09-11T21:58:31.769 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.764+0000 7f5dc16ae640 1 -- 172.21.15.1:0/3962070621 shutdown_connections 2024-09-11T21:58:31.769 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.764+0000 7f5dc16ae640 1 --2- 172.21.15.1:0/3962070621 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f5da40a2e90 0x7f5da40a5350 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:31.769 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.764+0000 7f5dc16ae640 1 --2- 172.21.15.1:0/3962070621 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5dbc107b70 0x7f5dbc19cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:31.769 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.764+0000 7f5dc16ae640 1 --2- 172.21.15.1:0/3962070621 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5dbc102470 0x7f5dbc1988b0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:31.769 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.764+0000 7f5dc16ae640 1 --2- 172.21.15.1:0/3962070621 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5dbc101ad0 0x7f5dbc198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:31.769 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.764+0000 7f5dc16ae640 1 -- 172.21.15.1:0/3962070621 >> 172.21.15.1:0/3962070621 conn(0x7f5dbc0fd7d0 msgr2=0x7f5dbc0fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:31.769 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.765+0000 7f5dc16ae640 1 -- 172.21.15.1:0/3962070621 shutdown_connections 2024-09-11T21:58:31.769 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:31.765+0000 7f5dc16ae640 1 -- 172.21.15.1:0/3962070621 wait complete. 2024-09-11T21:58:33.411 INFO:tasks.workunit.client.0.smithi001.stderr:Updating files: 94% (12228/12903) Updating files: 95% (12258/12903) Updating files: 96% (12387/12903) Updating files: 97% (12516/12903) Updating files: 98% (12645/12903) Updating files: 99% (12774/12903) Updating files: 100% (12903/12903) Updating files: 100% (12903/12903), done. 2024-09-11T21:58:33.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:58:33 smithi001 ceph-mon[30935]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-09-11T21:58:33.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:58:33 smithi053 ceph-mon[34908]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-09-11T21:58:33.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:58:33 smithi161 ceph-mon[35615]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-09-11T21:58:34.139 INFO:tasks.workunit.client.0.smithi001.stderr:Note: switching to '086c737fa75c22d521a3932c59b31c5e1d1c9ef4'. 2024-09-11T21:58:34.139 INFO:tasks.workunit.client.0.smithi001.stderr: 2024-09-11T21:58:34.139 INFO:tasks.workunit.client.0.smithi001.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-11T21:58:34.140 INFO:tasks.workunit.client.0.smithi001.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-11T21:58:34.140 INFO:tasks.workunit.client.0.smithi001.stderr:state without impacting any branches by switching back to a branch. 2024-09-11T21:58:34.140 INFO:tasks.workunit.client.0.smithi001.stderr: 2024-09-11T21:58:34.140 INFO:tasks.workunit.client.0.smithi001.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-11T21:58:34.140 INFO:tasks.workunit.client.0.smithi001.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-11T21:58:34.140 INFO:tasks.workunit.client.0.smithi001.stderr: 2024-09-11T21:58:34.140 INFO:tasks.workunit.client.0.smithi001.stderr: git switch -c 2024-09-11T21:58:34.140 INFO:tasks.workunit.client.0.smithi001.stderr: 2024-09-11T21:58:34.140 INFO:tasks.workunit.client.0.smithi001.stderr:Or undo this operation with: 2024-09-11T21:58:34.140 INFO:tasks.workunit.client.0.smithi001.stderr: 2024-09-11T21:58:34.140 INFO:tasks.workunit.client.0.smithi001.stderr: git switch - 2024-09-11T21:58:34.140 INFO:tasks.workunit.client.0.smithi001.stderr: 2024-09-11T21:58:34.140 INFO:tasks.workunit.client.0.smithi001.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-11T21:58:34.141 INFO:tasks.workunit.client.0.smithi001.stderr: 2024-09-11T21:58:34.141 INFO:tasks.workunit.client.0.smithi001.stderr:HEAD is now at 086c737fa75 Merge pull request #59733 from zdover23/wip-doc-2024-09-11-backport-59635-to-squid 2024-09-11T21:58:34.146 DEBUG:teuthology.orchestra.run.smithi001:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-11T21:58:34.216 INFO:tasks.workunit.client.0.smithi001.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-11T21:58:34.218 INFO:tasks.workunit.client.0.smithi001.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-11T21:58:34.219 INFO:tasks.workunit.client.0.smithi001.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-11T21:58:34.828 INFO:tasks.workunit.client.0.smithi001.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-11T21:58:34.910 INFO:tasks.workunit.client.0.smithi001.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-11T21:58:34.951 INFO:tasks.workunit.client.0.smithi001.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-11T21:58:34.953 INFO:tasks.workunit.client.0.smithi001.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-11T21:58:34.953 INFO:tasks.workunit.client.0.smithi001.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-11T21:58:34.992 INFO:tasks.workunit.client.0.smithi001.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-11T21:58:34.996 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-11T21:58:34.996 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-11T21:58:35.053 INFO:tasks.workunit:Running workunits matching fs/misc on client.0... 2024-09-11T21:58:35.054 INFO:tasks.workunit:Running workunit fs/misc/acl.sh... 2024-09-11T21:58:35.055 DEBUG:teuthology.orchestra.run.smithi001:workunit test fs/misc/acl.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=086c737fa75c22d521a3932c59b31c5e1d1c9ef4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/acl.sh 2024-09-11T21:58:35.117 INFO:tasks.workunit.client.0.smithi001.stderr:+ set -e 2024-09-11T21:58:35.117 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir -p testdir 2024-09-11T21:58:35.119 INFO:tasks.workunit.client.0.smithi001.stderr:+ cd testdir 2024-09-11T21:58:35.119 INFO:tasks.workunit.client.0.smithi001.stderr:+ set +e 2024-09-11T21:58:35.119 INFO:tasks.workunit.client.0.smithi001.stderr:+ setfacl -d -m u:nobody:rw . 2024-09-11T21:58:35.128 INFO:tasks.workunit.client.0.smithi001.stderr:+ test 0 '!=' 0 2024-09-11T21:58:35.128 INFO:tasks.workunit.client.0.smithi001.stderr:+ set -e 2024-09-11T21:58:35.128 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=0 2024-09-11T21:58:35.128 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 0 -lt 100 ']' 2024-09-11T21:58:35.128 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 0 + 1 2024-09-11T21:58:35.129 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=1 2024-09-11T21:58:35.129 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T21:58:35.129 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:58:35.129 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:58:35.134 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T21:58:35.134 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T21:58:35.134 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T21:58:35.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:58:35 smithi001 ceph-mon[30935]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T21:58:35.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:58:35 smithi053 ceph-mon[34908]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T21:58:35.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:58:35 smithi161 ceph-mon[35615]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T21:58:35.919 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:58:35.919 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:58:36.101 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T21:58:36.102 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T21:58:36.107 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:58:36.107 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:58:36.107 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 1 -lt 100 ']' 2024-09-11T21:58:36.107 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 1 + 1 2024-09-11T21:58:36.131 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=2 2024-09-11T21:58:36.131 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T21:58:36.151 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:58:36.151 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:58:36.153 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T21:58:36.153 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T21:58:36.153 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T21:58:36.593 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:58:36.593 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:58:36.768 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T21:58:36.768 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T21:58:36.774 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:58:36.774 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:58:36.774 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -lt 100 ']' 2024-09-11T21:58:36.774 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 2 + 1 2024-09-11T21:58:36.823 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=3 2024-09-11T21:58:36.823 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T21:58:36.841 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:58:36.842 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:58:36.843 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T21:58:36.843 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T21:58:36.843 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T21:58:37.259 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:58:37.259 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:58:37.394 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:58:37 smithi161 ceph-mon[35615]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-11T21:58:37.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:58:37 smithi001 ceph-mon[30935]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-11T21:58:37.535 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T21:58:37.535 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T21:58:37.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:58:37 smithi053 ceph-mon[34908]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-11T21:58:37.558 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:58:37.558 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:58:37.559 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 3 -lt 100 ']' 2024-09-11T21:58:37.559 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 3 + 1 2024-09-11T21:58:37.589 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=4 2024-09-11T21:58:37.589 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T21:58:37.610 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:58:37.611 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:58:37.612 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T21:58:37.612 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T21:58:37.612 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T21:58:38.352 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:58:38.352 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:58:38.519 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T21:58:38.519 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T21:58:38.533 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:58:38.533 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:58:38.533 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 4 -lt 100 ']' 2024-09-11T21:58:38.533 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 4 + 1 2024-09-11T21:58:38.579 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=5 2024-09-11T21:58:38.579 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T21:58:38.647 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:58:38.647 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:58:38.648 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T21:58:38.649 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T21:58:38.649 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T21:58:39.360 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:58:39.360 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:58:39.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:58:39 smithi001 ceph-mon[30935]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-11T21:58:39.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:58:39 smithi053 ceph-mon[34908]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-11T21:58:39.550 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T21:58:39.550 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T21:58:39.558 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:58:39.558 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:58:39.558 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 5 -lt 100 ']' 2024-09-11T21:58:39.558 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 5 + 1 2024-09-11T21:58:39.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:58:39 smithi161 ceph-mon[35615]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-11T21:58:39.671 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=6 2024-09-11T21:58:39.671 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T21:58:39.701 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:58:39.701 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:58:39.702 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T21:58:39.703 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T21:58:39.703 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T21:58:40.377 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:58:40.377 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:58:40.641 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T21:58:40.641 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T21:58:40.649 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:58:40.649 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:58:40.649 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 6 -lt 100 ']' 2024-09-11T21:58:40.649 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 6 + 1 2024-09-11T21:58:40.739 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=7 2024-09-11T21:58:40.739 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T21:58:40.802 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:58:40.802 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:58:40.803 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T21:58:40.804 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T21:58:40.804 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T21:58:41.404 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:58:41 smithi001 ceph-mon[30935]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 2.5 KiB/s wr, 13 op/s 2024-09-11T21:58:41.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:58:41 smithi053 ceph-mon[34908]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 2.5 KiB/s wr, 13 op/s 2024-09-11T21:58:41.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:58:41 smithi161 ceph-mon[35615]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 2.5 KiB/s wr, 13 op/s 2024-09-11T21:58:41.643 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:58:41.643 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:58:41.877 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T21:58:41.877 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T21:58:41.891 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:58:41.891 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:58:41.891 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 7 -lt 100 ']' 2024-09-11T21:58:41.891 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 7 + 1 2024-09-11T21:58:41.962 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=8 2024-09-11T21:58:41.963 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T21:58:42.058 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:58:42.058 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:58:42.059 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T21:58:42.060 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T21:58:42.060 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T21:58:42.801 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:58:42.801 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:58:43.016 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T21:58:43.016 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T21:58:43.033 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:58:43.033 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:58:43.033 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 8 -lt 100 ']' 2024-09-11T21:58:43.033 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 8 + 1 2024-09-11T21:58:43.081 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=9 2024-09-11T21:58:43.081 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T21:58:43.134 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:58:43.134 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:58:43.135 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T21:58:43.135 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T21:58:43.135 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T21:58:43.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:58:43 smithi001 ceph-mon[30935]: pgmap v49: 97 pgs: 97 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-09-11T21:58:43.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:58:43 smithi053 ceph-mon[34908]: pgmap v49: 97 pgs: 97 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-09-11T21:58:43.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:58:43 smithi161 ceph-mon[35615]: pgmap v49: 97 pgs: 97 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-09-11T21:58:43.766 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:58:43.766 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:58:43.926 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T21:58:43.927 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T21:58:43.957 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:58:43.957 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:58:43.957 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 9 -lt 100 ']' 2024-09-11T21:58:43.957 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 9 + 1 2024-09-11T21:58:44.006 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=10 2024-09-11T21:58:44.006 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T21:58:44.050 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:58:44.050 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:58:44.051 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T21:58:44.052 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T21:58:44.052 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T21:58:44.406 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T21:58:44.667 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:58:44.668 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:58:44.900 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T21:58:44.900 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T21:58:44.923 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:58:44.923 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:58:44.923 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 10 -lt 100 ']' 2024-09-11T21:58:44.923 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 10 + 1 2024-09-11T21:58:44.988 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=11 2024-09-11T21:58:44.989 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T21:58:45.001 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:58:45.001 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:58:45.002 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T21:58:45.002 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T21:58:45.002 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T21:58:45.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:58:45 smithi001 ceph-mon[30935]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-09-11T21:58:45.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:58:45 smithi053 ceph-mon[34908]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-09-11T21:58:45.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:58:45 smithi161 ceph-mon[35615]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-09-11T21:58:45.851 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:58:45.851 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:58:46.059 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T21:58:46.059 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T21:58:46.082 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:58:46.082 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:58:46.082 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 11 -lt 100 ']' 2024-09-11T21:58:46.082 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 11 + 1 2024-09-11T21:58:46.152 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=12 2024-09-11T21:58:46.152 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T21:58:46.196 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:58:46.196 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:58:46.197 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T21:58:46.197 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T21:58:46.198 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T21:58:46.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:58:46 smithi001 ceph-mon[30935]: mgrmap e26: x(active, since 91s), standbys: y, z 2024-09-11T21:58:46.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:58:46 smithi053 ceph-mon[34908]: mgrmap e26: x(active, since 91s), standbys: y, z 2024-09-11T21:58:46.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:58:46 smithi161 ceph-mon[35615]: mgrmap e26: x(active, since 91s), standbys: y, z 2024-09-11T21:58:47.101 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:58:47.101 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:58:47.357 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T21:58:47.357 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T21:58:47.374 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:58:47.374 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:58:47.374 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 12 -lt 100 ']' 2024-09-11T21:58:47.375 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 12 + 1 2024-09-11T21:58:47.441 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=13 2024-09-11T21:58:47.441 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T21:58:47.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:58:47 smithi001 ceph-mon[30935]: pgmap v51: 97 pgs: 97 active+clean; 1.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-09-11T21:58:47.476 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:58:47.476 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:58:47.477 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T21:58:47.477 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T21:58:47.477 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T21:58:47.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:58:47 smithi053 ceph-mon[34908]: pgmap v51: 97 pgs: 97 active+clean; 1.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-09-11T21:58:47.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:58:47 smithi161 ceph-mon[35615]: pgmap v51: 97 pgs: 97 active+clean; 1.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-09-11T21:58:48.181 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.180+0000 7f5c1414b640 1 -- 172.21.15.1:0/3259690168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c0c107ad0 msgr2=0x7f5c0c107eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:48.181 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.180+0000 7f5c1414b640 1 --2- 172.21.15.1:0/3259690168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c0c107ad0 0x7f5c0c107eb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5c08009ca0 tx=0x7f5c0802a8c0 comp rx=0 tx=0).stop 2024-09-11T21:58:48.181 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.180+0000 7f5c1414b640 1 -- 172.21.15.1:0/3259690168 shutdown_connections 2024-09-11T21:58:48.181 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.180+0000 7f5c1414b640 1 --2- 172.21.15.1:0/3259690168 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c0c102460 0x7f5c0c10e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:48.181 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.180+0000 7f5c1414b640 1 --2- 172.21.15.1:0/3259690168 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c0c101ac0 0x7f5c0c101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:48.181 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.180+0000 7f5c1414b640 1 --2- 172.21.15.1:0/3259690168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c0c107ad0 0x7f5c0c107eb0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:48.181 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.180+0000 7f5c1414b640 1 -- 172.21.15.1:0/3259690168 >> 172.21.15.1:0/3259690168 conn(0x7f5c0c0fd7a0 msgr2=0x7f5c0c0ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:48.181 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.180+0000 7f5c1414b640 1 -- 172.21.15.1:0/3259690168 shutdown_connections 2024-09-11T21:58:48.182 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.180+0000 7f5c1414b640 1 -- 172.21.15.1:0/3259690168 wait complete. 2024-09-11T21:58:48.201 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.199+0000 7f5c1414b640 1 Processor -- start 2024-09-11T21:58:48.201 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.199+0000 7f5c1414b640 1 -- start start 2024-09-11T21:58:48.201 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.200+0000 7f5c1414b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c0c101ac0 0x7f5c0c1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:48.201 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.200+0000 7f5c1414b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c0c102460 0x7f5c0c198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:48.201 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.200+0000 7f5c1414b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c0c107ad0 0x7f5c0c19cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:48.201 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.200+0000 7f5c1414b640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5c0c111f60 con 0x7f5c0c101ac0 2024-09-11T21:58:48.202 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.200+0000 7f5c1414b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5c0c111de0 con 0x7f5c0c107ad0 2024-09-11T21:58:48.202 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.200+0000 7f5c1414b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5c0c1120e0 con 0x7f5c0c102460 2024-09-11T21:58:48.202 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.200+0000 7f5c11ee7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c0c101ac0 0x7f5c0c1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:48.202 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.200+0000 7f5c11ee7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c0c101ac0 0x7f5c0c1983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47624/0 (socket says 172.21.15.1:47624) 2024-09-11T21:58:48.202 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.200+0000 7f5c11ee7640 1 -- 172.21.15.1:0/1539218487 learned_addr learned my addr 172.21.15.1:0/1539218487 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:58:48.202 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.200+0000 7f5c126e8640 1 --2- 172.21.15.1:0/1539218487 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c0c107ad0 0x7f5c0c19cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:48.202 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.200+0000 7f5c116e6640 1 --2- 172.21.15.1:0/1539218487 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c0c102460 0x7f5c0c198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:48.202 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.200+0000 7f5c11ee7640 1 -- 172.21.15.1:0/1539218487 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c0c102460 msgr2=0x7f5c0c198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:48.202 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.200+0000 7f5c11ee7640 1 --2- 172.21.15.1:0/1539218487 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c0c102460 0x7f5c0c198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:48.202 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.200+0000 7f5c11ee7640 1 -- 172.21.15.1:0/1539218487 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c0c107ad0 msgr2=0x7f5c0c19cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:48.202 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.200+0000 7f5c11ee7640 1 --2- 172.21.15.1:0/1539218487 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c0c107ad0 0x7f5c0c19cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:48.202 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.200+0000 7f5c11ee7640 1 -- 172.21.15.1:0/1539218487 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c0c19d3c0 con 0x7f5c0c101ac0 2024-09-11T21:58:48.202 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.201+0000 7f5c11ee7640 1 --2- 172.21.15.1:0/1539218487 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c0c101ac0 0x7f5c0c1983d0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f5c08005e30 tx=0x7f5c08003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:48.208 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.208+0000 7f5c02ffd640 1 -- 172.21.15.1:0/1539218487 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c08031650 con 0x7f5c0c101ac0 2024-09-11T21:58:48.209 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.208+0000 7f5c1414b640 1 -- 172.21.15.1:0/1539218487 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c0c19d650 con 0x7f5c0c101ac0 2024-09-11T21:58:48.209 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.208+0000 7f5c02ffd640 1 -- 172.21.15.1:0/1539218487 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c080317f0 con 0x7f5c0c101ac0 2024-09-11T21:58:48.209 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.208+0000 7f5c02ffd640 1 -- 172.21.15.1:0/1539218487 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c0802faa0 con 0x7f5c0c101ac0 2024-09-11T21:58:48.209 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.209+0000 7f5c1414b640 1 -- 172.21.15.1:0/1539218487 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c0c1a4f40 con 0x7f5c0c101ac0 2024-09-11T21:58:48.211 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.210+0000 7f5c02ffd640 1 -- 172.21.15.1:0/1539218487 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f5c0802fc40 con 0x7f5c0c101ac0 2024-09-11T21:58:48.211 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.211+0000 7f5c02ffd640 1 --2- 172.21.15.1:0/1539218487 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f5bf80a2e40 0x7f5bf80a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:48.212 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.211+0000 7f5c116e6640 1 --2- 172.21.15.1:0/1539218487 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f5bf80a2e40 0x7f5bf80a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:48.212 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.211+0000 7f5c02ffd640 1 -- 172.21.15.1:0/1539218487 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f5c080f0730 con 0x7f5c0c101ac0 2024-09-11T21:58:48.212 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.212+0000 7f5c116e6640 1 --2- 172.21.15.1:0/1539218487 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f5bf80a2e40 0x7f5bf80a5300 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc0095f0 tx=0x7f5bfc005f90 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:48.626 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.626+0000 7f5c16fc3740 1 Processor -- start 2024-09-11T21:58:48.628 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.626+0000 7f5c16fc3740 1 -- start start 2024-09-11T21:58:48.628 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.626+0000 7f5c16fc3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a3e0b73af0 0x55a3e0b73ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:48.628 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.626+0000 7f5c16fc3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a3e0b74410 0x55a3e0b7e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:48.628 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.626+0000 7f5c16fc3740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3e0b7ecb0 0x55a3e0b810a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:48.628 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.626+0000 7f5c16fc3740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55a3e0ad82f0 con 0x55a3e0b7ecb0 2024-09-11T21:58:48.628 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.626+0000 7f5c16fc3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55a3e0a80270 con 0x55a3e0b73af0 2024-09-11T21:58:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.626+0000 7f5c16fc3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a3e0ad42b0 con 0x55a3e0b74410 2024-09-11T21:58:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.627+0000 7f5c126e8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3e0b7ecb0 0x55a3e0b810a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.627+0000 7f5c126e8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3e0b7ecb0 0x55a3e0b810a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47634/0 (socket says 172.21.15.1:47634) 2024-09-11T21:58:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.627+0000 7f5c126e8640 1 -- 172.21.15.1:0/3285346937 learned_addr learned my addr 172.21.15.1:0/3285346937 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:58:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.627+0000 7f5c11ee7640 1 --2- 172.21.15.1:0/3285346937 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a3e0b73af0 0x55a3e0b73ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.627+0000 7f5c116e6640 1 --2- 172.21.15.1:0/3285346937 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a3e0b74410 0x55a3e0b7e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.627+0000 7f5c126e8640 1 -- 172.21.15.1:0/3285346937 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a3e0b74410 msgr2=0x55a3e0b7e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.627+0000 7f5c126e8640 1 --2- 172.21.15.1:0/3285346937 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a3e0b74410 0x55a3e0b7e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.627+0000 7f5c126e8640 1 -- 172.21.15.1:0/3285346937 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a3e0b73af0 msgr2=0x55a3e0b73ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.627+0000 7f5c126e8640 1 --2- 172.21.15.1:0/3285346937 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a3e0b73af0 0x55a3e0b73ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.627+0000 7f5c126e8640 1 -- 172.21.15.1:0/3285346937 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3e0a7f340 con 0x55a3e0b7ecb0 2024-09-11T21:58:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.627+0000 7f5c11ee7640 1 --2- 172.21.15.1:0/3285346937 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a3e0b73af0 0x55a3e0b73ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T21:58:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.627+0000 7f5c126e8640 1 --2- 172.21.15.1:0/3285346937 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3e0b7ecb0 0x55a3e0b810a0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f5c04009c20 tx=0x7f5c04011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83a6eecdfa76c422 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:48.630 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.628+0000 7f5bf7fff640 1 -- 172.21.15.1:0/3285346937 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c0401e070 con 0x55a3e0b7ecb0 2024-09-11T21:58:48.630 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.628+0000 7f5bf7fff640 1 -- 172.21.15.1:0/3285346937 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c04019b10 con 0x55a3e0b7ecb0 2024-09-11T21:58:48.630 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.628+0000 7f5bf7fff640 1 -- 172.21.15.1:0/3285346937 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c0400e9e0 con 0x55a3e0b7ecb0 2024-09-11T21:58:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.629+0000 7f5c16fc3740 1 -- 172.21.15.1:0/3285346937 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3e0b7ecb0 msgr2=0x55a3e0b810a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.629+0000 7f5c16fc3740 1 --2- 172.21.15.1:0/3285346937 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3e0b7ecb0 0x55a3e0b810a0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f5c04009c20 tx=0x7f5c04011810 comp rx=0 tx=0).stop 2024-09-11T21:58:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.629+0000 7f5c16fc3740 1 -- 172.21.15.1:0/3285346937 shutdown_connections 2024-09-11T21:58:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.629+0000 7f5c16fc3740 1 --2- 172.21.15.1:0/3285346937 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3e0b7ecb0 0x55a3e0b810a0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.629+0000 7f5c16fc3740 1 --2- 172.21.15.1:0/3285346937 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a3e0b74410 0x55a3e0b7e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.629+0000 7f5c16fc3740 1 --2- 172.21.15.1:0/3285346937 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a3e0b73af0 0x55a3e0b73ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.629+0000 7f5c16fc3740 1 -- 172.21.15.1:0/3285346937 >> 172.21.15.1:0/3285346937 conn(0x55a3e0b70060 msgr2=0x55a3e0b70440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.629+0000 7f5c16fc3740 1 -- 172.21.15.1:0/3285346937 shutdown_connections 2024-09-11T21:58:48.632 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.629+0000 7f5c16fc3740 1 -- 172.21.15.1:0/3285346937 wait complete. 2024-09-11T21:58:48.719 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.719+0000 7f5c16fc3740 1 -- start start 2024-09-11T21:58:48.731 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.731+0000 7f5c16fc3740 1 Processor -- start 2024-09-11T21:58:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.732+0000 7f5c16fc3740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3e0b73af0 0x55a3e0b7bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.732+0000 7f5c16fc3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a3e0b74410 0x55a3e0b7c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.732+0000 7f5c16fc3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a3e0b7ecb0 0x55a3e0b7c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.732+0000 7f5c16fc3740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55a3e0b81880 con 0x55a3e0b73af0 2024-09-11T21:58:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.732+0000 7f5c16fc3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55a3e0a61290 con 0x55a3e0b7ecb0 2024-09-11T21:58:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.732+0000 7f5c16fc3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a3e0b81a00 con 0x55a3e0b74410 2024-09-11T21:58:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.732+0000 7f5c11ee7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3e0b73af0 0x55a3e0b7bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.732+0000 7f5c11ee7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3e0b73af0 0x55a3e0b7bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47642/0 (socket says 172.21.15.1:47642) 2024-09-11T21:58:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.732+0000 7f5c11ee7640 1 -- 172.21.15.1:0/1261720880 learned_addr learned my addr 172.21.15.1:0/1261720880 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:58:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.732+0000 7f5c116e6640 1 --2- 172.21.15.1:0/1261720880 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a3e0b74410 0x55a3e0b7c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.732+0000 7f5c126e8640 1 --2- 172.21.15.1:0/1261720880 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a3e0b7ecb0 0x55a3e0b7c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.732+0000 7f5c116e6640 1 -- 172.21.15.1:0/1261720880 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a3e0b7ecb0 msgr2=0x55a3e0b7c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.732+0000 7f5c116e6640 1 --2- 172.21.15.1:0/1261720880 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a3e0b7ecb0 0x55a3e0b7c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.732+0000 7f5c116e6640 1 -- 172.21.15.1:0/1261720880 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3e0b73af0 msgr2=0x55a3e0b7bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.732+0000 7f5c116e6640 1 --2- 172.21.15.1:0/1261720880 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3e0b73af0 0x55a3e0b7bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.732+0000 7f5c116e6640 1 -- 172.21.15.1:0/1261720880 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3e07eb210 con 0x55a3e0b74410 2024-09-11T21:58:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.732+0000 7f5c11ee7640 1 --2- 172.21.15.1:0/1261720880 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3e0b73af0 0x55a3e0b7bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T21:58:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.732+0000 7f5c126e8640 1 --2- 172.21.15.1:0/1261720880 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a3e0b7ecb0 0x55a3e0b7c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:58:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.733+0000 7f5c116e6640 1 --2- 172.21.15.1:0/1261720880 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a3e0b74410 0x55a3e0b7c0f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c0ffd20 tx=0x7f5bfc004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbf90d1dc7b6f2ad server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.733+0000 7f5bf57fa640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-11T21:58:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.733+0000 7f5bf57fa640 1 -- 172.21.15.1:0/1261720880 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bfc02b060 con 0x55a3e0b74410 2024-09-11T21:58:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.733+0000 7f5bf57fa640 1 -- 172.21.15.1:0/1261720880 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f5bfc01daa0 con 0x55a3e0b74410 2024-09-11T21:58:48.789 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.789+0000 7f5c16fc3740 10 client.0 _ll_register_callbacks cb 0x55a3e0a92140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:58:48.790 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.790+0000 7f5c16fc3740 1 -- 172.21.15.1:0/1261720880 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c04019b10 con 0x55a3e0b74410 2024-09-11T21:58:48.791 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.791+0000 7f5bf57fa640 1 -- 172.21.15.1:0/1261720880 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5bfc01dca0 con 0x55a3e0b74410 2024-09-11T21:58:48.791 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.791+0000 7f5c16fc3740 10 client.25387 fetch_fsmap learned FSMap version 28 2024-09-11T21:58:48.791 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.791+0000 7f5c16fc3740 1 -- 172.21.15.1:0/1261720880 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({fsmap=28}) -- 0x55a3e08b7620 con 0x55a3e0b74410 2024-09-11T21:58:48.852 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.852+0000 7f5bf57fa640 1 -- 172.21.15.1:0/1261720880 <== mon.1 v2:172.21.15.161:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5bfc026410 con 0x55a3e0b74410 2024-09-11T21:58:48.852 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.852+0000 7f5c16fc3740 10 client.25387 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T21:58:48.867 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:58:48.867 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:58:48.908 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.908+0000 7f5c16fc3740 10 client.25387 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T21:58:48.909 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.908+0000 7f5c16fc3740 20 client.25387 populate_metadata read hostname 'smithi001' 2024-09-11T21:58:48.909 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.908+0000 7f5c16fc3740 1 --2- 172.21.15.1:0/1261720880 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55a3e0b8eac0 0x55a3e0b90f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:48.909 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.908+0000 7f5c16fc3740 4 client.25387 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T21:58:48.909 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.908+0000 7f5c16fc3740 1 -- 172.21.15.1:0/1261720880 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a3e0ae5a20 con 0x55a3e0b8eac0 2024-09-11T21:58:48.910 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.909+0000 7f5c11ee7640 1 --2- 172.21.15.1:0/1261720880 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55a3e0b8eac0 0x55a3e0b90f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:48.910 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.909+0000 7f5c11ee7640 1 --2- 172.21.15.1:0/1261720880 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55a3e0b8eac0 0x55a3e0b90f80 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4dc3a1ae964a5e4e server_cookie=a4e9af5b404b7b0 in_seq=0 out_seq=0 2024-09-11T21:58:48.910 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.909+0000 7f5bf57fa640 10 client.25387 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T21:58:48.911 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.910+0000 7f5bf57fa640 1 -- 172.21.15.1:0/1261720880 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a3e0ae5a20 con 0x55a3e0b8eac0 2024-09-11T21:58:48.911 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.910+0000 7f5bf57fa640 10 client.25387 handle_command_reply: tid=0 2024-09-11T21:58:48.911 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.910+0000 7f5c16fc3740 1 client.25387 shutdown 2024-09-11T21:58:48.912 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.911+0000 7f5c16fc3740 1 -- 172.21.15.1:0/1261720880 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a3e0b74410 msgr2=0x55a3e0b7c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:48.912 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.911+0000 7f5c16fc3740 1 --2- 172.21.15.1:0/1261720880 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a3e0b74410 0x55a3e0b7c0f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c0ffd20 tx=0x7f5bfc004730 comp rx=0 tx=0).stop 2024-09-11T21:58:48.912 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.911+0000 7f5c16fc3740 1 -- 172.21.15.1:0/1261720880 shutdown_connections 2024-09-11T21:58:48.912 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.911+0000 7f5c16fc3740 1 --2- 172.21.15.1:0/1261720880 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55a3e0b8eac0 0x55a3e0b90f80 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:48.912 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.911+0000 7f5c16fc3740 1 --2- 172.21.15.1:0/1261720880 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a3e0b7ecb0 0x55a3e0b7c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:48.912 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.911+0000 7f5c16fc3740 1 --2- 172.21.15.1:0/1261720880 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a3e0b74410 0x55a3e0b7c0f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:48.913 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.911+0000 7f5c16fc3740 1 --2- 172.21.15.1:0/1261720880 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3e0b73af0 0x55a3e0b7bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:48.913 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.911+0000 7f5bf57fa640 0 client.25387 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T21:58:48.913 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.911+0000 7f5c16fc3740 1 -- 172.21.15.1:0/1261720880 >> 172.21.15.1:0/1261720880 conn(0x55a3e0b70060 msgr2=0x55a3e0b75d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:48.913 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.911+0000 7f5c16fc3740 1 -- 172.21.15.1:0/1261720880 shutdown_connections 2024-09-11T21:58:48.913 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.911+0000 7f5c16fc3740 1 -- 172.21.15.1:0/1261720880 wait complete. 2024-09-11T21:58:48.913 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.911+0000 7f5c16fc3740 20 client.25387 trim_cache size 0 max 16384 2024-09-11T21:58:48.940 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.938+0000 7f5c16fc3740 1 Processor -- start 2024-09-11T21:58:48.940 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.938+0000 7f5c16fc3740 1 -- start start 2024-09-11T21:58:48.940 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.938+0000 7f5c16fc3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a3e0b73af0 0x55a3e0b7c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:48.941 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.938+0000 7f5c16fc3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a3e0b74410 0x55a3e0b96850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:48.941 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.938+0000 7f5c16fc3740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3e0b7ecb0 0x55a3e0b96d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:48.941 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.938+0000 7f5c16fc3740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5c0c1ad8b0 con 0x55a3e0b7ecb0 2024-09-11T21:58:48.941 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.938+0000 7f5c16fc3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5c0c1af3f0 con 0x55a3e0b74410 2024-09-11T21:58:48.941 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.938+0000 7f5c16fc3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5c0c1adf80 con 0x55a3e0b73af0 2024-09-11T21:58:48.941 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.939+0000 7f5c126e8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3e0b7ecb0 0x55a3e0b96d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:48.941 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.939+0000 7f5c11ee7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a3e0b73af0 0x55a3e0b7c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:48.941 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.939+0000 7f5c126e8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3e0b7ecb0 0x55a3e0b96d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47652/0 (socket says 172.21.15.1:47652) 2024-09-11T21:58:48.941 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.939+0000 7f5c126e8640 1 -- 172.21.15.1:0/1018178577 learned_addr learned my addr 172.21.15.1:0/1018178577 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:58:48.941 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.939+0000 7f5c11ee7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a3e0b73af0 0x55a3e0b7c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:54392/0 (socket says 172.21.15.1:54392) 2024-09-11T21:58:48.941 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.939+0000 7f5c116e6640 1 --2- 172.21.15.1:0/1018178577 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a3e0b74410 0x55a3e0b96850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:48.941 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.939+0000 7f5c11ee7640 1 -- 172.21.15.1:0/1018178577 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a3e0b74410 msgr2=0x55a3e0b96850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:48.942 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.939+0000 7f5c11ee7640 1 --2- 172.21.15.1:0/1018178577 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a3e0b74410 0x55a3e0b96850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:48.942 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.939+0000 7f5c11ee7640 1 -- 172.21.15.1:0/1018178577 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3e0b7ecb0 msgr2=0x55a3e0b96d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:48.942 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.939+0000 7f5c11ee7640 1 --2- 172.21.15.1:0/1018178577 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3e0b7ecb0 0x55a3e0b96d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:48.942 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.939+0000 7f5c11ee7640 1 -- 172.21.15.1:0/1018178577 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3e0b6a2f0 con 0x55a3e0b73af0 2024-09-11T21:58:48.942 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.939+0000 7f5c126e8640 1 --2- 172.21.15.1:0/1018178577 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3e0b7ecb0 0x55a3e0b96d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T21:58:48.942 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.939+0000 7f5c11ee7640 1 --2- 172.21.15.1:0/1018178577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a3e0b73af0 0x55a3e0b7c460 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5c080ee440 tx=0x7f5c0802f500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41b09456d8f46e7b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:48.942 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.940+0000 7f5bf77fe640 1 -- 172.21.15.1:0/1018178577 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c080bd420 con 0x55a3e0b73af0 2024-09-11T21:58:48.942 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.940+0000 7f5bf77fe640 1 -- 172.21.15.1:0/1018178577 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c080d9070 con 0x55a3e0b73af0 2024-09-11T21:58:48.942 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.940+0000 7f5bf77fe640 1 -- 172.21.15.1:0/1018178577 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c080b9390 con 0x55a3e0b73af0 2024-09-11T21:58:48.946 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.940+0000 7f5c16fc3740 1 -- 172.21.15.1:0/1018178577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a3e0b73af0 msgr2=0x55a3e0b7c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:48.946 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.940+0000 7f5c16fc3740 1 --2- 172.21.15.1:0/1018178577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a3e0b73af0 0x55a3e0b7c460 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5c080ee440 tx=0x7f5c0802f500 comp rx=0 tx=0).stop 2024-09-11T21:58:48.946 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.940+0000 7f5c16fc3740 1 -- 172.21.15.1:0/1018178577 shutdown_connections 2024-09-11T21:58:48.946 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.940+0000 7f5c16fc3740 1 --2- 172.21.15.1:0/1018178577 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3e0b7ecb0 0x55a3e0b96d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:48.947 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.940+0000 7f5c16fc3740 1 --2- 172.21.15.1:0/1018178577 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a3e0b74410 0x55a3e0b96850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:48.947 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.940+0000 7f5c16fc3740 1 --2- 172.21.15.1:0/1018178577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a3e0b73af0 0x55a3e0b7c460 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:48.947 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.941+0000 7f5c16fc3740 1 -- 172.21.15.1:0/1018178577 >> 172.21.15.1:0/1018178577 conn(0x55a3e0b70060 msgr2=0x55a3e0b7dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:48.947 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.941+0000 7f5c16fc3740 1 -- 172.21.15.1:0/1018178577 shutdown_connections 2024-09-11T21:58:48.947 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.941+0000 7f5c16fc3740 1 -- 172.21.15.1:0/1018178577 wait complete. 2024-09-11T21:58:48.947 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.941+0000 7f5c16fc3740 1 -- start start 2024-09-11T21:58:48.947 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.941+0000 7f5c16fc3740 1 Processor -- start 2024-09-11T21:58:48.947 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.942+0000 7f5c16fc3740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3e0b73af0 0x55a3e0b87150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:48.947 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.942+0000 7f5c16fc3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a3e0b74410 0x55a3e0b78600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:48.947 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.942+0000 7f5c16fc3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a3e0b7ecb0 0x55a3e0b78b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:48.947 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.942+0000 7f5c16fc3740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5c0c1aed20 con 0x55a3e0b73af0 2024-09-11T21:58:48.947 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.942+0000 7f5c16fc3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5c0c1ae650 con 0x55a3e0b74410 2024-09-11T21:58:48.948 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.942+0000 7f5c16fc3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5c0c1ab890 con 0x55a3e0b7ecb0 2024-09-11T21:58:48.948 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.942+0000 7f5c16fc3740 10 client.0 _ll_register_callbacks cb 0x7f5bfc00e4b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:58:48.948 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.942+0000 7f5c11ee7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3e0b73af0 0x55a3e0b87150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:48.948 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.942+0000 7f5c11ee7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3e0b73af0 0x55a3e0b87150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47658/0 (socket says 172.21.15.1:47658) 2024-09-11T21:58:48.948 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.942+0000 7f5c11ee7640 1 -- 172.21.15.1:0/413462580 learned_addr learned my addr 172.21.15.1:0/413462580 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:58:48.948 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.942+0000 7f5c126e8640 1 --2- 172.21.15.1:0/413462580 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a3e0b7ecb0 0x55a3e0b78b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:48.948 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.942+0000 7f5c116e6640 1 --2- 172.21.15.1:0/413462580 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a3e0b74410 0x55a3e0b78600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:48.948 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.942+0000 7f5c11ee7640 1 -- 172.21.15.1:0/413462580 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a3e0b7ecb0 msgr2=0x55a3e0b78b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:48.948 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.942+0000 7f5c11ee7640 1 --2- 172.21.15.1:0/413462580 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a3e0b7ecb0 0x55a3e0b78b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:48.948 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.942+0000 7f5c11ee7640 1 -- 172.21.15.1:0/413462580 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a3e0b74410 msgr2=0x55a3e0b78600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:48.948 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.942+0000 7f5c11ee7640 1 --2- 172.21.15.1:0/413462580 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a3e0b74410 0x55a3e0b78600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:48.948 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.942+0000 7f5c11ee7640 1 -- 172.21.15.1:0/413462580 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3e0b79080 con 0x55a3e0b73af0 2024-09-11T21:58:48.948 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.942+0000 7f5c116e6640 1 --2- 172.21.15.1:0/413462580 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a3e0b74410 0x55a3e0b78600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T21:58:48.949 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.943+0000 7f5c11ee7640 1 --2- 172.21.15.1:0/413462580 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3e0b73af0 0x55a3e0b87150 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f5c080b9180 tx=0x7f5c080bce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6bc67209b3e291d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:48.949 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.943+0000 7f5bf57fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T21:58:48.949 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.943+0000 7f5bf57fa640 1 -- 172.21.15.1:0/413462580 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c080bd420 con 0x55a3e0b73af0 2024-09-11T21:58:48.949 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.943+0000 7f5bf57fa640 1 -- 172.21.15.1:0/413462580 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f5c080b9390 con 0x55a3e0b73af0 2024-09-11T21:58:48.949 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.943+0000 7f5c16fc3740 1 -- 172.21.15.1:0/413462580 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c080d9070 con 0x55a3e0b73af0 2024-09-11T21:58:48.949 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.943+0000 7f5bf57fa640 1 -- 172.21.15.1:0/413462580 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5c080c5990 con 0x55a3e0b73af0 2024-09-11T21:58:48.949 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.943+0000 7f5c16fc3740 10 client.15831 fetch_fsmap learned FSMap version 28 2024-09-11T21:58:48.949 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.943+0000 7f5c16fc3740 1 -- 172.21.15.1:0/413462580 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a3e0bf9580 con 0x55a3e0b73af0 2024-09-11T21:58:48.949 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.943+0000 7f5bf57fa640 1 -- 172.21.15.1:0/413462580 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c080d9070 con 0x55a3e0b73af0 2024-09-11T21:58:48.949 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.944+0000 7f5bf57fa640 1 -- 172.21.15.1:0/413462580 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5c080c5500 con 0x55a3e0b73af0 2024-09-11T21:58:48.949 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.944+0000 7f5c16fc3740 10 client.15831 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T21:58:48.949 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.944+0000 7f5c16fc3740 10 client.15831 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T21:58:48.950 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.944+0000 7f5c16fc3740 20 client.15831 populate_metadata read hostname 'smithi001' 2024-09-11T21:58:48.950 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.944+0000 7f5c16fc3740 1 --2- 172.21.15.1:0/413462580 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55a3e0bf9ca0 0x55a3e0bfc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:48.950 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.944+0000 7f5c16fc3740 4 client.15831 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T21:58:48.950 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.944+0000 7f5c16fc3740 1 -- 172.21.15.1:0/413462580 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a3e0adfd70 con 0x55a3e0bf9ca0 2024-09-11T21:58:48.950 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.944+0000 7f5c116e6640 1 --2- 172.21.15.1:0/413462580 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55a3e0bf9ca0 0x55a3e0bfc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:48.950 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.944+0000 7f5bf57fa640 10 client.15831 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T21:58:48.950 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.944+0000 7f5c116e6640 1 --2- 172.21.15.1:0/413462580 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55a3e0bf9ca0 0x55a3e0bfc0c0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce8e22e9f0f73a01 server_cookie=da6261047818c94c in_seq=0 out_seq=0 2024-09-11T21:58:48.964 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.963+0000 7f5bf57fa640 1 -- 172.21.15.1:0/413462580 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a3e0adfd70 con 0x55a3e0bf9ca0 2024-09-11T21:58:48.964 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.963+0000 7f5bf57fa640 10 client.15831 handle_command_reply: tid=0 2024-09-11T21:58:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.963+0000 7f5c16fc3740 1 client.15831 shutdown 2024-09-11T21:58:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.963+0000 7f5c16fc3740 1 -- 172.21.15.1:0/413462580 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3e0b73af0 msgr2=0x55a3e0b87150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.963+0000 7f5c16fc3740 1 --2- 172.21.15.1:0/413462580 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3e0b73af0 0x55a3e0b87150 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f5c080b9180 tx=0x7f5c080bce60 comp rx=0 tx=0).stop 2024-09-11T21:58:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.964+0000 7f5c16fc3740 1 -- 172.21.15.1:0/413462580 shutdown_connections 2024-09-11T21:58:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.964+0000 7f5c16fc3740 1 --2- 172.21.15.1:0/413462580 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55a3e0bf9ca0 0x55a3e0bfc0c0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.964+0000 7f5c16fc3740 1 --2- 172.21.15.1:0/413462580 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a3e0b7ecb0 0x55a3e0b78b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.964+0000 7f5c16fc3740 1 --2- 172.21.15.1:0/413462580 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a3e0b74410 0x55a3e0b78600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.964+0000 7f5bf57fa640 0 client.15831 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T21:58:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.964+0000 7f5c16fc3740 1 --2- 172.21.15.1:0/413462580 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3e0b73af0 0x55a3e0b87150 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.964+0000 7f5c16fc3740 1 -- 172.21.15.1:0/413462580 >> 172.21.15.1:0/413462580 conn(0x55a3e0b70060 msgr2=0x55a3e0b75550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.964+0000 7f5c16fc3740 1 -- 172.21.15.1:0/413462580 shutdown_connections 2024-09-11T21:58:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.964+0000 7f5c16fc3740 1 -- 172.21.15.1:0/413462580 wait complete. 2024-09-11T21:58:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:48.964+0000 7f5c16fc3740 20 client.15831 trim_cache size 0 max 16384 2024-09-11T21:58:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-09-11T21:58:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-09-11T21:58:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-09-11T21:58:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-09-11T21:58:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:49.006+0000 7f5c1414b640 1 -- 172.21.15.1:0/1539218487 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f5bf80a2e40 msgr2=0x7f5bf80a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:49.006+0000 7f5c1414b640 1 --2- 172.21.15.1:0/1539218487 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f5bf80a2e40 0x7f5bf80a5300 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc0095f0 tx=0x7f5bfc005f90 comp rx=0 tx=0).stop 2024-09-11T21:58:49.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:49.006+0000 7f5c1414b640 1 -- 172.21.15.1:0/1539218487 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c0c101ac0 msgr2=0x7f5c0c1983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:49.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:49.006+0000 7f5c1414b640 1 --2- 172.21.15.1:0/1539218487 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c0c101ac0 0x7f5c0c1983d0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f5c08005e30 tx=0x7f5c08003f90 comp rx=0 tx=0).stop 2024-09-11T21:58:49.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:49.006+0000 7f5c1414b640 1 -- 172.21.15.1:0/1539218487 shutdown_connections 2024-09-11T21:58:49.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:49.006+0000 7f5c1414b640 1 --2- 172.21.15.1:0/1539218487 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f5bf80a2e40 0x7f5bf80a5300 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:49.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:49.007+0000 7f5c1414b640 1 --2- 172.21.15.1:0/1539218487 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c0c107ad0 0x7f5c0c19cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:49.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:49.007+0000 7f5c1414b640 1 --2- 172.21.15.1:0/1539218487 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c0c102460 0x7f5c0c198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:49.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:49.007+0000 7f5c1414b640 1 --2- 172.21.15.1:0/1539218487 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c0c101ac0 0x7f5c0c1983d0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:49.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:49.007+0000 7f5c1414b640 1 -- 172.21.15.1:0/1539218487 >> 172.21.15.1:0/1539218487 conn(0x7f5c0c0fd7a0 msgr2=0x7f5c0c0fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:49.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:49.007+0000 7f5c1414b640 1 -- 172.21.15.1:0/1539218487 shutdown_connections 2024-09-11T21:58:49.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:58:49.007+0000 7f5c1414b640 1 -- 172.21.15.1:0/1539218487 wait complete. 2024-09-11T21:58:49.307 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T21:58:49.307 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T21:58:49.323 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:58:49.323 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:58:49.323 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 13 -lt 100 ']' 2024-09-11T21:58:49.324 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 13 + 1 2024-09-11T21:58:49.389 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=14 2024-09-11T21:58:49.389 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T21:58:49.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:58:49 smithi001 ceph-mon[30935]: pgmap v52: 97 pgs: 97 active+clean; 1.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-11T21:58:49.459 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:58:49.459 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:58:49.460 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T21:58:49.460 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T21:58:49.461 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T21:58:49.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:58:49 smithi053 ceph-mon[34908]: pgmap v52: 97 pgs: 97 active+clean; 1.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-11T21:58:49.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:58:49 smithi161 ceph-mon[35615]: pgmap v52: 97 pgs: 97 active+clean; 1.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-11T21:58:51.183 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:58:51.183 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:58:51.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:58:51 smithi001 ceph-mon[30935]: pgmap v53: 97 pgs: 97 active+clean; 1.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-09-11T21:58:51.498 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T21:58:51.498 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T21:58:51.523 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:58:51.523 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:58:51.523 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 14 -lt 100 ']' 2024-09-11T21:58:51.523 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 14 + 1 2024-09-11T21:58:51.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:58:51 smithi053 ceph-mon[34908]: pgmap v53: 97 pgs: 97 active+clean; 1.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-09-11T21:58:51.557 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=15 2024-09-11T21:58:51.557 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T21:58:51.583 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:58:51.584 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:58:51.585 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T21:58:51.585 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T21:58:51.585 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T21:58:51.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:58:51 smithi161 ceph-mon[35615]: pgmap v53: 97 pgs: 97 active+clean; 1.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-09-11T21:58:52.487 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:58:52.558 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:58:52.558 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:58:52.781 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T21:58:52.781 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T21:58:52.798 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:58:52.799 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:58:52.799 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 15 -lt 100 ']' 2024-09-11T21:58:52.799 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 15 + 1 2024-09-11T21:58:52.828 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=16 2024-09-11T21:58:52.828 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T21:58:52.843 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:58:52.843 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:58:52.844 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T21:58:52.845 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T21:58:52.845 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T21:58:53.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:58:53 smithi001 ceph-mon[30935]: pgmap v54: 97 pgs: 97 active+clean; 1.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-11T21:58:53.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:58:53 smithi053 ceph-mon[34908]: pgmap v54: 97 pgs: 97 active+clean; 1.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-11T21:58:53.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:58:53 smithi161 ceph-mon[35615]: pgmap v54: 97 pgs: 97 active+clean; 1.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-11T21:58:53.966 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:58:53.966 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:58:54.185 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T21:58:54.185 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T21:58:54.433 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:58:54.433 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:58:54.433 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 16 -lt 100 ']' 2024-09-11T21:58:54.433 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 16 + 1 2024-09-11T21:58:54.638 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=17 2024-09-11T21:58:54.638 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T21:58:54.698 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:58:54.699 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:58:54.699 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T21:58:54.699 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T21:58:54.699 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T21:58:55.366 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:58:55 smithi001 ceph-mon[30935]: pgmap v55: 97 pgs: 97 active+clean; 1.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-11T21:58:55.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:58:55 smithi053 ceph-mon[34908]: pgmap v55: 97 pgs: 97 active+clean; 1.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-11T21:58:55.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:58:55 smithi161 ceph-mon[35615]: pgmap v55: 97 pgs: 97 active+clean; 1.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-09-11T21:58:55.724 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:58:55.724 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:58:56.156 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T21:58:56.156 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T21:58:56.180 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:58:56.180 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:58:56.180 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 17 -lt 100 ']' 2024-09-11T21:58:56.180 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 17 + 1 2024-09-11T21:58:56.231 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=18 2024-09-11T21:58:56.231 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T21:58:56.247 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:58:56.247 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:58:56.248 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T21:58:56.249 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T21:58:56.249 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T21:58:56.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:58:56 smithi001 ceph-mon[30935]: pgmap v56: 97 pgs: 97 active+clean; 1.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 22 op/s 2024-09-11T21:58:56.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:58:56 smithi053 ceph-mon[34908]: pgmap v56: 97 pgs: 97 active+clean; 1.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 22 op/s 2024-09-11T21:58:56.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:58:56 smithi161 ceph-mon[35615]: pgmap v56: 97 pgs: 97 active+clean; 1.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 22 op/s 2024-09-11T21:58:57.099 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:58:57.099 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:58:57.383 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T21:58:57.383 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T21:58:57.397 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:58:57.397 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:58:57.398 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 18 -lt 100 ']' 2024-09-11T21:58:57.398 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 18 + 1 2024-09-11T21:58:57.461 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=19 2024-09-11T21:58:57.461 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T21:58:57.481 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:58:57.481 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:58:57.482 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T21:58:57.482 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T21:58:57.482 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T21:58:58.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:58:57 smithi053 ceph-mon[34908]: pgmap v57: 97 pgs: 97 active+clean; 1.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-09-11T21:58:58.091 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:58:57 smithi161 ceph-mon[35615]: pgmap v57: 97 pgs: 97 active+clean; 1.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-09-11T21:58:58.120 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:58:57 smithi001 ceph-mon[30935]: pgmap v57: 97 pgs: 97 active+clean; 1.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-09-11T21:58:58.549 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:58:58.549 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:58:58.931 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T21:58:58.931 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T21:58:58.948 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:58:58.948 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:58:58.948 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 19 -lt 100 ']' 2024-09-11T21:58:58.948 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 19 + 1 2024-09-11T21:58:59.095 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=20 2024-09-11T21:58:59.096 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T21:58:59.157 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:58:59.157 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:58:59.158 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T21:58:59.159 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T21:58:59.159 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T21:58:59.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.305+0000 7fba2dd73640 1 -- 172.21.15.1:0/3111665728 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba28102d40 msgr2=0x7fba28103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:59.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.305+0000 7fba2dd73640 1 --2- 172.21.15.1:0/3111665728 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba28102d40 0x7fba28103120 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fba18009900 tx=0x7fba1802a480 comp rx=0 tx=0).stop 2024-09-11T21:58:59.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.305+0000 7fba2dd73640 1 -- 172.21.15.1:0/3111665728 shutdown_connections 2024-09-11T21:58:59.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.305+0000 7fba2dd73640 1 --2- 172.21.15.1:0/3111665728 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba2810c170 0x7fba2810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:59.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.305+0000 7fba2dd73640 1 --2- 172.21.15.1:0/3111665728 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba28103660 0x7fba2810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:59.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.305+0000 7fba2dd73640 1 --2- 172.21.15.1:0/3111665728 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba28102d40 0x7fba28103120 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:59.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.305+0000 7fba2dd73640 1 -- 172.21.15.1:0/3111665728 >> 172.21.15.1:0/3111665728 conn(0x7fba280fc7a0 msgr2=0x7fba280febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:59.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.305+0000 7fba2dd73640 1 -- 172.21.15.1:0/3111665728 shutdown_connections 2024-09-11T21:58:59.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.305+0000 7fba2dd73640 1 -- 172.21.15.1:0/3111665728 wait complete. 2024-09-11T21:58:59.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.306+0000 7fba2dd73640 1 Processor -- start 2024-09-11T21:58:59.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.307+0000 7fba2dd73640 1 -- start start 2024-09-11T21:58:59.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.307+0000 7fba2dd73640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba28102d40 0x7fba2819a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:59.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.307+0000 7fba2dd73640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba28103660 0x7fba2819ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:59.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.307+0000 7fba2dd73640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba2810c170 0x7fba2819eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:59.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.307+0000 7fba2dd73640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fba28111f70 con 0x7fba28103660 2024-09-11T21:58:59.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.307+0000 7fba2dd73640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fba28111df0 con 0x7fba2810c170 2024-09-11T21:58:59.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.307+0000 7fba2dd73640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fba281120f0 con 0x7fba28102d40 2024-09-11T21:58:59.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.307+0000 7fba26ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba28103660 0x7fba2819ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:59.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.307+0000 7fba26ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba28103660 0x7fba2819ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53834/0 (socket says 172.21.15.1:53834) 2024-09-11T21:58:59.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.307+0000 7fba26ffd640 1 -- 172.21.15.1:0/3033598554 learned_addr learned my addr 172.21.15.1:0/3033598554 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:58:59.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.307+0000 7fba277fe640 1 --2- 172.21.15.1:0/3033598554 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba28102d40 0x7fba2819a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:59.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.307+0000 7fba27fff640 1 --2- 172.21.15.1:0/3033598554 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba2810c170 0x7fba2819eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:59.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.307+0000 7fba26ffd640 1 -- 172.21.15.1:0/3033598554 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba28102d40 msgr2=0x7fba2819a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:59.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.308+0000 7fba26ffd640 1 --2- 172.21.15.1:0/3033598554 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba28102d40 0x7fba2819a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:59.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.308+0000 7fba26ffd640 1 -- 172.21.15.1:0/3033598554 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba2810c170 msgr2=0x7fba2819eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:59.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.308+0000 7fba26ffd640 1 --2- 172.21.15.1:0/3033598554 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba2810c170 0x7fba2819eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:59.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.308+0000 7fba26ffd640 1 -- 172.21.15.1:0/3033598554 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba2819f5c0 con 0x7fba28103660 2024-09-11T21:58:59.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.308+0000 7fba27fff640 1 --2- 172.21.15.1:0/3033598554 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba2810c170 0x7fba2819eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:58:59.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.308+0000 7fba26ffd640 1 --2- 172.21.15.1:0/3033598554 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba28103660 0x7fba2819ab50 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fba1c00e9f0 tx=0x7fba1c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:59.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.308+0000 7fba24ff9640 1 -- 172.21.15.1:0/3033598554 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1c00ccc0 con 0x7fba28103660 2024-09-11T21:58:59.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.308+0000 7fba24ff9640 1 -- 172.21.15.1:0/3033598554 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba1c010910 con 0x7fba28103660 2024-09-11T21:58:59.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.308+0000 7fba2dd73640 1 -- 172.21.15.1:0/3033598554 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba2819f8b0 con 0x7fba28103660 2024-09-11T21:58:59.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.308+0000 7fba24ff9640 1 -- 172.21.15.1:0/3033598554 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1c018920 con 0x7fba28103660 2024-09-11T21:58:59.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.308+0000 7fba2dd73640 1 -- 172.21.15.1:0/3033598554 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba281a7190 con 0x7fba28103660 2024-09-11T21:58:59.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.311+0000 7fba24ff9640 1 -- 172.21.15.1:0/3033598554 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7fba1c010430 con 0x7fba28103660 2024-09-11T21:58:59.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.312+0000 7fba24ff9640 1 --2- 172.21.15.1:0/3033598554 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fba100a2d70 0x7fba100a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:59.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.312+0000 7fba277fe640 1 --2- 172.21.15.1:0/3033598554 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fba100a2d70 0x7fba100a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:59.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.312+0000 7fba24ff9640 1 -- 172.21.15.1:0/3033598554 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7fba1c0cf790 con 0x7fba28103660 2024-09-11T21:58:59.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.312+0000 7fba277fe640 1 --2- 172.21.15.1:0/3033598554 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fba100a2d70 0x7fba100a5230 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fba18009230 tx=0x7fba180058d0 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:59.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.348+0000 7fba3014c740 1 Processor -- start 2024-09-11T21:58:59.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.348+0000 7fba3014c740 1 -- start start 2024-09-11T21:58:59.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.349+0000 7fba3014c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ed55fb34a0 0x55ed55fb3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:59.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.349+0000 7fba3014c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed55fb3dc0 0x55ed55fbe140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:59.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.349+0000 7fba3014c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed55fbe680 0x55ed55fc0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:59.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.349+0000 7fba3014c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55ed55f192e0 con 0x55ed55fbe680 2024-09-11T21:58:59.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.349+0000 7fba3014c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55ed55ec0ce0 con 0x55ed55fb34a0 2024-09-11T21:58:59.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.349+0000 7fba3014c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ed55f152a0 con 0x55ed55fb3dc0 2024-09-11T21:58:59.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.349+0000 7fba27fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed55fbe680 0x55ed55fc0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:59.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.349+0000 7fba27fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed55fbe680 0x55ed55fc0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53850/0 (socket says 172.21.15.1:53850) 2024-09-11T21:58:59.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.349+0000 7fba27fff640 1 -- 172.21.15.1:0/817411544 learned_addr learned my addr 172.21.15.1:0/817411544 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:58:59.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.349+0000 7fba277fe640 1 --2- 172.21.15.1:0/817411544 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ed55fb34a0 0x55ed55fb3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:59.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.349+0000 7fba26ffd640 1 --2- 172.21.15.1:0/817411544 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed55fb3dc0 0x55ed55fbe140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:59.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.349+0000 7fba27fff640 1 -- 172.21.15.1:0/817411544 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed55fb3dc0 msgr2=0x55ed55fbe140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:59.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.349+0000 7fba27fff640 1 --2- 172.21.15.1:0/817411544 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed55fb3dc0 0x55ed55fbe140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:59.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.349+0000 7fba27fff640 1 -- 172.21.15.1:0/817411544 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ed55fb34a0 msgr2=0x55ed55fb3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:59.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.349+0000 7fba27fff640 1 --2- 172.21.15.1:0/817411544 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ed55fb34a0 0x55ed55fb3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:59.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.349+0000 7fba27fff640 1 -- 172.21.15.1:0/817411544 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed55ea5440 con 0x55ed55fbe680 2024-09-11T21:58:59.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.349+0000 7fba277fe640 1 --2- 172.21.15.1:0/817411544 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ed55fb34a0 0x55ed55fb3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T21:58:59.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.350+0000 7fba27fff640 1 --2- 172.21.15.1:0/817411544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed55fbe680 0x55ed55fc0a70 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fba20009fd0 tx=0x7fba20010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19c09f9e035c59bd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:59.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.350+0000 7fba0dffb640 1 -- 172.21.15.1:0/817411544 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba20011700 con 0x55ed55fbe680 2024-09-11T21:58:59.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.350+0000 7fba0dffb640 1 -- 172.21.15.1:0/817411544 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba20011d80 con 0x55ed55fbe680 2024-09-11T21:58:59.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.350+0000 7fba0dffb640 1 -- 172.21.15.1:0/817411544 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2000e3c0 con 0x55ed55fbe680 2024-09-11T21:58:59.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.351+0000 7fba3014c740 1 -- 172.21.15.1:0/817411544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed55fbe680 msgr2=0x55ed55fc0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:59.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.351+0000 7fba3014c740 1 --2- 172.21.15.1:0/817411544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed55fbe680 0x55ed55fc0a70 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fba20009fd0 tx=0x7fba20010860 comp rx=0 tx=0).stop 2024-09-11T21:58:59.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.351+0000 7fba3014c740 1 -- 172.21.15.1:0/817411544 shutdown_connections 2024-09-11T21:58:59.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.351+0000 7fba3014c740 1 --2- 172.21.15.1:0/817411544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed55fbe680 0x55ed55fc0a70 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:59.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.351+0000 7fba3014c740 1 --2- 172.21.15.1:0/817411544 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed55fb3dc0 0x55ed55fbe140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:59.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.351+0000 7fba3014c740 1 --2- 172.21.15.1:0/817411544 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ed55fb34a0 0x55ed55fb3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:59.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.351+0000 7fba3014c740 1 -- 172.21.15.1:0/817411544 >> 172.21.15.1:0/817411544 conn(0x55ed55fafa30 msgr2=0x55ed55fafe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:59.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.351+0000 7fba3014c740 1 -- 172.21.15.1:0/817411544 shutdown_connections 2024-09-11T21:58:59.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.351+0000 7fba3014c740 1 -- 172.21.15.1:0/817411544 wait complete. 2024-09-11T21:58:59.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.351+0000 7fba3014c740 1 -- start start 2024-09-11T21:58:59.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.352+0000 7fba3014c740 1 Processor -- start 2024-09-11T21:58:59.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.352+0000 7fba3014c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed55fb34a0 0x55ed55fbdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:59.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.352+0000 7fba3014c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ed55fb3dc0 0x55ed55fbb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:59.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.352+0000 7fba3014c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed55fbe680 0x55ed55fbb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:59.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.352+0000 7fba3014c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55ed55fc1250 con 0x55ed55fb34a0 2024-09-11T21:58:59.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.352+0000 7fba3014c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55ed55ed4050 con 0x55ed55fb3dc0 2024-09-11T21:58:59.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.352+0000 7fba3014c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ed55fc13d0 con 0x55ed55fbe680 2024-09-11T21:58:59.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.352+0000 7fba3014c740 10 client.0 _ll_register_callbacks cb 0x55ed55f14160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:58:59.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.352+0000 7fba277fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed55fb34a0 0x55ed55fbdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:59.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.353+0000 7fba27fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed55fbe680 0x55ed55fbb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:59.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.353+0000 7fba277fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed55fb34a0 0x55ed55fbdf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53864/0 (socket says 172.21.15.1:53864) 2024-09-11T21:58:59.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.353+0000 7fba277fe640 1 -- 172.21.15.1:0/530837332 learned_addr learned my addr 172.21.15.1:0/530837332 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:58:59.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.353+0000 7fba26ffd640 1 --2- 172.21.15.1:0/530837332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ed55fb3dc0 0x55ed55fbb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:59.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.353+0000 7fba277fe640 1 -- 172.21.15.1:0/530837332 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed55fbe680 msgr2=0x55ed55fbb730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:59.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.353+0000 7fba277fe640 1 --2- 172.21.15.1:0/530837332 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed55fbe680 0x55ed55fbb730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:59.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.353+0000 7fba277fe640 1 -- 172.21.15.1:0/530837332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ed55fb3dc0 msgr2=0x55ed55fbb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:59.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.353+0000 7fba277fe640 1 --2- 172.21.15.1:0/530837332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ed55fb3dc0 0x55ed55fbb1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:59.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.353+0000 7fba277fe640 1 -- 172.21.15.1:0/530837332 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed55c8a3c0 con 0x55ed55fb34a0 2024-09-11T21:58:59.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.353+0000 7fba26ffd640 1 --2- 172.21.15.1:0/530837332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ed55fb3dc0 0x55ed55fbb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T21:58:59.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.353+0000 7fba277fe640 1 --2- 172.21.15.1:0/530837332 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed55fb34a0 0x55ed55fbdf40 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fba18009900 tx=0x7fba18009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9796148b1de77836 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:59.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.353+0000 7fba06ffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T21:58:59.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.354+0000 7fba06ffd640 1 -- 172.21.15.1:0/530837332 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba18035040 con 0x55ed55fb34a0 2024-09-11T21:58:59.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.354+0000 7fba3014c740 1 -- 172.21.15.1:0/530837332 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba20011d80 con 0x55ed55fb34a0 2024-09-11T21:58:59.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.354+0000 7fba06ffd640 1 -- 172.21.15.1:0/530837332 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7fba18003a00 con 0x55ed55fb34a0 2024-09-11T21:58:59.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.354+0000 7fba06ffd640 1 -- 172.21.15.1:0/530837332 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fba18003c00 con 0x55ed55fb34a0 2024-09-11T21:58:59.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.354+0000 7fba3014c740 10 client.15843 fetch_fsmap learned FSMap version 28 2024-09-11T21:58:59.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.354+0000 7fba3014c740 1 -- 172.21.15.1:0/530837332 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ed55fccbf0 con 0x55ed55fb34a0 2024-09-11T21:58:59.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.354+0000 7fba06ffd640 1 -- 172.21.15.1:0/530837332 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba20011d80 con 0x55ed55fb34a0 2024-09-11T21:58:59.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.354+0000 7fba06ffd640 1 -- 172.21.15.1:0/530837332 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fba1804a430 con 0x55ed55fb34a0 2024-09-11T21:58:59.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.354+0000 7fba3014c740 10 client.15843 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T21:58:59.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.354+0000 7fba3014c740 10 client.15843 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T21:58:59.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.354+0000 7fba3014c740 20 client.15843 populate_metadata read hostname 'smithi001' 2024-09-11T21:58:59.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.354+0000 7fba3014c740 1 --2- 172.21.15.1:0/530837332 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55ed55fce0b0 0x55ed55fd04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:59.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.355+0000 7fba3014c740 4 client.15843 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T21:58:59.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.355+0000 7fba3014c740 1 -- 172.21.15.1:0/530837332 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed55f26850 con 0x55ed55fce0b0 2024-09-11T21:58:59.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.355+0000 7fba26ffd640 1 --2- 172.21.15.1:0/530837332 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55ed55fce0b0 0x55ed55fd04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:59.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.355+0000 7fba26ffd640 1 --2- 172.21.15.1:0/530837332 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55ed55fce0b0 0x55ed55fd04e0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67c07546f9a6ee31 server_cookie=467da79e2d199c93 in_seq=0 out_seq=0 2024-09-11T21:58:59.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.355+0000 7fba06ffd640 10 client.15843 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T21:58:59.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.356+0000 7fba06ffd640 1 -- 172.21.15.1:0/530837332 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ed55f26850 con 0x55ed55fce0b0 2024-09-11T21:58:59.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.356+0000 7fba06ffd640 10 client.15843 handle_command_reply: tid=0 2024-09-11T21:58:59.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.356+0000 7fba3014c740 1 client.15843 shutdown 2024-09-11T21:58:59.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.356+0000 7fba3014c740 1 -- 172.21.15.1:0/530837332 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed55fb34a0 msgr2=0x55ed55fbdf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:59.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.356+0000 7fba3014c740 1 --2- 172.21.15.1:0/530837332 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed55fb34a0 0x55ed55fbdf40 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fba18009900 tx=0x7fba18009260 comp rx=0 tx=0).stop 2024-09-11T21:58:59.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.357+0000 7fba3014c740 1 -- 172.21.15.1:0/530837332 shutdown_connections 2024-09-11T21:58:59.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.357+0000 7fba3014c740 1 --2- 172.21.15.1:0/530837332 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55ed55fce0b0 0x55ed55fd04e0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:59.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.357+0000 7fba3014c740 1 --2- 172.21.15.1:0/530837332 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed55fbe680 0x55ed55fbb730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:59.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.357+0000 7fba3014c740 1 --2- 172.21.15.1:0/530837332 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ed55fb3dc0 0x55ed55fbb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:59.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.357+0000 7fba3014c740 1 --2- 172.21.15.1:0/530837332 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed55fb34a0 0x55ed55fbdf40 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:59.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.357+0000 7fba06ffd640 0 client.15843 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T21:58:59.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.357+0000 7fba3014c740 1 -- 172.21.15.1:0/530837332 >> 172.21.15.1:0/530837332 conn(0x55ed55fafa30 msgr2=0x55ed55fb5710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:59.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.357+0000 7fba3014c740 1 -- 172.21.15.1:0/530837332 shutdown_connections 2024-09-11T21:58:59.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.357+0000 7fba3014c740 1 -- 172.21.15.1:0/530837332 wait complete. 2024-09-11T21:58:59.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.357+0000 7fba3014c740 20 client.15843 trim_cache size 0 max 16384 2024-09-11T21:58:59.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.375+0000 7fba3014c740 1 Processor -- start 2024-09-11T21:58:59.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.375+0000 7fba3014c740 1 -- start start 2024-09-11T21:58:59.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.375+0000 7fba3014c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed55fb34a0 0x55ed55fca4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:59.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.375+0000 7fba3014c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ed55fb3dc0 0x55ed55fd5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:59.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.375+0000 7fba3014c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed55fbe680 0x55ed55fd62c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:59.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.375+0000 7fba3014c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fba281afbc0 con 0x55ed55fbe680 2024-09-11T21:58:59.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.375+0000 7fba3014c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fba281b1700 con 0x55ed55fb3dc0 2024-09-11T21:58:59.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.375+0000 7fba3014c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fba281b0290 con 0x55ed55fb34a0 2024-09-11T21:58:59.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.375+0000 7fba27fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed55fbe680 0x55ed55fd62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:59.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.376+0000 7fba27fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed55fbe680 0x55ed55fd62c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53868/0 (socket says 172.21.15.1:53868) 2024-09-11T21:58:59.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.376+0000 7fba27fff640 1 -- 172.21.15.1:0/1552541781 learned_addr learned my addr 172.21.15.1:0/1552541781 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:58:59.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.376+0000 7fba277fe640 1 --2- 172.21.15.1:0/1552541781 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed55fb34a0 0x55ed55fca4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:59.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.376+0000 7fba26ffd640 1 --2- 172.21.15.1:0/1552541781 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ed55fb3dc0 0x55ed55fd5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:59.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.376+0000 7fba27fff640 1 -- 172.21.15.1:0/1552541781 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed55fb34a0 msgr2=0x55ed55fca4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:59.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.376+0000 7fba27fff640 1 --2- 172.21.15.1:0/1552541781 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed55fb34a0 0x55ed55fca4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:59.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.376+0000 7fba27fff640 1 -- 172.21.15.1:0/1552541781 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ed55fb3dc0 msgr2=0x55ed55fd5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:59.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.376+0000 7fba27fff640 1 --2- 172.21.15.1:0/1552541781 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ed55fb3dc0 0x55ed55fd5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:59.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.376+0000 7fba27fff640 1 -- 172.21.15.1:0/1552541781 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed55fbe2b0 con 0x55ed55fbe680 2024-09-11T21:58:59.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.376+0000 7fba277fe640 1 --2- 172.21.15.1:0/1552541781 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed55fb34a0 0x55ed55fca4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:58:59.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.376+0000 7fba26ffd640 1 --2- 172.21.15.1:0/1552541781 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ed55fb3dc0 0x55ed55fd5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:58:59.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.376+0000 7fba27fff640 1 --2- 172.21.15.1:0/1552541781 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed55fbe680 0x55ed55fd62c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fba2001bc00 tx=0x7fba2001bb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1242b0556784144 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:59.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.376+0000 7fba0d7fa640 1 -- 172.21.15.1:0/1552541781 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba20004bc0 con 0x55ed55fbe680 2024-09-11T21:58:59.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.376+0000 7fba0d7fa640 1 -- 172.21.15.1:0/1552541781 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba2002b070 con 0x55ed55fbe680 2024-09-11T21:58:59.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.376+0000 7fba0d7fa640 1 -- 172.21.15.1:0/1552541781 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba20011be0 con 0x55ed55fbe680 2024-09-11T21:58:59.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.377+0000 7fba3014c740 1 -- 172.21.15.1:0/1552541781 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed55fbe680 msgr2=0x55ed55fd62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:59.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.377+0000 7fba3014c740 1 --2- 172.21.15.1:0/1552541781 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed55fbe680 0x55ed55fd62c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fba2001bc00 tx=0x7fba2001bb90 comp rx=0 tx=0).stop 2024-09-11T21:58:59.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.377+0000 7fba3014c740 1 -- 172.21.15.1:0/1552541781 shutdown_connections 2024-09-11T21:58:59.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.377+0000 7fba3014c740 1 --2- 172.21.15.1:0/1552541781 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed55fbe680 0x55ed55fd62c0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:59.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.377+0000 7fba3014c740 1 --2- 172.21.15.1:0/1552541781 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ed55fb3dc0 0x55ed55fd5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:59.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.377+0000 7fba3014c740 1 --2- 172.21.15.1:0/1552541781 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed55fb34a0 0x55ed55fca4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:59.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.377+0000 7fba3014c740 1 -- 172.21.15.1:0/1552541781 >> 172.21.15.1:0/1552541781 conn(0x55ed55fafa30 msgr2=0x55ed55fb9ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:59.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.377+0000 7fba3014c740 1 -- 172.21.15.1:0/1552541781 shutdown_connections 2024-09-11T21:58:59.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.377+0000 7fba3014c740 1 -- 172.21.15.1:0/1552541781 wait complete. 2024-09-11T21:58:59.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.378+0000 7fba3014c740 1 -- start start 2024-09-11T21:58:59.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.378+0000 7fba3014c740 1 Processor -- start 2024-09-11T21:58:59.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.378+0000 7fba3014c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed55fb34a0 0x55ed55fc6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:59.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.378+0000 7fba3014c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed55fb3dc0 0x55ed55fc1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:59.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.378+0000 7fba3014c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ed55fbe680 0x55ed55fc23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:59.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.378+0000 7fba3014c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fba281b1030 con 0x55ed55fb34a0 2024-09-11T21:58:59.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.378+0000 7fba3014c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fba281b0960 con 0x55ed55fbe680 2024-09-11T21:58:59.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.378+0000 7fba3014c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fba281adba0 con 0x55ed55fb3dc0 2024-09-11T21:58:59.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.378+0000 7fba3014c740 10 client.0 _ll_register_callbacks cb 0x7fba1c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:58:59.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.379+0000 7fba277fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed55fb34a0 0x55ed55fc6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:59.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.379+0000 7fba277fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed55fb34a0 0x55ed55fc6b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53880/0 (socket says 172.21.15.1:53880) 2024-09-11T21:58:59.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.379+0000 7fba277fe640 1 -- 172.21.15.1:0/3808892117 learned_addr learned my addr 172.21.15.1:0/3808892117 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:58:59.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.379+0000 7fba26ffd640 1 --2- 172.21.15.1:0/3808892117 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed55fb3dc0 0x55ed55fc1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:59.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.379+0000 7fba27fff640 1 --2- 172.21.15.1:0/3808892117 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ed55fbe680 0x55ed55fc23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:59.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.379+0000 7fba277fe640 1 -- 172.21.15.1:0/3808892117 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed55fb3dc0 msgr2=0x55ed55fc1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:59.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.379+0000 7fba277fe640 1 --2- 172.21.15.1:0/3808892117 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed55fb3dc0 0x55ed55fc1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:59.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.379+0000 7fba277fe640 1 -- 172.21.15.1:0/3808892117 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ed55fbe680 msgr2=0x55ed55fc23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:58:59.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.379+0000 7fba277fe640 1 --2- 172.21.15.1:0/3808892117 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ed55fbe680 0x55ed55fc23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:59.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.379+0000 7fba277fe640 1 -- 172.21.15.1:0/3808892117 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed55fc29a0 con 0x55ed55fb34a0 2024-09-11T21:58:59.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.379+0000 7fba27fff640 1 --2- 172.21.15.1:0/3808892117 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ed55fbe680 0x55ed55fc23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:58:59.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.379+0000 7fba277fe640 1 --2- 172.21.15.1:0/3808892117 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed55fb34a0 0x55ed55fc6b20 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fba1804a850 tx=0x7fba18003fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad495aefa0bf96ab server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:58:59.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.379+0000 7fba06ffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T21:58:59.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.379+0000 7fba06ffd640 1 -- 172.21.15.1:0/3808892117 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba18035040 con 0x55ed55fb34a0 2024-09-11T21:58:59.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.379+0000 7fba06ffd640 1 -- 172.21.15.1:0/3808892117 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7fba18003a00 con 0x55ed55fb34a0 2024-09-11T21:58:59.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.379+0000 7fba3014c740 1 -- 172.21.15.1:0/3808892117 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba2002b070 con 0x55ed55fb34a0 2024-09-11T21:58:59.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.380+0000 7fba06ffd640 1 -- 172.21.15.1:0/3808892117 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fba1804a640 con 0x55ed55fb34a0 2024-09-11T21:58:59.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.380+0000 7fba3014c740 10 client.15849 fetch_fsmap learned FSMap version 28 2024-09-11T21:58:59.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.380+0000 7fba3014c740 1 -- 172.21.15.1:0/3808892117 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ed56038b10 con 0x55ed55fb34a0 2024-09-11T21:58:59.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.380+0000 7fba06ffd640 1 -- 172.21.15.1:0/3808892117 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba2002b070 con 0x55ed55fb34a0 2024-09-11T21:58:59.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.380+0000 7fba06ffd640 1 -- 172.21.15.1:0/3808892117 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fba1803c440 con 0x55ed55fb34a0 2024-09-11T21:58:59.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.380+0000 7fba3014c740 10 client.15849 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T21:58:59.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.380+0000 7fba3014c740 10 client.15849 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T21:58:59.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.380+0000 7fba3014c740 20 client.15849 populate_metadata read hostname 'smithi001' 2024-09-11T21:58:59.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.380+0000 7fba3014c740 1 --2- 172.21.15.1:0/3808892117 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55ed56039280 0x55ed5603b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:58:59.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.380+0000 7fba3014c740 4 client.15849 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T21:58:59.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.380+0000 7fba3014c740 1 -- 172.21.15.1:0/3808892117 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ed55f20d60 con 0x55ed56039280 2024-09-11T21:58:59.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.381+0000 7fba26ffd640 1 --2- 172.21.15.1:0/3808892117 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55ed56039280 0x55ed5603b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:58:59.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.381+0000 7fba26ffd640 1 --2- 172.21.15.1:0/3808892117 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55ed56039280 0x55ed5603b670 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=574269a3354293de server_cookie=59d0a173b8ee6a42 in_seq=0 out_seq=0 2024-09-11T21:58:59.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.381+0000 7fba06ffd640 10 client.15849 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T21:58:59.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.381+0000 7fba06ffd640 1 -- 172.21.15.1:0/3808892117 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ed55f20d60 con 0x55ed56039280 2024-09-11T21:58:59.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.381+0000 7fba06ffd640 10 client.15849 handle_command_reply: tid=0 2024-09-11T21:58:59.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.381+0000 7fba3014c740 1 client.15849 shutdown 2024-09-11T21:58:59.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.382+0000 7fba3014c740 1 -- 172.21.15.1:0/3808892117 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed55fb34a0 msgr2=0x55ed55fc6b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:59.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.382+0000 7fba3014c740 1 --2- 172.21.15.1:0/3808892117 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed55fb34a0 0x55ed55fc6b20 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fba1804a850 tx=0x7fba18003fd0 comp rx=0 tx=0).stop 2024-09-11T21:58:59.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.382+0000 7fba3014c740 1 -- 172.21.15.1:0/3808892117 shutdown_connections 2024-09-11T21:58:59.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.382+0000 7fba3014c740 1 --2- 172.21.15.1:0/3808892117 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55ed56039280 0x55ed5603b670 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:59.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.382+0000 7fba3014c740 1 --2- 172.21.15.1:0/3808892117 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ed55fbe680 0x55ed55fc23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:59.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.382+0000 7fba3014c740 1 --2- 172.21.15.1:0/3808892117 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ed55fb3dc0 0x55ed55fc1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:59.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.382+0000 7fba3014c740 1 --2- 172.21.15.1:0/3808892117 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ed55fb34a0 0x55ed55fc6b20 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:59.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.382+0000 7fba3014c740 1 -- 172.21.15.1:0/3808892117 >> 172.21.15.1:0/3808892117 conn(0x55ed55fafa30 msgr2=0x55ed55faeb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:59.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.382+0000 7fba06ffd640 0 client.15849 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T21:58:59.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.382+0000 7fba3014c740 1 -- 172.21.15.1:0/3808892117 shutdown_connections 2024-09-11T21:58:59.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.382+0000 7fba3014c740 1 -- 172.21.15.1:0/3808892117 wait complete. 2024-09-11T21:58:59.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.382+0000 7fba3014c740 20 client.15849 trim_cache size 0 max 16384 2024-09-11T21:58:59.388 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-11T21:58:59.388 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-09-11T21:58:59.388 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-09-11T21:58:59.388 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-11T21:58:59.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.383+0000 7fba2dd73640 1 -- 172.21.15.1:0/3033598554 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fba100a2d70 msgr2=0x7fba100a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:59.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.383+0000 7fba2dd73640 1 --2- 172.21.15.1:0/3033598554 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fba100a2d70 0x7fba100a5230 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fba18009230 tx=0x7fba180058d0 comp rx=0 tx=0).stop 2024-09-11T21:58:59.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.383+0000 7fba2dd73640 1 -- 172.21.15.1:0/3033598554 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba28103660 msgr2=0x7fba2819ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:58:59.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.383+0000 7fba2dd73640 1 --2- 172.21.15.1:0/3033598554 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba28103660 0x7fba2819ab50 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fba1c00e9f0 tx=0x7fba1c00eeb0 comp rx=0 tx=0).stop 2024-09-11T21:58:59.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.383+0000 7fba2dd73640 1 -- 172.21.15.1:0/3033598554 shutdown_connections 2024-09-11T21:58:59.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.383+0000 7fba2dd73640 1 --2- 172.21.15.1:0/3033598554 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fba100a2d70 0x7fba100a5230 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:59.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.383+0000 7fba2dd73640 1 --2- 172.21.15.1:0/3033598554 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba2810c170 0x7fba2819eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:59.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.383+0000 7fba2dd73640 1 --2- 172.21.15.1:0/3033598554 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba28103660 0x7fba2819ab50 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:59.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.383+0000 7fba2dd73640 1 --2- 172.21.15.1:0/3033598554 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba28102d40 0x7fba2819a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:58:59.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.383+0000 7fba2dd73640 1 -- 172.21.15.1:0/3033598554 >> 172.21.15.1:0/3033598554 conn(0x7fba280fc7a0 msgr2=0x7fba2810d490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:58:59.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.383+0000 7fba2dd73640 1 -- 172.21.15.1:0/3033598554 shutdown_connections 2024-09-11T21:58:59.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:58:59.383+0000 7fba2dd73640 1 -- 172.21.15.1:0/3033598554 wait complete. 2024-09-11T21:59:00.474 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:59:00.474 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:59:00.805 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T21:59:00.806 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T21:59:00.826 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:59:00.826 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:59:00.826 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 20 -lt 100 ']' 2024-09-11T21:59:00.826 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 20 + 1 2024-09-11T21:59:00.905 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=21 2024-09-11T21:59:00.905 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T21:59:00.948 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:59:00.948 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:59:00.949 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T21:59:00.949 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T21:59:00.949 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T21:59:01.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:59:00 smithi053 ceph-mon[34908]: pgmap v58: 97 pgs: 97 active+clean; 1.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 20 op/s 2024-09-11T21:59:01.091 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:59:00 smithi161 ceph-mon[35615]: pgmap v58: 97 pgs: 97 active+clean; 1.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 20 op/s 2024-09-11T21:59:01.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:59:00 smithi001 ceph-mon[30935]: pgmap v58: 97 pgs: 97 active+clean; 1.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 20 op/s 2024-09-11T21:59:03.038 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:59:02 smithi001 ceph-mon[30935]: pgmap v59: 97 pgs: 97 active+clean; 1.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.0 KiB/s wr, 10 op/s 2024-09-11T21:59:03.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:59:02 smithi053 ceph-mon[34908]: pgmap v59: 97 pgs: 97 active+clean; 1.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.0 KiB/s wr, 10 op/s 2024-09-11T21:59:03.091 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:59:02 smithi161 ceph-mon[35615]: pgmap v59: 97 pgs: 97 active+clean; 1.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.0 KiB/s wr, 10 op/s 2024-09-11T21:59:03.849 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:59:03.849 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:59:05.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:59:04 smithi053 ceph-mon[34908]: pgmap v60: 97 pgs: 97 active+clean; 1.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.0 KiB/s wr, 10 op/s 2024-09-11T21:59:05.091 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:59:04 smithi161 ceph-mon[35615]: pgmap v60: 97 pgs: 97 active+clean; 1.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.0 KiB/s wr, 10 op/s 2024-09-11T21:59:05.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:59:04 smithi001 ceph-mon[30935]: pgmap v60: 97 pgs: 97 active+clean; 1.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.0 KiB/s wr, 10 op/s 2024-09-11T21:59:05.231 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T21:59:05.231 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T21:59:05.272 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:59:05.272 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:59:05.272 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 21 -lt 100 ']' 2024-09-11T21:59:05.272 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 21 + 1 2024-09-11T21:59:05.354 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=22 2024-09-11T21:59:05.355 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T21:59:05.404 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T21:59:05.405 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T21:59:05.406 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:59:05.406 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:59:05.408 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T21:59:05.408 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T21:59:05.408 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T21:59:06.305 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:59:06.306 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:59:06.555 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T21:59:06.555 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T21:59:06.572 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:59:06.573 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:59:06.573 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 22 -lt 100 ']' 2024-09-11T21:59:06.573 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 22 + 1 2024-09-11T21:59:06.695 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=23 2024-09-11T21:59:06.695 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T21:59:06.731 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:59:06.731 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:59:06.732 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T21:59:06.732 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T21:59:06.732 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T21:59:07.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:59:06 smithi053 ceph-mon[34908]: pgmap v61: 97 pgs: 97 active+clean; 1.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 33 KiB/s wr, 30 op/s 2024-09-11T21:59:07.091 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:59:06 smithi161 ceph-mon[35615]: pgmap v61: 97 pgs: 97 active+clean; 1.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 33 KiB/s wr, 30 op/s 2024-09-11T21:59:07.119 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:59:06 smithi001 ceph-mon[30935]: pgmap v61: 97 pgs: 97 active+clean; 1.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 33 KiB/s wr, 30 op/s 2024-09-11T21:59:07.356 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:59:07.356 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:59:07.638 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T21:59:07.638 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T21:59:07.648 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:59:07.648 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:59:07.648 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 23 -lt 100 ']' 2024-09-11T21:59:07.648 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 23 + 1 2024-09-11T21:59:07.651 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:07.649+0000 7f54942a4640 1 -- 172.21.15.1:0/4244285339 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f548c106e60 msgr2=0x7f548c10d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:59:07.651 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:07.649+0000 7f54942a4640 1 --2- 172.21.15.1:0/4244285339 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f548c106e60 0x7f548c10d8d0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f5484009ca0 tx=0x7f548402a8c0 comp rx=0 tx=0).stop 2024-09-11T21:59:07.651 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:07.650+0000 7f54942a4640 1 -- 172.21.15.1:0/4244285339 shutdown_connections 2024-09-11T21:59:07.651 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:07.650+0000 7f54942a4640 1 --2- 172.21.15.1:0/4244285339 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f548c106e60 0x7f548c10d8d0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:07.652 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:07.650+0000 7f54942a4640 1 --2- 172.21.15.1:0/4244285339 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f548c1022d0 0x7f548c106700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:07.652 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:07.650+0000 7f54942a4640 1 --2- 172.21.15.1:0/4244285339 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f548c101920 0x7f548c101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:07.652 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:07.650+0000 7f54942a4640 1 -- 172.21.15.1:0/4244285339 >> 172.21.15.1:0/4244285339 conn(0x7f548c075eb0 msgr2=0x7f548c0762c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:59:07.652 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:07.650+0000 7f54942a4640 1 -- 172.21.15.1:0/4244285339 shutdown_connections 2024-09-11T21:59:07.652 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:07.650+0000 7f54942a4640 1 -- 172.21.15.1:0/4244285339 wait complete. 2024-09-11T21:59:07.685 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=24 2024-09-11T21:59:07.685 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T21:59:07.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:07.732+0000 7f54942a4640 1 Processor -- start 2024-09-11T21:59:07.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:07.732+0000 7f54942a4640 1 -- start start 2024-09-11T21:59:07.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:07.733+0000 7f54942a4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f548c101920 0x7f548c19a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:07.736 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:07.733+0000 7f54942a4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f548c106e60 0x7f548c19ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:07.736 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:07.733+0000 7f54942a4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f548c19eee0 0x7f548c19f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:07.736 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:07.733+0000 7f54942a4640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f548c10fd20 con 0x7f548c19eee0 2024-09-11T21:59:07.736 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:07.733+0000 7f54942a4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f548c10fba0 con 0x7f548c101920 2024-09-11T21:59:07.736 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:07.733+0000 7f54942a4640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f548c10fea0 con 0x7f548c106e60 2024-09-11T21:59:07.736 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:07.733+0000 7f5492841640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f548c19eee0 0x7f548c19f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:07.736 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:07.733+0000 7f5492841640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f548c19eee0 0x7f548c19f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55010/0 (socket says 172.21.15.1:55010) 2024-09-11T21:59:07.736 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:07.733+0000 7f5492841640 1 -- 172.21.15.1:0/3187459119 learned_addr learned my addr 172.21.15.1:0/3187459119 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:59:07.736 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:07.733+0000 7f5492040640 1 --2- 172.21.15.1:0/3187459119 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f548c101920 0x7f548c19a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:07.736 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:07.733+0000 7f549183f640 1 --2- 172.21.15.1:0/3187459119 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f548c106e60 0x7f548c19ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:07.736 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:07.733+0000 7f5492841640 1 -- 172.21.15.1:0/3187459119 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f548c106e60 msgr2=0x7f548c19ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:59:07.736 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:07.733+0000 7f5492841640 1 --2- 172.21.15.1:0/3187459119 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f548c106e60 0x7f548c19ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:07.737 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:07.733+0000 7f5492841640 1 -- 172.21.15.1:0/3187459119 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f548c101920 msgr2=0x7f548c19a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:59:07.737 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:07.733+0000 7f5492841640 1 --2- 172.21.15.1:0/3187459119 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f548c101920 0x7f548c19a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:07.737 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:07.733+0000 7f5492841640 1 -- 172.21.15.1:0/3187459119 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f548c19f9d0 con 0x7f548c19eee0 2024-09-11T21:59:07.737 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:07.733+0000 7f549183f640 1 --2- 172.21.15.1:0/3187459119 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f548c106e60 0x7f548c19ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T21:59:07.737 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:07.733+0000 7f5492040640 1 --2- 172.21.15.1:0/3187459119 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f548c101920 0x7f548c19a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T21:59:07.737 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:07.734+0000 7f5492841640 1 --2- 172.21.15.1:0/3187459119 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f548c19eee0 0x7f548c19f390 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f5484005e30 tx=0x7f5484003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:59:07.756 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:07.755+0000 7f54837fe640 1 -- 172.21.15.1:0/3187459119 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5484031650 con 0x7f548c19eee0 2024-09-11T21:59:07.756 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:07.755+0000 7f54837fe640 1 -- 172.21.15.1:0/3187459119 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54840317f0 con 0x7f548c19eee0 2024-09-11T21:59:07.756 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:07.755+0000 7f54837fe640 1 -- 172.21.15.1:0/3187459119 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f548402f9d0 con 0x7f548c19eee0 2024-09-11T21:59:07.757 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:07.755+0000 7f54942a4640 1 -- 172.21.15.1:0/3187459119 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f548c19fc60 con 0x7f548c19eee0 2024-09-11T21:59:07.759 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:07.757+0000 7f54837fe640 1 -- 172.21.15.1:0/3187459119 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f548402fbb0 con 0x7f548c19eee0 2024-09-11T21:59:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:07.758+0000 7f54837fe640 1 --2- 172.21.15.1:0/3187459119 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f54780a2e40 0x7f54780a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:07.758+0000 7f5492040640 1 --2- 172.21.15.1:0/3187459119 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f54780a2e40 0x7f54780a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:07.759+0000 7f5492040640 1 --2- 172.21.15.1:0/3187459119 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f54780a2e40 0x7f54780a5300 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5488009fc0 tx=0x7f5488005f90 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:59:07.765 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:59:07.765 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:59:07.766 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T21:59:07.766 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T21:59:07.766 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T21:59:07.830 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:07.828+0000 7f54942a4640 1 -- 172.21.15.1:0/3187459119 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f548c1a7480 con 0x7f548c19eee0 2024-09-11T21:59:07.830 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:07.828+0000 7f54837fe640 1 -- 172.21.15.1:0/3187459119 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f54840ba900 con 0x7f548c19eee0 2024-09-11T21:59:07.969 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:07.967+0000 7f5497037740 1 Processor -- start 2024-09-11T21:59:07.970 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:07.967+0000 7f5497037740 1 -- start start 2024-09-11T21:59:07.970 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:07.968+0000 7f5497037740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d0ca4baf0 0x564d0ca4bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:07.970 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:07.968+0000 7f5497037740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564d0ca4c410 0x564d0ca56770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:07.970 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:07.968+0000 7f5497037740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d0ca56cb0 0x564d0ca590a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:07.970 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:07.968+0000 7f5497037740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x564d0c9b02f0 con 0x564d0ca56cb0 2024-09-11T21:59:07.970 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:07.968+0000 7f5497037740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x564d0c958270 con 0x564d0ca4baf0 2024-09-11T21:59:07.970 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:07.968+0000 7f5497037740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x564d0c9ac2b0 con 0x564d0ca4c410 2024-09-11T21:59:07.970 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:07.968+0000 7f5492841640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d0ca56cb0 0x564d0ca590a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:07.970 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:07.968+0000 7f5492841640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d0ca56cb0 0x564d0ca590a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55020/0 (socket says 172.21.15.1:55020) 2024-09-11T21:59:07.970 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:07.968+0000 7f549183f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564d0ca4c410 0x564d0ca56770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:07.970 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:07.968+0000 7f5492040640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d0ca4baf0 0x564d0ca4bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:07.970 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:07.968+0000 7f5492841640 1 -- 172.21.15.1:0/1758251955 learned_addr learned my addr 172.21.15.1:0/1758251955 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:59:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:07.968+0000 7f5492841640 1 -- 172.21.15.1:0/1758251955 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564d0ca4c410 msgr2=0x564d0ca56770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:59:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:07.968+0000 7f5492841640 1 --2- 172.21.15.1:0/1758251955 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564d0ca4c410 0x564d0ca56770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:07.968+0000 7f5492841640 1 -- 172.21.15.1:0/1758251955 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d0ca4baf0 msgr2=0x564d0ca4bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:59:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:07.968+0000 7f5492841640 1 --2- 172.21.15.1:0/1758251955 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d0ca4baf0 0x564d0ca4bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:07.968+0000 7f5492841640 1 -- 172.21.15.1:0/1758251955 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d0c957340 con 0x564d0ca56cb0 2024-09-11T21:59:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:07.969+0000 7f5492841640 1 --2- 172.21.15.1:0/1758251955 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d0ca56cb0 0x564d0ca590a0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f5484006010 tx=0x7f54840093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36250c101be26776 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:59:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:07.969+0000 7f5480bfe640 1 -- 172.21.15.1:0/1758251955 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54840bcf60 con 0x564d0ca56cb0 2024-09-11T21:59:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:07.969+0000 7f5480bfe640 1 -- 172.21.15.1:0/1758251955 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54840d4070 con 0x564d0ca56cb0 2024-09-11T21:59:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:07.969+0000 7f5480bfe640 1 -- 172.21.15.1:0/1758251955 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54840c5040 con 0x564d0ca56cb0 2024-09-11T21:59:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:07.970+0000 7f5497037740 1 -- 172.21.15.1:0/1758251955 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d0ca56cb0 msgr2=0x564d0ca590a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:59:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:07.970+0000 7f5497037740 1 --2- 172.21.15.1:0/1758251955 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d0ca56cb0 0x564d0ca590a0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f5484006010 tx=0x7f54840093d0 comp rx=0 tx=0).stop 2024-09-11T21:59:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:07.970+0000 7f5497037740 1 -- 172.21.15.1:0/1758251955 shutdown_connections 2024-09-11T21:59:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:07.970+0000 7f5497037740 1 --2- 172.21.15.1:0/1758251955 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d0ca56cb0 0x564d0ca590a0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:07.970+0000 7f5497037740 1 --2- 172.21.15.1:0/1758251955 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564d0ca4c410 0x564d0ca56770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:07.970+0000 7f5497037740 1 --2- 172.21.15.1:0/1758251955 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d0ca4baf0 0x564d0ca4bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:07.970+0000 7f5497037740 1 -- 172.21.15.1:0/1758251955 >> 172.21.15.1:0/1758251955 conn(0x564d0ca48060 msgr2=0x564d0ca48440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:59:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:07.970+0000 7f5497037740 1 -- 172.21.15.1:0/1758251955 shutdown_connections 2024-09-11T21:59:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:07.970+0000 7f5497037740 1 -- 172.21.15.1:0/1758251955 wait complete. 2024-09-11T21:59:08.033 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.031+0000 7f5497037740 1 -- start start 2024-09-11T21:59:08.047 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.045+0000 7f5497037740 1 Processor -- start 2024-09-11T21:59:08.048 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.046+0000 7f5497037740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564d0ca4baf0 0x564d0ca53c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:08.048 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.046+0000 7f5497037740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d0ca4c410 0x564d0ca541c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:08.049 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.046+0000 7f5497037740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d0ca56cb0 0x564d0ca54700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:08.049 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.046+0000 7f5497037740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x564d0c9b02f0 con 0x564d0ca56cb0 2024-09-11T21:59:08.049 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.046+0000 7f5497037740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x564d0c939290 con 0x564d0ca4c410 2024-09-11T21:59:08.049 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.046+0000 7f5497037740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x564d0ca59640 con 0x564d0ca4baf0 2024-09-11T21:59:08.049 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.046+0000 7f5492841640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d0ca56cb0 0x564d0ca54700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:08.049 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.046+0000 7f5492841640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d0ca56cb0 0x564d0ca54700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55024/0 (socket says 172.21.15.1:55024) 2024-09-11T21:59:08.049 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.046+0000 7f5492841640 1 -- 172.21.15.1:0/959090724 learned_addr learned my addr 172.21.15.1:0/959090724 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:59:08.049 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.046+0000 7f5492040640 1 --2- 172.21.15.1:0/959090724 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564d0ca4baf0 0x564d0ca53c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:08.049 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.046+0000 7f549183f640 1 --2- 172.21.15.1:0/959090724 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d0ca4c410 0x564d0ca541c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:08.049 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.046+0000 7f5492841640 1 -- 172.21.15.1:0/959090724 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564d0ca4baf0 msgr2=0x564d0ca53c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:59:08.049 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.046+0000 7f5492841640 1 --2- 172.21.15.1:0/959090724 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564d0ca4baf0 0x564d0ca53c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:08.049 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.046+0000 7f5492841640 1 -- 172.21.15.1:0/959090724 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d0ca4c410 msgr2=0x564d0ca541c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:59:08.050 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.046+0000 7f5492841640 1 --2- 172.21.15.1:0/959090724 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d0ca4c410 0x564d0ca541c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:08.050 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.046+0000 7f5492841640 1 -- 172.21.15.1:0/959090724 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d0c6c3210 con 0x564d0ca56cb0 2024-09-11T21:59:08.050 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.047+0000 7f5492841640 1 --2- 172.21.15.1:0/959090724 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d0ca56cb0 0x564d0ca54700 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f54840bc560 tx=0x7f54840edeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c978e82d9bc5c4cd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:59:08.050 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.047+0000 7f5475ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T21:59:08.050 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.047+0000 7f5475ffb640 1 -- 172.21.15.1:0/959090724 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54840ca060 con 0x564d0ca56cb0 2024-09-11T21:59:08.050 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.047+0000 7f5475ffb640 1 -- 172.21.15.1:0/959090724 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f54840c5250 con 0x564d0ca56cb0 2024-09-11T21:59:08.078 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.077+0000 7f5497037740 10 client.0 _ll_register_callbacks cb 0x564d0c96a140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:59:08.080 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.078+0000 7f5497037740 1 -- 172.21.15.1:0/959090724 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54840d4070 con 0x564d0ca56cb0 2024-09-11T21:59:08.080 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.079+0000 7f5475ffb640 1 -- 172.21.15.1:0/959090724 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f54840c54d0 con 0x564d0ca56cb0 2024-09-11T21:59:08.080 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.079+0000 7f5497037740 10 client.15861 fetch_fsmap learned FSMap version 28 2024-09-11T21:59:08.080 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.079+0000 7f5497037740 1 -- 172.21.15.1:0/959090724 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({fsmap=28}) -- 0x564d0c78f620 con 0x564d0ca56cb0 2024-09-11T21:59:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.135+0000 7f5475ffb640 1 -- 172.21.15.1:0/959090724 <== mon.0 v2:172.21.15.1:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f54840bdb90 con 0x564d0ca56cb0 2024-09-11T21:59:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.135+0000 7f5497037740 10 client.15861 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T21:59:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.135+0000 7f5497037740 10 client.15861 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T21:59:08.140 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.138+0000 7f5497037740 20 client.15861 populate_metadata read hostname 'smithi001' 2024-09-11T21:59:08.140 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.138+0000 7f5497037740 1 --2- 172.21.15.1:0/959090724 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x564d0ca66550 0x564d0ca68940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:08.140 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.138+0000 7f5497037740 4 client.15861 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T21:59:08.140 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.138+0000 7f5497037740 1 -- 172.21.15.1:0/959090724 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d0c9bda20 con 0x564d0ca66550 2024-09-11T21:59:08.140 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.139+0000 7f5492040640 1 --2- 172.21.15.1:0/959090724 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x564d0ca66550 0x564d0ca68940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:08.141 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.139+0000 7f5492040640 1 --2- 172.21.15.1:0/959090724 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x564d0ca66550 0x564d0ca68940 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b817e71ff230f946 server_cookie=6ecd1050f6a1e450 in_seq=0 out_seq=0 2024-09-11T21:59:08.141 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.139+0000 7f5475ffb640 10 client.15861 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T21:59:08.142 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.140+0000 7f5475ffb640 1 -- 172.21.15.1:0/959090724 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564d0c9bda20 con 0x564d0ca66550 2024-09-11T21:59:08.142 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.140+0000 7f5475ffb640 10 client.15861 handle_command_reply: tid=0 2024-09-11T21:59:08.142 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.140+0000 7f5497037740 1 client.15861 shutdown 2024-09-11T21:59:08.142 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.140+0000 7f5497037740 1 -- 172.21.15.1:0/959090724 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d0ca56cb0 msgr2=0x564d0ca54700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:59:08.142 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.140+0000 7f5497037740 1 --2- 172.21.15.1:0/959090724 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d0ca56cb0 0x564d0ca54700 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f54840bc560 tx=0x7f54840edeb0 comp rx=0 tx=0).stop 2024-09-11T21:59:08.143 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.140+0000 7f5497037740 1 -- 172.21.15.1:0/959090724 shutdown_connections 2024-09-11T21:59:08.143 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.140+0000 7f5497037740 1 --2- 172.21.15.1:0/959090724 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x564d0ca66550 0x564d0ca68940 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:08.143 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.140+0000 7f5497037740 1 --2- 172.21.15.1:0/959090724 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d0ca56cb0 0x564d0ca54700 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:08.143 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.140+0000 7f5497037740 1 --2- 172.21.15.1:0/959090724 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d0ca4c410 0x564d0ca541c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:08.143 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.140+0000 7f5475ffb640 0 client.15861 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T21:59:08.143 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.140+0000 7f5497037740 1 --2- 172.21.15.1:0/959090724 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564d0ca4baf0 0x564d0ca53c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:08.143 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.140+0000 7f5497037740 1 -- 172.21.15.1:0/959090724 >> 172.21.15.1:0/959090724 conn(0x564d0ca48060 msgr2=0x564d0ca4dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:59:08.143 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.140+0000 7f5497037740 1 -- 172.21.15.1:0/959090724 shutdown_connections 2024-09-11T21:59:08.143 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.140+0000 7f5497037740 1 -- 172.21.15.1:0/959090724 wait complete. 2024-09-11T21:59:08.143 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.141+0000 7f5497037740 20 client.15861 trim_cache size 0 max 16384 2024-09-11T21:59:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.193+0000 7f5497037740 1 Processor -- start 2024-09-11T21:59:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.193+0000 7f5497037740 1 -- start start 2024-09-11T21:59:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.193+0000 7f5497037740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d0ca4baf0 0x564d0ca54530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.193+0000 7f5497037740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564d0ca4c410 0x564d0ca6e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.193+0000 7f5497037740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d0ca56cb0 0x564d0ca6e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.193+0000 7f5497037740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f548c1afb50 con 0x564d0ca4baf0 2024-09-11T21:59:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.193+0000 7f5497037740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f548c1b1690 con 0x564d0ca56cb0 2024-09-11T21:59:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.193+0000 7f5497037740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f548c1b0220 con 0x564d0ca4c410 2024-09-11T21:59:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.193+0000 7f5492040640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d0ca4baf0 0x564d0ca54530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.193+0000 7f5492040640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d0ca4baf0 0x564d0ca54530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55036/0 (socket says 172.21.15.1:55036) 2024-09-11T21:59:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.193+0000 7f5492040640 1 -- 172.21.15.1:0/4270605727 learned_addr learned my addr 172.21.15.1:0/4270605727 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:59:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.193+0000 7f5492841640 1 --2- 172.21.15.1:0/4270605727 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d0ca56cb0 0x564d0ca6e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.193+0000 7f549183f640 1 --2- 172.21.15.1:0/4270605727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564d0ca4c410 0x564d0ca6e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.194+0000 7f5492040640 1 -- 172.21.15.1:0/4270605727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564d0ca4c410 msgr2=0x564d0ca6e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:59:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.194+0000 7f5492040640 1 --2- 172.21.15.1:0/4270605727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564d0ca4c410 0x564d0ca6e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:08.201 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.194+0000 7f5492040640 1 -- 172.21.15.1:0/4270605727 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d0ca56cb0 msgr2=0x564d0ca6e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:59:08.202 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.194+0000 7f5492040640 1 --2- 172.21.15.1:0/4270605727 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d0ca56cb0 0x564d0ca6e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:08.202 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.194+0000 7f5492040640 1 -- 172.21.15.1:0/4270605727 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d0ca48cf0 con 0x564d0ca4baf0 2024-09-11T21:59:08.202 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.194+0000 7f5492040640 1 --2- 172.21.15.1:0/4270605727 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d0ca4baf0 0x564d0ca54530 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f5488009e80 tx=0x7f5488014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30a8527632daed51 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:59:08.202 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.194+0000 7f5477fff640 1 -- 172.21.15.1:0/4270605727 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f548800f680 con 0x564d0ca4baf0 2024-09-11T21:59:08.202 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.194+0000 7f5477fff640 1 -- 172.21.15.1:0/4270605727 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f548800fd00 con 0x564d0ca4baf0 2024-09-11T21:59:08.202 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.194+0000 7f5477fff640 1 -- 172.21.15.1:0/4270605727 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f548802adc0 con 0x564d0ca4baf0 2024-09-11T21:59:08.202 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.195+0000 7f5497037740 1 -- 172.21.15.1:0/4270605727 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d0ca4baf0 msgr2=0x564d0ca54530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:59:08.202 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.195+0000 7f5497037740 1 --2- 172.21.15.1:0/4270605727 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d0ca4baf0 0x564d0ca54530 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f5488009e80 tx=0x7f5488014000 comp rx=0 tx=0).stop 2024-09-11T21:59:08.202 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.195+0000 7f5497037740 1 -- 172.21.15.1:0/4270605727 shutdown_connections 2024-09-11T21:59:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.195+0000 7f5497037740 1 --2- 172.21.15.1:0/4270605727 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d0ca56cb0 0x564d0ca6e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.195+0000 7f5497037740 1 --2- 172.21.15.1:0/4270605727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564d0ca4c410 0x564d0ca6e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.195+0000 7f5497037740 1 --2- 172.21.15.1:0/4270605727 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d0ca4baf0 0x564d0ca54530 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.195+0000 7f5497037740 1 -- 172.21.15.1:0/4270605727 >> 172.21.15.1:0/4270605727 conn(0x564d0ca48060 msgr2=0x564d0ca4dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:59:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.195+0000 7f5497037740 1 -- 172.21.15.1:0/4270605727 shutdown_connections 2024-09-11T21:59:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.195+0000 7f5497037740 1 -- 172.21.15.1:0/4270605727 wait complete. 2024-09-11T21:59:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.196+0000 7f5497037740 1 -- start start 2024-09-11T21:59:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.196+0000 7f5497037740 1 Processor -- start 2024-09-11T21:59:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.196+0000 7f5497037740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564d0ca4baf0 0x564d0ca5e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.196+0000 7f5497037740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d0ca4c410 0x564d0ca5edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.196+0000 7f5497037740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d0ca56cb0 0x564d0ca5b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.196+0000 7f5497037740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f548c1b0fc0 con 0x564d0ca4c410 2024-09-11T21:59:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.196+0000 7f5497037740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f548c1b08f0 con 0x564d0ca56cb0 2024-09-11T21:59:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.196+0000 7f5497037740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f548c1adb30 con 0x564d0ca4baf0 2024-09-11T21:59:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.196+0000 7f5497037740 10 client.0 _ll_register_callbacks cb 0x7f5488009db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:59:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.196+0000 7f549183f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d0ca4c410 0x564d0ca5edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.196+0000 7f549183f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d0ca4c410 0x564d0ca5edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55046/0 (socket says 172.21.15.1:55046) 2024-09-11T21:59:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.196+0000 7f549183f640 1 -- 172.21.15.1:0/489849521 learned_addr learned my addr 172.21.15.1:0/489849521 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:59:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.196+0000 7f5492040640 1 --2- 172.21.15.1:0/489849521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564d0ca4baf0 0x564d0ca5e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.196+0000 7f5492841640 1 --2- 172.21.15.1:0/489849521 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d0ca56cb0 0x564d0ca5b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.197+0000 7f549183f640 1 -- 172.21.15.1:0/489849521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564d0ca4baf0 msgr2=0x564d0ca5e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:59:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.197+0000 7f549183f640 1 --2- 172.21.15.1:0/489849521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564d0ca4baf0 0x564d0ca5e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.197+0000 7f549183f640 1 -- 172.21.15.1:0/489849521 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d0ca56cb0 msgr2=0x564d0ca5b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:59:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.197+0000 7f549183f640 1 --2- 172.21.15.1:0/489849521 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d0ca56cb0 0x564d0ca5b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.197+0000 7f549183f640 1 -- 172.21.15.1:0/489849521 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d0ca54a70 con 0x564d0ca4c410 2024-09-11T21:59:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.197+0000 7f5492040640 1 --2- 172.21.15.1:0/489849521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564d0ca4baf0 0x564d0ca5e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:59:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.197+0000 7f549183f640 1 --2- 172.21.15.1:0/489849521 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d0ca4c410 0x564d0ca5edd0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f548c10d680 tx=0x7f547c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=815b9c8ec59a7cec server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:59:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.197+0000 7f5475ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T21:59:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.197+0000 7f5475ffb640 1 -- 172.21.15.1:0/489849521 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f547c022060 con 0x564d0ca4c410 2024-09-11T21:59:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.197+0000 7f5475ffb640 1 -- 172.21.15.1:0/489849521 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f547c004230 con 0x564d0ca4c410 2024-09-11T21:59:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.197+0000 7f5497037740 1 -- 172.21.15.1:0/489849521 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f548800fd00 con 0x564d0ca4c410 2024-09-11T21:59:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.197+0000 7f5475ffb640 1 -- 172.21.15.1:0/489849521 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f547c010e60 con 0x564d0ca4c410 2024-09-11T21:59:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.197+0000 7f5497037740 10 client.15867 fetch_fsmap learned FSMap version 28 2024-09-11T21:59:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.197+0000 7f5497037740 1 -- 172.21.15.1:0/489849521 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564d0cad1920 con 0x564d0ca4c410 2024-09-11T21:59:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.197+0000 7f5475ffb640 1 -- 172.21.15.1:0/489849521 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f548800fd00 con 0x564d0ca4c410 2024-09-11T21:59:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.198+0000 7f5475ffb640 1 -- 172.21.15.1:0/489849521 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f547c01d530 con 0x564d0ca4c410 2024-09-11T21:59:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.198+0000 7f5497037740 10 client.15867 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T21:59:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.198+0000 7f5497037740 10 client.15867 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T21:59:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.198+0000 7f5497037740 20 client.15867 populate_metadata read hostname 'smithi001' 2024-09-11T21:59:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.198+0000 7f5497037740 1 --2- 172.21.15.1:0/489849521 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x564d0cad20a0 0x564d0cad44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.198+0000 7f5497037740 4 client.15867 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T21:59:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.198+0000 7f5497037740 1 -- 172.21.15.1:0/489849521 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564d0c9b7d70 con 0x564d0cad20a0 2024-09-11T21:59:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.198+0000 7f5492040640 1 --2- 172.21.15.1:0/489849521 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x564d0cad20a0 0x564d0cad44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.199+0000 7f5475ffb640 10 client.15867 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T21:59:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.199+0000 7f5492040640 1 --2- 172.21.15.1:0/489849521 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x564d0cad20a0 0x564d0cad44c0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4deaeb10bc32baca server_cookie=c2343bcd5db687f3 in_seq=0 out_seq=0 2024-09-11T21:59:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.219+0000 7f5475ffb640 1 -- 172.21.15.1:0/489849521 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564d0c9b7d70 con 0x564d0cad20a0 2024-09-11T21:59:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.219+0000 7f5475ffb640 10 client.15867 handle_command_reply: tid=0 2024-09-11T21:59:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.220+0000 7f5497037740 1 client.15867 shutdown 2024-09-11T21:59:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.220+0000 7f5497037740 1 -- 172.21.15.1:0/489849521 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d0ca4c410 msgr2=0x564d0ca5edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:59:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.220+0000 7f5497037740 1 --2- 172.21.15.1:0/489849521 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d0ca4c410 0x564d0ca5edd0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f548c10d680 tx=0x7f547c00f790 comp rx=0 tx=0).stop 2024-09-11T21:59:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.220+0000 7f5497037740 1 -- 172.21.15.1:0/489849521 shutdown_connections 2024-09-11T21:59:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.220+0000 7f5497037740 1 --2- 172.21.15.1:0/489849521 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x564d0cad20a0 0x564d0cad44c0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.220+0000 7f5497037740 1 --2- 172.21.15.1:0/489849521 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d0ca56cb0 0x564d0ca5b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.220+0000 7f5497037740 1 --2- 172.21.15.1:0/489849521 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d0ca4c410 0x564d0ca5edd0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.220+0000 7f5497037740 1 --2- 172.21.15.1:0/489849521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564d0ca4baf0 0x564d0ca5e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.220+0000 7f5475ffb640 0 client.15867 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T21:59:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.220+0000 7f5497037740 1 -- 172.21.15.1:0/489849521 >> 172.21.15.1:0/489849521 conn(0x564d0ca48060 msgr2=0x564d0ca469a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:59:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.220+0000 7f5497037740 1 -- 172.21.15.1:0/489849521 shutdown_connections 2024-09-11T21:59:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.220+0000 7f5497037740 1 -- 172.21.15.1:0/489849521 wait complete. 2024-09-11T21:59:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.220+0000 7f5497037740 20 client.15867 trim_cache size 0 max 16384 2024-09-11T21:59:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-09-11T21:59:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-09-11T21:59:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-09-11T21:59:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-09-11T21:59:08.224 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.221+0000 7f54942a4640 1 -- 172.21.15.1:0/3187459119 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f54780a2e40 msgr2=0x7f54780a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:59:08.224 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.221+0000 7f54942a4640 1 --2- 172.21.15.1:0/3187459119 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f54780a2e40 0x7f54780a5300 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5488009fc0 tx=0x7f5488005f90 comp rx=0 tx=0).stop 2024-09-11T21:59:08.224 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.221+0000 7f54942a4640 1 -- 172.21.15.1:0/3187459119 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f548c19eee0 msgr2=0x7f548c19f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:59:08.224 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.221+0000 7f54942a4640 1 --2- 172.21.15.1:0/3187459119 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f548c19eee0 0x7f548c19f390 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f5484005e30 tx=0x7f5484003d00 comp rx=0 tx=0).stop 2024-09-11T21:59:08.224 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.221+0000 7f54942a4640 1 -- 172.21.15.1:0/3187459119 shutdown_connections 2024-09-11T21:59:08.224 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.221+0000 7f54942a4640 1 --2- 172.21.15.1:0/3187459119 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f54780a2e40 0x7f54780a5300 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:08.224 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.221+0000 7f54942a4640 1 --2- 172.21.15.1:0/3187459119 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f548c19eee0 0x7f548c19f390 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:08.224 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.221+0000 7f54942a4640 1 --2- 172.21.15.1:0/3187459119 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f548c106e60 0x7f548c19ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:08.225 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.221+0000 7f54942a4640 1 --2- 172.21.15.1:0/3187459119 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f548c101920 0x7f548c19a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:08.225 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.221+0000 7f54942a4640 1 -- 172.21.15.1:0/3187459119 >> 172.21.15.1:0/3187459119 conn(0x7f548c075eb0 msgr2=0x7f548c10bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:59:08.225 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.221+0000 7f54942a4640 1 -- 172.21.15.1:0/3187459119 shutdown_connections 2024-09-11T21:59:08.225 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:08.221+0000 7f54942a4640 1 -- 172.21.15.1:0/3187459119 wait complete. 2024-09-11T21:59:08.889 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:59:08.890 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:59:08.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:59:08 smithi001 ceph-mon[30935]: pgmap v62: 97 pgs: 97 active+clean; 1.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 32 KiB/s wr, 25 op/s 2024-09-11T21:59:09.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:59:08 smithi053 ceph-mon[34908]: pgmap v62: 97 pgs: 97 active+clean; 1.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 32 KiB/s wr, 25 op/s 2024-09-11T21:59:09.091 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:59:08 smithi161 ceph-mon[35615]: pgmap v62: 97 pgs: 97 active+clean; 1.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 32 KiB/s wr, 25 op/s 2024-09-11T21:59:09.158 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T21:59:09.158 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T21:59:09.188 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:59:09.188 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:59:09.188 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 24 -lt 100 ']' 2024-09-11T21:59:09.188 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 24 + 1 2024-09-11T21:59:09.247 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=25 2024-09-11T21:59:09.247 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T21:59:09.274 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:59:09.274 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:59:09.275 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T21:59:09.275 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T21:59:09.275 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T21:59:10.481 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:59:10.481 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:59:10.973 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T21:59:10.973 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T21:59:11.012 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:59:11.012 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:59:11.012 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 25 -lt 100 ']' 2024-09-11T21:59:11.012 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 25 + 1 2024-09-11T21:59:11.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:59:10 smithi053 ceph-mon[34908]: pgmap v63: 97 pgs: 97 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 45 KiB/s wr, 36 op/s 2024-09-11T21:59:11.091 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:59:10 smithi161 ceph-mon[35615]: pgmap v63: 97 pgs: 97 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 45 KiB/s wr, 36 op/s 2024-09-11T21:59:11.125 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=26 2024-09-11T21:59:11.125 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T21:59:11.171 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:59:11.171 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:59:11.173 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T21:59:11.173 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T21:59:11.173 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T21:59:11.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:59:10 smithi001 ceph-mon[30935]: pgmap v63: 97 pgs: 97 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 45 KiB/s wr, 36 op/s 2024-09-11T21:59:12.231 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:59:12.231 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:59:12.494 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 13.4 2024-09-11T21:59:12.494 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T21:59:12.514 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T21:59:12.514 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T21:59:12.529 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:59:12.529 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:59:12.529 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 26 -lt 100 ']' 2024-09-11T21:59:12.530 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 26 + 1 2024-09-11T21:59:12.605 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=27 2024-09-11T21:59:12.605 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T21:59:12.722 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:59:12.722 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:59:12.723 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T21:59:12.724 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T21:59:12.724 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T21:59:13.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:59:12 smithi053 ceph-mon[34908]: pgmap v64: 97 pgs: 97 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 40 KiB/s wr, 31 op/s 2024-09-11T21:59:13.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:59:12 smithi161 ceph-mon[35615]: pgmap v64: 97 pgs: 97 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 40 KiB/s wr, 31 op/s 2024-09-11T21:59:13.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:59:12 smithi001 ceph-mon[30935]: pgmap v64: 97 pgs: 97 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 40 KiB/s wr, 31 op/s 2024-09-11T21:59:13.739 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:59:13.739 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:59:14.014 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T21:59:14.014 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T21:59:14.038 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:59:14.038 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:59:14.038 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 27 -lt 100 ']' 2024-09-11T21:59:14.038 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 27 + 1 2024-09-11T21:59:14.093 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=28 2024-09-11T21:59:14.093 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T21:59:14.172 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:59:14.172 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:59:14.174 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T21:59:14.174 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T21:59:14.174 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T21:59:15.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:59:14 smithi053 ceph-mon[34908]: pgmap v65: 97 pgs: 97 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 40 KiB/s wr, 31 op/s 2024-09-11T21:59:15.091 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:59:14 smithi161 ceph-mon[35615]: pgmap v65: 97 pgs: 97 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 40 KiB/s wr, 31 op/s 2024-09-11T21:59:15.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:59:14 smithi001 ceph-mon[30935]: pgmap v65: 97 pgs: 97 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 40 KiB/s wr, 31 op/s 2024-09-11T21:59:15.356 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:59:15.356 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:59:15.797 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T21:59:15.798 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T21:59:15.812 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:59:15.812 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:59:15.812 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 28 -lt 100 ']' 2024-09-11T21:59:15.812 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 28 + 1 2024-09-11T21:59:15.859 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=29 2024-09-11T21:59:15.859 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T21:59:15.896 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:59:15.896 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:59:15.897 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T21:59:15.897 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T21:59:15.897 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T21:59:16.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:59:16 smithi001 ceph-mon[30935]: pgmap v66: 97 pgs: 97 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 42 KiB/s wr, 36 op/s 2024-09-11T21:59:17.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:59:16 smithi053 ceph-mon[34908]: pgmap v66: 97 pgs: 97 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 42 KiB/s wr, 36 op/s 2024-09-11T21:59:17.091 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:59:16 smithi161 ceph-mon[35615]: pgmap v66: 97 pgs: 97 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 42 KiB/s wr, 36 op/s 2024-09-11T21:59:17.163 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:59:17.163 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:59:17.353 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T21:59:17.353 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T21:59:17.370 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:59:17.370 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:59:17.370 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 29 -lt 100 ']' 2024-09-11T21:59:17.370 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 29 + 1 2024-09-11T21:59:17.416 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=30 2024-09-11T21:59:17.417 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T21:59:17.447 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:59:17.447 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:59:17.448 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T21:59:17.448 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T21:59:17.448 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T21:59:18.330 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:59:18.330 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:59:18.614 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T21:59:18.614 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T21:59:18.628 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:59:18.629 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:59:18.629 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 30 -lt 100 ']' 2024-09-11T21:59:18.629 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 30 + 1 2024-09-11T21:59:18.712 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=31 2024-09-11T21:59:18.712 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T21:59:18.781 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:59:18.781 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:59:18.782 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T21:59:18.782 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T21:59:18.782 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T21:59:19.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:59:18 smithi053 ceph-mon[34908]: pgmap v67: 97 pgs: 97 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-09-11T21:59:19.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:59:18 smithi161 ceph-mon[35615]: pgmap v67: 97 pgs: 97 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-09-11T21:59:19.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:59:18 smithi001 ceph-mon[30935]: pgmap v67: 97 pgs: 97 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-09-11T21:59:19.921 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:59:19.921 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:59:20.388 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T21:59:20.388 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T21:59:20.403 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:59:20.403 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:59:20.403 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 31 -lt 100 ']' 2024-09-11T21:59:20.403 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 31 + 1 2024-09-11T21:59:20.461 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=32 2024-09-11T21:59:20.461 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T21:59:20.520 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:59:20.520 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:59:20.521 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T21:59:20.521 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T21:59:20.521 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T21:59:21.026 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:59:21.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:59:20 smithi053 ceph-mon[34908]: pgmap v68: 97 pgs: 97 active+clean; 1.9 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-09-11T21:59:21.316 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:59:20 smithi001 ceph-mon[30935]: pgmap v68: 97 pgs: 97 active+clean; 1.9 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-09-11T21:59:21.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:59:20 smithi161 ceph-mon[35615]: pgmap v68: 97 pgs: 97 active+clean; 1.9 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-09-11T21:59:21.712 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:59:21.713 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:59:21.914 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T21:59:21.914 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T21:59:21.994 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:59:21.994 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:59:21.994 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 32 -lt 100 ']' 2024-09-11T21:59:21.995 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 32 + 1 2024-09-11T21:59:22.141 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=33 2024-09-11T21:59:22.142 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T21:59:22.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:59:21 smithi001 ceph-mon[30935]: pgmap v69: 97 pgs: 97 active+clean; 1.9 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.2 KiB/s wr, 10 op/s 2024-09-11T21:59:22.204 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:59:22.204 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:59:22.205 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T21:59:22.205 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T21:59:22.205 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T21:59:22.259 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:59:22.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:59:21 smithi053 ceph-mon[34908]: pgmap v69: 97 pgs: 97 active+clean; 1.9 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.2 KiB/s wr, 10 op/s 2024-09-11T21:59:22.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:59:21 smithi161 ceph-mon[35615]: pgmap v69: 97 pgs: 97 active+clean; 1.9 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.2 KiB/s wr, 10 op/s 2024-09-11T21:59:23.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:59:22 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:59:23.328 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:59:22 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:59:23.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:59:22 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T21:59:24.113 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:59:24.113 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:59:24.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:59:24 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:59:24.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:59:24 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:59:24.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:59:24 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:59:24.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:59:24 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:59:24.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:59:24 smithi001 ceph-mon[30935]: pgmap v70: 97 pgs: 97 active+clean; 1.9 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.2 KiB/s wr, 10 op/s 2024-09-11T21:59:24.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:59:24 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:59:24.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:59:24 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:59:24.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:59:24 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:59:24.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:59:24 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:59:24.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:59:24 smithi053 ceph-mon[34908]: pgmap v70: 97 pgs: 97 active+clean; 1.9 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.2 KiB/s wr, 10 op/s 2024-09-11T21:59:24.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:59:24 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:59:24.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:59:24 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:59:24.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:59:24 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:59:24.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:59:24 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:59:24.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:59:24 smithi161 ceph-mon[35615]: pgmap v70: 97 pgs: 97 active+clean; 1.9 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.2 KiB/s wr, 10 op/s 2024-09-11T21:59:24.644 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T21:59:24.644 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T21:59:24.677 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:59:24.677 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:59:24.677 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 33 -lt 100 ']' 2024-09-11T21:59:24.677 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 33 + 1 2024-09-11T21:59:24.710 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=34 2024-09-11T21:59:24.710 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T21:59:24.785 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:59:24.785 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:59:24.786 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T21:59:24.786 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T21:59:24.786 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T21:59:26.262 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:59:26.262 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:59:26.677 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T21:59:26.677 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T21:59:26.743 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:59:26.743 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:59:26.744 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 34 -lt 100 ']' 2024-09-11T21:59:26.744 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 34 + 1 2024-09-11T21:59:26.817 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=35 2024-09-11T21:59:26.817 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T21:59:26.854 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:59:26.854 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:59:26.855 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T21:59:26.855 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T21:59:26.855 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T21:59:27.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:59:26 smithi053 ceph-mon[34908]: pgmap v71: 97 pgs: 97 active+clean; 2.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-09-11T21:59:27.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:59:26 smithi161 ceph-mon[35615]: pgmap v71: 97 pgs: 97 active+clean; 2.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-09-11T21:59:27.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:59:26 smithi001 ceph-mon[30935]: pgmap v71: 97 pgs: 97 active+clean; 2.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-09-11T21:59:27.508 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.507+0000 7f20dacf1640 1 -- 172.21.15.1:0/3788250819 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f20d41011c0 msgr2=0x7f20d4104da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:59:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.507+0000 7f20dacf1640 1 --2- 172.21.15.1:0/3788250819 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f20d41011c0 0x7f20d4104da0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f20c000aa10 tx=0x7f20c002b720 comp rx=0 tx=0).stop 2024-09-11T21:59:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.507+0000 7f20dacf1640 1 -- 172.21.15.1:0/3788250819 shutdown_connections 2024-09-11T21:59:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.507+0000 7f20dacf1640 1 --2- 172.21.15.1:0/3788250819 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20d4105500 0x7f20d410d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.507+0000 7f20dacf1640 1 --2- 172.21.15.1:0/3788250819 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f20d41011c0 0x7f20d4104da0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.507+0000 7f20dacf1640 1 --2- 172.21.15.1:0/3788250819 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20d4100810 0x7f20d4100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.507+0000 7f20dacf1640 1 -- 172.21.15.1:0/3788250819 >> 172.21.15.1:0/3788250819 conn(0x7f20d4075eb0 msgr2=0x7f20d40762c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:59:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.507+0000 7f20dacf1640 1 -- 172.21.15.1:0/3788250819 shutdown_connections 2024-09-11T21:59:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.507+0000 7f20dacf1640 1 -- 172.21.15.1:0/3788250819 wait complete. 2024-09-11T21:59:27.522 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.520+0000 7f20dacf1640 1 Processor -- start 2024-09-11T21:59:27.522 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.520+0000 7f20dacf1640 1 -- start start 2024-09-11T21:59:27.522 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.520+0000 7f20dacf1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f20d4100810 0x7f20d4198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:27.523 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.520+0000 7f20dacf1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20d4105500 0x7f20d41989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:27.523 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.520+0000 7f20dacf1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20d419cd30 0x7f20d419d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:27.523 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.520+0000 7f20dacf1640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f20d410fd30 con 0x7f20d4100810 2024-09-11T21:59:27.523 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.520+0000 7f20dacf1640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f20d410fbb0 con 0x7f20d4105500 2024-09-11T21:59:27.523 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.520+0000 7f20dacf1640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f20d410feb0 con 0x7f20d419cd30 2024-09-11T21:59:27.523 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.520+0000 7f20d8a8d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f20d4100810 0x7f20d4198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:27.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.520+0000 7f20d8a8d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f20d4100810 0x7f20d4198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42166/0 (socket says 172.21.15.1:42166) 2024-09-11T21:59:27.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.520+0000 7f20d8a8d640 1 -- 172.21.15.1:0/426057544 learned_addr learned my addr 172.21.15.1:0/426057544 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:59:27.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.520+0000 7f20d3fff640 1 --2- 172.21.15.1:0/426057544 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20d4105500 0x7f20d41989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:27.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.520+0000 7f20d928e640 1 --2- 172.21.15.1:0/426057544 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20d419cd30 0x7f20d419d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:27.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.520+0000 7f20d8a8d640 1 -- 172.21.15.1:0/426057544 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20d419cd30 msgr2=0x7f20d419d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:59:27.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.520+0000 7f20d8a8d640 1 --2- 172.21.15.1:0/426057544 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20d419cd30 0x7f20d419d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:27.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.520+0000 7f20d8a8d640 1 -- 172.21.15.1:0/426057544 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20d4105500 msgr2=0x7f20d41989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:59:27.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.520+0000 7f20d8a8d640 1 --2- 172.21.15.1:0/426057544 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20d4105500 0x7f20d41989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:27.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.520+0000 7f20d8a8d640 1 -- 172.21.15.1:0/426057544 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20d419d820 con 0x7f20d4100810 2024-09-11T21:59:27.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.521+0000 7f20d8a8d640 1 --2- 172.21.15.1:0/426057544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f20d4100810 0x7f20d4198470 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f20c800d770 tx=0x7f20c800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:59:27.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.521+0000 7f20d1ffb640 1 -- 172.21.15.1:0/426057544 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c8017070 con 0x7f20d4100810 2024-09-11T21:59:27.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.521+0000 7f20d1ffb640 1 -- 172.21.15.1:0/426057544 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20c8005500 con 0x7f20d4100810 2024-09-11T21:59:27.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.521+0000 7f20dacf1640 1 -- 172.21.15.1:0/426057544 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20d419db10 con 0x7f20d4100810 2024-09-11T21:59:27.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.521+0000 7f20d1ffb640 1 -- 172.21.15.1:0/426057544 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c8004e00 con 0x7f20d4100810 2024-09-11T21:59:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.523+0000 7f20d1ffb640 1 -- 172.21.15.1:0/426057544 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f20c8004fa0 con 0x7f20d4100810 2024-09-11T21:59:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.524+0000 7f20d1ffb640 1 --2- 172.21.15.1:0/426057544 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f20c40a2e40 0x7f20c40a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.524+0000 7f20d3fff640 1 --2- 172.21.15.1:0/426057544 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f20c40a2e40 0x7f20c40a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.525+0000 7f20d3fff640 1 --2- 172.21.15.1:0/426057544 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f20c40a2e40 0x7f20c40a5300 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f20c002bd10 tx=0x7f20c0000bc0 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:59:27.537 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.537+0000 7f20dacf1640 1 -- 172.21.15.1:0/426057544 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20d41a5270 con 0x7f20d4100810 2024-09-11T21:59:27.537 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.537+0000 7f20d1ffb640 1 -- 172.21.15.1:0/426057544 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f20c8098840 con 0x7f20d4100810 2024-09-11T21:59:27.842 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.839+0000 7f20dda8f740 1 Processor -- start 2024-09-11T21:59:27.842 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.839+0000 7f20dda8f740 1 -- start start 2024-09-11T21:59:27.842 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.840+0000 7f20dda8f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d768af3990 0x55d768af3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:27.842 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.840+0000 7f20dda8f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d768af42b0 0x55d768b02b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:27.842 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.840+0000 7f20dda8f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d768b030c0 0x55d768b054b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:27.842 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.840+0000 7f20dda8f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55d768a582f0 con 0x55d768af3990 2024-09-11T21:59:27.843 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.840+0000 7f20dda8f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55d768a00270 con 0x55d768b030c0 2024-09-11T21:59:27.843 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.840+0000 7f20dda8f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d768a542b0 con 0x55d768af42b0 2024-09-11T21:59:27.843 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.840+0000 7f20d8a8d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d768af3990 0x55d768af3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:27.843 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.840+0000 7f20d8a8d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d768af3990 0x55d768af3d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42174/0 (socket says 172.21.15.1:42174) 2024-09-11T21:59:27.843 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.840+0000 7f20d8a8d640 1 -- 172.21.15.1:0/3733810241 learned_addr learned my addr 172.21.15.1:0/3733810241 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:59:27.843 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.840+0000 7f20d928e640 1 --2- 172.21.15.1:0/3733810241 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d768b030c0 0x55d768b054b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:27.843 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.840+0000 7f20d3fff640 1 --2- 172.21.15.1:0/3733810241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d768af42b0 0x55d768b02b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:27.843 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.840+0000 7f20d8a8d640 1 -- 172.21.15.1:0/3733810241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d768af42b0 msgr2=0x55d768b02b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:59:27.843 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.840+0000 7f20d8a8d640 1 --2- 172.21.15.1:0/3733810241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d768af42b0 0x55d768b02b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:27.843 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.840+0000 7f20d8a8d640 1 -- 172.21.15.1:0/3733810241 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d768b030c0 msgr2=0x55d768b054b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:59:27.843 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.840+0000 7f20d8a8d640 1 --2- 172.21.15.1:0/3733810241 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d768b030c0 0x55d768b054b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:27.843 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.840+0000 7f20d8a8d640 1 -- 172.21.15.1:0/3733810241 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7689ff340 con 0x55d768af3990 2024-09-11T21:59:27.843 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.840+0000 7f20d8a8d640 1 --2- 172.21.15.1:0/3733810241 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d768af3990 0x55d768af3d70 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f20c8009610 tx=0x7f20c8012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dd1dabe619a0cfd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:59:27.844 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.841+0000 7f20beffd640 1 -- 172.21.15.1:0/3733810241 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c809a7a0 con 0x55d768af3990 2024-09-11T21:59:27.844 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.841+0000 7f20beffd640 1 -- 172.21.15.1:0/3733810241 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20c809eb50 con 0x55d768af3990 2024-09-11T21:59:27.844 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.841+0000 7f20beffd640 1 -- 172.21.15.1:0/3733810241 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c80cbe50 con 0x55d768af3990 2024-09-11T21:59:27.844 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.841+0000 7f20dda8f740 1 -- 172.21.15.1:0/3733810241 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d768af3990 msgr2=0x55d768af3d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:59:27.844 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.841+0000 7f20dda8f740 1 --2- 172.21.15.1:0/3733810241 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d768af3990 0x55d768af3d70 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f20c8009610 tx=0x7f20c8012000 comp rx=0 tx=0).stop 2024-09-11T21:59:27.844 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.841+0000 7f20dda8f740 1 -- 172.21.15.1:0/3733810241 shutdown_connections 2024-09-11T21:59:27.844 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.841+0000 7f20dda8f740 1 --2- 172.21.15.1:0/3733810241 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d768b030c0 0x55d768b054b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:27.844 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.841+0000 7f20dda8f740 1 --2- 172.21.15.1:0/3733810241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d768af42b0 0x55d768b02b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:27.844 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.841+0000 7f20dda8f740 1 --2- 172.21.15.1:0/3733810241 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d768af3990 0x55d768af3d70 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:27.844 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.841+0000 7f20dda8f740 1 -- 172.21.15.1:0/3733810241 >> 172.21.15.1:0/3733810241 conn(0x55d768af0060 msgr2=0x55d768af0440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:59:27.844 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.841+0000 7f20dda8f740 1 -- 172.21.15.1:0/3733810241 shutdown_connections 2024-09-11T21:59:27.844 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.841+0000 7f20dda8f740 1 -- 172.21.15.1:0/3733810241 wait complete. 2024-09-11T21:59:27.884 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.883+0000 7f20dda8f740 1 -- start start 2024-09-11T21:59:27.885 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.885+0000 7f20dda8f740 1 Processor -- start 2024-09-11T21:59:27.887 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.885+0000 7f20dda8f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d768af3990 0x55d768b000e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:27.887 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.885+0000 7f20dda8f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d768af42b0 0x55d768b00620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:27.887 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.885+0000 7f20dda8f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d768b030c0 0x55d768b00b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:27.887 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.885+0000 7f20dda8f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55d768b05cf0 con 0x55d768af42b0 2024-09-11T21:59:27.888 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.885+0000 7f20dda8f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55d7689e1290 con 0x55d768b030c0 2024-09-11T21:59:27.888 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.885+0000 7f20dda8f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d768b05e70 con 0x55d768af3990 2024-09-11T21:59:27.888 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.885+0000 7f20d3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d768af42b0 0x55d768b00620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:27.888 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.886+0000 7f20d3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d768af42b0 0x55d768b00620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42190/0 (socket says 172.21.15.1:42190) 2024-09-11T21:59:27.888 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.886+0000 7f20d3fff640 1 -- 172.21.15.1:0/1834511790 learned_addr learned my addr 172.21.15.1:0/1834511790 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:59:27.888 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.886+0000 7f20d8a8d640 1 --2- 172.21.15.1:0/1834511790 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d768af3990 0x55d768b000e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:27.888 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.886+0000 7f20d928e640 1 --2- 172.21.15.1:0/1834511790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d768b030c0 0x55d768b00b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:27.888 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.886+0000 7f20d3fff640 1 -- 172.21.15.1:0/1834511790 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d768af3990 msgr2=0x55d768b000e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:59:27.888 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.886+0000 7f20d3fff640 1 --2- 172.21.15.1:0/1834511790 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d768af3990 0x55d768b000e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:27.889 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.886+0000 7f20d3fff640 1 -- 172.21.15.1:0/1834511790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d768b030c0 msgr2=0x55d768b00b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:59:27.889 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.886+0000 7f20d3fff640 1 --2- 172.21.15.1:0/1834511790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d768b030c0 0x55d768b00b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:27.889 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.886+0000 7f20d3fff640 1 -- 172.21.15.1:0/1834511790 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d76876b210 con 0x55d768af42b0 2024-09-11T21:59:27.889 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.886+0000 7f20d928e640 1 --2- 172.21.15.1:0/1834511790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d768b030c0 0x55d768b00b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:59:27.889 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.886+0000 7f20d3fff640 1 --2- 172.21.15.1:0/1834511790 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d768af42b0 0x55d768b00620 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f20c000aa10 tx=0x7f20c000a370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=974a08e91df83514 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:59:27.889 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.886+0000 7f20b3fff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T21:59:27.889 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.886+0000 7f20b3fff640 1 -- 172.21.15.1:0/1834511790 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c004e030 con 0x55d768af42b0 2024-09-11T21:59:27.889 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.886+0000 7f20b3fff640 1 -- 172.21.15.1:0/1834511790 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f20c0044040 con 0x55d768af42b0 2024-09-11T21:59:27.889 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.886+0000 7f20dda8f740 10 client.0 _ll_register_callbacks cb 0x55d768a12140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:59:27.890 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.888+0000 7f20dda8f740 1 -- 172.21.15.1:0/1834511790 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20c809eb50 con 0x55d768af42b0 2024-09-11T21:59:27.890 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.888+0000 7f20b3fff640 1 -- 172.21.15.1:0/1834511790 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f20c0036040 con 0x55d768af42b0 2024-09-11T21:59:27.890 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.888+0000 7f20dda8f740 10 client.15879 fetch_fsmap learned FSMap version 28 2024-09-11T21:59:27.890 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.888+0000 7f20dda8f740 1 -- 172.21.15.1:0/1834511790 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({fsmap=28}) -- 0x55d768837620 con 0x55d768af42b0 2024-09-11T21:59:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.896+0000 7f20b3fff640 1 -- 172.21.15.1:0/1834511790 <== mon.0 v2:172.21.15.1:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f20c000b040 con 0x55d768af42b0 2024-09-11T21:59:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.896+0000 7f20dda8f740 10 client.15879 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T21:59:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.896+0000 7f20dda8f740 10 client.15879 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T21:59:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.896+0000 7f20dda8f740 20 client.15879 populate_metadata read hostname 'smithi001' 2024-09-11T21:59:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.896+0000 7f20dda8f740 1 --2- 172.21.15.1:0/1834511790 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55d768b12f90 0x55d768b15450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.896+0000 7f20dda8f740 4 client.15879 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T21:59:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.896+0000 7f20dda8f740 1 -- 172.21.15.1:0/1834511790 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d768a65a20 con 0x55d768b12f90 2024-09-11T21:59:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.897+0000 7f20d8a8d640 1 --2- 172.21.15.1:0/1834511790 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55d768b12f90 0x55d768b15450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:27.898 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.897+0000 7f20d8a8d640 1 --2- 172.21.15.1:0/1834511790 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55d768b12f90 0x55d768b15450 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4ee8df83f9da80e server_cookie=8c53a6c8de896c96 in_seq=0 out_seq=0 2024-09-11T21:59:27.898 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.897+0000 7f20b3fff640 10 client.15879 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T21:59:27.899 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.898+0000 7f20b3fff640 1 -- 172.21.15.1:0/1834511790 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d768a65a20 con 0x55d768b12f90 2024-09-11T21:59:27.899 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.898+0000 7f20b3fff640 10 client.15879 handle_command_reply: tid=0 2024-09-11T21:59:27.899 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.898+0000 7f20dda8f740 1 client.15879 shutdown 2024-09-11T21:59:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.898+0000 7f20dda8f740 1 -- 172.21.15.1:0/1834511790 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d768af42b0 msgr2=0x55d768b00620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:59:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.898+0000 7f20dda8f740 1 --2- 172.21.15.1:0/1834511790 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d768af42b0 0x55d768b00620 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f20c000aa10 tx=0x7f20c000a370 comp rx=0 tx=0).stop 2024-09-11T21:59:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.898+0000 7f20dda8f740 1 -- 172.21.15.1:0/1834511790 shutdown_connections 2024-09-11T21:59:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.898+0000 7f20dda8f740 1 --2- 172.21.15.1:0/1834511790 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55d768b12f90 0x55d768b15450 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.898+0000 7f20dda8f740 1 --2- 172.21.15.1:0/1834511790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d768b030c0 0x55d768b00b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.898+0000 7f20dda8f740 1 --2- 172.21.15.1:0/1834511790 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d768af42b0 0x55d768b00620 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.898+0000 7f20dda8f740 1 --2- 172.21.15.1:0/1834511790 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d768af3990 0x55d768b000e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.898+0000 7f20b3fff640 0 client.15879 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T21:59:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.898+0000 7f20dda8f740 1 -- 172.21.15.1:0/1834511790 >> 172.21.15.1:0/1834511790 conn(0x55d768af0060 msgr2=0x55d768af5dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:59:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.899+0000 7f20dda8f740 1 -- 172.21.15.1:0/1834511790 shutdown_connections 2024-09-11T21:59:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.899+0000 7f20dda8f740 1 -- 172.21.15.1:0/1834511790 wait complete. 2024-09-11T21:59:27.901 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.899+0000 7f20dda8f740 20 client.15879 trim_cache size 0 max 16384 2024-09-11T21:59:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.913+0000 7f20dda8f740 1 Processor -- start 2024-09-11T21:59:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.913+0000 7f20dda8f740 1 -- start start 2024-09-11T21:59:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.913+0000 7f20dda8f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d768af3990 0x55d768b02050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.913+0000 7f20dda8f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d768af42b0 0x55d768b1ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.913+0000 7f20dda8f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d768b030c0 0x55d768b1b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.913+0000 7f20dda8f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f20d41ab7f0 con 0x55d768b030c0 2024-09-11T21:59:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.913+0000 7f20dda8f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f20d41ad330 con 0x55d768af3990 2024-09-11T21:59:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.913+0000 7f20dda8f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f20d41abec0 con 0x55d768af42b0 2024-09-11T21:59:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.914+0000 7f20d928e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d768b030c0 0x55d768b1b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.914+0000 7f20d928e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d768b030c0 0x55d768b1b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42206/0 (socket says 172.21.15.1:42206) 2024-09-11T21:59:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.914+0000 7f20d928e640 1 -- 172.21.15.1:0/3805744008 learned_addr learned my addr 172.21.15.1:0/3805744008 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:59:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.914+0000 7f20d8a8d640 1 --2- 172.21.15.1:0/3805744008 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d768af3990 0x55d768b02050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.914+0000 7f20d928e640 1 -- 172.21.15.1:0/3805744008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d768af42b0 msgr2=0x55d768b1ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:59:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.914+0000 7f20d3fff640 1 --2- 172.21.15.1:0/3805744008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d768af42b0 0x55d768b1ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.914+0000 7f20d928e640 1 --2- 172.21.15.1:0/3805744008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d768af42b0 0x55d768b1ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.914+0000 7f20d928e640 1 -- 172.21.15.1:0/3805744008 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d768af3990 msgr2=0x55d768b02050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:59:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.914+0000 7f20d928e640 1 --2- 172.21.15.1:0/3805744008 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d768af3990 0x55d768b02050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.914+0000 7f20d928e640 1 -- 172.21.15.1:0/3805744008 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d768afe610 con 0x55d768b030c0 2024-09-11T21:59:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.914+0000 7f20d8a8d640 1 --2- 172.21.15.1:0/3805744008 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d768af3990 0x55d768b02050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T21:59:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.914+0000 7f20d3fff640 1 --2- 172.21.15.1:0/3805744008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d768af42b0 0x55d768b1ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T21:59:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.914+0000 7f20d928e640 1 --2- 172.21.15.1:0/3805744008 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d768b030c0 0x55d768b1b260 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f20cc0124d0 tx=0x7f20cc012990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dd91153f49ec79d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:59:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.914+0000 7f20be7fc640 1 -- 172.21.15.1:0/3805744008 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20cc020070 con 0x55d768b030c0 2024-09-11T21:59:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.914+0000 7f20be7fc640 1 -- 172.21.15.1:0/3805744008 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20cc0109d0 con 0x55d768b030c0 2024-09-11T21:59:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.914+0000 7f20be7fc640 1 -- 172.21.15.1:0/3805744008 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20cc01bb30 con 0x55d768b030c0 2024-09-11T21:59:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.915+0000 7f20dda8f740 1 -- 172.21.15.1:0/3805744008 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d768b030c0 msgr2=0x55d768b1b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:59:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.915+0000 7f20dda8f740 1 --2- 172.21.15.1:0/3805744008 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d768b030c0 0x55d768b1b260 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f20cc0124d0 tx=0x7f20cc012990 comp rx=0 tx=0).stop 2024-09-11T21:59:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.915+0000 7f20dda8f740 1 -- 172.21.15.1:0/3805744008 shutdown_connections 2024-09-11T21:59:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.915+0000 7f20dda8f740 1 --2- 172.21.15.1:0/3805744008 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d768b030c0 0x55d768b1b260 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.915+0000 7f20dda8f740 1 --2- 172.21.15.1:0/3805744008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d768af42b0 0x55d768b1ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.915+0000 7f20dda8f740 1 --2- 172.21.15.1:0/3805744008 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d768af3990 0x55d768b02050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.915+0000 7f20dda8f740 1 -- 172.21.15.1:0/3805744008 >> 172.21.15.1:0/3805744008 conn(0x55d768af0060 msgr2=0x55d768b01940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:59:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.915+0000 7f20dda8f740 1 -- 172.21.15.1:0/3805744008 shutdown_connections 2024-09-11T21:59:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.915+0000 7f20dda8f740 1 -- 172.21.15.1:0/3805744008 wait complete. 2024-09-11T21:59:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.915+0000 7f20dda8f740 1 -- start start 2024-09-11T21:59:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.915+0000 7f20dda8f740 1 Processor -- start 2024-09-11T21:59:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.916+0000 7f20dda8f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d768af3990 0x55d768b06d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.916+0000 7f20dda8f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d768af42b0 0x55d768b072b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.916+0000 7f20dda8f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d768b030c0 0x55d768b077f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.916+0000 7f20dda8f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f20d41acc60 con 0x55d768af3990 2024-09-11T21:59:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.916+0000 7f20dda8f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f20d41ac590 con 0x55d768af42b0 2024-09-11T21:59:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.916+0000 7f20dda8f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f20d41a97d0 con 0x55d768b030c0 2024-09-11T21:59:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.916+0000 7f20dda8f740 10 client.0 _ll_register_callbacks cb 0x7f20c0041070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:59:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.916+0000 7f20d8a8d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d768af3990 0x55d768b06d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.916+0000 7f20d8a8d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d768af3990 0x55d768b06d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42220/0 (socket says 172.21.15.1:42220) 2024-09-11T21:59:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.916+0000 7f20d8a8d640 1 -- 172.21.15.1:0/2340416053 learned_addr learned my addr 172.21.15.1:0/2340416053 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:59:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.916+0000 7f20d3fff640 1 --2- 172.21.15.1:0/2340416053 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d768af42b0 0x55d768b072b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.916+0000 7f20d8a8d640 1 -- 172.21.15.1:0/2340416053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d768b030c0 msgr2=0x55d768b077f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:59:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.916+0000 7f20d928e640 1 --2- 172.21.15.1:0/2340416053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d768b030c0 0x55d768b077f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.916+0000 7f20d8a8d640 1 --2- 172.21.15.1:0/2340416053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d768b030c0 0x55d768b077f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.916+0000 7f20d8a8d640 1 -- 172.21.15.1:0/2340416053 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d768af42b0 msgr2=0x55d768b072b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:59:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.916+0000 7f20d8a8d640 1 --2- 172.21.15.1:0/2340416053 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d768af42b0 0x55d768b072b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.916+0000 7f20d8a8d640 1 -- 172.21.15.1:0/2340416053 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d768b7ccf0 con 0x55d768af3990 2024-09-11T21:59:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.916+0000 7f20d3fff640 1 --2- 172.21.15.1:0/2340416053 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d768af42b0 0x55d768b072b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:59:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.916+0000 7f20d928e640 1 --2- 172.21.15.1:0/2340416053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d768b030c0 0x55d768b077f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T21:59:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.916+0000 7f20d8a8d640 1 --2- 172.21.15.1:0/2340416053 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d768af3990 0x55d768b06d70 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f20c800d740 tx=0x7f20c801d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e212231121338ff1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:59:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.916+0000 7f20b3fff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T21:59:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.917+0000 7f20b3fff640 1 -- 172.21.15.1:0/2340416053 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c80a8060 con 0x55d768af3990 2024-09-11T21:59:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.917+0000 7f20b3fff640 1 -- 172.21.15.1:0/2340416053 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f20c8009cc0 con 0x55d768af3990 2024-09-11T21:59:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.917+0000 7f20dda8f740 1 -- 172.21.15.1:0/2340416053 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20cc0109d0 con 0x55d768af3990 2024-09-11T21:59:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.917+0000 7f20b3fff640 1 -- 172.21.15.1:0/2340416053 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f20c807e050 con 0x55d768af3990 2024-09-11T21:59:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.917+0000 7f20dda8f740 10 client.15885 fetch_fsmap learned FSMap version 28 2024-09-11T21:59:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.917+0000 7f20dda8f740 1 -- 172.21.15.1:0/2340416053 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d768b7ded0 con 0x55d768af3990 2024-09-11T21:59:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.917+0000 7f20b3fff640 1 -- 172.21.15.1:0/2340416053 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20cc0109d0 con 0x55d768af3990 2024-09-11T21:59:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.917+0000 7f20b3fff640 1 -- 172.21.15.1:0/2340416053 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f20c809bad0 con 0x55d768af3990 2024-09-11T21:59:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.917+0000 7f20dda8f740 10 client.15885 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T21:59:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.917+0000 7f20dda8f740 10 client.15885 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T21:59:27.926 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.917+0000 7f20dda8f740 20 client.15885 populate_metadata read hostname 'smithi001' 2024-09-11T21:59:27.926 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.917+0000 7f20dda8f740 1 --2- 172.21.15.1:0/2340416053 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55d768b7e210 0x55d768b80600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:27.926 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.917+0000 7f20dda8f740 4 client.15885 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T21:59:27.926 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.917+0000 7f20dda8f740 1 -- 172.21.15.1:0/2340416053 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d768a5fd70 con 0x55d768b7e210 2024-09-11T21:59:27.926 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.917+0000 7f20d3fff640 1 --2- 172.21.15.1:0/2340416053 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55d768b7e210 0x55d768b80600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:27.926 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.918+0000 7f20d3fff640 1 --2- 172.21.15.1:0/2340416053 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55d768b7e210 0x55d768b80600 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f20954debd2d699 server_cookie=7d316385f6361f65 in_seq=0 out_seq=0 2024-09-11T21:59:27.926 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.918+0000 7f20b3fff640 10 client.15885 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T21:59:27.938 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.937+0000 7f20b3fff640 1 -- 172.21.15.1:0/2340416053 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d768a5fd70 con 0x55d768b7e210 2024-09-11T21:59:27.938 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.937+0000 7f20b3fff640 10 client.15885 handle_command_reply: tid=0 2024-09-11T21:59:27.938 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.937+0000 7f20dda8f740 1 client.15885 shutdown 2024-09-11T21:59:27.938 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.937+0000 7f20dda8f740 1 -- 172.21.15.1:0/2340416053 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d768af3990 msgr2=0x55d768b06d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:59:27.938 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.937+0000 7f20dda8f740 1 --2- 172.21.15.1:0/2340416053 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d768af3990 0x55d768b06d70 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f20c800d740 tx=0x7f20c801d040 comp rx=0 tx=0).stop 2024-09-11T21:59:27.939 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.937+0000 7f20dda8f740 1 -- 172.21.15.1:0/2340416053 shutdown_connections 2024-09-11T21:59:27.939 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.937+0000 7f20dda8f740 1 --2- 172.21.15.1:0/2340416053 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55d768b7e210 0x55d768b80600 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:27.939 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.937+0000 7f20dda8f740 1 --2- 172.21.15.1:0/2340416053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d768b030c0 0x55d768b077f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:27.939 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.937+0000 7f20dda8f740 1 --2- 172.21.15.1:0/2340416053 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d768af42b0 0x55d768b072b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:27.939 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.937+0000 7f20b3fff640 0 client.15885 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T21:59:27.939 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.937+0000 7f20dda8f740 1 --2- 172.21.15.1:0/2340416053 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d768af3990 0x55d768b06d70 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:27.940 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.937+0000 7f20dda8f740 1 -- 172.21.15.1:0/2340416053 >> 172.21.15.1:0/2340416053 conn(0x55d768af0060 msgr2=0x55d768aef1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:59:27.940 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.937+0000 7f20dda8f740 1 -- 172.21.15.1:0/2340416053 shutdown_connections 2024-09-11T21:59:27.940 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.937+0000 7f20dda8f740 1 -- 172.21.15.1:0/2340416053 wait complete. 2024-09-11T21:59:27.940 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.937+0000 7f20dda8f740 20 client.15885 trim_cache size 0 max 16384 2024-09-11T21:59:27.940 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-09-11T21:59:27.940 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-09-11T21:59:27.940 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-09-11T21:59:27.940 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-09-11T21:59:27.941 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.938+0000 7f20dacf1640 1 -- 172.21.15.1:0/426057544 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f20c40a2e40 msgr2=0x7f20c40a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:59:27.941 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.938+0000 7f20dacf1640 1 --2- 172.21.15.1:0/426057544 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f20c40a2e40 0x7f20c40a5300 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f20c002bd10 tx=0x7f20c0000bc0 comp rx=0 tx=0).stop 2024-09-11T21:59:27.941 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.938+0000 7f20dacf1640 1 -- 172.21.15.1:0/426057544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f20d4100810 msgr2=0x7f20d4198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:59:27.941 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.938+0000 7f20dacf1640 1 --2- 172.21.15.1:0/426057544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f20d4100810 0x7f20d4198470 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f20c800d770 tx=0x7f20c800dc30 comp rx=0 tx=0).stop 2024-09-11T21:59:27.941 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.938+0000 7f20dacf1640 1 -- 172.21.15.1:0/426057544 shutdown_connections 2024-09-11T21:59:27.942 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.938+0000 7f20dacf1640 1 --2- 172.21.15.1:0/426057544 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f20c40a2e40 0x7f20c40a5300 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:27.942 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.938+0000 7f20dacf1640 1 --2- 172.21.15.1:0/426057544 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20d419cd30 0x7f20d419d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:27.942 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.938+0000 7f20dacf1640 1 --2- 172.21.15.1:0/426057544 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20d4105500 0x7f20d41989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:27.942 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.938+0000 7f20dacf1640 1 --2- 172.21.15.1:0/426057544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f20d4100810 0x7f20d4198470 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:27.942 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.938+0000 7f20dacf1640 1 -- 172.21.15.1:0/426057544 >> 172.21.15.1:0/426057544 conn(0x7f20d4075eb0 msgr2=0x7f20d40fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:59:27.942 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.938+0000 7f20dacf1640 1 -- 172.21.15.1:0/426057544 shutdown_connections 2024-09-11T21:59:27.942 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:27.938+0000 7f20dacf1640 1 -- 172.21.15.1:0/426057544 wait complete. 2024-09-11T21:59:27.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:59:27 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:59:27.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:59:27 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:59:27.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:59:27 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:59:28.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:59:27 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:59:28.048 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:59:27 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:59:28.048 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:59:27 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:59:28.051 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:59:28.051 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:59:28.091 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:59:27 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T21:59:28.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:59:27 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T21:59:28.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:59:27 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T21:59:28.235 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T21:59:28.235 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T21:59:28.240 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:59:28.241 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:59:28.241 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 35 -lt 100 ']' 2024-09-11T21:59:28.241 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 35 + 1 2024-09-11T21:59:28.287 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=36 2024-09-11T21:59:28.287 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T21:59:28.303 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:59:28.303 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:59:28.304 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T21:59:28.304 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T21:59:28.304 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T21:59:29.032 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:59:28 smithi001 ceph-mon[30935]: pgmap v72: 97 pgs: 97 active+clean; 2.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 9 op/s 2024-09-11T21:59:29.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:59:28 smithi053 ceph-mon[34908]: pgmap v72: 97 pgs: 97 active+clean; 2.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 9 op/s 2024-09-11T21:59:29.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:59:28 smithi161 ceph-mon[35615]: pgmap v72: 97 pgs: 97 active+clean; 2.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 9 op/s 2024-09-11T21:59:29.237 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:59:29.237 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:59:29.518 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T21:59:29.518 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T21:59:29.543 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:59:29.543 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:59:29.543 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 36 -lt 100 ']' 2024-09-11T21:59:29.543 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 36 + 1 2024-09-11T21:59:29.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.566+0000 7f7ad7ba2640 1 -- 172.21.15.1:0/431584010 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ad0103740 msgr2=0x7f7ad0107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:59:29.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.566+0000 7f7ad7ba2640 1 --2- 172.21.15.1:0/431584010 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ad0103740 0x7f7ad0107b80 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0009930 tx=0x7f7ac002a420 comp rx=0 tx=0).stop 2024-09-11T21:59:29.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.566+0000 7f7ad7ba2640 1 -- 172.21.15.1:0/431584010 shutdown_connections 2024-09-11T21:59:29.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.566+0000 7f7ad7ba2640 1 --2- 172.21.15.1:0/431584010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ad0108290 0x7f7ad010ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:29.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.566+0000 7f7ad7ba2640 1 --2- 172.21.15.1:0/431584010 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ad0103740 0x7f7ad0107b80 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:29.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.566+0000 7f7ad7ba2640 1 --2- 172.21.15.1:0/431584010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ad0102d90 0x7f7ad0103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:29.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.566+0000 7f7ad7ba2640 1 -- 172.21.15.1:0/431584010 >> 172.21.15.1:0/431584010 conn(0x7f7ad00fc7d0 msgr2=0x7f7ad00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:59:29.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.567+0000 7f7ad7ba2640 1 -- 172.21.15.1:0/431584010 shutdown_connections 2024-09-11T21:59:29.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.567+0000 7f7ad7ba2640 1 -- 172.21.15.1:0/431584010 wait complete. 2024-09-11T21:59:29.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.566+0000 7f75d4e40640 1 -- 172.21.15.1:0/1909311579 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75d0102d30 msgr2=0x7f75d0103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:59:29.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.566+0000 7f75d4e40640 1 --2- 172.21.15.1:0/1909311579 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75d0102d30 0x7f75d0103110 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f75c0009930 tx=0x7f75c002a420 comp rx=0 tx=0).stop 2024-09-11T21:59:29.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.566+0000 7f75d4e40640 1 -- 172.21.15.1:0/1909311579 shutdown_connections 2024-09-11T21:59:29.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.566+0000 7f75d4e40640 1 --2- 172.21.15.1:0/1909311579 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75d01082a0 0x7f75d010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:29.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.566+0000 7f75d4e40640 1 --2- 172.21.15.1:0/1909311579 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75d01036e0 0x7f75d0107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:29.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.566+0000 7f75d4e40640 1 --2- 172.21.15.1:0/1909311579 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75d0102d30 0x7f75d0103110 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:29.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.566+0000 7f75d4e40640 1 -- 172.21.15.1:0/1909311579 >> 172.21.15.1:0/1909311579 conn(0x7f75d00fc7d0 msgr2=0x7f75d00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:59:29.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.566+0000 7f75d4e40640 1 -- 172.21.15.1:0/1909311579 shutdown_connections 2024-09-11T21:59:29.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.566+0000 7f75d4e40640 1 -- 172.21.15.1:0/1909311579 wait complete. 2024-09-11T21:59:29.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.568+0000 7f7ad7ba2640 1 Processor -- start 2024-09-11T21:59:29.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.568+0000 7f75d4e40640 1 Processor -- start 2024-09-11T21:59:29.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.568+0000 7f7ad7ba2640 1 -- start start 2024-09-11T21:59:29.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.568+0000 7f75d4e40640 1 -- start start 2024-09-11T21:59:29.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.568+0000 7f7ad7ba2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ad0102d90 0x7f7ad01009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:29.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.568+0000 7f7ad7ba2640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ad0103740 0x7f7ad0100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:29.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.568+0000 7f75d4e40640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75d0102d30 0x7f75d01a0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:29.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.568+0000 7f7ad7ba2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ad0108290 0x7f7ad0101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:29.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.568+0000 7f7ad7ba2640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad0112080 con 0x7f7ad0108290 2024-09-11T21:59:29.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.568+0000 7f7ad7ba2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad0111f00 con 0x7f7ad0102d90 2024-09-11T21:59:29.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.568+0000 7f7ad7ba2640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad0112200 con 0x7f7ad0103740 2024-09-11T21:59:29.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.568+0000 7f7ad513d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ad0103740 0x7f7ad0100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:29.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.568+0000 7f7ad513d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ad0103740 0x7f7ad0100f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:32796/0 (socket says 172.21.15.1:32796) 2024-09-11T21:59:29.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.568+0000 7f7ad513d640 1 -- 172.21.15.1:0/3460419738 learned_addr learned my addr 172.21.15.1:0/3460419738 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:59:29.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.568+0000 7f7ad613f640 1 --2- 172.21.15.1:0/3460419738 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ad0108290 0x7f7ad0101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:29.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.568+0000 7f75d4e40640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75d01036e0 0x7f75d01a0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:29.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.568+0000 7f75ce59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75d0102d30 0x7f75d01a0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:29.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.568+0000 7f75d4e40640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75d01082a0 0x7f75d019a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:29.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.569+0000 7f75ce59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75d0102d30 0x7f75d01a0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42256/0 (socket says 172.21.15.1:42256) 2024-09-11T21:59:29.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.569+0000 7f7ad593e640 1 --2- 172.21.15.1:0/3460419738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ad0102d90 0x7f7ad01009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:29.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.569+0000 7f75ce59c640 1 -- 172.21.15.1:0/3193142418 learned_addr learned my addr 172.21.15.1:0/3193142418 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:59:29.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.569+0000 7f7ad513d640 1 -- 172.21.15.1:0/3460419738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ad0102d90 msgr2=0x7f7ad01009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:59:29.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.569+0000 7f7ad513d640 1 --2- 172.21.15.1:0/3460419738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ad0102d90 0x7f7ad01009f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:29.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.569+0000 7f7ad513d640 1 -- 172.21.15.1:0/3460419738 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ad0108290 msgr2=0x7f7ad0101470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:59:29.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.569+0000 7f75d4e40640 1 -- 172.21.15.1:0/3193142418 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f75d01120a0 con 0x7f75d0102d30 2024-09-11T21:59:29.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.569+0000 7f75cdd9b640 1 --2- 172.21.15.1:0/3193142418 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75d01036e0 0x7f75d01a0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:29.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.569+0000 7f7ad513d640 1 --2- 172.21.15.1:0/3460419738 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ad0108290 0x7f7ad0101470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:29.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.569+0000 7f75d4e40640 1 -- 172.21.15.1:0/3193142418 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f75d0111f20 con 0x7f75d01082a0 2024-09-11T21:59:29.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.569+0000 7f75d4e40640 1 -- 172.21.15.1:0/3193142418 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f75d0112220 con 0x7f75d01036e0 2024-09-11T21:59:29.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.569+0000 7f75ced9d640 1 --2- 172.21.15.1:0/3193142418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75d01082a0 0x7f75d019a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:29.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.569+0000 7f7ad513d640 1 -- 172.21.15.1:0/3460419738 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ad01a6ac0 con 0x7f7ad0103740 2024-09-11T21:59:29.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.569+0000 7f7ad513d640 1 --2- 172.21.15.1:0/3460419738 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ad0103740 0x7f7ad0100f30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7ac002a940 tx=0x7f7ac0003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:59:29.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.569+0000 7f75ced9d640 1 -- 172.21.15.1:0/3193142418 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75d01036e0 msgr2=0x7f75d01a0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:59:29.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.569+0000 7f75ced9d640 1 --2- 172.21.15.1:0/3193142418 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75d01036e0 0x7f75d01a0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:29.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.569+0000 7f75ced9d640 1 -- 172.21.15.1:0/3193142418 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75d0102d30 msgr2=0x7f75d01a0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:59:29.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.569+0000 7f75ced9d640 1 --2- 172.21.15.1:0/3193142418 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75d0102d30 0x7f75d01a0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:29.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.569+0000 7f75ced9d640 1 -- 172.21.15.1:0/3193142418 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75d019aea0 con 0x7f75d01082a0 2024-09-11T21:59:29.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.569+0000 7f7ac6ffd640 1 -- 172.21.15.1:0/3460419738 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac003f070 con 0x7f7ad0103740 2024-09-11T21:59:29.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.569+0000 7f75ce59c640 1 --2- 172.21.15.1:0/3193142418 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75d0102d30 0x7f75d01a0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T21:59:29.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.570+0000 7f7ad7ba2640 1 -- 172.21.15.1:0/3460419738 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ad01a6d50 con 0x7f7ad0103740 2024-09-11T21:59:29.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.570+0000 7f7ac6ffd640 1 -- 172.21.15.1:0/3460419738 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ac0031c90 con 0x7f7ad0103740 2024-09-11T21:59:29.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.570+0000 7f7ac6ffd640 1 -- 172.21.15.1:0/3460419738 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac0035030 con 0x7f7ad0103740 2024-09-11T21:59:29.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.570+0000 7f75ced9d640 1 --2- 172.21.15.1:0/3193142418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75d01082a0 0x7f75d019a630 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f75c800e940 tx=0x7f75c800ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:59:29.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.570+0000 7f7ad7ba2640 1 -- 172.21.15.1:0/3460419738 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ad01a71d0 con 0x7f7ad0103740 2024-09-11T21:59:29.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.570+0000 7f75bf7fe640 1 -- 172.21.15.1:0/3193142418 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c8004d20 con 0x7f75d01082a0 2024-09-11T21:59:29.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.570+0000 7f75bf7fe640 1 -- 172.21.15.1:0/3193142418 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75c8005020 con 0x7f75d01082a0 2024-09-11T21:59:29.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.570+0000 7f75d4e40640 1 -- 172.21.15.1:0/3193142418 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75d019b190 con 0x7f75d01082a0 2024-09-11T21:59:29.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.570+0000 7f75bf7fe640 1 -- 172.21.15.1:0/3193142418 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c80116d0 con 0x7f75d01082a0 2024-09-11T21:59:29.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.570+0000 7f75d4e40640 1 -- 172.21.15.1:0/3193142418 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75d01a74e0 con 0x7f75d01082a0 2024-09-11T21:59:29.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.572+0000 7f7ad7ba2640 1 -- 172.21.15.1:0/3460419738 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ad0103170 con 0x7f7ad0103740 2024-09-11T21:59:29.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.572+0000 7f75d4e40640 1 -- 172.21.15.1:0/3193142418 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75d006a850 con 0x7f75d01082a0 2024-09-11T21:59:29.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.573+0000 7f7ac6ffd640 1 -- 172.21.15.1:0/3460419738 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f7ac0042070 con 0x7f7ad0103740 2024-09-11T21:59:29.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.573+0000 7f75bf7fe640 1 -- 172.21.15.1:0/3193142418 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f75c80054d0 con 0x7f75d01082a0 2024-09-11T21:59:29.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.573+0000 7f7ac6ffd640 1 --2- 172.21.15.1:0/3460419738 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f7abc0a2cf0 0x7f7abc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:29.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.574+0000 7f7ad593e640 1 --2- 172.21.15.1:0/3460419738 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f7abc0a2cf0 0x7f7abc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:29.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.574+0000 7f7ac6ffd640 1 -- 172.21.15.1:0/3460419738 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f7ac00efc10 con 0x7f7ad0103740 2024-09-11T21:59:29.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.574+0000 7f75bf7fe640 1 --2- 172.21.15.1:0/3193142418 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f75b80a2ec0 0x7f75b80a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:29.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.574+0000 7f75ce59c640 1 --2- 172.21.15.1:0/3193142418 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f75b80a2ec0 0x7f75b80a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:29.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.574+0000 7f75bf7fe640 1 -- 172.21.15.1:0/3193142418 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f75c80cf3d0 con 0x7f75d01082a0 2024-09-11T21:59:29.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.575+0000 7f7ad593e640 1 --2- 172.21.15.1:0/3460419738 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f7abc0a2cf0 0x7f7abc0a51b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7acc005fd0 tx=0x7f7acc009870 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:59:29.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.575+0000 7f75ce59c640 1 --2- 172.21.15.1:0/3193142418 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f75b80a2ec0 0x7f75b80a5380 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f75c0000c00 tx=0x7f75c0009560 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:59:29.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.578+0000 7f7ac6ffd640 1 -- 172.21.15.1:0/3460419738 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7ac00bc790 con 0x7f7ad0103740 2024-09-11T21:59:29.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.578+0000 7f75bf7fe640 1 -- 172.21.15.1:0/3193142418 <== mon.2 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f75c8016030 con 0x7f75d01082a0 2024-09-11T21:59:29.588 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=37 2024-09-11T21:59:29.588 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T21:59:29.612 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:59:29.612 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:59:29.613 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T21:59:29.613 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T21:59:29.614 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T21:59:29.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.795+0000 7f7ad7ba2640 1 -- 172.21.15.1:0/3460419738 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7ad010a2a0 con 0x7f7ad0103740 2024-09-11T21:59:29.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.796+0000 7f7ac6ffd640 1 -- 172.21.15.1:0/3460419738 <== mon.1 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11949 (secure 0 0 0) 0x7f7ac00bc090 con 0x7f7ad0103740 2024-09-11T21:59:29.797 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:59:29.797 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":28,"btime":"2024-09-11T21:57:12:512925+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/421542832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":421542832},{"type":"v1","addr":"172.21.15.1:6835","nonce":421542832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2836864406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2836864406},{"type":"v1","addr":"172.21.15.1:6837","nonce":2836864406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T21:55:20.791263+0000","modified":"2024-09-11T21:57:12.462736+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24484,"mds_2":24467,"mds_3":24466,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14595":{"gid":14595,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/407834672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":407834672},{"type":"v1","addr":"172.21.15.1:6839","nonce":407834672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/1068827678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":1068827678},{"type":"v1","addr":"172.21.15.1:6841","nonce":1068827678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/872410024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":872410024},{"type":"v1","addr":"172.21.15.161:6833","nonce":872410024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/1648944570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1648944570},{"type":"v1","addr":"172.21.15.53:6833","nonce":1648944570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/2126933993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2126933993},{"type":"v1","addr":"172.21.15.53:6835","nonce":2126933993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.161:6835/828381592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":828381592},{"type":"v1","addr":"172.21.15.161:6835","nonce":828381592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.161:6837/1799318656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1799318656},{"type":"v1","addr":"172.21.15.161:6837","nonce":1799318656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.53:6837/3457131830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3457131830},{"type":"v1","addr":"172.21.15.53:6837","nonce":3457131830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/455365383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":455365383},{"type":"v1","addr":"172.21.15.161:6839","nonce":455365383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.53:6839/1378584447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1378584447},{"type":"v1","addr":"172.21.15.53:6839","nonce":1378584447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24460,24466,24467,24484,24494]},"id":1}]} 2024-09-11T21:59:29.797 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 28 2024-09-11T21:59:29.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.798+0000 7f7ad7ba2640 1 -- 172.21.15.1:0/3460419738 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f7abc0a2cf0 msgr2=0x7f7abc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:59:29.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.798+0000 7f7ad7ba2640 1 --2- 172.21.15.1:0/3460419738 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f7abc0a2cf0 0x7f7abc0a51b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7acc005fd0 tx=0x7f7acc009870 comp rx=0 tx=0).stop 2024-09-11T21:59:29.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.798+0000 7f7ad7ba2640 1 -- 172.21.15.1:0/3460419738 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ad0103740 msgr2=0x7f7ad0100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:59:29.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.798+0000 7f7ad7ba2640 1 --2- 172.21.15.1:0/3460419738 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ad0103740 0x7f7ad0100f30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7ac002a940 tx=0x7f7ac0003b90 comp rx=0 tx=0).stop 2024-09-11T21:59:29.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.798+0000 7f75d4e40640 1 -- 172.21.15.1:0/3193142418 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f75d019bb60 con 0x7f75d01082a0 2024-09-11T21:59:29.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.798+0000 7f7ad7ba2640 1 -- 172.21.15.1:0/3460419738 shutdown_connections 2024-09-11T21:59:29.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.798+0000 7f7ad7ba2640 1 --2- 172.21.15.1:0/3460419738 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f7abc0a2cf0 0x7f7abc0a51b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:29.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.798+0000 7f7ad7ba2640 1 --2- 172.21.15.1:0/3460419738 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ad0108290 0x7f7ad0101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:29.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.798+0000 7f7ad7ba2640 1 --2- 172.21.15.1:0/3460419738 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ad0103740 0x7f7ad0100f30 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:29.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.798+0000 7f7ad7ba2640 1 --2- 172.21.15.1:0/3460419738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7ad0102d90 0x7f7ad01009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:29.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.798+0000 7f7ad7ba2640 1 -- 172.21.15.1:0/3460419738 >> 172.21.15.1:0/3460419738 conn(0x7f7ad00fc7d0 msgr2=0x7f7ad0106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:59:29.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.798+0000 7f7ad7ba2640 1 -- 172.21.15.1:0/3460419738 shutdown_connections 2024-09-11T21:59:29.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.798+0000 7f7ad7ba2640 1 -- 172.21.15.1:0/3460419738 wait complete. 2024-09-11T21:59:29.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.799+0000 7f75bf7fe640 1 -- 172.21.15.1:0/3193142418 <== mon.2 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11949 (secure 0 0 0) 0x7f75c809b900 con 0x7f75d01082a0 2024-09-11T21:59:29.801 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T21:59:29.801 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":28,"btime":"2024-09-11T21:57:12:512925+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/421542832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":421542832},{"type":"v1","addr":"172.21.15.1:6835","nonce":421542832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2836864406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2836864406},{"type":"v1","addr":"172.21.15.1:6837","nonce":2836864406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T21:55:20.791263+0000","modified":"2024-09-11T21:57:12.462736+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24484,"mds_2":24467,"mds_3":24466,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14595":{"gid":14595,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/407834672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":407834672},{"type":"v1","addr":"172.21.15.1:6839","nonce":407834672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/1068827678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":1068827678},{"type":"v1","addr":"172.21.15.1:6841","nonce":1068827678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/872410024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":872410024},{"type":"v1","addr":"172.21.15.161:6833","nonce":872410024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/1648944570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1648944570},{"type":"v1","addr":"172.21.15.53:6833","nonce":1648944570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/2126933993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2126933993},{"type":"v1","addr":"172.21.15.53:6835","nonce":2126933993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.161:6835/828381592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":828381592},{"type":"v1","addr":"172.21.15.161:6835","nonce":828381592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.161:6837/1799318656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1799318656},{"type":"v1","addr":"172.21.15.161:6837","nonce":1799318656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.53:6837/3457131830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3457131830},{"type":"v1","addr":"172.21.15.53:6837","nonce":3457131830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/455365383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":455365383},{"type":"v1","addr":"172.21.15.161:6839","nonce":455365383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.53:6839/1378584447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1378584447},{"type":"v1","addr":"172.21.15.53:6839","nonce":1378584447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24460,24466,24467,24484,24494]},"id":1}]} 2024-09-11T21:59:29.801 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 28 2024-09-11T21:59:29.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.802+0000 7f75d4e40640 1 -- 172.21.15.1:0/3193142418 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f75b80a2ec0 msgr2=0x7f75b80a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:59:29.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.802+0000 7f75d4e40640 1 --2- 172.21.15.1:0/3193142418 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f75b80a2ec0 0x7f75b80a5380 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f75c0000c00 tx=0x7f75c0009560 comp rx=0 tx=0).stop 2024-09-11T21:59:29.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.802+0000 7f75d4e40640 1 -- 172.21.15.1:0/3193142418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75d01082a0 msgr2=0x7f75d019a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:59:29.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.802+0000 7f75d4e40640 1 --2- 172.21.15.1:0/3193142418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75d01082a0 0x7f75d019a630 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f75c800e940 tx=0x7f75c800ee00 comp rx=0 tx=0).stop 2024-09-11T21:59:29.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.802+0000 7f75d4e40640 1 -- 172.21.15.1:0/3193142418 shutdown_connections 2024-09-11T21:59:29.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.802+0000 7f75d4e40640 1 --2- 172.21.15.1:0/3193142418 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f75b80a2ec0 0x7f75b80a5380 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:29.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.802+0000 7f75d4e40640 1 --2- 172.21.15.1:0/3193142418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75d01082a0 0x7f75d019a630 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:29.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.802+0000 7f75d4e40640 1 --2- 172.21.15.1:0/3193142418 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f75d01036e0 0x7f75d01a0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:29.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.802+0000 7f75d4e40640 1 --2- 172.21.15.1:0/3193142418 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75d0102d30 0x7f75d01a0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:29.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.802+0000 7f75d4e40640 1 -- 172.21.15.1:0/3193142418 >> 172.21.15.1:0/3193142418 conn(0x7f75d00fc7d0 msgr2=0x7f75d00fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:59:29.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.802+0000 7f75d4e40640 1 -- 172.21.15.1:0/3193142418 shutdown_connections 2024-09-11T21:59:29.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:29.802+0000 7f75d4e40640 1 -- 172.21.15.1:0/3193142418 wait complete. 2024-09-11T21:59:30.220 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:59:30.221 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:59:30.627 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T21:59:30.627 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T21:59:30.688 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:59:30.689 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:59:30.689 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 37 -lt 100 ']' 2024-09-11T21:59:30.689 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 37 + 1 2024-09-11T21:59:30.822 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=38 2024-09-11T21:59:30.823 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T21:59:30.851 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:59:30.851 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:59:30.852 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T21:59:30.852 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T21:59:30.852 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T21:59:31.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:59:30 smithi053 ceph-mon[34908]: pgmap v73: 97 pgs: 97 active+clean; 2.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 KiB/s wr, 20 op/s 2024-09-11T21:59:31.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:59:30 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/3460419738' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:59:31.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:59:30 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/3193142418' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:59:31.091 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:59:30 smithi161 ceph-mon[35615]: pgmap v73: 97 pgs: 97 active+clean; 2.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 KiB/s wr, 20 op/s 2024-09-11T21:59:31.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:59:30 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/3460419738' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:59:31.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:59:30 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/3193142418' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:59:31.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:59:30 smithi001 ceph-mon[30935]: pgmap v73: 97 pgs: 97 active+clean; 2.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 KiB/s wr, 20 op/s 2024-09-11T21:59:31.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:59:30 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/3460419738' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:59:31.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:59:30 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/3193142418' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T21:59:31.779 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:59:31.779 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:59:32.186 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T21:59:32.186 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T21:59:32.228 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:59:32.228 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:59:32.228 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 38 -lt 100 ']' 2024-09-11T21:59:32.228 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 38 + 1 2024-09-11T21:59:32.304 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=39 2024-09-11T21:59:32.304 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T21:59:32.354 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:59:32.354 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:59:32.355 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T21:59:32.355 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T21:59:32.356 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T21:59:32.413 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T21:59:33.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:59:32 smithi053 ceph-mon[34908]: pgmap v74: 97 pgs: 97 active+clean; 2.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-09-11T21:59:33.074 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:59:32 smithi001 ceph-mon[30935]: pgmap v74: 97 pgs: 97 active+clean; 2.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-09-11T21:59:33.091 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:59:32 smithi161 ceph-mon[35615]: pgmap v74: 97 pgs: 97 active+clean; 2.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-09-11T21:59:33.259 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:59:33.259 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:59:33.300 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db / --timeout 90 --expiration 133.4 --await-for 45.0' 2024-09-11T21:59:33.300 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db / --timeout 90 --expiration 133.4 --await-for 45.0 2024-09-11T21:59:33.369 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T21:59:33.370 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T21:59:33.378 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:59:33.378 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:59:33.378 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 39 -lt 100 ']' 2024-09-11T21:59:33.378 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 39 + 1 2024-09-11T21:59:33.423 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=40 2024-09-11T21:59:33.423 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T21:59:33.454 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:59:33.454 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:59:33.455 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T21:59:33.455 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T21:59:33.455 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T21:59:33.718 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:59:33.718 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:59:33.934 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T21:59:33.934 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T21:59:33.942 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:59:33.942 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:59:33.942 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 40 -lt 100 ']' 2024-09-11T21:59:33.942 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 40 + 1 2024-09-11T21:59:33.964 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=41 2024-09-11T21:59:33.964 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T21:59:33.985 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:59:33.985 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:59:33.986 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T21:59:33.986 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T21:59:33.986 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T21:59:34.435 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:59:34.435 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:59:34.579 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T21:59:34.579 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T21:59:34.602 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:59:34.602 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:59:34.603 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 41 -lt 100 ']' 2024-09-11T21:59:34.603 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 41 + 1 2024-09-11T21:59:34.632 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=42 2024-09-11T21:59:34.632 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T21:59:34.658 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:59:34.658 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:59:34.659 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T21:59:34.659 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T21:59:34.659 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T21:59:34.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:59:34 smithi001 ceph-mon[30935]: pgmap v75: 97 pgs: 97 active+clean; 2.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-09-11T21:59:35.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:59:34 smithi053 ceph-mon[34908]: pgmap v75: 97 pgs: 97 active+clean; 2.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-09-11T21:59:35.091 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:59:34 smithi161 ceph-mon[35615]: pgmap v75: 97 pgs: 97 active+clean; 2.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-09-11T21:59:35.250 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:59:35.250 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:59:35.434 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T21:59:35.434 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T21:59:35.475 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:59:35.475 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:59:35.475 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 42 -lt 100 ']' 2024-09-11T21:59:35.476 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 42 + 1 2024-09-11T21:59:35.523 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=43 2024-09-11T21:59:35.523 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T21:59:35.560 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:59:35.561 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:59:35.562 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T21:59:35.562 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T21:59:35.562 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T21:59:36.218 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:59:36.218 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:59:36.460 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T21:59:36.461 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T21:59:36.517 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:59:36.517 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:59:36.518 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 43 -lt 100 ']' 2024-09-11T21:59:36.518 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 43 + 1 2024-09-11T21:59:36.567 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=44 2024-09-11T21:59:36.567 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T21:59:36.652 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:59:36.652 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:59:36.653 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T21:59:36.653 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T21:59:36.653 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T21:59:37.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:59:36 smithi053 ceph-mon[34908]: pgmap v76: 97 pgs: 97 active+clean; 2.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-11T21:59:37.091 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:59:36 smithi161 ceph-mon[35615]: pgmap v76: 97 pgs: 97 active+clean; 2.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-11T21:59:37.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:59:36 smithi001 ceph-mon[30935]: pgmap v76: 97 pgs: 97 active+clean; 2.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-11T21:59:37.661 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:59:37.661 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:59:37.859 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T21:59:37.859 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T21:59:37.884 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:59:37.885 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:59:37.885 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 44 -lt 100 ']' 2024-09-11T21:59:37.885 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 44 + 1 2024-09-11T21:59:37.940 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=45 2024-09-11T21:59:37.941 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T21:59:38.019 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:59:38.019 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:59:38.020 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T21:59:38.020 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T21:59:38.020 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T21:59:38.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:59:38 smithi001 ceph-mon[30935]: pgmap v77: 97 pgs: 97 active+clean; 2.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 15 op/s 2024-09-11T21:59:38.986 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:59:38.986 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:59:39.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:59:38 smithi053 ceph-mon[34908]: pgmap v77: 97 pgs: 97 active+clean; 2.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 15 op/s 2024-09-11T21:59:39.091 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:59:38 smithi161 ceph-mon[35615]: pgmap v77: 97 pgs: 97 active+clean; 2.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 15 op/s 2024-09-11T21:59:39.559 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T21:59:39.559 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T21:59:39.576 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:59:39.576 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:59:39.576 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 45 -lt 100 ']' 2024-09-11T21:59:39.576 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 45 + 1 2024-09-11T21:59:39.682 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=46 2024-09-11T21:59:39.682 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T21:59:39.728 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:59:39.728 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:59:39.729 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T21:59:39.729 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T21:59:39.729 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T21:59:40.894 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:59:40.894 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:59:40.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:59:40 smithi001 ceph-mon[30935]: pgmap v78: 97 pgs: 97 active+clean; 2.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 KiB/s wr, 21 op/s 2024-09-11T21:59:41.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:59:40 smithi053 ceph-mon[34908]: pgmap v78: 97 pgs: 97 active+clean; 2.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 KiB/s wr, 21 op/s 2024-09-11T21:59:41.091 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:59:40 smithi161 ceph-mon[35615]: pgmap v78: 97 pgs: 97 active+clean; 2.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 KiB/s wr, 21 op/s 2024-09-11T21:59:41.319 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T21:59:41.319 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T21:59:41.335 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:59:41.335 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:59:41.335 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 46 -lt 100 ']' 2024-09-11T21:59:41.335 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 46 + 1 2024-09-11T21:59:41.448 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=47 2024-09-11T21:59:41.448 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T21:59:41.553 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:59:41.553 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:59:41.554 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T21:59:41.554 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T21:59:41.554 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T21:59:42.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:59:42 smithi001 ceph-mon[30935]: pgmap v79: 97 pgs: 97 active+clean; 2.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-09-11T21:59:43.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:59:42 smithi053 ceph-mon[34908]: pgmap v79: 97 pgs: 97 active+clean; 2.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-09-11T21:59:43.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:59:42 smithi161 ceph-mon[35615]: pgmap v79: 97 pgs: 97 active+clean; 2.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-09-11T21:59:43.101 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:59:43.102 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:59:43.350 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T21:59:43.350 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T21:59:43.367 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:59:43.368 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:59:43.368 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 47 -lt 100 ']' 2024-09-11T21:59:43.368 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 47 + 1 2024-09-11T21:59:43.531 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=48 2024-09-11T21:59:43.532 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T21:59:43.569 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:59:43.569 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:59:43.570 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T21:59:43.570 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T21:59:43.570 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T21:59:44.985 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:59:44.985 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:59:45.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:59:44 smithi053 ceph-mon[34908]: pgmap v80: 97 pgs: 97 active+clean; 2.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-09-11T21:59:45.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:59:44 smithi161 ceph-mon[35615]: pgmap v80: 97 pgs: 97 active+clean; 2.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-09-11T21:59:45.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:59:44 smithi001 ceph-mon[30935]: pgmap v80: 97 pgs: 97 active+clean; 2.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-09-11T21:59:45.196 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:59:45.196 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:59:45.366 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T21:59:45.366 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T21:59:45.392 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:59:45.392 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:59:45.392 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 48 -lt 100 ']' 2024-09-11T21:59:45.392 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 48 + 1 2024-09-11T21:59:45.471 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=49 2024-09-11T21:59:45.472 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T21:59:45.518 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:59:45.518 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:59:45.520 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T21:59:45.520 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T21:59:45.520 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T21:59:45.788 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:45.787+0000 7f6b3edd6640 1 -- 172.21.15.1:0/1338547393 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b38101130 msgr2=0x7f6b3810dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:59:45.789 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:45.787+0000 7f6b3edd6640 1 --2- 172.21.15.1:0/1338547393 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b38101130 0x7f6b3810dd10 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f6b28009900 tx=0x7f6b2802a3f0 comp rx=0 tx=0).stop 2024-09-11T21:59:45.789 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:45.787+0000 7f6b3edd6640 1 -- 172.21.15.1:0/1338547393 shutdown_connections 2024-09-11T21:59:45.789 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:45.787+0000 7f6b3edd6640 1 --2- 172.21.15.1:0/1338547393 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b3810e2f0 0x7f6b38110740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:45.789 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:45.787+0000 7f6b3edd6640 1 --2- 172.21.15.1:0/1338547393 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b38101130 0x7f6b3810dd10 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:45.789 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:45.787+0000 7f6b3edd6640 1 --2- 172.21.15.1:0/1338547393 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b38100810 0x7f6b38100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:45.789 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:45.787+0000 7f6b3edd6640 1 -- 172.21.15.1:0/1338547393 >> 172.21.15.1:0/1338547393 conn(0x7f6b38075eb0 msgr2=0x7f6b380762c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:59:45.789 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:45.787+0000 7f6b3edd6640 1 -- 172.21.15.1:0/1338547393 shutdown_connections 2024-09-11T21:59:45.789 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:45.787+0000 7f6b3edd6640 1 -- 172.21.15.1:0/1338547393 wait complete. 2024-09-11T21:59:45.820 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:45.817+0000 7f6b3edd6640 1 Processor -- start 2024-09-11T21:59:45.820 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:45.817+0000 7f6b3edd6640 1 -- start start 2024-09-11T21:59:45.820 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:45.817+0000 7f6b3edd6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b38100810 0x7f6b3819a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:45.821 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:45.817+0000 7f6b3edd6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b38101130 0x7f6b3819aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:45.821 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:45.817+0000 7f6b3edd6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b3810e2f0 0x7f6b3819ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:45.821 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:45.817+0000 7f6b3edd6640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6b381140d0 con 0x7f6b38101130 2024-09-11T21:59:45.821 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:45.817+0000 7f6b3edd6640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6b38113f50 con 0x7f6b38100810 2024-09-11T21:59:45.821 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:45.817+0000 7f6b3edd6640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6b38114250 con 0x7f6b3810e2f0 2024-09-11T21:59:45.821 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:45.818+0000 7f6b2ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b38101130 0x7f6b3819aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:45.821 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:45.818+0000 7f6b2ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b38101130 0x7f6b3819aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51010/0 (socket says 172.21.15.1:51010) 2024-09-11T21:59:45.821 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:45.818+0000 7f6b2ffff640 1 -- 172.21.15.1:0/3446783294 learned_addr learned my addr 172.21.15.1:0/3446783294 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:59:45.821 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:45.818+0000 7f6b3d373640 1 --2- 172.21.15.1:0/3446783294 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b3810e2f0 0x7f6b3819ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:45.821 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:45.818+0000 7f6b3cb72640 1 --2- 172.21.15.1:0/3446783294 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b38100810 0x7f6b3819a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:45.821 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:45.818+0000 7f6b2ffff640 1 -- 172.21.15.1:0/3446783294 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b3810e2f0 msgr2=0x7f6b3819ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:59:45.822 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:45.818+0000 7f6b2ffff640 1 --2- 172.21.15.1:0/3446783294 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b3810e2f0 0x7f6b3819ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:45.822 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:45.818+0000 7f6b2ffff640 1 -- 172.21.15.1:0/3446783294 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b38100810 msgr2=0x7f6b3819a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:59:45.822 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:45.818+0000 7f6b2ffff640 1 --2- 172.21.15.1:0/3446783294 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b38100810 0x7f6b3819a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:45.822 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:45.818+0000 7f6b2ffff640 1 -- 172.21.15.1:0/3446783294 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b3819f590 con 0x7f6b38101130 2024-09-11T21:59:45.822 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:45.818+0000 7f6b2ffff640 1 --2- 172.21.15.1:0/3446783294 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b38101130 0x7f6b3819aae0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f6b2802a910 tx=0x7f6b28003b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:59:45.823 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:45.821+0000 7f6b2dffb640 1 -- 172.21.15.1:0/3446783294 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2803f070 con 0x7f6b38101130 2024-09-11T21:59:45.823 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:45.821+0000 7f6b2dffb640 1 -- 172.21.15.1:0/3446783294 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b28031c90 con 0x7f6b38101130 2024-09-11T21:59:45.823 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:45.821+0000 7f6b3edd6640 1 -- 172.21.15.1:0/3446783294 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b3819f820 con 0x7f6b38101130 2024-09-11T21:59:45.823 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:45.821+0000 7f6b2dffb640 1 -- 172.21.15.1:0/3446783294 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b28035030 con 0x7f6b38101130 2024-09-11T21:59:45.823 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:45.822+0000 7f6b3edd6640 1 -- 172.21.15.1:0/3446783294 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b381a7110 con 0x7f6b38101130 2024-09-11T21:59:45.824 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:45.824+0000 7f6b2dffb640 1 -- 172.21.15.1:0/3446783294 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f6b28042070 con 0x7f6b38101130 2024-09-11T21:59:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:45.824+0000 7f6b2dffb640 1 --2- 172.21.15.1:0/3446783294 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f6b200a2e40 0x7f6b200a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:45.825+0000 7f6b2dffb640 1 -- 172.21.15.1:0/3446783294 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f6b280ef720 con 0x7f6b38101130 2024-09-11T21:59:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:45.825+0000 7f6b3cb72640 1 --2- 172.21.15.1:0/3446783294 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f6b200a2e40 0x7f6b200a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:45.826 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:45.825+0000 7f6b3cb72640 1 --2- 172.21.15.1:0/3446783294 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f6b200a2e40 0x7f6b200a5300 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6b340095f0 tx=0x7f6b34005f90 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:59:46.132 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.130+0000 7f6b41c4f740 1 Processor -- start 2024-09-11T21:59:46.132 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.130+0000 7f6b41c4f740 1 -- start start 2024-09-11T21:59:46.132 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.130+0000 7f6b41c4f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605f61f3af0 0x5605f61f3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:46.132 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.130+0000 7f6b41c4f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605f61f4410 0x5605f61fe770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:46.132 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.130+0000 7f6b41c4f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605f61fecb0 0x5605f62010a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:46.132 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.130+0000 7f6b41c4f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5605f61582f0 con 0x5605f61fecb0 2024-09-11T21:59:46.132 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.130+0000 7f6b41c4f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5605f6100270 con 0x5605f61f3af0 2024-09-11T21:59:46.132 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.130+0000 7f6b41c4f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5605f61542b0 con 0x5605f61f4410 2024-09-11T21:59:46.132 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.131+0000 7f6b3d373640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605f61fecb0 0x5605f62010a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:46.132 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.131+0000 7f6b3d373640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605f61fecb0 0x5605f62010a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51018/0 (socket says 172.21.15.1:51018) 2024-09-11T21:59:46.133 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.131+0000 7f6b3d373640 1 -- 172.21.15.1:0/2729786124 learned_addr learned my addr 172.21.15.1:0/2729786124 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:59:46.133 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.131+0000 7f6b2ffff640 1 --2- 172.21.15.1:0/2729786124 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605f61f4410 0x5605f61fe770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:46.133 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.131+0000 7f6b3cb72640 1 --2- 172.21.15.1:0/2729786124 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605f61f3af0 0x5605f61f3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:46.133 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.131+0000 7f6b3d373640 1 -- 172.21.15.1:0/2729786124 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605f61f4410 msgr2=0x5605f61fe770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:59:46.133 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.131+0000 7f6b3d373640 1 --2- 172.21.15.1:0/2729786124 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605f61f4410 0x5605f61fe770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:46.133 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.131+0000 7f6b3d373640 1 -- 172.21.15.1:0/2729786124 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605f61f3af0 msgr2=0x5605f61f3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:59:46.133 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.131+0000 7f6b3d373640 1 --2- 172.21.15.1:0/2729786124 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605f61f3af0 0x5605f61f3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:46.133 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.131+0000 7f6b3d373640 1 -- 172.21.15.1:0/2729786124 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605f60ff340 con 0x5605f61fecb0 2024-09-11T21:59:46.133 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.131+0000 7f6b2ffff640 1 --2- 172.21.15.1:0/2729786124 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605f61f4410 0x5605f61fe770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T21:59:46.133 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.131+0000 7f6b3d373640 1 --2- 172.21.15.1:0/2729786124 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605f61fecb0 0x5605f62010a0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f6b30009fd0 tx=0x7f6b300118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e452ae064f00991 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:59:46.134 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.132+0000 7f6b1effd640 1 -- 172.21.15.1:0/2729786124 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b30004d20 con 0x5605f61fecb0 2024-09-11T21:59:46.134 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.132+0000 7f6b1effd640 1 -- 172.21.15.1:0/2729786124 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b30004050 con 0x5605f61fecb0 2024-09-11T21:59:46.134 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.132+0000 7f6b1effd640 1 -- 172.21.15.1:0/2729786124 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b30005280 con 0x5605f61fecb0 2024-09-11T21:59:46.134 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.133+0000 7f6b41c4f740 1 -- 172.21.15.1:0/2729786124 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605f61fecb0 msgr2=0x5605f62010a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:59:46.134 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.133+0000 7f6b41c4f740 1 --2- 172.21.15.1:0/2729786124 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605f61fecb0 0x5605f62010a0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f6b30009fd0 tx=0x7f6b300118d0 comp rx=0 tx=0).stop 2024-09-11T21:59:46.134 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.133+0000 7f6b41c4f740 1 -- 172.21.15.1:0/2729786124 shutdown_connections 2024-09-11T21:59:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.133+0000 7f6b41c4f740 1 --2- 172.21.15.1:0/2729786124 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605f61fecb0 0x5605f62010a0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.133+0000 7f6b41c4f740 1 --2- 172.21.15.1:0/2729786124 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605f61f4410 0x5605f61fe770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.133+0000 7f6b41c4f740 1 --2- 172.21.15.1:0/2729786124 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605f61f3af0 0x5605f61f3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.133+0000 7f6b41c4f740 1 -- 172.21.15.1:0/2729786124 >> 172.21.15.1:0/2729786124 conn(0x5605f61f0060 msgr2=0x5605f61f0440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:59:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.133+0000 7f6b41c4f740 1 -- 172.21.15.1:0/2729786124 shutdown_connections 2024-09-11T21:59:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.133+0000 7f6b41c4f740 1 -- 172.21.15.1:0/2729786124 wait complete. 2024-09-11T21:59:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.294+0000 7f6b41c4f740 1 -- start start 2024-09-11T21:59:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.301+0000 7f6b41c4f740 1 Processor -- start 2024-09-11T21:59:46.303 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.301+0000 7f6b41c4f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605f61f3af0 0x5605f61fbb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:46.303 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.301+0000 7f6b41c4f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605f61f4410 0x5605f61fc060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:46.303 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.301+0000 7f6b41c4f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605f61fecb0 0x5605f61fc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:46.303 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.301+0000 7f6b41c4f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5605f6201880 con 0x5605f61f4410 2024-09-11T21:59:46.303 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.301+0000 7f6b41c4f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5605f60e1290 con 0x5605f61f3af0 2024-09-11T21:59:46.303 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.301+0000 7f6b41c4f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5605f6201a00 con 0x5605f61fecb0 2024-09-11T21:59:46.304 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.302+0000 7f6b2ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605f61f4410 0x5605f61fc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:46.304 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.302+0000 7f6b3cb72640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605f61f3af0 0x5605f61fbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:46.304 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.302+0000 7f6b3cb72640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605f61f3af0 0x5605f61fbb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:37334/0 (socket says 172.21.15.1:37334) 2024-09-11T21:59:46.304 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.302+0000 7f6b3cb72640 1 -- 172.21.15.1:0/3085702899 learned_addr learned my addr 172.21.15.1:0/3085702899 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:59:46.304 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.302+0000 7f6b2ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605f61f4410 0x5605f61fc060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51028/0 (socket says 172.21.15.1:51028) 2024-09-11T21:59:46.304 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.302+0000 7f6b3d373640 1 --2- 172.21.15.1:0/3085702899 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605f61fecb0 0x5605f61fc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:46.304 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.302+0000 7f6b3cb72640 1 -- 172.21.15.1:0/3085702899 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605f61fecb0 msgr2=0x5605f61fc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:59:46.304 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.302+0000 7f6b3cb72640 1 --2- 172.21.15.1:0/3085702899 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605f61fecb0 0x5605f61fc5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:46.304 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.302+0000 7f6b3cb72640 1 -- 172.21.15.1:0/3085702899 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605f61f4410 msgr2=0x5605f61fc060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:59:46.304 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.302+0000 7f6b3cb72640 1 --2- 172.21.15.1:0/3085702899 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605f61f4410 0x5605f61fc060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:46.304 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.302+0000 7f6b3cb72640 1 -- 172.21.15.1:0/3085702899 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605f5e6b210 con 0x5605f61f3af0 2024-09-11T21:59:46.304 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.302+0000 7f6b41c4f740 10 client.0 _ll_register_callbacks cb 0x5605f6112140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:59:46.305 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.302+0000 7f6b3cb72640 1 --2- 172.21.15.1:0/3085702899 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605f61f3af0 0x5605f61fbb20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6b3400ed40 tx=0x7f6b34004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca151bb0374870a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:59:46.305 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.302+0000 7f6b17fff640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-11T21:59:46.305 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.303+0000 7f6b17fff640 1 -- 172.21.15.1:0/3085702899 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b3402b060 con 0x5605f61f3af0 2024-09-11T21:59:46.305 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.303+0000 7f6b41c4f740 1 -- 172.21.15.1:0/3085702899 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b30004050 con 0x5605f61f3af0 2024-09-11T21:59:46.305 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.303+0000 7f6b17fff640 1 -- 172.21.15.1:0/3085702899 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f6b3401db20 con 0x5605f61f3af0 2024-09-11T21:59:46.305 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.303+0000 7f6b17fff640 1 -- 172.21.15.1:0/3085702899 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6b3401dd20 con 0x5605f61f3af0 2024-09-11T21:59:46.306 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.303+0000 7f6b41c4f740 10 client.25493 fetch_fsmap learned FSMap version 28 2024-09-11T21:59:46.306 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.303+0000 7f6b41c4f740 1 -- 172.21.15.1:0/3085702899 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({fsmap=28}) -- 0x5605f5f37620 con 0x5605f61f3af0 2024-09-11T21:59:46.327 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.326+0000 7f6b17fff640 1 -- 172.21.15.1:0/3085702899 <== mon.2 v2:172.21.15.53:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6b34026410 con 0x5605f61f3af0 2024-09-11T21:59:46.327 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.327+0000 7f6b41c4f740 10 client.25493 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T21:59:46.327 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.327+0000 7f6b41c4f740 10 client.25493 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T21:59:46.328 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.328+0000 7f6b41c4f740 20 client.25493 populate_metadata read hostname 'smithi001' 2024-09-11T21:59:46.328 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.328+0000 7f6b41c4f740 1 --2- 172.21.15.1:0/3085702899 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5605f620e960 0x5605f6210e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:46.328 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.328+0000 7f6b41c4f740 4 client.25493 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T21:59:46.328 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.328+0000 7f6b41c4f740 1 -- 172.21.15.1:0/3085702899 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5605f6165a20 con 0x5605f620e960 2024-09-11T21:59:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.328+0000 7f6b2ffff640 1 --2- 172.21.15.1:0/3085702899 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5605f620e960 0x5605f6210e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.329+0000 7f6b17fff640 10 client.25493 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T21:59:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.329+0000 7f6b2ffff640 1 --2- 172.21.15.1:0/3085702899 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5605f620e960 0x5605f6210e20 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ebc42e518a8e326 server_cookie=42e9450435c04a63 in_seq=0 out_seq=0 2024-09-11T21:59:46.330 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.329+0000 7f6b17fff640 1 -- 172.21.15.1:0/3085702899 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5605f6165a20 con 0x5605f620e960 2024-09-11T21:59:46.330 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.329+0000 7f6b17fff640 10 client.25493 handle_command_reply: tid=0 2024-09-11T21:59:46.330 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.329+0000 7f6b41c4f740 1 client.25493 shutdown 2024-09-11T21:59:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.330+0000 7f6b41c4f740 1 -- 172.21.15.1:0/3085702899 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605f61f3af0 msgr2=0x5605f61fbb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:59:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.330+0000 7f6b41c4f740 1 --2- 172.21.15.1:0/3085702899 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605f61f3af0 0x5605f61fbb20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6b3400ed40 tx=0x7f6b34004730 comp rx=0 tx=0).stop 2024-09-11T21:59:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.330+0000 7f6b41c4f740 1 -- 172.21.15.1:0/3085702899 shutdown_connections 2024-09-11T21:59:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.330+0000 7f6b41c4f740 1 --2- 172.21.15.1:0/3085702899 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5605f620e960 0x5605f6210e20 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.330+0000 7f6b41c4f740 1 --2- 172.21.15.1:0/3085702899 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605f61fecb0 0x5605f61fc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.330+0000 7f6b41c4f740 1 --2- 172.21.15.1:0/3085702899 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605f61f4410 0x5605f61fc060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.330+0000 7f6b41c4f740 1 --2- 172.21.15.1:0/3085702899 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605f61f3af0 0x5605f61fbb20 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.330+0000 7f6b17fff640 0 client.25493 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T21:59:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.330+0000 7f6b41c4f740 1 -- 172.21.15.1:0/3085702899 >> 172.21.15.1:0/3085702899 conn(0x5605f61f0060 msgr2=0x5605f61f5d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:59:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.330+0000 7f6b41c4f740 1 -- 172.21.15.1:0/3085702899 shutdown_connections 2024-09-11T21:59:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.330+0000 7f6b41c4f740 1 -- 172.21.15.1:0/3085702899 wait complete. 2024-09-11T21:59:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.330+0000 7f6b41c4f740 20 client.25493 trim_cache size 0 max 16384 2024-09-11T21:59:46.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.372+0000 7f6b41c4f740 1 Processor -- start 2024-09-11T21:59:46.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.372+0000 7f6b41c4f740 1 -- start start 2024-09-11T21:59:46.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.372+0000 7f6b41c4f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605f61f3af0 0x5605f61fc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:46.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.372+0000 7f6b41c4f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605f61f4410 0x5605f62166f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:46.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.372+0000 7f6b41c4f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605f61fecb0 0x5605f6216c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:46.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.372+0000 7f6b41c4f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6b381afa50 con 0x5605f61f3af0 2024-09-11T21:59:46.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.372+0000 7f6b41c4f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6b381b1590 con 0x5605f61f4410 2024-09-11T21:59:46.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.372+0000 7f6b41c4f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6b381b0120 con 0x5605f61fecb0 2024-09-11T21:59:46.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.372+0000 7f6b3cb72640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605f61f3af0 0x5605f61fc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:46.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.373+0000 7f6b3cb72640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605f61f3af0 0x5605f61fc3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51038/0 (socket says 172.21.15.1:51038) 2024-09-11T21:59:46.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.373+0000 7f6b3cb72640 1 -- 172.21.15.1:0/326848080 learned_addr learned my addr 172.21.15.1:0/326848080 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:59:46.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.373+0000 7f6b2ffff640 1 --2- 172.21.15.1:0/326848080 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605f61f4410 0x5605f62166f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:46.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.373+0000 7f6b3d373640 1 --2- 172.21.15.1:0/326848080 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605f61fecb0 0x5605f6216c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:46.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.373+0000 7f6b3cb72640 1 -- 172.21.15.1:0/326848080 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605f61fecb0 msgr2=0x5605f6216c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:59:46.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.373+0000 7f6b3cb72640 1 --2- 172.21.15.1:0/326848080 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605f61fecb0 0x5605f6216c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:46.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.373+0000 7f6b3cb72640 1 -- 172.21.15.1:0/326848080 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605f61f4410 msgr2=0x5605f62166f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:59:46.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.373+0000 7f6b3cb72640 1 --2- 172.21.15.1:0/326848080 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605f61f4410 0x5605f62166f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:46.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.373+0000 7f6b3cb72640 1 -- 172.21.15.1:0/326848080 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605f61efe90 con 0x5605f61f3af0 2024-09-11T21:59:46.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.373+0000 7f6b3cb72640 1 --2- 172.21.15.1:0/326848080 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605f61f3af0 0x5605f61fc3d0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f6b34005870 tx=0x7f6b34022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=389337db30243548 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:59:46.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.373+0000 7f6b1e7fc640 1 -- 172.21.15.1:0/326848080 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b3400e920 con 0x5605f61f3af0 2024-09-11T21:59:46.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.373+0000 7f6b1e7fc640 1 -- 172.21.15.1:0/326848080 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b30004050 con 0x5605f61f3af0 2024-09-11T21:59:46.376 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.373+0000 7f6b1e7fc640 1 -- 172.21.15.1:0/326848080 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b340268c0 con 0x5605f61f3af0 2024-09-11T21:59:46.379 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.374+0000 7f6b41c4f740 1 -- 172.21.15.1:0/326848080 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605f61f3af0 msgr2=0x5605f61fc3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:59:46.379 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.374+0000 7f6b41c4f740 1 --2- 172.21.15.1:0/326848080 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605f61f3af0 0x5605f61fc3d0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f6b34005870 tx=0x7f6b34022000 comp rx=0 tx=0).stop 2024-09-11T21:59:46.379 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.374+0000 7f6b41c4f740 1 -- 172.21.15.1:0/326848080 shutdown_connections 2024-09-11T21:59:46.379 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.374+0000 7f6b41c4f740 1 --2- 172.21.15.1:0/326848080 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605f61fecb0 0x5605f6216c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:46.379 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.374+0000 7f6b41c4f740 1 --2- 172.21.15.1:0/326848080 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605f61f4410 0x5605f62166f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:46.379 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.374+0000 7f6b41c4f740 1 --2- 172.21.15.1:0/326848080 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605f61f3af0 0x5605f61fc3d0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:46.380 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.374+0000 7f6b41c4f740 1 -- 172.21.15.1:0/326848080 >> 172.21.15.1:0/326848080 conn(0x5605f61f0060 msgr2=0x5605f61fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:59:46.380 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.374+0000 7f6b41c4f740 1 -- 172.21.15.1:0/326848080 shutdown_connections 2024-09-11T21:59:46.380 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.374+0000 7f6b41c4f740 1 -- 172.21.15.1:0/326848080 wait complete. 2024-09-11T21:59:46.380 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.375+0000 7f6b41c4f740 1 -- start start 2024-09-11T21:59:46.380 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.375+0000 7f6b41c4f740 1 Processor -- start 2024-09-11T21:59:46.380 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.375+0000 7f6b41c4f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605f61f3af0 0x5605f62074b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:46.380 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.375+0000 7f6b41c4f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605f61f4410 0x5605f62027d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:46.380 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.375+0000 7f6b41c4f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605f61fecb0 0x5605f6202d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:46.380 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.375+0000 7f6b41c4f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6b381b0ec0 con 0x5605f61f3af0 2024-09-11T21:59:46.380 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.375+0000 7f6b41c4f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6b381b07f0 con 0x5605f61fecb0 2024-09-11T21:59:46.380 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.375+0000 7f6b41c4f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6b381ada30 con 0x5605f61f4410 2024-09-11T21:59:46.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.375+0000 7f6b41c4f740 10 client.0 _ll_register_callbacks cb 0x7f6b3400e4b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:59:46.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.375+0000 7f6b3cb72640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605f61f3af0 0x5605f62074b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:46.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.375+0000 7f6b3cb72640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605f61f3af0 0x5605f62074b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51044/0 (socket says 172.21.15.1:51044) 2024-09-11T21:59:46.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.375+0000 7f6b3cb72640 1 -- 172.21.15.1:0/2713551895 learned_addr learned my addr 172.21.15.1:0/2713551895 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:59:46.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.376+0000 7f6b2ffff640 1 --2- 172.21.15.1:0/2713551895 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605f61f4410 0x5605f62027d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:46.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.376+0000 7f6b3d373640 1 --2- 172.21.15.1:0/2713551895 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605f61fecb0 0x5605f6202d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:46.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.376+0000 7f6b3cb72640 1 -- 172.21.15.1:0/2713551895 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605f61f4410 msgr2=0x5605f62027d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:59:46.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.376+0000 7f6b3cb72640 1 --2- 172.21.15.1:0/2713551895 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605f61f4410 0x5605f62027d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:46.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.376+0000 7f6b3cb72640 1 -- 172.21.15.1:0/2713551895 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605f61fecb0 msgr2=0x5605f6202d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:59:46.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.376+0000 7f6b3cb72640 1 --2- 172.21.15.1:0/2713551895 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605f61fecb0 0x5605f6202d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:46.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.376+0000 7f6b3cb72640 1 -- 172.21.15.1:0/2713551895 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605f6203580 con 0x5605f61f3af0 2024-09-11T21:59:46.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.376+0000 7f6b3d373640 1 --2- 172.21.15.1:0/2713551895 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605f61fecb0 0x5605f6202d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T21:59:46.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.376+0000 7f6b3cb72640 1 --2- 172.21.15.1:0/2713551895 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605f61f3af0 0x5605f62074b0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f6b340267f0 tx=0x7f6b34014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a23a513abc804c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:59:46.382 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.376+0000 7f6b17fff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T21:59:46.382 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.376+0000 7f6b17fff640 1 -- 172.21.15.1:0/2713551895 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b34047060 con 0x5605f61f3af0 2024-09-11T21:59:46.382 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.376+0000 7f6b17fff640 1 -- 172.21.15.1:0/2713551895 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f6b3401db20 con 0x5605f61f3af0 2024-09-11T21:59:46.382 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.376+0000 7f6b41c4f740 1 -- 172.21.15.1:0/2713551895 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b30004050 con 0x5605f61f3af0 2024-09-11T21:59:46.382 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.376+0000 7f6b17fff640 1 -- 172.21.15.1:0/2713551895 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6b3400e920 con 0x5605f61f3af0 2024-09-11T21:59:46.382 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.376+0000 7f6b41c4f740 10 client.15915 fetch_fsmap learned FSMap version 28 2024-09-11T21:59:46.382 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.376+0000 7f6b41c4f740 1 -- 172.21.15.1:0/2713551895 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5605f6279770 con 0x5605f61f3af0 2024-09-11T21:59:46.382 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.377+0000 7f6b17fff640 1 -- 172.21.15.1:0/2713551895 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b30004050 con 0x5605f61f3af0 2024-09-11T21:59:46.382 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.377+0000 7f6b17fff640 1 -- 172.21.15.1:0/2713551895 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6b34003c70 con 0x5605f61f3af0 2024-09-11T21:59:46.382 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.377+0000 7f6b41c4f740 10 client.15915 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T21:59:46.382 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.377+0000 7f6b41c4f740 10 client.15915 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T21:59:46.382 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.377+0000 7f6b41c4f740 20 client.15915 populate_metadata read hostname 'smithi001' 2024-09-11T21:59:46.383 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.377+0000 7f6b41c4f740 1 --2- 172.21.15.1:0/2713551895 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5605f6279dd0 0x5605f627c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:46.383 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.377+0000 7f6b41c4f740 4 client.15915 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T21:59:46.383 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.377+0000 7f6b41c4f740 1 -- 172.21.15.1:0/2713551895 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5605f615fd70 con 0x5605f6279dd0 2024-09-11T21:59:46.383 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.377+0000 7f6b2ffff640 1 --2- 172.21.15.1:0/2713551895 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5605f6279dd0 0x5605f627c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:46.383 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.378+0000 7f6b2ffff640 1 --2- 172.21.15.1:0/2713551895 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5605f6279dd0 0x5605f627c1c0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2c2160a7d6589ae server_cookie=fbf481aef30dde2a in_seq=0 out_seq=0 2024-09-11T21:59:46.383 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.378+0000 7f6b17fff640 10 client.15915 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T21:59:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.397+0000 7f6b17fff640 1 -- 172.21.15.1:0/2713551895 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5605f615fd70 con 0x5605f6279dd0 2024-09-11T21:59:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.397+0000 7f6b17fff640 10 client.15915 handle_command_reply: tid=0 2024-09-11T21:59:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.397+0000 7f6b41c4f740 1 client.15915 shutdown 2024-09-11T21:59:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.397+0000 7f6b41c4f740 1 -- 172.21.15.1:0/2713551895 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605f61f3af0 msgr2=0x5605f62074b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:59:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.397+0000 7f6b41c4f740 1 --2- 172.21.15.1:0/2713551895 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605f61f3af0 0x5605f62074b0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f6b340267f0 tx=0x7f6b34014000 comp rx=0 tx=0).stop 2024-09-11T21:59:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.397+0000 7f6b41c4f740 1 -- 172.21.15.1:0/2713551895 shutdown_connections 2024-09-11T21:59:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.397+0000 7f6b41c4f740 1 --2- 172.21.15.1:0/2713551895 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5605f6279dd0 0x5605f627c1c0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.397+0000 7f6b41c4f740 1 --2- 172.21.15.1:0/2713551895 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605f61fecb0 0x5605f6202d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.397+0000 7f6b41c4f740 1 --2- 172.21.15.1:0/2713551895 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5605f61f4410 0x5605f62027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.397+0000 7f6b17fff640 0 client.15915 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T21:59:46.399 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.397+0000 7f6b41c4f740 1 --2- 172.21.15.1:0/2713551895 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605f61f3af0 0x5605f62074b0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:46.399 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.397+0000 7f6b41c4f740 1 -- 172.21.15.1:0/2713551895 >> 172.21.15.1:0/2713551895 conn(0x5605f61f0060 msgr2=0x5605f61f5550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:59:46.399 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.397+0000 7f6b41c4f740 1 -- 172.21.15.1:0/2713551895 shutdown_connections 2024-09-11T21:59:46.399 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.397+0000 7f6b41c4f740 1 -- 172.21.15.1:0/2713551895 wait complete. 2024-09-11T21:59:46.399 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.397+0000 7f6b41c4f740 20 client.15915 trim_cache size 0 max 16384 2024-09-11T21:59:46.399 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-09-11T21:59:46.399 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-09-11T21:59:46.399 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-09-11T21:59:46.399 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-09-11T21:59:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.398+0000 7f6b3edd6640 1 -- 172.21.15.1:0/3446783294 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f6b200a2e40 msgr2=0x7f6b200a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:59:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.398+0000 7f6b3edd6640 1 --2- 172.21.15.1:0/3446783294 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f6b200a2e40 0x7f6b200a5300 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6b340095f0 tx=0x7f6b34005f90 comp rx=0 tx=0).stop 2024-09-11T21:59:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.398+0000 7f6b3edd6640 1 -- 172.21.15.1:0/3446783294 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b38101130 msgr2=0x7f6b3819aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:59:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.398+0000 7f6b3edd6640 1 --2- 172.21.15.1:0/3446783294 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b38101130 0x7f6b3819aae0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f6b2802a910 tx=0x7f6b28003b20 comp rx=0 tx=0).stop 2024-09-11T21:59:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.398+0000 7f6b3edd6640 1 -- 172.21.15.1:0/3446783294 shutdown_connections 2024-09-11T21:59:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.398+0000 7f6b3edd6640 1 --2- 172.21.15.1:0/3446783294 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f6b200a2e40 0x7f6b200a5300 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:46.401 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.398+0000 7f6b3edd6640 1 --2- 172.21.15.1:0/3446783294 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b3810e2f0 0x7f6b3819ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:46.401 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.398+0000 7f6b3edd6640 1 --2- 172.21.15.1:0/3446783294 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b38101130 0x7f6b3819aae0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:46.401 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.398+0000 7f6b3edd6640 1 --2- 172.21.15.1:0/3446783294 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b38100810 0x7f6b3819a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:46.401 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.398+0000 7f6b3edd6640 1 -- 172.21.15.1:0/3446783294 >> 172.21.15.1:0/3446783294 conn(0x7f6b38075eb0 msgr2=0x7f6b3810e830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:59:46.401 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.398+0000 7f6b3edd6640 1 -- 172.21.15.1:0/3446783294 shutdown_connections 2024-09-11T21:59:46.401 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T21:59:46.398+0000 7f6b3edd6640 1 -- 172.21.15.1:0/3446783294 wait complete. 2024-09-11T21:59:47.001 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:59:47.001 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:59:47.091 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:59:46 smithi161 ceph-mon[35615]: pgmap v81: 97 pgs: 97 active+clean; 2.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-11T21:59:47.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:59:46 smithi001 ceph-mon[30935]: pgmap v81: 97 pgs: 97 active+clean; 2.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-11T21:59:47.210 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T21:59:47.210 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T21:59:47.232 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:59:47.232 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:59:47.233 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 49 -lt 100 ']' 2024-09-11T21:59:47.233 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 49 + 1 2024-09-11T21:59:47.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:59:46 smithi053 ceph-mon[34908]: pgmap v81: 97 pgs: 97 active+clean; 2.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-09-11T21:59:47.369 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=50 2024-09-11T21:59:47.369 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T21:59:47.527 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:59:47.527 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:59:47.528 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T21:59:47.528 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T21:59:47.528 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T21:59:48.091 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:59:47 smithi161 ceph-mon[35615]: pgmap v82: 97 pgs: 97 active+clean; 2.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 10 op/s 2024-09-11T21:59:48.139 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:59:47 smithi001 ceph-mon[30935]: pgmap v82: 97 pgs: 97 active+clean; 2.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 10 op/s 2024-09-11T21:59:48.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:59:47 smithi053 ceph-mon[34908]: pgmap v82: 97 pgs: 97 active+clean; 2.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 10 op/s 2024-09-11T21:59:48.758 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:59:48.758 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:59:49.194 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T21:59:49.195 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T21:59:49.241 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:59:49.241 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:59:49.241 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 50 -lt 100 ']' 2024-09-11T21:59:49.242 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 50 + 1 2024-09-11T21:59:49.344 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=51 2024-09-11T21:59:49.344 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T21:59:49.455 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:59:49.455 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:59:49.456 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T21:59:49.456 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T21:59:49.456 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T21:59:50.630 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:59:50.630 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:59:50.974 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T21:59:50.974 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T21:59:50.990 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:59:50.991 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:59:50.991 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 51 -lt 100 ']' 2024-09-11T21:59:50.991 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 51 + 1 2024-09-11T21:59:51.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:59:50 smithi053 ceph-mon[34908]: pgmap v83: 97 pgs: 97 active+clean; 2.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 28 KiB/s wr, 21 op/s 2024-09-11T21:59:51.071 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=52 2024-09-11T21:59:51.071 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T21:59:51.091 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:59:50 smithi161 ceph-mon[35615]: pgmap v83: 97 pgs: 97 active+clean; 2.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 28 KiB/s wr, 21 op/s 2024-09-11T21:59:51.135 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:59:51.135 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:59:51.136 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T21:59:51.136 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T21:59:51.137 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T21:59:51.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:59:50 smithi001 ceph-mon[30935]: pgmap v83: 97 pgs: 97 active+clean; 2.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 28 KiB/s wr, 21 op/s 2024-09-11T21:59:52.192 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:59:52.192 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:59:52.382 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T21:59:52.382 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T21:59:52.407 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:59:52.407 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T21:59:52.407 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 52 -lt 100 ']' 2024-09-11T21:59:52.408 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 52 + 1 2024-09-11T21:59:52.495 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=53 2024-09-11T21:59:52.495 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T21:59:52.567 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:59:52.567 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:59:52.568 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T21:59:52.569 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T21:59:52.569 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T21:59:52.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.890+0000 7f0b8b096640 1 -- 172.21.15.1:0/3433718566 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b84102d90 msgr2=0x7f0b84103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:59:52.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.890+0000 7f0b8b096640 1 --2- 172.21.15.1:0/3433718566 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b84102d90 0x7f0b84103170 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f0b74009930 tx=0x7f0b7402a420 comp rx=0 tx=0).stop 2024-09-11T21:59:52.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.890+0000 7f0b8b096640 1 -- 172.21.15.1:0/3433718566 shutdown_connections 2024-09-11T21:59:52.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.890+0000 7f0b8b096640 1 --2- 172.21.15.1:0/3433718566 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b84108290 0x7f0b8410ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:52.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.890+0000 7f0b8b096640 1 --2- 172.21.15.1:0/3433718566 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b84103740 0x7f0b84107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:52.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.890+0000 7f0b8b096640 1 --2- 172.21.15.1:0/3433718566 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b84102d90 0x7f0b84103170 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:52.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.890+0000 7f0b8b096640 1 -- 172.21.15.1:0/3433718566 >> 172.21.15.1:0/3433718566 conn(0x7f0b840fc7d0 msgr2=0x7f0b840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:59:52.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.890+0000 7f0b8b096640 1 -- 172.21.15.1:0/3433718566 shutdown_connections 2024-09-11T21:59:52.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.890+0000 7f0b8b096640 1 -- 172.21.15.1:0/3433718566 wait complete. 2024-09-11T21:59:52.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.891+0000 7f0b8b096640 1 Processor -- start 2024-09-11T21:59:52.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.891+0000 7f0b8b096640 1 -- start start 2024-09-11T21:59:52.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.891+0000 7f0b8b096640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b84102d90 0x7f0b841009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:52.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.891+0000 7f0b8b096640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b84103740 0x7f0b84100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:52.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.891+0000 7f0b8b096640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b84108290 0x7f0b84101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:52.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.891+0000 7f0b8b096640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0b841120b0 con 0x7f0b84102d90 2024-09-11T21:59:52.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.891+0000 7f0b8b096640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0b84111f30 con 0x7f0b84108290 2024-09-11T21:59:52.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.891+0000 7f0b8b096640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0b84112230 con 0x7f0b84103740 2024-09-11T21:59:52.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.892+0000 7f0b88e32640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b84102d90 0x7f0b841009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:52.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.892+0000 7f0b88e32640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b84102d90 0x7f0b841009f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35300/0 (socket says 172.21.15.1:35300) 2024-09-11T21:59:52.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.892+0000 7f0b88e32640 1 -- 172.21.15.1:0/2362790814 learned_addr learned my addr 172.21.15.1:0/2362790814 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:59:52.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.892+0000 7f0b89633640 1 --2- 172.21.15.1:0/2362790814 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b84108290 0x7f0b84101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:52.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.892+0000 7f0b83fff640 1 --2- 172.21.15.1:0/2362790814 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b84103740 0x7f0b84100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:52.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.892+0000 7f0b88e32640 1 -- 172.21.15.1:0/2362790814 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b84103740 msgr2=0x7f0b84100f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:59:52.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.892+0000 7f0b88e32640 1 --2- 172.21.15.1:0/2362790814 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b84103740 0x7f0b84100f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:52.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.892+0000 7f0b88e32640 1 -- 172.21.15.1:0/2362790814 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b84108290 msgr2=0x7f0b84101470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:59:52.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.892+0000 7f0b88e32640 1 --2- 172.21.15.1:0/2362790814 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b84108290 0x7f0b84101470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:52.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.892+0000 7f0b88e32640 1 -- 172.21.15.1:0/2362790814 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b841a6a80 con 0x7f0b84102d90 2024-09-11T21:59:52.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.892+0000 7f0b89633640 1 --2- 172.21.15.1:0/2362790814 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b84108290 0x7f0b84101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T21:59:52.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.893+0000 7f0b88e32640 1 --2- 172.21.15.1:0/2362790814 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b84102d90 0x7f0b841009f0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f0b7402a940 tx=0x7f0b74003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:59:52.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.893+0000 7f0b81ffb640 1 -- 172.21.15.1:0/2362790814 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7403f070 con 0x7f0b84102d90 2024-09-11T21:59:52.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.893+0000 7f0b81ffb640 1 -- 172.21.15.1:0/2362790814 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b74031c90 con 0x7f0b84102d90 2024-09-11T21:59:52.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.893+0000 7f0b8b096640 1 -- 172.21.15.1:0/2362790814 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b841a6d10 con 0x7f0b84102d90 2024-09-11T21:59:52.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.893+0000 7f0b81ffb640 1 -- 172.21.15.1:0/2362790814 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b74035030 con 0x7f0b84102d90 2024-09-11T21:59:52.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.893+0000 7f0b8b096640 1 -- 172.21.15.1:0/2362790814 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b841a71e0 con 0x7f0b84102d90 2024-09-11T21:59:52.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.897+0000 7f0b81ffb640 1 -- 172.21.15.1:0/2362790814 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f0b74042070 con 0x7f0b84102d90 2024-09-11T21:59:52.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.898+0000 7f0b81ffb640 1 --2- 172.21.15.1:0/2362790814 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f0b700a2d70 0x7f0b700a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:52.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.898+0000 7f0b83fff640 1 --2- 172.21.15.1:0/2362790814 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f0b700a2d70 0x7f0b700a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:52.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.898+0000 7f0b81ffb640 1 -- 172.21.15.1:0/2362790814 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f0b740efaa0 con 0x7f0b84102d90 2024-09-11T21:59:52.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.898+0000 7fc1a444a640 1 -- 172.21.15.1:0/348622736 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc19c102790 msgr2=0x7fc19c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:59:52.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.898+0000 7fc1a444a640 1 --2- 172.21.15.1:0/348622736 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc19c102790 0x7fc19c1067e0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fc18c009900 tx=0x7fc18c02a410 comp rx=0 tx=0).stop 2024-09-11T21:59:52.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.898+0000 7fc1a444a640 1 -- 172.21.15.1:0/348622736 shutdown_connections 2024-09-11T21:59:52.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.898+0000 7fc1a444a640 1 --2- 172.21.15.1:0/348622736 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc19c106f10 0x7fc19c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:52.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.898+0000 7fc1a444a640 1 --2- 172.21.15.1:0/348622736 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc19c102790 0x7fc19c1067e0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:52.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.898+0000 7fc1a444a640 1 --2- 172.21.15.1:0/348622736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc19c101de0 0x7fc19c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:52.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.898+0000 7fc1a444a640 1 -- 172.21.15.1:0/348622736 >> 172.21.15.1:0/348622736 conn(0x7fc19c0fd8e0 msgr2=0x7fc19c0ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:59:52.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.898+0000 7fc1a444a640 1 -- 172.21.15.1:0/348622736 shutdown_connections 2024-09-11T21:59:52.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.898+0000 7fc1a444a640 1 -- 172.21.15.1:0/348622736 wait complete. 2024-09-11T21:59:52.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.899+0000 7f0b83fff640 1 --2- 172.21.15.1:0/2362790814 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f0b700a2d70 0x7f0b700a5230 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0b84102480 tx=0x7f0b6c00b390 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:59:52.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.899+0000 7fc1a444a640 1 Processor -- start 2024-09-11T21:59:52.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.899+0000 7fc1a444a640 1 -- start start 2024-09-11T21:59:52.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.900+0000 7fc1a444a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc19c101de0 0x7fc19c19a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:52.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.900+0000 7fc1a444a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc19c102790 0x7fc19c19ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:52.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.900+0000 7fc1a444a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc19c106f10 0x7fc19c19ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:52.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.900+0000 7fc1a444a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc19c111f70 con 0x7fc19c106f10 2024-09-11T21:59:52.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.900+0000 7fc1a444a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc19c111df0 con 0x7fc19c101de0 2024-09-11T21:59:52.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.900+0000 7fc1a444a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc19c1120f0 con 0x7fc19c102790 2024-09-11T21:59:52.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.900+0000 7fc1a29e7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc19c106f10 0x7fc19c19ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:52.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.900+0000 7fc1a29e7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc19c106f10 0x7fc19c19ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35320/0 (socket says 172.21.15.1:35320) 2024-09-11T21:59:52.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.900+0000 7fc1a29e7640 1 -- 172.21.15.1:0/2585013667 learned_addr learned my addr 172.21.15.1:0/2585013667 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:59:52.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.900+0000 7fc1a21e6640 1 --2- 172.21.15.1:0/2585013667 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc19c101de0 0x7fc19c19a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:52.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.900+0000 7fc1a19e5640 1 --2- 172.21.15.1:0/2585013667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc19c102790 0x7fc19c19ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:52.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.900+0000 7fc1a29e7640 1 -- 172.21.15.1:0/2585013667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc19c102790 msgr2=0x7fc19c19ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:59:52.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.900+0000 7fc1a29e7640 1 --2- 172.21.15.1:0/2585013667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc19c102790 0x7fc19c19ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:52.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.900+0000 7fc1a29e7640 1 -- 172.21.15.1:0/2585013667 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc19c101de0 msgr2=0x7fc19c19a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:59:52.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.900+0000 7fc1a29e7640 1 --2- 172.21.15.1:0/2585013667 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc19c101de0 0x7fc19c19a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:52.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.900+0000 7fc1a29e7640 1 -- 172.21.15.1:0/2585013667 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc19c19f5e0 con 0x7fc19c106f10 2024-09-11T21:59:52.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.901+0000 7fc1a29e7640 1 --2- 172.21.15.1:0/2585013667 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc19c106f10 0x7fc19c19ef00 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fc19400eb10 tx=0x7fc19400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:59:52.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.901+0000 7fc1937fe640 1 -- 172.21.15.1:0/2585013667 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc194004b20 con 0x7fc19c106f10 2024-09-11T21:59:52.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.901+0000 7fc1a444a640 1 -- 172.21.15.1:0/2585013667 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc19c19f8d0 con 0x7fc19c106f10 2024-09-11T21:59:52.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.901+0000 7fc1937fe640 1 -- 172.21.15.1:0/2585013667 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc194004cc0 con 0x7fc19c106f10 2024-09-11T21:59:52.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.901+0000 7fc1937fe640 1 -- 172.21.15.1:0/2585013667 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc194010760 con 0x7fc19c106f10 2024-09-11T21:59:52.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.901+0000 7fc1a444a640 1 -- 172.21.15.1:0/2585013667 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc19c1a71b0 con 0x7fc19c106f10 2024-09-11T21:59:52.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.904+0000 7fc1937fe640 1 -- 172.21.15.1:0/2585013667 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7fc194005020 con 0x7fc19c106f10 2024-09-11T21:59:52.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.904+0000 7fc1937fe640 1 --2- 172.21.15.1:0/2585013667 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fc1880a2cf0 0x7fc1880a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:52.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.905+0000 7fc1a21e6640 1 --2- 172.21.15.1:0/2585013667 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fc1880a2cf0 0x7fc1880a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:52.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.905+0000 7fc1937fe640 1 -- 172.21.15.1:0/2585013667 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7fc1940cfad0 con 0x7fc19c106f10 2024-09-11T21:59:52.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.905+0000 7fc1a21e6640 1 --2- 172.21.15.1:0/2585013667 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fc1880a2cf0 0x7fc1880a51b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc198005d00 tx=0x7fc19800b410 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:59:52.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.958+0000 7f0b8d46f740 1 Processor -- start 2024-09-11T21:59:52.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.958+0000 7fc1a6823740 1 Processor -- start 2024-09-11T21:59:52.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.958+0000 7f0b8d46f740 1 -- start start 2024-09-11T21:59:52.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.959+0000 7f0b8d46f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d8e9b9b40 0x560d8e9b8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:52.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.959+0000 7f0b8d46f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d8e9b85d0 0x560d8e9b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:52.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.959+0000 7fc1a6823740 1 -- start start 2024-09-11T21:59:52.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.959+0000 7f0b8d46f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d8e9cbb80 0x560d8e9cdf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:52.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.959+0000 7f0b8d46f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x560d8e91e2f0 con 0x560d8e9b85d0 2024-09-11T21:59:52.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.959+0000 7f0b8d46f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x560d8e8c5cf0 con 0x560d8e9cbb80 2024-09-11T21:59:52.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.959+0000 7f0b8d46f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x560d8e91a2b0 con 0x560d8e9b9b40 2024-09-11T21:59:52.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.959+0000 7fc1a6823740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564fd8b8b760 0x564fd8b8bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:52.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.959+0000 7fc1a6823740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564fd8b8c080 0x564fd8b96400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:52.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.959+0000 7fc1a6823740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564fd8b96940 0x564fd8b98d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:52.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.959+0000 7f0b83fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d8e9b85d0 0x560d8e9b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:52.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.959+0000 7f0b88e32640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d8e9b9b40 0x560d8e9b8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:52.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.959+0000 7f0b89633640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d8e9cbb80 0x560d8e9cdf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:52.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.959+0000 7fc1a19e5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564fd8b8c080 0x564fd8b96400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:52.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.959+0000 7f0b88e32640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d8e9b9b40 0x560d8e9b8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:39688/0 (socket says 172.21.15.1:39688) 2024-09-11T21:59:52.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.959+0000 7f0b88e32640 1 -- 172.21.15.1:0/2328573792 learned_addr learned my addr 172.21.15.1:0/2328573792 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:59:52.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.959+0000 7f0b89633640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d8e9cbb80 0x560d8e9cdf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:47844/0 (socket says 172.21.15.1:47844) 2024-09-11T21:59:52.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.959+0000 7fc1a21e6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564fd8b8b760 0x564fd8b8bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:52.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.959+0000 7fc1a19e5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564fd8b8c080 0x564fd8b96400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:47852/0 (socket says 172.21.15.1:47852) 2024-09-11T21:59:52.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.959+0000 7fc1a19e5640 1 -- 172.21.15.1:0/468527047 learned_addr learned my addr 172.21.15.1:0/468527047 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:59:52.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.959+0000 7fc1a6823740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x564fd8af12a0 con 0x564fd8b8b760 2024-09-11T21:59:52.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.959+0000 7fc1a6823740 1 -- 172.21.15.1:0/468527047 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x564fd8a98b10 con 0x564fd8b8c080 2024-09-11T21:59:52.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.959+0000 7fc1a6823740 1 -- 172.21.15.1:0/468527047 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x564fd8aed260 con 0x564fd8b96940 2024-09-11T21:59:52.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.959+0000 7fc1a29e7640 1 --2- 172.21.15.1:0/468527047 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564fd8b96940 0x564fd8b98d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:52.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.959+0000 7f0b88e32640 1 -- 172.21.15.1:0/2328573792 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d8e9cbb80 msgr2=0x560d8e9cdf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:59:52.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.959+0000 7f0b88e32640 1 --2- 172.21.15.1:0/2328573792 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d8e9cbb80 0x560d8e9cdf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:52.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.959+0000 7f0b88e32640 1 -- 172.21.15.1:0/2328573792 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d8e9b85d0 msgr2=0x560d8e9b8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:59:52.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.959+0000 7f0b88e32640 1 --2- 172.21.15.1:0/2328573792 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d8e9b85d0 0x560d8e9b8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:52.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.960+0000 7f0b88e32640 1 -- 172.21.15.1:0/2328573792 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d8e8aa3a0 con 0x560d8e9b9b40 2024-09-11T21:59:52.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.960+0000 7fc1a29e7640 1 -- 172.21.15.1:0/468527047 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564fd8b8c080 msgr2=0x564fd8b96400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:59:52.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.960+0000 7fc1a29e7640 1 --2- 172.21.15.1:0/468527047 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564fd8b8c080 0x564fd8b96400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:52.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.960+0000 7fc1a29e7640 1 -- 172.21.15.1:0/468527047 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564fd8b8b760 msgr2=0x564fd8b8bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:59:52.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.960+0000 7fc1a29e7640 1 --2- 172.21.15.1:0/468527047 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564fd8b8b760 0x564fd8b8bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:52.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.960+0000 7fc1a29e7640 1 -- 172.21.15.1:0/468527047 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fd8862b90 con 0x564fd8b96940 2024-09-11T21:59:52.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.960+0000 7fc1a19e5640 1 --2- 172.21.15.1:0/468527047 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564fd8b8c080 0x564fd8b96400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T21:59:52.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.960+0000 7f0b88e32640 1 --2- 172.21.15.1:0/2328573792 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d8e9b9b40 0x560d8e9b8090 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0b740c46c0 tx=0x7f0b74005e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c69e163c4f167d90 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:59:52.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.960+0000 7fc1a29e7640 1 --2- 172.21.15.1:0/468527047 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564fd8b96940 0x564fd8b98d30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc1940cd1e0 tx=0x7fc194098370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=436c17224d166195 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:59:52.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.960+0000 7f0b7affd640 1 -- 172.21.15.1:0/2328573792 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b74049070 con 0x560d8e9b9b40 2024-09-11T21:59:52.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.960+0000 7f0b7affd640 1 -- 172.21.15.1:0/2328573792 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b740c4bf0 con 0x560d8e9b9b40 2024-09-11T21:59:52.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.960+0000 7f0b7affd640 1 -- 172.21.15.1:0/2328573792 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b740c4760 con 0x560d8e9b9b40 2024-09-11T21:59:52.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.960+0000 7fc190dea640 1 -- 172.21.15.1:0/468527047 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19409c6b0 con 0x564fd8b96940 2024-09-11T21:59:52.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.960+0000 7fc190dea640 1 -- 172.21.15.1:0/468527047 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1940bc070 con 0x564fd8b96940 2024-09-11T21:59:52.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.960+0000 7fc190dea640 1 -- 172.21.15.1:0/468527047 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc194005310 con 0x564fd8b96940 2024-09-11T21:59:52.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.961+0000 7f0b8d46f740 1 -- 172.21.15.1:0/2328573792 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d8e9b9b40 msgr2=0x560d8e9b8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:59:52.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.961+0000 7f0b8d46f740 1 --2- 172.21.15.1:0/2328573792 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d8e9b9b40 0x560d8e9b8090 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0b740c46c0 tx=0x7f0b74005e50 comp rx=0 tx=0).stop 2024-09-11T21:59:52.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.961+0000 7fc1a6823740 1 -- 172.21.15.1:0/468527047 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564fd8b96940 msgr2=0x564fd8b98d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:59:52.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.961+0000 7fc1a6823740 1 --2- 172.21.15.1:0/468527047 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564fd8b96940 0x564fd8b98d30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc1940cd1e0 tx=0x7fc194098370 comp rx=0 tx=0).stop 2024-09-11T21:59:52.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.961+0000 7f0b8d46f740 1 -- 172.21.15.1:0/2328573792 shutdown_connections 2024-09-11T21:59:52.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.961+0000 7f0b8d46f740 1 --2- 172.21.15.1:0/2328573792 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d8e9cbb80 0x560d8e9cdf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:52.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.961+0000 7f0b8d46f740 1 --2- 172.21.15.1:0/2328573792 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d8e9b85d0 0x560d8e9b8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:52.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.961+0000 7f0b8d46f740 1 --2- 172.21.15.1:0/2328573792 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d8e9b9b40 0x560d8e9b8090 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:52.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.961+0000 7fc1a6823740 1 -- 172.21.15.1:0/468527047 shutdown_connections 2024-09-11T21:59:52.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.961+0000 7fc1a6823740 1 --2- 172.21.15.1:0/468527047 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564fd8b96940 0x564fd8b98d30 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:52.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.961+0000 7f0b8d46f740 1 -- 172.21.15.1:0/2328573792 >> 172.21.15.1:0/2328573792 conn(0x560d8e981310 msgr2=0x560d8e9b4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:59:52.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.961+0000 7fc1a6823740 1 --2- 172.21.15.1:0/468527047 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564fd8b8c080 0x564fd8b96400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:52.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.961+0000 7fc1a6823740 1 --2- 172.21.15.1:0/468527047 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564fd8b8b760 0x564fd8b8bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:52.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.961+0000 7fc1a6823740 1 -- 172.21.15.1:0/468527047 >> 172.21.15.1:0/468527047 conn(0x564fd8b87cf0 msgr2=0x564fd8b880d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:59:52.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.962+0000 7fc1a6823740 1 -- 172.21.15.1:0/468527047 shutdown_connections 2024-09-11T21:59:52.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.962+0000 7fc1a6823740 1 -- 172.21.15.1:0/468527047 wait complete. 2024-09-11T21:59:52.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.962+0000 7f0b8d46f740 1 -- 172.21.15.1:0/2328573792 shutdown_connections 2024-09-11T21:59:52.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.962+0000 7f0b8d46f740 1 -- 172.21.15.1:0/2328573792 wait complete. 2024-09-11T21:59:52.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.962+0000 7fc1a6823740 1 -- start start 2024-09-11T21:59:52.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.962+0000 7f0b8d46f740 1 -- start start 2024-09-11T21:59:52.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.962+0000 7fc1a6823740 1 Processor -- start 2024-09-11T21:59:52.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.962+0000 7f0b8d46f740 1 Processor -- start 2024-09-11T21:59:52.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.963+0000 7fc1a6823740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564fd8b8b760 0x564fd8b937c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:52.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.963+0000 7fc1a6823740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564fd8b8c080 0x564fd8b93d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:52.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.963+0000 7fc1a6823740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564fd8b96940 0x564fd8b94240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:52.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.963+0000 7fc1a6823740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x564fd8b99510 con 0x564fd8b8b760 2024-09-11T21:59:52.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.963+0000 7fc1a6823740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x564fd8a796e0 con 0x564fd8b96940 2024-09-11T21:59:52.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.963+0000 7fc1a6823740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x564fd8b99690 con 0x564fd8b8c080 2024-09-11T21:59:52.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.963+0000 7fc1a6823740 10 client.0 _ll_register_callbacks cb 0x564fd8a9da80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:59:52.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.963+0000 7fc1a19e5640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564fd8b8c080 0x564fd8b93d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:52.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.963+0000 7fc1a21e6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564fd8b8b760 0x564fd8b937c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:52.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.963+0000 7fc1a19e5640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564fd8b8c080 0x564fd8b93d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:39708/0 (socket says 172.21.15.1:39708) 2024-09-11T21:59:52.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.963+0000 7fc1a19e5640 1 -- 172.21.15.1:0/3280272402 learned_addr learned my addr 172.21.15.1:0/3280272402 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:59:52.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.963+0000 7fc1a29e7640 1 --2- 172.21.15.1:0/3280272402 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564fd8b96940 0x564fd8b94240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:52.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.963+0000 7f0b8d46f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d8e9b85d0 0x560d8e9cb550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:52.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.963+0000 7f0b8d46f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d8e9b9b40 0x560d8e9c8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:52.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.963+0000 7f0b8d46f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d8e9cbb80 0x560d8e9c8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:52.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.963+0000 7f0b8d46f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x560d8e91a2b0 con 0x560d8e9b85d0 2024-09-11T21:59:52.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.963+0000 7f0b8d46f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x560d8e8d9060 con 0x560d8e9b9b40 2024-09-11T21:59:52.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.963+0000 7f0b8d46f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x560d8e9ce640 con 0x560d8e9cbb80 2024-09-11T21:59:52.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.963+0000 7f0b8d46f740 10 client.0 _ll_register_callbacks cb 0x560d8e62c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:59:52.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.963+0000 7fc1a19e5640 1 -- 172.21.15.1:0/3280272402 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564fd8b96940 msgr2=0x564fd8b94240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:59:52.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.963+0000 7fc1a19e5640 1 --2- 172.21.15.1:0/3280272402 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564fd8b96940 0x564fd8b94240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:52.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.963+0000 7fc1a19e5640 1 -- 172.21.15.1:0/3280272402 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564fd8b8b760 msgr2=0x564fd8b937c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:59:52.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.963+0000 7fc1a19e5640 1 --2- 172.21.15.1:0/3280272402 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564fd8b8b760 0x564fd8b937c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:52.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.963+0000 7fc1a19e5640 1 -- 172.21.15.1:0/3280272402 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fd8ba29b0 con 0x564fd8b8c080 2024-09-11T21:59:52.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.963+0000 7f0b83fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d8e9b9b40 0x560d8e9c8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:52.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.963+0000 7f0b83fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d8e9b9b40 0x560d8e9c8850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:47874/0 (socket says 172.21.15.1:47874) 2024-09-11T21:59:52.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.964+0000 7f0b83fff640 1 -- 172.21.15.1:0/149965296 learned_addr learned my addr 172.21.15.1:0/149965296 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:59:52.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.964+0000 7f0b88e32640 1 --2- 172.21.15.1:0/149965296 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d8e9b85d0 0x560d8e9cb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:52.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.964+0000 7fc1a19e5640 1 --2- 172.21.15.1:0/3280272402 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564fd8b8c080 0x564fd8b93d00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc18c009230 tx=0x7fc18c031860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=770a5692334bc033 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:59:52.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.964+0000 7fc185ffb640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-11T21:59:52.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.964+0000 7f0b89633640 1 --2- 172.21.15.1:0/149965296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d8e9cbb80 0x560d8e9c8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:52.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.964+0000 7f0b83fff640 1 -- 172.21.15.1:0/149965296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d8e9cbb80 msgr2=0x560d8e9c8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:59:52.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.964+0000 7f0b83fff640 1 --2- 172.21.15.1:0/149965296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d8e9cbb80 0x560d8e9c8eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:52.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.964+0000 7f0b83fff640 1 -- 172.21.15.1:0/149965296 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d8e9b85d0 msgr2=0x560d8e9cb550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:59:52.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.964+0000 7f0b83fff640 1 --2- 172.21.15.1:0/149965296 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d8e9b85d0 0x560d8e9cb550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:52.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.964+0000 7f0b83fff640 1 -- 172.21.15.1:0/149965296 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d8e68f3c0 con 0x560d8e9b9b40 2024-09-11T21:59:52.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.964+0000 7fc185ffb640 1 -- 172.21.15.1:0/3280272402 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c035040 con 0x564fd8b8c080 2024-09-11T21:59:52.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.964+0000 7fc185ffb640 1 -- 172.21.15.1:0/3280272402 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7fc18c004440 con 0x564fd8b8c080 2024-09-11T21:59:52.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.964+0000 7fc1a6823740 1 -- 172.21.15.1:0/3280272402 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1940bc070 con 0x564fd8b8c080 2024-09-11T21:59:52.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.964+0000 7f0b83fff640 1 --2- 172.21.15.1:0/149965296 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d8e9b9b40 0x560d8e9c8850 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c0116c0 tx=0x7f0b6c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b949f697e893db30 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:59:52.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.964+0000 7f0b63fff640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-11T21:59:52.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.965+0000 7fc185ffb640 1 -- 172.21.15.1:0/3280272402 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc18c005d50 con 0x564fd8b8c080 2024-09-11T21:59:52.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.965+0000 7fc1a6823740 10 client.25492 fetch_fsmap learned FSMap version 28 2024-09-11T21:59:52.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.965+0000 7fc1a6823740 1 -- 172.21.15.1:0/3280272402 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564fd8ba5320 con 0x564fd8b8c080 2024-09-11T21:59:52.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.965+0000 7f0b63fff640 1 -- 172.21.15.1:0/149965296 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c005220 con 0x560d8e9b9b40 2024-09-11T21:59:52.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.965+0000 7f0b8d46f740 1 -- 172.21.15.1:0/149965296 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b740c4bf0 con 0x560d8e9b9b40 2024-09-11T21:59:52.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.965+0000 7f0b63fff640 1 -- 172.21.15.1:0/149965296 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f0b6c0053c0 con 0x560d8e9b9b40 2024-09-11T21:59:52.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.965+0000 7f0b63fff640 1 -- 172.21.15.1:0/149965296 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0b6c0055c0 con 0x560d8e9b9b40 2024-09-11T21:59:52.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.965+0000 7f0b8d46f740 10 client.25520 fetch_fsmap learned FSMap version 28 2024-09-11T21:59:52.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.965+0000 7f0b8d46f740 1 -- 172.21.15.1:0/149965296 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560d8e9da020 con 0x560d8e9b9b40 2024-09-11T21:59:52.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.965+0000 7fc185ffb640 1 -- 172.21.15.1:0/3280272402 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1940bc070 con 0x564fd8b8c080 2024-09-11T21:59:52.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.965+0000 7fc185ffb640 1 -- 172.21.15.1:0/3280272402 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc18c0035f0 con 0x564fd8b8c080 2024-09-11T21:59:52.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.965+0000 7fc1a6823740 10 client.25492 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T21:59:52.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.965+0000 7fc1a6823740 10 client.25492 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-11T21:59:52.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.965+0000 7fc1a6823740 20 client.25492 populate_metadata read hostname 'smithi001' 2024-09-11T21:59:52.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.965+0000 7fc1a6823740 1 --2- 172.21.15.1:0/3280272402 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x564fd8ba67e0 0x564fd8ba8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:52.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.965+0000 7f0b63fff640 1 -- 172.21.15.1:0/149965296 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b740c4bf0 con 0x560d8e9b9b40 2024-09-11T21:59:52.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.965+0000 7fc1a6823740 4 client.25492 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T21:59:52.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.966+0000 7fc1a21e6640 1 --2- 172.21.15.1:0/3280272402 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x564fd8ba67e0 0x564fd8ba8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:52.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.966+0000 7fc1a6823740 1 -- 172.21.15.1:0/3280272402 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564fd8afe9d0 con 0x564fd8ba67e0 2024-09-11T21:59:52.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.966+0000 7f0b63fff640 1 -- 172.21.15.1:0/149965296 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f0b6c01fc70 con 0x560d8e9b9b40 2024-09-11T21:59:52.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.966+0000 7f0b8d46f740 10 client.25520 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T21:59:52.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.966+0000 7f0b8d46f740 10 client.25520 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-11T21:59:52.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.966+0000 7f0b8d46f740 20 client.25520 populate_metadata read hostname 'smithi001' 2024-09-11T21:59:52.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.966+0000 7f0b8d46f740 1 --2- 172.21.15.1:0/149965296 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x560d8e9db430 0x560d8e9dd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:52.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.966+0000 7f0b8d46f740 4 client.25520 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T21:59:52.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.966+0000 7f0b8d46f740 1 -- 172.21.15.1:0/149965296 --> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d8e92b860 con 0x560d8e9db430 2024-09-11T21:59:52.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.966+0000 7fc185ffb640 10 client.25492 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T21:59:52.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.966+0000 7f0b88e32640 1 --2- 172.21.15.1:0/149965296 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x560d8e9db430 0x560d8e9dd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:52.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.966+0000 7fc1a21e6640 1 --2- 172.21.15.1:0/3280272402 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x564fd8ba67e0 0x564fd8ba8c10 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36542548c367dd21 server_cookie=ffcfa50eb3803bdf in_seq=0 out_seq=0 2024-09-11T21:59:52.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.967+0000 7f0b63fff640 10 client.25520 ms_handle_connect on v2:172.21.15.161:6834/828381592 2024-09-11T21:59:52.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.967+0000 7f0b88e32640 1 --2- 172.21.15.1:0/149965296 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x560d8e9db430 0x560d8e9dd890 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5777f4882acfe947 server_cookie=23ba6a8938678e8c in_seq=0 out_seq=0 2024-09-11T21:59:52.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.967+0000 7fc185ffb640 1 -- 172.21.15.1:0/3280272402 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564fd8afe9d0 con 0x564fd8ba67e0 2024-09-11T21:59:52.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.967+0000 7fc185ffb640 10 client.25492 handle_command_reply: tid=0 2024-09-11T21:59:52.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.967+0000 7fc1a6823740 1 client.25492 shutdown 2024-09-11T21:59:52.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.967+0000 7fc1a6823740 1 -- 172.21.15.1:0/3280272402 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564fd8b8c080 msgr2=0x564fd8b93d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:59:52.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.967+0000 7fc1a6823740 1 --2- 172.21.15.1:0/3280272402 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564fd8b8c080 0x564fd8b93d00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc18c009230 tx=0x7fc18c031860 comp rx=0 tx=0).stop 2024-09-11T21:59:52.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.967+0000 7fc1a6823740 1 -- 172.21.15.1:0/3280272402 shutdown_connections 2024-09-11T21:59:52.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.967+0000 7fc1a6823740 1 --2- 172.21.15.1:0/3280272402 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x564fd8ba67e0 0x564fd8ba8c10 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:52.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.967+0000 7fc1a6823740 1 --2- 172.21.15.1:0/3280272402 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564fd8b96940 0x564fd8b94240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:52.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.967+0000 7fc1a6823740 1 --2- 172.21.15.1:0/3280272402 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564fd8b8c080 0x564fd8b93d00 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:52.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.967+0000 7fc1a6823740 1 --2- 172.21.15.1:0/3280272402 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564fd8b8b760 0x564fd8b937c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:52.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.967+0000 7fc185ffb640 0 client.25492 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T21:59:52.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.967+0000 7fc1a6823740 1 -- 172.21.15.1:0/3280272402 >> 172.21.15.1:0/3280272402 conn(0x564fd8b87cf0 msgr2=0x564fd8b8da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:59:52.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.967+0000 7fc1a6823740 1 -- 172.21.15.1:0/3280272402 shutdown_connections 2024-09-11T21:59:52.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.967+0000 7fc1a6823740 1 -- 172.21.15.1:0/3280272402 wait complete. 2024-09-11T21:59:52.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.968+0000 7fc1a6823740 20 client.25492 trim_cache size 0 max 16384 2024-09-11T21:59:52.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.968+0000 7f0b63fff640 1 -- 172.21.15.1:0/149965296 <== mds.4 v2:172.21.15.161:6834/828381592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f0b7409ae50 con 0x560d8e9db430 2024-09-11T21:59:52.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.968+0000 7f0b63fff640 10 client.25520 handle_command_reply: tid=0 2024-09-11T21:59:52.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.968+0000 7f0b8d46f740 1 client.25520 shutdown 2024-09-11T21:59:52.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.968+0000 7f0b8d46f740 1 -- 172.21.15.1:0/149965296 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d8e9b9b40 msgr2=0x560d8e9c8850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:59:52.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.968+0000 7f0b8d46f740 1 --2- 172.21.15.1:0/149965296 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d8e9b9b40 0x560d8e9c8850 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c0116c0 tx=0x7f0b6c004730 comp rx=0 tx=0).stop 2024-09-11T21:59:52.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.968+0000 7f0b8d46f740 1 -- 172.21.15.1:0/149965296 shutdown_connections 2024-09-11T21:59:52.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.968+0000 7f0b8d46f740 1 --2- 172.21.15.1:0/149965296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d8e9cbb80 0x560d8e9c8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:52.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.968+0000 7f0b8d46f740 1 --2- 172.21.15.1:0/149965296 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x560d8e9db430 0x560d8e9dd890 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:52.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.968+0000 7f0b8d46f740 1 --2- 172.21.15.1:0/149965296 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d8e9b9b40 0x560d8e9c8850 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:52.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.968+0000 7f0b63fff640 0 client.25520 ms_handle_reset on v2:172.21.15.161:6834/828381592 2024-09-11T21:59:52.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.968+0000 7f0b8d46f740 1 --2- 172.21.15.1:0/149965296 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d8e9b85d0 0x560d8e9cb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:52.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.968+0000 7f0b8d46f740 1 -- 172.21.15.1:0/149965296 >> 172.21.15.1:0/149965296 conn(0x560d8e981310 msgr2=0x560d8e9ba960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:59:52.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.968+0000 7f0b8d46f740 1 -- 172.21.15.1:0/149965296 shutdown_connections 2024-09-11T21:59:52.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.968+0000 7f0b8d46f740 1 -- 172.21.15.1:0/149965296 wait complete. 2024-09-11T21:59:52.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.968+0000 7f0b8d46f740 20 client.25520 trim_cache size 0 max 16384 2024-09-11T21:59:52.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.982+0000 7f0b8d46f740 1 Processor -- start 2024-09-11T21:59:52.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.982+0000 7f0b8d46f740 1 -- start start 2024-09-11T21:59:52.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.982+0000 7f0b8d46f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d8e9b85d0 0x560d8e9d7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:52.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.983+0000 7f0b8d46f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d8e9b9b40 0x560d8e9ca7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:52.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.983+0000 7f0b8d46f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d8e9cbb80 0x560d8e9e31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:52.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.983+0000 7f0b8d46f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0b841ab920 con 0x560d8e9cbb80 2024-09-11T21:59:52.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.983+0000 7f0b8d46f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0b841ad460 con 0x560d8e9b85d0 2024-09-11T21:59:52.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.983+0000 7f0b8d46f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0b841abff0 con 0x560d8e9b9b40 2024-09-11T21:59:52.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.983+0000 7f0b89633640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d8e9cbb80 0x560d8e9e31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:52.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.983+0000 7f0b89633640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d8e9cbb80 0x560d8e9e31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35370/0 (socket says 172.21.15.1:35370) 2024-09-11T21:59:52.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.983+0000 7f0b89633640 1 -- 172.21.15.1:0/1102827600 learned_addr learned my addr 172.21.15.1:0/1102827600 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:59:52.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.983+0000 7f0b88e32640 1 --2- 172.21.15.1:0/1102827600 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d8e9b85d0 0x560d8e9d7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:52.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.983+0000 7f0b83fff640 1 --2- 172.21.15.1:0/1102827600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d8e9b9b40 0x560d8e9ca7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:52.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.983+0000 7f0b89633640 1 -- 172.21.15.1:0/1102827600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d8e9b9b40 msgr2=0x560d8e9ca7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:59:52.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.983+0000 7f0b89633640 1 --2- 172.21.15.1:0/1102827600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d8e9b9b40 0x560d8e9ca7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:52.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.983+0000 7f0b89633640 1 -- 172.21.15.1:0/1102827600 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d8e9b85d0 msgr2=0x560d8e9d7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:59:52.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.983+0000 7f0b89633640 1 --2- 172.21.15.1:0/1102827600 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d8e9b85d0 0x560d8e9d7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:52.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.983+0000 7f0b89633640 1 -- 172.21.15.1:0/1102827600 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d8e9c9220 con 0x560d8e9cbb80 2024-09-11T21:59:52.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.983+0000 7f0b89633640 1 --2- 172.21.15.1:0/1102827600 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d8e9cbb80 0x560d8e9e31a0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c00f980 tx=0x7f0b7c00fe40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4312efaf942162a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:59:52.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.984+0000 7f0b7a7fc640 1 -- 172.21.15.1:0/1102827600 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c021070 con 0x560d8e9cbb80 2024-09-11T21:59:52.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.984+0000 7f0b7a7fc640 1 -- 172.21.15.1:0/1102827600 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b7c010e50 con 0x560d8e9cbb80 2024-09-11T21:59:52.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.984+0000 7f0b7a7fc640 1 -- 172.21.15.1:0/1102827600 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c004860 con 0x560d8e9cbb80 2024-09-11T21:59:52.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.985+0000 7f0b8d46f740 1 -- 172.21.15.1:0/1102827600 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d8e9cbb80 msgr2=0x560d8e9e31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:59:52.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.985+0000 7f0b8d46f740 1 --2- 172.21.15.1:0/1102827600 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d8e9cbb80 0x560d8e9e31a0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c00f980 tx=0x7f0b7c00fe40 comp rx=0 tx=0).stop 2024-09-11T21:59:52.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.985+0000 7f0b8d46f740 1 -- 172.21.15.1:0/1102827600 shutdown_connections 2024-09-11T21:59:52.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.985+0000 7f0b8d46f740 1 --2- 172.21.15.1:0/1102827600 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d8e9cbb80 0x560d8e9e31a0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:52.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.985+0000 7f0b8d46f740 1 --2- 172.21.15.1:0/1102827600 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d8e9b9b40 0x560d8e9ca7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:52.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.985+0000 7f0b8d46f740 1 --2- 172.21.15.1:0/1102827600 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d8e9b85d0 0x560d8e9d7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:52.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.985+0000 7f0b8d46f740 1 -- 172.21.15.1:0/1102827600 >> 172.21.15.1:0/1102827600 conn(0x560d8e981310 msgr2=0x560d8e9ca0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:59:52.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.985+0000 7f0b8d46f740 1 -- 172.21.15.1:0/1102827600 shutdown_connections 2024-09-11T21:59:52.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.985+0000 7f0b8d46f740 1 -- 172.21.15.1:0/1102827600 wait complete. 2024-09-11T21:59:52.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.985+0000 7f0b8d46f740 1 -- start start 2024-09-11T21:59:52.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.985+0000 7f0b8d46f740 1 Processor -- start 2024-09-11T21:59:52.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.986+0000 7f0b8d46f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d8e9b85d0 0x560d8e9cf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:52.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.986+0000 7f0b8d46f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d8e9b9b40 0x560d8e9cfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:52.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.986+0000 7f0b8d46f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d8e9cbb80 0x560d8e9d02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:52.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.986+0000 7f0b8d46f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0b841acd90 con 0x560d8e9cbb80 2024-09-11T21:59:52.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.986+0000 7f0b8d46f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0b841ac6c0 con 0x560d8e9b9b40 2024-09-11T21:59:52.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.986+0000 7f0b8d46f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0b841a9900 con 0x560d8e9b85d0 2024-09-11T21:59:52.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.986+0000 7f0b8d46f740 10 client.0 _ll_register_callbacks cb 0x7f0b6c005cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:59:52.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.986+0000 7f0b89633640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d8e9cbb80 0x560d8e9d02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:52.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.986+0000 7f0b89633640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d8e9cbb80 0x560d8e9d02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35384/0 (socket says 172.21.15.1:35384) 2024-09-11T21:59:52.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.986+0000 7f0b89633640 1 -- 172.21.15.1:0/1863841230 learned_addr learned my addr 172.21.15.1:0/1863841230 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:59:52.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.986+0000 7f0b88e32640 1 --2- 172.21.15.1:0/1863841230 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d8e9b85d0 0x560d8e9cf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:52.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.986+0000 7f0b83fff640 1 --2- 172.21.15.1:0/1863841230 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d8e9b9b40 0x560d8e9cfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:52.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.987+0000 7f0b89633640 1 -- 172.21.15.1:0/1863841230 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d8e9b85d0 msgr2=0x560d8e9cf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:59:52.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.987+0000 7f0b89633640 1 --2- 172.21.15.1:0/1863841230 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d8e9b85d0 0x560d8e9cf870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:52.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.987+0000 7f0b89633640 1 -- 172.21.15.1:0/1863841230 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d8e9b9b40 msgr2=0x560d8e9cfdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:59:52.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.987+0000 7f0b89633640 1 --2- 172.21.15.1:0/1863841230 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d8e9b9b40 0x560d8e9cfdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:52.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.987+0000 7f0b89633640 1 -- 172.21.15.1:0/1863841230 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d8ea453c0 con 0x560d8e9cbb80 2024-09-11T21:59:52.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.987+0000 7f0b89633640 1 --2- 172.21.15.1:0/1863841230 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d8e9cbb80 0x560d8e9d02f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c01f040 tx=0x7f0b7c004810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dc125a2701768a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:59:52.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.987+0000 7f0b63fff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T21:59:52.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.987+0000 7f0b63fff640 1 -- 172.21.15.1:0/1863841230 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c021070 con 0x560d8e9cbb80 2024-09-11T21:59:52.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.987+0000 7f0b63fff640 1 -- 172.21.15.1:0/1863841230 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f0b7c004a40 con 0x560d8e9cbb80 2024-09-11T21:59:52.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.987+0000 7f0b8d46f740 1 -- 172.21.15.1:0/1863841230 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b7c010e50 con 0x560d8e9cbb80 2024-09-11T21:59:52.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.987+0000 7f0b63fff640 1 -- 172.21.15.1:0/1863841230 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0b7c004c40 con 0x560d8e9cbb80 2024-09-11T21:59:52.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.987+0000 7f0b8d46f740 10 client.15945 fetch_fsmap learned FSMap version 28 2024-09-11T21:59:52.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.987+0000 7f0b8d46f740 1 -- 172.21.15.1:0/1863841230 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560d8ea466a0 con 0x560d8e9cbb80 2024-09-11T21:59:52.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.988+0000 7f0b63fff640 1 -- 172.21.15.1:0/1863841230 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b7c010e50 con 0x560d8e9cbb80 2024-09-11T21:59:52.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.988+0000 7f0b63fff640 1 -- 172.21.15.1:0/1863841230 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f0b7c01b030 con 0x560d8e9cbb80 2024-09-11T21:59:52.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.988+0000 7f0b8d46f740 10 client.15945 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T21:59:52.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.988+0000 7f0b8d46f740 10 client.15945 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-11T21:59:52.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.988+0000 7f0b8d46f740 20 client.15945 populate_metadata read hostname 'smithi001' 2024-09-11T21:59:52.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.988+0000 7f0b8d46f740 1 --2- 172.21.15.1:0/1863841230 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x560d8ea46cb0 0x560d8ea490a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:52.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.988+0000 7f0b8d46f740 4 client.15945 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T21:59:52.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.988+0000 7f0b8d46f740 1 -- 172.21.15.1:0/1863841230 --> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560d8e925d70 con 0x560d8ea46cb0 2024-09-11T21:59:52.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.989+0000 7f0b88e32640 1 --2- 172.21.15.1:0/1863841230 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x560d8ea46cb0 0x560d8ea490a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:52.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.989+0000 7f0b88e32640 1 --2- 172.21.15.1:0/1863841230 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x560d8ea46cb0 0x560d8ea490a0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c2cb4126511d2fd server_cookie=4c3e1ba252197947 in_seq=0 out_seq=0 2024-09-11T21:59:52.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.989+0000 7f0b63fff640 10 client.15945 ms_handle_connect on v2:172.21.15.161:6834/828381592 2024-09-11T21:59:52.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.989+0000 7f0b63fff640 1 -- 172.21.15.1:0/1863841230 <== mds.4 v2:172.21.15.161:6834/828381592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560d8e925d70 con 0x560d8ea46cb0 2024-09-11T21:59:52.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.989+0000 7f0b63fff640 10 client.15945 handle_command_reply: tid=0 2024-09-11T21:59:52.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.989+0000 7f0b8d46f740 1 client.15945 shutdown 2024-09-11T21:59:52.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.989+0000 7f0b8d46f740 1 -- 172.21.15.1:0/1863841230 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d8e9cbb80 msgr2=0x560d8e9d02f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:59:52.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.989+0000 7f0b8d46f740 1 --2- 172.21.15.1:0/1863841230 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d8e9cbb80 0x560d8e9d02f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c01f040 tx=0x7f0b7c004810 comp rx=0 tx=0).stop 2024-09-11T21:59:52.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.989+0000 7fc1a6823740 1 Processor -- start 2024-09-11T21:59:52.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.989+0000 7f0b8d46f740 1 -- 172.21.15.1:0/1863841230 shutdown_connections 2024-09-11T21:59:52.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.989+0000 7fc1a6823740 1 -- start start 2024-09-11T21:59:52.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.989+0000 7f0b8d46f740 1 --2- 172.21.15.1:0/1863841230 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d8e9cbb80 0x560d8e9d02f0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:52.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.989+0000 7f0b8d46f740 1 --2- 172.21.15.1:0/1863841230 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x560d8ea46cb0 0x560d8ea490a0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:52.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.990+0000 7f0b8d46f740 1 --2- 172.21.15.1:0/1863841230 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560d8e9b9b40 0x560d8e9cfdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:52.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.990+0000 7f0b8d46f740 1 --2- 172.21.15.1:0/1863841230 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560d8e9b85d0 0x560d8e9cf870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:52.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.990+0000 7f0b63fff640 0 client.15945 ms_handle_reset on v2:172.21.15.161:6834/828381592 2024-09-11T21:59:53.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.990+0000 7f0b8d46f740 1 -- 172.21.15.1:0/1863841230 >> 172.21.15.1:0/1863841230 conn(0x560d8e981310 msgr2=0x560d8e9b3d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:59:53.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.990+0000 7f0b8d46f740 1 -- 172.21.15.1:0/1863841230 shutdown_connections 2024-09-11T21:59:53.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.990+0000 7f0b8d46f740 1 -- 172.21.15.1:0/1863841230 wait complete. 2024-09-11T21:59:53.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.990+0000 7fc1a6823740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564fd8b8b760 0x564fd8bae4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:53.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.990+0000 7fc1a6823740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564fd8b8c080 0x564fd8baea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:53.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.990+0000 7fc1a6823740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564fd8b96940 0x564fd8b90a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:53.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.990+0000 7fc1a6823740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc19c1afcf0 con 0x564fd8b96940 2024-09-11T21:59:53.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.990+0000 7fc1a6823740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc19c1b1830 con 0x564fd8b8b760 2024-09-11T21:59:53.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.990+0000 7f0b8d46f740 20 client.15945 trim_cache size 0 max 16384 2024-09-11T21:59:53.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.990+0000 7fc1a6823740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc19c1b03c0 con 0x564fd8b8c080 2024-09-11T21:59:53.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.990+0000 7fc1a29e7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564fd8b96940 0x564fd8b90a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:53.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.990+0000 7fc1a29e7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564fd8b96940 0x564fd8b90a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35394/0 (socket says 172.21.15.1:35394) 2024-09-11T21:59:53.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.990+0000 7fc1a29e7640 1 -- 172.21.15.1:0/1909170977 learned_addr learned my addr 172.21.15.1:0/1909170977 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:59:53.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.990+0000 7fc1a19e5640 1 --2- 172.21.15.1:0/1909170977 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564fd8b8c080 0x564fd8baea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:53.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.990+0000 7fc1a21e6640 1 --2- 172.21.15.1:0/1909170977 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564fd8b8b760 0x564fd8bae4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:53.002 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-11T21:59:53.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.990+0000 7fc1a29e7640 1 -- 172.21.15.1:0/1909170977 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564fd8b8c080 msgr2=0x564fd8baea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:59:53.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.990+0000 7fc1a29e7640 1 --2- 172.21.15.1:0/1909170977 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564fd8b8c080 0x564fd8baea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:53.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.990+0000 7fc1a29e7640 1 -- 172.21.15.1:0/1909170977 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564fd8b8b760 msgr2=0x564fd8bae4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:59:53.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.990+0000 7fc1a29e7640 1 --2- 172.21.15.1:0/1909170977 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564fd8b8b760 0x564fd8bae4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:53.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.990+0000 7fc1a29e7640 1 -- 172.21.15.1:0/1909170977 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fd8b87b20 con 0x564fd8b96940 2024-09-11T21:59:53.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.990+0000 7fc1a19e5640 1 --2- 172.21.15.1:0/1909170977 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564fd8b8c080 0x564fd8baea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T21:59:53.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.991+0000 7fc1a29e7640 1 --2- 172.21.15.1:0/1909170977 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564fd8b96940 0x564fd8b90a40 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fc194098410 tx=0x7fc19400efd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa826209a1df47fd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:59:53.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.991+0000 7f0b8b096640 1 -- 172.21.15.1:0/2362790814 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f0b700a2d70 msgr2=0x7f0b700a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:59:53.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.991+0000 7f0b8b096640 1 --2- 172.21.15.1:0/2362790814 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f0b700a2d70 0x7f0b700a5230 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0b84102480 tx=0x7f0b6c00b390 comp rx=0 tx=0).stop 2024-09-11T21:59:53.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.991+0000 7f0b8b096640 1 -- 172.21.15.1:0/2362790814 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b84102d90 msgr2=0x7f0b841009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:59:53.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.991+0000 7f0b8b096640 1 --2- 172.21.15.1:0/2362790814 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b84102d90 0x7f0b841009f0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f0b7402a940 tx=0x7f0b74003b90 comp rx=0 tx=0).stop 2024-09-11T21:59:53.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.991+0000 7fc187fff640 1 -- 172.21.15.1:0/1909170977 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19409c6b0 con 0x564fd8b96940 2024-09-11T21:59:53.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.991+0000 7f0b8b096640 1 -- 172.21.15.1:0/2362790814 shutdown_connections 2024-09-11T21:59:53.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.991+0000 7f0b8b096640 1 --2- 172.21.15.1:0/2362790814 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f0b700a2d70 0x7f0b700a5230 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:53.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.991+0000 7f0b8b096640 1 --2- 172.21.15.1:0/2362790814 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0b84108290 0x7f0b84101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:53.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.991+0000 7f0b8b096640 1 --2- 172.21.15.1:0/2362790814 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b84103740 0x7f0b84100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:53.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.991+0000 7f0b8b096640 1 --2- 172.21.15.1:0/2362790814 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b84102d90 0x7f0b841009f0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:53.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.991+0000 7f0b8b096640 1 -- 172.21.15.1:0/2362790814 >> 172.21.15.1:0/2362790814 conn(0x7f0b840fc7d0 msgr2=0x7f0b84106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:59:53.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.991+0000 7fc187fff640 1 -- 172.21.15.1:0/1909170977 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc19409bfb0 con 0x564fd8b96940 2024-09-11T21:59:53.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.991+0000 7fc187fff640 1 -- 172.21.15.1:0/1909170977 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc194005570 con 0x564fd8b96940 2024-09-11T21:59:53.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.991+0000 7f0b8b096640 1 -- 172.21.15.1:0/2362790814 shutdown_connections 2024-09-11T21:59:53.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.991+0000 7f0b8b096640 1 -- 172.21.15.1:0/2362790814 wait complete. 2024-09-11T21:59:53.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.992+0000 7fc1a6823740 1 -- 172.21.15.1:0/1909170977 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564fd8b96940 msgr2=0x564fd8b90a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:59:53.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.992+0000 7fc1a6823740 1 --2- 172.21.15.1:0/1909170977 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564fd8b96940 0x564fd8b90a40 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fc194098410 tx=0x7fc19400efd0 comp rx=0 tx=0).stop 2024-09-11T21:59:53.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.992+0000 7fc1a6823740 1 -- 172.21.15.1:0/1909170977 shutdown_connections 2024-09-11T21:59:53.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.992+0000 7fc1a6823740 1 --2- 172.21.15.1:0/1909170977 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564fd8b96940 0x564fd8b90a40 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:53.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.992+0000 7fc1a6823740 1 --2- 172.21.15.1:0/1909170977 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564fd8b8c080 0x564fd8baea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:53.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.992+0000 7fc1a6823740 1 --2- 172.21.15.1:0/1909170977 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564fd8b8b760 0x564fd8bae4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:53.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.992+0000 7fc1a6823740 1 -- 172.21.15.1:0/1909170977 >> 172.21.15.1:0/1909170977 conn(0x564fd8b87cf0 msgr2=0x564fd8b95840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:59:53.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.992+0000 7fc1a6823740 1 -- 172.21.15.1:0/1909170977 shutdown_connections 2024-09-11T21:59:53.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.992+0000 7fc1a6823740 1 -- 172.21.15.1:0/1909170977 wait complete. 2024-09-11T21:59:53.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.993+0000 7fc1a6823740 1 -- start start 2024-09-11T21:59:53.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.993+0000 7fc1a6823740 1 Processor -- start 2024-09-11T21:59:53.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.993+0000 7fc1a6823740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564fd8b8b760 0x564fd8b9f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:53.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.993+0000 7fc1a6823740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564fd8b8c080 0x564fd8b906d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:53.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.993+0000 7fc1a6823740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564fd8b96940 0x564fd8b90e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:53.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.993+0000 7fc1a6823740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc19c1b1160 con 0x564fd8b8b760 2024-09-11T21:59:53.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.993+0000 7fc1a6823740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc19c1b0a90 con 0x564fd8b8c080 2024-09-11T21:59:53.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.993+0000 7fc1a6823740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc19c1adcd0 con 0x564fd8b96940 2024-09-11T21:59:53.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.993+0000 7fc1a6823740 10 client.0 _ll_register_callbacks cb 0x7fc18c005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T21:59:53.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.993+0000 7fc1a21e6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564fd8b8b760 0x564fd8b9f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:53.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.993+0000 7fc1a21e6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564fd8b8b760 0x564fd8b9f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35404/0 (socket says 172.21.15.1:35404) 2024-09-11T21:59:53.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.993+0000 7fc1a21e6640 1 -- 172.21.15.1:0/4224483968 learned_addr learned my addr 172.21.15.1:0/4224483968 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T21:59:53.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.993+0000 7fc1a29e7640 1 --2- 172.21.15.1:0/4224483968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564fd8b96940 0x564fd8b90e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:53.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.993+0000 7fc1a19e5640 1 --2- 172.21.15.1:0/4224483968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564fd8b8c080 0x564fd8b906d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:53.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.994+0000 7fc1a29e7640 1 -- 172.21.15.1:0/4224483968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564fd8b8c080 msgr2=0x564fd8b906d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:59:53.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.994+0000 7fc1a29e7640 1 --2- 172.21.15.1:0/4224483968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564fd8b8c080 0x564fd8b906d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:53.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.994+0000 7fc1a29e7640 1 -- 172.21.15.1:0/4224483968 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564fd8b8b760 msgr2=0x564fd8b9f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T21:59:53.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.994+0000 7fc1a29e7640 1 --2- 172.21.15.1:0/4224483968 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564fd8b8b760 0x564fd8b9f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:53.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.994+0000 7fc1a29e7640 1 -- 172.21.15.1:0/4224483968 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fd8c104b0 con 0x564fd8b96940 2024-09-11T21:59:53.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.994+0000 7fc1a21e6640 1 --2- 172.21.15.1:0/4224483968 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564fd8b8b760 0x564fd8b9f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T21:59:53.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.994+0000 7fc1a19e5640 1 --2- 172.21.15.1:0/4224483968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564fd8b8c080 0x564fd8b906d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T21:59:53.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.994+0000 7fc1a29e7640 1 --2- 172.21.15.1:0/4224483968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564fd8b96940 0x564fd8b90e60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc19409a260 tx=0x7fc194005ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7294bed63ca8ab48 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T21:59:53.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.994+0000 7fc185ffb640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-11T21:59:53.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.994+0000 7fc185ffb640 1 -- 172.21.15.1:0/4224483968 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19409c6b0 con 0x564fd8b96940 2024-09-11T21:59:53.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.994+0000 7fc185ffb640 1 -- 172.21.15.1:0/4224483968 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7fc19409a2b0 con 0x564fd8b96940 2024-09-11T21:59:53.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.994+0000 7fc1a6823740 1 -- 172.21.15.1:0/4224483968 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc19409bfb0 con 0x564fd8b96940 2024-09-11T21:59:53.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.995+0000 7fc185ffb640 1 -- 172.21.15.1:0/4224483968 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc1940cd460 con 0x564fd8b96940 2024-09-11T21:59:53.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.995+0000 7fc1a6823740 10 client.25507 fetch_fsmap learned FSMap version 28 2024-09-11T21:59:53.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.995+0000 7fc1a6823740 1 -- 172.21.15.1:0/4224483968 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564fd8c115e0 con 0x564fd8b96940 2024-09-11T21:59:53.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.995+0000 7fc185ffb640 1 -- 172.21.15.1:0/4224483968 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc19409bfb0 con 0x564fd8b96940 2024-09-11T21:59:53.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.995+0000 7fc185ffb640 1 -- 172.21.15.1:0/4224483968 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc1940aa070 con 0x564fd8b96940 2024-09-11T21:59:53.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.995+0000 7fc1a6823740 10 client.25507 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T21:59:53.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.995+0000 7fc1a6823740 10 client.25507 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-11T21:59:53.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.995+0000 7fc1a6823740 20 client.25507 populate_metadata read hostname 'smithi001' 2024-09-11T21:59:53.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.995+0000 7fc1a6823740 1 --2- 172.21.15.1:0/4224483968 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x564fd8c11860 0x564fd8c13c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T21:59:53.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.995+0000 7fc1a6823740 4 client.25507 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 133.4, "await_for": 45.0}] 2024-09-11T21:59:53.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.995+0000 7fc1a6823740 1 -- 172.21.15.1:0/4224483968 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 133.4, "await_for": 45.0}) -- 0x564fd8af8d20 con 0x564fd8c11860 2024-09-11T21:59:53.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.995+0000 7fc1a21e6640 1 --2- 172.21.15.1:0/4224483968 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x564fd8c11860 0x564fd8c13c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T21:59:53.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.996+0000 7fc1a21e6640 1 --2- 172.21.15.1:0/4224483968 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x564fd8c11860 0x564fd8c13c50 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab4be16bd84c2a1e server_cookie=cdf047f57c4da36 in_seq=0 out_seq=0 2024-09-11T21:59:53.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:52.996+0000 7fc185ffb640 10 client.25507 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T21:59:53.028 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:59:52 smithi001 ceph-mon[30935]: pgmap v84: 97 pgs: 97 active+clean; 2.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-09-11T21:59:53.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:59:52 smithi053 ceph-mon[34908]: pgmap v84: 97 pgs: 97 active+clean; 2.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-09-11T21:59:53.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:59:52 smithi161 ceph-mon[35615]: pgmap v84: 97 pgs: 97 active+clean; 2.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-09-11T21:59:53.525 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T21:59:53.525 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T21:59:53.893 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T21:59:53.893 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T21:59:55.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:59:54 smithi053 ceph-mon[34908]: pgmap v85: 97 pgs: 97 active+clean; 2.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-09-11T21:59:55.091 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:59:54 smithi161 ceph-mon[35615]: pgmap v85: 97 pgs: 97 active+clean; 2.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-09-11T21:59:55.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:59:54 smithi001 ceph-mon[30935]: pgmap v85: 97 pgs: 97 active+clean; 2.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-09-11T21:59:55.403 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T21:59:55.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:55.442+0000 7fc185ffb640 1 -- 172.21.15.1:0/4224483968 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x564fd8af8d20 con 0x564fd8c11860 2024-09-11T21:59:55.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:55.442+0000 7fc185ffb640 10 client.25507 handle_command_reply: tid=0 2024-09-11T21:59:55.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:55.442+0000 7fc1a6823740 1 client.25507 shutdown 2024-09-11T21:59:55.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:55.442+0000 7fc1a6823740 1 -- 172.21.15.1:0/4224483968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564fd8b96940 msgr2=0x564fd8b90e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:59:55.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:55.442+0000 7fc1a6823740 1 --2- 172.21.15.1:0/4224483968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564fd8b96940 0x564fd8b90e60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc19409a260 tx=0x7fc194005ea0 comp rx=0 tx=0).stop 2024-09-11T21:59:55.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:55.442+0000 7fc1a6823740 1 -- 172.21.15.1:0/4224483968 shutdown_connections 2024-09-11T21:59:55.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:55.442+0000 7fc1a6823740 1 --2- 172.21.15.1:0/4224483968 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x564fd8c11860 0x564fd8c13c50 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:55.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:55.442+0000 7fc1a6823740 1 --2- 172.21.15.1:0/4224483968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564fd8b96940 0x564fd8b90e60 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:55.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:55.442+0000 7fc1a6823740 1 --2- 172.21.15.1:0/4224483968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564fd8b8c080 0x564fd8b906d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:55.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:55.442+0000 7fc185ffb640 0 client.25507 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T21:59:55.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:55.442+0000 7fc1a6823740 1 --2- 172.21.15.1:0/4224483968 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564fd8b8b760 0x564fd8b9f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:55.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:55.442+0000 7fc1a6823740 1 -- 172.21.15.1:0/4224483968 >> 172.21.15.1:0/4224483968 conn(0x564fd8b87cf0 msgr2=0x564fd8b95020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:59:55.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:55.442+0000 7fc1a6823740 1 -- 172.21.15.1:0/4224483968 shutdown_connections 2024-09-11T21:59:55.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:55.442+0000 7fc1a6823740 1 -- 172.21.15.1:0/4224483968 wait complete. 2024-09-11T21:59:55.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:55.442+0000 7fc1a6823740 20 client.25507 trim_cache size 0 max 16384 2024-09-11T21:59:55.445 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-11T21:59:55.445 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 28, 2024-09-11T21:59:55.445 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24494, 2024-09-11T21:59:55.445 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 6, 2024-09-11T21:59:55.445 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-09-11T21:59:55.445 INFO:teuthology.orchestra.run.smithi001.stdout: "ff1ce167": { 2024-09-11T21:59:55.445 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 6, 2024-09-11T21:59:55.445 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-09-11T21:59:55.445 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-11T21:59:55.445 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-09-11T21:59:55.446 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-11T21:59:55.446 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T21:59:55.446 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 90.0, 2024-09-11T21:59:55.446 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 133.4, 2024-09-11T21:59:55.446 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-09-11T21:59:55.446 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-09-11T21:59:55.447 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-09-11T21:59:55.447 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-11T21:59:55.447 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-09-11T21:59:55.447 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-11T21:59:55.447 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T21:59:55.447 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T21:59:55.447 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T21:59:55.448 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T21:59:55.448 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T21:59:55.448 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-11T21:59:55.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:55.443+0000 7fc1a444a640 1 -- 172.21.15.1:0/2585013667 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fc1880a2cf0 msgr2=0x7fc1880a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:59:55.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:55.443+0000 7fc1a444a640 1 --2- 172.21.15.1:0/2585013667 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fc1880a2cf0 0x7fc1880a51b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc198005d00 tx=0x7fc19800b410 comp rx=0 tx=0).stop 2024-09-11T21:59:55.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:55.443+0000 7fc1a444a640 1 -- 172.21.15.1:0/2585013667 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc19c106f10 msgr2=0x7fc19c19ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T21:59:55.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:55.443+0000 7fc1a444a640 1 --2- 172.21.15.1:0/2585013667 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc19c106f10 0x7fc19c19ef00 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fc19400eb10 tx=0x7fc19400c2d0 comp rx=0 tx=0).stop 2024-09-11T21:59:55.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:55.443+0000 7fc1a444a640 1 -- 172.21.15.1:0/2585013667 shutdown_connections 2024-09-11T21:59:55.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:55.443+0000 7fc1a444a640 1 --2- 172.21.15.1:0/2585013667 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fc1880a2cf0 0x7fc1880a51b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:55.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:55.443+0000 7fc1a444a640 1 --2- 172.21.15.1:0/2585013667 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc19c106f10 0x7fc19c19ef00 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:55.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:55.443+0000 7fc1a444a640 1 --2- 172.21.15.1:0/2585013667 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc19c102790 0x7fc19c19ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:55.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:55.443+0000 7fc1a444a640 1 --2- 172.21.15.1:0/2585013667 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc19c101de0 0x7fc19c19a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T21:59:55.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:55.443+0000 7fc1a444a640 1 -- 172.21.15.1:0/2585013667 >> 172.21.15.1:0/2585013667 conn(0x7fc19c0fd8e0 msgr2=0x7fc19c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T21:59:55.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:55.443+0000 7fc1a444a640 1 -- 172.21.15.1:0/2585013667 shutdown_connections 2024-09-11T21:59:55.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T21:59:55.443+0000 7fc1a444a640 1 -- 172.21.15.1:0/2585013667 wait complete. 2024-09-11T21:59:56.926 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'ff1ce167', quiesce took 44.4 seconds. Will release after: -31.0 2024-09-11T21:59:56.927 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T21:59:57.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:59:56 smithi053 ceph-mon[34908]: pgmap v86: 97 pgs: 97 active+clean; 2.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-09-11T21:59:57.091 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:59:56 smithi161 ceph-mon[35615]: pgmap v86: 97 pgs: 97 active+clean; 2.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-09-11T21:59:57.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:59:56 smithi001 ceph-mon[30935]: pgmap v86: 97 pgs: 97 active+clean; 2.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-09-11T21:59:57.468 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:59:57.515 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T21:59:58.900 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 21:59:58 smithi001 ceph-mon[30935]: pgmap v87: 97 pgs: 97 active+clean; 2.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-09-11T21:59:59.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 21:59:58 smithi053 ceph-mon[34908]: pgmap v87: 97 pgs: 97 active+clean; 2.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-09-11T21:59:59.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 21:59:58 smithi161 ceph-mon[35615]: pgmap v87: 97 pgs: 97 active+clean; 2.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-09-11T22:00:00.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.514+0000 7f4ec80e2640 1 -- 172.21.15.1:0/4204294983 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ec0102570 msgr2=0x7f4ec01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:00.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.514+0000 7f4ec80e2640 1 --2- 172.21.15.1:0/4204294983 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ec0102570 0x7f4ec01029b0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f4eb4009900 tx=0x7f4eb402a480 comp rx=0 tx=0).stop 2024-09-11T22:00:00.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.514+0000 7f4ec80e2640 1 -- 172.21.15.1:0/4204294983 shutdown_connections 2024-09-11T22:00:00.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.514+0000 7f4ec80e2640 1 --2- 172.21.15.1:0/4204294983 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ec0102ef0 0x7f4ec010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:00.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.514+0000 7f4ec80e2640 1 --2- 172.21.15.1:0/4204294983 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ec0102570 0x7f4ec01029b0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:00.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.514+0000 7f4ec80e2640 1 --2- 172.21.15.1:0/4204294983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ec0100e20 0x7f4ec0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:00.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.514+0000 7f4ec80e2640 1 -- 172.21.15.1:0/4204294983 >> 172.21.15.1:0/4204294983 conn(0x7f4ec00fc7d0 msgr2=0x7f4ec00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:00:00.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.514+0000 7f4ec80e2640 1 -- 172.21.15.1:0/4204294983 shutdown_connections 2024-09-11T22:00:00.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.514+0000 7f4ec80e2640 1 -- 172.21.15.1:0/4204294983 wait complete. 2024-09-11T22:00:00.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.514+0000 7f4ec80e2640 1 Processor -- start 2024-09-11T22:00:00.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.515+0000 7f4ec80e2640 1 -- start start 2024-09-11T22:00:00.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.515+0000 7f4ec80e2640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ec0100e20 0x7f4ec0199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:00.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.515+0000 7f4ec80e2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ec0102570 0x7f4ec01998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:00.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.515+0000 7f4ec80e2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ec0102ef0 0x7f4ec01a43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:00.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.515+0000 7f4ec80e2640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec0110fa0 con 0x7f4ec0102ef0 2024-09-11T22:00:00.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.515+0000 7f4ec80e2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec0110e20 con 0x7f4ec0102570 2024-09-11T22:00:00.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.515+0000 7f4ec80e2640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec0111120 con 0x7f4ec0100e20 2024-09-11T22:00:00.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.515+0000 7f4ec667f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ec0102ef0 0x7f4ec01a43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:00.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.515+0000 7f4ec667f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ec0102ef0 0x7f4ec01a43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45598/0 (socket says 172.21.15.1:45598) 2024-09-11T22:00:00.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.515+0000 7f4ec667f640 1 -- 172.21.15.1:0/307072113 learned_addr learned my addr 172.21.15.1:0/307072113 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:00:00.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.515+0000 7f4ec5e7e640 1 --2- 172.21.15.1:0/307072113 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ec0100e20 0x7f4ec0199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:00.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.515+0000 7f4ec567d640 1 --2- 172.21.15.1:0/307072113 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ec0102570 0x7f4ec01998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:00.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.515+0000 7f4ec667f640 1 -- 172.21.15.1:0/307072113 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ec0100e20 msgr2=0x7f4ec0199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:00.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.515+0000 7f8666825640 1 -- 172.21.15.1:0/1136341804 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8660102570 msgr2=0x7f86601029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:00.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.516+0000 7f8666825640 1 --2- 172.21.15.1:0/1136341804 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8660102570 0x7f86601029b0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f8654009900 tx=0x7f865402a480 comp rx=0 tx=0).stop 2024-09-11T22:00:00.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.516+0000 7f4ec667f640 1 --2- 172.21.15.1:0/307072113 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ec0100e20 0x7f4ec0199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:00.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.516+0000 7f4ec667f640 1 -- 172.21.15.1:0/307072113 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ec0102570 msgr2=0x7f4ec01998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:00.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.516+0000 7f4ec667f640 1 --2- 172.21.15.1:0/307072113 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ec0102570 0x7f4ec01998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:00.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.516+0000 7f4ec667f640 1 -- 172.21.15.1:0/307072113 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ec01a4930 con 0x7f4ec0102ef0 2024-09-11T22:00:00.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.516+0000 7f8666825640 1 -- 172.21.15.1:0/1136341804 shutdown_connections 2024-09-11T22:00:00.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.516+0000 7f8666825640 1 --2- 172.21.15.1:0/1136341804 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8660102ef0 0x7f866010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:00.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.516+0000 7f4ec667f640 1 --2- 172.21.15.1:0/307072113 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ec0102ef0 0x7f4ec01a43f0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f4eb800d950 tx=0x7f4eb800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:00:00.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.516+0000 7f8666825640 1 --2- 172.21.15.1:0/1136341804 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8660102570 0x7f86601029b0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:00.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.516+0000 7f8666825640 1 --2- 172.21.15.1:0/1136341804 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8660100e20 0x7f8660101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:00.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.516+0000 7f8666825640 1 -- 172.21.15.1:0/1136341804 >> 172.21.15.1:0/1136341804 conn(0x7f86600fc7d0 msgr2=0x7f86600febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:00:00.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.516+0000 7f8666825640 1 -- 172.21.15.1:0/1136341804 shutdown_connections 2024-09-11T22:00:00.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.516+0000 7f8666825640 1 -- 172.21.15.1:0/1136341804 wait complete. 2024-09-11T22:00:00.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.516+0000 7f4ebeffd640 1 -- 172.21.15.1:0/307072113 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb8017030 con 0x7f4ec0102ef0 2024-09-11T22:00:00.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.516+0000 7f4ebeffd640 1 -- 172.21.15.1:0/307072113 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4eb8004020 con 0x7f4ec0102ef0 2024-09-11T22:00:00.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.516+0000 7f4ec80e2640 1 -- 172.21.15.1:0/307072113 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ec0197160 con 0x7f4ec0102ef0 2024-09-11T22:00:00.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.516+0000 7f4ebeffd640 1 -- 172.21.15.1:0/307072113 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb8003c00 con 0x7f4ec0102ef0 2024-09-11T22:00:00.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.516+0000 7f4ec80e2640 1 -- 172.21.15.1:0/307072113 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ec0197650 con 0x7f4ec0102ef0 2024-09-11T22:00:00.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.517+0000 7f8666825640 1 Processor -- start 2024-09-11T22:00:00.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.517+0000 7f8666825640 1 -- start start 2024-09-11T22:00:00.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.517+0000 7f8666825640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8660100e20 0x7f8660197460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:00.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.517+0000 7f8666825640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8660102570 0x7f86601979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:00.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.517+0000 7f8666825640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8660102ef0 0x7f866019bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:00.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.518+0000 7f8666825640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8660110fa0 con 0x7f8660100e20 2024-09-11T22:00:00.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.518+0000 7f8666825640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8660110e20 con 0x7f8660102ef0 2024-09-11T22:00:00.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.518+0000 7f8666825640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8660111120 con 0x7f8660102570 2024-09-11T22:00:00.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.518+0000 7f4ec80e2640 1 -- 172.21.15.1:0/307072113 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ec0101200 con 0x7f4ec0102ef0 2024-09-11T22:00:00.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.518+0000 7f8664dc2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8660102ef0 0x7f866019bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:00.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.518+0000 7f8664dc2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8660102ef0 0x7f866019bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:59824/0 (socket says 172.21.15.1:59824) 2024-09-11T22:00:00.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.518+0000 7f8664dc2640 1 -- 172.21.15.1:0/1604313260 learned_addr learned my addr 172.21.15.1:0/1604313260 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:00:00.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.518+0000 7f865ffff640 1 --2- 172.21.15.1:0/1604313260 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8660100e20 0x7f8660197460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:00.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.518+0000 7f865f7fe640 1 --2- 172.21.15.1:0/1604313260 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8660102570 0x7f86601979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:00.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.518+0000 7f8664dc2640 1 -- 172.21.15.1:0/1604313260 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8660102570 msgr2=0x7f86601979a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:00.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.518+0000 7f8664dc2640 1 --2- 172.21.15.1:0/1604313260 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8660102570 0x7f86601979a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:00.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.518+0000 7f8664dc2640 1 -- 172.21.15.1:0/1604313260 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8660100e20 msgr2=0x7f8660197460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:00.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.518+0000 7f8664dc2640 1 --2- 172.21.15.1:0/1604313260 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8660100e20 0x7f8660197460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:00.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.518+0000 7f8664dc2640 1 -- 172.21.15.1:0/1604313260 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f866019c410 con 0x7f8660102ef0 2024-09-11T22:00:00.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.518+0000 7f865f7fe640 1 --2- 172.21.15.1:0/1604313260 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8660102570 0x7f86601979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:00:00.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.519+0000 7f8664dc2640 1 --2- 172.21.15.1:0/1604313260 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8660102ef0 0x7f866019bd30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f865800cad0 tx=0x7f865800ced0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:00:00.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.519+0000 7f865d7fa640 1 -- 172.21.15.1:0/1604313260 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865800ad60 con 0x7f8660102ef0 2024-09-11T22:00:00.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.519+0000 7f8666825640 1 -- 172.21.15.1:0/1604313260 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f866019c700 con 0x7f8660102ef0 2024-09-11T22:00:00.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.519+0000 7f8666825640 1 -- 172.21.15.1:0/1604313260 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8660069f70 con 0x7f8660102ef0 2024-09-11T22:00:00.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.519+0000 7f4ebeffd640 1 -- 172.21.15.1:0/307072113 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f4eb801a070 con 0x7f4ec0102ef0 2024-09-11T22:00:00.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.519+0000 7f865d7fa640 1 -- 172.21.15.1:0/1604313260 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8658005020 con 0x7f8660102ef0 2024-09-11T22:00:00.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.519+0000 7f865d7fa640 1 -- 172.21.15.1:0/1604313260 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865800f6b0 con 0x7f8660102ef0 2024-09-11T22:00:00.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.520+0000 7f4ebeffd640 1 --2- 172.21.15.1:0/307072113 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f4ea80ab580 0x7f4ea80ada40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:00.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.520+0000 7f4ec5e7e640 1 --2- 172.21.15.1:0/307072113 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f4ea80ab580 0x7f4ea80ada40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:00.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.521+0000 7f4ebeffd640 1 -- 172.21.15.1:0/307072113 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f4eb80cec00 con 0x7f4ec0102ef0 2024-09-11T22:00:00.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.521+0000 7f4ec5e7e640 1 --2- 172.21.15.1:0/307072113 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f4ea80ab580 0x7f4ea80ada40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0005fd0 tx=0x7f4eb0009680 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:00:00.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.522+0000 7f865d7fa640 1 -- 172.21.15.1:0/1604313260 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f8658004a40 con 0x7f8660102ef0 2024-09-11T22:00:00.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.522+0000 7f865d7fa640 1 --2- 172.21.15.1:0/1604313260 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f86480a2d70 0x7f86480a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:00.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.523+0000 7f865ffff640 1 --2- 172.21.15.1:0/1604313260 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f86480a2d70 0x7f86480a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:00.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.523+0000 7f865d7fa640 1 -- 172.21.15.1:0/1604313260 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f86580cf390 con 0x7f8660102ef0 2024-09-11T22:00:00.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.523+0000 7f865ffff640 1 --2- 172.21.15.1:0/1604313260 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f86480a2d70 0x7f86480a5230 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8650009990 tx=0x7f8650009110 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:00:00.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.525+0000 7f4ebeffd640 1 -- 172.21.15.1:0/307072113 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4eb80d5080 con 0x7f4ec0102ef0 2024-09-11T22:00:00.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.549+0000 7f8668bfe740 1 Processor -- start 2024-09-11T22:00:00.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.549+0000 7f8668bfe740 1 -- start start 2024-09-11T22:00:00.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.549+0000 7f8668bfe740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eab8cc5b40 0x55eab8cc4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:00.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.549+0000 7f8668bfe740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eab8cc45d0 0x55eab8cc4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:00.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.549+0000 7f8668bfe740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eab8cd3780 0x55eab8cd5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:00.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.549+0000 7f8668bfe740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55eab8c2a2f0 con 0x55eab8cc5b40 2024-09-11T22:00:00.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.549+0000 7f8668bfe740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55eab8bd1cf0 con 0x55eab8cc45d0 2024-09-11T22:00:00.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.549+0000 7f8668bfe740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55eab8c262b0 con 0x55eab8cd3780 2024-09-11T22:00:00.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.550+0000 7f865ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eab8cc5b40 0x55eab8cc4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:00.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.550+0000 7f865ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eab8cc5b40 0x55eab8cc4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45624/0 (socket says 172.21.15.1:45624) 2024-09-11T22:00:00.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.550+0000 7f865ffff640 1 -- 172.21.15.1:0/739691682 learned_addr learned my addr 172.21.15.1:0/739691682 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:00:00.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.550+0000 7f865f7fe640 1 --2- 172.21.15.1:0/739691682 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eab8cc45d0 0x55eab8cc4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:00.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.550+0000 7f8664dc2640 1 --2- 172.21.15.1:0/739691682 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eab8cd3780 0x55eab8cd5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:00.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.550+0000 7f865ffff640 1 -- 172.21.15.1:0/739691682 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eab8cd3780 msgr2=0x55eab8cd5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:00:00.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.550+0000 7f865ffff640 1 --2- 172.21.15.1:0/739691682 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eab8cd3780 0x55eab8cd5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:00.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.550+0000 7f865ffff640 1 -- 172.21.15.1:0/739691682 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eab8cc45d0 msgr2=0x55eab8cc4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:00:00.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.550+0000 7f865ffff640 1 --2- 172.21.15.1:0/739691682 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eab8cc45d0 0x55eab8cc4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:00.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.550+0000 7f865ffff640 1 -- 172.21.15.1:0/739691682 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eab8bb63a0 con 0x55eab8cc5b40 2024-09-11T22:00:00.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.550+0000 7f865ffff640 1 --2- 172.21.15.1:0/739691682 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eab8cc5b40 0x55eab8cc4090 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f8650009fd0 tx=0x7f8650011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e91c5ffbe7557418 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:00:00.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.550+0000 7f86467fc640 1 -- 172.21.15.1:0/739691682 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865001b040 con 0x55eab8cc5b40 2024-09-11T22:00:00.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.551+0000 7f86467fc640 1 -- 172.21.15.1:0/739691682 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8650024070 con 0x55eab8cc5b40 2024-09-11T22:00:00.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.551+0000 7f86467fc640 1 -- 172.21.15.1:0/739691682 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865001f650 con 0x55eab8cc5b40 2024-09-11T22:00:00.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.551+0000 7f8668bfe740 1 -- 172.21.15.1:0/739691682 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eab8cc5b40 msgr2=0x55eab8cc4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:00.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.551+0000 7f8668bfe740 1 --2- 172.21.15.1:0/739691682 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eab8cc5b40 0x55eab8cc4090 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f8650009fd0 tx=0x7f8650011000 comp rx=0 tx=0).stop 2024-09-11T22:00:00.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.552+0000 7f8668bfe740 1 -- 172.21.15.1:0/739691682 shutdown_connections 2024-09-11T22:00:00.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.552+0000 7f8668bfe740 1 --2- 172.21.15.1:0/739691682 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eab8cd3780 0x55eab8cd5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:00.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.552+0000 7f8668bfe740 1 --2- 172.21.15.1:0/739691682 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eab8cc45d0 0x55eab8cc4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:00.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.552+0000 7f8668bfe740 1 --2- 172.21.15.1:0/739691682 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eab8cc5b40 0x55eab8cc4090 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:00.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.552+0000 7f8668bfe740 1 -- 172.21.15.1:0/739691682 >> 172.21.15.1:0/739691682 conn(0x55eab8c8d310 msgr2=0x55eab8cc0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:00:00.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.552+0000 7f8668bfe740 1 -- 172.21.15.1:0/739691682 shutdown_connections 2024-09-11T22:00:00.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.552+0000 7f8668bfe740 1 -- 172.21.15.1:0/739691682 wait complete. 2024-09-11T22:00:00.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.552+0000 7f8668bfe740 1 -- start start 2024-09-11T22:00:00.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.552+0000 7f8668bfe740 1 Processor -- start 2024-09-11T22:00:00.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.552+0000 7f8668bfe740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eab8cc45d0 0x55eab8cd30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:00.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.552+0000 7f8668bfe740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eab8cc5b40 0x55eab8cd0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:00.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.553+0000 7f8668bfe740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eab8cd3780 0x55eab8cd08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:00.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.553+0000 7f8668bfe740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55eab8cd6480 con 0x55eab8cd3780 2024-09-11T22:00:00.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.553+0000 7f8668bfe740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55eab8be5060 con 0x55eab8cc45d0 2024-09-11T22:00:00.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.553+0000 7f8668bfe740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55eab8cd6600 con 0x55eab8cc5b40 2024-09-11T22:00:00.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.553+0000 7f8668bfe740 10 client.0 _ll_register_callbacks cb 0x55eab89388c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:00:00.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.553+0000 7f8664dc2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eab8cd3780 0x55eab8cd08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:00.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.553+0000 7f865ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eab8cc45d0 0x55eab8cd30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:00.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.553+0000 7f865f7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eab8cc5b40 0x55eab8cd0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:00.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.553+0000 7f8664dc2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eab8cd3780 0x55eab8cd08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45632/0 (socket says 172.21.15.1:45632) 2024-09-11T22:00:00.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.553+0000 7f8664dc2640 1 -- 172.21.15.1:0/3056156016 learned_addr learned my addr 172.21.15.1:0/3056156016 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:00:00.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.553+0000 7f865ffff640 1 -- 172.21.15.1:0/3056156016 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eab8cc5b40 msgr2=0x55eab8cd0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:00:00.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.553+0000 7f865ffff640 1 --2- 172.21.15.1:0/3056156016 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eab8cc5b40 0x55eab8cd0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:00.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.553+0000 7f865ffff640 1 -- 172.21.15.1:0/3056156016 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eab8cd3780 msgr2=0x55eab8cd08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:00:00.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.553+0000 7f865ffff640 1 --2- 172.21.15.1:0/3056156016 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eab8cd3780 0x55eab8cd08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:00.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.553+0000 7f865ffff640 1 -- 172.21.15.1:0/3056156016 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eab899b3c0 con 0x55eab8cc45d0 2024-09-11T22:00:00.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.553+0000 7f8664dc2640 1 --2- 172.21.15.1:0/3056156016 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eab8cd3780 0x55eab8cd08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:00:00.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.553+0000 7f865ffff640 1 --2- 172.21.15.1:0/3056156016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eab8cc45d0 0x55eab8cd30d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8650015850 tx=0x7f86500047d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4fff8eaae61b051 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:00:00.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.553+0000 7f863f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-11T22:00:00.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.554+0000 7f863f7fe640 1 -- 172.21.15.1:0/3056156016 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865002d070 con 0x55eab8cc45d0 2024-09-11T22:00:00.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.554+0000 7f863f7fe640 1 -- 172.21.15.1:0/3056156016 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f865000e770 con 0x55eab8cc45d0 2024-09-11T22:00:00.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.554+0000 7f8668bfe740 1 -- 172.21.15.1:0/3056156016 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8650024070 con 0x55eab8cc45d0 2024-09-11T22:00:00.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.554+0000 7f863f7fe640 1 -- 172.21.15.1:0/3056156016 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8650015cc0 con 0x55eab8cc45d0 2024-09-11T22:00:00.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.554+0000 7f8668bfe740 10 client.25547 fetch_fsmap learned FSMap version 28 2024-09-11T22:00:00.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.554+0000 7f8668bfe740 1 -- 172.21.15.1:0/3056156016 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55eab8ce1ec0 con 0x55eab8cc45d0 2024-09-11T22:00:00.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.554+0000 7f863f7fe640 1 -- 172.21.15.1:0/3056156016 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8650024070 con 0x55eab8cc45d0 2024-09-11T22:00:00.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.555+0000 7f863f7fe640 1 -- 172.21.15.1:0/3056156016 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f865001e030 con 0x55eab8cc45d0 2024-09-11T22:00:00.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.555+0000 7f8668bfe740 10 client.25547 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:00:00.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.555+0000 7f8668bfe740 10 client.25547 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-11T22:00:00.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.555+0000 7f8668bfe740 20 client.25547 populate_metadata read hostname 'smithi001' 2024-09-11T22:00:00.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.555+0000 7f8668bfe740 1 --2- 172.21.15.1:0/3056156016 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x55eab8ce3270 0x55eab8ce5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:00.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.555+0000 7f8668bfe740 4 client.25547 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:00:00.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.555+0000 7f8668bfe740 1 -- 172.21.15.1:0/3056156016 --> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eab8c37860 con 0x55eab8ce3270 2024-09-11T22:00:00.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.555+0000 7f865f7fe640 1 --2- 172.21.15.1:0/3056156016 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x55eab8ce3270 0x55eab8ce5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:00.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.555+0000 7f863f7fe640 10 client.25547 ms_handle_connect on v2:172.21.15.161:6836/1799318656 2024-09-11T22:00:00.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.555+0000 7f865f7fe640 1 --2- 172.21.15.1:0/3056156016 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x55eab8ce3270 0x55eab8ce5660 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b52be359d4098d0b server_cookie=82cfd9302c3aa999 in_seq=0 out_seq=0 2024-09-11T22:00:00.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.556+0000 7f863f7fe640 1 -- 172.21.15.1:0/3056156016 <== mds.3 v2:172.21.15.161:6836/1799318656 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eab8c37860 con 0x55eab8ce3270 2024-09-11T22:00:00.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.556+0000 7f863f7fe640 10 client.25547 handle_command_reply: tid=0 2024-09-11T22:00:00.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.556+0000 7f8668bfe740 1 client.25547 shutdown 2024-09-11T22:00:00.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.556+0000 7f8668bfe740 1 -- 172.21.15.1:0/3056156016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eab8cc45d0 msgr2=0x55eab8cd30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:00.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.557+0000 7f8668bfe740 1 --2- 172.21.15.1:0/3056156016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eab8cc45d0 0x55eab8cd30d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8650015850 tx=0x7f86500047d0 comp rx=0 tx=0).stop 2024-09-11T22:00:00.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.557+0000 7f8668bfe740 1 -- 172.21.15.1:0/3056156016 shutdown_connections 2024-09-11T22:00:00.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.557+0000 7f8668bfe740 1 --2- 172.21.15.1:0/3056156016 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x55eab8ce3270 0x55eab8ce5660 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:00.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.557+0000 7f863f7fe640 0 client.25547 ms_handle_reset on v2:172.21.15.161:6836/1799318656 2024-09-11T22:00:00.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.557+0000 7f8668bfe740 1 --2- 172.21.15.1:0/3056156016 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eab8cd3780 0x55eab8cd08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:00.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.557+0000 7f8668bfe740 1 --2- 172.21.15.1:0/3056156016 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eab8cc5b40 0x55eab8cd0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:00.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.557+0000 7f8668bfe740 1 --2- 172.21.15.1:0/3056156016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eab8cc45d0 0x55eab8cd30d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:00.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.557+0000 7f8668bfe740 1 -- 172.21.15.1:0/3056156016 >> 172.21.15.1:0/3056156016 conn(0x55eab8c8d310 msgr2=0x55eab8cc6930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:00:00.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.557+0000 7f8668bfe740 1 -- 172.21.15.1:0/3056156016 shutdown_connections 2024-09-11T22:00:00.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.557+0000 7f8668bfe740 1 -- 172.21.15.1:0/3056156016 wait complete. 2024-09-11T22:00:00.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.557+0000 7f8668bfe740 20 client.25547 trim_cache size 0 max 16384 2024-09-11T22:00:00.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.583+0000 7f8668bfe740 1 Processor -- start 2024-09-11T22:00:00.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.583+0000 7f8668bfe740 1 -- start start 2024-09-11T22:00:00.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.583+0000 7f8668bfe740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eab8cc45d0 0x55eab8cdf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:00.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.584+0000 7f8668bfe740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eab8cc5b40 0x55eab8ccfc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:00.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.584+0000 7f8668bfe740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eab8cd3780 0x55eab8ceac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:00.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.584+0000 7f8668bfe740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f864c00e8d0 con 0x55eab8cc5b40 2024-09-11T22:00:00.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.584+0000 7f8668bfe740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f864c010410 con 0x55eab8cc45d0 2024-09-11T22:00:00.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.584+0000 7f8668bfe740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f864c00efa0 con 0x55eab8cd3780 2024-09-11T22:00:00.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.584+0000 7f865f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eab8cc5b40 0x55eab8ccfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:00.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.584+0000 7f865f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eab8cc5b40 0x55eab8ccfc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45648/0 (socket says 172.21.15.1:45648) 2024-09-11T22:00:00.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.584+0000 7f865f7fe640 1 -- 172.21.15.1:0/895992170 learned_addr learned my addr 172.21.15.1:0/895992170 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:00:00.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.584+0000 7f865ffff640 1 --2- 172.21.15.1:0/895992170 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eab8cc45d0 0x55eab8cdf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:00.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.584+0000 7f865f7fe640 1 -- 172.21.15.1:0/895992170 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eab8cd3780 msgr2=0x55eab8ceac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:00:00.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.584+0000 7f8664dc2640 1 --2- 172.21.15.1:0/895992170 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eab8cd3780 0x55eab8ceac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:00.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.584+0000 7f865f7fe640 1 --2- 172.21.15.1:0/895992170 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eab8cd3780 0x55eab8ceac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:00.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.584+0000 7f865f7fe640 1 -- 172.21.15.1:0/895992170 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eab8cc45d0 msgr2=0x55eab8cdf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:00:00.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.584+0000 7f865f7fe640 1 --2- 172.21.15.1:0/895992170 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eab8cc45d0 0x55eab8cdf750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:00.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.584+0000 7f865f7fe640 1 -- 172.21.15.1:0/895992170 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eab8cc1730 con 0x55eab8cc5b40 2024-09-11T22:00:00.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.584+0000 7f8664dc2640 1 --2- 172.21.15.1:0/895992170 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eab8cd3780 0x55eab8ceac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:00:00.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.584+0000 7f865f7fe640 1 --2- 172.21.15.1:0/895992170 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eab8cc5b40 0x55eab8ccfc60 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f865402a9a0 tx=0x7f8654005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be9d90abba96a66f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:00:00.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.585+0000 7f8645ffb640 1 -- 172.21.15.1:0/895992170 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8654035040 con 0x55eab8cc5b40 2024-09-11T22:00:00.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.585+0000 7f8645ffb640 1 -- 172.21.15.1:0/895992170 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8654005500 con 0x55eab8cc5b40 2024-09-11T22:00:00.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.585+0000 7f8645ffb640 1 -- 172.21.15.1:0/895992170 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86540318e0 con 0x55eab8cc5b40 2024-09-11T22:00:00.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.585+0000 7f8668bfe740 1 -- 172.21.15.1:0/895992170 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eab8cc5b40 msgr2=0x55eab8ccfc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:00.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.585+0000 7f8668bfe740 1 --2- 172.21.15.1:0/895992170 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eab8cc5b40 0x55eab8ccfc60 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f865402a9a0 tx=0x7f8654005b50 comp rx=0 tx=0).stop 2024-09-11T22:00:00.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.586+0000 7f8668bfe740 1 -- 172.21.15.1:0/895992170 shutdown_connections 2024-09-11T22:00:00.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.586+0000 7f8668bfe740 1 --2- 172.21.15.1:0/895992170 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eab8cd3780 0x55eab8ceac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:00.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.586+0000 7f8668bfe740 1 --2- 172.21.15.1:0/895992170 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eab8cc5b40 0x55eab8ccfc60 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:00.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.586+0000 7f8668bfe740 1 --2- 172.21.15.1:0/895992170 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eab8cc45d0 0x55eab8cdf750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:00.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.586+0000 7f8668bfe740 1 -- 172.21.15.1:0/895992170 >> 172.21.15.1:0/895992170 conn(0x55eab8c8d310 msgr2=0x55eab8cbfd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:00:00.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.586+0000 7f8668bfe740 1 -- 172.21.15.1:0/895992170 shutdown_connections 2024-09-11T22:00:00.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.586+0000 7f8668bfe740 1 -- 172.21.15.1:0/895992170 wait complete. 2024-09-11T22:00:00.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.586+0000 7f8668bfe740 1 -- start start 2024-09-11T22:00:00.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.586+0000 7f8668bfe740 1 Processor -- start 2024-09-11T22:00:00.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.587+0000 7f8668bfe740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eab8cc45d0 0x55eab8cdbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:00.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.587+0000 7f8668bfe740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eab8cc5b40 0x55eab8cdc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:00.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.587+0000 7f8668bfe740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eab8cd3780 0x55eab8cd6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:00.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.587+0000 7f8668bfe740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f864c00fd40 con 0x55eab8cd3780 2024-09-11T22:00:00.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.587+0000 7f8668bfe740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f864c00f670 con 0x55eab8cc45d0 2024-09-11T22:00:00.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.587+0000 7f8668bfe740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f864c00c8b0 con 0x55eab8cc5b40 2024-09-11T22:00:00.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.587+0000 7f8668bfe740 10 client.0 _ll_register_callbacks cb 0x7f8654009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:00:00.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.587+0000 7f865f7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eab8cc5b40 0x55eab8cdc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:00.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.587+0000 7f8664dc2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eab8cd3780 0x55eab8cd6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:00.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.587+0000 7f865f7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eab8cc5b40 0x55eab8cdc0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:35670/0 (socket says 172.21.15.1:35670) 2024-09-11T22:00:00.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.587+0000 7f865f7fe640 1 -- 172.21.15.1:0/3690670712 learned_addr learned my addr 172.21.15.1:0/3690670712 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:00:00.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.587+0000 7f865ffff640 1 --2- 172.21.15.1:0/3690670712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eab8cc45d0 0x55eab8cdbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:00.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.587+0000 7f8664dc2640 1 -- 172.21.15.1:0/3690670712 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eab8cc5b40 msgr2=0x55eab8cdc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:00:00.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.587+0000 7f8664dc2640 1 --2- 172.21.15.1:0/3690670712 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eab8cc5b40 0x55eab8cdc0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:00.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.587+0000 7f8664dc2640 1 -- 172.21.15.1:0/3690670712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eab8cc45d0 msgr2=0x55eab8cdbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:00:00.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.587+0000 7f8664dc2640 1 --2- 172.21.15.1:0/3690670712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eab8cc45d0 0x55eab8cdbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:00.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.587+0000 7f8664dc2640 1 -- 172.21.15.1:0/3690670712 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eab8cd7750 con 0x55eab8cd3780 2024-09-11T22:00:00.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.588+0000 7f865ffff640 1 --2- 172.21.15.1:0/3690670712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eab8cc45d0 0x55eab8cdbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:00:00.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.588+0000 7f8664dc2640 1 --2- 172.21.15.1:0/3690670712 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eab8cd3780 0x55eab8cd6f20 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f8658005a70 tx=0x7f865809b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b2d61bc9424625a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:00:00.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.588+0000 7f863f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:00:00.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.588+0000 7f863f7fe640 1 -- 172.21.15.1:0/3690670712 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865801d040 con 0x55eab8cd3780 2024-09-11T22:00:00.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.588+0000 7f863f7fe640 1 -- 172.21.15.1:0/3690670712 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f865809cb40 con 0x55eab8cd3780 2024-09-11T22:00:00.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.588+0000 7f8668bfe740 1 -- 172.21.15.1:0/3690670712 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8654005500 con 0x55eab8cd3780 2024-09-11T22:00:00.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.588+0000 7f863f7fe640 1 -- 172.21.15.1:0/3690670712 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f865809cd40 con 0x55eab8cd3780 2024-09-11T22:00:00.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.588+0000 7f8668bfe740 10 client.15975 fetch_fsmap learned FSMap version 28 2024-09-11T22:00:00.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.588+0000 7f8668bfe740 1 -- 172.21.15.1:0/3690670712 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55eab8cc7310 con 0x55eab8cd3780 2024-09-11T22:00:00.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.588+0000 7f863f7fe640 1 -- 172.21.15.1:0/3690670712 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8654005500 con 0x55eab8cd3780 2024-09-11T22:00:00.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.589+0000 7f863f7fe640 1 -- 172.21.15.1:0/3690670712 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f865809a6d0 con 0x55eab8cd3780 2024-09-11T22:00:00.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.589+0000 7f8668bfe740 10 client.15975 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:00:00.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.589+0000 7f8668bfe740 10 client.15975 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-11T22:00:00.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.589+0000 7f8668bfe740 20 client.15975 populate_metadata read hostname 'smithi001' 2024-09-11T22:00:00.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.589+0000 7f8668bfe740 1 --2- 172.21.15.1:0/3690670712 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x55eab8ce3270 0x55eab8cc9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:00.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.589+0000 7f8668bfe740 4 client.15975 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:00:00.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.589+0000 7f8668bfe740 1 -- 172.21.15.1:0/3690670712 --> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55eab8c31d70 con 0x55eab8ce3270 2024-09-11T22:00:00.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.589+0000 7f865ffff640 1 --2- 172.21.15.1:0/3690670712 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x55eab8ce3270 0x55eab8cc9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:00.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.590+0000 7f865ffff640 1 --2- 172.21.15.1:0/3690670712 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x55eab8ce3270 0x55eab8cc9b10 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a9bb1d59501dc204 server_cookie=7ec5cf58de4a4502 in_seq=0 out_seq=0 2024-09-11T22:00:00.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.590+0000 7f863f7fe640 10 client.15975 ms_handle_connect on v2:172.21.15.161:6836/1799318656 2024-09-11T22:00:00.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.590+0000 7f863f7fe640 1 -- 172.21.15.1:0/3690670712 <== mds.3 v2:172.21.15.161:6836/1799318656 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55eab8c31d70 con 0x55eab8ce3270 2024-09-11T22:00:00.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.590+0000 7f863f7fe640 10 client.15975 handle_command_reply: tid=0 2024-09-11T22:00:00.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.590+0000 7f8668bfe740 1 client.15975 shutdown 2024-09-11T22:00:00.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.590+0000 7f8668bfe740 1 -- 172.21.15.1:0/3690670712 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eab8cd3780 msgr2=0x55eab8cd6f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:00.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.590+0000 7f8668bfe740 1 --2- 172.21.15.1:0/3690670712 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eab8cd3780 0x55eab8cd6f20 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f8658005a70 tx=0x7f865809b810 comp rx=0 tx=0).stop 2024-09-11T22:00:00.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.590+0000 7f8668bfe740 1 -- 172.21.15.1:0/3690670712 shutdown_connections 2024-09-11T22:00:00.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.590+0000 7f8668bfe740 1 --2- 172.21.15.1:0/3690670712 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x55eab8ce3270 0x55eab8cc9b10 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:00.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.590+0000 7f8668bfe740 1 --2- 172.21.15.1:0/3690670712 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eab8cd3780 0x55eab8cd6f20 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:00.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.590+0000 7f8668bfe740 1 --2- 172.21.15.1:0/3690670712 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eab8cc5b40 0x55eab8cdc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:00.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.590+0000 7f8668bfe740 1 --2- 172.21.15.1:0/3690670712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eab8cc45d0 0x55eab8cdbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:00.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.590+0000 7f863f7fe640 0 client.15975 ms_handle_reset on v2:172.21.15.161:6836/1799318656 2024-09-11T22:00:00.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.590+0000 7f8668bfe740 1 -- 172.21.15.1:0/3690670712 >> 172.21.15.1:0/3690670712 conn(0x55eab8c8d310 msgr2=0x55eab8cc6110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:00:00.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.590+0000 7f8668bfe740 1 -- 172.21.15.1:0/3690670712 shutdown_connections 2024-09-11T22:00:00.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.590+0000 7f8668bfe740 1 -- 172.21.15.1:0/3690670712 wait complete. 2024-09-11T22:00:00.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.590+0000 7f8668bfe740 20 client.15975 trim_cache size 0 max 16384 2024-09-11T22:00:00.597 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-11T22:00:00.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.591+0000 7f8666825640 1 -- 172.21.15.1:0/1604313260 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f86480a2d70 msgr2=0x7f86480a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:00.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.591+0000 7f8666825640 1 --2- 172.21.15.1:0/1604313260 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f86480a2d70 0x7f86480a5230 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8650009990 tx=0x7f8650009110 comp rx=0 tx=0).stop 2024-09-11T22:00:00.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.591+0000 7f8666825640 1 -- 172.21.15.1:0/1604313260 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8660102ef0 msgr2=0x7f866019bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:00.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.591+0000 7f8666825640 1 --2- 172.21.15.1:0/1604313260 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8660102ef0 0x7f866019bd30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f865800cad0 tx=0x7f865800ced0 comp rx=0 tx=0).stop 2024-09-11T22:00:00.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.591+0000 7f8666825640 1 -- 172.21.15.1:0/1604313260 shutdown_connections 2024-09-11T22:00:00.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.591+0000 7f8666825640 1 --2- 172.21.15.1:0/1604313260 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f86480a2d70 0x7f86480a5230 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:00.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.591+0000 7f8666825640 1 --2- 172.21.15.1:0/1604313260 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8660102ef0 0x7f866019bd30 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:00.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.591+0000 7f8666825640 1 --2- 172.21.15.1:0/1604313260 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8660102570 0x7f86601979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:00.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.591+0000 7f8666825640 1 --2- 172.21.15.1:0/1604313260 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8660100e20 0x7f8660197460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:00.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.592+0000 7f8666825640 1 -- 172.21.15.1:0/1604313260 >> 172.21.15.1:0/1604313260 conn(0x7f86600fc7d0 msgr2=0x7f86600febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:00:00.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.592+0000 7f8666825640 1 -- 172.21.15.1:0/1604313260 shutdown_connections 2024-09-11T22:00:00.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.592+0000 7f8666825640 1 -- 172.21.15.1:0/1604313260 wait complete. 2024-09-11T22:00:00.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.784+0000 7f4ec80e2640 1 -- 172.21.15.1:0/307072113 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4ec0109090 con 0x7f4ec0102ef0 2024-09-11T22:00:00.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.785+0000 7f4ebeffd640 1 -- 172.21.15.1:0/307072113 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11949 (secure 0 0 0) 0x7f4eb8021070 con 0x7f4ec0102ef0 2024-09-11T22:00:00.786 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T22:00:00.786 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":28,"btime":"2024-09-11T21:57:12:512925+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/421542832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":421542832},{"type":"v1","addr":"172.21.15.1:6835","nonce":421542832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2836864406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2836864406},{"type":"v1","addr":"172.21.15.1:6837","nonce":2836864406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T21:55:20.791263+0000","modified":"2024-09-11T21:57:12.462736+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24484,"mds_2":24467,"mds_3":24466,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14595":{"gid":14595,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/407834672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":407834672},{"type":"v1","addr":"172.21.15.1:6839","nonce":407834672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/1068827678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":1068827678},{"type":"v1","addr":"172.21.15.1:6841","nonce":1068827678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/872410024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":872410024},{"type":"v1","addr":"172.21.15.161:6833","nonce":872410024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/1648944570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1648944570},{"type":"v1","addr":"172.21.15.53:6833","nonce":1648944570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/2126933993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2126933993},{"type":"v1","addr":"172.21.15.53:6835","nonce":2126933993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.161:6835/828381592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":828381592},{"type":"v1","addr":"172.21.15.161:6835","nonce":828381592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.161:6837/1799318656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1799318656},{"type":"v1","addr":"172.21.15.161:6837","nonce":1799318656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.53:6837/3457131830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3457131830},{"type":"v1","addr":"172.21.15.53:6837","nonce":3457131830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/455365383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":455365383},{"type":"v1","addr":"172.21.15.161:6839","nonce":455365383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.53:6839/1378584447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1378584447},{"type":"v1","addr":"172.21.15.53:6839","nonce":1378584447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24460,24466,24467,24484,24494]},"id":1}]} 2024-09-11T22:00:00.787 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 28 2024-09-11T22:00:00.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.787+0000 7f4ec80e2640 1 -- 172.21.15.1:0/307072113 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f4ea80ab580 msgr2=0x7f4ea80ada40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:00.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.787+0000 7f4ec80e2640 1 --2- 172.21.15.1:0/307072113 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f4ea80ab580 0x7f4ea80ada40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0005fd0 tx=0x7f4eb0009680 comp rx=0 tx=0).stop 2024-09-11T22:00:00.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.788+0000 7f4ec80e2640 1 -- 172.21.15.1:0/307072113 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ec0102ef0 msgr2=0x7f4ec01a43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:00.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.788+0000 7f4ec80e2640 1 --2- 172.21.15.1:0/307072113 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ec0102ef0 0x7f4ec01a43f0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f4eb800d950 tx=0x7f4eb800de10 comp rx=0 tx=0).stop 2024-09-11T22:00:00.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.788+0000 7f4ec80e2640 1 -- 172.21.15.1:0/307072113 shutdown_connections 2024-09-11T22:00:00.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.788+0000 7f4ec80e2640 1 --2- 172.21.15.1:0/307072113 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f4ea80ab580 0x7f4ea80ada40 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:00.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.788+0000 7f4ec80e2640 1 --2- 172.21.15.1:0/307072113 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ec0102ef0 0x7f4ec01a43f0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:00.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.788+0000 7f4ec80e2640 1 --2- 172.21.15.1:0/307072113 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ec0102570 0x7f4ec01998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:00.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.788+0000 7f4ec80e2640 1 --2- 172.21.15.1:0/307072113 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ec0100e20 0x7f4ec0199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:00.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.788+0000 7f4ec80e2640 1 -- 172.21.15.1:0/307072113 >> 172.21.15.1:0/307072113 conn(0x7f4ec00fc7d0 msgr2=0x7f4ec00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:00:00.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.788+0000 7f4ec80e2640 1 -- 172.21.15.1:0/307072113 shutdown_connections 2024-09-11T22:00:00.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:00.788+0000 7f4ec80e2640 1 -- 172.21.15.1:0/307072113 wait complete. 2024-09-11T22:00:01.046 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:00:00 smithi001 ceph-mon[30935]: pgmap v88: 97 pgs: 97 active+clean; 2.5 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 21 op/s 2024-09-11T22:00:01.046 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:00:00 smithi001 ceph-mon[30935]: overall HEALTH_OK 2024-09-11T22:00:01.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:00:00 smithi053 ceph-mon[34908]: pgmap v88: 97 pgs: 97 active+clean; 2.5 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 21 op/s 2024-09-11T22:00:01.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:00:00 smithi053 ceph-mon[34908]: overall HEALTH_OK 2024-09-11T22:00:01.091 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:00:00 smithi161 ceph-mon[35615]: pgmap v88: 97 pgs: 97 active+clean; 2.5 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 21 op/s 2024-09-11T22:00:01.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:00:00 smithi161 ceph-mon[35615]: overall HEALTH_OK 2024-09-11T22:00:01.851 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:01.850+0000 7f67d3b8d640 1 -- 172.21.15.1:0/1703648583 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67cc107ad0 msgr2=0x7f67cc107eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:01.851 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:01.850+0000 7f67d3b8d640 1 --2- 172.21.15.1:0/1703648583 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67cc107ad0 0x7f67cc107eb0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f67c8009930 tx=0x7f67c802a420 comp rx=0 tx=0).stop 2024-09-11T22:00:01.851 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:01.850+0000 7f67d3b8d640 1 -- 172.21.15.1:0/1703648583 shutdown_connections 2024-09-11T22:00:01.851 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:01.850+0000 7f67d3b8d640 1 --2- 172.21.15.1:0/1703648583 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f67cc102460 0x7f67cc10e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:01.851 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:01.850+0000 7f67d3b8d640 1 --2- 172.21.15.1:0/1703648583 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67cc101ac0 0x7f67cc101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:01.851 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:01.850+0000 7f67d3b8d640 1 --2- 172.21.15.1:0/1703648583 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67cc107ad0 0x7f67cc107eb0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:01.852 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:01.850+0000 7f67d3b8d640 1 -- 172.21.15.1:0/1703648583 >> 172.21.15.1:0/1703648583 conn(0x7f67cc0fd7a0 msgr2=0x7f67cc0ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:00:01.852 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:01.850+0000 7f67d3b8d640 1 -- 172.21.15.1:0/1703648583 shutdown_connections 2024-09-11T22:00:01.852 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:01.850+0000 7f67d3b8d640 1 -- 172.21.15.1:0/1703648583 wait complete. 2024-09-11T22:00:01.893 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:01.891+0000 7f67d3b8d640 1 Processor -- start 2024-09-11T22:00:01.893 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:01.891+0000 7f67d3b8d640 1 -- start start 2024-09-11T22:00:01.893 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:01.891+0000 7f67d3b8d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f67cc101ac0 0x7f67cc0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:01.893 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:01.891+0000 7f67d3b8d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67cc102460 0x7f67cc074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:01.893 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:01.891+0000 7f67d3b8d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67cc107ad0 0x7f67cc074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:01.893 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:01.891+0000 7f67d3b8d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f67cc111ed0 con 0x7f67cc107ad0 2024-09-11T22:00:01.893 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:01.891+0000 7f67d3b8d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f67cc111d50 con 0x7f67cc102460 2024-09-11T22:00:01.894 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:01.891+0000 7f67d3b8d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f67cc112050 con 0x7f67cc101ac0 2024-09-11T22:00:01.894 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:01.892+0000 7f67d212a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67cc107ad0 0x7f67cc074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:01.894 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:01.892+0000 7f67d212a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67cc107ad0 0x7f67cc074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45664/0 (socket says 172.21.15.1:45664) 2024-09-11T22:00:01.894 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:01.892+0000 7f67d212a640 1 -- 172.21.15.1:0/2524559341 learned_addr learned my addr 172.21.15.1:0/2524559341 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:00:01.894 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:01.892+0000 7f67d1929640 1 --2- 172.21.15.1:0/2524559341 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f67cc101ac0 0x7f67cc0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:01.894 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:01.892+0000 7f67d1128640 1 --2- 172.21.15.1:0/2524559341 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67cc102460 0x7f67cc074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:01.894 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:01.892+0000 7f67d212a640 1 -- 172.21.15.1:0/2524559341 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f67cc101ac0 msgr2=0x7f67cc0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:01.894 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:01.892+0000 7f67d212a640 1 --2- 172.21.15.1:0/2524559341 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f67cc101ac0 0x7f67cc0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:01.894 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:01.892+0000 7f67d212a640 1 -- 172.21.15.1:0/2524559341 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67cc102460 msgr2=0x7f67cc074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:01.894 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:01.892+0000 7f67d212a640 1 --2- 172.21.15.1:0/2524559341 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67cc102460 0x7f67cc074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:01.894 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:01.892+0000 7f67d212a640 1 -- 172.21.15.1:0/2524559341 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67cc1a6960 con 0x7f67cc107ad0 2024-09-11T22:00:01.894 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:01.892+0000 7f67d1128640 1 --2- 172.21.15.1:0/2524559341 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67cc102460 0x7f67cc074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:00:01.894 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:01.892+0000 7f67d212a640 1 --2- 172.21.15.1:0/2524559341 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67cc107ad0 0x7f67cc074f70 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f67c400cad0 tx=0x7f67c400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:00:01.908 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:01.907+0000 7f67c2ffd640 1 -- 172.21.15.1:0/2524559341 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c400ae20 con 0x7f67cc107ad0 2024-09-11T22:00:01.908 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:01.907+0000 7f67d3b8d640 1 -- 172.21.15.1:0/2524559341 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67cc1a6c50 con 0x7f67cc107ad0 2024-09-11T22:00:01.908 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:01.907+0000 7f67c2ffd640 1 -- 172.21.15.1:0/2524559341 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67c4005020 con 0x7f67cc107ad0 2024-09-11T22:00:01.908 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:01.907+0000 7f67c2ffd640 1 -- 172.21.15.1:0/2524559341 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c400f6f0 con 0x7f67cc107ad0 2024-09-11T22:00:01.908 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:01.907+0000 7f67d3b8d640 1 -- 172.21.15.1:0/2524559341 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67cc1a71a0 con 0x7f67cc107ad0 2024-09-11T22:00:01.910 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:01.909+0000 7f67c2ffd640 1 -- 172.21.15.1:0/2524559341 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f67c40051d0 con 0x7f67cc107ad0 2024-09-11T22:00:01.911 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:01.910+0000 7f67c2ffd640 1 --2- 172.21.15.1:0/2524559341 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f67b80a2e40 0x7f67b80a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:01.911 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:01.910+0000 7f67c2ffd640 1 -- 172.21.15.1:0/2524559341 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f67c40cf3f0 con 0x7f67cc107ad0 2024-09-11T22:00:01.911 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:01.910+0000 7f67d1929640 1 --2- 172.21.15.1:0/2524559341 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f67b80a2e40 0x7f67b80a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:01.911 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:01.911+0000 7f67d1929640 1 --2- 172.21.15.1:0/2524559341 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f67b80a2e40 0x7f67b80a5300 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f67c8009900 tx=0x7f67c80058d0 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:00:02.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:00:01 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/307072113' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:00:02.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:00:01 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/307072113' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:00:02.123 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.122+0000 7f67d6a07740 1 Processor -- start 2024-09-11T22:00:02.123 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.122+0000 7f67d6a07740 1 -- start start 2024-09-11T22:00:02.124 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.123+0000 7f67d6a07740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56488f311af0 0x56488f311ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:02.124 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.123+0000 7f67d6a07740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56488f312410 0x56488f31c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:02.124 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.123+0000 7f67d6a07740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56488f31ccb0 0x56488f31f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:02.124 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.123+0000 7f67d6a07740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56488f2762f0 con 0x56488f31ccb0 2024-09-11T22:00:02.124 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.123+0000 7f67d6a07740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56488f21e270 con 0x56488f312410 2024-09-11T22:00:02.125 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.123+0000 7f67d6a07740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56488f2722b0 con 0x56488f311af0 2024-09-11T22:00:02.125 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.123+0000 7f67d212a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56488f31ccb0 0x56488f31f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:02.125 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.123+0000 7f67d212a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56488f31ccb0 0x56488f31f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45666/0 (socket says 172.21.15.1:45666) 2024-09-11T22:00:02.125 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.123+0000 7f67d212a640 1 -- 172.21.15.1:0/142327372 learned_addr learned my addr 172.21.15.1:0/142327372 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:00:02.125 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.123+0000 7f67d1929640 1 --2- 172.21.15.1:0/142327372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56488f311af0 0x56488f311ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:02.125 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.123+0000 7f67d1128640 1 --2- 172.21.15.1:0/142327372 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56488f312410 0x56488f31c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:02.125 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.123+0000 7f67d212a640 1 -- 172.21.15.1:0/142327372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56488f311af0 msgr2=0x56488f311ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:00:02.125 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.123+0000 7f67d212a640 1 --2- 172.21.15.1:0/142327372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56488f311af0 0x56488f311ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:02.125 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.123+0000 7f67d212a640 1 -- 172.21.15.1:0/142327372 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56488f312410 msgr2=0x56488f31c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:00:02.125 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.123+0000 7f67d212a640 1 --2- 172.21.15.1:0/142327372 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56488f312410 0x56488f31c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:02.125 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.123+0000 7f67d212a640 1 -- 172.21.15.1:0/142327372 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56488f21d340 con 0x56488f31ccb0 2024-09-11T22:00:02.125 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.123+0000 7f67d212a640 1 --2- 172.21.15.1:0/142327372 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56488f31ccb0 0x56488f31f0a0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f67c40a3fb0 tx=0x7f67c409bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84d3b469ba8676c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:00:02.125 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.124+0000 7f67b7fff640 1 -- 172.21.15.1:0/142327372 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c401d070 con 0x56488f31ccb0 2024-09-11T22:00:02.126 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.124+0000 7f67b7fff640 1 -- 172.21.15.1:0/142327372 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67c40b4070 con 0x56488f31ccb0 2024-09-11T22:00:02.126 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.124+0000 7f67b7fff640 1 -- 172.21.15.1:0/142327372 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c409cba0 con 0x56488f31ccb0 2024-09-11T22:00:02.126 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.124+0000 7f67d6a07740 1 -- 172.21.15.1:0/142327372 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56488f31ccb0 msgr2=0x56488f31f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:02.127 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.124+0000 7f67d6a07740 1 --2- 172.21.15.1:0/142327372 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56488f31ccb0 0x56488f31f0a0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f67c40a3fb0 tx=0x7f67c409bf70 comp rx=0 tx=0).stop 2024-09-11T22:00:02.127 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.124+0000 7f67d6a07740 1 -- 172.21.15.1:0/142327372 shutdown_connections 2024-09-11T22:00:02.127 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.124+0000 7f67d6a07740 1 --2- 172.21.15.1:0/142327372 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56488f31ccb0 0x56488f31f0a0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:02.127 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.124+0000 7f67d6a07740 1 --2- 172.21.15.1:0/142327372 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56488f312410 0x56488f31c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:02.127 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.124+0000 7f67d6a07740 1 --2- 172.21.15.1:0/142327372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56488f311af0 0x56488f311ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:02.127 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.124+0000 7f67d6a07740 1 -- 172.21.15.1:0/142327372 >> 172.21.15.1:0/142327372 conn(0x56488f30e060 msgr2=0x56488f30e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:00:02.127 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.125+0000 7f67d6a07740 1 -- 172.21.15.1:0/142327372 shutdown_connections 2024-09-11T22:00:02.127 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.125+0000 7f67d6a07740 1 -- 172.21.15.1:0/142327372 wait complete. 2024-09-11T22:00:02.168 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.168+0000 7f67d6a07740 1 -- start start 2024-09-11T22:00:02.192 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.192+0000 7f67d6a07740 1 Processor -- start 2024-09-11T22:00:02.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:00:01 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/307072113' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:00:02.194 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.192+0000 7f67d6a07740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56488f311af0 0x56488f319c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:02.195 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.192+0000 7f67d6a07740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56488f312410 0x56488f31a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:02.195 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.192+0000 7f67d6a07740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56488f31ccb0 0x56488f31a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:02.195 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.192+0000 7f67d6a07740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56488f31f880 con 0x56488f312410 2024-09-11T22:00:02.195 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.192+0000 7f67d6a07740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56488f1ff290 con 0x56488f31ccb0 2024-09-11T22:00:02.195 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.193+0000 7f67d6a07740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56488f31fa00 con 0x56488f311af0 2024-09-11T22:00:02.195 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.193+0000 7f67d1128640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56488f312410 0x56488f31a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:02.195 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.193+0000 7f67d1128640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56488f312410 0x56488f31a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45670/0 (socket says 172.21.15.1:45670) 2024-09-11T22:00:02.195 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.193+0000 7f67d1128640 1 -- 172.21.15.1:0/2193927351 learned_addr learned my addr 172.21.15.1:0/2193927351 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:00:02.196 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.193+0000 7f67d1929640 1 --2- 172.21.15.1:0/2193927351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56488f311af0 0x56488f319c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:02.196 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.193+0000 7f67d212a640 1 --2- 172.21.15.1:0/2193927351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56488f31ccb0 0x56488f31a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:02.196 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.193+0000 7f67d1128640 1 -- 172.21.15.1:0/2193927351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56488f311af0 msgr2=0x56488f319c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:00:02.196 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.193+0000 7f67d1128640 1 --2- 172.21.15.1:0/2193927351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56488f311af0 0x56488f319c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:02.196 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.193+0000 7f67d1128640 1 -- 172.21.15.1:0/2193927351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56488f31ccb0 msgr2=0x56488f31a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:00:02.196 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.193+0000 7f67d1128640 1 --2- 172.21.15.1:0/2193927351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56488f31ccb0 0x56488f31a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:02.196 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.193+0000 7f67d1128640 1 -- 172.21.15.1:0/2193927351 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56488ef89210 con 0x56488f312410 2024-09-11T22:00:02.196 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.193+0000 7f67d1128640 1 --2- 172.21.15.1:0/2193927351 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56488f312410 0x56488f31a180 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f67bc003b30 tx=0x7f67bc0113e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f74a84e89d9d83a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:00:02.196 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.193+0000 7f67b57fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:00:02.197 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.193+0000 7f67b57fa640 1 -- 172.21.15.1:0/2193927351 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67bc011d90 con 0x56488f312410 2024-09-11T22:00:02.197 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.193+0000 7f67b57fa640 1 -- 172.21.15.1:0/2193927351 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f67bc005020 con 0x56488f312410 2024-09-11T22:00:02.220 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.220+0000 7f67d6a07740 10 client.0 _ll_register_callbacks cb 0x56488f230140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:00:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.225+0000 7f67d6a07740 1 -- 172.21.15.1:0/2193927351 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67c40b4070 con 0x56488f312410 2024-09-11T22:00:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.225+0000 7f67b57fa640 1 -- 172.21.15.1:0/2193927351 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f67bc005220 con 0x56488f312410 2024-09-11T22:00:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.225+0000 7f67d6a07740 10 client.15987 fetch_fsmap learned FSMap version 28 2024-09-11T22:00:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.225+0000 7f67d6a07740 1 -- 172.21.15.1:0/2193927351 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({fsmap=28}) -- 0x56488f055620 con 0x56488f312410 2024-09-11T22:00:02.228 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.226+0000 7f67b57fa640 1 -- 172.21.15.1:0/2193927351 <== mon.0 v2:172.21.15.1:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f67bc0053c0 con 0x56488f312410 2024-09-11T22:00:02.228 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.226+0000 7f67d6a07740 10 client.15987 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:00:02.228 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.226+0000 7f67d6a07740 10 client.15987 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:00:02.228 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.226+0000 7f67d6a07740 20 client.15987 populate_metadata read hostname 'smithi001' 2024-09-11T22:00:02.228 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.226+0000 7f67d6a07740 1 --2- 172.21.15.1:0/2193927351 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x56488f32ca40 0x56488f32ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:02.229 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.226+0000 7f67d6a07740 4 client.15987 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:00:02.229 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.226+0000 7f67d6a07740 1 -- 172.21.15.1:0/2193927351 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56488f283a20 con 0x56488f32ca40 2024-09-11T22:00:02.229 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.227+0000 7f67d1929640 1 --2- 172.21.15.1:0/2193927351 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x56488f32ca40 0x56488f32ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:02.229 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.227+0000 7f67d1929640 1 --2- 172.21.15.1:0/2193927351 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x56488f32ca40 0x56488f32ef00 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a72eb7da3b986bb1 server_cookie=c14a8c13853dcee5 in_seq=0 out_seq=0 2024-09-11T22:00:02.229 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.227+0000 7f67b57fa640 10 client.15987 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:00:02.230 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.228+0000 7f67b57fa640 1 -- 172.21.15.1:0/2193927351 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56488f283a20 con 0x56488f32ca40 2024-09-11T22:00:02.230 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.228+0000 7f67b57fa640 10 client.15987 handle_command_reply: tid=0 2024-09-11T22:00:02.230 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.228+0000 7f67d6a07740 1 client.15987 shutdown 2024-09-11T22:00:02.231 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.228+0000 7f67d6a07740 1 -- 172.21.15.1:0/2193927351 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56488f312410 msgr2=0x56488f31a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:02.231 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.228+0000 7f67d6a07740 1 --2- 172.21.15.1:0/2193927351 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56488f312410 0x56488f31a180 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f67bc003b30 tx=0x7f67bc0113e0 comp rx=0 tx=0).stop 2024-09-11T22:00:02.231 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.228+0000 7f67d6a07740 1 -- 172.21.15.1:0/2193927351 shutdown_connections 2024-09-11T22:00:02.231 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.228+0000 7f67d6a07740 1 --2- 172.21.15.1:0/2193927351 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x56488f32ca40 0x56488f32ef00 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:02.231 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.228+0000 7f67d6a07740 1 --2- 172.21.15.1:0/2193927351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56488f31ccb0 0x56488f31a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:02.231 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.228+0000 7f67d6a07740 1 --2- 172.21.15.1:0/2193927351 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56488f312410 0x56488f31a180 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:02.231 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.228+0000 7f67d6a07740 1 --2- 172.21.15.1:0/2193927351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56488f311af0 0x56488f319c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:02.231 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.228+0000 7f67b57fa640 0 client.15987 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:00:02.231 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.228+0000 7f67d6a07740 1 -- 172.21.15.1:0/2193927351 >> 172.21.15.1:0/2193927351 conn(0x56488f30e060 msgr2=0x56488f313d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:00:02.231 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.229+0000 7f67d6a07740 1 -- 172.21.15.1:0/2193927351 shutdown_connections 2024-09-11T22:00:02.231 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.229+0000 7f67d6a07740 1 -- 172.21.15.1:0/2193927351 wait complete. 2024-09-11T22:00:02.232 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.229+0000 7f67d6a07740 20 client.15987 trim_cache size 0 max 16384 2024-09-11T22:00:02.246 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.243+0000 7f67d6a07740 1 Processor -- start 2024-09-11T22:00:02.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.244+0000 7f67d6a07740 1 -- start start 2024-09-11T22:00:02.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.244+0000 7f67d6a07740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56488f311af0 0x56488f31bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:02.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.244+0000 7f67d6a07740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56488f312410 0x56488f31a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:02.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.244+0000 7f67d6a07740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56488f31ccb0 0x56488f3347d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:02.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.244+0000 7f67d6a07740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f67cc1afc30 con 0x56488f311af0 2024-09-11T22:00:02.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.244+0000 7f67d6a07740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f67cc1b1770 con 0x56488f312410 2024-09-11T22:00:02.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.244+0000 7f67d6a07740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f67cc1b0300 con 0x56488f31ccb0 2024-09-11T22:00:02.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.244+0000 7f67d1929640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56488f311af0 0x56488f31bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:02.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.244+0000 7f67d1929640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56488f311af0 0x56488f31bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45684/0 (socket says 172.21.15.1:45684) 2024-09-11T22:00:02.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.244+0000 7f67d1929640 1 -- 172.21.15.1:0/2891320979 learned_addr learned my addr 172.21.15.1:0/2891320979 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:00:02.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.244+0000 7f67d212a640 1 --2- 172.21.15.1:0/2891320979 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56488f31ccb0 0x56488f3347d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:02.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.244+0000 7f67d1128640 1 --2- 172.21.15.1:0/2891320979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56488f312410 0x56488f31a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:02.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.244+0000 7f67d1929640 1 -- 172.21.15.1:0/2891320979 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56488f31ccb0 msgr2=0x56488f3347d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:00:02.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.244+0000 7f67d1929640 1 --2- 172.21.15.1:0/2891320979 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56488f31ccb0 0x56488f3347d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:02.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.244+0000 7f67d1929640 1 -- 172.21.15.1:0/2891320979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56488f312410 msgr2=0x56488f31a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:00:02.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.244+0000 7f67d1929640 1 --2- 172.21.15.1:0/2891320979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56488f312410 0x56488f31a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:02.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.244+0000 7f67d1929640 1 -- 172.21.15.1:0/2891320979 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56488f3157e0 con 0x56488f311af0 2024-09-11T22:00:02.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.244+0000 7f67d1128640 1 --2- 172.21.15.1:0/2891320979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56488f312410 0x56488f31a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:00:02.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.244+0000 7f67d1929640 1 --2- 172.21.15.1:0/2891320979 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56488f311af0 0x56488f31bf70 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f67c8054fd0 tx=0x7f67c8040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60e1dd9715affe45 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:00:02.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.245+0000 7f67b77fe640 1 -- 172.21.15.1:0/2891320979 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c8035040 con 0x56488f311af0 2024-09-11T22:00:02.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.245+0000 7f67b77fe640 1 -- 172.21.15.1:0/2891320979 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67c8003e50 con 0x56488f311af0 2024-09-11T22:00:02.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.245+0000 7f67b77fe640 1 -- 172.21.15.1:0/2891320979 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c804b440 con 0x56488f311af0 2024-09-11T22:00:02.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.245+0000 7f67d6a07740 1 -- 172.21.15.1:0/2891320979 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56488f311af0 msgr2=0x56488f31bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.245+0000 7f67d6a07740 1 --2- 172.21.15.1:0/2891320979 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56488f311af0 0x56488f31bf70 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f67c8054fd0 tx=0x7f67c8040000 comp rx=0 tx=0).stop 2024-09-11T22:00:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.246+0000 7f67d6a07740 1 -- 172.21.15.1:0/2891320979 shutdown_connections 2024-09-11T22:00:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.246+0000 7f67d6a07740 1 --2- 172.21.15.1:0/2891320979 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56488f31ccb0 0x56488f3347d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.246+0000 7f67d6a07740 1 --2- 172.21.15.1:0/2891320979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56488f312410 0x56488f31a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.246+0000 7f67d6a07740 1 --2- 172.21.15.1:0/2891320979 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56488f311af0 0x56488f31bf70 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.246+0000 7f67d6a07740 1 -- 172.21.15.1:0/2891320979 >> 172.21.15.1:0/2891320979 conn(0x56488f30e060 msgr2=0x56488f30d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:00:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.246+0000 7f67d6a07740 1 -- 172.21.15.1:0/2891320979 shutdown_connections 2024-09-11T22:00:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.246+0000 7f67d6a07740 1 -- 172.21.15.1:0/2891320979 wait complete. 2024-09-11T22:00:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.246+0000 7f67d6a07740 1 -- start start 2024-09-11T22:00:02.249 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.246+0000 7f67d6a07740 1 Processor -- start 2024-09-11T22:00:02.252 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.247+0000 7f67d6a07740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56488f311af0 0x56488f325050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:02.252 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.247+0000 7f67d6a07740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56488f312410 0x56488f325590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:02.252 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.247+0000 7f67d6a07740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56488f31ccb0 0x56488f3215d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:02.253 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.247+0000 7f67d6a07740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f67cc1b10a0 con 0x56488f31ccb0 2024-09-11T22:00:02.253 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.247+0000 7f67d6a07740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f67cc1b09d0 con 0x56488f311af0 2024-09-11T22:00:02.253 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.247+0000 7f67d6a07740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f67cc1adc10 con 0x56488f312410 2024-09-11T22:00:02.253 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.247+0000 7f67d6a07740 10 client.0 _ll_register_callbacks cb 0x7f67bc0040e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:00:02.253 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.247+0000 7f67d212a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56488f31ccb0 0x56488f3215d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:02.253 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.247+0000 7f67d1929640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56488f311af0 0x56488f325050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:02.253 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.247+0000 7f67d212a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56488f31ccb0 0x56488f3215d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45692/0 (socket says 172.21.15.1:45692) 2024-09-11T22:00:02.253 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.247+0000 7f67d212a640 1 -- 172.21.15.1:0/3376862354 learned_addr learned my addr 172.21.15.1:0/3376862354 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:00:02.253 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.247+0000 7f67d1128640 1 --2- 172.21.15.1:0/3376862354 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56488f312410 0x56488f325590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:02.253 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.247+0000 7f67d1929640 1 -- 172.21.15.1:0/3376862354 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56488f312410 msgr2=0x56488f325590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:00:02.253 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.247+0000 7f67d1929640 1 --2- 172.21.15.1:0/3376862354 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56488f312410 0x56488f325590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:02.253 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.247+0000 7f67d1929640 1 -- 172.21.15.1:0/3376862354 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56488f31ccb0 msgr2=0x56488f3215d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:00:02.253 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.247+0000 7f67d1929640 1 --2- 172.21.15.1:0/3376862354 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56488f31ccb0 0x56488f3215d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.247+0000 7f67d1929640 1 -- 172.21.15.1:0/3376862354 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56488f321dd0 con 0x56488f311af0 2024-09-11T22:00:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.247+0000 7f67d212a640 1 --2- 172.21.15.1:0/3376862354 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56488f31ccb0 0x56488f3215d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:00:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.247+0000 7f67d1929640 1 --2- 172.21.15.1:0/3376862354 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56488f311af0 0x56488f325050 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f67c8005ec0 tx=0x7f67c8054ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17f96f9392331561 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:00:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.247+0000 7f67b57fa640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-11T22:00:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.248+0000 7f67b57fa640 1 -- 172.21.15.1:0/3376862354 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c8035070 con 0x56488f311af0 2024-09-11T22:00:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.248+0000 7f67d6a07740 1 -- 172.21.15.1:0/3376862354 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67c8003e50 con 0x56488f311af0 2024-09-11T22:00:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.248+0000 7f67b57fa640 1 -- 172.21.15.1:0/3376862354 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f67c8003970 con 0x56488f311af0 2024-09-11T22:00:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.248+0000 7f67b57fa640 1 -- 172.21.15.1:0/3376862354 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f67c8003b90 con 0x56488f311af0 2024-09-11T22:00:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.248+0000 7f67d6a07740 10 client.25571 fetch_fsmap learned FSMap version 28 2024-09-11T22:00:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.248+0000 7f67d6a07740 1 -- 172.21.15.1:0/3376862354 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56488f398440 con 0x56488f311af0 2024-09-11T22:00:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.248+0000 7f67b57fa640 1 -- 172.21.15.1:0/3376862354 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67c8003e50 con 0x56488f311af0 2024-09-11T22:00:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.248+0000 7f67b57fa640 1 -- 172.21.15.1:0/3376862354 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f67c804b960 con 0x56488f311af0 2024-09-11T22:00:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.248+0000 7f67d6a07740 10 client.25571 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:00:02.255 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.248+0000 7f67d6a07740 10 client.25571 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:00:02.255 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.248+0000 7f67d6a07740 20 client.25571 populate_metadata read hostname 'smithi001' 2024-09-11T22:00:02.255 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.248+0000 7f67d6a07740 1 --2- 172.21.15.1:0/3376862354 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x56488f398b10 0x56488f39af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:02.255 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.248+0000 7f67d6a07740 4 client.25571 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:00:02.255 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.248+0000 7f67d6a07740 1 -- 172.21.15.1:0/3376862354 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56488f27dd70 con 0x56488f398b10 2024-09-11T22:00:02.255 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.249+0000 7f67d1128640 1 --2- 172.21.15.1:0/3376862354 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x56488f398b10 0x56488f39af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:02.255 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.249+0000 7f67b57fa640 10 client.25571 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:00:02.255 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.249+0000 7f67d1128640 1 --2- 172.21.15.1:0/3376862354 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x56488f398b10 0x56488f39af00 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea4dbc1801bc6969 server_cookie=55e1fd6d7803a8d8 in_seq=0 out_seq=0 2024-09-11T22:00:02.269 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.268+0000 7f67b57fa640 1 -- 172.21.15.1:0/3376862354 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56488f27dd70 con 0x56488f398b10 2024-09-11T22:00:02.269 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.268+0000 7f67b57fa640 10 client.25571 handle_command_reply: tid=0 2024-09-11T22:00:02.269 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.268+0000 7f67d6a07740 1 client.25571 shutdown 2024-09-11T22:00:02.269 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.268+0000 7f67d6a07740 1 -- 172.21.15.1:0/3376862354 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56488f311af0 msgr2=0x56488f325050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:02.270 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.268+0000 7f67d6a07740 1 --2- 172.21.15.1:0/3376862354 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56488f311af0 0x56488f325050 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f67c8005ec0 tx=0x7f67c8054ae0 comp rx=0 tx=0).stop 2024-09-11T22:00:02.270 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.268+0000 7f67d6a07740 1 -- 172.21.15.1:0/3376862354 shutdown_connections 2024-09-11T22:00:02.270 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.268+0000 7f67d6a07740 1 --2- 172.21.15.1:0/3376862354 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x56488f398b10 0x56488f39af00 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:02.270 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.268+0000 7f67d6a07740 1 --2- 172.21.15.1:0/3376862354 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56488f31ccb0 0x56488f3215d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:02.270 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.268+0000 7f67d6a07740 1 --2- 172.21.15.1:0/3376862354 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56488f312410 0x56488f325590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:02.270 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.268+0000 7f67b57fa640 0 client.25571 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:00:02.270 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.268+0000 7f67d6a07740 1 --2- 172.21.15.1:0/3376862354 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56488f311af0 0x56488f325050 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:02.270 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.268+0000 7f67d6a07740 1 -- 172.21.15.1:0/3376862354 >> 172.21.15.1:0/3376862354 conn(0x56488f30e060 msgr2=0x56488f30c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:00:02.270 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.268+0000 7f67d6a07740 1 -- 172.21.15.1:0/3376862354 shutdown_connections 2024-09-11T22:00:02.270 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.268+0000 7f67d6a07740 1 -- 172.21.15.1:0/3376862354 wait complete. 2024-09-11T22:00:02.270 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.268+0000 7f67d6a07740 20 client.25571 trim_cache size 0 max 16384 2024-09-11T22:00:02.271 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-09-11T22:00:02.271 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-09-11T22:00:02.271 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-09-11T22:00:02.271 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-09-11T22:00:02.272 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.269+0000 7f67d3b8d640 1 -- 172.21.15.1:0/2524559341 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f67b80a2e40 msgr2=0x7f67b80a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:02.272 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.269+0000 7f67d3b8d640 1 --2- 172.21.15.1:0/2524559341 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f67b80a2e40 0x7f67b80a5300 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f67c8009900 tx=0x7f67c80058d0 comp rx=0 tx=0).stop 2024-09-11T22:00:02.272 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.269+0000 7f67d3b8d640 1 -- 172.21.15.1:0/2524559341 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67cc107ad0 msgr2=0x7f67cc074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:02.272 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.269+0000 7f67d3b8d640 1 --2- 172.21.15.1:0/2524559341 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67cc107ad0 0x7f67cc074f70 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f67c400cad0 tx=0x7f67c400cf90 comp rx=0 tx=0).stop 2024-09-11T22:00:02.272 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.269+0000 7f67d3b8d640 1 -- 172.21.15.1:0/2524559341 shutdown_connections 2024-09-11T22:00:02.272 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.269+0000 7f67d3b8d640 1 --2- 172.21.15.1:0/2524559341 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f67b80a2e40 0x7f67b80a5300 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:02.272 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.269+0000 7f67d3b8d640 1 --2- 172.21.15.1:0/2524559341 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67cc107ad0 0x7f67cc074f70 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:02.272 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.269+0000 7f67d3b8d640 1 --2- 172.21.15.1:0/2524559341 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67cc102460 0x7f67cc074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:02.272 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.269+0000 7f67d3b8d640 1 --2- 172.21.15.1:0/2524559341 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f67cc101ac0 0x7f67cc0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:02.272 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.269+0000 7f67d3b8d640 1 -- 172.21.15.1:0/2524559341 >> 172.21.15.1:0/2524559341 conn(0x7f67cc0fd7a0 msgr2=0x7f67cc0fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:00:02.273 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.269+0000 7f67d3b8d640 1 -- 172.21.15.1:0/2524559341 shutdown_connections 2024-09-11T22:00:02.273 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:02.269+0000 7f67d3b8d640 1 -- 172.21.15.1:0/2524559341 wait complete. 2024-09-11T22:00:02.443 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db --set-id ff1ce167 --release --await' 2024-09-11T22:00:02.444 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db --set-id ff1ce167 --release --await 2024-09-11T22:00:03.028 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T22:00:03.131 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:00:03.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:00:02 smithi001 ceph-mon[30935]: pgmap v89: 97 pgs: 97 active+clean; 2.5 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.5 KiB/s wr, 10 op/s 2024-09-11T22:00:03.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:00:02 smithi053 ceph-mon[34908]: pgmap v89: 97 pgs: 97 active+clean; 2.5 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.5 KiB/s wr, 10 op/s 2024-09-11T22:00:03.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:00:02 smithi161 ceph-mon[35615]: pgmap v89: 97 pgs: 97 active+clean; 2.5 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.5 KiB/s wr, 10 op/s 2024-09-11T22:00:03.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.813+0000 7f4f15076640 1 -- 172.21.15.1:0/1725112860 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f10100e20 msgr2=0x7f4f10101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:03.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.813+0000 7f4f15076640 1 --2- 172.21.15.1:0/1725112860 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f10100e20 0x7f4f10101200 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f4f00009ca0 tx=0x7f4f0002a950 comp rx=0 tx=0).stop 2024-09-11T22:00:03.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.813+0000 7f4f15076640 1 -- 172.21.15.1:0/1725112860 shutdown_connections 2024-09-11T22:00:03.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.813+0000 7f4f15076640 1 --2- 172.21.15.1:0/1725112860 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f10102ef0 0x7f4f1010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:03.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.813+0000 7f4f15076640 1 --2- 172.21.15.1:0/1725112860 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f10102570 0x7f4f101029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:03.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.813+0000 7f4f15076640 1 --2- 172.21.15.1:0/1725112860 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f10100e20 0x7f4f10101200 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:03.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.813+0000 7f4f15076640 1 -- 172.21.15.1:0/1725112860 >> 172.21.15.1:0/1725112860 conn(0x7f4f100fc7d0 msgr2=0x7f4f100febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:00:03.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.813+0000 7f4f15076640 1 -- 172.21.15.1:0/1725112860 shutdown_connections 2024-09-11T22:00:03.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.813+0000 7f4f15076640 1 -- 172.21.15.1:0/1725112860 wait complete. 2024-09-11T22:00:03.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.814+0000 7f4f15076640 1 Processor -- start 2024-09-11T22:00:03.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.814+0000 7f4f15076640 1 -- start start 2024-09-11T22:00:03.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.814+0000 7f4f15076640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f10100e20 0x7f4f10199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:03.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.814+0000 7f4f15076640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f10102570 0x7f4f10199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:03.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.814+0000 7f4f15076640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f10102ef0 0x7f4f101a4480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:03.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.814+0000 7f4f15076640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4f10110f50 con 0x7f4f10102ef0 2024-09-11T22:00:03.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.814+0000 7f4f15076640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4f10110dd0 con 0x7f4f10100e20 2024-09-11T22:00:03.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.814+0000 7f4f15076640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4f101110d0 con 0x7f4f10102570 2024-09-11T22:00:03.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.814+0000 7f4f0f59e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f10102ef0 0x7f4f101a4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:03.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.814+0000 7f4f0f59e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f10102ef0 0x7f4f101a4480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45706/0 (socket says 172.21.15.1:45706) 2024-09-11T22:00:03.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.814+0000 7f4f0f59e640 1 -- 172.21.15.1:0/2550491007 learned_addr learned my addr 172.21.15.1:0/2550491007 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:00:03.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.814+0000 7f4f0ed9d640 1 --2- 172.21.15.1:0/2550491007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f10100e20 0x7f4f10199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:03.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.814+0000 7f4f0e59c640 1 --2- 172.21.15.1:0/2550491007 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f10102570 0x7f4f10199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:03.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.815+0000 7f4f0f59e640 1 -- 172.21.15.1:0/2550491007 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f10102570 msgr2=0x7f4f10199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:03.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.815+0000 7f4f0f59e640 1 --2- 172.21.15.1:0/2550491007 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f10102570 0x7f4f10199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:03.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.815+0000 7f4f0f59e640 1 -- 172.21.15.1:0/2550491007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f10100e20 msgr2=0x7f4f10199400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:03.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.815+0000 7f4f0f59e640 1 --2- 172.21.15.1:0/2550491007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f10100e20 0x7f4f10199400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:03.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.815+0000 7f4f0f59e640 1 -- 172.21.15.1:0/2550491007 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f101a49c0 con 0x7f4f10102ef0 2024-09-11T22:00:03.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.815+0000 7f4f0ed9d640 1 --2- 172.21.15.1:0/2550491007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f10100e20 0x7f4f10199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:00:03.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.815+0000 7f4f0f59e640 1 --2- 172.21.15.1:0/2550491007 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f10102ef0 0x7f4f101a4480 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f4f0800c970 tx=0x7f4f0800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:00:03.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.815+0000 7f4ef7fff640 1 -- 172.21.15.1:0/2550491007 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f08007bb0 con 0x7f4f10102ef0 2024-09-11T22:00:03.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.815+0000 7f4ef7fff640 1 -- 172.21.15.1:0/2550491007 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f08007d50 con 0x7f4f10102ef0 2024-09-11T22:00:03.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.815+0000 7f4f15076640 1 -- 172.21.15.1:0/2550491007 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f101a4b90 con 0x7f4f10102ef0 2024-09-11T22:00:03.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.815+0000 7f4ef7fff640 1 -- 172.21.15.1:0/2550491007 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0800f450 con 0x7f4f10102ef0 2024-09-11T22:00:03.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.815+0000 7f4f15076640 1 -- 172.21.15.1:0/2550491007 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f101a50a0 con 0x7f4f10102ef0 2024-09-11T22:00:03.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.818+0000 7f4ef7fff640 1 -- 172.21.15.1:0/2550491007 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f4f08004f60 con 0x7f4f10102ef0 2024-09-11T22:00:03.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.818+0000 7f4f1744f740 1 Processor -- start 2024-09-11T22:00:03.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.818+0000 7f4f1744f740 1 -- start start 2024-09-11T22:00:03.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.819+0000 7f4f1744f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d89495e060 0x55d89495e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:03.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.819+0000 7f4f1744f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d89495e980 0x55d894968d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:03.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.819+0000 7f4ef7fff640 1 --2- 172.21.15.1:0/2550491007 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f4ef80a2d70 0x7f4ef80a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:03.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.819+0000 7f4f1744f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d894969240 0x55d89496b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:03.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.819+0000 7f4f1744f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55d8948c3620 con 0x55d89495e980 2024-09-11T22:00:03.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.819+0000 7f4f1744f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55d89486ae50 con 0x55d894969240 2024-09-11T22:00:03.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.819+0000 7f4f1744f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d8948bf5e0 con 0x55d89495e060 2024-09-11T22:00:03.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.819+0000 7f4f0ed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d89495e980 0x55d894968d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:03.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.819+0000 7f4f0e59c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d89495e060 0x55d89495e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:03.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.819+0000 7f4f0ed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d89495e980 0x55d894968d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45710/0 (socket says 172.21.15.1:45710) 2024-09-11T22:00:03.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.819+0000 7f4ef7fff640 1 -- 172.21.15.1:0/2550491007 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f4f080cf070 con 0x7f4f10102ef0 2024-09-11T22:00:03.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.819+0000 7f4f0ed9d640 1 -- 172.21.15.1:0/1075777001 learned_addr learned my addr 172.21.15.1:0/1075777001 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:00:03.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.819+0000 7f4f0e59c640 1 --2- 172.21.15.1:0/2550491007 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f4ef80a2d70 0x7f4ef80a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:03.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.819+0000 7f4f0f59e640 1 --2- 172.21.15.1:0/1075777001 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d894969240 0x55d89496b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:03.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.819+0000 7f4f0e59c640 1 -- 172.21.15.1:0/1075777001 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d894969240 msgr2=0x55d89496b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:00:03.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.819+0000 7f4f0e59c640 1 --2- 172.21.15.1:0/1075777001 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d894969240 0x55d89496b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:03.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.819+0000 7f4f0e59c640 1 -- 172.21.15.1:0/1075777001 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d89495e980 msgr2=0x55d894968d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:00:03.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.819+0000 7f4f0e59c640 1 --2- 172.21.15.1:0/1075777001 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d89495e980 0x55d894968d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:03.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.819+0000 7f4f0e59c640 1 -- 172.21.15.1:0/1075777001 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d89485bd40 con 0x55d89495e060 2024-09-11T22:00:03.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.820+0000 7f4f0e59c640 1 --2- 172.21.15.1:0/1075777001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d89495e060 0x55d89495e440 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4f1006aac0 tx=0x7f4f04012e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a43db3d70cdfb6cf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:00:03.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.820+0000 7f4f0e59c640 1 --2- 172.21.15.1:0/2550491007 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f4ef80a2d70 0x7f4ef80a5230 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4f04010740 tx=0x7f4f04017000 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:00:03.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.820+0000 7f4ef57fa640 1 -- 172.21.15.1:0/1075777001 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f04024070 con 0x55d89495e060 2024-09-11T22:00:03.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.820+0000 7f4ef57fa640 1 -- 172.21.15.1:0/1075777001 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f0400b9b0 con 0x55d89495e060 2024-09-11T22:00:03.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.820+0000 7f4ef57fa640 1 -- 172.21.15.1:0/1075777001 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f040047a0 con 0x55d89495e060 2024-09-11T22:00:03.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.821+0000 7f4f1744f740 1 -- 172.21.15.1:0/1075777001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d89495e060 msgr2=0x55d89495e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:03.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.821+0000 7f4f1744f740 1 --2- 172.21.15.1:0/1075777001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d89495e060 0x55d89495e440 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4f1006aac0 tx=0x7f4f04012e70 comp rx=0 tx=0).stop 2024-09-11T22:00:03.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.821+0000 7f4f1744f740 1 -- 172.21.15.1:0/1075777001 shutdown_connections 2024-09-11T22:00:03.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.821+0000 7f4f1744f740 1 --2- 172.21.15.1:0/1075777001 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d894969240 0x55d89496b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:03.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.821+0000 7f4f1744f740 1 --2- 172.21.15.1:0/1075777001 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d89495e980 0x55d894968d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:03.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.821+0000 7f4f1744f740 1 --2- 172.21.15.1:0/1075777001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d89495e060 0x55d89495e440 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:03.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.821+0000 7f4f1744f740 1 -- 172.21.15.1:0/1075777001 >> 172.21.15.1:0/1075777001 conn(0x55d89495a5f0 msgr2=0x55d89495a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:00:03.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.821+0000 7f4f1744f740 1 -- 172.21.15.1:0/1075777001 shutdown_connections 2024-09-11T22:00:03.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.821+0000 7f4f1744f740 1 -- 172.21.15.1:0/1075777001 wait complete. 2024-09-11T22:00:03.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.821+0000 7f4f1744f740 1 -- start start 2024-09-11T22:00:03.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.821+0000 7f4f1744f740 1 Processor -- start 2024-09-11T22:00:03.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.822+0000 7f4f1744f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d89495e060 0x55d894966120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:03.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.822+0000 7f4f1744f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d89495e980 0x55d894966660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:03.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.822+0000 7f4f1744f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d894969240 0x55d894966ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:03.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.822+0000 7f4f1744f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55d89496be10 con 0x55d894969240 2024-09-11T22:00:03.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.822+0000 7f4f1744f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55d89484bf10 con 0x55d89495e060 2024-09-11T22:00:03.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.822+0000 7f4f1744f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d89496bf90 con 0x55d89495e980 2024-09-11T22:00:03.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.822+0000 7f4f1744f740 10 client.0 _ll_register_callbacks cb 0x55d89486fdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:00:03.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.822+0000 7f4f0f59e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d894969240 0x55d894966ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:03.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.822+0000 7f4f0f59e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d894969240 0x55d894966ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45720/0 (socket says 172.21.15.1:45720) 2024-09-11T22:00:03.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.822+0000 7f4f0f59e640 1 -- 172.21.15.1:0/4020383754 learned_addr learned my addr 172.21.15.1:0/4020383754 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:00:03.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.822+0000 7f4f0ed9d640 1 --2- 172.21.15.1:0/4020383754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d89495e060 0x55d894966120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:03.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.822+0000 7f4f0e59c640 1 --2- 172.21.15.1:0/4020383754 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d89495e980 0x55d894966660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:03.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.822+0000 7f4f0f59e640 1 -- 172.21.15.1:0/4020383754 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d89495e980 msgr2=0x55d894966660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:00:03.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.822+0000 7f4f0f59e640 1 --2- 172.21.15.1:0/4020383754 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d89495e980 0x55d894966660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:03.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.822+0000 7f4f0f59e640 1 -- 172.21.15.1:0/4020383754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d89495e060 msgr2=0x55d894966120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:00:03.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.822+0000 7f4f0f59e640 1 --2- 172.21.15.1:0/4020383754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d89495e060 0x55d894966120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:03.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.822+0000 7f4f0f59e640 1 -- 172.21.15.1:0/4020383754 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8945d2580 con 0x55d894969240 2024-09-11T22:00:03.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.822+0000 7f4f0ed9d640 1 --2- 172.21.15.1:0/4020383754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d89495e060 0x55d894966120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:00:03.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.822+0000 7f4f0f59e640 1 --2- 172.21.15.1:0/4020383754 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d894969240 0x55d894966ba0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f4f0800c940 tx=0x7f4f08097940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f28ef36043e08656 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:00:03.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.822+0000 7f4ef2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:00:03.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.823+0000 7f4ef2ff5640 1 -- 172.21.15.1:0/4020383754 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0801d070 con 0x55d894969240 2024-09-11T22:00:03.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.823+0000 7f4ef2ff5640 1 -- 172.21.15.1:0/4020383754 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f4f0801dd30 con 0x55d894969240 2024-09-11T22:00:03.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.823+0000 7f4f1744f740 1 -- 172.21.15.1:0/4020383754 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d8948d0b90 con 0x55d894969240 2024-09-11T22:00:03.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.823+0000 7f4ef2ff5640 1 -- 172.21.15.1:0/4020383754 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4f0809b5a0 con 0x55d894969240 2024-09-11T22:00:03.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.823+0000 7f4f1744f740 10 client.16005 fetch_fsmap learned FSMap version 28 2024-09-11T22:00:03.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.823+0000 7f4f1744f740 1 -- 172.21.15.1:0/4020383754 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d89469c960 con 0x55d894969240 2024-09-11T22:00:03.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.823+0000 7f4ef2ff5640 1 -- 172.21.15.1:0/4020383754 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55d8948d0b90 con 0x55d894969240 2024-09-11T22:00:03.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.823+0000 7f4ef2ff5640 1 -- 172.21.15.1:0/4020383754 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4f080cc9a0 con 0x55d894969240 2024-09-11T22:00:03.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.823+0000 7f4f1744f740 10 client.16005 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:00:03.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.823+0000 7f4f1744f740 10 client.16005 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-11T22:00:03.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.823+0000 7f4f1744f740 20 client.16005 populate_metadata read hostname 'smithi001' 2024-09-11T22:00:03.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.823+0000 7f4f1744f740 1 --2- 172.21.15.1:0/4020383754 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55d894978e80 0x55d89497b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:03.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.823+0000 7f4f1744f740 4 client.16005 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:00:03.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.823+0000 7f4f1744f740 1 -- 172.21.15.1:0/4020383754 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d8948cb0a0 con 0x55d894978e80 2024-09-11T22:00:03.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.823+0000 7f4f0ed9d640 1 --2- 172.21.15.1:0/4020383754 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55d894978e80 0x55d89497b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:03.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.824+0000 7f4ef2ff5640 10 client.16005 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:00:03.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.824+0000 7f4f0ed9d640 1 --2- 172.21.15.1:0/4020383754 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55d894978e80 0x55d89497b270 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=646bf19a43d753c4 server_cookie=de6ff8a2189c7b21 in_seq=0 out_seq=0 2024-09-11T22:00:03.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.825+0000 7f4ef2ff5640 1 -- 172.21.15.1:0/4020383754 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d8948cb0a0 con 0x55d894978e80 2024-09-11T22:00:03.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.825+0000 7f4ef2ff5640 10 client.16005 handle_command_reply: tid=0 2024-09-11T22:00:03.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.825+0000 7f4f1744f740 1 client.16005 shutdown 2024-09-11T22:00:03.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.825+0000 7f4f1744f740 1 -- 172.21.15.1:0/4020383754 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d894969240 msgr2=0x55d894966ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:03.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.825+0000 7f4f1744f740 1 --2- 172.21.15.1:0/4020383754 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d894969240 0x55d894966ba0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f4f0800c940 tx=0x7f4f08097940 comp rx=0 tx=0).stop 2024-09-11T22:00:03.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.825+0000 7f4f1744f740 1 -- 172.21.15.1:0/4020383754 shutdown_connections 2024-09-11T22:00:03.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.825+0000 7f4f1744f740 1 --2- 172.21.15.1:0/4020383754 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55d894978e80 0x55d89497b270 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:03.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.825+0000 7f4f1744f740 1 --2- 172.21.15.1:0/4020383754 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d894969240 0x55d894966ba0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:03.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.825+0000 7f4f1744f740 1 --2- 172.21.15.1:0/4020383754 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d89495e980 0x55d894966660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:03.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.825+0000 7f4f1744f740 1 --2- 172.21.15.1:0/4020383754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d89495e060 0x55d894966120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:03.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.825+0000 7f4ef2ff5640 0 client.16005 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:00:03.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.825+0000 7f4f1744f740 1 -- 172.21.15.1:0/4020383754 >> 172.21.15.1:0/4020383754 conn(0x55d89495a5f0 msgr2=0x55d894960320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:00:03.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.825+0000 7f4f1744f740 1 -- 172.21.15.1:0/4020383754 shutdown_connections 2024-09-11T22:00:03.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.825+0000 7f4f1744f740 1 -- 172.21.15.1:0/4020383754 wait complete. 2024-09-11T22:00:03.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.825+0000 7f4f1744f740 20 client.16005 trim_cache size 0 max 16384 2024-09-11T22:00:03.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.842+0000 7f4f1744f740 1 Processor -- start 2024-09-11T22:00:03.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.842+0000 7f4f1744f740 1 -- start start 2024-09-11T22:00:03.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.842+0000 7f4f1744f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d89495e060 0x55d894960b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:03.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.842+0000 7f4f1744f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d89495e980 0x55d8949649f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:03.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.842+0000 7f4f1744f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d894969240 0x55d894959640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:03.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.842+0000 7f4f1744f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4f101ad9c0 con 0x55d89495e060 2024-09-11T22:00:03.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.842+0000 7f4f1744f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4f101af500 con 0x55d89495e980 2024-09-11T22:00:03.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.842+0000 7f4f1744f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4f101ae090 con 0x55d894969240 2024-09-11T22:00:03.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.842+0000 7f4f0ed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d89495e060 0x55d894960b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:03.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.843+0000 7f4f0ed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d89495e060 0x55d894960b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45736/0 (socket says 172.21.15.1:45736) 2024-09-11T22:00:03.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.843+0000 7f4f0ed9d640 1 -- 172.21.15.1:0/1597737779 learned_addr learned my addr 172.21.15.1:0/1597737779 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:00:03.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.843+0000 7f4f0e59c640 1 --2- 172.21.15.1:0/1597737779 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d89495e980 0x55d8949649f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:03.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.843+0000 7f4f0f59e640 1 --2- 172.21.15.1:0/1597737779 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d894969240 0x55d894959640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:03.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.843+0000 7f4f0ed9d640 1 -- 172.21.15.1:0/1597737779 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d894969240 msgr2=0x55d894959640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:00:03.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.843+0000 7f4f0ed9d640 1 --2- 172.21.15.1:0/1597737779 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d894969240 0x55d894959640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:03.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.843+0000 7f4f0ed9d640 1 -- 172.21.15.1:0/1597737779 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d89495e980 msgr2=0x55d8949649f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:00:03.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.843+0000 7f4f0ed9d640 1 --2- 172.21.15.1:0/1597737779 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d89495e980 0x55d8949649f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:03.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.843+0000 7f4f0ed9d640 1 -- 172.21.15.1:0/1597737779 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d89497b8a0 con 0x55d89495e060 2024-09-11T22:00:03.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.843+0000 7f4f0e59c640 1 --2- 172.21.15.1:0/1597737779 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d89495e980 0x55d8949649f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:00:03.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.843+0000 7f4f0ed9d640 1 --2- 172.21.15.1:0/1597737779 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d89495e060 0x55d894960b10 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f4f00006010 tx=0x7f4f0002fdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdbc04ca8ed11a16 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:00:03.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.845+0000 7f4ef4ff9640 1 -- 172.21.15.1:0/1597737779 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f00034040 con 0x55d89495e060 2024-09-11T22:00:03.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.845+0000 7f4ef4ff9640 1 -- 172.21.15.1:0/1597737779 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f00003c30 con 0x55d89495e060 2024-09-11T22:00:03.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.845+0000 7f4ef4ff9640 1 -- 172.21.15.1:0/1597737779 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f00045970 con 0x55d89495e060 2024-09-11T22:00:03.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.846+0000 7f4f1744f740 1 -- 172.21.15.1:0/1597737779 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d89495e060 msgr2=0x55d894960b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:03.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.846+0000 7f4f1744f740 1 --2- 172.21.15.1:0/1597737779 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d89495e060 0x55d894960b10 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f4f00006010 tx=0x7f4f0002fdd0 comp rx=0 tx=0).stop 2024-09-11T22:00:03.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.846+0000 7f4f1744f740 1 -- 172.21.15.1:0/1597737779 shutdown_connections 2024-09-11T22:00:03.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.846+0000 7f4f1744f740 1 --2- 172.21.15.1:0/1597737779 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d894969240 0x55d894959640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:03.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.846+0000 7f4f1744f740 1 --2- 172.21.15.1:0/1597737779 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d89495e980 0x55d8949649f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:03.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.846+0000 7f4f1744f740 1 --2- 172.21.15.1:0/1597737779 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d89495e060 0x55d894960b10 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:03.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.846+0000 7f4f1744f740 1 -- 172.21.15.1:0/1597737779 >> 172.21.15.1:0/1597737779 conn(0x55d89495a5f0 msgr2=0x55d894958f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:00:03.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.846+0000 7f4f1744f740 1 -- 172.21.15.1:0/1597737779 shutdown_connections 2024-09-11T22:00:03.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.846+0000 7f4f1744f740 1 -- 172.21.15.1:0/1597737779 wait complete. 2024-09-11T22:00:03.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.847+0000 7f4f1744f740 1 -- start start 2024-09-11T22:00:03.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.847+0000 7f4f1744f740 1 Processor -- start 2024-09-11T22:00:03.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.847+0000 7f4f1744f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d89495e060 0x55d8949df350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:03.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.847+0000 7f4f1744f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d89495e980 0x55d894971210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:03.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.847+0000 7f4f1744f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d894969240 0x55d894971750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:03.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.847+0000 7f4f1744f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4f101aee30 con 0x55d894969240 2024-09-11T22:00:03.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.847+0000 7f4f1744f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4f101ae760 con 0x55d89495e060 2024-09-11T22:00:03.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.847+0000 7f4f1744f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4f101ab9a0 con 0x55d89495e980 2024-09-11T22:00:03.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.847+0000 7f4f1744f740 10 client.0 _ll_register_callbacks cb 0x7f4f0809b370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:00:03.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.847+0000 7f4f0e59c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d89495e980 0x55d894971210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:03.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.847+0000 7f4f0f59e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d894969240 0x55d894971750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:03.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.847+0000 7f4f0ed9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d89495e060 0x55d8949df350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:03.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.847+0000 7f4f0f59e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d894969240 0x55d894971750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45740/0 (socket says 172.21.15.1:45740) 2024-09-11T22:00:03.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.847+0000 7f4f0e59c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d89495e980 0x55d894971210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:35800/0 (socket says 172.21.15.1:35800) 2024-09-11T22:00:03.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.847+0000 7f4f0f59e640 1 -- 172.21.15.1:0/3759899983 learned_addr learned my addr 172.21.15.1:0/3759899983 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:00:03.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.847+0000 7f4f0f59e640 1 -- 172.21.15.1:0/3759899983 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d89495e980 msgr2=0x55d894971210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:00:03.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.847+0000 7f4f0f59e640 1 --2- 172.21.15.1:0/3759899983 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d89495e980 0x55d894971210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:03.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.847+0000 7f4f0f59e640 1 -- 172.21.15.1:0/3759899983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d89495e060 msgr2=0x55d8949df350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:00:03.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.848+0000 7f4f0f59e640 1 --2- 172.21.15.1:0/3759899983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d89495e060 0x55d8949df350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:03.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.848+0000 7f4f0f59e640 1 -- 172.21.15.1:0/3759899983 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8949dc6a0 con 0x55d894969240 2024-09-11T22:00:03.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.848+0000 7f4f0f59e640 1 --2- 172.21.15.1:0/3759899983 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d894969240 0x55d894971750 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f4f080a3c30 tx=0x7f4f08007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7346381641429c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:00:03.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.848+0000 7f4ef2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:00:03.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.848+0000 7f4ef2ff5640 1 -- 172.21.15.1:0/3759899983 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0801d040 con 0x55d894969240 2024-09-11T22:00:03.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.848+0000 7f4ef2ff5640 1 -- 172.21.15.1:0/3759899983 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f4f080972f0 con 0x55d894969240 2024-09-11T22:00:03.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.848+0000 7f4f1744f740 1 -- 172.21.15.1:0/3759899983 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f00003c30 con 0x55d894969240 2024-09-11T22:00:03.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.848+0000 7f4ef2ff5640 1 -- 172.21.15.1:0/3759899983 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4f0809af00 con 0x55d894969240 2024-09-11T22:00:03.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.848+0000 7f4f1744f740 10 client.16011 fetch_fsmap learned FSMap version 28 2024-09-11T22:00:03.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.848+0000 7f4f1744f740 1 -- 172.21.15.1:0/3759899983 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d8949e8090 con 0x55d894969240 2024-09-11T22:00:03.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.848+0000 7f4ef2ff5640 1 -- 172.21.15.1:0/3759899983 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f00003c30 con 0x55d894969240 2024-09-11T22:00:03.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.848+0000 7f4ef2ff5640 1 -- 172.21.15.1:0/3759899983 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4f080974f0 con 0x55d894969240 2024-09-11T22:00:03.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.848+0000 7f4f1744f740 10 client.16011 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:00:03.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.849+0000 7f4f1744f740 10 client.16011 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-11T22:00:03.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.849+0000 7f4f1744f740 20 client.16011 populate_metadata read hostname 'smithi001' 2024-09-11T22:00:03.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.849+0000 7f4f1744f740 1 --2- 172.21.15.1:0/3759899983 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55d8949e87b0 0x55d8949eabd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:03.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.849+0000 7f4f1744f740 4 client.16011 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "ff1ce167", "release": true, "await": true}] 2024-09-11T22:00:03.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.849+0000 7f4f1744f740 1 -- 172.21.15.1:0/3759899983 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "quiesce db", "set_id": "ff1ce167", "release": true, "await": true}) -- 0x55d8948475a0 con 0x55d8949e87b0 2024-09-11T22:00:03.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.849+0000 7f4f0ed9d640 1 --2- 172.21.15.1:0/3759899983 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55d8949e87b0 0x55d8949eabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:03.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.849+0000 7f4f0ed9d640 1 --2- 172.21.15.1:0/3759899983 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55d8949e87b0 0x55d8949eabd0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=796f076f6ae7f989 server_cookie=8c2d6367beff0dbf in_seq=0 out_seq=0 2024-09-11T22:00:03.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.849+0000 7f4ef2ff5640 10 client.16011 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:00:03.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.852+0000 7f4ef2ff5640 1 -- 172.21.15.1:0/3759899983 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55d8948475a0 con 0x55d8949e87b0 2024-09-11T22:00:03.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.852+0000 7f4ef2ff5640 10 client.16011 handle_command_reply: tid=0 2024-09-11T22:00:03.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.852+0000 7f4f1744f740 1 client.16011 shutdown 2024-09-11T22:00:03.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.852+0000 7f4f1744f740 1 -- 172.21.15.1:0/3759899983 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d894969240 msgr2=0x55d894971750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:03.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.852+0000 7f4f1744f740 1 --2- 172.21.15.1:0/3759899983 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d894969240 0x55d894971750 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f4f080a3c30 tx=0x7f4f08007f70 comp rx=0 tx=0).stop 2024-09-11T22:00:03.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.852+0000 7f4f1744f740 1 -- 172.21.15.1:0/3759899983 shutdown_connections 2024-09-11T22:00:03.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.852+0000 7f4f1744f740 1 --2- 172.21.15.1:0/3759899983 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55d8949e87b0 0x55d8949eabd0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:03.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.852+0000 7f4ef2ff5640 0 client.16011 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:00:03.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.852+0000 7f4f1744f740 1 --2- 172.21.15.1:0/3759899983 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d894969240 0x55d894971750 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:03.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.852+0000 7f4f1744f740 1 --2- 172.21.15.1:0/3759899983 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d89495e980 0x55d894971210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:03.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.852+0000 7f4f1744f740 1 --2- 172.21.15.1:0/3759899983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d89495e060 0x55d8949df350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:03.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.852+0000 7f4f1744f740 1 -- 172.21.15.1:0/3759899983 >> 172.21.15.1:0/3759899983 conn(0x55d89495a5f0 msgr2=0x55d894966900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:00:03.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.852+0000 7f4f1744f740 1 -- 172.21.15.1:0/3759899983 shutdown_connections 2024-09-11T22:00:03.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.853+0000 7f4f1744f740 1 -- 172.21.15.1:0/3759899983 wait complete. 2024-09-11T22:00:03.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.853+0000 7f4f1744f740 20 client.16011 trim_cache size 0 max 16384 2024-09-11T22:00:03.857 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-11T22:00:03.857 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 28, 2024-09-11T22:00:03.857 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24494, 2024-09-11T22:00:03.857 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 8, 2024-09-11T22:00:03.857 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-09-11T22:00:03.857 INFO:teuthology.orchestra.run.smithi001.stdout: "ff1ce167": { 2024-09-11T22:00:03.858 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 8, 2024-09-11T22:00:03.858 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-09-11T22:00:03.858 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-11T22:00:03.858 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RELEASED", 2024-09-11T22:00:03.858 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-11T22:00:03.858 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:00:03.858 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 90.0, 2024-09-11T22:00:03.858 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 133.4, 2024-09-11T22:00:03.858 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-09-11T22:00:03.858 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-09-11T22:00:03.858 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-09-11T22:00:03.858 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-11T22:00:03.858 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RELEASED", 2024-09-11T22:00:03.858 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-11T22:00:03.859 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:00:03.859 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:00:03.859 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:00:03.859 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:00:03.859 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:00:03.859 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-11T22:00:03.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.853+0000 7f4f15076640 1 -- 172.21.15.1:0/2550491007 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f4ef80a2d70 msgr2=0x7f4ef80a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:03.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.853+0000 7f4f15076640 1 --2- 172.21.15.1:0/2550491007 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f4ef80a2d70 0x7f4ef80a5230 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4f04010740 tx=0x7f4f04017000 comp rx=0 tx=0).stop 2024-09-11T22:00:03.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.853+0000 7f4f15076640 1 -- 172.21.15.1:0/2550491007 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f10102ef0 msgr2=0x7f4f101a4480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:03.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.853+0000 7f4f15076640 1 --2- 172.21.15.1:0/2550491007 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f10102ef0 0x7f4f101a4480 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f4f0800c970 tx=0x7f4f0800ce30 comp rx=0 tx=0).stop 2024-09-11T22:00:03.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.853+0000 7f4f15076640 1 -- 172.21.15.1:0/2550491007 shutdown_connections 2024-09-11T22:00:03.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.853+0000 7f4f15076640 1 --2- 172.21.15.1:0/2550491007 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f4ef80a2d70 0x7f4ef80a5230 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:03.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.853+0000 7f4f15076640 1 --2- 172.21.15.1:0/2550491007 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f10102ef0 0x7f4f101a4480 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:03.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.853+0000 7f4f15076640 1 --2- 172.21.15.1:0/2550491007 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4f10102570 0x7f4f10199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:03.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.853+0000 7f4f15076640 1 --2- 172.21.15.1:0/2550491007 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4f10100e20 0x7f4f10199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:03.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.853+0000 7f4f15076640 1 -- 172.21.15.1:0/2550491007 >> 172.21.15.1:0/2550491007 conn(0x7f4f100fc7d0 msgr2=0x7f4f100febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:00:03.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.854+0000 7f4f15076640 1 -- 172.21.15.1:0/2550491007 shutdown_connections 2024-09-11T22:00:03.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:03.854+0000 7f4f15076640 1 -- 172.21.15.1:0/2550491007 wait complete. 2024-09-11T22:00:03.861 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:03.861 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:03.861 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 53 -lt 100 ']' 2024-09-11T22:00:03.861 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 53 + 1 2024-09-11T22:00:03.955 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=54 2024-09-11T22:00:03.955 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T22:00:03.999 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:03.999 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:04.000 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T22:00:04.000 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T22:00:04.000 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T22:00:04.249 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:04.249 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:04.550 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T22:00:04.550 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T22:00:04.563 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:04.563 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:04.564 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 54 -lt 100 ']' 2024-09-11T22:00:04.564 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 54 + 1 2024-09-11T22:00:04.583 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=55 2024-09-11T22:00:04.583 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T22:00:04.598 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:04.598 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:04.599 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T22:00:04.599 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T22:00:04.600 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T22:00:05.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:00:04 smithi001 ceph-mon[30935]: pgmap v90: 97 pgs: 97 active+clean; 2.5 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.5 KiB/s wr, 10 op/s 2024-09-11T22:00:05.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:00:04 smithi053 ceph-mon[34908]: pgmap v90: 97 pgs: 97 active+clean; 2.5 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.5 KiB/s wr, 10 op/s 2024-09-11T22:00:05.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:00:04 smithi161 ceph-mon[35615]: pgmap v90: 97 pgs: 97 active+clean; 2.5 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.5 KiB/s wr, 10 op/s 2024-09-11T22:00:05.505 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:00:05.589 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:05.589 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:05.589 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'ff1ce167', total seconds elapsed: 53.1 2024-09-11T22:00:05.849 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T22:00:05.849 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T22:00:05.855 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:05.855 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:05.856 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 55 -lt 100 ']' 2024-09-11T22:00:05.856 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 55 + 1 2024-09-11T22:00:05.894 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=56 2024-09-11T22:00:05.894 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T22:00:05.907 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:05.907 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:05.909 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T22:00:05.909 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T22:00:05.909 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T22:00:06.705 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:06.705 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:06.991 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T22:00:06.991 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T22:00:07.014 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:07.014 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:07.014 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 56 -lt 100 ']' 2024-09-11T22:00:07.014 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 56 + 1 2024-09-11T22:00:07.052 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=57 2024-09-11T22:00:07.052 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T22:00:07.071 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:07.072 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:07.073 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T22:00:07.073 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T22:00:07.073 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T22:00:07.150 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:00:06 smithi001 ceph-mon[30935]: pgmap v91: 97 pgs: 97 active+clean; 2.5 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 39 KiB/s wr, 34 op/s 2024-09-11T22:00:07.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:00:06 smithi053 ceph-mon[34908]: pgmap v91: 97 pgs: 97 active+clean; 2.5 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 39 KiB/s wr, 34 op/s 2024-09-11T22:00:07.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:00:06 smithi161 ceph-mon[35615]: pgmap v91: 97 pgs: 97 active+clean; 2.5 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 39 KiB/s wr, 34 op/s 2024-09-11T22:00:07.773 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:07.773 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:08.013 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T22:00:08.014 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T22:00:08.022 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:08.022 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:08.022 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 57 -lt 100 ']' 2024-09-11T22:00:08.022 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 57 + 1 2024-09-11T22:00:08.103 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=58 2024-09-11T22:00:08.104 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T22:00:08.140 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:08.140 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:08.141 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T22:00:08.141 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T22:00:08.142 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T22:00:08.764 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:08.764 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:09.046 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T22:00:09.047 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T22:00:09.057 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:09.057 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:09.057 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 58 -lt 100 ']' 2024-09-11T22:00:09.057 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 58 + 1 2024-09-11T22:00:09.120 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=59 2024-09-11T22:00:09.120 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T22:00:09.143 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:00:08 smithi161 ceph-mon[35615]: pgmap v92: 97 pgs: 97 active+clean; 2.5 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 35 KiB/s wr, 29 op/s 2024-09-11T22:00:09.171 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:09.171 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:09.172 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T22:00:09.172 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T22:00:09.173 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T22:00:09.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:00:08 smithi001 ceph-mon[30935]: pgmap v92: 97 pgs: 97 active+clean; 2.5 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 35 KiB/s wr, 29 op/s 2024-09-11T22:00:09.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:00:08 smithi053 ceph-mon[34908]: pgmap v92: 97 pgs: 97 active+clean; 2.5 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 35 KiB/s wr, 29 op/s 2024-09-11T22:00:09.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.863+0000 7f8d415b3640 1 -- 172.21.15.1:0/321698502 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8d3c1049a0 msgr2=0x7f8d3c1089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:09.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.863+0000 7f8d415b3640 1 --2- 172.21.15.1:0/321698502 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8d3c1049a0 0x7f8d3c1089f0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f8d30009900 tx=0x7f8d3002a410 comp rx=0 tx=0).stop 2024-09-11T22:00:09.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.863+0000 7f8d415b3640 1 -- 172.21.15.1:0/321698502 shutdown_connections 2024-09-11T22:00:09.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.863+0000 7f8d415b3640 1 --2- 172.21.15.1:0/321698502 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d3c109120 0x7f8d3c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:09.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.863+0000 7f8d415b3640 1 --2- 172.21.15.1:0/321698502 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8d3c1049a0 0x7f8d3c1089f0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:09.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.863+0000 7f8d415b3640 1 --2- 172.21.15.1:0/321698502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d3c103ff0 0x7f8d3c1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:09.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.863+0000 7f8d415b3640 1 -- 172.21.15.1:0/321698502 >> 172.21.15.1:0/321698502 conn(0x7f8d3c0ffab0 msgr2=0x7f8d3c101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:00:09.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.863+0000 7f8d415b3640 1 -- 172.21.15.1:0/321698502 shutdown_connections 2024-09-11T22:00:09.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.863+0000 7f8d415b3640 1 -- 172.21.15.1:0/321698502 wait complete. 2024-09-11T22:00:09.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.864+0000 7f8d415b3640 1 Processor -- start 2024-09-11T22:00:09.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.864+0000 7f8d415b3640 1 -- start start 2024-09-11T22:00:09.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.865+0000 7f8d415b3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8d3c103ff0 0x7f8d3c19a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:09.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.865+0000 7f8d415b3640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d3c1049a0 0x7f8d3c19ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:09.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.865+0000 7f8d415b3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d3c109120 0x7f8d3c19eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:09.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.865+0000 7f8d415b3640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8d3c114140 con 0x7f8d3c103ff0 2024-09-11T22:00:09.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.865+0000 7f8d415b3640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8d3c113fc0 con 0x7f8d3c109120 2024-09-11T22:00:09.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.865+0000 7f8d415b3640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8d3c1142c0 con 0x7f8d3c1049a0 2024-09-11T22:00:09.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.865+0000 7f8d3affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8d3c103ff0 0x7f8d3c19a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:09.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.865+0000 7f8d3affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8d3c103ff0 0x7f8d3c19a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48732/0 (socket says 172.21.15.1:48732) 2024-09-11T22:00:09.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.865+0000 7f8d3a7fc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d3c1049a0 0x7f8d3c19ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:09.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.865+0000 7f8d3affd640 1 -- 172.21.15.1:0/3999690070 learned_addr learned my addr 172.21.15.1:0/3999690070 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:00:09.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.865+0000 7f8d3b7fe640 1 --2- 172.21.15.1:0/3999690070 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d3c109120 0x7f8d3c19eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:09.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.865+0000 7f8d3affd640 1 -- 172.21.15.1:0/3999690070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d3c1049a0 msgr2=0x7f8d3c19ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:09.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.865+0000 7f8d3affd640 1 --2- 172.21.15.1:0/3999690070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d3c1049a0 0x7f8d3c19ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:09.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.865+0000 7f8d3affd640 1 -- 172.21.15.1:0/3999690070 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d3c109120 msgr2=0x7f8d3c19eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:09.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.865+0000 7f8d3affd640 1 --2- 172.21.15.1:0/3999690070 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d3c109120 0x7f8d3c19eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:09.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.865+0000 7f8d3affd640 1 -- 172.21.15.1:0/3999690070 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d3c19f5d0 con 0x7f8d3c103ff0 2024-09-11T22:00:09.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.865+0000 7f8d3affd640 1 --2- 172.21.15.1:0/3999690070 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8d3c103ff0 0x7f8d3c19a620 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c00ebc0 tx=0x7f8d2c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:00:09.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.866+0000 7f8d23fff640 1 -- 172.21.15.1:0/3999690070 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d2c010630 con 0x7f8d3c103ff0 2024-09-11T22:00:09.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.866+0000 7f8d415b3640 1 -- 172.21.15.1:0/3999690070 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d3c19f8c0 con 0x7f8d3c103ff0 2024-09-11T22:00:09.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.866+0000 7f8d23fff640 1 -- 172.21.15.1:0/3999690070 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d2c010cb0 con 0x7f8d3c103ff0 2024-09-11T22:00:09.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.866+0000 7f8d23fff640 1 -- 172.21.15.1:0/3999690070 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d2c018bd0 con 0x7f8d3c103ff0 2024-09-11T22:00:09.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.866+0000 7f8d415b3640 1 -- 172.21.15.1:0/3999690070 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d3c1a71a0 con 0x7f8d3c103ff0 2024-09-11T22:00:09.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.868+0000 7f8d23fff640 1 -- 172.21.15.1:0/3999690070 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f8d2c0107d0 con 0x7f8d3c103ff0 2024-09-11T22:00:09.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.869+0000 7f8d23fff640 1 --2- 172.21.15.1:0/3999690070 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f8d240a2cf0 0x7f8d240a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:09.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.869+0000 7f8d3a7fc640 1 --2- 172.21.15.1:0/3999690070 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f8d240a2cf0 0x7f8d240a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:09.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.869+0000 7f8d23fff640 1 -- 172.21.15.1:0/3999690070 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f8d2c0cfa70 con 0x7f8d3c103ff0 2024-09-11T22:00:09.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.869+0000 7f8d3a7fc640 1 --2- 172.21.15.1:0/3999690070 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f8d240a2cf0 0x7f8d240a51b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8d30009230 tx=0x7f8d300058d0 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:00:09.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.981+0000 7f8d4398c740 1 Processor -- start 2024-09-11T22:00:09.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.981+0000 7f8d4398c740 1 -- start start 2024-09-11T22:00:09.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.981+0000 7f8d4398c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560dfaa57b40 0x560dfaa56090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:09.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.981+0000 7f8d4398c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560dfaa565d0 0x560dfaa56a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:09.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.981+0000 7f8d4398c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dfaa61370 0x560dfaa63760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:09.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.981+0000 7f8d4398c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x560dfa9bc2f0 con 0x560dfaa565d0 2024-09-11T22:00:09.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.981+0000 7f8d4398c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x560dfa963cf0 con 0x560dfaa57b40 2024-09-11T22:00:09.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.981+0000 7f8d4398c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x560dfa9b82b0 con 0x560dfaa61370 2024-09-11T22:00:09.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.981+0000 7f8d3a7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560dfaa565d0 0x560dfaa56a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:09.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.981+0000 7f8d3a7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560dfaa565d0 0x560dfaa56a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48742/0 (socket says 172.21.15.1:48742) 2024-09-11T22:00:09.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.981+0000 7f8d3a7fc640 1 -- 172.21.15.1:0/2009543376 learned_addr learned my addr 172.21.15.1:0/2009543376 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:00:09.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.981+0000 7f8d3affd640 1 --2- 172.21.15.1:0/2009543376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560dfaa57b40 0x560dfaa56090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:09.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.981+0000 7f8d3b7fe640 1 --2- 172.21.15.1:0/2009543376 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dfaa61370 0x560dfaa63760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:09.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.982+0000 7f8d3a7fc640 1 -- 172.21.15.1:0/2009543376 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dfaa61370 msgr2=0x560dfaa63760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:00:09.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.982+0000 7f8d3a7fc640 1 --2- 172.21.15.1:0/2009543376 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dfaa61370 0x560dfaa63760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:09.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.982+0000 7f8d3a7fc640 1 -- 172.21.15.1:0/2009543376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560dfaa57b40 msgr2=0x560dfaa56090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:00:09.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.982+0000 7f8d3a7fc640 1 --2- 172.21.15.1:0/2009543376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560dfaa57b40 0x560dfaa56090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:09.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.982+0000 7f8d3a7fc640 1 -- 172.21.15.1:0/2009543376 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560dfa9483a0 con 0x560dfaa565d0 2024-09-11T22:00:09.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.982+0000 7f8d3affd640 1 --2- 172.21.15.1:0/2009543376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560dfaa57b40 0x560dfaa56090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:00:09.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.982+0000 7f8d3a7fc640 1 --2- 172.21.15.1:0/2009543376 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560dfaa565d0 0x560dfaa56a10 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f8d30009900 tx=0x7f8d3003d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=922d8d53107eb47a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:00:09.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.982+0000 7f8d217fa640 1 -- 172.21.15.1:0/2009543376 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d30035040 con 0x560dfaa565d0 2024-09-11T22:00:09.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.982+0000 7f8d217fa640 1 -- 172.21.15.1:0/2009543376 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d3004e070 con 0x560dfaa565d0 2024-09-11T22:00:09.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.982+0000 7f8d217fa640 1 -- 172.21.15.1:0/2009543376 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d300496e0 con 0x560dfaa565d0 2024-09-11T22:00:09.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.983+0000 7f8d4398c740 1 -- 172.21.15.1:0/2009543376 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560dfaa565d0 msgr2=0x560dfaa56a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:09.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.983+0000 7f8d4398c740 1 --2- 172.21.15.1:0/2009543376 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560dfaa565d0 0x560dfaa56a10 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f8d30009900 tx=0x7f8d3003d000 comp rx=0 tx=0).stop 2024-09-11T22:00:09.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.983+0000 7f8d4398c740 1 -- 172.21.15.1:0/2009543376 shutdown_connections 2024-09-11T22:00:09.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.983+0000 7f8d4398c740 1 --2- 172.21.15.1:0/2009543376 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dfaa61370 0x560dfaa63760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:09.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.983+0000 7f8d4398c740 1 --2- 172.21.15.1:0/2009543376 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560dfaa565d0 0x560dfaa56a10 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:09.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.983+0000 7f8d4398c740 1 --2- 172.21.15.1:0/2009543376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560dfaa57b40 0x560dfaa56090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:09.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.983+0000 7f8d4398c740 1 -- 172.21.15.1:0/2009543376 >> 172.21.15.1:0/2009543376 conn(0x560dfaa1f310 msgr2=0x560dfaa52c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:00:09.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.983+0000 7f8d4398c740 1 -- 172.21.15.1:0/2009543376 shutdown_connections 2024-09-11T22:00:09.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.983+0000 7f8d4398c740 1 -- 172.21.15.1:0/2009543376 wait complete. 2024-09-11T22:00:09.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.983+0000 7f8d4398c740 1 -- start start 2024-09-11T22:00:09.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.984+0000 7f8d4398c740 1 Processor -- start 2024-09-11T22:00:09.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.984+0000 7f8d4398c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dfaa565d0 0x560dfaa60d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:09.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.984+0000 7f8d4398c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560dfaa57b40 0x560dfaa5e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:09.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.984+0000 7f8d4398c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560dfaa61370 0x560dfaa5e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:09.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.984+0000 7f8d4398c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x560dfaa64010 con 0x560dfaa61370 2024-09-11T22:00:09.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.984+0000 7f8d4398c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x560dfa977060 con 0x560dfaa57b40 2024-09-11T22:00:09.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.984+0000 7f8d4398c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x560dfaa64190 con 0x560dfaa565d0 2024-09-11T22:00:09.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.984+0000 7f8d4398c740 10 client.0 _ll_register_callbacks cb 0x560dfa6ca8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:00:09.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.984+0000 7f8d3b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560dfaa61370 0x560dfaa5e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:09.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.984+0000 7f8d3b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560dfaa61370 0x560dfaa5e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48746/0 (socket says 172.21.15.1:48746) 2024-09-11T22:00:09.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.984+0000 7f8d3b7fe640 1 -- 172.21.15.1:0/1617688636 learned_addr learned my addr 172.21.15.1:0/1617688636 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:00:09.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.984+0000 7f8d3a7fc640 1 --2- 172.21.15.1:0/1617688636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560dfaa57b40 0x560dfaa5e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:09.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.984+0000 7f8d3affd640 1 --2- 172.21.15.1:0/1617688636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dfaa565d0 0x560dfaa60d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:09.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.984+0000 7f8d3b7fe640 1 -- 172.21.15.1:0/1617688636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dfaa565d0 msgr2=0x560dfaa60d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:00:09.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.984+0000 7f8d3b7fe640 1 --2- 172.21.15.1:0/1617688636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dfaa565d0 0x560dfaa60d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:09.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.984+0000 7f8d3b7fe640 1 -- 172.21.15.1:0/1617688636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560dfaa57b40 msgr2=0x560dfaa5e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:00:09.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.984+0000 7f8d3b7fe640 1 --2- 172.21.15.1:0/1617688636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560dfaa57b40 0x560dfaa5e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:09.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.984+0000 7f8d3b7fe640 1 -- 172.21.15.1:0/1617688636 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560dfa72d3c0 con 0x560dfaa61370 2024-09-11T22:00:09.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.985+0000 7f8d3b7fe640 1 --2- 172.21.15.1:0/1617688636 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560dfaa61370 0x560dfaa5e650 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f8d3400b920 tx=0x7f8d3400bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6a701f9024c69a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:00:09.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.985+0000 7f8d1eff5640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:00:09.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.985+0000 7f8d1eff5640 1 -- 172.21.15.1:0/1617688636 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d34003e20 con 0x560dfaa61370 2024-09-11T22:00:09.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.985+0000 7f8d1eff5640 1 -- 172.21.15.1:0/1617688636 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f8d34005020 con 0x560dfaa61370 2024-09-11T22:00:09.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.985+0000 7f8d4398c740 1 -- 172.21.15.1:0/1617688636 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d3004e070 con 0x560dfaa61370 2024-09-11T22:00:09.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.985+0000 7f8d1eff5640 1 -- 172.21.15.1:0/1617688636 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8d34005220 con 0x560dfaa61370 2024-09-11T22:00:09.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.985+0000 7f8d4398c740 10 client.16023 fetch_fsmap learned FSMap version 28 2024-09-11T22:00:09.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.985+0000 7f8d4398c740 1 -- 172.21.15.1:0/1617688636 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560dfaa6fb40 con 0x560dfaa61370 2024-09-11T22:00:09.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.985+0000 7f8d1eff5640 1 -- 172.21.15.1:0/1617688636 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d3004e070 con 0x560dfaa61370 2024-09-11T22:00:09.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.986+0000 7f8d1eff5640 1 -- 172.21.15.1:0/1617688636 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8d3401b520 con 0x560dfaa61370 2024-09-11T22:00:09.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.986+0000 7f8d4398c740 10 client.16023 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:00:09.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.986+0000 7f8d4398c740 10 client.16023 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-11T22:00:09.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.986+0000 7f8d4398c740 20 client.16023 populate_metadata read hostname 'smithi001' 2024-09-11T22:00:09.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.986+0000 7f8d4398c740 1 --2- 172.21.15.1:0/1617688636 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x560dfaa71000 0x560dfaa733f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:09.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.986+0000 7f8d4398c740 4 client.16023 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:00:09.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.986+0000 7f8d4398c740 1 -- 172.21.15.1:0/1617688636 --> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560dfa9c9860 con 0x560dfaa71000 2024-09-11T22:00:09.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.986+0000 7f8d3affd640 1 --2- 172.21.15.1:0/1617688636 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x560dfaa71000 0x560dfaa733f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:09.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.987+0000 7f8d3affd640 1 --2- 172.21.15.1:0/1617688636 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x560dfaa71000 0x560dfaa733f0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cfa015f2ef8516b4 server_cookie=5772b398791053e1 in_seq=0 out_seq=0 2024-09-11T22:00:09.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.987+0000 7f8d1eff5640 10 client.16023 ms_handle_connect on v2:172.21.15.53:6836/3457131830 2024-09-11T22:00:09.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.987+0000 7f8d1eff5640 1 -- 172.21.15.1:0/1617688636 <== mds.2 v2:172.21.15.53:6836/3457131830 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560dfa9c9860 con 0x560dfaa71000 2024-09-11T22:00:09.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.987+0000 7f8d1eff5640 10 client.16023 handle_command_reply: tid=0 2024-09-11T22:00:09.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.987+0000 7f8d4398c740 1 client.16023 shutdown 2024-09-11T22:00:09.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.987+0000 7f8d4398c740 1 -- 172.21.15.1:0/1617688636 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560dfaa61370 msgr2=0x560dfaa5e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:09.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.987+0000 7f8d4398c740 1 --2- 172.21.15.1:0/1617688636 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560dfaa61370 0x560dfaa5e650 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f8d3400b920 tx=0x7f8d3400bde0 comp rx=0 tx=0).stop 2024-09-11T22:00:09.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.988+0000 7f8d4398c740 1 -- 172.21.15.1:0/1617688636 shutdown_connections 2024-09-11T22:00:09.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.988+0000 7f8d4398c740 1 --2- 172.21.15.1:0/1617688636 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x560dfaa71000 0x560dfaa733f0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:09.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.988+0000 7f8d4398c740 1 --2- 172.21.15.1:0/1617688636 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560dfaa61370 0x560dfaa5e650 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:09.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.988+0000 7f8d4398c740 1 --2- 172.21.15.1:0/1617688636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560dfaa57b40 0x560dfaa5e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:09.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.988+0000 7f8d4398c740 1 --2- 172.21.15.1:0/1617688636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dfaa565d0 0x560dfaa60d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:09.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.988+0000 7f8d1eff5640 0 client.16023 ms_handle_reset on v2:172.21.15.53:6836/3457131830 2024-09-11T22:00:09.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.988+0000 7f8d4398c740 1 -- 172.21.15.1:0/1617688636 >> 172.21.15.1:0/1617688636 conn(0x560dfaa1f310 msgr2=0x560dfaa58930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:00:09.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.988+0000 7f8d4398c740 1 -- 172.21.15.1:0/1617688636 shutdown_connections 2024-09-11T22:00:09.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.988+0000 7f8d4398c740 1 -- 172.21.15.1:0/1617688636 wait complete. 2024-09-11T22:00:09.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:09.988+0000 7f8d4398c740 20 client.16023 trim_cache size 0 max 16384 2024-09-11T22:00:09.998 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:09.998 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:10.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.001+0000 7f8d4398c740 1 Processor -- start 2024-09-11T22:00:10.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.001+0000 7f8d4398c740 1 -- start start 2024-09-11T22:00:10.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.001+0000 7f8d4398c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dfaa565d0 0x560dfaa6d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:10.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.001+0000 7f8d4398c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560dfaa57b40 0x560dfaa78cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:10.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.001+0000 7f8d4398c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560dfaa61370 0x560dfaa79200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:10.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.001+0000 7f8d4398c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8d3c1afc30 con 0x560dfaa61370 2024-09-11T22:00:10.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.001+0000 7f8d4398c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8d3c1b1770 con 0x560dfaa57b40 2024-09-11T22:00:10.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.001+0000 7f8d4398c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8d3c1b0300 con 0x560dfaa565d0 2024-09-11T22:00:10.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.002+0000 7f8d3b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560dfaa61370 0x560dfaa79200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:10.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.002+0000 7f8d3b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560dfaa61370 0x560dfaa79200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48756/0 (socket says 172.21.15.1:48756) 2024-09-11T22:00:10.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.002+0000 7f8d3b7fe640 1 -- 172.21.15.1:0/2610429433 learned_addr learned my addr 172.21.15.1:0/2610429433 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:00:10.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.002+0000 7f8d3affd640 1 --2- 172.21.15.1:0/2610429433 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dfaa565d0 0x560dfaa6d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:10.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.002+0000 7f8d3a7fc640 1 --2- 172.21.15.1:0/2610429433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560dfaa57b40 0x560dfaa78cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:10.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.002+0000 7f8d3b7fe640 1 -- 172.21.15.1:0/2610429433 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dfaa565d0 msgr2=0x560dfaa6d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:00:10.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.002+0000 7f8d3b7fe640 1 --2- 172.21.15.1:0/2610429433 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dfaa565d0 0x560dfaa6d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:10.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.002+0000 7f8d3b7fe640 1 -- 172.21.15.1:0/2610429433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560dfaa57b40 msgr2=0x560dfaa78cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:00:10.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.002+0000 7f8d3b7fe640 1 --2- 172.21.15.1:0/2610429433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560dfaa57b40 0x560dfaa78cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:10.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.002+0000 7f8d3b7fe640 1 -- 172.21.15.1:0/2610429433 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560dfaa5c590 con 0x560dfaa61370 2024-09-11T22:00:10.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.002+0000 7f8d3b7fe640 1 --2- 172.21.15.1:0/2610429433 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560dfaa61370 0x560dfaa79200 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f8d3401b990 tx=0x7f8d3400bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed1fe1905ba7eaa3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:00:10.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.003+0000 7f8d20ff9640 1 -- 172.21.15.1:0/2610429433 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d34003ea0 con 0x560dfaa61370 2024-09-11T22:00:10.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.003+0000 7f8d20ff9640 1 -- 172.21.15.1:0/2610429433 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d3401b470 con 0x560dfaa61370 2024-09-11T22:00:10.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.003+0000 7f8d20ff9640 1 -- 172.21.15.1:0/2610429433 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d34011920 con 0x560dfaa61370 2024-09-11T22:00:10.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.003+0000 7f8d4398c740 1 -- 172.21.15.1:0/2610429433 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560dfaa61370 msgr2=0x560dfaa79200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:10.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.003+0000 7f8d4398c740 1 --2- 172.21.15.1:0/2610429433 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560dfaa61370 0x560dfaa79200 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f8d3401b990 tx=0x7f8d3400bf90 comp rx=0 tx=0).stop 2024-09-11T22:00:10.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.003+0000 7f8d4398c740 1 -- 172.21.15.1:0/2610429433 shutdown_connections 2024-09-11T22:00:10.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.003+0000 7f8d4398c740 1 --2- 172.21.15.1:0/2610429433 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560dfaa61370 0x560dfaa79200 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:10.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.003+0000 7f8d4398c740 1 --2- 172.21.15.1:0/2610429433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560dfaa57b40 0x560dfaa78cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:10.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.003+0000 7f8d4398c740 1 --2- 172.21.15.1:0/2610429433 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dfaa565d0 0x560dfaa6d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:10.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.003+0000 7f8d4398c740 1 -- 172.21.15.1:0/2610429433 >> 172.21.15.1:0/2610429433 conn(0x560dfaa1f310 msgr2=0x560dfaa600c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:00:10.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.003+0000 7f8d4398c740 1 -- 172.21.15.1:0/2610429433 shutdown_connections 2024-09-11T22:00:10.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.003+0000 7f8d4398c740 1 -- 172.21.15.1:0/2610429433 wait complete. 2024-09-11T22:00:10.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.004+0000 7f8d4398c740 1 -- start start 2024-09-11T22:00:10.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.004+0000 7f8d4398c740 1 Processor -- start 2024-09-11T22:00:10.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.004+0000 7f8d4398c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560dfaa565d0 0x560dfaa65230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:10.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.004+0000 7f8d4398c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dfaa57b40 0x560dfaa65770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:10.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.004+0000 7f8d4398c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560dfaa61370 0x560dfaa65cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:10.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.004+0000 7f8d4398c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8d3c1b10a0 con 0x560dfaa61370 2024-09-11T22:00:10.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.004+0000 7f8d4398c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8d3c1b09d0 con 0x560dfaa565d0 2024-09-11T22:00:10.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.004+0000 7f8d4398c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8d3c1adc10 con 0x560dfaa57b40 2024-09-11T22:00:10.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.004+0000 7f8d4398c740 10 client.0 _ll_register_callbacks cb 0x7f8d340041f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:00:10.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.004+0000 7f8d3b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560dfaa61370 0x560dfaa65cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:10.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.004+0000 7f8d3b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560dfaa61370 0x560dfaa65cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48766/0 (socket says 172.21.15.1:48766) 2024-09-11T22:00:10.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.004+0000 7f8d3b7fe640 1 -- 172.21.15.1:0/3221495303 learned_addr learned my addr 172.21.15.1:0/3221495303 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:00:10.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.004+0000 7f8d3a7fc640 1 --2- 172.21.15.1:0/3221495303 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dfaa57b40 0x560dfaa65770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:10.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.005+0000 7f8d3affd640 1 --2- 172.21.15.1:0/3221495303 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560dfaa565d0 0x560dfaa65230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:10.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.005+0000 7f8d3b7fe640 1 -- 172.21.15.1:0/3221495303 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dfaa57b40 msgr2=0x560dfaa65770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:00:10.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.005+0000 7f8d3b7fe640 1 --2- 172.21.15.1:0/3221495303 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dfaa57b40 0x560dfaa65770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:10.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.005+0000 7f8d3b7fe640 1 -- 172.21.15.1:0/3221495303 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560dfaa565d0 msgr2=0x560dfaa65230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:00:10.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.005+0000 7f8d3b7fe640 1 --2- 172.21.15.1:0/3221495303 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560dfaa565d0 0x560dfaa65230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:10.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.005+0000 7f8d3b7fe640 1 -- 172.21.15.1:0/3221495303 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560dfaadad10 con 0x560dfaa61370 2024-09-11T22:00:10.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.005+0000 7f8d3affd640 1 --2- 172.21.15.1:0/3221495303 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560dfaa565d0 0x560dfaa65230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:00:10.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.005+0000 7f8d3b7fe640 1 --2- 172.21.15.1:0/3221495303 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560dfaa61370 0x560dfaa65cb0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f8d3400b2e0 tx=0x7f8d3400b310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dd8aea6a203688c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:00:10.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.005+0000 7f8d1eff5640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:00:10.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.005+0000 7f8d1eff5640 1 -- 172.21.15.1:0/3221495303 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d3401bd10 con 0x560dfaa61370 2024-09-11T22:00:10.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.005+0000 7f8d1eff5640 1 -- 172.21.15.1:0/3221495303 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f8d34017040 con 0x560dfaa61370 2024-09-11T22:00:10.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.005+0000 7f8d4398c740 1 -- 172.21.15.1:0/3221495303 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d3401b470 con 0x560dfaa61370 2024-09-11T22:00:10.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.005+0000 7f8d1eff5640 1 -- 172.21.15.1:0/3221495303 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8d34011440 con 0x560dfaa61370 2024-09-11T22:00:10.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.005+0000 7f8d4398c740 10 client.16029 fetch_fsmap learned FSMap version 28 2024-09-11T22:00:10.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.005+0000 7f8d4398c740 1 -- 172.21.15.1:0/3221495303 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560dfaadbfd0 con 0x560dfaa61370 2024-09-11T22:00:10.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.006+0000 7f8d1eff5640 1 -- 172.21.15.1:0/3221495303 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d3401b470 con 0x560dfaa61370 2024-09-11T22:00:10.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.006+0000 7f8d1eff5640 1 -- 172.21.15.1:0/3221495303 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8d3402a070 con 0x560dfaa61370 2024-09-11T22:00:10.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.006+0000 7f8d4398c740 10 client.16029 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:00:10.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.006+0000 7f8d4398c740 10 client.16029 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-11T22:00:10.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.006+0000 7f8d4398c740 20 client.16029 populate_metadata read hostname 'smithi001' 2024-09-11T22:00:10.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.006+0000 7f8d4398c740 1 --2- 172.21.15.1:0/3221495303 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x560dfaadc620 0x560dfaadea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:10.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.006+0000 7f8d4398c740 4 client.16029 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:00:10.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.006+0000 7f8d4398c740 1 -- 172.21.15.1:0/3221495303 --> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560dfa9c3d70 con 0x560dfaadc620 2024-09-11T22:00:10.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.006+0000 7f8d3affd640 1 --2- 172.21.15.1:0/3221495303 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x560dfaadc620 0x560dfaadea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:10.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.007+0000 7f8d3affd640 1 --2- 172.21.15.1:0/3221495303 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x560dfaadc620 0x560dfaadea10 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2566976f4f263cbf server_cookie=866b0eb9a37d838b in_seq=0 out_seq=0 2024-09-11T22:00:10.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.007+0000 7f8d1eff5640 10 client.16029 ms_handle_connect on v2:172.21.15.53:6836/3457131830 2024-09-11T22:00:10.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.007+0000 7f8d1eff5640 1 -- 172.21.15.1:0/3221495303 <== mds.2 v2:172.21.15.53:6836/3457131830 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560dfa9c3d70 con 0x560dfaadc620 2024-09-11T22:00:10.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.007+0000 7f8d1eff5640 10 client.16029 handle_command_reply: tid=0 2024-09-11T22:00:10.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.007+0000 7f8d4398c740 1 client.16029 shutdown 2024-09-11T22:00:10.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.007+0000 7f8d4398c740 1 -- 172.21.15.1:0/3221495303 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560dfaa61370 msgr2=0x560dfaa65cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:10.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.007+0000 7f8d4398c740 1 --2- 172.21.15.1:0/3221495303 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560dfaa61370 0x560dfaa65cb0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f8d3400b2e0 tx=0x7f8d3400b310 comp rx=0 tx=0).stop 2024-09-11T22:00:10.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.007+0000 7f8d4398c740 1 -- 172.21.15.1:0/3221495303 shutdown_connections 2024-09-11T22:00:10.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.007+0000 7f8d4398c740 1 --2- 172.21.15.1:0/3221495303 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x560dfaadc620 0x560dfaadea10 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:10.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.007+0000 7f8d4398c740 1 --2- 172.21.15.1:0/3221495303 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560dfaa61370 0x560dfaa65cb0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:10.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.007+0000 7f8d4398c740 1 --2- 172.21.15.1:0/3221495303 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560dfaa57b40 0x560dfaa65770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:10.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.007+0000 7f8d4398c740 1 --2- 172.21.15.1:0/3221495303 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560dfaa565d0 0x560dfaa65230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:10.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.007+0000 7f8d1eff5640 0 client.16029 ms_handle_reset on v2:172.21.15.53:6836/3457131830 2024-09-11T22:00:10.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.007+0000 7f8d4398c740 1 -- 172.21.15.1:0/3221495303 >> 172.21.15.1:0/3221495303 conn(0x560dfaa1f310 msgr2=0x560dfaa5f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:00:10.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.007+0000 7f8d4398c740 1 -- 172.21.15.1:0/3221495303 shutdown_connections 2024-09-11T22:00:10.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.007+0000 7f8d4398c740 1 -- 172.21.15.1:0/3221495303 wait complete. 2024-09-11T22:00:10.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.007+0000 7f8d4398c740 20 client.16029 trim_cache size 0 max 16384 2024-09-11T22:00:10.014 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-11T22:00:10.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.008+0000 7f8d415b3640 1 -- 172.21.15.1:0/3999690070 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f8d240a2cf0 msgr2=0x7f8d240a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:10.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.008+0000 7f8d415b3640 1 --2- 172.21.15.1:0/3999690070 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f8d240a2cf0 0x7f8d240a51b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8d30009230 tx=0x7f8d300058d0 comp rx=0 tx=0).stop 2024-09-11T22:00:10.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.008+0000 7f8d415b3640 1 -- 172.21.15.1:0/3999690070 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8d3c103ff0 msgr2=0x7f8d3c19a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:10.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.008+0000 7f8d415b3640 1 --2- 172.21.15.1:0/3999690070 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8d3c103ff0 0x7f8d3c19a620 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c00ebc0 tx=0x7f8d2c00c6a0 comp rx=0 tx=0).stop 2024-09-11T22:00:10.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.008+0000 7f8d415b3640 1 -- 172.21.15.1:0/3999690070 shutdown_connections 2024-09-11T22:00:10.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.008+0000 7f8d415b3640 1 --2- 172.21.15.1:0/3999690070 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f8d240a2cf0 0x7f8d240a51b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:10.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.008+0000 7f8d415b3640 1 --2- 172.21.15.1:0/3999690070 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8d3c109120 0x7f8d3c19eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:10.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.008+0000 7f8d415b3640 1 --2- 172.21.15.1:0/3999690070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d3c1049a0 0x7f8d3c19ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:10.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.008+0000 7f8d415b3640 1 --2- 172.21.15.1:0/3999690070 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8d3c103ff0 0x7f8d3c19a620 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:10.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.008+0000 7f8d415b3640 1 -- 172.21.15.1:0/3999690070 >> 172.21.15.1:0/3999690070 conn(0x7f8d3c0ffab0 msgr2=0x7f8d3c100d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:00:10.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.008+0000 7f8d415b3640 1 -- 172.21.15.1:0/3999690070 shutdown_connections 2024-09-11T22:00:10.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:10.008+0000 7f8d415b3640 1 -- 172.21.15.1:0/3999690070 wait complete. 2024-09-11T22:00:10.323 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T22:00:10.323 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T22:00:10.344 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:10.345 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:10.345 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 59 -lt 100 ']' 2024-09-11T22:00:10.345 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 59 + 1 2024-09-11T22:00:10.458 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=60 2024-09-11T22:00:10.459 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T22:00:10.515 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:10.516 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:10.517 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T22:00:10.517 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T22:00:10.517 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T22:00:11.096 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:11.096 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:11.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:00:10 smithi001 ceph-mon[30935]: pgmap v93: 97 pgs: 97 active+clean; 2.6 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 36 KiB/s wr, 34 op/s 2024-09-11T22:00:11.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:00:10 smithi053 ceph-mon[34908]: pgmap v93: 97 pgs: 97 active+clean; 2.6 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 36 KiB/s wr, 34 op/s 2024-09-11T22:00:11.307 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T22:00:11.307 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T22:00:11.330 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:11.330 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:11.330 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 60 -lt 100 ']' 2024-09-11T22:00:11.330 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 60 + 1 2024-09-11T22:00:11.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:00:10 smithi161 ceph-mon[35615]: pgmap v93: 97 pgs: 97 active+clean; 2.6 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 36 KiB/s wr, 34 op/s 2024-09-11T22:00:11.378 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=61 2024-09-11T22:00:11.379 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T22:00:11.404 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:11.404 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:11.405 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T22:00:11.405 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T22:00:11.406 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T22:00:12.436 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:12.437 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:12.454 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T22:00:12.454 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T22:00:12.457 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:12.457 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:12.458 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 61 -lt 100 ']' 2024-09-11T22:00:12.458 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 61 + 1 2024-09-11T22:00:12.461 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=62 2024-09-11T22:00:12.462 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T22:00:12.468 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:12.468 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:12.469 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T22:00:12.469 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T22:00:12.469 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T22:00:12.480 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T22:00:12.706 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:12.706 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:12.725 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T22:00:12.725 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T22:00:12.729 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:12.729 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:12.729 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 62 -lt 100 ']' 2024-09-11T22:00:12.730 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 62 + 1 2024-09-11T22:00:12.736 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=63 2024-09-11T22:00:12.736 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T22:00:12.739 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:12.739 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:12.740 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T22:00:12.741 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T22:00:12.741 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T22:00:12.818 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:12.818 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:12.833 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T22:00:12.834 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T22:00:12.836 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:12.836 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:12.836 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 63 -lt 100 ']' 2024-09-11T22:00:12.837 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 63 + 1 2024-09-11T22:00:12.847 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=64 2024-09-11T22:00:12.847 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T22:00:12.853 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:12.854 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:12.855 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T22:00:12.855 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T22:00:12.855 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T22:00:12.895 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:00:12 smithi001 ceph-mon[30935]: pgmap v94: 97 pgs: 97 active+clean; 2.6 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 31 KiB/s wr, 28 op/s 2024-09-11T22:00:12.939 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:12.940 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:13.024 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T22:00:13.024 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T22:00:13.048 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:13.048 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:13.049 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 64 -lt 100 ']' 2024-09-11T22:00:13.049 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 64 + 1 2024-09-11T22:00:13.102 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=65 2024-09-11T22:00:13.102 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T22:00:13.121 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:13.121 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:13.122 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T22:00:13.122 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T22:00:13.122 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T22:00:13.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:00:12 smithi053 ceph-mon[34908]: pgmap v94: 97 pgs: 97 active+clean; 2.6 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 31 KiB/s wr, 28 op/s 2024-09-11T22:00:13.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:00:12 smithi161 ceph-mon[35615]: pgmap v94: 97 pgs: 97 active+clean; 2.6 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 31 KiB/s wr, 28 op/s 2024-09-11T22:00:13.596 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:13.596 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:13.788 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T22:00:13.788 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T22:00:13.796 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:13.796 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:13.797 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 65 -lt 100 ']' 2024-09-11T22:00:13.797 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 65 + 1 2024-09-11T22:00:13.828 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=66 2024-09-11T22:00:13.828 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T22:00:13.853 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:13.853 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:13.855 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T22:00:13.855 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T22:00:13.855 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T22:00:14.429 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:14.429 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:14.597 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T22:00:14.598 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T22:00:14.611 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:14.612 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:14.612 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 66 -lt 100 ']' 2024-09-11T22:00:14.612 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 66 + 1 2024-09-11T22:00:14.673 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=67 2024-09-11T22:00:14.673 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T22:00:14.681 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:14.681 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:14.682 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T22:00:14.683 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T22:00:14.683 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T22:00:15.127 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:00:14 smithi001 ceph-mon[30935]: pgmap v95: 97 pgs: 97 active+clean; 2.6 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 31 KiB/s wr, 28 op/s 2024-09-11T22:00:15.263 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:15.263 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:15.300 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:00:14 smithi053 ceph-mon[34908]: pgmap v95: 97 pgs: 97 active+clean; 2.6 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 31 KiB/s wr, 28 op/s 2024-09-11T22:00:15.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:00:14 smithi161 ceph-mon[35615]: pgmap v95: 97 pgs: 97 active+clean; 2.6 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 31 KiB/s wr, 28 op/s 2024-09-11T22:00:15.657 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T22:00:15.657 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T22:00:15.687 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:15.687 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:15.688 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 67 -lt 100 ']' 2024-09-11T22:00:15.688 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 67 + 1 2024-09-11T22:00:15.844 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=68 2024-09-11T22:00:15.844 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T22:00:15.905 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:15.905 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:15.906 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T22:00:15.907 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T22:00:15.907 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T22:00:17.098 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:17.098 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:17.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:00:16 smithi001 ceph-mon[30935]: pgmap v96: 97 pgs: 97 active+clean; 2.7 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 43 KiB/s wr, 33 op/s 2024-09-11T22:00:17.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:00:16 smithi053 ceph-mon[34908]: pgmap v96: 97 pgs: 97 active+clean; 2.7 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 43 KiB/s wr, 33 op/s 2024-09-11T22:00:17.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:00:16 smithi161 ceph-mon[35615]: pgmap v96: 97 pgs: 97 active+clean; 2.7 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 43 KiB/s wr, 33 op/s 2024-09-11T22:00:17.506 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T22:00:17.506 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T22:00:17.521 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:17.521 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:17.521 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 68 -lt 100 ']' 2024-09-11T22:00:17.521 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 68 + 1 2024-09-11T22:00:17.848 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=69 2024-09-11T22:00:17.848 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T22:00:17.882 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:17.882 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:17.883 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T22:00:17.884 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T22:00:17.884 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T22:00:19.039 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:19.039 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:19.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:00:18 smithi001 ceph-mon[30935]: pgmap v97: 97 pgs: 97 active+clean; 2.7 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 9 op/s 2024-09-11T22:00:19.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:00:18 smithi053 ceph-mon[34908]: pgmap v97: 97 pgs: 97 active+clean; 2.7 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 9 op/s 2024-09-11T22:00:19.337 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T22:00:19.337 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T22:00:19.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:00:18 smithi161 ceph-mon[35615]: pgmap v97: 97 pgs: 97 active+clean; 2.7 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 9 op/s 2024-09-11T22:00:19.346 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:19.346 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:19.346 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 69 -lt 100 ']' 2024-09-11T22:00:19.346 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 69 + 1 2024-09-11T22:00:19.394 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=70 2024-09-11T22:00:19.394 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T22:00:19.406 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:19.406 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:19.407 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T22:00:19.408 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T22:00:19.408 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T22:00:20.026 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.025+0000 7f2a91c53640 1 -- 172.21.15.1:0/4034073887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a8c109090 msgr2=0x7f2a8c110be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:20.027 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.025+0000 7f2a91c53640 1 --2- 172.21.15.1:0/4034073887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a8c109090 0x7f2a8c110be0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2a84009ca0 tx=0x7f2a8402a8b0 comp rx=0 tx=0).stop 2024-09-11T22:00:20.027 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.025+0000 7f2a91c53640 1 -- 172.21.15.1:0/4034073887 shutdown_connections 2024-09-11T22:00:20.027 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.025+0000 7f2a91c53640 1 --2- 172.21.15.1:0/4034073887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a8c109090 0x7f2a8c110be0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:20.027 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.025+0000 7f2a91c53640 1 --2- 172.21.15.1:0/4034073887 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a8c1048f0 0x7f2a8c108930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:20.027 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.025+0000 7f2a91c53640 1 --2- 172.21.15.1:0/4034073887 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a8c103f40 0x7f2a8c104320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:20.027 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.025+0000 7f2a91c53640 1 -- 172.21.15.1:0/4034073887 >> 172.21.15.1:0/4034073887 conn(0x7f2a8c0ff970 msgr2=0x7f2a8c101d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:00:20.027 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.025+0000 7f2a91c53640 1 -- 172.21.15.1:0/4034073887 shutdown_connections 2024-09-11T22:00:20.027 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.025+0000 7f2a91c53640 1 -- 172.21.15.1:0/4034073887 wait complete. 2024-09-11T22:00:20.033 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.032+0000 7f2a91c53640 1 Processor -- start 2024-09-11T22:00:20.035 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.033+0000 7f2a91c53640 1 -- start start 2024-09-11T22:00:20.035 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.033+0000 7f2a91c53640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a8c103f40 0x7f2a8c19a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:20.035 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.033+0000 7f2a91c53640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a8c109090 0x7f2a8c19aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:20.035 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.033+0000 7f2a91c53640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a8c19ee40 0x7f2a8c19f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:20.035 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.033+0000 7f2a91c53640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8c1140b0 con 0x7f2a8c19ee40 2024-09-11T22:00:20.035 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.033+0000 7f2a91c53640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8c113f30 con 0x7f2a8c109090 2024-09-11T22:00:20.036 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.033+0000 7f2a91c53640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8c114230 con 0x7f2a8c103f40 2024-09-11T22:00:20.036 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.033+0000 7f2a8bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a8c19ee40 0x7f2a8c19f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:20.036 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.033+0000 7f2a8bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a8c19ee40 0x7f2a8c19f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36640/0 (socket says 172.21.15.1:36640) 2024-09-11T22:00:20.036 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.033+0000 7f2a8bfff640 1 -- 172.21.15.1:0/3985660532 learned_addr learned my addr 172.21.15.1:0/3985660532 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:00:20.036 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.033+0000 7f2a8b7fe640 1 --2- 172.21.15.1:0/3985660532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a8c103f40 0x7f2a8c19a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:20.036 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.033+0000 7f2a8affd640 1 --2- 172.21.15.1:0/3985660532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a8c109090 0x7f2a8c19aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:20.036 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.034+0000 7f2a8bfff640 1 -- 172.21.15.1:0/3985660532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a8c103f40 msgr2=0x7f2a8c19a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:20.036 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.034+0000 7f2a8bfff640 1 --2- 172.21.15.1:0/3985660532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a8c103f40 0x7f2a8c19a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:20.037 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.034+0000 7f2a8bfff640 1 -- 172.21.15.1:0/3985660532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a8c109090 msgr2=0x7f2a8c19aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:20.037 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.034+0000 7f2a8bfff640 1 --2- 172.21.15.1:0/3985660532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a8c109090 0x7f2a8c19aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:20.037 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.034+0000 7f2a8bfff640 1 -- 172.21.15.1:0/3985660532 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a8c19f930 con 0x7f2a8c19ee40 2024-09-11T22:00:20.037 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.034+0000 7f2a8bfff640 1 --2- 172.21.15.1:0/3985660532 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a8c19ee40 0x7f2a8c19f2f0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f2a84003ea0 tx=0x7f2a84003ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:00:20.044 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.043+0000 7f2a88ff9640 1 -- 172.21.15.1:0/3985660532 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a84030dc0 con 0x7f2a8c19ee40 2024-09-11T22:00:20.044 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.043+0000 7f2a88ff9640 1 -- 172.21.15.1:0/3985660532 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a84040070 con 0x7f2a8c19ee40 2024-09-11T22:00:20.044 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.043+0000 7f2a91c53640 1 -- 172.21.15.1:0/3985660532 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a8c19fbc0 con 0x7f2a8c19ee40 2024-09-11T22:00:20.044 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.043+0000 7f2a88ff9640 1 -- 172.21.15.1:0/3985660532 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a84045a60 con 0x7f2a8c19ee40 2024-09-11T22:00:20.044 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.044+0000 7f2a91c53640 1 -- 172.21.15.1:0/3985660532 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a8c1a73e0 con 0x7f2a8c19ee40 2024-09-11T22:00:20.046 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.045+0000 7f2a88ff9640 1 -- 172.21.15.1:0/3985660532 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f2a84004040 con 0x7f2a8c19ee40 2024-09-11T22:00:20.047 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.046+0000 7f2a88ff9640 1 --2- 172.21.15.1:0/3985660532 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f2a740a2e80 0x7f2a740a5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:20.047 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.046+0000 7f2a8b7fe640 1 --2- 172.21.15.1:0/3985660532 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f2a740a2e80 0x7f2a740a5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:20.048 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.046+0000 7f2a88ff9640 1 -- 172.21.15.1:0/3985660532 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f2a840f0730 con 0x7f2a8c19ee40 2024-09-11T22:00:20.048 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.047+0000 7f2a8b7fe640 1 --2- 172.21.15.1:0/3985660532 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f2a740a2e80 0x7f2a740a5340 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c0095f0 tx=0x7f2a7c005f90 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:00:20.246 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.244+0000 7f2a949f0740 1 Processor -- start 2024-09-11T22:00:20.246 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.244+0000 7f2a949f0740 1 -- start start 2024-09-11T22:00:20.246 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.244+0000 7f2a949f0740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557dcba12af0 0x557dcba12ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:20.246 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.244+0000 7f2a949f0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557dcba13410 0x557dcba1d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:20.246 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.244+0000 7f2a949f0740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557dcba1dcb0 0x557dcba200a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:20.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.244+0000 7f2a949f0740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x557dcb9772f0 con 0x557dcba12af0 2024-09-11T22:00:20.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.244+0000 7f2a949f0740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x557dcb91f270 con 0x557dcba1dcb0 2024-09-11T22:00:20.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.244+0000 7f2a949f0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x557dcb9732b0 con 0x557dcba13410 2024-09-11T22:00:20.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.245+0000 7f2a8affd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557dcba13410 0x557dcba1d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:20.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.245+0000 7f2a8bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557dcba1dcb0 0x557dcba200a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:20.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.245+0000 7f2a8b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557dcba12af0 0x557dcba12ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:20.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.245+0000 7f2a8affd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557dcba13410 0x557dcba1d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:38344/0 (socket says 172.21.15.1:38344) 2024-09-11T22:00:20.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.245+0000 7f2a8bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557dcba1dcb0 0x557dcba200a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:41492/0 (socket says 172.21.15.1:41492) 2024-09-11T22:00:20.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.245+0000 7f2a8affd640 1 -- 172.21.15.1:0/2633516531 learned_addr learned my addr 172.21.15.1:0/2633516531 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:00:20.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.245+0000 7f2a8affd640 1 -- 172.21.15.1:0/2633516531 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557dcba1dcb0 msgr2=0x557dcba200a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:00:20.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.245+0000 7f2a8affd640 1 --2- 172.21.15.1:0/2633516531 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557dcba1dcb0 0x557dcba200a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:20.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.245+0000 7f2a8affd640 1 -- 172.21.15.1:0/2633516531 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557dcba12af0 msgr2=0x557dcba12ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:00:20.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.245+0000 7f2a8affd640 1 --2- 172.21.15.1:0/2633516531 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557dcba12af0 0x557dcba12ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:20.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.245+0000 7f2a8affd640 1 -- 172.21.15.1:0/2633516531 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557dcb91e340 con 0x557dcba13410 2024-09-11T22:00:20.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.245+0000 7f2a8b7fe640 1 --2- 172.21.15.1:0/2633516531 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557dcba12af0 0x557dcba12ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:00:20.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.245+0000 7f2a8bfff640 1 --2- 172.21.15.1:0/2633516531 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557dcba1dcb0 0x557dcba200a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:00:20.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.245+0000 7f2a8affd640 1 --2- 172.21.15.1:0/2633516531 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557dcba13410 0x557dcba1d770 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2a80009c20 tx=0x7f2a80011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5423595868f179fc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:00:20.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.246+0000 7f2a71ffb640 1 -- 172.21.15.1:0/2633516531 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8001e070 con 0x557dcba13410 2024-09-11T22:00:20.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.246+0000 7f2a71ffb640 1 -- 172.21.15.1:0/2633516531 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a80009dd0 con 0x557dcba13410 2024-09-11T22:00:20.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.246+0000 7f2a71ffb640 1 -- 172.21.15.1:0/2633516531 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a80018730 con 0x557dcba13410 2024-09-11T22:00:20.249 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.246+0000 7f2a949f0740 1 -- 172.21.15.1:0/2633516531 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557dcba13410 msgr2=0x557dcba1d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:20.249 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.246+0000 7f2a949f0740 1 --2- 172.21.15.1:0/2633516531 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557dcba13410 0x557dcba1d770 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2a80009c20 tx=0x7f2a80011810 comp rx=0 tx=0).stop 2024-09-11T22:00:20.249 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.246+0000 7f2a949f0740 1 -- 172.21.15.1:0/2633516531 shutdown_connections 2024-09-11T22:00:20.249 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.246+0000 7f2a949f0740 1 --2- 172.21.15.1:0/2633516531 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557dcba1dcb0 0x557dcba200a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:20.249 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.246+0000 7f2a949f0740 1 --2- 172.21.15.1:0/2633516531 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557dcba13410 0x557dcba1d770 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:20.250 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.246+0000 7f2a949f0740 1 --2- 172.21.15.1:0/2633516531 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557dcba12af0 0x557dcba12ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:20.250 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.246+0000 7f2a949f0740 1 -- 172.21.15.1:0/2633516531 >> 172.21.15.1:0/2633516531 conn(0x557dcba0f060 msgr2=0x557dcba0f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:00:20.250 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.247+0000 7f2a949f0740 1 -- 172.21.15.1:0/2633516531 shutdown_connections 2024-09-11T22:00:20.250 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.247+0000 7f2a949f0740 1 -- 172.21.15.1:0/2633516531 wait complete. 2024-09-11T22:00:20.250 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:00:20.294 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.293+0000 7f2a949f0740 1 -- start start 2024-09-11T22:00:20.295 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.295+0000 7f2a949f0740 1 Processor -- start 2024-09-11T22:00:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.298+0000 7f2a949f0740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557dcba12af0 0x557dcba1ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.298+0000 7f2a949f0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557dcba13410 0x557dcba1b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.298+0000 7f2a949f0740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557dcba1dcb0 0x557dcba1b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.298+0000 7f2a949f0740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x557dcba20880 con 0x557dcba1dcb0 2024-09-11T22:00:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.298+0000 7f2a949f0740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x557dcb900290 con 0x557dcba12af0 2024-09-11T22:00:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.298+0000 7f2a949f0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x557dcba20a00 con 0x557dcba13410 2024-09-11T22:00:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.298+0000 7f2a8bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557dcba1dcb0 0x557dcba1b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.298+0000 7f2a8b7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557dcba12af0 0x557dcba1ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.298+0000 7f2a8bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557dcba1dcb0 0x557dcba1b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36666/0 (socket says 172.21.15.1:36666) 2024-09-11T22:00:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.298+0000 7f2a8bfff640 1 -- 172.21.15.1:0/2864156287 learned_addr learned my addr 172.21.15.1:0/2864156287 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:00:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.298+0000 7f2a8affd640 1 --2- 172.21.15.1:0/2864156287 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557dcba13410 0x557dcba1b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.298+0000 7f2a8b7fe640 1 -- 172.21.15.1:0/2864156287 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557dcba13410 msgr2=0x557dcba1b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:00:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.298+0000 7f2a8b7fe640 1 --2- 172.21.15.1:0/2864156287 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557dcba13410 0x557dcba1b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.298+0000 7f2a8b7fe640 1 -- 172.21.15.1:0/2864156287 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557dcba1dcb0 msgr2=0x557dcba1b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:00:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.298+0000 7f2a8b7fe640 1 --2- 172.21.15.1:0/2864156287 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557dcba1dcb0 0x557dcba1b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.298+0000 7f2a8b7fe640 1 -- 172.21.15.1:0/2864156287 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557dcb68a210 con 0x557dcba12af0 2024-09-11T22:00:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.298+0000 7f2a8bfff640 1 --2- 172.21.15.1:0/2864156287 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557dcba1dcb0 0x557dcba1b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:00:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.299+0000 7f2a949f0740 10 client.0 _ll_register_callbacks cb 0x557dcb931140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:00:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.299+0000 7f2a8b7fe640 1 --2- 172.21.15.1:0/2864156287 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557dcba12af0 0x557dcba1ac20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c0118d0 tx=0x7f2a7c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd4a5b9704603220 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:00:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.299+0000 7f2a6affd640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-11T22:00:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.299+0000 7f2a6affd640 1 -- 172.21.15.1:0/2864156287 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c01db20 con 0x557dcba12af0 2024-09-11T22:00:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.299+0000 7f2a6affd640 1 -- 172.21.15.1:0/2864156287 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f2a7c01dcc0 con 0x557dcba12af0 2024-09-11T22:00:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.299+0000 7f2a949f0740 1 -- 172.21.15.1:0/2864156287 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a80009dd0 con 0x557dcba12af0 2024-09-11T22:00:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.301+0000 7f2a6affd640 1 -- 172.21.15.1:0/2864156287 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2a7c004030 con 0x557dcba12af0 2024-09-11T22:00:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.301+0000 7f2a949f0740 10 client.25616 fetch_fsmap learned FSMap version 28 2024-09-11T22:00:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.301+0000 7f2a949f0740 1 -- 172.21.15.1:0/2864156287 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({fsmap=28}) -- 0x557dcb756620 con 0x557dcba12af0 2024-09-11T22:00:20.365 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.365+0000 7f2a6affd640 1 -- 172.21.15.1:0/2864156287 <== mon.2 v2:172.21.15.53:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2a7c0041d0 con 0x557dcba12af0 2024-09-11T22:00:20.366 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.365+0000 7f2a949f0740 10 client.25616 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:00:20.366 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.365+0000 7f2a949f0740 10 client.25616 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:00:20.366 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.365+0000 7f2a949f0740 20 client.25616 populate_metadata read hostname 'smithi001' 2024-09-11T22:00:20.366 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.365+0000 7f2a949f0740 1 --2- 172.21.15.1:0/2864156287 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x557dcba2dab0 0x557dcba2ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:20.366 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.365+0000 7f2a949f0740 4 client.25616 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:00:20.366 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.365+0000 7f2a949f0740 1 -- 172.21.15.1:0/2864156287 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557dcb984a20 con 0x557dcba2dab0 2024-09-11T22:00:20.367 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.365+0000 7f2a8affd640 1 --2- 172.21.15.1:0/2864156287 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x557dcba2dab0 0x557dcba2ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:20.367 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.366+0000 7f2a8affd640 1 --2- 172.21.15.1:0/2864156287 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x557dcba2dab0 0x557dcba2ff70 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1444add900faf23 server_cookie=acad5a6d9f64ea77 in_seq=0 out_seq=0 2024-09-11T22:00:20.367 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.366+0000 7f2a6affd640 10 client.25616 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:00:20.367 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.367+0000 7f2a6affd640 1 -- 172.21.15.1:0/2864156287 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557dcb984a20 con 0x557dcba2dab0 2024-09-11T22:00:20.367 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.367+0000 7f2a6affd640 10 client.25616 handle_command_reply: tid=0 2024-09-11T22:00:20.369 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.367+0000 7f2a949f0740 1 client.25616 shutdown 2024-09-11T22:00:20.369 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.368+0000 7f2a949f0740 1 -- 172.21.15.1:0/2864156287 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557dcba12af0 msgr2=0x557dcba1ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:20.369 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.368+0000 7f2a949f0740 1 --2- 172.21.15.1:0/2864156287 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557dcba12af0 0x557dcba1ac20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c0118d0 tx=0x7f2a7c004730 comp rx=0 tx=0).stop 2024-09-11T22:00:20.369 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.368+0000 7f2a949f0740 1 -- 172.21.15.1:0/2864156287 shutdown_connections 2024-09-11T22:00:20.369 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.368+0000 7f2a949f0740 1 --2- 172.21.15.1:0/2864156287 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x557dcba2dab0 0x557dcba2ff70 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:20.369 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.368+0000 7f2a949f0740 1 --2- 172.21.15.1:0/2864156287 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557dcba1dcb0 0x557dcba1b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:20.369 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.368+0000 7f2a949f0740 1 --2- 172.21.15.1:0/2864156287 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557dcba13410 0x557dcba1b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:20.369 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.368+0000 7f2a949f0740 1 --2- 172.21.15.1:0/2864156287 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557dcba12af0 0x557dcba1ac20 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:20.369 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.368+0000 7f2a6affd640 0 client.25616 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:00:20.369 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.368+0000 7f2a949f0740 1 -- 172.21.15.1:0/2864156287 >> 172.21.15.1:0/2864156287 conn(0x557dcba0f060 msgr2=0x557dcba0e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:00:20.369 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.368+0000 7f2a949f0740 1 -- 172.21.15.1:0/2864156287 shutdown_connections 2024-09-11T22:00:20.369 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.368+0000 7f2a949f0740 1 -- 172.21.15.1:0/2864156287 wait complete. 2024-09-11T22:00:20.370 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.368+0000 7f2a949f0740 20 client.25616 trim_cache size 0 max 16384 2024-09-11T22:00:20.404 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.401+0000 7f2a949f0740 1 Processor -- start 2024-09-11T22:00:20.404 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.401+0000 7f2a949f0740 1 -- start start 2024-09-11T22:00:20.404 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.401+0000 7f2a949f0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557dcba12af0 0x557dcba1b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:20.404 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.401+0000 7f2a949f0740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557dcba13410 0x557dcba1cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:20.405 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.401+0000 7f2a949f0740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557dcba1dcb0 0x557dcba35880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:20.405 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.401+0000 7f2a949f0740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8c1af990 con 0x557dcba13410 2024-09-11T22:00:20.405 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.401+0000 7f2a949f0740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8c1b14d0 con 0x557dcba1dcb0 2024-09-11T22:00:20.405 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.401+0000 7f2a949f0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8c1b0060 con 0x557dcba12af0 2024-09-11T22:00:20.405 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.401+0000 7f2a8affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557dcba13410 0x557dcba1cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:20.405 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.401+0000 7f2a8affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557dcba13410 0x557dcba1cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36672/0 (socket says 172.21.15.1:36672) 2024-09-11T22:00:20.405 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.401+0000 7f2a8affd640 1 -- 172.21.15.1:0/2365709448 learned_addr learned my addr 172.21.15.1:0/2365709448 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:00:20.405 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.401+0000 7f2a8b7fe640 1 --2- 172.21.15.1:0/2365709448 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557dcba12af0 0x557dcba1b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:20.405 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.401+0000 7f2a8bfff640 1 --2- 172.21.15.1:0/2365709448 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557dcba1dcb0 0x557dcba35880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:20.405 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.402+0000 7f2a8affd640 1 -- 172.21.15.1:0/2365709448 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557dcba12af0 msgr2=0x557dcba1b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:00:20.405 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.402+0000 7f2a8affd640 1 --2- 172.21.15.1:0/2365709448 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557dcba12af0 0x557dcba1b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:20.406 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.402+0000 7f2a8affd640 1 -- 172.21.15.1:0/2365709448 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557dcba1dcb0 msgr2=0x557dcba35880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:00:20.406 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.402+0000 7f2a8affd640 1 --2- 172.21.15.1:0/2365709448 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557dcba1dcb0 0x557dcba35880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:20.406 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.402+0000 7f2a8affd640 1 -- 172.21.15.1:0/2365709448 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557dcba20410 con 0x557dcba13410 2024-09-11T22:00:20.406 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.402+0000 7f2a8affd640 1 --2- 172.21.15.1:0/2365709448 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557dcba13410 0x557dcba1cb90 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f2a80009c20 tx=0x7f2a800095b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33b71a64256f8540 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:00:20.406 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.402+0000 7f2a717fa640 1 -- 172.21.15.1:0/2365709448 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8001e040 con 0x557dcba13410 2024-09-11T22:00:20.406 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.402+0000 7f2a717fa640 1 -- 172.21.15.1:0/2365709448 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a8001f040 con 0x557dcba13410 2024-09-11T22:00:20.406 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.402+0000 7f2a717fa640 1 -- 172.21.15.1:0/2365709448 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a80019ab0 con 0x557dcba13410 2024-09-11T22:00:20.406 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.403+0000 7f2a949f0740 1 -- 172.21.15.1:0/2365709448 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557dcba13410 msgr2=0x557dcba1cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:20.406 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.403+0000 7f2a949f0740 1 --2- 172.21.15.1:0/2365709448 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557dcba13410 0x557dcba1cb90 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f2a80009c20 tx=0x7f2a800095b0 comp rx=0 tx=0).stop 2024-09-11T22:00:20.406 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.403+0000 7f2a949f0740 1 -- 172.21.15.1:0/2365709448 shutdown_connections 2024-09-11T22:00:20.406 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.403+0000 7f2a949f0740 1 --2- 172.21.15.1:0/2365709448 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557dcba1dcb0 0x557dcba35880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:20.406 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.403+0000 7f2a949f0740 1 --2- 172.21.15.1:0/2365709448 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557dcba13410 0x557dcba1cb90 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:20.406 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.403+0000 7f2a949f0740 1 --2- 172.21.15.1:0/2365709448 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557dcba12af0 0x557dcba1b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:20.407 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.403+0000 7f2a949f0740 1 -- 172.21.15.1:0/2365709448 >> 172.21.15.1:0/2365709448 conn(0x557dcba0f060 msgr2=0x557dcba1c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:00:20.407 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.403+0000 7f2a949f0740 1 -- 172.21.15.1:0/2365709448 shutdown_connections 2024-09-11T22:00:20.407 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.403+0000 7f2a949f0740 1 -- 172.21.15.1:0/2365709448 wait complete. 2024-09-11T22:00:20.407 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.403+0000 7f2a949f0740 1 -- start start 2024-09-11T22:00:20.407 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.404+0000 7f2a949f0740 1 Processor -- start 2024-09-11T22:00:20.410 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.404+0000 7f2a949f0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557dcba12af0 0x557dcba25c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:20.410 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.404+0000 7f2a949f0740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557dcba13410 0x557dcba261d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:20.410 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.404+0000 7f2a949f0740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557dcba1dcb0 0x557dcba22c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:20.410 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.404+0000 7f2a949f0740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8c1b0e00 con 0x557dcba13410 2024-09-11T22:00:20.410 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.404+0000 7f2a949f0740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8c1b0730 con 0x557dcba1dcb0 2024-09-11T22:00:20.410 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.404+0000 7f2a949f0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8c1ad970 con 0x557dcba12af0 2024-09-11T22:00:20.411 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.404+0000 7f2a949f0740 10 client.0 _ll_register_callbacks cb 0x7f2a7c00e4b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:00:20.411 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.404+0000 7f2a8affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557dcba13410 0x557dcba261d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:20.411 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.404+0000 7f2a8affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557dcba13410 0x557dcba261d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36686/0 (socket says 172.21.15.1:36686) 2024-09-11T22:00:20.411 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.404+0000 7f2a8affd640 1 -- 172.21.15.1:0/733997830 learned_addr learned my addr 172.21.15.1:0/733997830 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:00:20.411 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.404+0000 7f2a8bfff640 1 --2- 172.21.15.1:0/733997830 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557dcba1dcb0 0x557dcba22c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:20.411 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.404+0000 7f2a8b7fe640 1 --2- 172.21.15.1:0/733997830 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557dcba12af0 0x557dcba25c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:20.411 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.405+0000 7f2a8affd640 1 -- 172.21.15.1:0/733997830 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557dcba12af0 msgr2=0x557dcba25c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:00:20.411 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.405+0000 7f2a8affd640 1 --2- 172.21.15.1:0/733997830 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557dcba12af0 0x557dcba25c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:20.411 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.405+0000 7f2a8affd640 1 -- 172.21.15.1:0/733997830 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557dcba1dcb0 msgr2=0x557dcba22c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:00:20.411 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.405+0000 7f2a8affd640 1 --2- 172.21.15.1:0/733997830 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557dcba1dcb0 0x557dcba22c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:20.411 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.405+0000 7f2a8affd640 1 -- 172.21.15.1:0/733997830 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557dcba26870 con 0x557dcba13410 2024-09-11T22:00:20.412 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.405+0000 7f2a8bfff640 1 --2- 172.21.15.1:0/733997830 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557dcba1dcb0 0x557dcba22c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:00:20.412 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.405+0000 7f2a8affd640 1 --2- 172.21.15.1:0/733997830 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557dcba13410 0x557dcba261d0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f2a8001c040 tx=0x7f2a80005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=893773a38a36700e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:00:20.412 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.405+0000 7f2a6affd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:00:20.412 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.405+0000 7f2a6affd640 1 -- 172.21.15.1:0/733997830 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8001e040 con 0x557dcba13410 2024-09-11T22:00:20.412 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.405+0000 7f2a6affd640 1 -- 172.21.15.1:0/733997830 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f2a80019410 con 0x557dcba13410 2024-09-11T22:00:20.412 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.405+0000 7f2a949f0740 1 -- 172.21.15.1:0/733997830 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a8001f040 con 0x557dcba13410 2024-09-11T22:00:20.412 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.405+0000 7f2a6affd640 1 -- 172.21.15.1:0/733997830 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2a80041050 con 0x557dcba13410 2024-09-11T22:00:20.412 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.405+0000 7f2a949f0740 10 client.16047 fetch_fsmap learned FSMap version 28 2024-09-11T22:00:20.412 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.405+0000 7f2a949f0740 1 -- 172.21.15.1:0/733997830 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557dcba98e10 con 0x557dcba13410 2024-09-11T22:00:20.412 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.405+0000 7f2a6affd640 1 -- 172.21.15.1:0/733997830 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a8001f040 con 0x557dcba13410 2024-09-11T22:00:20.412 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.406+0000 7f2a6affd640 1 -- 172.21.15.1:0/733997830 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2a80025040 con 0x557dcba13410 2024-09-11T22:00:20.412 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.406+0000 7f2a949f0740 10 client.16047 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:00:20.412 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.406+0000 7f2a949f0740 10 client.16047 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:00:20.413 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.406+0000 7f2a949f0740 20 client.16047 populate_metadata read hostname 'smithi001' 2024-09-11T22:00:20.413 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.406+0000 7f2a949f0740 1 --2- 172.21.15.1:0/733997830 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x557dcba992e0 0x557dcba9b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:20.413 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.406+0000 7f2a949f0740 4 client.16047 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:00:20.413 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.406+0000 7f2a949f0740 1 -- 172.21.15.1:0/733997830 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557dcb97ed70 con 0x557dcba992e0 2024-09-11T22:00:20.413 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.406+0000 7f2a8b7fe640 1 --2- 172.21.15.1:0/733997830 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x557dcba992e0 0x557dcba9b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:20.413 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.406+0000 7f2a6affd640 10 client.16047 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:00:20.413 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.406+0000 7f2a8b7fe640 1 --2- 172.21.15.1:0/733997830 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x557dcba992e0 0x557dcba9b6d0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6a98d3ed04d7de4 server_cookie=9152390c14f80213 in_seq=0 out_seq=0 2024-09-11T22:00:20.436 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.435+0000 7f2a6affd640 1 -- 172.21.15.1:0/733997830 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557dcb97ed70 con 0x557dcba992e0 2024-09-11T22:00:20.437 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.435+0000 7f2a6affd640 10 client.16047 handle_command_reply: tid=0 2024-09-11T22:00:20.437 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.435+0000 7f2a949f0740 1 client.16047 shutdown 2024-09-11T22:00:20.437 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.435+0000 7f2a949f0740 1 -- 172.21.15.1:0/733997830 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557dcba13410 msgr2=0x557dcba261d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:20.437 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.435+0000 7f2a949f0740 1 --2- 172.21.15.1:0/733997830 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557dcba13410 0x557dcba261d0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f2a8001c040 tx=0x7f2a80005f90 comp rx=0 tx=0).stop 2024-09-11T22:00:20.437 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.435+0000 7f2a949f0740 1 -- 172.21.15.1:0/733997830 shutdown_connections 2024-09-11T22:00:20.437 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.435+0000 7f2a949f0740 1 --2- 172.21.15.1:0/733997830 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x557dcba992e0 0x557dcba9b6d0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:20.437 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.435+0000 7f2a949f0740 1 --2- 172.21.15.1:0/733997830 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557dcba1dcb0 0x557dcba22c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:20.437 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.435+0000 7f2a949f0740 1 --2- 172.21.15.1:0/733997830 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557dcba13410 0x557dcba261d0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:20.437 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.435+0000 7f2a949f0740 1 --2- 172.21.15.1:0/733997830 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557dcba12af0 0x557dcba25c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:20.437 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.435+0000 7f2a6affd640 0 client.16047 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:00:20.437 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.435+0000 7f2a949f0740 1 -- 172.21.15.1:0/733997830 >> 172.21.15.1:0/733997830 conn(0x557dcba0f060 msgr2=0x557dcba190a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:00:20.438 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.435+0000 7f2a949f0740 1 -- 172.21.15.1:0/733997830 shutdown_connections 2024-09-11T22:00:20.438 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.435+0000 7f2a949f0740 1 -- 172.21.15.1:0/733997830 wait complete. 2024-09-11T22:00:20.438 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.435+0000 7f2a949f0740 20 client.16047 trim_cache size 0 max 16384 2024-09-11T22:00:20.438 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-09-11T22:00:20.438 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-09-11T22:00:20.438 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-09-11T22:00:20.438 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-09-11T22:00:20.439 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.436+0000 7f2a91c53640 1 -- 172.21.15.1:0/3985660532 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f2a740a2e80 msgr2=0x7f2a740a5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:20.439 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.436+0000 7f2a91c53640 1 --2- 172.21.15.1:0/3985660532 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f2a740a2e80 0x7f2a740a5340 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c0095f0 tx=0x7f2a7c005f90 comp rx=0 tx=0).stop 2024-09-11T22:00:20.439 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.436+0000 7f2a91c53640 1 -- 172.21.15.1:0/3985660532 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a8c19ee40 msgr2=0x7f2a8c19f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:20.439 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.436+0000 7f2a91c53640 1 --2- 172.21.15.1:0/3985660532 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a8c19ee40 0x7f2a8c19f2f0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f2a84003ea0 tx=0x7f2a84003ed0 comp rx=0 tx=0).stop 2024-09-11T22:00:20.439 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.436+0000 7f2a91c53640 1 -- 172.21.15.1:0/3985660532 shutdown_connections 2024-09-11T22:00:20.439 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.436+0000 7f2a91c53640 1 --2- 172.21.15.1:0/3985660532 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f2a740a2e80 0x7f2a740a5340 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.436+0000 7f2a91c53640 1 --2- 172.21.15.1:0/3985660532 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a8c19ee40 0x7f2a8c19f2f0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.436+0000 7f2a91c53640 1 --2- 172.21.15.1:0/3985660532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a8c109090 0x7f2a8c19aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.436+0000 7f2a91c53640 1 --2- 172.21.15.1:0/3985660532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a8c103f40 0x7f2a8c19a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.436+0000 7f2a91c53640 1 -- 172.21.15.1:0/3985660532 >> 172.21.15.1:0/3985660532 conn(0x7f2a8c0ff970 msgr2=0x7f2a8c100d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:00:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.436+0000 7f2a91c53640 1 -- 172.21.15.1:0/3985660532 shutdown_connections 2024-09-11T22:00:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:20.436+0000 7f2a91c53640 1 -- 172.21.15.1:0/3985660532 wait complete. 2024-09-11T22:00:20.514 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:20.514 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:20.953 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T22:00:20.953 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T22:00:20.962 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:20.962 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:20.962 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 70 -lt 100 ']' 2024-09-11T22:00:20.962 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 70 + 1 2024-09-11T22:00:21.035 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=71 2024-09-11T22:00:21.036 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T22:00:21.105 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:21.105 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:21.106 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T22:00:21.106 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T22:00:21.106 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T22:00:21.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:00:20 smithi001 ceph-mon[30935]: pgmap v98: 97 pgs: 97 active+clean; 2.8 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 22 KiB/s wr, 14 op/s 2024-09-11T22:00:21.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:00:20 smithi053 ceph-mon[34908]: pgmap v98: 97 pgs: 97 active+clean; 2.8 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 22 KiB/s wr, 14 op/s 2024-09-11T22:00:21.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:00:20 smithi161 ceph-mon[35615]: pgmap v98: 97 pgs: 97 active+clean; 2.8 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 22 KiB/s wr, 14 op/s 2024-09-11T22:00:22.164 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:22.164 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:22.630 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T22:00:22.630 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T22:00:22.646 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:22.646 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:22.646 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 71 -lt 100 ']' 2024-09-11T22:00:22.646 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 71 + 1 2024-09-11T22:00:22.684 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=72 2024-09-11T22:00:22.684 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T22:00:22.739 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:22.739 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:22.741 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T22:00:22.741 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T22:00:22.741 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T22:00:23.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:00:23 smithi161 ceph-mon[35615]: pgmap v99: 97 pgs: 97 active+clean; 2.8 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 21 KiB/s wr, 9 op/s 2024-09-11T22:00:23.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:00:23 smithi001 ceph-mon[30935]: pgmap v99: 97 pgs: 97 active+clean; 2.8 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 21 KiB/s wr, 9 op/s 2024-09-11T22:00:23.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:00:23 smithi053 ceph-mon[34908]: pgmap v99: 97 pgs: 97 active+clean; 2.8 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 21 KiB/s wr, 9 op/s 2024-09-11T22:00:24.214 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:24.214 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:24.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:00:24 smithi161 ceph-mon[35615]: pgmap v100: 97 pgs: 97 active+clean; 2.8 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 21 KiB/s wr, 9 op/s 2024-09-11T22:00:24.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:00:24 smithi001 ceph-mon[30935]: pgmap v100: 97 pgs: 97 active+clean; 2.8 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 21 KiB/s wr, 9 op/s 2024-09-11T22:00:24.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:00:24 smithi053 ceph-mon[34908]: pgmap v100: 97 pgs: 97 active+clean; 2.8 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 21 KiB/s wr, 9 op/s 2024-09-11T22:00:24.703 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T22:00:24.703 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T22:00:24.720 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:24.720 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:24.720 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 72 -lt 100 ']' 2024-09-11T22:00:24.720 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 72 + 1 2024-09-11T22:00:24.770 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=73 2024-09-11T22:00:24.770 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T22:00:24.864 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:24.864 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:24.865 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T22:00:24.865 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T22:00:24.866 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T22:00:26.180 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:26.180 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:26.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:00:26 smithi001 ceph-mon[30935]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:00:26.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:00:26 smithi001 ceph-mon[30935]: pgmap v101: 97 pgs: 97 active+clean; 2.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 19 op/s 2024-09-11T22:00:26.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:00:26 smithi053 ceph-mon[34908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:00:26.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:00:26 smithi053 ceph-mon[34908]: pgmap v101: 97 pgs: 97 active+clean; 2.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 19 op/s 2024-09-11T22:00:26.838 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T22:00:26.839 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T22:00:26.841 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:00:26 smithi161 ceph-mon[35615]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:00:26.841 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:00:26 smithi161 ceph-mon[35615]: pgmap v101: 97 pgs: 97 active+clean; 2.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 19 op/s 2024-09-11T22:00:26.862 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:26.862 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:26.862 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 73 -lt 100 ']' 2024-09-11T22:00:26.862 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 73 + 1 2024-09-11T22:00:26.933 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=74 2024-09-11T22:00:26.933 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T22:00:26.994 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:26.994 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:26.996 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T22:00:26.996 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T22:00:26.996 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T22:00:28.371 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:28.372 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:28.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:00:28 smithi053 ceph-mon[34908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:00:28.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:00:28 smithi053 ceph-mon[34908]: pgmap v102: 97 pgs: 97 active+clean; 2.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 14 op/s 2024-09-11T22:00:28.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:00:28 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:00:28.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:00:28 smithi161 ceph-mon[35615]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:00:28.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:00:28 smithi161 ceph-mon[35615]: pgmap v102: 97 pgs: 97 active+clean; 2.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 14 op/s 2024-09-11T22:00:28.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:00:28 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:00:28.862 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T22:00:28.862 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T22:00:28.877 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:28.877 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:28.877 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 74 -lt 100 ']' 2024-09-11T22:00:28.878 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 74 + 1 2024-09-11T22:00:28.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:00:28 smithi001 ceph-mon[30935]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:00:28.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:00:28 smithi001 ceph-mon[30935]: pgmap v102: 97 pgs: 97 active+clean; 2.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 14 op/s 2024-09-11T22:00:28.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:00:28 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:00:28.974 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=75 2024-09-11T22:00:28.974 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T22:00:29.209 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:29.210 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:29.210 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T22:00:29.211 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T22:00:29.211 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T22:00:30.476 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.7 2024-09-11T22:00:30.476 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:00:30.480 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:30.480 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:30.953 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T22:00:30.953 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T22:00:30.994 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:30.994 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:30.994 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 75 -lt 100 ']' 2024-09-11T22:00:30.994 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 75 + 1 2024-09-11T22:00:31.062 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=76 2024-09-11T22:00:31.062 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T22:00:31.104 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:31.104 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:31.105 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T22:00:31.105 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T22:00:31.105 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T22:00:31.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:00:30 smithi001 ceph-mon[30935]: pgmap v103: 97 pgs: 97 active+clean; 3.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 KiB/s wr, 22 op/s 2024-09-11T22:00:31.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:00:30 smithi053 ceph-mon[34908]: pgmap v103: 97 pgs: 97 active+clean; 3.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 KiB/s wr, 22 op/s 2024-09-11T22:00:31.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:00:30 smithi161 ceph-mon[35615]: pgmap v103: 97 pgs: 97 active+clean; 3.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 KiB/s wr, 22 op/s 2024-09-11T22:00:32.270 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:32.271 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:32.577 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T22:00:32.577 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T22:00:32.594 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:32.595 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:32.595 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 76 -lt 100 ']' 2024-09-11T22:00:32.595 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 76 + 1 2024-09-11T22:00:32.682 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=77 2024-09-11T22:00:32.682 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T22:00:32.763 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:32.763 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:32.764 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T22:00:32.764 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T22:00:32.764 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T22:00:33.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:00:32 smithi161 ceph-mon[35615]: pgmap v104: 97 pgs: 97 active+clean; 3.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 17 op/s 2024-09-11T22:00:33.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:00:32 smithi001 ceph-mon[30935]: pgmap v104: 97 pgs: 97 active+clean; 3.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 17 op/s 2024-09-11T22:00:33.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:00:32 smithi053 ceph-mon[34908]: pgmap v104: 97 pgs: 97 active+clean; 3.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 17 op/s 2024-09-11T22:00:34.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:00:33 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:00:34.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:00:33 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:00:34.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:00:33 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T22:00:34.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:00:33 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:00:34.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:00:33 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:00:34.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:00:33 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T22:00:34.344 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:00:33 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:00:34.345 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:00:33 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:00:34.345 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:00:33 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T22:00:34.345 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:34.345 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:34.486 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T22:00:34.486 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T22:00:34.502 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:34.502 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:34.502 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 77 -lt 100 ']' 2024-09-11T22:00:34.502 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 77 + 1 2024-09-11T22:00:34.520 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=78 2024-09-11T22:00:34.520 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T22:00:34.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:34.551+0000 7f3fc2a23640 1 -- 172.21.15.1:0/4238751764 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3fbc10c170 msgr2=0x7f3fbc10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:34.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:34.551+0000 7f3fc2a23640 1 --2- 172.21.15.1:0/4238751764 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3fbc10c170 0x7f3fbc10e610 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f3fb4009ca0 tx=0x7f3fb402a950 comp rx=0 tx=0).stop 2024-09-11T22:00:34.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:34.551+0000 7f3fc2a23640 1 -- 172.21.15.1:0/4238751764 shutdown_connections 2024-09-11T22:00:34.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:34.551+0000 7f3fc2a23640 1 --2- 172.21.15.1:0/4238751764 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3fbc10c170 0x7f3fbc10e610 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:34.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:34.551+0000 7f3fc2a23640 1 --2- 172.21.15.1:0/4238751764 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fbc103660 0x7f3fbc10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:34.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:34.551+0000 7f3fc2a23640 1 --2- 172.21.15.1:0/4238751764 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3fbc102d40 0x7f3fbc103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:34.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:34.551+0000 7f3fc2a23640 1 -- 172.21.15.1:0/4238751764 >> 172.21.15.1:0/4238751764 conn(0x7f3fbc0fc7a0 msgr2=0x7f3fbc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:00:34.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:34.551+0000 7f3fc2a23640 1 -- 172.21.15.1:0/4238751764 shutdown_connections 2024-09-11T22:00:34.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:34.551+0000 7f3fc2a23640 1 -- 172.21.15.1:0/4238751764 wait complete. 2024-09-11T22:00:34.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:34.552+0000 7f3fc2a23640 1 Processor -- start 2024-09-11T22:00:34.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:34.552+0000 7f3fc2a23640 1 -- start start 2024-09-11T22:00:34.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:34.552+0000 7f3fc2a23640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3fbc102d40 0x7f3fbc1983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:34.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:34.552+0000 7f3fc2a23640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3fbc103660 0x7f3fbc198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:34.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:34.552+0000 7f3fc2a23640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fbc10c170 0x7f3fbc19cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:34.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:34.552+0000 7f3fc2a23640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3fbc111fc0 con 0x7f3fbc103660 2024-09-11T22:00:34.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:34.552+0000 7f3fc2a23640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3fbc111e40 con 0x7f3fbc10c170 2024-09-11T22:00:34.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:34.552+0000 7f3fc2a23640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3fbc112140 con 0x7f3fbc102d40 2024-09-11T22:00:34.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:34.552+0000 7f3fb3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3fbc103660 0x7f3fbc198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:34.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:34.552+0000 7f3fb3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3fbc103660 0x7f3fbc198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55922/0 (socket says 172.21.15.1:55922) 2024-09-11T22:00:34.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:34.552+0000 7f3fb3fff640 1 -- 172.21.15.1:0/2212610175 learned_addr learned my addr 172.21.15.1:0/2212610175 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:00:34.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:34.553+0000 7f3fbbfff640 1 --2- 172.21.15.1:0/2212610175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3fbc102d40 0x7f3fbc1983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:34.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:34.553+0000 7f3fc0fc0640 1 --2- 172.21.15.1:0/2212610175 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fbc10c170 0x7f3fbc19cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:34.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:34.553+0000 7f3fb3fff640 1 -- 172.21.15.1:0/2212610175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3fbc102d40 msgr2=0x7f3fbc1983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:34.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:34.553+0000 7f3fb3fff640 1 --2- 172.21.15.1:0/2212610175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3fbc102d40 0x7f3fbc1983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:34.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:34.553+0000 7f3fb3fff640 1 -- 172.21.15.1:0/2212610175 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fbc10c170 msgr2=0x7f3fbc19cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:34.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:34.553+0000 7f3fb3fff640 1 --2- 172.21.15.1:0/2212610175 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fbc10c170 0x7f3fbc19cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:34.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:34.553+0000 7f3fb3fff640 1 -- 172.21.15.1:0/2212610175 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fbc19d370 con 0x7f3fbc103660 2024-09-11T22:00:34.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:34.553+0000 7f3fbbfff640 1 --2- 172.21.15.1:0/2212610175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3fbc102d40 0x7f3fbc1983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:00:34.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:34.553+0000 7f3fc0fc0640 1 --2- 172.21.15.1:0/2212610175 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fbc10c170 0x7f3fbc19cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:00:34.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:34.553+0000 7f3fb3fff640 1 --2- 172.21.15.1:0/2212610175 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3fbc103660 0x7f3fbc198900 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f3fa400d950 tx=0x7f3fa400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:00:34.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:34.553+0000 7f3fb9ffb640 1 -- 172.21.15.1:0/2212610175 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa4017030 con 0x7f3fbc103660 2024-09-11T22:00:34.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:34.553+0000 7f3fc2a23640 1 -- 172.21.15.1:0/2212610175 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fbc19d660 con 0x7f3fbc103660 2024-09-11T22:00:34.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:34.553+0000 7f3fb9ffb640 1 -- 172.21.15.1:0/2212610175 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fa4004020 con 0x7f3fbc103660 2024-09-11T22:00:34.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:34.553+0000 7f3fb9ffb640 1 -- 172.21.15.1:0/2212610175 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa40047a0 con 0x7f3fbc103660 2024-09-11T22:00:34.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:34.553+0000 7f3fc2a23640 1 -- 172.21.15.1:0/2212610175 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fbc1a4f40 con 0x7f3fbc103660 2024-09-11T22:00:34.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:34.556+0000 7f3fb9ffb640 1 -- 172.21.15.1:0/2212610175 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f3fa40041d0 con 0x7f3fbc103660 2024-09-11T22:00:34.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:34.556+0000 7f3fb9ffb640 1 --2- 172.21.15.1:0/2212610175 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f3fa80a2cf0 0x7f3fa80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:34.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:34.557+0000 7f3fbbfff640 1 --2- 172.21.15.1:0/2212610175 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f3fa80a2cf0 0x7f3fa80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:34.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:34.557+0000 7f3fb9ffb640 1 -- 172.21.15.1:0/2212610175 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f3fa40cee80 con 0x7f3fbc103660 2024-09-11T22:00:34.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:34.557+0000 7f3fbbfff640 1 --2- 172.21.15.1:0/2212610175 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f3fa80a2cf0 0x7f3fa80a51b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3fac0058c0 tx=0x7f3fac005ec0 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:00:34.560 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:34.561 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:34.562 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T22:00:34.562 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T22:00:34.562 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T22:00:35.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.151+0000 7f3fc4dfc740 1 Processor -- start 2024-09-11T22:00:35.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.151+0000 7f3fc4dfc740 1 -- start start 2024-09-11T22:00:35.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.152+0000 7f3fc4dfc740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e2ed00b40 0x558e2ecff090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:35.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.152+0000 7f3fc4dfc740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e2ecff5d0 0x558e2ecffa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:35.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.152+0000 7f3fc4dfc740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558e2ed0a370 0x558e2ed0c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:35.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.152+0000 7f3fc4dfc740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x558e2ec652f0 con 0x558e2ed0a370 2024-09-11T22:00:35.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.152+0000 7f3fc4dfc740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x558e2ec0ccf0 con 0x558e2ecff5d0 2024-09-11T22:00:35.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.152+0000 7f3fc4dfc740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x558e2ec612b0 con 0x558e2ed00b40 2024-09-11T22:00:35.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.152+0000 7f3fc0fc0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558e2ed0a370 0x558e2ed0c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:35.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.152+0000 7f3fc0fc0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558e2ed0a370 0x558e2ed0c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55926/0 (socket says 172.21.15.1:55926) 2024-09-11T22:00:35.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.152+0000 7f3fc0fc0640 1 -- 172.21.15.1:0/4043524801 learned_addr learned my addr 172.21.15.1:0/4043524801 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:00:35.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.152+0000 7f3fbbfff640 1 --2- 172.21.15.1:0/4043524801 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e2ed00b40 0x558e2ecff090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:35.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.152+0000 7f3fb3fff640 1 --2- 172.21.15.1:0/4043524801 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e2ecff5d0 0x558e2ecffa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:35.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.152+0000 7f3fc0fc0640 1 -- 172.21.15.1:0/4043524801 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e2ed00b40 msgr2=0x558e2ecff090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:00:35.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.152+0000 7f3fc0fc0640 1 --2- 172.21.15.1:0/4043524801 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e2ed00b40 0x558e2ecff090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:35.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.152+0000 7f3fc0fc0640 1 -- 172.21.15.1:0/4043524801 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e2ecff5d0 msgr2=0x558e2ecffa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:00:35.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.152+0000 7f3fc0fc0640 1 --2- 172.21.15.1:0/4043524801 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e2ecff5d0 0x558e2ecffa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:35.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.152+0000 7f3fc0fc0640 1 -- 172.21.15.1:0/4043524801 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e2ebf13a0 con 0x558e2ed0a370 2024-09-11T22:00:35.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.152+0000 7f3fb3fff640 1 --2- 172.21.15.1:0/4043524801 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e2ecff5d0 0x558e2ecffa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:00:35.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.153+0000 7f3fc0fc0640 1 --2- 172.21.15.1:0/4043524801 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558e2ed0a370 0x558e2ed0c760 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f3fb402f450 tx=0x7f3fb402f480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3978404625a76621 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:00:35.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.153+0000 7f3fb27fc640 1 -- 172.21.15.1:0/4043524801 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb4034050 con 0x558e2ed0a370 2024-09-11T22:00:35.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.153+0000 7f3fb27fc640 1 -- 172.21.15.1:0/4043524801 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fb4003e50 con 0x558e2ed0a370 2024-09-11T22:00:35.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.153+0000 7f3fb27fc640 1 -- 172.21.15.1:0/4043524801 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb40044e0 con 0x558e2ed0a370 2024-09-11T22:00:35.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.154+0000 7f3fc4dfc740 1 -- 172.21.15.1:0/4043524801 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558e2ed0a370 msgr2=0x558e2ed0c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:35.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.154+0000 7f3fc4dfc740 1 --2- 172.21.15.1:0/4043524801 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558e2ed0a370 0x558e2ed0c760 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f3fb402f450 tx=0x7f3fb402f480 comp rx=0 tx=0).stop 2024-09-11T22:00:35.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.154+0000 7f3fc4dfc740 1 -- 172.21.15.1:0/4043524801 shutdown_connections 2024-09-11T22:00:35.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.154+0000 7f3fc4dfc740 1 --2- 172.21.15.1:0/4043524801 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558e2ed0a370 0x558e2ed0c760 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:35.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.154+0000 7f3fc4dfc740 1 --2- 172.21.15.1:0/4043524801 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e2ecff5d0 0x558e2ecffa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:35.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.154+0000 7f3fc4dfc740 1 --2- 172.21.15.1:0/4043524801 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e2ed00b40 0x558e2ecff090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:35.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.154+0000 7f3fc4dfc740 1 -- 172.21.15.1:0/4043524801 >> 172.21.15.1:0/4043524801 conn(0x558e2ecc8310 msgr2=0x558e2ecfbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:00:35.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.154+0000 7f3fc4dfc740 1 -- 172.21.15.1:0/4043524801 shutdown_connections 2024-09-11T22:00:35.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.154+0000 7f3fc4dfc740 1 -- 172.21.15.1:0/4043524801 wait complete. 2024-09-11T22:00:35.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.154+0000 7f3fc4dfc740 1 -- start start 2024-09-11T22:00:35.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.155+0000 7f3fc4dfc740 1 Processor -- start 2024-09-11T22:00:35.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.155+0000 7f3fc4dfc740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e2ecff5d0 0x558e2ed09ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:35.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.155+0000 7f3fc4dfc740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e2ed00b40 0x558e2ed06f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:35.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.155+0000 7f3fc4dfc740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558e2ed0a370 0x558e2ed074f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:35.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.155+0000 7f3fc4dfc740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x558e2ed0d010 con 0x558e2ed0a370 2024-09-11T22:00:35.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.155+0000 7f3fc4dfc740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x558e2ec20060 con 0x558e2ecff5d0 2024-09-11T22:00:35.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.155+0000 7f3fc4dfc740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x558e2ed0d190 con 0x558e2ed00b40 2024-09-11T22:00:35.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.155+0000 7f3fc4dfc740 10 client.0 _ll_register_callbacks cb 0x558e2e9738c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:00:35.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.155+0000 7f3fc0fc0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558e2ed0a370 0x558e2ed074f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:35.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.155+0000 7f3fb3fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e2ed00b40 0x558e2ed06f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:35.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.155+0000 7f3fc0fc0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558e2ed0a370 0x558e2ed074f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55938/0 (socket says 172.21.15.1:55938) 2024-09-11T22:00:35.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.155+0000 7f3fc0fc0640 1 -- 172.21.15.1:0/3813762801 learned_addr learned my addr 172.21.15.1:0/3813762801 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:00:35.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.155+0000 7f3fbbfff640 1 --2- 172.21.15.1:0/3813762801 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e2ecff5d0 0x558e2ed09ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:35.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.156+0000 7f3fb3fff640 1 -- 172.21.15.1:0/3813762801 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e2ecff5d0 msgr2=0x558e2ed09ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:00:35.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.156+0000 7f3fb3fff640 1 --2- 172.21.15.1:0/3813762801 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e2ecff5d0 0x558e2ed09ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:35.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.156+0000 7f3fb3fff640 1 -- 172.21.15.1:0/3813762801 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558e2ed0a370 msgr2=0x558e2ed074f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:00:35.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.156+0000 7f3fb3fff640 1 --2- 172.21.15.1:0/3813762801 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558e2ed0a370 0x558e2ed074f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:35.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.156+0000 7f3fb3fff640 1 -- 172.21.15.1:0/3813762801 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e2e9d63c0 con 0x558e2ed00b40 2024-09-11T22:00:35.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.156+0000 7f3fc0fc0640 1 --2- 172.21.15.1:0/3813762801 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558e2ed0a370 0x558e2ed074f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:00:35.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.156+0000 7f3fb3fff640 1 --2- 172.21.15.1:0/3813762801 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e2ed00b40 0x558e2ed06f90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3fa40cc530 tx=0x7f3fa409ac60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab7b0d1689470917 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:00:35.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.156+0000 7f3f9b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-11T22:00:35.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.156+0000 7f3f9b7fe640 1 -- 172.21.15.1:0/3813762801 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa409f6b0 con 0x558e2ed00b40 2024-09-11T22:00:35.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.156+0000 7f3f9b7fe640 1 -- 172.21.15.1:0/3813762801 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f3fa40a3ae0 con 0x558e2ed00b40 2024-09-11T22:00:35.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.156+0000 7f3fc4dfc740 1 -- 172.21.15.1:0/3813762801 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3fb4003e50 con 0x558e2ed00b40 2024-09-11T22:00:35.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.156+0000 7f3f9b7fe640 1 -- 172.21.15.1:0/3813762801 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3fa40a3f70 con 0x558e2ed00b40 2024-09-11T22:00:35.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.156+0000 7f3fc4dfc740 10 client.25603 fetch_fsmap learned FSMap version 30 2024-09-11T22:00:35.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.156+0000 7f3fc4dfc740 1 -- 172.21.15.1:0/3813762801 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558e2ed18990 con 0x558e2ed00b40 2024-09-11T22:00:35.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.157+0000 7f3f9b7fe640 1 -- 172.21.15.1:0/3813762801 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fb4003e50 con 0x558e2ed00b40 2024-09-11T22:00:35.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.157+0000 7f3f9b7fe640 1 -- 172.21.15.1:0/3813762801 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f3fa409a8e0 con 0x558e2ed00b40 2024-09-11T22:00:35.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.157+0000 7f3fc4dfc740 10 client.25603 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:00:35.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.157+0000 7f3fc4dfc740 10 client.25603 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T22:00:35.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.157+0000 7f3fc4dfc740 20 client.25603 populate_metadata read hostname 'smithi001' 2024-09-11T22:00:35.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.157+0000 7f3fc4dfc740 1 --2- 172.21.15.1:0/3813762801 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x558e2ed19df0 0x558e2ed1c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:35.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.157+0000 7f3fc4dfc740 4 client.25603 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:00:35.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.157+0000 7f3fc4dfc740 1 -- 172.21.15.1:0/3813762801 --> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e2ec72860 con 0x558e2ed19df0 2024-09-11T22:00:35.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.157+0000 7f3fbbfff640 1 --2- 172.21.15.1:0/3813762801 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x558e2ed19df0 0x558e2ed1c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:35.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.158+0000 7f3f9b7fe640 10 client.25603 ms_handle_connect on v2:172.21.15.161:6838/455365383 2024-09-11T22:00:35.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.158+0000 7f3fbbfff640 1 --2- 172.21.15.1:0/3813762801 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x558e2ed19df0 0x558e2ed1c210 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a178310d2dda8793 server_cookie=4b3814a970ca0940 in_seq=0 out_seq=0 2024-09-11T22:00:35.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.159+0000 7f3f9b7fe640 1 -- 172.21.15.1:0/3813762801 <== mds.1 v2:172.21.15.161:6838/455365383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558e2ec72860 con 0x558e2ed19df0 2024-09-11T22:00:35.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.159+0000 7f3f9b7fe640 10 client.25603 handle_command_reply: tid=0 2024-09-11T22:00:35.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.159+0000 7f3fc4dfc740 1 client.25603 shutdown 2024-09-11T22:00:35.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.159+0000 7f3fc4dfc740 1 -- 172.21.15.1:0/3813762801 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e2ed00b40 msgr2=0x558e2ed06f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:35.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.159+0000 7f3fc4dfc740 1 --2- 172.21.15.1:0/3813762801 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e2ed00b40 0x558e2ed06f90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3fa40cc530 tx=0x7f3fa409ac60 comp rx=0 tx=0).stop 2024-09-11T22:00:35.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.159+0000 7f3fc4dfc740 1 -- 172.21.15.1:0/3813762801 shutdown_connections 2024-09-11T22:00:35.169 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:00:34 smithi001 ceph-mon[30935]: pgmap v105: 97 pgs: 97 active+clean; 3.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 17 op/s 2024-09-11T22:00:35.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.159+0000 7f3fc4dfc740 1 --2- 172.21.15.1:0/3813762801 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x558e2ed19df0 0x558e2ed1c210 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:35.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.159+0000 7f3fc4dfc740 1 --2- 172.21.15.1:0/3813762801 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558e2ed0a370 0x558e2ed074f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:35.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.159+0000 7f3fc4dfc740 1 --2- 172.21.15.1:0/3813762801 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e2ed00b40 0x558e2ed06f90 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:35.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.159+0000 7f3fc4dfc740 1 --2- 172.21.15.1:0/3813762801 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e2ecff5d0 0x558e2ed09ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:35.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.159+0000 7f3fc4dfc740 1 -- 172.21.15.1:0/3813762801 >> 172.21.15.1:0/3813762801 conn(0x558e2ecc8310 msgr2=0x558e2ed01960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:00:35.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.159+0000 7f3f9b7fe640 0 client.25603 ms_handle_reset on v2:172.21.15.161:6838/455365383 2024-09-11T22:00:35.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.159+0000 7f3fc4dfc740 1 -- 172.21.15.1:0/3813762801 shutdown_connections 2024-09-11T22:00:35.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.159+0000 7f3fc4dfc740 1 -- 172.21.15.1:0/3813762801 wait complete. 2024-09-11T22:00:35.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.159+0000 7f3fc4dfc740 20 client.25603 trim_cache size 0 max 16384 2024-09-11T22:00:35.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.173+0000 7f3fc4dfc740 1 Processor -- start 2024-09-11T22:00:35.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.173+0000 7f3fc4dfc740 1 -- start start 2024-09-11T22:00:35.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.173+0000 7f3fc4dfc740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e2ecff5d0 0x558e2ed162b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:35.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.173+0000 7f3fc4dfc740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e2ed00b40 0x558e2ed02310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:35.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.173+0000 7f3fc4dfc740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558e2ed0a370 0x558e2ed21ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:35.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.173+0000 7f3fc4dfc740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3fbc1ad980 con 0x558e2ed0a370 2024-09-11T22:00:35.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.173+0000 7f3fc4dfc740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3fbc1af520 con 0x558e2ecff5d0 2024-09-11T22:00:35.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.173+0000 7f3fc4dfc740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3fbc1ae050 con 0x558e2ed00b40 2024-09-11T22:00:35.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.173+0000 7f3fc0fc0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558e2ed0a370 0x558e2ed21ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:35.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.173+0000 7f3fc0fc0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558e2ed0a370 0x558e2ed21ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55940/0 (socket says 172.21.15.1:55940) 2024-09-11T22:00:35.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.174+0000 7f3fc0fc0640 1 -- 172.21.15.1:0/1531969409 learned_addr learned my addr 172.21.15.1:0/1531969409 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:00:35.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.174+0000 7f3fb3fff640 1 --2- 172.21.15.1:0/1531969409 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e2ed00b40 0x558e2ed02310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:35.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.174+0000 7f3fbbfff640 1 --2- 172.21.15.1:0/1531969409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e2ecff5d0 0x558e2ed162b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:35.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.174+0000 7f3fc0fc0640 1 -- 172.21.15.1:0/1531969409 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e2ed00b40 msgr2=0x558e2ed02310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:00:35.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.174+0000 7f3fc0fc0640 1 --2- 172.21.15.1:0/1531969409 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e2ed00b40 0x558e2ed02310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:35.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.174+0000 7f3fc0fc0640 1 -- 172.21.15.1:0/1531969409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e2ecff5d0 msgr2=0x558e2ed162b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:00:35.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.174+0000 7f3fc0fc0640 1 --2- 172.21.15.1:0/1531969409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e2ecff5d0 0x558e2ed162b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:35.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.174+0000 7f3fc0fc0640 1 -- 172.21.15.1:0/1531969409 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e2ed02120 con 0x558e2ed0a370 2024-09-11T22:00:35.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.174+0000 7f3fbbfff640 1 --2- 172.21.15.1:0/1531969409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e2ecff5d0 0x558e2ed162b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:00:35.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.174+0000 7f3fc0fc0640 1 --2- 172.21.15.1:0/1531969409 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558e2ed0a370 0x558e2ed21ac0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f3fb402f450 tx=0x7f3fb40040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce2e6de25baa0ef4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:00:35.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.174+0000 7f3fb1ffb640 1 -- 172.21.15.1:0/1531969409 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb4009800 con 0x558e2ed0a370 2024-09-11T22:00:35.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.174+0000 7f3fb1ffb640 1 -- 172.21.15.1:0/1531969409 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fb402fe20 con 0x558e2ed0a370 2024-09-11T22:00:35.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.174+0000 7f3fb1ffb640 1 -- 172.21.15.1:0/1531969409 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb40046b0 con 0x558e2ed0a370 2024-09-11T22:00:35.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.175+0000 7f3fc4dfc740 1 -- 172.21.15.1:0/1531969409 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558e2ed0a370 msgr2=0x558e2ed21ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:35.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.175+0000 7f3fc4dfc740 1 --2- 172.21.15.1:0/1531969409 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558e2ed0a370 0x558e2ed21ac0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f3fb402f450 tx=0x7f3fb40040e0 comp rx=0 tx=0).stop 2024-09-11T22:00:35.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.175+0000 7f3fc4dfc740 1 -- 172.21.15.1:0/1531969409 shutdown_connections 2024-09-11T22:00:35.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.175+0000 7f3fc4dfc740 1 --2- 172.21.15.1:0/1531969409 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558e2ed0a370 0x558e2ed21ac0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:35.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.175+0000 7f3fc4dfc740 1 --2- 172.21.15.1:0/1531969409 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e2ed00b40 0x558e2ed02310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:35.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.175+0000 7f3fc4dfc740 1 --2- 172.21.15.1:0/1531969409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e2ecff5d0 0x558e2ed162b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:35.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.175+0000 7f3fc4dfc740 1 -- 172.21.15.1:0/1531969409 >> 172.21.15.1:0/1531969409 conn(0x558e2ecc8310 msgr2=0x558e2ed05860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:00:35.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.175+0000 7f3fc4dfc740 1 -- 172.21.15.1:0/1531969409 shutdown_connections 2024-09-11T22:00:35.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.175+0000 7f3fc4dfc740 1 -- 172.21.15.1:0/1531969409 wait complete. 2024-09-11T22:00:35.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.176+0000 7f3fc4dfc740 1 -- start start 2024-09-11T22:00:35.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.176+0000 7f3fc4dfc740 1 Processor -- start 2024-09-11T22:00:35.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.176+0000 7f3fc4dfc740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e2ecff5d0 0x558e2ed04140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:35.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.176+0000 7f3fc4dfc740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e2ed00b40 0x558e2ed04680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:35.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.176+0000 7f3fc4dfc740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558e2ed0a370 0x558e2ed04bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:35.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.176+0000 7f3fc4dfc740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3fbc1aee20 con 0x558e2ed0a370 2024-09-11T22:00:35.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.176+0000 7f3fc4dfc740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3fbc1ae720 con 0x558e2ecff5d0 2024-09-11T22:00:35.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.176+0000 7f3fc4dfc740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3fbc1ab960 con 0x558e2ed00b40 2024-09-11T22:00:35.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.176+0000 7f3fc4dfc740 10 client.0 _ll_register_callbacks cb 0x7f3fa4005d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:00:35.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.176+0000 7f3fc0fc0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558e2ed0a370 0x558e2ed04bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:35.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.177+0000 7f3fc0fc0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558e2ed0a370 0x558e2ed04bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55944/0 (socket says 172.21.15.1:55944) 2024-09-11T22:00:35.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.177+0000 7f3fc0fc0640 1 -- 172.21.15.1:0/744635034 learned_addr learned my addr 172.21.15.1:0/744635034 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:00:35.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.177+0000 7f3fb3fff640 1 --2- 172.21.15.1:0/744635034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e2ed00b40 0x558e2ed04680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:35.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.177+0000 7f3fbbfff640 1 --2- 172.21.15.1:0/744635034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e2ecff5d0 0x558e2ed04140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:35.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.177+0000 7f3fc0fc0640 1 -- 172.21.15.1:0/744635034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e2ed00b40 msgr2=0x558e2ed04680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:00:35.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.177+0000 7f3fc0fc0640 1 --2- 172.21.15.1:0/744635034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e2ed00b40 0x558e2ed04680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:35.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.177+0000 7f3fc0fc0640 1 -- 172.21.15.1:0/744635034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e2ecff5d0 msgr2=0x558e2ed04140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:00:35.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.177+0000 7f3fc0fc0640 1 --2- 172.21.15.1:0/744635034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e2ecff5d0 0x558e2ed04140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:35.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.177+0000 7f3fc0fc0640 1 -- 172.21.15.1:0/744635034 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e2ed83c90 con 0x558e2ed0a370 2024-09-11T22:00:35.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.177+0000 7f3fbbfff640 1 --2- 172.21.15.1:0/744635034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e2ecff5d0 0x558e2ed04140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:00:35.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.177+0000 7f3fc0fc0640 1 --2- 172.21.15.1:0/744635034 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558e2ed0a370 0x558e2ed04bc0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f3fb402ae70 tx=0x7f3fb4005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=706f3279e6fac0d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:00:35.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.177+0000 7f3f9b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:00:35.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.177+0000 7f3f9b7fe640 1 -- 172.21.15.1:0/744635034 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb4009800 con 0x558e2ed0a370 2024-09-11T22:00:35.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.177+0000 7f3f9b7fe640 1 -- 172.21.15.1:0/744635034 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f3fb4004430 con 0x558e2ed0a370 2024-09-11T22:00:35.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.177+0000 7f3fc4dfc740 1 -- 172.21.15.1:0/744635034 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3fb402fe20 con 0x558e2ed0a370 2024-09-11T22:00:35.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.178+0000 7f3f9b7fe640 1 -- 172.21.15.1:0/744635034 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3fb4063050 con 0x558e2ed0a370 2024-09-11T22:00:35.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.178+0000 7f3fc4dfc740 10 client.16065 fetch_fsmap learned FSMap version 30 2024-09-11T22:00:35.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.178+0000 7f3fc4dfc740 1 -- 172.21.15.1:0/744635034 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558e2ed84f10 con 0x558e2ed0a370 2024-09-11T22:00:35.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.178+0000 7f3f9b7fe640 1 -- 172.21.15.1:0/744635034 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fb402fe20 con 0x558e2ed0a370 2024-09-11T22:00:35.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.178+0000 7f3f9b7fe640 1 -- 172.21.15.1:0/744635034 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f3fb4041070 con 0x558e2ed0a370 2024-09-11T22:00:35.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.178+0000 7f3fc4dfc740 10 client.16065 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:00:35.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.178+0000 7f3fc4dfc740 10 client.16065 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T22:00:35.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.178+0000 7f3fc4dfc740 20 client.16065 populate_metadata read hostname 'smithi001' 2024-09-11T22:00:35.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.178+0000 7f3fc4dfc740 1 --2- 172.21.15.1:0/744635034 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x558e2ed85190 0x558e2ed87580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:35.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.178+0000 7f3fc4dfc740 4 client.16065 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:00:35.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.178+0000 7f3fc4dfc740 1 -- 172.21.15.1:0/744635034 --> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558e2ec6cd70 con 0x558e2ed85190 2024-09-11T22:00:35.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.178+0000 7f3fbbfff640 1 --2- 172.21.15.1:0/744635034 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x558e2ed85190 0x558e2ed87580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:35.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.179+0000 7f3fbbfff640 1 --2- 172.21.15.1:0/744635034 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x558e2ed85190 0x558e2ed87580 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cd01dc11b629f455 server_cookie=7fdb3e9d266c7d6c in_seq=0 out_seq=0 2024-09-11T22:00:35.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.179+0000 7f3f9b7fe640 10 client.16065 ms_handle_connect on v2:172.21.15.161:6838/455365383 2024-09-11T22:00:35.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.179+0000 7f3f9b7fe640 1 -- 172.21.15.1:0/744635034 <== mds.1 v2:172.21.15.161:6838/455365383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558e2ec6cd70 con 0x558e2ed85190 2024-09-11T22:00:35.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.179+0000 7f3f9b7fe640 10 client.16065 handle_command_reply: tid=0 2024-09-11T22:00:35.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.179+0000 7f3fc4dfc740 1 client.16065 shutdown 2024-09-11T22:00:35.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.179+0000 7f3fc4dfc740 1 -- 172.21.15.1:0/744635034 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558e2ed0a370 msgr2=0x558e2ed04bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:35.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.179+0000 7f3fc4dfc740 1 --2- 172.21.15.1:0/744635034 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558e2ed0a370 0x558e2ed04bc0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f3fb402ae70 tx=0x7f3fb4005d80 comp rx=0 tx=0).stop 2024-09-11T22:00:35.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.179+0000 7f3fc4dfc740 1 -- 172.21.15.1:0/744635034 shutdown_connections 2024-09-11T22:00:35.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.179+0000 7f3fc4dfc740 1 --2- 172.21.15.1:0/744635034 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x558e2ed85190 0x558e2ed87580 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:35.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.179+0000 7f3fc4dfc740 1 --2- 172.21.15.1:0/744635034 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558e2ed0a370 0x558e2ed04bc0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:35.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.179+0000 7f3f9b7fe640 0 client.16065 ms_handle_reset on v2:172.21.15.161:6838/455365383 2024-09-11T22:00:35.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.179+0000 7f3fc4dfc740 1 --2- 172.21.15.1:0/744635034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558e2ed00b40 0x558e2ed04680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:35.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.179+0000 7f3fc4dfc740 1 --2- 172.21.15.1:0/744635034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558e2ecff5d0 0x558e2ed04140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:35.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.179+0000 7f3fc4dfc740 1 -- 172.21.15.1:0/744635034 >> 172.21.15.1:0/744635034 conn(0x558e2ecc8310 msgr2=0x558e2ed01110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:00:35.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.179+0000 7f3fc4dfc740 1 -- 172.21.15.1:0/744635034 shutdown_connections 2024-09-11T22:00:35.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.180+0000 7f3fc4dfc740 1 -- 172.21.15.1:0/744635034 wait complete. 2024-09-11T22:00:35.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.180+0000 7f3fc4dfc740 20 client.16065 trim_cache size 0 max 16384 2024-09-11T22:00:35.186 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-11T22:00:35.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.180+0000 7f3fc2a23640 1 -- 172.21.15.1:0/2212610175 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f3fa80a2cf0 msgr2=0x7f3fa80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:35.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.180+0000 7f3fc2a23640 1 --2- 172.21.15.1:0/2212610175 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f3fa80a2cf0 0x7f3fa80a51b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3fac0058c0 tx=0x7f3fac005ec0 comp rx=0 tx=0).stop 2024-09-11T22:00:35.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.180+0000 7f3fc2a23640 1 -- 172.21.15.1:0/2212610175 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3fbc103660 msgr2=0x7f3fbc198900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:35.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.180+0000 7f3fc2a23640 1 --2- 172.21.15.1:0/2212610175 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3fbc103660 0x7f3fbc198900 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f3fa400d950 tx=0x7f3fa400de10 comp rx=0 tx=0).stop 2024-09-11T22:00:35.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.180+0000 7f3fc2a23640 1 -- 172.21.15.1:0/2212610175 shutdown_connections 2024-09-11T22:00:35.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.180+0000 7f3fc2a23640 1 --2- 172.21.15.1:0/2212610175 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f3fa80a2cf0 0x7f3fa80a51b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:35.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.180+0000 7f3fc2a23640 1 --2- 172.21.15.1:0/2212610175 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3fbc10c170 0x7f3fbc19cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:35.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.180+0000 7f3fc2a23640 1 --2- 172.21.15.1:0/2212610175 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3fbc103660 0x7f3fbc198900 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:35.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.180+0000 7f3fc2a23640 1 --2- 172.21.15.1:0/2212610175 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3fbc102d40 0x7f3fbc1983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:35.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.180+0000 7f3fc2a23640 1 -- 172.21.15.1:0/2212610175 >> 172.21.15.1:0/2212610175 conn(0x7f3fbc0fc7a0 msgr2=0x7f3fbc0fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:00:35.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.180+0000 7f3fc2a23640 1 -- 172.21.15.1:0/2212610175 shutdown_connections 2024-09-11T22:00:35.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:35.180+0000 7f3fc2a23640 1 -- 172.21.15.1:0/2212610175 wait complete. 2024-09-11T22:00:35.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:00:34 smithi053 ceph-mon[34908]: pgmap v105: 97 pgs: 97 active+clean; 3.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 17 op/s 2024-09-11T22:00:35.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:00:34 smithi161 ceph-mon[35615]: pgmap v105: 97 pgs: 97 active+clean; 3.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 17 op/s 2024-09-11T22:00:36.220 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:36.220 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:36.835 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T22:00:36.835 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T22:00:36.843 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:36.843 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:36.843 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 78 -lt 100 ']' 2024-09-11T22:00:36.844 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 78 + 1 2024-09-11T22:00:36.886 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=79 2024-09-11T22:00:36.886 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T22:00:36.920 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:36.920 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:36.921 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T22:00:36.922 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T22:00:36.922 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T22:00:37.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:00:36 smithi001 ceph-mon[30935]: pgmap v106: 97 pgs: 97 active+clean; 3.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 22 op/s 2024-09-11T22:00:37.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:00:36 smithi053 ceph-mon[34908]: pgmap v106: 97 pgs: 97 active+clean; 3.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 22 op/s 2024-09-11T22:00:37.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:00:36 smithi161 ceph-mon[35615]: pgmap v106: 97 pgs: 97 active+clean; 3.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 22 op/s 2024-09-11T22:00:38.415 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:38.416 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:38.849 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T22:00:38.849 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T22:00:38.894 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:38.894 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:38.894 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 79 -lt 100 ']' 2024-09-11T22:00:38.894 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 79 + 1 2024-09-11T22:00:39.000 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=80 2024-09-11T22:00:39.000 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T22:00:39.111 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:39.112 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:39.112 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T22:00:39.112 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T22:00:39.113 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T22:00:39.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:00:38 smithi001 ceph-mon[30935]: pgmap v107: 97 pgs: 97 active+clean; 3.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-09-11T22:00:39.307 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:00:38 smithi053 ceph-mon[34908]: pgmap v107: 97 pgs: 97 active+clean; 3.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-09-11T22:00:39.348 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:00:38 smithi161 ceph-mon[35615]: pgmap v107: 97 pgs: 97 active+clean; 3.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-09-11T22:00:39.949 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:39.947+0000 7f2119623640 1 -- 172.21.15.1:0/4183112326 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f21140745a0 msgr2=0x7f2114074980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:39.949 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:39.947+0000 7f2119623640 1 --2- 172.21.15.1:0/4183112326 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f21140745a0 0x7f2114074980 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f2104009ca0 tx=0x7f210402a950 comp rx=0 tx=0).stop 2024-09-11T22:00:39.949 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:39.947+0000 7f2119623640 1 -- 172.21.15.1:0/4183112326 shutdown_connections 2024-09-11T22:00:39.949 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:39.947+0000 7f2119623640 1 --2- 172.21.15.1:0/4183112326 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f211410e380 0x7f2114110770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:39.950 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:39.947+0000 7f2119623640 1 --2- 172.21.15.1:0/4183112326 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2114074ec0 0x7f211410de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:39.950 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:39.947+0000 7f2119623640 1 --2- 172.21.15.1:0/4183112326 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f21140745a0 0x7f2114074980 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:39.950 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:39.947+0000 7f2119623640 1 -- 172.21.15.1:0/4183112326 >> 172.21.15.1:0/4183112326 conn(0x7f21140fd7a0 msgr2=0x7f21140ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:00:39.950 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:39.947+0000 7f2119623640 1 -- 172.21.15.1:0/4183112326 shutdown_connections 2024-09-11T22:00:39.950 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:39.947+0000 7f2119623640 1 -- 172.21.15.1:0/4183112326 wait complete. 2024-09-11T22:00:39.980 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:39.978+0000 7f2119623640 1 Processor -- start 2024-09-11T22:00:39.981 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:39.979+0000 7f2119623640 1 -- start start 2024-09-11T22:00:39.981 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:39.979+0000 7f2119623640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f21140745a0 0x7f211419a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:39.981 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:39.979+0000 7f2119623640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2114074ec0 0x7f211419ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:39.981 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:39.979+0000 7f2119623640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f211410e380 0x7f211419ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:39.981 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:39.979+0000 7f2119623640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f21141140e0 con 0x7f21140745a0 2024-09-11T22:00:39.981 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:39.979+0000 7f2119623640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2114113f60 con 0x7f211410e380 2024-09-11T22:00:39.981 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:39.979+0000 7f2119623640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2114114260 con 0x7f2114074ec0 2024-09-11T22:00:39.981 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:39.979+0000 7f2112ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f21140745a0 0x7f211419a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:39.981 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:39.979+0000 7f2112ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f21140745a0 0x7f211419a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52450/0 (socket says 172.21.15.1:52450) 2024-09-11T22:00:39.982 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:39.979+0000 7f2112ffd640 1 -- 172.21.15.1:0/2763318311 learned_addr learned my addr 172.21.15.1:0/2763318311 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:00:39.982 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:39.979+0000 7f21127fc640 1 --2- 172.21.15.1:0/2763318311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2114074ec0 0x7f211419ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:39.982 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:39.980+0000 7f21137fe640 1 --2- 172.21.15.1:0/2763318311 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f211410e380 0x7f211419ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:39.982 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:39.980+0000 7f2112ffd640 1 -- 172.21.15.1:0/2763318311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2114074ec0 msgr2=0x7f211419ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:39.982 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:39.980+0000 7f2112ffd640 1 --2- 172.21.15.1:0/2763318311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2114074ec0 0x7f211419ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:39.983 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:39.980+0000 7f2112ffd640 1 -- 172.21.15.1:0/2763318311 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f211410e380 msgr2=0x7f211419ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:39.983 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:39.980+0000 7f2112ffd640 1 --2- 172.21.15.1:0/2763318311 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f211410e380 0x7f211419ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:39.983 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:39.980+0000 7f2112ffd640 1 -- 172.21.15.1:0/2763318311 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f211419f6c0 con 0x7f21140745a0 2024-09-11T22:00:39.983 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:39.980+0000 7f2112ffd640 1 --2- 172.21.15.1:0/2763318311 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f21140745a0 0x7f211419a6d0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f2104005e30 tx=0x7f2104003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:00:39.994 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:39.993+0000 7f20fbfff640 1 -- 172.21.15.1:0/2763318311 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2104034040 con 0x7f21140745a0 2024-09-11T22:00:39.994 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:39.993+0000 7f20fbfff640 1 -- 172.21.15.1:0/2763318311 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f210402fe20 con 0x7f21140745a0 2024-09-11T22:00:39.994 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:39.993+0000 7f20fbfff640 1 -- 172.21.15.1:0/2763318311 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2104004800 con 0x7f21140745a0 2024-09-11T22:00:39.995 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:39.993+0000 7f2119623640 1 -- 172.21.15.1:0/2763318311 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f211419f950 con 0x7f21140745a0 2024-09-11T22:00:39.995 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:39.993+0000 7f2119623640 1 -- 172.21.15.1:0/2763318311 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21141a7240 con 0x7f21140745a0 2024-09-11T22:00:39.997 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:39.996+0000 7f20fbfff640 1 -- 172.21.15.1:0/2763318311 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f2104037020 con 0x7f21140745a0 2024-09-11T22:00:39.998 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:39.996+0000 7f20fbfff640 1 --2- 172.21.15.1:0/2763318311 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f20fc0a2e40 0x7f20fc0a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:39.998 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:39.997+0000 7f20fbfff640 1 -- 172.21.15.1:0/2763318311 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f21040f00b0 con 0x7f21140745a0 2024-09-11T22:00:39.998 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:39.997+0000 7f21127fc640 1 --2- 172.21.15.1:0/2763318311 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f20fc0a2e40 0x7f20fc0a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:39.999 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:39.997+0000 7f21127fc640 1 --2- 172.21.15.1:0/2763318311 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f20fc0a2e40 0x7f20fc0a5300 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f211410f770 tx=0x7f210800b410 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:00:40.358 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.353+0000 7f211c3bc740 1 Processor -- start 2024-09-11T22:00:40.358 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.353+0000 7f211c3bc740 1 -- start start 2024-09-11T22:00:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.353+0000 7f211c3bc740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645d35e0af0 0x5645d35e0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.353+0000 7f211c3bc740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645d35e1410 0x5645d35eb770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.353+0000 7f211c3bc740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5645d35ebcb0 0x5645d35ee0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.353+0000 7f211c3bc740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5645d35452f0 con 0x5645d35ebcb0 2024-09-11T22:00:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.353+0000 7f211c3bc740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5645d34ed270 con 0x5645d35e1410 2024-09-11T22:00:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.353+0000 7f211c3bc740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5645d35412b0 con 0x5645d35e0af0 2024-09-11T22:00:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.354+0000 7f21137fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5645d35ebcb0 0x5645d35ee0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.354+0000 7f21137fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5645d35ebcb0 0x5645d35ee0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52466/0 (socket says 172.21.15.1:52466) 2024-09-11T22:00:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.354+0000 7f21137fe640 1 -- 172.21.15.1:0/366566797 learned_addr learned my addr 172.21.15.1:0/366566797 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:00:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.354+0000 7f2112ffd640 1 --2- 172.21.15.1:0/366566797 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645d35e0af0 0x5645d35e0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.354+0000 7f21127fc640 1 --2- 172.21.15.1:0/366566797 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645d35e1410 0x5645d35eb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.354+0000 7f21137fe640 1 -- 172.21.15.1:0/366566797 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645d35e0af0 msgr2=0x5645d35e0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:00:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.354+0000 7f21137fe640 1 --2- 172.21.15.1:0/366566797 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645d35e0af0 0x5645d35e0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.354+0000 7f21137fe640 1 -- 172.21.15.1:0/366566797 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645d35e1410 msgr2=0x5645d35eb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:00:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.354+0000 7f21137fe640 1 --2- 172.21.15.1:0/366566797 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645d35e1410 0x5645d35eb770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.354+0000 7f21137fe640 1 -- 172.21.15.1:0/366566797 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645d34ec340 con 0x5645d35ebcb0 2024-09-11T22:00:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.354+0000 7f21127fc640 1 --2- 172.21.15.1:0/366566797 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645d35e1410 0x5645d35eb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:00:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.354+0000 7f2112ffd640 1 --2- 172.21.15.1:0/366566797 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645d35e0af0 0x5645d35e0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:00:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.355+0000 7f21137fe640 1 --2- 172.21.15.1:0/366566797 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5645d35ebcb0 0x5645d35ee0a0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f210c009c20 tx=0x7f210c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13873a1857d1aa44 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:00:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.355+0000 7f20f97fa640 1 -- 172.21.15.1:0/366566797 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f210c01e070 con 0x5645d35ebcb0 2024-09-11T22:00:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.355+0000 7f20f97fa640 1 -- 172.21.15.1:0/366566797 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f210c019910 con 0x5645d35ebcb0 2024-09-11T22:00:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.355+0000 7f20f97fa640 1 -- 172.21.15.1:0/366566797 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f210c00e8c0 con 0x5645d35ebcb0 2024-09-11T22:00:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.356+0000 7f211c3bc740 1 -- 172.21.15.1:0/366566797 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5645d35ebcb0 msgr2=0x5645d35ee0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.356+0000 7f211c3bc740 1 --2- 172.21.15.1:0/366566797 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5645d35ebcb0 0x5645d35ee0a0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f210c009c20 tx=0x7f210c011810 comp rx=0 tx=0).stop 2024-09-11T22:00:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.357+0000 7f211c3bc740 1 -- 172.21.15.1:0/366566797 shutdown_connections 2024-09-11T22:00:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.357+0000 7f211c3bc740 1 --2- 172.21.15.1:0/366566797 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5645d35ebcb0 0x5645d35ee0a0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.357+0000 7f211c3bc740 1 --2- 172.21.15.1:0/366566797 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645d35e1410 0x5645d35eb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.357+0000 7f211c3bc740 1 --2- 172.21.15.1:0/366566797 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645d35e0af0 0x5645d35e0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.357+0000 7f211c3bc740 1 -- 172.21.15.1:0/366566797 >> 172.21.15.1:0/366566797 conn(0x5645d35dd060 msgr2=0x5645d35dd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:00:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.357+0000 7f211c3bc740 1 -- 172.21.15.1:0/366566797 shutdown_connections 2024-09-11T22:00:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.357+0000 7f211c3bc740 1 -- 172.21.15.1:0/366566797 wait complete. 2024-09-11T22:00:40.402 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T22:00:40.522 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.521+0000 7f211c3bc740 1 -- start start 2024-09-11T22:00:40.522 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.521+0000 7f211c3bc740 1 Processor -- start 2024-09-11T22:00:40.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.523+0000 7f211c3bc740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645d35e0af0 0x5645d35e8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:40.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.523+0000 7f211c3bc740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645d35e1410 0x5645d35e9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:40.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.523+0000 7f211c3bc740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5645d35ebcb0 0x5645d35e9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:40.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.523+0000 7f211c3bc740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5645d35ee880 con 0x5645d35ebcb0 2024-09-11T22:00:40.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.523+0000 7f211c3bc740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5645d34ce290 con 0x5645d35e0af0 2024-09-11T22:00:40.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.523+0000 7f211c3bc740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5645d35eea00 con 0x5645d35e1410 2024-09-11T22:00:40.526 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.523+0000 7f21137fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5645d35ebcb0 0x5645d35e9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.523+0000 7f21137fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5645d35ebcb0 0x5645d35e9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52472/0 (socket says 172.21.15.1:52472) 2024-09-11T22:00:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.523+0000 7f21137fe640 1 -- 172.21.15.1:0/657078716 learned_addr learned my addr 172.21.15.1:0/657078716 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:00:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.523+0000 7f211c3bc740 10 client.0 _ll_register_callbacks cb 0x5645d34ff140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:00:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.524+0000 7f2112ffd640 1 --2- 172.21.15.1:0/657078716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645d35e0af0 0x5645d35e8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.524+0000 7f21127fc640 1 --2- 172.21.15.1:0/657078716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645d35e1410 0x5645d35e9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.524+0000 7f21137fe640 1 -- 172.21.15.1:0/657078716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645d35e1410 msgr2=0x5645d35e9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:00:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.524+0000 7f21137fe640 1 --2- 172.21.15.1:0/657078716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645d35e1410 0x5645d35e9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.524+0000 7f21137fe640 1 -- 172.21.15.1:0/657078716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645d35e0af0 msgr2=0x5645d35e8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:00:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.524+0000 7f21137fe640 1 --2- 172.21.15.1:0/657078716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645d35e0af0 0x5645d35e8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.524+0000 7f21137fe640 1 -- 172.21.15.1:0/657078716 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645d3258210 con 0x5645d35ebcb0 2024-09-11T22:00:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.524+0000 7f21137fe640 1 --2- 172.21.15.1:0/657078716 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5645d35ebcb0 0x5645d35e9670 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f210c009870 tx=0x7f210c00e3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85fb4df0d388edfe server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:00:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.524+0000 7f20f6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:00:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.524+0000 7f211c3bc740 1 -- 172.21.15.1:0/657078716 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f210c019910 con 0x5645d35ebcb0 2024-09-11T22:00:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.524+0000 7f20f6ff5640 1 -- 172.21.15.1:0/657078716 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f210c027070 con 0x5645d35ebcb0 2024-09-11T22:00:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.524+0000 7f20f6ff5640 1 -- 172.21.15.1:0/657078716 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f210c019cd0 con 0x5645d35ebcb0 2024-09-11T22:00:40.553 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.553+0000 7f20f6ff5640 1 -- 172.21.15.1:0/657078716 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f210c01e040 con 0x5645d35ebcb0 2024-09-11T22:00:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.558+0000 7f211c3bc740 10 client.16077 fetch_fsmap learned FSMap version 30 2024-09-11T22:00:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.558+0000 7f211c3bc740 1 -- 172.21.15.1:0/657078716 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({fsmap=30}) -- 0x5645d3324620 con 0x5645d35ebcb0 2024-09-11T22:00:40.669 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.667+0000 7f20f6ff5640 1 -- 172.21.15.1:0/657078716 <== mon.0 v2:172.21.15.1:3300/0 4 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f210c005040 con 0x5645d35ebcb0 2024-09-11T22:00:40.676 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.674+0000 7f211c3bc740 10 client.16077 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:00:40.676 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.674+0000 7f211c3bc740 10 client.16077 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:00:40.677 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.675+0000 7f211c3bc740 20 client.16077 populate_metadata read hostname 'smithi001' 2024-09-11T22:00:40.677 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.675+0000 7f211c3bc740 1 --2- 172.21.15.1:0/657078716 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5645d35fba90 0x5645d35fde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:40.677 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.675+0000 7f211c3bc740 4 client.16077 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:00:40.677 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.675+0000 7f211c3bc740 1 -- 172.21.15.1:0/657078716 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5645d3552a20 con 0x5645d35fba90 2024-09-11T22:00:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.676+0000 7f2112ffd640 1 --2- 172.21.15.1:0/657078716 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5645d35fba90 0x5645d35fde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.676+0000 7f2112ffd640 1 --2- 172.21.15.1:0/657078716 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5645d35fba90 0x5645d35fde80 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8b0bdfc34529fa9 server_cookie=f4eb7ab2cbdede1b in_seq=0 out_seq=0 2024-09-11T22:00:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.676+0000 7f20f6ff5640 10 client.16077 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:00:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.677+0000 7f20f6ff5640 1 -- 172.21.15.1:0/657078716 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5645d3552a20 con 0x5645d35fba90 2024-09-11T22:00:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.677+0000 7f20f6ff5640 10 client.16077 handle_command_reply: tid=0 2024-09-11T22:00:40.679 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.678+0000 7f211c3bc740 1 client.16077 shutdown 2024-09-11T22:00:40.680 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.678+0000 7f211c3bc740 1 -- 172.21.15.1:0/657078716 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5645d35ebcb0 msgr2=0x5645d35e9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:40.680 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.678+0000 7f211c3bc740 1 --2- 172.21.15.1:0/657078716 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5645d35ebcb0 0x5645d35e9670 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f210c009870 tx=0x7f210c00e3b0 comp rx=0 tx=0).stop 2024-09-11T22:00:40.680 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.678+0000 7f211c3bc740 1 -- 172.21.15.1:0/657078716 shutdown_connections 2024-09-11T22:00:40.681 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.678+0000 7f211c3bc740 1 --2- 172.21.15.1:0/657078716 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5645d35fba90 0x5645d35fde80 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:40.681 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.678+0000 7f211c3bc740 1 --2- 172.21.15.1:0/657078716 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5645d35ebcb0 0x5645d35e9670 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:40.681 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.678+0000 7f20f6ff5640 0 client.16077 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:00:40.681 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.678+0000 7f211c3bc740 1 --2- 172.21.15.1:0/657078716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645d35e1410 0x5645d35e9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:40.681 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.678+0000 7f211c3bc740 1 --2- 172.21.15.1:0/657078716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645d35e0af0 0x5645d35e8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:40.681 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.678+0000 7f211c3bc740 1 -- 172.21.15.1:0/657078716 >> 172.21.15.1:0/657078716 conn(0x5645d35dd060 msgr2=0x5645d35e2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:00:40.681 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.678+0000 7f211c3bc740 1 -- 172.21.15.1:0/657078716 shutdown_connections 2024-09-11T22:00:40.681 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.678+0000 7f211c3bc740 1 -- 172.21.15.1:0/657078716 wait complete. 2024-09-11T22:00:40.681 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.678+0000 7f211c3bc740 20 client.16077 trim_cache size 0 max 16384 2024-09-11T22:00:40.708 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.705+0000 7f211c3bc740 1 Processor -- start 2024-09-11T22:00:40.708 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.705+0000 7f211c3bc740 1 -- start start 2024-09-11T22:00:40.708 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.705+0000 7f211c3bc740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5645d35e0af0 0x5645d35ea450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:40.709 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.705+0000 7f211c3bc740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645d35e1410 0x5645d35e94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:40.709 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.705+0000 7f211c3bc740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645d35ebcb0 0x5645d3603650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:40.709 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.705+0000 7f211c3bc740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f21141afb10 con 0x5645d35e0af0 2024-09-11T22:00:40.709 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.705+0000 7f211c3bc740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f21141b16b0 con 0x5645d35e1410 2024-09-11T22:00:40.709 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.705+0000 7f211c3bc740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f21141b01e0 con 0x5645d35ebcb0 2024-09-11T22:00:40.709 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.706+0000 7f2112ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5645d35e0af0 0x5645d35ea450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:40.709 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.706+0000 7f2112ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5645d35e0af0 0x5645d35ea450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52488/0 (socket says 172.21.15.1:52488) 2024-09-11T22:00:40.709 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.706+0000 7f2112ffd640 1 -- 172.21.15.1:0/2057252708 learned_addr learned my addr 172.21.15.1:0/2057252708 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:00:40.709 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.706+0000 7f21127fc640 1 --2- 172.21.15.1:0/2057252708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645d35e1410 0x5645d35e94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:40.709 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.706+0000 7f21137fe640 1 --2- 172.21.15.1:0/2057252708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645d35ebcb0 0x5645d3603650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:40.709 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.706+0000 7f2112ffd640 1 -- 172.21.15.1:0/2057252708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645d35ebcb0 msgr2=0x5645d3603650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:00:40.709 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.706+0000 7f2112ffd640 1 --2- 172.21.15.1:0/2057252708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645d35ebcb0 0x5645d3603650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.706+0000 7f2112ffd640 1 -- 172.21.15.1:0/2057252708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645d35e1410 msgr2=0x5645d35e94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:00:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.706+0000 7f2112ffd640 1 --2- 172.21.15.1:0/2057252708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645d35e1410 0x5645d35e94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.706+0000 7f2112ffd640 1 -- 172.21.15.1:0/2057252708 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645d35d82d0 con 0x5645d35e0af0 2024-09-11T22:00:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.706+0000 7f2112ffd640 1 --2- 172.21.15.1:0/2057252708 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5645d35e0af0 0x5645d35ea450 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f21040b9070 tx=0x7f21040bf040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cddd7bd5d86b48a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:00:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.706+0000 7f20f8ff9640 1 -- 172.21.15.1:0/2057252708 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21040d4070 con 0x5645d35e0af0 2024-09-11T22:00:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.706+0000 7f20f8ff9640 1 -- 172.21.15.1:0/2057252708 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21040c0a50 con 0x5645d35e0af0 2024-09-11T22:00:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.706+0000 7f20f8ff9640 1 -- 172.21.15.1:0/2057252708 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f210402c040 con 0x5645d35e0af0 2024-09-11T22:00:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.707+0000 7f211c3bc740 1 -- 172.21.15.1:0/2057252708 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5645d35e0af0 msgr2=0x5645d35ea450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.707+0000 7f211c3bc740 1 --2- 172.21.15.1:0/2057252708 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5645d35e0af0 0x5645d35ea450 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f21040b9070 tx=0x7f21040bf040 comp rx=0 tx=0).stop 2024-09-11T22:00:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.707+0000 7f211c3bc740 1 -- 172.21.15.1:0/2057252708 shutdown_connections 2024-09-11T22:00:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.707+0000 7f211c3bc740 1 --2- 172.21.15.1:0/2057252708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645d35ebcb0 0x5645d3603650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.707+0000 7f211c3bc740 1 --2- 172.21.15.1:0/2057252708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645d35e1410 0x5645d35e94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.707+0000 7f211c3bc740 1 --2- 172.21.15.1:0/2057252708 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5645d35e0af0 0x5645d35ea450 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.707+0000 7f211c3bc740 1 -- 172.21.15.1:0/2057252708 >> 172.21.15.1:0/2057252708 conn(0x5645d35dd060 msgr2=0x5645d35dc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:00:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.707+0000 7f211c3bc740 1 -- 172.21.15.1:0/2057252708 shutdown_connections 2024-09-11T22:00:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.707+0000 7f211c3bc740 1 -- 172.21.15.1:0/2057252708 wait complete. 2024-09-11T22:00:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.708+0000 7f211c3bc740 1 -- start start 2024-09-11T22:00:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.708+0000 7f211c3bc740 1 Processor -- start 2024-09-11T22:00:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.708+0000 7f211c3bc740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645d35e0af0 0x5645d35f3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.708+0000 7f211c3bc740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5645d35e1410 0x5645d35f4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.708+0000 7f211c3bc740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645d35ebcb0 0x5645d35f0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.708+0000 7f211c3bc740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f21141b0fb0 con 0x5645d35e1410 2024-09-11T22:00:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.708+0000 7f211c3bc740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f21141b08b0 con 0x5645d35e0af0 2024-09-11T22:00:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.708+0000 7f211c3bc740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f21141adaf0 con 0x5645d35ebcb0 2024-09-11T22:00:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.708+0000 7f211c3bc740 10 client.0 _ll_register_callbacks cb 0x7f210c01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:00:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.708+0000 7f21127fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5645d35e1410 0x5645d35f4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.708+0000 7f21127fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5645d35e1410 0x5645d35f4200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52498/0 (socket says 172.21.15.1:52498) 2024-09-11T22:00:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.708+0000 7f21127fc640 1 -- 172.21.15.1:0/1228772255 learned_addr learned my addr 172.21.15.1:0/1228772255 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:00:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.708+0000 7f2112ffd640 1 --2- 172.21.15.1:0/1228772255 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645d35e0af0 0x5645d35f3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.708+0000 7f21137fe640 1 --2- 172.21.15.1:0/1228772255 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645d35ebcb0 0x5645d35f0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.708+0000 7f21127fc640 1 -- 172.21.15.1:0/1228772255 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645d35ebcb0 msgr2=0x5645d35f0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:00:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.708+0000 7f21127fc640 1 --2- 172.21.15.1:0/1228772255 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645d35ebcb0 0x5645d35f0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.709+0000 7f21127fc640 1 -- 172.21.15.1:0/1228772255 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645d35e0af0 msgr2=0x5645d35f3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:00:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.709+0000 7f21127fc640 1 --2- 172.21.15.1:0/1228772255 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645d35e0af0 0x5645d35f3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.709+0000 7f21127fc640 1 -- 172.21.15.1:0/1228772255 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645d35f11d0 con 0x5645d35e1410 2024-09-11T22:00:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.709+0000 7f2112ffd640 1 --2- 172.21.15.1:0/1228772255 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645d35e0af0 0x5645d35f3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:00:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.709+0000 7f21127fc640 1 --2- 172.21.15.1:0/1228772255 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5645d35e1410 0x5645d35f4200 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f2108005d00 tx=0x7f2108004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5db23021c424b689 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:00:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.709+0000 7f20f6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:00:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.709+0000 7f20f6ff5640 1 -- 172.21.15.1:0/1228772255 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f210802e060 con 0x5645d35e1410 2024-09-11T22:00:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.709+0000 7f211c3bc740 1 -- 172.21.15.1:0/1228772255 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21040c0a50 con 0x5645d35e1410 2024-09-11T22:00:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.709+0000 7f20f6ff5640 1 -- 172.21.15.1:0/1228772255 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f210801fd10 con 0x5645d35e1410 2024-09-11T22:00:40.717 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.709+0000 7f20f6ff5640 1 -- 172.21.15.1:0/1228772255 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f21080313f0 con 0x5645d35e1410 2024-09-11T22:00:40.717 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.709+0000 7f211c3bc740 10 client.16083 fetch_fsmap learned FSMap version 30 2024-09-11T22:00:40.717 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.709+0000 7f211c3bc740 1 -- 172.21.15.1:0/1228772255 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5645d3666e70 con 0x5645d35e1410 2024-09-11T22:00:40.717 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.709+0000 7f20f6ff5640 1 -- 172.21.15.1:0/1228772255 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21040c0a50 con 0x5645d35e1410 2024-09-11T22:00:40.717 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.710+0000 7f20f6ff5640 1 -- 172.21.15.1:0/1228772255 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f2108028070 con 0x5645d35e1410 2024-09-11T22:00:40.717 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.710+0000 7f211c3bc740 10 client.16083 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:00:40.717 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.710+0000 7f211c3bc740 10 client.16083 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:00:40.717 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.710+0000 7f211c3bc740 20 client.16083 populate_metadata read hostname 'smithi001' 2024-09-11T22:00:40.717 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.710+0000 7f211c3bc740 1 --2- 172.21.15.1:0/1228772255 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5645d3667600 0x5645d3669a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:40.717 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.710+0000 7f211c3bc740 4 client.16083 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:00:40.717 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.710+0000 7f211c3bc740 1 -- 172.21.15.1:0/1228772255 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5645d354cd70 con 0x5645d3667600 2024-09-11T22:00:40.717 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.710+0000 7f2112ffd640 1 --2- 172.21.15.1:0/1228772255 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5645d3667600 0x5645d3669a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:40.718 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.710+0000 7f2112ffd640 1 --2- 172.21.15.1:0/1228772255 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5645d3667600 0x5645d3669a20 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53ce6b5bda9c7928 server_cookie=f5028bce204aa100 in_seq=0 out_seq=0 2024-09-11T22:00:40.718 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.710+0000 7f20f6ff5640 10 client.16083 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:00:40.729 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.728+0000 7f20f6ff5640 1 -- 172.21.15.1:0/1228772255 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5645d354cd70 con 0x5645d3667600 2024-09-11T22:00:40.729 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.728+0000 7f20f6ff5640 10 client.16083 handle_command_reply: tid=0 2024-09-11T22:00:40.729 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.728+0000 7f211c3bc740 1 client.16083 shutdown 2024-09-11T22:00:40.729 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.728+0000 7f211c3bc740 1 -- 172.21.15.1:0/1228772255 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5645d35e1410 msgr2=0x5645d35f4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:40.729 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.728+0000 7f211c3bc740 1 --2- 172.21.15.1:0/1228772255 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5645d35e1410 0x5645d35f4200 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f2108005d00 tx=0x7f2108004730 comp rx=0 tx=0).stop 2024-09-11T22:00:40.730 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.728+0000 7f211c3bc740 1 -- 172.21.15.1:0/1228772255 shutdown_connections 2024-09-11T22:00:40.730 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.728+0000 7f211c3bc740 1 --2- 172.21.15.1:0/1228772255 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5645d3667600 0x5645d3669a20 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:40.730 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.728+0000 7f211c3bc740 1 --2- 172.21.15.1:0/1228772255 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5645d35ebcb0 0x5645d35f0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:40.730 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.728+0000 7f211c3bc740 1 --2- 172.21.15.1:0/1228772255 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5645d35e1410 0x5645d35f4200 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:40.730 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.728+0000 7f20f6ff5640 0 client.16083 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:00:40.730 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.728+0000 7f211c3bc740 1 --2- 172.21.15.1:0/1228772255 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5645d35e0af0 0x5645d35f3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:40.730 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.728+0000 7f211c3bc740 1 -- 172.21.15.1:0/1228772255 >> 172.21.15.1:0/1228772255 conn(0x5645d35dd060 msgr2=0x5645d35f6b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:00:40.730 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.728+0000 7f211c3bc740 1 -- 172.21.15.1:0/1228772255 shutdown_connections 2024-09-11T22:00:40.730 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.728+0000 7f211c3bc740 1 -- 172.21.15.1:0/1228772255 wait complete. 2024-09-11T22:00:40.730 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.728+0000 7f211c3bc740 20 client.16083 trim_cache size 0 max 16384 2024-09-11T22:00:40.730 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-09-11T22:00:40.731 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-09-11T22:00:40.731 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-09-11T22:00:40.731 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-09-11T22:00:40.743 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.741+0000 7f2119623640 1 -- 172.21.15.1:0/2763318311 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f20fc0a2e40 msgr2=0x7f20fc0a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:40.743 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.741+0000 7f2119623640 1 --2- 172.21.15.1:0/2763318311 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f20fc0a2e40 0x7f20fc0a5300 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f211410f770 tx=0x7f210800b410 comp rx=0 tx=0).stop 2024-09-11T22:00:40.743 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.741+0000 7f2119623640 1 -- 172.21.15.1:0/2763318311 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f21140745a0 msgr2=0x7f211419a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:40.743 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.741+0000 7f2119623640 1 --2- 172.21.15.1:0/2763318311 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f21140745a0 0x7f211419a6d0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f2104005e30 tx=0x7f2104003ce0 comp rx=0 tx=0).stop 2024-09-11T22:00:40.743 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.741+0000 7f2119623640 1 -- 172.21.15.1:0/2763318311 shutdown_connections 2024-09-11T22:00:40.743 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.741+0000 7f2119623640 1 --2- 172.21.15.1:0/2763318311 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f20fc0a2e40 0x7f20fc0a5300 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:40.743 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.741+0000 7f2119623640 1 --2- 172.21.15.1:0/2763318311 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f211410e380 0x7f211419ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:40.743 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.741+0000 7f2119623640 1 --2- 172.21.15.1:0/2763318311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2114074ec0 0x7f211419ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:40.743 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.741+0000 7f2119623640 1 --2- 172.21.15.1:0/2763318311 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f21140745a0 0x7f211419a6d0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:40.743 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.741+0000 7f2119623640 1 -- 172.21.15.1:0/2763318311 >> 172.21.15.1:0/2763318311 conn(0x7f21140fd7a0 msgr2=0x7f21140fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:00:40.743 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.741+0000 7f2119623640 1 -- 172.21.15.1:0/2763318311 shutdown_connections 2024-09-11T22:00:40.743 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:00:40.741+0000 7f2119623640 1 -- 172.21.15.1:0/2763318311 wait complete. 2024-09-11T22:00:40.926 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:40.926 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:41.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:00:40 smithi001 ceph-mon[30935]: pgmap v108: 97 pgs: 97 active+clean; 3.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 23 KiB/s wr, 17 op/s 2024-09-11T22:00:41.252 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T22:00:41.252 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T22:00:41.284 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:41.285 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:41.285 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 80 -lt 100 ']' 2024-09-11T22:00:41.285 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 80 + 1 2024-09-11T22:00:41.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:00:40 smithi053 ceph-mon[34908]: pgmap v108: 97 pgs: 97 active+clean; 3.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 23 KiB/s wr, 17 op/s 2024-09-11T22:00:41.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:00:40 smithi161 ceph-mon[35615]: pgmap v108: 97 pgs: 97 active+clean; 3.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 23 KiB/s wr, 17 op/s 2024-09-11T22:00:41.428 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=81 2024-09-11T22:00:41.428 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T22:00:41.455 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:41.455 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:41.456 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T22:00:41.456 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T22:00:41.456 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T22:00:42.500 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:42.501 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:42.751 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T22:00:42.751 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T22:00:42.769 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:42.770 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:42.770 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 81 -lt 100 ']' 2024-09-11T22:00:42.770 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 81 + 1 2024-09-11T22:00:43.031 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=82 2024-09-11T22:00:43.031 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T22:00:43.137 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:43.137 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:43.138 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T22:00:43.138 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T22:00:43.138 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T22:00:43.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:00:42 smithi001 ceph-mon[30935]: pgmap v109: 97 pgs: 97 active+clean; 3.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-11T22:00:43.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:00:42 smithi053 ceph-mon[34908]: pgmap v109: 97 pgs: 97 active+clean; 3.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-11T22:00:43.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:00:42 smithi161 ceph-mon[35615]: pgmap v109: 97 pgs: 97 active+clean; 3.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-11T22:00:44.719 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:44.719 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:45.192 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T22:00:45.192 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T22:00:45.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:00:44 smithi001 ceph-mon[30935]: pgmap v110: 97 pgs: 97 active+clean; 3.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-11T22:00:45.227 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:45.227 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:45.227 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 82 -lt 100 ']' 2024-09-11T22:00:45.227 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 82 + 1 2024-09-11T22:00:45.274 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=83 2024-09-11T22:00:45.274 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T22:00:45.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:00:44 smithi053 ceph-mon[34908]: pgmap v110: 97 pgs: 97 active+clean; 3.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-11T22:00:45.308 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:45.308 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:45.309 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T22:00:45.309 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T22:00:45.309 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T22:00:45.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:00:44 smithi161 ceph-mon[35615]: pgmap v110: 97 pgs: 97 active+clean; 3.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-09-11T22:00:46.181 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:00:46.228 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:00:47.069 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:47.069 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:47.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:00:46 smithi001 ceph-mon[30935]: pgmap v111: 97 pgs: 97 active+clean; 3.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-11T22:00:47.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:00:46 smithi053 ceph-mon[34908]: pgmap v111: 97 pgs: 97 active+clean; 3.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-11T22:00:47.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:00:46 smithi161 ceph-mon[35615]: pgmap v111: 97 pgs: 97 active+clean; 3.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-11T22:00:47.767 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T22:00:47.767 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T22:00:47.809 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:47.809 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:47.809 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 83 -lt 100 ']' 2024-09-11T22:00:47.809 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 83 + 1 2024-09-11T22:00:47.907 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=84 2024-09-11T22:00:47.907 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T22:00:47.935 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:47.935 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:47.937 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T22:00:47.937 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T22:00:47.937 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T22:00:49.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:00:49 smithi053 ceph-mon[34908]: pgmap v112: 97 pgs: 97 active+clean; 3.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-09-11T22:00:49.312 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:00:49 smithi001 ceph-mon[30935]: pgmap v112: 97 pgs: 97 active+clean; 3.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-09-11T22:00:49.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:00:49 smithi161 ceph-mon[35615]: pgmap v112: 97 pgs: 97 active+clean; 3.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-09-11T22:00:49.460 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:49.460 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:49.878 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T22:00:49.878 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T22:00:49.909 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:49.909 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:49.909 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 84 -lt 100 ']' 2024-09-11T22:00:49.909 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 84 + 1 2024-09-11T22:00:50.003 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=85 2024-09-11T22:00:50.004 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T22:00:50.085 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:50.085 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:50.086 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T22:00:50.086 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T22:00:50.086 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T22:00:50.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:00:50 smithi053 ceph-mon[34908]: pgmap v113: 97 pgs: 97 active+clean; 3.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-09-11T22:00:50.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:00:50 smithi161 ceph-mon[35615]: pgmap v113: 97 pgs: 97 active+clean; 3.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-09-11T22:00:50.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:00:50 smithi001 ceph-mon[30935]: pgmap v113: 97 pgs: 97 active+clean; 3.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-09-11T22:00:51.225 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:51.226 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:51.499 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T22:00:51.500 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T22:00:51.516 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:51.516 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:51.516 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 85 -lt 100 ']' 2024-09-11T22:00:51.517 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 85 + 1 2024-09-11T22:00:51.590 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=86 2024-09-11T22:00:51.590 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T22:00:51.634 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:51.634 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:51.636 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T22:00:51.636 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T22:00:51.636 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T22:00:52.392 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:52.392 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:52.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.617+0000 7f9d64a45640 1 -- 172.21.15.1:0/1056935067 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d60101de0 msgr2=0x7f9d601021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:52.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.617+0000 7f9d64a45640 1 --2- 172.21.15.1:0/1056935067 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d60101de0 0x7f9d601021c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9d50009860 tx=0x7f9d5002a340 comp rx=0 tx=0).stop 2024-09-11T22:00:52.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.617+0000 7f9d64a45640 1 -- 172.21.15.1:0/1056935067 shutdown_connections 2024-09-11T22:00:52.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.617+0000 7f9d64a45640 1 --2- 172.21.15.1:0/1056935067 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d60106f10 0x7f9d6010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:52.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.617+0000 7f9d64a45640 1 --2- 172.21.15.1:0/1056935067 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d60102790 0x7f9d601067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:52.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.617+0000 7f9d64a45640 1 --2- 172.21.15.1:0/1056935067 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d60101de0 0x7f9d601021c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:52.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.617+0000 7f9d64a45640 1 -- 172.21.15.1:0/1056935067 >> 172.21.15.1:0/1056935067 conn(0x7f9d600fd8e0 msgr2=0x7f9d600ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:00:52.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.618+0000 7f9d64a45640 1 -- 172.21.15.1:0/1056935067 shutdown_connections 2024-09-11T22:00:52.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.617+0000 7fe787d7b640 1 -- 172.21.15.1:0/872694820 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe780102570 msgr2=0x7fe7801029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:52.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.617+0000 7fe787d7b640 1 --2- 172.21.15.1:0/872694820 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe780102570 0x7fe7801029b0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fe770009900 tx=0x7fe77002a480 comp rx=0 tx=0).stop 2024-09-11T22:00:52.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.617+0000 7fe787d7b640 1 -- 172.21.15.1:0/872694820 shutdown_connections 2024-09-11T22:00:52.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.617+0000 7fe787d7b640 1 --2- 172.21.15.1:0/872694820 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe780102ef0 0x7fe78010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:52.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.617+0000 7fe787d7b640 1 --2- 172.21.15.1:0/872694820 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe780102570 0x7fe7801029b0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:52.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.617+0000 7fe787d7b640 1 --2- 172.21.15.1:0/872694820 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe780100e20 0x7fe780101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:52.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.617+0000 7fe787d7b640 1 -- 172.21.15.1:0/872694820 >> 172.21.15.1:0/872694820 conn(0x7fe7800fc7d0 msgr2=0x7fe7800febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:00:52.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.617+0000 7fe787d7b640 1 -- 172.21.15.1:0/872694820 shutdown_connections 2024-09-11T22:00:52.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.617+0000 7fe787d7b640 1 -- 172.21.15.1:0/872694820 wait complete. 2024-09-11T22:00:52.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.618+0000 7fe787d7b640 1 Processor -- start 2024-09-11T22:00:52.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.618+0000 7fe787d7b640 1 -- start start 2024-09-11T22:00:52.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.618+0000 7f9d64a45640 1 -- 172.21.15.1:0/1056935067 wait complete. 2024-09-11T22:00:52.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.619+0000 7fe787d7b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe780100e20 0x7fe78019a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:52.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.619+0000 7fe787d7b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe780102570 0x7fe78019af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:52.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.619+0000 7fe787d7b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe780102ef0 0x7fe78019eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:52.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.619+0000 7f9d64a45640 1 Processor -- start 2024-09-11T22:00:52.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.619+0000 7fe787d7b640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe780110f80 con 0x7fe780100e20 2024-09-11T22:00:52.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.619+0000 7fe787d7b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe780110e00 con 0x7fe780102ef0 2024-09-11T22:00:52.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.619+0000 7fe787d7b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe780111100 con 0x7fe780102570 2024-09-11T22:00:52.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.619+0000 7f9d64a45640 1 -- start start 2024-09-11T22:00:52.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.619+0000 7fe785b17640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe780100e20 0x7fe78019a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:52.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.619+0000 7fe785316640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe780102570 0x7fe78019af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:52.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.619+0000 7fe785316640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe780102570 0x7fe78019af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:50590/0 (socket says 172.21.15.1:50590) 2024-09-11T22:00:52.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.619+0000 7fe785316640 1 -- 172.21.15.1:0/1859742144 learned_addr learned my addr 172.21.15.1:0/1859742144 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:00:52.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.619+0000 7fe786318640 1 --2- 172.21.15.1:0/1859742144 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe780102ef0 0x7fe78019eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:52.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.619+0000 7fe785316640 1 -- 172.21.15.1:0/1859742144 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe780102ef0 msgr2=0x7fe78019eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:52.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.619+0000 7fe785316640 1 --2- 172.21.15.1:0/1859742144 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe780102ef0 0x7fe78019eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:52.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.619+0000 7fe785316640 1 -- 172.21.15.1:0/1859742144 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe780100e20 msgr2=0x7fe78019a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:52.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.619+0000 7fe785316640 1 --2- 172.21.15.1:0/1859742144 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe780100e20 0x7fe78019a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:52.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.620+0000 7f9d64a45640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d60101de0 0x7f9d6010dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:52.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.620+0000 7fe785316640 1 -- 172.21.15.1:0/1859742144 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe78019f720 con 0x7fe780102570 2024-09-11T22:00:52.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.620+0000 7fe785316640 1 --2- 172.21.15.1:0/1859742144 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe780102570 0x7fe78019af10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe770006010 tx=0x7fe770003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:00:52.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.620+0000 7f9d64a45640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d60102790 0x7f9d6010e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:52.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.620+0000 7f9d64a45640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d60106f10 0x7f9d601a3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:52.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.620+0000 7f9d5e59c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d60101de0 0x7f9d6010dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:52.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.620+0000 7fe776ffd640 1 -- 172.21.15.1:0/1859742144 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe77003f070 con 0x7fe780102570 2024-09-11T22:00:52.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.620+0000 7fe776ffd640 1 -- 172.21.15.1:0/1859742144 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe770031c30 con 0x7fe780102570 2024-09-11T22:00:52.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.620+0000 7f9d5e59c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d60101de0 0x7f9d6010dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:50592/0 (socket says 172.21.15.1:50592) 2024-09-11T22:00:52.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.620+0000 7fe787d7b640 1 -- 172.21.15.1:0/1859742144 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe78019f9b0 con 0x7fe780102570 2024-09-11T22:00:52.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.620+0000 7fe787d7b640 1 -- 172.21.15.1:0/1859742144 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7801a7260 con 0x7fe780102570 2024-09-11T22:00:52.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.620+0000 7f9d5dd9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d60102790 0x7f9d6010e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:52.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.620+0000 7f9d5dd9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d60102790 0x7f9d6010e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:47508/0 (socket says 172.21.15.1:47508) 2024-09-11T22:00:52.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.621+0000 7fe776ffd640 1 -- 172.21.15.1:0/1859742144 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe770035030 con 0x7fe780102570 2024-09-11T22:00:52.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.620+0000 7f9d5ed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d60106f10 0x7f9d601a3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:52.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.621+0000 7f9d5ed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d60106f10 0x7f9d601a3120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:32998/0 (socket says 172.21.15.1:32998) 2024-09-11T22:00:52.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.621+0000 7f9d5e59c640 1 -- 172.21.15.1:0/367937544 learned_addr learned my addr 172.21.15.1:0/367937544 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:00:52.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.621+0000 7f9d64a45640 1 -- 172.21.15.1:0/367937544 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9d60111f70 con 0x7f9d60106f10 2024-09-11T22:00:52.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.621+0000 7f9d64a45640 1 -- 172.21.15.1:0/367937544 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9d60111df0 con 0x7f9d60102790 2024-09-11T22:00:52.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.621+0000 7f9d64a45640 1 -- 172.21.15.1:0/367937544 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9d601120f0 con 0x7f9d60101de0 2024-09-11T22:00:52.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.621+0000 7f9d5ed9d640 1 -- 172.21.15.1:0/367937544 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d60101de0 msgr2=0x7f9d6010dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:52.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.621+0000 7f9d5ed9d640 1 --2- 172.21.15.1:0/367937544 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d60101de0 0x7f9d6010dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:52.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.621+0000 7f9d5ed9d640 1 -- 172.21.15.1:0/367937544 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d60102790 msgr2=0x7f9d6010e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:52.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.621+0000 7f9d5ed9d640 1 --2- 172.21.15.1:0/367937544 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d60102790 0x7f9d6010e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:52.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.621+0000 7f9d5ed9d640 1 -- 172.21.15.1:0/367937544 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d601a3820 con 0x7f9d60106f10 2024-09-11T22:00:52.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.622+0000 7f9d5ed9d640 1 --2- 172.21.15.1:0/367937544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d60106f10 0x7f9d601a3120 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f9d5800ef00 tx=0x7f9d5800c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:00:52.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.622+0000 7f9d4f7fe640 1 -- 172.21.15.1:0/367937544 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d58004b20 con 0x7f9d60106f10 2024-09-11T22:00:52.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.622+0000 7f9d4f7fe640 1 -- 172.21.15.1:0/367937544 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d58003e50 con 0x7f9d60106f10 2024-09-11T22:00:52.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.622+0000 7f9d4f7fe640 1 -- 172.21.15.1:0/367937544 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d5800b570 con 0x7f9d60106f10 2024-09-11T22:00:52.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.622+0000 7f9d64a45640 1 -- 172.21.15.1:0/367937544 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d601a3b10 con 0x7f9d60106f10 2024-09-11T22:00:52.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.622+0000 7f9d64a45640 1 -- 172.21.15.1:0/367937544 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d601ab3d0 con 0x7f9d60106f10 2024-09-11T22:00:52.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.624+0000 7fe776ffd640 1 -- 172.21.15.1:0/1859742144 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7fe770042070 con 0x7fe780102570 2024-09-11T22:00:52.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.625+0000 7fe776ffd640 1 --2- 172.21.15.1:0/1859742144 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fe76c0a2d70 0x7fe76c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:52.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.625+0000 7fe785b17640 1 --2- 172.21.15.1:0/1859742144 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fe76c0a2d70 0x7fe76c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:52.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.625+0000 7fe776ffd640 1 -- 172.21.15.1:0/1859742144 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7fe7700efad0 con 0x7fe780102570 2024-09-11T22:00:52.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.625+0000 7f9d4f7fe640 1 -- 172.21.15.1:0/367937544 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f9d5801d020 con 0x7f9d60106f10 2024-09-11T22:00:52.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.625+0000 7fe785b17640 1 --2- 172.21.15.1:0/1859742144 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fe76c0a2d70 0x7fe76c0a5230 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe77c005fd0 tx=0x7fe77c009500 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:00:52.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.626+0000 7f9d4f7fe640 1 --2- 172.21.15.1:0/367937544 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f9d480a2d70 0x7f9d480a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:52.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.626+0000 7f9d4f7fe640 1 -- 172.21.15.1:0/367937544 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f9d580d0370 con 0x7f9d60106f10 2024-09-11T22:00:52.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.626+0000 7f9d5e59c640 1 --2- 172.21.15.1:0/367937544 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f9d480a2d70 0x7f9d480a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:52.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.637+0000 7fe787d7b640 1 -- 172.21.15.1:0/1859742144 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7801021b0 con 0x7fe780102570 2024-09-11T22:00:52.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.643+0000 7fe776ffd640 1 -- 172.21.15.1:0/1859742144 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe7700bc000 con 0x7fe780102570 2024-09-11T22:00:52.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.691+0000 7f9d5e59c640 1 --2- 172.21.15.1:0/367937544 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f9d480a2d70 0x7f9d480a5230 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9d500095f0 tx=0x7f9d5000b040 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:00:52.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.692+0000 7f9d66e1e740 1 Processor -- start 2024-09-11T22:00:52.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.693+0000 7f9d66e1e740 1 -- start start 2024-09-11T22:00:52.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.693+0000 7f9d66e1e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556537ea4b40 0x556537ea3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:52.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.693+0000 7f9d66e1e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556537ea35d0 0x556537ea3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:52.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.693+0000 7f9d66e1e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556537eae370 0x556537eb0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:52.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.693+0000 7f9d66e1e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x556537e092f0 con 0x556537ea35d0 2024-09-11T22:00:52.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.693+0000 7f9d66e1e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x556537db0cf0 con 0x556537eae370 2024-09-11T22:00:52.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.693+0000 7f9d66e1e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556537e052b0 con 0x556537ea4b40 2024-09-11T22:00:52.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.693+0000 7f9d5dd9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556537ea35d0 0x556537ea3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:52.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.693+0000 7f9d5dd9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556537ea35d0 0x556537ea3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33002/0 (socket says 172.21.15.1:33002) 2024-09-11T22:00:52.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.693+0000 7f9d5dd9b640 1 -- 172.21.15.1:0/628597858 learned_addr learned my addr 172.21.15.1:0/628597858 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:00:52.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.693+0000 7f9d5ed9d640 1 --2- 172.21.15.1:0/628597858 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556537eae370 0x556537eb0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:52.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.693+0000 7f9d5e59c640 1 --2- 172.21.15.1:0/628597858 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556537ea4b40 0x556537ea3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:52.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.694+0000 7f9d5dd9b640 1 -- 172.21.15.1:0/628597858 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556537ea4b40 msgr2=0x556537ea3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:00:52.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.694+0000 7f9d5dd9b640 1 --2- 172.21.15.1:0/628597858 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556537ea4b40 0x556537ea3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:52.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.694+0000 7f9d5dd9b640 1 -- 172.21.15.1:0/628597858 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556537eae370 msgr2=0x556537eb0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:00:52.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.694+0000 7f9d5dd9b640 1 --2- 172.21.15.1:0/628597858 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556537eae370 0x556537eb0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:52.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.694+0000 7f9d5dd9b640 1 -- 172.21.15.1:0/628597858 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556537d953a0 con 0x556537ea35d0 2024-09-11T22:00:52.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.694+0000 7f9d5ed9d640 1 --2- 172.21.15.1:0/628597858 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556537eae370 0x556537eb0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:00:52.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.694+0000 7f9d5dd9b640 1 --2- 172.21.15.1:0/628597858 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556537ea35d0 0x556537ea3a10 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f9d5400bcb0 tx=0x7f9d540107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=988dc9e2a74deca5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:00:52.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.694+0000 7f9d4cff9640 1 -- 172.21.15.1:0/628597858 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d54004030 con 0x556537ea35d0 2024-09-11T22:00:52.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.694+0000 7f9d4cff9640 1 -- 172.21.15.1:0/628597858 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d540041d0 con 0x556537ea35d0 2024-09-11T22:00:52.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.694+0000 7f9d4cff9640 1 -- 172.21.15.1:0/628597858 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d54011750 con 0x556537ea35d0 2024-09-11T22:00:52.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.695+0000 7f9d66e1e740 1 -- 172.21.15.1:0/628597858 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556537ea35d0 msgr2=0x556537ea3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:52.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.695+0000 7f9d66e1e740 1 --2- 172.21.15.1:0/628597858 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556537ea35d0 0x556537ea3a10 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f9d5400bcb0 tx=0x7f9d540107a0 comp rx=0 tx=0).stop 2024-09-11T22:00:52.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.695+0000 7f9d66e1e740 1 -- 172.21.15.1:0/628597858 shutdown_connections 2024-09-11T22:00:52.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.695+0000 7f9d66e1e740 1 --2- 172.21.15.1:0/628597858 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556537eae370 0x556537eb0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:52.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.695+0000 7f9d66e1e740 1 --2- 172.21.15.1:0/628597858 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556537ea35d0 0x556537ea3a10 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:52.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.695+0000 7f9d66e1e740 1 --2- 172.21.15.1:0/628597858 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556537ea4b40 0x556537ea3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:52.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.695+0000 7f9d66e1e740 1 -- 172.21.15.1:0/628597858 >> 172.21.15.1:0/628597858 conn(0x556537e6c310 msgr2=0x556537e9fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:00:52.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.695+0000 7f9d66e1e740 1 -- 172.21.15.1:0/628597858 shutdown_connections 2024-09-11T22:00:52.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.695+0000 7f9d66e1e740 1 -- 172.21.15.1:0/628597858 wait complete. 2024-09-11T22:00:52.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.696+0000 7f9d66e1e740 1 -- start start 2024-09-11T22:00:52.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.696+0000 7f9d66e1e740 1 Processor -- start 2024-09-11T22:00:52.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.696+0000 7f9d66e1e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556537ea35d0 0x556537eadd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:52.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.696+0000 7f9d66e1e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556537ea4b40 0x556537eab010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:52.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.696+0000 7f9d66e1e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556537eae370 0x556537eab600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:52.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.696+0000 7f9d66e1e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x556537eb1070 con 0x556537ea35d0 2024-09-11T22:00:52.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.696+0000 7f9d66e1e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x556537dc4060 con 0x556537ea4b40 2024-09-11T22:00:52.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.696+0000 7f9d66e1e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556537eb11f0 con 0x556537eae370 2024-09-11T22:00:52.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.696+0000 7f9d5e59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556537ea35d0 0x556537eadd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:52.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.696+0000 7f9d5e59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556537ea35d0 0x556537eadd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33010/0 (socket says 172.21.15.1:33010) 2024-09-11T22:00:52.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.696+0000 7f9d5e59c640 1 -- 172.21.15.1:0/1055629543 learned_addr learned my addr 172.21.15.1:0/1055629543 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:00:52.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.696+0000 7f9d66e1e740 10 client.0 _ll_register_callbacks cb 0x556537b178c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:00:52.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.697+0000 7f9d5ed9d640 1 --2- 172.21.15.1:0/1055629543 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556537eae370 0x556537eab600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:52.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.697+0000 7f9d5dd9b640 1 --2- 172.21.15.1:0/1055629543 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556537ea4b40 0x556537eab010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:52.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.697+0000 7f9d5e59c640 1 -- 172.21.15.1:0/1055629543 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556537eae370 msgr2=0x556537eab600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:00:52.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.697+0000 7f9d5e59c640 1 --2- 172.21.15.1:0/1055629543 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556537eae370 0x556537eab600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:52.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.697+0000 7f9d5e59c640 1 -- 172.21.15.1:0/1055629543 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556537ea4b40 msgr2=0x556537eab010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:00:52.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.697+0000 7f9d5e59c640 1 --2- 172.21.15.1:0/1055629543 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556537ea4b40 0x556537eab010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:52.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.697+0000 7f9d5e59c640 1 -- 172.21.15.1:0/1055629543 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556537b7a3c0 con 0x556537ea35d0 2024-09-11T22:00:52.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.697+0000 7f9d5dd9b640 1 --2- 172.21.15.1:0/1055629543 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556537ea4b40 0x556537eab010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:00:52.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.697+0000 7f9d5e59c640 1 --2- 172.21.15.1:0/1055629543 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556537ea35d0 0x556537eadd60 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f9d50004650 tx=0x7f9d5002a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d01bbd865a040256 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:00:52.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.697+0000 7f9d45ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:00:52.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.697+0000 7f9d45ffb640 1 -- 172.21.15.1:0/1055629543 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d5004e020 con 0x556537ea35d0 2024-09-11T22:00:52.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.697+0000 7f9d45ffb640 1 -- 172.21.15.1:0/1055629543 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f9d50005510 con 0x556537ea35d0 2024-09-11T22:00:52.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.697+0000 7f9d66e1e740 1 -- 172.21.15.1:0/1055629543 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d540041d0 con 0x556537ea35d0 2024-09-11T22:00:52.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.698+0000 7f9d45ffb640 1 -- 172.21.15.1:0/1055629543 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9d50044070 con 0x556537ea35d0 2024-09-11T22:00:52.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.698+0000 7f9d66e1e740 10 client.16101 fetch_fsmap learned FSMap version 30 2024-09-11T22:00:52.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.698+0000 7f9d66e1e740 1 -- 172.21.15.1:0/1055629543 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556537ebcb40 con 0x556537ea35d0 2024-09-11T22:00:52.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.698+0000 7f9d45ffb640 1 -- 172.21.15.1:0/1055629543 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d50045040 con 0x556537ea35d0 2024-09-11T22:00:52.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.698+0000 7f9d45ffb640 1 -- 172.21.15.1:0/1055629543 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f9d50003b50 con 0x556537ea35d0 2024-09-11T22:00:52.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.698+0000 7f9d66e1e740 10 client.16101 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:00:52.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.698+0000 7f9d66e1e740 10 client.16101 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:00:52.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.698+0000 7f9d66e1e740 20 client.16101 populate_metadata read hostname 'smithi001' 2024-09-11T22:00:52.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.698+0000 7f9d66e1e740 1 --2- 172.21.15.1:0/1055629543 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x556537ebe050 0x556537ec04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:52.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.698+0000 7f9d66e1e740 4 client.16101 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:00:52.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.698+0000 7f9d66e1e740 1 -- 172.21.15.1:0/1055629543 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9d540041d0 con 0x556537ebe050 2024-09-11T22:00:52.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.699+0000 7f9d5dd9b640 1 --2- 172.21.15.1:0/1055629543 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x556537ebe050 0x556537ec04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:52.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.699+0000 7f9d45ffb640 10 client.16101 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:00:52.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.699+0000 7f9d5dd9b640 1 --2- 172.21.15.1:0/1055629543 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x556537ebe050 0x556537ec04d0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3b960c6f01ee621 server_cookie=3143366b224731c2 in_seq=0 out_seq=0 2024-09-11T22:00:52.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.700+0000 7f9d45ffb640 1 -- 172.21.15.1:0/1055629543 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f9d540041d0 con 0x556537ebe050 2024-09-11T22:00:52.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.700+0000 7f9d45ffb640 10 client.16101 handle_command_reply: tid=0 2024-09-11T22:00:52.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.700+0000 7f9d66e1e740 1 client.16101 shutdown 2024-09-11T22:00:52.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.700+0000 7f9d66e1e740 1 -- 172.21.15.1:0/1055629543 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556537ea35d0 msgr2=0x556537eadd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:52.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.700+0000 7f9d66e1e740 1 --2- 172.21.15.1:0/1055629543 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556537ea35d0 0x556537eadd60 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f9d50004650 tx=0x7f9d5002a340 comp rx=0 tx=0).stop 2024-09-11T22:00:52.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.700+0000 7f9d66e1e740 1 -- 172.21.15.1:0/1055629543 shutdown_connections 2024-09-11T22:00:52.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.700+0000 7f9d66e1e740 1 --2- 172.21.15.1:0/1055629543 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x556537ebe050 0x556537ec04d0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:52.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.700+0000 7f9d66e1e740 1 --2- 172.21.15.1:0/1055629543 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556537eae370 0x556537eab600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:52.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.700+0000 7f9d45ffb640 0 client.16101 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:00:52.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.700+0000 7f9d66e1e740 1 --2- 172.21.15.1:0/1055629543 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556537ea4b40 0x556537eab010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:52.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.700+0000 7f9d66e1e740 1 --2- 172.21.15.1:0/1055629543 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556537ea35d0 0x556537eadd60 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:52.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.700+0000 7f9d66e1e740 1 -- 172.21.15.1:0/1055629543 >> 172.21.15.1:0/1055629543 conn(0x556537e6c310 msgr2=0x556537ea5930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:00:52.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.700+0000 7f9d66e1e740 1 -- 172.21.15.1:0/1055629543 shutdown_connections 2024-09-11T22:00:52.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.700+0000 7f9d66e1e740 1 -- 172.21.15.1:0/1055629543 wait complete. 2024-09-11T22:00:52.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.700+0000 7f9d66e1e740 20 client.16101 trim_cache size 0 max 16384 2024-09-11T22:00:52.710 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T22:00:52.710 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T22:00:52.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.722+0000 7f9d66e1e740 1 Processor -- start 2024-09-11T22:00:52.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.722+0000 7f9d66e1e740 1 -- start start 2024-09-11T22:00:52.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.722+0000 7f9d66e1e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556537ea35d0 0x556537e9fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:52.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.722+0000 7f9d66e1e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556537ea4b40 0x556537eba340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:52.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.722+0000 7f9d66e1e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556537eae370 0x556537ec5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:52.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.722+0000 7f9d66e1e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9d601b4070 con 0x556537ea4b40 2024-09-11T22:00:52.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.722+0000 7f9d66e1e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9d601b5c10 con 0x556537eae370 2024-09-11T22:00:52.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.722+0000 7f9d66e1e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9d601b4740 con 0x556537ea35d0 2024-09-11T22:00:52.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.722+0000 7f9d5dd9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556537ea4b40 0x556537eba340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:52.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.722+0000 7f9d5dd9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556537ea4b40 0x556537eba340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33020/0 (socket says 172.21.15.1:33020) 2024-09-11T22:00:52.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.722+0000 7f9d5dd9b640 1 -- 172.21.15.1:0/2580213120 learned_addr learned my addr 172.21.15.1:0/2580213120 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:00:52.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.722+0000 7f9d5ed9d640 1 --2- 172.21.15.1:0/2580213120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556537eae370 0x556537ec5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:52.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.722+0000 7f9d5e59c640 1 --2- 172.21.15.1:0/2580213120 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556537ea35d0 0x556537e9fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:52.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.722+0000 7f9d5dd9b640 1 -- 172.21.15.1:0/2580213120 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556537ea35d0 msgr2=0x556537e9fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:00:52.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.722+0000 7f9d5dd9b640 1 --2- 172.21.15.1:0/2580213120 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556537ea35d0 0x556537e9fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:52.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.722+0000 7f9d5dd9b640 1 -- 172.21.15.1:0/2580213120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556537eae370 msgr2=0x556537ec5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:00:52.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.722+0000 7f9d5dd9b640 1 --2- 172.21.15.1:0/2580213120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556537eae370 0x556537ec5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:52.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.723+0000 7f9d5dd9b640 1 -- 172.21.15.1:0/2580213120 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556537eab970 con 0x556537ea4b40 2024-09-11T22:00:52.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.723+0000 7f9d5e59c640 1 --2- 172.21.15.1:0/2580213120 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556537ea35d0 0x556537e9fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:00:52.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.723+0000 7f9d5dd9b640 1 --2- 172.21.15.1:0/2580213120 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556537ea4b40 0x556537eba340 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f9d5400bbd0 tx=0x7f9d5400b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee3d7759d1db5487 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:00:52.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.723+0000 7f9d47fff640 1 -- 172.21.15.1:0/2580213120 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d54005040 con 0x556537ea4b40 2024-09-11T22:00:52.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.723+0000 7f9d47fff640 1 -- 172.21.15.1:0/2580213120 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d540035f0 con 0x556537ea4b40 2024-09-11T22:00:52.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.723+0000 7f9d47fff640 1 -- 172.21.15.1:0/2580213120 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d54011680 con 0x556537ea4b40 2024-09-11T22:00:52.726 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:52.726 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:52.726 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 86 -lt 100 ']' 2024-09-11T22:00:52.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.724+0000 7f9d66e1e740 1 -- 172.21.15.1:0/2580213120 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556537ea4b40 msgr2=0x556537eba340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:52.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.724+0000 7f9d66e1e740 1 --2- 172.21.15.1:0/2580213120 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556537ea4b40 0x556537eba340 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f9d5400bbd0 tx=0x7f9d5400b390 comp rx=0 tx=0).stop 2024-09-11T22:00:52.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.724+0000 7f9d66e1e740 1 -- 172.21.15.1:0/2580213120 shutdown_connections 2024-09-11T22:00:52.727 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 86 + 1 2024-09-11T22:00:52.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.724+0000 7f9d66e1e740 1 --2- 172.21.15.1:0/2580213120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556537eae370 0x556537ec5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:52.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.724+0000 7f9d66e1e740 1 --2- 172.21.15.1:0/2580213120 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556537ea4b40 0x556537eba340 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:52.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.724+0000 7f9d66e1e740 1 --2- 172.21.15.1:0/2580213120 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556537ea35d0 0x556537e9fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:52.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.724+0000 7f9d66e1e740 1 -- 172.21.15.1:0/2580213120 >> 172.21.15.1:0/2580213120 conn(0x556537e6c310 msgr2=0x556537e9f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:00:52.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.724+0000 7f9d66e1e740 1 -- 172.21.15.1:0/2580213120 shutdown_connections 2024-09-11T22:00:52.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.724+0000 7f9d66e1e740 1 -- 172.21.15.1:0/2580213120 wait complete. 2024-09-11T22:00:52.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.724+0000 7f9d66e1e740 1 -- start start 2024-09-11T22:00:52.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.724+0000 7f9d66e1e740 1 Processor -- start 2024-09-11T22:00:52.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.725+0000 7f9d66e1e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556537ea35d0 0x556537eb6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:52.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.725+0000 7f9d66e1e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556537ea4b40 0x556537eb1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:52.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.725+0000 7f9d66e1e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556537eae370 0x556537eb23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:52.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.725+0000 7f9d66e1e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9d601b5510 con 0x556537ea35d0 2024-09-11T22:00:52.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.725+0000 7f9d66e1e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9d601b4e10 con 0x556537eae370 2024-09-11T22:00:52.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.725+0000 7f9d66e1e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9d601b2050 con 0x556537ea4b40 2024-09-11T22:00:52.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.725+0000 7f9d66e1e740 10 client.0 _ll_register_callbacks cb 0x7f9d58009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:00:52.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.725+0000 7f9d5e59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556537ea35d0 0x556537eb6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:52.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.725+0000 7f9d5e59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556537ea35d0 0x556537eb6b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33024/0 (socket says 172.21.15.1:33024) 2024-09-11T22:00:52.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.725+0000 7f9d5e59c640 1 -- 172.21.15.1:0/1246001543 learned_addr learned my addr 172.21.15.1:0/1246001543 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:00:52.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.725+0000 7f9d5ed9d640 1 --2- 172.21.15.1:0/1246001543 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556537eae370 0x556537eb23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:52.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.725+0000 7f9d5dd9b640 1 --2- 172.21.15.1:0/1246001543 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556537ea4b40 0x556537eb1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:52.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.725+0000 7f9d5e59c640 1 -- 172.21.15.1:0/1246001543 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556537ea4b40 msgr2=0x556537eb1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:00:52.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.725+0000 7f9d5e59c640 1 --2- 172.21.15.1:0/1246001543 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556537ea4b40 0x556537eb1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:52.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.725+0000 7f9d5e59c640 1 -- 172.21.15.1:0/1246001543 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556537eae370 msgr2=0x556537eb23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:00:52.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.725+0000 7f9d5e59c640 1 --2- 172.21.15.1:0/1246001543 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556537eae370 0x556537eb23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:52.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.725+0000 7f9d5e59c640 1 -- 172.21.15.1:0/1246001543 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556537eb2bd0 con 0x556537ea35d0 2024-09-11T22:00:52.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.725+0000 7f9d5ed9d640 1 --2- 172.21.15.1:0/1246001543 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556537eae370 0x556537eb23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:00:52.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.725+0000 7f9d5dd9b640 1 --2- 172.21.15.1:0/1246001543 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556537ea4b40 0x556537eb1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:00:52.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.725+0000 7f9d5e59c640 1 --2- 172.21.15.1:0/1246001543 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556537ea35d0 0x556537eb6b60 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f9d50000c00 tx=0x7f9d5003d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fb0a5b2eddcddad server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:00:52.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.725+0000 7f9d45ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:00:52.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.726+0000 7f9d45ffb640 1 -- 172.21.15.1:0/1246001543 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d50062060 con 0x556537ea35d0 2024-09-11T22:00:52.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.726+0000 7f9d66e1e740 1 -- 172.21.15.1:0/1246001543 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d540035f0 con 0x556537ea35d0 2024-09-11T22:00:52.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.726+0000 7f9d45ffb640 1 -- 172.21.15.1:0/1246001543 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f9d5003d990 con 0x556537ea35d0 2024-09-11T22:00:52.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.726+0000 7f9d45ffb640 1 -- 172.21.15.1:0/1246001543 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9d5004e040 con 0x556537ea35d0 2024-09-11T22:00:52.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.726+0000 7f9d66e1e740 10 client.16107 fetch_fsmap learned FSMap version 30 2024-09-11T22:00:52.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.726+0000 7f9d66e1e740 1 -- 172.21.15.1:0/1246001543 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556537f28ce0 con 0x556537ea35d0 2024-09-11T22:00:52.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.726+0000 7f9d45ffb640 1 -- 172.21.15.1:0/1246001543 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d540035f0 con 0x556537ea35d0 2024-09-11T22:00:52.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.726+0000 7f9d45ffb640 1 -- 172.21.15.1:0/1246001543 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f9d500046c0 con 0x556537ea35d0 2024-09-11T22:00:52.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.726+0000 7f9d66e1e740 10 client.16107 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:00:52.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.726+0000 7f9d66e1e740 10 client.16107 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:00:52.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.726+0000 7f9d66e1e740 20 client.16107 populate_metadata read hostname 'smithi001' 2024-09-11T22:00:52.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.726+0000 7f9d66e1e740 1 --2- 172.21.15.1:0/1246001543 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x556537ebe050 0x556537f2b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:00:52.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.726+0000 7f9d66e1e740 4 client.16107 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:00:52.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.726+0000 7f9d66e1e740 1 -- 172.21.15.1:0/1246001543 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556537e16860 con 0x556537ebe050 2024-09-11T22:00:52.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.727+0000 7f9d5dd9b640 1 --2- 172.21.15.1:0/1246001543 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x556537ebe050 0x556537f2b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:00:52.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.727+0000 7f9d45ffb640 10 client.16107 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:00:52.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.727+0000 7f9d5dd9b640 1 --2- 172.21.15.1:0/1246001543 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x556537ebe050 0x556537f2b3f0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5796d110fe95825b server_cookie=b6f6caf52d351455 in_seq=0 out_seq=0 2024-09-11T22:00:52.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.727+0000 7f9d45ffb640 1 -- 172.21.15.1:0/1246001543 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556537e16860 con 0x556537ebe050 2024-09-11T22:00:52.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.727+0000 7f9d45ffb640 10 client.16107 handle_command_reply: tid=0 2024-09-11T22:00:52.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.727+0000 7f9d66e1e740 1 client.16107 shutdown 2024-09-11T22:00:52.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.727+0000 7f9d66e1e740 1 -- 172.21.15.1:0/1246001543 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556537ea35d0 msgr2=0x556537eb6b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:52.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.727+0000 7f9d66e1e740 1 --2- 172.21.15.1:0/1246001543 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556537ea35d0 0x556537eb6b60 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f9d50000c00 tx=0x7f9d5003d890 comp rx=0 tx=0).stop 2024-09-11T22:00:52.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.727+0000 7f9d66e1e740 1 -- 172.21.15.1:0/1246001543 shutdown_connections 2024-09-11T22:00:52.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.727+0000 7f9d66e1e740 1 --2- 172.21.15.1:0/1246001543 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x556537ebe050 0x556537f2b3f0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:52.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.728+0000 7f9d66e1e740 1 --2- 172.21.15.1:0/1246001543 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556537eae370 0x556537eb23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:52.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.728+0000 7f9d45ffb640 0 client.16107 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:00:52.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.728+0000 7f9d66e1e740 1 --2- 172.21.15.1:0/1246001543 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556537ea4b40 0x556537eb1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:52.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.728+0000 7f9d66e1e740 1 --2- 172.21.15.1:0/1246001543 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556537ea35d0 0x556537eb6b60 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:52.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.728+0000 7f9d66e1e740 1 -- 172.21.15.1:0/1246001543 >> 172.21.15.1:0/1246001543 conn(0x556537e6c310 msgr2=0x556537ea5930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:00:52.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.728+0000 7f9d66e1e740 1 -- 172.21.15.1:0/1246001543 shutdown_connections 2024-09-11T22:00:52.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.728+0000 7f9d66e1e740 1 -- 172.21.15.1:0/1246001543 wait complete. 2024-09-11T22:00:52.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.728+0000 7f9d66e1e740 20 client.16107 trim_cache size 0 max 16384 2024-09-11T22:00:52.736 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-11T22:00:52.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.728+0000 7f9d64a45640 1 -- 172.21.15.1:0/367937544 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f9d480a2d70 msgr2=0x7f9d480a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:52.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.728+0000 7f9d64a45640 1 --2- 172.21.15.1:0/367937544 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f9d480a2d70 0x7f9d480a5230 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9d500095f0 tx=0x7f9d5000b040 comp rx=0 tx=0).stop 2024-09-11T22:00:52.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.728+0000 7f9d64a45640 1 -- 172.21.15.1:0/367937544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d60106f10 msgr2=0x7f9d601a3120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:52.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.728+0000 7f9d64a45640 1 --2- 172.21.15.1:0/367937544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d60106f10 0x7f9d601a3120 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f9d5800ef00 tx=0x7f9d5800c740 comp rx=0 tx=0).stop 2024-09-11T22:00:52.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.728+0000 7f9d64a45640 1 -- 172.21.15.1:0/367937544 shutdown_connections 2024-09-11T22:00:52.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.728+0000 7f9d64a45640 1 --2- 172.21.15.1:0/367937544 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f9d480a2d70 0x7f9d480a5230 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:52.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.728+0000 7f9d64a45640 1 --2- 172.21.15.1:0/367937544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d60106f10 0x7f9d601a3120 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:52.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.728+0000 7f9d64a45640 1 --2- 172.21.15.1:0/367937544 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d60102790 0x7f9d6010e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:52.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.728+0000 7f9d64a45640 1 --2- 172.21.15.1:0/367937544 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d60101de0 0x7f9d6010dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:52.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.728+0000 7f9d64a45640 1 -- 172.21.15.1:0/367937544 >> 172.21.15.1:0/367937544 conn(0x7f9d600fd8e0 msgr2=0x7f9d600747b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:00:52.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.729+0000 7f9d64a45640 1 -- 172.21.15.1:0/367937544 shutdown_connections 2024-09-11T22:00:52.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.729+0000 7f9d64a45640 1 -- 172.21.15.1:0/367937544 wait complete. 2024-09-11T22:00:52.772 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=87 2024-09-11T22:00:52.772 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T22:00:52.812 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:52.812 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:52.812 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T22:00:52.812 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T22:00:52.812 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T22:00:52.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.863+0000 7fe787d7b640 1 -- 172.21.15.1:0/1859742144 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe78019ba10 con 0x7fe780102570 2024-09-11T22:00:52.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.864+0000 7fe776ffd640 1 -- 172.21.15.1:0/1859742144 <== mon.1 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11951 (secure 0 0 0) 0x7fe7700c0300 con 0x7fe780102570 2024-09-11T22:00:52.866 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T22:00:52.866 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":30,"btime":"2024-09-11T22:00:27:488718+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/421542832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":421542832},{"type":"v1","addr":"172.21.15.1:6835","nonce":421542832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2836864406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2836864406},{"type":"v1","addr":"172.21.15.1:6837","nonce":2836864406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T21:55:20.791263+0000","modified":"2024-09-11T22:00:26.715155+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24484,"mds_2":24467,"mds_3":24466,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14595":{"gid":14595,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/407834672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":407834672},{"type":"v1","addr":"172.21.15.1:6839","nonce":407834672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/1068827678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":1068827678},{"type":"v1","addr":"172.21.15.1:6841","nonce":1068827678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/872410024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":872410024},{"type":"v1","addr":"172.21.15.161:6833","nonce":872410024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/1648944570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1648944570},{"type":"v1","addr":"172.21.15.53:6833","nonce":1648944570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/2126933993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2126933993},{"type":"v1","addr":"172.21.15.53:6835","nonce":2126933993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.161:6835/828381592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":828381592},{"type":"v1","addr":"172.21.15.161:6835","nonce":828381592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.161:6837/1799318656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1799318656},{"type":"v1","addr":"172.21.15.161:6837","nonce":1799318656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.53:6837/3457131830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3457131830},{"type":"v1","addr":"172.21.15.53:6837","nonce":3457131830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/455365383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":455365383},{"type":"v1","addr":"172.21.15.161:6839","nonce":455365383}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.53:6839/1378584447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1378584447},{"type":"v1","addr":"172.21.15.53:6839","nonce":1378584447}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24460,24466,24467,24484,24494]},"id":1}]} 2024-09-11T22:00:52.866 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 30 2024-09-11T22:00:52.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.866+0000 7fe787d7b640 1 -- 172.21.15.1:0/1859742144 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fe76c0a2d70 msgr2=0x7fe76c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:52.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.866+0000 7fe787d7b640 1 --2- 172.21.15.1:0/1859742144 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fe76c0a2d70 0x7fe76c0a5230 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe77c005fd0 tx=0x7fe77c009500 comp rx=0 tx=0).stop 2024-09-11T22:00:52.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.866+0000 7fe787d7b640 1 -- 172.21.15.1:0/1859742144 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe780102570 msgr2=0x7fe78019af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:00:52.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.866+0000 7fe787d7b640 1 --2- 172.21.15.1:0/1859742144 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe780102570 0x7fe78019af10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe770006010 tx=0x7fe770003c10 comp rx=0 tx=0).stop 2024-09-11T22:00:52.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.866+0000 7fe787d7b640 1 -- 172.21.15.1:0/1859742144 shutdown_connections 2024-09-11T22:00:52.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.866+0000 7fe787d7b640 1 --2- 172.21.15.1:0/1859742144 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fe76c0a2d70 0x7fe76c0a5230 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:52.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.866+0000 7fe787d7b640 1 --2- 172.21.15.1:0/1859742144 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe780102ef0 0x7fe78019eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:52.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.866+0000 7fe787d7b640 1 --2- 172.21.15.1:0/1859742144 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe780102570 0x7fe78019af10 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:52.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.866+0000 7fe787d7b640 1 --2- 172.21.15.1:0/1859742144 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe780100e20 0x7fe78019a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:00:52.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.866+0000 7fe787d7b640 1 -- 172.21.15.1:0/1859742144 >> 172.21.15.1:0/1859742144 conn(0x7fe7800fc7d0 msgr2=0x7fe7801037f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:00:52.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.866+0000 7fe787d7b640 1 -- 172.21.15.1:0/1859742144 shutdown_connections 2024-09-11T22:00:52.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:00:52.866+0000 7fe787d7b640 1 -- 172.21.15.1:0/1859742144 wait complete. 2024-09-11T22:00:53.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:00:52 smithi001 ceph-mon[30935]: pgmap v114: 97 pgs: 97 active+clean; 3.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 15 op/s 2024-09-11T22:00:53.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:00:52 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/1859742144' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:00:53.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:00:52 smithi053 ceph-mon[34908]: pgmap v114: 97 pgs: 97 active+clean; 3.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 15 op/s 2024-09-11T22:00:53.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:00:52 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/1859742144' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:00:53.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:00:52 smithi161 ceph-mon[35615]: pgmap v114: 97 pgs: 97 active+clean; 3.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 15 op/s 2024-09-11T22:00:53.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:00:52 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/1859742144' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:00:53.991 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:53.991 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:54.317 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T22:00:54.317 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T22:00:54.349 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:54.349 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:54.349 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 87 -lt 100 ']' 2024-09-11T22:00:54.349 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 87 + 1 2024-09-11T22:00:54.423 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=88 2024-09-11T22:00:54.423 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T22:00:54.475 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:54.475 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:54.476 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T22:00:54.476 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T22:00:54.476 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T22:00:55.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:00:54 smithi001 ceph-mon[30935]: pgmap v115: 97 pgs: 97 active+clean; 3.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 15 op/s 2024-09-11T22:00:55.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:00:54 smithi053 ceph-mon[34908]: pgmap v115: 97 pgs: 97 active+clean; 3.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 15 op/s 2024-09-11T22:00:55.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:00:54 smithi161 ceph-mon[35615]: pgmap v115: 97 pgs: 97 active+clean; 3.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 15 op/s 2024-09-11T22:00:55.667 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:55.667 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:56.365 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T22:00:56.366 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T22:00:56.418 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:56.419 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:56.419 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 88 -lt 100 ']' 2024-09-11T22:00:56.419 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 88 + 1 2024-09-11T22:00:56.622 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=89 2024-09-11T22:00:56.622 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T22:00:56.692 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:56.692 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:56.693 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T22:00:56.693 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T22:00:56.693 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T22:00:57.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:00:56 smithi001 ceph-mon[30935]: pgmap v116: 97 pgs: 97 active+clean; 3.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-09-11T22:00:57.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:00:56 smithi053 ceph-mon[34908]: pgmap v116: 97 pgs: 97 active+clean; 3.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-09-11T22:00:57.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:00:56 smithi161 ceph-mon[35615]: pgmap v116: 97 pgs: 97 active+clean; 3.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-09-11T22:00:57.502 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db / --timeout 90 --expiration 144.7 --await-for 45.0' 2024-09-11T22:00:57.502 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db / --timeout 90 --expiration 144.7 --await-for 45.0 2024-09-11T22:00:58.067 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:58.068 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:58.225 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T22:00:58.225 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T22:00:58.240 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:58.240 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:58.241 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 89 -lt 100 ']' 2024-09-11T22:00:58.241 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 89 + 1 2024-09-11T22:00:58.420 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=90 2024-09-11T22:00:58.420 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T22:00:58.460 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:58.460 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:58.461 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T22:00:58.461 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T22:00:58.461 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T22:00:58.696 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T22:00:58.697 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T22:00:59.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:00:58 smithi001 ceph-mon[30935]: pgmap v117: 97 pgs: 97 active+clean; 3.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-11T22:00:59.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:00:58 smithi053 ceph-mon[34908]: pgmap v117: 97 pgs: 97 active+clean; 3.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-11T22:00:59.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:00:58 smithi161 ceph-mon[35615]: pgmap v117: 97 pgs: 97 active+clean; 3.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-11T22:00:59.742 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:00:59.742 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:00:59.940 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T22:00:59.940 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T22:00:59.949 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:59.949 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:00:59.949 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 90 -lt 100 ']' 2024-09-11T22:00:59.949 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 90 + 1 2024-09-11T22:01:00.045 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=91 2024-09-11T22:01:00.058 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T22:01:00.066 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:01:00.066 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:01:00.067 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T22:01:00.067 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T22:01:00.068 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T22:01:00.296 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.295+0000 7f5f551f6640 1 -- 172.21.15.1:0/2856043035 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5f50107ad0 msgr2=0x7f5f50107eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:00.296 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.295+0000 7f5f551f6640 1 --2- 172.21.15.1:0/2856043035 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5f50107ad0 0x7f5f50107eb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5f40009930 tx=0x7f5f4002a420 comp rx=0 tx=0).stop 2024-09-11T22:01:00.296 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.295+0000 7f5f551f6640 1 -- 172.21.15.1:0/2856043035 shutdown_connections 2024-09-11T22:01:00.296 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.295+0000 7f5f551f6640 1 --2- 172.21.15.1:0/2856043035 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f50102460 0x7f5f5010e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:00.296 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.295+0000 7f5f551f6640 1 --2- 172.21.15.1:0/2856043035 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f50101ac0 0x7f5f50101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:00.297 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.295+0000 7f5f551f6640 1 --2- 172.21.15.1:0/2856043035 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5f50107ad0 0x7f5f50107eb0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:00.297 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.295+0000 7f5f551f6640 1 -- 172.21.15.1:0/2856043035 >> 172.21.15.1:0/2856043035 conn(0x7f5f500fd7a0 msgr2=0x7f5f500ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:01:00.297 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.295+0000 7f5f551f6640 1 -- 172.21.15.1:0/2856043035 shutdown_connections 2024-09-11T22:01:00.297 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.295+0000 7f5f551f6640 1 -- 172.21.15.1:0/2856043035 wait complete. 2024-09-11T22:01:00.328 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.327+0000 7f5f551f6640 1 Processor -- start 2024-09-11T22:01:00.330 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.328+0000 7f5f551f6640 1 -- start start 2024-09-11T22:01:00.330 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.328+0000 7f5f551f6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f50101ac0 0x7f5f50198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:00.331 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.328+0000 7f5f551f6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f50102460 0x7f5f50198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:00.331 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.328+0000 7f5f551f6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5f50107ad0 0x7f5f5019cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:00.331 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.328+0000 7f5f551f6640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5f50111f60 con 0x7f5f50102460 2024-09-11T22:01:00.331 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.328+0000 7f5f551f6640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5f50111de0 con 0x7f5f50101ac0 2024-09-11T22:01:00.331 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.328+0000 7f5f551f6640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5f501120e0 con 0x7f5f50107ad0 2024-09-11T22:01:00.331 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.328+0000 7f5f4659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f50102460 0x7f5f50198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:00.331 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.328+0000 7f5f4659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f50102460 0x7f5f50198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45388/0 (socket says 172.21.15.1:45388) 2024-09-11T22:01:00.331 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.328+0000 7f5f4659c640 1 -- 172.21.15.1:0/804916181 learned_addr learned my addr 172.21.15.1:0/804916181 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:01:00.331 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.328+0000 7f5f4f59e640 1 --2- 172.21.15.1:0/804916181 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5f50107ad0 0x7f5f5019cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:00.331 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.328+0000 7f5f4ed9d640 1 --2- 172.21.15.1:0/804916181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f50101ac0 0x7f5f50198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:00.331 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.329+0000 7f5f4659c640 1 -- 172.21.15.1:0/804916181 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5f50107ad0 msgr2=0x7f5f5019cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:00.332 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.329+0000 7f5f4659c640 1 --2- 172.21.15.1:0/804916181 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5f50107ad0 0x7f5f5019cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:00.332 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.329+0000 7f5f4659c640 1 -- 172.21.15.1:0/804916181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f50101ac0 msgr2=0x7f5f50198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:00.332 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.329+0000 7f5f4659c640 1 --2- 172.21.15.1:0/804916181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f50101ac0 0x7f5f50198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:00.332 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.329+0000 7f5f4659c640 1 -- 172.21.15.1:0/804916181 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f5019d440 con 0x7f5f50102460 2024-09-11T22:01:00.332 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.329+0000 7f5f4f59e640 1 --2- 172.21.15.1:0/804916181 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5f50107ad0 0x7f5f5019cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:01:00.332 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.329+0000 7f5f4659c640 1 --2- 172.21.15.1:0/804916181 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f50102460 0x7f5f50198990 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f5f3800e9f0 tx=0x7f5f3800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:01:00.341 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.341+0000 7f5f4cd99640 1 -- 172.21.15.1:0/804916181 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f3800ccc0 con 0x7f5f50102460 2024-09-11T22:01:00.341 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.341+0000 7f5f4cd99640 1 -- 172.21.15.1:0/804916181 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f3800ce60 con 0x7f5f50102460 2024-09-11T22:01:00.342 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.341+0000 7f5f4cd99640 1 -- 172.21.15.1:0/804916181 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f38010690 con 0x7f5f50102460 2024-09-11T22:01:00.342 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.341+0000 7f5f551f6640 1 -- 172.21.15.1:0/804916181 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f5019d730 con 0x7f5f50102460 2024-09-11T22:01:00.344 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.344+0000 7f5f4cd99640 1 -- 172.21.15.1:0/804916181 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f5f380046c0 con 0x7f5f50102460 2024-09-11T22:01:00.345 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.344+0000 7f5f4cd99640 1 --2- 172.21.15.1:0/804916181 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f5f3c0a2e80 0x7f5f3c0a5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:00.345 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.345+0000 7f5f4ed9d640 1 --2- 172.21.15.1:0/804916181 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f5f3c0a2e80 0x7f5f3c0a5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:00.345 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.345+0000 7f5f4ed9d640 1 --2- 172.21.15.1:0/804916181 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f5f3c0a2e80 0x7f5f3c0a5340 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5f4002a990 tx=0x7f5f4002a920 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:01:00.373 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.373+0000 7f5f551f6640 1 -- 172.21.15.1:0/804916181 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f501a4fc0 con 0x7f5f50102460 2024-09-11T22:01:00.373 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.373+0000 7f5f4cd99640 1 -- 172.21.15.1:0/804916181 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f5f380995d0 con 0x7f5f50102460 2024-09-11T22:01:00.665 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.664+0000 7f5f5806b740 1 Processor -- start 2024-09-11T22:01:00.665 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.664+0000 7f5f5806b740 1 -- start start 2024-09-11T22:01:00.666 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.665+0000 7f5f5806b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b239cf9d0 0x559b239cfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:00.666 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.665+0000 7f5f5806b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559b239d02f0 0x559b239deb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:00.666 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.665+0000 7f5f5806b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559b239df0c0 0x559b239e14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:00.666 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.665+0000 7f5f5806b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x559b239342f0 con 0x559b239d02f0 2024-09-11T22:01:00.666 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.665+0000 7f5f5806b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x559b238dc270 con 0x559b239df0c0 2024-09-11T22:01:00.667 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.665+0000 7f5f5806b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x559b239302b0 con 0x559b239cf9d0 2024-09-11T22:01:00.667 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.665+0000 7f5f4659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559b239d02f0 0x559b239deb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:00.667 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.665+0000 7f5f4ed9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b239cf9d0 0x559b239cfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:00.667 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.665+0000 7f5f4659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559b239d02f0 0x559b239deb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45404/0 (socket says 172.21.15.1:45404) 2024-09-11T22:01:00.667 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.665+0000 7f5f4659c640 1 -- 172.21.15.1:0/3359672413 learned_addr learned my addr 172.21.15.1:0/3359672413 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:01:00.667 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.666+0000 7f5f4f59e640 1 --2- 172.21.15.1:0/3359672413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559b239df0c0 0x559b239e14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:00.667 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.666+0000 7f5f4659c640 1 -- 172.21.15.1:0/3359672413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b239cf9d0 msgr2=0x559b239cfdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:00.667 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.666+0000 7f5f4659c640 1 --2- 172.21.15.1:0/3359672413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b239cf9d0 0x559b239cfdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:00.667 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.666+0000 7f5f4659c640 1 -- 172.21.15.1:0/3359672413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559b239df0c0 msgr2=0x559b239e14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:00.667 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.666+0000 7f5f4659c640 1 --2- 172.21.15.1:0/3359672413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559b239df0c0 0x559b239e14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:00.667 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.666+0000 7f5f4659c640 1 -- 172.21.15.1:0/3359672413 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b238db340 con 0x559b239d02f0 2024-09-11T22:01:00.667 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.666+0000 7f5f4ed9d640 1 --2- 172.21.15.1:0/3359672413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b239cf9d0 0x559b239cfdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:01:00.668 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.666+0000 7f5f4f59e640 1 --2- 172.21.15.1:0/3359672413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559b239df0c0 0x559b239e14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:01:00.668 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.666+0000 7f5f4659c640 1 --2- 172.21.15.1:0/3359672413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559b239d02f0 0x559b239deb80 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f5f3800d010 tx=0x7f5f3809afa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dfebbc7a5bb173d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:01:00.669 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.666+0000 7f5f4559a640 1 -- 172.21.15.1:0/3359672413 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f380b8070 con 0x559b239d02f0 2024-09-11T22:01:00.669 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.666+0000 7f5f4559a640 1 -- 172.21.15.1:0/3359672413 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f3809f8e0 con 0x559b239d02f0 2024-09-11T22:01:00.669 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.666+0000 7f5f4559a640 1 -- 172.21.15.1:0/3359672413 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f380bb030 con 0x559b239d02f0 2024-09-11T22:01:00.669 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.667+0000 7f5f5806b740 1 -- 172.21.15.1:0/3359672413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559b239d02f0 msgr2=0x559b239deb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:00.669 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.667+0000 7f5f5806b740 1 --2- 172.21.15.1:0/3359672413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559b239d02f0 0x559b239deb80 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f5f3800d010 tx=0x7f5f3809afa0 comp rx=0 tx=0).stop 2024-09-11T22:01:00.669 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.668+0000 7f5f5806b740 1 -- 172.21.15.1:0/3359672413 shutdown_connections 2024-09-11T22:01:00.669 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.668+0000 7f5f5806b740 1 --2- 172.21.15.1:0/3359672413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559b239df0c0 0x559b239e14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:00.669 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.668+0000 7f5f5806b740 1 --2- 172.21.15.1:0/3359672413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559b239d02f0 0x559b239deb80 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:00.669 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.668+0000 7f5f5806b740 1 --2- 172.21.15.1:0/3359672413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b239cf9d0 0x559b239cfdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:00.669 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.668+0000 7f5f5806b740 1 -- 172.21.15.1:0/3359672413 >> 172.21.15.1:0/3359672413 conn(0x559b239cc060 msgr2=0x559b239cc440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:01:00.669 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.668+0000 7f5f5806b740 1 -- 172.21.15.1:0/3359672413 shutdown_connections 2024-09-11T22:01:00.669 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.668+0000 7f5f5806b740 1 -- 172.21.15.1:0/3359672413 wait complete. 2024-09-11T22:01:00.707 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.706+0000 7f5f5806b740 1 -- start start 2024-09-11T22:01:00.758 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.757+0000 7f5f5806b740 1 Processor -- start 2024-09-11T22:01:00.758 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.757+0000 7f5f5806b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b239cf9d0 0x559b239dc030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:00.758 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.757+0000 7f5f5806b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559b239d02f0 0x559b239dc570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:00.758 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.757+0000 7f5f5806b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559b239df0c0 0x559b239dcab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:00.758 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.757+0000 7f5f5806b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x559b239e1cf0 con 0x559b239df0c0 2024-09-11T22:01:00.758 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.757+0000 7f5f5806b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x559b238bd290 con 0x559b239d02f0 2024-09-11T22:01:00.758 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.757+0000 7f5f5806b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x559b239e1e70 con 0x559b239cf9d0 2024-09-11T22:01:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.758+0000 7f5f4f59e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559b239df0c0 0x559b239dcab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.758+0000 7f5f4f59e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559b239df0c0 0x559b239dcab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45410/0 (socket says 172.21.15.1:45410) 2024-09-11T22:01:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.758+0000 7f5f4f59e640 1 -- 172.21.15.1:0/3828901495 learned_addr learned my addr 172.21.15.1:0/3828901495 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:01:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.758+0000 7f5f4659c640 1 --2- 172.21.15.1:0/3828901495 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559b239d02f0 0x559b239dc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.758+0000 7f5f4ed9d640 1 --2- 172.21.15.1:0/3828901495 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b239cf9d0 0x559b239dc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.758+0000 7f5f4f59e640 1 -- 172.21.15.1:0/3828901495 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b239cf9d0 msgr2=0x559b239dc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.758+0000 7f5f4f59e640 1 --2- 172.21.15.1:0/3828901495 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b239cf9d0 0x559b239dc030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.758+0000 7f5f4f59e640 1 -- 172.21.15.1:0/3828901495 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559b239d02f0 msgr2=0x559b239dc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.758+0000 7f5f4f59e640 1 --2- 172.21.15.1:0/3828901495 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559b239d02f0 0x559b239dc570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.758+0000 7f5f4f59e640 1 -- 172.21.15.1:0/3828901495 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b23647210 con 0x559b239df0c0 2024-09-11T22:01:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.758+0000 7f5f4659c640 1 --2- 172.21.15.1:0/3828901495 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559b239d02f0 0x559b239dc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:01:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.758+0000 7f5f4f59e640 1 --2- 172.21.15.1:0/3828901495 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559b239df0c0 0x559b239dcab0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f5f4800fd70 tx=0x7f5f48012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9835d7980ce5819c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:01:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.758+0000 7f5f367fc640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:01:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.758+0000 7f5f367fc640 1 -- 172.21.15.1:0/3828901495 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f48010430 con 0x559b239df0c0 2024-09-11T22:01:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.758+0000 7f5f367fc640 1 -- 172.21.15.1:0/3828901495 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f5f480105d0 con 0x559b239df0c0 2024-09-11T22:01:00.772 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.772+0000 7f5f5806b740 10 client.0 _ll_register_callbacks cb 0x559b238ee140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:01:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.782+0000 7f5f5806b740 1 -- 172.21.15.1:0/3828901495 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f3809f8e0 con 0x559b239df0c0 2024-09-11T22:01:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.782+0000 7f5f367fc640 1 -- 172.21.15.1:0/3828901495 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5f480107d0 con 0x559b239df0c0 2024-09-11T22:01:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.783+0000 7f5f5806b740 10 client.16119 fetch_fsmap learned FSMap version 30 2024-09-11T22:01:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.783+0000 7f5f5806b740 1 -- 172.21.15.1:0/3828901495 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({fsmap=30}) -- 0x559b23713620 con 0x559b239df0c0 2024-09-11T22:01:00.814 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.813+0000 7f5f367fc640 1 -- 172.21.15.1:0/3828901495 <== mon.0 v2:172.21.15.1:3300/0 4 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f5f48010970 con 0x559b239df0c0 2024-09-11T22:01:00.814 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.813+0000 7f5f5806b740 10 client.16119 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:01:00.814 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.813+0000 7f5f5806b740 10 client.16119 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:01:00.814 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.813+0000 7f5f5806b740 20 client.16119 populate_metadata read hostname 'smithi001' 2024-09-11T22:01:00.814 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.814+0000 7f5f5806b740 1 --2- 172.21.15.1:0/3828901495 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x559b239eeef0 0x559b239f1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:00.815 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.814+0000 7f5f5806b740 4 client.16119 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:01:00.815 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.814+0000 7f5f5806b740 1 -- 172.21.15.1:0/3828901495 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b23941a20 con 0x559b239eeef0 2024-09-11T22:01:00.815 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.814+0000 7f5f4ed9d640 1 --2- 172.21.15.1:0/3828901495 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x559b239eeef0 0x559b239f1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:00.815 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.814+0000 7f5f4ed9d640 1 --2- 172.21.15.1:0/3828901495 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x559b239eeef0 0x559b239f1380 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90dec40bdeb9715 server_cookie=1a2bd3a0fdf6ec09 in_seq=0 out_seq=0 2024-09-11T22:01:00.815 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.814+0000 7f5f367fc640 10 client.16119 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:01:00.816 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.815+0000 7f5f367fc640 1 -- 172.21.15.1:0/3828901495 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559b23941a20 con 0x559b239eeef0 2024-09-11T22:01:00.816 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.815+0000 7f5f367fc640 10 client.16119 handle_command_reply: tid=0 2024-09-11T22:01:00.828 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.828+0000 7f5f5806b740 1 client.16119 shutdown 2024-09-11T22:01:00.829 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.828+0000 7f5f5806b740 1 -- 172.21.15.1:0/3828901495 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559b239df0c0 msgr2=0x559b239dcab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:00.829 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.828+0000 7f5f5806b740 1 --2- 172.21.15.1:0/3828901495 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559b239df0c0 0x559b239dcab0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f5f4800fd70 tx=0x7f5f48012820 comp rx=0 tx=0).stop 2024-09-11T22:01:00.829 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.828+0000 7f5f5806b740 1 -- 172.21.15.1:0/3828901495 shutdown_connections 2024-09-11T22:01:00.829 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.828+0000 7f5f5806b740 1 --2- 172.21.15.1:0/3828901495 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x559b239eeef0 0x559b239f1380 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:00.829 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.828+0000 7f5f5806b740 1 --2- 172.21.15.1:0/3828901495 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559b239df0c0 0x559b239dcab0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:00.829 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.828+0000 7f5f5806b740 1 --2- 172.21.15.1:0/3828901495 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559b239d02f0 0x559b239dc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:00.829 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.828+0000 7f5f367fc640 0 client.16119 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:01:00.829 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.828+0000 7f5f5806b740 1 --2- 172.21.15.1:0/3828901495 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b239cf9d0 0x559b239dc030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:00.829 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.828+0000 7f5f5806b740 1 -- 172.21.15.1:0/3828901495 >> 172.21.15.1:0/3828901495 conn(0x559b239cc060 msgr2=0x559b239d1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:01:00.829 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.828+0000 7f5f5806b740 1 -- 172.21.15.1:0/3828901495 shutdown_connections 2024-09-11T22:01:00.830 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.828+0000 7f5f5806b740 1 -- 172.21.15.1:0/3828901495 wait complete. 2024-09-11T22:01:00.830 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.828+0000 7f5f5806b740 20 client.16119 trim_cache size 0 max 16384 2024-09-11T22:01:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.849+0000 7f5f5806b740 1 Processor -- start 2024-09-11T22:01:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.849+0000 7f5f5806b740 1 -- start start 2024-09-11T22:01:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.849+0000 7f5f5806b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b239cf9d0 0x559b239cb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.849+0000 7f5f5806b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559b239d02f0 0x559b239f6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.849+0000 7f5f5806b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559b239df0c0 0x559b239f7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.849+0000 7f5f5806b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5f501ab840 con 0x559b239d02f0 2024-09-11T22:01:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.849+0000 7f5f5806b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5f501ad3e0 con 0x559b239df0c0 2024-09-11T22:01:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.849+0000 7f5f5806b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5f501abf10 con 0x559b239cf9d0 2024-09-11T22:01:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.849+0000 7f5f4659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559b239d02f0 0x559b239f6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.850+0000 7f5f4659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559b239d02f0 0x559b239f6c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45414/0 (socket says 172.21.15.1:45414) 2024-09-11T22:01:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.850+0000 7f5f4659c640 1 -- 172.21.15.1:0/3158649648 learned_addr learned my addr 172.21.15.1:0/3158649648 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:01:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.850+0000 7f5f4ed9d640 1 --2- 172.21.15.1:0/3158649648 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b239cf9d0 0x559b239cb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.850+0000 7f5f4f59e640 1 --2- 172.21.15.1:0/3158649648 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559b239df0c0 0x559b239f7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.850+0000 7f5f4659c640 1 -- 172.21.15.1:0/3158649648 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b239cf9d0 msgr2=0x559b239cb0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.850+0000 7f5f4659c640 1 --2- 172.21.15.1:0/3158649648 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b239cf9d0 0x559b239cb0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.850+0000 7f5f4659c640 1 -- 172.21.15.1:0/3158649648 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559b239df0c0 msgr2=0x559b239f7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.850+0000 7f5f4659c640 1 --2- 172.21.15.1:0/3158649648 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559b239df0c0 0x559b239f7190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.850+0000 7f5f4659c640 1 -- 172.21.15.1:0/3158649648 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b239d08c0 con 0x559b239d02f0 2024-09-11T22:01:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.850+0000 7f5f4f59e640 1 --2- 172.21.15.1:0/3158649648 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559b239df0c0 0x559b239f7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:01:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.850+0000 7f5f4659c640 1 --2- 172.21.15.1:0/3158649648 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559b239d02f0 0x559b239f6c50 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f5f38004ab0 tx=0x7f5f38004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c64a5eb6dffa2023 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:01:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.850+0000 7f5f44d99640 1 -- 172.21.15.1:0/3158649648 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f380b8040 con 0x559b239d02f0 2024-09-11T22:01:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.850+0000 7f5f44d99640 1 -- 172.21.15.1:0/3158649648 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f3809a980 con 0x559b239d02f0 2024-09-11T22:01:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.850+0000 7f5f44d99640 1 -- 172.21.15.1:0/3158649648 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f380bb040 con 0x559b239d02f0 2024-09-11T22:01:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.851+0000 7f5f5806b740 1 -- 172.21.15.1:0/3158649648 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559b239d02f0 msgr2=0x559b239f6c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.851+0000 7f5f5806b740 1 --2- 172.21.15.1:0/3158649648 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559b239d02f0 0x559b239f6c50 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f5f38004ab0 tx=0x7f5f38004a40 comp rx=0 tx=0).stop 2024-09-11T22:01:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.851+0000 7f5f5806b740 1 -- 172.21.15.1:0/3158649648 shutdown_connections 2024-09-11T22:01:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.851+0000 7f5f5806b740 1 --2- 172.21.15.1:0/3158649648 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559b239df0c0 0x559b239f7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.851+0000 7f5f5806b740 1 --2- 172.21.15.1:0/3158649648 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559b239d02f0 0x559b239f6c50 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.851+0000 7f5f5806b740 1 --2- 172.21.15.1:0/3158649648 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b239cf9d0 0x559b239cb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.851+0000 7f5f5806b740 1 -- 172.21.15.1:0/3158649648 >> 172.21.15.1:0/3158649648 conn(0x559b239cc060 msgr2=0x559b239ca9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:01:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.851+0000 7f5f5806b740 1 -- 172.21.15.1:0/3158649648 shutdown_connections 2024-09-11T22:01:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.851+0000 7f5f5806b740 1 -- 172.21.15.1:0/3158649648 wait complete. 2024-09-11T22:01:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.851+0000 7f5f5806b740 1 -- start start 2024-09-11T22:01:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.852+0000 7f5f5806b740 1 Processor -- start 2024-09-11T22:01:00.857 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.852+0000 7f5f5806b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b239cf9d0 0x559b239e78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:00.857 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.852+0000 7f5f5806b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559b239d02f0 0x559b239e2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:00.857 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.852+0000 7f5f5806b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559b239df0c0 0x559b239e3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:00.857 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.852+0000 7f5f5806b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5f501acce0 con 0x559b239df0c0 2024-09-11T22:01:00.857 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.852+0000 7f5f5806b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5f501ac5e0 con 0x559b239d02f0 2024-09-11T22:01:00.857 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.852+0000 7f5f5806b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5f501a9820 con 0x559b239cf9d0 2024-09-11T22:01:00.857 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.852+0000 7f5f5806b740 10 client.0 _ll_register_callbacks cb 0x7f5f48009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:01:00.857 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.852+0000 7f5f4f59e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559b239df0c0 0x559b239e3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:00.858 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.852+0000 7f5f4ed9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b239cf9d0 0x559b239e78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:00.858 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.852+0000 7f5f4f59e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559b239df0c0 0x559b239e3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45422/0 (socket says 172.21.15.1:45422) 2024-09-11T22:01:00.858 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.852+0000 7f5f4ed9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b239cf9d0 0x559b239e78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:45890/0 (socket says 172.21.15.1:45890) 2024-09-11T22:01:00.858 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.852+0000 7f5f4f59e640 1 -- 172.21.15.1:0/3150482319 learned_addr learned my addr 172.21.15.1:0/3150482319 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:01:00.858 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.852+0000 7f5f4659c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559b239d02f0 0x559b239e2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:00.858 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.852+0000 7f5f4f59e640 1 -- 172.21.15.1:0/3150482319 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b239cf9d0 msgr2=0x559b239e78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:00.858 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.852+0000 7f5f4f59e640 1 --2- 172.21.15.1:0/3150482319 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b239cf9d0 0x559b239e78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:00.858 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.852+0000 7f5f4f59e640 1 -- 172.21.15.1:0/3150482319 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559b239d02f0 msgr2=0x559b239e2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:00.858 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.852+0000 7f5f4f59e640 1 --2- 172.21.15.1:0/3150482319 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559b239d02f0 0x559b239e2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:00.858 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.852+0000 7f5f4f59e640 1 -- 172.21.15.1:0/3150482319 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b239e39c0 con 0x559b239df0c0 2024-09-11T22:01:00.858 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.852+0000 7f5f4ed9d640 1 --2- 172.21.15.1:0/3150482319 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b239cf9d0 0x559b239e78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:01:00.858 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.852+0000 7f5f4f59e640 1 --2- 172.21.15.1:0/3150482319 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559b239df0c0 0x559b239e3190 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f5f48012860 tx=0x7f5f48004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4a3ca4e87e27344 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:01:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.852+0000 7f5f367fc640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:01:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.853+0000 7f5f367fc640 1 -- 172.21.15.1:0/3150482319 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f48026680 con 0x559b239df0c0 2024-09-11T22:01:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.853+0000 7f5f5806b740 1 -- 172.21.15.1:0/3150482319 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f3809a980 con 0x559b239df0c0 2024-09-11T22:01:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.853+0000 7f5f367fc640 1 -- 172.21.15.1:0/3150482319 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f5f48026820 con 0x559b239df0c0 2024-09-11T22:01:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.853+0000 7f5f367fc640 1 -- 172.21.15.1:0/3150482319 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5f48026a20 con 0x559b239df0c0 2024-09-11T22:01:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.853+0000 7f5f5806b740 10 client.16125 fetch_fsmap learned FSMap version 30 2024-09-11T22:01:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.853+0000 7f5f5806b740 1 -- 172.21.15.1:0/3150482319 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559b23a59cd0 con 0x559b239df0c0 2024-09-11T22:01:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.853+0000 7f5f367fc640 1 -- 172.21.15.1:0/3150482319 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f3809a980 con 0x559b239df0c0 2024-09-11T22:01:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.853+0000 7f5f367fc640 1 -- 172.21.15.1:0/3150482319 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f5f48039950 con 0x559b239df0c0 2024-09-11T22:01:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.853+0000 7f5f5806b740 10 client.16125 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:01:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.853+0000 7f5f5806b740 10 client.16125 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:01:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.853+0000 7f5f5806b740 20 client.16125 populate_metadata read hostname 'smithi001' 2024-09-11T22:01:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.853+0000 7f5f5806b740 1 --2- 172.21.15.1:0/3150482319 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x559b23a5a330 0x559b23a5c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.853+0000 7f5f5806b740 4 client.16125 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:01:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.853+0000 7f5f5806b740 1 -- 172.21.15.1:0/3150482319 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559b2393bd70 con 0x559b23a5a330 2024-09-11T22:01:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.854+0000 7f5f4ed9d640 1 --2- 172.21.15.1:0/3150482319 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x559b23a5a330 0x559b23a5c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.854+0000 7f5f367fc640 10 client.16125 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:01:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.854+0000 7f5f4ed9d640 1 --2- 172.21.15.1:0/3150482319 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x559b23a5a330 0x559b23a5c720 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2699cc37dbf6196f server_cookie=4d8fd29c045a0b40 in_seq=0 out_seq=0 2024-09-11T22:01:00.872 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.871+0000 7f5f367fc640 1 -- 172.21.15.1:0/3150482319 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559b2393bd70 con 0x559b23a5a330 2024-09-11T22:01:00.872 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.871+0000 7f5f367fc640 10 client.16125 handle_command_reply: tid=0 2024-09-11T22:01:00.873 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.871+0000 7f5f5806b740 1 client.16125 shutdown 2024-09-11T22:01:00.873 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.872+0000 7f5f5806b740 1 -- 172.21.15.1:0/3150482319 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559b239df0c0 msgr2=0x559b239e3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:00.873 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.872+0000 7f5f5806b740 1 --2- 172.21.15.1:0/3150482319 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559b239df0c0 0x559b239e3190 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f5f48012860 tx=0x7f5f48004a40 comp rx=0 tx=0).stop 2024-09-11T22:01:00.873 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.872+0000 7f5f5806b740 1 -- 172.21.15.1:0/3150482319 shutdown_connections 2024-09-11T22:01:00.873 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.872+0000 7f5f5806b740 1 --2- 172.21.15.1:0/3150482319 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x559b23a5a330 0x559b23a5c720 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:00.873 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.872+0000 7f5f5806b740 1 --2- 172.21.15.1:0/3150482319 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559b239df0c0 0x559b239e3190 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:00.873 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.872+0000 7f5f5806b740 1 --2- 172.21.15.1:0/3150482319 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559b239d02f0 0x559b239e2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:00.873 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.872+0000 7f5f5806b740 1 --2- 172.21.15.1:0/3150482319 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559b239cf9d0 0x559b239e78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:00.873 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.872+0000 7f5f367fc640 0 client.16125 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:01:00.873 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.872+0000 7f5f5806b740 1 -- 172.21.15.1:0/3150482319 >> 172.21.15.1:0/3150482319 conn(0x559b239cc060 msgr2=0x559b239dbe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:01:00.873 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.872+0000 7f5f5806b740 1 -- 172.21.15.1:0/3150482319 shutdown_connections 2024-09-11T22:01:00.873 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.872+0000 7f5f5806b740 1 -- 172.21.15.1:0/3150482319 wait complete. 2024-09-11T22:01:00.874 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.872+0000 7f5f5806b740 20 client.16125 trim_cache size 0 max 16384 2024-09-11T22:01:00.874 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-09-11T22:01:00.874 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-09-11T22:01:00.874 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-09-11T22:01:00.874 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-09-11T22:01:00.875 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.872+0000 7f5f551f6640 1 -- 172.21.15.1:0/804916181 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f5f3c0a2e80 msgr2=0x7f5f3c0a5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:00.875 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.872+0000 7f5f551f6640 1 --2- 172.21.15.1:0/804916181 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f5f3c0a2e80 0x7f5f3c0a5340 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5f4002a990 tx=0x7f5f4002a920 comp rx=0 tx=0).stop 2024-09-11T22:01:00.875 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.872+0000 7f5f551f6640 1 -- 172.21.15.1:0/804916181 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f50102460 msgr2=0x7f5f50198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:00.875 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.872+0000 7f5f551f6640 1 --2- 172.21.15.1:0/804916181 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f50102460 0x7f5f50198990 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f5f3800e9f0 tx=0x7f5f3800edf0 comp rx=0 tx=0).stop 2024-09-11T22:01:00.875 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.872+0000 7f5f551f6640 1 -- 172.21.15.1:0/804916181 shutdown_connections 2024-09-11T22:01:00.875 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.872+0000 7f5f551f6640 1 --2- 172.21.15.1:0/804916181 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f5f3c0a2e80 0x7f5f3c0a5340 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:00.875 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.872+0000 7f5f551f6640 1 --2- 172.21.15.1:0/804916181 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5f50107ad0 0x7f5f5019cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:00.875 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.872+0000 7f5f551f6640 1 --2- 172.21.15.1:0/804916181 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f50102460 0x7f5f50198990 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:00.876 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.872+0000 7f5f551f6640 1 --2- 172.21.15.1:0/804916181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5f50101ac0 0x7f5f50198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:00.876 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.872+0000 7f5f551f6640 1 -- 172.21.15.1:0/804916181 >> 172.21.15.1:0/804916181 conn(0x7f5f500fd7a0 msgr2=0x7f5f500fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:01:00.876 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.873+0000 7f5f551f6640 1 -- 172.21.15.1:0/804916181 shutdown_connections 2024-09-11T22:01:00.876 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:00.873+0000 7f5f551f6640 1 -- 172.21.15.1:0/804916181 wait complete. 2024-09-11T22:01:00.900 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:01:00.900 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:01:01.180 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:01:00 smithi053 ceph-mon[34908]: pgmap v118: 97 pgs: 97 active+clean; 3.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-11T22:01:01.191 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:01:00 smithi161 ceph-mon[35615]: pgmap v118: 97 pgs: 97 active+clean; 3.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-11T22:01:01.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:01:00 smithi001 ceph-mon[30935]: pgmap v118: 97 pgs: 97 active+clean; 3.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-09-11T22:01:01.211 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T22:01:01.211 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T22:01:01.217 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:01:01.218 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:01:01.218 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 91 -lt 100 ']' 2024-09-11T22:01:01.218 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 91 + 1 2024-09-11T22:01:01.299 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=92 2024-09-11T22:01:01.300 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T22:01:01.382 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:01:01.382 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:01:01.383 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T22:01:01.384 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T22:01:01.384 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T22:01:02.175 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:01:02.175 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:01:02.325 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T22:01:02.326 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T22:01:02.340 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:01:02.340 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:01:02.340 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 92 -lt 100 ']' 2024-09-11T22:01:02.340 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 92 + 1 2024-09-11T22:01:02.409 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=93 2024-09-11T22:01:02.409 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T22:01:02.442 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:01:02.442 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:01:02.443 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T22:01:02.443 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T22:01:02.443 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T22:01:03.073 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:01:03.073 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:01:03.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:01:02 smithi001 ceph-mon[30935]: pgmap v119: 97 pgs: 97 active+clean; 3.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-09-11T22:01:03.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:01:02 smithi053 ceph-mon[34908]: pgmap v119: 97 pgs: 97 active+clean; 3.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-09-11T22:01:03.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:01:02 smithi161 ceph-mon[35615]: pgmap v119: 97 pgs: 97 active+clean; 3.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-09-11T22:01:03.456 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T22:01:03.457 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T22:01:03.473 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:01:03.474 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:01:03.474 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 93 -lt 100 ']' 2024-09-11T22:01:03.474 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 93 + 1 2024-09-11T22:01:03.520 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=94 2024-09-11T22:01:03.520 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T22:01:03.566 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:01:03.566 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:01:03.567 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T22:01:03.567 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T22:01:03.567 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T22:01:04.691 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:01:04.691 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:01:04.925 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T22:01:04.926 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T22:01:04.949 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:01:04.949 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:01:04.949 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 94 -lt 100 ']' 2024-09-11T22:01:04.949 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 94 + 1 2024-09-11T22:01:05.013 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=95 2024-09-11T22:01:05.013 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T22:01:05.040 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:01:05.040 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:01:05.041 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T22:01:05.042 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T22:01:05.042 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T22:01:05.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:01:04 smithi001 ceph-mon[30935]: pgmap v120: 97 pgs: 97 active+clean; 3.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-09-11T22:01:05.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:01:04 smithi053 ceph-mon[34908]: pgmap v120: 97 pgs: 97 active+clean; 3.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-09-11T22:01:05.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:01:04 smithi161 ceph-mon[35615]: pgmap v120: 97 pgs: 97 active+clean; 3.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-09-11T22:01:05.858 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:01:05.858 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:01:06.031 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T22:01:06.031 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T22:01:06.053 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:01:06.053 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:01:06.053 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 95 -lt 100 ']' 2024-09-11T22:01:06.054 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 95 + 1 2024-09-11T22:01:06.087 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=96 2024-09-11T22:01:06.087 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T22:01:06.117 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:01:06.117 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:01:06.119 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T22:01:06.119 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T22:01:06.119 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T22:01:06.740 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:01:06.741 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:01:06.966 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T22:01:06.966 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T22:01:06.989 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:01:06.990 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:01:06.990 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 96 -lt 100 ']' 2024-09-11T22:01:06.990 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 96 + 1 2024-09-11T22:01:07.063 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=97 2024-09-11T22:01:07.063 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T22:01:07.124 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:01:07.125 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:01:07.126 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T22:01:07.126 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T22:01:07.126 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T22:01:07.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:01:06 smithi001 ceph-mon[30935]: pgmap v121: 97 pgs: 97 active+clean; 3.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 50 KiB/s wr, 36 op/s 2024-09-11T22:01:07.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:01:06 smithi053 ceph-mon[34908]: pgmap v121: 97 pgs: 97 active+clean; 3.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 50 KiB/s wr, 36 op/s 2024-09-11T22:01:07.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:01:06 smithi161 ceph-mon[35615]: pgmap v121: 97 pgs: 97 active+clean; 3.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 50 KiB/s wr, 36 op/s 2024-09-11T22:01:08.141 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:01:08.141 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:01:08.514 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T22:01:08.514 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T22:01:08.523 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:01:08.523 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:01:08.523 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 97 -lt 100 ']' 2024-09-11T22:01:08.523 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 97 + 1 2024-09-11T22:01:08.589 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=98 2024-09-11T22:01:08.589 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T22:01:08.631 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:01:08.631 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:01:08.633 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T22:01:08.633 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T22:01:08.633 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T22:01:08.997 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:01:09.020 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:01:09.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:01:08 smithi001 ceph-mon[30935]: pgmap v122: 97 pgs: 97 active+clean; 3.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-09-11T22:01:09.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:01:08 smithi053 ceph-mon[34908]: pgmap v122: 97 pgs: 97 active+clean; 3.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-09-11T22:01:09.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:01:08 smithi161 ceph-mon[35615]: pgmap v122: 97 pgs: 97 active+clean; 3.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-09-11T22:01:09.582 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:01:09.583 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:01:09.933 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T22:01:09.933 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T22:01:09.964 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:01:09.965 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:01:09.965 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 98 -lt 100 ']' 2024-09-11T22:01:09.965 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 98 + 1 2024-09-11T22:01:10.011 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=99 2024-09-11T22:01:10.011 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T22:01:10.048 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:01:10.048 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:01:10.049 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T22:01:10.049 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T22:01:10.049 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T22:01:10.874 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:01:10.874 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:01:11.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:01:10 smithi001 ceph-mon[30935]: pgmap v123: 97 pgs: 97 active+clean; 3.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 53 KiB/s wr, 35 op/s 2024-09-11T22:01:11.233 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T22:01:11.233 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T22:01:11.272 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:01:11.272 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:01:11.273 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 99 -lt 100 ']' 2024-09-11T22:01:11.273 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 99 + 1 2024-09-11T22:01:11.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:01:10 smithi053 ceph-mon[34908]: pgmap v123: 97 pgs: 97 active+clean; 3.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 53 KiB/s wr, 35 op/s 2024-09-11T22:01:11.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:01:10 smithi161 ceph-mon[35615]: pgmap v123: 97 pgs: 97 active+clean; 3.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 53 KiB/s wr, 35 op/s 2024-09-11T22:01:11.346 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=100 2024-09-11T22:01:11.346 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T22:01:11.391 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:01:11.391 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:01:11.392 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2024-09-11T22:01:11.392 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2024-09-11T22:01:11.392 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2024-09-11T22:01:12.382 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2024-09-11T22:01:12.382 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2024-09-11T22:01:12.623 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2024-09-11T22:01:12.623 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T22:01:12.663 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:01:12.663 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2024-09-11T22:01:12.663 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 100 -lt 100 ']' 2024-09-11T22:01:12.675 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2024-09-11T22:01:12.749 INFO:tasks.workunit.client.0.smithi001.stderr:+ setfattr -n system.posix_acl_access -v 0x02000000 d1 2024-09-11T22:01:12.784 INFO:tasks.workunit.client.0.smithi001.stderr:+ setfattr -n system.posix_acl_default -v 0x02000000 . 2024-09-11T22:01:12.786 INFO:tasks.workunit.client.0.smithi001.stderr:+ expect_failure getfattr -n system.posix_acl_access d1 2024-09-11T22:01:12.786 INFO:tasks.workunit.client.0.smithi001.stderr:+ getfattr -n system.posix_acl_access d1 2024-09-11T22:01:12.794 INFO:tasks.workunit.client.0.smithi001.stderr:d1: system.posix_acl_access: No such attribute 2024-09-11T22:01:12.795 INFO:tasks.workunit.client.0.smithi001.stderr:+ return 0 2024-09-11T22:01:12.795 INFO:tasks.workunit.client.0.smithi001.stderr:+ expect_failure getfattr -n system.posix_acl_default . 2024-09-11T22:01:12.795 INFO:tasks.workunit.client.0.smithi001.stderr:+ getfattr -n system.posix_acl_default . 2024-09-11T22:01:12.795 INFO:tasks.workunit.client.0.smithi001.stderr:.: system.posix_acl_default: No such attribute 2024-09-11T22:01:12.795 INFO:tasks.workunit.client.0.smithi001.stderr:+ return 0 2024-09-11T22:01:12.795 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2024-09-11T22:01:12.798 INFO:tasks.workunit.client.0.smithi001.stderr:+ cd .. 2024-09-11T22:01:12.805 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir testdir 2024-09-11T22:01:12.812 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo OK 2024-09-11T22:01:12.813 INFO:tasks.workunit.client.0.smithi001.stdout:OK 2024-09-11T22:01:12.814 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-11T22:01:12.814 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-11T22:01:13.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:01:12 smithi001 ceph-mon[30935]: pgmap v124: 97 pgs: 97 active+clean; 3.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 46 KiB/s wr, 30 op/s 2024-09-11T22:01:13.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:01:12 smithi053 ceph-mon[34908]: pgmap v124: 97 pgs: 97 active+clean; 3.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 46 KiB/s wr, 30 op/s 2024-09-11T22:01:13.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:01:12 smithi161 ceph-mon[35615]: pgmap v124: 97 pgs: 97 active+clean; 3.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 46 KiB/s wr, 30 op/s 2024-09-11T22:01:13.887 INFO:tasks.workunit:Running workunit fs/misc/chmod.sh... 2024-09-11T22:01:13.887 DEBUG:teuthology.orchestra.run.smithi001:workunit test fs/misc/chmod.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=086c737fa75c22d521a3932c59b31c5e1d1c9ef4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/chmod.sh 2024-09-11T22:01:14.093 INFO:tasks.workunit.client.0.smithi001.stderr:+ set -e 2024-09-11T22:01:14.093 INFO:tasks.workunit.client.0.smithi001.stderr:+ file=test_chmod.91205 2024-09-11T22:01:14.093 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo foo 2024-09-11T22:01:14.097 INFO:tasks.workunit.client.0.smithi001.stderr:+ test 0 '!=' 0 2024-09-11T22:01:14.097 INFO:tasks.workunit.client.0.smithi001.stderr:+ chmod 400 test_chmod.91205 2024-09-11T22:01:14.143 INFO:tasks.workunit.client.0.smithi001.stderr:+ test 0 '!=' 0 2024-09-11T22:01:14.144 INFO:tasks.workunit.client.0.smithi001.stderr:+ check_perms test_chmod.91205 -r-------- 2024-09-11T22:01:14.144 INFO:tasks.workunit.client.0.smithi001.stderr:+ file=test_chmod.91205 2024-09-11T22:01:14.144 INFO:tasks.workunit.client.0.smithi001.stderr:++ ls -la test_chmod.91205 2024-09-11T22:01:14.205 INFO:tasks.workunit.client.0.smithi001.stderr:+ r='-r--------. 1 ubuntu ubuntu 4 Sep 11 22:01 test_chmod.91205' 2024-09-11T22:01:14.205 INFO:tasks.workunit.client.0.smithi001.stderr:+ test 0 '!=' 0 2024-09-11T22:01:14.205 INFO:tasks.workunit.client.0.smithi001.stderr:+ perms=-r-------- 2024-09-11T22:01:14.205 INFO:tasks.workunit.client.0.smithi001.stderr:++ echo -r--------. 1 ubuntu ubuntu 4 Sep 11 22:01 test_chmod.91205 2024-09-11T22:01:14.205 INFO:tasks.workunit.client.0.smithi001.stderr:++ awk '{print $1}' 2024-09-11T22:01:14.341 INFO:tasks.workunit.client.0.smithi001.stderr:+ test -r-------- '!=' -r--------. 2024-09-11T22:01:14.342 INFO:tasks.workunit.client.0.smithi001.stderr:++ echo -r--------. 1 ubuntu ubuntu 4 Sep 11 22:01 test_chmod.91205 2024-09-11T22:01:14.342 INFO:tasks.workunit.client.0.smithi001.stderr:++ awk '{print $1}' 2024-09-11T22:01:14.344 INFO:tasks.workunit.client.0.smithi001.stderr:+ test -r--------. '!=' -r--------. 2024-09-11T22:01:14.344 INFO:tasks.workunit.client.0.smithi001.stderr:+ set +e 2024-09-11T22:01:14.344 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo bar 2024-09-11T22:01:14.344 INFO:tasks.workunit.client.0.smithi001.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/chmod.sh: line 40: test_chmod.91205: Permission denied 2024-09-11T22:01:14.344 INFO:tasks.workunit.client.0.smithi001.stderr:+ test 1 = 0 2024-09-11T22:01:14.344 INFO:tasks.workunit.client.0.smithi001.stderr:+ set -e 2024-09-11T22:01:14.344 INFO:tasks.workunit.client.0.smithi001.stderr:+ chmod 600 test_chmod.91205 2024-09-11T22:01:14.345 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo bar 2024-09-11T22:01:14.345 INFO:tasks.workunit.client.0.smithi001.stderr:+ test 0 '!=' 0 2024-09-11T22:01:14.345 INFO:tasks.workunit.client.0.smithi001.stderr:+ check_perms test_chmod.91205 -rw------- 2024-09-11T22:01:14.345 INFO:tasks.workunit.client.0.smithi001.stderr:+ file=test_chmod.91205 2024-09-11T22:01:14.345 INFO:tasks.workunit.client.0.smithi001.stderr:++ ls -la test_chmod.91205 2024-09-11T22:01:14.346 INFO:tasks.workunit.client.0.smithi001.stderr:+ r='-rw-------. 1 ubuntu ubuntu 8 Sep 11 22:01 test_chmod.91205' 2024-09-11T22:01:14.346 INFO:tasks.workunit.client.0.smithi001.stderr:+ test 0 '!=' 0 2024-09-11T22:01:14.346 INFO:tasks.workunit.client.0.smithi001.stderr:+ perms=-rw------- 2024-09-11T22:01:14.347 INFO:tasks.workunit.client.0.smithi001.stderr:++ echo -rw-------. 1 ubuntu ubuntu 8 Sep 11 22:01 test_chmod.91205 2024-09-11T22:01:14.347 INFO:tasks.workunit.client.0.smithi001.stderr:++ awk '{print $1}' 2024-09-11T22:01:14.348 INFO:tasks.workunit.client.0.smithi001.stderr:+ test -rw------- '!=' -rw-------. 2024-09-11T22:01:14.349 INFO:tasks.workunit.client.0.smithi001.stderr:++ echo -rw-------. 1 ubuntu ubuntu 8 Sep 11 22:01 test_chmod.91205 2024-09-11T22:01:14.349 INFO:tasks.workunit.client.0.smithi001.stderr:++ awk '{print $1}' 2024-09-11T22:01:14.351 INFO:tasks.workunit.client.0.smithi001.stderr:+ test -rw-------. '!=' -rw-------. 2024-09-11T22:01:14.351 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo foo 2024-09-11T22:01:14.351 INFO:tasks.workunit.client.0.smithi001.stderr:+ test 0 '!=' 0 2024-09-11T22:01:14.352 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-11T22:01:14.352 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-11T22:01:14.401 INFO:tasks.workunit:Running workunit fs/misc/dac_override.sh... 2024-09-11T22:01:14.401 DEBUG:teuthology.orchestra.run.smithi001:workunit test fs/misc/dac_override.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=086c737fa75c22d521a3932c59b31c5e1d1c9ef4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/dac_override.sh 2024-09-11T22:01:15.188 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:01:14 smithi001 ceph-mon[30935]: pgmap v125: 97 pgs: 97 active+clean; 3.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 46 KiB/s wr, 30 op/s 2024-09-11T22:01:15.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:01:14 smithi053 ceph-mon[34908]: pgmap v125: 97 pgs: 97 active+clean; 3.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 46 KiB/s wr, 30 op/s 2024-09-11T22:01:15.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:01:14 smithi161 ceph-mon[35615]: pgmap v125: 97 pgs: 97 active+clean; 3.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 46 KiB/s wr, 30 op/s 2024-09-11T22:01:15.485 INFO:tasks.workunit.client.0.smithi001.stderr:+ set -e 2024-09-11T22:01:15.485 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir -p testdir 2024-09-11T22:01:15.487 INFO:tasks.workunit.client.0.smithi001.stderr:+ file=test_chmod.91270 2024-09-11T22:01:15.487 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo foo 2024-09-11T22:01:15.489 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo chmod 600 testdir 2024-09-11T22:01:15.504 INFO:tasks.workunit.client.0.smithi001.stderr:+ expect_failure cat testdir/test_chmod.91270 2024-09-11T22:01:15.505 INFO:tasks.workunit.client.0.smithi001.stderr:+ cat testdir/test_chmod.91270 2024-09-11T22:01:15.505 INFO:tasks.workunit.client.0.smithi001.stderr:cat: testdir/test_chmod.91270: Permission denied 2024-09-11T22:01:15.505 INFO:tasks.workunit.client.0.smithi001.stderr:+ return 0 2024-09-11T22:01:15.505 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo cat testdir/test_chmod.91270 2024-09-11T22:01:15.517 INFO:tasks.workunit.client.0.smithi001.stdout:foo 2024-09-11T22:01:15.519 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-11T22:01:15.519 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-11T22:01:15.571 INFO:tasks.workunit:Running workunit fs/misc/direct_io.py... 2024-09-11T22:01:15.571 DEBUG:teuthology.orchestra.run.smithi001:workunit test fs/misc/direct_io.py> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=086c737fa75c22d521a3932c59b31c5e1d1c9ef4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/direct_io.py 2024-09-11T22:01:16.886 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:16.885+0000 7f8ebefff640 1 -- 172.21.15.1:0/2874924404 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8eb8101ac0 msgr2=0x7f8eb8101f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:16.886 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:16.885+0000 7f8ebefff640 1 --2- 172.21.15.1:0/2874924404 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8eb8101ac0 0x7f8eb8101f20 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f8ea0009930 tx=0x7f8ea002a440 comp rx=0 tx=0).stop 2024-09-11T22:01:16.886 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:16.885+0000 7f8ebefff640 1 -- 172.21.15.1:0/2874924404 shutdown_connections 2024-09-11T22:01:16.886 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:16.885+0000 7f8ebefff640 1 --2- 172.21.15.1:0/2874924404 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8eb8102460 0x7f8eb810e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:16.886 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:16.885+0000 7f8ebefff640 1 --2- 172.21.15.1:0/2874924404 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8eb8101ac0 0x7f8eb8101f20 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:16.886 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:16.885+0000 7f8ebefff640 1 --2- 172.21.15.1:0/2874924404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8eb8107ad0 0x7f8eb8107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:16.886 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:16.885+0000 7f8ebefff640 1 -- 172.21.15.1:0/2874924404 >> 172.21.15.1:0/2874924404 conn(0x7f8eb80fd7a0 msgr2=0x7f8eb80ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:01:16.886 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:16.885+0000 7f8ebefff640 1 -- 172.21.15.1:0/2874924404 shutdown_connections 2024-09-11T22:01:16.887 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:16.885+0000 7f8ebefff640 1 -- 172.21.15.1:0/2874924404 wait complete. 2024-09-11T22:01:16.907 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:16.905+0000 7f8ebefff640 1 Processor -- start 2024-09-11T22:01:16.907 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:16.906+0000 7f8ebefff640 1 -- start start 2024-09-11T22:01:16.907 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:16.906+0000 7f8ebefff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8eb8101ac0 0x7f8eb8198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:16.907 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:16.906+0000 7f8ebefff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8eb8102460 0x7f8eb81989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:16.908 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:16.906+0000 7f8ebefff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8eb8107ad0 0x7f8eb819cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:16.908 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:16.906+0000 7f8ebefff640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb8111f10 con 0x7f8eb8107ad0 2024-09-11T22:01:16.908 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:16.906+0000 7f8ebefff640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb8111d90 con 0x7f8eb8102460 2024-09-11T22:01:16.908 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:16.906+0000 7f8ebefff640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb8112090 con 0x7f8eb8101ac0 2024-09-11T22:01:16.908 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:16.906+0000 7f8ebd59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8eb8107ad0 0x7f8eb819cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:16.908 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:16.906+0000 7f8ebd59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8eb8107ad0 0x7f8eb819cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58578/0 (socket says 172.21.15.1:58578) 2024-09-11T22:01:16.908 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:16.906+0000 7f8ebd59c640 1 -- 172.21.15.1:0/2130968134 learned_addr learned my addr 172.21.15.1:0/2130968134 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:01:16.908 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:16.906+0000 7f8ebcd9b640 1 --2- 172.21.15.1:0/2130968134 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8eb8101ac0 0x7f8eb8198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:16.908 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:16.906+0000 7f8eb7fff640 1 --2- 172.21.15.1:0/2130968134 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8eb8102460 0x7f8eb81989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:16.908 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:16.906+0000 7f8ebd59c640 1 -- 172.21.15.1:0/2130968134 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8eb8101ac0 msgr2=0x7f8eb8198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:16.908 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:16.906+0000 7f8ebd59c640 1 --2- 172.21.15.1:0/2130968134 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8eb8101ac0 0x7f8eb8198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:16.908 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:16.906+0000 7f8ebd59c640 1 -- 172.21.15.1:0/2130968134 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8eb8102460 msgr2=0x7f8eb81989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:16.908 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:16.906+0000 7f8ebd59c640 1 --2- 172.21.15.1:0/2130968134 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8eb8102460 0x7f8eb81989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:16.909 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:16.906+0000 7f8ebd59c640 1 -- 172.21.15.1:0/2130968134 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eb819d450 con 0x7f8eb8107ad0 2024-09-11T22:01:16.909 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:16.907+0000 7f8ebd59c640 1 --2- 172.21.15.1:0/2130968134 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8eb8107ad0 0x7f8eb819cd20 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f8eb000efd0 tx=0x7f8eb000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:01:16.909 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:16.907+0000 7f8eb5ffb640 1 -- 172.21.15.1:0/2130968134 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb0003b70 con 0x7f8eb8107ad0 2024-09-11T22:01:16.909 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:16.907+0000 7f8ebefff640 1 -- 172.21.15.1:0/2130968134 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8eb819d740 con 0x7f8eb8107ad0 2024-09-11T22:01:16.910 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:16.907+0000 7f8eb5ffb640 1 -- 172.21.15.1:0/2130968134 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8eb0003d10 con 0x7f8eb8107ad0 2024-09-11T22:01:16.910 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:16.907+0000 7f8eb5ffb640 1 -- 172.21.15.1:0/2130968134 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb00109f0 con 0x7f8eb8107ad0 2024-09-11T22:01:16.910 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:16.908+0000 7f8ebefff640 1 -- 172.21.15.1:0/2130968134 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8eb81a4fd0 con 0x7f8eb8107ad0 2024-09-11T22:01:16.910 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:16.910+0000 7f8eb5ffb640 1 -- 172.21.15.1:0/2130968134 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f8eb0004020 con 0x7f8eb8107ad0 2024-09-11T22:01:16.911 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:16.910+0000 7f8eb5ffb640 1 --2- 172.21.15.1:0/2130968134 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f8ea40a2e40 0x7f8ea40a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:16.911 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:16.911+0000 7f8eb5ffb640 1 -- 172.21.15.1:0/2130968134 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f8eb00cf7b0 con 0x7f8eb8107ad0 2024-09-11T22:01:16.911 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:16.911+0000 7f8ebcd9b640 1 --2- 172.21.15.1:0/2130968134 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f8ea40a2e40 0x7f8ea40a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:16.912 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:16.911+0000 7f8ebcd9b640 1 --2- 172.21.15.1:0/2130968134 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f8ea40a2e40 0x7f8ea40a5300 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8005d00 tx=0x7f8ea800b410 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:01:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.004+0000 7f8ec1e6e740 1 Processor -- start 2024-09-11T22:01:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.004+0000 7f8ec1e6e740 1 -- start start 2024-09-11T22:01:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.004+0000 7f8ec1e6e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c6cf81faf0 0x55c6cf81fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.004+0000 7f8ec1e6e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c6cf820410 0x55c6cf82a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.004+0000 7f8ec1e6e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6cf82acb0 0x55c6cf82d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.004+0000 7f8ec1e6e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55c6cf7842f0 con 0x55c6cf820410 2024-09-11T22:01:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.004+0000 7f8ec1e6e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55c6cf72c270 con 0x55c6cf81faf0 2024-09-11T22:01:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.004+0000 7f8ec1e6e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c6cf7802b0 con 0x55c6cf82acb0 2024-09-11T22:01:17.010 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.005+0000 7f8eb7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c6cf820410 0x55c6cf82a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:17.010 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.005+0000 7f8eb7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c6cf820410 0x55c6cf82a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58580/0 (socket says 172.21.15.1:58580) 2024-09-11T22:01:17.010 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.005+0000 7f8eb7fff640 1 -- 172.21.15.1:0/2676111214 learned_addr learned my addr 172.21.15.1:0/2676111214 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:01:17.010 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.005+0000 7f8ebcd9b640 1 --2- 172.21.15.1:0/2676111214 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c6cf81faf0 0x55c6cf81fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:17.010 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.005+0000 7f8ebd59c640 1 --2- 172.21.15.1:0/2676111214 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6cf82acb0 0x55c6cf82d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:17.010 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.005+0000 7f8eb7fff640 1 -- 172.21.15.1:0/2676111214 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6cf82acb0 msgr2=0x55c6cf82d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:17.010 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.005+0000 7f8eb7fff640 1 --2- 172.21.15.1:0/2676111214 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6cf82acb0 0x55c6cf82d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.010 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.005+0000 7f8eb7fff640 1 -- 172.21.15.1:0/2676111214 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c6cf81faf0 msgr2=0x55c6cf81fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:17.010 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.005+0000 7f8eb7fff640 1 --2- 172.21.15.1:0/2676111214 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c6cf81faf0 0x55c6cf81fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.010 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.005+0000 7f8eb7fff640 1 -- 172.21.15.1:0/2676111214 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6cf72b340 con 0x55c6cf820410 2024-09-11T22:01:17.010 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.005+0000 7f8ebcd9b640 1 --2- 172.21.15.1:0/2676111214 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c6cf81faf0 0x55c6cf81fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:01:17.010 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.005+0000 7f8eb7fff640 1 --2- 172.21.15.1:0/2676111214 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c6cf820410 0x55c6cf82a770 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f8ea002a960 tx=0x7f8ea0005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40749c294d9f6790 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:01:17.010 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.005+0000 7f8eaeffd640 1 -- 172.21.15.1:0/2676111214 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea0040070 con 0x55c6cf820410 2024-09-11T22:01:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.006+0000 7f8eaeffd640 1 -- 172.21.15.1:0/2676111214 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ea00318f0 con 0x55c6cf820410 2024-09-11T22:01:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.006+0000 7f8eaeffd640 1 -- 172.21.15.1:0/2676111214 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea0035040 con 0x55c6cf820410 2024-09-11T22:01:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.006+0000 7f8ec1e6e740 1 -- 172.21.15.1:0/2676111214 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c6cf820410 msgr2=0x55c6cf82a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.006+0000 7f8ec1e6e740 1 --2- 172.21.15.1:0/2676111214 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c6cf820410 0x55c6cf82a770 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f8ea002a960 tx=0x7f8ea0005c00 comp rx=0 tx=0).stop 2024-09-11T22:01:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.006+0000 7f8ec1e6e740 1 -- 172.21.15.1:0/2676111214 shutdown_connections 2024-09-11T22:01:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.006+0000 7f8ec1e6e740 1 --2- 172.21.15.1:0/2676111214 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6cf82acb0 0x55c6cf82d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.006+0000 7f8ec1e6e740 1 --2- 172.21.15.1:0/2676111214 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c6cf820410 0x55c6cf82a770 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.006+0000 7f8ec1e6e740 1 --2- 172.21.15.1:0/2676111214 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c6cf81faf0 0x55c6cf81fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.006+0000 7f8ec1e6e740 1 -- 172.21.15.1:0/2676111214 >> 172.21.15.1:0/2676111214 conn(0x55c6cf81c060 msgr2=0x55c6cf81c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:01:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.007+0000 7f8ec1e6e740 1 -- 172.21.15.1:0/2676111214 shutdown_connections 2024-09-11T22:01:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.007+0000 7f8ec1e6e740 1 -- 172.21.15.1:0/2676111214 wait complete. 2024-09-11T22:01:17.032 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.032+0000 7f8ec1e6e740 1 -- start start 2024-09-11T22:01:17.055 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.055+0000 7f8ec1e6e740 1 Processor -- start 2024-09-11T22:01:17.057 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.056+0000 7f8ec1e6e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c6cf81faf0 0x55c6cf827b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:17.057 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.056+0000 7f8ec1e6e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6cf820410 0x55c6cf828070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:17.057 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.056+0000 7f8ec1e6e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c6cf82acb0 0x55c6cf8285b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:17.057 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.056+0000 7f8ec1e6e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55c6cf82d880 con 0x55c6cf81faf0 2024-09-11T22:01:17.057 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.056+0000 7f8ec1e6e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55c6cf70d290 con 0x55c6cf82acb0 2024-09-11T22:01:17.057 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.056+0000 7f8ec1e6e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c6cf82da00 con 0x55c6cf820410 2024-09-11T22:01:17.057 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.056+0000 7f8ebcd9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c6cf81faf0 0x55c6cf827b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:17.057 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.056+0000 7f8ebcd9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c6cf81faf0 0x55c6cf827b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58582/0 (socket says 172.21.15.1:58582) 2024-09-11T22:01:17.057 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.056+0000 7f8ebcd9b640 1 -- 172.21.15.1:0/3758379176 learned_addr learned my addr 172.21.15.1:0/3758379176 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:01:17.058 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.056+0000 7f8ebd59c640 1 --2- 172.21.15.1:0/3758379176 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c6cf82acb0 0x55c6cf8285b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:17.058 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.056+0000 7f8eb7fff640 1 --2- 172.21.15.1:0/3758379176 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6cf820410 0x55c6cf828070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:17.058 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.056+0000 7f8ebcd9b640 1 -- 172.21.15.1:0/3758379176 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6cf820410 msgr2=0x55c6cf828070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:17.058 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.056+0000 7f8ebcd9b640 1 --2- 172.21.15.1:0/3758379176 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6cf820410 0x55c6cf828070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.058 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.056+0000 7f8ebcd9b640 1 -- 172.21.15.1:0/3758379176 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c6cf82acb0 msgr2=0x55c6cf8285b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:17.058 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.056+0000 7f8ebcd9b640 1 --2- 172.21.15.1:0/3758379176 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c6cf82acb0 0x55c6cf8285b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.058 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.056+0000 7f8ebcd9b640 1 -- 172.21.15.1:0/3758379176 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6cf497210 con 0x55c6cf81faf0 2024-09-11T22:01:17.058 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.057+0000 7f8ebcd9b640 1 --2- 172.21.15.1:0/3758379176 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c6cf81faf0 0x55c6cf827b30 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f8ea80117c0 tx=0x7f8ea8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27cb1c9e705c0d6f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:01:17.058 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.057+0000 7f8e97fff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:01:17.059 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.057+0000 7f8e97fff640 1 -- 172.21.15.1:0/3758379176 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea801fbc0 con 0x55c6cf81faf0 2024-09-11T22:01:17.059 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.057+0000 7f8e97fff640 1 -- 172.21.15.1:0/3758379176 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f8ea801fd60 con 0x55c6cf81faf0 2024-09-11T22:01:17.073 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.070+0000 7f8ec1e6e740 10 client.0 _ll_register_callbacks cb 0x55c6cf73e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:01:17.073 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.071+0000 7f8ec1e6e740 1 -- 172.21.15.1:0/3758379176 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ea00318f0 con 0x55c6cf81faf0 2024-09-11T22:01:17.073 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.071+0000 7f8e97fff640 1 -- 172.21.15.1:0/3758379176 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8ea8028440 con 0x55c6cf81faf0 2024-09-11T22:01:17.073 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.071+0000 7f8ec1e6e740 10 client.16137 fetch_fsmap learned FSMap version 30 2024-09-11T22:01:17.074 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.071+0000 7f8ec1e6e740 1 -- 172.21.15.1:0/3758379176 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({fsmap=30}) -- 0x55c6cf563620 con 0x55c6cf81faf0 2024-09-11T22:01:17.078 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.077+0000 7f8e97fff640 1 -- 172.21.15.1:0/3758379176 <== mon.0 v2:172.21.15.1:3300/0 4 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f8ea8026070 con 0x55c6cf81faf0 2024-09-11T22:01:17.078 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.077+0000 7f8ec1e6e740 10 client.16137 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:01:17.078 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.077+0000 7f8ec1e6e740 10 client.16137 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:01:17.085 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.085+0000 7f8ec1e6e740 20 client.16137 populate_metadata read hostname 'smithi001' 2024-09-11T22:01:17.085 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.085+0000 7f8ec1e6e740 1 --2- 172.21.15.1:0/3758379176 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55c6cf83ab30 0x55c6cf83cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:17.086 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.085+0000 7f8ec1e6e740 4 client.16137 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:01:17.086 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.085+0000 7f8ec1e6e740 1 -- 172.21.15.1:0/3758379176 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6cf791a20 con 0x55c6cf83ab30 2024-09-11T22:01:17.086 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.085+0000 7f8eb7fff640 1 --2- 172.21.15.1:0/3758379176 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55c6cf83ab30 0x55c6cf83cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:17.086 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.086+0000 7f8eb7fff640 1 --2- 172.21.15.1:0/3758379176 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55c6cf83ab30 0x55c6cf83cfc0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1335956378ef6e9e server_cookie=69f8da3d799e4339 in_seq=0 out_seq=0 2024-09-11T22:01:17.086 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.086+0000 7f8e97fff640 10 client.16137 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:01:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.087+0000 7f8e97fff640 1 -- 172.21.15.1:0/3758379176 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c6cf791a20 con 0x55c6cf83ab30 2024-09-11T22:01:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.087+0000 7f8e97fff640 10 client.16137 handle_command_reply: tid=0 2024-09-11T22:01:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.087+0000 7f8ec1e6e740 1 client.16137 shutdown 2024-09-11T22:01:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.087+0000 7f8ec1e6e740 1 -- 172.21.15.1:0/3758379176 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c6cf81faf0 msgr2=0x55c6cf827b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.087+0000 7f8ec1e6e740 1 --2- 172.21.15.1:0/3758379176 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c6cf81faf0 0x55c6cf827b30 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f8ea80117c0 tx=0x7f8ea8004730 comp rx=0 tx=0).stop 2024-09-11T22:01:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.087+0000 7f8ec1e6e740 1 -- 172.21.15.1:0/3758379176 shutdown_connections 2024-09-11T22:01:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.087+0000 7f8ec1e6e740 1 --2- 172.21.15.1:0/3758379176 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55c6cf83ab30 0x55c6cf83cfc0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.087+0000 7f8ec1e6e740 1 --2- 172.21.15.1:0/3758379176 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c6cf82acb0 0x55c6cf8285b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.090 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.087+0000 7f8ec1e6e740 1 --2- 172.21.15.1:0/3758379176 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6cf820410 0x55c6cf828070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.090 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.087+0000 7f8ec1e6e740 1 --2- 172.21.15.1:0/3758379176 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c6cf81faf0 0x55c6cf827b30 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.090 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.087+0000 7f8e97fff640 0 client.16137 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:01:17.090 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.087+0000 7f8ec1e6e740 1 -- 172.21.15.1:0/3758379176 >> 172.21.15.1:0/3758379176 conn(0x55c6cf81c060 msgr2=0x55c6cf821dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:01:17.090 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.087+0000 7f8ec1e6e740 1 -- 172.21.15.1:0/3758379176 shutdown_connections 2024-09-11T22:01:17.090 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.087+0000 7f8ec1e6e740 1 -- 172.21.15.1:0/3758379176 wait complete. 2024-09-11T22:01:17.090 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.087+0000 7f8ec1e6e740 20 client.16137 trim_cache size 0 max 16384 2024-09-11T22:01:17.105 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.102+0000 7f8ec1e6e740 1 Processor -- start 2024-09-11T22:01:17.105 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.102+0000 7f8ec1e6e740 1 -- start start 2024-09-11T22:01:17.106 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.102+0000 7f8ec1e6e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c6cf81faf0 0x55c6cf8283e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:17.106 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.102+0000 7f8ec1e6e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6cf820410 0x55c6cf8428d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:17.106 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.102+0000 7f8ec1e6e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c6cf82acb0 0x55c6cf842e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:17.106 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.102+0000 7f8ec1e6e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb81ad930 con 0x55c6cf81faf0 2024-09-11T22:01:17.106 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.102+0000 7f8ec1e6e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb81af4d0 con 0x55c6cf82acb0 2024-09-11T22:01:17.106 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.103+0000 7f8ec1e6e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb81ae000 con 0x55c6cf820410 2024-09-11T22:01:17.106 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.103+0000 7f8ebcd9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c6cf81faf0 0x55c6cf8283e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:17.106 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.103+0000 7f8ebcd9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c6cf81faf0 0x55c6cf8283e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58594/0 (socket says 172.21.15.1:58594) 2024-09-11T22:01:17.106 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.103+0000 7f8ebcd9b640 1 -- 172.21.15.1:0/3043569643 learned_addr learned my addr 172.21.15.1:0/3043569643 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:01:17.107 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.103+0000 7f8ebd59c640 1 --2- 172.21.15.1:0/3043569643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c6cf82acb0 0x55c6cf842e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:17.107 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.103+0000 7f8eb7fff640 1 --2- 172.21.15.1:0/3043569643 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6cf820410 0x55c6cf8428d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:17.107 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.103+0000 7f8ebcd9b640 1 -- 172.21.15.1:0/3043569643 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6cf820410 msgr2=0x55c6cf8428d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:17.107 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.103+0000 7f8ebcd9b640 1 --2- 172.21.15.1:0/3043569643 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6cf820410 0x55c6cf8428d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.107 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.103+0000 7f8ebcd9b640 1 -- 172.21.15.1:0/3043569643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c6cf82acb0 msgr2=0x55c6cf842e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:17.107 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.103+0000 7f8ebcd9b640 1 --2- 172.21.15.1:0/3043569643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c6cf82acb0 0x55c6cf842e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.107 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.103+0000 7f8ebcd9b640 1 -- 172.21.15.1:0/3043569643 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6cf822940 con 0x55c6cf81faf0 2024-09-11T22:01:17.107 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.103+0000 7f8ebd59c640 1 --2- 172.21.15.1:0/3043569643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c6cf82acb0 0x55c6cf842e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:01:17.107 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.103+0000 7f8ebcd9b640 1 --2- 172.21.15.1:0/3043569643 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c6cf81faf0 0x55c6cf8283e0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8010fa0 tx=0x7f8ea8021040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7381bf64c873c0ec server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:01:17.107 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.103+0000 7f8eae7fc640 1 -- 172.21.15.1:0/3043569643 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea801fbc0 con 0x55c6cf81faf0 2024-09-11T22:01:17.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.104+0000 7f8eae7fc640 1 -- 172.21.15.1:0/3043569643 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ea00318f0 con 0x55c6cf81faf0 2024-09-11T22:01:17.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.104+0000 7f8eae7fc640 1 -- 172.21.15.1:0/3043569643 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea803d930 con 0x55c6cf81faf0 2024-09-11T22:01:17.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.104+0000 7f8ec1e6e740 1 -- 172.21.15.1:0/3043569643 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c6cf81faf0 msgr2=0x55c6cf8283e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:17.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.104+0000 7f8ec1e6e740 1 --2- 172.21.15.1:0/3043569643 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c6cf81faf0 0x55c6cf8283e0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8010fa0 tx=0x7f8ea8021040 comp rx=0 tx=0).stop 2024-09-11T22:01:17.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.104+0000 7f8ec1e6e740 1 -- 172.21.15.1:0/3043569643 shutdown_connections 2024-09-11T22:01:17.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.104+0000 7f8ec1e6e740 1 --2- 172.21.15.1:0/3043569643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c6cf82acb0 0x55c6cf842e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.104+0000 7f8ec1e6e740 1 --2- 172.21.15.1:0/3043569643 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6cf820410 0x55c6cf8428d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.104+0000 7f8ec1e6e740 1 --2- 172.21.15.1:0/3043569643 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c6cf81faf0 0x55c6cf8283e0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.104+0000 7f8ec1e6e740 1 -- 172.21.15.1:0/3043569643 >> 172.21.15.1:0/3043569643 conn(0x55c6cf81c060 msgr2=0x55c6cf829bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:01:17.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.104+0000 7f8ec1e6e740 1 -- 172.21.15.1:0/3043569643 shutdown_connections 2024-09-11T22:01:17.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.104+0000 7f8ec1e6e740 1 -- 172.21.15.1:0/3043569643 wait complete. 2024-09-11T22:01:17.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.105+0000 7f8ec1e6e740 1 -- start start 2024-09-11T22:01:17.109 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.105+0000 7f8ec1e6e740 1 Processor -- start 2024-09-11T22:01:17.111 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.105+0000 7f8ec1e6e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c6cf81faf0 0x55c6cf82ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:17.111 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.105+0000 7f8ec1e6e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c6cf820410 0x55c6cf82f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:17.111 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.105+0000 7f8ec1e6e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6cf82acb0 0x55c6cf8a48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:17.112 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.105+0000 7f8ec1e6e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb81aedd0 con 0x55c6cf81faf0 2024-09-11T22:01:17.112 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.105+0000 7f8ec1e6e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb81ae6d0 con 0x55c6cf820410 2024-09-11T22:01:17.112 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.105+0000 7f8ec1e6e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb81ab910 con 0x55c6cf82acb0 2024-09-11T22:01:17.112 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.105+0000 7f8ec1e6e740 10 client.0 _ll_register_callbacks cb 0x7f8ea8010c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:01:17.112 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.105+0000 7f8ebcd9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c6cf81faf0 0x55c6cf82ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:17.112 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.106+0000 7f8ebcd9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c6cf81faf0 0x55c6cf82ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58602/0 (socket says 172.21.15.1:58602) 2024-09-11T22:01:17.112 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.106+0000 7f8ebcd9b640 1 -- 172.21.15.1:0/881281586 learned_addr learned my addr 172.21.15.1:0/881281586 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:01:17.112 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.106+0000 7f8ebd59c640 1 --2- 172.21.15.1:0/881281586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6cf82acb0 0x55c6cf8a48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:17.112 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.106+0000 7f8eb7fff640 1 --2- 172.21.15.1:0/881281586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c6cf820410 0x55c6cf82f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:17.113 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.106+0000 7f8ebcd9b640 1 -- 172.21.15.1:0/881281586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6cf82acb0 msgr2=0x55c6cf8a48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:17.113 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.106+0000 7f8ebcd9b640 1 --2- 172.21.15.1:0/881281586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6cf82acb0 0x55c6cf8a48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.113 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.106+0000 7f8ebcd9b640 1 -- 172.21.15.1:0/881281586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c6cf820410 msgr2=0x55c6cf82f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:17.113 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.106+0000 7f8ebcd9b640 1 --2- 172.21.15.1:0/881281586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c6cf820410 0x55c6cf82f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.113 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.106+0000 7f8ebcd9b640 1 -- 172.21.15.1:0/881281586 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6cf82fb40 con 0x55c6cf81faf0 2024-09-11T22:01:17.113 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.106+0000 7f8ebcd9b640 1 --2- 172.21.15.1:0/881281586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c6cf81faf0 0x55c6cf82ed40 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8011010 tx=0x7f8ea8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22133d3c31e60d98 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:01:17.113 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.106+0000 7f8e97fff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:01:17.113 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.106+0000 7f8e97fff640 1 -- 172.21.15.1:0/881281586 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea80289f0 con 0x55c6cf81faf0 2024-09-11T22:01:17.113 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.106+0000 7f8e97fff640 1 -- 172.21.15.1:0/881281586 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f8ea8028b90 con 0x55c6cf81faf0 2024-09-11T22:01:17.114 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.106+0000 7f8ec1e6e740 1 -- 172.21.15.1:0/881281586 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ea00318f0 con 0x55c6cf81faf0 2024-09-11T22:01:17.114 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.107+0000 7f8e97fff640 1 -- 172.21.15.1:0/881281586 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8ea8028d90 con 0x55c6cf81faf0 2024-09-11T22:01:17.114 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.107+0000 7f8ec1e6e740 10 client.16143 fetch_fsmap learned FSMap version 30 2024-09-11T22:01:17.114 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.107+0000 7f8ec1e6e740 1 -- 172.21.15.1:0/881281586 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c6cf8a5ef0 con 0x55c6cf81faf0 2024-09-11T22:01:17.114 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.107+0000 7f8e97fff640 1 -- 172.21.15.1:0/881281586 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ea00318f0 con 0x55c6cf81faf0 2024-09-11T22:01:17.114 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.107+0000 7f8e97fff640 1 -- 172.21.15.1:0/881281586 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f8ea8026040 con 0x55c6cf81faf0 2024-09-11T22:01:17.114 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.107+0000 7f8ec1e6e740 10 client.16143 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:01:17.114 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.107+0000 7f8ec1e6e740 10 client.16143 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:01:17.114 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.107+0000 7f8ec1e6e740 20 client.16143 populate_metadata read hostname 'smithi001' 2024-09-11T22:01:17.114 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.107+0000 7f8ec1e6e740 1 --2- 172.21.15.1:0/881281586 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55c6cf8a6540 0x55c6cf8a8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:17.114 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.107+0000 7f8ec1e6e740 4 client.16143 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:01:17.114 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.107+0000 7f8ec1e6e740 1 -- 172.21.15.1:0/881281586 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c6cf78bd70 con 0x55c6cf8a6540 2024-09-11T22:01:17.114 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.107+0000 7f8eb7fff640 1 --2- 172.21.15.1:0/881281586 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55c6cf8a6540 0x55c6cf8a8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:17.115 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.108+0000 7f8eb7fff640 1 --2- 172.21.15.1:0/881281586 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55c6cf8a6540 0x55c6cf8a8960 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17a09e25855b413a server_cookie=9a66b4cdb0381778 in_seq=0 out_seq=0 2024-09-11T22:01:17.115 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.108+0000 7f8e97fff640 10 client.16143 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:01:17.273 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:01:16 smithi001 ceph-mon[30935]: pgmap v126: 97 pgs: 97 active+clean; 3.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 68 KiB/s wr, 36 op/s 2024-09-11T22:01:17.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.272+0000 7f3666013640 1 -- 172.21.15.1:0/2007517071 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3660108300 msgr2=0x7f366010ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:17.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.272+0000 7f3666013640 1 --2- 172.21.15.1:0/2007517071 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3660108300 0x7f366010ec70 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f3658009cd0 tx=0x7f365802a8d0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.272+0000 7f3666013640 1 -- 172.21.15.1:0/2007517071 shutdown_connections 2024-09-11T22:01:17.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.272+0000 7f3666013640 1 --2- 172.21.15.1:0/2007517071 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3660108300 0x7f366010ec70 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.272+0000 7f3666013640 1 --2- 172.21.15.1:0/2007517071 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3660103780 0x7f3660107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.272+0000 7f3666013640 1 --2- 172.21.15.1:0/2007517071 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3660102dd0 0x7f36601031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.272+0000 7f3666013640 1 -- 172.21.15.1:0/2007517071 >> 172.21.15.1:0/2007517071 conn(0x7f36600fc7d0 msgr2=0x7f36600febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:01:17.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.272+0000 7f3666013640 1 -- 172.21.15.1:0/2007517071 shutdown_connections 2024-09-11T22:01:17.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.272+0000 7f3666013640 1 -- 172.21.15.1:0/2007517071 wait complete. 2024-09-11T22:01:17.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.272+0000 7f3666013640 1 Processor -- start 2024-09-11T22:01:17.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.273+0000 7f3666013640 1 -- start start 2024-09-11T22:01:17.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.273+0000 7f3666013640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3660102dd0 0x7f36601985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:17.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.273+0000 7f3666013640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3660103780 0x7f3660198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:17.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.273+0000 7f3666013640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3660108300 0x7f366019ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:17.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.273+0000 7f3666013640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f36601120d0 con 0x7f3660102dd0 2024-09-11T22:01:17.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.273+0000 7f3666013640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3660111f50 con 0x7f3660108300 2024-09-11T22:01:17.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.273+0000 7f3666013640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3660112250 con 0x7f3660103780 2024-09-11T22:01:17.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.273+0000 7f365f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3660102dd0 0x7f36601985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:17.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.273+0000 7f365f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3660102dd0 0x7f36601985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58622/0 (socket says 172.21.15.1:58622) 2024-09-11T22:01:17.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.273+0000 7f365f7fe640 1 -- 172.21.15.1:0/3150267863 learned_addr learned my addr 172.21.15.1:0/3150267863 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:01:17.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.273+0000 7f365ffff640 1 --2- 172.21.15.1:0/3150267863 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3660108300 0x7f366019ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:17.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.273+0000 7f3657fff640 1 --2- 172.21.15.1:0/3150267863 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3660103780 0x7f3660198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:17.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.273+0000 7f365f7fe640 1 -- 172.21.15.1:0/3150267863 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3660103780 msgr2=0x7f3660198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:17.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.273+0000 7f365f7fe640 1 --2- 172.21.15.1:0/3150267863 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3660103780 0x7f3660198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.273+0000 7f365f7fe640 1 -- 172.21.15.1:0/3150267863 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3660108300 msgr2=0x7f366019ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:17.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.273+0000 7f365f7fe640 1 --2- 172.21.15.1:0/3150267863 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3660108300 0x7f366019ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.274+0000 7f365f7fe640 1 -- 172.21.15.1:0/3150267863 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f366019d560 con 0x7f3660102dd0 2024-09-11T22:01:17.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.274+0000 7f3657fff640 1 --2- 172.21.15.1:0/3150267863 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3660103780 0x7f3660198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:01:17.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.274+0000 7f365f7fe640 1 --2- 172.21.15.1:0/3150267863 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3660102dd0 0x7f36601985b0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f365000ea60 tx=0x7f365000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:01:17.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.274+0000 7f365d7fa640 1 -- 172.21.15.1:0/3150267863 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f365000cde0 con 0x7f3660102dd0 2024-09-11T22:01:17.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.274+0000 7f365d7fa640 1 -- 172.21.15.1:0/3150267863 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36500035f0 con 0x7f3660102dd0 2024-09-11T22:01:17.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.274+0000 7f3666013640 1 -- 172.21.15.1:0/3150267863 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f366019d850 con 0x7f3660102dd0 2024-09-11T22:01:17.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.274+0000 7f365d7fa640 1 -- 172.21.15.1:0/3150267863 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3650010670 con 0x7f3660102dd0 2024-09-11T22:01:17.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.275+0000 7f3666013640 1 -- 172.21.15.1:0/3150267863 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36601a5130 con 0x7f3660102dd0 2024-09-11T22:01:17.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.277+0000 7f365d7fa640 1 -- 172.21.15.1:0/3150267863 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f36500037a0 con 0x7f3660102dd0 2024-09-11T22:01:17.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.277+0000 7f365d7fa640 1 --2- 172.21.15.1:0/3150267863 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f364c0a2d70 0x7f364c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:17.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.278+0000 7f365d7fa640 1 -- 172.21.15.1:0/3150267863 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f36500cf810 con 0x7f3660102dd0 2024-09-11T22:01:17.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.278+0000 7f3657fff640 1 --2- 172.21.15.1:0/3150267863 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f364c0a2d70 0x7f364c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:17.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.278+0000 7f3657fff640 1 --2- 172.21.15.1:0/3150267863 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f364c0a2d70 0x7f364c0a5230 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f36600fedf0 tx=0x7f3648005f90 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:01:17.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.282+0000 7f5d1b7ca640 1 -- 172.21.15.1:0/1102836295 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d14102d90 msgr2=0x7f5d14103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:17.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.282+0000 7f5d1b7ca640 1 --2- 172.21.15.1:0/1102836295 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d14102d90 0x7f5d14103170 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f5d10009930 tx=0x7f5d1002a420 comp rx=0 tx=0).stop 2024-09-11T22:01:17.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.282+0000 7f5d1b7ca640 1 -- 172.21.15.1:0/1102836295 shutdown_connections 2024-09-11T22:01:17.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.282+0000 7f5d1b7ca640 1 --2- 172.21.15.1:0/1102836295 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d14108290 0x7f5d1410ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.282+0000 7f5d1b7ca640 1 --2- 172.21.15.1:0/1102836295 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d14103740 0x7f5d14107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.282+0000 7f5d1b7ca640 1 --2- 172.21.15.1:0/1102836295 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d14102d90 0x7f5d14103170 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.282+0000 7f5d1b7ca640 1 -- 172.21.15.1:0/1102836295 >> 172.21.15.1:0/1102836295 conn(0x7f5d140fc7d0 msgr2=0x7f5d140febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:01:17.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.282+0000 7f5d1b7ca640 1 -- 172.21.15.1:0/1102836295 shutdown_connections 2024-09-11T22:01:17.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.282+0000 7f5d1b7ca640 1 -- 172.21.15.1:0/1102836295 wait complete. 2024-09-11T22:01:17.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.283+0000 7f5d1b7ca640 1 Processor -- start 2024-09-11T22:01:17.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.283+0000 7f5d1b7ca640 1 -- start start 2024-09-11T22:01:17.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.283+0000 7f5d1b7ca640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d14102d90 0x7f5d14198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:17.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.283+0000 7f5d1b7ca640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d14103740 0x7f5d14198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:17.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.283+0000 7f5d1b7ca640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d14108290 0x7f5d1419ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:17.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.283+0000 7f5d1b7ca640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5d14112090 con 0x7f5d14108290 2024-09-11T22:01:17.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.283+0000 7f5d1b7ca640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5d14111f10 con 0x7f5d14103740 2024-09-11T22:01:17.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.283+0000 7f5d1b7ca640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5d14112210 con 0x7f5d14102d90 2024-09-11T22:01:17.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.283+0000 7f5d19d67640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d14108290 0x7f5d1419ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:17.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.283+0000 7f5d19d67640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d14108290 0x7f5d1419ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58630/0 (socket says 172.21.15.1:58630) 2024-09-11T22:01:17.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.283+0000 7f5d19d67640 1 -- 172.21.15.1:0/739985241 learned_addr learned my addr 172.21.15.1:0/739985241 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:01:17.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.283+0000 7f5d18d65640 1 --2- 172.21.15.1:0/739985241 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d14103740 0x7f5d14198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:17.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.284+0000 7f5d19566640 1 --2- 172.21.15.1:0/739985241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d14102d90 0x7f5d14198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:17.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.284+0000 7f5d19d67640 1 -- 172.21.15.1:0/739985241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d14102d90 msgr2=0x7f5d14198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:17.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.284+0000 7f5d19d67640 1 --2- 172.21.15.1:0/739985241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d14102d90 0x7f5d14198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.284+0000 7f5d19d67640 1 -- 172.21.15.1:0/739985241 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d14103740 msgr2=0x7f5d14198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:17.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.284+0000 7f5d19d67640 1 --2- 172.21.15.1:0/739985241 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d14103740 0x7f5d14198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.284+0000 7f5d19d67640 1 -- 172.21.15.1:0/739985241 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d1419d500 con 0x7f5d14108290 2024-09-11T22:01:17.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.284+0000 7f5d19d67640 1 --2- 172.21.15.1:0/739985241 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d14108290 0x7f5d1419ce20 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c00cce0 tx=0x7f5d0c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:01:17.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.284+0000 7f5d0a7fc640 1 -- 172.21.15.1:0/739985241 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0c00f450 con 0x7f5d14108290 2024-09-11T22:01:17.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.284+0000 7f5d1b7ca640 1 -- 172.21.15.1:0/739985241 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d1419d7f0 con 0x7f5d14108290 2024-09-11T22:01:17.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.284+0000 7f5d0a7fc640 1 -- 172.21.15.1:0/739985241 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d0c00fad0 con 0x7f5d14108290 2024-09-11T22:01:17.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.285+0000 7f5d0a7fc640 1 -- 172.21.15.1:0/739985241 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0c018c10 con 0x7f5d14108290 2024-09-11T22:01:17.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.285+0000 7f5d1b7ca640 1 -- 172.21.15.1:0/739985241 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d141a50d0 con 0x7f5d14108290 2024-09-11T22:01:17.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.287+0000 7f5d0a7fc640 1 -- 172.21.15.1:0/739985241 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f5d0c00fc80 con 0x7f5d14108290 2024-09-11T22:01:17.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.287+0000 7f5d0a7fc640 1 --2- 172.21.15.1:0/739985241 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f5d000a2bd0 0x7f5d000a5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:17.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.287+0000 7f5d19566640 1 --2- 172.21.15.1:0/739985241 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f5d000a2bd0 0x7f5d000a5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:17.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.288+0000 7f5d0a7fc640 1 -- 172.21.15.1:0/739985241 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f5d0c0cf9c0 con 0x7f5d14108290 2024-09-11T22:01:17.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.288+0000 7f5d19566640 1 --2- 172.21.15.1:0/739985241 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f5d000a2bd0 0x7f5d000a5090 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5d10009900 tx=0x7f5d100058d0 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:01:17.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:01:16 smithi053 ceph-mon[34908]: pgmap v126: 97 pgs: 97 active+clean; 3.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 68 KiB/s wr, 36 op/s 2024-09-11T22:01:17.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.297+0000 7f36683ec740 1 Processor -- start 2024-09-11T22:01:17.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.297+0000 7f36683ec740 1 -- start start 2024-09-11T22:01:17.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.297+0000 7f5d1dba3740 1 Processor -- start 2024-09-11T22:01:17.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.297+0000 7f5d1dba3740 1 -- start start 2024-09-11T22:01:17.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.298+0000 7f36683ec740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56371e0bea40 0x56371e0bee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:17.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.298+0000 7f36683ec740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56371e0bf360 0x56371e0cdc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:17.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.298+0000 7f36683ec740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56371e0ce150 0x56371e0d0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:17.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.298+0000 7f36683ec740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56371e024280 con 0x56371e0bf360 2024-09-11T22:01:17.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.298+0000 7f36683ec740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56371dfcbb10 con 0x56371e0ce150 2024-09-11T22:01:17.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.298+0000 7f36683ec740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56371e020240 con 0x56371e0bea40 2024-09-11T22:01:17.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.298+0000 7f365ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56371e0ce150 0x56371e0d0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:17.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.298+0000 7f365ffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56371e0ce150 0x56371e0d0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:35478/0 (socket says 172.21.15.1:35478) 2024-09-11T22:01:17.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.298+0000 7f5d1dba3740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cb41bf8660 0x55cb41bf8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:17.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.298+0000 7f5d1dba3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb41bf8f80 0x55cb41c07810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:17.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.298+0000 7f5d1dba3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cb41c07d50 0x55cb41c0a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:17.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.298+0000 7f5d1dba3740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55cb41b5e2a0 con 0x55cb41bf8660 2024-09-11T22:01:17.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.298+0000 7f5d1dba3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55cb41b05b10 con 0x55cb41bf8f80 2024-09-11T22:01:17.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.298+0000 7f5d1dba3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55cb41b5a260 con 0x55cb41c07d50 2024-09-11T22:01:17.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.298+0000 7f5d19566640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cb41bf8660 0x55cb41bf8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:17.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.298+0000 7f5d19566640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cb41bf8660 0x55cb41bf8a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58678/0 (socket says 172.21.15.1:58678) 2024-09-11T22:01:17.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.298+0000 7f5d19566640 1 -- 172.21.15.1:0/3552820584 learned_addr learned my addr 172.21.15.1:0/3552820584 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:01:17.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.298+0000 7f365ffff640 1 -- 172.21.15.1:0/2692743761 learned_addr learned my addr 172.21.15.1:0/2692743761 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:01:17.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.298+0000 7f365f7fe640 1 --2- 172.21.15.1:0/2692743761 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56371e0bea40 0x56371e0bee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:17.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.298+0000 7f3657fff640 1 --2- 172.21.15.1:0/2692743761 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56371e0bf360 0x56371e0cdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:17.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.298+0000 7f365ffff640 1 -- 172.21.15.1:0/2692743761 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56371e0bea40 msgr2=0x56371e0bee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:17.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.298+0000 7f365ffff640 1 --2- 172.21.15.1:0/2692743761 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56371e0bea40 0x56371e0bee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.298+0000 7f365ffff640 1 -- 172.21.15.1:0/2692743761 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56371e0bf360 msgr2=0x56371e0cdc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:17.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.298+0000 7f365ffff640 1 --2- 172.21.15.1:0/2692743761 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56371e0bf360 0x56371e0cdc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.298+0000 7f365ffff640 1 -- 172.21.15.1:0/2692743761 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56371dfd81f0 con 0x56371e0ce150 2024-09-11T22:01:17.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.298+0000 7f3657fff640 1 --2- 172.21.15.1:0/2692743761 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56371e0bf360 0x56371e0cdc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:01:17.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.298+0000 7f5d18d65640 1 --2- 172.21.15.1:0/3552820584 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb41bf8f80 0x55cb41c07810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:17.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.298+0000 7f5d19d67640 1 --2- 172.21.15.1:0/3552820584 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cb41c07d50 0x55cb41c0a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:17.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.299+0000 7f5d19566640 1 -- 172.21.15.1:0/3552820584 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cb41c07d50 msgr2=0x55cb41c0a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:17.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.299+0000 7f5d19566640 1 --2- 172.21.15.1:0/3552820584 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cb41c07d50 0x55cb41c0a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.299+0000 7f5d19566640 1 -- 172.21.15.1:0/3552820584 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb41bf8f80 msgr2=0x55cb41c07810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:17.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.299+0000 7f5d19566640 1 --2- 172.21.15.1:0/3552820584 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb41bf8f80 0x55cb41c07810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.299+0000 7f365ffff640 1 --2- 172.21.15.1:0/2692743761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56371e0ce150 0x56371e0d0540 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3658009cd0 tx=0x7f3658005f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30f7580bb98e0b59 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:01:17.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.299+0000 7f5d19566640 1 -- 172.21.15.1:0/3552820584 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb418cfb90 con 0x55cb41bf8660 2024-09-11T22:01:17.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.299+0000 7f5d18d65640 1 --2- 172.21.15.1:0/3552820584 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb41bf8f80 0x55cb41c07810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:01:17.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.299+0000 7f5d19d67640 1 --2- 172.21.15.1:0/3552820584 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cb41c07d50 0x55cb41c0a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:01:17.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.299+0000 7f3655ffb640 1 -- 172.21.15.1:0/2692743761 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3658031cb0 con 0x56371e0ce150 2024-09-11T22:01:17.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.299+0000 7f3655ffb640 1 -- 172.21.15.1:0/2692743761 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3658036030 con 0x56371e0ce150 2024-09-11T22:01:17.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.299+0000 7f5d19566640 1 --2- 172.21.15.1:0/3552820584 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cb41bf8660 0x55cb41bf8a40 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f5d10009930 tx=0x7f5d1003d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=431a7704f08f003a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:01:17.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.299+0000 7f3655ffb640 1 -- 172.21.15.1:0/2692743761 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36580042a0 con 0x56371e0ce150 2024-09-11T22:01:17.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.299+0000 7f5cff7fe640 1 -- 172.21.15.1:0/3552820584 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d10035040 con 0x55cb41bf8660 2024-09-11T22:01:17.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.299+0000 7f5cff7fe640 1 -- 172.21.15.1:0/3552820584 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d1004e070 con 0x55cb41bf8660 2024-09-11T22:01:17.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.299+0000 7f5cff7fe640 1 -- 172.21.15.1:0/3552820584 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d10049b90 con 0x55cb41bf8660 2024-09-11T22:01:17.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.300+0000 7f36683ec740 1 -- 172.21.15.1:0/2692743761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56371e0ce150 msgr2=0x56371e0d0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:17.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.300+0000 7f36683ec740 1 --2- 172.21.15.1:0/2692743761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56371e0ce150 0x56371e0d0540 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3658009cd0 tx=0x7f3658005f60 comp rx=0 tx=0).stop 2024-09-11T22:01:17.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.300+0000 7f36683ec740 1 -- 172.21.15.1:0/2692743761 shutdown_connections 2024-09-11T22:01:17.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.300+0000 7f36683ec740 1 --2- 172.21.15.1:0/2692743761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56371e0ce150 0x56371e0d0540 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.300+0000 7f36683ec740 1 --2- 172.21.15.1:0/2692743761 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56371e0bf360 0x56371e0cdc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.300+0000 7f36683ec740 1 --2- 172.21.15.1:0/2692743761 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56371e0bea40 0x56371e0bee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.300+0000 7f36683ec740 1 -- 172.21.15.1:0/2692743761 >> 172.21.15.1:0/2692743761 conn(0x56371e0bb0f0 msgr2=0x56371e0bb4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:01:17.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.300+0000 7f36683ec740 1 -- 172.21.15.1:0/2692743761 shutdown_connections 2024-09-11T22:01:17.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.300+0000 7f36683ec740 1 -- 172.21.15.1:0/2692743761 wait complete. 2024-09-11T22:01:17.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.301+0000 7f5d1dba3740 1 -- 172.21.15.1:0/3552820584 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cb41bf8660 msgr2=0x55cb41bf8a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:17.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.301+0000 7f36683ec740 1 -- start start 2024-09-11T22:01:17.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.301+0000 7f5d1dba3740 1 --2- 172.21.15.1:0/3552820584 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cb41bf8660 0x55cb41bf8a40 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f5d10009930 tx=0x7f5d1003d000 comp rx=0 tx=0).stop 2024-09-11T22:01:17.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.301+0000 7f36683ec740 1 Processor -- start 2024-09-11T22:01:17.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.301+0000 7f5d1dba3740 1 -- 172.21.15.1:0/3552820584 shutdown_connections 2024-09-11T22:01:17.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.301+0000 7f5d1dba3740 1 --2- 172.21.15.1:0/3552820584 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cb41c07d50 0x55cb41c0a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.301+0000 7f5d1dba3740 1 --2- 172.21.15.1:0/3552820584 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb41bf8f80 0x55cb41c07810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.301+0000 7f5d1dba3740 1 --2- 172.21.15.1:0/3552820584 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cb41bf8660 0x55cb41bf8a40 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.301+0000 7f5d1dba3740 1 -- 172.21.15.1:0/3552820584 >> 172.21.15.1:0/3552820584 conn(0x55cb41bf4cf0 msgr2=0x55cb41bf50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:01:17.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.301+0000 7f5d1dba3740 1 -- 172.21.15.1:0/3552820584 shutdown_connections 2024-09-11T22:01:17.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.301+0000 7f5d1dba3740 1 -- 172.21.15.1:0/3552820584 wait complete. 2024-09-11T22:01:17.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.301+0000 7f5d1dba3740 1 -- start start 2024-09-11T22:01:17.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.302+0000 7f36683ec740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56371e0bea40 0x56371e0cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:17.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.302+0000 7f36683ec740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56371e0bf360 0x56371e0cb520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:17.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.302+0000 7f36683ec740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56371e0ce150 0x56371e0cba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:17.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.302+0000 7f36683ec740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56371e0d0dc0 con 0x56371e0bf360 2024-09-11T22:01:17.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.302+0000 7f36683ec740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56371dfdef10 con 0x56371e0bea40 2024-09-11T22:01:17.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.302+0000 7f36683ec740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56371e0d0f40 con 0x56371e0ce150 2024-09-11T22:01:17.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.302+0000 7f365f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56371e0bea40 0x56371e0cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:17.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.302+0000 7f36683ec740 10 client.0 _ll_register_callbacks cb 0x56371dfa7a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:01:17.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.302+0000 7f365f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56371e0bea40 0x56371e0cafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:35496/0 (socket says 172.21.15.1:35496) 2024-09-11T22:01:17.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.302+0000 7f365f7fe640 1 -- 172.21.15.1:0/3750332906 learned_addr learned my addr 172.21.15.1:0/3750332906 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:01:17.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.302+0000 7f3657fff640 1 --2- 172.21.15.1:0/3750332906 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56371e0bf360 0x56371e0cb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:17.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.302+0000 7f365ffff640 1 --2- 172.21.15.1:0/3750332906 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56371e0ce150 0x56371e0cba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:17.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.302+0000 7f365f7fe640 1 -- 172.21.15.1:0/3750332906 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56371e0ce150 msgr2=0x56371e0cba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:17.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.302+0000 7f365f7fe640 1 --2- 172.21.15.1:0/3750332906 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56371e0ce150 0x56371e0cba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.302+0000 7f365f7fe640 1 -- 172.21.15.1:0/3750332906 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56371e0bf360 msgr2=0x56371e0cb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:17.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.302+0000 7f365f7fe640 1 --2- 172.21.15.1:0/3750332906 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56371e0bf360 0x56371e0cb520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.302+0000 7f365f7fe640 1 -- 172.21.15.1:0/3750332906 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56371dd956a0 con 0x56371e0bea40 2024-09-11T22:01:17.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.302+0000 7f5d1dba3740 1 Processor -- start 2024-09-11T22:01:17.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.302+0000 7f365ffff640 1 --2- 172.21.15.1:0/3750332906 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56371e0ce150 0x56371e0cba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:01:17.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.302+0000 7f365f7fe640 1 --2- 172.21.15.1:0/3750332906 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56371e0bea40 0x56371e0cafe0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f365009bc80 tx=0x7f365009c3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca5f8c446468f960 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:01:17.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.302+0000 7f363effd640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-11T22:01:17.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.303+0000 7f363effd640 1 -- 172.21.15.1:0/3750332906 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36500a00a0 con 0x56371e0bea40 2024-09-11T22:01:17.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.303+0000 7f363effd640 1 -- 172.21.15.1:0/3750332906 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f3650098360 con 0x56371e0bea40 2024-09-11T22:01:17.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.303+0000 7f36683ec740 1 -- 172.21.15.1:0/3750332906 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3658036030 con 0x56371e0bea40 2024-09-11T22:01:17.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.303+0000 7f363effd640 1 -- 172.21.15.1:0/3750332906 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f36500a4960 con 0x56371e0bea40 2024-09-11T22:01:17.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.303+0000 7f36683ec740 10 client.25739 fetch_fsmap learned FSMap version 30 2024-09-11T22:01:17.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.303+0000 7f36683ec740 1 -- 172.21.15.1:0/3750332906 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56371e0dc950 con 0x56371e0bea40 2024-09-11T22:01:17.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.303+0000 7f5d1dba3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cb41bf8660 0x55cb41c04bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:17.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.303+0000 7f5d1dba3740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cb41bf8f80 0x55cb41c05130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:17.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.303+0000 7f363effd640 1 -- 172.21.15.1:0/3750332906 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3658036030 con 0x56371e0bea40 2024-09-11T22:01:17.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.304+0000 7f363effd640 1 -- 172.21.15.1:0/3750332906 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f36500ba020 con 0x56371e0bea40 2024-09-11T22:01:17.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.304+0000 7f36683ec740 10 client.25739 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:01:17.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.304+0000 7f36683ec740 10 client.25739 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:01:17.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.304+0000 7f36683ec740 20 client.25739 populate_metadata read hostname 'smithi001' 2024-09-11T22:01:17.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.304+0000 7f36683ec740 1 --2- 172.21.15.1:0/3750332906 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x56371e0ddf50 0x56371e0e0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:17.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.304+0000 7f36683ec740 4 client.25739 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:01:17.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.304+0000 7f36683ec740 1 -- 172.21.15.1:0/3750332906 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56371e0317f0 con 0x56371e0ddf50 2024-09-11T22:01:17.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.303+0000 7f5d18d65640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cb41bf8f80 0x55cb41c05130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:17.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.304+0000 7f3657fff640 1 --2- 172.21.15.1:0/3750332906 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x56371e0ddf50 0x56371e0e0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:17.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.304+0000 7f5d1dba3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb41c07d50 0x55cb41c05670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:17.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.304+0000 7f5d18d65640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cb41bf8f80 0x55cb41c05130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58702/0 (socket says 172.21.15.1:58702) 2024-09-11T22:01:17.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.304+0000 7f5d1dba3740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55cb41c0a920 con 0x55cb41bf8f80 2024-09-11T22:01:17.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.304+0000 7f3657fff640 1 --2- 172.21.15.1:0/3750332906 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x56371e0ddf50 0x56371e0e0340 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fedd7c9102c80c7a server_cookie=71465d84c033d3e2 in_seq=0 out_seq=0 2024-09-11T22:01:17.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.304+0000 7f363effd640 10 client.25739 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:01:17.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.304+0000 7f5d18d65640 1 -- 172.21.15.1:0/2114801011 learned_addr learned my addr 172.21.15.1:0/2114801011 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:01:17.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.304+0000 7f5d1dba3740 1 -- 172.21.15.1:0/2114801011 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55cb41ae66e0 con 0x55cb41c07d50 2024-09-11T22:01:17.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.304+0000 7f5d1dba3740 1 -- 172.21.15.1:0/2114801011 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55cb41c0aaa0 con 0x55cb41bf8660 2024-09-11T22:01:17.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.304+0000 7f5d1dba3740 10 client.0 _ll_register_callbacks cb 0x55cb41b0aa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:01:17.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.304+0000 7f5d19566640 1 --2- 172.21.15.1:0/2114801011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cb41bf8660 0x55cb41c04bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:17.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.304+0000 7f5d19d67640 1 --2- 172.21.15.1:0/2114801011 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb41c07d50 0x55cb41c05670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:17.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.304+0000 7f5d18d65640 1 -- 172.21.15.1:0/2114801011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cb41bf8660 msgr2=0x55cb41c04bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:17.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.304+0000 7f5d18d65640 1 --2- 172.21.15.1:0/2114801011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cb41bf8660 0x55cb41c04bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.304+0000 7f5d18d65640 1 -- 172.21.15.1:0/2114801011 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb41c07d50 msgr2=0x55cb41c05670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:17.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.304+0000 7f5d18d65640 1 --2- 172.21.15.1:0/2114801011 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb41c07d50 0x55cb41c05670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.304+0000 7f5d18d65640 1 -- 172.21.15.1:0/2114801011 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb41c13e20 con 0x55cb41bf8f80 2024-09-11T22:01:17.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.304+0000 7f5d1dba3740 1 -- 172.21.15.1:0/2114801011 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d1004e070 con 0x55cb41bf8f80 2024-09-11T22:01:17.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.305+0000 7f363effd640 1 -- 172.21.15.1:0/3750332906 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56371e0317f0 con 0x56371e0ddf50 2024-09-11T22:01:17.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.305+0000 7f363effd640 10 client.25739 handle_command_reply: tid=0 2024-09-11T22:01:17.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.305+0000 7f36683ec740 1 client.25739 shutdown 2024-09-11T22:01:17.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.305+0000 7f5d18d65640 1 --2- 172.21.15.1:0/2114801011 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cb41bf8f80 0x55cb41c05130 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f5d04013710 tx=0x7f5d04013bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=727098e2b2c9d69a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:01:17.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.305+0000 7f5cfcff9640 10 client.16173 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:01:17.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.306+0000 7f36683ec740 1 -- 172.21.15.1:0/3750332906 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56371e0bea40 msgr2=0x56371e0cafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:17.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.306+0000 7f36683ec740 1 --2- 172.21.15.1:0/3750332906 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56371e0bea40 0x56371e0cafe0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f365009bc80 tx=0x7f365009c3f0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.306+0000 7f5cfcff9640 1 -- 172.21.15.1:0/2114801011 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d040119c0 con 0x55cb41bf8f80 2024-09-11T22:01:17.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.306+0000 7f5cfcff9640 1 -- 172.21.15.1:0/2114801011 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f5d04011b60 con 0x55cb41bf8f80 2024-09-11T22:01:17.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.306+0000 7f36683ec740 1 -- 172.21.15.1:0/3750332906 shutdown_connections 2024-09-11T22:01:17.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.306+0000 7f5cfcff9640 1 -- 172.21.15.1:0/2114801011 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5d04011da0 con 0x55cb41bf8f80 2024-09-11T22:01:17.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.306+0000 7f5d1dba3740 10 client.16173 fetch_fsmap learned FSMap version 30 2024-09-11T22:01:17.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.306+0000 7f5d1dba3740 1 -- 172.21.15.1:0/2114801011 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({fsmap=30}) -- 0x55cb41bfb490 con 0x55cb41bf8f80 2024-09-11T22:01:17.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.306+0000 7f36683ec740 1 --2- 172.21.15.1:0/3750332906 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x56371e0ddf50 0x56371e0e0340 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.306+0000 7f363effd640 0 client.25739 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:01:17.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.306+0000 7f36683ec740 1 --2- 172.21.15.1:0/3750332906 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56371e0ce150 0x56371e0cba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.306+0000 7f36683ec740 1 --2- 172.21.15.1:0/3750332906 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56371e0bf360 0x56371e0cb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.306+0000 7f36683ec740 1 --2- 172.21.15.1:0/3750332906 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56371e0bea40 0x56371e0cafe0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.306+0000 7f36683ec740 1 -- 172.21.15.1:0/3750332906 >> 172.21.15.1:0/3750332906 conn(0x56371e0bb0f0 msgr2=0x56371e0ba270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:01:17.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.306+0000 7f36683ec740 1 -- 172.21.15.1:0/3750332906 shutdown_connections 2024-09-11T22:01:17.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.306+0000 7f36683ec740 1 -- 172.21.15.1:0/3750332906 wait complete. 2024-09-11T22:01:17.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.306+0000 7f5cfcff9640 1 -- 172.21.15.1:0/2114801011 <== mon.0 v2:172.21.15.1:3300/0 4 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f5d0401a070 con 0x55cb41bf8f80 2024-09-11T22:01:17.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.306+0000 7f36683ec740 20 client.25739 trim_cache size 0 max 16384 2024-09-11T22:01:17.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.306+0000 7f5d1dba3740 10 client.16173 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:01:17.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.306+0000 7f5d1dba3740 10 client.16173 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-11T22:01:17.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.306+0000 7f5d1dba3740 20 client.16173 populate_metadata read hostname 'smithi001' 2024-09-11T22:01:17.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.306+0000 7f5d1dba3740 1 --2- 172.21.15.1:0/2114801011 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55cb41bfcd20 0x55cb41bff1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:17.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.306+0000 7f5d1dba3740 4 client.16173 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:01:17.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.306+0000 7f5d1dba3740 1 -- 172.21.15.1:0/2114801011 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb41b6b9d0 con 0x55cb41bfcd20 2024-09-11T22:01:17.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.307+0000 7f5d19566640 1 --2- 172.21.15.1:0/2114801011 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55cb41bfcd20 0x55cb41bff1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:17.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.307+0000 7f5d19566640 1 --2- 172.21.15.1:0/2114801011 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55cb41bfcd20 0x55cb41bff1e0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1c0e6367792e0b7 server_cookie=cf023919f9e0b7b in_seq=0 out_seq=0 2024-09-11T22:01:17.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.307+0000 7f5cfcff9640 10 client.16173 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:01:17.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.308+0000 7f5cfcff9640 1 -- 172.21.15.1:0/2114801011 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cb41b6b9d0 con 0x55cb41bfcd20 2024-09-11T22:01:17.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.308+0000 7f5cfcff9640 10 client.16173 handle_command_reply: tid=0 2024-09-11T22:01:17.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.308+0000 7f5d1dba3740 1 client.16173 shutdown 2024-09-11T22:01:17.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.308+0000 7f5d1dba3740 1 -- 172.21.15.1:0/2114801011 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cb41bf8f80 msgr2=0x55cb41c05130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:17.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.308+0000 7f5d1dba3740 1 --2- 172.21.15.1:0/2114801011 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cb41bf8f80 0x55cb41c05130 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f5d04013710 tx=0x7f5d04013bd0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.308+0000 7f5d1dba3740 1 -- 172.21.15.1:0/2114801011 shutdown_connections 2024-09-11T22:01:17.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.308+0000 7f5d1dba3740 1 --2- 172.21.15.1:0/2114801011 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55cb41bfcd20 0x55cb41bff1e0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.308+0000 7f5d1dba3740 1 --2- 172.21.15.1:0/2114801011 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb41c07d50 0x55cb41c05670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.308+0000 7f5cfcff9640 0 client.16173 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:01:17.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.308+0000 7f5d1dba3740 1 --2- 172.21.15.1:0/2114801011 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cb41bf8f80 0x55cb41c05130 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.308+0000 7f5d1dba3740 1 --2- 172.21.15.1:0/2114801011 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cb41bf8660 0x55cb41c04bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.308+0000 7f5d1dba3740 1 -- 172.21.15.1:0/2114801011 >> 172.21.15.1:0/2114801011 conn(0x55cb41bf4cf0 msgr2=0x55cb41bfaa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:01:17.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.308+0000 7f5d1dba3740 1 -- 172.21.15.1:0/2114801011 shutdown_connections 2024-09-11T22:01:17.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.308+0000 7f5d1dba3740 1 -- 172.21.15.1:0/2114801011 wait complete. 2024-09-11T22:01:17.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.309+0000 7f5d1dba3740 20 client.16173 trim_cache size 0 max 16384 2024-09-11T22:01:17.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.331+0000 7f36683ec740 1 Processor -- start 2024-09-11T22:01:17.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.331+0000 7f36683ec740 1 -- start start 2024-09-11T22:01:17.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.331+0000 7f36683ec740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56371e0bea40 0x56371e0e5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:17.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.331+0000 7f36683ec740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56371e0bf360 0x56371e0e6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:17.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.331+0000 7f36683ec740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56371e0ce150 0x56371e0c8280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:17.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.331+0000 7f36683ec740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f36601ab7f0 con 0x56371e0bea40 2024-09-11T22:01:17.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.331+0000 7f365f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56371e0bea40 0x56371e0e5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:17.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.331+0000 7f365f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56371e0bea40 0x56371e0e5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58718/0 (socket says 172.21.15.1:58718) 2024-09-11T22:01:17.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.331+0000 7f365f7fe640 1 -- 172.21.15.1:0/3098143377 learned_addr learned my addr 172.21.15.1:0/3098143377 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:01:17.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.331+0000 7f36683ec740 1 -- 172.21.15.1:0/3098143377 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f36601ad390 con 0x56371e0ce150 2024-09-11T22:01:17.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.331+0000 7f36683ec740 1 -- 172.21.15.1:0/3098143377 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f36601abec0 con 0x56371e0bf360 2024-09-11T22:01:17.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.331+0000 7f365ffff640 1 --2- 172.21.15.1:0/3098143377 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56371e0ce150 0x56371e0c8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:17.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.331+0000 7f3657fff640 1 --2- 172.21.15.1:0/3098143377 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56371e0bf360 0x56371e0e6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:17.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.331+0000 7f5d1dba3740 1 Processor -- start 2024-09-11T22:01:17.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.331+0000 7f5d1dba3740 1 -- start start 2024-09-11T22:01:17.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.331+0000 7f5d1dba3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cb41bf8660 0x55cb41c1b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:17.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.331+0000 7f5d1dba3740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cb41bf8f80 0x55cb41c1be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:17.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.332+0000 7f5d1dba3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb41c07d50 0x55cb41c01e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:17.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.332+0000 7f5d1dba3740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5d141ab510 con 0x55cb41bf8f80 2024-09-11T22:01:17.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.332+0000 7f5d1dba3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5d141ad0b0 con 0x55cb41c07d50 2024-09-11T22:01:17.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.332+0000 7f5d1dba3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5d141abbe0 con 0x55cb41bf8660 2024-09-11T22:01:17.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.332+0000 7f5d19566640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cb41bf8660 0x55cb41c1b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:17.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.332+0000 7f5d19566640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cb41bf8660 0x55cb41c1b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:45362/0 (socket says 172.21.15.1:45362) 2024-09-11T22:01:17.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.332+0000 7f5d19566640 1 -- 172.21.15.1:0/1586955597 learned_addr learned my addr 172.21.15.1:0/1586955597 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:01:17.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.332+0000 7f365f7fe640 1 -- 172.21.15.1:0/3098143377 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56371e0bf360 msgr2=0x56371e0e6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:17.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.332+0000 7f365f7fe640 1 --2- 172.21.15.1:0/3098143377 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56371e0bf360 0x56371e0e6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.332+0000 7f365f7fe640 1 -- 172.21.15.1:0/3098143377 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56371e0ce150 msgr2=0x56371e0c8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:17.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.332+0000 7f365f7fe640 1 --2- 172.21.15.1:0/3098143377 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56371e0ce150 0x56371e0c8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.332+0000 7f365f7fe640 1 -- 172.21.15.1:0/3098143377 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56371e0baf20 con 0x56371e0bea40 2024-09-11T22:01:17.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.332+0000 7f5d19d67640 1 --2- 172.21.15.1:0/1586955597 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb41c07d50 0x55cb41c01e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:17.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.332+0000 7f5d18d65640 1 --2- 172.21.15.1:0/1586955597 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cb41bf8f80 0x55cb41c1be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:17.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.332+0000 7f5d19566640 1 -- 172.21.15.1:0/1586955597 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb41c07d50 msgr2=0x55cb41c01e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:17.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.332+0000 7f5d19566640 1 --2- 172.21.15.1:0/1586955597 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb41c07d50 0x55cb41c01e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.332+0000 7f5d19566640 1 -- 172.21.15.1:0/1586955597 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cb41bf8f80 msgr2=0x55cb41c1be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:17.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.332+0000 7f5d19566640 1 --2- 172.21.15.1:0/1586955597 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cb41bf8f80 0x55cb41c1be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.332+0000 7f5d19566640 1 -- 172.21.15.1:0/1586955597 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb41bf4b20 con 0x55cb41bf8660 2024-09-11T22:01:17.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.332+0000 7f365f7fe640 1 --2- 172.21.15.1:0/3098143377 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56371e0bea40 0x56371e0e5bc0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f365000ea30 tx=0x7f365001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b824ed7820341869 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:01:17.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.332+0000 7f36557fa640 1 -- 172.21.15.1:0/3098143377 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3650004000 con 0x56371e0bea40 2024-09-11T22:01:17.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.332+0000 7f5d19566640 1 --2- 172.21.15.1:0/1586955597 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cb41bf8660 0x55cb41c1b8e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5d10006010 tx=0x7f5d1002a420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43881d2eac4886cf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:01:17.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.332+0000 7f36557fa640 1 -- 172.21.15.1:0/3098143377 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3650004210 con 0x56371e0bea40 2024-09-11T22:01:17.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.332+0000 7f36557fa640 1 -- 172.21.15.1:0/3098143377 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36500984c0 con 0x56371e0bea40 2024-09-11T22:01:17.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.332+0000 7f5cfeffd640 1 -- 172.21.15.1:0/1586955597 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d10068030 con 0x55cb41bf8660 2024-09-11T22:01:17.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.333+0000 7f5cfeffd640 1 -- 172.21.15.1:0/1586955597 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d10049be0 con 0x55cb41bf8660 2024-09-11T22:01:17.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.333+0000 7f5cfeffd640 1 -- 172.21.15.1:0/1586955597 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d10035040 con 0x55cb41bf8660 2024-09-11T22:01:17.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:01:16 smithi161 ceph-mon[35615]: pgmap v126: 97 pgs: 97 active+clean; 3.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 68 KiB/s wr, 36 op/s 2024-09-11T22:01:17.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.333+0000 7f36683ec740 1 -- 172.21.15.1:0/3098143377 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56371e0bea40 msgr2=0x56371e0e5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:17.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.333+0000 7f36683ec740 1 --2- 172.21.15.1:0/3098143377 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56371e0bea40 0x56371e0e5bc0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f365000ea30 tx=0x7f365001f000 comp rx=0 tx=0).stop 2024-09-11T22:01:17.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.333+0000 7f36683ec740 1 -- 172.21.15.1:0/3098143377 shutdown_connections 2024-09-11T22:01:17.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.333+0000 7f36683ec740 1 --2- 172.21.15.1:0/3098143377 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56371e0ce150 0x56371e0c8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.333+0000 7f36683ec740 1 --2- 172.21.15.1:0/3098143377 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56371e0bf360 0x56371e0e6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.333+0000 7f36683ec740 1 --2- 172.21.15.1:0/3098143377 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56371e0bea40 0x56371e0e5bc0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.333+0000 7f36683ec740 1 -- 172.21.15.1:0/3098143377 >> 172.21.15.1:0/3098143377 conn(0x56371e0bb0f0 msgr2=0x56371e0ba270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:01:17.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.333+0000 7f36683ec740 1 -- 172.21.15.1:0/3098143377 shutdown_connections 2024-09-11T22:01:17.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.333+0000 7f36683ec740 1 -- 172.21.15.1:0/3098143377 wait complete. 2024-09-11T22:01:17.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.334+0000 7f5d1dba3740 1 -- 172.21.15.1:0/1586955597 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cb41bf8660 msgr2=0x55cb41c1b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:17.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.334+0000 7f5d1dba3740 1 --2- 172.21.15.1:0/1586955597 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cb41bf8660 0x55cb41c1b8e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5d10006010 tx=0x7f5d1002a420 comp rx=0 tx=0).stop 2024-09-11T22:01:17.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.334+0000 7f5d1dba3740 1 -- 172.21.15.1:0/1586955597 shutdown_connections 2024-09-11T22:01:17.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.334+0000 7f5d1dba3740 1 --2- 172.21.15.1:0/1586955597 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb41c07d50 0x55cb41c01e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.334+0000 7f5d1dba3740 1 --2- 172.21.15.1:0/1586955597 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cb41bf8f80 0x55cb41c1be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.334+0000 7f5d1dba3740 1 --2- 172.21.15.1:0/1586955597 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cb41bf8660 0x55cb41c1b8e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.334+0000 7f5d1dba3740 1 -- 172.21.15.1:0/1586955597 >> 172.21.15.1:0/1586955597 conn(0x55cb41bf4cf0 msgr2=0x55cb41c06c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:01:17.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.334+0000 7f5d1dba3740 1 -- 172.21.15.1:0/1586955597 shutdown_connections 2024-09-11T22:01:17.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.334+0000 7f36683ec740 1 -- start start 2024-09-11T22:01:17.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.334+0000 7f5d1dba3740 1 -- 172.21.15.1:0/1586955597 wait complete. 2024-09-11T22:01:17.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.334+0000 7f36683ec740 1 Processor -- start 2024-09-11T22:01:17.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.334+0000 7f36683ec740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56371e0bea40 0x56371e0d63e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:17.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.334+0000 7f5d1dba3740 1 -- start start 2024-09-11T22:01:17.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.335+0000 7f5d1dba3740 1 Processor -- start 2024-09-11T22:01:17.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.334+0000 7f36683ec740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56371e0bf360 0x56371e0d6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:17.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.334+0000 7f36683ec740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56371e0ce150 0x56371e0d31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:17.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.335+0000 7f36683ec740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f36601acc90 con 0x56371e0bea40 2024-09-11T22:01:17.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.335+0000 7f36683ec740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f36601ac590 con 0x56371e0ce150 2024-09-11T22:01:17.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.335+0000 7f36683ec740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f36601a97d0 con 0x56371e0bf360 2024-09-11T22:01:17.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.335+0000 7f36683ec740 10 client.0 _ll_register_callbacks cb 0x7f3650009490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:01:17.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.335+0000 7f365f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56371e0bea40 0x56371e0d63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:17.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.335+0000 7f365f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56371e0bea40 0x56371e0d63e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58732/0 (socket says 172.21.15.1:58732) 2024-09-11T22:01:17.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.335+0000 7f365f7fe640 1 -- 172.21.15.1:0/2733391816 learned_addr learned my addr 172.21.15.1:0/2733391816 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:01:17.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.335+0000 7f365ffff640 1 --2- 172.21.15.1:0/2733391816 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56371e0ce150 0x56371e0d31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:17.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.335+0000 7f365f7fe640 1 -- 172.21.15.1:0/2733391816 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56371e0bf360 msgr2=0x56371e0d6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:17.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.335+0000 7f365f7fe640 1 --2- 172.21.15.1:0/2733391816 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56371e0bf360 0x56371e0d6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.335+0000 7f365f7fe640 1 -- 172.21.15.1:0/2733391816 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56371e0ce150 msgr2=0x56371e0d31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:17.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.335+0000 7f365f7fe640 1 --2- 172.21.15.1:0/2733391816 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56371e0ce150 0x56371e0d31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.335+0000 7f365f7fe640 1 -- 172.21.15.1:0/2733391816 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56371e0c5de0 con 0x56371e0bea40 2024-09-11T22:01:17.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.335+0000 7f365f7fe640 1 --2- 172.21.15.1:0/2733391816 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56371e0bea40 0x56371e0d63e0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f3650099fa0 tx=0x7f3650003f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66527eae4aad597f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:01:17.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.335+0000 7f363effd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:01:17.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.335+0000 7f363effd640 1 -- 172.21.15.1:0/2733391816 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3650097c00 con 0x56371e0bea40 2024-09-11T22:01:17.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.335+0000 7f363effd640 1 -- 172.21.15.1:0/2733391816 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f365009b140 con 0x56371e0bea40 2024-09-11T22:01:17.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.335+0000 7f5d1dba3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cb41bf8660 0x55cb41c0bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:17.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.336+0000 7f36683ec740 1 -- 172.21.15.1:0/2733391816 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3650004210 con 0x56371e0bea40 2024-09-11T22:01:17.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.336+0000 7f5d1dba3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb41bf8f80 0x55cb41c0c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:17.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.336+0000 7f363effd640 1 -- 172.21.15.1:0/2733391816 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3650097da0 con 0x56371e0bea40 2024-09-11T22:01:17.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.336+0000 7f5d1dba3740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cb41c07d50 0x55cb41c7d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:17.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.336+0000 7f5d1dba3740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5d141ac9b0 con 0x55cb41c07d50 2024-09-11T22:01:17.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.336+0000 7f5d18d65640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb41bf8f80 0x55cb41c0c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:17.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.336+0000 7f36683ec740 10 client.16182 fetch_fsmap learned FSMap version 30 2024-09-11T22:01:17.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.336+0000 7f36683ec740 1 -- 172.21.15.1:0/2733391816 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56371e0bbc30 con 0x56371e0bea40 2024-09-11T22:01:17.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.336+0000 7f5d18d65640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb41bf8f80 0x55cb41c0c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:35564/0 (socket says 172.21.15.1:35564) 2024-09-11T22:01:17.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.336+0000 7f5d18d65640 1 -- 172.21.15.1:0/3815439587 learned_addr learned my addr 172.21.15.1:0/3815439587 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:01:17.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.336+0000 7f5d19566640 1 --2- 172.21.15.1:0/3815439587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cb41bf8660 0x55cb41c0bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:17.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.336+0000 7f5d19d67640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cb41c07d50 0x55cb41c7d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:17.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.336+0000 7f5d1dba3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5d141ac2b0 con 0x55cb41bf8f80 2024-09-11T22:01:17.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.336+0000 7f5d1dba3740 1 -- 172.21.15.1:0/3815439587 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5d141a94f0 con 0x55cb41bf8660 2024-09-11T22:01:17.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.336+0000 7f363effd640 1 -- 172.21.15.1:0/2733391816 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3650004210 con 0x56371e0bea40 2024-09-11T22:01:17.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.336+0000 7f5d1dba3740 10 client.0 _ll_register_callbacks cb 0x7f5d04009dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:01:17.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.336+0000 7f363effd640 1 -- 172.21.15.1:0/2733391816 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f365009ac80 con 0x56371e0bea40 2024-09-11T22:01:17.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.336+0000 7f36683ec740 10 client.16182 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:01:17.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.336+0000 7f36683ec740 10 client.16182 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:01:17.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.336+0000 7f36683ec740 20 client.16182 populate_metadata read hostname 'smithi001' 2024-09-11T22:01:17.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.337+0000 7f5d18d65640 1 -- 172.21.15.1:0/3815439587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cb41bf8660 msgr2=0x55cb41c0bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:17.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.336+0000 7f36683ec740 1 --2- 172.21.15.1:0/2733391816 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x56371e0ddf50 0x56371e0c3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:17.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.337+0000 7f5d18d65640 1 --2- 172.21.15.1:0/3815439587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cb41bf8660 0x55cb41c0bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.337+0000 7f5d18d65640 1 -- 172.21.15.1:0/3815439587 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cb41c07d50 msgr2=0x55cb41c7d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:17.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.337+0000 7f5d18d65640 1 --2- 172.21.15.1:0/3815439587 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cb41c07d50 0x55cb41c7d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.337+0000 7f5d18d65640 1 -- 172.21.15.1:0/3815439587 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb41c0aa90 con 0x55cb41bf8f80 2024-09-11T22:01:17.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.336+0000 7f36683ec740 4 client.16182 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T22:01:17.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.336+0000 7f36683ec740 1 -- 172.21.15.1:0/2733391816 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56371e02bd00 con 0x56371e0ddf50 2024-09-11T22:01:17.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.337+0000 7f5d19d67640 1 --2- 172.21.15.1:0/3815439587 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cb41c07d50 0x55cb41c7d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:01:17.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.337+0000 7f3657fff640 1 --2- 172.21.15.1:0/2733391816 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x56371e0ddf50 0x56371e0c3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:17.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.337+0000 7f5d18d65640 1 --2- 172.21.15.1:0/3815439587 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb41bf8f80 0x55cb41c0c300 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x55cb41c04480 tx=0x7f5d040046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4973735b4f9b01b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:01:17.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.337+0000 7f5cfcff9640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-11T22:01:17.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.337+0000 7f363effd640 10 client.16182 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:01:17.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.337+0000 7f3657fff640 1 --2- 172.21.15.1:0/2733391816 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x56371e0ddf50 0x56371e0c3a00 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=199d0fd5c68171c9 server_cookie=9936077eef769069 in_seq=0 out_seq=0 2024-09-11T22:01:17.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.337+0000 7f5cfcff9640 1 -- 172.21.15.1:0/3815439587 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d040035f0 con 0x55cb41bf8f80 2024-09-11T22:01:17.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.337+0000 7f5d1dba3740 1 -- 172.21.15.1:0/3815439587 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d10049be0 con 0x55cb41bf8f80 2024-09-11T22:01:17.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.338+0000 7f5cfcff9640 1 -- 172.21.15.1:0/3815439587 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f5d04005020 con 0x55cb41bf8f80 2024-09-11T22:01:17.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.338+0000 7f5cfcff9640 1 -- 172.21.15.1:0/3815439587 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5d04003e90 con 0x55cb41bf8f80 2024-09-11T22:01:17.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.338+0000 7f5d1dba3740 10 client.25751 fetch_fsmap learned FSMap version 30 2024-09-11T22:01:17.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.338+0000 7f5d1dba3740 1 -- 172.21.15.1:0/3815439587 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cb41bf0260 con 0x55cb41bf8f80 2024-09-11T22:01:17.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.338+0000 7f5cfcff9640 1 -- 172.21.15.1:0/3815439587 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d10049be0 con 0x55cb41bf8f80 2024-09-11T22:01:17.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.338+0000 7f5cfcff9640 1 -- 172.21.15.1:0/3815439587 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f5d0401a040 con 0x55cb41bf8f80 2024-09-11T22:01:17.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.338+0000 7f5d1dba3740 10 client.25751 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:01:17.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.338+0000 7f5d1dba3740 10 client.25751 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-11T22:01:17.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.338+0000 7f5d1dba3740 20 client.25751 populate_metadata read hostname 'smithi001' 2024-09-11T22:01:17.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.339+0000 7f5d1dba3740 1 --2- 172.21.15.1:0/3815439587 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55cb41c7f280 0x55cb41c816a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:17.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.339+0000 7f5d1dba3740 4 client.25751 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.7, "await_for": 45.0}] 2024-09-11T22:01:17.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.339+0000 7f363effd640 1 -- 172.21.15.1:0/2733391816 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56371e02bd00 con 0x56371e0ddf50 2024-09-11T22:01:17.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.339+0000 7f5d1dba3740 1 -- 172.21.15.1:0/3815439587 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.7, "await_for": 45.0}) -- 0x55cb41b65d20 con 0x55cb41c7f280 2024-09-11T22:01:17.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.339+0000 7f363effd640 10 client.16182 handle_command_reply: tid=0 2024-09-11T22:01:17.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.339+0000 7f36683ec740 1 client.16182 shutdown 2024-09-11T22:01:17.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.339+0000 7f36683ec740 1 -- 172.21.15.1:0/2733391816 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56371e0bea40 msgr2=0x56371e0d63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:17.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.339+0000 7f5d19566640 1 --2- 172.21.15.1:0/3815439587 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55cb41c7f280 0x55cb41c816a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:17.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.339+0000 7f36683ec740 1 --2- 172.21.15.1:0/2733391816 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56371e0bea40 0x56371e0d63e0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f3650099fa0 tx=0x7f3650003f30 comp rx=0 tx=0).stop 2024-09-11T22:01:17.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.339+0000 7f36683ec740 1 -- 172.21.15.1:0/2733391816 shutdown_connections 2024-09-11T22:01:17.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.339+0000 7f36683ec740 1 --2- 172.21.15.1:0/2733391816 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x56371e0ddf50 0x56371e0c3a00 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.339+0000 7f36683ec740 1 --2- 172.21.15.1:0/2733391816 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56371e0ce150 0x56371e0d31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.339+0000 7f36683ec740 1 --2- 172.21.15.1:0/2733391816 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56371e0bf360 0x56371e0d6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.339+0000 7f36683ec740 1 --2- 172.21.15.1:0/2733391816 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56371e0bea40 0x56371e0d63e0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.339+0000 7f36683ec740 1 -- 172.21.15.1:0/2733391816 >> 172.21.15.1:0/2733391816 conn(0x56371e0bb0f0 msgr2=0x56371e0c0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:01:17.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.339+0000 7f363effd640 0 client.16182 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:01:17.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.339+0000 7f36683ec740 1 -- 172.21.15.1:0/2733391816 shutdown_connections 2024-09-11T22:01:17.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.339+0000 7f36683ec740 1 -- 172.21.15.1:0/2733391816 wait complete. 2024-09-11T22:01:17.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.339+0000 7f36683ec740 20 client.16182 trim_cache size 0 max 16384 2024-09-11T22:01:17.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.340+0000 7f5cfcff9640 10 client.25751 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:01:17.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.340+0000 7f5d19566640 1 --2- 172.21.15.1:0/3815439587 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55cb41c7f280 0x55cb41c816a0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2afd1c71f2e3d00 server_cookie=e8f91e82d2a35a54 in_seq=0 out_seq=0 2024-09-11T22:01:17.364 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-11T22:01:17.364 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-09-11T22:01:17.364 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "d15df636-adf5-49ee-97f8-ef1137c3b62b", 2024-09-11T22:01:17.364 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-09-11T22:01:17.365 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-11T22:01:17.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.340+0000 7f3666013640 1 -- 172.21.15.1:0/3150267863 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f364c0a2d70 msgr2=0x7f364c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:17.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.340+0000 7f3666013640 1 --2- 172.21.15.1:0/3150267863 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f364c0a2d70 0x7f364c0a5230 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f36600fedf0 tx=0x7f3648005f90 comp rx=0 tx=0).stop 2024-09-11T22:01:17.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.340+0000 7f3666013640 1 -- 172.21.15.1:0/3150267863 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3660102dd0 msgr2=0x7f36601985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:17.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.340+0000 7f3666013640 1 --2- 172.21.15.1:0/3150267863 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3660102dd0 0x7f36601985b0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f365000ea60 tx=0x7f365000ef20 comp rx=0 tx=0).stop 2024-09-11T22:01:17.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.340+0000 7f3666013640 1 -- 172.21.15.1:0/3150267863 shutdown_connections 2024-09-11T22:01:17.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.340+0000 7f3666013640 1 --2- 172.21.15.1:0/3150267863 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f364c0a2d70 0x7f364c0a5230 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.340+0000 7f3666013640 1 --2- 172.21.15.1:0/3150267863 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3660108300 0x7f366019ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.340+0000 7f3666013640 1 --2- 172.21.15.1:0/3150267863 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3660103780 0x7f3660198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.340+0000 7f3666013640 1 --2- 172.21.15.1:0/3150267863 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3660102dd0 0x7f36601985b0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.340+0000 7f3666013640 1 -- 172.21.15.1:0/3150267863 >> 172.21.15.1:0/3150267863 conn(0x7f36600fc7d0 msgr2=0x7f3660105d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:01:17.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.340+0000 7f3666013640 1 -- 172.21.15.1:0/3150267863 shutdown_connections 2024-09-11T22:01:17.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.340+0000 7f3666013640 1 -- 172.21.15.1:0/3150267863 wait complete. 2024-09-11T22:01:17.368 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.363+0000 7f8e97fff640 1 -- 172.21.15.1:0/881281586 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c6cf78bd70 con 0x55c6cf8a6540 2024-09-11T22:01:17.368 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.363+0000 7f8e97fff640 10 client.16143 handle_command_reply: tid=0 2024-09-11T22:01:17.368 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.363+0000 7f8ec1e6e740 1 client.16143 shutdown 2024-09-11T22:01:17.368 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.363+0000 7f8ec1e6e740 1 -- 172.21.15.1:0/881281586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c6cf81faf0 msgr2=0x55c6cf82ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:17.368 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.363+0000 7f8ec1e6e740 1 --2- 172.21.15.1:0/881281586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c6cf81faf0 0x55c6cf82ed40 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8011010 tx=0x7f8ea8013000 comp rx=0 tx=0).stop 2024-09-11T22:01:17.368 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.363+0000 7f8ec1e6e740 1 -- 172.21.15.1:0/881281586 shutdown_connections 2024-09-11T22:01:17.368 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.363+0000 7f8ec1e6e740 1 --2- 172.21.15.1:0/881281586 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55c6cf8a6540 0x55c6cf8a8960 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.363+0000 7f8ec1e6e740 1 --2- 172.21.15.1:0/881281586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c6cf82acb0 0x55c6cf8a48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.363+0000 7f8e97fff640 0 client.16143 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:01:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.363+0000 7f8ec1e6e740 1 --2- 172.21.15.1:0/881281586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c6cf820410 0x55c6cf82f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.363+0000 7f8ec1e6e740 1 --2- 172.21.15.1:0/881281586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c6cf81faf0 0x55c6cf82ed40 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.363+0000 7f8ec1e6e740 1 -- 172.21.15.1:0/881281586 >> 172.21.15.1:0/881281586 conn(0x55c6cf81c060 msgr2=0x55c6cf81a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:01:17.370 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.363+0000 7f8ec1e6e740 1 -- 172.21.15.1:0/881281586 shutdown_connections 2024-09-11T22:01:17.370 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.363+0000 7f8ec1e6e740 1 -- 172.21.15.1:0/881281586 wait complete. 2024-09-11T22:01:17.370 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.363+0000 7f8ec1e6e740 20 client.16143 trim_cache size 0 max 16384 2024-09-11T22:01:17.370 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-09-11T22:01:17.370 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-09-11T22:01:17.371 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-09-11T22:01:17.371 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-09-11T22:01:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.363+0000 7f8ebefff640 1 -- 172.21.15.1:0/2130968134 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f8ea40a2e40 msgr2=0x7f8ea40a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.363+0000 7f8ebefff640 1 --2- 172.21.15.1:0/2130968134 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f8ea40a2e40 0x7f8ea40a5300 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8005d00 tx=0x7f8ea800b410 comp rx=0 tx=0).stop 2024-09-11T22:01:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.363+0000 7f8ebefff640 1 -- 172.21.15.1:0/2130968134 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8eb8107ad0 msgr2=0x7f8eb819cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.363+0000 7f8ebefff640 1 --2- 172.21.15.1:0/2130968134 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8eb8107ad0 0x7f8eb819cd20 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f8eb000efd0 tx=0x7f8eb000c6a0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.364+0000 7f8ebefff640 1 -- 172.21.15.1:0/2130968134 shutdown_connections 2024-09-11T22:01:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.364+0000 7f8ebefff640 1 --2- 172.21.15.1:0/2130968134 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f8ea40a2e40 0x7f8ea40a5300 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.364+0000 7f8ebefff640 1 --2- 172.21.15.1:0/2130968134 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8eb8107ad0 0x7f8eb819cd20 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.364+0000 7f8ebefff640 1 --2- 172.21.15.1:0/2130968134 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8eb8102460 0x7f8eb81989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.364+0000 7f8ebefff640 1 --2- 172.21.15.1:0/2130968134 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8eb8101ac0 0x7f8eb8198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.364+0000 7f8ebefff640 1 -- 172.21.15.1:0/2130968134 >> 172.21.15.1:0/2130968134 conn(0x7f8eb80fd7a0 msgr2=0x7f8eb80fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:01:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.364+0000 7f8ebefff640 1 -- 172.21.15.1:0/2130968134 shutdown_connections 2024-09-11T22:01:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:17.364+0000 7f8ebefff640 1 -- 172.21.15.1:0/2130968134 wait complete. 2024-09-11T22:01:17.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.369+0000 7f5cfcff9640 1 -- 172.21.15.1:0/3815439587 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55cb41b65d20 con 0x55cb41c7f280 2024-09-11T22:01:17.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.369+0000 7f5cfcff9640 10 client.25751 handle_command_reply: tid=0 2024-09-11T22:01:17.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.369+0000 7f5d1dba3740 1 client.25751 shutdown 2024-09-11T22:01:17.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.370+0000 7f5d1dba3740 1 -- 172.21.15.1:0/3815439587 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb41bf8f80 msgr2=0x55cb41c0c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:17.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.370+0000 7f5d1dba3740 1 --2- 172.21.15.1:0/3815439587 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb41bf8f80 0x55cb41c0c300 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x55cb41c04480 tx=0x7f5d040046c0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.370+0000 7f5d1dba3740 1 -- 172.21.15.1:0/3815439587 shutdown_connections 2024-09-11T22:01:17.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.370+0000 7f5d1dba3740 1 --2- 172.21.15.1:0/3815439587 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55cb41c7f280 0x55cb41c816a0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.370+0000 7f5d1dba3740 1 --2- 172.21.15.1:0/3815439587 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cb41c07d50 0x55cb41c7d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.370+0000 7f5d1dba3740 1 --2- 172.21.15.1:0/3815439587 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cb41bf8f80 0x55cb41c0c300 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.370+0000 7f5cfcff9640 0 client.25751 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:01:17.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.370+0000 7f5d1dba3740 1 --2- 172.21.15.1:0/3815439587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cb41bf8660 0x55cb41c0bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.370+0000 7f5d1dba3740 1 -- 172.21.15.1:0/3815439587 >> 172.21.15.1:0/3815439587 conn(0x55cb41bf4cf0 msgr2=0x55cb41bf3650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:01:17.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.370+0000 7f5d1dba3740 1 -- 172.21.15.1:0/3815439587 shutdown_connections 2024-09-11T22:01:17.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.370+0000 7f5d1dba3740 1 -- 172.21.15.1:0/3815439587 wait complete. 2024-09-11T22:01:17.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.370+0000 7f5d1dba3740 20 client.25751 trim_cache size 0 max 16384 2024-09-11T22:01:17.375 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-11T22:01:17.375 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 30, 2024-09-11T22:01:17.375 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24494, 2024-09-11T22:01:17.375 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 10, 2024-09-11T22:01:17.375 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-09-11T22:01:17.375 INFO:teuthology.orchestra.run.smithi001.stdout: "d754d884": { 2024-09-11T22:01:17.375 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 10, 2024-09-11T22:01:17.375 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-09-11T22:01:17.375 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-11T22:01:17.376 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-09-11T22:01:17.376 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-11T22:01:17.376 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:01:17.376 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 90.0, 2024-09-11T22:01:17.376 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 144.7, 2024-09-11T22:01:17.376 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-09-11T22:01:17.376 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-09-11T22:01:17.376 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-09-11T22:01:17.376 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-11T22:01:17.376 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-09-11T22:01:17.376 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-11T22:01:17.377 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:01:17.377 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:01:17.377 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:01:17.377 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:01:17.377 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:01:17.377 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-11T22:01:17.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.370+0000 7f5d1b7ca640 1 -- 172.21.15.1:0/739985241 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f5d000a2bd0 msgr2=0x7f5d000a5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:17.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.370+0000 7f5d1b7ca640 1 --2- 172.21.15.1:0/739985241 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f5d000a2bd0 0x7f5d000a5090 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5d10009900 tx=0x7f5d100058d0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.370+0000 7f5d1b7ca640 1 -- 172.21.15.1:0/739985241 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d14108290 msgr2=0x7f5d1419ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:17.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.370+0000 7f5d1b7ca640 1 --2- 172.21.15.1:0/739985241 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d14108290 0x7f5d1419ce20 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c00cce0 tx=0x7f5d0c00a620 comp rx=0 tx=0).stop 2024-09-11T22:01:17.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.371+0000 7f5d1b7ca640 1 -- 172.21.15.1:0/739985241 shutdown_connections 2024-09-11T22:01:17.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.371+0000 7f5d1b7ca640 1 --2- 172.21.15.1:0/739985241 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f5d000a2bd0 0x7f5d000a5090 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.371+0000 7f5d1b7ca640 1 --2- 172.21.15.1:0/739985241 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d14108290 0x7f5d1419ce20 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.371+0000 7f5d1b7ca640 1 --2- 172.21.15.1:0/739985241 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d14103740 0x7f5d14198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.371+0000 7f5d1b7ca640 1 --2- 172.21.15.1:0/739985241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d14102d90 0x7f5d14198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:17.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.371+0000 7f5d1b7ca640 1 -- 172.21.15.1:0/739985241 >> 172.21.15.1:0/739985241 conn(0x7f5d140fc7d0 msgr2=0x7f5d14105d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:01:17.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.371+0000 7f5d1b7ca640 1 -- 172.21.15.1:0/739985241 shutdown_connections 2024-09-11T22:01:17.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:17.371+0000 7f5d1b7ca640 1 -- 172.21.15.1:0/739985241 wait complete. 2024-09-11T22:01:18.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:01:17 smithi001 ceph-mon[30935]: scrub queued for path: / 2024-09-11T22:01:18.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:01:17 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:01:18.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:01:17 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:01:18.194 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:01:17 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:01:18.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:01:17 smithi053 ceph-mon[34908]: scrub queued for path: / 2024-09-11T22:01:18.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:01:17 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:01:18.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:01:17 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:01:18.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:01:17 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:01:18.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:01:17 smithi161 ceph-mon[35615]: scrub queued for path: / 2024-09-11T22:01:18.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:01:17 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:01:18.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:01:17 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:01:18.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:01:17 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:01:18.665 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'd754d884', quiesce took 48.2 seconds. Will release after: -23.500000000000004 2024-09-11T22:01:18.666 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:01:19.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:01:18 smithi001 ceph-mon[30935]: pgmap v127: 97 pgs: 97 active+clean; 3.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 31 KiB/s wr, 10 op/s 2024-09-11T22:01:19.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:01:18 smithi053 ceph-mon[34908]: pgmap v127: 97 pgs: 97 active+clean; 3.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 31 KiB/s wr, 10 op/s 2024-09-11T22:01:19.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:01:18 smithi161 ceph-mon[35615]: pgmap v127: 97 pgs: 97 active+clean; 3.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 31 KiB/s wr, 10 op/s 2024-09-11T22:01:19.354 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:01:19.664 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:01:19.664 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:01:20.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.334+0000 7fd754df7640 1 -- 172.21.15.1:0/1828488944 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7501008d0 msgr2=0x7fd750100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:20.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.334+0000 7fd754df7640 1 --2- 172.21.15.1:0/1828488944 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7501008d0 0x7fd750100cb0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fd740009900 tx=0x7fd74002a410 comp rx=0 tx=0).stop 2024-09-11T22:01:20.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.334+0000 7fd754df7640 1 -- 172.21.15.1:0/1828488944 shutdown_connections 2024-09-11T22:01:20.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.334+0000 7fd754df7640 1 --2- 172.21.15.1:0/1828488944 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd75010e480 0x7fd750110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:20.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.334+0000 7fd754df7640 1 --2- 172.21.15.1:0/1828488944 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd750101280 0x7fd75010df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:20.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.334+0000 7fd754df7640 1 --2- 172.21.15.1:0/1828488944 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7501008d0 0x7fd750100cb0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:20.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.334+0000 7fd754df7640 1 -- 172.21.15.1:0/1828488944 >> 172.21.15.1:0/1828488944 conn(0x7fd7500fc7a0 msgr2=0x7fd7500febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:01:20.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.334+0000 7fd754df7640 1 -- 172.21.15.1:0/1828488944 shutdown_connections 2024-09-11T22:01:20.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.334+0000 7fd754df7640 1 -- 172.21.15.1:0/1828488944 wait complete. 2024-09-11T22:01:20.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.335+0000 7fd754df7640 1 Processor -- start 2024-09-11T22:01:20.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.335+0000 7fd754df7640 1 -- start start 2024-09-11T22:01:20.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.335+0000 7fd754df7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7501008d0 0x7fd75019a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:20.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.335+0000 7fd754df7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd750101280 0x7fd75019ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:20.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.336+0000 7fd754df7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd75010e480 0x7fd75019efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:20.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.336+0000 7fd754df7640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd7501141b0 con 0x7fd75010e480 2024-09-11T22:01:20.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.336+0000 7fd754df7640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd750114030 con 0x7fd7501008d0 2024-09-11T22:01:20.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.336+0000 7fd754df7640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd750114330 con 0x7fd750101280 2024-09-11T22:01:20.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.336+0000 7fd74ed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd75010e480 0x7fd75019efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:20.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.336+0000 7fd74ed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd75010e480 0x7fd75019efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58776/0 (socket says 172.21.15.1:58776) 2024-09-11T22:01:20.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.336+0000 7fd74ed9d640 1 -- 172.21.15.1:0/4248191613 learned_addr learned my addr 172.21.15.1:0/4248191613 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:01:20.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.336+0000 7fd747fff640 1 --2- 172.21.15.1:0/4248191613 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd750101280 0x7fd75019ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:20.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.336+0000 7fd74e59c640 1 --2- 172.21.15.1:0/4248191613 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7501008d0 0x7fd75019a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:20.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.336+0000 7fd74ed9d640 1 -- 172.21.15.1:0/4248191613 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd750101280 msgr2=0x7fd75019ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:20.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.336+0000 7fd74ed9d640 1 --2- 172.21.15.1:0/4248191613 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd750101280 0x7fd75019ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:20.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.336+0000 7fd74ed9d640 1 -- 172.21.15.1:0/4248191613 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7501008d0 msgr2=0x7fd75019a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:20.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.336+0000 7fd74ed9d640 1 --2- 172.21.15.1:0/4248191613 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7501008d0 0x7fd75019a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:20.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.336+0000 7fd74ed9d640 1 -- 172.21.15.1:0/4248191613 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd75019f680 con 0x7fd75010e480 2024-09-11T22:01:20.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.336+0000 7fd747fff640 1 --2- 172.21.15.1:0/4248191613 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd750101280 0x7fd75019ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:01:20.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.336+0000 7fd74e59c640 1 --2- 172.21.15.1:0/4248191613 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7501008d0 0x7fd75019a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:01:20.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.336+0000 7fd74ed9d640 1 --2- 172.21.15.1:0/4248191613 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd75010e480 0x7fd75019efa0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fd74800dac0 tx=0x7fd74800df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:01:20.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.336+0000 7fd7477fe640 1 -- 172.21.15.1:0/4248191613 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd748017030 con 0x7fd75010e480 2024-09-11T22:01:20.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.336+0000 7fd7477fe640 1 -- 172.21.15.1:0/4248191613 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd748003e50 con 0x7fd75010e480 2024-09-11T22:01:20.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.336+0000 7fd754df7640 1 -- 172.21.15.1:0/4248191613 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd75019f970 con 0x7fd75010e480 2024-09-11T22:01:20.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.336+0000 7fd7477fe640 1 -- 172.21.15.1:0/4248191613 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd748010460 con 0x7fd75010e480 2024-09-11T22:01:20.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.337+0000 7fd754df7640 1 -- 172.21.15.1:0/4248191613 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7501a7250 con 0x7fd75010e480 2024-09-11T22:01:20.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.339+0000 7fd7477fe640 1 -- 172.21.15.1:0/4248191613 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7fd74801a070 con 0x7fd75010e480 2024-09-11T22:01:20.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.340+0000 7fd7571d0740 1 Processor -- start 2024-09-11T22:01:20.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.340+0000 7fd7571d0740 1 -- start start 2024-09-11T22:01:20.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.340+0000 7fd7571d0740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a4eff6e4a0 0x55a4eff6e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:20.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.340+0000 7fd7571d0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a4eff6edc0 0x55a4eff79140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:20.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.340+0000 7fd7571d0740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a4eff79680 0x55a4eff7ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:20.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.340+0000 7fd7571d0740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55a4efed42e0 con 0x55a4eff79680 2024-09-11T22:01:20.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.340+0000 7fd7571d0740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55a4efe7bce0 con 0x55a4eff6e4a0 2024-09-11T22:01:20.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.340+0000 7fd7571d0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a4efed02a0 con 0x55a4eff6edc0 2024-09-11T22:01:20.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.340+0000 7fd747fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a4eff79680 0x55a4eff7ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:20.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.340+0000 7fd747fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a4eff79680 0x55a4eff7ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58790/0 (socket says 172.21.15.1:58790) 2024-09-11T22:01:20.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.340+0000 7fd747fff640 1 -- 172.21.15.1:0/118120957 learned_addr learned my addr 172.21.15.1:0/118120957 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:01:20.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.340+0000 7fd74e59c640 1 --2- 172.21.15.1:0/118120957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a4eff6edc0 0x55a4eff79140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:20.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.341+0000 7fd7477fe640 1 --2- 172.21.15.1:0/4248191613 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fd73c0a2cf0 0x7fd73c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:20.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.341+0000 7fd747fff640 1 --2- 172.21.15.1:0/118120957 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a4eff6e4a0 0x55a4eff6e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:20.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.341+0000 7fd747fff640 1 -- 172.21.15.1:0/118120957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a4eff6edc0 msgr2=0x55a4eff79140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:20.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.341+0000 7fd747fff640 1 --2- 172.21.15.1:0/118120957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a4eff6edc0 0x55a4eff79140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:20.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.341+0000 7fd747fff640 1 -- 172.21.15.1:0/118120957 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a4eff6e4a0 msgr2=0x55a4eff6e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:20.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.341+0000 7fd747fff640 1 --2- 172.21.15.1:0/118120957 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a4eff6e4a0 0x55a4eff6e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:20.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.341+0000 7fd747fff640 1 -- 172.21.15.1:0/118120957 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4efe60440 con 0x55a4eff79680 2024-09-11T22:01:20.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.341+0000 7fd74ed9d640 1 --2- 172.21.15.1:0/4248191613 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fd73c0a2cf0 0x7fd73c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:20.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.341+0000 7fd7477fe640 1 -- 172.21.15.1:0/4248191613 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7fd7480cf310 con 0x7fd75010e480 2024-09-11T22:01:20.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.341+0000 7fd747fff640 1 --2- 172.21.15.1:0/118120957 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a4eff79680 0x55a4eff7ba70 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fd738013690 tx=0x7fd73800c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6534b7093ae833a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:01:20.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.341+0000 7fd744ff9640 1 -- 172.21.15.1:0/118120957 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd738019030 con 0x55a4eff79680 2024-09-11T22:01:20.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.341+0000 7fd74ed9d640 1 --2- 172.21.15.1:0/4248191613 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fd73c0a2cf0 0x7fd73c0a51b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd7480ccad0 tx=0x7fd748004420 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:01:20.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.341+0000 7fd744ff9640 1 -- 172.21.15.1:0/118120957 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd738005500 con 0x55a4eff79680 2024-09-11T22:01:20.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.341+0000 7fd744ff9640 1 -- 172.21.15.1:0/118120957 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd738011830 con 0x55a4eff79680 2024-09-11T22:01:20.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.342+0000 7fd7571d0740 1 -- 172.21.15.1:0/118120957 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a4eff79680 msgr2=0x55a4eff7ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:20.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.342+0000 7fd7571d0740 1 --2- 172.21.15.1:0/118120957 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a4eff79680 0x55a4eff7ba70 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fd738013690 tx=0x7fd73800c750 comp rx=0 tx=0).stop 2024-09-11T22:01:20.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.342+0000 7fd7571d0740 1 -- 172.21.15.1:0/118120957 shutdown_connections 2024-09-11T22:01:20.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.342+0000 7fd7571d0740 1 --2- 172.21.15.1:0/118120957 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a4eff79680 0x55a4eff7ba70 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:20.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.342+0000 7fd7571d0740 1 --2- 172.21.15.1:0/118120957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a4eff6edc0 0x55a4eff79140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:20.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.342+0000 7fd7571d0740 1 --2- 172.21.15.1:0/118120957 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a4eff6e4a0 0x55a4eff6e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:20.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.342+0000 7fd7571d0740 1 -- 172.21.15.1:0/118120957 >> 172.21.15.1:0/118120957 conn(0x55a4eff6aa30 msgr2=0x55a4eff6ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:01:20.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.342+0000 7fd7571d0740 1 -- 172.21.15.1:0/118120957 shutdown_connections 2024-09-11T22:01:20.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.342+0000 7fd7571d0740 1 -- 172.21.15.1:0/118120957 wait complete. 2024-09-11T22:01:20.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.342+0000 7fd7571d0740 1 -- start start 2024-09-11T22:01:20.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.343+0000 7fd7571d0740 1 Processor -- start 2024-09-11T22:01:20.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.343+0000 7fd7571d0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a4eff6e4a0 0x55a4eff79010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:20.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.343+0000 7fd7571d0740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a4eff6edc0 0x55a4eff762c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:20.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.343+0000 7fd7571d0740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a4eff79680 0x55a4eff76840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:20.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.343+0000 7fd7571d0740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55a4eff7c250 con 0x55a4eff6edc0 2024-09-11T22:01:20.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.343+0000 7fd7571d0740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55a4efe8f050 con 0x55a4eff79680 2024-09-11T22:01:20.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.343+0000 7fd7571d0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a4eff7c3d0 con 0x55a4eff6e4a0 2024-09-11T22:01:20.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.343+0000 7fd7571d0740 10 client.0 _ll_register_callbacks cb 0x55a4efecf160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:01:20.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.343+0000 7fd74e59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a4eff6edc0 0x55a4eff762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:20.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.343+0000 7fd74e59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a4eff6edc0 0x55a4eff762c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58806/0 (socket says 172.21.15.1:58806) 2024-09-11T22:01:20.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.343+0000 7fd74e59c640 1 -- 172.21.15.1:0/463679193 learned_addr learned my addr 172.21.15.1:0/463679193 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:01:20.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.343+0000 7fd747fff640 1 --2- 172.21.15.1:0/463679193 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a4eff79680 0x55a4eff76840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:20.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.343+0000 7fd747fff640 1 --2- 172.21.15.1:0/463679193 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a4eff6e4a0 0x55a4eff79010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:20.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.343+0000 7fd74e59c640 1 -- 172.21.15.1:0/463679193 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a4eff6e4a0 msgr2=0x55a4eff79010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:20.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.343+0000 7fd74e59c640 1 --2- 172.21.15.1:0/463679193 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a4eff6e4a0 0x55a4eff79010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:20.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.343+0000 7fd74e59c640 1 -- 172.21.15.1:0/463679193 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a4eff79680 msgr2=0x55a4eff76840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:20.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.343+0000 7fd74e59c640 1 --2- 172.21.15.1:0/463679193 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a4eff79680 0x55a4eff76840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:20.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.343+0000 7fd74e59c640 1 -- 172.21.15.1:0/463679193 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4efc453c0 con 0x55a4eff6edc0 2024-09-11T22:01:20.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.343+0000 7fd747fff640 1 --2- 172.21.15.1:0/463679193 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a4eff79680 0x55a4eff76840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:01:20.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.344+0000 7fd74e59c640 1 --2- 172.21.15.1:0/463679193 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a4eff6edc0 0x55a4eff762c0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fd740009340 tx=0x7fd740031860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e58cebe177745148 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:01:20.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.344+0000 7fd735ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:01:20.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.344+0000 7fd735ffb640 1 -- 172.21.15.1:0/463679193 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd740035070 con 0x55a4eff6edc0 2024-09-11T22:01:20.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.344+0000 7fd735ffb640 1 -- 172.21.15.1:0/463679193 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7fd740004240 con 0x55a4eff6edc0 2024-09-11T22:01:20.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.344+0000 7fd7571d0740 1 -- 172.21.15.1:0/463679193 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd738005500 con 0x55a4eff6edc0 2024-09-11T22:01:20.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.344+0000 7fd735ffb640 1 -- 172.21.15.1:0/463679193 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd740004460 con 0x55a4eff6edc0 2024-09-11T22:01:20.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.344+0000 7fd7571d0740 10 client.16197 fetch_fsmap learned FSMap version 30 2024-09-11T22:01:20.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.344+0000 7fd7571d0740 1 -- 172.21.15.1:0/463679193 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a4eff87c40 con 0x55a4eff6edc0 2024-09-11T22:01:20.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.344+0000 7fd735ffb640 1 -- 172.21.15.1:0/463679193 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd738005500 con 0x55a4eff6edc0 2024-09-11T22:01:20.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.344+0000 7fd735ffb640 1 -- 172.21.15.1:0/463679193 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fd740004b30 con 0x55a4eff6edc0 2024-09-11T22:01:20.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.344+0000 7fd7571d0740 10 client.16197 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:01:20.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.344+0000 7fd7571d0740 10 client.16197 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:01:20.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.344+0000 7fd7571d0740 20 client.16197 populate_metadata read hostname 'smithi001' 2024-09-11T22:01:20.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.344+0000 7fd7571d0740 1 --2- 172.21.15.1:0/463679193 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55a4eff89150 0x55a4eff8b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:20.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.344+0000 7fd7571d0740 4 client.16197 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:01:20.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.344+0000 7fd7571d0740 1 -- 172.21.15.1:0/463679193 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a4efee1850 con 0x55a4eff89150 2024-09-11T22:01:20.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.345+0000 7fd747fff640 1 --2- 172.21.15.1:0/463679193 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55a4eff89150 0x55a4eff8b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:20.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.345+0000 7fd747fff640 1 --2- 172.21.15.1:0/463679193 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55a4eff89150 0x55a4eff8b590 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a5c260f661d9dbb server_cookie=deef66ebf2e1f962 in_seq=0 out_seq=0 2024-09-11T22:01:20.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.345+0000 7fd735ffb640 10 client.16197 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:01:20.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.346+0000 7fd735ffb640 1 -- 172.21.15.1:0/463679193 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a4efee1850 con 0x55a4eff89150 2024-09-11T22:01:20.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.346+0000 7fd735ffb640 10 client.16197 handle_command_reply: tid=0 2024-09-11T22:01:20.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.346+0000 7fd7571d0740 1 client.16197 shutdown 2024-09-11T22:01:20.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.346+0000 7fd7571d0740 1 -- 172.21.15.1:0/463679193 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a4eff6edc0 msgr2=0x55a4eff762c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:20.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.346+0000 7fd7571d0740 1 --2- 172.21.15.1:0/463679193 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a4eff6edc0 0x55a4eff762c0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fd740009340 tx=0x7fd740031860 comp rx=0 tx=0).stop 2024-09-11T22:01:20.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.346+0000 7fd7571d0740 1 -- 172.21.15.1:0/463679193 shutdown_connections 2024-09-11T22:01:20.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.346+0000 7fd7571d0740 1 --2- 172.21.15.1:0/463679193 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55a4eff89150 0x55a4eff8b590 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:20.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.346+0000 7fd7571d0740 1 --2- 172.21.15.1:0/463679193 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a4eff79680 0x55a4eff76840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:20.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.346+0000 7fd7571d0740 1 --2- 172.21.15.1:0/463679193 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a4eff6edc0 0x55a4eff762c0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:20.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.346+0000 7fd7571d0740 1 --2- 172.21.15.1:0/463679193 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a4eff6e4a0 0x55a4eff79010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:20.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.346+0000 7fd7571d0740 1 -- 172.21.15.1:0/463679193 >> 172.21.15.1:0/463679193 conn(0x55a4eff6aa30 msgr2=0x55a4eff70710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:01:20.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.346+0000 7fd735ffb640 0 client.16197 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:01:20.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.346+0000 7fd7571d0740 1 -- 172.21.15.1:0/463679193 shutdown_connections 2024-09-11T22:01:20.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.346+0000 7fd7571d0740 1 -- 172.21.15.1:0/463679193 wait complete. 2024-09-11T22:01:20.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.346+0000 7fd7571d0740 20 client.16197 trim_cache size 0 max 16384 2024-09-11T22:01:20.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.363+0000 7fd7571d0740 1 Processor -- start 2024-09-11T22:01:20.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.363+0000 7fd7571d0740 1 -- start start 2024-09-11T22:01:20.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.363+0000 7fd7571d0740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a4eff6e4a0 0x55a4eff854b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:20.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.363+0000 7fd7571d0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a4eff6edc0 0x55a4eff6ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:20.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.363+0000 7fd7571d0740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a4eff79680 0x55a4eff90e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:20.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.363+0000 7fd7571d0740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd7501afce0 con 0x55a4eff6e4a0 2024-09-11T22:01:20.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.363+0000 7fd7571d0740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd7501b1880 con 0x55a4eff79680 2024-09-11T22:01:20.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.363+0000 7fd7571d0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd7501b03b0 con 0x55a4eff6edc0 2024-09-11T22:01:20.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.364+0000 7fd747fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a4eff6e4a0 0x55a4eff854b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:20.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.364+0000 7fd747fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a4eff6e4a0 0x55a4eff854b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58808/0 (socket says 172.21.15.1:58808) 2024-09-11T22:01:20.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.364+0000 7fd747fff640 1 -- 172.21.15.1:0/952244684 learned_addr learned my addr 172.21.15.1:0/952244684 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:01:20.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.364+0000 7fd74e59c640 1 --2- 172.21.15.1:0/952244684 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a4eff6edc0 0x55a4eff6ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:20.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.364+0000 7fd747fff640 1 -- 172.21.15.1:0/952244684 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a4eff6edc0 msgr2=0x55a4eff6ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:20.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.364+0000 7fd747fff640 1 --2- 172.21.15.1:0/952244684 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a4eff6edc0 0x55a4eff6ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:20.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.364+0000 7fd747fff640 1 -- 172.21.15.1:0/952244684 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a4eff79680 msgr2=0x55a4eff90e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:20.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.364+0000 7fd747fff640 1 --2- 172.21.15.1:0/952244684 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a4eff79680 0x55a4eff90e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:20.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.364+0000 7fd747fff640 1 -- 172.21.15.1:0/952244684 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4eff6ebf0 con 0x55a4eff6e4a0 2024-09-11T22:01:20.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.364+0000 7fd747fff640 1 --2- 172.21.15.1:0/952244684 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a4eff6e4a0 0x55a4eff854b0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fd738004e50 tx=0x7fd738013460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c1136ec4c16ff6f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:01:20.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.364+0000 7fd737fff640 1 -- 172.21.15.1:0/952244684 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73802e020 con 0x55a4eff6e4a0 2024-09-11T22:01:20.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.364+0000 7fd737fff640 1 -- 172.21.15.1:0/952244684 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd738004ef0 con 0x55a4eff6e4a0 2024-09-11T22:01:20.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.364+0000 7fd737fff640 1 -- 172.21.15.1:0/952244684 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd738019040 con 0x55a4eff6e4a0 2024-09-11T22:01:20.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.365+0000 7fd7571d0740 1 -- 172.21.15.1:0/952244684 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a4eff6e4a0 msgr2=0x55a4eff854b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:20.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.365+0000 7fd7571d0740 1 --2- 172.21.15.1:0/952244684 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a4eff6e4a0 0x55a4eff854b0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fd738004e50 tx=0x7fd738013460 comp rx=0 tx=0).stop 2024-09-11T22:01:20.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.365+0000 7fd7571d0740 1 -- 172.21.15.1:0/952244684 shutdown_connections 2024-09-11T22:01:20.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.365+0000 7fd7571d0740 1 --2- 172.21.15.1:0/952244684 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a4eff79680 0x55a4eff90e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:20.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.365+0000 7fd7571d0740 1 --2- 172.21.15.1:0/952244684 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a4eff6edc0 0x55a4eff6ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:20.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.365+0000 7fd7571d0740 1 --2- 172.21.15.1:0/952244684 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a4eff6e4a0 0x55a4eff854b0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:20.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.365+0000 7fd7571d0740 1 -- 172.21.15.1:0/952244684 >> 172.21.15.1:0/952244684 conn(0x55a4eff6aa30 msgr2=0x55a4eff718b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:01:20.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.365+0000 7fd7571d0740 1 -- 172.21.15.1:0/952244684 shutdown_connections 2024-09-11T22:01:20.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.365+0000 7fd7571d0740 1 -- 172.21.15.1:0/952244684 wait complete. 2024-09-11T22:01:20.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.365+0000 7fd7571d0740 1 -- start start 2024-09-11T22:01:20.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.365+0000 7fd7571d0740 1 Processor -- start 2024-09-11T22:01:20.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.366+0000 7fd7571d0740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a4eff6e4a0 0x55a4eff81c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:20.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.366+0000 7fd7571d0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a4eff6edc0 0x55a4eff72f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:20.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.366+0000 7fd7571d0740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a4eff79680 0x55a4eff73460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:20.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.366+0000 7fd7571d0740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd7501b1180 con 0x55a4eff6e4a0 2024-09-11T22:01:20.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.366+0000 7fd7571d0740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd7501b0a80 con 0x55a4eff79680 2024-09-11T22:01:20.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.366+0000 7fd7571d0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd7501adcc0 con 0x55a4eff6edc0 2024-09-11T22:01:20.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.366+0000 7fd7571d0740 10 client.0 _ll_register_callbacks cb 0x7fd740009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:01:20.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.366+0000 7fd747fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a4eff6e4a0 0x55a4eff81c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:20.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.366+0000 7fd747fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a4eff6e4a0 0x55a4eff81c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58822/0 (socket says 172.21.15.1:58822) 2024-09-11T22:01:20.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.366+0000 7fd747fff640 1 -- 172.21.15.1:0/3760495033 learned_addr learned my addr 172.21.15.1:0/3760495033 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:01:20.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.366+0000 7fd74e59c640 1 --2- 172.21.15.1:0/3760495033 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a4eff6edc0 0x55a4eff72f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:20.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.366+0000 7fd747fff640 1 --2- 172.21.15.1:0/3760495033 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a4eff79680 0x55a4eff73460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:20.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.366+0000 7fd747fff640 1 -- 172.21.15.1:0/3760495033 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a4eff6edc0 msgr2=0x55a4eff72f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:20.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.366+0000 7fd747fff640 1 --2- 172.21.15.1:0/3760495033 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a4eff6edc0 0x55a4eff72f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:20.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.366+0000 7fd747fff640 1 -- 172.21.15.1:0/3760495033 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a4eff79680 msgr2=0x55a4eff73460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:20.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.366+0000 7fd747fff640 1 --2- 172.21.15.1:0/3760495033 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a4eff79680 0x55a4eff73460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:20.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.366+0000 7fd747fff640 1 -- 172.21.15.1:0/3760495033 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4eff76bb0 con 0x55a4eff6e4a0 2024-09-11T22:01:20.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.366+0000 7fd74e59c640 1 --2- 172.21.15.1:0/3760495033 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a4eff6edc0 0x55a4eff72f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:01:20.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.366+0000 7fd747fff640 1 --2- 172.21.15.1:0/3760495033 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a4eff6e4a0 0x55a4eff81c50 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fd73800c790 tx=0x7fd738005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc569fbf64cc8df2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:01:20.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.366+0000 7fd735ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:01:20.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.366+0000 7fd735ffb640 1 -- 172.21.15.1:0/3760495033 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73802e050 con 0x55a4eff6e4a0 2024-09-11T22:01:20.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.367+0000 7fd735ffb640 1 -- 172.21.15.1:0/3760495033 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7fd738004780 con 0x55a4eff6e4a0 2024-09-11T22:01:20.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.367+0000 7fd7571d0740 1 -- 172.21.15.1:0/3760495033 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd738004ef0 con 0x55a4eff6e4a0 2024-09-11T22:01:20.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.367+0000 7fd735ffb640 1 -- 172.21.15.1:0/3760495033 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd738046050 con 0x55a4eff6e4a0 2024-09-11T22:01:20.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.367+0000 7fd7571d0740 10 client.16203 fetch_fsmap learned FSMap version 30 2024-09-11T22:01:20.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.367+0000 7fd7571d0740 1 -- 172.21.15.1:0/3760495033 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a4efff3ba0 con 0x55a4eff6e4a0 2024-09-11T22:01:20.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.367+0000 7fd735ffb640 1 -- 172.21.15.1:0/3760495033 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd738004ef0 con 0x55a4eff6e4a0 2024-09-11T22:01:20.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.367+0000 7fd735ffb640 1 -- 172.21.15.1:0/3760495033 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fd73802a3d0 con 0x55a4eff6e4a0 2024-09-11T22:01:20.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.367+0000 7fd7571d0740 10 client.16203 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:01:20.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.367+0000 7fd7571d0740 10 client.16203 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:01:20.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.367+0000 7fd7571d0740 20 client.16203 populate_metadata read hostname 'smithi001' 2024-09-11T22:01:20.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.367+0000 7fd7571d0740 1 --2- 172.21.15.1:0/3760495033 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55a4efff4010 0x55a4efff6400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:20.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.367+0000 7fd7571d0740 4 client.16203 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:01:20.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.367+0000 7fd7571d0740 1 -- 172.21.15.1:0/3760495033 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a4efedbd60 con 0x55a4efff4010 2024-09-11T22:01:20.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.367+0000 7fd74e59c640 1 --2- 172.21.15.1:0/3760495033 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55a4efff4010 0x55a4efff6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:20.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.368+0000 7fd74e59c640 1 --2- 172.21.15.1:0/3760495033 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55a4efff4010 0x55a4efff6400 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3287412fd1751f01 server_cookie=70d2a2cc66337096 in_seq=0 out_seq=0 2024-09-11T22:01:20.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.368+0000 7fd735ffb640 10 client.16203 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:01:20.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.368+0000 7fd735ffb640 1 -- 172.21.15.1:0/3760495033 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55a4efedbd60 con 0x55a4efff4010 2024-09-11T22:01:20.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.368+0000 7fd735ffb640 10 client.16203 handle_command_reply: tid=0 2024-09-11T22:01:20.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.368+0000 7fd7571d0740 1 client.16203 shutdown 2024-09-11T22:01:20.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.368+0000 7fd7571d0740 1 -- 172.21.15.1:0/3760495033 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a4eff6e4a0 msgr2=0x55a4eff81c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:20.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.368+0000 7fd7571d0740 1 --2- 172.21.15.1:0/3760495033 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a4eff6e4a0 0x55a4eff81c50 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fd73800c790 tx=0x7fd738005850 comp rx=0 tx=0).stop 2024-09-11T22:01:20.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.368+0000 7fd7571d0740 1 -- 172.21.15.1:0/3760495033 shutdown_connections 2024-09-11T22:01:20.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.368+0000 7fd7571d0740 1 --2- 172.21.15.1:0/3760495033 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55a4efff4010 0x55a4efff6400 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:20.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.368+0000 7fd7571d0740 1 --2- 172.21.15.1:0/3760495033 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a4eff79680 0x55a4eff73460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:20.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.368+0000 7fd7571d0740 1 --2- 172.21.15.1:0/3760495033 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a4eff6edc0 0x55a4eff72f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:20.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.368+0000 7fd7571d0740 1 --2- 172.21.15.1:0/3760495033 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a4eff6e4a0 0x55a4eff81c50 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:20.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.368+0000 7fd7571d0740 1 -- 172.21.15.1:0/3760495033 >> 172.21.15.1:0/3760495033 conn(0x55a4eff6aa30 msgr2=0x55a4eff77b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:01:20.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.368+0000 7fd735ffb640 0 client.16203 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:01:20.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.368+0000 7fd7571d0740 1 -- 172.21.15.1:0/3760495033 shutdown_connections 2024-09-11T22:01:20.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.368+0000 7fd7571d0740 1 -- 172.21.15.1:0/3760495033 wait complete. 2024-09-11T22:01:20.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.368+0000 7fd7571d0740 20 client.16203 trim_cache size 0 max 16384 2024-09-11T22:01:20.376 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-11T22:01:20.376 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T22:01:20.376 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-09-11T22:01:20.376 INFO:teuthology.orchestra.run.smithi001.stdout: "d15df636-adf5-49ee-97f8-ef1137c3b62b": { 2024-09-11T22:01:20.376 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2024-09-11T22:01:20.376 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "d15df636-adf5-49ee-97f8-ef1137c3b62b", 2024-09-11T22:01:20.376 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2024-09-11T22:01:20.376 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:01:20.377 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:01:20.377 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-11T22:01:20.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.369+0000 7fd754df7640 1 -- 172.21.15.1:0/4248191613 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fd73c0a2cf0 msgr2=0x7fd73c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:20.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.369+0000 7fd754df7640 1 --2- 172.21.15.1:0/4248191613 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fd73c0a2cf0 0x7fd73c0a51b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd7480ccad0 tx=0x7fd748004420 comp rx=0 tx=0).stop 2024-09-11T22:01:20.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.369+0000 7fd754df7640 1 -- 172.21.15.1:0/4248191613 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd75010e480 msgr2=0x7fd75019efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:20.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.369+0000 7fd754df7640 1 --2- 172.21.15.1:0/4248191613 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd75010e480 0x7fd75019efa0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fd74800dac0 tx=0x7fd74800df80 comp rx=0 tx=0).stop 2024-09-11T22:01:20.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.369+0000 7fd754df7640 1 -- 172.21.15.1:0/4248191613 shutdown_connections 2024-09-11T22:01:20.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.369+0000 7fd754df7640 1 --2- 172.21.15.1:0/4248191613 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fd73c0a2cf0 0x7fd73c0a51b0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:20.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.369+0000 7fd754df7640 1 --2- 172.21.15.1:0/4248191613 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd75010e480 0x7fd75019efa0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:20.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.369+0000 7fd754df7640 1 --2- 172.21.15.1:0/4248191613 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd750101280 0x7fd75019ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:20.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.369+0000 7fd754df7640 1 --2- 172.21.15.1:0/4248191613 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7501008d0 0x7fd75019a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:20.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.369+0000 7fd754df7640 1 -- 172.21.15.1:0/4248191613 >> 172.21.15.1:0/4248191613 conn(0x7fd7500fc7a0 msgr2=0x7fd75010f860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:01:20.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.369+0000 7fd754df7640 1 -- 172.21.15.1:0/4248191613 shutdown_connections 2024-09-11T22:01:20.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.369+0000 7fd754df7640 1 -- 172.21.15.1:0/4248191613 wait complete. 2024-09-11T22:01:20.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.554+0000 7fb6fe34a640 1 -- 172.21.15.1:0/1565143851 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb6f8101de0 msgr2=0x7fb6f81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:20.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.554+0000 7fb6fe34a640 1 --2- 172.21.15.1:0/1565143851 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb6f8101de0 0x7fb6f81021c0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8009900 tx=0x7fb6e802a410 comp rx=0 tx=0).stop 2024-09-11T22:01:20.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.554+0000 7fb6fe34a640 1 -- 172.21.15.1:0/1565143851 shutdown_connections 2024-09-11T22:01:20.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.554+0000 7fb6fe34a640 1 --2- 172.21.15.1:0/1565143851 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6f8106f10 0x7fb6f810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:20.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.554+0000 7fb6fe34a640 1 --2- 172.21.15.1:0/1565143851 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6f8102790 0x7fb6f81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:20.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.554+0000 7fb6fe34a640 1 --2- 172.21.15.1:0/1565143851 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb6f8101de0 0x7fb6f81021c0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:20.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.554+0000 7fb6fe34a640 1 -- 172.21.15.1:0/1565143851 >> 172.21.15.1:0/1565143851 conn(0x7fb6f80fd8c0 msgr2=0x7fb6f80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:01:20.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.554+0000 7fb6fe34a640 1 -- 172.21.15.1:0/1565143851 shutdown_connections 2024-09-11T22:01:20.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.554+0000 7fb6fe34a640 1 -- 172.21.15.1:0/1565143851 wait complete. 2024-09-11T22:01:20.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.555+0000 7fb6fe34a640 1 Processor -- start 2024-09-11T22:01:20.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.555+0000 7fb6fe34a640 1 -- start start 2024-09-11T22:01:20.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.555+0000 7fb6fe34a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6f8101de0 0x7fb6f8198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:20.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.555+0000 7fb6fe34a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6f8102790 0x7fb6f81989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:20.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.555+0000 7fb6fe34a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb6f8106f10 0x7fb6f819cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:20.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.555+0000 7fb6fe34a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f8111f90 con 0x7fb6f8106f10 2024-09-11T22:01:20.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.555+0000 7fb6fe34a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f8111e10 con 0x7fb6f8101de0 2024-09-11T22:01:20.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.555+0000 7fb6fe34a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f8112110 con 0x7fb6f8102790 2024-09-11T22:01:20.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.555+0000 7fb6fc8e7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb6f8106f10 0x7fb6f819cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:20.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.555+0000 7fb6fc8e7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb6f8106f10 0x7fb6f819cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58846/0 (socket says 172.21.15.1:58846) 2024-09-11T22:01:20.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.555+0000 7fb6fc8e7640 1 -- 172.21.15.1:0/1254945076 learned_addr learned my addr 172.21.15.1:0/1254945076 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:01:20.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.556+0000 7fb6f7fff640 1 --2- 172.21.15.1:0/1254945076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6f8101de0 0x7fb6f8198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:20.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.556+0000 7fb6fc8e7640 1 -- 172.21.15.1:0/1254945076 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6f8102790 msgr2=0x7fb6f81989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:20.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.556+0000 7fb6f77fe640 1 --2- 172.21.15.1:0/1254945076 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6f8102790 0x7fb6f81989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:20.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.556+0000 7fb6fc8e7640 1 --2- 172.21.15.1:0/1254945076 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6f8102790 0x7fb6f81989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:20.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.556+0000 7fb6fc8e7640 1 -- 172.21.15.1:0/1254945076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6f8101de0 msgr2=0x7fb6f8198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:20.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.556+0000 7fb6fc8e7640 1 --2- 172.21.15.1:0/1254945076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6f8101de0 0x7fb6f8198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:20.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.556+0000 7fb6fc8e7640 1 -- 172.21.15.1:0/1254945076 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6f819d430 con 0x7fb6f8106f10 2024-09-11T22:01:20.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.556+0000 7fb6fc8e7640 1 --2- 172.21.15.1:0/1254945076 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb6f8106f10 0x7fb6f819cd50 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fb6f000b970 tx=0x7fb6f000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:01:20.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.556+0000 7fb6f57fa640 1 -- 172.21.15.1:0/1254945076 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f0019070 con 0x7fb6f8106f10 2024-09-11T22:01:20.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.556+0000 7fb6fe34a640 1 -- 172.21.15.1:0/1254945076 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6f819d720 con 0x7fb6f8106f10 2024-09-11T22:01:20.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.556+0000 7fb6f57fa640 1 -- 172.21.15.1:0/1254945076 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6f000cb40 con 0x7fb6f8106f10 2024-09-11T22:01:20.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.556+0000 7fb6f57fa640 1 -- 172.21.15.1:0/1254945076 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f0007610 con 0x7fb6f8106f10 2024-09-11T22:01:20.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.556+0000 7fb6fe34a640 1 -- 172.21.15.1:0/1254945076 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6f81a5000 con 0x7fb6f8106f10 2024-09-11T22:01:20.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.557+0000 7fb6fe34a640 1 -- 172.21.15.1:0/1254945076 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6f81021c0 con 0x7fb6f8106f10 2024-09-11T22:01:20.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.559+0000 7fb6f57fa640 1 -- 172.21.15.1:0/1254945076 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7fb6f001c070 con 0x7fb6f8106f10 2024-09-11T22:01:20.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.559+0000 7fb6f57fa640 1 --2- 172.21.15.1:0/1254945076 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fb6e00a2cf0 0x7fb6e00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:20.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.559+0000 7fb6f7fff640 1 --2- 172.21.15.1:0/1254945076 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fb6e00a2cf0 0x7fb6e00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:20.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.559+0000 7fb6f57fa640 1 -- 172.21.15.1:0/1254945076 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7fb6f00ce910 con 0x7fb6f8106f10 2024-09-11T22:01:20.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.560+0000 7fb6f7fff640 1 --2- 172.21.15.1:0/1254945076 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fb6e00a2cf0 0x7fb6e00a51b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8009230 tx=0x7fb6e80058d0 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:01:20.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.564+0000 7fb6f57fa640 1 -- 172.21.15.1:0/1254945076 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb6f009afb0 con 0x7fb6f8106f10 2024-09-11T22:01:20.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.783+0000 7fb6fe34a640 1 -- 172.21.15.1:0/1254945076 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb6f802d7c0 con 0x7fb6f8106f10 2024-09-11T22:01:20.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.784+0000 7fb6f57fa640 1 -- 172.21.15.1:0/1254945076 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11951 (secure 0 0 0) 0x7fb6f009f2b0 con 0x7fb6f8106f10 2024-09-11T22:01:20.785 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T22:01:20.785 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":30,"btime":"2024-09-11T22:00:27:488718+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/421542832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":421542832},{"type":"v1","addr":"172.21.15.1:6835","nonce":421542832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2836864406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2836864406},{"type":"v1","addr":"172.21.15.1:6837","nonce":2836864406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T21:55:20.791263+0000","modified":"2024-09-11T22:00:26.715155+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24484,"mds_2":24467,"mds_3":24466,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14595":{"gid":14595,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/407834672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":407834672},{"type":"v1","addr":"172.21.15.1:6839","nonce":407834672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/1068827678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":1068827678},{"type":"v1","addr":"172.21.15.1:6841","nonce":1068827678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/872410024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":872410024},{"type":"v1","addr":"172.21.15.161:6833","nonce":872410024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/1648944570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1648944570},{"type":"v1","addr":"172.21.15.53:6833","nonce":1648944570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/2126933993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2126933993},{"type":"v1","addr":"172.21.15.53:6835","nonce":2126933993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.161:6835/828381592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":828381592},{"type":"v1","addr":"172.21.15.161:6835","nonce":828381592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.161:6837/1799318656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1799318656},{"type":"v1","addr":"172.21.15.161:6837","nonce":1799318656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.53:6837/3457131830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3457131830},{"type":"v1","addr":"172.21.15.53:6837","nonce":3457131830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/455365383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":455365383},{"type":"v1","addr":"172.21.15.161:6839","nonce":455365383}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.53:6839/1378584447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1378584447},{"type":"v1","addr":"172.21.15.53:6839","nonce":1378584447}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24460,24466,24467,24484,24494]},"id":1}]} 2024-09-11T22:01:20.785 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 30 2024-09-11T22:01:20.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.786+0000 7fb6fe34a640 1 -- 172.21.15.1:0/1254945076 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fb6e00a2cf0 msgr2=0x7fb6e00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:20.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.786+0000 7fb6fe34a640 1 --2- 172.21.15.1:0/1254945076 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fb6e00a2cf0 0x7fb6e00a51b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8009230 tx=0x7fb6e80058d0 comp rx=0 tx=0).stop 2024-09-11T22:01:20.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.786+0000 7fb6fe34a640 1 -- 172.21.15.1:0/1254945076 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb6f8106f10 msgr2=0x7fb6f819cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:20.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.786+0000 7fb6fe34a640 1 --2- 172.21.15.1:0/1254945076 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb6f8106f10 0x7fb6f819cd50 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fb6f000b970 tx=0x7fb6f000be30 comp rx=0 tx=0).stop 2024-09-11T22:01:20.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.787+0000 7fb6fe34a640 1 -- 172.21.15.1:0/1254945076 shutdown_connections 2024-09-11T22:01:20.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.787+0000 7fb6fe34a640 1 --2- 172.21.15.1:0/1254945076 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fb6e00a2cf0 0x7fb6e00a51b0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:20.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.787+0000 7fb6fe34a640 1 --2- 172.21.15.1:0/1254945076 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb6f8106f10 0x7fb6f819cd50 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:20.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.787+0000 7fb6fe34a640 1 --2- 172.21.15.1:0/1254945076 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6f8102790 0x7fb6f81989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:20.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.787+0000 7fb6fe34a640 1 --2- 172.21.15.1:0/1254945076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6f8101de0 0x7fb6f8198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:20.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.787+0000 7fb6fe34a640 1 -- 172.21.15.1:0/1254945076 >> 172.21.15.1:0/1254945076 conn(0x7fb6f80fd8c0 msgr2=0x7fb6f80fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:01:20.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.787+0000 7fb6fe34a640 1 -- 172.21.15.1:0/1254945076 shutdown_connections 2024-09-11T22:01:20.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:20.787+0000 7fb6fe34a640 1 -- 172.21.15.1:0/1254945076 wait complete. 2024-09-11T22:01:21.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:01:20 smithi001 ceph-mon[30935]: pgmap v128: 97 pgs: 97 active+clean; 4.0 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-09-11T22:01:21.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:01:20 smithi001 ceph-mon[30935]: scrub summary: idle 2024-09-11T22:01:21.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:01:20 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/1254945076' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:01:21.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:01:20 smithi053 ceph-mon[34908]: pgmap v128: 97 pgs: 97 active+clean; 4.0 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-09-11T22:01:21.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:01:20 smithi053 ceph-mon[34908]: scrub summary: idle 2024-09-11T22:01:21.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:01:20 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/1254945076' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:01:21.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:01:20 smithi161 ceph-mon[35615]: pgmap v128: 97 pgs: 97 active+clean; 4.0 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-09-11T22:01:21.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:01:20 smithi161 ceph-mon[35615]: scrub summary: idle 2024-09-11T22:01:21.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:01:20 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/1254945076' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:01:21.686 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db --set-id d754d884 --release --await' 2024-09-11T22:01:21.687 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db --set-id d754d884 --release --await 2024-09-11T22:01:22.149 INFO:tasks.cephfs.filesystem:scrub status for tag:d15df636-adf5-49ee-97f8-ef1137c3b62b - {'path': '/', 'tag': 'd15df636-adf5-49ee-97f8-ef1137c3b62b', 'options': 'recursive,force'} 2024-09-11T22:01:22.283 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:01:22.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.891+0000 7f73d1554640 1 -- 172.21.15.1:0/823699430 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f73cc108290 msgr2=0x7f73cc10ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:22.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.891+0000 7f73d1554640 1 --2- 172.21.15.1:0/823699430 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f73cc108290 0x7f73cc10ec30 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f73c4009cd0 tx=0x7f73c402a8e0 comp rx=0 tx=0).stop 2024-09-11T22:01:22.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.891+0000 7f73d1554640 1 -- 172.21.15.1:0/823699430 shutdown_connections 2024-09-11T22:01:22.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.891+0000 7f73d1554640 1 --2- 172.21.15.1:0/823699430 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f73cc108290 0x7f73cc10ec30 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:22.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.891+0000 7f73d1554640 1 --2- 172.21.15.1:0/823699430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73cc103740 0x7f73cc107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:22.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.891+0000 7f73d1554640 1 --2- 172.21.15.1:0/823699430 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73cc102d90 0x7f73cc103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:22.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.891+0000 7f73d1554640 1 -- 172.21.15.1:0/823699430 >> 172.21.15.1:0/823699430 conn(0x7f73cc0fc7d0 msgr2=0x7f73cc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:01:22.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.891+0000 7f73d1554640 1 -- 172.21.15.1:0/823699430 shutdown_connections 2024-09-11T22:01:22.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.891+0000 7f73d1554640 1 -- 172.21.15.1:0/823699430 wait complete. 2024-09-11T22:01:22.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.891+0000 7f73d1554640 1 Processor -- start 2024-09-11T22:01:22.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.892+0000 7f73d1554640 1 -- start start 2024-09-11T22:01:22.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.892+0000 7f73d1554640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73cc102d90 0x7f73cc1a0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:22.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.892+0000 7f73d1554640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73cc103740 0x7f73cc1a0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:22.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.892+0000 7f73d1554640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f73cc108290 0x7f73cc19a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:22.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.892+0000 7f73d1554640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f73cc112090 con 0x7f73cc108290 2024-09-11T22:01:22.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.892+0000 7f73d1554640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f73cc111f10 con 0x7f73cc103740 2024-09-11T22:01:22.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.892+0000 7f73d1554640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f73cc112210 con 0x7f73cc102d90 2024-09-11T22:01:22.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.892+0000 7f73cb7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f73cc108290 0x7f73cc19a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:22.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.892+0000 7f73cb7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f73cc108290 0x7f73cc19a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58866/0 (socket says 172.21.15.1:58866) 2024-09-11T22:01:22.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.892+0000 7f73cb7fe640 1 -- 172.21.15.1:0/2999105489 learned_addr learned my addr 172.21.15.1:0/2999105489 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:01:22.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.892+0000 7f73ca7fc640 1 --2- 172.21.15.1:0/2999105489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73cc103740 0x7f73cc1a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:22.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.892+0000 7f73caffd640 1 --2- 172.21.15.1:0/2999105489 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73cc102d90 0x7f73cc1a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:22.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.892+0000 7f73cb7fe640 1 -- 172.21.15.1:0/2999105489 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73cc102d90 msgr2=0x7f73cc1a0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:22.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.892+0000 7f73cb7fe640 1 --2- 172.21.15.1:0/2999105489 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73cc102d90 0x7f73cc1a0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:22.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.892+0000 7f73cb7fe640 1 -- 172.21.15.1:0/2999105489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73cc103740 msgr2=0x7f73cc1a0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:22.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.892+0000 7f73cb7fe640 1 --2- 172.21.15.1:0/2999105489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73cc103740 0x7f73cc1a0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:22.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.892+0000 7f73cb7fe640 1 -- 172.21.15.1:0/2999105489 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73cc19acf0 con 0x7f73cc108290 2024-09-11T22:01:22.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.892+0000 7f73caffd640 1 --2- 172.21.15.1:0/2999105489 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73cc102d90 0x7f73cc1a0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:01:22.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.893+0000 7f73cb7fe640 1 --2- 172.21.15.1:0/2999105489 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f73cc108290 0x7f73cc19a5c0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f73c4005cc0 tx=0x7f73c4032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:01:22.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.893+0000 7f73b3fff640 1 -- 172.21.15.1:0/2999105489 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c4004040 con 0x7f73cc108290 2024-09-11T22:01:22.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.893+0000 7f73b3fff640 1 -- 172.21.15.1:0/2999105489 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73c40046c0 con 0x7f73cc108290 2024-09-11T22:01:22.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.893+0000 7f73d1554640 1 -- 172.21.15.1:0/2999105489 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73cc19af80 con 0x7f73cc108290 2024-09-11T22:01:22.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.893+0000 7f73b3fff640 1 -- 172.21.15.1:0/2999105489 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c4030e00 con 0x7f73cc108290 2024-09-11T22:01:22.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.893+0000 7f73d1554640 1 -- 172.21.15.1:0/2999105489 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73cc1a72d0 con 0x7f73cc108290 2024-09-11T22:01:22.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.896+0000 7f73b3fff640 1 -- 172.21.15.1:0/2999105489 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f73c40305a0 con 0x7f73cc108290 2024-09-11T22:01:22.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.896+0000 7f73d392d740 1 Processor -- start 2024-09-11T22:01:22.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.896+0000 7f73d392d740 1 -- start start 2024-09-11T22:01:22.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.896+0000 7f73d392d740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4808d5060 0x55f4808d5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:22.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.896+0000 7f73d392d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4808d5980 0x55f4808dfd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:22.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.896+0000 7f73d392d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4808e0240 0x55f4808e2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:22.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.896+0000 7f73d392d740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55f48083a620 con 0x55f4808d5060 2024-09-11T22:01:22.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.896+0000 7f73d392d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55f4807e1e50 con 0x55f4808e0240 2024-09-11T22:01:22.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.896+0000 7f73d392d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f4808365e0 con 0x55f4808d5980 2024-09-11T22:01:22.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.896+0000 7f73ca7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4808d5060 0x55f4808d5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:22.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.896+0000 7f73ca7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4808d5060 0x55f4808d5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58882/0 (socket says 172.21.15.1:58882) 2024-09-11T22:01:22.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.896+0000 7f73ca7fc640 1 -- 172.21.15.1:0/1642189095 learned_addr learned my addr 172.21.15.1:0/1642189095 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:01:22.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.897+0000 7f73caffd640 1 --2- 172.21.15.1:0/1642189095 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4808d5980 0x55f4808dfd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:22.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.897+0000 7f73ca7fc640 1 --2- 172.21.15.1:0/1642189095 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4808e0240 0x55f4808e2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:22.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.897+0000 7f73ca7fc640 1 -- 172.21.15.1:0/1642189095 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4808d5980 msgr2=0x55f4808dfd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:22.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.897+0000 7f73ca7fc640 1 --2- 172.21.15.1:0/1642189095 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4808d5980 0x55f4808dfd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:22.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.897+0000 7f73ca7fc640 1 -- 172.21.15.1:0/1642189095 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4808e0240 msgr2=0x55f4808e2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:22.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.897+0000 7f73ca7fc640 1 --2- 172.21.15.1:0/1642189095 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4808e0240 0x55f4808e2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:22.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.897+0000 7f73ca7fc640 1 -- 172.21.15.1:0/1642189095 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4807d2d40 con 0x55f4808d5060 2024-09-11T22:01:22.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.897+0000 7f73ca7fc640 1 --2- 172.21.15.1:0/1642189095 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4808d5060 0x55f4808d5440 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f73c0009f80 tx=0x7f73c0009910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=532e95b8e218def server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:01:22.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.897+0000 7f73b3fff640 1 --2- 172.21.15.1:0/2999105489 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f73b40a2cf0 0x7f73b40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:22.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.897+0000 7f73b17fa640 1 -- 172.21.15.1:0/1642189095 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c0004030 con 0x55f4808d5060 2024-09-11T22:01:22.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.897+0000 7f73caffd640 1 --2- 172.21.15.1:0/2999105489 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f73b40a2cf0 0x7f73b40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:22.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.897+0000 7f73b17fa640 1 -- 172.21.15.1:0/1642189095 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73c0005500 con 0x55f4808d5060 2024-09-11T22:01:22.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.897+0000 7f73b3fff640 1 -- 172.21.15.1:0/2999105489 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f73c402e030 con 0x7f73cc108290 2024-09-11T22:01:22.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.897+0000 7f73b17fa640 1 -- 172.21.15.1:0/1642189095 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c0004c60 con 0x55f4808d5060 2024-09-11T22:01:22.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.898+0000 7f73d392d740 1 -- 172.21.15.1:0/1642189095 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4808d5060 msgr2=0x55f4808d5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:22.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.898+0000 7f73caffd640 1 --2- 172.21.15.1:0/2999105489 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f73b40a2cf0 0x7f73b40a51b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x55f4808d2450 tx=0x7f73bc00b5a0 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:01:22.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.898+0000 7f73d392d740 1 --2- 172.21.15.1:0/1642189095 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4808d5060 0x55f4808d5440 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f73c0009f80 tx=0x7f73c0009910 comp rx=0 tx=0).stop 2024-09-11T22:01:22.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.898+0000 7f73d392d740 1 -- 172.21.15.1:0/1642189095 shutdown_connections 2024-09-11T22:01:22.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.898+0000 7f73d392d740 1 --2- 172.21.15.1:0/1642189095 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4808e0240 0x55f4808e2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:22.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.898+0000 7f73d392d740 1 --2- 172.21.15.1:0/1642189095 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4808d5980 0x55f4808dfd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:22.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.898+0000 7f73d392d740 1 --2- 172.21.15.1:0/1642189095 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4808d5060 0x55f4808d5440 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:22.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.898+0000 7f73d392d740 1 -- 172.21.15.1:0/1642189095 >> 172.21.15.1:0/1642189095 conn(0x55f4808d15f0 msgr2=0x55f4808d19d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:01:22.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.898+0000 7f73d392d740 1 -- 172.21.15.1:0/1642189095 shutdown_connections 2024-09-11T22:01:22.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.898+0000 7f73d392d740 1 -- 172.21.15.1:0/1642189095 wait complete. 2024-09-11T22:01:22.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.898+0000 7f73d392d740 1 -- start start 2024-09-11T22:01:22.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.898+0000 7f73d392d740 1 Processor -- start 2024-09-11T22:01:22.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.899+0000 7f73d392d740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4808d5060 0x55f4808dd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:22.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.899+0000 7f73d392d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4808d5980 0x55f4808dd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:22.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.899+0000 7f73d392d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4808e0240 0x55f4808ddc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:22.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.899+0000 7f73d392d740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55f4808e2e10 con 0x55f4808d5060 2024-09-11T22:01:22.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.899+0000 7f73d392d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55f4807c2f10 con 0x55f4808d5980 2024-09-11T22:01:22.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.899+0000 7f73d392d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f4808e2f90 con 0x55f4808e0240 2024-09-11T22:01:22.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.899+0000 7f73d392d740 10 client.0 _ll_register_callbacks cb 0x55f4807e6db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:01:22.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.899+0000 7f73caffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4808d5060 0x55f4808dd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:22.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.899+0000 7f73caffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4808d5060 0x55f4808dd1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58898/0 (socket says 172.21.15.1:58898) 2024-09-11T22:01:22.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.899+0000 7f73caffd640 1 -- 172.21.15.1:0/4221585771 learned_addr learned my addr 172.21.15.1:0/4221585771 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:01:22.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.899+0000 7f73cb7fe640 1 --2- 172.21.15.1:0/4221585771 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4808e0240 0x55f4808ddc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:22.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.899+0000 7f73ca7fc640 1 --2- 172.21.15.1:0/4221585771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4808d5980 0x55f4808dd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:22.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.899+0000 7f73caffd640 1 -- 172.21.15.1:0/4221585771 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4808e0240 msgr2=0x55f4808ddc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:22.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.899+0000 7f73caffd640 1 --2- 172.21.15.1:0/4221585771 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4808e0240 0x55f4808ddc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:22.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.899+0000 7f73caffd640 1 -- 172.21.15.1:0/4221585771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4808d5980 msgr2=0x55f4808dd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:22.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.899+0000 7f73caffd640 1 --2- 172.21.15.1:0/4221585771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4808d5980 0x55f4808dd710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:22.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.899+0000 7f73caffd640 1 -- 172.21.15.1:0/4221585771 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f480549580 con 0x55f4808d5060 2024-09-11T22:01:22.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.899+0000 7f73caffd640 1 --2- 172.21.15.1:0/4221585771 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4808d5060 0x55f4808dd1d0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f73bc00bfd0 tx=0x7f73bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2330e80b32e15b93 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:01:22.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.899+0000 7f73aeff5640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:01:22.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.899+0000 7f73aeff5640 1 -- 172.21.15.1:0/4221585771 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73bc00fac0 con 0x55f4808d5060 2024-09-11T22:01:22.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.900+0000 7f73aeff5640 1 -- 172.21.15.1:0/4221585771 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f73bc00fc60 con 0x55f4808d5060 2024-09-11T22:01:22.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.900+0000 7f73d392d740 1 -- 172.21.15.1:0/4221585771 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73c0005500 con 0x55f4808d5060 2024-09-11T22:01:22.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.900+0000 7f73aeff5640 1 -- 172.21.15.1:0/4221585771 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f73bc00fe60 con 0x55f4808d5060 2024-09-11T22:01:22.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.900+0000 7f73d392d740 10 client.16221 fetch_fsmap learned FSMap version 30 2024-09-11T22:01:22.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.900+0000 7f73d392d740 1 -- 172.21.15.1:0/4221585771 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f480613960 con 0x55f4808d5060 2024-09-11T22:01:22.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.900+0000 7f73aeff5640 1 -- 172.21.15.1:0/4221585771 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73c0005500 con 0x55f4808d5060 2024-09-11T22:01:22.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.900+0000 7f73aeff5640 1 -- 172.21.15.1:0/4221585771 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f73bc01f630 con 0x55f4808d5060 2024-09-11T22:01:22.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.900+0000 7f73d392d740 10 client.16221 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:01:22.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.900+0000 7f73d392d740 10 client.16221 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-11T22:01:22.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.900+0000 7f73d392d740 20 client.16221 populate_metadata read hostname 'smithi001' 2024-09-11T22:01:22.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.900+0000 7f73d392d740 1 --2- 172.21.15.1:0/4221585771 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55f4808eff00 0x55f4808f2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:22.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.900+0000 7f73d392d740 4 client.16221 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:01:22.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.900+0000 7f73d392d740 1 -- 172.21.15.1:0/4221585771 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f480847b90 con 0x55f4808eff00 2024-09-11T22:01:22.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.900+0000 7f73ca7fc640 1 --2- 172.21.15.1:0/4221585771 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55f4808eff00 0x55f4808f2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:22.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.901+0000 7f73ca7fc640 1 --2- 172.21.15.1:0/4221585771 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55f4808eff00 0x55f4808f2380 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31a343ce64e4fb68 server_cookie=3f27f51033f50c5a in_seq=0 out_seq=0 2024-09-11T22:01:22.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.901+0000 7f73aeff5640 10 client.16221 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:01:22.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.902+0000 7f73aeff5640 1 -- 172.21.15.1:0/4221585771 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f480847b90 con 0x55f4808eff00 2024-09-11T22:01:22.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.902+0000 7f73aeff5640 10 client.16221 handle_command_reply: tid=0 2024-09-11T22:01:22.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.902+0000 7f73d392d740 1 client.16221 shutdown 2024-09-11T22:01:22.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.902+0000 7f73d392d740 1 -- 172.21.15.1:0/4221585771 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4808d5060 msgr2=0x55f4808dd1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:22.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.902+0000 7f73d392d740 1 --2- 172.21.15.1:0/4221585771 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4808d5060 0x55f4808dd1d0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f73bc00bfd0 tx=0x7f73bc0046c0 comp rx=0 tx=0).stop 2024-09-11T22:01:22.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.902+0000 7f73d392d740 1 -- 172.21.15.1:0/4221585771 shutdown_connections 2024-09-11T22:01:22.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.902+0000 7f73d392d740 1 --2- 172.21.15.1:0/4221585771 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55f4808eff00 0x55f4808f2380 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:22.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.902+0000 7f73d392d740 1 --2- 172.21.15.1:0/4221585771 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4808e0240 0x55f4808ddc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:22.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.902+0000 7f73aeff5640 0 client.16221 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:01:22.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.902+0000 7f73d392d740 1 --2- 172.21.15.1:0/4221585771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4808d5980 0x55f4808dd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:22.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.902+0000 7f73d392d740 1 --2- 172.21.15.1:0/4221585771 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4808d5060 0x55f4808dd1d0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:22.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.902+0000 7f73d392d740 1 -- 172.21.15.1:0/4221585771 >> 172.21.15.1:0/4221585771 conn(0x55f4808d15f0 msgr2=0x55f4808d7320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:01:22.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.902+0000 7f73d392d740 1 -- 172.21.15.1:0/4221585771 shutdown_connections 2024-09-11T22:01:22.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.902+0000 7f73d392d740 1 -- 172.21.15.1:0/4221585771 wait complete. 2024-09-11T22:01:22.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.902+0000 7f73d392d740 20 client.16221 trim_cache size 0 max 16384 2024-09-11T22:01:22.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.919+0000 7f73d392d740 1 Processor -- start 2024-09-11T22:01:22.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.919+0000 7f73d392d740 1 -- start start 2024-09-11T22:01:22.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.919+0000 7f73d392d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4808d5060 0x55f4808d7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:22.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.919+0000 7f73d392d740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4808d5980 0x55f4808d0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:22.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.919+0000 7f73d392d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4808e0240 0x55f4808f7c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:22.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.919+0000 7f73d392d740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f73cc1afbb0 con 0x55f4808d5980 2024-09-11T22:01:22.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.919+0000 7f73d392d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f73cc1b1750 con 0x55f4808d5060 2024-09-11T22:01:22.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.919+0000 7f73d392d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f73cc1b0280 con 0x55f4808e0240 2024-09-11T22:01:22.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.919+0000 7f73ca7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4808d5980 0x55f4808d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:22.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.919+0000 7f73ca7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4808d5980 0x55f4808d0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58902/0 (socket says 172.21.15.1:58902) 2024-09-11T22:01:22.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.919+0000 7f73ca7fc640 1 -- 172.21.15.1:0/818589644 learned_addr learned my addr 172.21.15.1:0/818589644 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:01:22.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.919+0000 7f73cb7fe640 1 --2- 172.21.15.1:0/818589644 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4808e0240 0x55f4808f7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:22.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.919+0000 7f73ca7fc640 1 -- 172.21.15.1:0/818589644 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4808e0240 msgr2=0x55f4808f7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:22.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.919+0000 7f73ca7fc640 1 --2- 172.21.15.1:0/818589644 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4808e0240 0x55f4808f7c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:22.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.919+0000 7f73caffd640 1 --2- 172.21.15.1:0/818589644 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4808d5060 0x55f4808d7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:22.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.919+0000 7f73ca7fc640 1 -- 172.21.15.1:0/818589644 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4808d5060 msgr2=0x55f4808d7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:22.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.919+0000 7f73ca7fc640 1 --2- 172.21.15.1:0/818589644 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4808d5060 0x55f4808d7ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:22.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.920+0000 7f73ca7fc640 1 -- 172.21.15.1:0/818589644 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4808d26a0 con 0x55f4808d5980 2024-09-11T22:01:22.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.920+0000 7f73cb7fe640 1 --2- 172.21.15.1:0/818589644 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4808e0240 0x55f4808f7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:01:22.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.920+0000 7f73caffd640 1 --2- 172.21.15.1:0/818589644 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4808d5060 0x55f4808d7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:01:22.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.920+0000 7f73ca7fc640 1 --2- 172.21.15.1:0/818589644 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4808d5980 0x55f4808d0640 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f73c0016010 tx=0x7f73c0009ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3070c6b3b670c58 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:01:22.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.920+0000 7f73b0ff9640 1 -- 172.21.15.1:0/818589644 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c0013190 con 0x55f4808d5980 2024-09-11T22:01:22.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.920+0000 7f73b0ff9640 1 -- 172.21.15.1:0/818589644 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73c00109d0 con 0x55f4808d5980 2024-09-11T22:01:22.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.920+0000 7f73b0ff9640 1 -- 172.21.15.1:0/818589644 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c001f6b0 con 0x55f4808d5980 2024-09-11T22:01:22.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.920+0000 7f73d392d740 1 -- 172.21.15.1:0/818589644 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4808d5980 msgr2=0x55f4808d0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:22.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.920+0000 7f73d392d740 1 --2- 172.21.15.1:0/818589644 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4808d5980 0x55f4808d0640 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f73c0016010 tx=0x7f73c0009ef0 comp rx=0 tx=0).stop 2024-09-11T22:01:22.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.920+0000 7f73d392d740 1 -- 172.21.15.1:0/818589644 shutdown_connections 2024-09-11T22:01:22.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.920+0000 7f73d392d740 1 --2- 172.21.15.1:0/818589644 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4808e0240 0x55f4808f7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:22.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.920+0000 7f73d392d740 1 --2- 172.21.15.1:0/818589644 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4808d5980 0x55f4808d0640 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:22.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.920+0000 7f73d392d740 1 --2- 172.21.15.1:0/818589644 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4808d5060 0x55f4808d7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:22.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.920+0000 7f73d392d740 1 -- 172.21.15.1:0/818589644 >> 172.21.15.1:0/818589644 conn(0x55f4808d15f0 msgr2=0x55f4808cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:01:22.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.921+0000 7f73d392d740 1 -- 172.21.15.1:0/818589644 shutdown_connections 2024-09-11T22:01:22.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.921+0000 7f73d392d740 1 -- 172.21.15.1:0/818589644 wait complete. 2024-09-11T22:01:22.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.921+0000 7f73d392d740 1 -- start start 2024-09-11T22:01:22.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.921+0000 7f73d392d740 1 Processor -- start 2024-09-11T22:01:22.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.921+0000 7f73d392d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4808d5060 0x55f4808e81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:22.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.921+0000 7f73d392d740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4808d5980 0x55f4808e8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:22.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.921+0000 7f73d392d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4808e0240 0x55f4808e51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:22.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.921+0000 7f73d392d740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f73cc1b1050 con 0x55f4808d5980 2024-09-11T22:01:22.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.921+0000 7f73d392d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f73cc1b0950 con 0x55f4808e0240 2024-09-11T22:01:22.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.921+0000 7f73d392d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f73cc1adb90 con 0x55f4808d5060 2024-09-11T22:01:22.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.921+0000 7f73d392d740 10 client.0 _ll_register_callbacks cb 0x7f73bc010460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:01:22.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.921+0000 7f73ca7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4808d5980 0x55f4808e8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:22.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.921+0000 7f73ca7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4808d5980 0x55f4808e8720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58906/0 (socket says 172.21.15.1:58906) 2024-09-11T22:01:22.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.921+0000 7f73ca7fc640 1 -- 172.21.15.1:0/1932068818 learned_addr learned my addr 172.21.15.1:0/1932068818 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:01:22.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.922+0000 7f73caffd640 1 --2- 172.21.15.1:0/1932068818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4808d5060 0x55f4808e81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:22.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.922+0000 7f73cb7fe640 1 --2- 172.21.15.1:0/1932068818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4808e0240 0x55f4808e51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:22.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.922+0000 7f73ca7fc640 1 -- 172.21.15.1:0/1932068818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4808d5060 msgr2=0x55f4808e81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:22.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.922+0000 7f73ca7fc640 1 --2- 172.21.15.1:0/1932068818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4808d5060 0x55f4808e81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:22.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.922+0000 7f73ca7fc640 1 -- 172.21.15.1:0/1932068818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4808e0240 msgr2=0x55f4808e51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:22.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.922+0000 7f73ca7fc640 1 --2- 172.21.15.1:0/1932068818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4808e0240 0x55f4808e51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:22.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.922+0000 7f73ca7fc640 1 -- 172.21.15.1:0/1932068818 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4808d2280 con 0x55f4808d5980 2024-09-11T22:01:22.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.922+0000 7f73cb7fe640 1 --2- 172.21.15.1:0/1932068818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4808e0240 0x55f4808e51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:01:22.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.922+0000 7f73ca7fc640 1 --2- 172.21.15.1:0/1932068818 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4808d5980 0x55f4808e8720 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f73c0004800 tx=0x7f73c00133d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5865b0d116e170af server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:01:22.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.922+0000 7f73aeff5640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:01:22.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.922+0000 7f73aeff5640 1 -- 172.21.15.1:0/1932068818 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c0010b80 con 0x55f4808d5980 2024-09-11T22:01:22.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.922+0000 7f73aeff5640 1 -- 172.21.15.1:0/1932068818 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f73c0010d20 con 0x55f4808d5980 2024-09-11T22:01:22.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.922+0000 7f73d392d740 1 -- 172.21.15.1:0/1932068818 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73c00109d0 con 0x55f4808d5980 2024-09-11T22:01:22.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.922+0000 7f73aeff5640 1 -- 172.21.15.1:0/1932068818 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f73c00104f0 con 0x55f4808d5980 2024-09-11T22:01:22.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.922+0000 7f73d392d740 10 client.16227 fetch_fsmap learned FSMap version 30 2024-09-11T22:01:22.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.922+0000 7f73d392d740 1 -- 172.21.15.1:0/1932068818 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f48095b1a0 con 0x55f4808d5980 2024-09-11T22:01:22.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.922+0000 7f73aeff5640 1 -- 172.21.15.1:0/1932068818 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73c00109d0 con 0x55f4808d5980 2024-09-11T22:01:22.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.923+0000 7f73aeff5640 1 -- 172.21.15.1:0/1932068818 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f73c001d020 con 0x55f4808d5980 2024-09-11T22:01:22.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.923+0000 7f73d392d740 10 client.16227 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:01:22.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.923+0000 7f73d392d740 10 client.16227 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-11T22:01:22.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.923+0000 7f73d392d740 20 client.16227 populate_metadata read hostname 'smithi001' 2024-09-11T22:01:22.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.923+0000 7f73d392d740 1 --2- 172.21.15.1:0/1932068818 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55f48095b660 0x55f48095da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:22.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.923+0000 7f73d392d740 4 client.16227 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "d754d884", "release": true, "await": true}] 2024-09-11T22:01:22.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.923+0000 7f73d392d740 1 -- 172.21.15.1:0/1932068818 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "quiesce db", "set_id": "d754d884", "release": true, "await": true}) -- 0x55f4808420a0 con 0x55f48095b660 2024-09-11T22:01:22.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.923+0000 7f73caffd640 1 --2- 172.21.15.1:0/1932068818 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55f48095b660 0x55f48095da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:22.958 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-11T22:01:22.958 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 30, 2024-09-11T22:01:22.958 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24494, 2024-09-11T22:01:22.958 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 12, 2024-09-11T22:01:22.958 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-09-11T22:01:22.958 INFO:teuthology.orchestra.run.smithi001.stdout: "d754d884": { 2024-09-11T22:01:22.958 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 12, 2024-09-11T22:01:22.958 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-09-11T22:01:22.958 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-11T22:01:22.959 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RELEASED", 2024-09-11T22:01:22.959 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-11T22:01:22.959 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:01:22.959 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 90.0, 2024-09-11T22:01:22.959 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 144.7, 2024-09-11T22:01:22.959 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-09-11T22:01:22.959 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-09-11T22:01:22.959 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-09-11T22:01:22.959 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-11T22:01:22.959 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RELEASED", 2024-09-11T22:01:22.959 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-11T22:01:22.959 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:01:22.959 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:01:22.960 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:01:22.960 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:01:22.960 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:01:22.960 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-11T22:01:22.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.923+0000 7f73caffd640 1 --2- 172.21.15.1:0/1932068818 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55f48095b660 0x55f48095da50 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=954e7c09d22b5d96 server_cookie=52793768de661a8c in_seq=0 out_seq=0 2024-09-11T22:01:22.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.923+0000 7f73aeff5640 10 client.16227 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:01:22.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.929+0000 7f73aeff5640 1 -- 172.21.15.1:0/1932068818 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55f4808420a0 con 0x55f48095b660 2024-09-11T22:01:22.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.929+0000 7f73aeff5640 10 client.16227 handle_command_reply: tid=0 2024-09-11T22:01:22.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.929+0000 7f73d392d740 1 client.16227 shutdown 2024-09-11T22:01:22.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.929+0000 7f73d392d740 1 -- 172.21.15.1:0/1932068818 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4808d5980 msgr2=0x55f4808e8720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:22.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.929+0000 7f73d392d740 1 --2- 172.21.15.1:0/1932068818 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4808d5980 0x55f4808e8720 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f73c0004800 tx=0x7f73c00133d0 comp rx=0 tx=0).stop 2024-09-11T22:01:22.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.929+0000 7f73d392d740 1 -- 172.21.15.1:0/1932068818 shutdown_connections 2024-09-11T22:01:22.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.929+0000 7f73d392d740 1 --2- 172.21.15.1:0/1932068818 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55f48095b660 0x55f48095da50 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:22.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.929+0000 7f73d392d740 1 --2- 172.21.15.1:0/1932068818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4808e0240 0x55f4808e51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:22.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.929+0000 7f73d392d740 1 --2- 172.21.15.1:0/1932068818 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4808d5980 0x55f4808e8720 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:22.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.929+0000 7f73d392d740 1 --2- 172.21.15.1:0/1932068818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4808d5060 0x55f4808e81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:22.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.929+0000 7f73aeff5640 0 client.16227 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:01:22.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.929+0000 7f73d392d740 1 -- 172.21.15.1:0/1932068818 >> 172.21.15.1:0/1932068818 conn(0x55f4808d15f0 msgr2=0x55f4808d0750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:01:22.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.929+0000 7f73d392d740 1 -- 172.21.15.1:0/1932068818 shutdown_connections 2024-09-11T22:01:22.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.929+0000 7f73d392d740 1 -- 172.21.15.1:0/1932068818 wait complete. 2024-09-11T22:01:22.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.929+0000 7f73d392d740 20 client.16227 trim_cache size 0 max 16384 2024-09-11T22:01:22.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.930+0000 7f73d1554640 1 -- 172.21.15.1:0/2999105489 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f73b40a2cf0 msgr2=0x7f73b40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:22.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.930+0000 7f73d1554640 1 --2- 172.21.15.1:0/2999105489 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f73b40a2cf0 0x7f73b40a51b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x55f4808d2450 tx=0x7f73bc00b5a0 comp rx=0 tx=0).stop 2024-09-11T22:01:22.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.930+0000 7f73d1554640 1 -- 172.21.15.1:0/2999105489 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f73cc108290 msgr2=0x7f73cc19a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:22.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.930+0000 7f73d1554640 1 --2- 172.21.15.1:0/2999105489 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f73cc108290 0x7f73cc19a5c0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f73c4005cc0 tx=0x7f73c4032f40 comp rx=0 tx=0).stop 2024-09-11T22:01:22.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.930+0000 7f73d1554640 1 -- 172.21.15.1:0/2999105489 shutdown_connections 2024-09-11T22:01:22.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.930+0000 7f73d1554640 1 --2- 172.21.15.1:0/2999105489 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f73b40a2cf0 0x7f73b40a51b0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:22.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.930+0000 7f73d1554640 1 --2- 172.21.15.1:0/2999105489 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f73cc108290 0x7f73cc19a5c0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:22.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.930+0000 7f73d1554640 1 --2- 172.21.15.1:0/2999105489 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73cc103740 0x7f73cc1a0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:22.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.930+0000 7f73d1554640 1 --2- 172.21.15.1:0/2999105489 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73cc102d90 0x7f73cc1a0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:22.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.930+0000 7f73d1554640 1 -- 172.21.15.1:0/2999105489 >> 172.21.15.1:0/2999105489 conn(0x7f73cc0fc7d0 msgr2=0x7f73cc0fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:01:22.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.930+0000 7f73d1554640 1 -- 172.21.15.1:0/2999105489 shutdown_connections 2024-09-11T22:01:22.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:22.930+0000 7f73d1554640 1 -- 172.21.15.1:0/2999105489 wait complete. 2024-09-11T22:01:23.010 INFO:tasks.workunit.client.0.smithi001.stdout:ok 2024-09-11T22:01:23.015 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-11T22:01:23.015 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-11T22:01:23.060 INFO:tasks.workunit:Running workunit fs/misc/dirfrag.sh... 2024-09-11T22:01:23.061 DEBUG:teuthology.orchestra.run.smithi001:workunit test fs/misc/dirfrag.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=086c737fa75c22d521a3932c59b31c5e1d1c9ef4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/dirfrag.sh 2024-09-11T22:01:23.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:01:22 smithi053 ceph-mon[34908]: pgmap v129: 97 pgs: 97 active+clean; 4.0 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 KiB/s wr, 18 op/s 2024-09-11T22:01:23.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:01:22 smithi161 ceph-mon[35615]: pgmap v129: 97 pgs: 97 active+clean; 4.0 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 KiB/s wr, 18 op/s 2024-09-11T22:01:23.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:01:22 smithi001 ceph-mon[30935]: pgmap v129: 97 pgs: 97 active+clean; 4.0 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 KiB/s wr, 18 op/s 2024-09-11T22:01:24.013 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'd754d884', total seconds elapsed: 53.5 2024-09-11T22:01:25.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:01:24 smithi053 ceph-mon[34908]: pgmap v130: 97 pgs: 97 active+clean; 4.0 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 KiB/s wr, 18 op/s 2024-09-11T22:01:25.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:01:24 smithi161 ceph-mon[35615]: pgmap v130: 97 pgs: 97 active+clean; 4.0 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 KiB/s wr, 18 op/s 2024-09-11T22:01:25.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:01:24 smithi001 ceph-mon[30935]: pgmap v130: 97 pgs: 97 active+clean; 4.0 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 KiB/s wr, 18 op/s 2024-09-11T22:01:25.475 INFO:tasks.workunit.client.0.smithi001.stdout:creating folder hierarchy 2024-09-11T22:01:27.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:01:26 smithi001 ceph-mon[30935]: pgmap v131: 97 pgs: 97 active+clean; 4.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 40 KiB/s wr, 25 op/s 2024-09-11T22:01:27.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:01:26 smithi053 ceph-mon[34908]: pgmap v131: 97 pgs: 97 active+clean; 4.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 40 KiB/s wr, 25 op/s 2024-09-11T22:01:27.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:01:26 smithi161 ceph-mon[35615]: pgmap v131: 97 pgs: 97 active+clean; 4.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 40 KiB/s wr, 25 op/s 2024-09-11T22:01:29.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:01:28 smithi053 ceph-mon[34908]: pgmap v132: 97 pgs: 97 active+clean; 4.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 KiB/s wr, 18 op/s 2024-09-11T22:01:29.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:01:28 smithi161 ceph-mon[35615]: pgmap v132: 97 pgs: 97 active+clean; 4.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 KiB/s wr, 18 op/s 2024-09-11T22:01:29.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:01:28 smithi001 ceph-mon[30935]: pgmap v132: 97 pgs: 97 active+clean; 4.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 KiB/s wr, 18 op/s 2024-09-11T22:01:30.471 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.469+0000 7fcfc99fd640 1 -- 172.21.15.1:0/3893533146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfc4103620 msgr2=0x7fcfc410bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:30.471 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.469+0000 7fcfc99fd640 1 --2- 172.21.15.1:0/3893533146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfc4103620 0x7fcfc410bb00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8009860 tx=0x7fcfb802a3d0 comp rx=0 tx=0).stop 2024-09-11T22:01:30.471 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.469+0000 7fcfc99fd640 1 -- 172.21.15.1:0/3893533146 shutdown_connections 2024-09-11T22:01:30.471 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.469+0000 7fcfc99fd640 1 --2- 172.21.15.1:0/3893533146 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcfc410c0d0 0x7fcfc410e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:30.471 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.469+0000 7fcfc99fd640 1 --2- 172.21.15.1:0/3893533146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfc4103620 0x7fcfc410bb00 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:30.471 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.469+0000 7fcfc99fd640 1 --2- 172.21.15.1:0/3893533146 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfc4102d00 0x7fcfc41030e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:30.472 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.469+0000 7fcfc99fd640 1 -- 172.21.15.1:0/3893533146 >> 172.21.15.1:0/3893533146 conn(0x7fcfc40779a0 msgr2=0x7fcfc40fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:01:30.472 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.469+0000 7fcfc99fd640 1 -- 172.21.15.1:0/3893533146 shutdown_connections 2024-09-11T22:01:30.472 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.469+0000 7fcfc99fd640 1 -- 172.21.15.1:0/3893533146 wait complete. 2024-09-11T22:01:30.472 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.469+0000 7fcfc99fd640 1 Processor -- start 2024-09-11T22:01:30.472 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.469+0000 7fcfc99fd640 1 -- start start 2024-09-11T22:01:30.472 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.470+0000 7fcfc99fd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfc4102d00 0x7fcfc4100950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:30.472 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.470+0000 7fcfc99fd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfc4103620 0x7fcfc4100e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:30.472 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.470+0000 7fcfc99fd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcfc410c0d0 0x7fcfc41013d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:30.472 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.470+0000 7fcfc99fd640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc4111ea0 con 0x7fcfc410c0d0 2024-09-11T22:01:30.472 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.470+0000 7fcfc99fd640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc4111d20 con 0x7fcfc4103620 2024-09-11T22:01:30.472 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.470+0000 7fcfc99fd640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc4112020 con 0x7fcfc4102d00 2024-09-11T22:01:30.472 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.470+0000 7fcfc27fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfc4103620 0x7fcfc4100e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:30.473 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.470+0000 7fcfc37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcfc410c0d0 0x7fcfc41013d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:30.473 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.470+0000 7fcfc27fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfc4103620 0x7fcfc4100e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:40480/0 (socket says 172.21.15.1:40480) 2024-09-11T22:01:30.473 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.470+0000 7fcfc37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcfc410c0d0 0x7fcfc41013d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54300/0 (socket says 172.21.15.1:54300) 2024-09-11T22:01:30.473 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.470+0000 7fcfc27fc640 1 -- 172.21.15.1:0/510993861 learned_addr learned my addr 172.21.15.1:0/510993861 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:01:30.473 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.470+0000 7fcfc2ffd640 1 --2- 172.21.15.1:0/510993861 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfc4102d00 0x7fcfc4100950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:30.473 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.470+0000 7fcfc27fc640 1 -- 172.21.15.1:0/510993861 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfc4102d00 msgr2=0x7fcfc4100950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:30.473 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.470+0000 7fcfc27fc640 1 --2- 172.21.15.1:0/510993861 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfc4102d00 0x7fcfc4100950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:30.473 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.470+0000 7fcfc27fc640 1 -- 172.21.15.1:0/510993861 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcfc410c0d0 msgr2=0x7fcfc41013d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:30.473 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.470+0000 7fcfc27fc640 1 --2- 172.21.15.1:0/510993861 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcfc410c0d0 0x7fcfc41013d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:30.473 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.470+0000 7fcfc27fc640 1 -- 172.21.15.1:0/510993861 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfc41a68f0 con 0x7fcfc4103620 2024-09-11T22:01:30.473 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.470+0000 7fcfc37fe640 1 --2- 172.21.15.1:0/510993861 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcfc410c0d0 0x7fcfc41013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:01:30.473 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.470+0000 7fcfc27fc640 1 --2- 172.21.15.1:0/510993861 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfc4103620 0x7fcfc4100e90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fcfb802a8f0 tx=0x7fcfb8030790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:01:30.481 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.471+0000 7fcfc89fb640 1 -- 172.21.15.1:0/510993861 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb8040020 con 0x7fcfc4103620 2024-09-11T22:01:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.471+0000 7fcfc89fb640 1 -- 172.21.15.1:0/510993861 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfb8004460 con 0x7fcfc4103620 2024-09-11T22:01:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.471+0000 7fcfc99fd640 1 -- 172.21.15.1:0/510993861 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfc41a6b80 con 0x7fcfc4103620 2024-09-11T22:01:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.471+0000 7fcfc89fb640 1 -- 172.21.15.1:0/510993861 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb80046c0 con 0x7fcfc4103620 2024-09-11T22:01:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.471+0000 7fcfc99fd640 1 -- 172.21.15.1:0/510993861 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfc41a7070 con 0x7fcfc4103620 2024-09-11T22:01:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.473+0000 7fcfc89fb640 1 -- 172.21.15.1:0/510993861 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7fcfb8043020 con 0x7fcfc4103620 2024-09-11T22:01:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.473+0000 7fcfcc87c740 1 Processor -- start 2024-09-11T22:01:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.473+0000 7fcfcc87c740 1 -- start start 2024-09-11T22:01:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.474+0000 7fcfcc87c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dfb8574af0 0x55dfb8574ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.474+0000 7fcfcc87c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dfb8575410 0x55dfb857f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:30.483 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.474+0000 7fcfc89fb640 1 --2- 172.21.15.1:0/510993861 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fcfac0a2cf0 0x7fcfac0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:30.483 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.474+0000 7fcfcc87c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dfb857fcb0 0x55dfb85820a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:30.483 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.474+0000 7fcfcc87c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55dfb84d92f0 con 0x55dfb857fcb0 2024-09-11T22:01:30.483 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.474+0000 7fcfcc87c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55dfb8481270 con 0x55dfb8575410 2024-09-11T22:01:30.483 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.474+0000 7fcfcc87c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55dfb84d52b0 con 0x55dfb8574af0 2024-09-11T22:01:30.483 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.474+0000 7fcfc27fc640 1 --2- 172.21.15.1:0/510993861 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fcfac0a2cf0 0x7fcfac0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:30.483 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.474+0000 7fcfc37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dfb857fcb0 0x55dfb85820a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:30.483 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.474+0000 7fcfc2ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dfb8575410 0x55dfb857f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:30.483 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.474+0000 7fcfc2ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dfb8575410 0x55dfb857f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:40494/0 (socket says 172.21.15.1:40494) 2024-09-11T22:01:30.483 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.474+0000 7fcfc2ffd640 1 -- 172.21.15.1:0/1770168577 learned_addr learned my addr 172.21.15.1:0/1770168577 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:01:30.484 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.474+0000 7fcfc89fb640 1 -- 172.21.15.1:0/510993861 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7fcfb80f0430 con 0x7fcfc4103620 2024-09-11T22:01:30.484 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.474+0000 7fcfc37fe640 1 --2- 172.21.15.1:0/1770168577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dfb8574af0 0x55dfb8574ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:30.484 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.474+0000 7fcfc2ffd640 1 -- 172.21.15.1:0/1770168577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dfb8574af0 msgr2=0x55dfb8574ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:30.484 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.474+0000 7fcfc2ffd640 1 --2- 172.21.15.1:0/1770168577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dfb8574af0 0x55dfb8574ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:30.484 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.474+0000 7fcfc2ffd640 1 -- 172.21.15.1:0/1770168577 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dfb857fcb0 msgr2=0x55dfb85820a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:30.484 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.474+0000 7fcfc2ffd640 1 --2- 172.21.15.1:0/1770168577 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dfb857fcb0 0x55dfb85820a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:30.484 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.474+0000 7fcfc2ffd640 1 -- 172.21.15.1:0/1770168577 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfb8480340 con 0x55dfb8575410 2024-09-11T22:01:30.484 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.475+0000 7fcfc27fc640 1 --2- 172.21.15.1:0/510993861 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fcfac0a2cf0 0x7fcfac0a51b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcfb80bc5b0 tx=0x7fcfb8079040 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:01:30.484 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.475+0000 7fcfc2ffd640 1 --2- 172.21.15.1:0/1770168577 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dfb8575410 0x55dfb857f770 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fcfb400b760 tx=0x7fcfb400bc20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a49d346487434898 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:01:30.484 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.475+0000 7fcfa9ffb640 1 -- 172.21.15.1:0/1770168577 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb4003d50 con 0x55dfb8575410 2024-09-11T22:01:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.475+0000 7fcfa9ffb640 1 -- 172.21.15.1:0/1770168577 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfb4015070 con 0x55dfb8575410 2024-09-11T22:01:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.475+0000 7fcfa9ffb640 1 -- 172.21.15.1:0/1770168577 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb4004e00 con 0x55dfb8575410 2024-09-11T22:01:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.475+0000 7fcfcc87c740 1 -- 172.21.15.1:0/1770168577 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dfb8575410 msgr2=0x55dfb857f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.475+0000 7fcfcc87c740 1 --2- 172.21.15.1:0/1770168577 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dfb8575410 0x55dfb857f770 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fcfb400b760 tx=0x7fcfb400bc20 comp rx=0 tx=0).stop 2024-09-11T22:01:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.476+0000 7fcfcc87c740 1 -- 172.21.15.1:0/1770168577 shutdown_connections 2024-09-11T22:01:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.476+0000 7fcfcc87c740 1 --2- 172.21.15.1:0/1770168577 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dfb857fcb0 0x55dfb85820a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.476+0000 7fcfcc87c740 1 --2- 172.21.15.1:0/1770168577 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dfb8575410 0x55dfb857f770 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.476+0000 7fcfcc87c740 1 --2- 172.21.15.1:0/1770168577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dfb8574af0 0x55dfb8574ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.476+0000 7fcfcc87c740 1 -- 172.21.15.1:0/1770168577 >> 172.21.15.1:0/1770168577 conn(0x55dfb8571060 msgr2=0x55dfb8571440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:01:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.476+0000 7fcfcc87c740 1 -- 172.21.15.1:0/1770168577 shutdown_connections 2024-09-11T22:01:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.476+0000 7fcfcc87c740 1 -- 172.21.15.1:0/1770168577 wait complete. 2024-09-11T22:01:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.476+0000 7fcfcc87c740 1 -- start start 2024-09-11T22:01:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.476+0000 7fcfcc87c740 1 Processor -- start 2024-09-11T22:01:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.476+0000 7fcfcc87c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dfb8574af0 0x55dfb857cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.476+0000 7fcfcc87c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dfb8575410 0x55dfb857d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.476+0000 7fcfcc87c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dfb857fcb0 0x55dfb857d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.476+0000 7fcfcc87c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55dfb8582880 con 0x55dfb857fcb0 2024-09-11T22:01:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.476+0000 7fcfcc87c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55dfb8462290 con 0x55dfb8575410 2024-09-11T22:01:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.476+0000 7fcfcc87c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55dfb8582a00 con 0x55dfb8574af0 2024-09-11T22:01:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.477+0000 7fcfcc87c740 10 client.0 _ll_register_callbacks cb 0x55dfb8493140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:01:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.477+0000 7fcfc37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dfb857fcb0 0x55dfb857d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.477+0000 7fcfc2ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dfb8575410 0x55dfb857d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:30.487 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.477+0000 7fcfc37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dfb857fcb0 0x55dfb857d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54310/0 (socket says 172.21.15.1:54310) 2024-09-11T22:01:30.487 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.477+0000 7fcfc37fe640 1 -- 172.21.15.1:0/971516969 learned_addr learned my addr 172.21.15.1:0/971516969 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:01:30.487 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.477+0000 7fcfc37fe640 1 --2- 172.21.15.1:0/971516969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dfb8574af0 0x55dfb857cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:30.487 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.477+0000 7fcfc2ffd640 1 -- 172.21.15.1:0/971516969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dfb8574af0 msgr2=0x55dfb857cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:30.487 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.477+0000 7fcfc2ffd640 1 --2- 172.21.15.1:0/971516969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dfb8574af0 0x55dfb857cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:30.487 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.477+0000 7fcfc2ffd640 1 -- 172.21.15.1:0/971516969 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dfb857fcb0 msgr2=0x55dfb857d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:30.487 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.477+0000 7fcfc2ffd640 1 --2- 172.21.15.1:0/971516969 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dfb857fcb0 0x55dfb857d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:30.487 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.477+0000 7fcfc2ffd640 1 -- 172.21.15.1:0/971516969 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfb81ec210 con 0x55dfb8575410 2024-09-11T22:01:30.487 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.477+0000 7fcfc37fe640 1 --2- 172.21.15.1:0/971516969 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dfb857fcb0 0x55dfb857d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:01:30.487 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.477+0000 7fcfc2ffd640 1 --2- 172.21.15.1:0/971516969 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dfb8575410 0x55dfb857d190 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fcfb40039c0 tx=0x7fcfb4004440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85f022d2193ddba2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:01:30.487 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.477+0000 7fcfa2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-11T22:01:30.487 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.477+0000 7fcfa2ffd640 1 -- 172.21.15.1:0/971516969 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb4028060 con 0x55dfb8575410 2024-09-11T22:01:30.488 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.477+0000 7fcfa2ffd640 1 -- 172.21.15.1:0/971516969 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7fcfb4011440 con 0x55dfb8575410 2024-09-11T22:01:30.488 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.477+0000 7fcfcc87c740 1 -- 172.21.15.1:0/971516969 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfb4015070 con 0x55dfb8575410 2024-09-11T22:01:30.488 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.478+0000 7fcfa2ffd640 1 -- 172.21.15.1:0/971516969 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcfb4011640 con 0x55dfb8575410 2024-09-11T22:01:30.488 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.478+0000 7fcfcc87c740 10 client.25802 fetch_fsmap learned FSMap version 30 2024-09-11T22:01:30.488 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.478+0000 7fcfcc87c740 1 -- 172.21.15.1:0/971516969 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55dfb82b8620 con 0x55dfb8575410 2024-09-11T22:01:30.488 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.478+0000 7fcfa2ffd640 1 -- 172.21.15.1:0/971516969 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfb4015070 con 0x55dfb8575410 2024-09-11T22:01:30.488 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.478+0000 7fcfa2ffd640 1 -- 172.21.15.1:0/971516969 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fcfb4022030 con 0x55dfb8575410 2024-09-11T22:01:30.488 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.478+0000 7fcfcc87c740 10 client.25802 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:01:30.488 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.478+0000 7fcfcc87c740 10 client.25802 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:01:30.488 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.478+0000 7fcfcc87c740 20 client.25802 populate_metadata read hostname 'smithi001' 2024-09-11T22:01:30.488 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.478+0000 7fcfcc87c740 1 --2- 172.21.15.1:0/971516969 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55dfb858fc90 0x55dfb8592080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:30.488 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.478+0000 7fcfcc87c740 4 client.25802 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:01:30.488 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.478+0000 7fcfcc87c740 1 -- 172.21.15.1:0/971516969 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dfb84e6a20 con 0x55dfb858fc90 2024-09-11T22:01:30.489 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.478+0000 7fcfc37fe640 1 --2- 172.21.15.1:0/971516969 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55dfb858fc90 0x55dfb8592080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:30.489 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.479+0000 7fcfc37fe640 1 --2- 172.21.15.1:0/971516969 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55dfb858fc90 0x55dfb8592080 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a06f2fd7c5a1cc8c server_cookie=d5ca1fe8bac4e023 in_seq=0 out_seq=0 2024-09-11T22:01:30.489 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.479+0000 7fcfa2ffd640 10 client.25802 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:01:30.490 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.482+0000 7fcfa2ffd640 1 -- 172.21.15.1:0/971516969 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dfb84e6a20 con 0x55dfb858fc90 2024-09-11T22:01:30.490 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.482+0000 7fcfa2ffd640 10 client.25802 handle_command_reply: tid=0 2024-09-11T22:01:30.490 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.482+0000 7fcfcc87c740 1 client.25802 shutdown 2024-09-11T22:01:30.490 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.482+0000 7fcfcc87c740 1 -- 172.21.15.1:0/971516969 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dfb8575410 msgr2=0x55dfb857d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:30.490 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.482+0000 7fcfcc87c740 1 --2- 172.21.15.1:0/971516969 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dfb8575410 0x55dfb857d190 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fcfb40039c0 tx=0x7fcfb4004440 comp rx=0 tx=0).stop 2024-09-11T22:01:30.490 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.482+0000 7fcfcc87c740 1 -- 172.21.15.1:0/971516969 shutdown_connections 2024-09-11T22:01:30.490 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.482+0000 7fcfcc87c740 1 --2- 172.21.15.1:0/971516969 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55dfb858fc90 0x55dfb8592080 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:30.490 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.482+0000 7fcfcc87c740 1 --2- 172.21.15.1:0/971516969 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dfb857fcb0 0x55dfb857d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:30.491 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.482+0000 7fcfcc87c740 1 --2- 172.21.15.1:0/971516969 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dfb8575410 0x55dfb857d190 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:30.491 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.482+0000 7fcfa2ffd640 0 client.25802 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:01:30.491 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.482+0000 7fcfcc87c740 1 --2- 172.21.15.1:0/971516969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dfb8574af0 0x55dfb857cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:30.491 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.482+0000 7fcfcc87c740 1 -- 172.21.15.1:0/971516969 >> 172.21.15.1:0/971516969 conn(0x55dfb8571060 msgr2=0x55dfb8576d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:01:30.491 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.482+0000 7fcfcc87c740 1 -- 172.21.15.1:0/971516969 shutdown_connections 2024-09-11T22:01:30.491 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.482+0000 7fcfcc87c740 1 -- 172.21.15.1:0/971516969 wait complete. 2024-09-11T22:01:30.491 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.482+0000 7fcfcc87c740 20 client.25802 trim_cache size 0 max 16384 2024-09-11T22:01:30.502 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.497+0000 7fcfcc87c740 1 Processor -- start 2024-09-11T22:01:30.502 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.497+0000 7fcfcc87c740 1 -- start start 2024-09-11T22:01:30.502 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.497+0000 7fcfcc87c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dfb8574af0 0x55dfb8570b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:30.503 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.497+0000 7fcfcc87c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dfb8575410 0x55dfb857d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:30.503 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.497+0000 7fcfcc87c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dfb857fcb0 0x55dfb8597670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:30.503 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.497+0000 7fcfcc87c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc41afb90 con 0x55dfb8574af0 2024-09-11T22:01:30.503 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.497+0000 7fcfcc87c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc41b1730 con 0x55dfb857fcb0 2024-09-11T22:01:30.503 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.497+0000 7fcfcc87c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc41b0260 con 0x55dfb8575410 2024-09-11T22:01:30.503 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.497+0000 7fcfc37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dfb8574af0 0x55dfb8570b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:30.503 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.497+0000 7fcfc37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dfb8574af0 0x55dfb8570b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54326/0 (socket says 172.21.15.1:54326) 2024-09-11T22:01:30.503 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.497+0000 7fcfc37fe640 1 -- 172.21.15.1:0/1815815917 learned_addr learned my addr 172.21.15.1:0/1815815917 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:01:30.503 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.497+0000 7fcfc37fe640 1 --2- 172.21.15.1:0/1815815917 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dfb857fcb0 0x55dfb8597670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:30.503 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.497+0000 7fcfc2ffd640 1 --2- 172.21.15.1:0/1815815917 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dfb8575410 0x55dfb857d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:30.503 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.497+0000 7fcfc37fe640 1 -- 172.21.15.1:0/1815815917 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dfb8575410 msgr2=0x55dfb857d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:30.503 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.497+0000 7fcfc37fe640 1 --2- 172.21.15.1:0/1815815917 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dfb8575410 0x55dfb857d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.497+0000 7fcfc37fe640 1 -- 172.21.15.1:0/1815815917 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dfb857fcb0 msgr2=0x55dfb8597670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.497+0000 7fcfc37fe640 1 --2- 172.21.15.1:0/1815815917 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dfb857fcb0 0x55dfb8597670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.497+0000 7fcfc37fe640 1 -- 172.21.15.1:0/1815815917 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfb857bbe0 con 0x55dfb8574af0 2024-09-11T22:01:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.498+0000 7fcfc37fe640 1 --2- 172.21.15.1:0/1815815917 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dfb8574af0 0x55dfb8570b00 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc009fd0 tx=0x7fcfbc00e8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e38d56891a43512 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:01:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.498+0000 7fcfa97fa640 1 -- 172.21.15.1:0/1815815917 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfbc004b20 con 0x55dfb8574af0 2024-09-11T22:01:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.498+0000 7fcfa97fa640 1 -- 172.21.15.1:0/1815815917 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfbc004cc0 con 0x55dfb8574af0 2024-09-11T22:01:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.498+0000 7fcfa97fa640 1 -- 172.21.15.1:0/1815815917 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfbc021770 con 0x55dfb8574af0 2024-09-11T22:01:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.499+0000 7fcfcc87c740 1 -- 172.21.15.1:0/1815815917 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dfb8574af0 msgr2=0x55dfb8570b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.499+0000 7fcfcc87c740 1 --2- 172.21.15.1:0/1815815917 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dfb8574af0 0x55dfb8570b00 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc009fd0 tx=0x7fcfbc00e8a0 comp rx=0 tx=0).stop 2024-09-11T22:01:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.499+0000 7fcfcc87c740 1 -- 172.21.15.1:0/1815815917 shutdown_connections 2024-09-11T22:01:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.499+0000 7fcfcc87c740 1 --2- 172.21.15.1:0/1815815917 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dfb857fcb0 0x55dfb8597670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.499+0000 7fcfcc87c740 1 --2- 172.21.15.1:0/1815815917 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dfb8575410 0x55dfb857d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.499+0000 7fcfcc87c740 1 --2- 172.21.15.1:0/1815815917 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dfb8574af0 0x55dfb8570b00 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.499+0000 7fcfcc87c740 1 -- 172.21.15.1:0/1815815917 >> 172.21.15.1:0/1815815917 conn(0x55dfb8571060 msgr2=0x55dfb8576d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:01:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.499+0000 7fcfcc87c740 1 -- 172.21.15.1:0/1815815917 shutdown_connections 2024-09-11T22:01:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.499+0000 7fcfcc87c740 1 -- 172.21.15.1:0/1815815917 wait complete. 2024-09-11T22:01:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.499+0000 7fcfcc87c740 1 -- start start 2024-09-11T22:01:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.499+0000 7fcfcc87c740 1 Processor -- start 2024-09-11T22:01:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.499+0000 7fcfcc87c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dfb8574af0 0x55dfb85885e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.499+0000 7fcfcc87c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dfb8575410 0x55dfb8579ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.499+0000 7fcfcc87c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dfb857fcb0 0x55dfb857a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.499+0000 7fcfcc87c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc41b1030 con 0x55dfb8575410 2024-09-11T22:01:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.499+0000 7fcfcc87c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc41b0930 con 0x55dfb8574af0 2024-09-11T22:01:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.499+0000 7fcfcc87c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc41adb70 con 0x55dfb857fcb0 2024-09-11T22:01:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.500+0000 7fcfcc87c740 10 client.0 _ll_register_callbacks cb 0x7fcfbc009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:01:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.500+0000 7fcfc2ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dfb8575410 0x55dfb8579ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.500+0000 7fcfc2ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dfb8575410 0x55dfb8579ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54342/0 (socket says 172.21.15.1:54342) 2024-09-11T22:01:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.500+0000 7fcfc37fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dfb8574af0 0x55dfb85885e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.500+0000 7fcfc2ffd640 1 -- 172.21.15.1:0/42443080 learned_addr learned my addr 172.21.15.1:0/42443080 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:01:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.500+0000 7fcfc37fe640 1 --2- 172.21.15.1:0/42443080 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dfb857fcb0 0x55dfb857a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.500+0000 7fcfc2ffd640 1 -- 172.21.15.1:0/42443080 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dfb857fcb0 msgr2=0x55dfb857a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.500+0000 7fcfc2ffd640 1 --2- 172.21.15.1:0/42443080 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dfb857fcb0 0x55dfb857a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.500+0000 7fcfc2ffd640 1 -- 172.21.15.1:0/42443080 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dfb8574af0 msgr2=0x55dfb85885e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.500+0000 7fcfc2ffd640 1 --2- 172.21.15.1:0/42443080 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dfb8574af0 0x55dfb85885e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.500+0000 7fcfc2ffd640 1 -- 172.21.15.1:0/42443080 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfb85f9640 con 0x55dfb8575410 2024-09-11T22:01:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.500+0000 7fcfc2ffd640 1 --2- 172.21.15.1:0/42443080 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dfb8575410 0x55dfb8579ac0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fcfb4001eb0 tx=0x7fcfb4004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6999b997b069e0db server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:01:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.500+0000 7fcfa2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:01:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.500+0000 7fcfa2ffd640 1 -- 172.21.15.1:0/42443080 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb400be70 con 0x55dfb8575410 2024-09-11T22:01:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.500+0000 7fcfcc87c740 1 -- 172.21.15.1:0/42443080 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfbc004cc0 con 0x55dfb8575410 2024-09-11T22:01:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.500+0000 7fcfa2ffd640 1 -- 172.21.15.1:0/42443080 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7fcfb4019bb0 con 0x55dfb8575410 2024-09-11T22:01:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.500+0000 7fcfa2ffd640 1 -- 172.21.15.1:0/42443080 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcfb40117e0 con 0x55dfb8575410 2024-09-11T22:01:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.501+0000 7fcfcc87c740 10 client.16254 fetch_fsmap learned FSMap version 30 2024-09-11T22:01:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.501+0000 7fcfcc87c740 1 -- 172.21.15.1:0/42443080 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55dfb85fa7f0 con 0x55dfb8575410 2024-09-11T22:01:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.501+0000 7fcfa2ffd640 1 -- 172.21.15.1:0/42443080 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfbc004cc0 con 0x55dfb8575410 2024-09-11T22:01:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.501+0000 7fcfa2ffd640 1 -- 172.21.15.1:0/42443080 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fcfb4032030 con 0x55dfb8575410 2024-09-11T22:01:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.501+0000 7fcfcc87c740 10 client.16254 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:01:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.501+0000 7fcfcc87c740 10 client.16254 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:01:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.501+0000 7fcfcc87c740 20 client.16254 populate_metadata read hostname 'smithi001' 2024-09-11T22:01:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.501+0000 7fcfcc87c740 1 --2- 172.21.15.1:0/42443080 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55dfb85faf10 0x55dfb85fd330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.501+0000 7fcfcc87c740 4 client.16254 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:01:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.501+0000 7fcfcc87c740 1 -- 172.21.15.1:0/42443080 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dfb84e0d70 con 0x55dfb85faf10 2024-09-11T22:01:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.501+0000 7fcfc37fe640 1 --2- 172.21.15.1:0/42443080 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55dfb85faf10 0x55dfb85fd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.502+0000 7fcfc37fe640 1 --2- 172.21.15.1:0/42443080 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55dfb85faf10 0x55dfb85fd330 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fd7158384308f89 server_cookie=d3ebc3ace74b80e0 in_seq=0 out_seq=0 2024-09-11T22:01:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:30.502+0000 7fcfa2ffd640 10 client.16254 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:01:31.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:01:30 smithi053 ceph-mon[34908]: pgmap v133: 97 pgs: 97 active+clean; 11 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 707 KiB/s wr, 36 op/s 2024-09-11T22:01:31.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:01:30 smithi161 ceph-mon[35615]: pgmap v133: 97 pgs: 97 active+clean; 11 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 707 KiB/s wr, 36 op/s 2024-09-11T22:01:31.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:01:30 smithi001 ceph-mon[30935]: pgmap v133: 97 pgs: 97 active+clean; 11 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 707 KiB/s wr, 36 op/s 2024-09-11T22:01:33.270 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:33.268+0000 7fcfa2ffd640 1 -- 172.21.15.1:0/42443080 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dfb84e0d70 con 0x55dfb85faf10 2024-09-11T22:01:33.270 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:33.269+0000 7fcfa2ffd640 10 client.16254 handle_command_reply: tid=0 2024-09-11T22:01:33.270 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:33.269+0000 7fcfcc87c740 1 client.16254 shutdown 2024-09-11T22:01:33.270 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:33.269+0000 7fcfcc87c740 1 -- 172.21.15.1:0/42443080 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dfb8575410 msgr2=0x55dfb8579ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:33.270 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:33.269+0000 7fcfcc87c740 1 --2- 172.21.15.1:0/42443080 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dfb8575410 0x55dfb8579ac0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fcfb4001eb0 tx=0x7fcfb4004600 comp rx=0 tx=0).stop 2024-09-11T22:01:33.270 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:33.269+0000 7fcfcc87c740 1 -- 172.21.15.1:0/42443080 shutdown_connections 2024-09-11T22:01:33.270 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:33.269+0000 7fcfcc87c740 1 --2- 172.21.15.1:0/42443080 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55dfb85faf10 0x55dfb85fd330 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:33.270 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:33.269+0000 7fcfcc87c740 1 --2- 172.21.15.1:0/42443080 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55dfb857fcb0 0x55dfb857a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:33.271 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:33.269+0000 7fcfcc87c740 1 --2- 172.21.15.1:0/42443080 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dfb8575410 0x55dfb8579ac0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:33.271 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:33.269+0000 7fcfa2ffd640 0 client.16254 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:01:33.271 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:33.269+0000 7fcfcc87c740 1 --2- 172.21.15.1:0/42443080 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55dfb8574af0 0x55dfb85885e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:33.271 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:33.269+0000 7fcfcc87c740 1 -- 172.21.15.1:0/42443080 >> 172.21.15.1:0/42443080 conn(0x55dfb8571060 msgr2=0x55dfb856f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:01:33.271 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:33.269+0000 7fcfcc87c740 1 -- 172.21.15.1:0/42443080 shutdown_connections 2024-09-11T22:01:33.271 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:33.269+0000 7fcfcc87c740 1 -- 172.21.15.1:0/42443080 wait complete. 2024-09-11T22:01:33.271 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:33.269+0000 7fcfcc87c740 20 client.16254 trim_cache size 0 max 16384 2024-09-11T22:01:33.271 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-09-11T22:01:33.272 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-09-11T22:01:33.272 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-09-11T22:01:33.272 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-09-11T22:01:33.273 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:33.270+0000 7fcfc99fd640 1 -- 172.21.15.1:0/510993861 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fcfac0a2cf0 msgr2=0x7fcfac0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:33.273 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:33.270+0000 7fcfc99fd640 1 --2- 172.21.15.1:0/510993861 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fcfac0a2cf0 0x7fcfac0a51b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcfb80bc5b0 tx=0x7fcfb8079040 comp rx=0 tx=0).stop 2024-09-11T22:01:33.273 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:33.270+0000 7fcfc99fd640 1 -- 172.21.15.1:0/510993861 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfc4103620 msgr2=0x7fcfc4100e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:33.273 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:33.270+0000 7fcfc99fd640 1 --2- 172.21.15.1:0/510993861 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfc4103620 0x7fcfc4100e90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fcfb802a8f0 tx=0x7fcfb8030790 comp rx=0 tx=0).stop 2024-09-11T22:01:33.273 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:33.270+0000 7fcfc99fd640 1 -- 172.21.15.1:0/510993861 shutdown_connections 2024-09-11T22:01:33.273 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:33.270+0000 7fcfc99fd640 1 --2- 172.21.15.1:0/510993861 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fcfac0a2cf0 0x7fcfac0a51b0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:33.273 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:33.270+0000 7fcfc99fd640 1 --2- 172.21.15.1:0/510993861 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcfc410c0d0 0x7fcfc41013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:33.274 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:33.270+0000 7fcfc99fd640 1 --2- 172.21.15.1:0/510993861 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfc4103620 0x7fcfc4100e90 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:33.274 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:33.270+0000 7fcfc99fd640 1 --2- 172.21.15.1:0/510993861 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfc4102d00 0x7fcfc4100950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:33.274 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:33.270+0000 7fcfc99fd640 1 -- 172.21.15.1:0/510993861 >> 172.21.15.1:0/510993861 conn(0x7fcfc40779a0 msgr2=0x7fcfc40feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:01:33.274 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:33.270+0000 7fcfc99fd640 1 -- 172.21.15.1:0/510993861 shutdown_connections 2024-09-11T22:01:33.274 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:33.270+0000 7fcfc99fd640 1 -- 172.21.15.1:0/510993861 wait complete. 2024-09-11T22:01:33.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:01:32 smithi053 ceph-mon[34908]: pgmap v134: 97 pgs: 97 active+clean; 11 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 694 KiB/s wr, 24 op/s 2024-09-11T22:01:33.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:01:32 smithi161 ceph-mon[35615]: pgmap v134: 97 pgs: 97 active+clean; 11 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 694 KiB/s wr, 24 op/s 2024-09-11T22:01:33.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:01:32 smithi001 ceph-mon[30935]: pgmap v134: 97 pgs: 97 active+clean; 11 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 694 KiB/s wr, 24 op/s 2024-09-11T22:01:35.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:01:34 smithi053 ceph-mon[34908]: pgmap v135: 97 pgs: 97 active+clean; 11 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 694 KiB/s wr, 24 op/s 2024-09-11T22:01:35.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:01:34 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:01:35.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:01:34 smithi161 ceph-mon[35615]: pgmap v135: 97 pgs: 97 active+clean; 11 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 694 KiB/s wr, 24 op/s 2024-09-11T22:01:35.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:01:34 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:01:35.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:01:34 smithi001 ceph-mon[30935]: pgmap v135: 97 pgs: 97 active+clean; 11 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 694 KiB/s wr, 24 op/s 2024-09-11T22:01:35.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:01:34 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:01:36.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:01:35 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:01:36.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:01:35 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:01:36.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:01:35 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T22:01:36.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:01:35 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:01:36.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:01:35 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:01:36.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:01:35 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T22:01:36.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:01:35 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:01:36.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:01:35 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:01:36.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:01:35 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T22:01:37.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:01:36 smithi053 ceph-mon[34908]: pgmap v136: 97 pgs: 97 active+clean; 9.4 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 840 op/s 2024-09-11T22:01:37.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:01:36 smithi161 ceph-mon[35615]: pgmap v136: 97 pgs: 97 active+clean; 9.4 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 840 op/s 2024-09-11T22:01:37.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:01:36 smithi001 ceph-mon[30935]: pgmap v136: 97 pgs: 97 active+clean; 9.4 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 840 op/s 2024-09-11T22:01:39.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:01:38 smithi053 ceph-mon[34908]: pgmap v137: 97 pgs: 97 active+clean; 9.4 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 834 op/s 2024-09-11T22:01:39.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:01:38 smithi161 ceph-mon[35615]: pgmap v137: 97 pgs: 97 active+clean; 9.4 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 834 op/s 2024-09-11T22:01:39.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:01:38 smithi001 ceph-mon[30935]: pgmap v137: 97 pgs: 97 active+clean; 9.4 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 834 op/s 2024-09-11T22:01:41.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:01:40 smithi053 ceph-mon[34908]: pgmap v138: 97 pgs: 97 active+clean; 17 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 874 op/s 2024-09-11T22:01:41.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:01:40 smithi161 ceph-mon[35615]: pgmap v138: 97 pgs: 97 active+clean; 17 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 874 op/s 2024-09-11T22:01:41.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:01:40 smithi001 ceph-mon[30935]: pgmap v138: 97 pgs: 97 active+clean; 17 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 874 op/s 2024-09-11T22:01:42.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:01:41 smithi053 ceph-mon[34908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:01:42.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:01:41 smithi161 ceph-mon[35615]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:01:42.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:01:41 smithi001 ceph-mon[30935]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:01:43.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:01:42 smithi053 ceph-mon[34908]: pgmap v139: 97 pgs: 97 active+clean; 17 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 856 op/s 2024-09-11T22:01:43.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:01:42 smithi161 ceph-mon[35615]: pgmap v139: 97 pgs: 97 active+clean; 17 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 856 op/s 2024-09-11T22:01:43.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:01:42 smithi001 ceph-mon[30935]: pgmap v139: 97 pgs: 97 active+clean; 17 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 856 op/s 2024-09-11T22:01:45.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:01:44 smithi053 ceph-mon[34908]: pgmap v140: 97 pgs: 97 active+clean; 17 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 856 op/s 2024-09-11T22:01:45.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:01:44 smithi161 ceph-mon[35615]: pgmap v140: 97 pgs: 97 active+clean; 17 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 856 op/s 2024-09-11T22:01:45.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:01:44 smithi001 ceph-mon[30935]: pgmap v140: 97 pgs: 97 active+clean; 17 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 856 op/s 2024-09-11T22:01:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.360+0000 7ff82b781640 1 -- 172.21.15.1:0/363531871 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff8241035c0 msgr2=0x7ff82410baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.360+0000 7ff82b781640 1 --2- 172.21.15.1:0/363531871 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff8241035c0 0x7ff82410baa0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7ff820009900 tx=0x7ff82002a480 comp rx=0 tx=0).stop 2024-09-11T22:01:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.360+0000 7ff82b781640 1 -- 172.21.15.1:0/363531871 shutdown_connections 2024-09-11T22:01:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.360+0000 7ff82b781640 1 --2- 172.21.15.1:0/363531871 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff82410c0f0 0x7ff82410e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.360+0000 7ff82b781640 1 --2- 172.21.15.1:0/363531871 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff8241035c0 0x7ff82410baa0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.360+0000 7ff82b781640 1 --2- 172.21.15.1:0/363531871 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff824102ca0 0x7ff824103080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.360+0000 7ff82b781640 1 -- 172.21.15.1:0/363531871 >> 172.21.15.1:0/363531871 conn(0x7ff8240fc6b0 msgr2=0x7ff8240fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:01:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.360+0000 7ff82b781640 1 -- 172.21.15.1:0/363531871 shutdown_connections 2024-09-11T22:01:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.361+0000 7ff82b781640 1 -- 172.21.15.1:0/363531871 wait complete. 2024-09-11T22:01:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.361+0000 7ff82b781640 1 Processor -- start 2024-09-11T22:01:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.361+0000 7ff82b781640 1 -- start start 2024-09-11T22:01:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.361+0000 7ff82b781640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff824102ca0 0x7ff824198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.361+0000 7ff82b781640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff8241035c0 0x7ff824198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.361+0000 7ff82b781640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff82410c0f0 0x7ff82419cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.361+0000 7ff82b781640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff824111ed0 con 0x7ff824102ca0 2024-09-11T22:01:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.361+0000 7ff82b781640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff824111d50 con 0x7ff82410c0f0 2024-09-11T22:01:46.366 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.361+0000 7ff82b781640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff824112050 con 0x7ff8241035c0 2024-09-11T22:01:46.366 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.361+0000 7ff82951d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff824102ca0 0x7ff824198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:46.366 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.362+0000 7ff82951d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff824102ca0 0x7ff824198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38292/0 (socket says 172.21.15.1:38292) 2024-09-11T22:01:46.366 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.362+0000 7ff82951d640 1 -- 172.21.15.1:0/4255209483 learned_addr learned my addr 172.21.15.1:0/4255209483 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:01:46.366 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.362+0000 7ff828d1c640 1 --2- 172.21.15.1:0/4255209483 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff8241035c0 0x7ff824198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:46.366 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.362+0000 7ff829d1e640 1 --2- 172.21.15.1:0/4255209483 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff82410c0f0 0x7ff82419cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:46.366 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.362+0000 7ff82951d640 1 -- 172.21.15.1:0/4255209483 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff8241035c0 msgr2=0x7ff824198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:46.366 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.362+0000 7ff82951d640 1 --2- 172.21.15.1:0/4255209483 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff8241035c0 0x7ff824198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:46.366 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.362+0000 7ff82951d640 1 -- 172.21.15.1:0/4255209483 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff82410c0f0 msgr2=0x7ff82419cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:46.366 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.362+0000 7ff82951d640 1 --2- 172.21.15.1:0/4255209483 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff82410c0f0 0x7ff82419cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:46.366 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.362+0000 7ff82951d640 1 -- 172.21.15.1:0/4255209483 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff82419d410 con 0x7ff824102ca0 2024-09-11T22:01:46.366 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.362+0000 7ff82951d640 1 --2- 172.21.15.1:0/4255209483 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff824102ca0 0x7ff824198420 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7ff81400e9f0 tx=0x7ff81400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:01:46.366 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.362+0000 7ff81a7fc640 1 -- 172.21.15.1:0/4255209483 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81400cd80 con 0x7ff824102ca0 2024-09-11T22:01:46.367 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.362+0000 7ff82b781640 1 -- 172.21.15.1:0/4255209483 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff82419d700 con 0x7ff824102ca0 2024-09-11T22:01:46.367 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.362+0000 7ff81a7fc640 1 -- 172.21.15.1:0/4255209483 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff814005500 con 0x7ff824102ca0 2024-09-11T22:01:46.367 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.362+0000 7ff81a7fc640 1 -- 172.21.15.1:0/4255209483 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff814010690 con 0x7ff824102ca0 2024-09-11T22:01:46.367 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.362+0000 7ff82b781640 1 -- 172.21.15.1:0/4255209483 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8241a4f90 con 0x7ff824102ca0 2024-09-11T22:01:46.373 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.365+0000 7ff81a7fc640 1 -- 172.21.15.1:0/4255209483 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7ff8140046c0 con 0x7ff824102ca0 2024-09-11T22:01:46.373 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.365+0000 7ff82e5f4740 1 Processor -- start 2024-09-11T22:01:46.373 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.365+0000 7ff82e5f4740 1 -- start start 2024-09-11T22:01:46.373 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.366+0000 7ff82e5f4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653266ca9f0 0x5653266cadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:46.373 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.366+0000 7ff82e5f4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5653266cb310 0x5653266d9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:46.373 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.366+0000 7ff82e5f4740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5653266da0c0 0x5653266dc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:46.373 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.366+0000 7ff82e5f4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56532662f2f0 con 0x5653266cb310 2024-09-11T22:01:46.373 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.366+0000 7ff82e5f4740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5653265d7270 con 0x5653266da0c0 2024-09-11T22:01:46.373 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.366+0000 7ff82e5f4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56532662b2b0 con 0x5653266ca9f0 2024-09-11T22:01:46.373 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.366+0000 7ff82951d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5653266cb310 0x5653266d9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:46.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.366+0000 7ff829d1e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653266ca9f0 0x5653266cadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:46.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.366+0000 7ff829d1e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653266ca9f0 0x5653266cadd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:48646/0 (socket says 172.21.15.1:48646) 2024-09-11T22:01:46.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.366+0000 7ff82951d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5653266cb310 0x5653266d9b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38300/0 (socket says 172.21.15.1:38300) 2024-09-11T22:01:46.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.366+0000 7ff829d1e640 1 -- 172.21.15.1:0/2616638680 learned_addr learned my addr 172.21.15.1:0/2616638680 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:01:46.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.366+0000 7ff828d1c640 1 --2- 172.21.15.1:0/2616638680 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5653266da0c0 0x5653266dc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:46.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.366+0000 7ff81a7fc640 1 --2- 172.21.15.1:0/4255209483 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7ff8100a2cf0 0x7ff8100a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:46.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.366+0000 7ff829d1e640 1 -- 172.21.15.1:0/2616638680 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5653266da0c0 msgr2=0x5653266dc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:46.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.366+0000 7ff829d1e640 1 --2- 172.21.15.1:0/2616638680 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5653266da0c0 0x5653266dc4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:46.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.366+0000 7ff829d1e640 1 -- 172.21.15.1:0/2616638680 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5653266cb310 msgr2=0x5653266d9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:46.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.366+0000 7ff829d1e640 1 --2- 172.21.15.1:0/2616638680 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5653266cb310 0x5653266d9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:46.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.366+0000 7ff829d1e640 1 -- 172.21.15.1:0/2616638680 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653265d6340 con 0x5653266ca9f0 2024-09-11T22:01:46.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.366+0000 7ff81a7fc640 1 -- 172.21.15.1:0/4255209483 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7ff8140cf8b0 con 0x7ff824102ca0 2024-09-11T22:01:46.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.366+0000 7ff829d1e640 1 --2- 172.21.15.1:0/4255209483 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7ff8100a2cf0 0x7ff8100a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:46.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.366+0000 7ff829d1e640 1 --2- 172.21.15.1:0/2616638680 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653266ca9f0 0x5653266cadd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff81c011590 tx=0x7ff81c011a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51089885d28a0274 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:01:46.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.367+0000 7ff80f7fe640 1 -- 172.21.15.1:0/2616638680 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81c017050 con 0x5653266ca9f0 2024-09-11T22:01:46.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.367+0000 7ff829d1e640 1 --2- 172.21.15.1:0/4255209483 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7ff8100a2cf0 0x7ff8100a51b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff81c022000 tx=0x7ff81c023000 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:01:46.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.367+0000 7ff80f7fe640 1 -- 172.21.15.1:0/2616638680 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff81c01bb90 con 0x5653266ca9f0 2024-09-11T22:01:46.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.367+0000 7ff80f7fe640 1 -- 172.21.15.1:0/2616638680 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81c01a360 con 0x5653266ca9f0 2024-09-11T22:01:46.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.367+0000 7ff82e5f4740 1 -- 172.21.15.1:0/2616638680 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653266ca9f0 msgr2=0x5653266cadd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:46.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.367+0000 7ff82e5f4740 1 --2- 172.21.15.1:0/2616638680 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653266ca9f0 0x5653266cadd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff81c011590 tx=0x7ff81c011a50 comp rx=0 tx=0).stop 2024-09-11T22:01:46.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.367+0000 7ff82e5f4740 1 -- 172.21.15.1:0/2616638680 shutdown_connections 2024-09-11T22:01:46.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.367+0000 7ff82e5f4740 1 --2- 172.21.15.1:0/2616638680 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5653266da0c0 0x5653266dc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:46.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.367+0000 7ff82e5f4740 1 --2- 172.21.15.1:0/2616638680 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5653266cb310 0x5653266d9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:46.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.367+0000 7ff82e5f4740 1 --2- 172.21.15.1:0/2616638680 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653266ca9f0 0x5653266cadd0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:46.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.367+0000 7ff82e5f4740 1 -- 172.21.15.1:0/2616638680 >> 172.21.15.1:0/2616638680 conn(0x5653266c7060 msgr2=0x5653266c7440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:01:46.376 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.368+0000 7ff82e5f4740 1 -- 172.21.15.1:0/2616638680 shutdown_connections 2024-09-11T22:01:46.376 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.368+0000 7ff82e5f4740 1 -- 172.21.15.1:0/2616638680 wait complete. 2024-09-11T22:01:46.376 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.368+0000 7ff82e5f4740 1 -- start start 2024-09-11T22:01:46.376 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.368+0000 7ff82e5f4740 1 Processor -- start 2024-09-11T22:01:46.376 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.368+0000 7ff82e5f4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653266ca9f0 0x5653266d70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:46.376 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.368+0000 7ff82e5f4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5653266cb310 0x5653266d75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:46.376 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.368+0000 7ff82e5f4740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5653266da0c0 0x5653266d7b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:46.376 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.368+0000 7ff82e5f4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5653266dcd20 con 0x5653266cb310 2024-09-11T22:01:46.376 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.368+0000 7ff82e5f4740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5653265b8290 con 0x5653266da0c0 2024-09-11T22:01:46.376 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.368+0000 7ff82e5f4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5653266dcea0 con 0x5653266ca9f0 2024-09-11T22:01:46.376 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.368+0000 7ff82e5f4740 10 client.0 _ll_register_callbacks cb 0x5653265e9140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:01:46.376 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.368+0000 7ff828d1c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5653266cb310 0x5653266d75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:46.377 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.368+0000 7ff82951d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653266ca9f0 0x5653266d70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:46.377 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.369+0000 7ff828d1c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5653266cb310 0x5653266d75e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38312/0 (socket says 172.21.15.1:38312) 2024-09-11T22:01:46.377 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.369+0000 7ff82951d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653266ca9f0 0x5653266d70a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:48650/0 (socket says 172.21.15.1:48650) 2024-09-11T22:01:46.377 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.369+0000 7ff829d1e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5653266da0c0 0x5653266d7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:46.377 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.369+0000 7ff828d1c640 1 -- 172.21.15.1:0/2868510979 learned_addr learned my addr 172.21.15.1:0/2868510979 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:01:46.377 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.369+0000 7ff828d1c640 1 -- 172.21.15.1:0/2868510979 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653266ca9f0 msgr2=0x5653266d70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:46.377 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.369+0000 7ff828d1c640 1 --2- 172.21.15.1:0/2868510979 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653266ca9f0 0x5653266d70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:46.377 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.369+0000 7ff828d1c640 1 -- 172.21.15.1:0/2868510979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5653266da0c0 msgr2=0x5653266d7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:46.377 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.369+0000 7ff828d1c640 1 --2- 172.21.15.1:0/2868510979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5653266da0c0 0x5653266d7b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:46.377 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.369+0000 7ff828d1c640 1 -- 172.21.15.1:0/2868510979 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565326342210 con 0x5653266cb310 2024-09-11T22:01:46.377 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.369+0000 7ff82951d640 1 --2- 172.21.15.1:0/2868510979 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653266ca9f0 0x5653266d70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:01:46.377 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.369+0000 7ff828d1c640 1 --2- 172.21.15.1:0/2868510979 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5653266cb310 0x5653266d75e0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7ff82002a9a0 tx=0x7ff8200317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ae3e42943a24244 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:01:46.377 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.369+0000 7ff80cff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:01:46.378 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.369+0000 7ff80cff9640 1 -- 172.21.15.1:0/2868510979 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff820035040 con 0x5653266cb310 2024-09-11T22:01:46.378 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.369+0000 7ff80cff9640 1 -- 172.21.15.1:0/2868510979 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7ff820004040 con 0x5653266cb310 2024-09-11T22:01:46.378 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.369+0000 7ff82e5f4740 1 -- 172.21.15.1:0/2868510979 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff81c01bb90 con 0x5653266cb310 2024-09-11T22:01:46.378 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.369+0000 7ff80cff9640 1 -- 172.21.15.1:0/2868510979 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff820004240 con 0x5653266cb310 2024-09-11T22:01:46.378 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.369+0000 7ff82e5f4740 10 client.16266 fetch_fsmap learned FSMap version 32 2024-09-11T22:01:46.378 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.369+0000 7ff82e5f4740 1 -- 172.21.15.1:0/2868510979 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56532640e620 con 0x5653266cb310 2024-09-11T22:01:46.378 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.370+0000 7ff80cff9640 1 -- 172.21.15.1:0/2868510979 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff81c01bb90 con 0x5653266cb310 2024-09-11T22:01:46.378 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.370+0000 7ff80cff9640 1 -- 172.21.15.1:0/2868510979 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 32) ==== 8378+0+0 (secure 0 0 0) 0x7ff82004c030 con 0x5653266cb310 2024-09-11T22:01:46.378 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.370+0000 7ff82e5f4740 10 client.16266 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T22:01:46.378 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.370+0000 7ff82e5f4740 10 client.16266 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:01:46.378 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.370+0000 7ff82e5f4740 20 client.16266 populate_metadata read hostname 'smithi001' 2024-09-11T22:01:46.378 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.370+0000 7ff82e5f4740 1 --2- 172.21.15.1:0/2868510979 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5653266ea1f0 0x5653266ec620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:46.378 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.370+0000 7ff82e5f4740 4 client.16266 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:01:46.379 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.370+0000 7ff82e5f4740 1 -- 172.21.15.1:0/2868510979 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56532663ca20 con 0x5653266ea1f0 2024-09-11T22:01:46.379 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.370+0000 7ff82951d640 1 --2- 172.21.15.1:0/2868510979 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5653266ea1f0 0x5653266ec620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:46.379 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.371+0000 7ff82951d640 1 --2- 172.21.15.1:0/2868510979 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5653266ea1f0 0x5653266ec620 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0bd8c70b5ab8036 server_cookie=94972e69d0760c9a in_seq=0 out_seq=0 2024-09-11T22:01:46.379 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.371+0000 7ff80cff9640 10 client.16266 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:01:46.380 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.376+0000 7ff80cff9640 1 -- 172.21.15.1:0/2868510979 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56532663ca20 con 0x5653266ea1f0 2024-09-11T22:01:46.380 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.376+0000 7ff80cff9640 10 client.16266 handle_command_reply: tid=0 2024-09-11T22:01:46.380 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.376+0000 7ff82e5f4740 1 client.16266 shutdown 2024-09-11T22:01:46.380 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.376+0000 7ff82e5f4740 1 -- 172.21.15.1:0/2868510979 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5653266cb310 msgr2=0x5653266d75e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:46.380 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.376+0000 7ff82e5f4740 1 --2- 172.21.15.1:0/2868510979 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5653266cb310 0x5653266d75e0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7ff82002a9a0 tx=0x7ff8200317e0 comp rx=0 tx=0).stop 2024-09-11T22:01:46.380 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.376+0000 7ff82e5f4740 1 -- 172.21.15.1:0/2868510979 shutdown_connections 2024-09-11T22:01:46.380 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.376+0000 7ff82e5f4740 1 --2- 172.21.15.1:0/2868510979 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5653266ea1f0 0x5653266ec620 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:46.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.376+0000 7ff82e5f4740 1 --2- 172.21.15.1:0/2868510979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5653266da0c0 0x5653266d7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:46.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.376+0000 7ff80cff9640 0 client.16266 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:01:46.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.376+0000 7ff82e5f4740 1 --2- 172.21.15.1:0/2868510979 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5653266cb310 0x5653266d75e0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:46.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.376+0000 7ff82e5f4740 1 --2- 172.21.15.1:0/2868510979 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653266ca9f0 0x5653266d70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:46.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.376+0000 7ff82e5f4740 1 -- 172.21.15.1:0/2868510979 >> 172.21.15.1:0/2868510979 conn(0x5653266c7060 msgr2=0x5653266ccd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:01:46.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.376+0000 7ff82e5f4740 1 -- 172.21.15.1:0/2868510979 shutdown_connections 2024-09-11T22:01:46.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.376+0000 7ff82e5f4740 1 -- 172.21.15.1:0/2868510979 wait complete. 2024-09-11T22:01:46.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.377+0000 7ff82e5f4740 20 client.16266 trim_cache size 0 max 16384 2024-09-11T22:01:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.392+0000 7ff82e5f4740 1 Processor -- start 2024-09-11T22:01:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.392+0000 7ff82e5f4740 1 -- start start 2024-09-11T22:01:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.392+0000 7ff82e5f4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653266ca9f0 0x5653266f1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.392+0000 7ff82e5f4740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5653266cb310 0x5653266f2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.392+0000 7ff82e5f4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5653266da0c0 0x5653266d42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.392+0000 7ff82e5f4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff8241ab720 con 0x5653266da0c0 2024-09-11T22:01:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.392+0000 7ff82e5f4740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff8241ad320 con 0x5653266cb310 2024-09-11T22:01:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.392+0000 7ff82e5f4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff8241abdf0 con 0x5653266ca9f0 2024-09-11T22:01:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.392+0000 7ff829d1e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5653266da0c0 0x5653266d42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.392+0000 7ff829d1e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5653266da0c0 0x5653266d42e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38322/0 (socket says 172.21.15.1:38322) 2024-09-11T22:01:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.392+0000 7ff829d1e640 1 -- 172.21.15.1:0/1227724003 learned_addr learned my addr 172.21.15.1:0/1227724003 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:01:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.392+0000 7ff82951d640 1 --2- 172.21.15.1:0/1227724003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653266ca9f0 0x5653266f1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.392+0000 7ff828d1c640 1 --2- 172.21.15.1:0/1227724003 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5653266cb310 0x5653266f2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.393+0000 7ff829d1e640 1 -- 172.21.15.1:0/1227724003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653266ca9f0 msgr2=0x5653266f1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.393+0000 7ff829d1e640 1 --2- 172.21.15.1:0/1227724003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653266ca9f0 0x5653266f1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.393+0000 7ff829d1e640 1 -- 172.21.15.1:0/1227724003 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5653266cb310 msgr2=0x5653266f2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.393+0000 7ff829d1e640 1 --2- 172.21.15.1:0/1227724003 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5653266cb310 0x5653266f2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.393+0000 7ff829d1e640 1 -- 172.21.15.1:0/1227724003 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653266d6030 con 0x5653266da0c0 2024-09-11T22:01:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.393+0000 7ff829d1e640 1 --2- 172.21.15.1:0/1227724003 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5653266da0c0 0x5653266d42e0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7ff81c011590 tx=0x7ff81c005c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39649e4f94192fa6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:01:46.397 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.393+0000 7ff80effd640 1 -- 172.21.15.1:0/1227724003 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81c00ebe0 con 0x5653266da0c0 2024-09-11T22:01:46.397 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.393+0000 7ff80effd640 1 -- 172.21.15.1:0/1227724003 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff81c00ed80 con 0x5653266da0c0 2024-09-11T22:01:46.397 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.393+0000 7ff80effd640 1 -- 172.21.15.1:0/1227724003 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81c00e6a0 con 0x5653266da0c0 2024-09-11T22:01:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.394+0000 7ff82e5f4740 1 -- 172.21.15.1:0/1227724003 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5653266da0c0 msgr2=0x5653266d42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:46.401 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.394+0000 7ff82e5f4740 1 --2- 172.21.15.1:0/1227724003 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5653266da0c0 0x5653266d42e0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7ff81c011590 tx=0x7ff81c005c40 comp rx=0 tx=0).stop 2024-09-11T22:01:46.401 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.394+0000 7ff82e5f4740 1 -- 172.21.15.1:0/1227724003 shutdown_connections 2024-09-11T22:01:46.401 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.394+0000 7ff82e5f4740 1 --2- 172.21.15.1:0/1227724003 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5653266da0c0 0x5653266d42e0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:46.401 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.394+0000 7ff82e5f4740 1 --2- 172.21.15.1:0/1227724003 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5653266cb310 0x5653266f2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:46.401 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.394+0000 7ff82e5f4740 1 --2- 172.21.15.1:0/1227724003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653266ca9f0 0x5653266f1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:46.401 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.394+0000 7ff82e5f4740 1 -- 172.21.15.1:0/1227724003 >> 172.21.15.1:0/1227724003 conn(0x5653266c7060 msgr2=0x5653266d7880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:01:46.401 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.394+0000 7ff82e5f4740 1 -- 172.21.15.1:0/1227724003 shutdown_connections 2024-09-11T22:01:46.401 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.394+0000 7ff82e5f4740 1 -- 172.21.15.1:0/1227724003 wait complete. 2024-09-11T22:01:46.401 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.394+0000 7ff82e5f4740 1 -- start start 2024-09-11T22:01:46.401 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.395+0000 7ff82e5f4740 1 Processor -- start 2024-09-11T22:01:46.401 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.395+0000 7ff82e5f4740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5653266ca9f0 0x5653266de1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:46.401 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.395+0000 7ff82e5f4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653266cb310 0x5653266de6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.395+0000 7ff82e5f4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5653266da0c0 0x565326753f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.395+0000 7ff82e5f4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff8241acbc0 con 0x5653266da0c0 2024-09-11T22:01:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.395+0000 7ff82e5f4740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff8241ac4c0 con 0x5653266ca9f0 2024-09-11T22:01:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.395+0000 7ff82e5f4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff8241a9700 con 0x5653266cb310 2024-09-11T22:01:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.395+0000 7ff82e5f4740 10 client.0 _ll_register_callbacks cb 0x7ff814005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:01:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.395+0000 7ff829d1e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5653266da0c0 0x565326753f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.395+0000 7ff829d1e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5653266da0c0 0x565326753f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38332/0 (socket says 172.21.15.1:38332) 2024-09-11T22:01:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.395+0000 7ff829d1e640 1 -- 172.21.15.1:0/3262695773 learned_addr learned my addr 172.21.15.1:0/3262695773 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:01:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.395+0000 7ff82951d640 1 --2- 172.21.15.1:0/3262695773 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5653266ca9f0 0x5653266de1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.395+0000 7ff828d1c640 1 --2- 172.21.15.1:0/3262695773 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653266cb310 0x5653266de6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.395+0000 7ff829d1e640 1 -- 172.21.15.1:0/3262695773 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653266cb310 msgr2=0x5653266de6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.395+0000 7ff829d1e640 1 --2- 172.21.15.1:0/3262695773 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653266cb310 0x5653266de6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.395+0000 7ff829d1e640 1 -- 172.21.15.1:0/3262695773 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5653266ca9f0 msgr2=0x5653266de1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.395+0000 7ff829d1e640 1 --2- 172.21.15.1:0/3262695773 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5653266ca9f0 0x5653266de1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.395+0000 7ff829d1e640 1 -- 172.21.15.1:0/3262695773 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653266cb140 con 0x5653266da0c0 2024-09-11T22:01:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.396+0000 7ff829d1e640 1 --2- 172.21.15.1:0/3262695773 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5653266da0c0 0x565326753f00 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7ff81c004e90 tx=0x7ff81c004ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f15104dbb6eb839d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:01:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.396+0000 7ff80cff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:01:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.396+0000 7ff80cff9640 1 -- 172.21.15.1:0/3262695773 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81c03a020 con 0x5653266da0c0 2024-09-11T22:01:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.396+0000 7ff80cff9640 1 -- 172.21.15.1:0/3262695773 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7ff81c00e3b0 con 0x5653266da0c0 2024-09-11T22:01:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.396+0000 7ff82e5f4740 1 -- 172.21.15.1:0/3262695773 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff81c00ed80 con 0x5653266da0c0 2024-09-11T22:01:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.396+0000 7ff80cff9640 1 -- 172.21.15.1:0/3262695773 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff81c017070 con 0x5653266da0c0 2024-09-11T22:01:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.396+0000 7ff82e5f4740 10 client.16272 fetch_fsmap learned FSMap version 32 2024-09-11T22:01:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.396+0000 7ff82e5f4740 1 -- 172.21.15.1:0/3262695773 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x565326755340 con 0x5653266da0c0 2024-09-11T22:01:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.396+0000 7ff80cff9640 1 -- 172.21.15.1:0/3262695773 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff81c00ed80 con 0x5653266da0c0 2024-09-11T22:01:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.396+0000 7ff80cff9640 1 -- 172.21.15.1:0/3262695773 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 32) ==== 8378+0+0 (secure 0 0 0) 0x7ff81c036780 con 0x5653266da0c0 2024-09-11T22:01:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.396+0000 7ff82e5f4740 10 client.16272 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T22:01:46.404 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.397+0000 7ff82e5f4740 10 client.16272 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:01:46.404 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.397+0000 7ff82e5f4740 20 client.16272 populate_metadata read hostname 'smithi001' 2024-09-11T22:01:46.404 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.397+0000 7ff82e5f4740 1 --2- 172.21.15.1:0/3262695773 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x565326755890 0x565326757cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:46.404 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.397+0000 7ff82e5f4740 4 client.16272 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:01:46.404 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.397+0000 7ff82e5f4740 1 -- 172.21.15.1:0/3262695773 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "flush journal"}) -- 0x565326636d70 con 0x565326755890 2024-09-11T22:01:46.404 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.397+0000 7ff82951d640 1 --2- 172.21.15.1:0/3262695773 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x565326755890 0x565326757cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:46.404 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.397+0000 7ff80cff9640 10 client.16272 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:01:46.404 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:46.397+0000 7ff82951d640 1 --2- 172.21.15.1:0/3262695773 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x565326755890 0x565326757cb0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8961b135b3edf26d server_cookie=b7e5086a5b85b81b in_seq=0 out_seq=0 2024-09-11T22:01:47.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:01:47 smithi053 ceph-mon[34908]: pgmap v141: 97 pgs: 97 active+clean; 27 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 876 op/s 2024-09-11T22:01:47.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:01:47 smithi053 ceph-mon[34908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:01:47.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:01:47 smithi161 ceph-mon[35615]: pgmap v141: 97 pgs: 97 active+clean; 27 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 876 op/s 2024-09-11T22:01:47.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:01:47 smithi161 ceph-mon[35615]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:01:47.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:01:46 smithi001 ceph-mon[30935]: pgmap v141: 97 pgs: 97 active+clean; 27 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 876 op/s 2024-09-11T22:01:47.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:01:46 smithi001 ceph-mon[30935]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:01:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:48.406+0000 7ff80cff9640 1 -- 172.21.15.1:0/3262695773 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x565326636d70 con 0x565326755890 2024-09-11T22:01:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:48.406+0000 7ff80cff9640 10 client.16272 handle_command_reply: tid=0 2024-09-11T22:01:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:48.406+0000 7ff82e5f4740 1 client.16272 shutdown 2024-09-11T22:01:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:48.407+0000 7ff82e5f4740 1 -- 172.21.15.1:0/3262695773 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5653266da0c0 msgr2=0x565326753f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:48.407+0000 7ff82e5f4740 1 --2- 172.21.15.1:0/3262695773 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5653266da0c0 0x565326753f00 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7ff81c004e90 tx=0x7ff81c004ec0 comp rx=0 tx=0).stop 2024-09-11T22:01:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:48.407+0000 7ff82e5f4740 1 -- 172.21.15.1:0/3262695773 shutdown_connections 2024-09-11T22:01:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:48.407+0000 7ff82e5f4740 1 --2- 172.21.15.1:0/3262695773 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x565326755890 0x565326757cb0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:48.407+0000 7ff80cff9640 0 client.16272 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:01:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:48.407+0000 7ff82e5f4740 1 --2- 172.21.15.1:0/3262695773 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5653266da0c0 0x565326753f00 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:48.407+0000 7ff82e5f4740 1 --2- 172.21.15.1:0/3262695773 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5653266cb310 0x5653266de6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:48.407+0000 7ff82e5f4740 1 --2- 172.21.15.1:0/3262695773 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5653266ca9f0 0x5653266de1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:48.410 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:48.407+0000 7ff82e5f4740 1 -- 172.21.15.1:0/3262695773 >> 172.21.15.1:0/3262695773 conn(0x5653266c7060 msgr2=0x5653266d8900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:01:48.410 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:48.407+0000 7ff82e5f4740 1 -- 172.21.15.1:0/3262695773 shutdown_connections 2024-09-11T22:01:48.410 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:48.407+0000 7ff82e5f4740 1 -- 172.21.15.1:0/3262695773 wait complete. 2024-09-11T22:01:48.410 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:48.407+0000 7ff82e5f4740 20 client.16272 trim_cache size 0 max 16384 2024-09-11T22:01:48.410 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-09-11T22:01:48.410 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-09-11T22:01:48.410 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-09-11T22:01:48.410 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-09-11T22:01:48.411 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:48.408+0000 7ff82b781640 1 -- 172.21.15.1:0/4255209483 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7ff8100a2cf0 msgr2=0x7ff8100a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:48.411 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:48.408+0000 7ff82b781640 1 --2- 172.21.15.1:0/4255209483 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7ff8100a2cf0 0x7ff8100a51b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff81c022000 tx=0x7ff81c023000 comp rx=0 tx=0).stop 2024-09-11T22:01:48.411 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:48.408+0000 7ff82b781640 1 -- 172.21.15.1:0/4255209483 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff824102ca0 msgr2=0x7ff824198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:48.411 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:48.408+0000 7ff82b781640 1 --2- 172.21.15.1:0/4255209483 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff824102ca0 0x7ff824198420 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7ff81400e9f0 tx=0x7ff81400edf0 comp rx=0 tx=0).stop 2024-09-11T22:01:48.412 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:48.408+0000 7ff82b781640 1 -- 172.21.15.1:0/4255209483 shutdown_connections 2024-09-11T22:01:48.412 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:48.408+0000 7ff82b781640 1 --2- 172.21.15.1:0/4255209483 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7ff8100a2cf0 0x7ff8100a51b0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:48.412 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:48.408+0000 7ff82b781640 1 --2- 172.21.15.1:0/4255209483 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff82410c0f0 0x7ff82419cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:48.412 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:48.408+0000 7ff82b781640 1 --2- 172.21.15.1:0/4255209483 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff8241035c0 0x7ff824198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:48.412 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:48.408+0000 7ff82b781640 1 --2- 172.21.15.1:0/4255209483 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff824102ca0 0x7ff824198420 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:48.412 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:48.408+0000 7ff82b781640 1 -- 172.21.15.1:0/4255209483 >> 172.21.15.1:0/4255209483 conn(0x7ff8240fc6b0 msgr2=0x7ff8240fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:01:48.412 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:48.408+0000 7ff82b781640 1 -- 172.21.15.1:0/4255209483 shutdown_connections 2024-09-11T22:01:48.412 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:01:48.408+0000 7ff82b781640 1 -- 172.21.15.1:0/4255209483 wait complete. 2024-09-11T22:01:49.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:01:49 smithi053 ceph-mon[34908]: pgmap v142: 97 pgs: 97 active+clean; 27 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 59 op/s 2024-09-11T22:01:49.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:01:49 smithi161 ceph-mon[35615]: pgmap v142: 97 pgs: 97 active+clean; 27 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 59 op/s 2024-09-11T22:01:49.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:01:49 smithi001 ceph-mon[30935]: pgmap v142: 97 pgs: 97 active+clean; 27 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 59 op/s 2024-09-11T22:01:51.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:01:51 smithi053 ceph-mon[34908]: pgmap v143: 97 pgs: 97 active+clean; 9.6 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.0 MiB/s wr, 1.51k op/s 2024-09-11T22:01:51.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:01:51 smithi161 ceph-mon[35615]: pgmap v143: 97 pgs: 97 active+clean; 9.6 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.0 MiB/s wr, 1.51k op/s 2024-09-11T22:01:51.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:01:51 smithi001 ceph-mon[30935]: pgmap v143: 97 pgs: 97 active+clean; 9.6 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.0 MiB/s wr, 1.51k op/s 2024-09-11T22:01:52.151 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:01:52.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:01:52 smithi053 ceph-mon[34908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:01:52.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:01:52 smithi161 ceph-mon[35615]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:01:52.438 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:01:52.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:01:52 smithi001 ceph-mon[30935]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:01:52.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.924+0000 7fb3b7440640 1 -- 172.21.15.1:0/457892886 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3b0102dd0 msgr2=0x7fb3b01031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:52.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.924+0000 7fb3b7440640 1 --2- 172.21.15.1:0/457892886 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3b0102dd0 0x7fb3b01031b0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac009930 tx=0x7fb3ac02a420 comp rx=0 tx=0).stop 2024-09-11T22:01:52.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.924+0000 7fb3b7440640 1 -- 172.21.15.1:0/457892886 shutdown_connections 2024-09-11T22:01:52.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.925+0000 7fb3b7440640 1 --2- 172.21.15.1:0/457892886 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb3b0108300 0x7fb3b010ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:52.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.925+0000 7fb3b7440640 1 --2- 172.21.15.1:0/457892886 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3b0103780 0x7fb3b0107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:52.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.925+0000 7fb3b7440640 1 --2- 172.21.15.1:0/457892886 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3b0102dd0 0x7fb3b01031b0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:52.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.925+0000 7fb3b7440640 1 -- 172.21.15.1:0/457892886 >> 172.21.15.1:0/457892886 conn(0x7fb3b00fc7d0 msgr2=0x7fb3b00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:01:52.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.925+0000 7fb3b7440640 1 -- 172.21.15.1:0/457892886 shutdown_connections 2024-09-11T22:01:52.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.925+0000 7fb3b7440640 1 -- 172.21.15.1:0/457892886 wait complete. 2024-09-11T22:01:52.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.925+0000 7fb3b7440640 1 Processor -- start 2024-09-11T22:01:52.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.925+0000 7fb3b7440640 1 -- start start 2024-09-11T22:01:52.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.926+0000 7fb3b7440640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb3b0102dd0 0x7fb3b0100a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:52.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.926+0000 7fb3b7440640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3b0103780 0x7fb3b0100f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:52.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.926+0000 7fb3b7440640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3b0108300 0x7fb3b01014b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:52.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.926+0000 7fb3b7440640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b01120e0 con 0x7fb3b0108300 2024-09-11T22:01:52.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.926+0000 7fb3b7440640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b0111f60 con 0x7fb3b0103780 2024-09-11T22:01:52.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.926+0000 7fb3b7440640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b0112260 con 0x7fb3b0102dd0 2024-09-11T22:01:52.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.926+0000 7fb3b51dc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb3b0102dd0 0x7fb3b0100a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:52.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.926+0000 7fb3b59dd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3b0108300 0x7fb3b01014b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:52.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.926+0000 7fb3b51dc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb3b0102dd0 0x7fb3b0100a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:41270/0 (socket says 172.21.15.1:41270) 2024-09-11T22:01:52.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.926+0000 7fb3b51dc640 1 -- 172.21.15.1:0/1839095810 learned_addr learned my addr 172.21.15.1:0/1839095810 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:01:52.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.926+0000 7fb3b49db640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3b0103780 0x7fb3b0100f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:52.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.926+0000 7fb3b51dc640 1 -- 172.21.15.1:0/1839095810 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3b0103780 msgr2=0x7fb3b0100f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:52.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.926+0000 7fb3b51dc640 1 --2- 172.21.15.1:0/1839095810 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3b0103780 0x7fb3b0100f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:52.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.926+0000 7fb3b51dc640 1 -- 172.21.15.1:0/1839095810 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3b0108300 msgr2=0x7fb3b01014b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:52.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.926+0000 7fb3b51dc640 1 --2- 172.21.15.1:0/1839095810 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3b0108300 0x7fb3b01014b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:52.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.926+0000 7fb3b51dc640 1 -- 172.21.15.1:0/1839095810 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3b01a6a80 con 0x7fb3b0102dd0 2024-09-11T22:01:52.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.926+0000 7fb3b59dd640 1 --2- 172.21.15.1:0/1839095810 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3b0108300 0x7fb3b01014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:01:52.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.926+0000 7fb3b51dc640 1 --2- 172.21.15.1:0/1839095810 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb3b0102dd0 0x7fb3b0100a30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac02a940 tx=0x7fb3ac003b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:01:52.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.926+0000 7fb3a67fc640 1 -- 172.21.15.1:0/1839095810 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac03f070 con 0x7fb3b0102dd0 2024-09-11T22:01:52.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.927+0000 7fb3b7440640 1 -- 172.21.15.1:0/1839095810 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3b01a6d10 con 0x7fb3b0102dd0 2024-09-11T22:01:52.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.927+0000 7fb3a67fc640 1 -- 172.21.15.1:0/1839095810 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3ac031c90 con 0x7fb3b0102dd0 2024-09-11T22:01:52.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.927+0000 7fb3a67fc640 1 -- 172.21.15.1:0/1839095810 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac035030 con 0x7fb3b0102dd0 2024-09-11T22:01:52.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.927+0000 7fb3b7440640 1 -- 172.21.15.1:0/1839095810 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3b01a71a0 con 0x7fb3b0102dd0 2024-09-11T22:01:52.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.929+0000 7fb3a67fc640 1 -- 172.21.15.1:0/1839095810 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7fb3ac042070 con 0x7fb3b0102dd0 2024-09-11T22:01:52.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.930+0000 7fb3b9819740 1 Processor -- start 2024-09-11T22:01:52.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.930+0000 7fb3b9819740 1 -- start start 2024-09-11T22:01:52.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.930+0000 7fb3b9819740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561cbe0904a0 0x561cbe090880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:52.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.930+0000 7fb3b9819740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cbe090dc0 0x561cbe09b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:52.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.930+0000 7fb3b9819740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cbe09b680 0x561cbe09da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:52.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.930+0000 7fb3b9819740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x561cbdff62e0 con 0x561cbe090dc0 2024-09-11T22:01:52.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.930+0000 7fb3b9819740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x561cbdf9dce0 con 0x561cbe0904a0 2024-09-11T22:01:52.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.930+0000 7fb3b9819740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x561cbdff22a0 con 0x561cbe09b680 2024-09-11T22:01:52.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.930+0000 7fb3b59dd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561cbe0904a0 0x561cbe090880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:52.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.930+0000 7fb3b59dd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561cbe0904a0 0x561cbe090880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:49752/0 (socket says 172.21.15.1:49752) 2024-09-11T22:01:52.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.930+0000 7fb3b59dd640 1 -- 172.21.15.1:0/3942146630 learned_addr learned my addr 172.21.15.1:0/3942146630 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:01:52.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.930+0000 7fb3b51dc640 1 --2- 172.21.15.1:0/3942146630 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cbe090dc0 0x561cbe09b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:52.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.930+0000 7fb3b49db640 1 --2- 172.21.15.1:0/3942146630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cbe09b680 0x561cbe09da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:52.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.931+0000 7fb3a67fc640 1 --2- 172.21.15.1:0/1839095810 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fb39c0a2cf0 0x7fb39c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:52.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.931+0000 7fb3b59dd640 1 -- 172.21.15.1:0/3942146630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cbe09b680 msgr2=0x561cbe09da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:52.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.931+0000 7fb3b59dd640 1 --2- 172.21.15.1:0/3942146630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cbe09b680 0x561cbe09da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:52.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.931+0000 7fb3b59dd640 1 -- 172.21.15.1:0/3942146630 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cbe090dc0 msgr2=0x561cbe09b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:52.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.931+0000 7fb3b59dd640 1 --2- 172.21.15.1:0/3942146630 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cbe090dc0 0x561cbe09b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:52.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.931+0000 7fb3b59dd640 1 -- 172.21.15.1:0/3942146630 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cbdf82440 con 0x561cbe0904a0 2024-09-11T22:01:52.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.931+0000 7fb3a67fc640 1 -- 172.21.15.1:0/1839095810 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7fb3ac0efaf0 con 0x7fb3b0102dd0 2024-09-11T22:01:52.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.931+0000 7fb3b59dd640 1 --2- 172.21.15.1:0/1839095810 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fb39c0a2cf0 0x7fb39c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:52.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.931+0000 7fb3b59dd640 1 --2- 172.21.15.1:0/3942146630 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561cbe0904a0 0x561cbe090880 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8011790 tx=0x7fb3a8011c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af7e3073cd481ebc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:01:52.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.931+0000 7fb39b7fe640 1 -- 172.21.15.1:0/3942146630 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a8004050 con 0x561cbe0904a0 2024-09-11T22:01:52.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.931+0000 7fb39b7fe640 1 -- 172.21.15.1:0/3942146630 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3a80046d0 con 0x561cbe0904a0 2024-09-11T22:01:52.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.931+0000 7fb39b7fe640 1 -- 172.21.15.1:0/3942146630 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a801cac0 con 0x561cbe0904a0 2024-09-11T22:01:52.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.932+0000 7fb3b9819740 1 -- 172.21.15.1:0/3942146630 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561cbe0904a0 msgr2=0x561cbe090880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:52.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.932+0000 7fb3b59dd640 1 --2- 172.21.15.1:0/1839095810 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fb39c0a2cf0 0x7fb39c0a51b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8020000 tx=0x7fb3a8021000 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:01:52.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.932+0000 7fb3b9819740 1 --2- 172.21.15.1:0/3942146630 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561cbe0904a0 0x561cbe090880 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8011790 tx=0x7fb3a8011c50 comp rx=0 tx=0).stop 2024-09-11T22:01:52.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.932+0000 7fb3b9819740 1 -- 172.21.15.1:0/3942146630 shutdown_connections 2024-09-11T22:01:52.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.932+0000 7fb3b9819740 1 --2- 172.21.15.1:0/3942146630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cbe09b680 0x561cbe09da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:52.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.932+0000 7fb3b9819740 1 --2- 172.21.15.1:0/3942146630 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cbe090dc0 0x561cbe09b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:52.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.932+0000 7fb3b9819740 1 --2- 172.21.15.1:0/3942146630 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561cbe0904a0 0x561cbe090880 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:52.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.932+0000 7fb3b9819740 1 -- 172.21.15.1:0/3942146630 >> 172.21.15.1:0/3942146630 conn(0x561cbe08ca30 msgr2=0x561cbe08ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:01:52.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.932+0000 7fb3b9819740 1 -- 172.21.15.1:0/3942146630 shutdown_connections 2024-09-11T22:01:52.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.932+0000 7fb3b9819740 1 -- 172.21.15.1:0/3942146630 wait complete. 2024-09-11T22:01:52.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.933+0000 7fb3b9819740 1 -- start start 2024-09-11T22:01:52.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.933+0000 7fb3b9819740 1 Processor -- start 2024-09-11T22:01:52.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.933+0000 7fb3b9819740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cbe0904a0 0x561cbe09af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:52.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.933+0000 7fb3b9819740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cbe090dc0 0x561cbe098220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:52.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.933+0000 7fb3b9819740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561cbe09b680 0x561cbe098760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:52.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.933+0000 7fb3b9819740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x561cbe09e250 con 0x561cbe0904a0 2024-09-11T22:01:52.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.933+0000 7fb3b9819740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x561cbdfb1050 con 0x561cbe09b680 2024-09-11T22:01:52.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.933+0000 7fb3b9819740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x561cbe09e3d0 con 0x561cbe090dc0 2024-09-11T22:01:52.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.933+0000 7fb3b9819740 10 client.0 _ll_register_callbacks cb 0x561cbdff1160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:01:52.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.933+0000 7fb3b51dc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cbe0904a0 0x561cbe09af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:52.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.933+0000 7fb3b49db640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cbe090dc0 0x561cbe098220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:52.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.933+0000 7fb3b51dc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cbe0904a0 0x561cbe09af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37826/0 (socket says 172.21.15.1:37826) 2024-09-11T22:01:52.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.934+0000 7fb3b51dc640 1 -- 172.21.15.1:0/455217545 learned_addr learned my addr 172.21.15.1:0/455217545 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:01:52.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.934+0000 7fb3b59dd640 1 --2- 172.21.15.1:0/455217545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561cbe09b680 0x561cbe098760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:52.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.934+0000 7fb3b51dc640 1 -- 172.21.15.1:0/455217545 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cbe090dc0 msgr2=0x561cbe098220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:52.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.934+0000 7fb3b51dc640 1 --2- 172.21.15.1:0/455217545 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cbe090dc0 0x561cbe098220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:52.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.934+0000 7fb3b51dc640 1 -- 172.21.15.1:0/455217545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561cbe09b680 msgr2=0x561cbe098760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:52.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.934+0000 7fb3b51dc640 1 --2- 172.21.15.1:0/455217545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561cbe09b680 0x561cbe098760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:52.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.934+0000 7fb3b51dc640 1 -- 172.21.15.1:0/455217545 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cbdd673c0 con 0x561cbe0904a0 2024-09-11T22:01:52.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.934+0000 7fb3b49db640 1 --2- 172.21.15.1:0/455217545 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cbe090dc0 0x561cbe098220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:01:52.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.934+0000 7fb3b51dc640 1 --2- 172.21.15.1:0/455217545 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cbe0904a0 0x561cbe09af70 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac000c00 tx=0x7fb3ac03b720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6251b4663b26207 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:01:52.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.934+0000 7fb398ff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:01:52.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.934+0000 7fb398ff9640 1 -- 172.21.15.1:0/455217545 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac049030 con 0x561cbe0904a0 2024-09-11T22:01:52.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.934+0000 7fb3b9819740 1 -- 172.21.15.1:0/455217545 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3a80046d0 con 0x561cbe0904a0 2024-09-11T22:01:52.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.934+0000 7fb398ff9640 1 -- 172.21.15.1:0/455217545 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7fb3ac03b460 con 0x561cbe0904a0 2024-09-11T22:01:52.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.934+0000 7fb398ff9640 1 -- 172.21.15.1:0/455217545 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb3ac0bb980 con 0x561cbe0904a0 2024-09-11T22:01:52.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.934+0000 7fb3b9819740 10 client.16284 fetch_fsmap learned FSMap version 33 2024-09-11T22:01:52.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.934+0000 7fb3b9819740 1 -- 172.21.15.1:0/455217545 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x561cbe0a9bc0 con 0x561cbe0904a0 2024-09-11T22:01:52.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.935+0000 7fb398ff9640 1 -- 172.21.15.1:0/455217545 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3a80046d0 con 0x561cbe0904a0 2024-09-11T22:01:52.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.935+0000 7fb398ff9640 1 -- 172.21.15.1:0/455217545 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7fb3ac0b8690 con 0x561cbe0904a0 2024-09-11T22:01:52.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.935+0000 7fb3b9819740 10 client.16284 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T22:01:52.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.935+0000 7fb3b9819740 10 client.16284 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:01:52.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.935+0000 7fb3b9819740 20 client.16284 populate_metadata read hostname 'smithi001' 2024-09-11T22:01:52.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.935+0000 7fb3b9819740 1 --2- 172.21.15.1:0/455217545 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x561cbe0ab000 0x561cbe0ad460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:52.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.935+0000 7fb3b9819740 4 client.16284 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:01:52.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.935+0000 7fb3b9819740 1 -- 172.21.15.1:0/455217545 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561cbe003850 con 0x561cbe0ab000 2024-09-11T22:01:52.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.935+0000 7fb3b49db640 1 --2- 172.21.15.1:0/455217545 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x561cbe0ab000 0x561cbe0ad460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:52.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.936+0000 7fb398ff9640 10 client.16284 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:01:52.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.936+0000 7fb3b49db640 1 --2- 172.21.15.1:0/455217545 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x561cbe0ab000 0x561cbe0ad460 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39e5e9d7631f9d7b server_cookie=44bf8bc1e3e3b78c in_seq=0 out_seq=0 2024-09-11T22:01:52.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.937+0000 7fb398ff9640 1 -- 172.21.15.1:0/455217545 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561cbe003850 con 0x561cbe0ab000 2024-09-11T22:01:52.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.937+0000 7fb398ff9640 10 client.16284 handle_command_reply: tid=0 2024-09-11T22:01:52.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.937+0000 7fb3b9819740 1 client.16284 shutdown 2024-09-11T22:01:52.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.937+0000 7fb3b9819740 1 -- 172.21.15.1:0/455217545 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cbe0904a0 msgr2=0x561cbe09af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:52.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.937+0000 7fb3b9819740 1 --2- 172.21.15.1:0/455217545 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cbe0904a0 0x561cbe09af70 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac000c00 tx=0x7fb3ac03b720 comp rx=0 tx=0).stop 2024-09-11T22:01:52.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.937+0000 7fb3b9819740 1 -- 172.21.15.1:0/455217545 shutdown_connections 2024-09-11T22:01:52.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.937+0000 7fb3b9819740 1 --2- 172.21.15.1:0/455217545 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x561cbe0ab000 0x561cbe0ad460 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:52.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.937+0000 7fb3b9819740 1 --2- 172.21.15.1:0/455217545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561cbe09b680 0x561cbe098760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:52.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.937+0000 7fb3b9819740 1 --2- 172.21.15.1:0/455217545 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cbe090dc0 0x561cbe098220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:52.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.937+0000 7fb3b9819740 1 --2- 172.21.15.1:0/455217545 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cbe0904a0 0x561cbe09af70 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:52.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.937+0000 7fb3b9819740 1 -- 172.21.15.1:0/455217545 >> 172.21.15.1:0/455217545 conn(0x561cbe08ca30 msgr2=0x561cbe08bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:01:52.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.937+0000 7fb398ff9640 0 client.16284 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:01:52.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.937+0000 7fb3b9819740 1 -- 172.21.15.1:0/455217545 shutdown_connections 2024-09-11T22:01:52.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.937+0000 7fb3b9819740 1 -- 172.21.15.1:0/455217545 wait complete. 2024-09-11T22:01:52.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.937+0000 7fb3b9819740 20 client.16284 trim_cache size 0 max 16384 2024-09-11T22:01:52.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.955+0000 7fb3b9819740 1 Processor -- start 2024-09-11T22:01:52.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.955+0000 7fb3b9819740 1 -- start start 2024-09-11T22:01:52.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.955+0000 7fb3b9819740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cbe08ca30 0x561cbe08ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:52.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.955+0000 7fb3b9819740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cbe0904a0 0x561cbe0b2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:52.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.955+0000 7fb3b9819740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561cbe090dc0 0x561cbe0b32b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:52.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.955+0000 7fb3b9819740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b01afbe0 con 0x561cbe08ca30 2024-09-11T22:01:52.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.955+0000 7fb3b51dc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cbe08ca30 0x561cbe08ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:52.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.955+0000 7fb3b51dc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cbe08ca30 0x561cbe08ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37842/0 (socket says 172.21.15.1:37842) 2024-09-11T22:01:52.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.955+0000 7fb3b51dc640 1 -- 172.21.15.1:0/4205007363 learned_addr learned my addr 172.21.15.1:0/4205007363 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:01:52.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.955+0000 7fb3b49db640 1 --2- 172.21.15.1:0/4205007363 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cbe0904a0 0x561cbe0b2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:52.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.955+0000 7fb3b9819740 1 -- 172.21.15.1:0/4205007363 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b01b1810 con 0x561cbe090dc0 2024-09-11T22:01:52.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.955+0000 7fb3b9819740 1 -- 172.21.15.1:0/4205007363 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b01b02b0 con 0x561cbe0904a0 2024-09-11T22:01:52.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.955+0000 7fb3b59dd640 1 --2- 172.21.15.1:0/4205007363 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561cbe090dc0 0x561cbe0b32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:52.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.956+0000 7fb3b49db640 1 -- 172.21.15.1:0/4205007363 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561cbe090dc0 msgr2=0x561cbe0b32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:52.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.956+0000 7fb3b49db640 1 --2- 172.21.15.1:0/4205007363 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561cbe090dc0 0x561cbe0b32b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:52.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.956+0000 7fb3b49db640 1 -- 172.21.15.1:0/4205007363 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cbe08ca30 msgr2=0x561cbe08ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:52.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.956+0000 7fb3b49db640 1 --2- 172.21.15.1:0/4205007363 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cbe08ca30 0x561cbe08ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:52.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.956+0000 7fb3b49db640 1 -- 172.21.15.1:0/4205007363 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cbe08c4d0 con 0x561cbe0904a0 2024-09-11T22:01:52.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.956+0000 7fb3b59dd640 1 --2- 172.21.15.1:0/4205007363 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561cbe090dc0 0x561cbe0b32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:01:52.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.956+0000 7fb3b49db640 1 --2- 172.21.15.1:0/4205007363 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cbe0904a0 0x561cbe0b2d70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x561cbe097ab0 tx=0x7fb3a0005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba817302a748945e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:01:52.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.956+0000 7fb39affd640 1 -- 172.21.15.1:0/4205007363 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a00122d0 con 0x561cbe0904a0 2024-09-11T22:01:52.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.956+0000 7fb39affd640 1 -- 172.21.15.1:0/4205007363 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3a0005020 con 0x561cbe0904a0 2024-09-11T22:01:52.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.956+0000 7fb39affd640 1 -- 172.21.15.1:0/4205007363 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a000fad0 con 0x561cbe0904a0 2024-09-11T22:01:52.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.957+0000 7fb3b9819740 1 -- 172.21.15.1:0/4205007363 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cbe0904a0 msgr2=0x561cbe0b2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:52.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.957+0000 7fb3b9819740 1 --2- 172.21.15.1:0/4205007363 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cbe0904a0 0x561cbe0b2d70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x561cbe097ab0 tx=0x7fb3a0005f90 comp rx=0 tx=0).stop 2024-09-11T22:01:52.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.957+0000 7fb3b9819740 1 -- 172.21.15.1:0/4205007363 shutdown_connections 2024-09-11T22:01:52.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.957+0000 7fb3b9819740 1 --2- 172.21.15.1:0/4205007363 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561cbe090dc0 0x561cbe0b32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:52.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.957+0000 7fb3b9819740 1 --2- 172.21.15.1:0/4205007363 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cbe0904a0 0x561cbe0b2d70 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:52.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.957+0000 7fb3b9819740 1 --2- 172.21.15.1:0/4205007363 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cbe08ca30 0x561cbe08ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:52.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.957+0000 7fb3b9819740 1 -- 172.21.15.1:0/4205007363 >> 172.21.15.1:0/4205007363 conn(0x561cbe0988c0 msgr2=0x561cbe08b370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:01:52.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.957+0000 7fb3b9819740 1 -- 172.21.15.1:0/4205007363 shutdown_connections 2024-09-11T22:01:52.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.957+0000 7fb3b9819740 1 -- 172.21.15.1:0/4205007363 wait complete. 2024-09-11T22:01:52.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.957+0000 7fb3b9819740 1 -- start start 2024-09-11T22:01:52.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.958+0000 7fb3b9819740 1 Processor -- start 2024-09-11T22:01:52.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.958+0000 7fb3b9819740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cbe08ca30 0x561cbe10d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:52.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.958+0000 7fb3b9819740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cbe0904a0 0x561cbe10d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:52.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.958+0000 7fb3b9819740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561cbe090dc0 0x561cbe0a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:52.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.958+0000 7fb3b9819740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b01b1080 con 0x561cbe0904a0 2024-09-11T22:01:52.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.958+0000 7fb3b9819740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b01b0980 con 0x561cbe090dc0 2024-09-11T22:01:52.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.958+0000 7fb3b9819740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b01adbc0 con 0x561cbe08ca30 2024-09-11T22:01:52.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.958+0000 7fb3b9819740 10 client.0 _ll_register_callbacks cb 0x7fb3ac0f3290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:01:52.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.958+0000 7fb3b49db640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cbe0904a0 0x561cbe10d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:52.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.958+0000 7fb3b51dc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cbe08ca30 0x561cbe10d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:52.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.958+0000 7fb3b59dd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561cbe090dc0 0x561cbe0a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:52.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.958+0000 7fb3b49db640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cbe0904a0 0x561cbe10d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37858/0 (socket says 172.21.15.1:37858) 2024-09-11T22:01:52.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.958+0000 7fb3b49db640 1 -- 172.21.15.1:0/611685336 learned_addr learned my addr 172.21.15.1:0/611685336 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:01:52.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.959+0000 7fb3b51dc640 1 -- 172.21.15.1:0/611685336 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561cbe090dc0 msgr2=0x561cbe0a0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:52.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.959+0000 7fb3b51dc640 1 --2- 172.21.15.1:0/611685336 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561cbe090dc0 0x561cbe0a0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:52.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.959+0000 7fb3b51dc640 1 -- 172.21.15.1:0/611685336 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cbe0904a0 msgr2=0x561cbe10d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:52.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.959+0000 7fb3b51dc640 1 --2- 172.21.15.1:0/611685336 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cbe0904a0 0x561cbe10d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:52.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.959+0000 7fb3b51dc640 1 -- 172.21.15.1:0/611685336 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cbe0a1460 con 0x561cbe08ca30 2024-09-11T22:01:52.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.959+0000 7fb3b49db640 1 --2- 172.21.15.1:0/611685336 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cbe0904a0 0x561cbe10d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:01:52.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.959+0000 7fb3b51dc640 1 --2- 172.21.15.1:0/611685336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cbe08ca30 0x561cbe10d0c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac0b8480 tx=0x7fb3ac003f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c5a0aee25a14e4e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:01:52.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.959+0000 7fb398ff9640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-11T22:01:52.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.959+0000 7fb398ff9640 1 -- 172.21.15.1:0/611685336 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac0d6020 con 0x561cbe08ca30 2024-09-11T22:01:52.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.959+0000 7fb398ff9640 1 -- 172.21.15.1:0/611685336 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7fb3ac0f6070 con 0x561cbe08ca30 2024-09-11T22:01:52.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.959+0000 7fb3b9819740 1 -- 172.21.15.1:0/611685336 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3a0005020 con 0x561cbe08ca30 2024-09-11T22:01:52.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.959+0000 7fb398ff9640 1 -- 172.21.15.1:0/611685336 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb3ac049070 con 0x561cbe08ca30 2024-09-11T22:01:52.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.959+0000 7fb3b9819740 10 client.25813 fetch_fsmap learned FSMap version 33 2024-09-11T22:01:52.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.959+0000 7fb3b9819740 1 -- 172.21.15.1:0/611685336 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x561cbe115890 con 0x561cbe08ca30 2024-09-11T22:01:52.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.960+0000 7fb398ff9640 1 -- 172.21.15.1:0/611685336 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3ac0bc020 con 0x561cbe08ca30 2024-09-11T22:01:52.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.960+0000 7fb398ff9640 1 -- 172.21.15.1:0/611685336 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7fb3ac0bc280 con 0x561cbe08ca30 2024-09-11T22:01:52.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.960+0000 7fb3b9819740 10 client.25813 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T22:01:52.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.960+0000 7fb3b9819740 10 client.25813 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:01:52.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.960+0000 7fb3b9819740 20 client.25813 populate_metadata read hostname 'smithi001' 2024-09-11T22:01:52.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.960+0000 7fb3b9819740 1 --2- 172.21.15.1:0/611685336 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x561cbe09b680 0x561cbe117f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:52.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.960+0000 7fb3b9819740 4 client.25813 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:01:52.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.960+0000 7fb3b9819740 1 -- 172.21.15.1:0/611685336 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fb3a0005020 con 0x561cbe09b680 2024-09-11T22:01:52.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.960+0000 7fb3b49db640 1 --2- 172.21.15.1:0/611685336 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x561cbe09b680 0x561cbe117f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:52.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.961+0000 7fb398ff9640 10 client.25813 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:01:52.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.961+0000 7fb3b49db640 1 --2- 172.21.15.1:0/611685336 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x561cbe09b680 0x561cbe117f40 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1016217a7987af10 server_cookie=1431d4b45640300b in_seq=0 out_seq=0 2024-09-11T22:01:52.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.962+0000 7fb398ff9640 1 -- 172.21.15.1:0/611685336 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7fb3a0005020 con 0x561cbe09b680 2024-09-11T22:01:52.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.962+0000 7fb398ff9640 10 client.25813 handle_command_reply: tid=0 2024-09-11T22:01:52.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.962+0000 7fb3b9819740 1 client.25813 shutdown 2024-09-11T22:01:52.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.962+0000 7fb3b9819740 1 -- 172.21.15.1:0/611685336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cbe08ca30 msgr2=0x561cbe10d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:52.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.962+0000 7fb3b9819740 1 --2- 172.21.15.1:0/611685336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cbe08ca30 0x561cbe10d0c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac0b8480 tx=0x7fb3ac003f50 comp rx=0 tx=0).stop 2024-09-11T22:01:52.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.962+0000 7fb3b9819740 1 -- 172.21.15.1:0/611685336 shutdown_connections 2024-09-11T22:01:52.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.962+0000 7fb3b9819740 1 --2- 172.21.15.1:0/611685336 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x561cbe09b680 0x561cbe117f40 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:52.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.962+0000 7fb3b9819740 1 --2- 172.21.15.1:0/611685336 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561cbe090dc0 0x561cbe0a0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:52.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.962+0000 7fb3b9819740 1 --2- 172.21.15.1:0/611685336 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cbe0904a0 0x561cbe10d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:52.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.962+0000 7fb398ff9640 0 client.25813 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:01:52.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.962+0000 7fb3b9819740 1 --2- 172.21.15.1:0/611685336 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561cbe08ca30 0x561cbe10d0c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:52.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.962+0000 7fb3b9819740 1 -- 172.21.15.1:0/611685336 >> 172.21.15.1:0/611685336 conn(0x561cbe0988c0 msgr2=0x561cbe08bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:01:52.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.963+0000 7fb3b9819740 1 -- 172.21.15.1:0/611685336 shutdown_connections 2024-09-11T22:01:52.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.963+0000 7fb3b9819740 1 -- 172.21.15.1:0/611685336 wait complete. 2024-09-11T22:01:52.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.963+0000 7fb3b9819740 20 client.25813 trim_cache size 0 max 16384 2024-09-11T22:01:52.976 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-11T22:01:52.976 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-09-11T22:01:52.977 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-09-11T22:01:52.977 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-11T22:01:52.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.964+0000 7fb3b7440640 1 -- 172.21.15.1:0/1839095810 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fb39c0a2cf0 msgr2=0x7fb39c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:52.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.964+0000 7fb3b7440640 1 --2- 172.21.15.1:0/1839095810 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fb39c0a2cf0 0x7fb39c0a51b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8020000 tx=0x7fb3a8021000 comp rx=0 tx=0).stop 2024-09-11T22:01:52.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.964+0000 7fb3b7440640 1 -- 172.21.15.1:0/1839095810 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb3b0102dd0 msgr2=0x7fb3b0100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:52.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.964+0000 7fb3b7440640 1 --2- 172.21.15.1:0/1839095810 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb3b0102dd0 0x7fb3b0100a30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac02a940 tx=0x7fb3ac003b20 comp rx=0 tx=0).stop 2024-09-11T22:01:52.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.964+0000 7fb3b7440640 1 -- 172.21.15.1:0/1839095810 shutdown_connections 2024-09-11T22:01:52.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.964+0000 7fb3b7440640 1 --2- 172.21.15.1:0/1839095810 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fb39c0a2cf0 0x7fb39c0a51b0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:52.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.964+0000 7fb3b7440640 1 --2- 172.21.15.1:0/1839095810 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3b0108300 0x7fb3b01014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:52.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.964+0000 7fb3b7440640 1 --2- 172.21.15.1:0/1839095810 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3b0103780 0x7fb3b0100f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:52.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.964+0000 7fb3b7440640 1 --2- 172.21.15.1:0/1839095810 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb3b0102dd0 0x7fb3b0100a30 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:52.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.964+0000 7fb3b7440640 1 -- 172.21.15.1:0/1839095810 >> 172.21.15.1:0/1839095810 conn(0x7fb3b00fc7d0 msgr2=0x7fb3b0105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:01:52.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.964+0000 7fb3b7440640 1 -- 172.21.15.1:0/1839095810 shutdown_connections 2024-09-11T22:01:52.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:52.964+0000 7fb3b7440640 1 -- 172.21.15.1:0/1839095810 wait complete. 2024-09-11T22:01:53.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:01:53 smithi001 ceph-mon[30935]: pgmap v144: 97 pgs: 97 active+clean; 9.6 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 1.47k op/s 2024-09-11T22:01:53.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:01:53 smithi053 ceph-mon[34908]: pgmap v144: 97 pgs: 97 active+clean; 9.6 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 1.47k op/s 2024-09-11T22:01:53.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:01:53 smithi161 ceph-mon[35615]: pgmap v144: 97 pgs: 97 active+clean; 9.6 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 1.47k op/s 2024-09-11T22:01:53.803 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T22:01:53.803 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:01:54.183 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:01:54.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:54.700+0000 7fc9987e1640 1 -- 172.21.15.1:0/1699598046 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc990102570 msgr2=0x7fc9901029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:54.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:54.700+0000 7fc9987e1640 1 --2- 172.21.15.1:0/1699598046 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc990102570 0x7fc9901029b0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fc980009900 tx=0x7fc98002a480 comp rx=0 tx=0).stop 2024-09-11T22:01:54.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:54.700+0000 7fc9987e1640 1 -- 172.21.15.1:0/1699598046 shutdown_connections 2024-09-11T22:01:54.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:54.700+0000 7fc9987e1640 1 --2- 172.21.15.1:0/1699598046 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc990102ef0 0x7fc99010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:54.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:54.700+0000 7fc9987e1640 1 --2- 172.21.15.1:0/1699598046 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc990102570 0x7fc9901029b0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:54.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:54.700+0000 7fc9987e1640 1 --2- 172.21.15.1:0/1699598046 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc990100e20 0x7fc990101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:54.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:54.700+0000 7fc9987e1640 1 -- 172.21.15.1:0/1699598046 >> 172.21.15.1:0/1699598046 conn(0x7fc9900fc7d0 msgr2=0x7fc9900febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:01:54.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:54.700+0000 7fc9987e1640 1 -- 172.21.15.1:0/1699598046 shutdown_connections 2024-09-11T22:01:54.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:54.700+0000 7fc9987e1640 1 -- 172.21.15.1:0/1699598046 wait complete. 2024-09-11T22:01:54.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:54.701+0000 7fc9987e1640 1 Processor -- start 2024-09-11T22:01:54.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:54.701+0000 7fc9987e1640 1 -- start start 2024-09-11T22:01:54.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:54.701+0000 7fc9987e1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc990100e20 0x7fc9901a0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:54.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:54.701+0000 7fc9987e1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc990102570 0x7fc9901a0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:54.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:54.701+0000 7fc9987e1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc990102ef0 0x7fc99010ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:54.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:54.701+0000 7fc9987e1640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc99006abc0 con 0x7fc990102ef0 2024-09-11T22:01:54.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:54.701+0000 7fc9987e1640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc99006aa40 con 0x7fc990102570 2024-09-11T22:01:54.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:54.701+0000 7fc9987e1640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc99006ad40 con 0x7fc990100e20 2024-09-11T22:01:54.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:54.701+0000 7fc996d7e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc990102ef0 0x7fc99010ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:54.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:54.701+0000 7fc99657d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc990100e20 0x7fc9901a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:54.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:54.701+0000 7fc99657d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc990100e20 0x7fc9901a0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:41320/0 (socket says 172.21.15.1:41320) 2024-09-11T22:01:54.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:54.701+0000 7fc99657d640 1 -- 172.21.15.1:0/472016017 learned_addr learned my addr 172.21.15.1:0/472016017 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:01:54.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:54.701+0000 7fc995d7c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc990102570 0x7fc9901a0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:54.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:54.702+0000 7fc996d7e640 1 -- 172.21.15.1:0/472016017 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc990100e20 msgr2=0x7fc9901a0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:54.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:54.702+0000 7fc996d7e640 1 --2- 172.21.15.1:0/472016017 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc990100e20 0x7fc9901a0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:54.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:54.702+0000 7fc996d7e640 1 -- 172.21.15.1:0/472016017 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc990102570 msgr2=0x7fc9901a0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:54.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:54.702+0000 7fc996d7e640 1 --2- 172.21.15.1:0/472016017 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc990102570 0x7fc9901a0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:54.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:54.702+0000 7fc996d7e640 1 -- 172.21.15.1:0/472016017 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc99010d650 con 0x7fc990102ef0 2024-09-11T22:01:54.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:54.702+0000 7fc996d7e640 1 --2- 172.21.15.1:0/472016017 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc990102ef0 0x7fc99010ced0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fc988009fd0 tx=0x7fc98800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:01:54.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:54.702+0000 7fc9877fe640 1 -- 172.21.15.1:0/472016017 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98800ce20 con 0x7fc990102ef0 2024-09-11T22:01:54.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:54.702+0000 7fc9877fe640 1 -- 172.21.15.1:0/472016017 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc988005020 con 0x7fc990102ef0 2024-09-11T22:01:54.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:54.702+0000 7fc9877fe640 1 -- 172.21.15.1:0/472016017 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc988010720 con 0x7fc990102ef0 2024-09-11T22:01:54.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:54.702+0000 7fc9987e1640 1 -- 172.21.15.1:0/472016017 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc99010d940 con 0x7fc990102ef0 2024-09-11T22:01:54.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:54.703+0000 7fc9987e1640 1 -- 172.21.15.1:0/472016017 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9901a7280 con 0x7fc990102ef0 2024-09-11T22:01:54.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:54.704+0000 7fc9987e1640 1 -- 172.21.15.1:0/472016017 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc990101200 con 0x7fc990102ef0 2024-09-11T22:01:54.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:54.705+0000 7fc9877fe640 1 -- 172.21.15.1:0/472016017 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7fc988005540 con 0x7fc990102ef0 2024-09-11T22:01:54.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:54.706+0000 7fc9877fe640 1 --2- 172.21.15.1:0/472016017 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fc97c0a2d70 0x7fc97c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:54.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:54.706+0000 7fc99657d640 1 --2- 172.21.15.1:0/472016017 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fc97c0a2d70 0x7fc97c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:54.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:54.706+0000 7fc9877fe640 1 -- 172.21.15.1:0/472016017 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7fc9880cf9b0 con 0x7fc990102ef0 2024-09-11T22:01:54.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:54.706+0000 7fc99657d640 1 --2- 172.21.15.1:0/472016017 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fc97c0a2d70 0x7fc97c0a5230 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc98c009c90 tx=0x7fc98c005f90 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:01:54.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:54.711+0000 7fc9877fe640 1 -- 172.21.15.1:0/472016017 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc98809c590 con 0x7fc990102ef0 2024-09-11T22:01:54.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:54.926+0000 7fc9987e1640 1 -- 172.21.15.1:0/472016017 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc990106840 con 0x7fc990102ef0 2024-09-11T22:01:54.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:54.927+0000 7fc9877fe640 1 -- 172.21.15.1:0/472016017 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11957 (secure 0 0 0) 0x7fc98809be90 con 0x7fc990102ef0 2024-09-11T22:01:54.929 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T22:01:54.929 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":33,"btime":"2024-09-11T22:01:51:001787+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/421542832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":421542832},{"type":"v1","addr":"172.21.15.1:6835","nonce":421542832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2836864406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2836864406},{"type":"v1","addr":"172.21.15.1:6837","nonce":2836864406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T21:55:20.791263+0000","modified":"2024-09-11T22:01:50.436812+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24484,"mds_2":24467,"mds_3":24466,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14595":{"gid":14595,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/407834672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":407834672},{"type":"v1","addr":"172.21.15.1:6839","nonce":407834672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/1068827678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":1068827678},{"type":"v1","addr":"172.21.15.1:6841","nonce":1068827678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/872410024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":872410024},{"type":"v1","addr":"172.21.15.161:6833","nonce":872410024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/1648944570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1648944570},{"type":"v1","addr":"172.21.15.53:6833","nonce":1648944570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/2126933993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2126933993},{"type":"v1","addr":"172.21.15.53:6835","nonce":2126933993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.161:6835/828381592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":828381592},{"type":"v1","addr":"172.21.15.161:6835","nonce":828381592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.161:6837/1799318656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1799318656},{"type":"v1","addr":"172.21.15.161:6837","nonce":1799318656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.53:6837/3457131830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3457131830},{"type":"v1","addr":"172.21.15.53:6837","nonce":3457131830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/455365383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":455365383},{"type":"v1","addr":"172.21.15.161:6839","nonce":455365383}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.53:6839/1378584447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1378584447},{"type":"v1","addr":"172.21.15.53:6839","nonce":1378584447}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24484,24467,24466,24460]},"id":1}]} 2024-09-11T22:01:54.929 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 33 2024-09-11T22:01:54.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:54.929+0000 7fc9987e1640 1 -- 172.21.15.1:0/472016017 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fc97c0a2d70 msgr2=0x7fc97c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:54.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:54.929+0000 7fc9987e1640 1 --2- 172.21.15.1:0/472016017 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fc97c0a2d70 0x7fc97c0a5230 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc98c009c90 tx=0x7fc98c005f90 comp rx=0 tx=0).stop 2024-09-11T22:01:54.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:54.929+0000 7fc9987e1640 1 -- 172.21.15.1:0/472016017 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc990102ef0 msgr2=0x7fc99010ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:54.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:54.929+0000 7fc9987e1640 1 --2- 172.21.15.1:0/472016017 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc990102ef0 0x7fc99010ced0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fc988009fd0 tx=0x7fc98800ef10 comp rx=0 tx=0).stop 2024-09-11T22:01:54.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:54.929+0000 7fc9987e1640 1 -- 172.21.15.1:0/472016017 shutdown_connections 2024-09-11T22:01:54.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:54.929+0000 7fc9987e1640 1 --2- 172.21.15.1:0/472016017 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fc97c0a2d70 0x7fc97c0a5230 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:54.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:54.929+0000 7fc9987e1640 1 --2- 172.21.15.1:0/472016017 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc990102ef0 0x7fc99010ced0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:54.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:54.929+0000 7fc9987e1640 1 --2- 172.21.15.1:0/472016017 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc990102570 0x7fc9901a0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:54.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:54.929+0000 7fc9987e1640 1 --2- 172.21.15.1:0/472016017 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc990100e20 0x7fc9901a0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:54.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:54.929+0000 7fc9987e1640 1 -- 172.21.15.1:0/472016017 >> 172.21.15.1:0/472016017 conn(0x7fc9900fc7d0 msgr2=0x7fc9901048c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:01:54.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:54.929+0000 7fc9987e1640 1 -- 172.21.15.1:0/472016017 shutdown_connections 2024-09-11T22:01:54.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:54.929+0000 7fc9987e1640 1 -- 172.21.15.1:0/472016017 wait complete. 2024-09-11T22:01:55.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:01:55 smithi001 ceph-mon[30935]: pgmap v145: 97 pgs: 97 active+clean; 9.6 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 1.47k op/s 2024-09-11T22:01:55.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:01:55 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/472016017' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:01:55.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:01:55 smithi053 ceph-mon[34908]: pgmap v145: 97 pgs: 97 active+clean; 9.6 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 1.47k op/s 2024-09-11T22:01:55.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:01:55 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/472016017' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:01:55.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:01:55 smithi161 ceph-mon[35615]: pgmap v145: 97 pgs: 97 active+clean; 9.6 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 1.47k op/s 2024-09-11T22:01:55.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:01:55 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/472016017' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:01:55.696 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T22:01:56.003 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:01:56.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.491+0000 7fee7c382640 1 -- 172.21.15.1:0/516474188 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fee741036e0 msgr2=0x7fee74107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:56.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.491+0000 7fee7c382640 1 --2- 172.21.15.1:0/516474188 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fee741036e0 0x7fee74107b90 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fee70009930 tx=0x7fee7002a420 comp rx=0 tx=0).stop 2024-09-11T22:01:56.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.491+0000 7fee7c382640 1 -- 172.21.15.1:0/516474188 shutdown_connections 2024-09-11T22:01:56.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.491+0000 7fee7c382640 1 --2- 172.21.15.1:0/516474188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee741082a0 0x7fee7410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:56.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.491+0000 7fee7c382640 1 --2- 172.21.15.1:0/516474188 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fee741036e0 0x7fee74107b90 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:56.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.491+0000 7fee7c382640 1 --2- 172.21.15.1:0/516474188 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee74102d30 0x7fee74103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:56.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.491+0000 7fee7c382640 1 -- 172.21.15.1:0/516474188 >> 172.21.15.1:0/516474188 conn(0x7fee740fc7d0 msgr2=0x7fee740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:01:56.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.491+0000 7fee7c382640 1 -- 172.21.15.1:0/516474188 shutdown_connections 2024-09-11T22:01:56.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.491+0000 7fee7c382640 1 -- 172.21.15.1:0/516474188 wait complete. 2024-09-11T22:01:56.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.492+0000 7fee7c382640 1 Processor -- start 2024-09-11T22:01:56.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.492+0000 7fee7c382640 1 -- start start 2024-09-11T22:01:56.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.492+0000 7fee7c382640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee74102d30 0x7fee74100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:56.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.492+0000 7fee7c382640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee741036e0 0x7fee74100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:56.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.492+0000 7fee7c382640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fee741082a0 0x7fee74101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:56.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.492+0000 7fee7c382640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fee741120a0 con 0x7fee741082a0 2024-09-11T22:01:56.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.492+0000 7fee7c382640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fee74111f20 con 0x7fee74102d30 2024-09-11T22:01:56.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.492+0000 7fee7c382640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fee74112220 con 0x7fee741036e0 2024-09-11T22:01:56.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.492+0000 7fee7a91f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fee741082a0 0x7fee74101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:56.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.492+0000 7fee7991d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee741036e0 0x7fee74100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:56.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.492+0000 7fee7a91f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fee741082a0 0x7fee74101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44530/0 (socket says 172.21.15.1:44530) 2024-09-11T22:01:56.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.492+0000 7fee7a91f640 1 -- 172.21.15.1:0/3164430640 learned_addr learned my addr 172.21.15.1:0/3164430640 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:01:56.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.492+0000 7fee7a11e640 1 --2- 172.21.15.1:0/3164430640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee74102d30 0x7fee74100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:56.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.493+0000 7fee7a91f640 1 -- 172.21.15.1:0/3164430640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee741036e0 msgr2=0x7fee74100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:56.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.493+0000 7fee7a91f640 1 --2- 172.21.15.1:0/3164430640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee741036e0 0x7fee74100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:56.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.493+0000 7fee7a91f640 1 -- 172.21.15.1:0/3164430640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee74102d30 msgr2=0x7fee74100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:56.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.493+0000 7fee7a91f640 1 --2- 172.21.15.1:0/3164430640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee74102d30 0x7fee74100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:56.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.493+0000 7fee7a91f640 1 -- 172.21.15.1:0/3164430640 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee741a6a60 con 0x7fee741082a0 2024-09-11T22:01:56.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.493+0000 7fee7991d640 1 --2- 172.21.15.1:0/3164430640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee741036e0 0x7fee74100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:01:56.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.493+0000 7fee7a11e640 1 --2- 172.21.15.1:0/3164430640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee74102d30 0x7fee74100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:01:56.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.493+0000 7fee7a91f640 1 --2- 172.21.15.1:0/3164430640 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fee741082a0 0x7fee74101410 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fee6c00eb10 tx=0x7fee6c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:01:56.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.493+0000 7fee677fe640 1 -- 172.21.15.1:0/3164430640 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6c004b20 con 0x7fee741082a0 2024-09-11T22:01:56.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.493+0000 7fee7c382640 1 -- 172.21.15.1:0/3164430640 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee741a6d50 con 0x7fee741082a0 2024-09-11T22:01:56.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.493+0000 7fee677fe640 1 -- 172.21.15.1:0/3164430640 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee6c004cc0 con 0x7fee741082a0 2024-09-11T22:01:56.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.493+0000 7fee677fe640 1 -- 172.21.15.1:0/3164430640 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6c010760 con 0x7fee741082a0 2024-09-11T22:01:56.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.493+0000 7fee7c382640 1 -- 172.21.15.1:0/3164430640 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee741a7260 con 0x7fee741082a0 2024-09-11T22:01:56.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.496+0000 7fee7e75b740 1 Processor -- start 2024-09-11T22:01:56.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.496+0000 7fee677fe640 1 -- 172.21.15.1:0/3164430640 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7fee6c005020 con 0x7fee741082a0 2024-09-11T22:01:56.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.497+0000 7fee7e75b740 1 -- start start 2024-09-11T22:01:56.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.497+0000 7fee7e75b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a3ab5fb40 0x563a3ab5e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:56.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.497+0000 7fee7e75b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563a3ab5e5d0 0x563a3ab5ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:56.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.497+0000 7fee7e75b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a3ab69370 0x563a3ab6b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:56.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.497+0000 7fee7e75b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x563a3aac42f0 con 0x563a3ab5fb40 2024-09-11T22:01:56.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.497+0000 7fee7e75b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x563a3aa6bcf0 con 0x563a3ab5e5d0 2024-09-11T22:01:56.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.497+0000 7fee7e75b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x563a3aac02b0 con 0x563a3ab69370 2024-09-11T22:01:56.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.497+0000 7fee7991d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a3ab5fb40 0x563a3ab5e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:56.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.497+0000 7fee7a11e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563a3ab5e5d0 0x563a3ab5ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:56.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.497+0000 7fee7991d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a3ab5fb40 0x563a3ab5e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44538/0 (socket says 172.21.15.1:44538) 2024-09-11T22:01:56.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.497+0000 7fee7a11e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563a3ab5e5d0 0x563a3ab5ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:50356/0 (socket says 172.21.15.1:50356) 2024-09-11T22:01:56.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.497+0000 7fee7991d640 1 -- 172.21.15.1:0/2250353238 learned_addr learned my addr 172.21.15.1:0/2250353238 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:01:56.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.497+0000 7fee7991d640 1 --2- 172.21.15.1:0/2250353238 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a3ab69370 0x563a3ab6b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:56.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.497+0000 7fee7a11e640 1 -- 172.21.15.1:0/2250353238 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a3ab69370 msgr2=0x563a3ab6b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:56.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.497+0000 7fee7a11e640 1 --2- 172.21.15.1:0/2250353238 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a3ab69370 0x563a3ab6b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:56.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.497+0000 7fee7a11e640 1 -- 172.21.15.1:0/2250353238 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a3ab5fb40 msgr2=0x563a3ab5e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:56.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.497+0000 7fee7a11e640 1 --2- 172.21.15.1:0/2250353238 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a3ab5fb40 0x563a3ab5e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:56.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.497+0000 7fee677fe640 1 --2- 172.21.15.1:0/3164430640 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fee600a2d70 0x7fee600a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:56.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.497+0000 7fee7a11e640 1 -- 172.21.15.1:0/2250353238 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a3aa503a0 con 0x563a3ab5e5d0 2024-09-11T22:01:56.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.497+0000 7fee7991d640 1 --2- 172.21.15.1:0/2250353238 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a3ab69370 0x563a3ab6b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:01:56.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.498+0000 7fee677fe640 1 -- 172.21.15.1:0/3164430640 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7fee6c0cf920 con 0x7fee741082a0 2024-09-11T22:01:56.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.498+0000 7fee7991d640 1 --2- 172.21.15.1:0/3164430640 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fee600a2d70 0x7fee600a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:56.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.498+0000 7fee7a11e640 1 --2- 172.21.15.1:0/2250353238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563a3ab5e5d0 0x563a3ab5ea10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fee680103d0 tx=0x7fee68010890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61470d5ffa636821 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:01:56.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.498+0000 7fee64dea640 1 -- 172.21.15.1:0/2250353238 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee68011700 con 0x563a3ab5e5d0 2024-09-11T22:01:56.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.498+0000 7fee64dea640 1 -- 172.21.15.1:0/2250353238 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee68011d80 con 0x563a3ab5e5d0 2024-09-11T22:01:56.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.498+0000 7fee64dea640 1 -- 172.21.15.1:0/2250353238 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6801a380 con 0x563a3ab5e5d0 2024-09-11T22:01:56.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.499+0000 7fee7e75b740 1 -- 172.21.15.1:0/2250353238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563a3ab5e5d0 msgr2=0x563a3ab5ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:56.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.499+0000 7fee7e75b740 1 --2- 172.21.15.1:0/2250353238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563a3ab5e5d0 0x563a3ab5ea10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fee680103d0 tx=0x7fee68010890 comp rx=0 tx=0).stop 2024-09-11T22:01:56.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.499+0000 7fee7991d640 1 --2- 172.21.15.1:0/3164430640 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fee600a2d70 0x7fee600a5230 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fee70005ec0 tx=0x7fee70005fd0 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:01:56.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.499+0000 7fee7e75b740 1 -- 172.21.15.1:0/2250353238 shutdown_connections 2024-09-11T22:01:56.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.499+0000 7fee7e75b740 1 --2- 172.21.15.1:0/2250353238 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a3ab69370 0x563a3ab6b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:56.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.499+0000 7fee7e75b740 1 --2- 172.21.15.1:0/2250353238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563a3ab5e5d0 0x563a3ab5ea10 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:56.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.499+0000 7fee7e75b740 1 --2- 172.21.15.1:0/2250353238 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a3ab5fb40 0x563a3ab5e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:56.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.499+0000 7fee7e75b740 1 -- 172.21.15.1:0/2250353238 >> 172.21.15.1:0/2250353238 conn(0x563a3ab27310 msgr2=0x563a3ab5ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:01:56.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.499+0000 7fee7e75b740 1 -- 172.21.15.1:0/2250353238 shutdown_connections 2024-09-11T22:01:56.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.499+0000 7fee7e75b740 1 -- 172.21.15.1:0/2250353238 wait complete. 2024-09-11T22:01:56.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.499+0000 7fee7e75b740 1 -- start start 2024-09-11T22:01:56.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.499+0000 7fee7e75b740 1 Processor -- start 2024-09-11T22:01:56.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.500+0000 7fee7e75b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563a3ab5e5d0 0x563a3ab68c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:56.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.500+0000 7fee7e75b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a3ab5fb40 0x563a3ab65f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:56.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.500+0000 7fee7e75b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a3ab69370 0x563a3ab66480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:56.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.500+0000 7fee7e75b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x563a3ab6c070 con 0x563a3ab69370 2024-09-11T22:01:56.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.500+0000 7fee7e75b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x563a3aa7f060 con 0x563a3ab5e5d0 2024-09-11T22:01:56.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.500+0000 7fee7e75b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x563a3ab6c1f0 con 0x563a3ab5fb40 2024-09-11T22:01:56.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.500+0000 7fee7e75b740 10 client.0 _ll_register_callbacks cb 0x563a3a7d28c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:01:56.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.500+0000 7fee7a11e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563a3ab5e5d0 0x563a3ab68c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:56.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.500+0000 7fee7991d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a3ab5fb40 0x563a3ab65f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:56.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.500+0000 7fee7a91f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a3ab69370 0x563a3ab66480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:56.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.500+0000 7fee7991d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a3ab5fb40 0x563a3ab65f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:38486/0 (socket says 172.21.15.1:38486) 2024-09-11T22:01:56.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.500+0000 7fee7991d640 1 -- 172.21.15.1:0/2019527224 learned_addr learned my addr 172.21.15.1:0/2019527224 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:01:56.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.500+0000 7fee7991d640 1 -- 172.21.15.1:0/2019527224 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563a3ab5e5d0 msgr2=0x563a3ab68c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:56.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.500+0000 7fee7991d640 1 --2- 172.21.15.1:0/2019527224 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563a3ab5e5d0 0x563a3ab68c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:56.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.500+0000 7fee7991d640 1 -- 172.21.15.1:0/2019527224 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a3ab69370 msgr2=0x563a3ab66480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:56.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.500+0000 7fee7991d640 1 --2- 172.21.15.1:0/2019527224 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a3ab69370 0x563a3ab66480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:56.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.500+0000 7fee7991d640 1 -- 172.21.15.1:0/2019527224 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a3a8353c0 con 0x563a3ab5fb40 2024-09-11T22:01:56.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.500+0000 7fee7a91f640 1 --2- 172.21.15.1:0/2019527224 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a3ab69370 0x563a3ab66480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:01:56.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.501+0000 7fee7991d640 1 --2- 172.21.15.1:0/2019527224 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a3ab5fb40 0x563a3ab65f40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fee70005940 tx=0x7fee700058d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1599e1d1fd34e7d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:01:56.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.501+0000 7fee5dffb640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-11T22:01:56.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.501+0000 7fee5dffb640 1 -- 172.21.15.1:0/2019527224 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7004c070 con 0x563a3ab5fb40 2024-09-11T22:01:56.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.501+0000 7fee5dffb640 1 -- 172.21.15.1:0/2019527224 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7fee70004770 con 0x563a3ab5fb40 2024-09-11T22:01:56.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.501+0000 7fee7e75b740 1 -- 172.21.15.1:0/2019527224 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee68011d80 con 0x563a3ab5fb40 2024-09-11T22:01:56.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.501+0000 7fee5dffb640 1 -- 172.21.15.1:0/2019527224 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fee70058050 con 0x563a3ab5fb40 2024-09-11T22:01:56.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.501+0000 7fee7e75b740 10 client.25831 fetch_fsmap learned FSMap version 33 2024-09-11T22:01:56.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.501+0000 7fee7e75b740 1 -- 172.21.15.1:0/2019527224 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x563a3ab77a60 con 0x563a3ab5fb40 2024-09-11T22:01:56.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.501+0000 7fee5dffb640 1 -- 172.21.15.1:0/2019527224 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee68011d80 con 0x563a3ab5fb40 2024-09-11T22:01:56.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.501+0000 7fee5dffb640 1 -- 172.21.15.1:0/2019527224 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7fee70003af0 con 0x563a3ab5fb40 2024-09-11T22:01:56.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.501+0000 7fee7e75b740 10 client.25831 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T22:01:56.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.501+0000 7fee7e75b740 10 client.25831 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-11T22:01:56.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.501+0000 7fee7e75b740 20 client.25831 populate_metadata read hostname 'smithi001' 2024-09-11T22:01:56.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.502+0000 7fee7e75b740 1 --2- 172.21.15.1:0/2019527224 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x563a3ab78ec0 0x563a3ab7b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:56.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.502+0000 7fee7e75b740 4 client.25831 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:01:56.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.502+0000 7fee7e75b740 1 -- 172.21.15.1:0/2019527224 --> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a3aad1860 con 0x563a3ab78ec0 2024-09-11T22:01:56.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.502+0000 7fee7a11e640 1 --2- 172.21.15.1:0/2019527224 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x563a3ab78ec0 0x563a3ab7b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:56.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.502+0000 7fee7a11e640 1 --2- 172.21.15.1:0/2019527224 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x563a3ab78ec0 0x563a3ab7b300 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6aab86ed1ba1fd2b server_cookie=76b70ec1fd7b17b in_seq=0 out_seq=0 2024-09-11T22:01:56.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.502+0000 7fee5dffb640 10 client.25831 ms_handle_connect on v2:172.21.15.161:6834/828381592 2024-09-11T22:01:56.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.503+0000 7fee5dffb640 1 -- 172.21.15.1:0/2019527224 <== mds.4 v2:172.21.15.161:6834/828381592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563a3aad1860 con 0x563a3ab78ec0 2024-09-11T22:01:56.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.503+0000 7fee5dffb640 10 client.25831 handle_command_reply: tid=0 2024-09-11T22:01:56.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.503+0000 7fee7e75b740 1 client.25831 shutdown 2024-09-11T22:01:56.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.503+0000 7fee7e75b740 1 -- 172.21.15.1:0/2019527224 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a3ab5fb40 msgr2=0x563a3ab65f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:56.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.503+0000 7fee7e75b740 1 --2- 172.21.15.1:0/2019527224 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a3ab5fb40 0x563a3ab65f40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fee70005940 tx=0x7fee700058d0 comp rx=0 tx=0).stop 2024-09-11T22:01:56.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.503+0000 7fee7e75b740 1 -- 172.21.15.1:0/2019527224 shutdown_connections 2024-09-11T22:01:56.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.503+0000 7fee7e75b740 1 --2- 172.21.15.1:0/2019527224 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a3ab69370 0x563a3ab66480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:56.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.503+0000 7fee7e75b740 1 --2- 172.21.15.1:0/2019527224 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a3ab5fb40 0x563a3ab65f40 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:56.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.503+0000 7fee7e75b740 1 --2- 172.21.15.1:0/2019527224 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x563a3ab78ec0 0x563a3ab7b300 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:56.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.503+0000 7fee7e75b740 1 --2- 172.21.15.1:0/2019527224 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563a3ab5e5d0 0x563a3ab68c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:56.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.503+0000 7fee7e75b740 1 -- 172.21.15.1:0/2019527224 >> 172.21.15.1:0/2019527224 conn(0x563a3ab27310 msgr2=0x563a3ab60930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:01:56.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.503+0000 7fee5dffb640 0 client.25831 ms_handle_reset on v2:172.21.15.161:6834/828381592 2024-09-11T22:01:56.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.503+0000 7fee7e75b740 1 -- 172.21.15.1:0/2019527224 shutdown_connections 2024-09-11T22:01:56.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.504+0000 7fee7e75b740 1 -- 172.21.15.1:0/2019527224 wait complete. 2024-09-11T22:01:56.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.504+0000 7fee7e75b740 20 client.25831 trim_cache size 0 max 16384 2024-09-11T22:01:56.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.517+0000 7fee7e75b740 1 Processor -- start 2024-09-11T22:01:56.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.517+0000 7fee7e75b740 1 -- start start 2024-09-11T22:01:56.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.517+0000 7fee7e75b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563a3ab5e5d0 0x563a3ab5ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:56.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.517+0000 7fee7e75b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a3ab5fb40 0x563a3ab75340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:56.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.517+0000 7fee7e75b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a3ab69370 0x563a3ab80bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:56.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.517+0000 7fee7e75b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fee741afb90 con 0x563a3ab69370 2024-09-11T22:01:56.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.517+0000 7fee7e75b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fee741b17c0 con 0x563a3ab5e5d0 2024-09-11T22:01:56.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.517+0000 7fee7e75b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fee741b0260 con 0x563a3ab5fb40 2024-09-11T22:01:56.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.517+0000 7fee7a91f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a3ab69370 0x563a3ab80bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:56.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.517+0000 7fee7a11e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563a3ab5e5d0 0x563a3ab5ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:56.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.517+0000 7fee7991d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a3ab5fb40 0x563a3ab75340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:56.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.517+0000 7fee7a91f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a3ab69370 0x563a3ab80bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44564/0 (socket says 172.21.15.1:44564) 2024-09-11T22:01:56.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.517+0000 7fee7a91f640 1 -- 172.21.15.1:0/882283649 learned_addr learned my addr 172.21.15.1:0/882283649 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:01:56.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.517+0000 7fee7a91f640 1 -- 172.21.15.1:0/882283649 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a3ab5fb40 msgr2=0x563a3ab75340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:56.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.517+0000 7fee7a91f640 1 --2- 172.21.15.1:0/882283649 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a3ab5fb40 0x563a3ab75340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:56.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.517+0000 7fee7a91f640 1 -- 172.21.15.1:0/882283649 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563a3ab5e5d0 msgr2=0x563a3ab5ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:56.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.517+0000 7fee7a91f640 1 --2- 172.21.15.1:0/882283649 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563a3ab5e5d0 0x563a3ab5ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:56.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.517+0000 7fee7a91f640 1 -- 172.21.15.1:0/882283649 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a3ab5b730 con 0x563a3ab69370 2024-09-11T22:01:56.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.517+0000 7fee7a11e640 1 --2- 172.21.15.1:0/882283649 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563a3ab5e5d0 0x563a3ab5ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:01:56.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.518+0000 7fee7a91f640 1 --2- 172.21.15.1:0/882283649 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a3ab69370 0x563a3ab80bd0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fee6c09bd90 tx=0x7fee6c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aed7d09a7a241cb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:01:56.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.518+0000 7fee5ffff640 1 -- 172.21.15.1:0/882283649 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6c0b9070 con 0x563a3ab69370 2024-09-11T22:01:56.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.518+0000 7fee5ffff640 1 -- 172.21.15.1:0/882283649 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee6c09c500 con 0x563a3ab69370 2024-09-11T22:01:56.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.518+0000 7fee5ffff640 1 -- 172.21.15.1:0/882283649 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6c0a4840 con 0x563a3ab69370 2024-09-11T22:01:56.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.519+0000 7fee7e75b740 1 -- 172.21.15.1:0/882283649 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a3ab69370 msgr2=0x563a3ab80bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:56.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.519+0000 7fee7e75b740 1 --2- 172.21.15.1:0/882283649 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a3ab69370 0x563a3ab80bd0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fee6c09bd90 tx=0x7fee6c004a40 comp rx=0 tx=0).stop 2024-09-11T22:01:56.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.519+0000 7fee7e75b740 1 -- 172.21.15.1:0/882283649 shutdown_connections 2024-09-11T22:01:56.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.519+0000 7fee7e75b740 1 --2- 172.21.15.1:0/882283649 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a3ab69370 0x563a3ab80bd0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:56.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.519+0000 7fee7e75b740 1 --2- 172.21.15.1:0/882283649 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a3ab5fb40 0x563a3ab75340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:56.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.519+0000 7fee7e75b740 1 --2- 172.21.15.1:0/882283649 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563a3ab5e5d0 0x563a3ab5ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:56.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.519+0000 7fee7e75b740 1 -- 172.21.15.1:0/882283649 >> 172.21.15.1:0/882283649 conn(0x563a3ab27310 msgr2=0x563a3ab5a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:01:56.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.519+0000 7fee7e75b740 1 -- 172.21.15.1:0/882283649 shutdown_connections 2024-09-11T22:01:56.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.519+0000 7fee7e75b740 1 -- 172.21.15.1:0/882283649 wait complete. 2024-09-11T22:01:56.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.519+0000 7fee7e75b740 1 -- start start 2024-09-11T22:01:56.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.519+0000 7fee7e75b740 1 Processor -- start 2024-09-11T22:01:56.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.519+0000 7fee7e75b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a3ab5e5d0 0x563a3ab6cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:56.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.519+0000 7fee7e75b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a3ab5fb40 0x563a3ab6d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:56.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.519+0000 7fee7e75b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563a3ab69370 0x563a3abe1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:56.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.520+0000 7fee7e75b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fee741b1030 con 0x563a3ab5e5d0 2024-09-11T22:01:56.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.520+0000 7fee7e75b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fee741b0930 con 0x563a3ab69370 2024-09-11T22:01:56.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.520+0000 7fee7e75b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fee741adb70 con 0x563a3ab5fb40 2024-09-11T22:01:56.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.520+0000 7fee7e75b740 10 client.0 _ll_register_callbacks cb 0x7fee70031ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:01:56.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.520+0000 7fee7a11e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a3ab5e5d0 0x563a3ab6cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:56.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.520+0000 7fee7a11e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a3ab5e5d0 0x563a3ab6cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44574/0 (socket says 172.21.15.1:44574) 2024-09-11T22:01:56.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.520+0000 7fee7a11e640 1 -- 172.21.15.1:0/3495720676 learned_addr learned my addr 172.21.15.1:0/3495720676 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:01:56.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.520+0000 7fee7991d640 1 --2- 172.21.15.1:0/3495720676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a3ab5fb40 0x563a3ab6d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:56.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.520+0000 7fee7a91f640 1 --2- 172.21.15.1:0/3495720676 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563a3ab69370 0x563a3abe1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:56.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.520+0000 7fee7a11e640 1 -- 172.21.15.1:0/3495720676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a3ab5fb40 msgr2=0x563a3ab6d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:56.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.520+0000 7fee7a11e640 1 --2- 172.21.15.1:0/3495720676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a3ab5fb40 0x563a3ab6d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:56.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.520+0000 7fee7a11e640 1 -- 172.21.15.1:0/3495720676 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563a3ab69370 msgr2=0x563a3abe1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:56.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.520+0000 7fee7a11e640 1 --2- 172.21.15.1:0/3495720676 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563a3ab69370 0x563a3abe1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:56.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.520+0000 7fee7a11e640 1 -- 172.21.15.1:0/3495720676 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a3abe2280 con 0x563a3ab5e5d0 2024-09-11T22:01:56.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.520+0000 7fee7a11e640 1 --2- 172.21.15.1:0/3495720676 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a3ab5e5d0 0x563a3ab6cbf0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fee680103d0 tx=0x7fee6801cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6f3d07b462c911e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:01:56.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.520+0000 7fee5dffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:01:56.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.520+0000 7fee5dffb640 1 -- 172.21.15.1:0/3495720676 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee68005040 con 0x563a3ab5e5d0 2024-09-11T22:01:56.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.520+0000 7fee5dffb640 1 -- 172.21.15.1:0/3495720676 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7fee680051e0 con 0x563a3ab5e5d0 2024-09-11T22:01:56.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.520+0000 7fee7e75b740 1 -- 172.21.15.1:0/3495720676 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee6c09c500 con 0x563a3ab5e5d0 2024-09-11T22:01:56.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.520+0000 7fee5dffb640 1 -- 172.21.15.1:0/3495720676 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fee68039050 con 0x563a3ab5e5d0 2024-09-11T22:01:56.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.520+0000 7fee7e75b740 10 client.16314 fetch_fsmap learned FSMap version 33 2024-09-11T22:01:56.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.520+0000 7fee7e75b740 1 -- 172.21.15.1:0/3495720676 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x563a3abe3560 con 0x563a3ab5e5d0 2024-09-11T22:01:56.733 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-11T22:01:56.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.521+0000 7fee5dffb640 1 -- 172.21.15.1:0/3495720676 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee6c09c500 con 0x563a3ab5e5d0 2024-09-11T22:01:56.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.521+0000 7fee5dffb640 1 -- 172.21.15.1:0/3495720676 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7fee68011500 con 0x563a3ab5e5d0 2024-09-11T22:01:56.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.521+0000 7fee7e75b740 10 client.16314 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T22:01:56.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.521+0000 7fee7e75b740 10 client.16314 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-11T22:01:56.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.521+0000 7fee7e75b740 20 client.16314 populate_metadata read hostname 'smithi001' 2024-09-11T22:01:56.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.521+0000 7fee7e75b740 1 --2- 172.21.15.1:0/3495720676 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x563a3abe38f0 0x563a3abe5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:56.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.521+0000 7fee7e75b740 4 client.16314 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:01:56.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.521+0000 7fee7e75b740 1 -- 172.21.15.1:0/3495720676 --> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563a3aacbd70 con 0x563a3abe38f0 2024-09-11T22:01:56.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.521+0000 7fee7991d640 1 --2- 172.21.15.1:0/3495720676 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x563a3abe38f0 0x563a3abe5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:56.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.522+0000 7fee7991d640 1 --2- 172.21.15.1:0/3495720676 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x563a3abe38f0 0x563a3abe5ce0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8648997b7788e57c server_cookie=289c58c93ea1320e in_seq=0 out_seq=0 2024-09-11T22:01:56.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.522+0000 7fee5dffb640 10 client.16314 ms_handle_connect on v2:172.21.15.161:6834/828381592 2024-09-11T22:01:56.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.522+0000 7fee5dffb640 1 -- 172.21.15.1:0/3495720676 <== mds.4 v2:172.21.15.161:6834/828381592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563a3aacbd70 con 0x563a3abe38f0 2024-09-11T22:01:56.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.522+0000 7fee5dffb640 10 client.16314 handle_command_reply: tid=0 2024-09-11T22:01:56.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.522+0000 7fee7e75b740 1 client.16314 shutdown 2024-09-11T22:01:56.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.522+0000 7fee7e75b740 1 -- 172.21.15.1:0/3495720676 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a3ab5e5d0 msgr2=0x563a3ab6cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:56.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.522+0000 7fee7e75b740 1 --2- 172.21.15.1:0/3495720676 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a3ab5e5d0 0x563a3ab6cbf0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fee680103d0 tx=0x7fee6801cc60 comp rx=0 tx=0).stop 2024-09-11T22:01:56.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.522+0000 7fee7e75b740 1 -- 172.21.15.1:0/3495720676 shutdown_connections 2024-09-11T22:01:56.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.522+0000 7fee7e75b740 1 --2- 172.21.15.1:0/3495720676 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x563a3abe38f0 0x563a3abe5ce0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:56.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.522+0000 7fee7e75b740 1 --2- 172.21.15.1:0/3495720676 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563a3ab69370 0x563a3abe1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:56.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.522+0000 7fee7e75b740 1 --2- 172.21.15.1:0/3495720676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a3ab5fb40 0x563a3ab6d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:56.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.522+0000 7fee5dffb640 0 client.16314 ms_handle_reset on v2:172.21.15.161:6834/828381592 2024-09-11T22:01:56.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.522+0000 7fee7e75b740 1 --2- 172.21.15.1:0/3495720676 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a3ab5e5d0 0x563a3ab6cbf0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:56.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.522+0000 7fee7e75b740 1 -- 172.21.15.1:0/3495720676 >> 172.21.15.1:0/3495720676 conn(0x563a3ab27310 msgr2=0x563a3ab59d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:01:56.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.522+0000 7fee7e75b740 1 -- 172.21.15.1:0/3495720676 shutdown_connections 2024-09-11T22:01:56.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.522+0000 7fee7e75b740 1 -- 172.21.15.1:0/3495720676 wait complete. 2024-09-11T22:01:56.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.522+0000 7fee7e75b740 20 client.16314 trim_cache size 0 max 16384 2024-09-11T22:01:56.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.523+0000 7fee7c382640 1 -- 172.21.15.1:0/3164430640 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fee600a2d70 msgr2=0x7fee600a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:56.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.523+0000 7fee7c382640 1 --2- 172.21.15.1:0/3164430640 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fee600a2d70 0x7fee600a5230 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fee70005ec0 tx=0x7fee70005fd0 comp rx=0 tx=0).stop 2024-09-11T22:01:56.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.523+0000 7fee7c382640 1 -- 172.21.15.1:0/3164430640 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fee741082a0 msgr2=0x7fee74101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:56.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.523+0000 7fee7c382640 1 --2- 172.21.15.1:0/3164430640 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fee741082a0 0x7fee74101410 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fee6c00eb10 tx=0x7fee6c00c2d0 comp rx=0 tx=0).stop 2024-09-11T22:01:56.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.523+0000 7fee7c382640 1 -- 172.21.15.1:0/3164430640 shutdown_connections 2024-09-11T22:01:56.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.523+0000 7fee7c382640 1 --2- 172.21.15.1:0/3164430640 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fee600a2d70 0x7fee600a5230 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:56.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.523+0000 7fee7c382640 1 --2- 172.21.15.1:0/3164430640 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fee741082a0 0x7fee74101410 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:56.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.523+0000 7fee7c382640 1 --2- 172.21.15.1:0/3164430640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee741036e0 0x7fee74100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:56.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.523+0000 7fee7c382640 1 --2- 172.21.15.1:0/3164430640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fee74102d30 0x7fee74100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:56.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.523+0000 7fee7c382640 1 -- 172.21.15.1:0/3164430640 >> 172.21.15.1:0/3164430640 conn(0x7fee740fc7d0 msgr2=0x7fee74105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:01:56.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.523+0000 7fee7c382640 1 -- 172.21.15.1:0/3164430640 shutdown_connections 2024-09-11T22:01:56.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:56.523+0000 7fee7c382640 1 -- 172.21.15.1:0/3164430640 wait complete. 2024-09-11T22:01:57.292 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:01:57 smithi001 ceph-mon[30935]: pgmap v146: 97 pgs: 97 active+clean; 22 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.2 MiB/s wr, 1.49k op/s 2024-09-11T22:01:57.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:01:57 smithi053 ceph-mon[34908]: pgmap v146: 97 pgs: 97 active+clean; 22 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.2 MiB/s wr, 1.49k op/s 2024-09-11T22:01:57.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:01:57 smithi161 ceph-mon[35615]: pgmap v146: 97 pgs: 97 active+clean; 22 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.2 MiB/s wr, 1.49k op/s 2024-09-11T22:01:57.794 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T22:01:58.071 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:01:58.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.588+0000 7f1f5483c640 1 -- 172.21.15.1:0/1444485867 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f501027a0 msgr2=0x7f1f501067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:58.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.588+0000 7f1f5483c640 1 --2- 172.21.15.1:0/1444485867 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f501027a0 0x7f1f501067f0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f1f44009900 tx=0x7f1f4402a410 comp rx=0 tx=0).stop 2024-09-11T22:01:58.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.588+0000 7f1f5483c640 1 -- 172.21.15.1:0/1444485867 shutdown_connections 2024-09-11T22:01:58.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.588+0000 7f1f5483c640 1 --2- 172.21.15.1:0/1444485867 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f50106f20 0x7f1f5010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:58.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.588+0000 7f1f5483c640 1 --2- 172.21.15.1:0/1444485867 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f501027a0 0x7f1f501067f0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:58.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.588+0000 7f1f5483c640 1 --2- 172.21.15.1:0/1444485867 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f50101df0 0x7f1f501021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:58.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.588+0000 7f1f5483c640 1 -- 172.21.15.1:0/1444485867 >> 172.21.15.1:0/1444485867 conn(0x7f1f500fd8f0 msgr2=0x7f1f500ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:01:58.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.588+0000 7f1f5483c640 1 -- 172.21.15.1:0/1444485867 shutdown_connections 2024-09-11T22:01:58.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.588+0000 7f1f5483c640 1 -- 172.21.15.1:0/1444485867 wait complete. 2024-09-11T22:01:58.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.589+0000 7f1f5483c640 1 Processor -- start 2024-09-11T22:01:58.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.589+0000 7f1f5483c640 1 -- start start 2024-09-11T22:01:58.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.589+0000 7f1f5483c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f50101df0 0x7f1f500744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:58.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.589+0000 7f1f5483c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f501027a0 0x7f1f50074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:58.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.589+0000 7f1f5483c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f50106f20 0x7f1f50074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:58.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.589+0000 7f1f5483c640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1f50111f80 con 0x7f1f50101df0 2024-09-11T22:01:58.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.589+0000 7f1f5483c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1f50111e00 con 0x7f1f501027a0 2024-09-11T22:01:58.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.589+0000 7f1f5483c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1f50112100 con 0x7f1f50106f20 2024-09-11T22:01:58.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.590+0000 7f1f4e59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f50101df0 0x7f1f500744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:58.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.590+0000 7f1f4e59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f50101df0 0x7f1f500744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44596/0 (socket says 172.21.15.1:44596) 2024-09-11T22:01:58.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.590+0000 7f1f4e59c640 1 -- 172.21.15.1:0/3450205396 learned_addr learned my addr 172.21.15.1:0/3450205396 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:01:58.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.590+0000 7f1f4dd9b640 1 --2- 172.21.15.1:0/3450205396 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f501027a0 0x7f1f50074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:58.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.590+0000 7f1f4ed9d640 1 --2- 172.21.15.1:0/3450205396 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f50106f20 0x7f1f50074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:58.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.590+0000 7f1f4e59c640 1 -- 172.21.15.1:0/3450205396 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f50106f20 msgr2=0x7f1f50074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:58.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.590+0000 7f1f4e59c640 1 --2- 172.21.15.1:0/3450205396 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f50106f20 0x7f1f50074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:58.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.590+0000 7f1f4e59c640 1 -- 172.21.15.1:0/3450205396 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f501027a0 msgr2=0x7f1f50074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:58.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.590+0000 7f1f4e59c640 1 --2- 172.21.15.1:0/3450205396 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f501027a0 0x7f1f50074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:58.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.590+0000 7f1f4e59c640 1 -- 172.21.15.1:0/3450205396 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f501a6920 con 0x7f1f50101df0 2024-09-11T22:01:58.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.590+0000 7f1f4ed9d640 1 --2- 172.21.15.1:0/3450205396 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f50106f20 0x7f1f50074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:01:58.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.590+0000 7f1f4dd9b640 1 --2- 172.21.15.1:0/3450205396 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f501027a0 0x7f1f50074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:01:58.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.590+0000 7f1f4e59c640 1 --2- 172.21.15.1:0/3450205396 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f50101df0 0x7f1f500744f0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f1f4000e9f0 tx=0x7f1f4000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:01:58.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.590+0000 7f1f3f7fe640 1 -- 172.21.15.1:0/3450205396 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f4000cd80 con 0x7f1f50101df0 2024-09-11T22:01:58.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.590+0000 7f1f5483c640 1 -- 172.21.15.1:0/3450205396 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f501a6c10 con 0x7f1f50101df0 2024-09-11T22:01:58.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.590+0000 7f1f3f7fe640 1 -- 172.21.15.1:0/3450205396 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f40005500 con 0x7f1f50101df0 2024-09-11T22:01:58.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.590+0000 7f1f3f7fe640 1 -- 172.21.15.1:0/3450205396 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f40010690 con 0x7f1f50101df0 2024-09-11T22:01:58.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.591+0000 7f1f5483c640 1 -- 172.21.15.1:0/3450205396 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f501a7120 con 0x7f1f50101df0 2024-09-11T22:01:58.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.593+0000 7f1f56c15740 1 Processor -- start 2024-09-11T22:01:58.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.593+0000 7f1f56c15740 1 -- start start 2024-09-11T22:01:58.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.594+0000 7f1f56c15740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557529d11b40 0x557529d10090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:58.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.594+0000 7f1f3f7fe640 1 -- 172.21.15.1:0/3450205396 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f1f400046c0 con 0x7f1f50101df0 2024-09-11T22:01:58.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.594+0000 7f1f56c15740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557529d105d0 0x557529d10a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:58.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.594+0000 7f1f56c15740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557529d1b370 0x557529d1d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:58.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.594+0000 7f1f56c15740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x557529c762f0 con 0x557529d11b40 2024-09-11T22:01:58.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.594+0000 7f1f56c15740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x557529c1dcf0 con 0x557529d105d0 2024-09-11T22:01:58.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.594+0000 7f1f56c15740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x557529c722b0 con 0x557529d1b370 2024-09-11T22:01:58.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.594+0000 7f1f4ed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557529d11b40 0x557529d10090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:58.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.594+0000 7f1f4ed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557529d11b40 0x557529d10090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44604/0 (socket says 172.21.15.1:44604) 2024-09-11T22:01:58.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.594+0000 7f1f4ed9d640 1 -- 172.21.15.1:0/3730602003 learned_addr learned my addr 172.21.15.1:0/3730602003 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:01:58.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.594+0000 7f1f4dd9b640 1 --2- 172.21.15.1:0/3730602003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557529d1b370 0x557529d1d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:58.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.594+0000 7f1f4ed9d640 1 -- 172.21.15.1:0/3730602003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557529d1b370 msgr2=0x557529d1d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:58.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.594+0000 7f1f4ed9d640 1 --2- 172.21.15.1:0/3730602003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557529d1b370 0x557529d1d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:58.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.594+0000 7f1f4ed9d640 1 -- 172.21.15.1:0/3730602003 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557529d105d0 msgr2=0x557529d10a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:58.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.595+0000 7f1f4e59c640 1 --2- 172.21.15.1:0/3730602003 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557529d105d0 0x557529d10a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:58.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.595+0000 7f1f4ed9d640 1 --2- 172.21.15.1:0/3730602003 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557529d105d0 0x557529d10a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:58.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.595+0000 7f1f4ed9d640 1 -- 172.21.15.1:0/3730602003 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557529c023a0 con 0x557529d11b40 2024-09-11T22:01:58.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.595+0000 7f1f4e59c640 1 --2- 172.21.15.1:0/3730602003 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557529d105d0 0x557529d10a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:01:58.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.596+0000 7f1f4ed9d640 1 --2- 172.21.15.1:0/3730602003 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557529d11b40 0x557529d10090 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f1f48011790 tx=0x7f1f48011c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f14ab1c47892b9f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:01:58.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.596+0000 7f1f3f7fe640 1 --2- 172.21.15.1:0/3450205396 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f1f380a2d70 0x7f1f380a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:58.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.596+0000 7f1f3cff9640 1 -- 172.21.15.1:0/3730602003 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f48003c30 con 0x557529d11b40 2024-09-11T22:01:58.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.596+0000 7f1f3cff9640 1 -- 172.21.15.1:0/3730602003 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f4800fe80 con 0x557529d11b40 2024-09-11T22:01:58.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.596+0000 7f1f3cff9640 1 -- 172.21.15.1:0/3730602003 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f480193e0 con 0x557529d11b40 2024-09-11T22:01:58.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.596+0000 7f1f4e59c640 1 --2- 172.21.15.1:0/3450205396 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f1f380a2d70 0x7f1f380a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:58.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.596+0000 7f1f3f7fe640 1 -- 172.21.15.1:0/3450205396 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f1f400cf810 con 0x7f1f50101df0 2024-09-11T22:01:58.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.597+0000 7f1f56c15740 1 -- 172.21.15.1:0/3730602003 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557529d11b40 msgr2=0x557529d10090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:58.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.597+0000 7f1f56c15740 1 --2- 172.21.15.1:0/3730602003 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557529d11b40 0x557529d10090 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f1f48011790 tx=0x7f1f48011c50 comp rx=0 tx=0).stop 2024-09-11T22:01:58.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.597+0000 7f1f4e59c640 1 --2- 172.21.15.1:0/3450205396 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f1f380a2d70 0x7f1f380a5230 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1f400ccfd0 tx=0x7f1f40004a40 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:01:58.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.597+0000 7f1f56c15740 1 -- 172.21.15.1:0/3730602003 shutdown_connections 2024-09-11T22:01:58.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.597+0000 7f1f56c15740 1 --2- 172.21.15.1:0/3730602003 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557529d1b370 0x557529d1d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:58.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.597+0000 7f1f56c15740 1 --2- 172.21.15.1:0/3730602003 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557529d105d0 0x557529d10a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:58.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.597+0000 7f1f56c15740 1 --2- 172.21.15.1:0/3730602003 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557529d11b40 0x557529d10090 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:58.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.597+0000 7f1f56c15740 1 -- 172.21.15.1:0/3730602003 >> 172.21.15.1:0/3730602003 conn(0x557529cd9310 msgr2=0x557529d0cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:01:58.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.597+0000 7f1f56c15740 1 -- 172.21.15.1:0/3730602003 shutdown_connections 2024-09-11T22:01:58.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.597+0000 7f1f56c15740 1 -- 172.21.15.1:0/3730602003 wait complete. 2024-09-11T22:01:58.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.598+0000 7f1f56c15740 1 -- start start 2024-09-11T22:01:58.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.598+0000 7f1f56c15740 1 Processor -- start 2024-09-11T22:01:58.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.598+0000 7f1f56c15740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557529d105d0 0x557529d1ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:58.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.598+0000 7f1f56c15740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557529d11b40 0x557529d18040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:58.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.598+0000 7f1f56c15740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557529d1b370 0x557529d18650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:58.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.598+0000 7f1f56c15740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x557529d1e070 con 0x557529d1b370 2024-09-11T22:01:58.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.598+0000 7f1f56c15740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x557529c31060 con 0x557529d11b40 2024-09-11T22:01:58.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.598+0000 7f1f56c15740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x557529d1e1f0 con 0x557529d105d0 2024-09-11T22:01:58.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.598+0000 7f1f4ed9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557529d105d0 0x557529d1ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:58.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.598+0000 7f1f4ed9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557529d105d0 0x557529d1ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:38520/0 (socket says 172.21.15.1:38520) 2024-09-11T22:01:58.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.598+0000 7f1f4ed9d640 1 -- 172.21.15.1:0/2414139874 learned_addr learned my addr 172.21.15.1:0/2414139874 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:01:58.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.598+0000 7f1f56c15740 10 client.0 _ll_register_callbacks cb 0x5575299848c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:01:58.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.598+0000 7f1f4ed9d640 1 --2- 172.21.15.1:0/2414139874 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557529d1b370 0x557529d18650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:58.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.599+0000 7f1f4dd9b640 1 --2- 172.21.15.1:0/2414139874 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557529d11b40 0x557529d18040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:58.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.599+0000 7f1f4ed9d640 1 -- 172.21.15.1:0/2414139874 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557529d11b40 msgr2=0x557529d18040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:58.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.599+0000 7f1f4ed9d640 1 --2- 172.21.15.1:0/2414139874 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557529d11b40 0x557529d18040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:58.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.599+0000 7f1f4ed9d640 1 -- 172.21.15.1:0/2414139874 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557529d1b370 msgr2=0x557529d18650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:58.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.599+0000 7f1f4ed9d640 1 --2- 172.21.15.1:0/2414139874 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557529d1b370 0x557529d18650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:58.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.599+0000 7f1f4ed9d640 1 -- 172.21.15.1:0/2414139874 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575299e73c0 con 0x557529d105d0 2024-09-11T22:01:58.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.599+0000 7f1f4ed9d640 1 --2- 172.21.15.1:0/2414139874 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557529d105d0 0x557529d1ad90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1f48004480 tx=0x7f1f480045c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5fd73a3557019cdf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:01:58.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.599+0000 7f1f35ffb640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-11T22:01:58.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.599+0000 7f1f35ffb640 1 -- 172.21.15.1:0/2414139874 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f4802f060 con 0x557529d105d0 2024-09-11T22:01:58.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.599+0000 7f1f56c15740 1 -- 172.21.15.1:0/2414139874 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557529c83860 con 0x557529d105d0 2024-09-11T22:01:58.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.599+0000 7f1f35ffb640 1 -- 172.21.15.1:0/2414139874 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f1f48003c40 con 0x557529d105d0 2024-09-11T22:01:58.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.600+0000 7f1f35ffb640 1 -- 172.21.15.1:0/2414139874 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1f48003e40 con 0x557529d105d0 2024-09-11T22:01:58.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.600+0000 7f1f56c15740 10 client.25849 fetch_fsmap learned FSMap version 33 2024-09-11T22:01:58.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.600+0000 7f1f56c15740 1 -- 172.21.15.1:0/2414139874 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x557529d29a90 con 0x557529d105d0 2024-09-11T22:01:58.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.600+0000 7f1f35ffb640 1 -- 172.21.15.1:0/2414139874 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x557529c83860 con 0x557529d105d0 2024-09-11T22:01:58.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.600+0000 7f1f35ffb640 1 -- 172.21.15.1:0/2414139874 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7f1f48018070 con 0x557529d105d0 2024-09-11T22:01:58.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.600+0000 7f1f56c15740 10 client.25849 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T22:01:58.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.600+0000 7f1f56c15740 10 client.25849 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-11T22:01:58.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.600+0000 7f1f56c15740 20 client.25849 populate_metadata read hostname 'smithi001' 2024-09-11T22:01:58.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.600+0000 7f1f56c15740 1 --2- 172.21.15.1:0/2414139874 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x557529d2b220 0x557529d2d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:58.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.600+0000 7f1f56c15740 4 client.25849 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:01:58.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.600+0000 7f1f56c15740 1 -- 172.21.15.1:0/2414139874 --> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557529c7dd70 con 0x557529d2b220 2024-09-11T22:01:58.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.600+0000 7f1f4dd9b640 1 --2- 172.21.15.1:0/2414139874 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x557529d2b220 0x557529d2d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:58.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.601+0000 7f1f35ffb640 10 client.25849 ms_handle_connect on v2:172.21.15.161:6836/1799318656 2024-09-11T22:01:58.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.601+0000 7f1f4dd9b640 1 --2- 172.21.15.1:0/2414139874 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x557529d2b220 0x557529d2d690 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f017a720015f77a9 server_cookie=7310413a941712a in_seq=0 out_seq=0 2024-09-11T22:01:58.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.602+0000 7f1f35ffb640 1 -- 172.21.15.1:0/2414139874 <== mds.3 v2:172.21.15.161:6836/1799318656 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557529c7dd70 con 0x557529d2b220 2024-09-11T22:01:58.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.602+0000 7f1f35ffb640 10 client.25849 handle_command_reply: tid=0 2024-09-11T22:01:58.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.602+0000 7f1f56c15740 1 client.25849 shutdown 2024-09-11T22:01:58.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.602+0000 7f1f56c15740 1 -- 172.21.15.1:0/2414139874 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557529d105d0 msgr2=0x557529d1ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:58.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.602+0000 7f1f56c15740 1 --2- 172.21.15.1:0/2414139874 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557529d105d0 0x557529d1ad90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1f48004480 tx=0x7f1f480045c0 comp rx=0 tx=0).stop 2024-09-11T22:01:58.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.602+0000 7f1f56c15740 1 -- 172.21.15.1:0/2414139874 shutdown_connections 2024-09-11T22:01:58.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.602+0000 7f1f56c15740 1 --2- 172.21.15.1:0/2414139874 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x557529d2b220 0x557529d2d690 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:58.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.602+0000 7f1f56c15740 1 --2- 172.21.15.1:0/2414139874 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557529d1b370 0x557529d18650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:58.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.602+0000 7f1f56c15740 1 --2- 172.21.15.1:0/2414139874 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557529d11b40 0x557529d18040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:58.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.602+0000 7f1f56c15740 1 --2- 172.21.15.1:0/2414139874 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557529d105d0 0x557529d1ad90 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:58.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.602+0000 7f1f56c15740 1 -- 172.21.15.1:0/2414139874 >> 172.21.15.1:0/2414139874 conn(0x557529cd9310 msgr2=0x557529d12930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:01:58.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.602+0000 7f1f35ffb640 0 client.25849 ms_handle_reset on v2:172.21.15.161:6836/1799318656 2024-09-11T22:01:58.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.602+0000 7f1f56c15740 1 -- 172.21.15.1:0/2414139874 shutdown_connections 2024-09-11T22:01:58.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.602+0000 7f1f56c15740 1 -- 172.21.15.1:0/2414139874 wait complete. 2024-09-11T22:01:58.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.602+0000 7f1f56c15740 20 client.25849 trim_cache size 0 max 16384 2024-09-11T22:01:58.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.619+0000 7f1f56c15740 1 Processor -- start 2024-09-11T22:01:58.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.619+0000 7f1f56c15740 1 -- start start 2024-09-11T22:01:58.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.619+0000 7f1f56c15740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557529d105d0 0x557529d27310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:58.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.619+0000 7f1f56c15740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557529d11b40 0x557529d32f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:58.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.619+0000 7f1f56c15740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557529d1b370 0x557529d334a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:58.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.619+0000 7f1f56c15740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1f501afcb0 con 0x557529d105d0 2024-09-11T22:01:58.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.619+0000 7f1f56c15740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1f501b18e0 con 0x557529d11b40 2024-09-11T22:01:58.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.619+0000 7f1f56c15740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1f501b0380 con 0x557529d1b370 2024-09-11T22:01:58.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.619+0000 7f1f4ed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557529d105d0 0x557529d27310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:58.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.619+0000 7f1f4ed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557529d105d0 0x557529d27310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44612/0 (socket says 172.21.15.1:44612) 2024-09-11T22:01:58.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.619+0000 7f1f4ed9d640 1 -- 172.21.15.1:0/1731808718 learned_addr learned my addr 172.21.15.1:0/1731808718 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:01:58.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.619+0000 7f1f4dd9b640 1 --2- 172.21.15.1:0/1731808718 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557529d11b40 0x557529d32f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:58.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.619+0000 7f1f4ed9d640 1 --2- 172.21.15.1:0/1731808718 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557529d1b370 0x557529d334a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:58.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.620+0000 7f1f4dd9b640 1 -- 172.21.15.1:0/1731808718 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557529d1b370 msgr2=0x557529d334a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:58.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.620+0000 7f1f4dd9b640 1 --2- 172.21.15.1:0/1731808718 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557529d1b370 0x557529d334a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:58.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.620+0000 7f1f4dd9b640 1 -- 172.21.15.1:0/1731808718 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557529d105d0 msgr2=0x557529d27310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:58.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.620+0000 7f1f4dd9b640 1 --2- 172.21.15.1:0/1731808718 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557529d105d0 0x557529d27310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:58.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.620+0000 7f1f4dd9b640 1 -- 172.21.15.1:0/1731808718 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557529d0d960 con 0x557529d11b40 2024-09-11T22:01:58.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.620+0000 7f1f4ed9d640 1 --2- 172.21.15.1:0/1731808718 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557529d105d0 0x557529d27310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:01:58.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.620+0000 7f1f4dd9b640 1 --2- 172.21.15.1:0/1731808718 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557529d11b40 0x557529d32f60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1f44005940 tx=0x7f1f440058d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30438c568457894f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:01:58.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.620+0000 7f1f37fff640 1 -- 172.21.15.1:0/1731808718 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f4404a030 con 0x557529d11b40 2024-09-11T22:01:58.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.620+0000 7f1f37fff640 1 -- 172.21.15.1:0/1731808718 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f44005020 con 0x557529d11b40 2024-09-11T22:01:58.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.620+0000 7f1f37fff640 1 -- 172.21.15.1:0/1731808718 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f44035040 con 0x557529d11b40 2024-09-11T22:01:58.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.621+0000 7f1f56c15740 1 -- 172.21.15.1:0/1731808718 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557529d11b40 msgr2=0x557529d32f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:58.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.621+0000 7f1f56c15740 1 --2- 172.21.15.1:0/1731808718 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557529d11b40 0x557529d32f60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1f44005940 tx=0x7f1f440058d0 comp rx=0 tx=0).stop 2024-09-11T22:01:58.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.621+0000 7f1f56c15740 1 -- 172.21.15.1:0/1731808718 shutdown_connections 2024-09-11T22:01:58.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.621+0000 7f1f56c15740 1 --2- 172.21.15.1:0/1731808718 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557529d1b370 0x557529d334a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:58.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.621+0000 7f1f56c15740 1 --2- 172.21.15.1:0/1731808718 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557529d11b40 0x557529d32f60 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:58.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.621+0000 7f1f56c15740 1 --2- 172.21.15.1:0/1731808718 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557529d105d0 0x557529d27310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:58.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.621+0000 7f1f56c15740 1 -- 172.21.15.1:0/1731808718 >> 172.21.15.1:0/1731808718 conn(0x557529cd9310 msgr2=0x557529d132c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:01:58.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.621+0000 7f1f56c15740 1 -- 172.21.15.1:0/1731808718 shutdown_connections 2024-09-11T22:01:58.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.621+0000 7f1f56c15740 1 -- 172.21.15.1:0/1731808718 wait complete. 2024-09-11T22:01:58.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.622+0000 7f1f56c15740 1 -- start start 2024-09-11T22:01:58.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.622+0000 7f1f56c15740 1 Processor -- start 2024-09-11T22:01:58.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.622+0000 7f1f56c15740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557529d105d0 0x557529d23860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:58.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.622+0000 7f1f56c15740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557529d11b40 0x557529d23da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:58.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.622+0000 7f1f56c15740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557529d1b370 0x557529d1ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:58.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.622+0000 7f1f56c15740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1f501b1150 con 0x557529d11b40 2024-09-11T22:01:58.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.622+0000 7f1f56c15740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1f501b0a50 con 0x557529d1b370 2024-09-11T22:01:58.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.622+0000 7f1f56c15740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1f501adc90 con 0x557529d105d0 2024-09-11T22:01:58.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.622+0000 7f1f56c15740 10 client.0 _ll_register_callbacks cb 0x7f1f40005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:01:58.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.622+0000 7f1f4dd9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557529d11b40 0x557529d23da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:58.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.622+0000 7f1f4ed9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557529d105d0 0x557529d23860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:58.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.622+0000 7f1f4dd9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557529d11b40 0x557529d23da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44624/0 (socket says 172.21.15.1:44624) 2024-09-11T22:01:58.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.622+0000 7f1f4dd9b640 1 -- 172.21.15.1:0/118799192 learned_addr learned my addr 172.21.15.1:0/118799192 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:01:58.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.622+0000 7f1f4ed9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557529d105d0 0x557529d23860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:38526/0 (socket says 172.21.15.1:38526) 2024-09-11T22:01:58.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.622+0000 7f1f4ed9d640 1 --2- 172.21.15.1:0/118799192 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557529d1b370 0x557529d1ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:58.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.623+0000 7f1f4ed9d640 1 -- 172.21.15.1:0/118799192 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557529d1b370 msgr2=0x557529d1ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:58.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.623+0000 7f1f4ed9d640 1 --2- 172.21.15.1:0/118799192 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557529d1b370 0x557529d1ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:58.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.623+0000 7f1f4ed9d640 1 -- 172.21.15.1:0/118799192 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557529d11b40 msgr2=0x557529d23da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:01:58.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.623+0000 7f1f4ed9d640 1 --2- 172.21.15.1:0/118799192 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557529d11b40 0x557529d23da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:58.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.623+0000 7f1f4ed9d640 1 -- 172.21.15.1:0/118799192 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557529d1f610 con 0x557529d105d0 2024-09-11T22:01:58.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.623+0000 7f1f4dd9b640 1 --2- 172.21.15.1:0/118799192 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557529d11b40 0x557529d23da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:01:58.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.623+0000 7f1f4ed9d640 1 --2- 172.21.15.1:0/118799192 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557529d105d0 0x557529d23860 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1f480059b0 tx=0x7f1f48004480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b35aedc1dff5c53e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:01:58.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.623+0000 7f1f35ffb640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-11T22:01:58.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.623+0000 7f1f35ffb640 1 -- 172.21.15.1:0/118799192 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f4802f040 con 0x557529d105d0 2024-09-11T22:01:58.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.623+0000 7f1f35ffb640 1 -- 172.21.15.1:0/118799192 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f1f48003c40 con 0x557529d105d0 2024-09-11T22:01:58.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.623+0000 7f1f56c15740 1 -- 172.21.15.1:0/118799192 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f44005020 con 0x557529d105d0 2024-09-11T22:01:58.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.624+0000 7f1f35ffb640 1 -- 172.21.15.1:0/118799192 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1f4801a6e0 con 0x557529d105d0 2024-09-11T22:01:58.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.624+0000 7f1f56c15740 10 client.25855 fetch_fsmap learned FSMap version 33 2024-09-11T22:01:58.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.624+0000 7f1f56c15740 1 -- 172.21.15.1:0/118799192 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x557529d95be0 con 0x557529d105d0 2024-09-11T22:01:58.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.624+0000 7f1f35ffb640 1 -- 172.21.15.1:0/118799192 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f44005020 con 0x557529d105d0 2024-09-11T22:01:58.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.624+0000 7f1f35ffb640 1 -- 172.21.15.1:0/118799192 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7f1f48018070 con 0x557529d105d0 2024-09-11T22:01:58.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.624+0000 7f1f56c15740 10 client.25855 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T22:01:58.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.624+0000 7f1f56c15740 10 client.25855 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-11T22:01:58.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.624+0000 7f1f56c15740 20 client.25855 populate_metadata read hostname 'smithi001' 2024-09-11T22:01:58.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.624+0000 7f1f56c15740 1 --2- 172.21.15.1:0/118799192 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x557529d961e0 0x557529d985d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:01:58.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.624+0000 7f1f56c15740 4 client.25855 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:01:58.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.624+0000 7f1f56c15740 1 -- 172.21.15.1:0/118799192 --> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557529bfa170 con 0x557529d961e0 2024-09-11T22:01:58.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.625+0000 7f1f4dd9b640 1 --2- 172.21.15.1:0/118799192 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x557529d961e0 0x557529d985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:01:58.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.625+0000 7f1f4dd9b640 1 --2- 172.21.15.1:0/118799192 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x557529d961e0 0x557529d985d0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=20e683b778095cf1 server_cookie=3704f9cb7344130f in_seq=0 out_seq=0 2024-09-11T22:01:58.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.625+0000 7f1f35ffb640 10 client.25855 ms_handle_connect on v2:172.21.15.161:6836/1799318656 2024-09-11T22:01:58.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.626+0000 7f1f35ffb640 1 -- 172.21.15.1:0/118799192 <== mds.3 v2:172.21.15.161:6836/1799318656 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557529bfa170 con 0x557529d961e0 2024-09-11T22:01:58.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.626+0000 7f1f35ffb640 10 client.25855 handle_command_reply: tid=0 2024-09-11T22:01:58.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.626+0000 7f1f56c15740 1 client.25855 shutdown 2024-09-11T22:01:58.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.626+0000 7f1f56c15740 1 -- 172.21.15.1:0/118799192 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557529d105d0 msgr2=0x557529d23860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:58.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.626+0000 7f1f56c15740 1 --2- 172.21.15.1:0/118799192 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557529d105d0 0x557529d23860 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1f480059b0 tx=0x7f1f48004480 comp rx=0 tx=0).stop 2024-09-11T22:01:58.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.626+0000 7f1f56c15740 1 -- 172.21.15.1:0/118799192 shutdown_connections 2024-09-11T22:01:58.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.626+0000 7f1f56c15740 1 --2- 172.21.15.1:0/118799192 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x557529d961e0 0x557529d985d0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:58.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.626+0000 7f1f56c15740 1 --2- 172.21.15.1:0/118799192 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557529d1b370 0x557529d1ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:58.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.626+0000 7f1f56c15740 1 --2- 172.21.15.1:0/118799192 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557529d11b40 0x557529d23da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:58.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.626+0000 7f1f56c15740 1 --2- 172.21.15.1:0/118799192 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557529d105d0 0x557529d23860 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:58.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.626+0000 7f1f56c15740 1 -- 172.21.15.1:0/118799192 >> 172.21.15.1:0/118799192 conn(0x557529cd9310 msgr2=0x557529d12110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:01:58.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.626+0000 7f1f35ffb640 0 client.25855 ms_handle_reset on v2:172.21.15.161:6836/1799318656 2024-09-11T22:01:58.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.626+0000 7f1f56c15740 1 -- 172.21.15.1:0/118799192 shutdown_connections 2024-09-11T22:01:58.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.626+0000 7f1f56c15740 1 -- 172.21.15.1:0/118799192 wait complete. 2024-09-11T22:01:58.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.626+0000 7f1f56c15740 20 client.25855 trim_cache size 0 max 16384 2024-09-11T22:01:58.632 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-11T22:01:58.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.627+0000 7f1f5483c640 1 -- 172.21.15.1:0/3450205396 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f1f380a2d70 msgr2=0x7f1f380a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:58.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.627+0000 7f1f5483c640 1 --2- 172.21.15.1:0/3450205396 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f1f380a2d70 0x7f1f380a5230 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1f400ccfd0 tx=0x7f1f40004a40 comp rx=0 tx=0).stop 2024-09-11T22:01:58.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.627+0000 7f1f5483c640 1 -- 172.21.15.1:0/3450205396 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f50101df0 msgr2=0x7f1f500744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:01:58.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.627+0000 7f1f5483c640 1 --2- 172.21.15.1:0/3450205396 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f50101df0 0x7f1f500744f0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f1f4000e9f0 tx=0x7f1f4000edf0 comp rx=0 tx=0).stop 2024-09-11T22:01:58.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.627+0000 7f1f5483c640 1 -- 172.21.15.1:0/3450205396 shutdown_connections 2024-09-11T22:01:58.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.627+0000 7f1f5483c640 1 --2- 172.21.15.1:0/3450205396 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f1f380a2d70 0x7f1f380a5230 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:58.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.627+0000 7f1f5483c640 1 --2- 172.21.15.1:0/3450205396 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f50106f20 0x7f1f50074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:58.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.627+0000 7f1f5483c640 1 --2- 172.21.15.1:0/3450205396 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1f501027a0 0x7f1f50074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:58.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.627+0000 7f1f5483c640 1 --2- 172.21.15.1:0/3450205396 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f50101df0 0x7f1f500744f0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:01:58.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.627+0000 7f1f5483c640 1 -- 172.21.15.1:0/3450205396 >> 172.21.15.1:0/3450205396 conn(0x7f1f500fd8f0 msgr2=0x7f1f500fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:01:58.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.627+0000 7f1f5483c640 1 -- 172.21.15.1:0/3450205396 shutdown_connections 2024-09-11T22:01:58.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:01:58.628+0000 7f1f5483c640 1 -- 172.21.15.1:0/3450205396 wait complete. 2024-09-11T22:01:59.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:01:59 smithi053 ceph-mon[34908]: pgmap v147: 97 pgs: 97 active+clean; 22 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 1.47k op/s 2024-09-11T22:01:59.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:01:59 smithi161 ceph-mon[35615]: pgmap v147: 97 pgs: 97 active+clean; 22 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 1.47k op/s 2024-09-11T22:01:59.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:01:59 smithi001 ceph-mon[30935]: pgmap v147: 97 pgs: 97 active+clean; 22 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 1.47k op/s 2024-09-11T22:01:59.559 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T22:01:59.836 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:02:00.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.472+0000 7fe53b173640 1 -- 172.21.15.1:0/3327358885 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe534101df0 msgr2=0x7fe5341021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:00.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.472+0000 7fe53b173640 1 --2- 172.21.15.1:0/3327358885 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe534101df0 0x7fe5341021d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe528009900 tx=0x7fe52802a410 comp rx=0 tx=0).stop 2024-09-11T22:02:00.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.472+0000 7fe53b173640 1 -- 172.21.15.1:0/3327358885 shutdown_connections 2024-09-11T22:02:00.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.472+0000 7fe53b173640 1 --2- 172.21.15.1:0/3327358885 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe534106f20 0x7fe53410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:00.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.472+0000 7fe53b173640 1 --2- 172.21.15.1:0/3327358885 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe5341027a0 0x7fe5341067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:00.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.472+0000 7fe53b173640 1 --2- 172.21.15.1:0/3327358885 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe534101df0 0x7fe5341021d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:00.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.472+0000 7fe53b173640 1 -- 172.21.15.1:0/3327358885 >> 172.21.15.1:0/3327358885 conn(0x7fe5340fd8f0 msgr2=0x7fe5340ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:00.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.472+0000 7fe53b173640 1 -- 172.21.15.1:0/3327358885 shutdown_connections 2024-09-11T22:02:00.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.472+0000 7fe53b173640 1 -- 172.21.15.1:0/3327358885 wait complete. 2024-09-11T22:02:00.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.473+0000 7fe53b173640 1 Processor -- start 2024-09-11T22:02:00.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.473+0000 7fe53b173640 1 -- start start 2024-09-11T22:02:00.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.473+0000 7fe53b173640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe534101df0 0x7fe5340744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:00.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.473+0000 7fe53b173640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe5341027a0 0x7fe534074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:00.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.473+0000 7fe53b173640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe534106f20 0x7fe534074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:00.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.473+0000 7fe53b173640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe534111f80 con 0x7fe5341027a0 2024-09-11T22:02:00.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.473+0000 7fe53b173640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe534111e00 con 0x7fe534106f20 2024-09-11T22:02:00.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.473+0000 7fe53b173640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe534112100 con 0x7fe534101df0 2024-09-11T22:02:00.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.474+0000 7fe539710640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe534106f20 0x7fe534074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:00.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.474+0000 7fe533fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe5341027a0 0x7fe534074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:00.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.474+0000 7fe539710640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe534106f20 0x7fe534074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:50462/0 (socket says 172.21.15.1:50462) 2024-09-11T22:02:00.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.474+0000 7fe539710640 1 -- 172.21.15.1:0/667684887 learned_addr learned my addr 172.21.15.1:0/667684887 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:00.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.474+0000 7fe538f0f640 1 --2- 172.21.15.1:0/667684887 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe534101df0 0x7fe5340744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:00.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.474+0000 7fe533fff640 1 -- 172.21.15.1:0/667684887 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe534101df0 msgr2=0x7fe5340744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:00.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.474+0000 7fe533fff640 1 --2- 172.21.15.1:0/667684887 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe534101df0 0x7fe5340744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:00.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.474+0000 7fe533fff640 1 -- 172.21.15.1:0/667684887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe534106f20 msgr2=0x7fe534074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:00.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.474+0000 7fe533fff640 1 --2- 172.21.15.1:0/667684887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe534106f20 0x7fe534074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:00.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.474+0000 7fe533fff640 1 -- 172.21.15.1:0/667684887 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5341a69a0 con 0x7fe5341027a0 2024-09-11T22:02:00.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.474+0000 7fe539710640 1 --2- 172.21.15.1:0/667684887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe534106f20 0x7fe534074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:02:00.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.474+0000 7fe533fff640 1 --2- 172.21.15.1:0/667684887 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe5341027a0 0x7fe534074a30 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fe52000e9f0 tx=0x7fe52000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:00.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.475+0000 7fe531ffb640 1 -- 172.21.15.1:0/667684887 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52000ccc0 con 0x7fe5341027a0 2024-09-11T22:02:00.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.475+0000 7fe531ffb640 1 -- 172.21.15.1:0/667684887 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe520010910 con 0x7fe5341027a0 2024-09-11T22:02:00.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.475+0000 7fe53b173640 1 -- 172.21.15.1:0/667684887 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5341a6c90 con 0x7fe5341027a0 2024-09-11T22:02:00.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.475+0000 7fe531ffb640 1 -- 172.21.15.1:0/667684887 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe520018960 con 0x7fe5341027a0 2024-09-11T22:02:00.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.475+0000 7fe53b173640 1 -- 172.21.15.1:0/667684887 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5341a71c0 con 0x7fe5341027a0 2024-09-11T22:02:00.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.478+0000 7fe531ffb640 1 -- 172.21.15.1:0/667684887 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7fe520010ac0 con 0x7fe5341027a0 2024-09-11T22:02:00.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.478+0000 7fe53d54c740 1 Processor -- start 2024-09-11T22:02:00.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.479+0000 7fe53d54c740 1 -- start start 2024-09-11T22:02:00.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.479+0000 7fe531ffb640 1 --2- 172.21.15.1:0/667684887 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fe5240a2d70 0x7fe5240a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:00.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.479+0000 7fe53d54c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f8ea0fb40 0x555f8ea0e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:00.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.479+0000 7fe53d54c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555f8ea0e5d0 0x555f8ea0ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:00.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.479+0000 7fe53d54c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f8ea19370 0x555f8ea1b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:00.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.479+0000 7fe53d54c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x555f8e9742f0 con 0x555f8ea0fb40 2024-09-11T22:02:00.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.479+0000 7fe53d54c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x555f8e91bcf0 con 0x555f8ea0e5d0 2024-09-11T22:02:00.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.479+0000 7fe53d54c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x555f8e9702b0 con 0x555f8ea19370 2024-09-11T22:02:00.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.479+0000 7fe533fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555f8ea0e5d0 0x555f8ea0ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:00.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.479+0000 7fe538f0f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f8ea0fb40 0x555f8ea0e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:00.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.479+0000 7fe533fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555f8ea0e5d0 0x555f8ea0ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:50464/0 (socket says 172.21.15.1:50464) 2024-09-11T22:02:00.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.479+0000 7fe533fff640 1 -- 172.21.15.1:0/2496325324 learned_addr learned my addr 172.21.15.1:0/2496325324 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:00.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.479+0000 7fe531ffb640 1 -- 172.21.15.1:0/667684887 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7fe5200cf800 con 0x7fe5341027a0 2024-09-11T22:02:00.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.479+0000 7fe539710640 1 --2- 172.21.15.1:0/667684887 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fe5240a2d70 0x7fe5240a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:00.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.479+0000 7fe539710640 1 --2- 172.21.15.1:0/2496325324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f8ea19370 0x555f8ea1b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:00.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.479+0000 7fe533fff640 1 -- 172.21.15.1:0/2496325324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f8ea19370 msgr2=0x555f8ea1b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:00.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.479+0000 7fe533fff640 1 --2- 172.21.15.1:0/2496325324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f8ea19370 0x555f8ea1b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:00.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.479+0000 7fe533fff640 1 -- 172.21.15.1:0/2496325324 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f8ea0fb40 msgr2=0x555f8ea0e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:00.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.479+0000 7fe533fff640 1 --2- 172.21.15.1:0/2496325324 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f8ea0fb40 0x555f8ea0e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:00.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.480+0000 7fe533fff640 1 -- 172.21.15.1:0/2496325324 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f8e9003a0 con 0x555f8ea0e5d0 2024-09-11T22:02:00.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.480+0000 7fe538f0f640 1 --2- 172.21.15.1:0/2496325324 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f8ea0fb40 0x555f8ea0e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:02:00.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.480+0000 7fe533fff640 1 --2- 172.21.15.1:0/2496325324 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555f8ea0e5d0 0x555f8ea0ea10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe5200ccf10 tx=0x7fe52001f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e8422050f9c1e0d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:00.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.480+0000 7fe51effd640 1 -- 172.21.15.1:0/2496325324 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe520026040 con 0x555f8ea0e5d0 2024-09-11T22:02:00.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.480+0000 7fe539710640 1 --2- 172.21.15.1:0/667684887 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fe5240a2d70 0x7fe5240a5230 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x555f8ea0f070 tx=0x7fe52c009960 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:00.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.480+0000 7fe51effd640 1 -- 172.21.15.1:0/2496325324 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe520078f40 con 0x555f8ea0e5d0 2024-09-11T22:02:00.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.480+0000 7fe51effd640 1 -- 172.21.15.1:0/2496325324 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5200a48f0 con 0x555f8ea0e5d0 2024-09-11T22:02:00.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.481+0000 7fe53d54c740 1 -- 172.21.15.1:0/2496325324 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555f8ea0e5d0 msgr2=0x555f8ea0ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:00.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.481+0000 7fe53d54c740 1 --2- 172.21.15.1:0/2496325324 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555f8ea0e5d0 0x555f8ea0ea10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe5200ccf10 tx=0x7fe52001f000 comp rx=0 tx=0).stop 2024-09-11T22:02:00.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.481+0000 7fe53d54c740 1 -- 172.21.15.1:0/2496325324 shutdown_connections 2024-09-11T22:02:00.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.481+0000 7fe53d54c740 1 --2- 172.21.15.1:0/2496325324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f8ea19370 0x555f8ea1b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:00.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.481+0000 7fe53d54c740 1 --2- 172.21.15.1:0/2496325324 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555f8ea0e5d0 0x555f8ea0ea10 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:00.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.481+0000 7fe53d54c740 1 --2- 172.21.15.1:0/2496325324 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f8ea0fb40 0x555f8ea0e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:00.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.481+0000 7fe53d54c740 1 -- 172.21.15.1:0/2496325324 >> 172.21.15.1:0/2496325324 conn(0x555f8e9d7310 msgr2=0x555f8ea0ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:00.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.481+0000 7fe53d54c740 1 -- 172.21.15.1:0/2496325324 shutdown_connections 2024-09-11T22:02:00.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.481+0000 7fe53d54c740 1 -- 172.21.15.1:0/2496325324 wait complete. 2024-09-11T22:02:00.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.481+0000 7fe53d54c740 1 -- start start 2024-09-11T22:02:00.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.482+0000 7fe53d54c740 1 Processor -- start 2024-09-11T22:02:00.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.482+0000 7fe53d54c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555f8ea0e5d0 0x555f8ea18d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:00.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.482+0000 7fe53d54c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f8ea0fb40 0x555f8ea16040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:00.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.482+0000 7fe538f0f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555f8ea0e5d0 0x555f8ea18d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:00.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.482+0000 7fe538f0f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555f8ea0e5d0 0x555f8ea18d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:50466/0 (socket says 172.21.15.1:50466) 2024-09-11T22:02:00.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.482+0000 7fe53d54c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f8ea19370 0x555f8ea16650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:00.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.483+0000 7fe533fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f8ea0fb40 0x555f8ea16040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:00.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.483+0000 7fe533fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f8ea0fb40 0x555f8ea16040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:38552/0 (socket says 172.21.15.1:38552) 2024-09-11T22:02:00.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.483+0000 7fe53d54c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x555f8ea1c010 con 0x555f8ea19370 2024-09-11T22:02:00.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.483+0000 7fe53d54c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x555f8e92f060 con 0x555f8ea0e5d0 2024-09-11T22:02:00.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.483+0000 7fe53d54c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x555f8ea1c190 con 0x555f8ea0fb40 2024-09-11T22:02:00.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.483+0000 7fe539710640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f8ea19370 0x555f8ea16650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:00.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.483+0000 7fe538f0f640 1 -- 172.21.15.1:0/3029156425 learned_addr learned my addr 172.21.15.1:0/3029156425 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:00.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.483+0000 7fe533fff640 1 -- 172.21.15.1:0/3029156425 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555f8ea0e5d0 msgr2=0x555f8ea18d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:00.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.483+0000 7fe533fff640 1 --2- 172.21.15.1:0/3029156425 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555f8ea0e5d0 0x555f8ea18d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:00.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.483+0000 7fe533fff640 1 -- 172.21.15.1:0/3029156425 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f8ea19370 msgr2=0x555f8ea16650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:00.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.483+0000 7fe53d54c740 10 client.0 _ll_register_callbacks cb 0x555f8e6828c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:02:00.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.483+0000 7fe533fff640 1 --2- 172.21.15.1:0/3029156425 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f8ea19370 0x555f8ea16650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:00.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.483+0000 7fe533fff640 1 -- 172.21.15.1:0/3029156425 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f8e6e53c0 con 0x555f8ea0fb40 2024-09-11T22:02:00.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.483+0000 7fe53d54c740 1 -- 172.21.15.1:0/3029156425 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe520078f40 con 0x555f8ea0fb40 2024-09-11T22:02:00.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.483+0000 7fe539710640 1 --2- 172.21.15.1:0/3029156425 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f8ea19370 0x555f8ea16650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:02:00.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.484+0000 7fe533fff640 1 --2- 172.21.15.1:0/3029156425 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f8ea0fb40 0x555f8ea16040 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe520098190 tx=0x7fe52009b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5294cb6283178d47 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:00.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.484+0000 7fe513fff640 10 client.25864 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-11T22:02:00.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.484+0000 7fe513fff640 1 -- 172.21.15.1:0/3029156425 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5200cd340 con 0x555f8ea0fb40 2024-09-11T22:02:00.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.484+0000 7fe513fff640 1 -- 172.21.15.1:0/3029156425 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7fe520076600 con 0x555f8ea0fb40 2024-09-11T22:02:00.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.484+0000 7fe513fff640 1 -- 172.21.15.1:0/3029156425 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe5200a48f0 con 0x555f8ea0fb40 2024-09-11T22:02:00.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.484+0000 7fe53d54c740 10 client.25864 fetch_fsmap learned FSMap version 33 2024-09-11T22:02:00.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.484+0000 7fe53d54c740 1 -- 172.21.15.1:0/3029156425 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({fsmap=33}) -- 0x555f8ea1c010 con 0x555f8ea0fb40 2024-09-11T22:02:00.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.484+0000 7fe513fff640 1 -- 172.21.15.1:0/3029156425 <== mon.1 v2:172.21.15.161:3300/0 4 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7fe52009b180 con 0x555f8ea0fb40 2024-09-11T22:02:00.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.485+0000 7fe53d54c740 10 client.25864 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T22:02:00.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.485+0000 7fe53d54c740 10 client.25864 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-11T22:02:00.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.485+0000 7fe53d54c740 20 client.25864 populate_metadata read hostname 'smithi001' 2024-09-11T22:02:00.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.485+0000 7fe53d54c740 1 --2- 172.21.15.1:0/3029156425 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x555f8ea31340 0x555f8ea33730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:00.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.485+0000 7fe53d54c740 4 client.25864 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:02:00.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.485+0000 7fe53d54c740 1 -- 172.21.15.1:0/3029156425 --> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f8e981860 con 0x555f8ea31340 2024-09-11T22:02:00.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.485+0000 7fe538f0f640 1 --2- 172.21.15.1:0/3029156425 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x555f8ea31340 0x555f8ea33730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:00.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.485+0000 7fe513fff640 10 client.25864 ms_handle_connect on v2:172.21.15.53:6836/3457131830 2024-09-11T22:02:00.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.485+0000 7fe538f0f640 1 --2- 172.21.15.1:0/3029156425 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x555f8ea31340 0x555f8ea33730 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6541ca446f873569 server_cookie=2b713c8f80b99fd7 in_seq=0 out_seq=0 2024-09-11T22:02:00.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.486+0000 7fe513fff640 1 -- 172.21.15.1:0/3029156425 <== mds.2 v2:172.21.15.53:6836/3457131830 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555f8e981860 con 0x555f8ea31340 2024-09-11T22:02:00.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.486+0000 7fe513fff640 10 client.25864 handle_command_reply: tid=0 2024-09-11T22:02:00.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.486+0000 7fe53d54c740 1 client.25864 shutdown 2024-09-11T22:02:00.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.486+0000 7fe53d54c740 1 -- 172.21.15.1:0/3029156425 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f8ea0fb40 msgr2=0x555f8ea16040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:00.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.486+0000 7fe53d54c740 1 --2- 172.21.15.1:0/3029156425 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f8ea0fb40 0x555f8ea16040 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe520098190 tx=0x7fe52009b790 comp rx=0 tx=0).stop 2024-09-11T22:02:00.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.486+0000 7fe53d54c740 1 -- 172.21.15.1:0/3029156425 shutdown_connections 2024-09-11T22:02:00.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.486+0000 7fe53d54c740 1 --2- 172.21.15.1:0/3029156425 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x555f8ea31340 0x555f8ea33730 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:00.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.486+0000 7fe53d54c740 1 --2- 172.21.15.1:0/3029156425 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f8ea19370 0x555f8ea16650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:00.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.486+0000 7fe53d54c740 1 --2- 172.21.15.1:0/3029156425 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f8ea0fb40 0x555f8ea16040 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:00.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.486+0000 7fe513fff640 0 client.25864 ms_handle_reset on v2:172.21.15.53:6836/3457131830 2024-09-11T22:02:00.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.486+0000 7fe53d54c740 1 --2- 172.21.15.1:0/3029156425 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555f8ea0e5d0 0x555f8ea18d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:00.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.486+0000 7fe53d54c740 1 -- 172.21.15.1:0/3029156425 >> 172.21.15.1:0/3029156425 conn(0x555f8e9d7310 msgr2=0x555f8ea109c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:00.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.487+0000 7fe53d54c740 1 -- 172.21.15.1:0/3029156425 shutdown_connections 2024-09-11T22:02:00.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.487+0000 7fe53d54c740 1 -- 172.21.15.1:0/3029156425 wait complete. 2024-09-11T22:02:00.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.487+0000 7fe53d54c740 20 client.25864 trim_cache size 0 max 16384 2024-09-11T22:02:00.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.504+0000 7fe53d54c740 1 Processor -- start 2024-09-11T22:02:00.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.504+0000 7fe53d54c740 1 -- start start 2024-09-11T22:02:00.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.504+0000 7fe53d54c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f8ea0e5d0 0x555f8ea0f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:00.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.504+0000 7fe53d54c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f8ea0fb40 0x555f8ea2dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:00.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.504+0000 7fe53d54c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555f8ea19370 0x555f8ea38d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:00.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.504+0000 7fe53d54c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe5341ab880 con 0x555f8ea0fb40 2024-09-11T22:02:00.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.504+0000 7fe53d54c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe5341ad4b0 con 0x555f8ea19370 2024-09-11T22:02:00.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.504+0000 7fe53d54c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe5341abf50 con 0x555f8ea0e5d0 2024-09-11T22:02:00.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.504+0000 7fe533fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f8ea0fb40 0x555f8ea2dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:00.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.504+0000 7fe533fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f8ea0fb40 0x555f8ea2dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44666/0 (socket says 172.21.15.1:44666) 2024-09-11T22:02:00.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.504+0000 7fe533fff640 1 -- 172.21.15.1:0/226567726 learned_addr learned my addr 172.21.15.1:0/226567726 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:00.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.504+0000 7fe538f0f640 1 --2- 172.21.15.1:0/226567726 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f8ea0e5d0 0x555f8ea0f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:00.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.504+0000 7fe539710640 1 --2- 172.21.15.1:0/226567726 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555f8ea19370 0x555f8ea38d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:00.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.505+0000 7fe533fff640 1 -- 172.21.15.1:0/226567726 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f8ea0e5d0 msgr2=0x555f8ea0f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:00.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.505+0000 7fe533fff640 1 --2- 172.21.15.1:0/226567726 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f8ea0e5d0 0x555f8ea0f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:00.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.505+0000 7fe533fff640 1 -- 172.21.15.1:0/226567726 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555f8ea19370 msgr2=0x555f8ea38d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:00.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.505+0000 7fe533fff640 1 --2- 172.21.15.1:0/226567726 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555f8ea19370 0x555f8ea38d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:00.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.505+0000 7fe533fff640 1 -- 172.21.15.1:0/226567726 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f8ea11120 con 0x555f8ea0fb40 2024-09-11T22:02:00.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.505+0000 7fe538f0f640 1 --2- 172.21.15.1:0/226567726 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f8ea0e5d0 0x555f8ea0f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:02:00.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.505+0000 7fe533fff640 1 --2- 172.21.15.1:0/226567726 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f8ea0fb40 0x555f8ea2dba0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fe520099fe0 tx=0x7fe520023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4de2741f64e30618 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:00.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.505+0000 7fe51e7fc640 1 -- 172.21.15.1:0/226567726 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52009b690 con 0x555f8ea0fb40 2024-09-11T22:02:00.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.505+0000 7fe51e7fc640 1 -- 172.21.15.1:0/226567726 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe520078f40 con 0x555f8ea0fb40 2024-09-11T22:02:00.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.505+0000 7fe51e7fc640 1 -- 172.21.15.1:0/226567726 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52009ae50 con 0x555f8ea0fb40 2024-09-11T22:02:00.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.506+0000 7fe53d54c740 1 -- 172.21.15.1:0/226567726 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f8ea0fb40 msgr2=0x555f8ea2dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:00.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.506+0000 7fe53d54c740 1 --2- 172.21.15.1:0/226567726 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f8ea0fb40 0x555f8ea2dba0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fe520099fe0 tx=0x7fe520023000 comp rx=0 tx=0).stop 2024-09-11T22:02:00.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.506+0000 7fe53d54c740 1 -- 172.21.15.1:0/226567726 shutdown_connections 2024-09-11T22:02:00.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.506+0000 7fe53d54c740 1 --2- 172.21.15.1:0/226567726 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555f8ea19370 0x555f8ea38d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:00.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.506+0000 7fe53d54c740 1 --2- 172.21.15.1:0/226567726 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f8ea0fb40 0x555f8ea2dba0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:00.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.506+0000 7fe53d54c740 1 --2- 172.21.15.1:0/226567726 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f8ea0e5d0 0x555f8ea0f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:00.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.506+0000 7fe53d54c740 1 -- 172.21.15.1:0/226567726 >> 172.21.15.1:0/226567726 conn(0x555f8e9d7310 msgr2=0x555f8ea15130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:00.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.506+0000 7fe53d54c740 1 -- 172.21.15.1:0/226567726 shutdown_connections 2024-09-11T22:02:00.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.506+0000 7fe53d54c740 1 -- 172.21.15.1:0/226567726 wait complete. 2024-09-11T22:02:00.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.506+0000 7fe53d54c740 1 -- start start 2024-09-11T22:02:00.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.507+0000 7fe53d54c740 1 Processor -- start 2024-09-11T22:02:00.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.507+0000 7fe53d54c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f8ea0e5d0 0x555f8ea2a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:00.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.507+0000 7fe53d54c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f8ea0fb40 0x555f8ea25980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:00.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.507+0000 7fe53d54c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555f8ea19370 0x555f8ea260c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:00.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.507+0000 7fe533fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f8ea0fb40 0x555f8ea25980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:00.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.507+0000 7fe533fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f8ea0fb40 0x555f8ea25980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44670/0 (socket says 172.21.15.1:44670) 2024-09-11T22:02:00.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.507+0000 7fe533fff640 1 -- 172.21.15.1:0/2345085140 learned_addr learned my addr 172.21.15.1:0/2345085140 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:00.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.507+0000 7fe53d54c740 1 -- 172.21.15.1:0/2345085140 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe5341acd20 con 0x555f8ea0fb40 2024-09-11T22:02:00.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.507+0000 7fe53d54c740 1 -- 172.21.15.1:0/2345085140 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe5341ac620 con 0x555f8ea19370 2024-09-11T22:02:00.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.507+0000 7fe53d54c740 1 -- 172.21.15.1:0/2345085140 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe5341a9860 con 0x555f8ea0e5d0 2024-09-11T22:02:00.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.507+0000 7fe538f0f640 1 --2- 172.21.15.1:0/2345085140 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f8ea0e5d0 0x555f8ea2a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:00.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.507+0000 7fe539710640 1 --2- 172.21.15.1:0/2345085140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555f8ea19370 0x555f8ea260c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:00.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.508+0000 7fe53d54c740 10 client.0 _ll_register_callbacks cb 0x7fe520098060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:02:00.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.508+0000 7fe533fff640 1 -- 172.21.15.1:0/2345085140 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f8ea0e5d0 msgr2=0x555f8ea2a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:00.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.508+0000 7fe533fff640 1 --2- 172.21.15.1:0/2345085140 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f8ea0e5d0 0x555f8ea2a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:00.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.508+0000 7fe533fff640 1 -- 172.21.15.1:0/2345085140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555f8ea19370 msgr2=0x555f8ea260c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:00.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.508+0000 7fe533fff640 1 --2- 172.21.15.1:0/2345085140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555f8ea19370 0x555f8ea260c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:00.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.508+0000 7fe533fff640 1 -- 172.21.15.1:0/2345085140 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f8ea9ad00 con 0x555f8ea0fb40 2024-09-11T22:02:00.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.508+0000 7fe539710640 1 --2- 172.21.15.1:0/2345085140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555f8ea19370 0x555f8ea260c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:02:00.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.508+0000 7fe538f0f640 1 --2- 172.21.15.1:0/2345085140 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f8ea0e5d0 0x555f8ea2a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:02:00.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.508+0000 7fe533fff640 1 --2- 172.21.15.1:0/2345085140 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f8ea0fb40 0x555f8ea25980 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fe520098700 tx=0x7fe520097c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=949b69f77c28ac05 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:00.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.508+0000 7fe513fff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:02:00.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.508+0000 7fe513fff640 1 -- 172.21.15.1:0/2345085140 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe520097d00 con 0x555f8ea0fb40 2024-09-11T22:02:00.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.508+0000 7fe513fff640 1 -- 172.21.15.1:0/2345085140 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7fe52007ab50 con 0x555f8ea0fb40 2024-09-11T22:02:00.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.508+0000 7fe53d54c740 1 -- 172.21.15.1:0/2345085140 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe520078f40 con 0x555f8ea0fb40 2024-09-11T22:02:00.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.509+0000 7fe513fff640 1 -- 172.21.15.1:0/2345085140 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe5200a0030 con 0x555f8ea0fb40 2024-09-11T22:02:00.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.509+0000 7fe53d54c740 10 client.16350 fetch_fsmap learned FSMap version 33 2024-09-11T22:02:00.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.509+0000 7fe53d54c740 1 -- 172.21.15.1:0/2345085140 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x555f8ea9ad00 con 0x555f8ea0fb40 2024-09-11T22:02:00.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.509+0000 7fe513fff640 1 -- 172.21.15.1:0/2345085140 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe520078f40 con 0x555f8ea0fb40 2024-09-11T22:02:00.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.509+0000 7fe513fff640 1 -- 172.21.15.1:0/2345085140 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7fe5200a11c0 con 0x555f8ea0fb40 2024-09-11T22:02:00.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.509+0000 7fe53d54c740 10 client.16350 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T22:02:00.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.509+0000 7fe53d54c740 10 client.16350 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-11T22:02:00.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.509+0000 7fe53d54c740 20 client.16350 populate_metadata read hostname 'smithi001' 2024-09-11T22:02:00.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.509+0000 7fe53d54c740 1 --2- 172.21.15.1:0/2345085140 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x555f8ea9c190 0x555f8ea9e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:00.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.509+0000 7fe53d54c740 4 client.16350 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:02:00.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.509+0000 7fe53d54c740 1 -- 172.21.15.1:0/2345085140 --> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555f8e97bd70 con 0x555f8ea9c190 2024-09-11T22:02:00.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.510+0000 7fe538f0f640 1 --2- 172.21.15.1:0/2345085140 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x555f8ea9c190 0x555f8ea9e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:00.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.510+0000 7fe513fff640 10 client.16350 ms_handle_connect on v2:172.21.15.53:6836/3457131830 2024-09-11T22:02:00.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.510+0000 7fe538f0f640 1 --2- 172.21.15.1:0/2345085140 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x555f8ea9c190 0x555f8ea9e580 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=57a394f30a82c7bc server_cookie=529994ac777a9ab0 in_seq=0 out_seq=0 2024-09-11T22:02:00.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.510+0000 7fe513fff640 1 -- 172.21.15.1:0/2345085140 <== mds.2 v2:172.21.15.53:6836/3457131830 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555f8e97bd70 con 0x555f8ea9c190 2024-09-11T22:02:00.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.510+0000 7fe513fff640 10 client.16350 handle_command_reply: tid=0 2024-09-11T22:02:00.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.510+0000 7fe53d54c740 1 client.16350 shutdown 2024-09-11T22:02:00.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.510+0000 7fe53d54c740 1 -- 172.21.15.1:0/2345085140 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f8ea0fb40 msgr2=0x555f8ea25980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:00.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.510+0000 7fe53d54c740 1 --2- 172.21.15.1:0/2345085140 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f8ea0fb40 0x555f8ea25980 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fe520098700 tx=0x7fe520097c90 comp rx=0 tx=0).stop 2024-09-11T22:02:00.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.510+0000 7fe53d54c740 1 -- 172.21.15.1:0/2345085140 shutdown_connections 2024-09-11T22:02:00.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.510+0000 7fe53d54c740 1 --2- 172.21.15.1:0/2345085140 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x555f8ea9c190 0x555f8ea9e580 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:00.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.510+0000 7fe513fff640 0 client.16350 ms_handle_reset on v2:172.21.15.53:6836/3457131830 2024-09-11T22:02:00.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.511+0000 7fe53d54c740 1 --2- 172.21.15.1:0/2345085140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555f8ea19370 0x555f8ea260c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:00.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.511+0000 7fe53d54c740 1 --2- 172.21.15.1:0/2345085140 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f8ea0fb40 0x555f8ea25980 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:00.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.511+0000 7fe53d54c740 1 --2- 172.21.15.1:0/2345085140 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x555f8ea0e5d0 0x555f8ea2a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:00.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.511+0000 7fe53d54c740 1 -- 172.21.15.1:0/2345085140 >> 172.21.15.1:0/2345085140 conn(0x555f8e9d7310 msgr2=0x555f8ea178a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:00.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.511+0000 7fe53d54c740 1 -- 172.21.15.1:0/2345085140 shutdown_connections 2024-09-11T22:02:00.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.511+0000 7fe53d54c740 1 -- 172.21.15.1:0/2345085140 wait complete. 2024-09-11T22:02:00.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.511+0000 7fe53d54c740 20 client.16350 trim_cache size 0 max 16384 2024-09-11T22:02:00.522 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-11T22:02:00.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.512+0000 7fe53b173640 1 -- 172.21.15.1:0/667684887 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fe5240a2d70 msgr2=0x7fe5240a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:00.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.512+0000 7fe53b173640 1 --2- 172.21.15.1:0/667684887 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fe5240a2d70 0x7fe5240a5230 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x555f8ea0f070 tx=0x7fe52c009960 comp rx=0 tx=0).stop 2024-09-11T22:02:00.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.512+0000 7fe53b173640 1 -- 172.21.15.1:0/667684887 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe5341027a0 msgr2=0x7fe534074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:00.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.512+0000 7fe53b173640 1 --2- 172.21.15.1:0/667684887 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe5341027a0 0x7fe534074a30 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fe52000e9f0 tx=0x7fe52000eeb0 comp rx=0 tx=0).stop 2024-09-11T22:02:00.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.512+0000 7fe53b173640 1 -- 172.21.15.1:0/667684887 shutdown_connections 2024-09-11T22:02:00.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.512+0000 7fe53b173640 1 --2- 172.21.15.1:0/667684887 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fe5240a2d70 0x7fe5240a5230 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:00.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.512+0000 7fe53b173640 1 --2- 172.21.15.1:0/667684887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe534106f20 0x7fe534074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:00.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.512+0000 7fe53b173640 1 --2- 172.21.15.1:0/667684887 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe5341027a0 0x7fe534074a30 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:00.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.512+0000 7fe53b173640 1 --2- 172.21.15.1:0/667684887 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe534101df0 0x7fe5340744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:00.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.512+0000 7fe53b173640 1 -- 172.21.15.1:0/667684887 >> 172.21.15.1:0/667684887 conn(0x7fe5340fd8f0 msgr2=0x7fe5340feda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:00.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.512+0000 7fe53b173640 1 -- 172.21.15.1:0/667684887 shutdown_connections 2024-09-11T22:02:00.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:00.512+0000 7fe53b173640 1 -- 172.21.15.1:0/667684887 wait complete. 2024-09-11T22:02:01.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:01 smithi001 ceph-mon[30935]: pgmap v148: 97 pgs: 97 active+clean; 39 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.9 MiB/s wr, 1.50k op/s 2024-09-11T22:02:01.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:01 smithi053 ceph-mon[34908]: pgmap v148: 97 pgs: 97 active+clean; 39 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.9 MiB/s wr, 1.50k op/s 2024-09-11T22:02:01.317 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T22:02:01.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:01 smithi161 ceph-mon[35615]: pgmap v148: 97 pgs: 97 active+clean; 39 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.9 MiB/s wr, 1.50k op/s 2024-09-11T22:02:01.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.500+0000 7f046e037640 1 -- 172.21.15.1:0/2304336343 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04681011c0 msgr2=0x7f0468107970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:01.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.500+0000 7f046e037640 1 --2- 172.21.15.1:0/2304336343 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04681011c0 0x7f0468107970 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f0450009930 tx=0x7f045002a420 comp rx=0 tx=0).stop 2024-09-11T22:02:01.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.500+0000 7f046e037640 1 -- 172.21.15.1:0/2304336343 shutdown_connections 2024-09-11T22:02:01.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.500+0000 7f046e037640 1 --2- 172.21.15.1:0/2304336343 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04681080d0 0x7f046810ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:01.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.500+0000 7f046e037640 1 --2- 172.21.15.1:0/2304336343 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04681011c0 0x7f0468107970 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:01.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.500+0000 7f046e037640 1 --2- 172.21.15.1:0/2304336343 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0468100810 0x7f0468100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:01.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.500+0000 7f046e037640 1 -- 172.21.15.1:0/2304336343 >> 172.21.15.1:0/2304336343 conn(0x7f0468075eb0 msgr2=0x7f04680762c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:01.505 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.500+0000 7f046e037640 1 -- 172.21.15.1:0/2304336343 shutdown_connections 2024-09-11T22:02:01.505 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.500+0000 7f046e037640 1 -- 172.21.15.1:0/2304336343 wait complete. 2024-09-11T22:02:01.505 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.500+0000 7f046e037640 1 Processor -- start 2024-09-11T22:02:01.505 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.501+0000 7f046e037640 1 -- start start 2024-09-11T22:02:01.505 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.501+0000 7f046e037640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0468100810 0x7f0468101780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:01.505 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.501+0000 7f046e037640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04681080d0 0x7f0468101cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:01.505 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.501+0000 7f046e037640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0468102200 0x7f04681a6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:01.505 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.501+0000 7f046e037640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0468111ef0 con 0x7f04681080d0 2024-09-11T22:02:01.505 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.501+0000 7f046e037640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0468111d70 con 0x7f0468102200 2024-09-11T22:02:01.505 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.501+0000 7f046e037640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0468112070 con 0x7f0468100810 2024-09-11T22:02:01.505 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.501+0000 7f045ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04681080d0 0x7f0468101cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:01.505 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.501+0000 7f045ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04681080d0 0x7f0468101cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44682/0 (socket says 172.21.15.1:44682) 2024-09-11T22:02:01.508 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.501+0000 7f045ffff640 1 -- 172.21.15.1:0/2978622010 learned_addr learned my addr 172.21.15.1:0/2978622010 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:01.509 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.501+0000 7f0467fff640 1 --2- 172.21.15.1:0/2978622010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0468102200 0x7f04681a6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:01.509 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.501+0000 7f04677fe640 1 --2- 172.21.15.1:0/2978622010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0468100810 0x7f0468101780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:01.509 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.501+0000 7f045ffff640 1 -- 172.21.15.1:0/2978622010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0468100810 msgr2=0x7f0468101780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:01.509 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.501+0000 7f045ffff640 1 --2- 172.21.15.1:0/2978622010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0468100810 0x7f0468101780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:01.509 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.501+0000 7f045ffff640 1 -- 172.21.15.1:0/2978622010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0468102200 msgr2=0x7f04681a6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:01.509 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.501+0000 7f045ffff640 1 --2- 172.21.15.1:0/2978622010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0468102200 0x7f04681a6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:01.510 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.501+0000 7f045ffff640 1 -- 172.21.15.1:0/2978622010 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04681a6d10 con 0x7f04681080d0 2024-09-11T22:02:01.510 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.501+0000 7f045ffff640 1 --2- 172.21.15.1:0/2978622010 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04681080d0 0x7f0468101cc0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f0450009230 tx=0x7f0450004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:01.510 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.502+0000 7f04657fa640 1 -- 172.21.15.1:0/2978622010 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045003f030 con 0x7f04681080d0 2024-09-11T22:02:01.510 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.502+0000 7f04657fa640 1 -- 172.21.15.1:0/2978622010 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0450003ad0 con 0x7f04681080d0 2024-09-11T22:02:01.510 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.502+0000 7f046e037640 1 -- 172.21.15.1:0/2978622010 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04681a6fa0 con 0x7f04681080d0 2024-09-11T22:02:01.510 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.502+0000 7f04657fa640 1 -- 172.21.15.1:0/2978622010 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0450035070 con 0x7f04681080d0 2024-09-11T22:02:01.510 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.502+0000 7f046e037640 1 -- 172.21.15.1:0/2978622010 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04681a7410 con 0x7f04681080d0 2024-09-11T22:02:01.517 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.505+0000 7f0470dcf740 1 Processor -- start 2024-09-11T22:02:01.517 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.505+0000 7f0470dcf740 1 -- start start 2024-09-11T22:02:01.517 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.505+0000 7f04657fa640 1 -- 172.21.15.1:0/2978622010 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f0450042030 con 0x7f04681080d0 2024-09-11T22:02:01.518 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.505+0000 7f0470dcf740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601c00aaaf0 0x5601c00aaed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:01.518 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.505+0000 7f0470dcf740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601c00ab410 0x5601c00b5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:01.518 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.505+0000 7f0470dcf740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601c00b5cb0 0x5601c00b80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:01.518 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.505+0000 7f0470dcf740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5601c000f2f0 con 0x5601c00aaaf0 2024-09-11T22:02:01.518 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.505+0000 7f0470dcf740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5601bffb7270 con 0x5601c00b5cb0 2024-09-11T22:02:01.518 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.505+0000 7f0470dcf740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5601c000b2b0 con 0x5601c00ab410 2024-09-11T22:02:01.518 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.505+0000 7f0467fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601c00aaaf0 0x5601c00aaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:01.518 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.505+0000 7f0467fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601c00aaaf0 0x5601c00aaed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44690/0 (socket says 172.21.15.1:44690) 2024-09-11T22:02:01.518 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.505+0000 7f0467fff640 1 -- 172.21.15.1:0/1224323716 learned_addr learned my addr 172.21.15.1:0/1224323716 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:01.518 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.505+0000 7f04677fe640 1 --2- 172.21.15.1:0/1224323716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601c00ab410 0x5601c00b5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:01.518 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.505+0000 7f045ffff640 1 --2- 172.21.15.1:0/1224323716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601c00b5cb0 0x5601c00b80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:01.518 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.506+0000 7f0467fff640 1 -- 172.21.15.1:0/1224323716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601c00ab410 msgr2=0x5601c00b5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:01.518 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.506+0000 7f0467fff640 1 --2- 172.21.15.1:0/1224323716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601c00ab410 0x5601c00b5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.506+0000 7f0467fff640 1 -- 172.21.15.1:0/1224323716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601c00b5cb0 msgr2=0x5601c00b80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.506+0000 7f0467fff640 1 --2- 172.21.15.1:0/1224323716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601c00b5cb0 0x5601c00b80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.506+0000 7f0467fff640 1 -- 172.21.15.1:0/1224323716 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601bffb6340 con 0x5601c00aaaf0 2024-09-11T22:02:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.506+0000 7f04657fa640 1 --2- 172.21.15.1:0/2978622010 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f04540a2da0 0x7f04540a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.506+0000 7f0467fff640 1 --2- 172.21.15.1:0/1224323716 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601c00aaaf0 0x5601c00aaed0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f046000b6b0 tx=0x7f046000bb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8ce627e6812f501 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.506+0000 7f04677fe640 1 --2- 172.21.15.1:0/2978622010 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f04540a2da0 0x7f04540a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.506+0000 7f04657fa640 1 -- 172.21.15.1:0/2978622010 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f04500ef8c0 con 0x7f04681080d0 2024-09-11T22:02:01.520 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.506+0000 7f045dffb640 1 -- 172.21.15.1:0/1224323716 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0460003bd0 con 0x5601c00aaaf0 2024-09-11T22:02:01.520 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.506+0000 7f045dffb640 1 -- 172.21.15.1:0/1224323716 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0460015070 con 0x5601c00aaaf0 2024-09-11T22:02:01.520 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.506+0000 7f045dffb640 1 -- 172.21.15.1:0/1224323716 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0460011640 con 0x5601c00aaaf0 2024-09-11T22:02:01.520 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.507+0000 7f04677fe640 1 --2- 172.21.15.1:0/2978622010 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f04540a2da0 0x7f04540a5260 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x5601c00a7ec0 tx=0x7f045800e040 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:01.520 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.507+0000 7f0470dcf740 1 -- 172.21.15.1:0/1224323716 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601c00aaaf0 msgr2=0x5601c00aaed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.507+0000 7f0470dcf740 1 --2- 172.21.15.1:0/1224323716 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601c00aaaf0 0x5601c00aaed0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f046000b6b0 tx=0x7f046000bb70 comp rx=0 tx=0).stop 2024-09-11T22:02:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.507+0000 7f0470dcf740 1 -- 172.21.15.1:0/1224323716 shutdown_connections 2024-09-11T22:02:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.507+0000 7f0470dcf740 1 --2- 172.21.15.1:0/1224323716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601c00b5cb0 0x5601c00b80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.507+0000 7f0470dcf740 1 --2- 172.21.15.1:0/1224323716 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601c00ab410 0x5601c00b5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.507+0000 7f0470dcf740 1 --2- 172.21.15.1:0/1224323716 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601c00aaaf0 0x5601c00aaed0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:01.522 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.507+0000 7f0470dcf740 1 -- 172.21.15.1:0/1224323716 >> 172.21.15.1:0/1224323716 conn(0x5601c00a7060 msgr2=0x5601c00a7440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:01.522 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.507+0000 7f0470dcf740 1 -- 172.21.15.1:0/1224323716 shutdown_connections 2024-09-11T22:02:01.522 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.507+0000 7f0470dcf740 1 -- 172.21.15.1:0/1224323716 wait complete. 2024-09-11T22:02:01.522 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.508+0000 7f0470dcf740 1 -- start start 2024-09-11T22:02:01.522 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.508+0000 7f0470dcf740 1 Processor -- start 2024-09-11T22:02:01.522 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.508+0000 7f0470dcf740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601c00aaaf0 0x5601c00b2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:01.523 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.508+0000 7f0470dcf740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601c00ab410 0x5601c00b3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:01.523 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.508+0000 7f0470dcf740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601c00b5cb0 0x5601c00b3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:01.523 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.508+0000 7f0470dcf740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5601c00b8880 con 0x5601c00ab410 2024-09-11T22:02:01.523 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.508+0000 7f0470dcf740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5601bff98290 con 0x5601c00b5cb0 2024-09-11T22:02:01.523 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.508+0000 7f0470dcf740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5601c00b8a00 con 0x5601c00aaaf0 2024-09-11T22:02:01.523 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.508+0000 7f0470dcf740 10 client.0 _ll_register_callbacks cb 0x5601bffc9140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:02:01.523 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.509+0000 7f045ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601c00ab410 0x5601c00b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:01.523 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.509+0000 7f04677fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601c00aaaf0 0x5601c00b2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:01.523 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.509+0000 7f045ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601c00ab410 0x5601c00b3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44702/0 (socket says 172.21.15.1:44702) 2024-09-11T22:02:01.523 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.509+0000 7f045ffff640 1 -- 172.21.15.1:0/3505518800 learned_addr learned my addr 172.21.15.1:0/3505518800 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:01.523 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.509+0000 7f0467fff640 1 --2- 172.21.15.1:0/3505518800 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601c00b5cb0 0x5601c00b3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:01.523 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.509+0000 7f045ffff640 1 -- 172.21.15.1:0/3505518800 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601c00aaaf0 msgr2=0x5601c00b2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:01.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.509+0000 7f045ffff640 1 --2- 172.21.15.1:0/3505518800 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601c00aaaf0 0x5601c00b2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:01.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.509+0000 7f045ffff640 1 -- 172.21.15.1:0/3505518800 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601c00b5cb0 msgr2=0x5601c00b3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:01.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.509+0000 7f045ffff640 1 --2- 172.21.15.1:0/3505518800 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601c00b5cb0 0x5601c00b3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:01.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.509+0000 7f045ffff640 1 -- 172.21.15.1:0/3505518800 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601bfd22210 con 0x5601c00ab410 2024-09-11T22:02:01.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.509+0000 7f0467fff640 1 --2- 172.21.15.1:0/3505518800 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601c00b5cb0 0x5601c00b3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:02:01.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.509+0000 7f045ffff640 1 --2- 172.21.15.1:0/3505518800 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601c00ab410 0x5601c00b3110 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f0450009a50 tx=0x7f04500b8040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0e58a22b80cebd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:01.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.509+0000 7f0446ffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:02:01.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.509+0000 7f0446ffd640 1 -- 172.21.15.1:0/3505518800 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0450049070 con 0x5601c00ab410 2024-09-11T22:02:01.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.509+0000 7f0446ffd640 1 -- 172.21.15.1:0/3505518800 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f0450003c80 con 0x5601c00ab410 2024-09-11T22:02:01.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.510+0000 7f0470dcf740 1 -- 172.21.15.1:0/3505518800 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0460015070 con 0x5601c00ab410 2024-09-11T22:02:01.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.510+0000 7f0446ffd640 1 -- 172.21.15.1:0/3505518800 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f04500bb700 con 0x5601c00ab410 2024-09-11T22:02:01.526 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.510+0000 7f0470dcf740 10 client.16362 fetch_fsmap learned FSMap version 33 2024-09-11T22:02:01.526 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.510+0000 7f0470dcf740 1 -- 172.21.15.1:0/3505518800 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5601bfdee620 con 0x5601c00ab410 2024-09-11T22:02:01.526 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.510+0000 7f0446ffd640 1 -- 172.21.15.1:0/3505518800 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0460015070 con 0x5601c00ab410 2024-09-11T22:02:01.526 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.510+0000 7f0446ffd640 1 -- 172.21.15.1:0/3505518800 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7f04500b8410 con 0x5601c00ab410 2024-09-11T22:02:01.526 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.510+0000 7f0470dcf740 10 client.16362 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T22:02:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.510+0000 7f0470dcf740 10 client.16362 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:02:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.510+0000 7f0470dcf740 20 client.16362 populate_metadata read hostname 'smithi001' 2024-09-11T22:02:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.510+0000 7f0470dcf740 1 --2- 172.21.15.1:0/3505518800 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5601c00c5d40 0x5601c00c8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.510+0000 7f0470dcf740 4 client.16362 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:02:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.510+0000 7f0470dcf740 1 -- 172.21.15.1:0/3505518800 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5601c001ca20 con 0x5601c00c5d40 2024-09-11T22:02:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.511+0000 7f04677fe640 1 --2- 172.21.15.1:0/3505518800 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5601c00c5d40 0x5601c00c8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.511+0000 7f0446ffd640 10 client.16362 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:02:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.511+0000 7f04677fe640 1 --2- 172.21.15.1:0/3505518800 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5601c00c5d40 0x5601c00c8160 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2aac130f637f200d server_cookie=d5cc43715391ff1a in_seq=0 out_seq=0 2024-09-11T22:02:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.512+0000 7f0446ffd640 1 -- 172.21.15.1:0/3505518800 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5601c001ca20 con 0x5601c00c5d40 2024-09-11T22:02:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.512+0000 7f0446ffd640 10 client.16362 handle_command_reply: tid=0 2024-09-11T22:02:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.512+0000 7f0470dcf740 1 client.16362 shutdown 2024-09-11T22:02:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.512+0000 7f0470dcf740 1 -- 172.21.15.1:0/3505518800 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601c00ab410 msgr2=0x5601c00b3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.512+0000 7f0470dcf740 1 --2- 172.21.15.1:0/3505518800 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601c00ab410 0x5601c00b3110 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f0450009a50 tx=0x7f04500b8040 comp rx=0 tx=0).stop 2024-09-11T22:02:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.512+0000 7f0470dcf740 1 -- 172.21.15.1:0/3505518800 shutdown_connections 2024-09-11T22:02:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.512+0000 7f0470dcf740 1 --2- 172.21.15.1:0/3505518800 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5601c00c5d40 0x5601c00c8160 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.512+0000 7f0470dcf740 1 --2- 172.21.15.1:0/3505518800 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601c00b5cb0 0x5601c00b3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.512+0000 7f0446ffd640 0 client.16362 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:02:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.512+0000 7f0470dcf740 1 --2- 172.21.15.1:0/3505518800 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601c00ab410 0x5601c00b3110 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.512+0000 7f0470dcf740 1 --2- 172.21.15.1:0/3505518800 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601c00aaaf0 0x5601c00b2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.512+0000 7f0470dcf740 1 -- 172.21.15.1:0/3505518800 >> 172.21.15.1:0/3505518800 conn(0x5601c00a7060 msgr2=0x5601c00a61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.512+0000 7f0470dcf740 1 -- 172.21.15.1:0/3505518800 shutdown_connections 2024-09-11T22:02:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.512+0000 7f0470dcf740 1 -- 172.21.15.1:0/3505518800 wait complete. 2024-09-11T22:02:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.513+0000 7f0470dcf740 20 client.16362 trim_cache size 0 max 16384 2024-09-11T22:02:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.527+0000 7f0470dcf740 1 Processor -- start 2024-09-11T22:02:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.528+0000 7f0470dcf740 1 -- start start 2024-09-11T22:02:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.528+0000 7f0470dcf740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601c00aaaf0 0x5601c00b3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.528+0000 7f0470dcf740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601c00ab410 0x5601c00cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.528+0000 7f0470dcf740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601c00b5cb0 0x5601c00cdf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.528+0000 7f0470dcf740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f04681afbc0 con 0x5601c00ab410 2024-09-11T22:02:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.528+0000 7f0470dcf740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f04681b17f0 con 0x5601c00b5cb0 2024-09-11T22:02:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.528+0000 7f0470dcf740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f04681b0290 con 0x5601c00aaaf0 2024-09-11T22:02:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.528+0000 7f045ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601c00ab410 0x5601c00cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.528+0000 7f045ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601c00ab410 0x5601c00cd9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44704/0 (socket says 172.21.15.1:44704) 2024-09-11T22:02:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.528+0000 7f045ffff640 1 -- 172.21.15.1:0/3824483674 learned_addr learned my addr 172.21.15.1:0/3824483674 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.528+0000 7f0467fff640 1 --2- 172.21.15.1:0/3824483674 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601c00b5cb0 0x5601c00cdf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.528+0000 7f04677fe640 1 --2- 172.21.15.1:0/3824483674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601c00aaaf0 0x5601c00b3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.528+0000 7f045ffff640 1 -- 172.21.15.1:0/3824483674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601c00aaaf0 msgr2=0x5601c00b3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.528+0000 7f045ffff640 1 --2- 172.21.15.1:0/3824483674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601c00aaaf0 0x5601c00b3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.528+0000 7f045ffff640 1 -- 172.21.15.1:0/3824483674 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601c00b5cb0 msgr2=0x5601c00cdf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.528+0000 7f045ffff640 1 --2- 172.21.15.1:0/3824483674 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601c00b5cb0 0x5601c00cdf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.528+0000 7f045ffff640 1 -- 172.21.15.1:0/3824483674 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601c00b83e0 con 0x5601c00ab410 2024-09-11T22:02:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.529+0000 7f045ffff640 1 --2- 172.21.15.1:0/3824483674 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601c00ab410 0x5601c00cd9c0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f04500b8200 tx=0x7f04500b8170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4231c86e9c1c7ee server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.529+0000 7f045d7fa640 1 -- 172.21.15.1:0/3824483674 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0450049070 con 0x5601c00ab410 2024-09-11T22:02:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.529+0000 7f045d7fa640 1 -- 172.21.15.1:0/3824483674 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04500efbc0 con 0x5601c00ab410 2024-09-11T22:02:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.529+0000 7f045d7fa640 1 -- 172.21.15.1:0/3824483674 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04500b8410 con 0x5601c00ab410 2024-09-11T22:02:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.530+0000 7f0470dcf740 1 -- 172.21.15.1:0/3824483674 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601c00ab410 msgr2=0x5601c00cd9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.530+0000 7f0470dcf740 1 --2- 172.21.15.1:0/3824483674 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601c00ab410 0x5601c00cd9c0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f04500b8200 tx=0x7f04500b8170 comp rx=0 tx=0).stop 2024-09-11T22:02:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.530+0000 7f0470dcf740 1 -- 172.21.15.1:0/3824483674 shutdown_connections 2024-09-11T22:02:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.530+0000 7f0470dcf740 1 --2- 172.21.15.1:0/3824483674 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601c00b5cb0 0x5601c00cdf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.530+0000 7f0470dcf740 1 --2- 172.21.15.1:0/3824483674 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601c00ab410 0x5601c00cd9c0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.530+0000 7f0470dcf740 1 --2- 172.21.15.1:0/3824483674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601c00aaaf0 0x5601c00b3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.530+0000 7f0470dcf740 1 -- 172.21.15.1:0/3824483674 >> 172.21.15.1:0/3824483674 conn(0x5601c00a7060 msgr2=0x5601c00b4c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.530+0000 7f0470dcf740 1 -- 172.21.15.1:0/3824483674 shutdown_connections 2024-09-11T22:02:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.530+0000 7f0470dcf740 1 -- 172.21.15.1:0/3824483674 wait complete. 2024-09-11T22:02:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.530+0000 7f0470dcf740 1 -- start start 2024-09-11T22:02:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.530+0000 7f0470dcf740 1 Processor -- start 2024-09-11T22:02:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.531+0000 7f0470dcf740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601c00aaaf0 0x5601c00b9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.531+0000 7f0470dcf740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601c00ab410 0x5601c00ba0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.531+0000 7f0470dcf740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601c00b5cb0 0x5601c00ba600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.531+0000 7f0470dcf740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f04681b1060 con 0x5601c00ab410 2024-09-11T22:02:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.531+0000 7f0470dcf740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f04681b0960 con 0x5601c00b5cb0 2024-09-11T22:02:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.531+0000 7f0470dcf740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f04681adba0 con 0x5601c00aaaf0 2024-09-11T22:02:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.531+0000 7f0470dcf740 10 client.0 _ll_register_callbacks cb 0x7f0450009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:02:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.531+0000 7f045ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601c00ab410 0x5601c00ba0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.531+0000 7f045ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601c00ab410 0x5601c00ba0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44710/0 (socket says 172.21.15.1:44710) 2024-09-11T22:02:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.531+0000 7f045ffff640 1 -- 172.21.15.1:0/978390237 learned_addr learned my addr 172.21.15.1:0/978390237 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.531+0000 7f04677fe640 1 --2- 172.21.15.1:0/978390237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601c00aaaf0 0x5601c00b9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.531+0000 7f0467fff640 1 --2- 172.21.15.1:0/978390237 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601c00b5cb0 0x5601c00ba600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.531+0000 7f045ffff640 1 -- 172.21.15.1:0/978390237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601c00aaaf0 msgr2=0x5601c00b9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.531+0000 7f045ffff640 1 --2- 172.21.15.1:0/978390237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601c00aaaf0 0x5601c00b9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.531+0000 7f045ffff640 1 -- 172.21.15.1:0/978390237 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601c00b5cb0 msgr2=0x5601c00ba600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.531+0000 7f045ffff640 1 --2- 172.21.15.1:0/978390237 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601c00b5cb0 0x5601c00ba600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.531+0000 7f045ffff640 1 -- 172.21.15.1:0/978390237 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601c012fb30 con 0x5601c00ab410 2024-09-11T22:02:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.531+0000 7f04677fe640 1 --2- 172.21.15.1:0/978390237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601c00aaaf0 0x5601c00b9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:02:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.531+0000 7f045ffff640 1 --2- 172.21.15.1:0/978390237 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601c00ab410 0x5601c00ba0c0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f0450000c00 tx=0x7f04500b8040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16ebb5a33e1584ed server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.531+0000 7f0446ffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:02:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.531+0000 7f0446ffd640 1 -- 172.21.15.1:0/978390237 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0450049070 con 0x5601c00ab410 2024-09-11T22:02:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.532+0000 7f0446ffd640 1 -- 172.21.15.1:0/978390237 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f04500c45d0 con 0x5601c00ab410 2024-09-11T22:02:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.532+0000 7f0470dcf740 1 -- 172.21.15.1:0/978390237 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04500efbc0 con 0x5601c00ab410 2024-09-11T22:02:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.532+0000 7f0446ffd640 1 -- 172.21.15.1:0/978390237 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f04500c47d0 con 0x5601c00ab410 2024-09-11T22:02:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.532+0000 7f0470dcf740 10 client.16368 fetch_fsmap learned FSMap version 33 2024-09-11T22:02:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.532+0000 7f0470dcf740 1 -- 172.21.15.1:0/978390237 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5601c0130e20 con 0x5601c00ab410 2024-09-11T22:02:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.532+0000 7f0446ffd640 1 -- 172.21.15.1:0/978390237 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04500efbc0 con 0x5601c00ab410 2024-09-11T22:02:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.532+0000 7f0446ffd640 1 -- 172.21.15.1:0/978390237 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7f045009abc0 con 0x5601c00ab410 2024-09-11T22:02:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.532+0000 7f0470dcf740 10 client.16368 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T22:02:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.532+0000 7f0470dcf740 10 client.16368 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:02:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.532+0000 7f0470dcf740 20 client.16368 populate_metadata read hostname 'smithi001' 2024-09-11T22:02:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.532+0000 7f0470dcf740 1 --2- 172.21.15.1:0/978390237 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5601c01312c0 0x5601c01336e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.532+0000 7f0470dcf740 4 client.16368 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:02:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.532+0000 7f0470dcf740 1 -- 172.21.15.1:0/978390237 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5601c0016d70 con 0x5601c01312c0 2024-09-11T22:02:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.532+0000 7f04677fe640 1 --2- 172.21.15.1:0/978390237 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5601c01312c0 0x5601c01336e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.533+0000 7f04677fe640 1 --2- 172.21.15.1:0/978390237 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5601c01312c0 0x5601c01336e0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=432f340c13487afb server_cookie=6b8a0636b71565ce in_seq=0 out_seq=0 2024-09-11T22:02:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:01.533+0000 7f0446ffd640 10 client.16368 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:02:01.580 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:02:02.555 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:02.554+0000 7f0446ffd640 1 -- 172.21.15.1:0/978390237 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5601c0016d70 con 0x5601c01312c0 2024-09-11T22:02:02.556 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:02.554+0000 7f0446ffd640 10 client.16368 handle_command_reply: tid=0 2024-09-11T22:02:02.556 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:02.554+0000 7f0470dcf740 1 client.16368 shutdown 2024-09-11T22:02:02.556 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:02.554+0000 7f0470dcf740 1 -- 172.21.15.1:0/978390237 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601c00ab410 msgr2=0x5601c00ba0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:02.556 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:02.554+0000 7f0470dcf740 1 --2- 172.21.15.1:0/978390237 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601c00ab410 0x5601c00ba0c0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f0450000c00 tx=0x7f04500b8040 comp rx=0 tx=0).stop 2024-09-11T22:02:02.556 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:02.554+0000 7f0470dcf740 1 -- 172.21.15.1:0/978390237 shutdown_connections 2024-09-11T22:02:02.556 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:02.554+0000 7f0470dcf740 1 --2- 172.21.15.1:0/978390237 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5601c01312c0 0x5601c01336e0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:02.556 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:02.554+0000 7f0470dcf740 1 --2- 172.21.15.1:0/978390237 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5601c00b5cb0 0x5601c00ba600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:02.556 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:02.554+0000 7f0470dcf740 1 --2- 172.21.15.1:0/978390237 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601c00ab410 0x5601c00ba0c0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:02.556 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:02.554+0000 7f0470dcf740 1 --2- 172.21.15.1:0/978390237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5601c00aaaf0 0x5601c00b9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:02.557 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:02.554+0000 7f0470dcf740 1 -- 172.21.15.1:0/978390237 >> 172.21.15.1:0/978390237 conn(0x5601c00a7060 msgr2=0x5601c00b4430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:02.557 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:02.554+0000 7f0446ffd640 0 client.16368 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:02:02.557 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:02.554+0000 7f0470dcf740 1 -- 172.21.15.1:0/978390237 shutdown_connections 2024-09-11T22:02:02.557 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:02.554+0000 7f0470dcf740 1 -- 172.21.15.1:0/978390237 wait complete. 2024-09-11T22:02:02.557 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:02.554+0000 7f0470dcf740 20 client.16368 trim_cache size 0 max 16384 2024-09-11T22:02:02.557 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-09-11T22:02:02.557 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-09-11T22:02:02.557 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-09-11T22:02:02.557 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-09-11T22:02:02.558 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:02.555+0000 7f046e037640 1 -- 172.21.15.1:0/2978622010 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f04540a2da0 msgr2=0x7f04540a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:02.558 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:02.555+0000 7f046e037640 1 --2- 172.21.15.1:0/2978622010 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f04540a2da0 0x7f04540a5260 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x5601c00a7ec0 tx=0x7f045800e040 comp rx=0 tx=0).stop 2024-09-11T22:02:02.558 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:02.555+0000 7f046e037640 1 -- 172.21.15.1:0/2978622010 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04681080d0 msgr2=0x7f0468101cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:02.558 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:02.555+0000 7f046e037640 1 --2- 172.21.15.1:0/2978622010 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04681080d0 0x7f0468101cc0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f0450009230 tx=0x7f0450004270 comp rx=0 tx=0).stop 2024-09-11T22:02:02.558 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:02.555+0000 7f046e037640 1 -- 172.21.15.1:0/2978622010 shutdown_connections 2024-09-11T22:02:02.559 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:02.555+0000 7f046e037640 1 --2- 172.21.15.1:0/2978622010 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f04540a2da0 0x7f04540a5260 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:02.559 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:02.555+0000 7f046e037640 1 --2- 172.21.15.1:0/2978622010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0468102200 0x7f04681a6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:02.559 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:02.555+0000 7f046e037640 1 --2- 172.21.15.1:0/2978622010 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04681080d0 0x7f0468101cc0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:02.559 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:02.555+0000 7f046e037640 1 --2- 172.21.15.1:0/2978622010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0468100810 0x7f0468101780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:02.559 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:02.555+0000 7f046e037640 1 -- 172.21.15.1:0/2978622010 >> 172.21.15.1:0/2978622010 conn(0x7f0468075eb0 msgr2=0x7f0468105bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:02.559 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:02.556+0000 7f046e037640 1 -- 172.21.15.1:0/2978622010 shutdown_connections 2024-09-11T22:02:02.559 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:02.556+0000 7f046e037640 1 -- 172.21.15.1:0/2978622010 wait complete. 2024-09-11T22:02:03.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.171+0000 7f7b68f2e640 1 -- 172.21.15.1:0/2894052851 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b64102d90 msgr2=0x7f7b64103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:03.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.171+0000 7f7b68f2e640 1 --2- 172.21.15.1:0/2894052851 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b64102d90 0x7f7b64103170 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f7b54009930 tx=0x7f7b5402a420 comp rx=0 tx=0).stop 2024-09-11T22:02:03.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.172+0000 7f7b68f2e640 1 -- 172.21.15.1:0/2894052851 shutdown_connections 2024-09-11T22:02:03.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.172+0000 7f7b68f2e640 1 --2- 172.21.15.1:0/2894052851 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b64108290 0x7f7b6410ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:03.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.172+0000 7f7b68f2e640 1 --2- 172.21.15.1:0/2894052851 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b64103740 0x7f7b64107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:03.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.172+0000 7f7b68f2e640 1 --2- 172.21.15.1:0/2894052851 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b64102d90 0x7f7b64103170 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:03.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.172+0000 7f7b68f2e640 1 -- 172.21.15.1:0/2894052851 >> 172.21.15.1:0/2894052851 conn(0x7f7b640fc7d0 msgr2=0x7f7b640febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:03.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.172+0000 7f7b68f2e640 1 -- 172.21.15.1:0/2894052851 shutdown_connections 2024-09-11T22:02:03.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.172+0000 7f7b68f2e640 1 -- 172.21.15.1:0/2894052851 wait complete. 2024-09-11T22:02:03.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.172+0000 7f7b68f2e640 1 Processor -- start 2024-09-11T22:02:03.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.172+0000 7f7b68f2e640 1 -- start start 2024-09-11T22:02:03.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.173+0000 7f7b68f2e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b64102d90 0x7f7b641009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:03.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.173+0000 7f7b68f2e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b64103740 0x7f7b64100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:03.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.173+0000 7f7b68f2e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b64108290 0x7f7b64101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:03.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.173+0000 7f7b68f2e640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7b64112090 con 0x7f7b64102d90 2024-09-11T22:02:03.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.173+0000 7f7b68f2e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7b64111f10 con 0x7f7b64103740 2024-09-11T22:02:03.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.173+0000 7f7b68f2e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7b64112210 con 0x7f7b64108290 2024-09-11T22:02:03.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.173+0000 7f7b6259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b64102d90 0x7f7b641009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:03.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.173+0000 7f7b6259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b64102d90 0x7f7b641009f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44728/0 (socket says 172.21.15.1:44728) 2024-09-11T22:02:03.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.173+0000 7f7b6259c640 1 -- 172.21.15.1:0/3425729607 learned_addr learned my addr 172.21.15.1:0/3425729607 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:03.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.173+0000 7f7b62d9d640 1 --2- 172.21.15.1:0/3425729607 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b64108290 0x7f7b64101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:03.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.173+0000 7f7b61d9b640 1 --2- 172.21.15.1:0/3425729607 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b64103740 0x7f7b64100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:03.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.173+0000 7f7b6259c640 1 -- 172.21.15.1:0/3425729607 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b64108290 msgr2=0x7f7b64101470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:03.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.173+0000 7f7b6259c640 1 --2- 172.21.15.1:0/3425729607 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b64108290 0x7f7b64101470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:03.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.173+0000 7f7b6259c640 1 -- 172.21.15.1:0/3425729607 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b64103740 msgr2=0x7f7b64100f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:03.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.173+0000 7f7b6259c640 1 --2- 172.21.15.1:0/3425729607 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b64103740 0x7f7b64100f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:03.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.173+0000 7f7b6259c640 1 -- 172.21.15.1:0/3425729607 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b641a6a60 con 0x7f7b64102d90 2024-09-11T22:02:03.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.173+0000 7f7b62d9d640 1 --2- 172.21.15.1:0/3425729607 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b64108290 0x7f7b64101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:02:03.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.173+0000 7f7b61d9b640 1 --2- 172.21.15.1:0/3425729607 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b64103740 0x7f7b64100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:02:03.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.173+0000 7f7b6259c640 1 --2- 172.21.15.1:0/3425729607 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b64102d90 0x7f7b641009f0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f7b54009230 tx=0x7f7b54004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:03.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.173+0000 7f7b537fe640 1 -- 172.21.15.1:0/3425729607 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b5403f030 con 0x7f7b64102d90 2024-09-11T22:02:03.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.174+0000 7f7b537fe640 1 -- 172.21.15.1:0/3425729607 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b54003ad0 con 0x7f7b64102d90 2024-09-11T22:02:03.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.174+0000 7f7b68f2e640 1 -- 172.21.15.1:0/3425729607 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b641a6cf0 con 0x7f7b64102d90 2024-09-11T22:02:03.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.174+0000 7f7b537fe640 1 -- 172.21.15.1:0/3425729607 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b54035070 con 0x7f7b64102d90 2024-09-11T22:02:03.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.174+0000 7f7b68f2e640 1 -- 172.21.15.1:0/3425729607 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b641a71c0 con 0x7f7b64102d90 2024-09-11T22:02:03.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.176+0000 7f7b537fe640 1 -- 172.21.15.1:0/3425729607 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f7b54042030 con 0x7f7b64102d90 2024-09-11T22:02:03.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.177+0000 7f7b537fe640 1 --2- 172.21.15.1:0/3425729607 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f7b4c0a2d70 0x7f7b4c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:03.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.177+0000 7f7b61d9b640 1 --2- 172.21.15.1:0/3425729607 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f7b4c0a2d70 0x7f7b4c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:03.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.177+0000 7f7b6b307740 1 Processor -- start 2024-09-11T22:02:03.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.177+0000 7f7b537fe640 1 -- 172.21.15.1:0/3425729607 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f7b540ef9e0 con 0x7f7b64102d90 2024-09-11T22:02:03.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.177+0000 7f7b6b307740 1 -- start start 2024-09-11T22:02:03.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.177+0000 7f7b61d9b640 1 --2- 172.21.15.1:0/3425729607 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f7b4c0a2d70 0x7f7b4c0a5230 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7b580058c0 tx=0x7f7b58005850 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:03.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.177+0000 7f7b6b307740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558509c27b40 0x558509c26090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:03.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.178+0000 7f7b6b307740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558509c265d0 0x558509c26a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:03.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.178+0000 7f7b6b307740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558509c31370 0x558509c33760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:03.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.178+0000 7f7b6b307740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x558509b8c2f0 con 0x558509c27b40 2024-09-11T22:02:03.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.178+0000 7f7b6b307740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x558509b33cf0 con 0x558509c265d0 2024-09-11T22:02:03.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.178+0000 7f7b6b307740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x558509b882b0 con 0x558509c31370 2024-09-11T22:02:03.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.178+0000 7f7b6259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558509c27b40 0x558509c26090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:03.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.178+0000 7f7b6259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558509c27b40 0x558509c26090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44734/0 (socket says 172.21.15.1:44734) 2024-09-11T22:02:03.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.178+0000 7f7b6259c640 1 -- 172.21.15.1:0/3295996477 learned_addr learned my addr 172.21.15.1:0/3295996477 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:03.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.178+0000 7f7b61d9b640 1 --2- 172.21.15.1:0/3295996477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558509c265d0 0x558509c26a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:03.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.178+0000 7f7b6259c640 1 -- 172.21.15.1:0/3295996477 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558509c31370 msgr2=0x558509c33760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:03.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.178+0000 7f7b62d9d640 1 --2- 172.21.15.1:0/3295996477 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558509c31370 0x558509c33760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:03.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.178+0000 7f7b6259c640 1 --2- 172.21.15.1:0/3295996477 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558509c31370 0x558509c33760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:03.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.178+0000 7f7b6259c640 1 -- 172.21.15.1:0/3295996477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558509c265d0 msgr2=0x558509c26a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:03.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.178+0000 7f7b6259c640 1 --2- 172.21.15.1:0/3295996477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558509c265d0 0x558509c26a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:03.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.178+0000 7f7b6259c640 1 -- 172.21.15.1:0/3295996477 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558509b183a0 con 0x558509c27b40 2024-09-11T22:02:03.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.178+0000 7f7b61d9b640 1 --2- 172.21.15.1:0/3295996477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558509c265d0 0x558509c26a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:02:03.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.178+0000 7f7b62d9d640 1 --2- 172.21.15.1:0/3295996477 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558509c31370 0x558509c33760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:02:03.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.178+0000 7f7b6259c640 1 --2- 172.21.15.1:0/3295996477 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558509c27b40 0x558509c26090 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f7b54032c00 tx=0x7f7b5402a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b532a02348b32f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:03.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.178+0000 7f7b50ff9640 1 -- 172.21.15.1:0/3295996477 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b54049040 con 0x558509c27b40 2024-09-11T22:02:03.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.178+0000 7f7b50ff9640 1 -- 172.21.15.1:0/3295996477 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b540c4ad0 con 0x558509c27b40 2024-09-11T22:02:03.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.178+0000 7f7b50ff9640 1 -- 172.21.15.1:0/3295996477 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b540bc0c0 con 0x558509c27b40 2024-09-11T22:02:03.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.179+0000 7f7b6b307740 1 -- 172.21.15.1:0/3295996477 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558509c27b40 msgr2=0x558509c26090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:03.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.179+0000 7f7b6b307740 1 --2- 172.21.15.1:0/3295996477 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558509c27b40 0x558509c26090 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f7b54032c00 tx=0x7f7b5402a420 comp rx=0 tx=0).stop 2024-09-11T22:02:03.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.179+0000 7f7b6b307740 1 -- 172.21.15.1:0/3295996477 shutdown_connections 2024-09-11T22:02:03.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.179+0000 7f7b6b307740 1 --2- 172.21.15.1:0/3295996477 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558509c31370 0x558509c33760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:03.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.179+0000 7f7b6b307740 1 --2- 172.21.15.1:0/3295996477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558509c265d0 0x558509c26a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:03.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.179+0000 7f7b6b307740 1 --2- 172.21.15.1:0/3295996477 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558509c27b40 0x558509c26090 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:03.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.179+0000 7f7b6b307740 1 -- 172.21.15.1:0/3295996477 >> 172.21.15.1:0/3295996477 conn(0x558509bef310 msgr2=0x558509c22c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:03.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.179+0000 7f7b6b307740 1 -- 172.21.15.1:0/3295996477 shutdown_connections 2024-09-11T22:02:03.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.179+0000 7f7b6b307740 1 -- 172.21.15.1:0/3295996477 wait complete. 2024-09-11T22:02:03.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.180+0000 7f7b6b307740 1 -- start start 2024-09-11T22:02:03.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.180+0000 7f7b6b307740 1 Processor -- start 2024-09-11T22:02:03.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.180+0000 7f7b6b307740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558509c265d0 0x558509c30dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:03.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.180+0000 7f7b6b307740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558509c27b40 0x558509c2e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:03.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.180+0000 7f7b6b307740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558509c31370 0x558509c2e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:03.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.180+0000 7f7b6b307740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x558509c33e30 con 0x558509c265d0 2024-09-11T22:02:03.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.180+0000 7f7b6b307740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x558509b47060 con 0x558509c27b40 2024-09-11T22:02:03.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.180+0000 7f7b6b307740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x558509c33fb0 con 0x558509c31370 2024-09-11T22:02:03.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.180+0000 7f7b6b307740 10 client.0 _ll_register_callbacks cb 0x55850989a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:02:03.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.180+0000 7f7b6259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558509c265d0 0x558509c30dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:03.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.180+0000 7f7b6259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558509c265d0 0x558509c30dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44738/0 (socket says 172.21.15.1:44738) 2024-09-11T22:02:03.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.180+0000 7f7b6259c640 1 -- 172.21.15.1:0/3533943935 learned_addr learned my addr 172.21.15.1:0/3533943935 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:03.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.180+0000 7f7b61d9b640 1 --2- 172.21.15.1:0/3533943935 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558509c27b40 0x558509c2e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:03.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.180+0000 7f7b6259c640 1 -- 172.21.15.1:0/3533943935 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558509c31370 msgr2=0x558509c2e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:03.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.180+0000 7f7b6259c640 1 --2- 172.21.15.1:0/3533943935 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558509c31370 0x558509c2e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:03.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.180+0000 7f7b6259c640 1 -- 172.21.15.1:0/3533943935 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558509c27b40 msgr2=0x558509c2e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:03.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.180+0000 7f7b6259c640 1 --2- 172.21.15.1:0/3533943935 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558509c27b40 0x558509c2e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:03.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.180+0000 7f7b6259c640 1 -- 172.21.15.1:0/3533943935 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585098fd3c0 con 0x558509c265d0 2024-09-11T22:02:03.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.181+0000 7f7b6259c640 1 --2- 172.21.15.1:0/3533943935 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558509c265d0 0x558509c30dd0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f7b540c45a0 tx=0x7f7b54005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a12f6a96db428587 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:03.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.181+0000 7f7b49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:02:03.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.181+0000 7f7b49ffb640 1 -- 172.21.15.1:0/3533943935 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b540d9070 con 0x558509c265d0 2024-09-11T22:02:03.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.181+0000 7f7b49ffb640 1 -- 172.21.15.1:0/3533943935 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f7b54003c80 con 0x558509c265d0 2024-09-11T22:02:03.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.181+0000 7f7b6b307740 1 -- 172.21.15.1:0/3533943935 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b540c4ad0 con 0x558509c265d0 2024-09-11T22:02:03.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.181+0000 7f7b49ffb640 1 -- 172.21.15.1:0/3533943935 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7b54049040 con 0x558509c265d0 2024-09-11T22:02:03.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.181+0000 7f7b6b307740 10 client.16380 fetch_fsmap learned FSMap version 33 2024-09-11T22:02:03.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.181+0000 7f7b6b307740 1 -- 172.21.15.1:0/3533943935 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x558509c33e30 con 0x558509c265d0 2024-09-11T22:02:03.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.181+0000 7f7b49ffb640 1 -- 172.21.15.1:0/3533943935 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b540c4ad0 con 0x558509c265d0 2024-09-11T22:02:03.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.181+0000 7f7b49ffb640 1 -- 172.21.15.1:0/3533943935 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7f7b540c4640 con 0x558509c265d0 2024-09-11T22:02:03.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.181+0000 7f7b6b307740 10 client.16380 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T22:02:03.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.181+0000 7f7b6b307740 10 client.16380 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T22:02:03.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.181+0000 7f7b6b307740 20 client.16380 populate_metadata read hostname 'smithi001' 2024-09-11T22:02:03.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.181+0000 7f7b6b307740 1 --2- 172.21.15.1:0/3533943935 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x558509c40a90 0x558509c42f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:03.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.181+0000 7f7b6b307740 4 client.16380 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:02:03.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.181+0000 7f7b6b307740 1 -- 172.21.15.1:0/3533943935 --> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558509b99860 con 0x558509c40a90 2024-09-11T22:02:03.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.182+0000 7f7b61d9b640 1 --2- 172.21.15.1:0/3533943935 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x558509c40a90 0x558509c42f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:03.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.182+0000 7f7b61d9b640 1 --2- 172.21.15.1:0/3533943935 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x558509c40a90 0x558509c42f10 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=18862ff54154d846 server_cookie=f4e1e39ce0c8efb5 in_seq=0 out_seq=0 2024-09-11T22:02:03.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.182+0000 7f7b49ffb640 10 client.16380 ms_handle_connect on v2:172.21.15.161:6838/455365383 2024-09-11T22:02:03.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.184+0000 7f7b49ffb640 1 -- 172.21.15.1:0/3533943935 <== mds.1 v2:172.21.15.161:6838/455365383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558509b99860 con 0x558509c40a90 2024-09-11T22:02:03.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.184+0000 7f7b49ffb640 10 client.16380 handle_command_reply: tid=0 2024-09-11T22:02:03.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.184+0000 7f7b6b307740 1 client.16380 shutdown 2024-09-11T22:02:03.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.184+0000 7f7b6b307740 1 -- 172.21.15.1:0/3533943935 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558509c265d0 msgr2=0x558509c30dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:03.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.184+0000 7f7b6b307740 1 --2- 172.21.15.1:0/3533943935 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558509c265d0 0x558509c30dd0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f7b540c45a0 tx=0x7f7b54005e50 comp rx=0 tx=0).stop 2024-09-11T22:02:03.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.184+0000 7f7b6b307740 1 -- 172.21.15.1:0/3533943935 shutdown_connections 2024-09-11T22:02:03.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.184+0000 7f7b6b307740 1 --2- 172.21.15.1:0/3533943935 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x558509c40a90 0x558509c42f10 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:03.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.184+0000 7f7b6b307740 1 --2- 172.21.15.1:0/3533943935 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558509c31370 0x558509c2e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:03.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.184+0000 7f7b6b307740 1 --2- 172.21.15.1:0/3533943935 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558509c27b40 0x558509c2e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:03.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.184+0000 7f7b49ffb640 0 client.16380 ms_handle_reset on v2:172.21.15.161:6838/455365383 2024-09-11T22:02:03.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.184+0000 7f7b6b307740 1 --2- 172.21.15.1:0/3533943935 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558509c265d0 0x558509c30dd0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:03.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.184+0000 7f7b6b307740 1 -- 172.21.15.1:0/3533943935 >> 172.21.15.1:0/3533943935 conn(0x558509bef310 msgr2=0x558509c28930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:03.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.184+0000 7f7b6b307740 1 -- 172.21.15.1:0/3533943935 shutdown_connections 2024-09-11T22:02:03.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.184+0000 7f7b6b307740 1 -- 172.21.15.1:0/3533943935 wait complete. 2024-09-11T22:02:03.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.184+0000 7f7b6b307740 20 client.16380 trim_cache size 0 max 16384 2024-09-11T22:02:03.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.197+0000 7f7b6b307740 1 Processor -- start 2024-09-11T22:02:03.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.197+0000 7f7b6b307740 1 -- start start 2024-09-11T22:02:03.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.197+0000 7f7b6b307740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558509c265d0 0x558509c2d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:03.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.197+0000 7f7b6b307740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558509c27b40 0x558509c21c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:03.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.197+0000 7f7b6b307740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558509c31370 0x558509c29120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:03.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.197+0000 7f7b6b307740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7b641afc90 con 0x558509c31370 2024-09-11T22:02:03.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.197+0000 7f7b6b307740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7b641b18c0 con 0x558509c265d0 2024-09-11T22:02:03.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.197+0000 7f7b6b307740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7b641b0360 con 0x558509c27b40 2024-09-11T22:02:03.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.197+0000 7f7b62d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558509c31370 0x558509c29120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:03.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.197+0000 7f7b62d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558509c31370 0x558509c29120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44746/0 (socket says 172.21.15.1:44746) 2024-09-11T22:02:03.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.197+0000 7f7b62d9d640 1 -- 172.21.15.1:0/1481913730 learned_addr learned my addr 172.21.15.1:0/1481913730 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:03.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.198+0000 7f7b61d9b640 1 --2- 172.21.15.1:0/1481913730 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558509c27b40 0x558509c21c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:03.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.198+0000 7f7b6259c640 1 --2- 172.21.15.1:0/1481913730 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558509c265d0 0x558509c2d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:03.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.198+0000 7f7b62d9d640 1 -- 172.21.15.1:0/1481913730 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558509c27b40 msgr2=0x558509c21c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:03.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.198+0000 7f7b62d9d640 1 --2- 172.21.15.1:0/1481913730 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558509c27b40 0x558509c21c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:03.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.198+0000 7f7b62d9d640 1 -- 172.21.15.1:0/1481913730 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558509c265d0 msgr2=0x558509c2d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:03.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.198+0000 7f7b62d9d640 1 --2- 172.21.15.1:0/1481913730 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558509c265d0 0x558509c2d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:03.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.198+0000 7f7b62d9d640 1 -- 172.21.15.1:0/1481913730 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558509c2f010 con 0x558509c31370 2024-09-11T22:02:03.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.198+0000 7f7b6259c640 1 --2- 172.21.15.1:0/1481913730 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558509c265d0 0x558509c2d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:02:03.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.198+0000 7f7b62d9d640 1 --2- 172.21.15.1:0/1481913730 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558509c31370 0x558509c29120 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c009d80 tx=0x7f7b5c011750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa0b970215a26831 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:03.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.198+0000 7f7b4bfff640 1 -- 172.21.15.1:0/1481913730 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c01a5f0 con 0x558509c31370 2024-09-11T22:02:03.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.198+0000 7f7b4bfff640 1 -- 172.21.15.1:0/1481913730 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b5c011e60 con 0x558509c31370 2024-09-11T22:02:03.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.198+0000 7f7b4bfff640 1 -- 172.21.15.1:0/1481913730 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c012bf0 con 0x558509c31370 2024-09-11T22:02:03.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.199+0000 7f7b6b307740 1 -- 172.21.15.1:0/1481913730 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558509c31370 msgr2=0x558509c29120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:03.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.199+0000 7f7b6b307740 1 --2- 172.21.15.1:0/1481913730 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558509c31370 0x558509c29120 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c009d80 tx=0x7f7b5c011750 comp rx=0 tx=0).stop 2024-09-11T22:02:03.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.199+0000 7f7b6b307740 1 -- 172.21.15.1:0/1481913730 shutdown_connections 2024-09-11T22:02:03.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.199+0000 7f7b6b307740 1 --2- 172.21.15.1:0/1481913730 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558509c31370 0x558509c29120 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:03.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.199+0000 7f7b6b307740 1 --2- 172.21.15.1:0/1481913730 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558509c27b40 0x558509c21c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:03.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.199+0000 7f7b6b307740 1 --2- 172.21.15.1:0/1481913730 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558509c265d0 0x558509c2d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:03.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.199+0000 7f7b6b307740 1 -- 172.21.15.1:0/1481913730 >> 172.21.15.1:0/1481913730 conn(0x558509bef310 msgr2=0x558509c21500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:03.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.199+0000 7f7b6b307740 1 -- 172.21.15.1:0/1481913730 shutdown_connections 2024-09-11T22:02:03.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.199+0000 7f7b6b307740 1 -- 172.21.15.1:0/1481913730 wait complete. 2024-09-11T22:02:03.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.199+0000 7f7b6b307740 1 -- start start 2024-09-11T22:02:03.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.199+0000 7f7b6b307740 1 Processor -- start 2024-09-11T22:02:03.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.200+0000 7f7b6b307740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558509c265d0 0x558509c39910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:03.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.200+0000 7f7b6b307740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558509c27b40 0x558509c34c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:03.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.200+0000 7f7b6b307740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558509c31370 0x558509c351a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:03.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.200+0000 7f7b6b307740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7b641b1130 con 0x558509c265d0 2024-09-11T22:02:03.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.200+0000 7f7b6b307740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7b641b0a30 con 0x558509c27b40 2024-09-11T22:02:03.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.200+0000 7f7b6b307740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7b641adc70 con 0x558509c31370 2024-09-11T22:02:03.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.200+0000 7f7b6b307740 10 client.0 _ll_register_callbacks cb 0x7f7b540bb7c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:02:03.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.200+0000 7f7b6259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558509c265d0 0x558509c39910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:03.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.200+0000 7f7b6259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558509c265d0 0x558509c39910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44756/0 (socket says 172.21.15.1:44756) 2024-09-11T22:02:03.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.200+0000 7f7b6259c640 1 -- 172.21.15.1:0/2384211365 learned_addr learned my addr 172.21.15.1:0/2384211365 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:03.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.200+0000 7f7b61d9b640 1 --2- 172.21.15.1:0/2384211365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558509c27b40 0x558509c34c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:03.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.200+0000 7f7b62d9d640 1 --2- 172.21.15.1:0/2384211365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558509c31370 0x558509c351a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:03.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.200+0000 7f7b6259c640 1 -- 172.21.15.1:0/2384211365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558509c31370 msgr2=0x558509c351a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:03.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.200+0000 7f7b6259c640 1 --2- 172.21.15.1:0/2384211365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558509c31370 0x558509c351a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:03.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.200+0000 7f7b6259c640 1 -- 172.21.15.1:0/2384211365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558509c27b40 msgr2=0x558509c34c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:03.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.200+0000 7f7b6259c640 1 --2- 172.21.15.1:0/2384211365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558509c27b40 0x558509c34c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:03.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.200+0000 7f7b6259c640 1 -- 172.21.15.1:0/2384211365 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558509c359d0 con 0x558509c265d0 2024-09-11T22:02:03.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.200+0000 7f7b62d9d640 1 --2- 172.21.15.1:0/2384211365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558509c31370 0x558509c351a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:02:03.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.200+0000 7f7b61d9b640 1 --2- 172.21.15.1:0/2384211365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558509c27b40 0x558509c34c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:02:03.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.200+0000 7f7b6259c640 1 --2- 172.21.15.1:0/2384211365 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558509c265d0 0x558509c39910 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f7b54000c00 tx=0x7f7b5403bbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16bcdabeb6875130 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:03.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.200+0000 7f7b49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:02:03.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.200+0000 7f7b49ffb640 1 -- 172.21.15.1:0/2384211365 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b540d9070 con 0x558509c265d0 2024-09-11T22:02:03.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.200+0000 7f7b49ffb640 1 -- 172.21.15.1:0/2384211365 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f7b540ed310 con 0x558509c265d0 2024-09-11T22:02:03.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.200+0000 7f7b6b307740 1 -- 172.21.15.1:0/2384211365 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b5c011e60 con 0x558509c265d0 2024-09-11T22:02:03.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.201+0000 7f7b49ffb640 1 -- 172.21.15.1:0/2384211365 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7b540a5170 con 0x558509c265d0 2024-09-11T22:02:03.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.201+0000 7f7b6b307740 10 client.16386 fetch_fsmap learned FSMap version 33 2024-09-11T22:02:03.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.201+0000 7f7b6b307740 1 -- 172.21.15.1:0/2384211365 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x558509c359d0 con 0x558509c265d0 2024-09-11T22:02:03.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.201+0000 7f7b49ffb640 1 -- 172.21.15.1:0/2384211365 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b5c011e60 con 0x558509c265d0 2024-09-11T22:02:03.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.201+0000 7f7b49ffb640 1 -- 172.21.15.1:0/2384211365 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7f7b54031bd0 con 0x558509c265d0 2024-09-11T22:02:03.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.201+0000 7f7b6b307740 10 client.16386 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T22:02:03.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.201+0000 7f7b6b307740 10 client.16386 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T22:02:03.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.201+0000 7f7b6b307740 20 client.16386 populate_metadata read hostname 'smithi001' 2024-09-11T22:02:03.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.201+0000 7f7b6b307740 1 --2- 172.21.15.1:0/2384211365 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x558509cabba0 0x558509cadf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:03.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.201+0000 7f7b6b307740 4 client.16386 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:02:03.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.201+0000 7f7b6b307740 1 -- 172.21.15.1:0/2384211365 --> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558509b93d70 con 0x558509cabba0 2024-09-11T22:02:03.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.201+0000 7f7b61d9b640 1 --2- 172.21.15.1:0/2384211365 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x558509cabba0 0x558509cadf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:03.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.202+0000 7f7b61d9b640 1 --2- 172.21.15.1:0/2384211365 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x558509cabba0 0x558509cadf90 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=68874645f011d5b3 server_cookie=3a9d731a836a9495 in_seq=0 out_seq=0 2024-09-11T22:02:03.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.202+0000 7f7b49ffb640 10 client.16386 ms_handle_connect on v2:172.21.15.161:6838/455365383 2024-09-11T22:02:03.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.202+0000 7f7b49ffb640 1 -- 172.21.15.1:0/2384211365 <== mds.1 v2:172.21.15.161:6838/455365383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558509b93d70 con 0x558509cabba0 2024-09-11T22:02:03.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.202+0000 7f7b49ffb640 10 client.16386 handle_command_reply: tid=0 2024-09-11T22:02:03.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.202+0000 7f7b6b307740 1 client.16386 shutdown 2024-09-11T22:02:03.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.202+0000 7f7b6b307740 1 -- 172.21.15.1:0/2384211365 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558509c265d0 msgr2=0x558509c39910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:03.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.202+0000 7f7b6b307740 1 --2- 172.21.15.1:0/2384211365 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558509c265d0 0x558509c39910 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f7b54000c00 tx=0x7f7b5403bbb0 comp rx=0 tx=0).stop 2024-09-11T22:02:03.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.202+0000 7f7b6b307740 1 -- 172.21.15.1:0/2384211365 shutdown_connections 2024-09-11T22:02:03.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.202+0000 7f7b6b307740 1 --2- 172.21.15.1:0/2384211365 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x558509cabba0 0x558509cadf90 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:03.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.202+0000 7f7b6b307740 1 --2- 172.21.15.1:0/2384211365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558509c31370 0x558509c351a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:03.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.202+0000 7f7b6b307740 1 --2- 172.21.15.1:0/2384211365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558509c27b40 0x558509c34c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:03.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.202+0000 7f7b49ffb640 0 client.16386 ms_handle_reset on v2:172.21.15.161:6838/455365383 2024-09-11T22:02:03.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.202+0000 7f7b6b307740 1 --2- 172.21.15.1:0/2384211365 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558509c265d0 0x558509c39910 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:03.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.202+0000 7f7b6b307740 1 -- 172.21.15.1:0/2384211365 >> 172.21.15.1:0/2384211365 conn(0x558509bef310 msgr2=0x558509c21d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:03.223 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-11T22:02:03.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.202+0000 7f7b6b307740 1 -- 172.21.15.1:0/2384211365 shutdown_connections 2024-09-11T22:02:03.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.202+0000 7f7b6b307740 1 -- 172.21.15.1:0/2384211365 wait complete. 2024-09-11T22:02:03.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.203+0000 7f7b6b307740 20 client.16386 trim_cache size 0 max 16384 2024-09-11T22:02:03.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.203+0000 7f7b68f2e640 1 -- 172.21.15.1:0/3425729607 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f7b4c0a2d70 msgr2=0x7f7b4c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:03.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.203+0000 7f7b68f2e640 1 --2- 172.21.15.1:0/3425729607 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f7b4c0a2d70 0x7f7b4c0a5230 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7b580058c0 tx=0x7f7b58005850 comp rx=0 tx=0).stop 2024-09-11T22:02:03.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.203+0000 7f7b68f2e640 1 -- 172.21.15.1:0/3425729607 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b64102d90 msgr2=0x7f7b641009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:03.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.203+0000 7f7b68f2e640 1 --2- 172.21.15.1:0/3425729607 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b64102d90 0x7f7b641009f0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f7b54009230 tx=0x7f7b54004270 comp rx=0 tx=0).stop 2024-09-11T22:02:03.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.203+0000 7f7b68f2e640 1 -- 172.21.15.1:0/3425729607 shutdown_connections 2024-09-11T22:02:03.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.203+0000 7f7b68f2e640 1 --2- 172.21.15.1:0/3425729607 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f7b4c0a2d70 0x7f7b4c0a5230 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:03.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.203+0000 7f7b68f2e640 1 --2- 172.21.15.1:0/3425729607 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b64108290 0x7f7b64101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:03.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.203+0000 7f7b68f2e640 1 --2- 172.21.15.1:0/3425729607 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b64103740 0x7f7b64100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:03.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.203+0000 7f7b68f2e640 1 --2- 172.21.15.1:0/3425729607 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b64102d90 0x7f7b641009f0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:03.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.203+0000 7f7b68f2e640 1 -- 172.21.15.1:0/3425729607 >> 172.21.15.1:0/3425729607 conn(0x7f7b640fc7d0 msgr2=0x7f7b64105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:03.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.203+0000 7f7b68f2e640 1 -- 172.21.15.1:0/3425729607 shutdown_connections 2024-09-11T22:02:03.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:02.203+0000 7f7b68f2e640 1 -- 172.21.15.1:0/3425729607 wait complete. 2024-09-11T22:02:03.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:03 smithi053 ceph-mon[34908]: pgmap v149: 97 pgs: 97 active+clean; 39 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 51 op/s 2024-09-11T22:02:03.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:03 smithi161 ceph-mon[35615]: pgmap v149: 97 pgs: 97 active+clean; 39 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 51 op/s 2024-09-11T22:02:03.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:03 smithi001 ceph-mon[30935]: pgmap v149: 97 pgs: 97 active+clean; 39 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 51 op/s 2024-09-11T22:02:04.767 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T22:02:05.044 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:02:05.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:05 smithi053 ceph-mon[34908]: pgmap v150: 97 pgs: 97 active+clean; 39 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 51 op/s 2024-09-11T22:02:05.320 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:05 smithi001 ceph-mon[30935]: pgmap v150: 97 pgs: 97 active+clean; 39 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 51 op/s 2024-09-11T22:02:05.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:05 smithi161 ceph-mon[35615]: pgmap v150: 97 pgs: 97 active+clean; 39 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 51 op/s 2024-09-11T22:02:05.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.792+0000 7f48137fe640 1 -- 172.21.15.1:0/1128627679 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f480c1027a0 msgr2=0x7f480c1067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:05.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.792+0000 7f48137fe640 1 --2- 172.21.15.1:0/1128627679 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f480c1027a0 0x7f480c1067f0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f47fc009900 tx=0x7f47fc02a410 comp rx=0 tx=0).stop 2024-09-11T22:02:05.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.792+0000 7f48137fe640 1 -- 172.21.15.1:0/1128627679 shutdown_connections 2024-09-11T22:02:05.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.792+0000 7f48137fe640 1 --2- 172.21.15.1:0/1128627679 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f480c106f20 0x7f480c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:05.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.792+0000 7f48137fe640 1 --2- 172.21.15.1:0/1128627679 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f480c1027a0 0x7f480c1067f0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:05.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.792+0000 7f48137fe640 1 --2- 172.21.15.1:0/1128627679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f480c101df0 0x7f480c1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:05.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.792+0000 7f48137fe640 1 -- 172.21.15.1:0/1128627679 >> 172.21.15.1:0/1128627679 conn(0x7f480c0fd8f0 msgr2=0x7f480c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:05.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.792+0000 7f48137fe640 1 -- 172.21.15.1:0/1128627679 shutdown_connections 2024-09-11T22:02:05.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.792+0000 7f48137fe640 1 -- 172.21.15.1:0/1128627679 wait complete. 2024-09-11T22:02:05.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.793+0000 7f48137fe640 1 Processor -- start 2024-09-11T22:02:05.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.793+0000 7f48137fe640 1 -- start start 2024-09-11T22:02:05.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.793+0000 7f48137fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f480c101df0 0x7f480c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:05.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.793+0000 7f48137fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f480c1027a0 0x7f480c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:05.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.793+0000 7f48137fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f480c106f20 0x7f480c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:05.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.793+0000 7f48137fe640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f480c111fd0 con 0x7f480c1027a0 2024-09-11T22:02:05.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.793+0000 7f48137fe640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f480c111e50 con 0x7f480c106f20 2024-09-11T22:02:05.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.793+0000 7f48137fe640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f480c112150 con 0x7f480c101df0 2024-09-11T22:02:05.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.793+0000 7f4810d99640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f480c1027a0 0x7f480c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:05.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.793+0000 7f4810d99640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f480c1027a0 0x7f480c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44774/0 (socket says 172.21.15.1:44774) 2024-09-11T22:02:05.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.793+0000 7f481159a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f480c101df0 0x7f480c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:05.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.793+0000 7f4810d99640 1 -- 172.21.15.1:0/3163188264 learned_addr learned my addr 172.21.15.1:0/3163188264 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:05.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.793+0000 7f4811d9b640 1 --2- 172.21.15.1:0/3163188264 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f480c106f20 0x7f480c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:05.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.794+0000 7f4810d99640 1 -- 172.21.15.1:0/3163188264 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f480c101df0 msgr2=0x7f480c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:05.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.794+0000 7f4810d99640 1 --2- 172.21.15.1:0/3163188264 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f480c101df0 0x7f480c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:05.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.794+0000 7f4810d99640 1 -- 172.21.15.1:0/3163188264 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f480c106f20 msgr2=0x7f480c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:05.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.794+0000 7f4810d99640 1 --2- 172.21.15.1:0/3163188264 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f480c106f20 0x7f480c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:05.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.794+0000 7f4810d99640 1 -- 172.21.15.1:0/3163188264 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f480c1a69b0 con 0x7f480c1027a0 2024-09-11T22:02:05.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.794+0000 7f4810d99640 1 --2- 172.21.15.1:0/3163188264 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f480c1027a0 0x7f480c074a30 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f47fc004770 tx=0x7f47fc003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:05.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.794+0000 7f48027fc640 1 -- 172.21.15.1:0/3163188264 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47fc03f070 con 0x7f480c1027a0 2024-09-11T22:02:05.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.794+0000 7f48137fe640 1 -- 172.21.15.1:0/3163188264 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f480c1a6c40 con 0x7f480c1027a0 2024-09-11T22:02:05.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.794+0000 7f48027fc640 1 -- 172.21.15.1:0/3163188264 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47fc031ca0 con 0x7f480c1027a0 2024-09-11T22:02:05.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.794+0000 7f48027fc640 1 -- 172.21.15.1:0/3163188264 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47fc035030 con 0x7f480c1027a0 2024-09-11T22:02:05.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.794+0000 7f48137fe640 1 -- 172.21.15.1:0/3163188264 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f480c1a7170 con 0x7f480c1027a0 2024-09-11T22:02:05.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.797+0000 7f48027fc640 1 -- 172.21.15.1:0/3163188264 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f47fc042070 con 0x7f480c1027a0 2024-09-11T22:02:05.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.797+0000 7f4815bd7740 1 Processor -- start 2024-09-11T22:02:05.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.798+0000 7f4815bd7740 1 -- start start 2024-09-11T22:02:05.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.798+0000 7f4815bd7740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557840df0b40 0x557840def090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:05.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.798+0000 7f4815bd7740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557840def5d0 0x557840defa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:05.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.798+0000 7f4815bd7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557840dfa370 0x557840dfc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:05.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.798+0000 7f4815bd7740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x557840d552f0 con 0x557840def5d0 2024-09-11T22:02:05.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.798+0000 7f4815bd7740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x557840cfccf0 con 0x557840df0b40 2024-09-11T22:02:05.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.798+0000 7f4815bd7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x557840d512b0 con 0x557840dfa370 2024-09-11T22:02:05.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.798+0000 7f4811d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557840df0b40 0x557840def090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:05.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.798+0000 7f4811d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557840df0b40 0x557840def090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:50630/0 (socket says 172.21.15.1:50630) 2024-09-11T22:02:05.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.798+0000 7f4811d9b640 1 -- 172.21.15.1:0/1232530371 learned_addr learned my addr 172.21.15.1:0/1232530371 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:05.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.798+0000 7f481159a640 1 --2- 172.21.15.1:0/1232530371 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557840def5d0 0x557840defa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:05.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.798+0000 7f4810d99640 1 --2- 172.21.15.1:0/1232530371 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557840dfa370 0x557840dfc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:05.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.798+0000 7f4811d9b640 1 -- 172.21.15.1:0/1232530371 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557840dfa370 msgr2=0x557840dfc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:05.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.798+0000 7f4811d9b640 1 --2- 172.21.15.1:0/1232530371 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557840dfa370 0x557840dfc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:05.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.798+0000 7f4811d9b640 1 -- 172.21.15.1:0/1232530371 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557840def5d0 msgr2=0x557840defa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:05.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.798+0000 7f4811d9b640 1 --2- 172.21.15.1:0/1232530371 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557840def5d0 0x557840defa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:05.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.798+0000 7f4811d9b640 1 -- 172.21.15.1:0/1232530371 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557840ce13a0 con 0x557840df0b40 2024-09-11T22:02:05.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.799+0000 7f48027fc640 1 --2- 172.21.15.1:0/3163188264 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f47f80a2ef0 0x7f47f80a53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:05.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.799+0000 7f4811d9b640 1 --2- 172.21.15.1:0/1232530371 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557840df0b40 0x557840def090 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4804009f50 tx=0x7f48040118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=373974a3dd4c5f5a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:05.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.799+0000 7f481159a640 1 --2- 172.21.15.1:0/3163188264 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f47f80a2ef0 0x7f47f80a53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:05.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.799+0000 7f48027fc640 1 -- 172.21.15.1:0/3163188264 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f47fc03bb90 con 0x7f480c1027a0 2024-09-11T22:02:05.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.799+0000 7f47f77fe640 1 -- 172.21.15.1:0/1232530371 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4804004d20 con 0x557840df0b40 2024-09-11T22:02:05.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.799+0000 7f47f77fe640 1 -- 172.21.15.1:0/1232530371 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4804004050 con 0x557840df0b40 2024-09-11T22:02:05.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.799+0000 7f47f77fe640 1 -- 172.21.15.1:0/1232530371 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4804005280 con 0x557840df0b40 2024-09-11T22:02:05.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.800+0000 7f4815bd7740 1 -- 172.21.15.1:0/1232530371 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557840df0b40 msgr2=0x557840def090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:05.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.800+0000 7f4815bd7740 1 --2- 172.21.15.1:0/1232530371 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557840df0b40 0x557840def090 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4804009f50 tx=0x7f48040118d0 comp rx=0 tx=0).stop 2024-09-11T22:02:05.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.800+0000 7f481159a640 1 --2- 172.21.15.1:0/3163188264 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f47f80a2ef0 0x7f47f80a53b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x557840decac0 tx=0x7f4808005f90 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:05.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.800+0000 7f4815bd7740 1 -- 172.21.15.1:0/1232530371 shutdown_connections 2024-09-11T22:02:05.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.800+0000 7f4815bd7740 1 --2- 172.21.15.1:0/1232530371 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557840dfa370 0x557840dfc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:05.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.800+0000 7f4815bd7740 1 --2- 172.21.15.1:0/1232530371 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557840def5d0 0x557840defa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:05.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.800+0000 7f4815bd7740 1 --2- 172.21.15.1:0/1232530371 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557840df0b40 0x557840def090 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:05.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.800+0000 7f4815bd7740 1 -- 172.21.15.1:0/1232530371 >> 172.21.15.1:0/1232530371 conn(0x557840db8310 msgr2=0x557840debc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:05.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.800+0000 7f4815bd7740 1 -- 172.21.15.1:0/1232530371 shutdown_connections 2024-09-11T22:02:05.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.800+0000 7f4815bd7740 1 -- 172.21.15.1:0/1232530371 wait complete. 2024-09-11T22:02:05.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.801+0000 7f4815bd7740 1 -- start start 2024-09-11T22:02:05.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.801+0000 7f4815bd7740 1 Processor -- start 2024-09-11T22:02:05.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.801+0000 7f4815bd7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557840def5d0 0x557840df9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:05.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.801+0000 7f4815bd7740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557840df0b40 0x557840df6f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:05.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.801+0000 7f4815bd7740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557840dfa370 0x557840df74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:05.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.801+0000 7f4815bd7740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x557840dfd070 con 0x557840df0b40 2024-09-11T22:02:05.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.801+0000 7f4815bd7740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x557840d10060 con 0x557840dfa370 2024-09-11T22:02:05.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.801+0000 7f4815bd7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x557840dfd1f0 con 0x557840def5d0 2024-09-11T22:02:05.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.801+0000 7f4815bd7740 10 client.0 _ll_register_callbacks cb 0x557840a638c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:02:05.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.801+0000 7f4810d99640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557840df0b40 0x557840df6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:05.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.801+0000 7f481159a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557840def5d0 0x557840df9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:05.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.801+0000 7f4810d99640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557840df0b40 0x557840df6f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44784/0 (socket says 172.21.15.1:44784) 2024-09-11T22:02:05.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.801+0000 7f4810d99640 1 -- 172.21.15.1:0/399355297 learned_addr learned my addr 172.21.15.1:0/399355297 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:05.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.801+0000 7f4811d9b640 1 --2- 172.21.15.1:0/399355297 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557840dfa370 0x557840df74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:05.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.802+0000 7f4810d99640 1 -- 172.21.15.1:0/399355297 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557840def5d0 msgr2=0x557840df9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:05.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.802+0000 7f4810d99640 1 --2- 172.21.15.1:0/399355297 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557840def5d0 0x557840df9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:05.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.802+0000 7f4810d99640 1 -- 172.21.15.1:0/399355297 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557840dfa370 msgr2=0x557840df74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:05.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.802+0000 7f4810d99640 1 --2- 172.21.15.1:0/399355297 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557840dfa370 0x557840df74e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:05.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.802+0000 7f4810d99640 1 -- 172.21.15.1:0/399355297 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557840ac63c0 con 0x557840df0b40 2024-09-11T22:02:05.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.802+0000 7f481159a640 1 --2- 172.21.15.1:0/399355297 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557840def5d0 0x557840df9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:02:05.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.802+0000 7f4810d99640 1 --2- 172.21.15.1:0/399355297 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557840df0b40 0x557840df6f80 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f47fc0bb820 tx=0x7f47fc0f29e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feb4e93033a4179 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:05.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.802+0000 7f47f4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:02:05.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.802+0000 7f47f4ff9640 1 -- 172.21.15.1:0/399355297 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47fc049070 con 0x557840df0b40 2024-09-11T22:02:05.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.802+0000 7f47f4ff9640 1 -- 172.21.15.1:0/399355297 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f47fc0b8190 con 0x557840df0b40 2024-09-11T22:02:05.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.802+0000 7f4815bd7740 1 -- 172.21.15.1:0/399355297 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4804004050 con 0x557840df0b40 2024-09-11T22:02:05.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.802+0000 7f47f4ff9640 1 -- 172.21.15.1:0/399355297 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f47fc0bb480 con 0x557840df0b40 2024-09-11T22:02:05.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.802+0000 7f4815bd7740 10 client.16398 fetch_fsmap learned FSMap version 33 2024-09-11T22:02:05.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.802+0000 7f4815bd7740 1 -- 172.21.15.1:0/399355297 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x557840e08a80 con 0x557840df0b40 2024-09-11T22:02:05.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.803+0000 7f47f4ff9640 1 -- 172.21.15.1:0/399355297 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4804004050 con 0x557840df0b40 2024-09-11T22:02:05.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.803+0000 7f47f4ff9640 1 -- 172.21.15.1:0/399355297 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7f47fc04c060 con 0x557840df0b40 2024-09-11T22:02:05.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.803+0000 7f4815bd7740 10 client.16398 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T22:02:05.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.803+0000 7f4815bd7740 10 client.16398 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:02:05.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.803+0000 7f4815bd7740 20 client.16398 populate_metadata read hostname 'smithi001' 2024-09-11T22:02:05.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.803+0000 7f4815bd7740 1 --2- 172.21.15.1:0/399355297 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x557840e09f10 0x557840e0c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:05.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.803+0000 7f4815bd7740 4 client.16398 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:02:05.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.803+0000 7f4815bd7740 1 -- 172.21.15.1:0/399355297 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557840d62860 con 0x557840e09f10 2024-09-11T22:02:05.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.803+0000 7f481159a640 1 --2- 172.21.15.1:0/399355297 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x557840e09f10 0x557840e0c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:05.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.804+0000 7f481159a640 1 --2- 172.21.15.1:0/399355297 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x557840e09f10 0x557840e0c330 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1be108e4369b67c server_cookie=482e67869c651104 in_seq=0 out_seq=0 2024-09-11T22:02:05.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.804+0000 7f47f4ff9640 10 client.16398 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:02:05.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.807+0000 7f47f4ff9640 1 -- 172.21.15.1:0/399355297 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557840d62860 con 0x557840e09f10 2024-09-11T22:02:05.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.807+0000 7f47f4ff9640 10 client.16398 handle_command_reply: tid=0 2024-09-11T22:02:05.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.807+0000 7f4815bd7740 1 client.16398 shutdown 2024-09-11T22:02:05.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.807+0000 7f4815bd7740 1 -- 172.21.15.1:0/399355297 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557840df0b40 msgr2=0x557840df6f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:05.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.807+0000 7f4815bd7740 1 --2- 172.21.15.1:0/399355297 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557840df0b40 0x557840df6f80 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f47fc0bb820 tx=0x7f47fc0f29e0 comp rx=0 tx=0).stop 2024-09-11T22:02:05.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.807+0000 7f4815bd7740 1 -- 172.21.15.1:0/399355297 shutdown_connections 2024-09-11T22:02:05.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.807+0000 7f4815bd7740 1 --2- 172.21.15.1:0/399355297 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x557840e09f10 0x557840e0c330 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:05.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.807+0000 7f4815bd7740 1 --2- 172.21.15.1:0/399355297 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557840dfa370 0x557840df74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:05.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.807+0000 7f4815bd7740 1 --2- 172.21.15.1:0/399355297 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557840df0b40 0x557840df6f80 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:05.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.807+0000 7f4815bd7740 1 --2- 172.21.15.1:0/399355297 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557840def5d0 0x557840df9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:05.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.808+0000 7f4815bd7740 1 -- 172.21.15.1:0/399355297 >> 172.21.15.1:0/399355297 conn(0x557840db8310 msgr2=0x557840df1930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:05.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.808+0000 7f47f4ff9640 0 client.16398 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:02:05.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.808+0000 7f4815bd7740 1 -- 172.21.15.1:0/399355297 shutdown_connections 2024-09-11T22:02:05.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.808+0000 7f4815bd7740 1 -- 172.21.15.1:0/399355297 wait complete. 2024-09-11T22:02:05.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.808+0000 7f4815bd7740 20 client.16398 trim_cache size 0 max 16384 2024-09-11T22:02:05.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.826+0000 7f4815bd7740 1 Processor -- start 2024-09-11T22:02:05.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.826+0000 7f4815bd7740 1 -- start start 2024-09-11T22:02:05.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.826+0000 7f4815bd7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557840def5d0 0x557840deb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:05.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.826+0000 7f4815bd7740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557840df0b40 0x557840deba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:05.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.826+0000 7f4815bd7740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557840dfa370 0x557840e11bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:05.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.826+0000 7f4815bd7740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f480c1afbf0 con 0x557840df0b40 2024-09-11T22:02:05.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.826+0000 7f4815bd7740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f480c1b1820 con 0x557840dfa370 2024-09-11T22:02:05.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.826+0000 7f4815bd7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f480c1b02c0 con 0x557840def5d0 2024-09-11T22:02:05.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.826+0000 7f4810d99640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557840df0b40 0x557840deba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:05.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.826+0000 7f4811d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557840dfa370 0x557840e11bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:05.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.826+0000 7f4810d99640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557840df0b40 0x557840deba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44800/0 (socket says 172.21.15.1:44800) 2024-09-11T22:02:05.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.826+0000 7f4810d99640 1 -- 172.21.15.1:0/238260587 learned_addr learned my addr 172.21.15.1:0/238260587 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:05.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.826+0000 7f481159a640 1 --2- 172.21.15.1:0/238260587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557840def5d0 0x557840deb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:05.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.826+0000 7f4810d99640 1 -- 172.21.15.1:0/238260587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557840def5d0 msgr2=0x557840deb4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:05.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.826+0000 7f4810d99640 1 --2- 172.21.15.1:0/238260587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557840def5d0 0x557840deb4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:05.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.826+0000 7f4810d99640 1 -- 172.21.15.1:0/238260587 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557840dfa370 msgr2=0x557840e11bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:05.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.826+0000 7f4810d99640 1 --2- 172.21.15.1:0/238260587 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557840dfa370 0x557840e11bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:05.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.826+0000 7f4810d99640 1 -- 172.21.15.1:0/238260587 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557840df2120 con 0x557840df0b40 2024-09-11T22:02:05.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.827+0000 7f4810d99640 1 --2- 172.21.15.1:0/238260587 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557840df0b40 0x557840deba30 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f47fc03b5e0 tx=0x7f47fc03b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2fb3ad54bbe0d97 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:05.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.827+0000 7f47f6ffd640 1 -- 172.21.15.1:0/238260587 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47fc049070 con 0x557840df0b40 2024-09-11T22:02:05.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.827+0000 7f47f6ffd640 1 -- 172.21.15.1:0/238260587 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47fc0bbb20 con 0x557840df0b40 2024-09-11T22:02:05.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.827+0000 7f47f6ffd640 1 -- 172.21.15.1:0/238260587 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47fc005d50 con 0x557840df0b40 2024-09-11T22:02:05.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.828+0000 7f4815bd7740 1 -- 172.21.15.1:0/238260587 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557840df0b40 msgr2=0x557840deba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:05.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.828+0000 7f4815bd7740 1 --2- 172.21.15.1:0/238260587 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557840df0b40 0x557840deba30 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f47fc03b5e0 tx=0x7f47fc03b890 comp rx=0 tx=0).stop 2024-09-11T22:02:05.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.828+0000 7f4815bd7740 1 -- 172.21.15.1:0/238260587 shutdown_connections 2024-09-11T22:02:05.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.828+0000 7f4815bd7740 1 --2- 172.21.15.1:0/238260587 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557840dfa370 0x557840e11bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:05.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.828+0000 7f4815bd7740 1 --2- 172.21.15.1:0/238260587 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557840df0b40 0x557840deba30 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:05.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.828+0000 7f4815bd7740 1 --2- 172.21.15.1:0/238260587 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557840def5d0 0x557840deb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:05.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.828+0000 7f4815bd7740 1 -- 172.21.15.1:0/238260587 >> 172.21.15.1:0/238260587 conn(0x557840db8310 msgr2=0x557840df9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:05.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.828+0000 7f4815bd7740 1 -- 172.21.15.1:0/238260587 shutdown_connections 2024-09-11T22:02:05.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.828+0000 7f4815bd7740 1 -- 172.21.15.1:0/238260587 wait complete. 2024-09-11T22:02:05.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.828+0000 7f4815bd7740 1 -- start start 2024-09-11T22:02:05.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.828+0000 7f4815bd7740 1 Processor -- start 2024-09-11T22:02:05.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.829+0000 7f4815bd7740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557840def5d0 0x557840e02d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:05.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.829+0000 7f4815bd7740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557840df0b40 0x557840dfe110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:05.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.829+0000 7f4815bd7740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557840dfa370 0x557840dfe8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:05.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.829+0000 7f4815bd7740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f480c1b1090 con 0x557840dfa370 2024-09-11T22:02:05.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.829+0000 7f4815bd7740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f480c1b0990 con 0x557840df0b40 2024-09-11T22:02:05.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.829+0000 7f4815bd7740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f480c1adbd0 con 0x557840def5d0 2024-09-11T22:02:05.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.829+0000 7f4815bd7740 10 client.0 _ll_register_callbacks cb 0x7f47fc005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:02:05.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.829+0000 7f4811d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557840dfa370 0x557840dfe8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:05.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.829+0000 7f481159a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557840def5d0 0x557840e02d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:05.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.829+0000 7f4810d99640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557840df0b40 0x557840dfe110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:05.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.829+0000 7f481159a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557840def5d0 0x557840e02d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:38716/0 (socket says 172.21.15.1:38716) 2024-09-11T22:02:05.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.829+0000 7f481159a640 1 -- 172.21.15.1:0/2876976939 learned_addr learned my addr 172.21.15.1:0/2876976939 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:05.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.829+0000 7f4810d99640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557840df0b40 0x557840dfe110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:50660/0 (socket says 172.21.15.1:50660) 2024-09-11T22:02:05.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.829+0000 7f481159a640 1 -- 172.21.15.1:0/2876976939 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557840df0b40 msgr2=0x557840dfe110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:05.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.829+0000 7f481159a640 1 --2- 172.21.15.1:0/2876976939 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557840df0b40 0x557840dfe110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:05.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.829+0000 7f481159a640 1 -- 172.21.15.1:0/2876976939 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557840dfa370 msgr2=0x557840dfe8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:05.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.829+0000 7f481159a640 1 --2- 172.21.15.1:0/2876976939 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557840dfa370 0x557840dfe8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:05.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.829+0000 7f481159a640 1 -- 172.21.15.1:0/2876976939 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557840e73b80 con 0x557840def5d0 2024-09-11T22:02:05.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.829+0000 7f4811d9b640 1 --2- 172.21.15.1:0/2876976939 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557840dfa370 0x557840dfe8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:02:05.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.829+0000 7f4810d99640 1 --2- 172.21.15.1:0/2876976939 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557840df0b40 0x557840dfe110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:02:05.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.830+0000 7f481159a640 1 --2- 172.21.15.1:0/2876976939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557840def5d0 0x557840e02d90 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x557840e0cb50 tx=0x7f48080046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=713b015fecc44224 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:05.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.830+0000 7f47f4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-11T22:02:05.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.830+0000 7f47f4ff9640 1 -- 172.21.15.1:0/2876976939 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4808005020 con 0x557840def5d0 2024-09-11T22:02:05.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.830+0000 7f47f4ff9640 1 -- 172.21.15.1:0/2876976939 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f48080051c0 con 0x557840def5d0 2024-09-11T22:02:05.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.830+0000 7f4815bd7740 1 -- 172.21.15.1:0/2876976939 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47fc0bbb20 con 0x557840def5d0 2024-09-11T22:02:05.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.830+0000 7f47f4ff9640 1 -- 172.21.15.1:0/2876976939 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f48080053e0 con 0x557840def5d0 2024-09-11T22:02:05.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.830+0000 7f4815bd7740 10 client.25912 fetch_fsmap learned FSMap version 33 2024-09-11T22:02:05.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.830+0000 7f4815bd7740 1 -- 172.21.15.1:0/2876976939 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x557840e74da0 con 0x557840def5d0 2024-09-11T22:02:05.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.830+0000 7f47f4ff9640 1 -- 172.21.15.1:0/2876976939 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47fc0bbb20 con 0x557840def5d0 2024-09-11T22:02:05.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.831+0000 7f47f4ff9640 1 -- 172.21.15.1:0/2876976939 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7f4808039030 con 0x557840def5d0 2024-09-11T22:02:05.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.831+0000 7f4815bd7740 10 client.25912 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T22:02:05.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.831+0000 7f4815bd7740 10 client.25912 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:02:05.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.831+0000 7f4815bd7740 20 client.25912 populate_metadata read hostname 'smithi001' 2024-09-11T22:02:05.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.831+0000 7f4815bd7740 1 --2- 172.21.15.1:0/2876976939 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x557840e75630 0x557840e77a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:05.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.831+0000 7f4815bd7740 4 client.25912 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:02:05.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.831+0000 7f4815bd7740 1 -- 172.21.15.1:0/2876976939 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557840d5cd70 con 0x557840e75630 2024-09-11T22:02:05.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.831+0000 7f4810d99640 1 --2- 172.21.15.1:0/2876976939 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x557840e75630 0x557840e77a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:05.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.831+0000 7f47f4ff9640 10 client.25912 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:02:05.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.831+0000 7f4810d99640 1 --2- 172.21.15.1:0/2876976939 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x557840e75630 0x557840e77a50 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ceba710829bf28da server_cookie=46035a29741c9ae9 in_seq=0 out_seq=0 2024-09-11T22:02:05.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.834+0000 7f47f4ff9640 1 -- 172.21.15.1:0/2876976939 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557840d5cd70 con 0x557840e75630 2024-09-11T22:02:05.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.834+0000 7f47f4ff9640 10 client.25912 handle_command_reply: tid=0 2024-09-11T22:02:05.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.834+0000 7f4815bd7740 1 client.25912 shutdown 2024-09-11T22:02:05.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.834+0000 7f4815bd7740 1 -- 172.21.15.1:0/2876976939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557840def5d0 msgr2=0x557840e02d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:05.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.834+0000 7f4815bd7740 1 --2- 172.21.15.1:0/2876976939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557840def5d0 0x557840e02d90 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x557840e0cb50 tx=0x7f48080046c0 comp rx=0 tx=0).stop 2024-09-11T22:02:05.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.834+0000 7f4815bd7740 1 -- 172.21.15.1:0/2876976939 shutdown_connections 2024-09-11T22:02:05.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.834+0000 7f4815bd7740 1 --2- 172.21.15.1:0/2876976939 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x557840e75630 0x557840e77a50 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:05.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.834+0000 7f4815bd7740 1 --2- 172.21.15.1:0/2876976939 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557840dfa370 0x557840dfe8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:05.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.834+0000 7f4815bd7740 1 --2- 172.21.15.1:0/2876976939 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557840df0b40 0x557840dfe110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:05.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.834+0000 7f47f4ff9640 0 client.25912 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:02:05.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.834+0000 7f4815bd7740 1 --2- 172.21.15.1:0/2876976939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557840def5d0 0x557840e02d90 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:05.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.834+0000 7f4815bd7740 1 -- 172.21.15.1:0/2876976939 >> 172.21.15.1:0/2876976939 conn(0x557840db8310 msgr2=0x557840df1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:05.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.834+0000 7f4815bd7740 1 -- 172.21.15.1:0/2876976939 shutdown_connections 2024-09-11T22:02:05.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.834+0000 7f4815bd7740 1 -- 172.21.15.1:0/2876976939 wait complete. 2024-09-11T22:02:05.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.834+0000 7f4815bd7740 20 client.25912 trim_cache size 0 max 16384 2024-09-11T22:02:05.852 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-11T22:02:05.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.835+0000 7f48137fe640 1 -- 172.21.15.1:0/3163188264 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f47f80a2ef0 msgr2=0x7f47f80a53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:05.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.835+0000 7f48137fe640 1 --2- 172.21.15.1:0/3163188264 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f47f80a2ef0 0x7f47f80a53b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x557840decac0 tx=0x7f4808005f90 comp rx=0 tx=0).stop 2024-09-11T22:02:05.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.835+0000 7f48137fe640 1 -- 172.21.15.1:0/3163188264 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f480c1027a0 msgr2=0x7f480c074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:05.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.835+0000 7f48137fe640 1 --2- 172.21.15.1:0/3163188264 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f480c1027a0 0x7f480c074a30 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f47fc004770 tx=0x7f47fc003c10 comp rx=0 tx=0).stop 2024-09-11T22:02:05.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.835+0000 7f48137fe640 1 -- 172.21.15.1:0/3163188264 shutdown_connections 2024-09-11T22:02:05.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.835+0000 7f48137fe640 1 --2- 172.21.15.1:0/3163188264 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f47f80a2ef0 0x7f47f80a53b0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:05.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.835+0000 7f48137fe640 1 --2- 172.21.15.1:0/3163188264 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f480c106f20 0x7f480c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:05.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.835+0000 7f48137fe640 1 --2- 172.21.15.1:0/3163188264 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f480c1027a0 0x7f480c074a30 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:05.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.835+0000 7f48137fe640 1 --2- 172.21.15.1:0/3163188264 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f480c101df0 0x7f480c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:05.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.835+0000 7f48137fe640 1 -- 172.21.15.1:0/3163188264 >> 172.21.15.1:0/3163188264 conn(0x7f480c0fd8f0 msgr2=0x7f480c0fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:05.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.835+0000 7f48137fe640 1 -- 172.21.15.1:0/3163188264 shutdown_connections 2024-09-11T22:02:05.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:05.835+0000 7f48137fe640 1 -- 172.21.15.1:0/3163188264 wait complete. 2024-09-11T22:02:07.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:07 smithi001 ceph-mon[30935]: pgmap v151: 97 pgs: 97 active+clean; 53 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.3 MiB/s wr, 1.12k op/s 2024-09-11T22:02:07.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:07 smithi053 ceph-mon[34908]: pgmap v151: 97 pgs: 97 active+clean; 53 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.3 MiB/s wr, 1.12k op/s 2024-09-11T22:02:07.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:07 smithi161 ceph-mon[35615]: pgmap v151: 97 pgs: 97 active+clean; 53 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 6.3 MiB/s wr, 1.12k op/s 2024-09-11T22:02:07.897 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T22:02:07.897 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T22:02:08.176 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:02:08.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.669+0000 7fd05312a640 1 -- 172.21.15.1:0/2441459937 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd04c1036e0 msgr2=0x7fd04c107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:08.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.669+0000 7fd05312a640 1 --2- 172.21.15.1:0/2441459937 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd04c1036e0 0x7fd04c107b90 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fd038009930 tx=0x7fd03802a420 comp rx=0 tx=0).stop 2024-09-11T22:02:08.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.669+0000 7fd05312a640 1 -- 172.21.15.1:0/2441459937 shutdown_connections 2024-09-11T22:02:08.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.669+0000 7fd05312a640 1 --2- 172.21.15.1:0/2441459937 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd04c1082a0 0x7fd04c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:08.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.669+0000 7fd05312a640 1 --2- 172.21.15.1:0/2441459937 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd04c1036e0 0x7fd04c107b90 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:08.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.669+0000 7fd05312a640 1 --2- 172.21.15.1:0/2441459937 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd04c102d30 0x7fd04c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:08.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.669+0000 7fd05312a640 1 -- 172.21.15.1:0/2441459937 >> 172.21.15.1:0/2441459937 conn(0x7fd04c0fc7d0 msgr2=0x7fd04c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:08.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.669+0000 7fd05312a640 1 -- 172.21.15.1:0/2441459937 shutdown_connections 2024-09-11T22:02:08.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.669+0000 7fd05312a640 1 -- 172.21.15.1:0/2441459937 wait complete. 2024-09-11T22:02:08.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.670+0000 7fd05312a640 1 Processor -- start 2024-09-11T22:02:08.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.670+0000 7fd05312a640 1 -- start start 2024-09-11T22:02:08.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.670+0000 7fd05312a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd04c102d30 0x7fd04c198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:08.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.670+0000 7fd05312a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd04c1036e0 0x7fd04c198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:08.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.670+0000 7fd05312a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd04c1082a0 0x7fd04c19ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:08.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.670+0000 7fd05312a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd04c1120a0 con 0x7fd04c1082a0 2024-09-11T22:02:08.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.670+0000 7fd05312a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd04c111f20 con 0x7fd04c1036e0 2024-09-11T22:02:08.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.670+0000 7fd05312a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd04c112220 con 0x7fd04c102d30 2024-09-11T22:02:08.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.670+0000 7fd0516c7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd04c1082a0 0x7fd04c19ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:08.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.670+0000 7fd0516c7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd04c1082a0 0x7fd04c19ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49324/0 (socket says 172.21.15.1:49324) 2024-09-11T22:02:08.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.670+0000 7fd0516c7640 1 -- 172.21.15.1:0/2268461192 learned_addr learned my addr 172.21.15.1:0/2268461192 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:08.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.670+0000 7fd04bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd04c1036e0 0x7fd04c198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:08.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.670+0000 7fd050ec6640 1 --2- 172.21.15.1:0/2268461192 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd04c102d30 0x7fd04c198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:08.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.671+0000 7fd0516c7640 1 -- 172.21.15.1:0/2268461192 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd04c102d30 msgr2=0x7fd04c198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:08.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.671+0000 7fd0516c7640 1 --2- 172.21.15.1:0/2268461192 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd04c102d30 0x7fd04c198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:08.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.671+0000 7fd0516c7640 1 -- 172.21.15.1:0/2268461192 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd04c1036e0 msgr2=0x7fd04c198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:08.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.671+0000 7fd0516c7640 1 --2- 172.21.15.1:0/2268461192 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd04c1036e0 0x7fd04c198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:08.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.671+0000 7fd0516c7640 1 -- 172.21.15.1:0/2268461192 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd04c19d510 con 0x7fd04c1082a0 2024-09-11T22:02:08.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.671+0000 7fd0516c7640 1 --2- 172.21.15.1:0/2268461192 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd04c1082a0 0x7fd04c19ce30 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fd04400cce0 tx=0x7fd04400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:08.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.671+0000 7fd049ffb640 1 -- 172.21.15.1:0/2268461192 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd04400f450 con 0x7fd04c1082a0 2024-09-11T22:02:08.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.671+0000 7fd049ffb640 1 -- 172.21.15.1:0/2268461192 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd04400fad0 con 0x7fd04c1082a0 2024-09-11T22:02:08.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.671+0000 7fd049ffb640 1 -- 172.21.15.1:0/2268461192 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd044018c10 con 0x7fd04c1082a0 2024-09-11T22:02:08.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.671+0000 7fd05312a640 1 -- 172.21.15.1:0/2268461192 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd04c19d800 con 0x7fd04c1082a0 2024-09-11T22:02:08.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.671+0000 7fd05312a640 1 -- 172.21.15.1:0/2268461192 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd04c1a50e0 con 0x7fd04c1082a0 2024-09-11T22:02:08.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.674+0000 7fd049ffb640 1 -- 172.21.15.1:0/2268461192 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7fd04400f5f0 con 0x7fd04c1082a0 2024-09-11T22:02:08.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.674+0000 7fd055503740 1 Processor -- start 2024-09-11T22:02:08.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.674+0000 7fd055503740 1 -- start start 2024-09-11T22:02:08.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.674+0000 7fd055503740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56422b3baa20 0x56422b3bae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:08.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.674+0000 7fd055503740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56422b3bb340 0x56422b3c9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:08.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.674+0000 7fd055503740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56422b3ca150 0x56422b3cc540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:08.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.674+0000 7fd055503740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56422b320280 con 0x56422b3baa20 2024-09-11T22:02:08.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.674+0000 7fd055503740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56422b2c7b10 con 0x56422b3ca150 2024-09-11T22:02:08.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.674+0000 7fd055503740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56422b31c240 con 0x56422b3bb340 2024-09-11T22:02:08.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.675+0000 7fd04bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56422b3baa20 0x56422b3bae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:08.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.675+0000 7fd04bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56422b3baa20 0x56422b3bae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49326/0 (socket says 172.21.15.1:49326) 2024-09-11T22:02:08.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.675+0000 7fd04bfff640 1 -- 172.21.15.1:0/3922025641 learned_addr learned my addr 172.21.15.1:0/3922025641 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:08.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.675+0000 7fd04bfff640 1 --2- 172.21.15.1:0/3922025641 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56422b3ca150 0x56422b3cc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:08.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.675+0000 7fd049ffb640 1 --2- 172.21.15.1:0/2268461192 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fd03c0a2cf0 0x7fd03c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:08.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.675+0000 7fd050ec6640 1 --2- 172.21.15.1:0/3922025641 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56422b3bb340 0x56422b3c9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:08.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.675+0000 7fd04bfff640 1 -- 172.21.15.1:0/3922025641 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56422b3bb340 msgr2=0x56422b3c9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:08.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.675+0000 7fd04bfff640 1 --2- 172.21.15.1:0/3922025641 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56422b3bb340 0x56422b3c9c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:08.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.675+0000 7fd04bfff640 1 -- 172.21.15.1:0/3922025641 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56422b3ca150 msgr2=0x56422b3cc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:08.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.675+0000 7fd04bfff640 1 --2- 172.21.15.1:0/3922025641 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56422b3ca150 0x56422b3cc540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:08.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.675+0000 7fd04bfff640 1 -- 172.21.15.1:0/3922025641 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56422b2d41f0 con 0x56422b3baa20 2024-09-11T22:02:08.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.675+0000 7fd0516c7640 1 --2- 172.21.15.1:0/2268461192 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fd03c0a2cf0 0x7fd03c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:08.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.675+0000 7fd049ffb640 1 -- 172.21.15.1:0/2268461192 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7fd0440cfa10 con 0x7fd04c1082a0 2024-09-11T22:02:08.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.675+0000 7fd04bfff640 1 --2- 172.21.15.1:0/3922025641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56422b3baa20 0x56422b3bae00 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fd038009900 tx=0x7fd038000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d3fa260247dc80e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:08.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.675+0000 7fd036ffd640 1 -- 172.21.15.1:0/3922025641 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd038035040 con 0x56422b3baa20 2024-09-11T22:02:08.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.676+0000 7fd0516c7640 1 --2- 172.21.15.1:0/2268461192 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fd03c0a2cf0 0x7fd03c0a51b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd0440cd1d0 tx=0x7fd04406deb0 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:08.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.676+0000 7fd036ffd640 1 -- 172.21.15.1:0/3922025641 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0380035f0 con 0x56422b3baa20 2024-09-11T22:02:08.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.676+0000 7fd036ffd640 1 -- 172.21.15.1:0/3922025641 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03803a650 con 0x56422b3baa20 2024-09-11T22:02:08.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.676+0000 7fd055503740 1 -- 172.21.15.1:0/3922025641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56422b3baa20 msgr2=0x56422b3bae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:08.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.676+0000 7fd055503740 1 --2- 172.21.15.1:0/3922025641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56422b3baa20 0x56422b3bae00 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fd038009900 tx=0x7fd038000bc0 comp rx=0 tx=0).stop 2024-09-11T22:02:08.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.676+0000 7fd055503740 1 -- 172.21.15.1:0/3922025641 shutdown_connections 2024-09-11T22:02:08.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.676+0000 7fd055503740 1 --2- 172.21.15.1:0/3922025641 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56422b3ca150 0x56422b3cc540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:08.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.676+0000 7fd055503740 1 --2- 172.21.15.1:0/3922025641 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56422b3bb340 0x56422b3c9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:08.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.676+0000 7fd055503740 1 --2- 172.21.15.1:0/3922025641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56422b3baa20 0x56422b3bae00 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:08.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.676+0000 7fd055503740 1 -- 172.21.15.1:0/3922025641 >> 172.21.15.1:0/3922025641 conn(0x56422b3b70f0 msgr2=0x56422b3b74d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:08.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.677+0000 7fd055503740 1 -- 172.21.15.1:0/3922025641 shutdown_connections 2024-09-11T22:02:08.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.677+0000 7fd055503740 1 -- 172.21.15.1:0/3922025641 wait complete. 2024-09-11T22:02:08.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.677+0000 7fd055503740 1 -- start start 2024-09-11T22:02:08.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.677+0000 7fd055503740 1 Processor -- start 2024-09-11T22:02:08.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.678+0000 7fd055503740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56422b3baa20 0x56422b3c70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:08.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.678+0000 7fd055503740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56422b3bb340 0x56422b3c75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:08.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.678+0000 7fd055503740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56422b3ca150 0x56422b3c7b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:08.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.678+0000 7fd04bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56422b3baa20 0x56422b3c70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:08.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.678+0000 7fd055503740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56422b3cccc0 con 0x56422b3baa20 2024-09-11T22:02:08.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.678+0000 7fd04bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56422b3baa20 0x56422b3c70b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49340/0 (socket says 172.21.15.1:49340) 2024-09-11T22:02:08.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.678+0000 7fd050ec6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56422b3bb340 0x56422b3c75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:08.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.678+0000 7fd04bfff640 1 -- 172.21.15.1:0/3616777408 learned_addr learned my addr 172.21.15.1:0/3616777408 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:08.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.678+0000 7fd055503740 1 -- 172.21.15.1:0/3616777408 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56422b2daf10 con 0x56422b3ca150 2024-09-11T22:02:08.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.678+0000 7fd055503740 1 -- 172.21.15.1:0/3616777408 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56422b3cce40 con 0x56422b3bb340 2024-09-11T22:02:08.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.678+0000 7fd055503740 10 client.0 _ll_register_callbacks cb 0x56422b2a3a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:02:08.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.678+0000 7fd04bfff640 1 --2- 172.21.15.1:0/3616777408 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56422b3ca150 0x56422b3c7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:08.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.679+0000 7fd04bfff640 1 -- 172.21.15.1:0/3616777408 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56422b3bb340 msgr2=0x56422b3c75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:08.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.679+0000 7fd04bfff640 1 --2- 172.21.15.1:0/3616777408 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56422b3bb340 0x56422b3c75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:08.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.679+0000 7fd04bfff640 1 -- 172.21.15.1:0/3616777408 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56422b3ca150 msgr2=0x56422b3c7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:08.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.679+0000 7fd04bfff640 1 --2- 172.21.15.1:0/3616777408 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56422b3ca150 0x56422b3c7b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:08.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.679+0000 7fd04bfff640 1 -- 172.21.15.1:0/3616777408 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56422b0916a0 con 0x56422b3baa20 2024-09-11T22:02:08.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.679+0000 7fd04bfff640 1 --2- 172.21.15.1:0/3616777408 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56422b3baa20 0x56422b3c70b0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fd0380040b0 tx=0x7fd038031510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5a4d98ddcfd548f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:08.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.679+0000 7fd02bfff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:02:08.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.679+0000 7fd02bfff640 1 -- 172.21.15.1:0/3616777408 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd038035040 con 0x56422b3baa20 2024-09-11T22:02:08.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.679+0000 7fd02bfff640 1 -- 172.21.15.1:0/3616777408 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7fd0380037a0 con 0x56422b3baa20 2024-09-11T22:02:08.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.679+0000 7fd055503740 1 -- 172.21.15.1:0/3616777408 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0380035f0 con 0x56422b3baa20 2024-09-11T22:02:08.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.680+0000 7fd02bfff640 1 -- 172.21.15.1:0/3616777408 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd0380039a0 con 0x56422b3baa20 2024-09-11T22:02:08.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.680+0000 7fd055503740 10 client.16416 fetch_fsmap learned FSMap version 33 2024-09-11T22:02:08.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.680+0000 7fd055503740 1 -- 172.21.15.1:0/3616777408 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56422b3cccc0 con 0x56422b3baa20 2024-09-11T22:02:08.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.680+0000 7fd02bfff640 1 -- 172.21.15.1:0/3616777408 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd038005510 con 0x56422b3baa20 2024-09-11T22:02:08.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.681+0000 7fd02bfff640 1 -- 172.21.15.1:0/3616777408 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7fd038054030 con 0x56422b3baa20 2024-09-11T22:02:08.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.681+0000 7fd055503740 10 client.16416 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T22:02:08.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.681+0000 7fd055503740 10 client.16416 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:02:08.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.681+0000 7fd055503740 20 client.16416 populate_metadata read hostname 'smithi001' 2024-09-11T22:02:08.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.681+0000 7fd055503740 1 --2- 172.21.15.1:0/3616777408 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x56422b3d9ac0 0x56422b3dbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:08.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.681+0000 7fd055503740 4 client.16416 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:02:08.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.681+0000 7fd055503740 1 -- 172.21.15.1:0/3616777408 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd0380035f0 con 0x56422b3d9ac0 2024-09-11T22:02:08.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.681+0000 7fd050ec6640 1 --2- 172.21.15.1:0/3616777408 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x56422b3d9ac0 0x56422b3dbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:08.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.681+0000 7fd02bfff640 10 client.16416 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:02:08.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.681+0000 7fd050ec6640 1 --2- 172.21.15.1:0/3616777408 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x56422b3d9ac0 0x56422b3dbeb0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f25c5bf751c606c8 server_cookie=536c23d3f3993373 in_seq=0 out_seq=0 2024-09-11T22:02:08.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.683+0000 7fd02bfff640 1 -- 172.21.15.1:0/3616777408 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fd0380035f0 con 0x56422b3d9ac0 2024-09-11T22:02:08.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.683+0000 7fd02bfff640 10 client.16416 handle_command_reply: tid=0 2024-09-11T22:02:08.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.683+0000 7fd055503740 1 client.16416 shutdown 2024-09-11T22:02:08.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.683+0000 7fd055503740 1 -- 172.21.15.1:0/3616777408 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56422b3baa20 msgr2=0x56422b3c70b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:08.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.683+0000 7fd055503740 1 --2- 172.21.15.1:0/3616777408 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56422b3baa20 0x56422b3c70b0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fd0380040b0 tx=0x7fd038031510 comp rx=0 tx=0).stop 2024-09-11T22:02:08.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.684+0000 7fd055503740 1 -- 172.21.15.1:0/3616777408 shutdown_connections 2024-09-11T22:02:08.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.684+0000 7fd055503740 1 --2- 172.21.15.1:0/3616777408 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x56422b3d9ac0 0x56422b3dbeb0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:08.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.684+0000 7fd055503740 1 --2- 172.21.15.1:0/3616777408 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56422b3ca150 0x56422b3c7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:08.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.684+0000 7fd055503740 1 --2- 172.21.15.1:0/3616777408 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56422b3bb340 0x56422b3c75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:08.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.684+0000 7fd055503740 1 --2- 172.21.15.1:0/3616777408 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56422b3baa20 0x56422b3c70b0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:08.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.684+0000 7fd055503740 1 -- 172.21.15.1:0/3616777408 >> 172.21.15.1:0/3616777408 conn(0x56422b3b70f0 msgr2=0x56422b3bce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:08.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.684+0000 7fd02bfff640 0 client.16416 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:02:08.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.684+0000 7fd055503740 1 -- 172.21.15.1:0/3616777408 shutdown_connections 2024-09-11T22:02:08.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.684+0000 7fd055503740 1 -- 172.21.15.1:0/3616777408 wait complete. 2024-09-11T22:02:08.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.684+0000 7fd055503740 20 client.16416 trim_cache size 0 max 16384 2024-09-11T22:02:08.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.706+0000 7fd055503740 1 Processor -- start 2024-09-11T22:02:08.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.706+0000 7fd055503740 1 -- start start 2024-09-11T22:02:08.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.706+0000 7fd055503740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56422b3baa20 0x56422b3e1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:08.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.706+0000 7fd055503740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56422b3bb340 0x56422b3e1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:08.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.706+0000 7fd055503740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56422b3ca150 0x56422b3c4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:08.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.706+0000 7fd055503740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd04c1ab6b0 con 0x56422b3baa20 2024-09-11T22:02:08.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.706+0000 7fd055503740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd04c1ad2e0 con 0x56422b3ca150 2024-09-11T22:02:08.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.706+0000 7fd055503740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd04c1abd80 con 0x56422b3bb340 2024-09-11T22:02:08.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.707+0000 7fd04bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56422b3baa20 0x56422b3e1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:08.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.707+0000 7fd04bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56422b3baa20 0x56422b3e1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49342/0 (socket says 172.21.15.1:49342) 2024-09-11T22:02:08.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.707+0000 7fd04bfff640 1 -- 172.21.15.1:0/1314889806 learned_addr learned my addr 172.21.15.1:0/1314889806 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:08.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.707+0000 7fd04bfff640 1 --2- 172.21.15.1:0/1314889806 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56422b3ca150 0x56422b3c4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:08.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.707+0000 7fd04bfff640 1 -- 172.21.15.1:0/1314889806 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56422b3bb340 msgr2=0x56422b3e1a50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T22:02:08.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.707+0000 7fd04bfff640 1 --2- 172.21.15.1:0/1314889806 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56422b3bb340 0x56422b3e1a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:08.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.707+0000 7fd04bfff640 1 -- 172.21.15.1:0/1314889806 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56422b3ca150 msgr2=0x56422b3c4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:08.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.707+0000 7fd04bfff640 1 --2- 172.21.15.1:0/1314889806 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56422b3ca150 0x56422b3c4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:08.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.707+0000 7fd04bfff640 1 -- 172.21.15.1:0/1314889806 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56422b3b2830 con 0x56422b3baa20 2024-09-11T22:02:08.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.707+0000 7fd04bfff640 1 --2- 172.21.15.1:0/1314889806 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56422b3baa20 0x56422b3e1510 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fd0380040e0 tx=0x7fd038000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=834217d6c34ff9af server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:08.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.707+0000 7fd0367fc640 1 -- 172.21.15.1:0/1314889806 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd038035040 con 0x56422b3baa20 2024-09-11T22:02:08.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.708+0000 7fd0367fc640 1 -- 172.21.15.1:0/1314889806 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd038009520 con 0x56422b3baa20 2024-09-11T22:02:08.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.708+0000 7fd0367fc640 1 -- 172.21.15.1:0/1314889806 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd038003e80 con 0x56422b3baa20 2024-09-11T22:02:08.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.708+0000 7fd055503740 1 -- 172.21.15.1:0/1314889806 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56422b3baa20 msgr2=0x56422b3e1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:08.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.708+0000 7fd055503740 1 --2- 172.21.15.1:0/1314889806 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56422b3baa20 0x56422b3e1510 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fd0380040e0 tx=0x7fd038000bc0 comp rx=0 tx=0).stop 2024-09-11T22:02:08.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.709+0000 7fd055503740 1 -- 172.21.15.1:0/1314889806 shutdown_connections 2024-09-11T22:02:08.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.709+0000 7fd055503740 1 --2- 172.21.15.1:0/1314889806 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56422b3ca150 0x56422b3c4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:08.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.709+0000 7fd055503740 1 --2- 172.21.15.1:0/1314889806 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56422b3bb340 0x56422b3e1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:08.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.709+0000 7fd055503740 1 --2- 172.21.15.1:0/1314889806 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56422b3baa20 0x56422b3e1510 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:08.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.709+0000 7fd055503740 1 -- 172.21.15.1:0/1314889806 >> 172.21.15.1:0/1314889806 conn(0x56422b3b70f0 msgr2=0x56422b3b6270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:08.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.709+0000 7fd055503740 1 -- 172.21.15.1:0/1314889806 shutdown_connections 2024-09-11T22:02:08.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.709+0000 7fd055503740 1 -- 172.21.15.1:0/1314889806 wait complete. 2024-09-11T22:02:08.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.709+0000 7fd055503740 1 -- start start 2024-09-11T22:02:08.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.709+0000 7fd055503740 1 Processor -- start 2024-09-11T22:02:08.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.710+0000 7fd055503740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56422b3baa20 0x56422b3ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:08.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.710+0000 7fd055503740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56422b3bb340 0x56422b3cea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:08.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.710+0000 7fd055503740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56422b3ca150 0x56422b4434e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:08.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.710+0000 7fd055503740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd04c1acb50 con 0x56422b3ca150 2024-09-11T22:02:08.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.710+0000 7fd055503740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd04c1ac450 con 0x56422b3baa20 2024-09-11T22:02:08.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.710+0000 7fd055503740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd04c1a9690 con 0x56422b3bb340 2024-09-11T22:02:08.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.710+0000 7fd055503740 10 client.0 _ll_register_callbacks cb 0x7fd038032dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:02:08.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.710+0000 7fd050ec6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56422b3bb340 0x56422b3cea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:08.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.710+0000 7fd04bfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56422b3baa20 0x56422b3ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:08.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.710+0000 7fd050ec6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56422b3bb340 0x56422b3cea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:58440/0 (socket says 172.21.15.1:58440) 2024-09-11T22:02:08.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.710+0000 7fd050ec6640 1 -- 172.21.15.1:0/1627769870 learned_addr learned my addr 172.21.15.1:0/1627769870 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:08.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.710+0000 7fd04bfff640 1 --2- 172.21.15.1:0/1627769870 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56422b3ca150 0x56422b4434e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:08.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.710+0000 7fd050ec6640 1 -- 172.21.15.1:0/1627769870 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56422b3baa20 msgr2=0x56422b3ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:08.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.710+0000 7fd050ec6640 1 --2- 172.21.15.1:0/1627769870 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56422b3baa20 0x56422b3ce510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:08.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.710+0000 7fd050ec6640 1 -- 172.21.15.1:0/1627769870 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56422b3ca150 msgr2=0x56422b4434e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:08.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.710+0000 7fd050ec6640 1 --2- 172.21.15.1:0/1627769870 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56422b3ca150 0x56422b4434e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:08.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.710+0000 7fd050ec6640 1 -- 172.21.15.1:0/1627769870 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56422b443a20 con 0x56422b3bb340 2024-09-11T22:02:08.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.711+0000 7fd050ec6640 1 --2- 172.21.15.1:0/1627769870 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56422b3bb340 0x56422b3cea50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd04000b7e0 tx=0x7fd040013830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c77c7e55dea5a253 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:08.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.711+0000 7fd02bfff640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-11T22:02:08.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.711+0000 7fd02bfff640 1 -- 172.21.15.1:0/1627769870 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd040004220 con 0x56422b3bb340 2024-09-11T22:02:08.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.711+0000 7fd02bfff640 1 -- 172.21.15.1:0/1627769870 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7fd0400043c0 con 0x56422b3bb340 2024-09-11T22:02:08.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.711+0000 7fd055503740 1 -- 172.21.15.1:0/1627769870 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd038009520 con 0x56422b3bb340 2024-09-11T22:02:08.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.711+0000 7fd02bfff640 1 -- 172.21.15.1:0/1627769870 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd04002b050 con 0x56422b3bb340 2024-09-11T22:02:08.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.711+0000 7fd055503740 10 client.25927 fetch_fsmap learned FSMap version 33 2024-09-11T22:02:08.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.711+0000 7fd055503740 1 -- 172.21.15.1:0/1627769870 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56422b444d00 con 0x56422b3bb340 2024-09-11T22:02:08.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.711+0000 7fd02bfff640 1 -- 172.21.15.1:0/1627769870 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd038009520 con 0x56422b3bb340 2024-09-11T22:02:08.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.712+0000 7fd02bfff640 1 -- 172.21.15.1:0/1627769870 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7fd04001c4f0 con 0x56422b3bb340 2024-09-11T22:02:08.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.712+0000 7fd055503740 10 client.25927 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T22:02:08.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.712+0000 7fd055503740 10 client.25927 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:02:08.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.712+0000 7fd055503740 20 client.25927 populate_metadata read hostname 'smithi001' 2024-09-11T22:02:08.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.712+0000 7fd055503740 1 --2- 172.21.15.1:0/1627769870 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x56422b4455c0 0x56422b4479e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:08.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.712+0000 7fd055503740 4 client.25927 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T22:02:08.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.712+0000 7fd055503740 1 -- 172.21.15.1:0/1627769870 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56422b32d7f0 con 0x56422b4455c0 2024-09-11T22:02:08.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.712+0000 7fd04bfff640 1 --2- 172.21.15.1:0/1627769870 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x56422b4455c0 0x56422b4479e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:08.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.712+0000 7fd04bfff640 1 --2- 172.21.15.1:0/1627769870 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x56422b4455c0 0x56422b4479e0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a67a126316b4b192 server_cookie=fbc0f88184f5ee4e in_seq=0 out_seq=0 2024-09-11T22:02:08.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.713+0000 7fd02bfff640 10 client.25927 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:02:08.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.754+0000 7fd02bfff640 1 -- 172.21.15.1:0/1627769870 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56422b32d7f0 con 0x56422b4455c0 2024-09-11T22:02:08.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.754+0000 7fd02bfff640 10 client.25927 handle_command_reply: tid=0 2024-09-11T22:02:08.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.754+0000 7fd055503740 1 client.25927 shutdown 2024-09-11T22:02:08.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.754+0000 7fd055503740 1 -- 172.21.15.1:0/1627769870 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56422b3bb340 msgr2=0x56422b3cea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:08.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.754+0000 7fd055503740 1 --2- 172.21.15.1:0/1627769870 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56422b3bb340 0x56422b3cea50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd04000b7e0 tx=0x7fd040013830 comp rx=0 tx=0).stop 2024-09-11T22:02:08.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.754+0000 7fd055503740 1 -- 172.21.15.1:0/1627769870 shutdown_connections 2024-09-11T22:02:08.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.754+0000 7fd055503740 1 --2- 172.21.15.1:0/1627769870 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x56422b4455c0 0x56422b4479e0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:08.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.754+0000 7fd055503740 1 --2- 172.21.15.1:0/1627769870 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56422b3ca150 0x56422b4434e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:08.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.754+0000 7fd055503740 1 --2- 172.21.15.1:0/1627769870 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56422b3bb340 0x56422b3cea50 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:08.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.754+0000 7fd055503740 1 --2- 172.21.15.1:0/1627769870 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56422b3baa20 0x56422b3ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:08.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.754+0000 7fd055503740 1 -- 172.21.15.1:0/1627769870 >> 172.21.15.1:0/1627769870 conn(0x56422b3b70f0 msgr2=0x56422b3c7070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:08.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.754+0000 7fd02bfff640 0 client.25927 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:02:08.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.754+0000 7fd055503740 1 -- 172.21.15.1:0/1627769870 shutdown_connections 2024-09-11T22:02:08.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.754+0000 7fd055503740 1 -- 172.21.15.1:0/1627769870 wait complete. 2024-09-11T22:02:08.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.755+0000 7fd055503740 20 client.25927 trim_cache size 0 max 16384 2024-09-11T22:02:08.757 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-11T22:02:08.757 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-09-11T22:02:08.757 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "77bfa38d-1de7-4e47-ae87-02330e512923", 2024-09-11T22:02:08.757 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-09-11T22:02:08.757 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-11T22:02:08.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.755+0000 7fd05312a640 1 -- 172.21.15.1:0/2268461192 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fd03c0a2cf0 msgr2=0x7fd03c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:08.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.755+0000 7fd05312a640 1 --2- 172.21.15.1:0/2268461192 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fd03c0a2cf0 0x7fd03c0a51b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd0440cd1d0 tx=0x7fd04406deb0 comp rx=0 tx=0).stop 2024-09-11T22:02:08.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.755+0000 7fd05312a640 1 -- 172.21.15.1:0/2268461192 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd04c1082a0 msgr2=0x7fd04c19ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:08.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.755+0000 7fd05312a640 1 --2- 172.21.15.1:0/2268461192 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd04c1082a0 0x7fd04c19ce30 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fd04400cce0 tx=0x7fd04400a620 comp rx=0 tx=0).stop 2024-09-11T22:02:08.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.755+0000 7fd05312a640 1 -- 172.21.15.1:0/2268461192 shutdown_connections 2024-09-11T22:02:08.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.755+0000 7fd05312a640 1 --2- 172.21.15.1:0/2268461192 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fd03c0a2cf0 0x7fd03c0a51b0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:08.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.755+0000 7fd05312a640 1 --2- 172.21.15.1:0/2268461192 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd04c1082a0 0x7fd04c19ce30 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:08.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.755+0000 7fd05312a640 1 --2- 172.21.15.1:0/2268461192 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd04c1036e0 0x7fd04c198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:08.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.755+0000 7fd05312a640 1 --2- 172.21.15.1:0/2268461192 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd04c102d30 0x7fd04c198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:08.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.755+0000 7fd05312a640 1 -- 172.21.15.1:0/2268461192 >> 172.21.15.1:0/2268461192 conn(0x7fd04c0fc7d0 msgr2=0x7fd04c105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:08.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.756+0000 7fd05312a640 1 -- 172.21.15.1:0/2268461192 shutdown_connections 2024-09-11T22:02:08.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:08.756+0000 7fd05312a640 1 -- 172.21.15.1:0/2268461192 wait complete. 2024-09-11T22:02:09.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:09 smithi053 ceph-mon[34908]: pgmap v152: 97 pgs: 97 active+clean; 53 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.2 MiB/s wr, 1.09k op/s 2024-09-11T22:02:09.320 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:09 smithi001 ceph-mon[30935]: pgmap v152: 97 pgs: 97 active+clean; 53 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.2 MiB/s wr, 1.09k op/s 2024-09-11T22:02:09.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:09 smithi161 ceph-mon[35615]: pgmap v152: 97 pgs: 97 active+clean; 53 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.2 MiB/s wr, 1.09k op/s 2024-09-11T22:02:09.682 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:02:09.885 INFO:tasks.quiescer.fs.[cephfs]:Total duration (40.3) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-11T22:02:09.885 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.15 2024-09-11T22:02:09.886 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:02:10.006 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:02:10.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:10 smithi001 ceph-mon[30935]: scrub queued for path: / 2024-09-11T22:02:10.337 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:10 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:10.338 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:10 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:10.338 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:10 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:10.338 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:10 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:10.338 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:10 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:10.338 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:10 smithi053 ceph-mon[34908]: scrub queued for path: / 2024-09-11T22:02:10.339 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:10 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:10.339 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:10 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:10.339 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:10 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:10.339 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:10 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:10.339 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:10 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:10.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:10 smithi161 ceph-mon[35615]: scrub queued for path: / 2024-09-11T22:02:10.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:10 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:10.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:10 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:10.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:10 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:10.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:10 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:10.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:10 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:10.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.535+0000 7f086c387640 1 -- 172.21.15.1:0/1719949491 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0864100e20 msgr2=0x7f0864101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:10.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.535+0000 7f086c387640 1 --2- 172.21.15.1:0/1719949491 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0864100e20 0x7f0864101200 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0860009930 tx=0x7f086002a4d0 comp rx=0 tx=0).stop 2024-09-11T22:02:10.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.535+0000 7f086c387640 1 -- 172.21.15.1:0/1719949491 shutdown_connections 2024-09-11T22:02:10.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.535+0000 7f086c387640 1 --2- 172.21.15.1:0/1719949491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0864102ef0 0x7f086410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:10.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.535+0000 7f086c387640 1 --2- 172.21.15.1:0/1719949491 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0864102570 0x7f08641029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:10.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.535+0000 7f086c387640 1 --2- 172.21.15.1:0/1719949491 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0864100e20 0x7f0864101200 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:10.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.535+0000 7f086c387640 1 -- 172.21.15.1:0/1719949491 >> 172.21.15.1:0/1719949491 conn(0x7f08640fc7d0 msgr2=0x7f08640febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:10.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.535+0000 7f086c387640 1 -- 172.21.15.1:0/1719949491 shutdown_connections 2024-09-11T22:02:10.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.535+0000 7f086c387640 1 -- 172.21.15.1:0/1719949491 wait complete. 2024-09-11T22:02:10.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.536+0000 7f086c387640 1 Processor -- start 2024-09-11T22:02:10.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.536+0000 7f086c387640 1 -- start start 2024-09-11T22:02:10.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.536+0000 7f086c387640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0864100e20 0x7f0864198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:10.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.536+0000 7f086c387640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0864102570 0x7f0864198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:10.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.536+0000 7f086c387640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0864102ef0 0x7f086419ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:10.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.536+0000 7f086c387640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0864110fd0 con 0x7f0864100e20 2024-09-11T22:02:10.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.536+0000 7f086c387640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0864110e50 con 0x7f0864102ef0 2024-09-11T22:02:10.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.536+0000 7f086c387640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0864111150 con 0x7f0864102570 2024-09-11T22:02:10.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.536+0000 7f0869922640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0864102570 0x7f0864198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:10.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.536+0000 7f086a123640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0864100e20 0x7f0864198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:10.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.536+0000 7f086a924640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0864102ef0 0x7f086419ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:10.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.536+0000 7f086a924640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0864102ef0 0x7f086419ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:46964/0 (socket says 172.21.15.1:46964) 2024-09-11T22:02:10.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.536+0000 7f086a123640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0864100e20 0x7f0864198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49366/0 (socket says 172.21.15.1:49366) 2024-09-11T22:02:10.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.536+0000 7f086a924640 1 -- 172.21.15.1:0/3710344706 learned_addr learned my addr 172.21.15.1:0/3710344706 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:10.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.536+0000 7f086a123640 1 -- 172.21.15.1:0/3710344706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0864102570 msgr2=0x7f0864198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:10.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.536+0000 7f086a123640 1 --2- 172.21.15.1:0/3710344706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0864102570 0x7f0864198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:10.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.536+0000 7f086a123640 1 -- 172.21.15.1:0/3710344706 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0864102ef0 msgr2=0x7f086419ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:10.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.536+0000 7f086a123640 1 --2- 172.21.15.1:0/3710344706 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0864102ef0 0x7f086419ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:10.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.536+0000 7f086a123640 1 -- 172.21.15.1:0/3710344706 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f086419d510 con 0x7f0864100e20 2024-09-11T22:02:10.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.537+0000 7f086a123640 1 --2- 172.21.15.1:0/3710344706 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0864100e20 0x7f0864198560 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f086002a9f0 tx=0x7f0860003b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:10.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.537+0000 7f085b7fe640 1 -- 172.21.15.1:0/3710344706 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f086003f070 con 0x7f0864100e20 2024-09-11T22:02:10.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.537+0000 7f086c387640 1 -- 172.21.15.1:0/3710344706 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f086419d7a0 con 0x7f0864100e20 2024-09-11T22:02:10.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.537+0000 7f085b7fe640 1 -- 172.21.15.1:0/3710344706 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0860031c90 con 0x7f0864100e20 2024-09-11T22:02:10.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.537+0000 7f085b7fe640 1 -- 172.21.15.1:0/3710344706 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0860035030 con 0x7f0864100e20 2024-09-11T22:02:10.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.537+0000 7f086c387640 1 -- 172.21.15.1:0/3710344706 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08641a50e0 con 0x7f0864100e20 2024-09-11T22:02:10.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.539+0000 7f085b7fe640 1 -- 172.21.15.1:0/3710344706 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f0860042070 con 0x7f0864100e20 2024-09-11T22:02:10.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.540+0000 7f086e760740 1 Processor -- start 2024-09-11T22:02:10.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.540+0000 7f086e760740 1 -- start start 2024-09-11T22:02:10.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.540+0000 7f085b7fe640 1 --2- 172.21.15.1:0/3710344706 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f08500a2cf0 0x7f08500a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:10.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.541+0000 7f086e760740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eff91db380 0x55eff91db760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:10.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.541+0000 7f086e760740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eff91dbca0 0x55eff91ea550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:10.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.541+0000 7f086e760740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eff91eaa90 0x55eff91ece80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:10.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.541+0000 7f086a924640 1 --2- 172.21.15.1:0/3710344706 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f08500a2cf0 0x7f08500a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:10.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.541+0000 7f086e760740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55eff91412e0 con 0x55eff91dbca0 2024-09-11T22:02:10.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.541+0000 7f086e760740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55eff90e8ce0 con 0x55eff91db380 2024-09-11T22:02:10.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.541+0000 7f086e760740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55eff913d2a0 con 0x55eff91eaa90 2024-09-11T22:02:10.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.541+0000 7f085b7fe640 1 -- 172.21.15.1:0/3710344706 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f08600efd20 con 0x7f0864100e20 2024-09-11T22:02:10.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.541+0000 7f0869922640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eff91dbca0 0x55eff91ea550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:10.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.541+0000 7f0869922640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eff91dbca0 0x55eff91ea550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49376/0 (socket says 172.21.15.1:49376) 2024-09-11T22:02:10.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.541+0000 7f0869922640 1 -- 172.21.15.1:0/1906800383 learned_addr learned my addr 172.21.15.1:0/1906800383 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:10.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.541+0000 7f086a924640 1 --2- 172.21.15.1:0/1906800383 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eff91eaa90 0x55eff91ece80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:10.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.541+0000 7f086a123640 1 --2- 172.21.15.1:0/1906800383 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eff91db380 0x55eff91db760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:10.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.541+0000 7f0869922640 1 -- 172.21.15.1:0/1906800383 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eff91eaa90 msgr2=0x55eff91ece80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:10.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.541+0000 7f0869922640 1 --2- 172.21.15.1:0/1906800383 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eff91eaa90 0x55eff91ece80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:10.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.541+0000 7f0869922640 1 -- 172.21.15.1:0/1906800383 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eff91db380 msgr2=0x55eff91db760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:10.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.541+0000 7f0869922640 1 --2- 172.21.15.1:0/1906800383 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eff91db380 0x55eff91db760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:10.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.541+0000 7f0869922640 1 -- 172.21.15.1:0/1906800383 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eff90cd440 con 0x55eff91dbca0 2024-09-11T22:02:10.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.541+0000 7f086a924640 1 --2- 172.21.15.1:0/3710344706 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f08500a2cf0 0x7f08500a51b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f085c009c30 tx=0x7f085c009f00 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:10.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.542+0000 7f0869922640 1 --2- 172.21.15.1:0/1906800383 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eff91dbca0 0x55eff91ea550 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f0854009ca0 tx=0x7f0854011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81750b91d0c31ce4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:10.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.542+0000 7f0858dea640 1 -- 172.21.15.1:0/1906800383 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f085401e070 con 0x55eff91dbca0 2024-09-11T22:02:10.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.542+0000 7f0858dea640 1 -- 172.21.15.1:0/1906800383 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0854019910 con 0x55eff91dbca0 2024-09-11T22:02:10.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.542+0000 7f0858dea640 1 -- 172.21.15.1:0/1906800383 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f085400e950 con 0x55eff91dbca0 2024-09-11T22:02:10.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.543+0000 7f086e760740 1 -- 172.21.15.1:0/1906800383 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eff91dbca0 msgr2=0x55eff91ea550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:10.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.543+0000 7f086e760740 1 --2- 172.21.15.1:0/1906800383 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eff91dbca0 0x55eff91ea550 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f0854009ca0 tx=0x7f0854011810 comp rx=0 tx=0).stop 2024-09-11T22:02:10.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.543+0000 7f086e760740 1 -- 172.21.15.1:0/1906800383 shutdown_connections 2024-09-11T22:02:10.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.543+0000 7f086e760740 1 --2- 172.21.15.1:0/1906800383 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eff91eaa90 0x55eff91ece80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:10.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.543+0000 7f086e760740 1 --2- 172.21.15.1:0/1906800383 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eff91dbca0 0x55eff91ea550 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:10.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.543+0000 7f086e760740 1 --2- 172.21.15.1:0/1906800383 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eff91db380 0x55eff91db760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:10.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.543+0000 7f086e760740 1 -- 172.21.15.1:0/1906800383 >> 172.21.15.1:0/1906800383 conn(0x55eff91d7a30 msgr2=0x55eff91d7e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:10.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.543+0000 7f086e760740 1 -- 172.21.15.1:0/1906800383 shutdown_connections 2024-09-11T22:02:10.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.543+0000 7f086e760740 1 -- 172.21.15.1:0/1906800383 wait complete. 2024-09-11T22:02:10.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.543+0000 7f086e760740 1 -- start start 2024-09-11T22:02:10.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.543+0000 7f086e760740 1 Processor -- start 2024-09-11T22:02:10.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.544+0000 7f086e760740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eff91db380 0x55eff91ea460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:10.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.544+0000 7f086e760740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eff91dbca0 0x55eff91e7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:10.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.544+0000 7f086e760740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eff91eaa90 0x55eff91e7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:10.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.544+0000 7f086e760740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55eff91ed660 con 0x55eff91eaa90 2024-09-11T22:02:10.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.544+0000 7f086e760740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55eff90fc050 con 0x55eff91db380 2024-09-11T22:02:10.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.544+0000 7f086e760740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55eff91ed7e0 con 0x55eff91dbca0 2024-09-11T22:02:10.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.544+0000 7f086e760740 10 client.0 _ll_register_callbacks cb 0x55eff913c160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:02:10.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.544+0000 7f086a924640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eff91eaa90 0x55eff91e7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:10.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.544+0000 7f0869922640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eff91dbca0 0x55eff91e7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:10.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.544+0000 7f086a123640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eff91db380 0x55eff91ea460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:10.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.544+0000 7f0869922640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eff91dbca0 0x55eff91e7710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:58482/0 (socket says 172.21.15.1:58482) 2024-09-11T22:02:10.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.544+0000 7f086a123640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eff91db380 0x55eff91ea460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:46972/0 (socket says 172.21.15.1:46972) 2024-09-11T22:02:10.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.544+0000 7f0869922640 1 -- 172.21.15.1:0/2581864920 learned_addr learned my addr 172.21.15.1:0/2581864920 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:10.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.544+0000 7f086a924640 1 -- 172.21.15.1:0/2581864920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eff91dbca0 msgr2=0x55eff91e7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:10.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.544+0000 7f086a924640 1 --2- 172.21.15.1:0/2581864920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eff91dbca0 0x55eff91e7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:10.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.544+0000 7f086a924640 1 -- 172.21.15.1:0/2581864920 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eff91db380 msgr2=0x55eff91ea460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:10.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.544+0000 7f086a924640 1 --2- 172.21.15.1:0/2581864920 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eff91db380 0x55eff91ea460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:10.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.544+0000 7f086a924640 1 -- 172.21.15.1:0/2581864920 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eff8eb23c0 con 0x55eff91eaa90 2024-09-11T22:02:10.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.544+0000 7f086a924640 1 --2- 172.21.15.1:0/2581864920 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eff91eaa90 0x55eff91e7d00 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x55eff91d8db0 tx=0x7f085c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1688ef5a964acd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:10.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.544+0000 7f084dffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:02:10.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.545+0000 7f084dffb640 1 -- 172.21.15.1:0/2581864920 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f085c0216e0 con 0x55eff91eaa90 2024-09-11T22:02:10.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.545+0000 7f084dffb640 1 -- 172.21.15.1:0/2581864920 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f085c021880 con 0x55eff91eaa90 2024-09-11T22:02:10.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.545+0000 7f086e760740 1 -- 172.21.15.1:0/2581864920 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0854019910 con 0x55eff91eaa90 2024-09-11T22:02:10.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.545+0000 7f084dffb640 1 -- 172.21.15.1:0/2581864920 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f085c021a80 con 0x55eff91eaa90 2024-09-11T22:02:10.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.545+0000 7f086e760740 10 client.16434 fetch_fsmap learned FSMap version 33 2024-09-11T22:02:10.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.545+0000 7f086e760740 1 -- 172.21.15.1:0/2581864920 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55eff91f90d0 con 0x55eff91eaa90 2024-09-11T22:02:10.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.545+0000 7f084dffb640 1 -- 172.21.15.1:0/2581864920 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0854019910 con 0x55eff91eaa90 2024-09-11T22:02:10.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.545+0000 7f084dffb640 1 -- 172.21.15.1:0/2581864920 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7f085c020050 con 0x55eff91eaa90 2024-09-11T22:02:10.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.545+0000 7f086e760740 10 client.16434 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T22:02:10.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.545+0000 7f086e760740 10 client.16434 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:02:10.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.545+0000 7f086e760740 20 client.16434 populate_metadata read hostname 'smithi001' 2024-09-11T22:02:10.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.545+0000 7f086e760740 1 --2- 172.21.15.1:0/2581864920 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55eff91fa5e0 0x55eff91fca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:10.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.545+0000 7f086e760740 4 client.16434 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:02:10.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.545+0000 7f086e760740 1 -- 172.21.15.1:0/2581864920 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eff914e850 con 0x55eff91fa5e0 2024-09-11T22:02:10.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.546+0000 7f086a123640 1 --2- 172.21.15.1:0/2581864920 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55eff91fa5e0 0x55eff91fca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:10.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.546+0000 7f084dffb640 10 client.16434 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:02:10.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.546+0000 7f086a123640 1 --2- 172.21.15.1:0/2581864920 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55eff91fa5e0 0x55eff91fca10 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25c2541f5b38d7fd server_cookie=14485aae6351d866 in_seq=0 out_seq=0 2024-09-11T22:02:10.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.548+0000 7f084dffb640 1 -- 172.21.15.1:0/2581864920 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eff914e850 con 0x55eff91fa5e0 2024-09-11T22:02:10.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.548+0000 7f084dffb640 10 client.16434 handle_command_reply: tid=0 2024-09-11T22:02:10.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.548+0000 7f086e760740 1 client.16434 shutdown 2024-09-11T22:02:10.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.549+0000 7f086e760740 1 -- 172.21.15.1:0/2581864920 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eff91eaa90 msgr2=0x55eff91e7d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:10.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.549+0000 7f086e760740 1 --2- 172.21.15.1:0/2581864920 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eff91eaa90 0x55eff91e7d00 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x55eff91d8db0 tx=0x7f085c004ab0 comp rx=0 tx=0).stop 2024-09-11T22:02:10.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.549+0000 7f086e760740 1 -- 172.21.15.1:0/2581864920 shutdown_connections 2024-09-11T22:02:10.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.549+0000 7f086e760740 1 --2- 172.21.15.1:0/2581864920 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55eff91fa5e0 0x55eff91fca10 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:10.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.549+0000 7f086e760740 1 --2- 172.21.15.1:0/2581864920 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eff91eaa90 0x55eff91e7d00 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:10.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.549+0000 7f084dffb640 0 client.16434 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:02:10.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.549+0000 7f086e760740 1 --2- 172.21.15.1:0/2581864920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eff91dbca0 0x55eff91e7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:10.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.549+0000 7f086e760740 1 --2- 172.21.15.1:0/2581864920 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eff91db380 0x55eff91ea460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:10.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.549+0000 7f086e760740 1 -- 172.21.15.1:0/2581864920 >> 172.21.15.1:0/2581864920 conn(0x55eff91d7a30 msgr2=0x55eff91dd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:10.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.549+0000 7f086e760740 1 -- 172.21.15.1:0/2581864920 shutdown_connections 2024-09-11T22:02:10.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.549+0000 7f086e760740 1 -- 172.21.15.1:0/2581864920 wait complete. 2024-09-11T22:02:10.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.549+0000 7f086e760740 20 client.16434 trim_cache size 0 max 16384 2024-09-11T22:02:10.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.570+0000 7f086e760740 1 Processor -- start 2024-09-11T22:02:10.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.570+0000 7f086e760740 1 -- start start 2024-09-11T22:02:10.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.570+0000 7f086e760740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eff91db380 0x55eff91f6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:10.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.570+0000 7f086e760740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eff91dbca0 0x55eff9202320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:10.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.570+0000 7f086e760740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eff91eaa90 0x55eff9202860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:10.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.570+0000 7f086e760740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f08641ab750 con 0x55eff91dbca0 2024-09-11T22:02:10.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.570+0000 7f086e760740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f08641ad380 con 0x55eff91eaa90 2024-09-11T22:02:10.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.570+0000 7f086e760740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f08641abe20 con 0x55eff91db380 2024-09-11T22:02:10.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.570+0000 7f0869922640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eff91dbca0 0x55eff9202320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:10.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.571+0000 7f0869922640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eff91dbca0 0x55eff9202320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49400/0 (socket says 172.21.15.1:49400) 2024-09-11T22:02:10.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.571+0000 7f0869922640 1 -- 172.21.15.1:0/2495760784 learned_addr learned my addr 172.21.15.1:0/2495760784 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:10.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.571+0000 7f086a123640 1 --2- 172.21.15.1:0/2495760784 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eff91db380 0x55eff91f6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:10.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.571+0000 7f086a924640 1 --2- 172.21.15.1:0/2495760784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eff91eaa90 0x55eff9202860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:10.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.571+0000 7f0869922640 1 -- 172.21.15.1:0/2495760784 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eff91db380 msgr2=0x55eff91f6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:10.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.571+0000 7f0869922640 1 --2- 172.21.15.1:0/2495760784 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eff91db380 0x55eff91f6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:10.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.571+0000 7f0869922640 1 -- 172.21.15.1:0/2495760784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eff91eaa90 msgr2=0x55eff9202860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:10.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.571+0000 7f0869922640 1 --2- 172.21.15.1:0/2495760784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eff91eaa90 0x55eff9202860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:10.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.571+0000 7f0869922640 1 -- 172.21.15.1:0/2495760784 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eff91d74d0 con 0x55eff91dbca0 2024-09-11T22:02:10.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.571+0000 7f0869922640 1 --2- 172.21.15.1:0/2495760784 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eff91dbca0 0x55eff9202320 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f085401c010 tx=0x7f08540097e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1d30153a0af845a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:10.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.571+0000 7f084ffff640 1 -- 172.21.15.1:0/2495760784 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f085401e040 con 0x55eff91dbca0 2024-09-11T22:02:10.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.571+0000 7f084ffff640 1 -- 172.21.15.1:0/2495760784 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0854004220 con 0x55eff91dbca0 2024-09-11T22:02:10.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.571+0000 7f084ffff640 1 -- 172.21.15.1:0/2495760784 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0854023800 con 0x55eff91dbca0 2024-09-11T22:02:10.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.572+0000 7f086e760740 1 -- 172.21.15.1:0/2495760784 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eff91dbca0 msgr2=0x55eff9202320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:10.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.572+0000 7f086e760740 1 --2- 172.21.15.1:0/2495760784 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eff91dbca0 0x55eff9202320 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f085401c010 tx=0x7f08540097e0 comp rx=0 tx=0).stop 2024-09-11T22:02:10.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.572+0000 7f086e760740 1 -- 172.21.15.1:0/2495760784 shutdown_connections 2024-09-11T22:02:10.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.572+0000 7f086e760740 1 --2- 172.21.15.1:0/2495760784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eff91eaa90 0x55eff9202860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:10.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.572+0000 7f086e760740 1 --2- 172.21.15.1:0/2495760784 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eff91dbca0 0x55eff9202320 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:10.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.572+0000 7f086e760740 1 --2- 172.21.15.1:0/2495760784 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eff91db380 0x55eff91f6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:10.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.572+0000 7f086e760740 1 -- 172.21.15.1:0/2495760784 >> 172.21.15.1:0/2495760784 conn(0x55eff91d7a30 msgr2=0x55eff91e2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:10.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.572+0000 7f086e760740 1 -- 172.21.15.1:0/2495760784 shutdown_connections 2024-09-11T22:02:10.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.572+0000 7f086e760740 1 -- 172.21.15.1:0/2495760784 wait complete. 2024-09-11T22:02:10.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.573+0000 7f086e760740 1 -- start start 2024-09-11T22:02:10.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.573+0000 7f086e760740 1 Processor -- start 2024-09-11T22:02:10.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.573+0000 7f086e760740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eff91db380 0x55eff91f2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:10.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.573+0000 7f086e760740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eff91dbca0 0x55eff91f2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:10.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.573+0000 7f086e760740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eff91eaa90 0x55eff91efa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:10.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.573+0000 7f086e760740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f08641acbf0 con 0x55eff91eaa90 2024-09-11T22:02:10.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.573+0000 7f086e760740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f08641ac4f0 con 0x55eff91dbca0 2024-09-11T22:02:10.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.573+0000 7f086e760740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f08641a9730 con 0x55eff91db380 2024-09-11T22:02:10.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.573+0000 7f086e760740 10 client.0 _ll_register_callbacks cb 0x7f085c0098a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:02:10.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.573+0000 7f086a924640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eff91eaa90 0x55eff91efa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:10.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.573+0000 7f0869922640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eff91dbca0 0x55eff91f2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:10.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.573+0000 7f086a123640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eff91db380 0x55eff91f2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:10.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.573+0000 7f0869922640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eff91dbca0 0x55eff91f2fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:46978/0 (socket says 172.21.15.1:46978) 2024-09-11T22:02:10.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.573+0000 7f086a924640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eff91eaa90 0x55eff91efa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49416/0 (socket says 172.21.15.1:49416) 2024-09-11T22:02:10.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.573+0000 7f0869922640 1 -- 172.21.15.1:0/2205060853 learned_addr learned my addr 172.21.15.1:0/2205060853 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:10.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.574+0000 7f086a924640 1 -- 172.21.15.1:0/2205060853 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eff91db380 msgr2=0x55eff91f2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:10.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.574+0000 7f086a924640 1 --2- 172.21.15.1:0/2205060853 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eff91db380 0x55eff91f2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:10.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.574+0000 7f086a924640 1 -- 172.21.15.1:0/2205060853 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eff91dbca0 msgr2=0x55eff91f2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:10.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.574+0000 7f086a924640 1 --2- 172.21.15.1:0/2205060853 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eff91dbca0 0x55eff91f2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:10.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.574+0000 7f086a924640 1 -- 172.21.15.1:0/2205060853 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eff91f35a0 con 0x55eff91eaa90 2024-09-11T22:02:10.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.574+0000 7f0869922640 1 --2- 172.21.15.1:0/2205060853 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eff91dbca0 0x55eff91f2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:02:10.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.574+0000 7f086a924640 1 --2- 172.21.15.1:0/2205060853 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eff91eaa90 0x55eff91efa40 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x55eff913c1a0 tx=0x7f085c021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39b9544d05b683c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:10.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.574+0000 7f084dffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:02:10.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.574+0000 7f084dffb640 1 -- 172.21.15.1:0/2205060853 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f085c00e5c0 con 0x55eff91eaa90 2024-09-11T22:02:10.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.574+0000 7f084dffb640 1 -- 172.21.15.1:0/2205060853 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f085c01c040 con 0x55eff91eaa90 2024-09-11T22:02:10.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.574+0000 7f086e760740 1 -- 172.21.15.1:0/2205060853 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0854004220 con 0x55eff91eaa90 2024-09-11T22:02:10.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.574+0000 7f084dffb640 1 -- 172.21.15.1:0/2205060853 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f085c013580 con 0x55eff91eaa90 2024-09-11T22:02:10.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.574+0000 7f086e760740 10 client.16440 fetch_fsmap learned FSMap version 33 2024-09-11T22:02:10.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.574+0000 7f086e760740 1 -- 172.21.15.1:0/2205060853 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55eff9265820 con 0x55eff91eaa90 2024-09-11T22:02:10.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.574+0000 7f084dffb640 1 -- 172.21.15.1:0/2205060853 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0854004220 con 0x55eff91eaa90 2024-09-11T22:02:10.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.574+0000 7f084dffb640 1 -- 172.21.15.1:0/2205060853 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7f085c043030 con 0x55eff91eaa90 2024-09-11T22:02:10.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.575+0000 7f086e760740 10 client.16440 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T22:02:10.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.575+0000 7f086e760740 10 client.16440 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:02:10.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.575+0000 7f086e760740 20 client.16440 populate_metadata read hostname 'smithi001' 2024-09-11T22:02:10.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.575+0000 7f086e760740 1 --2- 172.21.15.1:0/2205060853 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55eff91fa5e0 0x55eff9268000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:10.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.575+0000 7f086e760740 4 client.16440 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:02:10.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.575+0000 7f086e760740 1 -- 172.21.15.1:0/2205060853 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55eff9148d60 con 0x55eff91fa5e0 2024-09-11T22:02:10.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.575+0000 7f086a123640 1 --2- 172.21.15.1:0/2205060853 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55eff91fa5e0 0x55eff9268000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:10.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.575+0000 7f086a123640 1 --2- 172.21.15.1:0/2205060853 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55eff91fa5e0 0x55eff9268000 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=835431be0bcced2d server_cookie=27de76d0dd1bb5f6 in_seq=0 out_seq=0 2024-09-11T22:02:10.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.575+0000 7f084dffb640 10 client.16440 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:02:10.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.577+0000 7f084dffb640 1 -- 172.21.15.1:0/2205060853 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x55eff9148d60 con 0x55eff91fa5e0 2024-09-11T22:02:10.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.577+0000 7f084dffb640 10 client.16440 handle_command_reply: tid=0 2024-09-11T22:02:10.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.577+0000 7f086e760740 1 client.16440 shutdown 2024-09-11T22:02:10.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.577+0000 7f086e760740 1 -- 172.21.15.1:0/2205060853 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eff91eaa90 msgr2=0x55eff91efa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:10.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.578+0000 7f086e760740 1 --2- 172.21.15.1:0/2205060853 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eff91eaa90 0x55eff91efa40 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x55eff913c1a0 tx=0x7f085c021000 comp rx=0 tx=0).stop 2024-09-11T22:02:10.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.578+0000 7f086e760740 1 -- 172.21.15.1:0/2205060853 shutdown_connections 2024-09-11T22:02:10.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.578+0000 7f086e760740 1 --2- 172.21.15.1:0/2205060853 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55eff91fa5e0 0x55eff9268000 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:10.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.578+0000 7f086e760740 1 --2- 172.21.15.1:0/2205060853 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eff91eaa90 0x55eff91efa40 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:10.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.578+0000 7f086e760740 1 --2- 172.21.15.1:0/2205060853 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eff91dbca0 0x55eff91f2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:10.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.578+0000 7f086e760740 1 --2- 172.21.15.1:0/2205060853 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eff91db380 0x55eff91f2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:10.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.578+0000 7f086e760740 1 -- 172.21.15.1:0/2205060853 >> 172.21.15.1:0/2205060853 conn(0x55eff91d7a30 msgr2=0x55eff91e8f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:10.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.578+0000 7f084dffb640 0 client.16440 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:02:10.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.578+0000 7f086e760740 1 -- 172.21.15.1:0/2205060853 shutdown_connections 2024-09-11T22:02:10.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.578+0000 7f086e760740 1 -- 172.21.15.1:0/2205060853 wait complete. 2024-09-11T22:02:10.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.578+0000 7f086e760740 20 client.16440 trim_cache size 0 max 16384 2024-09-11T22:02:10.589 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-11T22:02:10.590 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "scrub active (4226 inodes in the stack)", 2024-09-11T22:02:10.590 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-09-11T22:02:10.590 INFO:teuthology.orchestra.run.smithi001.stdout: "77bfa38d-1de7-4e47-ae87-02330e512923": { 2024-09-11T22:02:10.590 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2024-09-11T22:02:10.590 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "77bfa38d-1de7-4e47-ae87-02330e512923", 2024-09-11T22:02:10.590 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2024-09-11T22:02:10.590 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:02:10.590 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:02:10.590 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-11T22:02:10.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.578+0000 7f086c387640 1 -- 172.21.15.1:0/3710344706 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f08500a2cf0 msgr2=0x7f08500a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:10.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.578+0000 7f086c387640 1 --2- 172.21.15.1:0/3710344706 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f08500a2cf0 0x7f08500a51b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f085c009c30 tx=0x7f085c009f00 comp rx=0 tx=0).stop 2024-09-11T22:02:10.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.578+0000 7f086c387640 1 -- 172.21.15.1:0/3710344706 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0864100e20 msgr2=0x7f0864198560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:10.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.579+0000 7f086c387640 1 --2- 172.21.15.1:0/3710344706 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0864100e20 0x7f0864198560 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f086002a9f0 tx=0x7f0860003b20 comp rx=0 tx=0).stop 2024-09-11T22:02:10.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.579+0000 7f086c387640 1 -- 172.21.15.1:0/3710344706 shutdown_connections 2024-09-11T22:02:10.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.579+0000 7f086c387640 1 --2- 172.21.15.1:0/3710344706 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f08500a2cf0 0x7f08500a51b0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:10.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.579+0000 7f086c387640 1 --2- 172.21.15.1:0/3710344706 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0864102ef0 0x7f086419ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:10.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.579+0000 7f086c387640 1 --2- 172.21.15.1:0/3710344706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0864102570 0x7f0864198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:10.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.579+0000 7f086c387640 1 --2- 172.21.15.1:0/3710344706 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0864100e20 0x7f0864198560 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:10.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.579+0000 7f086c387640 1 -- 172.21.15.1:0/3710344706 >> 172.21.15.1:0/3710344706 conn(0x7f08640fc7d0 msgr2=0x7f086406a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:10.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.579+0000 7f086c387640 1 -- 172.21.15.1:0/3710344706 shutdown_connections 2024-09-11T22:02:10.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:10.579+0000 7f086c387640 1 -- 172.21.15.1:0/3710344706 wait complete. 2024-09-11T22:02:10.872 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:02:11.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:11 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:11.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:11 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:11.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:11 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:11.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:11 smithi053 ceph-mon[34908]: pgmap v153: 97 pgs: 97 active+clean; 82 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 8.0 MiB/s wr, 1.28k op/s 2024-09-11T22:02:11.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:11 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:11.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:11 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:11.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:11 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:11.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:11 smithi161 ceph-mon[35615]: pgmap v153: 97 pgs: 97 active+clean; 82 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 8.0 MiB/s wr, 1.28k op/s 2024-09-11T22:02:11.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:11 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:11.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:11 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:11.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:11 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:11.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:11 smithi001 ceph-mon[30935]: pgmap v153: 97 pgs: 97 active+clean; 82 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 8.0 MiB/s wr, 1.28k op/s 2024-09-11T22:02:11.635 INFO:tasks.cephfs.filesystem:scrub status for tag:77bfa38d-1de7-4e47-ae87-02330e512923 - {'path': '/', 'tag': '77bfa38d-1de7-4e47-ae87-02330e512923', 'options': 'recursive,force'} 2024-09-11T22:02:11.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:11.833+0000 7ff357db6640 1 -- 172.21.15.1:0/4083312084 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff3501027c0 msgr2=0x7ff350106810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:11.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:11.833+0000 7ff357db6640 1 --2- 172.21.15.1:0/4083312084 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff3501027c0 0x7ff350106810 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7ff340009900 tx=0x7ff34002a410 comp rx=0 tx=0).stop 2024-09-11T22:02:11.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:11.833+0000 7ff357db6640 1 -- 172.21.15.1:0/4083312084 shutdown_connections 2024-09-11T22:02:11.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:11.833+0000 7ff357db6640 1 --2- 172.21.15.1:0/4083312084 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff350106f40 0x7ff35010eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:11.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:11.833+0000 7ff357db6640 1 --2- 172.21.15.1:0/4083312084 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff3501027c0 0x7ff350106810 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:11.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:11.833+0000 7ff357db6640 1 --2- 172.21.15.1:0/4083312084 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff350101e10 0x7ff3501021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:11.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:11.833+0000 7ff357db6640 1 -- 172.21.15.1:0/4083312084 >> 172.21.15.1:0/4083312084 conn(0x7ff3500fd8d0 msgr2=0x7ff3500ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:11.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:11.833+0000 7ff357db6640 1 -- 172.21.15.1:0/4083312084 shutdown_connections 2024-09-11T22:02:11.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:11.833+0000 7ff357db6640 1 -- 172.21.15.1:0/4083312084 wait complete. 2024-09-11T22:02:11.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:11.833+0000 7ff357db6640 1 Processor -- start 2024-09-11T22:02:11.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:11.834+0000 7ff357db6640 1 -- start start 2024-09-11T22:02:11.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:11.834+0000 7ff357db6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff350101e10 0x7ff350198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:11.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:11.834+0000 7ff357db6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff3501027c0 0x7ff350198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:11.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:11.834+0000 7ff357db6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff350106f40 0x7ff35019cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:11.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:11.834+0000 7ff357db6640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff350111fa0 con 0x7ff350106f40 2024-09-11T22:02:11.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:11.834+0000 7ff357db6640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff350111e20 con 0x7ff350101e10 2024-09-11T22:02:11.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:11.834+0000 7ff357db6640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff350112120 con 0x7ff3501027c0 2024-09-11T22:02:11.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:11.834+0000 7ff356353640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff350106f40 0x7ff35019cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:11.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:11.834+0000 7ff355b52640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff350101e10 0x7ff350198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:11.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:11.834+0000 7ff356353640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff350106f40 0x7ff35019cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49434/0 (socket says 172.21.15.1:49434) 2024-09-11T22:02:11.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:11.834+0000 7ff356353640 1 -- 172.21.15.1:0/1260307948 learned_addr learned my addr 172.21.15.1:0/1260307948 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:11.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:11.834+0000 7ff355351640 1 --2- 172.21.15.1:0/1260307948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff3501027c0 0x7ff350198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:11.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:11.834+0000 7ff356353640 1 -- 172.21.15.1:0/1260307948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff3501027c0 msgr2=0x7ff350198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:11.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:11.834+0000 7ff356353640 1 --2- 172.21.15.1:0/1260307948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff3501027c0 0x7ff350198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:11.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:11.834+0000 7ff356353640 1 -- 172.21.15.1:0/1260307948 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff350101e10 msgr2=0x7ff350198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:11.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:11.834+0000 7ff356353640 1 --2- 172.21.15.1:0/1260307948 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff350101e10 0x7ff350198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:11.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:11.834+0000 7ff356353640 1 -- 172.21.15.1:0/1260307948 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff35019d3e0 con 0x7ff350106f40 2024-09-11T22:02:11.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:11.835+0000 7ff356353640 1 --2- 172.21.15.1:0/1260307948 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff350106f40 0x7ff35019cd00 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7ff34800eb10 tx=0x7ff34800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:11.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:11.835+0000 7ff346ffd640 1 -- 172.21.15.1:0/1260307948 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff348004b20 con 0x7ff350106f40 2024-09-11T22:02:11.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:11.835+0000 7ff357db6640 1 -- 172.21.15.1:0/1260307948 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff35019d6d0 con 0x7ff350106f40 2024-09-11T22:02:11.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:11.835+0000 7ff357db6640 1 -- 172.21.15.1:0/1260307948 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3501a4fb0 con 0x7ff350106f40 2024-09-11T22:02:11.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:11.835+0000 7ff346ffd640 1 -- 172.21.15.1:0/1260307948 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff348004cc0 con 0x7ff350106f40 2024-09-11T22:02:11.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:11.836+0000 7ff346ffd640 1 -- 172.21.15.1:0/1260307948 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff348010760 con 0x7ff350106f40 2024-09-11T22:02:11.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:11.836+0000 7ff357db6640 1 -- 172.21.15.1:0/1260307948 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3501021f0 con 0x7ff350106f40 2024-09-11T22:02:11.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:11.837+0000 7ff346ffd640 1 -- 172.21.15.1:0/1260307948 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7ff348005000 con 0x7ff350106f40 2024-09-11T22:02:11.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:11.838+0000 7ff346ffd640 1 --2- 172.21.15.1:0/1260307948 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7ff33c0a2d70 0x7ff33c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:11.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:11.838+0000 7ff355b52640 1 --2- 172.21.15.1:0/1260307948 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7ff33c0a2d70 0x7ff33c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:11.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:11.838+0000 7ff346ffd640 1 -- 172.21.15.1:0/1260307948 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7ff3480d0060 con 0x7ff350106f40 2024-09-11T22:02:11.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:11.839+0000 7ff355b52640 1 --2- 172.21.15.1:0/1260307948 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7ff33c0a2d70 0x7ff33c0a5230 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff34c005fd0 tx=0x7ff34c00b390 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:11.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:11.842+0000 7ff346ffd640 1 -- 172.21.15.1:0/1260307948 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff34809cc40 con 0x7ff350106f40 2024-09-11T22:02:12.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:12.099+0000 7ff357db6640 1 -- 172.21.15.1:0/1260307948 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff35010a1e0 con 0x7ff350106f40 2024-09-11T22:02:12.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:12.100+0000 7ff346ffd640 1 -- 172.21.15.1:0/1260307948 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11957 (secure 0 0 0) 0x7ff34809c540 con 0x7ff350106f40 2024-09-11T22:02:12.101 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T22:02:12.101 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":33,"btime":"2024-09-11T22:01:51:001787+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/421542832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":421542832},{"type":"v1","addr":"172.21.15.1:6835","nonce":421542832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2836864406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2836864406},{"type":"v1","addr":"172.21.15.1:6837","nonce":2836864406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T21:55:20.791263+0000","modified":"2024-09-11T22:01:50.436812+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24484,"mds_2":24467,"mds_3":24466,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14595":{"gid":14595,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/407834672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":407834672},{"type":"v1","addr":"172.21.15.1:6839","nonce":407834672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/1068827678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":1068827678},{"type":"v1","addr":"172.21.15.1:6841","nonce":1068827678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/872410024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":872410024},{"type":"v1","addr":"172.21.15.161:6833","nonce":872410024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/1648944570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1648944570},{"type":"v1","addr":"172.21.15.53:6833","nonce":1648944570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/2126933993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2126933993},{"type":"v1","addr":"172.21.15.53:6835","nonce":2126933993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.161:6835/828381592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":828381592},{"type":"v1","addr":"172.21.15.161:6835","nonce":828381592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.161:6837/1799318656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1799318656},{"type":"v1","addr":"172.21.15.161:6837","nonce":1799318656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.53:6837/3457131830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3457131830},{"type":"v1","addr":"172.21.15.53:6837","nonce":3457131830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/455365383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":455365383},{"type":"v1","addr":"172.21.15.161:6839","nonce":455365383}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.53:6839/1378584447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1378584447},{"type":"v1","addr":"172.21.15.53:6839","nonce":1378584447}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24484,24467,24466,24460]},"id":1}]} 2024-09-11T22:02:12.101 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 33 2024-09-11T22:02:12.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:12.103+0000 7ff357db6640 1 -- 172.21.15.1:0/1260307948 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7ff33c0a2d70 msgr2=0x7ff33c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:12.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:12.103+0000 7ff357db6640 1 --2- 172.21.15.1:0/1260307948 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7ff33c0a2d70 0x7ff33c0a5230 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff34c005fd0 tx=0x7ff34c00b390 comp rx=0 tx=0).stop 2024-09-11T22:02:12.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:12.103+0000 7ff357db6640 1 -- 172.21.15.1:0/1260307948 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff350106f40 msgr2=0x7ff35019cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:12.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:12.103+0000 7ff357db6640 1 --2- 172.21.15.1:0/1260307948 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff350106f40 0x7ff35019cd00 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7ff34800eb10 tx=0x7ff34800c2d0 comp rx=0 tx=0).stop 2024-09-11T22:02:12.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:12.103+0000 7ff357db6640 1 -- 172.21.15.1:0/1260307948 shutdown_connections 2024-09-11T22:02:12.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:12.103+0000 7ff357db6640 1 --2- 172.21.15.1:0/1260307948 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7ff33c0a2d70 0x7ff33c0a5230 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:12.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:12.103+0000 7ff357db6640 1 --2- 172.21.15.1:0/1260307948 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff350106f40 0x7ff35019cd00 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:12.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:12.103+0000 7ff357db6640 1 --2- 172.21.15.1:0/1260307948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff3501027c0 0x7ff350198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:12.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:12.103+0000 7ff357db6640 1 --2- 172.21.15.1:0/1260307948 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff350101e10 0x7ff350198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:12.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:12.103+0000 7ff357db6640 1 -- 172.21.15.1:0/1260307948 >> 172.21.15.1:0/1260307948 conn(0x7ff3500fd8d0 msgr2=0x7ff3500fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:12.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:12.103+0000 7ff357db6640 1 -- 172.21.15.1:0/1260307948 shutdown_connections 2024-09-11T22:02:12.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:12.103+0000 7ff357db6640 1 -- 172.21.15.1:0/1260307948 wait complete. 2024-09-11T22:02:12.807 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db / --timeout 90 --expiration 140.15 --await-for 45.0' 2024-09-11T22:02:12.807 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db / --timeout 90 --expiration 140.15 --await-for 45.0 2024-09-11T22:02:13.145 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:02:13.294 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:13 smithi001 ceph-mon[30935]: pgmap v154: 97 pgs: 97 active+clean; 82 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.5 MiB/s wr, 1.25k op/s 2024-09-11T22:02:13.294 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:13 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:13.294 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:13 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:13.294 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:13 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:13.294 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:13 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:13.294 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:13 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:13.294 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:13 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/1260307948' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:02:13.295 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:13 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:13.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:13 smithi053 ceph-mon[34908]: pgmap v154: 97 pgs: 97 active+clean; 82 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.5 MiB/s wr, 1.25k op/s 2024-09-11T22:02:13.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:13 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:13.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:13 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:13.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:13 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:13.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:13 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:13.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:13 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:13.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:13 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/1260307948' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:02:13.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:13 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:13.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:13 smithi161 ceph-mon[35615]: pgmap v154: 97 pgs: 97 active+clean; 82 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.5 MiB/s wr, 1.25k op/s 2024-09-11T22:02:13.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:13 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:13.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:13 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:13.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:13 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:13.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:13 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:13.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:13 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:13.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:13 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/1260307948' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:02:13.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:13 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:13.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.821+0000 7f6af2c4b640 1 -- 172.21.15.1:0/792104689 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6aec103740 msgr2=0x7f6aec107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:13.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.821+0000 7f6af2c4b640 1 --2- 172.21.15.1:0/792104689 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6aec103740 0x7f6aec107b80 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f6ad8009930 tx=0x7f6ad802a420 comp rx=0 tx=0).stop 2024-09-11T22:02:13.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.821+0000 7f6af2c4b640 1 -- 172.21.15.1:0/792104689 shutdown_connections 2024-09-11T22:02:13.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.821+0000 7f6af2c4b640 1 --2- 172.21.15.1:0/792104689 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6aec108290 0x7f6aec10ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:13.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.821+0000 7f6af2c4b640 1 --2- 172.21.15.1:0/792104689 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6aec103740 0x7f6aec107b80 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:13.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.821+0000 7f6af2c4b640 1 --2- 172.21.15.1:0/792104689 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6aec102d90 0x7f6aec103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:13.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.821+0000 7f6af2c4b640 1 -- 172.21.15.1:0/792104689 >> 172.21.15.1:0/792104689 conn(0x7f6aec0fc7d0 msgr2=0x7f6aec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:13.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.821+0000 7f6af2c4b640 1 -- 172.21.15.1:0/792104689 shutdown_connections 2024-09-11T22:02:13.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.821+0000 7f6af2c4b640 1 -- 172.21.15.1:0/792104689 wait complete. 2024-09-11T22:02:13.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.822+0000 7f6af2c4b640 1 Processor -- start 2024-09-11T22:02:13.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.822+0000 7f6af2c4b640 1 -- start start 2024-09-11T22:02:13.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.822+0000 7f6af2c4b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6aec102d90 0x7f6aec1963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:13.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.822+0000 7f6af2c4b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6aec103740 0x7f6aec1968e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:13.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.822+0000 7f6af2c4b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6aec108290 0x7f6aec19ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:13.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.822+0000 7f6af2c4b640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6aec10fe80 con 0x7f6aec103740 2024-09-11T22:02:13.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.822+0000 7f6af2c4b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6aec10fd00 con 0x7f6aec102d90 2024-09-11T22:02:13.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.822+0000 7f6af2c4b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6aec110000 con 0x7f6aec108290 2024-09-11T22:02:13.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.823+0000 7f6af09e7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6aec102d90 0x7f6aec1963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:13.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.823+0000 7f6af11e8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6aec108290 0x7f6aec19ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:13.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.823+0000 7f6aebfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6aec103740 0x7f6aec1968e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:13.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.823+0000 7f6af09e7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6aec102d90 0x7f6aec1963a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:47022/0 (socket says 172.21.15.1:47022) 2024-09-11T22:02:13.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.823+0000 7f6af09e7640 1 -- 172.21.15.1:0/2074501376 learned_addr learned my addr 172.21.15.1:0/2074501376 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:13.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.823+0000 7f6af11e8640 1 -- 172.21.15.1:0/2074501376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6aec102d90 msgr2=0x7f6aec1963a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:13.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.823+0000 7f6af11e8640 1 --2- 172.21.15.1:0/2074501376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6aec102d90 0x7f6aec1963a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:13.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.823+0000 7f6af11e8640 1 -- 172.21.15.1:0/2074501376 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6aec103740 msgr2=0x7f6aec1968e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:13.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.823+0000 7f6af11e8640 1 --2- 172.21.15.1:0/2074501376 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6aec103740 0x7f6aec1968e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:13.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.823+0000 7f6af11e8640 1 -- 172.21.15.1:0/2074501376 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6aec19b350 con 0x7f6aec108290 2024-09-11T22:02:13.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.823+0000 7f6af11e8640 1 --2- 172.21.15.1:0/2074501376 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6aec108290 0x7f6aec19ac70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6ae400ed90 tx=0x7f6ae4012490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:13.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.823+0000 7f6ae9ffb640 1 -- 172.21.15.1:0/2074501376 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae401c070 con 0x7f6aec108290 2024-09-11T22:02:13.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.824+0000 7f6af2c4b640 1 -- 172.21.15.1:0/2074501376 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6aec19b640 con 0x7f6aec108290 2024-09-11T22:02:13.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.824+0000 7f6ae9ffb640 1 -- 172.21.15.1:0/2074501376 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ae4005020 con 0x7f6aec108290 2024-09-11T22:02:13.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.824+0000 7f6ae9ffb640 1 -- 172.21.15.1:0/2074501376 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae4018660 con 0x7f6aec108290 2024-09-11T22:02:13.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.824+0000 7f6af2c4b640 1 -- 172.21.15.1:0/2074501376 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6aec1a2f20 con 0x7f6aec108290 2024-09-11T22:02:13.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.827+0000 7f6ae9ffb640 1 -- 172.21.15.1:0/2074501376 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f6ae4017070 con 0x7f6aec108290 2024-09-11T22:02:13.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.827+0000 7f6af5024740 1 Processor -- start 2024-09-11T22:02:13.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.827+0000 7f6af5024740 1 -- start start 2024-09-11T22:02:13.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.827+0000 7f6af5024740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55838f5d1850 0x55838f5d1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:13.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.827+0000 7f6af5024740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55838f5d2170 0x55838f5dc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:13.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.827+0000 7f6af5024740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55838f5dca30 0x55838f5dee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:13.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.827+0000 7f6af5024740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55838f536160 con 0x55838f5dca30 2024-09-11T22:02:13.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.827+0000 7f6af5024740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55838f4dd720 con 0x55838f5d1850 2024-09-11T22:02:13.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.827+0000 7f6af5024740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55838f532120 con 0x55838f5d2170 2024-09-11T22:02:13.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.828+0000 7f6aebfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55838f5d1850 0x55838f5d1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:13.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.828+0000 7f6af09e7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55838f5d2170 0x55838f5dc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:13.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.828+0000 7f6aebfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55838f5d1850 0x55838f5d1c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:47026/0 (socket says 172.21.15.1:47026) 2024-09-11T22:02:13.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.828+0000 7f6aebfff640 1 -- 172.21.15.1:0/1315182304 learned_addr learned my addr 172.21.15.1:0/1315182304 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:13.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.828+0000 7f6aebfff640 1 --2- 172.21.15.1:0/1315182304 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55838f5dca30 0x55838f5dee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:13.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.828+0000 7f6af09e7640 1 -- 172.21.15.1:0/1315182304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55838f5d1850 msgr2=0x55838f5d1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:13.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.828+0000 7f6af09e7640 1 --2- 172.21.15.1:0/1315182304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55838f5d1850 0x55838f5d1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:13.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.828+0000 7f6af09e7640 1 -- 172.21.15.1:0/1315182304 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55838f5dca30 msgr2=0x55838f5dee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:13.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.828+0000 7f6af09e7640 1 --2- 172.21.15.1:0/1315182304 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55838f5dca30 0x55838f5dee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:13.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.828+0000 7f6af09e7640 1 -- 172.21.15.1:0/1315182304 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55838f2a7ba0 con 0x55838f5d2170 2024-09-11T22:02:13.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.828+0000 7f6ae9ffb640 1 --2- 172.21.15.1:0/2074501376 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f6adc0a2cf0 0x7f6adc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:13.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.828+0000 7f6af09e7640 1 --2- 172.21.15.1:0/1315182304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55838f5d2170 0x55838f5dc4f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0009c80 tx=0x7f6ae0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1d243aedf6d95b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:13.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.828+0000 7f6ad6ffd640 1 -- 172.21.15.1:0/1315182304 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae0004250 con 0x55838f5d2170 2024-09-11T22:02:13.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.829+0000 7f6aebfff640 1 --2- 172.21.15.1:0/2074501376 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f6adc0a2cf0 0x7f6adc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:13.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.829+0000 7f6ad6ffd640 1 -- 172.21.15.1:0/1315182304 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ae0009e30 con 0x55838f5d2170 2024-09-11T22:02:13.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.829+0000 7f6ad6ffd640 1 -- 172.21.15.1:0/1315182304 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae0004e20 con 0x55838f5d2170 2024-09-11T22:02:13.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.829+0000 7f6ae9ffb640 1 -- 172.21.15.1:0/2074501376 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f6ae40d32b0 con 0x7f6aec108290 2024-09-11T22:02:13.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.829+0000 7f6af5024740 1 -- 172.21.15.1:0/1315182304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55838f5d2170 msgr2=0x55838f5dc4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:13.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.829+0000 7f6aebfff640 1 --2- 172.21.15.1:0/2074501376 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f6adc0a2cf0 0x7f6adc0a51b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x55838f5cf160 tx=0x7f6ad80058d0 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:13.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.829+0000 7f6af5024740 1 --2- 172.21.15.1:0/1315182304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55838f5d2170 0x55838f5dc4f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0009c80 tx=0x7f6ae0011810 comp rx=0 tx=0).stop 2024-09-11T22:02:13.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.829+0000 7f6af5024740 1 -- 172.21.15.1:0/1315182304 shutdown_connections 2024-09-11T22:02:13.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.829+0000 7f6af5024740 1 --2- 172.21.15.1:0/1315182304 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55838f5dca30 0x55838f5dee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:13.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.829+0000 7f6af5024740 1 --2- 172.21.15.1:0/1315182304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55838f5d2170 0x55838f5dc4f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:13.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.829+0000 7f6af5024740 1 --2- 172.21.15.1:0/1315182304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55838f5d1850 0x55838f5d1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:13.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.829+0000 7f6af5024740 1 -- 172.21.15.1:0/1315182304 >> 172.21.15.1:0/1315182304 conn(0x55838f5cdde0 msgr2=0x55838f5ce1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:13.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.830+0000 7f6af5024740 1 -- 172.21.15.1:0/1315182304 shutdown_connections 2024-09-11T22:02:13.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.830+0000 7f6af5024740 1 -- 172.21.15.1:0/1315182304 wait complete. 2024-09-11T22:02:13.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.830+0000 7f6af5024740 1 -- start start 2024-09-11T22:02:13.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.830+0000 7f6af5024740 1 Processor -- start 2024-09-11T22:02:13.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.830+0000 7f6af5024740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55838f5d1850 0x55838f5d9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:13.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.830+0000 7f6af5024740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55838f5d2170 0x55838f5d9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:13.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.831+0000 7f6af5024740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55838f5dca30 0x55838f5da310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:13.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.831+0000 7f6af5024740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55838f5df600 con 0x55838f5d2170 2024-09-11T22:02:13.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.831+0000 7f6af5024740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55838f4be900 con 0x55838f5d1850 2024-09-11T22:02:13.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.831+0000 7f6af5024740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55838f5df780 con 0x55838f5dca30 2024-09-11T22:02:13.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.831+0000 7f6af5024740 10 client.0 _ll_register_callbacks cb 0x55838f4bc780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:02:13.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.831+0000 7f6aebfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55838f5d2170 0x55838f5d9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:13.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.831+0000 7f6aebfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55838f5d2170 0x55838f5d9dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49468/0 (socket says 172.21.15.1:49468) 2024-09-11T22:02:13.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.831+0000 7f6aebfff640 1 -- 172.21.15.1:0/4253416990 learned_addr learned my addr 172.21.15.1:0/4253416990 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:13.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.831+0000 7f6af09e7640 1 --2- 172.21.15.1:0/4253416990 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55838f5d1850 0x55838f5d9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:13.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.831+0000 7f6af11e8640 1 --2- 172.21.15.1:0/4253416990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55838f5dca30 0x55838f5da310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:13.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.831+0000 7f6aebfff640 1 -- 172.21.15.1:0/4253416990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55838f5dca30 msgr2=0x55838f5da310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:13.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.831+0000 7f6aebfff640 1 --2- 172.21.15.1:0/4253416990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55838f5dca30 0x55838f5da310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:13.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.831+0000 7f6aebfff640 1 -- 172.21.15.1:0/4253416990 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55838f5d1850 msgr2=0x55838f5d9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:13.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.831+0000 7f6aebfff640 1 --2- 172.21.15.1:0/4253416990 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55838f5d1850 0x55838f5d9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:13.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.831+0000 7f6aebfff640 1 -- 172.21.15.1:0/4253416990 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55838f4fec50 con 0x55838f5d2170 2024-09-11T22:02:13.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.831+0000 7f6aebfff640 1 --2- 172.21.15.1:0/4253416990 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55838f5d2170 0x55838f5d9dd0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f6ad8005780 tx=0x7f6ad802a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=533eba75a98d6273 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:13.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.831+0000 7f6acbfff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:02:13.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.831+0000 7f6acbfff640 1 -- 172.21.15.1:0/4253416990 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad804f030 con 0x55838f5d2170 2024-09-11T22:02:13.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.831+0000 7f6acbfff640 1 -- 172.21.15.1:0/4253416990 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f6ad80035f0 con 0x55838f5d2170 2024-09-11T22:02:13.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.831+0000 7f6af5024740 1 -- 172.21.15.1:0/4253416990 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ae0009e30 con 0x55838f5d2170 2024-09-11T22:02:13.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.832+0000 7f6acbfff640 1 -- 172.21.15.1:0/4253416990 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f6ad8035040 con 0x55838f5d2170 2024-09-11T22:02:13.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.832+0000 7f6af5024740 10 client.16458 fetch_fsmap learned FSMap version 33 2024-09-11T22:02:13.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.832+0000 7f6af5024740 1 -- 172.21.15.1:0/4253416990 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55838f5eb230 con 0x55838f5d2170 2024-09-11T22:02:13.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.832+0000 7f6acbfff640 1 -- 172.21.15.1:0/4253416990 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ae0009e30 con 0x55838f5d2170 2024-09-11T22:02:13.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.832+0000 7f6acbfff640 1 -- 172.21.15.1:0/4253416990 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7f6ad8048050 con 0x55838f5d2170 2024-09-11T22:02:13.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.832+0000 7f6af5024740 10 client.16458 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T22:02:13.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.832+0000 7f6af5024740 10 client.16458 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-11T22:02:13.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.832+0000 7f6af5024740 20 client.16458 populate_metadata read hostname 'smithi001' 2024-09-11T22:02:13.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.832+0000 7f6af5024740 1 --2- 172.21.15.1:0/4253416990 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55838f5ec700 0x55838f5eebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:13.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.832+0000 7f6af5024740 4 client.16458 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:02:13.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.832+0000 7f6af5024740 1 -- 172.21.15.1:0/4253416990 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55838f543890 con 0x55838f5ec700 2024-09-11T22:02:13.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.832+0000 7f6af09e7640 1 --2- 172.21.15.1:0/4253416990 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55838f5ec700 0x55838f5eebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:13.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.833+0000 7f6acbfff640 10 client.16458 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:02:13.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.833+0000 7f6af09e7640 1 --2- 172.21.15.1:0/4253416990 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55838f5ec700 0x55838f5eebc0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1451d9e34228bcc2 server_cookie=a14a089201cbe2d6 in_seq=0 out_seq=0 2024-09-11T22:02:13.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.838+0000 7f6acbfff640 1 -- 172.21.15.1:0/4253416990 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55838f543890 con 0x55838f5ec700 2024-09-11T22:02:13.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.838+0000 7f6acbfff640 10 client.16458 handle_command_reply: tid=0 2024-09-11T22:02:13.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.838+0000 7f6af5024740 1 client.16458 shutdown 2024-09-11T22:02:13.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.838+0000 7f6af5024740 1 -- 172.21.15.1:0/4253416990 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55838f5d2170 msgr2=0x55838f5d9dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:13.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.838+0000 7f6af5024740 1 --2- 172.21.15.1:0/4253416990 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55838f5d2170 0x55838f5d9dd0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f6ad8005780 tx=0x7f6ad802a420 comp rx=0 tx=0).stop 2024-09-11T22:02:13.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.838+0000 7f6af5024740 1 -- 172.21.15.1:0/4253416990 shutdown_connections 2024-09-11T22:02:13.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.838+0000 7f6af5024740 1 --2- 172.21.15.1:0/4253416990 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55838f5ec700 0x55838f5eebc0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:13.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.838+0000 7f6af5024740 1 --2- 172.21.15.1:0/4253416990 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55838f5dca30 0x55838f5da310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:13.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.838+0000 7f6af5024740 1 --2- 172.21.15.1:0/4253416990 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55838f5d2170 0x55838f5d9dd0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:13.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.838+0000 7f6af5024740 1 --2- 172.21.15.1:0/4253416990 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55838f5d1850 0x55838f5d9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:13.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.838+0000 7f6acbfff640 0 client.16458 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:02:13.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.838+0000 7f6af5024740 1 -- 172.21.15.1:0/4253416990 >> 172.21.15.1:0/4253416990 conn(0x55838f5cdde0 msgr2=0x55838f5d3ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:13.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.839+0000 7f6af5024740 1 -- 172.21.15.1:0/4253416990 shutdown_connections 2024-09-11T22:02:13.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.839+0000 7f6af5024740 1 -- 172.21.15.1:0/4253416990 wait complete. 2024-09-11T22:02:13.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.839+0000 7f6af5024740 20 client.16458 trim_cache size 0 max 16384 2024-09-11T22:02:13.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.860+0000 7f6af5024740 1 Processor -- start 2024-09-11T22:02:13.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.860+0000 7f6af5024740 1 -- start start 2024-09-11T22:02:13.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.860+0000 7f6af5024740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55838f5d1850 0x55838f5f44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:13.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.860+0000 7f6af5024740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55838f5d2170 0x55838f5f4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:13.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.860+0000 7f6af5024740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55838f5dca30 0x55838f5d6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:13.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.860+0000 7f6af5024740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6aec1ab980 con 0x55838f5dca30 2024-09-11T22:02:13.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.860+0000 7f6af5024740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6aec1ad5b0 con 0x55838f5d2170 2024-09-11T22:02:13.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.860+0000 7f6af5024740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6aec1ac050 con 0x55838f5d1850 2024-09-11T22:02:13.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.860+0000 7f6af11e8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55838f5dca30 0x55838f5d6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:13.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.860+0000 7f6af11e8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55838f5dca30 0x55838f5d6af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49476/0 (socket says 172.21.15.1:49476) 2024-09-11T22:02:13.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.860+0000 7f6af11e8640 1 -- 172.21.15.1:0/2344009662 learned_addr learned my addr 172.21.15.1:0/2344009662 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:13.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.860+0000 7f6af09e7640 1 --2- 172.21.15.1:0/2344009662 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55838f5d1850 0x55838f5f44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:13.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.860+0000 7f6aebfff640 1 --2- 172.21.15.1:0/2344009662 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55838f5d2170 0x55838f5f4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:13.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.860+0000 7f6af11e8640 1 -- 172.21.15.1:0/2344009662 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55838f5d1850 msgr2=0x55838f5f44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:13.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.860+0000 7f6af11e8640 1 --2- 172.21.15.1:0/2344009662 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55838f5d1850 0x55838f5f44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:13.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.860+0000 7f6af11e8640 1 -- 172.21.15.1:0/2344009662 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55838f5d2170 msgr2=0x55838f5f4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:13.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.860+0000 7f6af11e8640 1 --2- 172.21.15.1:0/2344009662 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55838f5d2170 0x55838f5f4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:13.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.860+0000 7f6af11e8640 1 -- 172.21.15.1:0/2344009662 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55838f5cea70 con 0x55838f5dca30 2024-09-11T22:02:13.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.861+0000 7f6af11e8640 1 --2- 172.21.15.1:0/2344009662 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55838f5dca30 0x55838f5d6af0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4012990 tx=0x7f6ae409b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=747268d2b4b805af server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:13.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.861+0000 7f6ad67fc640 1 -- 172.21.15.1:0/2344009662 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae40b7070 con 0x55838f5dca30 2024-09-11T22:02:13.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.861+0000 7f6ad67fc640 1 -- 172.21.15.1:0/2344009662 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ae40d3040 con 0x55838f5dca30 2024-09-11T22:02:13.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.861+0000 7f6ad67fc640 1 -- 172.21.15.1:0/2344009662 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae40055b0 con 0x55838f5dca30 2024-09-11T22:02:13.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.862+0000 7f6af5024740 1 -- 172.21.15.1:0/2344009662 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55838f5dca30 msgr2=0x55838f5d6af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:13.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.862+0000 7f6af5024740 1 --2- 172.21.15.1:0/2344009662 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55838f5dca30 0x55838f5d6af0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4012990 tx=0x7f6ae409b3e0 comp rx=0 tx=0).stop 2024-09-11T22:02:13.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.862+0000 7f6af5024740 1 -- 172.21.15.1:0/2344009662 shutdown_connections 2024-09-11T22:02:13.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.862+0000 7f6af5024740 1 --2- 172.21.15.1:0/2344009662 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55838f5dca30 0x55838f5d6af0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:13.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.862+0000 7f6af5024740 1 --2- 172.21.15.1:0/2344009662 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55838f5d2170 0x55838f5f4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:13.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.862+0000 7f6af5024740 1 --2- 172.21.15.1:0/2344009662 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55838f5d1850 0x55838f5f44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:13.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.862+0000 7f6af5024740 1 -- 172.21.15.1:0/2344009662 >> 172.21.15.1:0/2344009662 conn(0x55838f5cdde0 msgr2=0x55838f5db910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:13.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.862+0000 7f6af5024740 1 -- 172.21.15.1:0/2344009662 shutdown_connections 2024-09-11T22:02:13.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.862+0000 7f6af5024740 1 -- 172.21.15.1:0/2344009662 wait complete. 2024-09-11T22:02:13.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.862+0000 7f6af5024740 1 -- start start 2024-09-11T22:02:13.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.862+0000 7f6af5024740 1 Processor -- start 2024-09-11T22:02:13.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.863+0000 7f6af5024740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55838f5d1850 0x55838f5e4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:13.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.863+0000 7f6af5024740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55838f5d2170 0x55838f5e1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:13.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.863+0000 7f6af5024740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55838f5dca30 0x55838f5e16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:13.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.863+0000 7f6af5024740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6aec1ace20 con 0x55838f5dca30 2024-09-11T22:02:13.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.863+0000 7f6af5024740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6aec1ac720 con 0x55838f5d2170 2024-09-11T22:02:13.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.863+0000 7f6af5024740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6aec1a9960 con 0x55838f5d1850 2024-09-11T22:02:13.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.863+0000 7f6af5024740 10 client.0 _ll_register_callbacks cb 0x7f6ad8041050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:02:13.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.863+0000 7f6af09e7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55838f5d1850 0x55838f5e4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:13.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.863+0000 7f6af09e7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55838f5d1850 0x55838f5e4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:58574/0 (socket says 172.21.15.1:58574) 2024-09-11T22:02:13.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.863+0000 7f6aebfff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55838f5d2170 0x55838f5e1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:13.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.863+0000 7f6af09e7640 1 -- 172.21.15.1:0/1324922187 learned_addr learned my addr 172.21.15.1:0/1324922187 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:13.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.863+0000 7f6af11e8640 1 --2- 172.21.15.1:0/1324922187 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55838f5dca30 0x55838f5e16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:13.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.863+0000 7f6af09e7640 1 -- 172.21.15.1:0/1324922187 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55838f5d2170 msgr2=0x55838f5e1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:13.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.863+0000 7f6af09e7640 1 --2- 172.21.15.1:0/1324922187 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55838f5d2170 0x55838f5e1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:13.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.863+0000 7f6af09e7640 1 -- 172.21.15.1:0/1324922187 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55838f5dca30 msgr2=0x55838f5e16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:13.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.863+0000 7f6af09e7640 1 --2- 172.21.15.1:0/1324922187 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55838f5dca30 0x55838f5e16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:13.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.863+0000 7f6af09e7640 1 -- 172.21.15.1:0/1324922187 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55838f5e1bf0 con 0x55838f5d1850 2024-09-11T22:02:13.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.863+0000 7f6aebfff640 1 --2- 172.21.15.1:0/1324922187 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55838f5d2170 0x55838f5e1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:02:13.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.864+0000 7f6af09e7640 1 --2- 172.21.15.1:0/1324922187 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55838f5d1850 0x55838f5e4e30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6ae001b040 tx=0x7f6ae0011dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51a67bf2cb8e8798 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:13.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.864+0000 7f6acbfff640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-11T22:02:13.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.864+0000 7f6acbfff640 1 -- 172.21.15.1:0/1324922187 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae0004240 con 0x55838f5d1850 2024-09-11T22:02:13.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.864+0000 7f6acbfff640 1 -- 172.21.15.1:0/1324922187 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f6ae00043e0 con 0x55838f5d1850 2024-09-11T22:02:13.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.864+0000 7f6af5024740 1 -- 172.21.15.1:0/1324922187 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ae40d3040 con 0x55838f5d1850 2024-09-11T22:02:13.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.864+0000 7f6acbfff640 1 -- 172.21.15.1:0/1324922187 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f6ae003e050 con 0x55838f5d1850 2024-09-11T22:02:13.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.865+0000 7f6af5024740 10 client.25969 fetch_fsmap learned FSMap version 33 2024-09-11T22:02:13.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.865+0000 7f6af5024740 1 -- 172.21.15.1:0/1324922187 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55838f6581a0 con 0x55838f5d1850 2024-09-11T22:02:13.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.865+0000 7f6acbfff640 1 -- 172.21.15.1:0/1324922187 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ae0032030 con 0x55838f5d1850 2024-09-11T22:02:13.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.865+0000 7f6acbfff640 1 -- 172.21.15.1:0/1324922187 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7f6ae0037580 con 0x55838f5d1850 2024-09-11T22:02:13.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.865+0000 7f6af5024740 10 client.25969 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T22:02:13.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.865+0000 7f6af5024740 10 client.25969 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-11T22:02:13.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.865+0000 7f6af5024740 20 client.25969 populate_metadata read hostname 'smithi001' 2024-09-11T22:02:13.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.865+0000 7f6af5024740 1 --2- 172.21.15.1:0/1324922187 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55838f658830 0x55838f65ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:13.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.865+0000 7f6af5024740 4 client.25969 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.15, "await_for": 45.0}] 2024-09-11T22:02:13.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.865+0000 7f6af5024740 1 -- 172.21.15.1:0/1324922187 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.15, "await_for": 45.0}) -- 0x7f6ae40d3040 con 0x55838f658830 2024-09-11T22:02:13.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.865+0000 7f6aebfff640 1 --2- 172.21.15.1:0/1324922187 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55838f658830 0x55838f65ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:13.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.866+0000 7f6acbfff640 10 client.25969 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:02:13.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:13.866+0000 7f6aebfff640 1 --2- 172.21.15.1:0/1324922187 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55838f658830 0x55838f65ac50 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad005a63b05dc9e8 server_cookie=265f904dfe784b35 in_seq=0 out_seq=0 2024-09-11T22:02:14.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:14 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:14.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:14 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:14.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:14 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:15.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:15 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:15.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:15 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:15.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:15 smithi053 ceph-mon[34908]: pgmap v155: 97 pgs: 97 active+clean; 82 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.5 MiB/s wr, 1.25k op/s 2024-09-11T22:02:15.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:15 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:15.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:15 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:15.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:15 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:15.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:15 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:15.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:15 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:15.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:15 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:15.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:15 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:15.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:15 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:15.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:15 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:15.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:15 smithi161 ceph-mon[35615]: pgmap v155: 97 pgs: 97 active+clean; 82 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.5 MiB/s wr, 1.25k op/s 2024-09-11T22:02:15.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:15 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:15.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:15 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:15.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:15 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:15.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:15 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:15.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:15 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:15.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:15 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:15.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:15 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:15.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:15 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:15.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:15 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:15.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:15 smithi001 ceph-mon[30935]: pgmap v155: 97 pgs: 97 active+clean; 82 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.5 MiB/s wr, 1.25k op/s 2024-09-11T22:02:15.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:15 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:15.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:15 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:15.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:15 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:15.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:15 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:15.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:15 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:15.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:15 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:15.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:15 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:15.656 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.655+0000 7f290ab0a640 1 -- 172.21.15.1:0/3114212465 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2904102d60 msgr2=0x7f2904103140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:15.656 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.655+0000 7f290ab0a640 1 --2- 172.21.15.1:0/3114212465 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2904102d60 0x7f2904103140 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f28f8009900 tx=0x7f28f802a480 comp rx=0 tx=0).stop 2024-09-11T22:02:15.656 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.655+0000 7f290ab0a640 1 -- 172.21.15.1:0/3114212465 shutdown_connections 2024-09-11T22:02:15.656 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.655+0000 7f290ab0a640 1 --2- 172.21.15.1:0/3114212465 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f290410c0a0 0x7f290410e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:15.656 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.655+0000 7f290ab0a640 1 --2- 172.21.15.1:0/3114212465 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2904103680 0x7f290410bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:15.656 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.655+0000 7f290ab0a640 1 --2- 172.21.15.1:0/3114212465 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2904102d60 0x7f2904103140 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:15.656 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.655+0000 7f290ab0a640 1 -- 172.21.15.1:0/3114212465 >> 172.21.15.1:0/3114212465 conn(0x7f29040779a0 msgr2=0x7f29040fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:15.656 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.655+0000 7f290ab0a640 1 -- 172.21.15.1:0/3114212465 shutdown_connections 2024-09-11T22:02:15.657 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.655+0000 7f290ab0a640 1 -- 172.21.15.1:0/3114212465 wait complete. 2024-09-11T22:02:15.658 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.656+0000 7f290ab0a640 1 Processor -- start 2024-09-11T22:02:15.658 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.656+0000 7f290ab0a640 1 -- start start 2024-09-11T22:02:15.658 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.656+0000 7f290ab0a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2904102d60 0x7f29041009b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:15.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.656+0000 7f290ab0a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2904103680 0x7f2904100ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:15.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.656+0000 7f290ab0a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f290410c0a0 0x7f2904101430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:15.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.656+0000 7f290ab0a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2904111ea0 con 0x7f290410c0a0 2024-09-11T22:02:15.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.656+0000 7f290ab0a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2904111d20 con 0x7f2904102d60 2024-09-11T22:02:15.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.656+0000 7f290ab0a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2904112020 con 0x7f2904103680 2024-09-11T22:02:15.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.656+0000 7f29090a7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f290410c0a0 0x7f2904101430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:15.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.656+0000 7f29088a6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2904102d60 0x7f29041009b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:15.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.656+0000 7f29090a7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f290410c0a0 0x7f2904101430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49512/0 (socket says 172.21.15.1:49512) 2024-09-11T22:02:15.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.656+0000 7f29088a6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2904102d60 0x7f29041009b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:47072/0 (socket says 172.21.15.1:47072) 2024-09-11T22:02:15.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.656+0000 7f29090a7640 1 -- 172.21.15.1:0/1001761841 learned_addr learned my addr 172.21.15.1:0/1001761841 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:15.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.656+0000 7f29090a7640 1 -- 172.21.15.1:0/1001761841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2904103680 msgr2=0x7f2904100ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:15.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.656+0000 7f29090a7640 1 --2- 172.21.15.1:0/1001761841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2904103680 0x7f2904100ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:15.660 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.656+0000 7f29090a7640 1 -- 172.21.15.1:0/1001761841 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2904102d60 msgr2=0x7f29041009b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:15.660 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.656+0000 7f29090a7640 1 --2- 172.21.15.1:0/1001761841 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2904102d60 0x7f29041009b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:15.660 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.656+0000 7f29090a7640 1 -- 172.21.15.1:0/1001761841 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29041a6910 con 0x7f290410c0a0 2024-09-11T22:02:15.660 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.656+0000 7f29090a7640 1 --2- 172.21.15.1:0/1001761841 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f290410c0a0 0x7f2904101430 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f28fc00eb10 tx=0x7f28fc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:15.660 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.657+0000 7f2901ffb640 1 -- 172.21.15.1:0/1001761841 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28fc004b20 con 0x7f290410c0a0 2024-09-11T22:02:15.660 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.657+0000 7f2901ffb640 1 -- 172.21.15.1:0/1001761841 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28fc004cc0 con 0x7f290410c0a0 2024-09-11T22:02:15.660 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.657+0000 7f290ab0a640 1 -- 172.21.15.1:0/1001761841 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29041a6c00 con 0x7f290410c0a0 2024-09-11T22:02:15.660 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.657+0000 7f2901ffb640 1 -- 172.21.15.1:0/1001761841 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28fc010760 con 0x7f290410c0a0 2024-09-11T22:02:15.660 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.657+0000 7f290ab0a640 1 -- 172.21.15.1:0/1001761841 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29041a7150 con 0x7f290410c0a0 2024-09-11T22:02:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.660+0000 7f290d89e740 1 Processor -- start 2024-09-11T22:02:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.660+0000 7f2901ffb640 1 -- 172.21.15.1:0/1001761841 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f28fc005020 con 0x7f290410c0a0 2024-09-11T22:02:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.660+0000 7f290d89e740 1 -- start start 2024-09-11T22:02:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.660+0000 7f290d89e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c510bdbaf0 0x55c510bdbed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.660+0000 7f290d89e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c510bdc410 0x55c510be6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.660+0000 7f290d89e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c510be6cb0 0x55c510be90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.660+0000 7f290d89e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55c510b402f0 con 0x55c510bdc410 2024-09-11T22:02:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.660+0000 7f290d89e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55c510ae8270 con 0x55c510bdbaf0 2024-09-11T22:02:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.660+0000 7f290d89e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c510b3c2b0 con 0x55c510be6cb0 2024-09-11T22:02:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.660+0000 7f29088a6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c510bdc410 0x55c510be6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.660+0000 7f2903fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c510bdbaf0 0x55c510bdbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.660+0000 7f29088a6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c510bdc410 0x55c510be6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49526/0 (socket says 172.21.15.1:49526) 2024-09-11T22:02:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.660+0000 7f2903fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c510bdbaf0 0x55c510bdbed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:47088/0 (socket says 172.21.15.1:47088) 2024-09-11T22:02:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.660+0000 7f29088a6640 1 -- 172.21.15.1:0/1080673089 learned_addr learned my addr 172.21.15.1:0/1080673089 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.660+0000 7f2903fff640 1 --2- 172.21.15.1:0/1080673089 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c510be6cb0 0x55c510be90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.661+0000 7f2901ffb640 1 --2- 172.21.15.1:0/1001761841 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f28f40a2d70 0x7f28f40a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.661+0000 7f29088a6640 1 -- 172.21.15.1:0/1080673089 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c510be6cb0 msgr2=0x55c510be90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.661+0000 7f29088a6640 1 --2- 172.21.15.1:0/1080673089 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c510be6cb0 0x55c510be90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.661+0000 7f29088a6640 1 -- 172.21.15.1:0/1080673089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c510bdbaf0 msgr2=0x55c510bdbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.661+0000 7f29088a6640 1 --2- 172.21.15.1:0/1080673089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c510bdbaf0 0x55c510bdbed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.661+0000 7f29088a6640 1 -- 172.21.15.1:0/1080673089 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c510ae7340 con 0x55c510bdc410 2024-09-11T22:02:15.665 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.661+0000 7f2903fff640 1 --2- 172.21.15.1:0/1080673089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c510bdbaf0 0x55c510bdbed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:02:15.665 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.661+0000 7f2901ffb640 1 -- 172.21.15.1:0/1001761841 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f28fc0cf9d0 con 0x7f290410c0a0 2024-09-11T22:02:15.665 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.661+0000 7f29090a7640 1 --2- 172.21.15.1:0/1001761841 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f28f40a2d70 0x7f28f40a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:15.665 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.661+0000 7f29088a6640 1 --2- 172.21.15.1:0/1080673089 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c510bdc410 0x55c510be6770 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f28f8009900 tx=0x7f28f8005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc46a2aeb1adb3f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:15.665 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.661+0000 7f28eeffd640 1 -- 172.21.15.1:0/1080673089 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f8040070 con 0x55c510bdc410 2024-09-11T22:02:15.665 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.661+0000 7f28eeffd640 1 -- 172.21.15.1:0/1080673089 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28f8004030 con 0x55c510bdc410 2024-09-11T22:02:15.665 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.661+0000 7f28eeffd640 1 -- 172.21.15.1:0/1080673089 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f8035040 con 0x55c510bdc410 2024-09-11T22:02:15.665 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.662+0000 7f290d89e740 1 -- 172.21.15.1:0/1080673089 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c510bdc410 msgr2=0x55c510be6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:15.665 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.662+0000 7f29090a7640 1 --2- 172.21.15.1:0/1001761841 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f28f40a2d70 0x7f28f40a5230 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f28fc0cd190 tx=0x7f28fc0bc040 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:15.665 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.662+0000 7f290d89e740 1 --2- 172.21.15.1:0/1080673089 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c510bdc410 0x55c510be6770 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f28f8009900 tx=0x7f28f8005be0 comp rx=0 tx=0).stop 2024-09-11T22:02:15.666 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.662+0000 7f290d89e740 1 -- 172.21.15.1:0/1080673089 shutdown_connections 2024-09-11T22:02:15.666 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.662+0000 7f290d89e740 1 --2- 172.21.15.1:0/1080673089 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c510be6cb0 0x55c510be90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:15.666 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.662+0000 7f290d89e740 1 --2- 172.21.15.1:0/1080673089 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c510bdc410 0x55c510be6770 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:15.666 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.662+0000 7f290d89e740 1 --2- 172.21.15.1:0/1080673089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c510bdbaf0 0x55c510bdbed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:15.666 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.662+0000 7f290d89e740 1 -- 172.21.15.1:0/1080673089 >> 172.21.15.1:0/1080673089 conn(0x55c510bd8060 msgr2=0x55c510bd8440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:15.666 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.662+0000 7f290d89e740 1 -- 172.21.15.1:0/1080673089 shutdown_connections 2024-09-11T22:02:15.666 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.662+0000 7f290d89e740 1 -- 172.21.15.1:0/1080673089 wait complete. 2024-09-11T22:02:15.666 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.662+0000 7f290d89e740 1 -- start start 2024-09-11T22:02:15.666 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.663+0000 7f290d89e740 1 Processor -- start 2024-09-11T22:02:15.670 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.663+0000 7f290d89e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c510bdbaf0 0x55c510be3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:15.671 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.663+0000 7f290d89e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c510bdc410 0x55c510be4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:15.671 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.663+0000 7f290d89e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c510be6cb0 0x55c510be4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:15.671 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.663+0000 7f290d89e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55c510be9880 con 0x55c510bdbaf0 2024-09-11T22:02:15.671 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.663+0000 7f290d89e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55c510ac9290 con 0x55c510be6cb0 2024-09-11T22:02:15.671 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.663+0000 7f290d89e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c510be9a00 con 0x55c510bdc410 2024-09-11T22:02:15.671 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.663+0000 7f290d89e740 10 client.0 _ll_register_callbacks cb 0x55c510afa140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:02:15.671 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.663+0000 7f2903fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c510bdbaf0 0x55c510be3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:15.671 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.663+0000 7f2903fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c510bdbaf0 0x55c510be3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49528/0 (socket says 172.21.15.1:49528) 2024-09-11T22:02:15.671 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.663+0000 7f2903fff640 1 -- 172.21.15.1:0/3503059037 learned_addr learned my addr 172.21.15.1:0/3503059037 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:15.672 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.663+0000 7f29088a6640 1 --2- 172.21.15.1:0/3503059037 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c510bdc410 0x55c510be4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:15.672 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.663+0000 7f2903fff640 1 --2- 172.21.15.1:0/3503059037 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c510be6cb0 0x55c510be4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:15.672 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.664+0000 7f2903fff640 1 -- 172.21.15.1:0/3503059037 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c510bdc410 msgr2=0x55c510be4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:15.672 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.664+0000 7f2903fff640 1 --2- 172.21.15.1:0/3503059037 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c510bdc410 0x55c510be4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:15.672 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.664+0000 7f2903fff640 1 -- 172.21.15.1:0/3503059037 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c510be6cb0 msgr2=0x55c510be4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:15.672 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.664+0000 7f2903fff640 1 --2- 172.21.15.1:0/3503059037 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c510be6cb0 0x55c510be4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:15.672 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.664+0000 7f2903fff640 1 -- 172.21.15.1:0/3503059037 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c510853210 con 0x55c510bdbaf0 2024-09-11T22:02:15.672 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.664+0000 7f2903fff640 1 --2- 172.21.15.1:0/3503059037 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c510bdbaf0 0x55c510be3b00 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f28f00104d0 tx=0x7f28f0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e21a3626c570fb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:15.672 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.664+0000 7f28e3fff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:02:15.672 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.664+0000 7f28e3fff640 1 -- 172.21.15.1:0/3503059037 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f0004460 con 0x55c510bdbaf0 2024-09-11T22:02:15.672 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.664+0000 7f28e3fff640 1 -- 172.21.15.1:0/3503059037 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f28f00035f0 con 0x55c510bdbaf0 2024-09-11T22:02:15.672 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.664+0000 7f290d89e740 1 -- 172.21.15.1:0/3503059037 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28f8004030 con 0x55c510bdbaf0 2024-09-11T22:02:15.673 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.664+0000 7f28e3fff640 1 -- 172.21.15.1:0/3503059037 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f28f0026050 con 0x55c510bdbaf0 2024-09-11T22:02:15.673 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.664+0000 7f290d89e740 10 client.16476 fetch_fsmap learned FSMap version 33 2024-09-11T22:02:15.673 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.664+0000 7f290d89e740 1 -- 172.21.15.1:0/3503059037 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c51091f620 con 0x55c510bdbaf0 2024-09-11T22:02:15.673 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.665+0000 7f28e3fff640 1 -- 172.21.15.1:0/3503059037 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28f8004030 con 0x55c510bdbaf0 2024-09-11T22:02:15.673 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.665+0000 7f28e3fff640 1 -- 172.21.15.1:0/3503059037 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7f28f0021710 con 0x55c510bdbaf0 2024-09-11T22:02:15.673 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.665+0000 7f290d89e740 10 client.16476 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T22:02:15.673 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.665+0000 7f290d89e740 10 client.16476 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:02:15.673 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.665+0000 7f290d89e740 20 client.16476 populate_metadata read hostname 'smithi001' 2024-09-11T22:02:15.673 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.665+0000 7f290d89e740 1 --2- 172.21.15.1:0/3503059037 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55c510bf6c20 0x55c510bf9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:15.673 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.665+0000 7f290d89e740 4 client.16476 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:02:15.673 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.665+0000 7f290d89e740 1 -- 172.21.15.1:0/3503059037 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c510b4da20 con 0x55c510bf6c20 2024-09-11T22:02:15.673 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.665+0000 7f29088a6640 1 --2- 172.21.15.1:0/3503059037 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55c510bf6c20 0x55c510bf9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:15.673 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.666+0000 7f29088a6640 1 --2- 172.21.15.1:0/3503059037 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55c510bf6c20 0x55c510bf9080 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1a2c830058120dd server_cookie=5868d015b0c8c9a5 in_seq=0 out_seq=0 2024-09-11T22:02:15.674 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.666+0000 7f28e3fff640 10 client.16476 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:02:15.674 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.666+0000 7f28e3fff640 1 -- 172.21.15.1:0/3503059037 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c510b4da20 con 0x55c510bf6c20 2024-09-11T22:02:15.674 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.666+0000 7f28e3fff640 10 client.16476 handle_command_reply: tid=0 2024-09-11T22:02:15.674 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.667+0000 7f290d89e740 1 client.16476 shutdown 2024-09-11T22:02:15.674 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.667+0000 7f290d89e740 1 -- 172.21.15.1:0/3503059037 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c510bdbaf0 msgr2=0x55c510be3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:15.674 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.667+0000 7f290d89e740 1 --2- 172.21.15.1:0/3503059037 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c510bdbaf0 0x55c510be3b00 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f28f00104d0 tx=0x7f28f0005f90 comp rx=0 tx=0).stop 2024-09-11T22:02:15.674 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.667+0000 7f290d89e740 1 -- 172.21.15.1:0/3503059037 shutdown_connections 2024-09-11T22:02:15.674 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.667+0000 7f290d89e740 1 --2- 172.21.15.1:0/3503059037 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55c510bf6c20 0x55c510bf9080 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:15.674 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.667+0000 7f28e3fff640 0 client.16476 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:02:15.674 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.667+0000 7f290d89e740 1 --2- 172.21.15.1:0/3503059037 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c510be6cb0 0x55c510be4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:15.674 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.667+0000 7f290d89e740 1 --2- 172.21.15.1:0/3503059037 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c510bdc410 0x55c510be4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:15.674 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.667+0000 7f290d89e740 1 --2- 172.21.15.1:0/3503059037 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c510bdbaf0 0x55c510be3b00 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:15.675 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.667+0000 7f290d89e740 1 -- 172.21.15.1:0/3503059037 >> 172.21.15.1:0/3503059037 conn(0x55c510bd8060 msgr2=0x55c510bdddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:15.675 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.667+0000 7f290d89e740 1 -- 172.21.15.1:0/3503059037 shutdown_connections 2024-09-11T22:02:15.675 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.667+0000 7f290d89e740 1 -- 172.21.15.1:0/3503059037 wait complete. 2024-09-11T22:02:15.675 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.667+0000 7f290d89e740 20 client.16476 trim_cache size 0 max 16384 2024-09-11T22:02:15.687 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.685+0000 7f290d89e740 1 Processor -- start 2024-09-11T22:02:15.687 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.685+0000 7f290d89e740 1 -- start start 2024-09-11T22:02:15.687 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.685+0000 7f290d89e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c510bdbaf0 0x55c510bde560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:15.687 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.685+0000 7f290d89e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c510bdc410 0x55c510be4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:15.688 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.685+0000 7f290d89e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c510be6cb0 0x55c510be4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:15.688 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.685+0000 7f290d89e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f29041afc30 con 0x55c510bdbaf0 2024-09-11T22:02:15.688 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.685+0000 7f290d89e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f29041b1860 con 0x55c510bdc410 2024-09-11T22:02:15.688 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.685+0000 7f290d89e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f29041b0300 con 0x55c510be6cb0 2024-09-11T22:02:15.688 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.685+0000 7f2903fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c510bdbaf0 0x55c510bde560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:15.688 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.685+0000 7f2903fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c510bdbaf0 0x55c510bde560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49540/0 (socket says 172.21.15.1:49540) 2024-09-11T22:02:15.688 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.685+0000 7f29088a6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c510bdc410 0x55c510be4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:15.688 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.685+0000 7f2903fff640 1 -- 172.21.15.1:0/2691535662 learned_addr learned my addr 172.21.15.1:0/2691535662 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.685+0000 7f2903fff640 1 --2- 172.21.15.1:0/2691535662 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c510be6cb0 0x55c510be4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.686+0000 7f29088a6640 1 -- 172.21.15.1:0/2691535662 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c510be6cb0 msgr2=0x55c510be4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.686+0000 7f29088a6640 1 --2- 172.21.15.1:0/2691535662 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c510be6cb0 0x55c510be4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.686+0000 7f29088a6640 1 -- 172.21.15.1:0/2691535662 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c510bdbaf0 msgr2=0x55c510bde560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.686+0000 7f29088a6640 1 --2- 172.21.15.1:0/2691535662 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c510bdbaf0 0x55c510bde560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.686+0000 7f29088a6640 1 -- 172.21.15.1:0/2691535662 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c510bd7e90 con 0x55c510bdc410 2024-09-11T22:02:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.686+0000 7f2903fff640 1 --2- 172.21.15.1:0/2691535662 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c510be6cb0 0x55c510be4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:02:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.686+0000 7f29088a6640 1 --2- 172.21.15.1:0/2691535662 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c510bdc410 0x55c510be4030 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f28f802a9a0 tx=0x7f28f80050c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54cb0ff16bdc8de2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.686+0000 7f28ee7fc640 1 -- 172.21.15.1:0/2691535662 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f8040040 con 0x55c510bdc410 2024-09-11T22:02:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.686+0000 7f28ee7fc640 1 -- 172.21.15.1:0/2691535662 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28f80315b0 con 0x55c510bdc410 2024-09-11T22:02:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.686+0000 7f28ee7fc640 1 -- 172.21.15.1:0/2691535662 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f8035040 con 0x55c510bdc410 2024-09-11T22:02:15.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.687+0000 7f290d89e740 1 -- 172.21.15.1:0/2691535662 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c510bdc410 msgr2=0x55c510be4030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:15.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.687+0000 7f290d89e740 1 --2- 172.21.15.1:0/2691535662 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c510bdc410 0x55c510be4030 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f28f802a9a0 tx=0x7f28f80050c0 comp rx=0 tx=0).stop 2024-09-11T22:02:15.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.687+0000 7f290d89e740 1 -- 172.21.15.1:0/2691535662 shutdown_connections 2024-09-11T22:02:15.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.687+0000 7f290d89e740 1 --2- 172.21.15.1:0/2691535662 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c510be6cb0 0x55c510be4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:15.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.687+0000 7f290d89e740 1 --2- 172.21.15.1:0/2691535662 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c510bdc410 0x55c510be4030 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:15.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.687+0000 7f290d89e740 1 --2- 172.21.15.1:0/2691535662 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c510bdbaf0 0x55c510bde560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:15.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.687+0000 7f290d89e740 1 -- 172.21.15.1:0/2691535662 >> 172.21.15.1:0/2691535662 conn(0x55c510bd8060 msgr2=0x55c510be5b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:15.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.687+0000 7f290d89e740 1 -- 172.21.15.1:0/2691535662 shutdown_connections 2024-09-11T22:02:15.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.687+0000 7f290d89e740 1 -- 172.21.15.1:0/2691535662 wait complete. 2024-09-11T22:02:15.694 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.687+0000 7f290d89e740 1 -- start start 2024-09-11T22:02:15.694 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.687+0000 7f290d89e740 1 Processor -- start 2024-09-11T22:02:15.694 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.688+0000 7f290d89e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c510bdbaf0 0x55c510beefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:15.694 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.688+0000 7f290d89e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c510bdc410 0x55c510bef500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:15.694 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.688+0000 7f290d89e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c510be6cb0 0x55c510bea390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:15.694 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.688+0000 7f290d89e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f29041b10d0 con 0x55c510bdbaf0 2024-09-11T22:02:15.694 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.688+0000 7f290d89e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f29041b09d0 con 0x55c510bdc410 2024-09-11T22:02:15.694 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.688+0000 7f290d89e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f29041adc10 con 0x55c510be6cb0 2024-09-11T22:02:15.694 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.688+0000 7f290d89e740 10 client.0 _ll_register_callbacks cb 0x7f28f0011ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:02:15.694 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.688+0000 7f2903fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c510bdbaf0 0x55c510beefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:15.694 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.688+0000 7f2903fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c510bdbaf0 0x55c510beefc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49546/0 (socket says 172.21.15.1:49546) 2024-09-11T22:02:15.694 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.688+0000 7f2903fff640 1 -- 172.21.15.1:0/1660592216 learned_addr learned my addr 172.21.15.1:0/1660592216 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:15.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.688+0000 7f29088a6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c510bdc410 0x55c510bef500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:15.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.688+0000 7f2903fff640 1 --2- 172.21.15.1:0/1660592216 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c510be6cb0 0x55c510bea390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:15.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.688+0000 7f2903fff640 1 -- 172.21.15.1:0/1660592216 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c510be6cb0 msgr2=0x55c510bea390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:15.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.688+0000 7f2903fff640 1 --2- 172.21.15.1:0/1660592216 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c510be6cb0 0x55c510bea390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:15.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.688+0000 7f2903fff640 1 -- 172.21.15.1:0/1660592216 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c510bdc410 msgr2=0x55c510bef500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:15.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.688+0000 7f2903fff640 1 --2- 172.21.15.1:0/1660592216 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c510bdc410 0x55c510bef500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:15.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.688+0000 7f2903fff640 1 -- 172.21.15.1:0/1660592216 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c510bd32d0 con 0x55c510bdbaf0 2024-09-11T22:02:15.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.688+0000 7f29088a6640 1 --2- 172.21.15.1:0/1660592216 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c510bdc410 0x55c510bef500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:02:15.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.689+0000 7f2903fff640 1 --2- 172.21.15.1:0/1660592216 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c510bdbaf0 0x55c510beefc0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f28f0021d20 tx=0x7f28f0009c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d90a85b39f793127 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:15.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.689+0000 7f28e3fff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:02:15.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.689+0000 7f28e3fff640 1 -- 172.21.15.1:0/1660592216 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f0021440 con 0x55c510bdbaf0 2024-09-11T22:02:15.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.689+0000 7f28e3fff640 1 -- 172.21.15.1:0/1660592216 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f28f00215e0 con 0x55c510bdbaf0 2024-09-11T22:02:15.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.689+0000 7f290d89e740 1 -- 172.21.15.1:0/1660592216 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28f80315b0 con 0x55c510bdbaf0 2024-09-11T22:02:15.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.689+0000 7f28e3fff640 1 -- 172.21.15.1:0/1660592216 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f28f0021840 con 0x55c510bdbaf0 2024-09-11T22:02:15.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.689+0000 7f290d89e740 10 client.16482 fetch_fsmap learned FSMap version 33 2024-09-11T22:02:15.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.689+0000 7f290d89e740 1 -- 172.21.15.1:0/1660592216 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c510c61350 con 0x55c510bdbaf0 2024-09-11T22:02:15.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.689+0000 7f28e3fff640 1 -- 172.21.15.1:0/1660592216 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28f80315b0 con 0x55c510bdbaf0 2024-09-11T22:02:15.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.689+0000 7f28e3fff640 1 -- 172.21.15.1:0/1660592216 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7f28f0037420 con 0x55c510bdbaf0 2024-09-11T22:02:15.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.690+0000 7f290d89e740 10 client.16482 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T22:02:15.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.690+0000 7f290d89e740 10 client.16482 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:02:15.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.690+0000 7f290d89e740 20 client.16482 populate_metadata read hostname 'smithi001' 2024-09-11T22:02:15.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.690+0000 7f290d89e740 1 --2- 172.21.15.1:0/1660592216 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55c510bf6c20 0x55c510c63af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:15.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.690+0000 7f290d89e740 4 client.16482 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:02:15.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.690+0000 7f290d89e740 1 -- 172.21.15.1:0/1660592216 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c510b47d70 con 0x55c510bf6c20 2024-09-11T22:02:15.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.690+0000 7f29088a6640 1 --2- 172.21.15.1:0/1660592216 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55c510bf6c20 0x55c510c63af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:15.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.690+0000 7f29088a6640 1 --2- 172.21.15.1:0/1660592216 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55c510bf6c20 0x55c510c63af0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be7fc726f3db56de server_cookie=b5acd996f751f231 in_seq=0 out_seq=0 2024-09-11T22:02:15.697 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:15.690+0000 7f28e3fff640 10 client.16482 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:02:16.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:16 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:16.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:16 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:16.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:16 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:16.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:16 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:16.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:16 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:16.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:16 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:16.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:16 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:16.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:16 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:16.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:16 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:16.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:16 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:16.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:16 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:16.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:16 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:16.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:16 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:16.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:16 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:16.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:16 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:16.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:16 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:16.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:16 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:16.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:16 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:16.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:16 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:16.300 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:16 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:16.300 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:16 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:16.300 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:16 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:16.300 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:16 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:16.300 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:16 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:16.300 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:16 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:16.300 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:16 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:16.300 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:16 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:16.300 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:16 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:16.300 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:16 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:16.300 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:16 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:16.300 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:16 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:16.300 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:16 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:16.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:16 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:16.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:16 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:16.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:16 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:16.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:16 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:16.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:16 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:16.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:16 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:16.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:16 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:16.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:16 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:16.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:16 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:16.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:16 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:16.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:16 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:16.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:16 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:16.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:16 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:16.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:16 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:16.344 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:16 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:16.344 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:16 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:16.344 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:16 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:16.344 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:16 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:16.344 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:16 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:16.344 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:16 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:16.344 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:16 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:16.344 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:16 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:16.344 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:16 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:16.344 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:16 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:16.345 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:16 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:16.345 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:16 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:16.345 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:16 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:16.345 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:16 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:16.345 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:16 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:16.345 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:16 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:16.345 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:16 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:16.345 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:16 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:16.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:16 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:16.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:16 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:16.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:16 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:16.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:16 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:16.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:16 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:16.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:16 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:16.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:16 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:16.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:16 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:16.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:16 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:16.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:16 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:16.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:16 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:16.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:16 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:16.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:16 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:16.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:16 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:16.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:16 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:16.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:16 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:16.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:16 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:16.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:16 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:16.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:16 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:16.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:16 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:16.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:16 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:16.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:16 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:16.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:16 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:16.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:16 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:16.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:16 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:16.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:16 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:16.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:16 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:16.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:16 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:16.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:16 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:16.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:16 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:16.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:16 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:16.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:16 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:16.881 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:16.880+0000 7f28e3fff640 1 -- 172.21.15.1:0/1660592216 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c510b47d70 con 0x55c510bf6c20 2024-09-11T22:02:16.881 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:16.880+0000 7f28e3fff640 10 client.16482 handle_command_reply: tid=0 2024-09-11T22:02:16.882 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:16.880+0000 7f290d89e740 1 client.16482 shutdown 2024-09-11T22:02:16.882 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:16.880+0000 7f290d89e740 1 -- 172.21.15.1:0/1660592216 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c510bdbaf0 msgr2=0x55c510beefc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:16.882 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:16.880+0000 7f290d89e740 1 --2- 172.21.15.1:0/1660592216 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c510bdbaf0 0x55c510beefc0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f28f0021d20 tx=0x7f28f0009c50 comp rx=0 tx=0).stop 2024-09-11T22:02:16.882 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:16.880+0000 7f290d89e740 1 -- 172.21.15.1:0/1660592216 shutdown_connections 2024-09-11T22:02:16.882 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:16.880+0000 7f290d89e740 1 --2- 172.21.15.1:0/1660592216 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55c510bf6c20 0x55c510c63af0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:16.882 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:16.880+0000 7f290d89e740 1 --2- 172.21.15.1:0/1660592216 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c510be6cb0 0x55c510bea390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:16.882 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:16.880+0000 7f290d89e740 1 --2- 172.21.15.1:0/1660592216 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c510bdc410 0x55c510bef500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:16.882 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:16.880+0000 7f290d89e740 1 --2- 172.21.15.1:0/1660592216 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c510bdbaf0 0x55c510beefc0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:16.882 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:16.880+0000 7f290d89e740 1 -- 172.21.15.1:0/1660592216 >> 172.21.15.1:0/1660592216 conn(0x55c510bd8060 msgr2=0x55c510bd69a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:16.882 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:16.880+0000 7f28e3fff640 0 client.16482 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:02:16.882 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:16.881+0000 7f290d89e740 1 -- 172.21.15.1:0/1660592216 shutdown_connections 2024-09-11T22:02:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:16.881+0000 7f290d89e740 1 -- 172.21.15.1:0/1660592216 wait complete. 2024-09-11T22:02:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:16.881+0000 7f290d89e740 20 client.16482 trim_cache size 0 max 16384 2024-09-11T22:02:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-09-11T22:02:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-09-11T22:02:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-09-11T22:02:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-09-11T22:02:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:16.882+0000 7f290ab0a640 1 -- 172.21.15.1:0/1001761841 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f28f40a2d70 msgr2=0x7f28f40a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:16.882+0000 7f290ab0a640 1 --2- 172.21.15.1:0/1001761841 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f28f40a2d70 0x7f28f40a5230 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f28fc0cd190 tx=0x7f28fc0bc040 comp rx=0 tx=0).stop 2024-09-11T22:02:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:16.882+0000 7f290ab0a640 1 -- 172.21.15.1:0/1001761841 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f290410c0a0 msgr2=0x7f2904101430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:16.882+0000 7f290ab0a640 1 --2- 172.21.15.1:0/1001761841 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f290410c0a0 0x7f2904101430 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f28fc00eb10 tx=0x7f28fc00c2d0 comp rx=0 tx=0).stop 2024-09-11T22:02:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:16.882+0000 7f290ab0a640 1 -- 172.21.15.1:0/1001761841 shutdown_connections 2024-09-11T22:02:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:16.882+0000 7f290ab0a640 1 --2- 172.21.15.1:0/1001761841 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f28f40a2d70 0x7f28f40a5230 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:16.882+0000 7f290ab0a640 1 --2- 172.21.15.1:0/1001761841 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f290410c0a0 0x7f2904101430 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:16.882+0000 7f290ab0a640 1 --2- 172.21.15.1:0/1001761841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2904103680 0x7f2904100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:16.882+0000 7f290ab0a640 1 --2- 172.21.15.1:0/1001761841 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2904102d60 0x7f29041009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:16.882+0000 7f290ab0a640 1 -- 172.21.15.1:0/1001761841 >> 172.21.15.1:0/1001761841 conn(0x7f29040779a0 msgr2=0x7f29040fd170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:16.882+0000 7f290ab0a640 1 -- 172.21.15.1:0/1001761841 shutdown_connections 2024-09-11T22:02:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:16.882+0000 7f290ab0a640 1 -- 172.21.15.1:0/1001761841 wait complete. 2024-09-11T22:02:17.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:17 smithi161 ceph-mon[35615]: pgmap v156: 97 pgs: 97 active+clean; 106 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 9.1 MiB/s wr, 1.95k op/s 2024-09-11T22:02:17.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:17 smithi001 ceph-mon[30935]: pgmap v156: 97 pgs: 97 active+clean; 106 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 9.1 MiB/s wr, 1.95k op/s 2024-09-11T22:02:17.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:17 smithi053 ceph-mon[34908]: pgmap v156: 97 pgs: 97 active+clean; 106 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 9.1 MiB/s wr, 1.95k op/s 2024-09-11T22:02:18.344 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:18.344 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.344 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:18.344 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.344 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:18.344 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.344 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:18.344 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.344 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:18.344 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.345 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:18.345 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.345 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:18.345 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.345 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:18.345 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.345 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:18.345 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.345 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:18.345 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.345 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:18.346 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.346 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:18.346 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.346 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:18.346 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.346 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:18.346 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.346 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:18.346 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.346 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:18.346 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.346 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:18.346 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.347 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:18.347 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.347 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:18.347 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.347 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:18.347 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.347 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:18.347 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.347 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:18.347 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.347 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:18.347 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.348 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:18.348 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.348 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:18.348 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.348 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:18.348 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.348 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:18.348 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.348 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:18.348 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.348 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:18.349 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.349 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:18.349 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.349 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:18.349 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.349 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:18.349 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.349 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:18.349 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.349 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:18.349 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.349 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:18.349 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.350 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:18.350 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.350 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:18.350 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.350 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:18.350 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.350 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:18.350 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.350 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:18.350 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.350 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:18.350 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.350 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:18.351 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.351 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:18.351 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.351 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:18.351 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.351 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:18.351 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.351 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:18.351 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.351 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:18.351 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.351 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:18.351 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.352 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:18.352 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.352 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:18.352 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.352 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:18.352 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.352 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:18.352 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.352 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:18.353 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.353 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:18.353 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.353 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:18.353 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.353 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:18.353 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.353 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:18.353 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.353 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:18.353 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.353 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:18.354 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.354 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:18.354 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.354 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:18.354 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.354 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:18.354 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.354 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:18.354 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:18 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:18.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:18.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:18.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:18.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:18.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:18.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:18.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:18.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:18.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:18.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:18.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:18.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:18.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:18.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:18.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:18.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:18.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:18.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:18.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:18.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:18.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:18.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:18.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:18.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.448 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:18.448 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.448 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:18.448 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.448 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:18.448 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.449 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:18.449 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.449 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:18.449 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.449 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:18.449 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.449 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:18.449 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.449 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:18.449 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.449 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:18.449 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.449 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:18.450 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.450 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:18.450 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.450 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:18.450 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.450 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:18.450 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.450 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:18.450 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.450 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:18.450 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.450 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:18.450 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:18.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:18.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:18.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:18.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:18.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:18.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:18.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:18.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:18.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:18.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:18.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:18.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:18.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:18.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:18.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:18.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:18.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:18.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:18.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:18.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:18.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:18.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:18.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:18 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:18.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:18.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:18.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:18.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:18.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:18.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:18.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:18.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.552 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:18.552 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.552 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:18.552 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.552 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:18.552 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.552 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:18.552 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.552 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:18.552 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.552 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:18.552 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.552 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:18.553 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.553 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:18.553 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.553 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:18.553 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.553 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:18.553 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.553 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:18.553 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.553 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:18.553 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.553 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:18.554 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.554 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:18.554 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.554 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:18.554 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.554 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:18.554 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.554 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:18.554 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.554 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:18.554 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.554 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:18.554 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.554 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:18.555 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.555 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:18.555 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.555 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:18.555 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.555 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:18.555 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.555 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:18.555 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.555 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:18.555 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.555 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:18.555 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.556 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:18.556 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.556 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:18.556 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.556 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:18.556 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.556 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:18.556 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.556 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:18.556 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.556 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:18.556 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.556 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:18.556 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.557 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:18.557 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.557 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:18.557 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.557 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:18.557 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.557 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:18.557 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.557 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:18.557 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.557 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:18.557 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.557 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:18.557 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.558 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:18.558 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.558 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:18.558 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.558 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:18.558 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.558 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:18.558 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.558 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:18.558 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.558 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:18.558 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.558 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:18.559 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.559 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:18.559 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.559 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:18.559 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.559 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:18.559 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.559 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:18.559 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.559 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:18.559 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.559 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:18.559 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.560 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:18.560 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:18.560 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:18.560 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:18 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:19.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:19 smithi161 ceph-mon[35615]: pgmap v157: 97 pgs: 97 active+clean; 106 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.4 MiB/s wr, 888 op/s 2024-09-11T22:02:19.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:19 smithi001 ceph-mon[30935]: pgmap v157: 97 pgs: 97 active+clean; 106 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.4 MiB/s wr, 888 op/s 2024-09-11T22:02:19.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:19 smithi053 ceph-mon[34908]: pgmap v157: 97 pgs: 97 active+clean; 106 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.4 MiB/s wr, 888 op/s 2024-09-11T22:02:20.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:20 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:20.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:20 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:20.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:20 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:20.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:20 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:20.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:20 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:20.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:20 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:20.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:20 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:20.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:20 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:20.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:20 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:20.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:20 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:20.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:20 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:20.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:20 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:20.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:20 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:20.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:20 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:20.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:20 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:20.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:20 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:20.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:20 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:20.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:20 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:21.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:21 smithi161 ceph-mon[35615]: pgmap v158: 97 pgs: 97 active+clean; 88 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.5 MiB/s wr, 1.86k op/s 2024-09-11T22:02:21.343 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:21 smithi001 ceph-mon[30935]: pgmap v158: 97 pgs: 97 active+clean; 88 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.5 MiB/s wr, 1.86k op/s 2024-09-11T22:02:21.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:21 smithi053 ceph-mon[34908]: pgmap v158: 97 pgs: 97 active+clean; 88 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.5 MiB/s wr, 1.86k op/s 2024-09-11T22:02:23.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:23 smithi001 ceph-mon[30935]: pgmap v159: 97 pgs: 97 active+clean; 88 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.7 MiB/s wr, 1.68k op/s 2024-09-11T22:02:23.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:23 smithi053 ceph-mon[34908]: pgmap v159: 97 pgs: 97 active+clean; 88 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.7 MiB/s wr, 1.68k op/s 2024-09-11T22:02:23.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:23 smithi161 ceph-mon[35615]: pgmap v159: 97 pgs: 97 active+clean; 88 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.7 MiB/s wr, 1.68k op/s 2024-09-11T22:02:24.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:24 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:24.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:24 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:24.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:24 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:24.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:24 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:24.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:24 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:24.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:24 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:24.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:24 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:24.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:24 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:24.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:24 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:24.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:24 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:24.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:24 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:24.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:24 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:24.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:24 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:24.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:24 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:24.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:24 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:24.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:24 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:24.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:24 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:24.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:24 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:24.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:24 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:24.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:24 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:24.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:24 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:24.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:24 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:24.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:24 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:24.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:24 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:24.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:24 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:24.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:24 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:24.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:24 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:24.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:24 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:24.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:24 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:24.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:24 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:24.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:24 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:24.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:24 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:24.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:24 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:24.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:24 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:24.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:24 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:24.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:24 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:24.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:24 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:24.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:24 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:24.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:24 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:24.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:24 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:24.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:24 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:24.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:24 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:24.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:24 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:24.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:24 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:24.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:24 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:24.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:24 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:24.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:24 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:24.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:24 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:24.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:24 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:24.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:24 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:24.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:24 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:24.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:24 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:24.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:24 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:24.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:24 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:24.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:24 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:24.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:24 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:24.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:24 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:24.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:24 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:24.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:24 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:24.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:24 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:24.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:24 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:24.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:24 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:24.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:24 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:24.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:24 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:24.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:24 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:24.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:24 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:24.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:24 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:24.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:24 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:24.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:24 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:24.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:24 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:24.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:24 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:24.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:24 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:24.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:24 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:24.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:24 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:24.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:24 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:24.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:24 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:24.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:24 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:24.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:24 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:24.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:24 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:24.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:24 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:24.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:24 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:24.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:24 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:24.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:24 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:24.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:24 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:24.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:24 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:24.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:24 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:24.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:24 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:24.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:24 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:24.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:24 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:24.594 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:24 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:24.594 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:24 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:24.594 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:24 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:24.594 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:24 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:24.594 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:24 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:24.594 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:24 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:24.594 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:24 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:24.594 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:24 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:24.594 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:24 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:24.594 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:24 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:24.594 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:24 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:24.594 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:24 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:24.595 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:24 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:24.595 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:24 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:24.595 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:24 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:24.595 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:24 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:24.595 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:24 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:24.595 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:24 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:24.595 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:24 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:24.595 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:24 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:24.595 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:24 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:24.595 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:24 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:24.595 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:24 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:24.595 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:24 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:24.595 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:24 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:25.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:25 smithi001 ceph-mon[30935]: pgmap v160: 97 pgs: 97 active+clean; 88 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.7 MiB/s wr, 1.68k op/s 2024-09-11T22:02:25.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:25 smithi053 ceph-mon[34908]: pgmap v160: 97 pgs: 97 active+clean; 88 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.7 MiB/s wr, 1.68k op/s 2024-09-11T22:02:25.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:25 smithi161 ceph-mon[35615]: pgmap v160: 97 pgs: 97 active+clean; 88 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.7 MiB/s wr, 1.68k op/s 2024-09-11T22:02:27.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:27 smithi001 ceph-mon[30935]: pgmap v161: 97 pgs: 97 active+clean; 109 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 6.9 MiB/s wr, 2.68k op/s 2024-09-11T22:02:27.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:27 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:27.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:27 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:27.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:27 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:27.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:27 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:27.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:27 smithi053 ceph-mon[34908]: pgmap v161: 97 pgs: 97 active+clean; 109 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 6.9 MiB/s wr, 2.68k op/s 2024-09-11T22:02:27.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:27 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:27.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:27 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:27.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:27 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:27.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:27 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:27.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:27 smithi161 ceph-mon[35615]: pgmap v161: 97 pgs: 97 active+clean; 109 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 6.9 MiB/s wr, 2.68k op/s 2024-09-11T22:02:27.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:27 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:27.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:27 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:27.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:27 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:27.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:27 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:28.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:28 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:28.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:28 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:28.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:28 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:28.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:28 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:28.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:28 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:28.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:28 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:28.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:28 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:28.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:28 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:28.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:28 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:28.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:28 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:28.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:28 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:28.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:28 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:28.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:28 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:28.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:28 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:28.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:28 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:28.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:28 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:28.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:28 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:28.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:28 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:28.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:28 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:28.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:28 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:28.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:28 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:28.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:28 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:28.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:28 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:28.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:28 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:28.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:28 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:28.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:28 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:28.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:28 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:28.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:28 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:28.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:28 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:28.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:28 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:28.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:28 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:28.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:28 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:28.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:28 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:28.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:28 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:28.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:28 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:28.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:28 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:28.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:28 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:28.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:28 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:28.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:28 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:28.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:28 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:28.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:28 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:28.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:28 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:28.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:28 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:28.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:28 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:28.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:28 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:28.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:28 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:28.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:28 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:28.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:28 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:28.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:28 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:28.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:28 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:28.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:28 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:28.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:28 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:28.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:28 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:28.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:28 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:28.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:28 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:28.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:28 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:28.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:28 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:28.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:28 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:28.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:28 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:28.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:28 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:28.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:28 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:28.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:28 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:28.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:28 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:28.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:28 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:28.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:28 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:28.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:28 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:28.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:28 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:28.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:28 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:28.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:28 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:28.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:28 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:28.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:28 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:28.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:28 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:28.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:28 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:28.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:28 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:28.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:28 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:28.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:28 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:28.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:28 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:28.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:28 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:28.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:28 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:28.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:28 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:28.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:28 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:28.594 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:28 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:28.594 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:28 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:28.594 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:28 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:28.594 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:28 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:28.594 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:28 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:28.594 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:28 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:28.594 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:28 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:28.594 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:28 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:28.594 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:28 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:28.594 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:28 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:28.594 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:28 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:28.594 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:28 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:28.594 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:28 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:28.594 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:28 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:28.595 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:28 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:28.595 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:28 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:28.595 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:28 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:28.595 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:28 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:29.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:29.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:29.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:29.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:29.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:29.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:29.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:29.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: pgmap v162: 97 pgs: 97 active+clean; 109 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.3 MiB/s wr, 1.97k op/s 2024-09-11T22:02:29.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:29.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:29.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:29.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:29.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:29.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:29.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:29.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:29.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:29.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:29.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:29.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:29.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:29.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:29.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.448 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:29.448 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.448 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:29.448 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.448 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:29.448 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.448 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:29.448 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.448 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:29.449 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.449 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:29.449 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.449 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:29.449 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.449 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:29.449 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.449 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:29.449 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.449 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:29.449 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.449 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.449 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:29.450 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.450 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:29.450 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.450 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:29.450 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.450 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:29.450 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.450 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:29.450 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.450 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:29.450 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.450 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:29.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:29.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:29.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:29.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:29.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:29.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:29.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:29.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:29.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:29.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:29 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:29.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:29.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:29.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:29.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:29.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:29.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:29.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:29.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: pgmap v162: 97 pgs: 97 active+clean; 109 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.3 MiB/s wr, 1.97k op/s 2024-09-11T22:02:29.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:29.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:29.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:29.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:29.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:29.552 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.552 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:29.552 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.552 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:29.552 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.552 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:29.552 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.552 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:29.552 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.552 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:29.552 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.552 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:29.553 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.553 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:29.553 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.553 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:29.553 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.553 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:29.553 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.553 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:29.553 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.553 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:29.553 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.553 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:29.554 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.554 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:29.554 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.554 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:29.554 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.554 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:29.554 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.554 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:29.554 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.554 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:29.554 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.555 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:29.555 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.555 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:29.555 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.555 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.555 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:29.555 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.555 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:29.555 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.555 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:29.555 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.555 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:29.555 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.556 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:29.556 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.556 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:29.556 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.556 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:29.556 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.556 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:29.556 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.556 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:29.556 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.556 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:29.556 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.556 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:29.556 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.557 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:29.557 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.557 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:29.557 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.557 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:29.557 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.557 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:29.557 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.557 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:29.557 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:29 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:29.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:29.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:29.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:29.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:29.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:29.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:29.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:29.594 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.594 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: pgmap v162: 97 pgs: 97 active+clean; 109 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.3 MiB/s wr, 1.97k op/s 2024-09-11T22:02:29.594 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:29.594 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:29.594 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.594 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:29.594 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.594 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:29.594 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.594 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:29.594 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.594 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:29.594 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.594 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:29.595 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.595 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:29.595 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.595 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:29.595 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.595 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:29.595 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.595 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:29.595 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.595 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:29.595 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.595 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:29.595 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.596 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:29.596 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.596 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:29.596 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.596 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:29.596 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.596 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:29.596 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.596 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:29.597 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.597 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:29.597 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.597 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:29.597 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.597 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:29.597 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.597 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:29.597 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.597 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:29.597 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.597 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:29.597 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.598 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.598 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:29.598 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.598 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:29.598 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.598 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:29.598 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.598 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:29.598 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.598 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:29.598 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.598 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:29.598 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.599 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:29.599 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.599 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:29.599 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.599 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:29.599 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.599 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:29.599 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.599 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:29.599 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.599 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:29.599 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.599 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:29.600 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.600 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:29.600 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.600 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:29.600 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.600 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:29.600 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:29 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:29.993 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:29.992+0000 7f1784a4c640 1 -- 172.21.15.1:0/976169428 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17801009a0 msgr2=0x7f1780100e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:29.993 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:29.992+0000 7f1784a4c640 1 --2- 172.21.15.1:0/976169428 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17801009a0 0x7f1780100e00 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f1774009930 tx=0x7f177402a420 comp rx=0 tx=0).stop 2024-09-11T22:02:29.994 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:29.992+0000 7f1784a4c640 1 -- 172.21.15.1:0/976169428 shutdown_connections 2024-09-11T22:02:29.994 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:29.992+0000 7f1784a4c640 1 --2- 172.21.15.1:0/976169428 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1780101340 0x7f178010d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:29.994 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:29.992+0000 7f1784a4c640 1 --2- 172.21.15.1:0/976169428 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17801009a0 0x7f1780100e00 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:29.994 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:29.992+0000 7f1784a4c640 1 --2- 172.21.15.1:0/976169428 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17801069b0 0x7f1780106d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:29.994 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:29.992+0000 7f1784a4c640 1 -- 172.21.15.1:0/976169428 >> 172.21.15.1:0/976169428 conn(0x7f17800779a0 msgr2=0x7f17800fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:29.994 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:29.993+0000 7f1784a4c640 1 -- 172.21.15.1:0/976169428 shutdown_connections 2024-09-11T22:02:29.994 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:29.993+0000 7f1784a4c640 1 -- 172.21.15.1:0/976169428 wait complete. 2024-09-11T22:02:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:29.994+0000 7f1784a4c640 1 Processor -- start 2024-09-11T22:02:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:29.994+0000 7f1784a4c640 1 -- start start 2024-09-11T22:02:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:29.994+0000 7f1784a4c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17801009a0 0x7f1780199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:29.994+0000 7f1784a4c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1780101340 0x7f17801997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:29.994+0000 7f1784a4c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f17801069b0 0x7f17801a42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:29.997 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:29.994+0000 7f1784a4c640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1780110df0 con 0x7f1780101340 2024-09-11T22:02:29.997 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:29.994+0000 7f1784a4c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1780110c70 con 0x7f17801009a0 2024-09-11T22:02:29.997 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:29.994+0000 7f1784a4c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1780110f70 con 0x7f17801069b0 2024-09-11T22:02:29.997 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:29.994+0000 7f177dd9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1780101340 0x7f17801997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:29.997 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:29.994+0000 7f177ed9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f17801069b0 0x7f17801a42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:29.997 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:29.994+0000 7f177e59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17801009a0 0x7f1780199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:29.997 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:29.994+0000 7f177e59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17801009a0 0x7f1780199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:60590/0 (socket says 172.21.15.1:60590) 2024-09-11T22:02:29.997 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:29.994+0000 7f177e59c640 1 -- 172.21.15.1:0/3494491397 learned_addr learned my addr 172.21.15.1:0/3494491397 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:29.997 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:29.995+0000 7f177e59c640 1 -- 172.21.15.1:0/3494491397 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f17801069b0 msgr2=0x7f17801a42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:29.997 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:29.995+0000 7f177e59c640 1 --2- 172.21.15.1:0/3494491397 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f17801069b0 0x7f17801a42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:29.997 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:29.995+0000 7f177e59c640 1 -- 172.21.15.1:0/3494491397 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1780101340 msgr2=0x7f17801997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:29.997 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:29.995+0000 7f177e59c640 1 --2- 172.21.15.1:0/3494491397 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1780101340 0x7f17801997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:29.998 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:29.995+0000 7f177e59c640 1 -- 172.21.15.1:0/3494491397 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17801a4820 con 0x7f17801009a0 2024-09-11T22:02:29.998 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:29.995+0000 7f177e59c640 1 --2- 172.21.15.1:0/3494491397 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17801009a0 0x7f1780199260 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f177000ea60 tx=0x7f177000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:29.998 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:29.995+0000 7f176f7fe640 1 -- 172.21.15.1:0/3494491397 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f177000cde0 con 0x7f17801009a0 2024-09-11T22:02:29.998 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:29.995+0000 7f1784a4c640 1 -- 172.21.15.1:0/3494491397 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17801a49f0 con 0x7f17801009a0 2024-09-11T22:02:29.998 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:29.995+0000 7f176f7fe640 1 -- 172.21.15.1:0/3494491397 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17700035f0 con 0x7f17801009a0 2024-09-11T22:02:29.998 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:29.995+0000 7f176f7fe640 1 -- 172.21.15.1:0/3494491397 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1770010670 con 0x7f17801009a0 2024-09-11T22:02:29.998 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:29.995+0000 7f1784a4c640 1 -- 172.21.15.1:0/3494491397 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17801a4f60 con 0x7f17801009a0 2024-09-11T22:02:30.000 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:29.998+0000 7f176f7fe640 1 -- 172.21.15.1:0/3494491397 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f17700037a0 con 0x7f17801009a0 2024-09-11T22:02:30.000 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:29.998+0000 7f17877e6740 1 Processor -- start 2024-09-11T22:02:30.000 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:29.998+0000 7f17877e6740 1 -- start start 2024-09-11T22:02:30.000 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:29.998+0000 7f17877e6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c9fda129d0 0x55c9fda12db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:30.000 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:29.998+0000 7f17877e6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9fda132f0 0x55c9fda21b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:30.000 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:29.998+0000 7f17877e6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9fda220c0 0x55c9fda244b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:30.000 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:29.998+0000 7f17877e6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55c9fd9772f0 con 0x55c9fda129d0 2024-09-11T22:02:30.000 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:29.998+0000 7f17877e6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55c9fd91f270 con 0x55c9fda220c0 2024-09-11T22:02:30.000 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:29.998+0000 7f17877e6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c9fd9732b0 con 0x55c9fda132f0 2024-09-11T22:02:30.000 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:29.999+0000 7f177dd9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9fda220c0 0x55c9fda244b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:30.000 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:29.999+0000 7f177dd9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9fda220c0 0x55c9fda244b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:60606/0 (socket says 172.21.15.1:60606) 2024-09-11T22:02:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:29.999+0000 7f177dd9b640 1 -- 172.21.15.1:0/1845852594 learned_addr learned my addr 172.21.15.1:0/1845852594 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:29.999+0000 7f177ed9d640 1 --2- 172.21.15.1:0/1845852594 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c9fda129d0 0x55c9fda12db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:29.999+0000 7f177e59c640 1 --2- 172.21.15.1:0/1845852594 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9fda132f0 0x55c9fda21b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:29.999+0000 7f176f7fe640 1 --2- 172.21.15.1:0/3494491397 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f17680a2cf0 0x7f17680a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:29.999+0000 7f177dd9b640 1 -- 172.21.15.1:0/1845852594 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9fda132f0 msgr2=0x55c9fda21b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:29.999+0000 7f177dd9b640 1 --2- 172.21.15.1:0/1845852594 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9fda132f0 0x55c9fda21b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:29.999+0000 7f177dd9b640 1 -- 172.21.15.1:0/1845852594 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c9fda129d0 msgr2=0x55c9fda12db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:29.999+0000 7f177dd9b640 1 --2- 172.21.15.1:0/1845852594 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c9fda129d0 0x55c9fda12db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:29.999+0000 7f177dd9b640 1 -- 172.21.15.1:0/1845852594 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9fd91e340 con 0x55c9fda220c0 2024-09-11T22:02:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:29.999+0000 7f176f7fe640 1 -- 172.21.15.1:0/3494491397 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f17700cf700 con 0x7f17801009a0 2024-09-11T22:02:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:29.999+0000 7f177ed9d640 1 --2- 172.21.15.1:0/3494491397 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f17680a2cf0 0x7f17680a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:29.999+0000 7f177dd9b640 1 --2- 172.21.15.1:0/1845852594 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9fda220c0 0x55c9fda244b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1774009930 tx=0x7f1774005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ff1c305f2b0e157 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:30.005 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.000+0000 7f176cff9640 1 -- 172.21.15.1:0/1845852594 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1774040070 con 0x55c9fda220c0 2024-09-11T22:02:30.005 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.000+0000 7f176cff9640 1 -- 172.21.15.1:0/1845852594 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17740318f0 con 0x55c9fda220c0 2024-09-11T22:02:30.005 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.000+0000 7f177ed9d640 1 --2- 172.21.15.1:0/3494491397 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f17680a2cf0 0x7f17680a51b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1778005fd0 tx=0x7f1778004a40 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:30.005 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.000+0000 7f176cff9640 1 -- 172.21.15.1:0/1845852594 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1774035040 con 0x55c9fda220c0 2024-09-11T22:02:30.005 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.001+0000 7f17877e6740 1 -- 172.21.15.1:0/1845852594 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9fda220c0 msgr2=0x55c9fda244b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:30.005 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.001+0000 7f17877e6740 1 --2- 172.21.15.1:0/1845852594 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9fda220c0 0x55c9fda244b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1774009930 tx=0x7f1774005c00 comp rx=0 tx=0).stop 2024-09-11T22:02:30.005 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.001+0000 7f17877e6740 1 -- 172.21.15.1:0/1845852594 shutdown_connections 2024-09-11T22:02:30.005 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.001+0000 7f17877e6740 1 --2- 172.21.15.1:0/1845852594 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9fda220c0 0x55c9fda244b0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:30.005 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.001+0000 7f17877e6740 1 --2- 172.21.15.1:0/1845852594 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9fda132f0 0x55c9fda21b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:30.005 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.001+0000 7f17877e6740 1 --2- 172.21.15.1:0/1845852594 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c9fda129d0 0x55c9fda12db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:30.005 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.001+0000 7f17877e6740 1 -- 172.21.15.1:0/1845852594 >> 172.21.15.1:0/1845852594 conn(0x55c9fda0f060 msgr2=0x55c9fda0f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:30.005 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.001+0000 7f17877e6740 1 -- 172.21.15.1:0/1845852594 shutdown_connections 2024-09-11T22:02:30.006 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.001+0000 7f17877e6740 1 -- 172.21.15.1:0/1845852594 wait complete. 2024-09-11T22:02:30.006 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.001+0000 7f17877e6740 1 -- start start 2024-09-11T22:02:30.006 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.002+0000 7f17877e6740 1 Processor -- start 2024-09-11T22:02:30.006 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.002+0000 7f17877e6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c9fda129d0 0x55c9fda1f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:30.006 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.002+0000 7f17877e6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9fda132f0 0x55c9fda1f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:30.006 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.002+0000 7f17877e6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9fda220c0 0x55c9fda1fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:30.006 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.002+0000 7f17877e6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55c9fda24c90 con 0x55c9fda129d0 2024-09-11T22:02:30.006 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.002+0000 7f17877e6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55c9fd900290 con 0x55c9fda132f0 2024-09-11T22:02:30.006 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.002+0000 7f17877e6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c9fda24e10 con 0x55c9fda220c0 2024-09-11T22:02:30.006 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.002+0000 7f17877e6740 10 client.0 _ll_register_callbacks cb 0x55c9fd931140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:02:30.006 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.002+0000 7f177e59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c9fda129d0 0x55c9fda1f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:30.006 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.002+0000 7f177dd9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9fda132f0 0x55c9fda1f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:30.006 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.002+0000 7f177e59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c9fda129d0 0x55c9fda1f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33788/0 (socket says 172.21.15.1:33788) 2024-09-11T22:02:30.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.002+0000 7f177e59c640 1 -- 172.21.15.1:0/410252420 learned_addr learned my addr 172.21.15.1:0/410252420 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:30.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.002+0000 7f177ed9d640 1 --2- 172.21.15.1:0/410252420 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9fda220c0 0x55c9fda1fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:30.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.002+0000 7f177e59c640 1 -- 172.21.15.1:0/410252420 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9fda220c0 msgr2=0x55c9fda1fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:30.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.002+0000 7f177e59c640 1 --2- 172.21.15.1:0/410252420 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9fda220c0 0x55c9fda1fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:30.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.002+0000 7f177e59c640 1 -- 172.21.15.1:0/410252420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9fda132f0 msgr2=0x55c9fda1f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:30.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.002+0000 7f177e59c640 1 --2- 172.21.15.1:0/410252420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9fda132f0 0x55c9fda1f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:30.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.003+0000 7f177e59c640 1 -- 172.21.15.1:0/410252420 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9fd68a210 con 0x55c9fda129d0 2024-09-11T22:02:30.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.003+0000 7f177e59c640 1 --2- 172.21.15.1:0/410252420 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c9fda129d0 0x55c9fda1f060 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f177000c760 tx=0x7f1770097fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad755a453120260f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:30.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.003+0000 7f1765ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:02:30.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.003+0000 7f1765ffb640 1 -- 172.21.15.1:0/410252420 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f177009c2e0 con 0x55c9fda129d0 2024-09-11T22:02:30.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.003+0000 7f1765ffb640 1 -- 172.21.15.1:0/410252420 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f1770098250 con 0x55c9fda129d0 2024-09-11T22:02:30.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.003+0000 7f17877e6740 1 -- 172.21.15.1:0/410252420 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17740318f0 con 0x55c9fda129d0 2024-09-11T22:02:30.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.003+0000 7f1765ffb640 1 -- 172.21.15.1:0/410252420 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f17700b9050 con 0x55c9fda129d0 2024-09-11T22:02:30.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.003+0000 7f17877e6740 10 client.16494 fetch_fsmap learned FSMap version 33 2024-09-11T22:02:30.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.003+0000 7f17877e6740 1 -- 172.21.15.1:0/410252420 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c9fd756620 con 0x55c9fda129d0 2024-09-11T22:02:30.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.003+0000 7f1765ffb640 1 -- 172.21.15.1:0/410252420 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17740318f0 con 0x55c9fda129d0 2024-09-11T22:02:30.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.004+0000 7f1765ffb640 1 -- 172.21.15.1:0/410252420 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7f17700cfbb0 con 0x55c9fda129d0 2024-09-11T22:02:30.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.004+0000 7f17877e6740 10 client.16494 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T22:02:30.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.004+0000 7f17877e6740 10 client.16494 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:02:30.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.004+0000 7f17877e6740 20 client.16494 populate_metadata read hostname 'smithi001' 2024-09-11T22:02:30.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.004+0000 7f17877e6740 1 --2- 172.21.15.1:0/410252420 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55c9fda320d0 0x55c9fda34540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:30.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.004+0000 7f17877e6740 4 client.16494 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:02:30.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.004+0000 7f17877e6740 1 -- 172.21.15.1:0/410252420 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c9fd984a20 con 0x55c9fda320d0 2024-09-11T22:02:30.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.004+0000 7f177dd9b640 1 --2- 172.21.15.1:0/410252420 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55c9fda320d0 0x55c9fda34540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:30.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.004+0000 7f177dd9b640 1 --2- 172.21.15.1:0/410252420 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55c9fda320d0 0x55c9fda34540 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86cefaf762434638 server_cookie=98cbee0705c2cb4c in_seq=0 out_seq=0 2024-09-11T22:02:30.009 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.004+0000 7f1765ffb640 10 client.16494 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:02:30.010 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.008+0000 7f1765ffb640 1 -- 172.21.15.1:0/410252420 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c9fd984a20 con 0x55c9fda320d0 2024-09-11T22:02:30.010 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.008+0000 7f1765ffb640 10 client.16494 handle_command_reply: tid=0 2024-09-11T22:02:30.010 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.008+0000 7f17877e6740 1 client.16494 shutdown 2024-09-11T22:02:30.010 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.008+0000 7f17877e6740 1 -- 172.21.15.1:0/410252420 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c9fda129d0 msgr2=0x55c9fda1f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:30.010 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.008+0000 7f17877e6740 1 --2- 172.21.15.1:0/410252420 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c9fda129d0 0x55c9fda1f060 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f177000c760 tx=0x7f1770097fb0 comp rx=0 tx=0).stop 2024-09-11T22:02:30.010 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.008+0000 7f17877e6740 1 -- 172.21.15.1:0/410252420 shutdown_connections 2024-09-11T22:02:30.010 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.008+0000 7f17877e6740 1 --2- 172.21.15.1:0/410252420 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55c9fda320d0 0x55c9fda34540 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:30.010 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.008+0000 7f17877e6740 1 --2- 172.21.15.1:0/410252420 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9fda220c0 0x55c9fda1fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:30.010 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.008+0000 7f17877e6740 1 --2- 172.21.15.1:0/410252420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9fda132f0 0x55c9fda1f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:30.010 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.008+0000 7f17877e6740 1 --2- 172.21.15.1:0/410252420 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c9fda129d0 0x55c9fda1f060 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:30.010 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.008+0000 7f17877e6740 1 -- 172.21.15.1:0/410252420 >> 172.21.15.1:0/410252420 conn(0x55c9fda0f060 msgr2=0x55c9fda14d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:30.010 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.008+0000 7f1765ffb640 0 client.16494 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:02:30.010 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.008+0000 7f17877e6740 1 -- 172.21.15.1:0/410252420 shutdown_connections 2024-09-11T22:02:30.011 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.008+0000 7f17877e6740 1 -- 172.21.15.1:0/410252420 wait complete. 2024-09-11T22:02:30.011 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.008+0000 7f17877e6740 20 client.16494 trim_cache size 0 max 16384 2024-09-11T22:02:30.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:30.022+0000 7f6acbfff640 1 -- 172.21.15.1:0/1324922187 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7f6ae40d3040 con 0x55838f658830 2024-09-11T22:02:30.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:30.022+0000 7f6acbfff640 10 client.25969 handle_command_reply: tid=0 2024-09-11T22:02:30.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:30.022+0000 7f6af5024740 1 client.25969 shutdown 2024-09-11T22:02:30.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:30.022+0000 7f6af5024740 1 -- 172.21.15.1:0/1324922187 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55838f5d1850 msgr2=0x55838f5e4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:30.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:30.022+0000 7f6af5024740 1 --2- 172.21.15.1:0/1324922187 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55838f5d1850 0x55838f5e4e30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6ae001b040 tx=0x7f6ae0011dc0 comp rx=0 tx=0).stop 2024-09-11T22:02:30.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:30.023+0000 7f6af5024740 1 -- 172.21.15.1:0/1324922187 shutdown_connections 2024-09-11T22:02:30.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:30.023+0000 7f6af5024740 1 --2- 172.21.15.1:0/1324922187 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55838f658830 0x55838f65ac50 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:30.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:30.023+0000 7f6af5024740 1 --2- 172.21.15.1:0/1324922187 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55838f5dca30 0x55838f5e16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:30.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:30.023+0000 7f6af5024740 1 --2- 172.21.15.1:0/1324922187 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55838f5d2170 0x55838f5e1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:30.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:30.023+0000 7f6acbfff640 0 client.25969 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:02:30.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:30.023+0000 7f6af5024740 1 --2- 172.21.15.1:0/1324922187 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55838f5d1850 0x55838f5e4e30 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:30.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:30.023+0000 7f6af5024740 1 -- 172.21.15.1:0/1324922187 >> 172.21.15.1:0/1324922187 conn(0x55838f5cdde0 msgr2=0x55838f5d9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:30.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:30.023+0000 7f6af5024740 1 -- 172.21.15.1:0/1324922187 shutdown_connections 2024-09-11T22:02:30.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:30.023+0000 7f6af5024740 1 -- 172.21.15.1:0/1324922187 wait complete. 2024-09-11T22:02:30.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:30.023+0000 7f6af5024740 20 client.25969 trim_cache size 0 max 16384 2024-09-11T22:02:30.025 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-11T22:02:30.026 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 33, 2024-09-11T22:02:30.026 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24494, 2024-09-11T22:02:30.026 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 14, 2024-09-11T22:02:30.026 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-09-11T22:02:30.026 INFO:teuthology.orchestra.run.smithi001.stdout: "19475138": { 2024-09-11T22:02:30.026 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 14, 2024-09-11T22:02:30.026 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-09-11T22:02:30.026 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-11T22:02:30.026 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-09-11T22:02:30.026 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-11T22:02:30.026 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:02:30.026 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 90.0, 2024-09-11T22:02:30.027 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 140.2, 2024-09-11T22:02:30.027 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-09-11T22:02:30.027 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-09-11T22:02:30.027 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-09-11T22:02:30.027 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-11T22:02:30.027 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-09-11T22:02:30.027 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-11T22:02:30.027 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:02:30.027 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:02:30.027 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:02:30.027 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:02:30.027 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:02:30.027 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-11T22:02:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.023+0000 7f17877e6740 1 Processor -- start 2024-09-11T22:02:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.023+0000 7f17877e6740 1 -- start start 2024-09-11T22:02:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.024+0000 7f17877e6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9fda129d0 0x55c9fda1f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:30.029 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.024+0000 7f17877e6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9fda132f0 0x55c9fda39e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:30.029 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.024+0000 7f17877e6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c9fda220c0 0x55c9fda3a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:30.029 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.024+0000 7f17877e6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f17801ab7c0 con 0x55c9fda220c0 2024-09-11T22:02:30.029 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.024+0000 7f17877e6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f17801ad3f0 con 0x55c9fda132f0 2024-09-11T22:02:30.029 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.024+0000 7f17877e6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f17801abe90 con 0x55c9fda129d0 2024-09-11T22:02:30.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:30.024+0000 7f6af2c4b640 1 -- 172.21.15.1:0/2074501376 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f6adc0a2cf0 msgr2=0x7f6adc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:30.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:30.024+0000 7f6af2c4b640 1 --2- 172.21.15.1:0/2074501376 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f6adc0a2cf0 0x7f6adc0a51b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x55838f5cf160 tx=0x7f6ad80058d0 comp rx=0 tx=0).stop 2024-09-11T22:02:30.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:30.024+0000 7f6af2c4b640 1 -- 172.21.15.1:0/2074501376 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6aec108290 msgr2=0x7f6aec19ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:30.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:30.024+0000 7f6af2c4b640 1 --2- 172.21.15.1:0/2074501376 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6aec108290 0x7f6aec19ac70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6ae400ed90 tx=0x7f6ae4012490 comp rx=0 tx=0).stop 2024-09-11T22:02:30.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:30.024+0000 7f6af2c4b640 1 -- 172.21.15.1:0/2074501376 shutdown_connections 2024-09-11T22:02:30.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:30.024+0000 7f6af2c4b640 1 --2- 172.21.15.1:0/2074501376 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f6adc0a2cf0 0x7f6adc0a51b0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:30.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:30.024+0000 7f6af2c4b640 1 --2- 172.21.15.1:0/2074501376 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6aec108290 0x7f6aec19ac70 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:30.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:30.024+0000 7f6af2c4b640 1 --2- 172.21.15.1:0/2074501376 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6aec103740 0x7f6aec1968e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:30.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:30.024+0000 7f6af2c4b640 1 --2- 172.21.15.1:0/2074501376 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6aec102d90 0x7f6aec1963a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:30.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:30.024+0000 7f6af2c4b640 1 -- 172.21.15.1:0/2074501376 >> 172.21.15.1:0/2074501376 conn(0x7f6aec0fc7d0 msgr2=0x7f6aec106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:30.030 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.024+0000 7f177e59c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9fda129d0 0x55c9fda1f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:30.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:30.024+0000 7f6af2c4b640 1 -- 172.21.15.1:0/2074501376 shutdown_connections 2024-09-11T22:02:30.031 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.024+0000 7f177e59c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9fda129d0 0x55c9fda1f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:46380/0 (socket says 172.21.15.1:46380) 2024-09-11T22:02:30.031 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.024+0000 7f177e59c640 1 -- 172.21.15.1:0/466788821 learned_addr learned my addr 172.21.15.1:0/466788821 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:30.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:30.024+0000 7f6af2c4b640 1 -- 172.21.15.1:0/2074501376 wait complete. 2024-09-11T22:02:30.035 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.024+0000 7f177ed9d640 1 --2- 172.21.15.1:0/466788821 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c9fda220c0 0x55c9fda3a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:30.035 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.024+0000 7f177dd9b640 1 --2- 172.21.15.1:0/466788821 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9fda132f0 0x55c9fda39e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:30.035 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.024+0000 7f177e59c640 1 -- 172.21.15.1:0/466788821 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9fda132f0 msgr2=0x55c9fda39e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:30.036 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.024+0000 7f177e59c640 1 --2- 172.21.15.1:0/466788821 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9fda132f0 0x55c9fda39e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:30.036 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.024+0000 7f177e59c640 1 -- 172.21.15.1:0/466788821 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c9fda220c0 msgr2=0x55c9fda3a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:30.036 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.024+0000 7f177e59c640 1 --2- 172.21.15.1:0/466788821 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c9fda220c0 0x55c9fda3a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:30.036 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.024+0000 7f177e59c640 1 -- 172.21.15.1:0/466788821 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9fda0a2d0 con 0x55c9fda129d0 2024-09-11T22:02:30.036 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.024+0000 7f177dd9b640 1 --2- 172.21.15.1:0/466788821 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9fda132f0 0x55c9fda39e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:02:30.036 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.025+0000 7f177e59c640 1 --2- 172.21.15.1:0/466788821 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9fda129d0 0x55c9fda1f730 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f177000ea30 tx=0x7f17700d5140 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3241a8c22056c4ed server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:30.036 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.025+0000 7f1767fff640 1 -- 172.21.15.1:0/466788821 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17700a4850 con 0x55c9fda129d0 2024-09-11T22:02:30.036 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.025+0000 7f1767fff640 1 -- 172.21.15.1:0/466788821 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f177009c2e0 con 0x55c9fda129d0 2024-09-11T22:02:30.036 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.025+0000 7f1767fff640 1 -- 172.21.15.1:0/466788821 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17700cfa00 con 0x55c9fda129d0 2024-09-11T22:02:30.036 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.026+0000 7f17877e6740 1 -- 172.21.15.1:0/466788821 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9fda129d0 msgr2=0x55c9fda1f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:30.036 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.026+0000 7f17877e6740 1 --2- 172.21.15.1:0/466788821 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9fda129d0 0x55c9fda1f730 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f177000ea30 tx=0x7f17700d5140 comp rx=0 tx=0).stop 2024-09-11T22:02:30.036 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.026+0000 7f17877e6740 1 -- 172.21.15.1:0/466788821 shutdown_connections 2024-09-11T22:02:30.037 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.026+0000 7f17877e6740 1 --2- 172.21.15.1:0/466788821 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c9fda220c0 0x55c9fda3a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:30.037 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.026+0000 7f17877e6740 1 --2- 172.21.15.1:0/466788821 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9fda132f0 0x55c9fda39e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:30.037 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.026+0000 7f17877e6740 1 --2- 172.21.15.1:0/466788821 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9fda129d0 0x55c9fda1f730 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:30.037 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.026+0000 7f17877e6740 1 -- 172.21.15.1:0/466788821 >> 172.21.15.1:0/466788821 conn(0x55c9fda0f060 msgr2=0x55c9fda1f020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:30.037 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.026+0000 7f17877e6740 1 -- 172.21.15.1:0/466788821 shutdown_connections 2024-09-11T22:02:30.037 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.026+0000 7f17877e6740 1 -- 172.21.15.1:0/466788821 wait complete. 2024-09-11T22:02:30.037 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.026+0000 7f17877e6740 1 -- start start 2024-09-11T22:02:30.037 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.026+0000 7f17877e6740 1 Processor -- start 2024-09-11T22:02:30.037 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.026+0000 7f17877e6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9fda129d0 0x55c9fda2a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:30.037 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.026+0000 7f17877e6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c9fda132f0 0x55c9fda1bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:30.038 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.026+0000 7f17877e6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9fda220c0 0x55c9fda1c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:30.038 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.026+0000 7f17877e6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f17801acc60 con 0x55c9fda132f0 2024-09-11T22:02:30.038 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.026+0000 7f17877e6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f17801ac560 con 0x55c9fda129d0 2024-09-11T22:02:30.038 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.026+0000 7f17877e6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f17801a97a0 con 0x55c9fda220c0 2024-09-11T22:02:30.038 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.026+0000 7f17877e6740 10 client.0 _ll_register_callbacks cb 0x7f1770003e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:02:30.038 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.026+0000 7f177dd9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c9fda132f0 0x55c9fda1bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:30.038 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.026+0000 7f177ed9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9fda220c0 0x55c9fda1c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:30.038 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.026+0000 7f177e59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9fda129d0 0x55c9fda2a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:30.038 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.026+0000 7f177ed9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9fda220c0 0x55c9fda1c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:46390/0 (socket says 172.21.15.1:46390) 2024-09-11T22:02:30.038 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.027+0000 7f177ed9d640 1 -- 172.21.15.1:0/4246722044 learned_addr learned my addr 172.21.15.1:0/4246722044 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.026+0000 7f177dd9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c9fda132f0 0x55c9fda1bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33802/0 (socket says 172.21.15.1:33802) 2024-09-11T22:02:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.027+0000 7f177ed9d640 1 -- 172.21.15.1:0/4246722044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9fda129d0 msgr2=0x55c9fda2a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.027+0000 7f177ed9d640 1 --2- 172.21.15.1:0/4246722044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9fda129d0 0x55c9fda2a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.027+0000 7f177ed9d640 1 -- 172.21.15.1:0/4246722044 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c9fda132f0 msgr2=0x55c9fda1bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.027+0000 7f177ed9d640 1 --2- 172.21.15.1:0/4246722044 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c9fda132f0 0x55c9fda1bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.027+0000 7f177ed9d640 1 -- 172.21.15.1:0/4246722044 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9fda1cb10 con 0x55c9fda220c0 2024-09-11T22:02:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.027+0000 7f177e59c640 1 --2- 172.21.15.1:0/4246722044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9fda129d0 0x55c9fda2a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:02:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.027+0000 7f177ed9d640 1 --2- 172.21.15.1:0/4246722044 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9fda220c0 0x55c9fda1c310 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f17780126f0 tx=0x7f1778004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12ec99058ab26127 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.027+0000 7f1765ffb640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-11T22:02:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.027+0000 7f1765ffb640 1 -- 172.21.15.1:0/4246722044 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1778021760 con 0x55c9fda220c0 2024-09-11T22:02:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.027+0000 7f17877e6740 1 -- 172.21.15.1:0/4246722044 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f177009c2e0 con 0x55c9fda220c0 2024-09-11T22:02:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.027+0000 7f1765ffb640 1 -- 172.21.15.1:0/4246722044 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f1778021900 con 0x55c9fda220c0 2024-09-11T22:02:30.040 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.027+0000 7f1765ffb640 1 -- 172.21.15.1:0/4246722044 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1778021b00 con 0x55c9fda220c0 2024-09-11T22:02:30.040 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.027+0000 7f17877e6740 10 client.26002 fetch_fsmap learned FSMap version 33 2024-09-11T22:02:30.040 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.028+0000 7f17877e6740 1 -- 172.21.15.1:0/4246722044 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c9fda9ced0 con 0x55c9fda220c0 2024-09-11T22:02:30.040 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.028+0000 7f1765ffb640 1 -- 172.21.15.1:0/4246722044 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f177009c2e0 con 0x55c9fda220c0 2024-09-11T22:02:30.040 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.028+0000 7f1765ffb640 1 -- 172.21.15.1:0/4246722044 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7f177800f040 con 0x55c9fda220c0 2024-09-11T22:02:30.040 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.028+0000 7f17877e6740 10 client.26002 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T22:02:30.040 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.028+0000 7f17877e6740 10 client.26002 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:02:30.040 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.028+0000 7f17877e6740 20 client.26002 populate_metadata read hostname 'smithi001' 2024-09-11T22:02:30.040 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.028+0000 7f17877e6740 1 --2- 172.21.15.1:0/4246722044 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55c9fda9d600 0x55c9fda9fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:30.040 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.028+0000 7f17877e6740 4 client.26002 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:02:30.040 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.028+0000 7f17877e6740 1 -- 172.21.15.1:0/4246722044 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c9fd97ed70 con 0x55c9fda9d600 2024-09-11T22:02:30.040 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.028+0000 7f177e59c640 1 --2- 172.21.15.1:0/4246722044 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55c9fda9d600 0x55c9fda9fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:30.040 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.029+0000 7f1765ffb640 10 client.26002 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:02:30.041 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.029+0000 7f177e59c640 1 --2- 172.21.15.1:0/4246722044 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55c9fda9d600 0x55c9fda9fa20 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6a1573afa49c44a server_cookie=a16e239f06135bbd in_seq=0 out_seq=0 2024-09-11T22:02:30.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.448 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.448 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.448 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.448 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.448 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.449 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.449 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.450 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.450 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.450 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.450 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.450 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.450 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.450 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.450 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.450 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.450 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.450 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.450 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.450 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.455 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.455 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.455 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.455 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.455 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.455 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.455 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.455 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.455 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.455 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.455 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.455 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.455 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.455 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.456 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.456 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.456 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.456 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.456 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.456 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.456 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.456 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.456 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.456 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.456 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.457 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.457 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.457 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.457 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.462 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.462 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.462 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.462 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.463 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.463 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.463 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.463 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.463 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.463 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.464 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.464 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.464 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.464 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.464 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.464 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.464 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.464 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.464 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.464 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.464 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.464 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.464 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.465 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.465 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.465 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.465 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.465 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.465 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.465 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.465 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.465 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.465 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.465 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.465 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.465 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.466 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.466 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.466 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.466 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.466 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.466 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.466 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.466 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.466 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.466 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.466 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.466 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.467 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.467 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.467 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.467 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.467 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.467 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.467 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.467 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.467 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.467 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.467 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.467 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.467 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.467 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.468 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.468 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.468 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.468 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.468 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.468 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.468 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.468 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.468 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.468 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.468 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.468 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.469 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.469 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.469 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.469 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.469 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.469 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.469 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.469 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.469 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.469 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.469 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.469 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.469 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.469 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.470 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.470 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.470 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.470 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.470 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.470 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.470 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.470 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.470 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.470 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.470 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.470 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.470 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.470 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.471 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.471 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.471 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.471 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.471 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.471 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.471 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.471 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.471 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.471 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.471 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.471 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.471 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.471 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.472 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.472 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.472 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.472 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.472 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.472 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.472 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.472 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.472 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.472 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.472 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.472 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.473 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.473 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.473 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.473 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.473 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.473 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.473 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.473 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.473 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.473 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.473 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.473 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.473 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.473 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.474 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.474 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.474 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.474 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.474 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.474 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.474 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.474 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.474 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.474 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.474 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.474 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.474 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.475 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.475 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.475 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.475 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.475 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.475 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.475 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.475 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.475 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.475 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.475 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.475 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.475 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.475 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.476 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.476 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.476 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.476 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.476 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.476 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.476 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.476 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.476 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.476 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.476 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.476 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.476 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.476 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.477 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.477 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.477 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.477 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.477 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.477 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.477 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.477 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.477 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.477 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.477 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.477 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.477 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.478 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.478 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.478 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.478 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.478 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.478 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.478 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.478 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.478 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.478 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.478 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.478 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.478 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.478 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.479 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.479 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.479 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.479 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.479 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.479 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.479 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.479 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.479 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.479 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.479 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.479 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.479 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.480 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.480 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.480 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.480 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.480 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.480 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.480 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.480 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.480 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.480 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.480 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.480 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.480 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.480 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.481 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.481 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.481 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.481 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.481 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.481 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.481 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.481 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.481 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.481 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.481 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.481 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.482 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.482 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.482 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.482 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.482 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.482 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.482 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.482 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.482 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.482 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.482 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.482 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.482 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.482 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.483 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.483 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.483 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.483 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.483 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.483 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.483 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.483 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.483 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.483 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.483 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.483 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.483 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.484 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.484 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.484 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.484 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.484 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.484 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.484 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.484 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.484 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.484 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.484 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.484 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.484 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.484 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.485 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.485 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.485 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.485 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.485 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.485 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.485 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.485 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.485 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.485 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.485 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.485 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.485 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.485 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.486 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.486 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.486 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.486 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.486 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.486 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.486 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.486 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.486 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.486 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.486 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.486 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.486 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.487 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.487 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.487 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.487 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.487 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.487 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.487 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.487 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.487 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.487 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.487 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.487 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.487 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.487 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.488 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.488 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.488 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.488 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.488 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.488 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.488 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.488 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.488 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.488 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.488 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.488 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.488 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.488 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.489 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.489 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.489 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.489 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.489 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.489 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.489 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.489 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.489 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.489 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.489 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.489 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.489 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.490 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.490 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.490 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.490 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.490 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.490 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.490 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.490 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.490 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.490 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.490 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.490 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.490 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.490 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.491 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.491 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.491 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.491 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.492 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.492 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.492 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.492 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.492 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.492 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.492 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.492 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.492 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.492 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.493 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.493 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.493 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.493 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.493 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.493 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.493 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.493 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.493 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.493 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.493 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.493 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.494 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.494 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.494 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.494 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.494 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.494 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.494 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.494 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.494 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.494 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.494 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.494 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.494 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.495 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.495 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.495 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.495 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.495 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.495 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.495 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.495 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.495 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.495 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.496 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.496 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.496 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.496 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.496 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.496 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.496 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.496 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.496 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.496 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.497 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.497 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.497 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.497 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.497 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.497 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.497 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.497 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.497 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.497 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.497 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.497 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.497 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.497 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.498 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.498 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.498 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.498 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.498 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.498 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.498 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.498 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.498 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.498 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.498 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.498 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.498 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.498 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.499 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.499 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.499 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.499 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.499 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.499 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.499 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.499 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.499 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.499 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.499 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.499 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.499 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.499 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.499 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.500 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.500 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.500 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.500 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.500 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.500 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.500 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.500 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.500 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.500 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.500 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.500 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.500 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.501 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.501 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.501 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.501 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.501 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.501 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.501 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.501 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.501 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.501 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.501 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.501 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.501 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.502 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.502 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.502 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.502 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.502 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.502 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.502 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.502 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.502 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.502 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.502 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.502 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.502 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.503 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.503 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.503 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.503 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.503 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.503 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.503 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.503 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.503 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.503 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.503 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.503 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.503 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.503 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.503 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.504 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.504 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.504 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.504 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.504 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.504 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.504 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.504 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.504 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.504 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.504 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.504 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.504 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.505 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.505 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.505 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.505 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.505 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.505 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.505 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.505 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.505 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.505 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.505 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.505 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.505 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.505 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.506 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.506 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.506 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:30.506 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.506 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:30 smithi001 ceph-mon[30935]: pgmap v163: 97 pgs: 97 active+clean; 132 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 6.6 MiB/s wr, 2.84k op/s 2024-09-11T22:02:30.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.552 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.552 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.552 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.552 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.552 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.553 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.553 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.554 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.554 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.554 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.554 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.554 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.554 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.554 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.554 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.554 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.554 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.554 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.554 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.555 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.555 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.555 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.555 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.555 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.555 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.555 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.555 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.555 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.555 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.555 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.555 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.555 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.555 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.556 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.556 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.556 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.556 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.556 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.556 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.556 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.556 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.556 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.556 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.556 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.556 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.557 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.557 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.557 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.557 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.557 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.557 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.557 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.557 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.557 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.557 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.557 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.557 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.557 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.557 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.558 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.558 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.558 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.558 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.558 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.558 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.558 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.558 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.558 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.558 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.558 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.558 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.558 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.558 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.559 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.559 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.559 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.559 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.559 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.559 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.559 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.559 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.559 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.559 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.559 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.559 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.559 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.560 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.560 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.560 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.560 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.560 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.560 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.560 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.560 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.560 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.560 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.560 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.560 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.560 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.560 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.561 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.561 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.561 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.561 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.561 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.561 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.561 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.561 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.561 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.561 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.561 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.561 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.561 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.562 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.562 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.562 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.562 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.562 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.562 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.562 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.562 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.562 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.562 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.562 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.562 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.562 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.562 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.563 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.563 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.563 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.563 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.563 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.563 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.563 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.563 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.564 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.564 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.564 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.564 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.564 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.564 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.564 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.564 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.564 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.564 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.564 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.565 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.565 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.565 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.565 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.565 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.565 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.565 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.565 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.565 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.565 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.565 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.565 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.565 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.566 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.566 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.566 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.566 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.566 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.566 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.566 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.566 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.566 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.566 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.566 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.566 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.566 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.566 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.567 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.567 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.567 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.567 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.567 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.567 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.567 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.567 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.567 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.567 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.567 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.567 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.567 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.567 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.568 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.568 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.568 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.568 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.568 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.568 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.568 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.568 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.568 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.568 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.568 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.568 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.568 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.568 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.569 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.569 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.569 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.569 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.569 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.569 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.569 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.569 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.569 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.569 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.569 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.569 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.569 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.569 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.570 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.570 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.570 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.570 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.570 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.570 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.570 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.570 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.570 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.570 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.570 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.570 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.570 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.570 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.571 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.571 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.571 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.571 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.571 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.571 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.571 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.571 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.571 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.571 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.571 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.571 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.571 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.571 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.572 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.572 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.572 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.572 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.572 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.572 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.572 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.572 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.572 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.572 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.572 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.572 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.572 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.573 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.573 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.573 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.573 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.573 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.573 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.573 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.573 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.573 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.573 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.573 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.573 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.573 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.573 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.574 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.574 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.574 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.574 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.574 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.574 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.574 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.574 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.574 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.574 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.574 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.574 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.574 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.575 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.575 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.575 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.575 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.575 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.575 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.575 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.575 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.575 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.575 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.575 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.575 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.575 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.575 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.575 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.576 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.576 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.576 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.576 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.576 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.576 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.576 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.576 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.576 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.576 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.576 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.576 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.576 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.577 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.577 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.577 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.577 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.577 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.577 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.577 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.577 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.577 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.577 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.577 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.577 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.577 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.578 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.578 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.578 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.578 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.578 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.578 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.578 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.578 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.578 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.578 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.578 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.578 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.578 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.579 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.579 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.579 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.579 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.579 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.579 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.579 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.579 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.579 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.579 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.579 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.579 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.579 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.579 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.579 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.580 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.580 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.580 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.580 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.580 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.580 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.580 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.580 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.580 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.580 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.580 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.580 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.580 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.581 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.581 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.581 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.581 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.581 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.581 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.581 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.581 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.581 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.581 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.581 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.581 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.581 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.581 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.582 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.582 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.582 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.582 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.582 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.582 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.582 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.582 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.582 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.582 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.582 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.582 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.582 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.583 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.583 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.583 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.583 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.583 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.583 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.583 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.583 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.583 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.583 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.583 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.583 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.583 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.584 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.584 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.584 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.584 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.584 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.584 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.584 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.584 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.584 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.584 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.584 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.584 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.584 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.584 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.585 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.585 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.585 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.585 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.585 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.585 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.585 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.585 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.585 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.585 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.585 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.585 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.585 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.586 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.586 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.586 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.586 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.586 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.586 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.586 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.586 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.586 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.586 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.586 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.586 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.586 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.586 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.587 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.587 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.587 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.587 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.587 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.587 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.587 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.587 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.587 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.587 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.587 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.587 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.587 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.587 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.588 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.588 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.588 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.588 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.588 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.588 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.588 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.588 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.588 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.588 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.588 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.588 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.589 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.589 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.589 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.589 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.589 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.589 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.589 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.589 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.589 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.589 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.589 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.589 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.589 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.590 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.590 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.590 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.590 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.590 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.590 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.590 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.590 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.590 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.590 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.590 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.590 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.590 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.591 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.591 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.591 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.591 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.591 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.591 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.591 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.591 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.591 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.591 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.591 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.591 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.591 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.591 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.591 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.592 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.592 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.592 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.592 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.592 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.592 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.592 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.592 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.592 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.592 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.592 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.592 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.593 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.593 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.593 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.593 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.593 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.594 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.594 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.594 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.594 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.594 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.595 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.595 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.595 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.595 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.595 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.595 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.595 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.595 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.595 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.595 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.595 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.595 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.595 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.596 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.596 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.596 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.596 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.596 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.596 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.596 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.596 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.596 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.596 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.597 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.597 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.597 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.597 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.597 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.597 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.597 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.597 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.597 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.597 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.597 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.597 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.597 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.597 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.598 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.598 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.598 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.598 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.598 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.598 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.598 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.598 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.598 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.598 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.598 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.598 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.598 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.599 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.599 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.599 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.599 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.599 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.599 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.599 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.599 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.599 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.599 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.599 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.599 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.599 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.599 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.600 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.600 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.600 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.600 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.600 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.600 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.600 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.600 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.600 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.600 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.600 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.600 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.600 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.601 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.601 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.601 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.601 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.601 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.601 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.601 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.601 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.601 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.601 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.601 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.601 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.601 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.601 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.602 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.602 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.602 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.602 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.602 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.602 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.602 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.602 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.602 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.602 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.602 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.602 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.602 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.603 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.603 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.603 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.603 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.603 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.603 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.603 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.603 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.603 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.603 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.603 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.603 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.603 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.603 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.604 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.604 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.604 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.604 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.604 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.604 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.604 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:30.604 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.604 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:30 smithi053 ceph-mon[34908]: pgmap v163: 97 pgs: 97 active+clean; 132 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 6.6 MiB/s wr, 2.84k op/s 2024-09-11T22:02:30.604 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.604 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.604 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.605 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.605 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.605 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.605 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.605 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.605 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.605 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.605 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.605 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.605 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.605 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.605 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.605 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.606 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.606 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.606 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.606 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.606 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.606 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.606 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.606 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.606 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.606 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.606 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.606 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.606 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.607 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.607 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.607 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.607 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.607 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.607 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.607 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.607 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.607 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.607 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.607 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.607 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.607 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.607 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.608 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.608 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.608 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.608 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.608 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.608 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.608 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.608 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.608 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.608 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.608 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.608 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.608 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.609 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.609 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.609 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.609 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.609 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.609 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.609 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.609 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.609 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.609 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.609 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.609 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.610 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.610 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.610 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.610 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.610 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.610 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.610 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.610 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.610 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.610 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.610 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.610 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.610 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.610 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.611 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.611 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.611 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.611 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.611 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.611 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.611 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.611 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.611 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.611 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.611 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.611 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.611 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.611 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.612 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.612 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.612 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.612 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.612 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.612 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.612 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.612 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.612 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.612 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.612 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.612 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.612 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.613 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.615 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.615 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.615 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.615 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.615 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.615 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.615 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.615 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.615 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.615 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.615 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.615 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.615 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.615 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.616 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.616 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.616 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.616 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.616 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.616 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.616 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.616 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.616 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.616 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.616 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.616 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.617 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.617 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.617 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.617 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.617 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.617 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.617 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.617 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.617 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.617 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.617 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.617 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.617 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.618 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.618 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.618 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.618 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.618 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.618 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.618 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.618 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.618 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.618 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.618 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.618 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.618 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.618 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.619 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.619 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.619 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.619 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.619 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.619 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.619 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.619 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.619 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.619 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.619 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.619 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.619 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.620 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.620 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.620 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.620 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.620 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.620 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.620 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.620 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.620 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.627 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.627 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.627 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.627 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.627 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.627 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.627 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.627 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.628 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.628 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.628 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.628 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.628 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.628 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.628 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.628 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.628 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.628 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.628 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.628 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.629 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.629 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.629 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.629 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.629 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.629 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.629 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.629 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.629 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.629 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.629 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.629 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.629 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.629 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.630 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.630 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.630 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.630 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.630 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.630 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.630 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.630 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.630 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.630 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.630 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.630 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.631 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.631 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.631 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.631 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.631 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.631 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.631 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.631 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.631 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.631 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.631 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.631 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.631 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.631 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.632 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.632 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.632 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.632 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.632 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.632 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.632 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.632 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.632 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.632 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.632 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.632 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.633 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.633 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.633 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.633 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.633 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.633 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.633 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.633 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.633 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.633 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.633 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.633 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.633 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.634 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.634 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.634 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.634 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.634 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.634 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.634 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.634 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.634 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.634 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.634 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.634 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.634 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.634 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.635 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.635 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.635 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.635 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.635 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.635 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.635 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.635 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.635 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.635 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.635 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.635 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.635 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.636 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.636 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.636 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.636 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.636 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.636 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.636 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.636 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.636 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.636 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.636 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.636 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.636 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.637 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.637 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.637 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.637 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.637 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.637 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.637 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.637 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.637 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.637 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.637 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.637 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.640 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.640 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.640 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.640 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.640 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.640 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.640 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.640 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.640 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.640 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.640 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.640 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.640 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.641 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.641 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.641 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.641 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.641 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.641 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.641 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.641 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.641 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.641 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.641 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.641 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.641 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.642 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.642 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.642 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.642 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.642 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.642 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.642 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.642 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.642 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.642 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.642 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.642 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.642 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.643 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.643 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.643 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.643 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.643 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.643 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.643 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.643 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.643 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.643 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.643 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.643 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.643 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.643 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.644 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.644 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.644 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.644 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.644 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.644 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.644 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.644 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.644 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.644 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.644 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.644 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.644 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.645 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.645 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.645 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.645 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.645 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.645 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.645 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.645 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.645 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.645 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.645 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.645 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.645 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.646 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.646 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.646 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.646 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.646 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.646 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.646 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.646 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.646 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.646 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.646 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.646 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.646 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.647 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.647 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.647 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.647 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.647 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.647 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.647 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.647 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.647 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.647 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.647 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.647 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.647 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.648 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.648 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.648 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.648 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.648 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.648 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.648 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.648 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.648 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.648 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.648 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.648 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.648 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.649 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.649 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.649 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.649 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.649 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.649 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.649 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.649 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.649 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.649 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.649 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.649 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.649 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.650 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.650 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.650 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.650 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.650 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.650 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.650 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.650 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.650 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.650 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.650 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.650 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.650 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.650 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.650 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.651 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.651 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.651 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.651 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.651 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.651 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.651 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.651 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.651 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.651 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.651 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.651 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.651 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.651 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.652 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.652 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.652 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.652 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.652 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.652 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.652 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.652 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.652 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.652 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.652 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.652 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.652 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.653 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.653 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.653 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.653 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.653 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.653 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.653 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.653 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.653 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.653 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.653 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.653 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.653 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.654 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.654 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.654 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.654 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.654 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.654 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.654 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.654 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.654 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.654 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.654 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.654 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.654 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.654 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.655 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.655 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.655 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.655 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.655 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.655 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.655 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.655 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.655 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.655 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.655 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.655 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.655 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.655 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.656 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.656 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.656 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.656 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.656 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.656 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.656 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.656 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.656 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.656 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.656 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.656 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.656 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.657 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.657 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.657 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.657 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.657 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.657 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.657 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.657 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.657 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.657 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.657 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.657 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.657 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.658 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.658 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.658 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.658 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.658 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.658 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.658 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.658 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.658 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.658 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.658 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.658 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.658 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.658 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.659 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.659 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.659 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.659 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.659 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.659 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.659 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.659 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.659 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.659 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.659 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.659 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.659 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.660 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.660 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.660 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.660 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.660 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.660 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.660 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.660 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.660 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.660 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.660 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.660 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.660 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.661 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.661 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.661 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.661 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.661 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.661 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.661 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.661 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.661 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.661 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.661 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.661 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.661 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.661 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.662 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.662 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.662 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.662 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.662 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.662 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.662 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.662 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.662 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.662 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.662 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.662 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.662 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.663 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.663 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.663 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.663 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.663 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.663 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.663 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.663 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.663 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.663 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.663 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.663 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.663 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.663 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.664 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.664 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.664 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.664 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.664 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.664 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.664 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.664 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.664 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.665 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.665 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.665 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.665 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:30.665 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:30.665 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:30 smithi161 ceph-mon[35615]: pgmap v163: 97 pgs: 97 active+clean; 132 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 6.6 MiB/s wr, 2.84k op/s 2024-09-11T22:02:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.594+0000 7f1765ffb640 1 -- 172.21.15.1:0/4246722044 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c9fd97ed70 con 0x55c9fda9d600 2024-09-11T22:02:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.594+0000 7f1765ffb640 10 client.26002 handle_command_reply: tid=0 2024-09-11T22:02:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.594+0000 7f17877e6740 1 client.26002 shutdown 2024-09-11T22:02:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.594+0000 7f17877e6740 1 -- 172.21.15.1:0/4246722044 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9fda220c0 msgr2=0x55c9fda1c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.595+0000 7f17877e6740 1 --2- 172.21.15.1:0/4246722044 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9fda220c0 0x55c9fda1c310 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f17780126f0 tx=0x7f1778004ab0 comp rx=0 tx=0).stop 2024-09-11T22:02:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.595+0000 7f17877e6740 1 -- 172.21.15.1:0/4246722044 shutdown_connections 2024-09-11T22:02:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.595+0000 7f17877e6740 1 --2- 172.21.15.1:0/4246722044 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55c9fda9d600 0x55c9fda9fa20 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.595+0000 7f17877e6740 1 --2- 172.21.15.1:0/4246722044 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9fda220c0 0x55c9fda1c310 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.595+0000 7f17877e6740 1 --2- 172.21.15.1:0/4246722044 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c9fda132f0 0x55c9fda1bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.595+0000 7f17877e6740 1 --2- 172.21.15.1:0/4246722044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9fda129d0 0x55c9fda2a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.595+0000 7f17877e6740 1 -- 172.21.15.1:0/4246722044 >> 172.21.15.1:0/4246722044 conn(0x55c9fda0f060 msgr2=0x55c9fda0e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.595+0000 7f1765ffb640 0 client.26002 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:02:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.595+0000 7f17877e6740 1 -- 172.21.15.1:0/4246722044 shutdown_connections 2024-09-11T22:02:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.595+0000 7f17877e6740 1 -- 172.21.15.1:0/4246722044 wait complete. 2024-09-11T22:02:30.666 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.595+0000 7f17877e6740 20 client.26002 trim_cache size 0 max 16384 2024-09-11T22:02:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-09-11T22:02:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-09-11T22:02:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-09-11T22:02:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-09-11T22:02:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.596+0000 7f1784a4c640 1 -- 172.21.15.1:0/3494491397 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f17680a2cf0 msgr2=0x7f17680a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.596+0000 7f1784a4c640 1 --2- 172.21.15.1:0/3494491397 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f17680a2cf0 0x7f17680a51b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1778005fd0 tx=0x7f1778004a40 comp rx=0 tx=0).stop 2024-09-11T22:02:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.596+0000 7f1784a4c640 1 -- 172.21.15.1:0/3494491397 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17801009a0 msgr2=0x7f1780199260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.596+0000 7f1784a4c640 1 --2- 172.21.15.1:0/3494491397 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17801009a0 0x7f1780199260 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f177000ea60 tx=0x7f177000ef20 comp rx=0 tx=0).stop 2024-09-11T22:02:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.597+0000 7f1784a4c640 1 -- 172.21.15.1:0/3494491397 shutdown_connections 2024-09-11T22:02:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.597+0000 7f1784a4c640 1 --2- 172.21.15.1:0/3494491397 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f17680a2cf0 0x7f17680a51b0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.597+0000 7f1784a4c640 1 --2- 172.21.15.1:0/3494491397 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f17801069b0 0x7f17801a42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.597+0000 7f1784a4c640 1 --2- 172.21.15.1:0/3494491397 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1780101340 0x7f17801997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.597+0000 7f1784a4c640 1 --2- 172.21.15.1:0/3494491397 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17801009a0 0x7f1780199260 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.597+0000 7f1784a4c640 1 -- 172.21.15.1:0/3494491397 >> 172.21.15.1:0/3494491397 conn(0x7f17800779a0 msgr2=0x7f1780103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.597+0000 7f1784a4c640 1 -- 172.21.15.1:0/3494491397 shutdown_connections 2024-09-11T22:02:30.669 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:30.597+0000 7f1784a4c640 1 -- 172.21.15.1:0/3494491397 wait complete. 2024-09-11T22:02:31.414 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.414 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.414 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.414 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.414 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.414 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.415 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.415 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.415 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.415 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.415 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.415 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.415 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.415 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.415 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.415 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.415 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.415 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.415 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.416 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.416 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.416 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.416 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.416 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.416 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.416 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.416 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.416 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.416 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.416 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.416 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.416 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.417 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.417 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.417 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.417 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.417 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.417 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.417 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.417 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.417 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.417 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.417 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.417 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.417 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.418 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.418 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.418 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.418 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.418 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.418 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.418 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.419 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.419 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.420 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.420 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.420 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.420 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.420 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.420 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.420 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.420 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.420 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.420 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.420 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.420 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.420 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.421 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.421 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.421 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.421 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.421 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.421 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.421 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.421 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.421 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.421 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.421 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.421 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.421 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.422 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.422 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.422 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.422 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.422 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.422 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.422 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.422 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.422 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.422 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.422 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.422 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.422 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.423 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.423 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.423 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.423 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.423 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.423 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.423 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.423 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.423 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.423 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.423 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.423 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.423 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.424 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.424 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.424 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.424 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.424 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.424 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.424 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.424 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.424 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.424 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.424 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.424 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.424 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.424 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.425 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.425 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.425 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.425 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.425 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.425 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.425 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.425 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.425 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.425 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.425 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.425 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.425 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.426 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.426 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.426 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.426 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.426 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.426 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.426 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.426 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.426 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.426 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.426 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.426 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.426 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.427 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.428 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.429 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.429 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.429 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.429 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.429 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.429 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.429 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.429 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.429 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.429 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.429 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.429 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.429 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.430 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.430 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.430 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.430 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.430 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.430 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.430 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.430 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.430 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.430 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.430 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.430 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.430 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.430 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.431 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.431 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.431 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.431 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.431 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.431 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.431 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.431 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.431 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.431 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.431 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.431 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.431 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.434 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.434 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.434 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.434 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.434 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.434 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.434 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.434 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.434 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.434 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.434 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.434 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.434 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.434 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.435 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.435 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.435 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.435 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.435 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.435 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.435 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.435 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.435 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.435 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.435 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.435 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.436 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.436 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.436 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.436 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.436 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.436 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.436 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.436 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.436 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.436 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.436 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.437 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.437 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.437 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.437 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.437 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.437 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.437 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.437 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.437 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.437 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.437 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.437 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.437 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.438 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.438 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.438 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.438 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.438 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.438 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.438 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.438 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.438 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.438 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.438 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.438 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.438 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.439 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.439 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.439 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.439 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.439 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.439 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.439 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.439 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.439 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.439 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.439 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.439 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.442 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.442 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.442 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.442 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.442 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.442 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.442 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.442 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.442 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.442 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.442 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.442 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.442 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.442 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.448 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.449 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.449 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.449 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.449 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.449 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.449 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.449 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.449 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.449 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.450 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.450 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.450 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.450 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.450 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.450 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.450 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.450 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.450 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.450 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.450 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.450 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.450 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.455 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.455 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.455 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.455 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.455 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.455 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.455 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.455 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.455 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.455 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.455 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.455 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.455 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.455 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.456 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.456 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.456 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.456 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.456 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.456 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.456 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.456 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.456 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.456 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.456 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.456 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.456 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.456 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.457 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.457 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.457 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.457 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.457 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.457 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.457 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.457 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.457 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.457 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.457 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.457 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.457 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.458 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.458 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.458 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.458 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.458 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.458 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.458 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.458 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.459 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.459 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.459 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.459 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.459 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.459 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.459 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.459 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.459 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.459 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.459 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.459 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.459 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.460 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.460 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.460 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.460 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.460 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.460 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.460 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.460 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.460 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.460 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.460 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.460 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.460 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.461 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.461 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.461 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.461 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.461 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.461 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.461 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.461 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.461 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.461 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.462 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.462 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.462 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.462 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.462 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.462 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.462 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.462 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.462 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.462 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.462 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.462 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.462 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.463 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.463 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.463 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.463 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.463 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.463 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.463 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.463 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.463 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.463 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.463 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.463 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.463 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.463 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.464 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.464 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.464 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.464 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.464 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.464 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.464 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.464 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.465 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.465 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.465 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.465 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.465 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.465 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.465 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.465 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.465 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.465 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.465 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.465 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.466 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.466 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.466 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.466 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.466 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.466 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.466 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.466 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.466 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.466 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.466 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.466 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.466 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.467 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.467 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.467 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.467 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.467 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.467 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.467 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.467 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.467 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.467 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.468 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.468 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.468 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.468 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.468 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.468 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.468 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.468 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.468 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.468 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.468 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.468 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.468 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.468 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.469 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.469 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.469 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.469 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.469 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.469 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.469 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.469 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.469 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.469 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.469 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.469 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.469 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.470 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.470 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.470 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.470 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.470 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.470 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.470 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.470 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.470 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.470 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.470 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.470 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.470 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.470 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.471 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.471 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.471 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.471 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.471 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.471 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.471 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.471 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.471 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.471 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.471 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.471 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.471 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.472 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.472 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.472 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.472 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.472 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.472 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.472 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.472 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.472 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.472 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.472 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.472 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.472 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.473 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.473 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.473 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.473 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.473 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.473 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.473 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.473 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.473 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.473 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.473 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.473 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.473 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.474 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.474 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.474 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.474 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.474 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.474 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.474 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.474 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.474 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.474 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.474 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.474 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.474 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.475 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.475 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.475 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.475 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.475 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.475 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.475 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.475 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.475 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.475 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.475 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.475 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.475 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.476 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.476 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.476 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.476 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.476 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.476 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.476 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.476 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.476 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.476 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.476 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.476 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.476 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.477 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.477 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.477 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.477 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.477 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:31.477 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:31 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.552 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.552 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.552 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.552 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.552 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.553 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.553 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.553 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.553 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.554 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.554 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.554 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.554 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.554 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.554 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.554 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.554 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.554 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.554 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.554 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.554 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.555 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.555 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.555 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.555 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.555 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.555 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.555 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.555 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.555 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.555 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.555 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.555 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.555 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.556 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.556 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.556 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.556 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.556 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.556 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.556 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.556 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.556 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.556 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.556 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.556 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.556 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.557 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.557 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.557 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.557 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.557 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.557 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.557 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.557 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.557 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.557 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.557 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.557 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.558 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.558 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.558 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.558 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.558 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.558 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.558 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.558 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.558 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.558 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.558 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.559 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.559 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.559 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.559 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.559 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.559 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.559 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.559 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.559 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.559 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.559 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.560 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.560 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.560 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.560 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.560 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.560 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.560 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.560 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.560 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.560 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.560 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.560 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.560 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.561 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.561 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.561 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.561 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.561 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.561 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.561 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.561 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.561 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.561 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.561 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.562 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.562 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.562 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.562 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.562 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.562 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.562 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.562 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.562 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.562 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.562 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.562 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.562 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.563 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.563 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.563 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.563 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.563 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.563 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.563 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.563 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.563 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.563 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.563 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.563 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.563 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.563 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.564 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.564 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.564 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.564 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.564 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.564 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.564 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.564 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.564 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.564 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.564 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.564 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.565 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.565 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.565 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.565 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.565 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.565 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.565 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.565 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.565 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.565 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.566 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.566 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.566 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.566 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.566 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.566 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.566 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.566 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.566 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.566 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.566 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.566 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.566 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.567 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.567 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.567 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.567 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.567 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.567 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.567 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.567 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.567 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.567 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.568 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.568 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.568 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.568 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.568 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.568 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.568 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.568 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.568 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.568 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.568 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.568 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.569 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.569 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.569 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.569 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.569 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.569 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.569 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.569 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.569 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.569 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.569 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.569 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.569 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.569 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.570 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.570 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.570 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.570 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.570 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.570 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.570 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.570 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.570 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.570 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.570 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.570 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.570 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.570 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.571 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.571 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.571 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.571 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.571 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.571 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.571 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.571 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.571 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.571 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.571 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.571 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.571 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.572 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.572 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.572 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.572 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.572 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.572 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.572 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.572 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.572 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.572 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.572 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.572 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.572 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.573 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.573 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.573 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.573 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.573 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.573 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.573 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.573 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.573 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.573 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.573 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.573 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.573 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.573 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.574 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.574 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.574 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.574 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.574 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.574 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.574 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.574 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.574 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.574 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.574 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.574 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.574 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.575 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.575 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.575 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.575 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.575 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.575 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.575 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.575 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.575 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.575 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.575 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.575 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.575 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.575 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.576 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.576 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.576 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.576 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.576 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.576 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.576 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.576 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.576 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.576 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.576 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.576 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.577 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.577 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.577 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.577 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.577 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.577 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.577 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.577 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.577 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.577 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.577 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.578 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.578 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.578 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.578 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.578 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.578 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.578 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.578 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.578 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.578 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.578 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.578 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.578 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.579 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.579 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.579 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.579 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.579 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.579 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.579 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.579 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.579 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.579 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.579 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.580 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.580 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.580 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.580 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.580 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.580 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.580 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.580 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.580 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.580 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.580 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.580 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.580 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.581 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.581 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.581 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.581 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.581 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.581 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.581 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.581 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.581 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.581 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.581 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.581 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.582 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.582 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.582 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.582 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.582 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.582 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.582 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.582 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.582 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.582 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.582 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.582 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.583 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.583 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.583 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.583 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.583 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.583 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.583 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.583 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.583 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.583 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.583 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.583 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.583 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.583 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.584 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.584 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.584 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.584 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.584 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.584 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.584 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.584 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.584 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.584 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.584 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.584 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.584 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.585 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.585 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.585 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.585 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.585 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.585 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.585 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.585 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.585 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.585 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.585 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.585 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.585 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.585 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.586 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.586 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.586 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.586 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.586 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.586 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.586 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.586 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.586 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.586 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.586 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.586 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.586 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.586 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.587 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.587 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.588 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.588 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.588 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.588 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.588 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.588 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.588 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.588 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.588 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.588 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.588 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.589 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.589 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.589 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.589 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.589 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.589 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.589 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.589 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.589 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.589 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.589 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.589 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.589 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.589 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.590 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.590 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.590 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.590 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.590 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.590 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.590 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.590 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.590 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.590 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.590 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.590 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.590 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.590 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.591 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.591 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.591 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.591 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.591 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.591 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.591 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.591 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.591 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.591 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.591 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.591 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.591 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.592 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.592 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.592 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.592 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.592 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.592 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.592 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.592 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.592 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.592 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.592 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.592 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.593 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.593 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.593 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.593 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.593 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.593 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.593 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.593 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.593 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.593 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.593 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.593 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.593 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.594 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.594 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.594 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.594 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.594 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.594 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.594 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.594 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.594 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.594 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.594 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.594 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.595 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.595 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.595 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.595 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.595 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.595 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.595 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.595 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.595 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.595 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.595 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.595 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.595 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.595 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.596 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.596 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.596 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.596 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.596 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.596 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.596 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.596 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.596 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.596 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.596 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.596 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.596 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.597 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.597 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.597 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.597 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.597 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.597 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.597 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.597 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.597 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.597 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.597 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.598 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.598 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.598 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.598 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.598 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.598 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.598 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.598 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.598 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.598 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.598 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.598 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.598 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.599 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.599 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.599 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.599 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.599 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.599 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.599 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.599 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.599 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.599 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.599 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.599 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.599 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.599 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.600 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.600 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.600 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.600 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.600 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.600 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.600 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.600 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.600 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.600 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.600 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.600 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.600 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.601 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.601 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.601 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.601 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.601 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.601 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.601 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.601 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.601 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.601 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.601 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.601 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.601 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.602 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.602 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.602 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.602 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.602 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.602 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.602 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.602 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.602 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.602 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.602 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.602 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.602 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.602 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.603 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.603 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.603 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.603 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.603 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.603 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.603 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.603 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.603 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.603 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.603 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.604 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.604 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.604 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.604 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.604 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.604 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.604 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.604 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.604 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.604 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.604 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.604 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.605 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.605 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.605 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.605 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.605 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.605 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.605 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.605 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.605 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.605 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.605 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.605 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.605 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.606 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.606 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.606 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.606 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.606 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.606 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.606 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.606 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.606 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.606 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.606 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.606 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.606 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.607 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.607 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.607 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.607 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.607 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.607 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.607 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.607 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.607 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.607 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.607 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.607 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.607 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.607 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.608 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.608 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.608 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.608 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.608 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.608 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.608 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.608 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.608 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.609 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.609 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.609 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.609 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.609 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.609 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.609 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.609 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.609 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.609 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.609 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.610 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.610 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.610 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.610 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.610 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.610 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.610 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.610 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.610 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.610 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.610 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.610 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.610 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.611 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.611 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.611 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.611 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.611 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.611 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.611 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.611 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.611 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.611 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.611 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:31.611 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:31 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.614 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.615 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.615 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.615 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.615 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.615 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.615 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.615 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.615 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.615 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.615 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.615 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.615 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.615 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.616 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.616 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.616 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.616 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.616 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.616 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.616 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.616 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.616 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.616 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.616 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.616 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.616 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.617 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.617 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.617 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.617 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.617 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.617 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.617 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.617 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.617 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.617 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.617 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.617 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.617 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.617 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.618 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.618 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.618 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.618 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.618 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.618 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.618 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.618 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.618 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.618 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.618 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.618 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.618 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.618 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.619 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.619 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.619 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.619 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.619 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.619 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.619 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.619 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.619 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.619 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.619 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.619 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.620 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.620 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.620 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.620 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.620 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.620 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.620 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.620 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.620 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.620 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.620 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.620 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.620 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.621 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.621 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.621 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.621 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.621 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.621 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.621 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.621 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.621 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.621 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.621 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.621 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.621 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.621 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.622 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.622 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.622 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.622 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.622 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.622 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.622 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.622 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.622 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.622 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.622 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.622 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.622 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.623 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.623 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.623 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.623 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.623 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.623 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.623 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.623 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.623 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.623 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.623 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.623 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.623 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.624 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.624 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.624 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.624 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.624 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.624 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.624 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.624 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.624 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.624 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.624 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.624 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.624 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.625 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.625 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.625 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.625 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.625 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.625 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.625 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.625 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.625 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.625 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.625 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.625 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.625 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.626 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.626 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.626 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.626 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.626 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.626 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.626 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.626 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.626 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.626 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.626 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.626 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.626 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.627 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.627 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.627 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.627 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.627 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.627 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.627 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.627 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.627 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.627 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.627 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.627 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.627 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.627 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.628 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.628 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.628 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.628 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.628 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.628 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.628 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.628 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.628 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.628 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.628 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.628 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.628 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.628 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.629 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.629 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.629 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.629 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.629 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.629 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.629 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.629 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.629 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.629 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.629 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.629 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.629 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.630 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.630 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.630 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.630 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.630 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.630 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.630 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.630 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.630 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.630 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.630 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.630 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.630 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.630 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.631 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.631 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.631 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.631 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.631 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.631 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.631 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.631 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.631 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.631 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.631 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.631 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.631 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.631 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.632 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.632 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.632 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.632 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.632 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.632 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.632 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.632 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.632 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.632 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.632 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.632 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.632 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.633 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.633 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.633 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.633 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.633 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.633 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.633 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.633 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.633 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.633 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.633 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.633 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.633 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.634 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.634 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.634 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.634 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.634 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.634 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.634 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.634 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.634 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.634 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.635 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.635 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.635 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.635 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.635 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.635 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.635 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.635 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.635 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.635 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.635 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.635 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.635 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.636 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.636 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.636 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.636 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.636 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.636 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.636 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.636 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.636 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.636 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.636 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.636 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.636 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.637 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.637 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.637 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.637 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.637 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.637 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.637 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.637 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.637 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.637 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.637 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.637 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.637 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.638 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.639 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.640 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.640 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.640 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.640 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.640 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.640 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.640 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.640 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.640 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.640 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.640 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.640 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.640 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.641 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.641 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.641 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.641 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.641 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.641 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.641 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.641 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.641 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.641 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.641 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.641 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.641 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.642 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.642 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.642 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.642 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.642 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.642 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.642 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.642 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.642 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.642 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.642 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.642 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.642 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.642 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.643 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.643 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.643 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.643 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.643 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.643 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.643 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.643 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.643 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.643 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.643 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.643 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.643 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.644 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.644 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.644 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.644 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.644 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.644 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.644 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.644 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.644 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.644 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.644 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.644 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.644 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.645 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.645 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.645 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.645 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.645 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.645 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.645 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.645 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.645 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.645 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.645 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.645 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.645 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.645 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.646 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.646 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.646 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.646 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.646 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.646 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.646 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.646 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.646 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.646 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.646 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.646 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.646 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.647 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.647 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.647 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.647 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.647 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.647 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.647 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.647 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.648 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.648 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.648 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.648 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.648 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.648 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.649 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.649 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.649 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.649 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.649 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.649 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.649 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.649 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.649 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.649 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.650 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.650 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.650 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.650 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.650 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.650 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.651 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.651 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.651 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.651 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.651 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.651 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.651 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.651 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.651 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.651 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.651 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.651 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.651 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.652 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.652 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.652 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.652 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.652 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.652 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.652 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.652 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.652 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.652 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.652 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.652 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.652 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.653 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.653 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.653 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.653 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.653 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.653 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.653 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.653 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.653 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.653 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.653 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.653 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.653 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.653 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.654 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.654 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.654 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.654 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.654 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.654 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.654 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.654 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.654 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.654 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.654 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.654 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.655 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.655 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.655 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.655 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.655 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.655 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.655 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.655 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.655 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.655 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.655 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.655 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.655 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.656 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.656 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.656 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.656 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.656 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.656 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.656 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.656 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.656 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.656 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.656 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.656 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.656 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.657 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.657 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.657 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.657 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.657 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.657 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.657 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.657 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.657 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.657 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.657 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.657 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.658 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.658 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.658 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.658 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.658 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.658 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.658 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.658 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.658 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.658 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.658 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.658 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.658 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.658 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.658 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.659 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.659 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.659 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.659 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.659 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.659 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.659 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.659 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.660 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.660 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.660 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.660 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.660 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.660 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.660 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.660 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.660 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.660 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.660 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.660 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.661 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.661 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.661 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.661 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.661 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.661 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.661 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.661 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.661 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.661 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.661 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.661 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.661 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.662 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.662 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.662 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.662 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.662 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.662 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.663 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.663 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.663 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.663 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.663 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.663 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.664 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.664 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.664 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.664 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.664 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.664 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.664 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.664 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.664 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.664 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.664 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.664 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.664 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.664 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.665 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.665 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.665 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.665 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.665 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.665 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.665 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.665 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.665 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.665 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.665 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.665 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.666 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.666 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.666 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.666 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.666 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.666 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.666 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.666 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.666 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.666 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.667 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.667 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.667 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.667 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.667 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.667 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.667 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.667 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.667 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.667 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.667 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.667 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.667 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.668 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.668 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.668 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.668 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.668 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.668 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.668 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.668 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.668 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.668 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.668 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.668 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.668 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.669 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.669 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.669 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.669 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.669 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.669 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.669 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.669 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.669 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.669 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.669 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.670 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.670 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.670 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.670 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.670 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.670 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.671 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.671 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.671 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.671 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.671 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.671 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.671 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.671 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.671 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.671 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.672 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.672 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.672 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.672 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.672 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.672 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.672 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.673 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.673 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.673 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.673 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.673 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.673 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.673 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.673 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.673 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.674 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.674 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.674 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.674 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.674 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.674 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.674 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.674 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.674 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.674 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.674 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.674 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.674 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.674 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.675 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.676 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.677 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.678 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:31.679 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:31.680 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:31 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.148 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.149 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.149 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.149 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.149 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.149 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.149 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.149 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.149 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.149 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.149 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.149 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.150 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.150 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.150 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.150 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.150 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.150 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.150 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.150 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.150 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.150 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.150 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.150 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.150 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.150 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.151 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.151 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.151 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.151 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.151 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.152 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.152 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.152 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.152 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.152 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.152 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.152 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.152 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.152 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.153 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.153 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.153 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.153 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.153 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.153 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.153 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.153 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.153 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.153 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.153 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.153 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.154 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.154 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.154 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.154 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.155 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.155 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.155 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.155 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.155 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.155 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.155 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.155 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.155 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.155 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.155 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.155 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.155 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.156 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.156 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.156 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.156 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.156 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.156 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.156 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.156 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.157 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.157 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.157 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.157 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.157 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.157 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.157 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.157 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.157 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.157 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.157 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.157 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.157 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.158 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.158 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.158 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.158 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.158 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.158 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.158 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.158 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.158 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.158 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.158 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.158 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.158 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.159 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.159 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.159 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.159 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.159 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.159 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.159 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.159 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.159 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.160 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.160 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.160 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.160 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.160 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.161 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.161 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.161 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.162 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.162 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.162 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.162 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.162 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.162 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.162 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.162 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.162 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.162 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.162 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.162 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.162 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.162 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.163 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.163 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.163 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.163 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.163 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.163 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.163 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.163 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.163 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.163 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.163 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.164 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.165 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.165 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.165 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.165 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.165 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.165 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.165 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.165 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.165 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.165 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.165 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.166 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.166 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.166 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.166 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.166 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.166 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.166 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.166 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.166 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.166 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.166 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.166 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.166 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.167 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.167 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.167 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.167 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.167 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.167 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.167 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.167 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.167 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.167 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.167 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.167 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.167 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.168 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.168 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.168 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.168 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.168 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.168 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.168 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.168 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.168 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.168 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.168 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.168 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.168 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.169 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.169 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.169 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.169 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.169 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.169 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.169 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.169 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.169 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.169 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.170 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.170 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.170 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.170 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.170 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.170 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.170 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.170 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.170 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.171 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.171 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.171 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.171 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.171 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.171 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.171 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.171 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.171 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.171 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.171 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.171 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.171 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.171 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.171 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.172 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.172 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.172 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.172 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.172 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.172 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.172 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.172 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.172 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.172 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.172 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.172 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.172 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.172 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.173 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.173 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.173 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.173 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.173 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.173 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.173 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.173 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.173 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.173 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.173 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.173 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.173 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.174 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.174 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.174 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.174 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.174 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.174 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.174 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.174 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.174 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.174 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.174 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.174 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.174 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.175 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.175 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.175 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.175 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.175 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.175 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.175 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.175 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.175 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.175 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.175 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.175 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.175 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.176 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:32.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.177 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:32 smithi161 ceph-mon[35615]: pgmap v164: 97 pgs: 97 active+clean; 132 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.5 MiB/s wr, 1.87k op/s 2024-09-11T22:02:32.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.448 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.448 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.448 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.448 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.448 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.449 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.449 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.449 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.449 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.449 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.449 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.449 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.449 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.449 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.449 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.450 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.450 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.450 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.450 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.450 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.450 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.450 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.450 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.450 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.450 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.450 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.450 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.450 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.452 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.455 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.455 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.455 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.455 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.455 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.455 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.455 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.455 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.455 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.455 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.455 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.455 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.455 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.456 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.456 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.456 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.456 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.456 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.456 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.456 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.456 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.456 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.456 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.456 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.456 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.456 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.457 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.457 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.457 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.457 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.457 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.457 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.457 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.457 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.457 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.457 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.457 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.457 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.457 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.457 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.458 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.458 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.458 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.458 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.458 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.458 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.458 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.458 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.458 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.458 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.458 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.458 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.458 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.458 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.459 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.459 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.459 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.459 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.459 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.459 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.459 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.459 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.459 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.459 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.459 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.459 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.459 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.460 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.460 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.460 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.460 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.460 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.460 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.460 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.460 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.460 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.460 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.460 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.460 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.461 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.461 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.461 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.461 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.461 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.461 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.461 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.461 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.461 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.461 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.461 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.461 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.462 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.462 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.462 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.462 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.462 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.462 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.462 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.462 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.462 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.462 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.462 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.462 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.462 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.462 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.463 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.463 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.463 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.463 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.463 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.463 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.463 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.463 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.463 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.463 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.463 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.463 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.463 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.464 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.464 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.464 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.464 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.464 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.464 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.464 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.464 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.464 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.464 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.464 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.464 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.465 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.465 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.465 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.465 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.465 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.465 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.465 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.465 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.465 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.465 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.465 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.465 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.465 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.466 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.466 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.466 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.466 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.466 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.466 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.466 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.466 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.466 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.466 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.466 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.466 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.466 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.467 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.467 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.467 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.467 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.467 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.467 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.467 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.467 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.467 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.467 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.467 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.467 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.467 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.467 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.468 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.468 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.468 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.468 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.468 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.468 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:32.468 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.468 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.468 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:32 smithi001 ceph-mon[30935]: pgmap v164: 97 pgs: 97 active+clean; 132 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.5 MiB/s wr, 1.87k op/s 2024-09-11T22:02:32.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.552 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.552 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.552 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.552 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.552 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.552 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.552 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.552 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.553 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.553 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.553 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.553 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.553 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.553 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.554 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.554 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.554 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.554 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.554 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.554 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.554 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.554 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.554 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.554 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.554 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.554 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.554 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.555 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.555 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.555 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.555 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.555 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.555 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.555 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.555 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.555 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.555 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.555 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.555 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.555 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.556 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.556 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.556 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.556 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.556 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.556 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.556 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.556 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.556 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.556 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.556 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.556 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.556 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.557 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.557 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.557 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.557 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.557 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.557 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.557 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.557 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.557 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.557 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.557 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.557 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.558 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.558 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.558 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.558 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.558 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.558 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.558 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.558 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.558 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.558 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.558 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.558 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.558 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.558 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.559 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.559 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.559 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.559 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.559 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.559 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.559 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.559 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.559 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.559 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.559 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.559 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.559 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.560 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.560 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.560 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.560 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.560 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.560 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.560 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.560 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.560 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.560 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.560 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.560 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.560 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.561 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.561 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.561 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.561 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.561 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.561 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.561 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.561 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.561 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.561 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.561 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.561 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.562 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.562 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.562 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.562 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.562 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.562 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.562 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.562 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.562 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.562 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.562 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.562 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.562 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.562 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.563 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.563 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.563 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.563 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.563 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.563 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.563 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.563 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.563 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.563 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.563 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.563 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.563 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.564 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.564 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.564 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.564 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.564 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.564 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.564 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.564 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.564 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.564 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.564 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.564 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.564 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.565 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.565 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.565 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.565 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.565 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.565 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.565 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.565 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.565 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.565 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.565 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.565 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.565 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.565 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.566 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.566 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.566 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.566 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.566 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.566 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.566 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.566 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.566 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.566 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.566 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.566 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.566 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.567 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.567 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.567 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.567 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.567 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.567 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.567 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.567 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.567 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.567 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.567 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.567 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.567 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.567 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.568 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.568 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.568 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.568 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.568 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.568 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.568 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.568 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.568 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.568 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.568 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.568 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.568 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.569 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.569 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.569 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.569 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.569 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.569 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.569 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.569 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.569 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.569 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.569 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.569 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.569 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.569 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.570 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.570 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.570 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.570 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.570 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.570 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.570 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.570 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.570 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.570 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.570 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.570 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.570 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.571 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.571 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.571 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.571 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.571 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.571 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.571 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.571 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.571 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.571 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.572 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.572 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.572 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.572 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.572 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.572 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.572 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.572 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.572 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.572 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.572 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:32.572 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.572 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:32.573 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:32 smithi053 ceph-mon[34908]: pgmap v164: 97 pgs: 97 active+clean; 132 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.5 MiB/s wr, 1.87k op/s 2024-09-11T22:02:32.649 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '19475138', quiesce took 22.8 seconds. Will release after: -2.650000000000002 2024-09-11T22:02:32.650 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:02:32.948 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:02:33.348 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:33 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:33.348 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:33 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:02:33.348 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:33 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:33.348 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:33 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:33.349 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:33 smithi001 ceph-mon[30935]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:02:33.349 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:33 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:33.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:33.433+0000 7f74f4a38640 1 -- 172.21.15.1:0/2623465197 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f74f01027a0 msgr2=0x7f74f01067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:33.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:33.433+0000 7f74f4a38640 1 --2- 172.21.15.1:0/2623465197 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f74f01027a0 0x7f74f01067f0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f74e4009900 tx=0x7f74e402a410 comp rx=0 tx=0).stop 2024-09-11T22:02:33.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:33.433+0000 7f74f4a38640 1 -- 172.21.15.1:0/2623465197 shutdown_connections 2024-09-11T22:02:33.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:33.433+0000 7f74f4a38640 1 --2- 172.21.15.1:0/2623465197 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74f0106f20 0x7f74f010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:33.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:33.433+0000 7f74f4a38640 1 --2- 172.21.15.1:0/2623465197 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f74f01027a0 0x7f74f01067f0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:33.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:33.433+0000 7f74f4a38640 1 --2- 172.21.15.1:0/2623465197 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f74f0101df0 0x7f74f01021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:33.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:33.433+0000 7f74f4a38640 1 -- 172.21.15.1:0/2623465197 >> 172.21.15.1:0/2623465197 conn(0x7f74f00fd8f0 msgr2=0x7f74f00ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:33.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:33.433+0000 7f74f4a38640 1 -- 172.21.15.1:0/2623465197 shutdown_connections 2024-09-11T22:02:33.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:33.433+0000 7f74f4a38640 1 -- 172.21.15.1:0/2623465197 wait complete. 2024-09-11T22:02:33.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:33.434+0000 7f74f4a38640 1 Processor -- start 2024-09-11T22:02:33.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:33.434+0000 7f74f4a38640 1 -- start start 2024-09-11T22:02:33.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:33.434+0000 7f74f4a38640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74f0101df0 0x7f74f0198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:33.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:33.434+0000 7f74f4a38640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f74f01027a0 0x7f74f01989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:33.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:33.434+0000 7f74f4a38640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f74f0106f20 0x7f74f019cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:33.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:33.434+0000 7f74f4a38640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f74f0111f80 con 0x7f74f01027a0 2024-09-11T22:02:33.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:33.434+0000 7f74f4a38640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f74f0111e00 con 0x7f74f0101df0 2024-09-11T22:02:33.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:33.434+0000 7f74f4a38640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f74f0112100 con 0x7f74f0106f20 2024-09-11T22:02:33.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:33.434+0000 7f74edd9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f74f01027a0 0x7f74f01989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:33.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:33.435+0000 7f74edd9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f74f01027a0 0x7f74f01989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33828/0 (socket says 172.21.15.1:33828) 2024-09-11T22:02:33.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:33.435+0000 7f74edd9b640 1 -- 172.21.15.1:0/2485506308 learned_addr learned my addr 172.21.15.1:0/2485506308 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:33.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:33.435+0000 7f74eed9d640 1 --2- 172.21.15.1:0/2485506308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f74f0106f20 0x7f74f019cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:33.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:33.435+0000 7f74ee59c640 1 --2- 172.21.15.1:0/2485506308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74f0101df0 0x7f74f0198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:33.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:33.435+0000 7f74edd9b640 1 -- 172.21.15.1:0/2485506308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f74f0106f20 msgr2=0x7f74f019cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:33.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:33.435+0000 7f74edd9b640 1 --2- 172.21.15.1:0/2485506308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f74f0106f20 0x7f74f019cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:33.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:33.435+0000 7f74edd9b640 1 -- 172.21.15.1:0/2485506308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74f0101df0 msgr2=0x7f74f0198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:33.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:33.435+0000 7f74edd9b640 1 --2- 172.21.15.1:0/2485506308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74f0101df0 0x7f74f0198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:33.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:33.435+0000 7f74edd9b640 1 -- 172.21.15.1:0/2485506308 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74f019d410 con 0x7f74f01027a0 2024-09-11T22:02:33.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:33.435+0000 7f74edd9b640 1 --2- 172.21.15.1:0/2485506308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f74f01027a0 0x7f74f01989a0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f74e4006010 tx=0x7f74e40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:33.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:33.435+0000 7f74df7fe640 1 -- 172.21.15.1:0/2485506308 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e403f030 con 0x7f74f01027a0 2024-09-11T22:02:33.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:33.435+0000 7f74df7fe640 1 -- 172.21.15.1:0/2485506308 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74e4003ce0 con 0x7f74f01027a0 2024-09-11T22:02:33.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:33.435+0000 7f74f4a38640 1 -- 172.21.15.1:0/2485506308 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74f019d6a0 con 0x7f74f01027a0 2024-09-11T22:02:33.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:33.435+0000 7f74df7fe640 1 -- 172.21.15.1:0/2485506308 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e4035070 con 0x7f74f01027a0 2024-09-11T22:02:33.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:33.436+0000 7f74f4a38640 1 -- 172.21.15.1:0/2485506308 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74f01a4fe0 con 0x7f74f01027a0 2024-09-11T22:02:33.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:33.436+0000 7f74f4a38640 1 -- 172.21.15.1:0/2485506308 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74f01032d0 con 0x7f74f01027a0 2024-09-11T22:02:33.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:33.439+0000 7f74df7fe640 1 -- 172.21.15.1:0/2485506308 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f74e4042030 con 0x7f74f01027a0 2024-09-11T22:02:33.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:33.439+0000 7f74df7fe640 1 --2- 172.21.15.1:0/2485506308 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f74d80a2cf0 0x7f74d80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:33.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:33.439+0000 7f74ee59c640 1 --2- 172.21.15.1:0/2485506308 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f74d80a2cf0 0x7f74d80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:33.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:33.439+0000 7f74df7fe640 1 -- 172.21.15.1:0/2485506308 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f74e40efba0 con 0x7f74f01027a0 2024-09-11T22:02:33.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:33.440+0000 7f74ee59c640 1 --2- 172.21.15.1:0/2485506308 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f74d80a2cf0 0x7f74d80a51b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f74e0005fd0 tx=0x7f74e000b390 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:33.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:33.443+0000 7f74df7fe640 1 -- 172.21.15.1:0/2485506308 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f74e40bc1f0 con 0x7f74f01027a0 2024-09-11T22:02:33.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:33 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:33.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:33 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:02:33.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:33 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:33.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:33 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:33.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:33 smithi053 ceph-mon[34908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:02:33.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:33 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:33.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:33 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:33.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:33 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:02:33.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:33 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:33.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:33 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:33.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:33 smithi161 ceph-mon[35615]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:02:33.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:33 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:33.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:33.655+0000 7f74f4a38640 1 -- 172.21.15.1:0/2485506308 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f74f01021d0 con 0x7f74f01027a0 2024-09-11T22:02:33.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:33.656+0000 7f74df7fe640 1 -- 172.21.15.1:0/2485506308 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11956 (secure 0 0 0) 0x7f74f01021d0 con 0x7f74f01027a0 2024-09-11T22:02:33.657 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T22:02:33.657 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":34,"btime":"2024-09-11T22:02:32:134518+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/421542832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":421542832},{"type":"v1","addr":"172.21.15.1:6835","nonce":421542832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2836864406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2836864406},{"type":"v1","addr":"172.21.15.1:6837","nonce":2836864406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T21:55:20.791263+0000","modified":"2024-09-11T22:02:31.740162+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24484,"mds_2":24467,"mds_3":24466,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14595":{"gid":14595,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/407834672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":407834672},{"type":"v1","addr":"172.21.15.1:6839","nonce":407834672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/1068827678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":1068827678},{"type":"v1","addr":"172.21.15.1:6841","nonce":1068827678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/872410024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":872410024},{"type":"v1","addr":"172.21.15.161:6833","nonce":872410024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/1648944570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1648944570},{"type":"v1","addr":"172.21.15.53:6833","nonce":1648944570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/2126933993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2126933993},{"type":"v1","addr":"172.21.15.53:6835","nonce":2126933993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.161:6835/828381592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":828381592},{"type":"v1","addr":"172.21.15.161:6835","nonce":828381592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.161:6837/1799318656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1799318656},{"type":"v1","addr":"172.21.15.161:6837","nonce":1799318656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.53:6837/3457131830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3457131830},{"type":"v1","addr":"172.21.15.53:6837","nonce":3457131830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/455365383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":455365383},{"type":"v1","addr":"172.21.15.161:6839","nonce":455365383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.53:6839/1378584447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1378584447},{"type":"v1","addr":"172.21.15.53:6839","nonce":1378584447}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24460,24466,24467,24484,24494]},"id":1}]} 2024-09-11T22:02:33.657 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 34 2024-09-11T22:02:33.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:33.658+0000 7f74f4a38640 1 -- 172.21.15.1:0/2485506308 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f74d80a2cf0 msgr2=0x7f74d80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:33.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:33.658+0000 7f74f4a38640 1 --2- 172.21.15.1:0/2485506308 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f74d80a2cf0 0x7f74d80a51b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f74e0005fd0 tx=0x7f74e000b390 comp rx=0 tx=0).stop 2024-09-11T22:02:33.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:33.658+0000 7f74f4a38640 1 -- 172.21.15.1:0/2485506308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f74f01027a0 msgr2=0x7f74f01989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:33.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:33.658+0000 7f74f4a38640 1 --2- 172.21.15.1:0/2485506308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f74f01027a0 0x7f74f01989a0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f74e4006010 tx=0x7f74e40042f0 comp rx=0 tx=0).stop 2024-09-11T22:02:33.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:33.659+0000 7f74f4a38640 1 -- 172.21.15.1:0/2485506308 shutdown_connections 2024-09-11T22:02:33.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:33.659+0000 7f74f4a38640 1 --2- 172.21.15.1:0/2485506308 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f74d80a2cf0 0x7f74d80a51b0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:33.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:33.659+0000 7f74f4a38640 1 --2- 172.21.15.1:0/2485506308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f74f0106f20 0x7f74f019cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:33.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:33.659+0000 7f74f4a38640 1 --2- 172.21.15.1:0/2485506308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f74f01027a0 0x7f74f01989a0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:33.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:33.659+0000 7f74f4a38640 1 --2- 172.21.15.1:0/2485506308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74f0101df0 0x7f74f0198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:33.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:33.659+0000 7f74f4a38640 1 -- 172.21.15.1:0/2485506308 >> 172.21.15.1:0/2485506308 conn(0x7f74f00fd8f0 msgr2=0x7f74f00fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:33.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:33.659+0000 7f74f4a38640 1 -- 172.21.15.1:0/2485506308 shutdown_connections 2024-09-11T22:02:33.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:33.659+0000 7f74f4a38640 1 -- 172.21.15.1:0/2485506308 wait complete. 2024-09-11T22:02:34.404 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:34 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:34.404 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:34 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/2485506308' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:02:34.404 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:34 smithi001 ceph-mon[30935]: pgmap v165: 97 pgs: 97 active+clean; 132 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.5 MiB/s wr, 1.87k op/s 2024-09-11T22:02:34.431 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db --set-id 19475138 --release --await' 2024-09-11T22:02:34.432 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db --set-id 19475138 --release --await 2024-09-11T22:02:34.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:34 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:34.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:34 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/2485506308' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:02:34.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:34 smithi053 ceph-mon[34908]: pgmap v165: 97 pgs: 97 active+clean; 132 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.5 MiB/s wr, 1.87k op/s 2024-09-11T22:02:34.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:34 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:02:34.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:34 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/2485506308' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:02:34.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:34 smithi161 ceph-mon[35615]: pgmap v165: 97 pgs: 97 active+clean; 132 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.5 MiB/s wr, 1.87k op/s 2024-09-11T22:02:34.692 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:02:35.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.172+0000 7f0af5932640 1 -- 172.21.15.1:0/3670849455 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0af0103780 msgr2=0x7f0af0107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:35.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.172+0000 7f0af5932640 1 --2- 172.21.15.1:0/3670849455 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0af0103780 0x7f0af0107bc0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f0ae4009930 tx=0x7f0ae402a420 comp rx=0 tx=0).stop 2024-09-11T22:02:35.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.172+0000 7f0af5932640 1 -- 172.21.15.1:0/3670849455 shutdown_connections 2024-09-11T22:02:35.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.172+0000 7f0af5932640 1 --2- 172.21.15.1:0/3670849455 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0af0108300 0x7f0af010ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:35.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.172+0000 7f0af5932640 1 --2- 172.21.15.1:0/3670849455 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0af0103780 0x7f0af0107bc0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:35.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.172+0000 7f0af5932640 1 --2- 172.21.15.1:0/3670849455 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0af0102dd0 0x7f0af01031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:35.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.172+0000 7f0af5932640 1 -- 172.21.15.1:0/3670849455 >> 172.21.15.1:0/3670849455 conn(0x7f0af00fc7d0 msgr2=0x7f0af00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:35.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.172+0000 7f0af5932640 1 -- 172.21.15.1:0/3670849455 shutdown_connections 2024-09-11T22:02:35.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.172+0000 7f0af5932640 1 -- 172.21.15.1:0/3670849455 wait complete. 2024-09-11T22:02:35.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.172+0000 7f0af5932640 1 Processor -- start 2024-09-11T22:02:35.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.173+0000 7f0af5932640 1 -- start start 2024-09-11T22:02:35.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.173+0000 7f0af5932640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0af0102dd0 0x7f0af0100a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:35.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.173+0000 7f0af5932640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0af0103780 0x7f0af0100f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:35.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.173+0000 7f0af5932640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0af0108300 0x7f0af01014b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:35.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.173+0000 7f0af5932640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0af01120c0 con 0x7f0af0102dd0 2024-09-11T22:02:35.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.173+0000 7f0af5932640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0af0111f40 con 0x7f0af0108300 2024-09-11T22:02:35.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.173+0000 7f0af5932640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0af0112240 con 0x7f0af0103780 2024-09-11T22:02:35.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.173+0000 7f0aeeffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0af0102dd0 0x7f0af0100a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:35.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.173+0000 7f0aeeffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0af0102dd0 0x7f0af0100a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33844/0 (socket says 172.21.15.1:33844) 2024-09-11T22:02:35.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.173+0000 7f0aeeffd640 1 -- 172.21.15.1:0/1949745848 learned_addr learned my addr 172.21.15.1:0/1949745848 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:35.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.173+0000 7f0aee7fc640 1 --2- 172.21.15.1:0/1949745848 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0af0103780 0x7f0af0100f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:35.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.173+0000 7f0aef7fe640 1 --2- 172.21.15.1:0/1949745848 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0af0108300 0x7f0af01014b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:35.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.173+0000 7f0aeeffd640 1 -- 172.21.15.1:0/1949745848 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0af0103780 msgr2=0x7f0af0100f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:35.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.173+0000 7f0aeeffd640 1 --2- 172.21.15.1:0/1949745848 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0af0103780 0x7f0af0100f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:35.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.173+0000 7f0aeeffd640 1 -- 172.21.15.1:0/1949745848 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0af0108300 msgr2=0x7f0af01014b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:35.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.173+0000 7f0aeeffd640 1 --2- 172.21.15.1:0/1949745848 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0af0108300 0x7f0af01014b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:35.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.173+0000 7f0aeeffd640 1 -- 172.21.15.1:0/1949745848 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0af01a6b00 con 0x7f0af0102dd0 2024-09-11T22:02:35.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.174+0000 7f0aeeffd640 1 --2- 172.21.15.1:0/1949745848 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0af0102dd0 0x7f0af0100a30 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f0ae000d770 tx=0x7f0ae000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:35.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.174+0000 7f0af4930640 1 -- 172.21.15.1:0/1949745848 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae0017070 con 0x7f0af0102dd0 2024-09-11T22:02:35.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.174+0000 7f0af4930640 1 -- 172.21.15.1:0/1949745848 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ae0005500 con 0x7f0af0102dd0 2024-09-11T22:02:35.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.174+0000 7f0af5932640 1 -- 172.21.15.1:0/1949745848 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0af01a6df0 con 0x7f0af0102dd0 2024-09-11T22:02:35.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.174+0000 7f0af4930640 1 -- 172.21.15.1:0/1949745848 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae0004e00 con 0x7f0af0102dd0 2024-09-11T22:02:35.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.174+0000 7f0af5932640 1 -- 172.21.15.1:0/1949745848 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0af01a7250 con 0x7f0af0102dd0 2024-09-11T22:02:35.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.177+0000 7f0af4930640 1 -- 172.21.15.1:0/1949745848 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f0ae0004fa0 con 0x7f0af0102dd0 2024-09-11T22:02:35.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.177+0000 7f0af7d0b740 1 Processor -- start 2024-09-11T22:02:35.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.177+0000 7f0af7d0b740 1 -- start start 2024-09-11T22:02:35.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.177+0000 7f0af7d0b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ca56b1060 0x560ca56b1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:35.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.177+0000 7f0af7d0b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ca56b1980 0x560ca56bbd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:35.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.177+0000 7f0af7d0b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ca56bc240 0x560ca56be630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:35.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.177+0000 7f0af7d0b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x560ca5616620 con 0x560ca56b1980 2024-09-11T22:02:35.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.177+0000 7f0af7d0b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x560ca55bde50 con 0x560ca56b1060 2024-09-11T22:02:35.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.177+0000 7f0af7d0b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x560ca56125e0 con 0x560ca56bc240 2024-09-11T22:02:35.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.177+0000 7f0aeeffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ca56b1980 0x560ca56bbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:35.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.177+0000 7f0aeeffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ca56b1980 0x560ca56bbd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33852/0 (socket says 172.21.15.1:33852) 2024-09-11T22:02:35.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.177+0000 7f0aeeffd640 1 -- 172.21.15.1:0/3523112053 learned_addr learned my addr 172.21.15.1:0/3523112053 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:35.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.177+0000 7f0aef7fe640 1 --2- 172.21.15.1:0/3523112053 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ca56b1060 0x560ca56b1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:35.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.177+0000 7f0aee7fc640 1 --2- 172.21.15.1:0/3523112053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ca56bc240 0x560ca56be630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:35.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.178+0000 7f0af4930640 1 --2- 172.21.15.1:0/1949745848 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f0ad80a2cf0 0x7f0ad80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:35.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.178+0000 7f0aeeffd640 1 -- 172.21.15.1:0/3523112053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ca56bc240 msgr2=0x560ca56be630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:35.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.178+0000 7f0aeeffd640 1 --2- 172.21.15.1:0/3523112053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ca56bc240 0x560ca56be630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:35.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.178+0000 7f0aeeffd640 1 -- 172.21.15.1:0/3523112053 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ca56b1060 msgr2=0x560ca56b1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:35.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.178+0000 7f0aeeffd640 1 --2- 172.21.15.1:0/3523112053 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ca56b1060 0x560ca56b1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:35.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.178+0000 7f0aeeffd640 1 -- 172.21.15.1:0/3523112053 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ca55aed40 con 0x560ca56b1980 2024-09-11T22:02:35.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.178+0000 7f0aef7fe640 1 --2- 172.21.15.1:0/1949745848 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f0ad80a2cf0 0x7f0ad80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:35.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.178+0000 7f0af4930640 1 -- 172.21.15.1:0/1949745848 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f0ae00cef70 con 0x7f0af0102dd0 2024-09-11T22:02:35.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.178+0000 7f0aeeffd640 1 --2- 172.21.15.1:0/3523112053 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ca56b1980 0x560ca56bbd00 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f0ae00cc5f0 tx=0x7f0ae00c5fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a26ef55fbba8d56 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:35.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.178+0000 7f0ad5ffb640 1 -- 172.21.15.1:0/3523112053 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae0020070 con 0x560ca56b1980 2024-09-11T22:02:35.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.178+0000 7f0aef7fe640 1 --2- 172.21.15.1:0/1949745848 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f0ad80a2cf0 0x7f0ad80a51b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8005fd0 tx=0x7f0ae8004a40 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:35.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.178+0000 7f0ad5ffb640 1 -- 172.21.15.1:0/3523112053 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ae00046c0 con 0x560ca56b1980 2024-09-11T22:02:35.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.178+0000 7f0ad5ffb640 1 -- 172.21.15.1:0/3523112053 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae00056b0 con 0x560ca56b1980 2024-09-11T22:02:35.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.179+0000 7f0af7d0b740 1 -- 172.21.15.1:0/3523112053 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ca56b1980 msgr2=0x560ca56bbd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:35.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.179+0000 7f0af7d0b740 1 --2- 172.21.15.1:0/3523112053 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ca56b1980 0x560ca56bbd00 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f0ae00cc5f0 tx=0x7f0ae00c5fe0 comp rx=0 tx=0).stop 2024-09-11T22:02:35.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.179+0000 7f0af7d0b740 1 -- 172.21.15.1:0/3523112053 shutdown_connections 2024-09-11T22:02:35.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.179+0000 7f0af7d0b740 1 --2- 172.21.15.1:0/3523112053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ca56bc240 0x560ca56be630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:35.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.179+0000 7f0af7d0b740 1 --2- 172.21.15.1:0/3523112053 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ca56b1980 0x560ca56bbd00 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:35.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.179+0000 7f0af7d0b740 1 --2- 172.21.15.1:0/3523112053 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ca56b1060 0x560ca56b1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:35.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.179+0000 7f0af7d0b740 1 -- 172.21.15.1:0/3523112053 >> 172.21.15.1:0/3523112053 conn(0x560ca56ad5f0 msgr2=0x560ca56ad9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:35.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.179+0000 7f0af7d0b740 1 -- 172.21.15.1:0/3523112053 shutdown_connections 2024-09-11T22:02:35.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.179+0000 7f0af7d0b740 1 -- 172.21.15.1:0/3523112053 wait complete. 2024-09-11T22:02:35.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.179+0000 7f0af7d0b740 1 -- start start 2024-09-11T22:02:35.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.180+0000 7f0af7d0b740 1 Processor -- start 2024-09-11T22:02:35.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.180+0000 7f0af7d0b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ca56b1060 0x560ca56b90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:35.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.180+0000 7f0af7d0b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ca56b1980 0x560ca56b9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:35.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.180+0000 7f0af7d0b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ca56bc240 0x560ca56b9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:35.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.180+0000 7f0af7d0b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x560ca56bedb0 con 0x560ca56bc240 2024-09-11T22:02:35.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.180+0000 7f0af7d0b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x560ca559ef10 con 0x560ca56b1980 2024-09-11T22:02:35.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.180+0000 7f0af7d0b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x560ca56bef30 con 0x560ca56b1060 2024-09-11T22:02:35.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.180+0000 7f0af7d0b740 10 client.0 _ll_register_callbacks cb 0x560ca55c2db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:02:35.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.180+0000 7f0aef7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ca56bc240 0x560ca56b9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:35.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.180+0000 7f0aef7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ca56bc240 0x560ca56b9b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33862/0 (socket says 172.21.15.1:33862) 2024-09-11T22:02:35.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.180+0000 7f0aef7fe640 1 -- 172.21.15.1:0/3536975856 learned_addr learned my addr 172.21.15.1:0/3536975856 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:35.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.180+0000 7f0aeeffd640 1 --2- 172.21.15.1:0/3536975856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ca56b1060 0x560ca56b90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:35.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.180+0000 7f0aee7fc640 1 --2- 172.21.15.1:0/3536975856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ca56b1980 0x560ca56b9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:35.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.180+0000 7f0aef7fe640 1 -- 172.21.15.1:0/3536975856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ca56b1060 msgr2=0x560ca56b90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:35.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.180+0000 7f0aef7fe640 1 --2- 172.21.15.1:0/3536975856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ca56b1060 0x560ca56b90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:35.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.180+0000 7f0aef7fe640 1 -- 172.21.15.1:0/3536975856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ca56b1980 msgr2=0x560ca56b9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:35.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.180+0000 7f0aef7fe640 1 --2- 172.21.15.1:0/3536975856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ca56b1980 0x560ca56b9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:35.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.180+0000 7f0aef7fe640 1 -- 172.21.15.1:0/3536975856 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ca5325580 con 0x560ca56bc240 2024-09-11T22:02:35.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.180+0000 7f0aee7fc640 1 --2- 172.21.15.1:0/3536975856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ca56b1980 0x560ca56b9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:02:35.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.181+0000 7f0aef7fe640 1 --2- 172.21.15.1:0/3536975856 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ca56bc240 0x560ca56b9b60 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f0ae80116c0 tx=0x7f0ae8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b851943f0697450 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:35.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.181+0000 7f0aceffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:02:35.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.181+0000 7f0aceffd640 1 -- 172.21.15.1:0/3536975856 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae8013730 con 0x560ca56bc240 2024-09-11T22:02:35.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.181+0000 7f0aceffd640 1 -- 172.21.15.1:0/3536975856 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f0ae80138d0 con 0x560ca56bc240 2024-09-11T22:02:35.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.181+0000 7f0af7d0b740 1 -- 172.21.15.1:0/3536975856 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560ca5623b90 con 0x560ca56bc240 2024-09-11T22:02:35.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.181+0000 7f0aceffd640 1 -- 172.21.15.1:0/3536975856 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0ae802f050 con 0x560ca56bc240 2024-09-11T22:02:35.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.181+0000 7f0af7d0b740 10 client.16518 fetch_fsmap learned FSMap version 34 2024-09-11T22:02:35.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.181+0000 7f0af7d0b740 1 -- 172.21.15.1:0/3536975856 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x560ca53ef960 con 0x560ca56bc240 2024-09-11T22:02:35.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.181+0000 7f0aceffd640 1 -- 172.21.15.1:0/3536975856 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x560ca5623b90 con 0x560ca56bc240 2024-09-11T22:02:35.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.181+0000 7f0aceffd640 1 -- 172.21.15.1:0/3536975856 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7f0ae800f3f0 con 0x560ca56bc240 2024-09-11T22:02:35.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.181+0000 7f0af7d0b740 10 client.16518 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:02:35.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.181+0000 7f0af7d0b740 10 client.16518 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-11T22:02:35.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.182+0000 7f0af7d0b740 20 client.16518 populate_metadata read hostname 'smithi001' 2024-09-11T22:02:35.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.182+0000 7f0af7d0b740 1 --2- 172.21.15.1:0/3536975856 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x560ca56cbe80 0x560ca56ce2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:35.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.182+0000 7f0af7d0b740 4 client.16518 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:02:35.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.182+0000 7f0af7d0b740 1 -- 172.21.15.1:0/3536975856 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ca561e0a0 con 0x560ca56cbe80 2024-09-11T22:02:35.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.182+0000 7f0aeeffd640 1 --2- 172.21.15.1:0/3536975856 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x560ca56cbe80 0x560ca56ce2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:35.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.182+0000 7f0aeeffd640 1 --2- 172.21.15.1:0/3536975856 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x560ca56cbe80 0x560ca56ce2e0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a2cf6cf734ddfc2 server_cookie=699bd44b44af7340 in_seq=0 out_seq=0 2024-09-11T22:02:35.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.182+0000 7f0aceffd640 10 client.16518 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:02:35.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.183+0000 7f0aceffd640 1 -- 172.21.15.1:0/3536975856 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560ca561e0a0 con 0x560ca56cbe80 2024-09-11T22:02:35.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.183+0000 7f0aceffd640 10 client.16518 handle_command_reply: tid=0 2024-09-11T22:02:35.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.183+0000 7f0af7d0b740 1 client.16518 shutdown 2024-09-11T22:02:35.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.183+0000 7f0af7d0b740 1 -- 172.21.15.1:0/3536975856 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ca56bc240 msgr2=0x560ca56b9b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:35.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.183+0000 7f0af7d0b740 1 --2- 172.21.15.1:0/3536975856 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ca56bc240 0x560ca56b9b60 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f0ae80116c0 tx=0x7f0ae8004ab0 comp rx=0 tx=0).stop 2024-09-11T22:02:35.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.183+0000 7f0af7d0b740 1 -- 172.21.15.1:0/3536975856 shutdown_connections 2024-09-11T22:02:35.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.183+0000 7f0af7d0b740 1 --2- 172.21.15.1:0/3536975856 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x560ca56cbe80 0x560ca56ce2e0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:35.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.183+0000 7f0af7d0b740 1 --2- 172.21.15.1:0/3536975856 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ca56bc240 0x560ca56b9b60 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:35.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.183+0000 7f0af7d0b740 1 --2- 172.21.15.1:0/3536975856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ca56b1980 0x560ca56b9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:35.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.183+0000 7f0af7d0b740 1 --2- 172.21.15.1:0/3536975856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ca56b1060 0x560ca56b90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:35.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.183+0000 7f0aceffd640 0 client.16518 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:02:35.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.183+0000 7f0af7d0b740 1 -- 172.21.15.1:0/3536975856 >> 172.21.15.1:0/3536975856 conn(0x560ca56ad5f0 msgr2=0x560ca56b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:35.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.183+0000 7f0af7d0b740 1 -- 172.21.15.1:0/3536975856 shutdown_connections 2024-09-11T22:02:35.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.183+0000 7f0af7d0b740 1 -- 172.21.15.1:0/3536975856 wait complete. 2024-09-11T22:02:35.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.183+0000 7f0af7d0b740 20 client.16518 trim_cache size 0 max 16384 2024-09-11T22:02:35.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.201+0000 7f0af7d0b740 1 Processor -- start 2024-09-11T22:02:35.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.201+0000 7f0af7d0b740 1 -- start start 2024-09-11T22:02:35.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.201+0000 7f0af7d0b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ca56b1060 0x560ca56d3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:35.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.201+0000 7f0af7d0b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ca56b1980 0x560ca56d40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:35.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.201+0000 7f0af7d0b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ca56bc240 0x560ca56b6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:35.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.201+0000 7f0af7d0b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0af01afc20 con 0x560ca56bc240 2024-09-11T22:02:35.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.201+0000 7f0af7d0b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0af01b1820 con 0x560ca56b1060 2024-09-11T22:02:35.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.201+0000 7f0af7d0b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0af01b02f0 con 0x560ca56b1980 2024-09-11T22:02:35.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.201+0000 7f0aef7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ca56bc240 0x560ca56b6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:35.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.201+0000 7f0aef7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ca56bc240 0x560ca56b6340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33872/0 (socket says 172.21.15.1:33872) 2024-09-11T22:02:35.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.201+0000 7f0aef7fe640 1 -- 172.21.15.1:0/4294439669 learned_addr learned my addr 172.21.15.1:0/4294439669 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:35.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.201+0000 7f0aeeffd640 1 --2- 172.21.15.1:0/4294439669 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ca56b1060 0x560ca56d3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:35.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.201+0000 7f0aee7fc640 1 --2- 172.21.15.1:0/4294439669 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ca56b1980 0x560ca56d40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:35.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.201+0000 7f0aef7fe640 1 -- 172.21.15.1:0/4294439669 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ca56b1980 msgr2=0x560ca56d40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:35.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.201+0000 7f0aef7fe640 1 --2- 172.21.15.1:0/4294439669 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ca56b1980 0x560ca56d40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:35.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.201+0000 7f0aef7fe640 1 -- 172.21.15.1:0/4294439669 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ca56b1060 msgr2=0x560ca56d3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:35.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.201+0000 7f0aef7fe640 1 --2- 172.21.15.1:0/4294439669 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ca56b1060 0x560ca56d3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:35.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.201+0000 7f0aef7fe640 1 -- 172.21.15.1:0/4294439669 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ca56ad420 con 0x560ca56bc240 2024-09-11T22:02:35.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.201+0000 7f0aee7fc640 1 --2- 172.21.15.1:0/4294439669 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ca56b1980 0x560ca56d40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:02:35.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.202+0000 7f0aef7fe640 1 --2- 172.21.15.1:0/4294439669 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ca56bc240 0x560ca56b6340 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8000f80 tx=0x7f0ae8012fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fbc3914fda0ae76 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:35.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.202+0000 7f0ad57fa640 1 -- 172.21.15.1:0/4294439669 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae8039070 con 0x560ca56bc240 2024-09-11T22:02:35.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.202+0000 7f0ad57fa640 1 -- 172.21.15.1:0/4294439669 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ae8024020 con 0x560ca56bc240 2024-09-11T22:02:35.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.202+0000 7f0ad57fa640 1 -- 172.21.15.1:0/4294439669 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae8023020 con 0x560ca56bc240 2024-09-11T22:02:35.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.202+0000 7f0af7d0b740 1 -- 172.21.15.1:0/4294439669 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ca56bc240 msgr2=0x560ca56b6340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:35.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.202+0000 7f0af7d0b740 1 --2- 172.21.15.1:0/4294439669 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ca56bc240 0x560ca56b6340 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8000f80 tx=0x7f0ae8012fe0 comp rx=0 tx=0).stop 2024-09-11T22:02:35.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.203+0000 7f0af7d0b740 1 -- 172.21.15.1:0/4294439669 shutdown_connections 2024-09-11T22:02:35.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.203+0000 7f0af7d0b740 1 --2- 172.21.15.1:0/4294439669 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ca56bc240 0x560ca56b6340 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:35.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.203+0000 7f0af7d0b740 1 --2- 172.21.15.1:0/4294439669 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ca56b1980 0x560ca56d40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:35.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.203+0000 7f0af7d0b740 1 --2- 172.21.15.1:0/4294439669 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ca56b1060 0x560ca56d3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:35.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.203+0000 7f0af7d0b740 1 -- 172.21.15.1:0/4294439669 >> 172.21.15.1:0/4294439669 conn(0x560ca56ad5f0 msgr2=0x560ca56b79b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:35.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.203+0000 7f0af7d0b740 1 -- 172.21.15.1:0/4294439669 shutdown_connections 2024-09-11T22:02:35.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.203+0000 7f0af7d0b740 1 -- 172.21.15.1:0/4294439669 wait complete. 2024-09-11T22:02:35.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.203+0000 7f0af7d0b740 1 -- start start 2024-09-11T22:02:35.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.203+0000 7f0af7d0b740 1 Processor -- start 2024-09-11T22:02:35.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.203+0000 7f0af7d0b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ca56b1060 0x560ca56c4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:35.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.203+0000 7f0af7d0b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ca56b1980 0x560ca56c0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:35.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.203+0000 7f0af7d0b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ca56bc240 0x560ca56c11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:35.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.203+0000 7f0af7d0b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0af01b1090 con 0x560ca56b1060 2024-09-11T22:02:35.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.203+0000 7f0af7d0b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0af01b09c0 con 0x560ca56bc240 2024-09-11T22:02:35.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.203+0000 7f0af7d0b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0af01adc00 con 0x560ca56b1980 2024-09-11T22:02:35.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.204+0000 7f0af7d0b740 10 client.0 _ll_register_callbacks cb 0x7f0ae8004790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:02:35.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.204+0000 7f0aeeffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ca56b1060 0x560ca56c4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:35.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.204+0000 7f0aeeffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ca56b1060 0x560ca56c4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33888/0 (socket says 172.21.15.1:33888) 2024-09-11T22:02:35.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.204+0000 7f0aeeffd640 1 -- 172.21.15.1:0/3644353345 learned_addr learned my addr 172.21.15.1:0/3644353345 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:35.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.204+0000 7f0aee7fc640 1 --2- 172.21.15.1:0/3644353345 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ca56b1980 0x560ca56c0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:35.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.204+0000 7f0aef7fe640 1 --2- 172.21.15.1:0/3644353345 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ca56bc240 0x560ca56c11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:35.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.204+0000 7f0aeeffd640 1 -- 172.21.15.1:0/3644353345 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ca56b1980 msgr2=0x560ca56c0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:35.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.204+0000 7f0aeeffd640 1 --2- 172.21.15.1:0/3644353345 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ca56b1980 0x560ca56c0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:35.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.204+0000 7f0aeeffd640 1 -- 172.21.15.1:0/3644353345 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ca56bc240 msgr2=0x560ca56c11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:35.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.204+0000 7f0aeeffd640 1 --2- 172.21.15.1:0/3644353345 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ca56bc240 0x560ca56c11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:35.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.204+0000 7f0aeeffd640 1 -- 172.21.15.1:0/3644353345 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ca56c19c0 con 0x560ca56b1060 2024-09-11T22:02:35.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.204+0000 7f0aef7fe640 1 --2- 172.21.15.1:0/3644353345 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ca56bc240 0x560ca56c11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:02:35.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.204+0000 7f0aeeffd640 1 --2- 172.21.15.1:0/3644353345 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ca56b1060 0x560ca56c4940 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f0ae00ccc80 tx=0x7f0ae0005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f56a8674f39c8ab1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:35.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.204+0000 7f0aceffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:02:35.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.204+0000 7f0aceffd640 1 -- 172.21.15.1:0/3644353345 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae0020040 con 0x560ca56b1060 2024-09-11T22:02:35.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.204+0000 7f0aceffd640 1 -- 172.21.15.1:0/3644353345 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f0ae0005290 con 0x560ca56b1060 2024-09-11T22:02:35.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.204+0000 7f0af7d0b740 1 -- 172.21.15.1:0/3644353345 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ae8024020 con 0x560ca56b1060 2024-09-11T22:02:35.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.205+0000 7f0aceffd640 1 -- 172.21.15.1:0/3644353345 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0ae00b4050 con 0x560ca56b1060 2024-09-11T22:02:35.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.205+0000 7f0af7d0b740 10 client.16524 fetch_fsmap learned FSMap version 34 2024-09-11T22:02:35.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.205+0000 7f0af7d0b740 1 -- 172.21.15.1:0/3644353345 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x560ca5737c60 con 0x560ca56b1060 2024-09-11T22:02:35.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.205+0000 7f0aceffd640 1 -- 172.21.15.1:0/3644353345 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ae8024020 con 0x560ca56b1060 2024-09-11T22:02:35.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.205+0000 7f0aceffd640 1 -- 172.21.15.1:0/3644353345 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7f0ae00cc900 con 0x560ca56b1060 2024-09-11T22:02:35.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.205+0000 7f0af7d0b740 10 client.16524 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:02:35.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.205+0000 7f0af7d0b740 10 client.16524 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-11T22:02:35.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.205+0000 7f0af7d0b740 20 client.16524 populate_metadata read hostname 'smithi001' 2024-09-11T22:02:35.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.205+0000 7f0af7d0b740 1 --2- 172.21.15.1:0/3644353345 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x560ca57380b0 0x560ca573a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:35.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.205+0000 7f0af7d0b740 4 client.16524 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "19475138", "release": true, "await": true}] 2024-09-11T22:02:35.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.205+0000 7f0af7d0b740 1 -- 172.21.15.1:0/3644353345 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "quiesce db", "set_id": "19475138", "release": true, "await": true}) -- 0x560ca559a5a0 con 0x560ca57380b0 2024-09-11T22:02:35.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.205+0000 7f0aee7fc640 1 --2- 172.21.15.1:0/3644353345 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x560ca57380b0 0x560ca573a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:35.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.206+0000 7f0aee7fc640 1 --2- 172.21.15.1:0/3644353345 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x560ca57380b0 0x560ca573a4a0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b20bab15700925a server_cookie=5aedf24f8bc81d59 in_seq=0 out_seq=0 2024-09-11T22:02:35.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:35.206+0000 7f0aceffd640 10 client.16524 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:02:35.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:35 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:02:36.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:35 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:02:36.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:35 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:02:36.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:36.127+0000 7f0aceffd640 1 -- 172.21.15.1:0/3644353345 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x560ca559a5a0 con 0x560ca57380b0 2024-09-11T22:02:36.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:36.127+0000 7f0aceffd640 10 client.16524 handle_command_reply: tid=0 2024-09-11T22:02:36.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:36.127+0000 7f0af7d0b740 1 client.16524 shutdown 2024-09-11T22:02:36.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:36.128+0000 7f0af7d0b740 1 -- 172.21.15.1:0/3644353345 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ca56b1060 msgr2=0x560ca56c4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:36.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:36.128+0000 7f0af7d0b740 1 --2- 172.21.15.1:0/3644353345 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ca56b1060 0x560ca56c4940 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f0ae00ccc80 tx=0x7f0ae0005d30 comp rx=0 tx=0).stop 2024-09-11T22:02:36.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:36.128+0000 7f0af7d0b740 1 -- 172.21.15.1:0/3644353345 shutdown_connections 2024-09-11T22:02:36.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:36.128+0000 7f0af7d0b740 1 --2- 172.21.15.1:0/3644353345 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x560ca57380b0 0x560ca573a4a0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:36.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:36.128+0000 7f0af7d0b740 1 --2- 172.21.15.1:0/3644353345 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560ca56bc240 0x560ca56c11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:36.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:36.128+0000 7f0aceffd640 0 client.16524 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:02:36.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:36.128+0000 7f0af7d0b740 1 --2- 172.21.15.1:0/3644353345 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560ca56b1980 0x560ca56c0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:36.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:36.128+0000 7f0af7d0b740 1 --2- 172.21.15.1:0/3644353345 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560ca56b1060 0x560ca56c4940 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:36.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:36.128+0000 7f0af7d0b740 1 -- 172.21.15.1:0/3644353345 >> 172.21.15.1:0/3644353345 conn(0x560ca56ad5f0 msgr2=0x560ca56abf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:36.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:36.128+0000 7f0af7d0b740 1 -- 172.21.15.1:0/3644353345 shutdown_connections 2024-09-11T22:02:36.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:36.128+0000 7f0af7d0b740 1 -- 172.21.15.1:0/3644353345 wait complete. 2024-09-11T22:02:36.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:36.128+0000 7f0af7d0b740 20 client.16524 trim_cache size 0 max 16384 2024-09-11T22:02:36.131 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-11T22:02:36.131 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 34, 2024-09-11T22:02:36.131 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24494, 2024-09-11T22:02:36.131 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 16, 2024-09-11T22:02:36.131 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-09-11T22:02:36.131 INFO:teuthology.orchestra.run.smithi001.stdout: "19475138": { 2024-09-11T22:02:36.131 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 16, 2024-09-11T22:02:36.131 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-09-11T22:02:36.131 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-11T22:02:36.132 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RELEASED", 2024-09-11T22:02:36.132 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-11T22:02:36.132 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:02:36.132 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 90.0, 2024-09-11T22:02:36.132 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 140.2, 2024-09-11T22:02:36.132 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-09-11T22:02:36.132 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-09-11T22:02:36.132 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-09-11T22:02:36.132 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-11T22:02:36.132 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RELEASED", 2024-09-11T22:02:36.132 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-11T22:02:36.133 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:02:36.133 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:02:36.133 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:02:36.133 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:02:36.133 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:02:36.133 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-11T22:02:36.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:36.129+0000 7f0af5932640 1 -- 172.21.15.1:0/1949745848 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f0ad80a2cf0 msgr2=0x7f0ad80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:36.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:36.129+0000 7f0af5932640 1 --2- 172.21.15.1:0/1949745848 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f0ad80a2cf0 0x7f0ad80a51b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8005fd0 tx=0x7f0ae8004a40 comp rx=0 tx=0).stop 2024-09-11T22:02:36.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:36.129+0000 7f0af5932640 1 -- 172.21.15.1:0/1949745848 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0af0102dd0 msgr2=0x7f0af0100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:36.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:36.129+0000 7f0af5932640 1 --2- 172.21.15.1:0/1949745848 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0af0102dd0 0x7f0af0100a30 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f0ae000d770 tx=0x7f0ae000dc30 comp rx=0 tx=0).stop 2024-09-11T22:02:36.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:36.129+0000 7f0af5932640 1 -- 172.21.15.1:0/1949745848 shutdown_connections 2024-09-11T22:02:36.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:36.129+0000 7f0af5932640 1 --2- 172.21.15.1:0/1949745848 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f0ad80a2cf0 0x7f0ad80a51b0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:36.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:36.129+0000 7f0af5932640 1 --2- 172.21.15.1:0/1949745848 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0af0108300 0x7f0af01014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:36.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:36.129+0000 7f0af5932640 1 --2- 172.21.15.1:0/1949745848 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0af0103780 0x7f0af0100f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:36.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:36.129+0000 7f0af5932640 1 --2- 172.21.15.1:0/1949745848 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0af0102dd0 0x7f0af0100a30 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:36.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:36.129+0000 7f0af5932640 1 -- 172.21.15.1:0/1949745848 >> 172.21.15.1:0/1949745848 conn(0x7f0af00fc7d0 msgr2=0x7f0af0105e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:36.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:36.129+0000 7f0af5932640 1 -- 172.21.15.1:0/1949745848 shutdown_connections 2024-09-11T22:02:36.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:36.129+0000 7f0af5932640 1 -- 172.21.15.1:0/1949745848 wait complete. 2024-09-11T22:02:36.931 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:36 smithi001 ceph-mon[30935]: pgmap v166: 97 pgs: 97 active+clean; 118 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.2 MiB/s wr, 2.80k op/s 2024-09-11T22:02:36.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:36 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:02:36.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:36 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:02:36.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:36 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T22:02:36.965 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '19475138', total seconds elapsed: 27.1 2024-09-11T22:02:36.965 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.15 2024-09-11T22:02:36.965 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:02:37.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:36 smithi053 ceph-mon[34908]: pgmap v166: 97 pgs: 97 active+clean; 118 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.2 MiB/s wr, 2.80k op/s 2024-09-11T22:02:37.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:36 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:02:37.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:36 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:02:37.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:36 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T22:02:37.091 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:36 smithi161 ceph-mon[35615]: pgmap v166: 97 pgs: 97 active+clean; 118 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.2 MiB/s wr, 2.80k op/s 2024-09-11T22:02:37.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:36 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:02:37.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:36 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:02:37.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:36 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T22:02:37.252 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:02:37.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:37.901+0000 7f53ba7f0640 1 -- 172.21.15.1:0/3504074939 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f53b4100e20 msgr2=0x7f53b4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:37.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:37.901+0000 7f53ba7f0640 1 --2- 172.21.15.1:0/3504074939 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f53b4100e20 0x7f53b4101200 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f53a4009900 tx=0x7f53a402a480 comp rx=0 tx=0).stop 2024-09-11T22:02:37.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:37.901+0000 7f53ba7f0640 1 -- 172.21.15.1:0/3504074939 shutdown_connections 2024-09-11T22:02:37.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:37.901+0000 7f53ba7f0640 1 --2- 172.21.15.1:0/3504074939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f53b4102ef0 0x7f53b410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:37.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:37.901+0000 7f53ba7f0640 1 --2- 172.21.15.1:0/3504074939 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f53b4102570 0x7f53b41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:37.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:37.901+0000 7f53ba7f0640 1 --2- 172.21.15.1:0/3504074939 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f53b4100e20 0x7f53b4101200 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:37.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:37.901+0000 7f53ba7f0640 1 -- 172.21.15.1:0/3504074939 >> 172.21.15.1:0/3504074939 conn(0x7f53b40fc7d0 msgr2=0x7f53b40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:37.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:37.902+0000 7f53ba7f0640 1 -- 172.21.15.1:0/3504074939 shutdown_connections 2024-09-11T22:02:37.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:37.902+0000 7f53ba7f0640 1 -- 172.21.15.1:0/3504074939 wait complete. 2024-09-11T22:02:37.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:37.903+0000 7f53ba7f0640 1 Processor -- start 2024-09-11T22:02:37.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:37.903+0000 7f53ba7f0640 1 -- start start 2024-09-11T22:02:37.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:37.903+0000 7f53ba7f0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f53b4100e20 0x7f53b419a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:37.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:37.903+0000 7f53ba7f0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f53b4102570 0x7f53b419ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:37.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:37.903+0000 7f53ba7f0640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f53b4102ef0 0x7f53b419efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:37.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:37.903+0000 7f53ba7f0640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f53b406ad10 con 0x7f53b4100e20 2024-09-11T22:02:37.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:37.903+0000 7f53ba7f0640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f53b406ab90 con 0x7f53b4102ef0 2024-09-11T22:02:37.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:37.903+0000 7f53ba7f0640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f53b406ae90 con 0x7f53b4102570 2024-09-11T22:02:37.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:37.904+0000 7f53b37fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f53b4102570 0x7f53b419ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:37.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:37.904+0000 7f53b3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f53b4100e20 0x7f53b419a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:37.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:37.904+0000 7f53b8d8d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f53b4102ef0 0x7f53b419efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:37.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:37.904+0000 7f53b37fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f53b4102570 0x7f53b419ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:35418/0 (socket says 172.21.15.1:35418) 2024-09-11T22:02:37.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:37.904+0000 7f53b37fe640 1 -- 172.21.15.1:0/2553253361 learned_addr learned my addr 172.21.15.1:0/2553253361 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:37.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:37.904+0000 7f53b3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f53b4100e20 0x7f53b419a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48244/0 (socket says 172.21.15.1:48244) 2024-09-11T22:02:37.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:37.904+0000 7f53b3fff640 1 -- 172.21.15.1:0/2553253361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f53b4102570 msgr2=0x7f53b419ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:37.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:37.904+0000 7f53b3fff640 1 --2- 172.21.15.1:0/2553253361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f53b4102570 0x7f53b419ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:37.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:37.904+0000 7f53b3fff640 1 -- 172.21.15.1:0/2553253361 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f53b4102ef0 msgr2=0x7f53b419efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:37.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:37.904+0000 7f53b3fff640 1 --2- 172.21.15.1:0/2553253361 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f53b4102ef0 0x7f53b419efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:37.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:37.904+0000 7f53b3fff640 1 -- 172.21.15.1:0/2553253361 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53b419f680 con 0x7f53b4100e20 2024-09-11T22:02:37.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:37.904+0000 7f53b8d8d640 1 --2- 172.21.15.1:0/2553253361 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f53b4102ef0 0x7f53b419efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:02:37.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:37.904+0000 7f53b3fff640 1 --2- 172.21.15.1:0/2553253361 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f53b4100e20 0x7f53b419a6d0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f53a4005940 tx=0x7f53a4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:37.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:37.904+0000 7f53b17fa640 1 -- 172.21.15.1:0/2553253361 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53a403f070 con 0x7f53b4100e20 2024-09-11T22:02:37.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:37.905+0000 7f53ba7f0640 1 -- 172.21.15.1:0/2553253361 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53b419f910 con 0x7f53b4100e20 2024-09-11T22:02:37.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:37.905+0000 7f53b17fa640 1 -- 172.21.15.1:0/2553253361 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53a4031c30 con 0x7f53b4100e20 2024-09-11T22:02:37.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:37.905+0000 7f53b17fa640 1 -- 172.21.15.1:0/2553253361 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53a4035030 con 0x7f53b4100e20 2024-09-11T22:02:37.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:37.905+0000 7f53ba7f0640 1 -- 172.21.15.1:0/2553253361 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53b41a7250 con 0x7f53b4100e20 2024-09-11T22:02:37.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:37.906+0000 7f53ba7f0640 1 -- 172.21.15.1:0/2553253361 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53b4101200 con 0x7f53b4100e20 2024-09-11T22:02:37.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:37.907+0000 7f53b17fa640 1 -- 172.21.15.1:0/2553253361 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f53a4042070 con 0x7f53b4100e20 2024-09-11T22:02:37.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:37.908+0000 7f53b17fa640 1 --2- 172.21.15.1:0/2553253361 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f539c0a2cf0 0x7f539c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:37.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:37.908+0000 7f53b17fa640 1 -- 172.21.15.1:0/2553253361 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f53a40efa80 con 0x7f53b4100e20 2024-09-11T22:02:37.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:37.908+0000 7f53b37fe640 1 --2- 172.21.15.1:0/2553253361 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f539c0a2cf0 0x7f539c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:37.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:37.909+0000 7f53b37fe640 1 --2- 172.21.15.1:0/2553253361 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f539c0a2cf0 0x7f539c0a51b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f53b410de80 tx=0x7f53a8005f90 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:37.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:37.913+0000 7f53b17fa640 1 -- 172.21.15.1:0/2553253361 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f53a40bc660 con 0x7f53b4100e20 2024-09-11T22:02:38.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:38.147+0000 7f53ba7f0640 1 -- 172.21.15.1:0/2553253361 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f53b410d4a0 con 0x7f53b4100e20 2024-09-11T22:02:38.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:38.148+0000 7f53b17fa640 1 -- 172.21.15.1:0/2553253361 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11956 (secure 0 0 0) 0x7f53a4031de0 con 0x7f53b4100e20 2024-09-11T22:02:38.149 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T22:02:38.149 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":34,"btime":"2024-09-11T22:02:32:134518+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/421542832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":421542832},{"type":"v1","addr":"172.21.15.1:6835","nonce":421542832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2836864406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2836864406},{"type":"v1","addr":"172.21.15.1:6837","nonce":2836864406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T21:55:20.791263+0000","modified":"2024-09-11T22:02:31.740162+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24484,"mds_2":24467,"mds_3":24466,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14595":{"gid":14595,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/407834672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":407834672},{"type":"v1","addr":"172.21.15.1:6839","nonce":407834672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/1068827678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":1068827678},{"type":"v1","addr":"172.21.15.1:6841","nonce":1068827678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/872410024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":872410024},{"type":"v1","addr":"172.21.15.161:6833","nonce":872410024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/1648944570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1648944570},{"type":"v1","addr":"172.21.15.53:6833","nonce":1648944570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/2126933993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2126933993},{"type":"v1","addr":"172.21.15.53:6835","nonce":2126933993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.161:6835/828381592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":828381592},{"type":"v1","addr":"172.21.15.161:6835","nonce":828381592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.161:6837/1799318656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1799318656},{"type":"v1","addr":"172.21.15.161:6837","nonce":1799318656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.53:6837/3457131830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3457131830},{"type":"v1","addr":"172.21.15.53:6837","nonce":3457131830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/455365383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":455365383},{"type":"v1","addr":"172.21.15.161:6839","nonce":455365383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.53:6839/1378584447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1378584447},{"type":"v1","addr":"172.21.15.53:6839","nonce":1378584447}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24460,24466,24467,24484,24494]},"id":1}]} 2024-09-11T22:02:38.149 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 34 2024-09-11T22:02:38.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:38.150+0000 7f53ba7f0640 1 -- 172.21.15.1:0/2553253361 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f539c0a2cf0 msgr2=0x7f539c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:38.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:38.150+0000 7f53ba7f0640 1 --2- 172.21.15.1:0/2553253361 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f539c0a2cf0 0x7f539c0a51b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f53b410de80 tx=0x7f53a8005f90 comp rx=0 tx=0).stop 2024-09-11T22:02:38.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:38.150+0000 7f53ba7f0640 1 -- 172.21.15.1:0/2553253361 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f53b4100e20 msgr2=0x7f53b419a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:38.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:38.150+0000 7f53ba7f0640 1 --2- 172.21.15.1:0/2553253361 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f53b4100e20 0x7f53b419a6d0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f53a4005940 tx=0x7f53a4003c10 comp rx=0 tx=0).stop 2024-09-11T22:02:38.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:38.150+0000 7f53ba7f0640 1 -- 172.21.15.1:0/2553253361 shutdown_connections 2024-09-11T22:02:38.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:38.150+0000 7f53ba7f0640 1 --2- 172.21.15.1:0/2553253361 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f539c0a2cf0 0x7f539c0a51b0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:38.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:38.150+0000 7f53ba7f0640 1 --2- 172.21.15.1:0/2553253361 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f53b4102ef0 0x7f53b419efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:38.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:38.150+0000 7f53ba7f0640 1 --2- 172.21.15.1:0/2553253361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f53b4102570 0x7f53b419ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:38.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:38.150+0000 7f53ba7f0640 1 --2- 172.21.15.1:0/2553253361 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f53b4100e20 0x7f53b419a6d0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:38.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:38.150+0000 7f53ba7f0640 1 -- 172.21.15.1:0/2553253361 >> 172.21.15.1:0/2553253361 conn(0x7f53b40fc7d0 msgr2=0x7f53b4105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:38.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:38.150+0000 7f53ba7f0640 1 -- 172.21.15.1:0/2553253361 shutdown_connections 2024-09-11T22:02:38.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:38.150+0000 7f53ba7f0640 1 -- 172.21.15.1:0/2553253361 wait complete. 2024-09-11T22:02:38.975 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db / --timeout 90 --expiration 140.15 --await-for 45.0' 2024-09-11T22:02:38.975 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db / --timeout 90 --expiration 140.15 --await-for 45.0 2024-09-11T22:02:39.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:38 smithi001 ceph-mon[30935]: pgmap v167: 97 pgs: 97 active+clean; 118 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.9 MiB/s wr, 1.80k op/s 2024-09-11T22:02:39.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:38 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/2553253361' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:02:39.258 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:02:39.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:38 smithi053 ceph-mon[34908]: pgmap v167: 97 pgs: 97 active+clean; 118 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.9 MiB/s wr, 1.80k op/s 2024-09-11T22:02:39.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:38 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/2553253361' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:02:39.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:38 smithi161 ceph-mon[35615]: pgmap v167: 97 pgs: 97 active+clean; 118 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.9 MiB/s wr, 1.80k op/s 2024-09-11T22:02:39.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:38 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/2553253361' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:02:39.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.766+0000 7f1c65563640 1 -- 172.21.15.1:0/3348080786 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c60102d50 msgr2=0x7f1c60103130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:39.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.766+0000 7f1c65563640 1 --2- 172.21.15.1:0/3348080786 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c60102d50 0x7f1c60103130 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1c50009860 tx=0x7f1c5002a3d0 comp rx=0 tx=0).stop 2024-09-11T22:02:39.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.766+0000 7f1c65563640 1 -- 172.21.15.1:0/3348080786 shutdown_connections 2024-09-11T22:02:39.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.766+0000 7f1c65563640 1 --2- 172.21.15.1:0/3348080786 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c6010c180 0x7f1c6010e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:39.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.766+0000 7f1c65563640 1 --2- 172.21.15.1:0/3348080786 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c60103670 0x7f1c6010bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:39.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.766+0000 7f1c65563640 1 --2- 172.21.15.1:0/3348080786 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c60102d50 0x7f1c60103130 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:39.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.766+0000 7f1c65563640 1 -- 172.21.15.1:0/3348080786 >> 172.21.15.1:0/3348080786 conn(0x7f1c600fc7d0 msgr2=0x7f1c600febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:39.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.766+0000 7f1c65563640 1 -- 172.21.15.1:0/3348080786 shutdown_connections 2024-09-11T22:02:39.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.766+0000 7f1c65563640 1 -- 172.21.15.1:0/3348080786 wait complete. 2024-09-11T22:02:39.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.767+0000 7f1c65563640 1 Processor -- start 2024-09-11T22:02:39.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.767+0000 7f1c65563640 1 -- start start 2024-09-11T22:02:39.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.767+0000 7f1c65563640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c60102d50 0x7f1c601009b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:39.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.767+0000 7f1c65563640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c60103670 0x7f1c60100ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:39.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.767+0000 7f1c65563640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c6010c180 0x7f1c60101430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:39.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.767+0000 7f1c65563640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1c60111f80 con 0x7f1c6010c180 2024-09-11T22:02:39.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.767+0000 7f1c65563640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1c60111e00 con 0x7f1c60103670 2024-09-11T22:02:39.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.767+0000 7f1c65563640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1c60112100 con 0x7f1c60102d50 2024-09-11T22:02:39.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.767+0000 7f1c5effd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c60102d50 0x7f1c601009b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:39.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.768+0000 7f1c5effd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c60102d50 0x7f1c601009b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:35444/0 (socket says 172.21.15.1:35444) 2024-09-11T22:02:39.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.768+0000 7f1c5effd640 1 -- 172.21.15.1:0/3139740460 learned_addr learned my addr 172.21.15.1:0/3139740460 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:39.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.768+0000 7f1c5e7fc640 1 --2- 172.21.15.1:0/3139740460 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c60103670 0x7f1c60100ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:39.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.768+0000 7f1c5f7fe640 1 --2- 172.21.15.1:0/3139740460 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c6010c180 0x7f1c60101430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:39.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.768+0000 7f1c5effd640 1 -- 172.21.15.1:0/3139740460 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c60103670 msgr2=0x7f1c60100ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:39.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.768+0000 7f1c5effd640 1 --2- 172.21.15.1:0/3139740460 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c60103670 0x7f1c60100ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:39.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.768+0000 7f1c5effd640 1 -- 172.21.15.1:0/3139740460 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c6010c180 msgr2=0x7f1c60101430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:39.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.768+0000 7f1c5effd640 1 --2- 172.21.15.1:0/3139740460 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c6010c180 0x7f1c60101430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:39.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.768+0000 7f1c5effd640 1 -- 172.21.15.1:0/3139740460 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c601a6970 con 0x7f1c60102d50 2024-09-11T22:02:39.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.768+0000 7f1c5f7fe640 1 --2- 172.21.15.1:0/3139740460 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c6010c180 0x7f1c60101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:02:39.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.768+0000 7f1c5e7fc640 1 --2- 172.21.15.1:0/3139740460 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c60103670 0x7f1c60100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:02:39.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.768+0000 7f1c5effd640 1 --2- 172.21.15.1:0/3139740460 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c60102d50 0x7f1c601009b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1c5002a8f0 tx=0x7f1c50030650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:39.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.768+0000 7f1c47fff640 1 -- 172.21.15.1:0/3139740460 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c50040020 con 0x7f1c60102d50 2024-09-11T22:02:39.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.768+0000 7f1c65563640 1 -- 172.21.15.1:0/3139740460 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c601a6c00 con 0x7f1c60102d50 2024-09-11T22:02:39.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.768+0000 7f1c47fff640 1 -- 172.21.15.1:0/3139740460 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c50004460 con 0x7f1c60102d50 2024-09-11T22:02:39.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.768+0000 7f1c47fff640 1 -- 172.21.15.1:0/3139740460 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c500046c0 con 0x7f1c60102d50 2024-09-11T22:02:39.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.769+0000 7f1c65563640 1 -- 172.21.15.1:0/3139740460 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c601a70e0 con 0x7f1c60102d50 2024-09-11T22:02:39.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.771+0000 7f1c47fff640 1 -- 172.21.15.1:0/3139740460 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f1c50043020 con 0x7f1c60102d50 2024-09-11T22:02:39.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.771+0000 7f1c6793c740 1 Processor -- start 2024-09-11T22:02:39.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.772+0000 7f1c6793c740 1 -- start start 2024-09-11T22:02:39.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.772+0000 7f1c6793c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a362b3d770 0x55a362b3db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:39.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.772+0000 7f1c6793c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a362b3e090 0x55a362b50d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:39.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.772+0000 7f1c6793c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a362b51240 0x55a362b53630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:39.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.772+0000 7f1c6793c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55a362aa2160 con 0x55a362b3d770 2024-09-11T22:02:39.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.772+0000 7f1c6793c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55a362a49720 con 0x55a362b3e090 2024-09-11T22:02:39.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.772+0000 7f1c6793c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a362a9e120 con 0x55a362b51240 2024-09-11T22:02:39.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.772+0000 7f1c5e7fc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a362b51240 0x55a362b53630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:39.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.772+0000 7f1c5f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a362b3d770 0x55a362b3db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:39.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.772+0000 7f1c5e7fc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a362b51240 0x55a362b53630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:35446/0 (socket says 172.21.15.1:35446) 2024-09-11T22:02:39.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.772+0000 7f1c5e7fc640 1 -- 172.21.15.1:0/1845144362 learned_addr learned my addr 172.21.15.1:0/1845144362 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:39.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.772+0000 7f1c47fff640 1 --2- 172.21.15.1:0/3139740460 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f1c480a2cf0 0x7f1c480a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:39.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.772+0000 7f1c5e7fc640 1 -- 172.21.15.1:0/1845144362 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a362b3e090 msgr2=0x55a362b50d00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T22:02:39.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.772+0000 7f1c5e7fc640 1 --2- 172.21.15.1:0/1845144362 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a362b3e090 0x55a362b50d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:39.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.772+0000 7f1c5e7fc640 1 -- 172.21.15.1:0/1845144362 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a362b3d770 msgr2=0x55a362b3db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:39.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.773+0000 7f1c5e7fc640 1 --2- 172.21.15.1:0/1845144362 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a362b3d770 0x55a362b3db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:39.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.773+0000 7f1c5e7fc640 1 -- 172.21.15.1:0/1845144362 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a362813ba0 con 0x55a362b51240 2024-09-11T22:02:39.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.773+0000 7f1c5f7fe640 1 --2- 172.21.15.1:0/1845144362 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a362b3d770 0x55a362b3db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:02:39.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.773+0000 7f1c5e7fc640 1 --2- 172.21.15.1:0/1845144362 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a362b51240 0x55a362b53630 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1c540114f0 tx=0x7f1c540119b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=565522df0575909d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:39.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.773+0000 7f1c5f7fe640 1 --2- 172.21.15.1:0/3139740460 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f1c480a2cf0 0x7f1c480a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:39.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.773+0000 7f1c457fa640 1 -- 172.21.15.1:0/1845144362 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c54005020 con 0x55a362b51240 2024-09-11T22:02:39.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.773+0000 7f1c457fa640 1 -- 172.21.15.1:0/1845144362 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c540047a0 con 0x55a362b51240 2024-09-11T22:02:39.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.773+0000 7f1c457fa640 1 -- 172.21.15.1:0/1845144362 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c5400f990 con 0x55a362b51240 2024-09-11T22:02:39.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.774+0000 7f1c47fff640 1 -- 172.21.15.1:0/3139740460 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f1c500f03b0 con 0x7f1c60102d50 2024-09-11T22:02:39.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.774+0000 7f1c6793c740 1 -- 172.21.15.1:0/1845144362 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a362b51240 msgr2=0x55a362b53630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:39.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.774+0000 7f1c5f7fe640 1 --2- 172.21.15.1:0/3139740460 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f1c480a2cf0 0x7f1c480a51b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1c58009fd0 tx=0x7f1c58004a40 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:39.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.774+0000 7f1c6793c740 1 --2- 172.21.15.1:0/1845144362 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a362b51240 0x55a362b53630 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1c540114f0 tx=0x7f1c540119b0 comp rx=0 tx=0).stop 2024-09-11T22:02:39.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.775+0000 7f1c6793c740 1 -- 172.21.15.1:0/1845144362 shutdown_connections 2024-09-11T22:02:39.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.775+0000 7f1c6793c740 1 --2- 172.21.15.1:0/1845144362 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a362b51240 0x55a362b53630 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:39.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.775+0000 7f1c6793c740 1 --2- 172.21.15.1:0/1845144362 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a362b3e090 0x55a362b50d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:39.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.775+0000 7f1c6793c740 1 --2- 172.21.15.1:0/1845144362 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a362b3d770 0x55a362b3db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:39.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.775+0000 7f1c6793c740 1 -- 172.21.15.1:0/1845144362 >> 172.21.15.1:0/1845144362 conn(0x55a362b39de0 msgr2=0x55a362b3a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:39.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.775+0000 7f1c6793c740 1 -- 172.21.15.1:0/1845144362 shutdown_connections 2024-09-11T22:02:39.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.775+0000 7f1c6793c740 1 -- 172.21.15.1:0/1845144362 wait complete. 2024-09-11T22:02:39.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.775+0000 7f1c6793c740 1 -- start start 2024-09-11T22:02:39.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.775+0000 7f1c6793c740 1 Processor -- start 2024-09-11T22:02:39.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.776+0000 7f1c6793c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a362b3d770 0x55a362b4e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:39.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.776+0000 7f1c6793c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a362b3e090 0x55a362b4e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:39.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.776+0000 7f1c6793c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a362b51240 0x55a362b4ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:39.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.776+0000 7f1c5effd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a362b3d770 0x55a362b4e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:39.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.776+0000 7f1c5f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a362b51240 0x55a362b4ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:39.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.776+0000 7f1c5effd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a362b3d770 0x55a362b4e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:35450/0 (socket says 172.21.15.1:35450) 2024-09-11T22:02:39.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.776+0000 7f1c5f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a362b51240 0x55a362b4ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48288/0 (socket says 172.21.15.1:48288) 2024-09-11T22:02:39.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.776+0000 7f1c5e7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a362b3e090 0x55a362b4e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:39.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.776+0000 7f1c5effd640 1 -- 172.21.15.1:0/3461157586 learned_addr learned my addr 172.21.15.1:0/3461157586 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:39.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.776+0000 7f1c6793c740 1 -- 172.21.15.1:0/3461157586 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55a362b53db0 con 0x55a362b51240 2024-09-11T22:02:39.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.776+0000 7f1c6793c740 1 -- 172.21.15.1:0/3461157586 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55a362a2a900 con 0x55a362b3e090 2024-09-11T22:02:39.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.776+0000 7f1c6793c740 1 -- 172.21.15.1:0/3461157586 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a362b53f30 con 0x55a362b3d770 2024-09-11T22:02:39.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.776+0000 7f1c6793c740 10 client.0 _ll_register_callbacks cb 0x55a362a28780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:02:39.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.776+0000 7f1c5effd640 1 -- 172.21.15.1:0/3461157586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a362b3e090 msgr2=0x55a362b4e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:39.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.776+0000 7f1c5effd640 1 --2- 172.21.15.1:0/3461157586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a362b3e090 0x55a362b4e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:39.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.776+0000 7f1c5effd640 1 -- 172.21.15.1:0/3461157586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a362b51240 msgr2=0x55a362b4ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:39.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.776+0000 7f1c5effd640 1 --2- 172.21.15.1:0/3461157586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a362b51240 0x55a362b4ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:39.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.776+0000 7f1c5effd640 1 -- 172.21.15.1:0/3461157586 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a362a6ac50 con 0x55a362b3d770 2024-09-11T22:02:39.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.776+0000 7f1c6793c740 1 -- 172.21.15.1:0/3461157586 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c540047a0 con 0x55a362b3d770 2024-09-11T22:02:39.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.777+0000 7f1c5effd640 1 --2- 172.21.15.1:0/3461157586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a362b3d770 0x55a362b4e200 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1c50005940 tx=0x7f1c500bc2b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3cdb89adeeeb1b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:39.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.777+0000 7f1c42ff5640 10 client.26044 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-11T22:02:39.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.777+0000 7f1c42ff5640 1 -- 172.21.15.1:0/3461157586 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c500c0c40 con 0x55a362b3d770 2024-09-11T22:02:39.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.777+0000 7f1c42ff5640 1 -- 172.21.15.1:0/3461157586 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f1c500b8f00 con 0x55a362b3d770 2024-09-11T22:02:39.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.777+0000 7f1c42ff5640 1 -- 172.21.15.1:0/3461157586 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f1c500c5500 con 0x55a362b3d770 2024-09-11T22:02:39.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.777+0000 7f1c6793c740 10 client.26044 fetch_fsmap learned FSMap version 34 2024-09-11T22:02:39.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.777+0000 7f1c6793c740 1 -- 172.21.15.1:0/3461157586 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({fsmap=34}) -- 0x55a362b53e10 con 0x55a362b3d770 2024-09-11T22:02:39.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.777+0000 7f1c42ff5640 1 -- 172.21.15.1:0/3461157586 <== mon.1 v2:172.21.15.161:3300/0 4 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7f1c500a7070 con 0x55a362b3d770 2024-09-11T22:02:39.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.777+0000 7f1c6793c740 10 client.26044 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:02:39.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.777+0000 7f1c6793c740 10 client.26044 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-11T22:02:39.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.777+0000 7f1c6793c740 20 client.26044 populate_metadata read hostname 'smithi001' 2024-09-11T22:02:39.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.777+0000 7f1c6793c740 1 --2- 172.21.15.1:0/3461157586 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55a362b41b40 0x55a362b43f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:39.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.777+0000 7f1c6793c740 4 client.26044 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:02:39.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.777+0000 7f1c6793c740 1 -- 172.21.15.1:0/3461157586 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a362aaf890 con 0x55a362b41b40 2024-09-11T22:02:39.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.778+0000 7f1c5e7fc640 1 --2- 172.21.15.1:0/3461157586 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55a362b41b40 0x55a362b43f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:39.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.778+0000 7f1c5e7fc640 1 --2- 172.21.15.1:0/3461157586 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55a362b41b40 0x55a362b43f60 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=726418f349b58e78 server_cookie=3d188a1037bf1c3d in_seq=0 out_seq=0 2024-09-11T22:02:39.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.778+0000 7f1c42ff5640 10 client.26044 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:02:39.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.779+0000 7f1c42ff5640 1 -- 172.21.15.1:0/3461157586 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a362aaf890 con 0x55a362b41b40 2024-09-11T22:02:39.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.779+0000 7f1c42ff5640 10 client.26044 handle_command_reply: tid=0 2024-09-11T22:02:39.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.779+0000 7f1c6793c740 1 client.26044 shutdown 2024-09-11T22:02:39.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.779+0000 7f1c6793c740 1 -- 172.21.15.1:0/3461157586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a362b3d770 msgr2=0x55a362b4e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:39.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.779+0000 7f1c6793c740 1 --2- 172.21.15.1:0/3461157586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a362b3d770 0x55a362b4e200 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1c50005940 tx=0x7f1c500bc2b0 comp rx=0 tx=0).stop 2024-09-11T22:02:39.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.779+0000 7f1c6793c740 1 -- 172.21.15.1:0/3461157586 shutdown_connections 2024-09-11T22:02:39.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.779+0000 7f1c6793c740 1 --2- 172.21.15.1:0/3461157586 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55a362b41b40 0x55a362b43f60 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:39.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.779+0000 7f1c6793c740 1 --2- 172.21.15.1:0/3461157586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a362b51240 0x55a362b4ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:39.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.779+0000 7f1c6793c740 1 --2- 172.21.15.1:0/3461157586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a362b3e090 0x55a362b4e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:39.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.779+0000 7f1c6793c740 1 --2- 172.21.15.1:0/3461157586 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a362b3d770 0x55a362b4e200 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:39.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.779+0000 7f1c6793c740 1 -- 172.21.15.1:0/3461157586 >> 172.21.15.1:0/3461157586 conn(0x55a362b39de0 msgr2=0x55a362b3fa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:39.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.779+0000 7f1c42ff5640 0 client.26044 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:02:39.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.779+0000 7f1c6793c740 1 -- 172.21.15.1:0/3461157586 shutdown_connections 2024-09-11T22:02:39.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.779+0000 7f1c6793c740 1 -- 172.21.15.1:0/3461157586 wait complete. 2024-09-11T22:02:39.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.779+0000 7f1c6793c740 20 client.26044 trim_cache size 0 max 16384 2024-09-11T22:02:39.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.796+0000 7f1c6793c740 1 Processor -- start 2024-09-11T22:02:39.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.796+0000 7f1c6793c740 1 -- start start 2024-09-11T22:02:39.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.796+0000 7f1c6793c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a362b3d770 0x55a362b4cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:39.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.796+0000 7f1c6793c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a362b3e090 0x55a362b4b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:39.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.796+0000 7f1c6793c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a362b51240 0x55a362b4b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:39.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.796+0000 7f1c6793c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1c60070050 con 0x55a362b3e090 2024-09-11T22:02:39.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.797+0000 7f1c6793c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1c60071c50 con 0x55a362b51240 2024-09-11T22:02:39.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.797+0000 7f1c6793c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1c60070720 con 0x55a362b3d770 2024-09-11T22:02:39.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.797+0000 7f1c5e7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a362b3e090 0x55a362b4b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:39.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.797+0000 7f1c5e7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a362b3e090 0x55a362b4b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48292/0 (socket says 172.21.15.1:48292) 2024-09-11T22:02:39.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.797+0000 7f1c5e7fc640 1 -- 172.21.15.1:0/3967751507 learned_addr learned my addr 172.21.15.1:0/3967751507 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:39.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.797+0000 7f1c5f7fe640 1 --2- 172.21.15.1:0/3967751507 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a362b51240 0x55a362b4b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:39.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.797+0000 7f1c5effd640 1 --2- 172.21.15.1:0/3967751507 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a362b3d770 0x55a362b4cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:39.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.797+0000 7f1c5e7fc640 1 -- 172.21.15.1:0/3967751507 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a362b3d770 msgr2=0x55a362b4cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:39.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.797+0000 7f1c5e7fc640 1 --2- 172.21.15.1:0/3967751507 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a362b3d770 0x55a362b4cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:39.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.797+0000 7f1c5e7fc640 1 -- 172.21.15.1:0/3967751507 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a362b51240 msgr2=0x55a362b4b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:39.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.797+0000 7f1c5e7fc640 1 --2- 172.21.15.1:0/3967751507 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a362b51240 0x55a362b4b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:39.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.797+0000 7f1c5e7fc640 1 -- 172.21.15.1:0/3967751507 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a362b49ba0 con 0x55a362b3e090 2024-09-11T22:02:39.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.797+0000 7f1c5f7fe640 1 --2- 172.21.15.1:0/3967751507 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a362b51240 0x55a362b4b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:02:39.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.797+0000 7f1c5e7fc640 1 --2- 172.21.15.1:0/3967751507 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a362b3e090 0x55a362b4b480 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f1c54013010 tx=0x7f1c54005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=324501dad7e6512b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:39.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.798+0000 7f1c44ff9640 1 -- 172.21.15.1:0/3967751507 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c54004a70 con 0x55a362b3e090 2024-09-11T22:02:39.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.798+0000 7f1c44ff9640 1 -- 172.21.15.1:0/3967751507 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c540054b0 con 0x55a362b3e090 2024-09-11T22:02:39.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.798+0000 7f1c44ff9640 1 -- 172.21.15.1:0/3967751507 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c5400f9e0 con 0x55a362b3e090 2024-09-11T22:02:39.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.798+0000 7f1c6793c740 1 -- 172.21.15.1:0/3967751507 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a362b3e090 msgr2=0x55a362b4b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:39.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.798+0000 7f1c6793c740 1 --2- 172.21.15.1:0/3967751507 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a362b3e090 0x55a362b4b480 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f1c54013010 tx=0x7f1c54005fd0 comp rx=0 tx=0).stop 2024-09-11T22:02:39.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.798+0000 7f1c6793c740 1 -- 172.21.15.1:0/3967751507 shutdown_connections 2024-09-11T22:02:39.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.798+0000 7f1c6793c740 1 --2- 172.21.15.1:0/3967751507 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a362b51240 0x55a362b4b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:39.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.799+0000 7f1c6793c740 1 --2- 172.21.15.1:0/3967751507 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a362b3e090 0x55a362b4b480 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:39.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.799+0000 7f1c6793c740 1 --2- 172.21.15.1:0/3967751507 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a362b3d770 0x55a362b4cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:39.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.799+0000 7f1c6793c740 1 -- 172.21.15.1:0/3967751507 >> 172.21.15.1:0/3967751507 conn(0x55a362b39de0 msgr2=0x55a362b3f9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:39.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.799+0000 7f1c6793c740 1 -- 172.21.15.1:0/3967751507 shutdown_connections 2024-09-11T22:02:39.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.799+0000 7f1c6793c740 1 -- 172.21.15.1:0/3967751507 wait complete. 2024-09-11T22:02:39.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.799+0000 7f1c6793c740 1 -- start start 2024-09-11T22:02:39.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.799+0000 7f1c6793c740 1 Processor -- start 2024-09-11T22:02:39.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.799+0000 7f1c6793c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a362b3d770 0x55a362b62c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:39.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.799+0000 7f1c6793c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a362b3e090 0x55a362b63160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:39.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.799+0000 7f1c6793c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a362b51240 0x55a362bc2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:39.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.799+0000 7f1c6793c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1c600714c0 con 0x55a362b3d770 2024-09-11T22:02:39.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.799+0000 7f1c6793c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1c60070df0 con 0x55a362b3e090 2024-09-11T22:02:39.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.799+0000 7f1c6793c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6006e030 con 0x55a362b51240 2024-09-11T22:02:39.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.799+0000 7f1c6793c740 10 client.0 _ll_register_callbacks cb 0x7f1c50005ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:02:39.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.799+0000 7f1c5effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a362b3d770 0x55a362b62c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:39.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.800+0000 7f1c5f7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a362b51240 0x55a362bc2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:39.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.800+0000 7f1c5e7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a362b3e090 0x55a362b63160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:39.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.800+0000 7f1c5f7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a362b51240 0x55a362bc2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:35460/0 (socket says 172.21.15.1:35460) 2024-09-11T22:02:39.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.800+0000 7f1c5f7fe640 1 -- 172.21.15.1:0/3854064932 learned_addr learned my addr 172.21.15.1:0/3854064932 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:39.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.800+0000 7f1c5e7fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a362b3e090 0x55a362b63160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:37096/0 (socket says 172.21.15.1:37096) 2024-09-11T22:02:39.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.800+0000 7f1c5f7fe640 1 -- 172.21.15.1:0/3854064932 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a362b3e090 msgr2=0x55a362b63160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:39.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.800+0000 7f1c5f7fe640 1 --2- 172.21.15.1:0/3854064932 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a362b3e090 0x55a362b63160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:39.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.800+0000 7f1c5f7fe640 1 -- 172.21.15.1:0/3854064932 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a362b3d770 msgr2=0x55a362b62c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:39.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.800+0000 7f1c5f7fe640 1 --2- 172.21.15.1:0/3854064932 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a362b3d770 0x55a362b62c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:39.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.800+0000 7f1c5f7fe640 1 -- 172.21.15.1:0/3854064932 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a362b4d1b0 con 0x55a362b51240 2024-09-11T22:02:39.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.800+0000 7f1c5effd640 1 --2- 172.21.15.1:0/3854064932 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a362b3d770 0x55a362b62c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:02:39.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.800+0000 7f1c5e7fc640 1 --2- 172.21.15.1:0/3854064932 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a362b3e090 0x55a362b63160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:02:39.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.800+0000 7f1c5f7fe640 1 --2- 172.21.15.1:0/3854064932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a362b51240 0x55a362bc2a90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1c5802d6a0 tx=0x7f1c58004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd30075103a403c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:39.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.800+0000 7f1c42ff5640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-11T22:02:39.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.800+0000 7f1c42ff5640 1 -- 172.21.15.1:0/3854064932 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c5803f070 con 0x55a362b51240 2024-09-11T22:02:39.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.800+0000 7f1c42ff5640 1 -- 172.21.15.1:0/3854064932 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f1c5803a730 con 0x55a362b51240 2024-09-11T22:02:39.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.800+0000 7f1c6793c740 1 -- 172.21.15.1:0/3854064932 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c540054b0 con 0x55a362b51240 2024-09-11T22:02:39.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.801+0000 7f1c42ff5640 1 -- 172.21.15.1:0/3854064932 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f1c5804b050 con 0x55a362b51240 2024-09-11T22:02:39.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.801+0000 7f1c6793c740 10 client.26050 fetch_fsmap learned FSMap version 34 2024-09-11T22:02:39.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.801+0000 7f1c6793c740 1 -- 172.21.15.1:0/3854064932 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55a362bc3fe0 con 0x55a362b51240 2024-09-11T22:02:39.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.801+0000 7f1c42ff5640 1 -- 172.21.15.1:0/3854064932 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c540054b0 con 0x55a362b51240 2024-09-11T22:02:39.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.801+0000 7f1c42ff5640 1 -- 172.21.15.1:0/3854064932 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7f1c5803bb20 con 0x55a362b51240 2024-09-11T22:02:39.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.801+0000 7f1c6793c740 10 client.26050 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:02:39.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.801+0000 7f1c6793c740 10 client.26050 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-11T22:02:39.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.801+0000 7f1c6793c740 20 client.26050 populate_metadata read hostname 'smithi001' 2024-09-11T22:02:39.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.801+0000 7f1c6793c740 1 --2- 172.21.15.1:0/3854064932 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55a362bc4800 0x55a362bc6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:39.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.801+0000 7f1c6793c740 4 client.26050 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.15, "await_for": 45.0}] 2024-09-11T22:02:39.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.801+0000 7f1c6793c740 1 -- 172.21.15.1:0/3854064932 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.15, "await_for": 45.0}) -- 0x55a362b35410 con 0x55a362bc4800 2024-09-11T22:02:39.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.802+0000 7f1c5effd640 1 --2- 172.21.15.1:0/3854064932 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55a362bc4800 0x55a362bc6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:39.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.802+0000 7f1c5effd640 1 --2- 172.21.15.1:0/3854064932 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55a362bc4800 0x55a362bc6c20 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1891090751c8529 server_cookie=b73d5df35ed38c09 in_seq=0 out_seq=0 2024-09-11T22:02:39.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:39.802+0000 7f1c42ff5640 10 client.26050 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:02:41.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:40 smithi001 ceph-mon[30935]: pgmap v168: 97 pgs: 97 active+clean; 128 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.0 MiB/s wr, 1.86k op/s 2024-09-11T22:02:41.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:40 smithi053 ceph-mon[34908]: pgmap v168: 97 pgs: 97 active+clean; 128 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.0 MiB/s wr, 1.86k op/s 2024-09-11T22:02:41.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:40 smithi161 ceph-mon[35615]: pgmap v168: 97 pgs: 97 active+clean; 128 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.0 MiB/s wr, 1.86k op/s 2024-09-11T22:02:41.636 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:02:41.907 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:02:42.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.458+0000 7fcac6422640 1 -- 172.21.15.1:0/3202960573 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcac0101430 msgr2=0x7fcac010d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:42.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.458+0000 7fcac6422640 1 --2- 172.21.15.1:0/3202960573 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcac0101430 0x7fcac010d960 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcab8009cd0 tx=0x7fcab802a8f0 comp rx=0 tx=0).stop 2024-09-11T22:02:42.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.458+0000 7fcac6422640 1 -- 172.21.15.1:0/3202960573 shutdown_connections 2024-09-11T22:02:42.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.458+0000 7fcac6422640 1 --2- 172.21.15.1:0/3202960573 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcac0101430 0x7fcac010d960 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:42.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.458+0000 7fcac6422640 1 --2- 172.21.15.1:0/3202960573 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcac0100a90 0x7fcac0100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:42.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.458+0000 7fcac6422640 1 --2- 172.21.15.1:0/3202960573 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcac0106a90 0x7fcac0106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:42.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.458+0000 7fcac6422640 1 -- 172.21.15.1:0/3202960573 >> 172.21.15.1:0/3202960573 conn(0x7fcac00fc7d0 msgr2=0x7fcac00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:42.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.458+0000 7fcac6422640 1 -- 172.21.15.1:0/3202960573 shutdown_connections 2024-09-11T22:02:42.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.458+0000 7fcac6422640 1 -- 172.21.15.1:0/3202960573 wait complete. 2024-09-11T22:02:42.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.459+0000 7fcac6422640 1 Processor -- start 2024-09-11T22:02:42.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.459+0000 7fcac6422640 1 -- start start 2024-09-11T22:02:42.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.459+0000 7fcac6422640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcac0100a90 0x7fcac0196200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:42.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.459+0000 7fcac6422640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcac0101430 0x7fcac0196740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:42.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.459+0000 7fcac6422640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcac0106a90 0x7fcac019aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:42.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.459+0000 7fcac6422640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcac010fd80 con 0x7fcac0100a90 2024-09-11T22:02:42.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.459+0000 7fcac6422640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcac010fc00 con 0x7fcac0106a90 2024-09-11T22:02:42.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.459+0000 7fcac6422640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcac010ff00 con 0x7fcac0101430 2024-09-11T22:02:42.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.459+0000 7fcab7fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcac0101430 0x7fcac0196740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:42.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.459+0000 7fcac49bf640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcac0106a90 0x7fcac019aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:42.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.459+0000 7fcab7fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcac0101430 0x7fcac0196740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:35478/0 (socket says 172.21.15.1:35478) 2024-09-11T22:02:42.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.459+0000 7fcab7fff640 1 -- 172.21.15.1:0/1903185089 learned_addr learned my addr 172.21.15.1:0/1903185089 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:42.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.460+0000 7fcabffff640 1 --2- 172.21.15.1:0/1903185089 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcac0100a90 0x7fcac0196200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:42.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.460+0000 7fcab7fff640 1 -- 172.21.15.1:0/1903185089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcac0106a90 msgr2=0x7fcac019aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:42.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.460+0000 7fcab7fff640 1 --2- 172.21.15.1:0/1903185089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcac0106a90 0x7fcac019aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:42.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.460+0000 7fcab7fff640 1 -- 172.21.15.1:0/1903185089 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcac0100a90 msgr2=0x7fcac0196200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:42.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.460+0000 7fcab7fff640 1 --2- 172.21.15.1:0/1903185089 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcac0100a90 0x7fcac0196200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:42.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.460+0000 7fcab7fff640 1 -- 172.21.15.1:0/1903185089 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcac019b1b0 con 0x7fcac0101430 2024-09-11T22:02:42.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.460+0000 7fcabffff640 1 --2- 172.21.15.1:0/1903185089 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcac0100a90 0x7fcac0196200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:02:42.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.460+0000 7fcac49bf640 1 --2- 172.21.15.1:0/1903185089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcac0106a90 0x7fcac019aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:02:42.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.460+0000 7fcab7fff640 1 --2- 172.21.15.1:0/1903185089 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcac0101430 0x7fcac0196740 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8009960 tx=0x7fcaa8009e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:42.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.460+0000 7fcabdffb640 1 -- 172.21.15.1:0/1903185089 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa8007c70 con 0x7fcac0101430 2024-09-11T22:02:42.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.460+0000 7fcabdffb640 1 -- 172.21.15.1:0/1903185089 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcaa801c070 con 0x7fcac0101430 2024-09-11T22:02:42.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.460+0000 7fcabdffb640 1 -- 172.21.15.1:0/1903185089 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa8016640 con 0x7fcac0101430 2024-09-11T22:02:42.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.460+0000 7fcac6422640 1 -- 172.21.15.1:0/1903185089 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcac019b4a0 con 0x7fcac0101430 2024-09-11T22:02:42.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.460+0000 7fcac6422640 1 -- 172.21.15.1:0/1903185089 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcac01a2d80 con 0x7fcac0101430 2024-09-11T22:02:42.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.463+0000 7fcac87fb740 1 Processor -- start 2024-09-11T22:02:42.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.463+0000 7fcac87fb740 1 -- start start 2024-09-11T22:02:42.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.464+0000 7fcac87fb740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562fc5dd84a0 0x562fc5dd8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:42.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.464+0000 7fcac87fb740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562fc5dd8dc0 0x562fc5de3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:42.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.464+0000 7fcabdffb640 1 -- 172.21.15.1:0/1903185089 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7fcaa80167e0 con 0x7fcac0101430 2024-09-11T22:02:42.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.464+0000 7fcac87fb740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562fc5de3680 0x562fc5de5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:42.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.464+0000 7fcac87fb740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x562fc5d3e2e0 con 0x562fc5dd8dc0 2024-09-11T22:02:42.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.464+0000 7fcac87fb740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x562fc5ce5ce0 con 0x562fc5dd84a0 2024-09-11T22:02:42.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.464+0000 7fcac87fb740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x562fc5d3a2a0 con 0x562fc5de3680 2024-09-11T22:02:42.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.464+0000 7fcac49bf640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562fc5dd84a0 0x562fc5dd8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:42.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.464+0000 7fcac49bf640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562fc5dd84a0 0x562fc5dd8880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:37114/0 (socket says 172.21.15.1:37114) 2024-09-11T22:02:42.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.464+0000 7fcac49bf640 1 -- 172.21.15.1:0/398649515 learned_addr learned my addr 172.21.15.1:0/398649515 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:42.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.464+0000 7fcabffff640 1 --2- 172.21.15.1:0/398649515 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562fc5dd8dc0 0x562fc5de3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:42.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.465+0000 7fcab7fff640 1 --2- 172.21.15.1:0/398649515 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562fc5de3680 0x562fc5de5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:42.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.465+0000 7fcac49bf640 1 -- 172.21.15.1:0/398649515 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562fc5de3680 msgr2=0x562fc5de5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:42.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.465+0000 7fcac49bf640 1 --2- 172.21.15.1:0/398649515 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562fc5de3680 0x562fc5de5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:42.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.465+0000 7fcac49bf640 1 -- 172.21.15.1:0/398649515 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562fc5dd8dc0 msgr2=0x562fc5de3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:42.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.465+0000 7fcac49bf640 1 --2- 172.21.15.1:0/398649515 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562fc5dd8dc0 0x562fc5de3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:42.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.465+0000 7fcac49bf640 1 -- 172.21.15.1:0/398649515 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fc5cca440 con 0x562fc5dd84a0 2024-09-11T22:02:42.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.465+0000 7fcab7fff640 1 --2- 172.21.15.1:0/398649515 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562fc5de3680 0x562fc5de5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:02:42.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.465+0000 7fcabdffb640 1 --2- 172.21.15.1:0/1903185089 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fcaac0a2cf0 0x7fcaac0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:42.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.466+0000 7fcabffff640 1 --2- 172.21.15.1:0/1903185089 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fcaac0a2cf0 0x7fcaac0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:42.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.466+0000 7fcabdffb640 1 -- 172.21.15.1:0/1903185089 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7fcaa80d1350 con 0x7fcac0101430 2024-09-11T22:02:42.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.466+0000 7fcac49bf640 1 --2- 172.21.15.1:0/398649515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562fc5dd84a0 0x562fc5dd8880 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fcab80093e0 tx=0x7fcab8009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4bc6750e757bb1e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:42.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.466+0000 7fcab67fc640 1 -- 172.21.15.1:0/398649515 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab8030620 con 0x562fc5dd84a0 2024-09-11T22:02:42.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.466+0000 7fcab67fc640 1 -- 172.21.15.1:0/398649515 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcab8036040 con 0x562fc5dd84a0 2024-09-11T22:02:42.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.466+0000 7fcabffff640 1 --2- 172.21.15.1:0/1903185089 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fcaac0a2cf0 0x7fcaac0a51b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fcab00095f0 tx=0x7fcab0014040 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:42.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.467+0000 7fcab67fc640 1 -- 172.21.15.1:0/398649515 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab8004480 con 0x562fc5dd84a0 2024-09-11T22:02:42.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.468+0000 7fcac87fb740 1 -- 172.21.15.1:0/398649515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562fc5dd84a0 msgr2=0x562fc5dd8880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:42.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.468+0000 7fcac87fb740 1 --2- 172.21.15.1:0/398649515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562fc5dd84a0 0x562fc5dd8880 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fcab80093e0 tx=0x7fcab8009a20 comp rx=0 tx=0).stop 2024-09-11T22:02:42.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.468+0000 7fcac87fb740 1 -- 172.21.15.1:0/398649515 shutdown_connections 2024-09-11T22:02:42.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.468+0000 7fcac87fb740 1 --2- 172.21.15.1:0/398649515 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562fc5de3680 0x562fc5de5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:42.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.468+0000 7fcac87fb740 1 --2- 172.21.15.1:0/398649515 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562fc5dd8dc0 0x562fc5de3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:42.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.468+0000 7fcac87fb740 1 --2- 172.21.15.1:0/398649515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562fc5dd84a0 0x562fc5dd8880 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:42.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.468+0000 7fcac87fb740 1 -- 172.21.15.1:0/398649515 >> 172.21.15.1:0/398649515 conn(0x562fc5dd4a30 msgr2=0x562fc5dd4e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:42.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.468+0000 7fcac87fb740 1 -- 172.21.15.1:0/398649515 shutdown_connections 2024-09-11T22:02:42.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.468+0000 7fcac87fb740 1 -- 172.21.15.1:0/398649515 wait complete. 2024-09-11T22:02:42.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.468+0000 7fcac87fb740 1 -- start start 2024-09-11T22:02:42.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.469+0000 7fcac87fb740 1 Processor -- start 2024-09-11T22:02:42.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.469+0000 7fcac87fb740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562fc5dd84a0 0x562fc5de2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:42.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.469+0000 7fcac87fb740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562fc5dd8dc0 0x562fc5de0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:42.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.469+0000 7fcac87fb740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562fc5de3680 0x562fc5de0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:42.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.469+0000 7fcac87fb740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x562fc5de62a0 con 0x562fc5dd8dc0 2024-09-11T22:02:42.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.469+0000 7fcac87fb740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x562fc5cf9050 con 0x562fc5dd84a0 2024-09-11T22:02:42.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.469+0000 7fcac87fb740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x562fc5de6420 con 0x562fc5de3680 2024-09-11T22:02:42.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.469+0000 7fcac87fb740 10 client.0 _ll_register_callbacks cb 0x562fc5d39160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:02:42.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.469+0000 7fcac49bf640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562fc5de3680 0x562fc5de0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:42.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.469+0000 7fcabffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562fc5dd84a0 0x562fc5de2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:42.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.469+0000 7fcabffff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562fc5dd84a0 0x562fc5de2f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:37124/0 (socket says 172.21.15.1:37124) 2024-09-11T22:02:42.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.469+0000 7fcabffff640 1 -- 172.21.15.1:0/910577807 learned_addr learned my addr 172.21.15.1:0/910577807 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:42.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.470+0000 7fcab7fff640 1 --2- 172.21.15.1:0/910577807 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562fc5dd8dc0 0x562fc5de0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:42.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.470+0000 7fcabffff640 1 -- 172.21.15.1:0/910577807 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562fc5de3680 msgr2=0x562fc5de0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:42.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.470+0000 7fcabffff640 1 --2- 172.21.15.1:0/910577807 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562fc5de3680 0x562fc5de0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:42.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.470+0000 7fcabffff640 1 -- 172.21.15.1:0/910577807 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562fc5dd8dc0 msgr2=0x562fc5de0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:42.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.470+0000 7fcabffff640 1 --2- 172.21.15.1:0/910577807 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562fc5dd8dc0 0x562fc5de0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:42.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.470+0000 7fcabffff640 1 -- 172.21.15.1:0/910577807 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fc5aaf3c0 con 0x562fc5dd84a0 2024-09-11T22:02:42.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.470+0000 7fcabffff640 1 --2- 172.21.15.1:0/910577807 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562fc5dd84a0 0x562fc5de2f60 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fcab001e670 tx=0x7fcab0004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19e68bf9f645c07c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:42.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.470+0000 7fca9f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-11T22:02:42.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.470+0000 7fca9f7fe640 1 -- 172.21.15.1:0/910577807 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab002b060 con 0x562fc5dd84a0 2024-09-11T22:02:42.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.470+0000 7fca9f7fe640 1 -- 172.21.15.1:0/910577807 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7fcab001c680 con 0x562fc5dd84a0 2024-09-11T22:02:42.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.470+0000 7fcac87fb740 1 -- 172.21.15.1:0/910577807 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcab8036040 con 0x562fc5dd84a0 2024-09-11T22:02:42.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.471+0000 7fca9f7fe640 1 -- 172.21.15.1:0/910577807 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fcab001c880 con 0x562fc5dd84a0 2024-09-11T22:02:42.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.471+0000 7fcac87fb740 10 client.26102 fetch_fsmap learned FSMap version 34 2024-09-11T22:02:42.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.471+0000 7fcac87fb740 1 -- 172.21.15.1:0/910577807 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562fc5dfa4d0 con 0x562fc5dd84a0 2024-09-11T22:02:42.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.471+0000 7fca9f7fe640 1 -- 172.21.15.1:0/910577807 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcab8036040 con 0x562fc5dd84a0 2024-09-11T22:02:42.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.471+0000 7fca9f7fe640 1 -- 172.21.15.1:0/910577807 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7fcab000fc00 con 0x562fc5dd84a0 2024-09-11T22:02:42.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.471+0000 7fcac87fb740 10 client.26102 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:02:42.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.471+0000 7fcac87fb740 10 client.26102 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:02:42.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.471+0000 7fcac87fb740 20 client.26102 populate_metadata read hostname 'smithi001' 2024-09-11T22:02:42.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.472+0000 7fcac87fb740 1 --2- 172.21.15.1:0/910577807 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x562fc5dfb8e0 0x562fc5dfdd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:42.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.472+0000 7fcac87fb740 4 client.26102 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:02:42.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.472+0000 7fcac87fb740 1 -- 172.21.15.1:0/910577807 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562fc5d4b850 con 0x562fc5dfb8e0 2024-09-11T22:02:42.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.472+0000 7fcab7fff640 1 --2- 172.21.15.1:0/910577807 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x562fc5dfb8e0 0x562fc5dfdd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:42.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.472+0000 7fcab7fff640 1 --2- 172.21.15.1:0/910577807 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x562fc5dfb8e0 0x562fc5dfdd10 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6e5cf8af7a4e7fa server_cookie=15c18951d336ffac in_seq=0 out_seq=0 2024-09-11T22:02:42.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.472+0000 7fca9f7fe640 10 client.26102 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:02:42.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.473+0000 7fca9f7fe640 1 -- 172.21.15.1:0/910577807 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562fc5d4b850 con 0x562fc5dfb8e0 2024-09-11T22:02:42.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.473+0000 7fca9f7fe640 10 client.26102 handle_command_reply: tid=0 2024-09-11T22:02:42.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.473+0000 7fcac87fb740 1 client.26102 shutdown 2024-09-11T22:02:42.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.473+0000 7fcac87fb740 1 -- 172.21.15.1:0/910577807 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562fc5dd84a0 msgr2=0x562fc5de2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:42.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.473+0000 7fcac87fb740 1 --2- 172.21.15.1:0/910577807 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562fc5dd84a0 0x562fc5de2f60 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fcab001e670 tx=0x7fcab0004730 comp rx=0 tx=0).stop 2024-09-11T22:02:42.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.473+0000 7fcac87fb740 1 -- 172.21.15.1:0/910577807 shutdown_connections 2024-09-11T22:02:42.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.473+0000 7fcac87fb740 1 --2- 172.21.15.1:0/910577807 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x562fc5dfb8e0 0x562fc5dfdd10 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:42.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.474+0000 7fcac87fb740 1 --2- 172.21.15.1:0/910577807 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562fc5de3680 0x562fc5de0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:42.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.474+0000 7fcac87fb740 1 --2- 172.21.15.1:0/910577807 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562fc5dd8dc0 0x562fc5de0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:42.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.474+0000 7fcac87fb740 1 --2- 172.21.15.1:0/910577807 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562fc5dd84a0 0x562fc5de2f60 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:42.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.474+0000 7fca9f7fe640 0 client.26102 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:02:42.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.474+0000 7fcac87fb740 1 -- 172.21.15.1:0/910577807 >> 172.21.15.1:0/910577807 conn(0x562fc5dd4a30 msgr2=0x562fc5dda760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:42.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.474+0000 7fcac87fb740 1 -- 172.21.15.1:0/910577807 shutdown_connections 2024-09-11T22:02:42.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.474+0000 7fcac87fb740 1 -- 172.21.15.1:0/910577807 wait complete. 2024-09-11T22:02:42.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.474+0000 7fcac87fb740 20 client.26102 trim_cache size 0 max 16384 2024-09-11T22:02:42.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.497+0000 7fcac87fb740 1 Processor -- start 2024-09-11T22:02:42.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.498+0000 7fcac87fb740 1 -- start start 2024-09-11T22:02:42.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.498+0000 7fcac87fb740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562fc5dd84a0 0x562fc5e03620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:42.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.498+0000 7fcac87fb740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562fc5dd8dc0 0x562fc5e03b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:42.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.498+0000 7fcac87fb740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562fc5de3680 0x562fc5ddd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:42.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.498+0000 7fcac87fb740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcac01ab9c0 con 0x562fc5dd84a0 2024-09-11T22:02:42.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.498+0000 7fcac87fb740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcac01ad5c0 con 0x562fc5de3680 2024-09-11T22:02:42.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.498+0000 7fcac87fb740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcac01ac090 con 0x562fc5dd8dc0 2024-09-11T22:02:42.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.498+0000 7fcab7fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562fc5dd8dc0 0x562fc5e03b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:42.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.498+0000 7fcabffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562fc5dd84a0 0x562fc5e03620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:42.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.498+0000 7fcac49bf640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562fc5de3680 0x562fc5ddd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:42.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.498+0000 7fcab7fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562fc5dd8dc0 0x562fc5e03b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:35510/0 (socket says 172.21.15.1:35510) 2024-09-11T22:02:42.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.498+0000 7fcab7fff640 1 -- 172.21.15.1:0/996121404 learned_addr learned my addr 172.21.15.1:0/996121404 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:42.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.498+0000 7fcac49bf640 1 -- 172.21.15.1:0/996121404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562fc5dd8dc0 msgr2=0x562fc5e03b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:42.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.498+0000 7fcac49bf640 1 --2- 172.21.15.1:0/996121404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562fc5dd8dc0 0x562fc5e03b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:42.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.498+0000 7fcac49bf640 1 -- 172.21.15.1:0/996121404 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562fc5dd84a0 msgr2=0x562fc5e03620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:42.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.498+0000 7fcac49bf640 1 --2- 172.21.15.1:0/996121404 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562fc5dd84a0 0x562fc5e03620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:42.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.498+0000 7fcac49bf640 1 -- 172.21.15.1:0/996121404 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fc5dd44d0 con 0x562fc5de3680 2024-09-11T22:02:42.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.498+0000 7fcab7fff640 1 --2- 172.21.15.1:0/996121404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562fc5dd8dc0 0x562fc5e03b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:02:42.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.499+0000 7fcac49bf640 1 --2- 172.21.15.1:0/996121404 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562fc5de3680 0x562fc5ddd470 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fcab802ae10 tx=0x7fcab8003f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f6c972706bdfe57 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:42.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.499+0000 7fcab5ffb640 1 -- 172.21.15.1:0/996121404 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab8030420 con 0x562fc5de3680 2024-09-11T22:02:42.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.499+0000 7fcab5ffb640 1 -- 172.21.15.1:0/996121404 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcab8036040 con 0x562fc5de3680 2024-09-11T22:02:42.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.499+0000 7fcab5ffb640 1 -- 172.21.15.1:0/996121404 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab8030db0 con 0x562fc5de3680 2024-09-11T22:02:42.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.500+0000 7fcac87fb740 1 -- 172.21.15.1:0/996121404 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562fc5de3680 msgr2=0x562fc5ddd470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:42.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.500+0000 7fcac87fb740 1 --2- 172.21.15.1:0/996121404 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562fc5de3680 0x562fc5ddd470 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fcab802ae10 tx=0x7fcab8003f10 comp rx=0 tx=0).stop 2024-09-11T22:02:42.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.500+0000 7fcac87fb740 1 -- 172.21.15.1:0/996121404 shutdown_connections 2024-09-11T22:02:42.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.500+0000 7fcac87fb740 1 --2- 172.21.15.1:0/996121404 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562fc5de3680 0x562fc5ddd470 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:42.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.500+0000 7fcac87fb740 1 --2- 172.21.15.1:0/996121404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562fc5dd8dc0 0x562fc5e03b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:42.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.500+0000 7fcac87fb740 1 --2- 172.21.15.1:0/996121404 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562fc5dd84a0 0x562fc5e03620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:42.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.500+0000 7fcac87fb740 1 -- 172.21.15.1:0/996121404 >> 172.21.15.1:0/996121404 conn(0x562fc5dd4a30 msgr2=0x562fc5ddeae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:42.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.500+0000 7fcac87fb740 1 -- 172.21.15.1:0/996121404 shutdown_connections 2024-09-11T22:02:42.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.500+0000 7fcac87fb740 1 -- 172.21.15.1:0/996121404 wait complete. 2024-09-11T22:02:42.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.501+0000 7fcac87fb740 1 -- start start 2024-09-11T22:02:42.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.501+0000 7fcac87fb740 1 Processor -- start 2024-09-11T22:02:42.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.501+0000 7fcac87fb740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562fc5dd84a0 0x562fc5df4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:42.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.501+0000 7fcac87fb740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562fc5dd8dc0 0x562fc5def960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:42.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.501+0000 7fcac87fb740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562fc5de3680 0x562fc5defea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:42.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.501+0000 7fcac87fb740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcac01ace30 con 0x562fc5de3680 2024-09-11T22:02:42.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.501+0000 7fcac87fb740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcac01ac760 con 0x562fc5dd8dc0 2024-09-11T22:02:42.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.501+0000 7fcac87fb740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcac01a99a0 con 0x562fc5dd84a0 2024-09-11T22:02:42.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.501+0000 7fcabffff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562fc5dd84a0 0x562fc5df4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:42.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.501+0000 7fcabffff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562fc5dd84a0 0x562fc5df4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:35514/0 (socket says 172.21.15.1:35514) 2024-09-11T22:02:42.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.501+0000 7fcabffff640 1 -- 172.21.15.1:0/1319287168 learned_addr learned my addr 172.21.15.1:0/1319287168 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:42.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.501+0000 7fcac87fb740 10 client.0 _ll_register_callbacks cb 0x7fcab0010530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:02:42.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.501+0000 7fcac49bf640 1 --2- 172.21.15.1:0/1319287168 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562fc5de3680 0x562fc5defea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:42.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.501+0000 7fcab7fff640 1 --2- 172.21.15.1:0/1319287168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562fc5dd8dc0 0x562fc5def960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:42.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.502+0000 7fcabffff640 1 -- 172.21.15.1:0/1319287168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562fc5dd8dc0 msgr2=0x562fc5def960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:42.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.502+0000 7fcabffff640 1 --2- 172.21.15.1:0/1319287168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562fc5dd8dc0 0x562fc5def960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:42.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.502+0000 7fcabffff640 1 -- 172.21.15.1:0/1319287168 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562fc5de3680 msgr2=0x562fc5defea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:42.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.502+0000 7fcabffff640 1 --2- 172.21.15.1:0/1319287168 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562fc5de3680 0x562fc5defea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:42.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.502+0000 7fcabffff640 1 -- 172.21.15.1:0/1319287168 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fc5df06d0 con 0x562fc5dd84a0 2024-09-11T22:02:42.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.502+0000 7fcabffff640 1 --2- 172.21.15.1:0/1319287168 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562fc5dd84a0 0x562fc5df4630 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fcab0004700 tx=0x7fcab0005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42db7e6425210015 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:42.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.502+0000 7fca9f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-11T22:02:42.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.502+0000 7fca9f7fe640 1 -- 172.21.15.1:0/1319287168 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab004c070 con 0x562fc5dd84a0 2024-09-11T22:02:42.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.502+0000 7fca9f7fe640 1 -- 172.21.15.1:0/1319287168 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7fcab001cc20 con 0x562fc5dd84a0 2024-09-11T22:02:42.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.502+0000 7fcac87fb740 1 -- 172.21.15.1:0/1319287168 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcab8036040 con 0x562fc5dd84a0 2024-09-11T22:02:42.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.502+0000 7fca9f7fe640 1 -- 172.21.15.1:0/1319287168 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fcab0057050 con 0x562fc5dd84a0 2024-09-11T22:02:42.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.502+0000 7fcac87fb740 10 client.26068 fetch_fsmap learned FSMap version 34 2024-09-11T22:02:42.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.502+0000 7fcac87fb740 1 -- 172.21.15.1:0/1319287168 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562fc5de66b0 con 0x562fc5dd84a0 2024-09-11T22:02:42.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.503+0000 7fca9f7fe640 1 -- 172.21.15.1:0/1319287168 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcab0022030 con 0x562fc5dd84a0 2024-09-11T22:02:42.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.503+0000 7fca9f7fe640 1 -- 172.21.15.1:0/1319287168 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7fcab00047a0 con 0x562fc5dd84a0 2024-09-11T22:02:42.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.503+0000 7fcac87fb740 10 client.26068 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:02:42.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.503+0000 7fcac87fb740 10 client.26068 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:02:42.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.503+0000 7fcac87fb740 20 client.26068 populate_metadata read hostname 'smithi001' 2024-09-11T22:02:42.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.503+0000 7fcac87fb740 1 --2- 172.21.15.1:0/1319287168 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x562fc5dfb8e0 0x562fc5de8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:42.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.503+0000 7fcac87fb740 4 client.26068 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:02:42.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.503+0000 7fcac87fb740 1 -- 172.21.15.1:0/1319287168 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fcab8036040 con 0x562fc5dfb8e0 2024-09-11T22:02:42.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.503+0000 7fcab7fff640 1 --2- 172.21.15.1:0/1319287168 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x562fc5dfb8e0 0x562fc5de8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:42.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.504+0000 7fcab7fff640 1 --2- 172.21.15.1:0/1319287168 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x562fc5dfb8e0 0x562fc5de8ee0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5edad8cb6861990 server_cookie=23bab3fede13eb03 in_seq=0 out_seq=0 2024-09-11T22:02:42.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.504+0000 7fca9f7fe640 10 client.26068 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:02:42.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.504+0000 7fca9f7fe640 1 -- 172.21.15.1:0/1319287168 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7fcab8036040 con 0x562fc5dfb8e0 2024-09-11T22:02:42.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.504+0000 7fca9f7fe640 10 client.26068 handle_command_reply: tid=0 2024-09-11T22:02:42.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.504+0000 7fcac87fb740 1 client.26068 shutdown 2024-09-11T22:02:42.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.504+0000 7fcac87fb740 1 -- 172.21.15.1:0/1319287168 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562fc5dd84a0 msgr2=0x562fc5df4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:42.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.504+0000 7fcac87fb740 1 --2- 172.21.15.1:0/1319287168 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562fc5dd84a0 0x562fc5df4630 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fcab0004700 tx=0x7fcab0005f90 comp rx=0 tx=0).stop 2024-09-11T22:02:42.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.505+0000 7fcac87fb740 1 -- 172.21.15.1:0/1319287168 shutdown_connections 2024-09-11T22:02:42.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.505+0000 7fcac87fb740 1 --2- 172.21.15.1:0/1319287168 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x562fc5dfb8e0 0x562fc5de8ee0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:42.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.505+0000 7fcac87fb740 1 --2- 172.21.15.1:0/1319287168 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562fc5de3680 0x562fc5defea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:42.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.505+0000 7fca9f7fe640 0 client.26068 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:02:42.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.505+0000 7fcac87fb740 1 --2- 172.21.15.1:0/1319287168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562fc5dd8dc0 0x562fc5def960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:42.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.505+0000 7fcac87fb740 1 --2- 172.21.15.1:0/1319287168 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562fc5dd84a0 0x562fc5df4630 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:42.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.505+0000 7fcac87fb740 1 -- 172.21.15.1:0/1319287168 >> 172.21.15.1:0/1319287168 conn(0x562fc5dd4a30 msgr2=0x562fc5dd3b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:42.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.505+0000 7fcac87fb740 1 -- 172.21.15.1:0/1319287168 shutdown_connections 2024-09-11T22:02:42.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.505+0000 7fcac87fb740 1 -- 172.21.15.1:0/1319287168 wait complete. 2024-09-11T22:02:42.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.505+0000 7fcac87fb740 20 client.26068 trim_cache size 0 max 16384 2024-09-11T22:02:42.511 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-11T22:02:42.511 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T22:02:42.511 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-09-11T22:02:42.511 INFO:teuthology.orchestra.run.smithi001.stdout: "77bfa38d-1de7-4e47-ae87-02330e512923": { 2024-09-11T22:02:42.511 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2024-09-11T22:02:42.511 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "77bfa38d-1de7-4e47-ae87-02330e512923", 2024-09-11T22:02:42.511 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2024-09-11T22:02:42.511 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:02:42.511 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:02:42.511 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-11T22:02:42.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.506+0000 7fcac6422640 1 -- 172.21.15.1:0/1903185089 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fcaac0a2cf0 msgr2=0x7fcaac0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:42.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.506+0000 7fcac6422640 1 --2- 172.21.15.1:0/1903185089 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fcaac0a2cf0 0x7fcaac0a51b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fcab00095f0 tx=0x7fcab0014040 comp rx=0 tx=0).stop 2024-09-11T22:02:42.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.506+0000 7fcac6422640 1 -- 172.21.15.1:0/1903185089 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcac0101430 msgr2=0x7fcac0196740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:42.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.506+0000 7fcac6422640 1 --2- 172.21.15.1:0/1903185089 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcac0101430 0x7fcac0196740 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8009960 tx=0x7fcaa8009e20 comp rx=0 tx=0).stop 2024-09-11T22:02:42.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.506+0000 7fcac6422640 1 -- 172.21.15.1:0/1903185089 shutdown_connections 2024-09-11T22:02:42.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.506+0000 7fcac6422640 1 --2- 172.21.15.1:0/1903185089 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fcaac0a2cf0 0x7fcaac0a51b0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:42.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.506+0000 7fcac6422640 1 --2- 172.21.15.1:0/1903185089 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcac0106a90 0x7fcac019aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:42.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.506+0000 7fcac6422640 1 --2- 172.21.15.1:0/1903185089 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcac0101430 0x7fcac0196740 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:42.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.506+0000 7fcac6422640 1 --2- 172.21.15.1:0/1903185089 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcac0100a90 0x7fcac0196200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:42.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.506+0000 7fcac6422640 1 -- 172.21.15.1:0/1903185089 >> 172.21.15.1:0/1903185089 conn(0x7fcac00fc7d0 msgr2=0x7fcac00fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:42.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.506+0000 7fcac6422640 1 -- 172.21.15.1:0/1903185089 shutdown_connections 2024-09-11T22:02:42.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:42.506+0000 7fcac6422640 1 -- 172.21.15.1:0/1903185089 wait complete. 2024-09-11T22:02:43.192 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:42 smithi001 ceph-mon[30935]: pgmap v169: 97 pgs: 97 active+clean; 128 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.7 MiB/s wr, 985 op/s 2024-09-11T22:02:43.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:42 smithi053 ceph-mon[34908]: pgmap v169: 97 pgs: 97 active+clean; 128 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.7 MiB/s wr, 985 op/s 2024-09-11T22:02:43.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:42 smithi161 ceph-mon[35615]: pgmap v169: 97 pgs: 97 active+clean; 128 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.7 MiB/s wr, 985 op/s 2024-09-11T22:02:43.673 INFO:tasks.cephfs.filesystem:scrub status for tag:77bfa38d-1de7-4e47-ae87-02330e512923 - {'path': '/', 'tag': '77bfa38d-1de7-4e47-ae87-02330e512923', 'options': 'recursive,force'} 2024-09-11T22:02:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.732+0000 7fb9bd8ac640 1 -- 172.21.15.1:0/3337882456 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9b8107ad0 msgr2=0x7fb9b8107eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.732+0000 7fb9bd8ac640 1 --2- 172.21.15.1:0/3337882456 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9b8107ad0 0x7fb9b8107eb0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8009930 tx=0x7fb9a802a420 comp rx=0 tx=0).stop 2024-09-11T22:02:43.734 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.732+0000 7fb9bd8ac640 1 -- 172.21.15.1:0/3337882456 shutdown_connections 2024-09-11T22:02:43.734 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.732+0000 7fb9bd8ac640 1 --2- 172.21.15.1:0/3337882456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9b8102460 0x7fb9b810e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:43.734 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.732+0000 7fb9bd8ac640 1 --2- 172.21.15.1:0/3337882456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb9b8101ac0 0x7fb9b8101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:43.734 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.732+0000 7fb9bd8ac640 1 --2- 172.21.15.1:0/3337882456 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9b8107ad0 0x7fb9b8107eb0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:43.734 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.732+0000 7fb9bd8ac640 1 -- 172.21.15.1:0/3337882456 >> 172.21.15.1:0/3337882456 conn(0x7fb9b80fd7a0 msgr2=0x7fb9b80ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:43.734 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.732+0000 7fb9bd8ac640 1 -- 172.21.15.1:0/3337882456 shutdown_connections 2024-09-11T22:02:43.734 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.732+0000 7fb9bd8ac640 1 -- 172.21.15.1:0/3337882456 wait complete. 2024-09-11T22:02:43.734 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.733+0000 7fb9bd8ac640 1 Processor -- start 2024-09-11T22:02:43.734 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.733+0000 7fb9bd8ac640 1 -- start start 2024-09-11T22:02:43.736 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.733+0000 7fb9bd8ac640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9b8101ac0 0x7fb9b81983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:43.736 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.733+0000 7fb9bd8ac640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb9b8102460 0x7fb9b8198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:43.736 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.733+0000 7fb9bd8ac640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9b8107ad0 0x7fb9b819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:43.736 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.733+0000 7fb9b6ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9b8101ac0 0x7fb9b81983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:43.736 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.734+0000 7fb9b6ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9b8101ac0 0x7fb9b81983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:37168/0 (socket says 172.21.15.1:37168) 2024-09-11T22:02:43.736 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.734+0000 7fb9b67fc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb9b8102460 0x7fb9b8198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:43.736 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.734+0000 7fb9b77fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9b8107ad0 0x7fb9b819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:43.736 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.734+0000 7fb9b77fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9b8107ad0 0x7fb9b819cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48356/0 (socket says 172.21.15.1:48356) 2024-09-11T22:02:43.737 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.734+0000 7fb9b67fc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb9b8102460 0x7fb9b8198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:35530/0 (socket says 172.21.15.1:35530) 2024-09-11T22:02:43.737 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.734+0000 7fb9bd8ac640 1 -- 172.21.15.1:0/78499790 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b8111f10 con 0x7fb9b8107ad0 2024-09-11T22:02:43.737 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.734+0000 7fb9bd8ac640 1 -- 172.21.15.1:0/78499790 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b8111d90 con 0x7fb9b8101ac0 2024-09-11T22:02:43.737 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.734+0000 7fb9bd8ac640 1 -- 172.21.15.1:0/78499790 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b8112090 con 0x7fb9b8102460 2024-09-11T22:02:43.737 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.734+0000 7fb9b6ffd640 1 -- 172.21.15.1:0/78499790 learned_addr learned my addr 172.21.15.1:0/78499790 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:43.737 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.734+0000 7fb9b6ffd640 1 -- 172.21.15.1:0/78499790 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb9b8102460 msgr2=0x7fb9b8198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:43.737 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.734+0000 7fb9b6ffd640 1 --2- 172.21.15.1:0/78499790 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb9b8102460 0x7fb9b8198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:43.737 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.734+0000 7fb9b6ffd640 1 -- 172.21.15.1:0/78499790 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9b8107ad0 msgr2=0x7fb9b819cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:43.737 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.734+0000 7fb9b6ffd640 1 --2- 172.21.15.1:0/78499790 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9b8107ad0 0x7fb9b819cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:43.737 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.734+0000 7fb9b6ffd640 1 -- 172.21.15.1:0/78499790 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9b819d3d0 con 0x7fb9b8101ac0 2024-09-11T22:02:43.737 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.734+0000 7fb9b6ffd640 1 --2- 172.21.15.1:0/78499790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9b8101ac0 0x7fb9b81983e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8009230 tx=0x7fb9a8004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:43.737 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.735+0000 7fb9bc8aa640 1 -- 172.21.15.1:0/78499790 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a803f030 con 0x7fb9b8101ac0 2024-09-11T22:02:43.738 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.735+0000 7fb9bc8aa640 1 -- 172.21.15.1:0/78499790 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9a8004440 con 0x7fb9b8101ac0 2024-09-11T22:02:43.738 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.735+0000 7fb9bd8ac640 1 -- 172.21.15.1:0/78499790 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9b819d660 con 0x7fb9b8101ac0 2024-09-11T22:02:43.738 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.735+0000 7fb9bc8aa640 1 -- 172.21.15.1:0/78499790 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a8035070 con 0x7fb9b8101ac0 2024-09-11T22:02:43.738 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.735+0000 7fb9bd8ac640 1 -- 172.21.15.1:0/78499790 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9b81a4f50 con 0x7fb9b8101ac0 2024-09-11T22:02:43.738 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.738+0000 7fb9bc8aa640 1 -- 172.21.15.1:0/78499790 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7fb9a8042030 con 0x7fb9b8101ac0 2024-09-11T22:02:43.739 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.738+0000 7fb9c064c740 1 Processor -- start 2024-09-11T22:02:43.739 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.738+0000 7fb9c064c740 1 -- start start 2024-09-11T22:02:43.740 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.738+0000 7fb9c064c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564611cfaaf0 0x564611cfaed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:43.740 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.738+0000 7fb9c064c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564611cfb410 0x564611d05770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:43.741 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.738+0000 7fb9c064c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564611d05cb0 0x564611d080a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:43.741 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.738+0000 7fb9c064c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x564611c5f2f0 con 0x564611cfaaf0 2024-09-11T22:02:43.741 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.738+0000 7fb9c064c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x564611c07270 con 0x564611d05cb0 2024-09-11T22:02:43.741 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.738+0000 7fb9c064c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x564611c5b2b0 con 0x564611cfb410 2024-09-11T22:02:43.741 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.739+0000 7fb9b6ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564611cfb410 0x564611d05770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:43.741 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.739+0000 7fb9b67fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564611d05cb0 0x564611d080a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:43.741 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.739+0000 7fb9b77fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564611cfaaf0 0x564611cfaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:43.741 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.739+0000 7fb9b77fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564611cfaaf0 0x564611cfaed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48366/0 (socket says 172.21.15.1:48366) 2024-09-11T22:02:43.741 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.739+0000 7fb9b67fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564611d05cb0 0x564611d080a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:37174/0 (socket says 172.21.15.1:37174) 2024-09-11T22:02:43.741 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.739+0000 7fb9b77fe640 1 -- 172.21.15.1:0/402801176 learned_addr learned my addr 172.21.15.1:0/402801176 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:43.741 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.739+0000 7fb9bc8aa640 1 --2- 172.21.15.1:0/78499790 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fb9a00a2df0 0x7fb9a00a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:43.741 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.739+0000 7fb9b67fc640 1 -- 172.21.15.1:0/402801176 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564611cfb410 msgr2=0x564611d05770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:43.741 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.739+0000 7fb9b67fc640 1 --2- 172.21.15.1:0/402801176 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564611cfb410 0x564611d05770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:43.742 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.739+0000 7fb9b67fc640 1 -- 172.21.15.1:0/402801176 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564611cfaaf0 msgr2=0x564611cfaed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:43.742 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.739+0000 7fb9b67fc640 1 --2- 172.21.15.1:0/402801176 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564611cfaaf0 0x564611cfaed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:43.742 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.739+0000 7fb9b67fc640 1 -- 172.21.15.1:0/402801176 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564611c06340 con 0x564611d05cb0 2024-09-11T22:02:43.742 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.739+0000 7fb9bc8aa640 1 -- 172.21.15.1:0/78499790 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7fb9a80ef730 con 0x7fb9b8101ac0 2024-09-11T22:02:43.742 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.739+0000 7fb9b77fe640 1 --2- 172.21.15.1:0/78499790 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fb9a00a2df0 0x7fb9a00a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:43.742 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.739+0000 7fb9b67fc640 1 --2- 172.21.15.1:0/402801176 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564611d05cb0 0x564611d080a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac009bf0 tx=0x7fb9ac011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec93e2278f7d5ab9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:43.742 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.740+0000 7fb99dffb640 1 -- 172.21.15.1:0/402801176 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac009da0 con 0x564611d05cb0 2024-09-11T22:02:43.742 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.740+0000 7fb99dffb640 1 -- 172.21.15.1:0/402801176 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9ac005020 con 0x564611d05cb0 2024-09-11T22:02:43.742 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.740+0000 7fb99dffb640 1 -- 172.21.15.1:0/402801176 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac004c60 con 0x564611d05cb0 2024-09-11T22:02:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.741+0000 7fb9c064c740 1 -- 172.21.15.1:0/402801176 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564611d05cb0 msgr2=0x564611d080a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.741+0000 7fb9b77fe640 1 --2- 172.21.15.1:0/78499790 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fb9a00a2df0 0x7fb9a00a52b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0010610 tx=0x7fb9b0004a40 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.741+0000 7fb9c064c740 1 --2- 172.21.15.1:0/402801176 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564611d05cb0 0x564611d080a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac009bf0 tx=0x7fb9ac011810 comp rx=0 tx=0).stop 2024-09-11T22:02:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.741+0000 7fb9c064c740 1 -- 172.21.15.1:0/402801176 shutdown_connections 2024-09-11T22:02:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.741+0000 7fb9c064c740 1 --2- 172.21.15.1:0/402801176 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564611d05cb0 0x564611d080a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.741+0000 7fb9c064c740 1 --2- 172.21.15.1:0/402801176 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564611cfb410 0x564611d05770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:43.746 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.741+0000 7fb9c064c740 1 --2- 172.21.15.1:0/402801176 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564611cfaaf0 0x564611cfaed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:43.746 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.741+0000 7fb9c064c740 1 -- 172.21.15.1:0/402801176 >> 172.21.15.1:0/402801176 conn(0x564611cf7060 msgr2=0x564611cf7440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:43.746 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.741+0000 7fb9c064c740 1 -- 172.21.15.1:0/402801176 shutdown_connections 2024-09-11T22:02:43.746 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.741+0000 7fb9c064c740 1 -- 172.21.15.1:0/402801176 wait complete. 2024-09-11T22:02:43.746 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.741+0000 7fb9c064c740 1 -- start start 2024-09-11T22:02:43.746 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.741+0000 7fb9c064c740 1 Processor -- start 2024-09-11T22:02:43.746 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.742+0000 7fb9c064c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564611cfaaf0 0x564611d02b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:43.746 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.742+0000 7fb9c064c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564611cfb410 0x564611d03080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:43.746 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.742+0000 7fb9c064c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564611d05cb0 0x564611d035c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:43.746 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.742+0000 7fb9c064c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x564611d08880 con 0x564611cfaaf0 2024-09-11T22:02:43.746 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.742+0000 7fb9c064c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x564611be8290 con 0x564611cfb410 2024-09-11T22:02:43.746 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.742+0000 7fb9c064c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x564611d08a00 con 0x564611d05cb0 2024-09-11T22:02:43.746 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.742+0000 7fb9b77fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564611d05cb0 0x564611d035c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:43.747 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.742+0000 7fb9b77fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564611d05cb0 0x564611d035c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:35544/0 (socket says 172.21.15.1:35544) 2024-09-11T22:02:43.747 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.742+0000 7fb9b67fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564611cfb410 0x564611d03080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:43.747 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.742+0000 7fb9b77fe640 1 -- 172.21.15.1:0/2005995758 learned_addr learned my addr 172.21.15.1:0/2005995758 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:43.747 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.742+0000 7fb9b6ffd640 1 --2- 172.21.15.1:0/2005995758 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564611cfaaf0 0x564611d02b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:43.747 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.742+0000 7fb9c064c740 10 client.0 _ll_register_callbacks cb 0x564611c19140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:02:43.747 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.743+0000 7fb9b67fc640 1 -- 172.21.15.1:0/2005995758 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564611d05cb0 msgr2=0x564611d035c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:43.747 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.743+0000 7fb9b67fc640 1 --2- 172.21.15.1:0/2005995758 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564611d05cb0 0x564611d035c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:43.747 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.743+0000 7fb9b67fc640 1 -- 172.21.15.1:0/2005995758 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564611cfaaf0 msgr2=0x564611d02b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:43.747 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.743+0000 7fb9b67fc640 1 --2- 172.21.15.1:0/2005995758 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564611cfaaf0 0x564611d02b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:43.747 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.743+0000 7fb9b67fc640 1 -- 172.21.15.1:0/2005995758 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564611972210 con 0x564611cfb410 2024-09-11T22:02:43.747 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.743+0000 7fb9b67fc640 1 --2- 172.21.15.1:0/2005995758 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564611cfb410 0x564611d03080 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac01b040 tx=0x7fb9ac009fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe89aca00725df20 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:43.747 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.743+0000 7fb996ffd640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-11T22:02:43.747 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.743+0000 7fb996ffd640 1 -- 172.21.15.1:0/2005995758 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac004bf0 con 0x564611cfb410 2024-09-11T22:02:43.748 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.743+0000 7fb9c064c740 1 -- 172.21.15.1:0/2005995758 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9ac005020 con 0x564611cfb410 2024-09-11T22:02:43.748 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.743+0000 7fb996ffd640 1 -- 172.21.15.1:0/2005995758 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7fb9ac016040 con 0x564611cfb410 2024-09-11T22:02:43.748 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.743+0000 7fb996ffd640 1 -- 172.21.15.1:0/2005995758 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb9ac037050 con 0x564611cfb410 2024-09-11T22:02:43.748 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.743+0000 7fb9c064c740 10 client.26120 fetch_fsmap learned FSMap version 34 2024-09-11T22:02:43.748 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.743+0000 7fb9c064c740 1 -- 172.21.15.1:0/2005995758 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x564611a3e620 con 0x564611cfb410 2024-09-11T22:02:43.748 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.744+0000 7fb996ffd640 1 -- 172.21.15.1:0/2005995758 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9ac005020 con 0x564611cfb410 2024-09-11T22:02:43.748 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.744+0000 7fb996ffd640 1 -- 172.21.15.1:0/2005995758 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7fb9ac032030 con 0x564611cfb410 2024-09-11T22:02:43.748 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.744+0000 7fb9c064c740 10 client.26120 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:02:43.748 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.744+0000 7fb9c064c740 10 client.26120 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:02:43.748 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.744+0000 7fb9c064c740 20 client.26120 populate_metadata read hostname 'smithi001' 2024-09-11T22:02:43.748 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.744+0000 7fb9c064c740 1 --2- 172.21.15.1:0/2005995758 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x564611d15aa0 0x564611d17e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:43.748 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.744+0000 7fb9c064c740 4 client.26120 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:02:43.748 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.744+0000 7fb9c064c740 1 -- 172.21.15.1:0/2005995758 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564611c6ca20 con 0x564611d15aa0 2024-09-11T22:02:43.749 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.744+0000 7fb9b6ffd640 1 --2- 172.21.15.1:0/2005995758 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x564611d15aa0 0x564611d17e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:43.749 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.744+0000 7fb996ffd640 10 client.26120 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:02:43.749 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:43.744+0000 7fb9b6ffd640 1 --2- 172.21.15.1:0/2005995758 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x564611d15aa0 0x564611d17e90 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea5b1aa65cfcaa28 server_cookie=90d609bb64dc22cf in_seq=0 out_seq=0 2024-09-11T22:02:44.436 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.435+0000 7fb996ffd640 1 -- 172.21.15.1:0/2005995758 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564611c6ca20 con 0x564611d15aa0 2024-09-11T22:02:44.437 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.435+0000 7fb996ffd640 10 client.26120 handle_command_reply: tid=0 2024-09-11T22:02:44.437 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.435+0000 7fb9c064c740 1 client.26120 shutdown 2024-09-11T22:02:44.437 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.435+0000 7fb9c064c740 1 -- 172.21.15.1:0/2005995758 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564611cfb410 msgr2=0x564611d03080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:44.437 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.435+0000 7fb9c064c740 1 --2- 172.21.15.1:0/2005995758 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564611cfb410 0x564611d03080 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac01b040 tx=0x7fb9ac009fd0 comp rx=0 tx=0).stop 2024-09-11T22:02:44.437 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.435+0000 7fb9c064c740 1 -- 172.21.15.1:0/2005995758 shutdown_connections 2024-09-11T22:02:44.437 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.435+0000 7fb9c064c740 1 --2- 172.21.15.1:0/2005995758 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x564611d15aa0 0x564611d17e90 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:44.437 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.435+0000 7fb996ffd640 0 client.26120 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:02:44.437 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.435+0000 7fb9c064c740 1 --2- 172.21.15.1:0/2005995758 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564611d05cb0 0x564611d035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:44.437 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.435+0000 7fb9c064c740 1 --2- 172.21.15.1:0/2005995758 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564611cfb410 0x564611d03080 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:44.437 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.435+0000 7fb9c064c740 1 --2- 172.21.15.1:0/2005995758 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564611cfaaf0 0x564611d02b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:44.437 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.435+0000 7fb9c064c740 1 -- 172.21.15.1:0/2005995758 >> 172.21.15.1:0/2005995758 conn(0x564611cf7060 msgr2=0x564611cfcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:44.437 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.436+0000 7fb9c064c740 1 -- 172.21.15.1:0/2005995758 shutdown_connections 2024-09-11T22:02:44.438 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.436+0000 7fb9c064c740 1 -- 172.21.15.1:0/2005995758 wait complete. 2024-09-11T22:02:44.438 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.436+0000 7fb9c064c740 20 client.26120 trim_cache size 0 max 16384 2024-09-11T22:02:44.464 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.463+0000 7fb9c064c740 1 Processor -- start 2024-09-11T22:02:44.464 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.463+0000 7fb9c064c740 1 -- start start 2024-09-11T22:02:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.464+0000 7fb9c064c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564611cfaaf0 0x564611d033f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.464+0000 7fb9c064c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564611cfb410 0x564611d1d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.464+0000 7fb9c064c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564611d05cb0 0x564611d1dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.464+0000 7fb9c064c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b81ab7d0 con 0x564611d05cb0 2024-09-11T22:02:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.464+0000 7fb9c064c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b81ad3d0 con 0x564611cfaaf0 2024-09-11T22:02:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.464+0000 7fb9c064c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b81abea0 con 0x564611cfb410 2024-09-11T22:02:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.464+0000 7fb9b67fc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564611cfb410 0x564611d1d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.464+0000 7fb9b6ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564611cfaaf0 0x564611d033f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.464+0000 7fb9b67fc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564611cfb410 0x564611d1d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:35546/0 (socket says 172.21.15.1:35546) 2024-09-11T22:02:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.464+0000 7fb9b67fc640 1 -- 172.21.15.1:0/2702998575 learned_addr learned my addr 172.21.15.1:0/2702998575 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:44.466 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.464+0000 7fb9b77fe640 1 --2- 172.21.15.1:0/2702998575 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564611d05cb0 0x564611d1dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.465+0000 7fb9b67fc640 1 -- 172.21.15.1:0/2702998575 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564611cfaaf0 msgr2=0x564611d033f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.465+0000 7fb9b67fc640 1 --2- 172.21.15.1:0/2702998575 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564611cfaaf0 0x564611d033f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.465+0000 7fb9b67fc640 1 -- 172.21.15.1:0/2702998575 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564611d05cb0 msgr2=0x564611d1dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.465+0000 7fb9b67fc640 1 --2- 172.21.15.1:0/2702998575 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564611d05cb0 0x564611d1dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.465+0000 7fb9b67fc640 1 -- 172.21.15.1:0/2702998575 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564611cfa920 con 0x564611cfb410 2024-09-11T22:02:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.465+0000 7fb9b67fc640 1 --2- 172.21.15.1:0/2702998575 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564611cfb410 0x564611d1d700 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac0056e0 tx=0x7fb9ac009640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4433fcf6d17a8f4f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.465+0000 7fb99d7fa640 1 -- 172.21.15.1:0/2702998575 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac016070 con 0x564611cfb410 2024-09-11T22:02:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.465+0000 7fb99d7fa640 1 -- 172.21.15.1:0/2702998575 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9ac0097a0 con 0x564611cfb410 2024-09-11T22:02:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.465+0000 7fb99d7fa640 1 -- 172.21.15.1:0/2702998575 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac004e50 con 0x564611cfb410 2024-09-11T22:02:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.466+0000 7fb9c064c740 1 -- 172.21.15.1:0/2702998575 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564611cfb410 msgr2=0x564611d1d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.466+0000 7fb9c064c740 1 --2- 172.21.15.1:0/2702998575 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564611cfb410 0x564611d1d700 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac0056e0 tx=0x7fb9ac009640 comp rx=0 tx=0).stop 2024-09-11T22:02:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.467+0000 7fb9c064c740 1 -- 172.21.15.1:0/2702998575 shutdown_connections 2024-09-11T22:02:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.467+0000 7fb9c064c740 1 --2- 172.21.15.1:0/2702998575 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564611d05cb0 0x564611d1dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.467+0000 7fb9c064c740 1 --2- 172.21.15.1:0/2702998575 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564611cfb410 0x564611d1d700 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.467+0000 7fb9c064c740 1 --2- 172.21.15.1:0/2702998575 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564611cfaaf0 0x564611d033f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.467+0000 7fb9c064c740 1 -- 172.21.15.1:0/2702998575 >> 172.21.15.1:0/2702998575 conn(0x564611cf7060 msgr2=0x564611d04bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.467+0000 7fb9c064c740 1 -- 172.21.15.1:0/2702998575 shutdown_connections 2024-09-11T22:02:44.469 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.467+0000 7fb9c064c740 1 -- 172.21.15.1:0/2702998575 wait complete. 2024-09-11T22:02:44.470 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.467+0000 7fb9c064c740 1 -- start start 2024-09-11T22:02:44.470 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.468+0000 7fb9c064c740 1 Processor -- start 2024-09-11T22:02:44.470 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.468+0000 7fb9c064c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564611cfaaf0 0x564611d0e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:44.470 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.468+0000 7fb9c064c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564611cfb410 0x564611cff7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:44.470 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.468+0000 7fb9c064c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564611d05cb0 0x564611cffd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.468+0000 7fb9c064c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b81acc40 con 0x564611d05cb0 2024-09-11T22:02:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.468+0000 7fb9c064c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b81ac570 con 0x564611cfaaf0 2024-09-11T22:02:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.468+0000 7fb9c064c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b81a97b0 con 0x564611cfb410 2024-09-11T22:02:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.468+0000 7fb9c064c740 10 client.0 _ll_register_callbacks cb 0x7fb9b0009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:02:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.469+0000 7fb9b6ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564611cfaaf0 0x564611d0e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.469+0000 7fb9b6ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564611cfaaf0 0x564611d0e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:37206/0 (socket says 172.21.15.1:37206) 2024-09-11T22:02:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.469+0000 7fb9b6ffd640 1 -- 172.21.15.1:0/579052204 learned_addr learned my addr 172.21.15.1:0/579052204 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.469+0000 7fb9b77fe640 1 --2- 172.21.15.1:0/579052204 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564611d05cb0 0x564611cffd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.469+0000 7fb9b67fc640 1 --2- 172.21.15.1:0/579052204 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564611cfb410 0x564611cff7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.469+0000 7fb9b6ffd640 1 -- 172.21.15.1:0/579052204 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564611cfb410 msgr2=0x564611cff7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.469+0000 7fb9b6ffd640 1 --2- 172.21.15.1:0/579052204 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564611cfb410 0x564611cff7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.469+0000 7fb9b6ffd640 1 -- 172.21.15.1:0/579052204 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564611d05cb0 msgr2=0x564611cffd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.469+0000 7fb9b6ffd640 1 --2- 172.21.15.1:0/579052204 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564611d05cb0 0x564611cffd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:44.472 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.469+0000 7fb9b6ffd640 1 -- 172.21.15.1:0/579052204 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564611cf22d0 con 0x564611cfaaf0 2024-09-11T22:02:44.472 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.469+0000 7fb9b77fe640 1 --2- 172.21.15.1:0/579052204 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564611d05cb0 0x564611cffd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:02:44.472 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.470+0000 7fb9b6ffd640 1 --2- 172.21.15.1:0/579052204 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564611cfaaf0 0x564611d0e410 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb9a80bbbf0 tx=0x7fb9a802a420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e96355f14bc61ab server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:44.472 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.470+0000 7fb996ffd640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-11T22:02:44.473 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.470+0000 7fb996ffd640 1 -- 172.21.15.1:0/579052204 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a80d2070 con 0x564611cfaaf0 2024-09-11T22:02:44.473 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.470+0000 7fb996ffd640 1 -- 172.21.15.1:0/579052204 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7fb9a80ed060 con 0x564611cfaaf0 2024-09-11T22:02:44.473 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.471+0000 7fb9c064c740 1 -- 172.21.15.1:0/579052204 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9ac0097a0 con 0x564611cfaaf0 2024-09-11T22:02:44.473 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.471+0000 7fb996ffd640 1 -- 172.21.15.1:0/579052204 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb9a8049040 con 0x564611cfaaf0 2024-09-11T22:02:44.473 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.471+0000 7fb9c064c740 10 client.26126 fetch_fsmap learned FSMap version 34 2024-09-11T22:02:44.473 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.471+0000 7fb9c064c740 1 -- 172.21.15.1:0/579052204 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x564611cfd940 con 0x564611cfaaf0 2024-09-11T22:02:44.473 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.471+0000 7fb996ffd640 1 -- 172.21.15.1:0/579052204 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9ac0097a0 con 0x564611cfaaf0 2024-09-11T22:02:44.473 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.471+0000 7fb996ffd640 1 -- 172.21.15.1:0/579052204 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7fb9a80f1c70 con 0x564611cfaaf0 2024-09-11T22:02:44.474 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.471+0000 7fb9c064c740 10 client.26126 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:02:44.474 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.471+0000 7fb9c064c740 10 client.26126 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:02:44.474 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.471+0000 7fb9c064c740 20 client.26126 populate_metadata read hostname 'smithi001' 2024-09-11T22:02:44.474 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.471+0000 7fb9c064c740 1 --2- 172.21.15.1:0/579052204 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x564611d80ae0 0x564611d82f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:44.474 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.471+0000 7fb9c064c740 4 client.26126 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:02:44.474 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.471+0000 7fb9c064c740 1 -- 172.21.15.1:0/579052204 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564611c66d70 con 0x564611d80ae0 2024-09-11T22:02:44.474 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.472+0000 7fb9b67fc640 1 --2- 172.21.15.1:0/579052204 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x564611d80ae0 0x564611d82f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:44.474 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.472+0000 7fb9b67fc640 1 --2- 172.21.15.1:0/579052204 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x564611d80ae0 0x564611d82f00 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3734c0caf3e8579b server_cookie=18f3f3cd8c46fd26 in_seq=0 out_seq=0 2024-09-11T22:02:44.474 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:44.472+0000 7fb996ffd640 10 client.26126 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:02:45.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:44 smithi001 ceph-mon[30935]: pgmap v170: 97 pgs: 97 active+clean; 128 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.7 MiB/s wr, 985 op/s 2024-09-11T22:02:45.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:44 smithi053 ceph-mon[34908]: pgmap v170: 97 pgs: 97 active+clean; 128 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.7 MiB/s wr, 985 op/s 2024-09-11T22:02:45.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:44 smithi161 ceph-mon[35615]: pgmap v170: 97 pgs: 97 active+clean; 128 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.7 MiB/s wr, 985 op/s 2024-09-11T22:02:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:46.249+0000 7fb996ffd640 1 -- 172.21.15.1:0/579052204 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564611c66d70 con 0x564611d80ae0 2024-09-11T22:02:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:46.249+0000 7fb996ffd640 10 client.26126 handle_command_reply: tid=0 2024-09-11T22:02:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:46.249+0000 7fb9c064c740 1 client.26126 shutdown 2024-09-11T22:02:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:46.250+0000 7fb9c064c740 1 -- 172.21.15.1:0/579052204 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564611cfaaf0 msgr2=0x564611d0e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:46.250+0000 7fb9c064c740 1 --2- 172.21.15.1:0/579052204 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564611cfaaf0 0x564611d0e410 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb9a80bbbf0 tx=0x7fb9a802a420 comp rx=0 tx=0).stop 2024-09-11T22:02:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:46.250+0000 7fb9c064c740 1 -- 172.21.15.1:0/579052204 shutdown_connections 2024-09-11T22:02:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:46.250+0000 7fb9c064c740 1 --2- 172.21.15.1:0/579052204 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x564611d80ae0 0x564611d82f00 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:46.250+0000 7fb9c064c740 1 --2- 172.21.15.1:0/579052204 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564611d05cb0 0x564611cffd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:46.250+0000 7fb996ffd640 0 client.26126 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:02:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:46.250+0000 7fb9c064c740 1 --2- 172.21.15.1:0/579052204 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564611cfb410 0x564611cff7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:46.250+0000 7fb9c064c740 1 --2- 172.21.15.1:0/579052204 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564611cfaaf0 0x564611d0e410 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:46.250+0000 7fb9c064c740 1 -- 172.21.15.1:0/579052204 >> 172.21.15.1:0/579052204 conn(0x564611cf7060 msgr2=0x564611cf59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:46.250+0000 7fb9c064c740 1 -- 172.21.15.1:0/579052204 shutdown_connections 2024-09-11T22:02:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:46.250+0000 7fb9c064c740 1 -- 172.21.15.1:0/579052204 wait complete. 2024-09-11T22:02:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:46.250+0000 7fb9c064c740 20 client.26126 trim_cache size 0 max 16384 2024-09-11T22:02:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-09-11T22:02:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-09-11T22:02:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-09-11T22:02:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-09-11T22:02:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:46.251+0000 7fb9bd8ac640 1 -- 172.21.15.1:0/78499790 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fb9a00a2df0 msgr2=0x7fb9a00a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:46.251+0000 7fb9bd8ac640 1 --2- 172.21.15.1:0/78499790 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fb9a00a2df0 0x7fb9a00a52b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0010610 tx=0x7fb9b0004a40 comp rx=0 tx=0).stop 2024-09-11T22:02:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:46.251+0000 7fb9bd8ac640 1 -- 172.21.15.1:0/78499790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9b8101ac0 msgr2=0x7fb9b81983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:46.251+0000 7fb9bd8ac640 1 --2- 172.21.15.1:0/78499790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9b8101ac0 0x7fb9b81983e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8009230 tx=0x7fb9a8004040 comp rx=0 tx=0).stop 2024-09-11T22:02:46.254 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:46.251+0000 7fb9bd8ac640 1 -- 172.21.15.1:0/78499790 shutdown_connections 2024-09-11T22:02:46.254 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:46.251+0000 7fb9bd8ac640 1 --2- 172.21.15.1:0/78499790 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fb9a00a2df0 0x7fb9a00a52b0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:46.254 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:46.251+0000 7fb9bd8ac640 1 --2- 172.21.15.1:0/78499790 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9b8107ad0 0x7fb9b819cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:46.254 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:46.251+0000 7fb9bd8ac640 1 --2- 172.21.15.1:0/78499790 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb9b8102460 0x7fb9b8198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:46.254 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:46.251+0000 7fb9bd8ac640 1 --2- 172.21.15.1:0/78499790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb9b8101ac0 0x7fb9b81983e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:46.254 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:46.251+0000 7fb9bd8ac640 1 -- 172.21.15.1:0/78499790 >> 172.21.15.1:0/78499790 conn(0x7fb9b80fd7a0 msgr2=0x7fb9b80ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:46.254 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:46.251+0000 7fb9bd8ac640 1 -- 172.21.15.1:0/78499790 shutdown_connections 2024-09-11T22:02:46.254 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:46.251+0000 7fb9bd8ac640 1 -- 172.21.15.1:0/78499790 wait complete. 2024-09-11T22:02:47.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:46 smithi053 ceph-mon[34908]: scrub summary: idle 2024-09-11T22:02:47.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:46 smithi053 ceph-mon[34908]: scrub summary: idle 2024-09-11T22:02:47.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:46 smithi053 ceph-mon[34908]: scrub summary: idle 2024-09-11T22:02:47.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:46 smithi053 ceph-mon[34908]: scrub summary: idle 2024-09-11T22:02:47.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:46 smithi053 ceph-mon[34908]: pgmap v171: 97 pgs: 97 active+clean; 139 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 1.47k op/s 2024-09-11T22:02:47.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:46 smithi161 ceph-mon[35615]: scrub summary: idle 2024-09-11T22:02:47.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:46 smithi161 ceph-mon[35615]: scrub summary: idle 2024-09-11T22:02:47.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:46 smithi161 ceph-mon[35615]: scrub summary: idle 2024-09-11T22:02:47.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:46 smithi161 ceph-mon[35615]: scrub summary: idle 2024-09-11T22:02:47.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:46 smithi161 ceph-mon[35615]: pgmap v171: 97 pgs: 97 active+clean; 139 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 1.47k op/s 2024-09-11T22:02:47.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:46 smithi001 ceph-mon[30935]: scrub summary: idle 2024-09-11T22:02:47.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:46 smithi001 ceph-mon[30935]: scrub summary: idle 2024-09-11T22:02:47.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:46 smithi001 ceph-mon[30935]: scrub summary: idle 2024-09-11T22:02:47.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:46 smithi001 ceph-mon[30935]: scrub summary: idle 2024-09-11T22:02:47.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:46 smithi001 ceph-mon[30935]: pgmap v171: 97 pgs: 97 active+clean; 139 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 1.47k op/s 2024-09-11T22:02:47.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:47.774+0000 7f1c42ff5640 1 -- 172.21.15.1:0/3854064932 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55a362b35410 con 0x55a362bc4800 2024-09-11T22:02:47.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:47.774+0000 7f1c42ff5640 10 client.26050 handle_command_reply: tid=0 2024-09-11T22:02:47.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:47.774+0000 7f1c6793c740 1 client.26050 shutdown 2024-09-11T22:02:47.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:47.774+0000 7f1c6793c740 1 -- 172.21.15.1:0/3854064932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a362b51240 msgr2=0x55a362bc2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:47.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:47.774+0000 7f1c6793c740 1 --2- 172.21.15.1:0/3854064932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a362b51240 0x55a362bc2a90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1c5802d6a0 tx=0x7f1c58004ab0 comp rx=0 tx=0).stop 2024-09-11T22:02:47.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:47.774+0000 7f1c6793c740 1 -- 172.21.15.1:0/3854064932 shutdown_connections 2024-09-11T22:02:47.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:47.774+0000 7f1c6793c740 1 --2- 172.21.15.1:0/3854064932 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55a362bc4800 0x55a362bc6c20 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:47.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:47.774+0000 7f1c6793c740 1 --2- 172.21.15.1:0/3854064932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a362b51240 0x55a362bc2a90 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:47.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:47.774+0000 7f1c6793c740 1 --2- 172.21.15.1:0/3854064932 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a362b3e090 0x55a362b63160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:47.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:47.774+0000 7f1c6793c740 1 --2- 172.21.15.1:0/3854064932 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a362b3d770 0x55a362b62c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:47.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:47.774+0000 7f1c42ff5640 0 client.26050 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:02:47.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:47.774+0000 7f1c6793c740 1 -- 172.21.15.1:0/3854064932 >> 172.21.15.1:0/3854064932 conn(0x55a362b39de0 msgr2=0x55a362b3f9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:47.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:47.774+0000 7f1c6793c740 1 -- 172.21.15.1:0/3854064932 shutdown_connections 2024-09-11T22:02:47.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:47.774+0000 7f1c6793c740 1 -- 172.21.15.1:0/3854064932 wait complete. 2024-09-11T22:02:47.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:47.775+0000 7f1c6793c740 20 client.26050 trim_cache size 0 max 16384 2024-09-11T22:02:47.777 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-11T22:02:47.777 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 34, 2024-09-11T22:02:47.777 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24494, 2024-09-11T22:02:47.777 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 18, 2024-09-11T22:02:47.777 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-09-11T22:02:47.777 INFO:teuthology.orchestra.run.smithi001.stdout: "125e9d94": { 2024-09-11T22:02:47.777 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 18, 2024-09-11T22:02:47.778 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-09-11T22:02:47.778 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-11T22:02:47.778 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-09-11T22:02:47.778 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-11T22:02:47.778 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:02:47.778 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 90.0, 2024-09-11T22:02:47.778 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 140.2, 2024-09-11T22:02:47.778 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-09-11T22:02:47.778 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-09-11T22:02:47.778 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-09-11T22:02:47.778 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-11T22:02:47.778 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-09-11T22:02:47.778 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-11T22:02:47.779 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:02:47.779 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:02:47.779 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:02:47.779 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:02:47.779 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:02:47.779 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-11T22:02:47.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:47.775+0000 7f1c65563640 1 -- 172.21.15.1:0/3139740460 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f1c480a2cf0 msgr2=0x7f1c480a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:47.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:47.775+0000 7f1c65563640 1 --2- 172.21.15.1:0/3139740460 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f1c480a2cf0 0x7f1c480a51b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1c58009fd0 tx=0x7f1c58004a40 comp rx=0 tx=0).stop 2024-09-11T22:02:47.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:47.775+0000 7f1c65563640 1 -- 172.21.15.1:0/3139740460 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c60102d50 msgr2=0x7f1c601009b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:47.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:47.775+0000 7f1c65563640 1 --2- 172.21.15.1:0/3139740460 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c60102d50 0x7f1c601009b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1c5002a8f0 tx=0x7f1c50030650 comp rx=0 tx=0).stop 2024-09-11T22:02:47.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:47.776+0000 7f1c65563640 1 -- 172.21.15.1:0/3139740460 shutdown_connections 2024-09-11T22:02:47.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:47.776+0000 7f1c65563640 1 --2- 172.21.15.1:0/3139740460 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f1c480a2cf0 0x7f1c480a51b0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:47.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:47.776+0000 7f1c65563640 1 --2- 172.21.15.1:0/3139740460 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c6010c180 0x7f1c60101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:47.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:47.776+0000 7f1c65563640 1 --2- 172.21.15.1:0/3139740460 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c60103670 0x7f1c60100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:47.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:47.776+0000 7f1c65563640 1 --2- 172.21.15.1:0/3139740460 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c60102d50 0x7f1c601009b0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:47.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:47.776+0000 7f1c65563640 1 -- 172.21.15.1:0/3139740460 >> 172.21.15.1:0/3139740460 conn(0x7f1c600fc7d0 msgr2=0x7f1c600fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:47.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:47.776+0000 7f1c65563640 1 -- 172.21.15.1:0/3139740460 shutdown_connections 2024-09-11T22:02:47.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:47.776+0000 7f1c65563640 1 -- 172.21.15.1:0/3139740460 wait complete. 2024-09-11T22:02:48.107 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:47 smithi001 ceph-mon[30935]: scrub summary: idle 2024-09-11T22:02:48.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:47 smithi053 ceph-mon[34908]: scrub summary: idle 2024-09-11T22:02:48.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:47 smithi161 ceph-mon[35615]: scrub summary: idle 2024-09-11T22:02:48.477 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '125e9d94', quiesce took 11.5 seconds. Will release after: 8.649999999999999 2024-09-11T22:02:49.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:48 smithi053 ceph-mon[34908]: pgmap v172: 97 pgs: 97 active+clean; 139 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 543 op/s 2024-09-11T22:02:49.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:48 smithi161 ceph-mon[35615]: pgmap v172: 97 pgs: 97 active+clean; 139 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 543 op/s 2024-09-11T22:02:49.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:48 smithi001 ceph-mon[30935]: pgmap v172: 97 pgs: 97 active+clean; 139 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 543 op/s 2024-09-11T22:02:51.246 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:50 smithi001 ceph-mon[30935]: pgmap v173: 97 pgs: 97 active+clean; 146 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.4 MiB/s wr, 835 op/s 2024-09-11T22:02:51.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:50 smithi053 ceph-mon[34908]: pgmap v173: 97 pgs: 97 active+clean; 146 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.4 MiB/s wr, 835 op/s 2024-09-11T22:02:51.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:50 smithi161 ceph-mon[35615]: pgmap v173: 97 pgs: 97 active+clean; 146 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.4 MiB/s wr, 835 op/s 2024-09-11T22:02:53.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:52 smithi053 ceph-mon[34908]: pgmap v174: 97 pgs: 97 active+clean; 146 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 778 op/s 2024-09-11T22:02:53.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:52 smithi161 ceph-mon[35615]: pgmap v174: 97 pgs: 97 active+clean; 146 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 778 op/s 2024-09-11T22:02:53.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:52 smithi001 ceph-mon[30935]: pgmap v174: 97 pgs: 97 active+clean; 146 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 778 op/s 2024-09-11T22:02:55.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:54 smithi053 ceph-mon[34908]: pgmap v175: 97 pgs: 97 active+clean; 146 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 778 op/s 2024-09-11T22:02:55.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:54 smithi161 ceph-mon[35615]: pgmap v175: 97 pgs: 97 active+clean; 146 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 778 op/s 2024-09-11T22:02:55.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:54 smithi001 ceph-mon[30935]: pgmap v175: 97 pgs: 97 active+clean; 146 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 778 op/s 2024-09-11T22:02:57.128 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:02:57.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:56 smithi053 ceph-mon[34908]: pgmap v176: 97 pgs: 97 active+clean; 146 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 784 op/s 2024-09-11T22:02:57.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:56 smithi161 ceph-mon[35615]: pgmap v176: 97 pgs: 97 active+clean; 146 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 784 op/s 2024-09-11T22:02:57.421 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:02:57.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:56 smithi001 ceph-mon[30935]: pgmap v176: 97 pgs: 97 active+clean; 146 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 784 op/s 2024-09-11T22:02:57.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:57.860+0000 7f91710d6640 1 -- 172.21.15.1:0/55834407 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f916c1036e0 msgr2=0x7f916c107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:57.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:57.860+0000 7f91710d6640 1 --2- 172.21.15.1:0/55834407 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f916c1036e0 0x7f916c107b90 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f9154009930 tx=0x7f915402a420 comp rx=0 tx=0).stop 2024-09-11T22:02:57.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:57.861+0000 7f91710d6640 1 -- 172.21.15.1:0/55834407 shutdown_connections 2024-09-11T22:02:57.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:57.861+0000 7f91710d6640 1 --2- 172.21.15.1:0/55834407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f916c1082a0 0x7f916c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:57.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:57.861+0000 7f91710d6640 1 --2- 172.21.15.1:0/55834407 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f916c1036e0 0x7f916c107b90 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:57.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:57.861+0000 7f91710d6640 1 --2- 172.21.15.1:0/55834407 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f916c102d30 0x7f916c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:57.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:57.861+0000 7f91710d6640 1 -- 172.21.15.1:0/55834407 >> 172.21.15.1:0/55834407 conn(0x7f916c0fc7d0 msgr2=0x7f916c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:57.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:57.861+0000 7f91710d6640 1 -- 172.21.15.1:0/55834407 shutdown_connections 2024-09-11T22:02:57.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:57.861+0000 7f91710d6640 1 -- 172.21.15.1:0/55834407 wait complete. 2024-09-11T22:02:57.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:57.861+0000 7f91710d6640 1 Processor -- start 2024-09-11T22:02:57.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:57.862+0000 7f91710d6640 1 -- start start 2024-09-11T22:02:57.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:57.862+0000 7f91710d6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f916c102d30 0x7f916c198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:57.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:57.862+0000 7f91710d6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f916c1036e0 0x7f916c198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:57.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:57.862+0000 7f91710d6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f916c1082a0 0x7f916c19cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:57.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:57.862+0000 7f91710d6640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f916c112060 con 0x7f916c102d30 2024-09-11T22:02:57.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:57.862+0000 7f91710d6640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f916c111ee0 con 0x7f916c1036e0 2024-09-11T22:02:57.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:57.862+0000 7f91710d6640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f916c1121e0 con 0x7f916c1082a0 2024-09-11T22:02:57.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:57.862+0000 7f916ad9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f916c102d30 0x7f916c198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:57.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:57.862+0000 7f916ad9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f916c102d30 0x7f916c198640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59602/0 (socket says 172.21.15.1:59602) 2024-09-11T22:02:57.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:57.862+0000 7f916ad9d640 1 -- 172.21.15.1:0/1148367741 learned_addr learned my addr 172.21.15.1:0/1148367741 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:57.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:57.862+0000 7f916a59c640 1 --2- 172.21.15.1:0/1148367741 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f916c1036e0 0x7f916c198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:57.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:57.862+0000 7f916b59e640 1 --2- 172.21.15.1:0/1148367741 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f916c1082a0 0x7f916c19cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:57.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:57.862+0000 7f916ad9d640 1 -- 172.21.15.1:0/1148367741 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f916c1082a0 msgr2=0x7f916c19cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:57.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:57.862+0000 7f916ad9d640 1 --2- 172.21.15.1:0/1148367741 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f916c1082a0 0x7f916c19cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:57.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:57.863+0000 7f916ad9d640 1 -- 172.21.15.1:0/1148367741 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f916c1036e0 msgr2=0x7f916c198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:57.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:57.863+0000 7f916ad9d640 1 --2- 172.21.15.1:0/1148367741 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f916c1036e0 0x7f916c198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:57.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:57.863+0000 7f916ad9d640 1 -- 172.21.15.1:0/1148367741 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f916c19d5f0 con 0x7f916c102d30 2024-09-11T22:02:57.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:57.863+0000 7f916b59e640 1 --2- 172.21.15.1:0/1148367741 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f916c1082a0 0x7f916c19cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:02:57.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:57.863+0000 7f916ad9d640 1 --2- 172.21.15.1:0/1148367741 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f916c102d30 0x7f916c198640 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f916400c890 tx=0x7f916400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:57.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:57.863+0000 7f9163fff640 1 -- 172.21.15.1:0/1148367741 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9164004220 con 0x7f916c102d30 2024-09-11T22:02:57.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:57.863+0000 7f9163fff640 1 -- 172.21.15.1:0/1148367741 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91640043c0 con 0x7f916c102d30 2024-09-11T22:02:57.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:57.863+0000 7f91710d6640 1 -- 172.21.15.1:0/1148367741 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f916c19d8e0 con 0x7f916c102d30 2024-09-11T22:02:57.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:57.863+0000 7f9163fff640 1 -- 172.21.15.1:0/1148367741 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f916400f450 con 0x7f916c102d30 2024-09-11T22:02:57.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:57.863+0000 7f91710d6640 1 -- 172.21.15.1:0/1148367741 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f916c1a51c0 con 0x7f916c102d30 2024-09-11T22:02:57.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:57.864+0000 7f91710d6640 1 -- 172.21.15.1:0/1148367741 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f916c103110 con 0x7f916c102d30 2024-09-11T22:02:57.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:57.865+0000 7f9163fff640 1 -- 172.21.15.1:0/1148367741 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f91640079b0 con 0x7f916c102d30 2024-09-11T22:02:57.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:57.866+0000 7f9163fff640 1 --2- 172.21.15.1:0/1148367741 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f91580a2cf0 0x7f91580a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:57.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:57.866+0000 7f9163fff640 1 -- 172.21.15.1:0/1148367741 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f91640ceee0 con 0x7f916c102d30 2024-09-11T22:02:57.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:57.866+0000 7f916a59c640 1 --2- 172.21.15.1:0/1148367741 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f91580a2cf0 0x7f91580a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:57.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:57.867+0000 7f916a59c640 1 --2- 172.21.15.1:0/1148367741 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f91580a2cf0 0x7f91580a51b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9154005940 tx=0x7f91540058d0 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:57.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:57.870+0000 7f9163fff640 1 -- 172.21.15.1:0/1148367741 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9164014030 con 0x7f916c102d30 2024-09-11T22:02:58.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:58.082+0000 7f91710d6640 1 -- 172.21.15.1:0/1148367741 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f916c10a2b0 con 0x7f916c102d30 2024-09-11T22:02:58.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:58.083+0000 7f9163fff640 1 -- 172.21.15.1:0/1148367741 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11956 (secure 0 0 0) 0x7f916409ba60 con 0x7f916c102d30 2024-09-11T22:02:58.084 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T22:02:58.084 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":34,"btime":"2024-09-11T22:02:32:134518+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/421542832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":421542832},{"type":"v1","addr":"172.21.15.1:6835","nonce":421542832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2836864406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2836864406},{"type":"v1","addr":"172.21.15.1:6837","nonce":2836864406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T21:55:20.791263+0000","modified":"2024-09-11T22:02:31.740162+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24484,"mds_2":24467,"mds_3":24466,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14595":{"gid":14595,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/407834672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":407834672},{"type":"v1","addr":"172.21.15.1:6839","nonce":407834672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/1068827678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":1068827678},{"type":"v1","addr":"172.21.15.1:6841","nonce":1068827678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/872410024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":872410024},{"type":"v1","addr":"172.21.15.161:6833","nonce":872410024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/1648944570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1648944570},{"type":"v1","addr":"172.21.15.53:6833","nonce":1648944570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/2126933993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2126933993},{"type":"v1","addr":"172.21.15.53:6835","nonce":2126933993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.161:6835/828381592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":828381592},{"type":"v1","addr":"172.21.15.161:6835","nonce":828381592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.161:6837/1799318656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1799318656},{"type":"v1","addr":"172.21.15.161:6837","nonce":1799318656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.53:6837/3457131830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3457131830},{"type":"v1","addr":"172.21.15.53:6837","nonce":3457131830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/455365383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":455365383},{"type":"v1","addr":"172.21.15.161:6839","nonce":455365383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.53:6839/1378584447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1378584447},{"type":"v1","addr":"172.21.15.53:6839","nonce":1378584447}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24460,24466,24467,24484,24494]},"id":1}]} 2024-09-11T22:02:58.084 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 34 2024-09-11T22:02:58.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:58.085+0000 7f91710d6640 1 -- 172.21.15.1:0/1148367741 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f91580a2cf0 msgr2=0x7f91580a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:58.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:58.085+0000 7f91710d6640 1 --2- 172.21.15.1:0/1148367741 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f91580a2cf0 0x7f91580a51b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9154005940 tx=0x7f91540058d0 comp rx=0 tx=0).stop 2024-09-11T22:02:58.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:58.085+0000 7f91710d6640 1 -- 172.21.15.1:0/1148367741 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f916c102d30 msgr2=0x7f916c198640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:58.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:58.085+0000 7f91710d6640 1 --2- 172.21.15.1:0/1148367741 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f916c102d30 0x7f916c198640 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f916400c890 tx=0x7f916400cd50 comp rx=0 tx=0).stop 2024-09-11T22:02:58.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:58.085+0000 7f91710d6640 1 -- 172.21.15.1:0/1148367741 shutdown_connections 2024-09-11T22:02:58.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:58.085+0000 7f91710d6640 1 --2- 172.21.15.1:0/1148367741 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f91580a2cf0 0x7f91580a51b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:58.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:58.085+0000 7f91710d6640 1 --2- 172.21.15.1:0/1148367741 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f916c1082a0 0x7f916c19cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:58.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:58.085+0000 7f91710d6640 1 --2- 172.21.15.1:0/1148367741 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f916c1036e0 0x7f916c198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:58.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:58.085+0000 7f91710d6640 1 --2- 172.21.15.1:0/1148367741 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f916c102d30 0x7f916c198640 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:58.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:58.085+0000 7f91710d6640 1 -- 172.21.15.1:0/1148367741 >> 172.21.15.1:0/1148367741 conn(0x7f916c0fc7d0 msgr2=0x7f916c105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:58.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:58.085+0000 7f91710d6640 1 -- 172.21.15.1:0/1148367741 shutdown_connections 2024-09-11T22:02:58.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:58.085+0000 7f91710d6640 1 -- 172.21.15.1:0/1148367741 wait complete. 2024-09-11T22:02:58.860 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db --set-id 125e9d94 --release --await' 2024-09-11T22:02:58.861 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db --set-id 125e9d94 --release --await 2024-09-11T22:02:59.126 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:02:59.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:58 smithi001 ceph-mon[30935]: pgmap v177: 97 pgs: 97 active+clean; 146 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1011 KiB/s rd, 1.2 MiB/s wr, 299 op/s 2024-09-11T22:02:59.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:02:58 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/1148367741' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:02:59.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:58 smithi053 ceph-mon[34908]: pgmap v177: 97 pgs: 97 active+clean; 146 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1011 KiB/s rd, 1.2 MiB/s wr, 299 op/s 2024-09-11T22:02:59.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:02:58 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/1148367741' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:02:59.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:58 smithi161 ceph-mon[35615]: pgmap v177: 97 pgs: 97 active+clean; 146 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1011 KiB/s rd, 1.2 MiB/s wr, 299 op/s 2024-09-11T22:02:59.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:02:58 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/1148367741' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:02:59.344 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.343+0000 7fd8ecd49640 1 -- 172.21.15.1:0/1018784081 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd8e8101340 msgr2=0x7fd8e810d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:59.344 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.343+0000 7fd8ecd49640 1 --2- 172.21.15.1:0/1018784081 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd8e8101340 0x7fd8e810d870 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fd8e0009cd0 tx=0x7fd8e002a8e0 comp rx=0 tx=0).stop 2024-09-11T22:02:59.345 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.344+0000 7fd8ecd49640 1 -- 172.21.15.1:0/1018784081 shutdown_connections 2024-09-11T22:02:59.345 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.344+0000 7fd8ecd49640 1 --2- 172.21.15.1:0/1018784081 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd8e8101340 0x7fd8e810d870 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:59.345 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.344+0000 7fd8ecd49640 1 --2- 172.21.15.1:0/1018784081 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8e81009a0 0x7fd8e8100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:59.345 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.344+0000 7fd8ecd49640 1 --2- 172.21.15.1:0/1018784081 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8e81069b0 0x7fd8e8106d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:59.345 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.344+0000 7fd8ecd49640 1 -- 172.21.15.1:0/1018784081 >> 172.21.15.1:0/1018784081 conn(0x7fd8e80779a0 msgr2=0x7fd8e80fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:59.345 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.344+0000 7fd8ecd49640 1 -- 172.21.15.1:0/1018784081 shutdown_connections 2024-09-11T22:02:59.345 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.344+0000 7fd8ecd49640 1 -- 172.21.15.1:0/1018784081 wait complete. 2024-09-11T22:02:59.347 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.344+0000 7fd8ecd49640 1 Processor -- start 2024-09-11T22:02:59.347 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.344+0000 7fd8ecd49640 1 -- start start 2024-09-11T22:02:59.347 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.345+0000 7fd8ecd49640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8e81009a0 0x7fd8e8198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:59.347 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.345+0000 7fd8ecd49640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd8e8101340 0x7fd8e8198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:59.347 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.345+0000 7fd8ecd49640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8e81069b0 0x7fd8e819ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:59.347 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.345+0000 7fd8ecd49640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e8110df0 con 0x7fd8e8101340 2024-09-11T22:02:59.348 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.345+0000 7fd8ecd49640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e8110c70 con 0x7fd8e81009a0 2024-09-11T22:02:59.348 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.345+0000 7fd8ecd49640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e8110f70 con 0x7fd8e81069b0 2024-09-11T22:02:59.348 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.345+0000 7fd8e5d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd8e8101340 0x7fd8e8198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:59.348 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.345+0000 7fd8e5d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd8e8101340 0x7fd8e8198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59620/0 (socket says 172.21.15.1:59620) 2024-09-11T22:02:59.348 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.345+0000 7fd8e5d9b640 1 -- 172.21.15.1:0/986805869 learned_addr learned my addr 172.21.15.1:0/986805869 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:59.348 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.345+0000 7fd8e659c640 1 --2- 172.21.15.1:0/986805869 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8e81009a0 0x7fd8e8198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:59.348 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.345+0000 7fd8e6d9d640 1 --2- 172.21.15.1:0/986805869 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8e81069b0 0x7fd8e819ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:59.348 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.345+0000 7fd8e5d9b640 1 -- 172.21.15.1:0/986805869 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8e81069b0 msgr2=0x7fd8e819ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:59.348 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.345+0000 7fd8e5d9b640 1 --2- 172.21.15.1:0/986805869 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8e81069b0 0x7fd8e819ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:59.348 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.345+0000 7fd8e5d9b640 1 -- 172.21.15.1:0/986805869 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8e81009a0 msgr2=0x7fd8e8198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:59.348 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.345+0000 7fd8e5d9b640 1 --2- 172.21.15.1:0/986805869 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8e81009a0 0x7fd8e8198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:59.348 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.345+0000 7fd8e5d9b640 1 -- 172.21.15.1:0/986805869 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8e819d3f0 con 0x7fd8e8101340 2024-09-11T22:02:59.349 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.345+0000 7fd8e5d9b640 1 --2- 172.21.15.1:0/986805869 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd8e8101340 0x7fd8e8198940 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc00c890 tx=0x7fd8dc00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:59.349 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.345+0000 7fd8d77fe640 1 -- 172.21.15.1:0/986805869 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc004220 con 0x7fd8e8101340 2024-09-11T22:02:59.349 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.345+0000 7fd8d77fe640 1 -- 172.21.15.1:0/986805869 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8dc0043c0 con 0x7fd8e8101340 2024-09-11T22:02:59.349 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.345+0000 7fd8ecd49640 1 -- 172.21.15.1:0/986805869 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8e819d6e0 con 0x7fd8e8101340 2024-09-11T22:02:59.349 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.345+0000 7fd8d77fe640 1 -- 172.21.15.1:0/986805869 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc0049e0 con 0x7fd8e8101340 2024-09-11T22:02:59.349 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.345+0000 7fd8ecd49640 1 -- 172.21.15.1:0/986805869 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8e81a4f70 con 0x7fd8e8101340 2024-09-11T22:02:59.356 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.347+0000 7fd8d77fe640 1 -- 172.21.15.1:0/986805869 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7fd8dc0079b0 con 0x7fd8e8101340 2024-09-11T22:02:59.356 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.348+0000 7fd8efbbe740 1 Processor -- start 2024-09-11T22:02:59.356 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.348+0000 7fd8efbbe740 1 -- start start 2024-09-11T22:02:59.356 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.348+0000 7fd8efbbe740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56028cb799f0 0x56028cb79dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:59.356 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.348+0000 7fd8efbbe740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56028cb7a310 0x56028cb88b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:59.356 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.348+0000 7fd8efbbe740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56028cb890c0 0x56028cb8b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:59.356 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.348+0000 7fd8efbbe740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56028cade2f0 con 0x56028cb799f0 2024-09-11T22:02:59.356 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.348+0000 7fd8efbbe740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56028ca86270 con 0x56028cb890c0 2024-09-11T22:02:59.356 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.348+0000 7fd8efbbe740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56028cada2b0 con 0x56028cb7a310 2024-09-11T22:02:59.357 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.348+0000 7fd8e6d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56028cb799f0 0x56028cb79dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:59.357 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.348+0000 7fd8e6d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56028cb799f0 0x56028cb79dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59624/0 (socket says 172.21.15.1:59624) 2024-09-11T22:02:59.357 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.348+0000 7fd8e6d9d640 1 -- 172.21.15.1:0/4145470409 learned_addr learned my addr 172.21.15.1:0/4145470409 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:59.357 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.348+0000 7fd8d77fe640 1 --2- 172.21.15.1:0/986805869 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fd8d00a2d70 0x7fd8d00a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:59.357 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.348+0000 7fd8e5d9b640 1 --2- 172.21.15.1:0/4145470409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56028cb890c0 0x56028cb8b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:59.357 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.349+0000 7fd8e659c640 1 --2- 172.21.15.1:0/4145470409 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56028cb7a310 0x56028cb88b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:59.357 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.349+0000 7fd8e6d9d640 1 --2- 172.21.15.1:0/986805869 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fd8d00a2d70 0x7fd8d00a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:59.357 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.349+0000 7fd8e6d9d640 1 -- 172.21.15.1:0/4145470409 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56028cb7a310 msgr2=0x56028cb88b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:59.357 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.349+0000 7fd8d77fe640 1 -- 172.21.15.1:0/986805869 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7fd8dc0cf3c0 con 0x7fd8e8101340 2024-09-11T22:02:59.357 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.349+0000 7fd8e6d9d640 1 --2- 172.21.15.1:0/4145470409 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56028cb7a310 0x56028cb88b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:59.357 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.349+0000 7fd8e6d9d640 1 -- 172.21.15.1:0/4145470409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56028cb890c0 msgr2=0x56028cb8b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:59.357 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.349+0000 7fd8e6d9d640 1 --2- 172.21.15.1:0/4145470409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56028cb890c0 0x56028cb8b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:59.358 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.349+0000 7fd8e6d9d640 1 -- 172.21.15.1:0/4145470409 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56028ca85340 con 0x56028cb799f0 2024-09-11T22:02:59.358 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.349+0000 7fd8e6d9d640 1 --2- 172.21.15.1:0/4145470409 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56028cb799f0 0x56028cb79dd0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fd8e0004ab0 tx=0x7fd8e0031b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d68e4566d2a73158 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:59.358 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.349+0000 7fd8d4ff9640 1 -- 172.21.15.1:0/4145470409 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e0004d40 con 0x56028cb799f0 2024-09-11T22:02:59.358 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.349+0000 7fd8d4ff9640 1 -- 172.21.15.1:0/4145470409 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8e004f070 con 0x56028cb799f0 2024-09-11T22:02:59.358 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.349+0000 7fd8d4ff9640 1 -- 172.21.15.1:0/4145470409 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e0030b40 con 0x56028cb799f0 2024-09-11T22:02:59.358 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.350+0000 7fd8efbbe740 1 -- 172.21.15.1:0/4145470409 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56028cb799f0 msgr2=0x56028cb79dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:59.358 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.350+0000 7fd8e6d9d640 1 --2- 172.21.15.1:0/986805869 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fd8d00a2d70 0x7fd8d00a5230 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd8e004e000 tx=0x7fd8e004c000 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:59.358 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.350+0000 7fd8efbbe740 1 --2- 172.21.15.1:0/4145470409 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56028cb799f0 0x56028cb79dd0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fd8e0004ab0 tx=0x7fd8e0031b00 comp rx=0 tx=0).stop 2024-09-11T22:02:59.358 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.350+0000 7fd8efbbe740 1 -- 172.21.15.1:0/4145470409 shutdown_connections 2024-09-11T22:02:59.358 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.350+0000 7fd8efbbe740 1 --2- 172.21.15.1:0/4145470409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56028cb890c0 0x56028cb8b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:59.358 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.350+0000 7fd8efbbe740 1 --2- 172.21.15.1:0/4145470409 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56028cb7a310 0x56028cb88b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:59.358 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.350+0000 7fd8efbbe740 1 --2- 172.21.15.1:0/4145470409 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56028cb799f0 0x56028cb79dd0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:59.358 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.350+0000 7fd8efbbe740 1 -- 172.21.15.1:0/4145470409 >> 172.21.15.1:0/4145470409 conn(0x56028cb76060 msgr2=0x56028cb76440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:59.359 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.350+0000 7fd8efbbe740 1 -- 172.21.15.1:0/4145470409 shutdown_connections 2024-09-11T22:02:59.359 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.350+0000 7fd8efbbe740 1 -- 172.21.15.1:0/4145470409 wait complete. 2024-09-11T22:02:59.359 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.350+0000 7fd8efbbe740 1 -- start start 2024-09-11T22:02:59.359 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.351+0000 7fd8efbbe740 1 Processor -- start 2024-09-11T22:02:59.359 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.351+0000 7fd8efbbe740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56028cb799f0 0x56028cb86060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:59.359 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.351+0000 7fd8efbbe740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56028cb7a310 0x56028cb865a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:59.359 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.351+0000 7fd8efbbe740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56028cb890c0 0x56028cb86ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:59.359 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.351+0000 7fd8efbbe740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56028cb8bd20 con 0x56028cb7a310 2024-09-11T22:02:59.359 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.351+0000 7fd8efbbe740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56028ca67290 con 0x56028cb799f0 2024-09-11T22:02:59.359 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.351+0000 7fd8efbbe740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56028cb8bea0 con 0x56028cb890c0 2024-09-11T22:02:59.359 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.351+0000 7fd8efbbe740 10 client.0 _ll_register_callbacks cb 0x56028ca98140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:02:59.359 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.351+0000 7fd8e5d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56028cb7a310 0x56028cb865a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:59.360 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.351+0000 7fd8e5d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56028cb7a310 0x56028cb865a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59636/0 (socket says 172.21.15.1:59636) 2024-09-11T22:02:59.360 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.351+0000 7fd8e5d9b640 1 -- 172.21.15.1:0/2094134369 learned_addr learned my addr 172.21.15.1:0/2094134369 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:59.360 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.351+0000 7fd8e6d9d640 1 --2- 172.21.15.1:0/2094134369 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56028cb890c0 0x56028cb86ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:59.360 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.351+0000 7fd8e659c640 1 --2- 172.21.15.1:0/2094134369 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56028cb799f0 0x56028cb86060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:59.360 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.351+0000 7fd8e5d9b640 1 -- 172.21.15.1:0/2094134369 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56028cb890c0 msgr2=0x56028cb86ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:59.360 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.351+0000 7fd8e5d9b640 1 --2- 172.21.15.1:0/2094134369 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56028cb890c0 0x56028cb86ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:59.360 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.351+0000 7fd8e5d9b640 1 -- 172.21.15.1:0/2094134369 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56028cb799f0 msgr2=0x56028cb86060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:59.360 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.351+0000 7fd8e5d9b640 1 --2- 172.21.15.1:0/2094134369 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56028cb799f0 0x56028cb86060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:59.360 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.351+0000 7fd8e5d9b640 1 -- 172.21.15.1:0/2094134369 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56028c7f1210 con 0x56028cb7a310 2024-09-11T22:02:59.360 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.351+0000 7fd8e6d9d640 1 --2- 172.21.15.1:0/2094134369 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56028cb890c0 0x56028cb86ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:02:59.360 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.351+0000 7fd8e659c640 1 --2- 172.21.15.1:0/2094134369 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56028cb799f0 0x56028cb86060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:02:59.360 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.351+0000 7fd8e5d9b640 1 --2- 172.21.15.1:0/2094134369 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56028cb7a310 0x56028cb865a0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc0cd0d0 tx=0x7fd8dc0975c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f3e9a1df0f73e8e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:59.361 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.351+0000 7fd8cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:02:59.361 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.351+0000 7fd8cdffb640 1 -- 172.21.15.1:0/2094134369 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc09b950 con 0x56028cb7a310 2024-09-11T22:02:59.361 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.351+0000 7fd8cdffb640 1 -- 172.21.15.1:0/2094134369 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7fd8dc09feb0 con 0x56028cb7a310 2024-09-11T22:02:59.361 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.351+0000 7fd8efbbe740 1 -- 172.21.15.1:0/2094134369 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8e004f070 con 0x56028cb7a310 2024-09-11T22:02:59.361 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.352+0000 7fd8cdffb640 1 -- 172.21.15.1:0/2094134369 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fd8dc0b9050 con 0x56028cb7a310 2024-09-11T22:02:59.361 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.352+0000 7fd8efbbe740 10 client.16599 fetch_fsmap learned FSMap version 34 2024-09-11T22:02:59.361 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.352+0000 7fd8efbbe740 1 -- 172.21.15.1:0/2094134369 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56028c8bd620 con 0x56028cb7a310 2024-09-11T22:02:59.361 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.352+0000 7fd8cdffb640 1 -- 172.21.15.1:0/2094134369 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8e004f070 con 0x56028cb7a310 2024-09-11T22:02:59.361 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.352+0000 7fd8cdffb640 1 -- 172.21.15.1:0/2094134369 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7fd8dc09cbd0 con 0x56028cb7a310 2024-09-11T22:02:59.361 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.352+0000 7fd8efbbe740 10 client.16599 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:02:59.361 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.352+0000 7fd8efbbe740 10 client.16599 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:02:59.361 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.352+0000 7fd8efbbe740 20 client.16599 populate_metadata read hostname 'smithi001' 2024-09-11T22:02:59.361 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.352+0000 7fd8efbbe740 1 --2- 172.21.15.1:0/2094134369 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x56028cb99160 0x56028cb9b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:59.362 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.352+0000 7fd8efbbe740 4 client.16599 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:02:59.362 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.352+0000 7fd8efbbe740 1 -- 172.21.15.1:0/2094134369 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56028caeba20 con 0x56028cb99160 2024-09-11T22:02:59.362 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.352+0000 7fd8e659c640 1 --2- 172.21.15.1:0/2094134369 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x56028cb99160 0x56028cb9b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:59.362 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.353+0000 7fd8cdffb640 10 client.16599 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:02:59.362 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.353+0000 7fd8e659c640 1 --2- 172.21.15.1:0/2094134369 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x56028cb99160 0x56028cb9b620 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=499c5a17c94d6365 server_cookie=14886872bce160f5 in_seq=0 out_seq=0 2024-09-11T22:02:59.362 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.354+0000 7fd8cdffb640 1 -- 172.21.15.1:0/2094134369 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56028caeba20 con 0x56028cb99160 2024-09-11T22:02:59.362 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.354+0000 7fd8cdffb640 10 client.16599 handle_command_reply: tid=0 2024-09-11T22:02:59.362 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.354+0000 7fd8efbbe740 1 client.16599 shutdown 2024-09-11T22:02:59.362 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.354+0000 7fd8efbbe740 1 -- 172.21.15.1:0/2094134369 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56028cb7a310 msgr2=0x56028cb865a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:59.362 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.354+0000 7fd8efbbe740 1 --2- 172.21.15.1:0/2094134369 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56028cb7a310 0x56028cb865a0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc0cd0d0 tx=0x7fd8dc0975c0 comp rx=0 tx=0).stop 2024-09-11T22:02:59.362 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.354+0000 7fd8efbbe740 1 -- 172.21.15.1:0/2094134369 shutdown_connections 2024-09-11T22:02:59.362 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.354+0000 7fd8efbbe740 1 --2- 172.21.15.1:0/2094134369 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x56028cb99160 0x56028cb9b620 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:59.363 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.354+0000 7fd8efbbe740 1 --2- 172.21.15.1:0/2094134369 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56028cb890c0 0x56028cb86ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:59.363 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.354+0000 7fd8efbbe740 1 --2- 172.21.15.1:0/2094134369 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56028cb7a310 0x56028cb865a0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:59.363 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.354+0000 7fd8cdffb640 0 client.16599 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:02:59.363 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.354+0000 7fd8efbbe740 1 --2- 172.21.15.1:0/2094134369 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56028cb799f0 0x56028cb86060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:59.363 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.354+0000 7fd8efbbe740 1 -- 172.21.15.1:0/2094134369 >> 172.21.15.1:0/2094134369 conn(0x56028cb76060 msgr2=0x56028cb7bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:59.363 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.354+0000 7fd8efbbe740 1 -- 172.21.15.1:0/2094134369 shutdown_connections 2024-09-11T22:02:59.363 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.354+0000 7fd8efbbe740 1 -- 172.21.15.1:0/2094134369 wait complete. 2024-09-11T22:02:59.363 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.354+0000 7fd8efbbe740 20 client.16599 trim_cache size 0 max 16384 2024-09-11T22:02:59.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.369+0000 7fd8efbbe740 1 Processor -- start 2024-09-11T22:02:59.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.369+0000 7fd8efbbe740 1 -- start start 2024-09-11T22:02:59.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.369+0000 7fd8efbbe740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56028cb799f0 0x56028cb86730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:59.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.369+0000 7fd8efbbe740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56028cb7a310 0x56028cba0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:59.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.369+0000 7fd8efbbe740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56028cb890c0 0x56028cba1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:59.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.369+0000 7fd8efbbe740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e81ab790 con 0x56028cb799f0 2024-09-11T22:02:59.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.369+0000 7fd8efbbe740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e81ad390 con 0x56028cb7a310 2024-09-11T22:02:59.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.369+0000 7fd8efbbe740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e81abe60 con 0x56028cb890c0 2024-09-11T22:02:59.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.369+0000 7fd8e659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56028cb799f0 0x56028cb86730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:59.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.369+0000 7fd8e659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56028cb799f0 0x56028cb86730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59642/0 (socket says 172.21.15.1:59642) 2024-09-11T22:02:59.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.369+0000 7fd8e659c640 1 -- 172.21.15.1:0/159570223 learned_addr learned my addr 172.21.15.1:0/159570223 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:59.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.369+0000 7fd8e5d9b640 1 --2- 172.21.15.1:0/159570223 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56028cb7a310 0x56028cba0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:59.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.369+0000 7fd8e659c640 1 -- 172.21.15.1:0/159570223 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56028cb890c0 msgr2=0x56028cba1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:59.375 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.369+0000 7fd8e6d9d640 1 --2- 172.21.15.1:0/159570223 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56028cb890c0 0x56028cba1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:59.376 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.369+0000 7fd8e659c640 1 --2- 172.21.15.1:0/159570223 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56028cb890c0 0x56028cba1470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:59.376 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.369+0000 7fd8e659c640 1 -- 172.21.15.1:0/159570223 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56028cb7a310 msgr2=0x56028cba0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:59.376 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.369+0000 7fd8e659c640 1 --2- 172.21.15.1:0/159570223 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56028cb7a310 0x56028cba0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:59.376 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.369+0000 7fd8e659c640 1 -- 172.21.15.1:0/159570223 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56028cb9bca0 con 0x56028cb799f0 2024-09-11T22:02:59.376 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.370+0000 7fd8e659c640 1 --2- 172.21.15.1:0/159570223 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56028cb799f0 0x56028cb86730 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x56028cb9bed0 tx=0x7fd8d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d973fc4735b62e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:59.376 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.370+0000 7fd8cffff640 1 -- 172.21.15.1:0/159570223 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8d8005020 con 0x56028cb799f0 2024-09-11T22:02:59.376 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.370+0000 7fd8cffff640 1 -- 172.21.15.1:0/159570223 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8d80047a0 con 0x56028cb799f0 2024-09-11T22:02:59.376 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.370+0000 7fd8cffff640 1 -- 172.21.15.1:0/159570223 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8d8010690 con 0x56028cb799f0 2024-09-11T22:02:59.376 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.371+0000 7fd8efbbe740 1 -- 172.21.15.1:0/159570223 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56028cb799f0 msgr2=0x56028cb86730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:59.376 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.371+0000 7fd8efbbe740 1 --2- 172.21.15.1:0/159570223 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56028cb799f0 0x56028cb86730 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x56028cb9bed0 tx=0x7fd8d80046c0 comp rx=0 tx=0).stop 2024-09-11T22:02:59.376 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.371+0000 7fd8efbbe740 1 -- 172.21.15.1:0/159570223 shutdown_connections 2024-09-11T22:02:59.376 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.371+0000 7fd8efbbe740 1 --2- 172.21.15.1:0/159570223 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56028cb890c0 0x56028cba1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:59.377 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.371+0000 7fd8efbbe740 1 --2- 172.21.15.1:0/159570223 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56028cb7a310 0x56028cba0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:59.377 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.371+0000 7fd8efbbe740 1 --2- 172.21.15.1:0/159570223 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56028cb799f0 0x56028cb86730 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:59.377 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.371+0000 7fd8efbbe740 1 -- 172.21.15.1:0/159570223 >> 172.21.15.1:0/159570223 conn(0x56028cb76060 msgr2=0x56028cb86020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:59.377 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.371+0000 7fd8efbbe740 1 -- 172.21.15.1:0/159570223 shutdown_connections 2024-09-11T22:02:59.377 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.371+0000 7fd8efbbe740 1 -- 172.21.15.1:0/159570223 wait complete. 2024-09-11T22:02:59.377 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.371+0000 7fd8efbbe740 1 -- start start 2024-09-11T22:02:59.377 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.371+0000 7fd8efbbe740 1 Processor -- start 2024-09-11T22:02:59.377 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.371+0000 7fd8efbbe740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56028cb799f0 0x56028cb91130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:59.377 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.371+0000 7fd8efbbe740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56028cb7a310 0x56028cb91670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:59.377 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.371+0000 7fd8efbbe740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56028cb890c0 0x56028cb8e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:59.377 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.371+0000 7fd8efbbe740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e81acc00 con 0x56028cb890c0 2024-09-11T22:02:59.377 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.371+0000 7fd8efbbe740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e81ac530 con 0x56028cb7a310 2024-09-11T22:02:59.377 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.371+0000 7fd8efbbe740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e81a9770 con 0x56028cb799f0 2024-09-11T22:02:59.378 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.371+0000 7fd8efbbe740 10 client.0 _ll_register_callbacks cb 0x7fd8dc0058c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:02:59.378 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.371+0000 7fd8e6d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56028cb890c0 0x56028cb8e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:59.378 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.371+0000 7fd8e6d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56028cb890c0 0x56028cb8e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59658/0 (socket says 172.21.15.1:59658) 2024-09-11T22:02:59.378 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.371+0000 7fd8e6d9d640 1 -- 172.21.15.1:0/2542993854 learned_addr learned my addr 172.21.15.1:0/2542993854 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:59.378 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.371+0000 7fd8e659c640 1 --2- 172.21.15.1:0/2542993854 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56028cb799f0 0x56028cb91130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:59.378 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.372+0000 7fd8e5d9b640 1 --2- 172.21.15.1:0/2542993854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56028cb7a310 0x56028cb91670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:59.378 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.372+0000 7fd8e6d9d640 1 -- 172.21.15.1:0/2542993854 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56028cb799f0 msgr2=0x56028cb91130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:59.378 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.372+0000 7fd8e6d9d640 1 --2- 172.21.15.1:0/2542993854 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56028cb799f0 0x56028cb91130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:59.378 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.372+0000 7fd8e6d9d640 1 -- 172.21.15.1:0/2542993854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56028cb7a310 msgr2=0x56028cb91670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:59.378 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.372+0000 7fd8e6d9d640 1 --2- 172.21.15.1:0/2542993854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56028cb7a310 0x56028cb91670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:59.378 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.372+0000 7fd8e6d9d640 1 -- 172.21.15.1:0/2542993854 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56028cb8e640 con 0x56028cb890c0 2024-09-11T22:02:59.378 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.372+0000 7fd8e6d9d640 1 --2- 172.21.15.1:0/2542993854 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56028cb890c0 0x56028cb8e100 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fd8e002ae00 tx=0x7fd8e0030fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65c6d0f1a865aaf3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:59.379 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.372+0000 7fd8cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:02:59.379 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.372+0000 7fd8cdffb640 1 -- 172.21.15.1:0/2542993854 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e0004d50 con 0x56028cb890c0 2024-09-11T22:02:59.379 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.372+0000 7fd8cdffb640 1 -- 172.21.15.1:0/2542993854 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7fd8e002e040 con 0x56028cb890c0 2024-09-11T22:02:59.379 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.372+0000 7fd8efbbe740 1 -- 172.21.15.1:0/2542993854 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8d80047a0 con 0x56028cb890c0 2024-09-11T22:02:59.379 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.372+0000 7fd8cdffb640 1 -- 172.21.15.1:0/2542993854 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fd8e0004410 con 0x56028cb890c0 2024-09-11T22:02:59.379 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.372+0000 7fd8efbbe740 10 client.16605 fetch_fsmap learned FSMap version 34 2024-09-11T22:02:59.379 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.372+0000 7fd8efbbe740 1 -- 172.21.15.1:0/2542993854 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56028cc045d0 con 0x56028cb890c0 2024-09-11T22:02:59.379 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.372+0000 7fd8cdffb640 1 -- 172.21.15.1:0/2542993854 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8d80047a0 con 0x56028cb890c0 2024-09-11T22:02:59.379 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.372+0000 7fd8cdffb640 1 -- 172.21.15.1:0/2542993854 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7fd8e0030070 con 0x56028cb890c0 2024-09-11T22:02:59.379 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.373+0000 7fd8efbbe740 10 client.16605 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:02:59.379 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.373+0000 7fd8efbbe740 10 client.16605 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:02:59.379 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.373+0000 7fd8efbbe740 20 client.16605 populate_metadata read hostname 'smithi001' 2024-09-11T22:02:59.379 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.373+0000 7fd8efbbe740 1 --2- 172.21.15.1:0/2542993854 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x56028cc04970 0x56028cc06d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:59.380 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.373+0000 7fd8efbbe740 4 client.16605 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:02:59.380 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.373+0000 7fd8efbbe740 1 -- 172.21.15.1:0/2542993854 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56028cae5d70 con 0x56028cc04970 2024-09-11T22:02:59.380 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.373+0000 7fd8e659c640 1 --2- 172.21.15.1:0/2542993854 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x56028cc04970 0x56028cc06d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:59.380 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.373+0000 7fd8e659c640 1 --2- 172.21.15.1:0/2542993854 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x56028cc04970 0x56028cc06d60 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=339df514c0134f51 server_cookie=17e268fcc32c2a49 in_seq=0 out_seq=0 2024-09-11T22:02:59.380 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.373+0000 7fd8cdffb640 10 client.16605 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:02:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.523+0000 7fd8cdffb640 1 -- 172.21.15.1:0/2542993854 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56028cae5d70 con 0x56028cc04970 2024-09-11T22:02:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.523+0000 7fd8cdffb640 10 client.16605 handle_command_reply: tid=0 2024-09-11T22:02:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.523+0000 7fd8efbbe740 1 client.16605 shutdown 2024-09-11T22:02:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.523+0000 7fd8efbbe740 1 -- 172.21.15.1:0/2542993854 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56028cb890c0 msgr2=0x56028cb8e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.523+0000 7fd8efbbe740 1 --2- 172.21.15.1:0/2542993854 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56028cb890c0 0x56028cb8e100 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fd8e002ae00 tx=0x7fd8e0030fd0 comp rx=0 tx=0).stop 2024-09-11T22:02:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.523+0000 7fd8efbbe740 1 -- 172.21.15.1:0/2542993854 shutdown_connections 2024-09-11T22:02:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.523+0000 7fd8efbbe740 1 --2- 172.21.15.1:0/2542993854 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x56028cc04970 0x56028cc06d60 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.523+0000 7fd8efbbe740 1 --2- 172.21.15.1:0/2542993854 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56028cb890c0 0x56028cb8e100 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.523+0000 7fd8cdffb640 0 client.16605 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:02:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.523+0000 7fd8efbbe740 1 --2- 172.21.15.1:0/2542993854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56028cb7a310 0x56028cb91670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.523+0000 7fd8efbbe740 1 --2- 172.21.15.1:0/2542993854 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56028cb799f0 0x56028cb91130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.523+0000 7fd8efbbe740 1 -- 172.21.15.1:0/2542993854 >> 172.21.15.1:0/2542993854 conn(0x56028cb76060 msgr2=0x56028cb86840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.523+0000 7fd8efbbe740 1 -- 172.21.15.1:0/2542993854 shutdown_connections 2024-09-11T22:02:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.523+0000 7fd8efbbe740 1 -- 172.21.15.1:0/2542993854 wait complete. 2024-09-11T22:02:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.523+0000 7fd8efbbe740 20 client.16605 trim_cache size 0 max 16384 2024-09-11T22:02:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-09-11T22:02:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-09-11T22:02:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-09-11T22:02:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-09-11T22:02:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.524+0000 7fd8ecd49640 1 -- 172.21.15.1:0/986805869 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fd8d00a2d70 msgr2=0x7fd8d00a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.524+0000 7fd8ecd49640 1 --2- 172.21.15.1:0/986805869 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fd8d00a2d70 0x7fd8d00a5230 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd8e004e000 tx=0x7fd8e004c000 comp rx=0 tx=0).stop 2024-09-11T22:02:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.524+0000 7fd8ecd49640 1 -- 172.21.15.1:0/986805869 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd8e8101340 msgr2=0x7fd8e8198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.524+0000 7fd8ecd49640 1 --2- 172.21.15.1:0/986805869 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd8e8101340 0x7fd8e8198940 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc00c890 tx=0x7fd8dc00cd50 comp rx=0 tx=0).stop 2024-09-11T22:02:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.524+0000 7fd8ecd49640 1 -- 172.21.15.1:0/986805869 shutdown_connections 2024-09-11T22:02:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.524+0000 7fd8ecd49640 1 --2- 172.21.15.1:0/986805869 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fd8d00a2d70 0x7fd8d00a5230 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.524+0000 7fd8ecd49640 1 --2- 172.21.15.1:0/986805869 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8e81069b0 0x7fd8e819ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.524+0000 7fd8ecd49640 1 --2- 172.21.15.1:0/986805869 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd8e8101340 0x7fd8e8198940 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.524+0000 7fd8ecd49640 1 --2- 172.21.15.1:0/986805869 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8e81009a0 0x7fd8e8198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.524+0000 7fd8ecd49640 1 -- 172.21.15.1:0/986805869 >> 172.21.15.1:0/986805869 conn(0x7fd8e80779a0 msgr2=0x7fd8e8069d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:59.528 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.524+0000 7fd8ecd49640 1 -- 172.21.15.1:0/986805869 shutdown_connections 2024-09-11T22:02:59.528 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:02:59.524+0000 7fd8ecd49640 1 -- 172.21.15.1:0/986805869 wait complete. 2024-09-11T22:02:59.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.729+0000 7f810831d640 1 -- 172.21.15.1:0/2608851886 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8100106f10 msgr2=0x7f810010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:59.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.729+0000 7f810831d640 1 --2- 172.21.15.1:0/2608851886 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8100106f10 0x7f810010ea90 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f80f8009ca0 tx=0x7f80f802a8c0 comp rx=0 tx=0).stop 2024-09-11T22:02:59.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.729+0000 7f810831d640 1 -- 172.21.15.1:0/2608851886 shutdown_connections 2024-09-11T22:02:59.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.729+0000 7f810831d640 1 --2- 172.21.15.1:0/2608851886 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8100106f10 0x7f810010ea90 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:59.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.729+0000 7f810831d640 1 --2- 172.21.15.1:0/2608851886 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8100102790 0x7f81001067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:59.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.729+0000 7f810831d640 1 --2- 172.21.15.1:0/2608851886 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8100101de0 0x7f81001021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:59.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.729+0000 7f810831d640 1 -- 172.21.15.1:0/2608851886 >> 172.21.15.1:0/2608851886 conn(0x7f81000fd8c0 msgr2=0x7f81000ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:59.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.729+0000 7f810831d640 1 -- 172.21.15.1:0/2608851886 shutdown_connections 2024-09-11T22:02:59.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.729+0000 7f810831d640 1 -- 172.21.15.1:0/2608851886 wait complete. 2024-09-11T22:02:59.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.730+0000 7f810831d640 1 Processor -- start 2024-09-11T22:02:59.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.730+0000 7f810831d640 1 -- start start 2024-09-11T22:02:59.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.730+0000 7f810831d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8100101de0 0x7f81001983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:59.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.730+0000 7f810831d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8100102790 0x7f81001988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:59.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.730+0000 7f810831d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8100106f10 0x7f810019cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:59.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.730+0000 7f810831d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8100111f70 con 0x7f8100101de0 2024-09-11T22:02:59.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.730+0000 7f810831d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8100111df0 con 0x7f8100102790 2024-09-11T22:02:59.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.730+0000 7f810831d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f81001120f0 con 0x7f8100106f10 2024-09-11T22:02:59.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.730+0000 7f81060b9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8100101de0 0x7f81001983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:59.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.730+0000 7f81060b9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8100101de0 0x7f81001983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59662/0 (socket says 172.21.15.1:59662) 2024-09-11T22:02:59.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.730+0000 7f81060b9640 1 -- 172.21.15.1:0/1172676970 learned_addr learned my addr 172.21.15.1:0/1172676970 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:59.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.730+0000 7f81058b8640 1 --2- 172.21.15.1:0/1172676970 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8100102790 0x7f81001988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:59.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.730+0000 7f81068ba640 1 --2- 172.21.15.1:0/1172676970 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8100106f10 0x7f810019cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:59.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.730+0000 7f81060b9640 1 -- 172.21.15.1:0/1172676970 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8100106f10 msgr2=0x7f810019cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:59.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.730+0000 7f81060b9640 1 --2- 172.21.15.1:0/1172676970 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8100106f10 0x7f810019cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:59.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.730+0000 7f81060b9640 1 -- 172.21.15.1:0/1172676970 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8100102790 msgr2=0x7f81001988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:59.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.730+0000 7f81060b9640 1 --2- 172.21.15.1:0/1172676970 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8100102790 0x7f81001988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:59.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.730+0000 7f81060b9640 1 -- 172.21.15.1:0/1172676970 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f810019d360 con 0x7f8100101de0 2024-09-11T22:02:59.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.731+0000 7f81060b9640 1 --2- 172.21.15.1:0/1172676970 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8100101de0 0x7f81001983b0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f80fc00d950 tx=0x7f80fc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:59.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.731+0000 7f80f77fe640 1 -- 172.21.15.1:0/1172676970 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80fc017030 con 0x7f8100101de0 2024-09-11T22:02:59.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.731+0000 7f80f77fe640 1 -- 172.21.15.1:0/1172676970 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80fc004020 con 0x7f8100101de0 2024-09-11T22:02:59.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.731+0000 7f810831d640 1 -- 172.21.15.1:0/1172676970 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f810019d650 con 0x7f8100101de0 2024-09-11T22:02:59.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.731+0000 7f80f77fe640 1 -- 172.21.15.1:0/1172676970 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80fc0047a0 con 0x7f8100101de0 2024-09-11T22:02:59.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.731+0000 7f810831d640 1 -- 172.21.15.1:0/1172676970 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81001a4f30 con 0x7f8100101de0 2024-09-11T22:02:59.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.733+0000 7f80f77fe640 1 -- 172.21.15.1:0/1172676970 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f80fc0041d0 con 0x7f8100101de0 2024-09-11T22:02:59.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.733+0000 7f810a6f6740 1 Processor -- start 2024-09-11T22:02:59.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.734+0000 7f810a6f6740 1 -- start start 2024-09-11T22:02:59.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.734+0000 7f810a6f6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639e5824f20 0x5639e5825300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:59.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.734+0000 7f810a6f6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5639e5825840 0x5639e5834110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:59.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.734+0000 7f810a6f6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639e5834650 0x5639e5836a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:59.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.734+0000 7f810a6f6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5639e578a620 con 0x5639e5825840 2024-09-11T22:02:59.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.734+0000 7f810a6f6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5639e5731e50 con 0x5639e5834650 2024-09-11T22:02:59.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.734+0000 7f810a6f6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5639e57865e0 con 0x5639e5824f20 2024-09-11T22:02:59.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.734+0000 7f81060b9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5639e5825840 0x5639e5834110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:59.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.734+0000 7f81068ba640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639e5824f20 0x5639e5825300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:59.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.734+0000 7f81058b8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639e5834650 0x5639e5836a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:59.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.734+0000 7f81068ba640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639e5824f20 0x5639e5825300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:43758/0 (socket says 172.21.15.1:43758) 2024-09-11T22:02:59.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.734+0000 7f81060b9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5639e5825840 0x5639e5834110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59664/0 (socket says 172.21.15.1:59664) 2024-09-11T22:02:59.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.734+0000 7f81068ba640 1 -- 172.21.15.1:0/3430016342 learned_addr learned my addr 172.21.15.1:0/3430016342 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:59.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.734+0000 7f80f77fe640 1 --2- 172.21.15.1:0/1172676970 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f80ec0a2d70 0x7f80ec0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:59.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.734+0000 7f81068ba640 1 --2- 172.21.15.1:0/1172676970 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f80ec0a2d70 0x7f80ec0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:59.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.735+0000 7f80f77fe640 1 -- 172.21.15.1:0/1172676970 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f80fc0ced60 con 0x7f8100101de0 2024-09-11T22:02:59.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.735+0000 7f81058b8640 1 -- 172.21.15.1:0/3430016342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639e5824f20 msgr2=0x5639e5825300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:59.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.735+0000 7f81058b8640 1 --2- 172.21.15.1:0/3430016342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639e5824f20 0x5639e5825300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:59.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.735+0000 7f81058b8640 1 -- 172.21.15.1:0/3430016342 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5639e5825840 msgr2=0x5639e5834110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:59.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.735+0000 7f81058b8640 1 --2- 172.21.15.1:0/3430016342 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5639e5825840 0x5639e5834110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:59.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.735+0000 7f81058b8640 1 -- 172.21.15.1:0/3430016342 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639e5722d40 con 0x5639e5834650 2024-09-11T22:02:59.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.735+0000 7f81068ba640 1 --2- 172.21.15.1:0/3430016342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639e5824f20 0x5639e5825300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:02:59.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.735+0000 7f81058b8640 1 --2- 172.21.15.1:0/3430016342 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639e5834650 0x5639e5836a40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f80f0005fd0 tx=0x7f80f000ba60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f94352c780d8668 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:59.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.735+0000 7f81068ba640 1 --2- 172.21.15.1:0/1172676970 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f80ec0a2d70 0x7f80ec0a5230 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f80f8005fe0 tx=0x7f80f802a8c0 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:59.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.735+0000 7f80f4dea640 1 -- 172.21.15.1:0/3430016342 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f0003a90 con 0x5639e5834650 2024-09-11T22:02:59.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.735+0000 7f80f4dea640 1 -- 172.21.15.1:0/3430016342 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80f0015070 con 0x5639e5834650 2024-09-11T22:02:59.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.735+0000 7f80f4dea640 1 -- 172.21.15.1:0/3430016342 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f0004cb0 con 0x5639e5834650 2024-09-11T22:02:59.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.736+0000 7f810a6f6740 1 -- 172.21.15.1:0/3430016342 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639e5834650 msgr2=0x5639e5836a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:59.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.736+0000 7f810a6f6740 1 --2- 172.21.15.1:0/3430016342 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639e5834650 0x5639e5836a40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f80f0005fd0 tx=0x7f80f000ba60 comp rx=0 tx=0).stop 2024-09-11T22:02:59.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.736+0000 7f810a6f6740 1 -- 172.21.15.1:0/3430016342 shutdown_connections 2024-09-11T22:02:59.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.736+0000 7f810a6f6740 1 --2- 172.21.15.1:0/3430016342 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639e5834650 0x5639e5836a40 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:59.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.736+0000 7f810a6f6740 1 --2- 172.21.15.1:0/3430016342 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5639e5825840 0x5639e5834110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:59.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.736+0000 7f810a6f6740 1 --2- 172.21.15.1:0/3430016342 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639e5824f20 0x5639e5825300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:59.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.736+0000 7f810a6f6740 1 -- 172.21.15.1:0/3430016342 >> 172.21.15.1:0/3430016342 conn(0x5639e58215f0 msgr2=0x5639e58219d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:59.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.736+0000 7f810a6f6740 1 -- 172.21.15.1:0/3430016342 shutdown_connections 2024-09-11T22:02:59.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.736+0000 7f810a6f6740 1 -- 172.21.15.1:0/3430016342 wait complete. 2024-09-11T22:02:59.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.736+0000 7f810a6f6740 1 -- start start 2024-09-11T22:02:59.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.736+0000 7f810a6f6740 1 Processor -- start 2024-09-11T22:02:59.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.737+0000 7f810a6f6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5639e5824f20 0x5639e58315d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:59.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.737+0000 7f810a6f6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639e5825840 0x5639e5831b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:59.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.737+0000 7f810a6f6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639e5834650 0x5639e5832050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:59.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.737+0000 7f810a6f6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5639e5837220 con 0x5639e5824f20 2024-09-11T22:02:59.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.737+0000 7f810a6f6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5639e5712f10 con 0x5639e5825840 2024-09-11T22:02:59.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.737+0000 7f810a6f6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5639e58373a0 con 0x5639e5834650 2024-09-11T22:02:59.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.737+0000 7f810a6f6740 10 client.0 _ll_register_callbacks cb 0x5639e5736db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:02:59.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.737+0000 7f81060b9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5639e5824f20 0x5639e58315d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:59.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.737+0000 7f81058b8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639e5825840 0x5639e5831b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:59.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.737+0000 7f81060b9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5639e5824f20 0x5639e58315d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59674/0 (socket says 172.21.15.1:59674) 2024-09-11T22:02:59.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.737+0000 7f81060b9640 1 -- 172.21.15.1:0/445887413 learned_addr learned my addr 172.21.15.1:0/445887413 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:59.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.737+0000 7f81068ba640 1 --2- 172.21.15.1:0/445887413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639e5834650 0x5639e5832050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:59.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.737+0000 7f81058b8640 1 -- 172.21.15.1:0/445887413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639e5834650 msgr2=0x5639e5832050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:59.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.737+0000 7f81058b8640 1 --2- 172.21.15.1:0/445887413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639e5834650 0x5639e5832050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:59.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.737+0000 7f81058b8640 1 -- 172.21.15.1:0/445887413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5639e5824f20 msgr2=0x5639e58315d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:59.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.737+0000 7f81058b8640 1 --2- 172.21.15.1:0/445887413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5639e5824f20 0x5639e58315d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:59.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.737+0000 7f81058b8640 1 -- 172.21.15.1:0/445887413 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639e5499580 con 0x5639e5825840 2024-09-11T22:02:59.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.737+0000 7f81068ba640 1 --2- 172.21.15.1:0/445887413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639e5834650 0x5639e5832050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:02:59.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.737+0000 7f81060b9640 1 --2- 172.21.15.1:0/445887413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5639e5824f20 0x5639e58315d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:02:59.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.737+0000 7f81058b8640 1 --2- 172.21.15.1:0/445887413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639e5825840 0x5639e5831b10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f80f000b5f0 tx=0x7f80f0003ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d309f9af9baa6431 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:59.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.737+0000 7f80e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-11T22:02:59.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.738+0000 7f80e9ffb640 1 -- 172.21.15.1:0/445887413 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f0028060 con 0x5639e5825840 2024-09-11T22:02:59.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.738+0000 7f810a6f6740 1 -- 172.21.15.1:0/445887413 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80f0015070 con 0x5639e5825840 2024-09-11T22:02:59.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.738+0000 7f80e9ffb640 1 -- 172.21.15.1:0/445887413 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f80f0011640 con 0x5639e5825840 2024-09-11T22:02:59.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.738+0000 7f80e9ffb640 1 -- 172.21.15.1:0/445887413 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f80f0011840 con 0x5639e5825840 2024-09-11T22:02:59.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.738+0000 7f810a6f6740 10 client.26150 fetch_fsmap learned FSMap version 34 2024-09-11T22:02:59.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.738+0000 7f810a6f6740 1 -- 172.21.15.1:0/445887413 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5639e5563960 con 0x5639e5825840 2024-09-11T22:02:59.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.738+0000 7f80e9ffb640 1 -- 172.21.15.1:0/445887413 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80f0015070 con 0x5639e5825840 2024-09-11T22:02:59.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.738+0000 7f80e9ffb640 1 -- 172.21.15.1:0/445887413 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7f80f0022070 con 0x5639e5825840 2024-09-11T22:02:59.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.738+0000 7f810a6f6740 10 client.26150 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:02:59.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.738+0000 7f810a6f6740 10 client.26150 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-11T22:02:59.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.738+0000 7f810a6f6740 20 client.26150 populate_metadata read hostname 'smithi001' 2024-09-11T22:02:59.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.738+0000 7f810a6f6740 1 --2- 172.21.15.1:0/445887413 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5639e58441d0 0x5639e58465c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:59.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.738+0000 7f810a6f6740 4 client.26150 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:02:59.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.738+0000 7f810a6f6740 1 -- 172.21.15.1:0/445887413 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5639e5797b90 con 0x5639e58441d0 2024-09-11T22:02:59.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.739+0000 7f81060b9640 1 --2- 172.21.15.1:0/445887413 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5639e58441d0 0x5639e58465c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:59.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.739+0000 7f81060b9640 1 --2- 172.21.15.1:0/445887413 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5639e58441d0 0x5639e58465c0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2264f7fc1c16496c server_cookie=ccb7e58c47a260f2 in_seq=0 out_seq=0 2024-09-11T22:02:59.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.739+0000 7f80e9ffb640 10 client.26150 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:02:59.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.739+0000 7f80e9ffb640 1 -- 172.21.15.1:0/445887413 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5639e5797b90 con 0x5639e58441d0 2024-09-11T22:02:59.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.739+0000 7f80e9ffb640 10 client.26150 handle_command_reply: tid=0 2024-09-11T22:02:59.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.740+0000 7f810a6f6740 1 client.26150 shutdown 2024-09-11T22:02:59.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.740+0000 7f810a6f6740 1 -- 172.21.15.1:0/445887413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639e5825840 msgr2=0x5639e5831b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:59.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.740+0000 7f810a6f6740 1 --2- 172.21.15.1:0/445887413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639e5825840 0x5639e5831b10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f80f000b5f0 tx=0x7f80f0003ac0 comp rx=0 tx=0).stop 2024-09-11T22:02:59.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.740+0000 7f810a6f6740 1 -- 172.21.15.1:0/445887413 shutdown_connections 2024-09-11T22:02:59.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.740+0000 7f810a6f6740 1 --2- 172.21.15.1:0/445887413 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5639e58441d0 0x5639e58465c0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:59.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.740+0000 7f810a6f6740 1 --2- 172.21.15.1:0/445887413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639e5834650 0x5639e5832050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:59.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.740+0000 7f810a6f6740 1 --2- 172.21.15.1:0/445887413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639e5825840 0x5639e5831b10 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:59.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.740+0000 7f80e9ffb640 0 client.26150 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:02:59.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.740+0000 7f810a6f6740 1 --2- 172.21.15.1:0/445887413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5639e5824f20 0x5639e58315d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:59.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.740+0000 7f810a6f6740 1 -- 172.21.15.1:0/445887413 >> 172.21.15.1:0/445887413 conn(0x5639e58215f0 msgr2=0x5639e5827350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:59.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.740+0000 7f810a6f6740 1 -- 172.21.15.1:0/445887413 shutdown_connections 2024-09-11T22:02:59.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.740+0000 7f810a6f6740 1 -- 172.21.15.1:0/445887413 wait complete. 2024-09-11T22:02:59.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.740+0000 7f810a6f6740 20 client.26150 trim_cache size 0 max 16384 2024-09-11T22:02:59.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.757+0000 7f810a6f6740 1 Processor -- start 2024-09-11T22:02:59.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.757+0000 7f810a6f6740 1 -- start start 2024-09-11T22:02:59.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.757+0000 7f810a6f6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5639e5824f20 0x5639e5833540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:59.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.757+0000 7f810a6f6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639e5825840 0x5639e584bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:59.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.757+0000 7f810a6f6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639e5834650 0x5639e584c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:59.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.757+0000 7f810a6f6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f81001ab7c0 con 0x5639e5824f20 2024-09-11T22:02:59.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.757+0000 7f810a6f6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f81001ad3c0 con 0x5639e5825840 2024-09-11T22:02:59.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.757+0000 7f810a6f6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f81001abe90 con 0x5639e5834650 2024-09-11T22:02:59.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.757+0000 7f81060b9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5639e5824f20 0x5639e5833540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:59.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.757+0000 7f81060b9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5639e5824f20 0x5639e5833540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59690/0 (socket says 172.21.15.1:59690) 2024-09-11T22:02:59.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.757+0000 7f81060b9640 1 -- 172.21.15.1:0/3927625010 learned_addr learned my addr 172.21.15.1:0/3927625010 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:59.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.757+0000 7f81068ba640 1 --2- 172.21.15.1:0/3927625010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639e5834650 0x5639e584c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:59.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.757+0000 7f81058b8640 1 --2- 172.21.15.1:0/3927625010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639e5825840 0x5639e584bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:59.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.757+0000 7f81060b9640 1 -- 172.21.15.1:0/3927625010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639e5834650 msgr2=0x5639e584c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:59.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.757+0000 7f81060b9640 1 --2- 172.21.15.1:0/3927625010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639e5834650 0x5639e584c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:59.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.757+0000 7f81060b9640 1 -- 172.21.15.1:0/3927625010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639e5825840 msgr2=0x5639e584bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:59.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.757+0000 7f81060b9640 1 --2- 172.21.15.1:0/3927625010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639e5825840 0x5639e584bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:59.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.757+0000 7f81060b9640 1 -- 172.21.15.1:0/3927625010 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639e5837360 con 0x5639e5824f20 2024-09-11T22:02:59.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.758+0000 7f81060b9640 1 --2- 172.21.15.1:0/3927625010 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5639e5824f20 0x5639e5833540 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f80fc005250 tx=0x7f80fc0bb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d851d67a689252f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:59.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.758+0000 7f80ebfff640 1 -- 172.21.15.1:0/3927625010 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80fc09b940 con 0x5639e5824f20 2024-09-11T22:02:59.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.758+0000 7f80ebfff640 1 -- 172.21.15.1:0/3927625010 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80fc09b240 con 0x5639e5824f20 2024-09-11T22:02:59.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.758+0000 7f80ebfff640 1 -- 172.21.15.1:0/3927625010 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80fc0cc880 con 0x5639e5824f20 2024-09-11T22:02:59.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.759+0000 7f810a6f6740 1 -- 172.21.15.1:0/3927625010 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5639e5824f20 msgr2=0x5639e5833540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:02:59.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.759+0000 7f810a6f6740 1 --2- 172.21.15.1:0/3927625010 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5639e5824f20 0x5639e5833540 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f80fc005250 tx=0x7f80fc0bb040 comp rx=0 tx=0).stop 2024-09-11T22:02:59.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.759+0000 7f810a6f6740 1 -- 172.21.15.1:0/3927625010 shutdown_connections 2024-09-11T22:02:59.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.759+0000 7f810a6f6740 1 --2- 172.21.15.1:0/3927625010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639e5834650 0x5639e584c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:59.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.759+0000 7f810a6f6740 1 --2- 172.21.15.1:0/3927625010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639e5825840 0x5639e584bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:59.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.759+0000 7f810a6f6740 1 --2- 172.21.15.1:0/3927625010 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5639e5824f20 0x5639e5833540 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:59.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.759+0000 7f810a6f6740 1 -- 172.21.15.1:0/3927625010 >> 172.21.15.1:0/3927625010 conn(0x5639e58215f0 msgr2=0x5639e5832e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:02:59.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.759+0000 7f810a6f6740 1 -- 172.21.15.1:0/3927625010 shutdown_connections 2024-09-11T22:02:59.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.759+0000 7f810a6f6740 1 -- 172.21.15.1:0/3927625010 wait complete. 2024-09-11T22:02:59.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.759+0000 7f810a6f6740 1 -- start start 2024-09-11T22:02:59.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.759+0000 7f810a6f6740 1 Processor -- start 2024-09-11T22:02:59.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.759+0000 7f810a6f6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639e5824f20 0x5639e5838c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:59.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.759+0000 7f810a6f6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5639e5825840 0x5639e58391a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:59.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.760+0000 7f810a6f6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639e5834650 0x5639e58a7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:59.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.760+0000 7f810a6f6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f81001acc30 con 0x5639e5825840 2024-09-11T22:02:59.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.760+0000 7f810a6f6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f81001ac560 con 0x5639e5834650 2024-09-11T22:02:59.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.760+0000 7f810a6f6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f81001a97a0 con 0x5639e5824f20 2024-09-11T22:02:59.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.760+0000 7f810a6f6740 10 client.0 _ll_register_callbacks cb 0x7f80f0004430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:02:59.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.760+0000 7f81058b8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5639e5825840 0x5639e58391a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:59.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.760+0000 7f81068ba640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639e5834650 0x5639e58a7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:59.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.760+0000 7f81068ba640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639e5834650 0x5639e58a7c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:47298/0 (socket says 172.21.15.1:47298) 2024-09-11T22:02:59.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.760+0000 7f81060b9640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639e5824f20 0x5639e5838c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:59.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.760+0000 7f81068ba640 1 -- 172.21.15.1:0/3931316845 learned_addr learned my addr 172.21.15.1:0/3931316845 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:02:59.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.760+0000 7f81058b8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5639e5825840 0x5639e58391a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59696/0 (socket says 172.21.15.1:59696) 2024-09-11T22:02:59.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.760+0000 7f81068ba640 1 -- 172.21.15.1:0/3931316845 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639e5824f20 msgr2=0x5639e5838c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:59.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.760+0000 7f81068ba640 1 --2- 172.21.15.1:0/3931316845 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639e5824f20 0x5639e5838c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:59.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.760+0000 7f81068ba640 1 -- 172.21.15.1:0/3931316845 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5639e5825840 msgr2=0x5639e58391a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:02:59.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.760+0000 7f81068ba640 1 --2- 172.21.15.1:0/3931316845 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5639e5825840 0x5639e58391a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:02:59.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.760+0000 7f81068ba640 1 -- 172.21.15.1:0/3931316845 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639e58a8240 con 0x5639e5834650 2024-09-11T22:02:59.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.760+0000 7f81068ba640 1 --2- 172.21.15.1:0/3931316845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639e5834650 0x5639e58a7c00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f80f8005e30 tx=0x7f80f80095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52f4b693342b5b2e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:02:59.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.760+0000 7f80e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-11T22:02:59.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.761+0000 7f80e9ffb640 1 -- 172.21.15.1:0/3931316845 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f80305c0 con 0x5639e5834650 2024-09-11T22:02:59.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.761+0000 7f80e9ffb640 1 -- 172.21.15.1:0/3931316845 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f80f8030760 con 0x5639e5834650 2024-09-11T22:02:59.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.761+0000 7f810a6f6740 1 -- 172.21.15.1:0/3931316845 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80fc09b240 con 0x5639e5834650 2024-09-11T22:02:59.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.761+0000 7f80e9ffb640 1 -- 172.21.15.1:0/3931316845 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f80f8030960 con 0x5639e5834650 2024-09-11T22:02:59.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.761+0000 7f810a6f6740 10 client.26156 fetch_fsmap learned FSMap version 34 2024-09-11T22:02:59.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.761+0000 7f810a6f6740 1 -- 172.21.15.1:0/3931316845 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5639e58af5f0 con 0x5639e5834650 2024-09-11T22:02:59.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.761+0000 7f80e9ffb640 1 -- 172.21.15.1:0/3931316845 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80fc09b240 con 0x5639e5834650 2024-09-11T22:02:59.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.761+0000 7f80e9ffb640 1 -- 172.21.15.1:0/3931316845 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7f80f804a040 con 0x5639e5834650 2024-09-11T22:02:59.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.761+0000 7f810a6f6740 10 client.26156 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:02:59.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.761+0000 7f810a6f6740 10 client.26156 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-11T22:02:59.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.761+0000 7f810a6f6740 20 client.26156 populate_metadata read hostname 'smithi001' 2024-09-11T22:02:59.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.761+0000 7f810a6f6740 1 --2- 172.21.15.1:0/3931316845 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5639e58afe00 0x5639e58b2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:02:59.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.761+0000 7f810a6f6740 4 client.26156 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "125e9d94", "release": true, "await": true}] 2024-09-11T22:02:59.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.761+0000 7f810a6f6740 1 -- 172.21.15.1:0/3931316845 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "quiesce db", "set_id": "125e9d94", "release": true, "await": true}) -- 0x5639e57920a0 con 0x5639e58afe00 2024-09-11T22:02:59.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.761+0000 7f81060b9640 1 --2- 172.21.15.1:0/3931316845 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5639e58afe00 0x5639e58b2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:02:59.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.762+0000 7f81060b9640 1 --2- 172.21.15.1:0/3931316845 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5639e58afe00 0x5639e58b2220 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c00f14dfbc0dbc99 server_cookie=c0fa32dfd05e698a in_seq=0 out_seq=0 2024-09-11T22:02:59.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:02:59.762+0000 7f80e9ffb640 10 client.26156 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:03:00.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:00.450+0000 7f80e9ffb640 1 -- 172.21.15.1:0/3931316845 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5639e57920a0 con 0x5639e58afe00 2024-09-11T22:03:00.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:00.450+0000 7f80e9ffb640 10 client.26156 handle_command_reply: tid=0 2024-09-11T22:03:00.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:00.450+0000 7f810a6f6740 1 client.26156 shutdown 2024-09-11T22:03:00.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:00.451+0000 7f810a6f6740 1 -- 172.21.15.1:0/3931316845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639e5834650 msgr2=0x5639e58a7c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:00.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:00.451+0000 7f810a6f6740 1 --2- 172.21.15.1:0/3931316845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639e5834650 0x5639e58a7c00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f80f8005e30 tx=0x7f80f80095c0 comp rx=0 tx=0).stop 2024-09-11T22:03:00.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:00.451+0000 7f810a6f6740 1 -- 172.21.15.1:0/3931316845 shutdown_connections 2024-09-11T22:03:00.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:00.451+0000 7f810a6f6740 1 --2- 172.21.15.1:0/3931316845 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5639e58afe00 0x5639e58b2220 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:00.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:00.451+0000 7f810a6f6740 1 --2- 172.21.15.1:0/3931316845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5639e5834650 0x5639e58a7c00 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:00.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:00.451+0000 7f810a6f6740 1 --2- 172.21.15.1:0/3931316845 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5639e5825840 0x5639e58391a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:00.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:00.451+0000 7f80e9ffb640 0 client.26156 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:03:00.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:00.451+0000 7f810a6f6740 1 --2- 172.21.15.1:0/3931316845 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5639e5824f20 0x5639e5838c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:00.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:00.451+0000 7f810a6f6740 1 -- 172.21.15.1:0/3931316845 >> 172.21.15.1:0/3931316845 conn(0x5639e58215f0 msgr2=0x5639e5833650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:00.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:00.451+0000 7f810a6f6740 1 -- 172.21.15.1:0/3931316845 shutdown_connections 2024-09-11T22:03:00.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:00.451+0000 7f810a6f6740 1 -- 172.21.15.1:0/3931316845 wait complete. 2024-09-11T22:03:00.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:00.451+0000 7f810a6f6740 20 client.26156 trim_cache size 0 max 16384 2024-09-11T22:03:00.453 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-11T22:03:00.453 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 34, 2024-09-11T22:03:00.454 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24494, 2024-09-11T22:03:00.454 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 20, 2024-09-11T22:03:00.454 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-09-11T22:03:00.454 INFO:teuthology.orchestra.run.smithi001.stdout: "125e9d94": { 2024-09-11T22:03:00.454 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 20, 2024-09-11T22:03:00.454 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-09-11T22:03:00.454 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-11T22:03:00.454 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RELEASED", 2024-09-11T22:03:00.454 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-11T22:03:00.454 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:03:00.454 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 90.0, 2024-09-11T22:03:00.454 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 140.2, 2024-09-11T22:03:00.454 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-09-11T22:03:00.455 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-09-11T22:03:00.455 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-09-11T22:03:00.455 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-11T22:03:00.455 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RELEASED", 2024-09-11T22:03:00.455 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-11T22:03:00.455 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:03:00.455 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:03:00.455 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:03:00.455 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:03:00.455 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:03:00.455 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-11T22:03:00.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:00.452+0000 7f810831d640 1 -- 172.21.15.1:0/1172676970 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f80ec0a2d70 msgr2=0x7f80ec0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:00.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:00.452+0000 7f810831d640 1 --2- 172.21.15.1:0/1172676970 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f80ec0a2d70 0x7f80ec0a5230 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f80f8005fe0 tx=0x7f80f802a8c0 comp rx=0 tx=0).stop 2024-09-11T22:03:00.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:00.452+0000 7f810831d640 1 -- 172.21.15.1:0/1172676970 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8100101de0 msgr2=0x7f81001983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:00.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:00.452+0000 7f810831d640 1 --2- 172.21.15.1:0/1172676970 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8100101de0 0x7f81001983b0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f80fc00d950 tx=0x7f80fc00de10 comp rx=0 tx=0).stop 2024-09-11T22:03:00.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:00.452+0000 7f810831d640 1 -- 172.21.15.1:0/1172676970 shutdown_connections 2024-09-11T22:03:00.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:00.452+0000 7f810831d640 1 --2- 172.21.15.1:0/1172676970 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f80ec0a2d70 0x7f80ec0a5230 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:00.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:00.452+0000 7f810831d640 1 --2- 172.21.15.1:0/1172676970 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8100106f10 0x7f810019cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:00.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:00.452+0000 7f810831d640 1 --2- 172.21.15.1:0/1172676970 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8100102790 0x7f81001988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:00.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:00.452+0000 7f810831d640 1 --2- 172.21.15.1:0/1172676970 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8100101de0 0x7f81001983b0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:00.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:00.452+0000 7f810831d640 1 -- 172.21.15.1:0/1172676970 >> 172.21.15.1:0/1172676970 conn(0x7f81000fd8c0 msgr2=0x7f81000fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:00.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:00.452+0000 7f810831d640 1 -- 172.21.15.1:0/1172676970 shutdown_connections 2024-09-11T22:03:00.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:00.452+0000 7f810831d640 1 -- 172.21.15.1:0/1172676970 wait complete. 2024-09-11T22:03:01.145 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:00 smithi001 ceph-mon[30935]: pgmap v178: 97 pgs: 97 active+clean; 147 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 393 op/s 2024-09-11T22:03:01.171 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '125e9d94', total seconds elapsed: 24.2 2024-09-11T22:03:01.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:00 smithi053 ceph-mon[34908]: pgmap v178: 97 pgs: 97 active+clean; 147 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 393 op/s 2024-09-11T22:03:01.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:00 smithi161 ceph-mon[35615]: pgmap v178: 97 pgs: 97 active+clean; 147 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 393 op/s 2024-09-11T22:03:03.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:02 smithi053 ceph-mon[34908]: pgmap v179: 97 pgs: 97 active+clean; 147 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 205 KiB/s wr, 101 op/s 2024-09-11T22:03:03.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:02 smithi161 ceph-mon[35615]: pgmap v179: 97 pgs: 97 active+clean; 147 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 205 KiB/s wr, 101 op/s 2024-09-11T22:03:03.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:02 smithi001 ceph-mon[30935]: pgmap v179: 97 pgs: 97 active+clean; 147 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 205 KiB/s wr, 101 op/s 2024-09-11T22:03:05.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:04 smithi053 ceph-mon[34908]: pgmap v180: 97 pgs: 97 active+clean; 147 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 205 KiB/s wr, 101 op/s 2024-09-11T22:03:05.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:04 smithi161 ceph-mon[35615]: pgmap v180: 97 pgs: 97 active+clean; 147 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 205 KiB/s wr, 101 op/s 2024-09-11T22:03:05.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:04 smithi001 ceph-mon[30935]: pgmap v180: 97 pgs: 97 active+clean; 147 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 205 KiB/s wr, 101 op/s 2024-09-11T22:03:07.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:06 smithi053 ceph-mon[34908]: pgmap v181: 97 pgs: 97 active+clean; 158 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 133 op/s 2024-09-11T22:03:07.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:06 smithi161 ceph-mon[35615]: pgmap v181: 97 pgs: 97 active+clean; 158 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 133 op/s 2024-09-11T22:03:07.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:06 smithi001 ceph-mon[30935]: pgmap v181: 97 pgs: 97 active+clean; 158 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 133 op/s 2024-09-11T22:03:09.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:08 smithi053 ceph-mon[34908]: pgmap v182: 97 pgs: 97 active+clean; 158 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 1.1 MiB/s wr, 127 op/s 2024-09-11T22:03:09.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:08 smithi161 ceph-mon[35615]: pgmap v182: 97 pgs: 97 active+clean; 158 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 1.1 MiB/s wr, 127 op/s 2024-09-11T22:03:09.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:08 smithi001 ceph-mon[30935]: pgmap v182: 97 pgs: 97 active+clean; 158 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 1.1 MiB/s wr, 127 op/s 2024-09-11T22:03:09.913 INFO:tasks.workunit.client.0.smithi001.stdout:created hierarchy, now cleaning up 2024-09-11T22:03:11.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:10 smithi053 ceph-mon[34908]: pgmap v183: 97 pgs: 97 active+clean; 165 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 213 op/s 2024-09-11T22:03:11.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:10 smithi161 ceph-mon[35615]: pgmap v183: 97 pgs: 97 active+clean; 165 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 213 op/s 2024-09-11T22:03:11.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:10 smithi001 ceph-mon[30935]: pgmap v183: 97 pgs: 97 active+clean; 165 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 213 op/s 2024-09-11T22:03:12.622 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.618+0000 7f1d8b396640 1 -- 172.21.15.1:0/3782470864 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d84101ac0 msgr2=0x7f1d84101f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:12.622 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.618+0000 7f1d8b396640 1 --2- 172.21.15.1:0/3782470864 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d84101ac0 0x7f1d84101f20 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f1d74009930 tx=0x7f1d7402a420 comp rx=0 tx=0).stop 2024-09-11T22:03:12.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.618+0000 7f1d8b396640 1 -- 172.21.15.1:0/3782470864 shutdown_connections 2024-09-11T22:03:12.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.618+0000 7f1d8b396640 1 --2- 172.21.15.1:0/3782470864 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d84102460 0x7f1d8410e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:12.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.618+0000 7f1d8b396640 1 --2- 172.21.15.1:0/3782470864 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d84101ac0 0x7f1d84101f20 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:12.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.618+0000 7f1d8b396640 1 --2- 172.21.15.1:0/3782470864 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d84107ad0 0x7f1d84107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:12.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.618+0000 7f1d8b396640 1 -- 172.21.15.1:0/3782470864 >> 172.21.15.1:0/3782470864 conn(0x7f1d840fd7a0 msgr2=0x7f1d840ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:12.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.618+0000 7f1d8b396640 1 -- 172.21.15.1:0/3782470864 shutdown_connections 2024-09-11T22:03:12.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.618+0000 7f1d8b396640 1 -- 172.21.15.1:0/3782470864 wait complete. 2024-09-11T22:03:12.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.619+0000 7f1d8b396640 1 Processor -- start 2024-09-11T22:03:12.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.619+0000 7f1d8b396640 1 -- start start 2024-09-11T22:03:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.619+0000 7f1d8b396640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d84101ac0 0x7f1d84198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.619+0000 7f1d8b396640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d84102460 0x7f1d841989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.619+0000 7f1d8b396640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d84107ad0 0x7f1d8419cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.619+0000 7f1d8b396640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1d84111f10 con 0x7f1d84107ad0 2024-09-11T22:03:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.619+0000 7f1d8b396640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1d84111d90 con 0x7f1d84101ac0 2024-09-11T22:03:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.619+0000 7f1d8b396640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1d84112090 con 0x7f1d84102460 2024-09-11T22:03:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.620+0000 7f1d89132640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d84101ac0 0x7f1d84198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.620+0000 7f1d89132640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d84101ac0 0x7f1d84198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:46306/0 (socket says 172.21.15.1:46306) 2024-09-11T22:03:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.620+0000 7f1d88931640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d84102460 0x7f1d841989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.620+0000 7f1d89132640 1 -- 172.21.15.1:0/3953953623 learned_addr learned my addr 172.21.15.1:0/3953953623 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:03:12.625 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.620+0000 7f1d89933640 1 --2- 172.21.15.1:0/3953953623 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d84107ad0 0x7f1d8419cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:12.625 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.620+0000 7f1d89132640 1 -- 172.21.15.1:0/3953953623 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d84102460 msgr2=0x7f1d841989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:12.625 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.620+0000 7f1d89132640 1 --2- 172.21.15.1:0/3953953623 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d84102460 0x7f1d841989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:12.625 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.620+0000 7f1d89132640 1 -- 172.21.15.1:0/3953953623 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d84107ad0 msgr2=0x7f1d8419cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:12.625 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.620+0000 7f1d89132640 1 --2- 172.21.15.1:0/3953953623 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d84107ad0 0x7f1d8419cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:12.625 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.620+0000 7f1d89132640 1 -- 172.21.15.1:0/3953953623 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d8419d450 con 0x7f1d84101ac0 2024-09-11T22:03:12.625 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.620+0000 7f1d89132640 1 --2- 172.21.15.1:0/3953953623 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d84101ac0 0x7f1d84198460 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1d8000e9f0 tx=0x7f1d8000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:12.625 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.620+0000 7f1d7a7fc640 1 -- 172.21.15.1:0/3953953623 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d8000cd80 con 0x7f1d84101ac0 2024-09-11T22:03:12.625 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.620+0000 7f1d8b396640 1 -- 172.21.15.1:0/3953953623 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d8419d740 con 0x7f1d84101ac0 2024-09-11T22:03:12.625 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.620+0000 7f1d7a7fc640 1 -- 172.21.15.1:0/3953953623 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d80005500 con 0x7f1d84101ac0 2024-09-11T22:03:12.625 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.620+0000 7f1d7a7fc640 1 -- 172.21.15.1:0/3953953623 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d80010690 con 0x7f1d84101ac0 2024-09-11T22:03:12.626 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.620+0000 7f1d8b396640 1 -- 172.21.15.1:0/3953953623 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d84074420 con 0x7f1d84101ac0 2024-09-11T22:03:12.631 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.623+0000 7f1d8e208740 1 Processor -- start 2024-09-11T22:03:12.631 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.623+0000 7f1d7a7fc640 1 -- 172.21.15.1:0/3953953623 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f1d800046c0 con 0x7f1d84101ac0 2024-09-11T22:03:12.631 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.623+0000 7f1d8e208740 1 -- start start 2024-09-11T22:03:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.624+0000 7f1d8e208740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a8bc984af0 0x55a8bc984ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.624+0000 7f1d8e208740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8bc985410 0x55a8bc98f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.624+0000 7f1d8e208740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8bc98fcb0 0x55a8bc9920a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.624+0000 7f1d89132640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8bc985410 0x55a8bc98f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.624+0000 7f1d89933640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a8bc984af0 0x55a8bc984ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.624+0000 7f1d89132640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8bc985410 0x55a8bc98f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:46320/0 (socket says 172.21.15.1:46320) 2024-09-11T22:03:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.624+0000 7f1d89132640 1 -- 172.21.15.1:0/1072934230 learned_addr learned my addr 172.21.15.1:0/1072934230 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:03:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.624+0000 7f1d8e208740 1 -- 172.21.15.1:0/1072934230 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55a8bc8e92f0 con 0x55a8bc98fcb0 2024-09-11T22:03:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.624+0000 7f1d8e208740 1 -- 172.21.15.1:0/1072934230 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55a8bc891270 con 0x55a8bc985410 2024-09-11T22:03:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.624+0000 7f1d8e208740 1 -- 172.21.15.1:0/1072934230 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a8bc8e52b0 con 0x55a8bc984af0 2024-09-11T22:03:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.624+0000 7f1d88931640 1 --2- 172.21.15.1:0/1072934230 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8bc98fcb0 0x55a8bc9920a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.624+0000 7f1d89933640 1 -- 172.21.15.1:0/1072934230 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8bc985410 msgr2=0x55a8bc98f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.624+0000 7f1d89933640 1 --2- 172.21.15.1:0/1072934230 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8bc985410 0x55a8bc98f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.624+0000 7f1d89933640 1 -- 172.21.15.1:0/1072934230 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8bc98fcb0 msgr2=0x55a8bc9920a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.624+0000 7f1d89933640 1 --2- 172.21.15.1:0/1072934230 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8bc98fcb0 0x55a8bc9920a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.624+0000 7f1d89933640 1 -- 172.21.15.1:0/1072934230 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8bc890340 con 0x55a8bc984af0 2024-09-11T22:03:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.625+0000 7f1d89933640 1 --2- 172.21.15.1:0/1072934230 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a8bc984af0 0x55a8bc984ed0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c009f00 tx=0x7f1d7c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=351039ffa45f3c15 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.625+0000 7f1d7a7fc640 1 --2- 172.21.15.1:0/3953953623 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f1d700a2cf0 0x7f1d700a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.625+0000 7f1d6f7fe640 1 -- 172.21.15.1:0/1072934230 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d7c004d20 con 0x55a8bc984af0 2024-09-11T22:03:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.625+0000 7f1d89132640 1 --2- 172.21.15.1:0/3953953623 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f1d700a2cf0 0x7f1d700a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.625+0000 7f1d6f7fe640 1 -- 172.21.15.1:0/1072934230 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d7c004030 con 0x55a8bc984af0 2024-09-11T22:03:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.625+0000 7f1d6f7fe640 1 -- 172.21.15.1:0/1072934230 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d7c005280 con 0x55a8bc984af0 2024-09-11T22:03:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.625+0000 7f1d7a7fc640 1 -- 172.21.15.1:0/3953953623 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f1d800cf890 con 0x7f1d84101ac0 2024-09-11T22:03:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.626+0000 7f1d8e208740 1 -- 172.21.15.1:0/1072934230 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a8bc984af0 msgr2=0x55a8bc984ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.626+0000 7f1d8e208740 1 --2- 172.21.15.1:0/1072934230 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a8bc984af0 0x55a8bc984ed0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c009f00 tx=0x7f1d7c011810 comp rx=0 tx=0).stop 2024-09-11T22:03:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.626+0000 7f1d89132640 1 --2- 172.21.15.1:0/3953953623 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f1d700a2cf0 0x7f1d700a51b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1d8000c6a0 tx=0x7f1d800d5530 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.626+0000 7f1d8e208740 1 -- 172.21.15.1:0/1072934230 shutdown_connections 2024-09-11T22:03:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.626+0000 7f1d8e208740 1 --2- 172.21.15.1:0/1072934230 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8bc98fcb0 0x55a8bc9920a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.626+0000 7f1d8e208740 1 --2- 172.21.15.1:0/1072934230 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8bc985410 0x55a8bc98f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.626+0000 7f1d8e208740 1 --2- 172.21.15.1:0/1072934230 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a8bc984af0 0x55a8bc984ed0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.626+0000 7f1d8e208740 1 -- 172.21.15.1:0/1072934230 >> 172.21.15.1:0/1072934230 conn(0x55a8bc981060 msgr2=0x55a8bc981440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.626+0000 7f1d8e208740 1 -- 172.21.15.1:0/1072934230 shutdown_connections 2024-09-11T22:03:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.626+0000 7f1d8e208740 1 -- 172.21.15.1:0/1072934230 wait complete. 2024-09-11T22:03:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.626+0000 7f1d8e208740 1 -- start start 2024-09-11T22:03:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.626+0000 7f1d8e208740 1 Processor -- start 2024-09-11T22:03:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.627+0000 7f1d8e208740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8bc984af0 0x55a8bc98cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.627+0000 7f1d8e208740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8bc985410 0x55a8bc98d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.627+0000 7f1d8e208740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a8bc98fcb0 0x55a8bc98d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.627+0000 7f1d8e208740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55a8bc992880 con 0x55a8bc985410 2024-09-11T22:03:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.627+0000 7f1d8e208740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55a8bc872290 con 0x55a8bc984af0 2024-09-11T22:03:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.627+0000 7f1d8e208740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55a8bc992a00 con 0x55a8bc98fcb0 2024-09-11T22:03:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.627+0000 7f1d8e208740 10 client.0 _ll_register_callbacks cb 0x55a8bc8a3140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:03:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.627+0000 7f1d89933640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8bc984af0 0x55a8bc98cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:12.636 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.627+0000 7f1d88931640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8bc985410 0x55a8bc98d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:12.636 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.627+0000 7f1d89933640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8bc984af0 0x55a8bc98cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:46328/0 (socket says 172.21.15.1:46328) 2024-09-11T22:03:12.636 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.627+0000 7f1d89933640 1 -- 172.21.15.1:0/4288904233 learned_addr learned my addr 172.21.15.1:0/4288904233 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:03:12.636 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.627+0000 7f1d89933640 1 --2- 172.21.15.1:0/4288904233 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a8bc98fcb0 0x55a8bc98d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:12.636 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.627+0000 7f1d89933640 1 -- 172.21.15.1:0/4288904233 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a8bc98fcb0 msgr2=0x55a8bc98d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:12.636 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.627+0000 7f1d89933640 1 --2- 172.21.15.1:0/4288904233 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a8bc98fcb0 0x55a8bc98d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:12.636 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.627+0000 7f1d89933640 1 -- 172.21.15.1:0/4288904233 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8bc985410 msgr2=0x55a8bc98d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:12.636 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.627+0000 7f1d89933640 1 --2- 172.21.15.1:0/4288904233 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8bc985410 0x55a8bc98d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:12.636 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.627+0000 7f1d89933640 1 -- 172.21.15.1:0/4288904233 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8bc5fc210 con 0x55a8bc984af0 2024-09-11T22:03:12.636 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.627+0000 7f1d88931640 1 --2- 172.21.15.1:0/4288904233 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8bc985410 0x55a8bc98d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:03:12.637 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.628+0000 7f1d89933640 1 --2- 172.21.15.1:0/4288904233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8bc984af0 0x55a8bc98cbf0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c005a60 tx=0x7f1d7c009be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb6a7cf82abf1d5c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:12.637 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.628+0000 7f1d6cff9640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-11T22:03:12.637 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.628+0000 7f1d6cff9640 1 -- 172.21.15.1:0/4288904233 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d7c02d060 con 0x55a8bc984af0 2024-09-11T22:03:12.637 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.628+0000 7f1d6cff9640 1 -- 172.21.15.1:0/4288904233 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f1d7c004a40 con 0x55a8bc984af0 2024-09-11T22:03:12.637 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.628+0000 7f1d8e208740 1 -- 172.21.15.1:0/4288904233 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d7c004030 con 0x55a8bc984af0 2024-09-11T22:03:12.637 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.628+0000 7f1d6cff9640 1 -- 172.21.15.1:0/4288904233 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f1d7c01e040 con 0x55a8bc984af0 2024-09-11T22:03:12.637 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.628+0000 7f1d8e208740 10 client.26168 fetch_fsmap learned FSMap version 35 2024-09-11T22:03:12.637 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.628+0000 7f1d8e208740 1 -- 172.21.15.1:0/4288904233 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a8bc6c8620 con 0x55a8bc984af0 2024-09-11T22:03:12.637 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.628+0000 7f1d6cff9640 1 -- 172.21.15.1:0/4288904233 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d7c004030 con 0x55a8bc984af0 2024-09-11T22:03:12.638 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.628+0000 7f1d6cff9640 1 -- 172.21.15.1:0/4288904233 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 35) ==== 8382+0+0 (secure 0 0 0) 0x7f1d7c024070 con 0x55a8bc984af0 2024-09-11T22:03:12.638 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.628+0000 7f1d8e208740 10 client.26168 fetch_fsmap finished waiting for FSMap version 35 2024-09-11T22:03:12.638 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.628+0000 7f1d8e208740 10 client.26168 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:03:12.638 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.628+0000 7f1d8e208740 20 client.26168 populate_metadata read hostname 'smithi001' 2024-09-11T22:03:12.638 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.629+0000 7f1d8e208740 1 --2- 172.21.15.1:0/4288904233 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55a8bc99fc50 0x55a8bc9a2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:12.638 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.629+0000 7f1d8e208740 4 client.26168 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:03:12.638 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.629+0000 7f1d8e208740 1 -- 172.21.15.1:0/4288904233 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a8bc8f6a20 con 0x55a8bc99fc50 2024-09-11T22:03:12.638 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.629+0000 7f1d88931640 1 --2- 172.21.15.1:0/4288904233 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55a8bc99fc50 0x55a8bc9a2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:12.638 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.629+0000 7f1d6cff9640 10 client.26168 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:03:12.639 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.629+0000 7f1d88931640 1 --2- 172.21.15.1:0/4288904233 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55a8bc99fc50 0x55a8bc9a2040 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5099c73b7d6862f9 server_cookie=116eba28b344dcbd in_seq=0 out_seq=0 2024-09-11T22:03:12.640 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.631+0000 7f1d6cff9640 1 -- 172.21.15.1:0/4288904233 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a8bc8f6a20 con 0x55a8bc99fc50 2024-09-11T22:03:12.640 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.631+0000 7f1d6cff9640 10 client.26168 handle_command_reply: tid=0 2024-09-11T22:03:12.640 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.631+0000 7f1d8e208740 1 client.26168 shutdown 2024-09-11T22:03:12.640 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.631+0000 7f1d8e208740 1 -- 172.21.15.1:0/4288904233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8bc984af0 msgr2=0x55a8bc98cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:12.640 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.631+0000 7f1d8e208740 1 --2- 172.21.15.1:0/4288904233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8bc984af0 0x55a8bc98cbf0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c005a60 tx=0x7f1d7c009be0 comp rx=0 tx=0).stop 2024-09-11T22:03:12.640 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.631+0000 7f1d8e208740 1 -- 172.21.15.1:0/4288904233 shutdown_connections 2024-09-11T22:03:12.641 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.631+0000 7f1d8e208740 1 --2- 172.21.15.1:0/4288904233 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55a8bc99fc50 0x55a8bc9a2040 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:12.641 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.631+0000 7f1d8e208740 1 --2- 172.21.15.1:0/4288904233 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a8bc98fcb0 0x55a8bc98d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:12.641 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.631+0000 7f1d8e208740 1 --2- 172.21.15.1:0/4288904233 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8bc985410 0x55a8bc98d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:12.641 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.631+0000 7f1d6cff9640 0 client.26168 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:03:12.641 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.631+0000 7f1d8e208740 1 --2- 172.21.15.1:0/4288904233 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8bc984af0 0x55a8bc98cbf0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:12.641 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.631+0000 7f1d8e208740 1 -- 172.21.15.1:0/4288904233 >> 172.21.15.1:0/4288904233 conn(0x55a8bc981060 msgr2=0x55a8bc9801c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:12.641 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.632+0000 7f1d8e208740 1 -- 172.21.15.1:0/4288904233 shutdown_connections 2024-09-11T22:03:12.641 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.632+0000 7f1d8e208740 1 -- 172.21.15.1:0/4288904233 wait complete. 2024-09-11T22:03:12.641 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.632+0000 7f1d8e208740 20 client.26168 trim_cache size 0 max 16384 2024-09-11T22:03:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.647+0000 7f1d8e208740 1 Processor -- start 2024-09-11T22:03:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.647+0000 7f1d8e208740 1 -- start start 2024-09-11T22:03:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.647+0000 7f1d8e208740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a8bc984af0 0x55a8bc98cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.647+0000 7f1d8e208740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8bc985410 0x55a8bc98d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.647+0000 7f1d8e208740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8bc98fcb0 0x55a8bc9a7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.647+0000 7f1d8e208740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1d841ab820 con 0x55a8bc985410 2024-09-11T22:03:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.647+0000 7f1d8e208740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1d841ad450 con 0x55a8bc98fcb0 2024-09-11T22:03:12.650 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.647+0000 7f1d8e208740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1d841abef0 con 0x55a8bc984af0 2024-09-11T22:03:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.647+0000 7f1d88931640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8bc985410 0x55a8bc98d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.647+0000 7f1d89933640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a8bc984af0 0x55a8bc98cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.647+0000 7f1d88931640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8bc985410 0x55a8bc98d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42048/0 (socket says 172.21.15.1:42048) 2024-09-11T22:03:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.647+0000 7f1d88931640 1 -- 172.21.15.1:0/1027851480 learned_addr learned my addr 172.21.15.1:0/1027851480 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:03:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.647+0000 7f1d89933640 1 --2- 172.21.15.1:0/1027851480 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8bc98fcb0 0x55a8bc9a7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.648+0000 7f1d88931640 1 -- 172.21.15.1:0/1027851480 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a8bc984af0 msgr2=0x55a8bc98cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.648+0000 7f1d88931640 1 --2- 172.21.15.1:0/1027851480 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a8bc984af0 0x55a8bc98cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.648+0000 7f1d88931640 1 -- 172.21.15.1:0/1027851480 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8bc98fcb0 msgr2=0x55a8bc9a7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.648+0000 7f1d88931640 1 --2- 172.21.15.1:0/1027851480 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8bc98fcb0 0x55a8bc9a7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.648+0000 7f1d88931640 1 -- 172.21.15.1:0/1027851480 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8bc981cf0 con 0x55a8bc985410 2024-09-11T22:03:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.648+0000 7f1d89933640 1 --2- 172.21.15.1:0/1027851480 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a8bc984af0 0x55a8bc98cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:03:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.648+0000 7f1d88931640 1 --2- 172.21.15.1:0/1027851480 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8bc985410 0x55a8bc98d4e0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f1d74031550 tx=0x7f1d740058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa4fe3c43ca93732 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.648+0000 7f1d6effd640 1 -- 172.21.15.1:0/1027851480 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d74035040 con 0x55a8bc985410 2024-09-11T22:03:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.648+0000 7f1d6effd640 1 -- 172.21.15.1:0/1027851480 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d74005500 con 0x55a8bc985410 2024-09-11T22:03:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.648+0000 7f1d6effd640 1 -- 172.21.15.1:0/1027851480 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d74004c40 con 0x55a8bc985410 2024-09-11T22:03:12.656 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.649+0000 7f1d8e208740 1 -- 172.21.15.1:0/1027851480 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8bc985410 msgr2=0x55a8bc98d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:12.656 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.649+0000 7f1d8e208740 1 --2- 172.21.15.1:0/1027851480 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8bc985410 0x55a8bc98d4e0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f1d74031550 tx=0x7f1d740058d0 comp rx=0 tx=0).stop 2024-09-11T22:03:12.656 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.649+0000 7f1d8e208740 1 -- 172.21.15.1:0/1027851480 shutdown_connections 2024-09-11T22:03:12.656 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.649+0000 7f1d8e208740 1 --2- 172.21.15.1:0/1027851480 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8bc98fcb0 0x55a8bc9a7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:12.657 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.649+0000 7f1d8e208740 1 --2- 172.21.15.1:0/1027851480 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8bc985410 0x55a8bc98d4e0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:12.657 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.649+0000 7f1d8e208740 1 --2- 172.21.15.1:0/1027851480 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a8bc984af0 0x55a8bc98cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:12.657 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.649+0000 7f1d8e208740 1 -- 172.21.15.1:0/1027851480 >> 172.21.15.1:0/1027851480 conn(0x55a8bc981060 msgr2=0x55a8bc98ec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:12.657 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.649+0000 7f1d8e208740 1 -- 172.21.15.1:0/1027851480 shutdown_connections 2024-09-11T22:03:12.657 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.649+0000 7f1d8e208740 1 -- 172.21.15.1:0/1027851480 wait complete. 2024-09-11T22:03:12.657 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.649+0000 7f1d8e208740 1 -- start start 2024-09-11T22:03:12.657 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.649+0000 7f1d8e208740 1 Processor -- start 2024-09-11T22:03:12.657 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.650+0000 7f1d8e208740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8bc984af0 0x55a8bc998110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:12.657 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.650+0000 7f1d8e208740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a8bc985410 0x55a8bc989690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:12.657 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.650+0000 7f1d8e208740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8bc98fcb0 0x55a8bc989bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:12.658 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.650+0000 7f1d8e208740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1d841accc0 con 0x55a8bc984af0 2024-09-11T22:03:12.658 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.650+0000 7f1d8e208740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1d841ac5c0 con 0x55a8bc98fcb0 2024-09-11T22:03:12.658 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.650+0000 7f1d8e208740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1d841a9800 con 0x55a8bc985410 2024-09-11T22:03:12.658 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.650+0000 7f1d8e208740 10 client.0 _ll_register_callbacks cb 0x7f1d80005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:03:12.658 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.650+0000 7f1d89933640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8bc984af0 0x55a8bc998110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:12.658 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.650+0000 7f1d89933640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8bc984af0 0x55a8bc998110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42062/0 (socket says 172.21.15.1:42062) 2024-09-11T22:03:12.658 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.650+0000 7f1d89933640 1 -- 172.21.15.1:0/2963445660 learned_addr learned my addr 172.21.15.1:0/2963445660 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:03:12.658 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.650+0000 7f1d88931640 1 --2- 172.21.15.1:0/2963445660 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a8bc985410 0x55a8bc989690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:12.658 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.650+0000 7f1d89933640 1 --2- 172.21.15.1:0/2963445660 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8bc98fcb0 0x55a8bc989bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:12.658 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.650+0000 7f1d89933640 1 -- 172.21.15.1:0/2963445660 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a8bc985410 msgr2=0x55a8bc989690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.650+0000 7f1d89933640 1 --2- 172.21.15.1:0/2963445660 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a8bc985410 0x55a8bc989690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.650+0000 7f1d89933640 1 -- 172.21.15.1:0/2963445660 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8bc98fcb0 msgr2=0x55a8bc989bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.650+0000 7f1d89933640 1 --2- 172.21.15.1:0/2963445660 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8bc98fcb0 0x55a8bc989bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.650+0000 7f1d89933640 1 -- 172.21.15.1:0/2963445660 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8bc98a140 con 0x55a8bc984af0 2024-09-11T22:03:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.651+0000 7f1d89933640 1 --2- 172.21.15.1:0/2963445660 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8bc984af0 0x55a8bc998110 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c009bb0 tx=0x7f1d7c009be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88c8c7dd485234bb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.651+0000 7f1d6cff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:03:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.651+0000 7f1d6cff9640 1 -- 172.21.15.1:0/2963445660 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d7c041060 con 0x55a8bc984af0 2024-09-11T22:03:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.651+0000 7f1d6cff9640 1 -- 172.21.15.1:0/2963445660 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f1d7c03c660 con 0x55a8bc984af0 2024-09-11T22:03:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.651+0000 7f1d8e208740 1 -- 172.21.15.1:0/2963445660 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d74005500 con 0x55a8bc984af0 2024-09-11T22:03:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.651+0000 7f1d6cff9640 1 -- 172.21.15.1:0/2963445660 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f1d7c004030 con 0x55a8bc984af0 2024-09-11T22:03:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.651+0000 7f1d8e208740 10 client.16641 fetch_fsmap learned FSMap version 35 2024-09-11T22:03:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.651+0000 7f1d8e208740 1 -- 172.21.15.1:0/2963445660 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a8bca0a4c0 con 0x55a8bc984af0 2024-09-11T22:03:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.651+0000 7f1d6cff9640 1 -- 172.21.15.1:0/2963445660 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d74005500 con 0x55a8bc984af0 2024-09-11T22:03:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.652+0000 7f1d6cff9640 1 -- 172.21.15.1:0/2963445660 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 35) ==== 8382+0+0 (secure 0 0 0) 0x7f1d7c024070 con 0x55a8bc984af0 2024-09-11T22:03:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.652+0000 7f1d8e208740 10 client.16641 fetch_fsmap finished waiting for FSMap version 35 2024-09-11T22:03:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.652+0000 7f1d8e208740 10 client.16641 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:03:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.652+0000 7f1d8e208740 20 client.16641 populate_metadata read hostname 'smithi001' 2024-09-11T22:03:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.652+0000 7f1d8e208740 1 --2- 172.21.15.1:0/2963445660 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55a8bca0ac20 0x55a8bca0d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.652+0000 7f1d8e208740 4 client.16641 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:03:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.652+0000 7f1d8e208740 1 -- 172.21.15.1:0/2963445660 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a8bc8f0d70 con 0x55a8bca0ac20 2024-09-11T22:03:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.652+0000 7f1d88931640 1 --2- 172.21.15.1:0/2963445660 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55a8bca0ac20 0x55a8bca0d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.652+0000 7f1d6cff9640 10 client.16641 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:03:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:12.652+0000 7f1d88931640 1 --2- 172.21.15.1:0/2963445660 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55a8bca0ac20 0x55a8bca0d040 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6f5b0483c3d2962 server_cookie=ec6a68314a16f5cd in_seq=0 out_seq=0 2024-09-11T22:03:13.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:12 smithi053 ceph-mon[34908]: pgmap v184: 97 pgs: 97 active+clean; 165 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 118 op/s 2024-09-11T22:03:13.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:12 smithi053 ceph-mon[34908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:03:13.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:12 smithi161 ceph-mon[35615]: pgmap v184: 97 pgs: 97 active+clean; 165 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 118 op/s 2024-09-11T22:03:13.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:12 smithi161 ceph-mon[35615]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:03:13.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:12 smithi001 ceph-mon[30935]: pgmap v184: 97 pgs: 97 active+clean; 165 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 118 op/s 2024-09-11T22:03:13.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:12 smithi001 ceph-mon[30935]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:03:13.674 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:03:13.954 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:03:14.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.723+0000 7f140f59e640 1 -- 172.21.15.1:0/4156116500 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1410101270 msgr2=0x7f141010d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:14.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.723+0000 7f140f59e640 1 --2- 172.21.15.1:0/4156116500 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1410101270 0x7f141010d7a0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f1408009cd0 tx=0x7f140802a910 comp rx=0 tx=0).stop 2024-09-11T22:03:14.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.723+0000 7f140f59e640 1 -- 172.21.15.1:0/4156116500 shutdown_connections 2024-09-11T22:03:14.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.723+0000 7f140f59e640 1 --2- 172.21.15.1:0/4156116500 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1410101270 0x7f141010d7a0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:14.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.723+0000 7f140f59e640 1 --2- 172.21.15.1:0/4156116500 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14101008d0 0x7f1410100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:14.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.723+0000 7f140f59e640 1 --2- 172.21.15.1:0/4156116500 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f14101068d0 0x7f1410106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:14.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.723+0000 7f140f59e640 1 -- 172.21.15.1:0/4156116500 >> 172.21.15.1:0/4156116500 conn(0x7f14100fc610 msgr2=0x7f14100fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:14.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.724+0000 7f140f59e640 1 -- 172.21.15.1:0/4156116500 shutdown_connections 2024-09-11T22:03:14.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.724+0000 7f140f59e640 1 -- 172.21.15.1:0/4156116500 wait complete. 2024-09-11T22:03:14.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.725+0000 7f140f59e640 1 Processor -- start 2024-09-11T22:03:14.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.726+0000 7f140f59e640 1 -- start start 2024-09-11T22:03:14.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.726+0000 7f140f59e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f14101008d0 0x7f14101a45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:14.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.726+0000 7f140f59e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1410101270 0x7f14101a4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:14.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.726+0000 7f140f59e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14101068d0 0x7f141019e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:14.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.726+0000 7f140f59e640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f141010fc00 con 0x7f14101008d0 2024-09-11T22:03:14.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.726+0000 7f140f59e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f141010fa80 con 0x7f1410101270 2024-09-11T22:03:14.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.726+0000 7f140f59e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f141010fd80 con 0x7f14101068d0 2024-09-11T22:03:14.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.727+0000 7f140dd9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1410101270 0x7f14101a4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:14.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.727+0000 7f140ed9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14101068d0 0x7f141019e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:14.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.727+0000 7f140dd9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1410101270 0x7f14101a4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:46342/0 (socket says 172.21.15.1:46342) 2024-09-11T22:03:14.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.727+0000 7f140dd9b640 1 -- 172.21.15.1:0/4284938447 learned_addr learned my addr 172.21.15.1:0/4284938447 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:03:14.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.727+0000 7f140ed9d640 1 -- 172.21.15.1:0/4284938447 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1410101270 msgr2=0x7f14101a4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:14.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.727+0000 7f140ed9d640 1 --2- 172.21.15.1:0/4284938447 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1410101270 0x7f14101a4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:14.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.727+0000 7f140ed9d640 1 -- 172.21.15.1:0/4284938447 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f14101008d0 msgr2=0x7f14101a45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:14.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.727+0000 7f140ed9d640 1 --2- 172.21.15.1:0/4284938447 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f14101008d0 0x7f14101a45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:14.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.727+0000 7f140ed9d640 1 -- 172.21.15.1:0/4284938447 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f141019ee80 con 0x7f14101068d0 2024-09-11T22:03:14.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.728+0000 7f140dd9b640 1 --2- 172.21.15.1:0/4284938447 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1410101270 0x7f14101a4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:03:14.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.728+0000 7f140ed9d640 1 --2- 172.21.15.1:0/4284938447 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14101068d0 0x7f141019e6c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f140802af00 tx=0x7f1408003f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:14.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.731+0000 7f13ff7fe640 1 -- 172.21.15.1:0/4284938447 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1408031660 con 0x7f14101068d0 2024-09-11T22:03:14.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.731+0000 7f140f59e640 1 -- 172.21.15.1:0/4284938447 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f141019f110 con 0x7f14101068d0 2024-09-11T22:03:14.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.731+0000 7f140f59e640 1 -- 172.21.15.1:0/4284938447 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f141006bc10 con 0x7f14101068d0 2024-09-11T22:03:14.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.731+0000 7f13ff7fe640 1 -- 172.21.15.1:0/4284938447 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1408031800 con 0x7f14101068d0 2024-09-11T22:03:14.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.732+0000 7f13ff7fe640 1 -- 172.21.15.1:0/4284938447 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f140802fb50 con 0x7f14101068d0 2024-09-11T22:03:14.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.737+0000 7f13ff7fe640 1 -- 172.21.15.1:0/4284938447 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f140802fcf0 con 0x7f14101068d0 2024-09-11T22:03:14.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.738+0000 7f1416cb4740 1 Processor -- start 2024-09-11T22:03:14.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.738+0000 7f1416cb4740 1 -- start start 2024-09-11T22:03:14.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.739+0000 7f1416cb4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5556b7cec670 0x5556b7ceca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:14.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.739+0000 7f13ff7fe640 1 --2- 172.21.15.1:0/4284938447 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f13f80a2f20 0x7f13f80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:14.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.739+0000 7f1416cb4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5556b7cecf90 0x5556b7cf7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:14.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.739+0000 7f1416cb4740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5556b7cf7850 0x5556b7cf9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:14.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.739+0000 7f1416cb4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5556b7c522e0 con 0x5556b7cec670 2024-09-11T22:03:14.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.739+0000 7f1416cb4740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5556b7bf9ce0 con 0x5556b7cf7850 2024-09-11T22:03:14.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.739+0000 7f1416cb4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5556b7c4e2a0 con 0x5556b7cecf90 2024-09-11T22:03:14.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.739+0000 7f140ed9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5556b7cf7850 0x5556b7cf9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:14.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.739+0000 7f140ed9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5556b7cf7850 0x5556b7cf9c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:46350/0 (socket says 172.21.15.1:46350) 2024-09-11T22:03:14.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.739+0000 7f140ed9d640 1 -- 172.21.15.1:0/1314443757 learned_addr learned my addr 172.21.15.1:0/1314443757 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:03:14.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.739+0000 7f140dd9b640 1 --2- 172.21.15.1:0/1314443757 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5556b7cec670 0x5556b7ceca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:14.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.739+0000 7f140dd9b640 1 --2- 172.21.15.1:0/1314443757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5556b7cecf90 0x5556b7cf7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:14.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.740+0000 7f140ed9d640 1 -- 172.21.15.1:0/1314443757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5556b7cecf90 msgr2=0x5556b7cf7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:14.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.740+0000 7f140ed9d640 1 --2- 172.21.15.1:0/1314443757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5556b7cecf90 0x5556b7cf7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:14.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.740+0000 7f140ed9d640 1 -- 172.21.15.1:0/1314443757 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5556b7cec670 msgr2=0x5556b7ceca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:14.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.740+0000 7f140ed9d640 1 --2- 172.21.15.1:0/1314443757 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5556b7cec670 0x5556b7ceca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:14.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.740+0000 7f140ed9d640 1 -- 172.21.15.1:0/1314443757 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556b7bde440 con 0x5556b7cf7850 2024-09-11T22:03:14.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.740+0000 7f13ff7fe640 1 -- 172.21.15.1:0/4284938447 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f14080f0520 con 0x7f14101068d0 2024-09-11T22:03:14.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.741+0000 7f140ed9d640 1 --2- 172.21.15.1:0/1314443757 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5556b7cf7850 0x5556b7cf9c40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f14080c51f0 tx=0x7f14080b8d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=178d7ec6b6c6d456 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:14.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.742+0000 7f140e59c640 1 --2- 172.21.15.1:0/4284938447 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f13f80a2f20 0x7f13f80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:14.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.742+0000 7f13fcff9640 1 -- 172.21.15.1:0/1314443757 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14080bc410 con 0x5556b7cf7850 2024-09-11T22:03:14.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.742+0000 7f13fcff9640 1 -- 172.21.15.1:0/1314443757 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14080bd1b0 con 0x5556b7cf7850 2024-09-11T22:03:14.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.743+0000 7f13fcff9640 1 -- 172.21.15.1:0/1314443757 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1408035180 con 0x5556b7cf7850 2024-09-11T22:03:14.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.744+0000 7f140e59c640 1 --2- 172.21.15.1:0/4284938447 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f13f80a2f20 0x7f13f80a53e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1400005fd0 tx=0x7f1400005850 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:14.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.744+0000 7f1416cb4740 1 -- 172.21.15.1:0/1314443757 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5556b7cf7850 msgr2=0x5556b7cf9c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:14.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.744+0000 7f1416cb4740 1 --2- 172.21.15.1:0/1314443757 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5556b7cf7850 0x5556b7cf9c40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f14080c51f0 tx=0x7f14080b8d50 comp rx=0 tx=0).stop 2024-09-11T22:03:14.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.744+0000 7f1416cb4740 1 -- 172.21.15.1:0/1314443757 shutdown_connections 2024-09-11T22:03:14.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.744+0000 7f1416cb4740 1 --2- 172.21.15.1:0/1314443757 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5556b7cf7850 0x5556b7cf9c40 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:14.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.744+0000 7f1416cb4740 1 --2- 172.21.15.1:0/1314443757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5556b7cecf90 0x5556b7cf7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:14.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.744+0000 7f1416cb4740 1 --2- 172.21.15.1:0/1314443757 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5556b7cec670 0x5556b7ceca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:14.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.744+0000 7f1416cb4740 1 -- 172.21.15.1:0/1314443757 >> 172.21.15.1:0/1314443757 conn(0x5556b7ce8c00 msgr2=0x5556b7ce8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:14.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.744+0000 7f1416cb4740 1 -- 172.21.15.1:0/1314443757 shutdown_connections 2024-09-11T22:03:14.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.744+0000 7f1416cb4740 1 -- 172.21.15.1:0/1314443757 wait complete. 2024-09-11T22:03:14.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.744+0000 7f1416cb4740 1 -- start start 2024-09-11T22:03:14.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.745+0000 7f1416cb4740 1 Processor -- start 2024-09-11T22:03:14.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.745+0000 7f1416cb4740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5556b7cec670 0x5556b7cf7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:14.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.745+0000 7f1416cb4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5556b7cecf90 0x5556b7cf4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:14.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.745+0000 7f1416cb4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5556b7cf7850 0x5556b7cf4950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:14.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.745+0000 7f1416cb4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5556b7cfa1e0 con 0x5556b7cecf90 2024-09-11T22:03:14.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.745+0000 7f1416cb4740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5556b7c0d050 con 0x5556b7cec670 2024-09-11T22:03:14.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.745+0000 7f1416cb4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5556b7cfa360 con 0x5556b7cf7850 2024-09-11T22:03:14.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.745+0000 7f1416cb4740 10 client.0 _ll_register_callbacks cb 0x5556b7c4d160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:03:14.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.745+0000 7f140e59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5556b7cec670 0x5556b7cf7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:14.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.746+0000 7f140e59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5556b7cec670 0x5556b7cf7160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:46352/0 (socket says 172.21.15.1:46352) 2024-09-11T22:03:14.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.746+0000 7f140e59c640 1 -- 172.21.15.1:0/389027686 learned_addr learned my addr 172.21.15.1:0/389027686 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:03:14.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.746+0000 7f140ed9d640 1 --2- 172.21.15.1:0/389027686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5556b7cf7850 0x5556b7cf4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:14.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.746+0000 7f140dd9b640 1 --2- 172.21.15.1:0/389027686 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5556b7cecf90 0x5556b7cf4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:14.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.746+0000 7f140e59c640 1 -- 172.21.15.1:0/389027686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5556b7cf7850 msgr2=0x5556b7cf4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:14.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.746+0000 7f140e59c640 1 --2- 172.21.15.1:0/389027686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5556b7cf7850 0x5556b7cf4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:14.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.746+0000 7f140e59c640 1 -- 172.21.15.1:0/389027686 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5556b7cecf90 msgr2=0x5556b7cf4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:14.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.746+0000 7f140e59c640 1 --2- 172.21.15.1:0/389027686 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5556b7cecf90 0x5556b7cf4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:14.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.746+0000 7f140e59c640 1 -- 172.21.15.1:0/389027686 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556b79c33c0 con 0x5556b7cec670 2024-09-11T22:03:14.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.747+0000 7f140e59c640 1 --2- 172.21.15.1:0/389027686 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5556b7cec670 0x5556b7cf7160 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f14000176a0 tx=0x7f14000046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe7fda0ffd8f0fd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:14.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.747+0000 7f13f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-11T22:03:14.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.747+0000 7f13f5ffb640 1 -- 172.21.15.1:0/389027686 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1400027060 con 0x5556b7cec670 2024-09-11T22:03:14.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.748+0000 7f13f5ffb640 1 -- 172.21.15.1:0/389027686 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f1400011030 con 0x5556b7cec670 2024-09-11T22:03:14.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.748+0000 7f1416cb4740 1 -- 172.21.15.1:0/389027686 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14080bd1b0 con 0x5556b7cec670 2024-09-11T22:03:14.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.749+0000 7f13f5ffb640 1 -- 172.21.15.1:0/389027686 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f1400017c10 con 0x5556b7cec670 2024-09-11T22:03:14.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.750+0000 7f1416cb4740 10 client.26186 fetch_fsmap learned FSMap version 35 2024-09-11T22:03:14.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.750+0000 7f1416cb4740 1 -- 172.21.15.1:0/389027686 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5556b7d05b80 con 0x5556b7cec670 2024-09-11T22:03:14.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.751+0000 7f13f5ffb640 1 -- 172.21.15.1:0/389027686 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14080bd1b0 con 0x5556b7cec670 2024-09-11T22:03:14.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.751+0000 7f13f5ffb640 1 -- 172.21.15.1:0/389027686 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 35) ==== 8382+0+0 (secure 0 0 0) 0x7f14000154b0 con 0x5556b7cec670 2024-09-11T22:03:14.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.752+0000 7f1416cb4740 10 client.26186 fetch_fsmap finished waiting for FSMap version 35 2024-09-11T22:03:14.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.752+0000 7f1416cb4740 10 client.26186 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:03:14.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.752+0000 7f1416cb4740 20 client.26186 populate_metadata read hostname 'smithi001' 2024-09-11T22:03:14.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.752+0000 7f1416cb4740 1 --2- 172.21.15.1:0/389027686 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5556b7d074e0 0x5556b7d09920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:14.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.752+0000 7f1416cb4740 4 client.26186 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:03:14.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.752+0000 7f1416cb4740 1 -- 172.21.15.1:0/389027686 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5556b7c5f850 con 0x5556b7d074e0 2024-09-11T22:03:14.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.753+0000 7f140dd9b640 1 --2- 172.21.15.1:0/389027686 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5556b7d074e0 0x5556b7d09920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:14.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.755+0000 7f140dd9b640 1 --2- 172.21.15.1:0/389027686 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5556b7d074e0 0x5556b7d09920 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92a7188c65df6016 server_cookie=bee35b45056fc396 in_seq=0 out_seq=0 2024-09-11T22:03:14.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.755+0000 7f13f5ffb640 10 client.26186 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:03:14.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.758+0000 7f13f5ffb640 1 -- 172.21.15.1:0/389027686 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5556b7c5f850 con 0x5556b7d074e0 2024-09-11T22:03:14.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.758+0000 7f13f5ffb640 10 client.26186 handle_command_reply: tid=0 2024-09-11T22:03:14.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.758+0000 7f1416cb4740 1 client.26186 shutdown 2024-09-11T22:03:14.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.759+0000 7f1416cb4740 1 -- 172.21.15.1:0/389027686 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5556b7cec670 msgr2=0x5556b7cf7160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:14.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.759+0000 7f1416cb4740 1 --2- 172.21.15.1:0/389027686 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5556b7cec670 0x5556b7cf7160 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f14000176a0 tx=0x7f14000046c0 comp rx=0 tx=0).stop 2024-09-11T22:03:14.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.759+0000 7f1416cb4740 1 -- 172.21.15.1:0/389027686 shutdown_connections 2024-09-11T22:03:14.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.759+0000 7f1416cb4740 1 --2- 172.21.15.1:0/389027686 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5556b7d074e0 0x5556b7d09920 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:14.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.759+0000 7f1416cb4740 1 --2- 172.21.15.1:0/389027686 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5556b7cf7850 0x5556b7cf4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:14.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.759+0000 7f1416cb4740 1 --2- 172.21.15.1:0/389027686 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5556b7cecf90 0x5556b7cf4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:14.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.759+0000 7f1416cb4740 1 --2- 172.21.15.1:0/389027686 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5556b7cec670 0x5556b7cf7160 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:14.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.759+0000 7f1416cb4740 1 -- 172.21.15.1:0/389027686 >> 172.21.15.1:0/389027686 conn(0x5556b7ce8c00 msgr2=0x5556b7cee930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:14.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.759+0000 7f13f5ffb640 0 client.26186 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:03:14.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.760+0000 7f1416cb4740 1 -- 172.21.15.1:0/389027686 shutdown_connections 2024-09-11T22:03:14.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.760+0000 7f1416cb4740 1 -- 172.21.15.1:0/389027686 wait complete. 2024-09-11T22:03:14.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.760+0000 7f1416cb4740 20 client.26186 trim_cache size 0 max 16384 2024-09-11T22:03:14.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.795+0000 7f1416cb4740 1 Processor -- start 2024-09-11T22:03:14.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.795+0000 7f1416cb4740 1 -- start start 2024-09-11T22:03:14.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.795+0000 7f1416cb4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5556b7cec670 0x5556b7d03440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:14.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.795+0000 7f1416cb4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5556b7cecf90 0x5556b7d0f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:14.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.795+0000 7f1416cb4740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5556b7cf7850 0x5556b7d0f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:14.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.795+0000 7f1416cb4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f13f000e9e0 con 0x5556b7cecf90 2024-09-11T22:03:14.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.795+0000 7f1416cb4740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f13f0010610 con 0x5556b7cf7850 2024-09-11T22:03:14.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.795+0000 7f1416cb4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f13f000f0b0 con 0x5556b7cec670 2024-09-11T22:03:14.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.796+0000 7f140e59c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5556b7cec670 0x5556b7d03440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:14.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.796+0000 7f140e59c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5556b7cec670 0x5556b7d03440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:44392/0 (socket says 172.21.15.1:44392) 2024-09-11T22:03:14.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.796+0000 7f140e59c640 1 -- 172.21.15.1:0/477536627 learned_addr learned my addr 172.21.15.1:0/477536627 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:03:14.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.796+0000 7f140ed9d640 1 --2- 172.21.15.1:0/477536627 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5556b7cf7850 0x5556b7d0f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:14.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.796+0000 7f140dd9b640 1 --2- 172.21.15.1:0/477536627 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5556b7cecf90 0x5556b7d0f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:14.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.796+0000 7f140e59c640 1 -- 172.21.15.1:0/477536627 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5556b7cf7850 msgr2=0x5556b7d0f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:14.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.796+0000 7f140e59c640 1 --2- 172.21.15.1:0/477536627 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5556b7cf7850 0x5556b7d0f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:14.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.796+0000 7f140e59c640 1 -- 172.21.15.1:0/477536627 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5556b7cecf90 msgr2=0x5556b7d0f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:14.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.796+0000 7f140e59c640 1 --2- 172.21.15.1:0/477536627 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5556b7cecf90 0x5556b7d0f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:14.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.796+0000 7f140e59c640 1 -- 172.21.15.1:0/477536627 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556b7ced5e0 con 0x5556b7cec670 2024-09-11T22:03:14.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.796+0000 7f140dd9b640 1 --2- 172.21.15.1:0/477536627 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5556b7cecf90 0x5556b7d0f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:03:14.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.796+0000 7f140e59c640 1 --2- 172.21.15.1:0/477536627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5556b7cec670 0x5556b7d03440 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1400012040 tx=0x7f1400000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1d34879e52bd2cd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:14.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.797+0000 7f13f7fff640 1 -- 172.21.15.1:0/477536627 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1400031080 con 0x5556b7cec670 2024-09-11T22:03:14.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.797+0000 7f13f7fff640 1 -- 172.21.15.1:0/477536627 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1400017db0 con 0x5556b7cec670 2024-09-11T22:03:14.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.797+0000 7f13f7fff640 1 -- 172.21.15.1:0/477536627 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1400022b60 con 0x5556b7cec670 2024-09-11T22:03:14.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.798+0000 7f1416cb4740 1 -- 172.21.15.1:0/477536627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5556b7cec670 msgr2=0x5556b7d03440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:14.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.798+0000 7f1416cb4740 1 --2- 172.21.15.1:0/477536627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5556b7cec670 0x5556b7d03440 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1400012040 tx=0x7f1400000bc0 comp rx=0 tx=0).stop 2024-09-11T22:03:14.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.798+0000 7f1416cb4740 1 -- 172.21.15.1:0/477536627 shutdown_connections 2024-09-11T22:03:14.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.798+0000 7f1416cb4740 1 --2- 172.21.15.1:0/477536627 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5556b7cf7850 0x5556b7d0f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:14.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.798+0000 7f1416cb4740 1 --2- 172.21.15.1:0/477536627 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5556b7cecf90 0x5556b7d0f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:14.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.798+0000 7f1416cb4740 1 --2- 172.21.15.1:0/477536627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5556b7cec670 0x5556b7d03440 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:14.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.799+0000 7f1416cb4740 1 -- 172.21.15.1:0/477536627 >> 172.21.15.1:0/477536627 conn(0x5556b7ce8c00 msgr2=0x5556b7cefa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:14.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.799+0000 7f1416cb4740 1 -- 172.21.15.1:0/477536627 shutdown_connections 2024-09-11T22:03:14.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.799+0000 7f1416cb4740 1 -- 172.21.15.1:0/477536627 wait complete. 2024-09-11T22:03:14.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.800+0000 7f1416cb4740 1 -- start start 2024-09-11T22:03:14.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.800+0000 7f1416cb4740 1 Processor -- start 2024-09-11T22:03:14.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.800+0000 7f1416cb4740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5556b7cec670 0x5556b7cfab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:14.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.800+0000 7f1416cb4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5556b7cecf90 0x5556b7cfb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:14.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.800+0000 7f1416cb4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5556b7cf7850 0x5556b7d701f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:14.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.801+0000 7f1416cb4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f13f000fe80 con 0x5556b7cecf90 2024-09-11T22:03:14.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.801+0000 7f1416cb4740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f13f000f780 con 0x5556b7cec670 2024-09-11T22:03:14.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.801+0000 7f1416cb4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f13f000c9c0 con 0x5556b7cf7850 2024-09-11T22:03:14.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.801+0000 7f140dd9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5556b7cecf90 0x5556b7cfb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:14.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.801+0000 7f1416cb4740 10 client.0 _ll_register_callbacks cb 0x7f140000b7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:03:14.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.801+0000 7f140dd9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5556b7cecf90 0x5556b7cfb070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42114/0 (socket says 172.21.15.1:42114) 2024-09-11T22:03:14.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.801+0000 7f140dd9b640 1 -- 172.21.15.1:0/1320760754 learned_addr learned my addr 172.21.15.1:0/1320760754 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:03:14.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.801+0000 7f140ed9d640 1 --2- 172.21.15.1:0/1320760754 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5556b7cf7850 0x5556b7d701f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:14.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.801+0000 7f140dd9b640 1 -- 172.21.15.1:0/1320760754 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5556b7cf7850 msgr2=0x5556b7d701f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:14.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.801+0000 7f140dd9b640 1 --2- 172.21.15.1:0/1320760754 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5556b7cf7850 0x5556b7d701f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:14.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.801+0000 7f140dd9b640 1 -- 172.21.15.1:0/1320760754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5556b7cec670 msgr2=0x5556b7cfab30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T22:03:14.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.801+0000 7f140dd9b640 1 --2- 172.21.15.1:0/1320760754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5556b7cec670 0x5556b7cfab30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:14.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.801+0000 7f140dd9b640 1 -- 172.21.15.1:0/1320760754 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556b7d70730 con 0x5556b7cecf90 2024-09-11T22:03:14.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.802+0000 7f140dd9b640 1 --2- 172.21.15.1:0/1320760754 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5556b7cecf90 0x5556b7cfb070 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f1404020fd0 tx=0x7f14040046e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0f23857d8dcb841 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:14.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.802+0000 7f13f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:03:14.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.802+0000 7f13f5ffb640 1 -- 172.21.15.1:0/1320760754 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1404019930 con 0x5556b7cecf90 2024-09-11T22:03:14.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.803+0000 7f1416cb4740 1 -- 172.21.15.1:0/1320760754 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1400017db0 con 0x5556b7cecf90 2024-09-11T22:03:14.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.803+0000 7f13f5ffb640 1 -- 172.21.15.1:0/1320760754 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f1404019ad0 con 0x5556b7cecf90 2024-09-11T22:03:14.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.803+0000 7f13f5ffb640 1 -- 172.21.15.1:0/1320760754 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f1404030050 con 0x5556b7cecf90 2024-09-11T22:03:14.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.804+0000 7f1416cb4740 10 client.16653 fetch_fsmap learned FSMap version 35 2024-09-11T22:03:14.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.804+0000 7f1416cb4740 1 -- 172.21.15.1:0/1320760754 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5556b7d71a10 con 0x5556b7cecf90 2024-09-11T22:03:14.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.804+0000 7f13f5ffb640 1 -- 172.21.15.1:0/1320760754 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1404019cd0 con 0x5556b7cecf90 2024-09-11T22:03:14.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.805+0000 7f13f5ffb640 1 -- 172.21.15.1:0/1320760754 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 35) ==== 8382+0+0 (secure 0 0 0) 0x7f1404017440 con 0x5556b7cecf90 2024-09-11T22:03:14.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.805+0000 7f1416cb4740 10 client.16653 fetch_fsmap finished waiting for FSMap version 35 2024-09-11T22:03:14.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.805+0000 7f1416cb4740 10 client.16653 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:03:14.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.805+0000 7f1416cb4740 20 client.16653 populate_metadata read hostname 'smithi001' 2024-09-11T22:03:14.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.805+0000 7f1416cb4740 1 --2- 172.21.15.1:0/1320760754 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5556b7d72230 0x5556b7d74650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:14.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.805+0000 7f1416cb4740 4 client.16653 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:03:14.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.805+0000 7f1416cb4740 1 -- 172.21.15.1:0/1320760754 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f1400017db0 con 0x5556b7d72230 2024-09-11T22:03:14.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.805+0000 7f140e59c640 1 --2- 172.21.15.1:0/1320760754 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5556b7d72230 0x5556b7d74650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:14.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.806+0000 7f140e59c640 1 --2- 172.21.15.1:0/1320760754 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5556b7d72230 0x5556b7d74650 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2591f9a48f2485b server_cookie=1a7a5305bfd86e8c in_seq=0 out_seq=0 2024-09-11T22:03:14.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.807+0000 7f13f5ffb640 10 client.16653 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:03:14.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.810+0000 7f13f5ffb640 1 -- 172.21.15.1:0/1320760754 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f1400017db0 con 0x5556b7d72230 2024-09-11T22:03:14.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.810+0000 7f13f5ffb640 10 client.16653 handle_command_reply: tid=0 2024-09-11T22:03:14.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.810+0000 7f1416cb4740 1 client.16653 shutdown 2024-09-11T22:03:14.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.810+0000 7f1416cb4740 1 -- 172.21.15.1:0/1320760754 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5556b7cecf90 msgr2=0x5556b7cfb070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:14.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.810+0000 7f1416cb4740 1 --2- 172.21.15.1:0/1320760754 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5556b7cecf90 0x5556b7cfb070 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f1404020fd0 tx=0x7f14040046e0 comp rx=0 tx=0).stop 2024-09-11T22:03:14.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.810+0000 7f1416cb4740 1 -- 172.21.15.1:0/1320760754 shutdown_connections 2024-09-11T22:03:14.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.810+0000 7f1416cb4740 1 --2- 172.21.15.1:0/1320760754 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5556b7d72230 0x5556b7d74650 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:14.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.810+0000 7f1416cb4740 1 --2- 172.21.15.1:0/1320760754 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5556b7cf7850 0x5556b7d701f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:14.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.810+0000 7f1416cb4740 1 --2- 172.21.15.1:0/1320760754 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5556b7cecf90 0x5556b7cfb070 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:14.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.810+0000 7f1416cb4740 1 --2- 172.21.15.1:0/1320760754 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5556b7cec670 0x5556b7cfab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:14.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.810+0000 7f13f5ffb640 0 client.16653 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:03:14.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.810+0000 7f1416cb4740 1 -- 172.21.15.1:0/1320760754 >> 172.21.15.1:0/1320760754 conn(0x5556b7ce8c00 msgr2=0x5556b7ce7500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:14.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.810+0000 7f1416cb4740 1 -- 172.21.15.1:0/1320760754 shutdown_connections 2024-09-11T22:03:14.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.810+0000 7f1416cb4740 1 -- 172.21.15.1:0/1320760754 wait complete. 2024-09-11T22:03:14.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.811+0000 7f1416cb4740 20 client.16653 trim_cache size 0 max 16384 2024-09-11T22:03:14.814 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-11T22:03:14.814 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-09-11T22:03:14.814 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-09-11T22:03:14.814 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-11T22:03:14.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.811+0000 7f140f59e640 1 -- 172.21.15.1:0/4284938447 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f13f80a2f20 msgr2=0x7f13f80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:14.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.811+0000 7f140f59e640 1 --2- 172.21.15.1:0/4284938447 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f13f80a2f20 0x7f13f80a53e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1400005fd0 tx=0x7f1400005850 comp rx=0 tx=0).stop 2024-09-11T22:03:14.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.811+0000 7f140f59e640 1 -- 172.21.15.1:0/4284938447 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14101068d0 msgr2=0x7f141019e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:14.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.811+0000 7f140f59e640 1 --2- 172.21.15.1:0/4284938447 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14101068d0 0x7f141019e6c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f140802af00 tx=0x7f1408003f10 comp rx=0 tx=0).stop 2024-09-11T22:03:14.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.812+0000 7f140f59e640 1 -- 172.21.15.1:0/4284938447 shutdown_connections 2024-09-11T22:03:14.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.812+0000 7f140f59e640 1 --2- 172.21.15.1:0/4284938447 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f13f80a2f20 0x7f13f80a53e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:14.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.812+0000 7f140f59e640 1 --2- 172.21.15.1:0/4284938447 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14101068d0 0x7f141019e6c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:14.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.812+0000 7f140f59e640 1 --2- 172.21.15.1:0/4284938447 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1410101270 0x7f14101a4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:14.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.812+0000 7f140f59e640 1 --2- 172.21.15.1:0/4284938447 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f14101008d0 0x7f14101a45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:14.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.812+0000 7f140f59e640 1 -- 172.21.15.1:0/4284938447 >> 172.21.15.1:0/4284938447 conn(0x7f14100fc610 msgr2=0x7f14100fcc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:14.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.812+0000 7f140f59e640 1 -- 172.21.15.1:0/4284938447 shutdown_connections 2024-09-11T22:03:14.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:14.812+0000 7f140f59e640 1 -- 172.21.15.1:0/4284938447 wait complete. 2024-09-11T22:03:15.152 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:14 smithi001 ceph-mon[30935]: pgmap v185: 97 pgs: 97 active+clean; 165 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 118 op/s 2024-09-11T22:03:15.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:14 smithi053 ceph-mon[34908]: pgmap v185: 97 pgs: 97 active+clean; 165 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 118 op/s 2024-09-11T22:03:15.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:14 smithi161 ceph-mon[35615]: pgmap v185: 97 pgs: 97 active+clean; 165 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 118 op/s 2024-09-11T22:03:15.744 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T22:03:15.745 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:03:16.070 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:03:16.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:16.681+0000 7f1143d6c640 1 -- 172.21.15.1:0/946706429 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f113c106a80 msgr2=0x7f113c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:16.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:16.681+0000 7f1143d6c640 1 --2- 172.21.15.1:0/946706429 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f113c106a80 0x7f113c106e60 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f1138009930 tx=0x7f113802a420 comp rx=0 tx=0).stop 2024-09-11T22:03:16.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:16.681+0000 7f1143d6c640 1 -- 172.21.15.1:0/946706429 shutdown_connections 2024-09-11T22:03:16.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:16.681+0000 7f1143d6c640 1 --2- 172.21.15.1:0/946706429 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f113c101420 0x7f113c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:16.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:16.681+0000 7f1143d6c640 1 --2- 172.21.15.1:0/946706429 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f113c100a80 0x7f113c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:16.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:16.681+0000 7f1143d6c640 1 --2- 172.21.15.1:0/946706429 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f113c106a80 0x7f113c106e60 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:16.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:16.681+0000 7f1143d6c640 1 -- 172.21.15.1:0/946706429 >> 172.21.15.1:0/946706429 conn(0x7f113c0fc7a0 msgr2=0x7f113c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:16.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:16.681+0000 7f1143d6c640 1 -- 172.21.15.1:0/946706429 shutdown_connections 2024-09-11T22:03:16.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:16.681+0000 7f1143d6c640 1 -- 172.21.15.1:0/946706429 wait complete. 2024-09-11T22:03:16.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:16.681+0000 7f1143d6c640 1 Processor -- start 2024-09-11T22:03:16.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:16.681+0000 7f1143d6c640 1 -- start start 2024-09-11T22:03:16.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:16.682+0000 7f1143d6c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f113c100a80 0x7f113c0fcdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:16.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:16.682+0000 7f1143d6c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f113c101420 0x7f113c0fd300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:16.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:16.682+0000 7f1143d6c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f113c106a80 0x7f113c0fd840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:16.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:16.682+0000 7f1143d6c640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f113c110e80 con 0x7f113c106a80 2024-09-11T22:03:16.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:16.682+0000 7f1143d6c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f113c110d00 con 0x7f113c101420 2024-09-11T22:03:16.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:16.682+0000 7f1143d6c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f113c111000 con 0x7f113c100a80 2024-09-11T22:03:16.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:16.682+0000 7f1142309640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f113c106a80 0x7f113c0fd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:16.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:16.682+0000 7f1141b08640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f113c100a80 0x7f113c0fcdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:16.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:16.682+0000 7f1141307640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f113c101420 0x7f113c0fd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:16.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:16.682+0000 7f1142309640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f113c106a80 0x7f113c0fd840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52084/0 (socket says 172.21.15.1:52084) 2024-09-11T22:03:16.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:16.682+0000 7f1142309640 1 -- 172.21.15.1:0/1616198614 learned_addr learned my addr 172.21.15.1:0/1616198614 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:03:16.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:16.682+0000 7f1142309640 1 -- 172.21.15.1:0/1616198614 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f113c100a80 msgr2=0x7f113c0fcdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:16.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:16.682+0000 7f1142309640 1 --2- 172.21.15.1:0/1616198614 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f113c100a80 0x7f113c0fcdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:16.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:16.682+0000 7f1142309640 1 -- 172.21.15.1:0/1616198614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f113c101420 msgr2=0x7f113c0fd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:16.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:16.682+0000 7f1142309640 1 --2- 172.21.15.1:0/1616198614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f113c101420 0x7f113c0fd300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:16.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:16.682+0000 7f1142309640 1 -- 172.21.15.1:0/1616198614 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f113c1a6930 con 0x7f113c106a80 2024-09-11T22:03:16.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:16.682+0000 7f1142309640 1 --2- 172.21.15.1:0/1616198614 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f113c106a80 0x7f113c0fd840 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f113400eb10 tx=0x7f113400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:16.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:16.683+0000 7f1132ffd640 1 -- 172.21.15.1:0/1616198614 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1134004b20 con 0x7f113c106a80 2024-09-11T22:03:16.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:16.683+0000 7f1132ffd640 1 -- 172.21.15.1:0/1616198614 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1134004cc0 con 0x7f113c106a80 2024-09-11T22:03:16.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:16.683+0000 7f1143d6c640 1 -- 172.21.15.1:0/1616198614 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f113c1a6c20 con 0x7f113c106a80 2024-09-11T22:03:16.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:16.683+0000 7f1132ffd640 1 -- 172.21.15.1:0/1616198614 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1134010760 con 0x7f113c106a80 2024-09-11T22:03:16.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:16.683+0000 7f1143d6c640 1 -- 172.21.15.1:0/1616198614 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f113c1a7160 con 0x7f113c106a80 2024-09-11T22:03:16.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:16.684+0000 7f1143d6c640 1 -- 172.21.15.1:0/1616198614 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f113c100ee0 con 0x7f113c106a80 2024-09-11T22:03:16.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:16.685+0000 7f1132ffd640 1 -- 172.21.15.1:0/1616198614 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f1134010900 con 0x7f113c106a80 2024-09-11T22:03:16.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:16.686+0000 7f1132ffd640 1 --2- 172.21.15.1:0/1616198614 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f11280a2d70 0x7f11280a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:16.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:16.686+0000 7f1132ffd640 1 -- 172.21.15.1:0/1616198614 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f11340cfb70 con 0x7f113c106a80 2024-09-11T22:03:16.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:16.686+0000 7f1141b08640 1 --2- 172.21.15.1:0/1616198614 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f11280a2d70 0x7f11280a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:16.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:16.687+0000 7f1141b08640 1 --2- 172.21.15.1:0/1616198614 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f11280a2d70 0x7f11280a5230 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1138009900 tx=0x7f11380058d0 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:16.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:16.689+0000 7f1132ffd640 1 -- 172.21.15.1:0/1616198614 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f11340541b0 con 0x7f113c106a80 2024-09-11T22:03:17.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:17.026+0000 7f1143d6c640 1 -- 172.21.15.1:0/1616198614 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f113c0fe2f0 con 0x7f113c106a80 2024-09-11T22:03:17.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:17.027+0000 7f1132ffd640 1 -- 172.21.15.1:0/1616198614 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11963 (secure 0 0 0) 0x7f113c0fe2f0 con 0x7f113c106a80 2024-09-11T22:03:17.029 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T22:03:17.029 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":36,"btime":"2024-09-11T22:03:16:984637+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/421542832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":421542832},{"type":"v1","addr":"172.21.15.1:6835","nonce":421542832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2836864406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2836864406},{"type":"v1","addr":"172.21.15.1:6837","nonce":2836864406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":36,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T21:55:20.791263+0000","modified":"2024-09-11T22:03:16.791541+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24484,"mds_2":24467,"mds_3":24466,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14595":{"gid":14595,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/407834672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":407834672},{"type":"v1","addr":"172.21.15.1:6839","nonce":407834672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/1068827678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":1068827678},{"type":"v1","addr":"172.21.15.1:6841","nonce":1068827678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/872410024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":872410024},{"type":"v1","addr":"172.21.15.161:6833","nonce":872410024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/1648944570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1648944570},{"type":"v1","addr":"172.21.15.53:6833","nonce":1648944570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/2126933993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2126933993},{"type":"v1","addr":"172.21.15.53:6835","nonce":2126933993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.161:6835/828381592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":828381592},{"type":"v1","addr":"172.21.15.161:6835","nonce":828381592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.161:6837/1799318656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1799318656},{"type":"v1","addr":"172.21.15.161:6837","nonce":1799318656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.53:6837/3457131830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3457131830},{"type":"v1","addr":"172.21.15.53:6837","nonce":3457131830}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/455365383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":455365383},{"type":"v1","addr":"172.21.15.161:6839","nonce":455365383}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.53:6839/1378584447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1378584447},{"type":"v1","addr":"172.21.15.53:6839","nonce":1378584447}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24460,24466,24467,24484,24494]},"id":1}]} 2024-09-11T22:03:17.029 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 36 2024-09-11T22:03:17.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:17.029+0000 7f1143d6c640 1 -- 172.21.15.1:0/1616198614 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f11280a2d70 msgr2=0x7f11280a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:17.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:17.029+0000 7f1143d6c640 1 --2- 172.21.15.1:0/1616198614 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f11280a2d70 0x7f11280a5230 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1138009900 tx=0x7f11380058d0 comp rx=0 tx=0).stop 2024-09-11T22:03:17.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:17.029+0000 7f1143d6c640 1 -- 172.21.15.1:0/1616198614 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f113c106a80 msgr2=0x7f113c0fd840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:17.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:17.029+0000 7f1143d6c640 1 --2- 172.21.15.1:0/1616198614 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f113c106a80 0x7f113c0fd840 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f113400eb10 tx=0x7f113400c2d0 comp rx=0 tx=0).stop 2024-09-11T22:03:17.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:17.029+0000 7f1143d6c640 1 -- 172.21.15.1:0/1616198614 shutdown_connections 2024-09-11T22:03:17.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:17.029+0000 7f1143d6c640 1 --2- 172.21.15.1:0/1616198614 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f11280a2d70 0x7f11280a5230 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:17.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:17.029+0000 7f1143d6c640 1 --2- 172.21.15.1:0/1616198614 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f113c106a80 0x7f113c0fd840 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:17.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:17.029+0000 7f1143d6c640 1 --2- 172.21.15.1:0/1616198614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f113c101420 0x7f113c0fd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:17.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:17.029+0000 7f1143d6c640 1 --2- 172.21.15.1:0/1616198614 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f113c100a80 0x7f113c0fcdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:17.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:17.029+0000 7f1143d6c640 1 -- 172.21.15.1:0/1616198614 >> 172.21.15.1:0/1616198614 conn(0x7f113c0fc7a0 msgr2=0x7f113c06aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:17.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:17.029+0000 7f1143d6c640 1 -- 172.21.15.1:0/1616198614 shutdown_connections 2024-09-11T22:03:17.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:17.029+0000 7f1143d6c640 1 -- 172.21.15.1:0/1616198614 wait complete. 2024-09-11T22:03:17.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:16 smithi053 ceph-mon[34908]: pgmap v186: 97 pgs: 97 active+clean; 182 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 196 op/s 2024-09-11T22:03:17.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:16 smithi161 ceph-mon[35615]: pgmap v186: 97 pgs: 97 active+clean; 182 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 196 op/s 2024-09-11T22:03:17.418 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:16 smithi001 ceph-mon[30935]: pgmap v186: 97 pgs: 97 active+clean; 182 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 196 op/s 2024-09-11T22:03:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:17.467+0000 7f1d6cff9640 1 -- 172.21.15.1:0/2963445660 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a8bc8f0d70 con 0x55a8bca0ac20 2024-09-11T22:03:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:17.467+0000 7f1d6cff9640 10 client.16641 handle_command_reply: tid=0 2024-09-11T22:03:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:17.467+0000 7f1d8e208740 1 client.16641 shutdown 2024-09-11T22:03:17.470 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:17.467+0000 7f1d8e208740 1 -- 172.21.15.1:0/2963445660 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8bc984af0 msgr2=0x55a8bc998110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:17.470 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:17.467+0000 7f1d8e208740 1 --2- 172.21.15.1:0/2963445660 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8bc984af0 0x55a8bc998110 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c009bb0 tx=0x7f1d7c009be0 comp rx=0 tx=0).stop 2024-09-11T22:03:17.470 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:17.467+0000 7f1d8e208740 1 -- 172.21.15.1:0/2963445660 shutdown_connections 2024-09-11T22:03:17.470 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:17.467+0000 7f1d8e208740 1 --2- 172.21.15.1:0/2963445660 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55a8bca0ac20 0x55a8bca0d040 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:17.470 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:17.467+0000 7f1d8e208740 1 --2- 172.21.15.1:0/2963445660 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8bc98fcb0 0x55a8bc989bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:17.470 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:17.467+0000 7f1d8e208740 1 --2- 172.21.15.1:0/2963445660 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a8bc985410 0x55a8bc989690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:17.470 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:17.467+0000 7f1d6cff9640 0 client.16641 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:03:17.470 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:17.467+0000 7f1d8e208740 1 --2- 172.21.15.1:0/2963445660 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8bc984af0 0x55a8bc998110 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:17.470 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:17.467+0000 7f1d8e208740 1 -- 172.21.15.1:0/2963445660 >> 172.21.15.1:0/2963445660 conn(0x55a8bc981060 msgr2=0x55a8bc9864c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:17.470 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:17.468+0000 7f1d8e208740 1 -- 172.21.15.1:0/2963445660 shutdown_connections 2024-09-11T22:03:17.470 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:17.468+0000 7f1d8e208740 1 -- 172.21.15.1:0/2963445660 wait complete. 2024-09-11T22:03:17.471 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:17.468+0000 7f1d8e208740 20 client.16641 trim_cache size 0 max 16384 2024-09-11T22:03:17.471 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-09-11T22:03:17.471 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-09-11T22:03:17.471 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-09-11T22:03:17.471 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-09-11T22:03:17.472 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:17.468+0000 7f1d8b396640 1 -- 172.21.15.1:0/3953953623 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f1d700a2cf0 msgr2=0x7f1d700a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:17.472 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:17.468+0000 7f1d8b396640 1 --2- 172.21.15.1:0/3953953623 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f1d700a2cf0 0x7f1d700a51b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1d8000c6a0 tx=0x7f1d800d5530 comp rx=0 tx=0).stop 2024-09-11T22:03:17.472 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:17.468+0000 7f1d8b396640 1 -- 172.21.15.1:0/3953953623 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d84101ac0 msgr2=0x7f1d84198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:17.472 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:17.468+0000 7f1d8b396640 1 --2- 172.21.15.1:0/3953953623 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d84101ac0 0x7f1d84198460 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1d8000e9f0 tx=0x7f1d8000edf0 comp rx=0 tx=0).stop 2024-09-11T22:03:17.472 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:17.469+0000 7f1d8b396640 1 -- 172.21.15.1:0/3953953623 shutdown_connections 2024-09-11T22:03:17.472 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:17.469+0000 7f1d8b396640 1 --2- 172.21.15.1:0/3953953623 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f1d700a2cf0 0x7f1d700a51b0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:17.472 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:17.469+0000 7f1d8b396640 1 --2- 172.21.15.1:0/3953953623 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d84107ad0 0x7f1d8419cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:17.472 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:17.469+0000 7f1d8b396640 1 --2- 172.21.15.1:0/3953953623 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d84102460 0x7f1d841989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:17.472 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:17.469+0000 7f1d8b396640 1 --2- 172.21.15.1:0/3953953623 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d84101ac0 0x7f1d84198460 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:17.473 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:17.469+0000 7f1d8b396640 1 -- 172.21.15.1:0/3953953623 >> 172.21.15.1:0/3953953623 conn(0x7f1d840fd7a0 msgr2=0x7f1d840fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:17.473 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:17.469+0000 7f1d8b396640 1 -- 172.21.15.1:0/3953953623 shutdown_connections 2024-09-11T22:03:17.473 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:17.469+0000 7f1d8b396640 1 -- 172.21.15.1:0/3953953623 wait complete. 2024-09-11T22:03:17.912 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T22:03:18.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:17 smithi001 ceph-mon[30935]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:03:18.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:17 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/1616198614' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:03:18.280 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:03:18.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:17 smithi053 ceph-mon[34908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:03:18.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:17 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/1616198614' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:03:18.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:17 smithi161 ceph-mon[35615]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:03:18.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:17 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/1616198614' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:03:18.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.861+0000 7fd8bafcd640 1 -- 172.21.15.1:0/203700478 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd8b41082a0 msgr2=0x7fd8b410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:18.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.861+0000 7fd8bafcd640 1 --2- 172.21.15.1:0/203700478 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd8b41082a0 0x7fd8b410ec40 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac009cd0 tx=0x7fd8ac02a8e0 comp rx=0 tx=0).stop 2024-09-11T22:03:18.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.861+0000 7fd8bafcd640 1 -- 172.21.15.1:0/203700478 shutdown_connections 2024-09-11T22:03:18.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.861+0000 7fd8bafcd640 1 --2- 172.21.15.1:0/203700478 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd8b41082a0 0x7fd8b410ec40 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:18.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.861+0000 7fd8bafcd640 1 --2- 172.21.15.1:0/203700478 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8b41036e0 0x7fd8b4107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:18.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.861+0000 7fd8bafcd640 1 --2- 172.21.15.1:0/203700478 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8b4102d30 0x7fd8b4103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:18.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.861+0000 7fd8bafcd640 1 -- 172.21.15.1:0/203700478 >> 172.21.15.1:0/203700478 conn(0x7fd8b40fc7d0 msgr2=0x7fd8b40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:18.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.861+0000 7fd8bafcd640 1 -- 172.21.15.1:0/203700478 shutdown_connections 2024-09-11T22:03:18.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.861+0000 7fd8bafcd640 1 -- 172.21.15.1:0/203700478 wait complete. 2024-09-11T22:03:18.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.862+0000 7fd8bafcd640 1 Processor -- start 2024-09-11T22:03:18.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.862+0000 7fd8bafcd640 1 -- start start 2024-09-11T22:03:18.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.862+0000 7fd8bafcd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8b4102d30 0x7fd8b4100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:18.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.862+0000 7fd8bafcd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8b41036e0 0x7fd8b4100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:18.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.862+0000 7fd8bafcd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd8b41082a0 0x7fd8b4101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:18.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.862+0000 7fd8bafcd640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b41120a0 con 0x7fd8b41082a0 2024-09-11T22:03:18.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.862+0000 7fd8bafcd640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b4111f20 con 0x7fd8b41036e0 2024-09-11T22:03:18.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.862+0000 7fd8bafcd640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b4112220 con 0x7fd8b4102d30 2024-09-11T22:03:18.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.862+0000 7fd8b8d69640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8b4102d30 0x7fd8b4100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:18.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.862+0000 7fd8b956a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd8b41082a0 0x7fd8b4101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:18.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.862+0000 7fd8b8d69640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8b4102d30 0x7fd8b4100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:42070/0 (socket says 172.21.15.1:42070) 2024-09-11T22:03:18.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.862+0000 7fd8b8d69640 1 -- 172.21.15.1:0/3194603310 learned_addr learned my addr 172.21.15.1:0/3194603310 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:03:18.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.862+0000 7fd8b956a640 1 --2- 172.21.15.1:0/3194603310 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd8b41082a0 0x7fd8b4101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52102/0 (socket says 172.21.15.1:52102) 2024-09-11T22:03:18.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.862+0000 7fd8b3fff640 1 --2- 172.21.15.1:0/3194603310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8b41036e0 0x7fd8b4100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:18.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.862+0000 7fd8b956a640 1 -- 172.21.15.1:0/3194603310 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8b4102d30 msgr2=0x7fd8b4100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:18.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.862+0000 7fd8b956a640 1 --2- 172.21.15.1:0/3194603310 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8b4102d30 0x7fd8b4100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:18.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.863+0000 7fd8b956a640 1 -- 172.21.15.1:0/3194603310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8b41036e0 msgr2=0x7fd8b4100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:18.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.863+0000 7fd8b956a640 1 --2- 172.21.15.1:0/3194603310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8b41036e0 0x7fd8b4100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:18.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.863+0000 7fd8b956a640 1 -- 172.21.15.1:0/3194603310 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8b41a6a10 con 0x7fd8b41082a0 2024-09-11T22:03:18.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.863+0000 7fd8b8d69640 1 --2- 172.21.15.1:0/3194603310 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8b4102d30 0x7fd8b4100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:03:18.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.863+0000 7fd8b3fff640 1 --2- 172.21.15.1:0/3194603310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8b41036e0 0x7fd8b4100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:03:18.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.863+0000 7fd8b956a640 1 --2- 172.21.15.1:0/3194603310 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd8b41082a0 0x7fd8b4101410 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac02ae00 tx=0x7fd8ac031df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:18.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.863+0000 7fd8b1ffb640 1 -- 172.21.15.1:0/3194603310 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac003d80 con 0x7fd8b41082a0 2024-09-11T22:03:18.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.863+0000 7fd8bafcd640 1 -- 172.21.15.1:0/3194603310 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8b41a6ca0 con 0x7fd8b41082a0 2024-09-11T22:03:18.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.863+0000 7fd8b1ffb640 1 -- 172.21.15.1:0/3194603310 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8ac004520 con 0x7fd8b41082a0 2024-09-11T22:03:18.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.863+0000 7fd8b1ffb640 1 -- 172.21.15.1:0/3194603310 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac030d00 con 0x7fd8b41082a0 2024-09-11T22:03:18.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.863+0000 7fd8bafcd640 1 -- 172.21.15.1:0/3194603310 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8b41a71e0 con 0x7fd8b41082a0 2024-09-11T22:03:18.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.867+0000 7fd8b1ffb640 1 -- 172.21.15.1:0/3194603310 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7fd8ac0046d0 con 0x7fd8b41082a0 2024-09-11T22:03:18.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.867+0000 7fd8bd3a6740 1 Processor -- start 2024-09-11T22:03:18.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.867+0000 7fd8bd3a6740 1 -- start start 2024-09-11T22:03:18.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.867+0000 7fd8bd3a6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e85aacfb40 0x55e85aace090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:18.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.867+0000 7fd8bd3a6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e85aace5d0 0x55e85aacea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:18.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.867+0000 7fd8bd3a6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e85aad9370 0x55e85aadb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:18.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.867+0000 7fd8bd3a6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55e85aa342f0 con 0x55e85aad9370 2024-09-11T22:03:18.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.867+0000 7fd8bd3a6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55e85a9dbcf0 con 0x55e85aacfb40 2024-09-11T22:03:18.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.867+0000 7fd8bd3a6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55e85aa302b0 con 0x55e85aace5d0 2024-09-11T22:03:18.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.867+0000 7fd8b3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e85aad9370 0x55e85aadb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:18.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.867+0000 7fd8b8d69640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e85aace5d0 0x55e85aacea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:18.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.868+0000 7fd8b1ffb640 1 --2- 172.21.15.1:0/3194603310 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fd8a40a2cf0 0x7fd8a40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:18.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.867+0000 7fd8b3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e85aad9370 0x55e85aadb760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52104/0 (socket says 172.21.15.1:52104) 2024-09-11T22:03:18.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.868+0000 7fd8b8d69640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e85aace5d0 0x55e85aacea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:42084/0 (socket says 172.21.15.1:42084) 2024-09-11T22:03:18.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.868+0000 7fd8b3fff640 1 -- 172.21.15.1:0/536354430 learned_addr learned my addr 172.21.15.1:0/536354430 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:03:18.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.868+0000 7fd8b3fff640 1 --2- 172.21.15.1:0/536354430 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e85aacfb40 0x55e85aace090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:18.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.868+0000 7fd8b956a640 1 --2- 172.21.15.1:0/3194603310 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fd8a40a2cf0 0x7fd8a40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:18.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.868+0000 7fd8b3fff640 1 -- 172.21.15.1:0/536354430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e85aace5d0 msgr2=0x55e85aacea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:18.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.868+0000 7fd8b3fff640 1 --2- 172.21.15.1:0/536354430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e85aace5d0 0x55e85aacea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:18.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.868+0000 7fd8b3fff640 1 -- 172.21.15.1:0/536354430 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e85aacfb40 msgr2=0x55e85aace090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:18.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.868+0000 7fd8b3fff640 1 --2- 172.21.15.1:0/536354430 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e85aacfb40 0x55e85aace090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:18.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.868+0000 7fd8b3fff640 1 -- 172.21.15.1:0/536354430 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e85a9c03a0 con 0x55e85aad9370 2024-09-11T22:03:18.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.868+0000 7fd8b1ffb640 1 -- 172.21.15.1:0/3194603310 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7fd8ac02e030 con 0x7fd8b41082a0 2024-09-11T22:03:18.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.868+0000 7fd8b3fff640 1 --2- 172.21.15.1:0/536354430 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e85aad9370 0x55e85aadb760 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0011b90 tx=0x7fd8a0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66b13bcd60d6e1d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:18.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.868+0000 7fd89effd640 1 -- 172.21.15.1:0/536354430 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a000fc50 con 0x55e85aad9370 2024-09-11T22:03:18.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.868+0000 7fd8b956a640 1 --2- 172.21.15.1:0/3194603310 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fd8a40a2cf0 0x7fd8a40a51b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac0b9370 tx=0x7fd8ac08eed0 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:18.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.868+0000 7fd89effd640 1 -- 172.21.15.1:0/536354430 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8a000fdf0 con 0x55e85aad9370 2024-09-11T22:03:18.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.868+0000 7fd89effd640 1 -- 172.21.15.1:0/536354430 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a0004b00 con 0x55e85aad9370 2024-09-11T22:03:18.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.869+0000 7fd8bd3a6740 1 -- 172.21.15.1:0/536354430 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e85aad9370 msgr2=0x55e85aadb760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:18.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.869+0000 7fd8bd3a6740 1 --2- 172.21.15.1:0/536354430 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e85aad9370 0x55e85aadb760 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0011b90 tx=0x7fd8a0004730 comp rx=0 tx=0).stop 2024-09-11T22:03:18.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.869+0000 7fd8bd3a6740 1 -- 172.21.15.1:0/536354430 shutdown_connections 2024-09-11T22:03:18.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.869+0000 7fd8bd3a6740 1 --2- 172.21.15.1:0/536354430 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e85aad9370 0x55e85aadb760 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:18.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.869+0000 7fd8bd3a6740 1 --2- 172.21.15.1:0/536354430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e85aace5d0 0x55e85aacea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:18.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.869+0000 7fd8bd3a6740 1 --2- 172.21.15.1:0/536354430 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e85aacfb40 0x55e85aace090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:18.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.869+0000 7fd8bd3a6740 1 -- 172.21.15.1:0/536354430 >> 172.21.15.1:0/536354430 conn(0x55e85aa97310 msgr2=0x55e85aacac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:18.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.869+0000 7fd8bd3a6740 1 -- 172.21.15.1:0/536354430 shutdown_connections 2024-09-11T22:03:18.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.869+0000 7fd8bd3a6740 1 -- 172.21.15.1:0/536354430 wait complete. 2024-09-11T22:03:18.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.870+0000 7fd8bd3a6740 1 -- start start 2024-09-11T22:03:18.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.870+0000 7fd8bd3a6740 1 Processor -- start 2024-09-11T22:03:18.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.870+0000 7fd8bd3a6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e85aace5d0 0x55e85aad8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:18.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.870+0000 7fd8bd3a6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e85aacfb40 0x55e85aad5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:18.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.870+0000 7fd8bd3a6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e85aad9370 0x55e85aad65d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:18.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.870+0000 7fd8bd3a6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55e85aadc070 con 0x55e85aace5d0 2024-09-11T22:03:18.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.870+0000 7fd8bd3a6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55e85a9ef060 con 0x55e85aacfb40 2024-09-11T22:03:18.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.870+0000 7fd8bd3a6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55e85aadc1f0 con 0x55e85aad9370 2024-09-11T22:03:18.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.870+0000 7fd8bd3a6740 10 client.0 _ll_register_callbacks cb 0x55e85a7428c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:03:18.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.870+0000 7fd8b3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e85aace5d0 0x55e85aad8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:18.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.870+0000 7fd8b3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e85aace5d0 0x55e85aad8d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52110/0 (socket says 172.21.15.1:52110) 2024-09-11T22:03:18.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.870+0000 7fd8b3fff640 1 -- 172.21.15.1:0/2389841752 learned_addr learned my addr 172.21.15.1:0/2389841752 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:03:18.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.870+0000 7fd8b8d69640 1 --2- 172.21.15.1:0/2389841752 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e85aacfb40 0x55e85aad5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:18.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.871+0000 7fd8b3fff640 1 --2- 172.21.15.1:0/2389841752 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e85aad9370 0x55e85aad65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:18.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.871+0000 7fd8b8d69640 1 -- 172.21.15.1:0/2389841752 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e85aad9370 msgr2=0x55e85aad65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:18.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.871+0000 7fd8b8d69640 1 --2- 172.21.15.1:0/2389841752 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e85aad9370 0x55e85aad65d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:18.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.871+0000 7fd8b8d69640 1 -- 172.21.15.1:0/2389841752 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e85aace5d0 msgr2=0x55e85aad8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:18.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.871+0000 7fd8b8d69640 1 --2- 172.21.15.1:0/2389841752 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e85aace5d0 0x55e85aad8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:18.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.871+0000 7fd8b8d69640 1 -- 172.21.15.1:0/2389841752 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e85a7a53c0 con 0x55e85aacfb40 2024-09-11T22:03:18.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.871+0000 7fd8b8d69640 1 --2- 172.21.15.1:0/2389841752 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e85aacfb40 0x55e85aad5fe0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x55e85aacbac0 tx=0x7fd8a800f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42ec24242ec229a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:18.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.871+0000 7fd893fff640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-11T22:03:18.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.871+0000 7fd893fff640 1 -- 172.21.15.1:0/2389841752 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a8004050 con 0x55e85aacfb40 2024-09-11T22:03:18.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.871+0000 7fd893fff640 1 -- 172.21.15.1:0/2389841752 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7fd8a80041f0 con 0x55e85aacfb40 2024-09-11T22:03:18.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.872+0000 7fd8bd3a6740 1 -- 172.21.15.1:0/2389841752 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8a000fdf0 con 0x55e85aacfb40 2024-09-11T22:03:18.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.872+0000 7fd893fff640 1 -- 172.21.15.1:0/2389841752 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fd8a80043f0 con 0x55e85aacfb40 2024-09-11T22:03:18.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.872+0000 7fd8bd3a6740 10 client.26207 fetch_fsmap learned FSMap version 36 2024-09-11T22:03:18.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.872+0000 7fd8bd3a6740 1 -- 172.21.15.1:0/2389841752 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55e85aaf8b70 con 0x55e85aacfb40 2024-09-11T22:03:18.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.872+0000 7fd893fff640 1 -- 172.21.15.1:0/2389841752 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8a000fdf0 con 0x55e85aacfb40 2024-09-11T22:03:18.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.872+0000 7fd893fff640 1 -- 172.21.15.1:0/2389841752 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7fd8a8010550 con 0x55e85aacfb40 2024-09-11T22:03:18.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.872+0000 7fd8bd3a6740 10 client.26207 fetch_fsmap finished waiting for FSMap version 36 2024-09-11T22:03:18.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.872+0000 7fd8bd3a6740 10 client.26207 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-11T22:03:18.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.872+0000 7fd8bd3a6740 20 client.26207 populate_metadata read hostname 'smithi001' 2024-09-11T22:03:18.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.872+0000 7fd8bd3a6740 1 --2- 172.21.15.1:0/2389841752 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x55e85aaf9fd0 0x55e85aafc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:18.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.872+0000 7fd8bd3a6740 4 client.26207 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:03:18.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.872+0000 7fd8bd3a6740 1 -- 172.21.15.1:0/2389841752 --> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e85aa41860 con 0x55e85aaf9fd0 2024-09-11T22:03:18.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.873+0000 7fd8b3fff640 1 --2- 172.21.15.1:0/2389841752 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x55e85aaf9fd0 0x55e85aafc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:18.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.873+0000 7fd893fff640 10 client.26207 ms_handle_connect on v2:172.21.15.161:6834/828381592 2024-09-11T22:03:18.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.873+0000 7fd8b3fff640 1 --2- 172.21.15.1:0/2389841752 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x55e85aaf9fd0 0x55e85aafc410 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ab166e5b6ef775a2 server_cookie=4867adad7bd94d85 in_seq=0 out_seq=0 2024-09-11T22:03:18.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.874+0000 7fd893fff640 1 -- 172.21.15.1:0/2389841752 <== mds.4 v2:172.21.15.161:6834/828381592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e85aa41860 con 0x55e85aaf9fd0 2024-09-11T22:03:18.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.874+0000 7fd893fff640 10 client.26207 handle_command_reply: tid=0 2024-09-11T22:03:18.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.874+0000 7fd8bd3a6740 1 client.26207 shutdown 2024-09-11T22:03:18.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.874+0000 7fd8bd3a6740 1 -- 172.21.15.1:0/2389841752 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e85aacfb40 msgr2=0x55e85aad5fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:18.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.874+0000 7fd8bd3a6740 1 --2- 172.21.15.1:0/2389841752 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e85aacfb40 0x55e85aad5fe0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x55e85aacbac0 tx=0x7fd8a800f790 comp rx=0 tx=0).stop 2024-09-11T22:03:18.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.874+0000 7fd8bd3a6740 1 -- 172.21.15.1:0/2389841752 shutdown_connections 2024-09-11T22:03:18.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.874+0000 7fd8bd3a6740 1 --2- 172.21.15.1:0/2389841752 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e85aad9370 0x55e85aad65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:18.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.874+0000 7fd8bd3a6740 1 --2- 172.21.15.1:0/2389841752 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x55e85aaf9fd0 0x55e85aafc410 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:18.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.874+0000 7fd893fff640 0 client.26207 ms_handle_reset on v2:172.21.15.161:6834/828381592 2024-09-11T22:03:18.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.874+0000 7fd8bd3a6740 1 --2- 172.21.15.1:0/2389841752 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e85aacfb40 0x55e85aad5fe0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:18.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.874+0000 7fd8bd3a6740 1 --2- 172.21.15.1:0/2389841752 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e85aace5d0 0x55e85aad8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:18.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.874+0000 7fd8bd3a6740 1 -- 172.21.15.1:0/2389841752 >> 172.21.15.1:0/2389841752 conn(0x55e85aa97310 msgr2=0x55e85aad0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:18.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.874+0000 7fd8bd3a6740 1 -- 172.21.15.1:0/2389841752 shutdown_connections 2024-09-11T22:03:18.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.874+0000 7fd8bd3a6740 1 -- 172.21.15.1:0/2389841752 wait complete. 2024-09-11T22:03:18.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.874+0000 7fd8bd3a6740 20 client.26207 trim_cache size 0 max 16384 2024-09-11T22:03:18.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.888+0000 7fd8bd3a6740 1 Processor -- start 2024-09-11T22:03:18.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.888+0000 7fd8bd3a6740 1 -- start start 2024-09-11T22:03:18.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.888+0000 7fd8bd3a6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e85aace5d0 0x55e85aacae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:18.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.888+0000 7fd8bd3a6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e85aacfb40 0x55e85aacf300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:18.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.888+0000 7fd8bd3a6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e85aad9370 0x55e85aad58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:18.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.888+0000 7fd8bd3a6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b41a7410 con 0x55e85aad9370 2024-09-11T22:03:18.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.888+0000 7fd8bd3a6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b41a9030 con 0x55e85aace5d0 2024-09-11T22:03:18.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.888+0000 7fd8bd3a6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b41a7a40 con 0x55e85aacfb40 2024-09-11T22:03:18.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.888+0000 7fd8b8d69640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e85aacfb40 0x55e85aacf300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:18.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.888+0000 7fd8b3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e85aace5d0 0x55e85aacae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:18.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.888+0000 7fd8b3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e85aace5d0 0x55e85aacae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:42582/0 (socket says 172.21.15.1:42582) 2024-09-11T22:03:18.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.888+0000 7fd8b8d69640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e85aacfb40 0x55e85aacf300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:42090/0 (socket says 172.21.15.1:42090) 2024-09-11T22:03:18.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.888+0000 7fd8b3fff640 1 -- 172.21.15.1:0/3181434241 learned_addr learned my addr 172.21.15.1:0/3181434241 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:03:18.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.888+0000 7fd8b3fff640 1 --2- 172.21.15.1:0/3181434241 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e85aad9370 0x55e85aad58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:18.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.889+0000 7fd8b8d69640 1 -- 172.21.15.1:0/3181434241 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e85aace5d0 msgr2=0x55e85aacae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:18.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.889+0000 7fd8b8d69640 1 --2- 172.21.15.1:0/3181434241 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e85aace5d0 0x55e85aacae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:18.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.889+0000 7fd8b8d69640 1 -- 172.21.15.1:0/3181434241 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e85aad9370 msgr2=0x55e85aad58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:18.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.889+0000 7fd8b8d69640 1 --2- 172.21.15.1:0/3181434241 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e85aad9370 0x55e85aad58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:18.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.889+0000 7fd8b8d69640 1 -- 172.21.15.1:0/3181434241 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e85aaca4f0 con 0x55e85aacfb40 2024-09-11T22:03:18.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.889+0000 7fd8b3fff640 1 --2- 172.21.15.1:0/3181434241 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e85aace5d0 0x55e85aacae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:03:18.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.889+0000 7fd8b8d69640 1 --2- 172.21.15.1:0/3181434241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e85aacfb40 0x55e85aacf300 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8010a70 tx=0x7fd8a8009c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eecfd705631fbf30 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:18.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.889+0000 7fd89e7fc640 1 -- 172.21.15.1:0/3181434241 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a8010af0 con 0x55e85aacfb40 2024-09-11T22:03:18.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.889+0000 7fd89e7fc640 1 -- 172.21.15.1:0/3181434241 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8a8010c90 con 0x55e85aacfb40 2024-09-11T22:03:18.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.889+0000 7fd89e7fc640 1 -- 172.21.15.1:0/3181434241 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a8005450 con 0x55e85aacfb40 2024-09-11T22:03:18.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.890+0000 7fd8bd3a6740 1 -- 172.21.15.1:0/3181434241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e85aacfb40 msgr2=0x55e85aacf300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:18.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.890+0000 7fd8bd3a6740 1 --2- 172.21.15.1:0/3181434241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e85aacfb40 0x55e85aacf300 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8010a70 tx=0x7fd8a8009c10 comp rx=0 tx=0).stop 2024-09-11T22:03:18.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.890+0000 7fd8bd3a6740 1 -- 172.21.15.1:0/3181434241 shutdown_connections 2024-09-11T22:03:18.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.890+0000 7fd8bd3a6740 1 --2- 172.21.15.1:0/3181434241 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e85aad9370 0x55e85aad58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:18.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.890+0000 7fd8bd3a6740 1 --2- 172.21.15.1:0/3181434241 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e85aacfb40 0x55e85aacf300 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:18.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.890+0000 7fd8bd3a6740 1 --2- 172.21.15.1:0/3181434241 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e85aace5d0 0x55e85aacae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:18.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.890+0000 7fd8bd3a6740 1 -- 172.21.15.1:0/3181434241 >> 172.21.15.1:0/3181434241 conn(0x55e85aa97310 msgr2=0x55e85aaca700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:18.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.890+0000 7fd8bd3a6740 1 -- 172.21.15.1:0/3181434241 shutdown_connections 2024-09-11T22:03:18.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.890+0000 7fd8bd3a6740 1 -- 172.21.15.1:0/3181434241 wait complete. 2024-09-11T22:03:18.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.890+0000 7fd8bd3a6740 1 -- start start 2024-09-11T22:03:18.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.890+0000 7fd8bd3a6740 1 Processor -- start 2024-09-11T22:03:18.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.891+0000 7fd8bd3a6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e85aace5d0 0x55e85aae1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:18.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.891+0000 7fd8bd3a6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e85aacfb40 0x55e85aad2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:18.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.891+0000 7fd8bd3a6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e85aad9370 0x55e85aad3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:18.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.891+0000 7fd8bd3a6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b41a88a0 con 0x55e85aacfb40 2024-09-11T22:03:18.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.891+0000 7fd8bd3a6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b41a8110 con 0x55e85aace5d0 2024-09-11T22:03:18.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.891+0000 7fd8bd3a6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b4072350 con 0x55e85aad9370 2024-09-11T22:03:18.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.891+0000 7fd8bd3a6740 10 client.0 _ll_register_callbacks cb 0x7fd8a8009cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:03:18.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.891+0000 7fd8b8d69640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e85aacfb40 0x55e85aad2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:18.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.891+0000 7fd8b3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e85aace5d0 0x55e85aae1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:18.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.891+0000 7fd8b3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e85aace5d0 0x55e85aae1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:42598/0 (socket says 172.21.15.1:42598) 2024-09-11T22:03:18.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.891+0000 7fd8b3fff640 1 -- 172.21.15.1:0/2484773385 learned_addr learned my addr 172.21.15.1:0/2484773385 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:03:18.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.891+0000 7fd8b8d69640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e85aacfb40 0x55e85aad2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52122/0 (socket says 172.21.15.1:52122) 2024-09-11T22:03:18.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.891+0000 7fd8b3fff640 1 --2- 172.21.15.1:0/2484773385 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e85aad9370 0x55e85aad3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:18.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.891+0000 7fd8b3fff640 1 -- 172.21.15.1:0/2484773385 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e85aad9370 msgr2=0x55e85aad3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:18.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.891+0000 7fd8b3fff640 1 --2- 172.21.15.1:0/2484773385 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e85aad9370 0x55e85aad3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:18.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.891+0000 7fd8b3fff640 1 -- 172.21.15.1:0/2484773385 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e85aacfb40 msgr2=0x55e85aad2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:18.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.891+0000 7fd8b3fff640 1 --2- 172.21.15.1:0/2484773385 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e85aacfb40 0x55e85aad2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:18.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.891+0000 7fd8b3fff640 1 -- 172.21.15.1:0/2484773385 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e85aad3840 con 0x55e85aace5d0 2024-09-11T22:03:18.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.892+0000 7fd8b3fff640 1 --2- 172.21.15.1:0/2484773385 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e85aace5d0 0x55e85aae1ad0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd8a00059d0 tx=0x7fd8a0021040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a8a797848341cd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:18.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.892+0000 7fd893fff640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-11T22:03:18.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.892+0000 7fd893fff640 1 -- 172.21.15.1:0/2484773385 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a0034070 con 0x55e85aace5d0 2024-09-11T22:03:18.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.892+0000 7fd893fff640 1 -- 172.21.15.1:0/2484773385 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7fd8a00054b0 con 0x55e85aace5d0 2024-09-11T22:03:18.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.892+0000 7fd8bd3a6740 1 -- 172.21.15.1:0/2484773385 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8a8010c90 con 0x55e85aace5d0 2024-09-11T22:03:18.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.892+0000 7fd893fff640 1 -- 172.21.15.1:0/2484773385 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fd8a0043050 con 0x55e85aace5d0 2024-09-11T22:03:18.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.892+0000 7fd8bd3a6740 10 client.26213 fetch_fsmap learned FSMap version 36 2024-09-11T22:03:18.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.892+0000 7fd8bd3a6740 1 -- 172.21.15.1:0/2484773385 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55e85ab64b70 con 0x55e85aace5d0 2024-09-11T22:03:18.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.892+0000 7fd893fff640 1 -- 172.21.15.1:0/2484773385 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8a8010c90 con 0x55e85aace5d0 2024-09-11T22:03:18.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.892+0000 7fd893fff640 1 -- 172.21.15.1:0/2484773385 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 36) ==== 8394+0+0 (secure 0 0 0) 0x7fd8a003a070 con 0x55e85aace5d0 2024-09-11T22:03:18.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.893+0000 7fd8bd3a6740 10 client.26213 fetch_fsmap finished waiting for FSMap version 36 2024-09-11T22:03:18.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.893+0000 7fd8bd3a6740 10 client.26213 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-11T22:03:18.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.893+0000 7fd8bd3a6740 20 client.26213 populate_metadata read hostname 'smithi001' 2024-09-11T22:03:18.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.893+0000 7fd8bd3a6740 1 --2- 172.21.15.1:0/2484773385 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x55e85aaf9fd0 0x55e85ab67090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:18.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.893+0000 7fd8bd3a6740 4 client.26213 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:03:18.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.893+0000 7fd8bd3a6740 1 -- 172.21.15.1:0/2484773385 --> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e85aa3bd70 con 0x55e85aaf9fd0 2024-09-11T22:03:18.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.893+0000 7fd8b8d69640 1 --2- 172.21.15.1:0/2484773385 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x55e85aaf9fd0 0x55e85ab67090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:18.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.893+0000 7fd893fff640 10 client.26213 ms_handle_connect on v2:172.21.15.161:6834/828381592 2024-09-11T22:03:18.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.893+0000 7fd8b8d69640 1 --2- 172.21.15.1:0/2484773385 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x55e85aaf9fd0 0x55e85ab67090 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=31bcb6e0722274b6 server_cookie=77be4c047dea09df in_seq=0 out_seq=0 2024-09-11T22:03:18.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.893+0000 7fd893fff640 1 -- 172.21.15.1:0/2484773385 <== mds.4 v2:172.21.15.161:6834/828381592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e85aa3bd70 con 0x55e85aaf9fd0 2024-09-11T22:03:18.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.893+0000 7fd893fff640 10 client.26213 handle_command_reply: tid=0 2024-09-11T22:03:18.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.893+0000 7fd8bd3a6740 1 client.26213 shutdown 2024-09-11T22:03:18.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.894+0000 7fd8bd3a6740 1 -- 172.21.15.1:0/2484773385 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e85aace5d0 msgr2=0x55e85aae1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:18.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.894+0000 7fd8bd3a6740 1 --2- 172.21.15.1:0/2484773385 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e85aace5d0 0x55e85aae1ad0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd8a00059d0 tx=0x7fd8a0021040 comp rx=0 tx=0).stop 2024-09-11T22:03:18.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.894+0000 7fd8bd3a6740 1 -- 172.21.15.1:0/2484773385 shutdown_connections 2024-09-11T22:03:18.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.894+0000 7fd8bd3a6740 1 --2- 172.21.15.1:0/2484773385 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e85aad9370 0x55e85aad3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:18.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.894+0000 7fd8bd3a6740 1 --2- 172.21.15.1:0/2484773385 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e85aacfb40 0x55e85aad2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:18.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.894+0000 7fd8bd3a6740 1 --2- 172.21.15.1:0/2484773385 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x55e85aaf9fd0 0x55e85ab67090 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:18.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.894+0000 7fd8bd3a6740 1 --2- 172.21.15.1:0/2484773385 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e85aace5d0 0x55e85aae1ad0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:18.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.894+0000 7fd8bd3a6740 1 -- 172.21.15.1:0/2484773385 >> 172.21.15.1:0/2484773385 conn(0x55e85aa97310 msgr2=0x55e85aad0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:18.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.894+0000 7fd893fff640 0 client.26213 ms_handle_reset on v2:172.21.15.161:6834/828381592 2024-09-11T22:03:18.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.894+0000 7fd8bd3a6740 1 -- 172.21.15.1:0/2484773385 shutdown_connections 2024-09-11T22:03:18.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.894+0000 7fd8bd3a6740 1 -- 172.21.15.1:0/2484773385 wait complete. 2024-09-11T22:03:18.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.894+0000 7fd8bd3a6740 20 client.26213 trim_cache size 0 max 16384 2024-09-11T22:03:18.902 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-11T22:03:18.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.895+0000 7fd8bafcd640 1 -- 172.21.15.1:0/3194603310 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fd8a40a2cf0 msgr2=0x7fd8a40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:18.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.895+0000 7fd8bafcd640 1 --2- 172.21.15.1:0/3194603310 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fd8a40a2cf0 0x7fd8a40a51b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac0b9370 tx=0x7fd8ac08eed0 comp rx=0 tx=0).stop 2024-09-11T22:03:18.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.895+0000 7fd8bafcd640 1 -- 172.21.15.1:0/3194603310 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd8b41082a0 msgr2=0x7fd8b4101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:18.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.895+0000 7fd8bafcd640 1 --2- 172.21.15.1:0/3194603310 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd8b41082a0 0x7fd8b4101410 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac02ae00 tx=0x7fd8ac031df0 comp rx=0 tx=0).stop 2024-09-11T22:03:18.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.895+0000 7fd8bafcd640 1 -- 172.21.15.1:0/3194603310 shutdown_connections 2024-09-11T22:03:18.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.895+0000 7fd8bafcd640 1 --2- 172.21.15.1:0/3194603310 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fd8a40a2cf0 0x7fd8a40a51b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:18.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.895+0000 7fd8bafcd640 1 --2- 172.21.15.1:0/3194603310 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd8b41082a0 0x7fd8b4101410 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:18.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.895+0000 7fd8bafcd640 1 --2- 172.21.15.1:0/3194603310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd8b41036e0 0x7fd8b4100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:18.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.895+0000 7fd8bafcd640 1 --2- 172.21.15.1:0/3194603310 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8b4102d30 0x7fd8b4100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:18.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.895+0000 7fd8bafcd640 1 -- 172.21.15.1:0/3194603310 >> 172.21.15.1:0/3194603310 conn(0x7fd8b40fc7d0 msgr2=0x7fd8b4106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:18.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.895+0000 7fd8bafcd640 1 -- 172.21.15.1:0/3194603310 shutdown_connections 2024-09-11T22:03:18.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:18.895+0000 7fd8bafcd640 1 -- 172.21.15.1:0/3194603310 wait complete. 2024-09-11T22:03:19.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:19 smithi001 ceph-mon[30935]: pgmap v187: 97 pgs: 97 active+clean; 182 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 164 op/s 2024-09-11T22:03:19.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:19 smithi053 ceph-mon[34908]: pgmap v187: 97 pgs: 97 active+clean; 182 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 164 op/s 2024-09-11T22:03:19.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:19 smithi161 ceph-mon[35615]: pgmap v187: 97 pgs: 97 active+clean; 182 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 164 op/s 2024-09-11T22:03:19.891 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T22:03:20.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:20 smithi001 ceph-mon[30935]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:03:20.198 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:03:20.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:20 smithi053 ceph-mon[34908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:03:20.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:20 smithi161 ceph-mon[35615]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:03:20.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.721+0000 7fdb6e32d640 1 -- 172.21.15.1:0/2944543790 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb68102570 msgr2=0x7fdb681029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:20.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.721+0000 7fdb6e32d640 1 --2- 172.21.15.1:0/2944543790 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb68102570 0x7fdb681029b0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c009900 tx=0x7fdb5c02a480 comp rx=0 tx=0).stop 2024-09-11T22:03:20.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.721+0000 7fdb6e32d640 1 -- 172.21.15.1:0/2944543790 shutdown_connections 2024-09-11T22:03:20.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.721+0000 7fdb6e32d640 1 --2- 172.21.15.1:0/2944543790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb68102ef0 0x7fdb6810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:20.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.721+0000 7fdb6e32d640 1 --2- 172.21.15.1:0/2944543790 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb68102570 0x7fdb681029b0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:20.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.721+0000 7fdb6e32d640 1 --2- 172.21.15.1:0/2944543790 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdb68100e20 0x7fdb68101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:20.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.721+0000 7fdb6e32d640 1 -- 172.21.15.1:0/2944543790 >> 172.21.15.1:0/2944543790 conn(0x7fdb680fc7d0 msgr2=0x7fdb680febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:20.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.721+0000 7fdb6e32d640 1 -- 172.21.15.1:0/2944543790 shutdown_connections 2024-09-11T22:03:20.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.721+0000 7fdb6e32d640 1 -- 172.21.15.1:0/2944543790 wait complete. 2024-09-11T22:03:20.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.722+0000 7fdb6e32d640 1 Processor -- start 2024-09-11T22:03:20.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.722+0000 7fdb6e32d640 1 -- start start 2024-09-11T22:03:20.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.722+0000 7fdb6e32d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb68100e20 0x7fdb681984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:20.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.722+0000 7fdb6e32d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdb68102570 0x7fdb681989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:20.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.722+0000 7fdb6e32d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb68102ef0 0x7fdb6819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:20.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.723+0000 7fdb6e32d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdb68110f80 con 0x7fdb68100e20 2024-09-11T22:03:20.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.723+0000 7fdb6e32d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fdb68110e00 con 0x7fdb68102ef0 2024-09-11T22:03:20.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.723+0000 7fdb6e32d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fdb68111100 con 0x7fdb68102570 2024-09-11T22:03:20.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.723+0000 7fdb677fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdb68102570 0x7fdb681989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:20.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.723+0000 7fdb677fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdb68102570 0x7fdb681989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:42128/0 (socket says 172.21.15.1:42128) 2024-09-11T22:03:20.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.723+0000 7fdb677fe640 1 -- 172.21.15.1:0/4066299578 learned_addr learned my addr 172.21.15.1:0/4066299578 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:03:20.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.723+0000 7fdb67fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb68100e20 0x7fdb681984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:20.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.723+0000 7fdb6c8ca640 1 --2- 172.21.15.1:0/4066299578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb68102ef0 0x7fdb6819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:20.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.723+0000 7fdb677fe640 1 -- 172.21.15.1:0/4066299578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb68102ef0 msgr2=0x7fdb6819cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:20.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.723+0000 7fdb677fe640 1 --2- 172.21.15.1:0/4066299578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb68102ef0 0x7fdb6819cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:20.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.723+0000 7fdb677fe640 1 -- 172.21.15.1:0/4066299578 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb68100e20 msgr2=0x7fdb681984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:20.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.723+0000 7fdb677fe640 1 --2- 172.21.15.1:0/4066299578 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb68100e20 0x7fdb681984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:20.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.723+0000 7fdb677fe640 1 -- 172.21.15.1:0/4066299578 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb6819d460 con 0x7fdb68102570 2024-09-11T22:03:20.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.723+0000 7fdb677fe640 1 --2- 172.21.15.1:0/4066299578 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdb68102570 0x7fdb681989f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c006010 tx=0x7fdb5c003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:20.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.724+0000 7fdb657fa640 1 -- 172.21.15.1:0/4066299578 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb5c03f070 con 0x7fdb68102570 2024-09-11T22:03:20.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.724+0000 7fdb6e32d640 1 -- 172.21.15.1:0/4066299578 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb6819d6f0 con 0x7fdb68102570 2024-09-11T22:03:20.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.724+0000 7fdb6e32d640 1 -- 172.21.15.1:0/4066299578 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb681a5030 con 0x7fdb68102570 2024-09-11T22:03:20.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.724+0000 7fdb657fa640 1 -- 172.21.15.1:0/4066299578 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb5c031c30 con 0x7fdb68102570 2024-09-11T22:03:20.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.724+0000 7fdb657fa640 1 -- 172.21.15.1:0/4066299578 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb5c035030 con 0x7fdb68102570 2024-09-11T22:03:20.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.726+0000 7fdb70706740 1 Processor -- start 2024-09-11T22:03:20.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.726+0000 7fdb70706740 1 -- start start 2024-09-11T22:03:20.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.727+0000 7fdb657fa640 1 -- 172.21.15.1:0/4066299578 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7fdb5c042070 con 0x7fdb68102570 2024-09-11T22:03:20.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.727+0000 7fdb70706740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558f1a77c840 0x558f1a77cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:20.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.727+0000 7fdb70706740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f1a77b120 0x558f1a77b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:20.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.727+0000 7fdb70706740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f1a77bb80 0x558f1a790f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:20.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.727+0000 7fdb70706740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x558f1a6e12f0 con 0x558f1a77c840 2024-09-11T22:03:20.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.727+0000 7fdb70706740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x558f1a688cf0 con 0x558f1a77bb80 2024-09-11T22:03:20.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.727+0000 7fdb70706740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x558f1a6dd2b0 con 0x558f1a77b120 2024-09-11T22:03:20.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.727+0000 7fdb67fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f1a77b120 0x558f1a77b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:20.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.727+0000 7fdb6c8ca640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558f1a77c840 0x558f1a77cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:20.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.727+0000 7fdb67fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f1a77b120 0x558f1a77b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:42138/0 (socket says 172.21.15.1:42138) 2024-09-11T22:03:20.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.727+0000 7fdb67fff640 1 -- 172.21.15.1:0/2128207936 learned_addr learned my addr 172.21.15.1:0/2128207936 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:03:20.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.727+0000 7fdb677fe640 1 --2- 172.21.15.1:0/2128207936 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f1a77bb80 0x558f1a790f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:20.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.727+0000 7fdb67fff640 1 -- 172.21.15.1:0/2128207936 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f1a77bb80 msgr2=0x558f1a790f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:20.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.727+0000 7fdb67fff640 1 --2- 172.21.15.1:0/2128207936 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f1a77bb80 0x558f1a790f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:20.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.727+0000 7fdb67fff640 1 -- 172.21.15.1:0/2128207936 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558f1a77c840 msgr2=0x558f1a77cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:20.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.727+0000 7fdb67fff640 1 --2- 172.21.15.1:0/2128207936 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558f1a77c840 0x558f1a77cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:20.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.727+0000 7fdb67fff640 1 -- 172.21.15.1:0/2128207936 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f1a66d3a0 con 0x558f1a77b120 2024-09-11T22:03:20.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.727+0000 7fdb6c8ca640 1 --2- 172.21.15.1:0/2128207936 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558f1a77c840 0x558f1a77cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:03:20.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.728+0000 7fdb67fff640 1 --2- 172.21.15.1:0/2128207936 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f1a77b120 0x558f1a77b560 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fdb58009d20 tx=0x7fdb58011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a1f8c5f11f589af server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:20.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.728+0000 7fdb4e7fc640 1 -- 172.21.15.1:0/2128207936 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb58004230 con 0x558f1a77b120 2024-09-11T22:03:20.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.728+0000 7fdb4e7fc640 1 -- 172.21.15.1:0/2128207936 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb580043d0 con 0x558f1a77b120 2024-09-11T22:03:20.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.728+0000 7fdb4e7fc640 1 -- 172.21.15.1:0/2128207936 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb58019930 con 0x558f1a77b120 2024-09-11T22:03:20.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.728+0000 7fdb657fa640 1 --2- 172.21.15.1:0/4066299578 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fdb500a2cf0 0x7fdb500a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:20.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.729+0000 7fdb657fa640 1 -- 172.21.15.1:0/4066299578 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7fdb5c0efc10 con 0x7fdb68102570 2024-09-11T22:03:20.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.729+0000 7fdb6c8ca640 1 --2- 172.21.15.1:0/4066299578 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fdb500a2cf0 0x7fdb500a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:20.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.729+0000 7fdb70706740 1 -- 172.21.15.1:0/2128207936 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f1a77b120 msgr2=0x558f1a77b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:20.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.729+0000 7fdb70706740 1 --2- 172.21.15.1:0/2128207936 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f1a77b120 0x558f1a77b560 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fdb58009d20 tx=0x7fdb58011810 comp rx=0 tx=0).stop 2024-09-11T22:03:20.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.729+0000 7fdb70706740 1 -- 172.21.15.1:0/2128207936 shutdown_connections 2024-09-11T22:03:20.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.729+0000 7fdb70706740 1 --2- 172.21.15.1:0/2128207936 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f1a77bb80 0x558f1a790f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:20.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.729+0000 7fdb70706740 1 --2- 172.21.15.1:0/2128207936 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f1a77b120 0x558f1a77b560 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:20.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.729+0000 7fdb70706740 1 --2- 172.21.15.1:0/2128207936 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558f1a77c840 0x558f1a77cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:20.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.729+0000 7fdb70706740 1 -- 172.21.15.1:0/2128207936 >> 172.21.15.1:0/2128207936 conn(0x558f1a744310 msgr2=0x558f1a777c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:20.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.729+0000 7fdb6c8ca640 1 --2- 172.21.15.1:0/4066299578 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fdb500a2cf0 0x7fdb500a51b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdb60009fd0 tx=0x7fdb60009500 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:20.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.729+0000 7fdb70706740 1 -- 172.21.15.1:0/2128207936 shutdown_connections 2024-09-11T22:03:20.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.729+0000 7fdb70706740 1 -- 172.21.15.1:0/2128207936 wait complete. 2024-09-11T22:03:20.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.730+0000 7fdb70706740 1 -- start start 2024-09-11T22:03:20.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.730+0000 7fdb70706740 1 Processor -- start 2024-09-11T22:03:20.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.730+0000 7fdb70706740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558f1a77b120 0x558f1a790cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:20.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.730+0000 7fdb70706740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f1a77bb80 0x558f1a78dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:20.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.730+0000 7fdb70706740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f1a77c840 0x558f1a78e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:20.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.730+0000 7fdb70706740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x558f1a791930 con 0x558f1a77b120 2024-09-11T22:03:20.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.730+0000 7fdb70706740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x558f1a69c060 con 0x558f1a77bb80 2024-09-11T22:03:20.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.730+0000 7fdb70706740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x558f1a791ab0 con 0x558f1a77c840 2024-09-11T22:03:20.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.730+0000 7fdb70706740 10 client.0 _ll_register_callbacks cb 0x558f1a3ef8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:03:20.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.730+0000 7fdb6c8ca640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f1a77c840 0x558f1a78e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:20.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.730+0000 7fdb67fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558f1a77b120 0x558f1a790cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:20.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.730+0000 7fdb6c8ca640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f1a77c840 0x558f1a78e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:42154/0 (socket says 172.21.15.1:42154) 2024-09-11T22:03:20.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.730+0000 7fdb67fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558f1a77b120 0x558f1a790cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52146/0 (socket says 172.21.15.1:52146) 2024-09-11T22:03:20.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.730+0000 7fdb6c8ca640 1 -- 172.21.15.1:0/3962594438 learned_addr learned my addr 172.21.15.1:0/3962594438 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:03:20.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.731+0000 7fdb677fe640 1 --2- 172.21.15.1:0/3962594438 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f1a77bb80 0x558f1a78dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:20.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.731+0000 7fdb6c8ca640 1 -- 172.21.15.1:0/3962594438 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f1a77bb80 msgr2=0x558f1a78dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:20.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.731+0000 7fdb6c8ca640 1 --2- 172.21.15.1:0/3962594438 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f1a77bb80 0x558f1a78dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:20.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.731+0000 7fdb6c8ca640 1 -- 172.21.15.1:0/3962594438 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558f1a77b120 msgr2=0x558f1a790cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:20.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.731+0000 7fdb6c8ca640 1 --2- 172.21.15.1:0/3962594438 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558f1a77b120 0x558f1a790cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:20.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.731+0000 7fdb6c8ca640 1 -- 172.21.15.1:0/3962594438 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f1a4523c0 con 0x558f1a77c840 2024-09-11T22:03:20.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.731+0000 7fdb6c8ca640 1 --2- 172.21.15.1:0/3962594438 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f1a77c840 0x558f1a78e4e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fdb60009cb0 tx=0x7fdb60004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9091b9685c3be2d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:20.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.731+0000 7fdb477fe640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-11T22:03:20.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.731+0000 7fdb477fe640 1 -- 172.21.15.1:0/3962594438 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb6002c060 con 0x558f1a77c840 2024-09-11T22:03:20.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.731+0000 7fdb477fe640 1 -- 172.21.15.1:0/3962594438 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7fdb60017030 con 0x558f1a77c840 2024-09-11T22:03:20.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.731+0000 7fdb70706740 1 -- 172.21.15.1:0/3962594438 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb580043d0 con 0x558f1a77c840 2024-09-11T22:03:20.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.732+0000 7fdb477fe640 1 -- 172.21.15.1:0/3962594438 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fdb6001f9b0 con 0x558f1a77c840 2024-09-11T22:03:20.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.732+0000 7fdb70706740 10 client.26188 fetch_fsmap learned FSMap version 38 2024-09-11T22:03:20.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.732+0000 7fdb70706740 1 -- 172.21.15.1:0/3962594438 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x558f1a79d330 con 0x558f1a77c840 2024-09-11T22:03:20.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.732+0000 7fdb477fe640 1 -- 172.21.15.1:0/3962594438 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb580043d0 con 0x558f1a77c840 2024-09-11T22:03:20.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.732+0000 7fdb477fe640 1 -- 172.21.15.1:0/3962594438 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 38) ==== 8414+0+0 (secure 0 0 0) 0x7fdb60027c40 con 0x558f1a77c840 2024-09-11T22:03:20.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.732+0000 7fdb70706740 10 client.26188 fetch_fsmap finished waiting for FSMap version 38 2024-09-11T22:03:20.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.732+0000 7fdb70706740 10 client.26188 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-11T22:03:20.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.732+0000 7fdb70706740 20 client.26188 populate_metadata read hostname 'smithi001' 2024-09-11T22:03:20.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.732+0000 7fdb70706740 1 --2- 172.21.15.1:0/3962594438 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x558f1a79e740 0x558f1a7a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:20.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.732+0000 7fdb70706740 4 client.26188 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:03:20.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.732+0000 7fdb70706740 1 -- 172.21.15.1:0/3962594438 --> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f1a6ee860 con 0x558f1a79e740 2024-09-11T22:03:20.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.732+0000 7fdb67fff640 1 --2- 172.21.15.1:0/3962594438 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x558f1a79e740 0x558f1a7a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:20.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.733+0000 7fdb67fff640 1 --2- 172.21.15.1:0/3962594438 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x558f1a79e740 0x558f1a7a0be0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=39d78a6704217567 server_cookie=f0c8ff7474d63e4c in_seq=0 out_seq=0 2024-09-11T22:03:20.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.733+0000 7fdb477fe640 10 client.26188 ms_handle_connect on v2:172.21.15.161:6836/1799318656 2024-09-11T22:03:20.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.734+0000 7fdb477fe640 1 -- 172.21.15.1:0/3962594438 <== mds.3 v2:172.21.15.161:6836/1799318656 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558f1a6ee860 con 0x558f1a79e740 2024-09-11T22:03:20.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.734+0000 7fdb477fe640 10 client.26188 handle_command_reply: tid=0 2024-09-11T22:03:20.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.734+0000 7fdb70706740 1 client.26188 shutdown 2024-09-11T22:03:20.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.734+0000 7fdb70706740 1 -- 172.21.15.1:0/3962594438 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f1a77c840 msgr2=0x558f1a78e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:20.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.734+0000 7fdb70706740 1 --2- 172.21.15.1:0/3962594438 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f1a77c840 0x558f1a78e4e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fdb60009cb0 tx=0x7fdb60004ab0 comp rx=0 tx=0).stop 2024-09-11T22:03:20.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.734+0000 7fdb70706740 1 -- 172.21.15.1:0/3962594438 shutdown_connections 2024-09-11T22:03:20.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.734+0000 7fdb70706740 1 --2- 172.21.15.1:0/3962594438 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x558f1a79e740 0x558f1a7a0be0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:20.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.734+0000 7fdb477fe640 0 client.26188 ms_handle_reset on v2:172.21.15.161:6836/1799318656 2024-09-11T22:03:20.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.734+0000 7fdb70706740 1 --2- 172.21.15.1:0/3962594438 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f1a77c840 0x558f1a78e4e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:20.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.734+0000 7fdb70706740 1 --2- 172.21.15.1:0/3962594438 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f1a77bb80 0x558f1a78dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:20.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.734+0000 7fdb70706740 1 --2- 172.21.15.1:0/3962594438 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558f1a77b120 0x558f1a790cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:20.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.734+0000 7fdb70706740 1 -- 172.21.15.1:0/3962594438 >> 172.21.15.1:0/3962594438 conn(0x558f1a744310 msgr2=0x558f1a77d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:20.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.734+0000 7fdb70706740 1 -- 172.21.15.1:0/3962594438 shutdown_connections 2024-09-11T22:03:20.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.734+0000 7fdb70706740 1 -- 172.21.15.1:0/3962594438 wait complete. 2024-09-11T22:03:20.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.735+0000 7fdb70706740 20 client.26188 trim_cache size 0 max 16384 2024-09-11T22:03:20.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.750+0000 7fdb70706740 1 Processor -- start 2024-09-11T22:03:20.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.750+0000 7fdb70706740 1 -- start start 2024-09-11T22:03:20.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.750+0000 7fdb70706740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558f1a77b120 0x558f1a79ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:20.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.750+0000 7fdb70706740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f1a77bb80 0x558f1a781da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:20.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.751+0000 7fdb70706740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f1a77c840 0x558f1a7a64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:20.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.751+0000 7fdb70706740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5400e8d0 con 0x558f1a77b120 2024-09-11T22:03:20.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.751+0000 7fdb70706740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fdb54010680 con 0x558f1a77c840 2024-09-11T22:03:20.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.751+0000 7fdb67fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558f1a77b120 0x558f1a79ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:20.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.751+0000 7fdb677fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f1a77bb80 0x558f1a781da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:20.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.751+0000 7fdb67fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558f1a77b120 0x558f1a79ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52148/0 (socket says 172.21.15.1:52148) 2024-09-11T22:03:20.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.751+0000 7fdb67fff640 1 -- 172.21.15.1:0/1544377435 learned_addr learned my addr 172.21.15.1:0/1544377435 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:03:20.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.751+0000 7fdb70706740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5400f000 con 0x558f1a77bb80 2024-09-11T22:03:20.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.751+0000 7fdb6c8ca640 1 --2- 172.21.15.1:0/1544377435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f1a77c840 0x558f1a7a64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:20.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.751+0000 7fdb6c8ca640 1 -- 172.21.15.1:0/1544377435 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f1a77bb80 msgr2=0x558f1a781da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:20.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.751+0000 7fdb6c8ca640 1 --2- 172.21.15.1:0/1544377435 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f1a77bb80 0x558f1a781da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:20.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.751+0000 7fdb6c8ca640 1 -- 172.21.15.1:0/1544377435 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558f1a77b120 msgr2=0x558f1a79ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:20.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.751+0000 7fdb6c8ca640 1 --2- 172.21.15.1:0/1544377435 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558f1a77b120 0x558f1a79ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:20.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.751+0000 7fdb6c8ca640 1 -- 172.21.15.1:0/1544377435 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f1a78cf50 con 0x558f1a77c840 2024-09-11T22:03:20.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.752+0000 7fdb6c8ca640 1 --2- 172.21.15.1:0/1544377435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f1a77c840 0x558f1a7a64b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fdb6001c630 tx=0x7fdb60005c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=794ee2a3eec93415 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:20.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.752+0000 7fdb4dffb640 1 -- 172.21.15.1:0/1544377435 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb60011470 con 0x558f1a77c840 2024-09-11T22:03:20.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.752+0000 7fdb4dffb640 1 -- 172.21.15.1:0/1544377435 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb6001c660 con 0x558f1a77c840 2024-09-11T22:03:20.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.752+0000 7fdb4dffb640 1 -- 172.21.15.1:0/1544377435 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb60004280 con 0x558f1a77c840 2024-09-11T22:03:20.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.753+0000 7fdb70706740 1 -- 172.21.15.1:0/1544377435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f1a77c840 msgr2=0x558f1a7a64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:20.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.753+0000 7fdb70706740 1 --2- 172.21.15.1:0/1544377435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f1a77c840 0x558f1a7a64b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fdb6001c630 tx=0x7fdb60005c10 comp rx=0 tx=0).stop 2024-09-11T22:03:20.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.753+0000 7fdb70706740 1 -- 172.21.15.1:0/1544377435 shutdown_connections 2024-09-11T22:03:20.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.753+0000 7fdb70706740 1 --2- 172.21.15.1:0/1544377435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f1a77c840 0x558f1a7a64b0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:20.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.753+0000 7fdb70706740 1 --2- 172.21.15.1:0/1544377435 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f1a77bb80 0x558f1a781da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:20.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.753+0000 7fdb70706740 1 --2- 172.21.15.1:0/1544377435 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558f1a77b120 0x558f1a79ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:20.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.753+0000 7fdb70706740 1 -- 172.21.15.1:0/1544377435 >> 172.21.15.1:0/1544377435 conn(0x558f1a744310 msgr2=0x558f1a781690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:20.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.753+0000 7fdb70706740 1 -- 172.21.15.1:0/1544377435 shutdown_connections 2024-09-11T22:03:20.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.754+0000 7fdb70706740 1 -- 172.21.15.1:0/1544377435 wait complete. 2024-09-11T22:03:20.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.754+0000 7fdb70706740 1 -- start start 2024-09-11T22:03:20.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.754+0000 7fdb70706740 1 Processor -- start 2024-09-11T22:03:20.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.755+0000 7fdb70706740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f1a77b120 0x558f1a797640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:20.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.755+0000 7fdb70706740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f1a77bb80 0x558f1a776fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:20.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.755+0000 7fdb70706740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558f1a77c840 0x558f1a777600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:20.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.755+0000 7fdb67fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f1a77b120 0x558f1a797640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:20.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.755+0000 7fdb67fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f1a77b120 0x558f1a797640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:42182/0 (socket says 172.21.15.1:42182) 2024-09-11T22:03:20.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.755+0000 7fdb70706740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5400fef0 con 0x558f1a77c840 2024-09-11T22:03:20.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.755+0000 7fdb70706740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5400f760 con 0x558f1a77bb80 2024-09-11T22:03:20.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.755+0000 7fdb70706740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5400c8b0 con 0x558f1a77b120 2024-09-11T22:03:20.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.755+0000 7fdb70706740 10 client.0 _ll_register_callbacks cb 0x7fdb60011ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:03:20.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.755+0000 7fdb67fff640 1 -- 172.21.15.1:0/3453816565 learned_addr learned my addr 172.21.15.1:0/3453816565 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:03:20.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.756+0000 7fdb6c8ca640 1 --2- 172.21.15.1:0/3453816565 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558f1a77c840 0x558f1a777600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:20.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.756+0000 7fdb677fe640 1 --2- 172.21.15.1:0/3453816565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f1a77bb80 0x558f1a776fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:20.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.756+0000 7fdb6c8ca640 1 -- 172.21.15.1:0/3453816565 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f1a77b120 msgr2=0x558f1a797640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:20.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.756+0000 7fdb6c8ca640 1 --2- 172.21.15.1:0/3453816565 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f1a77b120 0x558f1a797640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:20.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.756+0000 7fdb6c8ca640 1 -- 172.21.15.1:0/3453816565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f1a77bb80 msgr2=0x558f1a776fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:20.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.756+0000 7fdb6c8ca640 1 --2- 172.21.15.1:0/3453816565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f1a77bb80 0x558f1a776fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:20.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.756+0000 7fdb6c8ca640 1 -- 172.21.15.1:0/3453816565 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f1a808480 con 0x558f1a77c840 2024-09-11T22:03:20.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.756+0000 7fdb67fff640 1 --2- 172.21.15.1:0/3453816565 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f1a77b120 0x558f1a797640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:03:20.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.756+0000 7fdb6c8ca640 1 --2- 172.21.15.1:0/3453816565 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558f1a77c840 0x558f1a777600 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fdb60009e80 tx=0x7fdb60003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93199baea013ae5f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:20.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.756+0000 7fdb477fe640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:03:20.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.757+0000 7fdb477fe640 1 -- 172.21.15.1:0/3453816565 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb60004a40 con 0x558f1a77c840 2024-09-11T22:03:20.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.757+0000 7fdb477fe640 1 -- 172.21.15.1:0/3453816565 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7fdb60004be0 con 0x558f1a77c840 2024-09-11T22:03:20.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.757+0000 7fdb70706740 1 -- 172.21.15.1:0/3453816565 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb6001c660 con 0x558f1a77c840 2024-09-11T22:03:20.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.757+0000 7fdb477fe640 1 -- 172.21.15.1:0/3453816565 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fdb60004de0 con 0x558f1a77c840 2024-09-11T22:03:20.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.757+0000 7fdb70706740 10 client.16695 fetch_fsmap learned FSMap version 38 2024-09-11T22:03:20.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.757+0000 7fdb70706740 1 -- 172.21.15.1:0/3453816565 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x558f1a8095d0 con 0x558f1a77c840 2024-09-11T22:03:20.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.757+0000 7fdb477fe640 1 -- 172.21.15.1:0/3453816565 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb6001c660 con 0x558f1a77c840 2024-09-11T22:03:20.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.758+0000 7fdb477fe640 1 -- 172.21.15.1:0/3453816565 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 38) ==== 8414+0+0 (secure 0 0 0) 0x7fdb60004510 con 0x558f1a77c840 2024-09-11T22:03:20.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.758+0000 7fdb70706740 10 client.16695 fetch_fsmap finished waiting for FSMap version 38 2024-09-11T22:03:20.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.758+0000 7fdb70706740 10 client.16695 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-11T22:03:20.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.758+0000 7fdb70706740 20 client.16695 populate_metadata read hostname 'smithi001' 2024-09-11T22:03:20.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.758+0000 7fdb70706740 1 --2- 172.21.15.1:0/3453816565 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x558f1a79e740 0x558f1a80bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:20.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.758+0000 7fdb70706740 4 client.16695 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:03:20.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.758+0000 7fdb70706740 1 -- 172.21.15.1:0/3453816565 --> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558f1a6e8d70 con 0x558f1a79e740 2024-09-11T22:03:20.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.758+0000 7fdb67fff640 1 --2- 172.21.15.1:0/3453816565 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x558f1a79e740 0x558f1a80bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:20.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.759+0000 7fdb67fff640 1 --2- 172.21.15.1:0/3453816565 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x558f1a79e740 0x558f1a80bc90 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e75af077695a687e server_cookie=3c2cca88d09767a0 in_seq=0 out_seq=0 2024-09-11T22:03:20.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.759+0000 7fdb477fe640 10 client.16695 ms_handle_connect on v2:172.21.15.161:6836/1799318656 2024-09-11T22:03:20.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.759+0000 7fdb477fe640 1 -- 172.21.15.1:0/3453816565 <== mds.3 v2:172.21.15.161:6836/1799318656 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558f1a6e8d70 con 0x558f1a79e740 2024-09-11T22:03:20.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.759+0000 7fdb477fe640 10 client.16695 handle_command_reply: tid=0 2024-09-11T22:03:20.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.759+0000 7fdb70706740 1 client.16695 shutdown 2024-09-11T22:03:20.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.760+0000 7fdb70706740 1 -- 172.21.15.1:0/3453816565 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558f1a77c840 msgr2=0x558f1a777600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:20.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.760+0000 7fdb70706740 1 --2- 172.21.15.1:0/3453816565 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558f1a77c840 0x558f1a777600 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fdb60009e80 tx=0x7fdb60003ba0 comp rx=0 tx=0).stop 2024-09-11T22:03:20.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.760+0000 7fdb70706740 1 -- 172.21.15.1:0/3453816565 shutdown_connections 2024-09-11T22:03:20.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.760+0000 7fdb70706740 1 --2- 172.21.15.1:0/3453816565 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x558f1a79e740 0x558f1a80bc90 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:20.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.760+0000 7fdb70706740 1 --2- 172.21.15.1:0/3453816565 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558f1a77c840 0x558f1a777600 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:20.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.760+0000 7fdb70706740 1 --2- 172.21.15.1:0/3453816565 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f1a77bb80 0x558f1a776fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:20.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.760+0000 7fdb477fe640 0 client.16695 ms_handle_reset on v2:172.21.15.161:6836/1799318656 2024-09-11T22:03:20.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.760+0000 7fdb70706740 1 --2- 172.21.15.1:0/3453816565 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f1a77b120 0x558f1a797640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:20.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.760+0000 7fdb70706740 1 -- 172.21.15.1:0/3453816565 >> 172.21.15.1:0/3453816565 conn(0x558f1a744310 msgr2=0x558f1a780b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:20.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.760+0000 7fdb70706740 1 -- 172.21.15.1:0/3453816565 shutdown_connections 2024-09-11T22:03:20.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.760+0000 7fdb70706740 1 -- 172.21.15.1:0/3453816565 wait complete. 2024-09-11T22:03:20.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.760+0000 7fdb70706740 20 client.16695 trim_cache size 0 max 16384 2024-09-11T22:03:20.764 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-11T22:03:20.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.761+0000 7fdb6e32d640 1 -- 172.21.15.1:0/4066299578 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fdb500a2cf0 msgr2=0x7fdb500a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:20.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.761+0000 7fdb6e32d640 1 --2- 172.21.15.1:0/4066299578 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fdb500a2cf0 0x7fdb500a51b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdb60009fd0 tx=0x7fdb60009500 comp rx=0 tx=0).stop 2024-09-11T22:03:20.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.761+0000 7fdb6e32d640 1 -- 172.21.15.1:0/4066299578 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdb68102570 msgr2=0x7fdb681989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:20.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.761+0000 7fdb6e32d640 1 --2- 172.21.15.1:0/4066299578 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdb68102570 0x7fdb681989f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c006010 tx=0x7fdb5c003c10 comp rx=0 tx=0).stop 2024-09-11T22:03:20.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.761+0000 7fdb6e32d640 1 -- 172.21.15.1:0/4066299578 shutdown_connections 2024-09-11T22:03:20.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.761+0000 7fdb6e32d640 1 --2- 172.21.15.1:0/4066299578 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fdb500a2cf0 0x7fdb500a51b0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:20.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.761+0000 7fdb6e32d640 1 --2- 172.21.15.1:0/4066299578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb68102ef0 0x7fdb6819cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:20.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.761+0000 7fdb6e32d640 1 --2- 172.21.15.1:0/4066299578 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdb68102570 0x7fdb681989f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:20.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.761+0000 7fdb6e32d640 1 --2- 172.21.15.1:0/4066299578 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb68100e20 0x7fdb681984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:20.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.761+0000 7fdb6e32d640 1 -- 172.21.15.1:0/4066299578 >> 172.21.15.1:0/4066299578 conn(0x7fdb680fc7d0 msgr2=0x7fdb6806a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:20.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.761+0000 7fdb6e32d640 1 -- 172.21.15.1:0/4066299578 shutdown_connections 2024-09-11T22:03:20.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:20.761+0000 7fdb6e32d640 1 -- 172.21.15.1:0/4066299578 wait complete. 2024-09-11T22:03:21.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:21 smithi053 ceph-mon[34908]: pgmap v188: 97 pgs: 97 active+clean; 205 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.3 MiB/s wr, 670 op/s 2024-09-11T22:03:21.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:21 smithi053 ceph-mon[34908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:03:21.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:21 smithi161 ceph-mon[35615]: pgmap v188: 97 pgs: 97 active+clean; 205 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.3 MiB/s wr, 670 op/s 2024-09-11T22:03:21.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:21 smithi161 ceph-mon[35615]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:03:21.383 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:21 smithi001 ceph-mon[30935]: pgmap v188: 97 pgs: 97 active+clean; 205 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.3 MiB/s wr, 670 op/s 2024-09-11T22:03:21.383 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:21 smithi001 ceph-mon[30935]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:03:21.413 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T22:03:21.749 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:03:23.133 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:23 smithi001 ceph-mon[30935]: pgmap v189: 97 pgs: 97 active+clean; 205 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.2 MiB/s wr, 585 op/s 2024-09-11T22:03:23.134 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:23 smithi001 ceph-mon[30935]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:03:23.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.126+0000 7efe6f850640 1 -- 172.21.15.1:0/1084863231 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe68106f10 msgr2=0x7efe6810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:23.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.126+0000 7efe6f850640 1 --2- 172.21.15.1:0/1084863231 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe68106f10 0x7efe6810ea90 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7efe60009ca0 tx=0x7efe6002a8c0 comp rx=0 tx=0).stop 2024-09-11T22:03:23.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.126+0000 7efe6f850640 1 -- 172.21.15.1:0/1084863231 shutdown_connections 2024-09-11T22:03:23.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.126+0000 7efe6f850640 1 --2- 172.21.15.1:0/1084863231 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe68106f10 0x7efe6810ea90 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:23.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.126+0000 7efe6f850640 1 --2- 172.21.15.1:0/1084863231 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe68102790 0x7efe681067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:23.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.126+0000 7efe6f850640 1 --2- 172.21.15.1:0/1084863231 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe68101de0 0x7efe681021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:23.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.126+0000 7efe6f850640 1 -- 172.21.15.1:0/1084863231 >> 172.21.15.1:0/1084863231 conn(0x7efe680fd8e0 msgr2=0x7efe680ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:23.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.126+0000 7efe6f850640 1 -- 172.21.15.1:0/1084863231 shutdown_connections 2024-09-11T22:03:23.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.126+0000 7efe6f850640 1 -- 172.21.15.1:0/1084863231 wait complete. 2024-09-11T22:03:23.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.126+0000 7efe6f850640 1 Processor -- start 2024-09-11T22:03:23.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.126+0000 7efe6f850640 1 -- start start 2024-09-11T22:03:23.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.127+0000 7efe6f850640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe68101de0 0x7efe681983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:23.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.127+0000 7efe6f850640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe68102790 0x7efe68198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:23.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.127+0000 7efe6f850640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe68106f10 0x7efe6819ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:23.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.127+0000 7efe6f850640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7efe68111f70 con 0x7efe68101de0 2024-09-11T22:03:23.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.127+0000 7efe6f850640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7efe68111df0 con 0x7efe68102790 2024-09-11T22:03:23.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.127+0000 7efe6f850640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7efe681120f0 con 0x7efe68106f10 2024-09-11T22:03:23.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.127+0000 7efe6d5ec640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe68101de0 0x7efe681983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:23.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.127+0000 7efe6d5ec640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe68101de0 0x7efe681983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52172/0 (socket says 172.21.15.1:52172) 2024-09-11T22:03:23.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.127+0000 7efe6d5ec640 1 -- 172.21.15.1:0/1328917102 learned_addr learned my addr 172.21.15.1:0/1328917102 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:03:23.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.127+0000 7efe6dded640 1 --2- 172.21.15.1:0/1328917102 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe68106f10 0x7efe6819ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:23.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.127+0000 7efe6cdeb640 1 --2- 172.21.15.1:0/1328917102 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe68102790 0x7efe68198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:23.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.127+0000 7efe6d5ec640 1 -- 172.21.15.1:0/1328917102 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe68106f10 msgr2=0x7efe6819ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:23.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.127+0000 7efe6d5ec640 1 --2- 172.21.15.1:0/1328917102 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe68106f10 0x7efe6819ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:23.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.127+0000 7efe6d5ec640 1 -- 172.21.15.1:0/1328917102 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe68102790 msgr2=0x7efe68198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:23.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.127+0000 7efe6d5ec640 1 --2- 172.21.15.1:0/1328917102 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe68102790 0x7efe68198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:23.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.127+0000 7efe6d5ec640 1 -- 172.21.15.1:0/1328917102 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe6819d3a0 con 0x7efe68101de0 2024-09-11T22:03:23.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.127+0000 7efe6dded640 1 --2- 172.21.15.1:0/1328917102 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe68106f10 0x7efe6819ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:03:23.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.127+0000 7efe6d5ec640 1 --2- 172.21.15.1:0/1328917102 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe68101de0 0x7efe681983f0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7efe6400e9f0 tx=0x7efe6400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:23.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.127+0000 7efe5e7fc640 1 -- 172.21.15.1:0/1328917102 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6400cd80 con 0x7efe68101de0 2024-09-11T22:03:23.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.128+0000 7efe6f850640 1 -- 172.21.15.1:0/1328917102 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe6819d690 con 0x7efe68101de0 2024-09-11T22:03:23.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.128+0000 7efe5e7fc640 1 -- 172.21.15.1:0/1328917102 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe64010910 con 0x7efe68101de0 2024-09-11T22:03:23.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.128+0000 7efe5e7fc640 1 -- 172.21.15.1:0/1328917102 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe640188f0 con 0x7efe68101de0 2024-09-11T22:03:23.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.128+0000 7efe6f850640 1 -- 172.21.15.1:0/1328917102 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe681a4f70 con 0x7efe68101de0 2024-09-11T22:03:23.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.131+0000 7efe71c29740 1 Processor -- start 2024-09-11T22:03:23.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.131+0000 7efe71c29740 1 -- start start 2024-09-11T22:03:23.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.131+0000 7efe5e7fc640 1 -- 172.21.15.1:0/1328917102 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7efe64010ac0 con 0x7efe68101de0 2024-09-11T22:03:23.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.132+0000 7efe71c29740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55653c0d8840 0x55653c0d8c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:23.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.132+0000 7efe71c29740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55653c0d7120 0x55653c0d7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:23.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.132+0000 7efe71c29740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55653c0d7b80 0x55653c0ecf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:23.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.132+0000 7efe71c29740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55653c03d2f0 con 0x55653c0d8840 2024-09-11T22:03:23.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.132+0000 7efe71c29740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55653bfe4cf0 con 0x55653c0d7b80 2024-09-11T22:03:23.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.132+0000 7efe71c29740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55653c0392b0 con 0x55653c0d7120 2024-09-11T22:03:23.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.132+0000 7efe6cdeb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55653c0d7b80 0x55653c0ecf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:23.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.132+0000 7efe6cdeb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55653c0d7b80 0x55653c0ecf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:42674/0 (socket says 172.21.15.1:42674) 2024-09-11T22:03:23.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.132+0000 7efe6cdeb640 1 -- 172.21.15.1:0/2988850271 learned_addr learned my addr 172.21.15.1:0/2988850271 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:03:23.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.132+0000 7efe6dded640 1 --2- 172.21.15.1:0/2988850271 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55653c0d8840 0x55653c0d8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:23.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.132+0000 7efe6d5ec640 1 --2- 172.21.15.1:0/2988850271 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55653c0d7120 0x55653c0d7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:23.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.132+0000 7efe6dded640 1 -- 172.21.15.1:0/2988850271 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55653c0d7120 msgr2=0x55653c0d7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:23.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.132+0000 7efe6dded640 1 --2- 172.21.15.1:0/2988850271 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55653c0d7120 0x55653c0d7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:23.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.132+0000 7efe6dded640 1 -- 172.21.15.1:0/2988850271 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55653c0d7b80 msgr2=0x55653c0ecf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:23.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.132+0000 7efe6dded640 1 --2- 172.21.15.1:0/2988850271 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55653c0d7b80 0x55653c0ecf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:23.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.133+0000 7efe6dded640 1 -- 172.21.15.1:0/2988850271 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55653bfc93a0 con 0x55653c0d8840 2024-09-11T22:03:23.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.133+0000 7efe6cdeb640 1 --2- 172.21.15.1:0/2988850271 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55653c0d7b80 0x55653c0ecf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:03:23.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.133+0000 7efe6dded640 1 --2- 172.21.15.1:0/2988850271 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55653c0d8840 0x55653c0d8c20 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7efe60006010 tx=0x7efe60009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a42775e1a4e4698 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:23.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.133+0000 7efe5e7fc640 1 --2- 172.21.15.1:0/1328917102 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7efe540a2cf0 0x7efe540a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:23.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.133+0000 7efe537fe640 1 -- 172.21.15.1:0/2988850271 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe60003b70 con 0x55653c0d8840 2024-09-11T22:03:23.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.133+0000 7efe537fe640 1 -- 172.21.15.1:0/2988850271 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe60036040 con 0x55653c0d8840 2024-09-11T22:03:23.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.133+0000 7efe537fe640 1 -- 172.21.15.1:0/2988850271 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe600046a0 con 0x55653c0d8840 2024-09-11T22:03:23.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.133+0000 7efe5e7fc640 1 -- 172.21.15.1:0/1328917102 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7efe640cf8a0 con 0x7efe68101de0 2024-09-11T22:03:23.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.133+0000 7efe6d5ec640 1 --2- 172.21.15.1:0/1328917102 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7efe540a2cf0 0x7efe540a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:23.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.134+0000 7efe71c29740 1 -- 172.21.15.1:0/2988850271 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55653c0d8840 msgr2=0x55653c0d8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:23.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.134+0000 7efe6d5ec640 1 --2- 172.21.15.1:0/1328917102 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7efe540a2cf0 0x7efe540a51b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x55653c0d4ac0 tx=0x7efe640d5530 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:23.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.134+0000 7efe71c29740 1 --2- 172.21.15.1:0/2988850271 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55653c0d8840 0x55653c0d8c20 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7efe60006010 tx=0x7efe60009800 comp rx=0 tx=0).stop 2024-09-11T22:03:23.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.134+0000 7efe71c29740 1 -- 172.21.15.1:0/2988850271 shutdown_connections 2024-09-11T22:03:23.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.134+0000 7efe71c29740 1 --2- 172.21.15.1:0/2988850271 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55653c0d7b80 0x55653c0ecf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:23.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.134+0000 7efe71c29740 1 --2- 172.21.15.1:0/2988850271 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55653c0d7120 0x55653c0d7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:23.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.134+0000 7efe71c29740 1 --2- 172.21.15.1:0/2988850271 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55653c0d8840 0x55653c0d8c20 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:23.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.134+0000 7efe71c29740 1 -- 172.21.15.1:0/2988850271 >> 172.21.15.1:0/2988850271 conn(0x55653c0a0310 msgr2=0x55653c0d3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:23.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.134+0000 7efe71c29740 1 -- 172.21.15.1:0/2988850271 shutdown_connections 2024-09-11T22:03:23.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.134+0000 7efe71c29740 1 -- 172.21.15.1:0/2988850271 wait complete. 2024-09-11T22:03:23.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.134+0000 7efe71c29740 1 -- start start 2024-09-11T22:03:23.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.134+0000 7efe71c29740 1 Processor -- start 2024-09-11T22:03:23.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.135+0000 7efe71c29740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55653c0d7120 0x55653c0e6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:23.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.135+0000 7efe71c29740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55653c0d7b80 0x55653c0e7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:23.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.135+0000 7efe71c29740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55653c0d8840 0x55653c0ec930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:23.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.135+0000 7efe71c29740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55653c0de940 con 0x55653c0d8840 2024-09-11T22:03:23.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.135+0000 7efe71c29740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55653bff8060 con 0x55653c0d7b80 2024-09-11T22:03:23.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.135+0000 7efe71c29740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55653c0deac0 con 0x55653c0d7120 2024-09-11T22:03:23.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.135+0000 7efe71c29740 10 client.0 _ll_register_callbacks cb 0x55653bd4b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:03:23.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.135+0000 7efe6dded640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55653c0d8840 0x55653c0ec930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:23.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.135+0000 7efe6cdeb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55653c0d7b80 0x55653c0e7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:23.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.135+0000 7efe6dded640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55653c0d8840 0x55653c0ec930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52184/0 (socket says 172.21.15.1:52184) 2024-09-11T22:03:23.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.135+0000 7efe6dded640 1 -- 172.21.15.1:0/1503257528 learned_addr learned my addr 172.21.15.1:0/1503257528 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:03:23.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.135+0000 7efe6dded640 1 --2- 172.21.15.1:0/1503257528 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55653c0d7120 0x55653c0e6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:23.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.135+0000 7efe6cdeb640 1 -- 172.21.15.1:0/1503257528 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55653c0d7120 msgr2=0x55653c0e6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:23.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.135+0000 7efe6cdeb640 1 --2- 172.21.15.1:0/1503257528 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55653c0d7120 0x55653c0e6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:23.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.135+0000 7efe6cdeb640 1 -- 172.21.15.1:0/1503257528 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55653c0d8840 msgr2=0x55653c0ec930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:23.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.135+0000 7efe6cdeb640 1 --2- 172.21.15.1:0/1503257528 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55653c0d8840 0x55653c0ec930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:23.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.135+0000 7efe6cdeb640 1 -- 172.21.15.1:0/1503257528 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55653bdae3c0 con 0x55653c0d7b80 2024-09-11T22:03:23.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.135+0000 7efe6dded640 1 --2- 172.21.15.1:0/1503257528 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55653c0d8840 0x55653c0ec930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:03:23.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.136+0000 7efe6cdeb640 1 --2- 172.21.15.1:0/1503257528 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55653c0d7b80 0x55653c0e7140 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7efe58005fd0 tx=0x7efe5800bf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdf161b76cc9f591 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:23.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.136+0000 7efe50ff9640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-11T22:03:23.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.136+0000 7efe50ff9640 1 -- 172.21.15.1:0/1503257528 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe58004020 con 0x55653c0d7b80 2024-09-11T22:03:23.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.136+0000 7efe50ff9640 1 -- 172.21.15.1:0/1503257528 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7efe580041c0 con 0x55653c0d7b80 2024-09-11T22:03:23.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.136+0000 7efe71c29740 1 -- 172.21.15.1:0/1503257528 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe60036040 con 0x55653c0d7b80 2024-09-11T22:03:23.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.136+0000 7efe50ff9640 1 -- 172.21.15.1:0/1503257528 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7efe580043c0 con 0x55653c0d7b80 2024-09-11T22:03:23.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.136+0000 7efe71c29740 10 client.26243 fetch_fsmap learned FSMap version 39 2024-09-11T22:03:23.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.136+0000 7efe71c29740 1 -- 172.21.15.1:0/1503257528 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55653c101ba0 con 0x55653c0d7b80 2024-09-11T22:03:23.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.137+0000 7efe50ff9640 1 -- 172.21.15.1:0/1503257528 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe60036040 con 0x55653c0d7b80 2024-09-11T22:03:23.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.137+0000 7efe50ff9640 1 -- 172.21.15.1:0/1503257528 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 39) ==== 8418+0+0 (secure 0 0 0) 0x7efe58010700 con 0x55653c0d7b80 2024-09-11T22:03:23.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.137+0000 7efe71c29740 10 client.26243 fetch_fsmap finished waiting for FSMap version 39 2024-09-11T22:03:23.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.137+0000 7efe71c29740 10 client.26243 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-11T22:03:23.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.137+0000 7efe71c29740 20 client.26243 populate_metadata read hostname 'smithi001' 2024-09-11T22:03:23.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.137+0000 7efe71c29740 1 --2- 172.21.15.1:0/1503257528 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x55653c103000 0x55653c1053f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:23.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.137+0000 7efe71c29740 4 client.26243 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:03:23.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.137+0000 7efe71c29740 1 -- 172.21.15.1:0/1503257528 --> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55653c04a860 con 0x55653c103000 2024-09-11T22:03:23.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.137+0000 7efe6dded640 1 --2- 172.21.15.1:0/1503257528 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x55653c103000 0x55653c1053f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:23.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.138+0000 7efe6dded640 1 --2- 172.21.15.1:0/1503257528 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x55653c103000 0x55653c1053f0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3a1ff7fea925eba2 server_cookie=63f222e64bdb773f in_seq=0 out_seq=0 2024-09-11T22:03:23.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.138+0000 7efe50ff9640 10 client.26243 ms_handle_connect on v2:172.21.15.53:6836/3457131830 2024-09-11T22:03:23.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.139+0000 7efe50ff9640 1 -- 172.21.15.1:0/1503257528 <== mds.2 v2:172.21.15.53:6836/3457131830 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55653c04a860 con 0x55653c103000 2024-09-11T22:03:23.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.139+0000 7efe50ff9640 10 client.26243 handle_command_reply: tid=0 2024-09-11T22:03:23.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.139+0000 7efe71c29740 1 client.26243 shutdown 2024-09-11T22:03:23.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.139+0000 7efe71c29740 1 -- 172.21.15.1:0/1503257528 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55653c0d7b80 msgr2=0x55653c0e7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:23.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.139+0000 7efe71c29740 1 --2- 172.21.15.1:0/1503257528 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55653c0d7b80 0x55653c0e7140 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7efe58005fd0 tx=0x7efe5800bf90 comp rx=0 tx=0).stop 2024-09-11T22:03:23.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.139+0000 7efe71c29740 1 -- 172.21.15.1:0/1503257528 shutdown_connections 2024-09-11T22:03:23.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.139+0000 7efe71c29740 1 --2- 172.21.15.1:0/1503257528 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x55653c103000 0x55653c1053f0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:23.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.139+0000 7efe71c29740 1 --2- 172.21.15.1:0/1503257528 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55653c0d8840 0x55653c0ec930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:23.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.139+0000 7efe71c29740 1 --2- 172.21.15.1:0/1503257528 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55653c0d7b80 0x55653c0e7140 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:23.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.139+0000 7efe71c29740 1 --2- 172.21.15.1:0/1503257528 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55653c0d7120 0x55653c0e6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:23.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.139+0000 7efe71c29740 1 -- 172.21.15.1:0/1503257528 >> 172.21.15.1:0/1503257528 conn(0x55653c0a0310 msgr2=0x55653c0d99a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:23.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.139+0000 7efe50ff9640 0 client.26243 ms_handle_reset on v2:172.21.15.53:6836/3457131830 2024-09-11T22:03:23.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.140+0000 7efe71c29740 1 -- 172.21.15.1:0/1503257528 shutdown_connections 2024-09-11T22:03:23.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.140+0000 7efe71c29740 1 -- 172.21.15.1:0/1503257528 wait complete. 2024-09-11T22:03:23.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.140+0000 7efe71c29740 20 client.26243 trim_cache size 0 max 16384 2024-09-11T22:03:23.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.154+0000 7efe71c29740 1 Processor -- start 2024-09-11T22:03:23.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.154+0000 7efe71c29740 1 -- start start 2024-09-11T22:03:23.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.154+0000 7efe71c29740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55653c0d7120 0x55653c0e0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:23.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.154+0000 7efe71c29740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55653c0d7b80 0x55653c10acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:23.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.154+0000 7efe71c29740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55653c0d8840 0x55653c10b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:23.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.154+0000 7efe71c29740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7efe681ab700 con 0x55653c0d7b80 2024-09-11T22:03:23.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.154+0000 7efe71c29740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7efe681ad4b0 con 0x55653c0d8840 2024-09-11T22:03:23.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.154+0000 7efe71c29740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7efe681abe30 con 0x55653c0d7120 2024-09-11T22:03:23.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.154+0000 7efe6cdeb640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55653c0d7b80 0x55653c10acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:23.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.154+0000 7efe6dded640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55653c0d7120 0x55653c0e0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:23.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.154+0000 7efe6cdeb640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55653c0d7b80 0x55653c10acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52188/0 (socket says 172.21.15.1:52188) 2024-09-11T22:03:23.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.154+0000 7efe6cdeb640 1 -- 172.21.15.1:0/1109607995 learned_addr learned my addr 172.21.15.1:0/1109607995 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:03:23.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.154+0000 7efe6dded640 1 --2- 172.21.15.1:0/1109607995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55653c0d8840 0x55653c10b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:23.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.155+0000 7efe6dded640 1 -- 172.21.15.1:0/1109607995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55653c0d8840 msgr2=0x55653c10b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:23.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.155+0000 7efe6dded640 1 --2- 172.21.15.1:0/1109607995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55653c0d8840 0x55653c10b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:23.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.155+0000 7efe6dded640 1 -- 172.21.15.1:0/1109607995 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55653c0d7b80 msgr2=0x55653c10acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:23.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.155+0000 7efe6dded640 1 --2- 172.21.15.1:0/1109607995 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55653c0d7b80 0x55653c10acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:23.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.155+0000 7efe6dded640 1 -- 172.21.15.1:0/1109607995 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55653c0ecf10 con 0x55653c0d7120 2024-09-11T22:03:23.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.155+0000 7efe6cdeb640 1 --2- 172.21.15.1:0/1109607995 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55653c0d7b80 0x55653c10acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:03:23.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.155+0000 7efe6dded640 1 --2- 172.21.15.1:0/1109607995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55653c0d7120 0x55653c0e0f00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7efe60009600 tx=0x7efe6002a8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6485bfb67fc57d9c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:23.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.155+0000 7efe52ffd640 1 -- 172.21.15.1:0/1109607995 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe60009870 con 0x55653c0d7120 2024-09-11T22:03:23.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.155+0000 7efe52ffd640 1 -- 172.21.15.1:0/1109607995 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe60036030 con 0x55653c0d7120 2024-09-11T22:03:23.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.155+0000 7efe52ffd640 1 -- 172.21.15.1:0/1109607995 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6002f870 con 0x55653c0d7120 2024-09-11T22:03:23.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.156+0000 7efe71c29740 1 -- 172.21.15.1:0/1109607995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55653c0d7120 msgr2=0x55653c0e0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:23.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.156+0000 7efe71c29740 1 --2- 172.21.15.1:0/1109607995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55653c0d7120 0x55653c0e0f00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7efe60009600 tx=0x7efe6002a8c0 comp rx=0 tx=0).stop 2024-09-11T22:03:23.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.156+0000 7efe71c29740 1 -- 172.21.15.1:0/1109607995 shutdown_connections 2024-09-11T22:03:23.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.156+0000 7efe71c29740 1 --2- 172.21.15.1:0/1109607995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55653c0d8840 0x55653c10b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:23.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.156+0000 7efe71c29740 1 --2- 172.21.15.1:0/1109607995 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55653c0d7b80 0x55653c10acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:23.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.156+0000 7efe71c29740 1 --2- 172.21.15.1:0/1109607995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55653c0d7120 0x55653c0e0f00 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:23.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.156+0000 7efe71c29740 1 -- 172.21.15.1:0/1109607995 >> 172.21.15.1:0/1109607995 conn(0x55653c0a0310 msgr2=0x55653c0e07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:23.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.156+0000 7efe71c29740 1 -- 172.21.15.1:0/1109607995 shutdown_connections 2024-09-11T22:03:23.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.156+0000 7efe71c29740 1 -- 172.21.15.1:0/1109607995 wait complete. 2024-09-11T22:03:23.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.157+0000 7efe71c29740 1 -- start start 2024-09-11T22:03:23.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.157+0000 7efe71c29740 1 Processor -- start 2024-09-11T22:03:23.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.157+0000 7efe71c29740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55653c0d7120 0x55653c116e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:23.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.157+0000 7efe71c29740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55653c0d7b80 0x55653c1141b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:23.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.158+0000 7efe6dded640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55653c0d7120 0x55653c116e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:23.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.158+0000 7efe6cdeb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55653c0d7b80 0x55653c1141b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:23.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.158+0000 7efe6cdeb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55653c0d7b80 0x55653c1141b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:42698/0 (socket says 172.21.15.1:42698) 2024-09-11T22:03:23.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.158+0000 7efe6dded640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55653c0d7120 0x55653c116e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:42238/0 (socket says 172.21.15.1:42238) 2024-09-11T22:03:23.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.158+0000 7efe71c29740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55653c0d8840 0x55653c1146f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:23.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.158+0000 7efe6cdeb640 1 -- 172.21.15.1:0/34588666 learned_addr learned my addr 172.21.15.1:0/34588666 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:03:23.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.158+0000 7efe71c29740 1 -- 172.21.15.1:0/34588666 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7efe681acd20 con 0x55653c0d8840 2024-09-11T22:03:23.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.158+0000 7efe71c29740 1 -- 172.21.15.1:0/34588666 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7efe681ac590 con 0x55653c0d7b80 2024-09-11T22:03:23.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.158+0000 7efe71c29740 1 -- 172.21.15.1:0/34588666 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7efe681a96e0 con 0x55653c0d7120 2024-09-11T22:03:23.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.158+0000 7efe71c29740 10 client.0 _ll_register_callbacks cb 0x7efe5800bb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:03:23.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.158+0000 7efe6cdeb640 1 -- 172.21.15.1:0/34588666 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55653c0d7120 msgr2=0x55653c116e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:23.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.158+0000 7efe6cdeb640 1 --2- 172.21.15.1:0/34588666 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55653c0d7120 0x55653c116e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:23.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.158+0000 7efe6cdeb640 1 -- 172.21.15.1:0/34588666 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55653c0d8840 msgr2=0x55653c1146f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:23.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.158+0000 7efe6cdeb640 1 --2- 172.21.15.1:0/34588666 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55653c0d8840 0x55653c1146f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:23.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.158+0000 7efe6cdeb640 1 -- 172.21.15.1:0/34588666 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55653c114c30 con 0x55653c0d7b80 2024-09-11T22:03:23.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.158+0000 7efe6dded640 1 --2- 172.21.15.1:0/34588666 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55653c0d7120 0x55653c116e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:03:23.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.158+0000 7efe6cdeb640 1 --2- 172.21.15.1:0/34588666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55653c0d7b80 0x55653c1141b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7efe58010d30 tx=0x7efe58004870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a1f8c9a92140d62 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:23.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.158+0000 7efe50ff9640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-11T22:03:23.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.159+0000 7efe50ff9640 1 -- 172.21.15.1:0/34588666 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe58031020 con 0x55653c0d7b80 2024-09-11T22:03:23.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.159+0000 7efe50ff9640 1 -- 172.21.15.1:0/34588666 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7efe5800b410 con 0x55653c0d7b80 2024-09-11T22:03:23.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.159+0000 7efe71c29740 1 -- 172.21.15.1:0/34588666 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe60036030 con 0x55653c0d7b80 2024-09-11T22:03:23.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.159+0000 7efe50ff9640 1 -- 172.21.15.1:0/34588666 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7efe58028040 con 0x55653c0d7b80 2024-09-11T22:03:23.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.159+0000 7efe71c29740 10 client.26249 fetch_fsmap learned FSMap version 39 2024-09-11T22:03:23.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.159+0000 7efe71c29740 1 -- 172.21.15.1:0/34588666 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55653c16d820 con 0x55653c0d7b80 2024-09-11T22:03:23.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.159+0000 7efe50ff9640 1 -- 172.21.15.1:0/34588666 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe60036030 con 0x55653c0d7b80 2024-09-11T22:03:23.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.160+0000 7efe50ff9640 1 -- 172.21.15.1:0/34588666 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 39) ==== 8418+0+0 (secure 0 0 0) 0x7efe58024430 con 0x55653c0d7b80 2024-09-11T22:03:23.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.160+0000 7efe71c29740 10 client.26249 fetch_fsmap finished waiting for FSMap version 39 2024-09-11T22:03:23.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.160+0000 7efe71c29740 10 client.26249 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-11T22:03:23.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.160+0000 7efe71c29740 20 client.26249 populate_metadata read hostname 'smithi001' 2024-09-11T22:03:23.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.160+0000 7efe71c29740 1 --2- 172.21.15.1:0/34588666 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x55653c16e0b0 0x55653c1704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:23.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.160+0000 7efe71c29740 4 client.26249 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:03:23.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.160+0000 7efe71c29740 1 -- 172.21.15.1:0/34588666 --> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55653c044d70 con 0x55653c16e0b0 2024-09-11T22:03:23.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.160+0000 7efe6dded640 1 --2- 172.21.15.1:0/34588666 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x55653c16e0b0 0x55653c1704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:23.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.161+0000 7efe6dded640 1 --2- 172.21.15.1:0/34588666 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x55653c16e0b0 0x55653c1704d0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4b635e90241e598d server_cookie=3da9e21641cda08e in_seq=0 out_seq=0 2024-09-11T22:03:23.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.161+0000 7efe50ff9640 10 client.26249 ms_handle_connect on v2:172.21.15.53:6836/3457131830 2024-09-11T22:03:23.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.161+0000 7efe50ff9640 1 -- 172.21.15.1:0/34588666 <== mds.2 v2:172.21.15.53:6836/3457131830 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55653c044d70 con 0x55653c16e0b0 2024-09-11T22:03:23.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.161+0000 7efe50ff9640 10 client.26249 handle_command_reply: tid=0 2024-09-11T22:03:23.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.161+0000 7efe71c29740 1 client.26249 shutdown 2024-09-11T22:03:23.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.163+0000 7efe71c29740 1 -- 172.21.15.1:0/34588666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55653c0d7b80 msgr2=0x55653c1141b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:23.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.163+0000 7efe71c29740 1 --2- 172.21.15.1:0/34588666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55653c0d7b80 0x55653c1141b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7efe58010d30 tx=0x7efe58004870 comp rx=0 tx=0).stop 2024-09-11T22:03:23.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.163+0000 7efe71c29740 1 -- 172.21.15.1:0/34588666 shutdown_connections 2024-09-11T22:03:23.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.163+0000 7efe71c29740 1 --2- 172.21.15.1:0/34588666 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x55653c16e0b0 0x55653c1704d0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:23.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.163+0000 7efe71c29740 1 --2- 172.21.15.1:0/34588666 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55653c0d8840 0x55653c1146f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:23.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.163+0000 7efe71c29740 1 --2- 172.21.15.1:0/34588666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55653c0d7b80 0x55653c1141b0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:23.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.163+0000 7efe71c29740 1 --2- 172.21.15.1:0/34588666 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55653c0d7120 0x55653c116e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:23.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.163+0000 7efe50ff9640 0 client.26249 ms_handle_reset on v2:172.21.15.53:6836/3457131830 2024-09-11T22:03:23.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.163+0000 7efe71c29740 1 -- 172.21.15.1:0/34588666 >> 172.21.15.1:0/34588666 conn(0x55653c0a0310 msgr2=0x55653c0e1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:23.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.163+0000 7efe71c29740 1 -- 172.21.15.1:0/34588666 shutdown_connections 2024-09-11T22:03:23.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.163+0000 7efe71c29740 1 -- 172.21.15.1:0/34588666 wait complete. 2024-09-11T22:03:23.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.163+0000 7efe71c29740 20 client.26249 trim_cache size 0 max 16384 2024-09-11T22:03:23.178 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-11T22:03:23.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.163+0000 7efe6f850640 1 -- 172.21.15.1:0/1328917102 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7efe540a2cf0 msgr2=0x7efe540a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:23.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.163+0000 7efe6f850640 1 --2- 172.21.15.1:0/1328917102 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7efe540a2cf0 0x7efe540a51b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x55653c0d4ac0 tx=0x7efe640d5530 comp rx=0 tx=0).stop 2024-09-11T22:03:23.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.163+0000 7efe6f850640 1 -- 172.21.15.1:0/1328917102 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe68101de0 msgr2=0x7efe681983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:23.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.163+0000 7efe6f850640 1 --2- 172.21.15.1:0/1328917102 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe68101de0 0x7efe681983f0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7efe6400e9f0 tx=0x7efe6400eeb0 comp rx=0 tx=0).stop 2024-09-11T22:03:23.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.163+0000 7efe6f850640 1 -- 172.21.15.1:0/1328917102 shutdown_connections 2024-09-11T22:03:23.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.163+0000 7efe6f850640 1 --2- 172.21.15.1:0/1328917102 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7efe540a2cf0 0x7efe540a51b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:23.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.164+0000 7efe6f850640 1 --2- 172.21.15.1:0/1328917102 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efe68106f10 0x7efe6819ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:23.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.164+0000 7efe6f850640 1 --2- 172.21.15.1:0/1328917102 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efe68102790 0x7efe68198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:23.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.164+0000 7efe6f850640 1 --2- 172.21.15.1:0/1328917102 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe68101de0 0x7efe681983f0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:23.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.164+0000 7efe6f850640 1 -- 172.21.15.1:0/1328917102 >> 172.21.15.1:0/1328917102 conn(0x7efe680fd8e0 msgr2=0x7efe680febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:23.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.164+0000 7efe6f850640 1 -- 172.21.15.1:0/1328917102 shutdown_connections 2024-09-11T22:03:23.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:23.164+0000 7efe6f850640 1 -- 172.21.15.1:0/1328917102 wait complete. 2024-09-11T22:03:23.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:23 smithi053 ceph-mon[34908]: pgmap v189: 97 pgs: 97 active+clean; 205 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.2 MiB/s wr, 585 op/s 2024-09-11T22:03:23.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:23 smithi053 ceph-mon[34908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:03:23.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:23 smithi161 ceph-mon[35615]: pgmap v189: 97 pgs: 97 active+clean; 205 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.2 MiB/s wr, 585 op/s 2024-09-11T22:03:23.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:23 smithi161 ceph-mon[35615]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:03:24.729 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T22:03:25.142 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:03:25.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:25 smithi053 ceph-mon[34908]: pgmap v190: 97 pgs: 97 active+clean; 205 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.2 MiB/s wr, 585 op/s 2024-09-11T22:03:25.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:25 smithi053 ceph-mon[34908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:03:25.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:25 smithi161 ceph-mon[35615]: pgmap v190: 97 pgs: 97 active+clean; 205 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.2 MiB/s wr, 585 op/s 2024-09-11T22:03:25.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:25 smithi161 ceph-mon[35615]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:03:25.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:25 smithi001 ceph-mon[30935]: pgmap v190: 97 pgs: 97 active+clean; 205 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.2 MiB/s wr, 585 op/s 2024-09-11T22:03:25.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:25 smithi001 ceph-mon[30935]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:03:26.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.053+0000 7f894dbd8640 1 -- 172.21.15.1:0/1520296100 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8948100e20 msgr2=0x7f8948101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:26.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.053+0000 7f894dbd8640 1 --2- 172.21.15.1:0/1520296100 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8948100e20 0x7f8948101200 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8938009930 tx=0x7f893802a4b0 comp rx=0 tx=0).stop 2024-09-11T22:03:26.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.053+0000 7f894dbd8640 1 -- 172.21.15.1:0/1520296100 shutdown_connections 2024-09-11T22:03:26.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.053+0000 7f894dbd8640 1 --2- 172.21.15.1:0/1520296100 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8948102ef0 0x7f894810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:26.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.053+0000 7f894dbd8640 1 --2- 172.21.15.1:0/1520296100 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8948102570 0x7f89481029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:26.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.053+0000 7f894dbd8640 1 --2- 172.21.15.1:0/1520296100 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8948100e20 0x7f8948101200 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:26.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.053+0000 7f894dbd8640 1 -- 172.21.15.1:0/1520296100 >> 172.21.15.1:0/1520296100 conn(0x7f89480fc7d0 msgr2=0x7f89480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:26.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.053+0000 7f894dbd8640 1 -- 172.21.15.1:0/1520296100 shutdown_connections 2024-09-11T22:03:26.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.053+0000 7f894dbd8640 1 -- 172.21.15.1:0/1520296100 wait complete. 2024-09-11T22:03:26.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.053+0000 7f894dbd8640 1 Processor -- start 2024-09-11T22:03:26.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.054+0000 7f894dbd8640 1 -- start start 2024-09-11T22:03:26.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.054+0000 7f894dbd8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8948100e20 0x7f89480fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:26.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.054+0000 7f894dbd8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8948102570 0x7f89480fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:26.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.054+0000 7f894dbd8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8948102ef0 0x7f89480fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:26.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.054+0000 7f894dbd8640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8948110f80 con 0x7f8948102ef0 2024-09-11T22:03:26.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.054+0000 7f894dbd8640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8948110e00 con 0x7f8948100e20 2024-09-11T22:03:26.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.054+0000 7f894dbd8640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8948111100 con 0x7f8948102570 2024-09-11T22:03:26.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.054+0000 7f8947fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8948102ef0 0x7f89480fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:26.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.054+0000 7f8946ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8948102570 0x7f89480fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:26.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.054+0000 7f8947fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8948102ef0 0x7f89480fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52206/0 (socket says 172.21.15.1:52206) 2024-09-11T22:03:26.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.054+0000 7f8947fff640 1 -- 172.21.15.1:0/2588869908 learned_addr learned my addr 172.21.15.1:0/2588869908 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:03:26.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.054+0000 7f89477fe640 1 --2- 172.21.15.1:0/2588869908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8948100e20 0x7f89480fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:26.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.054+0000 7f8947fff640 1 -- 172.21.15.1:0/2588869908 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8948102570 msgr2=0x7f89480fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:26.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.054+0000 7f8947fff640 1 --2- 172.21.15.1:0/2588869908 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8948102570 0x7f89480fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:26.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.054+0000 7f8947fff640 1 -- 172.21.15.1:0/2588869908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8948100e20 msgr2=0x7f89480fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:26.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.054+0000 7f8947fff640 1 --2- 172.21.15.1:0/2588869908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8948100e20 0x7f89480fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:26.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.054+0000 7f8947fff640 1 -- 172.21.15.1:0/2588869908 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89481a6a30 con 0x7f8948102ef0 2024-09-11T22:03:26.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.055+0000 7f8946ffd640 1 --2- 172.21.15.1:0/2588869908 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8948102570 0x7f89480fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:03:26.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.055+0000 7f89477fe640 1 --2- 172.21.15.1:0/2588869908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8948100e20 0x7f89480fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:03:26.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.055+0000 7f8947fff640 1 --2- 172.21.15.1:0/2588869908 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8948102ef0 0x7f89480fd700 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f894000efd0 tx=0x7f894000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:26.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.055+0000 7f8944ff9640 1 -- 172.21.15.1:0/2588869908 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8940003b70 con 0x7f8948102ef0 2024-09-11T22:03:26.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.055+0000 7f894dbd8640 1 -- 172.21.15.1:0/2588869908 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89481a6d20 con 0x7f8948102ef0 2024-09-11T22:03:26.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.055+0000 7f8944ff9640 1 -- 172.21.15.1:0/2588869908 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8940003d10 con 0x7f8948102ef0 2024-09-11T22:03:26.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.055+0000 7f8944ff9640 1 -- 172.21.15.1:0/2588869908 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8940010a90 con 0x7f8948102ef0 2024-09-11T22:03:26.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.055+0000 7f894dbd8640 1 -- 172.21.15.1:0/2588869908 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89481a7210 con 0x7f8948102ef0 2024-09-11T22:03:26.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.058+0000 7f8944ff9640 1 -- 172.21.15.1:0/2588869908 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f8940003ec0 con 0x7f8948102ef0 2024-09-11T22:03:26.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.058+0000 7f894ffb1740 1 Processor -- start 2024-09-11T22:03:26.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.058+0000 7f894ffb1740 1 -- start start 2024-09-11T22:03:26.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.058+0000 7f8944ff9640 1 --2- 172.21.15.1:0/2588869908 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f89300a2cf0 0x7f89300a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:26.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.059+0000 7f894ffb1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5638e7c1ab40 0x5638e7c19090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:26.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.059+0000 7f894ffb1740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638e7c195d0 0x5638e7c19a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:26.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.059+0000 7f894ffb1740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5638e7c24370 0x5638e7c26760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:26.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.059+0000 7f894ffb1740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5638e7b7f2f0 con 0x5638e7c195d0 2024-09-11T22:03:26.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.059+0000 7f894ffb1740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5638e7b26cf0 con 0x5638e7c24370 2024-09-11T22:03:26.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.059+0000 7f894ffb1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5638e7b7b2b0 con 0x5638e7c1ab40 2024-09-11T22:03:26.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.059+0000 7f8944ff9640 1 -- 172.21.15.1:0/2588869908 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f89400cf840 con 0x7f8948102ef0 2024-09-11T22:03:26.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.059+0000 7f8946ffd640 1 --2- 172.21.15.1:0/2588869908 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f89300a2cf0 0x7f89300a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:26.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.059+0000 7f8946ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638e7c195d0 0x5638e7c19a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:26.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.059+0000 7f89477fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5638e7c1ab40 0x5638e7c19090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:26.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.059+0000 7f8946ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638e7c195d0 0x5638e7c19a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52220/0 (socket says 172.21.15.1:52220) 2024-09-11T22:03:26.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.059+0000 7f89477fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5638e7c1ab40 0x5638e7c19090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:42258/0 (socket says 172.21.15.1:42258) 2024-09-11T22:03:26.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.059+0000 7f8946ffd640 1 -- 172.21.15.1:0/4165416426 learned_addr learned my addr 172.21.15.1:0/4165416426 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:03:26.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.059+0000 7f8947fff640 1 --2- 172.21.15.1:0/4165416426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5638e7c24370 0x5638e7c26760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:26.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.059+0000 7f89477fe640 1 -- 172.21.15.1:0/4165416426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5638e7c24370 msgr2=0x5638e7c26760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:26.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.059+0000 7f89477fe640 1 --2- 172.21.15.1:0/4165416426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5638e7c24370 0x5638e7c26760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:26.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.059+0000 7f89477fe640 1 -- 172.21.15.1:0/4165416426 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638e7c195d0 msgr2=0x5638e7c19a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:26.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.059+0000 7f89477fe640 1 --2- 172.21.15.1:0/4165416426 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638e7c195d0 0x5638e7c19a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:26.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.059+0000 7f89477fe640 1 -- 172.21.15.1:0/4165416426 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638e7b0b3a0 con 0x5638e7c1ab40 2024-09-11T22:03:26.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.059+0000 7f8946ffd640 1 --2- 172.21.15.1:0/4165416426 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638e7c195d0 0x5638e7c19a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:03:26.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.059+0000 7f8946ffd640 1 --2- 172.21.15.1:0/2588869908 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f89300a2cf0 0x7f89300a51b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f893c009fd0 tx=0x7f893c011440 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:26.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.059+0000 7f89477fe640 1 --2- 172.21.15.1:0/4165416426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5638e7c1ab40 0x5638e7c19090 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8938005940 tx=0x7f8938005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ade9949d4c944d9c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:26.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.060+0000 7f892dffb640 1 -- 172.21.15.1:0/4165416426 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8938035040 con 0x5638e7c1ab40 2024-09-11T22:03:26.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.060+0000 7f892dffb640 1 -- 172.21.15.1:0/4165416426 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89380317b0 con 0x5638e7c1ab40 2024-09-11T22:03:26.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.060+0000 7f892dffb640 1 -- 172.21.15.1:0/4165416426 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89380047a0 con 0x5638e7c1ab40 2024-09-11T22:03:26.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.060+0000 7f894ffb1740 1 -- 172.21.15.1:0/4165416426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5638e7c1ab40 msgr2=0x5638e7c19090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:26.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.060+0000 7f894ffb1740 1 --2- 172.21.15.1:0/4165416426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5638e7c1ab40 0x5638e7c19090 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8938005940 tx=0x7f8938005c00 comp rx=0 tx=0).stop 2024-09-11T22:03:26.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.061+0000 7f894ffb1740 1 -- 172.21.15.1:0/4165416426 shutdown_connections 2024-09-11T22:03:26.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.061+0000 7f894ffb1740 1 --2- 172.21.15.1:0/4165416426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5638e7c24370 0x5638e7c26760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:26.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.061+0000 7f894ffb1740 1 --2- 172.21.15.1:0/4165416426 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638e7c195d0 0x5638e7c19a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:26.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.061+0000 7f894ffb1740 1 --2- 172.21.15.1:0/4165416426 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5638e7c1ab40 0x5638e7c19090 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:26.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.061+0000 7f894ffb1740 1 -- 172.21.15.1:0/4165416426 >> 172.21.15.1:0/4165416426 conn(0x5638e7be2310 msgr2=0x5638e7c15c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:26.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.061+0000 7f894ffb1740 1 -- 172.21.15.1:0/4165416426 shutdown_connections 2024-09-11T22:03:26.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.061+0000 7f894ffb1740 1 -- 172.21.15.1:0/4165416426 wait complete. 2024-09-11T22:03:26.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.061+0000 7f894ffb1740 1 -- start start 2024-09-11T22:03:26.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.062+0000 7f894ffb1740 1 Processor -- start 2024-09-11T22:03:26.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.062+0000 7f894ffb1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5638e7c195d0 0x5638e7c23c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:26.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.062+0000 7f894ffb1740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638e7c1ab40 0x5638e7c20f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:26.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.062+0000 7f894ffb1740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5638e7c24370 0x5638e7c21490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:26.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.062+0000 7f894ffb1740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5638e7c27010 con 0x5638e7c1ab40 2024-09-11T22:03:26.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.062+0000 7f894ffb1740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5638e7b3a060 con 0x5638e7c24370 2024-09-11T22:03:26.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.062+0000 7f894ffb1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5638e7c27190 con 0x5638e7c195d0 2024-09-11T22:03:26.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.062+0000 7f894ffb1740 10 client.0 _ll_register_callbacks cb 0x5638e788d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:03:26.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.062+0000 7f89477fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5638e7c195d0 0x5638e7c23c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:26.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.062+0000 7f8946ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638e7c1ab40 0x5638e7c20f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:26.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.062+0000 7f8946ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638e7c1ab40 0x5638e7c20f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52224/0 (socket says 172.21.15.1:52224) 2024-09-11T22:03:26.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.062+0000 7f8946ffd640 1 -- 172.21.15.1:0/1287139858 learned_addr learned my addr 172.21.15.1:0/1287139858 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:03:26.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.062+0000 7f89477fe640 1 --2- 172.21.15.1:0/1287139858 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5638e7c195d0 0x5638e7c23c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:42262/0 (socket says 172.21.15.1:42262) 2024-09-11T22:03:26.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.062+0000 7f8947fff640 1 --2- 172.21.15.1:0/1287139858 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5638e7c24370 0x5638e7c21490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:26.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.063+0000 7f8946ffd640 1 -- 172.21.15.1:0/1287139858 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5638e7c195d0 msgr2=0x5638e7c23c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:26.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.063+0000 7f8946ffd640 1 --2- 172.21.15.1:0/1287139858 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5638e7c195d0 0x5638e7c23c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:26.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.063+0000 7f8946ffd640 1 -- 172.21.15.1:0/1287139858 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5638e7c24370 msgr2=0x5638e7c21490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:26.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.063+0000 7f8946ffd640 1 --2- 172.21.15.1:0/1287139858 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5638e7c24370 0x5638e7c21490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:26.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.063+0000 7f8946ffd640 1 -- 172.21.15.1:0/1287139858 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638e78f03c0 con 0x5638e7c1ab40 2024-09-11T22:03:26.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.063+0000 7f8947fff640 1 --2- 172.21.15.1:0/1287139858 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5638e7c24370 0x5638e7c21490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:03:26.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.063+0000 7f8946ffd640 1 --2- 172.21.15.1:0/1287139858 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638e7c1ab40 0x5638e7c20f50 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f893c009710 tx=0x7f893c01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0c0237769b43021 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:26.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.063+0000 7f8926ffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:03:26.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.063+0000 7f8926ffd640 1 -- 172.21.15.1:0/1287139858 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f893c011da0 con 0x5638e7c1ab40 2024-09-11T22:03:26.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.063+0000 7f8926ffd640 1 -- 172.21.15.1:0/1287139858 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f893c01f460 con 0x5638e7c1ab40 2024-09-11T22:03:26.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.063+0000 7f894ffb1740 1 -- 172.21.15.1:0/1287139858 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89380317b0 con 0x5638e7c1ab40 2024-09-11T22:03:26.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.063+0000 7f8926ffd640 1 -- 172.21.15.1:0/1287139858 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f893c01f660 con 0x5638e7c1ab40 2024-09-11T22:03:26.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.063+0000 7f894ffb1740 10 client.16722 fetch_fsmap learned FSMap version 41 2024-09-11T22:03:26.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.063+0000 7f894ffb1740 1 -- 172.21.15.1:0/1287139858 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5638e7c32a00 con 0x5638e7c1ab40 2024-09-11T22:03:26.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.064+0000 7f8926ffd640 1 -- 172.21.15.1:0/1287139858 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89380317b0 con 0x5638e7c1ab40 2024-09-11T22:03:26.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.064+0000 7f8926ffd640 1 -- 172.21.15.1:0/1287139858 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 41) ==== 8430+0+0 (secure 0 0 0) 0x7f893c02e030 con 0x5638e7c1ab40 2024-09-11T22:03:26.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.064+0000 7f894ffb1740 10 client.16722 fetch_fsmap finished waiting for FSMap version 41 2024-09-11T22:03:26.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.064+0000 7f894ffb1740 10 client.16722 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T22:03:26.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.064+0000 7f894ffb1740 20 client.16722 populate_metadata read hostname 'smithi001' 2024-09-11T22:03:26.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.064+0000 7f894ffb1740 1 --2- 172.21.15.1:0/1287139858 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x5638e7c33f60 0x5638e7c363e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:26.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.064+0000 7f894ffb1740 4 client.16722 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:03:26.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.064+0000 7f894ffb1740 1 -- 172.21.15.1:0/1287139858 --> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5638e7b8c860 con 0x5638e7c33f60 2024-09-11T22:03:26.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.064+0000 7f89477fe640 1 --2- 172.21.15.1:0/1287139858 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x5638e7c33f60 0x5638e7c363e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:26.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.065+0000 7f89477fe640 1 --2- 172.21.15.1:0/1287139858 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x5638e7c33f60 0x5638e7c363e0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2f82c710a7abb04d server_cookie=27d84fbfb556b806 in_seq=0 out_seq=0 2024-09-11T22:03:26.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.065+0000 7f8926ffd640 10 client.16722 ms_handle_connect on v2:172.21.15.161:6838/455365383 2024-09-11T22:03:26.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.065+0000 7f8926ffd640 1 -- 172.21.15.1:0/1287139858 <== mds.1 v2:172.21.15.161:6838/455365383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5638e7b8c860 con 0x5638e7c33f60 2024-09-11T22:03:26.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.065+0000 7f8926ffd640 10 client.16722 handle_command_reply: tid=0 2024-09-11T22:03:26.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.066+0000 7f894ffb1740 1 client.16722 shutdown 2024-09-11T22:03:26.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.066+0000 7f894ffb1740 1 -- 172.21.15.1:0/1287139858 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638e7c1ab40 msgr2=0x5638e7c20f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:26.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.066+0000 7f894ffb1740 1 --2- 172.21.15.1:0/1287139858 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638e7c1ab40 0x5638e7c20f50 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f893c009710 tx=0x7f893c01d000 comp rx=0 tx=0).stop 2024-09-11T22:03:26.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.066+0000 7f894ffb1740 1 -- 172.21.15.1:0/1287139858 shutdown_connections 2024-09-11T22:03:26.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.066+0000 7f894ffb1740 1 --2- 172.21.15.1:0/1287139858 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x5638e7c33f60 0x5638e7c363e0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:26.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.066+0000 7f894ffb1740 1 --2- 172.21.15.1:0/1287139858 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5638e7c24370 0x5638e7c21490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:26.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.066+0000 7f894ffb1740 1 --2- 172.21.15.1:0/1287139858 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638e7c1ab40 0x5638e7c20f50 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:26.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.066+0000 7f894ffb1740 1 --2- 172.21.15.1:0/1287139858 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5638e7c195d0 0x5638e7c23c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:26.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.066+0000 7f8926ffd640 0 client.16722 ms_handle_reset on v2:172.21.15.161:6838/455365383 2024-09-11T22:03:26.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.066+0000 7f894ffb1740 1 -- 172.21.15.1:0/1287139858 >> 172.21.15.1:0/1287139858 conn(0x5638e7be2310 msgr2=0x5638e7c1b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:26.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.066+0000 7f894ffb1740 1 -- 172.21.15.1:0/1287139858 shutdown_connections 2024-09-11T22:03:26.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.066+0000 7f894ffb1740 1 -- 172.21.15.1:0/1287139858 wait complete. 2024-09-11T22:03:26.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.066+0000 7f894ffb1740 20 client.16722 trim_cache size 0 max 16384 2024-09-11T22:03:26.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.080+0000 7f894ffb1740 1 Processor -- start 2024-09-11T22:03:26.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.080+0000 7f894ffb1740 1 -- start start 2024-09-11T22:03:26.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.080+0000 7f894ffb1740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5638e7c195d0 0x5638e7c302b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:26.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.080+0000 7f894ffb1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5638e7c1ab40 0x5638e7c1f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:26.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.080+0000 7f894ffb1740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638e7c24370 0x5638e7c1fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:26.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.080+0000 7f894ffb1740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f89481afc90 con 0x5638e7c24370 2024-09-11T22:03:26.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.080+0000 7f894ffb1740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f89481b1ad0 con 0x5638e7c195d0 2024-09-11T22:03:26.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.080+0000 7f894ffb1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f89481b0420 con 0x5638e7c1ab40 2024-09-11T22:03:26.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.080+0000 7f89477fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5638e7c195d0 0x5638e7c302b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:26.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.080+0000 7f8946ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5638e7c1ab40 0x5638e7c1f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:26.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.080+0000 7f89477fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5638e7c195d0 0x5638e7c302b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:42738/0 (socket says 172.21.15.1:42738) 2024-09-11T22:03:26.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.080+0000 7f89477fe640 1 -- 172.21.15.1:0/1294822423 learned_addr learned my addr 172.21.15.1:0/1294822423 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:03:26.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.081+0000 7f8947fff640 1 --2- 172.21.15.1:0/1294822423 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638e7c24370 0x5638e7c1fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:26.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.081+0000 7f89477fe640 1 -- 172.21.15.1:0/1294822423 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5638e7c1ab40 msgr2=0x5638e7c1f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:26.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.081+0000 7f89477fe640 1 --2- 172.21.15.1:0/1294822423 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5638e7c1ab40 0x5638e7c1f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:26.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.081+0000 7f89477fe640 1 -- 172.21.15.1:0/1294822423 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638e7c24370 msgr2=0x5638e7c1fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:26.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.081+0000 7f89477fe640 1 --2- 172.21.15.1:0/1294822423 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638e7c24370 0x5638e7c1fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:26.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.081+0000 7f89477fe640 1 -- 172.21.15.1:0/1294822423 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638e7c23850 con 0x5638e7c195d0 2024-09-11T22:03:26.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.081+0000 7f8947fff640 1 --2- 172.21.15.1:0/1294822423 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638e7c24370 0x5638e7c1fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:03:26.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.081+0000 7f89477fe640 1 --2- 172.21.15.1:0/1294822423 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5638e7c195d0 0x5638e7c302b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f89380096c0 tx=0x7f8938005390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0a982440eabda2e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:26.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.081+0000 7f892d7fa640 1 -- 172.21.15.1:0/1294822423 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8938035040 con 0x5638e7c195d0 2024-09-11T22:03:26.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.081+0000 7f892d7fa640 1 -- 172.21.15.1:0/1294822423 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8938005400 con 0x5638e7c195d0 2024-09-11T22:03:26.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.081+0000 7f892d7fa640 1 -- 172.21.15.1:0/1294822423 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89380035f0 con 0x5638e7c195d0 2024-09-11T22:03:26.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.082+0000 7f894ffb1740 1 -- 172.21.15.1:0/1294822423 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5638e7c195d0 msgr2=0x5638e7c302b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:26.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.082+0000 7f894ffb1740 1 --2- 172.21.15.1:0/1294822423 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5638e7c195d0 0x5638e7c302b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f89380096c0 tx=0x7f8938005390 comp rx=0 tx=0).stop 2024-09-11T22:03:26.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.082+0000 7f894ffb1740 1 -- 172.21.15.1:0/1294822423 shutdown_connections 2024-09-11T22:03:26.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.082+0000 7f894ffb1740 1 --2- 172.21.15.1:0/1294822423 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638e7c24370 0x5638e7c1fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:26.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.082+0000 7f894ffb1740 1 --2- 172.21.15.1:0/1294822423 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5638e7c1ab40 0x5638e7c1f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:26.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.082+0000 7f894ffb1740 1 --2- 172.21.15.1:0/1294822423 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5638e7c195d0 0x5638e7c302b0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:26.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.082+0000 7f894ffb1740 1 -- 172.21.15.1:0/1294822423 >> 172.21.15.1:0/1294822423 conn(0x5638e7be2310 msgr2=0x5638e7c20f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:26.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.082+0000 7f894ffb1740 1 -- 172.21.15.1:0/1294822423 shutdown_connections 2024-09-11T22:03:26.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.082+0000 7f894ffb1740 1 -- 172.21.15.1:0/1294822423 wait complete. 2024-09-11T22:03:26.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.083+0000 7f894ffb1740 1 -- start start 2024-09-11T22:03:26.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.083+0000 7f894ffb1740 1 Processor -- start 2024-09-11T22:03:26.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.083+0000 7f894ffb1740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5638e7c195d0 0x5638e7c2c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:26.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.083+0000 7f894ffb1740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638e7c1ab40 0x5638e7c2c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:26.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.083+0000 7f894ffb1740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5638e7c24370 0x5638e7c1d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:26.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.083+0000 7f894ffb1740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f89481b1340 con 0x5638e7c1ab40 2024-09-11T22:03:26.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.083+0000 7f894ffb1740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f89481b0bb0 con 0x5638e7c24370 2024-09-11T22:03:26.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.083+0000 7f894ffb1740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f89481adc70 con 0x5638e7c195d0 2024-09-11T22:03:26.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.083+0000 7f894ffb1740 10 client.0 _ll_register_callbacks cb 0x7f893c009e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:03:26.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.083+0000 7f8946ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638e7c1ab40 0x5638e7c2c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:26.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.083+0000 7f89477fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5638e7c195d0 0x5638e7c2c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:26.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.083+0000 7f8946ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638e7c1ab40 0x5638e7c2c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52232/0 (socket says 172.21.15.1:52232) 2024-09-11T22:03:26.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.083+0000 7f8946ffd640 1 -- 172.21.15.1:0/3350537017 learned_addr learned my addr 172.21.15.1:0/3350537017 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:03:26.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.083+0000 7f8947fff640 1 --2- 172.21.15.1:0/3350537017 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5638e7c24370 0x5638e7c1d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:26.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.084+0000 7f89477fe640 1 -- 172.21.15.1:0/3350537017 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5638e7c24370 msgr2=0x5638e7c1d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:26.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.084+0000 7f89477fe640 1 --2- 172.21.15.1:0/3350537017 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5638e7c24370 0x5638e7c1d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:26.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.084+0000 7f89477fe640 1 -- 172.21.15.1:0/3350537017 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638e7c1ab40 msgr2=0x5638e7c2c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:26.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.084+0000 7f89477fe640 1 --2- 172.21.15.1:0/3350537017 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638e7c1ab40 0x5638e7c2c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:26.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.084+0000 7f89477fe640 1 -- 172.21.15.1:0/3350537017 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638e7c1daf0 con 0x5638e7c195d0 2024-09-11T22:03:26.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.084+0000 7f89477fe640 1 --2- 172.21.15.1:0/3350537017 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5638e7c195d0 0x5638e7c2c420 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8938033000 tx=0x7f8938005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94c6341c6df809e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:26.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.084+0000 7f8926ffd640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-11T22:03:26.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.084+0000 7f8926ffd640 1 -- 172.21.15.1:0/3350537017 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f893805e060 con 0x5638e7c195d0 2024-09-11T22:03:26.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.084+0000 7f894ffb1740 1 -- 172.21.15.1:0/3350537017 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8938005400 con 0x5638e7c195d0 2024-09-11T22:03:26.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.084+0000 7f8926ffd640 1 -- 172.21.15.1:0/3350537017 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f8938004310 con 0x5638e7c195d0 2024-09-11T22:03:26.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.084+0000 7f8926ffd640 1 -- 172.21.15.1:0/3350537017 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f8938035040 con 0x5638e7c195d0 2024-09-11T22:03:26.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.084+0000 7f894ffb1740 10 client.26230 fetch_fsmap learned FSMap version 41 2024-09-11T22:03:26.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.084+0000 7f894ffb1740 1 -- 172.21.15.1:0/3350537017 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5638e7c9e4c0 con 0x5638e7c195d0 2024-09-11T22:03:26.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.085+0000 7f8926ffd640 1 -- 172.21.15.1:0/3350537017 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8938005400 con 0x5638e7c195d0 2024-09-11T22:03:26.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.085+0000 7f8926ffd640 1 -- 172.21.15.1:0/3350537017 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 41) ==== 8430+0+0 (secure 0 0 0) 0x7f8938003a80 con 0x5638e7c195d0 2024-09-11T22:03:26.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.085+0000 7f894ffb1740 10 client.26230 fetch_fsmap finished waiting for FSMap version 41 2024-09-11T22:03:26.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.085+0000 7f894ffb1740 10 client.26230 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T22:03:26.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.085+0000 7f894ffb1740 20 client.26230 populate_metadata read hostname 'smithi001' 2024-09-11T22:03:26.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.085+0000 7f894ffb1740 1 --2- 172.21.15.1:0/3350537017 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x5638e7c9e7b0 0x5638e7ca0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:26.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.085+0000 7f894ffb1740 4 client.26230 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:03:26.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.085+0000 7f894ffb1740 1 -- 172.21.15.1:0/3350537017 --> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5638e7b86d70 con 0x5638e7c9e7b0 2024-09-11T22:03:26.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.085+0000 7f8946ffd640 1 --2- 172.21.15.1:0/3350537017 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x5638e7c9e7b0 0x5638e7ca0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:26.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.086+0000 7f8946ffd640 1 --2- 172.21.15.1:0/3350537017 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x5638e7c9e7b0 0x5638e7ca0bd0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=19feba375cec96dc server_cookie=2e627011c7ce4e15 in_seq=0 out_seq=0 2024-09-11T22:03:26.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.086+0000 7f8926ffd640 10 client.26230 ms_handle_connect on v2:172.21.15.161:6838/455365383 2024-09-11T22:03:26.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.086+0000 7f8926ffd640 1 -- 172.21.15.1:0/3350537017 <== mds.1 v2:172.21.15.161:6838/455365383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5638e7b86d70 con 0x5638e7c9e7b0 2024-09-11T22:03:26.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.086+0000 7f8926ffd640 10 client.26230 handle_command_reply: tid=0 2024-09-11T22:03:26.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.086+0000 7f894ffb1740 1 client.26230 shutdown 2024-09-11T22:03:26.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.086+0000 7f894ffb1740 1 -- 172.21.15.1:0/3350537017 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5638e7c195d0 msgr2=0x5638e7c2c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:26.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.086+0000 7f894ffb1740 1 --2- 172.21.15.1:0/3350537017 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5638e7c195d0 0x5638e7c2c420 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8938033000 tx=0x7f8938005850 comp rx=0 tx=0).stop 2024-09-11T22:03:26.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.086+0000 7f894ffb1740 1 -- 172.21.15.1:0/3350537017 shutdown_connections 2024-09-11T22:03:26.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.086+0000 7f894ffb1740 1 --2- 172.21.15.1:0/3350537017 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x5638e7c9e7b0 0x5638e7ca0bd0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:26.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.086+0000 7f894ffb1740 1 --2- 172.21.15.1:0/3350537017 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5638e7c24370 0x5638e7c1d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:26.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.086+0000 7f894ffb1740 1 --2- 172.21.15.1:0/3350537017 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638e7c1ab40 0x5638e7c2c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:26.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.086+0000 7f8926ffd640 0 client.26230 ms_handle_reset on v2:172.21.15.161:6838/455365383 2024-09-11T22:03:26.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.086+0000 7f894ffb1740 1 --2- 172.21.15.1:0/3350537017 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5638e7c195d0 0x5638e7c2c420 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:26.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.086+0000 7f894ffb1740 1 -- 172.21.15.1:0/3350537017 >> 172.21.15.1:0/3350537017 conn(0x5638e7be2310 msgr2=0x5638e7c1f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:26.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.086+0000 7f894ffb1740 1 -- 172.21.15.1:0/3350537017 shutdown_connections 2024-09-11T22:03:26.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.086+0000 7f894ffb1740 1 -- 172.21.15.1:0/3350537017 wait complete. 2024-09-11T22:03:26.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.087+0000 7f894ffb1740 20 client.26230 trim_cache size 0 max 16384 2024-09-11T22:03:26.094 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-11T22:03:26.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.087+0000 7f894dbd8640 1 -- 172.21.15.1:0/2588869908 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f89300a2cf0 msgr2=0x7f89300a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:26.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.087+0000 7f894dbd8640 1 --2- 172.21.15.1:0/2588869908 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f89300a2cf0 0x7f89300a51b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f893c009fd0 tx=0x7f893c011440 comp rx=0 tx=0).stop 2024-09-11T22:03:26.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.087+0000 7f894dbd8640 1 -- 172.21.15.1:0/2588869908 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8948102ef0 msgr2=0x7f89480fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:26.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.087+0000 7f894dbd8640 1 --2- 172.21.15.1:0/2588869908 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8948102ef0 0x7f89480fd700 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f894000efd0 tx=0x7f894000c6a0 comp rx=0 tx=0).stop 2024-09-11T22:03:26.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.087+0000 7f894dbd8640 1 -- 172.21.15.1:0/2588869908 shutdown_connections 2024-09-11T22:03:26.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.087+0000 7f894dbd8640 1 --2- 172.21.15.1:0/2588869908 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f89300a2cf0 0x7f89300a51b0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:26.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.087+0000 7f894dbd8640 1 --2- 172.21.15.1:0/2588869908 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8948102ef0 0x7f89480fd700 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:26.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.087+0000 7f894dbd8640 1 --2- 172.21.15.1:0/2588869908 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8948102570 0x7f89480fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:26.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.087+0000 7f894dbd8640 1 --2- 172.21.15.1:0/2588869908 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8948100e20 0x7f89480fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:26.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.087+0000 7f894dbd8640 1 -- 172.21.15.1:0/2588869908 >> 172.21.15.1:0/2588869908 conn(0x7f89480fc7d0 msgr2=0x7f894806a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:26.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.088+0000 7f894dbd8640 1 -- 172.21.15.1:0/2588869908 shutdown_connections 2024-09-11T22:03:26.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:26.088+0000 7f894dbd8640 1 -- 172.21.15.1:0/2588869908 wait complete. 2024-09-11T22:03:26.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:26 smithi053 ceph-mon[34908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:03:26.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:26 smithi161 ceph-mon[35615]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:03:26.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:26 smithi001 ceph-mon[30935]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:03:26.989 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T22:03:27.269 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:03:27.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:27 smithi053 ceph-mon[34908]: pgmap v191: 97 pgs: 97 active+clean; 233 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.6 MiB/s wr, 767 op/s 2024-09-11T22:03:27.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:27 smithi161 ceph-mon[35615]: pgmap v191: 97 pgs: 97 active+clean; 233 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.6 MiB/s wr, 767 op/s 2024-09-11T22:03:27.391 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:27 smithi001 ceph-mon[30935]: pgmap v191: 97 pgs: 97 active+clean; 233 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.6 MiB/s wr, 767 op/s 2024-09-11T22:03:27.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.856+0000 7f68095e2640 1 -- 172.21.15.1:0/478804453 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6804106b50 msgr2=0x7f6804114cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:27.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.856+0000 7f68095e2640 1 --2- 172.21.15.1:0/478804453 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6804106b50 0x7f6804114cf0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f67fc009ca0 tx=0x7f67fc02a8c0 comp rx=0 tx=0).stop 2024-09-11T22:03:27.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.856+0000 7f68095e2640 1 -- 172.21.15.1:0/478804453 shutdown_connections 2024-09-11T22:03:27.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.856+0000 7f68095e2640 1 --2- 172.21.15.1:0/478804453 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6804106b50 0x7f6804114cf0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:27.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.856+0000 7f68095e2640 1 --2- 172.21.15.1:0/478804453 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6804101360 0x7f6804112780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:27.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.856+0000 7f68095e2640 1 --2- 172.21.15.1:0/478804453 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68041009b0 0x7f6804100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:27.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.856+0000 7f68095e2640 1 -- 172.21.15.1:0/478804453 >> 172.21.15.1:0/478804453 conn(0x7f68040fc7d0 msgr2=0x7f68040febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:27.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.856+0000 7f68095e2640 1 -- 172.21.15.1:0/478804453 shutdown_connections 2024-09-11T22:03:27.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.856+0000 7f68095e2640 1 -- 172.21.15.1:0/478804453 wait complete. 2024-09-11T22:03:27.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.857+0000 7f68095e2640 1 Processor -- start 2024-09-11T22:03:27.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.857+0000 7f68095e2640 1 -- start start 2024-09-11T22:03:27.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.858+0000 7f68095e2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68041009b0 0x7f68041a47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:27.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.858+0000 7f68095e2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6804101360 0x7f68041a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:27.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.858+0000 7f68095e2640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f680419e9b0 0x7f680419ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:27.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.858+0000 7f68095e2640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6804118680 con 0x7f68041009b0 2024-09-11T22:03:27.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.858+0000 7f68095e2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6804118500 con 0x7f6804101360 2024-09-11T22:03:27.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.858+0000 7f68095e2640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6804118800 con 0x7f680419e9b0 2024-09-11T22:03:27.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.858+0000 7f6802ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68041009b0 0x7f68041a47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:27.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.858+0000 7f6802ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68041009b0 0x7f68041a47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59226/0 (socket says 172.21.15.1:59226) 2024-09-11T22:03:27.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.858+0000 7f6802ffd640 1 -- 172.21.15.1:0/3004344955 learned_addr learned my addr 172.21.15.1:0/3004344955 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:03:27.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.858+0000 7f68027fc640 1 --2- 172.21.15.1:0/3004344955 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6804101360 0x7f68041a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:27.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.858+0000 7f68037fe640 1 --2- 172.21.15.1:0/3004344955 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f680419e9b0 0x7f680419ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:27.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.859+0000 7f6802ffd640 1 -- 172.21.15.1:0/3004344955 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f680419e9b0 msgr2=0x7f680419ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:27.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.859+0000 7f6802ffd640 1 --2- 172.21.15.1:0/3004344955 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f680419e9b0 0x7f680419ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:27.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.859+0000 7f6802ffd640 1 -- 172.21.15.1:0/3004344955 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6804101360 msgr2=0x7f68041a4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:27.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.859+0000 7f6802ffd640 1 --2- 172.21.15.1:0/3004344955 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6804101360 0x7f68041a4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:27.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.859+0000 7f6802ffd640 1 -- 172.21.15.1:0/3004344955 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f680419f4d0 con 0x7f68041009b0 2024-09-11T22:03:27.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.859+0000 7f6802ffd640 1 --2- 172.21.15.1:0/3004344955 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68041009b0 0x7f68041a47f0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f67f400e9f0 tx=0x7f67f400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:27.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.859+0000 7f67ebfff640 1 -- 172.21.15.1:0/3004344955 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f400cd80 con 0x7f68041009b0 2024-09-11T22:03:27.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.859+0000 7f67ebfff640 1 -- 172.21.15.1:0/3004344955 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67f4010910 con 0x7f68041009b0 2024-09-11T22:03:27.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.860+0000 7f67ebfff640 1 -- 172.21.15.1:0/3004344955 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f4018990 con 0x7f68041009b0 2024-09-11T22:03:27.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.860+0000 7f68095e2640 1 -- 172.21.15.1:0/3004344955 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f680419f7c0 con 0x7f68041009b0 2024-09-11T22:03:27.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.860+0000 7f68095e2640 1 -- 172.21.15.1:0/3004344955 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68041ab9b0 con 0x7f68041009b0 2024-09-11T22:03:27.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.864+0000 7f680b9bb740 1 Processor -- start 2024-09-11T22:03:27.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.864+0000 7f680b9bb740 1 -- start start 2024-09-11T22:03:27.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.864+0000 7f680b9bb740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d19d3a6b40 0x55d19d3a5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:27.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.864+0000 7f680b9bb740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d19d3a55d0 0x55d19d3a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:27.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.864+0000 7f680b9bb740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d19d3b0370 0x55d19d3b2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:27.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.864+0000 7f680b9bb740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55d19d30b2f0 con 0x55d19d3b0370 2024-09-11T22:03:27.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.864+0000 7f680b9bb740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55d19d2b2cf0 con 0x55d19d3a55d0 2024-09-11T22:03:27.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.864+0000 7f680b9bb740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d19d3072b0 con 0x55d19d3a6b40 2024-09-11T22:03:27.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.864+0000 7f67ebfff640 1 -- 172.21.15.1:0/3004344955 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f67f4010ac0 con 0x7f68041009b0 2024-09-11T22:03:27.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.864+0000 7f68027fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d19d3b0370 0x55d19d3b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:27.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.864+0000 7f68037fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d19d3a6b40 0x55d19d3a5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:27.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.864+0000 7f68027fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d19d3b0370 0x55d19d3b2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59230/0 (socket says 172.21.15.1:59230) 2024-09-11T22:03:27.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.865+0000 7f68027fc640 1 -- 172.21.15.1:0/3863029367 learned_addr learned my addr 172.21.15.1:0/3863029367 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:03:27.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.865+0000 7f6802ffd640 1 --2- 172.21.15.1:0/3863029367 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d19d3a55d0 0x55d19d3a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:27.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.865+0000 7f68027fc640 1 -- 172.21.15.1:0/3863029367 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d19d3a6b40 msgr2=0x55d19d3a5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:27.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.865+0000 7f68027fc640 1 --2- 172.21.15.1:0/3863029367 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d19d3a6b40 0x55d19d3a5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:27.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.865+0000 7f68027fc640 1 -- 172.21.15.1:0/3863029367 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d19d3a55d0 msgr2=0x55d19d3a5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:27.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.865+0000 7f68027fc640 1 --2- 172.21.15.1:0/3863029367 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d19d3a55d0 0x55d19d3a5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:27.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.865+0000 7f68027fc640 1 -- 172.21.15.1:0/3863029367 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d19d2973a0 con 0x55d19d3b0370 2024-09-11T22:03:27.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.865+0000 7f68027fc640 1 --2- 172.21.15.1:0/3863029367 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d19d3b0370 0x55d19d3b2760 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f67f8009c00 tx=0x7f67f8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf5cf4b9021ccdab server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:27.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.866+0000 7f67ebfff640 1 --2- 172.21.15.1:0/3004344955 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f67ec0a2da0 0x7f67ec0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:27.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.866+0000 7f67ebfff640 1 -- 172.21.15.1:0/3004344955 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f67f40cf6a0 con 0x7f68041009b0 2024-09-11T22:03:27.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.866+0000 7f67e97fa640 1 -- 172.21.15.1:0/3863029367 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f8004230 con 0x55d19d3b0370 2024-09-11T22:03:27.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.867+0000 7f67e97fa640 1 -- 172.21.15.1:0/3863029367 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67f80043d0 con 0x55d19d3b0370 2024-09-11T22:03:27.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.867+0000 7f67e97fa640 1 -- 172.21.15.1:0/3863029367 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f8019730 con 0x55d19d3b0370 2024-09-11T22:03:27.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.868+0000 7f680b9bb740 1 -- 172.21.15.1:0/3863029367 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d19d3b0370 msgr2=0x55d19d3b2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:27.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.868+0000 7f680b9bb740 1 --2- 172.21.15.1:0/3863029367 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d19d3b0370 0x55d19d3b2760 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f67f8009c00 tx=0x7f67f8011810 comp rx=0 tx=0).stop 2024-09-11T22:03:27.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.868+0000 7f68037fe640 1 --2- 172.21.15.1:0/3004344955 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f67ec0a2da0 0x7f67ec0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:27.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.868+0000 7f680b9bb740 1 -- 172.21.15.1:0/3863029367 shutdown_connections 2024-09-11T22:03:27.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.868+0000 7f680b9bb740 1 --2- 172.21.15.1:0/3863029367 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d19d3b0370 0x55d19d3b2760 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:27.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.868+0000 7f680b9bb740 1 --2- 172.21.15.1:0/3863029367 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d19d3a55d0 0x55d19d3a5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:27.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.868+0000 7f680b9bb740 1 --2- 172.21.15.1:0/3863029367 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d19d3a6b40 0x55d19d3a5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:27.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.868+0000 7f680b9bb740 1 -- 172.21.15.1:0/3863029367 >> 172.21.15.1:0/3863029367 conn(0x55d19d36e310 msgr2=0x55d19d3a1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:27.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.869+0000 7f680b9bb740 1 -- 172.21.15.1:0/3863029367 shutdown_connections 2024-09-11T22:03:27.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.869+0000 7f680b9bb740 1 -- 172.21.15.1:0/3863029367 wait complete. 2024-09-11T22:03:27.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.869+0000 7f68037fe640 1 --2- 172.21.15.1:0/3004344955 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f67ec0a2da0 0x7f67ec0a5260 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f67fc006010 tx=0x7f67fc005d80 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:27.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.869+0000 7f680b9bb740 1 -- start start 2024-09-11T22:03:27.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.869+0000 7f680b9bb740 1 Processor -- start 2024-09-11T22:03:27.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.870+0000 7f680b9bb740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d19d3a55d0 0x55d19d3afd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:27.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.870+0000 7f680b9bb740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d19d3a6b40 0x55d19d3ad030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:27.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.870+0000 7f680b9bb740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d19d3b0370 0x55d19d3ad640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:27.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.870+0000 7f680b9bb740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55d19d3b3070 con 0x55d19d3a55d0 2024-09-11T22:03:27.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.870+0000 7f680b9bb740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55d19d2c6060 con 0x55d19d3b0370 2024-09-11T22:03:27.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.870+0000 7f68037fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d19d3b0370 0x55d19d3ad640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:27.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.870+0000 7f6802ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d19d3a55d0 0x55d19d3afd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:27.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.870+0000 7f68037fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d19d3b0370 0x55d19d3ad640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:54432/0 (socket says 172.21.15.1:54432) 2024-09-11T22:03:27.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.870+0000 7f68037fe640 1 -- 172.21.15.1:0/2602047965 learned_addr learned my addr 172.21.15.1:0/2602047965 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:03:27.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.870+0000 7f680b9bb740 1 -- 172.21.15.1:0/2602047965 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d19d3b31f0 con 0x55d19d3a6b40 2024-09-11T22:03:27.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.870+0000 7f68027fc640 1 --2- 172.21.15.1:0/2602047965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d19d3a6b40 0x55d19d3ad030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:27.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.870+0000 7f680b9bb740 10 client.0 _ll_register_callbacks cb 0x55d19d0198c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:03:27.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.871+0000 7f68037fe640 1 -- 172.21.15.1:0/2602047965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d19d3a6b40 msgr2=0x55d19d3ad030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:27.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.871+0000 7f68037fe640 1 --2- 172.21.15.1:0/2602047965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d19d3a6b40 0x55d19d3ad030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:27.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.871+0000 7f68037fe640 1 -- 172.21.15.1:0/2602047965 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d19d3a55d0 msgr2=0x55d19d3afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:27.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.871+0000 7f68037fe640 1 --2- 172.21.15.1:0/2602047965 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d19d3a55d0 0x55d19d3afd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:27.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.871+0000 7f68037fe640 1 -- 172.21.15.1:0/2602047965 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d19d07c3c0 con 0x55d19d3b0370 2024-09-11T22:03:27.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.872+0000 7f68037fe640 1 --2- 172.21.15.1:0/2602047965 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d19d3b0370 0x55d19d3ad640 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f67fc035000 tx=0x7f67fc02a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c24336828c7c16a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:27.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.872+0000 7f67e6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-11T22:03:27.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.872+0000 7f67e6ff5640 1 -- 172.21.15.1:0/2602047965 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67fc031720 con 0x55d19d3b0370 2024-09-11T22:03:27.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.872+0000 7f67e6ff5640 1 -- 172.21.15.1:0/2602047965 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f67fc0318c0 con 0x55d19d3b0370 2024-09-11T22:03:27.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.872+0000 7f680b9bb740 1 -- 172.21.15.1:0/2602047965 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67f80043d0 con 0x55d19d3b0370 2024-09-11T22:03:27.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.873+0000 7f67e6ff5640 1 -- 172.21.15.1:0/2602047965 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f67fc003fb0 con 0x55d19d3b0370 2024-09-11T22:03:27.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.873+0000 7f680b9bb740 10 client.26279 fetch_fsmap learned FSMap version 42 2024-09-11T22:03:27.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.873+0000 7f680b9bb740 1 -- 172.21.15.1:0/2602047965 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55d19d3cfb80 con 0x55d19d3b0370 2024-09-11T22:03:27.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.873+0000 7f67e6ff5640 1 -- 172.21.15.1:0/2602047965 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67fc036050 con 0x55d19d3b0370 2024-09-11T22:03:27.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.873+0000 7f67e6ff5640 1 -- 172.21.15.1:0/2602047965 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f67fc059030 con 0x55d19d3b0370 2024-09-11T22:03:27.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.874+0000 7f680b9bb740 10 client.26279 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:03:27.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.874+0000 7f680b9bb740 10 client.26279 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:03:27.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.874+0000 7f680b9bb740 20 client.26279 populate_metadata read hostname 'smithi001' 2024-09-11T22:03:27.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.874+0000 7f680b9bb740 1 --2- 172.21.15.1:0/2602047965 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55d19d3d1060 0x55d19d3d34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:27.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.874+0000 7f680b9bb740 4 client.26279 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:03:27.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.874+0000 7f6802ffd640 1 --2- 172.21.15.1:0/2602047965 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55d19d3d1060 0x55d19d3d34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:27.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.874+0000 7f680b9bb740 1 -- 172.21.15.1:0/2602047965 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f67f80043d0 con 0x55d19d3d1060 2024-09-11T22:03:27.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.874+0000 7f6802ffd640 1 --2- 172.21.15.1:0/2602047965 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55d19d3d1060 0x55d19d3d34c0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f55a7f2f3b41e00a server_cookie=45cf81bb5d5f7f42 in_seq=0 out_seq=0 2024-09-11T22:03:27.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.874+0000 7f67e6ff5640 10 client.26279 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:03:27.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.876+0000 7f67e6ff5640 1 -- 172.21.15.1:0/2602047965 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f67f80043d0 con 0x55d19d3d1060 2024-09-11T22:03:27.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.876+0000 7f67e6ff5640 10 client.26279 handle_command_reply: tid=0 2024-09-11T22:03:27.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.876+0000 7f680b9bb740 1 client.26279 shutdown 2024-09-11T22:03:27.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.876+0000 7f680b9bb740 1 -- 172.21.15.1:0/2602047965 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d19d3b0370 msgr2=0x55d19d3ad640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:27.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.876+0000 7f680b9bb740 1 --2- 172.21.15.1:0/2602047965 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d19d3b0370 0x55d19d3ad640 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f67fc035000 tx=0x7f67fc02a8c0 comp rx=0 tx=0).stop 2024-09-11T22:03:27.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.876+0000 7f680b9bb740 1 -- 172.21.15.1:0/2602047965 shutdown_connections 2024-09-11T22:03:27.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.876+0000 7f680b9bb740 1 --2- 172.21.15.1:0/2602047965 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55d19d3d1060 0x55d19d3d34c0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:27.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.876+0000 7f680b9bb740 1 --2- 172.21.15.1:0/2602047965 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d19d3b0370 0x55d19d3ad640 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:27.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.876+0000 7f680b9bb740 1 --2- 172.21.15.1:0/2602047965 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d19d3a6b40 0x55d19d3ad030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:27.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.876+0000 7f67e6ff5640 0 client.26279 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:03:27.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.876+0000 7f680b9bb740 1 --2- 172.21.15.1:0/2602047965 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d19d3a55d0 0x55d19d3afd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:27.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.876+0000 7f680b9bb740 1 -- 172.21.15.1:0/2602047965 >> 172.21.15.1:0/2602047965 conn(0x55d19d36e310 msgr2=0x55d19d3a7960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:27.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.876+0000 7f680b9bb740 1 -- 172.21.15.1:0/2602047965 shutdown_connections 2024-09-11T22:03:27.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.876+0000 7f680b9bb740 1 -- 172.21.15.1:0/2602047965 wait complete. 2024-09-11T22:03:27.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.876+0000 7f680b9bb740 20 client.26279 trim_cache size 0 max 16384 2024-09-11T22:03:27.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.896+0000 7f680b9bb740 1 Processor -- start 2024-09-11T22:03:27.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.896+0000 7f680b9bb740 1 -- start start 2024-09-11T22:03:27.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.896+0000 7f680b9bb740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d19d3a55d0 0x55d19d3aefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:27.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.896+0000 7f680b9bb740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d19d3a6b40 0x55d19d3a8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:27.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.897+0000 7f680b9bb740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d19d3b0370 0x55d19d3a8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:27.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.897+0000 7f680b9bb740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f68041adf00 con 0x55d19d3b0370 2024-09-11T22:03:27.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.897+0000 7f680b9bb740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6804112490 con 0x55d19d3a6b40 2024-09-11T22:03:27.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.897+0000 7f680b9bb740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f68041ad770 con 0x55d19d3a55d0 2024-09-11T22:03:27.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.897+0000 7f6802ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d19d3a55d0 0x55d19d3aefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:27.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.897+0000 7f68037fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d19d3b0370 0x55d19d3a8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:27.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.897+0000 7f68037fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d19d3b0370 0x55d19d3a8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59250/0 (socket says 172.21.15.1:59250) 2024-09-11T22:03:27.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.897+0000 7f68037fe640 1 -- 172.21.15.1:0/3064724627 learned_addr learned my addr 172.21.15.1:0/3064724627 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:03:27.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.897+0000 7f68027fc640 1 --2- 172.21.15.1:0/3064724627 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d19d3a6b40 0x55d19d3a8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:27.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.897+0000 7f68037fe640 1 -- 172.21.15.1:0/3064724627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d19d3a55d0 msgr2=0x55d19d3aefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:27.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.897+0000 7f68037fe640 1 --2- 172.21.15.1:0/3064724627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d19d3a55d0 0x55d19d3aefa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:27.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.897+0000 7f68037fe640 1 -- 172.21.15.1:0/3064724627 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d19d3a6b40 msgr2=0x55d19d3a8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:27.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.897+0000 7f68037fe640 1 --2- 172.21.15.1:0/3064724627 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d19d3a6b40 0x55d19d3a8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:27.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.898+0000 7f68037fe640 1 -- 172.21.15.1:0/3064724627 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d19d3b00f0 con 0x55d19d3b0370 2024-09-11T22:03:27.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.898+0000 7f68037fe640 1 --2- 172.21.15.1:0/3064724627 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d19d3b0370 0x55d19d3a8660 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f67fc02ade0 tx=0x7f67fc02afc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36ca6c326be1d999 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:27.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.899+0000 7f67e8ff9640 1 -- 172.21.15.1:0/3064724627 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67fc031920 con 0x55d19d3b0370 2024-09-11T22:03:27.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.899+0000 7f67e8ff9640 1 -- 172.21.15.1:0/3064724627 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67fc036070 con 0x55d19d3b0370 2024-09-11T22:03:27.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.899+0000 7f67e8ff9640 1 -- 172.21.15.1:0/3064724627 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67fc030c50 con 0x55d19d3b0370 2024-09-11T22:03:27.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.900+0000 7f680b9bb740 1 -- 172.21.15.1:0/3064724627 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d19d3b0370 msgr2=0x55d19d3a8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:27.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.900+0000 7f680b9bb740 1 --2- 172.21.15.1:0/3064724627 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d19d3b0370 0x55d19d3a8660 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f67fc02ade0 tx=0x7f67fc02afc0 comp rx=0 tx=0).stop 2024-09-11T22:03:27.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.900+0000 7f680b9bb740 1 -- 172.21.15.1:0/3064724627 shutdown_connections 2024-09-11T22:03:27.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.900+0000 7f680b9bb740 1 --2- 172.21.15.1:0/3064724627 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d19d3b0370 0x55d19d3a8660 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:27.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.900+0000 7f680b9bb740 1 --2- 172.21.15.1:0/3064724627 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d19d3a6b40 0x55d19d3a8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:27.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.900+0000 7f680b9bb740 1 --2- 172.21.15.1:0/3064724627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d19d3a55d0 0x55d19d3aefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:27.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.900+0000 7f680b9bb740 1 -- 172.21.15.1:0/3064724627 >> 172.21.15.1:0/3064724627 conn(0x55d19d36e310 msgr2=0x55d19d3ae890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:27.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.901+0000 7f680b9bb740 1 -- 172.21.15.1:0/3064724627 shutdown_connections 2024-09-11T22:03:27.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.901+0000 7f680b9bb740 1 -- 172.21.15.1:0/3064724627 wait complete. 2024-09-11T22:03:27.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.901+0000 7f680b9bb740 1 -- start start 2024-09-11T22:03:27.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.901+0000 7f680b9bb740 1 Processor -- start 2024-09-11T22:03:27.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.902+0000 7f680b9bb740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d19d3a55d0 0x55d19d3b48c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:27.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.902+0000 7f680b9bb740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d19d3a6b40 0x55d19d3b4e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:27.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.902+0000 7f680b9bb740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d19d3b0370 0x55d19d43ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:27.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.902+0000 7f680b9bb740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f68041ac850 con 0x55d19d3b0370 2024-09-11T22:03:27.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.902+0000 7f6802ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d19d3a55d0 0x55d19d3b48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:27.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.902+0000 7f680b9bb740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f68041ac0c0 con 0x55d19d3a55d0 2024-09-11T22:03:27.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.902+0000 7f680b9bb740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f68041acfe0 con 0x55d19d3a6b40 2024-09-11T22:03:27.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.902+0000 7f6802ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d19d3a55d0 0x55d19d3b48c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:54442/0 (socket says 172.21.15.1:54442) 2024-09-11T22:03:27.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.902+0000 7f6802ffd640 1 -- 172.21.15.1:0/2070866155 learned_addr learned my addr 172.21.15.1:0/2070866155 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:03:27.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.902+0000 7f680b9bb740 10 client.0 _ll_register_callbacks cb 0x7f67fc009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:03:27.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.902+0000 7f68027fc640 1 --2- 172.21.15.1:0/2070866155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d19d3a6b40 0x55d19d3b4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:27.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.902+0000 7f68037fe640 1 --2- 172.21.15.1:0/2070866155 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d19d3b0370 0x55d19d43ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:27.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.902+0000 7f6802ffd640 1 -- 172.21.15.1:0/2070866155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d19d3a6b40 msgr2=0x55d19d3b4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:27.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.902+0000 7f6802ffd640 1 --2- 172.21.15.1:0/2070866155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d19d3a6b40 0x55d19d3b4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:27.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.902+0000 7f6802ffd640 1 -- 172.21.15.1:0/2070866155 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d19d3b0370 msgr2=0x55d19d43ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:27.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.902+0000 7f6802ffd640 1 --2- 172.21.15.1:0/2070866155 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d19d3b0370 0x55d19d43ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:27.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.902+0000 7f6802ffd640 1 -- 172.21.15.1:0/2070866155 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d19d43b2e0 con 0x55d19d3a55d0 2024-09-11T22:03:27.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.902+0000 7f68027fc640 1 --2- 172.21.15.1:0/2070866155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d19d3a6b40 0x55d19d3b4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:03:27.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.903+0000 7f6802ffd640 1 --2- 172.21.15.1:0/2070866155 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d19d3a55d0 0x55d19d3b48c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f67f409b930 tx=0x7f67f4018420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c7d9d65c3db72d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:27.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.903+0000 7f67e6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-11T22:03:27.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.903+0000 7f67e6ff5640 1 -- 172.21.15.1:0/2070866155 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f4086040 con 0x55d19d3a55d0 2024-09-11T22:03:27.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.903+0000 7f67e6ff5640 1 -- 172.21.15.1:0/2070866155 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f67f40982a0 con 0x55d19d3a55d0 2024-09-11T22:03:27.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.903+0000 7f680b9bb740 1 -- 172.21.15.1:0/2070866155 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67fc036070 con 0x55d19d3a55d0 2024-09-11T22:03:27.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.903+0000 7f67e6ff5640 1 -- 172.21.15.1:0/2070866155 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f67f40a47f0 con 0x55d19d3a55d0 2024-09-11T22:03:27.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.904+0000 7f680b9bb740 10 client.26285 fetch_fsmap learned FSMap version 42 2024-09-11T22:03:27.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.904+0000 7f680b9bb740 1 -- 172.21.15.1:0/2070866155 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55d19d43c4e0 con 0x55d19d3a55d0 2024-09-11T22:03:27.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.904+0000 7f67e6ff5640 1 -- 172.21.15.1:0/2070866155 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67f40cfb50 con 0x55d19d3a55d0 2024-09-11T22:03:27.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.904+0000 7f67e6ff5640 1 -- 172.21.15.1:0/2070866155 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f67f409b080 con 0x55d19d3a55d0 2024-09-11T22:03:27.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.904+0000 7f680b9bb740 10 client.26285 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:03:27.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.904+0000 7f680b9bb740 10 client.26285 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:03:27.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.904+0000 7f680b9bb740 20 client.26285 populate_metadata read hostname 'smithi001' 2024-09-11T22:03:27.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.904+0000 7f680b9bb740 1 --2- 172.21.15.1:0/2070866155 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55d19d3d1060 0x55d19d43ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:27.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.904+0000 7f680b9bb740 4 client.26285 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:03:27.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.904+0000 7f680b9bb740 1 -- 172.21.15.1:0/2070866155 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f67fc036070 con 0x55d19d3d1060 2024-09-11T22:03:27.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.905+0000 7f68027fc640 1 --2- 172.21.15.1:0/2070866155 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55d19d3d1060 0x55d19d43ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:27.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.906+0000 7f67e6ff5640 10 client.26285 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:03:27.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.906+0000 7f68027fc640 1 --2- 172.21.15.1:0/2070866155 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55d19d3d1060 0x55d19d43ecf0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11910371c8aea8a0 server_cookie=2aeac37289e0c5eb in_seq=0 out_seq=0 2024-09-11T22:03:27.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.906+0000 7f67e6ff5640 1 -- 172.21.15.1:0/2070866155 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f67fc036070 con 0x55d19d3d1060 2024-09-11T22:03:27.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.906+0000 7f67e6ff5640 10 client.26285 handle_command_reply: tid=0 2024-09-11T22:03:27.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.906+0000 7f680b9bb740 1 client.26285 shutdown 2024-09-11T22:03:27.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.907+0000 7f680b9bb740 1 -- 172.21.15.1:0/2070866155 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d19d3a55d0 msgr2=0x55d19d3b48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:27.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.907+0000 7f680b9bb740 1 --2- 172.21.15.1:0/2070866155 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d19d3a55d0 0x55d19d3b48c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f67f409b930 tx=0x7f67f4018420 comp rx=0 tx=0).stop 2024-09-11T22:03:27.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.907+0000 7f680b9bb740 1 -- 172.21.15.1:0/2070866155 shutdown_connections 2024-09-11T22:03:27.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.907+0000 7f680b9bb740 1 --2- 172.21.15.1:0/2070866155 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55d19d3d1060 0x55d19d43ecf0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:27.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.907+0000 7f680b9bb740 1 --2- 172.21.15.1:0/2070866155 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d19d3b0370 0x55d19d43ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:27.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.907+0000 7f67e6ff5640 0 client.26285 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:03:27.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.907+0000 7f680b9bb740 1 --2- 172.21.15.1:0/2070866155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d19d3a6b40 0x55d19d3b4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:27.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.907+0000 7f680b9bb740 1 --2- 172.21.15.1:0/2070866155 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d19d3a55d0 0x55d19d3b48c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:27.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.907+0000 7f680b9bb740 1 -- 172.21.15.1:0/2070866155 >> 172.21.15.1:0/2070866155 conn(0x55d19d36e310 msgr2=0x55d19d3a0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:27.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.907+0000 7f680b9bb740 1 -- 172.21.15.1:0/2070866155 shutdown_connections 2024-09-11T22:03:27.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.907+0000 7f680b9bb740 1 -- 172.21.15.1:0/2070866155 wait complete. 2024-09-11T22:03:27.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.907+0000 7f680b9bb740 20 client.26285 trim_cache size 0 max 16384 2024-09-11T22:03:27.909 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-11T22:03:27.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.908+0000 7f68095e2640 1 -- 172.21.15.1:0/3004344955 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f67ec0a2da0 msgr2=0x7f67ec0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:27.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.908+0000 7f68095e2640 1 --2- 172.21.15.1:0/3004344955 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f67ec0a2da0 0x7f67ec0a5260 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f67fc006010 tx=0x7f67fc005d80 comp rx=0 tx=0).stop 2024-09-11T22:03:27.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.908+0000 7f68095e2640 1 -- 172.21.15.1:0/3004344955 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68041009b0 msgr2=0x7f68041a47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:27.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.908+0000 7f68095e2640 1 --2- 172.21.15.1:0/3004344955 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68041009b0 0x7f68041a47f0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f67f400e9f0 tx=0x7f67f400eeb0 comp rx=0 tx=0).stop 2024-09-11T22:03:27.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.908+0000 7f68095e2640 1 -- 172.21.15.1:0/3004344955 shutdown_connections 2024-09-11T22:03:27.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.908+0000 7f68095e2640 1 --2- 172.21.15.1:0/3004344955 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f67ec0a2da0 0x7f67ec0a5260 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:27.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.908+0000 7f68095e2640 1 --2- 172.21.15.1:0/3004344955 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f680419e9b0 0x7f680419ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:27.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.908+0000 7f68095e2640 1 --2- 172.21.15.1:0/3004344955 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6804101360 0x7f68041a4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:27.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.908+0000 7f68095e2640 1 --2- 172.21.15.1:0/3004344955 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68041009b0 0x7f68041a47f0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:27.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.908+0000 7f68095e2640 1 -- 172.21.15.1:0/3004344955 >> 172.21.15.1:0/3004344955 conn(0x7f68040fc7d0 msgr2=0x7f68040fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:27.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.909+0000 7f68095e2640 1 -- 172.21.15.1:0/3004344955 shutdown_connections 2024-09-11T22:03:27.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:27.909+0000 7f68095e2640 1 -- 172.21.15.1:0/3004344955 wait complete. 2024-09-11T22:03:28.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:28 smithi161 ceph-mon[35615]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:03:28.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:28 smithi001 ceph-mon[30935]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:03:28.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:28 smithi053 ceph-mon[34908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:03:29.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:29 smithi161 ceph-mon[35615]: pgmap v192: 97 pgs: 97 active+clean; 233 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.1 MiB/s wr, 689 op/s 2024-09-11T22:03:29.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:29 smithi001 ceph-mon[30935]: pgmap v192: 97 pgs: 97 active+clean; 233 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.1 MiB/s wr, 689 op/s 2024-09-11T22:03:29.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:29 smithi053 ceph-mon[34908]: pgmap v192: 97 pgs: 97 active+clean; 233 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.1 MiB/s wr, 689 op/s 2024-09-11T22:03:29.665 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T22:03:29.666 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T22:03:29.997 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:03:30.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.535+0000 7f0554a29640 1 -- 172.21.15.1:0/555130737 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f05501027a0 msgr2=0x7f05501067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:30.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.535+0000 7f0554a29640 1 --2- 172.21.15.1:0/555130737 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f05501027a0 0x7f05501067f0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f0538009900 tx=0x7f053802a3f0 comp rx=0 tx=0).stop 2024-09-11T22:03:30.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.535+0000 7f0554a29640 1 -- 172.21.15.1:0/555130737 shutdown_connections 2024-09-11T22:03:30.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.535+0000 7f0554a29640 1 --2- 172.21.15.1:0/555130737 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0550106f20 0x7f055010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:30.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.535+0000 7f0554a29640 1 --2- 172.21.15.1:0/555130737 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f05501027a0 0x7f05501067f0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:30.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.535+0000 7f0554a29640 1 --2- 172.21.15.1:0/555130737 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0550101df0 0x7f05501021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:30.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.535+0000 7f0554a29640 1 -- 172.21.15.1:0/555130737 >> 172.21.15.1:0/555130737 conn(0x7f05500fd8f0 msgr2=0x7f05500ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:30.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.535+0000 7f0554a29640 1 -- 172.21.15.1:0/555130737 shutdown_connections 2024-09-11T22:03:30.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.535+0000 7f0554a29640 1 -- 172.21.15.1:0/555130737 wait complete. 2024-09-11T22:03:30.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.536+0000 7f0554a29640 1 Processor -- start 2024-09-11T22:03:30.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.536+0000 7f0554a29640 1 -- start start 2024-09-11T22:03:30.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.537+0000 7f0554a29640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0550101df0 0x7f05500744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:30.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.537+0000 7f0554a29640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05501027a0 0x7f0550074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:30.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.537+0000 7f0554a29640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0550106f20 0x7f0550074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:30.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.537+0000 7f0554a29640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0550111f40 con 0x7f0550106f20 2024-09-11T22:03:30.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.537+0000 7f0554a29640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0550111dc0 con 0x7f05501027a0 2024-09-11T22:03:30.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.537+0000 7f0554a29640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f05501120c0 con 0x7f0550101df0 2024-09-11T22:03:30.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.537+0000 7f054dd9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05501027a0 0x7f0550074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:30.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.537+0000 7f054ed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0550106f20 0x7f0550074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:30.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.537+0000 7f054dd9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05501027a0 0x7f0550074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:54454/0 (socket says 172.21.15.1:54454) 2024-09-11T22:03:30.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.537+0000 7f054dd9b640 1 -- 172.21.15.1:0/1841723468 learned_addr learned my addr 172.21.15.1:0/1841723468 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:03:30.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.537+0000 7f054e59c640 1 --2- 172.21.15.1:0/1841723468 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0550101df0 0x7f05500744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:30.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.537+0000 7f054dd9b640 1 -- 172.21.15.1:0/1841723468 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0550101df0 msgr2=0x7f05500744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:30.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.537+0000 7f054dd9b640 1 --2- 172.21.15.1:0/1841723468 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0550101df0 0x7f05500744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:30.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.537+0000 7f054dd9b640 1 -- 172.21.15.1:0/1841723468 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0550106f20 msgr2=0x7f0550074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:30.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.537+0000 7f054dd9b640 1 --2- 172.21.15.1:0/1841723468 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0550106f20 0x7f0550074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:30.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.537+0000 7f054dd9b640 1 -- 172.21.15.1:0/1841723468 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05501a69a0 con 0x7f05501027a0 2024-09-11T22:03:30.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.537+0000 7f054dd9b640 1 --2- 172.21.15.1:0/1841723468 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05501027a0 0x7f0550074a30 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0538006010 tx=0x7f0538004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:30.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.538+0000 7f05477fe640 1 -- 172.21.15.1:0/1841723468 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f053803f030 con 0x7f05501027a0 2024-09-11T22:03:30.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.538+0000 7f05477fe640 1 -- 172.21.15.1:0/1841723468 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0538004440 con 0x7f05501027a0 2024-09-11T22:03:30.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.538+0000 7f0554a29640 1 -- 172.21.15.1:0/1841723468 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05501a6c30 con 0x7f05501027a0 2024-09-11T22:03:30.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.538+0000 7f05477fe640 1 -- 172.21.15.1:0/1841723468 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0538035070 con 0x7f05501027a0 2024-09-11T22:03:30.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.538+0000 7f0554a29640 1 -- 172.21.15.1:0/1841723468 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05501a7160 con 0x7f05501027a0 2024-09-11T22:03:30.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.540+0000 7f05477fe640 1 -- 172.21.15.1:0/1841723468 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f0538042030 con 0x7f05501027a0 2024-09-11T22:03:30.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.542+0000 7f0556e02740 1 Processor -- start 2024-09-11T22:03:30.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.542+0000 7f05477fe640 1 --2- 172.21.15.1:0/1841723468 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f053c0a2d70 0x7f053c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:30.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.542+0000 7f054ed9d640 1 --2- 172.21.15.1:0/1841723468 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f053c0a2d70 0x7f053c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:30.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.542+0000 7f0556e02740 1 -- start start 2024-09-11T22:03:30.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.542+0000 7f05477fe640 1 -- 172.21.15.1:0/1841723468 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f05380ef750 con 0x7f05501027a0 2024-09-11T22:03:30.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.542+0000 7f0556e02740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558f53c42b60 0x558f53c42f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:30.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.542+0000 7f0556e02740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f53c43480 0x558f53c4d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:30.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.542+0000 7f0556e02740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f53c4dd40 0x558f53c50130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:30.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.542+0000 7f0556e02740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x558f53ba8280 con 0x558f53c42b60 2024-09-11T22:03:30.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.542+0000 7f0556e02740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x558f53b4fb10 con 0x558f53c4dd40 2024-09-11T22:03:30.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.542+0000 7f0556e02740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x558f53ba4240 con 0x558f53c43480 2024-09-11T22:03:30.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.543+0000 7f054dd9b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f53c43480 0x558f53c4d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:30.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.543+0000 7f054dd9b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f53c43480 0x558f53c4d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:44548/0 (socket says 172.21.15.1:44548) 2024-09-11T22:03:30.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.543+0000 7f054dd9b640 1 -- 172.21.15.1:0/2583524998 learned_addr learned my addr 172.21.15.1:0/2583524998 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:03:30.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.543+0000 7f054e59c640 1 --2- 172.21.15.1:0/2583524998 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558f53c42b60 0x558f53c42f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:30.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.543+0000 7f054ed9d640 1 --2- 172.21.15.1:0/2583524998 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f53c4dd40 0x558f53c50130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:30.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.543+0000 7f054ed9d640 1 --2- 172.21.15.1:0/1841723468 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f053c0a2d70 0x7f053c0a5230 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0548005fd0 tx=0x7f0548009500 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:30.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.543+0000 7f054dd9b640 1 -- 172.21.15.1:0/2583524998 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f53c4dd40 msgr2=0x558f53c50130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:30.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.543+0000 7f054dd9b640 1 --2- 172.21.15.1:0/2583524998 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f53c4dd40 0x558f53c50130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:30.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.543+0000 7f054dd9b640 1 -- 172.21.15.1:0/2583524998 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558f53c42b60 msgr2=0x558f53c42f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:30.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.543+0000 7f054dd9b640 1 --2- 172.21.15.1:0/2583524998 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558f53c42b60 0x558f53c42f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:30.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.543+0000 7f054dd9b640 1 -- 172.21.15.1:0/2583524998 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f53b5c1f0 con 0x558f53c43480 2024-09-11T22:03:30.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.543+0000 7f054ed9d640 1 --2- 172.21.15.1:0/2583524998 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f53c4dd40 0x558f53c50130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:03:30.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.543+0000 7f054dd9b640 1 --2- 172.21.15.1:0/2583524998 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f53c43480 0x558f53c4d800 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f05380b86b0 tx=0x7f05380bb640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0bc46d4eeaf41a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:30.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.544+0000 7f0544ff9640 1 -- 172.21.15.1:0/2583524998 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0538049040 con 0x558f53c43480 2024-09-11T22:03:30.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.544+0000 7f0544ff9640 1 -- 172.21.15.1:0/2583524998 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05380c48a0 con 0x558f53c43480 2024-09-11T22:03:30.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.544+0000 7f0544ff9640 1 -- 172.21.15.1:0/2583524998 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0538031e30 con 0x558f53c43480 2024-09-11T22:03:30.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.544+0000 7f0556e02740 1 -- 172.21.15.1:0/2583524998 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f53c43480 msgr2=0x558f53c4d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:30.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.544+0000 7f0556e02740 1 --2- 172.21.15.1:0/2583524998 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f53c43480 0x558f53c4d800 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f05380b86b0 tx=0x7f05380bb640 comp rx=0 tx=0).stop 2024-09-11T22:03:30.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.544+0000 7f0556e02740 1 -- 172.21.15.1:0/2583524998 shutdown_connections 2024-09-11T22:03:30.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.544+0000 7f0556e02740 1 --2- 172.21.15.1:0/2583524998 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f53c4dd40 0x558f53c50130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:30.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.544+0000 7f0556e02740 1 --2- 172.21.15.1:0/2583524998 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f53c43480 0x558f53c4d800 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:30.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.544+0000 7f0556e02740 1 --2- 172.21.15.1:0/2583524998 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558f53c42b60 0x558f53c42f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:30.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.544+0000 7f0556e02740 1 -- 172.21.15.1:0/2583524998 >> 172.21.15.1:0/2583524998 conn(0x558f53c3f0f0 msgr2=0x558f53c3f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:30.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.545+0000 7f0556e02740 1 -- 172.21.15.1:0/2583524998 shutdown_connections 2024-09-11T22:03:30.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.545+0000 7f0556e02740 1 -- 172.21.15.1:0/2583524998 wait complete. 2024-09-11T22:03:30.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.545+0000 7f0556e02740 1 -- start start 2024-09-11T22:03:30.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.545+0000 7f0556e02740 1 Processor -- start 2024-09-11T22:03:30.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.546+0000 7f0556e02740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f53c42b60 0x558f53c4ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:30.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.546+0000 7f0556e02740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f53c43480 0x558f53c4b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:30.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.546+0000 7f0556e02740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558f53c4dd40 0x558f53c4b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:30.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.546+0000 7f054e59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f53c42b60 0x558f53c4ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:30.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.546+0000 7f054e59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f53c42b60 0x558f53c4ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:54464/0 (socket says 172.21.15.1:54464) 2024-09-11T22:03:30.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.546+0000 7f054e59c640 1 -- 172.21.15.1:0/4201893969 learned_addr learned my addr 172.21.15.1:0/4201893969 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:03:30.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.546+0000 7f0556e02740 1 -- 172.21.15.1:0/4201893969 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x558f53c506d0 con 0x558f53c4dd40 2024-09-11T22:03:30.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.546+0000 7f0556e02740 1 -- 172.21.15.1:0/4201893969 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x558f53b62f10 con 0x558f53c42b60 2024-09-11T22:03:30.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.546+0000 7f0556e02740 1 -- 172.21.15.1:0/4201893969 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x558f53c50850 con 0x558f53c43480 2024-09-11T22:03:30.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.546+0000 7f054dd9b640 1 --2- 172.21.15.1:0/4201893969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f53c43480 0x558f53c4b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:30.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.546+0000 7f0556e02740 10 client.0 _ll_register_callbacks cb 0x558f53b2ba40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:03:30.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.546+0000 7f054e59c640 1 -- 172.21.15.1:0/4201893969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f53c43480 msgr2=0x558f53c4b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:30.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.546+0000 7f054e59c640 1 --2- 172.21.15.1:0/4201893969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f53c43480 0x558f53c4b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:30.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.546+0000 7f054ed9d640 1 --2- 172.21.15.1:0/4201893969 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558f53c4dd40 0x558f53c4b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:30.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.546+0000 7f054e59c640 1 -- 172.21.15.1:0/4201893969 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558f53c4dd40 msgr2=0x558f53c4b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:30.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.546+0000 7f054e59c640 1 --2- 172.21.15.1:0/4201893969 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558f53c4dd40 0x558f53c4b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:30.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.546+0000 7f054e59c640 1 -- 172.21.15.1:0/4201893969 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f539196a0 con 0x558f53c42b60 2024-09-11T22:03:30.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.546+0000 7f0556e02740 1 -- 172.21.15.1:0/4201893969 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05380c48a0 con 0x558f53c42b60 2024-09-11T22:03:30.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.546+0000 7f054ed9d640 1 --2- 172.21.15.1:0/4201893969 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558f53c4dd40 0x558f53c4b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:03:30.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.546+0000 7f054e59c640 1 --2- 172.21.15.1:0/4201893969 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f53c42b60 0x558f53c4ac10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0540012840 tx=0x7f0540012d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba32914932f967ee server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:30.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.547+0000 7f0535ffb640 10 client.26297 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-11T22:03:30.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.547+0000 7f0535ffb640 1 -- 172.21.15.1:0/4201893969 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f054000f8a0 con 0x558f53c42b60 2024-09-11T22:03:30.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.547+0000 7f0535ffb640 1 -- 172.21.15.1:0/4201893969 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f054000fa40 con 0x558f53c42b60 2024-09-11T22:03:30.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.547+0000 7f0535ffb640 1 -- 172.21.15.1:0/4201893969 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f054000fc80 con 0x558f53c42b60 2024-09-11T22:03:30.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.547+0000 7f0556e02740 10 client.26297 fetch_fsmap learned FSMap version 42 2024-09-11T22:03:30.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.547+0000 7f0556e02740 1 -- 172.21.15.1:0/4201893969 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({fsmap=42}) -- 0x558f53c5c1c0 con 0x558f53c42b60 2024-09-11T22:03:30.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.547+0000 7f0535ffb640 1 -- 172.21.15.1:0/4201893969 <== mon.2 v2:172.21.15.53:3300/0 4 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f0540026470 con 0x558f53c42b60 2024-09-11T22:03:30.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.547+0000 7f0556e02740 10 client.26297 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:03:30.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.547+0000 7f0556e02740 10 client.26297 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:03:30.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.547+0000 7f0556e02740 20 client.26297 populate_metadata read hostname 'smithi001' 2024-09-11T22:03:30.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.547+0000 7f0556e02740 1 --2- 172.21.15.1:0/4201893969 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x558f53c5d5c0 0x558f53c5fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:30.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.547+0000 7f0556e02740 4 client.26297 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:03:30.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.547+0000 7f0556e02740 1 -- 172.21.15.1:0/4201893969 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f53bb57f0 con 0x558f53c5d5c0 2024-09-11T22:03:30.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.548+0000 7f054dd9b640 1 --2- 172.21.15.1:0/4201893969 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x558f53c5d5c0 0x558f53c5fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:30.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.548+0000 7f0535ffb640 10 client.26297 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:03:30.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.548+0000 7f054dd9b640 1 --2- 172.21.15.1:0/4201893969 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x558f53c5d5c0 0x558f53c5fa50 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b4b049d470c4bad server_cookie=f6c558a6516ca2e4 in_seq=0 out_seq=0 2024-09-11T22:03:30.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.549+0000 7f0535ffb640 1 -- 172.21.15.1:0/4201893969 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558f53bb57f0 con 0x558f53c5d5c0 2024-09-11T22:03:30.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.549+0000 7f0535ffb640 10 client.26297 handle_command_reply: tid=0 2024-09-11T22:03:30.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.549+0000 7f0556e02740 1 client.26297 shutdown 2024-09-11T22:03:30.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.549+0000 7f0556e02740 1 -- 172.21.15.1:0/4201893969 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f53c42b60 msgr2=0x558f53c4ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:30.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.549+0000 7f0556e02740 1 --2- 172.21.15.1:0/4201893969 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f53c42b60 0x558f53c4ac10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0540012840 tx=0x7f0540012d00 comp rx=0 tx=0).stop 2024-09-11T22:03:30.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.549+0000 7f0556e02740 1 -- 172.21.15.1:0/4201893969 shutdown_connections 2024-09-11T22:03:30.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.549+0000 7f0556e02740 1 --2- 172.21.15.1:0/4201893969 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x558f53c5d5c0 0x558f53c5fa50 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:30.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.549+0000 7f0556e02740 1 --2- 172.21.15.1:0/4201893969 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558f53c4dd40 0x558f53c4b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:30.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.549+0000 7f0556e02740 1 --2- 172.21.15.1:0/4201893969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f53c43480 0x558f53c4b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:30.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.549+0000 7f0556e02740 1 --2- 172.21.15.1:0/4201893969 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f53c42b60 0x558f53c4ac10 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:30.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.549+0000 7f0556e02740 1 -- 172.21.15.1:0/4201893969 >> 172.21.15.1:0/4201893969 conn(0x558f53c3f0f0 msgr2=0x558f53c44e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:30.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.549+0000 7f0535ffb640 0 client.26297 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:03:30.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.549+0000 7f0556e02740 1 -- 172.21.15.1:0/4201893969 shutdown_connections 2024-09-11T22:03:30.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.550+0000 7f0556e02740 1 -- 172.21.15.1:0/4201893969 wait complete. 2024-09-11T22:03:30.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.550+0000 7f0556e02740 20 client.26297 trim_cache size 0 max 16384 2024-09-11T22:03:30.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.568+0000 7f0556e02740 1 Processor -- start 2024-09-11T22:03:30.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.568+0000 7f0556e02740 1 -- start start 2024-09-11T22:03:30.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.569+0000 7f0556e02740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f53c42b60 0x558f53c45610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:30.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.569+0000 7f0556e02740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558f53c43480 0x558f53c4ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:30.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.569+0000 7f0556e02740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f53c4dd40 0x558f53c65360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:30.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.569+0000 7f0556e02740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f05501afc10 con 0x558f53c43480 2024-09-11T22:03:30.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.569+0000 7f0556e02740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f05501b1a50 con 0x558f53c4dd40 2024-09-11T22:03:30.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.569+0000 7f0556e02740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f05501b03a0 con 0x558f53c42b60 2024-09-11T22:03:30.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.569+0000 7f054e59c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f53c42b60 0x558f53c45610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:30.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.569+0000 7f054e59c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f53c42b60 0x558f53c45610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:44568/0 (socket says 172.21.15.1:44568) 2024-09-11T22:03:30.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.569+0000 7f054e59c640 1 -- 172.21.15.1:0/495573478 learned_addr learned my addr 172.21.15.1:0/495573478 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:03:30.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.569+0000 7f054dd9b640 1 --2- 172.21.15.1:0/495573478 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558f53c43480 0x558f53c4ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:30.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.569+0000 7f054ed9d640 1 --2- 172.21.15.1:0/495573478 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f53c4dd40 0x558f53c65360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:30.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.569+0000 7f054e59c640 1 -- 172.21.15.1:0/495573478 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f53c4dd40 msgr2=0x558f53c65360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:30.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.569+0000 7f054e59c640 1 --2- 172.21.15.1:0/495573478 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f53c4dd40 0x558f53c65360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:30.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.569+0000 7f054e59c640 1 -- 172.21.15.1:0/495573478 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558f53c43480 msgr2=0x558f53c4ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:30.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.569+0000 7f054e59c640 1 --2- 172.21.15.1:0/495573478 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558f53c43480 0x558f53c4ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:30.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.569+0000 7f054e59c640 1 -- 172.21.15.1:0/495573478 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f53c4d4e0 con 0x558f53c42b60 2024-09-11T22:03:30.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.569+0000 7f054ed9d640 1 --2- 172.21.15.1:0/495573478 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f53c4dd40 0x558f53c65360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:03:30.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.570+0000 7f054dd9b640 1 --2- 172.21.15.1:0/495573478 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558f53c43480 0x558f53c4ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:03:30.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.570+0000 7f054e59c640 1 --2- 172.21.15.1:0/495573478 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f53c42b60 0x558f53c45610 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0540012d40 tx=0x7f0540014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47028b04d3e0cc3c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:30.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.570+0000 7f0537fff640 1 -- 172.21.15.1:0/495573478 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05400122e0 con 0x558f53c42b60 2024-09-11T22:03:30.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.570+0000 7f0537fff640 1 -- 172.21.15.1:0/495573478 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f054002c020 con 0x558f53c42b60 2024-09-11T22:03:30.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.570+0000 7f0537fff640 1 -- 172.21.15.1:0/495573478 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f054000f8a0 con 0x558f53c42b60 2024-09-11T22:03:30.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.571+0000 7f0556e02740 1 -- 172.21.15.1:0/495573478 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f53c42b60 msgr2=0x558f53c45610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:30.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.571+0000 7f0556e02740 1 --2- 172.21.15.1:0/495573478 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f53c42b60 0x558f53c45610 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0540012d40 tx=0x7f0540014000 comp rx=0 tx=0).stop 2024-09-11T22:03:30.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.571+0000 7f0556e02740 1 -- 172.21.15.1:0/495573478 shutdown_connections 2024-09-11T22:03:30.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.571+0000 7f0556e02740 1 --2- 172.21.15.1:0/495573478 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f53c4dd40 0x558f53c65360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:30.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.571+0000 7f0556e02740 1 --2- 172.21.15.1:0/495573478 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558f53c43480 0x558f53c4ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:30.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.571+0000 7f0556e02740 1 --2- 172.21.15.1:0/495573478 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f53c42b60 0x558f53c45610 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:30.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.571+0000 7f0556e02740 1 -- 172.21.15.1:0/495573478 >> 172.21.15.1:0/495573478 conn(0x558f53c3f0f0 msgr2=0x558f53c4c360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:30.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.571+0000 7f0556e02740 1 -- 172.21.15.1:0/495573478 shutdown_connections 2024-09-11T22:03:30.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.571+0000 7f0556e02740 1 -- 172.21.15.1:0/495573478 wait complete. 2024-09-11T22:03:30.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.572+0000 7f0556e02740 1 -- start start 2024-09-11T22:03:30.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.572+0000 7f0556e02740 1 Processor -- start 2024-09-11T22:03:30.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.572+0000 7f0556e02740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f53c42b60 0x558f53c51670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:30.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.572+0000 7f0556e02740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558f53c43480 0x558f53c51bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:30.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.573+0000 7f0556e02740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f53c4dd40 0x558f53c520f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:30.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.573+0000 7f054dd9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558f53c43480 0x558f53c51bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:30.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.573+0000 7f054e59c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f53c42b60 0x558f53c51670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:30.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.573+0000 7f054dd9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558f53c43480 0x558f53c51bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59310/0 (socket says 172.21.15.1:59310) 2024-09-11T22:03:30.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.573+0000 7f054e59c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f53c42b60 0x558f53c51670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:44570/0 (socket says 172.21.15.1:44570) 2024-09-11T22:03:30.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.573+0000 7f054dd9b640 1 -- 172.21.15.1:0/1583718948 learned_addr learned my addr 172.21.15.1:0/1583718948 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:03:30.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.573+0000 7f0556e02740 1 -- 172.21.15.1:0/1583718948 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f05501b12c0 con 0x558f53c43480 2024-09-11T22:03:30.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.573+0000 7f0556e02740 1 -- 172.21.15.1:0/1583718948 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f05501b0b30 con 0x558f53c4dd40 2024-09-11T22:03:30.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.573+0000 7f054ed9d640 1 --2- 172.21.15.1:0/1583718948 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f53c4dd40 0x558f53c520f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:30.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.573+0000 7f0556e02740 1 -- 172.21.15.1:0/1583718948 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f05501adbf0 con 0x558f53c42b60 2024-09-11T22:03:30.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.573+0000 7f0556e02740 10 client.0 _ll_register_callbacks cb 0x7f0540009890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:03:30.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.573+0000 7f054dd9b640 1 -- 172.21.15.1:0/1583718948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f53c42b60 msgr2=0x558f53c51670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:30.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.573+0000 7f054dd9b640 1 --2- 172.21.15.1:0/1583718948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f53c42b60 0x558f53c51670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:30.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.573+0000 7f054dd9b640 1 -- 172.21.15.1:0/1583718948 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f53c4dd40 msgr2=0x558f53c520f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:30.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.573+0000 7f054dd9b640 1 --2- 172.21.15.1:0/1583718948 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f53c4dd40 0x558f53c520f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:30.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.573+0000 7f054dd9b640 1 -- 172.21.15.1:0/1583718948 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f53cc7330 con 0x558f53c43480 2024-09-11T22:03:30.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.574+0000 7f054dd9b640 1 --2- 172.21.15.1:0/1583718948 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558f53c43480 0x558f53c51bb0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f05380bbc70 tx=0x7f05380bbe60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad5416fda2151cc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:30.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.574+0000 7f0535ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:03:30.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.574+0000 7f0535ffb640 1 -- 172.21.15.1:0/1583718948 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0538049040 con 0x558f53c43480 2024-09-11T22:03:30.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.574+0000 7f0535ffb640 1 -- 172.21.15.1:0/1583718948 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f05380b8350 con 0x558f53c43480 2024-09-11T22:03:30.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.574+0000 7f0556e02740 1 -- 172.21.15.1:0/1583718948 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f054002c020 con 0x558f53c43480 2024-09-11T22:03:30.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.574+0000 7f0535ffb640 1 -- 172.21.15.1:0/1583718948 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f05380c4410 con 0x558f53c43480 2024-09-11T22:03:30.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.574+0000 7f0556e02740 10 client.16761 fetch_fsmap learned FSMap version 42 2024-09-11T22:03:30.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.574+0000 7f0556e02740 1 -- 172.21.15.1:0/1583718948 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x558f53cc8590 con 0x558f53c43480 2024-09-11T22:03:30.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.574+0000 7f0535ffb640 1 -- 172.21.15.1:0/1583718948 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f054002c020 con 0x558f53c43480 2024-09-11T22:03:30.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.575+0000 7f0535ffb640 1 -- 172.21.15.1:0/1583718948 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f05380ca030 con 0x558f53c43480 2024-09-11T22:03:30.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.575+0000 7f0556e02740 10 client.16761 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:03:30.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.575+0000 7f0556e02740 10 client.16761 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:03:30.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.575+0000 7f0556e02740 20 client.16761 populate_metadata read hostname 'smithi001' 2024-09-11T22:03:30.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.575+0000 7f0556e02740 1 --2- 172.21.15.1:0/1583718948 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x558f53cc8980 0x558f53ccad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:30.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.575+0000 7f0556e02740 4 client.16761 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T22:03:30.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.575+0000 7f0556e02740 1 -- 172.21.15.1:0/1583718948 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558f53bafd00 con 0x558f53cc8980 2024-09-11T22:03:30.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.575+0000 7f054e59c640 1 --2- 172.21.15.1:0/1583718948 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x558f53cc8980 0x558f53ccad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:30.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.576+0000 7f054e59c640 1 --2- 172.21.15.1:0/1583718948 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x558f53cc8980 0x558f53ccad70 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac1381f6392730ce server_cookie=2245d7ada58ec71b in_seq=0 out_seq=0 2024-09-11T22:03:30.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.576+0000 7f0535ffb640 10 client.16761 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:03:30.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.578+0000 7f0535ffb640 1 -- 172.21.15.1:0/1583718948 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558f53bafd00 con 0x558f53cc8980 2024-09-11T22:03:30.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.578+0000 7f0535ffb640 10 client.16761 handle_command_reply: tid=0 2024-09-11T22:03:30.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.578+0000 7f0556e02740 1 client.16761 shutdown 2024-09-11T22:03:30.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.578+0000 7f0556e02740 1 -- 172.21.15.1:0/1583718948 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558f53c43480 msgr2=0x558f53c51bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:30.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.578+0000 7f0556e02740 1 --2- 172.21.15.1:0/1583718948 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558f53c43480 0x558f53c51bb0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f05380bbc70 tx=0x7f05380bbe60 comp rx=0 tx=0).stop 2024-09-11T22:03:30.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.579+0000 7f0556e02740 1 -- 172.21.15.1:0/1583718948 shutdown_connections 2024-09-11T22:03:30.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.579+0000 7f0556e02740 1 --2- 172.21.15.1:0/1583718948 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x558f53cc8980 0x558f53ccad70 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:30.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.579+0000 7f0556e02740 1 --2- 172.21.15.1:0/1583718948 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558f53c4dd40 0x558f53c520f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:30.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.579+0000 7f0535ffb640 0 client.16761 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:03:30.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.579+0000 7f0556e02740 1 --2- 172.21.15.1:0/1583718948 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558f53c43480 0x558f53c51bb0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:30.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.579+0000 7f0556e02740 1 --2- 172.21.15.1:0/1583718948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558f53c42b60 0x558f53c51670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:30.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.579+0000 7f0556e02740 1 -- 172.21.15.1:0/1583718948 >> 172.21.15.1:0/1583718948 conn(0x558f53c3f0f0 msgr2=0x558f53c3e270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:30.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.579+0000 7f0556e02740 1 -- 172.21.15.1:0/1583718948 shutdown_connections 2024-09-11T22:03:30.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.579+0000 7f0556e02740 1 -- 172.21.15.1:0/1583718948 wait complete. 2024-09-11T22:03:30.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.579+0000 7f0556e02740 20 client.16761 trim_cache size 0 max 16384 2024-09-11T22:03:30.581 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-11T22:03:30.581 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-09-11T22:03:30.581 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "4bb42c4e-eaef-4b43-9290-d831431460f1", 2024-09-11T22:03:30.582 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-09-11T22:03:30.582 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-11T22:03:30.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.580+0000 7f0554a29640 1 -- 172.21.15.1:0/1841723468 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f053c0a2d70 msgr2=0x7f053c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:30.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.580+0000 7f0554a29640 1 --2- 172.21.15.1:0/1841723468 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f053c0a2d70 0x7f053c0a5230 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0548005fd0 tx=0x7f0548009500 comp rx=0 tx=0).stop 2024-09-11T22:03:30.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.580+0000 7f0554a29640 1 -- 172.21.15.1:0/1841723468 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05501027a0 msgr2=0x7f0550074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:30.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.580+0000 7f0554a29640 1 --2- 172.21.15.1:0/1841723468 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05501027a0 0x7f0550074a30 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0538006010 tx=0x7f0538004040 comp rx=0 tx=0).stop 2024-09-11T22:03:30.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.580+0000 7f0554a29640 1 -- 172.21.15.1:0/1841723468 shutdown_connections 2024-09-11T22:03:30.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.580+0000 7f0554a29640 1 --2- 172.21.15.1:0/1841723468 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f053c0a2d70 0x7f053c0a5230 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:30.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.580+0000 7f0554a29640 1 --2- 172.21.15.1:0/1841723468 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0550106f20 0x7f0550074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:30.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.580+0000 7f0554a29640 1 --2- 172.21.15.1:0/1841723468 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f05501027a0 0x7f0550074a30 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:30.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.580+0000 7f0554a29640 1 --2- 172.21.15.1:0/1841723468 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0550101df0 0x7f05500744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:30.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.580+0000 7f0554a29640 1 -- 172.21.15.1:0/1841723468 >> 172.21.15.1:0/1841723468 conn(0x7f05500fd8f0 msgr2=0x7f05500feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:30.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.580+0000 7f0554a29640 1 -- 172.21.15.1:0/1841723468 shutdown_connections 2024-09-11T22:03:30.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:30.580+0000 7f0554a29640 1 -- 172.21.15.1:0/1841723468 wait complete. 2024-09-11T22:03:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.581+0000 7f6f18c65640 1 -- 172.21.15.1:0/159872605 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f14101340 msgr2=0x7f6f1410d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.581+0000 7f6f18c65640 1 --2- 172.21.15.1:0/159872605 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f14101340 0x7f6f1410d870 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c009cd0 tx=0x7f6f0c02a8e0 comp rx=0 tx=0).stop 2024-09-11T22:03:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.581+0000 7f6f18c65640 1 -- 172.21.15.1:0/159872605 shutdown_connections 2024-09-11T22:03:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.581+0000 7f6f18c65640 1 --2- 172.21.15.1:0/159872605 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f14101340 0x7f6f1410d870 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.581+0000 7f6f18c65640 1 --2- 172.21.15.1:0/159872605 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f141009a0 0x7f6f14100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.581+0000 7f6f18c65640 1 --2- 172.21.15.1:0/159872605 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f141069b0 0x7f6f14106d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.581+0000 7f6f18c65640 1 -- 172.21.15.1:0/159872605 >> 172.21.15.1:0/159872605 conn(0x7f6f140779a0 msgr2=0x7f6f140fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.581+0000 7f6f18c65640 1 -- 172.21.15.1:0/159872605 shutdown_connections 2024-09-11T22:03:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.581+0000 7f6f18c65640 1 -- 172.21.15.1:0/159872605 wait complete. 2024-09-11T22:03:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.582+0000 7f6f18c65640 1 Processor -- start 2024-09-11T22:03:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.582+0000 7f6f18c65640 1 -- start start 2024-09-11T22:03:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.582+0000 7f6f18c65640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f141009a0 0x7f6f141983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.583+0000 7f6f18c65640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f14101340 0x7f6f14198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.583+0000 7f6f18c65640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f141069b0 0x7f6f1419cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.583+0000 7f6f18c65640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6f14111f30 con 0x7f6f14101340 2024-09-11T22:03:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.583+0000 7f6f18c65640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6f14111db0 con 0x7f6f141069b0 2024-09-11T22:03:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.583+0000 7f6f18c65640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6f141120b0 con 0x7f6f141009a0 2024-09-11T22:03:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.583+0000 7f6f11d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f14101340 0x7f6f14198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.583+0000 7f6f1259c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f141009a0 0x7f6f141983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.583+0000 7f6f1259c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f141009a0 0x7f6f141983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:44594/0 (socket says 172.21.15.1:44594) 2024-09-11T22:03:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.583+0000 7f6f1259c640 1 -- 172.21.15.1:0/1503181939 learned_addr learned my addr 172.21.15.1:0/1503181939 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:03:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.583+0000 7f6f1259c640 1 -- 172.21.15.1:0/1503181939 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f141069b0 msgr2=0x7f6f1419cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.583+0000 7f6f1259c640 1 --2- 172.21.15.1:0/1503181939 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f141069b0 0x7f6f1419cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.583+0000 7f6f1259c640 1 -- 172.21.15.1:0/1503181939 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f14101340 msgr2=0x7f6f14198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.583+0000 7f6f1259c640 1 --2- 172.21.15.1:0/1503181939 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f14101340 0x7f6f14198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.583+0000 7f6f1259c640 1 -- 172.21.15.1:0/1503181939 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f1419d3b0 con 0x7f6f141009a0 2024-09-11T22:03:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.583+0000 7f6f11d9b640 1 --2- 172.21.15.1:0/1503181939 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f14101340 0x7f6f14198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:03:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.584+0000 7f6f1259c640 1 --2- 172.21.15.1:0/1503181939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f141009a0 0x7f6f141983c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6f0400d770 tx=0x7f6f0400dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.584+0000 7f6f037fe640 1 -- 172.21.15.1:0/1503181939 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f04017070 con 0x7f6f141009a0 2024-09-11T22:03:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.584+0000 7f6f037fe640 1 -- 172.21.15.1:0/1503181939 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f04005500 con 0x7f6f141009a0 2024-09-11T22:03:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.584+0000 7f6f037fe640 1 -- 172.21.15.1:0/1503181939 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f04004e00 con 0x7f6f141009a0 2024-09-11T22:03:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.584+0000 7f6f18c65640 1 -- 172.21.15.1:0/1503181939 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f1419d6a0 con 0x7f6f141009a0 2024-09-11T22:03:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.584+0000 7f6f18c65640 1 -- 172.21.15.1:0/1503181939 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f141a4f30 con 0x7f6f141009a0 2024-09-11T22:03:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.587+0000 7f6f037fe640 1 -- 172.21.15.1:0/1503181939 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f6f04004fa0 con 0x7f6f141009a0 2024-09-11T22:03:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.587+0000 7f6f1ba05740 1 Processor -- start 2024-09-11T22:03:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.588+0000 7f6f1ba05740 1 -- start start 2024-09-11T22:03:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.588+0000 7f6f1ba05740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5609541819b0 0x560954181d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.588+0000 7f6f1ba05740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5609541822d0 0x560954190b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.588+0000 7f6f1ba05740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5609541910c0 0x5609541934b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.588+0000 7f6f1ba05740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5609540e62f0 con 0x5609541819b0 2024-09-11T22:03:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.588+0000 7f6f1ba05740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56095408e270 con 0x5609541822d0 2024-09-11T22:03:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.588+0000 7f6f1ba05740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5609540e22b0 con 0x5609541910c0 2024-09-11T22:03:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.588+0000 7f6f12d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5609541819b0 0x560954181d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.588+0000 7f6f12d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5609541819b0 0x560954181d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59332/0 (socket says 172.21.15.1:59332) 2024-09-11T22:03:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.588+0000 7f6f12d9d640 1 -- 172.21.15.1:0/33985177 learned_addr learned my addr 172.21.15.1:0/33985177 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:03:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.588+0000 7f6f1259c640 1 --2- 172.21.15.1:0/33985177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5609541822d0 0x560954190b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.588+0000 7f6f12d9d640 1 -- 172.21.15.1:0/33985177 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5609541910c0 msgr2=0x5609541934b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.588+0000 7f6f12d9d640 1 --2- 172.21.15.1:0/33985177 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5609541910c0 0x5609541934b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.588+0000 7f6f12d9d640 1 -- 172.21.15.1:0/33985177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5609541822d0 msgr2=0x560954190b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.588+0000 7f6f12d9d640 1 --2- 172.21.15.1:0/33985177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5609541822d0 0x560954190b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.589+0000 7f6f12d9d640 1 -- 172.21.15.1:0/33985177 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56095408d340 con 0x5609541819b0 2024-09-11T22:03:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.589+0000 7f6f037fe640 1 --2- 172.21.15.1:0/1503181939 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f6efc0a2d70 0x7f6efc0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.589+0000 7f6f12d9d640 1 --2- 172.21.15.1:0/33985177 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5609541819b0 0x560954181d90 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c009ca0 tx=0x7f6f0c009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b04f1652ef4baf08 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.589+0000 7f6f1259c640 1 --2- 172.21.15.1:0/1503181939 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f6efc0a2d70 0x7f6efc0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.589+0000 7f6f037fe640 1 -- 172.21.15.1:0/1503181939 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f6f040cedf0 con 0x7f6f141009a0 2024-09-11T22:03:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.589+0000 7f6f1259c640 1 --2- 172.21.15.1:0/1503181939 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f6efc0a2d70 0x7f6efc0a5230 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6f040cc5b0 tx=0x7f6f040bb040 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.590+0000 7f6f00ff9640 1 -- 172.21.15.1:0/33985177 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c03f070 con 0x5609541819b0 2024-09-11T22:03:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.590+0000 7f6f00ff9640 1 -- 172.21.15.1:0/33985177 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f0c036040 con 0x5609541819b0 2024-09-11T22:03:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.590+0000 7f6f00ff9640 1 -- 172.21.15.1:0/33985177 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c004330 con 0x5609541819b0 2024-09-11T22:03:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.590+0000 7f6f1ba05740 1 -- 172.21.15.1:0/33985177 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5609541819b0 msgr2=0x560954181d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.590+0000 7f6f1ba05740 1 --2- 172.21.15.1:0/33985177 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5609541819b0 0x560954181d90 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c009ca0 tx=0x7f6f0c009a20 comp rx=0 tx=0).stop 2024-09-11T22:03:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.591+0000 7f6f1ba05740 1 -- 172.21.15.1:0/33985177 shutdown_connections 2024-09-11T22:03:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.591+0000 7f6f1ba05740 1 --2- 172.21.15.1:0/33985177 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5609541910c0 0x5609541934b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.591+0000 7f6f1ba05740 1 --2- 172.21.15.1:0/33985177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5609541822d0 0x560954190b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.591+0000 7f6f1ba05740 1 --2- 172.21.15.1:0/33985177 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5609541819b0 0x560954181d90 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.591+0000 7f6f1ba05740 1 -- 172.21.15.1:0/33985177 >> 172.21.15.1:0/33985177 conn(0x56095417e060 msgr2=0x56095417e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.591+0000 7f6f1ba05740 1 -- 172.21.15.1:0/33985177 shutdown_connections 2024-09-11T22:03:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.591+0000 7f6f1ba05740 1 -- 172.21.15.1:0/33985177 wait complete. 2024-09-11T22:03:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.591+0000 7f6f1ba05740 1 -- start start 2024-09-11T22:03:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.591+0000 7f6f1ba05740 1 Processor -- start 2024-09-11T22:03:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.591+0000 7f6f1ba05740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5609541819b0 0x56095418e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.591+0000 7f6f1ba05740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5609541822d0 0x56095418e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.591+0000 7f6f1ba05740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5609541910c0 0x56095418eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.592+0000 7f6f1ba05740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x560954193c30 con 0x5609541822d0 2024-09-11T22:03:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.592+0000 7f6f1ba05740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56095406f290 con 0x5609541910c0 2024-09-11T22:03:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.592+0000 7f6f1ba05740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x560954193db0 con 0x5609541819b0 2024-09-11T22:03:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.592+0000 7f6f1ba05740 10 client.0 _ll_register_callbacks cb 0x5609540a0140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:03:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.592+0000 7f6f11d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5609541822d0 0x56095418e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.592+0000 7f6f11d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5609541822d0 0x56095418e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59348/0 (socket says 172.21.15.1:59348) 2024-09-11T22:03:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.592+0000 7f6f11d9b640 1 -- 172.21.15.1:0/2230088171 learned_addr learned my addr 172.21.15.1:0/2230088171 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:03:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.592+0000 7f6f12d9d640 1 --2- 172.21.15.1:0/2230088171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5609541819b0 0x56095418e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.592+0000 7f6f12d9d640 1 --2- 172.21.15.1:0/2230088171 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5609541910c0 0x56095418eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.592+0000 7f6f11d9b640 1 -- 172.21.15.1:0/2230088171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5609541819b0 msgr2=0x56095418e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.592+0000 7f6f11d9b640 1 --2- 172.21.15.1:0/2230088171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5609541819b0 0x56095418e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.592+0000 7f6f11d9b640 1 -- 172.21.15.1:0/2230088171 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5609541910c0 msgr2=0x56095418eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.592+0000 7f6f11d9b640 1 --2- 172.21.15.1:0/2230088171 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5609541910c0 0x56095418eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.592+0000 7f6f11d9b640 1 -- 172.21.15.1:0/2230088171 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560953df9210 con 0x5609541822d0 2024-09-11T22:03:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.592+0000 7f6f11d9b640 1 --2- 172.21.15.1:0/2230088171 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5609541822d0 0x56095418e650 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f6f080119c0 tx=0x7f6f08011e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21070adea699a4fe server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.592+0000 7f6ef9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:03:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.592+0000 7f6ef9ffb640 1 -- 172.21.15.1:0/2230088171 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f0800fbb0 con 0x5609541822d0 2024-09-11T22:03:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.592+0000 7f6ef9ffb640 1 -- 172.21.15.1:0/2230088171 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f6f0800fd50 con 0x5609541822d0 2024-09-11T22:03:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.593+0000 7f6f1ba05740 1 -- 172.21.15.1:0/2230088171 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f0c036040 con 0x5609541822d0 2024-09-11T22:03:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.593+0000 7f6ef9ffb640 1 -- 172.21.15.1:0/2230088171 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f6f08019410 con 0x5609541822d0 2024-09-11T22:03:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.593+0000 7f6f1ba05740 10 client.16773 fetch_fsmap learned FSMap version 42 2024-09-11T22:03:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.593+0000 7f6f1ba05740 1 -- 172.21.15.1:0/2230088171 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x560953ec5620 con 0x5609541822d0 2024-09-11T22:03:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.593+0000 7f6ef9ffb640 1 -- 172.21.15.1:0/2230088171 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f0c036040 con 0x5609541822d0 2024-09-11T22:03:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.593+0000 7f6ef9ffb640 1 -- 172.21.15.1:0/2230088171 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f6f080226a0 con 0x5609541822d0 2024-09-11T22:03:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.593+0000 7f6f1ba05740 10 client.16773 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:03:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.593+0000 7f6f1ba05740 10 client.16773 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:03:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.593+0000 7f6f1ba05740 20 client.16773 populate_metadata read hostname 'smithi001' 2024-09-11T22:03:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.593+0000 7f6f1ba05740 1 --2- 172.21.15.1:0/2230088171 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5609541a1250 0x5609541a3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.593+0000 7f6f1ba05740 4 client.16773 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:03:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.593+0000 7f6f1ba05740 1 -- 172.21.15.1:0/2230088171 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5609540f3a20 con 0x5609541a1250 2024-09-11T22:03:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.594+0000 7f6f12d9d640 1 --2- 172.21.15.1:0/2230088171 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5609541a1250 0x5609541a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.594+0000 7f6ef9ffb640 10 client.16773 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:03:30.602 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.594+0000 7f6f12d9d640 1 --2- 172.21.15.1:0/2230088171 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5609541a1250 0x5609541a3670 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9025ec0192bd5c56 server_cookie=71c020ccb253dd89 in_seq=0 out_seq=0 2024-09-11T22:03:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.623+0000 7f6ef9ffb640 1 -- 172.21.15.1:0/2230088171 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5609540f3a20 con 0x5609541a1250 2024-09-11T22:03:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.623+0000 7f6ef9ffb640 10 client.16773 handle_command_reply: tid=0 2024-09-11T22:03:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.623+0000 7f6f1ba05740 1 client.16773 shutdown 2024-09-11T22:03:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.623+0000 7f6f1ba05740 1 -- 172.21.15.1:0/2230088171 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5609541822d0 msgr2=0x56095418e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.623+0000 7f6f1ba05740 1 --2- 172.21.15.1:0/2230088171 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5609541822d0 0x56095418e650 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f6f080119c0 tx=0x7f6f08011e80 comp rx=0 tx=0).stop 2024-09-11T22:03:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.623+0000 7f6f1ba05740 1 -- 172.21.15.1:0/2230088171 shutdown_connections 2024-09-11T22:03:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.623+0000 7f6f1ba05740 1 --2- 172.21.15.1:0/2230088171 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5609541a1250 0x5609541a3670 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.623+0000 7f6f1ba05740 1 --2- 172.21.15.1:0/2230088171 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5609541910c0 0x56095418eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.623+0000 7f6f1ba05740 1 --2- 172.21.15.1:0/2230088171 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5609541822d0 0x56095418e650 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.623+0000 7f6f1ba05740 1 --2- 172.21.15.1:0/2230088171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5609541819b0 0x56095418e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.623+0000 7f6ef9ffb640 0 client.16773 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:03:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.623+0000 7f6f1ba05740 1 -- 172.21.15.1:0/2230088171 >> 172.21.15.1:0/2230088171 conn(0x56095417e060 msgr2=0x56095417d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.624+0000 7f6f1ba05740 1 -- 172.21.15.1:0/2230088171 shutdown_connections 2024-09-11T22:03:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.624+0000 7f6f1ba05740 1 -- 172.21.15.1:0/2230088171 wait complete. 2024-09-11T22:03:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.624+0000 7f6f1ba05740 20 client.16773 trim_cache size 0 max 16384 2024-09-11T22:03:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.638+0000 7f6f1ba05740 1 Processor -- start 2024-09-11T22:03:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.638+0000 7f6f1ba05740 1 -- start start 2024-09-11T22:03:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.639+0000 7f6f1ba05740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5609541819b0 0x56095418c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.639+0000 7f6f1ba05740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5609541822d0 0x56095418e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.639+0000 7f6f1ba05740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5609541910c0 0x5609541a8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.639+0000 7f6f1ba05740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6f141ab810 con 0x5609541819b0 2024-09-11T22:03:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.639+0000 7f6f1ba05740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6f141ad650 con 0x5609541910c0 2024-09-11T22:03:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.639+0000 7f6f1ba05740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6f141abfa0 con 0x5609541822d0 2024-09-11T22:03:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.639+0000 7f6f12d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5609541819b0 0x56095418c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.639+0000 7f6f12d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5609541819b0 0x56095418c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59362/0 (socket says 172.21.15.1:59362) 2024-09-11T22:03:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.639+0000 7f6f12d9d640 1 -- 172.21.15.1:0/3516950128 learned_addr learned my addr 172.21.15.1:0/3516950128 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:03:30.641 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.639+0000 7f6f11d9b640 1 --2- 172.21.15.1:0/3516950128 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5609541822d0 0x56095418e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.639+0000 7f6f12d9d640 1 --2- 172.21.15.1:0/3516950128 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5609541910c0 0x5609541a8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.639+0000 7f6f12d9d640 1 -- 172.21.15.1:0/3516950128 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5609541822d0 msgr2=0x56095418e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.639+0000 7f6f12d9d640 1 --2- 172.21.15.1:0/3516950128 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5609541822d0 0x56095418e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.639+0000 7f6f12d9d640 1 -- 172.21.15.1:0/3516950128 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5609541910c0 msgr2=0x5609541a8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.639+0000 7f6f12d9d640 1 --2- 172.21.15.1:0/3516950128 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5609541910c0 0x5609541a8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.639+0000 7f6f12d9d640 1 -- 172.21.15.1:0/3516950128 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609541a3cd0 con 0x5609541819b0 2024-09-11T22:03:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.639+0000 7f6f12d9d640 1 --2- 172.21.15.1:0/3516950128 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5609541819b0 0x56095418c9c0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c009cd0 tx=0x7f6f0c009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb40c39c64d501b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.640+0000 7f6efbfff640 1 -- 172.21.15.1:0/3516950128 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c03f040 con 0x5609541819b0 2024-09-11T22:03:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.640+0000 7f6efbfff640 1 -- 172.21.15.1:0/3516950128 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f0c036070 con 0x5609541819b0 2024-09-11T22:03:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.640+0000 7f6efbfff640 1 -- 172.21.15.1:0/3516950128 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c004500 con 0x5609541819b0 2024-09-11T22:03:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.641+0000 7f6f1ba05740 1 -- 172.21.15.1:0/3516950128 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5609541819b0 msgr2=0x56095418c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.641+0000 7f6f1ba05740 1 --2- 172.21.15.1:0/3516950128 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5609541819b0 0x56095418c9c0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c009cd0 tx=0x7f6f0c009a20 comp rx=0 tx=0).stop 2024-09-11T22:03:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.641+0000 7f6f1ba05740 1 -- 172.21.15.1:0/3516950128 shutdown_connections 2024-09-11T22:03:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.641+0000 7f6f1ba05740 1 --2- 172.21.15.1:0/3516950128 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5609541910c0 0x5609541a8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.641+0000 7f6f1ba05740 1 --2- 172.21.15.1:0/3516950128 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5609541822d0 0x56095418e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.641+0000 7f6f1ba05740 1 --2- 172.21.15.1:0/3516950128 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5609541819b0 0x56095418c9c0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.641+0000 7f6f1ba05740 1 -- 172.21.15.1:0/3516950128 >> 172.21.15.1:0/3516950128 conn(0x56095417e060 msgr2=0x56095417d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.641+0000 7f6f1ba05740 1 -- 172.21.15.1:0/3516950128 shutdown_connections 2024-09-11T22:03:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.641+0000 7f6f1ba05740 1 -- 172.21.15.1:0/3516950128 wait complete. 2024-09-11T22:03:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.641+0000 7f6f1ba05740 1 -- start start 2024-09-11T22:03:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.641+0000 7f6f1ba05740 1 Processor -- start 2024-09-11T22:03:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.641+0000 7f6f1ba05740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5609541819b0 0x560954199580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.641+0000 7f6f1ba05740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5609541822d0 0x5609541948a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.641+0000 7f6f1ba05740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5609541910c0 0x560954194de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.641+0000 7f6f1ba05740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6f141acec0 con 0x5609541822d0 2024-09-11T22:03:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.641+0000 7f6f1ba05740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6f141ac730 con 0x5609541819b0 2024-09-11T22:03:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.641+0000 7f6f1ba05740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6f141a9810 con 0x5609541910c0 2024-09-11T22:03:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.641+0000 7f6f1ba05740 10 client.0 _ll_register_callbacks cb 0x7f6f08009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:03:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.642+0000 7f6f11d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5609541822d0 0x5609541948a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.642+0000 7f6f11d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5609541822d0 0x5609541948a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59374/0 (socket says 172.21.15.1:59374) 2024-09-11T22:03:30.648 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.642+0000 7f6f11d9b640 1 -- 172.21.15.1:0/166798106 learned_addr learned my addr 172.21.15.1:0/166798106 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:03:30.648 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.642+0000 7f6f12d9d640 1 --2- 172.21.15.1:0/166798106 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5609541910c0 0x560954194de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:30.648 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.642+0000 7f6f12d9d640 1 --2- 172.21.15.1:0/166798106 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5609541819b0 0x560954199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:30.648 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.642+0000 7f6f11d9b640 1 -- 172.21.15.1:0/166798106 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5609541910c0 msgr2=0x560954194de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:30.648 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.642+0000 7f6f11d9b640 1 --2- 172.21.15.1:0/166798106 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5609541910c0 0x560954194de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:30.648 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.642+0000 7f6f11d9b640 1 -- 172.21.15.1:0/166798106 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5609541819b0 msgr2=0x560954199580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:30.648 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.642+0000 7f6f11d9b640 1 --2- 172.21.15.1:0/166798106 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5609541819b0 0x560954199580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:30.648 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.642+0000 7f6f11d9b640 1 -- 172.21.15.1:0/166798106 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609541953a0 con 0x5609541822d0 2024-09-11T22:03:30.648 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.642+0000 7f6f12d9d640 1 --2- 172.21.15.1:0/166798106 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5609541819b0 0x560954199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:03:30.648 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.642+0000 7f6f11d9b640 1 --2- 172.21.15.1:0/166798106 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5609541822d0 0x5609541948a0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f6f08022cb0 tx=0x7f6f08011540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed6cdbd5d5f9d389 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:30.648 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.642+0000 7f6ef9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:03:30.648 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.642+0000 7f6ef9ffb640 1 -- 172.21.15.1:0/166798106 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f080225e0 con 0x5609541822d0 2024-09-11T22:03:30.649 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.642+0000 7f6ef9ffb640 1 -- 172.21.15.1:0/166798106 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f6f08022780 con 0x5609541822d0 2024-09-11T22:03:30.649 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.642+0000 7f6f1ba05740 1 -- 172.21.15.1:0/166798106 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f0c036070 con 0x5609541822d0 2024-09-11T22:03:30.649 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.642+0000 7f6ef9ffb640 1 -- 172.21.15.1:0/166798106 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f6f080050b0 con 0x5609541822d0 2024-09-11T22:03:30.649 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.642+0000 7f6f1ba05740 10 client.16779 fetch_fsmap learned FSMap version 42 2024-09-11T22:03:30.649 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.642+0000 7f6f1ba05740 1 -- 172.21.15.1:0/166798106 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x56095420bcb0 con 0x5609541822d0 2024-09-11T22:03:30.649 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.643+0000 7f6ef9ffb640 1 -- 172.21.15.1:0/166798106 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f0c036070 con 0x5609541822d0 2024-09-11T22:03:30.649 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.643+0000 7f6ef9ffb640 1 -- 172.21.15.1:0/166798106 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f6f08003bc0 con 0x5609541822d0 2024-09-11T22:03:30.649 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.643+0000 7f6f1ba05740 10 client.16779 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:03:30.649 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.643+0000 7f6f1ba05740 10 client.16779 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:03:30.649 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.643+0000 7f6f1ba05740 20 client.16779 populate_metadata read hostname 'smithi001' 2024-09-11T22:03:30.649 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.643+0000 7f6f1ba05740 1 --2- 172.21.15.1:0/166798106 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x56095420c520 0x56095420e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:30.649 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.643+0000 7f6f1ba05740 4 client.16779 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:03:30.650 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.643+0000 7f6f1ba05740 1 -- 172.21.15.1:0/166798106 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5609540edd70 con 0x56095420c520 2024-09-11T22:03:30.650 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.643+0000 7f6f12d9d640 1 --2- 172.21.15.1:0/166798106 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x56095420c520 0x56095420e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:30.650 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.644+0000 7f6ef9ffb640 10 client.16779 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:03:30.650 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:30.644+0000 7f6f12d9d640 1 --2- 172.21.15.1:0/166798106 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x56095420c520 0x56095420e940 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d7a6a67a9d659f4 server_cookie=2bd3365bbfdf3933 in_seq=0 out_seq=0 2024-09-11T22:03:31.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:31 smithi161 ceph-mon[35615]: pgmap v193: 97 pgs: 97 active+clean; 263 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 7.7 MiB/s wr, 897 op/s 2024-09-11T22:03:31.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:31 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:03:31.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:31 smithi001 ceph-mon[30935]: pgmap v193: 97 pgs: 97 active+clean; 263 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 7.7 MiB/s wr, 897 op/s 2024-09-11T22:03:31.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:31 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:03:31.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:31 smithi053 ceph-mon[34908]: pgmap v193: 97 pgs: 97 active+clean; 263 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 7.7 MiB/s wr, 897 op/s 2024-09-11T22:03:31.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:31 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:03:31.603 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:03:32.015 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:03:32.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:32 smithi161 ceph-mon[35615]: scrub queued for path: / 2024-09-11T22:03:32.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:03:32.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:03:32.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:03:32.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:03:32.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:32 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:03:32.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:03:32.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:32 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:03:32.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:32 smithi001 ceph-mon[30935]: scrub queued for path: / 2024-09-11T22:03:32.442 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:03:32.442 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:03:32.442 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:03:32.442 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:03:32.442 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:32 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:03:32.442 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:03:32.442 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:32 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:03:32.472 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:32.468+0000 7f6ef9ffb640 1 -- 172.21.15.1:0/166798106 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5609540edd70 con 0x56095420c520 2024-09-11T22:03:32.472 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:32.468+0000 7f6ef9ffb640 10 client.16779 handle_command_reply: tid=0 2024-09-11T22:03:32.472 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:32.468+0000 7f6f1ba05740 1 client.16779 shutdown 2024-09-11T22:03:32.472 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:32.468+0000 7f6f1ba05740 1 -- 172.21.15.1:0/166798106 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5609541822d0 msgr2=0x5609541948a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:32.472 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:32.468+0000 7f6f1ba05740 1 --2- 172.21.15.1:0/166798106 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5609541822d0 0x5609541948a0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f6f08022cb0 tx=0x7f6f08011540 comp rx=0 tx=0).stop 2024-09-11T22:03:32.473 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:32.471+0000 7f6f1ba05740 1 -- 172.21.15.1:0/166798106 shutdown_connections 2024-09-11T22:03:32.473 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:32.471+0000 7f6f1ba05740 1 --2- 172.21.15.1:0/166798106 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x56095420c520 0x56095420e940 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:32.473 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:32.471+0000 7f6f1ba05740 1 --2- 172.21.15.1:0/166798106 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5609541910c0 0x560954194de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:32.473 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:32.471+0000 7f6f1ba05740 1 --2- 172.21.15.1:0/166798106 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5609541822d0 0x5609541948a0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:32.473 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:32.471+0000 7f6f1ba05740 1 --2- 172.21.15.1:0/166798106 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5609541819b0 0x560954199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:32.473 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:32.471+0000 7f6f1ba05740 1 -- 172.21.15.1:0/166798106 >> 172.21.15.1:0/166798106 conn(0x56095417e060 msgr2=0x56095419bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:32.473 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:32.471+0000 7f6ef9ffb640 0 client.16779 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:03:32.473 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:32.471+0000 7f6f1ba05740 1 -- 172.21.15.1:0/166798106 shutdown_connections 2024-09-11T22:03:32.473 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:32.472+0000 7f6f1ba05740 1 -- 172.21.15.1:0/166798106 wait complete. 2024-09-11T22:03:32.473 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:32.472+0000 7f6f1ba05740 20 client.16779 trim_cache size 0 max 16384 2024-09-11T22:03:32.474 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-09-11T22:03:32.474 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-09-11T22:03:32.474 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-09-11T22:03:32.474 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-09-11T22:03:32.475 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:32.475+0000 7f6f18c65640 1 -- 172.21.15.1:0/1503181939 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f6efc0a2d70 msgr2=0x7f6efc0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:32.475 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:32.475+0000 7f6f18c65640 1 --2- 172.21.15.1:0/1503181939 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f6efc0a2d70 0x7f6efc0a5230 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6f040cc5b0 tx=0x7f6f040bb040 comp rx=0 tx=0).stop 2024-09-11T22:03:32.475 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:32.475+0000 7f6f18c65640 1 -- 172.21.15.1:0/1503181939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f141009a0 msgr2=0x7f6f141983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:32.476 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:32.475+0000 7f6f18c65640 1 --2- 172.21.15.1:0/1503181939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f141009a0 0x7f6f141983c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6f0400d770 tx=0x7f6f0400dc30 comp rx=0 tx=0).stop 2024-09-11T22:03:32.477 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:32.475+0000 7f6f18c65640 1 -- 172.21.15.1:0/1503181939 shutdown_connections 2024-09-11T22:03:32.477 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:32.475+0000 7f6f18c65640 1 --2- 172.21.15.1:0/1503181939 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f6efc0a2d70 0x7f6efc0a5230 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:32.477 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:32.475+0000 7f6f18c65640 1 --2- 172.21.15.1:0/1503181939 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6f141069b0 0x7f6f1419cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:32.477 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:32.475+0000 7f6f18c65640 1 --2- 172.21.15.1:0/1503181939 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f14101340 0x7f6f14198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:32.477 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:32.475+0000 7f6f18c65640 1 --2- 172.21.15.1:0/1503181939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f141009a0 0x7f6f141983c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:32.477 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:32.475+0000 7f6f18c65640 1 -- 172.21.15.1:0/1503181939 >> 172.21.15.1:0/1503181939 conn(0x7f6f140779a0 msgr2=0x7f6f14103770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:32.477 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:32.475+0000 7f6f18c65640 1 -- 172.21.15.1:0/1503181939 shutdown_connections 2024-09-11T22:03:32.477 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:32.475+0000 7f6f18c65640 1 -- 172.21.15.1:0/1503181939 wait complete. 2024-09-11T22:03:32.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:32 smithi053 ceph-mon[34908]: scrub queued for path: / 2024-09-11T22:03:32.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:03:32.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:03:32.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:03:32.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:03:32.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:32 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:03:32.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:03:32.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:32 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:03:32.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.573+0000 7f11c08ca640 1 -- 172.21.15.1:0/478144877 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11bc103ff0 msgr2=0x7f11bc1043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:32.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.573+0000 7f11c08ca640 1 --2- 172.21.15.1:0/478144877 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11bc103ff0 0x7f11bc1043d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f11ac009900 tx=0x7f11ac02a3f0 comp rx=0 tx=0).stop 2024-09-11T22:03:32.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.573+0000 7f11c08ca640 1 -- 172.21.15.1:0/478144877 shutdown_connections 2024-09-11T22:03:32.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.573+0000 7f11c08ca640 1 --2- 172.21.15.1:0/478144877 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11bc109120 0x7f11bc110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:32.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.573+0000 7f11c08ca640 1 --2- 172.21.15.1:0/478144877 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f11bc1049a0 0x7f11bc1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:32.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.573+0000 7f11c08ca640 1 --2- 172.21.15.1:0/478144877 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11bc103ff0 0x7f11bc1043d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:32.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.573+0000 7f11c08ca640 1 -- 172.21.15.1:0/478144877 >> 172.21.15.1:0/478144877 conn(0x7f11bc0ffab0 msgr2=0x7f11bc101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:32.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.573+0000 7f11c08ca640 1 -- 172.21.15.1:0/478144877 shutdown_connections 2024-09-11T22:03:32.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.574+0000 7f11c08ca640 1 -- 172.21.15.1:0/478144877 wait complete. 2024-09-11T22:03:32.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.574+0000 7f11c08ca640 1 Processor -- start 2024-09-11T22:03:32.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.574+0000 7f11c08ca640 1 -- start start 2024-09-11T22:03:32.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.575+0000 7f11c08ca640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11bc103ff0 0x7f11bc19a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:32.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.575+0000 7f11c08ca640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f11bc1049a0 0x7f11bc19abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:32.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.575+0000 7f11c08ca640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11bc109120 0x7f11bc19ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:32.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.575+0000 7f11c08ca640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f11bc114170 con 0x7f11bc1049a0 2024-09-11T22:03:32.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.575+0000 7f11c08ca640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f11bc113ff0 con 0x7f11bc103ff0 2024-09-11T22:03:32.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.575+0000 7f11c08ca640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f11bc1142f0 con 0x7f11bc109120 2024-09-11T22:03:32.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.575+0000 7f11ba59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11bc103ff0 0x7f11bc19a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:32.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.575+0000 7f11ba59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11bc103ff0 0x7f11bc19a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:54582/0 (socket says 172.21.15.1:54582) 2024-09-11T22:03:32.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.575+0000 7f11ba59c640 1 -- 172.21.15.1:0/2159226158 learned_addr learned my addr 172.21.15.1:0/2159226158 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:03:32.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.575+0000 7f11b9d9b640 1 --2- 172.21.15.1:0/2159226158 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f11bc1049a0 0x7f11bc19abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:32.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.575+0000 7f11bad9d640 1 --2- 172.21.15.1:0/2159226158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11bc109120 0x7f11bc19ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:32.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.575+0000 7f11ba59c640 1 -- 172.21.15.1:0/2159226158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11bc109120 msgr2=0x7f11bc19ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:32.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.575+0000 7f11ba59c640 1 --2- 172.21.15.1:0/2159226158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11bc109120 0x7f11bc19ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:32.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.575+0000 7f11ba59c640 1 -- 172.21.15.1:0/2159226158 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f11bc1049a0 msgr2=0x7f11bc19abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:32.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.575+0000 7f11ba59c640 1 --2- 172.21.15.1:0/2159226158 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f11bc1049a0 0x7f11bc19abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:32.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.575+0000 7f11ba59c640 1 -- 172.21.15.1:0/2159226158 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11bc19f640 con 0x7f11bc103ff0 2024-09-11T22:03:32.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.576+0000 7f11ba59c640 1 --2- 172.21.15.1:0/2159226158 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11bc103ff0 0x7f11bc19a690 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f11ac004770 tx=0x7f11ac003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:32.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.576+0000 7f11ab7fe640 1 -- 172.21.15.1:0/2159226158 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ac03f070 con 0x7f11bc103ff0 2024-09-11T22:03:32.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.576+0000 7f11c08ca640 1 -- 172.21.15.1:0/2159226158 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11bc19f8d0 con 0x7f11bc103ff0 2024-09-11T22:03:32.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.576+0000 7f11ab7fe640 1 -- 172.21.15.1:0/2159226158 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11ac031ca0 con 0x7f11bc103ff0 2024-09-11T22:03:32.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.576+0000 7f11c08ca640 1 -- 172.21.15.1:0/2159226158 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11bc1a7210 con 0x7f11bc103ff0 2024-09-11T22:03:32.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.576+0000 7f11ab7fe640 1 -- 172.21.15.1:0/2159226158 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ac035030 con 0x7f11bc103ff0 2024-09-11T22:03:32.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.579+0000 7f11ab7fe640 1 -- 172.21.15.1:0/2159226158 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f11ac042070 con 0x7f11bc103ff0 2024-09-11T22:03:32.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.579+0000 7f11c2ca3740 1 Processor -- start 2024-09-11T22:03:32.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.580+0000 7f11c2ca3740 1 -- start start 2024-09-11T22:03:32.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.581+0000 7f11c2ca3740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ee1df84a0 0x558ee1df8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:32.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.581+0000 7f11c2ca3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ee1df8dc0 0x558ee1e03140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:32.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.581+0000 7f11c2ca3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ee1e03680 0x558ee1e05a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:32.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.581+0000 7f11c2ca3740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x558ee1d5e2e0 con 0x558ee1df84a0 2024-09-11T22:03:32.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.581+0000 7f11c2ca3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x558ee1d05ce0 con 0x558ee1e03680 2024-09-11T22:03:32.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.581+0000 7f11c2ca3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x558ee1d5a2a0 con 0x558ee1df8dc0 2024-09-11T22:03:32.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.581+0000 7f11ab7fe640 1 --2- 172.21.15.1:0/2159226158 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f11a40a2df0 0x7f11a40a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:32.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.581+0000 7f11ba59c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ee1df8dc0 0x558ee1e03140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:32.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.581+0000 7f11b9d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ee1e03680 0x558ee1e05a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:32.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.581+0000 7f11bad9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ee1df84a0 0x558ee1df8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:32.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.581+0000 7f11b9d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ee1e03680 0x558ee1e05a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:54586/0 (socket says 172.21.15.1:54586) 2024-09-11T22:03:32.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.581+0000 7f11b9d9b640 1 -- 172.21.15.1:0/2002033108 learned_addr learned my addr 172.21.15.1:0/2002033108 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:03:32.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.581+0000 7f11ab7fe640 1 -- 172.21.15.1:0/2159226158 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f11ac0efb40 con 0x7f11bc103ff0 2024-09-11T22:03:32.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.581+0000 7f11bad9d640 1 --2- 172.21.15.1:0/2159226158 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f11a40a2df0 0x7f11a40a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:32.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.582+0000 7f11bad9d640 1 -- 172.21.15.1:0/2002033108 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ee1df8dc0 msgr2=0x558ee1e03140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:32.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.582+0000 7f11bad9d640 1 --2- 172.21.15.1:0/2002033108 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ee1df8dc0 0x558ee1e03140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:32.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.582+0000 7f11bad9d640 1 -- 172.21.15.1:0/2002033108 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ee1e03680 msgr2=0x558ee1e05a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:32.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.582+0000 7f11bad9d640 1 --2- 172.21.15.1:0/2002033108 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ee1e03680 0x558ee1e05a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:32.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.582+0000 7f11bad9d640 1 -- 172.21.15.1:0/2002033108 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ee1cea440 con 0x558ee1df84a0 2024-09-11T22:03:32.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.582+0000 7f11b9d9b640 1 --2- 172.21.15.1:0/2002033108 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ee1e03680 0x558ee1e05a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:03:32.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.582+0000 7f11bad9d640 1 --2- 172.21.15.1:0/2002033108 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ee1df84a0 0x558ee1df8880 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f11b4005fd0 tx=0x7f11b4010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe33d06cfea5dd64 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:32.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.582+0000 7f11bad9d640 1 --2- 172.21.15.1:0/2159226158 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f11a40a2df0 0x7f11a40a52b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f11b4010c40 tx=0x7f11b400c000 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:32.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.582+0000 7f11a8ff9640 1 -- 172.21.15.1:0/2002033108 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b4024070 con 0x558ee1df84a0 2024-09-11T22:03:32.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.582+0000 7f11a8ff9640 1 -- 172.21.15.1:0/2002033108 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11b4009c40 con 0x558ee1df84a0 2024-09-11T22:03:32.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.582+0000 7f11a8ff9640 1 -- 172.21.15.1:0/2002033108 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b4005280 con 0x558ee1df84a0 2024-09-11T22:03:32.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.583+0000 7f11c2ca3740 1 -- 172.21.15.1:0/2002033108 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ee1df84a0 msgr2=0x558ee1df8880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:32.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.583+0000 7f11c2ca3740 1 --2- 172.21.15.1:0/2002033108 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ee1df84a0 0x558ee1df8880 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f11b4005fd0 tx=0x7f11b4010740 comp rx=0 tx=0).stop 2024-09-11T22:03:32.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.583+0000 7f11c2ca3740 1 -- 172.21.15.1:0/2002033108 shutdown_connections 2024-09-11T22:03:32.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.583+0000 7f11c2ca3740 1 --2- 172.21.15.1:0/2002033108 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ee1e03680 0x558ee1e05a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:32.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.583+0000 7f11c2ca3740 1 --2- 172.21.15.1:0/2002033108 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ee1df8dc0 0x558ee1e03140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:32.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.583+0000 7f11c2ca3740 1 --2- 172.21.15.1:0/2002033108 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ee1df84a0 0x558ee1df8880 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:32.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.583+0000 7f11c2ca3740 1 -- 172.21.15.1:0/2002033108 >> 172.21.15.1:0/2002033108 conn(0x558ee1df4a30 msgr2=0x558ee1df4e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:32.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.583+0000 7f11c2ca3740 1 -- 172.21.15.1:0/2002033108 shutdown_connections 2024-09-11T22:03:32.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.583+0000 7f11c2ca3740 1 -- 172.21.15.1:0/2002033108 wait complete. 2024-09-11T22:03:32.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.584+0000 7f11c2ca3740 1 -- start start 2024-09-11T22:03:32.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.584+0000 7f11c2ca3740 1 Processor -- start 2024-09-11T22:03:32.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.584+0000 7f11c2ca3740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ee1df84a0 0x558ee1e02fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:32.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.584+0000 7f11c2ca3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ee1df8dc0 0x558ee1e00270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:32.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.584+0000 7f11c2ca3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ee1e03680 0x558ee1e007b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:32.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.584+0000 7f11c2ca3740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x558ee1e06250 con 0x558ee1df84a0 2024-09-11T22:03:32.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.584+0000 7f11c2ca3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x558ee1d19050 con 0x558ee1df8dc0 2024-09-11T22:03:32.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.584+0000 7f11c2ca3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x558ee1e063d0 con 0x558ee1e03680 2024-09-11T22:03:32.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.584+0000 7f11c2ca3740 10 client.0 _ll_register_callbacks cb 0x558ee1d59160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:03:32.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.584+0000 7f11b9d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ee1df8dc0 0x558ee1e00270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:32.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.584+0000 7f11b9d9b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ee1df8dc0 0x558ee1e00270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:54594/0 (socket says 172.21.15.1:54594) 2024-09-11T22:03:32.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.584+0000 7f11b9d9b640 1 -- 172.21.15.1:0/2502571072 learned_addr learned my addr 172.21.15.1:0/2502571072 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:03:32.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.584+0000 7f11ba59c640 1 --2- 172.21.15.1:0/2502571072 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ee1df84a0 0x558ee1e02fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:32.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.585+0000 7f11bad9d640 1 --2- 172.21.15.1:0/2502571072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ee1e03680 0x558ee1e007b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:32.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.585+0000 7f11b9d9b640 1 -- 172.21.15.1:0/2502571072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ee1e03680 msgr2=0x558ee1e007b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:32.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.585+0000 7f11b9d9b640 1 --2- 172.21.15.1:0/2502571072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ee1e03680 0x558ee1e007b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:32.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.585+0000 7f11b9d9b640 1 -- 172.21.15.1:0/2502571072 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ee1df84a0 msgr2=0x558ee1e02fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:32.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.585+0000 7f11b9d9b640 1 --2- 172.21.15.1:0/2502571072 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ee1df84a0 0x558ee1e02fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:32.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.585+0000 7f11b9d9b640 1 -- 172.21.15.1:0/2502571072 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ee1acf3c0 con 0x558ee1df8dc0 2024-09-11T22:03:32.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.585+0000 7f11b9d9b640 1 --2- 172.21.15.1:0/2502571072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ee1df8dc0 0x558ee1e00270 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f11b00126d0 tx=0x7f11b0012b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=600e3c2a0ce88cea server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:32.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.585+0000 7f11a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-11T22:03:32.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.585+0000 7f11a1ffb640 1 -- 172.21.15.1:0/2502571072 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b0004020 con 0x558ee1df8dc0 2024-09-11T22:03:32.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.585+0000 7f11a1ffb640 1 -- 172.21.15.1:0/2502571072 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f11b00041c0 con 0x558ee1df8dc0 2024-09-11T22:03:32.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.585+0000 7f11c2ca3740 1 -- 172.21.15.1:0/2502571072 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11b4009c40 con 0x558ee1df8dc0 2024-09-11T22:03:32.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.586+0000 7f11a1ffb640 1 -- 172.21.15.1:0/2502571072 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f11b00043c0 con 0x558ee1df8dc0 2024-09-11T22:03:32.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.586+0000 7f11c2ca3740 10 client.26327 fetch_fsmap learned FSMap version 42 2024-09-11T22:03:32.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.586+0000 7f11c2ca3740 1 -- 172.21.15.1:0/2502571072 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x558ee1e11c90 con 0x558ee1df8dc0 2024-09-11T22:03:32.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.586+0000 7f11a1ffb640 1 -- 172.21.15.1:0/2502571072 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11b4009c40 con 0x558ee1df8dc0 2024-09-11T22:03:32.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.586+0000 7f11a1ffb640 1 -- 172.21.15.1:0/2502571072 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f11b0003b20 con 0x558ee1df8dc0 2024-09-11T22:03:32.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.586+0000 7f11c2ca3740 10 client.26327 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:03:32.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.586+0000 7f11c2ca3740 10 client.26327 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:03:32.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.586+0000 7f11c2ca3740 20 client.26327 populate_metadata read hostname 'smithi001' 2024-09-11T22:03:32.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.586+0000 7f11c2ca3740 1 --2- 172.21.15.1:0/2502571072 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x558ee1e131d0 0x558ee1e15610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:32.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.586+0000 7f11c2ca3740 4 client.26327 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:03:32.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.586+0000 7f11c2ca3740 1 -- 172.21.15.1:0/2502571072 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ee1d6b850 con 0x558ee1e131d0 2024-09-11T22:03:32.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.587+0000 7f11ba59c640 1 --2- 172.21.15.1:0/2502571072 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x558ee1e131d0 0x558ee1e15610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:32.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.587+0000 7f11a1ffb640 10 client.26327 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:03:32.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.587+0000 7f11ba59c640 1 --2- 172.21.15.1:0/2502571072 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x558ee1e131d0 0x558ee1e15610 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55421080738be4b2 server_cookie=bbcc064e3036cc04 in_seq=0 out_seq=0 2024-09-11T22:03:32.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.589+0000 7f11a1ffb640 1 -- 172.21.15.1:0/2502571072 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558ee1d6b850 con 0x558ee1e131d0 2024-09-11T22:03:32.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.589+0000 7f11a1ffb640 10 client.26327 handle_command_reply: tid=0 2024-09-11T22:03:32.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.589+0000 7f11c2ca3740 1 client.26327 shutdown 2024-09-11T22:03:32.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.590+0000 7f11c2ca3740 1 -- 172.21.15.1:0/2502571072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ee1df8dc0 msgr2=0x558ee1e00270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:32.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.590+0000 7f11c2ca3740 1 --2- 172.21.15.1:0/2502571072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ee1df8dc0 0x558ee1e00270 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f11b00126d0 tx=0x7f11b0012b90 comp rx=0 tx=0).stop 2024-09-11T22:03:32.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.590+0000 7f11c2ca3740 1 -- 172.21.15.1:0/2502571072 shutdown_connections 2024-09-11T22:03:32.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.590+0000 7f11c2ca3740 1 --2- 172.21.15.1:0/2502571072 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x558ee1e131d0 0x558ee1e15610 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:32.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.590+0000 7f11c2ca3740 1 --2- 172.21.15.1:0/2502571072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ee1e03680 0x558ee1e007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:32.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.590+0000 7f11c2ca3740 1 --2- 172.21.15.1:0/2502571072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ee1df8dc0 0x558ee1e00270 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:32.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.590+0000 7f11a1ffb640 0 client.26327 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:03:32.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.590+0000 7f11c2ca3740 1 --2- 172.21.15.1:0/2502571072 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ee1df84a0 0x558ee1e02fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:32.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.590+0000 7f11c2ca3740 1 -- 172.21.15.1:0/2502571072 >> 172.21.15.1:0/2502571072 conn(0x558ee1df4a30 msgr2=0x558ee1dfa710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:32.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.590+0000 7f11c2ca3740 1 -- 172.21.15.1:0/2502571072 shutdown_connections 2024-09-11T22:03:32.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.590+0000 7f11c2ca3740 1 -- 172.21.15.1:0/2502571072 wait complete. 2024-09-11T22:03:32.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.590+0000 7f11c2ca3740 20 client.26327 trim_cache size 0 max 16384 2024-09-11T22:03:32.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.611+0000 7f11c2ca3740 1 Processor -- start 2024-09-11T22:03:32.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.611+0000 7f11c2ca3740 1 -- start start 2024-09-11T22:03:32.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.611+0000 7f11c2ca3740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ee1df84a0 0x558ee1e0f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:32.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.611+0000 7f11c2ca3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ee1df8dc0 0x558ee1e1aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:32.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.611+0000 7f11c2ca3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ee1e03680 0x558ee1e1b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:32.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.611+0000 7f11c2ca3740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f11bc1afb40 con 0x558ee1df84a0 2024-09-11T22:03:32.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.611+0000 7f11c2ca3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f11bc1b1980 con 0x558ee1e03680 2024-09-11T22:03:32.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.611+0000 7f11c2ca3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f11bc1b02d0 con 0x558ee1df8dc0 2024-09-11T22:03:32.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.611+0000 7f11ba59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ee1df84a0 0x558ee1e0f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:32.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.611+0000 7f11ba59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ee1df84a0 0x558ee1e0f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59428/0 (socket says 172.21.15.1:59428) 2024-09-11T22:03:32.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.611+0000 7f11ba59c640 1 -- 172.21.15.1:0/3952335989 learned_addr learned my addr 172.21.15.1:0/3952335989 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:03:32.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.611+0000 7f11b9d9b640 1 --2- 172.21.15.1:0/3952335989 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ee1df8dc0 0x558ee1e1aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:32.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.612+0000 7f11ba59c640 1 -- 172.21.15.1:0/3952335989 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ee1df8dc0 msgr2=0x558ee1e1aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:32.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.612+0000 7f11ba59c640 1 --2- 172.21.15.1:0/3952335989 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ee1df8dc0 0x558ee1e1aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:32.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.612+0000 7f11ba59c640 1 -- 172.21.15.1:0/3952335989 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ee1e03680 msgr2=0x558ee1e1b420 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T22:03:32.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.612+0000 7f11ba59c640 1 --2- 172.21.15.1:0/3952335989 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ee1e03680 0x558ee1e1b420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:32.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.612+0000 7f11ba59c640 1 -- 172.21.15.1:0/3952335989 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ee1df8bf0 con 0x558ee1df84a0 2024-09-11T22:03:32.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.612+0000 7f11ba59c640 1 --2- 172.21.15.1:0/3952335989 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ee1df84a0 0x558ee1e0f4e0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f11ac0b84d0 tx=0x7f11ac03d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9ebac1557329135 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:32.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.612+0000 7f11a3fff640 1 -- 172.21.15.1:0/3952335989 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ac049040 con 0x558ee1df84a0 2024-09-11T22:03:32.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.612+0000 7f11a3fff640 1 -- 172.21.15.1:0/3952335989 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11ac0bc070 con 0x558ee1df84a0 2024-09-11T22:03:32.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.612+0000 7f11a3fff640 1 -- 172.21.15.1:0/3952335989 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ac0b86e0 con 0x558ee1df84a0 2024-09-11T22:03:32.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.613+0000 7f11c2ca3740 1 -- 172.21.15.1:0/3952335989 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ee1df84a0 msgr2=0x558ee1e0f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:32.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.613+0000 7f11c2ca3740 1 --2- 172.21.15.1:0/3952335989 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ee1df84a0 0x558ee1e0f4e0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f11ac0b84d0 tx=0x7f11ac03d040 comp rx=0 tx=0).stop 2024-09-11T22:03:32.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.613+0000 7f11c2ca3740 1 -- 172.21.15.1:0/3952335989 shutdown_connections 2024-09-11T22:03:32.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.613+0000 7f11c2ca3740 1 --2- 172.21.15.1:0/3952335989 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ee1e03680 0x558ee1e1b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:32.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.613+0000 7f11c2ca3740 1 --2- 172.21.15.1:0/3952335989 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ee1df8dc0 0x558ee1e1aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:32.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.613+0000 7f11c2ca3740 1 --2- 172.21.15.1:0/3952335989 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ee1df84a0 0x558ee1e0f4e0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:32.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.613+0000 7f11c2ca3740 1 -- 172.21.15.1:0/3952335989 >> 172.21.15.1:0/3952335989 conn(0x558ee1df4a30 msgr2=0x558ee1dff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:32.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.614+0000 7f11c2ca3740 1 -- 172.21.15.1:0/3952335989 shutdown_connections 2024-09-11T22:03:32.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.614+0000 7f11c2ca3740 1 -- 172.21.15.1:0/3952335989 wait complete. 2024-09-11T22:03:32.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.614+0000 7f11c2ca3740 1 -- start start 2024-09-11T22:03:32.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.614+0000 7f11c2ca3740 1 Processor -- start 2024-09-11T22:03:32.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.615+0000 7f11c2ca3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ee1df84a0 0x558ee1e0bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:32.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.615+0000 7f11c2ca3740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ee1df8dc0 0x558ee1e09020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:32.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.615+0000 7f11c2ca3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ee1e03680 0x558ee1e09560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:32.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.615+0000 7f11c2ca3740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f11bc1b11f0 con 0x558ee1df8dc0 2024-09-11T22:03:32.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.615+0000 7f11c2ca3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f11bc1b0a60 con 0x558ee1e03680 2024-09-11T22:03:32.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.615+0000 7f11c2ca3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f11bc1adb20 con 0x558ee1df84a0 2024-09-11T22:03:32.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.615+0000 7f11b9d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ee1df8dc0 0x558ee1e09020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:32.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.615+0000 7f11c2ca3740 10 client.0 _ll_register_callbacks cb 0x7f11b0009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:03:32.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.615+0000 7f11ba59c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ee1df84a0 0x558ee1e0bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:32.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.615+0000 7f11b9d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ee1df8dc0 0x558ee1e09020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59440/0 (socket says 172.21.15.1:59440) 2024-09-11T22:03:32.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.615+0000 7f11b9d9b640 1 -- 172.21.15.1:0/2939795625 learned_addr learned my addr 172.21.15.1:0/2939795625 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:03:32.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.615+0000 7f11bad9d640 1 --2- 172.21.15.1:0/2939795625 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ee1e03680 0x558ee1e09560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:32.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.615+0000 7f11ba59c640 1 -- 172.21.15.1:0/2939795625 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ee1e03680 msgr2=0x558ee1e09560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:32.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.615+0000 7f11ba59c640 1 --2- 172.21.15.1:0/2939795625 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ee1e03680 0x558ee1e09560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:32.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.615+0000 7f11ba59c640 1 -- 172.21.15.1:0/2939795625 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ee1df8dc0 msgr2=0x558ee1e09020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:32.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.615+0000 7f11ba59c640 1 --2- 172.21.15.1:0/2939795625 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ee1df8dc0 0x558ee1e09020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:32.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.615+0000 7f11ba59c640 1 -- 172.21.15.1:0/2939795625 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ee1e09c50 con 0x558ee1df84a0 2024-09-11T22:03:32.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.615+0000 7f11b9d9b640 1 --2- 172.21.15.1:0/2939795625 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ee1df8dc0 0x558ee1e09020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:03:32.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.615+0000 7f11ba59c640 1 --2- 172.21.15.1:0/2939795625 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ee1df84a0 0x558ee1e0bc80 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f11ac0c0530 tx=0x7f11ac03b460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=240862f53510a214 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:32.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.615+0000 7f11a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-11T22:03:32.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.616+0000 7f11a1ffb640 1 -- 172.21.15.1:0/2939795625 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ac049050 con 0x558ee1df84a0 2024-09-11T22:03:32.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.616+0000 7f11a1ffb640 1 -- 172.21.15.1:0/2939795625 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f11ac0b86e0 con 0x558ee1df84a0 2024-09-11T22:03:32.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.616+0000 7f11c2ca3740 1 -- 172.21.15.1:0/2939795625 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11ac0bc070 con 0x558ee1df84a0 2024-09-11T22:03:32.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.616+0000 7f11a1ffb640 1 -- 172.21.15.1:0/2939795625 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f11ac03b4f0 con 0x558ee1df84a0 2024-09-11T22:03:32.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.616+0000 7f11c2ca3740 10 client.26296 fetch_fsmap learned FSMap version 42 2024-09-11T22:03:32.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.616+0000 7f11c2ca3740 1 -- 172.21.15.1:0/2939795625 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x558ee1e865c0 con 0x558ee1df84a0 2024-09-11T22:03:32.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.617+0000 7f11a1ffb640 1 -- 172.21.15.1:0/2939795625 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11ac03b690 con 0x558ee1df84a0 2024-09-11T22:03:32.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.617+0000 7f11a1ffb640 1 -- 172.21.15.1:0/2939795625 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f11ac0ca030 con 0x558ee1df84a0 2024-09-11T22:03:32.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.617+0000 7f11c2ca3740 10 client.26296 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:03:32.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.617+0000 7f11c2ca3740 10 client.26296 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:03:32.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.617+0000 7f11c2ca3740 20 client.26296 populate_metadata read hostname 'smithi001' 2024-09-11T22:03:32.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.617+0000 7f11c2ca3740 1 --2- 172.21.15.1:0/2939795625 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x558ee1e131d0 0x558ee1e88d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:32.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.617+0000 7f11c2ca3740 4 client.26296 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:03:32.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.617+0000 7f11c2ca3740 1 -- 172.21.15.1:0/2939795625 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f11ac0bc070 con 0x558ee1e131d0 2024-09-11T22:03:32.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.617+0000 7f11b9d9b640 1 --2- 172.21.15.1:0/2939795625 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x558ee1e131d0 0x558ee1e88d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:32.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.618+0000 7f11b9d9b640 1 --2- 172.21.15.1:0/2939795625 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x558ee1e131d0 0x558ee1e88d50 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1578e63d12fbc21 server_cookie=4710c9f4a233cdab in_seq=0 out_seq=0 2024-09-11T22:03:32.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.618+0000 7f11a1ffb640 10 client.26296 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:03:32.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.620+0000 7f11a1ffb640 1 -- 172.21.15.1:0/2939795625 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x7f11ac0bc070 con 0x558ee1e131d0 2024-09-11T22:03:32.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.620+0000 7f11a1ffb640 10 client.26296 handle_command_reply: tid=0 2024-09-11T22:03:32.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.620+0000 7f11c2ca3740 1 client.26296 shutdown 2024-09-11T22:03:32.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.620+0000 7f11c2ca3740 1 -- 172.21.15.1:0/2939795625 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ee1df84a0 msgr2=0x558ee1e0bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:32.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.620+0000 7f11c2ca3740 1 --2- 172.21.15.1:0/2939795625 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ee1df84a0 0x558ee1e0bc80 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f11ac0c0530 tx=0x7f11ac03b460 comp rx=0 tx=0).stop 2024-09-11T22:03:32.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.620+0000 7f11c2ca3740 1 -- 172.21.15.1:0/2939795625 shutdown_connections 2024-09-11T22:03:32.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.620+0000 7f11c2ca3740 1 --2- 172.21.15.1:0/2939795625 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x558ee1e131d0 0x558ee1e88d50 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:32.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.620+0000 7f11c2ca3740 1 --2- 172.21.15.1:0/2939795625 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558ee1e03680 0x558ee1e09560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:32.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.620+0000 7f11c2ca3740 1 --2- 172.21.15.1:0/2939795625 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558ee1df8dc0 0x558ee1e09020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:32.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.620+0000 7f11c2ca3740 1 --2- 172.21.15.1:0/2939795625 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x558ee1df84a0 0x558ee1e0bc80 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:32.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.620+0000 7f11c2ca3740 1 -- 172.21.15.1:0/2939795625 >> 172.21.15.1:0/2939795625 conn(0x558ee1df4a30 msgr2=0x558ee1e01ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:32.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.620+0000 7f11a1ffb640 0 client.26296 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:03:32.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.621+0000 7f11c2ca3740 1 -- 172.21.15.1:0/2939795625 shutdown_connections 2024-09-11T22:03:32.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.621+0000 7f11c2ca3740 1 -- 172.21.15.1:0/2939795625 wait complete. 2024-09-11T22:03:32.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.621+0000 7f11c2ca3740 20 client.26296 trim_cache size 0 max 16384 2024-09-11T22:03:32.624 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-11T22:03:32.624 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "scrub active (3025 inodes in the stack)", 2024-09-11T22:03:32.624 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-09-11T22:03:32.624 INFO:teuthology.orchestra.run.smithi001.stdout: "4bb42c4e-eaef-4b43-9290-d831431460f1": { 2024-09-11T22:03:32.624 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2024-09-11T22:03:32.624 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "4bb42c4e-eaef-4b43-9290-d831431460f1", 2024-09-11T22:03:32.624 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2024-09-11T22:03:32.625 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:03:32.625 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:03:32.625 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-11T22:03:32.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.621+0000 7f11c08ca640 1 -- 172.21.15.1:0/2159226158 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f11a40a2df0 msgr2=0x7f11a40a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:32.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.621+0000 7f11c08ca640 1 --2- 172.21.15.1:0/2159226158 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f11a40a2df0 0x7f11a40a52b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f11b4010c40 tx=0x7f11b400c000 comp rx=0 tx=0).stop 2024-09-11T22:03:32.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.621+0000 7f11c08ca640 1 -- 172.21.15.1:0/2159226158 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11bc103ff0 msgr2=0x7f11bc19a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:32.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.621+0000 7f11c08ca640 1 --2- 172.21.15.1:0/2159226158 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11bc103ff0 0x7f11bc19a690 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f11ac004770 tx=0x7f11ac003c10 comp rx=0 tx=0).stop 2024-09-11T22:03:32.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.621+0000 7f11c08ca640 1 -- 172.21.15.1:0/2159226158 shutdown_connections 2024-09-11T22:03:32.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.621+0000 7f11c08ca640 1 --2- 172.21.15.1:0/2159226158 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f11a40a2df0 0x7f11a40a52b0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:32.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.621+0000 7f11c08ca640 1 --2- 172.21.15.1:0/2159226158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11bc109120 0x7f11bc19ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:32.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.621+0000 7f11c08ca640 1 --2- 172.21.15.1:0/2159226158 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f11bc1049a0 0x7f11bc19abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:32.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.621+0000 7f11c08ca640 1 --2- 172.21.15.1:0/2159226158 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11bc103ff0 0x7f11bc19a690 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:32.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.621+0000 7f11c08ca640 1 -- 172.21.15.1:0/2159226158 >> 172.21.15.1:0/2159226158 conn(0x7f11bc0ffab0 msgr2=0x7f11bc100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:32.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.622+0000 7f11c08ca640 1 -- 172.21.15.1:0/2159226158 shutdown_connections 2024-09-11T22:03:32.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:03:32.622+0000 7f11c08ca640 1 -- 172.21.15.1:0/2159226158 wait complete. 2024-09-11T22:03:33.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:33 smithi161 ceph-mon[35615]: pgmap v194: 97 pgs: 97 active+clean; 263 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.9 MiB/s wr, 390 op/s 2024-09-11T22:03:33.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:33 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:03:33.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:33 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:03:33.383 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:33 smithi001 ceph-mon[30935]: pgmap v194: 97 pgs: 97 active+clean; 263 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.9 MiB/s wr, 390 op/s 2024-09-11T22:03:33.383 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:33 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:03:33.383 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:33 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:03:33.424 INFO:tasks.cephfs.filesystem:scrub status for tag:4bb42c4e-eaef-4b43-9290-d831431460f1 - {'path': '/', 'tag': '4bb42c4e-eaef-4b43-9290-d831431460f1', 'options': 'recursive,force'} 2024-09-11T22:03:33.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:33 smithi053 ceph-mon[34908]: pgmap v194: 97 pgs: 97 active+clean; 263 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.9 MiB/s wr, 390 op/s 2024-09-11T22:03:33.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:33 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:03:33.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:33 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:03:34.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:34 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:03:34.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:34 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:03:34.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:34 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:03:34.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:34 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:03:34.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:34 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:03:34.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:34 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:03:35.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:35 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:03:35.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:35 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:03:35.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:35 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:03:35.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:35 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:03:35.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:35 smithi161 ceph-mon[35615]: pgmap v195: 97 pgs: 97 active+clean; 263 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.9 MiB/s wr, 390 op/s 2024-09-11T22:03:35.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:35 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:03:35.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:35 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:03:35.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:35 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:03:35.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:35 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:03:35.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:35 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:03:35.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:35 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:03:35.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:35 smithi001 ceph-mon[30935]: pgmap v195: 97 pgs: 97 active+clean; 263 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.9 MiB/s wr, 390 op/s 2024-09-11T22:03:35.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:35 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:03:35.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:35 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:03:35.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:35 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:03:35.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:35 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:03:35.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:35 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:03:35.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:35 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:03:35.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:35 smithi053 ceph-mon[34908]: pgmap v195: 97 pgs: 97 active+clean; 263 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.9 MiB/s wr, 390 op/s 2024-09-11T22:03:35.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:35 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:03:35.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:35 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:03:37.345 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:37 smithi161 ceph-mon[35615]: pgmap v196: 97 pgs: 97 active+clean; 262 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 7.4 MiB/s wr, 1.75k op/s 2024-09-11T22:03:37.345 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:37 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:03:37.345 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:37 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:03:37.345 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:37 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:03:37.345 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:37 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T22:03:37.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:37 smithi001 ceph-mon[30935]: pgmap v196: 97 pgs: 97 active+clean; 262 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 7.4 MiB/s wr, 1.75k op/s 2024-09-11T22:03:37.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:37 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:03:37.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:37 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:03:37.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:37 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:03:37.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:37 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T22:03:37.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:37 smithi053 ceph-mon[34908]: pgmap v196: 97 pgs: 97 active+clean; 262 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 7.4 MiB/s wr, 1.75k op/s 2024-09-11T22:03:37.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:37 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:03:37.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:37 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:03:37.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:37 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:03:37.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:37 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T22:03:39.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:39 smithi161 ceph-mon[35615]: pgmap v197: 97 pgs: 97 active+clean; 262 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 5.0 MiB/s wr, 1.57k op/s 2024-09-11T22:03:39.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:39 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:03:39.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:39 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:03:39.348 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:39 smithi053 ceph-mon[34908]: pgmap v197: 97 pgs: 97 active+clean; 262 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 5.0 MiB/s wr, 1.57k op/s 2024-09-11T22:03:39.348 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:39 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:03:39.349 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:39 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:03:39.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:39 smithi001 ceph-mon[30935]: pgmap v197: 97 pgs: 97 active+clean; 262 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 5.0 MiB/s wr, 1.57k op/s 2024-09-11T22:03:39.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:39 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:03:39.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:39 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:03:40.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:40 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:03:40.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:40 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:03:40.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:40 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:03:41.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:41 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:03:41.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:41 smithi161 ceph-mon[35615]: pgmap v198: 97 pgs: 97 active+clean; 284 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 6.8 MiB/s wr, 2.05k op/s 2024-09-11T22:03:41.654 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:41 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:03:41.654 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:41 smithi001 ceph-mon[30935]: pgmap v198: 97 pgs: 97 active+clean; 284 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 6.8 MiB/s wr, 2.05k op/s 2024-09-11T22:03:41.754 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:41 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:03:41.755 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:41 smithi053 ceph-mon[34908]: pgmap v198: 97 pgs: 97 active+clean; 284 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 6.8 MiB/s wr, 2.05k op/s 2024-09-11T22:03:42.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:42 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:03:42.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:42 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:03:42.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:42 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:03:43.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:43 smithi001 ceph-mon[30935]: pgmap v199: 97 pgs: 97 active+clean; 284 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 4.3 MiB/s wr, 1.84k op/s 2024-09-11T22:03:43.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:43 smithi053 ceph-mon[34908]: pgmap v199: 97 pgs: 97 active+clean; 284 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 4.3 MiB/s wr, 1.84k op/s 2024-09-11T22:03:43.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:43 smithi161 ceph-mon[35615]: pgmap v199: 97 pgs: 97 active+clean; 284 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 4.3 MiB/s wr, 1.84k op/s 2024-09-11T22:03:45.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:45 smithi001 ceph-mon[30935]: pgmap v200: 97 pgs: 97 active+clean; 284 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 4.3 MiB/s wr, 1.84k op/s 2024-09-11T22:03:45.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:45 smithi053 ceph-mon[34908]: pgmap v200: 97 pgs: 97 active+clean; 284 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 4.3 MiB/s wr, 1.84k op/s 2024-09-11T22:03:45.591 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.588+0000 7f9fce1cf640 1 -- 172.21.15.1:0/3269690899 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9fc8101ac0 msgr2=0x7f9fc8101f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:45.591 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.588+0000 7f9fce1cf640 1 --2- 172.21.15.1:0/3269690899 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9fc8101ac0 0x7f9fc8101f20 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc009930 tx=0x7f9fbc02a420 comp rx=0 tx=0).stop 2024-09-11T22:03:45.591 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.588+0000 7f9fce1cf640 1 -- 172.21.15.1:0/3269690899 shutdown_connections 2024-09-11T22:03:45.591 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.588+0000 7f9fce1cf640 1 --2- 172.21.15.1:0/3269690899 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9fc8102460 0x7f9fc810e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:45.591 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.588+0000 7f9fce1cf640 1 --2- 172.21.15.1:0/3269690899 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9fc8101ac0 0x7f9fc8101f20 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:45.591 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.588+0000 7f9fce1cf640 1 --2- 172.21.15.1:0/3269690899 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fc8107ad0 0x7f9fc8107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:45.592 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.588+0000 7f9fce1cf640 1 -- 172.21.15.1:0/3269690899 >> 172.21.15.1:0/3269690899 conn(0x7f9fc80fd7a0 msgr2=0x7f9fc80ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:45.592 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.588+0000 7f9fce1cf640 1 -- 172.21.15.1:0/3269690899 shutdown_connections 2024-09-11T22:03:45.592 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.588+0000 7f9fce1cf640 1 -- 172.21.15.1:0/3269690899 wait complete. 2024-09-11T22:03:45.592 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.589+0000 7f9fce1cf640 1 Processor -- start 2024-09-11T22:03:45.592 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.589+0000 7f9fce1cf640 1 -- start start 2024-09-11T22:03:45.592 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.589+0000 7f9fce1cf640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9fc8101ac0 0x7f9fc8198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:45.592 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.589+0000 7f9fce1cf640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fc8102460 0x7f9fc8198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:45.592 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.589+0000 7f9fce1cf640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9fc8107ad0 0x7f9fc819cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:45.592 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.589+0000 7f9fce1cf640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc8111f10 con 0x7f9fc8101ac0 2024-09-11T22:03:45.592 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.589+0000 7f9fce1cf640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc8111d90 con 0x7f9fc8102460 2024-09-11T22:03:45.592 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.589+0000 7f9fce1cf640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc8112090 con 0x7f9fc8107ad0 2024-09-11T22:03:45.592 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.589+0000 7f9fc77fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9fc8101ac0 0x7f9fc8198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:45.593 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.589+0000 7f9fc6ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fc8102460 0x7f9fc8198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:45.593 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.589+0000 7f9fc77fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9fc8101ac0 0x7f9fc8198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58646/0 (socket says 172.21.15.1:58646) 2024-09-11T22:03:45.593 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.589+0000 7f9fc77fe640 1 -- 172.21.15.1:0/321690865 learned_addr learned my addr 172.21.15.1:0/321690865 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:03:45.593 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.589+0000 7f9fc7fff640 1 --2- 172.21.15.1:0/321690865 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9fc8107ad0 0x7f9fc819cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:45.593 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.590+0000 7f9fc77fe640 1 -- 172.21.15.1:0/321690865 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9fc8107ad0 msgr2=0x7f9fc819cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:45.593 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.590+0000 7f9fc77fe640 1 --2- 172.21.15.1:0/321690865 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9fc8107ad0 0x7f9fc819cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:45.593 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.590+0000 7f9fc77fe640 1 -- 172.21.15.1:0/321690865 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fc8102460 msgr2=0x7f9fc8198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:45.593 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.590+0000 7f9fc77fe640 1 --2- 172.21.15.1:0/321690865 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fc8102460 0x7f9fc8198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:45.593 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.590+0000 7f9fc77fe640 1 -- 172.21.15.1:0/321690865 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fc819d410 con 0x7f9fc8101ac0 2024-09-11T22:03:45.593 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.590+0000 7f9fc6ffd640 1 --2- 172.21.15.1:0/321690865 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fc8102460 0x7f9fc8198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:03:45.593 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.590+0000 7f9fc77fe640 1 --2- 172.21.15.1:0/321690865 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9fc8101ac0 0x7f9fc8198420 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f9fb800e9f0 tx=0x7f9fb800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:45.593 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.590+0000 7f9fc4ff9640 1 -- 172.21.15.1:0/321690865 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb800cd80 con 0x7f9fc8101ac0 2024-09-11T22:03:45.594 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.590+0000 7f9fc4ff9640 1 -- 172.21.15.1:0/321690865 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fb8010910 con 0x7f9fc8101ac0 2024-09-11T22:03:45.594 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.590+0000 7f9fce1cf640 1 -- 172.21.15.1:0/321690865 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fc819d700 con 0x7f9fc8101ac0 2024-09-11T22:03:45.594 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.590+0000 7f9fc4ff9640 1 -- 172.21.15.1:0/321690865 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb8018990 con 0x7f9fc8101ac0 2024-09-11T22:03:45.594 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.590+0000 7f9fce1cf640 1 -- 172.21.15.1:0/321690865 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fc81a4f90 con 0x7f9fc8101ac0 2024-09-11T22:03:45.594 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:45 smithi161 ceph-mon[35615]: pgmap v200: 97 pgs: 97 active+clean; 284 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 4.3 MiB/s wr, 1.84k op/s 2024-09-11T22:03:45.599 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.593+0000 7f9fd1044740 1 Processor -- start 2024-09-11T22:03:45.599 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.593+0000 7f9fd1044740 1 -- start start 2024-09-11T22:03:45.600 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.593+0000 7f9fc4ff9640 1 -- 172.21.15.1:0/321690865 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f9fb8010ac0 con 0x7f9fc8101ac0 2024-09-11T22:03:45.600 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.593+0000 7f9fd1044740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5649099e19b0 0x5649099e1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:45.600 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.593+0000 7f9fd1044740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649099e22d0 0x5649099f0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:45.600 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.593+0000 7f9fd1044740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5649099f10c0 0x5649099f34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:45.600 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.593+0000 7f9fd1044740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5649099462f0 con 0x5649099f10c0 2024-09-11T22:03:45.600 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.593+0000 7f9fd1044740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5649098ee270 con 0x5649099e19b0 2024-09-11T22:03:45.600 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.593+0000 7f9fd1044740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5649099422b0 con 0x5649099e22d0 2024-09-11T22:03:45.600 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.593+0000 7f9fc77fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649099e22d0 0x5649099f0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:45.600 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.593+0000 7f9fc6ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5649099f10c0 0x5649099f34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:45.600 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.594+0000 7f9fc6ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5649099f10c0 0x5649099f34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58648/0 (socket says 172.21.15.1:58648) 2024-09-11T22:03:45.600 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.594+0000 7f9fc6ffd640 1 -- 172.21.15.1:0/1450196601 learned_addr learned my addr 172.21.15.1:0/1450196601 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:03:45.601 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.594+0000 7f9fc7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5649099e19b0 0x5649099e1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:45.601 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.594+0000 7f9fc77fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649099e22d0 0x5649099f0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:46702/0 (socket says 172.21.15.1:46702) 2024-09-11T22:03:45.601 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.594+0000 7f9fc6ffd640 1 -- 172.21.15.1:0/1450196601 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649099e22d0 msgr2=0x5649099f0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:45.601 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.594+0000 7f9fc6ffd640 1 --2- 172.21.15.1:0/1450196601 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649099e22d0 0x5649099f0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:45.601 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.594+0000 7f9fc6ffd640 1 -- 172.21.15.1:0/1450196601 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5649099e19b0 msgr2=0x5649099e1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:45.601 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.594+0000 7f9fc6ffd640 1 --2- 172.21.15.1:0/1450196601 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5649099e19b0 0x5649099e1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:45.601 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.594+0000 7f9fc6ffd640 1 -- 172.21.15.1:0/1450196601 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649098ed340 con 0x5649099f10c0 2024-09-11T22:03:45.601 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.594+0000 7f9fc7fff640 1 --2- 172.21.15.1:0/1450196601 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5649099e19b0 0x5649099e1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:03:45.601 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.594+0000 7f9fc4ff9640 1 --2- 172.21.15.1:0/321690865 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f9fb00a2d70 0x7f9fb00a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:45.601 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.594+0000 7f9fc6ffd640 1 --2- 172.21.15.1:0/1450196601 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5649099f10c0 0x5649099f34b0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc000c00 tx=0x7f9fbc005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbe8e1107c8054cd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:45.601 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.594+0000 7f9fc7fff640 1 --2- 172.21.15.1:0/321690865 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f9fb00a2d70 0x7f9fb00a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:45.601 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.594+0000 7f9fc4ff9640 1 -- 172.21.15.1:0/321690865 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f9fb80cf660 con 0x7f9fc8101ac0 2024-09-11T22:03:45.602 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.594+0000 7f9fadffb640 1 -- 172.21.15.1:0/1450196601 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fbc035040 con 0x5649099f10c0 2024-09-11T22:03:45.602 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.594+0000 7f9fadffb640 1 -- 172.21.15.1:0/1450196601 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fbc031830 con 0x5649099f10c0 2024-09-11T22:03:45.602 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.594+0000 7f9fadffb640 1 -- 172.21.15.1:0/1450196601 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fbc003db0 con 0x5649099f10c0 2024-09-11T22:03:45.602 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.595+0000 7f9fd1044740 1 -- 172.21.15.1:0/1450196601 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5649099f10c0 msgr2=0x5649099f34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:45.602 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.595+0000 7f9fd1044740 1 --2- 172.21.15.1:0/1450196601 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5649099f10c0 0x5649099f34b0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc000c00 tx=0x7f9fbc005d50 comp rx=0 tx=0).stop 2024-09-11T22:03:45.602 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.595+0000 7f9fc7fff640 1 --2- 172.21.15.1:0/321690865 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f9fb00a2d70 0x7f9fb00a5230 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9fc0009fd0 tx=0x7f9fc0009500 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:45.602 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.595+0000 7f9fd1044740 1 -- 172.21.15.1:0/1450196601 shutdown_connections 2024-09-11T22:03:45.602 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.595+0000 7f9fd1044740 1 --2- 172.21.15.1:0/1450196601 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5649099f10c0 0x5649099f34b0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:45.602 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.595+0000 7f9fd1044740 1 --2- 172.21.15.1:0/1450196601 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649099e22d0 0x5649099f0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:45.602 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.595+0000 7f9fd1044740 1 --2- 172.21.15.1:0/1450196601 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5649099e19b0 0x5649099e1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:45.602 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.595+0000 7f9fd1044740 1 -- 172.21.15.1:0/1450196601 >> 172.21.15.1:0/1450196601 conn(0x5649099de060 msgr2=0x5649099de440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:45.603 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.595+0000 7f9fd1044740 1 -- 172.21.15.1:0/1450196601 shutdown_connections 2024-09-11T22:03:45.603 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.595+0000 7f9fd1044740 1 -- 172.21.15.1:0/1450196601 wait complete. 2024-09-11T22:03:45.603 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.596+0000 7f9fd1044740 1 -- start start 2024-09-11T22:03:45.603 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.596+0000 7f9fd1044740 1 Processor -- start 2024-09-11T22:03:45.603 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.596+0000 7f9fd1044740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5649099e19b0 0x5649099edfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:45.603 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.596+0000 7f9fd1044740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649099e22d0 0x5649099ee510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:45.603 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.596+0000 7f9fd1044740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5649099f10c0 0x5649099eea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:45.603 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.596+0000 7f9fd1044740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5649099f3c30 con 0x5649099e19b0 2024-09-11T22:03:45.603 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.596+0000 7f9fd1044740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5649098cf290 con 0x5649099f10c0 2024-09-11T22:03:45.603 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.596+0000 7f9fd1044740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5649099f3db0 con 0x5649099e22d0 2024-09-11T22:03:45.603 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.596+0000 7f9fd1044740 10 client.0 _ll_register_callbacks cb 0x564909900140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:03:45.603 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.596+0000 7f9fc77fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5649099e19b0 0x5649099edfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:45.603 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.596+0000 7f9fc6ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649099e22d0 0x5649099ee510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:45.604 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.596+0000 7f9fc6ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649099e22d0 0x5649099ee510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:46704/0 (socket says 172.21.15.1:46704) 2024-09-11T22:03:45.604 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.596+0000 7f9fc6ffd640 1 -- 172.21.15.1:0/1777642477 learned_addr learned my addr 172.21.15.1:0/1777642477 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:03:45.604 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.596+0000 7f9fc7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5649099f10c0 0x5649099eea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:45.604 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.597+0000 7f9fc6ffd640 1 -- 172.21.15.1:0/1777642477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5649099f10c0 msgr2=0x5649099eea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:45.604 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.597+0000 7f9fc6ffd640 1 --2- 172.21.15.1:0/1777642477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5649099f10c0 0x5649099eea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:45.604 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.597+0000 7f9fc6ffd640 1 -- 172.21.15.1:0/1777642477 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5649099e19b0 msgr2=0x5649099edfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:45.604 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.597+0000 7f9fc6ffd640 1 --2- 172.21.15.1:0/1777642477 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5649099e19b0 0x5649099edfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:45.604 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.597+0000 7f9fc6ffd640 1 -- 172.21.15.1:0/1777642477 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564909659210 con 0x5649099e22d0 2024-09-11T22:03:45.604 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.597+0000 7f9fc77fe640 1 --2- 172.21.15.1:0/1777642477 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5649099e19b0 0x5649099edfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:03:45.604 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.597+0000 7f9fc6ffd640 1 --2- 172.21.15.1:0/1777642477 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649099e22d0 0x5649099ee510 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc009900 tx=0x7f9fbc0038a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=434903e7d01be308 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:45.604 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.597+0000 7f9fa6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-11T22:03:45.604 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.597+0000 7f9fa6ffd640 1 -- 172.21.15.1:0/1777642477 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fbc04e020 con 0x5649099e22d0 2024-09-11T22:03:45.604 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.597+0000 7f9fd1044740 1 -- 172.21.15.1:0/1777642477 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fbc031830 con 0x5649099e22d0 2024-09-11T22:03:45.605 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.597+0000 7f9fa6ffd640 1 -- 172.21.15.1:0/1777642477 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f9fbc005250 con 0x5649099e22d0 2024-09-11T22:03:45.605 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.597+0000 7f9fa6ffd640 1 -- 172.21.15.1:0/1777642477 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f9fbc059050 con 0x5649099e22d0 2024-09-11T22:03:45.605 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.597+0000 7f9fd1044740 10 client.26308 fetch_fsmap learned FSMap version 42 2024-09-11T22:03:45.605 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.597+0000 7f9fd1044740 1 -- 172.21.15.1:0/1777642477 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x564909725620 con 0x5649099e22d0 2024-09-11T22:03:45.605 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.598+0000 7f9fa6ffd640 1 -- 172.21.15.1:0/1777642477 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fbc031830 con 0x5649099e22d0 2024-09-11T22:03:45.605 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.598+0000 7f9fa6ffd640 1 -- 172.21.15.1:0/1777642477 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f9fbc003bc0 con 0x5649099e22d0 2024-09-11T22:03:45.605 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.598+0000 7f9fd1044740 10 client.26308 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:03:45.605 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.598+0000 7f9fd1044740 10 client.26308 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:03:45.605 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.598+0000 7f9fd1044740 20 client.26308 populate_metadata read hostname 'smithi001' 2024-09-11T22:03:45.605 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.598+0000 7f9fd1044740 1 --2- 172.21.15.1:0/1777642477 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x564909a00e90 0x564909a03280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:45.605 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.598+0000 7f9fd1044740 4 client.26308 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:03:45.605 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.598+0000 7f9fd1044740 1 -- 172.21.15.1:0/1777642477 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564909953a20 con 0x564909a00e90 2024-09-11T22:03:45.605 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.598+0000 7f9fc77fe640 1 --2- 172.21.15.1:0/1777642477 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x564909a00e90 0x564909a03280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:45.605 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.598+0000 7f9fa6ffd640 10 client.26308 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:03:45.606 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.599+0000 7f9fc77fe640 1 --2- 172.21.15.1:0/1777642477 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x564909a00e90 0x564909a03280 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc171e9b735bddb2 server_cookie=425982677b9bf313 in_seq=0 out_seq=0 2024-09-11T22:03:45.607 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.600+0000 7f9fa6ffd640 1 -- 172.21.15.1:0/1777642477 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564909953a20 con 0x564909a00e90 2024-09-11T22:03:45.607 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.600+0000 7f9fa6ffd640 10 client.26308 handle_command_reply: tid=0 2024-09-11T22:03:45.607 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.600+0000 7f9fd1044740 1 client.26308 shutdown 2024-09-11T22:03:45.607 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.600+0000 7f9fd1044740 1 -- 172.21.15.1:0/1777642477 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649099e22d0 msgr2=0x5649099ee510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:45.607 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.600+0000 7f9fd1044740 1 --2- 172.21.15.1:0/1777642477 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649099e22d0 0x5649099ee510 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc009900 tx=0x7f9fbc0038a0 comp rx=0 tx=0).stop 2024-09-11T22:03:45.607 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.600+0000 7f9fd1044740 1 -- 172.21.15.1:0/1777642477 shutdown_connections 2024-09-11T22:03:45.607 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.600+0000 7f9fd1044740 1 --2- 172.21.15.1:0/1777642477 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x564909a00e90 0x564909a03280 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:45.607 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.600+0000 7f9fd1044740 1 --2- 172.21.15.1:0/1777642477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5649099f10c0 0x5649099eea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:45.607 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.600+0000 7f9fa6ffd640 0 client.26308 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:03:45.607 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.600+0000 7f9fd1044740 1 --2- 172.21.15.1:0/1777642477 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649099e22d0 0x5649099ee510 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:45.607 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.600+0000 7f9fd1044740 1 --2- 172.21.15.1:0/1777642477 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5649099e19b0 0x5649099edfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:45.607 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.600+0000 7f9fd1044740 1 -- 172.21.15.1:0/1777642477 >> 172.21.15.1:0/1777642477 conn(0x5649099de060 msgr2=0x5649099e3dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:45.608 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.600+0000 7f9fd1044740 1 -- 172.21.15.1:0/1777642477 shutdown_connections 2024-09-11T22:03:45.608 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.600+0000 7f9fd1044740 1 -- 172.21.15.1:0/1777642477 wait complete. 2024-09-11T22:03:45.608 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.600+0000 7f9fd1044740 20 client.26308 trim_cache size 0 max 16384 2024-09-11T22:03:45.616 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.615+0000 7f9fd1044740 1 Processor -- start 2024-09-11T22:03:45.617 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.615+0000 7f9fd1044740 1 -- start start 2024-09-11T22:03:45.617 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.615+0000 7f9fd1044740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5649099e19b0 0x5649099ec880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:45.617 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.615+0000 7f9fd1044740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649099e22d0 0x564909a088b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:45.617 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.615+0000 7f9fd1044740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5649099f10c0 0x564909a08df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:45.617 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.615+0000 7f9fd1044740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc81ab860 con 0x5649099e19b0 2024-09-11T22:03:45.617 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.615+0000 7f9fd1044740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc81ad6a0 con 0x5649099f10c0 2024-09-11T22:03:45.617 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.615+0000 7f9fd1044740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc81abff0 con 0x5649099e22d0 2024-09-11T22:03:45.617 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.615+0000 7f9fc77fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5649099e19b0 0x5649099ec880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:45.617 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.615+0000 7f9fc77fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5649099e19b0 0x5649099ec880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58652/0 (socket says 172.21.15.1:58652) 2024-09-11T22:03:45.617 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.615+0000 7f9fc77fe640 1 -- 172.21.15.1:0/1628421296 learned_addr learned my addr 172.21.15.1:0/1628421296 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:03:45.617 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.615+0000 7f9fc6ffd640 1 --2- 172.21.15.1:0/1628421296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649099e22d0 0x564909a088b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:45.618 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.615+0000 7f9fc7fff640 1 --2- 172.21.15.1:0/1628421296 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5649099f10c0 0x564909a08df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:45.618 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.616+0000 7f9fc77fe640 1 -- 172.21.15.1:0/1628421296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649099e22d0 msgr2=0x564909a088b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:45.618 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.616+0000 7f9fc77fe640 1 --2- 172.21.15.1:0/1628421296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649099e22d0 0x564909a088b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:45.618 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.616+0000 7f9fc77fe640 1 -- 172.21.15.1:0/1628421296 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5649099f10c0 msgr2=0x564909a08df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:45.618 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.616+0000 7f9fc77fe640 1 --2- 172.21.15.1:0/1628421296 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5649099f10c0 0x564909a08df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:45.618 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.616+0000 7f9fc77fe640 1 -- 172.21.15.1:0/1628421296 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649099e2100 con 0x5649099e19b0 2024-09-11T22:03:45.618 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.616+0000 7f9fc77fe640 1 --2- 172.21.15.1:0/1628421296 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5649099e19b0 0x5649099ec880 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f9fb809b8f0 tx=0x7f9fb80d5550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4ffe2d8830fc70e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:45.618 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.616+0000 7f9fad7fa640 1 -- 172.21.15.1:0/1628421296 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb809b550 con 0x5649099e19b0 2024-09-11T22:03:45.618 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.616+0000 7f9fad7fa640 1 -- 172.21.15.1:0/1628421296 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fb809bbf0 con 0x5649099e19b0 2024-09-11T22:03:45.618 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.616+0000 7f9fad7fa640 1 -- 172.21.15.1:0/1628421296 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb8018b30 con 0x5649099e19b0 2024-09-11T22:03:45.622 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.617+0000 7f9fd1044740 1 -- 172.21.15.1:0/1628421296 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5649099e19b0 msgr2=0x5649099ec880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:45.622 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.617+0000 7f9fd1044740 1 --2- 172.21.15.1:0/1628421296 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5649099e19b0 0x5649099ec880 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f9fb809b8f0 tx=0x7f9fb80d5550 comp rx=0 tx=0).stop 2024-09-11T22:03:45.622 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.617+0000 7f9fd1044740 1 -- 172.21.15.1:0/1628421296 shutdown_connections 2024-09-11T22:03:45.622 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.617+0000 7f9fd1044740 1 --2- 172.21.15.1:0/1628421296 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5649099f10c0 0x564909a08df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:45.622 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.617+0000 7f9fd1044740 1 --2- 172.21.15.1:0/1628421296 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649099e22d0 0x564909a088b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:45.622 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.617+0000 7f9fd1044740 1 --2- 172.21.15.1:0/1628421296 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5649099e19b0 0x5649099ec880 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:45.622 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.617+0000 7f9fd1044740 1 -- 172.21.15.1:0/1628421296 >> 172.21.15.1:0/1628421296 conn(0x5649099de060 msgr2=0x5649099f0050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:45.622 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.617+0000 7f9fd1044740 1 -- 172.21.15.1:0/1628421296 shutdown_connections 2024-09-11T22:03:45.622 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.617+0000 7f9fd1044740 1 -- 172.21.15.1:0/1628421296 wait complete. 2024-09-11T22:03:45.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.617+0000 7f9fd1044740 1 -- start start 2024-09-11T22:03:45.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.617+0000 7f9fd1044740 1 Processor -- start 2024-09-11T22:03:45.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.618+0000 7f9fd1044740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5649099e19b0 0x5649099f98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:45.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.618+0000 7f9fd1044740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5649099e22d0 0x5649099f4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:45.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.618+0000 7f9fd1044740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649099f10c0 0x5649099f5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:45.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.618+0000 7f9fd1044740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc81acf10 con 0x5649099e22d0 2024-09-11T22:03:45.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.618+0000 7f9fd1044740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc81ac780 con 0x5649099e19b0 2024-09-11T22:03:45.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.618+0000 7f9fd1044740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc81a9840 con 0x5649099f10c0 2024-09-11T22:03:45.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.618+0000 7f9fd1044740 10 client.0 _ll_register_callbacks cb 0x7f9fbc040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:03:45.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.618+0000 7f9fc6ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5649099e22d0 0x5649099f4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:45.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.618+0000 7f9fc77fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5649099e19b0 0x5649099f98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:45.624 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.618+0000 7f9fc77fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5649099e19b0 0x5649099f98f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:55090/0 (socket says 172.21.15.1:55090) 2024-09-11T22:03:45.624 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.618+0000 7f9fc6ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5649099e22d0 0x5649099f4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58654/0 (socket says 172.21.15.1:58654) 2024-09-11T22:03:45.624 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.618+0000 7f9fc77fe640 1 -- 172.21.15.1:0/3220584757 learned_addr learned my addr 172.21.15.1:0/3220584757 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:03:45.624 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.618+0000 7f9fc7fff640 1 --2- 172.21.15.1:0/3220584757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649099f10c0 0x5649099f5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:45.624 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.618+0000 7f9fc77fe640 1 -- 172.21.15.1:0/3220584757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649099f10c0 msgr2=0x5649099f5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:45.624 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.618+0000 7f9fc77fe640 1 --2- 172.21.15.1:0/3220584757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649099f10c0 0x5649099f5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:45.624 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.618+0000 7f9fc77fe640 1 -- 172.21.15.1:0/3220584757 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5649099e22d0 msgr2=0x5649099f4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:45.624 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.618+0000 7f9fc77fe640 1 --2- 172.21.15.1:0/3220584757 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5649099e22d0 0x5649099f4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:45.624 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.618+0000 7f9fc77fe640 1 -- 172.21.15.1:0/3220584757 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649099f5a50 con 0x5649099e19b0 2024-09-11T22:03:45.624 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.618+0000 7f9fc6ffd640 1 --2- 172.21.15.1:0/3220584757 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5649099e22d0 0x5649099f4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:03:45.624 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.618+0000 7f9fc77fe640 1 --2- 172.21.15.1:0/3220584757 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5649099e19b0 0x5649099f98f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9fb80d4010 tx=0x7f9fb8005bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ee3dbb7e751583 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:45.624 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.618+0000 7f9fa6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-11T22:03:45.625 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.619+0000 7f9fa6ffd640 1 -- 172.21.15.1:0/3220584757 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb80b9060 con 0x5649099e19b0 2024-09-11T22:03:45.625 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.619+0000 7f9fa6ffd640 1 -- 172.21.15.1:0/3220584757 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f9fb80a4320 con 0x5649099e19b0 2024-09-11T22:03:45.625 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.619+0000 7f9fd1044740 1 -- 172.21.15.1:0/3220584757 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fb809bbf0 con 0x5649099e19b0 2024-09-11T22:03:45.625 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.619+0000 7f9fa6ffd640 1 -- 172.21.15.1:0/3220584757 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f9fb80a47b0 con 0x5649099e19b0 2024-09-11T22:03:45.625 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.619+0000 7f9fd1044740 10 client.26348 fetch_fsmap learned FSMap version 42 2024-09-11T22:03:45.625 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.619+0000 7f9fd1044740 1 -- 172.21.15.1:0/3220584757 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x564909a6b9c0 con 0x5649099e19b0 2024-09-11T22:03:45.625 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.619+0000 7f9fa6ffd640 1 -- 172.21.15.1:0/3220584757 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fb809bbf0 con 0x5649099e19b0 2024-09-11T22:03:45.625 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.619+0000 7f9fa6ffd640 1 -- 172.21.15.1:0/3220584757 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f9fb8018b30 con 0x5649099e19b0 2024-09-11T22:03:45.625 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.619+0000 7f9fd1044740 10 client.26348 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:03:45.625 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.619+0000 7f9fd1044740 10 client.26348 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:03:45.625 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.619+0000 7f9fd1044740 20 client.26348 populate_metadata read hostname 'smithi001' 2024-09-11T22:03:45.625 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.619+0000 7f9fd1044740 1 --2- 172.21.15.1:0/3220584757 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x564909a6c180 0x564909a6e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:45.626 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.619+0000 7f9fd1044740 4 client.26348 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:03:45.626 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.619+0000 7f9fd1044740 1 -- 172.21.15.1:0/3220584757 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56490994dd70 con 0x564909a6c180 2024-09-11T22:03:45.626 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.619+0000 7f9fc6ffd640 1 --2- 172.21.15.1:0/3220584757 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x564909a6c180 0x564909a6e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:45.626 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.620+0000 7f9fa6ffd640 10 client.26348 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:03:45.626 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:45.620+0000 7f9fc6ffd640 1 --2- 172.21.15.1:0/3220584757 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x564909a6c180 0x564909a6e5a0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d92d48ac734ab69f server_cookie=9d0b44c929776b61 in_seq=0 out_seq=0 2024-09-11T22:03:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:46.212+0000 7f9fa6ffd640 1 -- 172.21.15.1:0/3220584757 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56490994dd70 con 0x564909a6c180 2024-09-11T22:03:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:46.212+0000 7f9fa6ffd640 10 client.26348 handle_command_reply: tid=0 2024-09-11T22:03:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:46.212+0000 7f9fd1044740 1 client.26348 shutdown 2024-09-11T22:03:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:46.212+0000 7f9fd1044740 1 -- 172.21.15.1:0/3220584757 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5649099e19b0 msgr2=0x5649099f98f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:46.212+0000 7f9fd1044740 1 --2- 172.21.15.1:0/3220584757 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5649099e19b0 0x5649099f98f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9fb80d4010 tx=0x7f9fb8005bf0 comp rx=0 tx=0).stop 2024-09-11T22:03:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:46.212+0000 7f9fd1044740 1 -- 172.21.15.1:0/3220584757 shutdown_connections 2024-09-11T22:03:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:46.212+0000 7f9fd1044740 1 --2- 172.21.15.1:0/3220584757 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x564909a6c180 0x564909a6e5a0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:46.212+0000 7f9fd1044740 1 --2- 172.21.15.1:0/3220584757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5649099f10c0 0x5649099f5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:46.212+0000 7f9fd1044740 1 --2- 172.21.15.1:0/3220584757 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5649099e22d0 0x5649099f4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:46.212+0000 7f9fd1044740 1 --2- 172.21.15.1:0/3220584757 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5649099e19b0 0x5649099f98f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:46.212+0000 7f9fd1044740 1 -- 172.21.15.1:0/3220584757 >> 172.21.15.1:0/3220584757 conn(0x5649099de060 msgr2=0x5649099e3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:46.212+0000 7f9fa6ffd640 0 client.26348 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:03:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:46.213+0000 7f9fd1044740 1 -- 172.21.15.1:0/3220584757 shutdown_connections 2024-09-11T22:03:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:46.213+0000 7f9fd1044740 1 -- 172.21.15.1:0/3220584757 wait complete. 2024-09-11T22:03:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:46.213+0000 7f9fd1044740 20 client.26348 trim_cache size 0 max 16384 2024-09-11T22:03:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-09-11T22:03:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-09-11T22:03:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-09-11T22:03:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-09-11T22:03:46.216 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:46.214+0000 7f9fce1cf640 1 -- 172.21.15.1:0/321690865 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f9fb00a2d70 msgr2=0x7f9fb00a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:46.216 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:46.214+0000 7f9fce1cf640 1 --2- 172.21.15.1:0/321690865 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f9fb00a2d70 0x7f9fb00a5230 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9fc0009fd0 tx=0x7f9fc0009500 comp rx=0 tx=0).stop 2024-09-11T22:03:46.216 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:46.214+0000 7f9fce1cf640 1 -- 172.21.15.1:0/321690865 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9fc8101ac0 msgr2=0x7f9fc8198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:46.216 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:46.214+0000 7f9fce1cf640 1 --2- 172.21.15.1:0/321690865 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9fc8101ac0 0x7f9fc8198420 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f9fb800e9f0 tx=0x7f9fb800eeb0 comp rx=0 tx=0).stop 2024-09-11T22:03:46.216 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:46.214+0000 7f9fce1cf640 1 -- 172.21.15.1:0/321690865 shutdown_connections 2024-09-11T22:03:46.216 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:46.214+0000 7f9fce1cf640 1 --2- 172.21.15.1:0/321690865 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f9fb00a2d70 0x7f9fb00a5230 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:46.216 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:46.214+0000 7f9fce1cf640 1 --2- 172.21.15.1:0/321690865 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9fc8107ad0 0x7f9fc819cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:46.216 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:46.214+0000 7f9fce1cf640 1 --2- 172.21.15.1:0/321690865 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fc8102460 0x7f9fc8198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:46.216 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:46.214+0000 7f9fce1cf640 1 --2- 172.21.15.1:0/321690865 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9fc8101ac0 0x7f9fc8198420 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:46.216 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:46.214+0000 7f9fce1cf640 1 -- 172.21.15.1:0/321690865 >> 172.21.15.1:0/321690865 conn(0x7f9fc80fd7a0 msgr2=0x7f9fc80fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:46.217 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:46.214+0000 7f9fce1cf640 1 -- 172.21.15.1:0/321690865 shutdown_connections 2024-09-11T22:03:46.217 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:46.214+0000 7f9fce1cf640 1 -- 172.21.15.1:0/321690865 wait complete. 2024-09-11T22:03:47.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:47 smithi001 ceph-mon[30935]: pgmap v201: 97 pgs: 97 active+clean; 313 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 6.7 MiB/s wr, 2.05k op/s 2024-09-11T22:03:47.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:47 smithi053 ceph-mon[34908]: pgmap v201: 97 pgs: 97 active+clean; 313 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 6.7 MiB/s wr, 2.05k op/s 2024-09-11T22:03:47.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:47 smithi161 ceph-mon[35615]: pgmap v201: 97 pgs: 97 active+clean; 313 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 6.7 MiB/s wr, 2.05k op/s 2024-09-11T22:03:49.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:49 smithi001 ceph-mon[30935]: pgmap v202: 97 pgs: 97 active+clean; 313 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.3 MiB/s wr, 693 op/s 2024-09-11T22:03:49.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:49 smithi053 ceph-mon[34908]: pgmap v202: 97 pgs: 97 active+clean; 313 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.3 MiB/s wr, 693 op/s 2024-09-11T22:03:49.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:49 smithi161 ceph-mon[35615]: pgmap v202: 97 pgs: 97 active+clean; 313 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.3 MiB/s wr, 693 op/s 2024-09-11T22:03:51.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:51 smithi001 ceph-mon[30935]: pgmap v203: 97 pgs: 97 active+clean; 319 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.3 MiB/s wr, 965 op/s 2024-09-11T22:03:51.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:51 smithi001 ceph-mon[30935]: scrub summary: idle 2024-09-11T22:03:51.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:51 smithi001 ceph-mon[30935]: scrub summary: idle 2024-09-11T22:03:51.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:51 smithi053 ceph-mon[34908]: pgmap v203: 97 pgs: 97 active+clean; 319 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.3 MiB/s wr, 965 op/s 2024-09-11T22:03:51.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:51 smithi053 ceph-mon[34908]: scrub summary: idle 2024-09-11T22:03:51.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:51 smithi053 ceph-mon[34908]: scrub summary: idle 2024-09-11T22:03:51.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:51 smithi161 ceph-mon[35615]: pgmap v203: 97 pgs: 97 active+clean; 319 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.3 MiB/s wr, 965 op/s 2024-09-11T22:03:51.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:51 smithi161 ceph-mon[35615]: scrub summary: idle 2024-09-11T22:03:51.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:51 smithi161 ceph-mon[35615]: scrub summary: idle 2024-09-11T22:03:52.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:52 smithi001 ceph-mon[30935]: scrub summary: idle 2024-09-11T22:03:52.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:52 smithi001 ceph-mon[30935]: scrub summary: idle 2024-09-11T22:03:52.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:52 smithi001 ceph-mon[30935]: scrub summary: idle 2024-09-11T22:03:52.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:52 smithi053 ceph-mon[34908]: scrub summary: idle 2024-09-11T22:03:52.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:52 smithi053 ceph-mon[34908]: scrub summary: idle 2024-09-11T22:03:52.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:52 smithi053 ceph-mon[34908]: scrub summary: idle 2024-09-11T22:03:52.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:52 smithi161 ceph-mon[35615]: scrub summary: idle 2024-09-11T22:03:52.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:52 smithi161 ceph-mon[35615]: scrub summary: idle 2024-09-11T22:03:52.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:52 smithi161 ceph-mon[35615]: scrub summary: idle 2024-09-11T22:03:53.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:53 smithi001 ceph-mon[30935]: pgmap v204: 97 pgs: 97 active+clean; 319 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.4 MiB/s wr, 483 op/s 2024-09-11T22:03:53.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:53 smithi053 ceph-mon[34908]: pgmap v204: 97 pgs: 97 active+clean; 319 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.4 MiB/s wr, 483 op/s 2024-09-11T22:03:53.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:53 smithi161 ceph-mon[35615]: pgmap v204: 97 pgs: 97 active+clean; 319 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.4 MiB/s wr, 483 op/s 2024-09-11T22:03:55.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:55 smithi001 ceph-mon[30935]: pgmap v205: 97 pgs: 97 active+clean; 319 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.4 MiB/s wr, 483 op/s 2024-09-11T22:03:55.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:55 smithi053 ceph-mon[34908]: pgmap v205: 97 pgs: 97 active+clean; 319 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.4 MiB/s wr, 483 op/s 2024-09-11T22:03:55.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:55 smithi161 ceph-mon[35615]: pgmap v205: 97 pgs: 97 active+clean; 319 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.4 MiB/s wr, 483 op/s 2024-09-11T22:03:57.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:57 smithi001 ceph-mon[30935]: pgmap v206: 97 pgs: 97 active+clean; 344 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.5 MiB/s wr, 724 op/s 2024-09-11T22:03:57.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:57 smithi053 ceph-mon[34908]: pgmap v206: 97 pgs: 97 active+clean; 344 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.5 MiB/s wr, 724 op/s 2024-09-11T22:03:57.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:57 smithi161 ceph-mon[35615]: pgmap v206: 97 pgs: 97 active+clean; 344 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.5 MiB/s wr, 724 op/s 2024-09-11T22:03:59.314 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.311+0000 7f6dbeeba640 1 -- 172.21.15.1:0/451838753 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6db81009a0 msgr2=0x7f6db8100e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:59.315 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.311+0000 7f6dbeeba640 1 --2- 172.21.15.1:0/451838753 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6db81009a0 0x7f6db8100e00 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f6da0009cd0 tx=0x7f6da002a8e0 comp rx=0 tx=0).stop 2024-09-11T22:03:59.315 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.311+0000 7f6dbeeba640 1 -- 172.21.15.1:0/451838753 shutdown_connections 2024-09-11T22:03:59.315 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.311+0000 7f6dbeeba640 1 --2- 172.21.15.1:0/451838753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6db8101340 0x7f6db810d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:59.315 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.311+0000 7f6dbeeba640 1 --2- 172.21.15.1:0/451838753 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6db81009a0 0x7f6db8100e00 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:59.315 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.311+0000 7f6dbeeba640 1 --2- 172.21.15.1:0/451838753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6db81069b0 0x7f6db8106d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:59.315 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.311+0000 7f6dbeeba640 1 -- 172.21.15.1:0/451838753 >> 172.21.15.1:0/451838753 conn(0x7f6db80779a0 msgr2=0x7f6db80fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:59.315 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.311+0000 7f6dbeeba640 1 -- 172.21.15.1:0/451838753 shutdown_connections 2024-09-11T22:03:59.315 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.311+0000 7f6dbeeba640 1 -- 172.21.15.1:0/451838753 wait complete. 2024-09-11T22:03:59.315 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.312+0000 7f6dbeeba640 1 Processor -- start 2024-09-11T22:03:59.315 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.312+0000 7f6dbeeba640 1 -- start start 2024-09-11T22:03:59.315 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.312+0000 7f6dbeeba640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6db81009a0 0x7f6db81983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:59.315 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.312+0000 7f6dbeeba640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6db8101340 0x7f6db8198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:59.315 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.312+0000 7f6dbeeba640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6db81069b0 0x7f6db819ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:59.316 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.312+0000 7f6dbeeba640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6db8110df0 con 0x7f6db8101340 2024-09-11T22:03:59.316 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.312+0000 7f6dbeeba640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6db8110c70 con 0x7f6db81069b0 2024-09-11T22:03:59.316 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.312+0000 7f6dbeeba640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6db8110f70 con 0x7f6db81009a0 2024-09-11T22:03:59.316 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.312+0000 7f6db7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6db8101340 0x7f6db8198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:59.316 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.312+0000 7f6dbcc56640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6db81009a0 0x7f6db81983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:59.316 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.312+0000 7f6db7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6db8101340 0x7f6db8198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56826/0 (socket says 172.21.15.1:56826) 2024-09-11T22:03:59.316 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.312+0000 7f6db7fff640 1 -- 172.21.15.1:0/2418141790 learned_addr learned my addr 172.21.15.1:0/2418141790 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:03:59.316 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.312+0000 7f6dbd457640 1 --2- 172.21.15.1:0/2418141790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6db81069b0 0x7f6db819ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:59.316 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.312+0000 7f6db7fff640 1 -- 172.21.15.1:0/2418141790 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6db81009a0 msgr2=0x7f6db81983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:59.316 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.312+0000 7f6db7fff640 1 --2- 172.21.15.1:0/2418141790 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6db81009a0 0x7f6db81983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:59.316 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.312+0000 7f6db7fff640 1 -- 172.21.15.1:0/2418141790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6db81069b0 msgr2=0x7f6db819ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:59.316 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.312+0000 7f6db7fff640 1 --2- 172.21.15.1:0/2418141790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6db81069b0 0x7f6db819ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:59.317 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.313+0000 7f6db7fff640 1 -- 172.21.15.1:0/2418141790 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6db819d3e0 con 0x7f6db8101340 2024-09-11T22:03:59.317 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.313+0000 7f6db7fff640 1 --2- 172.21.15.1:0/2418141790 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6db8101340 0x7f6db8198930 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f6da002ae00 tx=0x7f6da0031e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:59.317 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.313+0000 7f6db5ffb640 1 -- 172.21.15.1:0/2418141790 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da0003e00 con 0x7f6db8101340 2024-09-11T22:03:59.317 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.313+0000 7f6dbeeba640 1 -- 172.21.15.1:0/2418141790 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6db819d670 con 0x7f6db8101340 2024-09-11T22:03:59.317 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.313+0000 7f6db5ffb640 1 -- 172.21.15.1:0/2418141790 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6da0004520 con 0x7f6db8101340 2024-09-11T22:03:59.317 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.313+0000 7f6db5ffb640 1 -- 172.21.15.1:0/2418141790 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da0030de0 con 0x7f6db8101340 2024-09-11T22:03:59.317 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.313+0000 7f6dbeeba640 1 -- 172.21.15.1:0/2418141790 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6db81a4f60 con 0x7f6db8101340 2024-09-11T22:03:59.322 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.315+0000 7f6db5ffb640 1 -- 172.21.15.1:0/2418141790 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f6da00046d0 con 0x7f6db8101340 2024-09-11T22:03:59.322 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.316+0000 7f6dc1c5b740 1 Processor -- start 2024-09-11T22:03:59.322 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.316+0000 7f6dc1c5b740 1 -- start start 2024-09-11T22:03:59.322 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.316+0000 7f6dc1c5b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557ece7c29d0 0x557ece7c2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:59.322 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.316+0000 7f6dc1c5b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557ece7c32f0 0x557ece7d1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:59.322 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.316+0000 7f6dc1c5b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557ece7d20c0 0x557ece7d44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:59.323 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.316+0000 7f6dc1c5b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x557ece7272f0 con 0x557ece7d20c0 2024-09-11T22:03:59.323 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.316+0000 7f6dc1c5b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x557ece6cf270 con 0x557ece7c32f0 2024-09-11T22:03:59.323 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.316+0000 7f6dc1c5b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x557ece7232b0 con 0x557ece7c29d0 2024-09-11T22:03:59.323 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.316+0000 7f6db5ffb640 1 --2- 172.21.15.1:0/2418141790 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f6da40a2cf0 0x7f6da40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:59.323 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.316+0000 7f6dbd457640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557ece7c29d0 0x557ece7c2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:59.323 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.316+0000 7f6dbd457640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557ece7c29d0 0x557ece7c2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:37104/0 (socket says 172.21.15.1:37104) 2024-09-11T22:03:59.323 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.316+0000 7f6dbd457640 1 -- 172.21.15.1:0/3304495317 learned_addr learned my addr 172.21.15.1:0/3304495317 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:03:59.323 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.316+0000 7f6dbcc56640 1 --2- 172.21.15.1:0/3304495317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557ece7c32f0 0x557ece7d1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:59.323 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.316+0000 7f6db7fff640 1 --2- 172.21.15.1:0/3304495317 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557ece7d20c0 0x557ece7d44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:59.323 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.317+0000 7f6dbd457640 1 --2- 172.21.15.1:0/2418141790 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f6da40a2cf0 0x7f6da40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:59.323 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.317+0000 7f6dbd457640 1 -- 172.21.15.1:0/3304495317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557ece7c32f0 msgr2=0x557ece7d1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:59.323 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.317+0000 7f6db5ffb640 1 -- 172.21.15.1:0/2418141790 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f6da002e030 con 0x7f6db8101340 2024-09-11T22:03:59.324 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.317+0000 7f6dbd457640 1 --2- 172.21.15.1:0/3304495317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557ece7c32f0 0x557ece7d1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:59.324 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.317+0000 7f6dbd457640 1 -- 172.21.15.1:0/3304495317 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557ece7d20c0 msgr2=0x557ece7d44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:59.324 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.317+0000 7f6dbd457640 1 --2- 172.21.15.1:0/3304495317 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557ece7d20c0 0x557ece7d44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:59.324 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.317+0000 7f6dbd457640 1 -- 172.21.15.1:0/3304495317 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ece6ce340 con 0x557ece7c29d0 2024-09-11T22:03:59.324 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.317+0000 7f6dbd457640 1 --2- 172.21.15.1:0/3304495317 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557ece7c29d0 0x557ece7c2db0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6db00138b0 tx=0x7f6db0013cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96ce9f6f53551b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:59.324 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.317+0000 7f6dbd457640 1 --2- 172.21.15.1:0/2418141790 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f6da40a2cf0 0x7f6da40a51b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6db001e000 tx=0x7f6db0020000 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:59.324 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.317+0000 7f6daeffd640 1 -- 172.21.15.1:0/3304495317 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db0012280 con 0x557ece7c29d0 2024-09-11T22:03:59.324 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.317+0000 7f6daeffd640 1 -- 172.21.15.1:0/3304495317 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6db0023070 con 0x557ece7c29d0 2024-09-11T22:03:59.324 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.317+0000 7f6daeffd640 1 -- 172.21.15.1:0/3304495317 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db00035f0 con 0x557ece7c29d0 2024-09-11T22:03:59.324 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.318+0000 7f6dc1c5b740 1 -- 172.21.15.1:0/3304495317 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557ece7c29d0 msgr2=0x557ece7c2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:59.324 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.318+0000 7f6dc1c5b740 1 --2- 172.21.15.1:0/3304495317 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557ece7c29d0 0x557ece7c2db0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6db00138b0 tx=0x7f6db0013cb0 comp rx=0 tx=0).stop 2024-09-11T22:03:59.324 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.318+0000 7f6dc1c5b740 1 -- 172.21.15.1:0/3304495317 shutdown_connections 2024-09-11T22:03:59.325 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.318+0000 7f6dc1c5b740 1 --2- 172.21.15.1:0/3304495317 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557ece7d20c0 0x557ece7d44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:59.325 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.318+0000 7f6dc1c5b740 1 --2- 172.21.15.1:0/3304495317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557ece7c32f0 0x557ece7d1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:59.325 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.318+0000 7f6dc1c5b740 1 --2- 172.21.15.1:0/3304495317 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557ece7c29d0 0x557ece7c2db0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:59.325 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.318+0000 7f6dc1c5b740 1 -- 172.21.15.1:0/3304495317 >> 172.21.15.1:0/3304495317 conn(0x557ece7bf060 msgr2=0x557ece7bf440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:59.325 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.318+0000 7f6dc1c5b740 1 -- 172.21.15.1:0/3304495317 shutdown_connections 2024-09-11T22:03:59.325 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.318+0000 7f6dc1c5b740 1 -- 172.21.15.1:0/3304495317 wait complete. 2024-09-11T22:03:59.325 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.318+0000 7f6dc1c5b740 1 -- start start 2024-09-11T22:03:59.325 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.318+0000 7f6dc1c5b740 1 Processor -- start 2024-09-11T22:03:59.325 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.319+0000 7f6dc1c5b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557ece7c29d0 0x557ece7cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:59.325 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.319+0000 7f6dc1c5b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557ece7c32f0 0x557ece7cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:59.325 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.319+0000 7f6dc1c5b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557ece7d20c0 0x557ece7cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:59.325 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.319+0000 7f6dc1c5b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x557ece7d4c90 con 0x557ece7c29d0 2024-09-11T22:03:59.325 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.319+0000 7f6dc1c5b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x557ece6b0290 con 0x557ece7d20c0 2024-09-11T22:03:59.326 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.319+0000 7f6dc1c5b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x557ece7d4e10 con 0x557ece7c32f0 2024-09-11T22:03:59.326 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.319+0000 7f6dc1c5b740 10 client.0 _ll_register_callbacks cb 0x557ece6e1140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:03:59.326 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.319+0000 7f6dbcc56640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557ece7c29d0 0x557ece7cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:59.326 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.319+0000 7f6dbcc56640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557ece7c29d0 0x557ece7cf050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56858/0 (socket says 172.21.15.1:56858) 2024-09-11T22:03:59.326 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.319+0000 7f6dbcc56640 1 -- 172.21.15.1:0/1441696344 learned_addr learned my addr 172.21.15.1:0/1441696344 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:03:59.326 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.319+0000 7f6dbd457640 1 --2- 172.21.15.1:0/1441696344 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557ece7d20c0 0x557ece7cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:59.326 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.319+0000 7f6db7fff640 1 --2- 172.21.15.1:0/1441696344 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557ece7c32f0 0x557ece7cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:59.326 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.319+0000 7f6dbcc56640 1 -- 172.21.15.1:0/1441696344 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557ece7c32f0 msgr2=0x557ece7cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:59.326 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.319+0000 7f6dbcc56640 1 --2- 172.21.15.1:0/1441696344 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557ece7c32f0 0x557ece7cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:59.326 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.319+0000 7f6dbcc56640 1 -- 172.21.15.1:0/1441696344 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557ece7d20c0 msgr2=0x557ece7cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:59.326 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.319+0000 7f6dbcc56640 1 --2- 172.21.15.1:0/1441696344 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557ece7d20c0 0x557ece7cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:59.326 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.319+0000 7f6dbcc56640 1 -- 172.21.15.1:0/1441696344 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ece43a210 con 0x557ece7c29d0 2024-09-11T22:03:59.327 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.320+0000 7f6dbcc56640 1 --2- 172.21.15.1:0/1441696344 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557ece7c29d0 0x557ece7cf050 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f6da80126e0 tx=0x7f6da8012ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60217a5d6b4ca0bb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:59.327 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.320+0000 7f6d97fff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:03:59.327 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.320+0000 7f6d97fff640 1 -- 172.21.15.1:0/1441696344 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da80120d0 con 0x557ece7c29d0 2024-09-11T22:03:59.327 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.320+0000 7f6d97fff640 1 -- 172.21.15.1:0/1441696344 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f6da800f920 con 0x557ece7c29d0 2024-09-11T22:03:59.328 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.320+0000 7f6dc1c5b740 1 -- 172.21.15.1:0/1441696344 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6db0023070 con 0x557ece7c29d0 2024-09-11T22:03:59.328 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.320+0000 7f6d97fff640 1 -- 172.21.15.1:0/1441696344 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f6da8012270 con 0x557ece7c29d0 2024-09-11T22:03:59.329 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.320+0000 7f6dc1c5b740 10 client.16827 fetch_fsmap learned FSMap version 42 2024-09-11T22:03:59.329 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.320+0000 7f6dc1c5b740 1 -- 172.21.15.1:0/1441696344 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x557ece506620 con 0x557ece7c29d0 2024-09-11T22:03:59.329 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.320+0000 7f6d97fff640 1 -- 172.21.15.1:0/1441696344 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6db0023070 con 0x557ece7c29d0 2024-09-11T22:03:59.329 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.320+0000 7f6d97fff640 1 -- 172.21.15.1:0/1441696344 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f6da8022030 con 0x557ece7c29d0 2024-09-11T22:03:59.329 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.320+0000 7f6dc1c5b740 10 client.16827 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:03:59.329 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.321+0000 7f6dc1c5b740 10 client.16827 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:03:59.329 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.321+0000 7f6dc1c5b740 20 client.16827 populate_metadata read hostname 'smithi001' 2024-09-11T22:03:59.329 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.321+0000 7f6dc1c5b740 1 --2- 172.21.15.1:0/1441696344 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x557ece7e2220 0x557ece7e4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:59.329 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.321+0000 7f6dc1c5b740 4 client.16827 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:03:59.329 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.321+0000 7f6dc1c5b740 1 -- 172.21.15.1:0/1441696344 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ece734a20 con 0x557ece7e2220 2024-09-11T22:03:59.329 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.321+0000 7f6db7fff640 1 --2- 172.21.15.1:0/1441696344 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x557ece7e2220 0x557ece7e4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:59.329 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.321+0000 7f6db7fff640 1 --2- 172.21.15.1:0/1441696344 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x557ece7e2220 0x557ece7e4680 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=963402420832d2b1 server_cookie=75fbdb89fc26d528 in_seq=0 out_seq=0 2024-09-11T22:03:59.329 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.321+0000 7f6d97fff640 10 client.16827 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:03:59.331 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.326+0000 7f6d97fff640 1 -- 172.21.15.1:0/1441696344 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557ece734a20 con 0x557ece7e2220 2024-09-11T22:03:59.331 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.326+0000 7f6d97fff640 10 client.16827 handle_command_reply: tid=0 2024-09-11T22:03:59.331 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.326+0000 7f6dc1c5b740 1 client.16827 shutdown 2024-09-11T22:03:59.331 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.327+0000 7f6dc1c5b740 1 -- 172.21.15.1:0/1441696344 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557ece7c29d0 msgr2=0x557ece7cf050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:59.331 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.327+0000 7f6dc1c5b740 1 --2- 172.21.15.1:0/1441696344 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557ece7c29d0 0x557ece7cf050 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f6da80126e0 tx=0x7f6da8012ba0 comp rx=0 tx=0).stop 2024-09-11T22:03:59.331 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.327+0000 7f6dc1c5b740 1 -- 172.21.15.1:0/1441696344 shutdown_connections 2024-09-11T22:03:59.331 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.327+0000 7f6dc1c5b740 1 --2- 172.21.15.1:0/1441696344 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x557ece7e2220 0x557ece7e4680 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:59.331 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.327+0000 7f6dc1c5b740 1 --2- 172.21.15.1:0/1441696344 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557ece7d20c0 0x557ece7cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:59.331 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.327+0000 7f6dc1c5b740 1 --2- 172.21.15.1:0/1441696344 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557ece7c32f0 0x557ece7cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:59.331 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.327+0000 7f6dc1c5b740 1 --2- 172.21.15.1:0/1441696344 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557ece7c29d0 0x557ece7cf050 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:59.331 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.327+0000 7f6dc1c5b740 1 -- 172.21.15.1:0/1441696344 >> 172.21.15.1:0/1441696344 conn(0x557ece7bf060 msgr2=0x557ece7c4dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:59.332 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.327+0000 7f6d97fff640 0 client.16827 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:03:59.332 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.327+0000 7f6dc1c5b740 1 -- 172.21.15.1:0/1441696344 shutdown_connections 2024-09-11T22:03:59.332 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.327+0000 7f6dc1c5b740 1 -- 172.21.15.1:0/1441696344 wait complete. 2024-09-11T22:03:59.332 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.327+0000 7f6dc1c5b740 20 client.16827 trim_cache size 0 max 16384 2024-09-11T22:03:59.348 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.342+0000 7f6dc1c5b740 1 Processor -- start 2024-09-11T22:03:59.348 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.342+0000 7f6dc1c5b740 1 -- start start 2024-09-11T22:03:59.348 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.342+0000 7f6dc1c5b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557ece7c29d0 0x557ece7e9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:59.349 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.342+0000 7f6dc1c5b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557ece7c32f0 0x557ece7ea4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:59.349 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.342+0000 7f6dc1c5b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557ece7d20c0 0x557ece7d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:59.349 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.342+0000 7f6dc1c5b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6db81ab6d0 con 0x557ece7d20c0 2024-09-11T22:03:59.349 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.342+0000 7f6dc1c5b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6db81ad510 con 0x557ece7c29d0 2024-09-11T22:03:59.349 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.342+0000 7f6dc1c5b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6db81abe60 con 0x557ece7c32f0 2024-09-11T22:03:59.349 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.342+0000 7f6dbd457640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557ece7d20c0 0x557ece7d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:59.349 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.342+0000 7f6db7fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557ece7c32f0 0x557ece7ea4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:59.349 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.342+0000 7f6dbd457640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557ece7d20c0 0x557ece7d08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56868/0 (socket says 172.21.15.1:56868) 2024-09-11T22:03:59.349 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.342+0000 7f6dbd457640 1 -- 172.21.15.1:0/112596902 learned_addr learned my addr 172.21.15.1:0/112596902 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:03:59.349 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.342+0000 7f6dbcc56640 1 --2- 172.21.15.1:0/112596902 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557ece7c29d0 0x557ece7e9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:59.349 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.343+0000 7f6dbd457640 1 -- 172.21.15.1:0/112596902 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557ece7c32f0 msgr2=0x557ece7ea4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:59.349 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.343+0000 7f6dbd457640 1 --2- 172.21.15.1:0/112596902 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557ece7c32f0 0x557ece7ea4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:59.350 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.343+0000 7f6dbd457640 1 -- 172.21.15.1:0/112596902 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557ece7c29d0 msgr2=0x557ece7e9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:59.350 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.343+0000 7f6dbd457640 1 --2- 172.21.15.1:0/112596902 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557ece7c29d0 0x557ece7e9f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:59.350 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.343+0000 7f6dbd457640 1 -- 172.21.15.1:0/112596902 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ece7b92f0 con 0x557ece7d20c0 2024-09-11T22:03:59.350 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.343+0000 7f6db7fff640 1 --2- 172.21.15.1:0/112596902 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557ece7c32f0 0x557ece7ea4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:03:59.350 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.343+0000 7f6dbd457640 1 --2- 172.21.15.1:0/112596902 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557ece7d20c0 0x557ece7d08b0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f6db001fc10 tx=0x7f6db001fc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7cb41b223621ed1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:59.350 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.343+0000 7f6dae7fc640 1 -- 172.21.15.1:0/112596902 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db00097d0 con 0x557ece7d20c0 2024-09-11T22:03:59.350 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.343+0000 7f6dae7fc640 1 -- 172.21.15.1:0/112596902 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6db0023030 con 0x557ece7d20c0 2024-09-11T22:03:59.350 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.343+0000 7f6dae7fc640 1 -- 172.21.15.1:0/112596902 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db0004660 con 0x557ece7d20c0 2024-09-11T22:03:59.350 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.344+0000 7f6dc1c5b740 1 -- 172.21.15.1:0/112596902 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557ece7d20c0 msgr2=0x557ece7d08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:03:59.350 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.344+0000 7f6dc1c5b740 1 --2- 172.21.15.1:0/112596902 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557ece7d20c0 0x557ece7d08b0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f6db001fc10 tx=0x7f6db001fc40 comp rx=0 tx=0).stop 2024-09-11T22:03:59.350 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.344+0000 7f6dc1c5b740 1 -- 172.21.15.1:0/112596902 shutdown_connections 2024-09-11T22:03:59.351 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.344+0000 7f6dc1c5b740 1 --2- 172.21.15.1:0/112596902 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557ece7d20c0 0x557ece7d08b0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:59.351 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.344+0000 7f6dc1c5b740 1 --2- 172.21.15.1:0/112596902 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557ece7c32f0 0x557ece7ea4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:59.351 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.344+0000 7f6dc1c5b740 1 --2- 172.21.15.1:0/112596902 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557ece7c29d0 0x557ece7e9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:59.351 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.344+0000 7f6dc1c5b740 1 -- 172.21.15.1:0/112596902 >> 172.21.15.1:0/112596902 conn(0x557ece7bf060 msgr2=0x557ece7ce120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:03:59.351 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.344+0000 7f6dc1c5b740 1 -- 172.21.15.1:0/112596902 shutdown_connections 2024-09-11T22:03:59.351 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.344+0000 7f6dc1c5b740 1 -- 172.21.15.1:0/112596902 wait complete. 2024-09-11T22:03:59.351 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.344+0000 7f6dc1c5b740 1 -- start start 2024-09-11T22:03:59.351 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.344+0000 7f6dc1c5b740 1 Processor -- start 2024-09-11T22:03:59.352 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.345+0000 7f6dc1c5b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557ece7c29d0 0x557ece7da330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:59.352 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.345+0000 7f6dc1c5b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557ece7c32f0 0x557ece7da870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:59.352 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.345+0000 7f6dc1c5b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557ece7d20c0 0x557ece7d5700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:59.352 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.345+0000 7f6dc1c5b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6db81acd80 con 0x557ece7d20c0 2024-09-11T22:03:59.352 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.345+0000 7f6dc1c5b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6db81ac5f0 con 0x557ece7c29d0 2024-09-11T22:03:59.352 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.345+0000 7f6dc1c5b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6db81a96b0 con 0x557ece7c32f0 2024-09-11T22:03:59.352 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.345+0000 7f6dc1c5b740 10 client.0 _ll_register_callbacks cb 0x7f6da8009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:03:59.352 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.345+0000 7f6dbd457640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557ece7d20c0 0x557ece7d5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:59.352 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.345+0000 7f6dbcc56640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557ece7c29d0 0x557ece7da330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:59.352 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.345+0000 7f6db7fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557ece7c32f0 0x557ece7da870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:59.352 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.345+0000 7f6dbcc56640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557ece7c29d0 0x557ece7da330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:37624/0 (socket says 172.21.15.1:37624) 2024-09-11T22:03:59.353 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.345+0000 7f6dbd457640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557ece7d20c0 0x557ece7d5700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56884/0 (socket says 172.21.15.1:56884) 2024-09-11T22:03:59.353 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.345+0000 7f6db7fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557ece7c32f0 0x557ece7da870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:37124/0 (socket says 172.21.15.1:37124) 2024-09-11T22:03:59.353 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.345+0000 7f6dbcc56640 1 -- 172.21.15.1:0/1269170135 learned_addr learned my addr 172.21.15.1:0/1269170135 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:03:59.353 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.345+0000 7f6dbd457640 1 -- 172.21.15.1:0/1269170135 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557ece7c32f0 msgr2=0x557ece7da870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:59.353 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.345+0000 7f6dbd457640 1 --2- 172.21.15.1:0/1269170135 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557ece7c32f0 0x557ece7da870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:59.353 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.345+0000 7f6dbd457640 1 -- 172.21.15.1:0/1269170135 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557ece7c29d0 msgr2=0x557ece7da330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:03:59.353 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.345+0000 7f6dbd457640 1 --2- 172.21.15.1:0/1269170135 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557ece7c29d0 0x557ece7da330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:03:59.353 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.345+0000 7f6dbd457640 1 -- 172.21.15.1:0/1269170135 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ece7d5f30 con 0x557ece7d20c0 2024-09-11T22:03:59.353 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.345+0000 7f6dbd457640 1 --2- 172.21.15.1:0/1269170135 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557ece7d20c0 0x557ece7d5700 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f6db0016090 tx=0x7f6db001fb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55460d9dff3927bd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:03:59.353 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.345+0000 7f6d97fff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:03:59.353 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.345+0000 7f6d97fff640 1 -- 172.21.15.1:0/1269170135 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db003f020 con 0x557ece7d20c0 2024-09-11T22:03:59.353 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.346+0000 7f6dc1c5b740 1 -- 172.21.15.1:0/1269170135 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6db0023030 con 0x557ece7d20c0 2024-09-11T22:03:59.353 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.346+0000 7f6d97fff640 1 -- 172.21.15.1:0/1269170135 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f6db0005040 con 0x557ece7d20c0 2024-09-11T22:03:59.354 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.346+0000 7f6d97fff640 1 -- 172.21.15.1:0/1269170135 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f6db00097d0 con 0x557ece7d20c0 2024-09-11T22:03:59.354 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.346+0000 7f6dc1c5b740 10 client.16833 fetch_fsmap learned FSMap version 42 2024-09-11T22:03:59.354 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.346+0000 7f6dc1c5b740 1 -- 172.21.15.1:0/1269170135 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x557ece84caf0 con 0x557ece7d20c0 2024-09-11T22:03:59.354 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.346+0000 7f6d97fff640 1 -- 172.21.15.1:0/1269170135 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6db0023030 con 0x557ece7d20c0 2024-09-11T22:03:59.354 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.346+0000 7f6d97fff640 1 -- 172.21.15.1:0/1269170135 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f6db00049c0 con 0x557ece7d20c0 2024-09-11T22:03:59.354 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.346+0000 7f6dc1c5b740 10 client.16833 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:03:59.354 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.346+0000 7f6dc1c5b740 10 client.16833 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:03:59.354 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.346+0000 7f6dc1c5b740 20 client.16833 populate_metadata read hostname 'smithi001' 2024-09-11T22:03:59.354 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.346+0000 7f6dc1c5b740 1 --2- 172.21.15.1:0/1269170135 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x557ece84d100 0x557ece84f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:03:59.354 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.346+0000 7f6dc1c5b740 4 client.16833 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:03:59.354 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.346+0000 7f6dc1c5b740 1 -- 172.21.15.1:0/1269170135 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557ece72ed70 con 0x557ece84d100 2024-09-11T22:03:59.354 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.346+0000 7f6dbcc56640 1 --2- 172.21.15.1:0/1269170135 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x557ece84d100 0x557ece84f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:03:59.354 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.347+0000 7f6dbcc56640 1 --2- 172.21.15.1:0/1269170135 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x557ece84d100 0x557ece84f520 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=651c25e4a1889182 server_cookie=349532278bcb6f27 in_seq=0 out_seq=0 2024-09-11T22:03:59.355 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:03:59.347+0000 7f6d97fff640 10 client.16833 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:03:59.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:03:59 smithi001 ceph-mon[30935]: pgmap v207: 97 pgs: 97 active+clean; 344 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.1 MiB/s wr, 512 op/s 2024-09-11T22:03:59.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:03:59 smithi053 ceph-mon[34908]: pgmap v207: 97 pgs: 97 active+clean; 344 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.1 MiB/s wr, 512 op/s 2024-09-11T22:03:59.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:03:59 smithi161 ceph-mon[35615]: pgmap v207: 97 pgs: 97 active+clean; 344 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.1 MiB/s wr, 512 op/s 2024-09-11T22:04:00.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:04:00 smithi001 ceph-mon[30935]: pgmap v208: 97 pgs: 97 active+clean; 362 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.7 MiB/s wr, 738 op/s 2024-09-11T22:04:00.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:04:00 smithi053 ceph-mon[34908]: pgmap v208: 97 pgs: 97 active+clean; 362 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.7 MiB/s wr, 738 op/s 2024-09-11T22:04:00.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:04:00 smithi161 ceph-mon[35615]: pgmap v208: 97 pgs: 97 active+clean; 362 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.7 MiB/s wr, 738 op/s 2024-09-11T22:04:00.618 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:00.613+0000 7f6d97fff640 1 -- 172.21.15.1:0/1269170135 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557ece72ed70 con 0x557ece84d100 2024-09-11T22:04:00.618 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:00.613+0000 7f6d97fff640 10 client.16833 handle_command_reply: tid=0 2024-09-11T22:04:00.618 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:00.613+0000 7f6dc1c5b740 1 client.16833 shutdown 2024-09-11T22:04:00.618 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:00.613+0000 7f6dc1c5b740 1 -- 172.21.15.1:0/1269170135 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557ece7d20c0 msgr2=0x557ece7d5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:00.618 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:00.613+0000 7f6dc1c5b740 1 --2- 172.21.15.1:0/1269170135 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557ece7d20c0 0x557ece7d5700 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f6db0016090 tx=0x7f6db001fb20 comp rx=0 tx=0).stop 2024-09-11T22:04:00.618 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:00.613+0000 7f6dc1c5b740 1 -- 172.21.15.1:0/1269170135 shutdown_connections 2024-09-11T22:04:00.619 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:00.613+0000 7f6dc1c5b740 1 --2- 172.21.15.1:0/1269170135 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x557ece84d100 0x557ece84f520 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:00.619 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:00.613+0000 7f6dc1c5b740 1 --2- 172.21.15.1:0/1269170135 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557ece7d20c0 0x557ece7d5700 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:00.619 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:00.614+0000 7f6d97fff640 0 client.16833 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:04:00.619 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:00.614+0000 7f6dc1c5b740 1 --2- 172.21.15.1:0/1269170135 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557ece7c32f0 0x557ece7da870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:00.619 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:00.614+0000 7f6dc1c5b740 1 --2- 172.21.15.1:0/1269170135 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557ece7c29d0 0x557ece7da330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:00.619 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:00.614+0000 7f6dc1c5b740 1 -- 172.21.15.1:0/1269170135 >> 172.21.15.1:0/1269170135 conn(0x557ece7bf060 msgr2=0x557ece7ceff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:00.619 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:00.614+0000 7f6dc1c5b740 1 -- 172.21.15.1:0/1269170135 shutdown_connections 2024-09-11T22:04:00.619 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:00.614+0000 7f6dc1c5b740 1 -- 172.21.15.1:0/1269170135 wait complete. 2024-09-11T22:04:00.619 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:00.614+0000 7f6dc1c5b740 20 client.16833 trim_cache size 0 max 16384 2024-09-11T22:04:00.620 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-09-11T22:04:00.620 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-09-11T22:04:00.620 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-09-11T22:04:00.620 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-09-11T22:04:00.621 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:00.614+0000 7f6dbeeba640 1 -- 172.21.15.1:0/2418141790 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f6da40a2cf0 msgr2=0x7f6da40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:00.621 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:00.615+0000 7f6dbeeba640 1 --2- 172.21.15.1:0/2418141790 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f6da40a2cf0 0x7f6da40a51b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6db001e000 tx=0x7f6db0020000 comp rx=0 tx=0).stop 2024-09-11T22:04:00.621 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:00.615+0000 7f6dbeeba640 1 -- 172.21.15.1:0/2418141790 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6db8101340 msgr2=0x7f6db8198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:00.621 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:00.615+0000 7f6dbeeba640 1 --2- 172.21.15.1:0/2418141790 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6db8101340 0x7f6db8198930 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f6da002ae00 tx=0x7f6da0031e60 comp rx=0 tx=0).stop 2024-09-11T22:04:00.622 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:00.615+0000 7f6dbeeba640 1 -- 172.21.15.1:0/2418141790 shutdown_connections 2024-09-11T22:04:00.622 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:00.615+0000 7f6dbeeba640 1 --2- 172.21.15.1:0/2418141790 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f6da40a2cf0 0x7f6da40a51b0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:00.622 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:00.615+0000 7f6dbeeba640 1 --2- 172.21.15.1:0/2418141790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6db81069b0 0x7f6db819ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:00.622 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:00.615+0000 7f6dbeeba640 1 --2- 172.21.15.1:0/2418141790 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6db8101340 0x7f6db8198930 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:00.622 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:00.615+0000 7f6dbeeba640 1 --2- 172.21.15.1:0/2418141790 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6db81009a0 0x7f6db81983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:00.622 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:00.615+0000 7f6dbeeba640 1 -- 172.21.15.1:0/2418141790 >> 172.21.15.1:0/2418141790 conn(0x7f6db80779a0 msgr2=0x7f6db8069ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:00.622 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:00.615+0000 7f6dbeeba640 1 -- 172.21.15.1:0/2418141790 shutdown_connections 2024-09-11T22:04:00.622 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:00.615+0000 7f6dbeeba640 1 -- 172.21.15.1:0/2418141790 wait complete. 2024-09-11T22:04:03.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:04:02 smithi001 ceph-mon[30935]: pgmap v209: 97 pgs: 97 active+clean; 362 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.7 MiB/s wr, 467 op/s 2024-09-11T22:04:03.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:04:02 smithi053 ceph-mon[34908]: pgmap v209: 97 pgs: 97 active+clean; 362 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.7 MiB/s wr, 467 op/s 2024-09-11T22:04:03.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:04:02 smithi161 ceph-mon[35615]: pgmap v209: 97 pgs: 97 active+clean; 362 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.7 MiB/s wr, 467 op/s 2024-09-11T22:04:03.425 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:04:03.712 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:04:04.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.381+0000 7f0bbee88640 1 -- 172.21.15.1:0/778154761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0bb8100e20 msgr2=0x7f0bb8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:04.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.381+0000 7f0bbee88640 1 --2- 172.21.15.1:0/778154761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0bb8100e20 0x7f0bb8101200 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0bac009890 tx=0x7f0bac02a410 comp rx=0 tx=0).stop 2024-09-11T22:04:04.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.381+0000 7f0bbee88640 1 -- 172.21.15.1:0/778154761 shutdown_connections 2024-09-11T22:04:04.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.381+0000 7f0bbee88640 1 --2- 172.21.15.1:0/778154761 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0bb8102ef0 0x7f0bb810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:04.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.381+0000 7f0bbee88640 1 --2- 172.21.15.1:0/778154761 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0bb8102570 0x7f0bb81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:04.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.381+0000 7f0bbee88640 1 --2- 172.21.15.1:0/778154761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0bb8100e20 0x7f0bb8101200 secure :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0bac009890 tx=0x7f0bac02a410 comp rx=0 tx=0).stop 2024-09-11T22:04:04.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.381+0000 7f0bbee88640 1 -- 172.21.15.1:0/778154761 >> 172.21.15.1:0/778154761 conn(0x7f0bb80fc7d0 msgr2=0x7f0bb80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:04.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.382+0000 7f0bbee88640 1 -- 172.21.15.1:0/778154761 shutdown_connections 2024-09-11T22:04:04.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.382+0000 7f0bbee88640 1 -- 172.21.15.1:0/778154761 wait complete. 2024-09-11T22:04:04.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.382+0000 7f0bbee88640 1 Processor -- start 2024-09-11T22:04:04.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.382+0000 7f0bbee88640 1 -- start start 2024-09-11T22:04:04.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.383+0000 7f0bbee88640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0bb8102570 0x7f0bb81a0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:04.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.383+0000 7f0bbee88640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0bb8102ef0 0x7f0bb81a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:04.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.383+0000 7f0bbee88640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0bb819ada0 0x7f0bb819b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:04.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.383+0000 7f0bbee88640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb810feb0 con 0x7f0bb819ada0 2024-09-11T22:04:04.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.383+0000 7f0bbee88640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb810fd30 con 0x7f0bb8102ef0 2024-09-11T22:04:04.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.383+0000 7f0bbee88640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb8110030 con 0x7f0bb8102570 2024-09-11T22:04:04.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.383+0000 7f0bb7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0bb8102ef0 0x7f0bb81a0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:04.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.383+0000 7f0bb7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0bb8102ef0 0x7f0bb81a0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:37640/0 (socket says 172.21.15.1:37640) 2024-09-11T22:04:04.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.383+0000 7f0bb7fff640 1 -- 172.21.15.1:0/2844405585 learned_addr learned my addr 172.21.15.1:0/2844405585 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:04.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.383+0000 7f0bbcc24640 1 --2- 172.21.15.1:0/2844405585 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0bb8102570 0x7f0bb81a0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:04.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.383+0000 7f0bbd425640 1 --2- 172.21.15.1:0/2844405585 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0bb819ada0 0x7f0bb819b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:04.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.383+0000 7f0bbcc24640 1 -- 172.21.15.1:0/2844405585 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0bb8102ef0 msgr2=0x7f0bb81a0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:04.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.383+0000 7f0bbcc24640 1 --2- 172.21.15.1:0/2844405585 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0bb8102ef0 0x7f0bb81a0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:04.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.383+0000 7f0bbcc24640 1 -- 172.21.15.1:0/2844405585 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0bb819ada0 msgr2=0x7f0bb819b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:04.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.384+0000 7f0bbcc24640 1 --2- 172.21.15.1:0/2844405585 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0bb819ada0 0x7f0bb819b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:04.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.384+0000 7f0bbcc24640 1 -- 172.21.15.1:0/2844405585 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bb819b980 con 0x7f0bb8102570 2024-09-11T22:04:04.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.384+0000 7f0bbd425640 1 --2- 172.21.15.1:0/2844405585 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0bb819ada0 0x7f0bb819b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:04:04.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.384+0000 7f0bbcc24640 1 --2- 172.21.15.1:0/2844405585 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0bb8102570 0x7f0bb81a0710 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0bac008000 tx=0x7f0bac0309c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:04.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.384+0000 7f0bb5ffb640 1 -- 172.21.15.1:0/2844405585 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bac040070 con 0x7f0bb8102570 2024-09-11T22:04:04.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.384+0000 7f0bbee88640 1 -- 172.21.15.1:0/2844405585 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bb81a7530 con 0x7f0bb8102570 2024-09-11T22:04:04.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.385+0000 7f0bbee88640 1 -- 172.21.15.1:0/2844405585 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bb81a78d0 con 0x7f0bb8102570 2024-09-11T22:04:04.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.385+0000 7f0bb5ffb640 1 -- 172.21.15.1:0/2844405585 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bac0055d0 con 0x7f0bb8102570 2024-09-11T22:04:04.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.385+0000 7f0bb5ffb640 1 -- 172.21.15.1:0/2844405585 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bac004e90 con 0x7f0bb8102570 2024-09-11T22:04:04.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.391+0000 7f0bc1261740 1 Processor -- start 2024-09-11T22:04:04.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.391+0000 7f0bc1261740 1 -- start start 2024-09-11T22:04:04.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.391+0000 7f0bb5ffb640 1 -- 172.21.15.1:0/2844405585 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f0bac039040 con 0x7f0bb8102570 2024-09-11T22:04:04.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.391+0000 7f0bc1261740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5648ce8c83a0 0x5648ce8c8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:04.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.392+0000 7f0bb5ffb640 1 --2- 172.21.15.1:0/2844405585 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f0ba80a2f90 0x7f0ba80a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:04.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.393+0000 7f0bb5ffb640 1 -- 172.21.15.1:0/2844405585 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f0bac0f0090 con 0x7f0bb8102570 2024-09-11T22:04:04.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.393+0000 7f0bc1261740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5648ce8c8cc0 0x5648ce8db950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:04.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.393+0000 7f0bc1261740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648ce8dbe90 0x5648ce8de280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:04.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.393+0000 7f0bc1261740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5648ce82e2e0 con 0x5648ce8c83a0 2024-09-11T22:04:04.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.393+0000 7f0bc1261740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5648ce7d5ce0 con 0x5648ce8c8cc0 2024-09-11T22:04:04.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.393+0000 7f0bc1261740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5648ce82a2a0 con 0x5648ce8dbe90 2024-09-11T22:04:04.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.394+0000 7f0bbd425640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648ce8dbe90 0x5648ce8de280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:04.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.394+0000 7f0bbd425640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648ce8dbe90 0x5648ce8de280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:37144/0 (socket says 172.21.15.1:37144) 2024-09-11T22:04:04.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.394+0000 7f0bbd425640 1 -- 172.21.15.1:0/22072279 learned_addr learned my addr 172.21.15.1:0/22072279 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:04.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.394+0000 7f0bbd425640 1 --2- 172.21.15.1:0/22072279 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5648ce8c83a0 0x5648ce8c8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:04.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.394+0000 7f0bbd425640 1 -- 172.21.15.1:0/22072279 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5648ce8c8cc0 msgr2=0x5648ce8db950 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T22:04:04.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.394+0000 7f0bbd425640 1 --2- 172.21.15.1:0/22072279 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5648ce8c8cc0 0x5648ce8db950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:04.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.394+0000 7f0bbd425640 1 -- 172.21.15.1:0/22072279 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5648ce8c83a0 msgr2=0x5648ce8c8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:04.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.394+0000 7f0bbd425640 1 --2- 172.21.15.1:0/22072279 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5648ce8c83a0 0x5648ce8c8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:04.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.394+0000 7f0bbd425640 1 -- 172.21.15.1:0/22072279 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648ce7ba440 con 0x5648ce8dbe90 2024-09-11T22:04:04.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.395+0000 7f0bbcc24640 1 --2- 172.21.15.1:0/2844405585 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f0ba80a2f90 0x7f0ba80a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:04.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.395+0000 7f0bbd425640 1 --2- 172.21.15.1:0/22072279 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648ce8dbe90 0x5648ce8de280 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0012df0 tx=0x7f0bb0009dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2effce5afc8aead0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:04.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.395+0000 7f0bbcc24640 1 --2- 172.21.15.1:0/2844405585 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f0ba80a2f90 0x7f0ba80a5450 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0bac0c4c50 tx=0x7f0bac02a410 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:04.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.397+0000 7f0ba2ffd640 1 -- 172.21.15.1:0/22072279 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb0010c00 con 0x5648ce8dbe90 2024-09-11T22:04:04.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.397+0000 7f0ba2ffd640 1 -- 172.21.15.1:0/22072279 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bb0010da0 con 0x5648ce8dbe90 2024-09-11T22:04:04.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.397+0000 7f0ba2ffd640 1 -- 172.21.15.1:0/22072279 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb001d370 con 0x5648ce8dbe90 2024-09-11T22:04:04.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.399+0000 7f0bc1261740 1 -- 172.21.15.1:0/22072279 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648ce8dbe90 msgr2=0x5648ce8de280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:04.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.399+0000 7f0bc1261740 1 --2- 172.21.15.1:0/22072279 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648ce8dbe90 0x5648ce8de280 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0012df0 tx=0x7f0bb0009dd0 comp rx=0 tx=0).stop 2024-09-11T22:04:04.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.399+0000 7f0bc1261740 1 -- 172.21.15.1:0/22072279 shutdown_connections 2024-09-11T22:04:04.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.399+0000 7f0bc1261740 1 --2- 172.21.15.1:0/22072279 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648ce8dbe90 0x5648ce8de280 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:04.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.399+0000 7f0bc1261740 1 --2- 172.21.15.1:0/22072279 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5648ce8c8cc0 0x5648ce8db950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:04.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.399+0000 7f0bc1261740 1 --2- 172.21.15.1:0/22072279 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5648ce8c83a0 0x5648ce8c8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:04.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.399+0000 7f0bc1261740 1 -- 172.21.15.1:0/22072279 >> 172.21.15.1:0/22072279 conn(0x5648ce8c4a30 msgr2=0x5648ce8c4e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:04.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.399+0000 7f0bc1261740 1 -- 172.21.15.1:0/22072279 shutdown_connections 2024-09-11T22:04:04.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.400+0000 7f0bc1261740 1 -- 172.21.15.1:0/22072279 wait complete. 2024-09-11T22:04:04.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.401+0000 7f0bc1261740 1 -- start start 2024-09-11T22:04:04.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.401+0000 7f0bc1261740 1 Processor -- start 2024-09-11T22:04:04.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.402+0000 7f0bc1261740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648ce8c83a0 0x5648ce8db880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:04.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.402+0000 7f0bc1261740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5648ce8c8cc0 0x5648ce8d8b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:04.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.402+0000 7f0bc1261740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5648ce8dbe90 0x5648ce8d9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:04.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.402+0000 7f0bc1261740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5648ce8deac0 con 0x5648ce8c8cc0 2024-09-11T22:04:04.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.402+0000 7f0bc1261740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5648ce7e9050 con 0x5648ce8dbe90 2024-09-11T22:04:04.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.402+0000 7f0bc1261740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5648ce8dec40 con 0x5648ce8c83a0 2024-09-11T22:04:04.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.403+0000 7f0bc1261740 10 client.0 _ll_register_callbacks cb 0x5648ce829160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:04:04.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.403+0000 7f0bbd425640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648ce8c83a0 0x5648ce8db880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:04.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.403+0000 7f0bbd425640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648ce8c83a0 0x5648ce8db880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:37154/0 (socket says 172.21.15.1:37154) 2024-09-11T22:04:04.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.403+0000 7f0bbd425640 1 -- 172.21.15.1:0/87372873 learned_addr learned my addr 172.21.15.1:0/87372873 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:04.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.403+0000 7f0bb7fff640 1 --2- 172.21.15.1:0/87372873 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5648ce8c8cc0 0x5648ce8d8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:04.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.404+0000 7f0bbd425640 1 --2- 172.21.15.1:0/87372873 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5648ce8dbe90 0x5648ce8d9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:04.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.404+0000 7f0bbd425640 1 -- 172.21.15.1:0/87372873 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5648ce8dbe90 msgr2=0x5648ce8d9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:04.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.404+0000 7f0bbd425640 1 --2- 172.21.15.1:0/87372873 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5648ce8dbe90 0x5648ce8d9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:04.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.404+0000 7f0bbd425640 1 -- 172.21.15.1:0/87372873 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5648ce8c8cc0 msgr2=0x5648ce8d8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:04.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.404+0000 7f0bbd425640 1 --2- 172.21.15.1:0/87372873 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5648ce8c8cc0 0x5648ce8d8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:04.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.404+0000 7f0bbd425640 1 -- 172.21.15.1:0/87372873 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648ce59f3c0 con 0x5648ce8c83a0 2024-09-11T22:04:04.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.404+0000 7f0bbd425640 1 --2- 172.21.15.1:0/87372873 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648ce8c83a0 0x5648ce8db880 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0012dc0 tx=0x7f0bb0005580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66e155dc8c145903 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:04.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.404+0000 7f0b97fff640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-11T22:04:04.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.405+0000 7f0b97fff640 1 -- 172.21.15.1:0/87372873 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb0012a80 con 0x5648ce8c83a0 2024-09-11T22:04:04.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.405+0000 7f0b97fff640 1 -- 172.21.15.1:0/87372873 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f0bb0004a40 con 0x5648ce8c83a0 2024-09-11T22:04:04.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.406+0000 7f0bc1261740 1 -- 172.21.15.1:0/87372873 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bb0010da0 con 0x5648ce8c83a0 2024-09-11T22:04:04.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.407+0000 7f0b97fff640 1 -- 172.21.15.1:0/87372873 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f0bb0032050 con 0x5648ce8c83a0 2024-09-11T22:04:04.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.407+0000 7f0bc1261740 10 client.26344 fetch_fsmap learned FSMap version 42 2024-09-11T22:04:04.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.407+0000 7f0bc1261740 1 -- 172.21.15.1:0/87372873 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5648ce8ea5f0 con 0x5648ce8c83a0 2024-09-11T22:04:04.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.408+0000 7f0b97fff640 1 -- 172.21.15.1:0/87372873 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bb0004c40 con 0x5648ce8c83a0 2024-09-11T22:04:04.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.408+0000 7f0b97fff640 1 -- 172.21.15.1:0/87372873 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f0bb000f040 con 0x5648ce8c83a0 2024-09-11T22:04:04.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.409+0000 7f0bc1261740 10 client.26344 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:04:04.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.409+0000 7f0bc1261740 10 client.26344 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:04:04.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.409+0000 7f0bc1261740 20 client.26344 populate_metadata read hostname 'smithi001' 2024-09-11T22:04:04.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.409+0000 7f0bc1261740 1 --2- 172.21.15.1:0/87372873 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5648ce8eb940 0x5648ce8edd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:04.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.409+0000 7f0bc1261740 4 client.26344 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:04:04.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.409+0000 7f0bc1261740 1 -- 172.21.15.1:0/87372873 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0bb0010da0 con 0x5648ce8eb940 2024-09-11T22:04:04.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.409+0000 7f0bb7fff640 1 --2- 172.21.15.1:0/87372873 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5648ce8eb940 0x5648ce8edd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:04.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.410+0000 7f0bb7fff640 1 --2- 172.21.15.1:0/87372873 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5648ce8eb940 0x5648ce8edd30 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75f18512aeeaf1b8 server_cookie=2d9944229c2e5a7 in_seq=0 out_seq=0 2024-09-11T22:04:04.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.410+0000 7f0b97fff640 10 client.26344 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:04:04.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.411+0000 7f0b97fff640 1 -- 172.21.15.1:0/87372873 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f0bb0010da0 con 0x5648ce8eb940 2024-09-11T22:04:04.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.411+0000 7f0b97fff640 10 client.26344 handle_command_reply: tid=0 2024-09-11T22:04:04.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.411+0000 7f0bc1261740 1 client.26344 shutdown 2024-09-11T22:04:04.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.411+0000 7f0bc1261740 1 -- 172.21.15.1:0/87372873 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648ce8c83a0 msgr2=0x5648ce8db880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:04.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.411+0000 7f0bc1261740 1 --2- 172.21.15.1:0/87372873 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648ce8c83a0 0x5648ce8db880 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0012dc0 tx=0x7f0bb0005580 comp rx=0 tx=0).stop 2024-09-11T22:04:04.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.411+0000 7f0bc1261740 1 -- 172.21.15.1:0/87372873 shutdown_connections 2024-09-11T22:04:04.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.411+0000 7f0bc1261740 1 --2- 172.21.15.1:0/87372873 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5648ce8eb940 0x5648ce8edd30 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:04.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.412+0000 7f0bc1261740 1 --2- 172.21.15.1:0/87372873 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5648ce8dbe90 0x5648ce8d9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:04.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.412+0000 7f0bc1261740 1 --2- 172.21.15.1:0/87372873 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5648ce8c8cc0 0x5648ce8d8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:04.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.412+0000 7f0bc1261740 1 --2- 172.21.15.1:0/87372873 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648ce8c83a0 0x5648ce8db880 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:04.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.412+0000 7f0bc1261740 1 -- 172.21.15.1:0/87372873 >> 172.21.15.1:0/87372873 conn(0x5648ce8c4a30 msgr2=0x5648ce8ca790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:04.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.412+0000 7f0b97fff640 0 client.26344 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:04:04.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.412+0000 7f0bc1261740 1 -- 172.21.15.1:0/87372873 shutdown_connections 2024-09-11T22:04:04.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.412+0000 7f0bc1261740 1 -- 172.21.15.1:0/87372873 wait complete. 2024-09-11T22:04:04.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.412+0000 7f0bc1261740 20 client.26344 trim_cache size 0 max 16384 2024-09-11T22:04:04.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.453+0000 7f0bc1261740 1 Processor -- start 2024-09-11T22:04:04.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.453+0000 7f0bc1261740 1 -- start start 2024-09-11T22:04:04.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.453+0000 7f0bc1261740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5648ce8c83a0 0x5648ce8e7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:04.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.454+0000 7f0bc1261740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648ce8c8cc0 0x5648ce8f35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:04.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.454+0000 7f0bbd425640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5648ce8c83a0 0x5648ce8e7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:04.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.454+0000 7f0bbd425640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5648ce8c83a0 0x5648ce8e7df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:37664/0 (socket says 172.21.15.1:37664) 2024-09-11T22:04:04.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.454+0000 7f0bc1261740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5648ce8dbe90 0x5648ce8f3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:04.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.454+0000 7f0bbd425640 1 -- 172.21.15.1:0/163732986 learned_addr learned my addr 172.21.15.1:0/163732986 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:04.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.454+0000 7f0bb7fff640 1 --2- 172.21.15.1:0/163732986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648ce8c8cc0 0x5648ce8f35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:04.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.454+0000 7f0bc1261740 1 -- 172.21.15.1:0/163732986 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0b9c00e920 con 0x5648ce8dbe90 2024-09-11T22:04:04.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.454+0000 7f0bc1261740 1 -- 172.21.15.1:0/163732986 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0b9c010760 con 0x5648ce8c83a0 2024-09-11T22:04:04.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.454+0000 7f0bc1261740 1 -- 172.21.15.1:0/163732986 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0b9c00f0b0 con 0x5648ce8c8cc0 2024-09-11T22:04:04.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.455+0000 7f0bbd425640 1 --2- 172.21.15.1:0/163732986 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5648ce8dbe90 0x5648ce8f3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:04.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.455+0000 7f0bbd425640 1 -- 172.21.15.1:0/163732986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648ce8c8cc0 msgr2=0x5648ce8f35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:04.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.455+0000 7f0bbd425640 1 --2- 172.21.15.1:0/163732986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648ce8c8cc0 0x5648ce8f35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:04.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.455+0000 7f0bbd425640 1 -- 172.21.15.1:0/163732986 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5648ce8dbe90 msgr2=0x5648ce8f3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:04.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.455+0000 7f0bbd425640 1 --2- 172.21.15.1:0/163732986 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5648ce8dbe90 0x5648ce8f3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:04.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.455+0000 7f0bbd425640 1 -- 172.21.15.1:0/163732986 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648ce8d7080 con 0x5648ce8c83a0 2024-09-11T22:04:04.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.456+0000 7f0bbd425640 1 --2- 172.21.15.1:0/163732986 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5648ce8c83a0 0x5648ce8e7df0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0bb00129d0 tx=0x7f0bb0012960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4465d2195ff2d2fb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:04.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.456+0000 7f0ba27fc640 1 -- 172.21.15.1:0/163732986 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb003c070 con 0x5648ce8c83a0 2024-09-11T22:04:04.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.456+0000 7f0ba27fc640 1 -- 172.21.15.1:0/163732986 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bb0010d30 con 0x5648ce8c83a0 2024-09-11T22:04:04.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.456+0000 7f0ba27fc640 1 -- 172.21.15.1:0/163732986 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb002e0d0 con 0x5648ce8c83a0 2024-09-11T22:04:04.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.458+0000 7f0bc1261740 1 -- 172.21.15.1:0/163732986 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5648ce8c83a0 msgr2=0x5648ce8e7df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:04.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.458+0000 7f0bc1261740 1 --2- 172.21.15.1:0/163732986 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5648ce8c83a0 0x5648ce8e7df0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0bb00129d0 tx=0x7f0bb0012960 comp rx=0 tx=0).stop 2024-09-11T22:04:04.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.458+0000 7f0bc1261740 1 -- 172.21.15.1:0/163732986 shutdown_connections 2024-09-11T22:04:04.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.458+0000 7f0bc1261740 1 --2- 172.21.15.1:0/163732986 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5648ce8dbe90 0x5648ce8f3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:04.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.458+0000 7f0bc1261740 1 --2- 172.21.15.1:0/163732986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648ce8c8cc0 0x5648ce8f35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:04.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.458+0000 7f0bc1261740 1 --2- 172.21.15.1:0/163732986 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5648ce8c83a0 0x5648ce8e7df0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:04.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.458+0000 7f0bc1261740 1 -- 172.21.15.1:0/163732986 >> 172.21.15.1:0/163732986 conn(0x5648ce8c4a30 msgr2=0x5648ce8d7c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:04.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.458+0000 7f0bc1261740 1 -- 172.21.15.1:0/163732986 shutdown_connections 2024-09-11T22:04:04.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.458+0000 7f0bc1261740 1 -- 172.21.15.1:0/163732986 wait complete. 2024-09-11T22:04:04.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.459+0000 7f0bc1261740 1 -- start start 2024-09-11T22:04:04.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.459+0000 7f0bc1261740 1 Processor -- start 2024-09-11T22:04:04.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.459+0000 7f0bc1261740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648ce8c83a0 0x5648ce8df4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:04.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.460+0000 7f0bc1261740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5648ce8c8cc0 0x5648ce8dfa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:04.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.460+0000 7f0bc1261740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5648ce8dbe90 0x5648ce954570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:04.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.460+0000 7f0bc1261740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0b9c00ffd0 con 0x5648ce8dbe90 2024-09-11T22:04:04.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.460+0000 7f0bc1261740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0b9c00f840 con 0x5648ce8c8cc0 2024-09-11T22:04:04.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.460+0000 7f0bc1261740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0b9c00c900 con 0x5648ce8c83a0 2024-09-11T22:04:04.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.460+0000 7f0bc1261740 10 client.0 _ll_register_callbacks cb 0x7f0bb00128b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:04:04.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.460+0000 7f0bb7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5648ce8c8cc0 0x5648ce8dfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:04.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.460+0000 7f0bbd425640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648ce8c83a0 0x5648ce8df4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:04.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.460+0000 7f0bbd425640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648ce8c83a0 0x5648ce8df4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:37166/0 (socket says 172.21.15.1:37166) 2024-09-11T22:04:04.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.460+0000 7f0bbd425640 1 -- 172.21.15.1:0/2328799229 learned_addr learned my addr 172.21.15.1:0/2328799229 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:04.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.462+0000 7f0bbd425640 1 -- 172.21.15.1:0/2328799229 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5648ce8c8cc0 msgr2=0x5648ce8dfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:04.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.462+0000 7f0bbd425640 1 --2- 172.21.15.1:0/2328799229 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5648ce8c8cc0 0x5648ce8dfa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:04.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.462+0000 7f0bbd425640 1 -- 172.21.15.1:0/2328799229 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5648ce8dbe90 msgr2=0x5648ce954570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:04.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.462+0000 7f0bbd425640 1 --2- 172.21.15.1:0/2328799229 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5648ce8dbe90 0x5648ce954570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:04.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.462+0000 7f0bbd425640 1 -- 172.21.15.1:0/2328799229 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648ce954ab0 con 0x5648ce8c83a0 2024-09-11T22:04:04.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.463+0000 7f0bbd425640 1 --2- 172.21.15.1:0/2328799229 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648ce8c83a0 0x5648ce8df4c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0bb00098b0 tx=0x7f0bb0005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=141312bb9019b2f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:04.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.463+0000 7f0b97fff640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-11T22:04:04.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.464+0000 7f0b97fff640 1 -- 172.21.15.1:0/2328799229 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb003c070 con 0x5648ce8c83a0 2024-09-11T22:04:04.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.464+0000 7f0bc1261740 1 -- 172.21.15.1:0/2328799229 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bb0010d30 con 0x5648ce8c83a0 2024-09-11T22:04:04.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.464+0000 7f0b97fff640 1 -- 172.21.15.1:0/2328799229 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8585+0+0 (secure 0 0 0) 0x7f0bb002eb80 con 0x5648ce8c83a0 2024-09-11T22:04:04.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.464+0000 7f0b97fff640 1 -- 172.21.15.1:0/2328799229 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f0bb0048050 con 0x5648ce8c83a0 2024-09-11T22:04:04.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.465+0000 7f0bc1261740 10 client.26350 fetch_fsmap learned FSMap version 42 2024-09-11T22:04:04.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.465+0000 7f0bc1261740 1 -- 172.21.15.1:0/2328799229 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5648ce955d00 con 0x5648ce8c83a0 2024-09-11T22:04:04.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.468+0000 7f0b97fff640 1 -- 172.21.15.1:0/2328799229 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bb0010d30 con 0x5648ce8c83a0 2024-09-11T22:04:04.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.468+0000 7f0b97fff640 1 -- 172.21.15.1:0/2328799229 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f0bb001f500 con 0x5648ce8c83a0 2024-09-11T22:04:04.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.468+0000 7f0bc1261740 10 client.26350 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:04:04.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.468+0000 7f0bc1261740 10 client.26350 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:04:04.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.468+0000 7f0bc1261740 20 client.26350 populate_metadata read hostname 'smithi001' 2024-09-11T22:04:04.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.468+0000 7f0bc1261740 1 --2- 172.21.15.1:0/2328799229 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5648ce956400 0x5648ce958820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:04.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.468+0000 7f0bc1261740 4 client.26350 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:04:04.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.468+0000 7f0bc1261740 1 -- 172.21.15.1:0/2328799229 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5648ce83b850 con 0x5648ce956400 2024-09-11T22:04:04.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.469+0000 7f0bb7fff640 1 --2- 172.21.15.1:0/2328799229 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5648ce956400 0x5648ce958820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:04.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.469+0000 7f0b97fff640 10 client.26350 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:04:04.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.470+0000 7f0bb7fff640 1 --2- 172.21.15.1:0/2328799229 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5648ce956400 0x5648ce958820 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9000fcce21e3965 server_cookie=47bc207901009e28 in_seq=0 out_seq=0 2024-09-11T22:04:04.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.470+0000 7f0b97fff640 1 -- 172.21.15.1:0/2328799229 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5648ce83b850 con 0x5648ce956400 2024-09-11T22:04:04.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.470+0000 7f0b97fff640 10 client.26350 handle_command_reply: tid=0 2024-09-11T22:04:04.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.470+0000 7f0bc1261740 1 client.26350 shutdown 2024-09-11T22:04:04.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.470+0000 7f0bc1261740 1 -- 172.21.15.1:0/2328799229 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648ce8c83a0 msgr2=0x5648ce8df4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:04.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.470+0000 7f0bc1261740 1 --2- 172.21.15.1:0/2328799229 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648ce8c83a0 0x5648ce8df4c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0bb00098b0 tx=0x7f0bb0005fd0 comp rx=0 tx=0).stop 2024-09-11T22:04:04.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.471+0000 7f0bc1261740 1 -- 172.21.15.1:0/2328799229 shutdown_connections 2024-09-11T22:04:04.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.471+0000 7f0bc1261740 1 --2- 172.21.15.1:0/2328799229 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5648ce956400 0x5648ce958820 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:04.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.471+0000 7f0bc1261740 1 --2- 172.21.15.1:0/2328799229 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5648ce8dbe90 0x5648ce954570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:04.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.471+0000 7f0bc1261740 1 --2- 172.21.15.1:0/2328799229 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5648ce8c8cc0 0x5648ce8dfa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:04.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.471+0000 7f0bc1261740 1 --2- 172.21.15.1:0/2328799229 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5648ce8c83a0 0x5648ce8df4c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:04.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.471+0000 7f0bc1261740 1 -- 172.21.15.1:0/2328799229 >> 172.21.15.1:0/2328799229 conn(0x5648ce8c4a30 msgr2=0x5648ce8d38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:04.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.471+0000 7f0b97fff640 0 client.26350 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:04:04.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.472+0000 7f0bc1261740 1 -- 172.21.15.1:0/2328799229 shutdown_connections 2024-09-11T22:04:04.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.472+0000 7f0bc1261740 1 -- 172.21.15.1:0/2328799229 wait complete. 2024-09-11T22:04:04.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.472+0000 7f0bc1261740 20 client.26350 trim_cache size 0 max 16384 2024-09-11T22:04:04.475 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-11T22:04:04.475 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-09-11T22:04:04.475 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-09-11T22:04:04.475 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-11T22:04:04.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.475+0000 7f0bbee88640 1 -- 172.21.15.1:0/2844405585 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f0ba80a2f90 msgr2=0x7f0ba80a5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:04.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.475+0000 7f0bbee88640 1 --2- 172.21.15.1:0/2844405585 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f0ba80a2f90 0x7f0ba80a5450 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0bac0c4c50 tx=0x7f0bac02a410 comp rx=0 tx=0).stop 2024-09-11T22:04:04.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.475+0000 7f0bbee88640 1 -- 172.21.15.1:0/2844405585 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0bb8102570 msgr2=0x7f0bb81a0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:04.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.475+0000 7f0bbee88640 1 --2- 172.21.15.1:0/2844405585 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0bb8102570 0x7f0bb81a0710 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0bac008000 tx=0x7f0bac0309c0 comp rx=0 tx=0).stop 2024-09-11T22:04:04.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.475+0000 7f0bbee88640 1 -- 172.21.15.1:0/2844405585 shutdown_connections 2024-09-11T22:04:04.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.475+0000 7f0bbee88640 1 --2- 172.21.15.1:0/2844405585 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f0ba80a2f90 0x7f0ba80a5450 secure :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0bac0c4c50 tx=0x7f0bac02a410 comp rx=0 tx=0).stop 2024-09-11T22:04:04.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.475+0000 7f0bbee88640 1 --2- 172.21.15.1:0/2844405585 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0bb819ada0 0x7f0bb819b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:04.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.475+0000 7f0bbee88640 1 --2- 172.21.15.1:0/2844405585 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0bb8102ef0 0x7f0bb81a0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:04.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.475+0000 7f0bbee88640 1 --2- 172.21.15.1:0/2844405585 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0bb8102570 0x7f0bb81a0710 secure :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0bac008000 tx=0x7f0bac0309c0 comp rx=0 tx=0).stop 2024-09-11T22:04:04.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.475+0000 7f0bbee88640 1 -- 172.21.15.1:0/2844405585 >> 172.21.15.1:0/2844405585 conn(0x7f0bb80fc7d0 msgr2=0x7f0bb8110630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:04.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.477+0000 7f0bbee88640 1 -- 172.21.15.1:0/2844405585 shutdown_connections 2024-09-11T22:04:04.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:04.478+0000 7f0bbee88640 1 -- 172.21.15.1:0/2844405585 wait complete. 2024-09-11T22:04:05.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:04:04 smithi001 ceph-mon[30935]: pgmap v210: 97 pgs: 97 active+clean; 362 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.7 MiB/s wr, 467 op/s 2024-09-11T22:04:05.252 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T22:04:05.252 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:04:05.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:04:04 smithi053 ceph-mon[34908]: pgmap v210: 97 pgs: 97 active+clean; 362 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.7 MiB/s wr, 467 op/s 2024-09-11T22:04:05.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:04:04 smithi161 ceph-mon[35615]: pgmap v210: 97 pgs: 97 active+clean; 362 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.7 MiB/s wr, 467 op/s 2024-09-11T22:04:05.604 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:04:06.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:06.147+0000 7f5736a0d640 1 -- 172.21.15.1:0/3784388449 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5730102ef0 msgr2=0x7f573010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:06.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:06.147+0000 7f5736a0d640 1 --2- 172.21.15.1:0/3784388449 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5730102ef0 0x7f573010da40 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f5728009ca0 tx=0x7f572802a950 comp rx=0 tx=0).stop 2024-09-11T22:04:06.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:06.147+0000 7f5736a0d640 1 -- 172.21.15.1:0/3784388449 shutdown_connections 2024-09-11T22:04:06.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:06.147+0000 7f5736a0d640 1 --2- 172.21.15.1:0/3784388449 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5730102ef0 0x7f573010da40 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:06.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:06.147+0000 7f5736a0d640 1 --2- 172.21.15.1:0/3784388449 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5730102570 0x7f57301029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:06.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:06.147+0000 7f5736a0d640 1 --2- 172.21.15.1:0/3784388449 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5730100e20 0x7f5730101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:06.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:06.147+0000 7f5736a0d640 1 -- 172.21.15.1:0/3784388449 >> 172.21.15.1:0/3784388449 conn(0x7f57300fc7d0 msgr2=0x7f57300febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:06.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:06.147+0000 7f5736a0d640 1 -- 172.21.15.1:0/3784388449 shutdown_connections 2024-09-11T22:04:06.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:06.147+0000 7f5736a0d640 1 -- 172.21.15.1:0/3784388449 wait complete. 2024-09-11T22:04:06.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:06.148+0000 7f5736a0d640 1 Processor -- start 2024-09-11T22:04:06.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:06.148+0000 7f5736a0d640 1 -- start start 2024-09-11T22:04:06.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:06.149+0000 7f5736a0d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5730100e20 0x7f57301984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:06.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:06.149+0000 7f5736a0d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5730102570 0x7f57301989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:06.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:06.149+0000 7f5736a0d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5730102ef0 0x7f573019cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:06.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:06.149+0000 7f5736a0d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f57301120d0 con 0x7f5730102570 2024-09-11T22:04:06.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:06.149+0000 7f5736a0d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5730111f50 con 0x7f5730100e20 2024-09-11T22:04:06.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:06.149+0000 7f5736a0d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5730112250 con 0x7f5730102ef0 2024-09-11T22:04:06.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:06.149+0000 7f5727fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5730102570 0x7f57301989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:06.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:06.149+0000 7f5734faa640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5730102ef0 0x7f573019cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:06.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:06.149+0000 7f5727fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5730102570 0x7f57301989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56970/0 (socket says 172.21.15.1:56970) 2024-09-11T22:04:06.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:06.149+0000 7f5734faa640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5730102ef0 0x7f573019cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:37176/0 (socket says 172.21.15.1:37176) 2024-09-11T22:04:06.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:06.149+0000 7f5727fff640 1 -- 172.21.15.1:0/586186786 learned_addr learned my addr 172.21.15.1:0/586186786 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:06.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:06.149+0000 7f572ffff640 1 --2- 172.21.15.1:0/586186786 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5730100e20 0x7f57301984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:06.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:06.149+0000 7f5734faa640 1 -- 172.21.15.1:0/586186786 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5730100e20 msgr2=0x7f57301984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:06.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:06.149+0000 7f5734faa640 1 --2- 172.21.15.1:0/586186786 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5730100e20 0x7f57301984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:06.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:06.149+0000 7f5734faa640 1 -- 172.21.15.1:0/586186786 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5730102570 msgr2=0x7f57301989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:06.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:06.149+0000 7f5734faa640 1 --2- 172.21.15.1:0/586186786 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5730102570 0x7f57301989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:06.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:06.149+0000 7f5734faa640 1 -- 172.21.15.1:0/586186786 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f573019d460 con 0x7f5730102ef0 2024-09-11T22:04:06.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:06.149+0000 7f5727fff640 1 --2- 172.21.15.1:0/586186786 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5730102570 0x7f57301989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:04:06.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:06.149+0000 7f572ffff640 1 --2- 172.21.15.1:0/586186786 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5730100e20 0x7f57301984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:04:06.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:06.150+0000 7f5734faa640 1 --2- 172.21.15.1:0/586186786 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5730102ef0 0x7f573019cd80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5728005e30 tx=0x7f5728003b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:06.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:06.150+0000 7f572dffb640 1 -- 172.21.15.1:0/586186786 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5728034040 con 0x7f5730102ef0 2024-09-11T22:04:06.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:06.150+0000 7f572dffb640 1 -- 172.21.15.1:0/586186786 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f572802f930 con 0x7f5730102ef0 2024-09-11T22:04:06.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:06.150+0000 7f5736a0d640 1 -- 172.21.15.1:0/586186786 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f573019d6f0 con 0x7f5730102ef0 2024-09-11T22:04:06.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:06.150+0000 7f572dffb640 1 -- 172.21.15.1:0/586186786 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57280045d0 con 0x7f5730102ef0 2024-09-11T22:04:06.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:06.150+0000 7f5736a0d640 1 -- 172.21.15.1:0/586186786 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f573010e400 con 0x7f5730102ef0 2024-09-11T22:04:06.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:06.151+0000 7f5736a0d640 1 -- 172.21.15.1:0/586186786 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57301021b0 con 0x7f5730102ef0 2024-09-11T22:04:06.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:06.152+0000 7f572dffb640 1 -- 172.21.15.1:0/586186786 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f5728037020 con 0x7f5730102ef0 2024-09-11T22:04:06.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:06.153+0000 7f572dffb640 1 --2- 172.21.15.1:0/586186786 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f571c0a2cf0 0x7f571c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:06.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:06.153+0000 7f572dffb640 1 -- 172.21.15.1:0/586186786 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8673+0+0 (secure 0 0 0) 0x7f57280effe0 con 0x7f5730102ef0 2024-09-11T22:04:06.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:06.153+0000 7f572ffff640 1 --2- 172.21.15.1:0/586186786 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f571c0a2cf0 0x7f571c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:06.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:06.154+0000 7f572ffff640 1 --2- 172.21.15.1:0/586186786 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f571c0a2cf0 0x7f571c0a51b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5720009fc0 tx=0x7f5720005f90 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:06.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:06.157+0000 7f572dffb640 1 -- 172.21.15.1:0/586186786 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f57280bcc50 con 0x7f5730102ef0 2024-09-11T22:04:06.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:06.462+0000 7f5736a0d640 1 -- 172.21.15.1:0/586186786 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f573010e660 con 0x7f5730102ef0 2024-09-11T22:04:06.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:06.463+0000 7f572dffb640 1 -- 172.21.15.1:0/586186786 <== mon.1 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11984 (secure 0 0 0) 0x7f57280bc550 con 0x7f5730102ef0 2024-09-11T22:04:06.465 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T22:04:06.465 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":42,"btime":"2024-09-11T22:03:27:040852+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/421542832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":421542832},{"type":"v1","addr":"172.21.15.1:6835","nonce":421542832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2836864406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2836864406},{"type":"v1","addr":"172.21.15.1:6837","nonce":2836864406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":42,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T21:55:20.791263+0000","modified":"2024-09-11T22:03:26.485134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24484,"mds_2":24467,"mds_3":24466,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14595":{"gid":14595,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/407834672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":407834672},{"type":"v1","addr":"172.21.15.1:6839","nonce":407834672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/1068827678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":1068827678},{"type":"v1","addr":"172.21.15.1:6841","nonce":1068827678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/872410024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":872410024},{"type":"v1","addr":"172.21.15.161:6833","nonce":872410024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/1648944570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1648944570},{"type":"v1","addr":"172.21.15.53:6833","nonce":1648944570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/2126933993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2126933993},{"type":"v1","addr":"172.21.15.53:6835","nonce":2126933993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.161:6835/828381592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":828381592},{"type":"v1","addr":"172.21.15.161:6835","nonce":828381592}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.161:6837/1799318656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1799318656},{"type":"v1","addr":"172.21.15.161:6837","nonce":1799318656}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.53:6837/3457131830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3457131830},{"type":"v1","addr":"172.21.15.53:6837","nonce":3457131830}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/455365383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":455365383},{"type":"v1","addr":"172.21.15.161:6839","nonce":455365383}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.53:6839/1378584447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1378584447},{"type":"v1","addr":"172.21.15.53:6839","nonce":1378584447}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24460,24466,24467,24484,24494]},"id":1}]} 2024-09-11T22:04:06.465 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 42 2024-09-11T22:04:06.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:06.466+0000 7f5736a0d640 1 -- 172.21.15.1:0/586186786 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f571c0a2cf0 msgr2=0x7f571c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:06.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:06.466+0000 7f5736a0d640 1 --2- 172.21.15.1:0/586186786 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f571c0a2cf0 0x7f571c0a51b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5720009fc0 tx=0x7f5720005f90 comp rx=0 tx=0).stop 2024-09-11T22:04:06.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:06.466+0000 7f5736a0d640 1 -- 172.21.15.1:0/586186786 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5730102ef0 msgr2=0x7f573019cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:06.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:06.466+0000 7f5736a0d640 1 --2- 172.21.15.1:0/586186786 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5730102ef0 0x7f573019cd80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5728005e30 tx=0x7f5728003b80 comp rx=0 tx=0).stop 2024-09-11T22:04:06.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:06.466+0000 7f5736a0d640 1 -- 172.21.15.1:0/586186786 shutdown_connections 2024-09-11T22:04:06.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:06.466+0000 7f5736a0d640 1 --2- 172.21.15.1:0/586186786 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f571c0a2cf0 0x7f571c0a51b0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:06.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:06.466+0000 7f5736a0d640 1 --2- 172.21.15.1:0/586186786 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5730102ef0 0x7f573019cd80 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:06.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:06.466+0000 7f5736a0d640 1 --2- 172.21.15.1:0/586186786 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5730102570 0x7f57301989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:06.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:06.466+0000 7f5736a0d640 1 --2- 172.21.15.1:0/586186786 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5730100e20 0x7f57301984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:06.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:06.466+0000 7f5736a0d640 1 -- 172.21.15.1:0/586186786 >> 172.21.15.1:0/586186786 conn(0x7f57300fc7d0 msgr2=0x7f5730106830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:06.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:06.466+0000 7f5736a0d640 1 -- 172.21.15.1:0/586186786 shutdown_connections 2024-09-11T22:04:06.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:06.466+0000 7f5736a0d640 1 -- 172.21.15.1:0/586186786 wait complete. 2024-09-11T22:04:07.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:04:06 smithi001 ceph-mon[30935]: pgmap v211: 97 pgs: 97 active+clean; 380 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.0 MiB/s wr, 776 op/s 2024-09-11T22:04:07.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:04:06 smithi001 ceph-mon[30935]: osdmap e90: 12 total, 12 up, 12 in 2024-09-11T22:04:07.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:04:06 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/586186786' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:04:07.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:04:06 smithi053 ceph-mon[34908]: pgmap v211: 97 pgs: 97 active+clean; 380 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.0 MiB/s wr, 776 op/s 2024-09-11T22:04:07.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:04:06 smithi053 ceph-mon[34908]: osdmap e90: 12 total, 12 up, 12 in 2024-09-11T22:04:07.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:04:06 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/586186786' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:04:07.300 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T22:04:07.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:04:06 smithi161 ceph-mon[35615]: pgmap v211: 97 pgs: 97 active+clean; 380 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.0 MiB/s wr, 776 op/s 2024-09-11T22:04:07.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:04:06 smithi161 ceph-mon[35615]: osdmap e90: 12 total, 12 up, 12 in 2024-09-11T22:04:07.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:04:06 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/586186786' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:04:07.634 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:04:08.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.167+0000 7fab74749640 1 -- 172.21.15.1:0/2296260556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab6c101dd0 msgr2=0x7fab6c1021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:08.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.167+0000 7fab74749640 1 --2- 172.21.15.1:0/2296260556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab6c101dd0 0x7fab6c1021b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fab68009900 tx=0x7fab6802a3f0 comp rx=0 tx=0).stop 2024-09-11T22:04:08.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.167+0000 7fab74749640 1 -- 172.21.15.1:0/2296260556 shutdown_connections 2024-09-11T22:04:08.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.167+0000 7fab74749640 1 --2- 172.21.15.1:0/2296260556 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fab6c106f00 0x7fab6c10ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:08.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.167+0000 7fab74749640 1 --2- 172.21.15.1:0/2296260556 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab6c102780 0x7fab6c1067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:08.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.167+0000 7fab74749640 1 --2- 172.21.15.1:0/2296260556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab6c101dd0 0x7fab6c1021b0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:08.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.167+0000 7fab74749640 1 -- 172.21.15.1:0/2296260556 >> 172.21.15.1:0/2296260556 conn(0x7fab6c0fd890 msgr2=0x7fab6c0ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:08.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.167+0000 7fab74749640 1 -- 172.21.15.1:0/2296260556 shutdown_connections 2024-09-11T22:04:08.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.167+0000 7fab74749640 1 -- 172.21.15.1:0/2296260556 wait complete. 2024-09-11T22:04:08.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.168+0000 7fab74749640 1 Processor -- start 2024-09-11T22:04:08.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.168+0000 7fab74749640 1 -- start start 2024-09-11T22:04:08.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.168+0000 7fab74749640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab6c101dd0 0x7fab6c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:08.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.168+0000 7fab74749640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab6c102780 0x7fab6c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:08.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.168+0000 7fab74749640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fab6c106f00 0x7fab6c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:08.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.168+0000 7fab74749640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fab6c111f20 con 0x7fab6c106f00 2024-09-11T22:04:08.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.168+0000 7fab74749640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fab6c111da0 con 0x7fab6c101dd0 2024-09-11T22:04:08.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.168+0000 7fab74749640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fab6c1120a0 con 0x7fab6c102780 2024-09-11T22:04:08.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.168+0000 7fab724e5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab6c101dd0 0x7fab6c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:08.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.168+0000 7fab71ce4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab6c102780 0x7fab6c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:08.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.168+0000 7fab72ce6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fab6c106f00 0x7fab6c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:08.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.168+0000 7fab724e5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab6c101dd0 0x7fab6c0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:50672/0 (socket says 172.21.15.1:50672) 2024-09-11T22:04:08.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.168+0000 7fab724e5640 1 -- 172.21.15.1:0/2036818651 learned_addr learned my addr 172.21.15.1:0/2036818651 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:08.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.168+0000 7fab724e5640 1 -- 172.21.15.1:0/2036818651 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab6c102780 msgr2=0x7fab6c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:08.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.168+0000 7fab724e5640 1 --2- 172.21.15.1:0/2036818651 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab6c102780 0x7fab6c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:08.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.168+0000 7fab724e5640 1 -- 172.21.15.1:0/2036818651 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fab6c106f00 msgr2=0x7fab6c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:08.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.168+0000 7fab724e5640 1 --2- 172.21.15.1:0/2036818651 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fab6c106f00 0x7fab6c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:08.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.168+0000 7fab724e5640 1 -- 172.21.15.1:0/2036818651 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab6c1a68f0 con 0x7fab6c101dd0 2024-09-11T22:04:08.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.168+0000 7fab72ce6640 1 --2- 172.21.15.1:0/2036818651 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fab6c106f00 0x7fab6c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:04:08.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.169+0000 7fab724e5640 1 --2- 172.21.15.1:0/2036818651 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab6c101dd0 0x7fab6c0744f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fab68006010 tx=0x7fab68004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:08.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.169+0000 7fab637fe640 1 -- 172.21.15.1:0/2036818651 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab6803f030 con 0x7fab6c101dd0 2024-09-11T22:04:08.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.169+0000 7fab74749640 1 -- 172.21.15.1:0/2036818651 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab6c1a6b80 con 0x7fab6c101dd0 2024-09-11T22:04:08.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.169+0000 7fab637fe640 1 -- 172.21.15.1:0/2036818651 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab68004440 con 0x7fab6c101dd0 2024-09-11T22:04:08.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.169+0000 7fab637fe640 1 -- 172.21.15.1:0/2036818651 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab68035070 con 0x7fab6c101dd0 2024-09-11T22:04:08.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.169+0000 7fab74749640 1 -- 172.21.15.1:0/2036818651 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab6c1a70b0 con 0x7fab6c101dd0 2024-09-11T22:04:08.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.171+0000 7fab637fe640 1 -- 172.21.15.1:0/2036818651 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7fab68042030 con 0x7fab6c101dd0 2024-09-11T22:04:08.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.172+0000 7fab76b22740 1 Processor -- start 2024-09-11T22:04:08.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.172+0000 7fab76b22740 1 -- start start 2024-09-11T22:04:08.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.172+0000 7fab76b22740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f5c3ea1b40 0x55f5c3ea0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:08.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.172+0000 7fab76b22740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5c3ea05d0 0x55f5c3ea0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:08.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.172+0000 7fab76b22740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5c3eab370 0x55f5c3ead760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:08.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.172+0000 7fab76b22740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55f5c3e062f0 con 0x55f5c3ea05d0 2024-09-11T22:04:08.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.172+0000 7fab76b22740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55f5c3dadcf0 con 0x55f5c3eab370 2024-09-11T22:04:08.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.172+0000 7fab76b22740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f5c3e022b0 con 0x55f5c3ea1b40 2024-09-11T22:04:08.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.172+0000 7fab724e5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5c3ea05d0 0x55f5c3ea0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:08.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.172+0000 7fab72ce6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f5c3ea1b40 0x55f5c3ea0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:08.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.172+0000 7fab72ce6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f5c3ea1b40 0x55f5c3ea0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:50722/0 (socket says 172.21.15.1:50722) 2024-09-11T22:04:08.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.172+0000 7fab72ce6640 1 -- 172.21.15.1:0/35229203 learned_addr learned my addr 172.21.15.1:0/35229203 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:08.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.172+0000 7fab71ce4640 1 --2- 172.21.15.1:0/35229203 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5c3eab370 0x55f5c3ead760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:08.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.172+0000 7fab637fe640 1 --2- 172.21.15.1:0/2036818651 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fab580a2cf0 0x7fab580a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:08.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.173+0000 7fab72ce6640 1 -- 172.21.15.1:0/35229203 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5c3eab370 msgr2=0x55f5c3ead760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:08.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.173+0000 7fab72ce6640 1 --2- 172.21.15.1:0/35229203 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5c3eab370 0x55f5c3ead760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:08.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.173+0000 7fab72ce6640 1 -- 172.21.15.1:0/35229203 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5c3ea05d0 msgr2=0x55f5c3ea0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:08.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.173+0000 7fab72ce6640 1 --2- 172.21.15.1:0/35229203 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5c3ea05d0 0x55f5c3ea0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:08.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.173+0000 7fab72ce6640 1 -- 172.21.15.1:0/35229203 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5c3d923a0 con 0x55f5c3ea1b40 2024-09-11T22:04:08.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.173+0000 7fab71ce4640 1 --2- 172.21.15.1:0/35229203 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5c3eab370 0x55f5c3ead760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:04:08.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.173+0000 7fab637fe640 1 -- 172.21.15.1:0/2036818651 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8673+0+0 (secure 0 0 0) 0x7fab680ef830 con 0x7fab6c101dd0 2024-09-11T22:04:08.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.173+0000 7fab72ce6640 1 --2- 172.21.15.1:0/2036818651 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fab580a2cf0 0x7fab580a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:08.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.173+0000 7fab72ce6640 1 --2- 172.21.15.1:0/35229203 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f5c3ea1b40 0x55f5c3ea0090 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fab640118d0 tx=0x7fab64011d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=626bec1fad080cce server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:08.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.173+0000 7fab60ff9640 1 -- 172.21.15.1:0/35229203 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab64017050 con 0x55f5c3ea1b40 2024-09-11T22:04:08.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.173+0000 7fab72ce6640 1 --2- 172.21.15.1:0/2036818651 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fab580a2cf0 0x7fab580a51b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fab64023000 tx=0x7fab64024000 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:08.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.173+0000 7fab60ff9640 1 -- 172.21.15.1:0/35229203 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab6401bb90 con 0x55f5c3ea1b40 2024-09-11T22:04:08.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.173+0000 7fab60ff9640 1 -- 172.21.15.1:0/35229203 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab6401a360 con 0x55f5c3ea1b40 2024-09-11T22:04:08.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.174+0000 7fab76b22740 1 -- 172.21.15.1:0/35229203 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f5c3ea1b40 msgr2=0x55f5c3ea0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:08.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.174+0000 7fab76b22740 1 --2- 172.21.15.1:0/35229203 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f5c3ea1b40 0x55f5c3ea0090 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fab640118d0 tx=0x7fab64011d90 comp rx=0 tx=0).stop 2024-09-11T22:04:08.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.174+0000 7fab76b22740 1 -- 172.21.15.1:0/35229203 shutdown_connections 2024-09-11T22:04:08.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.174+0000 7fab76b22740 1 --2- 172.21.15.1:0/35229203 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5c3eab370 0x55f5c3ead760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:08.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.174+0000 7fab76b22740 1 --2- 172.21.15.1:0/35229203 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5c3ea05d0 0x55f5c3ea0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:08.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.174+0000 7fab76b22740 1 --2- 172.21.15.1:0/35229203 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f5c3ea1b40 0x55f5c3ea0090 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:08.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.174+0000 7fab76b22740 1 -- 172.21.15.1:0/35229203 >> 172.21.15.1:0/35229203 conn(0x55f5c3e69310 msgr2=0x55f5c3e9cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:08.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.174+0000 7fab76b22740 1 -- 172.21.15.1:0/35229203 shutdown_connections 2024-09-11T22:04:08.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.175+0000 7fab76b22740 1 -- 172.21.15.1:0/35229203 wait complete. 2024-09-11T22:04:08.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.175+0000 7fab76b22740 1 -- start start 2024-09-11T22:04:08.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.175+0000 7fab76b22740 1 Processor -- start 2024-09-11T22:04:08.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.175+0000 7fab76b22740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5c3ea05d0 0x55f5c3eaad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:08.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.176+0000 7fab76b22740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f5c3ea1b40 0x55f5c3ea8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:08.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.176+0000 7fab76b22740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5c3eab370 0x55f5c3ea86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:08.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.176+0000 7fab76b22740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55f5c3eae070 con 0x55f5c3eab370 2024-09-11T22:04:08.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.176+0000 7fab76b22740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55f5c3dc1060 con 0x55f5c3ea05d0 2024-09-11T22:04:08.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.176+0000 7fab76b22740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f5c3eae1f0 con 0x55f5c3ea1b40 2024-09-11T22:04:08.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.176+0000 7fab76b22740 10 client.0 _ll_register_callbacks cb 0x55f5c3b148c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:04:08.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.176+0000 7fab72ce6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5c3eab370 0x55f5c3ea86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:08.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.176+0000 7fab724e5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5c3ea05d0 0x55f5c3eaad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:08.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.176+0000 7fab72ce6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5c3eab370 0x55f5c3ea86a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55654/0 (socket says 172.21.15.1:55654) 2024-09-11T22:04:08.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.176+0000 7fab72ce6640 1 -- 172.21.15.1:0/1562156948 learned_addr learned my addr 172.21.15.1:0/1562156948 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:08.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.176+0000 7fab71ce4640 1 --2- 172.21.15.1:0/1562156948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f5c3ea1b40 0x55f5c3ea8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:08.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.176+0000 7fab72ce6640 1 -- 172.21.15.1:0/1562156948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f5c3ea1b40 msgr2=0x55f5c3ea8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:08.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.176+0000 7fab72ce6640 1 --2- 172.21.15.1:0/1562156948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f5c3ea1b40 0x55f5c3ea8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:08.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.176+0000 7fab72ce6640 1 -- 172.21.15.1:0/1562156948 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5c3ea05d0 msgr2=0x55f5c3eaad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:08.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.176+0000 7fab72ce6640 1 --2- 172.21.15.1:0/1562156948 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5c3ea05d0 0x55f5c3eaad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:08.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.176+0000 7fab72ce6640 1 -- 172.21.15.1:0/1562156948 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5c3b773c0 con 0x55f5c3eab370 2024-09-11T22:04:08.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.176+0000 7fab72ce6640 1 --2- 172.21.15.1:0/1562156948 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5c3eab370 0x55f5c3ea86a0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fab6400eaf0 tx=0x7fab6400e5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61be4f7d23b19da3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:08.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.176+0000 7fab55ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:04:08.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.176+0000 7fab55ffb640 1 -- 172.21.15.1:0/1562156948 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab64034060 con 0x55f5c3eab370 2024-09-11T22:04:08.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.176+0000 7fab55ffb640 1 -- 172.21.15.1:0/1562156948 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8673+0+0 (secure 0 0 0) 0x7fab64004c50 con 0x55f5c3eab370 2024-09-11T22:04:08.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.176+0000 7fab76b22740 1 -- 172.21.15.1:0/1562156948 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab6401bb90 con 0x55f5c3eab370 2024-09-11T22:04:08.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.177+0000 7fab55ffb640 1 -- 172.21.15.1:0/1562156948 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fab64017020 con 0x55f5c3eab370 2024-09-11T22:04:08.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.177+0000 7fab76b22740 10 client.16866 fetch_fsmap learned FSMap version 42 2024-09-11T22:04:08.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.177+0000 7fab76b22740 1 -- 172.21.15.1:0/1562156948 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55f5c3eb9b20 con 0x55f5c3eab370 2024-09-11T22:04:08.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.177+0000 7fab55ffb640 1 -- 172.21.15.1:0/1562156948 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab6401bb90 con 0x55f5c3eab370 2024-09-11T22:04:08.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.177+0000 7fab55ffb640 1 -- 172.21.15.1:0/1562156948 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7fab64005510 con 0x55f5c3eab370 2024-09-11T22:04:08.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.177+0000 7fab76b22740 10 client.16866 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:04:08.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.177+0000 7fab76b22740 10 client.16866 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-11T22:04:08.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.177+0000 7fab76b22740 20 client.16866 populate_metadata read hostname 'smithi001' 2024-09-11T22:04:08.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.177+0000 7fab76b22740 1 --2- 172.21.15.1:0/1562156948 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x55f5c3ebafe0 0x55f5c3ebd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:08.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.177+0000 7fab76b22740 4 client.16866 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:04:08.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.177+0000 7fab76b22740 1 -- 172.21.15.1:0/1562156948 --> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f5c3e13860 con 0x55f5c3ebafe0 2024-09-11T22:04:08.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.177+0000 7fab724e5640 1 --2- 172.21.15.1:0/1562156948 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x55f5c3ebafe0 0x55f5c3ebd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:08.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.178+0000 7fab724e5640 1 --2- 172.21.15.1:0/1562156948 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x55f5c3ebafe0 0x55f5c3ebd3d0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8226e26b051c84fa server_cookie=aedeb2b301d4014e in_seq=0 out_seq=0 2024-09-11T22:04:08.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.178+0000 7fab55ffb640 10 client.16866 ms_handle_connect on v2:172.21.15.161:6834/828381592 2024-09-11T22:04:08.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.181+0000 7fab55ffb640 1 -- 172.21.15.1:0/1562156948 <== mds.4 v2:172.21.15.161:6834/828381592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f5c3e13860 con 0x55f5c3ebafe0 2024-09-11T22:04:08.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.181+0000 7fab55ffb640 10 client.16866 handle_command_reply: tid=0 2024-09-11T22:04:08.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.182+0000 7fab76b22740 1 client.16866 shutdown 2024-09-11T22:04:08.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.182+0000 7fab76b22740 1 -- 172.21.15.1:0/1562156948 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5c3eab370 msgr2=0x55f5c3ea86a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:08.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.182+0000 7fab76b22740 1 --2- 172.21.15.1:0/1562156948 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5c3eab370 0x55f5c3ea86a0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fab6400eaf0 tx=0x7fab6400e5e0 comp rx=0 tx=0).stop 2024-09-11T22:04:08.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.182+0000 7fab76b22740 1 -- 172.21.15.1:0/1562156948 shutdown_connections 2024-09-11T22:04:08.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.182+0000 7fab76b22740 1 --2- 172.21.15.1:0/1562156948 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5c3eab370 0x55f5c3ea86a0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:08.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.182+0000 7fab76b22740 1 --2- 172.21.15.1:0/1562156948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f5c3ea1b40 0x55f5c3ea8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:08.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.182+0000 7fab76b22740 1 --2- 172.21.15.1:0/1562156948 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x55f5c3ebafe0 0x55f5c3ebd3d0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:08.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.182+0000 7fab76b22740 1 --2- 172.21.15.1:0/1562156948 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5c3ea05d0 0x55f5c3eaad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:08.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.182+0000 7fab76b22740 1 -- 172.21.15.1:0/1562156948 >> 172.21.15.1:0/1562156948 conn(0x55f5c3e69310 msgr2=0x55f5c3ea2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:08.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.182+0000 7fab55ffb640 0 client.16866 ms_handle_reset on v2:172.21.15.161:6834/828381592 2024-09-11T22:04:08.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.182+0000 7fab76b22740 1 -- 172.21.15.1:0/1562156948 shutdown_connections 2024-09-11T22:04:08.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.182+0000 7fab76b22740 1 -- 172.21.15.1:0/1562156948 wait complete. 2024-09-11T22:04:08.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.182+0000 7fab76b22740 20 client.16866 trim_cache size 0 max 16384 2024-09-11T22:04:08.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.195+0000 7fab76b22740 1 Processor -- start 2024-09-11T22:04:08.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.195+0000 7fab76b22740 1 -- start start 2024-09-11T22:04:08.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.195+0000 7fab76b22740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f5c3ea05d0 0x55f5c3eb7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:08.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.195+0000 7fab76b22740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5c3ea1b40 0x55f5c3ec2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:08.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.195+0000 7fab76b22740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5c3eab370 0x55f5c3ec31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:08.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.195+0000 7fab76b22740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fab6c1afd30 con 0x55f5c3eab370 2024-09-11T22:04:08.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.195+0000 7fab76b22740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fab6c1b1b70 con 0x55f5c3ea1b40 2024-09-11T22:04:08.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.195+0000 7fab76b22740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fab6c1b04c0 con 0x55f5c3ea05d0 2024-09-11T22:04:08.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.195+0000 7fab72ce6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5c3eab370 0x55f5c3ec31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:08.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.195+0000 7fab71ce4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5c3ea1b40 0x55f5c3ec2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:08.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.196+0000 7fab724e5640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f5c3ea05d0 0x55f5c3eb7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:08.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.196+0000 7fab71ce4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5c3ea1b40 0x55f5c3ec2c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:50704/0 (socket says 172.21.15.1:50704) 2024-09-11T22:04:08.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.196+0000 7fab724e5640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f5c3ea05d0 0x55f5c3eb7310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:50740/0 (socket says 172.21.15.1:50740) 2024-09-11T22:04:08.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.196+0000 7fab71ce4640 1 -- 172.21.15.1:0/932599017 learned_addr learned my addr 172.21.15.1:0/932599017 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:08.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.196+0000 7fab71ce4640 1 -- 172.21.15.1:0/932599017 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f5c3ea05d0 msgr2=0x55f5c3eb7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:08.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.196+0000 7fab71ce4640 1 --2- 172.21.15.1:0/932599017 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f5c3ea05d0 0x55f5c3eb7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:08.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.196+0000 7fab71ce4640 1 -- 172.21.15.1:0/932599017 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5c3eab370 msgr2=0x55f5c3ec31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:08.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.196+0000 7fab71ce4640 1 --2- 172.21.15.1:0/932599017 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5c3eab370 0x55f5c3ec31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:08.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.196+0000 7fab71ce4640 1 -- 172.21.15.1:0/932599017 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5c3e9c510 con 0x55f5c3ea1b40 2024-09-11T22:04:08.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.196+0000 7fab71ce4640 1 --2- 172.21.15.1:0/932599017 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5c3ea1b40 0x55f5c3ec2c80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fab5c009d30 tx=0x7fab5c005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c077b914e663f7f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:08.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.196+0000 7fab57fff640 1 -- 172.21.15.1:0/932599017 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5c0120d0 con 0x55f5c3ea1b40 2024-09-11T22:04:08.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.196+0000 7fab57fff640 1 -- 172.21.15.1:0/932599017 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab5c012270 con 0x55f5c3ea1b40 2024-09-11T22:04:08.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.196+0000 7fab57fff640 1 -- 172.21.15.1:0/932599017 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5c004e50 con 0x55f5c3ea1b40 2024-09-11T22:04:08.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.197+0000 7fab76b22740 1 -- 172.21.15.1:0/932599017 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5c3ea1b40 msgr2=0x55f5c3ec2c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:08.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.197+0000 7fab76b22740 1 --2- 172.21.15.1:0/932599017 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5c3ea1b40 0x55f5c3ec2c80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fab5c009d30 tx=0x7fab5c005f90 comp rx=0 tx=0).stop 2024-09-11T22:04:08.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.197+0000 7fab76b22740 1 -- 172.21.15.1:0/932599017 shutdown_connections 2024-09-11T22:04:08.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.197+0000 7fab76b22740 1 --2- 172.21.15.1:0/932599017 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5c3eab370 0x55f5c3ec31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:08.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.197+0000 7fab76b22740 1 --2- 172.21.15.1:0/932599017 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5c3ea1b40 0x55f5c3ec2c80 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:08.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.197+0000 7fab76b22740 1 --2- 172.21.15.1:0/932599017 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f5c3ea05d0 0x55f5c3eb7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:08.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.197+0000 7fab76b22740 1 -- 172.21.15.1:0/932599017 >> 172.21.15.1:0/932599017 conn(0x55f5c3e69310 msgr2=0x55f5c3ea6910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:08.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.197+0000 7fab76b22740 1 -- 172.21.15.1:0/932599017 shutdown_connections 2024-09-11T22:04:08.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.197+0000 7fab76b22740 1 -- 172.21.15.1:0/932599017 wait complete. 2024-09-11T22:04:08.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.198+0000 7fab76b22740 1 -- start start 2024-09-11T22:04:08.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.198+0000 7fab76b22740 1 Processor -- start 2024-09-11T22:04:08.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.198+0000 7fab76b22740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5c3ea05d0 0x55f5c3eb3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:08.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.198+0000 7fab76b22740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f5c3ea1b40 0x55f5c3ea4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:08.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.198+0000 7fab76b22740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5c3eab370 0x55f5c3ea51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:08.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.198+0000 7fab76b22740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fab6c1b13e0 con 0x55f5c3eab370 2024-09-11T22:04:08.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.198+0000 7fab76b22740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fab6c1b0c50 con 0x55f5c3ea05d0 2024-09-11T22:04:08.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.198+0000 7fab76b22740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fab6c1add30 con 0x55f5c3ea1b40 2024-09-11T22:04:08.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.198+0000 7fab76b22740 10 client.0 _ll_register_callbacks cb 0x7fab680b8080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:04:08.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.198+0000 7fab724e5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5c3ea05d0 0x55f5c3eb3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:08.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.198+0000 7fab71ce4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f5c3ea1b40 0x55f5c3ea4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:08.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.198+0000 7fab72ce6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5c3eab370 0x55f5c3ea51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:08.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.198+0000 7fab724e5640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5c3ea05d0 0x55f5c3eb3ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:50718/0 (socket says 172.21.15.1:50718) 2024-09-11T22:04:08.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.198+0000 7fab724e5640 1 -- 172.21.15.1:0/2545679393 learned_addr learned my addr 172.21.15.1:0/2545679393 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:08.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.198+0000 7fab72ce6640 1 --2- 172.21.15.1:0/2545679393 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5c3eab370 0x55f5c3ea51f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55668/0 (socket says 172.21.15.1:55668) 2024-09-11T22:04:08.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.198+0000 7fab724e5640 1 -- 172.21.15.1:0/2545679393 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f5c3ea1b40 msgr2=0x55f5c3ea4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:08.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.198+0000 7fab724e5640 1 --2- 172.21.15.1:0/2545679393 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f5c3ea1b40 0x55f5c3ea4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:08.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.198+0000 7fab724e5640 1 -- 172.21.15.1:0/2545679393 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5c3eab370 msgr2=0x55f5c3ea51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:08.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.198+0000 7fab724e5640 1 --2- 172.21.15.1:0/2545679393 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5c3eab370 0x55f5c3ea51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:08.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.198+0000 7fab724e5640 1 -- 172.21.15.1:0/2545679393 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5c3ea5930 con 0x55f5c3ea05d0 2024-09-11T22:04:08.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.198+0000 7fab72ce6640 1 --2- 172.21.15.1:0/2545679393 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5c3eab370 0x55f5c3ea51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:04:08.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.199+0000 7fab724e5640 1 --2- 172.21.15.1:0/2545679393 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5c3ea05d0 0x55f5c3eb3ac0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fab680ecf90 tx=0x7fab680bb780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a0c726de627ba2b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:08.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.199+0000 7fab55ffb640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-11T22:04:08.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.199+0000 7fab55ffb640 1 -- 172.21.15.1:0/2545679393 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab680f5040 con 0x55f5c3ea05d0 2024-09-11T22:04:08.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.199+0000 7fab55ffb640 1 -- 172.21.15.1:0/2545679393 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8673+0+0 (secure 0 0 0) 0x7fab68003cd0 con 0x55f5c3ea05d0 2024-09-11T22:04:08.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.199+0000 7fab76b22740 1 -- 172.21.15.1:0/2545679393 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab5c012270 con 0x55f5c3ea05d0 2024-09-11T22:04:08.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.199+0000 7fab55ffb640 1 -- 172.21.15.1:0/2545679393 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fab68049040 con 0x55f5c3ea05d0 2024-09-11T22:04:08.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.199+0000 7fab76b22740 10 client.26405 fetch_fsmap learned FSMap version 42 2024-09-11T22:04:08.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.199+0000 7fab76b22740 1 -- 172.21.15.1:0/2545679393 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55f5c3f25be0 con 0x55f5c3ea05d0 2024-09-11T22:04:08.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.199+0000 7fab55ffb640 1 -- 172.21.15.1:0/2545679393 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab5c012270 con 0x55f5c3ea05d0 2024-09-11T22:04:08.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.199+0000 7fab55ffb640 1 -- 172.21.15.1:0/2545679393 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7fab680efce0 con 0x55f5c3ea05d0 2024-09-11T22:04:08.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.200+0000 7fab76b22740 10 client.26405 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:04:08.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.200+0000 7fab76b22740 10 client.26405 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-11T22:04:08.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.200+0000 7fab76b22740 20 client.26405 populate_metadata read hostname 'smithi001' 2024-09-11T22:04:08.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.200+0000 7fab76b22740 1 --2- 172.21.15.1:0/2545679393 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x55f5c3f26370 0x55f5c3f28790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:08.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.200+0000 7fab76b22740 4 client.26405 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:04:08.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.200+0000 7fab76b22740 1 -- 172.21.15.1:0/2545679393 --> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f5c3e0dd70 con 0x55f5c3f26370 2024-09-11T22:04:08.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.200+0000 7fab71ce4640 1 --2- 172.21.15.1:0/2545679393 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x55f5c3f26370 0x55f5c3f28790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:08.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.200+0000 7fab71ce4640 1 --2- 172.21.15.1:0/2545679393 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x55f5c3f26370 0x55f5c3f28790 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4f2b6e2b1bef313c server_cookie=3fcb65ae85d4b653 in_seq=0 out_seq=0 2024-09-11T22:04:08.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.201+0000 7fab55ffb640 10 client.26405 ms_handle_connect on v2:172.21.15.161:6834/828381592 2024-09-11T22:04:08.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.204+0000 7fab55ffb640 1 -- 172.21.15.1:0/2545679393 <== mds.4 v2:172.21.15.161:6834/828381592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f5c3e0dd70 con 0x55f5c3f26370 2024-09-11T22:04:08.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.204+0000 7fab55ffb640 10 client.26405 handle_command_reply: tid=0 2024-09-11T22:04:08.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.204+0000 7fab76b22740 1 client.26405 shutdown 2024-09-11T22:04:08.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.204+0000 7fab76b22740 1 -- 172.21.15.1:0/2545679393 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5c3ea05d0 msgr2=0x55f5c3eb3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:08.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.204+0000 7fab76b22740 1 --2- 172.21.15.1:0/2545679393 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5c3ea05d0 0x55f5c3eb3ac0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fab680ecf90 tx=0x7fab680bb780 comp rx=0 tx=0).stop 2024-09-11T22:04:08.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.204+0000 7fab76b22740 1 -- 172.21.15.1:0/2545679393 shutdown_connections 2024-09-11T22:04:08.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.204+0000 7fab76b22740 1 --2- 172.21.15.1:0/2545679393 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f5c3eab370 0x55f5c3ea51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:08.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.204+0000 7fab76b22740 1 --2- 172.21.15.1:0/2545679393 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f5c3ea1b40 0x55f5c3ea4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:08.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.204+0000 7fab76b22740 1 --2- 172.21.15.1:0/2545679393 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x55f5c3f26370 0x55f5c3f28790 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:08.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.204+0000 7fab76b22740 1 --2- 172.21.15.1:0/2545679393 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f5c3ea05d0 0x55f5c3eb3ac0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:08.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.204+0000 7fab55ffb640 0 client.26405 ms_handle_reset on v2:172.21.15.161:6834/828381592 2024-09-11T22:04:08.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.204+0000 7fab76b22740 1 -- 172.21.15.1:0/2545679393 >> 172.21.15.1:0/2545679393 conn(0x55f5c3e69310 msgr2=0x55f5c3ea6d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:08.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.204+0000 7fab76b22740 1 -- 172.21.15.1:0/2545679393 shutdown_connections 2024-09-11T22:04:08.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.204+0000 7fab76b22740 1 -- 172.21.15.1:0/2545679393 wait complete. 2024-09-11T22:04:08.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.204+0000 7fab76b22740 20 client.26405 trim_cache size 0 max 16384 2024-09-11T22:04:08.208 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-11T22:04:08.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.205+0000 7fab74749640 1 -- 172.21.15.1:0/2036818651 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fab580a2cf0 msgr2=0x7fab580a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:08.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.205+0000 7fab74749640 1 --2- 172.21.15.1:0/2036818651 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fab580a2cf0 0x7fab580a51b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fab64023000 tx=0x7fab64024000 comp rx=0 tx=0).stop 2024-09-11T22:04:08.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.205+0000 7fab74749640 1 -- 172.21.15.1:0/2036818651 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab6c101dd0 msgr2=0x7fab6c0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:08.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.205+0000 7fab74749640 1 --2- 172.21.15.1:0/2036818651 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab6c101dd0 0x7fab6c0744f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fab68006010 tx=0x7fab68004040 comp rx=0 tx=0).stop 2024-09-11T22:04:08.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.205+0000 7fab74749640 1 -- 172.21.15.1:0/2036818651 shutdown_connections 2024-09-11T22:04:08.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.205+0000 7fab74749640 1 --2- 172.21.15.1:0/2036818651 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fab580a2cf0 0x7fab580a51b0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:08.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.205+0000 7fab74749640 1 --2- 172.21.15.1:0/2036818651 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fab6c106f00 0x7fab6c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:08.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.205+0000 7fab74749640 1 --2- 172.21.15.1:0/2036818651 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab6c102780 0x7fab6c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:08.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.205+0000 7fab74749640 1 --2- 172.21.15.1:0/2036818651 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fab6c101dd0 0x7fab6c0744f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:08.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.205+0000 7fab74749640 1 -- 172.21.15.1:0/2036818651 >> 172.21.15.1:0/2036818651 conn(0x7fab6c0fd890 msgr2=0x7fab6c0fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:08.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.205+0000 7fab74749640 1 -- 172.21.15.1:0/2036818651 shutdown_connections 2024-09-11T22:04:08.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:08.205+0000 7fab74749640 1 -- 172.21.15.1:0/2036818651 wait complete. 2024-09-11T22:04:08.914 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T22:04:09.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:04:08 smithi001 ceph-mon[30935]: pgmap v213: 97 pgs: 97 active+clean; 380 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.7 MiB/s wr, 642 op/s 2024-09-11T22:04:09.251 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:04:09.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:04:08 smithi053 ceph-mon[34908]: pgmap v213: 97 pgs: 97 active+clean; 380 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.7 MiB/s wr, 642 op/s 2024-09-11T22:04:09.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:04:08 smithi161 ceph-mon[35615]: pgmap v213: 97 pgs: 97 active+clean; 380 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.7 MiB/s wr, 642 op/s 2024-09-11T22:04:09.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.771+0000 7f8275a96640 1 -- 172.21.15.1:0/1229389453 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8270074620 msgr2=0x7f8270074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:09.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.771+0000 7f8275a96640 1 --2- 172.21.15.1:0/1229389453 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8270074620 0x7f8270074a00 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f8260009900 tx=0x7f826002a480 comp rx=0 tx=0).stop 2024-09-11T22:04:09.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.771+0000 7f8275a96640 1 -- 172.21.15.1:0/1229389453 shutdown_connections 2024-09-11T22:04:09.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.771+0000 7f8275a96640 1 --2- 172.21.15.1:0/1229389453 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f827010e3c0 0x7f8270110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:09.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.771+0000 7f8275a96640 1 --2- 172.21.15.1:0/1229389453 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8270074f40 0x7f827010de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:09.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.771+0000 7f8275a96640 1 --2- 172.21.15.1:0/1229389453 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8270074620 0x7f8270074a00 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:09.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.771+0000 7f8275a96640 1 -- 172.21.15.1:0/1229389453 >> 172.21.15.1:0/1229389453 conn(0x7f82700fd8c0 msgr2=0x7f82700ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:09.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.771+0000 7f8275a96640 1 -- 172.21.15.1:0/1229389453 shutdown_connections 2024-09-11T22:04:09.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.771+0000 7f8275a96640 1 -- 172.21.15.1:0/1229389453 wait complete. 2024-09-11T22:04:09.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.771+0000 7f8275a96640 1 Processor -- start 2024-09-11T22:04:09.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.772+0000 7f8275a96640 1 -- start start 2024-09-11T22:04:09.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.772+0000 7f8275a96640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8270074620 0x7f827019a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:09.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.772+0000 7f8275a96640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8270074f40 0x7f827019ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:09.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.772+0000 7f8275a96640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f827010e3c0 0x7f827019ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:09.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.772+0000 7f8275a96640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8270114150 con 0x7f827010e3c0 2024-09-11T22:04:09.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.772+0000 7f8275a96640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8270113fd0 con 0x7f8270074f40 2024-09-11T22:04:09.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.772+0000 7f8275a96640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f82701142d0 con 0x7f8270074620 2024-09-11T22:04:09.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.772+0000 7f826ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f827010e3c0 0x7f827019ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:09.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.772+0000 7f826ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f827010e3c0 0x7f827019ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55672/0 (socket says 172.21.15.1:55672) 2024-09-11T22:04:09.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.772+0000 7f826ffff640 1 -- 172.21.15.1:0/962079997 learned_addr learned my addr 172.21.15.1:0/962079997 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:09.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.772+0000 7f826f7fe640 1 --2- 172.21.15.1:0/962079997 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8270074620 0x7f827019a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:09.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.772+0000 7f826effd640 1 --2- 172.21.15.1:0/962079997 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8270074f40 0x7f827019ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:09.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.772+0000 7f826effd640 1 -- 172.21.15.1:0/962079997 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8270074620 msgr2=0x7f827019a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:09.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.772+0000 7f826effd640 1 --2- 172.21.15.1:0/962079997 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8270074620 0x7f827019a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:09.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.772+0000 7f826effd640 1 -- 172.21.15.1:0/962079997 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f827010e3c0 msgr2=0x7f827019ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:09.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.772+0000 7f826effd640 1 --2- 172.21.15.1:0/962079997 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f827010e3c0 0x7f827019ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:09.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.772+0000 7f826effd640 1 -- 172.21.15.1:0/962079997 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f827019f600 con 0x7f8270074f40 2024-09-11T22:04:09.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.772+0000 7f826f7fe640 1 --2- 172.21.15.1:0/962079997 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8270074620 0x7f827019a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:04:09.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.772+0000 7f826ffff640 1 --2- 172.21.15.1:0/962079997 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f827010e3c0 0x7f827019ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:04:09.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.773+0000 7f826effd640 1 --2- 172.21.15.1:0/962079997 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8270074f40 0x7f827019ab90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f826400e9f0 tx=0x7f826400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:09.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.773+0000 7f826cff9640 1 -- 172.21.15.1:0/962079997 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826400ccc0 con 0x7f8270074f40 2024-09-11T22:04:09.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.773+0000 7f8275a96640 1 -- 172.21.15.1:0/962079997 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f827019f8f0 con 0x7f8270074f40 2024-09-11T22:04:09.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.773+0000 7f826cff9640 1 -- 172.21.15.1:0/962079997 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8264010910 con 0x7f8270074f40 2024-09-11T22:04:09.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.773+0000 7f8275a96640 1 -- 172.21.15.1:0/962079997 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82701a71d0 con 0x7f8270074f40 2024-09-11T22:04:09.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.773+0000 7f826cff9640 1 -- 172.21.15.1:0/962079997 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8264018960 con 0x7f8270074f40 2024-09-11T22:04:09.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.775+0000 7f826cff9640 1 -- 172.21.15.1:0/962079997 <== mon.2 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f8264018b00 con 0x7f8270074f40 2024-09-11T22:04:09.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.776+0000 7f8277e6f740 1 Processor -- start 2024-09-11T22:04:09.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.776+0000 7f8277e6f740 1 -- start start 2024-09-11T22:04:09.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.776+0000 7f8277e6f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556607d41b40 0x556607d40090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:09.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.776+0000 7f8277e6f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556607d405d0 0x556607d40a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:09.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.776+0000 7f8277e6f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556607d4b370 0x556607d4d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:09.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.776+0000 7f8277e6f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x556607ca62f0 con 0x556607d41b40 2024-09-11T22:04:09.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.776+0000 7f8277e6f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x556607c4dcf0 con 0x556607d405d0 2024-09-11T22:04:09.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.776+0000 7f8277e6f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556607ca22b0 con 0x556607d4b370 2024-09-11T22:04:09.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.776+0000 7f826ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556607d41b40 0x556607d40090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:09.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.776+0000 7f826f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556607d405d0 0x556607d40a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:09.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.776+0000 7f826f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556607d405d0 0x556607d40a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:50736/0 (socket says 172.21.15.1:50736) 2024-09-11T22:04:09.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.776+0000 7f826f7fe640 1 -- 172.21.15.1:0/3795128553 learned_addr learned my addr 172.21.15.1:0/3795128553 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:09.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.776+0000 7f826ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556607d41b40 0x556607d40090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55674/0 (socket says 172.21.15.1:55674) 2024-09-11T22:04:09.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.776+0000 7f826effd640 1 --2- 172.21.15.1:0/3795128553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556607d4b370 0x556607d4d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:09.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.776+0000 7f826cff9640 1 --2- 172.21.15.1:0/962079997 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f82580a2cf0 0x7f82580a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:09.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.777+0000 7f826ffff640 1 -- 172.21.15.1:0/3795128553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556607d4b370 msgr2=0x556607d4d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:09.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.777+0000 7f826ffff640 1 --2- 172.21.15.1:0/3795128553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556607d4b370 0x556607d4d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:09.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.777+0000 7f826ffff640 1 -- 172.21.15.1:0/3795128553 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556607d405d0 msgr2=0x556607d40a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:09.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.777+0000 7f826ffff640 1 --2- 172.21.15.1:0/3795128553 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556607d405d0 0x556607d40a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:09.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.777+0000 7f826ffff640 1 -- 172.21.15.1:0/3795128553 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556607c323a0 con 0x556607d41b40 2024-09-11T22:04:09.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.777+0000 7f826ffff640 1 --2- 172.21.15.1:0/962079997 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f82580a2cf0 0x7f82580a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:09.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.777+0000 7f826cff9640 1 -- 172.21.15.1:0/962079997 <== mon.2 v2:172.21.15.53:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8673+0+0 (secure 0 0 0) 0x7f82640cf8e0 con 0x7f8270074f40 2024-09-11T22:04:09.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.777+0000 7f826f7fe640 1 --2- 172.21.15.1:0/3795128553 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556607d405d0 0x556607d40a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:04:09.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.777+0000 7f826ffff640 1 --2- 172.21.15.1:0/3795128553 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556607d41b40 0x556607d40090 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f82680118a0 tx=0x7f8268011d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1739fe76e3facce0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:09.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.777+0000 7f8255ffb640 1 -- 172.21.15.1:0/3795128553 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8268017050 con 0x556607d41b40 2024-09-11T22:04:09.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.777+0000 7f8255ffb640 1 -- 172.21.15.1:0/3795128553 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f826801bb90 con 0x556607d41b40 2024-09-11T22:04:09.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.777+0000 7f8255ffb640 1 -- 172.21.15.1:0/3795128553 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826800e3b0 con 0x556607d41b40 2024-09-11T22:04:09.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.778+0000 7f8277e6f740 1 -- 172.21.15.1:0/3795128553 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556607d41b40 msgr2=0x556607d40090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:09.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.778+0000 7f8277e6f740 1 --2- 172.21.15.1:0/3795128553 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556607d41b40 0x556607d40090 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f82680118a0 tx=0x7f8268011d60 comp rx=0 tx=0).stop 2024-09-11T22:04:09.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.778+0000 7f826ffff640 1 --2- 172.21.15.1:0/962079997 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f82580a2cf0 0x7f82580a51b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8268023000 tx=0x7f8268024000 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:09.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.778+0000 7f8277e6f740 1 -- 172.21.15.1:0/3795128553 shutdown_connections 2024-09-11T22:04:09.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.778+0000 7f8277e6f740 1 --2- 172.21.15.1:0/3795128553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556607d4b370 0x556607d4d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:09.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.778+0000 7f8277e6f740 1 --2- 172.21.15.1:0/3795128553 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556607d405d0 0x556607d40a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:09.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.778+0000 7f8277e6f740 1 --2- 172.21.15.1:0/3795128553 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556607d41b40 0x556607d40090 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:09.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.778+0000 7f8277e6f740 1 -- 172.21.15.1:0/3795128553 >> 172.21.15.1:0/3795128553 conn(0x556607d09310 msgr2=0x556607d3cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:09.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.778+0000 7f8277e6f740 1 -- 172.21.15.1:0/3795128553 shutdown_connections 2024-09-11T22:04:09.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.778+0000 7f8277e6f740 1 -- 172.21.15.1:0/3795128553 wait complete. 2024-09-11T22:04:09.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.779+0000 7f8277e6f740 1 -- start start 2024-09-11T22:04:09.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.779+0000 7f8277e6f740 1 Processor -- start 2024-09-11T22:04:09.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.779+0000 7f8277e6f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556607d405d0 0x556607d4ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:09.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.779+0000 7f8277e6f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556607d41b40 0x556607d47f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:09.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.779+0000 7f8277e6f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556607d4b370 0x556607d484d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:09.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.779+0000 7f8277e6f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x556607d4e070 con 0x556607d4b370 2024-09-11T22:04:09.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.779+0000 7f8277e6f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x556607c61060 con 0x556607d405d0 2024-09-11T22:04:09.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.779+0000 7f8277e6f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556607d4e1f0 con 0x556607d41b40 2024-09-11T22:04:09.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.779+0000 7f8277e6f740 10 client.0 _ll_register_callbacks cb 0x5566079b48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:04:09.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.779+0000 7f826ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556607d4b370 0x556607d484d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:09.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.779+0000 7f826ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556607d4b370 0x556607d484d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55686/0 (socket says 172.21.15.1:55686) 2024-09-11T22:04:09.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.779+0000 7f826ffff640 1 -- 172.21.15.1:0/3805121943 learned_addr learned my addr 172.21.15.1:0/3805121943 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:09.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.779+0000 7f826effd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556607d41b40 0x556607d47f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:09.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.779+0000 7f826f7fe640 1 --2- 172.21.15.1:0/3805121943 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556607d405d0 0x556607d4ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:09.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.780+0000 7f826ffff640 1 -- 172.21.15.1:0/3805121943 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556607d41b40 msgr2=0x556607d47f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:09.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.780+0000 7f826ffff640 1 --2- 172.21.15.1:0/3805121943 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556607d41b40 0x556607d47f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:09.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.780+0000 7f826ffff640 1 -- 172.21.15.1:0/3805121943 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556607d405d0 msgr2=0x556607d4ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:09.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.780+0000 7f826ffff640 1 --2- 172.21.15.1:0/3805121943 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556607d405d0 0x556607d4ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:09.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.780+0000 7f826ffff640 1 -- 172.21.15.1:0/3805121943 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556607a173c0 con 0x556607d4b370 2024-09-11T22:04:09.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.780+0000 7f826f7fe640 1 --2- 172.21.15.1:0/3805121943 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556607d405d0 0x556607d4ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:04:09.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.780+0000 7f826ffff640 1 --2- 172.21.15.1:0/3805121943 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556607d4b370 0x556607d484d0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f8268026b80 tx=0x7f8268026f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b26b499c79b706e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:09.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.780+0000 7f824effd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:04:09.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.780+0000 7f824effd640 1 -- 172.21.15.1:0/3805121943 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8268034030 con 0x556607d4b370 2024-09-11T22:04:09.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.780+0000 7f824effd640 1 -- 172.21.15.1:0/3805121943 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8673+0+0 (secure 0 0 0) 0x7f826801b990 con 0x556607d4b370 2024-09-11T22:04:09.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.780+0000 7f8277e6f740 1 -- 172.21.15.1:0/3805121943 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f826801bb90 con 0x556607d4b370 2024-09-11T22:04:09.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.781+0000 7f824effd640 1 -- 172.21.15.1:0/3805121943 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f8268017020 con 0x556607d4b370 2024-09-11T22:04:09.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.781+0000 7f8277e6f740 10 client.16884 fetch_fsmap learned FSMap version 42 2024-09-11T22:04:09.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.781+0000 7f8277e6f740 1 -- 172.21.15.1:0/3805121943 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x556607d59a80 con 0x556607d4b370 2024-09-11T22:04:09.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.781+0000 7f824effd640 1 -- 172.21.15.1:0/3805121943 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f826801bb90 con 0x556607d4b370 2024-09-11T22:04:09.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.781+0000 7f824effd640 1 -- 172.21.15.1:0/3805121943 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f8268030400 con 0x556607d4b370 2024-09-11T22:04:09.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.781+0000 7f8277e6f740 10 client.16884 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:04:09.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.781+0000 7f8277e6f740 10 client.16884 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-11T22:04:09.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.781+0000 7f8277e6f740 20 client.16884 populate_metadata read hostname 'smithi001' 2024-09-11T22:04:09.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.781+0000 7f8277e6f740 1 --2- 172.21.15.1:0/3805121943 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x556607d5b140 0x556607d5d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:09.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.781+0000 7f8277e6f740 4 client.16884 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:04:09.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.781+0000 7f8277e6f740 1 -- 172.21.15.1:0/3805121943 --> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556607cb3860 con 0x556607d5b140 2024-09-11T22:04:09.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.782+0000 7f826f7fe640 1 --2- 172.21.15.1:0/3805121943 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x556607d5b140 0x556607d5d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:09.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.782+0000 7f826f7fe640 1 --2- 172.21.15.1:0/3805121943 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x556607d5b140 0x556607d5d5e0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7a34ebbce7efa13b server_cookie=102a582cd75cd63e in_seq=0 out_seq=0 2024-09-11T22:04:09.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.782+0000 7f824effd640 10 client.16884 ms_handle_connect on v2:172.21.15.161:6836/1799318656 2024-09-11T22:04:09.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.783+0000 7f824effd640 1 -- 172.21.15.1:0/3805121943 <== mds.3 v2:172.21.15.161:6836/1799318656 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556607cb3860 con 0x556607d5b140 2024-09-11T22:04:09.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.783+0000 7f824effd640 10 client.16884 handle_command_reply: tid=0 2024-09-11T22:04:09.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.783+0000 7f8277e6f740 1 client.16884 shutdown 2024-09-11T22:04:09.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.783+0000 7f8277e6f740 1 -- 172.21.15.1:0/3805121943 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556607d4b370 msgr2=0x556607d484d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:09.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.783+0000 7f8277e6f740 1 --2- 172.21.15.1:0/3805121943 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556607d4b370 0x556607d484d0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f8268026b80 tx=0x7f8268026f90 comp rx=0 tx=0).stop 2024-09-11T22:04:09.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.783+0000 7f8277e6f740 1 -- 172.21.15.1:0/3805121943 shutdown_connections 2024-09-11T22:04:09.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.783+0000 7f8277e6f740 1 --2- 172.21.15.1:0/3805121943 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x556607d5b140 0x556607d5d5e0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:09.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.783+0000 7f8277e6f740 1 --2- 172.21.15.1:0/3805121943 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556607d4b370 0x556607d484d0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:09.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.783+0000 7f8277e6f740 1 --2- 172.21.15.1:0/3805121943 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556607d41b40 0x556607d47f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:09.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.783+0000 7f8277e6f740 1 --2- 172.21.15.1:0/3805121943 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556607d405d0 0x556607d4ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:09.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.783+0000 7f8277e6f740 1 -- 172.21.15.1:0/3805121943 >> 172.21.15.1:0/3805121943 conn(0x556607d09310 msgr2=0x556607d42960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:09.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.783+0000 7f824effd640 0 client.16884 ms_handle_reset on v2:172.21.15.161:6836/1799318656 2024-09-11T22:04:09.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.783+0000 7f8277e6f740 1 -- 172.21.15.1:0/3805121943 shutdown_connections 2024-09-11T22:04:09.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.783+0000 7f8277e6f740 1 -- 172.21.15.1:0/3805121943 wait complete. 2024-09-11T22:04:09.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.783+0000 7f8277e6f740 20 client.16884 trim_cache size 0 max 16384 2024-09-11T22:04:09.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.798+0000 7f8277e6f740 1 Processor -- start 2024-09-11T22:04:09.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.798+0000 7f8277e6f740 1 -- start start 2024-09-11T22:04:09.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.799+0000 7f8277e6f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556607d405d0 0x556607d57310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:09.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.799+0000 7f8277e6f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556607d41b40 0x556607d43120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:09.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.799+0000 7f8277e6f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556607d4b370 0x556607d43660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:09.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.799+0000 7f8277e6f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f82701afdb0 con 0x556607d41b40 2024-09-11T22:04:09.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.799+0000 7f8277e6f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f82701b1bf0 con 0x556607d405d0 2024-09-11T22:04:09.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.799+0000 7f8277e6f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f82701b0540 con 0x556607d4b370 2024-09-11T22:04:09.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.799+0000 7f826f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556607d405d0 0x556607d57310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:09.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.799+0000 7f826effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556607d41b40 0x556607d43120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:09.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.799+0000 7f826f7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556607d405d0 0x556607d57310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:50752/0 (socket says 172.21.15.1:50752) 2024-09-11T22:04:09.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.799+0000 7f826f7fe640 1 -- 172.21.15.1:0/3098107243 learned_addr learned my addr 172.21.15.1:0/3098107243 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:09.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.799+0000 7f826ffff640 1 --2- 172.21.15.1:0/3098107243 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556607d4b370 0x556607d43660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:09.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.799+0000 7f826f7fe640 1 -- 172.21.15.1:0/3098107243 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556607d4b370 msgr2=0x556607d43660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:09.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.799+0000 7f826f7fe640 1 --2- 172.21.15.1:0/3098107243 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556607d4b370 0x556607d43660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:09.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.799+0000 7f826f7fe640 1 -- 172.21.15.1:0/3098107243 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556607d41b40 msgr2=0x556607d43120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:09.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.799+0000 7f826f7fe640 1 --2- 172.21.15.1:0/3098107243 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556607d41b40 0x556607d43120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:09.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.799+0000 7f826f7fe640 1 -- 172.21.15.1:0/3098107243 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556607d3d730 con 0x556607d405d0 2024-09-11T22:04:09.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.799+0000 7f826f7fe640 1 --2- 172.21.15.1:0/3098107243 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556607d405d0 0x556607d57310 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8260009900 tx=0x7f82600329b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d585f85346f5a08 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:09.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.800+0000 7f82557fa640 1 -- 172.21.15.1:0/3098107243 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8260035040 con 0x556607d405d0 2024-09-11T22:04:09.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.800+0000 7f82557fa640 1 -- 172.21.15.1:0/3098107243 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f826004c070 con 0x556607d405d0 2024-09-11T22:04:09.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.800+0000 7f82557fa640 1 -- 172.21.15.1:0/3098107243 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82600049b0 con 0x556607d405d0 2024-09-11T22:04:09.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.800+0000 7f8277e6f740 1 -- 172.21.15.1:0/3098107243 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556607d405d0 msgr2=0x556607d57310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:09.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.800+0000 7f8277e6f740 1 --2- 172.21.15.1:0/3098107243 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556607d405d0 0x556607d57310 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8260009900 tx=0x7f82600329b0 comp rx=0 tx=0).stop 2024-09-11T22:04:09.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.800+0000 7f8277e6f740 1 -- 172.21.15.1:0/3098107243 shutdown_connections 2024-09-11T22:04:09.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.800+0000 7f8277e6f740 1 --2- 172.21.15.1:0/3098107243 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556607d4b370 0x556607d43660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:09.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.800+0000 7f8277e6f740 1 --2- 172.21.15.1:0/3098107243 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556607d41b40 0x556607d43120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:09.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.800+0000 7f8277e6f740 1 --2- 172.21.15.1:0/3098107243 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556607d405d0 0x556607d57310 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:09.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.800+0000 7f8277e6f740 1 -- 172.21.15.1:0/3098107243 >> 172.21.15.1:0/3098107243 conn(0x556607d09310 msgr2=0x556607d47060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:09.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.801+0000 7f8277e6f740 1 -- 172.21.15.1:0/3098107243 shutdown_connections 2024-09-11T22:04:09.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.801+0000 7f8277e6f740 1 -- 172.21.15.1:0/3098107243 wait complete. 2024-09-11T22:04:09.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.801+0000 7f8277e6f740 1 -- start start 2024-09-11T22:04:09.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.801+0000 7f8277e6f740 1 Processor -- start 2024-09-11T22:04:09.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.801+0000 7f8277e6f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556607d405d0 0x556607d4e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:09.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.801+0000 7f8277e6f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556607d41b40 0x556607d4ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:09.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.801+0000 7f8277e6f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556607d4b370 0x556607dc3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:09.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.801+0000 7f8277e6f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f82701b1460 con 0x556607d41b40 2024-09-11T22:04:09.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.802+0000 7f8277e6f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f82701b0cd0 con 0x556607d405d0 2024-09-11T22:04:09.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.802+0000 7f8277e6f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f82701addb0 con 0x556607d4b370 2024-09-11T22:04:09.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.802+0000 7f826effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556607d41b40 0x556607d4ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:09.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.802+0000 7f8277e6f740 10 client.0 _ll_register_callbacks cb 0x7f8268011780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:04:09.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.802+0000 7f826ffff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556607d4b370 0x556607dc3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:09.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.802+0000 7f826effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556607d41b40 0x556607d4ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55712/0 (socket says 172.21.15.1:55712) 2024-09-11T22:04:09.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.802+0000 7f826effd640 1 -- 172.21.15.1:0/4264318157 learned_addr learned my addr 172.21.15.1:0/4264318157 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:09.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.802+0000 7f826f7fe640 1 --2- 172.21.15.1:0/4264318157 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556607d405d0 0x556607d4e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:09.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.802+0000 7f826effd640 1 -- 172.21.15.1:0/4264318157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556607d4b370 msgr2=0x556607dc3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:09.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.802+0000 7f826effd640 1 --2- 172.21.15.1:0/4264318157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556607d4b370 0x556607dc3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:09.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.802+0000 7f826effd640 1 -- 172.21.15.1:0/4264318157 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556607d405d0 msgr2=0x556607d4e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:09.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.802+0000 7f826effd640 1 --2- 172.21.15.1:0/4264318157 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556607d405d0 0x556607d4e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:09.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.802+0000 7f826effd640 1 -- 172.21.15.1:0/4264318157 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556607dc4190 con 0x556607d41b40 2024-09-11T22:04:09.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.802+0000 7f826effd640 1 --2- 172.21.15.1:0/4264318157 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556607d41b40 0x556607d4ed40 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f826409bb10 tx=0x7f826400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd30256453111265 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:09.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.802+0000 7f824effd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:04:09.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.802+0000 7f824effd640 1 -- 172.21.15.1:0/4264318157 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82640d5040 con 0x556607d41b40 2024-09-11T22:04:09.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.802+0000 7f824effd640 1 -- 172.21.15.1:0/4264318157 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8673+0+0 (secure 0 0 0) 0x7f826409b260 con 0x556607d41b40 2024-09-11T22:04:09.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.802+0000 7f8277e6f740 1 -- 172.21.15.1:0/4264318157 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f826004c070 con 0x556607d41b40 2024-09-11T22:04:09.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.803+0000 7f824effd640 1 -- 172.21.15.1:0/4264318157 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f826409b770 con 0x556607d41b40 2024-09-11T22:04:09.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.803+0000 7f8277e6f740 10 client.16890 fetch_fsmap learned FSMap version 42 2024-09-11T22:04:09.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.803+0000 7f8277e6f740 1 -- 172.21.15.1:0/4264318157 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x556607dc53b0 con 0x556607d41b40 2024-09-11T22:04:09.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.803+0000 7f824effd640 1 -- 172.21.15.1:0/4264318157 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f826004c070 con 0x556607d41b40 2024-09-11T22:04:09.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.803+0000 7f824effd640 1 -- 172.21.15.1:0/4264318157 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f8264097bf0 con 0x556607d41b40 2024-09-11T22:04:09.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.803+0000 7f8277e6f740 10 client.16890 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:04:09.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.803+0000 7f8277e6f740 10 client.16890 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-11T22:04:09.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.803+0000 7f8277e6f740 20 client.16890 populate_metadata read hostname 'smithi001' 2024-09-11T22:04:09.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.803+0000 7f8277e6f740 1 --2- 172.21.15.1:0/4264318157 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x556607dc5c20 0x556607dc8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:09.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.803+0000 7f8277e6f740 4 client.16890 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:04:09.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.803+0000 7f8277e6f740 1 -- 172.21.15.1:0/4264318157 --> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556607cadd70 con 0x556607dc5c20 2024-09-11T22:04:09.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.803+0000 7f826f7fe640 1 --2- 172.21.15.1:0/4264318157 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x556607dc5c20 0x556607dc8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:09.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.804+0000 7f826f7fe640 1 --2- 172.21.15.1:0/4264318157 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x556607dc5c20 0x556607dc8040 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4a0f0f009d08ea35 server_cookie=8be0c50d73d1396d in_seq=0 out_seq=0 2024-09-11T22:04:09.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.804+0000 7f824effd640 10 client.16890 ms_handle_connect on v2:172.21.15.161:6836/1799318656 2024-09-11T22:04:09.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.804+0000 7f824effd640 1 -- 172.21.15.1:0/4264318157 <== mds.3 v2:172.21.15.161:6836/1799318656 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556607cadd70 con 0x556607dc5c20 2024-09-11T22:04:09.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.804+0000 7f824effd640 10 client.16890 handle_command_reply: tid=0 2024-09-11T22:04:09.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.804+0000 7f8277e6f740 1 client.16890 shutdown 2024-09-11T22:04:09.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.804+0000 7f8277e6f740 1 -- 172.21.15.1:0/4264318157 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556607d41b40 msgr2=0x556607d4ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:09.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.804+0000 7f8277e6f740 1 --2- 172.21.15.1:0/4264318157 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556607d41b40 0x556607d4ed40 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f826409bb10 tx=0x7f826400cf60 comp rx=0 tx=0).stop 2024-09-11T22:04:09.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.804+0000 7f8277e6f740 1 -- 172.21.15.1:0/4264318157 shutdown_connections 2024-09-11T22:04:09.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.804+0000 7f8277e6f740 1 --2- 172.21.15.1:0/4264318157 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x556607dc5c20 0x556607dc8040 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:09.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.804+0000 7f8277e6f740 1 --2- 172.21.15.1:0/4264318157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556607d4b370 0x556607dc3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:09.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.804+0000 7f8277e6f740 1 --2- 172.21.15.1:0/4264318157 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556607d41b40 0x556607d4ed40 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:09.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.804+0000 7f8277e6f740 1 --2- 172.21.15.1:0/4264318157 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556607d405d0 0x556607d4e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:09.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.804+0000 7f8277e6f740 1 -- 172.21.15.1:0/4264318157 >> 172.21.15.1:0/4264318157 conn(0x556607d09310 msgr2=0x556607d3b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:09.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.804+0000 7f824effd640 0 client.16890 ms_handle_reset on v2:172.21.15.161:6836/1799318656 2024-09-11T22:04:09.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.804+0000 7f8277e6f740 1 -- 172.21.15.1:0/4264318157 shutdown_connections 2024-09-11T22:04:09.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.804+0000 7f8277e6f740 1 -- 172.21.15.1:0/4264318157 wait complete. 2024-09-11T22:04:09.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.804+0000 7f8277e6f740 20 client.16890 trim_cache size 0 max 16384 2024-09-11T22:04:09.810 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-11T22:04:09.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.805+0000 7f8275a96640 1 -- 172.21.15.1:0/962079997 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f82580a2cf0 msgr2=0x7f82580a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:09.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.805+0000 7f8275a96640 1 --2- 172.21.15.1:0/962079997 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f82580a2cf0 0x7f82580a51b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8268023000 tx=0x7f8268024000 comp rx=0 tx=0).stop 2024-09-11T22:04:09.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.805+0000 7f8275a96640 1 -- 172.21.15.1:0/962079997 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8270074f40 msgr2=0x7f827019ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:09.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.805+0000 7f8275a96640 1 --2- 172.21.15.1:0/962079997 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8270074f40 0x7f827019ab90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f826400e9f0 tx=0x7f826400eeb0 comp rx=0 tx=0).stop 2024-09-11T22:04:09.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.805+0000 7f8275a96640 1 -- 172.21.15.1:0/962079997 shutdown_connections 2024-09-11T22:04:09.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.805+0000 7f8275a96640 1 --2- 172.21.15.1:0/962079997 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f82580a2cf0 0x7f82580a51b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:09.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.805+0000 7f8275a96640 1 --2- 172.21.15.1:0/962079997 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f827010e3c0 0x7f827019ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:09.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.805+0000 7f8275a96640 1 --2- 172.21.15.1:0/962079997 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8270074f40 0x7f827019ab90 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:09.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.805+0000 7f8275a96640 1 --2- 172.21.15.1:0/962079997 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8270074620 0x7f827019a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:09.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.805+0000 7f8275a96640 1 -- 172.21.15.1:0/962079997 >> 172.21.15.1:0/962079997 conn(0x7f82700fd8c0 msgr2=0x7f82700fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:09.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.805+0000 7f8275a96640 1 -- 172.21.15.1:0/962079997 shutdown_connections 2024-09-11T22:04:09.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:09.805+0000 7f8275a96640 1 -- 172.21.15.1:0/962079997 wait complete. 2024-09-11T22:04:10.511 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T22:04:10.789 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:04:11.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:04:10 smithi001 ceph-mon[30935]: pgmap v214: 97 pgs: 97 active+clean; 402 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.9 MiB/s wr, 573 op/s 2024-09-11T22:04:11.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:04:10 smithi053 ceph-mon[34908]: pgmap v214: 97 pgs: 97 active+clean; 402 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.9 MiB/s wr, 573 op/s 2024-09-11T22:04:11.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:04:10 smithi161 ceph-mon[35615]: pgmap v214: 97 pgs: 97 active+clean; 402 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.9 MiB/s wr, 573 op/s 2024-09-11T22:04:11.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.442+0000 7f104ba0f640 1 -- 172.21.15.1:0/2150964308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1044101de0 msgr2=0x7f10441021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:11.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.442+0000 7f104ba0f640 1 --2- 172.21.15.1:0/2150964308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1044101de0 0x7f10441021c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1040009900 tx=0x7f104002a410 comp rx=0 tx=0).stop 2024-09-11T22:04:11.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.442+0000 7f104ba0f640 1 -- 172.21.15.1:0/2150964308 shutdown_connections 2024-09-11T22:04:11.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.442+0000 7f104ba0f640 1 --2- 172.21.15.1:0/2150964308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1044106f10 0x7f104410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:11.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.442+0000 7f104ba0f640 1 --2- 172.21.15.1:0/2150964308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1044102790 0x7f10441067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:11.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.442+0000 7f104ba0f640 1 --2- 172.21.15.1:0/2150964308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1044101de0 0x7f10441021c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:11.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.442+0000 7f104ba0f640 1 -- 172.21.15.1:0/2150964308 >> 172.21.15.1:0/2150964308 conn(0x7f10440fd880 msgr2=0x7f10440ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:11.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.442+0000 7f104ba0f640 1 -- 172.21.15.1:0/2150964308 shutdown_connections 2024-09-11T22:04:11.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.442+0000 7f104ba0f640 1 -- 172.21.15.1:0/2150964308 wait complete. 2024-09-11T22:04:11.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.443+0000 7f104ba0f640 1 Processor -- start 2024-09-11T22:04:11.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.443+0000 7f104ba0f640 1 -- start start 2024-09-11T22:04:11.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.443+0000 7f104ba0f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1044101de0 0x7f104419f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:11.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.443+0000 7f104ba0f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1044102790 0x7f104419f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:11.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.443+0000 7f104ba0f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1044106f10 0x7f104410de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:11.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.444+0000 7f104ba0f640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1044111f70 con 0x7f1044106f10 2024-09-11T22:04:11.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.444+0000 7f104ba0f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1044111df0 con 0x7f1044102790 2024-09-11T22:04:11.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.444+0000 7f104ba0f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f10441120f0 con 0x7f1044101de0 2024-09-11T22:04:11.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.444+0000 7f10497ab640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1044101de0 0x7f104419f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:11.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.444+0000 7f1048faa640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1044102790 0x7f104419f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:11.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.444+0000 7f10497ab640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1044101de0 0x7f104419f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:50818/0 (socket says 172.21.15.1:50818) 2024-09-11T22:04:11.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.444+0000 7f10497ab640 1 -- 172.21.15.1:0/3671210751 learned_addr learned my addr 172.21.15.1:0/3671210751 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:11.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.444+0000 7f1049fac640 1 --2- 172.21.15.1:0/3671210751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1044106f10 0x7f104410de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:11.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.445+0000 7f10497ab640 1 -- 172.21.15.1:0/3671210751 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1044102790 msgr2=0x7f104419f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:11.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.445+0000 7f10497ab640 1 --2- 172.21.15.1:0/3671210751 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1044102790 0x7f104419f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:11.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.445+0000 7f10497ab640 1 -- 172.21.15.1:0/3671210751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1044106f10 msgr2=0x7f104410de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:11.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.445+0000 7f10497ab640 1 --2- 172.21.15.1:0/3671210751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1044106f10 0x7f104410de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:11.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.445+0000 7f10497ab640 1 -- 172.21.15.1:0/3671210751 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f104410e720 con 0x7f1044101de0 2024-09-11T22:04:11.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.445+0000 7f1048faa640 1 --2- 172.21.15.1:0/3671210751 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1044102790 0x7f104419f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:04:11.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.445+0000 7f10497ab640 1 --2- 172.21.15.1:0/3671210751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1044101de0 0x7f104419f190 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1040006010 tx=0x7f1040004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:11.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.445+0000 7f103a7fc640 1 -- 172.21.15.1:0/3671210751 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f104003f030 con 0x7f1044101de0 2024-09-11T22:04:11.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.445+0000 7f104ba0f640 1 -- 172.21.15.1:0/3671210751 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f104410e9b0 con 0x7f1044101de0 2024-09-11T22:04:11.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.446+0000 7f104ba0f640 1 -- 172.21.15.1:0/3671210751 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f104419ad40 con 0x7f1044101de0 2024-09-11T22:04:11.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.446+0000 7f103a7fc640 1 -- 172.21.15.1:0/3671210751 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1040004440 con 0x7f1044101de0 2024-09-11T22:04:11.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.446+0000 7f103a7fc640 1 -- 172.21.15.1:0/3671210751 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1040049060 con 0x7f1044101de0 2024-09-11T22:04:11.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.449+0000 7f104dde8740 1 Processor -- start 2024-09-11T22:04:11.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.450+0000 7f104dde8740 1 -- start start 2024-09-11T22:04:11.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.450+0000 7f104dde8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562601be2b40 0x562601be1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:11.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.450+0000 7f104dde8740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562601be15d0 0x562601be1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:11.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.450+0000 7f104dde8740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562601bec370 0x562601bee760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:11.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.450+0000 7f104dde8740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x562601b472f0 con 0x562601bec370 2024-09-11T22:04:11.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.450+0000 7f104dde8740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x562601aeecf0 con 0x562601be15d0 2024-09-11T22:04:11.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.450+0000 7f104dde8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x562601b432b0 con 0x562601be2b40 2024-09-11T22:04:11.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.451+0000 7f1048faa640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562601bec370 0x562601bee760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:11.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.451+0000 7f1048faa640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562601bec370 0x562601bee760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55736/0 (socket says 172.21.15.1:55736) 2024-09-11T22:04:11.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.451+0000 7f1048faa640 1 -- 172.21.15.1:0/1755308124 learned_addr learned my addr 172.21.15.1:0/1755308124 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:11.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.451+0000 7f1049fac640 1 --2- 172.21.15.1:0/1755308124 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562601be2b40 0x562601be1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:11.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.452+0000 7f1049fac640 1 -- 172.21.15.1:0/1755308124 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562601be15d0 msgr2=0x562601be1a10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T22:04:11.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.452+0000 7f1049fac640 1 --2- 172.21.15.1:0/1755308124 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562601be15d0 0x562601be1a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:11.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.452+0000 7f1049fac640 1 -- 172.21.15.1:0/1755308124 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562601bec370 msgr2=0x562601bee760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:11.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.452+0000 7f1049fac640 1 --2- 172.21.15.1:0/1755308124 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562601bec370 0x562601bee760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:11.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.452+0000 7f1049fac640 1 -- 172.21.15.1:0/1755308124 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562601ad33a0 con 0x562601be2b40 2024-09-11T22:04:11.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.453+0000 7f103a7fc640 1 -- 172.21.15.1:0/3671210751 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f1040042030 con 0x7f1044101de0 2024-09-11T22:04:11.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.454+0000 7f103a7fc640 1 --2- 172.21.15.1:0/3671210751 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f10300a2d70 0x7f10300a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:11.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.454+0000 7f103a7fc640 1 -- 172.21.15.1:0/3671210751 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8673+0+0 (secure 0 0 0) 0x7f10400ef740 con 0x7f1044101de0 2024-09-11T22:04:11.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.454+0000 7f1049fac640 1 --2- 172.21.15.1:0/1755308124 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562601be2b40 0x562601be1090 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f103c009f00 tx=0x7f103c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=486b17fd3c3cb703 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:11.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.454+0000 7f10497ab640 1 --2- 172.21.15.1:0/3671210751 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f10300a2d70 0x7f10300a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:11.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.455+0000 7f10497ab640 1 --2- 172.21.15.1:0/3671210751 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f10300a2d70 0x7f10300a5230 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x562601bdeac0 tx=0x7f10400058b0 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:11.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.455+0000 7f102f7fe640 1 -- 172.21.15.1:0/1755308124 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f103c004d20 con 0x562601be2b40 2024-09-11T22:04:11.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.455+0000 7f102f7fe640 1 -- 172.21.15.1:0/1755308124 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f103c004050 con 0x562601be2b40 2024-09-11T22:04:11.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.455+0000 7f102f7fe640 1 -- 172.21.15.1:0/1755308124 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f103c005280 con 0x562601be2b40 2024-09-11T22:04:11.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.456+0000 7f104dde8740 1 -- 172.21.15.1:0/1755308124 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562601be2b40 msgr2=0x562601be1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:11.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.456+0000 7f104dde8740 1 --2- 172.21.15.1:0/1755308124 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562601be2b40 0x562601be1090 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f103c009f00 tx=0x7f103c011810 comp rx=0 tx=0).stop 2024-09-11T22:04:11.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.456+0000 7f104dde8740 1 -- 172.21.15.1:0/1755308124 shutdown_connections 2024-09-11T22:04:11.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.456+0000 7f104dde8740 1 --2- 172.21.15.1:0/1755308124 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562601bec370 0x562601bee760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:11.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.456+0000 7f104dde8740 1 --2- 172.21.15.1:0/1755308124 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562601be15d0 0x562601be1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:11.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.456+0000 7f104dde8740 1 --2- 172.21.15.1:0/1755308124 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562601be2b40 0x562601be1090 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:11.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.456+0000 7f104dde8740 1 -- 172.21.15.1:0/1755308124 >> 172.21.15.1:0/1755308124 conn(0x562601baa310 msgr2=0x562601bddc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:11.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.456+0000 7f104dde8740 1 -- 172.21.15.1:0/1755308124 shutdown_connections 2024-09-11T22:04:11.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.456+0000 7f104dde8740 1 -- 172.21.15.1:0/1755308124 wait complete. 2024-09-11T22:04:11.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.457+0000 7f104dde8740 1 -- start start 2024-09-11T22:04:11.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.457+0000 7f104dde8740 1 Processor -- start 2024-09-11T22:04:11.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.458+0000 7f104dde8740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562601be15d0 0x562601bebd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:11.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.458+0000 7f104dde8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562601be2b40 0x562601be9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:11.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.458+0000 7f104dde8740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562601bec370 0x562601be9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:11.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.458+0000 7f1049fac640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562601be15d0 0x562601bebd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:11.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.458+0000 7f104dde8740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x562601bef070 con 0x562601be15d0 2024-09-11T22:04:11.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.458+0000 7f104dde8740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x562601b02060 con 0x562601bec370 2024-09-11T22:04:11.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.458+0000 7f104dde8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x562601bef1f0 con 0x562601be2b40 2024-09-11T22:04:11.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.458+0000 7f104dde8740 10 client.0 _ll_register_callbacks cb 0x5626018558c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:04:11.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.458+0000 7f1049fac640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562601bec370 0x562601be9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:11.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.458+0000 7f1048faa640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562601be2b40 0x562601be9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:11.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.458+0000 7f1049fac640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562601bec370 0x562601be9600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:50784/0 (socket says 172.21.15.1:50784) 2024-09-11T22:04:11.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.458+0000 7f1049fac640 1 -- 172.21.15.1:0/17404989 learned_addr learned my addr 172.21.15.1:0/17404989 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:11.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.459+0000 7f1048faa640 1 -- 172.21.15.1:0/17404989 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562601bec370 msgr2=0x562601be9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:11.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.459+0000 7f1048faa640 1 --2- 172.21.15.1:0/17404989 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562601bec370 0x562601be9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:11.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.459+0000 7f1048faa640 1 -- 172.21.15.1:0/17404989 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562601be15d0 msgr2=0x562601bebd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:11.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.459+0000 7f1048faa640 1 --2- 172.21.15.1:0/17404989 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562601be15d0 0x562601bebd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:11.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.459+0000 7f1048faa640 1 -- 172.21.15.1:0/17404989 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626018b83c0 con 0x562601be2b40 2024-09-11T22:04:11.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.459+0000 7f1048faa640 1 --2- 172.21.15.1:0/17404989 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562601be2b40 0x562601be9010 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1034012cf0 tx=0x7f103400f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f46b69ab9e488ba server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:11.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.459+0000 7f102cff9640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-11T22:04:11.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.462+0000 7f102cff9640 1 -- 172.21.15.1:0/17404989 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f103400fe60 con 0x562601be2b40 2024-09-11T22:04:11.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.462+0000 7f102cff9640 1 -- 172.21.15.1:0/17404989 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8673+0+0 (secure 0 0 0) 0x7f10340120d0 con 0x562601be2b40 2024-09-11T22:04:11.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.462+0000 7f104dde8740 1 -- 172.21.15.1:0/17404989 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f103c004050 con 0x562601be2b40 2024-09-11T22:04:11.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.463+0000 7f102cff9640 1 -- 172.21.15.1:0/17404989 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f1034005020 con 0x562601be2b40 2024-09-11T22:04:11.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.463+0000 7f104dde8740 10 client.26404 fetch_fsmap learned FSMap version 42 2024-09-11T22:04:11.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.463+0000 7f104dde8740 1 -- 172.21.15.1:0/17404989 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x562601bfac10 con 0x562601be2b40 2024-09-11T22:04:11.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.464+0000 7f102cff9640 1 -- 172.21.15.1:0/17404989 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10340056a0 con 0x562601be2b40 2024-09-11T22:04:11.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.465+0000 7f102cff9640 1 -- 172.21.15.1:0/17404989 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f10340051c0 con 0x562601be2b40 2024-09-11T22:04:11.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.465+0000 7f104dde8740 10 client.26404 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:04:11.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.465+0000 7f104dde8740 10 client.26404 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-11T22:04:11.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.465+0000 7f104dde8740 20 client.26404 populate_metadata read hostname 'smithi001' 2024-09-11T22:04:11.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.465+0000 7f104dde8740 1 --2- 172.21.15.1:0/17404989 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x562601bfc5c0 0x562601bfea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:11.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.465+0000 7f104dde8740 4 client.26404 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:04:11.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.465+0000 7f104dde8740 1 -- 172.21.15.1:0/17404989 --> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f103c004050 con 0x562601bfc5c0 2024-09-11T22:04:11.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.466+0000 7f1049fac640 1 --2- 172.21.15.1:0/17404989 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x562601bfc5c0 0x562601bfea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:11.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.466+0000 7f1049fac640 1 --2- 172.21.15.1:0/17404989 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x562601bfc5c0 0x562601bfea20 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=68503de936e1d398 server_cookie=64cd5a32388723b in_seq=0 out_seq=0 2024-09-11T22:04:11.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.466+0000 7f102cff9640 10 client.26404 ms_handle_connect on v2:172.21.15.53:6836/3457131830 2024-09-11T22:04:11.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.468+0000 7f102cff9640 1 -- 172.21.15.1:0/17404989 <== mds.2 v2:172.21.15.53:6836/3457131830 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f103c004050 con 0x562601bfc5c0 2024-09-11T22:04:11.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.468+0000 7f102cff9640 10 client.26404 handle_command_reply: tid=0 2024-09-11T22:04:11.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.468+0000 7f104dde8740 1 client.26404 shutdown 2024-09-11T22:04:11.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.468+0000 7f104dde8740 1 -- 172.21.15.1:0/17404989 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562601be2b40 msgr2=0x562601be9010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:11.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.468+0000 7f104dde8740 1 --2- 172.21.15.1:0/17404989 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562601be2b40 0x562601be9010 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1034012cf0 tx=0x7f103400f790 comp rx=0 tx=0).stop 2024-09-11T22:04:11.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.468+0000 7f104dde8740 1 -- 172.21.15.1:0/17404989 shutdown_connections 2024-09-11T22:04:11.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.468+0000 7f104dde8740 1 --2- 172.21.15.1:0/17404989 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x562601bfc5c0 0x562601bfea20 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:11.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.468+0000 7f104dde8740 1 --2- 172.21.15.1:0/17404989 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562601bec370 0x562601be9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:11.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.468+0000 7f104dde8740 1 --2- 172.21.15.1:0/17404989 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562601be2b40 0x562601be9010 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:11.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.468+0000 7f104dde8740 1 --2- 172.21.15.1:0/17404989 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562601be15d0 0x562601bebd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:11.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.468+0000 7f104dde8740 1 -- 172.21.15.1:0/17404989 >> 172.21.15.1:0/17404989 conn(0x562601baa310 msgr2=0x562601be3930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:11.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.469+0000 7f102cff9640 0 client.26404 ms_handle_reset on v2:172.21.15.53:6836/3457131830 2024-09-11T22:04:11.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.469+0000 7f104dde8740 1 -- 172.21.15.1:0/17404989 shutdown_connections 2024-09-11T22:04:11.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.469+0000 7f104dde8740 1 -- 172.21.15.1:0/17404989 wait complete. 2024-09-11T22:04:11.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.469+0000 7f104dde8740 20 client.26404 trim_cache size 0 max 16384 2024-09-11T22:04:11.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.490+0000 7f104dde8740 1 Processor -- start 2024-09-11T22:04:11.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.490+0000 7f104dde8740 1 -- start start 2024-09-11T22:04:11.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.490+0000 7f104dde8740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562601be15d0 0x562601bf83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:11.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.490+0000 7f104dde8740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562601be2b40 0x562601be88d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:11.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.490+0000 7f104dde8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562601bec370 0x562601c04330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:11.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.490+0000 7f104dde8740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f102800eab0 con 0x562601be2b40 2024-09-11T22:04:11.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.490+0000 7f104dde8740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f10280108f0 con 0x562601be15d0 2024-09-11T22:04:11.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.490+0000 7f104dde8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f102800f240 con 0x562601bec370 2024-09-11T22:04:11.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.490+0000 7f1048faa640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562601be2b40 0x562601be88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:11.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.491+0000 7f1048faa640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562601be2b40 0x562601be88d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55748/0 (socket says 172.21.15.1:55748) 2024-09-11T22:04:11.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.491+0000 7f1048faa640 1 -- 172.21.15.1:0/240118868 learned_addr learned my addr 172.21.15.1:0/240118868 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:11.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.491+0000 7f1048faa640 1 -- 172.21.15.1:0/240118868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562601bec370 msgr2=0x562601c04330 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T22:04:11.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.491+0000 7f1048faa640 1 --2- 172.21.15.1:0/240118868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562601bec370 0x562601c04330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:11.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.491+0000 7f1048faa640 1 -- 172.21.15.1:0/240118868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562601be15d0 msgr2=0x562601bf83a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T22:04:11.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.491+0000 7f1048faa640 1 --2- 172.21.15.1:0/240118868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562601be15d0 0x562601bf83a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:11.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.491+0000 7f1048faa640 1 -- 172.21.15.1:0/240118868 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562601bec0d0 con 0x562601be2b40 2024-09-11T22:04:11.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.492+0000 7f1048faa640 1 --2- 172.21.15.1:0/240118868 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562601be2b40 0x562601be88d0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f1034012870 tx=0x7f1034012370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6758a451bbb6e64f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:11.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.492+0000 7f102effd640 1 -- 172.21.15.1:0/240118868 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f103402c070 con 0x562601be2b40 2024-09-11T22:04:11.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.492+0000 7f102effd640 1 -- 172.21.15.1:0/240118868 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f103400fe60 con 0x562601be2b40 2024-09-11T22:04:11.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.492+0000 7f102effd640 1 -- 172.21.15.1:0/240118868 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10340047e0 con 0x562601be2b40 2024-09-11T22:04:11.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.493+0000 7f104dde8740 1 -- 172.21.15.1:0/240118868 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562601be2b40 msgr2=0x562601be88d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:11.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.493+0000 7f104dde8740 1 --2- 172.21.15.1:0/240118868 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562601be2b40 0x562601be88d0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f1034012870 tx=0x7f1034012370 comp rx=0 tx=0).stop 2024-09-11T22:04:11.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.494+0000 7f104dde8740 1 -- 172.21.15.1:0/240118868 shutdown_connections 2024-09-11T22:04:11.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.494+0000 7f104dde8740 1 --2- 172.21.15.1:0/240118868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562601bec370 0x562601c04330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:11.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.494+0000 7f104dde8740 1 --2- 172.21.15.1:0/240118868 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562601be2b40 0x562601be88d0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:11.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.494+0000 7f104dde8740 1 --2- 172.21.15.1:0/240118868 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562601be15d0 0x562601bf83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:11.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.494+0000 7f104dde8740 1 -- 172.21.15.1:0/240118868 >> 172.21.15.1:0/240118868 conn(0x562601baa310 msgr2=0x562601be78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:11.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.494+0000 7f104dde8740 1 -- 172.21.15.1:0/240118868 shutdown_connections 2024-09-11T22:04:11.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.494+0000 7f104dde8740 1 -- 172.21.15.1:0/240118868 wait complete. 2024-09-11T22:04:11.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.494+0000 7f104dde8740 1 -- start start 2024-09-11T22:04:11.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.495+0000 7f104dde8740 1 Processor -- start 2024-09-11T22:04:11.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.496+0000 7f104dde8740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562601be15d0 0x562601bf4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:11.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.496+0000 7f104dde8740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562601be2b40 0x562601befd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:11.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.496+0000 7f104dde8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562601bec370 0x562601bf02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:11.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.496+0000 7f104dde8740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1028010160 con 0x562601be15d0 2024-09-11T22:04:11.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.496+0000 7f1049fac640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562601be15d0 0x562601bf4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:11.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.496+0000 7f1049fac640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562601be15d0 0x562601bf4a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55752/0 (socket says 172.21.15.1:55752) 2024-09-11T22:04:11.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.496+0000 7f1049fac640 1 -- 172.21.15.1:0/2491064258 learned_addr learned my addr 172.21.15.1:0/2491064258 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:11.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.496+0000 7f104dde8740 1 -- 172.21.15.1:0/2491064258 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f102800f9d0 con 0x562601be2b40 2024-09-11T22:04:11.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.496+0000 7f104dde8740 1 -- 172.21.15.1:0/2491064258 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f102800cab0 con 0x562601bec370 2024-09-11T22:04:11.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.496+0000 7f104dde8740 10 client.0 _ll_register_callbacks cb 0x7f1034009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:04:11.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.496+0000 7f1049fac640 1 --2- 172.21.15.1:0/2491064258 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562601bec370 0x562601bf02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:11.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.496+0000 7f1048faa640 1 --2- 172.21.15.1:0/2491064258 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562601be2b40 0x562601befd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:11.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.496+0000 7f1049fac640 1 -- 172.21.15.1:0/2491064258 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562601bec370 msgr2=0x562601bf02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:11.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.496+0000 7f1049fac640 1 --2- 172.21.15.1:0/2491064258 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562601bec370 0x562601bf02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:11.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.496+0000 7f1049fac640 1 -- 172.21.15.1:0/2491064258 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562601be2b40 msgr2=0x562601befd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:11.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.496+0000 7f1049fac640 1 --2- 172.21.15.1:0/2491064258 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562601be2b40 0x562601befd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:11.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.496+0000 7f1049fac640 1 -- 172.21.15.1:0/2491064258 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562601bf08a0 con 0x562601be15d0 2024-09-11T22:04:11.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.497+0000 7f1049fac640 1 --2- 172.21.15.1:0/2491064258 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562601be15d0 0x562601bf4a40 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f103c009c50 tx=0x7f103c009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a226b9a93035d95e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:11.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.497+0000 7f102cff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:04:11.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.497+0000 7f102cff9640 1 -- 172.21.15.1:0/2491064258 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f103c028680 con 0x562601be15d0 2024-09-11T22:04:11.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.497+0000 7f102cff9640 1 -- 172.21.15.1:0/2491064258 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8673+0+0 (secure 0 0 0) 0x7f103c028820 con 0x562601be15d0 2024-09-11T22:04:11.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.497+0000 7f104dde8740 1 -- 172.21.15.1:0/2491064258 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f103400fe60 con 0x562601be15d0 2024-09-11T22:04:11.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.497+0000 7f102cff9640 1 -- 172.21.15.1:0/2491064258 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f103c028a20 con 0x562601be15d0 2024-09-11T22:04:11.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.497+0000 7f104dde8740 10 client.16908 fetch_fsmap learned FSMap version 42 2024-09-11T22:04:11.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.497+0000 7f104dde8740 1 -- 172.21.15.1:0/2491064258 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x562601c670c0 con 0x562601be15d0 2024-09-11T22:04:11.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.498+0000 7f102cff9640 1 -- 172.21.15.1:0/2491064258 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f103c028bc0 con 0x562601be15d0 2024-09-11T22:04:11.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.498+0000 7f102cff9640 1 -- 172.21.15.1:0/2491064258 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f103c0265d0 con 0x562601be15d0 2024-09-11T22:04:11.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.498+0000 7f104dde8740 10 client.16908 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:04:11.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.498+0000 7f104dde8740 10 client.16908 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-11T22:04:11.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.498+0000 7f104dde8740 20 client.16908 populate_metadata read hostname 'smithi001' 2024-09-11T22:04:11.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.498+0000 7f104dde8740 1 --2- 172.21.15.1:0/2491064258 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x562601c67670 0x562601c69a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:11.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.498+0000 7f104dde8740 4 client.16908 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:04:11.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.498+0000 7f104dde8740 1 -- 172.21.15.1:0/2491064258 --> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f103400fe60 con 0x562601c67670 2024-09-11T22:04:11.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.499+0000 7f1048faa640 1 --2- 172.21.15.1:0/2491064258 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x562601c67670 0x562601c69a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:11.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.499+0000 7f102cff9640 10 client.16908 ms_handle_connect on v2:172.21.15.53:6836/3457131830 2024-09-11T22:04:11.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.499+0000 7f1048faa640 1 --2- 172.21.15.1:0/2491064258 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x562601c67670 0x562601c69a90 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2eae82bb4b586fe2 server_cookie=834e1c296cff7114 in_seq=0 out_seq=0 2024-09-11T22:04:11.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.500+0000 7f102cff9640 1 -- 172.21.15.1:0/2491064258 <== mds.2 v2:172.21.15.53:6836/3457131830 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f103400fe60 con 0x562601c67670 2024-09-11T22:04:11.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.500+0000 7f102cff9640 10 client.16908 handle_command_reply: tid=0 2024-09-11T22:04:11.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.500+0000 7f104dde8740 1 client.16908 shutdown 2024-09-11T22:04:11.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.500+0000 7f104dde8740 1 -- 172.21.15.1:0/2491064258 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562601be15d0 msgr2=0x562601bf4a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:11.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.500+0000 7f104dde8740 1 --2- 172.21.15.1:0/2491064258 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562601be15d0 0x562601bf4a40 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f103c009c50 tx=0x7f103c009ca0 comp rx=0 tx=0).stop 2024-09-11T22:04:11.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.500+0000 7f104dde8740 1 -- 172.21.15.1:0/2491064258 shutdown_connections 2024-09-11T22:04:11.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.500+0000 7f104dde8740 1 --2- 172.21.15.1:0/2491064258 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x562601c67670 0x562601c69a90 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:11.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.500+0000 7f104dde8740 1 --2- 172.21.15.1:0/2491064258 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562601bec370 0x562601bf02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:11.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.500+0000 7f104dde8740 1 --2- 172.21.15.1:0/2491064258 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562601be2b40 0x562601befd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:11.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.500+0000 7f102cff9640 0 client.16908 ms_handle_reset on v2:172.21.15.53:6836/3457131830 2024-09-11T22:04:11.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.500+0000 7f104dde8740 1 --2- 172.21.15.1:0/2491064258 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562601be15d0 0x562601bf4a40 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:11.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.500+0000 7f104dde8740 1 -- 172.21.15.1:0/2491064258 >> 172.21.15.1:0/2491064258 conn(0x562601baa310 msgr2=0x562601be78a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:11.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.501+0000 7f104dde8740 1 -- 172.21.15.1:0/2491064258 shutdown_connections 2024-09-11T22:04:11.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.501+0000 7f104dde8740 1 -- 172.21.15.1:0/2491064258 wait complete. 2024-09-11T22:04:11.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.501+0000 7f104dde8740 20 client.16908 trim_cache size 0 max 16384 2024-09-11T22:04:11.503 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-11T22:04:11.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.502+0000 7f104ba0f640 1 -- 172.21.15.1:0/3671210751 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f10300a2d70 msgr2=0x7f10300a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:11.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.502+0000 7f104ba0f640 1 --2- 172.21.15.1:0/3671210751 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f10300a2d70 0x7f10300a5230 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x562601bdeac0 tx=0x7f10400058b0 comp rx=0 tx=0).stop 2024-09-11T22:04:11.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.502+0000 7f104ba0f640 1 -- 172.21.15.1:0/3671210751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1044101de0 msgr2=0x7f104419f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:11.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.502+0000 7f104ba0f640 1 --2- 172.21.15.1:0/3671210751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1044101de0 0x7f104419f190 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1040006010 tx=0x7f1040004040 comp rx=0 tx=0).stop 2024-09-11T22:04:11.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.502+0000 7f104ba0f640 1 -- 172.21.15.1:0/3671210751 shutdown_connections 2024-09-11T22:04:11.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.502+0000 7f104ba0f640 1 --2- 172.21.15.1:0/3671210751 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f10300a2d70 0x7f10300a5230 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:11.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.502+0000 7f104ba0f640 1 --2- 172.21.15.1:0/3671210751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1044106f10 0x7f104410de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:11.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.502+0000 7f104ba0f640 1 --2- 172.21.15.1:0/3671210751 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1044102790 0x7f104419f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:11.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.502+0000 7f104ba0f640 1 --2- 172.21.15.1:0/3671210751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1044101de0 0x7f104419f190 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:11.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.502+0000 7f104ba0f640 1 -- 172.21.15.1:0/3671210751 >> 172.21.15.1:0/3671210751 conn(0x7f10440fd880 msgr2=0x7f10440febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:11.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.502+0000 7f104ba0f640 1 -- 172.21.15.1:0/3671210751 shutdown_connections 2024-09-11T22:04:11.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:11.502+0000 7f104ba0f640 1 -- 172.21.15.1:0/3671210751 wait complete. 2024-09-11T22:04:12.581 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T22:04:12.877 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:04:13.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:04:12 smithi053 ceph-mon[34908]: pgmap v215: 97 pgs: 97 active+clean; 402 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.9 MiB/s wr, 573 op/s 2024-09-11T22:04:13.313 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:04:12 smithi001 ceph-mon[30935]: pgmap v215: 97 pgs: 97 active+clean; 402 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.9 MiB/s wr, 573 op/s 2024-09-11T22:04:13.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:04:12 smithi161 ceph-mon[35615]: pgmap v215: 97 pgs: 97 active+clean; 402 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.9 MiB/s wr, 573 op/s 2024-09-11T22:04:13.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.400+0000 7eff36c62640 1 -- 172.21.15.1:0/4262450881 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff30112640 msgr2=0x7eff30112a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:13.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.400+0000 7eff36c62640 1 --2- 172.21.15.1:0/4262450881 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff30112640 0x7eff30112a20 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7eff24009900 tx=0x7eff2402a410 comp rx=0 tx=0).stop 2024-09-11T22:04:13.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.400+0000 7eff36c62640 1 -- 172.21.15.1:0/4262450881 shutdown_connections 2024-09-11T22:04:13.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.400+0000 7eff36c62640 1 --2- 172.21.15.1:0/4262450881 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff3010cf30 0x7eff30119410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:13.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.400+0000 7eff36c62640 1 --2- 172.21.15.1:0/4262450881 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff3010c590 0x7eff3010c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:13.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.400+0000 7eff36c62640 1 --2- 172.21.15.1:0/4262450881 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff30112640 0x7eff30112a20 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:13.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.400+0000 7eff36c62640 1 -- 172.21.15.1:0/4262450881 >> 172.21.15.1:0/4262450881 conn(0x7eff3006cdc0 msgr2=0x7eff3006d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:13.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.400+0000 7eff36c62640 1 -- 172.21.15.1:0/4262450881 shutdown_connections 2024-09-11T22:04:13.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.400+0000 7eff36c62640 1 -- 172.21.15.1:0/4262450881 wait complete. 2024-09-11T22:04:13.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.401+0000 7eff36c62640 1 Processor -- start 2024-09-11T22:04:13.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.401+0000 7eff36c62640 1 -- start start 2024-09-11T22:04:13.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.401+0000 7eff36c62640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff3010c590 0x7eff301a2e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:13.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.401+0000 7eff36c62640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff3010cf30 0x7eff301a33b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:13.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.401+0000 7eff36c62640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff30112640 0x7eff301a7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:13.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.401+0000 7eff36c62640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7eff3011c9a0 con 0x7eff3010c590 2024-09-11T22:04:13.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.401+0000 7eff36c62640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7eff3011c820 con 0x7eff3010cf30 2024-09-11T22:04:13.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.401+0000 7eff36c62640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7eff3011cb20 con 0x7eff30112640 2024-09-11T22:04:13.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.401+0000 7eff349fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff3010c590 0x7eff301a2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:13.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.401+0000 7eff349fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff3010c590 0x7eff301a2e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55764/0 (socket says 172.21.15.1:55764) 2024-09-11T22:04:13.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.401+0000 7eff349fe640 1 -- 172.21.15.1:0/2172592609 learned_addr learned my addr 172.21.15.1:0/2172592609 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:13.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.401+0000 7eff351ff640 1 --2- 172.21.15.1:0/2172592609 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff30112640 0x7eff301a7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:13.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.401+0000 7eff2ffff640 1 --2- 172.21.15.1:0/2172592609 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff3010cf30 0x7eff301a33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:13.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.402+0000 7eff349fe640 1 -- 172.21.15.1:0/2172592609 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff30112640 msgr2=0x7eff301a7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:13.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.402+0000 7eff349fe640 1 --2- 172.21.15.1:0/2172592609 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff30112640 0x7eff301a7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:13.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.402+0000 7eff349fe640 1 -- 172.21.15.1:0/2172592609 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff3010cf30 msgr2=0x7eff301a33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:13.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.402+0000 7eff349fe640 1 --2- 172.21.15.1:0/2172592609 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff3010cf30 0x7eff301a33b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:13.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.402+0000 7eff349fe640 1 -- 172.21.15.1:0/2172592609 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff301a7e20 con 0x7eff3010c590 2024-09-11T22:04:13.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.402+0000 7eff349fe640 1 --2- 172.21.15.1:0/2172592609 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff3010c590 0x7eff301a2e70 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7eff24006010 tx=0x7eff24003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:13.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.402+0000 7eff2dffb640 1 -- 172.21.15.1:0/2172592609 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff2403f070 con 0x7eff3010c590 2024-09-11T22:04:13.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.402+0000 7eff36c62640 1 -- 172.21.15.1:0/2172592609 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff301a80b0 con 0x7eff3010c590 2024-09-11T22:04:13.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.402+0000 7eff2dffb640 1 -- 172.21.15.1:0/2172592609 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff24031c30 con 0x7eff3010c590 2024-09-11T22:04:13.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.402+0000 7eff2dffb640 1 -- 172.21.15.1:0/2172592609 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff24035030 con 0x7eff3010c590 2024-09-11T22:04:13.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.402+0000 7eff36c62640 1 -- 172.21.15.1:0/2172592609 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff301af9f0 con 0x7eff3010c590 2024-09-11T22:04:13.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.405+0000 7eff2dffb640 1 -- 172.21.15.1:0/2172592609 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7eff24042070 con 0x7eff3010c590 2024-09-11T22:04:13.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.405+0000 7eff3903b740 1 Processor -- start 2024-09-11T22:04:13.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.405+0000 7eff3903b740 1 -- start start 2024-09-11T22:04:13.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.405+0000 7eff3903b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e0cddbb40 0x556e0cdda090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:13.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.405+0000 7eff3903b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e0cdda5d0 0x556e0cddaa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:13.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.405+0000 7eff3903b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556e0cde5370 0x556e0cde7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:13.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.405+0000 7eff3903b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x556e0cd402f0 con 0x556e0cde5370 2024-09-11T22:04:13.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.405+0000 7eff3903b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x556e0cce7cf0 con 0x556e0cddbb40 2024-09-11T22:04:13.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.405+0000 7eff3903b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556e0cd3c2b0 con 0x556e0cdda5d0 2024-09-11T22:04:13.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.405+0000 7eff349fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e0cdda5d0 0x556e0cddaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:13.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.405+0000 7eff2ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556e0cde5370 0x556e0cde7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:13.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.405+0000 7eff349fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e0cdda5d0 0x556e0cddaa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:50884/0 (socket says 172.21.15.1:50884) 2024-09-11T22:04:13.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.405+0000 7eff349fe640 1 -- 172.21.15.1:0/105140452 learned_addr learned my addr 172.21.15.1:0/105140452 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:13.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.405+0000 7eff351ff640 1 --2- 172.21.15.1:0/105140452 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e0cddbb40 0x556e0cdda090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:13.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.406+0000 7eff349fe640 1 -- 172.21.15.1:0/105140452 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e0cddbb40 msgr2=0x556e0cdda090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:13.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.406+0000 7eff349fe640 1 --2- 172.21.15.1:0/105140452 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e0cddbb40 0x556e0cdda090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:13.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.406+0000 7eff349fe640 1 -- 172.21.15.1:0/105140452 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556e0cde5370 msgr2=0x556e0cde7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:13.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.406+0000 7eff349fe640 1 --2- 172.21.15.1:0/105140452 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556e0cde5370 0x556e0cde7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:13.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.406+0000 7eff349fe640 1 -- 172.21.15.1:0/105140452 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e0cccc3a0 con 0x556e0cdda5d0 2024-09-11T22:04:13.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.406+0000 7eff351ff640 1 --2- 172.21.15.1:0/105140452 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e0cddbb40 0x556e0cdda090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:04:13.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.406+0000 7eff2dffb640 1 --2- 172.21.15.1:0/2172592609 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7eff200a2cf0 0x7eff200a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:13.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.406+0000 7eff351ff640 1 --2- 172.21.15.1:0/2172592609 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7eff200a2cf0 0x7eff200a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:13.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.406+0000 7eff349fe640 1 --2- 172.21.15.1:0/105140452 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e0cdda5d0 0x556e0cddaa10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7eff240ed170 tx=0x7eff240f3e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd62761c40e9db41 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:13.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.406+0000 7eff2dffb640 1 -- 172.21.15.1:0/2172592609 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8673+0+0 (secure 0 0 0) 0x7eff240efb50 con 0x7eff3010c590 2024-09-11T22:04:13.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.406+0000 7eff1affd640 1 -- 172.21.15.1:0/105140452 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff24049030 con 0x556e0cdda5d0 2024-09-11T22:04:13.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.406+0000 7eff1affd640 1 -- 172.21.15.1:0/105140452 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff24031de0 con 0x556e0cdda5d0 2024-09-11T22:04:13.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.406+0000 7eff1affd640 1 -- 172.21.15.1:0/105140452 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff240bbe50 con 0x556e0cdda5d0 2024-09-11T22:04:13.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.407+0000 7eff3903b740 1 -- 172.21.15.1:0/105140452 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e0cdda5d0 msgr2=0x556e0cddaa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:13.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.407+0000 7eff351ff640 1 --2- 172.21.15.1:0/2172592609 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7eff200a2cf0 0x7eff200a51b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7eff28005fd0 tx=0x7eff28009500 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:13.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.407+0000 7eff3903b740 1 --2- 172.21.15.1:0/105140452 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e0cdda5d0 0x556e0cddaa10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7eff240ed170 tx=0x7eff240f3e20 comp rx=0 tx=0).stop 2024-09-11T22:04:13.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.407+0000 7eff3903b740 1 -- 172.21.15.1:0/105140452 shutdown_connections 2024-09-11T22:04:13.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.407+0000 7eff3903b740 1 --2- 172.21.15.1:0/105140452 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556e0cde5370 0x556e0cde7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:13.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.407+0000 7eff3903b740 1 --2- 172.21.15.1:0/105140452 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e0cdda5d0 0x556e0cddaa10 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:13.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.407+0000 7eff3903b740 1 --2- 172.21.15.1:0/105140452 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e0cddbb40 0x556e0cdda090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:13.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.407+0000 7eff3903b740 1 -- 172.21.15.1:0/105140452 >> 172.21.15.1:0/105140452 conn(0x556e0cda3310 msgr2=0x556e0cdd6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:13.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.407+0000 7eff3903b740 1 -- 172.21.15.1:0/105140452 shutdown_connections 2024-09-11T22:04:13.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.407+0000 7eff3903b740 1 -- 172.21.15.1:0/105140452 wait complete. 2024-09-11T22:04:13.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.407+0000 7eff3903b740 1 -- start start 2024-09-11T22:04:13.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.408+0000 7eff3903b740 1 Processor -- start 2024-09-11T22:04:13.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.408+0000 7eff3903b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e0cdda5d0 0x556e0cde4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:13.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.408+0000 7eff3903b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556e0cddbb40 0x556e0cde2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:13.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.408+0000 7eff3903b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e0cde5370 0x556e0cde27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:13.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.408+0000 7eff3903b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x556e0cde7e30 con 0x556e0cddbb40 2024-09-11T22:04:13.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.408+0000 7eff3903b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x556e0ccfb060 con 0x556e0cdda5d0 2024-09-11T22:04:13.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.408+0000 7eff3903b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556e0cde7fb0 con 0x556e0cde5370 2024-09-11T22:04:13.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.408+0000 7eff3903b740 10 client.0 _ll_register_callbacks cb 0x556e0ca4e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:04:13.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.408+0000 7eff2ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556e0cddbb40 0x556e0cde2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:13.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.408+0000 7eff349fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e0cdda5d0 0x556e0cde4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:13.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.408+0000 7eff349fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e0cdda5d0 0x556e0cde4da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:50830/0 (socket says 172.21.15.1:50830) 2024-09-11T22:04:13.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.408+0000 7eff2ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556e0cddbb40 0x556e0cde2050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55790/0 (socket says 172.21.15.1:55790) 2024-09-11T22:04:13.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.408+0000 7eff349fe640 1 -- 172.21.15.1:0/614872328 learned_addr learned my addr 172.21.15.1:0/614872328 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:13.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.408+0000 7eff351ff640 1 --2- 172.21.15.1:0/614872328 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e0cde5370 0x556e0cde27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:13.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.409+0000 7eff349fe640 1 -- 172.21.15.1:0/614872328 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e0cde5370 msgr2=0x556e0cde27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:13.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.409+0000 7eff349fe640 1 --2- 172.21.15.1:0/614872328 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e0cde5370 0x556e0cde27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:13.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.409+0000 7eff349fe640 1 -- 172.21.15.1:0/614872328 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556e0cddbb40 msgr2=0x556e0cde2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:13.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.409+0000 7eff349fe640 1 --2- 172.21.15.1:0/614872328 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556e0cddbb40 0x556e0cde2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:13.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.409+0000 7eff349fe640 1 -- 172.21.15.1:0/614872328 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e0cab13c0 con 0x556e0cdda5d0 2024-09-11T22:04:13.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.409+0000 7eff349fe640 1 --2- 172.21.15.1:0/614872328 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e0cdda5d0 0x556e0cde4da0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7eff240c4710 tx=0x7eff2403b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acb074c836c8e56 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:13.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.409+0000 7eff0ffff640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-11T22:04:13.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.409+0000 7eff0ffff640 1 -- 172.21.15.1:0/614872328 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff24049050 con 0x556e0cdda5d0 2024-09-11T22:04:13.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.409+0000 7eff0ffff640 1 -- 172.21.15.1:0/614872328 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8673+0+0 (secure 0 0 0) 0x7eff240ed480 con 0x556e0cdda5d0 2024-09-11T22:04:13.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.409+0000 7eff3903b740 1 -- 172.21.15.1:0/614872328 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff24031de0 con 0x556e0cdda5d0 2024-09-11T22:04:13.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.409+0000 7eff0ffff640 1 -- 172.21.15.1:0/614872328 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7eff240bb9e0 con 0x556e0cdda5d0 2024-09-11T22:04:13.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.409+0000 7eff3903b740 10 client.26447 fetch_fsmap learned FSMap version 42 2024-09-11T22:04:13.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.409+0000 7eff3903b740 1 -- 172.21.15.1:0/614872328 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x556e0cde7e30 con 0x556e0cdda5d0 2024-09-11T22:04:13.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.409+0000 7eff0ffff640 1 -- 172.21.15.1:0/614872328 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff24031de0 con 0x556e0cdda5d0 2024-09-11T22:04:13.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.410+0000 7eff0ffff640 1 -- 172.21.15.1:0/614872328 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7eff240b86f0 con 0x556e0cdda5d0 2024-09-11T22:04:13.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.410+0000 7eff3903b740 10 client.26447 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:04:13.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.410+0000 7eff3903b740 10 client.26447 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T22:04:13.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.410+0000 7eff3903b740 20 client.26447 populate_metadata read hostname 'smithi001' 2024-09-11T22:04:13.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.410+0000 7eff3903b740 1 --2- 172.21.15.1:0/614872328 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x556e0cdf4a70 0x556e0cdf6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:13.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.410+0000 7eff3903b740 4 client.26447 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:04:13.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.410+0000 7eff3903b740 1 -- 172.21.15.1:0/614872328 --> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e0cd4d860 con 0x556e0cdf4a70 2024-09-11T22:04:13.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.410+0000 7eff2ffff640 1 --2- 172.21.15.1:0/614872328 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x556e0cdf4a70 0x556e0cdf6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:13.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.410+0000 7eff2ffff640 1 --2- 172.21.15.1:0/614872328 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x556e0cdf4a70 0x556e0cdf6ea0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6495b5f2af635f04 server_cookie=955f2783debc5ceb in_seq=0 out_seq=0 2024-09-11T22:04:13.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.410+0000 7eff0ffff640 10 client.26447 ms_handle_connect on v2:172.21.15.161:6838/455365383 2024-09-11T22:04:13.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.411+0000 7eff0ffff640 1 -- 172.21.15.1:0/614872328 <== mds.1 v2:172.21.15.161:6838/455365383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556e0cd4d860 con 0x556e0cdf4a70 2024-09-11T22:04:13.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.411+0000 7eff0ffff640 10 client.26447 handle_command_reply: tid=0 2024-09-11T22:04:13.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.411+0000 7eff3903b740 1 client.26447 shutdown 2024-09-11T22:04:13.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.411+0000 7eff3903b740 1 -- 172.21.15.1:0/614872328 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e0cdda5d0 msgr2=0x556e0cde4da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:13.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.411+0000 7eff3903b740 1 --2- 172.21.15.1:0/614872328 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e0cdda5d0 0x556e0cde4da0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7eff240c4710 tx=0x7eff2403b790 comp rx=0 tx=0).stop 2024-09-11T22:04:13.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.411+0000 7eff3903b740 1 -- 172.21.15.1:0/614872328 shutdown_connections 2024-09-11T22:04:13.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.411+0000 7eff3903b740 1 --2- 172.21.15.1:0/614872328 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x556e0cdf4a70 0x556e0cdf6ea0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:13.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.411+0000 7eff3903b740 1 --2- 172.21.15.1:0/614872328 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e0cde5370 0x556e0cde27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:13.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.411+0000 7eff3903b740 1 --2- 172.21.15.1:0/614872328 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556e0cddbb40 0x556e0cde2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:13.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.411+0000 7eff3903b740 1 --2- 172.21.15.1:0/614872328 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e0cdda5d0 0x556e0cde4da0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:13.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.411+0000 7eff3903b740 1 -- 172.21.15.1:0/614872328 >> 172.21.15.1:0/614872328 conn(0x556e0cda3310 msgr2=0x556e0cddc960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:13.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.411+0000 7eff0ffff640 0 client.26447 ms_handle_reset on v2:172.21.15.161:6838/455365383 2024-09-11T22:04:13.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.412+0000 7eff3903b740 1 -- 172.21.15.1:0/614872328 shutdown_connections 2024-09-11T22:04:13.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.412+0000 7eff3903b740 1 -- 172.21.15.1:0/614872328 wait complete. 2024-09-11T22:04:13.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.412+0000 7eff3903b740 20 client.26447 trim_cache size 0 max 16384 2024-09-11T22:04:13.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.425+0000 7eff3903b740 1 Processor -- start 2024-09-11T22:04:13.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.425+0000 7eff3903b740 1 -- start start 2024-09-11T22:04:13.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.425+0000 7eff3903b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556e0cdda5d0 0x556e0cdfc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:13.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.425+0000 7eff3903b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e0cddbb40 0x556e0cdfccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:13.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.425+0000 7eff3903b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e0cde5370 0x556e0cddf2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:13.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.425+0000 7eff3903b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7eff301b8580 con 0x556e0cdda5d0 2024-09-11T22:04:13.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.425+0000 7eff3903b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7eff301ba3c0 con 0x556e0cde5370 2024-09-11T22:04:13.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.425+0000 7eff3903b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7eff301b8d10 con 0x556e0cddbb40 2024-09-11T22:04:13.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.425+0000 7eff349fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556e0cdda5d0 0x556e0cdfc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:13.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.425+0000 7eff2ffff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e0cddbb40 0x556e0cdfccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:13.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.425+0000 7eff349fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556e0cdda5d0 0x556e0cdfc7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55800/0 (socket says 172.21.15.1:55800) 2024-09-11T22:04:13.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.425+0000 7eff351ff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e0cde5370 0x556e0cddf2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:13.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.425+0000 7eff349fe640 1 -- 172.21.15.1:0/2905858407 learned_addr learned my addr 172.21.15.1:0/2905858407 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:13.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.426+0000 7eff351ff640 1 -- 172.21.15.1:0/2905858407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e0cddbb40 msgr2=0x556e0cdfccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:13.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.426+0000 7eff351ff640 1 --2- 172.21.15.1:0/2905858407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e0cddbb40 0x556e0cdfccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:13.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.426+0000 7eff351ff640 1 -- 172.21.15.1:0/2905858407 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556e0cdda5d0 msgr2=0x556e0cdfc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:13.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.426+0000 7eff351ff640 1 --2- 172.21.15.1:0/2905858407 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556e0cdda5d0 0x556e0cdfc7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:13.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.426+0000 7eff351ff640 1 -- 172.21.15.1:0/2905858407 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e0cdf7520 con 0x556e0cde5370 2024-09-11T22:04:13.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.426+0000 7eff2ffff640 1 --2- 172.21.15.1:0/2905858407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e0cddbb40 0x556e0cdfccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:04:13.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.426+0000 7eff351ff640 1 --2- 172.21.15.1:0/2905858407 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e0cde5370 0x556e0cddf2f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7eff28009fd0 tx=0x7eff2801b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b9379430faf065d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:13.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.426+0000 7eff1a7fc640 1 -- 172.21.15.1:0/2905858407 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff28004b20 con 0x556e0cde5370 2024-09-11T22:04:13.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.426+0000 7eff1a7fc640 1 -- 172.21.15.1:0/2905858407 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff28004cc0 con 0x556e0cde5370 2024-09-11T22:04:13.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.426+0000 7eff1a7fc640 1 -- 172.21.15.1:0/2905858407 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff2800f880 con 0x556e0cde5370 2024-09-11T22:04:13.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.427+0000 7eff3903b740 1 -- 172.21.15.1:0/2905858407 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e0cde5370 msgr2=0x556e0cddf2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:13.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.427+0000 7eff3903b740 1 --2- 172.21.15.1:0/2905858407 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e0cde5370 0x556e0cddf2f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7eff28009fd0 tx=0x7eff2801b000 comp rx=0 tx=0).stop 2024-09-11T22:04:13.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.427+0000 7eff3903b740 1 -- 172.21.15.1:0/2905858407 shutdown_connections 2024-09-11T22:04:13.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.427+0000 7eff3903b740 1 --2- 172.21.15.1:0/2905858407 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e0cde5370 0x556e0cddf2f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:13.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.427+0000 7eff3903b740 1 --2- 172.21.15.1:0/2905858407 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e0cddbb40 0x556e0cdfccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:13.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.427+0000 7eff3903b740 1 --2- 172.21.15.1:0/2905858407 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556e0cdda5d0 0x556e0cdfc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:13.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.427+0000 7eff3903b740 1 -- 172.21.15.1:0/2905858407 >> 172.21.15.1:0/2905858407 conn(0x556e0cda3310 msgr2=0x556e0cde1160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:13.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.427+0000 7eff3903b740 1 -- 172.21.15.1:0/2905858407 shutdown_connections 2024-09-11T22:04:13.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.427+0000 7eff3903b740 1 -- 172.21.15.1:0/2905858407 wait complete. 2024-09-11T22:04:13.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.427+0000 7eff3903b740 1 -- start start 2024-09-11T22:04:13.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.428+0000 7eff3903b740 1 Processor -- start 2024-09-11T22:04:13.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.428+0000 7eff3903b740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e0cdda5d0 0x556e0cded570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:13.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.428+0000 7eff3903b740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e0cddbb40 0x556e0cdedab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:13.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.428+0000 7eff3903b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556e0cde5370 0x556e0cdea420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:13.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.428+0000 7eff3903b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7eff301b9c30 con 0x556e0cde5370 2024-09-11T22:04:13.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.428+0000 7eff3903b740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7eff301b94a0 con 0x556e0cdda5d0 2024-09-11T22:04:13.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.428+0000 7eff3903b740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7eff301b6580 con 0x556e0cddbb40 2024-09-11T22:04:13.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.428+0000 7eff3903b740 10 client.0 _ll_register_callbacks cb 0x7eff1c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:04:13.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.428+0000 7eff351ff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556e0cde5370 0x556e0cdea420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:13.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.428+0000 7eff349fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e0cdda5d0 0x556e0cded570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:13.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.428+0000 7eff351ff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556e0cde5370 0x556e0cdea420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55810/0 (socket says 172.21.15.1:55810) 2024-09-11T22:04:13.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.428+0000 7eff351ff640 1 -- 172.21.15.1:0/2757457179 learned_addr learned my addr 172.21.15.1:0/2757457179 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:13.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.428+0000 7eff2ffff640 1 --2- 172.21.15.1:0/2757457179 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e0cddbb40 0x556e0cdedab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:13.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.428+0000 7eff351ff640 1 -- 172.21.15.1:0/2757457179 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e0cddbb40 msgr2=0x556e0cdedab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:13.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.428+0000 7eff351ff640 1 --2- 172.21.15.1:0/2757457179 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e0cddbb40 0x556e0cdedab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:13.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.429+0000 7eff351ff640 1 -- 172.21.15.1:0/2757457179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e0cdda5d0 msgr2=0x556e0cded570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:13.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.429+0000 7eff351ff640 1 --2- 172.21.15.1:0/2757457179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e0cdda5d0 0x556e0cded570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:13.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.429+0000 7eff351ff640 1 -- 172.21.15.1:0/2757457179 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e0cdd7960 con 0x556e0cde5370 2024-09-11T22:04:13.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.429+0000 7eff349fe640 1 --2- 172.21.15.1:0/2757457179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e0cdda5d0 0x556e0cded570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:04:13.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.429+0000 7eff351ff640 1 --2- 172.21.15.1:0/2757457179 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556e0cde5370 0x556e0cdea420 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7eff28019040 tx=0x7eff28012f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f19d3d044db0bac6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:13.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.429+0000 7eff0ffff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:04:13.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.429+0000 7eff0ffff640 1 -- 172.21.15.1:0/2757457179 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff28037060 con 0x556e0cde5370 2024-09-11T22:04:13.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.429+0000 7eff0ffff640 1 -- 172.21.15.1:0/2757457179 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8673+0+0 (secure 0 0 0) 0x7eff28005070 con 0x556e0cde5370 2024-09-11T22:04:13.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.429+0000 7eff3903b740 1 -- 172.21.15.1:0/2757457179 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff28004cc0 con 0x556e0cde5370 2024-09-11T22:04:13.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.429+0000 7eff0ffff640 1 -- 172.21.15.1:0/2757457179 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7eff2803e050 con 0x556e0cde5370 2024-09-11T22:04:13.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.429+0000 7eff3903b740 10 client.16926 fetch_fsmap learned FSMap version 42 2024-09-11T22:04:13.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.429+0000 7eff3903b740 1 -- 172.21.15.1:0/2757457179 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x556e0ce601d0 con 0x556e0cde5370 2024-09-11T22:04:13.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.429+0000 7eff0ffff640 1 -- 172.21.15.1:0/2757457179 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff28004cc0 con 0x556e0cde5370 2024-09-11T22:04:13.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.430+0000 7eff0ffff640 1 -- 172.21.15.1:0/2757457179 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7eff28031030 con 0x556e0cde5370 2024-09-11T22:04:13.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.430+0000 7eff3903b740 10 client.16926 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:04:13.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.430+0000 7eff3903b740 10 client.16926 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T22:04:13.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.430+0000 7eff3903b740 20 client.16926 populate_metadata read hostname 'smithi001' 2024-09-11T22:04:13.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.430+0000 7eff3903b740 1 --2- 172.21.15.1:0/2757457179 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x556e0ce608d0 0x556e0ce62cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:13.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.430+0000 7eff3903b740 4 client.16926 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:04:13.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.430+0000 7eff3903b740 1 -- 172.21.15.1:0/2757457179 --> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556e0cd47d70 con 0x556e0ce608d0 2024-09-11T22:04:13.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.430+0000 7eff349fe640 1 --2- 172.21.15.1:0/2757457179 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x556e0ce608d0 0x556e0ce62cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:13.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.430+0000 7eff0ffff640 10 client.16926 ms_handle_connect on v2:172.21.15.161:6838/455365383 2024-09-11T22:04:13.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.430+0000 7eff349fe640 1 --2- 172.21.15.1:0/2757457179 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x556e0ce608d0 0x556e0ce62cf0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d6a3ae3fcd7e8b28 server_cookie=ef1c14107c670c5 in_seq=0 out_seq=0 2024-09-11T22:04:13.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.431+0000 7eff0ffff640 1 -- 172.21.15.1:0/2757457179 <== mds.1 v2:172.21.15.161:6838/455365383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556e0cd47d70 con 0x556e0ce608d0 2024-09-11T22:04:13.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.431+0000 7eff0ffff640 10 client.16926 handle_command_reply: tid=0 2024-09-11T22:04:13.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.431+0000 7eff3903b740 1 client.16926 shutdown 2024-09-11T22:04:13.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.431+0000 7eff3903b740 1 -- 172.21.15.1:0/2757457179 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556e0cde5370 msgr2=0x556e0cdea420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:13.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.431+0000 7eff3903b740 1 --2- 172.21.15.1:0/2757457179 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556e0cde5370 0x556e0cdea420 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7eff28019040 tx=0x7eff28012f10 comp rx=0 tx=0).stop 2024-09-11T22:04:13.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.431+0000 7eff3903b740 1 -- 172.21.15.1:0/2757457179 shutdown_connections 2024-09-11T22:04:13.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.431+0000 7eff3903b740 1 --2- 172.21.15.1:0/2757457179 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x556e0ce608d0 0x556e0ce62cf0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:13.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.431+0000 7eff0ffff640 0 client.16926 ms_handle_reset on v2:172.21.15.161:6838/455365383 2024-09-11T22:04:13.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.431+0000 7eff3903b740 1 --2- 172.21.15.1:0/2757457179 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556e0cde5370 0x556e0cdea420 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:13.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.431+0000 7eff3903b740 1 --2- 172.21.15.1:0/2757457179 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556e0cddbb40 0x556e0cdedab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:13.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.431+0000 7eff3903b740 1 --2- 172.21.15.1:0/2757457179 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556e0cdda5d0 0x556e0cded570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:13.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.431+0000 7eff3903b740 1 -- 172.21.15.1:0/2757457179 >> 172.21.15.1:0/2757457179 conn(0x556e0cda3310 msgr2=0x556e0cdd54e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:13.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.431+0000 7eff3903b740 1 -- 172.21.15.1:0/2757457179 shutdown_connections 2024-09-11T22:04:13.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.431+0000 7eff3903b740 1 -- 172.21.15.1:0/2757457179 wait complete. 2024-09-11T22:04:13.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.431+0000 7eff3903b740 20 client.16926 trim_cache size 0 max 16384 2024-09-11T22:04:13.448 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-11T22:04:13.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.432+0000 7eff36c62640 1 -- 172.21.15.1:0/2172592609 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7eff200a2cf0 msgr2=0x7eff200a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:13.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.432+0000 7eff36c62640 1 --2- 172.21.15.1:0/2172592609 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7eff200a2cf0 0x7eff200a51b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7eff28005fd0 tx=0x7eff28009500 comp rx=0 tx=0).stop 2024-09-11T22:04:13.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.432+0000 7eff36c62640 1 -- 172.21.15.1:0/2172592609 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff3010c590 msgr2=0x7eff301a2e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:13.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.432+0000 7eff36c62640 1 --2- 172.21.15.1:0/2172592609 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff3010c590 0x7eff301a2e70 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7eff24006010 tx=0x7eff24003c10 comp rx=0 tx=0).stop 2024-09-11T22:04:13.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.432+0000 7eff36c62640 1 -- 172.21.15.1:0/2172592609 shutdown_connections 2024-09-11T22:04:13.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.432+0000 7eff36c62640 1 --2- 172.21.15.1:0/2172592609 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7eff200a2cf0 0x7eff200a51b0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:13.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.432+0000 7eff36c62640 1 --2- 172.21.15.1:0/2172592609 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff30112640 0x7eff301a7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:13.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.432+0000 7eff36c62640 1 --2- 172.21.15.1:0/2172592609 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff3010cf30 0x7eff301a33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:13.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.432+0000 7eff36c62640 1 --2- 172.21.15.1:0/2172592609 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff3010c590 0x7eff301a2e70 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:13.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.432+0000 7eff36c62640 1 -- 172.21.15.1:0/2172592609 >> 172.21.15.1:0/2172592609 conn(0x7eff3006cdc0 msgr2=0x7eff3010afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:13.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.432+0000 7eff36c62640 1 -- 172.21.15.1:0/2172592609 shutdown_connections 2024-09-11T22:04:13.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:13.432+0000 7eff36c62640 1 -- 172.21.15.1:0/2172592609 wait complete. 2024-09-11T22:04:13.714 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.713+0000 7faed6a1e640 1 -- 172.21.15.1:0/3550421474 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faed0101340 msgr2=0x7faed010d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:13.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.713+0000 7faed6a1e640 1 --2- 172.21.15.1:0/3550421474 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faed0101340 0x7faed010d870 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7faec8009ca0 tx=0x7faec802a8c0 comp rx=0 tx=0).stop 2024-09-11T22:04:13.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.713+0000 7faed6a1e640 1 -- 172.21.15.1:0/3550421474 shutdown_connections 2024-09-11T22:04:13.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.713+0000 7faed6a1e640 1 --2- 172.21.15.1:0/3550421474 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faed0101340 0x7faed010d870 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:13.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.713+0000 7faed6a1e640 1 --2- 172.21.15.1:0/3550421474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faed01009a0 0x7faed0100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:13.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.713+0000 7faed6a1e640 1 --2- 172.21.15.1:0/3550421474 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faed01069b0 0x7faed0106d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:13.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.713+0000 7faed6a1e640 1 -- 172.21.15.1:0/3550421474 >> 172.21.15.1:0/3550421474 conn(0x7faed00779a0 msgr2=0x7faed00fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:13.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.713+0000 7faed6a1e640 1 -- 172.21.15.1:0/3550421474 shutdown_connections 2024-09-11T22:04:13.715 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.713+0000 7faed6a1e640 1 -- 172.21.15.1:0/3550421474 wait complete. 2024-09-11T22:04:13.717 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.714+0000 7faed6a1e640 1 Processor -- start 2024-09-11T22:04:13.717 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.714+0000 7faed6a1e640 1 -- start start 2024-09-11T22:04:13.717 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.715+0000 7faed6a1e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faed01009a0 0x7faed010e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:13.717 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.715+0000 7faed6a1e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faed0101340 0x7faed010e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:13.718 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.715+0000 7faed6a1e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faed01069b0 0x7faed010ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:13.718 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.715+0000 7faed6a1e640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7faed0111f30 con 0x7faed01069b0 2024-09-11T22:04:13.718 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.715+0000 7faed6a1e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7faed0111db0 con 0x7faed0101340 2024-09-11T22:04:13.718 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.715+0000 7faed6a1e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7faed01120b0 con 0x7faed01009a0 2024-09-11T22:04:13.718 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.715+0000 7faed4fbb640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faed01069b0 0x7faed010ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:13.718 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.715+0000 7faed4fbb640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faed01069b0 0x7faed010ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55830/0 (socket says 172.21.15.1:55830) 2024-09-11T22:04:13.718 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.715+0000 7faecf7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faed0101340 0x7faed010e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:13.718 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.715+0000 7faed4fbb640 1 -- 172.21.15.1:0/3699407637 learned_addr learned my addr 172.21.15.1:0/3699407637 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:13.718 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.715+0000 7faecffff640 1 --2- 172.21.15.1:0/3699407637 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faed01009a0 0x7faed010e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:13.718 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.715+0000 7faed4fbb640 1 -- 172.21.15.1:0/3699407637 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faed01009a0 msgr2=0x7faed010e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:13.718 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.715+0000 7faed4fbb640 1 --2- 172.21.15.1:0/3699407637 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faed01009a0 0x7faed010e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:13.718 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.715+0000 7faed4fbb640 1 -- 172.21.15.1:0/3699407637 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faed0101340 msgr2=0x7faed010e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:13.718 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.715+0000 7faed4fbb640 1 --2- 172.21.15.1:0/3699407637 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faed0101340 0x7faed010e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:13.719 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.715+0000 7faed4fbb640 1 -- 172.21.15.1:0/3699407637 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faed01a6900 con 0x7faed01069b0 2024-09-11T22:04:13.719 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.715+0000 7faecf7fe640 1 --2- 172.21.15.1:0/3699407637 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faed0101340 0x7faed010e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:04:13.719 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.716+0000 7faed4fbb640 1 --2- 172.21.15.1:0/3699407637 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faed01069b0 0x7faed010ea80 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7faec8033040 tx=0x7faec8003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:13.719 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.716+0000 7faecd7fa640 1 -- 172.21.15.1:0/3699407637 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec8031650 con 0x7faed01069b0 2024-09-11T22:04:13.719 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.716+0000 7faecd7fa640 1 -- 172.21.15.1:0/3699407637 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faec80317f0 con 0x7faed01069b0 2024-09-11T22:04:13.719 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.716+0000 7faed6a1e640 1 -- 172.21.15.1:0/3699407637 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faed01a6b90 con 0x7faed01069b0 2024-09-11T22:04:13.719 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.716+0000 7faecd7fa640 1 -- 172.21.15.1:0/3699407637 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec802fa40 con 0x7faed01069b0 2024-09-11T22:04:13.719 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.716+0000 7faed6a1e640 1 -- 172.21.15.1:0/3699407637 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faed01a70e0 con 0x7faed01069b0 2024-09-11T22:04:13.724 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.718+0000 7faecd7fa640 1 -- 172.21.15.1:0/3699407637 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7faec802fcc0 con 0x7faed01069b0 2024-09-11T22:04:13.724 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.719+0000 7faed988d740 1 Processor -- start 2024-09-11T22:04:13.724 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.719+0000 7faed988d740 1 -- start start 2024-09-11T22:04:13.724 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.719+0000 7faed988d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559d8612af0 0x5559d8612ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:13.724 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.719+0000 7faed988d740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5559d8613410 0x5559d861d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:13.724 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.719+0000 7faed988d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5559d861dcb0 0x5559d86200a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:13.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.719+0000 7faed988d740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5559d85772f0 con 0x5559d8613410 2024-09-11T22:04:13.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.719+0000 7faed988d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5559d851f270 con 0x5559d861dcb0 2024-09-11T22:04:13.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.719+0000 7faed988d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5559d85732b0 con 0x5559d8612af0 2024-09-11T22:04:13.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.719+0000 7faecffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5559d8613410 0x5559d861d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:13.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.719+0000 7faecffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5559d8613410 0x5559d861d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55834/0 (socket says 172.21.15.1:55834) 2024-09-11T22:04:13.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.719+0000 7faecffff640 1 -- 172.21.15.1:0/4265897288 learned_addr learned my addr 172.21.15.1:0/4265897288 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:13.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.719+0000 7faecf7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559d8612af0 0x5559d8612ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:13.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.719+0000 7faecd7fa640 1 --2- 172.21.15.1:0/3699407637 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7faeb80a2cf0 0x7faeb80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:13.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.719+0000 7faecf7fe640 1 --2- 172.21.15.1:0/4265897288 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5559d861dcb0 0x5559d86200a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:13.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.719+0000 7faecffff640 1 -- 172.21.15.1:0/4265897288 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559d8612af0 msgr2=0x5559d8612ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:13.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.719+0000 7faecffff640 1 --2- 172.21.15.1:0/4265897288 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559d8612af0 0x5559d8612ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:13.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.719+0000 7faed4fbb640 1 --2- 172.21.15.1:0/3699407637 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7faeb80a2cf0 0x7faeb80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:13.726 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.719+0000 7faecffff640 1 -- 172.21.15.1:0/4265897288 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5559d861dcb0 msgr2=0x5559d86200a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:13.726 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.719+0000 7faecffff640 1 --2- 172.21.15.1:0/4265897288 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5559d861dcb0 0x5559d86200a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:13.726 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.719+0000 7faecffff640 1 -- 172.21.15.1:0/4265897288 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559d851e340 con 0x5559d8613410 2024-09-11T22:04:13.726 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.719+0000 7faecf7fe640 1 --2- 172.21.15.1:0/4265897288 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559d8612af0 0x5559d8612ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:04:13.726 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.719+0000 7faecd7fa640 1 -- 172.21.15.1:0/3699407637 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7faec80f07f0 con 0x7faed01069b0 2024-09-11T22:04:13.726 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.720+0000 7faecffff640 1 --2- 172.21.15.1:0/4265897288 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5559d8613410 0x5559d861d770 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7faec000bc90 tx=0x7faec00107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dc151d506c390b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:13.726 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.720+0000 7faeb67fc640 1 -- 172.21.15.1:0/4265897288 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec001c480 con 0x5559d8613410 2024-09-11T22:04:13.726 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.720+0000 7faed4fbb640 1 --2- 172.21.15.1:0/3699407637 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7faeb80a2cf0 0x7faeb80a51b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7faec80ee090 tx=0x7faec8041910 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:13.726 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.720+0000 7faeb67fc640 1 -- 172.21.15.1:0/4265897288 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faec001cb00 con 0x5559d8613410 2024-09-11T22:04:13.726 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.720+0000 7faeb67fc640 1 -- 172.21.15.1:0/4265897288 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec0011b40 con 0x5559d8613410 2024-09-11T22:04:13.726 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.721+0000 7faed988d740 1 -- 172.21.15.1:0/4265897288 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5559d8613410 msgr2=0x5559d861d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:13.726 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.721+0000 7faed988d740 1 --2- 172.21.15.1:0/4265897288 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5559d8613410 0x5559d861d770 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7faec000bc90 tx=0x7faec00107a0 comp rx=0 tx=0).stop 2024-09-11T22:04:13.727 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.721+0000 7faed988d740 1 -- 172.21.15.1:0/4265897288 shutdown_connections 2024-09-11T22:04:13.727 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.721+0000 7faed988d740 1 --2- 172.21.15.1:0/4265897288 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5559d861dcb0 0x5559d86200a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:13.727 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.721+0000 7faed988d740 1 --2- 172.21.15.1:0/4265897288 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5559d8613410 0x5559d861d770 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:13.727 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.721+0000 7faed988d740 1 --2- 172.21.15.1:0/4265897288 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559d8612af0 0x5559d8612ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:13.727 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.721+0000 7faed988d740 1 -- 172.21.15.1:0/4265897288 >> 172.21.15.1:0/4265897288 conn(0x5559d860f060 msgr2=0x5559d860f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:13.727 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.721+0000 7faed988d740 1 -- 172.21.15.1:0/4265897288 shutdown_connections 2024-09-11T22:04:13.727 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.721+0000 7faed988d740 1 -- 172.21.15.1:0/4265897288 wait complete. 2024-09-11T22:04:13.727 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.721+0000 7faed988d740 1 -- start start 2024-09-11T22:04:13.727 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.721+0000 7faed988d740 1 Processor -- start 2024-09-11T22:04:13.727 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.722+0000 7faed988d740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5559d8612af0 0x5559d861ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:13.727 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.722+0000 7faed988d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559d8613410 0x5559d861b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:13.727 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.722+0000 7faed988d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5559d861dcb0 0x5559d861b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:13.728 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.722+0000 7faed988d740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5559d8620880 con 0x5559d8612af0 2024-09-11T22:04:13.728 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.722+0000 7faed988d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5559d8500290 con 0x5559d861dcb0 2024-09-11T22:04:13.728 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.722+0000 7faed988d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5559d8620a00 con 0x5559d8613410 2024-09-11T22:04:13.728 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.722+0000 7faed988d740 10 client.0 _ll_register_callbacks cb 0x5559d8531140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:04:13.728 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.722+0000 7faecf7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5559d8612af0 0x5559d861ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:13.728 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.722+0000 7faecf7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5559d8612af0 0x5559d861ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55848/0 (socket says 172.21.15.1:55848) 2024-09-11T22:04:13.728 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.722+0000 7faecf7fe640 1 -- 172.21.15.1:0/3662137958 learned_addr learned my addr 172.21.15.1:0/3662137958 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:13.728 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.722+0000 7faecffff640 1 --2- 172.21.15.1:0/3662137958 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559d8613410 0x5559d861b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:13.728 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.722+0000 7faecf7fe640 1 --2- 172.21.15.1:0/3662137958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5559d861dcb0 0x5559d861b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:13.728 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.722+0000 7faecffff640 1 -- 172.21.15.1:0/3662137958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5559d861dcb0 msgr2=0x5559d861b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:13.728 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.722+0000 7faecffff640 1 --2- 172.21.15.1:0/3662137958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5559d861dcb0 0x5559d861b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:13.728 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.722+0000 7faecffff640 1 -- 172.21.15.1:0/3662137958 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5559d8612af0 msgr2=0x5559d861ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:13.729 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.722+0000 7faecffff640 1 --2- 172.21.15.1:0/3662137958 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5559d8612af0 0x5559d861ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:13.729 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.722+0000 7faecffff640 1 -- 172.21.15.1:0/3662137958 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559d828a210 con 0x5559d8613410 2024-09-11T22:04:13.729 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.722+0000 7faecffff640 1 --2- 172.21.15.1:0/3662137958 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559d8613410 0x5559d861b0d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7faec0000c00 tx=0x7faec001c710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7e512f7e0f0341a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:13.729 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.722+0000 7faeaf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-11T22:04:13.729 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.723+0000 7faeaf7fe640 1 -- 172.21.15.1:0/3662137958 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec000bd80 con 0x5559d8613410 2024-09-11T22:04:13.729 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.723+0000 7faeaf7fe640 1 -- 172.21.15.1:0/3662137958 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7faec001c900 con 0x5559d8613410 2024-09-11T22:04:13.729 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.723+0000 7faed988d740 1 -- 172.21.15.1:0/3662137958 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faec001cb00 con 0x5559d8613410 2024-09-11T22:04:13.729 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.723+0000 7faeaf7fe640 1 -- 172.21.15.1:0/3662137958 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7faec0011ad0 con 0x5559d8613410 2024-09-11T22:04:13.729 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.723+0000 7faed988d740 10 client.26437 fetch_fsmap learned FSMap version 42 2024-09-11T22:04:13.729 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.723+0000 7faed988d740 1 -- 172.21.15.1:0/3662137958 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5559d8356620 con 0x5559d8613410 2024-09-11T22:04:13.729 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.723+0000 7faeaf7fe640 1 -- 172.21.15.1:0/3662137958 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faec001cb00 con 0x5559d8613410 2024-09-11T22:04:13.729 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.723+0000 7faeaf7fe640 1 -- 172.21.15.1:0/3662137958 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7faec0005340 con 0x5559d8613410 2024-09-11T22:04:13.730 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.723+0000 7faed988d740 10 client.26437 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:04:13.730 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.723+0000 7faed988d740 10 client.26437 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:04:13.730 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.723+0000 7faed988d740 20 client.26437 populate_metadata read hostname 'smithi001' 2024-09-11T22:04:13.730 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.723+0000 7faed988d740 1 --2- 172.21.15.1:0/3662137958 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5559d862dbc0 0x5559d862ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:13.730 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.723+0000 7faed988d740 4 client.26437 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:04:13.730 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.723+0000 7faed988d740 1 -- 172.21.15.1:0/3662137958 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5559d8584a20 con 0x5559d862dbc0 2024-09-11T22:04:13.730 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.724+0000 7faecf7fe640 1 --2- 172.21.15.1:0/3662137958 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5559d862dbc0 0x5559d862ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:13.731 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.724+0000 7faecf7fe640 1 --2- 172.21.15.1:0/3662137958 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5559d862dbc0 0x5559d862ffb0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29acb139ea9c96d0 server_cookie=26b34740ed4c9aa in_seq=0 out_seq=0 2024-09-11T22:04:13.731 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.724+0000 7faeaf7fe640 10 client.26437 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:04:13.731 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.725+0000 7faeaf7fe640 1 -- 172.21.15.1:0/3662137958 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5559d8584a20 con 0x5559d862dbc0 2024-09-11T22:04:13.732 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.725+0000 7faeaf7fe640 10 client.26437 handle_command_reply: tid=0 2024-09-11T22:04:13.732 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.725+0000 7faed988d740 1 client.26437 shutdown 2024-09-11T22:04:13.732 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.725+0000 7faed988d740 1 -- 172.21.15.1:0/3662137958 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559d8613410 msgr2=0x5559d861b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:13.732 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.725+0000 7faed988d740 1 --2- 172.21.15.1:0/3662137958 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559d8613410 0x5559d861b0d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7faec0000c00 tx=0x7faec001c710 comp rx=0 tx=0).stop 2024-09-11T22:04:13.732 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.725+0000 7faed988d740 1 -- 172.21.15.1:0/3662137958 shutdown_connections 2024-09-11T22:04:13.732 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.725+0000 7faed988d740 1 --2- 172.21.15.1:0/3662137958 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5559d862dbc0 0x5559d862ffb0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:13.732 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.725+0000 7faed988d740 1 --2- 172.21.15.1:0/3662137958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5559d861dcb0 0x5559d861b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:13.732 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.725+0000 7faed988d740 1 --2- 172.21.15.1:0/3662137958 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559d8613410 0x5559d861b0d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:13.732 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.725+0000 7faeaf7fe640 0 client.26437 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:04:13.732 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.725+0000 7faed988d740 1 --2- 172.21.15.1:0/3662137958 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5559d8612af0 0x5559d861ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:13.732 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.725+0000 7faed988d740 1 -- 172.21.15.1:0/3662137958 >> 172.21.15.1:0/3662137958 conn(0x5559d860f060 msgr2=0x5559d8614d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:13.732 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.725+0000 7faed988d740 1 -- 172.21.15.1:0/3662137958 shutdown_connections 2024-09-11T22:04:13.733 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.725+0000 7faed988d740 1 -- 172.21.15.1:0/3662137958 wait complete. 2024-09-11T22:04:13.733 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.726+0000 7faed988d740 20 client.26437 trim_cache size 0 max 16384 2024-09-11T22:04:13.746 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.744+0000 7faed988d740 1 Processor -- start 2024-09-11T22:04:13.746 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.744+0000 7faed988d740 1 -- start start 2024-09-11T22:04:13.746 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.744+0000 7faed988d740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5559d8612af0 0x5559d8615560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:13.746 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.744+0000 7faed988d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559d8613410 0x5559d861b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:13.746 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.744+0000 7faed988d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5559d861dcb0 0x5559d8635760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:13.746 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.744+0000 7faed988d740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7faed01afcb0 con 0x5559d8612af0 2024-09-11T22:04:13.746 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.744+0000 7faed988d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7faed01b1af0 con 0x5559d861dcb0 2024-09-11T22:04:13.746 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.744+0000 7faed988d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7faed01b0440 con 0x5559d8613410 2024-09-11T22:04:13.747 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.744+0000 7faecf7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5559d8612af0 0x5559d8615560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:13.747 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.744+0000 7faecf7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5559d8612af0 0x5559d8615560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55862/0 (socket says 172.21.15.1:55862) 2024-09-11T22:04:13.747 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.744+0000 7faecf7fe640 1 -- 172.21.15.1:0/2980736959 learned_addr learned my addr 172.21.15.1:0/2980736959 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:13.747 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.744+0000 7faecf7fe640 1 --2- 172.21.15.1:0/2980736959 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5559d861dcb0 0x5559d8635760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:13.747 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.744+0000 7faecffff640 1 --2- 172.21.15.1:0/2980736959 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559d8613410 0x5559d861b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:13.747 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.744+0000 7faecf7fe640 1 -- 172.21.15.1:0/2980736959 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559d8613410 msgr2=0x5559d861b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:13.747 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.744+0000 7faecf7fe640 1 --2- 172.21.15.1:0/2980736959 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559d8613410 0x5559d861b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:13.747 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.744+0000 7faecf7fe640 1 -- 172.21.15.1:0/2980736959 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5559d861dcb0 msgr2=0x5559d8635760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:13.747 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.744+0000 7faecf7fe640 1 --2- 172.21.15.1:0/2980736959 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5559d861dcb0 0x5559d8635760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:13.747 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.744+0000 7faecf7fe640 1 -- 172.21.15.1:0/2980736959 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559d8612870 con 0x5559d8612af0 2024-09-11T22:04:13.747 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.745+0000 7faecf7fe640 1 --2- 172.21.15.1:0/2980736959 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5559d8612af0 0x5559d8615560 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7faec4000c00 tx=0x7faec4013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dd1ba3af0ed6ce3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:13.748 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.745+0000 7faeb5ffb640 1 -- 172.21.15.1:0/2980736959 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec4011050 con 0x5559d8612af0 2024-09-11T22:04:13.748 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.745+0000 7faeb5ffb640 1 -- 172.21.15.1:0/2980736959 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faec40111f0 con 0x5559d8612af0 2024-09-11T22:04:13.748 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.745+0000 7faeb5ffb640 1 -- 172.21.15.1:0/2980736959 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec4003e90 con 0x5559d8612af0 2024-09-11T22:04:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.746+0000 7faed988d740 1 -- 172.21.15.1:0/2980736959 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5559d8612af0 msgr2=0x5559d8615560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.746+0000 7faed988d740 1 --2- 172.21.15.1:0/2980736959 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5559d8612af0 0x5559d8615560 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7faec4000c00 tx=0x7faec4013040 comp rx=0 tx=0).stop 2024-09-11T22:04:13.752 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.746+0000 7faed988d740 1 -- 172.21.15.1:0/2980736959 shutdown_connections 2024-09-11T22:04:13.752 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.746+0000 7faed988d740 1 --2- 172.21.15.1:0/2980736959 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5559d861dcb0 0x5559d8635760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:13.752 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.746+0000 7faed988d740 1 --2- 172.21.15.1:0/2980736959 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559d8613410 0x5559d861b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:13.752 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.746+0000 7faed988d740 1 --2- 172.21.15.1:0/2980736959 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5559d8612af0 0x5559d8615560 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:13.752 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.746+0000 7faed988d740 1 -- 172.21.15.1:0/2980736959 >> 172.21.15.1:0/2980736959 conn(0x5559d860f060 msgr2=0x5559d861cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:13.752 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.746+0000 7faed988d740 1 -- 172.21.15.1:0/2980736959 shutdown_connections 2024-09-11T22:04:13.752 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.746+0000 7faed988d740 1 -- 172.21.15.1:0/2980736959 wait complete. 2024-09-11T22:04:13.752 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.746+0000 7faed988d740 1 -- start start 2024-09-11T22:04:13.752 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.747+0000 7faed988d740 1 Processor -- start 2024-09-11T22:04:13.752 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.747+0000 7faed988d740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5559d8612af0 0x5559d86213f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:13.752 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.747+0000 7faed988d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559d8613410 0x5559d8621930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:13.752 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.747+0000 7faed988d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5559d861dcb0 0x5559d8696720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:13.753 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.747+0000 7faed988d740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7faed01b1360 con 0x5559d8612af0 2024-09-11T22:04:13.753 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.747+0000 7faed988d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7faed01b0bd0 con 0x5559d861dcb0 2024-09-11T22:04:13.753 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.747+0000 7faed988d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7faed01adcb0 con 0x5559d8613410 2024-09-11T22:04:13.753 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.747+0000 7faed988d740 10 client.0 _ll_register_callbacks cb 0x7faec4011610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:04:13.753 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.748+0000 7faecf7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5559d861dcb0 0x5559d8696720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:13.753 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.748+0000 7faecf7fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5559d861dcb0 0x5559d8696720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:50920/0 (socket says 172.21.15.1:50920) 2024-09-11T22:04:13.753 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.748+0000 7faecf7fe640 1 -- 172.21.15.1:0/4235224307 learned_addr learned my addr 172.21.15.1:0/4235224307 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:13.753 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.748+0000 7faecffff640 1 --2- 172.21.15.1:0/4235224307 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559d8613410 0x5559d8621930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:13.753 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.748+0000 7faecf7fe640 1 --2- 172.21.15.1:0/4235224307 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5559d8612af0 0x5559d86213f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:13.753 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.748+0000 7faecf7fe640 1 -- 172.21.15.1:0/4235224307 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559d8613410 msgr2=0x5559d8621930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:13.753 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.748+0000 7faecf7fe640 1 --2- 172.21.15.1:0/4235224307 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559d8613410 0x5559d8621930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:13.753 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.748+0000 7faecf7fe640 1 -- 172.21.15.1:0/4235224307 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5559d8612af0 msgr2=0x5559d86213f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:13.754 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.748+0000 7faecf7fe640 1 --2- 172.21.15.1:0/4235224307 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5559d8612af0 0x5559d86213f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:13.754 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.748+0000 7faecf7fe640 1 -- 172.21.15.1:0/4235224307 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559d8696d60 con 0x5559d861dcb0 2024-09-11T22:04:13.754 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.748+0000 7faecf7fe640 1 --2- 172.21.15.1:0/4235224307 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5559d861dcb0 0x5559d8696720 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7faec4011480 tx=0x7faec40040f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=826a58919e125fe0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:13.754 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.748+0000 7faeaf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-11T22:04:13.754 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.749+0000 7faeaf7fe640 1 -- 172.21.15.1:0/4235224307 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec4032030 con 0x5559d861dcb0 2024-09-11T22:04:13.754 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.749+0000 7faed988d740 1 -- 172.21.15.1:0/4235224307 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faec40111f0 con 0x5559d861dcb0 2024-09-11T22:04:13.754 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.749+0000 7faeaf7fe640 1 -- 172.21.15.1:0/4235224307 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7faec4039070 con 0x5559d861dcb0 2024-09-11T22:04:13.754 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.749+0000 7faeaf7fe640 1 -- 172.21.15.1:0/4235224307 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7faec4005730 con 0x5559d861dcb0 2024-09-11T22:04:13.754 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.749+0000 7faed988d740 10 client.26471 fetch_fsmap learned FSMap version 42 2024-09-11T22:04:13.754 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.749+0000 7faed988d740 1 -- 172.21.15.1:0/4235224307 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5559d8698070 con 0x5559d861dcb0 2024-09-11T22:04:13.754 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.749+0000 7faeaf7fe640 1 -- 172.21.15.1:0/4235224307 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faec40111f0 con 0x5559d861dcb0 2024-09-11T22:04:13.754 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.749+0000 7faeaf7fe640 1 -- 172.21.15.1:0/4235224307 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7faec402a070 con 0x5559d861dcb0 2024-09-11T22:04:13.754 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.749+0000 7faed988d740 10 client.26471 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:04:13.755 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.749+0000 7faed988d740 10 client.26471 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:04:13.755 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.749+0000 7faed988d740 20 client.26471 populate_metadata read hostname 'smithi001' 2024-09-11T22:04:13.755 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.749+0000 7faed988d740 1 --2- 172.21.15.1:0/4235224307 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5559d8698750 0x5559d869ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:13.755 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.749+0000 7faed988d740 4 client.26471 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:04:13.755 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.749+0000 7faed988d740 1 -- 172.21.15.1:0/4235224307 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5559d857ed70 con 0x5559d8698750 2024-09-11T22:04:13.755 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.750+0000 7faecffff640 1 --2- 172.21.15.1:0/4235224307 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5559d8698750 0x5559d869ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:13.755 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.750+0000 7faeaf7fe640 10 client.26471 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:04:13.755 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.750+0000 7faecffff640 1 --2- 172.21.15.1:0/4235224307 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5559d8698750 0x5559d869ab70 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a875715d4af815de server_cookie=c5391c9abc28aef3 in_seq=0 out_seq=0 2024-09-11T22:04:13.953 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.952+0000 7faeaf7fe640 1 -- 172.21.15.1:0/4235224307 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5559d857ed70 con 0x5559d8698750 2024-09-11T22:04:13.954 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.952+0000 7faeaf7fe640 10 client.26471 handle_command_reply: tid=0 2024-09-11T22:04:13.954 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.952+0000 7faed988d740 1 client.26471 shutdown 2024-09-11T22:04:13.954 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.952+0000 7faed988d740 1 -- 172.21.15.1:0/4235224307 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5559d861dcb0 msgr2=0x5559d8696720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:13.954 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.952+0000 7faed988d740 1 --2- 172.21.15.1:0/4235224307 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5559d861dcb0 0x5559d8696720 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7faec4011480 tx=0x7faec40040f0 comp rx=0 tx=0).stop 2024-09-11T22:04:13.954 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.952+0000 7faed988d740 1 -- 172.21.15.1:0/4235224307 shutdown_connections 2024-09-11T22:04:13.954 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.952+0000 7faed988d740 1 --2- 172.21.15.1:0/4235224307 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5559d8698750 0x5559d869ab70 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:13.954 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.952+0000 7faed988d740 1 --2- 172.21.15.1:0/4235224307 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5559d861dcb0 0x5559d8696720 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:13.954 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.952+0000 7faed988d740 1 --2- 172.21.15.1:0/4235224307 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5559d8613410 0x5559d8621930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:13.954 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.952+0000 7faed988d740 1 --2- 172.21.15.1:0/4235224307 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5559d8612af0 0x5559d86213f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:13.954 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.952+0000 7faeaf7fe640 0 client.26471 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:04:13.954 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.952+0000 7faed988d740 1 -- 172.21.15.1:0/4235224307 >> 172.21.15.1:0/4235224307 conn(0x5559d860f060 msgr2=0x5559d861c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:13.954 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.952+0000 7faed988d740 1 -- 172.21.15.1:0/4235224307 shutdown_connections 2024-09-11T22:04:13.955 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.953+0000 7faed988d740 1 -- 172.21.15.1:0/4235224307 wait complete. 2024-09-11T22:04:13.955 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.953+0000 7faed988d740 20 client.26471 trim_cache size 0 max 16384 2024-09-11T22:04:13.955 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-09-11T22:04:13.955 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-09-11T22:04:13.955 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-09-11T22:04:13.955 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-09-11T22:04:13.956 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.953+0000 7faed6a1e640 1 -- 172.21.15.1:0/3699407637 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7faeb80a2cf0 msgr2=0x7faeb80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:13.956 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.953+0000 7faed6a1e640 1 --2- 172.21.15.1:0/3699407637 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7faeb80a2cf0 0x7faeb80a51b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7faec80ee090 tx=0x7faec8041910 comp rx=0 tx=0).stop 2024-09-11T22:04:13.956 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.953+0000 7faed6a1e640 1 -- 172.21.15.1:0/3699407637 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faed01069b0 msgr2=0x7faed010ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:13.956 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.953+0000 7faed6a1e640 1 --2- 172.21.15.1:0/3699407637 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faed01069b0 0x7faed010ea80 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7faec8033040 tx=0x7faec8003f10 comp rx=0 tx=0).stop 2024-09-11T22:04:13.956 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.953+0000 7faed6a1e640 1 -- 172.21.15.1:0/3699407637 shutdown_connections 2024-09-11T22:04:13.956 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.953+0000 7faed6a1e640 1 --2- 172.21.15.1:0/3699407637 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7faeb80a2cf0 0x7faeb80a51b0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:13.957 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.953+0000 7faed6a1e640 1 --2- 172.21.15.1:0/3699407637 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faed01069b0 0x7faed010ea80 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:13.957 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.953+0000 7faed6a1e640 1 --2- 172.21.15.1:0/3699407637 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faed0101340 0x7faed010e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:13.957 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.953+0000 7faed6a1e640 1 --2- 172.21.15.1:0/3699407637 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faed01009a0 0x7faed010e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:13.957 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.953+0000 7faed6a1e640 1 -- 172.21.15.1:0/3699407637 >> 172.21.15.1:0/3699407637 conn(0x7faed00779a0 msgr2=0x7faed0103740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:13.957 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.953+0000 7faed6a1e640 1 -- 172.21.15.1:0/3699407637 shutdown_connections 2024-09-11T22:04:13.957 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:13.953+0000 7faed6a1e640 1 -- 172.21.15.1:0/3699407637 wait complete. 2024-09-11T22:04:14.154 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T22:04:14.468 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:04:14.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:04:14 smithi001 ceph-mon[30935]: osdmap e91: 12 total, 12 up, 12 in 2024-09-11T22:04:14.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:04:14 smithi001 ceph-mon[30935]: pgmap v217: 97 pgs: 97 active+clean; 402 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.8 MiB/s wr, 253 op/s 2024-09-11T22:04:15.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:04:14 smithi053 ceph-mon[34908]: osdmap e91: 12 total, 12 up, 12 in 2024-09-11T22:04:15.048 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:04:14 smithi053 ceph-mon[34908]: pgmap v217: 97 pgs: 97 active+clean; 402 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.8 MiB/s wr, 253 op/s 2024-09-11T22:04:15.091 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:04:14 smithi161 ceph-mon[35615]: osdmap e91: 12 total, 12 up, 12 in 2024-09-11T22:04:15.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:04:14 smithi161 ceph-mon[35615]: pgmap v217: 97 pgs: 97 active+clean; 402 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.8 MiB/s wr, 253 op/s 2024-09-11T22:04:15.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.240+0000 7f6dcc2f4640 1 -- 172.21.15.1:0/1953675370 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6dc4102d40 msgr2=0x7f6dc4103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:15.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.240+0000 7f6dcc2f4640 1 --2- 172.21.15.1:0/1953675370 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6dc4102d40 0x7f6dc4103120 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0009900 tx=0x7f6dc002a480 comp rx=0 tx=0).stop 2024-09-11T22:04:15.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.240+0000 7f6dcc2f4640 1 -- 172.21.15.1:0/1953675370 shutdown_connections 2024-09-11T22:04:15.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.240+0000 7f6dcc2f4640 1 --2- 172.21.15.1:0/1953675370 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6dc410c170 0x7f6dc410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:15.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.240+0000 7f6dcc2f4640 1 --2- 172.21.15.1:0/1953675370 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6dc4103660 0x7f6dc410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:15.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.240+0000 7f6dcc2f4640 1 --2- 172.21.15.1:0/1953675370 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6dc4102d40 0x7f6dc4103120 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:15.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.240+0000 7f6dcc2f4640 1 -- 172.21.15.1:0/1953675370 >> 172.21.15.1:0/1953675370 conn(0x7f6dc40fc7a0 msgr2=0x7f6dc40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:15.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.240+0000 7f6dcc2f4640 1 -- 172.21.15.1:0/1953675370 shutdown_connections 2024-09-11T22:04:15.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.240+0000 7f6dcc2f4640 1 -- 172.21.15.1:0/1953675370 wait complete. 2024-09-11T22:04:15.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.241+0000 7f6dcc2f4640 1 Processor -- start 2024-09-11T22:04:15.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.241+0000 7f6dcc2f4640 1 -- start start 2024-09-11T22:04:15.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.241+0000 7f6dcc2f4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6dc4102d40 0x7f6dc4198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:15.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.241+0000 7f6dcc2f4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6dc4103660 0x7f6dc4198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:15.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.241+0000 7f6dcc2f4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6dc410c170 0x7f6dc419cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:15.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.241+0000 7f6dcc2f4640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc4111f70 con 0x7f6dc4102d40 2024-09-11T22:04:15.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.241+0000 7f6dcc2f4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc4111df0 con 0x7f6dc410c170 2024-09-11T22:04:15.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.241+0000 7f6dcc2f4640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc41120f0 con 0x7f6dc4103660 2024-09-11T22:04:15.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.241+0000 7f6dca090640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6dc4102d40 0x7f6dc4198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:15.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.241+0000 7f6dca090640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6dc4102d40 0x7f6dc4198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55894/0 (socket says 172.21.15.1:55894) 2024-09-11T22:04:15.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.241+0000 7f6dca090640 1 -- 172.21.15.1:0/1841032433 learned_addr learned my addr 172.21.15.1:0/1841032433 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:15.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.241+0000 7f6dc988f640 1 --2- 172.21.15.1:0/1841032433 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6dc4103660 0x7f6dc4198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:15.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.241+0000 7f6dca891640 1 --2- 172.21.15.1:0/1841032433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6dc410c170 0x7f6dc419cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:15.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.241+0000 7f6dca090640 1 -- 172.21.15.1:0/1841032433 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6dc4103660 msgr2=0x7f6dc4198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:15.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.241+0000 7f6dca090640 1 --2- 172.21.15.1:0/1841032433 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6dc4103660 0x7f6dc4198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:15.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.241+0000 7f6dca090640 1 -- 172.21.15.1:0/1841032433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6dc410c170 msgr2=0x7f6dc419cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:15.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.241+0000 7f6dca090640 1 --2- 172.21.15.1:0/1841032433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6dc410c170 0x7f6dc419cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:15.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.241+0000 7f6dca090640 1 -- 172.21.15.1:0/1841032433 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dc419d3e0 con 0x7f6dc4102d40 2024-09-11T22:04:15.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.242+0000 7f6dca891640 1 --2- 172.21.15.1:0/1841032433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6dc410c170 0x7f6dc419cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:04:15.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.242+0000 7f6dc988f640 1 --2- 172.21.15.1:0/1841032433 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6dc4103660 0x7f6dc4198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:04:15.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.242+0000 7f6dca090640 1 --2- 172.21.15.1:0/1841032433 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6dc4102d40 0x7f6dc4198430 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0006010 tx=0x7f6dc0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:15.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.242+0000 7f6dbb7fe640 1 -- 172.21.15.1:0/1841032433 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc003f030 con 0x7f6dc4102d40 2024-09-11T22:04:15.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.242+0000 7f6dbb7fe640 1 -- 172.21.15.1:0/1841032433 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6dc0004440 con 0x7f6dc4102d40 2024-09-11T22:04:15.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.242+0000 7f6dcc2f4640 1 -- 172.21.15.1:0/1841032433 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dc419d670 con 0x7f6dc4102d40 2024-09-11T22:04:15.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.242+0000 7f6dbb7fe640 1 -- 172.21.15.1:0/1841032433 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0035070 con 0x7f6dc4102d40 2024-09-11T22:04:15.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.242+0000 7f6dcc2f4640 1 -- 172.21.15.1:0/1841032433 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dc41a4fb0 con 0x7f6dc4102d40 2024-09-11T22:04:15.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.245+0000 7f6dbb7fe640 1 -- 172.21.15.1:0/1841032433 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f6dc0042030 con 0x7f6dc4102d40 2024-09-11T22:04:15.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.245+0000 7f6dce6cd740 1 Processor -- start 2024-09-11T22:04:15.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.245+0000 7f6dce6cd740 1 -- start start 2024-09-11T22:04:15.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.246+0000 7f6dce6cd740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556733b7cc50 0x556733b7d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:15.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.246+0000 7f6dce6cd740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556733b7b4a0 0x556733b7b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:15.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.246+0000 7f6dce6cd740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556733b7bf20 0x556733b8cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:15.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.246+0000 7f6dce6cd740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x556733add2f0 con 0x556733b7bf20 2024-09-11T22:04:15.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.246+0000 7f6dce6cd740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x556733a84cf0 con 0x556733b7cc50 2024-09-11T22:04:15.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.246+0000 7f6dce6cd740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556733ad92b0 con 0x556733b7b4a0 2024-09-11T22:04:15.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.246+0000 7f6dbb7fe640 1 --2- 172.21.15.1:0/1841032433 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f6db00a2cf0 0x7f6db00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:15.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.246+0000 7f6dca891640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556733b7cc50 0x556733b7d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:15.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.246+0000 7f6dca090640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556733b7b4a0 0x556733b7b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:15.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.246+0000 7f6dca891640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556733b7cc50 0x556733b7d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:50940/0 (socket says 172.21.15.1:50940) 2024-09-11T22:04:15.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.246+0000 7f6dc988f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556733b7bf20 0x556733b8cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:15.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.246+0000 7f6dca891640 1 -- 172.21.15.1:0/1096783874 learned_addr learned my addr 172.21.15.1:0/1096783874 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:15.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.247+0000 7f6dbb7fe640 1 -- 172.21.15.1:0/1841032433 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7f6dc00ef3a0 con 0x7f6dc4102d40 2024-09-11T22:04:15.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.247+0000 7f6dca891640 1 --2- 172.21.15.1:0/1841032433 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f6db00a2cf0 0x7f6db00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:15.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.247+0000 7f6dca891640 1 -- 172.21.15.1:0/1096783874 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556733b7b4a0 msgr2=0x556733b7b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:15.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.247+0000 7f6dca891640 1 --2- 172.21.15.1:0/1096783874 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556733b7b4a0 0x556733b7b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:15.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.247+0000 7f6dca891640 1 -- 172.21.15.1:0/1096783874 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556733b7bf20 msgr2=0x556733b8cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:15.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.247+0000 7f6dca891640 1 --2- 172.21.15.1:0/1096783874 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556733b7bf20 0x556733b8cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:15.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.247+0000 7f6dca891640 1 -- 172.21.15.1:0/1096783874 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556733a693a0 con 0x556733b7cc50 2024-09-11T22:04:15.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.247+0000 7f6dca090640 1 --2- 172.21.15.1:0/1096783874 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556733b7b4a0 0x556733b7b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:04:15.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.247+0000 7f6dca891640 1 --2- 172.21.15.1:0/1096783874 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556733b7cc50 0x556733b7d030 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc009fd0 tx=0x7f6dbc00f7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28701cdc8f2dda04 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:15.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.247+0000 7f6db8dea640 1 -- 172.21.15.1:0/1096783874 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc01ee60 con 0x556733b7cc50 2024-09-11T22:04:15.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.247+0000 7f6db8dea640 1 -- 172.21.15.1:0/1096783874 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6dbc013860 con 0x556733b7cc50 2024-09-11T22:04:15.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.247+0000 7f6db8dea640 1 -- 172.21.15.1:0/1096783874 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc00e990 con 0x556733b7cc50 2024-09-11T22:04:15.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.248+0000 7f6dce6cd740 1 -- 172.21.15.1:0/1096783874 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556733b7cc50 msgr2=0x556733b7d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:15.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.248+0000 7f6dca891640 1 --2- 172.21.15.1:0/1841032433 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f6db00a2cf0 0x7f6db00a51b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc024000 tx=0x7f6dbc022000 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:15.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.248+0000 7f6dce6cd740 1 --2- 172.21.15.1:0/1096783874 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556733b7cc50 0x556733b7d030 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc009fd0 tx=0x7f6dbc00f7d0 comp rx=0 tx=0).stop 2024-09-11T22:04:15.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.248+0000 7f6dce6cd740 1 -- 172.21.15.1:0/1096783874 shutdown_connections 2024-09-11T22:04:15.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.248+0000 7f6dce6cd740 1 --2- 172.21.15.1:0/1096783874 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556733b7bf20 0x556733b8cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:15.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.248+0000 7f6dce6cd740 1 --2- 172.21.15.1:0/1096783874 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556733b7b4a0 0x556733b7b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:15.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.248+0000 7f6dce6cd740 1 --2- 172.21.15.1:0/1096783874 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556733b7cc50 0x556733b7d030 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:15.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.248+0000 7f6dce6cd740 1 -- 172.21.15.1:0/1096783874 >> 172.21.15.1:0/1096783874 conn(0x556733b40310 msgr2=0x556733b77fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:15.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.248+0000 7f6dce6cd740 1 -- 172.21.15.1:0/1096783874 shutdown_connections 2024-09-11T22:04:15.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.248+0000 7f6dce6cd740 1 -- 172.21.15.1:0/1096783874 wait complete. 2024-09-11T22:04:15.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.249+0000 7f6dce6cd740 1 -- start start 2024-09-11T22:04:15.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.249+0000 7f6dce6cd740 1 Processor -- start 2024-09-11T22:04:15.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.249+0000 7f6dce6cd740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556733b7b4a0 0x556733b8ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:15.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.249+0000 7f6dce6cd740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556733b7bf20 0x556733b8a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:15.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.249+0000 7f6dce6cd740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556733b7cc50 0x556733b8a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:15.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.249+0000 7f6dce6cd740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x556733b8d8f0 con 0x556733b7cc50 2024-09-11T22:04:15.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.249+0000 7f6dce6cd740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x556733a98060 con 0x556733b7bf20 2024-09-11T22:04:15.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.249+0000 7f6dce6cd740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556733b8da70 con 0x556733b7b4a0 2024-09-11T22:04:15.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.249+0000 7f6dce6cd740 10 client.0 _ll_register_callbacks cb 0x5567337eb8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:04:15.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.250+0000 7f6dca090640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556733b7b4a0 0x556733b8ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:15.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.250+0000 7f6dca090640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556733b7b4a0 0x556733b8ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:51018/0 (socket says 172.21.15.1:51018) 2024-09-11T22:04:15.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.250+0000 7f6dca090640 1 -- 172.21.15.1:0/2106415357 learned_addr learned my addr 172.21.15.1:0/2106415357 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:15.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.250+0000 7f6dca891640 1 --2- 172.21.15.1:0/2106415357 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556733b7cc50 0x556733b8a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:15.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.250+0000 7f6dc988f640 1 --2- 172.21.15.1:0/2106415357 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556733b7bf20 0x556733b8a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:15.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.250+0000 7f6dca090640 1 -- 172.21.15.1:0/2106415357 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556733b7bf20 msgr2=0x556733b8a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:15.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.250+0000 7f6dca090640 1 --2- 172.21.15.1:0/2106415357 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556733b7bf20 0x556733b8a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:15.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.250+0000 7f6dca090640 1 -- 172.21.15.1:0/2106415357 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556733b7cc50 msgr2=0x556733b8a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:15.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.250+0000 7f6dca090640 1 --2- 172.21.15.1:0/2106415357 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556733b7cc50 0x556733b8a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:15.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.250+0000 7f6dca090640 1 -- 172.21.15.1:0/2106415357 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55673384e3c0 con 0x556733b7b4a0 2024-09-11T22:04:15.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.250+0000 7f6dc988f640 1 --2- 172.21.15.1:0/2106415357 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556733b7bf20 0x556733b8a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:04:15.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.251+0000 7f6dca090640 1 --2- 172.21.15.1:0/2106415357 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556733b7b4a0 0x556733b8ce10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6dc00ecae0 tx=0x7f6dc00bb3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ce0c6421e6f1f10 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:15.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.251+0000 7f6dadffb640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-11T22:04:15.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.251+0000 7f6dadffb640 1 -- 172.21.15.1:0/2106415357 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0049040 con 0x556733b7b4a0 2024-09-11T22:04:15.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.251+0000 7f6dce6cd740 1 -- 172.21.15.1:0/2106415357 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6dbc013860 con 0x556733b7b4a0 2024-09-11T22:04:15.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.251+0000 7f6dadffb640 1 -- 172.21.15.1:0/2106415357 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7f6dc00b8030 con 0x556733b7b4a0 2024-09-11T22:04:15.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.251+0000 7f6dadffb640 1 -- 172.21.15.1:0/2106415357 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f6dc00c4580 con 0x556733b7b4a0 2024-09-11T22:04:15.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.251+0000 7f6dce6cd740 10 client.26449 fetch_fsmap learned FSMap version 42 2024-09-11T22:04:15.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.251+0000 7f6dce6cd740 1 -- 172.21.15.1:0/2106415357 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x556733b8d940 con 0x556733b7b4a0 2024-09-11T22:04:15.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.251+0000 7f6dadffb640 1 -- 172.21.15.1:0/2106415357 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6dbc013860 con 0x556733b7b4a0 2024-09-11T22:04:15.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.251+0000 7f6dadffb640 1 -- 172.21.15.1:0/2106415357 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f6dc00a60e0 con 0x556733b7b4a0 2024-09-11T22:04:15.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.251+0000 7f6dce6cd740 10 client.26449 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:04:15.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.251+0000 7f6dce6cd740 10 client.26449 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:04:15.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.252+0000 7f6dce6cd740 20 client.26449 populate_metadata read hostname 'smithi001' 2024-09-11T22:04:15.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.252+0000 7f6dce6cd740 1 --2- 172.21.15.1:0/2106415357 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x556733b9a960 0x556733b9cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:15.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.252+0000 7f6dce6cd740 4 client.26449 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:04:15.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.252+0000 7f6dce6cd740 1 -- 172.21.15.1:0/2106415357 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556733aea860 con 0x556733b9a960 2024-09-11T22:04:15.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.252+0000 7f6dc988f640 1 --2- 172.21.15.1:0/2106415357 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x556733b9a960 0x556733b9cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:15.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.252+0000 7f6dadffb640 10 client.26449 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:04:15.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.252+0000 7f6dc988f640 1 --2- 172.21.15.1:0/2106415357 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x556733b9a960 0x556733b9cda0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd42daae0b3da355 server_cookie=a7aec4a0e725f2bb in_seq=0 out_seq=0 2024-09-11T22:04:15.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.253+0000 7f6dadffb640 1 -- 172.21.15.1:0/2106415357 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556733aea860 con 0x556733b9a960 2024-09-11T22:04:15.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.253+0000 7f6dadffb640 10 client.26449 handle_command_reply: tid=0 2024-09-11T22:04:15.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.253+0000 7f6dce6cd740 1 client.26449 shutdown 2024-09-11T22:04:15.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.253+0000 7f6dce6cd740 1 -- 172.21.15.1:0/2106415357 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556733b7b4a0 msgr2=0x556733b8ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:15.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.253+0000 7f6dce6cd740 1 --2- 172.21.15.1:0/2106415357 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556733b7b4a0 0x556733b8ce10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6dc00ecae0 tx=0x7f6dc00bb3e0 comp rx=0 tx=0).stop 2024-09-11T22:04:15.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.253+0000 7f6dce6cd740 1 -- 172.21.15.1:0/2106415357 shutdown_connections 2024-09-11T22:04:15.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.253+0000 7f6dce6cd740 1 --2- 172.21.15.1:0/2106415357 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x556733b9a960 0x556733b9cda0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:15.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.253+0000 7f6dce6cd740 1 --2- 172.21.15.1:0/2106415357 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556733b7cc50 0x556733b8a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:15.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.253+0000 7f6dce6cd740 1 --2- 172.21.15.1:0/2106415357 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556733b7bf20 0x556733b8a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:15.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.253+0000 7f6dce6cd740 1 --2- 172.21.15.1:0/2106415357 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556733b7b4a0 0x556733b8ce10 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:15.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.253+0000 7f6dce6cd740 1 -- 172.21.15.1:0/2106415357 >> 172.21.15.1:0/2106415357 conn(0x556733b40310 msgr2=0x556733b7d850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:15.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.253+0000 7f6dadffb640 0 client.26449 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:04:15.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.253+0000 7f6dce6cd740 1 -- 172.21.15.1:0/2106415357 shutdown_connections 2024-09-11T22:04:15.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.253+0000 7f6dce6cd740 1 -- 172.21.15.1:0/2106415357 wait complete. 2024-09-11T22:04:15.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.253+0000 7f6dce6cd740 20 client.26449 trim_cache size 0 max 16384 2024-09-11T22:04:15.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.267+0000 7f6dce6cd740 1 Processor -- start 2024-09-11T22:04:15.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.267+0000 7f6dce6cd740 1 -- start start 2024-09-11T22:04:15.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.268+0000 7f6dce6cd740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556733b40310 0x556733b96b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:15.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.268+0000 7f6dce6cd740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556733b7bf20 0x556733b7e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:15.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.268+0000 7f6dce6cd740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556733b7cc50 0x556733ba26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:15.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.268+0000 7f6dce6cd740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc41ab860 con 0x556733b40310 2024-09-11T22:04:15.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.268+0000 7f6dce6cd740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc41ad6a0 con 0x556733b7bf20 2024-09-11T22:04:15.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.268+0000 7f6dce6cd740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc41abff0 con 0x556733b7cc50 2024-09-11T22:04:15.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.268+0000 7f6dca090640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556733b40310 0x556733b96b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:15.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.268+0000 7f6dc988f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556733b7bf20 0x556733b7e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:15.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.268+0000 7f6dca090640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556733b40310 0x556733b96b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55912/0 (socket says 172.21.15.1:55912) 2024-09-11T22:04:15.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.268+0000 7f6dca090640 1 -- 172.21.15.1:0/2557421946 learned_addr learned my addr 172.21.15.1:0/2557421946 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:15.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.268+0000 7f6dca891640 1 --2- 172.21.15.1:0/2557421946 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556733b7cc50 0x556733ba26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:15.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.268+0000 7f6dc988f640 1 -- 172.21.15.1:0/2557421946 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556733b7cc50 msgr2=0x556733ba26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:15.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.268+0000 7f6dc988f640 1 --2- 172.21.15.1:0/2557421946 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556733b7cc50 0x556733ba26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:15.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.268+0000 7f6dc988f640 1 -- 172.21.15.1:0/2557421946 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556733b40310 msgr2=0x556733b96b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:15.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.268+0000 7f6dc988f640 1 --2- 172.21.15.1:0/2557421946 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556733b40310 0x556733b96b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:15.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.268+0000 7f6dc988f640 1 -- 172.21.15.1:0/2557421946 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556733b78fd0 con 0x556733b7bf20 2024-09-11T22:04:15.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.268+0000 7f6dca090640 1 --2- 172.21.15.1:0/2557421946 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556733b40310 0x556733b96b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:04:15.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.269+0000 7f6dc988f640 1 --2- 172.21.15.1:0/2557421946 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556733b7bf20 0x556733b7e040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x556733b9d650 tx=0x7f6db4005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=117c7d2bc20a064f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:15.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.269+0000 7f6daffff640 1 -- 172.21.15.1:0/2557421946 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db4004220 con 0x556733b7bf20 2024-09-11T22:04:15.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.269+0000 7f6daffff640 1 -- 172.21.15.1:0/2557421946 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6db400fdd0 con 0x556733b7bf20 2024-09-11T22:04:15.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.269+0000 7f6daffff640 1 -- 172.21.15.1:0/2557421946 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db4003d80 con 0x556733b7bf20 2024-09-11T22:04:15.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.270+0000 7f6dce6cd740 1 -- 172.21.15.1:0/2557421946 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556733b7bf20 msgr2=0x556733b7e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:15.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.270+0000 7f6dce6cd740 1 --2- 172.21.15.1:0/2557421946 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556733b7bf20 0x556733b7e040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x556733b9d650 tx=0x7f6db4005f90 comp rx=0 tx=0).stop 2024-09-11T22:04:15.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.270+0000 7f6dce6cd740 1 -- 172.21.15.1:0/2557421946 shutdown_connections 2024-09-11T22:04:15.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.270+0000 7f6dce6cd740 1 --2- 172.21.15.1:0/2557421946 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556733b7cc50 0x556733ba26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:15.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.270+0000 7f6dce6cd740 1 --2- 172.21.15.1:0/2557421946 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556733b7bf20 0x556733b7e040 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:15.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.270+0000 7f6dce6cd740 1 --2- 172.21.15.1:0/2557421946 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556733b40310 0x556733b96b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:15.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.270+0000 7f6dce6cd740 1 -- 172.21.15.1:0/2557421946 >> 172.21.15.1:0/2557421946 conn(0x556733b77390 msgr2=0x556733b8c140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:15.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.270+0000 7f6dce6cd740 1 -- 172.21.15.1:0/2557421946 shutdown_connections 2024-09-11T22:04:15.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.270+0000 7f6dce6cd740 1 -- 172.21.15.1:0/2557421946 wait complete. 2024-09-11T22:04:15.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.270+0000 7f6dce6cd740 1 -- start start 2024-09-11T22:04:15.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.270+0000 7f6dce6cd740 1 Processor -- start 2024-09-11T22:04:15.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.271+0000 7f6dce6cd740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556733b40310 0x556733b908f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:15.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.271+0000 7f6dce6cd740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556733b7cc50 0x556733b90e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:15.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.271+0000 7f6dce6cd740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556733b91370 0x556733c04690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:15.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.271+0000 7f6dce6cd740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc41acf10 con 0x556733b40310 2024-09-11T22:04:15.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.271+0000 7f6dce6cd740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc41ac780 con 0x556733b7cc50 2024-09-11T22:04:15.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.271+0000 7f6dce6cd740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc41a9860 con 0x556733b91370 2024-09-11T22:04:15.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.271+0000 7f6dce6cd740 10 client.0 _ll_register_callbacks cb 0x7f6dc003b460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:04:15.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.271+0000 7f6dc988f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556733b7cc50 0x556733b90e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:15.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.271+0000 7f6dca090640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556733b40310 0x556733b908f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:15.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.271+0000 7f6dc988f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556733b7cc50 0x556733b90e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:50976/0 (socket says 172.21.15.1:50976) 2024-09-11T22:04:15.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.271+0000 7f6dc988f640 1 -- 172.21.15.1:0/1007968912 learned_addr learned my addr 172.21.15.1:0/1007968912 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:15.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.271+0000 7f6dca891640 1 --2- 172.21.15.1:0/1007968912 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556733b91370 0x556733c04690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:15.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.271+0000 7f6dc988f640 1 -- 172.21.15.1:0/1007968912 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556733b91370 msgr2=0x556733c04690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:15.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.271+0000 7f6dc988f640 1 --2- 172.21.15.1:0/1007968912 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556733b91370 0x556733c04690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:15.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.271+0000 7f6dc988f640 1 -- 172.21.15.1:0/1007968912 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556733b40310 msgr2=0x556733b908f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:15.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.271+0000 7f6dc988f640 1 --2- 172.21.15.1:0/1007968912 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556733b40310 0x556733b908f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:15.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.271+0000 7f6dc988f640 1 -- 172.21.15.1:0/1007968912 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556733b93950 con 0x556733b7cc50 2024-09-11T22:04:15.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.271+0000 7f6dca090640 1 --2- 172.21.15.1:0/1007968912 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556733b40310 0x556733b908f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:04:15.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.272+0000 7f6dc988f640 1 --2- 172.21.15.1:0/1007968912 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556733b7cc50 0x556733b90e30 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6db4003920 tx=0x7f6db4009e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fcb8dad7256f5e29 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:15.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.272+0000 7f6dadffb640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-11T22:04:15.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.272+0000 7f6dadffb640 1 -- 172.21.15.1:0/1007968912 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db400faf0 con 0x556733b7cc50 2024-09-11T22:04:15.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.272+0000 7f6dadffb640 1 -- 172.21.15.1:0/1007968912 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7f6db4030420 con 0x556733b7cc50 2024-09-11T22:04:15.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.272+0000 7f6dce6cd740 1 -- 172.21.15.1:0/1007968912 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6db400fdd0 con 0x556733b7cc50 2024-09-11T22:04:15.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.272+0000 7f6dadffb640 1 -- 172.21.15.1:0/1007968912 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f6db4030620 con 0x556733b7cc50 2024-09-11T22:04:15.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.272+0000 7f6dce6cd740 10 client.26489 fetch_fsmap learned FSMap version 42 2024-09-11T22:04:15.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.272+0000 7f6dce6cd740 1 -- 172.21.15.1:0/1007968912 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x556733c05c50 con 0x556733b7cc50 2024-09-11T22:04:15.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.272+0000 7f6dadffb640 1 -- 172.21.15.1:0/1007968912 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6db402b030 con 0x556733b7cc50 2024-09-11T22:04:15.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.272+0000 7f6dadffb640 1 -- 172.21.15.1:0/1007968912 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f6db4025030 con 0x556733b7cc50 2024-09-11T22:04:15.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.272+0000 7f6dce6cd740 10 client.26489 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:04:15.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.272+0000 7f6dce6cd740 10 client.26489 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:04:15.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.272+0000 7f6dce6cd740 20 client.26489 populate_metadata read hostname 'smithi001' 2024-09-11T22:04:15.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.272+0000 7f6dce6cd740 1 --2- 172.21.15.1:0/1007968912 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x556733c063b0 0x556733c087d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:15.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.272+0000 7f6dce6cd740 4 client.26489 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:04:15.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.272+0000 7f6dce6cd740 1 -- 172.21.15.1:0/1007968912 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f6db400fdd0 con 0x556733c063b0 2024-09-11T22:04:15.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.273+0000 7f6dca090640 1 --2- 172.21.15.1:0/1007968912 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x556733c063b0 0x556733c087d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:15.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.273+0000 7f6dadffb640 10 client.26489 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:04:15.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.273+0000 7f6dca090640 1 --2- 172.21.15.1:0/1007968912 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x556733c063b0 0x556733c087d0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e98dd513170c1232 server_cookie=e63cc888a8338f42 in_seq=0 out_seq=0 2024-09-11T22:04:15.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.273+0000 7f6dadffb640 1 -- 172.21.15.1:0/1007968912 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f6db400fdd0 con 0x556733c063b0 2024-09-11T22:04:15.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.273+0000 7f6dadffb640 10 client.26489 handle_command_reply: tid=0 2024-09-11T22:04:15.297 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-11T22:04:15.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.273+0000 7f6dce6cd740 1 client.26489 shutdown 2024-09-11T22:04:15.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.273+0000 7f6dce6cd740 1 -- 172.21.15.1:0/1007968912 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556733b7cc50 msgr2=0x556733b90e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:15.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.273+0000 7f6dce6cd740 1 --2- 172.21.15.1:0/1007968912 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556733b7cc50 0x556733b90e30 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6db4003920 tx=0x7f6db4009e70 comp rx=0 tx=0).stop 2024-09-11T22:04:15.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.273+0000 7f6dce6cd740 1 -- 172.21.15.1:0/1007968912 shutdown_connections 2024-09-11T22:04:15.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.273+0000 7f6dce6cd740 1 --2- 172.21.15.1:0/1007968912 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x556733c063b0 0x556733c087d0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:15.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.274+0000 7f6dce6cd740 1 --2- 172.21.15.1:0/1007968912 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556733b91370 0x556733c04690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:15.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.274+0000 7f6dadffb640 0 client.26489 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:04:15.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.274+0000 7f6dce6cd740 1 --2- 172.21.15.1:0/1007968912 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556733b7cc50 0x556733b90e30 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:15.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.274+0000 7f6dce6cd740 1 --2- 172.21.15.1:0/1007968912 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556733b40310 0x556733b908f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:15.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.274+0000 7f6dce6cd740 1 -- 172.21.15.1:0/1007968912 >> 172.21.15.1:0/1007968912 conn(0x556733b77390 msgr2=0x556733b89980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:15.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.274+0000 7f6dce6cd740 1 -- 172.21.15.1:0/1007968912 shutdown_connections 2024-09-11T22:04:15.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.274+0000 7f6dce6cd740 1 -- 172.21.15.1:0/1007968912 wait complete. 2024-09-11T22:04:15.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.274+0000 7f6dce6cd740 20 client.26489 trim_cache size 0 max 16384 2024-09-11T22:04:15.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.274+0000 7f6dcc2f4640 1 -- 172.21.15.1:0/1841032433 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f6db00a2cf0 msgr2=0x7f6db00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:15.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.274+0000 7f6dcc2f4640 1 --2- 172.21.15.1:0/1841032433 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f6db00a2cf0 0x7f6db00a51b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc024000 tx=0x7f6dbc022000 comp rx=0 tx=0).stop 2024-09-11T22:04:15.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.274+0000 7f6dcc2f4640 1 -- 172.21.15.1:0/1841032433 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6dc4102d40 msgr2=0x7f6dc4198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:15.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.274+0000 7f6dcc2f4640 1 --2- 172.21.15.1:0/1841032433 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6dc4102d40 0x7f6dc4198430 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0006010 tx=0x7f6dc0004040 comp rx=0 tx=0).stop 2024-09-11T22:04:15.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.275+0000 7f6dcc2f4640 1 -- 172.21.15.1:0/1841032433 shutdown_connections 2024-09-11T22:04:15.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.275+0000 7f6dcc2f4640 1 --2- 172.21.15.1:0/1841032433 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f6db00a2cf0 0x7f6db00a51b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:15.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.275+0000 7f6dcc2f4640 1 --2- 172.21.15.1:0/1841032433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6dc410c170 0x7f6dc419cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:15.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.275+0000 7f6dcc2f4640 1 --2- 172.21.15.1:0/1841032433 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6dc4103660 0x7f6dc4198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:15.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.275+0000 7f6dcc2f4640 1 --2- 172.21.15.1:0/1841032433 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6dc4102d40 0x7f6dc4198430 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:15.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.275+0000 7f6dcc2f4640 1 -- 172.21.15.1:0/1841032433 >> 172.21.15.1:0/1841032433 conn(0x7f6dc40fc7a0 msgr2=0x7f6dc40fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:15.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.275+0000 7f6dcc2f4640 1 -- 172.21.15.1:0/1841032433 shutdown_connections 2024-09-11T22:04:15.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:15.275+0000 7f6dcc2f4640 1 -- 172.21.15.1:0/1841032433 wait complete. 2024-09-11T22:04:16.015 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.1 2024-09-11T22:04:16.015 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:04:16.328 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:04:17.011 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T22:04:17.012 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T22:04:17.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:04:16 smithi053 ceph-mon[34908]: pgmap v218: 97 pgs: 97 active+clean; 415 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.0 MiB/s wr, 479 op/s 2024-09-11T22:04:17.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:04:16 smithi161 ceph-mon[35615]: pgmap v218: 97 pgs: 97 active+clean; 415 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.0 MiB/s wr, 479 op/s 2024-09-11T22:04:17.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:04:16 smithi001 ceph-mon[30935]: pgmap v218: 97 pgs: 97 active+clean; 415 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.0 MiB/s wr, 479 op/s 2024-09-11T22:04:19.197 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:04:18 smithi001 ceph-mon[30935]: pgmap v219: 97 pgs: 97 active+clean; 415 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.0 MiB/s wr, 476 op/s 2024-09-11T22:04:19.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:04:18 smithi053 ceph-mon[34908]: pgmap v219: 97 pgs: 97 active+clean; 415 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.0 MiB/s wr, 476 op/s 2024-09-11T22:04:19.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:04:18 smithi161 ceph-mon[35615]: pgmap v219: 97 pgs: 97 active+clean; 415 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.0 MiB/s wr, 476 op/s 2024-09-11T22:04:19.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:19.841+0000 7f65db9fc640 1 -- 172.21.15.1:0/2412016045 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65d4102ef0 msgr2=0x7f65d410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:19.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:19.841+0000 7f65db9fc640 1 --2- 172.21.15.1:0/2412016045 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65d4102ef0 0x7f65d410da40 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f65cc009ca0 tx=0x7f65cc02a950 comp rx=0 tx=0).stop 2024-09-11T22:04:19.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:19.841+0000 7f65db9fc640 1 -- 172.21.15.1:0/2412016045 shutdown_connections 2024-09-11T22:04:19.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:19.841+0000 7f65db9fc640 1 --2- 172.21.15.1:0/2412016045 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65d4102ef0 0x7f65d410da40 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:19.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:19.841+0000 7f65db9fc640 1 --2- 172.21.15.1:0/2412016045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65d4102570 0x7f65d41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:19.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:19.841+0000 7f65db9fc640 1 --2- 172.21.15.1:0/2412016045 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65d4100e20 0x7f65d4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:19.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:19.841+0000 7f65db9fc640 1 -- 172.21.15.1:0/2412016045 >> 172.21.15.1:0/2412016045 conn(0x7f65d40fc7d0 msgr2=0x7f65d40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:19.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:19.841+0000 7f65db9fc640 1 -- 172.21.15.1:0/2412016045 shutdown_connections 2024-09-11T22:04:19.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:19.841+0000 7f65db9fc640 1 -- 172.21.15.1:0/2412016045 wait complete. 2024-09-11T22:04:19.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:19.842+0000 7f65db9fc640 1 Processor -- start 2024-09-11T22:04:19.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:19.842+0000 7f65db9fc640 1 -- start start 2024-09-11T22:04:19.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:19.842+0000 7f65db9fc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65d4100e20 0x7f65d4198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:19.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:19.842+0000 7f65db9fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65d4102570 0x7f65d41989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:19.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:19.842+0000 7f65db9fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65d4102ef0 0x7f65d419cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:19.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:19.842+0000 7f65db9fc640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f65d4110f80 con 0x7f65d4102ef0 2024-09-11T22:04:19.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:19.842+0000 7f65db9fc640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f65d4110e00 con 0x7f65d4102570 2024-09-11T22:04:19.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:19.842+0000 7f65db9fc640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f65d4111100 con 0x7f65d4100e20 2024-09-11T22:04:19.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:19.842+0000 7f65d9f99640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65d4102ef0 0x7f65d419cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:19.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:19.842+0000 7f65d8f97640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65d4102570 0x7f65d41989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:19.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:19.842+0000 7f65d9798640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65d4100e20 0x7f65d4198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:19.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:19.842+0000 7f65d9f99640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65d4102ef0 0x7f65d419cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53044/0 (socket says 172.21.15.1:53044) 2024-09-11T22:04:19.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:19.842+0000 7f65d8f97640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65d4102570 0x7f65d41989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:54468/0 (socket says 172.21.15.1:54468) 2024-09-11T22:04:19.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:19.842+0000 7f65d9f99640 1 -- 172.21.15.1:0/3243035787 learned_addr learned my addr 172.21.15.1:0/3243035787 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:19.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:19.842+0000 7f65d9f99640 1 -- 172.21.15.1:0/3243035787 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65d4100e20 msgr2=0x7f65d4198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:19.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:19.842+0000 7f65d9f99640 1 --2- 172.21.15.1:0/3243035787 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65d4100e20 0x7f65d4198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:19.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:19.842+0000 7f65d9f99640 1 -- 172.21.15.1:0/3243035787 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65d4102570 msgr2=0x7f65d41989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:19.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:19.842+0000 7f65d9f99640 1 --2- 172.21.15.1:0/3243035787 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65d4102570 0x7f65d41989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:19.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:19.842+0000 7f65d9f99640 1 -- 172.21.15.1:0/3243035787 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65d419d420 con 0x7f65d4102ef0 2024-09-11T22:04:19.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:19.843+0000 7f65d9f99640 1 --2- 172.21.15.1:0/3243035787 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65d4102ef0 0x7f65d419cd40 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f65cc004af0 tx=0x7f65cc02f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:19.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:19.843+0000 7f65ca7fc640 1 -- 172.21.15.1:0/3243035787 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65cc034040 con 0x7f65d4102ef0 2024-09-11T22:04:19.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:19.843+0000 7f65db9fc640 1 -- 172.21.15.1:0/3243035787 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65d419d6b0 con 0x7f65d4102ef0 2024-09-11T22:04:19.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:19.843+0000 7f65ca7fc640 1 -- 172.21.15.1:0/3243035787 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65cc030920 con 0x7f65d4102ef0 2024-09-11T22:04:19.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:19.843+0000 7f65ca7fc640 1 -- 172.21.15.1:0/3243035787 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65cc004630 con 0x7f65d4102ef0 2024-09-11T22:04:19.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:19.843+0000 7f65db9fc640 1 -- 172.21.15.1:0/3243035787 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65d41a4ff0 con 0x7f65d4102ef0 2024-09-11T22:04:19.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:19.844+0000 7f65db9fc640 1 -- 172.21.15.1:0/3243035787 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65d4101200 con 0x7f65d4102ef0 2024-09-11T22:04:19.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:19.845+0000 7f65ca7fc640 1 -- 172.21.15.1:0/3243035787 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f65cc037020 con 0x7f65d4102ef0 2024-09-11T22:04:19.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:19.846+0000 7f65ca7fc640 1 --2- 172.21.15.1:0/3243035787 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f65c00a2cf0 0x7f65c00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:19.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:19.846+0000 7f65d9798640 1 --2- 172.21.15.1:0/3243035787 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f65c00a2cf0 0x7f65c00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:19.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:19.846+0000 7f65ca7fc640 1 -- 172.21.15.1:0/3243035787 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7f65cc0f00f0 con 0x7f65d4102ef0 2024-09-11T22:04:19.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:19.847+0000 7f65d9798640 1 --2- 172.21.15.1:0/3243035787 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f65c00a2cf0 0x7f65c00a51b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f65d0009bd0 tx=0x7f65d0005f90 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:19.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:19.851+0000 7f65ca7fc640 1 -- 172.21.15.1:0/3243035787 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f65cc0bcdb0 con 0x7f65d4102ef0 2024-09-11T22:04:19.917 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:04:20.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:20.077+0000 7f65db9fc640 1 -- 172.21.15.1:0/3243035787 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f65d410a0a0 con 0x7f65d4102ef0 2024-09-11T22:04:20.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:20.078+0000 7f65ca7fc640 1 -- 172.21.15.1:0/3243035787 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11984 (secure 0 0 0) 0x7f65cc0bc6b0 con 0x7f65d4102ef0 2024-09-11T22:04:20.079 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T22:04:20.079 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":42,"btime":"2024-09-11T22:03:27:040852+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/421542832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":421542832},{"type":"v1","addr":"172.21.15.1:6835","nonce":421542832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2836864406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2836864406},{"type":"v1","addr":"172.21.15.1:6837","nonce":2836864406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":42,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T21:55:20.791263+0000","modified":"2024-09-11T22:03:26.485134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24484,"mds_2":24467,"mds_3":24466,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14595":{"gid":14595,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/407834672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":407834672},{"type":"v1","addr":"172.21.15.1:6839","nonce":407834672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/1068827678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":1068827678},{"type":"v1","addr":"172.21.15.1:6841","nonce":1068827678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/872410024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":872410024},{"type":"v1","addr":"172.21.15.161:6833","nonce":872410024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/1648944570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1648944570},{"type":"v1","addr":"172.21.15.53:6833","nonce":1648944570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/2126933993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2126933993},{"type":"v1","addr":"172.21.15.53:6835","nonce":2126933993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.161:6835/828381592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":828381592},{"type":"v1","addr":"172.21.15.161:6835","nonce":828381592}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.161:6837/1799318656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1799318656},{"type":"v1","addr":"172.21.15.161:6837","nonce":1799318656}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.53:6837/3457131830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3457131830},{"type":"v1","addr":"172.21.15.53:6837","nonce":3457131830}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/455365383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":455365383},{"type":"v1","addr":"172.21.15.161:6839","nonce":455365383}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.53:6839/1378584447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1378584447},{"type":"v1","addr":"172.21.15.53:6839","nonce":1378584447}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24460,24466,24467,24484,24494]},"id":1}]} 2024-09-11T22:04:20.079 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 42 2024-09-11T22:04:20.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:20.080+0000 7f65db9fc640 1 -- 172.21.15.1:0/3243035787 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f65c00a2cf0 msgr2=0x7f65c00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:20.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:20.080+0000 7f65db9fc640 1 --2- 172.21.15.1:0/3243035787 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f65c00a2cf0 0x7f65c00a51b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f65d0009bd0 tx=0x7f65d0005f90 comp rx=0 tx=0).stop 2024-09-11T22:04:20.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:20.080+0000 7f65db9fc640 1 -- 172.21.15.1:0/3243035787 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65d4102ef0 msgr2=0x7f65d419cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:20.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:20.080+0000 7f65db9fc640 1 --2- 172.21.15.1:0/3243035787 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65d4102ef0 0x7f65d419cd40 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f65cc004af0 tx=0x7f65cc02f3a0 comp rx=0 tx=0).stop 2024-09-11T22:04:20.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:20.080+0000 7f65db9fc640 1 -- 172.21.15.1:0/3243035787 shutdown_connections 2024-09-11T22:04:20.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:20.080+0000 7f65db9fc640 1 --2- 172.21.15.1:0/3243035787 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f65c00a2cf0 0x7f65c00a51b0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:20.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:20.080+0000 7f65db9fc640 1 --2- 172.21.15.1:0/3243035787 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f65d4102ef0 0x7f65d419cd40 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:20.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:20.080+0000 7f65db9fc640 1 --2- 172.21.15.1:0/3243035787 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f65d4102570 0x7f65d41989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:20.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:20.080+0000 7f65db9fc640 1 --2- 172.21.15.1:0/3243035787 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f65d4100e20 0x7f65d4198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:20.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:20.080+0000 7f65db9fc640 1 -- 172.21.15.1:0/3243035787 >> 172.21.15.1:0/3243035787 conn(0x7f65d40fc7d0 msgr2=0x7f65d406a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:20.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:20.080+0000 7f65db9fc640 1 -- 172.21.15.1:0/3243035787 shutdown_connections 2024-09-11T22:04:20.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:20.080+0000 7f65db9fc640 1 -- 172.21.15.1:0/3243035787 wait complete. 2024-09-11T22:04:21.070 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db / --timeout 90 --expiration 140.1 --await-for 45.0' 2024-09-11T22:04:21.071 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db / --timeout 90 --expiration 140.1 --await-for 45.0 2024-09-11T22:04:21.251 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:04:20 smithi001 ceph-mon[30935]: pgmap v220: 97 pgs: 97 active+clean; 433 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.6 MiB/s wr, 414 op/s 2024-09-11T22:04:21.251 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:04:20 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/3243035787' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:04:21.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.248+0000 7f79eadc7640 1 -- 172.21.15.1:0/1967245385 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f79e41027a0 msgr2=0x7f79e41067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:21.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.248+0000 7f79eadc7640 1 --2- 172.21.15.1:0/1967245385 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f79e41027a0 0x7f79e41067f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f79d0009860 tx=0x7f79d002a340 comp rx=0 tx=0).stop 2024-09-11T22:04:21.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.248+0000 7f79eadc7640 1 -- 172.21.15.1:0/1967245385 shutdown_connections 2024-09-11T22:04:21.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.248+0000 7f79eadc7640 1 --2- 172.21.15.1:0/1967245385 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f79e4106f20 0x7f79e410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:21.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.248+0000 7f79eadc7640 1 --2- 172.21.15.1:0/1967245385 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f79e41027a0 0x7f79e41067f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:21.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.248+0000 7f79eadc7640 1 --2- 172.21.15.1:0/1967245385 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f79e4101df0 0x7f79e41021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:21.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.248+0000 7f79eadc7640 1 -- 172.21.15.1:0/1967245385 >> 172.21.15.1:0/1967245385 conn(0x7f79e40fd8b0 msgr2=0x7f79e40ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:21.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.248+0000 7f79eadc7640 1 -- 172.21.15.1:0/1967245385 shutdown_connections 2024-09-11T22:04:21.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.248+0000 7f79eadc7640 1 -- 172.21.15.1:0/1967245385 wait complete. 2024-09-11T22:04:21.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.250+0000 7f79eadc7640 1 Processor -- start 2024-09-11T22:04:21.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.250+0000 7f79eadc7640 1 -- start start 2024-09-11T22:04:21.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.250+0000 7f79eadc7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f79e4101df0 0x7f79e40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:21.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.250+0000 7f79eadc7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f79e41027a0 0x7f79e4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:21.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.250+0000 7f79eadc7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f79e4106f20 0x7f79e4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:21.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.250+0000 7f79eadc7640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f79e4111f80 con 0x7f79e4106f20 2024-09-11T22:04:21.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.250+0000 7f79eadc7640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f79e4111e00 con 0x7f79e41027a0 2024-09-11T22:04:21.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.250+0000 7f79eadc7640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f79e4112100 con 0x7f79e4101df0 2024-09-11T22:04:21.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.250+0000 7f79e8b63640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f79e4101df0 0x7f79e40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:21.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.250+0000 7f79e9364640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f79e4106f20 0x7f79e4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:21.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.250+0000 7f79e3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f79e41027a0 0x7f79e4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:21.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.250+0000 7f79e9364640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f79e4106f20 0x7f79e4074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53062/0 (socket says 172.21.15.1:53062) 2024-09-11T22:04:21.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.250+0000 7f79e9364640 1 -- 172.21.15.1:0/3608588437 learned_addr learned my addr 172.21.15.1:0/3608588437 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:21.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.251+0000 7f79e8b63640 1 -- 172.21.15.1:0/3608588437 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f79e41027a0 msgr2=0x7f79e4074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:21.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.251+0000 7f79e8b63640 1 --2- 172.21.15.1:0/3608588437 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f79e41027a0 0x7f79e4074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:21.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.251+0000 7f79e8b63640 1 -- 172.21.15.1:0/3608588437 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f79e4106f20 msgr2=0x7f79e4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:21.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.251+0000 7f79e8b63640 1 --2- 172.21.15.1:0/3608588437 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f79e4106f20 0x7f79e4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:21.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.251+0000 7f79e8b63640 1 -- 172.21.15.1:0/3608588437 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79e41aae30 con 0x7f79e4101df0 2024-09-11T22:04:21.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.251+0000 7f79e9364640 1 --2- 172.21.15.1:0/3608588437 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f79e4106f20 0x7f79e4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:04:21.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.252+0000 7f79e8b63640 1 --2- 172.21.15.1:0/3608588437 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f79e4101df0 0x7f79e40744f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f79d801fa00 tx=0x7f79d801fec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:21.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.252+0000 7f79e1ffb640 1 -- 172.21.15.1:0/3608588437 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d800ccc0 con 0x7f79e4101df0 2024-09-11T22:04:21.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.252+0000 7f79e1ffb640 1 -- 172.21.15.1:0/3608588437 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79d8021920 con 0x7f79e4101df0 2024-09-11T22:04:21.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.252+0000 7f79e1ffb640 1 -- 172.21.15.1:0/3608588437 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d8029920 con 0x7f79e4101df0 2024-09-11T22:04:21.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.252+0000 7f79eadc7640 1 -- 172.21.15.1:0/3608588437 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79e406bc70 con 0x7f79e4101df0 2024-09-11T22:04:21.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.252+0000 7f79eadc7640 1 -- 172.21.15.1:0/3608588437 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79e406c1d0 con 0x7f79e4101df0 2024-09-11T22:04:21.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.255+0000 7f79e1ffb640 1 -- 172.21.15.1:0/3608588437 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f79d8021ad0 con 0x7f79e4101df0 2024-09-11T22:04:21.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.255+0000 7f79ed1a0740 1 Processor -- start 2024-09-11T22:04:21.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.255+0000 7f79ed1a0740 1 -- start start 2024-09-11T22:04:21.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.255+0000 7f79ed1a0740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562954729b60 0x562954729f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:21.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.255+0000 7f79ed1a0740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56295472a480 0x562954734800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:21.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.255+0000 7f79ed1a0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562954734d40 0x562954737130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:21.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.255+0000 7f79ed1a0740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56295468f280 con 0x562954729b60 2024-09-11T22:04:21.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.255+0000 7f79ed1a0740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x562954636b10 con 0x56295472a480 2024-09-11T22:04:21.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.255+0000 7f79ed1a0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56295468b240 con 0x562954734d40 2024-09-11T22:04:21.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.256+0000 7f79e8b63640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56295472a480 0x562954734800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:21.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.256+0000 7f79e9364640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562954729b60 0x562954729f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:21.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.256+0000 7f79e9364640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562954729b60 0x562954729f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53078/0 (socket says 172.21.15.1:53078) 2024-09-11T22:04:21.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.256+0000 7f79e9364640 1 -- 172.21.15.1:0/174248273 learned_addr learned my addr 172.21.15.1:0/174248273 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:21.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.256+0000 7f79e3fff640 1 --2- 172.21.15.1:0/174248273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562954734d40 0x562954737130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:21.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.256+0000 7f79e8b63640 1 -- 172.21.15.1:0/174248273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562954734d40 msgr2=0x562954737130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:21.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.256+0000 7f79e8b63640 1 --2- 172.21.15.1:0/174248273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562954734d40 0x562954737130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:21.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.256+0000 7f79e8b63640 1 -- 172.21.15.1:0/174248273 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562954729b60 msgr2=0x562954729f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:21.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.256+0000 7f79e8b63640 1 --2- 172.21.15.1:0/174248273 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562954729b60 0x562954729f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:21.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.256+0000 7f79e8b63640 1 -- 172.21.15.1:0/174248273 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629546431f0 con 0x56295472a480 2024-09-11T22:04:21.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.256+0000 7f79e8b63640 1 --2- 172.21.15.1:0/174248273 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56295472a480 0x562954734800 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f79d80dde70 tx=0x7f79d80e8670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83d5ba26fa0c170c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:21.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.256+0000 7f79e1ffb640 1 --2- 172.21.15.1:0/3608588437 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f79d40a2d70 0x7f79d40a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:21.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.256+0000 7f79e9364640 1 --2- 172.21.15.1:0/3608588437 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f79d40a2d70 0x7f79d40a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:21.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.256+0000 7f79ceffd640 1 -- 172.21.15.1:0/174248273 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d80ef070 con 0x56295472a480 2024-09-11T22:04:21.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.257+0000 7f79e1ffb640 1 -- 172.21.15.1:0/3608588437 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7f79d80e07f0 con 0x7f79e4101df0 2024-09-11T22:04:21.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.257+0000 7f79ceffd640 1 -- 172.21.15.1:0/174248273 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79d8027d10 con 0x56295472a480 2024-09-11T22:04:21.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.257+0000 7f79ceffd640 1 -- 172.21.15.1:0/174248273 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d80ef070 con 0x56295472a480 2024-09-11T22:04:21.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.257+0000 7f79ed1a0740 1 -- 172.21.15.1:0/174248273 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56295472a480 msgr2=0x562954734800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:21.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.257+0000 7f79ed1a0740 1 --2- 172.21.15.1:0/174248273 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56295472a480 0x562954734800 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f79d80dde70 tx=0x7f79d80e8670 comp rx=0 tx=0).stop 2024-09-11T22:04:21.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.258+0000 7f79e9364640 1 --2- 172.21.15.1:0/3608588437 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f79d40a2d70 0x7f79d40a5230 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f79dc009c40 tx=0x7f79dc0095c0 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:21.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.258+0000 7f79ed1a0740 1 -- 172.21.15.1:0/174248273 shutdown_connections 2024-09-11T22:04:21.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.258+0000 7f79ed1a0740 1 --2- 172.21.15.1:0/174248273 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562954734d40 0x562954737130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:21.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.258+0000 7f79ed1a0740 1 --2- 172.21.15.1:0/174248273 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56295472a480 0x562954734800 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:21.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.258+0000 7f79ed1a0740 1 --2- 172.21.15.1:0/174248273 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562954729b60 0x562954729f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:21.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.258+0000 7f79ed1a0740 1 -- 172.21.15.1:0/174248273 >> 172.21.15.1:0/174248273 conn(0x5629547260f0 msgr2=0x5629547264d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:21.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.258+0000 7f79ed1a0740 1 -- 172.21.15.1:0/174248273 shutdown_connections 2024-09-11T22:04:21.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.258+0000 7f79ed1a0740 1 -- 172.21.15.1:0/174248273 wait complete. 2024-09-11T22:04:21.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.258+0000 7f79ed1a0740 1 -- start start 2024-09-11T22:04:21.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.258+0000 7f79ed1a0740 1 Processor -- start 2024-09-11T22:04:21.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.259+0000 7f79ed1a0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562954729b60 0x5629547347b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:21.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.259+0000 7f79ed1a0740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56295472a480 0x562954731b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:21.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.259+0000 7f79ed1a0740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562954734d40 0x562954732240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:21.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.259+0000 7f79e8b63640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562954729b60 0x5629547347b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:21.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.259+0000 7f79ed1a0740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x562954737910 con 0x56295472a480 2024-09-11T22:04:21.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.259+0000 7f79e8b63640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562954729b60 0x5629547347b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:51776/0 (socket says 172.21.15.1:51776) 2024-09-11T22:04:21.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.259+0000 7f79e8b63640 1 -- 172.21.15.1:0/2042004553 learned_addr learned my addr 172.21.15.1:0/2042004553 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:21.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.259+0000 7f79ed1a0740 1 -- 172.21.15.1:0/2042004553 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x562954649f10 con 0x562954734d40 2024-09-11T22:04:21.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.259+0000 7f79e3fff640 1 --2- 172.21.15.1:0/2042004553 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56295472a480 0x562954731b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:21.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.259+0000 7f79ed1a0740 1 -- 172.21.15.1:0/2042004553 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x562954737a90 con 0x562954729b60 2024-09-11T22:04:21.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.259+0000 7f79e9364640 1 --2- 172.21.15.1:0/2042004553 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562954734d40 0x562954732240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:21.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.259+0000 7f79ed1a0740 10 client.0 _ll_register_callbacks cb 0x562954612a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:04:21.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.260+0000 7f79e8b63640 1 -- 172.21.15.1:0/2042004553 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562954734d40 msgr2=0x562954732240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:21.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.260+0000 7f79e8b63640 1 --2- 172.21.15.1:0/2042004553 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562954734d40 0x562954732240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:21.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.260+0000 7f79e8b63640 1 -- 172.21.15.1:0/2042004553 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56295472a480 msgr2=0x562954731b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:21.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.260+0000 7f79e8b63640 1 --2- 172.21.15.1:0/2042004553 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56295472a480 0x562954731b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:21.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.260+0000 7f79e8b63640 1 -- 172.21.15.1:0/2042004553 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629544006a0 con 0x562954729b60 2024-09-11T22:04:21.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.260+0000 7f79e8b63640 1 --2- 172.21.15.1:0/2042004553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562954729b60 0x5629547347b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f79d80ddea0 tx=0x7f79d80acbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df5e724165c8010b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:21.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.260+0000 7f79c3fff640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-11T22:04:21.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.260+0000 7f79c3fff640 1 -- 172.21.15.1:0/2042004553 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d80ef070 con 0x562954729b60 2024-09-11T22:04:21.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.260+0000 7f79ed1a0740 1 -- 172.21.15.1:0/2042004553 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79d8027d10 con 0x562954729b60 2024-09-11T22:04:21.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.260+0000 7f79c3fff640 1 -- 172.21.15.1:0/2042004553 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7f79d80a9360 con 0x562954729b60 2024-09-11T22:04:21.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.260+0000 7f79c3fff640 1 -- 172.21.15.1:0/2042004553 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f79d80ac830 con 0x562954729b60 2024-09-11T22:04:21.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.260+0000 7f79ed1a0740 10 client.26473 fetch_fsmap learned FSMap version 42 2024-09-11T22:04:21.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.260+0000 7f79ed1a0740 1 -- 172.21.15.1:0/2042004553 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x562954737910 con 0x562954729b60 2024-09-11T22:04:21.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.261+0000 7f79c3fff640 1 -- 172.21.15.1:0/2042004553 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79d8027d10 con 0x562954729b60 2024-09-11T22:04:21.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.261+0000 7f79c3fff640 1 -- 172.21.15.1:0/2042004553 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f79d80de0f0 con 0x562954729b60 2024-09-11T22:04:21.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.261+0000 7f79ed1a0740 10 client.26473 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:04:21.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.261+0000 7f79ed1a0740 10 client.26473 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:04:21.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.261+0000 7f79ed1a0740 20 client.26473 populate_metadata read hostname 'smithi001' 2024-09-11T22:04:21.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.261+0000 7f79ed1a0740 1 --2- 172.21.15.1:0/2042004553 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5629547445e0 0x5629547469d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:21.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.261+0000 7f79ed1a0740 4 client.26473 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:04:21.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.261+0000 7f79ed1a0740 1 -- 172.21.15.1:0/2042004553 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56295469c7f0 con 0x5629547445e0 2024-09-11T22:04:21.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.261+0000 7f79e3fff640 1 --2- 172.21.15.1:0/2042004553 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5629547445e0 0x5629547469d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:21.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.261+0000 7f79c3fff640 10 client.26473 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:04:21.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.262+0000 7f79e3fff640 1 --2- 172.21.15.1:0/2042004553 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5629547445e0 0x5629547469d0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2692704e95cd4d0e server_cookie=cbbf6091d3c001b0 in_seq=0 out_seq=0 2024-09-11T22:04:21.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.262+0000 7f79c3fff640 1 -- 172.21.15.1:0/2042004553 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56295469c7f0 con 0x5629547445e0 2024-09-11T22:04:21.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.262+0000 7f79c3fff640 10 client.26473 handle_command_reply: tid=0 2024-09-11T22:04:21.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.262+0000 7f79ed1a0740 1 client.26473 shutdown 2024-09-11T22:04:21.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.262+0000 7f79ed1a0740 1 -- 172.21.15.1:0/2042004553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562954729b60 msgr2=0x5629547347b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:21.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.262+0000 7f79ed1a0740 1 --2- 172.21.15.1:0/2042004553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562954729b60 0x5629547347b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f79d80ddea0 tx=0x7f79d80acbf0 comp rx=0 tx=0).stop 2024-09-11T22:04:21.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.263+0000 7f79ed1a0740 1 -- 172.21.15.1:0/2042004553 shutdown_connections 2024-09-11T22:04:21.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.263+0000 7f79ed1a0740 1 --2- 172.21.15.1:0/2042004553 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5629547445e0 0x5629547469d0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:21.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.263+0000 7f79c3fff640 0 client.26473 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:04:21.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.263+0000 7f79ed1a0740 1 --2- 172.21.15.1:0/2042004553 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562954734d40 0x562954732240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:21.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.263+0000 7f79ed1a0740 1 --2- 172.21.15.1:0/2042004553 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56295472a480 0x562954731b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:21.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.263+0000 7f79ed1a0740 1 --2- 172.21.15.1:0/2042004553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562954729b60 0x5629547347b0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:21.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.263+0000 7f79ed1a0740 1 -- 172.21.15.1:0/2042004553 >> 172.21.15.1:0/2042004553 conn(0x5629547260f0 msgr2=0x56295472be20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:21.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.263+0000 7f79ed1a0740 1 -- 172.21.15.1:0/2042004553 shutdown_connections 2024-09-11T22:04:21.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.263+0000 7f79ed1a0740 1 -- 172.21.15.1:0/2042004553 wait complete. 2024-09-11T22:04:21.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.263+0000 7f79ed1a0740 20 client.26473 trim_cache size 0 max 16384 2024-09-11T22:04:21.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.282+0000 7f79ed1a0740 1 Processor -- start 2024-09-11T22:04:21.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.282+0000 7f79ed1a0740 1 -- start start 2024-09-11T22:04:21.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.282+0000 7f79ed1a0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562954729b60 0x562954740ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:21.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.282+0000 7f79ed1a0740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56295472a480 0x562954730ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:21.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.282+0000 7f79ed1a0740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562954734d40 0x56295474c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:21.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.282+0000 7f79ed1a0740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f79e41ab8b0 con 0x56295472a480 2024-09-11T22:04:21.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.282+0000 7f79ed1a0740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f79e41ad6f0 con 0x562954734d40 2024-09-11T22:04:21.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.282+0000 7f79ed1a0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f79e41ac040 con 0x562954729b60 2024-09-11T22:04:21.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.283+0000 7f79e3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56295472a480 0x562954730ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:21.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.283+0000 7f79e9364640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562954734d40 0x56295474c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:21.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.283+0000 7f79e3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56295472a480 0x562954730ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53096/0 (socket says 172.21.15.1:53096) 2024-09-11T22:04:21.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.283+0000 7f79e9364640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562954734d40 0x56295474c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:54502/0 (socket says 172.21.15.1:54502) 2024-09-11T22:04:21.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.283+0000 7f79e3fff640 1 -- 172.21.15.1:0/3515938798 learned_addr learned my addr 172.21.15.1:0/3515938798 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:21.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.283+0000 7f79e3fff640 1 -- 172.21.15.1:0/3515938798 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562954729b60 msgr2=0x562954740ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T22:04:21.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.283+0000 7f79e3fff640 1 --2- 172.21.15.1:0/3515938798 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562954729b60 0x562954740ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:21.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.283+0000 7f79e3fff640 1 -- 172.21.15.1:0/3515938798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562954734d40 msgr2=0x56295474c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:21.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.283+0000 7f79e3fff640 1 --2- 172.21.15.1:0/3515938798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562954734d40 0x56295474c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:21.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.283+0000 7f79e3fff640 1 -- 172.21.15.1:0/3515938798 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562954725f20 con 0x56295472a480 2024-09-11T22:04:21.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.283+0000 7f79e9364640 1 --2- 172.21.15.1:0/3515938798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562954734d40 0x56295474c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:04:21.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.283+0000 7f79e3fff640 1 --2- 172.21.15.1:0/3515938798 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56295472a480 0x562954730ae0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f79d0005e00 tx=0x7f79d00058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=718b22d3050bc6b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:21.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.283+0000 7f79ce7fc640 1 -- 172.21.15.1:0/3515938798 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d004d070 con 0x56295472a480 2024-09-11T22:04:21.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.284+0000 7f79ce7fc640 1 -- 172.21.15.1:0/3515938798 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79d0030df0 con 0x56295472a480 2024-09-11T22:04:21.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.284+0000 7f79ce7fc640 1 -- 172.21.15.1:0/3515938798 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d0037040 con 0x56295472a480 2024-09-11T22:04:21.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.284+0000 7f79ed1a0740 1 -- 172.21.15.1:0/3515938798 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56295472a480 msgr2=0x562954730ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:21.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.284+0000 7f79ed1a0740 1 --2- 172.21.15.1:0/3515938798 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56295472a480 0x562954730ae0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f79d0005e00 tx=0x7f79d00058d0 comp rx=0 tx=0).stop 2024-09-11T22:04:21.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.285+0000 7f79ed1a0740 1 -- 172.21.15.1:0/3515938798 shutdown_connections 2024-09-11T22:04:21.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.285+0000 7f79ed1a0740 1 --2- 172.21.15.1:0/3515938798 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562954734d40 0x56295474c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:21.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.285+0000 7f79ed1a0740 1 --2- 172.21.15.1:0/3515938798 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56295472a480 0x562954730ae0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:21.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.285+0000 7f79ed1a0740 1 --2- 172.21.15.1:0/3515938798 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562954729b60 0x562954740ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:21.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.285+0000 7f79ed1a0740 1 -- 172.21.15.1:0/3515938798 >> 172.21.15.1:0/3515938798 conn(0x5629547260f0 msgr2=0x5629547303d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:21.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.285+0000 7f79ed1a0740 1 -- 172.21.15.1:0/3515938798 shutdown_connections 2024-09-11T22:04:21.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.285+0000 7f79ed1a0740 1 -- 172.21.15.1:0/3515938798 wait complete. 2024-09-11T22:04:21.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.285+0000 7f79ed1a0740 1 -- start start 2024-09-11T22:04:21.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.285+0000 7f79ed1a0740 1 Processor -- start 2024-09-11T22:04:21.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.286+0000 7f79ed1a0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562954729b60 0x562954736070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:21.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.286+0000 7f79ed1a0740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56295472a480 0x5629547365b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:21.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.286+0000 7f79ed1a0740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562954734d40 0x562954736af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:21.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.286+0000 7f79ed1a0740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f79e41acf60 con 0x562954734d40 2024-09-11T22:04:21.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.286+0000 7f79ed1a0740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f79e41ac7d0 con 0x56295472a480 2024-09-11T22:04:21.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.286+0000 7f79ed1a0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f79e4072cd0 con 0x562954729b60 2024-09-11T22:04:21.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.286+0000 7f79ed1a0740 10 client.0 _ll_register_callbacks cb 0x7f79dc009c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:04:21.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.286+0000 7f79e3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56295472a480 0x5629547365b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:21.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.286+0000 7f79e9364640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562954734d40 0x562954736af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:21.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.286+0000 7f79e3fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56295472a480 0x5629547365b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:54514/0 (socket says 172.21.15.1:54514) 2024-09-11T22:04:21.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.286+0000 7f79e3fff640 1 -- 172.21.15.1:0/2606065855 learned_addr learned my addr 172.21.15.1:0/2606065855 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:21.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.286+0000 7f79e8b63640 1 --2- 172.21.15.1:0/2606065855 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562954729b60 0x562954736070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:21.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.286+0000 7f79e3fff640 1 -- 172.21.15.1:0/2606065855 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562954729b60 msgr2=0x562954736070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:21.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.286+0000 7f79e3fff640 1 --2- 172.21.15.1:0/2606065855 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562954729b60 0x562954736070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:21.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.286+0000 7f79e3fff640 1 -- 172.21.15.1:0/2606065855 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562954734d40 msgr2=0x562954736af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:21.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.286+0000 7f79e3fff640 1 --2- 172.21.15.1:0/2606065855 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562954734d40 0x562954736af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:21.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.286+0000 7f79e3fff640 1 -- 172.21.15.1:0/2606065855 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56295473b6d0 con 0x56295472a480 2024-09-11T22:04:21.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.286+0000 7f79e8b63640 1 --2- 172.21.15.1:0/2606065855 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562954729b60 0x562954736070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:04:21.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.286+0000 7f79e3fff640 1 --2- 172.21.15.1:0/2606065855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56295472a480 0x5629547365b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f79d002c010 tx=0x7f79d002a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb56387ec5d925a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:21.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.286+0000 7f79c3fff640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-11T22:04:21.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.287+0000 7f79c3fff640 1 -- 172.21.15.1:0/2606065855 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d004d070 con 0x56295472a480 2024-09-11T22:04:21.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.287+0000 7f79ed1a0740 1 -- 172.21.15.1:0/2606065855 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79d0030df0 con 0x56295472a480 2024-09-11T22:04:21.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.287+0000 7f79c3fff640 1 -- 172.21.15.1:0/2606065855 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7f79d0030710 con 0x56295472a480 2024-09-11T22:04:21.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.287+0000 7f79c3fff640 1 -- 172.21.15.1:0/2606065855 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f79d0037050 con 0x56295472a480 2024-09-11T22:04:21.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.287+0000 7f79ed1a0740 10 client.26510 fetch_fsmap learned FSMap version 42 2024-09-11T22:04:21.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.287+0000 7f79ed1a0740 1 -- 172.21.15.1:0/2606065855 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5629547af740 con 0x56295472a480 2024-09-11T22:04:21.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.287+0000 7f79c3fff640 1 -- 172.21.15.1:0/2606065855 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79d0030df0 con 0x56295472a480 2024-09-11T22:04:21.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.287+0000 7f79c3fff640 1 -- 172.21.15.1:0/2606065855 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f79d0032440 con 0x56295472a480 2024-09-11T22:04:21.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.287+0000 7f79ed1a0740 10 client.26510 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:04:21.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.287+0000 7f79ed1a0740 10 client.26510 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:04:21.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.287+0000 7f79ed1a0740 20 client.26510 populate_metadata read hostname 'smithi001' 2024-09-11T22:04:21.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.287+0000 7f79ed1a0740 1 --2- 172.21.15.1:0/2606065855 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5629547445e0 0x5629547b1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:21.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.287+0000 7f79ed1a0740 4 client.26510 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T22:04:21.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.287+0000 7f79ed1a0740 1 -- 172.21.15.1:0/2606065855 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562954696d00 con 0x5629547445e0 2024-09-11T22:04:21.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.288+0000 7f79e8b63640 1 --2- 172.21.15.1:0/2606065855 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5629547445e0 0x5629547b1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:21.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.288+0000 7f79e8b63640 1 --2- 172.21.15.1:0/2606065855 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5629547445e0 0x5629547b1fa0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34a37ff5e1be9867 server_cookie=aaecc5c091a7334 in_seq=0 out_seq=0 2024-09-11T22:04:21.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.288+0000 7f79c3fff640 10 client.26510 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:04:21.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.290+0000 7f79c3fff640 1 -- 172.21.15.1:0/2606065855 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562954696d00 con 0x5629547445e0 2024-09-11T22:04:21.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.290+0000 7f79c3fff640 10 client.26510 handle_command_reply: tid=0 2024-09-11T22:04:21.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.290+0000 7f79ed1a0740 1 client.26510 shutdown 2024-09-11T22:04:21.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.290+0000 7f79ed1a0740 1 -- 172.21.15.1:0/2606065855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56295472a480 msgr2=0x5629547365b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:21.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.290+0000 7f79ed1a0740 1 --2- 172.21.15.1:0/2606065855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56295472a480 0x5629547365b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f79d002c010 tx=0x7f79d002a340 comp rx=0 tx=0).stop 2024-09-11T22:04:21.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.290+0000 7f79ed1a0740 1 -- 172.21.15.1:0/2606065855 shutdown_connections 2024-09-11T22:04:21.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.290+0000 7f79ed1a0740 1 --2- 172.21.15.1:0/2606065855 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5629547445e0 0x5629547b1fa0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:21.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.291+0000 7f79ed1a0740 1 --2- 172.21.15.1:0/2606065855 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562954734d40 0x562954736af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:21.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.291+0000 7f79ed1a0740 1 --2- 172.21.15.1:0/2606065855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56295472a480 0x5629547365b0 secure :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f79d002c010 tx=0x7f79d002a340 comp rx=0 tx=0).stop 2024-09-11T22:04:21.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.291+0000 7f79c3fff640 0 client.26510 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:04:21.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.291+0000 7f79ed1a0740 1 --2- 172.21.15.1:0/2606065855 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562954729b60 0x562954736070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:21.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.291+0000 7f79ed1a0740 1 -- 172.21.15.1:0/2606065855 >> 172.21.15.1:0/2606065855 conn(0x5629547260f0 msgr2=0x562954730bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:21.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.291+0000 7f79ed1a0740 1 -- 172.21.15.1:0/2606065855 shutdown_connections 2024-09-11T22:04:21.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.291+0000 7f79ed1a0740 1 -- 172.21.15.1:0/2606065855 wait complete. 2024-09-11T22:04:21.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.292+0000 7f79ed1a0740 20 client.26510 trim_cache size 0 max 16384 2024-09-11T22:04:21.295 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-11T22:04:21.295 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-09-11T22:04:21.295 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "deec3096-57b8-4c8f-ae1a-664e2a2c29f6", 2024-09-11T22:04:21.295 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-09-11T22:04:21.295 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-11T22:04:21.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.293+0000 7f79eadc7640 1 -- 172.21.15.1:0/3608588437 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f79d40a2d70 msgr2=0x7f79d40a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:21.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.293+0000 7f79eadc7640 1 --2- 172.21.15.1:0/3608588437 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f79d40a2d70 0x7f79d40a5230 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f79dc009c40 tx=0x7f79dc0095c0 comp rx=0 tx=0).stop 2024-09-11T22:04:21.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.293+0000 7f79eadc7640 1 -- 172.21.15.1:0/3608588437 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f79e4101df0 msgr2=0x7f79e40744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:21.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.293+0000 7f79eadc7640 1 --2- 172.21.15.1:0/3608588437 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f79e4101df0 0x7f79e40744f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f79d801fa00 tx=0x7f79d801fec0 comp rx=0 tx=0).stop 2024-09-11T22:04:21.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.293+0000 7f79eadc7640 1 -- 172.21.15.1:0/3608588437 shutdown_connections 2024-09-11T22:04:21.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.293+0000 7f79eadc7640 1 --2- 172.21.15.1:0/3608588437 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f79d40a2d70 0x7f79d40a5230 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:21.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.293+0000 7f79eadc7640 1 --2- 172.21.15.1:0/3608588437 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f79e4106f20 0x7f79e4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:21.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.293+0000 7f79eadc7640 1 --2- 172.21.15.1:0/3608588437 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f79e41027a0 0x7f79e4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:21.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.293+0000 7f79eadc7640 1 --2- 172.21.15.1:0/3608588437 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f79e4101df0 0x7f79e40744f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:21.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.293+0000 7f79eadc7640 1 -- 172.21.15.1:0/3608588437 >> 172.21.15.1:0/3608588437 conn(0x7f79e40fd8b0 msgr2=0x7f79e40feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:21.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.294+0000 7f79eadc7640 1 -- 172.21.15.1:0/3608588437 shutdown_connections 2024-09-11T22:04:21.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:21.294+0000 7f79eadc7640 1 -- 172.21.15.1:0/3608588437 wait complete. 2024-09-11T22:04:21.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:04:20 smithi053 ceph-mon[34908]: pgmap v220: 97 pgs: 97 active+clean; 433 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.6 MiB/s wr, 414 op/s 2024-09-11T22:04:21.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:04:20 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/3243035787' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:04:21.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:04:20 smithi161 ceph-mon[35615]: pgmap v220: 97 pgs: 97 active+clean; 433 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.6 MiB/s wr, 414 op/s 2024-09-11T22:04:21.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:04:20 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/3243035787' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:04:22.183 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:04:22.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:04:21 smithi053 ceph-mon[34908]: scrub queued for path: / 2024-09-11T22:04:22.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:04:21 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:04:22.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:04:21 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:04:22.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:04:21 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:04:22.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:04:21 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:04:22.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:04:21 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:04:22.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:04:21 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:04:22.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:04:21 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:04:22.315 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:04:22.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:04:21 smithi161 ceph-mon[35615]: scrub queued for path: / 2024-09-11T22:04:22.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:04:21 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:04:22.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:04:21 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:04:22.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:04:21 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:04:22.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:04:21 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:04:22.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:04:21 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:04:22.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:04:21 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:04:22.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:04:21 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:04:22.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:04:21 smithi001 ceph-mon[30935]: scrub queued for path: / 2024-09-11T22:04:22.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:04:21 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:04:22.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:04:21 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:04:22.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:04:21 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:04:22.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:04:21 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:04:22.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:04:21 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:04:22.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:04:21 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:04:22.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:04:21 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:04:22.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.870+0000 7f3549456640 1 -- 172.21.15.1:0/2068668022 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3544106a60 msgr2=0x7f3544106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:22.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.870+0000 7f3549456640 1 --2- 172.21.15.1:0/2068668022 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3544106a60 0x7f3544106e40 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f3534009930 tx=0x7f353402a420 comp rx=0 tx=0).stop 2024-09-11T22:04:22.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.870+0000 7f3549456640 1 -- 172.21.15.1:0/2068668022 shutdown_connections 2024-09-11T22:04:22.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.870+0000 7f3549456640 1 --2- 172.21.15.1:0/2068668022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3544101400 0x7f354410d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:22.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.870+0000 7f3549456640 1 --2- 172.21.15.1:0/2068668022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3544100a60 0x7f3544100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:22.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.870+0000 7f3549456640 1 --2- 172.21.15.1:0/2068668022 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3544106a60 0x7f3544106e40 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:22.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.870+0000 7f3549456640 1 -- 172.21.15.1:0/2068668022 >> 172.21.15.1:0/2068668022 conn(0x7f35440fc780 msgr2=0x7f35440feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:22.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.870+0000 7f3549456640 1 -- 172.21.15.1:0/2068668022 shutdown_connections 2024-09-11T22:04:22.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.870+0000 7f3549456640 1 -- 172.21.15.1:0/2068668022 wait complete. 2024-09-11T22:04:22.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.870+0000 7f3549456640 1 Processor -- start 2024-09-11T22:04:22.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.871+0000 7f3549456640 1 -- start start 2024-09-11T22:04:22.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.871+0000 7f3549456640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3544100a60 0x7f354419a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:22.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.871+0000 7f3549456640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3544101400 0x7f354419ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:22.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.871+0000 7f3549456640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3544106a60 0x7f354419ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:22.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.871+0000 7f3549456640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3544110e20 con 0x7f3544106a60 2024-09-11T22:04:22.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.871+0000 7f3549456640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3544110ca0 con 0x7f3544100a60 2024-09-11T22:04:22.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.871+0000 7f3549456640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3544110fa0 con 0x7f3544101400 2024-09-11T22:04:22.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.871+0000 7f353bfff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3544101400 0x7f354419ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:22.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.871+0000 7f35437fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3544106a60 0x7f354419ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:22.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.871+0000 7f353bfff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3544101400 0x7f354419ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:51804/0 (socket says 172.21.15.1:51804) 2024-09-11T22:04:22.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.871+0000 7f353bfff640 1 -- 172.21.15.1:0/4270560790 learned_addr learned my addr 172.21.15.1:0/4270560790 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:22.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.871+0000 7f35437fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3544106a60 0x7f354419ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53126/0 (socket says 172.21.15.1:53126) 2024-09-11T22:04:22.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.871+0000 7f3542ffd640 1 --2- 172.21.15.1:0/4270560790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3544100a60 0x7f354419a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:22.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.871+0000 7f353bfff640 1 -- 172.21.15.1:0/4270560790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3544100a60 msgr2=0x7f354419a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:22.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.871+0000 7f353bfff640 1 --2- 172.21.15.1:0/4270560790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3544100a60 0x7f354419a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:22.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.871+0000 7f353bfff640 1 -- 172.21.15.1:0/4270560790 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3544106a60 msgr2=0x7f354419ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:22.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.871+0000 7f353bfff640 1 --2- 172.21.15.1:0/4270560790 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3544106a60 0x7f354419ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:22.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.871+0000 7f353bfff640 1 -- 172.21.15.1:0/4270560790 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f354419f5f0 con 0x7f3544101400 2024-09-11T22:04:22.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.872+0000 7f35437fe640 1 --2- 172.21.15.1:0/4270560790 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3544106a60 0x7f354419ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:04:22.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.872+0000 7f353bfff640 1 --2- 172.21.15.1:0/4270560790 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3544101400 0x7f354419ab80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f352c00e9f0 tx=0x7f352c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:22.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.872+0000 7f3540ff9640 1 -- 172.21.15.1:0/4270560790 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f352c00cd80 con 0x7f3544101400 2024-09-11T22:04:22.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.872+0000 7f3549456640 1 -- 172.21.15.1:0/4270560790 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f354419f8e0 con 0x7f3544101400 2024-09-11T22:04:22.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.872+0000 7f3540ff9640 1 -- 172.21.15.1:0/4270560790 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f352c010910 con 0x7f3544101400 2024-09-11T22:04:22.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.872+0000 7f3540ff9640 1 -- 172.21.15.1:0/4270560790 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f352c0188f0 con 0x7f3544101400 2024-09-11T22:04:22.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.872+0000 7f3549456640 1 -- 172.21.15.1:0/4270560790 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35441a71c0 con 0x7f3544101400 2024-09-11T22:04:22.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.875+0000 7f354b82f740 1 Processor -- start 2024-09-11T22:04:22.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.875+0000 7f354b82f740 1 -- start start 2024-09-11T22:04:22.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.875+0000 7f3540ff9640 1 -- 172.21.15.1:0/4270560790 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f352c010ac0 con 0x7f3544101400 2024-09-11T22:04:22.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.875+0000 7f354b82f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585f5d71760 0x5585f5d71b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:22.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.875+0000 7f354b82f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585f5d72080 0x5585f5d7c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:22.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.875+0000 7f354b82f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585f5d7c940 0x5585f5d7ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:22.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.875+0000 7f354b82f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5585f5cd72a0 con 0x5585f5d72080 2024-09-11T22:04:22.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.875+0000 7f354b82f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5585f5c7eb10 con 0x5585f5d71760 2024-09-11T22:04:22.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.875+0000 7f354b82f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5585f5cd3260 con 0x5585f5d7c940 2024-09-11T22:04:22.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.875+0000 7f3542ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585f5d72080 0x5585f5d7c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:22.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.875+0000 7f353bfff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585f5d7c940 0x5585f5d7ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:22.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.875+0000 7f3542ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585f5d72080 0x5585f5d7c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53132/0 (socket says 172.21.15.1:53132) 2024-09-11T22:04:22.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.875+0000 7f3542ffd640 1 -- 172.21.15.1:0/1687548545 learned_addr learned my addr 172.21.15.1:0/1687548545 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:22.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.875+0000 7f35437fe640 1 --2- 172.21.15.1:0/1687548545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585f5d71760 0x5585f5d71b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:22.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.876+0000 7f353bfff640 1 -- 172.21.15.1:0/1687548545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585f5d71760 msgr2=0x5585f5d71b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:22.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.876+0000 7f353bfff640 1 --2- 172.21.15.1:0/1687548545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585f5d71760 0x5585f5d71b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:22.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.876+0000 7f353bfff640 1 -- 172.21.15.1:0/1687548545 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585f5d72080 msgr2=0x5585f5d7c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:22.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.876+0000 7f353bfff640 1 --2- 172.21.15.1:0/1687548545 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585f5d72080 0x5585f5d7c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:22.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.876+0000 7f353bfff640 1 -- 172.21.15.1:0/1687548545 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585f5a48b90 con 0x5585f5d7c940 2024-09-11T22:04:22.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.876+0000 7f35437fe640 1 --2- 172.21.15.1:0/1687548545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585f5d71760 0x5585f5d71b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:04:22.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.876+0000 7f3542ffd640 1 --2- 172.21.15.1:0/1687548545 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585f5d72080 0x5585f5d7c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:04:22.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.876+0000 7f3540ff9640 1 --2- 172.21.15.1:0/4270560790 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f35300a2cf0 0x7f35300a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:22.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.876+0000 7f353bfff640 1 --2- 172.21.15.1:0/1687548545 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585f5d7c940 0x5585f5d7ed30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f352c0cd010 tx=0x7f352c0d72a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c971fe2210023af3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:22.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.876+0000 7f3540ff9640 1 -- 172.21.15.1:0/4270560790 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7f352c0cf990 con 0x7f3544101400 2024-09-11T22:04:22.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.876+0000 7f35397fa640 1 -- 172.21.15.1:0/1687548545 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f352c098080 con 0x5585f5d7c940 2024-09-11T22:04:22.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.877+0000 7f35397fa640 1 -- 172.21.15.1:0/1687548545 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f352c09c4e0 con 0x5585f5d7c940 2024-09-11T22:04:22.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.877+0000 7f35397fa640 1 -- 172.21.15.1:0/1687548545 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f352c09b740 con 0x5585f5d7c940 2024-09-11T22:04:22.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.877+0000 7f35437fe640 1 --2- 172.21.15.1:0/4270560790 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f35300a2cf0 0x7f35300a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:22.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.877+0000 7f354b82f740 1 -- 172.21.15.1:0/1687548545 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585f5d7c940 msgr2=0x5585f5d7ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:22.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.877+0000 7f354b82f740 1 --2- 172.21.15.1:0/1687548545 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585f5d7c940 0x5585f5d7ed30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f352c0cd010 tx=0x7f352c0d72a0 comp rx=0 tx=0).stop 2024-09-11T22:04:22.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.877+0000 7f35437fe640 1 --2- 172.21.15.1:0/4270560790 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f35300a2cf0 0x7f35300a51b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f353c009fd0 tx=0x7f353c009500 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:22.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.878+0000 7f354b82f740 1 -- 172.21.15.1:0/1687548545 shutdown_connections 2024-09-11T22:04:22.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.878+0000 7f354b82f740 1 --2- 172.21.15.1:0/1687548545 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585f5d7c940 0x5585f5d7ed30 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:22.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.878+0000 7f354b82f740 1 --2- 172.21.15.1:0/1687548545 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585f5d72080 0x5585f5d7c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:22.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.878+0000 7f354b82f740 1 --2- 172.21.15.1:0/1687548545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585f5d71760 0x5585f5d71b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:22.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.878+0000 7f354b82f740 1 -- 172.21.15.1:0/1687548545 >> 172.21.15.1:0/1687548545 conn(0x5585f5d6dcf0 msgr2=0x5585f5d6e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:22.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.878+0000 7f354b82f740 1 -- 172.21.15.1:0/1687548545 shutdown_connections 2024-09-11T22:04:22.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.878+0000 7f354b82f740 1 -- 172.21.15.1:0/1687548545 wait complete. 2024-09-11T22:04:22.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.878+0000 7f354b82f740 1 -- start start 2024-09-11T22:04:22.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.878+0000 7f354b82f740 1 Processor -- start 2024-09-11T22:04:22.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.879+0000 7f354b82f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585f5d71760 0x5585f5d79800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:22.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.879+0000 7f354b82f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585f5d72080 0x5585f5d79d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:22.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.879+0000 7f354b82f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585f5d7c940 0x5585f5d7a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:22.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.879+0000 7f354b82f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5585f5d7f510 con 0x5585f5d72080 2024-09-11T22:04:22.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.879+0000 7f353bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585f5d72080 0x5585f5d79d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:22.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.879+0000 7f353bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585f5d72080 0x5585f5d79d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53138/0 (socket says 172.21.15.1:53138) 2024-09-11T22:04:22.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.879+0000 7f3542ffd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585f5d71760 0x5585f5d79800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:22.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.879+0000 7f353bfff640 1 -- 172.21.15.1:0/1043698575 learned_addr learned my addr 172.21.15.1:0/1043698575 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:22.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.879+0000 7f354b82f740 1 -- 172.21.15.1:0/1043698575 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5585f5c5f6e0 con 0x5585f5d7c940 2024-09-11T22:04:22.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.879+0000 7f35437fe640 1 --2- 172.21.15.1:0/1043698575 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585f5d7c940 0x5585f5d7a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:22.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.879+0000 7f354b82f740 1 -- 172.21.15.1:0/1043698575 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5585f5d7f690 con 0x5585f5d71760 2024-09-11T22:04:22.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.879+0000 7f354b82f740 10 client.0 _ll_register_callbacks cb 0x5585f5c83a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:04:22.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.879+0000 7f353bfff640 1 -- 172.21.15.1:0/1043698575 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585f5d71760 msgr2=0x5585f5d79800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:22.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.879+0000 7f353bfff640 1 --2- 172.21.15.1:0/1043698575 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585f5d71760 0x5585f5d79800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:22.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.879+0000 7f353bfff640 1 -- 172.21.15.1:0/1043698575 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585f5d7c940 msgr2=0x5585f5d7a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:22.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.879+0000 7f353bfff640 1 --2- 172.21.15.1:0/1043698575 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585f5d7c940 0x5585f5d7a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:22.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.879+0000 7f353bfff640 1 -- 172.21.15.1:0/1043698575 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585f5d889d0 con 0x5585f5d72080 2024-09-11T22:04:22.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.880+0000 7f353bfff640 1 --2- 172.21.15.1:0/1043698575 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585f5d72080 0x5585f5d79d40 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f352c0cd040 tx=0x7f352c0d5300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cad961a9dffa110 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:22.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.880+0000 7f352a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:04:22.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.880+0000 7f352a7fc640 1 -- 172.21.15.1:0/1043698575 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f352c09b920 con 0x5585f5d72080 2024-09-11T22:04:22.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.880+0000 7f354b82f740 1 -- 172.21.15.1:0/1043698575 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f352c09c4e0 con 0x5585f5d72080 2024-09-11T22:04:22.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.880+0000 7f352a7fc640 1 -- 172.21.15.1:0/1043698575 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7f352c0a45c0 con 0x5585f5d72080 2024-09-11T22:04:22.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.880+0000 7f352a7fc640 1 -- 172.21.15.1:0/1043698575 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f352c09bbb0 con 0x5585f5d72080 2024-09-11T22:04:22.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.880+0000 7f354b82f740 10 client.16998 fetch_fsmap learned FSMap version 42 2024-09-11T22:04:22.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.880+0000 7f354b82f740 1 -- 172.21.15.1:0/1043698575 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5585f5d8b320 con 0x5585f5d72080 2024-09-11T22:04:22.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.880+0000 7f352a7fc640 1 -- 172.21.15.1:0/1043698575 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f352c09c4e0 con 0x5585f5d72080 2024-09-11T22:04:22.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.880+0000 7f352a7fc640 1 -- 172.21.15.1:0/1043698575 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f352c09b230 con 0x5585f5d72080 2024-09-11T22:04:22.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.880+0000 7f354b82f740 10 client.16998 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:04:22.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.881+0000 7f354b82f740 10 client.16998 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-11T22:04:22.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.881+0000 7f354b82f740 20 client.16998 populate_metadata read hostname 'smithi001' 2024-09-11T22:04:22.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.881+0000 7f354b82f740 1 --2- 172.21.15.1:0/1043698575 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5585f5d8c960 0x5585f5d8edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:22.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.881+0000 7f354b82f740 4 client.16998 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:04:22.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.881+0000 7f354b82f740 1 -- 172.21.15.1:0/1043698575 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5585f5ce49d0 con 0x5585f5d8c960 2024-09-11T22:04:22.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.881+0000 7f3542ffd640 1 --2- 172.21.15.1:0/1043698575 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5585f5d8c960 0x5585f5d8edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:22.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.881+0000 7f3542ffd640 1 --2- 172.21.15.1:0/1043698575 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5585f5d8c960 0x5585f5d8edc0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41835229a302464b server_cookie=dab4e029c7023e5 in_seq=0 out_seq=0 2024-09-11T22:04:22.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.881+0000 7f352a7fc640 10 client.16998 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:04:22.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.882+0000 7f352a7fc640 1 -- 172.21.15.1:0/1043698575 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5585f5ce49d0 con 0x5585f5d8c960 2024-09-11T22:04:22.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.882+0000 7f352a7fc640 10 client.16998 handle_command_reply: tid=0 2024-09-11T22:04:22.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.882+0000 7f354b82f740 1 client.16998 shutdown 2024-09-11T22:04:22.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.882+0000 7f354b82f740 1 -- 172.21.15.1:0/1043698575 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585f5d72080 msgr2=0x5585f5d79d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:22.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.882+0000 7f354b82f740 1 --2- 172.21.15.1:0/1043698575 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585f5d72080 0x5585f5d79d40 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f352c0cd040 tx=0x7f352c0d5300 comp rx=0 tx=0).stop 2024-09-11T22:04:22.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.882+0000 7f354b82f740 1 -- 172.21.15.1:0/1043698575 shutdown_connections 2024-09-11T22:04:22.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.882+0000 7f354b82f740 1 --2- 172.21.15.1:0/1043698575 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5585f5d8c960 0x5585f5d8edc0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:22.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.882+0000 7f354b82f740 1 --2- 172.21.15.1:0/1043698575 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585f5d7c940 0x5585f5d7a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:22.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.882+0000 7f354b82f740 1 --2- 172.21.15.1:0/1043698575 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585f5d72080 0x5585f5d79d40 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:22.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.882+0000 7f354b82f740 1 --2- 172.21.15.1:0/1043698575 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585f5d71760 0x5585f5d79800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:22.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.882+0000 7f354b82f740 1 -- 172.21.15.1:0/1043698575 >> 172.21.15.1:0/1043698575 conn(0x5585f5d6dcf0 msgr2=0x5585f5d73a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:22.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.882+0000 7f352a7fc640 0 client.16998 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:04:22.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.882+0000 7f354b82f740 1 -- 172.21.15.1:0/1043698575 shutdown_connections 2024-09-11T22:04:22.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.883+0000 7f354b82f740 1 -- 172.21.15.1:0/1043698575 wait complete. 2024-09-11T22:04:22.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.883+0000 7f354b82f740 20 client.16998 trim_cache size 0 max 16384 2024-09-11T22:04:22.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.902+0000 7f354b82f740 1 Processor -- start 2024-09-11T22:04:22.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.902+0000 7f354b82f740 1 -- start start 2024-09-11T22:04:22.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.903+0000 7f354b82f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585f5d71760 0x5585f5d94690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:22.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.903+0000 7f354b82f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585f5d72080 0x5585f5d94bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:22.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.903+0000 7f354b82f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585f5d7c940 0x5585f5d76a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:22.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.903+0000 7f354b82f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f35441afc50 con 0x5585f5d72080 2024-09-11T22:04:22.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.903+0000 7f354b82f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f35441b1a90 con 0x5585f5d71760 2024-09-11T22:04:22.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.903+0000 7f354b82f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f35441b03e0 con 0x5585f5d7c940 2024-09-11T22:04:22.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.903+0000 7f3542ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585f5d71760 0x5585f5d94690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:22.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.903+0000 7f35437fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585f5d7c940 0x5585f5d76a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:22.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.903+0000 7f3542ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585f5d71760 0x5585f5d94690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:54542/0 (socket says 172.21.15.1:54542) 2024-09-11T22:04:22.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.903+0000 7f3542ffd640 1 -- 172.21.15.1:0/1500795944 learned_addr learned my addr 172.21.15.1:0/1500795944 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:22.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.903+0000 7f353bfff640 1 --2- 172.21.15.1:0/1500795944 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585f5d72080 0x5585f5d94bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:22.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.903+0000 7f3542ffd640 1 -- 172.21.15.1:0/1500795944 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585f5d7c940 msgr2=0x5585f5d76a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:22.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.903+0000 7f3542ffd640 1 --2- 172.21.15.1:0/1500795944 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585f5d7c940 0x5585f5d76a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:22.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.903+0000 7f3542ffd640 1 -- 172.21.15.1:0/1500795944 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585f5d72080 msgr2=0x5585f5d94bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:22.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.903+0000 7f3542ffd640 1 --2- 172.21.15.1:0/1500795944 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585f5d72080 0x5585f5d94bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:22.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.904+0000 7f3542ffd640 1 -- 172.21.15.1:0/1500795944 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585f5d69260 con 0x5585f5d71760 2024-09-11T22:04:22.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.904+0000 7f3542ffd640 1 --2- 172.21.15.1:0/1500795944 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585f5d71760 0x5585f5d94690 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3534009230 tx=0x7f35340058b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7fee791382e60741 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:22.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.904+0000 7f3538ff9640 1 -- 172.21.15.1:0/1500795944 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f353404a070 con 0x5585f5d71760 2024-09-11T22:04:22.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.904+0000 7f3538ff9640 1 -- 172.21.15.1:0/1500795944 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3534005520 con 0x5585f5d71760 2024-09-11T22:04:22.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.904+0000 7f3538ff9640 1 -- 172.21.15.1:0/1500795944 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3534035040 con 0x5585f5d71760 2024-09-11T22:04:22.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.905+0000 7f354b82f740 1 -- 172.21.15.1:0/1500795944 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585f5d71760 msgr2=0x5585f5d94690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:22.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.905+0000 7f354b82f740 1 --2- 172.21.15.1:0/1500795944 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585f5d71760 0x5585f5d94690 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3534009230 tx=0x7f35340058b0 comp rx=0 tx=0).stop 2024-09-11T22:04:22.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.905+0000 7f354b82f740 1 -- 172.21.15.1:0/1500795944 shutdown_connections 2024-09-11T22:04:22.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.905+0000 7f354b82f740 1 --2- 172.21.15.1:0/1500795944 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585f5d7c940 0x5585f5d76a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:22.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.905+0000 7f354b82f740 1 --2- 172.21.15.1:0/1500795944 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585f5d72080 0x5585f5d94bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:22.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.905+0000 7f354b82f740 1 --2- 172.21.15.1:0/1500795944 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585f5d71760 0x5585f5d94690 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:22.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.905+0000 7f354b82f740 1 -- 172.21.15.1:0/1500795944 >> 172.21.15.1:0/1500795944 conn(0x5585f5d6dcf0 msgr2=0x5585f5d6c650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:22.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.905+0000 7f354b82f740 1 -- 172.21.15.1:0/1500795944 shutdown_connections 2024-09-11T22:04:22.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.905+0000 7f354b82f740 1 -- 172.21.15.1:0/1500795944 wait complete. 2024-09-11T22:04:22.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.905+0000 7f354b82f740 1 -- start start 2024-09-11T22:04:22.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.906+0000 7f354b82f740 1 Processor -- start 2024-09-11T22:04:22.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.906+0000 7f354b82f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585f5d71760 0x5585f5d84d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:22.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.906+0000 7f354b82f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585f5d72080 0x5585f5d80040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:22.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.906+0000 7f354b82f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585f5d7c940 0x5585f5d80580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:22.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.906+0000 7f354b82f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f35441b1300 con 0x5585f5d72080 2024-09-11T22:04:22.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.906+0000 7f354b82f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f35441b0b70 con 0x5585f5d7c940 2024-09-11T22:04:22.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.906+0000 7f354b82f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f35441adc50 con 0x5585f5d71760 2024-09-11T22:04:22.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.906+0000 7f354b82f740 10 client.0 _ll_register_callbacks cb 0x7f352c0d59e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:04:22.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.906+0000 7f353bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585f5d72080 0x5585f5d80040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:22.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.906+0000 7f353bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585f5d72080 0x5585f5d80040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53152/0 (socket says 172.21.15.1:53152) 2024-09-11T22:04:22.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.906+0000 7f353bfff640 1 -- 172.21.15.1:0/1081173193 learned_addr learned my addr 172.21.15.1:0/1081173193 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:22.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.907+0000 7f35437fe640 1 --2- 172.21.15.1:0/1081173193 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585f5d7c940 0x5585f5d80580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:22.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.907+0000 7f353bfff640 1 -- 172.21.15.1:0/1081173193 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585f5d71760 msgr2=0x5585f5d84d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:22.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.907+0000 7f353bfff640 1 --2- 172.21.15.1:0/1081173193 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585f5d71760 0x5585f5d84d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:22.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.907+0000 7f353bfff640 1 -- 172.21.15.1:0/1081173193 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585f5d7c940 msgr2=0x5585f5d80580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:22.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.907+0000 7f353bfff640 1 --2- 172.21.15.1:0/1081173193 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585f5d7c940 0x5585f5d80580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:22.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.907+0000 7f353bfff640 1 -- 172.21.15.1:0/1081173193 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585f5d80ac0 con 0x5585f5d72080 2024-09-11T22:04:22.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.907+0000 7f35437fe640 1 --2- 172.21.15.1:0/1081173193 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585f5d7c940 0x5585f5d80580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:04:22.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.907+0000 7f353bfff640 1 --2- 172.21.15.1:0/1081173193 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585f5d72080 0x5585f5d80040 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f352c09a090 tx=0x7f352c09b4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85fafbce296cde0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:22.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.907+0000 7f352a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:04:22.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.907+0000 7f352a7fc640 1 -- 172.21.15.1:0/1081173193 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f352c09b920 con 0x5585f5d72080 2024-09-11T22:04:22.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.907+0000 7f352a7fc640 1 -- 172.21.15.1:0/1081173193 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7f352c010430 con 0x5585f5d72080 2024-09-11T22:04:22.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.907+0000 7f354b82f740 1 -- 172.21.15.1:0/1081173193 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3534005520 con 0x5585f5d72080 2024-09-11T22:04:22.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.908+0000 7f352a7fc640 1 -- 172.21.15.1:0/1081173193 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f352c098080 con 0x5585f5d72080 2024-09-11T22:04:22.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.908+0000 7f354b82f740 10 client.17004 fetch_fsmap learned FSMap version 42 2024-09-11T22:04:22.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.908+0000 7f354b82f740 1 -- 172.21.15.1:0/1081173193 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5585f5d80ac0 con 0x5585f5d72080 2024-09-11T22:04:22.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.908+0000 7f352a7fc640 1 -- 172.21.15.1:0/1081173193 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3534005520 con 0x5585f5d72080 2024-09-11T22:04:22.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.908+0000 7f352a7fc640 1 -- 172.21.15.1:0/1081173193 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f352c018a90 con 0x5585f5d72080 2024-09-11T22:04:22.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.908+0000 7f354b82f740 10 client.17004 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:04:22.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.909+0000 7f354b82f740 10 client.17004 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-11T22:04:22.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.909+0000 7f354b82f740 20 client.17004 populate_metadata read hostname 'smithi001' 2024-09-11T22:04:22.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.909+0000 7f354b82f740 1 --2- 172.21.15.1:0/1081173193 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5585f5dfffb0 0x5585f5e023d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:22.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.909+0000 7f354b82f740 4 client.17004 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.1, "await_for": 45.0}] 2024-09-11T22:04:22.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.909+0000 7f354b82f740 1 -- 172.21.15.1:0/1081173193 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.1, "await_for": 45.0}) -- 0x5585f5cded20 con 0x5585f5dfffb0 2024-09-11T22:04:22.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.909+0000 7f3542ffd640 1 --2- 172.21.15.1:0/1081173193 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5585f5dfffb0 0x5585f5e023d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:22.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.909+0000 7f3542ffd640 1 --2- 172.21.15.1:0/1081173193 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5585f5dfffb0 0x5585f5e023d0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca3fb0900dc61992 server_cookie=5d183c694fecd916 in_seq=0 out_seq=0 2024-09-11T22:04:22.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:22.909+0000 7f352a7fc640 10 client.17004 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:04:22.933 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:04:23.055 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:04:22 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:04:23.055 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:04:22 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:04:23.055 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:04:22 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:04:23.055 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:04:22 smithi001 ceph-mon[30935]: pgmap v221: 97 pgs: 97 active+clean; 433 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.6 MiB/s wr, 414 op/s 2024-09-11T22:04:23.056 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:04:22 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:04:23.056 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:04:22 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:04:23.056 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:04:22 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:04:23.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:04:22 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:04:23.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:04:22 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:04:23.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:04:22 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:04:23.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:04:22 smithi053 ceph-mon[34908]: pgmap v221: 97 pgs: 97 active+clean; 433 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.6 MiB/s wr, 414 op/s 2024-09-11T22:04:23.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:04:22 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:04:23.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:04:22 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:04:23.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:04:22 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:04:23.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:04:22 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:04:23.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:04:22 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:04:23.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:04:22 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:04:23.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:04:22 smithi161 ceph-mon[35615]: pgmap v221: 97 pgs: 97 active+clean; 433 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.6 MiB/s wr, 414 op/s 2024-09-11T22:04:23.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:04:22 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:04:23.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:04:22 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:04:23.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:04:22 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:04:23.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.483+0000 7f5c3ca6d640 1 -- 172.21.15.1:0/3862694726 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c38102ef0 msgr2=0x7f5c3810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:23.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.483+0000 7f5c3ca6d640 1 --2- 172.21.15.1:0/3862694726 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c38102ef0 0x7f5c3810da40 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f5c30009ca0 tx=0x7f5c3002a950 comp rx=0 tx=0).stop 2024-09-11T22:04:23.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.483+0000 7f5c3ca6d640 1 -- 172.21.15.1:0/3862694726 shutdown_connections 2024-09-11T22:04:23.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.483+0000 7f5c3ca6d640 1 --2- 172.21.15.1:0/3862694726 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c38102ef0 0x7f5c3810da40 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:23.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.483+0000 7f5c3ca6d640 1 --2- 172.21.15.1:0/3862694726 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c38102570 0x7f5c381029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:23.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.483+0000 7f5c3ca6d640 1 --2- 172.21.15.1:0/3862694726 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c38100e20 0x7f5c38101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:23.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.483+0000 7f5c3ca6d640 1 -- 172.21.15.1:0/3862694726 >> 172.21.15.1:0/3862694726 conn(0x7f5c380fc7d0 msgr2=0x7f5c380febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:23.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.483+0000 7f5c3ca6d640 1 -- 172.21.15.1:0/3862694726 shutdown_connections 2024-09-11T22:04:23.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.483+0000 7f5c3ca6d640 1 -- 172.21.15.1:0/3862694726 wait complete. 2024-09-11T22:04:23.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.484+0000 7f5c3ca6d640 1 Processor -- start 2024-09-11T22:04:23.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.484+0000 7f5c3ca6d640 1 -- start start 2024-09-11T22:04:23.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.484+0000 7f5c3ca6d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c38100e20 0x7f5c38198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:23.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.484+0000 7f5c3ca6d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c38102570 0x7f5c38198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:23.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.485+0000 7f5c3ca6d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c38102ef0 0x7f5c3819cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:23.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.485+0000 7f5c36d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c38102ef0 0x7f5c3819cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:23.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.485+0000 7f5c36d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c38102ef0 0x7f5c3819cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53170/0 (socket says 172.21.15.1:53170) 2024-09-11T22:04:23.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.485+0000 7f5c36d9d640 1 -- 172.21.15.1:0/1435865656 learned_addr learned my addr 172.21.15.1:0/1435865656 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:23.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.485+0000 7f5c2ffff640 1 --2- 172.21.15.1:0/1435865656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c38102570 0x7f5c38198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:23.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.485+0000 7f5c3ca6d640 1 -- 172.21.15.1:0/1435865656 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5c38110f80 con 0x7f5c38102ef0 2024-09-11T22:04:23.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.485+0000 7f5c3ca6d640 1 -- 172.21.15.1:0/1435865656 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5c38110e00 con 0x7f5c38100e20 2024-09-11T22:04:23.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.485+0000 7f5c3ca6d640 1 -- 172.21.15.1:0/1435865656 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5c38111100 con 0x7f5c38102570 2024-09-11T22:04:23.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.485+0000 7f5c3659c640 1 --2- 172.21.15.1:0/1435865656 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c38100e20 0x7f5c38198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:23.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.485+0000 7f5c36d9d640 1 -- 172.21.15.1:0/1435865656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c38102570 msgr2=0x7f5c38198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:23.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.485+0000 7f5c36d9d640 1 --2- 172.21.15.1:0/1435865656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c38102570 0x7f5c38198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:23.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.485+0000 7f5c36d9d640 1 -- 172.21.15.1:0/1435865656 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c38100e20 msgr2=0x7f5c38198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:23.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.485+0000 7f5c36d9d640 1 --2- 172.21.15.1:0/1435865656 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c38100e20 0x7f5c38198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:23.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.485+0000 7f5c36d9d640 1 -- 172.21.15.1:0/1435865656 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c3819d4b0 con 0x7f5c38102ef0 2024-09-11T22:04:23.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.486+0000 7f5c36d9d640 1 --2- 172.21.15.1:0/1435865656 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c38102ef0 0x7f5c3819cdd0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f5c30004af0 tx=0x7f5c3002f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:23.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.486+0000 7f5c2f7fe640 1 -- 172.21.15.1:0/1435865656 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c30034040 con 0x7f5c38102ef0 2024-09-11T22:04:23.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.486+0000 7f5c3ca6d640 1 -- 172.21.15.1:0/1435865656 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c3819d740 con 0x7f5c38102ef0 2024-09-11T22:04:23.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.486+0000 7f5c2f7fe640 1 -- 172.21.15.1:0/1435865656 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c30030920 con 0x7f5c38102ef0 2024-09-11T22:04:23.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.486+0000 7f5c2f7fe640 1 -- 172.21.15.1:0/1435865656 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c30004630 con 0x7f5c38102ef0 2024-09-11T22:04:23.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.486+0000 7f5c3ca6d640 1 -- 172.21.15.1:0/1435865656 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c381a5080 con 0x7f5c38102ef0 2024-09-11T22:04:23.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.488+0000 7f5c2f7fe640 1 -- 172.21.15.1:0/1435865656 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f5c30037020 con 0x7f5c38102ef0 2024-09-11T22:04:23.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.489+0000 7f5c3ee46740 1 Processor -- start 2024-09-11T22:04:23.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.489+0000 7f5c3ee46740 1 -- start start 2024-09-11T22:04:23.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.489+0000 7f5c3ee46740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561df11f14a0 0x561df11f1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:23.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.489+0000 7f5c3ee46740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561df11f1dc0 0x561df11fc140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:23.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.489+0000 7f5c3ee46740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561df11fc680 0x561df11fea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:23.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.489+0000 7f5c3ee46740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x561df11572e0 con 0x561df11fc680 2024-09-11T22:04:23.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.489+0000 7f5c3ee46740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x561df10fece0 con 0x561df11f1dc0 2024-09-11T22:04:23.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.489+0000 7f5c3ee46740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x561df11532a0 con 0x561df11f14a0 2024-09-11T22:04:23.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.489+0000 7f5c2f7fe640 1 --2- 172.21.15.1:0/1435865656 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f5c240a2cf0 0x7f5c240a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:23.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.489+0000 7f5c2ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561df11fc680 0x561df11fea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:23.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.490+0000 7f5c2ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561df11fc680 0x561df11fea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53176/0 (socket says 172.21.15.1:53176) 2024-09-11T22:04:23.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.490+0000 7f5c2ffff640 1 -- 172.21.15.1:0/2064245292 learned_addr learned my addr 172.21.15.1:0/2064245292 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:23.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.490+0000 7f5c3659c640 1 --2- 172.21.15.1:0/2064245292 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561df11f1dc0 0x561df11fc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:23.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.490+0000 7f5c36d9d640 1 --2- 172.21.15.1:0/1435865656 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f5c240a2cf0 0x7f5c240a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:23.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.490+0000 7f5c2ffff640 1 --2- 172.21.15.1:0/2064245292 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561df11f14a0 0x561df11f1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:23.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.490+0000 7f5c2f7fe640 1 -- 172.21.15.1:0/1435865656 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7f5c300f0170 con 0x7f5c38102ef0 2024-09-11T22:04:23.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.490+0000 7f5c3659c640 1 -- 172.21.15.1:0/2064245292 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561df11f14a0 msgr2=0x561df11f1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:23.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.490+0000 7f5c3659c640 1 --2- 172.21.15.1:0/2064245292 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561df11f14a0 0x561df11f1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:23.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.490+0000 7f5c3659c640 1 -- 172.21.15.1:0/2064245292 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561df11fc680 msgr2=0x561df11fea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:23.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.490+0000 7f5c3659c640 1 --2- 172.21.15.1:0/2064245292 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561df11fc680 0x561df11fea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:23.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.490+0000 7f5c3659c640 1 -- 172.21.15.1:0/2064245292 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561df10e3440 con 0x561df11f1dc0 2024-09-11T22:04:23.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.490+0000 7f5c2ffff640 1 --2- 172.21.15.1:0/2064245292 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561df11fc680 0x561df11fea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:04:23.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.490+0000 7f5c3659c640 1 --2- 172.21.15.1:0/2064245292 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561df11f1dc0 0x561df11fc140 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5c28009b70 tx=0x7f5c280117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b6bde00ffcb09d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:23.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.490+0000 7f5c36d9d640 1 --2- 172.21.15.1:0/1435865656 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f5c240a2cf0 0x7f5c240a51b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5c300edab0 tx=0x7f5c300b1c60 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:23.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.490+0000 7f5c2cff9640 1 -- 172.21.15.1:0/2064245292 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c28009e60 con 0x561df11f1dc0 2024-09-11T22:04:23.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.490+0000 7f5c2cff9640 1 -- 172.21.15.1:0/2064245292 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c28012930 con 0x561df11f1dc0 2024-09-11T22:04:23.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.490+0000 7f5c2cff9640 1 -- 172.21.15.1:0/2064245292 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c2800faf0 con 0x561df11f1dc0 2024-09-11T22:04:23.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.491+0000 7f5c3ee46740 1 -- 172.21.15.1:0/2064245292 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561df11f1dc0 msgr2=0x561df11fc140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:23.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.491+0000 7f5c3ee46740 1 --2- 172.21.15.1:0/2064245292 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561df11f1dc0 0x561df11fc140 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5c28009b70 tx=0x7f5c280117b0 comp rx=0 tx=0).stop 2024-09-11T22:04:23.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.491+0000 7f5c3ee46740 1 -- 172.21.15.1:0/2064245292 shutdown_connections 2024-09-11T22:04:23.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.491+0000 7f5c3ee46740 1 --2- 172.21.15.1:0/2064245292 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561df11fc680 0x561df11fea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:23.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.491+0000 7f5c3ee46740 1 --2- 172.21.15.1:0/2064245292 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561df11f1dc0 0x561df11fc140 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:23.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.491+0000 7f5c3ee46740 1 --2- 172.21.15.1:0/2064245292 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561df11f14a0 0x561df11f1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:23.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.491+0000 7f5c3ee46740 1 -- 172.21.15.1:0/2064245292 >> 172.21.15.1:0/2064245292 conn(0x561df11eda30 msgr2=0x561df11ede10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:23.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.491+0000 7f5c3ee46740 1 -- 172.21.15.1:0/2064245292 shutdown_connections 2024-09-11T22:04:23.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.491+0000 7f5c3ee46740 1 -- 172.21.15.1:0/2064245292 wait complete. 2024-09-11T22:04:23.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.492+0000 7f5c3ee46740 1 -- start start 2024-09-11T22:04:23.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.492+0000 7f5c3ee46740 1 Processor -- start 2024-09-11T22:04:23.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.492+0000 7f5c3ee46740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561df11f14a0 0x561df11fbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:23.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.492+0000 7f5c3ee46740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561df11f1dc0 0x561df11f9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:23.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.492+0000 7f5c3ee46740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561df11fc680 0x561df11f9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:23.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.492+0000 7f5c3ee46740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x561df11ff250 con 0x561df11f1dc0 2024-09-11T22:04:23.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.492+0000 7f5c3ee46740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x561df1112050 con 0x561df11f14a0 2024-09-11T22:04:23.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.492+0000 7f5c3ee46740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x561df11ff3d0 con 0x561df11fc680 2024-09-11T22:04:23.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.492+0000 7f5c3ee46740 10 client.0 _ll_register_callbacks cb 0x561df1152160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:04:23.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.492+0000 7f5c3659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561df11f1dc0 0x561df11f9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:23.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.492+0000 7f5c3659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561df11f1dc0 0x561df11f9220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53178/0 (socket says 172.21.15.1:53178) 2024-09-11T22:04:23.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.492+0000 7f5c3659c640 1 -- 172.21.15.1:0/3044291100 learned_addr learned my addr 172.21.15.1:0/3044291100 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:23.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.492+0000 7f5c2ffff640 1 --2- 172.21.15.1:0/3044291100 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561df11f14a0 0x561df11fbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:23.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.492+0000 7f5c2ffff640 1 --2- 172.21.15.1:0/3044291100 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561df11fc680 0x561df11f9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:23.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.493+0000 7f5c2ffff640 1 -- 172.21.15.1:0/3044291100 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561df11fc680 msgr2=0x561df11f9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:23.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.493+0000 7f5c2ffff640 1 --2- 172.21.15.1:0/3044291100 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561df11fc680 0x561df11f9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:23.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.493+0000 7f5c2ffff640 1 -- 172.21.15.1:0/3044291100 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561df11f1dc0 msgr2=0x561df11f9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:23.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.493+0000 7f5c2ffff640 1 --2- 172.21.15.1:0/3044291100 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561df11f1dc0 0x561df11f9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:23.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.493+0000 7f5c2ffff640 1 -- 172.21.15.1:0/3044291100 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561df0ec83c0 con 0x561df11f14a0 2024-09-11T22:04:23.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.493+0000 7f5c3659c640 1 --2- 172.21.15.1:0/3044291100 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561df11f1dc0 0x561df11f9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:04:23.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.493+0000 7f5c2ffff640 1 --2- 172.21.15.1:0/3044291100 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561df11f14a0 0x561df11fbf70 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5c20009ef0 tx=0x7f5c20004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d611008d1435bbd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:23.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.493+0000 7f5c1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-11T22:04:23.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.493+0000 7f5c1dffb640 1 -- 172.21.15.1:0/3044291100 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c20004050 con 0x561df11f14a0 2024-09-11T22:04:23.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.493+0000 7f5c1dffb640 1 -- 172.21.15.1:0/3044291100 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7f5c200041f0 con 0x561df11f14a0 2024-09-11T22:04:23.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.493+0000 7f5c3ee46740 1 -- 172.21.15.1:0/3044291100 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c28012930 con 0x561df11f14a0 2024-09-11T22:04:23.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.493+0000 7f5c1dffb640 1 -- 172.21.15.1:0/3044291100 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f5c20028050 con 0x561df11f14a0 2024-09-11T22:04:23.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.493+0000 7f5c3ee46740 10 client.26537 fetch_fsmap learned FSMap version 42 2024-09-11T22:04:23.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.493+0000 7f5c3ee46740 1 -- 172.21.15.1:0/3044291100 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x561df120ac20 con 0x561df11f14a0 2024-09-11T22:04:23.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.493+0000 7f5c1dffb640 1 -- 172.21.15.1:0/3044291100 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c28012930 con 0x561df11f14a0 2024-09-11T22:04:23.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.494+0000 7f5c1dffb640 1 -- 172.21.15.1:0/3044291100 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f5c20023380 con 0x561df11f14a0 2024-09-11T22:04:23.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.494+0000 7f5c3ee46740 10 client.26537 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:04:23.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.494+0000 7f5c3ee46740 10 client.26537 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:04:23.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.494+0000 7f5c3ee46740 20 client.26537 populate_metadata read hostname 'smithi001' 2024-09-11T22:04:23.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.494+0000 7f5c3ee46740 1 --2- 172.21.15.1:0/3044291100 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x561df120c130 0x561df120e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:23.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.494+0000 7f5c3ee46740 4 client.26537 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:04:23.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.494+0000 7f5c3ee46740 1 -- 172.21.15.1:0/3044291100 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561df1164850 con 0x561df120c130 2024-09-11T22:04:23.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.494+0000 7f5c3659c640 1 --2- 172.21.15.1:0/3044291100 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x561df120c130 0x561df120e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:23.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.494+0000 7f5c1dffb640 10 client.26537 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:04:23.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.494+0000 7f5c3659c640 1 --2- 172.21.15.1:0/3044291100 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x561df120c130 0x561df120e580 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97f7586c7fc17c49 server_cookie=fd193b15128d7b0 in_seq=0 out_seq=0 2024-09-11T22:04:23.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.495+0000 7f5c1dffb640 1 -- 172.21.15.1:0/3044291100 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561df1164850 con 0x561df120c130 2024-09-11T22:04:23.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.495+0000 7f5c1dffb640 10 client.26537 handle_command_reply: tid=0 2024-09-11T22:04:23.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.495+0000 7f5c3ee46740 1 client.26537 shutdown 2024-09-11T22:04:23.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.495+0000 7f5c3ee46740 1 -- 172.21.15.1:0/3044291100 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561df11f14a0 msgr2=0x561df11fbf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:23.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.495+0000 7f5c3ee46740 1 --2- 172.21.15.1:0/3044291100 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561df11f14a0 0x561df11fbf70 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5c20009ef0 tx=0x7f5c20004730 comp rx=0 tx=0).stop 2024-09-11T22:04:23.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.495+0000 7f5c3ee46740 1 -- 172.21.15.1:0/3044291100 shutdown_connections 2024-09-11T22:04:23.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.495+0000 7f5c3ee46740 1 --2- 172.21.15.1:0/3044291100 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x561df120c130 0x561df120e580 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:23.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.495+0000 7f5c3ee46740 1 --2- 172.21.15.1:0/3044291100 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561df11fc680 0x561df11f9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:23.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.495+0000 7f5c3ee46740 1 --2- 172.21.15.1:0/3044291100 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561df11f1dc0 0x561df11f9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:23.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.495+0000 7f5c1dffb640 0 client.26537 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:04:23.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.495+0000 7f5c3ee46740 1 --2- 172.21.15.1:0/3044291100 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561df11f14a0 0x561df11fbf70 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:23.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.495+0000 7f5c3ee46740 1 -- 172.21.15.1:0/3044291100 >> 172.21.15.1:0/3044291100 conn(0x561df11eda30 msgr2=0x561df11f3710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:23.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.495+0000 7f5c3ee46740 1 -- 172.21.15.1:0/3044291100 shutdown_connections 2024-09-11T22:04:23.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.495+0000 7f5c3ee46740 1 -- 172.21.15.1:0/3044291100 wait complete. 2024-09-11T22:04:23.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.495+0000 7f5c3ee46740 20 client.26537 trim_cache size 0 max 16384 2024-09-11T22:04:23.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.513+0000 7f5c3ee46740 1 Processor -- start 2024-09-11T22:04:23.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.513+0000 7f5c3ee46740 1 -- start start 2024-09-11T22:04:23.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.513+0000 7f5c3ee46740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561df11f14a0 0x561df11ede10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:23.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.513+0000 7f5c3ee46740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561df11f1dc0 0x561df1213e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:23.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.513+0000 7f5c3ee46740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561df11fc680 0x561df12143d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:23.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.513+0000 7f5c3ee46740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5c381ada40 con 0x561df11fc680 2024-09-11T22:04:23.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.513+0000 7f5c3ee46740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5c381af880 con 0x561df11f14a0 2024-09-11T22:04:23.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.513+0000 7f5c3ee46740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5c381ae1d0 con 0x561df11f1dc0 2024-09-11T22:04:23.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.513+0000 7f5c2ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561df11fc680 0x561df12143d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:23.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.513+0000 7f5c3659c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561df11f1dc0 0x561df1213e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:23.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.513+0000 7f5c2ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561df11fc680 0x561df12143d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53194/0 (socket says 172.21.15.1:53194) 2024-09-11T22:04:23.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.513+0000 7f5c2ffff640 1 -- 172.21.15.1:0/1346798838 learned_addr learned my addr 172.21.15.1:0/1346798838 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:23.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.513+0000 7f5c2ffff640 1 --2- 172.21.15.1:0/1346798838 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561df11f14a0 0x561df11ede10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:23.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.514+0000 7f5c3659c640 1 -- 172.21.15.1:0/1346798838 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561df11f14a0 msgr2=0x561df11ede10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:23.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.514+0000 7f5c3659c640 1 --2- 172.21.15.1:0/1346798838 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561df11f14a0 0x561df11ede10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:23.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.514+0000 7f5c3659c640 1 -- 172.21.15.1:0/1346798838 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561df11fc680 msgr2=0x561df12143d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:23.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.514+0000 7f5c3659c640 1 --2- 172.21.15.1:0/1346798838 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561df11fc680 0x561df12143d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:23.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.514+0000 7f5c3659c640 1 -- 172.21.15.1:0/1346798838 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561df11f81d0 con 0x561df11f1dc0 2024-09-11T22:04:23.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.514+0000 7f5c2ffff640 1 --2- 172.21.15.1:0/1346798838 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561df11f14a0 0x561df11ede10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:04:23.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.514+0000 7f5c3659c640 1 --2- 172.21.15.1:0/1346798838 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561df11f1dc0 0x561df1213e90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5c28005890 tx=0x7f5c2800f650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5cdfcaea5d880d26 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:23.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.514+0000 7f5c1ffff640 1 -- 172.21.15.1:0/1346798838 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c28005050 con 0x561df11f1dc0 2024-09-11T22:04:23.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.514+0000 7f5c1ffff640 1 -- 172.21.15.1:0/1346798838 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c28029070 con 0x561df11f1dc0 2024-09-11T22:04:23.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.514+0000 7f5c1ffff640 1 -- 172.21.15.1:0/1346798838 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c280035f0 con 0x561df11f1dc0 2024-09-11T22:04:23.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.515+0000 7f5c3ee46740 1 -- 172.21.15.1:0/1346798838 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561df11f1dc0 msgr2=0x561df1213e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:23.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.515+0000 7f5c3ee46740 1 --2- 172.21.15.1:0/1346798838 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561df11f1dc0 0x561df1213e90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5c28005890 tx=0x7f5c2800f650 comp rx=0 tx=0).stop 2024-09-11T22:04:23.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.515+0000 7f5c3ee46740 1 -- 172.21.15.1:0/1346798838 shutdown_connections 2024-09-11T22:04:23.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.515+0000 7f5c3ee46740 1 --2- 172.21.15.1:0/1346798838 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561df11fc680 0x561df12143d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:23.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.515+0000 7f5c3ee46740 1 --2- 172.21.15.1:0/1346798838 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561df11f1dc0 0x561df1213e90 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:23.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.515+0000 7f5c3ee46740 1 --2- 172.21.15.1:0/1346798838 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561df11f14a0 0x561df11ede10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:23.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.515+0000 7f5c3ee46740 1 -- 172.21.15.1:0/1346798838 >> 172.21.15.1:0/1346798838 conn(0x561df11eda30 msgr2=0x561df11ecb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:23.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.515+0000 7f5c3ee46740 1 -- 172.21.15.1:0/1346798838 shutdown_connections 2024-09-11T22:04:23.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.515+0000 7f5c3ee46740 1 -- 172.21.15.1:0/1346798838 wait complete. 2024-09-11T22:04:23.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.515+0000 7f5c3ee46740 1 -- start start 2024-09-11T22:04:23.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.515+0000 7f5c3ee46740 1 Processor -- start 2024-09-11T22:04:23.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.516+0000 7f5c3ee46740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561df11f14a0 0x561df1204a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:23.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.516+0000 7f5c3ee46740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561df11f1dc0 0x561df1204f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:23.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.516+0000 7f5c3ee46740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561df11fc680 0x561df1201fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:23.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.516+0000 7f5c3ee46740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5c381af0f0 con 0x561df11fc680 2024-09-11T22:04:23.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.516+0000 7f5c3ee46740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5c381ae960 con 0x561df11f14a0 2024-09-11T22:04:23.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.516+0000 7f5c3ee46740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5c381aba40 con 0x561df11f1dc0 2024-09-11T22:04:23.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.516+0000 7f5c3ee46740 10 client.0 _ll_register_callbacks cb 0x7f5c20009d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:04:23.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.516+0000 7f5c2ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561df11fc680 0x561df1201fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:23.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.516+0000 7f5c2ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561df11fc680 0x561df1201fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53210/0 (socket says 172.21.15.1:53210) 2024-09-11T22:04:23.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.516+0000 7f5c2ffff640 1 -- 172.21.15.1:0/1352161157 learned_addr learned my addr 172.21.15.1:0/1352161157 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:23.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.516+0000 7f5c3659c640 1 --2- 172.21.15.1:0/1352161157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561df11f1dc0 0x561df1204f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:23.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.516+0000 7f5c2ffff640 1 --2- 172.21.15.1:0/1352161157 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561df11f14a0 0x561df1204a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:23.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.516+0000 7f5c3659c640 1 -- 172.21.15.1:0/1352161157 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561df11f14a0 msgr2=0x561df1204a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:23.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.516+0000 7f5c3659c640 1 --2- 172.21.15.1:0/1352161157 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561df11f14a0 0x561df1204a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:23.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.516+0000 7f5c3659c640 1 -- 172.21.15.1:0/1352161157 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561df11fc680 msgr2=0x561df1201fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:23.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.516+0000 7f5c3659c640 1 --2- 172.21.15.1:0/1352161157 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561df11fc680 0x561df1201fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:23.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.516+0000 7f5c3659c640 1 -- 172.21.15.1:0/1352161157 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561df12027e0 con 0x561df11f1dc0 2024-09-11T22:04:23.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.516+0000 7f5c3659c640 1 --2- 172.21.15.1:0/1352161157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561df11f1dc0 0x561df1204f50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x561df11e95f0 tx=0x7f5c2800f4c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3bc8b31dfbace14b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:23.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.516+0000 7f5c1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-11T22:04:23.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.517+0000 7f5c1dffb640 1 -- 172.21.15.1:0/1352161157 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c2800fe70 con 0x561df11f1dc0 2024-09-11T22:04:23.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.517+0000 7f5c3ee46740 1 -- 172.21.15.1:0/1352161157 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c28029070 con 0x561df11f1dc0 2024-09-11T22:04:23.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.517+0000 7f5c1dffb640 1 -- 172.21.15.1:0/1352161157 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7f5c280126c0 con 0x561df11f1dc0 2024-09-11T22:04:23.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.517+0000 7f5c1dffb640 1 -- 172.21.15.1:0/1352161157 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f5c280128c0 con 0x561df11f1dc0 2024-09-11T22:04:23.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.517+0000 7f5c3ee46740 10 client.26509 fetch_fsmap learned FSMap version 42 2024-09-11T22:04:23.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.517+0000 7f5c3ee46740 1 -- 172.21.15.1:0/1352161157 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x561df1276aa0 con 0x561df11f1dc0 2024-09-11T22:04:23.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.517+0000 7f5c1dffb640 1 -- 172.21.15.1:0/1352161157 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c28029070 con 0x561df11f1dc0 2024-09-11T22:04:23.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.517+0000 7f5c1dffb640 1 -- 172.21.15.1:0/1352161157 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f5c280035f0 con 0x561df11f1dc0 2024-09-11T22:04:23.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.517+0000 7f5c3ee46740 10 client.26509 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:04:23.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.517+0000 7f5c3ee46740 10 client.26509 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:04:23.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.517+0000 7f5c3ee46740 20 client.26509 populate_metadata read hostname 'smithi001' 2024-09-11T22:04:23.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.517+0000 7f5c3ee46740 1 --2- 172.21.15.1:0/1352161157 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x561df1277030 0x561df1279450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:23.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.517+0000 7f5c3ee46740 4 client.26509 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:04:23.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.517+0000 7f5c3ee46740 1 -- 172.21.15.1:0/1352161157 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561df115ed60 con 0x561df1277030 2024-09-11T22:04:23.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.518+0000 7f5c2ffff640 1 --2- 172.21.15.1:0/1352161157 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x561df1277030 0x561df1279450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:23.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.518+0000 7f5c2ffff640 1 --2- 172.21.15.1:0/1352161157 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x561df1277030 0x561df1279450 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29aff22395c4d6d2 server_cookie=27f9dcf1ea4a6042 in_seq=0 out_seq=0 2024-09-11T22:04:23.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.518+0000 7f5c1dffb640 10 client.26509 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:04:23.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.518+0000 7f5c1dffb640 1 -- 172.21.15.1:0/1352161157 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x561df115ed60 con 0x561df1277030 2024-09-11T22:04:23.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.518+0000 7f5c1dffb640 10 client.26509 handle_command_reply: tid=0 2024-09-11T22:04:23.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.518+0000 7f5c3ee46740 1 client.26509 shutdown 2024-09-11T22:04:23.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.519+0000 7f5c3ee46740 1 -- 172.21.15.1:0/1352161157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561df11f1dc0 msgr2=0x561df1204f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:23.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.519+0000 7f5c3ee46740 1 --2- 172.21.15.1:0/1352161157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561df11f1dc0 0x561df1204f50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x561df11e95f0 tx=0x7f5c2800f4c0 comp rx=0 tx=0).stop 2024-09-11T22:04:23.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.519+0000 7f5c3ee46740 1 -- 172.21.15.1:0/1352161157 shutdown_connections 2024-09-11T22:04:23.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.519+0000 7f5c3ee46740 1 --2- 172.21.15.1:0/1352161157 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x561df1277030 0x561df1279450 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:23.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.519+0000 7f5c3ee46740 1 --2- 172.21.15.1:0/1352161157 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561df11fc680 0x561df1201fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:23.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.519+0000 7f5c3ee46740 1 --2- 172.21.15.1:0/1352161157 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561df11f1dc0 0x561df1204f50 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:23.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.519+0000 7f5c3ee46740 1 --2- 172.21.15.1:0/1352161157 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561df11f14a0 0x561df1204a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:23.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.519+0000 7f5c3ee46740 1 -- 172.21.15.1:0/1352161157 >> 172.21.15.1:0/1352161157 conn(0x561df11eda30 msgr2=0x561df11faa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:23.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.519+0000 7f5c1dffb640 0 client.26509 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:04:23.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.519+0000 7f5c3ee46740 1 -- 172.21.15.1:0/1352161157 shutdown_connections 2024-09-11T22:04:23.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.519+0000 7f5c3ee46740 1 -- 172.21.15.1:0/1352161157 wait complete. 2024-09-11T22:04:23.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.519+0000 7f5c3ee46740 20 client.26509 trim_cache size 0 max 16384 2024-09-11T22:04:23.526 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-11T22:04:23.526 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T22:04:23.527 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-09-11T22:04:23.527 INFO:teuthology.orchestra.run.smithi001.stdout: "deec3096-57b8-4c8f-ae1a-664e2a2c29f6": { 2024-09-11T22:04:23.527 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2024-09-11T22:04:23.527 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "deec3096-57b8-4c8f-ae1a-664e2a2c29f6", 2024-09-11T22:04:23.527 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2024-09-11T22:04:23.527 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:04:23.527 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:04:23.527 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-11T22:04:23.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.519+0000 7f5c3ca6d640 1 -- 172.21.15.1:0/1435865656 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f5c240a2cf0 msgr2=0x7f5c240a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:23.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.519+0000 7f5c3ca6d640 1 --2- 172.21.15.1:0/1435865656 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f5c240a2cf0 0x7f5c240a51b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5c300edab0 tx=0x7f5c300b1c60 comp rx=0 tx=0).stop 2024-09-11T22:04:23.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.519+0000 7f5c3ca6d640 1 -- 172.21.15.1:0/1435865656 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c38102ef0 msgr2=0x7f5c3819cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:23.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.519+0000 7f5c3ca6d640 1 --2- 172.21.15.1:0/1435865656 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c38102ef0 0x7f5c3819cdd0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f5c30004af0 tx=0x7f5c3002f3a0 comp rx=0 tx=0).stop 2024-09-11T22:04:23.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.519+0000 7f5c3ca6d640 1 -- 172.21.15.1:0/1435865656 shutdown_connections 2024-09-11T22:04:23.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.519+0000 7f5c3ca6d640 1 --2- 172.21.15.1:0/1435865656 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f5c240a2cf0 0x7f5c240a51b0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:23.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.519+0000 7f5c3ca6d640 1 --2- 172.21.15.1:0/1435865656 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c38102ef0 0x7f5c3819cdd0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:23.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.519+0000 7f5c3ca6d640 1 --2- 172.21.15.1:0/1435865656 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c38102570 0x7f5c38198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:23.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.519+0000 7f5c3ca6d640 1 --2- 172.21.15.1:0/1435865656 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c38100e20 0x7f5c38198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:23.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.519+0000 7f5c3ca6d640 1 -- 172.21.15.1:0/1435865656 >> 172.21.15.1:0/1435865656 conn(0x7f5c380fc7d0 msgr2=0x7f5c3806a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:23.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.520+0000 7f5c3ca6d640 1 -- 172.21.15.1:0/1435865656 shutdown_connections 2024-09-11T22:04:23.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:23.520+0000 7f5c3ca6d640 1 -- 172.21.15.1:0/1435865656 wait complete. 2024-09-11T22:04:24.254 INFO:tasks.cephfs.filesystem:scrub status for tag:deec3096-57b8-4c8f-ae1a-664e2a2c29f6 - {'path': '/', 'tag': 'deec3096-57b8-4c8f-ae1a-664e2a2c29f6', 'options': 'recursive,force'} 2024-09-11T22:04:24.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:24.491+0000 7f352a7fc640 1 -- 172.21.15.1:0/1081173193 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5585f5cded20 con 0x5585f5dfffb0 2024-09-11T22:04:24.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:24.491+0000 7f352a7fc640 10 client.17004 handle_command_reply: tid=0 2024-09-11T22:04:24.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:24.491+0000 7f354b82f740 1 client.17004 shutdown 2024-09-11T22:04:24.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:24.491+0000 7f354b82f740 1 -- 172.21.15.1:0/1081173193 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585f5d72080 msgr2=0x5585f5d80040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:24.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:24.491+0000 7f354b82f740 1 --2- 172.21.15.1:0/1081173193 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585f5d72080 0x5585f5d80040 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f352c09a090 tx=0x7f352c09b4e0 comp rx=0 tx=0).stop 2024-09-11T22:04:24.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:24.491+0000 7f354b82f740 1 -- 172.21.15.1:0/1081173193 shutdown_connections 2024-09-11T22:04:24.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:24.491+0000 7f354b82f740 1 --2- 172.21.15.1:0/1081173193 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5585f5dfffb0 0x5585f5e023d0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:24.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:24.491+0000 7f354b82f740 1 --2- 172.21.15.1:0/1081173193 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5585f5d7c940 0x5585f5d80580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:24.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:24.491+0000 7f352a7fc640 0 client.17004 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:04:24.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:24.491+0000 7f354b82f740 1 --2- 172.21.15.1:0/1081173193 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585f5d72080 0x5585f5d80040 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:24.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:24.491+0000 7f354b82f740 1 --2- 172.21.15.1:0/1081173193 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5585f5d71760 0x5585f5d84d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:24.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:24.491+0000 7f354b82f740 1 -- 172.21.15.1:0/1081173193 >> 172.21.15.1:0/1081173193 conn(0x5585f5d6dcf0 msgr2=0x5585f5d77fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:24.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:24.491+0000 7f354b82f740 1 -- 172.21.15.1:0/1081173193 shutdown_connections 2024-09-11T22:04:24.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:24.491+0000 7f354b82f740 1 -- 172.21.15.1:0/1081173193 wait complete. 2024-09-11T22:04:24.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:24.491+0000 7f354b82f740 20 client.17004 trim_cache size 0 max 16384 2024-09-11T22:04:24.495 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-11T22:04:24.495 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 42, 2024-09-11T22:04:24.495 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24494, 2024-09-11T22:04:24.495 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 22, 2024-09-11T22:04:24.495 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-09-11T22:04:24.496 INFO:teuthology.orchestra.run.smithi001.stdout: "42839ccd": { 2024-09-11T22:04:24.496 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 22, 2024-09-11T22:04:24.496 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-09-11T22:04:24.496 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-11T22:04:24.496 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-09-11T22:04:24.496 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-11T22:04:24.496 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:04:24.496 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 90.0, 2024-09-11T22:04:24.496 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 140.1, 2024-09-11T22:04:24.496 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-09-11T22:04:24.496 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-09-11T22:04:24.497 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-09-11T22:04:24.497 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-11T22:04:24.497 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-09-11T22:04:24.497 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-11T22:04:24.497 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:04:24.497 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:04:24.497 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:04:24.497 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:04:24.497 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:04:24.497 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-11T22:04:24.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:24.492+0000 7f3549456640 1 -- 172.21.15.1:0/4270560790 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f35300a2cf0 msgr2=0x7f35300a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:24.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:24.492+0000 7f3549456640 1 --2- 172.21.15.1:0/4270560790 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f35300a2cf0 0x7f35300a51b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f353c009fd0 tx=0x7f353c009500 comp rx=0 tx=0).stop 2024-09-11T22:04:24.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:24.492+0000 7f3549456640 1 -- 172.21.15.1:0/4270560790 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3544101400 msgr2=0x7f354419ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:24.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:24.492+0000 7f3549456640 1 --2- 172.21.15.1:0/4270560790 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3544101400 0x7f354419ab80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f352c00e9f0 tx=0x7f352c00eeb0 comp rx=0 tx=0).stop 2024-09-11T22:04:24.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:24.492+0000 7f3549456640 1 -- 172.21.15.1:0/4270560790 shutdown_connections 2024-09-11T22:04:24.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:24.492+0000 7f3549456640 1 --2- 172.21.15.1:0/4270560790 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f35300a2cf0 0x7f35300a51b0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:24.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:24.492+0000 7f3549456640 1 --2- 172.21.15.1:0/4270560790 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3544106a60 0x7f354419ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:24.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:24.492+0000 7f3549456640 1 --2- 172.21.15.1:0/4270560790 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3544101400 0x7f354419ab80 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:24.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:24.492+0000 7f3549456640 1 --2- 172.21.15.1:0/4270560790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3544100a60 0x7f354419a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:24.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:24.492+0000 7f3549456640 1 -- 172.21.15.1:0/4270560790 >> 172.21.15.1:0/4270560790 conn(0x7f35440fc780 msgr2=0x7f354406a910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:24.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:24.492+0000 7f3549456640 1 -- 172.21.15.1:0/4270560790 shutdown_connections 2024-09-11T22:04:24.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:24.492+0000 7f3549456640 1 -- 172.21.15.1:0/4270560790 wait complete. 2024-09-11T22:04:25.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:04:24 smithi001 ceph-mon[30935]: pgmap v222: 97 pgs: 97 active+clean; 433 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.5 MiB/s wr, 406 op/s 2024-09-11T22:04:25.297 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '42839ccd', quiesce took 9.3 seconds. Will release after: 10.8 2024-09-11T22:04:25.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:04:24 smithi053 ceph-mon[34908]: pgmap v222: 97 pgs: 97 active+clean; 433 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.5 MiB/s wr, 406 op/s 2024-09-11T22:04:25.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:04:24 smithi161 ceph-mon[35615]: pgmap v222: 97 pgs: 97 active+clean; 433 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.5 MiB/s wr, 406 op/s 2024-09-11T22:04:27.060 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.056+0000 7f16b77b9640 1 -- 172.21.15.1:0/595214647 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16b0102d20 msgr2=0x7f16b0103100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:27.060 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.056+0000 7f16b77b9640 1 --2- 172.21.15.1:0/595214647 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16b0102d20 0x7f16b0103100 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f16a0009900 tx=0x7f16a002a480 comp rx=0 tx=0).stop 2024-09-11T22:04:27.060 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.056+0000 7f16b77b9640 1 -- 172.21.15.1:0/595214647 shutdown_connections 2024-09-11T22:04:27.061 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.056+0000 7f16b77b9640 1 --2- 172.21.15.1:0/595214647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16b010c0f0 0x7f16b010e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:27.061 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.056+0000 7f16b77b9640 1 --2- 172.21.15.1:0/595214647 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f16b0103640 0x7f16b010bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:27.061 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.056+0000 7f16b77b9640 1 --2- 172.21.15.1:0/595214647 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16b0102d20 0x7f16b0103100 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:27.061 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.056+0000 7f16b77b9640 1 -- 172.21.15.1:0/595214647 >> 172.21.15.1:0/595214647 conn(0x7f16b00779a0 msgr2=0x7f16b00fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:27.061 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.056+0000 7f16b77b9640 1 -- 172.21.15.1:0/595214647 shutdown_connections 2024-09-11T22:04:27.061 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.056+0000 7f16b77b9640 1 -- 172.21.15.1:0/595214647 wait complete. 2024-09-11T22:04:27.061 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.057+0000 7f16b77b9640 1 Processor -- start 2024-09-11T22:04:27.061 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.057+0000 7f16b77b9640 1 -- start start 2024-09-11T22:04:27.061 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.057+0000 7f16b77b9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f16b0102d20 0x7f16b0100970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:27.061 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.057+0000 7f16b77b9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16b0103640 0x7f16b0100eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:27.062 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.057+0000 7f16b77b9640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16b010c0f0 0x7f16b01013f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:27.062 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.057+0000 7f16b77b9640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f16b0111ea0 con 0x7f16b0103640 2024-09-11T22:04:27.062 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.057+0000 7f16b77b9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f16b0111d20 con 0x7f16b0102d20 2024-09-11T22:04:27.062 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.057+0000 7f16b77b9640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f16b0112020 con 0x7f16b010c0f0 2024-09-11T22:04:27.062 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.057+0000 7f16b4d54640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16b0103640 0x7f16b0100eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:27.062 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.058+0000 7f16b4d54640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16b0103640 0x7f16b0100eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:32864/0 (socket says 172.21.15.1:32864) 2024-09-11T22:04:27.062 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.058+0000 7f16b4d54640 1 -- 172.21.15.1:0/1166434655 learned_addr learned my addr 172.21.15.1:0/1166434655 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:27.062 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.058+0000 7f16b5d56640 1 --2- 172.21.15.1:0/1166434655 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16b010c0f0 0x7f16b01013f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:27.062 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.058+0000 7f16b5555640 1 --2- 172.21.15.1:0/1166434655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f16b0102d20 0x7f16b0100970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:27.062 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.058+0000 7f16b4d54640 1 -- 172.21.15.1:0/1166434655 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16b010c0f0 msgr2=0x7f16b01013f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:27.062 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.058+0000 7f16b4d54640 1 --2- 172.21.15.1:0/1166434655 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16b010c0f0 0x7f16b01013f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:27.062 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.058+0000 7f16b4d54640 1 -- 172.21.15.1:0/1166434655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f16b0102d20 msgr2=0x7f16b0100970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:27.062 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.058+0000 7f16b4d54640 1 --2- 172.21.15.1:0/1166434655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f16b0102d20 0x7f16b0100970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:27.063 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.058+0000 7f16b4d54640 1 -- 172.21.15.1:0/1166434655 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16b01a68d0 con 0x7f16b0103640 2024-09-11T22:04:27.063 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.058+0000 7f16b5d56640 1 --2- 172.21.15.1:0/1166434655 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16b010c0f0 0x7f16b01013f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:04:27.063 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.058+0000 7f16b5555640 1 --2- 172.21.15.1:0/1166434655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f16b0102d20 0x7f16b0100970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:04:27.063 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.058+0000 7f16b4d54640 1 --2- 172.21.15.1:0/1166434655 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16b0103640 0x7f16b0100eb0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f16a400e9f0 tx=0x7f16a400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:27.063 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.058+0000 7f16ae7fc640 1 -- 172.21.15.1:0/1166434655 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a400cd80 con 0x7f16b0103640 2024-09-11T22:04:27.063 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.058+0000 7f16ae7fc640 1 -- 172.21.15.1:0/1166434655 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16a4010910 con 0x7f16b0103640 2024-09-11T22:04:27.063 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.058+0000 7f16b77b9640 1 -- 172.21.15.1:0/1166434655 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16b01a6bc0 con 0x7f16b0103640 2024-09-11T22:04:27.063 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.058+0000 7f16ae7fc640 1 -- 172.21.15.1:0/1166434655 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a4018990 con 0x7f16b0103640 2024-09-11T22:04:27.063 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.058+0000 7f16b77b9640 1 -- 172.21.15.1:0/1166434655 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16b01a7160 con 0x7f16b0103640 2024-09-11T22:04:27.068 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.061+0000 7f16ae7fc640 1 -- 172.21.15.1:0/1166434655 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f16a4010ac0 con 0x7f16b0103640 2024-09-11T22:04:27.068 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.061+0000 7f16ba627740 1 Processor -- start 2024-09-11T22:04:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.061+0000 7f16ba627740 1 -- start start 2024-09-11T22:04:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.061+0000 7f16ba627740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561f3b876af0 0x561f3b876ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.061+0000 7f16ba627740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561f3b877410 0x561f3b881770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.061+0000 7f16ba627740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f3b881cb0 0x561f3b8840a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.061+0000 7f16ba627740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x561f3b7db2f0 con 0x561f3b877410 2024-09-11T22:04:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.061+0000 7f16ba627740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x561f3b783270 con 0x561f3b881cb0 2024-09-11T22:04:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.061+0000 7f16ba627740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x561f3b7d72b0 con 0x561f3b876af0 2024-09-11T22:04:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.061+0000 7f16ae7fc640 1 --2- 172.21.15.1:0/1166434655 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f16980a2d70 0x7f16980a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.062+0000 7f16b5555640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561f3b877410 0x561f3b881770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.062+0000 7f16b5555640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561f3b877410 0x561f3b881770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:32880/0 (socket says 172.21.15.1:32880) 2024-09-11T22:04:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.062+0000 7f16b5555640 1 -- 172.21.15.1:0/803706642 learned_addr learned my addr 172.21.15.1:0/803706642 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.062+0000 7f16b5d56640 1 --2- 172.21.15.1:0/803706642 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561f3b876af0 0x561f3b876ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.062+0000 7f16b5d56640 1 --2- 172.21.15.1:0/1166434655 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f16980a2d70 0x7f16980a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.062+0000 7f16ae7fc640 1 -- 172.21.15.1:0/1166434655 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7f16a40cf890 con 0x7f16b0103640 2024-09-11T22:04:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.062+0000 7f16b4d54640 1 --2- 172.21.15.1:0/803706642 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f3b881cb0 0x561f3b8840a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.062+0000 7f16b5555640 1 -- 172.21.15.1:0/803706642 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561f3b876af0 msgr2=0x561f3b876ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.062+0000 7f16b5555640 1 --2- 172.21.15.1:0/803706642 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561f3b876af0 0x561f3b876ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.062+0000 7f16b5555640 1 -- 172.21.15.1:0/803706642 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f3b881cb0 msgr2=0x561f3b8840a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.062+0000 7f16b5555640 1 --2- 172.21.15.1:0/803706642 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f3b881cb0 0x561f3b8840a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.062+0000 7f16b5555640 1 -- 172.21.15.1:0/803706642 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f3b782340 con 0x561f3b877410 2024-09-11T22:04:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.062+0000 7f16b5d56640 1 --2- 172.21.15.1:0/1166434655 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f16980a2d70 0x7f16980a5230 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f16a8009fd0 tx=0x7f16a8005bd0 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.062+0000 7f16b5555640 1 --2- 172.21.15.1:0/803706642 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561f3b877410 0x561f3b881770 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f16a0031550 tx=0x7f16a0032bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de351aed6824c592 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.063+0000 7f16977fe640 1 -- 172.21.15.1:0/803706642 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a003f070 con 0x561f3b877410 2024-09-11T22:04:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.063+0000 7f16977fe640 1 -- 172.21.15.1:0/803706642 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16a0004030 con 0x561f3b877410 2024-09-11T22:04:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.063+0000 7f16977fe640 1 -- 172.21.15.1:0/803706642 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a0035070 con 0x561f3b877410 2024-09-11T22:04:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.063+0000 7f16ba627740 1 -- 172.21.15.1:0/803706642 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561f3b877410 msgr2=0x561f3b881770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.063+0000 7f16ba627740 1 --2- 172.21.15.1:0/803706642 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561f3b877410 0x561f3b881770 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f16a0031550 tx=0x7f16a0032bf0 comp rx=0 tx=0).stop 2024-09-11T22:04:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.063+0000 7f16ba627740 1 -- 172.21.15.1:0/803706642 shutdown_connections 2024-09-11T22:04:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.063+0000 7f16ba627740 1 --2- 172.21.15.1:0/803706642 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f3b881cb0 0x561f3b8840a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.063+0000 7f16ba627740 1 --2- 172.21.15.1:0/803706642 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561f3b877410 0x561f3b881770 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.063+0000 7f16ba627740 1 --2- 172.21.15.1:0/803706642 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561f3b876af0 0x561f3b876ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.063+0000 7f16ba627740 1 -- 172.21.15.1:0/803706642 >> 172.21.15.1:0/803706642 conn(0x561f3b873060 msgr2=0x561f3b873440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.063+0000 7f16ba627740 1 -- 172.21.15.1:0/803706642 shutdown_connections 2024-09-11T22:04:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.063+0000 7f16ba627740 1 -- 172.21.15.1:0/803706642 wait complete. 2024-09-11T22:04:27.072 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.064+0000 7f16ba627740 1 -- start start 2024-09-11T22:04:27.072 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.064+0000 7f16ba627740 1 Processor -- start 2024-09-11T22:04:27.072 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.064+0000 7f16ba627740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f3b876af0 0x561f3b87eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:27.072 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.064+0000 7f16ba627740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561f3b877410 0x561f3b87f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:27.072 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.064+0000 7f16ba627740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561f3b881cb0 0x561f3b87f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:27.072 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.064+0000 7f16ba627740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x561f3b884880 con 0x561f3b881cb0 2024-09-11T22:04:27.072 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.064+0000 7f16ba627740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x561f3b764290 con 0x561f3b876af0 2024-09-11T22:04:27.072 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.064+0000 7f16ba627740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x561f3b884a00 con 0x561f3b877410 2024-09-11T22:04:27.072 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.065+0000 7f16ba627740 10 client.0 _ll_register_callbacks cb 0x561f3b795140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:04:27.072 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.065+0000 7f16b5d56640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561f3b881cb0 0x561f3b87f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:27.072 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.065+0000 7f16b5d56640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561f3b881cb0 0x561f3b87f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:32886/0 (socket says 172.21.15.1:32886) 2024-09-11T22:04:27.072 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.065+0000 7f16b5d56640 1 -- 172.21.15.1:0/3056068749 learned_addr learned my addr 172.21.15.1:0/3056068749 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:27.073 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.065+0000 7f16b5555640 1 --2- 172.21.15.1:0/3056068749 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f3b876af0 0x561f3b87eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:27.073 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.065+0000 7f16b4d54640 1 --2- 172.21.15.1:0/3056068749 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561f3b877410 0x561f3b87f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:27.073 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.065+0000 7f16b5d56640 1 -- 172.21.15.1:0/3056068749 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561f3b877410 msgr2=0x561f3b87f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:27.073 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.065+0000 7f16b5d56640 1 --2- 172.21.15.1:0/3056068749 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561f3b877410 0x561f3b87f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:27.073 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.065+0000 7f16b5d56640 1 -- 172.21.15.1:0/3056068749 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f3b876af0 msgr2=0x561f3b87eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:27.073 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.065+0000 7f16b5d56640 1 --2- 172.21.15.1:0/3056068749 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f3b876af0 0x561f3b87eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:27.073 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.065+0000 7f16b5d56640 1 -- 172.21.15.1:0/3056068749 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f3b4ee210 con 0x561f3b881cb0 2024-09-11T22:04:27.073 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.065+0000 7f16b5d56640 1 --2- 172.21.15.1:0/3056068749 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561f3b881cb0 0x561f3b87f610 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f16a8012f90 tx=0x7f16a8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4be188e2583ecd68 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:27.073 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.065+0000 7f1694ff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:04:27.073 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.065+0000 7f1694ff9640 1 -- 172.21.15.1:0/3056068749 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a8025070 con 0x561f3b881cb0 2024-09-11T22:04:27.073 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.065+0000 7f1694ff9640 1 -- 172.21.15.1:0/3056068749 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7f16a800fc00 con 0x561f3b881cb0 2024-09-11T22:04:27.073 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.065+0000 7f16ba627740 1 -- 172.21.15.1:0/3056068749 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16a0004030 con 0x561f3b881cb0 2024-09-11T22:04:27.073 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.065+0000 7f1694ff9640 1 -- 172.21.15.1:0/3056068749 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f16a800fe40 con 0x561f3b881cb0 2024-09-11T22:04:27.074 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.065+0000 7f16ba627740 10 client.17034 fetch_fsmap learned FSMap version 42 2024-09-11T22:04:27.074 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.065+0000 7f16ba627740 1 -- 172.21.15.1:0/3056068749 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x561f3b5ba620 con 0x561f3b881cb0 2024-09-11T22:04:27.074 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.066+0000 7f1694ff9640 1 -- 172.21.15.1:0/3056068749 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16a0004030 con 0x561f3b881cb0 2024-09-11T22:04:27.074 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.066+0000 7f1694ff9640 1 -- 172.21.15.1:0/3056068749 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f16a801e040 con 0x561f3b881cb0 2024-09-11T22:04:27.074 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.066+0000 7f16ba627740 10 client.17034 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:04:27.074 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.066+0000 7f16ba627740 10 client.17034 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:04:27.074 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.066+0000 7f16ba627740 20 client.17034 populate_metadata read hostname 'smithi001' 2024-09-11T22:04:27.074 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.066+0000 7f16ba627740 1 --2- 172.21.15.1:0/3056068749 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x561f3b891cf0 0x561f3b894170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:27.074 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.066+0000 7f16ba627740 4 client.17034 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:04:27.074 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.066+0000 7f16ba627740 1 -- 172.21.15.1:0/3056068749 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f3b7e8a20 con 0x561f3b891cf0 2024-09-11T22:04:27.074 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.066+0000 7f16b5555640 1 --2- 172.21.15.1:0/3056068749 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x561f3b891cf0 0x561f3b894170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:27.074 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.067+0000 7f16b5555640 1 --2- 172.21.15.1:0/3056068749 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x561f3b891cf0 0x561f3b894170 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ac612a138cb7b4e server_cookie=f953b4c5e7b7651e in_seq=0 out_seq=0 2024-09-11T22:04:27.074 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.067+0000 7f1694ff9640 10 client.17034 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:04:27.076 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.067+0000 7f1694ff9640 1 -- 172.21.15.1:0/3056068749 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561f3b7e8a20 con 0x561f3b891cf0 2024-09-11T22:04:27.076 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.067+0000 7f1694ff9640 10 client.17034 handle_command_reply: tid=0 2024-09-11T22:04:27.076 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.068+0000 7f16ba627740 1 client.17034 shutdown 2024-09-11T22:04:27.076 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.068+0000 7f16ba627740 1 -- 172.21.15.1:0/3056068749 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561f3b881cb0 msgr2=0x561f3b87f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:27.076 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.068+0000 7f16ba627740 1 --2- 172.21.15.1:0/3056068749 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561f3b881cb0 0x561f3b87f610 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f16a8012f90 tx=0x7f16a8004ab0 comp rx=0 tx=0).stop 2024-09-11T22:04:27.076 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.068+0000 7f16ba627740 1 -- 172.21.15.1:0/3056068749 shutdown_connections 2024-09-11T22:04:27.076 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.068+0000 7f16ba627740 1 --2- 172.21.15.1:0/3056068749 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x561f3b891cf0 0x561f3b894170 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:27.076 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.068+0000 7f16ba627740 1 --2- 172.21.15.1:0/3056068749 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561f3b881cb0 0x561f3b87f610 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:27.076 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.068+0000 7f16ba627740 1 --2- 172.21.15.1:0/3056068749 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561f3b877410 0x561f3b87f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:27.076 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.068+0000 7f16ba627740 1 --2- 172.21.15.1:0/3056068749 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f3b876af0 0x561f3b87eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:27.076 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.068+0000 7f1694ff9640 0 client.17034 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:04:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.068+0000 7f16ba627740 1 -- 172.21.15.1:0/3056068749 >> 172.21.15.1:0/3056068749 conn(0x561f3b873060 msgr2=0x561f3b8721c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.068+0000 7f16ba627740 1 -- 172.21.15.1:0/3056068749 shutdown_connections 2024-09-11T22:04:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.068+0000 7f16ba627740 1 -- 172.21.15.1:0/3056068749 wait complete. 2024-09-11T22:04:27.077 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.068+0000 7f16ba627740 20 client.17034 trim_cache size 0 max 16384 2024-09-11T22:04:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.083+0000 7f16ba627740 1 Processor -- start 2024-09-11T22:04:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.083+0000 7f16ba627740 1 -- start start 2024-09-11T22:04:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.083+0000 7f16ba627740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561f3b876af0 0x561f3b87f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.083+0000 7f16ba627740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f3b877410 0x561f3b899a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.083+0000 7f16ba627740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561f3b881cb0 0x561f3b899fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.083+0000 7f16ba627740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f16b01afc40 con 0x561f3b881cb0 2024-09-11T22:04:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.083+0000 7f16ba627740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f16b01b1a80 con 0x561f3b877410 2024-09-11T22:04:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.083+0000 7f16ba627740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f16b01b03d0 con 0x561f3b876af0 2024-09-11T22:04:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.083+0000 7f16b5d56640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561f3b881cb0 0x561f3b899fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.083+0000 7f16b5d56640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561f3b881cb0 0x561f3b899fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:32894/0 (socket says 172.21.15.1:32894) 2024-09-11T22:04:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.083+0000 7f16b5d56640 1 -- 172.21.15.1:0/2490743069 learned_addr learned my addr 172.21.15.1:0/2490743069 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.083+0000 7f16b5d56640 1 -- 172.21.15.1:0/2490743069 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561f3b876af0 msgr2=0x561f3b87f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.083+0000 7f16b5d56640 1 --2- 172.21.15.1:0/2490743069 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561f3b876af0 0x561f3b87f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.083+0000 7f16b5d56640 1 -- 172.21.15.1:0/2490743069 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f3b877410 msgr2=0x561f3b899a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.083+0000 7f16b5d56640 1 --2- 172.21.15.1:0/2490743069 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f3b877410 0x561f3b899a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.083+0000 7f16b5d56640 1 -- 172.21.15.1:0/2490743069 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f3b876870 con 0x561f3b881cb0 2024-09-11T22:04:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.083+0000 7f16b5d56640 1 --2- 172.21.15.1:0/2490743069 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561f3b881cb0 0x561f3b899fc0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f16a8004a80 tx=0x7f16a8009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5abdedb86b8d8482 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:27.087 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.083+0000 7f1696ffd640 1 -- 172.21.15.1:0/2490743069 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a8036070 con 0x561f3b881cb0 2024-09-11T22:04:27.087 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.084+0000 7f1696ffd640 1 -- 172.21.15.1:0/2490743069 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16a8012980 con 0x561f3b881cb0 2024-09-11T22:04:27.087 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.084+0000 7f1696ffd640 1 -- 172.21.15.1:0/2490743069 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a8025070 con 0x561f3b881cb0 2024-09-11T22:04:27.090 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.084+0000 7f16ba627740 1 -- 172.21.15.1:0/2490743069 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561f3b881cb0 msgr2=0x561f3b899fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:27.090 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.084+0000 7f16ba627740 1 --2- 172.21.15.1:0/2490743069 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561f3b881cb0 0x561f3b899fc0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f16a8004a80 tx=0x7f16a8009c20 comp rx=0 tx=0).stop 2024-09-11T22:04:27.090 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.084+0000 7f16ba627740 1 -- 172.21.15.1:0/2490743069 shutdown_connections 2024-09-11T22:04:27.091 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.084+0000 7f16ba627740 1 --2- 172.21.15.1:0/2490743069 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561f3b881cb0 0x561f3b899fc0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:27.091 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.084+0000 7f16ba627740 1 --2- 172.21.15.1:0/2490743069 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f3b877410 0x561f3b899a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:27.091 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.084+0000 7f16ba627740 1 --2- 172.21.15.1:0/2490743069 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561f3b876af0 0x561f3b87f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:27.091 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.084+0000 7f16ba627740 1 -- 172.21.15.1:0/2490743069 >> 172.21.15.1:0/2490743069 conn(0x561f3b873060 msgr2=0x561f3b880c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:27.091 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.084+0000 7f16ba627740 1 -- 172.21.15.1:0/2490743069 shutdown_connections 2024-09-11T22:04:27.091 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.084+0000 7f16ba627740 1 -- 172.21.15.1:0/2490743069 wait complete. 2024-09-11T22:04:27.091 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.084+0000 7f16ba627740 1 -- start start 2024-09-11T22:04:27.091 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.085+0000 7f16ba627740 1 Processor -- start 2024-09-11T22:04:27.091 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.085+0000 7f16ba627740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561f3b876af0 0x561f3b885990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:27.091 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.085+0000 7f16ba627740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561f3b877410 0x561f3b885ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:27.091 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.085+0000 7f16ba627740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f3b881cb0 0x561f3b886410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:27.091 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.085+0000 7f16ba627740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f16b01b12f0 con 0x561f3b876af0 2024-09-11T22:04:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.085+0000 7f16ba627740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f16b01b0b60 con 0x561f3b881cb0 2024-09-11T22:04:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.085+0000 7f16ba627740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f16b01adc40 con 0x561f3b877410 2024-09-11T22:04:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.085+0000 7f16ba627740 10 client.0 _ll_register_callbacks cb 0x7f16a8009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:04:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.085+0000 7f16b5555640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561f3b876af0 0x561f3b885990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.085+0000 7f16b5555640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561f3b876af0 0x561f3b885990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:32902/0 (socket says 172.21.15.1:32902) 2024-09-11T22:04:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.085+0000 7f16b5555640 1 -- 172.21.15.1:0/3389408592 learned_addr learned my addr 172.21.15.1:0/3389408592 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.085+0000 7f16b5555640 1 -- 172.21.15.1:0/3389408592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561f3b877410 msgr2=0x561f3b885ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.085+0000 7f16b5d56640 1 --2- 172.21.15.1:0/3389408592 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f3b881cb0 0x561f3b886410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.085+0000 7f16b4d54640 1 --2- 172.21.15.1:0/3389408592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561f3b877410 0x561f3b885ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.085+0000 7f16b5555640 1 --2- 172.21.15.1:0/3389408592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561f3b877410 0x561f3b885ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.085+0000 7f16b5555640 1 -- 172.21.15.1:0/3389408592 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f3b881cb0 msgr2=0x561f3b886410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.085+0000 7f16b5555640 1 --2- 172.21.15.1:0/3389408592 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f3b881cb0 0x561f3b886410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.085+0000 7f16b5555640 1 -- 172.21.15.1:0/3389408592 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f3b8fba50 con 0x561f3b876af0 2024-09-11T22:04:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.085+0000 7f16b4d54640 1 --2- 172.21.15.1:0/3389408592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561f3b877410 0x561f3b885ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:04:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.085+0000 7f16b5d56640 1 --2- 172.21.15.1:0/3389408592 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f3b881cb0 0x561f3b886410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:04:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.086+0000 7f16b5555640 1 --2- 172.21.15.1:0/3389408592 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561f3b876af0 0x561f3b885990 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f16a0031550 tx=0x7f16a00058f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45ca4619ae589e0e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.086+0000 7f1694ff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:04:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.086+0000 7f1694ff9640 1 -- 172.21.15.1:0/3389408592 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a0035070 con 0x561f3b876af0 2024-09-11T22:04:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.086+0000 7f1694ff9640 1 -- 172.21.15.1:0/3389408592 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7f16a0004460 con 0x561f3b876af0 2024-09-11T22:04:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.086+0000 7f16ba627740 1 -- 172.21.15.1:0/3389408592 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16a8012980 con 0x561f3b876af0 2024-09-11T22:04:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.086+0000 7f1694ff9640 1 -- 172.21.15.1:0/3389408592 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f16a005d050 con 0x561f3b876af0 2024-09-11T22:04:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.086+0000 7f16ba627740 10 client.17040 fetch_fsmap learned FSMap version 42 2024-09-11T22:04:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.086+0000 7f16ba627740 1 -- 172.21.15.1:0/3389408592 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x561f3b8fccf0 con 0x561f3b876af0 2024-09-11T22:04:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.086+0000 7f1694ff9640 1 -- 172.21.15.1:0/3389408592 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16a8012980 con 0x561f3b876af0 2024-09-11T22:04:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.086+0000 7f1694ff9640 1 -- 172.21.15.1:0/3389408592 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f16a00316d0 con 0x561f3b876af0 2024-09-11T22:04:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.086+0000 7f16ba627740 10 client.17040 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:04:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.086+0000 7f16ba627740 10 client.17040 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:04:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.086+0000 7f16ba627740 20 client.17040 populate_metadata read hostname 'smithi001' 2024-09-11T22:04:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.087+0000 7f16ba627740 1 --2- 172.21.15.1:0/3389408592 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x561f3b8fd0f0 0x561f3b8ff4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.087+0000 7f16ba627740 4 client.17040 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:04:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.087+0000 7f16ba627740 1 -- 172.21.15.1:0/3389408592 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561f3b7e2d70 con 0x561f3b8fd0f0 2024-09-11T22:04:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.087+0000 7f16b4d54640 1 --2- 172.21.15.1:0/3389408592 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x561f3b8fd0f0 0x561f3b8ff4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.087+0000 7f16b4d54640 1 --2- 172.21.15.1:0/3389408592 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x561f3b8fd0f0 0x561f3b8ff4e0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b72b6625a7d3c210 server_cookie=df4a086e6dba70f8 in_seq=0 out_seq=0 2024-09-11T22:04:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.087+0000 7f1694ff9640 10 client.17040 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:04:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.222+0000 7f1694ff9640 1 -- 172.21.15.1:0/3389408592 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561f3b7e2d70 con 0x561f3b8fd0f0 2024-09-11T22:04:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.222+0000 7f1694ff9640 10 client.17040 handle_command_reply: tid=0 2024-09-11T22:04:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.222+0000 7f16ba627740 1 client.17040 shutdown 2024-09-11T22:04:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.222+0000 7f16ba627740 1 -- 172.21.15.1:0/3389408592 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561f3b876af0 msgr2=0x561f3b885990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.222+0000 7f16ba627740 1 --2- 172.21.15.1:0/3389408592 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561f3b876af0 0x561f3b885990 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f16a0031550 tx=0x7f16a00058f0 comp rx=0 tx=0).stop 2024-09-11T22:04:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.223+0000 7f16ba627740 1 -- 172.21.15.1:0/3389408592 shutdown_connections 2024-09-11T22:04:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.223+0000 7f16ba627740 1 --2- 172.21.15.1:0/3389408592 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x561f3b8fd0f0 0x561f3b8ff4e0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.223+0000 7f16ba627740 1 --2- 172.21.15.1:0/3389408592 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561f3b881cb0 0x561f3b886410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.223+0000 7f1694ff9640 0 client.17040 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:04:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.223+0000 7f16ba627740 1 --2- 172.21.15.1:0/3389408592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561f3b877410 0x561f3b885ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.223+0000 7f16ba627740 1 --2- 172.21.15.1:0/3389408592 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561f3b876af0 0x561f3b885990 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.223+0000 7f16ba627740 1 -- 172.21.15.1:0/3389408592 >> 172.21.15.1:0/3389408592 conn(0x561f3b873060 msgr2=0x561f3b878550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.223+0000 7f16ba627740 1 -- 172.21.15.1:0/3389408592 shutdown_connections 2024-09-11T22:04:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.223+0000 7f16ba627740 1 -- 172.21.15.1:0/3389408592 wait complete. 2024-09-11T22:04:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.223+0000 7f16ba627740 20 client.17040 trim_cache size 0 max 16384 2024-09-11T22:04:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-09-11T22:04:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-09-11T22:04:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-09-11T22:04:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-09-11T22:04:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.223+0000 7f16b77b9640 1 -- 172.21.15.1:0/1166434655 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f16980a2d70 msgr2=0x7f16980a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.223+0000 7f16b77b9640 1 --2- 172.21.15.1:0/1166434655 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f16980a2d70 0x7f16980a5230 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f16a8009fd0 tx=0x7f16a8005bd0 comp rx=0 tx=0).stop 2024-09-11T22:04:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.223+0000 7f16b77b9640 1 -- 172.21.15.1:0/1166434655 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16b0103640 msgr2=0x7f16b0100eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.223+0000 7f16b77b9640 1 --2- 172.21.15.1:0/1166434655 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16b0103640 0x7f16b0100eb0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f16a400e9f0 tx=0x7f16a400eeb0 comp rx=0 tx=0).stop 2024-09-11T22:04:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.223+0000 7f16b77b9640 1 -- 172.21.15.1:0/1166434655 shutdown_connections 2024-09-11T22:04:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.223+0000 7f16b77b9640 1 --2- 172.21.15.1:0/1166434655 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f16980a2d70 0x7f16980a5230 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.223+0000 7f16b77b9640 1 --2- 172.21.15.1:0/1166434655 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16b010c0f0 0x7f16b01013f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.223+0000 7f16b77b9640 1 --2- 172.21.15.1:0/1166434655 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16b0103640 0x7f16b0100eb0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.223+0000 7f16b77b9640 1 --2- 172.21.15.1:0/1166434655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f16b0102d20 0x7f16b0100970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.223+0000 7f16b77b9640 1 -- 172.21.15.1:0/1166434655 >> 172.21.15.1:0/1166434655 conn(0x7f16b00779a0 msgr2=0x7f16b00fd170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.224+0000 7f16b77b9640 1 -- 172.21.15.1:0/1166434655 shutdown_connections 2024-09-11T22:04:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:27.224+0000 7f16b77b9640 1 -- 172.21.15.1:0/1166434655 wait complete. 2024-09-11T22:04:27.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:04:26 smithi053 ceph-mon[34908]: pgmap v223: 97 pgs: 97 active+clean; 443 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 3.8 MiB/s wr, 684 op/s 2024-09-11T22:04:27.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:04:26 smithi161 ceph-mon[35615]: pgmap v223: 97 pgs: 97 active+clean; 443 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 3.8 MiB/s wr, 684 op/s 2024-09-11T22:04:27.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:04:26 smithi001 ceph-mon[30935]: pgmap v223: 97 pgs: 97 active+clean; 443 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 3.8 MiB/s wr, 684 op/s 2024-09-11T22:04:29.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:04:28 smithi053 ceph-mon[34908]: pgmap v224: 97 pgs: 97 active+clean; 443 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.3 MiB/s wr, 457 op/s 2024-09-11T22:04:29.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:04:28 smithi161 ceph-mon[35615]: pgmap v224: 97 pgs: 97 active+clean; 443 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.3 MiB/s wr, 457 op/s 2024-09-11T22:04:29.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:04:28 smithi001 ceph-mon[30935]: pgmap v224: 97 pgs: 97 active+clean; 443 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.3 MiB/s wr, 457 op/s 2024-09-11T22:04:31.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:04:30 smithi053 ceph-mon[34908]: pgmap v225: 97 pgs: 97 active+clean; 444 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.4 MiB/s wr, 558 op/s 2024-09-11T22:04:31.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:04:30 smithi161 ceph-mon[35615]: pgmap v225: 97 pgs: 97 active+clean; 444 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.4 MiB/s wr, 558 op/s 2024-09-11T22:04:31.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:04:30 smithi001 ceph-mon[30935]: pgmap v225: 97 pgs: 97 active+clean; 444 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.4 MiB/s wr, 558 op/s 2024-09-11T22:04:33.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:04:32 smithi053 ceph-mon[34908]: pgmap v226: 97 pgs: 97 active+clean; 444 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 930 KiB/s wr, 440 op/s 2024-09-11T22:04:33.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:04:32 smithi161 ceph-mon[35615]: pgmap v226: 97 pgs: 97 active+clean; 444 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 930 KiB/s wr, 440 op/s 2024-09-11T22:04:33.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:04:32 smithi001 ceph-mon[30935]: pgmap v226: 97 pgs: 97 active+clean; 444 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 930 KiB/s wr, 440 op/s 2024-09-11T22:04:35.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:04:34 smithi053 ceph-mon[34908]: pgmap v227: 97 pgs: 97 active+clean; 444 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 930 KiB/s wr, 440 op/s 2024-09-11T22:04:35.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:04:34 smithi161 ceph-mon[35615]: pgmap v227: 97 pgs: 97 active+clean; 444 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 930 KiB/s wr, 440 op/s 2024-09-11T22:04:35.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:04:34 smithi001 ceph-mon[30935]: pgmap v227: 97 pgs: 97 active+clean; 444 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 930 KiB/s wr, 440 op/s 2024-09-11T22:04:36.098 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:04:36.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:04:35 smithi053 ceph-mon[34908]: scrub summary: idle 2024-09-11T22:04:36.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:04:35 smithi053 ceph-mon[34908]: scrub summary: idle 2024-09-11T22:04:36.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:04:35 smithi161 ceph-mon[35615]: scrub summary: idle 2024-09-11T22:04:36.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:04:35 smithi161 ceph-mon[35615]: scrub summary: idle 2024-09-11T22:04:36.373 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:04:36.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:04:35 smithi001 ceph-mon[30935]: scrub summary: idle 2024-09-11T22:04:36.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:04:35 smithi001 ceph-mon[30935]: scrub summary: idle 2024-09-11T22:04:36.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:36.856+0000 7f818db1f640 1 -- 172.21.15.1:0/2551055381 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81881082a0 msgr2=0x7f818810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:36.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:36.856+0000 7f818db1f640 1 --2- 172.21.15.1:0/2551055381 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81881082a0 0x7f818810ec40 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f8180009cd0 tx=0x7f818002a8e0 comp rx=0 tx=0).stop 2024-09-11T22:04:36.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:36.856+0000 7f818db1f640 1 -- 172.21.15.1:0/2551055381 shutdown_connections 2024-09-11T22:04:36.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:36.856+0000 7f818db1f640 1 --2- 172.21.15.1:0/2551055381 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81881082a0 0x7f818810ec40 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:36.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:36.856+0000 7f818db1f640 1 --2- 172.21.15.1:0/2551055381 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81881036e0 0x7f8188107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:36.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:36.856+0000 7f818db1f640 1 --2- 172.21.15.1:0/2551055381 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8188102d30 0x7f8188103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:36.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:36.856+0000 7f818db1f640 1 -- 172.21.15.1:0/2551055381 >> 172.21.15.1:0/2551055381 conn(0x7f81880fc7d0 msgr2=0x7f81880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:36.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:36.856+0000 7f818db1f640 1 -- 172.21.15.1:0/2551055381 shutdown_connections 2024-09-11T22:04:36.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:36.856+0000 7f818db1f640 1 -- 172.21.15.1:0/2551055381 wait complete. 2024-09-11T22:04:36.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:36.857+0000 7f818db1f640 1 Processor -- start 2024-09-11T22:04:36.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:36.857+0000 7f818db1f640 1 -- start start 2024-09-11T22:04:36.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:36.857+0000 7f818db1f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8188102d30 0x7f8188198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:36.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:36.857+0000 7f818db1f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81881036e0 0x7f8188198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:36.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:36.857+0000 7f818db1f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81881082a0 0x7f818819cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:36.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:36.857+0000 7f818db1f640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f81881120a0 con 0x7f81881036e0 2024-09-11T22:04:36.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:36.857+0000 7f818db1f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8188111f20 con 0x7f81881082a0 2024-09-11T22:04:36.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:36.857+0000 7f818db1f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f8188112220 con 0x7f8188102d30 2024-09-11T22:04:36.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:36.857+0000 7f8186ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81881036e0 0x7f8188198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:36.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:36.857+0000 7f81877fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8188102d30 0x7f8188198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:36.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:36.857+0000 7f81877fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8188102d30 0x7f8188198510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:56956/0 (socket says 172.21.15.1:56956) 2024-09-11T22:04:36.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:36.857+0000 7f8186ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81881036e0 0x7f8188198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47432/0 (socket says 172.21.15.1:47432) 2024-09-11T22:04:36.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:36.857+0000 7f81877fe640 1 -- 172.21.15.1:0/3359193384 learned_addr learned my addr 172.21.15.1:0/3359193384 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:36.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:36.857+0000 7f8187fff640 1 --2- 172.21.15.1:0/3359193384 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81881082a0 0x7f818819cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:36.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:36.858+0000 7f81877fe640 1 -- 172.21.15.1:0/3359193384 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81881082a0 msgr2=0x7f818819cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:36.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:36.858+0000 7f81877fe640 1 --2- 172.21.15.1:0/3359193384 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81881082a0 0x7f818819cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:36.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:36.858+0000 7f81877fe640 1 -- 172.21.15.1:0/3359193384 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81881036e0 msgr2=0x7f8188198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:36.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:36.858+0000 7f81877fe640 1 --2- 172.21.15.1:0/3359193384 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81881036e0 0x7f8188198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:36.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:36.858+0000 7f81877fe640 1 -- 172.21.15.1:0/3359193384 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f818819d4c0 con 0x7f8188102d30 2024-09-11T22:04:36.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:36.858+0000 7f8186ffd640 1 --2- 172.21.15.1:0/3359193384 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81881036e0 0x7f8188198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:04:36.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:36.858+0000 7f81877fe640 1 --2- 172.21.15.1:0/3359193384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8188102d30 0x7f8188198510 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f817800ea60 tx=0x7f817800ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:36.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:36.858+0000 7f8184ff9640 1 -- 172.21.15.1:0/3359193384 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f817800cde0 con 0x7f8188102d30 2024-09-11T22:04:36.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:36.858+0000 7f818db1f640 1 -- 172.21.15.1:0/3359193384 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f818819d7b0 con 0x7f8188102d30 2024-09-11T22:04:36.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:36.858+0000 7f8184ff9640 1 -- 172.21.15.1:0/3359193384 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81780035f0 con 0x7f8188102d30 2024-09-11T22:04:36.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:36.858+0000 7f8184ff9640 1 -- 172.21.15.1:0/3359193384 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8178010670 con 0x7f8188102d30 2024-09-11T22:04:36.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:36.858+0000 7f818db1f640 1 -- 172.21.15.1:0/3359193384 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81881a5090 con 0x7f8188102d30 2024-09-11T22:04:36.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:36.859+0000 7f818db1f640 1 -- 172.21.15.1:0/3359193384 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8188103110 con 0x7f8188102d30 2024-09-11T22:04:36.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:36.861+0000 7f8184ff9640 1 -- 172.21.15.1:0/3359193384 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f81780037a0 con 0x7f8188102d30 2024-09-11T22:04:36.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:36.862+0000 7f8184ff9640 1 --2- 172.21.15.1:0/3359193384 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f81700a2cf0 0x7f81700a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:36.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:36.862+0000 7f8184ff9640 1 -- 172.21.15.1:0/3359193384 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7f81780cf750 con 0x7f8188102d30 2024-09-11T22:04:36.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:36.862+0000 7f8186ffd640 1 --2- 172.21.15.1:0/3359193384 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f81700a2cf0 0x7f81700a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:36.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:36.863+0000 7f8186ffd640 1 --2- 172.21.15.1:0/3359193384 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f81700a2cf0 0x7f81700a51b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f81880fedf0 tx=0x7f817c005f90 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:36.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:36.864+0000 7f8184ff9640 1 -- 172.21.15.1:0/3359193384 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f817809c300 con 0x7f8188102d30 2024-09-11T22:04:37.063 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:04:36 smithi001 ceph-mon[30935]: scrub summary: idle 2024-09-11T22:04:37.063 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:04:36 smithi001 ceph-mon[30935]: scrub summary: idle 2024-09-11T22:04:37.063 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:04:36 smithi001 ceph-mon[30935]: scrub summary: idle 2024-09-11T22:04:37.063 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:04:36 smithi001 ceph-mon[30935]: pgmap v228: 97 pgs: 97 active+clean; 444 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 933 KiB/s wr, 445 op/s 2024-09-11T22:04:37.063 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:04:36 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:04:37.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:37.082+0000 7f818db1f640 1 -- 172.21.15.1:0/3359193384 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8188199870 con 0x7f8188102d30 2024-09-11T22:04:37.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:37.083+0000 7f8184ff9640 1 -- 172.21.15.1:0/3359193384 <== mon.1 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11984 (secure 0 0 0) 0x7f8188199870 con 0x7f8188102d30 2024-09-11T22:04:37.085 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T22:04:37.086 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":42,"btime":"2024-09-11T22:03:27:040852+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/421542832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":421542832},{"type":"v1","addr":"172.21.15.1:6835","nonce":421542832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2836864406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2836864406},{"type":"v1","addr":"172.21.15.1:6837","nonce":2836864406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":42,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T21:55:20.791263+0000","modified":"2024-09-11T22:03:26.485134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24484,"mds_2":24467,"mds_3":24466,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14595":{"gid":14595,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/407834672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":407834672},{"type":"v1","addr":"172.21.15.1:6839","nonce":407834672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/1068827678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":1068827678},{"type":"v1","addr":"172.21.15.1:6841","nonce":1068827678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/872410024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":872410024},{"type":"v1","addr":"172.21.15.161:6833","nonce":872410024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/1648944570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1648944570},{"type":"v1","addr":"172.21.15.53:6833","nonce":1648944570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/2126933993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2126933993},{"type":"v1","addr":"172.21.15.53:6835","nonce":2126933993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.161:6835/828381592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":828381592},{"type":"v1","addr":"172.21.15.161:6835","nonce":828381592}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.161:6837/1799318656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1799318656},{"type":"v1","addr":"172.21.15.161:6837","nonce":1799318656}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.53:6837/3457131830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3457131830},{"type":"v1","addr":"172.21.15.53:6837","nonce":3457131830}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/455365383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":455365383},{"type":"v1","addr":"172.21.15.161:6839","nonce":455365383}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.53:6839/1378584447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1378584447},{"type":"v1","addr":"172.21.15.53:6839","nonce":1378584447}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24460,24466,24467,24484,24494]},"id":1}]} 2024-09-11T22:04:37.086 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 42 2024-09-11T22:04:37.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:37.085+0000 7f818db1f640 1 -- 172.21.15.1:0/3359193384 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f81700a2cf0 msgr2=0x7f81700a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:37.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:37.085+0000 7f818db1f640 1 --2- 172.21.15.1:0/3359193384 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f81700a2cf0 0x7f81700a51b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f81880fedf0 tx=0x7f817c005f90 comp rx=0 tx=0).stop 2024-09-11T22:04:37.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:37.085+0000 7f818db1f640 1 -- 172.21.15.1:0/3359193384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8188102d30 msgr2=0x7f8188198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:37.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:37.085+0000 7f818db1f640 1 --2- 172.21.15.1:0/3359193384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8188102d30 0x7f8188198510 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f817800ea60 tx=0x7f817800ef20 comp rx=0 tx=0).stop 2024-09-11T22:04:37.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:37.086+0000 7f818db1f640 1 -- 172.21.15.1:0/3359193384 shutdown_connections 2024-09-11T22:04:37.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:37.086+0000 7f818db1f640 1 --2- 172.21.15.1:0/3359193384 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f81700a2cf0 0x7f81700a51b0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:37.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:37.086+0000 7f818db1f640 1 --2- 172.21.15.1:0/3359193384 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81881082a0 0x7f818819cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:37.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:37.086+0000 7f818db1f640 1 --2- 172.21.15.1:0/3359193384 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81881036e0 0x7f8188198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:37.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:37.086+0000 7f818db1f640 1 --2- 172.21.15.1:0/3359193384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8188102d30 0x7f8188198510 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:37.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:37.086+0000 7f818db1f640 1 -- 172.21.15.1:0/3359193384 >> 172.21.15.1:0/3359193384 conn(0x7f81880fc7d0 msgr2=0x7f8188105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:37.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:37.086+0000 7f818db1f640 1 -- 172.21.15.1:0/3359193384 shutdown_connections 2024-09-11T22:04:37.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:37.086+0000 7f818db1f640 1 -- 172.21.15.1:0/3359193384 wait complete. 2024-09-11T22:04:37.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:04:36 smithi053 ceph-mon[34908]: scrub summary: idle 2024-09-11T22:04:37.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:04:36 smithi053 ceph-mon[34908]: scrub summary: idle 2024-09-11T22:04:37.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:04:36 smithi053 ceph-mon[34908]: scrub summary: idle 2024-09-11T22:04:37.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:04:36 smithi053 ceph-mon[34908]: pgmap v228: 97 pgs: 97 active+clean; 444 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 933 KiB/s wr, 445 op/s 2024-09-11T22:04:37.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:04:36 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:04:37.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:04:36 smithi161 ceph-mon[35615]: scrub summary: idle 2024-09-11T22:04:37.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:04:36 smithi161 ceph-mon[35615]: scrub summary: idle 2024-09-11T22:04:37.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:04:36 smithi161 ceph-mon[35615]: scrub summary: idle 2024-09-11T22:04:37.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:04:36 smithi161 ceph-mon[35615]: pgmap v228: 97 pgs: 97 active+clean; 444 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 933 KiB/s wr, 445 op/s 2024-09-11T22:04:37.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:04:36 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:04:37.909 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db --set-id 42839ccd --release --await' 2024-09-11T22:04:37.909 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db --set-id 42839ccd --release --await 2024-09-11T22:04:38.173 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:04:38.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:04:37 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/3359193384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:04:38.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:04:37 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:04:38.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:04:37 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:04:38.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:04:37 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T22:04:38.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:04:37 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/3359193384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:04:38.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:04:37 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:04:38.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:04:37 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:04:38.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:04:37 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T22:04:38.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:04:37 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/3359193384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:04:38.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:04:37 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:04:38.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:04:37 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:04:38.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:04:37 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T22:04:38.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.692+0000 7ff591f37640 1 -- 172.21.15.1:0/2557729299 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff58c100350 msgr2=0x7ff58c100730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:38.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.692+0000 7ff591f37640 1 --2- 172.21.15.1:0/2557729299 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff58c100350 0x7ff58c100730 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7ff57c009930 tx=0x7ff57c02a420 comp rx=0 tx=0).stop 2024-09-11T22:04:38.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.692+0000 7ff591f37640 1 -- 172.21.15.1:0/2557729299 shutdown_connections 2024-09-11T22:04:38.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.692+0000 7ff591f37640 1 --2- 172.21.15.1:0/2557729299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff58c106f50 0x7ff58c10eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:38.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.692+0000 7ff591f37640 1 --2- 172.21.15.1:0/2557729299 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff58c100d00 0x7ff58c106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:38.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.692+0000 7ff591f37640 1 --2- 172.21.15.1:0/2557729299 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff58c100350 0x7ff58c100730 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:38.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.692+0000 7ff591f37640 1 -- 172.21.15.1:0/2557729299 >> 172.21.15.1:0/2557729299 conn(0x7ff58c077210 msgr2=0x7ff58c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:38.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.692+0000 7ff591f37640 1 -- 172.21.15.1:0/2557729299 shutdown_connections 2024-09-11T22:04:38.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.692+0000 7ff591f37640 1 -- 172.21.15.1:0/2557729299 wait complete. 2024-09-11T22:04:38.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.692+0000 7ff591f37640 1 Processor -- start 2024-09-11T22:04:38.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.693+0000 7ff591f37640 1 -- start start 2024-09-11T22:04:38.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.693+0000 7ff591f37640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff58c100350 0x7ff58c1012d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:38.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.693+0000 7ff591f37640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff58c100d00 0x7ff58c101810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:38.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.693+0000 7ff591f37640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff58c106f50 0x7ff58c101d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:38.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.693+0000 7ff591f37640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff58c112000 con 0x7ff58c106f50 2024-09-11T22:04:38.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.693+0000 7ff591f37640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff58c111e80 con 0x7ff58c100d00 2024-09-11T22:04:38.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.693+0000 7ff591f37640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff58c112180 con 0x7ff58c100350 2024-09-11T22:04:38.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.693+0000 7ff58bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff58c106f50 0x7ff58c101d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:38.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.693+0000 7ff58bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff58c106f50 0x7ff58c101d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47444/0 (socket says 172.21.15.1:47444) 2024-09-11T22:04:38.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.693+0000 7ff58bfff640 1 -- 172.21.15.1:0/3274782745 learned_addr learned my addr 172.21.15.1:0/3274782745 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:38.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.693+0000 7ff58b7fe640 1 --2- 172.21.15.1:0/3274782745 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff58c100350 0x7ff58c1012d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:38.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.693+0000 7ff58affd640 1 --2- 172.21.15.1:0/3274782745 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff58c100d00 0x7ff58c101810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:38.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.693+0000 7ff58bfff640 1 -- 172.21.15.1:0/3274782745 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff58c100350 msgr2=0x7ff58c1012d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:38.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.693+0000 7ff58bfff640 1 --2- 172.21.15.1:0/3274782745 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff58c100350 0x7ff58c1012d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:38.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.693+0000 7ff58bfff640 1 -- 172.21.15.1:0/3274782745 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff58c100d00 msgr2=0x7ff58c101810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:38.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.693+0000 7ff58bfff640 1 --2- 172.21.15.1:0/3274782745 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff58c100d00 0x7ff58c101810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:38.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.693+0000 7ff58bfff640 1 -- 172.21.15.1:0/3274782745 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff58c1a69a0 con 0x7ff58c106f50 2024-09-11T22:04:38.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.694+0000 7ff58bfff640 1 --2- 172.21.15.1:0/3274782745 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff58c106f50 0x7ff58c101d50 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7ff58400cce0 tx=0x7ff58400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:38.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.694+0000 7ff588ff9640 1 -- 172.21.15.1:0/3274782745 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff58400f450 con 0x7ff58c106f50 2024-09-11T22:04:38.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.694+0000 7ff591f37640 1 -- 172.21.15.1:0/3274782745 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff58c1a6c90 con 0x7ff58c106f50 2024-09-11T22:04:38.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.694+0000 7ff588ff9640 1 -- 172.21.15.1:0/3274782745 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff58400fad0 con 0x7ff58c106f50 2024-09-11T22:04:38.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.694+0000 7ff588ff9640 1 -- 172.21.15.1:0/3274782745 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff584018b60 con 0x7ff58c106f50 2024-09-11T22:04:38.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.694+0000 7ff591f37640 1 -- 172.21.15.1:0/3274782745 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff58c1a71d0 con 0x7ff58c106f50 2024-09-11T22:04:38.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.696+0000 7ff588ff9640 1 -- 172.21.15.1:0/3274782745 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7ff58400fc80 con 0x7ff58c106f50 2024-09-11T22:04:38.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.696+0000 7ff594310740 1 Processor -- start 2024-09-11T22:04:38.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.697+0000 7ff594310740 1 -- start start 2024-09-11T22:04:38.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.697+0000 7ff594310740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56458cdd4060 0x56458cdd4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:38.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.697+0000 7ff594310740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56458cdd4980 0x56458cdded00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:38.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.697+0000 7ff594310740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56458cddf240 0x56458cde1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:38.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.697+0000 7ff594310740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56458cd39620 con 0x56458cddf240 2024-09-11T22:04:38.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.697+0000 7ff594310740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56458cce0e50 con 0x56458cdd4980 2024-09-11T22:04:38.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.697+0000 7ff594310740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56458cd355e0 con 0x56458cdd4060 2024-09-11T22:04:38.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.697+0000 7ff588ff9640 1 --2- 172.21.15.1:0/3274782745 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7ff5740a2cf0 0x7ff5740a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:38.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.697+0000 7ff58affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56458cddf240 0x56458cde1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:38.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.697+0000 7ff58affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56458cddf240 0x56458cde1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47458/0 (socket says 172.21.15.1:47458) 2024-09-11T22:04:38.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.697+0000 7ff58affd640 1 -- 172.21.15.1:0/3819933251 learned_addr learned my addr 172.21.15.1:0/3819933251 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:38.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.697+0000 7ff58affd640 1 --2- 172.21.15.1:0/3819933251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56458cdd4060 0x56458cdd4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:38.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.697+0000 7ff58bfff640 1 --2- 172.21.15.1:0/3274782745 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7ff5740a2cf0 0x7ff5740a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:38.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.697+0000 7ff58b7fe640 1 --2- 172.21.15.1:0/3819933251 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56458cdd4980 0x56458cdded00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:38.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.697+0000 7ff588ff9640 1 -- 172.21.15.1:0/3274782745 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7ff5840cf9b0 con 0x7ff58c106f50 2024-09-11T22:04:38.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.697+0000 7ff58affd640 1 -- 172.21.15.1:0/3819933251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56458cdd4060 msgr2=0x56458cdd4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:38.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.697+0000 7ff58affd640 1 --2- 172.21.15.1:0/3819933251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56458cdd4060 0x56458cdd4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:38.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.697+0000 7ff58affd640 1 -- 172.21.15.1:0/3819933251 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56458cdd4980 msgr2=0x56458cdded00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:38.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.698+0000 7ff58affd640 1 --2- 172.21.15.1:0/3819933251 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56458cdd4980 0x56458cdded00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:38.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.698+0000 7ff58affd640 1 -- 172.21.15.1:0/3819933251 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56458ccd1d40 con 0x56458cddf240 2024-09-11T22:04:38.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.698+0000 7ff58b7fe640 1 --2- 172.21.15.1:0/3819933251 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56458cdd4980 0x56458cdded00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:04:38.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.698+0000 7ff58affd640 1 --2- 172.21.15.1:0/3819933251 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56458cddf240 0x56458cde1630 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7ff580017700 tx=0x7ff5800094d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44eca30203d271e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:38.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.698+0000 7ff58bfff640 1 --2- 172.21.15.1:0/3274782745 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7ff5740a2cf0 0x7ff5740a51b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff58400afd0 tx=0x7ff58401f4b0 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:38.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.698+0000 7ff571ffb640 1 -- 172.21.15.1:0/3819933251 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff580015760 con 0x56458cddf240 2024-09-11T22:04:38.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.698+0000 7ff571ffb640 1 -- 172.21.15.1:0/3819933251 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff58001d070 con 0x56458cddf240 2024-09-11T22:04:38.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.698+0000 7ff571ffb640 1 -- 172.21.15.1:0/3819933251 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff580018a40 con 0x56458cddf240 2024-09-11T22:04:38.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.699+0000 7ff594310740 1 -- 172.21.15.1:0/3819933251 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56458cddf240 msgr2=0x56458cde1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:38.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.699+0000 7ff594310740 1 --2- 172.21.15.1:0/3819933251 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56458cddf240 0x56458cde1630 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7ff580017700 tx=0x7ff5800094d0 comp rx=0 tx=0).stop 2024-09-11T22:04:38.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.699+0000 7ff594310740 1 -- 172.21.15.1:0/3819933251 shutdown_connections 2024-09-11T22:04:38.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.699+0000 7ff594310740 1 --2- 172.21.15.1:0/3819933251 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56458cddf240 0x56458cde1630 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:38.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.699+0000 7ff594310740 1 --2- 172.21.15.1:0/3819933251 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56458cdd4980 0x56458cdded00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:38.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.699+0000 7ff594310740 1 --2- 172.21.15.1:0/3819933251 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56458cdd4060 0x56458cdd4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:38.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.699+0000 7ff594310740 1 -- 172.21.15.1:0/3819933251 >> 172.21.15.1:0/3819933251 conn(0x56458cdd05f0 msgr2=0x56458cdd09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:38.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.699+0000 7ff594310740 1 -- 172.21.15.1:0/3819933251 shutdown_connections 2024-09-11T22:04:38.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.699+0000 7ff594310740 1 -- 172.21.15.1:0/3819933251 wait complete. 2024-09-11T22:04:38.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.699+0000 7ff594310740 1 -- start start 2024-09-11T22:04:38.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.699+0000 7ff594310740 1 Processor -- start 2024-09-11T22:04:38.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.700+0000 7ff594310740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56458cdd4060 0x56458cddc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:38.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.700+0000 7ff594310740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56458cdd4980 0x56458cddc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:38.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.700+0000 7ff594310740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56458cddf240 0x56458cddcc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:38.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.700+0000 7ff594310740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56458cde1e10 con 0x56458cdd4980 2024-09-11T22:04:38.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.700+0000 7ff594310740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56458ccc1f10 con 0x56458cdd4060 2024-09-11T22:04:38.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.700+0000 7ff594310740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56458cde1f90 con 0x56458cddf240 2024-09-11T22:04:38.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.700+0000 7ff594310740 10 client.0 _ll_register_callbacks cb 0x56458cce5db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:04:38.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.700+0000 7ff58b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56458cdd4980 0x56458cddc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:38.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.700+0000 7ff58b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56458cdd4980 0x56458cddc730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47460/0 (socket says 172.21.15.1:47460) 2024-09-11T22:04:38.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.700+0000 7ff58b7fe640 1 -- 172.21.15.1:0/1183898941 learned_addr learned my addr 172.21.15.1:0/1183898941 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:38.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.700+0000 7ff58affd640 1 --2- 172.21.15.1:0/1183898941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56458cddf240 0x56458cddcc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:38.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.700+0000 7ff58affd640 1 --2- 172.21.15.1:0/1183898941 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56458cdd4060 0x56458cddc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:38.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.700+0000 7ff58b7fe640 1 -- 172.21.15.1:0/1183898941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56458cddf240 msgr2=0x56458cddcc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:38.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.700+0000 7ff58b7fe640 1 --2- 172.21.15.1:0/1183898941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56458cddf240 0x56458cddcc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:38.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.700+0000 7ff58b7fe640 1 -- 172.21.15.1:0/1183898941 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56458cdd4060 msgr2=0x56458cddc1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:38.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.700+0000 7ff58b7fe640 1 --2- 172.21.15.1:0/1183898941 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56458cdd4060 0x56458cddc1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:38.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.700+0000 7ff58b7fe640 1 -- 172.21.15.1:0/1183898941 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56458ca48580 con 0x56458cdd4980 2024-09-11T22:04:38.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.700+0000 7ff58affd640 1 --2- 172.21.15.1:0/1183898941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56458cddf240 0x56458cddcc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:04:38.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.700+0000 7ff58b7fe640 1 --2- 172.21.15.1:0/1183898941 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56458cdd4980 0x56458cddc730 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7ff57c009930 tx=0x7ff57c0319a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e415a2b300ac0f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:38.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.700+0000 7ff56affd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:04:38.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.700+0000 7ff56affd640 1 -- 172.21.15.1:0/1183898941 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57c035040 con 0x56458cdd4980 2024-09-11T22:04:38.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.701+0000 7ff594310740 1 -- 172.21.15.1:0/1183898941 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff58001d070 con 0x56458cdd4980 2024-09-11T22:04:38.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.701+0000 7ff56affd640 1 -- 172.21.15.1:0/1183898941 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7ff57c004300 con 0x56458cdd4980 2024-09-11T22:04:38.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.701+0000 7ff56affd640 1 -- 172.21.15.1:0/1183898941 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7ff57c005020 con 0x56458cdd4980 2024-09-11T22:04:38.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.701+0000 7ff594310740 10 client.17058 fetch_fsmap learned FSMap version 42 2024-09-11T22:04:38.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.701+0000 7ff594310740 1 -- 172.21.15.1:0/1183898941 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x56458cb12960 con 0x56458cdd4980 2024-09-11T22:04:38.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.701+0000 7ff56affd640 1 -- 172.21.15.1:0/1183898941 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff58001d070 con 0x56458cdd4980 2024-09-11T22:04:38.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.701+0000 7ff56affd640 1 -- 172.21.15.1:0/1183898941 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7ff57c0035f0 con 0x56458cdd4980 2024-09-11T22:04:38.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.701+0000 7ff594310740 10 client.17058 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:04:38.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.701+0000 7ff594310740 10 client.17058 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-11T22:04:38.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.701+0000 7ff594310740 20 client.17058 populate_metadata read hostname 'smithi001' 2024-09-11T22:04:38.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.701+0000 7ff594310740 1 --2- 172.21.15.1:0/1183898941 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x56458cdeeec0 0x56458cdf1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:38.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.701+0000 7ff594310740 4 client.17058 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:04:38.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.701+0000 7ff594310740 1 -- 172.21.15.1:0/1183898941 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56458cd46b90 con 0x56458cdeeec0 2024-09-11T22:04:38.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.701+0000 7ff58affd640 1 --2- 172.21.15.1:0/1183898941 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x56458cdeeec0 0x56458cdf1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:38.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.702+0000 7ff58affd640 1 --2- 172.21.15.1:0/1183898941 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x56458cdeeec0 0x56458cdf1380 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=660d4008479a4b3c server_cookie=86b73399305e8af4 in_seq=0 out_seq=0 2024-09-11T22:04:38.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.702+0000 7ff56affd640 10 client.17058 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:04:38.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.702+0000 7ff56affd640 1 -- 172.21.15.1:0/1183898941 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56458cd46b90 con 0x56458cdeeec0 2024-09-11T22:04:38.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.702+0000 7ff56affd640 10 client.17058 handle_command_reply: tid=0 2024-09-11T22:04:38.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.703+0000 7ff594310740 1 client.17058 shutdown 2024-09-11T22:04:38.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.703+0000 7ff594310740 1 -- 172.21.15.1:0/1183898941 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56458cdd4980 msgr2=0x56458cddc730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:38.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.703+0000 7ff594310740 1 --2- 172.21.15.1:0/1183898941 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56458cdd4980 0x56458cddc730 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7ff57c009930 tx=0x7ff57c0319a0 comp rx=0 tx=0).stop 2024-09-11T22:04:38.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.703+0000 7ff594310740 1 -- 172.21.15.1:0/1183898941 shutdown_connections 2024-09-11T22:04:38.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.703+0000 7ff594310740 1 --2- 172.21.15.1:0/1183898941 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x56458cdeeec0 0x56458cdf1380 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:38.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.703+0000 7ff594310740 1 --2- 172.21.15.1:0/1183898941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56458cddf240 0x56458cddcc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:38.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.703+0000 7ff594310740 1 --2- 172.21.15.1:0/1183898941 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56458cdd4980 0x56458cddc730 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:38.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.703+0000 7ff56affd640 0 client.17058 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:04:38.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.703+0000 7ff594310740 1 --2- 172.21.15.1:0/1183898941 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56458cdd4060 0x56458cddc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:38.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.703+0000 7ff594310740 1 -- 172.21.15.1:0/1183898941 >> 172.21.15.1:0/1183898941 conn(0x56458cdd05f0 msgr2=0x56458cdd62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:38.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.703+0000 7ff594310740 1 -- 172.21.15.1:0/1183898941 shutdown_connections 2024-09-11T22:04:38.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.703+0000 7ff594310740 1 -- 172.21.15.1:0/1183898941 wait complete. 2024-09-11T22:04:38.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.703+0000 7ff594310740 20 client.17058 trim_cache size 0 max 16384 2024-09-11T22:04:38.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.720+0000 7ff594310740 1 Processor -- start 2024-09-11T22:04:38.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.720+0000 7ff594310740 1 -- start start 2024-09-11T22:04:38.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.720+0000 7ff594310740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56458cdd4060 0x56458cdf6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:38.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.720+0000 7ff594310740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56458cdd4980 0x56458cdf7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:38.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.721+0000 7ff594310740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56458cddf240 0x56458cdd9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:38.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.721+0000 7ff594310740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff58c1afcb0 con 0x56458cdd4060 2024-09-11T22:04:38.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.721+0000 7ff594310740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff58c1b1af0 con 0x56458cddf240 2024-09-11T22:04:38.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.721+0000 7ff594310740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff58c1b0440 con 0x56458cdd4980 2024-09-11T22:04:38.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.721+0000 7ff58affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56458cdd4060 0x56458cdf6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:38.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.721+0000 7ff58affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56458cdd4060 0x56458cdf6c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47468/0 (socket says 172.21.15.1:47468) 2024-09-11T22:04:38.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.721+0000 7ff58affd640 1 -- 172.21.15.1:0/1531201791 learned_addr learned my addr 172.21.15.1:0/1531201791 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:38.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.721+0000 7ff58b7fe640 1 --2- 172.21.15.1:0/1531201791 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56458cdd4980 0x56458cdf7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:38.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.721+0000 7ff58affd640 1 -- 172.21.15.1:0/1531201791 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56458cdd4980 msgr2=0x56458cdf7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:38.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.721+0000 7ff58affd640 1 --2- 172.21.15.1:0/1531201791 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56458cdd4980 0x56458cdf7190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:38.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.721+0000 7ff58affd640 1 -- 172.21.15.1:0/1531201791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56458cddf240 msgr2=0x56458cdd9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:38.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.721+0000 7ff58affd640 1 --2- 172.21.15.1:0/1531201791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56458cddf240 0x56458cdd9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:38.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.721+0000 7ff58affd640 1 -- 172.21.15.1:0/1531201791 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56458cdd1780 con 0x56458cdd4060 2024-09-11T22:04:38.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.721+0000 7ff58b7fe640 1 --2- 172.21.15.1:0/1531201791 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56458cdd4980 0x56458cdf7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:04:38.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.721+0000 7ff58affd640 1 --2- 172.21.15.1:0/1531201791 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56458cdd4060 0x56458cdf6c50 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7ff580009b20 tx=0x7ff580018580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f7a563ff3817981 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:38.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.721+0000 7ff5717fa640 1 -- 172.21.15.1:0/1531201791 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff580018bd0 con 0x56458cdd4060 2024-09-11T22:04:38.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.721+0000 7ff5717fa640 1 -- 172.21.15.1:0/1531201791 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff58001d040 con 0x56458cdd4060 2024-09-11T22:04:38.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.721+0000 7ff5717fa640 1 -- 172.21.15.1:0/1531201791 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff580022730 con 0x56458cdd4060 2024-09-11T22:04:38.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.722+0000 7ff594310740 1 -- 172.21.15.1:0/1531201791 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56458cdd4060 msgr2=0x56458cdf6c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:38.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.722+0000 7ff594310740 1 --2- 172.21.15.1:0/1531201791 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56458cdd4060 0x56458cdf6c50 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7ff580009b20 tx=0x7ff580018580 comp rx=0 tx=0).stop 2024-09-11T22:04:38.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.722+0000 7ff594310740 1 -- 172.21.15.1:0/1531201791 shutdown_connections 2024-09-11T22:04:38.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.722+0000 7ff594310740 1 --2- 172.21.15.1:0/1531201791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56458cddf240 0x56458cdd9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:38.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.722+0000 7ff594310740 1 --2- 172.21.15.1:0/1531201791 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56458cdd4980 0x56458cdf7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:38.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.722+0000 7ff594310740 1 --2- 172.21.15.1:0/1531201791 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56458cdd4060 0x56458cdf6c50 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:38.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.722+0000 7ff594310740 1 -- 172.21.15.1:0/1531201791 >> 172.21.15.1:0/1531201791 conn(0x56458cdd05f0 msgr2=0x56458cdde270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:38.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.722+0000 7ff594310740 1 -- 172.21.15.1:0/1531201791 shutdown_connections 2024-09-11T22:04:38.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.722+0000 7ff594310740 1 -- 172.21.15.1:0/1531201791 wait complete. 2024-09-11T22:04:38.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.722+0000 7ff594310740 1 -- start start 2024-09-11T22:04:38.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.723+0000 7ff594310740 1 Processor -- start 2024-09-11T22:04:38.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.723+0000 7ff594310740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56458cdd4060 0x56458cde3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:38.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.723+0000 7ff594310740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56458cdd4980 0x56458cde38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:38.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.723+0000 7ff594310740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56458cddf240 0x56458ce58c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:38.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.723+0000 7ff594310740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff58c1b1360 con 0x56458cddf240 2024-09-11T22:04:38.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.723+0000 7ff594310740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff58c1b0bd0 con 0x56458cdd4060 2024-09-11T22:04:38.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.723+0000 7ff594310740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff58c1adcb0 con 0x56458cdd4980 2024-09-11T22:04:38.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.723+0000 7ff594310740 10 client.0 _ll_register_callbacks cb 0x7ff57c0315b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:04:38.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.723+0000 7ff58affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56458cddf240 0x56458ce58c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:38.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.723+0000 7ff58affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56458cddf240 0x56458ce58c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47472/0 (socket says 172.21.15.1:47472) 2024-09-11T22:04:38.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.723+0000 7ff58affd640 1 -- 172.21.15.1:0/1886696694 learned_addr learned my addr 172.21.15.1:0/1886696694 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:38.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.723+0000 7ff58affd640 1 --2- 172.21.15.1:0/1886696694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56458cdd4060 0x56458cde3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:38.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.723+0000 7ff58b7fe640 1 --2- 172.21.15.1:0/1886696694 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56458cdd4980 0x56458cde38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:38.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.723+0000 7ff58affd640 1 -- 172.21.15.1:0/1886696694 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56458cdd4980 msgr2=0x56458cde38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:38.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.723+0000 7ff58affd640 1 --2- 172.21.15.1:0/1886696694 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56458cdd4980 0x56458cde38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:38.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.723+0000 7ff58affd640 1 -- 172.21.15.1:0/1886696694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56458cdd4060 msgr2=0x56458cde3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:38.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.724+0000 7ff58affd640 1 --2- 172.21.15.1:0/1886696694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56458cdd4060 0x56458cde3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:38.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.724+0000 7ff58affd640 1 -- 172.21.15.1:0/1886696694 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56458ce59160 con 0x56458cddf240 2024-09-11T22:04:38.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.724+0000 7ff58affd640 1 --2- 172.21.15.1:0/1886696694 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56458cddf240 0x56458ce58c20 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7ff580016010 tx=0x7ff580015e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a192ac29428171c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:38.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.724+0000 7ff56affd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:04:38.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.724+0000 7ff56affd640 1 -- 172.21.15.1:0/1886696694 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff580015730 con 0x56458cddf240 2024-09-11T22:04:38.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.724+0000 7ff56affd640 1 -- 172.21.15.1:0/1886696694 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7ff5800158d0 con 0x56458cddf240 2024-09-11T22:04:38.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.724+0000 7ff594310740 1 -- 172.21.15.1:0/1886696694 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff58001d040 con 0x56458cddf240 2024-09-11T22:04:38.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.724+0000 7ff56affd640 1 -- 172.21.15.1:0/1886696694 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7ff580015ad0 con 0x56458cddf240 2024-09-11T22:04:38.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.724+0000 7ff594310740 10 client.17064 fetch_fsmap learned FSMap version 42 2024-09-11T22:04:38.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.724+0000 7ff594310740 1 -- 172.21.15.1:0/1886696694 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x56458ce5a2d0 con 0x56458cddf240 2024-09-11T22:04:38.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.724+0000 7ff56affd640 1 -- 172.21.15.1:0/1886696694 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff58001d040 con 0x56458cddf240 2024-09-11T22:04:38.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.724+0000 7ff56affd640 1 -- 172.21.15.1:0/1886696694 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7ff58003b020 con 0x56458cddf240 2024-09-11T22:04:38.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.724+0000 7ff594310740 10 client.17064 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:04:38.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.724+0000 7ff594310740 10 client.17064 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-11T22:04:38.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.724+0000 7ff594310740 20 client.17064 populate_metadata read hostname 'smithi001' 2024-09-11T22:04:38.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.724+0000 7ff594310740 1 --2- 172.21.15.1:0/1886696694 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x56458ce5a7d0 0x56458ce5cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:38.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.724+0000 7ff594310740 4 client.17064 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "42839ccd", "release": true, "await": true}] 2024-09-11T22:04:38.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.724+0000 7ff594310740 1 -- 172.21.15.1:0/1886696694 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "quiesce db", "set_id": "42839ccd", "release": true, "await": true}) -- 0x56458cd410a0 con 0x56458ce5a7d0 2024-09-11T22:04:38.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.725+0000 7ff58b7fe640 1 --2- 172.21.15.1:0/1886696694 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x56458ce5a7d0 0x56458ce5cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:38.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.725+0000 7ff58b7fe640 1 --2- 172.21.15.1:0/1886696694 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x56458ce5a7d0 0x56458ce5cbf0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed9b3ffa20a13220 server_cookie=42a21c32b74b490b in_seq=0 out_seq=0 2024-09-11T22:04:38.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.725+0000 7ff56affd640 10 client.17064 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:04:38.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.796+0000 7ff56affd640 1 -- 172.21.15.1:0/1886696694 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56458cd410a0 con 0x56458ce5a7d0 2024-09-11T22:04:38.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.796+0000 7ff56affd640 10 client.17064 handle_command_reply: tid=0 2024-09-11T22:04:38.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.796+0000 7ff594310740 1 client.17064 shutdown 2024-09-11T22:04:38.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.796+0000 7ff594310740 1 -- 172.21.15.1:0/1886696694 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56458cddf240 msgr2=0x56458ce58c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:38.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.796+0000 7ff594310740 1 --2- 172.21.15.1:0/1886696694 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56458cddf240 0x56458ce58c20 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7ff580016010 tx=0x7ff580015e50 comp rx=0 tx=0).stop 2024-09-11T22:04:38.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.796+0000 7ff594310740 1 -- 172.21.15.1:0/1886696694 shutdown_connections 2024-09-11T22:04:38.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.796+0000 7ff594310740 1 --2- 172.21.15.1:0/1886696694 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x56458ce5a7d0 0x56458ce5cbf0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:38.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.796+0000 7ff594310740 1 --2- 172.21.15.1:0/1886696694 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56458cddf240 0x56458ce58c20 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:38.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.796+0000 7ff594310740 1 --2- 172.21.15.1:0/1886696694 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56458cdd4980 0x56458cde38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:38.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.796+0000 7ff594310740 1 --2- 172.21.15.1:0/1886696694 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56458cdd4060 0x56458cde3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:38.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.796+0000 7ff56affd640 0 client.17064 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:04:38.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.796+0000 7ff594310740 1 -- 172.21.15.1:0/1886696694 >> 172.21.15.1:0/1886696694 conn(0x56458cdd05f0 msgr2=0x56458cddc190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:38.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.796+0000 7ff594310740 1 -- 172.21.15.1:0/1886696694 shutdown_connections 2024-09-11T22:04:38.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.796+0000 7ff594310740 1 -- 172.21.15.1:0/1886696694 wait complete. 2024-09-11T22:04:38.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.797+0000 7ff594310740 20 client.17064 trim_cache size 0 max 16384 2024-09-11T22:04:38.800 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-11T22:04:38.800 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 42, 2024-09-11T22:04:38.800 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24494, 2024-09-11T22:04:38.800 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 24, 2024-09-11T22:04:38.800 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-09-11T22:04:38.800 INFO:teuthology.orchestra.run.smithi001.stdout: "42839ccd": { 2024-09-11T22:04:38.800 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 24, 2024-09-11T22:04:38.800 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-09-11T22:04:38.800 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-11T22:04:38.800 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RELEASED", 2024-09-11T22:04:38.801 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-11T22:04:38.801 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:04:38.801 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 90.0, 2024-09-11T22:04:38.801 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 140.1, 2024-09-11T22:04:38.801 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-09-11T22:04:38.801 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-09-11T22:04:38.801 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-09-11T22:04:38.801 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-11T22:04:38.801 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RELEASED", 2024-09-11T22:04:38.801 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-11T22:04:38.801 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:04:38.801 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:04:38.801 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:04:38.801 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:04:38.802 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:04:38.802 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-11T22:04:38.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.797+0000 7ff591f37640 1 -- 172.21.15.1:0/3274782745 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7ff5740a2cf0 msgr2=0x7ff5740a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:38.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.797+0000 7ff591f37640 1 --2- 172.21.15.1:0/3274782745 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7ff5740a2cf0 0x7ff5740a51b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff58400afd0 tx=0x7ff58401f4b0 comp rx=0 tx=0).stop 2024-09-11T22:04:38.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.797+0000 7ff591f37640 1 -- 172.21.15.1:0/3274782745 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff58c106f50 msgr2=0x7ff58c101d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:38.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.797+0000 7ff591f37640 1 --2- 172.21.15.1:0/3274782745 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff58c106f50 0x7ff58c101d50 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7ff58400cce0 tx=0x7ff58400a620 comp rx=0 tx=0).stop 2024-09-11T22:04:38.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.797+0000 7ff591f37640 1 -- 172.21.15.1:0/3274782745 shutdown_connections 2024-09-11T22:04:38.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.797+0000 7ff591f37640 1 --2- 172.21.15.1:0/3274782745 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7ff5740a2cf0 0x7ff5740a51b0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:38.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.797+0000 7ff591f37640 1 --2- 172.21.15.1:0/3274782745 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff58c106f50 0x7ff58c101d50 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:38.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.797+0000 7ff591f37640 1 --2- 172.21.15.1:0/3274782745 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff58c100d00 0x7ff58c101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:38.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.797+0000 7ff591f37640 1 --2- 172.21.15.1:0/3274782745 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff58c100350 0x7ff58c1012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:38.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.797+0000 7ff591f37640 1 -- 172.21.15.1:0/3274782745 >> 172.21.15.1:0/3274782745 conn(0x7ff58c077210 msgr2=0x7ff58c105860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:38.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.797+0000 7ff591f37640 1 -- 172.21.15.1:0/3274782745 shutdown_connections 2024-09-11T22:04:38.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:38.797+0000 7ff591f37640 1 -- 172.21.15.1:0/3274782745 wait complete. 2024-09-11T22:04:39.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:04:38 smithi053 ceph-mon[34908]: pgmap v229: 97 pgs: 97 active+clean; 444 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 90 KiB/s wr, 106 op/s 2024-09-11T22:04:39.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:04:38 smithi161 ceph-mon[35615]: pgmap v229: 97 pgs: 97 active+clean; 444 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 90 KiB/s wr, 106 op/s 2024-09-11T22:04:39.343 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:04:38 smithi001 ceph-mon[30935]: pgmap v229: 97 pgs: 97 active+clean; 444 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 90 KiB/s wr, 106 op/s 2024-09-11T22:04:39.655 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '42839ccd', total seconds elapsed: 23.6 2024-09-11T22:04:40.343 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.342+0000 7f9d91d9b640 1 -- 172.21.15.1:0/2184457485 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d840319b0 con 0x7f9d941035c0 2024-09-11T22:04:40.344 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.342+0000 7f9d98cf1640 1 -- 172.21.15.1:0/2184457485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d941035c0 msgr2=0x7f9d9410baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:40.344 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.342+0000 7f9d98cf1640 1 --2- 172.21.15.1:0/2184457485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d941035c0 0x7f9d9410baa0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9d84009900 tx=0x7f9d8402a4b0 comp rx=0 tx=0).stop 2024-09-11T22:04:40.344 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.343+0000 7f9d98cf1640 1 -- 172.21.15.1:0/2184457485 shutdown_connections 2024-09-11T22:04:40.344 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.343+0000 7f9d98cf1640 1 --2- 172.21.15.1:0/2184457485 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d9410c0f0 0x7f9d9410e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:40.345 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.343+0000 7f9d98cf1640 1 --2- 172.21.15.1:0/2184457485 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d941035c0 0x7f9d9410baa0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:40.345 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.343+0000 7f9d98cf1640 1 --2- 172.21.15.1:0/2184457485 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d94102ca0 0x7f9d94103080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:40.345 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.343+0000 7f9d98cf1640 1 -- 172.21.15.1:0/2184457485 >> 172.21.15.1:0/2184457485 conn(0x7f9d940fc6b0 msgr2=0x7f9d940fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:40.345 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.343+0000 7f9d98cf1640 1 -- 172.21.15.1:0/2184457485 shutdown_connections 2024-09-11T22:04:40.345 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.343+0000 7f9d98cf1640 1 -- 172.21.15.1:0/2184457485 wait complete. 2024-09-11T22:04:40.347 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.344+0000 7f9d98cf1640 1 Processor -- start 2024-09-11T22:04:40.347 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.344+0000 7f9d98cf1640 1 -- start start 2024-09-11T22:04:40.347 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.344+0000 7f9d98cf1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d94102ca0 0x7f9d941984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:40.347 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.344+0000 7f9d98cf1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d941035c0 0x7f9d94198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:40.347 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.344+0000 7f9d98cf1640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d9410c0f0 0x7f9d9419cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:40.347 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.344+0000 7f9d98cf1640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9d94111f80 con 0x7f9d941035c0 2024-09-11T22:04:40.347 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.344+0000 7f9d98cf1640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9d94111e00 con 0x7f9d9410c0f0 2024-09-11T22:04:40.347 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.344+0000 7f9d98cf1640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9d94112100 con 0x7f9d94102ca0 2024-09-11T22:04:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.344+0000 7f9d8bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d941035c0 0x7f9d94198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.344+0000 7f9d8bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d941035c0 0x7f9d94198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47488/0 (socket says 172.21.15.1:47488) 2024-09-11T22:04:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.344+0000 7f9d8bfff640 1 -- 172.21.15.1:0/994539246 learned_addr learned my addr 172.21.15.1:0/994539246 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.344+0000 7f9d92d9d640 1 --2- 172.21.15.1:0/994539246 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d9410c0f0 0x7f9d9419cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.345+0000 7f9d8bfff640 1 -- 172.21.15.1:0/994539246 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d94102ca0 msgr2=0x7f9d941984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.345+0000 7f9d8bfff640 1 --2- 172.21.15.1:0/994539246 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d94102ca0 0x7f9d941984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.345+0000 7f9d8bfff640 1 -- 172.21.15.1:0/994539246 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d9410c0f0 msgr2=0x7f9d9419cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.345+0000 7f9d8bfff640 1 --2- 172.21.15.1:0/994539246 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d9410c0f0 0x7f9d9419cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.345+0000 7f9d8bfff640 1 -- 172.21.15.1:0/994539246 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d9419d4b0 con 0x7f9d941035c0 2024-09-11T22:04:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.345+0000 7f9d92d9d640 1 --2- 172.21.15.1:0/994539246 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d9410c0f0 0x7f9d9419cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:04:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.345+0000 7f9d8bfff640 1 --2- 172.21.15.1:0/994539246 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d941035c0 0x7f9d94198a00 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f9d84006010 tx=0x7f9d84004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.345+0000 7f9d8b7fe640 1 -- 172.21.15.1:0/994539246 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8403f030 con 0x7f9d941035c0 2024-09-11T22:04:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.345+0000 7f9d8b7fe640 1 -- 172.21.15.1:0/994539246 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d84004440 con 0x7f9d941035c0 2024-09-11T22:04:40.349 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.345+0000 7f9d98cf1640 1 -- 172.21.15.1:0/994539246 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d9419d740 con 0x7f9d941035c0 2024-09-11T22:04:40.349 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.345+0000 7f9d8b7fe640 1 -- 172.21.15.1:0/994539246 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d84035070 con 0x7f9d941035c0 2024-09-11T22:04:40.349 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.345+0000 7f9d98cf1640 1 -- 172.21.15.1:0/994539246 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d94100820 con 0x7f9d941035c0 2024-09-11T22:04:40.350 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.349+0000 7f9d8b7fe640 1 -- 172.21.15.1:0/994539246 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f9d84042030 con 0x7f9d941035c0 2024-09-11T22:04:40.350 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.349+0000 7f9d9ba8c740 1 Processor -- start 2024-09-11T22:04:40.351 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.349+0000 7f9d9ba8c740 1 -- start start 2024-09-11T22:04:40.351 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.350+0000 7f9d9ba8c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b88c111af0 0x55b88c111ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.350+0000 7f9d9ba8c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b88c112410 0x55b88c11c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.350+0000 7f9d9ba8c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b88c11ccb0 0x55b88c11f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.350+0000 7f9d9ba8c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55b88c0762f0 con 0x55b88c111af0 2024-09-11T22:04:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.350+0000 7f9d9ba8c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55b88c01e270 con 0x55b88c112410 2024-09-11T22:04:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.350+0000 7f9d9ba8c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b88c0722b0 con 0x55b88c11ccb0 2024-09-11T22:04:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.350+0000 7f9d92d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b88c111af0 0x55b88c111ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.350+0000 7f9d92d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b88c111af0 0x55b88c111ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47496/0 (socket says 172.21.15.1:47496) 2024-09-11T22:04:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.350+0000 7f9d92d9d640 1 -- 172.21.15.1:0/1275438666 learned_addr learned my addr 172.21.15.1:0/1275438666 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.350+0000 7f9d8bfff640 1 --2- 172.21.15.1:0/1275438666 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b88c11ccb0 0x55b88c11f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.350+0000 7f9d9259c640 1 --2- 172.21.15.1:0/1275438666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b88c112410 0x55b88c11c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:40.353 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.350+0000 7f9d92d9d640 1 -- 172.21.15.1:0/1275438666 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b88c11ccb0 msgr2=0x55b88c11f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:40.353 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.351+0000 7f9d92d9d640 1 --2- 172.21.15.1:0/1275438666 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b88c11ccb0 0x55b88c11f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:40.353 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.351+0000 7f9d92d9d640 1 -- 172.21.15.1:0/1275438666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b88c112410 msgr2=0x55b88c11c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:40.353 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.351+0000 7f9d92d9d640 1 --2- 172.21.15.1:0/1275438666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b88c112410 0x55b88c11c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:40.353 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.351+0000 7f9d92d9d640 1 -- 172.21.15.1:0/1275438666 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b88c01d340 con 0x55b88c111af0 2024-09-11T22:04:40.353 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.351+0000 7f9d8b7fe640 1 --2- 172.21.15.1:0/994539246 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f9d740a2cf0 0x7f9d740a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:40.353 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.351+0000 7f9d9259c640 1 --2- 172.21.15.1:0/1275438666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b88c112410 0x55b88c11c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:04:40.354 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.351+0000 7f9d8bfff640 1 --2- 172.21.15.1:0/1275438666 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b88c11ccb0 0x55b88c11f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:04:40.354 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.351+0000 7f9d92d9d640 1 --2- 172.21.15.1:0/1275438666 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b88c111af0 0x55b88c111ed0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c010a80 tx=0x7f9d8c009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b84b0d7ead9cadf1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:40.354 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.351+0000 7f9d8b7fe640 1 -- 172.21.15.1:0/994539246 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7f9d840ef9b0 con 0x7f9d941035c0 2024-09-11T22:04:40.354 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.351+0000 7f9d9259c640 1 --2- 172.21.15.1:0/994539246 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f9d740a2cf0 0x7f9d740a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:40.355 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.352+0000 7f9d88ff9640 1 -- 172.21.15.1:0/1275438666 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c003c30 con 0x55b88c111af0 2024-09-11T22:04:40.355 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.352+0000 7f9d88ff9640 1 -- 172.21.15.1:0/1275438666 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d8c01f070 con 0x55b88c111af0 2024-09-11T22:04:40.355 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.352+0000 7f9d88ff9640 1 -- 172.21.15.1:0/1275438666 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c01a400 con 0x55b88c111af0 2024-09-11T22:04:40.355 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.353+0000 7f9d9ba8c740 1 -- 172.21.15.1:0/1275438666 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b88c111af0 msgr2=0x55b88c111ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:40.355 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.353+0000 7f9d9ba8c740 1 --2- 172.21.15.1:0/1275438666 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b88c111af0 0x55b88c111ed0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c010a80 tx=0x7f9d8c009500 comp rx=0 tx=0).stop 2024-09-11T22:04:40.356 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.353+0000 7f9d9259c640 1 --2- 172.21.15.1:0/994539246 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f9d740a2cf0 0x7f9d740a51b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x55b88c10eec0 tx=0x7f9d7c008040 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:40.356 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.353+0000 7f9d9ba8c740 1 -- 172.21.15.1:0/1275438666 shutdown_connections 2024-09-11T22:04:40.356 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.353+0000 7f9d9ba8c740 1 --2- 172.21.15.1:0/1275438666 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b88c11ccb0 0x55b88c11f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:40.356 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.353+0000 7f9d9ba8c740 1 --2- 172.21.15.1:0/1275438666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b88c112410 0x55b88c11c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:40.356 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.353+0000 7f9d9ba8c740 1 --2- 172.21.15.1:0/1275438666 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b88c111af0 0x55b88c111ed0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:40.356 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.353+0000 7f9d9ba8c740 1 -- 172.21.15.1:0/1275438666 >> 172.21.15.1:0/1275438666 conn(0x55b88c10e060 msgr2=0x55b88c10e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:40.356 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.353+0000 7f9d9ba8c740 1 -- 172.21.15.1:0/1275438666 shutdown_connections 2024-09-11T22:04:40.356 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.353+0000 7f9d9ba8c740 1 -- 172.21.15.1:0/1275438666 wait complete. 2024-09-11T22:04:40.356 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.354+0000 7f9d9ba8c740 1 -- start start 2024-09-11T22:04:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.354+0000 7f9d9ba8c740 1 Processor -- start 2024-09-11T22:04:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.354+0000 7f9d9ba8c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b88c111af0 0x55b88c119bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.354+0000 7f9d9ba8c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b88c112410 0x55b88c11a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.354+0000 7f9d9ba8c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b88c11ccb0 0x55b88c11a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.354+0000 7f9d9ba8c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55b88c11f880 con 0x55b88c11ccb0 2024-09-11T22:04:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.354+0000 7f9d9ba8c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55b88bfff290 con 0x55b88c111af0 2024-09-11T22:04:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.354+0000 7f9d9ba8c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b88c11fa00 con 0x55b88c112410 2024-09-11T22:04:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.354+0000 7f9d9ba8c740 10 client.0 _ll_register_callbacks cb 0x55b88c030140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:04:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.355+0000 7f9d92d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b88c11ccb0 0x55b88c11a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.355+0000 7f9d92d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b88c11ccb0 0x55b88c11a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47498/0 (socket says 172.21.15.1:47498) 2024-09-11T22:04:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.355+0000 7f9d92d9d640 1 -- 172.21.15.1:0/1249779314 learned_addr learned my addr 172.21.15.1:0/1249779314 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.355+0000 7f9d8bfff640 1 --2- 172.21.15.1:0/1249779314 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b88c112410 0x55b88c11a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.355+0000 7f9d9259c640 1 --2- 172.21.15.1:0/1249779314 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b88c111af0 0x55b88c119bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.355+0000 7f9d8bfff640 1 -- 172.21.15.1:0/1249779314 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b88c111af0 msgr2=0x55b88c119bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.355+0000 7f9d8bfff640 1 --2- 172.21.15.1:0/1249779314 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b88c111af0 0x55b88c119bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.355+0000 7f9d8bfff640 1 -- 172.21.15.1:0/1249779314 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b88c11ccb0 msgr2=0x55b88c11a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.355+0000 7f9d8bfff640 1 --2- 172.21.15.1:0/1249779314 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b88c11ccb0 0x55b88c11a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.355+0000 7f9d8bfff640 1 -- 172.21.15.1:0/1249779314 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b88bd89210 con 0x55b88c112410 2024-09-11T22:04:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.355+0000 7f9d9259c640 1 --2- 172.21.15.1:0/1249779314 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b88c111af0 0x55b88c119bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:04:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.355+0000 7f9d92d9d640 1 --2- 172.21.15.1:0/1249779314 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b88c11ccb0 0x55b88c11a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:04:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.355+0000 7f9d8bfff640 1 --2- 172.21.15.1:0/1249779314 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b88c112410 0x55b88c11a130 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9d840ed000 tx=0x7f9d840bba00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=122607a034e56884 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.355+0000 7f9d81ffb640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-11T22:04:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.356+0000 7f9d81ffb640 1 -- 172.21.15.1:0/1249779314 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d84049040 con 0x55b88c112410 2024-09-11T22:04:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.356+0000 7f9d9ba8c740 1 -- 172.21.15.1:0/1249779314 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d8c01f070 con 0x55b88c112410 2024-09-11T22:04:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.356+0000 7f9d81ffb640 1 -- 172.21.15.1:0/1249779314 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7f9d840b8470 con 0x55b88c112410 2024-09-11T22:04:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.356+0000 7f9d81ffb640 1 -- 172.21.15.1:0/1249779314 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f9d840c01b0 con 0x55b88c112410 2024-09-11T22:04:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.356+0000 7f9d9ba8c740 10 client.26542 fetch_fsmap learned FSMap version 42 2024-09-11T22:04:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.356+0000 7f9d9ba8c740 1 -- 172.21.15.1:0/1249779314 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55b88be55620 con 0x55b88c112410 2024-09-11T22:04:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.356+0000 7f9d81ffb640 1 -- 172.21.15.1:0/1249779314 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d8c01f070 con 0x55b88c112410 2024-09-11T22:04:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.357+0000 7f9d81ffb640 1 -- 172.21.15.1:0/1249779314 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f9d840c45e0 con 0x55b88c112410 2024-09-11T22:04:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.357+0000 7f9d9ba8c740 10 client.26542 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:04:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.357+0000 7f9d9ba8c740 10 client.26542 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:04:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.357+0000 7f9d9ba8c740 20 client.26542 populate_metadata read hostname 'smithi001' 2024-09-11T22:04:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.357+0000 7f9d9ba8c740 1 --2- 172.21.15.1:0/1249779314 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55b88c12cc90 0x55b88c12f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.357+0000 7f9d9ba8c740 4 client.26542 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:04:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.357+0000 7f9d9ba8c740 1 -- 172.21.15.1:0/1249779314 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b88c083a20 con 0x55b88c12cc90 2024-09-11T22:04:40.362 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.357+0000 7f9d9259c640 1 --2- 172.21.15.1:0/1249779314 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55b88c12cc90 0x55b88c12f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.358+0000 7f9d9259c640 1 --2- 172.21.15.1:0/1249779314 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55b88c12cc90 0x55b88c12f0b0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d95f816a34d857bf server_cookie=b23de162303b1059 in_seq=0 out_seq=0 2024-09-11T22:04:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.358+0000 7f9d81ffb640 10 client.26542 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:04:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.359+0000 7f9d81ffb640 1 -- 172.21.15.1:0/1249779314 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b88c083a20 con 0x55b88c12cc90 2024-09-11T22:04:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.359+0000 7f9d81ffb640 10 client.26542 handle_command_reply: tid=0 2024-09-11T22:04:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.359+0000 7f9d9ba8c740 1 client.26542 shutdown 2024-09-11T22:04:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.359+0000 7f9d9ba8c740 1 -- 172.21.15.1:0/1249779314 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b88c112410 msgr2=0x55b88c11a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.359+0000 7f9d9ba8c740 1 --2- 172.21.15.1:0/1249779314 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b88c112410 0x55b88c11a130 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9d840ed000 tx=0x7f9d840bba00 comp rx=0 tx=0).stop 2024-09-11T22:04:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.359+0000 7f9d9ba8c740 1 -- 172.21.15.1:0/1249779314 shutdown_connections 2024-09-11T22:04:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.359+0000 7f9d9ba8c740 1 --2- 172.21.15.1:0/1249779314 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55b88c12cc90 0x55b88c12f0b0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.359+0000 7f9d9ba8c740 1 --2- 172.21.15.1:0/1249779314 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b88c11ccb0 0x55b88c11a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.359+0000 7f9d9ba8c740 1 --2- 172.21.15.1:0/1249779314 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b88c112410 0x55b88c11a130 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.359+0000 7f9d9ba8c740 1 --2- 172.21.15.1:0/1249779314 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b88c111af0 0x55b88c119bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.359+0000 7f9d9ba8c740 1 -- 172.21.15.1:0/1249779314 >> 172.21.15.1:0/1249779314 conn(0x55b88c10e060 msgr2=0x55b88c113dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.359+0000 7f9d81ffb640 0 client.26542 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:04:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.359+0000 7f9d9ba8c740 1 -- 172.21.15.1:0/1249779314 shutdown_connections 2024-09-11T22:04:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.359+0000 7f9d9ba8c740 1 -- 172.21.15.1:0/1249779314 wait complete. 2024-09-11T22:04:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.359+0000 7f9d9ba8c740 20 client.26542 trim_cache size 0 max 16384 2024-09-11T22:04:40.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.379+0000 7f9d9ba8c740 1 Processor -- start 2024-09-11T22:04:40.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.379+0000 7f9d9ba8c740 1 -- start start 2024-09-11T22:04:40.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.379+0000 7f9d9ba8c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b88c111af0 0x55b88c11a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:40.382 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.379+0000 7f9d9ba8c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b88c112410 0x55b88c11bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.379+0000 7f9d9ba8c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b88c11ccb0 0x55b88c134940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.379+0000 7f9d9ba8c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9d941ab950 con 0x55b88c111af0 2024-09-11T22:04:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.379+0000 7f9d9ba8c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9d941ad790 con 0x55b88c112410 2024-09-11T22:04:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.379+0000 7f9d9ba8c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9d941ac0e0 con 0x55b88c11ccb0 2024-09-11T22:04:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.379+0000 7f9d9259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b88c111af0 0x55b88c11a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.379+0000 7f9d9259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b88c111af0 0x55b88c11a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47508/0 (socket says 172.21.15.1:47508) 2024-09-11T22:04:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.379+0000 7f9d9259c640 1 -- 172.21.15.1:0/1490408778 learned_addr learned my addr 172.21.15.1:0/1490408778 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.379+0000 7f9d92d9d640 1 --2- 172.21.15.1:0/1490408778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b88c11ccb0 0x55b88c134940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.379+0000 7f9d8bfff640 1 --2- 172.21.15.1:0/1490408778 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b88c112410 0x55b88c11bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.380+0000 7f9d9259c640 1 -- 172.21.15.1:0/1490408778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b88c11ccb0 msgr2=0x55b88c134940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.380+0000 7f9d9259c640 1 --2- 172.21.15.1:0/1490408778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b88c11ccb0 0x55b88c134940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.380+0000 7f9d9259c640 1 -- 172.21.15.1:0/1490408778 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b88c112410 msgr2=0x55b88c11bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:40.384 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.380+0000 7f9d9259c640 1 --2- 172.21.15.1:0/1490408778 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b88c112410 0x55b88c11bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:40.384 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.380+0000 7f9d9259c640 1 -- 172.21.15.1:0/1490408778 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b88c10f110 con 0x55b88c111af0 2024-09-11T22:04:40.384 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.380+0000 7f9d9259c640 1 --2- 172.21.15.1:0/1490408778 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b88c111af0 0x55b88c11a4a0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c006ef0 tx=0x7f9d7c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=110b9a705d628928 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:40.387 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.380+0000 7f9d83fff640 1 -- 172.21.15.1:0/1490408778 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c016110 con 0x55b88c111af0 2024-09-11T22:04:40.387 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.380+0000 7f9d83fff640 1 -- 172.21.15.1:0/1490408778 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d7c026030 con 0x55b88c111af0 2024-09-11T22:04:40.387 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.380+0000 7f9d83fff640 1 -- 172.21.15.1:0/1490408778 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c0146d0 con 0x55b88c111af0 2024-09-11T22:04:40.387 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.381+0000 7f9d9ba8c740 1 -- 172.21.15.1:0/1490408778 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b88c111af0 msgr2=0x55b88c11a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:40.387 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.381+0000 7f9d9ba8c740 1 --2- 172.21.15.1:0/1490408778 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b88c111af0 0x55b88c11a4a0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c006ef0 tx=0x7f9d7c010000 comp rx=0 tx=0).stop 2024-09-11T22:04:40.387 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.381+0000 7f9d9ba8c740 1 -- 172.21.15.1:0/1490408778 shutdown_connections 2024-09-11T22:04:40.388 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.381+0000 7f9d9ba8c740 1 --2- 172.21.15.1:0/1490408778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b88c11ccb0 0x55b88c134940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:40.388 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.381+0000 7f9d9ba8c740 1 --2- 172.21.15.1:0/1490408778 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b88c112410 0x55b88c11bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:40.388 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.381+0000 7f9d9ba8c740 1 --2- 172.21.15.1:0/1490408778 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b88c111af0 0x55b88c11a4a0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:40.388 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.381+0000 7f9d9ba8c740 1 -- 172.21.15.1:0/1490408778 >> 172.21.15.1:0/1490408778 conn(0x55b88c10e060 msgr2=0x55b88c11b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:40.388 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.381+0000 7f9d9ba8c740 1 -- 172.21.15.1:0/1490408778 shutdown_connections 2024-09-11T22:04:40.388 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.381+0000 7f9d9ba8c740 1 -- 172.21.15.1:0/1490408778 wait complete. 2024-09-11T22:04:40.388 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.382+0000 7f9d9ba8c740 1 -- start start 2024-09-11T22:04:40.388 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.382+0000 7f9d9ba8c740 1 Processor -- start 2024-09-11T22:04:40.388 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.382+0000 7f9d9ba8c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b88c111af0 0x55b88c124e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:40.388 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.383+0000 7f9d9ba8c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b88c112410 0x55b88c1253c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:40.388 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.383+0000 7f9d9ba8c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b88c11ccb0 0x55b88c116420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:40.388 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.383+0000 7f9d9ba8c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9d941ad000 con 0x55b88c111af0 2024-09-11T22:04:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.383+0000 7f9d9ba8c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9d941ac870 con 0x55b88c112410 2024-09-11T22:04:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.383+0000 7f9d9ba8c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9d941a9950 con 0x55b88c11ccb0 2024-09-11T22:04:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.383+0000 7f9d9ba8c740 10 client.0 _ll_register_callbacks cb 0x7f9d84005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:04:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.383+0000 7f9d9259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b88c111af0 0x55b88c124e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.383+0000 7f9d9259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b88c111af0 0x55b88c124e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47522/0 (socket says 172.21.15.1:47522) 2024-09-11T22:04:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.383+0000 7f9d9259c640 1 -- 172.21.15.1:0/793202763 learned_addr learned my addr 172.21.15.1:0/793202763 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.383+0000 7f9d92d9d640 1 --2- 172.21.15.1:0/793202763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b88c11ccb0 0x55b88c116420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.383+0000 7f9d8bfff640 1 --2- 172.21.15.1:0/793202763 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b88c112410 0x55b88c1253c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.383+0000 7f9d9259c640 1 -- 172.21.15.1:0/793202763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b88c11ccb0 msgr2=0x55b88c116420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.383+0000 7f9d9259c640 1 --2- 172.21.15.1:0/793202763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b88c11ccb0 0x55b88c116420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.383+0000 7f9d9259c640 1 -- 172.21.15.1:0/793202763 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b88c112410 msgr2=0x55b88c1253c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.383+0000 7f9d9259c640 1 --2- 172.21.15.1:0/793202763 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b88c112410 0x55b88c1253c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.383+0000 7f9d9259c640 1 -- 172.21.15.1:0/793202763 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b88c1092d0 con 0x55b88c111af0 2024-09-11T22:04:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.383+0000 7f9d9259c640 1 --2- 172.21.15.1:0/793202763 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b88c111af0 0x55b88c124e80 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c0096f0 tx=0x7f9d7c0167c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70f655bb3515352b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.383+0000 7f9d81ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:04:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.384+0000 7f9d81ffb640 1 -- 172.21.15.1:0/793202763 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c014770 con 0x55b88c111af0 2024-09-11T22:04:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.384+0000 7f9d81ffb640 1 -- 172.21.15.1:0/793202763 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7f9d7c014910 con 0x55b88c111af0 2024-09-11T22:04:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.384+0000 7f9d9ba8c740 1 -- 172.21.15.1:0/793202763 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d7c026030 con 0x55b88c111af0 2024-09-11T22:04:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.384+0000 7f9d81ffb640 1 -- 172.21.15.1:0/793202763 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f9d7c016b70 con 0x55b88c111af0 2024-09-11T22:04:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.384+0000 7f9d9ba8c740 10 client.17082 fetch_fsmap learned FSMap version 42 2024-09-11T22:04:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.384+0000 7f9d9ba8c740 1 -- 172.21.15.1:0/793202763 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55b88c197020 con 0x55b88c111af0 2024-09-11T22:04:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.384+0000 7f9d81ffb640 1 -- 172.21.15.1:0/793202763 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d7c03b420 con 0x55b88c111af0 2024-09-11T22:04:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.385+0000 7f9d81ffb640 1 -- 172.21.15.1:0/793202763 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f9d7c032030 con 0x55b88c111af0 2024-09-11T22:04:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.385+0000 7f9d9ba8c740 10 client.17082 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:04:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.385+0000 7f9d9ba8c740 10 client.17082 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:04:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.385+0000 7f9d9ba8c740 20 client.17082 populate_metadata read hostname 'smithi001' 2024-09-11T22:04:40.391 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.385+0000 7f9d9ba8c740 1 --2- 172.21.15.1:0/793202763 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55b88c197650 0x55b88c199a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:40.391 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.385+0000 7f9d9ba8c740 4 client.17082 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:04:40.391 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.385+0000 7f9d9ba8c740 1 -- 172.21.15.1:0/793202763 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f9d7c026030 con 0x55b88c197650 2024-09-11T22:04:40.391 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.385+0000 7f9d8bfff640 1 --2- 172.21.15.1:0/793202763 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55b88c197650 0x55b88c199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:40.391 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.385+0000 7f9d8bfff640 1 --2- 172.21.15.1:0/793202763 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55b88c197650 0x55b88c199a70 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff0ecdbe4d95d99 server_cookie=db087abda7462616 in_seq=0 out_seq=0 2024-09-11T22:04:40.391 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.385+0000 7f9d81ffb640 10 client.17082 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:04:40.546 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.543+0000 7f9d81ffb640 1 -- 172.21.15.1:0/793202763 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f9d7c026030 con 0x55b88c197650 2024-09-11T22:04:40.546 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.543+0000 7f9d81ffb640 10 client.17082 handle_command_reply: tid=0 2024-09-11T22:04:40.546 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.543+0000 7f9d9ba8c740 1 client.17082 shutdown 2024-09-11T22:04:40.546 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.544+0000 7f9d9ba8c740 1 -- 172.21.15.1:0/793202763 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b88c111af0 msgr2=0x55b88c124e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:40.546 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.544+0000 7f9d9ba8c740 1 --2- 172.21.15.1:0/793202763 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b88c111af0 0x55b88c124e80 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c0096f0 tx=0x7f9d7c0167c0 comp rx=0 tx=0).stop 2024-09-11T22:04:40.547 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.544+0000 7f9d9ba8c740 1 -- 172.21.15.1:0/793202763 shutdown_connections 2024-09-11T22:04:40.547 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.544+0000 7f9d9ba8c740 1 --2- 172.21.15.1:0/793202763 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55b88c197650 0x55b88c199a70 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:40.547 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.544+0000 7f9d9ba8c740 1 --2- 172.21.15.1:0/793202763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b88c11ccb0 0x55b88c116420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:40.547 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.544+0000 7f9d81ffb640 0 client.17082 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:04:40.547 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.544+0000 7f9d9ba8c740 1 --2- 172.21.15.1:0/793202763 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b88c112410 0x55b88c1253c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:40.547 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.544+0000 7f9d9ba8c740 1 --2- 172.21.15.1:0/793202763 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b88c111af0 0x55b88c124e80 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:40.547 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.544+0000 7f9d9ba8c740 1 -- 172.21.15.1:0/793202763 >> 172.21.15.1:0/793202763 conn(0x55b88c10e060 msgr2=0x55b88c113dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:40.547 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.544+0000 7f9d9ba8c740 1 -- 172.21.15.1:0/793202763 shutdown_connections 2024-09-11T22:04:40.547 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.544+0000 7f9d9ba8c740 1 -- 172.21.15.1:0/793202763 wait complete. 2024-09-11T22:04:40.547 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.544+0000 7f9d9ba8c740 20 client.17082 trim_cache size 0 max 16384 2024-09-11T22:04:40.548 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-09-11T22:04:40.548 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-09-11T22:04:40.548 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-09-11T22:04:40.548 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-09-11T22:04:40.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.544+0000 7f9d98cf1640 1 -- 172.21.15.1:0/994539246 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f9d740a2cf0 msgr2=0x7f9d740a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:40.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.544+0000 7f9d98cf1640 1 --2- 172.21.15.1:0/994539246 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f9d740a2cf0 0x7f9d740a51b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x55b88c10eec0 tx=0x7f9d7c008040 comp rx=0 tx=0).stop 2024-09-11T22:04:40.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.544+0000 7f9d98cf1640 1 -- 172.21.15.1:0/994539246 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d941035c0 msgr2=0x7f9d94198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:40.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.544+0000 7f9d98cf1640 1 --2- 172.21.15.1:0/994539246 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d941035c0 0x7f9d94198a00 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f9d84006010 tx=0x7f9d84004040 comp rx=0 tx=0).stop 2024-09-11T22:04:40.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.545+0000 7f9d98cf1640 1 -- 172.21.15.1:0/994539246 shutdown_connections 2024-09-11T22:04:40.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.545+0000 7f9d98cf1640 1 --2- 172.21.15.1:0/994539246 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f9d740a2cf0 0x7f9d740a51b0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:40.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.545+0000 7f9d98cf1640 1 --2- 172.21.15.1:0/994539246 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d9410c0f0 0x7f9d9419cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:40.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.545+0000 7f9d98cf1640 1 --2- 172.21.15.1:0/994539246 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d941035c0 0x7f9d94198a00 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:40.550 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.545+0000 7f9d98cf1640 1 --2- 172.21.15.1:0/994539246 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d94102ca0 0x7f9d941984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:40.550 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.545+0000 7f9d98cf1640 1 -- 172.21.15.1:0/994539246 >> 172.21.15.1:0/994539246 conn(0x7f9d940fc6b0 msgr2=0x7f9d940fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:40.550 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.545+0000 7f9d98cf1640 1 -- 172.21.15.1:0/994539246 shutdown_connections 2024-09-11T22:04:40.550 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:40.545+0000 7f9d98cf1640 1 -- 172.21.15.1:0/994539246 wait complete. 2024-09-11T22:04:41.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:04:40 smithi053 ceph-mon[34908]: pgmap v230: 97 pgs: 97 active+clean; 446 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 333 KiB/s wr, 127 op/s 2024-09-11T22:04:41.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:04:40 smithi161 ceph-mon[35615]: pgmap v230: 97 pgs: 97 active+clean; 446 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 333 KiB/s wr, 127 op/s 2024-09-11T22:04:41.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:04:40 smithi001 ceph-mon[30935]: pgmap v230: 97 pgs: 97 active+clean; 446 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 333 KiB/s wr, 127 op/s 2024-09-11T22:04:43.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:04:43 smithi053 ceph-mon[34908]: pgmap v231: 97 pgs: 97 active+clean; 446 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 247 KiB/s wr, 26 op/s 2024-09-11T22:04:43.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:04:43 smithi161 ceph-mon[35615]: pgmap v231: 97 pgs: 97 active+clean; 446 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 247 KiB/s wr, 26 op/s 2024-09-11T22:04:43.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:04:43 smithi001 ceph-mon[30935]: pgmap v231: 97 pgs: 97 active+clean; 446 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 247 KiB/s wr, 26 op/s 2024-09-11T22:04:45.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:04:45 smithi053 ceph-mon[34908]: pgmap v232: 97 pgs: 97 active+clean; 446 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 247 KiB/s wr, 26 op/s 2024-09-11T22:04:45.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:04:45 smithi161 ceph-mon[35615]: pgmap v232: 97 pgs: 97 active+clean; 446 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 247 KiB/s wr, 26 op/s 2024-09-11T22:04:45.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:04:45 smithi001 ceph-mon[30935]: pgmap v232: 97 pgs: 97 active+clean; 446 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 247 KiB/s wr, 26 op/s 2024-09-11T22:04:47.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:04:47 smithi053 ceph-mon[34908]: pgmap v233: 97 pgs: 97 active+clean; 457 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 1.2 MiB/s wr, 192 op/s 2024-09-11T22:04:47.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:04:47 smithi161 ceph-mon[35615]: pgmap v233: 97 pgs: 97 active+clean; 457 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 1.2 MiB/s wr, 192 op/s 2024-09-11T22:04:47.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:04:47 smithi001 ceph-mon[30935]: pgmap v233: 97 pgs: 97 active+clean; 457 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 1.2 MiB/s wr, 192 op/s 2024-09-11T22:04:48.955 INFO:tasks.workunit.client.0.smithi001.stdout:cleaned up hierarchy 2024-09-11T22:04:49.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:04:49 smithi053 ceph-mon[34908]: pgmap v234: 97 pgs: 97 active+clean; 457 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 1.2 MiB/s wr, 186 op/s 2024-09-11T22:04:49.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:04:49 smithi161 ceph-mon[35615]: pgmap v234: 97 pgs: 97 active+clean; 457 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 1.2 MiB/s wr, 186 op/s 2024-09-11T22:04:49.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:04:49 smithi001 ceph-mon[30935]: pgmap v234: 97 pgs: 97 active+clean; 457 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 1.2 MiB/s wr, 186 op/s 2024-09-11T22:04:51.261 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:04:51 smithi001 ceph-mon[30935]: pgmap v235: 97 pgs: 97 active+clean; 465 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 283 op/s 2024-09-11T22:04:51.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:04:51 smithi053 ceph-mon[34908]: pgmap v235: 97 pgs: 97 active+clean; 465 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 283 op/s 2024-09-11T22:04:51.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:04:51 smithi161 ceph-mon[35615]: pgmap v235: 97 pgs: 97 active+clean; 465 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 283 op/s 2024-09-11T22:04:51.486 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-11T22:04:51.486 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-11T22:04:51.525 INFO:tasks.workunit:Running workunit fs/misc/filelock_deadlock.py... 2024-09-11T22:04:51.525 DEBUG:teuthology.orchestra.run.smithi001:workunit test fs/misc/filelock_deadlock.py> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=086c737fa75c22d521a3932c59b31c5e1d1c9ef4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/filelock_deadlock.py 2024-09-11T22:04:53.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:04:53 smithi053 ceph-mon[34908]: pgmap v236: 97 pgs: 97 active+clean; 465 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.5 MiB/s wr, 263 op/s 2024-09-11T22:04:53.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:04:53 smithi161 ceph-mon[35615]: pgmap v236: 97 pgs: 97 active+clean; 465 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.5 MiB/s wr, 263 op/s 2024-09-11T22:04:53.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:04:53 smithi001 ceph-mon[30935]: pgmap v236: 97 pgs: 97 active+clean; 465 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.5 MiB/s wr, 263 op/s 2024-09-11T22:04:53.650 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.646+0000 7fe267a42640 1 -- 172.21.15.1:0/426735987 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2601008b0 msgr2=0x7fe260100c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:53.650 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.646+0000 7fe267a42640 1 --2- 172.21.15.1:0/426735987 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2601008b0 0x7fe260100c90 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fe25c009900 tx=0x7fe25c02a410 comp rx=0 tx=0).stop 2024-09-11T22:04:53.650 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.646+0000 7fe267a42640 1 -- 172.21.15.1:0/426735987 shutdown_connections 2024-09-11T22:04:53.650 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.646+0000 7fe267a42640 1 --2- 172.21.15.1:0/426735987 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe26010e460 0x7fe260110850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:53.651 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.646+0000 7fe267a42640 1 --2- 172.21.15.1:0/426735987 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe260101260 0x7fe26010de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:53.651 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.646+0000 7fe267a42640 1 --2- 172.21.15.1:0/426735987 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2601008b0 0x7fe260100c90 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:53.651 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.646+0000 7fe267a42640 1 -- 172.21.15.1:0/426735987 >> 172.21.15.1:0/426735987 conn(0x7fe2600fc6b0 msgr2=0x7fe2600fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:53.651 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.646+0000 7fe267a42640 1 -- 172.21.15.1:0/426735987 shutdown_connections 2024-09-11T22:04:53.651 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.646+0000 7fe267a42640 1 -- 172.21.15.1:0/426735987 wait complete. 2024-09-11T22:04:53.651 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.647+0000 7fe267a42640 1 Processor -- start 2024-09-11T22:04:53.651 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.647+0000 7fe267a42640 1 -- start start 2024-09-11T22:04:53.651 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.647+0000 7fe267a42640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2601008b0 0x7fe26019a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:53.651 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.647+0000 7fe267a42640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe260101260 0x7fe26019ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:53.652 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.647+0000 7fe267a42640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe26010e460 0x7fe26019ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:53.652 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.647+0000 7fe267a42640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe2601140e0 con 0x7fe2601008b0 2024-09-11T22:04:53.652 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.647+0000 7fe267a42640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe260113f60 con 0x7fe26010e460 2024-09-11T22:04:53.652 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.647+0000 7fe267a42640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe260114260 con 0x7fe260101260 2024-09-11T22:04:53.652 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.647+0000 7fe2657de640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2601008b0 0x7fe26019a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:53.652 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.648+0000 7fe2657de640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2601008b0 0x7fe26019a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56148/0 (socket says 172.21.15.1:56148) 2024-09-11T22:04:53.652 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.648+0000 7fe2657de640 1 -- 172.21.15.1:0/416064637 learned_addr learned my addr 172.21.15.1:0/416064637 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:53.652 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.648+0000 7fe264fdd640 1 --2- 172.21.15.1:0/416064637 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe260101260 0x7fe26019ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.648+0000 7fe265fdf640 1 --2- 172.21.15.1:0/416064637 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe26010e460 0x7fe26019ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.648+0000 7fe2657de640 1 -- 172.21.15.1:0/416064637 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe260101260 msgr2=0x7fe26019ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.648+0000 7fe2657de640 1 --2- 172.21.15.1:0/416064637 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe260101260 0x7fe26019ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.648+0000 7fe2657de640 1 -- 172.21.15.1:0/416064637 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe26010e460 msgr2=0x7fe26019ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.648+0000 7fe2657de640 1 --2- 172.21.15.1:0/416064637 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe26010e460 0x7fe26019ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.648+0000 7fe2657de640 1 -- 172.21.15.1:0/416064637 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe26019f5c0 con 0x7fe2601008b0 2024-09-11T22:04:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.648+0000 7fe2657de640 1 --2- 172.21.15.1:0/416064637 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2601008b0 0x7fe26019a5d0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fe25c006010 tx=0x7fe25c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.648+0000 7fe2567fc640 1 -- 172.21.15.1:0/416064637 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe25c03f030 con 0x7fe2601008b0 2024-09-11T22:04:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.648+0000 7fe2567fc640 1 -- 172.21.15.1:0/416064637 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe25c003ce0 con 0x7fe2601008b0 2024-09-11T22:04:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.648+0000 7fe267a42640 1 -- 172.21.15.1:0/416064637 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe26019f850 con 0x7fe2601008b0 2024-09-11T22:04:53.654 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.648+0000 7fe2567fc640 1 -- 172.21.15.1:0/416064637 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe25c035070 con 0x7fe2601008b0 2024-09-11T22:04:53.654 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.648+0000 7fe267a42640 1 -- 172.21.15.1:0/416064637 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2601a7140 con 0x7fe2601008b0 2024-09-11T22:04:53.662 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.651+0000 7fe2567fc640 1 -- 172.21.15.1:0/416064637 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7fe25c042030 con 0x7fe2601008b0 2024-09-11T22:04:53.662 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.651+0000 7fe26a7d2740 1 Processor -- start 2024-09-11T22:04:53.662 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.651+0000 7fe26a7d2740 1 -- start start 2024-09-11T22:04:53.662 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.652+0000 7fe26a7d2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561caec22af0 0x561caec22ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:53.662 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.652+0000 7fe26a7d2740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561caec23410 0x561caec2d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:53.662 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.652+0000 7fe26a7d2740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561caec2dcb0 0x561caec300a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:53.663 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.652+0000 7fe26a7d2740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x561caeb872f0 con 0x561caec23410 2024-09-11T22:04:53.663 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.652+0000 7fe26a7d2740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x561caeb2f270 con 0x561caec2dcb0 2024-09-11T22:04:53.663 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.652+0000 7fe26a7d2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x561caeb832b0 con 0x561caec22af0 2024-09-11T22:04:53.663 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.652+0000 7fe2657de640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561caec23410 0x561caec2d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:53.663 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.652+0000 7fe2657de640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561caec23410 0x561caec2d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56154/0 (socket says 172.21.15.1:56154) 2024-09-11T22:04:53.663 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.652+0000 7fe2657de640 1 -- 172.21.15.1:0/1274861855 learned_addr learned my addr 172.21.15.1:0/1274861855 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:53.663 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.652+0000 7fe2567fc640 1 --2- 172.21.15.1:0/416064637 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fe24c0a2d70 0x7fe24c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:53.663 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.652+0000 7fe265fdf640 1 --2- 172.21.15.1:0/1274861855 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561caec22af0 0x561caec22ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.652+0000 7fe264fdd640 1 --2- 172.21.15.1:0/1274861855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561caec2dcb0 0x561caec300a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.652+0000 7fe265fdf640 1 --2- 172.21.15.1:0/416064637 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fe24c0a2d70 0x7fe24c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.652+0000 7fe2567fc640 1 -- 172.21.15.1:0/416064637 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7fe25c0ef9b0 con 0x7fe2601008b0 2024-09-11T22:04:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.652+0000 7fe2657de640 1 -- 172.21.15.1:0/1274861855 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561caec22af0 msgr2=0x561caec22ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.652+0000 7fe2657de640 1 --2- 172.21.15.1:0/1274861855 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561caec22af0 0x561caec22ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.652+0000 7fe2657de640 1 -- 172.21.15.1:0/1274861855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561caec2dcb0 msgr2=0x561caec300a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.652+0000 7fe2657de640 1 --2- 172.21.15.1:0/1274861855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561caec2dcb0 0x561caec300a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.652+0000 7fe2657de640 1 -- 172.21.15.1:0/1274861855 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561caeb2e340 con 0x561caec23410 2024-09-11T22:04:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.652+0000 7fe2657de640 1 --2- 172.21.15.1:0/1274861855 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561caec23410 0x561caec2d770 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fe25c0c4600 tx=0x7fe25c0b8170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf2df3cbc70cae5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:53.665 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.653+0000 7fe265fdf640 1 --2- 172.21.15.1:0/416064637 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fe24c0a2d70 0x7fe24c0a5230 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe258009fd0 tx=0x7fe258009600 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:53.665 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.653+0000 7fe24b7fe640 1 -- 172.21.15.1:0/1274861855 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe25c049070 con 0x561caec23410 2024-09-11T22:04:53.665 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.653+0000 7fe24b7fe640 1 -- 172.21.15.1:0/1274861855 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe25c0bbf70 con 0x561caec23410 2024-09-11T22:04:53.665 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.653+0000 7fe24b7fe640 1 -- 172.21.15.1:0/1274861855 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe25c0c0270 con 0x561caec23410 2024-09-11T22:04:53.665 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.653+0000 7fe26a7d2740 1 -- 172.21.15.1:0/1274861855 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561caec23410 msgr2=0x561caec2d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:53.665 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.653+0000 7fe26a7d2740 1 --2- 172.21.15.1:0/1274861855 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561caec23410 0x561caec2d770 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fe25c0c4600 tx=0x7fe25c0b8170 comp rx=0 tx=0).stop 2024-09-11T22:04:53.665 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.653+0000 7fe26a7d2740 1 -- 172.21.15.1:0/1274861855 shutdown_connections 2024-09-11T22:04:53.665 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.653+0000 7fe26a7d2740 1 --2- 172.21.15.1:0/1274861855 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561caec2dcb0 0x561caec300a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:53.665 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.654+0000 7fe26a7d2740 1 --2- 172.21.15.1:0/1274861855 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561caec23410 0x561caec2d770 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:53.665 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.654+0000 7fe26a7d2740 1 --2- 172.21.15.1:0/1274861855 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561caec22af0 0x561caec22ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:53.665 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.654+0000 7fe26a7d2740 1 -- 172.21.15.1:0/1274861855 >> 172.21.15.1:0/1274861855 conn(0x561caec1f060 msgr2=0x561caec1f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:53.666 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.654+0000 7fe26a7d2740 1 -- 172.21.15.1:0/1274861855 shutdown_connections 2024-09-11T22:04:53.666 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.654+0000 7fe26a7d2740 1 -- 172.21.15.1:0/1274861855 wait complete. 2024-09-11T22:04:53.666 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.654+0000 7fe26a7d2740 1 -- start start 2024-09-11T22:04:53.666 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.654+0000 7fe26a7d2740 1 Processor -- start 2024-09-11T22:04:53.666 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.655+0000 7fe26a7d2740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561caec22af0 0x561caec2ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:53.666 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.655+0000 7fe26a7d2740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561caec23410 0x561caec2b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:53.666 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.655+0000 7fe26a7d2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561caec2dcb0 0x561caec2b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:53.666 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.655+0000 7fe26a7d2740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x561caec30640 con 0x561caec22af0 2024-09-11T22:04:53.666 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.655+0000 7fe26a7d2740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x561caeb10290 con 0x561caec23410 2024-09-11T22:04:53.666 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.655+0000 7fe26a7d2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x561caec307c0 con 0x561caec2dcb0 2024-09-11T22:04:53.666 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.655+0000 7fe26a7d2740 10 client.0 _ll_register_callbacks cb 0x561caeb41140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:04:53.667 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.655+0000 7fe2657de640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561caec22af0 0x561caec2ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:53.667 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.655+0000 7fe2657de640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561caec22af0 0x561caec2ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56158/0 (socket says 172.21.15.1:56158) 2024-09-11T22:04:53.667 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.655+0000 7fe2657de640 1 -- 172.21.15.1:0/2984517351 learned_addr learned my addr 172.21.15.1:0/2984517351 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:53.667 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.655+0000 7fe264fdd640 1 --2- 172.21.15.1:0/2984517351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561caec23410 0x561caec2b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:53.667 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.655+0000 7fe265fdf640 1 --2- 172.21.15.1:0/2984517351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561caec2dcb0 0x561caec2b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:53.667 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.655+0000 7fe2657de640 1 -- 172.21.15.1:0/2984517351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561caec2dcb0 msgr2=0x561caec2b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:53.667 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.655+0000 7fe2657de640 1 --2- 172.21.15.1:0/2984517351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561caec2dcb0 0x561caec2b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:53.668 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.655+0000 7fe2657de640 1 -- 172.21.15.1:0/2984517351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561caec23410 msgr2=0x561caec2b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:53.668 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.655+0000 7fe2657de640 1 --2- 172.21.15.1:0/2984517351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561caec23410 0x561caec2b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:53.668 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.655+0000 7fe2657de640 1 -- 172.21.15.1:0/2984517351 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cae89a210 con 0x561caec22af0 2024-09-11T22:04:53.668 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.655+0000 7fe265fdf640 1 --2- 172.21.15.1:0/2984517351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561caec2dcb0 0x561caec2b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:04:53.668 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.655+0000 7fe2657de640 1 --2- 172.21.15.1:0/2984517351 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561caec22af0 0x561caec2ac20 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fe25c031aa0 tx=0x7fe25c009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c8032f257223714 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:53.668 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.655+0000 7fe248ff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:04:53.668 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.655+0000 7fe248ff9640 1 -- 172.21.15.1:0/2984517351 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe25c049030 con 0x561caec22af0 2024-09-11T22:04:53.668 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.655+0000 7fe248ff9640 1 -- 172.21.15.1:0/2984517351 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7fe25c0ed2b0 con 0x561caec22af0 2024-09-11T22:04:53.668 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.655+0000 7fe26a7d2740 1 -- 172.21.15.1:0/2984517351 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe25c0bbf70 con 0x561caec22af0 2024-09-11T22:04:53.668 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.656+0000 7fe248ff9640 1 -- 172.21.15.1:0/2984517351 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fe25c0bbab0 con 0x561caec22af0 2024-09-11T22:04:53.669 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.656+0000 7fe26a7d2740 10 client.17094 fetch_fsmap learned FSMap version 42 2024-09-11T22:04:53.669 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.656+0000 7fe26a7d2740 1 -- 172.21.15.1:0/2984517351 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x561cae966620 con 0x561caec22af0 2024-09-11T22:04:53.669 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.656+0000 7fe248ff9640 1 -- 172.21.15.1:0/2984517351 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe25c0bbf70 con 0x561caec22af0 2024-09-11T22:04:53.669 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.656+0000 7fe248ff9640 1 -- 172.21.15.1:0/2984517351 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7fe25c0c46a0 con 0x561caec22af0 2024-09-11T22:04:53.669 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.656+0000 7fe26a7d2740 10 client.17094 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:04:53.669 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.656+0000 7fe26a7d2740 10 client.17094 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:04:53.669 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.656+0000 7fe26a7d2740 20 client.17094 populate_metadata read hostname 'smithi001' 2024-09-11T22:04:53.669 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.656+0000 7fe26a7d2740 1 --2- 172.21.15.1:0/2984517351 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x561caec3d800 0x561caec3fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:53.669 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.656+0000 7fe26a7d2740 4 client.17094 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:04:53.669 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.656+0000 7fe26a7d2740 1 -- 172.21.15.1:0/2984517351 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561caeb94a20 con 0x561caec3d800 2024-09-11T22:04:53.669 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.656+0000 7fe264fdd640 1 --2- 172.21.15.1:0/2984517351 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x561caec3d800 0x561caec3fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:53.669 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.657+0000 7fe248ff9640 10 client.17094 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:04:53.670 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.657+0000 7fe264fdd640 1 --2- 172.21.15.1:0/2984517351 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x561caec3d800 0x561caec3fc20 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42b1dbed4af52484 server_cookie=2390bd43d8b0e7ca in_seq=0 out_seq=0 2024-09-11T22:04:53.670 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.657+0000 7fe248ff9640 1 -- 172.21.15.1:0/2984517351 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561caeb94a20 con 0x561caec3d800 2024-09-11T22:04:53.670 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.657+0000 7fe248ff9640 10 client.17094 handle_command_reply: tid=0 2024-09-11T22:04:53.670 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.657+0000 7fe26a7d2740 1 client.17094 shutdown 2024-09-11T22:04:53.670 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.658+0000 7fe26a7d2740 1 -- 172.21.15.1:0/2984517351 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561caec22af0 msgr2=0x561caec2ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:53.670 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.658+0000 7fe26a7d2740 1 --2- 172.21.15.1:0/2984517351 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561caec22af0 0x561caec2ac20 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fe25c031aa0 tx=0x7fe25c009560 comp rx=0 tx=0).stop 2024-09-11T22:04:53.670 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.658+0000 7fe26a7d2740 1 -- 172.21.15.1:0/2984517351 shutdown_connections 2024-09-11T22:04:53.670 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.658+0000 7fe26a7d2740 1 --2- 172.21.15.1:0/2984517351 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x561caec3d800 0x561caec3fc20 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:53.670 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.658+0000 7fe26a7d2740 1 --2- 172.21.15.1:0/2984517351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561caec2dcb0 0x561caec2b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:53.670 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.658+0000 7fe26a7d2740 1 --2- 172.21.15.1:0/2984517351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561caec23410 0x561caec2b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:53.671 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.658+0000 7fe26a7d2740 1 --2- 172.21.15.1:0/2984517351 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561caec22af0 0x561caec2ac20 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:53.671 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.658+0000 7fe248ff9640 0 client.17094 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:04:53.671 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.658+0000 7fe26a7d2740 1 -- 172.21.15.1:0/2984517351 >> 172.21.15.1:0/2984517351 conn(0x561caec1f060 msgr2=0x561caec24d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:53.671 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.658+0000 7fe26a7d2740 1 -- 172.21.15.1:0/2984517351 shutdown_connections 2024-09-11T22:04:53.671 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.658+0000 7fe26a7d2740 1 -- 172.21.15.1:0/2984517351 wait complete. 2024-09-11T22:04:53.671 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.658+0000 7fe26a7d2740 20 client.17094 trim_cache size 0 max 16384 2024-09-11T22:04:53.678 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.672+0000 7fe26a7d2740 1 Processor -- start 2024-09-11T22:04:53.679 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.672+0000 7fe26a7d2740 1 -- start start 2024-09-11T22:04:53.679 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.673+0000 7fe26a7d2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561caec22af0 0x561caec2b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:53.679 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.673+0000 7fe26a7d2740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561caec23410 0x561caec45530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:53.679 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.673+0000 7fe26a7d2740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561caec2dcb0 0x561caec45a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:53.679 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.673+0000 7fe26a7d2740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe2601afc20 con 0x561caec23410 2024-09-11T22:04:53.679 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.673+0000 7fe26a7d2740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe2601b1a60 con 0x561caec2dcb0 2024-09-11T22:04:53.679 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.673+0000 7fe26a7d2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe2601b03b0 con 0x561caec22af0 2024-09-11T22:04:53.679 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.673+0000 7fe264fdd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561caec23410 0x561caec45530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:53.680 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.673+0000 7fe264fdd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561caec23410 0x561caec45530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56166/0 (socket says 172.21.15.1:56166) 2024-09-11T22:04:53.680 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.673+0000 7fe264fdd640 1 -- 172.21.15.1:0/92274909 learned_addr learned my addr 172.21.15.1:0/92274909 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:53.680 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.673+0000 7fe265fdf640 1 --2- 172.21.15.1:0/92274909 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561caec2dcb0 0x561caec45a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:53.680 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.673+0000 7fe2657de640 1 --2- 172.21.15.1:0/92274909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561caec22af0 0x561caec2b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:53.680 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.673+0000 7fe264fdd640 1 -- 172.21.15.1:0/92274909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561caec22af0 msgr2=0x561caec2b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:53.680 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.673+0000 7fe264fdd640 1 --2- 172.21.15.1:0/92274909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561caec22af0 0x561caec2b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:53.680 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.673+0000 7fe264fdd640 1 -- 172.21.15.1:0/92274909 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561caec2dcb0 msgr2=0x561caec45a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:53.680 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.673+0000 7fe264fdd640 1 --2- 172.21.15.1:0/92274909 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561caec2dcb0 0x561caec45a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:53.680 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.673+0000 7fe264fdd640 1 -- 172.21.15.1:0/92274909 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561caec1d790 con 0x561caec23410 2024-09-11T22:04:53.680 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.673+0000 7fe265fdf640 1 --2- 172.21.15.1:0/92274909 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561caec2dcb0 0x561caec45a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:04:53.680 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.673+0000 7fe2657de640 1 --2- 172.21.15.1:0/92274909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561caec22af0 0x561caec2b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:04:53.681 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.673+0000 7fe264fdd640 1 --2- 172.21.15.1:0/92274909 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561caec23410 0x561caec45530 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x561caec203e0 tx=0x7fe250005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6145754c291b77fd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:53.681 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.673+0000 7fe24affd640 1 -- 172.21.15.1:0/92274909 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe250022020 con 0x561caec23410 2024-09-11T22:04:53.681 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.673+0000 7fe24affd640 1 -- 172.21.15.1:0/92274909 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe250003de0 con 0x561caec23410 2024-09-11T22:04:53.681 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.674+0000 7fe24affd640 1 -- 172.21.15.1:0/92274909 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe25001e840 con 0x561caec23410 2024-09-11T22:04:53.681 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.674+0000 7fe26a7d2740 1 -- 172.21.15.1:0/92274909 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561caec23410 msgr2=0x561caec45530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:53.681 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.674+0000 7fe26a7d2740 1 --2- 172.21.15.1:0/92274909 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561caec23410 0x561caec45530 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x561caec203e0 tx=0x7fe250005dc0 comp rx=0 tx=0).stop 2024-09-11T22:04:53.681 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.674+0000 7fe26a7d2740 1 -- 172.21.15.1:0/92274909 shutdown_connections 2024-09-11T22:04:53.681 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.674+0000 7fe26a7d2740 1 --2- 172.21.15.1:0/92274909 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561caec2dcb0 0x561caec45a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:53.681 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.674+0000 7fe26a7d2740 1 --2- 172.21.15.1:0/92274909 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561caec23410 0x561caec45530 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:53.681 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.674+0000 7fe26a7d2740 1 --2- 172.21.15.1:0/92274909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561caec22af0 0x561caec2b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:53.681 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.674+0000 7fe26a7d2740 1 -- 172.21.15.1:0/92274909 >> 172.21.15.1:0/92274909 conn(0x561caec1f060 msgr2=0x561caec2cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:53.681 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.674+0000 7fe26a7d2740 1 -- 172.21.15.1:0/92274909 shutdown_connections 2024-09-11T22:04:53.682 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.674+0000 7fe26a7d2740 1 -- 172.21.15.1:0/92274909 wait complete. 2024-09-11T22:04:53.682 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.674+0000 7fe26a7d2740 1 -- start start 2024-09-11T22:04:53.682 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.675+0000 7fe26a7d2740 1 Processor -- start 2024-09-11T22:04:53.682 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.675+0000 7fe26a7d2740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561caec22af0 0x561caec36100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:53.682 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.675+0000 7fe26a7d2740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561caec23410 0x561caec27b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:53.682 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.675+0000 7fe26a7d2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561caec2dcb0 0x561caec282d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:53.682 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.675+0000 7fe26a7d2740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe2601b12d0 con 0x561caec23410 2024-09-11T22:04:53.682 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.675+0000 7fe26a7d2740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe2601b0b40 con 0x561caec22af0 2024-09-11T22:04:53.682 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.675+0000 7fe26a7d2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe2601adc20 con 0x561caec2dcb0 2024-09-11T22:04:53.682 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.675+0000 7fe26a7d2740 10 client.0 _ll_register_callbacks cb 0x7fe2500097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:04:53.683 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.675+0000 7fe264fdd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561caec23410 0x561caec27b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:53.683 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.675+0000 7fe264fdd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561caec23410 0x561caec27b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56172/0 (socket says 172.21.15.1:56172) 2024-09-11T22:04:53.683 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.675+0000 7fe264fdd640 1 -- 172.21.15.1:0/3621845259 learned_addr learned my addr 172.21.15.1:0/3621845259 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:53.683 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.675+0000 7fe265fdf640 1 --2- 172.21.15.1:0/3621845259 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561caec2dcb0 0x561caec282d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:53.683 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.675+0000 7fe2657de640 1 --2- 172.21.15.1:0/3621845259 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561caec22af0 0x561caec36100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:53.683 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.675+0000 7fe264fdd640 1 -- 172.21.15.1:0/3621845259 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561caec2dcb0 msgr2=0x561caec282d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:53.683 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.675+0000 7fe264fdd640 1 --2- 172.21.15.1:0/3621845259 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561caec2dcb0 0x561caec282d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:53.683 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.675+0000 7fe264fdd640 1 -- 172.21.15.1:0/3621845259 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561caec22af0 msgr2=0x561caec36100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:53.683 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.675+0000 7fe264fdd640 1 --2- 172.21.15.1:0/3621845259 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561caec22af0 0x561caec36100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:53.683 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.675+0000 7fe264fdd640 1 -- 172.21.15.1:0/3621845259 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561caeca7500 con 0x561caec23410 2024-09-11T22:04:53.683 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.676+0000 7fe264fdd640 1 --2- 172.21.15.1:0/3621845259 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561caec23410 0x561caec27b40 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fe250003d40 tx=0x7fe250003d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f4be57ec2616a00 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:53.683 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.676+0000 7fe248ff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:04:53.684 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.676+0000 7fe248ff9640 1 -- 172.21.15.1:0/3621845259 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe250014070 con 0x561caec23410 2024-09-11T22:04:53.684 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.676+0000 7fe248ff9640 1 -- 172.21.15.1:0/3621845259 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7fe250005550 con 0x561caec23410 2024-09-11T22:04:53.684 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.676+0000 7fe26a7d2740 1 -- 172.21.15.1:0/3621845259 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe250003de0 con 0x561caec23410 2024-09-11T22:04:53.684 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.676+0000 7fe248ff9640 1 -- 172.21.15.1:0/3621845259 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fe250004700 con 0x561caec23410 2024-09-11T22:04:53.684 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.676+0000 7fe26a7d2740 10 client.17100 fetch_fsmap learned FSMap version 42 2024-09-11T22:04:53.684 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.676+0000 7fe26a7d2740 1 -- 172.21.15.1:0/3621845259 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x561caeca8610 con 0x561caec23410 2024-09-11T22:04:53.684 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.676+0000 7fe248ff9640 1 -- 172.21.15.1:0/3621845259 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe250003de0 con 0x561caec23410 2024-09-11T22:04:53.684 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.676+0000 7fe248ff9640 1 -- 172.21.15.1:0/3621845259 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7fe25001ebe0 con 0x561caec23410 2024-09-11T22:04:53.684 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.676+0000 7fe26a7d2740 10 client.17100 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:04:53.684 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.676+0000 7fe26a7d2740 10 client.17100 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:04:53.684 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.676+0000 7fe26a7d2740 20 client.17100 populate_metadata read hostname 'smithi001' 2024-09-11T22:04:53.685 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.676+0000 7fe26a7d2740 1 --2- 172.21.15.1:0/3621845259 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x561caeca8d40 0x561caecab160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:53.685 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.676+0000 7fe26a7d2740 4 client.17100 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:04:53.685 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.676+0000 7fe26a7d2740 1 -- 172.21.15.1:0/3621845259 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561caeb8ed70 con 0x561caeca8d40 2024-09-11T22:04:53.685 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.677+0000 7fe2657de640 1 --2- 172.21.15.1:0/3621845259 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x561caeca8d40 0x561caecab160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:53.685 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.677+0000 7fe2657de640 1 --2- 172.21.15.1:0/3621845259 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x561caeca8d40 0x561caecab160 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d93d9a8d51f17e7 server_cookie=3bb18bf79bf31e38 in_seq=0 out_seq=0 2024-09-11T22:04:53.685 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.677+0000 7fe248ff9640 10 client.17100 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:04:53.852 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.850+0000 7fe248ff9640 1 -- 172.21.15.1:0/3621845259 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561caeb8ed70 con 0x561caeca8d40 2024-09-11T22:04:53.852 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.850+0000 7fe248ff9640 10 client.17100 handle_command_reply: tid=0 2024-09-11T22:04:53.852 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.850+0000 7fe26a7d2740 1 client.17100 shutdown 2024-09-11T22:04:53.852 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.850+0000 7fe26a7d2740 1 -- 172.21.15.1:0/3621845259 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561caec23410 msgr2=0x561caec27b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:53.852 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.850+0000 7fe26a7d2740 1 --2- 172.21.15.1:0/3621845259 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561caec23410 0x561caec27b40 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fe250003d40 tx=0x7fe250003d70 comp rx=0 tx=0).stop 2024-09-11T22:04:53.852 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.850+0000 7fe26a7d2740 1 -- 172.21.15.1:0/3621845259 shutdown_connections 2024-09-11T22:04:53.852 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.850+0000 7fe26a7d2740 1 --2- 172.21.15.1:0/3621845259 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x561caeca8d40 0x561caecab160 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:53.853 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.850+0000 7fe26a7d2740 1 --2- 172.21.15.1:0/3621845259 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x561caec2dcb0 0x561caec282d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:53.853 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.850+0000 7fe248ff9640 0 client.17100 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:04:53.853 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.850+0000 7fe26a7d2740 1 --2- 172.21.15.1:0/3621845259 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561caec23410 0x561caec27b40 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:53.853 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.850+0000 7fe26a7d2740 1 --2- 172.21.15.1:0/3621845259 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561caec22af0 0x561caec36100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:53.853 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.850+0000 7fe26a7d2740 1 -- 172.21.15.1:0/3621845259 >> 172.21.15.1:0/3621845259 conn(0x561caec1f060 msgr2=0x561caec1d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:53.853 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.850+0000 7fe26a7d2740 1 -- 172.21.15.1:0/3621845259 shutdown_connections 2024-09-11T22:04:53.853 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.850+0000 7fe26a7d2740 1 -- 172.21.15.1:0/3621845259 wait complete. 2024-09-11T22:04:53.853 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.850+0000 7fe26a7d2740 20 client.17100 trim_cache size 0 max 16384 2024-09-11T22:04:53.853 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-09-11T22:04:53.854 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-09-11T22:04:53.854 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-09-11T22:04:53.854 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-09-11T22:04:53.855 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.851+0000 7fe267a42640 1 -- 172.21.15.1:0/416064637 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fe24c0a2d70 msgr2=0x7fe24c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:53.855 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.851+0000 7fe267a42640 1 --2- 172.21.15.1:0/416064637 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fe24c0a2d70 0x7fe24c0a5230 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe258009fd0 tx=0x7fe258009600 comp rx=0 tx=0).stop 2024-09-11T22:04:53.855 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.851+0000 7fe267a42640 1 -- 172.21.15.1:0/416064637 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2601008b0 msgr2=0x7fe26019a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:53.855 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.851+0000 7fe267a42640 1 --2- 172.21.15.1:0/416064637 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2601008b0 0x7fe26019a5d0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fe25c006010 tx=0x7fe25c0042f0 comp rx=0 tx=0).stop 2024-09-11T22:04:53.855 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.851+0000 7fe267a42640 1 -- 172.21.15.1:0/416064637 shutdown_connections 2024-09-11T22:04:53.855 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.851+0000 7fe267a42640 1 --2- 172.21.15.1:0/416064637 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fe24c0a2d70 0x7fe24c0a5230 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:53.855 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.851+0000 7fe267a42640 1 --2- 172.21.15.1:0/416064637 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe26010e460 0x7fe26019ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:53.855 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.851+0000 7fe267a42640 1 --2- 172.21.15.1:0/416064637 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe260101260 0x7fe26019ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:53.855 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.851+0000 7fe267a42640 1 --2- 172.21.15.1:0/416064637 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2601008b0 0x7fe26019a5d0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:53.856 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.851+0000 7fe267a42640 1 -- 172.21.15.1:0/416064637 >> 172.21.15.1:0/416064637 conn(0x7fe2600fc6b0 msgr2=0x7fe2600feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:53.856 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.851+0000 7fe267a42640 1 -- 172.21.15.1:0/416064637 shutdown_connections 2024-09-11T22:04:53.856 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:04:53.851+0000 7fe267a42640 1 -- 172.21.15.1:0/416064637 wait complete. 2024-09-11T22:04:54.256 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:04:54.516 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:04:55.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.023+0000 7fe7f28e4640 1 -- 172.21.15.1:0/3151095142 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe7ec102da0 msgr2=0x7fe7ec103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:55.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.023+0000 7fe7f28e4640 1 --2- 172.21.15.1:0/3151095142 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe7ec102da0 0x7fe7ec103180 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc009900 tx=0x7fe7dc02a480 comp rx=0 tx=0).stop 2024-09-11T22:04:55.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.023+0000 7fe7f28e4640 1 -- 172.21.15.1:0/3151095142 shutdown_connections 2024-09-11T22:04:55.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.023+0000 7fe7f28e4640 1 --2- 172.21.15.1:0/3151095142 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7ec10c180 0x7fe7ec10e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:55.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.023+0000 7fe7f28e4640 1 --2- 172.21.15.1:0/3151095142 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7ec1036c0 0x7fe7ec10bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:55.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.023+0000 7fe7f28e4640 1 --2- 172.21.15.1:0/3151095142 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe7ec102da0 0x7fe7ec103180 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:55.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.023+0000 7fe7f28e4640 1 -- 172.21.15.1:0/3151095142 >> 172.21.15.1:0/3151095142 conn(0x7fe7ec0fc7a0 msgr2=0x7fe7ec0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:55.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.023+0000 7fe7f28e4640 1 -- 172.21.15.1:0/3151095142 shutdown_connections 2024-09-11T22:04:55.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.023+0000 7fe7f28e4640 1 -- 172.21.15.1:0/3151095142 wait complete. 2024-09-11T22:04:55.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.024+0000 7fe7f28e4640 1 Processor -- start 2024-09-11T22:04:55.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.024+0000 7fe7f28e4640 1 -- start start 2024-09-11T22:04:55.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.024+0000 7fe7f28e4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7ec102da0 0x7fe7ec1984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:55.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.024+0000 7fe7f28e4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe7ec1036c0 0x7fe7ec1989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:55.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.024+0000 7fe7f28e4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7ec10c180 0x7fe7ec19cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:55.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.024+0000 7fe7f28e4640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec111f60 con 0x7fe7ec1036c0 2024-09-11T22:04:55.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.024+0000 7fe7f28e4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec111de0 con 0x7fe7ec10c180 2024-09-11T22:04:55.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.024+0000 7fe7f28e4640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec1120e0 con 0x7fe7ec102da0 2024-09-11T22:04:55.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.024+0000 7fe7eb7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe7ec1036c0 0x7fe7ec1989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:55.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.024+0000 7fe7eb7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe7ec1036c0 0x7fe7ec1989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56192/0 (socket says 172.21.15.1:56192) 2024-09-11T22:04:55.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.024+0000 7fe7eb7fe640 1 -- 172.21.15.1:0/4280517749 learned_addr learned my addr 172.21.15.1:0/4280517749 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:55.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.024+0000 7fe7f0e81640 1 --2- 172.21.15.1:0/4280517749 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7ec10c180 0x7fe7ec19cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:55.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.024+0000 7fe7ebfff640 1 --2- 172.21.15.1:0/4280517749 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7ec102da0 0x7fe7ec1984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:55.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.025+0000 7fe7eb7fe640 1 -- 172.21.15.1:0/4280517749 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7ec102da0 msgr2=0x7fe7ec1984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:55.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.025+0000 7fe7eb7fe640 1 --2- 172.21.15.1:0/4280517749 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7ec102da0 0x7fe7ec1984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:55.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.025+0000 7fe7eb7fe640 1 -- 172.21.15.1:0/4280517749 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7ec10c180 msgr2=0x7fe7ec19cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:55.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.025+0000 7fe7eb7fe640 1 --2- 172.21.15.1:0/4280517749 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7ec10c180 0x7fe7ec19cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:55.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.025+0000 7fe7eb7fe640 1 -- 172.21.15.1:0/4280517749 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7ec19d460 con 0x7fe7ec1036c0 2024-09-11T22:04:55.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.025+0000 7fe7eb7fe640 1 --2- 172.21.15.1:0/4280517749 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe7ec1036c0 0x7fe7ec1989f0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fe7e000d770 tx=0x7fe7e000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:55.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.025+0000 7fe7e97fa640 1 -- 172.21.15.1:0/4280517749 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e0017070 con 0x7fe7ec1036c0 2024-09-11T22:04:55.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.025+0000 7fe7e97fa640 1 -- 172.21.15.1:0/4280517749 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7e0005500 con 0x7fe7ec1036c0 2024-09-11T22:04:55.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.025+0000 7fe7e97fa640 1 -- 172.21.15.1:0/4280517749 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e0004e00 con 0x7fe7ec1036c0 2024-09-11T22:04:55.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.025+0000 7fe7f28e4640 1 -- 172.21.15.1:0/4280517749 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7ec19d750 con 0x7fe7ec1036c0 2024-09-11T22:04:55.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.025+0000 7fe7f28e4640 1 -- 172.21.15.1:0/4280517749 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7ec1a5030 con 0x7fe7ec1036c0 2024-09-11T22:04:55.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.028+0000 7fe7e97fa640 1 -- 172.21.15.1:0/4280517749 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7fe7e0004fa0 con 0x7fe7ec1036c0 2024-09-11T22:04:55.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.028+0000 7fe7f4cbd740 1 Processor -- start 2024-09-11T22:04:55.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.028+0000 7fe7f4cbd740 1 -- start start 2024-09-11T22:04:55.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.029+0000 7fe7f4cbd740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55598d8284a0 0x55598d828880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:55.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.029+0000 7fe7f4cbd740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55598d828dc0 0x55598d833140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:55.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.029+0000 7fe7f4cbd740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55598d833680 0x55598d835a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:55.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.029+0000 7fe7f4cbd740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55598d78e2e0 con 0x55598d8284a0 2024-09-11T22:04:55.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.029+0000 7fe7f4cbd740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55598d735ce0 con 0x55598d833680 2024-09-11T22:04:55.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.029+0000 7fe7f4cbd740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55598d78a2a0 con 0x55598d828dc0 2024-09-11T22:04:55.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.029+0000 7fe7f0e81640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55598d8284a0 0x55598d828880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:55.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.029+0000 7fe7f0e81640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55598d8284a0 0x55598d828880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56196/0 (socket says 172.21.15.1:56196) 2024-09-11T22:04:55.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.029+0000 7fe7f0e81640 1 -- 172.21.15.1:0/3986782742 learned_addr learned my addr 172.21.15.1:0/3986782742 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:55.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.029+0000 7fe7e97fa640 1 --2- 172.21.15.1:0/4280517749 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fe7d40a2cf0 0x7fe7d40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:55.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.029+0000 7fe7eb7fe640 1 --2- 172.21.15.1:0/3986782742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55598d833680 0x55598d835a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:55.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.029+0000 7fe7ebfff640 1 --2- 172.21.15.1:0/3986782742 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55598d828dc0 0x55598d833140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:55.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.029+0000 7fe7f0e81640 1 -- 172.21.15.1:0/3986782742 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55598d828dc0 msgr2=0x55598d833140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:55.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.029+0000 7fe7f0e81640 1 --2- 172.21.15.1:0/3986782742 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55598d828dc0 0x55598d833140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:55.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.029+0000 7fe7f0e81640 1 -- 172.21.15.1:0/3986782742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55598d833680 msgr2=0x55598d835a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:55.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.029+0000 7fe7f0e81640 1 --2- 172.21.15.1:0/3986782742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55598d833680 0x55598d835a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:55.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.029+0000 7fe7f0e81640 1 -- 172.21.15.1:0/3986782742 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55598d71a440 con 0x55598d8284a0 2024-09-11T22:04:55.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.029+0000 7fe7e97fa640 1 -- 172.21.15.1:0/4280517749 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7fe7e00cee90 con 0x7fe7ec1036c0 2024-09-11T22:04:55.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.029+0000 7fe7f0e81640 1 --2- 172.21.15.1:0/4280517749 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fe7d40a2cf0 0x7fe7d40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:55.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.029+0000 7fe7ebfff640 1 --2- 172.21.15.1:0/3986782742 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55598d828dc0 0x55598d833140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:04:55.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.029+0000 7fe7f0e81640 1 --2- 172.21.15.1:0/3986782742 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55598d8284a0 0x55598d828880 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4010440 tx=0x7fe7e4010900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f495b1a04d2e46ea server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:55.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.030+0000 7fe7d27fc640 1 -- 172.21.15.1:0/3986782742 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e401d400 con 0x55598d8284a0 2024-09-11T22:04:55.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.030+0000 7fe7d27fc640 1 -- 172.21.15.1:0/3986782742 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7e4013520 con 0x55598d8284a0 2024-09-11T22:04:55.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.030+0000 7fe7d27fc640 1 -- 172.21.15.1:0/3986782742 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e40118a0 con 0x55598d8284a0 2024-09-11T22:04:55.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.030+0000 7fe7f0e81640 1 --2- 172.21.15.1:0/4280517749 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fe7d40a2cf0 0x7fe7d40a51b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4021000 tx=0x7fe7e401e000 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:55.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.030+0000 7fe7f4cbd740 1 -- 172.21.15.1:0/3986782742 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55598d8284a0 msgr2=0x55598d828880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:55.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.030+0000 7fe7f4cbd740 1 --2- 172.21.15.1:0/3986782742 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55598d8284a0 0x55598d828880 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4010440 tx=0x7fe7e4010900 comp rx=0 tx=0).stop 2024-09-11T22:04:55.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.030+0000 7fe7f4cbd740 1 -- 172.21.15.1:0/3986782742 shutdown_connections 2024-09-11T22:04:55.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.030+0000 7fe7f4cbd740 1 --2- 172.21.15.1:0/3986782742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55598d833680 0x55598d835a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:55.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.030+0000 7fe7f4cbd740 1 --2- 172.21.15.1:0/3986782742 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55598d828dc0 0x55598d833140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:55.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.030+0000 7fe7f4cbd740 1 --2- 172.21.15.1:0/3986782742 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55598d8284a0 0x55598d828880 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:55.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.031+0000 7fe7f4cbd740 1 -- 172.21.15.1:0/3986782742 >> 172.21.15.1:0/3986782742 conn(0x55598d824a30 msgr2=0x55598d824e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:55.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.031+0000 7fe7f4cbd740 1 -- 172.21.15.1:0/3986782742 shutdown_connections 2024-09-11T22:04:55.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.031+0000 7fe7f4cbd740 1 -- 172.21.15.1:0/3986782742 wait complete. 2024-09-11T22:04:55.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.031+0000 7fe7f4cbd740 1 -- start start 2024-09-11T22:04:55.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.031+0000 7fe7f4cbd740 1 Processor -- start 2024-09-11T22:04:55.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.032+0000 7fe7f4cbd740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55598d8284a0 0x55598d832f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:55.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.032+0000 7fe7f4cbd740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55598d828dc0 0x55598d830210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:55.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.032+0000 7fe7f4cbd740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55598d833680 0x55598d830750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:55.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.032+0000 7fe7f4cbd740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55598d836250 con 0x55598d8284a0 2024-09-11T22:04:55.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.032+0000 7fe7f4cbd740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55598d749050 con 0x55598d828dc0 2024-09-11T22:04:55.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.032+0000 7fe7f4cbd740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55598d8363d0 con 0x55598d833680 2024-09-11T22:04:55.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.032+0000 7fe7f4cbd740 10 client.0 _ll_register_callbacks cb 0x55598d789160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:04:55.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.032+0000 7fe7ebfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55598d8284a0 0x55598d832f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:55.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.032+0000 7fe7ebfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55598d8284a0 0x55598d832f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56206/0 (socket says 172.21.15.1:56206) 2024-09-11T22:04:55.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.032+0000 7fe7ebfff640 1 -- 172.21.15.1:0/479413189 learned_addr learned my addr 172.21.15.1:0/479413189 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:55.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.032+0000 7fe7f0e81640 1 --2- 172.21.15.1:0/479413189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55598d833680 0x55598d830750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:55.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.032+0000 7fe7eb7fe640 1 --2- 172.21.15.1:0/479413189 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55598d828dc0 0x55598d830210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:55.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.032+0000 7fe7ebfff640 1 -- 172.21.15.1:0/479413189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55598d833680 msgr2=0x55598d830750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:55.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.032+0000 7fe7ebfff640 1 --2- 172.21.15.1:0/479413189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55598d833680 0x55598d830750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:55.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.032+0000 7fe7ebfff640 1 -- 172.21.15.1:0/479413189 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55598d828dc0 msgr2=0x55598d830210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:55.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.032+0000 7fe7ebfff640 1 --2- 172.21.15.1:0/479413189 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55598d828dc0 0x55598d830210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:55.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.032+0000 7fe7ebfff640 1 -- 172.21.15.1:0/479413189 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55598d4ff3c0 con 0x55598d8284a0 2024-09-11T22:04:55.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.032+0000 7fe7eb7fe640 1 --2- 172.21.15.1:0/479413189 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55598d828dc0 0x55598d830210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:04:55.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.032+0000 7fe7ebfff640 1 --2- 172.21.15.1:0/479413189 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55598d8284a0 0x55598d832f60 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc009900 tx=0x7fe7dc031920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c73899318cf0db server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:55.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.032+0000 7fe7cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:04:55.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.032+0000 7fe7cb7fe640 1 -- 172.21.15.1:0/479413189 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc040070 con 0x55598d8284a0 2024-09-11T22:04:55.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.032+0000 7fe7cb7fe640 1 -- 172.21.15.1:0/479413189 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7fe7dc004450 con 0x55598d8284a0 2024-09-11T22:04:55.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.032+0000 7fe7f4cbd740 1 -- 172.21.15.1:0/479413189 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7e4013520 con 0x55598d8284a0 2024-09-11T22:04:55.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.033+0000 7fe7cb7fe640 1 -- 172.21.15.1:0/479413189 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fe7dc035040 con 0x55598d8284a0 2024-09-11T22:04:55.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.033+0000 7fe7f4cbd740 10 client.17112 fetch_fsmap learned FSMap version 42 2024-09-11T22:04:55.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.033+0000 7fe7f4cbd740 1 -- 172.21.15.1:0/479413189 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55598d841b80 con 0x55598d8284a0 2024-09-11T22:04:55.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.033+0000 7fe7cb7fe640 1 -- 172.21.15.1:0/479413189 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7e4013520 con 0x55598d8284a0 2024-09-11T22:04:55.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.033+0000 7fe7cb7fe640 1 -- 172.21.15.1:0/479413189 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7fe7dc0035f0 con 0x55598d8284a0 2024-09-11T22:04:55.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.033+0000 7fe7f4cbd740 10 client.17112 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:04:55.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.033+0000 7fe7f4cbd740 10 client.17112 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:04:55.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.033+0000 7fe7f4cbd740 20 client.17112 populate_metadata read hostname 'smithi001' 2024-09-11T22:04:55.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.033+0000 7fe7f4cbd740 1 --2- 172.21.15.1:0/479413189 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55598d843090 0x55598d845510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:55.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.033+0000 7fe7f4cbd740 4 client.17112 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:04:55.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.033+0000 7fe7f4cbd740 1 -- 172.21.15.1:0/479413189 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55598d79b850 con 0x55598d843090 2024-09-11T22:04:55.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.033+0000 7fe7eb7fe640 1 --2- 172.21.15.1:0/479413189 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55598d843090 0x55598d845510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:55.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.034+0000 7fe7cb7fe640 10 client.17112 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:04:55.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.034+0000 7fe7eb7fe640 1 --2- 172.21.15.1:0/479413189 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55598d843090 0x55598d845510 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14cbec7cdcf6aa1e server_cookie=3fa4c37c35a9fc12 in_seq=0 out_seq=0 2024-09-11T22:04:55.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.034+0000 7fe7cb7fe640 1 -- 172.21.15.1:0/479413189 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55598d79b850 con 0x55598d843090 2024-09-11T22:04:55.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.034+0000 7fe7cb7fe640 10 client.17112 handle_command_reply: tid=0 2024-09-11T22:04:55.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.035+0000 7fe7f4cbd740 1 client.17112 shutdown 2024-09-11T22:04:55.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.035+0000 7fe7f4cbd740 1 -- 172.21.15.1:0/479413189 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55598d8284a0 msgr2=0x55598d832f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:55.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.035+0000 7fe7f4cbd740 1 --2- 172.21.15.1:0/479413189 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55598d8284a0 0x55598d832f60 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc009900 tx=0x7fe7dc031920 comp rx=0 tx=0).stop 2024-09-11T22:04:55.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.035+0000 7fe7f4cbd740 1 -- 172.21.15.1:0/479413189 shutdown_connections 2024-09-11T22:04:55.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.035+0000 7fe7f4cbd740 1 --2- 172.21.15.1:0/479413189 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55598d843090 0x55598d845510 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:55.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.035+0000 7fe7f4cbd740 1 --2- 172.21.15.1:0/479413189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55598d833680 0x55598d830750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:55.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.035+0000 7fe7f4cbd740 1 --2- 172.21.15.1:0/479413189 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55598d828dc0 0x55598d830210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:55.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.035+0000 7fe7f4cbd740 1 --2- 172.21.15.1:0/479413189 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55598d8284a0 0x55598d832f60 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:55.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.035+0000 7fe7f4cbd740 1 -- 172.21.15.1:0/479413189 >> 172.21.15.1:0/479413189 conn(0x55598d824a30 msgr2=0x55598d82a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:55.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.035+0000 7fe7cb7fe640 0 client.17112 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:04:55.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.035+0000 7fe7f4cbd740 1 -- 172.21.15.1:0/479413189 shutdown_connections 2024-09-11T22:04:55.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.035+0000 7fe7f4cbd740 1 -- 172.21.15.1:0/479413189 wait complete. 2024-09-11T22:04:55.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.035+0000 7fe7f4cbd740 20 client.17112 trim_cache size 0 max 16384 2024-09-11T22:04:55.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.052+0000 7fe7f4cbd740 1 Processor -- start 2024-09-11T22:04:55.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.052+0000 7fe7f4cbd740 1 -- start start 2024-09-11T22:04:55.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.053+0000 7fe7f4cbd740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55598d8284a0 0x55598d823a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:55.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.053+0000 7fe7f4cbd740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55598d828dc0 0x55598d84ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:55.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.053+0000 7fe7f4cbd740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55598d833680 0x55598d84b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:55.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.053+0000 7fe7f4cbd740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec1ada10 con 0x55598d828dc0 2024-09-11T22:04:55.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.053+0000 7fe7f4cbd740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec1af850 con 0x55598d8284a0 2024-09-11T22:04:55.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.053+0000 7fe7f4cbd740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec1ae1a0 con 0x55598d833680 2024-09-11T22:04:55.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.053+0000 7fe7eb7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55598d828dc0 0x55598d84ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:55.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.053+0000 7fe7eb7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55598d828dc0 0x55598d84ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56208/0 (socket says 172.21.15.1:56208) 2024-09-11T22:04:55.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.053+0000 7fe7eb7fe640 1 -- 172.21.15.1:0/1347938373 learned_addr learned my addr 172.21.15.1:0/1347938373 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:55.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.053+0000 7fe7f0e81640 1 --2- 172.21.15.1:0/1347938373 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55598d833680 0x55598d84b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:55.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.053+0000 7fe7ebfff640 1 --2- 172.21.15.1:0/1347938373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55598d8284a0 0x55598d823a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:55.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.053+0000 7fe7eb7fe640 1 -- 172.21.15.1:0/1347938373 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55598d833680 msgr2=0x55598d84b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:55.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.053+0000 7fe7eb7fe640 1 --2- 172.21.15.1:0/1347938373 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55598d833680 0x55598d84b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:55.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.053+0000 7fe7eb7fe640 1 -- 172.21.15.1:0/1347938373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55598d8284a0 msgr2=0x55598d823a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:55.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.053+0000 7fe7eb7fe640 1 --2- 172.21.15.1:0/1347938373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55598d8284a0 0x55598d823a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:55.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.053+0000 7fe7eb7fe640 1 -- 172.21.15.1:0/1347938373 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55598d830a90 con 0x55598d828dc0 2024-09-11T22:04:55.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.053+0000 7fe7ebfff640 1 --2- 172.21.15.1:0/1347938373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55598d8284a0 0x55598d823a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:04:55.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.053+0000 7fe7eb7fe640 1 --2- 172.21.15.1:0/1347938373 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55598d828dc0 0x55598d84ade0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fe7e00cc620 tx=0x7fe7e00d13a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b5e53932e0a0775 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:55.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.054+0000 7fe7d1ffb640 1 -- 172.21.15.1:0/1347938373 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e0020040 con 0x55598d828dc0 2024-09-11T22:04:55.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.054+0000 7fe7d1ffb640 1 -- 172.21.15.1:0/1347938373 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7e00a3fb0 con 0x55598d828dc0 2024-09-11T22:04:55.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.054+0000 7fe7d1ffb640 1 -- 172.21.15.1:0/1347938373 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e00cc7f0 con 0x55598d828dc0 2024-09-11T22:04:55.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.054+0000 7fe7f4cbd740 1 -- 172.21.15.1:0/1347938373 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55598d828dc0 msgr2=0x55598d84ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:55.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.054+0000 7fe7f4cbd740 1 --2- 172.21.15.1:0/1347938373 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55598d828dc0 0x55598d84ade0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fe7e00cc620 tx=0x7fe7e00d13a0 comp rx=0 tx=0).stop 2024-09-11T22:04:55.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.054+0000 7fe7f4cbd740 1 -- 172.21.15.1:0/1347938373 shutdown_connections 2024-09-11T22:04:55.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.054+0000 7fe7f4cbd740 1 --2- 172.21.15.1:0/1347938373 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55598d833680 0x55598d84b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:55.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.054+0000 7fe7f4cbd740 1 --2- 172.21.15.1:0/1347938373 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55598d828dc0 0x55598d84ade0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:55.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.054+0000 7fe7f4cbd740 1 --2- 172.21.15.1:0/1347938373 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55598d8284a0 0x55598d823a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:55.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.054+0000 7fe7f4cbd740 1 -- 172.21.15.1:0/1347938373 >> 172.21.15.1:0/1347938373 conn(0x55598d824a30 msgr2=0x55598d823370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:55.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.054+0000 7fe7f4cbd740 1 -- 172.21.15.1:0/1347938373 shutdown_connections 2024-09-11T22:04:55.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.054+0000 7fe7f4cbd740 1 -- 172.21.15.1:0/1347938373 wait complete. 2024-09-11T22:04:55.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.055+0000 7fe7f4cbd740 1 -- start start 2024-09-11T22:04:55.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.055+0000 7fe7f4cbd740 1 Processor -- start 2024-09-11T22:04:55.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.055+0000 7fe7f4cbd740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55598d8284a0 0x55598d83ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:55.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.055+0000 7fe7f4cbd740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55598d828dc0 0x55598d838dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:55.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.055+0000 7fe7f4cbd740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55598d833680 0x55598d839300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:55.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.055+0000 7fe7f4cbd740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec1af0c0 con 0x55598d833680 2024-09-11T22:04:55.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.055+0000 7fe7f4cbd740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec1ae930 con 0x55598d8284a0 2024-09-11T22:04:55.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.055+0000 7fe7f4cbd740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec1aba10 con 0x55598d828dc0 2024-09-11T22:04:55.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.055+0000 7fe7f4cbd740 10 client.0 _ll_register_callbacks cb 0x7fe7dc005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:04:55.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.055+0000 7fe7f0e81640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55598d833680 0x55598d839300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:55.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.055+0000 7fe7f0e81640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55598d833680 0x55598d839300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56216/0 (socket says 172.21.15.1:56216) 2024-09-11T22:04:55.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.055+0000 7fe7f0e81640 1 -- 172.21.15.1:0/2878651634 learned_addr learned my addr 172.21.15.1:0/2878651634 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:55.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.055+0000 7fe7eb7fe640 1 --2- 172.21.15.1:0/2878651634 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55598d828dc0 0x55598d838dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:55.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.056+0000 7fe7ebfff640 1 --2- 172.21.15.1:0/2878651634 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55598d8284a0 0x55598d83ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:55.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.056+0000 7fe7eb7fe640 1 -- 172.21.15.1:0/2878651634 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55598d8284a0 msgr2=0x55598d83ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:55.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.056+0000 7fe7eb7fe640 1 --2- 172.21.15.1:0/2878651634 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55598d8284a0 0x55598d83ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:55.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.056+0000 7fe7eb7fe640 1 -- 172.21.15.1:0/2878651634 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55598d833680 msgr2=0x55598d839300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:55.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.056+0000 7fe7eb7fe640 1 --2- 172.21.15.1:0/2878651634 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55598d833680 0x55598d839300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:55.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.056+0000 7fe7eb7fe640 1 -- 172.21.15.1:0/2878651634 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55598d839840 con 0x55598d828dc0 2024-09-11T22:04:55.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.056+0000 7fe7f0e81640 1 --2- 172.21.15.1:0/2878651634 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55598d833680 0x55598d839300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:04:55.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.056+0000 7fe7eb7fe640 1 --2- 172.21.15.1:0/2878651634 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55598d828dc0 0x55598d838dc0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe7e00ccb70 tx=0x7fe7e00996f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74498dddc0dd5bb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:55.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.056+0000 7fe7cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-11T22:04:55.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.056+0000 7fe7cb7fe640 1 -- 172.21.15.1:0/2878651634 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e00cc7f0 con 0x55598d828dc0 2024-09-11T22:04:55.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.056+0000 7fe7f4cbd740 1 -- 172.21.15.1:0/2878651634 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7e00a3fb0 con 0x55598d828dc0 2024-09-11T22:04:55.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.056+0000 7fe7cb7fe640 1 -- 172.21.15.1:0/2878651634 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7fe7e0097a60 con 0x55598d828dc0 2024-09-11T22:04:55.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.056+0000 7fe7cb7fe640 1 -- 172.21.15.1:0/2878651634 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fe7e00b41a0 con 0x55598d828dc0 2024-09-11T22:04:55.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.056+0000 7fe7f4cbd740 10 client.26575 fetch_fsmap learned FSMap version 42 2024-09-11T22:04:55.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.056+0000 7fe7f4cbd740 1 -- 172.21.15.1:0/2878651634 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55598d8b1e90 con 0x55598d828dc0 2024-09-11T22:04:55.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.057+0000 7fe7cb7fe640 1 -- 172.21.15.1:0/2878651634 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7e00a3fb0 con 0x55598d828dc0 2024-09-11T22:04:55.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.057+0000 7fe7cb7fe640 1 -- 172.21.15.1:0/2878651634 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7fe7e00bb0b0 con 0x55598d828dc0 2024-09-11T22:04:55.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.057+0000 7fe7f4cbd740 10 client.26575 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:04:55.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.057+0000 7fe7f4cbd740 10 client.26575 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:04:55.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.057+0000 7fe7f4cbd740 20 client.26575 populate_metadata read hostname 'smithi001' 2024-09-11T22:04:55.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.057+0000 7fe7f4cbd740 1 --2- 172.21.15.1:0/2878651634 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55598d8b25d0 0x55598d8b49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:55.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.057+0000 7fe7f4cbd740 4 client.26575 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:04:55.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.057+0000 7fe7f4cbd740 1 -- 172.21.15.1:0/2878651634 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55598d795d60 con 0x55598d8b25d0 2024-09-11T22:04:55.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.057+0000 7fe7ebfff640 1 --2- 172.21.15.1:0/2878651634 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55598d8b25d0 0x55598d8b49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:55.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.058+0000 7fe7cb7fe640 10 client.26575 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:04:55.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.058+0000 7fe7ebfff640 1 --2- 172.21.15.1:0/2878651634 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55598d8b25d0 0x55598d8b49f0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=303ed2d9980ef428 server_cookie=24cf6d7fc8fe8e69 in_seq=0 out_seq=0 2024-09-11T22:04:55.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.058+0000 7fe7cb7fe640 1 -- 172.21.15.1:0/2878651634 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55598d795d60 con 0x55598d8b25d0 2024-09-11T22:04:55.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.058+0000 7fe7cb7fe640 10 client.26575 handle_command_reply: tid=0 2024-09-11T22:04:55.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.058+0000 7fe7f4cbd740 1 client.26575 shutdown 2024-09-11T22:04:55.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.058+0000 7fe7f4cbd740 1 -- 172.21.15.1:0/2878651634 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55598d828dc0 msgr2=0x55598d838dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:55.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.058+0000 7fe7f4cbd740 1 --2- 172.21.15.1:0/2878651634 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55598d828dc0 0x55598d838dc0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe7e00ccb70 tx=0x7fe7e00996f0 comp rx=0 tx=0).stop 2024-09-11T22:04:55.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.058+0000 7fe7f4cbd740 1 -- 172.21.15.1:0/2878651634 shutdown_connections 2024-09-11T22:04:55.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.058+0000 7fe7f4cbd740 1 --2- 172.21.15.1:0/2878651634 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55598d8b25d0 0x55598d8b49f0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:55.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.058+0000 7fe7f4cbd740 1 --2- 172.21.15.1:0/2878651634 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55598d833680 0x55598d839300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:55.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.058+0000 7fe7f4cbd740 1 --2- 172.21.15.1:0/2878651634 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55598d828dc0 0x55598d838dc0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:55.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.058+0000 7fe7f4cbd740 1 --2- 172.21.15.1:0/2878651634 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55598d8284a0 0x55598d83ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:55.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.058+0000 7fe7cb7fe640 0 client.26575 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:04:55.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.058+0000 7fe7f4cbd740 1 -- 172.21.15.1:0/2878651634 >> 172.21.15.1:0/2878651634 conn(0x55598d824a30 msgr2=0x55598d823b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:55.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.058+0000 7fe7f4cbd740 1 -- 172.21.15.1:0/2878651634 shutdown_connections 2024-09-11T22:04:55.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.058+0000 7fe7f4cbd740 1 -- 172.21.15.1:0/2878651634 wait complete. 2024-09-11T22:04:55.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.058+0000 7fe7f4cbd740 20 client.26575 trim_cache size 0 max 16384 2024-09-11T22:04:55.069 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-11T22:04:55.070 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-09-11T22:04:55.070 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-09-11T22:04:55.070 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-11T22:04:55.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.059+0000 7fe7f28e4640 1 -- 172.21.15.1:0/4280517749 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fe7d40a2cf0 msgr2=0x7fe7d40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:55.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.059+0000 7fe7f28e4640 1 --2- 172.21.15.1:0/4280517749 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fe7d40a2cf0 0x7fe7d40a51b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4021000 tx=0x7fe7e401e000 comp rx=0 tx=0).stop 2024-09-11T22:04:55.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.059+0000 7fe7f28e4640 1 -- 172.21.15.1:0/4280517749 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe7ec1036c0 msgr2=0x7fe7ec1989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:55.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.059+0000 7fe7f28e4640 1 --2- 172.21.15.1:0/4280517749 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe7ec1036c0 0x7fe7ec1989f0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fe7e000d770 tx=0x7fe7e000dc30 comp rx=0 tx=0).stop 2024-09-11T22:04:55.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.059+0000 7fe7f28e4640 1 -- 172.21.15.1:0/4280517749 shutdown_connections 2024-09-11T22:04:55.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.059+0000 7fe7f28e4640 1 --2- 172.21.15.1:0/4280517749 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fe7d40a2cf0 0x7fe7d40a51b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:55.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.059+0000 7fe7f28e4640 1 --2- 172.21.15.1:0/4280517749 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe7ec10c180 0x7fe7ec19cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:55.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.059+0000 7fe7f28e4640 1 --2- 172.21.15.1:0/4280517749 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe7ec1036c0 0x7fe7ec1989f0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:55.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.059+0000 7fe7f28e4640 1 --2- 172.21.15.1:0/4280517749 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7ec102da0 0x7fe7ec1984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:55.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.059+0000 7fe7f28e4640 1 -- 172.21.15.1:0/4280517749 >> 172.21.15.1:0/4280517749 conn(0x7fe7ec0fc7a0 msgr2=0x7fe7ec0fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:55.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.059+0000 7fe7f28e4640 1 -- 172.21.15.1:0/4280517749 shutdown_connections 2024-09-11T22:04:55.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:55.059+0000 7fe7f28e4640 1 -- 172.21.15.1:0/4280517749 wait complete. 2024-09-11T22:04:55.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:04:55 smithi001 ceph-mon[30935]: pgmap v237: 97 pgs: 97 active+clean; 465 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.5 MiB/s wr, 263 op/s 2024-09-11T22:04:55.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:04:55 smithi053 ceph-mon[34908]: pgmap v237: 97 pgs: 97 active+clean; 465 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.5 MiB/s wr, 263 op/s 2024-09-11T22:04:55.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:04:55 smithi161 ceph-mon[35615]: pgmap v237: 97 pgs: 97 active+clean; 465 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.5 MiB/s wr, 263 op/s 2024-09-11T22:04:56.062 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T22:04:56.063 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:04:56.330 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:04:56.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:56.840+0000 7f64f07fc640 1 -- 172.21.15.1:0/3651385732 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f64e8101de0 msgr2=0x7f64e81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:56.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:56.840+0000 7f64f07fc640 1 --2- 172.21.15.1:0/3651385732 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f64e8101de0 0x7f64e81021c0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f64e4009900 tx=0x7f64e402a410 comp rx=0 tx=0).stop 2024-09-11T22:04:56.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:56.840+0000 7f64f07fc640 1 -- 172.21.15.1:0/3651385732 shutdown_connections 2024-09-11T22:04:56.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:56.840+0000 7f64f07fc640 1 --2- 172.21.15.1:0/3651385732 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64e8106f10 0x7f64e810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:56.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:56.840+0000 7f64f07fc640 1 --2- 172.21.15.1:0/3651385732 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64e8102790 0x7f64e81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:56.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:56.840+0000 7f64f07fc640 1 --2- 172.21.15.1:0/3651385732 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f64e8101de0 0x7f64e81021c0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:56.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:56.840+0000 7f64f07fc640 1 -- 172.21.15.1:0/3651385732 >> 172.21.15.1:0/3651385732 conn(0x7f64e80fd8c0 msgr2=0x7f64e80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:56.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:56.840+0000 7f64f07fc640 1 -- 172.21.15.1:0/3651385732 shutdown_connections 2024-09-11T22:04:56.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:56.840+0000 7f64f07fc640 1 -- 172.21.15.1:0/3651385732 wait complete. 2024-09-11T22:04:56.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:56.841+0000 7f64f07fc640 1 Processor -- start 2024-09-11T22:04:56.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:56.841+0000 7f64f07fc640 1 -- start start 2024-09-11T22:04:56.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:56.841+0000 7f64f07fc640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64e8101de0 0x7f64e81984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:56.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:56.841+0000 7f64f07fc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64e8102790 0x7f64e8198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:56.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:56.841+0000 7f64f07fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f64e8106f10 0x7f64e819cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:56.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:56.841+0000 7f64f07fc640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f64e8111f90 con 0x7f64e8106f10 2024-09-11T22:04:56.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:56.841+0000 7f64f07fc640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f64e8111e10 con 0x7f64e8101de0 2024-09-11T22:04:56.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:56.841+0000 7f64f07fc640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f64e8112110 con 0x7f64e8102790 2024-09-11T22:04:56.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:56.841+0000 7f64ee598640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64e8101de0 0x7f64e81984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:56.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:56.841+0000 7f64eed99640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f64e8106f10 0x7f64e819cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:56.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:56.841+0000 7f64ee598640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64e8101de0 0x7f64e81984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:32830/0 (socket says 172.21.15.1:32830) 2024-09-11T22:04:56.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:56.841+0000 7f64ee598640 1 -- 172.21.15.1:0/3047157713 learned_addr learned my addr 172.21.15.1:0/3047157713 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:56.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:56.841+0000 7f64edd97640 1 --2- 172.21.15.1:0/3047157713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64e8102790 0x7f64e8198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:56.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:56.842+0000 7f64eed99640 1 -- 172.21.15.1:0/3047157713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64e8102790 msgr2=0x7f64e8198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:56.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:56.842+0000 7f64eed99640 1 --2- 172.21.15.1:0/3047157713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64e8102790 0x7f64e8198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:56.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:56.842+0000 7f64eed99640 1 -- 172.21.15.1:0/3047157713 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64e8101de0 msgr2=0x7f64e81984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:56.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:56.842+0000 7f64eed99640 1 --2- 172.21.15.1:0/3047157713 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64e8101de0 0x7f64e81984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:56.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:56.842+0000 7f64eed99640 1 -- 172.21.15.1:0/3047157713 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64e819d480 con 0x7f64e8106f10 2024-09-11T22:04:56.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:56.842+0000 7f64eed99640 1 --2- 172.21.15.1:0/3047157713 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f64e8106f10 0x7f64e819cda0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f64e000ca30 tx=0x7f64e000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:56.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:56.842+0000 7f64df7fe640 1 -- 172.21.15.1:0/3047157713 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e0004d20 con 0x7f64e8106f10 2024-09-11T22:04:56.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:56.842+0000 7f64df7fe640 1 -- 172.21.15.1:0/3047157713 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64e0005020 con 0x7f64e8106f10 2024-09-11T22:04:56.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:56.842+0000 7f64f07fc640 1 -- 172.21.15.1:0/3047157713 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64e819d770 con 0x7f64e8106f10 2024-09-11T22:04:56.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:56.842+0000 7f64df7fe640 1 -- 172.21.15.1:0/3047157713 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e000f6b0 con 0x7f64e8106f10 2024-09-11T22:04:56.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:56.842+0000 7f64f07fc640 1 -- 172.21.15.1:0/3047157713 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64e81a5050 con 0x7f64e8106f10 2024-09-11T22:04:56.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:56.843+0000 7f64f07fc640 1 -- 172.21.15.1:0/3047157713 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64e81021c0 con 0x7f64e8106f10 2024-09-11T22:04:56.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:56.844+0000 7f64df7fe640 1 -- 172.21.15.1:0/3047157713 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f64e0005560 con 0x7f64e8106f10 2024-09-11T22:04:56.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:56.845+0000 7f64df7fe640 1 --2- 172.21.15.1:0/3047157713 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f64d40a2cf0 0x7f64d40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:56.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:56.845+0000 7f64ee598640 1 --2- 172.21.15.1:0/3047157713 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f64d40a2cf0 0x7f64d40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:56.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:56.845+0000 7f64df7fe640 1 -- 172.21.15.1:0/3047157713 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7f64e00cf670 con 0x7f64e8106f10 2024-09-11T22:04:56.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:56.845+0000 7f64ee598640 1 --2- 172.21.15.1:0/3047157713 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f64d40a2cf0 0x7f64d40a51b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f64e40096c0 tx=0x7f64e4005a20 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:56.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:56.849+0000 7f64df7fe640 1 -- 172.21.15.1:0/3047157713 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f64e0014070 con 0x7f64e8106f10 2024-09-11T22:04:57.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:57.063+0000 7f64f07fc640 1 -- 172.21.15.1:0/3047157713 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f64e802d7c0 con 0x7f64e8106f10 2024-09-11T22:04:57.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:57.064+0000 7f64df7fe640 1 -- 172.21.15.1:0/3047157713 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11984 (secure 0 0 0) 0x7f64e009c220 con 0x7f64e8106f10 2024-09-11T22:04:57.065 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T22:04:57.066 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":42,"btime":"2024-09-11T22:03:27:040852+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/421542832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":421542832},{"type":"v1","addr":"172.21.15.1:6835","nonce":421542832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2836864406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2836864406},{"type":"v1","addr":"172.21.15.1:6837","nonce":2836864406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":42,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T21:55:20.791263+0000","modified":"2024-09-11T22:03:26.485134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24484,"mds_2":24467,"mds_3":24466,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14595":{"gid":14595,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/407834672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":407834672},{"type":"v1","addr":"172.21.15.1:6839","nonce":407834672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/1068827678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":1068827678},{"type":"v1","addr":"172.21.15.1:6841","nonce":1068827678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/872410024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":872410024},{"type":"v1","addr":"172.21.15.161:6833","nonce":872410024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/1648944570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1648944570},{"type":"v1","addr":"172.21.15.53:6833","nonce":1648944570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/2126933993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2126933993},{"type":"v1","addr":"172.21.15.53:6835","nonce":2126933993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.161:6835/828381592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":828381592},{"type":"v1","addr":"172.21.15.161:6835","nonce":828381592}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.161:6837/1799318656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1799318656},{"type":"v1","addr":"172.21.15.161:6837","nonce":1799318656}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.53:6837/3457131830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3457131830},{"type":"v1","addr":"172.21.15.53:6837","nonce":3457131830}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/455365383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":455365383},{"type":"v1","addr":"172.21.15.161:6839","nonce":455365383}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.53:6839/1378584447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1378584447},{"type":"v1","addr":"172.21.15.53:6839","nonce":1378584447}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24460,24466,24467,24484,24494]},"id":1}]} 2024-09-11T22:04:57.066 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 42 2024-09-11T22:04:57.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:57.066+0000 7f64f07fc640 1 -- 172.21.15.1:0/3047157713 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f64d40a2cf0 msgr2=0x7f64d40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:57.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:57.066+0000 7f64f07fc640 1 --2- 172.21.15.1:0/3047157713 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f64d40a2cf0 0x7f64d40a51b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f64e40096c0 tx=0x7f64e4005a20 comp rx=0 tx=0).stop 2024-09-11T22:04:57.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:57.066+0000 7f64f07fc640 1 -- 172.21.15.1:0/3047157713 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f64e8106f10 msgr2=0x7f64e819cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:57.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:57.066+0000 7f64f07fc640 1 --2- 172.21.15.1:0/3047157713 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f64e8106f10 0x7f64e819cda0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f64e000ca30 tx=0x7f64e000ce30 comp rx=0 tx=0).stop 2024-09-11T22:04:57.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:57.066+0000 7f64f07fc640 1 -- 172.21.15.1:0/3047157713 shutdown_connections 2024-09-11T22:04:57.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:57.066+0000 7f64f07fc640 1 --2- 172.21.15.1:0/3047157713 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f64d40a2cf0 0x7f64d40a51b0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:57.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:57.066+0000 7f64f07fc640 1 --2- 172.21.15.1:0/3047157713 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f64e8106f10 0x7f64e819cda0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:57.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:57.066+0000 7f64f07fc640 1 --2- 172.21.15.1:0/3047157713 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64e8102790 0x7f64e8198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:57.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:57.066+0000 7f64f07fc640 1 --2- 172.21.15.1:0/3047157713 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64e8101de0 0x7f64e81984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:57.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:57.066+0000 7f64f07fc640 1 -- 172.21.15.1:0/3047157713 >> 172.21.15.1:0/3047157713 conn(0x7f64e80fd8c0 msgr2=0x7f64e80fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:57.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:57.066+0000 7f64f07fc640 1 -- 172.21.15.1:0/3047157713 shutdown_connections 2024-09-11T22:04:57.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:57.066+0000 7f64f07fc640 1 -- 172.21.15.1:0/3047157713 wait complete. 2024-09-11T22:04:57.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:04:57 smithi053 ceph-mon[34908]: pgmap v238: 97 pgs: 97 active+clean; 469 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.9 MiB/s wr, 451 op/s 2024-09-11T22:04:57.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:04:57 smithi161 ceph-mon[35615]: pgmap v238: 97 pgs: 97 active+clean; 469 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.9 MiB/s wr, 451 op/s 2024-09-11T22:04:57.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:04:57 smithi001 ceph-mon[30935]: pgmap v238: 97 pgs: 97 active+clean; 469 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.9 MiB/s wr, 451 op/s 2024-09-11T22:04:57.788 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T22:04:58.049 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:04:58.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:04:58 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/3047157713' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:04:58.308 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:04:58 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/3047157713' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:04:58.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:04:58 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/3047157713' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:04:58.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.536+0000 7f37fed8c640 1 -- 172.21.15.1:0/1145687365 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37f8103660 msgr2=0x7f37f810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:58.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.536+0000 7f37fed8c640 1 --2- 172.21.15.1:0/1145687365 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37f8103660 0x7f37f810bb50 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f37ec009900 tx=0x7f37ec02a480 comp rx=0 tx=0).stop 2024-09-11T22:04:58.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.536+0000 7f37fed8c640 1 -- 172.21.15.1:0/1145687365 shutdown_connections 2024-09-11T22:04:58.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.536+0000 7f37fed8c640 1 --2- 172.21.15.1:0/1145687365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f37f810c170 0x7f37f810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:58.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.536+0000 7f37fed8c640 1 --2- 172.21.15.1:0/1145687365 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37f8103660 0x7f37f810bb50 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:58.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.536+0000 7f37fed8c640 1 --2- 172.21.15.1:0/1145687365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37f8102d40 0x7f37f8103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:58.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.536+0000 7f37fed8c640 1 -- 172.21.15.1:0/1145687365 >> 172.21.15.1:0/1145687365 conn(0x7f37f80fc7a0 msgr2=0x7f37f80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:58.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.537+0000 7f37fed8c640 1 -- 172.21.15.1:0/1145687365 shutdown_connections 2024-09-11T22:04:58.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.537+0000 7f37fed8c640 1 -- 172.21.15.1:0/1145687365 wait complete. 2024-09-11T22:04:58.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.537+0000 7f37fed8c640 1 Processor -- start 2024-09-11T22:04:58.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.537+0000 7f37fed8c640 1 -- start start 2024-09-11T22:04:58.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.537+0000 7f37fed8c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37f8102d40 0x7f37f81009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:58.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.537+0000 7f37fed8c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37f8103660 0x7f37f8100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:58.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.537+0000 7f37fed8c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f37f810c170 0x7f37f8101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:58.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.537+0000 7f37fed8c640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f37f8111f70 con 0x7f37f8102d40 2024-09-11T22:04:58.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.537+0000 7f37fed8c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f37f8111df0 con 0x7f37f8103660 2024-09-11T22:04:58.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.537+0000 7f37fed8c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f37f81120f0 con 0x7f37f810c170 2024-09-11T22:04:58.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.538+0000 7f37fcb28640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37f8102d40 0x7f37f81009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:58.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.538+0000 7f37fcb28640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37f8102d40 0x7f37f81009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59450/0 (socket says 172.21.15.1:59450) 2024-09-11T22:04:58.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.538+0000 7f37fcb28640 1 -- 172.21.15.1:0/641641640 learned_addr learned my addr 172.21.15.1:0/641641640 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:58.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.538+0000 7f37f7fff640 1 --2- 172.21.15.1:0/641641640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37f8103660 0x7f37f8100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:58.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.538+0000 7f37fd329640 1 --2- 172.21.15.1:0/641641640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f37f810c170 0x7f37f8101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:58.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.538+0000 7f37fcb28640 1 -- 172.21.15.1:0/641641640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f37f810c170 msgr2=0x7f37f8101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:58.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.538+0000 7f37fcb28640 1 --2- 172.21.15.1:0/641641640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f37f810c170 0x7f37f8101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:58.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.538+0000 7f37fcb28640 1 -- 172.21.15.1:0/641641640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37f8103660 msgr2=0x7f37f8100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:58.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.538+0000 7f37fcb28640 1 --2- 172.21.15.1:0/641641640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37f8103660 0x7f37f8100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:58.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.538+0000 7f37fcb28640 1 -- 172.21.15.1:0/641641640 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37f81a6930 con 0x7f37f8102d40 2024-09-11T22:04:58.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.538+0000 7f37fd329640 1 --2- 172.21.15.1:0/641641640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f37f810c170 0x7f37f8101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:04:58.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.538+0000 7f37fcb28640 1 --2- 172.21.15.1:0/641641640 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37f8102d40 0x7f37f81009a0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f37e800d950 tx=0x7f37e800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:58.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.538+0000 7f37f5ffb640 1 -- 172.21.15.1:0/641641640 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e8017070 con 0x7f37f8102d40 2024-09-11T22:04:58.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.538+0000 7f37fed8c640 1 -- 172.21.15.1:0/641641640 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37f81a6c20 con 0x7f37f8102d40 2024-09-11T22:04:58.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.538+0000 7f37f5ffb640 1 -- 172.21.15.1:0/641641640 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37e8009c30 con 0x7f37f8102d40 2024-09-11T22:04:58.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.538+0000 7f37f5ffb640 1 -- 172.21.15.1:0/641641640 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e800f6b0 con 0x7f37f8102d40 2024-09-11T22:04:58.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.539+0000 7f37fed8c640 1 -- 172.21.15.1:0/641641640 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37f81a7160 con 0x7f37f8102d40 2024-09-11T22:04:58.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.540+0000 7f37f5ffb640 1 -- 172.21.15.1:0/641641640 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f37e80109d0 con 0x7f37f8102d40 2024-09-11T22:04:58.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.541+0000 7f3801165740 1 Processor -- start 2024-09-11T22:04:58.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.541+0000 7f3801165740 1 -- start start 2024-09-11T22:04:58.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.541+0000 7f37f5ffb640 1 --2- 172.21.15.1:0/641641640 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f37e00a2cf0 0x7f37e00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:58.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.541+0000 7f3801165740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55971662fb40 0x55971662e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:58.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.541+0000 7f3801165740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55971662e5d0 0x55971662ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:58.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.541+0000 7f37f5ffb640 1 -- 172.21.15.1:0/641641640 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7f37e80cf1e0 con 0x7f37f8102d40 2024-09-11T22:04:58.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.541+0000 7f3801165740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559716639370 0x55971663b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:58.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.541+0000 7f3801165740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5597165942f0 con 0x55971662fb40 2024-09-11T22:04:58.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.541+0000 7f3801165740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55971653bcf0 con 0x559716639370 2024-09-11T22:04:58.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.541+0000 7f3801165740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5597165902b0 con 0x55971662e5d0 2024-09-11T22:04:58.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.541+0000 7f37fcb28640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55971662fb40 0x55971662e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:58.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.542+0000 7f37fcb28640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55971662fb40 0x55971662e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59466/0 (socket says 172.21.15.1:59466) 2024-09-11T22:04:58.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.542+0000 7f37fcb28640 1 -- 172.21.15.1:0/2731072016 learned_addr learned my addr 172.21.15.1:0/2731072016 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:58.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.542+0000 7f37f7fff640 1 --2- 172.21.15.1:0/2731072016 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55971662e5d0 0x55971662ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:58.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.542+0000 7f37fcb28640 1 -- 172.21.15.1:0/2731072016 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55971662e5d0 msgr2=0x55971662ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:58.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.542+0000 7f37fcb28640 1 --2- 172.21.15.1:0/2731072016 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55971662e5d0 0x55971662ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:58.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.542+0000 7f37fd329640 1 --2- 172.21.15.1:0/2731072016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559716639370 0x55971663b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:58.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.542+0000 7f37fcb28640 1 -- 172.21.15.1:0/2731072016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559716639370 msgr2=0x55971663b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:58.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.542+0000 7f37fcb28640 1 --2- 172.21.15.1:0/2731072016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559716639370 0x55971663b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:58.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.542+0000 7f37fcb28640 1 -- 172.21.15.1:0/2731072016 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597165203a0 con 0x55971662fb40 2024-09-11T22:04:58.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.542+0000 7f37fd329640 1 --2- 172.21.15.1:0/2731072016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559716639370 0x55971663b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:04:58.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.542+0000 7f37fd329640 1 --2- 172.21.15.1:0/641641640 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f37e00a2cf0 0x7f37e00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:58.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.542+0000 7f37fcb28640 1 --2- 172.21.15.1:0/2731072016 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55971662fb40 0x55971662e090 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f37e80a3d70 tx=0x7f37e809b530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb93cef65ba5caf2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:58.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.542+0000 7f37deffd640 1 -- 172.21.15.1:0/2731072016 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e8020070 con 0x55971662fb40 2024-09-11T22:04:58.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.542+0000 7f37deffd640 1 -- 172.21.15.1:0/2731072016 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37e8009de0 con 0x55971662fb40 2024-09-11T22:04:58.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.542+0000 7f37deffd640 1 -- 172.21.15.1:0/2731072016 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e8013280 con 0x55971662fb40 2024-09-11T22:04:58.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.543+0000 7f3801165740 1 -- 172.21.15.1:0/2731072016 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55971662fb40 msgr2=0x55971662e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:58.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.543+0000 7f37fd329640 1 --2- 172.21.15.1:0/641641640 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f37e00a2cf0 0x7f37e00a51b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x55971662f070 tx=0x7f37f0004ab0 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:58.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.543+0000 7f3801165740 1 --2- 172.21.15.1:0/2731072016 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55971662fb40 0x55971662e090 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f37e80a3d70 tx=0x7f37e809b530 comp rx=0 tx=0).stop 2024-09-11T22:04:58.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.543+0000 7f3801165740 1 -- 172.21.15.1:0/2731072016 shutdown_connections 2024-09-11T22:04:58.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.543+0000 7f3801165740 1 --2- 172.21.15.1:0/2731072016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559716639370 0x55971663b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:58.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.543+0000 7f3801165740 1 --2- 172.21.15.1:0/2731072016 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55971662e5d0 0x55971662ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:58.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.543+0000 7f3801165740 1 --2- 172.21.15.1:0/2731072016 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55971662fb40 0x55971662e090 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:58.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.543+0000 7f3801165740 1 -- 172.21.15.1:0/2731072016 >> 172.21.15.1:0/2731072016 conn(0x5597165f7310 msgr2=0x55971662ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:58.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.543+0000 7f3801165740 1 -- 172.21.15.1:0/2731072016 shutdown_connections 2024-09-11T22:04:58.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.543+0000 7f3801165740 1 -- 172.21.15.1:0/2731072016 wait complete. 2024-09-11T22:04:58.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.543+0000 7f3801165740 1 -- start start 2024-09-11T22:04:58.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.544+0000 7f3801165740 1 Processor -- start 2024-09-11T22:04:58.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.544+0000 7f3801165740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55971662e5d0 0x559716638d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:58.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.544+0000 7f3801165740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55971662fb40 0x559716636010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:58.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.544+0000 7f3801165740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559716639370 0x559716636600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:58.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.544+0000 7f3801165740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55971663bfb0 con 0x55971662e5d0 2024-09-11T22:04:58.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.544+0000 7f3801165740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55971654f060 con 0x559716639370 2024-09-11T22:04:58.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.544+0000 7f3801165740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55971663c130 con 0x55971662fb40 2024-09-11T22:04:58.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.544+0000 7f3801165740 10 client.0 _ll_register_callbacks cb 0x5597162a28c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:04:58.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.544+0000 7f37fcb28640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55971662e5d0 0x559716638d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:58.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.544+0000 7f37fcb28640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55971662e5d0 0x559716638d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59478/0 (socket says 172.21.15.1:59478) 2024-09-11T22:04:58.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.544+0000 7f37fcb28640 1 -- 172.21.15.1:0/2836188054 learned_addr learned my addr 172.21.15.1:0/2836188054 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:58.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.544+0000 7f37f7fff640 1 --2- 172.21.15.1:0/2836188054 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55971662fb40 0x559716636010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:58.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.544+0000 7f37fd329640 1 --2- 172.21.15.1:0/2836188054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559716639370 0x559716636600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:58.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.544+0000 7f37fcb28640 1 -- 172.21.15.1:0/2836188054 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55971662fb40 msgr2=0x559716636010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:58.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.544+0000 7f37fcb28640 1 --2- 172.21.15.1:0/2836188054 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55971662fb40 0x559716636010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:58.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.544+0000 7f37fcb28640 1 -- 172.21.15.1:0/2836188054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559716639370 msgr2=0x559716636600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:58.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.544+0000 7f37fcb28640 1 --2- 172.21.15.1:0/2836188054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559716639370 0x559716636600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:58.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.544+0000 7f37fcb28640 1 -- 172.21.15.1:0/2836188054 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597163053c0 con 0x55971662e5d0 2024-09-11T22:04:58.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.544+0000 7f37fd329640 1 --2- 172.21.15.1:0/2836188054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559716639370 0x559716636600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:04:58.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.545+0000 7f37fcb28640 1 --2- 172.21.15.1:0/2836188054 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55971662e5d0 0x559716638d60 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f37e8000c00 tx=0x7f37e8097f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=923b32de902a053 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:58.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.545+0000 7f37d7fff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:04:58.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.545+0000 7f37d7fff640 1 -- 172.21.15.1:0/2836188054 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e80b4060 con 0x55971662e5d0 2024-09-11T22:04:58.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.545+0000 7f37d7fff640 1 -- 172.21.15.1:0/2836188054 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7f37e8084b70 con 0x55971662e5d0 2024-09-11T22:04:58.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.545+0000 7f3801165740 1 -- 172.21.15.1:0/2836188054 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37e8009de0 con 0x55971662e5d0 2024-09-11T22:04:58.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.545+0000 7f37d7fff640 1 -- 172.21.15.1:0/2836188054 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f37e8020070 con 0x55971662e5d0 2024-09-11T22:04:58.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.545+0000 7f3801165740 10 client.17136 fetch_fsmap learned FSMap version 42 2024-09-11T22:04:58.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.545+0000 7f3801165740 1 -- 172.21.15.1:0/2836188054 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5597166479c0 con 0x55971662e5d0 2024-09-11T22:04:58.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.545+0000 7f37d7fff640 1 -- 172.21.15.1:0/2836188054 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37e8009de0 con 0x55971662e5d0 2024-09-11T22:04:58.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.545+0000 7f37d7fff640 1 -- 172.21.15.1:0/2836188054 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f37e80afb50 con 0x55971662e5d0 2024-09-11T22:04:58.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.545+0000 7f3801165740 10 client.17136 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:04:58.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.545+0000 7f3801165740 10 client.17136 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-11T22:04:58.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.545+0000 7f3801165740 20 client.17136 populate_metadata read hostname 'smithi001' 2024-09-11T22:04:58.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.545+0000 7f3801165740 1 --2- 172.21.15.1:0/2836188054 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x559716648d10 0x55971664b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:58.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.545+0000 7f3801165740 4 client.17136 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:04:58.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.545+0000 7f3801165740 1 -- 172.21.15.1:0/2836188054 --> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5597165a1860 con 0x559716648d10 2024-09-11T22:04:58.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.546+0000 7f37f7fff640 1 --2- 172.21.15.1:0/2836188054 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x559716648d10 0x55971664b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:58.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.546+0000 7f37d7fff640 10 client.17136 ms_handle_connect on v2:172.21.15.161:6834/828381592 2024-09-11T22:04:58.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.546+0000 7f37f7fff640 1 --2- 172.21.15.1:0/2836188054 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x559716648d10 0x55971664b100 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5e43fbb8097a665 server_cookie=dcbf895e52ac83bd in_seq=0 out_seq=0 2024-09-11T22:04:58.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.547+0000 7f37d7fff640 1 -- 172.21.15.1:0/2836188054 <== mds.4 v2:172.21.15.161:6834/828381592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5597165a1860 con 0x559716648d10 2024-09-11T22:04:58.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.547+0000 7f37d7fff640 10 client.17136 handle_command_reply: tid=0 2024-09-11T22:04:58.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.547+0000 7f3801165740 1 client.17136 shutdown 2024-09-11T22:04:58.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.547+0000 7f3801165740 1 -- 172.21.15.1:0/2836188054 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55971662e5d0 msgr2=0x559716638d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:58.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.547+0000 7f3801165740 1 --2- 172.21.15.1:0/2836188054 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55971662e5d0 0x559716638d60 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f37e8000c00 tx=0x7f37e8097f30 comp rx=0 tx=0).stop 2024-09-11T22:04:58.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.547+0000 7f3801165740 1 -- 172.21.15.1:0/2836188054 shutdown_connections 2024-09-11T22:04:58.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.547+0000 7f3801165740 1 --2- 172.21.15.1:0/2836188054 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x559716648d10 0x55971664b100 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:58.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.547+0000 7f3801165740 1 --2- 172.21.15.1:0/2836188054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559716639370 0x559716636600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:58.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.547+0000 7f3801165740 1 --2- 172.21.15.1:0/2836188054 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55971662fb40 0x559716636010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:58.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.547+0000 7f3801165740 1 --2- 172.21.15.1:0/2836188054 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55971662e5d0 0x559716638d60 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:58.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.547+0000 7f37d7fff640 0 client.17136 ms_handle_reset on v2:172.21.15.161:6834/828381592 2024-09-11T22:04:58.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.547+0000 7f3801165740 1 -- 172.21.15.1:0/2836188054 >> 172.21.15.1:0/2836188054 conn(0x5597165f7310 msgr2=0x559716630990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:58.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.547+0000 7f3801165740 1 -- 172.21.15.1:0/2836188054 shutdown_connections 2024-09-11T22:04:58.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.547+0000 7f3801165740 1 -- 172.21.15.1:0/2836188054 wait complete. 2024-09-11T22:04:58.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.547+0000 7f3801165740 20 client.17136 trim_cache size 0 max 16384 2024-09-11T22:04:58.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.560+0000 7f3801165740 1 Processor -- start 2024-09-11T22:04:58.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.560+0000 7f3801165740 1 -- start start 2024-09-11T22:04:58.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.561+0000 7f3801165740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55971662e5d0 0x559716631120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:58.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.561+0000 7f3801165740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55971662fb40 0x559716631660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:58.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.561+0000 7f3801165740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559716639370 0x55971664e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:58.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.561+0000 7f3801165740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f37f81afd00 con 0x55971662fb40 2024-09-11T22:04:58.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.561+0000 7f3801165740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f37f81b1b40 con 0x55971662e5d0 2024-09-11T22:04:58.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.561+0000 7f3801165740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f37f81b0490 con 0x559716639370 2024-09-11T22:04:58.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.561+0000 7f37f7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55971662fb40 0x559716631660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:58.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.561+0000 7f37f7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55971662fb40 0x559716631660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59486/0 (socket says 172.21.15.1:59486) 2024-09-11T22:04:58.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.561+0000 7f37f7fff640 1 -- 172.21.15.1:0/3977644152 learned_addr learned my addr 172.21.15.1:0/3977644152 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:58.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.561+0000 7f37fcb28640 1 --2- 172.21.15.1:0/3977644152 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55971662e5d0 0x559716631120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:58.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.561+0000 7f37fd329640 1 --2- 172.21.15.1:0/3977644152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559716639370 0x55971664e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:58.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.561+0000 7f37f7fff640 1 -- 172.21.15.1:0/3977644152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559716639370 msgr2=0x55971664e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:58.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.561+0000 7f37f7fff640 1 --2- 172.21.15.1:0/3977644152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559716639370 0x55971664e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:58.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.561+0000 7f37f7fff640 1 -- 172.21.15.1:0/3977644152 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55971662e5d0 msgr2=0x559716631120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:58.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.561+0000 7f37f7fff640 1 --2- 172.21.15.1:0/3977644152 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55971662e5d0 0x559716631120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:58.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.561+0000 7f37f7fff640 1 -- 172.21.15.1:0/3977644152 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55971662b960 con 0x55971662fb40 2024-09-11T22:04:58.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.561+0000 7f37f7fff640 1 --2- 172.21.15.1:0/3977644152 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55971662fb40 0x559716631660 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f37ec005ab0 tx=0x7f37ec0058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=484a2193d49d7fe2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:58.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.562+0000 7f37de7fc640 1 -- 172.21.15.1:0/3977644152 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ec049070 con 0x55971662fb40 2024-09-11T22:04:58.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.562+0000 7f37de7fc640 1 -- 172.21.15.1:0/3977644152 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37ec005520 con 0x55971662fb40 2024-09-11T22:04:58.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.562+0000 7f37de7fc640 1 -- 172.21.15.1:0/3977644152 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ec035040 con 0x55971662fb40 2024-09-11T22:04:58.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.562+0000 7f3801165740 1 -- 172.21.15.1:0/3977644152 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55971662fb40 msgr2=0x559716631660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:58.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.562+0000 7f3801165740 1 --2- 172.21.15.1:0/3977644152 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55971662fb40 0x559716631660 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f37ec005ab0 tx=0x7f37ec0058b0 comp rx=0 tx=0).stop 2024-09-11T22:04:58.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.562+0000 7f3801165740 1 -- 172.21.15.1:0/3977644152 shutdown_connections 2024-09-11T22:04:58.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.562+0000 7f3801165740 1 --2- 172.21.15.1:0/3977644152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x559716639370 0x55971664e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:58.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.562+0000 7f3801165740 1 --2- 172.21.15.1:0/3977644152 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55971662fb40 0x559716631660 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:58.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.562+0000 7f3801165740 1 --2- 172.21.15.1:0/3977644152 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55971662e5d0 0x559716631120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:58.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.562+0000 7f3801165740 1 -- 172.21.15.1:0/3977644152 >> 172.21.15.1:0/3977644152 conn(0x5597165f7310 msgr2=0x559716635100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:58.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.563+0000 7f3801165740 1 -- 172.21.15.1:0/3977644152 shutdown_connections 2024-09-11T22:04:58.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.563+0000 7f3801165740 1 -- 172.21.15.1:0/3977644152 wait complete. 2024-09-11T22:04:58.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.563+0000 7f3801165740 1 -- start start 2024-09-11T22:04:58.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.563+0000 7f3801165740 1 Processor -- start 2024-09-11T22:04:58.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.563+0000 7f3801165740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55971662e5d0 0x55971664e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:58.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.563+0000 7f3801165740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55971662fb40 0x55971664ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:58.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.563+0000 7f3801165740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559716639370 0x55971664ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:58.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.563+0000 7f3801165740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f37f81b13b0 con 0x559716639370 2024-09-11T22:04:58.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.563+0000 7f3801165740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f37f81b0c20 con 0x55971662fb40 2024-09-11T22:04:58.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.563+0000 7f3801165740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f37f81add00 con 0x55971662e5d0 2024-09-11T22:04:58.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.564+0000 7f3801165740 10 client.0 _ll_register_callbacks cb 0x7f37e809b400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:04:58.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.564+0000 7f37fd329640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559716639370 0x55971664ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:58.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.564+0000 7f37fcb28640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55971662e5d0 0x55971664e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:58.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.564+0000 7f37fcb28640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55971662e5d0 0x55971664e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:48186/0 (socket says 172.21.15.1:48186) 2024-09-11T22:04:58.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.564+0000 7f37fcb28640 1 -- 172.21.15.1:0/3307854645 learned_addr learned my addr 172.21.15.1:0/3307854645 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:04:58.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.564+0000 7f37fd329640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559716639370 0x55971664ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59488/0 (socket says 172.21.15.1:59488) 2024-09-11T22:04:58.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.564+0000 7f37f7fff640 1 --2- 172.21.15.1:0/3307854645 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55971662fb40 0x55971664ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:58.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.564+0000 7f37fcb28640 1 -- 172.21.15.1:0/3307854645 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55971662fb40 msgr2=0x55971664ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:58.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.564+0000 7f37fcb28640 1 --2- 172.21.15.1:0/3307854645 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55971662fb40 0x55971664ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:58.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.564+0000 7f37fcb28640 1 -- 172.21.15.1:0/3307854645 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559716639370 msgr2=0x55971664ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:04:58.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.564+0000 7f37fcb28640 1 --2- 172.21.15.1:0/3307854645 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559716639370 0x55971664ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:58.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.564+0000 7f37fcb28640 1 -- 172.21.15.1:0/3307854645 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55971664c250 con 0x55971662e5d0 2024-09-11T22:04:58.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.564+0000 7f37f7fff640 1 --2- 172.21.15.1:0/3307854645 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55971662fb40 0x55971664ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:04:58.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.564+0000 7f37fcb28640 1 --2- 172.21.15.1:0/3307854645 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55971662e5d0 0x55971664e4d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f37e80cc9a0 tx=0x7f37e800dfa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c964833ca63ea6b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:04:58.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.564+0000 7f37d7fff640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-11T22:04:58.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.565+0000 7f37d7fff640 1 -- 172.21.15.1:0/3307854645 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e80b4070 con 0x55971662e5d0 2024-09-11T22:04:58.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.565+0000 7f3801165740 1 -- 172.21.15.1:0/3307854645 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37ec005520 con 0x55971662e5d0 2024-09-11T22:04:58.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.565+0000 7f37d7fff640 1 -- 172.21.15.1:0/3307854645 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7f37e8060d40 con 0x55971662e5d0 2024-09-11T22:04:58.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.565+0000 7f37d7fff640 1 -- 172.21.15.1:0/3307854645 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f37e8009de0 con 0x55971662e5d0 2024-09-11T22:04:58.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.565+0000 7f3801165740 10 client.26590 fetch_fsmap learned FSMap version 42 2024-09-11T22:04:58.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.565+0000 7f3801165740 1 -- 172.21.15.1:0/3307854645 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5597166b32b0 con 0x55971662e5d0 2024-09-11T22:04:58.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.565+0000 7f37d7fff640 1 -- 172.21.15.1:0/3307854645 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37ec005520 con 0x55971662e5d0 2024-09-11T22:04:58.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.565+0000 7f37d7fff640 1 -- 172.21.15.1:0/3307854645 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f37e8013ce0 con 0x55971662e5d0 2024-09-11T22:04:58.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.565+0000 7f3801165740 10 client.26590 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:04:58.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.565+0000 7f3801165740 10 client.26590 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-11T22:04:58.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.565+0000 7f3801165740 20 client.26590 populate_metadata read hostname 'smithi001' 2024-09-11T22:04:58.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.565+0000 7f3801165740 1 --2- 172.21.15.1:0/3307854645 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x5597166b3780 0x5597166b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:04:58.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.565+0000 7f3801165740 4 client.26590 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:04:58.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.565+0000 7f3801165740 1 -- 172.21.15.1:0/3307854645 --> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55971659bd70 con 0x5597166b3780 2024-09-11T22:04:58.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.566+0000 7f37f7fff640 1 --2- 172.21.15.1:0/3307854645 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x5597166b3780 0x5597166b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:04:58.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.566+0000 7f37f7fff640 1 --2- 172.21.15.1:0/3307854645 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x5597166b3780 0x5597166b5b70 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5de4be81319cbcd8 server_cookie=bb973a4b9c8e8b95 in_seq=0 out_seq=0 2024-09-11T22:04:58.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.566+0000 7f37d7fff640 10 client.26590 ms_handle_connect on v2:172.21.15.161:6834/828381592 2024-09-11T22:04:58.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.566+0000 7f37d7fff640 1 -- 172.21.15.1:0/3307854645 <== mds.4 v2:172.21.15.161:6834/828381592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55971659bd70 con 0x5597166b3780 2024-09-11T22:04:58.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.566+0000 7f37d7fff640 10 client.26590 handle_command_reply: tid=0 2024-09-11T22:04:58.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.566+0000 7f3801165740 1 client.26590 shutdown 2024-09-11T22:04:58.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.567+0000 7f3801165740 1 -- 172.21.15.1:0/3307854645 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55971662e5d0 msgr2=0x55971664e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:58.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.567+0000 7f3801165740 1 --2- 172.21.15.1:0/3307854645 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55971662e5d0 0x55971664e4d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f37e80cc9a0 tx=0x7f37e800dfa0 comp rx=0 tx=0).stop 2024-09-11T22:04:58.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.567+0000 7f3801165740 1 -- 172.21.15.1:0/3307854645 shutdown_connections 2024-09-11T22:04:58.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.567+0000 7f3801165740 1 --2- 172.21.15.1:0/3307854645 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559716639370 0x55971664ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:58.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.567+0000 7f3801165740 1 --2- 172.21.15.1:0/3307854645 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x5597166b3780 0x5597166b5b70 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:58.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.567+0000 7f3801165740 1 --2- 172.21.15.1:0/3307854645 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55971662fb40 0x55971664ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:58.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.567+0000 7f3801165740 1 --2- 172.21.15.1:0/3307854645 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55971662e5d0 0x55971664e4d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:58.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.567+0000 7f3801165740 1 -- 172.21.15.1:0/3307854645 >> 172.21.15.1:0/3307854645 conn(0x5597165f7310 msgr2=0x5597166294e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:58.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.567+0000 7f37d7fff640 0 client.26590 ms_handle_reset on v2:172.21.15.161:6834/828381592 2024-09-11T22:04:58.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.567+0000 7f3801165740 1 -- 172.21.15.1:0/3307854645 shutdown_connections 2024-09-11T22:04:58.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.567+0000 7f3801165740 1 -- 172.21.15.1:0/3307854645 wait complete. 2024-09-11T22:04:58.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.567+0000 7f3801165740 20 client.26590 trim_cache size 0 max 16384 2024-09-11T22:04:58.575 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-11T22:04:58.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.567+0000 7f37fed8c640 1 -- 172.21.15.1:0/641641640 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f37e00a2cf0 msgr2=0x7f37e00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:58.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.567+0000 7f37fed8c640 1 --2- 172.21.15.1:0/641641640 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f37e00a2cf0 0x7f37e00a51b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x55971662f070 tx=0x7f37f0004ab0 comp rx=0 tx=0).stop 2024-09-11T22:04:58.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.568+0000 7f37fed8c640 1 -- 172.21.15.1:0/641641640 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37f8102d40 msgr2=0x7f37f81009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:04:58.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.568+0000 7f37fed8c640 1 --2- 172.21.15.1:0/641641640 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37f8102d40 0x7f37f81009a0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f37e800d950 tx=0x7f37e800de10 comp rx=0 tx=0).stop 2024-09-11T22:04:58.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.568+0000 7f37fed8c640 1 -- 172.21.15.1:0/641641640 shutdown_connections 2024-09-11T22:04:58.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.568+0000 7f37fed8c640 1 --2- 172.21.15.1:0/641641640 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f37e00a2cf0 0x7f37e00a51b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:58.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.568+0000 7f37fed8c640 1 --2- 172.21.15.1:0/641641640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f37f810c170 0x7f37f8101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:58.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.568+0000 7f37fed8c640 1 --2- 172.21.15.1:0/641641640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f37f8103660 0x7f37f8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:58.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.568+0000 7f37fed8c640 1 --2- 172.21.15.1:0/641641640 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37f8102d40 0x7f37f81009a0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:04:58.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.568+0000 7f37fed8c640 1 -- 172.21.15.1:0/641641640 >> 172.21.15.1:0/641641640 conn(0x7f37f80fc7a0 msgr2=0x7f37f80fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:04:58.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.568+0000 7f37fed8c640 1 -- 172.21.15.1:0/641641640 shutdown_connections 2024-09-11T22:04:58.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:04:58.568+0000 7f37fed8c640 1 -- 172.21.15.1:0/641641640 wait complete. 2024-09-11T22:04:59.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:04:59 smithi001 ceph-mon[30935]: pgmap v239: 97 pgs: 97 active+clean; 469 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.0 MiB/s wr, 285 op/s 2024-09-11T22:04:59.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:04:59 smithi053 ceph-mon[34908]: pgmap v239: 97 pgs: 97 active+clean; 469 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.0 MiB/s wr, 285 op/s 2024-09-11T22:04:59.312 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T22:04:59.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:04:59 smithi161 ceph-mon[35615]: pgmap v239: 97 pgs: 97 active+clean; 469 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.0 MiB/s wr, 285 op/s 2024-09-11T22:04:59.570 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:05:00.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.070+0000 7f494c54c640 1 -- 172.21.15.1:0/3512779307 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4944102da0 msgr2=0x7f4944103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:00.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.070+0000 7f494c54c640 1 --2- 172.21.15.1:0/3512779307 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4944102da0 0x7f4944103180 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f4940009900 tx=0x7f494002a480 comp rx=0 tx=0).stop 2024-09-11T22:05:00.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.070+0000 7f494c54c640 1 -- 172.21.15.1:0/3512779307 shutdown_connections 2024-09-11T22:05:00.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.070+0000 7f494c54c640 1 --2- 172.21.15.1:0/3512779307 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f494410c180 0x7f494410e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:00.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.070+0000 7f494c54c640 1 --2- 172.21.15.1:0/3512779307 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49441036c0 0x7f494410bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:00.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.070+0000 7f494c54c640 1 --2- 172.21.15.1:0/3512779307 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4944102da0 0x7f4944103180 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:00.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.070+0000 7f494c54c640 1 -- 172.21.15.1:0/3512779307 >> 172.21.15.1:0/3512779307 conn(0x7f49440fc7a0 msgr2=0x7f49440febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:00.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.070+0000 7f494c54c640 1 -- 172.21.15.1:0/3512779307 shutdown_connections 2024-09-11T22:05:00.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.070+0000 7f494c54c640 1 -- 172.21.15.1:0/3512779307 wait complete. 2024-09-11T22:05:00.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.071+0000 7f494c54c640 1 Processor -- start 2024-09-11T22:05:00.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.071+0000 7f494c54c640 1 -- start start 2024-09-11T22:05:00.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.071+0000 7f494c54c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4944102da0 0x7f49441984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:00.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.071+0000 7f494c54c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49441036c0 0x7f49441989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:00.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.071+0000 7f494c54c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f494410c180 0x7f494419cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:00.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.071+0000 7f494c54c640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4944111f60 con 0x7f49441036c0 2024-09-11T22:05:00.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.071+0000 7f494c54c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4944111de0 con 0x7f4944102da0 2024-09-11T22:05:00.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.071+0000 7f494c54c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f49441120e0 con 0x7f494410c180 2024-09-11T22:05:00.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.071+0000 7f4949ae7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49441036c0 0x7f49441989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:00.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.071+0000 7f4949ae7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49441036c0 0x7f49441989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59512/0 (socket says 172.21.15.1:59512) 2024-09-11T22:05:00.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.071+0000 7f4949ae7640 1 -- 172.21.15.1:0/2240596938 learned_addr learned my addr 172.21.15.1:0/2240596938 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:00.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.071+0000 7f494a2e8640 1 --2- 172.21.15.1:0/2240596938 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4944102da0 0x7f49441984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:00.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.071+0000 7f494aae9640 1 --2- 172.21.15.1:0/2240596938 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f494410c180 0x7f494419cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:00.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.071+0000 7f4949ae7640 1 -- 172.21.15.1:0/2240596938 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f494410c180 msgr2=0x7f494419cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:00.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.071+0000 7f4949ae7640 1 --2- 172.21.15.1:0/2240596938 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f494410c180 0x7f494419cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:00.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.071+0000 7f4949ae7640 1 -- 172.21.15.1:0/2240596938 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4944102da0 msgr2=0x7f49441984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:00.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.071+0000 7f4949ae7640 1 --2- 172.21.15.1:0/2240596938 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4944102da0 0x7f49441984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:00.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.071+0000 7f4949ae7640 1 -- 172.21.15.1:0/2240596938 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f494419d460 con 0x7f49441036c0 2024-09-11T22:05:00.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.071+0000 7f494aae9640 1 --2- 172.21.15.1:0/2240596938 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f494410c180 0x7f494419cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:05:00.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.071+0000 7f494a2e8640 1 --2- 172.21.15.1:0/2240596938 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4944102da0 0x7f49441984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:05:00.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.072+0000 7f4949ae7640 1 --2- 172.21.15.1:0/2240596938 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49441036c0 0x7f49441989f0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f493400d770 tx=0x7f493400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:00.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.072+0000 7f493b7fe640 1 -- 172.21.15.1:0/2240596938 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4934017070 con 0x7f49441036c0 2024-09-11T22:05:00.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.072+0000 7f493b7fe640 1 -- 172.21.15.1:0/2240596938 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4934005500 con 0x7f49441036c0 2024-09-11T22:05:00.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.072+0000 7f493b7fe640 1 -- 172.21.15.1:0/2240596938 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4934004e00 con 0x7f49441036c0 2024-09-11T22:05:00.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.072+0000 7f494c54c640 1 -- 172.21.15.1:0/2240596938 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f494419d750 con 0x7f49441036c0 2024-09-11T22:05:00.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.072+0000 7f494c54c640 1 -- 172.21.15.1:0/2240596938 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49441a5030 con 0x7f49441036c0 2024-09-11T22:05:00.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.074+0000 7f493b7fe640 1 -- 172.21.15.1:0/2240596938 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f4934004fa0 con 0x7f49441036c0 2024-09-11T22:05:00.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.074+0000 7f494e925740 1 Processor -- start 2024-09-11T22:05:00.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.075+0000 7f494e925740 1 -- start start 2024-09-11T22:05:00.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.075+0000 7f494e925740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5654a5971b40 0x5654a5970090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:00.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.075+0000 7f494e925740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5654a59705d0 0x5654a5970a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:00.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.075+0000 7f494e925740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5654a597f780 0x5654a5981b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:00.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.075+0000 7f494e925740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5654a58d62f0 con 0x5654a59705d0 2024-09-11T22:05:00.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.075+0000 7f494e925740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5654a587dcf0 con 0x5654a597f780 2024-09-11T22:05:00.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.075+0000 7f494e925740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5654a58d22b0 con 0x5654a5971b40 2024-09-11T22:05:00.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.075+0000 7f494a2e8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5654a59705d0 0x5654a5970a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:00.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.075+0000 7f494a2e8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5654a59705d0 0x5654a5970a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59526/0 (socket says 172.21.15.1:59526) 2024-09-11T22:05:00.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.075+0000 7f494a2e8640 1 -- 172.21.15.1:0/3598154226 learned_addr learned my addr 172.21.15.1:0/3598154226 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:00.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.075+0000 7f494aae9640 1 --2- 172.21.15.1:0/3598154226 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5654a5971b40 0x5654a5970090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:00.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.075+0000 7f493b7fe640 1 --2- 172.21.15.1:0/2240596938 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f49300a2d70 0x7f49300a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:00.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.075+0000 7f4949ae7640 1 --2- 172.21.15.1:0/3598154226 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5654a597f780 0x5654a5981b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:00.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.075+0000 7f494a2e8640 1 -- 172.21.15.1:0/3598154226 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5654a5971b40 msgr2=0x5654a5970090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:00.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.075+0000 7f494aae9640 1 --2- 172.21.15.1:0/2240596938 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f49300a2d70 0x7f49300a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:00.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.075+0000 7f494a2e8640 1 --2- 172.21.15.1:0/3598154226 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5654a5971b40 0x5654a5970090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:00.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.075+0000 7f494a2e8640 1 -- 172.21.15.1:0/3598154226 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5654a597f780 msgr2=0x5654a5981b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:00.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.075+0000 7f494a2e8640 1 --2- 172.21.15.1:0/3598154226 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5654a597f780 0x5654a5981b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:00.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.075+0000 7f494a2e8640 1 -- 172.21.15.1:0/3598154226 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654a58623a0 con 0x5654a59705d0 2024-09-11T22:05:00.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.075+0000 7f4949ae7640 1 --2- 172.21.15.1:0/3598154226 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5654a597f780 0x5654a5981b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:05:00.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.076+0000 7f493b7fe640 1 -- 172.21.15.1:0/2240596938 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7f49340cef40 con 0x7f49441036c0 2024-09-11T22:05:00.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.076+0000 7f494a2e8640 1 --2- 172.21.15.1:0/3598154226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5654a59705d0 0x5654a5970a10 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f4940033000 tx=0x7f4940032cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=520c43c87e3c8c37 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:00.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.076+0000 7f494aae9640 1 --2- 172.21.15.1:0/2240596938 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f49300a2d70 0x7f49300a5230 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f493c005fd0 tx=0x7f493c004ab0 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:00.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.076+0000 7f4938ff9640 1 -- 172.21.15.1:0/3598154226 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f494003f070 con 0x5654a59705d0 2024-09-11T22:05:00.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.076+0000 7f4938ff9640 1 -- 172.21.15.1:0/3598154226 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49400317b0 con 0x5654a59705d0 2024-09-11T22:05:00.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.076+0000 7f4938ff9640 1 -- 172.21.15.1:0/3598154226 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4940035070 con 0x5654a59705d0 2024-09-11T22:05:00.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.077+0000 7f494e925740 1 -- 172.21.15.1:0/3598154226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5654a59705d0 msgr2=0x5654a5970a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:00.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.077+0000 7f494e925740 1 --2- 172.21.15.1:0/3598154226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5654a59705d0 0x5654a5970a10 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f4940033000 tx=0x7f4940032cb0 comp rx=0 tx=0).stop 2024-09-11T22:05:00.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.077+0000 7f494e925740 1 -- 172.21.15.1:0/3598154226 shutdown_connections 2024-09-11T22:05:00.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.077+0000 7f494e925740 1 --2- 172.21.15.1:0/3598154226 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5654a597f780 0x5654a5981b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:00.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.077+0000 7f494e925740 1 --2- 172.21.15.1:0/3598154226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5654a59705d0 0x5654a5970a10 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:00.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.077+0000 7f494e925740 1 --2- 172.21.15.1:0/3598154226 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5654a5971b40 0x5654a5970090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:00.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.077+0000 7f494e925740 1 -- 172.21.15.1:0/3598154226 >> 172.21.15.1:0/3598154226 conn(0x5654a5939310 msgr2=0x5654a596cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:00.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.077+0000 7f494e925740 1 -- 172.21.15.1:0/3598154226 shutdown_connections 2024-09-11T22:05:00.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.077+0000 7f494e925740 1 -- 172.21.15.1:0/3598154226 wait complete. 2024-09-11T22:05:00.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.077+0000 7f494e925740 1 -- start start 2024-09-11T22:05:00.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.077+0000 7f494e925740 1 Processor -- start 2024-09-11T22:05:00.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.077+0000 7f494e925740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5654a59705d0 0x5654a597f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:00.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.077+0000 7f494e925740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5654a5971b40 0x5654a597c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:00.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.077+0000 7f494e925740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5654a597f780 0x5654a597ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:00.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.077+0000 7f494e925740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5654a5982420 con 0x5654a59705d0 2024-09-11T22:05:00.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.077+0000 7f494e925740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5654a5891060 con 0x5654a597f780 2024-09-11T22:05:00.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.077+0000 7f494e925740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5654a59825a0 con 0x5654a5971b40 2024-09-11T22:05:00.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.078+0000 7f494e925740 10 client.0 _ll_register_callbacks cb 0x5654a55e48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:05:00.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.078+0000 7f494a2e8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5654a59705d0 0x5654a597f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:00.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.078+0000 7f494a2e8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5654a59705d0 0x5654a597f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59532/0 (socket says 172.21.15.1:59532) 2024-09-11T22:05:00.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.078+0000 7f494a2e8640 1 -- 172.21.15.1:0/2513930853 learned_addr learned my addr 172.21.15.1:0/2513930853 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:00.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.078+0000 7f4949ae7640 1 --2- 172.21.15.1:0/2513930853 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5654a5971b40 0x5654a597c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:00.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.078+0000 7f494aae9640 1 --2- 172.21.15.1:0/2513930853 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5654a597f780 0x5654a597ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:00.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.078+0000 7f494a2e8640 1 -- 172.21.15.1:0/2513930853 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5654a5971b40 msgr2=0x5654a597c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:00.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.078+0000 7f494a2e8640 1 --2- 172.21.15.1:0/2513930853 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5654a5971b40 0x5654a597c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:00.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.078+0000 7f494a2e8640 1 -- 172.21.15.1:0/2513930853 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5654a597f780 msgr2=0x5654a597ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:00.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.078+0000 7f494a2e8640 1 --2- 172.21.15.1:0/2513930853 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5654a597f780 0x5654a597ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:00.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.078+0000 7f494a2e8640 1 -- 172.21.15.1:0/2513930853 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654a56473c0 con 0x5654a59705d0 2024-09-11T22:05:00.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.078+0000 7f494a2e8640 1 --2- 172.21.15.1:0/2513930853 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5654a59705d0 0x5654a597f190 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f4940032d20 tx=0x7f49400052d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a057ad3d37266d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:00.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.078+0000 7f492dffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:05:00.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.078+0000 7f492dffb640 1 -- 172.21.15.1:0/2513930853 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4940035070 con 0x5654a59705d0 2024-09-11T22:05:00.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.078+0000 7f492dffb640 1 -- 172.21.15.1:0/2513930853 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7f4940031960 con 0x5654a59705d0 2024-09-11T22:05:00.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.078+0000 7f494e925740 1 -- 172.21.15.1:0/2513930853 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49400317b0 con 0x5654a59705d0 2024-09-11T22:05:00.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.078+0000 7f492dffb640 1 -- 172.21.15.1:0/2513930853 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f4940055050 con 0x5654a59705d0 2024-09-11T22:05:00.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.079+0000 7f494e925740 10 client.17154 fetch_fsmap learned FSMap version 42 2024-09-11T22:05:00.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.079+0000 7f494e925740 1 -- 172.21.15.1:0/2513930853 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5654a598de60 con 0x5654a59705d0 2024-09-11T22:05:00.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.079+0000 7f492dffb640 1 -- 172.21.15.1:0/2513930853 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49400317b0 con 0x5654a59705d0 2024-09-11T22:05:00.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.079+0000 7f492dffb640 1 -- 172.21.15.1:0/2513930853 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f494003bb70 con 0x5654a59705d0 2024-09-11T22:05:00.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.079+0000 7f494e925740 10 client.17154 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:05:00.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.079+0000 7f494e925740 10 client.17154 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-11T22:05:00.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.079+0000 7f494e925740 20 client.17154 populate_metadata read hostname 'smithi001' 2024-09-11T22:05:00.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.079+0000 7f494e925740 1 --2- 172.21.15.1:0/2513930853 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x5654a598f1b0 0x5654a59915a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:00.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.079+0000 7f494e925740 4 client.17154 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:05:00.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.079+0000 7f494e925740 1 -- 172.21.15.1:0/2513930853 --> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654a58e3860 con 0x5654a598f1b0 2024-09-11T22:05:00.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.079+0000 7f4949ae7640 1 --2- 172.21.15.1:0/2513930853 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x5654a598f1b0 0x5654a59915a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:00.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.080+0000 7f4949ae7640 1 --2- 172.21.15.1:0/2513930853 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x5654a598f1b0 0x5654a59915a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=12a40222d080dbe4 server_cookie=fbc9f6e0a1ba5473 in_seq=0 out_seq=0 2024-09-11T22:05:00.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.080+0000 7f492dffb640 10 client.17154 ms_handle_connect on v2:172.21.15.161:6836/1799318656 2024-09-11T22:05:00.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.080+0000 7f492dffb640 1 -- 172.21.15.1:0/2513930853 <== mds.3 v2:172.21.15.161:6836/1799318656 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5654a58e3860 con 0x5654a598f1b0 2024-09-11T22:05:00.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.080+0000 7f492dffb640 10 client.17154 handle_command_reply: tid=0 2024-09-11T22:05:00.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.081+0000 7f494e925740 1 client.17154 shutdown 2024-09-11T22:05:00.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.081+0000 7f494e925740 1 -- 172.21.15.1:0/2513930853 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5654a59705d0 msgr2=0x5654a597f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:00.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.081+0000 7f494e925740 1 --2- 172.21.15.1:0/2513930853 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5654a59705d0 0x5654a597f190 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f4940032d20 tx=0x7f49400052d0 comp rx=0 tx=0).stop 2024-09-11T22:05:00.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.081+0000 7f494e925740 1 -- 172.21.15.1:0/2513930853 shutdown_connections 2024-09-11T22:05:00.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.081+0000 7f494e925740 1 --2- 172.21.15.1:0/2513930853 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x5654a598f1b0 0x5654a59915a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:00.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.081+0000 7f494e925740 1 --2- 172.21.15.1:0/2513930853 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5654a597f780 0x5654a597ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:00.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.081+0000 7f494e925740 1 --2- 172.21.15.1:0/2513930853 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5654a5971b40 0x5654a597c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:00.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.081+0000 7f492dffb640 0 client.17154 ms_handle_reset on v2:172.21.15.161:6836/1799318656 2024-09-11T22:05:00.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.081+0000 7f494e925740 1 --2- 172.21.15.1:0/2513930853 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5654a59705d0 0x5654a597f190 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:00.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.081+0000 7f494e925740 1 -- 172.21.15.1:0/2513930853 >> 172.21.15.1:0/2513930853 conn(0x5654a5939310 msgr2=0x5654a5972960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:00.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.081+0000 7f494e925740 1 -- 172.21.15.1:0/2513930853 shutdown_connections 2024-09-11T22:05:00.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.081+0000 7f494e925740 1 -- 172.21.15.1:0/2513930853 wait complete. 2024-09-11T22:05:00.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.081+0000 7f494e925740 20 client.17154 trim_cache size 0 max 16384 2024-09-11T22:05:00.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.094+0000 7f494e925740 1 Processor -- start 2024-09-11T22:05:00.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.094+0000 7f494e925740 1 -- start start 2024-09-11T22:05:00.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.094+0000 7f494e925740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5654a59705d0 0x5654a598b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:00.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.094+0000 7f494e925740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5654a5971b40 0x5654a597b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:00.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.094+0000 7f494e925740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5654a597f780 0x5654a5996c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:00.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.094+0000 7f494e925740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f49441ab790 con 0x5654a597f780 2024-09-11T22:05:00.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.094+0000 7f494e925740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f49441ad5d0 con 0x5654a59705d0 2024-09-11T22:05:00.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.094+0000 7f494e925740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f49441abf20 con 0x5654a5971b40 2024-09-11T22:05:00.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.094+0000 7f494aae9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5654a597f780 0x5654a5996c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:00.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.094+0000 7f494aae9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5654a597f780 0x5654a5996c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59542/0 (socket says 172.21.15.1:59542) 2024-09-11T22:05:00.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.094+0000 7f494aae9640 1 -- 172.21.15.1:0/1815571100 learned_addr learned my addr 172.21.15.1:0/1815571100 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:00.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.094+0000 7f494a2e8640 1 --2- 172.21.15.1:0/1815571100 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5654a59705d0 0x5654a598b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:00.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.094+0000 7f4949ae7640 1 --2- 172.21.15.1:0/1815571100 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5654a5971b40 0x5654a597b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:00.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.094+0000 7f494aae9640 1 -- 172.21.15.1:0/1815571100 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5654a5971b40 msgr2=0x5654a597b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:00.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.094+0000 7f494aae9640 1 --2- 172.21.15.1:0/1815571100 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5654a5971b40 0x5654a597b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:00.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.094+0000 7f494aae9640 1 -- 172.21.15.1:0/1815571100 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5654a59705d0 msgr2=0x5654a598b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:00.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.094+0000 7f494aae9640 1 --2- 172.21.15.1:0/1815571100 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5654a59705d0 0x5654a598b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:00.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.094+0000 7f494aae9640 1 -- 172.21.15.1:0/1815571100 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654a5971740 con 0x5654a597f780 2024-09-11T22:05:00.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.094+0000 7f494a2e8640 1 --2- 172.21.15.1:0/1815571100 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5654a59705d0 0x5654a598b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:05:00.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.095+0000 7f494aae9640 1 --2- 172.21.15.1:0/1815571100 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5654a597f780 0x5654a5996c50 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f493c0126c0 tx=0x7f493c016070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9e42b699e2ee893 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:00.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.095+0000 7f492ffff640 1 -- 172.21.15.1:0/1815571100 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f493c021800 con 0x5654a597f780 2024-09-11T22:05:00.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.095+0000 7f492ffff640 1 -- 172.21.15.1:0/1815571100 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f493c02a440 con 0x5654a597f780 2024-09-11T22:05:00.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.095+0000 7f492ffff640 1 -- 172.21.15.1:0/1815571100 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f493c01d690 con 0x5654a597f780 2024-09-11T22:05:00.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.095+0000 7f494e925740 1 -- 172.21.15.1:0/1815571100 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5654a597f780 msgr2=0x5654a5996c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:00.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.095+0000 7f494e925740 1 --2- 172.21.15.1:0/1815571100 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5654a597f780 0x5654a5996c50 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f493c0126c0 tx=0x7f493c016070 comp rx=0 tx=0).stop 2024-09-11T22:05:00.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.095+0000 7f494e925740 1 -- 172.21.15.1:0/1815571100 shutdown_connections 2024-09-11T22:05:00.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.095+0000 7f494e925740 1 --2- 172.21.15.1:0/1815571100 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5654a597f780 0x5654a5996c50 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:00.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.095+0000 7f494e925740 1 --2- 172.21.15.1:0/1815571100 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5654a5971b40 0x5654a597b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:00.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.095+0000 7f494e925740 1 --2- 172.21.15.1:0/1815571100 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5654a59705d0 0x5654a598b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:00.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.095+0000 7f494e925740 1 -- 172.21.15.1:0/1815571100 >> 172.21.15.1:0/1815571100 conn(0x5654a5939310 msgr2=0x5654a597ad10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:00.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.096+0000 7f494e925740 1 -- 172.21.15.1:0/1815571100 shutdown_connections 2024-09-11T22:05:00.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.096+0000 7f494e925740 1 -- 172.21.15.1:0/1815571100 wait complete. 2024-09-11T22:05:00.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.096+0000 7f494e925740 1 -- start start 2024-09-11T22:05:00.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.096+0000 7f494e925740 1 Processor -- start 2024-09-11T22:05:00.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.096+0000 7f494e925740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5654a59705d0 0x5654a59880e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:00.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.096+0000 7f494e925740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5654a5971b40 0x5654a5983410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:00.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.096+0000 7f494e925740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5654a597f780 0x5654a5983ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:00.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.096+0000 7f494e925740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f49441ace40 con 0x5654a59705d0 2024-09-11T22:05:00.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.096+0000 7f494e925740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f49441ac6b0 con 0x5654a597f780 2024-09-11T22:05:00.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.096+0000 7f494e925740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f49441a9790 con 0x5654a5971b40 2024-09-11T22:05:00.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.096+0000 7f494e925740 10 client.0 _ll_register_callbacks cb 0x7f493c0096e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:05:00.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.096+0000 7f494a2e8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5654a59705d0 0x5654a59880e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:00.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.096+0000 7f494a2e8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5654a59705d0 0x5654a59880e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59546/0 (socket says 172.21.15.1:59546) 2024-09-11T22:05:00.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.096+0000 7f494a2e8640 1 -- 172.21.15.1:0/2750635936 learned_addr learned my addr 172.21.15.1:0/2750635936 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:00.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.096+0000 7f4949ae7640 1 --2- 172.21.15.1:0/2750635936 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5654a5971b40 0x5654a5983410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:00.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.097+0000 7f494aae9640 1 --2- 172.21.15.1:0/2750635936 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5654a597f780 0x5654a5983ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:00.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.097+0000 7f494a2e8640 1 -- 172.21.15.1:0/2750635936 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5654a5971b40 msgr2=0x5654a5983410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:00.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.097+0000 7f494a2e8640 1 --2- 172.21.15.1:0/2750635936 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5654a5971b40 0x5654a5983410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:00.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.097+0000 7f494a2e8640 1 -- 172.21.15.1:0/2750635936 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5654a597f780 msgr2=0x5654a5983ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:00.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.097+0000 7f494a2e8640 1 --2- 172.21.15.1:0/2750635936 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5654a597f780 0x5654a5983ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:00.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.097+0000 7f494a2e8640 1 -- 172.21.15.1:0/2750635936 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654a59f8c20 con 0x5654a59705d0 2024-09-11T22:05:00.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.097+0000 7f494aae9640 1 --2- 172.21.15.1:0/2750635936 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5654a597f780 0x5654a5983ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:05:00.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.097+0000 7f494a2e8640 1 --2- 172.21.15.1:0/2750635936 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5654a59705d0 0x5654a59880e0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f4940005340 tx=0x7f4940005920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bdf68e241414bc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:00.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.097+0000 7f492dffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:05:00.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.097+0000 7f492dffb640 1 -- 172.21.15.1:0/2750635936 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4940035040 con 0x5654a59705d0 2024-09-11T22:05:00.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.097+0000 7f492dffb640 1 -- 172.21.15.1:0/2750635936 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7f4940005370 con 0x5654a59705d0 2024-09-11T22:05:00.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.097+0000 7f494e925740 1 -- 172.21.15.1:0/2750635936 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f493c02a440 con 0x5654a59705d0 2024-09-11T22:05:00.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.097+0000 7f492dffb640 1 -- 172.21.15.1:0/2750635936 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f494003f040 con 0x5654a59705d0 2024-09-11T22:05:00.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.097+0000 7f494e925740 10 client.17160 fetch_fsmap learned FSMap version 42 2024-09-11T22:05:00.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.097+0000 7f494e925740 1 -- 172.21.15.1:0/2750635936 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5654a59f9d80 con 0x5654a59705d0 2024-09-11T22:05:00.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.097+0000 7f492dffb640 1 -- 172.21.15.1:0/2750635936 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f493c02a440 con 0x5654a59705d0 2024-09-11T22:05:00.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.098+0000 7f492dffb640 1 -- 172.21.15.1:0/2750635936 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f494003b430 con 0x5654a59705d0 2024-09-11T22:05:00.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.098+0000 7f494e925740 10 client.17160 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:05:00.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.098+0000 7f494e925740 10 client.17160 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-11T22:05:00.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.098+0000 7f494e925740 20 client.17160 populate_metadata read hostname 'smithi001' 2024-09-11T22:05:00.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.098+0000 7f494e925740 1 --2- 172.21.15.1:0/2750635936 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x5654a59fa550 0x5654a59fc970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:00.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.098+0000 7f494e925740 4 client.17160 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:05:00.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.098+0000 7f494e925740 1 -- 172.21.15.1:0/2750635936 --> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5654a58ddd70 con 0x5654a59fa550 2024-09-11T22:05:00.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.098+0000 7f4949ae7640 1 --2- 172.21.15.1:0/2750635936 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x5654a59fa550 0x5654a59fc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:00.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.099+0000 7f492dffb640 10 client.17160 ms_handle_connect on v2:172.21.15.161:6836/1799318656 2024-09-11T22:05:00.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.099+0000 7f4949ae7640 1 --2- 172.21.15.1:0/2750635936 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x5654a59fa550 0x5654a59fc970 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=66366b066480b2d2 server_cookie=3e51be01d7dcb5cd in_seq=0 out_seq=0 2024-09-11T22:05:00.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.099+0000 7f492dffb640 1 -- 172.21.15.1:0/2750635936 <== mds.3 v2:172.21.15.161:6836/1799318656 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5654a58ddd70 con 0x5654a59fa550 2024-09-11T22:05:00.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.099+0000 7f492dffb640 10 client.17160 handle_command_reply: tid=0 2024-09-11T22:05:00.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.099+0000 7f494e925740 1 client.17160 shutdown 2024-09-11T22:05:00.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.099+0000 7f494e925740 1 -- 172.21.15.1:0/2750635936 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5654a59705d0 msgr2=0x5654a59880e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:00.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.099+0000 7f494e925740 1 --2- 172.21.15.1:0/2750635936 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5654a59705d0 0x5654a59880e0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f4940005340 tx=0x7f4940005920 comp rx=0 tx=0).stop 2024-09-11T22:05:00.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.099+0000 7f494e925740 1 -- 172.21.15.1:0/2750635936 shutdown_connections 2024-09-11T22:05:00.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.099+0000 7f494e925740 1 --2- 172.21.15.1:0/2750635936 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x5654a59fa550 0x5654a59fc970 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:00.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.099+0000 7f494e925740 1 --2- 172.21.15.1:0/2750635936 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5654a597f780 0x5654a5983ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:00.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.099+0000 7f494e925740 1 --2- 172.21.15.1:0/2750635936 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5654a5971b40 0x5654a5983410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:00.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.099+0000 7f492dffb640 0 client.17160 ms_handle_reset on v2:172.21.15.161:6836/1799318656 2024-09-11T22:05:00.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.099+0000 7f494e925740 1 --2- 172.21.15.1:0/2750635936 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5654a59705d0 0x5654a59880e0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:00.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.099+0000 7f494e925740 1 -- 172.21.15.1:0/2750635936 >> 172.21.15.1:0/2750635936 conn(0x5654a5939310 msgr2=0x5654a597b530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:00.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.099+0000 7f494e925740 1 -- 172.21.15.1:0/2750635936 shutdown_connections 2024-09-11T22:05:00.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.099+0000 7f494e925740 1 -- 172.21.15.1:0/2750635936 wait complete. 2024-09-11T22:05:00.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.099+0000 7f494e925740 20 client.17160 trim_cache size 0 max 16384 2024-09-11T22:05:00.115 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-11T22:05:00.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.100+0000 7f494c54c640 1 -- 172.21.15.1:0/2240596938 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f49300a2d70 msgr2=0x7f49300a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:00.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.100+0000 7f494c54c640 1 --2- 172.21.15.1:0/2240596938 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f49300a2d70 0x7f49300a5230 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f493c005fd0 tx=0x7f493c004ab0 comp rx=0 tx=0).stop 2024-09-11T22:05:00.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.100+0000 7f494c54c640 1 -- 172.21.15.1:0/2240596938 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49441036c0 msgr2=0x7f49441989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:00.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.100+0000 7f494c54c640 1 --2- 172.21.15.1:0/2240596938 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49441036c0 0x7f49441989f0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f493400d770 tx=0x7f493400dc30 comp rx=0 tx=0).stop 2024-09-11T22:05:00.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.100+0000 7f494c54c640 1 -- 172.21.15.1:0/2240596938 shutdown_connections 2024-09-11T22:05:00.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.100+0000 7f494c54c640 1 --2- 172.21.15.1:0/2240596938 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f49300a2d70 0x7f49300a5230 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:00.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.100+0000 7f494c54c640 1 --2- 172.21.15.1:0/2240596938 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f494410c180 0x7f494419cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:00.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.100+0000 7f494c54c640 1 --2- 172.21.15.1:0/2240596938 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49441036c0 0x7f49441989f0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:00.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.100+0000 7f494c54c640 1 --2- 172.21.15.1:0/2240596938 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4944102da0 0x7f49441984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:00.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.100+0000 7f494c54c640 1 -- 172.21.15.1:0/2240596938 >> 172.21.15.1:0/2240596938 conn(0x7f49440fc7a0 msgr2=0x7f49440fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:00.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.100+0000 7f494c54c640 1 -- 172.21.15.1:0/2240596938 shutdown_connections 2024-09-11T22:05:00.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:00.100+0000 7f494c54c640 1 -- 172.21.15.1:0/2240596938 wait complete. 2024-09-11T22:05:00.983 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T22:05:01.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:01 smithi001 ceph-mon[30935]: pgmap v240: 97 pgs: 97 active+clean; 469 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.0 MiB/s wr, 291 op/s 2024-09-11T22:05:01.252 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:05:01.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:01 smithi053 ceph-mon[34908]: pgmap v240: 97 pgs: 97 active+clean; 469 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.0 MiB/s wr, 291 op/s 2024-09-11T22:05:01.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:01 smithi161 ceph-mon[35615]: pgmap v240: 97 pgs: 97 active+clean; 469 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.0 MiB/s wr, 291 op/s 2024-09-11T22:05:01.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.749+0000 7f455b065640 1 -- 172.21.15.1:0/4097306926 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4554102790 msgr2=0x7f45541067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:01.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.749+0000 7f455b065640 1 --2- 172.21.15.1:0/4097306926 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4554102790 0x7f45541067e0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f4544009900 tx=0x7f454402a410 comp rx=0 tx=0).stop 2024-09-11T22:05:01.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.749+0000 7f455b065640 1 -- 172.21.15.1:0/4097306926 shutdown_connections 2024-09-11T22:05:01.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.749+0000 7f455b065640 1 --2- 172.21.15.1:0/4097306926 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4554106f10 0x7f455410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:01.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.749+0000 7f455b065640 1 --2- 172.21.15.1:0/4097306926 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4554102790 0x7f45541067e0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:01.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.749+0000 7f455b065640 1 --2- 172.21.15.1:0/4097306926 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4554101de0 0x7f45541021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:01.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.749+0000 7f455b065640 1 -- 172.21.15.1:0/4097306926 >> 172.21.15.1:0/4097306926 conn(0x7f45540fd8c0 msgr2=0x7f45540ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:01.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.749+0000 7f455b065640 1 -- 172.21.15.1:0/4097306926 shutdown_connections 2024-09-11T22:05:01.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.749+0000 7f455b065640 1 -- 172.21.15.1:0/4097306926 wait complete. 2024-09-11T22:05:01.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.749+0000 7f455b065640 1 Processor -- start 2024-09-11T22:05:01.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.750+0000 7f455b065640 1 -- start start 2024-09-11T22:05:01.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.750+0000 7f455b065640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4554101de0 0x7f4554198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:01.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.750+0000 7f455b065640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4554102790 0x7f4554198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:01.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.750+0000 7f455b065640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4554106f10 0x7f455419cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:01.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.750+0000 7f455b065640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4554111f70 con 0x7f4554101de0 2024-09-11T22:05:01.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.750+0000 7f455b065640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4554111df0 con 0x7f4554102790 2024-09-11T22:05:01.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.750+0000 7f455b065640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f45541120f0 con 0x7f4554106f10 2024-09-11T22:05:01.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.750+0000 7f4558e01640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4554101de0 0x7f4554198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:01.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.750+0000 7f4558e01640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4554101de0 0x7f4554198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59562/0 (socket says 172.21.15.1:59562) 2024-09-11T22:05:01.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.750+0000 7f4558e01640 1 -- 172.21.15.1:0/4047807602 learned_addr learned my addr 172.21.15.1:0/4047807602 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:01.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.750+0000 7f454bfff640 1 --2- 172.21.15.1:0/4047807602 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4554102790 0x7f4554198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:01.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.750+0000 7f4559602640 1 --2- 172.21.15.1:0/4047807602 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4554106f10 0x7f455419cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:01.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.750+0000 7f4558e01640 1 -- 172.21.15.1:0/4047807602 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4554106f10 msgr2=0x7f455419cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:01.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.750+0000 7f4558e01640 1 --2- 172.21.15.1:0/4047807602 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4554106f10 0x7f455419cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:01.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.750+0000 7f4558e01640 1 -- 172.21.15.1:0/4047807602 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4554102790 msgr2=0x7f4554198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:01.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.750+0000 7f4558e01640 1 --2- 172.21.15.1:0/4047807602 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4554102790 0x7f4554198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:01.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.750+0000 7f4558e01640 1 -- 172.21.15.1:0/4047807602 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f455419d400 con 0x7f4554101de0 2024-09-11T22:05:01.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.750+0000 7f4558e01640 1 --2- 172.21.15.1:0/4047807602 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4554101de0 0x7f4554198450 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f455000c890 tx=0x7f455000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:01.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.751+0000 7f4549ffb640 1 -- 172.21.15.1:0/4047807602 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4550004220 con 0x7f4554101de0 2024-09-11T22:05:01.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.751+0000 7f455b065640 1 -- 172.21.15.1:0/4047807602 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f455419d6f0 con 0x7f4554101de0 2024-09-11T22:05:01.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.751+0000 7f4549ffb640 1 -- 172.21.15.1:0/4047807602 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45500043c0 con 0x7f4554101de0 2024-09-11T22:05:01.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.751+0000 7f4549ffb640 1 -- 172.21.15.1:0/4047807602 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4550004a00 con 0x7f4554101de0 2024-09-11T22:05:01.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.751+0000 7f455b065640 1 -- 172.21.15.1:0/4047807602 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45541a4fd0 con 0x7f4554101de0 2024-09-11T22:05:01.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.754+0000 7f455d43e740 1 Processor -- start 2024-09-11T22:05:01.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.754+0000 7f455d43e740 1 -- start start 2024-09-11T22:05:01.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.754+0000 7f4549ffb640 1 -- 172.21.15.1:0/4047807602 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f45500079b0 con 0x7f4554101de0 2024-09-11T22:05:01.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.754+0000 7f455d43e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5574842a2b40 0x5574842a1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:01.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.754+0000 7f455d43e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5574842a15d0 0x5574842a1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:01.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.754+0000 7f455d43e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5574842b0780 0x5574842b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:01.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.754+0000 7f455d43e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5574842072f0 con 0x5574842b0780 2024-09-11T22:05:01.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.754+0000 7f455d43e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5574841aecf0 con 0x5574842a2b40 2024-09-11T22:05:01.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.754+0000 7f455d43e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5574842032b0 con 0x5574842a15d0 2024-09-11T22:05:01.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.754+0000 7f454bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5574842b0780 0x5574842b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:01.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.754+0000 7f454bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5574842b0780 0x5574842b2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59570/0 (socket says 172.21.15.1:59570) 2024-09-11T22:05:01.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.754+0000 7f454bfff640 1 -- 172.21.15.1:0/923506062 learned_addr learned my addr 172.21.15.1:0/923506062 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:01.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.754+0000 7f4559602640 1 --2- 172.21.15.1:0/923506062 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5574842a2b40 0x5574842a1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:01.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.754+0000 7f4558e01640 1 --2- 172.21.15.1:0/923506062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5574842a15d0 0x5574842a1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:01.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.754+0000 7f454bfff640 1 -- 172.21.15.1:0/923506062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5574842a15d0 msgr2=0x5574842a1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:01.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.754+0000 7f454bfff640 1 --2- 172.21.15.1:0/923506062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5574842a15d0 0x5574842a1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:01.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.754+0000 7f454bfff640 1 -- 172.21.15.1:0/923506062 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5574842a2b40 msgr2=0x5574842a1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:01.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.754+0000 7f454bfff640 1 --2- 172.21.15.1:0/923506062 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5574842a2b40 0x5574842a1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:01.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.754+0000 7f454bfff640 1 -- 172.21.15.1:0/923506062 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574841933a0 con 0x5574842b0780 2024-09-11T22:05:01.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.754+0000 7f4558e01640 1 --2- 172.21.15.1:0/923506062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5574842a15d0 0x5574842a1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:05:01.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.755+0000 7f454bfff640 1 --2- 172.21.15.1:0/923506062 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5574842b0780 0x5574842b2b70 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f4544009900 tx=0x7f4544031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c008c2d27e3d1c2b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:01.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.755+0000 7f4549ffb640 1 --2- 172.21.15.1:0/4047807602 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f453c0a2cf0 0x7f453c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:01.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.755+0000 7f453affd640 1 -- 172.21.15.1:0/923506062 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4544040070 con 0x5574842b0780 2024-09-11T22:05:01.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.755+0000 7f4559602640 1 --2- 172.21.15.1:0/4047807602 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f453c0a2cf0 0x7f453c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:01.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.755+0000 7f4549ffb640 1 -- 172.21.15.1:0/4047807602 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7f45500cf1c0 con 0x7f4554101de0 2024-09-11T22:05:01.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.755+0000 7f453affd640 1 -- 172.21.15.1:0/923506062 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4544005020 con 0x5574842b0780 2024-09-11T22:05:01.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.755+0000 7f453affd640 1 -- 172.21.15.1:0/923506062 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4544035040 con 0x5574842b0780 2024-09-11T22:05:01.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.756+0000 7f455d43e740 1 -- 172.21.15.1:0/923506062 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5574842b0780 msgr2=0x5574842b2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:01.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.756+0000 7f4559602640 1 --2- 172.21.15.1:0/4047807602 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f453c0a2cf0 0x7f453c0a51b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f454c0092f0 tx=0x7f454c009280 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:01.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.756+0000 7f455d43e740 1 --2- 172.21.15.1:0/923506062 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5574842b0780 0x5574842b2b70 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f4544009900 tx=0x7f4544031630 comp rx=0 tx=0).stop 2024-09-11T22:05:01.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.756+0000 7f455d43e740 1 -- 172.21.15.1:0/923506062 shutdown_connections 2024-09-11T22:05:01.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.756+0000 7f455d43e740 1 --2- 172.21.15.1:0/923506062 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5574842b0780 0x5574842b2b70 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:01.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.756+0000 7f455d43e740 1 --2- 172.21.15.1:0/923506062 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5574842a15d0 0x5574842a1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:01.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.756+0000 7f455d43e740 1 --2- 172.21.15.1:0/923506062 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5574842a2b40 0x5574842a1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:01.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.756+0000 7f455d43e740 1 -- 172.21.15.1:0/923506062 >> 172.21.15.1:0/923506062 conn(0x55748426a310 msgr2=0x55748429dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:01.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.756+0000 7f455d43e740 1 -- 172.21.15.1:0/923506062 shutdown_connections 2024-09-11T22:05:01.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.756+0000 7f455d43e740 1 -- 172.21.15.1:0/923506062 wait complete. 2024-09-11T22:05:01.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.756+0000 7f455d43e740 1 -- start start 2024-09-11T22:05:01.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.756+0000 7f455d43e740 1 Processor -- start 2024-09-11T22:05:01.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.757+0000 7f455d43e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5574842a15d0 0x5574842b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:01.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.757+0000 7f455d43e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5574842a2b40 0x5574842ad370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:01.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.757+0000 7f455d43e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5574842b0780 0x5574842ad8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:01.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.757+0000 7f455d43e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5574842b3420 con 0x5574842a15d0 2024-09-11T22:05:01.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.757+0000 7f455d43e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5574841c2060 con 0x5574842a2b40 2024-09-11T22:05:01.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.757+0000 7f455d43e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5574842b35a0 con 0x5574842b0780 2024-09-11T22:05:01.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.757+0000 7f455d43e740 10 client.0 _ll_register_callbacks cb 0x557483f158c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:05:01.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.757+0000 7f4558e01640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5574842a15d0 0x5574842b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:01.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.757+0000 7f4558e01640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5574842a15d0 0x5574842b00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59582/0 (socket says 172.21.15.1:59582) 2024-09-11T22:05:01.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.757+0000 7f4558e01640 1 -- 172.21.15.1:0/2833812978 learned_addr learned my addr 172.21.15.1:0/2833812978 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:01.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.757+0000 7f454bfff640 1 --2- 172.21.15.1:0/2833812978 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5574842a2b40 0x5574842ad370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:01.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.757+0000 7f4559602640 1 --2- 172.21.15.1:0/2833812978 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5574842b0780 0x5574842ad8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:01.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.757+0000 7f4558e01640 1 -- 172.21.15.1:0/2833812978 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5574842b0780 msgr2=0x5574842ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:01.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.757+0000 7f4558e01640 1 --2- 172.21.15.1:0/2833812978 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5574842b0780 0x5574842ad8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:01.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.757+0000 7f4558e01640 1 -- 172.21.15.1:0/2833812978 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5574842a2b40 msgr2=0x5574842ad370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:01.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.757+0000 7f4558e01640 1 --2- 172.21.15.1:0/2833812978 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5574842a2b40 0x5574842ad370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:01.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.757+0000 7f4558e01640 1 -- 172.21.15.1:0/2833812978 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557483f783c0 con 0x5574842a15d0 2024-09-11T22:05:01.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.757+0000 7f454bfff640 1 --2- 172.21.15.1:0/2833812978 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5574842a2b40 0x5574842ad370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:05:01.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.758+0000 7f4558e01640 1 --2- 172.21.15.1:0/2833812978 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5574842a15d0 0x5574842b00c0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f45500cc900 tx=0x7f455009b440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=887f869b36c0915c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:01.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.758+0000 7f4533fff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:05:01.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.758+0000 7f4533fff640 1 -- 172.21.15.1:0/2833812978 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f455001d040 con 0x5574842a15d0 2024-09-11T22:05:01.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.758+0000 7f4533fff640 1 -- 172.21.15.1:0/2833812978 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7f4550007ca0 con 0x5574842a15d0 2024-09-11T22:05:01.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.758+0000 7f455d43e740 1 -- 172.21.15.1:0/2833812978 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4544005020 con 0x5574842a15d0 2024-09-11T22:05:01.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.758+0000 7f4533fff640 1 -- 172.21.15.1:0/2833812978 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f455009bd70 con 0x5574842a15d0 2024-09-11T22:05:01.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.758+0000 7f455d43e740 10 client.17172 fetch_fsmap learned FSMap version 42 2024-09-11T22:05:01.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.758+0000 7f455d43e740 1 -- 172.21.15.1:0/2833812978 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5574842bed80 con 0x5574842a15d0 2024-09-11T22:05:01.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.758+0000 7f4533fff640 1 -- 172.21.15.1:0/2833812978 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4544005020 con 0x5574842a15d0 2024-09-11T22:05:01.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.758+0000 7f4533fff640 1 -- 172.21.15.1:0/2833812978 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f4550084bb0 con 0x5574842a15d0 2024-09-11T22:05:01.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.759+0000 7f455d43e740 10 client.17172 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:05:01.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.759+0000 7f455d43e740 10 client.17172 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-11T22:05:01.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.759+0000 7f455d43e740 20 client.17172 populate_metadata read hostname 'smithi001' 2024-09-11T22:05:01.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.759+0000 7f455d43e740 1 --2- 172.21.15.1:0/2833812978 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x5574842c01e0 0x5574842c26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:01.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.759+0000 7f455d43e740 4 client.17172 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:05:01.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.759+0000 7f455d43e740 1 -- 172.21.15.1:0/2833812978 --> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557484214860 con 0x5574842c01e0 2024-09-11T22:05:01.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.759+0000 7f454bfff640 1 --2- 172.21.15.1:0/2833812978 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x5574842c01e0 0x5574842c26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:01.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.759+0000 7f454bfff640 1 --2- 172.21.15.1:0/2833812978 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x5574842c01e0 0x5574842c26a0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e320be3268ce4b90 server_cookie=c0471d30f881973c in_seq=0 out_seq=0 2024-09-11T22:05:01.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.759+0000 7f4533fff640 10 client.17172 ms_handle_connect on v2:172.21.15.53:6836/3457131830 2024-09-11T22:05:01.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.760+0000 7f4533fff640 1 -- 172.21.15.1:0/2833812978 <== mds.2 v2:172.21.15.53:6836/3457131830 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557484214860 con 0x5574842c01e0 2024-09-11T22:05:01.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.760+0000 7f4533fff640 10 client.17172 handle_command_reply: tid=0 2024-09-11T22:05:01.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.760+0000 7f455d43e740 1 client.17172 shutdown 2024-09-11T22:05:01.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.760+0000 7f455d43e740 1 -- 172.21.15.1:0/2833812978 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5574842a15d0 msgr2=0x5574842b00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:01.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.760+0000 7f455d43e740 1 --2- 172.21.15.1:0/2833812978 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5574842a15d0 0x5574842b00c0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f45500cc900 tx=0x7f455009b440 comp rx=0 tx=0).stop 2024-09-11T22:05:01.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.760+0000 7f455d43e740 1 -- 172.21.15.1:0/2833812978 shutdown_connections 2024-09-11T22:05:01.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.760+0000 7f455d43e740 1 --2- 172.21.15.1:0/2833812978 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x5574842c01e0 0x5574842c26a0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:01.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.760+0000 7f455d43e740 1 --2- 172.21.15.1:0/2833812978 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5574842b0780 0x5574842ad8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:01.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.760+0000 7f455d43e740 1 --2- 172.21.15.1:0/2833812978 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5574842a2b40 0x5574842ad370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:01.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.760+0000 7f455d43e740 1 --2- 172.21.15.1:0/2833812978 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5574842a15d0 0x5574842b00c0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:01.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.760+0000 7f455d43e740 1 -- 172.21.15.1:0/2833812978 >> 172.21.15.1:0/2833812978 conn(0x55748426a310 msgr2=0x5574842a3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:01.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.760+0000 7f4533fff640 0 client.17172 ms_handle_reset on v2:172.21.15.53:6836/3457131830 2024-09-11T22:05:01.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.761+0000 7f455d43e740 1 -- 172.21.15.1:0/2833812978 shutdown_connections 2024-09-11T22:05:01.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.761+0000 7f455d43e740 1 -- 172.21.15.1:0/2833812978 wait complete. 2024-09-11T22:05:01.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.761+0000 7f455d43e740 20 client.17172 trim_cache size 0 max 16384 2024-09-11T22:05:01.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.774+0000 7f455d43e740 1 Processor -- start 2024-09-11T22:05:01.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.774+0000 7f455d43e740 1 -- start start 2024-09-11T22:05:01.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.774+0000 7f455d43e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5574842a15d0 0x5574842bc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:01.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.774+0000 7f455d43e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5574842a2b40 0x5574842ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:01.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.774+0000 7f455d43e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5574842b0780 0x5574842c7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:01.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.774+0000 7f455d43e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f45541ab8c0 con 0x5574842b0780 2024-09-11T22:05:01.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.774+0000 7f455d43e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f45541ad700 con 0x5574842a15d0 2024-09-11T22:05:01.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.774+0000 7f455d43e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f45541ac050 con 0x5574842a2b40 2024-09-11T22:05:01.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.774+0000 7f4559602640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5574842b0780 0x5574842c7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:01.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.774+0000 7f4559602640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5574842b0780 0x5574842c7f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59594/0 (socket says 172.21.15.1:59594) 2024-09-11T22:05:01.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.774+0000 7f4559602640 1 -- 172.21.15.1:0/4183463025 learned_addr learned my addr 172.21.15.1:0/4183463025 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:01.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.774+0000 7f4558e01640 1 --2- 172.21.15.1:0/4183463025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5574842a15d0 0x5574842bc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:01.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.774+0000 7f454bfff640 1 --2- 172.21.15.1:0/4183463025 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5574842a2b40 0x5574842ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:01.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.774+0000 7f4559602640 1 -- 172.21.15.1:0/4183463025 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5574842a2b40 msgr2=0x5574842ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:01.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.774+0000 7f4559602640 1 --2- 172.21.15.1:0/4183463025 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5574842a2b40 0x5574842ac8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:01.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.774+0000 7f4559602640 1 -- 172.21.15.1:0/4183463025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5574842a15d0 msgr2=0x5574842bc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:01.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.774+0000 7f4559602640 1 --2- 172.21.15.1:0/4183463025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5574842a15d0 0x5574842bc6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:01.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.774+0000 7f4559602640 1 -- 172.21.15.1:0/4183463025 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574842afce0 con 0x5574842b0780 2024-09-11T22:05:01.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.775+0000 7f4559602640 1 --2- 172.21.15.1:0/4183463025 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5574842b0780 0x5574842c7f70 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f454c0127a0 tx=0x7f454c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8336fe61124db36 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:01.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.775+0000 7f453a7fc640 1 -- 172.21.15.1:0/4183463025 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c01f700 con 0x5574842b0780 2024-09-11T22:05:01.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.775+0000 7f453a7fc640 1 -- 172.21.15.1:0/4183463025 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f454c01fd80 con 0x5574842b0780 2024-09-11T22:05:01.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.775+0000 7f453a7fc640 1 -- 172.21.15.1:0/4183463025 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c0283d0 con 0x5574842b0780 2024-09-11T22:05:01.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.775+0000 7f455d43e740 1 -- 172.21.15.1:0/4183463025 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5574842b0780 msgr2=0x5574842c7f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:01.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.775+0000 7f455d43e740 1 --2- 172.21.15.1:0/4183463025 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5574842b0780 0x5574842c7f70 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f454c0127a0 tx=0x7f454c014000 comp rx=0 tx=0).stop 2024-09-11T22:05:01.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.775+0000 7f455d43e740 1 -- 172.21.15.1:0/4183463025 shutdown_connections 2024-09-11T22:05:01.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.775+0000 7f455d43e740 1 --2- 172.21.15.1:0/4183463025 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5574842b0780 0x5574842c7f70 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:01.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.775+0000 7f455d43e740 1 --2- 172.21.15.1:0/4183463025 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5574842a2b40 0x5574842ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:01.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.775+0000 7f455d43e740 1 --2- 172.21.15.1:0/4183463025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5574842a15d0 0x5574842bc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:01.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.775+0000 7f455d43e740 1 -- 172.21.15.1:0/4183463025 >> 172.21.15.1:0/4183463025 conn(0x55748426a310 msgr2=0x5574842abc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:01.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.776+0000 7f455d43e740 1 -- 172.21.15.1:0/4183463025 shutdown_connections 2024-09-11T22:05:01.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.776+0000 7f455d43e740 1 -- 172.21.15.1:0/4183463025 wait complete. 2024-09-11T22:05:01.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.776+0000 7f455d43e740 1 -- start start 2024-09-11T22:05:01.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.776+0000 7f455d43e740 1 Processor -- start 2024-09-11T22:05:01.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.776+0000 7f455d43e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5574842a15d0 0x5574842b48c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:01.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.776+0000 7f455d43e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5574842a2b40 0x5574842b4e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:01.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.776+0000 7f455d43e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5574842b0780 0x5574842b5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:01.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.776+0000 7f455d43e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f45541acf70 con 0x5574842a2b40 2024-09-11T22:05:01.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.777+0000 7f455d43e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f45541ac7e0 con 0x5574842a15d0 2024-09-11T22:05:01.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.777+0000 7f455d43e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f45541a98c0 con 0x5574842b0780 2024-09-11T22:05:01.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.777+0000 7f455d43e740 10 client.0 _ll_register_callbacks cb 0x7f45500058c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:05:01.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.777+0000 7f454bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5574842a2b40 0x5574842b4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:01.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.777+0000 7f454bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5574842a2b40 0x5574842b4e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59602/0 (socket says 172.21.15.1:59602) 2024-09-11T22:05:01.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.777+0000 7f454bfff640 1 -- 172.21.15.1:0/1255150079 learned_addr learned my addr 172.21.15.1:0/1255150079 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:01.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.777+0000 7f4558e01640 1 --2- 172.21.15.1:0/1255150079 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5574842a15d0 0x5574842b48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:01.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.777+0000 7f454bfff640 1 -- 172.21.15.1:0/1255150079 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5574842b0780 msgr2=0x5574842b5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:01.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.777+0000 7f454bfff640 1 --2- 172.21.15.1:0/1255150079 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5574842b0780 0x5574842b5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:01.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.777+0000 7f454bfff640 1 -- 172.21.15.1:0/1255150079 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5574842a15d0 msgr2=0x5574842b48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:01.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.777+0000 7f454bfff640 1 --2- 172.21.15.1:0/1255150079 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5574842a15d0 0x5574842b48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:01.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.777+0000 7f454bfff640 1 -- 172.21.15.1:0/1255150079 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55748432a210 con 0x5574842a2b40 2024-09-11T22:05:01.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.777+0000 7f4558e01640 1 --2- 172.21.15.1:0/1255150079 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5574842a15d0 0x5574842b48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:05:01.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.777+0000 7f454bfff640 1 --2- 172.21.15.1:0/1255150079 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5574842a2b40 0x5574842b4e00 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f4544000c00 tx=0x7f4544031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c275fcd4967f2ae2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:01.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.777+0000 7f4533fff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:05:01.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.777+0000 7f4533fff640 1 -- 172.21.15.1:0/1255150079 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4544035040 con 0x5574842a2b40 2024-09-11T22:05:01.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.777+0000 7f455d43e740 1 -- 172.21.15.1:0/1255150079 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f454c01fd80 con 0x5574842a2b40 2024-09-11T22:05:01.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.777+0000 7f4533fff640 1 -- 172.21.15.1:0/1255150079 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7f4544004240 con 0x5574842a2b40 2024-09-11T22:05:01.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.777+0000 7f4533fff640 1 -- 172.21.15.1:0/1255150079 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f454405d050 con 0x5574842a2b40 2024-09-11T22:05:01.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.778+0000 7f455d43e740 10 client.17178 fetch_fsmap learned FSMap version 42 2024-09-11T22:05:01.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.778+0000 7f455d43e740 1 -- 172.21.15.1:0/1255150079 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55748432b510 con 0x5574842a2b40 2024-09-11T22:05:01.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.778+0000 7f4533fff640 1 -- 172.21.15.1:0/1255150079 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f454c01fd80 con 0x5574842a2b40 2024-09-11T22:05:01.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.778+0000 7f4533fff640 1 -- 172.21.15.1:0/1255150079 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f45440048f0 con 0x5574842a2b40 2024-09-11T22:05:01.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.778+0000 7f455d43e740 10 client.17178 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:05:01.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.778+0000 7f455d43e740 10 client.17178 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-11T22:05:01.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.778+0000 7f455d43e740 20 client.17178 populate_metadata read hostname 'smithi001' 2024-09-11T22:05:01.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.778+0000 7f455d43e740 1 --2- 172.21.15.1:0/1255150079 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x55748432b9c0 0x55748432dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:01.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.778+0000 7f455d43e740 4 client.17178 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:05:01.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.778+0000 7f455d43e740 1 -- 172.21.15.1:0/1255150079 --> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55748420ed70 con 0x55748432b9c0 2024-09-11T22:05:01.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.778+0000 7f4558e01640 1 --2- 172.21.15.1:0/1255150079 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x55748432b9c0 0x55748432dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:01.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.779+0000 7f4558e01640 1 --2- 172.21.15.1:0/1255150079 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x55748432b9c0 0x55748432dde0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=97de598aae1a6a81 server_cookie=965f0d1d1b32072b in_seq=0 out_seq=0 2024-09-11T22:05:01.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.779+0000 7f4533fff640 10 client.17178 ms_handle_connect on v2:172.21.15.53:6836/3457131830 2024-09-11T22:05:01.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.779+0000 7f4533fff640 1 -- 172.21.15.1:0/1255150079 <== mds.2 v2:172.21.15.53:6836/3457131830 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55748420ed70 con 0x55748432b9c0 2024-09-11T22:05:01.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.779+0000 7f4533fff640 10 client.17178 handle_command_reply: tid=0 2024-09-11T22:05:01.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.779+0000 7f455d43e740 1 client.17178 shutdown 2024-09-11T22:05:01.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.779+0000 7f455d43e740 1 -- 172.21.15.1:0/1255150079 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5574842a2b40 msgr2=0x5574842b4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:01.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.779+0000 7f455d43e740 1 --2- 172.21.15.1:0/1255150079 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5574842a2b40 0x5574842b4e00 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f4544000c00 tx=0x7f4544031400 comp rx=0 tx=0).stop 2024-09-11T22:05:01.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.779+0000 7f455d43e740 1 -- 172.21.15.1:0/1255150079 shutdown_connections 2024-09-11T22:05:01.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.779+0000 7f455d43e740 1 --2- 172.21.15.1:0/1255150079 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x55748432b9c0 0x55748432dde0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:01.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.779+0000 7f455d43e740 1 --2- 172.21.15.1:0/1255150079 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5574842b0780 0x5574842b5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:01.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.779+0000 7f455d43e740 1 --2- 172.21.15.1:0/1255150079 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5574842a2b40 0x5574842b4e00 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:01.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.779+0000 7f455d43e740 1 --2- 172.21.15.1:0/1255150079 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5574842a15d0 0x5574842b48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:01.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.779+0000 7f455d43e740 1 -- 172.21.15.1:0/1255150079 >> 172.21.15.1:0/1255150079 conn(0x55748426a310 msgr2=0x5574842aebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:01.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.779+0000 7f4533fff640 0 client.17178 ms_handle_reset on v2:172.21.15.53:6836/3457131830 2024-09-11T22:05:01.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.779+0000 7f455d43e740 1 -- 172.21.15.1:0/1255150079 shutdown_connections 2024-09-11T22:05:01.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.779+0000 7f455d43e740 1 -- 172.21.15.1:0/1255150079 wait complete. 2024-09-11T22:05:01.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.779+0000 7f455d43e740 20 client.17178 trim_cache size 0 max 16384 2024-09-11T22:05:01.787 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-11T22:05:01.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.780+0000 7f455b065640 1 -- 172.21.15.1:0/4047807602 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f453c0a2cf0 msgr2=0x7f453c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:01.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.780+0000 7f455b065640 1 --2- 172.21.15.1:0/4047807602 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f453c0a2cf0 0x7f453c0a51b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f454c0092f0 tx=0x7f454c009280 comp rx=0 tx=0).stop 2024-09-11T22:05:01.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.780+0000 7f455b065640 1 -- 172.21.15.1:0/4047807602 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4554101de0 msgr2=0x7f4554198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:01.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.780+0000 7f455b065640 1 --2- 172.21.15.1:0/4047807602 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4554101de0 0x7f4554198450 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f455000c890 tx=0x7f455000cd50 comp rx=0 tx=0).stop 2024-09-11T22:05:01.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.780+0000 7f455b065640 1 -- 172.21.15.1:0/4047807602 shutdown_connections 2024-09-11T22:05:01.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.780+0000 7f455b065640 1 --2- 172.21.15.1:0/4047807602 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f453c0a2cf0 0x7f453c0a51b0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:01.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.780+0000 7f455b065640 1 --2- 172.21.15.1:0/4047807602 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4554106f10 0x7f455419cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:01.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.780+0000 7f455b065640 1 --2- 172.21.15.1:0/4047807602 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4554102790 0x7f4554198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:01.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.780+0000 7f455b065640 1 --2- 172.21.15.1:0/4047807602 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4554101de0 0x7f4554198450 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:01.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.780+0000 7f455b065640 1 -- 172.21.15.1:0/4047807602 >> 172.21.15.1:0/4047807602 conn(0x7f45540fd8c0 msgr2=0x7f45540fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:01.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.780+0000 7f455b065640 1 -- 172.21.15.1:0/4047807602 shutdown_connections 2024-09-11T22:05:01.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:01.780+0000 7f455b065640 1 -- 172.21.15.1:0/4047807602 wait complete. 2024-09-11T22:05:02.343 INFO:tasks.workunit.client.0.smithi001.stdout:ok 2024-09-11T22:05:02.481 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-11T22:05:02.481 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-11T22:05:02.528 INFO:tasks.workunit:Running workunit fs/misc/filelock_interrupt.py... 2024-09-11T22:05:02.529 DEBUG:teuthology.orchestra.run.smithi001:workunit test fs/misc/filelock_interrupt.py> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=086c737fa75c22d521a3932c59b31c5e1d1c9ef4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/filelock_interrupt.py 2024-09-11T22:05:02.561 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T22:05:02.858 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:05:03.276 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:03 smithi001 ceph-mon[30935]: pgmap v241: 97 pgs: 97 active+clean; 469 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 411 KiB/s wr, 193 op/s 2024-09-11T22:05:03.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:03 smithi053 ceph-mon[34908]: pgmap v241: 97 pgs: 97 active+clean; 469 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 411 KiB/s wr, 193 op/s 2024-09-11T22:05:03.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:03 smithi161 ceph-mon[35615]: pgmap v241: 97 pgs: 97 active+clean; 469 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 411 KiB/s wr, 193 op/s 2024-09-11T22:05:03.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.360+0000 7f30111a3640 1 -- 172.21.15.1:0/1882824631 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f300c108290 msgr2=0x7f300c10ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:03.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.360+0000 7f30111a3640 1 --2- 172.21.15.1:0/1882824631 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f300c108290 0x7f300c10ec30 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f3004009cd0 tx=0x7f300402a8e0 comp rx=0 tx=0).stop 2024-09-11T22:05:03.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.360+0000 7f30111a3640 1 -- 172.21.15.1:0/1882824631 shutdown_connections 2024-09-11T22:05:03.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.360+0000 7f30111a3640 1 --2- 172.21.15.1:0/1882824631 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f300c108290 0x7f300c10ec30 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:03.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.360+0000 7f30111a3640 1 --2- 172.21.15.1:0/1882824631 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f300c103740 0x7f300c107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:03.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.360+0000 7f30111a3640 1 --2- 172.21.15.1:0/1882824631 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f300c102d90 0x7f300c103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:03.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.360+0000 7f30111a3640 1 -- 172.21.15.1:0/1882824631 >> 172.21.15.1:0/1882824631 conn(0x7f300c0fc7d0 msgr2=0x7f300c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:03.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.360+0000 7f30111a3640 1 -- 172.21.15.1:0/1882824631 shutdown_connections 2024-09-11T22:05:03.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.360+0000 7f30111a3640 1 -- 172.21.15.1:0/1882824631 wait complete. 2024-09-11T22:05:03.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.361+0000 7f30111a3640 1 Processor -- start 2024-09-11T22:05:03.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.361+0000 7f30111a3640 1 -- start start 2024-09-11T22:05:03.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.361+0000 7f30111a3640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f300c102d90 0x7f300c198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:03.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.361+0000 7f30111a3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f300c103740 0x7f300c198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:03.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.361+0000 7f30111a3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f300c108290 0x7f300c19cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:03.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.361+0000 7f30111a3640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f300c1120d0 con 0x7f300c103740 2024-09-11T22:05:03.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.361+0000 7f30111a3640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f300c111f50 con 0x7f300c108290 2024-09-11T22:05:03.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.361+0000 7f30111a3640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f300c112250 con 0x7f300c102d90 2024-09-11T22:05:03.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.361+0000 7f300a59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f300c103740 0x7f300c198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:03.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.361+0000 7f300a59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f300c103740 0x7f300c198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59622/0 (socket says 172.21.15.1:59622) 2024-09-11T22:05:03.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.361+0000 7f300a59c640 1 -- 172.21.15.1:0/783885299 learned_addr learned my addr 172.21.15.1:0/783885299 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:03.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.361+0000 7f300ad9d640 1 --2- 172.21.15.1:0/783885299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f300c102d90 0x7f300c198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:03.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.361+0000 7f300b59e640 1 --2- 172.21.15.1:0/783885299 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f300c108290 0x7f300c19cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:03.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.362+0000 7f300a59c640 1 -- 172.21.15.1:0/783885299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f300c102d90 msgr2=0x7f300c198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:03.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.362+0000 7f300a59c640 1 --2- 172.21.15.1:0/783885299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f300c102d90 0x7f300c198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:03.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.362+0000 7f300a59c640 1 -- 172.21.15.1:0/783885299 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f300c108290 msgr2=0x7f300c19cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:03.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.362+0000 7f300a59c640 1 --2- 172.21.15.1:0/783885299 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f300c108290 0x7f300c19cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:03.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.362+0000 7f300a59c640 1 -- 172.21.15.1:0/783885299 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f300c19d4c0 con 0x7f300c103740 2024-09-11T22:05:03.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.362+0000 7f300a59c640 1 --2- 172.21.15.1:0/783885299 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f300c103740 0x7f300c198a50 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f300000c970 tx=0x7f300000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:03.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.362+0000 7f2ff3fff640 1 -- 172.21.15.1:0/783885299 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3000014070 con 0x7f300c103740 2024-09-11T22:05:03.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.362+0000 7f2ff3fff640 1 -- 172.21.15.1:0/783885299 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3000004440 con 0x7f300c103740 2024-09-11T22:05:03.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.362+0000 7f30111a3640 1 -- 172.21.15.1:0/783885299 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f300c19d7b0 con 0x7f300c103740 2024-09-11T22:05:03.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.362+0000 7f2ff3fff640 1 -- 172.21.15.1:0/783885299 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3000003e10 con 0x7f300c103740 2024-09-11T22:05:03.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.362+0000 7f30111a3640 1 -- 172.21.15.1:0/783885299 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f300c1a5090 con 0x7f300c103740 2024-09-11T22:05:03.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.364+0000 7f2ff3fff640 1 -- 172.21.15.1:0/783885299 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f3000017070 con 0x7f300c103740 2024-09-11T22:05:03.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.365+0000 7f301357c740 1 Processor -- start 2024-09-11T22:05:03.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.365+0000 7f301357c740 1 -- start start 2024-09-11T22:05:03.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.365+0000 7f301357c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af11a17b40 0x55af11a16090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:03.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.365+0000 7f301357c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af11a165d0 0x55af11a16a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:03.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.365+0000 7f301357c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55af11a21370 0x55af11a23760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:03.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.365+0000 7f301357c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55af1197c2f0 con 0x55af11a165d0 2024-09-11T22:05:03.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.365+0000 7f301357c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55af11923cf0 con 0x55af11a17b40 2024-09-11T22:05:03.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.365+0000 7f301357c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55af119782b0 con 0x55af11a21370 2024-09-11T22:05:03.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.365+0000 7f300ad9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af11a165d0 0x55af11a16a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:03.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.365+0000 7f300ad9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af11a165d0 0x55af11a16a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59638/0 (socket says 172.21.15.1:59638) 2024-09-11T22:05:03.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.365+0000 7f300ad9d640 1 -- 172.21.15.1:0/2733931627 learned_addr learned my addr 172.21.15.1:0/2733931627 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:03.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.365+0000 7f300b59e640 1 --2- 172.21.15.1:0/2733931627 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af11a17b40 0x55af11a16090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:03.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.365+0000 7f2ff3fff640 1 --2- 172.21.15.1:0/783885299 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f2ff40a2cf0 0x7f2ff40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:03.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.365+0000 7f300a59c640 1 --2- 172.21.15.1:0/2733931627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55af11a21370 0x55af11a23760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:03.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.365+0000 7f300ad9d640 1 -- 172.21.15.1:0/2733931627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55af11a21370 msgr2=0x55af11a23760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:03.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.365+0000 7f300ad9d640 1 --2- 172.21.15.1:0/2733931627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55af11a21370 0x55af11a23760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:03.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.365+0000 7f300ad9d640 1 -- 172.21.15.1:0/2733931627 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af11a17b40 msgr2=0x55af11a16090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:03.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.365+0000 7f300ad9d640 1 --2- 172.21.15.1:0/2733931627 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af11a17b40 0x55af11a16090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:03.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.365+0000 7f300ad9d640 1 -- 172.21.15.1:0/2733931627 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af119083a0 con 0x55af11a165d0 2024-09-11T22:05:03.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.365+0000 7f300a59c640 1 --2- 172.21.15.1:0/2733931627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55af11a21370 0x55af11a23760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:05:03.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.365+0000 7f300b59e640 1 --2- 172.21.15.1:0/783885299 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f2ff40a2cf0 0x7f2ff40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:03.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.366+0000 7f300ad9d640 1 --2- 172.21.15.1:0/2733931627 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af11a165d0 0x55af11a16a10 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc009a70 tx=0x7f2ffc009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd6f8011d0c70ab5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:03.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.366+0000 7f2ff3fff640 1 -- 172.21.15.1:0/783885299 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7f30000cf2e0 con 0x7f300c103740 2024-09-11T22:05:03.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.366+0000 7f2ff17fa640 1 -- 172.21.15.1:0/2733931627 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc017070 con 0x55af11a165d0 2024-09-11T22:05:03.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.366+0000 7f2ff17fa640 1 -- 172.21.15.1:0/2733931627 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ffc013470 con 0x55af11a165d0 2024-09-11T22:05:03.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.366+0000 7f2ff17fa640 1 -- 172.21.15.1:0/2733931627 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc0125f0 con 0x55af11a165d0 2024-09-11T22:05:03.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.367+0000 7f301357c740 1 -- 172.21.15.1:0/2733931627 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af11a165d0 msgr2=0x55af11a16a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:03.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.367+0000 7f300b59e640 1 --2- 172.21.15.1:0/783885299 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f2ff40a2cf0 0x7f2ff40a51b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f30040098d0 tx=0x7f30040095c0 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:03.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.367+0000 7f301357c740 1 --2- 172.21.15.1:0/2733931627 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af11a165d0 0x55af11a16a10 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc009a70 tx=0x7f2ffc009f30 comp rx=0 tx=0).stop 2024-09-11T22:05:03.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.367+0000 7f301357c740 1 -- 172.21.15.1:0/2733931627 shutdown_connections 2024-09-11T22:05:03.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.367+0000 7f301357c740 1 --2- 172.21.15.1:0/2733931627 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55af11a21370 0x55af11a23760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:03.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.367+0000 7f301357c740 1 --2- 172.21.15.1:0/2733931627 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af11a165d0 0x55af11a16a10 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:03.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.367+0000 7f301357c740 1 --2- 172.21.15.1:0/2733931627 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af11a17b40 0x55af11a16090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:03.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.367+0000 7f301357c740 1 -- 172.21.15.1:0/2733931627 >> 172.21.15.1:0/2733931627 conn(0x55af119df310 msgr2=0x55af11a12c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:03.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.367+0000 7f301357c740 1 -- 172.21.15.1:0/2733931627 shutdown_connections 2024-09-11T22:05:03.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.367+0000 7f301357c740 1 -- 172.21.15.1:0/2733931627 wait complete. 2024-09-11T22:05:03.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.367+0000 7f301357c740 1 -- start start 2024-09-11T22:05:03.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.367+0000 7f301357c740 1 Processor -- start 2024-09-11T22:05:03.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.367+0000 7f301357c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af11a165d0 0x55af11a20cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:03.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.367+0000 7f301357c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af11a17b40 0x55af11a1df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:03.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.367+0000 7f301357c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55af11a21370 0x55af11a1e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:03.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.367+0000 7f301357c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55af11a24070 con 0x55af11a17b40 2024-09-11T22:05:03.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.367+0000 7f301357c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55af11937060 con 0x55af11a165d0 2024-09-11T22:05:03.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.367+0000 7f301357c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55af11a241f0 con 0x55af11a21370 2024-09-11T22:05:03.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.367+0000 7f301357c740 10 client.0 _ll_register_callbacks cb 0x55af1168a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:05:03.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.367+0000 7f300a59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af11a17b40 0x55af11a1df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:03.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.367+0000 7f300a59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af11a17b40 0x55af11a1df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59652/0 (socket says 172.21.15.1:59652) 2024-09-11T22:05:03.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.367+0000 7f300a59c640 1 -- 172.21.15.1:0/1694023684 learned_addr learned my addr 172.21.15.1:0/1694023684 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:03.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.368+0000 7f300b59e640 1 --2- 172.21.15.1:0/1694023684 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55af11a21370 0x55af11a1e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:03.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.368+0000 7f300ad9d640 1 --2- 172.21.15.1:0/1694023684 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af11a165d0 0x55af11a20cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:03.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.368+0000 7f300a59c640 1 -- 172.21.15.1:0/1694023684 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55af11a21370 msgr2=0x55af11a1e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:03.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.368+0000 7f300a59c640 1 --2- 172.21.15.1:0/1694023684 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55af11a21370 0x55af11a1e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:03.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.368+0000 7f300a59c640 1 -- 172.21.15.1:0/1694023684 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af11a165d0 msgr2=0x55af11a20cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:03.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.368+0000 7f300a59c640 1 --2- 172.21.15.1:0/1694023684 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af11a165d0 0x55af11a20cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:03.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.368+0000 7f300a59c640 1 -- 172.21.15.1:0/1694023684 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af116ed3c0 con 0x55af11a17b40 2024-09-11T22:05:03.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.368+0000 7f300a59c640 1 --2- 172.21.15.1:0/1694023684 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af11a17b40 0x55af11a1df70 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f300000c940 tx=0x7f300009b2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb4e3b744d0cec8f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:03.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.368+0000 7f2feeff5640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:05:03.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.368+0000 7f2feeff5640 1 -- 172.21.15.1:0/1694023684 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f300009be90 con 0x55af11a17b40 2024-09-11T22:05:03.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.368+0000 7f2feeff5640 1 -- 172.21.15.1:0/1694023684 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7f300009f830 con 0x55af11a17b40 2024-09-11T22:05:03.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.368+0000 7f301357c740 1 -- 172.21.15.1:0/1694023684 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ffc013470 con 0x55af11a17b40 2024-09-11T22:05:03.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.368+0000 7f2feeff5640 1 -- 172.21.15.1:0/1694023684 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f30000a4400 con 0x55af11a17b40 2024-09-11T22:05:03.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.368+0000 7f301357c740 10 client.17190 fetch_fsmap learned FSMap version 42 2024-09-11T22:05:03.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.368+0000 7f301357c740 1 -- 172.21.15.1:0/1694023684 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55af11a2f9f0 con 0x55af11a17b40 2024-09-11T22:05:03.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.368+0000 7f2feeff5640 1 -- 172.21.15.1:0/1694023684 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ffc013470 con 0x55af11a17b40 2024-09-11T22:05:03.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.369+0000 7f2feeff5640 1 -- 172.21.15.1:0/1694023684 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f300009abe0 con 0x55af11a17b40 2024-09-11T22:05:03.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.369+0000 7f301357c740 10 client.17190 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:05:03.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.369+0000 7f301357c740 10 client.17190 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T22:05:03.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.369+0000 7f301357c740 20 client.17190 populate_metadata read hostname 'smithi001' 2024-09-11T22:05:03.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.369+0000 7f301357c740 1 --2- 172.21.15.1:0/1694023684 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x55af11a30f50 0x55af11a33360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:03.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.369+0000 7f301357c740 4 client.17190 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:05:03.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.369+0000 7f301357c740 1 -- 172.21.15.1:0/1694023684 --> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af11989860 con 0x55af11a30f50 2024-09-11T22:05:03.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.369+0000 7f300ad9d640 1 --2- 172.21.15.1:0/1694023684 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x55af11a30f50 0x55af11a33360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:03.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.369+0000 7f300ad9d640 1 --2- 172.21.15.1:0/1694023684 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x55af11a30f50 0x55af11a33360 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9c6fcfeaefb9c6f0 server_cookie=1369b265b2d8190d in_seq=0 out_seq=0 2024-09-11T22:05:03.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.369+0000 7f2feeff5640 10 client.17190 ms_handle_connect on v2:172.21.15.161:6838/455365383 2024-09-11T22:05:03.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.370+0000 7f2feeff5640 1 -- 172.21.15.1:0/1694023684 <== mds.1 v2:172.21.15.161:6838/455365383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55af11989860 con 0x55af11a30f50 2024-09-11T22:05:03.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.370+0000 7f2feeff5640 10 client.17190 handle_command_reply: tid=0 2024-09-11T22:05:03.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.370+0000 7f301357c740 1 client.17190 shutdown 2024-09-11T22:05:03.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.370+0000 7f301357c740 1 -- 172.21.15.1:0/1694023684 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af11a17b40 msgr2=0x55af11a1df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:03.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.370+0000 7f301357c740 1 --2- 172.21.15.1:0/1694023684 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af11a17b40 0x55af11a1df70 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f300000c940 tx=0x7f300009b2d0 comp rx=0 tx=0).stop 2024-09-11T22:05:03.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.370+0000 7f301357c740 1 -- 172.21.15.1:0/1694023684 shutdown_connections 2024-09-11T22:05:03.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.370+0000 7f301357c740 1 --2- 172.21.15.1:0/1694023684 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x55af11a30f50 0x55af11a33360 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:03.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.370+0000 7f301357c740 1 --2- 172.21.15.1:0/1694023684 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55af11a21370 0x55af11a1e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:03.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.370+0000 7f301357c740 1 --2- 172.21.15.1:0/1694023684 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af11a17b40 0x55af11a1df70 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:03.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.370+0000 7f2feeff5640 0 client.17190 ms_handle_reset on v2:172.21.15.161:6838/455365383 2024-09-11T22:05:03.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.370+0000 7f301357c740 1 --2- 172.21.15.1:0/1694023684 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af11a165d0 0x55af11a20cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:03.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.370+0000 7f301357c740 1 -- 172.21.15.1:0/1694023684 >> 172.21.15.1:0/1694023684 conn(0x55af119df310 msgr2=0x55af11a18960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:03.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.370+0000 7f301357c740 1 -- 172.21.15.1:0/1694023684 shutdown_connections 2024-09-11T22:05:03.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.370+0000 7f301357c740 1 -- 172.21.15.1:0/1694023684 wait complete. 2024-09-11T22:05:03.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.370+0000 7f301357c740 20 client.17190 trim_cache size 0 max 16384 2024-09-11T22:05:03.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.383+0000 7f301357c740 1 Processor -- start 2024-09-11T22:05:03.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.383+0000 7f301357c740 1 -- start start 2024-09-11T22:05:03.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.384+0000 7f301357c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55af11a165d0 0x55af11a12e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:03.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.384+0000 7f301357c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af11a17b40 0x55af11a2d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:03.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.384+0000 7f301357c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af11a21370 0x55af11a17300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:03.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.384+0000 7f301357c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f300c1ada90 con 0x55af11a21370 2024-09-11T22:05:03.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.384+0000 7f301357c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f300c1af8d0 con 0x55af11a17b40 2024-09-11T22:05:03.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.384+0000 7f301357c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f300c1ae220 con 0x55af11a165d0 2024-09-11T22:05:03.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.384+0000 7f300b59e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af11a21370 0x55af11a17300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:03.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.384+0000 7f300b59e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af11a21370 0x55af11a17300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59662/0 (socket says 172.21.15.1:59662) 2024-09-11T22:05:03.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.384+0000 7f300b59e640 1 -- 172.21.15.1:0/45537875 learned_addr learned my addr 172.21.15.1:0/45537875 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:03.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.384+0000 7f300ad9d640 1 --2- 172.21.15.1:0/45537875 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55af11a165d0 0x55af11a12e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:03.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.384+0000 7f300a59c640 1 --2- 172.21.15.1:0/45537875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af11a17b40 0x55af11a2d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:03.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.384+0000 7f300b59e640 1 -- 172.21.15.1:0/45537875 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55af11a165d0 msgr2=0x55af11a12e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:03.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.384+0000 7f300b59e640 1 --2- 172.21.15.1:0/45537875 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55af11a165d0 0x55af11a12e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:03.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.384+0000 7f300b59e640 1 -- 172.21.15.1:0/45537875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af11a17b40 msgr2=0x55af11a2d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:03.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.384+0000 7f300b59e640 1 --2- 172.21.15.1:0/45537875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af11a17b40 0x55af11a2d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:03.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.384+0000 7f300b59e640 1 -- 172.21.15.1:0/45537875 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af11a19b70 con 0x55af11a21370 2024-09-11T22:05:03.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.384+0000 7f300a59c640 1 --2- 172.21.15.1:0/45537875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af11a17b40 0x55af11a2d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:05:03.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.384+0000 7f300b59e640 1 --2- 172.21.15.1:0/45537875 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af11a21370 0x55af11a17300 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f3004009cd0 tx=0x7f3004044000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91e185ad929acdd5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:03.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.384+0000 7f2ff0ff9640 1 -- 172.21.15.1:0/45537875 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3004056070 con 0x55af11a21370 2024-09-11T22:05:03.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.385+0000 7f2ff0ff9640 1 -- 172.21.15.1:0/45537875 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3004036040 con 0x55af11a21370 2024-09-11T22:05:03.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.385+0000 7f2ff0ff9640 1 -- 172.21.15.1:0/45537875 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3004045410 con 0x55af11a21370 2024-09-11T22:05:03.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.385+0000 7f301357c740 1 -- 172.21.15.1:0/45537875 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af11a21370 msgr2=0x55af11a17300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:03.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.385+0000 7f301357c740 1 --2- 172.21.15.1:0/45537875 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af11a21370 0x55af11a17300 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f3004009cd0 tx=0x7f3004044000 comp rx=0 tx=0).stop 2024-09-11T22:05:03.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.385+0000 7f301357c740 1 -- 172.21.15.1:0/45537875 shutdown_connections 2024-09-11T22:05:03.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.385+0000 7f301357c740 1 --2- 172.21.15.1:0/45537875 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af11a21370 0x55af11a17300 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:03.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.385+0000 7f301357c740 1 --2- 172.21.15.1:0/45537875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af11a17b40 0x55af11a2d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:03.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.385+0000 7f301357c740 1 --2- 172.21.15.1:0/45537875 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55af11a165d0 0x55af11a12e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:03.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.385+0000 7f301357c740 1 -- 172.21.15.1:0/45537875 >> 172.21.15.1:0/45537875 conn(0x55af119df310 msgr2=0x55af11a12700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:03.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.385+0000 7f301357c740 1 -- 172.21.15.1:0/45537875 shutdown_connections 2024-09-11T22:05:03.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.385+0000 7f301357c740 1 -- 172.21.15.1:0/45537875 wait complete. 2024-09-11T22:05:03.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.385+0000 7f301357c740 1 -- start start 2024-09-11T22:05:03.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.386+0000 7f301357c740 1 Processor -- start 2024-09-11T22:05:03.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.386+0000 7f301357c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af11a165d0 0x55af11a298f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:03.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.386+0000 7f301357c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55af11a17b40 0x55af11a1aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:03.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.386+0000 7f301357c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af11a21370 0x55af11a1af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:03.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.386+0000 7f301357c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f300c1af140 con 0x55af11a165d0 2024-09-11T22:05:03.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.386+0000 7f301357c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f300c1ae9b0 con 0x55af11a21370 2024-09-11T22:05:03.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.386+0000 7f301357c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f300c1aba90 con 0x55af11a17b40 2024-09-11T22:05:03.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.386+0000 7f301357c740 10 client.0 _ll_register_callbacks cb 0x7f3000007ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:05:03.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.386+0000 7f300ad9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af11a165d0 0x55af11a298f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:03.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.386+0000 7f300ad9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af11a165d0 0x55af11a298f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59664/0 (socket says 172.21.15.1:59664) 2024-09-11T22:05:03.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.386+0000 7f300ad9d640 1 -- 172.21.15.1:0/3293592055 learned_addr learned my addr 172.21.15.1:0/3293592055 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:03.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.386+0000 7f300b59e640 1 --2- 172.21.15.1:0/3293592055 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af11a21370 0x55af11a1af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:03.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.386+0000 7f300a59c640 1 --2- 172.21.15.1:0/3293592055 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55af11a17b40 0x55af11a1aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:03.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.386+0000 7f300ad9d640 1 -- 172.21.15.1:0/3293592055 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55af11a17b40 msgr2=0x55af11a1aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:03.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.386+0000 7f300ad9d640 1 --2- 172.21.15.1:0/3293592055 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55af11a17b40 0x55af11a1aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:03.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.386+0000 7f300ad9d640 1 -- 172.21.15.1:0/3293592055 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af11a21370 msgr2=0x55af11a1af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:03.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.386+0000 7f300ad9d640 1 --2- 172.21.15.1:0/3293592055 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af11a21370 0x55af11a1af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:03.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.386+0000 7f300ad9d640 1 -- 172.21.15.1:0/3293592055 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af11a1b4b0 con 0x55af11a165d0 2024-09-11T22:05:03.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.386+0000 7f300a59c640 1 --2- 172.21.15.1:0/3293592055 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55af11a17b40 0x55af11a1aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:05:03.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.387+0000 7f300ad9d640 1 --2- 172.21.15.1:0/3293592055 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af11a165d0 0x55af11a298f0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc009790 tx=0x7f2ffc013da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1180aa15e672e90 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:03.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.387+0000 7f2feeff5640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:05:03.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.387+0000 7f2feeff5640 1 -- 172.21.15.1:0/3293592055 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc017040 con 0x55af11a165d0 2024-09-11T22:05:03.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.387+0000 7f2feeff5640 1 -- 172.21.15.1:0/3293592055 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7f2ffc013670 con 0x55af11a165d0 2024-09-11T22:05:03.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.387+0000 7f301357c740 1 -- 172.21.15.1:0/3293592055 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3004036040 con 0x55af11a165d0 2024-09-11T22:05:03.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.387+0000 7f2feeff5640 1 -- 172.21.15.1:0/3293592055 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f2ffc013870 con 0x55af11a165d0 2024-09-11T22:05:03.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.387+0000 7f301357c740 10 client.17196 fetch_fsmap learned FSMap version 42 2024-09-11T22:05:03.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.387+0000 7f301357c740 1 -- 172.21.15.1:0/3293592055 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55af11a9b900 con 0x55af11a165d0 2024-09-11T22:05:03.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.387+0000 7f2feeff5640 1 -- 172.21.15.1:0/3293592055 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3004036040 con 0x55af11a165d0 2024-09-11T22:05:03.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.387+0000 7f2feeff5640 1 -- 172.21.15.1:0/3293592055 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f2ffc01c450 con 0x55af11a165d0 2024-09-11T22:05:03.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.387+0000 7f301357c740 10 client.17196 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:05:03.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.387+0000 7f301357c740 10 client.17196 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T22:05:03.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.387+0000 7f301357c740 20 client.17196 populate_metadata read hostname 'smithi001' 2024-09-11T22:05:03.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.387+0000 7f301357c740 1 --2- 172.21.15.1:0/3293592055 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x55af11a9bd40 0x55af11a9e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:03.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.387+0000 7f301357c740 4 client.17196 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:05:03.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.387+0000 7f301357c740 1 -- 172.21.15.1:0/3293592055 --> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55af11983d70 con 0x55af11a9bd40 2024-09-11T22:05:03.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.388+0000 7f300a59c640 1 --2- 172.21.15.1:0/3293592055 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x55af11a9bd40 0x55af11a9e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:03.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.388+0000 7f300a59c640 1 --2- 172.21.15.1:0/3293592055 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x55af11a9bd40 0x55af11a9e160 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=22631e8ea94a6743 server_cookie=3e65b8019690f9e2 in_seq=0 out_seq=0 2024-09-11T22:05:03.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.388+0000 7f2feeff5640 10 client.17196 ms_handle_connect on v2:172.21.15.161:6838/455365383 2024-09-11T22:05:03.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.388+0000 7f2feeff5640 1 -- 172.21.15.1:0/3293592055 <== mds.1 v2:172.21.15.161:6838/455365383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55af11983d70 con 0x55af11a9bd40 2024-09-11T22:05:03.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.388+0000 7f2feeff5640 10 client.17196 handle_command_reply: tid=0 2024-09-11T22:05:03.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.388+0000 7f301357c740 1 client.17196 shutdown 2024-09-11T22:05:03.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.388+0000 7f301357c740 1 -- 172.21.15.1:0/3293592055 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af11a165d0 msgr2=0x55af11a298f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:03.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.388+0000 7f301357c740 1 --2- 172.21.15.1:0/3293592055 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af11a165d0 0x55af11a298f0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc009790 tx=0x7f2ffc013da0 comp rx=0 tx=0).stop 2024-09-11T22:05:03.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.388+0000 7f301357c740 1 -- 172.21.15.1:0/3293592055 shutdown_connections 2024-09-11T22:05:03.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.388+0000 7f301357c740 1 --2- 172.21.15.1:0/3293592055 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x55af11a9bd40 0x55af11a9e160 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:03.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.388+0000 7f301357c740 1 --2- 172.21.15.1:0/3293592055 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af11a21370 0x55af11a1af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:03.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.388+0000 7f301357c740 1 --2- 172.21.15.1:0/3293592055 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55af11a17b40 0x55af11a1aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:03.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.388+0000 7f2feeff5640 0 client.17196 ms_handle_reset on v2:172.21.15.161:6838/455365383 2024-09-11T22:05:03.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.388+0000 7f301357c740 1 --2- 172.21.15.1:0/3293592055 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af11a165d0 0x55af11a298f0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:03.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.388+0000 7f301357c740 1 -- 172.21.15.1:0/3293592055 >> 172.21.15.1:0/3293592055 conn(0x55af119df310 msgr2=0x55af11a12560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:03.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.389+0000 7f301357c740 1 -- 172.21.15.1:0/3293592055 shutdown_connections 2024-09-11T22:05:03.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.389+0000 7f301357c740 1 -- 172.21.15.1:0/3293592055 wait complete. 2024-09-11T22:05:03.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.389+0000 7f301357c740 20 client.17196 trim_cache size 0 max 16384 2024-09-11T22:05:03.411 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-11T22:05:03.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.389+0000 7f30111a3640 1 -- 172.21.15.1:0/783885299 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f2ff40a2cf0 msgr2=0x7f2ff40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:03.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.389+0000 7f30111a3640 1 --2- 172.21.15.1:0/783885299 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f2ff40a2cf0 0x7f2ff40a51b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f30040098d0 tx=0x7f30040095c0 comp rx=0 tx=0).stop 2024-09-11T22:05:03.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.389+0000 7f30111a3640 1 -- 172.21.15.1:0/783885299 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f300c103740 msgr2=0x7f300c198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:03.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.389+0000 7f30111a3640 1 --2- 172.21.15.1:0/783885299 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f300c103740 0x7f300c198a50 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f300000c970 tx=0x7f300000cd70 comp rx=0 tx=0).stop 2024-09-11T22:05:03.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.389+0000 7f30111a3640 1 -- 172.21.15.1:0/783885299 shutdown_connections 2024-09-11T22:05:03.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.389+0000 7f30111a3640 1 --2- 172.21.15.1:0/783885299 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f2ff40a2cf0 0x7f2ff40a51b0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:03.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.389+0000 7f30111a3640 1 --2- 172.21.15.1:0/783885299 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f300c108290 0x7f300c19cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:03.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.389+0000 7f30111a3640 1 --2- 172.21.15.1:0/783885299 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f300c103740 0x7f300c198a50 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:03.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.389+0000 7f30111a3640 1 --2- 172.21.15.1:0/783885299 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f300c102d90 0x7f300c198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:03.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.389+0000 7f30111a3640 1 -- 172.21.15.1:0/783885299 >> 172.21.15.1:0/783885299 conn(0x7f300c0fc7d0 msgr2=0x7f300c0fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:03.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.389+0000 7f30111a3640 1 -- 172.21.15.1:0/783885299 shutdown_connections 2024-09-11T22:05:03.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:03.389+0000 7f30111a3640 1 -- 172.21.15.1:0/783885299 wait complete. 2024-09-11T22:05:04.247 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T22:05:04.509 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:05:05.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.970+0000 7f2c1da73640 1 -- 172.21.15.1:0/1729058103 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c18102d20 msgr2=0x7f2c18103100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:05.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.970+0000 7f2c1da73640 1 --2- 172.21.15.1:0/1729058103 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c18102d20 0x7f2c18103100 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f2c08009900 tx=0x7f2c0802a480 comp rx=0 tx=0).stop 2024-09-11T22:05:05.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.970+0000 7f2c1da73640 1 -- 172.21.15.1:0/1729058103 shutdown_connections 2024-09-11T22:05:05.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.970+0000 7f2c1da73640 1 --2- 172.21.15.1:0/1729058103 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c1810c150 0x7f2c1810e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:05.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.970+0000 7f2c1da73640 1 --2- 172.21.15.1:0/1729058103 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c18103640 0x7f2c1810bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:05.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.970+0000 7f2c1da73640 1 --2- 172.21.15.1:0/1729058103 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c18102d20 0x7f2c18103100 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:05.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.970+0000 7f2c1da73640 1 -- 172.21.15.1:0/1729058103 >> 172.21.15.1:0/1729058103 conn(0x7f2c180fc780 msgr2=0x7f2c180feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:05.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.970+0000 7f2c1da73640 1 -- 172.21.15.1:0/1729058103 shutdown_connections 2024-09-11T22:05:05.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.970+0000 7f2c1da73640 1 -- 172.21.15.1:0/1729058103 wait complete. 2024-09-11T22:05:05.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.971+0000 7f2c1da73640 1 Processor -- start 2024-09-11T22:05:05.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.971+0000 7f2c1da73640 1 -- start start 2024-09-11T22:05:05.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.971+0000 7f2c1da73640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c18102d20 0x7f2c18198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:05.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.971+0000 7f2c1da73640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c18103640 0x7f2c18198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:05.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.971+0000 7f2c1da73640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c1810c150 0x7f2c1819cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:05.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.971+0000 7f2c1da73640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2c18111f90 con 0x7f2c18102d20 2024-09-11T22:05:05.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.971+0000 7f2c1da73640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2c18111e10 con 0x7f2c18103640 2024-09-11T22:05:05.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.971+0000 7f2c1da73640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2c18112110 con 0x7f2c1810c150 2024-09-11T22:05:05.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.971+0000 7f2c177fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c18102d20 0x7f2c18198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:05.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.971+0000 7f2c177fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c18102d20 0x7f2c18198520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59688/0 (socket says 172.21.15.1:59688) 2024-09-11T22:05:05.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.971+0000 7f2c177fe640 1 -- 172.21.15.1:0/3697989818 learned_addr learned my addr 172.21.15.1:0/3697989818 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:05.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.971+0000 7f2c177fe640 1 -- 172.21.15.1:0/3697989818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c1810c150 msgr2=0x7f2c1819cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:05.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.971+0000 7f2c0effd640 1 --2- 172.21.15.1:0/3697989818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c18103640 0x7f2c18198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:05.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.971+0000 7f2c17fff640 1 --2- 172.21.15.1:0/3697989818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c1810c150 0x7f2c1819cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:05.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.972+0000 7f2c177fe640 1 --2- 172.21.15.1:0/3697989818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c1810c150 0x7f2c1819cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:05.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.972+0000 7f2c177fe640 1 -- 172.21.15.1:0/3697989818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c18103640 msgr2=0x7f2c18198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:05.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.972+0000 7f2c177fe640 1 --2- 172.21.15.1:0/3697989818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c18103640 0x7f2c18198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:05.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.972+0000 7f2c177fe640 1 -- 172.21.15.1:0/3697989818 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c1819d4d0 con 0x7f2c18102d20 2024-09-11T22:05:05.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.972+0000 7f2c17fff640 1 --2- 172.21.15.1:0/3697989818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c1810c150 0x7f2c1819cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:05:05.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.972+0000 7f2c177fe640 1 --2- 172.21.15.1:0/3697989818 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c18102d20 0x7f2c18198520 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f2c08006010 tx=0x7f2c080042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:05.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.972+0000 7f2c157fa640 1 -- 172.21.15.1:0/3697989818 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c0803f030 con 0x7f2c18102d20 2024-09-11T22:05:05.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.972+0000 7f2c157fa640 1 -- 172.21.15.1:0/3697989818 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c08003ce0 con 0x7f2c18102d20 2024-09-11T22:05:05.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.972+0000 7f2c1da73640 1 -- 172.21.15.1:0/3697989818 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c1819d760 con 0x7f2c18102d20 2024-09-11T22:05:05.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.972+0000 7f2c157fa640 1 -- 172.21.15.1:0/3697989818 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c08035070 con 0x7f2c18102d20 2024-09-11T22:05:05.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.972+0000 7f2c1da73640 1 -- 172.21.15.1:0/3697989818 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c181a50a0 con 0x7f2c18102d20 2024-09-11T22:05:05.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.974+0000 7f2c157fa640 1 -- 172.21.15.1:0/3697989818 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f2c08042030 con 0x7f2c18102d20 2024-09-11T22:05:05.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.974+0000 7f2c1fe4c740 1 Processor -- start 2024-09-11T22:05:05.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.975+0000 7f2c1fe4c740 1 -- start start 2024-09-11T22:05:05.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.975+0000 7f2c1fe4c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4debc3b40 0x55f4debc2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:05.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.975+0000 7f2c1fe4c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4debc25d0 0x55f4debc2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:05.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.975+0000 7f2c1fe4c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4debd1780 0x55f4debd3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:05.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.975+0000 7f2c1fe4c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55f4deb282f0 con 0x55f4debd1780 2024-09-11T22:05:05.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.975+0000 7f2c1fe4c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55f4deacfcf0 con 0x55f4debc3b40 2024-09-11T22:05:05.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.975+0000 7f2c1fe4c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f4deb242b0 con 0x55f4debc25d0 2024-09-11T22:05:05.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.975+0000 7f2c0effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4debd1780 0x55f4debd3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:05.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.975+0000 7f2c0effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4debd1780 0x55f4debd3b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59702/0 (socket says 172.21.15.1:59702) 2024-09-11T22:05:05.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.975+0000 7f2c0effd640 1 -- 172.21.15.1:0/736126274 learned_addr learned my addr 172.21.15.1:0/736126274 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:05.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.975+0000 7f2c17fff640 1 --2- 172.21.15.1:0/736126274 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4debc3b40 0x55f4debc2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:05.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.975+0000 7f2c157fa640 1 --2- 172.21.15.1:0/3697989818 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f2c040a2cf0 0x7f2c040a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:05.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.975+0000 7f2c0effd640 1 -- 172.21.15.1:0/736126274 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4debc25d0 msgr2=0x55f4debc2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:05.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.975+0000 7f2c0effd640 1 --2- 172.21.15.1:0/736126274 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4debc25d0 0x55f4debc2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:05.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.975+0000 7f2c0effd640 1 -- 172.21.15.1:0/736126274 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4debc3b40 msgr2=0x55f4debc2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:05.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.975+0000 7f2c0effd640 1 --2- 172.21.15.1:0/736126274 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4debc3b40 0x55f4debc2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:05.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.975+0000 7f2c0effd640 1 -- 172.21.15.1:0/736126274 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4deab43a0 con 0x55f4debd1780 2024-09-11T22:05:05.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.975+0000 7f2c17fff640 1 --2- 172.21.15.1:0/3697989818 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f2c040a2cf0 0x7f2c040a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:05.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.975+0000 7f2c157fa640 1 -- 172.21.15.1:0/3697989818 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7f2c080efc30 con 0x7f2c18102d20 2024-09-11T22:05:05.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.976+0000 7f2c0effd640 1 --2- 172.21.15.1:0/736126274 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4debd1780 0x55f4debd3b70 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f2c0000a8d0 tx=0x7f2c0000ad90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b77a7254b1eef94 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:05.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.976+0000 7f2c0dffb640 1 -- 172.21.15.1:0/736126274 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c00015070 con 0x55f4debd1780 2024-09-11T22:05:05.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.976+0000 7f2c0dffb640 1 -- 172.21.15.1:0/736126274 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c0000cb40 con 0x55f4debd1780 2024-09-11T22:05:05.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.976+0000 7f2c0dffb640 1 -- 172.21.15.1:0/736126274 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c000185c0 con 0x55f4debd1780 2024-09-11T22:05:05.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.976+0000 7f2c1fe4c740 1 -- 172.21.15.1:0/736126274 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4debd1780 msgr2=0x55f4debd3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:05.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.976+0000 7f2c17fff640 1 --2- 172.21.15.1:0/3697989818 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f2c040a2cf0 0x7f2c040a51b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2c10005fd0 tx=0x7f2c10004a40 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:05.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.976+0000 7f2c1fe4c740 1 --2- 172.21.15.1:0/736126274 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4debd1780 0x55f4debd3b70 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f2c0000a8d0 tx=0x7f2c0000ad90 comp rx=0 tx=0).stop 2024-09-11T22:05:05.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.976+0000 7f2c1fe4c740 1 -- 172.21.15.1:0/736126274 shutdown_connections 2024-09-11T22:05:05.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.976+0000 7f2c1fe4c740 1 --2- 172.21.15.1:0/736126274 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4debd1780 0x55f4debd3b70 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:05.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.976+0000 7f2c1fe4c740 1 --2- 172.21.15.1:0/736126274 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4debc25d0 0x55f4debc2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:05.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.976+0000 7f2c1fe4c740 1 --2- 172.21.15.1:0/736126274 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4debc3b40 0x55f4debc2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:05.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.976+0000 7f2c1fe4c740 1 -- 172.21.15.1:0/736126274 >> 172.21.15.1:0/736126274 conn(0x55f4deb8b310 msgr2=0x55f4debbec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:05.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.977+0000 7f2c1fe4c740 1 -- 172.21.15.1:0/736126274 shutdown_connections 2024-09-11T22:05:05.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.977+0000 7f2c1fe4c740 1 -- 172.21.15.1:0/736126274 wait complete. 2024-09-11T22:05:05.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.977+0000 7f2c1fe4c740 1 -- start start 2024-09-11T22:05:05.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.977+0000 7f2c1fe4c740 1 Processor -- start 2024-09-11T22:05:05.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.977+0000 7f2c1fe4c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4debc25d0 0x55f4debd11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:05.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.977+0000 7f2c1fe4c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4debc3b40 0x55f4debce450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:05.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.977+0000 7f2c1fe4c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4debd1780 0x55f4debcea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:05.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.977+0000 7f2c1fe4c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55f4debd4480 con 0x55f4debd1780 2024-09-11T22:05:05.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.977+0000 7f2c1fe4c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55f4deae3060 con 0x55f4debc25d0 2024-09-11T22:05:05.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.977+0000 7f2c1fe4c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f4debd4600 con 0x55f4debc3b40 2024-09-11T22:05:05.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.977+0000 7f2c1fe4c740 10 client.0 _ll_register_callbacks cb 0x55f4de8368c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:05:05.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.977+0000 7f2c17fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4debd1780 0x55f4debcea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:05.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.978+0000 7f2c17fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4debd1780 0x55f4debcea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59710/0 (socket says 172.21.15.1:59710) 2024-09-11T22:05:05.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.978+0000 7f2c17fff640 1 -- 172.21.15.1:0/1902554753 learned_addr learned my addr 172.21.15.1:0/1902554753 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:05.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.978+0000 7f2c0effd640 1 --2- 172.21.15.1:0/1902554753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4debc3b40 0x55f4debce450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:05.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.978+0000 7f2c177fe640 1 --2- 172.21.15.1:0/1902554753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4debc25d0 0x55f4debd11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:05.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.978+0000 7f2c17fff640 1 -- 172.21.15.1:0/1902554753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4debc3b40 msgr2=0x55f4debce450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:05.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.978+0000 7f2c17fff640 1 --2- 172.21.15.1:0/1902554753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4debc3b40 0x55f4debce450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:05.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.978+0000 7f2c17fff640 1 -- 172.21.15.1:0/1902554753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4debc25d0 msgr2=0x55f4debd11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:05.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.978+0000 7f2c17fff640 1 --2- 172.21.15.1:0/1902554753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4debc25d0 0x55f4debd11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:05.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.978+0000 7f2c17fff640 1 -- 172.21.15.1:0/1902554753 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4de8993c0 con 0x55f4debd1780 2024-09-11T22:05:05.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.978+0000 7f2c0effd640 1 --2- 172.21.15.1:0/1902554753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4debc3b40 0x55f4debce450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:05:05.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.978+0000 7f2c17fff640 1 --2- 172.21.15.1:0/1902554753 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4debd1780 0x55f4debcea60 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f2c1001d9a0 tx=0x7f2c10004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d160a90dfc766f4e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:05.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.978+0000 7f2bf6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:05:05.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.978+0000 7f2bf6ffd640 1 -- 172.21.15.1:0/1902554753 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c10013980 con 0x55f4debd1780 2024-09-11T22:05:05.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.978+0000 7f2bf6ffd640 1 -- 172.21.15.1:0/1902554753 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7f2c10013b20 con 0x55f4debd1780 2024-09-11T22:05:05.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.978+0000 7f2c1fe4c740 1 -- 172.21.15.1:0/1902554753 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c0000cb40 con 0x55f4debd1780 2024-09-11T22:05:05.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.978+0000 7f2bf6ffd640 1 -- 172.21.15.1:0/1902554753 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f2c10013d60 con 0x55f4debd1780 2024-09-11T22:05:05.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.978+0000 7f2c1fe4c740 10 client.17208 fetch_fsmap learned FSMap version 42 2024-09-11T22:05:05.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.978+0000 7f2c1fe4c740 1 -- 172.21.15.1:0/1902554753 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55f4debdffb0 con 0x55f4debd1780 2024-09-11T22:05:05.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.979+0000 7f2bf6ffd640 1 -- 172.21.15.1:0/1902554753 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c0000cb40 con 0x55f4debd1780 2024-09-11T22:05:05.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.979+0000 7f2bf6ffd640 1 -- 172.21.15.1:0/1902554753 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f2c10024070 con 0x55f4debd1780 2024-09-11T22:05:05.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.979+0000 7f2c1fe4c740 10 client.17208 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:05:05.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.979+0000 7f2c1fe4c740 10 client.17208 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:05:05.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.979+0000 7f2c1fe4c740 20 client.17208 populate_metadata read hostname 'smithi001' 2024-09-11T22:05:05.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.979+0000 7f2c1fe4c740 1 --2- 172.21.15.1:0/1902554753 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55f4debe1510 0x55f4debe3930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:05.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.979+0000 7f2c1fe4c740 4 client.17208 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:05:05.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.979+0000 7f2c1fe4c740 1 -- 172.21.15.1:0/1902554753 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4deb35860 con 0x55f4debe1510 2024-09-11T22:05:05.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.979+0000 7f2c177fe640 1 --2- 172.21.15.1:0/1902554753 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55f4debe1510 0x55f4debe3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:05.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.980+0000 7f2c177fe640 1 --2- 172.21.15.1:0/1902554753 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55f4debe1510 0x55f4debe3930 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e688a78a9160df3 server_cookie=a3d175160978cf96 in_seq=0 out_seq=0 2024-09-11T22:05:05.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.980+0000 7f2bf6ffd640 10 client.17208 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:05:05.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.980+0000 7f2bf6ffd640 1 -- 172.21.15.1:0/1902554753 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f4deb35860 con 0x55f4debe1510 2024-09-11T22:05:05.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.980+0000 7f2bf6ffd640 10 client.17208 handle_command_reply: tid=0 2024-09-11T22:05:05.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.980+0000 7f2c1fe4c740 1 client.17208 shutdown 2024-09-11T22:05:05.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.981+0000 7f2c1fe4c740 1 -- 172.21.15.1:0/1902554753 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4debd1780 msgr2=0x55f4debcea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:05.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.981+0000 7f2c1fe4c740 1 --2- 172.21.15.1:0/1902554753 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4debd1780 0x55f4debcea60 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f2c1001d9a0 tx=0x7f2c10004ab0 comp rx=0 tx=0).stop 2024-09-11T22:05:05.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.981+0000 7f2c1fe4c740 1 -- 172.21.15.1:0/1902554753 shutdown_connections 2024-09-11T22:05:05.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.981+0000 7f2c1fe4c740 1 --2- 172.21.15.1:0/1902554753 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55f4debe1510 0x55f4debe3930 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:05.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.981+0000 7f2c1fe4c740 1 --2- 172.21.15.1:0/1902554753 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4debd1780 0x55f4debcea60 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:05.016 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-11T22:05:05.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.981+0000 7f2c1fe4c740 1 --2- 172.21.15.1:0/1902554753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4debc3b40 0x55f4debce450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:05.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.981+0000 7f2c1fe4c740 1 --2- 172.21.15.1:0/1902554753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4debc25d0 0x55f4debd11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:05.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.981+0000 7f2c1fe4c740 1 -- 172.21.15.1:0/1902554753 >> 172.21.15.1:0/1902554753 conn(0x55f4deb8b310 msgr2=0x55f4debc4960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:05.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.981+0000 7f2bf6ffd640 0 client.17208 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:05:05.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.981+0000 7f2c1fe4c740 1 -- 172.21.15.1:0/1902554753 shutdown_connections 2024-09-11T22:05:05.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.981+0000 7f2c1fe4c740 1 -- 172.21.15.1:0/1902554753 wait complete. 2024-09-11T22:05:05.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.981+0000 7f2c1fe4c740 20 client.17208 trim_cache size 0 max 16384 2024-09-11T22:05:05.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.994+0000 7f2c1fe4c740 1 Processor -- start 2024-09-11T22:05:05.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.994+0000 7f2c1fe4c740 1 -- start start 2024-09-11T22:05:05.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.994+0000 7f2c1fe4c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4debc25d0 0x55f4debbecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:05.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.994+0000 7f2c1fe4c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4debc3b40 0x55f4debdd720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:05.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.994+0000 7f2c1fe4c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4debd1780 0x55f4debe9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:05.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.994+0000 7f2c1fe4c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2c181ab8e0 con 0x55f4debc25d0 2024-09-11T22:05:05.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.994+0000 7f2c1fe4c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2c181ad720 con 0x55f4debd1780 2024-09-11T22:05:05.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.994+0000 7f2c1fe4c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2c181ac070 con 0x55f4debc3b40 2024-09-11T22:05:05.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.994+0000 7f2c177fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4debc25d0 0x55f4debbecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:05.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.994+0000 7f2c177fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4debc25d0 0x55f4debbecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59720/0 (socket says 172.21.15.1:59720) 2024-09-11T22:05:05.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.994+0000 7f2c177fe640 1 -- 172.21.15.1:0/3865850728 learned_addr learned my addr 172.21.15.1:0/3865850728 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:05.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.994+0000 7f2c0effd640 1 --2- 172.21.15.1:0/3865850728 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4debc3b40 0x55f4debdd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:05.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.995+0000 7f2c17fff640 1 --2- 172.21.15.1:0/3865850728 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4debd1780 0x55f4debe9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:05.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.995+0000 7f2c177fe640 1 -- 172.21.15.1:0/3865850728 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4debc3b40 msgr2=0x55f4debdd720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:05.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.995+0000 7f2c177fe640 1 --2- 172.21.15.1:0/3865850728 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4debc3b40 0x55f4debdd720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:05.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.995+0000 7f2c177fe640 1 -- 172.21.15.1:0/3865850728 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4debd1780 msgr2=0x55f4debe9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:05.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.995+0000 7f2c177fe640 1 --2- 172.21.15.1:0/3865850728 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4debd1780 0x55f4debe9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:05.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.995+0000 7f2c177fe640 1 -- 172.21.15.1:0/3865850728 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4debc5120 con 0x55f4debc25d0 2024-09-11T22:05:05.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.995+0000 7f2c17fff640 1 --2- 172.21.15.1:0/3865850728 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4debd1780 0x55f4debe9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:05:05.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.995+0000 7f2c177fe640 1 --2- 172.21.15.1:0/3865850728 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4debc25d0 0x55f4debbecd0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f2c080b8600 tx=0x7f2c08003e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35b04778b9884e59 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:05.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.995+0000 7f2c0d7fa640 1 -- 172.21.15.1:0/3865850728 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c08049070 con 0x55f4debc25d0 2024-09-11T22:05:05.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.995+0000 7f2c0d7fa640 1 -- 172.21.15.1:0/3865850728 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c0803b580 con 0x55f4debc25d0 2024-09-11T22:05:05.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.995+0000 7f2c0d7fa640 1 -- 172.21.15.1:0/3865850728 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c080c4e10 con 0x55f4debc25d0 2024-09-11T22:05:05.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.996+0000 7f2c1fe4c740 1 -- 172.21.15.1:0/3865850728 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4debc25d0 msgr2=0x55f4debbecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:05.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.996+0000 7f2c1fe4c740 1 --2- 172.21.15.1:0/3865850728 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4debc25d0 0x55f4debbecd0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f2c080b8600 tx=0x7f2c08003e90 comp rx=0 tx=0).stop 2024-09-11T22:05:05.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.996+0000 7f2c1fe4c740 1 -- 172.21.15.1:0/3865850728 shutdown_connections 2024-09-11T22:05:05.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.996+0000 7f2c1fe4c740 1 --2- 172.21.15.1:0/3865850728 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4debd1780 0x55f4debe9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:05.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.996+0000 7f2c1fe4c740 1 --2- 172.21.15.1:0/3865850728 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4debc3b40 0x55f4debdd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:05.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.996+0000 7f2c1fe4c740 1 --2- 172.21.15.1:0/3865850728 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4debc25d0 0x55f4debbecd0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:05.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.996+0000 7f2c1fe4c740 1 -- 172.21.15.1:0/3865850728 >> 172.21.15.1:0/3865850728 conn(0x55f4deb8b310 msgr2=0x55f4debbe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:05.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.996+0000 7f2c1fe4c740 1 -- 172.21.15.1:0/3865850728 shutdown_connections 2024-09-11T22:05:05.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.996+0000 7f2c1fe4c740 1 -- 172.21.15.1:0/3865850728 wait complete. 2024-09-11T22:05:05.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.996+0000 7f2c1fe4c740 1 -- start start 2024-09-11T22:05:05.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.996+0000 7f2c1fe4c740 1 Processor -- start 2024-09-11T22:05:05.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.996+0000 7f2c1fe4c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4debc25d0 0x55f4debd4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:05.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.996+0000 7f2c1fe4c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4debc3b40 0x55f4debd5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:05.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.996+0000 7f2c1fe4c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4debd1780 0x55f4debd9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:05.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.997+0000 7f2c1fe4c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2c181acf90 con 0x55f4debd1780 2024-09-11T22:05:05.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.997+0000 7f2c1fe4c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2c181ac800 con 0x55f4debc25d0 2024-09-11T22:05:05.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.997+0000 7f2c1fe4c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2c181a98e0 con 0x55f4debc3b40 2024-09-11T22:05:05.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.997+0000 7f2c1fe4c740 10 client.0 _ll_register_callbacks cb 0x7f2c10009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:05:05.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.997+0000 7f2c17fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4debd1780 0x55f4debd9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:05.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.997+0000 7f2c17fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4debd1780 0x55f4debd9580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59732/0 (socket says 172.21.15.1:59732) 2024-09-11T22:05:05.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.997+0000 7f2c17fff640 1 -- 172.21.15.1:0/809887207 learned_addr learned my addr 172.21.15.1:0/809887207 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:05.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.997+0000 7f2c177fe640 1 --2- 172.21.15.1:0/809887207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4debc25d0 0x55f4debd4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:05.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.997+0000 7f2c0effd640 1 --2- 172.21.15.1:0/809887207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4debc3b40 0x55f4debd5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:05.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.997+0000 7f2c17fff640 1 -- 172.21.15.1:0/809887207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4debc3b40 msgr2=0x55f4debd5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:05.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.997+0000 7f2c17fff640 1 --2- 172.21.15.1:0/809887207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4debc3b40 0x55f4debd5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:05.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.997+0000 7f2c17fff640 1 -- 172.21.15.1:0/809887207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4debc25d0 msgr2=0x55f4debd4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:05.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.997+0000 7f2c17fff640 1 --2- 172.21.15.1:0/809887207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4debc25d0 0x55f4debd4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:05.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.997+0000 7f2c17fff640 1 -- 172.21.15.1:0/809887207 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4debd9c20 con 0x55f4debd1780 2024-09-11T22:05:05.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.997+0000 7f2c0effd640 1 --2- 172.21.15.1:0/809887207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4debc3b40 0x55f4debd5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:05:05.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.997+0000 7f2c17fff640 1 --2- 172.21.15.1:0/809887207 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4debd1780 0x55f4debd9580 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f2c10009630 tx=0x7f2c1001d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bc8a03b31fd21da server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:05.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.997+0000 7f2bf6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:05:05.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.997+0000 7f2bf6ffd640 1 -- 172.21.15.1:0/809887207 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c100139d0 con 0x55f4debd1780 2024-09-11T22:05:05.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.997+0000 7f2c1fe4c740 1 -- 172.21.15.1:0/809887207 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f4deb2fd70 con 0x55f4debd1780 2024-09-11T22:05:05.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.997+0000 7f2bf6ffd640 1 -- 172.21.15.1:0/809887207 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7f2c10026440 con 0x55f4debd1780 2024-09-11T22:05:05.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.997+0000 7f2bf6ffd640 1 -- 172.21.15.1:0/809887207 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f2c10013b70 con 0x55f4debd1780 2024-09-11T22:05:05.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.997+0000 7f2c1fe4c740 10 client.17214 fetch_fsmap learned FSMap version 42 2024-09-11T22:05:05.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.997+0000 7f2c1fe4c740 1 -- 172.21.15.1:0/809887207 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55f4dec4bc40 con 0x55f4debd1780 2024-09-11T22:05:05.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.998+0000 7f2bf6ffd640 1 -- 172.21.15.1:0/809887207 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55f4deb2fd70 con 0x55f4debd1780 2024-09-11T22:05:05.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.998+0000 7f2bf6ffd640 1 -- 172.21.15.1:0/809887207 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f2c10035030 con 0x55f4debd1780 2024-09-11T22:05:05.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.998+0000 7f2c1fe4c740 10 client.17214 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:05:05.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.998+0000 7f2c1fe4c740 10 client.17214 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:05:05.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.998+0000 7f2c1fe4c740 20 client.17214 populate_metadata read hostname 'smithi001' 2024-09-11T22:05:05.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.998+0000 7f2c1fe4c740 1 --2- 172.21.15.1:0/809887207 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55f4dec4c420 0x55f4dec4e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:05.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.998+0000 7f2c1fe4c740 4 client.17214 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:05:05.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.998+0000 7f2c1fe4c740 1 -- 172.21.15.1:0/809887207 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f4deaac170 con 0x55f4dec4c420 2024-09-11T22:05:05.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.998+0000 7f2c177fe640 1 --2- 172.21.15.1:0/809887207 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55f4dec4c420 0x55f4dec4e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:05.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.999+0000 7f2c177fe640 1 --2- 172.21.15.1:0/809887207 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55f4dec4c420 0x55f4dec4e840 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47feb6a0a7163c57 server_cookie=897e58ab65e8400 in_seq=0 out_seq=0 2024-09-11T22:05:05.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.999+0000 7f2bf6ffd640 10 client.17214 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:05:05.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.999+0000 7f2bf6ffd640 1 -- 172.21.15.1:0/809887207 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f4deaac170 con 0x55f4dec4c420 2024-09-11T22:05:05.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.999+0000 7f2bf6ffd640 10 client.17214 handle_command_reply: tid=0 2024-09-11T22:05:05.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.999+0000 7f2c1fe4c740 1 client.17214 shutdown 2024-09-11T22:05:05.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.999+0000 7f2c1fe4c740 1 -- 172.21.15.1:0/809887207 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4debd1780 msgr2=0x55f4debd9580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:05.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.999+0000 7f2c1fe4c740 1 --2- 172.21.15.1:0/809887207 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4debd1780 0x55f4debd9580 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f2c10009630 tx=0x7f2c1001d560 comp rx=0 tx=0).stop 2024-09-11T22:05:05.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.999+0000 7f2c1fe4c740 1 -- 172.21.15.1:0/809887207 shutdown_connections 2024-09-11T22:05:05.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.999+0000 7f2c1fe4c740 1 --2- 172.21.15.1:0/809887207 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55f4dec4c420 0x55f4dec4e840 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:05.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.999+0000 7f2c1fe4c740 1 --2- 172.21.15.1:0/809887207 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4debd1780 0x55f4debd9580 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:05.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.999+0000 7f2c1fe4c740 1 --2- 172.21.15.1:0/809887207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f4debc3b40 0x55f4debd5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:05.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.999+0000 7f2bf6ffd640 0 client.17214 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:05:05.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.999+0000 7f2c1fe4c740 1 --2- 172.21.15.1:0/809887207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4debc25d0 0x55f4debd4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:05.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.999+0000 7f2c1fe4c740 1 -- 172.21.15.1:0/809887207 >> 172.21.15.1:0/809887207 conn(0x55f4deb8b310 msgr2=0x55f4debcd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:05.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.999+0000 7f2c1fe4c740 1 -- 172.21.15.1:0/809887207 shutdown_connections 2024-09-11T22:05:05.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.999+0000 7f2c1fe4c740 1 -- 172.21.15.1:0/809887207 wait complete. 2024-09-11T22:05:05.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:04.999+0000 7f2c1fe4c740 20 client.17214 trim_cache size 0 max 16384 2024-09-11T22:05:05.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:05.000+0000 7f2c1da73640 1 -- 172.21.15.1:0/3697989818 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f2c040a2cf0 msgr2=0x7f2c040a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:05.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:05.000+0000 7f2c1da73640 1 --2- 172.21.15.1:0/3697989818 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f2c040a2cf0 0x7f2c040a51b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2c10005fd0 tx=0x7f2c10004a40 comp rx=0 tx=0).stop 2024-09-11T22:05:05.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:05.000+0000 7f2c1da73640 1 -- 172.21.15.1:0/3697989818 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c18102d20 msgr2=0x7f2c18198520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:05.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:05.000+0000 7f2c1da73640 1 --2- 172.21.15.1:0/3697989818 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c18102d20 0x7f2c18198520 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f2c08006010 tx=0x7f2c080042f0 comp rx=0 tx=0).stop 2024-09-11T22:05:05.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:05.000+0000 7f2c1da73640 1 -- 172.21.15.1:0/3697989818 shutdown_connections 2024-09-11T22:05:05.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:05.000+0000 7f2c1da73640 1 --2- 172.21.15.1:0/3697989818 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f2c040a2cf0 0x7f2c040a51b0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:05.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:05.000+0000 7f2c1da73640 1 --2- 172.21.15.1:0/3697989818 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c1810c150 0x7f2c1819cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:05.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:05.000+0000 7f2c1da73640 1 --2- 172.21.15.1:0/3697989818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c18103640 0x7f2c18198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:05.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:05.000+0000 7f2c1da73640 1 --2- 172.21.15.1:0/3697989818 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c18102d20 0x7f2c18198520 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:05.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:05.000+0000 7f2c1da73640 1 -- 172.21.15.1:0/3697989818 >> 172.21.15.1:0/3697989818 conn(0x7f2c180fc780 msgr2=0x7f2c180fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:05.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:05.000+0000 7f2c1da73640 1 -- 172.21.15.1:0/3697989818 shutdown_connections 2024-09-11T22:05:05.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:05.000+0000 7f2c1da73640 1 -- 172.21.15.1:0/3697989818 wait complete. 2024-09-11T22:05:05.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:05 smithi001 ceph-mon[30935]: pgmap v242: 97 pgs: 97 active+clean; 469 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 411 KiB/s wr, 193 op/s 2024-09-11T22:05:05.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:05 smithi053 ceph-mon[34908]: pgmap v242: 97 pgs: 97 active+clean; 469 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 411 KiB/s wr, 193 op/s 2024-09-11T22:05:05.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:05 smithi161 ceph-mon[35615]: pgmap v242: 97 pgs: 97 active+clean; 469 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 411 KiB/s wr, 193 op/s 2024-09-11T22:05:06.702 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T22:05:06.702 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T22:05:06.950 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.946+0000 7f9de6d81640 1 -- 172.21.15.1:0/4283495451 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9de01069b0 msgr2=0x7f9de0106d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:06.950 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.946+0000 7f9de6d81640 1 --2- 172.21.15.1:0/4283495451 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9de01069b0 0x7f9de0106d90 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f9dd4009930 tx=0x7f9dd402a420 comp rx=0 tx=0).stop 2024-09-11T22:05:06.950 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.946+0000 7f9de6d81640 1 -- 172.21.15.1:0/4283495451 shutdown_connections 2024-09-11T22:05:06.950 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.946+0000 7f9de6d81640 1 --2- 172.21.15.1:0/4283495451 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9de0101340 0x7f9de010d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:06.950 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.946+0000 7f9de6d81640 1 --2- 172.21.15.1:0/4283495451 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9de01009a0 0x7f9de0100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:06.950 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.946+0000 7f9de6d81640 1 --2- 172.21.15.1:0/4283495451 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9de01069b0 0x7f9de0106d90 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:06.950 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.946+0000 7f9de6d81640 1 -- 172.21.15.1:0/4283495451 >> 172.21.15.1:0/4283495451 conn(0x7f9de00779a0 msgr2=0x7f9de00fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:06.950 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.946+0000 7f9de6d81640 1 -- 172.21.15.1:0/4283495451 shutdown_connections 2024-09-11T22:05:06.950 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.946+0000 7f9de6d81640 1 -- 172.21.15.1:0/4283495451 wait complete. 2024-09-11T22:05:06.951 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.947+0000 7f9de6d81640 1 Processor -- start 2024-09-11T22:05:06.951 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.947+0000 7f9de6d81640 1 -- start start 2024-09-11T22:05:06.951 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.947+0000 7f9de6d81640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9de01009a0 0x7f9de00fcde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:06.951 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.947+0000 7f9de6d81640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9de0101340 0x7f9de00fd320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:06.951 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.947+0000 7f9de6d81640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9de01069b0 0x7f9de00fd860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:06.951 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.947+0000 7f9de6d81640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9de0110e00 con 0x7f9de0101340 2024-09-11T22:05:06.951 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.947+0000 7f9de6d81640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9de0110c80 con 0x7f9de01009a0 2024-09-11T22:05:06.951 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.947+0000 7f9de6d81640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9de0110f80 con 0x7f9de01069b0 2024-09-11T22:05:06.951 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.947+0000 7f9ddffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9de0101340 0x7f9de00fd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:06.951 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.947+0000 7f9ddffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9de0101340 0x7f9de00fd320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50572/0 (socket says 172.21.15.1:50572) 2024-09-11T22:05:06.951 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.947+0000 7f9ddffff640 1 -- 172.21.15.1:0/2853944758 learned_addr learned my addr 172.21.15.1:0/2853944758 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:06.951 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.947+0000 7f9de531e640 1 --2- 172.21.15.1:0/2853944758 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9de01069b0 0x7f9de00fd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:06.952 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.948+0000 7f9de4b1d640 1 --2- 172.21.15.1:0/2853944758 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9de01009a0 0x7f9de00fcde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:06.952 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.948+0000 7f9ddffff640 1 -- 172.21.15.1:0/2853944758 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9de01069b0 msgr2=0x7f9de00fd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:06.952 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.948+0000 7f9ddffff640 1 --2- 172.21.15.1:0/2853944758 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9de01069b0 0x7f9de00fd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:06.952 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.948+0000 7f9ddffff640 1 -- 172.21.15.1:0/2853944758 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9de01009a0 msgr2=0x7f9de00fcde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:06.952 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.948+0000 7f9ddffff640 1 --2- 172.21.15.1:0/2853944758 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9de01009a0 0x7f9de00fcde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:06.952 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.948+0000 7f9ddffff640 1 -- 172.21.15.1:0/2853944758 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9de01a68d0 con 0x7f9de0101340 2024-09-11T22:05:06.952 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.948+0000 7f9de4b1d640 1 --2- 172.21.15.1:0/2853944758 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9de01009a0 0x7f9de00fcde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:05:06.952 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.948+0000 7f9ddffff640 1 --2- 172.21.15.1:0/2853944758 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9de0101340 0x7f9de00fd320 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc00c920 tx=0x7f9dcc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:06.952 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.948+0000 7f9dddffb640 1 -- 172.21.15.1:0/2853944758 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc007c70 con 0x7f9de0101340 2024-09-11T22:05:06.952 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.948+0000 7f9de6d81640 1 -- 172.21.15.1:0/2853944758 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9de01a6bc0 con 0x7f9de0101340 2024-09-11T22:05:06.952 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.948+0000 7f9dddffb640 1 -- 172.21.15.1:0/2853944758 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9dcc007e10 con 0x7f9de0101340 2024-09-11T22:05:06.952 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.948+0000 7f9dddffb640 1 -- 172.21.15.1:0/2853944758 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc018890 con 0x7f9de0101340 2024-09-11T22:05:06.953 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.949+0000 7f9de6d81640 1 -- 172.21.15.1:0/2853944758 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9de01a7110 con 0x7f9de0101340 2024-09-11T22:05:06.960 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.951+0000 7f9dddffb640 1 -- 172.21.15.1:0/2853944758 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f9dcc018a70 con 0x7f9de0101340 2024-09-11T22:05:06.961 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.951+0000 7f9de9bf4740 1 Processor -- start 2024-09-11T22:05:06.961 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.951+0000 7f9de9bf4740 1 -- start start 2024-09-11T22:05:06.961 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.951+0000 7f9de9bf4740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556483b29af0 0x556483b29ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:06.961 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.951+0000 7f9de9bf4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556483b2a410 0x556483b34770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:06.961 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.951+0000 7f9de9bf4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556483b34cb0 0x556483b370a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:06.961 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.951+0000 7f9de9bf4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x556483a8e2f0 con 0x556483b34cb0 2024-09-11T22:05:06.961 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.951+0000 7f9de9bf4740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x556483a36270 con 0x556483b29af0 2024-09-11T22:05:06.961 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.951+0000 7f9de9bf4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556483a8a2b0 con 0x556483b2a410 2024-09-11T22:05:06.962 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.952+0000 7f9ddffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556483b34cb0 0x556483b370a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:06.962 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.952+0000 7f9ddffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556483b34cb0 0x556483b370a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50586/0 (socket says 172.21.15.1:50586) 2024-09-11T22:05:06.962 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.952+0000 7f9ddffff640 1 -- 172.21.15.1:0/3194059903 learned_addr learned my addr 172.21.15.1:0/3194059903 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:06.962 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.952+0000 7f9de4b1d640 1 --2- 172.21.15.1:0/3194059903 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556483b2a410 0x556483b34770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:06.962 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.952+0000 7f9de531e640 1 --2- 172.21.15.1:0/3194059903 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556483b29af0 0x556483b29ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:06.962 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.952+0000 7f9dddffb640 1 --2- 172.21.15.1:0/2853944758 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f9dd00a2cf0 0x7f9dd00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:06.962 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.952+0000 7f9ddffff640 1 -- 172.21.15.1:0/3194059903 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556483b2a410 msgr2=0x556483b34770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:06.962 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.952+0000 7f9ddffff640 1 --2- 172.21.15.1:0/3194059903 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556483b2a410 0x556483b34770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:06.962 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.952+0000 7f9ddffff640 1 -- 172.21.15.1:0/3194059903 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556483b29af0 msgr2=0x556483b29ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:06.963 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.952+0000 7f9ddffff640 1 --2- 172.21.15.1:0/3194059903 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556483b29af0 0x556483b29ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:06.963 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.952+0000 7f9ddffff640 1 -- 172.21.15.1:0/3194059903 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556483a35340 con 0x556483b34cb0 2024-09-11T22:05:06.963 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.952+0000 7f9de4b1d640 1 --2- 172.21.15.1:0/3194059903 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556483b2a410 0x556483b34770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:05:06.963 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.952+0000 7f9de531e640 1 --2- 172.21.15.1:0/2853944758 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f9dd00a2cf0 0x7f9dd00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:06.963 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.952+0000 7f9dddffb640 1 -- 172.21.15.1:0/2853944758 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7f9dcc0cf610 con 0x7f9de0101340 2024-09-11T22:05:06.963 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.952+0000 7f9ddffff640 1 --2- 172.21.15.1:0/3194059903 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556483b34cb0 0x556483b370a0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc0ccc60 tx=0x7f9dcc066ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=100a53a96bcc3d9a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:06.963 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.952+0000 7f9dcaffd640 1 -- 172.21.15.1:0/3194059903 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc09b420 con 0x556483b34cb0 2024-09-11T22:05:06.963 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.952+0000 7f9dcaffd640 1 -- 172.21.15.1:0/3194059903 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9dcc066b50 con 0x556483b34cb0 2024-09-11T22:05:06.963 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.953+0000 7f9dcaffd640 1 -- 172.21.15.1:0/3194059903 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc066de0 con 0x556483b34cb0 2024-09-11T22:05:06.963 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.953+0000 7f9de531e640 1 --2- 172.21.15.1:0/2853944758 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f9dd00a2cf0 0x7f9dd00a51b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9dd8005fd0 tx=0x7f9dd8004a40 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:06.963 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.953+0000 7f9de9bf4740 1 -- 172.21.15.1:0/3194059903 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556483b34cb0 msgr2=0x556483b370a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:06.963 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.953+0000 7f9de9bf4740 1 --2- 172.21.15.1:0/3194059903 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556483b34cb0 0x556483b370a0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc0ccc60 tx=0x7f9dcc066ae0 comp rx=0 tx=0).stop 2024-09-11T22:05:06.963 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.954+0000 7f9de9bf4740 1 -- 172.21.15.1:0/3194059903 shutdown_connections 2024-09-11T22:05:06.964 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.954+0000 7f9de9bf4740 1 --2- 172.21.15.1:0/3194059903 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556483b34cb0 0x556483b370a0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:06.964 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.954+0000 7f9de9bf4740 1 --2- 172.21.15.1:0/3194059903 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556483b2a410 0x556483b34770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:06.964 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.954+0000 7f9de9bf4740 1 --2- 172.21.15.1:0/3194059903 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556483b29af0 0x556483b29ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:06.964 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.954+0000 7f9de9bf4740 1 -- 172.21.15.1:0/3194059903 >> 172.21.15.1:0/3194059903 conn(0x556483b26060 msgr2=0x556483b26440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:06.964 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.954+0000 7f9de9bf4740 1 -- 172.21.15.1:0/3194059903 shutdown_connections 2024-09-11T22:05:06.964 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.954+0000 7f9de9bf4740 1 -- 172.21.15.1:0/3194059903 wait complete. 2024-09-11T22:05:06.964 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.954+0000 7f9de9bf4740 1 -- start start 2024-09-11T22:05:06.964 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.954+0000 7f9de9bf4740 1 Processor -- start 2024-09-11T22:05:06.964 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.955+0000 7f9de9bf4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556483b29af0 0x556483b31b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:06.965 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.955+0000 7f9de9bf4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556483b2a410 0x556483b320c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:06.965 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.955+0000 7f9de9bf4740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556483b34cb0 0x556483b32600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:06.965 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.955+0000 7f9de9bf4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x556483b37880 con 0x556483b2a410 2024-09-11T22:05:06.965 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.955+0000 7f9de9bf4740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x556483a17290 con 0x556483b34cb0 2024-09-11T22:05:06.965 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.955+0000 7f9de9bf4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x556483b37a00 con 0x556483b29af0 2024-09-11T22:05:06.965 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.955+0000 7f9de9bf4740 10 client.0 _ll_register_callbacks cb 0x556483a48140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:05:06.965 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.955+0000 7f9ddffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556483b2a410 0x556483b320c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:06.965 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.955+0000 7f9ddffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556483b2a410 0x556483b320c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50602/0 (socket says 172.21.15.1:50602) 2024-09-11T22:05:06.966 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.955+0000 7f9ddffff640 1 -- 172.21.15.1:0/105844561 learned_addr learned my addr 172.21.15.1:0/105844561 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:06.966 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.955+0000 7f9de531e640 1 --2- 172.21.15.1:0/105844561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556483b34cb0 0x556483b32600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:06.966 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.955+0000 7f9de4b1d640 1 --2- 172.21.15.1:0/105844561 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556483b29af0 0x556483b31b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:06.966 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.955+0000 7f9ddffff640 1 -- 172.21.15.1:0/105844561 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556483b29af0 msgr2=0x556483b31b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:06.966 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.955+0000 7f9ddffff640 1 --2- 172.21.15.1:0/105844561 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556483b29af0 0x556483b31b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:06.966 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.955+0000 7f9ddffff640 1 -- 172.21.15.1:0/105844561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556483b34cb0 msgr2=0x556483b32600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:06.966 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.955+0000 7f9ddffff640 1 --2- 172.21.15.1:0/105844561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556483b34cb0 0x556483b32600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:06.966 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.955+0000 7f9ddffff640 1 -- 172.21.15.1:0/105844561 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564837a1210 con 0x556483b2a410 2024-09-11T22:05:06.966 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.956+0000 7f9ddffff640 1 --2- 172.21.15.1:0/105844561 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556483b2a410 0x556483b320c0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc0cd2f0 tx=0x7f9dcc00ff30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8525a0983e10880e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:06.966 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.956+0000 7f9dbffff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:05:06.967 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.956+0000 7f9dbffff640 1 -- 172.21.15.1:0/105844561 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc06b990 con 0x556483b2a410 2024-09-11T22:05:06.967 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.956+0000 7f9dbffff640 1 -- 172.21.15.1:0/105844561 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7f9dcc0d5120 con 0x556483b2a410 2024-09-11T22:05:06.967 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.956+0000 7f9de9bf4740 1 -- 172.21.15.1:0/105844561 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9dcc066b50 con 0x556483b2a410 2024-09-11T22:05:06.967 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.956+0000 7f9dbffff640 1 -- 172.21.15.1:0/105844561 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f9dcc0b4050 con 0x556483b2a410 2024-09-11T22:05:06.967 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.956+0000 7f9de9bf4740 10 client.17226 fetch_fsmap learned FSMap version 42 2024-09-11T22:05:06.967 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.956+0000 7f9de9bf4740 1 -- 172.21.15.1:0/105844561 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55648386d620 con 0x556483b2a410 2024-09-11T22:05:06.967 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.956+0000 7f9dbffff640 1 -- 172.21.15.1:0/105844561 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9dcc066b50 con 0x556483b2a410 2024-09-11T22:05:06.967 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.956+0000 7f9dbffff640 1 -- 172.21.15.1:0/105844561 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f9dcc098130 con 0x556483b2a410 2024-09-11T22:05:06.967 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.956+0000 7f9de9bf4740 10 client.17226 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:05:06.967 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.956+0000 7f9de9bf4740 10 client.17226 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:05:06.967 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.956+0000 7f9de9bf4740 20 client.17226 populate_metadata read hostname 'smithi001' 2024-09-11T22:05:06.967 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.956+0000 7f9de9bf4740 1 --2- 172.21.15.1:0/105844561 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x556483b44ab0 0x556483b46ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:06.967 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.956+0000 7f9de9bf4740 4 client.17226 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:05:06.968 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.956+0000 7f9de9bf4740 1 -- 172.21.15.1:0/105844561 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556483a9ba20 con 0x556483b44ab0 2024-09-11T22:05:06.968 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.957+0000 7f9de4b1d640 1 --2- 172.21.15.1:0/105844561 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x556483b44ab0 0x556483b46ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:06.968 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.957+0000 7f9dbffff640 10 client.17226 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:05:06.968 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.957+0000 7f9de4b1d640 1 --2- 172.21.15.1:0/105844561 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x556483b44ab0 0x556483b46ea0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e562f0fcefeedd90 server_cookie=53d637bd5a7371df in_seq=0 out_seq=0 2024-09-11T22:05:06.968 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.958+0000 7f9dbffff640 1 -- 172.21.15.1:0/105844561 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556483a9ba20 con 0x556483b44ab0 2024-09-11T22:05:06.968 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.958+0000 7f9dbffff640 10 client.17226 handle_command_reply: tid=0 2024-09-11T22:05:06.968 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.958+0000 7f9de9bf4740 1 client.17226 shutdown 2024-09-11T22:05:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.958+0000 7f9de9bf4740 1 -- 172.21.15.1:0/105844561 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556483b2a410 msgr2=0x556483b320c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.958+0000 7f9de9bf4740 1 --2- 172.21.15.1:0/105844561 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556483b2a410 0x556483b320c0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc0cd2f0 tx=0x7f9dcc00ff30 comp rx=0 tx=0).stop 2024-09-11T22:05:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.958+0000 7f9de9bf4740 1 -- 172.21.15.1:0/105844561 shutdown_connections 2024-09-11T22:05:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.958+0000 7f9de9bf4740 1 --2- 172.21.15.1:0/105844561 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x556483b44ab0 0x556483b46ea0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.958+0000 7f9de9bf4740 1 --2- 172.21.15.1:0/105844561 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556483b34cb0 0x556483b32600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.958+0000 7f9de9bf4740 1 --2- 172.21.15.1:0/105844561 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556483b2a410 0x556483b320c0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.958+0000 7f9dbffff640 0 client.17226 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:05:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.958+0000 7f9de9bf4740 1 --2- 172.21.15.1:0/105844561 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556483b29af0 0x556483b31b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.958+0000 7f9de9bf4740 1 -- 172.21.15.1:0/105844561 >> 172.21.15.1:0/105844561 conn(0x556483b26060 msgr2=0x556483b2bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:06.970 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.958+0000 7f9de9bf4740 1 -- 172.21.15.1:0/105844561 shutdown_connections 2024-09-11T22:05:06.970 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.958+0000 7f9de9bf4740 1 -- 172.21.15.1:0/105844561 wait complete. 2024-09-11T22:05:06.970 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.958+0000 7f9de9bf4740 20 client.17226 trim_cache size 0 max 16384 2024-09-11T22:05:06.980 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.974+0000 7f9de9bf4740 1 Processor -- start 2024-09-11T22:05:06.980 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.974+0000 7f9de9bf4740 1 -- start start 2024-09-11T22:05:06.980 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.975+0000 7f9de9bf4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556483b29af0 0x556483b32430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:06.981 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.975+0000 7f9de9bf4740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556483b2a410 0x556483b4c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:06.981 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.975+0000 7f9de9bf4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556483b34cb0 0x556483b4ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:06.981 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.975+0000 7f9de9bf4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9de01afc10 con 0x556483b29af0 2024-09-11T22:05:06.981 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.975+0000 7f9de9bf4740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9de01b1a50 con 0x556483b2a410 2024-09-11T22:05:06.981 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.975+0000 7f9de9bf4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9de01b03a0 con 0x556483b34cb0 2024-09-11T22:05:06.981 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.975+0000 7f9de4b1d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556483b29af0 0x556483b32430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:06.981 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.975+0000 7f9de4b1d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556483b29af0 0x556483b32430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50604/0 (socket says 172.21.15.1:50604) 2024-09-11T22:05:06.981 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.975+0000 7f9de4b1d640 1 -- 172.21.15.1:0/1753525516 learned_addr learned my addr 172.21.15.1:0/1753525516 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:06.981 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.975+0000 7f9ddffff640 1 --2- 172.21.15.1:0/1753525516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556483b2a410 0x556483b4c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:06.981 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.975+0000 7f9de531e640 1 --2- 172.21.15.1:0/1753525516 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556483b34cb0 0x556483b4ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:06.981 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.975+0000 7f9de4b1d640 1 -- 172.21.15.1:0/1753525516 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556483b34cb0 msgr2=0x556483b4ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:06.981 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.975+0000 7f9de4b1d640 1 --2- 172.21.15.1:0/1753525516 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556483b34cb0 0x556483b4ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:06.982 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.975+0000 7f9de4b1d640 1 -- 172.21.15.1:0/1753525516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556483b2a410 msgr2=0x556483b4c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:06.982 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.975+0000 7f9de4b1d640 1 --2- 172.21.15.1:0/1753525516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556483b2a410 0x556483b4c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:06.982 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.975+0000 7f9de4b1d640 1 -- 172.21.15.1:0/1753525516 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556483b25e90 con 0x556483b29af0 2024-09-11T22:05:06.982 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.975+0000 7f9ddffff640 1 --2- 172.21.15.1:0/1753525516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556483b2a410 0x556483b4c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:05:06.982 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.975+0000 7f9de4b1d640 1 --2- 172.21.15.1:0/1753525516 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556483b29af0 0x556483b32430 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f9dd402ab30 tx=0x7f9dd4031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f008724534ee1f4b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:06.982 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.976+0000 7f9dca7fc640 1 -- 172.21.15.1:0/1753525516 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd4035040 con 0x556483b29af0 2024-09-11T22:05:06.982 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.976+0000 7f9dca7fc640 1 -- 172.21.15.1:0/1753525516 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9dd404a070 con 0x556483b29af0 2024-09-11T22:05:06.982 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.976+0000 7f9dca7fc640 1 -- 172.21.15.1:0/1753525516 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd40319b0 con 0x556483b29af0 2024-09-11T22:05:06.982 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.976+0000 7f9de9bf4740 1 -- 172.21.15.1:0/1753525516 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556483b29af0 msgr2=0x556483b32430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:06.982 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.976+0000 7f9de9bf4740 1 --2- 172.21.15.1:0/1753525516 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556483b29af0 0x556483b32430 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f9dd402ab30 tx=0x7f9dd4031400 comp rx=0 tx=0).stop 2024-09-11T22:05:06.982 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.976+0000 7f9de9bf4740 1 -- 172.21.15.1:0/1753525516 shutdown_connections 2024-09-11T22:05:06.982 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.976+0000 7f9de9bf4740 1 --2- 172.21.15.1:0/1753525516 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556483b34cb0 0x556483b4ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:06.983 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.976+0000 7f9de9bf4740 1 --2- 172.21.15.1:0/1753525516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556483b2a410 0x556483b4c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:06.983 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.976+0000 7f9de9bf4740 1 --2- 172.21.15.1:0/1753525516 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556483b29af0 0x556483b32430 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:06.983 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.976+0000 7f9de9bf4740 1 -- 172.21.15.1:0/1753525516 >> 172.21.15.1:0/1753525516 conn(0x556483b26060 msgr2=0x556483b33c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:06.983 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.976+0000 7f9de9bf4740 1 -- 172.21.15.1:0/1753525516 shutdown_connections 2024-09-11T22:05:06.983 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.976+0000 7f9de9bf4740 1 -- 172.21.15.1:0/1753525516 wait complete. 2024-09-11T22:05:06.983 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.977+0000 7f9de9bf4740 1 -- start start 2024-09-11T22:05:06.983 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.977+0000 7f9de9bf4740 1 Processor -- start 2024-09-11T22:05:06.983 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.977+0000 7f9de9bf4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556483b29af0 0x556483b38b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:06.983 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.977+0000 7f9de9bf4740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556483b2a410 0x556483b39060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:06.983 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.977+0000 7f9de9bf4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556483b34cb0 0x556483b395a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:06.983 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.977+0000 7f9de9bf4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9de01b12c0 con 0x556483b34cb0 2024-09-11T22:05:06.983 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.977+0000 7f9de9bf4740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9de01b0b30 con 0x556483b2a410 2024-09-11T22:05:06.984 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.977+0000 7f9de9bf4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9de01adc10 con 0x556483b29af0 2024-09-11T22:05:06.984 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.977+0000 7f9de9bf4740 10 client.0 _ll_register_callbacks cb 0x7f9dd4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:05:06.984 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.977+0000 7f9de531e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556483b34cb0 0x556483b395a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:06.984 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.977+0000 7f9de531e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556483b34cb0 0x556483b395a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50610/0 (socket says 172.21.15.1:50610) 2024-09-11T22:05:06.984 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.977+0000 7f9de531e640 1 -- 172.21.15.1:0/4258849314 learned_addr learned my addr 172.21.15.1:0/4258849314 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:06.984 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.977+0000 7f9de4b1d640 1 --2- 172.21.15.1:0/4258849314 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556483b29af0 0x556483b38b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:06.984 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.978+0000 7f9ddffff640 1 --2- 172.21.15.1:0/4258849314 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556483b2a410 0x556483b39060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:06.984 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.978+0000 7f9de531e640 1 -- 172.21.15.1:0/4258849314 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556483b29af0 msgr2=0x556483b38b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:06.984 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.978+0000 7f9de531e640 1 --2- 172.21.15.1:0/4258849314 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556483b29af0 0x556483b38b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:06.984 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.978+0000 7f9de531e640 1 -- 172.21.15.1:0/4258849314 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556483b2a410 msgr2=0x556483b39060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:06.984 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.978+0000 7f9de531e640 1 --2- 172.21.15.1:0/4258849314 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556483b2a410 0x556483b39060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:06.984 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.978+0000 7f9de531e640 1 -- 172.21.15.1:0/4258849314 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556483b2c940 con 0x556483b34cb0 2024-09-11T22:05:06.985 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.978+0000 7f9de531e640 1 --2- 172.21.15.1:0/4258849314 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556483b34cb0 0x556483b395a0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f9dd801f6e0 tx=0x7f9dd8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d90652b0f82f177 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:06.985 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.978+0000 7f9dbffff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:05:06.985 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.978+0000 7f9dbffff640 1 -- 172.21.15.1:0/4258849314 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd8012a80 con 0x556483b34cb0 2024-09-11T22:05:06.985 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.978+0000 7f9dbffff640 1 -- 172.21.15.1:0/4258849314 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7f9dd8012c20 con 0x556483b34cb0 2024-09-11T22:05:06.985 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.978+0000 7f9de9bf4740 1 -- 172.21.15.1:0/4258849314 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9dd404a070 con 0x556483b34cb0 2024-09-11T22:05:06.985 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.978+0000 7f9dbffff640 1 -- 172.21.15.1:0/4258849314 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f9dd8031410 con 0x556483b34cb0 2024-09-11T22:05:06.985 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.978+0000 7f9de9bf4740 10 client.17232 fetch_fsmap learned FSMap version 42 2024-09-11T22:05:06.985 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.978+0000 7f9de9bf4740 1 -- 172.21.15.1:0/4258849314 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x556483baf750 con 0x556483b34cb0 2024-09-11T22:05:06.985 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.979+0000 7f9dbffff640 1 -- 172.21.15.1:0/4258849314 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9dd404a070 con 0x556483b34cb0 2024-09-11T22:05:06.985 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.979+0000 7f9dbffff640 1 -- 172.21.15.1:0/4258849314 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f9dd802a760 con 0x556483b34cb0 2024-09-11T22:05:06.985 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.979+0000 7f9de9bf4740 10 client.17232 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:05:06.985 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.979+0000 7f9de9bf4740 10 client.17232 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:05:06.985 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.979+0000 7f9de9bf4740 20 client.17232 populate_metadata read hostname 'smithi001' 2024-09-11T22:05:06.986 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.979+0000 7f9de9bf4740 1 --2- 172.21.15.1:0/4258849314 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x556483baffe0 0x556483bb2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:06.986 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.979+0000 7f9de9bf4740 4 client.17232 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:05:06.986 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.979+0000 7f9de9bf4740 1 -- 172.21.15.1:0/4258849314 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556483a95d70 con 0x556483baffe0 2024-09-11T22:05:06.986 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.979+0000 7f9de4b1d640 1 --2- 172.21.15.1:0/4258849314 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x556483baffe0 0x556483bb2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:06.986 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.980+0000 7f9de4b1d640 1 --2- 172.21.15.1:0/4258849314 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x556483baffe0 0x556483bb2400 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de8f78e3c0588f7 server_cookie=6747629a621900b2 in_seq=0 out_seq=0 2024-09-11T22:05:06.986 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:06.980+0000 7f9dbffff640 10 client.17232 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:05:06.986 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:05:07.036 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:07 smithi001 ceph-mon[30935]: pgmap v243: 97 pgs: 97 active+clean; 469 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 897 KiB/s rd, 456 KiB/s wr, 214 op/s 2024-09-11T22:05:07.123 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:07.122+0000 7f9dbffff640 1 -- 172.21.15.1:0/4258849314 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556483a95d70 con 0x556483baffe0 2024-09-11T22:05:07.123 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:07.122+0000 7f9dbffff640 10 client.17232 handle_command_reply: tid=0 2024-09-11T22:05:07.124 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:07.122+0000 7f9de9bf4740 1 client.17232 shutdown 2024-09-11T22:05:07.124 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:07.122+0000 7f9de9bf4740 1 -- 172.21.15.1:0/4258849314 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556483b34cb0 msgr2=0x556483b395a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:07.124 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:07.122+0000 7f9de9bf4740 1 --2- 172.21.15.1:0/4258849314 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556483b34cb0 0x556483b395a0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f9dd801f6e0 tx=0x7f9dd8004ab0 comp rx=0 tx=0).stop 2024-09-11T22:05:07.124 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:07.122+0000 7f9de9bf4740 1 -- 172.21.15.1:0/4258849314 shutdown_connections 2024-09-11T22:05:07.124 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:07.122+0000 7f9de9bf4740 1 --2- 172.21.15.1:0/4258849314 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x556483baffe0 0x556483bb2400 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:07.124 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:07.122+0000 7f9de9bf4740 1 --2- 172.21.15.1:0/4258849314 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556483b34cb0 0x556483b395a0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:07.124 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:07.122+0000 7f9dbffff640 0 client.17232 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:05:07.124 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:07.122+0000 7f9de9bf4740 1 --2- 172.21.15.1:0/4258849314 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556483b2a410 0x556483b39060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:07.125 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:07.122+0000 7f9de9bf4740 1 --2- 172.21.15.1:0/4258849314 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x556483b29af0 0x556483b38b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:07.125 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:07.122+0000 7f9de9bf4740 1 -- 172.21.15.1:0/4258849314 >> 172.21.15.1:0/4258849314 conn(0x556483b26060 msgr2=0x556483b249a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:07.125 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:07.122+0000 7f9de9bf4740 1 -- 172.21.15.1:0/4258849314 shutdown_connections 2024-09-11T22:05:07.125 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:07.122+0000 7f9de9bf4740 1 -- 172.21.15.1:0/4258849314 wait complete. 2024-09-11T22:05:07.125 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:07.122+0000 7f9de9bf4740 20 client.17232 trim_cache size 0 max 16384 2024-09-11T22:05:07.126 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-09-11T22:05:07.126 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-09-11T22:05:07.126 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-09-11T22:05:07.126 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-09-11T22:05:07.127 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:07.123+0000 7f9de6d81640 1 -- 172.21.15.1:0/2853944758 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f9dd00a2cf0 msgr2=0x7f9dd00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:07.127 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:07.123+0000 7f9de6d81640 1 --2- 172.21.15.1:0/2853944758 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f9dd00a2cf0 0x7f9dd00a51b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9dd8005fd0 tx=0x7f9dd8004a40 comp rx=0 tx=0).stop 2024-09-11T22:05:07.127 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:07.123+0000 7f9de6d81640 1 -- 172.21.15.1:0/2853944758 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9de0101340 msgr2=0x7f9de00fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:07.127 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:07.123+0000 7f9de6d81640 1 --2- 172.21.15.1:0/2853944758 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9de0101340 0x7f9de00fd320 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc00c920 tx=0x7f9dcc00cde0 comp rx=0 tx=0).stop 2024-09-11T22:05:07.128 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:07.123+0000 7f9de6d81640 1 -- 172.21.15.1:0/2853944758 shutdown_connections 2024-09-11T22:05:07.128 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:07.123+0000 7f9de6d81640 1 --2- 172.21.15.1:0/2853944758 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f9dd00a2cf0 0x7f9dd00a51b0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:07.128 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:07.123+0000 7f9de6d81640 1 --2- 172.21.15.1:0/2853944758 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9de01069b0 0x7f9de00fd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:07.128 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:07.123+0000 7f9de6d81640 1 --2- 172.21.15.1:0/2853944758 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9de0101340 0x7f9de00fd320 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:07.128 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:07.123+0000 7f9de6d81640 1 --2- 172.21.15.1:0/2853944758 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9de01009a0 0x7f9de00fcde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:07.128 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:07.123+0000 7f9de6d81640 1 -- 172.21.15.1:0/2853944758 >> 172.21.15.1:0/2853944758 conn(0x7f9de00779a0 msgr2=0x7f9de0069c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:07.128 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:07.123+0000 7f9de6d81640 1 -- 172.21.15.1:0/2853944758 shutdown_connections 2024-09-11T22:05:07.129 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:07.123+0000 7f9de6d81640 1 -- 172.21.15.1:0/2853944758 wait complete. 2024-09-11T22:05:07.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:07 smithi053 ceph-mon[34908]: pgmap v243: 97 pgs: 97 active+clean; 469 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 897 KiB/s rd, 456 KiB/s wr, 214 op/s 2024-09-11T22:05:07.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:07 smithi161 ceph-mon[35615]: pgmap v243: 97 pgs: 97 active+clean; 469 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 897 KiB/s rd, 456 KiB/s wr, 214 op/s 2024-09-11T22:05:07.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.524+0000 7fe922c45640 1 -- 172.21.15.1:0/1614879000 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe91c102570 msgr2=0x7fe91c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:07.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.524+0000 7fe922c45640 1 --2- 172.21.15.1:0/1614879000 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe91c102570 0x7fe91c1029b0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fe908009900 tx=0x7fe90802a480 comp rx=0 tx=0).stop 2024-09-11T22:05:07.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.525+0000 7fe922c45640 1 -- 172.21.15.1:0/1614879000 shutdown_connections 2024-09-11T22:05:07.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.525+0000 7fe922c45640 1 --2- 172.21.15.1:0/1614879000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe91c102ef0 0x7fe91c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:07.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.525+0000 7fe922c45640 1 --2- 172.21.15.1:0/1614879000 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe91c102570 0x7fe91c1029b0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:07.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.525+0000 7fe922c45640 1 --2- 172.21.15.1:0/1614879000 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe91c100e20 0x7fe91c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:07.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.525+0000 7fe922c45640 1 -- 172.21.15.1:0/1614879000 >> 172.21.15.1:0/1614879000 conn(0x7fe91c0fc7d0 msgr2=0x7fe91c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:07.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.525+0000 7fe922c45640 1 -- 172.21.15.1:0/1614879000 shutdown_connections 2024-09-11T22:05:07.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.525+0000 7fe922c45640 1 -- 172.21.15.1:0/1614879000 wait complete. 2024-09-11T22:05:07.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.525+0000 7fe922c45640 1 Processor -- start 2024-09-11T22:05:07.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.525+0000 7fe922c45640 1 -- start start 2024-09-11T22:05:07.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.526+0000 7fe922c45640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe91c100e20 0x7fe91c199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:07.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.526+0000 7fe922c45640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe91c102570 0x7fe91c199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:07.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.526+0000 7fe922c45640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe91c102ef0 0x7fe91c1a4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:07.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.526+0000 7fe922c45640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe91c110f80 con 0x7fe91c102ef0 2024-09-11T22:05:07.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.526+0000 7fe922c45640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe91c110e00 con 0x7fe91c102570 2024-09-11T22:05:07.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.526+0000 7fe922c45640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe91c111100 con 0x7fe91c100e20 2024-09-11T22:05:07.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.526+0000 7fe9211e2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe91c102ef0 0x7fe91c1a4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:07.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.526+0000 7fe9209e1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe91c100e20 0x7fe91c199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:07.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.526+0000 7fe9209e1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe91c100e20 0x7fe91c199310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:41206/0 (socket says 172.21.15.1:41206) 2024-09-11T22:05:07.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.526+0000 7fe9211e2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe91c102ef0 0x7fe91c1a4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50624/0 (socket says 172.21.15.1:50624) 2024-09-11T22:05:07.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.526+0000 7fe9209e1640 1 -- 172.21.15.1:0/3938865324 learned_addr learned my addr 172.21.15.1:0/3938865324 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:07.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.526+0000 7fe91bfff640 1 --2- 172.21.15.1:0/3938865324 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe91c102570 0x7fe91c199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:07.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.526+0000 7fe9209e1640 1 -- 172.21.15.1:0/3938865324 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe91c102570 msgr2=0x7fe91c199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:07.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.526+0000 7fe9209e1640 1 --2- 172.21.15.1:0/3938865324 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe91c102570 0x7fe91c199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:07.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.526+0000 7fe9209e1640 1 -- 172.21.15.1:0/3938865324 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe91c102ef0 msgr2=0x7fe91c1a4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:07.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.526+0000 7fe9209e1640 1 --2- 172.21.15.1:0/3938865324 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe91c102ef0 0x7fe91c1a4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:07.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.526+0000 7fe9209e1640 1 -- 172.21.15.1:0/3938865324 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe91c1a48d0 con 0x7fe91c100e20 2024-09-11T22:05:07.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.526+0000 7fe9211e2640 1 --2- 172.21.15.1:0/3938865324 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe91c102ef0 0x7fe91c1a4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:05:07.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.526+0000 7fe9209e1640 1 --2- 172.21.15.1:0/3938865324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe91c100e20 0x7fe91c199310 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe91000c890 tx=0x7fe91000cd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:07.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.527+0000 7fe919ffb640 1 -- 172.21.15.1:0/3938865324 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe910004220 con 0x7fe91c100e20 2024-09-11T22:05:07.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.527+0000 7fe919ffb640 1 -- 172.21.15.1:0/3938865324 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9100043c0 con 0x7fe91c100e20 2024-09-11T22:05:07.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.527+0000 7fe922c45640 1 -- 172.21.15.1:0/3938865324 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe91c1a4aa0 con 0x7fe91c100e20 2024-09-11T22:05:07.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.527+0000 7fe919ffb640 1 -- 172.21.15.1:0/3938865324 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9100049e0 con 0x7fe91c100e20 2024-09-11T22:05:07.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.527+0000 7fe922c45640 1 -- 172.21.15.1:0/3938865324 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe91c1a4fb0 con 0x7fe91c100e20 2024-09-11T22:05:07.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.529+0000 7fe919ffb640 1 -- 172.21.15.1:0/3938865324 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7fe9100079b0 con 0x7fe91c100e20 2024-09-11T22:05:07.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.529+0000 7fe92501e740 1 Processor -- start 2024-09-11T22:05:07.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.529+0000 7fe92501e740 1 -- start start 2024-09-11T22:05:07.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.530+0000 7fe92501e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557585084f70 0x557585085350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:07.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.530+0000 7fe92501e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557585085890 0x55758508fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:07.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.530+0000 7fe92501e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557585090150 0x557585092540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:07.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.530+0000 7fe92501e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x557584fe6280 con 0x557585085890 2024-09-11T22:05:07.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.530+0000 7fe92501e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x557584f8db10 con 0x557585090150 2024-09-11T22:05:07.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.530+0000 7fe92501e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x557584fe2240 con 0x557585084f70 2024-09-11T22:05:07.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.530+0000 7fe9209e1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557585085890 0x55758508fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:07.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.530+0000 7fe9209e1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557585085890 0x55758508fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50638/0 (socket says 172.21.15.1:50638) 2024-09-11T22:05:07.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.530+0000 7fe9209e1640 1 -- 172.21.15.1:0/4282086834 learned_addr learned my addr 172.21.15.1:0/4282086834 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:07.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.530+0000 7fe91bfff640 1 --2- 172.21.15.1:0/4282086834 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557585090150 0x557585092540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:07.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.530+0000 7fe9211e2640 1 --2- 172.21.15.1:0/4282086834 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557585084f70 0x557585085350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:07.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.530+0000 7fe919ffb640 1 --2- 172.21.15.1:0/3938865324 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fe90c0a2cf0 0x7fe90c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:07.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.530+0000 7fe9209e1640 1 -- 172.21.15.1:0/4282086834 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557585084f70 msgr2=0x557585085350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:07.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.530+0000 7fe9209e1640 1 --2- 172.21.15.1:0/4282086834 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557585084f70 0x557585085350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:07.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.530+0000 7fe9209e1640 1 -- 172.21.15.1:0/4282086834 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557585090150 msgr2=0x557585092540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:07.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.530+0000 7fe9209e1640 1 --2- 172.21.15.1:0/4282086834 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557585090150 0x557585092540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:07.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.530+0000 7fe9209e1640 1 -- 172.21.15.1:0/4282086834 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557584f9a1f0 con 0x557585085890 2024-09-11T22:05:07.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.530+0000 7fe9211e2640 1 --2- 172.21.15.1:0/4282086834 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557585084f70 0x557585085350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:05:07.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.530+0000 7fe9211e2640 1 --2- 172.21.15.1:0/3938865324 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fe90c0a2cf0 0x7fe90c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:07.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.530+0000 7fe919ffb640 1 -- 172.21.15.1:0/3938865324 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7fe9100cf580 con 0x7fe91c100e20 2024-09-11T22:05:07.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.530+0000 7fe9209e1640 1 --2- 172.21.15.1:0/4282086834 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557585085890 0x55758508fc10 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fe9100ccb70 tx=0x7fe910066600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5619c058111450b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:07.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.531+0000 7fe906ffd640 1 -- 172.21.15.1:0/4282086834 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91001d030 con 0x557585085890 2024-09-11T22:05:07.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.531+0000 7fe906ffd640 1 -- 172.21.15.1:0/4282086834 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe91009c0d0 con 0x557585085890 2024-09-11T22:05:07.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.531+0000 7fe906ffd640 1 -- 172.21.15.1:0/4282086834 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91006a710 con 0x557585085890 2024-09-11T22:05:07.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.531+0000 7fe9211e2640 1 --2- 172.21.15.1:0/3938865324 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fe90c0a2cf0 0x7fe90c0a51b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe914009fd0 tx=0x7fe914004a40 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:07.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.531+0000 7fe92501e740 1 -- 172.21.15.1:0/4282086834 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557585085890 msgr2=0x55758508fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:07.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.531+0000 7fe92501e740 1 --2- 172.21.15.1:0/4282086834 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557585085890 0x55758508fc10 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fe9100ccb70 tx=0x7fe910066600 comp rx=0 tx=0).stop 2024-09-11T22:05:07.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.531+0000 7fe92501e740 1 -- 172.21.15.1:0/4282086834 shutdown_connections 2024-09-11T22:05:07.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.531+0000 7fe92501e740 1 --2- 172.21.15.1:0/4282086834 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557585090150 0x557585092540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:07.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.531+0000 7fe92501e740 1 --2- 172.21.15.1:0/4282086834 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557585085890 0x55758508fc10 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:07.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.531+0000 7fe92501e740 1 --2- 172.21.15.1:0/4282086834 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557585084f70 0x557585085350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:07.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.531+0000 7fe92501e740 1 -- 172.21.15.1:0/4282086834 >> 172.21.15.1:0/4282086834 conn(0x557585081500 msgr2=0x5575850818e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:07.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.532+0000 7fe92501e740 1 -- 172.21.15.1:0/4282086834 shutdown_connections 2024-09-11T22:05:07.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.532+0000 7fe92501e740 1 -- 172.21.15.1:0/4282086834 wait complete. 2024-09-11T22:05:07.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.532+0000 7fe92501e740 1 -- start start 2024-09-11T22:05:07.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.532+0000 7fe92501e740 1 Processor -- start 2024-09-11T22:05:07.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.532+0000 7fe92501e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557585084f70 0x55758508cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:07.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.532+0000 7fe92501e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557585085890 0x55758508d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:07.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.532+0000 7fe92501e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557585090150 0x55758508da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:07.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.532+0000 7fe92501e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x557585092d20 con 0x557585085890 2024-09-11T22:05:07.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.532+0000 7fe92501e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x557584fa0f10 con 0x557585084f70 2024-09-11T22:05:07.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.532+0000 7fe92501e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x557585092ea0 con 0x557585090150 2024-09-11T22:05:07.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.532+0000 7fe92501e740 10 client.0 _ll_register_callbacks cb 0x557584f69a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:05:07.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.533+0000 7fe91bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557585085890 0x55758508d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:07.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.533+0000 7fe91bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557585085890 0x55758508d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50646/0 (socket says 172.21.15.1:50646) 2024-09-11T22:05:07.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.533+0000 7fe91bfff640 1 -- 172.21.15.1:0/2974106494 learned_addr learned my addr 172.21.15.1:0/2974106494 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:07.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.533+0000 7fe9211e2640 1 --2- 172.21.15.1:0/2974106494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557585090150 0x55758508da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:07.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.533+0000 7fe9209e1640 1 --2- 172.21.15.1:0/2974106494 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557585084f70 0x55758508cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:07.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.533+0000 7fe91bfff640 1 -- 172.21.15.1:0/2974106494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557585090150 msgr2=0x55758508da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:07.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.533+0000 7fe91bfff640 1 --2- 172.21.15.1:0/2974106494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557585090150 0x55758508da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:07.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.533+0000 7fe91bfff640 1 -- 172.21.15.1:0/2974106494 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557585084f70 msgr2=0x55758508cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:07.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.533+0000 7fe91bfff640 1 --2- 172.21.15.1:0/2974106494 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557585084f70 0x55758508cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:07.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.533+0000 7fe91bfff640 1 -- 172.21.15.1:0/2974106494 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557584d576a0 con 0x557585085890 2024-09-11T22:05:07.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.533+0000 7fe9209e1640 1 --2- 172.21.15.1:0/2974106494 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557585084f70 0x55758508cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:05:07.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.533+0000 7fe91bfff640 1 --2- 172.21.15.1:0/2974106494 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557585085890 0x55758508d4c0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fe908009900 tx=0x7fe908005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33891ad68e5e3daa server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:07.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.533+0000 7fe8fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:05:07.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.533+0000 7fe8fbfff640 1 -- 172.21.15.1:0/2974106494 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe908035040 con 0x557585085890 2024-09-11T22:05:07.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.533+0000 7fe8fbfff640 1 -- 172.21.15.1:0/2974106494 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7fe9080048c0 con 0x557585085890 2024-09-11T22:05:07.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.533+0000 7fe92501e740 1 -- 172.21.15.1:0/2974106494 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe91009c0d0 con 0x557585085890 2024-09-11T22:05:07.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.533+0000 7fe8fbfff640 1 -- 172.21.15.1:0/2974106494 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fe908004ac0 con 0x557585085890 2024-09-11T22:05:07.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.533+0000 7fe92501e740 10 client.17244 fetch_fsmap learned FSMap version 42 2024-09-11T22:05:07.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.533+0000 7fe92501e740 1 -- 172.21.15.1:0/2974106494 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55758509e870 con 0x557585085890 2024-09-11T22:05:07.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.534+0000 7fe8fbfff640 1 -- 172.21.15.1:0/2974106494 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe91009c0d0 con 0x557585085890 2024-09-11T22:05:07.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.534+0000 7fe8fbfff640 1 -- 172.21.15.1:0/2974106494 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7fe908031540 con 0x557585085890 2024-09-11T22:05:07.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.534+0000 7fe92501e740 10 client.17244 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:05:07.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.534+0000 7fe92501e740 10 client.17244 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:05:07.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.534+0000 7fe92501e740 20 client.17244 populate_metadata read hostname 'smithi001' 2024-09-11T22:05:07.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.534+0000 7fe92501e740 1 --2- 172.21.15.1:0/2974106494 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55758509fd30 0x5575850a2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:07.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.534+0000 7fe92501e740 4 client.17244 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:05:07.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.534+0000 7fe92501e740 1 -- 172.21.15.1:0/2974106494 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557584ff37f0 con 0x55758509fd30 2024-09-11T22:05:07.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.534+0000 7fe9209e1640 1 --2- 172.21.15.1:0/2974106494 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55758509fd30 0x5575850a2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:07.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.535+0000 7fe9209e1640 1 --2- 172.21.15.1:0/2974106494 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55758509fd30 0x5575850a2180 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b53a4400b3a6f260 server_cookie=fdacfa1cd800f9ba in_seq=0 out_seq=0 2024-09-11T22:05:07.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.535+0000 7fe8fbfff640 10 client.17244 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:05:07.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.535+0000 7fe8fbfff640 1 -- 172.21.15.1:0/2974106494 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557584ff37f0 con 0x55758509fd30 2024-09-11T22:05:07.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.535+0000 7fe8fbfff640 10 client.17244 handle_command_reply: tid=0 2024-09-11T22:05:07.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.535+0000 7fe92501e740 1 client.17244 shutdown 2024-09-11T22:05:07.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.535+0000 7fe92501e740 1 -- 172.21.15.1:0/2974106494 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557585085890 msgr2=0x55758508d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:07.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.535+0000 7fe92501e740 1 --2- 172.21.15.1:0/2974106494 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557585085890 0x55758508d4c0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fe908009900 tx=0x7fe908005250 comp rx=0 tx=0).stop 2024-09-11T22:05:07.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.535+0000 7fe92501e740 1 -- 172.21.15.1:0/2974106494 shutdown_connections 2024-09-11T22:05:07.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.535+0000 7fe92501e740 1 --2- 172.21.15.1:0/2974106494 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55758509fd30 0x5575850a2180 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:07.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.536+0000 7fe92501e740 1 --2- 172.21.15.1:0/2974106494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557585090150 0x55758508da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:07.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.536+0000 7fe92501e740 1 --2- 172.21.15.1:0/2974106494 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557585085890 0x55758508d4c0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:07.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.536+0000 7fe92501e740 1 --2- 172.21.15.1:0/2974106494 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557585084f70 0x55758508cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:07.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.536+0000 7fe8fbfff640 0 client.17244 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:05:07.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.536+0000 7fe92501e740 1 -- 172.21.15.1:0/2974106494 >> 172.21.15.1:0/2974106494 conn(0x557585081500 msgr2=0x55758507c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:07.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.536+0000 7fe92501e740 1 -- 172.21.15.1:0/2974106494 shutdown_connections 2024-09-11T22:05:07.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.536+0000 7fe92501e740 1 -- 172.21.15.1:0/2974106494 wait complete. 2024-09-11T22:05:07.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.536+0000 7fe92501e740 20 client.17244 trim_cache size 0 max 16384 2024-09-11T22:05:07.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.553+0000 7fe92501e740 1 Processor -- start 2024-09-11T22:05:07.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.553+0000 7fe92501e740 1 -- start start 2024-09-11T22:05:07.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.553+0000 7fe92501e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557585084f70 0x55758507c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:07.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.553+0000 7fe92501e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557585085890 0x5575850a7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:07.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.553+0000 7fe92501e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557585090150 0x5575850a7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:07.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.553+0000 7fe92501e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe91c1ab910 con 0x557585085890 2024-09-11T22:05:07.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.553+0000 7fe92501e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe91c1ad750 con 0x557585084f70 2024-09-11T22:05:07.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.553+0000 7fe92501e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe91c1ac0a0 con 0x557585090150 2024-09-11T22:05:07.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.553+0000 7fe91bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557585085890 0x5575850a7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:07.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.554+0000 7fe91bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557585085890 0x5575850a7a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50658/0 (socket says 172.21.15.1:50658) 2024-09-11T22:05:07.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.554+0000 7fe91bfff640 1 -- 172.21.15.1:0/2807042080 learned_addr learned my addr 172.21.15.1:0/2807042080 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:07.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.554+0000 7fe9209e1640 1 --2- 172.21.15.1:0/2807042080 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557585084f70 0x55758507c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:07.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.554+0000 7fe9211e2640 1 --2- 172.21.15.1:0/2807042080 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557585090150 0x5575850a7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:07.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.554+0000 7fe91bfff640 1 -- 172.21.15.1:0/2807042080 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557585090150 msgr2=0x5575850a7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:07.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.554+0000 7fe91bfff640 1 --2- 172.21.15.1:0/2807042080 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557585090150 0x5575850a7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:07.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.554+0000 7fe91bfff640 1 -- 172.21.15.1:0/2807042080 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557585084f70 msgr2=0x55758507c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:07.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.554+0000 7fe91bfff640 1 --2- 172.21.15.1:0/2807042080 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557585084f70 0x55758507c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:07.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.554+0000 7fe91bfff640 1 -- 172.21.15.1:0/2807042080 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55758508bf50 con 0x557585085890 2024-09-11T22:05:07.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.554+0000 7fe9209e1640 1 --2- 172.21.15.1:0/2807042080 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557585084f70 0x55758507c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:05:07.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.554+0000 7fe9211e2640 1 --2- 172.21.15.1:0/2807042080 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557585090150 0x5575850a7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:05:07.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.554+0000 7fe91bfff640 1 --2- 172.21.15.1:0/2807042080 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557585085890 0x5575850a7a90 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fe908006010 tx=0x7fe908005b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c24e3bb8332fc2fc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:07.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.554+0000 7fe9067fc640 1 -- 172.21.15.1:0/2807042080 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe908035070 con 0x557585085890 2024-09-11T22:05:07.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.554+0000 7fe9067fc640 1 -- 172.21.15.1:0/2807042080 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe908040070 con 0x557585085890 2024-09-11T22:05:07.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.554+0000 7fe9067fc640 1 -- 172.21.15.1:0/2807042080 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9080048e0 con 0x557585085890 2024-09-11T22:05:07.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.555+0000 7fe92501e740 1 -- 172.21.15.1:0/2807042080 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557585085890 msgr2=0x5575850a7a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:07.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.555+0000 7fe92501e740 1 --2- 172.21.15.1:0/2807042080 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557585085890 0x5575850a7a90 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fe908006010 tx=0x7fe908005b90 comp rx=0 tx=0).stop 2024-09-11T22:05:07.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.555+0000 7fe92501e740 1 -- 172.21.15.1:0/2807042080 shutdown_connections 2024-09-11T22:05:07.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.555+0000 7fe92501e740 1 --2- 172.21.15.1:0/2807042080 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557585090150 0x5575850a7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:07.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.555+0000 7fe92501e740 1 --2- 172.21.15.1:0/2807042080 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557585085890 0x5575850a7a90 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:07.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.555+0000 7fe92501e740 1 --2- 172.21.15.1:0/2807042080 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557585084f70 0x55758507c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:07.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.555+0000 7fe92501e740 1 -- 172.21.15.1:0/2807042080 >> 172.21.15.1:0/2807042080 conn(0x557585081500 msgr2=0x55758507ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:07.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.555+0000 7fe92501e740 1 -- 172.21.15.1:0/2807042080 shutdown_connections 2024-09-11T22:05:07.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.555+0000 7fe92501e740 1 -- 172.21.15.1:0/2807042080 wait complete. 2024-09-11T22:05:07.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.556+0000 7fe92501e740 1 -- start start 2024-09-11T22:05:07.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.556+0000 7fe92501e740 1 Processor -- start 2024-09-11T22:05:07.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.556+0000 7fe92501e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557585084f70 0x5575850980f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:07.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.556+0000 7fe92501e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557585085890 0x557585098630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:07.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.556+0000 7fe92501e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557585090150 0x5575850950c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:07.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.556+0000 7fe92501e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe91c1acfc0 con 0x557585090150 2024-09-11T22:05:07.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.556+0000 7fe92501e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fe91c1ac830 con 0x557585085890 2024-09-11T22:05:07.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.556+0000 7fe92501e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fe91c1a9910 con 0x557585084f70 2024-09-11T22:05:07.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.556+0000 7fe92501e740 10 client.0 _ll_register_callbacks cb 0x7fe908004520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:05:07.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.556+0000 7fe9211e2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557585090150 0x5575850950c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:07.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.556+0000 7fe9211e2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557585090150 0x5575850950c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50674/0 (socket says 172.21.15.1:50674) 2024-09-11T22:05:07.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.556+0000 7fe9211e2640 1 -- 172.21.15.1:0/1492375995 learned_addr learned my addr 172.21.15.1:0/1492375995 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:07.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.556+0000 7fe9209e1640 1 --2- 172.21.15.1:0/1492375995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557585084f70 0x5575850980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:07.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.557+0000 7fe91bfff640 1 --2- 172.21.15.1:0/1492375995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557585085890 0x557585098630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:07.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.557+0000 7fe9209e1640 1 -- 172.21.15.1:0/1492375995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557585085890 msgr2=0x557585098630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:07.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.557+0000 7fe9209e1640 1 --2- 172.21.15.1:0/1492375995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557585085890 0x557585098630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:07.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.557+0000 7fe9209e1640 1 -- 172.21.15.1:0/1492375995 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557585090150 msgr2=0x5575850950c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:07.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.557+0000 7fe9209e1640 1 --2- 172.21.15.1:0/1492375995 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557585090150 0x5575850950c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:07.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.557+0000 7fe9209e1640 1 -- 172.21.15.1:0/1492375995 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557585078830 con 0x557585084f70 2024-09-11T22:05:07.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.557+0000 7fe9211e2640 1 --2- 172.21.15.1:0/1492375995 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557585090150 0x5575850950c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:05:07.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.557+0000 7fe9209e1640 1 --2- 172.21.15.1:0/1492375995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557585084f70 0x5575850980f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x557585082360 tx=0x7fe9100054a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff3dd93f6e1022ce server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:07.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.557+0000 7fe8fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-11T22:05:07.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.557+0000 7fe8fbfff640 1 -- 172.21.15.1:0/1492375995 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91001d040 con 0x557585084f70 2024-09-11T22:05:07.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.557+0000 7fe8fbfff640 1 -- 172.21.15.1:0/1492375995 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7fe910099cd0 con 0x557585084f70 2024-09-11T22:05:07.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.557+0000 7fe92501e740 1 -- 172.21.15.1:0/1492375995 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe908040070 con 0x557585084f70 2024-09-11T22:05:07.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.557+0000 7fe8fbfff640 1 -- 172.21.15.1:0/1492375995 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fe91009c0d0 con 0x557585084f70 2024-09-11T22:05:07.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.557+0000 7fe92501e740 10 client.26668 fetch_fsmap learned FSMap version 42 2024-09-11T22:05:07.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.557+0000 7fe92501e740 1 -- 172.21.15.1:0/1492375995 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55758510b010 con 0x557585084f70 2024-09-11T22:05:07.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.558+0000 7fe8fbfff640 1 -- 172.21.15.1:0/1492375995 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe908040070 con 0x557585084f70 2024-09-11T22:05:07.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.558+0000 7fe8fbfff640 1 -- 172.21.15.1:0/1492375995 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7fe910098040 con 0x557585084f70 2024-09-11T22:05:07.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.558+0000 7fe92501e740 10 client.26668 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:05:07.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.558+0000 7fe92501e740 10 client.26668 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:05:07.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.558+0000 7fe92501e740 20 client.26668 populate_metadata read hostname 'smithi001' 2024-09-11T22:05:07.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.558+0000 7fe92501e740 1 --2- 172.21.15.1:0/1492375995 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55758510b460 0x55758510d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:07.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.558+0000 7fe92501e740 4 client.26668 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T22:05:07.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.558+0000 7fe92501e740 1 -- 172.21.15.1:0/1492375995 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557584fedd00 con 0x55758510b460 2024-09-11T22:05:07.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.558+0000 7fe91bfff640 1 --2- 172.21.15.1:0/1492375995 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55758510b460 0x55758510d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:07.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.559+0000 7fe8fbfff640 10 client.26668 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:05:07.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.559+0000 7fe91bfff640 1 --2- 172.21.15.1:0/1492375995 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55758510b460 0x55758510d870 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c028a221d6b957c7 server_cookie=be0674b03b96dca1 in_seq=0 out_seq=0 2024-09-11T22:05:07.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.560+0000 7fe8fbfff640 1 -- 172.21.15.1:0/1492375995 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557584fedd00 con 0x55758510b460 2024-09-11T22:05:07.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.560+0000 7fe8fbfff640 10 client.26668 handle_command_reply: tid=0 2024-09-11T22:05:07.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.560+0000 7fe92501e740 1 client.26668 shutdown 2024-09-11T22:05:07.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.560+0000 7fe92501e740 1 -- 172.21.15.1:0/1492375995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557585084f70 msgr2=0x5575850980f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:07.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.560+0000 7fe92501e740 1 --2- 172.21.15.1:0/1492375995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557585084f70 0x5575850980f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x557585082360 tx=0x7fe9100054a0 comp rx=0 tx=0).stop 2024-09-11T22:05:07.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.560+0000 7fe92501e740 1 -- 172.21.15.1:0/1492375995 shutdown_connections 2024-09-11T22:05:07.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.560+0000 7fe92501e740 1 --2- 172.21.15.1:0/1492375995 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55758510b460 0x55758510d870 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:07.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.560+0000 7fe92501e740 1 --2- 172.21.15.1:0/1492375995 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557585090150 0x5575850950c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:07.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.560+0000 7fe92501e740 1 --2- 172.21.15.1:0/1492375995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557585085890 0x557585098630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:07.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.560+0000 7fe92501e740 1 --2- 172.21.15.1:0/1492375995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557585084f70 0x5575850980f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:07.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.560+0000 7fe92501e740 1 -- 172.21.15.1:0/1492375995 >> 172.21.15.1:0/1492375995 conn(0x557585081500 msgr2=0x5575850871e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:07.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.560+0000 7fe8fbfff640 0 client.26668 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:05:07.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.560+0000 7fe92501e740 1 -- 172.21.15.1:0/1492375995 shutdown_connections 2024-09-11T22:05:07.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.560+0000 7fe92501e740 1 -- 172.21.15.1:0/1492375995 wait complete. 2024-09-11T22:05:07.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.560+0000 7fe92501e740 20 client.26668 trim_cache size 0 max 16384 2024-09-11T22:05:07.566 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-11T22:05:07.566 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-09-11T22:05:07.566 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "5b45b944-7717-43cb-828e-314edc1bcac9", 2024-09-11T22:05:07.566 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-09-11T22:05:07.566 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-11T22:05:07.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.561+0000 7fe922c45640 1 -- 172.21.15.1:0/3938865324 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fe90c0a2cf0 msgr2=0x7fe90c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:07.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.561+0000 7fe922c45640 1 --2- 172.21.15.1:0/3938865324 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fe90c0a2cf0 0x7fe90c0a51b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe914009fd0 tx=0x7fe914004a40 comp rx=0 tx=0).stop 2024-09-11T22:05:07.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.561+0000 7fe922c45640 1 -- 172.21.15.1:0/3938865324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe91c100e20 msgr2=0x7fe91c199310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:07.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.561+0000 7fe922c45640 1 --2- 172.21.15.1:0/3938865324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe91c100e20 0x7fe91c199310 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe91000c890 tx=0x7fe91000cd50 comp rx=0 tx=0).stop 2024-09-11T22:05:07.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.561+0000 7fe922c45640 1 -- 172.21.15.1:0/3938865324 shutdown_connections 2024-09-11T22:05:07.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.561+0000 7fe922c45640 1 --2- 172.21.15.1:0/3938865324 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fe90c0a2cf0 0x7fe90c0a51b0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:07.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.561+0000 7fe922c45640 1 --2- 172.21.15.1:0/3938865324 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe91c102ef0 0x7fe91c1a4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:07.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.561+0000 7fe922c45640 1 --2- 172.21.15.1:0/3938865324 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe91c102570 0x7fe91c199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:07.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.561+0000 7fe922c45640 1 --2- 172.21.15.1:0/3938865324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe91c100e20 0x7fe91c199310 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:07.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.561+0000 7fe922c45640 1 -- 172.21.15.1:0/3938865324 >> 172.21.15.1:0/3938865324 conn(0x7fe91c0fc7d0 msgr2=0x7fe91c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:07.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.561+0000 7fe922c45640 1 -- 172.21.15.1:0/3938865324 shutdown_connections 2024-09-11T22:05:07.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:07.561+0000 7fe922c45640 1 -- 172.21.15.1:0/3938865324 wait complete. 2024-09-11T22:05:08.252 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:05:08.598 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:05:09.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.127+0000 7f011c356640 1 -- 172.21.15.1:0/3146976760 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0114103ff0 msgr2=0x7f01141043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:09.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.127+0000 7f011c356640 1 --2- 172.21.15.1:0/3146976760 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0114103ff0 0x7f01141043d0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f0110009900 tx=0x7f011002a410 comp rx=0 tx=0).stop 2024-09-11T22:05:09.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.127+0000 7f011c356640 1 -- 172.21.15.1:0/3146976760 shutdown_connections 2024-09-11T22:05:09.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.127+0000 7f011c356640 1 --2- 172.21.15.1:0/3146976760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0114109120 0x7f0114110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:09.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.127+0000 7f011c356640 1 --2- 172.21.15.1:0/3146976760 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01141049a0 0x7f01141089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:09.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.127+0000 7f011c356640 1 --2- 172.21.15.1:0/3146976760 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0114103ff0 0x7f01141043d0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:09.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.127+0000 7f011c356640 1 -- 172.21.15.1:0/3146976760 >> 172.21.15.1:0/3146976760 conn(0x7f01140ffa90 msgr2=0x7f0114101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:09.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.127+0000 7f011c356640 1 -- 172.21.15.1:0/3146976760 shutdown_connections 2024-09-11T22:05:09.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.127+0000 7f011c356640 1 -- 172.21.15.1:0/3146976760 wait complete. 2024-09-11T22:05:09.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.127+0000 7f011c356640 1 Processor -- start 2024-09-11T22:05:09.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.128+0000 7f011c356640 1 -- start start 2024-09-11T22:05:09.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.128+0000 7f011c356640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0114103ff0 0x7f011419a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:09.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.128+0000 7f011c356640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01141049a0 0x7f011419abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:09.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.128+0000 7f011c356640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0114109120 0x7f011419ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:09.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.128+0000 7f011c356640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f01141141a0 con 0x7f0114109120 2024-09-11T22:05:09.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.128+0000 7f011c356640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0114114020 con 0x7f01141049a0 2024-09-11T22:05:09.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.128+0000 7f011c356640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0114114320 con 0x7f0114103ff0 2024-09-11T22:05:09.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.128+0000 7f011a8f3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0114109120 0x7f011419ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:09.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.128+0000 7f011a8f3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0114109120 0x7f011419ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50696/0 (socket says 172.21.15.1:50696) 2024-09-11T22:05:09.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.128+0000 7f011a8f3640 1 -- 172.21.15.1:0/791182827 learned_addr learned my addr 172.21.15.1:0/791182827 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:09.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.128+0000 7f01198f1640 1 --2- 172.21.15.1:0/791182827 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01141049a0 0x7f011419abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:09.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.128+0000 7f011a0f2640 1 --2- 172.21.15.1:0/791182827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0114103ff0 0x7f011419a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:09.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.128+0000 7f011a8f3640 1 -- 172.21.15.1:0/791182827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0114103ff0 msgr2=0x7f011419a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:09.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.128+0000 7f011a8f3640 1 --2- 172.21.15.1:0/791182827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0114103ff0 0x7f011419a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:09.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.128+0000 7f011a8f3640 1 -- 172.21.15.1:0/791182827 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01141049a0 msgr2=0x7f011419abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:09.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.128+0000 7f011a8f3640 1 --2- 172.21.15.1:0/791182827 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01141049a0 0x7f011419abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:09.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.128+0000 7f011a8f3640 1 -- 172.21.15.1:0/791182827 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f011419f630 con 0x7f0114109120 2024-09-11T22:05:09.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.128+0000 7f011a8f3640 1 --2- 172.21.15.1:0/791182827 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0114109120 0x7f011419ef50 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f010c00eaa0 tx=0x7f010c00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:09.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.129+0000 7f01077fe640 1 -- 172.21.15.1:0/791182827 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f010c004d20 con 0x7f0114109120 2024-09-11T22:05:09.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.129+0000 7f011c356640 1 -- 172.21.15.1:0/791182827 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f011419f920 con 0x7f0114109120 2024-09-11T22:05:09.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.129+0000 7f01077fe640 1 -- 172.21.15.1:0/791182827 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f010c005020 con 0x7f0114109120 2024-09-11T22:05:09.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.129+0000 7f01077fe640 1 -- 172.21.15.1:0/791182827 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f010c0116d0 con 0x7f0114109120 2024-09-11T22:05:09.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.129+0000 7f011c356640 1 -- 172.21.15.1:0/791182827 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01141a7200 con 0x7f0114109120 2024-09-11T22:05:09.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.131+0000 7f011e72f740 1 Processor -- start 2024-09-11T22:05:09.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.131+0000 7f01077fe640 1 -- 172.21.15.1:0/791182827 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f010c0051d0 con 0x7f0114109120 2024-09-11T22:05:09.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.131+0000 7f011e72f740 1 -- start start 2024-09-11T22:05:09.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.132+0000 7f011e72f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e081f874a0 0x55e081f87880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:09.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.132+0000 7f011e72f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e081f87dc0 0x55e081f92140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:09.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.132+0000 7f011e72f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e081f92680 0x55e081f94a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:09.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.132+0000 7f011e72f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55e081eed2e0 con 0x55e081f874a0 2024-09-11T22:05:09.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.132+0000 7f011e72f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55e081e94ce0 con 0x55e081f87dc0 2024-09-11T22:05:09.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.132+0000 7f011e72f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55e081ee92a0 con 0x55e081f92680 2024-09-11T22:05:09.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.132+0000 7f01198f1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e081f874a0 0x55e081f87880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:09.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.132+0000 7f01198f1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e081f874a0 0x55e081f87880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50700/0 (socket says 172.21.15.1:50700) 2024-09-11T22:05:09.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.132+0000 7f01198f1640 1 -- 172.21.15.1:0/168474512 learned_addr learned my addr 172.21.15.1:0/168474512 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:09.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.132+0000 7f011a0f2640 1 --2- 172.21.15.1:0/168474512 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e081f87dc0 0x55e081f92140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:09.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.132+0000 7f01198f1640 1 --2- 172.21.15.1:0/168474512 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e081f92680 0x55e081f94a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:09.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.132+0000 7f01077fe640 1 --2- 172.21.15.1:0/791182827 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f01000a2cf0 0x7f01000a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:09.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.132+0000 7f01198f1640 1 -- 172.21.15.1:0/168474512 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e081f92680 msgr2=0x55e081f94a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:09.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.132+0000 7f01198f1640 1 --2- 172.21.15.1:0/168474512 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e081f92680 0x55e081f94a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:09.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.132+0000 7f01198f1640 1 -- 172.21.15.1:0/168474512 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e081f87dc0 msgr2=0x55e081f92140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:09.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.132+0000 7f01198f1640 1 --2- 172.21.15.1:0/168474512 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e081f87dc0 0x55e081f92140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:09.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.132+0000 7f01198f1640 1 -- 172.21.15.1:0/168474512 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e081e79440 con 0x55e081f874a0 2024-09-11T22:05:09.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.132+0000 7f011a0f2640 1 --2- 172.21.15.1:0/168474512 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e081f87dc0 0x55e081f92140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:05:09.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.132+0000 7f011a8f3640 1 --2- 172.21.15.1:0/791182827 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f01000a2cf0 0x7f01000a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:09.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.133+0000 7f01077fe640 1 -- 172.21.15.1:0/791182827 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7f010c0cf6a0 con 0x7f0114109120 2024-09-11T22:05:09.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.133+0000 7f01198f1640 1 --2- 172.21.15.1:0/168474512 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e081f874a0 0x55e081f87880 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f0108006aa0 tx=0x7f01080095d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6333c020284a697 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:09.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.133+0000 7f0104dea640 1 -- 172.21.15.1:0/168474512 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0108017610 con 0x55e081f874a0 2024-09-11T22:05:09.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.133+0000 7f0104dea640 1 -- 172.21.15.1:0/168474512 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f010801c070 con 0x55e081f874a0 2024-09-11T22:05:09.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.133+0000 7f0104dea640 1 -- 172.21.15.1:0/168474512 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0108015950 con 0x55e081f874a0 2024-09-11T22:05:09.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.134+0000 7f011e72f740 1 -- 172.21.15.1:0/168474512 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e081f874a0 msgr2=0x55e081f87880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:09.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.134+0000 7f011e72f740 1 --2- 172.21.15.1:0/168474512 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e081f874a0 0x55e081f87880 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f0108006aa0 tx=0x7f01080095d0 comp rx=0 tx=0).stop 2024-09-11T22:05:09.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.134+0000 7f011a8f3640 1 --2- 172.21.15.1:0/791182827 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f01000a2cf0 0x7f01000a51b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f010c09bae0 tx=0x7f010c04b040 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:09.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.134+0000 7f011e72f740 1 -- 172.21.15.1:0/168474512 shutdown_connections 2024-09-11T22:05:09.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.134+0000 7f011e72f740 1 --2- 172.21.15.1:0/168474512 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e081f92680 0x55e081f94a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:09.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.134+0000 7f011e72f740 1 --2- 172.21.15.1:0/168474512 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e081f87dc0 0x55e081f92140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:09.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.134+0000 7f011e72f740 1 --2- 172.21.15.1:0/168474512 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e081f874a0 0x55e081f87880 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:09.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.134+0000 7f011e72f740 1 -- 172.21.15.1:0/168474512 >> 172.21.15.1:0/168474512 conn(0x55e081f83a30 msgr2=0x55e081f83e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:09.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.134+0000 7f011e72f740 1 -- 172.21.15.1:0/168474512 shutdown_connections 2024-09-11T22:05:09.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.134+0000 7f011e72f740 1 -- 172.21.15.1:0/168474512 wait complete. 2024-09-11T22:05:09.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.134+0000 7f011e72f740 1 -- start start 2024-09-11T22:05:09.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.134+0000 7f011e72f740 1 Processor -- start 2024-09-11T22:05:09.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.135+0000 7f011e72f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e081f874a0 0x55e081f92030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:09.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.135+0000 7f011e72f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e081f87dc0 0x55e081f8f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:09.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.135+0000 7f011e72f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e081f92680 0x55e081f8f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:09.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.135+0000 7f011e72f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55e081f95250 con 0x55e081f87dc0 2024-09-11T22:05:09.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.135+0000 7f011e72f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55e081ea8050 con 0x55e081f92680 2024-09-11T22:05:09.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.135+0000 7f011e72f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55e081f953d0 con 0x55e081f874a0 2024-09-11T22:05:09.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.135+0000 7f011a0f2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e081f87dc0 0x55e081f8f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:09.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.135+0000 7f011e72f740 10 client.0 _ll_register_callbacks cb 0x55e081ee8160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:05:09.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.135+0000 7f011a0f2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e081f87dc0 0x55e081f8f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50710/0 (socket says 172.21.15.1:50710) 2024-09-11T22:05:09.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.135+0000 7f011a0f2640 1 -- 172.21.15.1:0/2503548617 learned_addr learned my addr 172.21.15.1:0/2503548617 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:09.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.135+0000 7f01198f1640 1 --2- 172.21.15.1:0/2503548617 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e081f92680 0x55e081f8f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:09.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.135+0000 7f01198f1640 1 --2- 172.21.15.1:0/2503548617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e081f874a0 0x55e081f92030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:09.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.135+0000 7f011a0f2640 1 -- 172.21.15.1:0/2503548617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e081f874a0 msgr2=0x55e081f92030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:09.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.135+0000 7f011a0f2640 1 --2- 172.21.15.1:0/2503548617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e081f874a0 0x55e081f92030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:09.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.135+0000 7f011a0f2640 1 -- 172.21.15.1:0/2503548617 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e081f92680 msgr2=0x55e081f8f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:09.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.135+0000 7f011a0f2640 1 --2- 172.21.15.1:0/2503548617 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e081f92680 0x55e081f8f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:09.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.135+0000 7f011a0f2640 1 -- 172.21.15.1:0/2503548617 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e081c5e3c0 con 0x55e081f87dc0 2024-09-11T22:05:09.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.135+0000 7f01198f1640 1 --2- 172.21.15.1:0/2503548617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e081f874a0 0x55e081f92030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:05:09.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.135+0000 7f011a0f2640 1 --2- 172.21.15.1:0/2503548617 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e081f87dc0 0x55e081f8f2e0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f011002a930 tx=0x7f0110031b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b5913799cb77d62 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:09.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.135+0000 7f00fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:05:09.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.135+0000 7f00fdffb640 1 -- 172.21.15.1:0/2503548617 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0110035040 con 0x55e081f87dc0 2024-09-11T22:05:09.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.136+0000 7f00fdffb640 1 -- 172.21.15.1:0/2503548617 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7f0110004480 con 0x55e081f87dc0 2024-09-11T22:05:09.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.136+0000 7f011e72f740 1 -- 172.21.15.1:0/2503548617 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f010801c070 con 0x55e081f87dc0 2024-09-11T22:05:09.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.136+0000 7f00fdffb640 1 -- 172.21.15.1:0/2503548617 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f0110005020 con 0x55e081f87dc0 2024-09-11T22:05:09.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.136+0000 7f011e72f740 10 client.17262 fetch_fsmap learned FSMap version 42 2024-09-11T22:05:09.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.136+0000 7f011e72f740 1 -- 172.21.15.1:0/2503548617 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55e081fa0c60 con 0x55e081f87dc0 2024-09-11T22:05:09.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.136+0000 7f00fdffb640 1 -- 172.21.15.1:0/2503548617 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f010801c070 con 0x55e081f87dc0 2024-09-11T22:05:09.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.136+0000 7f00fdffb640 1 -- 172.21.15.1:0/2503548617 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f01100035f0 con 0x55e081f87dc0 2024-09-11T22:05:09.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.136+0000 7f011e72f740 10 client.17262 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:05:09.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.136+0000 7f011e72f740 10 client.17262 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:05:09.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.136+0000 7f011e72f740 20 client.17262 populate_metadata read hostname 'smithi001' 2024-09-11T22:05:09.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.136+0000 7f011e72f740 1 --2- 172.21.15.1:0/2503548617 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55e081fa21f0 0x55e081fa4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:09.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.136+0000 7f011e72f740 4 client.17262 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:05:09.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.136+0000 7f011e72f740 1 -- 172.21.15.1:0/2503548617 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e081efa850 con 0x55e081fa21f0 2024-09-11T22:05:09.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.136+0000 7f01198f1640 1 --2- 172.21.15.1:0/2503548617 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55e081fa21f0 0x55e081fa4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:09.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.137+0000 7f00fdffb640 10 client.17262 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:05:09.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.137+0000 7f01198f1640 1 --2- 172.21.15.1:0/2503548617 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55e081fa21f0 0x55e081fa4610 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e88b8d0d21882d6d server_cookie=167f385e1a9ebde6 in_seq=0 out_seq=0 2024-09-11T22:05:09.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.138+0000 7f00fdffb640 1 -- 172.21.15.1:0/2503548617 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e081efa850 con 0x55e081fa21f0 2024-09-11T22:05:09.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.138+0000 7f00fdffb640 10 client.17262 handle_command_reply: tid=0 2024-09-11T22:05:09.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.138+0000 7f011e72f740 1 client.17262 shutdown 2024-09-11T22:05:09.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.138+0000 7f011e72f740 1 -- 172.21.15.1:0/2503548617 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e081f87dc0 msgr2=0x55e081f8f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:09.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.138+0000 7f011e72f740 1 --2- 172.21.15.1:0/2503548617 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e081f87dc0 0x55e081f8f2e0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f011002a930 tx=0x7f0110031b10 comp rx=0 tx=0).stop 2024-09-11T22:05:09.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.138+0000 7f011e72f740 1 -- 172.21.15.1:0/2503548617 shutdown_connections 2024-09-11T22:05:09.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.138+0000 7f011e72f740 1 --2- 172.21.15.1:0/2503548617 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55e081fa21f0 0x55e081fa4610 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:09.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.138+0000 7f011e72f740 1 --2- 172.21.15.1:0/2503548617 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e081f92680 0x55e081f8f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:09.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.138+0000 7f011e72f740 1 --2- 172.21.15.1:0/2503548617 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e081f87dc0 0x55e081f8f2e0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:09.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.138+0000 7f011e72f740 1 --2- 172.21.15.1:0/2503548617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e081f874a0 0x55e081f92030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:09.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.138+0000 7f011e72f740 1 -- 172.21.15.1:0/2503548617 >> 172.21.15.1:0/2503548617 conn(0x55e081f83a30 msgr2=0x55e081f89760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:09.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.138+0000 7f00fdffb640 0 client.17262 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:05:09.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.138+0000 7f011e72f740 1 -- 172.21.15.1:0/2503548617 shutdown_connections 2024-09-11T22:05:09.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.138+0000 7f011e72f740 1 -- 172.21.15.1:0/2503548617 wait complete. 2024-09-11T22:05:09.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.138+0000 7f011e72f740 20 client.17262 trim_cache size 0 max 16384 2024-09-11T22:05:09.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.155+0000 7f011e72f740 1 Processor -- start 2024-09-11T22:05:09.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.155+0000 7f011e72f740 1 -- start start 2024-09-11T22:05:09.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.155+0000 7f011e72f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e081f874a0 0x55e081f9e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:09.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.155+0000 7f011e72f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e081f87dc0 0x55e081fa9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:09.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.155+0000 7f011e72f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e081f92680 0x55e081faa420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:09.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.155+0000 7f011e72f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f01141afc00 con 0x55e081f87dc0 2024-09-11T22:05:09.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.155+0000 7f011e72f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f01141b1a40 con 0x55e081f92680 2024-09-11T22:05:09.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.155+0000 7f011e72f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f01141b0390 con 0x55e081f874a0 2024-09-11T22:05:09.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.156+0000 7f011a0f2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e081f87dc0 0x55e081fa9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:09.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.156+0000 7f011a0f2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e081f87dc0 0x55e081fa9ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50718/0 (socket says 172.21.15.1:50718) 2024-09-11T22:05:09.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.156+0000 7f011a0f2640 1 -- 172.21.15.1:0/1815875467 learned_addr learned my addr 172.21.15.1:0/1815875467 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:09.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.156+0000 7f01198f1640 1 --2- 172.21.15.1:0/1815875467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e081f92680 0x55e081faa420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:09.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.156+0000 7f01198f1640 1 --2- 172.21.15.1:0/1815875467 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e081f874a0 0x55e081f9e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:09.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.156+0000 7f011a0f2640 1 -- 172.21.15.1:0/1815875467 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e081f874a0 msgr2=0x55e081f9e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:09.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.156+0000 7f011a0f2640 1 --2- 172.21.15.1:0/1815875467 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e081f874a0 0x55e081f9e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:09.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.156+0000 7f011a0f2640 1 -- 172.21.15.1:0/1815875467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e081f92680 msgr2=0x55e081faa420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:09.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.156+0000 7f011a0f2640 1 --2- 172.21.15.1:0/1815875467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e081f92680 0x55e081faa420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:09.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.156+0000 7f011a0f2640 1 -- 172.21.15.1:0/1815875467 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e081f8fbf0 con 0x55e081f87dc0 2024-09-11T22:05:09.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.156+0000 7f01198f1640 1 --2- 172.21.15.1:0/1815875467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e081f92680 0x55e081faa420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:05:09.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.156+0000 7f011a0f2640 1 --2- 172.21.15.1:0/1815875467 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e081f87dc0 0x55e081fa9ee0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f0110003a10 tx=0x7f0110009610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ffe2e6e513d6081 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:09.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.156+0000 7f00fffff640 1 -- 172.21.15.1:0/1815875467 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0110035030 con 0x55e081f87dc0 2024-09-11T22:05:09.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.156+0000 7f00fffff640 1 -- 172.21.15.1:0/1815875467 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0110003c00 con 0x55e081f87dc0 2024-09-11T22:05:09.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.156+0000 7f00fffff640 1 -- 172.21.15.1:0/1815875467 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0110052040 con 0x55e081f87dc0 2024-09-11T22:05:09.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.157+0000 7f011e72f740 1 -- 172.21.15.1:0/1815875467 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e081f87dc0 msgr2=0x55e081fa9ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:09.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.157+0000 7f011e72f740 1 --2- 172.21.15.1:0/1815875467 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e081f87dc0 0x55e081fa9ee0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f0110003a10 tx=0x7f0110009610 comp rx=0 tx=0).stop 2024-09-11T22:05:09.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.157+0000 7f011e72f740 1 -- 172.21.15.1:0/1815875467 shutdown_connections 2024-09-11T22:05:09.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.157+0000 7f011e72f740 1 --2- 172.21.15.1:0/1815875467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e081f92680 0x55e081faa420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:09.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.157+0000 7f011e72f740 1 --2- 172.21.15.1:0/1815875467 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e081f87dc0 0x55e081fa9ee0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:09.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.157+0000 7f011e72f740 1 --2- 172.21.15.1:0/1815875467 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e081f874a0 0x55e081f9e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:09.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.157+0000 7f011e72f740 1 -- 172.21.15.1:0/1815875467 >> 172.21.15.1:0/1815875467 conn(0x55e081f83a30 msgr2=0x55e081f8a900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:09.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.157+0000 7f011e72f740 1 -- 172.21.15.1:0/1815875467 shutdown_connections 2024-09-11T22:05:09.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.157+0000 7f011e72f740 1 -- 172.21.15.1:0/1815875467 wait complete. 2024-09-11T22:05:09.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.157+0000 7f011e72f740 1 -- start start 2024-09-11T22:05:09.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.157+0000 7f011e72f740 1 Processor -- start 2024-09-11T22:05:09.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.158+0000 7f011e72f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e081f874a0 0x55e081f95d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:09.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.158+0000 7f011e72f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e081f87dc0 0x55e081f962a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:09.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.158+0000 7f011e72f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e081f92680 0x55e08200aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:09.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.158+0000 7f011e72f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f01141b12b0 con 0x55e081f87dc0 2024-09-11T22:05:09.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.158+0000 7f011e72f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f01141b0b20 con 0x55e081f92680 2024-09-11T22:05:09.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.158+0000 7f011e72f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f01141adc00 con 0x55e081f874a0 2024-09-11T22:05:09.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.158+0000 7f011e72f740 10 client.0 _ll_register_callbacks cb 0x7f01100315b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:05:09.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.158+0000 7f011a0f2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e081f87dc0 0x55e081f962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:09.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.158+0000 7f011a0f2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e081f87dc0 0x55e081f962a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50734/0 (socket says 172.21.15.1:50734) 2024-09-11T22:05:09.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.158+0000 7f011a0f2640 1 -- 172.21.15.1:0/3859845840 learned_addr learned my addr 172.21.15.1:0/3859845840 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:09.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.158+0000 7f01198f1640 1 --2- 172.21.15.1:0/3859845840 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e081f92680 0x55e08200aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:09.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.158+0000 7f01198f1640 1 --2- 172.21.15.1:0/3859845840 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e081f874a0 0x55e081f95d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:09.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.158+0000 7f011a0f2640 1 -- 172.21.15.1:0/3859845840 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e081f874a0 msgr2=0x55e081f95d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:09.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.158+0000 7f011a0f2640 1 --2- 172.21.15.1:0/3859845840 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e081f874a0 0x55e081f95d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:09.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.158+0000 7f011a0f2640 1 -- 172.21.15.1:0/3859845840 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e081f92680 msgr2=0x55e08200aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:09.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.158+0000 7f011a0f2640 1 --2- 172.21.15.1:0/3859845840 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e081f92680 0x55e08200aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:09.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.158+0000 7f011a0f2640 1 -- 172.21.15.1:0/3859845840 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e08200b4e0 con 0x55e081f87dc0 2024-09-11T22:05:09.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.158+0000 7f01198f1640 1 --2- 172.21.15.1:0/3859845840 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e081f874a0 0x55e081f95d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:05:09.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.158+0000 7f011a0f2640 1 --2- 172.21.15.1:0/3859845840 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e081f87dc0 0x55e081f962a0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f0110003b80 tx=0x7f011002ab90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebc7d30268c10a36 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:09.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.158+0000 7f00fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:05:09.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.158+0000 7f00fdffb640 1 -- 172.21.15.1:0/3859845840 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0110065060 con 0x55e081f87dc0 2024-09-11T22:05:09.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.158+0000 7f00fdffb640 1 -- 172.21.15.1:0/3859845840 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7f0110004480 con 0x55e081f87dc0 2024-09-11T22:05:09.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.158+0000 7f011e72f740 1 -- 172.21.15.1:0/3859845840 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0110003c00 con 0x55e081f87dc0 2024-09-11T22:05:09.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.159+0000 7f00fdffb640 1 -- 172.21.15.1:0/3859845840 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f0110052040 con 0x55e081f87dc0 2024-09-11T22:05:09.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.159+0000 7f011e72f740 10 client.17268 fetch_fsmap learned FSMap version 42 2024-09-11T22:05:09.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.159+0000 7f011e72f740 1 -- 172.21.15.1:0/3859845840 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55e08200c7d0 con 0x55e081f87dc0 2024-09-11T22:05:09.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.159+0000 7f00fdffb640 1 -- 172.21.15.1:0/3859845840 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0110003c00 con 0x55e081f87dc0 2024-09-11T22:05:09.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.159+0000 7f00fdffb640 1 -- 172.21.15.1:0/3859845840 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f0110046480 con 0x55e081f87dc0 2024-09-11T22:05:09.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.159+0000 7f011e72f740 10 client.17268 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:05:09.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.159+0000 7f011e72f740 10 client.17268 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:05:09.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.159+0000 7f011e72f740 20 client.17268 populate_metadata read hostname 'smithi001' 2024-09-11T22:05:09.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.159+0000 7f011e72f740 1 --2- 172.21.15.1:0/3859845840 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55e08200cf00 0x55e08200f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:09.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.159+0000 7f011e72f740 4 client.17268 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:05:09.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.159+0000 7f011e72f740 1 -- 172.21.15.1:0/3859845840 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e081ef4d60 con 0x55e08200cf00 2024-09-11T22:05:09.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.159+0000 7f01198f1640 1 --2- 172.21.15.1:0/3859845840 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55e08200cf00 0x55e08200f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:09.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.160+0000 7f01198f1640 1 --2- 172.21.15.1:0/3859845840 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55e08200cf00 0x55e08200f320 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68d5fd734e8ed005 server_cookie=184378a48a3d6797 in_seq=0 out_seq=0 2024-09-11T22:05:09.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.160+0000 7f00fdffb640 10 client.17268 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:05:09.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.160+0000 7f00fdffb640 1 -- 172.21.15.1:0/3859845840 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e081ef4d60 con 0x55e08200cf00 2024-09-11T22:05:09.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.160+0000 7f00fdffb640 10 client.17268 handle_command_reply: tid=0 2024-09-11T22:05:09.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.160+0000 7f011e72f740 1 client.17268 shutdown 2024-09-11T22:05:09.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.160+0000 7f011e72f740 1 -- 172.21.15.1:0/3859845840 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e081f87dc0 msgr2=0x55e081f962a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:09.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.160+0000 7f011e72f740 1 --2- 172.21.15.1:0/3859845840 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e081f87dc0 0x55e081f962a0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f0110003b80 tx=0x7f011002ab90 comp rx=0 tx=0).stop 2024-09-11T22:05:09.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.160+0000 7f011e72f740 1 -- 172.21.15.1:0/3859845840 shutdown_connections 2024-09-11T22:05:09.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.160+0000 7f011e72f740 1 --2- 172.21.15.1:0/3859845840 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55e08200cf00 0x55e08200f320 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:09.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.160+0000 7f011e72f740 1 --2- 172.21.15.1:0/3859845840 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e081f92680 0x55e08200aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:09.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.160+0000 7f011e72f740 1 --2- 172.21.15.1:0/3859845840 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e081f87dc0 0x55e081f962a0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:09.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.160+0000 7f011e72f740 1 --2- 172.21.15.1:0/3859845840 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e081f874a0 0x55e081f95d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:09.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.160+0000 7f00fdffb640 0 client.17268 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:05:09.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.160+0000 7f011e72f740 1 -- 172.21.15.1:0/3859845840 >> 172.21.15.1:0/3859845840 conn(0x55e081f83a30 msgr2=0x55e081f8da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:09.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.160+0000 7f011e72f740 1 -- 172.21.15.1:0/3859845840 shutdown_connections 2024-09-11T22:05:09.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.160+0000 7f011e72f740 1 -- 172.21.15.1:0/3859845840 wait complete. 2024-09-11T22:05:09.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.160+0000 7f011e72f740 20 client.17268 trim_cache size 0 max 16384 2024-09-11T22:05:09.168 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-11T22:05:09.168 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T22:05:09.168 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-09-11T22:05:09.168 INFO:teuthology.orchestra.run.smithi001.stdout: "5b45b944-7717-43cb-828e-314edc1bcac9": { 2024-09-11T22:05:09.168 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2024-09-11T22:05:09.169 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "5b45b944-7717-43cb-828e-314edc1bcac9", 2024-09-11T22:05:09.169 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2024-09-11T22:05:09.169 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:05:09.169 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:05:09.169 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-11T22:05:09.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.161+0000 7f011c356640 1 -- 172.21.15.1:0/791182827 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f01000a2cf0 msgr2=0x7f01000a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:09.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.161+0000 7f011c356640 1 --2- 172.21.15.1:0/791182827 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f01000a2cf0 0x7f01000a51b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f010c09bae0 tx=0x7f010c04b040 comp rx=0 tx=0).stop 2024-09-11T22:05:09.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.161+0000 7f011c356640 1 -- 172.21.15.1:0/791182827 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0114109120 msgr2=0x7f011419ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:09.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.161+0000 7f011c356640 1 --2- 172.21.15.1:0/791182827 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0114109120 0x7f011419ef50 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f010c00eaa0 tx=0x7f010c00eea0 comp rx=0 tx=0).stop 2024-09-11T22:05:09.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.161+0000 7f011c356640 1 -- 172.21.15.1:0/791182827 shutdown_connections 2024-09-11T22:05:09.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.161+0000 7f011c356640 1 --2- 172.21.15.1:0/791182827 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f01000a2cf0 0x7f01000a51b0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:09.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.161+0000 7f011c356640 1 --2- 172.21.15.1:0/791182827 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0114109120 0x7f011419ef50 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:09.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.161+0000 7f011c356640 1 --2- 172.21.15.1:0/791182827 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01141049a0 0x7f011419abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:09.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.161+0000 7f011c356640 1 --2- 172.21.15.1:0/791182827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0114103ff0 0x7f011419a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:09.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.161+0000 7f011c356640 1 -- 172.21.15.1:0/791182827 >> 172.21.15.1:0/791182827 conn(0x7f01140ffa90 msgr2=0x7f0114100dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:09.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.161+0000 7f011c356640 1 -- 172.21.15.1:0/791182827 shutdown_connections 2024-09-11T22:05:09.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:09.161+0000 7f011c356640 1 -- 172.21.15.1:0/791182827 wait complete. 2024-09-11T22:05:09.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:09 smithi053 ceph-mon[34908]: scrub queued for path: / 2024-09-11T22:05:09.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:09 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:05:09.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:09 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:05:09.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:09 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:05:09.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:09 smithi053 ceph-mon[34908]: pgmap v244: 97 pgs: 97 active+clean; 469 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 60 KiB/s wr, 26 op/s 2024-09-11T22:05:09.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:09 smithi161 ceph-mon[35615]: scrub queued for path: / 2024-09-11T22:05:09.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:09 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:05:09.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:09 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:05:09.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:09 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:05:09.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:09 smithi161 ceph-mon[35615]: pgmap v244: 97 pgs: 97 active+clean; 469 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 60 KiB/s wr, 26 op/s 2024-09-11T22:05:09.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:09 smithi001 ceph-mon[30935]: scrub queued for path: / 2024-09-11T22:05:09.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:09 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:05:09.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:09 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:05:09.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:09 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:05:09.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:09 smithi001 ceph-mon[30935]: pgmap v244: 97 pgs: 97 active+clean; 469 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 60 KiB/s wr, 26 op/s 2024-09-11T22:05:09.842 INFO:tasks.cephfs.filesystem:scrub status for tag:5b45b944-7717-43cb-828e-314edc1bcac9 - {'path': '/', 'tag': '5b45b944-7717-43cb-828e-314edc1bcac9', 'options': 'recursive,force'} 2024-09-11T22:05:11.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:11 smithi053 ceph-mon[34908]: pgmap v245: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 69 KiB/s wr, 123 op/s 2024-09-11T22:05:11.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:11 smithi161 ceph-mon[35615]: pgmap v245: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 69 KiB/s wr, 123 op/s 2024-09-11T22:05:11.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:11 smithi001 ceph-mon[30935]: pgmap v245: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 69 KiB/s wr, 123 op/s 2024-09-11T22:05:12.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:12 smithi053 ceph-mon[34908]: scrub summary: idle 2024-09-11T22:05:12.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:12 smithi161 ceph-mon[35615]: scrub summary: idle 2024-09-11T22:05:12.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:12 smithi001 ceph-mon[30935]: scrub summary: idle 2024-09-11T22:05:12.590 INFO:tasks.workunit.client.0.smithi001.stdout:ok 2024-09-11T22:05:12.595 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-11T22:05:12.596 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-11T22:05:12.641 INFO:tasks.workunit:Running workunit fs/misc/i_complete_vs_rename.sh... 2024-09-11T22:05:12.641 DEBUG:teuthology.orchestra.run.smithi001:workunit test fs/misc/i_complete_vs_rename.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=086c737fa75c22d521a3932c59b31c5e1d1c9ef4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/i_complete_vs_rename.sh 2024-09-11T22:05:13.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:13 smithi053 ceph-mon[34908]: pgmap v246: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 53 KiB/s wr, 118 op/s 2024-09-11T22:05:13.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:13 smithi161 ceph-mon[35615]: pgmap v246: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 53 KiB/s wr, 118 op/s 2024-09-11T22:05:13.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:13 smithi001 ceph-mon[30935]: pgmap v246: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 53 KiB/s wr, 118 op/s 2024-09-11T22:05:15.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:15 smithi161 ceph-mon[35615]: pgmap v247: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 53 KiB/s wr, 118 op/s 2024-09-11T22:05:15.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:15 smithi001 ceph-mon[30935]: pgmap v247: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 53 KiB/s wr, 118 op/s 2024-09-11T22:05:15.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:15 smithi053 ceph-mon[34908]: pgmap v247: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 53 KiB/s wr, 118 op/s 2024-09-11T22:05:15.712 INFO:tasks.workunit.client.0.smithi001.stdout:a 2024-09-11T22:05:15.712 INFO:tasks.workunit.client.0.smithi001.stdout:b 2024-09-11T22:05:15.712 INFO:tasks.workunit.client.0.smithi001.stdout:c 2024-09-11T22:05:15.712 INFO:tasks.workunit.client.0.smithi001.stdout:d 2024-09-11T22:05:15.725 INFO:tasks.workunit.client.0.smithi001.stderr:stat: cannot statx 'e': No such file or directory 2024-09-11T22:05:15.730 INFO:tasks.workunit.client.0.smithi001.stdout:attempting rename over existing file... 2024-09-11T22:05:15.752 INFO:tasks.workunit.client.0.smithi001.stdout:f 2024-09-11T22:05:15.752 INFO:tasks.workunit.client.0.smithi001.stdout:rename over existing file is okay 2024-09-11T22:05:15.752 INFO:tasks.workunit.client.0.smithi001.stdout:attempting rename over negative dentry... 2024-09-11T22:05:15.762 INFO:tasks.workunit.client.0.smithi001.stdout:e 2024-09-11T22:05:15.762 INFO:tasks.workunit.client.0.smithi001.stdout:rename over negative dentry is ok 2024-09-11T22:05:15.762 INFO:tasks.workunit.client.0.smithi001.stdout:OK 2024-09-11T22:05:15.763 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-11T22:05:15.763 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-11T22:05:15.830 INFO:tasks.workunit:Running workunit fs/misc/layout_vxattrs.sh... 2024-09-11T22:05:15.831 DEBUG:teuthology.orchestra.run.smithi001:workunit test fs/misc/layout_vxattrs.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=086c737fa75c22d521a3932c59b31c5e1d1c9ef4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/layout_vxattrs.sh 2024-09-11T22:05:16.985 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 16.6 2024-09-11T22:05:16.985 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:05:17.258 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:05:17.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:17 smithi161 ceph-mon[35615]: pgmap v248: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 57 KiB/s wr, 123 op/s 2024-09-11T22:05:17.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:17 smithi001 ceph-mon[30935]: pgmap v248: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 57 KiB/s wr, 123 op/s 2024-09-11T22:05:17.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:17 smithi053 ceph-mon[34908]: pgmap v248: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 57 KiB/s wr, 123 op/s 2024-09-11T22:05:17.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:17.750+0000 7fd734c0e640 1 -- 172.21.15.1:0/1826111095 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd730103660 msgr2=0x7fd73010bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:17.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:17.750+0000 7fd734c0e640 1 --2- 172.21.15.1:0/1826111095 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd730103660 0x7fd73010bb50 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fd718009900 tx=0x7fd71802a480 comp rx=0 tx=0).stop 2024-09-11T22:05:17.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:17.750+0000 7fd734c0e640 1 -- 172.21.15.1:0/1826111095 shutdown_connections 2024-09-11T22:05:17.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:17.750+0000 7fd734c0e640 1 --2- 172.21.15.1:0/1826111095 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd73010c170 0x7fd73010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:17.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:17.750+0000 7fd734c0e640 1 --2- 172.21.15.1:0/1826111095 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd730103660 0x7fd73010bb50 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:17.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:17.750+0000 7fd734c0e640 1 --2- 172.21.15.1:0/1826111095 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd730102d40 0x7fd730103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:17.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:17.750+0000 7fd734c0e640 1 -- 172.21.15.1:0/1826111095 >> 172.21.15.1:0/1826111095 conn(0x7fd7300fc7a0 msgr2=0x7fd7300febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:17.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:17.750+0000 7fd734c0e640 1 -- 172.21.15.1:0/1826111095 shutdown_connections 2024-09-11T22:05:17.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:17.750+0000 7fd734c0e640 1 -- 172.21.15.1:0/1826111095 wait complete. 2024-09-11T22:05:17.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:17.751+0000 7fd734c0e640 1 Processor -- start 2024-09-11T22:05:17.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:17.751+0000 7fd734c0e640 1 -- start start 2024-09-11T22:05:17.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:17.751+0000 7fd734c0e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd730102d40 0x7fd730198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:17.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:17.751+0000 7fd734c0e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd730103660 0x7fd730198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:17.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:17.751+0000 7fd734c0e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd73010c170 0x7fd73019cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:17.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:17.751+0000 7fd734c0e640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd730111f70 con 0x7fd730102d40 2024-09-11T22:05:17.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:17.751+0000 7fd734c0e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd730111df0 con 0x7fd730103660 2024-09-11T22:05:17.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:17.751+0000 7fd734c0e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd7301120f0 con 0x7fd73010c170 2024-09-11T22:05:17.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:17.751+0000 7fd72e59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd730102d40 0x7fd730198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:17.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:17.751+0000 7fd72e59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd730102d40 0x7fd730198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49644/0 (socket says 172.21.15.1:49644) 2024-09-11T22:05:17.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:17.751+0000 7fd72e59c640 1 -- 172.21.15.1:0/2125453448 learned_addr learned my addr 172.21.15.1:0/2125453448 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:17.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:17.751+0000 7fd72ed9d640 1 --2- 172.21.15.1:0/2125453448 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd73010c170 0x7fd73019cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:17.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:17.752+0000 7fd725d9b640 1 --2- 172.21.15.1:0/2125453448 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd730103660 0x7fd730198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:17.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:17.752+0000 7fd72ed9d640 1 -- 172.21.15.1:0/2125453448 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd730103660 msgr2=0x7fd730198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:17.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:17.752+0000 7fd72ed9d640 1 --2- 172.21.15.1:0/2125453448 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd730103660 0x7fd730198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:17.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:17.752+0000 7fd72ed9d640 1 -- 172.21.15.1:0/2125453448 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd730102d40 msgr2=0x7fd730198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:17.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:17.752+0000 7fd72ed9d640 1 --2- 172.21.15.1:0/2125453448 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd730102d40 0x7fd730198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:17.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:17.752+0000 7fd72ed9d640 1 -- 172.21.15.1:0/2125453448 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd73019d400 con 0x7fd73010c170 2024-09-11T22:05:17.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:17.752+0000 7fd72e59c640 1 --2- 172.21.15.1:0/2125453448 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd730102d40 0x7fd730198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:05:17.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:17.752+0000 7fd72ed9d640 1 --2- 172.21.15.1:0/2125453448 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd73010c170 0x7fd73019cd20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd72800ca30 tx=0x7fd72800cef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:17.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:17.752+0000 7fd727fff640 1 -- 172.21.15.1:0/2125453448 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd728007d90 con 0x7fd73010c170 2024-09-11T22:05:17.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:17.752+0000 7fd734c0e640 1 -- 172.21.15.1:0/2125453448 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd73019d6f0 con 0x7fd73010c170 2024-09-11T22:05:17.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:17.753+0000 7fd727fff640 1 -- 172.21.15.1:0/2125453448 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd728005020 con 0x7fd73010c170 2024-09-11T22:05:17.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:17.753+0000 7fd727fff640 1 -- 172.21.15.1:0/2125453448 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd72800f780 con 0x7fd73010c170 2024-09-11T22:05:17.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:17.753+0000 7fd734c0e640 1 -- 172.21.15.1:0/2125453448 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7301a4fd0 con 0x7fd73010c170 2024-09-11T22:05:17.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:17.754+0000 7fd734c0e640 1 -- 172.21.15.1:0/2125453448 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd730103120 con 0x7fd73010c170 2024-09-11T22:05:17.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:17.755+0000 7fd727fff640 1 -- 172.21.15.1:0/2125453448 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7fd7280051d0 con 0x7fd73010c170 2024-09-11T22:05:17.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:17.756+0000 7fd727fff640 1 --2- 172.21.15.1:0/2125453448 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fd71c0a2cf0 0x7fd71c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:17.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:17.756+0000 7fd72e59c640 1 --2- 172.21.15.1:0/2125453448 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fd71c0a2cf0 0x7fd71c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:17.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:17.756+0000 7fd727fff640 1 -- 172.21.15.1:0/2125453448 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7fd7280cfa60 con 0x7fd73010c170 2024-09-11T22:05:17.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:17.756+0000 7fd72e59c640 1 --2- 172.21.15.1:0/2125453448 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fd71c0a2cf0 0x7fd71c0a51b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd720005fd0 tx=0x7fd72000b480 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:17.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:17.759+0000 7fd727fff640 1 -- 172.21.15.1:0/2125453448 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd728014070 con 0x7fd73010c170 2024-09-11T22:05:17.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:17.973+0000 7fd734c0e640 1 -- 172.21.15.1:0/2125453448 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd7301997b0 con 0x7fd73010c170 2024-09-11T22:05:17.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:17.974+0000 7fd727fff640 1 -- 172.21.15.1:0/2125453448 <== mon.1 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11984 (secure 0 0 0) 0x7fd7301997b0 con 0x7fd73010c170 2024-09-11T22:05:17.975 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T22:05:17.975 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":42,"btime":"2024-09-11T22:03:27:040852+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/421542832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":421542832},{"type":"v1","addr":"172.21.15.1:6835","nonce":421542832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2836864406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2836864406},{"type":"v1","addr":"172.21.15.1:6837","nonce":2836864406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":42,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T21:55:20.791263+0000","modified":"2024-09-11T22:03:26.485134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24484,"mds_2":24467,"mds_3":24466,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14595":{"gid":14595,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/407834672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":407834672},{"type":"v1","addr":"172.21.15.1:6839","nonce":407834672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/1068827678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":1068827678},{"type":"v1","addr":"172.21.15.1:6841","nonce":1068827678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/872410024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":872410024},{"type":"v1","addr":"172.21.15.161:6833","nonce":872410024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/1648944570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1648944570},{"type":"v1","addr":"172.21.15.53:6833","nonce":1648944570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/2126933993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2126933993},{"type":"v1","addr":"172.21.15.53:6835","nonce":2126933993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.161:6835/828381592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":828381592},{"type":"v1","addr":"172.21.15.161:6835","nonce":828381592}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.161:6837/1799318656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1799318656},{"type":"v1","addr":"172.21.15.161:6837","nonce":1799318656}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.53:6837/3457131830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3457131830},{"type":"v1","addr":"172.21.15.53:6837","nonce":3457131830}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/455365383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":455365383},{"type":"v1","addr":"172.21.15.161:6839","nonce":455365383}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.53:6839/1378584447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1378584447},{"type":"v1","addr":"172.21.15.53:6839","nonce":1378584447}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24460,24466,24467,24484,24494]},"id":1}]} 2024-09-11T22:05:17.976 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 42 2024-09-11T22:05:17.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:17.976+0000 7fd734c0e640 1 -- 172.21.15.1:0/2125453448 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fd71c0a2cf0 msgr2=0x7fd71c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:17.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:17.976+0000 7fd734c0e640 1 --2- 172.21.15.1:0/2125453448 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fd71c0a2cf0 0x7fd71c0a51b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd720005fd0 tx=0x7fd72000b480 comp rx=0 tx=0).stop 2024-09-11T22:05:17.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:17.976+0000 7fd734c0e640 1 -- 172.21.15.1:0/2125453448 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd73010c170 msgr2=0x7fd73019cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:17.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:17.976+0000 7fd734c0e640 1 --2- 172.21.15.1:0/2125453448 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd73010c170 0x7fd73019cd20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd72800ca30 tx=0x7fd72800cef0 comp rx=0 tx=0).stop 2024-09-11T22:05:17.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:17.976+0000 7fd734c0e640 1 -- 172.21.15.1:0/2125453448 shutdown_connections 2024-09-11T22:05:17.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:17.976+0000 7fd734c0e640 1 --2- 172.21.15.1:0/2125453448 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fd71c0a2cf0 0x7fd71c0a51b0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:17.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:17.976+0000 7fd734c0e640 1 --2- 172.21.15.1:0/2125453448 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd73010c170 0x7fd73019cd20 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:17.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:17.976+0000 7fd734c0e640 1 --2- 172.21.15.1:0/2125453448 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd730103660 0x7fd730198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:17.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:17.976+0000 7fd734c0e640 1 --2- 172.21.15.1:0/2125453448 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd730102d40 0x7fd730198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:17.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:17.976+0000 7fd734c0e640 1 -- 172.21.15.1:0/2125453448 >> 172.21.15.1:0/2125453448 conn(0x7fd7300fc7a0 msgr2=0x7fd7300fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:17.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:17.976+0000 7fd734c0e640 1 -- 172.21.15.1:0/2125453448 shutdown_connections 2024-09-11T22:05:17.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:17.976+0000 7fd734c0e640 1 -- 172.21.15.1:0/2125453448 wait complete. 2024-09-11T22:05:18.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:18 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/2125453448' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:05:18.348 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:18 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/2125453448' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:05:18.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:18 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/2125453448' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:05:18.782 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db / --timeout 90 --expiration 136.6 --await-for 45.0' 2024-09-11T22:05:18.782 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db / --timeout 90 --expiration 136.6 --await-for 45.0 2024-09-11T22:05:19.048 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:05:19.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:19 smithi161 ceph-mon[35615]: pgmap v249: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 102 op/s 2024-09-11T22:05:19.351 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:19 smithi001 ceph-mon[30935]: pgmap v249: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 102 op/s 2024-09-11T22:05:19.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:19 smithi053 ceph-mon[34908]: pgmap v249: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 102 op/s 2024-09-11T22:05:19.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.613+0000 7f4a0778e640 1 -- 172.21.15.1:0/4008585560 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4a0010e480 msgr2=0x7f4a00110870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:19.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.613+0000 7f4a0778e640 1 --2- 172.21.15.1:0/4008585560 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4a0010e480 0x7f4a00110870 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f49f8009ca0 tx=0x7f49f802a8c0 comp rx=0 tx=0).stop 2024-09-11T22:05:19.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.613+0000 7f4a0778e640 1 -- 172.21.15.1:0/4008585560 shutdown_connections 2024-09-11T22:05:19.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.613+0000 7f4a0778e640 1 --2- 172.21.15.1:0/4008585560 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4a0010e480 0x7f4a00110870 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:19.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.613+0000 7f4a0778e640 1 --2- 172.21.15.1:0/4008585560 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a00101280 0x7f4a0010df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:19.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.613+0000 7f4a0778e640 1 --2- 172.21.15.1:0/4008585560 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a001008d0 0x7f4a00100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:19.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.613+0000 7f4a0778e640 1 -- 172.21.15.1:0/4008585560 >> 172.21.15.1:0/4008585560 conn(0x7f4a000fc7a0 msgr2=0x7f4a000febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:19.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.613+0000 7f4a0778e640 1 -- 172.21.15.1:0/4008585560 shutdown_connections 2024-09-11T22:05:19.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.613+0000 7f4a0778e640 1 -- 172.21.15.1:0/4008585560 wait complete. 2024-09-11T22:05:19.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.614+0000 7f4a0778e640 1 Processor -- start 2024-09-11T22:05:19.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.614+0000 7f4a0778e640 1 -- start start 2024-09-11T22:05:19.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.614+0000 7f4a0778e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a001008d0 0x7f4a0019a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:19.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.614+0000 7f4a0778e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a00101280 0x7f4a0019aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:19.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.614+0000 7f4a0778e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4a0010e480 0x7f4a0019ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:19.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.614+0000 7f4a0778e640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4a00114160 con 0x7f4a0010e480 2024-09-11T22:05:19.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.614+0000 7f4a0778e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4a00113fe0 con 0x7f4a00101280 2024-09-11T22:05:19.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.614+0000 7f4a0778e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4a001142e0 con 0x7f4a001008d0 2024-09-11T22:05:19.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.614+0000 7f4a05d2b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4a0010e480 0x7f4a0019ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:19.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.614+0000 7f4a05d2b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4a0010e480 0x7f4a0019ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49662/0 (socket says 172.21.15.1:49662) 2024-09-11T22:05:19.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.614+0000 7f4a05d2b640 1 -- 172.21.15.1:0/2152084951 learned_addr learned my addr 172.21.15.1:0/2152084951 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:19.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.615+0000 7f4a0552a640 1 --2- 172.21.15.1:0/2152084951 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a001008d0 0x7f4a0019a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:19.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.615+0000 7f4a04d29640 1 --2- 172.21.15.1:0/2152084951 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a00101280 0x7f4a0019aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:19.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.615+0000 7f4a05d2b640 1 -- 172.21.15.1:0/2152084951 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a001008d0 msgr2=0x7f4a0019a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:19.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.615+0000 7f4a05d2b640 1 --2- 172.21.15.1:0/2152084951 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a001008d0 0x7f4a0019a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:19.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.615+0000 7f4a05d2b640 1 -- 172.21.15.1:0/2152084951 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a00101280 msgr2=0x7f4a0019aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:19.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.615+0000 7f4a05d2b640 1 --2- 172.21.15.1:0/2152084951 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a00101280 0x7f4a0019aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:19.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.615+0000 7f4a05d2b640 1 -- 172.21.15.1:0/2152084951 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a0019f610 con 0x7f4a0010e480 2024-09-11T22:05:19.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.615+0000 7f4a04d29640 1 --2- 172.21.15.1:0/2152084951 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a00101280 0x7f4a0019aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:05:19.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.615+0000 7f4a05d2b640 1 --2- 172.21.15.1:0/2152084951 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4a0010e480 0x7f4a0019ef30 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f49f802ade0 tx=0x7f49f80319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:19.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.615+0000 7f49f67fc640 1 -- 172.21.15.1:0/2152084951 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f8003c30 con 0x7f4a0010e480 2024-09-11T22:05:19.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.615+0000 7f4a0778e640 1 -- 172.21.15.1:0/2152084951 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a0019f8a0 con 0x7f4a0010e480 2024-09-11T22:05:19.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.615+0000 7f49f67fc640 1 -- 172.21.15.1:0/2152084951 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49f8003dd0 con 0x7f4a0010e480 2024-09-11T22:05:19.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.615+0000 7f49f67fc640 1 -- 172.21.15.1:0/2152084951 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f802fd60 con 0x7f4a0010e480 2024-09-11T22:05:19.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.615+0000 7f4a0778e640 1 -- 172.21.15.1:0/2152084951 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a001a71e0 con 0x7f4a0010e480 2024-09-11T22:05:19.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.618+0000 7f49f67fc640 1 -- 172.21.15.1:0/2152084951 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f49f8004050 con 0x7f4a0010e480 2024-09-11T22:05:19.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.618+0000 7f4a09b67740 1 Processor -- start 2024-09-11T22:05:19.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.618+0000 7f4a09b67740 1 -- start start 2024-09-11T22:05:19.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.618+0000 7f4a09b67740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562834f86760 0x562834f86b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:19.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.618+0000 7f4a09b67740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562834f87080 0x562834f91400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:19.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.618+0000 7f4a09b67740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562834f91940 0x562834f93d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:19.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.618+0000 7f4a09b67740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x562834eec2a0 con 0x562834f91940 2024-09-11T22:05:19.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.618+0000 7f4a09b67740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x562834e93b10 con 0x562834f86760 2024-09-11T22:05:19.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.618+0000 7f4a09b67740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x562834ee8260 con 0x562834f87080 2024-09-11T22:05:19.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.619+0000 7f49f67fc640 1 --2- 172.21.15.1:0/2152084951 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f49ec0a2cf0 0x7f49ec0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:19.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.619+0000 7f4a04d29640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562834f91940 0x562834f93d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:19.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.619+0000 7f4a04d29640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562834f91940 0x562834f93d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49666/0 (socket says 172.21.15.1:49666) 2024-09-11T22:05:19.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.619+0000 7f4a04d29640 1 -- 172.21.15.1:0/4136530748 learned_addr learned my addr 172.21.15.1:0/4136530748 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:19.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.619+0000 7f4a0552a640 1 --2- 172.21.15.1:0/4136530748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562834f87080 0x562834f91400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:19.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.619+0000 7f4a05d2b640 1 --2- 172.21.15.1:0/2152084951 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f49ec0a2cf0 0x7f49ec0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:19.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.619+0000 7f49f67fc640 1 -- 172.21.15.1:0/2152084951 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7f49f80f08a0 con 0x7f4a0010e480 2024-09-11T22:05:19.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.619+0000 7f4a04d29640 1 --2- 172.21.15.1:0/4136530748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562834f86760 0x562834f86b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:19.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.619+0000 7f4a0552a640 1 -- 172.21.15.1:0/4136530748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562834f86760 msgr2=0x562834f86b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:19.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.619+0000 7f4a0552a640 1 --2- 172.21.15.1:0/4136530748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562834f86760 0x562834f86b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:19.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.619+0000 7f4a0552a640 1 -- 172.21.15.1:0/4136530748 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562834f91940 msgr2=0x562834f93d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:19.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.619+0000 7f4a0552a640 1 --2- 172.21.15.1:0/4136530748 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562834f91940 0x562834f93d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:19.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.619+0000 7f4a0552a640 1 -- 172.21.15.1:0/4136530748 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562834c5db90 con 0x562834f87080 2024-09-11T22:05:19.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.619+0000 7f4a05d2b640 1 --2- 172.21.15.1:0/2152084951 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f49ec0a2cf0 0x7f49ec0a51b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f49f80bcd10 tx=0x7f49f808f0f0 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:19.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.619+0000 7f4a0552a640 1 --2- 172.21.15.1:0/4136530748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562834f87080 0x562834f91400 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f49fc00ab40 tx=0x7f49fc00f3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e1f3978cf71412e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:19.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.619+0000 7f49eb7fe640 1 -- 172.21.15.1:0/4136530748 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49fc004020 con 0x562834f87080 2024-09-11T22:05:19.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.620+0000 7f49eb7fe640 1 -- 172.21.15.1:0/4136530748 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49fc0041c0 con 0x562834f87080 2024-09-11T22:05:19.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.620+0000 7f49eb7fe640 1 -- 172.21.15.1:0/4136530748 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49fc018780 con 0x562834f87080 2024-09-11T22:05:19.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.620+0000 7f4a09b67740 1 -- 172.21.15.1:0/4136530748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562834f87080 msgr2=0x562834f91400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:19.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.620+0000 7f4a09b67740 1 --2- 172.21.15.1:0/4136530748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562834f87080 0x562834f91400 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f49fc00ab40 tx=0x7f49fc00f3c0 comp rx=0 tx=0).stop 2024-09-11T22:05:19.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.620+0000 7f4a09b67740 1 -- 172.21.15.1:0/4136530748 shutdown_connections 2024-09-11T22:05:19.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.620+0000 7f4a09b67740 1 --2- 172.21.15.1:0/4136530748 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562834f91940 0x562834f93d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:19.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.620+0000 7f4a09b67740 1 --2- 172.21.15.1:0/4136530748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562834f87080 0x562834f91400 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:19.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.620+0000 7f4a09b67740 1 --2- 172.21.15.1:0/4136530748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562834f86760 0x562834f86b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:19.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.620+0000 7f4a09b67740 1 -- 172.21.15.1:0/4136530748 >> 172.21.15.1:0/4136530748 conn(0x562834f82cf0 msgr2=0x562834f830d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:19.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.620+0000 7f4a09b67740 1 -- 172.21.15.1:0/4136530748 shutdown_connections 2024-09-11T22:05:19.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.620+0000 7f4a09b67740 1 -- 172.21.15.1:0/4136530748 wait complete. 2024-09-11T22:05:19.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.621+0000 7f4a09b67740 1 -- start start 2024-09-11T22:05:19.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.621+0000 7f4a09b67740 1 Processor -- start 2024-09-11T22:05:19.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.621+0000 7f4a09b67740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562834f86760 0x562834f8e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:19.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.621+0000 7f4a09b67740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562834f87080 0x562834f8ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:19.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.621+0000 7f4a09b67740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562834f91940 0x562834f8f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:19.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.621+0000 7f4a09b67740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x562834f94510 con 0x562834f91940 2024-09-11T22:05:19.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.621+0000 7f4a09b67740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x562834e746e0 con 0x562834f86760 2024-09-11T22:05:19.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.621+0000 7f4a09b67740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x562834f94690 con 0x562834f87080 2024-09-11T22:05:19.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.621+0000 7f4a09b67740 10 client.0 _ll_register_callbacks cb 0x562834e98a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:05:19.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.621+0000 7f4a0552a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562834f87080 0x562834f8ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:19.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.621+0000 7f4a04d29640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562834f91940 0x562834f8f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:19.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.621+0000 7f4a0552a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562834f87080 0x562834f8ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:44802/0 (socket says 172.21.15.1:44802) 2024-09-11T22:05:19.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.622+0000 7f4a0552a640 1 -- 172.21.15.1:0/1122095951 learned_addr learned my addr 172.21.15.1:0/1122095951 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:19.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.622+0000 7f4a04d29640 1 --2- 172.21.15.1:0/1122095951 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562834f86760 0x562834f8e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:19.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.622+0000 7f4a0552a640 1 -- 172.21.15.1:0/1122095951 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562834f86760 msgr2=0x562834f8e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:19.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.622+0000 7f4a0552a640 1 --2- 172.21.15.1:0/1122095951 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562834f86760 0x562834f8e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:19.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.622+0000 7f4a0552a640 1 -- 172.21.15.1:0/1122095951 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562834f91940 msgr2=0x562834f8f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:19.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.622+0000 7f4a0552a640 1 --2- 172.21.15.1:0/1122095951 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562834f91940 0x562834f8f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:19.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.622+0000 7f4a0552a640 1 -- 172.21.15.1:0/1122095951 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562834f9d9d0 con 0x562834f87080 2024-09-11T22:05:19.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.622+0000 7f4a04d29640 1 --2- 172.21.15.1:0/1122095951 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562834f91940 0x562834f8f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:05:19.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.622+0000 7f4a0552a640 1 --2- 172.21.15.1:0/1122095951 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562834f87080 0x562834f8ed20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f49fc00a740 tx=0x7f49fc003e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1259cdbe423e490b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:19.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.622+0000 7f49e8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.161:3300/0 2024-09-11T22:05:19.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.622+0000 7f49e8ff9640 1 -- 172.21.15.1:0/1122095951 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49fc027020 con 0x562834f87080 2024-09-11T22:05:19.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.622+0000 7f4a09b67740 1 -- 172.21.15.1:0/1122095951 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49fc0041c0 con 0x562834f87080 2024-09-11T22:05:19.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.622+0000 7f49e8ff9640 1 -- 172.21.15.1:0/1122095951 <== mon.1 v2:172.21.15.161:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7f49fc004370 con 0x562834f87080 2024-09-11T22:05:19.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.623+0000 7f49e8ff9640 1 -- 172.21.15.1:0/1122095951 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f49fc032050 con 0x562834f87080 2024-09-11T22:05:19.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.623+0000 7f4a09b67740 10 client.26701 fetch_fsmap learned FSMap version 42 2024-09-11T22:05:19.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.623+0000 7f4a09b67740 1 -- 172.21.15.1:0/1122095951 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x562834fa0340 con 0x562834f87080 2024-09-11T22:05:19.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.623+0000 7f49e8ff9640 1 -- 172.21.15.1:0/1122095951 <== mon.1 v2:172.21.15.161:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49fc0041c0 con 0x562834f87080 2024-09-11T22:05:19.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.623+0000 7f49e8ff9640 1 -- 172.21.15.1:0/1122095951 <== mon.1 v2:172.21.15.161:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f49fc004a90 con 0x562834f87080 2024-09-11T22:05:19.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.623+0000 7f4a09b67740 10 client.26701 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:05:19.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.623+0000 7f4a09b67740 10 client.26701 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-11T22:05:19.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.623+0000 7f4a09b67740 20 client.26701 populate_metadata read hostname 'smithi001' 2024-09-11T22:05:19.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.623+0000 7f4a09b67740 1 --2- 172.21.15.1:0/1122095951 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x562834fa1750 0x562834fa3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:19.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.623+0000 7f4a09b67740 4 client.26701 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:05:19.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.623+0000 7f4a09b67740 1 -- 172.21.15.1:0/1122095951 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562834ef99d0 con 0x562834fa1750 2024-09-11T22:05:19.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.624+0000 7f4a04d29640 1 --2- 172.21.15.1:0/1122095951 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x562834fa1750 0x562834fa3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:19.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.624+0000 7f49e8ff9640 10 client.26701 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:05:19.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.624+0000 7f4a04d29640 1 --2- 172.21.15.1:0/1122095951 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x562834fa1750 0x562834fa3b40 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3a184bd13a0e1ac server_cookie=56888b9062e4a8a1 in_seq=0 out_seq=0 2024-09-11T22:05:19.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.625+0000 7f49e8ff9640 1 -- 172.21.15.1:0/1122095951 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562834ef99d0 con 0x562834fa1750 2024-09-11T22:05:19.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.625+0000 7f49e8ff9640 10 client.26701 handle_command_reply: tid=0 2024-09-11T22:05:19.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.625+0000 7f4a09b67740 1 client.26701 shutdown 2024-09-11T22:05:19.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.625+0000 7f4a09b67740 1 -- 172.21.15.1:0/1122095951 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562834f87080 msgr2=0x562834f8ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:19.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.625+0000 7f4a09b67740 1 --2- 172.21.15.1:0/1122095951 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562834f87080 0x562834f8ed20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f49fc00a740 tx=0x7f49fc003e10 comp rx=0 tx=0).stop 2024-09-11T22:05:19.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.625+0000 7f4a09b67740 1 -- 172.21.15.1:0/1122095951 shutdown_connections 2024-09-11T22:05:19.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.625+0000 7f4a09b67740 1 --2- 172.21.15.1:0/1122095951 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x562834fa1750 0x562834fa3b40 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:19.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.625+0000 7f4a09b67740 1 --2- 172.21.15.1:0/1122095951 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562834f91940 0x562834f8f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:19.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.625+0000 7f4a09b67740 1 --2- 172.21.15.1:0/1122095951 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562834f87080 0x562834f8ed20 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:19.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.625+0000 7f49e8ff9640 0 client.26701 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:05:19.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.625+0000 7f4a09b67740 1 --2- 172.21.15.1:0/1122095951 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562834f86760 0x562834f8e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:19.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.625+0000 7f4a09b67740 1 -- 172.21.15.1:0/1122095951 >> 172.21.15.1:0/1122095951 conn(0x562834f82cf0 msgr2=0x562834f81e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:19.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.625+0000 7f4a09b67740 1 -- 172.21.15.1:0/1122095951 shutdown_connections 2024-09-11T22:05:19.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.625+0000 7f4a09b67740 1 -- 172.21.15.1:0/1122095951 wait complete. 2024-09-11T22:05:19.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.625+0000 7f4a09b67740 20 client.26701 trim_cache size 0 max 16384 2024-09-11T22:05:19.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.642+0000 7f4a09b67740 1 Processor -- start 2024-09-11T22:05:19.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.643+0000 7f4a09b67740 1 -- start start 2024-09-11T22:05:19.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.643+0000 7f4a09b67740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562834f86760 0x562834fa92b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:19.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.643+0000 7f4a09b67740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562834f87080 0x562834fa97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:19.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.643+0000 7f4a09b67740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562834f91940 0x562834f8ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:19.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.643+0000 7f4a09b67740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4a001afcc0 con 0x562834f91940 2024-09-11T22:05:19.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.643+0000 7f4a09b67740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4a001b1b00 con 0x562834f86760 2024-09-11T22:05:19.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.643+0000 7f4a09b67740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4a001b0450 con 0x562834f87080 2024-09-11T22:05:19.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.643+0000 7f4a04d29640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562834f91940 0x562834f8ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:19.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.643+0000 7f4a04d29640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562834f91940 0x562834f8ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49688/0 (socket says 172.21.15.1:49688) 2024-09-11T22:05:19.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.643+0000 7f4a04d29640 1 -- 172.21.15.1:0/2791428014 learned_addr learned my addr 172.21.15.1:0/2791428014 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:19.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.643+0000 7f4a0552a640 1 --2- 172.21.15.1:0/2791428014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562834f87080 0x562834fa97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:19.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.643+0000 7f4a04d29640 1 --2- 172.21.15.1:0/2791428014 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562834f86760 0x562834fa92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:19.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.643+0000 7f4a04d29640 1 -- 172.21.15.1:0/2791428014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562834f87080 msgr2=0x562834fa97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:19.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.643+0000 7f4a04d29640 1 --2- 172.21.15.1:0/2791428014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562834f87080 0x562834fa97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:19.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.643+0000 7f4a04d29640 1 -- 172.21.15.1:0/2791428014 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562834f86760 msgr2=0x562834fa92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:19.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.643+0000 7f4a04d29640 1 --2- 172.21.15.1:0/2791428014 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562834f86760 0x562834fa92b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:19.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.643+0000 7f4a04d29640 1 -- 172.21.15.1:0/2791428014 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562834f895d0 con 0x562834f91940 2024-09-11T22:05:19.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.643+0000 7f4a0552a640 1 --2- 172.21.15.1:0/2791428014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562834f87080 0x562834fa97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:05:19.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.644+0000 7f4a04d29640 1 --2- 172.21.15.1:0/2791428014 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562834f91940 0x562834f8ba40 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x562834f84070 tx=0x7f49f0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a64d86a63e5295c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:19.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.644+0000 7f49eaffd640 1 -- 172.21.15.1:0/2791428014 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f0027070 con 0x562834f91940 2024-09-11T22:05:19.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.644+0000 7f49eaffd640 1 -- 172.21.15.1:0/2791428014 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49f0013230 con 0x562834f91940 2024-09-11T22:05:19.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.644+0000 7f49eaffd640 1 -- 172.21.15.1:0/2791428014 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f001c040 con 0x562834f91940 2024-09-11T22:05:19.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.644+0000 7f4a09b67740 1 -- 172.21.15.1:0/2791428014 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562834f91940 msgr2=0x562834f8ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:19.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.644+0000 7f4a09b67740 1 --2- 172.21.15.1:0/2791428014 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562834f91940 0x562834f8ba40 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x562834f84070 tx=0x7f49f0004730 comp rx=0 tx=0).stop 2024-09-11T22:05:19.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.644+0000 7f4a09b67740 1 -- 172.21.15.1:0/2791428014 shutdown_connections 2024-09-11T22:05:19.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.644+0000 7f4a09b67740 1 --2- 172.21.15.1:0/2791428014 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562834f91940 0x562834f8ba40 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:19.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.644+0000 7f4a09b67740 1 --2- 172.21.15.1:0/2791428014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562834f87080 0x562834fa97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:19.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.644+0000 7f4a09b67740 1 --2- 172.21.15.1:0/2791428014 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562834f86760 0x562834fa92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:19.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.644+0000 7f4a09b67740 1 -- 172.21.15.1:0/2791428014 >> 172.21.15.1:0/2791428014 conn(0x562834f82cf0 msgr2=0x562834f8d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:19.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.644+0000 7f4a09b67740 1 -- 172.21.15.1:0/2791428014 shutdown_connections 2024-09-11T22:05:19.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.644+0000 7f4a09b67740 1 -- 172.21.15.1:0/2791428014 wait complete. 2024-09-11T22:05:19.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.645+0000 7f4a09b67740 1 -- start start 2024-09-11T22:05:19.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.645+0000 7f4a09b67740 1 Processor -- start 2024-09-11T22:05:19.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.645+0000 7f4a09b67740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562834f86760 0x562834f959a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:19.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.645+0000 7f4a09b67740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562834f87080 0x562834f95ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:19.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.645+0000 7f4a09b67740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562834f91940 0x56283500b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:19.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.645+0000 7f4a09b67740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4a001b1370 con 0x562834f87080 2024-09-11T22:05:19.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.645+0000 7f4a09b67740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4a001b0be0 con 0x562834f86760 2024-09-11T22:05:19.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.645+0000 7f4a09b67740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4a001adcc0 con 0x562834f91940 2024-09-11T22:05:19.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.645+0000 7f4a09b67740 10 client.0 _ll_register_callbacks cb 0x7f49fc003990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:05:19.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.645+0000 7f4a0552a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562834f87080 0x562834f95ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:19.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.645+0000 7f4a0552a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562834f87080 0x562834f95ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49702/0 (socket says 172.21.15.1:49702) 2024-09-11T22:05:19.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.645+0000 7f4a0552a640 1 -- 172.21.15.1:0/692883631 learned_addr learned my addr 172.21.15.1:0/692883631 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:19.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.646+0000 7f4a04d29640 1 --2- 172.21.15.1:0/692883631 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562834f86760 0x562834f959a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:19.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.646+0000 7f4a04d29640 1 --2- 172.21.15.1:0/692883631 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562834f91940 0x56283500b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:19.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.646+0000 7f4a0552a640 1 -- 172.21.15.1:0/692883631 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562834f91940 msgr2=0x56283500b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:19.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.646+0000 7f4a0552a640 1 --2- 172.21.15.1:0/692883631 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562834f91940 0x56283500b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:19.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.646+0000 7f4a0552a640 1 -- 172.21.15.1:0/692883631 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562834f86760 msgr2=0x562834f959a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:19.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.646+0000 7f4a0552a640 1 --2- 172.21.15.1:0/692883631 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562834f86760 0x562834f959a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:19.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.646+0000 7f4a0552a640 1 -- 172.21.15.1:0/692883631 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562834f967a0 con 0x562834f87080 2024-09-11T22:05:19.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.646+0000 7f4a04d29640 1 --2- 172.21.15.1:0/692883631 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562834f86760 0x562834f959a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:05:19.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.646+0000 7f4a0552a640 1 --2- 172.21.15.1:0/692883631 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562834f87080 0x562834f95ee0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f49fc0058c0 tx=0x7f49fc018720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9dc03c66f8b0a6d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:19.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.646+0000 7f49e8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:05:19.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.646+0000 7f49e8ff9640 1 -- 172.21.15.1:0/692883631 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49fc027040 con 0x562834f87080 2024-09-11T22:05:19.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.646+0000 7f4a09b67740 1 -- 172.21.15.1:0/692883631 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49f0013230 con 0x562834f87080 2024-09-11T22:05:19.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.646+0000 7f49e8ff9640 1 -- 172.21.15.1:0/692883631 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7f49fc004380 con 0x562834f87080 2024-09-11T22:05:19.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.646+0000 7f49e8ff9640 1 -- 172.21.15.1:0/692883631 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f49fc00f8c0 con 0x562834f87080 2024-09-11T22:05:19.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.646+0000 7f4a09b67740 10 client.17292 fetch_fsmap learned FSMap version 42 2024-09-11T22:05:19.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.646+0000 7f4a09b67740 1 -- 172.21.15.1:0/692883631 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x56283500c8a0 con 0x562834f87080 2024-09-11T22:05:19.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.647+0000 7f49e8ff9640 1 -- 172.21.15.1:0/692883631 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49f0013230 con 0x562834f87080 2024-09-11T22:05:19.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.647+0000 7f49e8ff9640 1 -- 172.21.15.1:0/692883631 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f49fc004f80 con 0x562834f87080 2024-09-11T22:05:19.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.647+0000 7f4a09b67740 10 client.17292 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:05:19.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.647+0000 7f4a09b67740 10 client.17292 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-11T22:05:19.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.647+0000 7f4a09b67740 20 client.17292 populate_metadata read hostname 'smithi001' 2024-09-11T22:05:19.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.647+0000 7f4a09b67740 1 --2- 172.21.15.1:0/692883631 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x562834fa1750 0x56283500f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:19.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.647+0000 7f4a09b67740 4 client.17292 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 136.6, "await_for": 45.0}] 2024-09-11T22:05:19.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.647+0000 7f4a09b67740 1 -- 172.21.15.1:0/692883631 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 136.6, "await_for": 45.0}) -- 0x562834ef3d20 con 0x562834fa1750 2024-09-11T22:05:19.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.647+0000 7f4a04d29640 1 --2- 172.21.15.1:0/692883631 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x562834fa1750 0x56283500f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:19.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.647+0000 7f49e8ff9640 10 client.17292 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:05:19.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:19.647+0000 7f4a04d29640 1 --2- 172.21.15.1:0/692883631 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x562834fa1750 0x56283500f0c0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf60267ca5803ab1 server_cookie=ea0d1ee05e9c974 in_seq=0 out_seq=0 2024-09-11T22:05:20.219 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.217+0000 7ff4080b7640 1 -- 172.21.15.1:0/3028550162 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff40010e460 msgr2=0x7ff400110850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:20.220 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.217+0000 7ff4080b7640 1 --2- 172.21.15.1:0/3028550162 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff40010e460 0x7ff400110850 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7ff3f8009ca0 tx=0x7ff3f802a8c0 comp rx=0 tx=0).stop 2024-09-11T22:05:20.220 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.217+0000 7ff4080b7640 1 -- 172.21.15.1:0/3028550162 shutdown_connections 2024-09-11T22:05:20.220 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.217+0000 7ff4080b7640 1 --2- 172.21.15.1:0/3028550162 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff40010e460 0x7ff400110850 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:20.220 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.217+0000 7ff4080b7640 1 --2- 172.21.15.1:0/3028550162 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff400101260 0x7ff40010de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:20.220 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.217+0000 7ff4080b7640 1 --2- 172.21.15.1:0/3028550162 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4001008b0 0x7ff400100c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:20.220 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.217+0000 7ff4080b7640 1 -- 172.21.15.1:0/3028550162 >> 172.21.15.1:0/3028550162 conn(0x7ff4000fc6b0 msgr2=0x7ff4000fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:20.220 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.217+0000 7ff4080b7640 1 -- 172.21.15.1:0/3028550162 shutdown_connections 2024-09-11T22:05:20.220 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.217+0000 7ff4080b7640 1 -- 172.21.15.1:0/3028550162 wait complete. 2024-09-11T22:05:20.220 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.217+0000 7ff4080b7640 1 Processor -- start 2024-09-11T22:05:20.221 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.218+0000 7ff4080b7640 1 -- start start 2024-09-11T22:05:20.221 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.218+0000 7ff4080b7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff4001008b0 0x7ff40019a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:20.221 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.218+0000 7ff4080b7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff400101260 0x7ff40019aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:20.221 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.218+0000 7ff4080b7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff40010e460 0x7ff40019edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:20.221 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.218+0000 7ff4080b7640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff4001140c0 con 0x7ff4001008b0 2024-09-11T22:05:20.221 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.218+0000 7ff4080b7640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff400113f40 con 0x7ff40010e460 2024-09-11T22:05:20.221 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.218+0000 7ff4080b7640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff400114240 con 0x7ff400101260 2024-09-11T22:05:20.221 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.218+0000 7ff405e53640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff4001008b0 0x7ff40019a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:20.221 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.218+0000 7ff405e53640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff4001008b0 0x7ff40019a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49724/0 (socket says 172.21.15.1:49724) 2024-09-11T22:05:20.221 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.218+0000 7ff405e53640 1 -- 172.21.15.1:0/4073749638 learned_addr learned my addr 172.21.15.1:0/4073749638 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:20.221 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.218+0000 7ff406654640 1 --2- 172.21.15.1:0/4073749638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff40010e460 0x7ff40019edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:20.221 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.218+0000 7ff405652640 1 --2- 172.21.15.1:0/4073749638 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff400101260 0x7ff40019aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:20.221 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.218+0000 7ff405e53640 1 -- 172.21.15.1:0/4073749638 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff400101260 msgr2=0x7ff40019aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:20.222 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.218+0000 7ff405e53640 1 --2- 172.21.15.1:0/4073749638 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff400101260 0x7ff40019aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:20.222 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.218+0000 7ff405e53640 1 -- 172.21.15.1:0/4073749638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff40010e460 msgr2=0x7ff40019edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:20.222 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.218+0000 7ff405e53640 1 --2- 172.21.15.1:0/4073749638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff40010e460 0x7ff40019edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:20.222 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.218+0000 7ff405e53640 1 -- 172.21.15.1:0/4073749638 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff40019f520 con 0x7ff4001008b0 2024-09-11T22:05:20.222 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.218+0000 7ff405e53640 1 --2- 172.21.15.1:0/4073749638 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff4001008b0 0x7ff40019a530 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc00d770 tx=0x7ff3fc00db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:20.222 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.219+0000 7ff3f6ffd640 1 -- 172.21.15.1:0/4073749638 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3fc017070 con 0x7ff4001008b0 2024-09-11T22:05:20.222 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.219+0000 7ff3f6ffd640 1 -- 172.21.15.1:0/4073749638 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3fc009820 con 0x7ff4001008b0 2024-09-11T22:05:20.222 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.219+0000 7ff4080b7640 1 -- 172.21.15.1:0/4073749638 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff40019f810 con 0x7ff4001008b0 2024-09-11T22:05:20.222 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.219+0000 7ff3f6ffd640 1 -- 172.21.15.1:0/4073749638 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3fc004cd0 con 0x7ff4001008b0 2024-09-11T22:05:20.222 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.219+0000 7ff4080b7640 1 -- 172.21.15.1:0/4073749638 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4001a70a0 con 0x7ff4001008b0 2024-09-11T22:05:20.229 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.221+0000 7ff3f6ffd640 1 -- 172.21.15.1:0/4073749638 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7ff3fc0099d0 con 0x7ff4001008b0 2024-09-11T22:05:20.229 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.221+0000 7ff40ae50740 1 Processor -- start 2024-09-11T22:05:20.229 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.222+0000 7ff40ae50740 1 -- start start 2024-09-11T22:05:20.229 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.222+0000 7ff40ae50740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b72ecabaf0 0x55b72ecabed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:20.229 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.222+0000 7ff3f6ffd640 1 --2- 172.21.15.1:0/4073749638 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7ff3ec0a2d70 0x7ff3ec0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:20.229 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.222+0000 7ff40ae50740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b72ecac410 0x55b72ecb6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:20.229 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.222+0000 7ff40ae50740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b72ecb6cb0 0x55b72ecb90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:20.229 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.222+0000 7ff40ae50740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55b72ec102f0 con 0x55b72ecabaf0 2024-09-11T22:05:20.229 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.222+0000 7ff40ae50740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55b72ebb8270 con 0x55b72ecac410 2024-09-11T22:05:20.230 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.222+0000 7ff40ae50740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b72ec0c2b0 con 0x55b72ecb6cb0 2024-09-11T22:05:20.230 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.222+0000 7ff406654640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b72ecabaf0 0x55b72ecabed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:20.230 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.222+0000 7ff406654640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b72ecabaf0 0x55b72ecabed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49726/0 (socket says 172.21.15.1:49726) 2024-09-11T22:05:20.230 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.222+0000 7ff405e53640 1 --2- 172.21.15.1:0/4073749638 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7ff3ec0a2d70 0x7ff3ec0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:20.230 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.222+0000 7ff406654640 1 -- 172.21.15.1:0/2384555423 learned_addr learned my addr 172.21.15.1:0/2384555423 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:20.230 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.222+0000 7ff405652640 1 --2- 172.21.15.1:0/2384555423 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b72ecac410 0x55b72ecb6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:20.230 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.222+0000 7ff3f6ffd640 1 -- 172.21.15.1:0/4073749638 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7ff3fc0cee30 con 0x7ff4001008b0 2024-09-11T22:05:20.230 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.222+0000 7ff406654640 1 --2- 172.21.15.1:0/2384555423 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b72ecb6cb0 0x55b72ecb90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:20.230 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.222+0000 7ff405652640 1 -- 172.21.15.1:0/2384555423 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b72ecb6cb0 msgr2=0x55b72ecb90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:20.230 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.222+0000 7ff405652640 1 --2- 172.21.15.1:0/2384555423 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b72ecb6cb0 0x55b72ecb90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:20.230 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.222+0000 7ff405652640 1 -- 172.21.15.1:0/2384555423 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b72ecabaf0 msgr2=0x55b72ecabed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:20.231 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.222+0000 7ff405652640 1 --2- 172.21.15.1:0/2384555423 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b72ecabaf0 0x55b72ecabed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:20.231 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.222+0000 7ff405652640 1 -- 172.21.15.1:0/2384555423 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b72ebb7340 con 0x55b72ecac410 2024-09-11T22:05:20.231 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.222+0000 7ff406654640 1 --2- 172.21.15.1:0/2384555423 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b72ecb6cb0 0x55b72ecb90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:05:20.231 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.222+0000 7ff405e53640 1 --2- 172.21.15.1:0/4073749638 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7ff3ec0a2d70 0x7ff3ec0a5230 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc0cc570 tx=0x7ff3fc0bb040 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:20.231 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.223+0000 7ff405652640 1 --2- 172.21.15.1:0/2384555423 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b72ecac410 0x55b72ecb6770 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff3f000ab40 tx=0x7ff3f000f3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f54285606f7ebd1b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:20.231 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.223+0000 7ff3ebfff640 1 -- 172.21.15.1:0/2384555423 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f0005020 con 0x55b72ecac410 2024-09-11T22:05:20.231 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.223+0000 7ff3ebfff640 1 -- 172.21.15.1:0/2384555423 <== mon.2 v2:172.21.15.53:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3f00047a0 con 0x55b72ecac410 2024-09-11T22:05:20.231 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.223+0000 7ff3ebfff640 1 -- 172.21.15.1:0/2384555423 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f0018840 con 0x55b72ecac410 2024-09-11T22:05:20.231 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.223+0000 7ff40ae50740 1 -- 172.21.15.1:0/2384555423 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b72ecac410 msgr2=0x55b72ecb6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:20.231 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.223+0000 7ff40ae50740 1 --2- 172.21.15.1:0/2384555423 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b72ecac410 0x55b72ecb6770 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff3f000ab40 tx=0x7ff3f000f3c0 comp rx=0 tx=0).stop 2024-09-11T22:05:20.231 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.223+0000 7ff40ae50740 1 -- 172.21.15.1:0/2384555423 shutdown_connections 2024-09-11T22:05:20.231 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.223+0000 7ff40ae50740 1 --2- 172.21.15.1:0/2384555423 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b72ecb6cb0 0x55b72ecb90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:20.231 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.223+0000 7ff40ae50740 1 --2- 172.21.15.1:0/2384555423 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b72ecac410 0x55b72ecb6770 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:20.232 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.223+0000 7ff40ae50740 1 --2- 172.21.15.1:0/2384555423 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b72ecabaf0 0x55b72ecabed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:20.232 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.224+0000 7ff40ae50740 1 -- 172.21.15.1:0/2384555423 >> 172.21.15.1:0/2384555423 conn(0x55b72eca8060 msgr2=0x55b72eca8440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:20.232 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.224+0000 7ff40ae50740 1 -- 172.21.15.1:0/2384555423 shutdown_connections 2024-09-11T22:05:20.232 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.224+0000 7ff40ae50740 1 -- 172.21.15.1:0/2384555423 wait complete. 2024-09-11T22:05:20.232 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.224+0000 7ff40ae50740 1 -- start start 2024-09-11T22:05:20.232 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.224+0000 7ff40ae50740 1 Processor -- start 2024-09-11T22:05:20.232 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.224+0000 7ff40ae50740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b72ecabaf0 0x55b72ecb3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:20.232 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.224+0000 7ff40ae50740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b72ecac410 0x55b72ecb4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:20.232 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.224+0000 7ff40ae50740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b72ecb6cb0 0x55b72ecb4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:20.232 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.224+0000 7ff40ae50740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55b72ecb9880 con 0x55b72ecac410 2024-09-11T22:05:20.232 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.224+0000 7ff40ae50740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55b72eb99290 con 0x55b72ecabaf0 2024-09-11T22:05:20.232 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.224+0000 7ff40ae50740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b72ecb9a00 con 0x55b72ecb6cb0 2024-09-11T22:05:20.233 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.224+0000 7ff40ae50740 10 client.0 _ll_register_callbacks cb 0x55b72ebca140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:05:20.233 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.225+0000 7ff405652640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b72ecac410 0x55b72ecb4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:20.233 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.225+0000 7ff406654640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b72ecabaf0 0x55b72ecb3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:20.233 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.225+0000 7ff405652640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b72ecac410 0x55b72ecb4030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49732/0 (socket says 172.21.15.1:49732) 2024-09-11T22:05:20.233 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.225+0000 7ff405652640 1 -- 172.21.15.1:0/904118071 learned_addr learned my addr 172.21.15.1:0/904118071 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:20.233 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.225+0000 7ff406654640 1 --2- 172.21.15.1:0/904118071 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b72ecb6cb0 0x55b72ecb4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:20.233 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.225+0000 7ff405652640 1 -- 172.21.15.1:0/904118071 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b72ecb6cb0 msgr2=0x55b72ecb4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:20.233 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.225+0000 7ff405652640 1 --2- 172.21.15.1:0/904118071 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b72ecb6cb0 0x55b72ecb4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:20.233 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.225+0000 7ff405652640 1 -- 172.21.15.1:0/904118071 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b72ecabaf0 msgr2=0x55b72ecb3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:20.233 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.225+0000 7ff405652640 1 --2- 172.21.15.1:0/904118071 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b72ecabaf0 0x55b72ecb3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:20.233 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.225+0000 7ff405652640 1 -- 172.21.15.1:0/904118071 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b72e923210 con 0x55b72ecac410 2024-09-11T22:05:20.233 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.225+0000 7ff406654640 1 --2- 172.21.15.1:0/904118071 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b72ecabaf0 0x55b72ecb3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:05:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.225+0000 7ff405652640 1 --2- 172.21.15.1:0/904118071 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b72ecac410 0x55b72ecb4030 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7ff3f001a040 tx=0x7ff3f0005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=145aafa7ca1a1518 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.225+0000 7ff3e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:05:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.225+0000 7ff3e97fa640 1 -- 172.21.15.1:0/904118071 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f0027070 con 0x55b72ecac410 2024-09-11T22:05:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.225+0000 7ff3e97fa640 1 -- 172.21.15.1:0/904118071 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7ff3f00051b0 con 0x55b72ecac410 2024-09-11T22:05:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.225+0000 7ff40ae50740 1 -- 172.21.15.1:0/904118071 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3f00047a0 con 0x55b72ecac410 2024-09-11T22:05:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.225+0000 7ff3e97fa640 1 -- 172.21.15.1:0/904118071 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7ff3f0033050 con 0x55b72ecac410 2024-09-11T22:05:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.225+0000 7ff40ae50740 10 client.17304 fetch_fsmap learned FSMap version 42 2024-09-11T22:05:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.225+0000 7ff40ae50740 1 -- 172.21.15.1:0/904118071 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55b72e9ef620 con 0x55b72ecac410 2024-09-11T22:05:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.226+0000 7ff3e97fa640 1 -- 172.21.15.1:0/904118071 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3f00047a0 con 0x55b72ecac410 2024-09-11T22:05:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.226+0000 7ff3e97fa640 1 -- 172.21.15.1:0/904118071 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7ff3f0023430 con 0x55b72ecac410 2024-09-11T22:05:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.226+0000 7ff40ae50740 10 client.17304 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:05:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.226+0000 7ff40ae50740 10 client.17304 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:05:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.226+0000 7ff40ae50740 20 client.17304 populate_metadata read hostname 'smithi001' 2024-09-11T22:05:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.226+0000 7ff40ae50740 1 --2- 172.21.15.1:0/904118071 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55b72ecc6a70 0x55b72ecc8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.226+0000 7ff40ae50740 4 client.17304 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:05:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.226+0000 7ff40ae50740 1 -- 172.21.15.1:0/904118071 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b72ec1da20 con 0x55b72ecc6a70 2024-09-11T22:05:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.226+0000 7ff406654640 1 --2- 172.21.15.1:0/904118071 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55b72ecc6a70 0x55b72ecc8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.227+0000 7ff3e97fa640 10 client.17304 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:05:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.227+0000 7ff406654640 1 --2- 172.21.15.1:0/904118071 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55b72ecc6a70 0x55b72ecc8e60 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47caab56f6445c63 server_cookie=f32950df6e578aaa in_seq=0 out_seq=0 2024-09-11T22:05:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.227+0000 7ff3e97fa640 1 -- 172.21.15.1:0/904118071 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b72ec1da20 con 0x55b72ecc6a70 2024-09-11T22:05:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.227+0000 7ff3e97fa640 10 client.17304 handle_command_reply: tid=0 2024-09-11T22:05:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.227+0000 7ff40ae50740 1 client.17304 shutdown 2024-09-11T22:05:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.227+0000 7ff40ae50740 1 -- 172.21.15.1:0/904118071 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b72ecac410 msgr2=0x55b72ecb4030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.227+0000 7ff40ae50740 1 --2- 172.21.15.1:0/904118071 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b72ecac410 0x55b72ecb4030 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7ff3f001a040 tx=0x7ff3f0005850 comp rx=0 tx=0).stop 2024-09-11T22:05:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.227+0000 7ff40ae50740 1 -- 172.21.15.1:0/904118071 shutdown_connections 2024-09-11T22:05:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.227+0000 7ff40ae50740 1 --2- 172.21.15.1:0/904118071 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55b72ecc6a70 0x55b72ecc8e60 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.228+0000 7ff40ae50740 1 --2- 172.21.15.1:0/904118071 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b72ecb6cb0 0x55b72ecb4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.228+0000 7ff40ae50740 1 --2- 172.21.15.1:0/904118071 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b72ecac410 0x55b72ecb4030 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.228+0000 7ff3e97fa640 0 client.17304 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:05:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.228+0000 7ff40ae50740 1 --2- 172.21.15.1:0/904118071 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b72ecabaf0 0x55b72ecb3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.228+0000 7ff40ae50740 1 -- 172.21.15.1:0/904118071 >> 172.21.15.1:0/904118071 conn(0x55b72eca8060 msgr2=0x55b72ecaddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.228+0000 7ff40ae50740 1 -- 172.21.15.1:0/904118071 shutdown_connections 2024-09-11T22:05:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.228+0000 7ff40ae50740 1 -- 172.21.15.1:0/904118071 wait complete. 2024-09-11T22:05:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.228+0000 7ff40ae50740 20 client.17304 trim_cache size 0 max 16384 2024-09-11T22:05:20.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.242+0000 7ff40ae50740 1 Processor -- start 2024-09-11T22:05:20.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.242+0000 7ff40ae50740 1 -- start start 2024-09-11T22:05:20.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.242+0000 7ff40ae50740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b72ecabaf0 0x55b72ecb43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:20.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.242+0000 7ff40ae50740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b72ecac410 0x55b72ecce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:20.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.242+0000 7ff40ae50740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b72ecb6cb0 0x55b72eccea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:20.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.242+0000 7ff40ae50740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff4001afc90 con 0x55b72ecac410 2024-09-11T22:05:20.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.242+0000 7ff40ae50740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff4001b1ad0 con 0x55b72ecb6cb0 2024-09-11T22:05:20.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.242+0000 7ff40ae50740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff4001b0420 con 0x55b72ecabaf0 2024-09-11T22:05:20.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.242+0000 7ff405652640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b72ecac410 0x55b72ecce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:20.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.242+0000 7ff405652640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b72ecac410 0x55b72ecce510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49748/0 (socket says 172.21.15.1:49748) 2024-09-11T22:05:20.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.242+0000 7ff405652640 1 -- 172.21.15.1:0/3258672745 learned_addr learned my addr 172.21.15.1:0/3258672745 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:20.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.242+0000 7ff406654640 1 --2- 172.21.15.1:0/3258672745 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b72ecabaf0 0x55b72ecb43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:20.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.242+0000 7ff406654640 1 --2- 172.21.15.1:0/3258672745 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b72ecb6cb0 0x55b72eccea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:20.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.243+0000 7ff405652640 1 -- 172.21.15.1:0/3258672745 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b72ecabaf0 msgr2=0x55b72ecb43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:20.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.243+0000 7ff405652640 1 --2- 172.21.15.1:0/3258672745 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b72ecabaf0 0x55b72ecb43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:20.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.243+0000 7ff405652640 1 -- 172.21.15.1:0/3258672745 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b72ecb6cb0 msgr2=0x55b72eccea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:20.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.243+0000 7ff405652640 1 --2- 172.21.15.1:0/3258672745 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b72ecb6cb0 0x55b72eccea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:20.249 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.243+0000 7ff405652640 1 -- 172.21.15.1:0/3258672745 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b72eca7e90 con 0x55b72ecac410 2024-09-11T22:05:20.249 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.243+0000 7ff406654640 1 --2- 172.21.15.1:0/3258672745 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b72ecabaf0 0x55b72ecb43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:05:20.249 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.243+0000 7ff405652640 1 --2- 172.21.15.1:0/3258672745 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b72ecac410 0x55b72ecce510 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7ff3f0010010 tx=0x7ff3f0005900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70d6a617f50c06ed server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:20.249 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.243+0000 7ff3eb7fe640 1 -- 172.21.15.1:0/3258672745 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f0027070 con 0x55b72ecac410 2024-09-11T22:05:20.249 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.243+0000 7ff3eb7fe640 1 -- 172.21.15.1:0/3258672745 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3f000a9c0 con 0x55b72ecac410 2024-09-11T22:05:20.249 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.243+0000 7ff3eb7fe640 1 -- 172.21.15.1:0/3258672745 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f002c440 con 0x55b72ecac410 2024-09-11T22:05:20.249 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.243+0000 7ff40ae50740 1 -- 172.21.15.1:0/3258672745 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b72ecac410 msgr2=0x55b72ecce510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:20.249 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.243+0000 7ff40ae50740 1 --2- 172.21.15.1:0/3258672745 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b72ecac410 0x55b72ecce510 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7ff3f0010010 tx=0x7ff3f0005900 comp rx=0 tx=0).stop 2024-09-11T22:05:20.249 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.243+0000 7ff40ae50740 1 -- 172.21.15.1:0/3258672745 shutdown_connections 2024-09-11T22:05:20.249 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.243+0000 7ff40ae50740 1 --2- 172.21.15.1:0/3258672745 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b72ecb6cb0 0x55b72eccea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:20.249 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.243+0000 7ff40ae50740 1 --2- 172.21.15.1:0/3258672745 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b72ecac410 0x55b72ecce510 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:20.249 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.243+0000 7ff40ae50740 1 --2- 172.21.15.1:0/3258672745 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b72ecabaf0 0x55b72ecb43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:20.250 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.243+0000 7ff40ae50740 1 -- 172.21.15.1:0/3258672745 >> 172.21.15.1:0/3258672745 conn(0x55b72eca8060 msgr2=0x55b72ecb5b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:20.250 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.244+0000 7ff40ae50740 1 -- 172.21.15.1:0/3258672745 shutdown_connections 2024-09-11T22:05:20.250 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.244+0000 7ff40ae50740 1 -- 172.21.15.1:0/3258672745 wait complete. 2024-09-11T22:05:20.250 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.244+0000 7ff40ae50740 1 -- start start 2024-09-11T22:05:20.250 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.244+0000 7ff40ae50740 1 Processor -- start 2024-09-11T22:05:20.250 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.244+0000 7ff40ae50740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b72ecabaf0 0x55b72ecbf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:20.250 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.244+0000 7ff40ae50740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b72ecac410 0x55b72ecba570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:20.250 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.244+0000 7ff40ae50740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b72ecb6cb0 0x55b72ecbaab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:20.250 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.244+0000 7ff40ae50740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff4001b1340 con 0x55b72ecac410 2024-09-11T22:05:20.250 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.244+0000 7ff40ae50740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff4001b0bb0 con 0x55b72ecabaf0 2024-09-11T22:05:20.250 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.244+0000 7ff40ae50740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff4001adc90 con 0x55b72ecb6cb0 2024-09-11T22:05:20.250 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.244+0000 7ff40ae50740 10 client.0 _ll_register_callbacks cb 0x7ff3f8005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:05:20.250 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.244+0000 7ff405652640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b72ecac410 0x55b72ecba570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:20.251 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.244+0000 7ff405652640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b72ecac410 0x55b72ecba570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49760/0 (socket says 172.21.15.1:49760) 2024-09-11T22:05:20.251 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.244+0000 7ff405652640 1 -- 172.21.15.1:0/2843726187 learned_addr learned my addr 172.21.15.1:0/2843726187 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:20.251 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.245+0000 7ff406654640 1 --2- 172.21.15.1:0/2843726187 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b72ecabaf0 0x55b72ecbf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:20.251 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.245+0000 7ff406654640 1 --2- 172.21.15.1:0/2843726187 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b72ecb6cb0 0x55b72ecbaab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:20.251 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.245+0000 7ff405652640 1 -- 172.21.15.1:0/2843726187 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b72ecb6cb0 msgr2=0x55b72ecbaab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:20.251 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.245+0000 7ff405652640 1 --2- 172.21.15.1:0/2843726187 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b72ecb6cb0 0x55b72ecbaab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:20.251 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.245+0000 7ff405652640 1 -- 172.21.15.1:0/2843726187 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b72ecabaf0 msgr2=0x55b72ecbf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:20.251 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.245+0000 7ff405652640 1 --2- 172.21.15.1:0/2843726187 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b72ecabaf0 0x55b72ecbf250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:20.251 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.245+0000 7ff405652640 1 -- 172.21.15.1:0/2843726187 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b72ecbb0f0 con 0x55b72ecac410 2024-09-11T22:05:20.251 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.245+0000 7ff406654640 1 --2- 172.21.15.1:0/2843726187 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b72ecb6cb0 0x55b72ecbaab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:05:20.251 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.245+0000 7ff405652640 1 --2- 172.21.15.1:0/2843726187 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b72ecac410 0x55b72ecba570 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7ff3f000f8c0 tx=0x7ff3f0023670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=547854f4fe434b10 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:20.251 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.245+0000 7ff3e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:05:20.252 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.245+0000 7ff3e97fa640 1 -- 172.21.15.1:0/2843726187 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f0027040 con 0x55b72ecac410 2024-09-11T22:05:20.252 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.245+0000 7ff3e97fa640 1 -- 172.21.15.1:0/2843726187 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7ff3f00053b0 con 0x55b72ecac410 2024-09-11T22:05:20.252 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.245+0000 7ff40ae50740 1 -- 172.21.15.1:0/2843726187 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3f000a9c0 con 0x55b72ecac410 2024-09-11T22:05:20.252 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.245+0000 7ff3e97fa640 1 -- 172.21.15.1:0/2843726187 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7ff3f0023c80 con 0x55b72ecac410 2024-09-11T22:05:20.252 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.245+0000 7ff40ae50740 10 client.17310 fetch_fsmap learned FSMap version 42 2024-09-11T22:05:20.252 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.245+0000 7ff40ae50740 1 -- 172.21.15.1:0/2843726187 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55b72ed31270 con 0x55b72ecac410 2024-09-11T22:05:20.252 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.245+0000 7ff3e97fa640 1 -- 172.21.15.1:0/2843726187 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3f000a9c0 con 0x55b72ecac410 2024-09-11T22:05:20.252 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.245+0000 7ff3e97fa640 1 -- 172.21.15.1:0/2843726187 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7ff3f0042500 con 0x55b72ecac410 2024-09-11T22:05:20.252 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.246+0000 7ff40ae50740 10 client.17310 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:05:20.252 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.246+0000 7ff40ae50740 10 client.17310 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:05:20.252 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.246+0000 7ff40ae50740 20 client.17310 populate_metadata read hostname 'smithi001' 2024-09-11T22:05:20.252 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.246+0000 7ff40ae50740 1 --2- 172.21.15.1:0/2843726187 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55b72ed31a10 0x55b72ed33e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:20.252 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.246+0000 7ff40ae50740 4 client.17310 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:05:20.253 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.246+0000 7ff40ae50740 1 -- 172.21.15.1:0/2843726187 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b72ec17d70 con 0x55b72ed31a10 2024-09-11T22:05:20.253 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.246+0000 7ff406654640 1 --2- 172.21.15.1:0/2843726187 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55b72ed31a10 0x55b72ed33e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:20.253 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.246+0000 7ff406654640 1 --2- 172.21.15.1:0/2843726187 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55b72ed31a10 0x55b72ed33e30 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58c90fa48d2114b8 server_cookie=4bce583500c9f2ad in_seq=0 out_seq=0 2024-09-11T22:05:20.253 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.246+0000 7ff3e97fa640 10 client.17310 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:05:20.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:20.253+0000 7f49e8ff9640 1 -- 172.21.15.1:0/692883631 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x562834ef3d20 con 0x562834fa1750 2024-09-11T22:05:20.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:20.253+0000 7f49e8ff9640 10 client.17292 handle_command_reply: tid=0 2024-09-11T22:05:20.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:20.253+0000 7f4a09b67740 1 client.17292 shutdown 2024-09-11T22:05:20.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:20.253+0000 7f4a09b67740 1 -- 172.21.15.1:0/692883631 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562834f87080 msgr2=0x562834f95ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:20.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:20.253+0000 7f4a09b67740 1 --2- 172.21.15.1:0/692883631 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562834f87080 0x562834f95ee0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f49fc0058c0 tx=0x7f49fc018720 comp rx=0 tx=0).stop 2024-09-11T22:05:20.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:20.253+0000 7f4a09b67740 1 -- 172.21.15.1:0/692883631 shutdown_connections 2024-09-11T22:05:20.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:20.253+0000 7f4a09b67740 1 --2- 172.21.15.1:0/692883631 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x562834fa1750 0x56283500f0c0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:20.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:20.253+0000 7f4a09b67740 1 --2- 172.21.15.1:0/692883631 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x562834f91940 0x56283500b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:20.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:20.253+0000 7f49e8ff9640 0 client.17292 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:05:20.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:20.253+0000 7f4a09b67740 1 --2- 172.21.15.1:0/692883631 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562834f87080 0x562834f95ee0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:20.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:20.253+0000 7f4a09b67740 1 --2- 172.21.15.1:0/692883631 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562834f86760 0x562834f959a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:20.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:20.253+0000 7f4a09b67740 1 -- 172.21.15.1:0/692883631 >> 172.21.15.1:0/692883631 conn(0x562834f82cf0 msgr2=0x562834f881b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:20.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:20.253+0000 7f4a09b67740 1 -- 172.21.15.1:0/692883631 shutdown_connections 2024-09-11T22:05:20.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:20.253+0000 7f4a09b67740 1 -- 172.21.15.1:0/692883631 wait complete. 2024-09-11T22:05:20.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:20.253+0000 7f4a09b67740 20 client.17292 trim_cache size 0 max 16384 2024-09-11T22:05:20.255 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-11T22:05:20.255 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 42, 2024-09-11T22:05:20.255 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24494, 2024-09-11T22:05:20.255 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 26, 2024-09-11T22:05:20.255 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-09-11T22:05:20.255 INFO:teuthology.orchestra.run.smithi001.stdout: "f8d2564c": { 2024-09-11T22:05:20.256 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 26, 2024-09-11T22:05:20.256 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-09-11T22:05:20.256 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-11T22:05:20.256 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-09-11T22:05:20.256 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-11T22:05:20.256 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:05:20.256 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 90.0, 2024-09-11T22:05:20.256 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 136.6, 2024-09-11T22:05:20.256 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-09-11T22:05:20.256 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-09-11T22:05:20.256 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-09-11T22:05:20.256 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-11T22:05:20.256 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-09-11T22:05:20.257 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-11T22:05:20.257 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:05:20.257 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:05:20.257 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:05:20.257 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:05:20.257 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:05:20.257 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-11T22:05:20.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:20.254+0000 7f4a0778e640 1 -- 172.21.15.1:0/2152084951 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f49ec0a2cf0 msgr2=0x7f49ec0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:20.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:20.254+0000 7f4a0778e640 1 --2- 172.21.15.1:0/2152084951 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f49ec0a2cf0 0x7f49ec0a51b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f49f80bcd10 tx=0x7f49f808f0f0 comp rx=0 tx=0).stop 2024-09-11T22:05:20.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:20.254+0000 7f4a0778e640 1 -- 172.21.15.1:0/2152084951 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4a0010e480 msgr2=0x7f4a0019ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:20.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:20.254+0000 7f4a0778e640 1 --2- 172.21.15.1:0/2152084951 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4a0010e480 0x7f4a0019ef30 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f49f802ade0 tx=0x7f49f80319f0 comp rx=0 tx=0).stop 2024-09-11T22:05:20.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:20.254+0000 7f4a0778e640 1 -- 172.21.15.1:0/2152084951 shutdown_connections 2024-09-11T22:05:20.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:20.254+0000 7f4a0778e640 1 --2- 172.21.15.1:0/2152084951 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f49ec0a2cf0 0x7f49ec0a51b0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:20.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:20.254+0000 7f4a0778e640 1 --2- 172.21.15.1:0/2152084951 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4a0010e480 0x7f4a0019ef30 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:20.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:20.254+0000 7f4a0778e640 1 --2- 172.21.15.1:0/2152084951 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a00101280 0x7f4a0019aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:20.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:20.254+0000 7f4a0778e640 1 --2- 172.21.15.1:0/2152084951 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a001008d0 0x7f4a0019a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:20.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:20.254+0000 7f4a0778e640 1 -- 172.21.15.1:0/2152084951 >> 172.21.15.1:0/2152084951 conn(0x7f4a000fc7a0 msgr2=0x7f4a000feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:20.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:20.254+0000 7f4a0778e640 1 -- 172.21.15.1:0/2152084951 shutdown_connections 2024-09-11T22:05:20.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:20.254+0000 7f4a0778e640 1 -- 172.21.15.1:0/2152084951 wait complete. 2024-09-11T22:05:20.270 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.269+0000 7ff3e97fa640 1 -- 172.21.15.1:0/2843726187 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b72ec17d70 con 0x55b72ed31a10 2024-09-11T22:05:20.270 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.269+0000 7ff3e97fa640 10 client.17310 handle_command_reply: tid=0 2024-09-11T22:05:20.271 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.269+0000 7ff40ae50740 1 client.17310 shutdown 2024-09-11T22:05:20.271 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.269+0000 7ff40ae50740 1 -- 172.21.15.1:0/2843726187 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b72ecac410 msgr2=0x55b72ecba570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:20.271 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.269+0000 7ff40ae50740 1 --2- 172.21.15.1:0/2843726187 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b72ecac410 0x55b72ecba570 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7ff3f000f8c0 tx=0x7ff3f0023670 comp rx=0 tx=0).stop 2024-09-11T22:05:20.271 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.269+0000 7ff40ae50740 1 -- 172.21.15.1:0/2843726187 shutdown_connections 2024-09-11T22:05:20.271 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.269+0000 7ff40ae50740 1 --2- 172.21.15.1:0/2843726187 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55b72ed31a10 0x55b72ed33e30 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:20.271 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.269+0000 7ff40ae50740 1 --2- 172.21.15.1:0/2843726187 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b72ecb6cb0 0x55b72ecbaab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:20.271 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.269+0000 7ff40ae50740 1 --2- 172.21.15.1:0/2843726187 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b72ecac410 0x55b72ecba570 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:20.271 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.270+0000 7ff3e97fa640 0 client.17310 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:05:20.271 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.269+0000 7ff40ae50740 1 --2- 172.21.15.1:0/2843726187 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b72ecabaf0 0x55b72ecbf250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:20.271 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.270+0000 7ff40ae50740 1 -- 172.21.15.1:0/2843726187 >> 172.21.15.1:0/2843726187 conn(0x55b72eca8060 msgr2=0x55b72eca69a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:20.271 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.270+0000 7ff40ae50740 1 -- 172.21.15.1:0/2843726187 shutdown_connections 2024-09-11T22:05:20.271 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.270+0000 7ff40ae50740 1 -- 172.21.15.1:0/2843726187 wait complete. 2024-09-11T22:05:20.271 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.270+0000 7ff40ae50740 20 client.17310 trim_cache size 0 max 16384 2024-09-11T22:05:20.272 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-09-11T22:05:20.272 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-09-11T22:05:20.272 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-09-11T22:05:20.272 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-09-11T22:05:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.270+0000 7ff4080b7640 1 -- 172.21.15.1:0/4073749638 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7ff3ec0a2d70 msgr2=0x7ff3ec0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.270+0000 7ff4080b7640 1 --2- 172.21.15.1:0/4073749638 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7ff3ec0a2d70 0x7ff3ec0a5230 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc0cc570 tx=0x7ff3fc0bb040 comp rx=0 tx=0).stop 2024-09-11T22:05:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.270+0000 7ff4080b7640 1 -- 172.21.15.1:0/4073749638 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff4001008b0 msgr2=0x7ff40019a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.270+0000 7ff4080b7640 1 --2- 172.21.15.1:0/4073749638 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff4001008b0 0x7ff40019a530 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc00d770 tx=0x7ff3fc00db70 comp rx=0 tx=0).stop 2024-09-11T22:05:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.270+0000 7ff4080b7640 1 -- 172.21.15.1:0/4073749638 shutdown_connections 2024-09-11T22:05:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.270+0000 7ff4080b7640 1 --2- 172.21.15.1:0/4073749638 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7ff3ec0a2d70 0x7ff3ec0a5230 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.270+0000 7ff4080b7640 1 --2- 172.21.15.1:0/4073749638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff40010e460 0x7ff40019edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.270+0000 7ff4080b7640 1 --2- 172.21.15.1:0/4073749638 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff400101260 0x7ff40019aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.270+0000 7ff4080b7640 1 --2- 172.21.15.1:0/4073749638 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff4001008b0 0x7ff40019a530 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:20.274 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.270+0000 7ff4080b7640 1 -- 172.21.15.1:0/4073749638 >> 172.21.15.1:0/4073749638 conn(0x7ff4000fc6b0 msgr2=0x7ff4000feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:20.274 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.271+0000 7ff4080b7640 1 -- 172.21.15.1:0/4073749638 shutdown_connections 2024-09-11T22:05:20.274 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:20.271+0000 7ff4080b7640 1 -- 172.21.15.1:0/4073749638 wait complete. 2024-09-11T22:05:21.052 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'f8d2564c', quiesce took 4.1 seconds. Will release after: 12.500000000000002 2024-09-11T22:05:21.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:21 smithi161 ceph-mon[35615]: pgmap v250: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 109 op/s 2024-09-11T22:05:21.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:21 smithi001 ceph-mon[30935]: pgmap v250: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 109 op/s 2024-09-11T22:05:21.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:21 smithi053 ceph-mon[34908]: pgmap v250: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 109 op/s 2024-09-11T22:05:23.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:23 smithi161 ceph-mon[35615]: pgmap v251: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-09-11T22:05:23.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:23 smithi001 ceph-mon[30935]: pgmap v251: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-09-11T22:05:23.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:23 smithi053 ceph-mon[34908]: pgmap v251: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-09-11T22:05:25.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:25 smithi161 ceph-mon[35615]: pgmap v252: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-09-11T22:05:25.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:25 smithi001 ceph-mon[30935]: pgmap v252: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-09-11T22:05:25.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:25 smithi053 ceph-mon[34908]: pgmap v252: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-09-11T22:05:27.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:27 smithi161 ceph-mon[35615]: pgmap v253: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-11T22:05:27.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:27 smithi001 ceph-mon[30935]: pgmap v253: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-11T22:05:27.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:27 smithi053 ceph-mon[34908]: pgmap v253: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-11T22:05:29.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:29 smithi161 ceph-mon[35615]: pgmap v254: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-09-11T22:05:29.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:29 smithi001 ceph-mon[30935]: pgmap v254: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-09-11T22:05:29.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:29 smithi053 ceph-mon[34908]: pgmap v254: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-09-11T22:05:31.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:31 smithi161 ceph-mon[35615]: pgmap v255: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 19 op/s 2024-09-11T22:05:31.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:31 smithi001 ceph-mon[30935]: pgmap v255: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 19 op/s 2024-09-11T22:05:31.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:31 smithi053 ceph-mon[34908]: pgmap v255: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 19 op/s 2024-09-11T22:05:33.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:33 smithi161 ceph-mon[35615]: pgmap v256: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-09-11T22:05:33.368 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.367+0000 7f15e9bfd640 1 -- 172.21.15.1:0/1719602665 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15e41069e0 msgr2=0x7f15e4106dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:33.368 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.367+0000 7f15e9bfd640 1 --2- 172.21.15.1:0/1719602665 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15e41069e0 0x7f15e4106dc0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f15d4009930 tx=0x7f15d402a420 comp rx=0 tx=0).stop 2024-09-11T22:05:33.368 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.367+0000 7f15e9bfd640 1 -- 172.21.15.1:0/1719602665 shutdown_connections 2024-09-11T22:05:33.369 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.367+0000 7f15e9bfd640 1 --2- 172.21.15.1:0/1719602665 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15e4101370 0x7f15e410d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:33.369 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.367+0000 7f15e9bfd640 1 --2- 172.21.15.1:0/1719602665 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15e41009d0 0x7f15e4100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:33.369 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.367+0000 7f15e9bfd640 1 --2- 172.21.15.1:0/1719602665 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15e41069e0 0x7f15e4106dc0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:33.369 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.367+0000 7f15e9bfd640 1 -- 172.21.15.1:0/1719602665 >> 172.21.15.1:0/1719602665 conn(0x7f15e40fc6b0 msgr2=0x7f15e40fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:33.369 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.367+0000 7f15e9bfd640 1 -- 172.21.15.1:0/1719602665 shutdown_connections 2024-09-11T22:05:33.369 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.367+0000 7f15e9bfd640 1 -- 172.21.15.1:0/1719602665 wait complete. 2024-09-11T22:05:33.369 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.368+0000 7f15e9bfd640 1 Processor -- start 2024-09-11T22:05:33.369 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.368+0000 7f15e9bfd640 1 -- start start 2024-09-11T22:05:33.371 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.368+0000 7f15e9bfd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15e41009d0 0x7f15e41972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:33.371 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.368+0000 7f15e9bfd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15e4101370 0x7f15e4197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:33.371 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.368+0000 7f15e9bfd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15e41069e0 0x7f15e419bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:33.371 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.368+0000 7f15e9bfd640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f15e4110e70 con 0x7f15e4101370 2024-09-11T22:05:33.371 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.368+0000 7f15e9bfd640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f15e4110cf0 con 0x7f15e41069e0 2024-09-11T22:05:33.371 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.368+0000 7f15e9bfd640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f15e4110ff0 con 0x7f15e41009d0 2024-09-11T22:05:33.371 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.368+0000 7f15daffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15e4101370 0x7f15e4197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:33.371 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.368+0000 7f15daffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15e4101370 0x7f15e4197820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42530/0 (socket says 172.21.15.1:42530) 2024-09-11T22:05:33.372 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.368+0000 7f15daffd640 1 -- 172.21.15.1:0/3152391803 learned_addr learned my addr 172.21.15.1:0/3152391803 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:33.372 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.368+0000 7f15e3fff640 1 --2- 172.21.15.1:0/3152391803 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15e41069e0 0x7f15e419bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:33.372 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.368+0000 7f15e37fe640 1 --2- 172.21.15.1:0/3152391803 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15e41009d0 0x7f15e41972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:33.372 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.369+0000 7f15daffd640 1 -- 172.21.15.1:0/3152391803 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15e41009d0 msgr2=0x7f15e41972e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:33.372 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.369+0000 7f15daffd640 1 --2- 172.21.15.1:0/3152391803 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15e41009d0 0x7f15e41972e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:33.372 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.369+0000 7f15daffd640 1 -- 172.21.15.1:0/3152391803 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15e41069e0 msgr2=0x7f15e419bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:33.372 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.369+0000 7f15daffd640 1 --2- 172.21.15.1:0/3152391803 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15e41069e0 0x7f15e419bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:33.372 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.369+0000 7f15daffd640 1 -- 172.21.15.1:0/3152391803 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15e419c2d0 con 0x7f15e4101370 2024-09-11T22:05:33.372 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.369+0000 7f15daffd640 1 --2- 172.21.15.1:0/3152391803 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15e4101370 0x7f15e4197820 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f15cc00c970 tx=0x7f15cc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:33.372 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.369+0000 7f15e17fa640 1 -- 172.21.15.1:0/3152391803 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15cc007bb0 con 0x7f15e4101370 2024-09-11T22:05:33.372 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.369+0000 7f15e17fa640 1 -- 172.21.15.1:0/3152391803 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15cc007d50 con 0x7f15e4101370 2024-09-11T22:05:33.372 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.369+0000 7f15e9bfd640 1 -- 172.21.15.1:0/3152391803 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15e419c5c0 con 0x7f15e4101370 2024-09-11T22:05:33.373 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.369+0000 7f15e17fa640 1 -- 172.21.15.1:0/3152391803 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15cc00f450 con 0x7f15e4101370 2024-09-11T22:05:33.373 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.369+0000 7f15e9bfd640 1 -- 172.21.15.1:0/3152391803 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15e41a3e50 con 0x7f15e4101370 2024-09-11T22:05:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.371+0000 7f15e17fa640 1 -- 172.21.15.1:0/3152391803 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f15cc005020 con 0x7f15e4101370 2024-09-11T22:05:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.372+0000 7f15eca80740 1 Processor -- start 2024-09-11T22:05:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.372+0000 7f15eca80740 1 -- start start 2024-09-11T22:05:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.372+0000 7f15eca80740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557354f91970 0x557354f91d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.372+0000 7f15eca80740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557354f92290 0x557354f9e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.372+0000 7f15eca80740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557354f9eec0 0x557354fa12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.372+0000 7f15eca80740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x557354ef62f0 con 0x557354f91970 2024-09-11T22:05:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.372+0000 7f15eca80740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x557354e9e270 con 0x557354f9eec0 2024-09-11T22:05:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.372+0000 7f15eca80740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x557354ef22b0 con 0x557354f92290 2024-09-11T22:05:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.372+0000 7f15e3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557354f91970 0x557354f91d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.372+0000 7f15e3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557354f91970 0x557354f91d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42546/0 (socket says 172.21.15.1:42546) 2024-09-11T22:05:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.372+0000 7f15e3fff640 1 -- 172.21.15.1:0/3104625455 learned_addr learned my addr 172.21.15.1:0/3104625455 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.372+0000 7f15e17fa640 1 --2- 172.21.15.1:0/3152391803 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f15d00a2cf0 0x7f15d00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:33.379 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.372+0000 7f15daffd640 1 --2- 172.21.15.1:0/3104625455 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557354f9eec0 0x557354fa12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:33.379 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.372+0000 7f15e37fe640 1 --2- 172.21.15.1:0/3104625455 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557354f92290 0x557354f9e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:33.379 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.372+0000 7f15e3fff640 1 --2- 172.21.15.1:0/3152391803 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f15d00a2cf0 0x7f15d00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:33.379 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.372+0000 7f15e17fa640 1 -- 172.21.15.1:0/3152391803 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7f15cc0cf360 con 0x7f15e4101370 2024-09-11T22:05:33.379 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.373+0000 7f15e3fff640 1 -- 172.21.15.1:0/3104625455 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557354f92290 msgr2=0x557354f9e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:33.379 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.373+0000 7f15e3fff640 1 --2- 172.21.15.1:0/3104625455 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557354f92290 0x557354f9e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:33.379 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.373+0000 7f15e3fff640 1 -- 172.21.15.1:0/3104625455 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557354f9eec0 msgr2=0x557354fa12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:33.379 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.373+0000 7f15e3fff640 1 --2- 172.21.15.1:0/3104625455 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557354f9eec0 0x557354fa12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:33.379 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.373+0000 7f15e3fff640 1 -- 172.21.15.1:0/3104625455 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557354e9d340 con 0x557354f91970 2024-09-11T22:05:33.379 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.373+0000 7f15daffd640 1 --2- 172.21.15.1:0/3104625455 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557354f9eec0 0x557354fa12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:05:33.379 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.373+0000 7f15e3fff640 1 --2- 172.21.15.1:0/3104625455 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557354f91970 0x557354f91d50 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f15dc012890 tx=0x7f15dc012d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=933188c4655078d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:33.379 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.373+0000 7f15d9ffb640 1 -- 172.21.15.1:0/3104625455 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15dc005020 con 0x557354f91970 2024-09-11T22:05:33.380 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.373+0000 7f15d9ffb640 1 -- 172.21.15.1:0/3104625455 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15dc0056a0 con 0x557354f91970 2024-09-11T22:05:33.380 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.373+0000 7f15d9ffb640 1 -- 172.21.15.1:0/3104625455 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15dc01c770 con 0x557354f91970 2024-09-11T22:05:33.380 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.373+0000 7f15e3fff640 1 --2- 172.21.15.1:0/3152391803 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f15d00a2cf0 0x7f15d00a51b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f15dc01f000 tx=0x7f15dc01e000 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:33.380 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.374+0000 7f15eca80740 1 -- 172.21.15.1:0/3104625455 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557354f91970 msgr2=0x557354f91d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:33.380 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.374+0000 7f15eca80740 1 --2- 172.21.15.1:0/3104625455 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557354f91970 0x557354f91d50 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f15dc012890 tx=0x7f15dc012d50 comp rx=0 tx=0).stop 2024-09-11T22:05:33.380 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.374+0000 7f15eca80740 1 -- 172.21.15.1:0/3104625455 shutdown_connections 2024-09-11T22:05:33.380 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.374+0000 7f15eca80740 1 --2- 172.21.15.1:0/3104625455 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557354f9eec0 0x557354fa12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:33.380 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.374+0000 7f15eca80740 1 --2- 172.21.15.1:0/3104625455 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557354f92290 0x557354f9e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:33.380 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.374+0000 7f15eca80740 1 --2- 172.21.15.1:0/3104625455 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557354f91970 0x557354f91d50 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:33.380 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.374+0000 7f15eca80740 1 -- 172.21.15.1:0/3104625455 >> 172.21.15.1:0/3104625455 conn(0x557354f8e060 msgr2=0x557354f8e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:33.380 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.374+0000 7f15eca80740 1 -- 172.21.15.1:0/3104625455 shutdown_connections 2024-09-11T22:05:33.380 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.374+0000 7f15eca80740 1 -- 172.21.15.1:0/3104625455 wait complete. 2024-09-11T22:05:33.380 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.374+0000 7f15eca80740 1 -- start start 2024-09-11T22:05:33.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.374+0000 7f15eca80740 1 Processor -- start 2024-09-11T22:05:33.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.375+0000 7f15eca80740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557354f91970 0x557354f9be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:33.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.375+0000 7f15eca80740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557354f92290 0x557354f9c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:33.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.375+0000 7f15eca80740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557354f9eec0 0x557354f9c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:33.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.375+0000 7f15eca80740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x557354fa1af0 con 0x557354f91970 2024-09-11T22:05:33.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.375+0000 7f15eca80740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x557354e7f290 con 0x557354f9eec0 2024-09-11T22:05:33.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.375+0000 7f15eca80740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x557354fa1c70 con 0x557354f92290 2024-09-11T22:05:33.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.375+0000 7f15eca80740 10 client.0 _ll_register_callbacks cb 0x557354eb0140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:05:33.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.375+0000 7f15e37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557354f91970 0x557354f9be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:33.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.375+0000 7f15e37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557354f91970 0x557354f9be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42548/0 (socket says 172.21.15.1:42548) 2024-09-11T22:05:33.382 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.375+0000 7f15e37fe640 1 -- 172.21.15.1:0/1902878391 learned_addr learned my addr 172.21.15.1:0/1902878391 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:33.382 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.375+0000 7f15daffd640 1 --2- 172.21.15.1:0/1902878391 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557354f92290 0x557354f9c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:33.382 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.375+0000 7f15e3fff640 1 --2- 172.21.15.1:0/1902878391 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557354f9eec0 0x557354f9c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:33.383 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.375+0000 7f15e37fe640 1 -- 172.21.15.1:0/1902878391 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557354f92290 msgr2=0x557354f9c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:33.383 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.375+0000 7f15e37fe640 1 --2- 172.21.15.1:0/1902878391 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557354f92290 0x557354f9c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:33.383 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.375+0000 7f15e37fe640 1 -- 172.21.15.1:0/1902878391 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557354f9eec0 msgr2=0x557354f9c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:33.383 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.375+0000 7f15e37fe640 1 --2- 172.21.15.1:0/1902878391 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557354f9eec0 0x557354f9c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:33.383 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.375+0000 7f15e37fe640 1 -- 172.21.15.1:0/1902878391 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557354c09210 con 0x557354f91970 2024-09-11T22:05:33.383 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.375+0000 7f15e3fff640 1 --2- 172.21.15.1:0/1902878391 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557354f9eec0 0x557354f9c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:05:33.383 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.375+0000 7f15e37fe640 1 --2- 172.21.15.1:0/1902878391 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557354f91970 0x557354f9be50 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f15d4009930 tx=0x7f15d4003800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5f3f93dd1133bb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:33.383 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.375+0000 7f15c2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:05:33.383 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.375+0000 7f15c2ffd640 1 -- 172.21.15.1:0/1902878391 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d4035070 con 0x557354f91970 2024-09-11T22:05:33.383 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.375+0000 7f15c2ffd640 1 -- 172.21.15.1:0/1902878391 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7f15d4003b70 con 0x557354f91970 2024-09-11T22:05:33.384 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.375+0000 7f15eca80740 1 -- 172.21.15.1:0/1902878391 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15dc0056a0 con 0x557354f91970 2024-09-11T22:05:33.384 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.376+0000 7f15c2ffd640 1 -- 172.21.15.1:0/1902878391 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f15d4003d70 con 0x557354f91970 2024-09-11T22:05:33.384 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.376+0000 7f15eca80740 10 client.17322 fetch_fsmap learned FSMap version 42 2024-09-11T22:05:33.384 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.376+0000 7f15eca80740 1 -- 172.21.15.1:0/1902878391 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x557354cd5620 con 0x557354f91970 2024-09-11T22:05:33.384 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.376+0000 7f15c2ffd640 1 -- 172.21.15.1:0/1902878391 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15dc0056a0 con 0x557354f91970 2024-09-11T22:05:33.384 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.376+0000 7f15c2ffd640 1 -- 172.21.15.1:0/1902878391 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f15d40446d0 con 0x557354f91970 2024-09-11T22:05:33.384 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.376+0000 7f15eca80740 10 client.17322 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:05:33.384 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.376+0000 7f15eca80740 10 client.17322 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:05:33.384 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.376+0000 7f15eca80740 20 client.17322 populate_metadata read hostname 'smithi001' 2024-09-11T22:05:33.384 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.376+0000 7f15eca80740 1 --2- 172.21.15.1:0/1902878391 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x557354faeec0 0x557354fb1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:33.384 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.376+0000 7f15eca80740 4 client.17322 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:05:33.384 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.376+0000 7f15eca80740 1 -- 172.21.15.1:0/1902878391 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557354f03a20 con 0x557354faeec0 2024-09-11T22:05:33.385 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.376+0000 7f15daffd640 1 --2- 172.21.15.1:0/1902878391 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x557354faeec0 0x557354fb1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:33.385 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.377+0000 7f15c2ffd640 10 client.17322 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:05:33.385 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.377+0000 7f15daffd640 1 --2- 172.21.15.1:0/1902878391 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x557354faeec0 0x557354fb1320 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e25c1f3e92c49f server_cookie=bfe94c44db7a2cfd in_seq=0 out_seq=0 2024-09-11T22:05:33.386 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.377+0000 7f15c2ffd640 1 -- 172.21.15.1:0/1902878391 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557354f03a20 con 0x557354faeec0 2024-09-11T22:05:33.386 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.377+0000 7f15c2ffd640 10 client.17322 handle_command_reply: tid=0 2024-09-11T22:05:33.386 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.377+0000 7f15eca80740 1 client.17322 shutdown 2024-09-11T22:05:33.386 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.378+0000 7f15eca80740 1 -- 172.21.15.1:0/1902878391 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557354f91970 msgr2=0x557354f9be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:33.386 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.378+0000 7f15eca80740 1 --2- 172.21.15.1:0/1902878391 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557354f91970 0x557354f9be50 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f15d4009930 tx=0x7f15d4003800 comp rx=0 tx=0).stop 2024-09-11T22:05:33.386 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.378+0000 7f15eca80740 1 -- 172.21.15.1:0/1902878391 shutdown_connections 2024-09-11T22:05:33.386 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.378+0000 7f15eca80740 1 --2- 172.21.15.1:0/1902878391 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x557354faeec0 0x557354fb1320 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:33.386 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.378+0000 7f15eca80740 1 --2- 172.21.15.1:0/1902878391 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557354f9eec0 0x557354f9c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:33.386 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.378+0000 7f15eca80740 1 --2- 172.21.15.1:0/1902878391 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557354f92290 0x557354f9c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:33.386 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.378+0000 7f15eca80740 1 --2- 172.21.15.1:0/1902878391 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557354f91970 0x557354f9be50 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:33.387 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.378+0000 7f15c2ffd640 0 client.17322 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:05:33.387 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.378+0000 7f15eca80740 1 -- 172.21.15.1:0/1902878391 >> 172.21.15.1:0/1902878391 conn(0x557354f8e060 msgr2=0x557354f93dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:33.387 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.378+0000 7f15eca80740 1 -- 172.21.15.1:0/1902878391 shutdown_connections 2024-09-11T22:05:33.387 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.378+0000 7f15eca80740 1 -- 172.21.15.1:0/1902878391 wait complete. 2024-09-11T22:05:33.387 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.378+0000 7f15eca80740 20 client.17322 trim_cache size 0 max 16384 2024-09-11T22:05:33.393 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.392+0000 7f15eca80740 1 Processor -- start 2024-09-11T22:05:33.393 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.392+0000 7f15eca80740 1 -- start start 2024-09-11T22:05:33.393 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.392+0000 7f15eca80740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557354f91970 0x557354fb6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:33.393 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.392+0000 7f15eca80740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557354f92290 0x557354fb7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.392+0000 7f15eca80740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557354f9eec0 0x557354f990b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.392+0000 7f15eca80740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f15e41ab770 con 0x557354f92290 2024-09-11T22:05:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.392+0000 7f15eca80740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f15e41ad5b0 con 0x557354f91970 2024-09-11T22:05:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.392+0000 7f15eca80740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f15e41abf00 con 0x557354f9eec0 2024-09-11T22:05:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.393+0000 7f15daffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557354f92290 0x557354fb7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.393+0000 7f15daffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557354f92290 0x557354fb7170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42554/0 (socket says 172.21.15.1:42554) 2024-09-11T22:05:33.394 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.393+0000 7f15daffd640 1 -- 172.21.15.1:0/420669474 learned_addr learned my addr 172.21.15.1:0/420669474 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:33.401 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.393+0000 7f15e3fff640 1 --2- 172.21.15.1:0/420669474 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557354f9eec0 0x557354f990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:33.401 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.393+0000 7f15daffd640 1 -- 172.21.15.1:0/420669474 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557354f9eec0 msgr2=0x557354f990b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:33.401 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.393+0000 7f15daffd640 1 --2- 172.21.15.1:0/420669474 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557354f9eec0 0x557354f990b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:33.401 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.393+0000 7f15daffd640 1 -- 172.21.15.1:0/420669474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557354f91970 msgr2=0x557354fb6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:33.401 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.393+0000 7f15e37fe640 1 --2- 172.21.15.1:0/420669474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557354f91970 0x557354fb6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:33.401 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.393+0000 7f15daffd640 1 --2- 172.21.15.1:0/420669474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557354f91970 0x557354fb6c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:33.402 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.393+0000 7f15daffd640 1 -- 172.21.15.1:0/420669474 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557354f920c0 con 0x557354f92290 2024-09-11T22:05:33.402 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.393+0000 7f15daffd640 1 --2- 172.21.15.1:0/420669474 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557354f92290 0x557354fb7170 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f15cc0981e0 tx=0x7f15cc06d7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f1992b53378c186 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:33.402 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.393+0000 7f15d97fa640 1 -- 172.21.15.1:0/420669474 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15cc01d040 con 0x557354f92290 2024-09-11T22:05:33.402 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.393+0000 7f15d97fa640 1 -- 172.21.15.1:0/420669474 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15cc09bf10 con 0x557354f92290 2024-09-11T22:05:33.402 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.393+0000 7f15d97fa640 1 -- 172.21.15.1:0/420669474 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15cc0cccc0 con 0x557354f92290 2024-09-11T22:05:33.402 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.394+0000 7f15eca80740 1 -- 172.21.15.1:0/420669474 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557354f92290 msgr2=0x557354fb7170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:33.402 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.394+0000 7f15eca80740 1 --2- 172.21.15.1:0/420669474 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557354f92290 0x557354fb7170 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f15cc0981e0 tx=0x7f15cc06d7d0 comp rx=0 tx=0).stop 2024-09-11T22:05:33.402 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.394+0000 7f15eca80740 1 -- 172.21.15.1:0/420669474 shutdown_connections 2024-09-11T22:05:33.402 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.394+0000 7f15eca80740 1 --2- 172.21.15.1:0/420669474 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557354f9eec0 0x557354f990b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:33.402 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.394+0000 7f15eca80740 1 --2- 172.21.15.1:0/420669474 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557354f92290 0x557354fb7170 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:33.402 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.394+0000 7f15eca80740 1 --2- 172.21.15.1:0/420669474 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557354f91970 0x557354fb6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:33.402 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.394+0000 7f15eca80740 1 -- 172.21.15.1:0/420669474 >> 172.21.15.1:0/420669474 conn(0x557354f8e060 msgr2=0x557354f9ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:33.403 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.394+0000 7f15eca80740 1 -- 172.21.15.1:0/420669474 shutdown_connections 2024-09-11T22:05:33.403 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.394+0000 7f15eca80740 1 -- 172.21.15.1:0/420669474 wait complete. 2024-09-11T22:05:33.403 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.394+0000 7f15eca80740 1 -- start start 2024-09-11T22:05:33.403 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.394+0000 7f15eca80740 1 Processor -- start 2024-09-11T22:05:33.403 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.395+0000 7f15eca80740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557354f91970 0x557354fa75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:33.403 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.395+0000 7f15eca80740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557354f92290 0x557354f98b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:33.403 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.395+0000 7f15eca80740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557354f9eec0 0x557354f99050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:33.403 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.395+0000 7f15eca80740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f15e41ace20 con 0x557354f92290 2024-09-11T22:05:33.403 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.395+0000 7f15eca80740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f15e41ac690 con 0x557354f9eec0 2024-09-11T22:05:33.403 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.395+0000 7f15eca80740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f15e41a9770 con 0x557354f91970 2024-09-11T22:05:33.403 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.395+0000 7f15eca80740 10 client.0 _ll_register_callbacks cb 0x7f15d4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:05:33.403 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.395+0000 7f15daffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557354f92290 0x557354f98b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:33.403 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.395+0000 7f15daffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557354f92290 0x557354f98b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42564/0 (socket says 172.21.15.1:42564) 2024-09-11T22:05:33.404 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.395+0000 7f15daffd640 1 -- 172.21.15.1:0/3996275184 learned_addr learned my addr 172.21.15.1:0/3996275184 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:33.404 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.395+0000 7f15e37fe640 1 --2- 172.21.15.1:0/3996275184 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557354f91970 0x557354fa75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:33.404 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.395+0000 7f15e3fff640 1 --2- 172.21.15.1:0/3996275184 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557354f9eec0 0x557354f99050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:33.404 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.395+0000 7f15daffd640 1 -- 172.21.15.1:0/3996275184 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557354f91970 msgr2=0x557354fa75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:33.404 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.395+0000 7f15daffd640 1 --2- 172.21.15.1:0/3996275184 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557354f91970 0x557354fa75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:33.404 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.395+0000 7f15daffd640 1 -- 172.21.15.1:0/3996275184 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557354f9eec0 msgr2=0x557354f99050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:33.404 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.395+0000 7f15daffd640 1 --2- 172.21.15.1:0/3996275184 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557354f9eec0 0x557354f99050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:33.404 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.395+0000 7f15daffd640 1 -- 172.21.15.1:0/3996275184 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557354f99850 con 0x557354f92290 2024-09-11T22:05:33.404 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.395+0000 7f15daffd640 1 --2- 172.21.15.1:0/3996275184 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557354f92290 0x557354f98b10 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f15cc0ccaf0 tx=0x7f15cc09b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d8b6d8b035c7d32 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:33.404 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.395+0000 7f15c2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:05:33.404 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.395+0000 7f15c2ffd640 1 -- 172.21.15.1:0/3996275184 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15cc0b3020 con 0x557354f92290 2024-09-11T22:05:33.405 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.395+0000 7f15c2ffd640 1 -- 172.21.15.1:0/3996275184 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7f15cc0860c0 con 0x557354f92290 2024-09-11T22:05:33.405 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.395+0000 7f15eca80740 1 -- 172.21.15.1:0/3996275184 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15cc09bf10 con 0x557354f92290 2024-09-11T22:05:33.405 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.395+0000 7f15c2ffd640 1 -- 172.21.15.1:0/3996275184 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f15cc0a4480 con 0x557354f92290 2024-09-11T22:05:33.405 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.396+0000 7f15eca80740 10 client.17328 fetch_fsmap learned FSMap version 42 2024-09-11T22:05:33.405 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.396+0000 7f15eca80740 1 -- 172.21.15.1:0/3996275184 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x557355019be0 con 0x557354f92290 2024-09-11T22:05:33.405 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.396+0000 7f15c2ffd640 1 -- 172.21.15.1:0/3996275184 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15cc09bf10 con 0x557354f92290 2024-09-11T22:05:33.405 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.396+0000 7f15c2ffd640 1 -- 172.21.15.1:0/3996275184 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f15cc005310 con 0x557354f92290 2024-09-11T22:05:33.405 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.396+0000 7f15eca80740 10 client.17328 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:05:33.405 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.396+0000 7f15eca80740 10 client.17328 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:05:33.405 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.396+0000 7f15eca80740 20 client.17328 populate_metadata read hostname 'smithi001' 2024-09-11T22:05:33.405 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.396+0000 7f15eca80740 1 --2- 172.21.15.1:0/3996275184 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55735501a420 0x55735501c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:33.405 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.396+0000 7f15eca80740 4 client.17328 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:05:33.406 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.396+0000 7f15eca80740 1 -- 172.21.15.1:0/3996275184 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557354efdd70 con 0x55735501a420 2024-09-11T22:05:33.406 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.396+0000 7f15e37fe640 1 --2- 172.21.15.1:0/3996275184 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55735501a420 0x55735501c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:33.406 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.396+0000 7f15c2ffd640 10 client.17328 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:05:33.406 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.396+0000 7f15e37fe640 1 --2- 172.21.15.1:0/3996275184 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55735501a420 0x55735501c840 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ebbc3c457768241 server_cookie=119d20727f064c51 in_seq=0 out_seq=0 2024-09-11T22:05:33.412 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.411+0000 7f15c2ffd640 1 -- 172.21.15.1:0/3996275184 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557354efdd70 con 0x55735501a420 2024-09-11T22:05:33.412 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.411+0000 7f15c2ffd640 10 client.17328 handle_command_reply: tid=0 2024-09-11T22:05:33.412 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.411+0000 7f15eca80740 1 client.17328 shutdown 2024-09-11T22:05:33.412 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.411+0000 7f15eca80740 1 -- 172.21.15.1:0/3996275184 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557354f92290 msgr2=0x557354f98b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:33.412 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.411+0000 7f15eca80740 1 --2- 172.21.15.1:0/3996275184 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557354f92290 0x557354f98b10 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f15cc0ccaf0 tx=0x7f15cc09b900 comp rx=0 tx=0).stop 2024-09-11T22:05:33.413 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.411+0000 7f15eca80740 1 -- 172.21.15.1:0/3996275184 shutdown_connections 2024-09-11T22:05:33.413 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.411+0000 7f15eca80740 1 --2- 172.21.15.1:0/3996275184 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55735501a420 0x55735501c840 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:33.413 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.411+0000 7f15eca80740 1 --2- 172.21.15.1:0/3996275184 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557354f9eec0 0x557354f99050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:33.413 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.411+0000 7f15c2ffd640 0 client.17328 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:05:33.413 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.411+0000 7f15eca80740 1 --2- 172.21.15.1:0/3996275184 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557354f92290 0x557354f98b10 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:33.413 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.411+0000 7f15eca80740 1 --2- 172.21.15.1:0/3996275184 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x557354f91970 0x557354fa75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:33.413 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.411+0000 7f15eca80740 1 -- 172.21.15.1:0/3996275184 >> 172.21.15.1:0/3996275184 conn(0x557354f8e060 msgr2=0x557354f8c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:33.413 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.411+0000 7f15eca80740 1 -- 172.21.15.1:0/3996275184 shutdown_connections 2024-09-11T22:05:33.413 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.411+0000 7f15eca80740 1 -- 172.21.15.1:0/3996275184 wait complete. 2024-09-11T22:05:33.413 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.411+0000 7f15eca80740 20 client.17328 trim_cache size 0 max 16384 2024-09-11T22:05:33.414 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-09-11T22:05:33.414 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-09-11T22:05:33.414 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-09-11T22:05:33.414 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-09-11T22:05:33.415 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.412+0000 7f15e9bfd640 1 -- 172.21.15.1:0/3152391803 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f15d00a2cf0 msgr2=0x7f15d00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:33.415 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.412+0000 7f15e9bfd640 1 --2- 172.21.15.1:0/3152391803 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f15d00a2cf0 0x7f15d00a51b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f15dc01f000 tx=0x7f15dc01e000 comp rx=0 tx=0).stop 2024-09-11T22:05:33.415 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.412+0000 7f15e9bfd640 1 -- 172.21.15.1:0/3152391803 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15e4101370 msgr2=0x7f15e4197820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:33.415 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.412+0000 7f15e9bfd640 1 --2- 172.21.15.1:0/3152391803 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15e4101370 0x7f15e4197820 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f15cc00c970 tx=0x7f15cc00ce30 comp rx=0 tx=0).stop 2024-09-11T22:05:33.415 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.412+0000 7f15e9bfd640 1 -- 172.21.15.1:0/3152391803 shutdown_connections 2024-09-11T22:05:33.415 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.412+0000 7f15e9bfd640 1 --2- 172.21.15.1:0/3152391803 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f15d00a2cf0 0x7f15d00a51b0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:33.415 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.412+0000 7f15e9bfd640 1 --2- 172.21.15.1:0/3152391803 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f15e41069e0 0x7f15e419bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:33.415 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.412+0000 7f15e9bfd640 1 --2- 172.21.15.1:0/3152391803 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15e4101370 0x7f15e4197820 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:33.415 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.412+0000 7f15e9bfd640 1 --2- 172.21.15.1:0/3152391803 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f15e41009d0 0x7f15e41972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:33.415 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.412+0000 7f15e9bfd640 1 -- 172.21.15.1:0/3152391803 >> 172.21.15.1:0/3152391803 conn(0x7f15e40fc6b0 msgr2=0x7f15e40fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:33.416 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.412+0000 7f15e9bfd640 1 -- 172.21.15.1:0/3152391803 shutdown_connections 2024-09-11T22:05:33.416 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:33.412+0000 7f15e9bfd640 1 -- 172.21.15.1:0/3152391803 wait complete. 2024-09-11T22:05:33.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:33 smithi001 ceph-mon[30935]: pgmap v256: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-09-11T22:05:33.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:33 smithi053 ceph-mon[34908]: pgmap v256: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-09-11T22:05:33.553 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:05:33.821 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:05:34.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:34.383+0000 7f9d3f9ba640 1 -- 172.21.15.1:0/4208378560 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d38102d30 msgr2=0x7f9d38103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:34.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:34.383+0000 7f9d3f9ba640 1 --2- 172.21.15.1:0/4208378560 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d38102d30 0x7f9d38103110 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f9d28009cd0 tx=0x7f9d2802a8e0 comp rx=0 tx=0).stop 2024-09-11T22:05:34.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:34.383+0000 7f9d3f9ba640 1 -- 172.21.15.1:0/4208378560 shutdown_connections 2024-09-11T22:05:34.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:34.383+0000 7f9d3f9ba640 1 --2- 172.21.15.1:0/4208378560 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d381082a0 0x7f9d3810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:34.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:34.383+0000 7f9d3f9ba640 1 --2- 172.21.15.1:0/4208378560 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d381036e0 0x7f9d38107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:34.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:34.383+0000 7f9d3f9ba640 1 --2- 172.21.15.1:0/4208378560 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d38102d30 0x7f9d38103110 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:34.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:34.383+0000 7f9d3f9ba640 1 -- 172.21.15.1:0/4208378560 >> 172.21.15.1:0/4208378560 conn(0x7f9d380fc7d0 msgr2=0x7f9d380febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:34.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:34.383+0000 7f9d3f9ba640 1 -- 172.21.15.1:0/4208378560 shutdown_connections 2024-09-11T22:05:34.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:34.383+0000 7f9d3f9ba640 1 -- 172.21.15.1:0/4208378560 wait complete. 2024-09-11T22:05:34.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:34.384+0000 7f9d3f9ba640 1 Processor -- start 2024-09-11T22:05:34.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:34.384+0000 7f9d3f9ba640 1 -- start start 2024-09-11T22:05:34.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:34.384+0000 7f9d3f9ba640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d38102d30 0x7f9d38100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:34.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:34.384+0000 7f9d3f9ba640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d381036e0 0x7f9d38100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:34.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:34.384+0000 7f9d3f9ba640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d381082a0 0x7f9d38101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:34.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:34.384+0000 7f9d3f9ba640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9d381120a0 con 0x7f9d381036e0 2024-09-11T22:05:34.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:34.384+0000 7f9d3f9ba640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9d38111f20 con 0x7f9d38102d30 2024-09-11T22:05:34.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:34.384+0000 7f9d3f9ba640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9d38112220 con 0x7f9d381082a0 2024-09-11T22:05:34.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:34.384+0000 7f9d3cf55640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d381036e0 0x7f9d38100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:34.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:34.384+0000 7f9d3cf55640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d381036e0 0x7f9d38100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42580/0 (socket says 172.21.15.1:42580) 2024-09-11T22:05:34.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:34.384+0000 7f9d3cf55640 1 -- 172.21.15.1:0/1412582147 learned_addr learned my addr 172.21.15.1:0/1412582147 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:34.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:34.384+0000 7f9d3d756640 1 --2- 172.21.15.1:0/1412582147 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d38102d30 0x7f9d38100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:34.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:34.384+0000 7f9d3df57640 1 --2- 172.21.15.1:0/1412582147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d381082a0 0x7f9d38101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:34.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:34.384+0000 7f9d3cf55640 1 -- 172.21.15.1:0/1412582147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d381082a0 msgr2=0x7f9d38101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:34.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:34.384+0000 7f9d3cf55640 1 --2- 172.21.15.1:0/1412582147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d381082a0 0x7f9d38101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:34.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:34.384+0000 7f9d3cf55640 1 -- 172.21.15.1:0/1412582147 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d38102d30 msgr2=0x7f9d38100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:34.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:34.384+0000 7f9d3cf55640 1 --2- 172.21.15.1:0/1412582147 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d38102d30 0x7f9d38100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:34.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:34.384+0000 7f9d3cf55640 1 -- 172.21.15.1:0/1412582147 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d381a69e0 con 0x7f9d381036e0 2024-09-11T22:05:34.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:34.385+0000 7f9d3cf55640 1 --2- 172.21.15.1:0/1412582147 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d381036e0 0x7f9d38100ed0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c00d740 tx=0x7f9d2c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:34.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:34.385+0000 7f9d367fc640 1 -- 172.21.15.1:0/1412582147 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c017070 con 0x7f9d381036e0 2024-09-11T22:05:34.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:34.385+0000 7f9d367fc640 1 -- 172.21.15.1:0/1412582147 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d2c00de50 con 0x7f9d381036e0 2024-09-11T22:05:34.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:34.385+0000 7f9d3f9ba640 1 -- 172.21.15.1:0/1412582147 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d381a6cd0 con 0x7f9d381036e0 2024-09-11T22:05:34.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:34.385+0000 7f9d367fc640 1 -- 172.21.15.1:0/1412582147 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c00f6b0 con 0x7f9d381036e0 2024-09-11T22:05:34.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:34.385+0000 7f9d3f9ba640 1 -- 172.21.15.1:0/1412582147 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d381a7200 con 0x7f9d381036e0 2024-09-11T22:05:34.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:34.386+0000 7f9d3f9ba640 1 -- 172.21.15.1:0/1412582147 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d38103110 con 0x7f9d381036e0 2024-09-11T22:05:34.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:34.387+0000 7f9d367fc640 1 -- 172.21.15.1:0/1412582147 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f9d2c01a070 con 0x7f9d381036e0 2024-09-11T22:05:34.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:34.388+0000 7f9d367fc640 1 --2- 172.21.15.1:0/1412582147 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f9d200a2d70 0x7f9d200a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:34.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:34.388+0000 7f9d3d756640 1 --2- 172.21.15.1:0/1412582147 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f9d200a2d70 0x7f9d200a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:34.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:34.388+0000 7f9d367fc640 1 -- 172.21.15.1:0/1412582147 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7f9d2c0ceeb0 con 0x7f9d381036e0 2024-09-11T22:05:34.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:34.388+0000 7f9d3d756640 1 --2- 172.21.15.1:0/1412582147 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f9d200a2d70 0x7f9d200a5230 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9d28006010 tx=0x7f9d28009360 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:34.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:34.392+0000 7f9d367fc640 1 -- 172.21.15.1:0/1412582147 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9d2c09ba00 con 0x7f9d381036e0 2024-09-11T22:05:34.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:34.602+0000 7f9d3f9ba640 1 -- 172.21.15.1:0/1412582147 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9d381a7430 con 0x7f9d381036e0 2024-09-11T22:05:34.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:34.603+0000 7f9d367fc640 1 -- 172.21.15.1:0/1412582147 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11984 (secure 0 0 0) 0x7f9d2c09b300 con 0x7f9d381036e0 2024-09-11T22:05:34.604 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T22:05:34.604 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":42,"btime":"2024-09-11T22:03:27:040852+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/421542832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":421542832},{"type":"v1","addr":"172.21.15.1:6835","nonce":421542832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2836864406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2836864406},{"type":"v1","addr":"172.21.15.1:6837","nonce":2836864406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":42,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T21:55:20.791263+0000","modified":"2024-09-11T22:03:26.485134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24484,"mds_2":24467,"mds_3":24466,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14595":{"gid":14595,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/407834672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":407834672},{"type":"v1","addr":"172.21.15.1:6839","nonce":407834672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/1068827678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":1068827678},{"type":"v1","addr":"172.21.15.1:6841","nonce":1068827678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/872410024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":872410024},{"type":"v1","addr":"172.21.15.161:6833","nonce":872410024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/1648944570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1648944570},{"type":"v1","addr":"172.21.15.53:6833","nonce":1648944570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/2126933993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2126933993},{"type":"v1","addr":"172.21.15.53:6835","nonce":2126933993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.161:6835/828381592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":828381592},{"type":"v1","addr":"172.21.15.161:6835","nonce":828381592}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.161:6837/1799318656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1799318656},{"type":"v1","addr":"172.21.15.161:6837","nonce":1799318656}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.53:6837/3457131830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3457131830},{"type":"v1","addr":"172.21.15.53:6837","nonce":3457131830}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/455365383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":455365383},{"type":"v1","addr":"172.21.15.161:6839","nonce":455365383}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.53:6839/1378584447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1378584447},{"type":"v1","addr":"172.21.15.53:6839","nonce":1378584447}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24460,24466,24467,24484,24494]},"id":1}]} 2024-09-11T22:05:34.604 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 42 2024-09-11T22:05:34.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:34.605+0000 7f9d3f9ba640 1 -- 172.21.15.1:0/1412582147 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f9d200a2d70 msgr2=0x7f9d200a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:34.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:34.605+0000 7f9d3f9ba640 1 --2- 172.21.15.1:0/1412582147 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f9d200a2d70 0x7f9d200a5230 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9d28006010 tx=0x7f9d28009360 comp rx=0 tx=0).stop 2024-09-11T22:05:34.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:34.605+0000 7f9d3f9ba640 1 -- 172.21.15.1:0/1412582147 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d381036e0 msgr2=0x7f9d38100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:34.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:34.605+0000 7f9d3f9ba640 1 --2- 172.21.15.1:0/1412582147 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d381036e0 0x7f9d38100ed0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c00d740 tx=0x7f9d2c00dc00 comp rx=0 tx=0).stop 2024-09-11T22:05:34.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:34.605+0000 7f9d3f9ba640 1 -- 172.21.15.1:0/1412582147 shutdown_connections 2024-09-11T22:05:34.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:34.605+0000 7f9d3f9ba640 1 --2- 172.21.15.1:0/1412582147 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f9d200a2d70 0x7f9d200a5230 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:34.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:34.605+0000 7f9d3f9ba640 1 --2- 172.21.15.1:0/1412582147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d381082a0 0x7f9d38101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:34.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:34.605+0000 7f9d3f9ba640 1 --2- 172.21.15.1:0/1412582147 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d381036e0 0x7f9d38100ed0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:34.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:34.605+0000 7f9d3f9ba640 1 --2- 172.21.15.1:0/1412582147 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9d38102d30 0x7f9d38100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:34.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:34.605+0000 7f9d3f9ba640 1 -- 172.21.15.1:0/1412582147 >> 172.21.15.1:0/1412582147 conn(0x7f9d380fc7d0 msgr2=0x7f9d38105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:34.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:34.605+0000 7f9d3f9ba640 1 -- 172.21.15.1:0/1412582147 shutdown_connections 2024-09-11T22:05:34.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:34.605+0000 7f9d3f9ba640 1 -- 172.21.15.1:0/1412582147 wait complete. 2024-09-11T22:05:35.307 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:35 smithi001 ceph-mon[30935]: pgmap v257: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-09-11T22:05:35.307 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:35 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/1412582147' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:05:35.339 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db --set-id f8d2564c --release --await' 2024-09-11T22:05:35.340 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db --set-id f8d2564c --release --await 2024-09-11T22:05:35.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:35 smithi161 ceph-mon[35615]: pgmap v257: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-09-11T22:05:35.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:35 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/1412582147' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:05:35.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:35 smithi053 ceph-mon[34908]: pgmap v257: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-09-11T22:05:35.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:35 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/1412582147' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:05:35.604 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:05:36.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.119+0000 7f4eee8a6640 1 -- 172.21.15.1:0/1431650405 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ee8100a80 msgr2=0x7f4ee8100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:36.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.119+0000 7f4eee8a6640 1 --2- 172.21.15.1:0/1431650405 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ee8100a80 0x7f4ee8100ee0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0009930 tx=0x7f4ed002a420 comp rx=0 tx=0).stop 2024-09-11T22:05:36.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.119+0000 7f4eee8a6640 1 -- 172.21.15.1:0/1431650405 shutdown_connections 2024-09-11T22:05:36.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.119+0000 7f4eee8a6640 1 --2- 172.21.15.1:0/1431650405 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ee8101420 0x7f4ee810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:36.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.119+0000 7f4eee8a6640 1 --2- 172.21.15.1:0/1431650405 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ee8100a80 0x7f4ee8100ee0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:36.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.119+0000 7f4eee8a6640 1 --2- 172.21.15.1:0/1431650405 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ee8106a80 0x7f4ee8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:36.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.119+0000 7f4eee8a6640 1 -- 172.21.15.1:0/1431650405 >> 172.21.15.1:0/1431650405 conn(0x7f4ee80fc7a0 msgr2=0x7f4ee80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:36.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.119+0000 7f4eee8a6640 1 -- 172.21.15.1:0/1431650405 shutdown_connections 2024-09-11T22:05:36.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.119+0000 7f4eee8a6640 1 -- 172.21.15.1:0/1431650405 wait complete. 2024-09-11T22:05:36.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.120+0000 7f4eee8a6640 1 Processor -- start 2024-09-11T22:05:36.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.120+0000 7f4eee8a6640 1 -- start start 2024-09-11T22:05:36.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.120+0000 7f4eee8a6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ee8100a80 0x7f4ee81992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:36.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.120+0000 7f4eee8a6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ee8101420 0x7f4ee8199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:36.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.120+0000 7f4eee8a6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ee8106a80 0x7f4ee81a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:36.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.120+0000 7f4eee8a6640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee8110e80 con 0x7f4ee8101420 2024-09-11T22:05:36.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.120+0000 7f4eee8a6640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee8110d00 con 0x7f4ee8100a80 2024-09-11T22:05:36.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.120+0000 7f4eee8a6640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee8111000 con 0x7f4ee8106a80 2024-09-11T22:05:36.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.120+0000 7f4edffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ee8101420 0x7f4ee8199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:36.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.120+0000 7f4edffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ee8101420 0x7f4ee8199810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42594/0 (socket says 172.21.15.1:42594) 2024-09-11T22:05:36.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.120+0000 7f4edffff640 1 -- 172.21.15.1:0/2591142995 learned_addr learned my addr 172.21.15.1:0/2591142995 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:36.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.120+0000 7f4eece43640 1 --2- 172.21.15.1:0/2591142995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ee8106a80 0x7f4ee81a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:36.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.120+0000 7f4ee7fff640 1 --2- 172.21.15.1:0/2591142995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ee8100a80 0x7f4ee81992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:36.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.121+0000 7f4edffff640 1 -- 172.21.15.1:0/2591142995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ee8106a80 msgr2=0x7f4ee81a4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:36.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.121+0000 7f4edffff640 1 --2- 172.21.15.1:0/2591142995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ee8106a80 0x7f4ee81a4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:36.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.121+0000 7f4edffff640 1 -- 172.21.15.1:0/2591142995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ee8100a80 msgr2=0x7f4ee81992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:36.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.121+0000 7f4edffff640 1 --2- 172.21.15.1:0/2591142995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ee8100a80 0x7f4ee81992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:36.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.121+0000 7f4edffff640 1 -- 172.21.15.1:0/2591142995 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ee81a4890 con 0x7f4ee8101420 2024-09-11T22:05:36.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.121+0000 7f4edffff640 1 --2- 172.21.15.1:0/2591142995 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ee8101420 0x7f4ee8199810 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0009230 tx=0x7f4ed0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:36.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.121+0000 7f4ee5ffb640 1 -- 172.21.15.1:0/2591142995 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed003f030 con 0x7f4ee8101420 2024-09-11T22:05:36.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.121+0000 7f4eee8a6640 1 -- 172.21.15.1:0/2591142995 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ee81a4a60 con 0x7f4ee8101420 2024-09-11T22:05:36.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.121+0000 7f4ee5ffb640 1 -- 172.21.15.1:0/2591142995 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ed0004440 con 0x7f4ee8101420 2024-09-11T22:05:36.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.121+0000 7f4ee5ffb640 1 -- 172.21.15.1:0/2591142995 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed0035070 con 0x7f4ee8101420 2024-09-11T22:05:36.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.122+0000 7f4eee8a6640 1 -- 172.21.15.1:0/2591142995 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ee81a4f10 con 0x7f4ee8101420 2024-09-11T22:05:36.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.123+0000 7f4ee5ffb640 1 -- 172.21.15.1:0/2591142995 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f4ed0042030 con 0x7f4ee8101420 2024-09-11T22:05:36.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.124+0000 7f4ef0c7f740 1 Processor -- start 2024-09-11T22:05:36.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.124+0000 7f4ee5ffb640 1 --2- 172.21.15.1:0/2591142995 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f4ed40a2cf0 0x7f4ed40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:36.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.124+0000 7f4ef0c7f740 1 -- start start 2024-09-11T22:05:36.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.124+0000 7f4ee5ffb640 1 -- 172.21.15.1:0/2591142995 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7f4ed00ef460 con 0x7f4ee8101420 2024-09-11T22:05:36.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.124+0000 7f4ee7fff640 1 --2- 172.21.15.1:0/2591142995 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f4ed40a2cf0 0x7f4ed40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:36.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.124+0000 7f4ef0c7f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd5bc0f060 0x55cd5bc0f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:36.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.124+0000 7f4ef0c7f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd5bc0f980 0x55cd5bc19d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:36.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.124+0000 7f4ef0c7f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd5bc1a240 0x55cd5bc1c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:36.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.125+0000 7f4ef0c7f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55cd5bb74620 con 0x55cd5bc0f980 2024-09-11T22:05:36.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.125+0000 7f4ef0c7f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55cd5bb1be50 con 0x55cd5bc0f060 2024-09-11T22:05:36.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.125+0000 7f4ef0c7f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55cd5bb705e0 con 0x55cd5bc1a240 2024-09-11T22:05:36.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.125+0000 7f4edffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd5bc0f980 0x55cd5bc19d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:36.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.125+0000 7f4edffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd5bc0f980 0x55cd5bc19d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42606/0 (socket says 172.21.15.1:42606) 2024-09-11T22:05:36.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.125+0000 7f4edffff640 1 -- 172.21.15.1:0/982450270 learned_addr learned my addr 172.21.15.1:0/982450270 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:36.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.125+0000 7f4ee7fff640 1 --2- 172.21.15.1:0/982450270 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd5bc0f060 0x55cd5bc0f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:36.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.125+0000 7f4eece43640 1 --2- 172.21.15.1:0/982450270 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd5bc1a240 0x55cd5bc1c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:36.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.125+0000 7f4ee7fff640 1 --2- 172.21.15.1:0/2591142995 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f4ed40a2cf0 0x7f4ed40a51b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4ed8009ba0 tx=0x7f4ed8011440 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:36.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.125+0000 7f4edffff640 1 -- 172.21.15.1:0/982450270 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd5bc1a240 msgr2=0x55cd5bc1c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:36.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.125+0000 7f4edffff640 1 --2- 172.21.15.1:0/982450270 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd5bc1a240 0x55cd5bc1c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:36.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.125+0000 7f4edffff640 1 -- 172.21.15.1:0/982450270 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd5bc0f060 msgr2=0x55cd5bc0f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:36.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.125+0000 7f4edffff640 1 --2- 172.21.15.1:0/982450270 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd5bc0f060 0x55cd5bc0f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:36.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.125+0000 7f4edffff640 1 -- 172.21.15.1:0/982450270 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd5bb0cd40 con 0x55cd5bc0f980 2024-09-11T22:05:36.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.125+0000 7f4ee7fff640 1 --2- 172.21.15.1:0/982450270 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd5bc0f060 0x55cd5bc0f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:05:36.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.125+0000 7f4edffff640 1 --2- 172.21.15.1:0/982450270 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd5bc0f980 0x55cd5bc19d00 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0032c00 tx=0x7f4ed00bb9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31810fe15a6fb1c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:36.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.125+0000 7f4ede7fc640 1 -- 172.21.15.1:0/982450270 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed0049070 con 0x55cd5bc0f980 2024-09-11T22:05:36.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.125+0000 7f4ede7fc640 1 -- 172.21.15.1:0/982450270 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ed00c4580 con 0x55cd5bc0f980 2024-09-11T22:05:36.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.125+0000 7f4ede7fc640 1 -- 172.21.15.1:0/982450270 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed00c42f0 con 0x55cd5bc0f980 2024-09-11T22:05:36.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.126+0000 7f4ef0c7f740 1 -- 172.21.15.1:0/982450270 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd5bc0f980 msgr2=0x55cd5bc19d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:36.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.126+0000 7f4ef0c7f740 1 --2- 172.21.15.1:0/982450270 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd5bc0f980 0x55cd5bc19d00 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0032c00 tx=0x7f4ed00bb9c0 comp rx=0 tx=0).stop 2024-09-11T22:05:36.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.126+0000 7f4ef0c7f740 1 -- 172.21.15.1:0/982450270 shutdown_connections 2024-09-11T22:05:36.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.126+0000 7f4ef0c7f740 1 --2- 172.21.15.1:0/982450270 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd5bc1a240 0x55cd5bc1c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:36.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.126+0000 7f4ef0c7f740 1 --2- 172.21.15.1:0/982450270 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd5bc0f980 0x55cd5bc19d00 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:36.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.126+0000 7f4ef0c7f740 1 --2- 172.21.15.1:0/982450270 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd5bc0f060 0x55cd5bc0f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:36.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.126+0000 7f4ef0c7f740 1 -- 172.21.15.1:0/982450270 >> 172.21.15.1:0/982450270 conn(0x55cd5bc0b5f0 msgr2=0x55cd5bc0b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:36.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.126+0000 7f4ef0c7f740 1 -- 172.21.15.1:0/982450270 shutdown_connections 2024-09-11T22:05:36.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.126+0000 7f4ef0c7f740 1 -- 172.21.15.1:0/982450270 wait complete. 2024-09-11T22:05:36.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.126+0000 7f4ef0c7f740 1 -- start start 2024-09-11T22:05:36.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.127+0000 7f4ef0c7f740 1 Processor -- start 2024-09-11T22:05:36.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.127+0000 7f4ef0c7f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd5bc0f060 0x55cd5bc171e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:36.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.127+0000 7f4ef0c7f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd5bc0f980 0x55cd5bc17720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:36.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.127+0000 7f4ef0c7f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd5bc1a240 0x55cd5bc17c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:36.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.127+0000 7f4ef0c7f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55cd5bc1cbd0 con 0x55cd5bc0f060 2024-09-11T22:05:36.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.127+0000 7f4ef0c7f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55cd5bafcf10 con 0x55cd5bc0f980 2024-09-11T22:05:36.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.127+0000 7f4ef0c7f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55cd5bc1cd50 con 0x55cd5bc1a240 2024-09-11T22:05:36.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.127+0000 7f4ef0c7f740 10 client.0 _ll_register_callbacks cb 0x55cd5bb20db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:05:36.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.127+0000 7f4ee7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd5bc0f060 0x55cd5bc171e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:36.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.127+0000 7f4ee7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd5bc0f060 0x55cd5bc171e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42622/0 (socket says 172.21.15.1:42622) 2024-09-11T22:05:36.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.127+0000 7f4ee7fff640 1 -- 172.21.15.1:0/1057182253 learned_addr learned my addr 172.21.15.1:0/1057182253 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:36.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.127+0000 7f4edffff640 1 --2- 172.21.15.1:0/1057182253 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd5bc0f980 0x55cd5bc17720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:36.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.127+0000 7f4ee7fff640 1 -- 172.21.15.1:0/1057182253 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd5bc1a240 msgr2=0x55cd5bc17c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:36.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.127+0000 7f4eece43640 1 --2- 172.21.15.1:0/1057182253 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd5bc1a240 0x55cd5bc17c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:36.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.127+0000 7f4ee7fff640 1 --2- 172.21.15.1:0/1057182253 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd5bc1a240 0x55cd5bc17c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:36.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.127+0000 7f4ee7fff640 1 -- 172.21.15.1:0/1057182253 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd5bc0f980 msgr2=0x55cd5bc17720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:36.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.127+0000 7f4ee7fff640 1 --2- 172.21.15.1:0/1057182253 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd5bc0f980 0x55cd5bc17720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:36.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.127+0000 7f4ee7fff640 1 -- 172.21.15.1:0/1057182253 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd5b883580 con 0x55cd5bc0f060 2024-09-11T22:05:36.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.127+0000 7f4edffff640 1 --2- 172.21.15.1:0/1057182253 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd5bc0f980 0x55cd5bc17720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:05:36.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.128+0000 7f4ee7fff640 1 --2- 172.21.15.1:0/1057182253 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd5bc0f060 0x55cd5bc171e0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f4ed801e990 tx=0x7f4ed8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32dd0a4dd02b9c05 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:36.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.128+0000 7f4ec77fe640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:05:36.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.128+0000 7f4ec77fe640 1 -- 172.21.15.1:0/1057182253 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed8017040 con 0x55cd5bc0f060 2024-09-11T22:05:36.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.128+0000 7f4ec77fe640 1 -- 172.21.15.1:0/1057182253 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7f4ed8009040 con 0x55cd5bc0f060 2024-09-11T22:05:36.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.128+0000 7f4ef0c7f740 1 -- 172.21.15.1:0/1057182253 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ed00c4580 con 0x55cd5bc0f060 2024-09-11T22:05:36.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.128+0000 7f4ec77fe640 1 -- 172.21.15.1:0/1057182253 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f4ed801e020 con 0x55cd5bc0f060 2024-09-11T22:05:36.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.128+0000 7f4ef0c7f740 10 client.17346 fetch_fsmap learned FSMap version 42 2024-09-11T22:05:36.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.128+0000 7f4ef0c7f740 1 -- 172.21.15.1:0/1057182253 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55cd5b94d960 con 0x55cd5bc0f060 2024-09-11T22:05:36.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.128+0000 7f4ec77fe640 1 -- 172.21.15.1:0/1057182253 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ed00c4580 con 0x55cd5bc0f060 2024-09-11T22:05:36.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.128+0000 7f4ec77fe640 1 -- 172.21.15.1:0/1057182253 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f4ed801d600 con 0x55cd5bc0f060 2024-09-11T22:05:36.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.128+0000 7f4ef0c7f740 10 client.17346 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:05:36.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.128+0000 7f4ef0c7f740 10 client.17346 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-11T22:05:36.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.128+0000 7f4ef0c7f740 20 client.17346 populate_metadata read hostname 'smithi001' 2024-09-11T22:05:36.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.128+0000 7f4ef0c7f740 1 --2- 172.21.15.1:0/1057182253 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55cd5bc29c50 0x55cd5bc2c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:36.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.128+0000 7f4ef0c7f740 4 client.17346 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:05:36.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.128+0000 7f4ef0c7f740 1 -- 172.21.15.1:0/1057182253 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd5bb81b90 con 0x55cd5bc29c50 2024-09-11T22:05:36.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.129+0000 7f4edffff640 1 --2- 172.21.15.1:0/1057182253 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55cd5bc29c50 0x55cd5bc2c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:36.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.129+0000 7f4ec77fe640 10 client.17346 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:05:36.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.129+0000 7f4edffff640 1 --2- 172.21.15.1:0/1057182253 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55cd5bc29c50 0x55cd5bc2c0f0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d63fa43a7f88a0d server_cookie=1f81e43d935815e5 in_seq=0 out_seq=0 2024-09-11T22:05:36.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.130+0000 7f4ec77fe640 1 -- 172.21.15.1:0/1057182253 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cd5bb81b90 con 0x55cd5bc29c50 2024-09-11T22:05:36.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.130+0000 7f4ec77fe640 10 client.17346 handle_command_reply: tid=0 2024-09-11T22:05:36.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.130+0000 7f4ef0c7f740 1 client.17346 shutdown 2024-09-11T22:05:36.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.130+0000 7f4ef0c7f740 1 -- 172.21.15.1:0/1057182253 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd5bc0f060 msgr2=0x55cd5bc171e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:36.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.130+0000 7f4ef0c7f740 1 --2- 172.21.15.1:0/1057182253 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd5bc0f060 0x55cd5bc171e0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f4ed801e990 tx=0x7f4ed8004730 comp rx=0 tx=0).stop 2024-09-11T22:05:36.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.130+0000 7f4ef0c7f740 1 -- 172.21.15.1:0/1057182253 shutdown_connections 2024-09-11T22:05:36.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.130+0000 7f4ef0c7f740 1 --2- 172.21.15.1:0/1057182253 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55cd5bc29c50 0x55cd5bc2c0f0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:36.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.130+0000 7f4ef0c7f740 1 --2- 172.21.15.1:0/1057182253 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd5bc1a240 0x55cd5bc17c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:36.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.130+0000 7f4ef0c7f740 1 --2- 172.21.15.1:0/1057182253 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd5bc0f980 0x55cd5bc17720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:36.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.130+0000 7f4ef0c7f740 1 --2- 172.21.15.1:0/1057182253 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd5bc0f060 0x55cd5bc171e0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:36.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.130+0000 7f4ec77fe640 0 client.17346 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:05:36.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.130+0000 7f4ef0c7f740 1 -- 172.21.15.1:0/1057182253 >> 172.21.15.1:0/1057182253 conn(0x55cd5bc0b5f0 msgr2=0x55cd5bc0a750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:36.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.130+0000 7f4ef0c7f740 1 -- 172.21.15.1:0/1057182253 shutdown_connections 2024-09-11T22:05:36.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.130+0000 7f4ef0c7f740 1 -- 172.21.15.1:0/1057182253 wait complete. 2024-09-11T22:05:36.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.130+0000 7f4ef0c7f740 20 client.17346 trim_cache size 0 max 16384 2024-09-11T22:05:36.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.148+0000 7f4ef0c7f740 1 Processor -- start 2024-09-11T22:05:36.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.148+0000 7f4ef0c7f740 1 -- start start 2024-09-11T22:05:36.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.148+0000 7f4ef0c7f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd5bc0f060 0x55cd5bc0a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:36.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.148+0000 7f4ef0c7f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd5bc0f980 0x55cd5bc22140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:36.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.148+0000 7f4ef0c7f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd5bc1a240 0x55cd5bc22680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:36.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.148+0000 7f4ef0c7f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee81adaa0 con 0x55cd5bc0f980 2024-09-11T22:05:36.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.148+0000 7f4ef0c7f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee81af8e0 con 0x55cd5bc1a240 2024-09-11T22:05:36.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.148+0000 7f4ef0c7f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee81ae230 con 0x55cd5bc0f060 2024-09-11T22:05:36.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.148+0000 7f4edffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd5bc0f980 0x55cd5bc22140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:36.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.148+0000 7f4edffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd5bc0f980 0x55cd5bc22140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42628/0 (socket says 172.21.15.1:42628) 2024-09-11T22:05:36.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.148+0000 7f4edffff640 1 -- 172.21.15.1:0/4108655113 learned_addr learned my addr 172.21.15.1:0/4108655113 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:36.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.148+0000 7f4eece43640 1 --2- 172.21.15.1:0/4108655113 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd5bc1a240 0x55cd5bc22680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:36.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.148+0000 7f4ee7fff640 1 --2- 172.21.15.1:0/4108655113 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd5bc0f060 0x55cd5bc0a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:36.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.148+0000 7f4edffff640 1 -- 172.21.15.1:0/4108655113 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd5bc0f060 msgr2=0x55cd5bc0a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:36.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.148+0000 7f4edffff640 1 --2- 172.21.15.1:0/4108655113 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd5bc0f060 0x55cd5bc0a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:36.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.148+0000 7f4edffff640 1 -- 172.21.15.1:0/4108655113 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd5bc1a240 msgr2=0x55cd5bc22680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:36.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.148+0000 7f4edffff640 1 --2- 172.21.15.1:0/4108655113 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd5bc1a240 0x55cd5bc22680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:36.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.148+0000 7f4edffff640 1 -- 172.21.15.1:0/4108655113 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd5bc19b50 con 0x55cd5bc0f980 2024-09-11T22:05:36.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.148+0000 7f4ee7fff640 1 --2- 172.21.15.1:0/4108655113 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd5bc0f060 0x55cd5bc0a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:05:36.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.148+0000 7f4edffff640 1 --2- 172.21.15.1:0/4108655113 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd5bc0f980 0x55cd5bc22140 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0009900 tx=0x7f4ed00a6040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a744b324818a2ed1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:36.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.149+0000 7f4eddffb640 1 -- 172.21.15.1:0/4108655113 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed00d9070 con 0x55cd5bc0f980 2024-09-11T22:05:36.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.149+0000 7f4eddffb640 1 -- 172.21.15.1:0/4108655113 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ed00c4580 con 0x55cd5bc0f980 2024-09-11T22:05:36.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.149+0000 7f4eddffb640 1 -- 172.21.15.1:0/4108655113 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed00d3030 con 0x55cd5bc0f980 2024-09-11T22:05:36.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.149+0000 7f4ef0c7f740 1 -- 172.21.15.1:0/4108655113 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd5bc0f980 msgr2=0x55cd5bc22140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:36.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.149+0000 7f4ef0c7f740 1 --2- 172.21.15.1:0/4108655113 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd5bc0f980 0x55cd5bc22140 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0009900 tx=0x7f4ed00a6040 comp rx=0 tx=0).stop 2024-09-11T22:05:36.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.149+0000 7f4ef0c7f740 1 -- 172.21.15.1:0/4108655113 shutdown_connections 2024-09-11T22:05:36.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.149+0000 7f4ef0c7f740 1 --2- 172.21.15.1:0/4108655113 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd5bc1a240 0x55cd5bc22680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:36.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.149+0000 7f4ef0c7f740 1 --2- 172.21.15.1:0/4108655113 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd5bc0f980 0x55cd5bc22140 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:36.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.149+0000 7f4ef0c7f740 1 --2- 172.21.15.1:0/4108655113 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd5bc0f060 0x55cd5bc0a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:36.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.149+0000 7f4ef0c7f740 1 -- 172.21.15.1:0/4108655113 >> 172.21.15.1:0/4108655113 conn(0x55cd5bc0b5f0 msgr2=0x55cd5bc09f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:36.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.149+0000 7f4ef0c7f740 1 -- 172.21.15.1:0/4108655113 shutdown_connections 2024-09-11T22:05:36.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.149+0000 7f4ef0c7f740 1 -- 172.21.15.1:0/4108655113 wait complete. 2024-09-11T22:05:36.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.150+0000 7f4ef0c7f740 1 -- start start 2024-09-11T22:05:36.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.150+0000 7f4ef0c7f740 1 Processor -- start 2024-09-11T22:05:36.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.150+0000 7f4ef0c7f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd5bc0f060 0x55cd5bc14d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:36.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.150+0000 7f4ef0c7f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd5bc0f980 0x55cd5bc13450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:36.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.150+0000 7f4ef0c7f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd5bc1a240 0x55cd5bc13990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:36.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.150+0000 7f4ef0c7f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee81af150 con 0x55cd5bc0f980 2024-09-11T22:05:36.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.150+0000 7f4ef0c7f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee81ae9c0 con 0x55cd5bc0f060 2024-09-11T22:05:36.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.150+0000 7f4ef0c7f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee81abaa0 con 0x55cd5bc1a240 2024-09-11T22:05:36.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.150+0000 7f4ef0c7f740 10 client.0 _ll_register_callbacks cb 0x7f4ed801e870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:05:36.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.150+0000 7f4ee7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd5bc0f060 0x55cd5bc14d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:36.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.150+0000 7f4edffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd5bc0f980 0x55cd5bc13450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:36.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.151+0000 7f4ee7fff640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd5bc0f060 0x55cd5bc14d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:58836/0 (socket says 172.21.15.1:58836) 2024-09-11T22:05:36.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.151+0000 7f4ee7fff640 1 -- 172.21.15.1:0/2914029013 learned_addr learned my addr 172.21.15.1:0/2914029013 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:36.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.151+0000 7f4eece43640 1 --2- 172.21.15.1:0/2914029013 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd5bc1a240 0x55cd5bc13990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:36.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.151+0000 7f4edffff640 1 -- 172.21.15.1:0/2914029013 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd5bc1a240 msgr2=0x55cd5bc13990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:36.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.151+0000 7f4edffff640 1 --2- 172.21.15.1:0/2914029013 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd5bc1a240 0x55cd5bc13990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:36.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.151+0000 7f4edffff640 1 -- 172.21.15.1:0/2914029013 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd5bc0f060 msgr2=0x55cd5bc14d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:36.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.151+0000 7f4edffff640 1 --2- 172.21.15.1:0/2914029013 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd5bc0f060 0x55cd5bc14d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:36.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.151+0000 7f4edffff640 1 -- 172.21.15.1:0/2914029013 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd5bc141a0 con 0x55cd5bc0f980 2024-09-11T22:05:36.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.151+0000 7f4eece43640 1 --2- 172.21.15.1:0/2914029013 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd5bc1a240 0x55cd5bc13990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:05:36.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.151+0000 7f4ee7fff640 1 --2- 172.21.15.1:0/2914029013 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd5bc0f060 0x55cd5bc14d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:05:36.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.151+0000 7f4edffff640 1 --2- 172.21.15.1:0/2914029013 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd5bc0f980 0x55cd5bc13450 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0009a50 tx=0x7f4ed00ecb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3cf70eca0c37616 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:36.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.151+0000 7f4ec77fe640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:05:36.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.151+0000 7f4ec77fe640 1 -- 172.21.15.1:0/2914029013 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed009f060 con 0x55cd5bc0f980 2024-09-11T22:05:36.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.151+0000 7f4ec77fe640 1 -- 172.21.15.1:0/2914029013 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7f4ed00c4240 con 0x55cd5bc0f980 2024-09-11T22:05:36.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.151+0000 7f4ef0c7f740 1 -- 172.21.15.1:0/2914029013 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ed00c4580 con 0x55cd5bc0f980 2024-09-11T22:05:36.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.151+0000 7f4ec77fe640 1 -- 172.21.15.1:0/2914029013 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f4ed00d3070 con 0x55cd5bc0f980 2024-09-11T22:05:36.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.151+0000 7f4ef0c7f740 10 client.17352 fetch_fsmap learned FSMap version 42 2024-09-11T22:05:36.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.151+0000 7f4ef0c7f740 1 -- 172.21.15.1:0/2914029013 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55cd5bc141a0 con 0x55cd5bc0f980 2024-09-11T22:05:36.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.152+0000 7f4ec77fe640 1 -- 172.21.15.1:0/2914029013 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ed00c4580 con 0x55cd5bc0f980 2024-09-11T22:05:36.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.152+0000 7f4ec77fe640 1 -- 172.21.15.1:0/2914029013 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f4ed004c060 con 0x55cd5bc0f980 2024-09-11T22:05:36.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.152+0000 7f4ef0c7f740 10 client.17352 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:05:36.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.152+0000 7f4ef0c7f740 10 client.17352 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-11T22:05:36.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.152+0000 7f4ef0c7f740 20 client.17352 populate_metadata read hostname 'smithi001' 2024-09-11T22:05:36.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.152+0000 7f4ef0c7f740 1 --2- 172.21.15.1:0/2914029013 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55cd5bc99ad0 0x55cd5bc9bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:36.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.152+0000 7f4ef0c7f740 4 client.17352 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "f8d2564c", "release": true, "await": true}] 2024-09-11T22:05:36.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.152+0000 7f4ef0c7f740 1 -- 172.21.15.1:0/2914029013 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "quiesce db", "set_id": "f8d2564c", "release": true, "await": true}) -- 0x55cd5bb7c0a0 con 0x55cd5bc99ad0 2024-09-11T22:05:36.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.152+0000 7f4ee7fff640 1 --2- 172.21.15.1:0/2914029013 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55cd5bc99ad0 0x55cd5bc9bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:36.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.152+0000 7f4ee7fff640 1 --2- 172.21.15.1:0/2914029013 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55cd5bc99ad0 0x55cd5bc9bec0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e474673e9a7e900a server_cookie=6af09866878a3711 in_seq=0 out_seq=0 2024-09-11T22:05:36.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.152+0000 7f4ec77fe640 10 client.17352 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:05:36.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.155+0000 7f4ec77fe640 1 -- 172.21.15.1:0/2914029013 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55cd5bb7c0a0 con 0x55cd5bc99ad0 2024-09-11T22:05:36.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.155+0000 7f4ec77fe640 10 client.17352 handle_command_reply: tid=0 2024-09-11T22:05:36.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.155+0000 7f4ef0c7f740 1 client.17352 shutdown 2024-09-11T22:05:36.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.155+0000 7f4ef0c7f740 1 -- 172.21.15.1:0/2914029013 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd5bc0f980 msgr2=0x55cd5bc13450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:36.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.155+0000 7f4ef0c7f740 1 --2- 172.21.15.1:0/2914029013 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd5bc0f980 0x55cd5bc13450 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0009a50 tx=0x7f4ed00ecb60 comp rx=0 tx=0).stop 2024-09-11T22:05:36.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.155+0000 7f4ef0c7f740 1 -- 172.21.15.1:0/2914029013 shutdown_connections 2024-09-11T22:05:36.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.155+0000 7f4ef0c7f740 1 --2- 172.21.15.1:0/2914029013 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55cd5bc99ad0 0x55cd5bc9bec0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:36.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.155+0000 7f4ef0c7f740 1 --2- 172.21.15.1:0/2914029013 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd5bc1a240 0x55cd5bc13990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:36.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.155+0000 7f4ef0c7f740 1 --2- 172.21.15.1:0/2914029013 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd5bc0f980 0x55cd5bc13450 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:36.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.155+0000 7f4ef0c7f740 1 --2- 172.21.15.1:0/2914029013 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd5bc0f060 0x55cd5bc14d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:36.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.155+0000 7f4ef0c7f740 1 -- 172.21.15.1:0/2914029013 >> 172.21.15.1:0/2914029013 conn(0x55cd5bc0b5f0 msgr2=0x55cd5bc0a750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:36.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.155+0000 7f4ec77fe640 0 client.17352 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:05:36.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.155+0000 7f4ef0c7f740 1 -- 172.21.15.1:0/2914029013 shutdown_connections 2024-09-11T22:05:36.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.155+0000 7f4ef0c7f740 1 -- 172.21.15.1:0/2914029013 wait complete. 2024-09-11T22:05:36.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.156+0000 7f4ef0c7f740 20 client.17352 trim_cache size 0 max 16384 2024-09-11T22:05:36.187 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-11T22:05:36.187 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 42, 2024-09-11T22:05:36.188 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24494, 2024-09-11T22:05:36.188 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 28, 2024-09-11T22:05:36.188 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-09-11T22:05:36.188 INFO:teuthology.orchestra.run.smithi001.stdout: "f8d2564c": { 2024-09-11T22:05:36.188 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 28, 2024-09-11T22:05:36.189 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-09-11T22:05:36.189 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-11T22:05:36.189 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RELEASED", 2024-09-11T22:05:36.189 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-11T22:05:36.189 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:05:36.190 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 90.0, 2024-09-11T22:05:36.190 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 136.6, 2024-09-11T22:05:36.190 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-09-11T22:05:36.190 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-09-11T22:05:36.191 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-09-11T22:05:36.191 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-11T22:05:36.191 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RELEASED", 2024-09-11T22:05:36.191 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-11T22:05:36.191 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:05:36.192 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:05:36.192 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:05:36.192 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:05:36.192 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:05:36.192 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-11T22:05:36.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.156+0000 7f4eee8a6640 1 -- 172.21.15.1:0/2591142995 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f4ed40a2cf0 msgr2=0x7f4ed40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:36.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.156+0000 7f4eee8a6640 1 --2- 172.21.15.1:0/2591142995 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f4ed40a2cf0 0x7f4ed40a51b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4ed8009ba0 tx=0x7f4ed8011440 comp rx=0 tx=0).stop 2024-09-11T22:05:36.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.156+0000 7f4eee8a6640 1 -- 172.21.15.1:0/2591142995 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ee8101420 msgr2=0x7f4ee8199810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:36.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.156+0000 7f4eee8a6640 1 --2- 172.21.15.1:0/2591142995 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ee8101420 0x7f4ee8199810 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0009230 tx=0x7f4ed0004040 comp rx=0 tx=0).stop 2024-09-11T22:05:36.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.156+0000 7f4eee8a6640 1 -- 172.21.15.1:0/2591142995 shutdown_connections 2024-09-11T22:05:36.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.156+0000 7f4eee8a6640 1 --2- 172.21.15.1:0/2591142995 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f4ed40a2cf0 0x7f4ed40a51b0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:36.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.156+0000 7f4eee8a6640 1 --2- 172.21.15.1:0/2591142995 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ee8106a80 0x7f4ee81a4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:36.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.156+0000 7f4eee8a6640 1 --2- 172.21.15.1:0/2591142995 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ee8101420 0x7f4ee8199810 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:36.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.156+0000 7f4eee8a6640 1 --2- 172.21.15.1:0/2591142995 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4ee8100a80 0x7f4ee81992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:36.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.156+0000 7f4eee8a6640 1 -- 172.21.15.1:0/2591142995 >> 172.21.15.1:0/2591142995 conn(0x7f4ee80fc7a0 msgr2=0x7f4ee80fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:36.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.156+0000 7f4eee8a6640 1 -- 172.21.15.1:0/2591142995 shutdown_connections 2024-09-11T22:05:36.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:36.156+0000 7f4eee8a6640 1 -- 172.21.15.1:0/2591142995 wait complete. 2024-09-11T22:05:36.195 INFO:tasks.workunit.client.0.smithi001.stderr:+ datapool= 2024-09-11T22:05:36.195 INFO:tasks.workunit.client.0.smithi001.stderr:+ dir=. 2024-09-11T22:05:36.195 INFO:tasks.workunit.client.0.smithi001.stdout:. 2024-09-11T22:05:36.196 INFO:tasks.workunit.client.0.smithi001.stderr:+ true 2024-09-11T22:05:36.196 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo . 2024-09-11T22:05:36.196 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfattr -n ceph.dir.layout.pool . --only-values 2024-09-11T22:05:36.197 INFO:tasks.workunit.client.0.smithi001.stderr:.: ceph.dir.layout.pool: No such attribute 2024-09-11T22:05:36.197 INFO:tasks.workunit.client.0.smithi001.stderr:+ datapool= 2024-09-11T22:05:36.197 INFO:tasks.workunit.client.0.smithi001.stderr:+ dir=./.. 2024-09-11T22:05:36.197 INFO:tasks.workunit.client.0.smithi001.stderr:+ true 2024-09-11T22:05:36.197 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo ./.. 2024-09-11T22:05:36.197 INFO:tasks.workunit.client.0.smithi001.stdout:./.. 2024-09-11T22:05:36.198 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfattr -n ceph.dir.layout.pool ./.. --only-values 2024-09-11T22:05:36.198 INFO:tasks.workunit.client.0.smithi001.stderr:./..: ceph.dir.layout.pool: No such attribute 2024-09-11T22:05:36.198 INFO:tasks.workunit.client.0.smithi001.stderr:+ datapool= 2024-09-11T22:05:36.198 INFO:tasks.workunit.client.0.smithi001.stderr:+ dir=./../.. 2024-09-11T22:05:36.198 INFO:tasks.workunit.client.0.smithi001.stdout:./../.. 2024-09-11T22:05:36.199 INFO:tasks.workunit.client.0.smithi001.stderr:+ true 2024-09-11T22:05:36.199 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo ./../.. 2024-09-11T22:05:36.199 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfattr -n ceph.dir.layout.pool ./../.. --only-values 2024-09-11T22:05:36.199 INFO:tasks.workunit.client.0.smithi001.stderr:+ datapool=cephfs_data 2024-09-11T22:05:36.199 INFO:tasks.workunit.client.0.smithi001.stderr:+ break 2024-09-11T22:05:36.199 INFO:tasks.workunit.client.0.smithi001.stderr:+ rm -f file file2 2024-09-11T22:05:36.199 INFO:tasks.workunit.client.0.smithi001.stderr:+ touch file file2 2024-09-11T22:05:36.199 INFO:tasks.workunit.client.0.smithi001.stderr:+ getfattr -n ceph.file.layout file 2024-09-11T22:05:36.200 INFO:tasks.workunit.client.0.smithi001.stdout:# file: file 2024-09-11T22:05:36.200 INFO:tasks.workunit.client.0.smithi001.stdout:ceph.file.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data" 2024-09-11T22:05:36.200 INFO:tasks.workunit.client.0.smithi001.stdout: 2024-09-11T22:05:36.201 INFO:tasks.workunit.client.0.smithi001.stderr:+ getfattr -n ceph.file.layout file 2024-09-11T22:05:36.201 INFO:tasks.workunit.client.0.smithi001.stderr:+ grep -q object_size= 2024-09-11T22:05:36.201 INFO:tasks.workunit.client.0.smithi001.stderr:+ getfattr -n ceph.file.layout file 2024-09-11T22:05:36.201 INFO:tasks.workunit.client.0.smithi001.stderr:+ grep -q stripe_count= 2024-09-11T22:05:36.201 INFO:tasks.workunit.client.0.smithi001.stderr:+ getfattr -n ceph.file.layout file 2024-09-11T22:05:36.201 INFO:tasks.workunit.client.0.smithi001.stderr:+ grep -q stripe_unit= 2024-09-11T22:05:36.201 INFO:tasks.workunit.client.0.smithi001.stderr:+ getfattr -n ceph.file.layout file 2024-09-11T22:05:36.201 INFO:tasks.workunit.client.0.smithi001.stderr:+ grep -q pool= 2024-09-11T22:05:36.202 INFO:tasks.workunit.client.0.smithi001.stderr:+ getfattr -n ceph.file.layout.pool file 2024-09-11T22:05:36.202 INFO:tasks.workunit.client.0.smithi001.stdout:# file: file 2024-09-11T22:05:36.202 INFO:tasks.workunit.client.0.smithi001.stdout:ceph.file.layout.pool="cephfs_data" 2024-09-11T22:05:36.203 INFO:tasks.workunit.client.0.smithi001.stdout: 2024-09-11T22:05:36.203 INFO:tasks.workunit.client.0.smithi001.stderr:+ getfattr -n ceph.file.layout.pool_namespace file 2024-09-11T22:05:36.204 INFO:tasks.workunit.client.0.smithi001.stdout:# file: file 2024-09-11T22:05:36.204 INFO:tasks.workunit.client.0.smithi001.stdout:ceph.file.layout.pool_namespace="" 2024-09-11T22:05:36.204 INFO:tasks.workunit.client.0.smithi001.stdout: 2024-09-11T22:05:36.204 INFO:tasks.workunit.client.0.smithi001.stderr:+ getfattr -n ceph.file.layout.stripe_unit file 2024-09-11T22:05:36.205 INFO:tasks.workunit.client.0.smithi001.stdout:# file: file 2024-09-11T22:05:36.205 INFO:tasks.workunit.client.0.smithi001.stdout:ceph.file.layout.stripe_unit="4194304" 2024-09-11T22:05:36.205 INFO:tasks.workunit.client.0.smithi001.stdout: 2024-09-11T22:05:36.206 INFO:tasks.workunit.client.0.smithi001.stderr:+ getfattr -n ceph.file.layout.stripe_count file 2024-09-11T22:05:36.207 INFO:tasks.workunit.client.0.smithi001.stdout:# file: file 2024-09-11T22:05:36.207 INFO:tasks.workunit.client.0.smithi001.stdout:ceph.file.layout.stripe_count="1" 2024-09-11T22:05:36.207 INFO:tasks.workunit.client.0.smithi001.stdout: 2024-09-11T22:05:36.207 INFO:tasks.workunit.client.0.smithi001.stderr:+ getfattr -n ceph.file.layout.object_size file 2024-09-11T22:05:36.208 INFO:tasks.workunit.client.0.smithi001.stdout:# file: file 2024-09-11T22:05:36.208 INFO:tasks.workunit.client.0.smithi001.stdout:ceph.file.layout.object_size="4194304" 2024-09-11T22:05:36.208 INFO:tasks.workunit.client.0.smithi001.stdout: 2024-09-11T22:05:36.209 INFO:tasks.workunit.client.0.smithi001.stderr:+ getfattr -n ceph.file.layout.bogus file 2024-09-11T22:05:36.209 INFO:tasks.workunit.client.0.smithi001.stderr:+ grep -q 'No such attribute' 2024-09-11T22:05:36.210 INFO:tasks.workunit.client.0.smithi001.stderr:+ getfattr -n ceph.dir.layout file 2024-09-11T22:05:36.210 INFO:tasks.workunit.client.0.smithi001.stderr:+ grep -q 'No such attribute' 2024-09-11T22:05:36.211 INFO:tasks.workunit.client.0.smithi001.stderr:+ setfattr -n ceph.file.layout.stripe_unit -v 1048576 file2 2024-09-11T22:05:36.216 INFO:tasks.workunit.client.0.smithi001.stderr:+ setfattr -n ceph.file.layout.stripe_count -v 8 file2 2024-09-11T22:05:36.220 INFO:tasks.workunit.client.0.smithi001.stderr:+ setfattr -n ceph.file.layout.object_size -v 10485760 file2 2024-09-11T22:05:36.226 INFO:tasks.workunit.client.0.smithi001.stderr:+ setfattr -n ceph.file.layout.pool -v cephfs_data file2 2024-09-11T22:05:36.234 INFO:tasks.workunit.client.0.smithi001.stderr:+ getfattr -n ceph.file.layout.pool file2 2024-09-11T22:05:36.234 INFO:tasks.workunit.client.0.smithi001.stderr:+ grep -q cephfs_data 2024-09-11T22:05:36.235 INFO:tasks.workunit.client.0.smithi001.stderr:+ setfattr -n ceph.file.layout.pool_namespace -v foons file2 2024-09-11T22:05:36.239 INFO:tasks.workunit.client.0.smithi001.stderr:+ getfattr -n ceph.file.layout.pool_namespace file2 2024-09-11T22:05:36.239 INFO:tasks.workunit.client.0.smithi001.stderr:+ grep -q foons 2024-09-11T22:05:36.240 INFO:tasks.workunit.client.0.smithi001.stderr:+ setfattr -x ceph.file.layout.pool_namespace file2 2024-09-11T22:05:36.243 INFO:tasks.workunit.client.0.smithi001.stderr:+ getfattr -n ceph.file.layout.pool_namespace file2 2024-09-11T22:05:36.244 INFO:tasks.workunit.client.0.smithi001.stderr:+ grep -q -v foons 2024-09-11T22:05:36.245 INFO:tasks.workunit.client.0.smithi001.stderr:+ getfattr -n ceph.file.layout.stripe_unit file2 2024-09-11T22:05:36.245 INFO:tasks.workunit.client.0.smithi001.stderr:+ grep -q 1048576 2024-09-11T22:05:36.246 INFO:tasks.workunit.client.0.smithi001.stderr:+ getfattr -n ceph.file.layout.stripe_count file2 2024-09-11T22:05:36.246 INFO:tasks.workunit.client.0.smithi001.stderr:+ grep -q 8 2024-09-11T22:05:36.248 INFO:tasks.workunit.client.0.smithi001.stderr:+ getfattr -n ceph.file.layout.object_size file2 2024-09-11T22:05:36.248 INFO:tasks.workunit.client.0.smithi001.stderr:+ grep -q 10485760 2024-09-11T22:05:36.249 INFO:tasks.workunit.client.0.smithi001.stderr:+ setfattr -n ceph.file.layout -v 'stripe_unit=4194304 stripe_count=16 object_size=41943040 pool=cephfs_data pool_namespace=foons' file2 2024-09-11T22:05:36.254 INFO:tasks.workunit.client.0.smithi001.stderr:+ getfattr -n ceph.file.layout.stripe_unit file2 2024-09-11T22:05:36.254 INFO:tasks.workunit.client.0.smithi001.stderr:+ grep -q 4194304 2024-09-11T22:05:36.255 INFO:tasks.workunit.client.0.smithi001.stderr:+ getfattr -n ceph.file.layout.stripe_count file2 2024-09-11T22:05:36.255 INFO:tasks.workunit.client.0.smithi001.stderr:+ grep -q 16 2024-09-11T22:05:36.256 INFO:tasks.workunit.client.0.smithi001.stderr:+ getfattr -n ceph.file.layout.object_size file2 2024-09-11T22:05:36.257 INFO:tasks.workunit.client.0.smithi001.stderr:+ grep -q 41943040 2024-09-11T22:05:36.258 INFO:tasks.workunit.client.0.smithi001.stderr:+ getfattr -n ceph.file.layout.pool file2 2024-09-11T22:05:36.258 INFO:tasks.workunit.client.0.smithi001.stderr:+ grep -q cephfs_data 2024-09-11T22:05:36.259 INFO:tasks.workunit.client.0.smithi001.stderr:+ getfattr -n ceph.file.layout.pool_namespace file2 2024-09-11T22:05:36.259 INFO:tasks.workunit.client.0.smithi001.stderr:+ grep -q foons 2024-09-11T22:05:36.260 INFO:tasks.workunit.client.0.smithi001.stderr:+ setfattr -n ceph.file.layout -v stripe_unit=1048576 file2 2024-09-11T22:05:36.267 INFO:tasks.workunit.client.0.smithi001.stderr:+ getfattr -n ceph.file.layout.stripe_unit file2 2024-09-11T22:05:36.267 INFO:tasks.workunit.client.0.smithi001.stderr:+ grep -q 1048576 2024-09-11T22:05:36.269 INFO:tasks.workunit.client.0.smithi001.stderr:+ getfattr -n ceph.file.layout.stripe_count file2 2024-09-11T22:05:36.269 INFO:tasks.workunit.client.0.smithi001.stderr:+ grep -q 16 2024-09-11T22:05:36.271 INFO:tasks.workunit.client.0.smithi001.stderr:+ getfattr -n ceph.file.layout.object_size file2 2024-09-11T22:05:36.271 INFO:tasks.workunit.client.0.smithi001.stderr:+ grep -q 41943040 2024-09-11T22:05:36.272 INFO:tasks.workunit.client.0.smithi001.stderr:+ getfattr -n ceph.file.layout.pool file2 2024-09-11T22:05:36.272 INFO:tasks.workunit.client.0.smithi001.stderr:+ grep -q cephfs_data 2024-09-11T22:05:36.273 INFO:tasks.workunit.client.0.smithi001.stderr:+ getfattr -n ceph.file.layout.pool_namespace file2 2024-09-11T22:05:36.274 INFO:tasks.workunit.client.0.smithi001.stderr:+ grep -q foons 2024-09-11T22:05:36.275 INFO:tasks.workunit.client.0.smithi001.stderr:+ setfattr -n ceph.file.layout -v 'stripe_unit=2097152 stripe_count=4 object_size=2097152 pool=cephfs_data pool_namespace=barns' file2 2024-09-11T22:05:36.279 INFO:tasks.workunit.client.0.smithi001.stderr:+ getfattr -n ceph.file.layout.stripe_unit file2 2024-09-11T22:05:36.279 INFO:tasks.workunit.client.0.smithi001.stderr:+ grep -q 2097152 2024-09-11T22:05:36.281 INFO:tasks.workunit.client.0.smithi001.stderr:+ getfattr -n ceph.file.layout.stripe_count file2 2024-09-11T22:05:36.281 INFO:tasks.workunit.client.0.smithi001.stderr:+ grep -q 4 2024-09-11T22:05:36.282 INFO:tasks.workunit.client.0.smithi001.stderr:+ getfattr -n ceph.file.layout.object_size file2 2024-09-11T22:05:36.282 INFO:tasks.workunit.client.0.smithi001.stderr:+ grep -q 2097152 2024-09-11T22:05:36.284 INFO:tasks.workunit.client.0.smithi001.stderr:+ getfattr -n ceph.file.layout.pool file2 2024-09-11T22:05:36.284 INFO:tasks.workunit.client.0.smithi001.stderr:+ grep -q cephfs_data 2024-09-11T22:05:36.285 INFO:tasks.workunit.client.0.smithi001.stderr:+ getfattr -n ceph.file.layout.pool_namespace file2 2024-09-11T22:05:36.285 INFO:tasks.workunit.client.0.smithi001.stderr:+ grep -q barns 2024-09-11T22:05:36.287 INFO:tasks.workunit.client.0.smithi001.stderr:+ rm -f dir/file 2024-09-11T22:05:36.287 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir dir 2024-09-11T22:05:36.288 INFO:tasks.workunit.client.0.smithi001.stderr:rmdir: failed to remove 'dir': No such file or directory 2024-09-11T22:05:36.288 INFO:tasks.workunit.client.0.smithi001.stderr:+ true 2024-09-11T22:05:36.288 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir -p dir 2024-09-11T22:05:36.290 INFO:tasks.workunit.client.0.smithi001.stderr:+ getfattr -d -m - dir 2024-09-11T22:05:36.291 INFO:tasks.workunit.client.0.smithi001.stderr:+ grep -q ceph.dir.layout 2024-09-11T22:05:36.292 INFO:tasks.workunit.client.0.smithi001.stderr:+ true 2024-09-11T22:05:36.292 INFO:tasks.workunit.client.0.smithi001.stderr:+ getfattr -d -m - dir 2024-09-11T22:05:36.292 INFO:tasks.workunit.client.0.smithi001.stderr:+ grep -q ceph.file.layout 2024-09-11T22:05:36.293 INFO:tasks.workunit.client.0.smithi001.stderr:+ true 2024-09-11T22:05:36.293 INFO:tasks.workunit.client.0.smithi001.stderr:+ getfattr -n ceph.dir.layout dir 2024-09-11T22:05:36.294 INFO:tasks.workunit.client.0.smithi001.stderr:dir: ceph.dir.layout: No such attribute 2024-09-11T22:05:36.294 INFO:tasks.workunit.client.0.smithi001.stderr:+ true 2024-09-11T22:05:36.294 INFO:tasks.workunit.client.0.smithi001.stderr:+ setfattr -n ceph.dir.layout.stripe_unit -v 1048576 dir 2024-09-11T22:05:36.298 INFO:tasks.workunit.client.0.smithi001.stderr:+ setfattr -n ceph.dir.layout.stripe_count -v 8 dir 2024-09-11T22:05:36.302 INFO:tasks.workunit.client.0.smithi001.stderr:+ setfattr -n ceph.dir.layout.object_size -v 10485760 dir 2024-09-11T22:05:36.306 INFO:tasks.workunit.client.0.smithi001.stderr:+ setfattr -n ceph.dir.layout.pool -v cephfs_data dir 2024-09-11T22:05:36.310 INFO:tasks.workunit.client.0.smithi001.stderr:+ setfattr -n ceph.dir.layout.pool_namespace -v dirns dir 2024-09-11T22:05:36.314 INFO:tasks.workunit.client.0.smithi001.stderr:+ getfattr -n ceph.dir.layout dir 2024-09-11T22:05:36.315 INFO:tasks.workunit.client.0.smithi001.stdout:# file: dir 2024-09-11T22:05:36.315 INFO:tasks.workunit.client.0.smithi001.stdout:ceph.dir.layout="stripe_unit=1048576 stripe_count=8 object_size=10485760 pool=cephfs_data pool_namespace=dirns" 2024-09-11T22:05:36.315 INFO:tasks.workunit.client.0.smithi001.stdout: 2024-09-11T22:05:36.315 INFO:tasks.workunit.client.0.smithi001.stderr:+ getfattr -n ceph.dir.layout dir 2024-09-11T22:05:36.316 INFO:tasks.workunit.client.0.smithi001.stderr:+ grep -q object_size=10485760 2024-09-11T22:05:36.317 INFO:tasks.workunit.client.0.smithi001.stderr:+ getfattr -n ceph.dir.layout dir 2024-09-11T22:05:36.317 INFO:tasks.workunit.client.0.smithi001.stderr:+ grep -q stripe_count=8 2024-09-11T22:05:36.318 INFO:tasks.workunit.client.0.smithi001.stderr:+ getfattr -n ceph.dir.layout dir 2024-09-11T22:05:36.318 INFO:tasks.workunit.client.0.smithi001.stderr:+ grep -q stripe_unit=1048576 2024-09-11T22:05:36.319 INFO:tasks.workunit.client.0.smithi001.stderr:+ getfattr -n ceph.dir.layout dir 2024-09-11T22:05:36.319 INFO:tasks.workunit.client.0.smithi001.stderr:+ grep -q pool=cephfs_data 2024-09-11T22:05:36.320 INFO:tasks.workunit.client.0.smithi001.stderr:+ getfattr -n ceph.dir.layout dir 2024-09-11T22:05:36.320 INFO:tasks.workunit.client.0.smithi001.stderr:+ grep -q pool_namespace=dirns 2024-09-11T22:05:36.322 INFO:tasks.workunit.client.0.smithi001.stderr:+ getfattr -n ceph.dir.layout.pool dir 2024-09-11T22:05:36.322 INFO:tasks.workunit.client.0.smithi001.stderr:+ grep -q cephfs_data 2024-09-11T22:05:36.323 INFO:tasks.workunit.client.0.smithi001.stderr:+ getfattr -n ceph.dir.layout.stripe_unit dir 2024-09-11T22:05:36.323 INFO:tasks.workunit.client.0.smithi001.stderr:+ grep -q 1048576 2024-09-11T22:05:36.324 INFO:tasks.workunit.client.0.smithi001.stderr:+ getfattr -n ceph.dir.layout.stripe_count dir 2024-09-11T22:05:36.324 INFO:tasks.workunit.client.0.smithi001.stderr:+ grep -q 8 2024-09-11T22:05:36.325 INFO:tasks.workunit.client.0.smithi001.stderr:+ getfattr -n ceph.dir.layout.object_size dir 2024-09-11T22:05:36.325 INFO:tasks.workunit.client.0.smithi001.stderr:+ grep -q 10485760 2024-09-11T22:05:36.327 INFO:tasks.workunit.client.0.smithi001.stderr:+ getfattr -n ceph.dir.layout.pool_namespace dir 2024-09-11T22:05:36.327 INFO:tasks.workunit.client.0.smithi001.stderr:+ grep -q dirns 2024-09-11T22:05:36.328 INFO:tasks.workunit.client.0.smithi001.stderr:+ setfattr -n ceph.file.layout -v stripe_count=16 file2 2024-09-11T22:05:36.329 INFO:tasks.workunit.client.0.smithi001.stderr:+ getfattr -n ceph.file.layout.stripe_count file2 2024-09-11T22:05:36.329 INFO:tasks.workunit.client.0.smithi001.stderr:+ grep -q 16 2024-09-11T22:05:36.330 INFO:tasks.workunit.client.0.smithi001.stderr:+ setfattr -n ceph.file.layout -v 'object_size=10485760 stripe_count=8 stripe_unit=1048576 pool=cephfs_data pool_namespace=dirns' file2 2024-09-11T22:05:36.335 INFO:tasks.workunit.client.0.smithi001.stderr:+ getfattr -n ceph.file.layout.stripe_count file2 2024-09-11T22:05:36.335 INFO:tasks.workunit.client.0.smithi001.stderr:+ grep -q 8 2024-09-11T22:05:36.336 INFO:tasks.workunit.client.0.smithi001.stderr:+ touch dir/file 2024-09-11T22:05:36.337 INFO:tasks.workunit.client.0.smithi001.stderr:+ getfattr -n ceph.file.layout.pool dir/file 2024-09-11T22:05:36.338 INFO:tasks.workunit.client.0.smithi001.stderr:+ grep -q cephfs_data 2024-09-11T22:05:36.339 INFO:tasks.workunit.client.0.smithi001.stderr:+ getfattr -n ceph.file.layout.stripe_unit dir/file 2024-09-11T22:05:36.339 INFO:tasks.workunit.client.0.smithi001.stderr:+ grep -q 1048576 2024-09-11T22:05:36.340 INFO:tasks.workunit.client.0.smithi001.stderr:+ getfattr -n ceph.file.layout.stripe_count dir/file 2024-09-11T22:05:36.340 INFO:tasks.workunit.client.0.smithi001.stderr:+ grep -q 8 2024-09-11T22:05:36.341 INFO:tasks.workunit.client.0.smithi001.stderr:+ getfattr -n ceph.file.layout.object_size dir/file 2024-09-11T22:05:36.341 INFO:tasks.workunit.client.0.smithi001.stderr:+ grep -q 10485760 2024-09-11T22:05:36.343 INFO:tasks.workunit.client.0.smithi001.stderr:+ getfattr -n ceph.file.layout.pool_namespace dir/file 2024-09-11T22:05:36.343 INFO:tasks.workunit.client.0.smithi001.stderr:+ grep -q dirns 2024-09-11T22:05:36.343 INFO:tasks.workunit.client.0.smithi001.stderr:+ setfattr -x ceph.dir.layout.pool_namespace dir 2024-09-11T22:05:36.350 INFO:tasks.workunit.client.0.smithi001.stderr:+ getfattr -n ceph.dir.layout dir 2024-09-11T22:05:36.350 INFO:tasks.workunit.client.0.smithi001.stderr:+ grep -q -v pool_namespace=dirns 2024-09-11T22:05:36.351 INFO:tasks.workunit.client.0.smithi001.stderr:+ setfattr -x ceph.dir.layout dir 2024-09-11T22:05:36.355 INFO:tasks.workunit.client.0.smithi001.stderr:+ getfattr -n ceph.dir.layout dir 2024-09-11T22:05:36.355 INFO:tasks.workunit.client.0.smithi001.stderr:+ grep -q 'No such attribute' 2024-09-11T22:05:36.356 INFO:tasks.workunit.client.0.smithi001.stdout:OK 2024-09-11T22:05:36.356 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo OK 2024-09-11T22:05:36.357 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-11T22:05:36.357 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-11T22:05:36.407 INFO:tasks.workunit:Running workunit fs/misc/mkpool_layout_vxattrs.sh... 2024-09-11T22:05:36.408 DEBUG:teuthology.orchestra.run.smithi001:workunit test fs/misc/mkpool_layout_vxattrs.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=086c737fa75c22d521a3932c59b31c5e1d1c9ef4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/mkpool_layout_vxattrs.sh 2024-09-11T22:05:36.837 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'f8d2564c', total seconds elapsed: 19.9 2024-09-11T22:05:37.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:37 smithi161 ceph-mon[35615]: pgmap v258: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 20 op/s 2024-09-11T22:05:37.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:37 smithi001 ceph-mon[30935]: pgmap v258: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 20 op/s 2024-09-11T22:05:37.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:37 smithi053 ceph-mon[34908]: pgmap v258: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 20 op/s 2024-09-11T22:05:38.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:38 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:05:38.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:38 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:05:38.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:38 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:05:38.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:38 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T22:05:38.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:38 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:05:38.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:38 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:05:38.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:38 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:05:38.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:38 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T22:05:38.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:38 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:05:38.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:38 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:05:38.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:38 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:05:38.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:38 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T22:05:39.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:39 smithi161 ceph-mon[35615]: pgmap v259: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 6.2 KiB/s wr, 11 op/s 2024-09-11T22:05:39.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:39 smithi001 ceph-mon[30935]: pgmap v259: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 6.2 KiB/s wr, 11 op/s 2024-09-11T22:05:39.448 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:39 smithi053 ceph-mon[34908]: pgmap v259: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 6.2 KiB/s wr, 11 op/s 2024-09-11T22:05:39.843 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:05:40.116 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:05:40.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.616+0000 7f2611fc4640 1 -- 172.21.15.1:0/2508813493 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f260c102ef0 msgr2=0x7f260c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:40.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.617+0000 7f2611fc4640 1 --2- 172.21.15.1:0/2508813493 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f260c102ef0 0x7f260c10da40 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f2604009ca0 tx=0x7f260402a950 comp rx=0 tx=0).stop 2024-09-11T22:05:40.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.617+0000 7f2611fc4640 1 -- 172.21.15.1:0/2508813493 shutdown_connections 2024-09-11T22:05:40.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.617+0000 7f2611fc4640 1 --2- 172.21.15.1:0/2508813493 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f260c102ef0 0x7f260c10da40 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:40.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.617+0000 7f2611fc4640 1 --2- 172.21.15.1:0/2508813493 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f260c102570 0x7f260c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:40.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.617+0000 7f2611fc4640 1 --2- 172.21.15.1:0/2508813493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f260c100e20 0x7f260c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:40.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.617+0000 7f2611fc4640 1 -- 172.21.15.1:0/2508813493 >> 172.21.15.1:0/2508813493 conn(0x7f260c0fc7d0 msgr2=0x7f260c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:40.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.617+0000 7f2611fc4640 1 -- 172.21.15.1:0/2508813493 shutdown_connections 2024-09-11T22:05:40.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.617+0000 7f2611fc4640 1 -- 172.21.15.1:0/2508813493 wait complete. 2024-09-11T22:05:40.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.617+0000 7f2611fc4640 1 Processor -- start 2024-09-11T22:05:40.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.618+0000 7f2611fc4640 1 -- start start 2024-09-11T22:05:40.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.618+0000 7f2611fc4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f260c100e20 0x7f260c198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:40.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.618+0000 7f2611fc4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f260c102570 0x7f260c1989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:40.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.618+0000 7f2611fc4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f260c102ef0 0x7f260c19cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:40.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.618+0000 7f2611fc4640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f260c110f80 con 0x7f260c102570 2024-09-11T22:05:40.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.618+0000 7f2611fc4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f260c110e00 con 0x7f260c102ef0 2024-09-11T22:05:40.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.618+0000 7f2611fc4640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f260c111100 con 0x7f260c100e20 2024-09-11T22:05:40.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.618+0000 7f260affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f260c102570 0x7f260c1989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:40.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.618+0000 7f260affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f260c102570 0x7f260c1989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43936/0 (socket says 172.21.15.1:43936) 2024-09-11T22:05:40.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.618+0000 7f260affd640 1 -- 172.21.15.1:0/942092009 learned_addr learned my addr 172.21.15.1:0/942092009 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:40.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.618+0000 7f260bfff640 1 --2- 172.21.15.1:0/942092009 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f260c102ef0 0x7f260c19cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:40.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.618+0000 7f260b7fe640 1 --2- 172.21.15.1:0/942092009 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f260c100e20 0x7f260c198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:40.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.618+0000 7f260affd640 1 -- 172.21.15.1:0/942092009 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f260c100e20 msgr2=0x7f260c198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:40.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.618+0000 7f260affd640 1 --2- 172.21.15.1:0/942092009 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f260c100e20 0x7f260c198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:40.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.618+0000 7f260affd640 1 -- 172.21.15.1:0/942092009 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f260c102ef0 msgr2=0x7f260c19cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:40.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.618+0000 7f260affd640 1 --2- 172.21.15.1:0/942092009 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f260c102ef0 0x7f260c19cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:40.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.618+0000 7f260affd640 1 -- 172.21.15.1:0/942092009 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f260c19d410 con 0x7f260c102570 2024-09-11T22:05:40.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.619+0000 7f260affd640 1 --2- 172.21.15.1:0/942092009 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f260c102570 0x7f260c1989a0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f260000c920 tx=0x7f260000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:40.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.619+0000 7f2608ff9640 1 -- 172.21.15.1:0/942092009 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2600007c70 con 0x7f260c102570 2024-09-11T22:05:40.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.619+0000 7f2608ff9640 1 -- 172.21.15.1:0/942092009 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2600007e10 con 0x7f260c102570 2024-09-11T22:05:40.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.619+0000 7f2611fc4640 1 -- 172.21.15.1:0/942092009 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f260c19d700 con 0x7f260c102570 2024-09-11T22:05:40.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.619+0000 7f2608ff9640 1 -- 172.21.15.1:0/942092009 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2600018890 con 0x7f260c102570 2024-09-11T22:05:40.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.619+0000 7f2611fc4640 1 -- 172.21.15.1:0/942092009 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f260c1a4fe0 con 0x7f260c102570 2024-09-11T22:05:40.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.622+0000 7f261439d740 1 Processor -- start 2024-09-11T22:05:40.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.622+0000 7f261439d740 1 -- start start 2024-09-11T22:05:40.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.622+0000 7f261439d740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fc38ba33c0 0x55fc38ba37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:40.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.622+0000 7f261439d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc38ba3ce0 0x55fc38bb2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:40.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.622+0000 7f261439d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc38bb2a90 0x55fc38bb4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:40.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.622+0000 7f261439d740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55fc38b092e0 con 0x55fc38ba33c0 2024-09-11T22:05:40.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.622+0000 7f261439d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55fc38ab0ce0 con 0x55fc38ba3ce0 2024-09-11T22:05:40.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.622+0000 7f261439d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55fc38b052a0 con 0x55fc38bb2a90 2024-09-11T22:05:40.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.622+0000 7f260bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fc38ba33c0 0x55fc38ba37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:40.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.622+0000 7f260bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fc38ba33c0 0x55fc38ba37a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43940/0 (socket says 172.21.15.1:43940) 2024-09-11T22:05:40.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.622+0000 7f260bfff640 1 -- 172.21.15.1:0/109188890 learned_addr learned my addr 172.21.15.1:0/109188890 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:40.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.622+0000 7f2608ff9640 1 -- 172.21.15.1:0/942092009 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f2600018a70 con 0x7f260c102570 2024-09-11T22:05:40.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.622+0000 7f260b7fe640 1 --2- 172.21.15.1:0/109188890 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc38ba3ce0 0x55fc38bb2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:40.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.622+0000 7f260bfff640 1 -- 172.21.15.1:0/109188890 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc38bb2a90 msgr2=0x55fc38bb4e80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T22:05:40.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.622+0000 7f260bfff640 1 --2- 172.21.15.1:0/109188890 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc38bb2a90 0x55fc38bb4e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:40.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.623+0000 7f260bfff640 1 -- 172.21.15.1:0/109188890 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc38ba3ce0 msgr2=0x55fc38bb2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:40.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.623+0000 7f260bfff640 1 --2- 172.21.15.1:0/109188890 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc38ba3ce0 0x55fc38bb2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:40.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.623+0000 7f260bfff640 1 -- 172.21.15.1:0/109188890 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc38a95440 con 0x55fc38ba33c0 2024-09-11T22:05:40.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.623+0000 7f260b7fe640 1 --2- 172.21.15.1:0/109188890 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc38ba3ce0 0x55fc38bb2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:05:40.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.623+0000 7f260bfff640 1 --2- 172.21.15.1:0/109188890 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fc38ba33c0 0x55fc38ba37a0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f2604035000 tx=0x7f260402ff40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30f2d598697f7d17 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:40.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.623+0000 7f25f1ffb640 1 -- 172.21.15.1:0/109188890 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2604048070 con 0x55fc38ba33c0 2024-09-11T22:05:40.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.623+0000 7f25f1ffb640 1 -- 172.21.15.1:0/109188890 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f260402faf0 con 0x55fc38ba33c0 2024-09-11T22:05:40.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.623+0000 7f25f1ffb640 1 -- 172.21.15.1:0/109188890 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2604034040 con 0x55fc38ba33c0 2024-09-11T22:05:40.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.623+0000 7f2608ff9640 1 --2- 172.21.15.1:0/942092009 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f25f40a2d70 0x7f25f40a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:40.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.623+0000 7f260b7fe640 1 --2- 172.21.15.1:0/942092009 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f25f40a2d70 0x7f25f40a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:40.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.623+0000 7f2608ff9640 1 -- 172.21.15.1:0/942092009 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7f26000cf520 con 0x7f260c102570 2024-09-11T22:05:40.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.624+0000 7f260b7fe640 1 --2- 172.21.15.1:0/942092009 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f25f40a2d70 0x7f25f40a5230 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f25fc0058c0 tx=0x7f25fc005ec0 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:40.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.624+0000 7f261439d740 1 -- 172.21.15.1:0/109188890 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fc38ba33c0 msgr2=0x55fc38ba37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:40.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.624+0000 7f261439d740 1 --2- 172.21.15.1:0/109188890 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fc38ba33c0 0x55fc38ba37a0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f2604035000 tx=0x7f260402ff40 comp rx=0 tx=0).stop 2024-09-11T22:05:40.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.624+0000 7f261439d740 1 -- 172.21.15.1:0/109188890 shutdown_connections 2024-09-11T22:05:40.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.624+0000 7f261439d740 1 --2- 172.21.15.1:0/109188890 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc38bb2a90 0x55fc38bb4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:40.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.624+0000 7f261439d740 1 --2- 172.21.15.1:0/109188890 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc38ba3ce0 0x55fc38bb2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:40.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.624+0000 7f261439d740 1 --2- 172.21.15.1:0/109188890 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fc38ba33c0 0x55fc38ba37a0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:40.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.624+0000 7f261439d740 1 -- 172.21.15.1:0/109188890 >> 172.21.15.1:0/109188890 conn(0x55fc38b9fa30 msgr2=0x55fc38b9fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:40.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.624+0000 7f261439d740 1 -- 172.21.15.1:0/109188890 shutdown_connections 2024-09-11T22:05:40.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.624+0000 7f261439d740 1 -- 172.21.15.1:0/109188890 wait complete. 2024-09-11T22:05:40.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.625+0000 7f261439d740 1 -- start start 2024-09-11T22:05:40.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.625+0000 7f261439d740 1 Processor -- start 2024-09-11T22:05:40.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.625+0000 7f261439d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc38ba33c0 0x55fc38bb24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:40.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.625+0000 7f261439d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc38ba3ce0 0x55fc38baf790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:40.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.625+0000 7f261439d740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fc38bb2a90 0x55fc38baff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:40.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.625+0000 7f261439d740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55fc38bb55a0 con 0x55fc38bb2a90 2024-09-11T22:05:40.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.625+0000 7f261439d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55fc38ac4050 con 0x55fc38ba33c0 2024-09-11T22:05:40.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.625+0000 7f261439d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55fc38bb5720 con 0x55fc38ba3ce0 2024-09-11T22:05:40.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.625+0000 7f261439d740 10 client.0 _ll_register_callbacks cb 0x55fc38b04160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:05:40.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.625+0000 7f260bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fc38bb2a90 0x55fc38baff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:40.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.625+0000 7f260bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fc38bb2a90 0x55fc38baff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43956/0 (socket says 172.21.15.1:43956) 2024-09-11T22:05:40.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.625+0000 7f260bfff640 1 -- 172.21.15.1:0/2910757629 learned_addr learned my addr 172.21.15.1:0/2910757629 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:40.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.625+0000 7f260affd640 1 --2- 172.21.15.1:0/2910757629 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc38ba3ce0 0x55fc38baf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:40.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.626+0000 7f260b7fe640 1 --2- 172.21.15.1:0/2910757629 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc38ba33c0 0x55fc38bb24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:40.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.626+0000 7f260bfff640 1 -- 172.21.15.1:0/2910757629 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc38ba3ce0 msgr2=0x55fc38baf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:40.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.626+0000 7f260bfff640 1 --2- 172.21.15.1:0/2910757629 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc38ba3ce0 0x55fc38baf790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:40.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.626+0000 7f260bfff640 1 -- 172.21.15.1:0/2910757629 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc38ba33c0 msgr2=0x55fc38bb24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:40.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.626+0000 7f260bfff640 1 --2- 172.21.15.1:0/2910757629 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc38ba33c0 0x55fc38bb24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:40.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.626+0000 7f260bfff640 1 -- 172.21.15.1:0/2910757629 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc3887a3c0 con 0x55fc38bb2a90 2024-09-11T22:05:40.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.626+0000 7f260b7fe640 1 --2- 172.21.15.1:0/2910757629 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc38ba33c0 0x55fc38bb24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:05:40.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.626+0000 7f260bfff640 1 --2- 172.21.15.1:0/2910757629 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fc38bb2a90 0x55fc38baff20 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f2604006010 tx=0x7f2604003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1efc9193b3813238 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:40.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.626+0000 7f25eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:05:40.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.626+0000 7f25eaffd640 1 -- 172.21.15.1:0/2910757629 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2604048030 con 0x55fc38bb2a90 2024-09-11T22:05:40.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.626+0000 7f261439d740 1 -- 172.21.15.1:0/2910757629 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f260402faf0 con 0x55fc38bb2a90 2024-09-11T22:05:40.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.626+0000 7f25eaffd640 1 -- 172.21.15.1:0/2910757629 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7f2604040050 con 0x55fc38bb2a90 2024-09-11T22:05:40.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.626+0000 7f25eaffd640 1 -- 172.21.15.1:0/2910757629 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f2604034040 con 0x55fc38bb2a90 2024-09-11T22:05:40.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.626+0000 7f261439d740 10 client.17364 fetch_fsmap learned FSMap version 42 2024-09-11T22:05:40.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.626+0000 7f261439d740 1 -- 172.21.15.1:0/2910757629 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55fc38bc1020 con 0x55fc38bb2a90 2024-09-11T22:05:40.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.627+0000 7f25eaffd640 1 -- 172.21.15.1:0/2910757629 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f260402faf0 con 0x55fc38bb2a90 2024-09-11T22:05:40.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.627+0000 7f25eaffd640 1 -- 172.21.15.1:0/2910757629 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f260404e8a0 con 0x55fc38bb2a90 2024-09-11T22:05:40.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.627+0000 7f261439d740 10 client.17364 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:05:40.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.627+0000 7f261439d740 10 client.17364 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:05:40.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.627+0000 7f261439d740 20 client.17364 populate_metadata read hostname 'smithi001' 2024-09-11T22:05:40.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.627+0000 7f261439d740 1 --2- 172.21.15.1:0/2910757629 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55fc38bc2370 0x55fc38bc4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:40.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.627+0000 7f261439d740 4 client.17364 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:05:40.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.627+0000 7f261439d740 1 -- 172.21.15.1:0/2910757629 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc38b16850 con 0x55fc38bc2370 2024-09-11T22:05:40.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.627+0000 7f260b7fe640 1 --2- 172.21.15.1:0/2910757629 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55fc38bc2370 0x55fc38bc4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:40.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.628+0000 7f25eaffd640 10 client.17364 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:05:40.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.628+0000 7f260b7fe640 1 --2- 172.21.15.1:0/2910757629 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55fc38bc2370 0x55fc38bc4760 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63c3aaa476e32fa1 server_cookie=5432ceb256767410 in_seq=0 out_seq=0 2024-09-11T22:05:40.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.629+0000 7f25eaffd640 1 -- 172.21.15.1:0/2910757629 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fc38b16850 con 0x55fc38bc2370 2024-09-11T22:05:40.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.629+0000 7f25eaffd640 10 client.17364 handle_command_reply: tid=0 2024-09-11T22:05:40.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.629+0000 7f261439d740 1 client.17364 shutdown 2024-09-11T22:05:40.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.629+0000 7f261439d740 1 -- 172.21.15.1:0/2910757629 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fc38bb2a90 msgr2=0x55fc38baff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:40.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.629+0000 7f261439d740 1 --2- 172.21.15.1:0/2910757629 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fc38bb2a90 0x55fc38baff20 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f2604006010 tx=0x7f2604003c20 comp rx=0 tx=0).stop 2024-09-11T22:05:40.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.629+0000 7f261439d740 1 -- 172.21.15.1:0/2910757629 shutdown_connections 2024-09-11T22:05:40.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.629+0000 7f261439d740 1 --2- 172.21.15.1:0/2910757629 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55fc38bc2370 0x55fc38bc4760 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:40.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.629+0000 7f261439d740 1 --2- 172.21.15.1:0/2910757629 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fc38bb2a90 0x55fc38baff20 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:40.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.629+0000 7f25eaffd640 0 client.17364 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:05:40.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.629+0000 7f261439d740 1 --2- 172.21.15.1:0/2910757629 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc38ba3ce0 0x55fc38baf790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:40.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.629+0000 7f261439d740 1 --2- 172.21.15.1:0/2910757629 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc38ba33c0 0x55fc38bb24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:40.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.629+0000 7f261439d740 1 -- 172.21.15.1:0/2910757629 >> 172.21.15.1:0/2910757629 conn(0x55fc38b9fa30 msgr2=0x55fc38ba56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:40.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.630+0000 7f261439d740 1 -- 172.21.15.1:0/2910757629 shutdown_connections 2024-09-11T22:05:40.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.630+0000 7f261439d740 1 -- 172.21.15.1:0/2910757629 wait complete. 2024-09-11T22:05:40.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.630+0000 7f261439d740 20 client.17364 trim_cache size 0 max 16384 2024-09-11T22:05:40.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.647+0000 7f261439d740 1 Processor -- start 2024-09-11T22:05:40.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.647+0000 7f261439d740 1 -- start start 2024-09-11T22:05:40.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.647+0000 7f261439d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc38ba33c0 0x55fc38b9f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:40.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.647+0000 7f261439d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc38ba3ce0 0x55fc38bb1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:40.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.647+0000 7f261439d740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fc38bb2a90 0x55fc38bc9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:40.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.647+0000 7f261439d740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f260c1ab6e0 con 0x55fc38bb2a90 2024-09-11T22:05:40.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.647+0000 7f261439d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f260c1ad520 con 0x55fc38ba3ce0 2024-09-11T22:05:40.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.647+0000 7f261439d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f260c1abe70 con 0x55fc38ba33c0 2024-09-11T22:05:40.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.647+0000 7f260bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fc38bb2a90 0x55fc38bc9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:40.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.647+0000 7f260bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fc38bb2a90 0x55fc38bc9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43958/0 (socket says 172.21.15.1:43958) 2024-09-11T22:05:40.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.647+0000 7f260bfff640 1 -- 172.21.15.1:0/3538316098 learned_addr learned my addr 172.21.15.1:0/3538316098 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:40.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.647+0000 7f260affd640 1 --2- 172.21.15.1:0/3538316098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc38ba3ce0 0x55fc38bb1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:40.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.647+0000 7f260b7fe640 1 --2- 172.21.15.1:0/3538316098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc38ba33c0 0x55fc38b9f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:40.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.648+0000 7f260bfff640 1 -- 172.21.15.1:0/3538316098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc38ba33c0 msgr2=0x55fc38b9f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:40.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.648+0000 7f260bfff640 1 --2- 172.21.15.1:0/3538316098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc38ba33c0 0x55fc38b9f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:40.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.648+0000 7f260bfff640 1 -- 172.21.15.1:0/3538316098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc38ba3ce0 msgr2=0x55fc38bb1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:40.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.648+0000 7f260bfff640 1 --2- 172.21.15.1:0/3538316098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc38ba3ce0 0x55fc38bb1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:40.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.648+0000 7f260bfff640 1 -- 172.21.15.1:0/3538316098 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc38bb2850 con 0x55fc38bb2a90 2024-09-11T22:05:40.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.648+0000 7f260bfff640 1 --2- 172.21.15.1:0/3538316098 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fc38bb2a90 0x55fc38bc9d30 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f260404eeb0 tx=0x7f2604000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f78e19a24635608a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:40.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.648+0000 7f25f17fa640 1 -- 172.21.15.1:0/3538316098 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2604048070 con 0x55fc38bb2a90 2024-09-11T22:05:40.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.648+0000 7f25f17fa640 1 -- 172.21.15.1:0/3538316098 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f260404a070 con 0x55fc38bb2a90 2024-09-11T22:05:40.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.648+0000 7f25f17fa640 1 -- 172.21.15.1:0/3538316098 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260404e460 con 0x55fc38bb2a90 2024-09-11T22:05:40.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.649+0000 7f261439d740 1 -- 172.21.15.1:0/3538316098 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fc38bb2a90 msgr2=0x55fc38bc9d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:40.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.649+0000 7f261439d740 1 --2- 172.21.15.1:0/3538316098 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fc38bb2a90 0x55fc38bc9d30 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f260404eeb0 tx=0x7f2604000f40 comp rx=0 tx=0).stop 2024-09-11T22:05:40.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.649+0000 7f261439d740 1 -- 172.21.15.1:0/3538316098 shutdown_connections 2024-09-11T22:05:40.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.649+0000 7f261439d740 1 --2- 172.21.15.1:0/3538316098 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fc38bb2a90 0x55fc38bc9d30 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:40.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.649+0000 7f261439d740 1 --2- 172.21.15.1:0/3538316098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc38ba3ce0 0x55fc38bb1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:40.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.649+0000 7f261439d740 1 --2- 172.21.15.1:0/3538316098 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc38ba33c0 0x55fc38b9f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:40.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.649+0000 7f261439d740 1 -- 172.21.15.1:0/3538316098 >> 172.21.15.1:0/3538316098 conn(0x55fc38b9fa30 msgr2=0x55fc38baf4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:40.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.649+0000 7f261439d740 1 -- 172.21.15.1:0/3538316098 shutdown_connections 2024-09-11T22:05:40.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.649+0000 7f261439d740 1 -- 172.21.15.1:0/3538316098 wait complete. 2024-09-11T22:05:40.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.649+0000 7f261439d740 1 -- start start 2024-09-11T22:05:40.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.650+0000 7f261439d740 1 Processor -- start 2024-09-11T22:05:40.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.650+0000 7f261439d740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fc38ba33c0 0x55fc38bac770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:40.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.650+0000 7f261439d740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc38ba3ce0 0x55fc38baccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:40.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.650+0000 7f261439d740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc38bb2a90 0x55fc38bad1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:40.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.650+0000 7f261439d740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f260c1acd90 con 0x55fc38ba33c0 2024-09-11T22:05:40.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.650+0000 7f261439d740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f260c1ac600 con 0x55fc38ba3ce0 2024-09-11T22:05:40.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.650+0000 7f261439d740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f260c1a96e0 con 0x55fc38bb2a90 2024-09-11T22:05:40.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.650+0000 7f261439d740 10 client.0 _ll_register_callbacks cb 0x7f2604046070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:05:40.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.650+0000 7f260b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fc38ba33c0 0x55fc38bac770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:40.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.650+0000 7f260b7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fc38ba33c0 0x55fc38bac770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43970/0 (socket says 172.21.15.1:43970) 2024-09-11T22:05:40.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.650+0000 7f260b7fe640 1 -- 172.21.15.1:0/2078604207 learned_addr learned my addr 172.21.15.1:0/2078604207 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:40.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.650+0000 7f260affd640 1 --2- 172.21.15.1:0/2078604207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc38ba3ce0 0x55fc38baccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:40.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.650+0000 7f260bfff640 1 --2- 172.21.15.1:0/2078604207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc38bb2a90 0x55fc38bad1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:40.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.651+0000 7f260affd640 1 -- 172.21.15.1:0/2078604207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc38bb2a90 msgr2=0x55fc38bad1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:40.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.651+0000 7f260affd640 1 --2- 172.21.15.1:0/2078604207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc38bb2a90 0x55fc38bad1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:40.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.651+0000 7f260affd640 1 -- 172.21.15.1:0/2078604207 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fc38ba33c0 msgr2=0x55fc38bac770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:40.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.651+0000 7f260affd640 1 --2- 172.21.15.1:0/2078604207 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fc38ba33c0 0x55fc38bac770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:40.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.651+0000 7f260affd640 1 -- 172.21.15.1:0/2078604207 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc38c2bd00 con 0x55fc38ba3ce0 2024-09-11T22:05:40.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.651+0000 7f260b7fe640 1 --2- 172.21.15.1:0/2078604207 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fc38ba33c0 0x55fc38bac770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:05:40.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.651+0000 7f260affd640 1 --2- 172.21.15.1:0/2078604207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc38ba3ce0 0x55fc38baccb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f26000ccc00 tx=0x7f260009b3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f94e2a2d90314b83 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:40.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.651+0000 7f25eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-11T22:05:40.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.651+0000 7f25eaffd640 1 -- 172.21.15.1:0/2078604207 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260009fd80 con 0x55fc38ba3ce0 2024-09-11T22:05:40.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.651+0000 7f25eaffd640 1 -- 172.21.15.1:0/2078604207 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7f26000ba380 con 0x55fc38ba3ce0 2024-09-11T22:05:40.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.651+0000 7f261439d740 1 -- 172.21.15.1:0/2078604207 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f260404a070 con 0x55fc38ba3ce0 2024-09-11T22:05:40.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.651+0000 7f25eaffd640 1 -- 172.21.15.1:0/2078604207 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f26000a4640 con 0x55fc38ba3ce0 2024-09-11T22:05:40.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.652+0000 7f261439d740 10 client.26801 fetch_fsmap learned FSMap version 42 2024-09-11T22:05:40.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.652+0000 7f261439d740 1 -- 172.21.15.1:0/2078604207 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55fc38c2cfe0 con 0x55fc38ba3ce0 2024-09-11T22:05:40.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.652+0000 7f25eaffd640 1 -- 172.21.15.1:0/2078604207 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f260404a070 con 0x55fc38ba3ce0 2024-09-11T22:05:40.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.652+0000 7f25eaffd640 1 -- 172.21.15.1:0/2078604207 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 42) ==== 8442+0+0 (secure 0 0 0) 0x7f26000a41b0 con 0x55fc38ba3ce0 2024-09-11T22:05:40.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.652+0000 7f261439d740 10 client.26801 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:05:40.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.652+0000 7f261439d740 10 client.26801 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:05:40.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.652+0000 7f261439d740 20 client.26801 populate_metadata read hostname 'smithi001' 2024-09-11T22:05:40.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.652+0000 7f261439d740 1 --2- 172.21.15.1:0/2078604207 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55fc38bc2370 0x55fc38c2f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:40.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.652+0000 7f261439d740 4 client.26801 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:05:40.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.652+0000 7f261439d740 1 -- 172.21.15.1:0/2078604207 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fc38b10d60 con 0x55fc38bc2370 2024-09-11T22:05:40.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.652+0000 7f260b7fe640 1 --2- 172.21.15.1:0/2078604207 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55fc38bc2370 0x55fc38c2f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:40.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.653+0000 7f25eaffd640 10 client.26801 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:05:40.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.653+0000 7f260b7fe640 1 --2- 172.21.15.1:0/2078604207 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55fc38bc2370 0x55fc38c2f860 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2d453a4b8300f43 server_cookie=b498ea59742250cf in_seq=0 out_seq=0 2024-09-11T22:05:40.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.653+0000 7f25eaffd640 1 -- 172.21.15.1:0/2078604207 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55fc38b10d60 con 0x55fc38bc2370 2024-09-11T22:05:40.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.653+0000 7f25eaffd640 10 client.26801 handle_command_reply: tid=0 2024-09-11T22:05:40.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.653+0000 7f261439d740 1 client.26801 shutdown 2024-09-11T22:05:40.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.653+0000 7f261439d740 1 -- 172.21.15.1:0/2078604207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc38ba3ce0 msgr2=0x55fc38baccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:40.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.653+0000 7f261439d740 1 --2- 172.21.15.1:0/2078604207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc38ba3ce0 0x55fc38baccb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f26000ccc00 tx=0x7f260009b3f0 comp rx=0 tx=0).stop 2024-09-11T22:05:40.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.653+0000 7f261439d740 1 -- 172.21.15.1:0/2078604207 shutdown_connections 2024-09-11T22:05:40.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.653+0000 7f261439d740 1 --2- 172.21.15.1:0/2078604207 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55fc38bc2370 0x55fc38c2f860 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:40.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.653+0000 7f261439d740 1 --2- 172.21.15.1:0/2078604207 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fc38bb2a90 0x55fc38bad1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:40.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.653+0000 7f261439d740 1 --2- 172.21.15.1:0/2078604207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fc38ba3ce0 0x55fc38baccb0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:40.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.653+0000 7f261439d740 1 --2- 172.21.15.1:0/2078604207 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fc38ba33c0 0x55fc38bac770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:40.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.653+0000 7f261439d740 1 -- 172.21.15.1:0/2078604207 >> 172.21.15.1:0/2078604207 conn(0x55fc38b9fa30 msgr2=0x55fc38bb1810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:40.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.653+0000 7f25eaffd640 0 client.26801 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:05:40.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.653+0000 7f261439d740 1 -- 172.21.15.1:0/2078604207 shutdown_connections 2024-09-11T22:05:40.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.654+0000 7f261439d740 1 -- 172.21.15.1:0/2078604207 wait complete. 2024-09-11T22:05:40.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.654+0000 7f261439d740 20 client.26801 trim_cache size 0 max 16384 2024-09-11T22:05:40.660 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-11T22:05:40.660 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-09-11T22:05:40.660 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-09-11T22:05:40.660 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-11T22:05:40.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.654+0000 7f2611fc4640 1 -- 172.21.15.1:0/942092009 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f25f40a2d70 msgr2=0x7f25f40a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:40.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.654+0000 7f2611fc4640 1 --2- 172.21.15.1:0/942092009 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f25f40a2d70 0x7f25f40a5230 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f25fc0058c0 tx=0x7f25fc005ec0 comp rx=0 tx=0).stop 2024-09-11T22:05:40.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.654+0000 7f2611fc4640 1 -- 172.21.15.1:0/942092009 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f260c102570 msgr2=0x7f260c1989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:40.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.654+0000 7f2611fc4640 1 --2- 172.21.15.1:0/942092009 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f260c102570 0x7f260c1989a0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f260000c920 tx=0x7f260000cde0 comp rx=0 tx=0).stop 2024-09-11T22:05:40.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.654+0000 7f2611fc4640 1 -- 172.21.15.1:0/942092009 shutdown_connections 2024-09-11T22:05:40.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.654+0000 7f2611fc4640 1 --2- 172.21.15.1:0/942092009 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f25f40a2d70 0x7f25f40a5230 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:40.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.654+0000 7f2611fc4640 1 --2- 172.21.15.1:0/942092009 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f260c102ef0 0x7f260c19cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:40.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.654+0000 7f2611fc4640 1 --2- 172.21.15.1:0/942092009 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f260c102570 0x7f260c1989a0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:40.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.654+0000 7f2611fc4640 1 --2- 172.21.15.1:0/942092009 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f260c100e20 0x7f260c198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:40.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.654+0000 7f2611fc4640 1 -- 172.21.15.1:0/942092009 >> 172.21.15.1:0/942092009 conn(0x7f260c0fc7d0 msgr2=0x7f260c06a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:40.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.655+0000 7f2611fc4640 1 -- 172.21.15.1:0/942092009 shutdown_connections 2024-09-11T22:05:40.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:40.655+0000 7f2611fc4640 1 -- 172.21.15.1:0/942092009 wait complete. 2024-09-11T22:05:40.792 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:40.789+0000 7f3026ff6640 1 -- 172.21.15.1:0/2413384973 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3020106a20 msgr2=0x7f3020106e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:40.792 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:40.789+0000 7f3026ff6640 1 --2- 172.21.15.1:0/2413384973 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3020106a20 0x7f3020106e00 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f3010009cd0 tx=0x7f301002a8e0 comp rx=0 tx=0).stop 2024-09-11T22:05:40.792 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:40.789+0000 7f3026ff6640 1 -- 172.21.15.1:0/2413384973 shutdown_connections 2024-09-11T22:05:40.792 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:40.789+0000 7f3026ff6640 1 --2- 172.21.15.1:0/2413384973 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30201013b0 0x7f302010d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:40.793 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:40.789+0000 7f3026ff6640 1 --2- 172.21.15.1:0/2413384973 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3020100a10 0x7f3020100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:40.793 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:40.789+0000 7f3026ff6640 1 --2- 172.21.15.1:0/2413384973 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3020106a20 0x7f3020106e00 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:40.793 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:40.789+0000 7f3026ff6640 1 -- 172.21.15.1:0/2413384973 >> 172.21.15.1:0/2413384973 conn(0x7f30200fc6f0 msgr2=0x7f30200feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:40.793 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:40.789+0000 7f3026ff6640 1 -- 172.21.15.1:0/2413384973 shutdown_connections 2024-09-11T22:05:40.793 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:40.790+0000 7f3026ff6640 1 -- 172.21.15.1:0/2413384973 wait complete. 2024-09-11T22:05:40.793 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:40.790+0000 7f3026ff6640 1 Processor -- start 2024-09-11T22:05:40.793 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:40.790+0000 7f3026ff6640 1 -- start start 2024-09-11T22:05:40.793 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:40.790+0000 7f3026ff6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3020100a10 0x7f30201992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:40.793 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:40.790+0000 7f3026ff6640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30201013b0 0x7f30201997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:40.793 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:40.790+0000 7f3026ff6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3020106a20 0x7f30201a4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:40.793 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:40.790+0000 7f3026ff6640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3020110e70 con 0x7f3020100a10 2024-09-11T22:05:40.793 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:40.790+0000 7f3026ff6640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3020110cf0 con 0x7f30201013b0 2024-09-11T22:05:40.794 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:40.790+0000 7f3026ff6640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3020110ff0 con 0x7f3020106a20 2024-09-11T22:05:40.794 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:40.790+0000 7f3024d92640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3020100a10 0x7f30201992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:40.794 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:40.791+0000 7f3024d92640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3020100a10 0x7f30201992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43982/0 (socket says 172.21.15.1:43982) 2024-09-11T22:05:40.794 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:40.791+0000 7f3024d92640 1 -- 172.21.15.1:0/3946002492 learned_addr learned my addr 172.21.15.1:0/3946002492 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:40.794 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:40.791+0000 7f3025593640 1 --2- 172.21.15.1:0/3946002492 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3020106a20 0x7f30201a4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:40.794 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:40.791+0000 7f301ffff640 1 --2- 172.21.15.1:0/3946002492 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30201013b0 0x7f30201997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:40.794 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:40.791+0000 7f3024d92640 1 -- 172.21.15.1:0/3946002492 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3020106a20 msgr2=0x7f30201a4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:40.794 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:40.791+0000 7f3024d92640 1 --2- 172.21.15.1:0/3946002492 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3020106a20 0x7f30201a4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:40.794 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:40.791+0000 7f3024d92640 1 -- 172.21.15.1:0/3946002492 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30201013b0 msgr2=0x7f30201997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:40.794 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:40.791+0000 7f3024d92640 1 --2- 172.21.15.1:0/3946002492 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30201013b0 0x7f30201997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:40.794 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:40.791+0000 7f3024d92640 1 -- 172.21.15.1:0/3946002492 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30201a4860 con 0x7f3020100a10 2024-09-11T22:05:40.794 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:40.791+0000 7f3025593640 1 --2- 172.21.15.1:0/3946002492 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3020106a20 0x7f30201a4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:05:40.794 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:40.791+0000 7f3024d92640 1 --2- 172.21.15.1:0/3946002492 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3020100a10 0x7f30201992a0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f3010033040 tx=0x7f3010003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:40.795 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:40.791+0000 7f301dffb640 1 -- 172.21.15.1:0/3946002492 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3010004040 con 0x7f3020100a10 2024-09-11T22:05:40.795 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:40.791+0000 7f301dffb640 1 -- 172.21.15.1:0/3946002492 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30100046c0 con 0x7f3020100a10 2024-09-11T22:05:40.795 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:40.791+0000 7f301dffb640 1 -- 172.21.15.1:0/3946002492 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3010046480 con 0x7f3020100a10 2024-09-11T22:05:40.795 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:40.791+0000 7f3026ff6640 1 -- 172.21.15.1:0/3946002492 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30201a4a30 con 0x7f3020100a10 2024-09-11T22:05:40.795 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:40.792+0000 7f3026ff6640 1 -- 172.21.15.1:0/3946002492 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30201a4f70 con 0x7f3020100a10 2024-09-11T22:05:40.795 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:40.795+0000 7f301dffb640 1 -- 172.21.15.1:0/3946002492 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f30100305a0 con 0x7f3020100a10 2024-09-11T22:05:40.795 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:40.795+0000 7f301dffb640 1 --2- 172.21.15.1:0/3946002492 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f300c0a2d80 0x7f300c0a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:40.796 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:40.795+0000 7f301dffb640 1 -- 172.21.15.1:0/3946002492 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8629+0+0 (secure 0 0 0) 0x7f301002e030 con 0x7f3020100a10 2024-09-11T22:05:40.796 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:40.795+0000 7f301ffff640 1 --2- 172.21.15.1:0/3946002492 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f300c0a2d80 0x7f300c0a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:40.823 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:40.792+0000 7f3026ff6640 1 -- 172.21.15.1:0/3946002492 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3020100e70 con 0x7f3020100a10 2024-09-11T22:05:40.823 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:40.823+0000 7f301ffff640 1 --2- 172.21.15.1:0/3946002492 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f300c0a2d80 0x7f300c0a5240 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f302006aef0 tx=0x7f300800b590 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:40.828 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:40.827+0000 7f301dffb640 1 -- 172.21.15.1:0/3946002492 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f301004b060 con 0x7f3020100a10 2024-09-11T22:05:41.009 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:41.009+0000 7f3026ff6640 1 -- 172.21.15.1:0/3946002492 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "foo.201307", "pg_num": 8} v 0) -- 0x7f3020108f80 con 0x7f3020100a10 2024-09-11T22:05:41.357 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T22:05:41.358 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:05:41.501 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:41.500+0000 7f301dffb640 1 -- 172.21.15.1:0/3946002492 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "foo.201307", "pg_num": 8}]=0 pool 'foo.201307' created v92) ==== 123+0+0 (secure 0 0 0) 0x7f30100bd5a0 con 0x7f3020100a10 2024-09-11T22:05:41.618 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:41.618+0000 7f3026ff6640 1 -- 172.21.15.1:0/3946002492 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "foo.201307", "pg_num": 8} v 0) -- 0x7f3020108bb0 con 0x7f3020100a10 2024-09-11T22:05:41.619 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:41.618+0000 7f301dffb640 1 -- 172.21.15.1:0/3946002492 <== mon.0 v2:172.21.15.1:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "foo.201307", "pg_num": 8}]=0 pool 'foo.201307' already exists v92) ==== 130+0+0 (secure 0 0 0) 0x7f3010045e80 con 0x7f3020100a10 2024-09-11T22:05:41.619 INFO:tasks.workunit.client.0.smithi001.stderr:pool 'foo.201307' already exists 2024-09-11T22:05:41.622 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:41.621+0000 7f3026ff6640 1 -- 172.21.15.1:0/3946002492 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f300c0a2d80 msgr2=0x7f300c0a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:41.622 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:41.621+0000 7f3026ff6640 1 --2- 172.21.15.1:0/3946002492 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f300c0a2d80 0x7f300c0a5240 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f302006aef0 tx=0x7f300800b590 comp rx=0 tx=0).stop 2024-09-11T22:05:41.622 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:41.621+0000 7f3026ff6640 1 -- 172.21.15.1:0/3946002492 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3020100a10 msgr2=0x7f30201992a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:41.622 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:41.621+0000 7f3026ff6640 1 --2- 172.21.15.1:0/3946002492 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3020100a10 0x7f30201992a0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f3010033040 tx=0x7f3010003970 comp rx=0 tx=0).stop 2024-09-11T22:05:41.622 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:41.621+0000 7f3026ff6640 1 -- 172.21.15.1:0/3946002492 shutdown_connections 2024-09-11T22:05:41.622 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:41.621+0000 7f3026ff6640 1 --2- 172.21.15.1:0/3946002492 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f300c0a2d80 0x7f300c0a5240 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:41.622 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:41.621+0000 7f3026ff6640 1 --2- 172.21.15.1:0/3946002492 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3020106a20 0x7f30201a4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:41.622 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:41.621+0000 7f3026ff6640 1 --2- 172.21.15.1:0/3946002492 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f30201013b0 0x7f30201997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:41.622 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:41.621+0000 7f3026ff6640 1 --2- 172.21.15.1:0/3946002492 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3020100a10 0x7f30201992a0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:41.622 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:41.621+0000 7f3026ff6640 1 -- 172.21.15.1:0/3946002492 >> 172.21.15.1:0/3946002492 conn(0x7f30200fc6f0 msgr2=0x7f30200fcf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:41.622 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:41.621+0000 7f3026ff6640 1 -- 172.21.15.1:0/3946002492 shutdown_connections 2024-09-11T22:05:41.623 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:41.621+0000 7f3026ff6640 1 -- 172.21.15.1:0/3946002492 wait complete. 2024-09-11T22:05:41.637 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:05:41.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:41 smithi001 ceph-mon[30935]: pgmap v260: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 18 op/s 2024-09-11T22:05:41.728 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:41.727+0000 7f6840bdf640 1 -- 172.21.15.1:0/2809659749 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f683c101870 msgr2=0x7f683c10d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:41.729 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:41.727+0000 7f6840bdf640 1 --2- 172.21.15.1:0/2809659749 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f683c101870 0x7f683c10d8d0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f6834009930 tx=0x7f683402a420 comp rx=0 tx=0).stop 2024-09-11T22:05:41.729 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:41.728+0000 7f6840bdf640 1 -- 172.21.15.1:0/2809659749 shutdown_connections 2024-09-11T22:05:41.729 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:41.728+0000 7f6840bdf640 1 --2- 172.21.15.1:0/2809659749 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f683c101870 0x7f683c10d8d0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:41.729 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:41.728+0000 7f6840bdf640 1 --2- 172.21.15.1:0/2809659749 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f683c100ed0 0x7f683c101330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:41.729 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:41.728+0000 7f6840bdf640 1 --2- 172.21.15.1:0/2809659749 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f683c106a80 0x7f683c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:41.729 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:41.728+0000 7f6840bdf640 1 -- 172.21.15.1:0/2809659749 >> 172.21.15.1:0/2809659749 conn(0x7f683c076830 msgr2=0x7f683c076c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:41.729 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:41.728+0000 7f6840bdf640 1 -- 172.21.15.1:0/2809659749 shutdown_connections 2024-09-11T22:05:41.729 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:41.728+0000 7f6840bdf640 1 -- 172.21.15.1:0/2809659749 wait complete. 2024-09-11T22:05:41.732 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:41.728+0000 7f6840bdf640 1 Processor -- start 2024-09-11T22:05:41.732 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:41.729+0000 7f6840bdf640 1 -- start start 2024-09-11T22:05:41.733 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:41.729+0000 7f6840bdf640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f683c100ed0 0x7f683c1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:41.733 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:41.729+0000 7f6840bdf640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f683c101870 0x7f683c198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:41.733 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:41.729+0000 7f6840bdf640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f683c106a80 0x7f683c19cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:41.733 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:41.729+0000 7f6840bdf640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f683c110e20 con 0x7f683c106a80 2024-09-11T22:05:41.733 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:41.729+0000 7f6840bdf640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f683c110ca0 con 0x7f683c100ed0 2024-09-11T22:05:41.733 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:41.729+0000 7f6840bdf640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f683c110fa0 con 0x7f683c101870 2024-09-11T22:05:41.733 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:41.729+0000 7f683ad9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f683c106a80 0x7f683c19cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:41.733 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:41.729+0000 7f683ad9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f683c106a80 0x7f683c19cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43998/0 (socket says 172.21.15.1:43998) 2024-09-11T22:05:41.734 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:41.729+0000 7f683ad9d640 1 -- 172.21.15.1:0/1218656940 learned_addr learned my addr 172.21.15.1:0/1218656940 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:41.734 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:41.729+0000 7f6839d9b640 1 --2- 172.21.15.1:0/1218656940 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f683c101870 0x7f683c198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:41.734 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:41.729+0000 7f683a59c640 1 --2- 172.21.15.1:0/1218656940 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f683c100ed0 0x7f683c1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:41.734 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:41.729+0000 7f683ad9d640 1 -- 172.21.15.1:0/1218656940 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f683c101870 msgr2=0x7f683c198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:41.734 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:41.729+0000 7f683ad9d640 1 --2- 172.21.15.1:0/1218656940 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f683c101870 0x7f683c198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:41.734 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:41.729+0000 7f683ad9d640 1 -- 172.21.15.1:0/1218656940 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f683c100ed0 msgr2=0x7f683c1983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:41.734 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:41.729+0000 7f683ad9d640 1 --2- 172.21.15.1:0/1218656940 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f683c100ed0 0x7f683c1983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:41.734 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:41.729+0000 7f683ad9d640 1 -- 172.21.15.1:0/1218656940 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f683c19d3c0 con 0x7f683c106a80 2024-09-11T22:05:41.734 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:41.729+0000 7f683ad9d640 1 --2- 172.21.15.1:0/1218656940 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f683c106a80 0x7f683c19cc90 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f6834005ab0 tx=0x7f68340042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:41.734 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:41.730+0000 7f682b7fe640 1 -- 172.21.15.1:0/1218656940 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f683403f030 con 0x7f683c106a80 2024-09-11T22:05:41.734 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:41.730+0000 7f682b7fe640 1 -- 172.21.15.1:0/1218656940 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6834003ce0 con 0x7f683c106a80 2024-09-11T22:05:41.734 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:41.730+0000 7f682b7fe640 1 -- 172.21.15.1:0/1218656940 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6834035070 con 0x7f683c106a80 2024-09-11T22:05:41.734 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:41.730+0000 7f6840bdf640 1 -- 172.21.15.1:0/1218656940 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f683c19d650 con 0x7f683c106a80 2024-09-11T22:05:41.735 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:41.730+0000 7f6840bdf640 1 -- 172.21.15.1:0/1218656940 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f683c1a4f40 con 0x7f683c106a80 2024-09-11T22:05:41.735 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:41.731+0000 7f6840bdf640 1 -- 172.21.15.1:0/1218656940 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f683c101330 con 0x7f683c106a80 2024-09-11T22:05:41.736 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:41.733+0000 7f682b7fe640 1 -- 172.21.15.1:0/1218656940 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f6834042030 con 0x7f683c106a80 2024-09-11T22:05:41.736 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:41.734+0000 7f682b7fe640 1 --2- 172.21.15.1:0/1218656940 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f68240a2d80 0x7f68240a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:41.736 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:41.734+0000 7f683a59c640 1 --2- 172.21.15.1:0/1218656940 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f68240a2d80 0x7f68240a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:41.736 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:41.734+0000 7f682b7fe640 1 -- 172.21.15.1:0/1218656940 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8934+0+0 (secure 0 0 0) 0x7f68340efee0 con 0x7f683c106a80 2024-09-11T22:05:41.736 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:41.734+0000 7f683a59c640 1 --2- 172.21.15.1:0/1218656940 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f68240a2d80 0x7f68240a5240 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f682c005c40 tx=0x7f682c005bd0 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:41.737 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:41.737+0000 7f682b7fe640 1 -- 172.21.15.1:0/1218656940 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f68340bc900 con 0x7f683c106a80 2024-09-11T22:05:41.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:41 smithi053 ceph-mon[34908]: pgmap v260: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 18 op/s 2024-09-11T22:05:41.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:41 smithi161 ceph-mon[35615]: pgmap v260: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 18 op/s 2024-09-11T22:05:41.958 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:41.957+0000 7f6840bdf640 1 -- 172.21.15.1:0/1218656940 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "foo.201307"} v 0) -- 0x7f683c199720 con 0x7f683c106a80 2024-09-11T22:05:42.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:42.130+0000 7ff2cd76f640 1 -- 172.21.15.1:0/328580253 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2c8102d40 msgr2=0x7ff2c8103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:42.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:42.130+0000 7ff2cd76f640 1 --2- 172.21.15.1:0/328580253 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2c8102d40 0x7ff2c8103120 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7ff2b8009900 tx=0x7ff2b802a480 comp rx=0 tx=0).stop 2024-09-11T22:05:42.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:42.131+0000 7ff2cd76f640 1 -- 172.21.15.1:0/328580253 shutdown_connections 2024-09-11T22:05:42.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:42.131+0000 7ff2cd76f640 1 --2- 172.21.15.1:0/328580253 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2c810c170 0x7ff2c810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:42.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:42.131+0000 7ff2cd76f640 1 --2- 172.21.15.1:0/328580253 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2c8103660 0x7ff2c810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:42.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:42.131+0000 7ff2cd76f640 1 --2- 172.21.15.1:0/328580253 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2c8102d40 0x7ff2c8103120 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:42.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:42.131+0000 7ff2cd76f640 1 -- 172.21.15.1:0/328580253 >> 172.21.15.1:0/328580253 conn(0x7ff2c80fc7a0 msgr2=0x7ff2c80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:42.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:42.131+0000 7ff2cd76f640 1 -- 172.21.15.1:0/328580253 shutdown_connections 2024-09-11T22:05:42.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:42.131+0000 7ff2cd76f640 1 -- 172.21.15.1:0/328580253 wait complete. 2024-09-11T22:05:42.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:42.131+0000 7ff2cd76f640 1 Processor -- start 2024-09-11T22:05:42.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:42.131+0000 7ff2cd76f640 1 -- start start 2024-09-11T22:05:42.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:42.132+0000 7ff2cd76f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2c8102d40 0x7ff2c81983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:42.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:42.132+0000 7ff2cd76f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2c8103660 0x7ff2c8198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:42.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:42.132+0000 7ff2cd76f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2c810c170 0x7ff2c819ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:42.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:42.132+0000 7ff2cd76f640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c8111f70 con 0x7ff2c8102d40 2024-09-11T22:05:42.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:42.132+0000 7ff2cd76f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c8111df0 con 0x7ff2c8103660 2024-09-11T22:05:42.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:42.132+0000 7ff2cd76f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c81120f0 con 0x7ff2c810c170 2024-09-11T22:05:42.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:42.132+0000 7ff2c6ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2c8102d40 0x7ff2c81983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:42.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:42.132+0000 7ff2c6ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2c8102d40 0x7ff2c81983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44010/0 (socket says 172.21.15.1:44010) 2024-09-11T22:05:42.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:42.132+0000 7ff2c6ffd640 1 -- 172.21.15.1:0/765612329 learned_addr learned my addr 172.21.15.1:0/765612329 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:42.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:42.132+0000 7ff2be7fc640 1 --2- 172.21.15.1:0/765612329 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2c8103660 0x7ff2c8198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:42.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:42.132+0000 7ff2c6ffd640 1 -- 172.21.15.1:0/765612329 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2c810c170 msgr2=0x7ff2c819ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:42.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:42.132+0000 7ff2c6ffd640 1 --2- 172.21.15.1:0/765612329 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2c810c170 0x7ff2c819ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:42.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:42.132+0000 7ff2c6ffd640 1 -- 172.21.15.1:0/765612329 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2c8103660 msgr2=0x7ff2c8198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:42.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:42.132+0000 7ff2c6ffd640 1 --2- 172.21.15.1:0/765612329 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2c8103660 0x7ff2c8198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:42.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:42.132+0000 7ff2c6ffd640 1 -- 172.21.15.1:0/765612329 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2c819d3a0 con 0x7ff2c8102d40 2024-09-11T22:05:42.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:42.132+0000 7ff2c6ffd640 1 --2- 172.21.15.1:0/765612329 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2c8102d40 0x7ff2c81983f0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7ff2b8006010 tx=0x7ff2b80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:42.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:42.133+0000 7ff2c4ff9640 1 -- 172.21.15.1:0/765612329 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b803f030 con 0x7ff2c8102d40 2024-09-11T22:05:42.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:42.133+0000 7ff2cd76f640 1 -- 172.21.15.1:0/765612329 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2c819d630 con 0x7ff2c8102d40 2024-09-11T22:05:42.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:42.133+0000 7ff2c4ff9640 1 -- 172.21.15.1:0/765612329 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2b8003ce0 con 0x7ff2c8102d40 2024-09-11T22:05:42.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:42.133+0000 7ff2c4ff9640 1 -- 172.21.15.1:0/765612329 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b8035070 con 0x7ff2c8102d40 2024-09-11T22:05:42.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:42.133+0000 7ff2cd76f640 1 -- 172.21.15.1:0/765612329 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2c81a4f70 con 0x7ff2c8102d40 2024-09-11T22:05:42.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:42.134+0000 7ff2cd76f640 1 -- 172.21.15.1:0/765612329 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2c8103120 con 0x7ff2c8102d40 2024-09-11T22:05:42.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:42.136+0000 7ff2c4ff9640 1 -- 172.21.15.1:0/765612329 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7ff2b8042030 con 0x7ff2c8102d40 2024-09-11T22:05:42.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:42.136+0000 7ff2c4ff9640 1 --2- 172.21.15.1:0/765612329 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7ff2b40a2cf0 0x7ff2b40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:42.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:42.137+0000 7ff2be7fc640 1 --2- 172.21.15.1:0/765612329 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7ff2b40a2cf0 0x7ff2b40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:42.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:42.137+0000 7ff2c4ff9640 1 -- 172.21.15.1:0/765612329 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8934+0+0 (secure 0 0 0) 0x7ff2b80effc0 con 0x7ff2c8102d40 2024-09-11T22:05:42.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:42.137+0000 7ff2be7fc640 1 --2- 172.21.15.1:0/765612329 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7ff2b40a2cf0 0x7ff2b40a51b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0005fd0 tx=0x7ff2b0005f60 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:42.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:42.140+0000 7ff2c4ff9640 1 -- 172.21.15.1:0/765612329 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff2b80bcb00 con 0x7ff2c8102d40 2024-09-11T22:05:42.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:42.353+0000 7ff2cd76f640 1 -- 172.21.15.1:0/765612329 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff2c8199750 con 0x7ff2c8102d40 2024-09-11T22:05:42.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:42.354+0000 7ff2c4ff9640 1 -- 172.21.15.1:0/765612329 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11984 (secure 0 0 0) 0x7ff2b80bc400 con 0x7ff2c8102d40 2024-09-11T22:05:42.355 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T22:05:42.355 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":42,"btime":"2024-09-11T22:03:27:040852+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/421542832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":421542832},{"type":"v1","addr":"172.21.15.1:6835","nonce":421542832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2836864406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2836864406},{"type":"v1","addr":"172.21.15.1:6837","nonce":2836864406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":42,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T21:55:20.791263+0000","modified":"2024-09-11T22:03:26.485134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24484,"mds_2":24467,"mds_3":24466,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14595":{"gid":14595,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/407834672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":407834672},{"type":"v1","addr":"172.21.15.1:6839","nonce":407834672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/1068827678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":1068827678},{"type":"v1","addr":"172.21.15.1:6841","nonce":1068827678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/872410024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":872410024},{"type":"v1","addr":"172.21.15.161:6833","nonce":872410024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/1648944570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1648944570},{"type":"v1","addr":"172.21.15.53:6833","nonce":1648944570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/2126933993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2126933993},{"type":"v1","addr":"172.21.15.53:6835","nonce":2126933993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.161:6835/828381592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":828381592},{"type":"v1","addr":"172.21.15.161:6835","nonce":828381592}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.161:6837/1799318656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1799318656},{"type":"v1","addr":"172.21.15.161:6837","nonce":1799318656}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.53:6837/3457131830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3457131830},{"type":"v1","addr":"172.21.15.53:6837","nonce":3457131830}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/455365383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":455365383},{"type":"v1","addr":"172.21.15.161:6839","nonce":455365383}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.53:6839/1378584447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1378584447},{"type":"v1","addr":"172.21.15.53:6839","nonce":1378584447}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24460,24466,24467,24484,24494]},"id":1}]} 2024-09-11T22:05:42.355 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 42 2024-09-11T22:05:42.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:42.356+0000 7ff2cd76f640 1 -- 172.21.15.1:0/765612329 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7ff2b40a2cf0 msgr2=0x7ff2b40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:42.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:42.356+0000 7ff2cd76f640 1 --2- 172.21.15.1:0/765612329 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7ff2b40a2cf0 0x7ff2b40a51b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0005fd0 tx=0x7ff2b0005f60 comp rx=0 tx=0).stop 2024-09-11T22:05:42.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:42.356+0000 7ff2cd76f640 1 -- 172.21.15.1:0/765612329 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2c8102d40 msgr2=0x7ff2c81983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:42.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:42.356+0000 7ff2cd76f640 1 --2- 172.21.15.1:0/765612329 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2c8102d40 0x7ff2c81983f0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7ff2b8006010 tx=0x7ff2b80042f0 comp rx=0 tx=0).stop 2024-09-11T22:05:42.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:42.356+0000 7ff2cd76f640 1 -- 172.21.15.1:0/765612329 shutdown_connections 2024-09-11T22:05:42.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:42.356+0000 7ff2cd76f640 1 --2- 172.21.15.1:0/765612329 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7ff2b40a2cf0 0x7ff2b40a51b0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:42.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:42.356+0000 7ff2cd76f640 1 --2- 172.21.15.1:0/765612329 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2c810c170 0x7ff2c819ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:42.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:42.356+0000 7ff2cd76f640 1 --2- 172.21.15.1:0/765612329 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff2c8103660 0x7ff2c8198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:42.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:42.356+0000 7ff2cd76f640 1 --2- 172.21.15.1:0/765612329 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2c8102d40 0x7ff2c81983f0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:42.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:42.356+0000 7ff2cd76f640 1 -- 172.21.15.1:0/765612329 >> 172.21.15.1:0/765612329 conn(0x7ff2c80fc7a0 msgr2=0x7ff2c80fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:42.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:42.356+0000 7ff2cd76f640 1 -- 172.21.15.1:0/765612329 shutdown_connections 2024-09-11T22:05:42.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:42.356+0000 7ff2cd76f640 1 -- 172.21.15.1:0/765612329 wait complete. 2024-09-11T22:05:42.505 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:42.505+0000 7f682b7fe640 1 -- 172.21.15.1:0/1218656940 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "foo.201307"}]=0 added data pool 4 to fsmap v43) ==== 133+0+0 (secure 0 0 0) 0x7f683c199720 con 0x7f683c106a80 2024-09-11T22:05:42.652 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:42.652+0000 7f6840bdf640 1 -- 172.21.15.1:0/1218656940 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "foo.201307"} v 0) -- 0x7f683c02d7c0 con 0x7f683c106a80 2024-09-11T22:05:42.653 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:42.652+0000 7f682b7fe640 1 -- 172.21.15.1:0/1218656940 <== mon.0 v2:172.21.15.1:3300/0 8 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "foo.201307"}]=-22 RADOS pool 'foo.201307' is already used by filesystem 'cephfs' as a 'data' pool for application 'cephfs' v43) ==== 211+0+0 (secure 0 0 0) 0x7f68340bc200 con 0x7f683c106a80 2024-09-11T22:05:42.653 INFO:tasks.workunit.client.0.smithi001.stderr:Second attempt of previously successful command failed with EINVAL: RADOS pool 'foo.201307' is already used by filesystem 'cephfs' as a 'data' pool for application 'cephfs' 2024-09-11T22:05:42.653 INFO:tasks.workunit.client.0.smithi001.stderr:RADOS pool 'foo.201307' is already used by filesystem 'cephfs' as a 'data' pool for application 'cephfs' 2024-09-11T22:05:42.655 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:42.654+0000 7f6840bdf640 1 -- 172.21.15.1:0/1218656940 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f68240a2d80 msgr2=0x7f68240a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:42.655 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:42.654+0000 7f6840bdf640 1 --2- 172.21.15.1:0/1218656940 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f68240a2d80 0x7f68240a5240 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f682c005c40 tx=0x7f682c005bd0 comp rx=0 tx=0).stop 2024-09-11T22:05:42.655 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:42.654+0000 7f6840bdf640 1 -- 172.21.15.1:0/1218656940 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f683c106a80 msgr2=0x7f683c19cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:42.655 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:42.654+0000 7f6840bdf640 1 --2- 172.21.15.1:0/1218656940 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f683c106a80 0x7f683c19cc90 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f6834005ab0 tx=0x7f68340042f0 comp rx=0 tx=0).stop 2024-09-11T22:05:42.655 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:42.654+0000 7f6840bdf640 1 -- 172.21.15.1:0/1218656940 shutdown_connections 2024-09-11T22:05:42.655 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:42.654+0000 7f6840bdf640 1 --2- 172.21.15.1:0/1218656940 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f68240a2d80 0x7f68240a5240 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:42.655 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:42.654+0000 7f6840bdf640 1 --2- 172.21.15.1:0/1218656940 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f683c106a80 0x7f683c19cc90 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:42.656 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:42.654+0000 7f6840bdf640 1 --2- 172.21.15.1:0/1218656940 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f683c101870 0x7f683c198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:42.656 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:42.654+0000 7f6840bdf640 1 --2- 172.21.15.1:0/1218656940 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f683c100ed0 0x7f683c1983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:42.656 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:42.654+0000 7f6840bdf640 1 -- 172.21.15.1:0/1218656940 >> 172.21.15.1:0/1218656940 conn(0x7f683c076830 msgr2=0x7f683c069950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:42.656 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:42.654+0000 7f6840bdf640 1 -- 172.21.15.1:0/1218656940 shutdown_connections 2024-09-11T22:05:42.656 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:42.654+0000 7f6840bdf640 1 -- 172.21.15.1:0/1218656940 wait complete. 2024-09-11T22:05:42.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:42 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/3946002492' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "foo.201307", "pg_num": 8}]: dispatch 2024-09-11T22:05:42.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:42 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/3946002492' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "foo.201307", "pg_num": 8}]': finished 2024-09-11T22:05:42.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:42 smithi001 ceph-mon[30935]: osdmap e92: 12 total, 12 up, 12 in 2024-09-11T22:05:42.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:42 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/3946002492' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "foo.201307", "pg_num": 8}]: dispatch 2024-09-11T22:05:42.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:42 smithi001 ceph-mon[30935]: pgmap v262: 105 pgs: 8 unknown, 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-09-11T22:05:42.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:42 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/1218656940' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "foo.201307"}]: dispatch 2024-09-11T22:05:42.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:42 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/765612329' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:05:42.761 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:42.760+0000 7f84411e8640 1 -- 172.21.15.1:0/1228829795 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f843c106a20 msgr2=0x7f843c106e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:42.761 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:42.760+0000 7f84411e8640 1 --2- 172.21.15.1:0/1228829795 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f843c106a20 0x7f843c106e00 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f842c009930 tx=0x7f842c02a420 comp rx=0 tx=0).stop 2024-09-11T22:05:42.762 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:42.760+0000 7f84411e8640 1 -- 172.21.15.1:0/1228829795 shutdown_connections 2024-09-11T22:05:42.762 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:42.760+0000 7f84411e8640 1 --2- 172.21.15.1:0/1228829795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f843c1013b0 0x7f843c10d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:42.762 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:42.760+0000 7f84411e8640 1 --2- 172.21.15.1:0/1228829795 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f843c100a10 0x7f843c100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:42.762 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:42.760+0000 7f84411e8640 1 --2- 172.21.15.1:0/1228829795 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f843c106a20 0x7f843c106e00 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:42.762 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:42.760+0000 7f84411e8640 1 -- 172.21.15.1:0/1228829795 >> 172.21.15.1:0/1228829795 conn(0x7f843c0fc6f0 msgr2=0x7f843c0feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:42.762 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:42.760+0000 7f84411e8640 1 -- 172.21.15.1:0/1228829795 shutdown_connections 2024-09-11T22:05:42.762 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:42.760+0000 7f84411e8640 1 -- 172.21.15.1:0/1228829795 wait complete. 2024-09-11T22:05:42.762 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:42.761+0000 7f84411e8640 1 Processor -- start 2024-09-11T22:05:42.762 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:42.761+0000 7f84411e8640 1 -- start start 2024-09-11T22:05:42.764 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:42.761+0000 7f84411e8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f843c100a10 0x7f843c199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:42.764 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:42.761+0000 7f84411e8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f843c1013b0 0x7f843c199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:42.764 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:42.761+0000 7f84411e8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f843c106a20 0x7f843c1a4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:42.764 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:42.761+0000 7f84411e8640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f843c110e60 con 0x7f843c106a20 2024-09-11T22:05:42.764 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:42.761+0000 7f84411e8640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f843c110ce0 con 0x7f843c100a10 2024-09-11T22:05:42.764 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:42.761+0000 7f84411e8640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f843c110fe0 con 0x7f843c1013b0 2024-09-11T22:05:42.765 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:42.762+0000 7f843b59e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f843c106a20 0x7f843c1a4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:42.765 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:42.762+0000 7f843b59e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f843c106a20 0x7f843c1a4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44028/0 (socket says 172.21.15.1:44028) 2024-09-11T22:05:42.765 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:42.762+0000 7f843b59e640 1 -- 172.21.15.1:0/110930524 learned_addr learned my addr 172.21.15.1:0/110930524 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:42.765 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:42.762+0000 7f843259c640 1 --2- 172.21.15.1:0/110930524 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f843c1013b0 0x7f843c199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:42.765 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:42.762+0000 7f843ad9d640 1 --2- 172.21.15.1:0/110930524 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f843c100a10 0x7f843c199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:42.765 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:42.762+0000 7f843b59e640 1 -- 172.21.15.1:0/110930524 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f843c1013b0 msgr2=0x7f843c199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:42.765 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:42.762+0000 7f843b59e640 1 --2- 172.21.15.1:0/110930524 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f843c1013b0 0x7f843c199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:42.765 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:42.762+0000 7f843b59e640 1 -- 172.21.15.1:0/110930524 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f843c100a10 msgr2=0x7f843c199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:42.765 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:42.762+0000 7f843b59e640 1 --2- 172.21.15.1:0/110930524 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f843c100a10 0x7f843c199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:42.765 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:42.762+0000 7f843b59e640 1 -- 172.21.15.1:0/110930524 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f843c1a48d0 con 0x7f843c106a20 2024-09-11T22:05:42.765 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:42.762+0000 7f843b59e640 1 --2- 172.21.15.1:0/110930524 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f843c106a20 0x7f843c1a4390 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f843400b970 tx=0x7f843400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:42.765 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:42.762+0000 7f8438d99640 1 -- 172.21.15.1:0/110930524 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8434019070 con 0x7f843c106a20 2024-09-11T22:05:42.766 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:42.762+0000 7f8438d99640 1 -- 172.21.15.1:0/110930524 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f843400cb40 con 0x7f843c106a20 2024-09-11T22:05:42.766 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:42.762+0000 7f8438d99640 1 -- 172.21.15.1:0/110930524 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8434007650 con 0x7f843c106a20 2024-09-11T22:05:42.766 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:42.762+0000 7f84411e8640 1 -- 172.21.15.1:0/110930524 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f843c1a4aa0 con 0x7f843c106a20 2024-09-11T22:05:42.766 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:42.762+0000 7f84411e8640 1 -- 172.21.15.1:0/110930524 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f843c1a4fc0 con 0x7f843c106a20 2024-09-11T22:05:42.766 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:42.763+0000 7f84411e8640 1 -- 172.21.15.1:0/110930524 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f843c100e70 con 0x7f843c106a20 2024-09-11T22:05:42.767 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:42.765+0000 7f8438d99640 1 -- 172.21.15.1:0/110930524 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f843401c070 con 0x7f843c106a20 2024-09-11T22:05:42.767 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:42.765+0000 7f8438d99640 1 --2- 172.21.15.1:0/110930524 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f84280a2d80 0x7f84280a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:42.767 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:42.766+0000 7f843ad9d640 1 --2- 172.21.15.1:0/110930524 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f84280a2d80 0x7f84280a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:42.767 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:42.766+0000 7f8438d99640 1 -- 172.21.15.1:0/110930524 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8966+0+0 (secure 0 0 0) 0x7f84340cea30 con 0x7f843c106a20 2024-09-11T22:05:42.767 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:42.766+0000 7f843ad9d640 1 --2- 172.21.15.1:0/110930524 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f84280a2d80 0x7f84280a5240 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f842c000c00 tx=0x7f842c031490 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:42.770 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:42.770+0000 7f8438d99640 1 -- 172.21.15.1:0/110930524 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f843409af50 con 0x7f843c106a20 2024-09-11T22:05:42.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:42 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/3946002492' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "foo.201307", "pg_num": 8}]: dispatch 2024-09-11T22:05:42.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:42 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/3946002492' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "foo.201307", "pg_num": 8}]': finished 2024-09-11T22:05:42.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:42 smithi053 ceph-mon[34908]: osdmap e92: 12 total, 12 up, 12 in 2024-09-11T22:05:42.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:42 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/3946002492' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "foo.201307", "pg_num": 8}]: dispatch 2024-09-11T22:05:42.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:42 smithi053 ceph-mon[34908]: pgmap v262: 105 pgs: 8 unknown, 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-09-11T22:05:42.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:42 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/1218656940' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "foo.201307"}]: dispatch 2024-09-11T22:05:42.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:42 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/765612329' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:05:42.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:42 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/3946002492' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "foo.201307", "pg_num": 8}]: dispatch 2024-09-11T22:05:42.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:42 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/3946002492' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "foo.201307", "pg_num": 8}]': finished 2024-09-11T22:05:42.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:42 smithi161 ceph-mon[35615]: osdmap e92: 12 total, 12 up, 12 in 2024-09-11T22:05:42.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:42 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/3946002492' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "foo.201307", "pg_num": 8}]: dispatch 2024-09-11T22:05:42.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:42 smithi161 ceph-mon[35615]: pgmap v262: 105 pgs: 8 unknown, 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-09-11T22:05:42.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:42 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/1218656940' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "foo.201307"}]: dispatch 2024-09-11T22:05:42.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:42 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/765612329' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:05:42.984 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:42.983+0000 7f84411e8640 1 -- 172.21.15.1:0/110930524 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs rm_data_pool", "fs_name": "cephfs", "pool": "foo.201307"} v 0) -- 0x7f843c02d7c0 con 0x7f843c106a20 2024-09-11T22:05:43.056 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T22:05:43.312 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:05:43.507 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:43.507+0000 7f8438d99640 1 -- 172.21.15.1:0/110930524 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm_data_pool", "fs_name": "cephfs", "pool": "foo.201307"}]=0 removed data pool 4 from fsmap v44) ==== 136+0+0 (secure 0 0 0) 0x7f843409f250 con 0x7f843c106a20 2024-09-11T22:05:43.652 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:43.651+0000 7f84411e8640 1 -- 172.21.15.1:0/110930524 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs rm_data_pool", "fs_name": "cephfs", "pool": "foo.201307"} v 0) -- 0x7f843c108f80 con 0x7f843c106a20 2024-09-11T22:05:43.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:43 smithi001 ceph-mon[30935]: osdmap e93: 12 total, 12 up, 12 in 2024-09-11T22:05:43.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:43 smithi001 ceph-mon[30935]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T22:05:43.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:43 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/1218656940' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "foo.201307"}]': finished 2024-09-11T22:05:43.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:43 smithi001 ceph-mon[30935]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:05:43.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:43 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/1218656940' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "foo.201307"}]: dispatch 2024-09-11T22:05:43.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:43 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/110930524' entity='client.admin' cmd=[{"prefix": "fs rm_data_pool", "fs_name": "cephfs", "pool": "foo.201307"}]: dispatch 2024-09-11T22:05:43.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:43 smithi053 ceph-mon[34908]: osdmap e93: 12 total, 12 up, 12 in 2024-09-11T22:05:43.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:43 smithi053 ceph-mon[34908]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T22:05:43.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:43 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/1218656940' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "foo.201307"}]': finished 2024-09-11T22:05:43.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:43 smithi053 ceph-mon[34908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:05:43.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:43 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/1218656940' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "foo.201307"}]: dispatch 2024-09-11T22:05:43.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:43 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/110930524' entity='client.admin' cmd=[{"prefix": "fs rm_data_pool", "fs_name": "cephfs", "pool": "foo.201307"}]: dispatch 2024-09-11T22:05:43.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.810+0000 7fcee52cb640 1 -- 172.21.15.1:0/1328784687 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcee010e4f0 msgr2=0x7fcee0110900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:43.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.810+0000 7fcee52cb640 1 --2- 172.21.15.1:0/1328784687 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcee010e4f0 0x7fcee0110900 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fced8009900 tx=0x7fced802a410 comp rx=0 tx=0).stop 2024-09-11T22:05:43.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.810+0000 7fcee52cb640 1 -- 172.21.15.1:0/1328784687 shutdown_connections 2024-09-11T22:05:43.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.810+0000 7fcee52cb640 1 --2- 172.21.15.1:0/1328784687 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcee010e4f0 0x7fcee0110900 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:43.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.810+0000 7fcee52cb640 1 --2- 172.21.15.1:0/1328784687 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcee0101380 0x7fcee010dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:43.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.810+0000 7fcee52cb640 1 --2- 172.21.15.1:0/1328784687 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcee01009d0 0x7fcee0100db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:43.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.810+0000 7fcee52cb640 1 -- 172.21.15.1:0/1328784687 >> 172.21.15.1:0/1328784687 conn(0x7fcee00fc7d0 msgr2=0x7fcee00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:43.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.810+0000 7fcee52cb640 1 -- 172.21.15.1:0/1328784687 shutdown_connections 2024-09-11T22:05:43.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.810+0000 7fcee52cb640 1 -- 172.21.15.1:0/1328784687 wait complete. 2024-09-11T22:05:43.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.811+0000 7fcee52cb640 1 Processor -- start 2024-09-11T22:05:43.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.811+0000 7fcee52cb640 1 -- start start 2024-09-11T22:05:43.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.811+0000 7fcee52cb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcee01009d0 0x7fcee019a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:43.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.811+0000 7fcee52cb640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcee0101380 0x7fcee019ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:43.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.811+0000 7fcee52cb640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcee010e4f0 0x7fcee019f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:43.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.811+0000 7fcee52cb640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcee01142a0 con 0x7fcee010e4f0 2024-09-11T22:05:43.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.811+0000 7fcee52cb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcee0114120 con 0x7fcee01009d0 2024-09-11T22:05:43.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.811+0000 7fcee52cb640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcee0114420 con 0x7fcee0101380 2024-09-11T22:05:43.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.812+0000 7fcedf7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcee010e4f0 0x7fcee019f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:43.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.812+0000 7fcedf7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcee010e4f0 0x7fcee019f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44044/0 (socket says 172.21.15.1:44044) 2024-09-11T22:05:43.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.812+0000 7fcede7fc640 1 --2- 172.21.15.1:0/2739795571 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcee0101380 0x7fcee019ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:43.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.812+0000 7fcedf7fe640 1 -- 172.21.15.1:0/2739795571 learned_addr learned my addr 172.21.15.1:0/2739795571 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:43.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.812+0000 7fcedeffd640 1 --2- 172.21.15.1:0/2739795571 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcee01009d0 0x7fcee019a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:43.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.812+0000 7fcedf7fe640 1 -- 172.21.15.1:0/2739795571 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcee0101380 msgr2=0x7fcee019ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:43.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.812+0000 7fcedf7fe640 1 --2- 172.21.15.1:0/2739795571 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcee0101380 0x7fcee019ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:43.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.812+0000 7fcedf7fe640 1 -- 172.21.15.1:0/2739795571 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcee01009d0 msgr2=0x7fcee019a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:43.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.812+0000 7fcedf7fe640 1 --2- 172.21.15.1:0/2739795571 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcee01009d0 0x7fcee019a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:43.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.812+0000 7fcedf7fe640 1 -- 172.21.15.1:0/2739795571 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcee019f6e0 con 0x7fcee010e4f0 2024-09-11T22:05:43.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.812+0000 7fcedf7fe640 1 --2- 172.21.15.1:0/2739795571 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcee010e4f0 0x7fcee019f000 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fced8005ab0 tx=0x7fced80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:43.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.812+0000 7fced7fff640 1 -- 172.21.15.1:0/2739795571 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced803f030 con 0x7fcee010e4f0 2024-09-11T22:05:43.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.812+0000 7fced7fff640 1 -- 172.21.15.1:0/2739795571 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fced8003ce0 con 0x7fcee010e4f0 2024-09-11T22:05:43.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.812+0000 7fcee52cb640 1 -- 172.21.15.1:0/2739795571 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcee019f970 con 0x7fcee010e4f0 2024-09-11T22:05:43.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.812+0000 7fced7fff640 1 -- 172.21.15.1:0/2739795571 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced8035070 con 0x7fcee010e4f0 2024-09-11T22:05:43.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.812+0000 7fcee52cb640 1 -- 172.21.15.1:0/2739795571 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcee01a72b0 con 0x7fcee010e4f0 2024-09-11T22:05:43.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.815+0000 7fced7fff640 1 -- 172.21.15.1:0/2739795571 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7fced8042030 con 0x7fcee010e4f0 2024-09-11T22:05:43.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.815+0000 7fcee76a4740 1 Processor -- start 2024-09-11T22:05:43.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.815+0000 7fcee76a4740 1 -- start start 2024-09-11T22:05:43.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.815+0000 7fcee76a4740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb23235b40 0x55eb23234090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:43.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.815+0000 7fcee76a4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eb232345d0 0x55eb23234a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:43.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.815+0000 7fcee76a4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb2323f370 0x55eb23241760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:43.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.815+0000 7fcee76a4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55eb2319a2f0 con 0x55eb232345d0 2024-09-11T22:05:43.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.815+0000 7fcee76a4740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55eb23141cf0 con 0x55eb23235b40 2024-09-11T22:05:43.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.815+0000 7fcee76a4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55eb231962b0 con 0x55eb2323f370 2024-09-11T22:05:43.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.815+0000 7fced7fff640 1 --2- 172.21.15.1:0/2739795571 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fcecc0a2d70 0x7fcecc0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:43.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.816+0000 7fcedeffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eb232345d0 0x55eb23234a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:43.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.816+0000 7fcedeffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eb232345d0 0x55eb23234a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44058/0 (socket says 172.21.15.1:44058) 2024-09-11T22:05:43.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.816+0000 7fcedeffd640 1 -- 172.21.15.1:0/2751442882 learned_addr learned my addr 172.21.15.1:0/2751442882 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:43.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.816+0000 7fcede7fc640 1 --2- 172.21.15.1:0/2751442882 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb23235b40 0x55eb23234090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:43.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.816+0000 7fced7fff640 1 -- 172.21.15.1:0/2739795571 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8966+0+0 (secure 0 0 0) 0x7fced80efea0 con 0x7fcee010e4f0 2024-09-11T22:05:43.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.816+0000 7fcedf7fe640 1 --2- 172.21.15.1:0/2739795571 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fcecc0a2d70 0x7fcecc0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:43.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.816+0000 7fcede7fc640 1 --2- 172.21.15.1:0/2751442882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb2323f370 0x55eb23241760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:43.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.816+0000 7fcedeffd640 1 -- 172.21.15.1:0/2751442882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb2323f370 msgr2=0x55eb23241760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:43.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.816+0000 7fcedeffd640 1 --2- 172.21.15.1:0/2751442882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb2323f370 0x55eb23241760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:43.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.816+0000 7fcedeffd640 1 -- 172.21.15.1:0/2751442882 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb23235b40 msgr2=0x55eb23234090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:43.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.816+0000 7fcedeffd640 1 --2- 172.21.15.1:0/2751442882 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb23235b40 0x55eb23234090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:43.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.816+0000 7fcedeffd640 1 -- 172.21.15.1:0/2751442882 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb231263a0 con 0x55eb232345d0 2024-09-11T22:05:43.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.816+0000 7fcedeffd640 1 --2- 172.21.15.1:0/2751442882 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eb232345d0 0x55eb23234a10 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fced000ae90 tx=0x7fced000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=596bb9807911c2f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:43.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.816+0000 7fcedf7fe640 1 --2- 172.21.15.1:0/2739795571 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fcecc0a2d70 0x7fcecc0a5230 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fced80ed490 tx=0x7fced803b600 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:43.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.816+0000 7fced57fa640 1 -- 172.21.15.1:0/2751442882 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced0011850 con 0x55eb232345d0 2024-09-11T22:05:43.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.816+0000 7fced57fa640 1 -- 172.21.15.1:0/2751442882 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fced001a480 con 0x55eb232345d0 2024-09-11T22:05:43.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.816+0000 7fced57fa640 1 -- 172.21.15.1:0/2751442882 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced0010630 con 0x55eb232345d0 2024-09-11T22:05:43.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.817+0000 7fcee76a4740 1 -- 172.21.15.1:0/2751442882 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eb232345d0 msgr2=0x55eb23234a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:43.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.817+0000 7fcee76a4740 1 --2- 172.21.15.1:0/2751442882 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eb232345d0 0x55eb23234a10 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fced000ae90 tx=0x7fced000f730 comp rx=0 tx=0).stop 2024-09-11T22:05:43.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.817+0000 7fcee76a4740 1 -- 172.21.15.1:0/2751442882 shutdown_connections 2024-09-11T22:05:43.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.817+0000 7fcee76a4740 1 --2- 172.21.15.1:0/2751442882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb2323f370 0x55eb23241760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:43.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.817+0000 7fcee76a4740 1 --2- 172.21.15.1:0/2751442882 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eb232345d0 0x55eb23234a10 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:43.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.817+0000 7fcee76a4740 1 --2- 172.21.15.1:0/2751442882 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb23235b40 0x55eb23234090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:43.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.817+0000 7fcee76a4740 1 -- 172.21.15.1:0/2751442882 >> 172.21.15.1:0/2751442882 conn(0x55eb231fd310 msgr2=0x55eb23230c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:43.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.817+0000 7fcee76a4740 1 -- 172.21.15.1:0/2751442882 shutdown_connections 2024-09-11T22:05:43.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.817+0000 7fcee76a4740 1 -- 172.21.15.1:0/2751442882 wait complete. 2024-09-11T22:05:43.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.818+0000 7fcee76a4740 1 -- start start 2024-09-11T22:05:43.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.818+0000 7fcee76a4740 1 Processor -- start 2024-09-11T22:05:43.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.818+0000 7fcee76a4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb232345d0 0x55eb2323ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:43.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.818+0000 7fcee76a4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eb23235b40 0x55eb2323c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:43.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.818+0000 7fcee76a4740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb2323f370 0x55eb2323c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:43.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.818+0000 7fcee76a4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55eb23242070 con 0x55eb23235b40 2024-09-11T22:05:43.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.818+0000 7fcee76a4740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55eb23155060 con 0x55eb2323f370 2024-09-11T22:05:43.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.818+0000 7fcee76a4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55eb232421f0 con 0x55eb232345d0 2024-09-11T22:05:43.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.818+0000 7fcee76a4740 10 client.0 _ll_register_callbacks cb 0x55eb22ea88c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:05:43.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.818+0000 7fcedeffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eb23235b40 0x55eb2323c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:43.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.818+0000 7fcedeffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eb23235b40 0x55eb2323c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44062/0 (socket says 172.21.15.1:44062) 2024-09-11T22:05:43.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.818+0000 7fcedeffd640 1 -- 172.21.15.1:0/3030413907 learned_addr learned my addr 172.21.15.1:0/3030413907 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:43.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.818+0000 7fcede7fc640 1 --2- 172.21.15.1:0/3030413907 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb232345d0 0x55eb2323ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:43.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.818+0000 7fcede7fc640 1 --2- 172.21.15.1:0/3030413907 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb2323f370 0x55eb2323c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:43.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.818+0000 7fcedeffd640 1 -- 172.21.15.1:0/3030413907 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb232345d0 msgr2=0x55eb2323ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:43.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.818+0000 7fcedeffd640 1 --2- 172.21.15.1:0/3030413907 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb232345d0 0x55eb2323ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:43.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.818+0000 7fcedeffd640 1 -- 172.21.15.1:0/3030413907 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb2323f370 msgr2=0x55eb2323c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:43.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.818+0000 7fcedeffd640 1 --2- 172.21.15.1:0/3030413907 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb2323f370 0x55eb2323c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:43.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.818+0000 7fcedeffd640 1 -- 172.21.15.1:0/3030413907 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb22f0b3c0 con 0x55eb23235b40 2024-09-11T22:05:43.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.818+0000 7fcede7fc640 1 --2- 172.21.15.1:0/3030413907 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb232345d0 0x55eb2323ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:05:43.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.819+0000 7fcedeffd640 1 --2- 172.21.15.1:0/3030413907 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eb23235b40 0x55eb2323c030 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fced000ac50 tx=0x7fced001a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac5696c220309eb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:43.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.819+0000 7fcec67fc640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:05:43.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.819+0000 7fcec67fc640 1 -- 172.21.15.1:0/3030413907 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced001a960 con 0x55eb23235b40 2024-09-11T22:05:43.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.819+0000 7fcee76a4740 1 -- 172.21.15.1:0/3030413907 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fced001a480 con 0x55eb23235b40 2024-09-11T22:05:43.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.819+0000 7fcec67fc640 1 -- 172.21.15.1:0/3030413907 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8966+0+0 (secure 0 0 0) 0x7fced0004a40 con 0x55eb23235b40 2024-09-11T22:05:43.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.819+0000 7fcec67fc640 1 -- 172.21.15.1:0/3030413907 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fced0004c40 con 0x55eb23235b40 2024-09-11T22:05:43.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.819+0000 7fcee76a4740 10 client.17406 fetch_fsmap learned FSMap version 44 2024-09-11T22:05:43.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.819+0000 7fcee76a4740 1 -- 172.21.15.1:0/3030413907 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55eb2324db40 con 0x55eb23235b40 2024-09-11T22:05:43.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.819+0000 7fcec67fc640 1 -- 172.21.15.1:0/3030413907 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fced001a480 con 0x55eb23235b40 2024-09-11T22:05:43.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.819+0000 7fcec67fc640 1 -- 172.21.15.1:0/3030413907 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 44) ==== 8442+0+0 (secure 0 0 0) 0x7fced0005340 con 0x55eb23235b40 2024-09-11T22:05:43.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.819+0000 7fcee76a4740 10 client.17406 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T22:05:43.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.819+0000 7fcee76a4740 10 client.17406 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-11T22:05:43.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.819+0000 7fcee76a4740 20 client.17406 populate_metadata read hostname 'smithi001' 2024-09-11T22:05:43.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.820+0000 7fcee76a4740 1 --2- 172.21.15.1:0/3030413907 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x55eb2324eef0 0x55eb232512e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:43.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.820+0000 7fcee76a4740 4 client.17406 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:05:43.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.820+0000 7fcee76a4740 1 -- 172.21.15.1:0/3030413907 --> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb231a7860 con 0x55eb2324eef0 2024-09-11T22:05:43.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.820+0000 7fcede7fc640 1 --2- 172.21.15.1:0/3030413907 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x55eb2324eef0 0x55eb232512e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:43.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.820+0000 7fcede7fc640 1 --2- 172.21.15.1:0/3030413907 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x55eb2324eef0 0x55eb232512e0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2cc34097da7f2527 server_cookie=6abc006d0e919c3c in_seq=0 out_seq=0 2024-09-11T22:05:43.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.820+0000 7fcec67fc640 10 client.17406 ms_handle_connect on v2:172.21.15.161:6834/828381592 2024-09-11T22:05:43.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.821+0000 7fcec67fc640 1 -- 172.21.15.1:0/3030413907 <== mds.4 v2:172.21.15.161:6834/828381592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eb231a7860 con 0x55eb2324eef0 2024-09-11T22:05:43.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.821+0000 7fcec67fc640 10 client.17406 handle_command_reply: tid=0 2024-09-11T22:05:43.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.821+0000 7fcee76a4740 1 client.17406 shutdown 2024-09-11T22:05:43.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.821+0000 7fcee76a4740 1 -- 172.21.15.1:0/3030413907 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eb23235b40 msgr2=0x55eb2323c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:43.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.821+0000 7fcee76a4740 1 --2- 172.21.15.1:0/3030413907 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eb23235b40 0x55eb2323c030 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fced000ac50 tx=0x7fced001a630 comp rx=0 tx=0).stop 2024-09-11T22:05:43.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.821+0000 7fcee76a4740 1 -- 172.21.15.1:0/3030413907 shutdown_connections 2024-09-11T22:05:43.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.821+0000 7fcee76a4740 1 --2- 172.21.15.1:0/3030413907 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x55eb2324eef0 0x55eb232512e0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:43.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.821+0000 7fcee76a4740 1 --2- 172.21.15.1:0/3030413907 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb2323f370 0x55eb2323c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:43.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.821+0000 7fcec67fc640 0 client.17406 ms_handle_reset on v2:172.21.15.161:6834/828381592 2024-09-11T22:05:43.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.821+0000 7fcee76a4740 1 --2- 172.21.15.1:0/3030413907 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eb23235b40 0x55eb2323c030 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:43.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.821+0000 7fcee76a4740 1 --2- 172.21.15.1:0/3030413907 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb232345d0 0x55eb2323ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:43.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.821+0000 7fcee76a4740 1 -- 172.21.15.1:0/3030413907 >> 172.21.15.1:0/3030413907 conn(0x55eb231fd310 msgr2=0x55eb23236930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:43.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.821+0000 7fcee76a4740 1 -- 172.21.15.1:0/3030413907 shutdown_connections 2024-09-11T22:05:43.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.822+0000 7fcee76a4740 1 -- 172.21.15.1:0/3030413907 wait complete. 2024-09-11T22:05:43.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.822+0000 7fcee76a4740 20 client.17406 trim_cache size 0 max 16384 2024-09-11T22:05:43.845 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:43 smithi161 ceph-mon[35615]: osdmap e93: 12 total, 12 up, 12 in 2024-09-11T22:05:43.845 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:43 smithi161 ceph-mon[35615]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T22:05:43.845 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:43 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/1218656940' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "foo.201307"}]': finished 2024-09-11T22:05:43.845 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:43 smithi161 ceph-mon[35615]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:05:43.846 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:43 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/1218656940' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "foo.201307"}]: dispatch 2024-09-11T22:05:43.846 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:43 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/110930524' entity='client.admin' cmd=[{"prefix": "fs rm_data_pool", "fs_name": "cephfs", "pool": "foo.201307"}]: dispatch 2024-09-11T22:05:43.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.835+0000 7fcee76a4740 1 Processor -- start 2024-09-11T22:05:43.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.835+0000 7fcee76a4740 1 -- start start 2024-09-11T22:05:43.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.835+0000 7fcee76a4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb232345d0 0x55eb2323bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:43.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.835+0000 7fcee76a4740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb23235b40 0x55eb2324b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:43.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.835+0000 7fcee76a4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eb2323f370 0x55eb23230700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:43.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.835+0000 7fcee76a4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcee01b0320 con 0x55eb2323f370 2024-09-11T22:05:43.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.835+0000 7fcee76a4740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcee01b2160 con 0x55eb23235b40 2024-09-11T22:05:43.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.835+0000 7fcee76a4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcee01b0ab0 con 0x55eb232345d0 2024-09-11T22:05:43.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.835+0000 7fcede7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eb2323f370 0x55eb23230700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:43.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.835+0000 7fcede7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eb2323f370 0x55eb23230700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44070/0 (socket says 172.21.15.1:44070) 2024-09-11T22:05:43.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.835+0000 7fcede7fc640 1 -- 172.21.15.1:0/679095523 learned_addr learned my addr 172.21.15.1:0/679095523 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:43.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.836+0000 7fcede7fc640 1 --2- 172.21.15.1:0/679095523 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb232345d0 0x55eb2323bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:43.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.836+0000 7fcedeffd640 1 --2- 172.21.15.1:0/679095523 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb23235b40 0x55eb2324b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:43.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.836+0000 7fcede7fc640 1 -- 172.21.15.1:0/679095523 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb232345d0 msgr2=0x55eb2323bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:43.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.836+0000 7fcede7fc640 1 --2- 172.21.15.1:0/679095523 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb232345d0 0x55eb2323bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:43.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.836+0000 7fcede7fc640 1 -- 172.21.15.1:0/679095523 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb23235b40 msgr2=0x55eb2324b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:43.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.836+0000 7fcede7fc640 1 --2- 172.21.15.1:0/679095523 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb23235b40 0x55eb2324b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:43.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.836+0000 7fcede7fc640 1 -- 172.21.15.1:0/679095523 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb2323cfd0 con 0x55eb2323f370 2024-09-11T22:05:43.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.836+0000 7fcedeffd640 1 --2- 172.21.15.1:0/679095523 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb23235b40 0x55eb2324b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:05:43.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.836+0000 7fcede7fc640 1 --2- 172.21.15.1:0/679095523 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eb2323f370 0x55eb23230700 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x55eb23251ae0 tx=0x7fcec8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d43fc0fe7a336f45 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:43.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.836+0000 7fced4ff9640 1 -- 172.21.15.1:0/679095523 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec80185d0 con 0x55eb2323f370 2024-09-11T22:05:43.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.836+0000 7fced4ff9640 1 -- 172.21.15.1:0/679095523 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcec8028070 con 0x55eb2323f370 2024-09-11T22:05:43.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.836+0000 7fced4ff9640 1 -- 172.21.15.1:0/679095523 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec8021a80 con 0x55eb2323f370 2024-09-11T22:05:43.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.837+0000 7fcee76a4740 1 -- 172.21.15.1:0/679095523 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eb2323f370 msgr2=0x55eb23230700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:43.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.837+0000 7fcee76a4740 1 --2- 172.21.15.1:0/679095523 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eb2323f370 0x55eb23230700 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x55eb23251ae0 tx=0x7fcec8008040 comp rx=0 tx=0).stop 2024-09-11T22:05:43.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.837+0000 7fcee76a4740 1 -- 172.21.15.1:0/679095523 shutdown_connections 2024-09-11T22:05:43.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.837+0000 7fcee76a4740 1 --2- 172.21.15.1:0/679095523 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eb2323f370 0x55eb23230700 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:43.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.837+0000 7fcee76a4740 1 --2- 172.21.15.1:0/679095523 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb23235b40 0x55eb2324b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:43.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.837+0000 7fcee76a4740 1 --2- 172.21.15.1:0/679095523 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb232345d0 0x55eb2323bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:43.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.837+0000 7fcee76a4740 1 -- 172.21.15.1:0/679095523 >> 172.21.15.1:0/679095523 conn(0x55eb231fd310 msgr2=0x55eb2322ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:43.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.837+0000 7fcee76a4740 1 -- 172.21.15.1:0/679095523 shutdown_connections 2024-09-11T22:05:43.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.837+0000 7fcee76a4740 1 -- 172.21.15.1:0/679095523 wait complete. 2024-09-11T22:05:43.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.837+0000 7fcee76a4740 1 -- start start 2024-09-11T22:05:43.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.837+0000 7fcee76a4740 1 Processor -- start 2024-09-11T22:05:43.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.838+0000 7fcee76a4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eb232345d0 0x55eb23247a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:43.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.838+0000 7fcee76a4740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb23235b40 0x55eb23238bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:43.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.838+0000 7fcee76a4740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb2323f370 0x55eb23239110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:43.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.838+0000 7fcee76a4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcee01b19d0 con 0x55eb232345d0 2024-09-11T22:05:43.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.838+0000 7fcee76a4740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcee01b1240 con 0x55eb23235b40 2024-09-11T22:05:43.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.838+0000 7fcee76a4740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcee01ae300 con 0x55eb2323f370 2024-09-11T22:05:43.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.838+0000 7fcee76a4740 10 client.0 _ll_register_callbacks cb 0x7fcec8009520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:05:43.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.838+0000 7fcede7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eb232345d0 0x55eb23247a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:43.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.838+0000 7fcede7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eb232345d0 0x55eb23247a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44072/0 (socket says 172.21.15.1:44072) 2024-09-11T22:05:43.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.838+0000 7fcede7fc640 1 -- 172.21.15.1:0/4150607190 learned_addr learned my addr 172.21.15.1:0/4150607190 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:43.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.838+0000 7fcedeffd640 1 --2- 172.21.15.1:0/4150607190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb23235b40 0x55eb23238bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:43.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.838+0000 7fcede7fc640 1 -- 172.21.15.1:0/4150607190 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb2323f370 msgr2=0x55eb23239110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:43.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.838+0000 7fcede7fc640 1 --2- 172.21.15.1:0/4150607190 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb2323f370 0x55eb23239110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:43.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.838+0000 7fcede7fc640 1 -- 172.21.15.1:0/4150607190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb23235b40 msgr2=0x55eb23238bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:43.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.838+0000 7fcede7fc640 1 --2- 172.21.15.1:0/4150607190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb23235b40 0x55eb23238bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:43.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.838+0000 7fcede7fc640 1 -- 172.21.15.1:0/4150607190 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb23237120 con 0x55eb232345d0 2024-09-11T22:05:43.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.838+0000 7fcede7fc640 1 --2- 172.21.15.1:0/4150607190 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eb232345d0 0x55eb23247a30 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fcec8006a90 tx=0x7fcec8009110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2ba15b2e2e8a9db server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:43.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.838+0000 7fcec67fc640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:05:43.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.838+0000 7fcec67fc640 1 -- 172.21.15.1:0/4150607190 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec8030020 con 0x55eb232345d0 2024-09-11T22:05:43.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.838+0000 7fcec67fc640 1 -- 172.21.15.1:0/4150607190 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8966+0+0 (secure 0 0 0) 0x7fcec80185d0 con 0x55eb232345d0 2024-09-11T22:05:43.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.838+0000 7fcee76a4740 1 -- 172.21.15.1:0/4150607190 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcec8028070 con 0x55eb232345d0 2024-09-11T22:05:43.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.839+0000 7fcec67fc640 1 -- 172.21.15.1:0/4150607190 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fcec80174e0 con 0x55eb232345d0 2024-09-11T22:05:43.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.839+0000 7fcee76a4740 10 client.17412 fetch_fsmap learned FSMap version 44 2024-09-11T22:05:43.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.839+0000 7fcee76a4740 1 -- 172.21.15.1:0/4150607190 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55eb232b9710 con 0x55eb232345d0 2024-09-11T22:05:43.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.839+0000 7fcec67fc640 1 -- 172.21.15.1:0/4150607190 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcec8028070 con 0x55eb232345d0 2024-09-11T22:05:43.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.839+0000 7fcec67fc640 1 -- 172.21.15.1:0/4150607190 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 44) ==== 8442+0+0 (secure 0 0 0) 0x7fcec802c920 con 0x55eb232345d0 2024-09-11T22:05:43.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.839+0000 7fcee76a4740 10 client.17412 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T22:05:43.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.839+0000 7fcee76a4740 10 client.17412 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-11T22:05:43.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.839+0000 7fcee76a4740 20 client.17412 populate_metadata read hostname 'smithi001' 2024-09-11T22:05:43.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.839+0000 7fcee76a4740 1 --2- 172.21.15.1:0/4150607190 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x55eb232b9e40 0x55eb232bc260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:43.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.839+0000 7fcee76a4740 4 client.17412 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:05:43.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.839+0000 7fcee76a4740 1 -- 172.21.15.1:0/4150607190 --> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55eb231a1d70 con 0x55eb232b9e40 2024-09-11T22:05:43.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.839+0000 7fcedeffd640 1 --2- 172.21.15.1:0/4150607190 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x55eb232b9e40 0x55eb232bc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:43.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.840+0000 7fcedeffd640 1 --2- 172.21.15.1:0/4150607190 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x55eb232b9e40 0x55eb232bc260 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=18de46d16e7e0838 server_cookie=43fcc2557e83398d in_seq=0 out_seq=0 2024-09-11T22:05:43.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.840+0000 7fcec67fc640 10 client.17412 ms_handle_connect on v2:172.21.15.161:6834/828381592 2024-09-11T22:05:43.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.840+0000 7fcec67fc640 1 -- 172.21.15.1:0/4150607190 <== mds.4 v2:172.21.15.161:6834/828381592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55eb231a1d70 con 0x55eb232b9e40 2024-09-11T22:05:43.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.840+0000 7fcec67fc640 10 client.17412 handle_command_reply: tid=0 2024-09-11T22:05:43.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.840+0000 7fcee76a4740 1 client.17412 shutdown 2024-09-11T22:05:43.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.840+0000 7fcee76a4740 1 -- 172.21.15.1:0/4150607190 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eb232345d0 msgr2=0x55eb23247a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:43.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.840+0000 7fcee76a4740 1 --2- 172.21.15.1:0/4150607190 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eb232345d0 0x55eb23247a30 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fcec8006a90 tx=0x7fcec8009110 comp rx=0 tx=0).stop 2024-09-11T22:05:43.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.840+0000 7fcee76a4740 1 -- 172.21.15.1:0/4150607190 shutdown_connections 2024-09-11T22:05:43.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.840+0000 7fcee76a4740 1 --2- 172.21.15.1:0/4150607190 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55eb2323f370 0x55eb23239110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:43.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.840+0000 7fcee76a4740 1 --2- 172.21.15.1:0/4150607190 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x55eb232b9e40 0x55eb232bc260 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:43.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.840+0000 7fcec67fc640 0 client.17412 ms_handle_reset on v2:172.21.15.161:6834/828381592 2024-09-11T22:05:43.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.840+0000 7fcee76a4740 1 --2- 172.21.15.1:0/4150607190 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55eb23235b40 0x55eb23238bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:43.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.840+0000 7fcee76a4740 1 --2- 172.21.15.1:0/4150607190 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eb232345d0 0x55eb23247a30 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:43.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.840+0000 7fcee76a4740 1 -- 172.21.15.1:0/4150607190 >> 172.21.15.1:0/4150607190 conn(0x55eb231fd310 msgr2=0x55eb23230700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:43.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.840+0000 7fcee76a4740 1 -- 172.21.15.1:0/4150607190 shutdown_connections 2024-09-11T22:05:43.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.840+0000 7fcee76a4740 1 -- 172.21.15.1:0/4150607190 wait complete. 2024-09-11T22:05:43.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.841+0000 7fcee76a4740 20 client.17412 trim_cache size 0 max 16384 2024-09-11T22:05:43.858 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-11T22:05:43.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.841+0000 7fcee52cb640 1 -- 172.21.15.1:0/2739795571 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fcecc0a2d70 msgr2=0x7fcecc0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:43.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.841+0000 7fcee52cb640 1 --2- 172.21.15.1:0/2739795571 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fcecc0a2d70 0x7fcecc0a5230 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fced80ed490 tx=0x7fced803b600 comp rx=0 tx=0).stop 2024-09-11T22:05:43.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.841+0000 7fcee52cb640 1 -- 172.21.15.1:0/2739795571 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcee010e4f0 msgr2=0x7fcee019f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:43.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.841+0000 7fcee52cb640 1 --2- 172.21.15.1:0/2739795571 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcee010e4f0 0x7fcee019f000 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fced8005ab0 tx=0x7fced80042f0 comp rx=0 tx=0).stop 2024-09-11T22:05:43.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.841+0000 7fcee52cb640 1 -- 172.21.15.1:0/2739795571 shutdown_connections 2024-09-11T22:05:43.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.841+0000 7fcee52cb640 1 --2- 172.21.15.1:0/2739795571 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fcecc0a2d70 0x7fcecc0a5230 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:43.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.841+0000 7fcee52cb640 1 --2- 172.21.15.1:0/2739795571 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcee010e4f0 0x7fcee019f000 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:43.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.841+0000 7fcee52cb640 1 --2- 172.21.15.1:0/2739795571 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcee0101380 0x7fcee019ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:43.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.841+0000 7fcee52cb640 1 --2- 172.21.15.1:0/2739795571 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcee01009d0 0x7fcee019a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:43.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.841+0000 7fcee52cb640 1 -- 172.21.15.1:0/2739795571 >> 172.21.15.1:0/2739795571 conn(0x7fcee00fc7d0 msgr2=0x7fcee00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:43.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.841+0000 7fcee52cb640 1 -- 172.21.15.1:0/2739795571 shutdown_connections 2024-09-11T22:05:43.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:43.841+0000 7fcee52cb640 1 -- 172.21.15.1:0/2739795571 wait complete. 2024-09-11T22:05:44.512 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:44.511+0000 7f8438d99640 1 -- 172.21.15.1:0/110930524 <== mon.0 v2:172.21.15.1:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm_data_pool", "fs_name": "cephfs", "pool": "foo.201307"}]=0 v45) ==== 106+0+0 (secure 0 0 0) 0x7f84340a3b10 con 0x7f843c106a20 2024-09-11T22:05:44.514 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:44.513+0000 7f84411e8640 1 -- 172.21.15.1:0/110930524 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f84280a2d80 msgr2=0x7f84280a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:44.514 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:44.513+0000 7f84411e8640 1 --2- 172.21.15.1:0/110930524 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f84280a2d80 0x7f84280a5240 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f842c000c00 tx=0x7f842c031490 comp rx=0 tx=0).stop 2024-09-11T22:05:44.514 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:44.513+0000 7f84411e8640 1 -- 172.21.15.1:0/110930524 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f843c106a20 msgr2=0x7f843c1a4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:44.514 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:44.513+0000 7f84411e8640 1 --2- 172.21.15.1:0/110930524 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f843c106a20 0x7f843c1a4390 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f843400b970 tx=0x7f843400be30 comp rx=0 tx=0).stop 2024-09-11T22:05:44.515 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:44.513+0000 7f84411e8640 1 -- 172.21.15.1:0/110930524 shutdown_connections 2024-09-11T22:05:44.515 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:44.513+0000 7f84411e8640 1 --2- 172.21.15.1:0/110930524 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f84280a2d80 0x7f84280a5240 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:44.515 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:44.513+0000 7f84411e8640 1 --2- 172.21.15.1:0/110930524 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f843c106a20 0x7f843c1a4390 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:44.515 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:44.513+0000 7f84411e8640 1 --2- 172.21.15.1:0/110930524 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f843c1013b0 0x7f843c199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:44.515 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:44.513+0000 7f84411e8640 1 --2- 172.21.15.1:0/110930524 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f843c100a10 0x7f843c199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:44.515 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:44.514+0000 7f84411e8640 1 -- 172.21.15.1:0/110930524 >> 172.21.15.1:0/110930524 conn(0x7f843c0fc6f0 msgr2=0x7f843c0fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:44.515 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:44.514+0000 7f84411e8640 1 -- 172.21.15.1:0/110930524 shutdown_connections 2024-09-11T22:05:44.515 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:44.514+0000 7f84411e8640 1 -- 172.21.15.1:0/110930524 wait complete. 2024-09-11T22:05:44.611 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:44.610+0000 7fc61e186640 1 -- 172.21.15.1:0/2795563214 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc618106a80 msgr2=0x7fc618106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:44.611 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:44.610+0000 7fc61e186640 1 --2- 172.21.15.1:0/2795563214 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc618106a80 0x7fc618106e60 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fc608009930 tx=0x7fc60802a420 comp rx=0 tx=0).stop 2024-09-11T22:05:44.611 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:44.610+0000 7fc61e186640 1 -- 172.21.15.1:0/2795563214 shutdown_connections 2024-09-11T22:05:44.611 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:44.610+0000 7fc61e186640 1 --2- 172.21.15.1:0/2795563214 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc618101870 0x7fc61810d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:44.611 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:44.610+0000 7fc61e186640 1 --2- 172.21.15.1:0/2795563214 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc618100ed0 0x7fc618101330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:44.611 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:44.610+0000 7fc61e186640 1 --2- 172.21.15.1:0/2795563214 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc618106a80 0x7fc618106e60 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:44.611 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:44.610+0000 7fc61e186640 1 -- 172.21.15.1:0/2795563214 >> 172.21.15.1:0/2795563214 conn(0x7fc618076830 msgr2=0x7fc618076c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:44.611 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:44.610+0000 7fc61e186640 1 -- 172.21.15.1:0/2795563214 shutdown_connections 2024-09-11T22:05:44.611 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:44.610+0000 7fc61e186640 1 -- 172.21.15.1:0/2795563214 wait complete. 2024-09-11T22:05:44.611 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:44.610+0000 7fc61e186640 1 Processor -- start 2024-09-11T22:05:44.612 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:44.610+0000 7fc61e186640 1 -- start start 2024-09-11T22:05:44.614 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:44.611+0000 7fc61e186640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc618100ed0 0x7fc618198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:44.614 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:44.611+0000 7fc61e186640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc618101870 0x7fc618198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:44.614 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:44.611+0000 7fc61e186640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc618106a80 0x7fc61819cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:44.614 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:44.611+0000 7fc61e186640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc618110e80 con 0x7fc618106a80 2024-09-11T22:05:44.614 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:44.611+0000 7fc61e186640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc618110d00 con 0x7fc618100ed0 2024-09-11T22:05:44.614 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:44.611+0000 7fc61e186640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc618111000 con 0x7fc618101870 2024-09-11T22:05:44.614 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:44.611+0000 7fc617fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc618106a80 0x7fc61819cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:44.614 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:44.611+0000 7fc617fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc618106a80 0x7fc61819cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44098/0 (socket says 172.21.15.1:44098) 2024-09-11T22:05:44.614 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:44.611+0000 7fc617fff640 1 -- 172.21.15.1:0/3263835799 learned_addr learned my addr 172.21.15.1:0/3263835799 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:44.614 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:44.611+0000 7fc6177fe640 1 --2- 172.21.15.1:0/3263835799 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc618100ed0 0x7fc618198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:44.614 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:44.611+0000 7fc60effd640 1 --2- 172.21.15.1:0/3263835799 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc618101870 0x7fc618198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:44.615 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:44.611+0000 7fc617fff640 1 -- 172.21.15.1:0/3263835799 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc618101870 msgr2=0x7fc618198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:44.615 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:44.611+0000 7fc617fff640 1 --2- 172.21.15.1:0/3263835799 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc618101870 0x7fc618198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:44.615 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:44.611+0000 7fc617fff640 1 -- 172.21.15.1:0/3263835799 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc618100ed0 msgr2=0x7fc618198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:44.615 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:44.611+0000 7fc617fff640 1 --2- 172.21.15.1:0/3263835799 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc618100ed0 0x7fc618198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:44.615 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:44.611+0000 7fc617fff640 1 -- 172.21.15.1:0/3263835799 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc61819d440 con 0x7fc618106a80 2024-09-11T22:05:44.615 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:44.611+0000 7fc60effd640 1 --2- 172.21.15.1:0/3263835799 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc618101870 0x7fc618198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:05:44.615 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:44.611+0000 7fc617fff640 1 --2- 172.21.15.1:0/3263835799 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc618106a80 0x7fc61819cd10 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fc61000cce0 tx=0x7fc61000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:44.615 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:44.612+0000 7fc6157fa640 1 -- 172.21.15.1:0/3263835799 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61000f450 con 0x7fc618106a80 2024-09-11T22:05:44.615 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:44.612+0000 7fc61e186640 1 -- 172.21.15.1:0/3263835799 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc61819d730 con 0x7fc618106a80 2024-09-11T22:05:44.615 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:44.612+0000 7fc6157fa640 1 -- 172.21.15.1:0/3263835799 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc61000fad0 con 0x7fc618106a80 2024-09-11T22:05:44.615 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:44.612+0000 7fc6157fa640 1 -- 172.21.15.1:0/3263835799 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc610018c10 con 0x7fc618106a80 2024-09-11T22:05:44.615 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:44.612+0000 7fc61e186640 1 -- 172.21.15.1:0/3263835799 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6181a4fc0 con 0x7fc618106a80 2024-09-11T22:05:44.615 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:44.612+0000 7fc61e186640 1 -- 172.21.15.1:0/3263835799 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc618101330 con 0x7fc618106a80 2024-09-11T22:05:44.616 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:44.613+0000 7fc6157fa640 1 -- 172.21.15.1:0/3263835799 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7fc61000f5f0 con 0x7fc618106a80 2024-09-11T22:05:44.616 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:44.614+0000 7fc6157fa640 1 --2- 172.21.15.1:0/3263835799 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fc6040a2d00 0x7fc6040a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:44.616 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:44.614+0000 7fc6177fe640 1 --2- 172.21.15.1:0/3263835799 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fc6040a2d00 0x7fc6040a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:44.617 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:44.614+0000 7fc6157fa640 1 -- 172.21.15.1:0/3263835799 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8966+0+0 (secure 0 0 0) 0x7fc6100cfd00 con 0x7fc618106a80 2024-09-11T22:05:44.617 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:44.614+0000 7fc6177fe640 1 --2- 172.21.15.1:0/3263835799 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fc6040a2d00 0x7fc6040a51c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc608009900 tx=0x7fc6080058d0 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:44.619 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:44.618+0000 7fc6157fa640 1 -- 172.21.15.1:0/3263835799 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc610014030 con 0x7fc618106a80 2024-09-11T22:05:44.640 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:44 smithi001 ceph-mon[30935]: osdmap e94: 12 total, 12 up, 12 in 2024-09-11T22:05:44.640 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:44 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/110930524' entity='client.admin' cmd='[{"prefix": "fs rm_data_pool", "fs_name": "cephfs", "pool": "foo.201307"}]': finished 2024-09-11T22:05:44.640 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:44 smithi001 ceph-mon[30935]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:05:44.641 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:44 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/110930524' entity='client.admin' cmd=[{"prefix": "fs rm_data_pool", "fs_name": "cephfs", "pool": "foo.201307"}]: dispatch 2024-09-11T22:05:44.641 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:44 smithi001 ceph-mon[30935]: pgmap v265: 105 pgs: 8 unknown, 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 14 op/s 2024-09-11T22:05:44.668 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T22:05:44.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:44 smithi053 ceph-mon[34908]: osdmap e94: 12 total, 12 up, 12 in 2024-09-11T22:05:44.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:44 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/110930524' entity='client.admin' cmd='[{"prefix": "fs rm_data_pool", "fs_name": "cephfs", "pool": "foo.201307"}]': finished 2024-09-11T22:05:44.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:44 smithi053 ceph-mon[34908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:05:44.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:44 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/110930524' entity='client.admin' cmd=[{"prefix": "fs rm_data_pool", "fs_name": "cephfs", "pool": "foo.201307"}]: dispatch 2024-09-11T22:05:44.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:44 smithi053 ceph-mon[34908]: pgmap v265: 105 pgs: 8 unknown, 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 14 op/s 2024-09-11T22:05:44.801 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:44.799+0000 7fc61e186640 1 -- 172.21.15.1:0/3263835799 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "foo.201307", "pool2": "foo.201307", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc61802d7c0 con 0x7fc618106a80 2024-09-11T22:05:44.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:44 smithi161 ceph-mon[35615]: osdmap e94: 12 total, 12 up, 12 in 2024-09-11T22:05:44.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:44 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/110930524' entity='client.admin' cmd='[{"prefix": "fs rm_data_pool", "fs_name": "cephfs", "pool": "foo.201307"}]': finished 2024-09-11T22:05:44.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:44 smithi161 ceph-mon[35615]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:05:44.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:44 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/110930524' entity='client.admin' cmd=[{"prefix": "fs rm_data_pool", "fs_name": "cephfs", "pool": "foo.201307"}]: dispatch 2024-09-11T22:05:44.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:44 smithi161 ceph-mon[35615]: pgmap v265: 105 pgs: 8 unknown, 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 14 op/s 2024-09-11T22:05:44.929 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:05:45.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.449+0000 7f80d8890640 1 -- 172.21.15.1:0/3609140608 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f80d4106f30 msgr2=0x7f80d410eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:45.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.449+0000 7f80d8890640 1 --2- 172.21.15.1:0/3609140608 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f80d4106f30 0x7f80d410eab0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f80cc009ca0 tx=0x7f80cc02a8c0 comp rx=0 tx=0).stop 2024-09-11T22:05:45.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.449+0000 7f80d8890640 1 -- 172.21.15.1:0/3609140608 shutdown_connections 2024-09-11T22:05:45.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.449+0000 7f80d8890640 1 --2- 172.21.15.1:0/3609140608 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f80d4106f30 0x7f80d410eab0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:45.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.449+0000 7f80d8890640 1 --2- 172.21.15.1:0/3609140608 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80d4100cc0 0x7f80d4106800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:45.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.449+0000 7f80d8890640 1 --2- 172.21.15.1:0/3609140608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80d4100310 0x7f80d41006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:45.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.449+0000 7f80d8890640 1 -- 172.21.15.1:0/3609140608 >> 172.21.15.1:0/3609140608 conn(0x7f80d4077210 msgr2=0x7f80d40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:45.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.449+0000 7f80d8890640 1 -- 172.21.15.1:0/3609140608 shutdown_connections 2024-09-11T22:05:45.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.449+0000 7f80d8890640 1 -- 172.21.15.1:0/3609140608 wait complete. 2024-09-11T22:05:45.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.450+0000 7f80d8890640 1 Processor -- start 2024-09-11T22:05:45.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.450+0000 7f80d8890640 1 -- start start 2024-09-11T22:05:45.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.450+0000 7f80d8890640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80d4100310 0x7f80d4198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:45.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.450+0000 7f80d8890640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80d4100cc0 0x7f80d41989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:45.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.450+0000 7f80d8890640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f80d4106f30 0x7f80d419cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:45.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.450+0000 7f80d8890640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f80d4111fd0 con 0x7f80d4106f30 2024-09-11T22:05:45.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.450+0000 7f80d8890640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f80d4111e50 con 0x7f80d4100cc0 2024-09-11T22:05:45.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.450+0000 7f80d8890640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f80d4112150 con 0x7f80d4100310 2024-09-11T22:05:45.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.450+0000 7f80d2d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f80d4106f30 0x7f80d419cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:45.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.450+0000 7f80d2d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f80d4106f30 0x7f80d419cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44120/0 (socket says 172.21.15.1:44120) 2024-09-11T22:05:45.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.450+0000 7f80d2d9d640 1 -- 172.21.15.1:0/3786077926 learned_addr learned my addr 172.21.15.1:0/3786077926 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:45.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.450+0000 7f80d259c640 1 --2- 172.21.15.1:0/3786077926 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80d4100310 0x7f80d4198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:45.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.450+0000 7f80d1d9b640 1 --2- 172.21.15.1:0/3786077926 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80d4100cc0 0x7f80d41989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:45.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.451+0000 7f80d2d9d640 1 -- 172.21.15.1:0/3786077926 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80d4100310 msgr2=0x7f80d4198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:45.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.451+0000 7f80d2d9d640 1 --2- 172.21.15.1:0/3786077926 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80d4100310 0x7f80d4198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:45.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.451+0000 7f80d2d9d640 1 -- 172.21.15.1:0/3786077926 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80d4100cc0 msgr2=0x7f80d41989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:45.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.451+0000 7f80d2d9d640 1 --2- 172.21.15.1:0/3786077926 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80d4100cc0 0x7f80d41989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:45.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.451+0000 7f80d2d9d640 1 -- 172.21.15.1:0/3786077926 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80d419d440 con 0x7f80d4106f30 2024-09-11T22:05:45.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.451+0000 7f80d2d9d640 1 --2- 172.21.15.1:0/3786077926 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f80d4106f30 0x7f80d419cd60 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f80cc031a20 tx=0x7f80cc031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:45.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.451+0000 7f80c37fe640 1 -- 172.21.15.1:0/3786077926 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80cc003b70 con 0x7f80d4106f30 2024-09-11T22:05:45.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.451+0000 7f80c37fe640 1 -- 172.21.15.1:0/3786077926 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80cc003d10 con 0x7f80d4106f30 2024-09-11T22:05:45.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.451+0000 7f80d8890640 1 -- 172.21.15.1:0/3786077926 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80d419d6d0 con 0x7f80d4106f30 2024-09-11T22:05:45.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.451+0000 7f80c37fe640 1 -- 172.21.15.1:0/3786077926 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80cc02fc80 con 0x7f80d4106f30 2024-09-11T22:05:45.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.451+0000 7f80d8890640 1 -- 172.21.15.1:0/3786077926 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80d41a5010 con 0x7f80d4106f30 2024-09-11T22:05:45.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.454+0000 7f80c37fe640 1 -- 172.21.15.1:0/3786077926 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f80cc0046a0 con 0x7f80d4106f30 2024-09-11T22:05:45.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.454+0000 7f80dac69740 1 Processor -- start 2024-09-11T22:05:45.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.454+0000 7f80dac69740 1 -- start start 2024-09-11T22:05:45.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.454+0000 7f80dac69740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564db9e2bb40 0x564db9e2a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:45.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.454+0000 7f80dac69740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564db9e2a5d0 0x564db9e2aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:45.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.454+0000 7f80dac69740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564db9e35370 0x564db9e37760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:45.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.454+0000 7f80dac69740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x564db9d902f0 con 0x564db9e2a5d0 2024-09-11T22:05:45.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.454+0000 7f80dac69740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x564db9d37cf0 con 0x564db9e35370 2024-09-11T22:05:45.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.454+0000 7f80dac69740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x564db9d8c2b0 con 0x564db9e2bb40 2024-09-11T22:05:45.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.454+0000 7f80d259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564db9e2a5d0 0x564db9e2aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:45.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.454+0000 7f80d259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564db9e2a5d0 0x564db9e2aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44136/0 (socket says 172.21.15.1:44136) 2024-09-11T22:05:45.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.454+0000 7f80d259c640 1 -- 172.21.15.1:0/612623446 learned_addr learned my addr 172.21.15.1:0/612623446 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:45.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.454+0000 7f80d1d9b640 1 --2- 172.21.15.1:0/612623446 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564db9e2bb40 0x564db9e2a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:45.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.455+0000 7f80d1d9b640 1 --2- 172.21.15.1:0/612623446 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564db9e35370 0x564db9e37760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:45.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.455+0000 7f80c37fe640 1 --2- 172.21.15.1:0/3786077926 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f80bc0a2cf0 0x7f80bc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:45.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.455+0000 7f80d259c640 1 -- 172.21.15.1:0/612623446 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564db9e2bb40 msgr2=0x564db9e2a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:45.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.455+0000 7f80d259c640 1 --2- 172.21.15.1:0/612623446 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564db9e2bb40 0x564db9e2a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:45.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.455+0000 7f80d259c640 1 -- 172.21.15.1:0/612623446 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564db9e35370 msgr2=0x564db9e37760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:45.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.455+0000 7f80d259c640 1 --2- 172.21.15.1:0/612623446 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564db9e35370 0x564db9e37760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:45.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.455+0000 7f80d259c640 1 -- 172.21.15.1:0/612623446 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564db9d1c3a0 con 0x564db9e2a5d0 2024-09-11T22:05:45.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.455+0000 7f80d1d9b640 1 --2- 172.21.15.1:0/612623446 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564db9e35370 0x564db9e37760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:05:45.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.455+0000 7f80d2d9d640 1 --2- 172.21.15.1:0/3786077926 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f80bc0a2cf0 0x7f80bc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:45.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.455+0000 7f80d259c640 1 --2- 172.21.15.1:0/612623446 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564db9e2a5d0 0x564db9e2aa10 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f80c4009a70 tx=0x7f80c4009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9be7f536438a30b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:45.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.455+0000 7f80c37fe640 1 -- 172.21.15.1:0/3786077926 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8966+0+0 (secure 0 0 0) 0x7f80cc0f0770 con 0x7f80d4106f30 2024-09-11T22:05:45.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.455+0000 7f80c0ff9640 1 -- 172.21.15.1:0/612623446 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c4017070 con 0x564db9e2a5d0 2024-09-11T22:05:45.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.455+0000 7f80c0ff9640 1 -- 172.21.15.1:0/612623446 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80c4013470 con 0x564db9e2a5d0 2024-09-11T22:05:45.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.455+0000 7f80c0ff9640 1 -- 172.21.15.1:0/612623446 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c40125f0 con 0x564db9e2a5d0 2024-09-11T22:05:45.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.456+0000 7f80dac69740 1 -- 172.21.15.1:0/612623446 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564db9e2a5d0 msgr2=0x564db9e2aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:45.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.456+0000 7f80d2d9d640 1 --2- 172.21.15.1:0/3786077926 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f80bc0a2cf0 0x7f80bc0a51b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f80cc0bc970 tx=0x7f80cc08fce0 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:45.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.456+0000 7f80dac69740 1 --2- 172.21.15.1:0/612623446 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564db9e2a5d0 0x564db9e2aa10 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f80c4009a70 tx=0x7f80c4009f30 comp rx=0 tx=0).stop 2024-09-11T22:05:45.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.456+0000 7f80dac69740 1 -- 172.21.15.1:0/612623446 shutdown_connections 2024-09-11T22:05:45.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.456+0000 7f80dac69740 1 --2- 172.21.15.1:0/612623446 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564db9e35370 0x564db9e37760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:45.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.456+0000 7f80dac69740 1 --2- 172.21.15.1:0/612623446 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564db9e2a5d0 0x564db9e2aa10 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:45.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.456+0000 7f80dac69740 1 --2- 172.21.15.1:0/612623446 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564db9e2bb40 0x564db9e2a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:45.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.456+0000 7f80dac69740 1 -- 172.21.15.1:0/612623446 >> 172.21.15.1:0/612623446 conn(0x564db9df3310 msgr2=0x564db9e26c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:45.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.456+0000 7f80dac69740 1 -- 172.21.15.1:0/612623446 shutdown_connections 2024-09-11T22:05:45.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.456+0000 7f80dac69740 1 -- 172.21.15.1:0/612623446 wait complete. 2024-09-11T22:05:45.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.456+0000 7f80dac69740 1 -- start start 2024-09-11T22:05:45.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.456+0000 7f80dac69740 1 Processor -- start 2024-09-11T22:05:45.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.457+0000 7f80dac69740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564db9e2a5d0 0x564db9e34d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:45.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.457+0000 7f80dac69740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564db9e2bb40 0x564db9e31fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:45.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.457+0000 7f80dac69740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564db9e35370 0x564db9e32570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:45.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.457+0000 7f80dac69740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x564db9e38070 con 0x564db9e2bb40 2024-09-11T22:05:45.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.457+0000 7f80dac69740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x564db9d4b060 con 0x564db9e35370 2024-09-11T22:05:45.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.457+0000 7f80dac69740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x564db9e381f0 con 0x564db9e2a5d0 2024-09-11T22:05:45.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.457+0000 7f80dac69740 10 client.0 _ll_register_callbacks cb 0x564db9a9e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:05:45.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.457+0000 7f80d259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564db9e2bb40 0x564db9e31fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:45.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.457+0000 7f80d259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564db9e2bb40 0x564db9e31fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44142/0 (socket says 172.21.15.1:44142) 2024-09-11T22:05:45.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.457+0000 7f80d259c640 1 -- 172.21.15.1:0/3375137477 learned_addr learned my addr 172.21.15.1:0/3375137477 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:45.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.457+0000 7f80d1d9b640 1 --2- 172.21.15.1:0/3375137477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564db9e35370 0x564db9e32570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:45.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.457+0000 7f80d1d9b640 1 --2- 172.21.15.1:0/3375137477 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564db9e2a5d0 0x564db9e34d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:45.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.457+0000 7f80d259c640 1 -- 172.21.15.1:0/3375137477 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564db9e2a5d0 msgr2=0x564db9e34d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:45.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.457+0000 7f80d259c640 1 --2- 172.21.15.1:0/3375137477 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564db9e2a5d0 0x564db9e34d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:45.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.457+0000 7f80d259c640 1 -- 172.21.15.1:0/3375137477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564db9e35370 msgr2=0x564db9e32570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:45.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.457+0000 7f80d259c640 1 --2- 172.21.15.1:0/3375137477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564db9e35370 0x564db9e32570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:45.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.457+0000 7f80d259c640 1 -- 172.21.15.1:0/3375137477 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564db9b013c0 con 0x564db9e2bb40 2024-09-11T22:05:45.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.457+0000 7f80d259c640 1 --2- 172.21.15.1:0/3375137477 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564db9e2bb40 0x564db9e31fd0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f80c4013f40 tx=0x7f80c40138d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd8a1048904131c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:45.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.457+0000 7f80b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:05:45.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.458+0000 7f80b9ffb640 1 -- 172.21.15.1:0/3375137477 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c4017050 con 0x564db9e2bb40 2024-09-11T22:05:45.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.458+0000 7f80b9ffb640 1 -- 172.21.15.1:0/3375137477 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8966+0+0 (secure 0 0 0) 0x7f80c401c450 con 0x564db9e2bb40 2024-09-11T22:05:45.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.458+0000 7f80dac69740 1 -- 172.21.15.1:0/3375137477 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80c4013470 con 0x564db9e2bb40 2024-09-11T22:05:45.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.461+0000 7f80b9ffb640 1 -- 172.21.15.1:0/3375137477 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f80c401c650 con 0x564db9e2bb40 2024-09-11T22:05:45.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.461+0000 7f80dac69740 10 client.17430 fetch_fsmap learned FSMap version 45 2024-09-11T22:05:45.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.461+0000 7f80dac69740 1 -- 172.21.15.1:0/3375137477 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x564db9e43a30 con 0x564db9e2bb40 2024-09-11T22:05:45.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.461+0000 7f80b9ffb640 1 -- 172.21.15.1:0/3375137477 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80c4013470 con 0x564db9e2bb40 2024-09-11T22:05:45.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.461+0000 7f80b9ffb640 1 -- 172.21.15.1:0/3375137477 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 45) ==== 8442+0+0 (secure 0 0 0) 0x7f80c4026840 con 0x564db9e2bb40 2024-09-11T22:05:45.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.461+0000 7f80dac69740 10 client.17430 fetch_fsmap finished waiting for FSMap version 45 2024-09-11T22:05:45.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.461+0000 7f80dac69740 10 client.17430 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-11T22:05:45.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.461+0000 7f80dac69740 20 client.17430 populate_metadata read hostname 'smithi001' 2024-09-11T22:05:45.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.461+0000 7f80dac69740 1 --2- 172.21.15.1:0/3375137477 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x564db9e44d80 0x564db9e47170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:45.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.461+0000 7f80dac69740 4 client.17430 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:05:45.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.461+0000 7f80dac69740 1 -- 172.21.15.1:0/3375137477 --> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564db9d9d860 con 0x564db9e44d80 2024-09-11T22:05:45.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.462+0000 7f80d1d9b640 1 --2- 172.21.15.1:0/3375137477 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x564db9e44d80 0x564db9e47170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:45.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.462+0000 7f80d1d9b640 1 --2- 172.21.15.1:0/3375137477 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x564db9e44d80 0x564db9e47170 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5c7148ef31130927 server_cookie=e01d4413c24f6028 in_seq=0 out_seq=0 2024-09-11T22:05:45.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.462+0000 7f80b9ffb640 10 client.17430 ms_handle_connect on v2:172.21.15.161:6836/1799318656 2024-09-11T22:05:45.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.463+0000 7f80b9ffb640 1 -- 172.21.15.1:0/3375137477 <== mds.3 v2:172.21.15.161:6836/1799318656 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564db9d9d860 con 0x564db9e44d80 2024-09-11T22:05:45.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.463+0000 7f80b9ffb640 10 client.17430 handle_command_reply: tid=0 2024-09-11T22:05:45.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.463+0000 7f80dac69740 1 client.17430 shutdown 2024-09-11T22:05:45.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.463+0000 7f80dac69740 1 -- 172.21.15.1:0/3375137477 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564db9e2bb40 msgr2=0x564db9e31fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:45.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.463+0000 7f80dac69740 1 --2- 172.21.15.1:0/3375137477 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564db9e2bb40 0x564db9e31fd0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f80c4013f40 tx=0x7f80c40138d0 comp rx=0 tx=0).stop 2024-09-11T22:05:45.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.463+0000 7f80dac69740 1 -- 172.21.15.1:0/3375137477 shutdown_connections 2024-09-11T22:05:45.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.463+0000 7f80dac69740 1 --2- 172.21.15.1:0/3375137477 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x564db9e44d80 0x564db9e47170 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:45.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.463+0000 7f80dac69740 1 --2- 172.21.15.1:0/3375137477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564db9e35370 0x564db9e32570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:45.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.463+0000 7f80dac69740 1 --2- 172.21.15.1:0/3375137477 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564db9e2bb40 0x564db9e31fd0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:45.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.463+0000 7f80dac69740 1 --2- 172.21.15.1:0/3375137477 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564db9e2a5d0 0x564db9e34d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:45.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.463+0000 7f80dac69740 1 -- 172.21.15.1:0/3375137477 >> 172.21.15.1:0/3375137477 conn(0x564db9df3310 msgr2=0x564db9e2c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:45.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.463+0000 7f80b9ffb640 0 client.17430 ms_handle_reset on v2:172.21.15.161:6836/1799318656 2024-09-11T22:05:45.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.463+0000 7f80dac69740 1 -- 172.21.15.1:0/3375137477 shutdown_connections 2024-09-11T22:05:45.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.463+0000 7f80dac69740 1 -- 172.21.15.1:0/3375137477 wait complete. 2024-09-11T22:05:45.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.463+0000 7f80dac69740 20 client.17430 trim_cache size 0 max 16384 2024-09-11T22:05:45.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.476+0000 7f80dac69740 1 Processor -- start 2024-09-11T22:05:45.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.476+0000 7f80dac69740 1 -- start start 2024-09-11T22:05:45.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.477+0000 7f80dac69740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564db9e2a5d0 0x564db9e41310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:45.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.477+0000 7f80dac69740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564db9e2bb40 0x564db9e31890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:45.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.477+0000 7f80dac69740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564db9e35370 0x564db9e36760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:45.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.477+0000 7f80dac69740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f80d41abf10 con 0x564db9e2bb40 2024-09-11T22:05:45.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.477+0000 7f80dac69740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f80d41add50 con 0x564db9e35370 2024-09-11T22:05:45.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.477+0000 7f80dac69740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f80d41ac6a0 con 0x564db9e2a5d0 2024-09-11T22:05:45.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.477+0000 7f80d259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564db9e2bb40 0x564db9e31890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:45.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.477+0000 7f80d259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564db9e2bb40 0x564db9e31890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44144/0 (socket says 172.21.15.1:44144) 2024-09-11T22:05:45.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.477+0000 7f80d259c640 1 -- 172.21.15.1:0/917054365 learned_addr learned my addr 172.21.15.1:0/917054365 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:45.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.477+0000 7f80d1d9b640 1 --2- 172.21.15.1:0/917054365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564db9e2a5d0 0x564db9e41310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:45.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.477+0000 7f80d1d9b640 1 --2- 172.21.15.1:0/917054365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564db9e35370 0x564db9e36760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:45.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.477+0000 7f80d259c640 1 -- 172.21.15.1:0/917054365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564db9e2a5d0 msgr2=0x564db9e41310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:45.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.477+0000 7f80d259c640 1 --2- 172.21.15.1:0/917054365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564db9e2a5d0 0x564db9e41310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:45.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.477+0000 7f80d259c640 1 -- 172.21.15.1:0/917054365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564db9e35370 msgr2=0x564db9e36760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:45.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.477+0000 7f80d259c640 1 --2- 172.21.15.1:0/917054365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564db9e35370 0x564db9e36760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:45.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.477+0000 7f80d259c640 1 -- 172.21.15.1:0/917054365 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564db9e2d120 con 0x564db9e2bb40 2024-09-11T22:05:45.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.477+0000 7f80d1d9b640 1 --2- 172.21.15.1:0/917054365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564db9e35370 0x564db9e36760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:05:45.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.477+0000 7f80d259c640 1 --2- 172.21.15.1:0/917054365 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564db9e2bb40 0x564db9e31890 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f80c4004810 tx=0x7f80c4009280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f36af25f96f4b9ed server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:45.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.477+0000 7f80bbfff640 1 -- 172.21.15.1:0/917054365 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c4017040 con 0x564db9e2bb40 2024-09-11T22:05:45.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.478+0000 7f80bbfff640 1 -- 172.21.15.1:0/917054365 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80c40098f0 con 0x564db9e2bb40 2024-09-11T22:05:45.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.478+0000 7f80bbfff640 1 -- 172.21.15.1:0/917054365 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c401ca50 con 0x564db9e2bb40 2024-09-11T22:05:45.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.478+0000 7f80dac69740 1 -- 172.21.15.1:0/917054365 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564db9e2bb40 msgr2=0x564db9e31890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:45.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.478+0000 7f80dac69740 1 --2- 172.21.15.1:0/917054365 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564db9e2bb40 0x564db9e31890 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f80c4004810 tx=0x7f80c4009280 comp rx=0 tx=0).stop 2024-09-11T22:05:45.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.478+0000 7f80dac69740 1 -- 172.21.15.1:0/917054365 shutdown_connections 2024-09-11T22:05:45.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.478+0000 7f80dac69740 1 --2- 172.21.15.1:0/917054365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564db9e35370 0x564db9e36760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:45.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.478+0000 7f80dac69740 1 --2- 172.21.15.1:0/917054365 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564db9e2bb40 0x564db9e31890 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:45.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.478+0000 7f80dac69740 1 --2- 172.21.15.1:0/917054365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564db9e2a5d0 0x564db9e41310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:45.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.478+0000 7f80dac69740 1 -- 172.21.15.1:0/917054365 >> 172.21.15.1:0/917054365 conn(0x564db9df3310 msgr2=0x564db9e46d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:45.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.478+0000 7f80dac69740 1 -- 172.21.15.1:0/917054365 shutdown_connections 2024-09-11T22:05:45.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.478+0000 7f80dac69740 1 -- 172.21.15.1:0/917054365 wait complete. 2024-09-11T22:05:45.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.479+0000 7f80dac69740 1 -- start start 2024-09-11T22:05:45.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.479+0000 7f80dac69740 1 Processor -- start 2024-09-11T22:05:45.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.479+0000 7f80dac69740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564db9e2a5d0 0x564db9e3d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:45.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.479+0000 7f80dac69740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564db9e2bb40 0x564db9e2e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:45.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.479+0000 7f80dac69740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564db9e35370 0x564db9e2ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:45.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.479+0000 7f80dac69740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f80d41ad5c0 con 0x564db9e2a5d0 2024-09-11T22:05:45.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.479+0000 7f80dac69740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f80d41ace30 con 0x564db9e35370 2024-09-11T22:05:45.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.479+0000 7f80dac69740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f80d41a9ef0 con 0x564db9e2bb40 2024-09-11T22:05:45.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.479+0000 7f80dac69740 10 client.0 _ll_register_callbacks cb 0x7f80c800c640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:05:45.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.479+0000 7f80d1d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564db9e2a5d0 0x564db9e3d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:45.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.479+0000 7f80d1d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564db9e2a5d0 0x564db9e3d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44156/0 (socket says 172.21.15.1:44156) 2024-09-11T22:05:45.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.479+0000 7f80d1d9b640 1 -- 172.21.15.1:0/912628930 learned_addr learned my addr 172.21.15.1:0/912628930 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:45.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.479+0000 7f80d259c640 1 --2- 172.21.15.1:0/912628930 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564db9e2bb40 0x564db9e2e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:45.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.479+0000 7f80d1d9b640 1 --2- 172.21.15.1:0/912628930 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564db9e35370 0x564db9e2ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:45.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.480+0000 7f80d1d9b640 1 -- 172.21.15.1:0/912628930 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564db9e2bb40 msgr2=0x564db9e2e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:45.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.480+0000 7f80d1d9b640 1 --2- 172.21.15.1:0/912628930 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564db9e2bb40 0x564db9e2e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:45.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.480+0000 7f80d1d9b640 1 -- 172.21.15.1:0/912628930 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564db9e35370 msgr2=0x564db9e2ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:45.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.480+0000 7f80d1d9b640 1 --2- 172.21.15.1:0/912628930 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564db9e35370 0x564db9e2ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:45.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.480+0000 7f80d1d9b640 1 -- 172.21.15.1:0/912628930 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564db9e2f440 con 0x564db9e2a5d0 2024-09-11T22:05:45.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.480+0000 7f80d1d9b640 1 --2- 172.21.15.1:0/912628930 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564db9e2a5d0 0x564db9e3d8d0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f80d4104780 tx=0x7f80c4013e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c84762f6ec4a5e53 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:45.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.480+0000 7f80b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:05:45.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.480+0000 7f80b9ffb640 1 -- 172.21.15.1:0/912628930 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c802c030 con 0x564db9e2a5d0 2024-09-11T22:05:45.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.480+0000 7f80b9ffb640 1 -- 172.21.15.1:0/912628930 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8966+0+0 (secure 0 0 0) 0x7f80c8022040 con 0x564db9e2a5d0 2024-09-11T22:05:45.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.480+0000 7f80dac69740 1 -- 172.21.15.1:0/912628930 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80c40098f0 con 0x564db9e2a5d0 2024-09-11T22:05:45.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.480+0000 7f80b9ffb640 1 -- 172.21.15.1:0/912628930 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f80c803d050 con 0x564db9e2a5d0 2024-09-11T22:05:45.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.480+0000 7f80dac69740 10 client.17436 fetch_fsmap learned FSMap version 45 2024-09-11T22:05:45.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.480+0000 7f80dac69740 1 -- 172.21.15.1:0/912628930 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x564db9e3c110 con 0x564db9e2a5d0 2024-09-11T22:05:45.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.480+0000 7f80b9ffb640 1 -- 172.21.15.1:0/912628930 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80c40098f0 con 0x564db9e2a5d0 2024-09-11T22:05:45.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.481+0000 7f80b9ffb640 1 -- 172.21.15.1:0/912628930 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 45) ==== 8442+0+0 (secure 0 0 0) 0x7f80c8011560 con 0x564db9e2a5d0 2024-09-11T22:05:45.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.481+0000 7f80dac69740 10 client.17436 fetch_fsmap finished waiting for FSMap version 45 2024-09-11T22:05:45.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.481+0000 7f80dac69740 10 client.17436 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-11T22:05:45.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.481+0000 7f80dac69740 20 client.17436 populate_metadata read hostname 'smithi001' 2024-09-11T22:05:45.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.481+0000 7f80dac69740 1 --2- 172.21.15.1:0/912628930 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x564db9eafd60 0x564db9eb2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:45.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.481+0000 7f80dac69740 4 client.17436 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:05:45.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.481+0000 7f80dac69740 1 -- 172.21.15.1:0/912628930 --> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564db9d97d70 con 0x564db9eafd60 2024-09-11T22:05:45.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.481+0000 7f80d259c640 1 --2- 172.21.15.1:0/912628930 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x564db9eafd60 0x564db9eb2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:45.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.481+0000 7f80b9ffb640 10 client.17436 ms_handle_connect on v2:172.21.15.161:6836/1799318656 2024-09-11T22:05:45.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.481+0000 7f80d259c640 1 --2- 172.21.15.1:0/912628930 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x564db9eafd60 0x564db9eb2180 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2da1cefefdb304d1 server_cookie=9eba0c5c961d983e in_seq=0 out_seq=0 2024-09-11T22:05:45.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.481+0000 7f80b9ffb640 1 -- 172.21.15.1:0/912628930 <== mds.3 v2:172.21.15.161:6836/1799318656 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564db9d97d70 con 0x564db9eafd60 2024-09-11T22:05:45.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.481+0000 7f80b9ffb640 10 client.17436 handle_command_reply: tid=0 2024-09-11T22:05:45.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.481+0000 7f80dac69740 1 client.17436 shutdown 2024-09-11T22:05:45.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.482+0000 7f80dac69740 1 -- 172.21.15.1:0/912628930 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564db9e2a5d0 msgr2=0x564db9e3d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:45.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.482+0000 7f80dac69740 1 --2- 172.21.15.1:0/912628930 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564db9e2a5d0 0x564db9e3d8d0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f80d4104780 tx=0x7f80c4013e40 comp rx=0 tx=0).stop 2024-09-11T22:05:45.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.482+0000 7f80dac69740 1 -- 172.21.15.1:0/912628930 shutdown_connections 2024-09-11T22:05:45.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.482+0000 7f80dac69740 1 --2- 172.21.15.1:0/912628930 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x564db9eafd60 0x564db9eb2180 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:45.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.482+0000 7f80dac69740 1 --2- 172.21.15.1:0/912628930 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564db9e35370 0x564db9e2ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:45.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.482+0000 7f80dac69740 1 --2- 172.21.15.1:0/912628930 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564db9e2bb40 0x564db9e2e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:45.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.482+0000 7f80dac69740 1 --2- 172.21.15.1:0/912628930 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564db9e2a5d0 0x564db9e3d8d0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:45.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.482+0000 7f80dac69740 1 -- 172.21.15.1:0/912628930 >> 172.21.15.1:0/912628930 conn(0x564db9df3310 msgr2=0x564db9e2c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:45.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.482+0000 7f80b9ffb640 0 client.17436 ms_handle_reset on v2:172.21.15.161:6836/1799318656 2024-09-11T22:05:45.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.482+0000 7f80dac69740 1 -- 172.21.15.1:0/912628930 shutdown_connections 2024-09-11T22:05:45.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.482+0000 7f80dac69740 1 -- 172.21.15.1:0/912628930 wait complete. 2024-09-11T22:05:45.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.482+0000 7f80dac69740 20 client.17436 trim_cache size 0 max 16384 2024-09-11T22:05:45.492 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-11T22:05:45.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.482+0000 7f80d8890640 1 -- 172.21.15.1:0/3786077926 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f80bc0a2cf0 msgr2=0x7f80bc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:45.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.482+0000 7f80d8890640 1 --2- 172.21.15.1:0/3786077926 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f80bc0a2cf0 0x7f80bc0a51b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f80cc0bc970 tx=0x7f80cc08fce0 comp rx=0 tx=0).stop 2024-09-11T22:05:45.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.482+0000 7f80d8890640 1 -- 172.21.15.1:0/3786077926 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f80d4106f30 msgr2=0x7f80d419cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:45.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.482+0000 7f80d8890640 1 --2- 172.21.15.1:0/3786077926 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f80d4106f30 0x7f80d419cd60 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f80cc031a20 tx=0x7f80cc031a50 comp rx=0 tx=0).stop 2024-09-11T22:05:45.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.482+0000 7f80d8890640 1 -- 172.21.15.1:0/3786077926 shutdown_connections 2024-09-11T22:05:45.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.482+0000 7f80d8890640 1 --2- 172.21.15.1:0/3786077926 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f80bc0a2cf0 0x7f80bc0a51b0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:45.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.483+0000 7f80d8890640 1 --2- 172.21.15.1:0/3786077926 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f80d4106f30 0x7f80d419cd60 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:45.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.483+0000 7f80d8890640 1 --2- 172.21.15.1:0/3786077926 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80d4100cc0 0x7f80d41989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:45.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.483+0000 7f80d8890640 1 --2- 172.21.15.1:0/3786077926 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80d4100310 0x7f80d4198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:45.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.483+0000 7f80d8890640 1 -- 172.21.15.1:0/3786077926 >> 172.21.15.1:0/3786077926 conn(0x7f80d4077210 msgr2=0x7f80d41057a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:45.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.483+0000 7f80d8890640 1 -- 172.21.15.1:0/3786077926 shutdown_connections 2024-09-11T22:05:45.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:45.483+0000 7f80d8890640 1 -- 172.21.15.1:0/3786077926 wait complete. 2024-09-11T22:05:45.513 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:45.511+0000 7fc6157fa640 1 -- 172.21.15.1:0/3263835799 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "foo.201307", "pool2": "foo.201307", "yes_i_really_really_mean_it": true}]=0 pool 'foo.201307' removed v95) ==== 166+0+0 (secure 0 0 0) 0x7fc61009c760 con 0x7fc618106a80 2024-09-11T22:05:45.619 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:45.618+0000 7fc61e186640 1 -- 172.21.15.1:0/3263835799 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "foo.201307", "pool2": "foo.201307", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc618109f30 con 0x7fc618106a80 2024-09-11T22:05:45.620 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:45.618+0000 7fc6157fa640 1 -- 172.21.15.1:0/3263835799 <== mon.0 v2:172.21.15.1:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "foo.201307", "pool2": "foo.201307", "yes_i_really_really_mean_it": true}]=0 pool 'foo.201307' does not exist v95) ==== 173+0+0 (secure 0 0 0) 0x7fc61009c060 con 0x7fc618106a80 2024-09-11T22:05:45.620 INFO:tasks.workunit.client.0.smithi001.stderr:pool 'foo.201307' does not exist 2024-09-11T22:05:45.622 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:45.620+0000 7fc61e186640 1 -- 172.21.15.1:0/3263835799 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fc6040a2d00 msgr2=0x7fc6040a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:45.622 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:45.620+0000 7fc61e186640 1 --2- 172.21.15.1:0/3263835799 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fc6040a2d00 0x7fc6040a51c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc608009900 tx=0x7fc6080058d0 comp rx=0 tx=0).stop 2024-09-11T22:05:45.622 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:45.620+0000 7fc61e186640 1 -- 172.21.15.1:0/3263835799 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc618106a80 msgr2=0x7fc61819cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:45.622 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:45.620+0000 7fc61e186640 1 --2- 172.21.15.1:0/3263835799 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc618106a80 0x7fc61819cd10 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fc61000cce0 tx=0x7fc61000a620 comp rx=0 tx=0).stop 2024-09-11T22:05:45.622 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:45.620+0000 7fc61e186640 1 -- 172.21.15.1:0/3263835799 shutdown_connections 2024-09-11T22:05:45.623 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:45.620+0000 7fc61e186640 1 --2- 172.21.15.1:0/3263835799 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fc6040a2d00 0x7fc6040a51c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:45.623 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:45.620+0000 7fc61e186640 1 --2- 172.21.15.1:0/3263835799 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc618106a80 0x7fc61819cd10 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:45.623 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:45.620+0000 7fc61e186640 1 --2- 172.21.15.1:0/3263835799 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc618101870 0x7fc618198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:45.623 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:45.620+0000 7fc61e186640 1 --2- 172.21.15.1:0/3263835799 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc618100ed0 0x7fc618198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:45.623 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:45.620+0000 7fc61e186640 1 -- 172.21.15.1:0/3263835799 >> 172.21.15.1:0/3263835799 conn(0x7fc618076830 msgr2=0x7fc618069960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:45.623 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:45.620+0000 7fc61e186640 1 -- 172.21.15.1:0/3263835799 shutdown_connections 2024-09-11T22:05:45.623 INFO:tasks.workunit.client.0.smithi001.stderr:2024-09-11T22:05:45.620+0000 7fc61e186640 1 -- 172.21.15.1:0/3263835799 wait complete. 2024-09-11T22:05:45.633 INFO:tasks.workunit.client.0.smithi001.stdout:OK 2024-09-11T22:05:45.633 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-11T22:05:45.633 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-11T22:05:45.656 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:45 smithi001 ceph-mon[30935]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T22:05:45.656 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:45 smithi001 ceph-mon[30935]: Cluster is now healthy 2024-09-11T22:05:45.656 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:45 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/110930524' entity='client.admin' cmd='[{"prefix": "fs rm_data_pool", "fs_name": "cephfs", "pool": "foo.201307"}]': finished 2024-09-11T22:05:45.656 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:45 smithi001 ceph-mon[30935]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:05:45.656 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:45 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/3263835799' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "foo.201307", "pool2": "foo.201307", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T22:05:45.679 INFO:tasks.workunit:Running workunit fs/misc/multiple_rsync.sh... 2024-09-11T22:05:45.679 DEBUG:teuthology.orchestra.run.smithi001:workunit test fs/misc/multiple_rsync.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=086c737fa75c22d521a3932c59b31c5e1d1c9ef4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/multiple_rsync.sh 2024-09-11T22:05:45.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:45 smithi053 ceph-mon[34908]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T22:05:45.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:45 smithi053 ceph-mon[34908]: Cluster is now healthy 2024-09-11T22:05:45.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:45 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/110930524' entity='client.admin' cmd='[{"prefix": "fs rm_data_pool", "fs_name": "cephfs", "pool": "foo.201307"}]': finished 2024-09-11T22:05:45.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:45 smithi053 ceph-mon[34908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:05:45.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:45 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/3263835799' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "foo.201307", "pool2": "foo.201307", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T22:05:45.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:45 smithi161 ceph-mon[35615]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T22:05:45.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:45 smithi161 ceph-mon[35615]: Cluster is now healthy 2024-09-11T22:05:45.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:45 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/110930524' entity='client.admin' cmd='[{"prefix": "fs rm_data_pool", "fs_name": "cephfs", "pool": "foo.201307"}]': finished 2024-09-11T22:05:45.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:45 smithi161 ceph-mon[35615]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:05:45.843 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:45 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/3263835799' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "foo.201307", "pool2": "foo.201307", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T22:05:46.179 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T22:05:46.440 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:05:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.510+0000 7f9198ae4640 1 -- 172.21.15.1:0/46625920 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9194103660 msgr2=0x7f919410bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.510+0000 7f9198ae4640 1 --2- 172.21.15.1:0/46625920 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9194103660 0x7f919410bb40 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f9188009900 tx=0x7f918802a480 comp rx=0 tx=0).stop 2024-09-11T22:05:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.510+0000 7f9198ae4640 1 -- 172.21.15.1:0/46625920 shutdown_connections 2024-09-11T22:05:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.510+0000 7f9198ae4640 1 --2- 172.21.15.1:0/46625920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f919410c110 0x7f919410e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.510+0000 7f9198ae4640 1 --2- 172.21.15.1:0/46625920 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9194103660 0x7f919410bb40 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.510+0000 7f9198ae4640 1 --2- 172.21.15.1:0/46625920 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9194102d40 0x7f9194103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.510+0000 7f9198ae4640 1 -- 172.21.15.1:0/46625920 >> 172.21.15.1:0/46625920 conn(0x7f91940fc6b0 msgr2=0x7f91940fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.510+0000 7f9198ae4640 1 -- 172.21.15.1:0/46625920 shutdown_connections 2024-09-11T22:05:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.510+0000 7f9198ae4640 1 -- 172.21.15.1:0/46625920 wait complete. 2024-09-11T22:05:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.511+0000 7f9198ae4640 1 Processor -- start 2024-09-11T22:05:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.511+0000 7f9198ae4640 1 -- start start 2024-09-11T22:05:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.511+0000 7f9198ae4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9194102d40 0x7f9194100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.511+0000 7f9198ae4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9194103660 0x7f9194100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.511+0000 7f9198ae4640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f919410c110 0x7f9194101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.511+0000 7f9198ae4640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9194111ee0 con 0x7f9194102d40 2024-09-11T22:05:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.511+0000 7f9198ae4640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9194111d60 con 0x7f919410c110 2024-09-11T22:05:46.516 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.511+0000 7f9198ae4640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9194112060 con 0x7f9194103660 2024-09-11T22:05:46.516 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.511+0000 7f919259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9194102d40 0x7f9194100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:46.516 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.511+0000 7f919259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9194102d40 0x7f9194100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56496/0 (socket says 172.21.15.1:56496) 2024-09-11T22:05:46.516 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.511+0000 7f919259c640 1 -- 172.21.15.1:0/1558569199 learned_addr learned my addr 172.21.15.1:0/1558569199 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:46.516 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.511+0000 7f9192d9d640 1 --2- 172.21.15.1:0/1558569199 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f919410c110 0x7f9194101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:46.516 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.511+0000 7f9191d9b640 1 --2- 172.21.15.1:0/1558569199 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9194103660 0x7f9194100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:46.516 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.511+0000 7f919259c640 1 -- 172.21.15.1:0/1558569199 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9194103660 msgr2=0x7f9194100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:46.516 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.511+0000 7f919259c640 1 --2- 172.21.15.1:0/1558569199 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9194103660 0x7f9194100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:46.516 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.511+0000 7f919259c640 1 -- 172.21.15.1:0/1558569199 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f919410c110 msgr2=0x7f9194101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:46.516 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.511+0000 7f919259c640 1 --2- 172.21.15.1:0/1558569199 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f919410c110 0x7f9194101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:46.517 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.511+0000 7f919259c640 1 -- 172.21.15.1:0/1558569199 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91941a68b0 con 0x7f9194102d40 2024-09-11T22:05:46.517 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.512+0000 7f919259c640 1 --2- 172.21.15.1:0/1558569199 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9194102d40 0x7f9194100990 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f918400d950 tx=0x7f918400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:46.517 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.512+0000 7f91837fe640 1 -- 172.21.15.1:0/1558569199 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9184017070 con 0x7f9194102d40 2024-09-11T22:05:46.517 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.512+0000 7f9198ae4640 1 -- 172.21.15.1:0/1558569199 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91941a6ba0 con 0x7f9194102d40 2024-09-11T22:05:46.517 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.512+0000 7f91837fe640 1 -- 172.21.15.1:0/1558569199 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9184009c30 con 0x7f9194102d40 2024-09-11T22:05:46.517 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.512+0000 7f91837fe640 1 -- 172.21.15.1:0/1558569199 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918400f6b0 con 0x7f9194102d40 2024-09-11T22:05:46.517 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.512+0000 7f9198ae4640 1 -- 172.21.15.1:0/1558569199 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91941a7140 con 0x7f9194102d40 2024-09-11T22:05:46.523 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.514+0000 7f91837fe640 1 -- 172.21.15.1:0/1558569199 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f91840109d0 con 0x7f9194102d40 2024-09-11T22:05:46.523 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.515+0000 7f919b887740 1 Processor -- start 2024-09-11T22:05:46.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.515+0000 7f919b887740 1 -- start start 2024-09-11T22:05:46.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.515+0000 7f919b887740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563bbeca1af0 0x563bbeca1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:46.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.515+0000 7f919b887740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bbeca2410 0x563bbecac770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:46.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.515+0000 7f919b887740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563bbecaccb0 0x563bbecaf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:46.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.515+0000 7f919b887740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x563bbec062f0 con 0x563bbecaccb0 2024-09-11T22:05:46.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.515+0000 7f919b887740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x563bbebae270 con 0x563bbeca2410 2024-09-11T22:05:46.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.515+0000 7f919b887740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x563bbec022b0 con 0x563bbeca1af0 2024-09-11T22:05:46.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.516+0000 7f91837fe640 1 --2- 172.21.15.1:0/1558569199 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f917c0a2cf0 0x7f917c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:46.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.516+0000 7f9191d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563bbecaccb0 0x563bbecaf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:46.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.516+0000 7f9191d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563bbecaccb0 0x563bbecaf0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56498/0 (socket says 172.21.15.1:56498) 2024-09-11T22:05:46.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.516+0000 7f9191d9b640 1 -- 172.21.15.1:0/3523248447 learned_addr learned my addr 172.21.15.1:0/3523248447 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:46.524 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.516+0000 7f9192d9d640 1 --2- 172.21.15.1:0/3523248447 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563bbeca1af0 0x563bbeca1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:46.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.516+0000 7f9192d9d640 1 --2- 172.21.15.1:0/1558569199 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f917c0a2cf0 0x7f917c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:46.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.516+0000 7f91837fe640 1 -- 172.21.15.1:0/1558569199 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7f9184097ca0 con 0x7f9194102d40 2024-09-11T22:05:46.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.516+0000 7f919259c640 1 --2- 172.21.15.1:0/3523248447 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bbeca2410 0x563bbecac770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:46.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.516+0000 7f9191d9b640 1 -- 172.21.15.1:0/3523248447 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563bbeca1af0 msgr2=0x563bbeca1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:46.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.516+0000 7f9191d9b640 1 --2- 172.21.15.1:0/3523248447 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563bbeca1af0 0x563bbeca1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:46.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.516+0000 7f9191d9b640 1 -- 172.21.15.1:0/3523248447 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bbeca2410 msgr2=0x563bbecac770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:46.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.516+0000 7f9191d9b640 1 --2- 172.21.15.1:0/3523248447 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bbeca2410 0x563bbecac770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:46.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.516+0000 7f9191d9b640 1 -- 172.21.15.1:0/3523248447 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bbebad340 con 0x563bbecaccb0 2024-09-11T22:05:46.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.516+0000 7f9192d9d640 1 --2- 172.21.15.1:0/3523248447 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563bbeca1af0 0x563bbeca1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:05:46.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.516+0000 7f9191d9b640 1 --2- 172.21.15.1:0/3523248447 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563bbecaccb0 0x563bbecaf0a0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f918802a9a0 tx=0x7f9188005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=251603ecfc6012d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:46.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.516+0000 7f9192d9d640 1 --2- 172.21.15.1:0/1558569199 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f917c0a2cf0 0x7f917c0a51b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f918c009fd0 tx=0x7f918c005bd0 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:46.525 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.516+0000 7f9180ff9640 1 -- 172.21.15.1:0/3523248447 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918803f070 con 0x563bbecaccb0 2024-09-11T22:05:46.526 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.516+0000 7f9180ff9640 1 -- 172.21.15.1:0/3523248447 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9188004030 con 0x563bbecaccb0 2024-09-11T22:05:46.526 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.516+0000 7f9180ff9640 1 -- 172.21.15.1:0/3523248447 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9188035070 con 0x563bbecaccb0 2024-09-11T22:05:46.526 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.517+0000 7f919b887740 1 -- 172.21.15.1:0/3523248447 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563bbecaccb0 msgr2=0x563bbecaf0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:46.526 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.517+0000 7f919b887740 1 --2- 172.21.15.1:0/3523248447 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563bbecaccb0 0x563bbecaf0a0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f918802a9a0 tx=0x7f9188005e80 comp rx=0 tx=0).stop 2024-09-11T22:05:46.526 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.517+0000 7f919b887740 1 -- 172.21.15.1:0/3523248447 shutdown_connections 2024-09-11T22:05:46.526 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.517+0000 7f919b887740 1 --2- 172.21.15.1:0/3523248447 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563bbecaccb0 0x563bbecaf0a0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:46.526 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.517+0000 7f919b887740 1 --2- 172.21.15.1:0/3523248447 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bbeca2410 0x563bbecac770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:46.526 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.517+0000 7f919b887740 1 --2- 172.21.15.1:0/3523248447 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563bbeca1af0 0x563bbeca1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:46.526 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.517+0000 7f919b887740 1 -- 172.21.15.1:0/3523248447 >> 172.21.15.1:0/3523248447 conn(0x563bbec9e060 msgr2=0x563bbec9e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:46.526 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.517+0000 7f919b887740 1 -- 172.21.15.1:0/3523248447 shutdown_connections 2024-09-11T22:05:46.526 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.517+0000 7f919b887740 1 -- 172.21.15.1:0/3523248447 wait complete. 2024-09-11T22:05:46.526 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.518+0000 7f919b887740 1 -- start start 2024-09-11T22:05:46.526 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.518+0000 7f919b887740 1 Processor -- start 2024-09-11T22:05:46.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.518+0000 7f919b887740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bbeca1af0 0x563bbeca9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:46.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.518+0000 7f919b887740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563bbeca2410 0x563bbecaa0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:46.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.518+0000 7f919b887740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563bbecaccb0 0x563bbecaa610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:46.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.518+0000 7f919b887740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x563bbecaf880 con 0x563bbeca2410 2024-09-11T22:05:46.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.518+0000 7f919b887740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x563bbeb8f290 con 0x563bbeca1af0 2024-09-11T22:05:46.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.518+0000 7f919b887740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x563bbecafa00 con 0x563bbecaccb0 2024-09-11T22:05:46.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.518+0000 7f919b887740 10 client.0 _ll_register_callbacks cb 0x563bbebc0140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:05:46.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.518+0000 7f9191d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563bbeca2410 0x563bbecaa0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:46.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.518+0000 7f9191d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563bbeca2410 0x563bbecaa0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56502/0 (socket says 172.21.15.1:56502) 2024-09-11T22:05:46.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.518+0000 7f9191d9b640 1 -- 172.21.15.1:0/885906225 learned_addr learned my addr 172.21.15.1:0/885906225 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:46.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.518+0000 7f919259c640 1 --2- 172.21.15.1:0/885906225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bbeca1af0 0x563bbeca9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:46.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.518+0000 7f9192d9d640 1 --2- 172.21.15.1:0/885906225 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563bbecaccb0 0x563bbecaa610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:46.527 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.519+0000 7f9191d9b640 1 -- 172.21.15.1:0/885906225 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563bbecaccb0 msgr2=0x563bbecaa610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:46.528 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.519+0000 7f9191d9b640 1 --2- 172.21.15.1:0/885906225 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563bbecaccb0 0x563bbecaa610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:46.528 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.519+0000 7f9191d9b640 1 -- 172.21.15.1:0/885906225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bbeca1af0 msgr2=0x563bbeca9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:46.528 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.519+0000 7f9191d9b640 1 --2- 172.21.15.1:0/885906225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bbeca1af0 0x563bbeca9b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:46.528 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.519+0000 7f9191d9b640 1 -- 172.21.15.1:0/885906225 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bbe919210 con 0x563bbeca2410 2024-09-11T22:05:46.528 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.519+0000 7f9191d9b640 1 --2- 172.21.15.1:0/885906225 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563bbeca2410 0x563bbecaa0d0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f9188005940 tx=0x7f91880051c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a03e8b32d351c5a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:46.528 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.519+0000 7f9179ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:05:46.528 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.521+0000 7f9179ffb640 1 -- 172.21.15.1:0/885906225 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9188035070 con 0x563bbeca2410 2024-09-11T22:05:46.528 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.521+0000 7f9179ffb640 1 -- 172.21.15.1:0/885906225 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7f91880317b0 con 0x563bbeca2410 2024-09-11T22:05:46.529 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.521+0000 7f919b887740 1 -- 172.21.15.1:0/885906225 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9188004030 con 0x563bbeca2410 2024-09-11T22:05:46.529 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.521+0000 7f9179ffb640 1 -- 172.21.15.1:0/885906225 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f918803f040 con 0x563bbeca2410 2024-09-11T22:05:46.529 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.521+0000 7f919b887740 10 client.17448 fetch_fsmap learned FSMap version 45 2024-09-11T22:05:46.529 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.521+0000 7f919b887740 1 -- 172.21.15.1:0/885906225 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x563bbe9e5620 con 0x563bbeca2410 2024-09-11T22:05:46.529 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.521+0000 7f9179ffb640 1 -- 172.21.15.1:0/885906225 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9188004030 con 0x563bbeca2410 2024-09-11T22:05:46.529 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.521+0000 7f9179ffb640 1 -- 172.21.15.1:0/885906225 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 45) ==== 8442+0+0 (secure 0 0 0) 0x7f9188004930 con 0x563bbeca2410 2024-09-11T22:05:46.529 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.521+0000 7f919b887740 10 client.17448 fetch_fsmap finished waiting for FSMap version 45 2024-09-11T22:05:46.529 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.521+0000 7f919b887740 10 client.17448 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:05:46.529 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.521+0000 7f919b887740 20 client.17448 populate_metadata read hostname 'smithi001' 2024-09-11T22:05:46.529 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.521+0000 7f919b887740 1 --2- 172.21.15.1:0/885906225 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x563bbecbc9f0 0x563bbecbede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:46.529 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.521+0000 7f919b887740 4 client.17448 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:05:46.529 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.521+0000 7f919b887740 1 -- 172.21.15.1:0/885906225 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563bbec13a20 con 0x563bbecbc9f0 2024-09-11T22:05:46.529 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.522+0000 7f919259c640 1 --2- 172.21.15.1:0/885906225 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x563bbecbc9f0 0x563bbecbede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:46.530 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.522+0000 7f9179ffb640 10 client.17448 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:05:46.530 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.522+0000 7f919259c640 1 --2- 172.21.15.1:0/885906225 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x563bbecbc9f0 0x563bbecbede0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1793ea4081a111fb server_cookie=3461d185b27edbc4 in_seq=0 out_seq=0 2024-09-11T22:05:46.531 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.523+0000 7f9179ffb640 1 -- 172.21.15.1:0/885906225 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563bbec13a20 con 0x563bbecbc9f0 2024-09-11T22:05:46.531 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.523+0000 7f9179ffb640 10 client.17448 handle_command_reply: tid=0 2024-09-11T22:05:46.531 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.523+0000 7f919b887740 1 client.17448 shutdown 2024-09-11T22:05:46.531 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.523+0000 7f919b887740 1 -- 172.21.15.1:0/885906225 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563bbeca2410 msgr2=0x563bbecaa0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:46.531 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.523+0000 7f919b887740 1 --2- 172.21.15.1:0/885906225 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563bbeca2410 0x563bbecaa0d0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f9188005940 tx=0x7f91880051c0 comp rx=0 tx=0).stop 2024-09-11T22:05:46.531 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.523+0000 7f919b887740 1 -- 172.21.15.1:0/885906225 shutdown_connections 2024-09-11T22:05:46.531 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.523+0000 7f919b887740 1 --2- 172.21.15.1:0/885906225 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x563bbecbc9f0 0x563bbecbede0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:46.531 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.523+0000 7f919b887740 1 --2- 172.21.15.1:0/885906225 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563bbecaccb0 0x563bbecaa610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:46.531 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.523+0000 7f919b887740 1 --2- 172.21.15.1:0/885906225 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563bbeca2410 0x563bbecaa0d0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:46.531 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.523+0000 7f9179ffb640 0 client.17448 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:05:46.531 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.523+0000 7f919b887740 1 --2- 172.21.15.1:0/885906225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bbeca1af0 0x563bbeca9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:46.532 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.523+0000 7f919b887740 1 -- 172.21.15.1:0/885906225 >> 172.21.15.1:0/885906225 conn(0x563bbec9e060 msgr2=0x563bbeca3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:46.532 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.523+0000 7f919b887740 1 -- 172.21.15.1:0/885906225 shutdown_connections 2024-09-11T22:05:46.532 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.523+0000 7f919b887740 1 -- 172.21.15.1:0/885906225 wait complete. 2024-09-11T22:05:46.532 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.523+0000 7f919b887740 20 client.17448 trim_cache size 0 max 16384 2024-09-11T22:05:46.543 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.537+0000 7f919b887740 1 Processor -- start 2024-09-11T22:05:46.544 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.537+0000 7f919b887740 1 -- start start 2024-09-11T22:05:46.544 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.538+0000 7f919b887740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563bbeca1af0 0x563bbecaa440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:46.544 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.538+0000 7f919b887740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563bbeca2410 0x563bbecabb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:46.544 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.538+0000 7f919b887740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bbecaccb0 0x563bbecc4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:46.544 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.538+0000 7f919b887740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f91941afaf0 con 0x563bbeca1af0 2024-09-11T22:05:46.544 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.538+0000 7f919b887740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f91941b1930 con 0x563bbecaccb0 2024-09-11T22:05:46.544 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.538+0000 7f919b887740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f91941b0280 con 0x563bbeca2410 2024-09-11T22:05:46.545 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.538+0000 7f919259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563bbeca1af0 0x563bbecaa440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:46.545 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.538+0000 7f919259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563bbeca1af0 0x563bbecaa440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56512/0 (socket says 172.21.15.1:56512) 2024-09-11T22:05:46.545 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.538+0000 7f919259c640 1 -- 172.21.15.1:0/4289031845 learned_addr learned my addr 172.21.15.1:0/4289031845 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:46.545 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.538+0000 7f9192d9d640 1 --2- 172.21.15.1:0/4289031845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bbecaccb0 0x563bbecc4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:46.545 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.538+0000 7f919259c640 1 -- 172.21.15.1:0/4289031845 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563bbeca2410 msgr2=0x563bbecabb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:46.545 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.538+0000 7f9191d9b640 1 --2- 172.21.15.1:0/4289031845 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563bbeca2410 0x563bbecabb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:46.545 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.538+0000 7f919259c640 1 --2- 172.21.15.1:0/4289031845 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563bbeca2410 0x563bbecabb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:46.545 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.538+0000 7f919259c640 1 -- 172.21.15.1:0/4289031845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bbecaccb0 msgr2=0x563bbecc4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:46.545 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.538+0000 7f919259c640 1 --2- 172.21.15.1:0/4289031845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bbecaccb0 0x563bbecc4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:46.545 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.538+0000 7f919259c640 1 -- 172.21.15.1:0/4289031845 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bbeca2240 con 0x563bbeca1af0 2024-09-11T22:05:46.545 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.538+0000 7f9191d9b640 1 --2- 172.21.15.1:0/4289031845 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563bbeca2410 0x563bbecabb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:05:46.546 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.538+0000 7f9192d9d640 1 --2- 172.21.15.1:0/4289031845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bbecaccb0 0x563bbecc4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:05:46.546 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.538+0000 7f919259c640 1 --2- 172.21.15.1:0/4289031845 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563bbeca1af0 0x563bbecaa440 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f9184098000 tx=0x7f918400bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfb9fb3c175566d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:46.546 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.538+0000 7f917bfff640 1 -- 172.21.15.1:0/4289031845 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91840b3070 con 0x563bbeca1af0 2024-09-11T22:05:46.546 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.538+0000 7f917bfff640 1 -- 172.21.15.1:0/4289031845 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f918409b630 con 0x563bbeca1af0 2024-09-11T22:05:46.546 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.538+0000 7f917bfff640 1 -- 172.21.15.1:0/4289031845 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9184020040 con 0x563bbeca1af0 2024-09-11T22:05:46.546 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.539+0000 7f919b887740 1 -- 172.21.15.1:0/4289031845 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563bbeca1af0 msgr2=0x563bbecaa440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:46.546 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.539+0000 7f919b887740 1 --2- 172.21.15.1:0/4289031845 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563bbeca1af0 0x563bbecaa440 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f9184098000 tx=0x7f918400bf60 comp rx=0 tx=0).stop 2024-09-11T22:05:46.546 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.539+0000 7f919b887740 1 -- 172.21.15.1:0/4289031845 shutdown_connections 2024-09-11T22:05:46.546 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.539+0000 7f919b887740 1 --2- 172.21.15.1:0/4289031845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bbecaccb0 0x563bbecc4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:46.547 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.539+0000 7f919b887740 1 --2- 172.21.15.1:0/4289031845 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563bbeca2410 0x563bbecabb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:46.547 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.539+0000 7f919b887740 1 --2- 172.21.15.1:0/4289031845 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563bbeca1af0 0x563bbecaa440 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:46.547 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.539+0000 7f919b887740 1 -- 172.21.15.1:0/4289031845 >> 172.21.15.1:0/4289031845 conn(0x563bbec9e060 msgr2=0x563bbecab3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:46.547 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.539+0000 7f919b887740 1 -- 172.21.15.1:0/4289031845 shutdown_connections 2024-09-11T22:05:46.547 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.539+0000 7f919b887740 1 -- 172.21.15.1:0/4289031845 wait complete. 2024-09-11T22:05:46.547 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.539+0000 7f919b887740 1 -- start start 2024-09-11T22:05:46.547 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.539+0000 7f919b887740 1 Processor -- start 2024-09-11T22:05:46.547 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.540+0000 7f919b887740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bbeca1af0 0x563bbecb5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:46.547 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.540+0000 7f919b887740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563bbeca2410 0x563bbecb0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:46.547 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.540+0000 7f919b887740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563bbecaccb0 0x563bbecb1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:46.547 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.540+0000 7f919b887740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f91941b11a0 con 0x563bbeca2410 2024-09-11T22:05:46.547 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.540+0000 7f919b887740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f91941b0a10 con 0x563bbeca1af0 2024-09-11T22:05:46.548 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.540+0000 7f919b887740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f91941adad0 con 0x563bbecaccb0 2024-09-11T22:05:46.548 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.540+0000 7f919b887740 10 client.0 _ll_register_callbacks cb 0x7f9188005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:05:46.548 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.540+0000 7f9191d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563bbeca2410 0x563bbecb0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:46.548 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.540+0000 7f9191d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563bbeca2410 0x563bbecb0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56524/0 (socket says 172.21.15.1:56524) 2024-09-11T22:05:46.548 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.540+0000 7f9191d9b640 1 -- 172.21.15.1:0/2412830012 learned_addr learned my addr 172.21.15.1:0/2412830012 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:46.548 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.540+0000 7f919259c640 1 --2- 172.21.15.1:0/2412830012 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bbeca1af0 0x563bbecb5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:46.548 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.540+0000 7f9192d9d640 1 --2- 172.21.15.1:0/2412830012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563bbecaccb0 0x563bbecb1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:46.548 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.540+0000 7f9191d9b640 1 -- 172.21.15.1:0/2412830012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563bbecaccb0 msgr2=0x563bbecb1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:46.548 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.540+0000 7f9191d9b640 1 --2- 172.21.15.1:0/2412830012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563bbecaccb0 0x563bbecb1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:46.548 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.540+0000 7f9191d9b640 1 -- 172.21.15.1:0/2412830012 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bbeca1af0 msgr2=0x563bbecb5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:46.548 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.540+0000 7f9191d9b640 1 --2- 172.21.15.1:0/2412830012 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563bbeca1af0 0x563bbecb5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:46.548 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.540+0000 7f9191d9b640 1 -- 172.21.15.1:0/2412830012 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bbed26410 con 0x563bbeca2410 2024-09-11T22:05:46.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.540+0000 7f9192d9d640 1 --2- 172.21.15.1:0/2412830012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563bbecaccb0 0x563bbecb1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:05:46.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.540+0000 7f9191d9b640 1 --2- 172.21.15.1:0/2412830012 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563bbeca2410 0x563bbecb0870 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f9188005c40 tx=0x7f9188005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc66edb1a166e23 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:46.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.540+0000 7f9179ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:05:46.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.540+0000 7f9179ffb640 1 -- 172.21.15.1:0/2412830012 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9188035070 con 0x563bbeca2410 2024-09-11T22:05:46.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.541+0000 7f919b887740 1 -- 172.21.15.1:0/2412830012 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f918409b630 con 0x563bbeca2410 2024-09-11T22:05:46.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.541+0000 7f9179ffb640 1 -- 172.21.15.1:0/2412830012 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7f91880048c0 con 0x563bbeca2410 2024-09-11T22:05:46.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.541+0000 7f9179ffb640 1 -- 172.21.15.1:0/2412830012 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f9188062050 con 0x563bbeca2410 2024-09-11T22:05:46.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.541+0000 7f919b887740 10 client.17454 fetch_fsmap learned FSMap version 45 2024-09-11T22:05:46.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.541+0000 7f919b887740 1 -- 172.21.15.1:0/2412830012 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x563bbed27510 con 0x563bbeca2410 2024-09-11T22:05:46.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.541+0000 7f9179ffb640 1 -- 172.21.15.1:0/2412830012 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f918409b630 con 0x563bbeca2410 2024-09-11T22:05:46.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.541+0000 7f9179ffb640 1 -- 172.21.15.1:0/2412830012 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 45) ==== 8442+0+0 (secure 0 0 0) 0x7f918803bb10 con 0x563bbeca2410 2024-09-11T22:05:46.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.541+0000 7f919b887740 10 client.17454 fetch_fsmap finished waiting for FSMap version 45 2024-09-11T22:05:46.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.541+0000 7f919b887740 10 client.17454 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:05:46.550 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.541+0000 7f919b887740 20 client.17454 populate_metadata read hostname 'smithi001' 2024-09-11T22:05:46.550 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.541+0000 7f919b887740 1 --2- 172.21.15.1:0/2412830012 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x563bbecbc9f0 0x563bbed29d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:46.550 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.541+0000 7f919b887740 4 client.17454 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:05:46.550 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.541+0000 7f919b887740 1 -- 172.21.15.1:0/2412830012 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563bbec0dd70 con 0x563bbecbc9f0 2024-09-11T22:05:46.550 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.541+0000 7f919259c640 1 --2- 172.21.15.1:0/2412830012 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x563bbecbc9f0 0x563bbed29d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:46.550 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.542+0000 7f9179ffb640 10 client.17454 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:05:46.550 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-09-11T22:05:46.542+0000 7f919259c640 1 --2- 172.21.15.1:0/2412830012 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x563bbecbc9f0 0x563bbed29d00 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4bf2ac03225f5cd server_cookie=6ec6d726c594e026 in_seq=0 out_seq=0 2024-09-11T22:05:46.591 INFO:tasks.workunit.client.0.smithi001.stderr:+ export PAYLOAD=/tmp/multiple_rsync_payload.201961 2024-09-11T22:05:46.591 INFO:tasks.workunit.client.0.smithi001.stderr:+ PAYLOAD=/tmp/multiple_rsync_payload.201961 2024-09-11T22:05:46.591 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo cp -r /usr/lib/ /tmp/multiple_rsync_payload.201961 2024-09-11T22:05:46.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:46 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/3263835799' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "foo.201307", "pool2": "foo.201307", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T22:05:46.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:46 smithi053 ceph-mon[34908]: osdmap e95: 12 total, 12 up, 12 in 2024-09-11T22:05:46.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:46 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/3263835799' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "foo.201307", "pool2": "foo.201307", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T22:05:46.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:46 smithi053 ceph-mon[34908]: pgmap v267: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-09-11T22:05:46.798 INFO:journalctl@ceph.mds.b.smithi053.stdout:Sep 11 22:05:46 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-b[64373]: 2024-09-11T22:05:46.556+0000 7f17b38c8640 -1 log_channel(cluster) log [ERR] : failed to store backtrace on ino 0x100000069d1 object, pool 2, errno -2 2024-09-11T22:05:46.798 INFO:journalctl@ceph.mds.b.smithi053.stdout:Sep 11 22:05:46 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-b[64373]: 2024-09-11T22:05:46.556+0000 7f17b38c8640 -1 mds.0.14 unhandled write error (2) No such file or directory, force readonly... 2024-09-11T22:05:46.798 INFO:journalctl@ceph.mds.b.smithi053.stdout:Sep 11 22:05:46 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-b[64373]: 2024-09-11T22:05:46.561+0000 7f17b38c8640 -1 mds.0.14 unhandled write error (2) No such file or directory, force readonly... 2024-09-11T22:05:46.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:46 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/3263835799' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "foo.201307", "pool2": "foo.201307", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T22:05:46.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:46 smithi161 ceph-mon[35615]: osdmap e95: 12 total, 12 up, 12 in 2024-09-11T22:05:46.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:46 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/3263835799' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "foo.201307", "pool2": "foo.201307", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T22:05:46.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:46 smithi161 ceph-mon[35615]: pgmap v267: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-09-11T22:05:46.925 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:46 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/3263835799' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "foo.201307", "pool2": "foo.201307", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T22:05:46.926 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:46 smithi001 ceph-mon[30935]: osdmap e95: 12 total, 12 up, 12 in 2024-09-11T22:05:46.926 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:46 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/3263835799' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "foo.201307", "pool2": "foo.201307", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T22:05:46.926 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:46 smithi001 ceph-mon[30935]: pgmap v267: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-09-11T22:05:47.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.008+0000 7fef2894c640 1 -- 172.21.15.1:0/1284495169 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef24102da0 msgr2=0x7fef24103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:47.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.008+0000 7fef2894c640 1 --2- 172.21.15.1:0/1284495169 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef24102da0 0x7fef24103180 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fef14009ca0 tx=0x7fef1402a950 comp rx=0 tx=0).stop 2024-09-11T22:05:47.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.008+0000 7fef2894c640 1 -- 172.21.15.1:0/1284495169 shutdown_connections 2024-09-11T22:05:47.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.008+0000 7fef2894c640 1 --2- 172.21.15.1:0/1284495169 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef2410c180 0x7fef2410e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:47.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.008+0000 7fef2894c640 1 --2- 172.21.15.1:0/1284495169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef241036c0 0x7fef2410bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:47.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.008+0000 7fef2894c640 1 --2- 172.21.15.1:0/1284495169 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef24102da0 0x7fef24103180 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:47.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.008+0000 7fef2894c640 1 -- 172.21.15.1:0/1284495169 >> 172.21.15.1:0/1284495169 conn(0x7fef240fc7a0 msgr2=0x7fef240febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:47.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.008+0000 7fef2894c640 1 -- 172.21.15.1:0/1284495169 shutdown_connections 2024-09-11T22:05:47.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.008+0000 7fef2894c640 1 -- 172.21.15.1:0/1284495169 wait complete. 2024-09-11T22:05:47.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.008+0000 7fef2894c640 1 Processor -- start 2024-09-11T22:05:47.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.009+0000 7fef2894c640 1 -- start start 2024-09-11T22:05:47.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.009+0000 7fef2894c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef24102da0 0x7fef24100a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:47.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.009+0000 7fef2894c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef241036c0 0x7fef24100f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:47.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.009+0000 7fef2894c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef2410c180 0x7fef24101480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:47.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.009+0000 7fef2894c640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fef24111f60 con 0x7fef2410c180 2024-09-11T22:05:47.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.009+0000 7fef2894c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fef24111de0 con 0x7fef24102da0 2024-09-11T22:05:47.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.009+0000 7fef2894c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fef241120e0 con 0x7fef241036c0 2024-09-11T22:05:47.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.009+0000 7fef22d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef2410c180 0x7fef24101480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:47.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.009+0000 7fef22d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef2410c180 0x7fef24101480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56538/0 (socket says 172.21.15.1:56538) 2024-09-11T22:05:47.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.009+0000 7fef22d9d640 1 -- 172.21.15.1:0/3117575348 learned_addr learned my addr 172.21.15.1:0/3117575348 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:47.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.009+0000 7fef2259c640 1 --2- 172.21.15.1:0/3117575348 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef24102da0 0x7fef24100a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:47.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.009+0000 7fef21d9b640 1 --2- 172.21.15.1:0/3117575348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef241036c0 0x7fef24100f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:47.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.009+0000 7fef22d9d640 1 -- 172.21.15.1:0/3117575348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef241036c0 msgr2=0x7fef24100f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:47.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.009+0000 7fef22d9d640 1 --2- 172.21.15.1:0/3117575348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef241036c0 0x7fef24100f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:47.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.009+0000 7fef22d9d640 1 -- 172.21.15.1:0/3117575348 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef24102da0 msgr2=0x7fef24100a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:47.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.009+0000 7fef22d9d640 1 --2- 172.21.15.1:0/3117575348 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef24102da0 0x7fef24100a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:47.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.009+0000 7fef22d9d640 1 -- 172.21.15.1:0/3117575348 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef241a6880 con 0x7fef2410c180 2024-09-11T22:05:47.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.009+0000 7fef2259c640 1 --2- 172.21.15.1:0/3117575348 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef24102da0 0x7fef24100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:05:47.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.009+0000 7fef22d9d640 1 --2- 172.21.15.1:0/3117575348 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef2410c180 0x7fef24101480 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fef1c00c970 tx=0x7fef1c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:47.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.010+0000 7fef1b7fe640 1 -- 172.21.15.1:0/3117575348 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef1c007bb0 con 0x7fef2410c180 2024-09-11T22:05:47.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.010+0000 7fef1b7fe640 1 -- 172.21.15.1:0/3117575348 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef1c007d50 con 0x7fef2410c180 2024-09-11T22:05:47.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.010+0000 7fef1b7fe640 1 -- 172.21.15.1:0/3117575348 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef1c018800 con 0x7fef2410c180 2024-09-11T22:05:47.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.010+0000 7fef2894c640 1 -- 172.21.15.1:0/3117575348 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef241a6b70 con 0x7fef2410c180 2024-09-11T22:05:47.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.010+0000 7fef2894c640 1 -- 172.21.15.1:0/3117575348 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef241a70b0 con 0x7fef2410c180 2024-09-11T22:05:47.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.012+0000 7fef2ad25740 1 Processor -- start 2024-09-11T22:05:47.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.012+0000 7fef1b7fe640 1 -- 172.21.15.1:0/3117575348 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7fef1c00fa50 con 0x7fef2410c180 2024-09-11T22:05:47.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.013+0000 7fef2ad25740 1 -- start start 2024-09-11T22:05:47.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.013+0000 7fef2ad25740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe9cb2cb40 0x55fe9cb2b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:47.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.013+0000 7fef2ad25740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe9cb2b5d0 0x55fe9cb2ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:47.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.013+0000 7fef2ad25740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe9cb36370 0x55fe9cb38760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:47.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.013+0000 7fef2ad25740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55fe9ca912f0 con 0x55fe9cb2b5d0 2024-09-11T22:05:47.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.013+0000 7fef2ad25740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55fe9ca38cf0 con 0x55fe9cb36370 2024-09-11T22:05:47.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.013+0000 7fef2ad25740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55fe9ca8d2b0 con 0x55fe9cb2cb40 2024-09-11T22:05:47.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.013+0000 7fef2259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe9cb2b5d0 0x55fe9cb2ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:47.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.013+0000 7fef2259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe9cb2b5d0 0x55fe9cb2ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56544/0 (socket says 172.21.15.1:56544) 2024-09-11T22:05:47.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.013+0000 7fef2259c640 1 -- 172.21.15.1:0/4272281838 learned_addr learned my addr 172.21.15.1:0/4272281838 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:47.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.013+0000 7fef1b7fe640 1 --2- 172.21.15.1:0/3117575348 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fef100a2d70 0x7fef100a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:47.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.013+0000 7fef21d9b640 1 --2- 172.21.15.1:0/4272281838 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe9cb36370 0x55fe9cb38760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:47.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.013+0000 7fef21d9b640 1 --2- 172.21.15.1:0/4272281838 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe9cb2cb40 0x55fe9cb2b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:47.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.013+0000 7fef22d9d640 1 --2- 172.21.15.1:0/3117575348 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fef100a2d70 0x7fef100a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:47.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.014+0000 7fef2259c640 1 -- 172.21.15.1:0/4272281838 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe9cb2cb40 msgr2=0x55fe9cb2b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:47.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.014+0000 7fef2259c640 1 --2- 172.21.15.1:0/4272281838 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe9cb2cb40 0x55fe9cb2b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:47.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.014+0000 7fef2259c640 1 -- 172.21.15.1:0/4272281838 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe9cb36370 msgr2=0x55fe9cb38760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:47.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.014+0000 7fef2259c640 1 --2- 172.21.15.1:0/4272281838 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe9cb36370 0x55fe9cb38760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:47.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.014+0000 7fef2259c640 1 -- 172.21.15.1:0/4272281838 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe9ca1d3a0 con 0x55fe9cb2b5d0 2024-09-11T22:05:47.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.014+0000 7fef21d9b640 1 --2- 172.21.15.1:0/4272281838 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe9cb2cb40 0x55fe9cb2b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:05:47.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.014+0000 7fef1b7fe640 1 -- 172.21.15.1:0/3117575348 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7fef1c0cf4a0 con 0x7fef2410c180 2024-09-11T22:05:47.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.014+0000 7fef2259c640 1 --2- 172.21.15.1:0/4272281838 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe9cb2b5d0 0x55fe9cb2ba10 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fef14035000 tx=0x7fef14009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=276c2cac40219974 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:47.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.014+0000 7fef18ff9640 1 -- 172.21.15.1:0/4272281838 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef14034040 con 0x55fe9cb2b5d0 2024-09-11T22:05:47.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.014+0000 7fef22d9d640 1 --2- 172.21.15.1:0/3117575348 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fef100a2d70 0x7fef100a5230 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fef1c0cd1b0 tx=0x7fef1c00fdf0 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:47.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.014+0000 7fef18ff9640 1 -- 172.21.15.1:0/4272281838 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef14003970 con 0x55fe9cb2b5d0 2024-09-11T22:05:47.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.014+0000 7fef18ff9640 1 -- 172.21.15.1:0/4272281838 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef14004700 con 0x55fe9cb2b5d0 2024-09-11T22:05:47.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.015+0000 7fef2ad25740 1 -- 172.21.15.1:0/4272281838 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe9cb2b5d0 msgr2=0x55fe9cb2ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:47.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.015+0000 7fef2ad25740 1 --2- 172.21.15.1:0/4272281838 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe9cb2b5d0 0x55fe9cb2ba10 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fef14035000 tx=0x7fef14009a20 comp rx=0 tx=0).stop 2024-09-11T22:05:47.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.015+0000 7fef2ad25740 1 -- 172.21.15.1:0/4272281838 shutdown_connections 2024-09-11T22:05:47.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.015+0000 7fef2ad25740 1 --2- 172.21.15.1:0/4272281838 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe9cb36370 0x55fe9cb38760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:47.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.015+0000 7fef2ad25740 1 --2- 172.21.15.1:0/4272281838 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe9cb2b5d0 0x55fe9cb2ba10 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:47.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.015+0000 7fef2ad25740 1 --2- 172.21.15.1:0/4272281838 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe9cb2cb40 0x55fe9cb2b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:47.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.015+0000 7fef2ad25740 1 -- 172.21.15.1:0/4272281838 >> 172.21.15.1:0/4272281838 conn(0x55fe9caf4310 msgr2=0x55fe9cb27c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:47.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.015+0000 7fef2ad25740 1 -- 172.21.15.1:0/4272281838 shutdown_connections 2024-09-11T22:05:47.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.015+0000 7fef2ad25740 1 -- 172.21.15.1:0/4272281838 wait complete. 2024-09-11T22:05:47.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.015+0000 7fef2ad25740 1 -- start start 2024-09-11T22:05:47.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.015+0000 7fef2ad25740 1 Processor -- start 2024-09-11T22:05:47.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.016+0000 7fef2ad25740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe9cb2b5d0 0x55fe9cb35d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:47.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.016+0000 7fef2ad25740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe9cb2cb40 0x55fe9cb32fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:47.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.016+0000 7fef2ad25740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe9cb36370 0x55fe9cb33530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:47.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.016+0000 7fef2ad25740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55fe9cb39010 con 0x55fe9cb2cb40 2024-09-11T22:05:47.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.016+0000 7fef2ad25740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55fe9ca4c060 con 0x55fe9cb2b5d0 2024-09-11T22:05:47.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.016+0000 7fef2ad25740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55fe9cb39190 con 0x55fe9cb36370 2024-09-11T22:05:47.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.016+0000 7fef2ad25740 10 client.0 _ll_register_callbacks cb 0x55fe9c79f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:05:47.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.016+0000 7fef2259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe9cb2cb40 0x55fe9cb32fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:47.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.016+0000 7fef2259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe9cb2cb40 0x55fe9cb32fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56554/0 (socket says 172.21.15.1:56554) 2024-09-11T22:05:47.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.016+0000 7fef2259c640 1 -- 172.21.15.1:0/2954720433 learned_addr learned my addr 172.21.15.1:0/2954720433 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:47.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.016+0000 7fef21d9b640 1 --2- 172.21.15.1:0/2954720433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe9cb2b5d0 0x55fe9cb35d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:47.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.016+0000 7fef2259c640 1 -- 172.21.15.1:0/2954720433 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe9cb36370 msgr2=0x55fe9cb33530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:47.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.016+0000 7fef21d9b640 1 --2- 172.21.15.1:0/2954720433 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe9cb36370 0x55fe9cb33530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:47.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.016+0000 7fef2259c640 1 --2- 172.21.15.1:0/2954720433 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe9cb36370 0x55fe9cb33530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:47.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.016+0000 7fef2259c640 1 -- 172.21.15.1:0/2954720433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe9cb2b5d0 msgr2=0x55fe9cb35d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:47.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.016+0000 7fef2259c640 1 --2- 172.21.15.1:0/2954720433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe9cb2b5d0 0x55fe9cb35d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:47.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.016+0000 7fef2259c640 1 -- 172.21.15.1:0/2954720433 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe9c8023c0 con 0x55fe9cb2cb40 2024-09-11T22:05:47.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.016+0000 7fef21d9b640 1 --2- 172.21.15.1:0/2954720433 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe9cb36370 0x55fe9cb33530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:05:47.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.016+0000 7fef2259c640 1 --2- 172.21.15.1:0/2954720433 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe9cb2cb40 0x55fe9cb32fb0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fef1402a990 tx=0x7fef14003e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb3dff5ab99ae318 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:47.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.016+0000 7fef09ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:05:47.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.016+0000 7fef09ffb640 1 -- 172.21.15.1:0/2954720433 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef14034070 con 0x55fe9cb2cb40 2024-09-11T22:05:47.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.016+0000 7fef09ffb640 1 -- 172.21.15.1:0/2954720433 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7fef14004480 con 0x55fe9cb2cb40 2024-09-11T22:05:47.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.017+0000 7fef2ad25740 1 -- 172.21.15.1:0/2954720433 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef14003970 con 0x55fe9cb2cb40 2024-09-11T22:05:47.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.017+0000 7fef09ffb640 1 -- 172.21.15.1:0/2954720433 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fef14004680 con 0x55fe9cb2cb40 2024-09-11T22:05:47.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.017+0000 7fef2ad25740 10 client.17466 fetch_fsmap learned FSMap version 45 2024-09-11T22:05:47.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.017+0000 7fef2ad25740 1 -- 172.21.15.1:0/2954720433 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55fe9cb39010 con 0x55fe9cb2cb40 2024-09-11T22:05:47.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.017+0000 7fef09ffb640 1 -- 172.21.15.1:0/2954720433 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef14003970 con 0x55fe9cb2cb40 2024-09-11T22:05:47.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.017+0000 7fef09ffb640 1 -- 172.21.15.1:0/2954720433 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 45) ==== 8442+0+0 (secure 0 0 0) 0x7fef14042070 con 0x55fe9cb2cb40 2024-09-11T22:05:47.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.017+0000 7fef2ad25740 10 client.17466 fetch_fsmap finished waiting for FSMap version 45 2024-09-11T22:05:47.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.017+0000 7fef2ad25740 10 client.17466 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-11T22:05:47.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.017+0000 7fef2ad25740 20 client.17466 populate_metadata read hostname 'smithi001' 2024-09-11T22:05:47.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.017+0000 7fef2ad25740 1 --2- 172.21.15.1:0/2954720433 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x55fe9cb45bf0 0x55fe9cb47fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:47.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.017+0000 7fef2ad25740 4 client.17466 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:05:47.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.017+0000 7fef2ad25740 1 -- 172.21.15.1:0/2954720433 --> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe9ca9e860 con 0x55fe9cb45bf0 2024-09-11T22:05:47.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.018+0000 7fef21d9b640 1 --2- 172.21.15.1:0/2954720433 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x55fe9cb45bf0 0x55fe9cb47fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:47.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.018+0000 7fef21d9b640 1 --2- 172.21.15.1:0/2954720433 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x55fe9cb45bf0 0x55fe9cb47fe0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a0ba15a62ee0e60a server_cookie=d73e87523755c2a0 in_seq=0 out_seq=0 2024-09-11T22:05:47.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.018+0000 7fef09ffb640 10 client.17466 ms_handle_connect on v2:172.21.15.53:6836/3457131830 2024-09-11T22:05:47.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.019+0000 7fef09ffb640 1 -- 172.21.15.1:0/2954720433 <== mds.2 v2:172.21.15.53:6836/3457131830 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fe9ca9e860 con 0x55fe9cb45bf0 2024-09-11T22:05:47.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.019+0000 7fef09ffb640 10 client.17466 handle_command_reply: tid=0 2024-09-11T22:05:47.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.019+0000 7fef2ad25740 1 client.17466 shutdown 2024-09-11T22:05:47.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.019+0000 7fef2ad25740 1 -- 172.21.15.1:0/2954720433 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe9cb2cb40 msgr2=0x55fe9cb32fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:47.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.019+0000 7fef2ad25740 1 --2- 172.21.15.1:0/2954720433 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe9cb2cb40 0x55fe9cb32fb0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fef1402a990 tx=0x7fef14003e90 comp rx=0 tx=0).stop 2024-09-11T22:05:47.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.019+0000 7fef2ad25740 1 -- 172.21.15.1:0/2954720433 shutdown_connections 2024-09-11T22:05:47.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.019+0000 7fef2ad25740 1 --2- 172.21.15.1:0/2954720433 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x55fe9cb45bf0 0x55fe9cb47fe0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:47.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.019+0000 7fef2ad25740 1 --2- 172.21.15.1:0/2954720433 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe9cb36370 0x55fe9cb33530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:47.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.019+0000 7fef2ad25740 1 --2- 172.21.15.1:0/2954720433 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe9cb2cb40 0x55fe9cb32fb0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:47.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.019+0000 7fef2ad25740 1 --2- 172.21.15.1:0/2954720433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe9cb2b5d0 0x55fe9cb35d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:47.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.019+0000 7fef09ffb640 0 client.17466 ms_handle_reset on v2:172.21.15.53:6836/3457131830 2024-09-11T22:05:47.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.019+0000 7fef2ad25740 1 -- 172.21.15.1:0/2954720433 >> 172.21.15.1:0/2954720433 conn(0x55fe9caf4310 msgr2=0x55fe9cb2d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:47.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.019+0000 7fef2ad25740 1 -- 172.21.15.1:0/2954720433 shutdown_connections 2024-09-11T22:05:47.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.019+0000 7fef2ad25740 1 -- 172.21.15.1:0/2954720433 wait complete. 2024-09-11T22:05:47.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.019+0000 7fef2ad25740 20 client.17466 trim_cache size 0 max 16384 2024-09-11T22:05:47.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.032+0000 7fef2ad25740 1 Processor -- start 2024-09-11T22:05:47.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.032+0000 7fef2ad25740 1 -- start start 2024-09-11T22:05:47.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.032+0000 7fef2ad25740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe9cb2b5d0 0x55fe9cb422e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:47.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.032+0000 7fef2ad25740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe9cb2cb40 0x55fe9cb37760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:47.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.032+0000 7fef2ad25740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe9cb36370 0x55fe9cb37ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:47.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.032+0000 7fef2ad25740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fef241afaf0 con 0x55fe9cb2b5d0 2024-09-11T22:05:47.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.032+0000 7fef2ad25740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fef241b1930 con 0x55fe9cb2cb40 2024-09-11T22:05:47.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.032+0000 7fef2ad25740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fef241b0280 con 0x55fe9cb36370 2024-09-11T22:05:47.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.032+0000 7fef21d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe9cb2b5d0 0x55fe9cb422e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:47.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.033+0000 7fef21d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe9cb2b5d0 0x55fe9cb422e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56558/0 (socket says 172.21.15.1:56558) 2024-09-11T22:05:47.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.033+0000 7fef21d9b640 1 -- 172.21.15.1:0/1564163764 learned_addr learned my addr 172.21.15.1:0/1564163764 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:47.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.033+0000 7fef21d9b640 1 -- 172.21.15.1:0/1564163764 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe9cb36370 msgr2=0x55fe9cb37ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:47.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.033+0000 7fef21d9b640 1 --2- 172.21.15.1:0/1564163764 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe9cb36370 0x55fe9cb37ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:47.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.033+0000 7fef21d9b640 1 -- 172.21.15.1:0/1564163764 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe9cb2cb40 msgr2=0x55fe9cb37760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:47.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.033+0000 7fef21d9b640 1 --2- 172.21.15.1:0/1564163764 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe9cb2cb40 0x55fe9cb37760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:47.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.033+0000 7fef21d9b640 1 -- 172.21.15.1:0/1564163764 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe9cb27ed0 con 0x55fe9cb2b5d0 2024-09-11T22:05:47.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.033+0000 7fef21d9b640 1 --2- 172.21.15.1:0/1564163764 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe9cb2b5d0 0x55fe9cb422e0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fef0c005fd0 tx=0x7fef0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50100fdf5cc8a35 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:47.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.033+0000 7fef0bfff640 1 -- 172.21.15.1:0/1564163764 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef0c004020 con 0x55fe9cb2b5d0 2024-09-11T22:05:47.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.033+0000 7fef0bfff640 1 -- 172.21.15.1:0/1564163764 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef0c005500 con 0x55fe9cb2b5d0 2024-09-11T22:05:47.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.033+0000 7fef0bfff640 1 -- 172.21.15.1:0/1564163764 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef0c004ca0 con 0x55fe9cb2b5d0 2024-09-11T22:05:47.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.034+0000 7fef2ad25740 1 -- 172.21.15.1:0/1564163764 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe9cb2b5d0 msgr2=0x55fe9cb422e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:47.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.034+0000 7fef2ad25740 1 --2- 172.21.15.1:0/1564163764 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe9cb2b5d0 0x55fe9cb422e0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fef0c005fd0 tx=0x7fef0c0046c0 comp rx=0 tx=0).stop 2024-09-11T22:05:47.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.034+0000 7fef2ad25740 1 -- 172.21.15.1:0/1564163764 shutdown_connections 2024-09-11T22:05:47.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.034+0000 7fef2ad25740 1 --2- 172.21.15.1:0/1564163764 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe9cb36370 0x55fe9cb37ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:47.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.034+0000 7fef2ad25740 1 --2- 172.21.15.1:0/1564163764 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe9cb2cb40 0x55fe9cb37760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:47.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.034+0000 7fef2ad25740 1 --2- 172.21.15.1:0/1564163764 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe9cb2b5d0 0x55fe9cb422e0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:47.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.034+0000 7fef2ad25740 1 -- 172.21.15.1:0/1564163764 >> 172.21.15.1:0/1564163764 conn(0x55fe9caf4310 msgr2=0x55fe9cb2e310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:47.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.034+0000 7fef2ad25740 1 -- 172.21.15.1:0/1564163764 shutdown_connections 2024-09-11T22:05:47.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.034+0000 7fef2ad25740 1 -- 172.21.15.1:0/1564163764 wait complete. 2024-09-11T22:05:47.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.034+0000 7fef2ad25740 1 -- start start 2024-09-11T22:05:47.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.034+0000 7fef2ad25740 1 Processor -- start 2024-09-11T22:05:47.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.034+0000 7fef2ad25740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe9cb2b5d0 0x55fe9cb3ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:47.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.034+0000 7fef2ad25740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe9cb2cb40 0x55fe9cb4d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:47.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.034+0000 7fef2ad25740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe9cb36370 0x55fe9cb4d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:47.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.035+0000 7fef2ad25740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fef241b11a0 con 0x55fe9cb36370 2024-09-11T22:05:47.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.035+0000 7fef2ad25740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fef241b0a10 con 0x55fe9cb2cb40 2024-09-11T22:05:47.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.035+0000 7fef2ad25740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fef241adad0 con 0x55fe9cb2b5d0 2024-09-11T22:05:47.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.035+0000 7fef2ad25740 10 client.0 _ll_register_callbacks cb 0x7fef0c00e9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:05:47.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.035+0000 7fef21d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe9cb36370 0x55fe9cb4d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:47.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.035+0000 7fef21d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe9cb36370 0x55fe9cb4d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56560/0 (socket says 172.21.15.1:56560) 2024-09-11T22:05:47.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.035+0000 7fef21d9b640 1 -- 172.21.15.1:0/182513146 learned_addr learned my addr 172.21.15.1:0/182513146 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:47.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.035+0000 7fef2259c640 1 --2- 172.21.15.1:0/182513146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe9cb2cb40 0x55fe9cb4d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:47.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.035+0000 7fef21d9b640 1 --2- 172.21.15.1:0/182513146 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe9cb2b5d0 0x55fe9cb3ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:47.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.035+0000 7fef21d9b640 1 -- 172.21.15.1:0/182513146 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe9cb2b5d0 msgr2=0x55fe9cb3ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:47.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.035+0000 7fef21d9b640 1 --2- 172.21.15.1:0/182513146 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe9cb2b5d0 0x55fe9cb3ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:47.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.035+0000 7fef21d9b640 1 -- 172.21.15.1:0/182513146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe9cb2cb40 msgr2=0x55fe9cb4d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:47.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.035+0000 7fef21d9b640 1 --2- 172.21.15.1:0/182513146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe9cb2cb40 0x55fe9cb4d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:47.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.035+0000 7fef21d9b640 1 -- 172.21.15.1:0/182513146 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe9cb4df80 con 0x55fe9cb36370 2024-09-11T22:05:47.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.035+0000 7fef2259c640 1 --2- 172.21.15.1:0/182513146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe9cb2cb40 0x55fe9cb4d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:05:47.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.035+0000 7fef21d9b640 1 --2- 172.21.15.1:0/182513146 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe9cb36370 0x55fe9cb4d780 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fef0c0121a0 tx=0x7fef0c0124a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60b0fca204719841 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:47.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.035+0000 7fef09ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:05:47.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.035+0000 7fef09ffb640 1 -- 172.21.15.1:0/182513146 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef0c013460 con 0x55fe9cb36370 2024-09-11T22:05:47.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.035+0000 7fef09ffb640 1 -- 172.21.15.1:0/182513146 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7fef0c004020 con 0x55fe9cb36370 2024-09-11T22:05:47.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.035+0000 7fef2ad25740 1 -- 172.21.15.1:0/182513146 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef0c005500 con 0x55fe9cb36370 2024-09-11T22:05:47.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.035+0000 7fef09ffb640 1 -- 172.21.15.1:0/182513146 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fef0c004220 con 0x55fe9cb36370 2024-09-11T22:05:47.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.035+0000 7fef2ad25740 10 client.17472 fetch_fsmap learned FSMap version 45 2024-09-11T22:05:47.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.035+0000 7fef2ad25740 1 -- 172.21.15.1:0/182513146 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55fe9cbb0880 con 0x55fe9cb36370 2024-09-11T22:05:47.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.036+0000 7fef09ffb640 1 -- 172.21.15.1:0/182513146 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef0c005500 con 0x55fe9cb36370 2024-09-11T22:05:47.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.036+0000 7fef09ffb640 1 -- 172.21.15.1:0/182513146 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 45) ==== 8442+0+0 (secure 0 0 0) 0x7fef0c025070 con 0x55fe9cb36370 2024-09-11T22:05:47.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.036+0000 7fef2ad25740 10 client.17472 fetch_fsmap finished waiting for FSMap version 45 2024-09-11T22:05:47.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.036+0000 7fef2ad25740 10 client.17472 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-11T22:05:47.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.036+0000 7fef2ad25740 20 client.17472 populate_metadata read hostname 'smithi001' 2024-09-11T22:05:47.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.036+0000 7fef2ad25740 1 --2- 172.21.15.1:0/182513146 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x55fe9cbb0f50 0x55fe9cbb3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:47.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.036+0000 7fef2ad25740 4 client.17472 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:05:47.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.036+0000 7fef2ad25740 1 -- 172.21.15.1:0/182513146 --> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fe9ca98d70 con 0x55fe9cbb0f50 2024-09-11T22:05:47.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.036+0000 7fef2259c640 1 --2- 172.21.15.1:0/182513146 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x55fe9cbb0f50 0x55fe9cbb3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:47.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.037+0000 7fef2259c640 1 --2- 172.21.15.1:0/182513146 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x55fe9cbb0f50 0x55fe9cbb3370 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8e8208323bca522c server_cookie=291fc6bba5e2122e in_seq=0 out_seq=0 2024-09-11T22:05:47.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.037+0000 7fef09ffb640 10 client.17472 ms_handle_connect on v2:172.21.15.53:6836/3457131830 2024-09-11T22:05:47.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.037+0000 7fef09ffb640 1 -- 172.21.15.1:0/182513146 <== mds.2 v2:172.21.15.53:6836/3457131830 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fe9ca98d70 con 0x55fe9cbb0f50 2024-09-11T22:05:47.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.037+0000 7fef09ffb640 10 client.17472 handle_command_reply: tid=0 2024-09-11T22:05:47.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.037+0000 7fef2ad25740 1 client.17472 shutdown 2024-09-11T22:05:47.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.037+0000 7fef2ad25740 1 -- 172.21.15.1:0/182513146 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe9cb36370 msgr2=0x55fe9cb4d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:47.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.037+0000 7fef2ad25740 1 --2- 172.21.15.1:0/182513146 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe9cb36370 0x55fe9cb4d780 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fef0c0121a0 tx=0x7fef0c0124a0 comp rx=0 tx=0).stop 2024-09-11T22:05:47.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.037+0000 7fef2ad25740 1 -- 172.21.15.1:0/182513146 shutdown_connections 2024-09-11T22:05:47.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.037+0000 7fef2ad25740 1 --2- 172.21.15.1:0/182513146 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x55fe9cbb0f50 0x55fe9cbb3370 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:47.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.037+0000 7fef2ad25740 1 --2- 172.21.15.1:0/182513146 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe9cb36370 0x55fe9cb4d780 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:47.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.037+0000 7fef2ad25740 1 --2- 172.21.15.1:0/182513146 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe9cb2cb40 0x55fe9cb4d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:47.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.037+0000 7fef2ad25740 1 --2- 172.21.15.1:0/182513146 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55fe9cb2b5d0 0x55fe9cb3ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:47.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.037+0000 7fef2ad25740 1 -- 172.21.15.1:0/182513146 >> 172.21.15.1:0/182513146 conn(0x55fe9caf4310 msgr2=0x55fe9cb33090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:47.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.037+0000 7fef09ffb640 0 client.17472 ms_handle_reset on v2:172.21.15.53:6836/3457131830 2024-09-11T22:05:47.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.037+0000 7fef2ad25740 1 -- 172.21.15.1:0/182513146 shutdown_connections 2024-09-11T22:05:47.063 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-11T22:05:47.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.037+0000 7fef2ad25740 1 -- 172.21.15.1:0/182513146 wait complete. 2024-09-11T22:05:47.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.037+0000 7fef2ad25740 20 client.17472 trim_cache size 0 max 16384 2024-09-11T22:05:47.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.038+0000 7fef2894c640 1 -- 172.21.15.1:0/3117575348 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fef100a2d70 msgr2=0x7fef100a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:47.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.038+0000 7fef2894c640 1 --2- 172.21.15.1:0/3117575348 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fef100a2d70 0x7fef100a5230 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fef1c0cd1b0 tx=0x7fef1c00fdf0 comp rx=0 tx=0).stop 2024-09-11T22:05:47.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.038+0000 7fef2894c640 1 -- 172.21.15.1:0/3117575348 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef2410c180 msgr2=0x7fef24101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:47.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.038+0000 7fef2894c640 1 --2- 172.21.15.1:0/3117575348 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef2410c180 0x7fef24101480 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fef1c00c970 tx=0x7fef1c00ce30 comp rx=0 tx=0).stop 2024-09-11T22:05:47.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.038+0000 7fef2894c640 1 -- 172.21.15.1:0/3117575348 shutdown_connections 2024-09-11T22:05:47.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.038+0000 7fef2894c640 1 --2- 172.21.15.1:0/3117575348 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fef100a2d70 0x7fef100a5230 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:47.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.038+0000 7fef2894c640 1 --2- 172.21.15.1:0/3117575348 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef2410c180 0x7fef24101480 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:47.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.038+0000 7fef2894c640 1 --2- 172.21.15.1:0/3117575348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fef241036c0 0x7fef24100f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:47.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.038+0000 7fef2894c640 1 --2- 172.21.15.1:0/3117575348 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef24102da0 0x7fef24100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:47.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.038+0000 7fef2894c640 1 -- 172.21.15.1:0/3117575348 >> 172.21.15.1:0/3117575348 conn(0x7fef240fc7a0 msgr2=0x7fef240fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:47.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.038+0000 7fef2894c640 1 -- 172.21.15.1:0/3117575348 shutdown_connections 2024-09-11T22:05:47.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:47.038+0000 7fef2894c640 1 -- 172.21.15.1:0/3117575348 wait complete. 2024-09-11T22:05:47.767 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:47 smithi001 ceph-mon[30935]: failed to store backtrace on ino 0x100000069d1 object, pool 2, errno -2 2024-09-11T22:05:47.767 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:47 smithi001 ceph-mon[30935]: force file system read-only 2024-09-11T22:05:47.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:47 smithi053 ceph-mon[34908]: failed to store backtrace on ino 0x100000069d1 object, pool 2, errno -2 2024-09-11T22:05:47.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:47 smithi053 ceph-mon[34908]: force file system read-only 2024-09-11T22:05:47.798 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T22:05:47.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:47 smithi161 ceph-mon[35615]: failed to store backtrace on ino 0x100000069d1 object, pool 2, errno -2 2024-09-11T22:05:47.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:47 smithi161 ceph-mon[35615]: force file system read-only 2024-09-11T22:05:48.059 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:05:48.689 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:48 smithi001 ceph-mon[30935]: pgmap v268: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-11T22:05:48.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.687+0000 7fec473e9640 1 -- 172.21.15.1:0/4196802441 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fec40074eb0 msgr2=0x7fec40112650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:48.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.687+0000 7fec473e9640 1 --2- 172.21.15.1:0/4196802441 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fec40074eb0 0x7fec40112650 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fec30009900 tx=0x7fec3002a480 comp rx=0 tx=0).stop 2024-09-11T22:05:48.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.687+0000 7fec473e9640 1 -- 172.21.15.1:0/4196802441 shutdown_connections 2024-09-11T22:05:48.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.687+0000 7fec473e9640 1 --2- 172.21.15.1:0/4196802441 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec40112b90 0x7fec40114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:48.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.687+0000 7fec473e9640 1 --2- 172.21.15.1:0/4196802441 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fec40074eb0 0x7fec40112650 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:48.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.687+0000 7fec473e9640 1 --2- 172.21.15.1:0/4196802441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec40074590 0x7fec40074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:48.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.687+0000 7fec473e9640 1 -- 172.21.15.1:0/4196802441 >> 172.21.15.1:0/4196802441 conn(0x7fec400fd8c0 msgr2=0x7fec400ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:48.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.687+0000 7fec473e9640 1 -- 172.21.15.1:0/4196802441 shutdown_connections 2024-09-11T22:05:48.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.687+0000 7fec473e9640 1 -- 172.21.15.1:0/4196802441 wait complete. 2024-09-11T22:05:48.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.687+0000 7fec473e9640 1 Processor -- start 2024-09-11T22:05:48.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.687+0000 7fec473e9640 1 -- start start 2024-09-11T22:05:48.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.688+0000 7fec473e9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fec40074590 0x7fec401a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:48.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.688+0000 7fec473e9640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec40074eb0 0x7fec401a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:48.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.688+0000 7fec473e9640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec40112b90 0x7fec4019e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:48.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.688+0000 7fec473e9640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fec40118540 con 0x7fec40074590 2024-09-11T22:05:48.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.688+0000 7fec473e9640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fec401183c0 con 0x7fec40074eb0 2024-09-11T22:05:48.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.688+0000 7fec473e9640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fec401186c0 con 0x7fec40112b90 2024-09-11T22:05:48.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.688+0000 7fec45185640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fec40074590 0x7fec401a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:48.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.688+0000 7fec45185640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fec40074590 0x7fec401a46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56584/0 (socket says 172.21.15.1:56584) 2024-09-11T22:05:48.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.688+0000 7fec45185640 1 -- 172.21.15.1:0/615526997 learned_addr learned my addr 172.21.15.1:0/615526997 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:48.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.688+0000 7fec45986640 1 --2- 172.21.15.1:0/615526997 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec40112b90 0x7fec4019e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:48.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.688+0000 7fec44984640 1 --2- 172.21.15.1:0/615526997 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec40074eb0 0x7fec401a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:48.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.688+0000 7fec45185640 1 -- 172.21.15.1:0/615526997 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec40112b90 msgr2=0x7fec4019e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:48.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.688+0000 7fec45185640 1 --2- 172.21.15.1:0/615526997 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec40112b90 0x7fec4019e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:48.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.688+0000 7fec45185640 1 -- 172.21.15.1:0/615526997 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec40074eb0 msgr2=0x7fec401a4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:48.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.688+0000 7fec45185640 1 --2- 172.21.15.1:0/615526997 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec40074eb0 0x7fec401a4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:48.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.688+0000 7fec45185640 1 -- 172.21.15.1:0/615526997 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec4019efc0 con 0x7fec40074590 2024-09-11T22:05:48.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.688+0000 7fec44984640 1 --2- 172.21.15.1:0/615526997 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec40074eb0 0x7fec401a4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:05:48.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.688+0000 7fec45185640 1 --2- 172.21.15.1:0/615526997 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fec40074590 0x7fec401a46d0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fec3c00c920 tx=0x7fec3c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:48.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.688+0000 7fec367fc640 1 -- 172.21.15.1:0/615526997 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3c007c70 con 0x7fec40074590 2024-09-11T22:05:48.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.689+0000 7fec367fc640 1 -- 172.21.15.1:0/615526997 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec3c007e10 con 0x7fec40074590 2024-09-11T22:05:48.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.689+0000 7fec473e9640 1 -- 172.21.15.1:0/615526997 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec4019f2b0 con 0x7fec40074590 2024-09-11T22:05:48.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.689+0000 7fec367fc640 1 -- 172.21.15.1:0/615526997 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3c018890 con 0x7fec40074590 2024-09-11T22:05:48.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.689+0000 7fec473e9640 1 -- 172.21.15.1:0/615526997 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec401ab5a0 con 0x7fec40074590 2024-09-11T22:05:48.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.691+0000 7fec367fc640 1 -- 172.21.15.1:0/615526997 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7fec3c018a70 con 0x7fec40074590 2024-09-11T22:05:48.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.691+0000 7fec497c2740 1 Processor -- start 2024-09-11T22:05:48.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.691+0000 7fec497c2740 1 -- start start 2024-09-11T22:05:48.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.692+0000 7fec367fc640 1 --2- 172.21.15.1:0/615526997 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fec2c0a2d70 0x7fec2c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:48.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.692+0000 7fec497c2740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b357e2cb40 0x55b357e2b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:48.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.692+0000 7fec497c2740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b357e2b5d0 0x55b357e2ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:48.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.692+0000 7fec497c2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b357e36370 0x55b357e38760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:48.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.692+0000 7fec497c2740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55b357d912f0 con 0x55b357e2b5d0 2024-09-11T22:05:48.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.692+0000 7fec497c2740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55b357d38cf0 con 0x55b357e2cb40 2024-09-11T22:05:48.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.692+0000 7fec497c2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b357d8d2b0 con 0x55b357e36370 2024-09-11T22:05:48.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.692+0000 7fec45986640 1 --2- 172.21.15.1:0/615526997 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fec2c0a2d70 0x7fec2c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:48.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.692+0000 7fec367fc640 1 -- 172.21.15.1:0/615526997 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7fec3c0cf660 con 0x7fec40074590 2024-09-11T22:05:48.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.692+0000 7fec44984640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b357e2b5d0 0x55b357e2ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:48.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.692+0000 7fec44984640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b357e2b5d0 0x55b357e2ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56586/0 (socket says 172.21.15.1:56586) 2024-09-11T22:05:48.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.692+0000 7fec44984640 1 -- 172.21.15.1:0/4079752430 learned_addr learned my addr 172.21.15.1:0/4079752430 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:48.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.692+0000 7fec45185640 1 --2- 172.21.15.1:0/4079752430 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b357e2cb40 0x55b357e2b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:48.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.692+0000 7fec45986640 1 --2- 172.21.15.1:0/4079752430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b357e36370 0x55b357e38760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:48.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.692+0000 7fec44984640 1 -- 172.21.15.1:0/4079752430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b357e36370 msgr2=0x55b357e38760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:48.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.692+0000 7fec44984640 1 --2- 172.21.15.1:0/4079752430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b357e36370 0x55b357e38760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:48.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.692+0000 7fec44984640 1 -- 172.21.15.1:0/4079752430 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b357e2cb40 msgr2=0x55b357e2b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:48.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.692+0000 7fec44984640 1 --2- 172.21.15.1:0/4079752430 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b357e2cb40 0x55b357e2b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:48.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.692+0000 7fec44984640 1 -- 172.21.15.1:0/4079752430 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b357d1d3a0 con 0x55b357e2b5d0 2024-09-11T22:05:48.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.692+0000 7fec45986640 1 --2- 172.21.15.1:0/615526997 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fec2c0a2d70 0x7fec2c0a5230 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fec380092f0 tx=0x7fec38014a70 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:48.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.692+0000 7fec44984640 1 --2- 172.21.15.1:0/4079752430 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b357e2b5d0 0x55b357e2ba10 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fec30005c40 tx=0x7fec30032cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71c47737ebcddb21 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:48.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.693+0000 7fec2b7fe640 1 -- 172.21.15.1:0/4079752430 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3003f070 con 0x55b357e2b5d0 2024-09-11T22:05:48.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.693+0000 7fec2b7fe640 1 -- 172.21.15.1:0/4079752430 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec300317b0 con 0x55b357e2b5d0 2024-09-11T22:05:48.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.693+0000 7fec2b7fe640 1 -- 172.21.15.1:0/4079752430 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec30035070 con 0x55b357e2b5d0 2024-09-11T22:05:48.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.693+0000 7fec497c2740 1 -- 172.21.15.1:0/4079752430 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b357e2b5d0 msgr2=0x55b357e2ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:48.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.693+0000 7fec497c2740 1 --2- 172.21.15.1:0/4079752430 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b357e2b5d0 0x55b357e2ba10 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fec30005c40 tx=0x7fec30032cb0 comp rx=0 tx=0).stop 2024-09-11T22:05:48.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.693+0000 7fec497c2740 1 -- 172.21.15.1:0/4079752430 shutdown_connections 2024-09-11T22:05:48.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.693+0000 7fec497c2740 1 --2- 172.21.15.1:0/4079752430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b357e36370 0x55b357e38760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:48.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.693+0000 7fec497c2740 1 --2- 172.21.15.1:0/4079752430 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b357e2b5d0 0x55b357e2ba10 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:48.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.693+0000 7fec497c2740 1 --2- 172.21.15.1:0/4079752430 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b357e2cb40 0x55b357e2b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:48.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.693+0000 7fec497c2740 1 -- 172.21.15.1:0/4079752430 >> 172.21.15.1:0/4079752430 conn(0x55b357df4310 msgr2=0x55b357e27c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:48.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.693+0000 7fec497c2740 1 -- 172.21.15.1:0/4079752430 shutdown_connections 2024-09-11T22:05:48.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.693+0000 7fec497c2740 1 -- 172.21.15.1:0/4079752430 wait complete. 2024-09-11T22:05:48.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.694+0000 7fec497c2740 1 -- start start 2024-09-11T22:05:48.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.694+0000 7fec497c2740 1 Processor -- start 2024-09-11T22:05:48.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.694+0000 7fec497c2740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b357e2b5d0 0x55b357e35ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:48.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.694+0000 7fec497c2740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b357e2cb40 0x55b357e32f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:48.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.694+0000 7fec497c2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b357e36370 0x55b357e334f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:48.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.694+0000 7fec497c2740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55b357e39010 con 0x55b357e2b5d0 2024-09-11T22:05:48.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.694+0000 7fec497c2740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55b357d4c060 con 0x55b357e2cb40 2024-09-11T22:05:48.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.694+0000 7fec497c2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b357e39190 con 0x55b357e36370 2024-09-11T22:05:48.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.694+0000 7fec497c2740 10 client.0 _ll_register_callbacks cb 0x55b357a9f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:05:48.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.695+0000 7fec45185640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b357e2b5d0 0x55b357e35ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:48.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.695+0000 7fec45185640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b357e2b5d0 0x55b357e35ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56590/0 (socket says 172.21.15.1:56590) 2024-09-11T22:05:48.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.695+0000 7fec45185640 1 -- 172.21.15.1:0/3427909352 learned_addr learned my addr 172.21.15.1:0/3427909352 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:48.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.695+0000 7fec45986640 1 --2- 172.21.15.1:0/3427909352 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b357e36370 0x55b357e334f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:48.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.695+0000 7fec44984640 1 --2- 172.21.15.1:0/3427909352 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b357e2cb40 0x55b357e32f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:48.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.695+0000 7fec45185640 1 -- 172.21.15.1:0/3427909352 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b357e36370 msgr2=0x55b357e334f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:48.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.695+0000 7fec45185640 1 --2- 172.21.15.1:0/3427909352 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b357e36370 0x55b357e334f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:48.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.695+0000 7fec45185640 1 -- 172.21.15.1:0/3427909352 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b357e2cb40 msgr2=0x55b357e32f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:48.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.695+0000 7fec45185640 1 --2- 172.21.15.1:0/3427909352 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b357e2cb40 0x55b357e32f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:48.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.695+0000 7fec45185640 1 -- 172.21.15.1:0/3427909352 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b357b023c0 con 0x55b357e2b5d0 2024-09-11T22:05:48.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.695+0000 7fec44984640 1 --2- 172.21.15.1:0/3427909352 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b357e2cb40 0x55b357e32f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:05:48.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.695+0000 7fec45185640 1 --2- 172.21.15.1:0/3427909352 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b357e2b5d0 0x55b357e35ce0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fec3c097fa0 tx=0x7fec3c09b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=837b229a844f6292 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:48.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.695+0000 7fec28ff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:05:48.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.695+0000 7fec28ff9640 1 -- 172.21.15.1:0/3427909352 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3c0a47b0 con 0x55b357e2b5d0 2024-09-11T22:05:48.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.695+0000 7fec28ff9640 1 -- 172.21.15.1:0/3427909352 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7fec3c018d60 con 0x55b357e2b5d0 2024-09-11T22:05:48.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.695+0000 7fec497c2740 1 -- 172.21.15.1:0/3427909352 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec300317b0 con 0x55b357e2b5d0 2024-09-11T22:05:48.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.695+0000 7fec28ff9640 1 -- 172.21.15.1:0/3427909352 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fec3c00fb30 con 0x55b357e2b5d0 2024-09-11T22:05:48.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.695+0000 7fec497c2740 10 client.17484 fetch_fsmap learned FSMap version 45 2024-09-11T22:05:48.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.695+0000 7fec497c2740 1 -- 172.21.15.1:0/3427909352 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55b357e44ab0 con 0x55b357e2b5d0 2024-09-11T22:05:48.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.696+0000 7fec28ff9640 1 -- 172.21.15.1:0/3427909352 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec3c0b4080 con 0x55b357e2b5d0 2024-09-11T22:05:48.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.696+0000 7fec28ff9640 1 -- 172.21.15.1:0/3427909352 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 45) ==== 8442+0+0 (secure 0 0 0) 0x7fec3c0a4320 con 0x55b357e2b5d0 2024-09-11T22:05:48.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.696+0000 7fec497c2740 10 client.17484 fetch_fsmap finished waiting for FSMap version 45 2024-09-11T22:05:48.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.696+0000 7fec497c2740 10 client.17484 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T22:05:48.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.696+0000 7fec497c2740 20 client.17484 populate_metadata read hostname 'smithi001' 2024-09-11T22:05:48.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.696+0000 7fec497c2740 1 --2- 172.21.15.1:0/3427909352 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x55b357e45f10 0x55b357e48360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:48.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.696+0000 7fec497c2740 4 client.17484 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:05:48.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.696+0000 7fec497c2740 1 -- 172.21.15.1:0/3427909352 --> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b357d9e860 con 0x55b357e45f10 2024-09-11T22:05:48.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.697+0000 7fec44984640 1 --2- 172.21.15.1:0/3427909352 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x55b357e45f10 0x55b357e48360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:48.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.699+0000 7fec44984640 1 --2- 172.21.15.1:0/3427909352 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x55b357e45f10 0x55b357e48360 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8fce6cd833605d7d server_cookie=5525d23a77841259 in_seq=0 out_seq=0 2024-09-11T22:05:48.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.699+0000 7fec28ff9640 10 client.17484 ms_handle_connect on v2:172.21.15.161:6838/455365383 2024-09-11T22:05:48.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.700+0000 7fec28ff9640 1 -- 172.21.15.1:0/3427909352 <== mds.1 v2:172.21.15.161:6838/455365383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b357d9e860 con 0x55b357e45f10 2024-09-11T22:05:48.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.700+0000 7fec28ff9640 10 client.17484 handle_command_reply: tid=0 2024-09-11T22:05:48.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.700+0000 7fec497c2740 1 client.17484 shutdown 2024-09-11T22:05:48.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.700+0000 7fec497c2740 1 -- 172.21.15.1:0/3427909352 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b357e2b5d0 msgr2=0x55b357e35ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:48.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.700+0000 7fec497c2740 1 --2- 172.21.15.1:0/3427909352 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b357e2b5d0 0x55b357e35ce0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fec3c097fa0 tx=0x7fec3c09b780 comp rx=0 tx=0).stop 2024-09-11T22:05:48.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.700+0000 7fec497c2740 1 -- 172.21.15.1:0/3427909352 shutdown_connections 2024-09-11T22:05:48.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.700+0000 7fec497c2740 1 --2- 172.21.15.1:0/3427909352 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x55b357e45f10 0x55b357e48360 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:48.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.700+0000 7fec497c2740 1 --2- 172.21.15.1:0/3427909352 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b357e36370 0x55b357e334f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:48.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.700+0000 7fec497c2740 1 --2- 172.21.15.1:0/3427909352 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b357e2cb40 0x55b357e32f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:48.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.700+0000 7fec28ff9640 0 client.17484 ms_handle_reset on v2:172.21.15.161:6838/455365383 2024-09-11T22:05:48.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.700+0000 7fec497c2740 1 --2- 172.21.15.1:0/3427909352 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b357e2b5d0 0x55b357e35ce0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:48.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.700+0000 7fec497c2740 1 -- 172.21.15.1:0/3427909352 >> 172.21.15.1:0/3427909352 conn(0x55b357df4310 msgr2=0x55b357e2d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:48.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.700+0000 7fec497c2740 1 -- 172.21.15.1:0/3427909352 shutdown_connections 2024-09-11T22:05:48.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.700+0000 7fec497c2740 1 -- 172.21.15.1:0/3427909352 wait complete. 2024-09-11T22:05:48.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.700+0000 7fec497c2740 20 client.17484 trim_cache size 0 max 16384 2024-09-11T22:05:48.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.713+0000 7fec497c2740 1 Processor -- start 2024-09-11T22:05:48.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.713+0000 7fec497c2740 1 -- start start 2024-09-11T22:05:48.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.713+0000 7fec497c2740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b357e2b5d0 0x55b357e2c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:48.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.713+0000 7fec497c2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b357e2cb40 0x55b357e26bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:48.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.713+0000 7fec497c2740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b357e36370 0x55b357e2e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:48.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.713+0000 7fec497c2740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fec401b3fd0 con 0x55b357e36370 2024-09-11T22:05:48.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.713+0000 7fec497c2740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fec401b5e10 con 0x55b357e2b5d0 2024-09-11T22:05:48.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.713+0000 7fec497c2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fec401b4760 con 0x55b357e2cb40 2024-09-11T22:05:48.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.714+0000 7fec45986640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b357e36370 0x55b357e2e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:48.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.714+0000 7fec45986640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b357e36370 0x55b357e2e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56594/0 (socket says 172.21.15.1:56594) 2024-09-11T22:05:48.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.714+0000 7fec45986640 1 -- 172.21.15.1:0/1755083303 learned_addr learned my addr 172.21.15.1:0/1755083303 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:48.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.714+0000 7fec45185640 1 --2- 172.21.15.1:0/1755083303 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b357e2b5d0 0x55b357e2c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:48.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.714+0000 7fec44984640 1 --2- 172.21.15.1:0/1755083303 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b357e2cb40 0x55b357e26bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:48.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.714+0000 7fec45986640 1 -- 172.21.15.1:0/1755083303 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b357e2cb40 msgr2=0x55b357e26bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:48.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.714+0000 7fec45986640 1 --2- 172.21.15.1:0/1755083303 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b357e2cb40 0x55b357e26bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:48.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.714+0000 7fec45986640 1 -- 172.21.15.1:0/1755083303 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b357e2b5d0 msgr2=0x55b357e2c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:48.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.714+0000 7fec45986640 1 --2- 172.21.15.1:0/1755083303 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b357e2b5d0 0x55b357e2c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:48.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.714+0000 7fec45986640 1 -- 172.21.15.1:0/1755083303 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b357e314e0 con 0x55b357e36370 2024-09-11T22:05:48.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.714+0000 7fec45986640 1 --2- 172.21.15.1:0/1755083303 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b357e36370 0x55b357e2e120 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fec38014380 tx=0x7fec38009280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3a2482e77dc74fc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:48.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.714+0000 7fec2affd640 1 -- 172.21.15.1:0/1755083303 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec38004d20 con 0x55b357e36370 2024-09-11T22:05:48.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.714+0000 7fec2affd640 1 -- 172.21.15.1:0/1755083303 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec38013a60 con 0x55b357e36370 2024-09-11T22:05:48.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.714+0000 7fec2affd640 1 -- 172.21.15.1:0/1755083303 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec38005760 con 0x55b357e36370 2024-09-11T22:05:48.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.715+0000 7fec497c2740 1 -- 172.21.15.1:0/1755083303 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b357e36370 msgr2=0x55b357e2e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:48.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.715+0000 7fec497c2740 1 --2- 172.21.15.1:0/1755083303 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b357e36370 0x55b357e2e120 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fec38014380 tx=0x7fec38009280 comp rx=0 tx=0).stop 2024-09-11T22:05:48.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.715+0000 7fec497c2740 1 -- 172.21.15.1:0/1755083303 shutdown_connections 2024-09-11T22:05:48.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.715+0000 7fec497c2740 1 --2- 172.21.15.1:0/1755083303 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b357e36370 0x55b357e2e120 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:48.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.715+0000 7fec497c2740 1 --2- 172.21.15.1:0/1755083303 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b357e2cb40 0x55b357e26bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:48.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.715+0000 7fec497c2740 1 --2- 172.21.15.1:0/1755083303 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b357e2b5d0 0x55b357e2c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:48.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.715+0000 7fec497c2740 1 -- 172.21.15.1:0/1755083303 >> 172.21.15.1:0/1755083303 conn(0x55b357df4310 msgr2=0x55b357e264e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:48.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.715+0000 7fec497c2740 1 -- 172.21.15.1:0/1755083303 shutdown_connections 2024-09-11T22:05:48.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.715+0000 7fec497c2740 1 -- 172.21.15.1:0/1755083303 wait complete. 2024-09-11T22:05:48.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.715+0000 7fec497c2740 1 -- start start 2024-09-11T22:05:48.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.715+0000 7fec497c2740 1 Processor -- start 2024-09-11T22:05:48.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.716+0000 7fec497c2740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b357e2b5d0 0x55b357e3ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:48.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.716+0000 7fec497c2740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b357e2cb40 0x55b357e2fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:48.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.716+0000 7fec497c2740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b357e36370 0x55b357e302f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:48.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.716+0000 7fec497c2740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fec401b5680 con 0x55b357e2b5d0 2024-09-11T22:05:48.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.716+0000 7fec497c2740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fec401b4ef0 con 0x55b357e36370 2024-09-11T22:05:48.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.716+0000 7fec497c2740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fec401b1fb0 con 0x55b357e2cb40 2024-09-11T22:05:48.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.716+0000 7fec497c2740 10 client.0 _ll_register_callbacks cb 0x7fec3c00c870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:05:48.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.716+0000 7fec45185640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b357e2b5d0 0x55b357e3ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:48.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.716+0000 7fec45185640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b357e2b5d0 0x55b357e3ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56604/0 (socket says 172.21.15.1:56604) 2024-09-11T22:05:48.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.716+0000 7fec45185640 1 -- 172.21.15.1:0/710343753 learned_addr learned my addr 172.21.15.1:0/710343753 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:48.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.716+0000 7fec45986640 1 --2- 172.21.15.1:0/710343753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b357e36370 0x55b357e302f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:48.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.716+0000 7fec44984640 1 --2- 172.21.15.1:0/710343753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b357e2cb40 0x55b357e2fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:48.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.716+0000 7fec45185640 1 -- 172.21.15.1:0/710343753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b357e2cb40 msgr2=0x55b357e2fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:48.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.716+0000 7fec45185640 1 --2- 172.21.15.1:0/710343753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b357e2cb40 0x55b357e2fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:48.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.716+0000 7fec45185640 1 -- 172.21.15.1:0/710343753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b357e36370 msgr2=0x55b357e302f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:48.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.716+0000 7fec45185640 1 --2- 172.21.15.1:0/710343753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b357e36370 0x55b357e302f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:48.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.716+0000 7fec45185640 1 -- 172.21.15.1:0/710343753 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b357e30b20 con 0x55b357e2b5d0 2024-09-11T22:05:48.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.716+0000 7fec45185640 1 --2- 172.21.15.1:0/710343753 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b357e2b5d0 0x55b357e3ec30 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fec3c09b7f0 tx=0x7fec3c00fcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c82b917d280e62f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:48.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.716+0000 7fec28ff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:05:48.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.716+0000 7fec28ff9640 1 -- 172.21.15.1:0/710343753 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3c0a47b0 con 0x55b357e2b5d0 2024-09-11T22:05:48.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.717+0000 7fec28ff9640 1 -- 172.21.15.1:0/710343753 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7fec3c0ba390 con 0x55b357e2b5d0 2024-09-11T22:05:48.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.717+0000 7fec497c2740 1 -- 172.21.15.1:0/710343753 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec38013a60 con 0x55b357e2b5d0 2024-09-11T22:05:48.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.717+0000 7fec28ff9640 1 -- 172.21.15.1:0/710343753 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fec3c09c240 con 0x55b357e2b5d0 2024-09-11T22:05:48.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.717+0000 7fec497c2740 10 client.17490 fetch_fsmap learned FSMap version 45 2024-09-11T22:05:48.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.717+0000 7fec497c2740 1 -- 172.21.15.1:0/710343753 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55b357eb0c90 con 0x55b357e2b5d0 2024-09-11T22:05:48.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.717+0000 7fec28ff9640 1 -- 172.21.15.1:0/710343753 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec38013a60 con 0x55b357e2b5d0 2024-09-11T22:05:48.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.717+0000 7fec28ff9640 1 -- 172.21.15.1:0/710343753 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 45) ==== 8442+0+0 (secure 0 0 0) 0x7fec3c09af90 con 0x55b357e2b5d0 2024-09-11T22:05:48.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.717+0000 7fec497c2740 10 client.17490 fetch_fsmap finished waiting for FSMap version 45 2024-09-11T22:05:48.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.717+0000 7fec497c2740 10 client.17490 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T22:05:48.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.717+0000 7fec497c2740 20 client.17490 populate_metadata read hostname 'smithi001' 2024-09-11T22:05:48.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.717+0000 7fec497c2740 1 --2- 172.21.15.1:0/710343753 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x55b357eb1260 0x55b357eb3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:48.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.717+0000 7fec497c2740 4 client.17490 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:05:48.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.717+0000 7fec497c2740 1 -- 172.21.15.1:0/710343753 --> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b357d98d70 con 0x55b357eb1260 2024-09-11T22:05:48.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.717+0000 7fec44984640 1 --2- 172.21.15.1:0/710343753 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x55b357eb1260 0x55b357eb3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:48.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.718+0000 7fec44984640 1 --2- 172.21.15.1:0/710343753 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x55b357eb1260 0x55b357eb3680 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e4af6cfedf7f79c6 server_cookie=fe5e803be2f2d78 in_seq=0 out_seq=0 2024-09-11T22:05:48.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.718+0000 7fec28ff9640 10 client.17490 ms_handle_connect on v2:172.21.15.161:6838/455365383 2024-09-11T22:05:48.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.718+0000 7fec28ff9640 1 -- 172.21.15.1:0/710343753 <== mds.1 v2:172.21.15.161:6838/455365383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b357d98d70 con 0x55b357eb1260 2024-09-11T22:05:48.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.718+0000 7fec28ff9640 10 client.17490 handle_command_reply: tid=0 2024-09-11T22:05:48.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.718+0000 7fec497c2740 1 client.17490 shutdown 2024-09-11T22:05:48.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.718+0000 7fec497c2740 1 -- 172.21.15.1:0/710343753 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b357e2b5d0 msgr2=0x55b357e3ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:48.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.718+0000 7fec497c2740 1 --2- 172.21.15.1:0/710343753 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b357e2b5d0 0x55b357e3ec30 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fec3c09b7f0 tx=0x7fec3c00fcd0 comp rx=0 tx=0).stop 2024-09-11T22:05:48.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.719+0000 7fec497c2740 1 -- 172.21.15.1:0/710343753 shutdown_connections 2024-09-11T22:05:48.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.719+0000 7fec497c2740 1 --2- 172.21.15.1:0/710343753 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x55b357eb1260 0x55b357eb3680 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:48.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.719+0000 7fec497c2740 1 --2- 172.21.15.1:0/710343753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b357e36370 0x55b357e302f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:48.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.719+0000 7fec497c2740 1 --2- 172.21.15.1:0/710343753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b357e2cb40 0x55b357e2fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:48.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.719+0000 7fec497c2740 1 --2- 172.21.15.1:0/710343753 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b357e2b5d0 0x55b357e3ec30 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:48.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.719+0000 7fec497c2740 1 -- 172.21.15.1:0/710343753 >> 172.21.15.1:0/710343753 conn(0x55b357df4310 msgr2=0x55b357e26d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:48.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.719+0000 7fec28ff9640 0 client.17490 ms_handle_reset on v2:172.21.15.161:6838/455365383 2024-09-11T22:05:48.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.719+0000 7fec497c2740 1 -- 172.21.15.1:0/710343753 shutdown_connections 2024-09-11T22:05:48.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.719+0000 7fec497c2740 1 -- 172.21.15.1:0/710343753 wait complete. 2024-09-11T22:05:48.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.719+0000 7fec497c2740 20 client.17490 trim_cache size 0 max 16384 2024-09-11T22:05:48.742 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-11T22:05:48.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.719+0000 7fec473e9640 1 -- 172.21.15.1:0/615526997 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fec2c0a2d70 msgr2=0x7fec2c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:48.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.719+0000 7fec473e9640 1 --2- 172.21.15.1:0/615526997 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fec2c0a2d70 0x7fec2c0a5230 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fec380092f0 tx=0x7fec38014a70 comp rx=0 tx=0).stop 2024-09-11T22:05:48.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.719+0000 7fec473e9640 1 -- 172.21.15.1:0/615526997 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fec40074590 msgr2=0x7fec401a46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:48.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.719+0000 7fec473e9640 1 --2- 172.21.15.1:0/615526997 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fec40074590 0x7fec401a46d0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fec3c00c920 tx=0x7fec3c00cde0 comp rx=0 tx=0).stop 2024-09-11T22:05:48.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.719+0000 7fec473e9640 1 -- 172.21.15.1:0/615526997 shutdown_connections 2024-09-11T22:05:48.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.720+0000 7fec473e9640 1 --2- 172.21.15.1:0/615526997 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fec2c0a2d70 0x7fec2c0a5230 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:48.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.720+0000 7fec473e9640 1 --2- 172.21.15.1:0/615526997 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec40112b90 0x7fec4019e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:48.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.720+0000 7fec473e9640 1 --2- 172.21.15.1:0/615526997 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fec40074eb0 0x7fec401a4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:48.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.720+0000 7fec473e9640 1 --2- 172.21.15.1:0/615526997 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fec40074590 0x7fec401a46d0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:48.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.720+0000 7fec473e9640 1 -- 172.21.15.1:0/615526997 >> 172.21.15.1:0/615526997 conn(0x7fec400fd8c0 msgr2=0x7fec400ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:48.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.720+0000 7fec473e9640 1 -- 172.21.15.1:0/615526997 shutdown_connections 2024-09-11T22:05:48.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:48.720+0000 7fec473e9640 1 -- 172.21.15.1:0/615526997 wait complete. 2024-09-11T22:05:48.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:48 smithi053 ceph-mon[34908]: pgmap v268: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-11T22:05:48.842 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:48 smithi161 ceph-mon[35615]: pgmap v268: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-09-11T22:05:49.651 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T22:05:49.923 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:05:50.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.472+0000 7f7ba0f43640 1 -- 172.21.15.1:0/2811813898 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b9c106a90 msgr2=0x7f7b9c106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:50.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.472+0000 7f7ba0f43640 1 --2- 172.21.15.1:0/2811813898 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b9c106a90 0x7f7b9c106e70 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c009930 tx=0x7f7b8c02a420 comp rx=0 tx=0).stop 2024-09-11T22:05:50.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.472+0000 7f7ba0f43640 1 -- 172.21.15.1:0/2811813898 shutdown_connections 2024-09-11T22:05:50.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.472+0000 7f7ba0f43640 1 --2- 172.21.15.1:0/2811813898 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b9c101430 0x7f7b9c10d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:50.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.472+0000 7f7ba0f43640 1 --2- 172.21.15.1:0/2811813898 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b9c100a90 0x7f7b9c100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:50.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.472+0000 7f7ba0f43640 1 --2- 172.21.15.1:0/2811813898 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b9c106a90 0x7f7b9c106e70 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:50.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.472+0000 7f7ba0f43640 1 -- 172.21.15.1:0/2811813898 >> 172.21.15.1:0/2811813898 conn(0x7f7b9c0fc7d0 msgr2=0x7f7b9c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:50.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.472+0000 7f7ba0f43640 1 -- 172.21.15.1:0/2811813898 shutdown_connections 2024-09-11T22:05:50.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.472+0000 7f7ba0f43640 1 -- 172.21.15.1:0/2811813898 wait complete. 2024-09-11T22:05:50.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.473+0000 7f7ba0f43640 1 Processor -- start 2024-09-11T22:05:50.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.473+0000 7f7ba0f43640 1 -- start start 2024-09-11T22:05:50.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.474+0000 7f7ba0f43640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b9c100a90 0x7f7b9c199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:50.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.474+0000 7f7ba0f43640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b9c101430 0x7f7b9c1997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:50.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.474+0000 7f7ba0f43640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b9c106a90 0x7f7b9c1a4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:50.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.474+0000 7f7ba0f43640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9c110e90 con 0x7f7b9c106a90 2024-09-11T22:05:50.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.474+0000 7f7ba0f43640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9c110d10 con 0x7f7b9c100a90 2024-09-11T22:05:50.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.474+0000 7f7ba0f43640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9c111010 con 0x7f7b9c101430 2024-09-11T22:05:50.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.474+0000 7f7b9ad9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b9c106a90 0x7f7b9c1a4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:50.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.474+0000 7f7b9ad9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b9c106a90 0x7f7b9c1a4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56626/0 (socket says 172.21.15.1:56626) 2024-09-11T22:05:50.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.474+0000 7f7b9ad9d640 1 -- 172.21.15.1:0/35825327 learned_addr learned my addr 172.21.15.1:0/35825327 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:50.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.474+0000 7f7b9a59c640 1 --2- 172.21.15.1:0/35825327 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b9c100a90 0x7f7b9c199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:50.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.474+0000 7f7b99d9b640 1 --2- 172.21.15.1:0/35825327 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b9c101430 0x7f7b9c1997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:50.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.474+0000 7f7b9ad9d640 1 -- 172.21.15.1:0/35825327 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b9c101430 msgr2=0x7f7b9c1997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:50.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.474+0000 7f7b9ad9d640 1 --2- 172.21.15.1:0/35825327 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b9c101430 0x7f7b9c1997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:50.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.474+0000 7f7b9ad9d640 1 -- 172.21.15.1:0/35825327 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b9c100a90 msgr2=0x7f7b9c199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:50.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.474+0000 7f7b9ad9d640 1 --2- 172.21.15.1:0/35825327 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b9c100a90 0x7f7b9c199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:50.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.474+0000 7f7b9ad9d640 1 -- 172.21.15.1:0/35825327 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b9c1a4850 con 0x7f7b9c106a90 2024-09-11T22:05:50.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.474+0000 7f7b9ad9d640 1 --2- 172.21.15.1:0/35825327 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b9c106a90 0x7f7b9c1a4310 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f7b9400eaa0 tx=0x7f7b9400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:50.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.474+0000 7f7b8b7fe640 1 -- 172.21.15.1:0/35825327 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b9400cdb0 con 0x7f7b9c106a90 2024-09-11T22:05:50.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.474+0000 7f7b8b7fe640 1 -- 172.21.15.1:0/35825327 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b94005020 con 0x7f7b9c106a90 2024-09-11T22:05:50.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.474+0000 7f7ba0f43640 1 -- 172.21.15.1:0/35825327 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b9c1a4a20 con 0x7f7b9c106a90 2024-09-11T22:05:50.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.474+0000 7f7b8b7fe640 1 -- 172.21.15.1:0/35825327 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b940116b0 con 0x7f7b9c106a90 2024-09-11T22:05:50.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.475+0000 7f7ba0f43640 1 -- 172.21.15.1:0/35825327 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b9c1a4f60 con 0x7f7b9c106a90 2024-09-11T22:05:50.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.477+0000 7f7b8b7fe640 1 -- 172.21.15.1:0/35825327 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f7b94005560 con 0x7f7b9c106a90 2024-09-11T22:05:50.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.477+0000 7f7ba331c740 1 Processor -- start 2024-09-11T22:05:50.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.477+0000 7f7ba331c740 1 -- start start 2024-09-11T22:05:50.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.477+0000 7f7ba331c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5617eb5b5b40 0x5617eb5b4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:50.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.477+0000 7f7ba331c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617eb5b45d0 0x5617eb5b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:50.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.477+0000 7f7ba331c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617eb5bf370 0x5617eb5c1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:50.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.477+0000 7f7ba331c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5617eb51a2f0 con 0x5617eb5b5b40 2024-09-11T22:05:50.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.477+0000 7f7ba331c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5617eb4c1cf0 con 0x5617eb5b45d0 2024-09-11T22:05:50.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.477+0000 7f7ba331c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5617eb5162b0 con 0x5617eb5bf370 2024-09-11T22:05:50.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.478+0000 7f7b99d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5617eb5b5b40 0x5617eb5b4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:50.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.478+0000 7f7b99d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5617eb5b5b40 0x5617eb5b4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56628/0 (socket says 172.21.15.1:56628) 2024-09-11T22:05:50.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.478+0000 7f7b99d9b640 1 -- 172.21.15.1:0/3785337305 learned_addr learned my addr 172.21.15.1:0/3785337305 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:50.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.478+0000 7f7b99d9b640 1 --2- 172.21.15.1:0/3785337305 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617eb5bf370 0x5617eb5c1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:50.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.478+0000 7f7b8b7fe640 1 --2- 172.21.15.1:0/35825327 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f7b840a2d70 0x7f7b840a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:50.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.478+0000 7f7b9a59c640 1 --2- 172.21.15.1:0/3785337305 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617eb5b45d0 0x5617eb5b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:50.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.478+0000 7f7b99d9b640 1 -- 172.21.15.1:0/3785337305 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617eb5bf370 msgr2=0x5617eb5c1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:50.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.478+0000 7f7b99d9b640 1 --2- 172.21.15.1:0/3785337305 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617eb5bf370 0x5617eb5c1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:50.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.478+0000 7f7b99d9b640 1 -- 172.21.15.1:0/3785337305 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617eb5b45d0 msgr2=0x5617eb5b4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:50.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.478+0000 7f7b99d9b640 1 --2- 172.21.15.1:0/3785337305 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617eb5b45d0 0x5617eb5b4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:50.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.478+0000 7f7b99d9b640 1 -- 172.21.15.1:0/3785337305 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617eb4a63a0 con 0x5617eb5b5b40 2024-09-11T22:05:50.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.478+0000 7f7b9ad9d640 1 --2- 172.21.15.1:0/35825327 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f7b840a2d70 0x7f7b840a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:50.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.478+0000 7f7b8b7fe640 1 -- 172.21.15.1:0/35825327 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7f7b940cf520 con 0x7f7b9c106a90 2024-09-11T22:05:50.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.478+0000 7f7b99d9b640 1 --2- 172.21.15.1:0/3785337305 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5617eb5b5b40 0x5617eb5b4090 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f7b900058a0 tx=0x7f7b9000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9370091cf33fd71 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:50.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.478+0000 7f7b88ff9640 1 -- 172.21.15.1:0/3785337305 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b90004030 con 0x5617eb5b5b40 2024-09-11T22:05:50.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.478+0000 7f7b88ff9640 1 -- 172.21.15.1:0/3785337305 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b900041d0 con 0x5617eb5b5b40 2024-09-11T22:05:50.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.478+0000 7f7b88ff9640 1 -- 172.21.15.1:0/3785337305 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b900106c0 con 0x5617eb5b5b40 2024-09-11T22:05:50.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.479+0000 7f7ba331c740 1 -- 172.21.15.1:0/3785337305 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5617eb5b5b40 msgr2=0x5617eb5b4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:50.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.479+0000 7f7b9ad9d640 1 --2- 172.21.15.1:0/35825327 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f7b840a2d70 0x7f7b840a5230 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7b940ccce0 tx=0x7f7b9406a9e0 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:50.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.479+0000 7f7ba331c740 1 --2- 172.21.15.1:0/3785337305 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5617eb5b5b40 0x5617eb5b4090 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f7b900058a0 tx=0x7f7b9000b840 comp rx=0 tx=0).stop 2024-09-11T22:05:50.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.479+0000 7f7ba331c740 1 -- 172.21.15.1:0/3785337305 shutdown_connections 2024-09-11T22:05:50.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.479+0000 7f7ba331c740 1 --2- 172.21.15.1:0/3785337305 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617eb5bf370 0x5617eb5c1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:50.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.479+0000 7f7ba331c740 1 --2- 172.21.15.1:0/3785337305 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617eb5b45d0 0x5617eb5b4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:50.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.479+0000 7f7ba331c740 1 --2- 172.21.15.1:0/3785337305 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5617eb5b5b40 0x5617eb5b4090 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:50.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.479+0000 7f7ba331c740 1 -- 172.21.15.1:0/3785337305 >> 172.21.15.1:0/3785337305 conn(0x5617eb57d310 msgr2=0x5617eb5b0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:50.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.479+0000 7f7ba331c740 1 -- 172.21.15.1:0/3785337305 shutdown_connections 2024-09-11T22:05:50.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.479+0000 7f7ba331c740 1 -- 172.21.15.1:0/3785337305 wait complete. 2024-09-11T22:05:50.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.480+0000 7f7ba331c740 1 -- start start 2024-09-11T22:05:50.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.480+0000 7f7ba331c740 1 Processor -- start 2024-09-11T22:05:50.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.480+0000 7f7ba331c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617eb5b45d0 0x5617eb5beda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:50.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.480+0000 7f7ba331c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617eb5b5b40 0x5617eb5bc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:50.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.480+0000 7f7ba331c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5617eb5bf370 0x5617eb5bc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:50.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.480+0000 7f7ba331c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5617eb5c2070 con 0x5617eb5bf370 2024-09-11T22:05:50.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.480+0000 7f7ba331c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5617eb4d5060 con 0x5617eb5b5b40 2024-09-11T22:05:50.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.480+0000 7f7ba331c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5617eb5c21f0 con 0x5617eb5b45d0 2024-09-11T22:05:50.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.480+0000 7f7ba331c740 10 client.0 _ll_register_callbacks cb 0x5617eb2288c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:05:50.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.480+0000 7f7b99d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5617eb5bf370 0x5617eb5bc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:50.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.480+0000 7f7b99d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5617eb5bf370 0x5617eb5bc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56632/0 (socket says 172.21.15.1:56632) 2024-09-11T22:05:50.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.480+0000 7f7b9a59c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617eb5b5b40 0x5617eb5bc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:50.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.480+0000 7f7b99d9b640 1 -- 172.21.15.1:0/3066116658 learned_addr learned my addr 172.21.15.1:0/3066116658 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:50.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.480+0000 7f7b99d9b640 1 --2- 172.21.15.1:0/3066116658 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617eb5b45d0 0x5617eb5beda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:50.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.480+0000 7f7b99d9b640 1 -- 172.21.15.1:0/3066116658 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617eb5b45d0 msgr2=0x5617eb5beda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:50.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.480+0000 7f7b99d9b640 1 --2- 172.21.15.1:0/3066116658 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617eb5b45d0 0x5617eb5beda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:50.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.480+0000 7f7b99d9b640 1 -- 172.21.15.1:0/3066116658 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617eb5b5b40 msgr2=0x5617eb5bc050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:50.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.480+0000 7f7b99d9b640 1 --2- 172.21.15.1:0/3066116658 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617eb5b5b40 0x5617eb5bc050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:50.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.480+0000 7f7b99d9b640 1 -- 172.21.15.1:0/3066116658 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617eb28b3c0 con 0x5617eb5bf370 2024-09-11T22:05:50.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.480+0000 7f7b9a59c640 1 --2- 172.21.15.1:0/3066116658 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617eb5b5b40 0x5617eb5bc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:05:50.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.481+0000 7f7b99d9b640 1 --2- 172.21.15.1:0/3066116658 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5617eb5bf370 0x5617eb5bc7e0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f7b9000bec0 tx=0x7f7b90003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f0e6d33fc0de099 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:50.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.481+0000 7f7b81ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:05:50.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.481+0000 7f7b81ffb640 1 -- 172.21.15.1:0/3066116658 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b90023060 con 0x5617eb5bf370 2024-09-11T22:05:50.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.481+0000 7f7b81ffb640 1 -- 172.21.15.1:0/3066116658 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7f7b900035f0 con 0x5617eb5bf370 2024-09-11T22:05:50.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.481+0000 7f7ba331c740 1 -- 172.21.15.1:0/3066116658 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b900041d0 con 0x5617eb5bf370 2024-09-11T22:05:50.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.481+0000 7f7b81ffb640 1 -- 172.21.15.1:0/3066116658 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f7b900037f0 con 0x5617eb5bf370 2024-09-11T22:05:50.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.481+0000 7f7ba331c740 10 client.17502 fetch_fsmap learned FSMap version 45 2024-09-11T22:05:50.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.481+0000 7f7ba331c740 1 -- 172.21.15.1:0/3066116658 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5617eb5cdb50 con 0x5617eb5bf370 2024-09-11T22:05:50.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.481+0000 7f7b81ffb640 1 -- 172.21.15.1:0/3066116658 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b900041d0 con 0x5617eb5bf370 2024-09-11T22:05:50.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.481+0000 7f7b81ffb640 1 -- 172.21.15.1:0/3066116658 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 45) ==== 8442+0+0 (secure 0 0 0) 0x7f7b900047c0 con 0x5617eb5bf370 2024-09-11T22:05:50.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.481+0000 7f7ba331c740 10 client.17502 fetch_fsmap finished waiting for FSMap version 45 2024-09-11T22:05:50.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.481+0000 7f7ba331c740 10 client.17502 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:05:50.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.481+0000 7f7ba331c740 20 client.17502 populate_metadata read hostname 'smithi001' 2024-09-11T22:05:50.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.481+0000 7f7ba331c740 1 --2- 172.21.15.1:0/3066116658 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5617eb5ceea0 0x5617eb5d1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:50.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.481+0000 7f7ba331c740 4 client.17502 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:05:50.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.481+0000 7f7ba331c740 1 -- 172.21.15.1:0/3066116658 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5617eb527860 con 0x5617eb5ceea0 2024-09-11T22:05:50.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.482+0000 7f7b9a59c640 1 --2- 172.21.15.1:0/3066116658 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5617eb5ceea0 0x5617eb5d1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:50.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.482+0000 7f7b9a59c640 1 --2- 172.21.15.1:0/3066116658 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5617eb5ceea0 0x5617eb5d1290 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75be8ecc0c296bfd server_cookie=4dc64425f0307b5c in_seq=0 out_seq=0 2024-09-11T22:05:50.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.482+0000 7f7b81ffb640 10 client.17502 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:05:50.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.483+0000 7f7b81ffb640 1 -- 172.21.15.1:0/3066116658 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5617eb527860 con 0x5617eb5ceea0 2024-09-11T22:05:50.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.483+0000 7f7b81ffb640 10 client.17502 handle_command_reply: tid=0 2024-09-11T22:05:50.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.483+0000 7f7ba331c740 1 client.17502 shutdown 2024-09-11T22:05:50.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.483+0000 7f7ba331c740 1 -- 172.21.15.1:0/3066116658 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5617eb5bf370 msgr2=0x5617eb5bc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:50.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.483+0000 7f7ba331c740 1 --2- 172.21.15.1:0/3066116658 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5617eb5bf370 0x5617eb5bc7e0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f7b9000bec0 tx=0x7f7b90003b90 comp rx=0 tx=0).stop 2024-09-11T22:05:50.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.483+0000 7f7ba331c740 1 -- 172.21.15.1:0/3066116658 shutdown_connections 2024-09-11T22:05:50.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.483+0000 7f7ba331c740 1 --2- 172.21.15.1:0/3066116658 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5617eb5ceea0 0x5617eb5d1290 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:50.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.483+0000 7f7b81ffb640 0 client.17502 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:05:50.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.483+0000 7f7ba331c740 1 --2- 172.21.15.1:0/3066116658 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5617eb5bf370 0x5617eb5bc7e0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:50.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.483+0000 7f7ba331c740 1 --2- 172.21.15.1:0/3066116658 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617eb5b5b40 0x5617eb5bc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:50.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.483+0000 7f7ba331c740 1 --2- 172.21.15.1:0/3066116658 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617eb5b45d0 0x5617eb5beda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:50.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.483+0000 7f7ba331c740 1 -- 172.21.15.1:0/3066116658 >> 172.21.15.1:0/3066116658 conn(0x5617eb57d310 msgr2=0x5617eb5b6930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:50.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.483+0000 7f7ba331c740 1 -- 172.21.15.1:0/3066116658 shutdown_connections 2024-09-11T22:05:50.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.483+0000 7f7ba331c740 1 -- 172.21.15.1:0/3066116658 wait complete. 2024-09-11T22:05:50.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.483+0000 7f7ba331c740 20 client.17502 trim_cache size 0 max 16384 2024-09-11T22:05:50.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.496+0000 7f7ba331c740 1 Processor -- start 2024-09-11T22:05:50.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.496+0000 7f7ba331c740 1 -- start start 2024-09-11T22:05:50.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.496+0000 7f7ba331c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5617eb5b45d0 0x5617eb5ba920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:50.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.496+0000 7f7ba331c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617eb5b5b40 0x5617eb5bae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:50.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.496+0000 7f7ba331c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617eb5bf370 0x5617eb5d6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:50.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.496+0000 7f7ba331c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9c1adac0 con 0x5617eb5b45d0 2024-09-11T22:05:50.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.496+0000 7f7ba331c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9c1af900 con 0x5617eb5bf370 2024-09-11T22:05:50.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.496+0000 7f7ba331c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9c1ae250 con 0x5617eb5b5b40 2024-09-11T22:05:50.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.496+0000 7f7b99d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5617eb5b45d0 0x5617eb5ba920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:50.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.497+0000 7f7b99d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5617eb5b45d0 0x5617eb5ba920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56646/0 (socket says 172.21.15.1:56646) 2024-09-11T22:05:50.535 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-11T22:05:50.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.497+0000 7f7b99d9b640 1 -- 172.21.15.1:0/3729582525 learned_addr learned my addr 172.21.15.1:0/3729582525 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:50.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.497+0000 7f7b9a59c640 1 --2- 172.21.15.1:0/3729582525 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617eb5b5b40 0x5617eb5bae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:50.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.497+0000 7f7b99d9b640 1 -- 172.21.15.1:0/3729582525 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617eb5b5b40 msgr2=0x5617eb5bae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:50.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.497+0000 7f7b99d9b640 1 --2- 172.21.15.1:0/3729582525 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617eb5b5b40 0x5617eb5bae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:50.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.497+0000 7f7b99d9b640 1 -- 172.21.15.1:0/3729582525 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617eb5bf370 msgr2=0x5617eb5d6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:50.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.497+0000 7f7b99d9b640 1 --2- 172.21.15.1:0/3729582525 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617eb5bf370 0x5617eb5d6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:50.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.497+0000 7f7b99d9b640 1 -- 172.21.15.1:0/3729582525 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617eb5b7120 con 0x5617eb5b45d0 2024-09-11T22:05:50.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.497+0000 7f7b99d9b640 1 --2- 172.21.15.1:0/3729582525 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5617eb5b45d0 0x5617eb5ba920 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f7b9000be20 tx=0x7f7b9000b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31d0206566be043c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:50.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.497+0000 7f7b83fff640 1 -- 172.21.15.1:0/3729582525 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b90023040 con 0x5617eb5b45d0 2024-09-11T22:05:50.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.497+0000 7f7b83fff640 1 -- 172.21.15.1:0/3729582525 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b900035f0 con 0x5617eb5b45d0 2024-09-11T22:05:50.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.497+0000 7f7b83fff640 1 -- 172.21.15.1:0/3729582525 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b90010670 con 0x5617eb5b45d0 2024-09-11T22:05:50.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.498+0000 7f7ba331c740 1 -- 172.21.15.1:0/3729582525 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5617eb5b45d0 msgr2=0x5617eb5ba920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:50.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.498+0000 7f7ba331c740 1 --2- 172.21.15.1:0/3729582525 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5617eb5b45d0 0x5617eb5ba920 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f7b9000be20 tx=0x7f7b9000b390 comp rx=0 tx=0).stop 2024-09-11T22:05:50.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.498+0000 7f7ba331c740 1 -- 172.21.15.1:0/3729582525 shutdown_connections 2024-09-11T22:05:50.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.498+0000 7f7ba331c740 1 --2- 172.21.15.1:0/3729582525 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617eb5bf370 0x5617eb5d6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:50.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.498+0000 7f7ba331c740 1 --2- 172.21.15.1:0/3729582525 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617eb5b5b40 0x5617eb5bae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:50.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.498+0000 7f7ba331c740 1 --2- 172.21.15.1:0/3729582525 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5617eb5b45d0 0x5617eb5ba920 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:50.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.498+0000 7f7ba331c740 1 -- 172.21.15.1:0/3729582525 >> 172.21.15.1:0/3729582525 conn(0x5617eb57d310 msgr2=0x5617eb5b7310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:50.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.498+0000 7f7ba331c740 1 -- 172.21.15.1:0/3729582525 shutdown_connections 2024-09-11T22:05:50.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.498+0000 7f7ba331c740 1 -- 172.21.15.1:0/3729582525 wait complete. 2024-09-11T22:05:50.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.498+0000 7f7ba331c740 1 -- start start 2024-09-11T22:05:50.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.498+0000 7f7ba331c740 1 Processor -- start 2024-09-11T22:05:50.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.498+0000 7f7ba331c740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617eb5b45d0 0x5617eb631d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:50.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.498+0000 7f7ba331c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617eb5b5b40 0x5617eb632260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:50.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.498+0000 7f7ba331c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5617eb5bf370 0x5617eb5c29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:50.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.498+0000 7f7ba331c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9c1af170 con 0x5617eb5bf370 2024-09-11T22:05:50.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.498+0000 7f7ba331c740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9c1ae9e0 con 0x5617eb5b45d0 2024-09-11T22:05:50.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.498+0000 7f7ba331c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9c1abaa0 con 0x5617eb5b5b40 2024-09-11T22:05:50.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.498+0000 7f7ba331c740 10 client.0 _ll_register_callbacks cb 0x7f7b90012480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:05:50.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.499+0000 7f7b99d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5617eb5bf370 0x5617eb5c29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:50.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.499+0000 7f7b99d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5617eb5bf370 0x5617eb5c29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56658/0 (socket says 172.21.15.1:56658) 2024-09-11T22:05:50.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.499+0000 7f7b99d9b640 1 -- 172.21.15.1:0/2504724668 learned_addr learned my addr 172.21.15.1:0/2504724668 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:50.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.499+0000 7f7b99d9b640 1 --2- 172.21.15.1:0/2504724668 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617eb5b45d0 0x5617eb631d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:50.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.499+0000 7f7b9a59c640 1 --2- 172.21.15.1:0/2504724668 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617eb5b5b40 0x5617eb632260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:50.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.499+0000 7f7b99d9b640 1 -- 172.21.15.1:0/2504724668 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617eb5b5b40 msgr2=0x5617eb632260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:50.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.499+0000 7f7b99d9b640 1 --2- 172.21.15.1:0/2504724668 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617eb5b5b40 0x5617eb632260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:50.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.499+0000 7f7b99d9b640 1 -- 172.21.15.1:0/2504724668 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617eb5b45d0 msgr2=0x5617eb631d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:50.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.499+0000 7f7b99d9b640 1 --2- 172.21.15.1:0/2504724668 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617eb5b45d0 0x5617eb631d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:50.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.499+0000 7f7b99d9b640 1 -- 172.21.15.1:0/2504724668 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617eb5c3210 con 0x5617eb5bf370 2024-09-11T22:05:50.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.499+0000 7f7b9a59c640 1 --2- 172.21.15.1:0/2504724668 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617eb5b5b40 0x5617eb632260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:05:50.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.499+0000 7f7b99d9b640 1 --2- 172.21.15.1:0/2504724668 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5617eb5bf370 0x5617eb5c29e0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f7b9000b880 tx=0x7f7b90003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=118d4599dd74402c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:50.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.499+0000 7f7b81ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:05:50.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.499+0000 7f7b81ffb640 1 -- 172.21.15.1:0/2504724668 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b90004860 con 0x5617eb5bf370 2024-09-11T22:05:50.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.499+0000 7f7b81ffb640 1 -- 172.21.15.1:0/2504724668 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7f7b90010670 con 0x5617eb5bf370 2024-09-11T22:05:50.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.499+0000 7f7ba331c740 1 -- 172.21.15.1:0/2504724668 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b900035f0 con 0x5617eb5bf370 2024-09-11T22:05:50.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.499+0000 7f7b81ffb640 1 -- 172.21.15.1:0/2504724668 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f7b90010870 con 0x5617eb5bf370 2024-09-11T22:05:50.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.499+0000 7f7ba331c740 10 client.17508 fetch_fsmap learned FSMap version 45 2024-09-11T22:05:50.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.499+0000 7f7ba331c740 1 -- 172.21.15.1:0/2504724668 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5617eb639530 con 0x5617eb5bf370 2024-09-11T22:05:50.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.499+0000 7f7b81ffb640 1 -- 172.21.15.1:0/2504724668 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b900035f0 con 0x5617eb5bf370 2024-09-11T22:05:50.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.500+0000 7f7b81ffb640 1 -- 172.21.15.1:0/2504724668 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 45) ==== 8442+0+0 (secure 0 0 0) 0x7f7b900052f0 con 0x5617eb5bf370 2024-09-11T22:05:50.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.500+0000 7f7ba331c740 10 client.17508 fetch_fsmap finished waiting for FSMap version 45 2024-09-11T22:05:50.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.500+0000 7f7ba331c740 10 client.17508 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:05:50.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.500+0000 7f7ba331c740 20 client.17508 populate_metadata read hostname 'smithi001' 2024-09-11T22:05:50.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.500+0000 7f7ba331c740 1 --2- 172.21.15.1:0/2504724668 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5617eb5ceea0 0x5617eb63bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:50.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.500+0000 7f7ba331c740 4 client.17508 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:05:50.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.500+0000 7f7ba331c740 1 -- 172.21.15.1:0/2504724668 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5617eb521d70 con 0x5617eb5ceea0 2024-09-11T22:05:50.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.500+0000 7f7b9a59c640 1 --2- 172.21.15.1:0/2504724668 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5617eb5ceea0 0x5617eb63bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:50.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.500+0000 7f7b81ffb640 10 client.17508 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:05:50.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.500+0000 7f7b9a59c640 1 --2- 172.21.15.1:0/2504724668 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5617eb5ceea0 0x5617eb63bc60 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b50f2b9099b0502e server_cookie=d636f37b8a7e006b in_seq=0 out_seq=0 2024-09-11T22:05:50.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.501+0000 7f7b81ffb640 1 -- 172.21.15.1:0/2504724668 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5617eb521d70 con 0x5617eb5ceea0 2024-09-11T22:05:50.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.501+0000 7f7b81ffb640 10 client.17508 handle_command_reply: tid=0 2024-09-11T22:05:50.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.501+0000 7f7ba331c740 1 client.17508 shutdown 2024-09-11T22:05:50.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.501+0000 7f7ba331c740 1 -- 172.21.15.1:0/2504724668 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5617eb5bf370 msgr2=0x5617eb5c29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:50.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.501+0000 7f7ba331c740 1 --2- 172.21.15.1:0/2504724668 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5617eb5bf370 0x5617eb5c29e0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f7b9000b880 tx=0x7f7b90003c80 comp rx=0 tx=0).stop 2024-09-11T22:05:50.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.501+0000 7f7ba331c740 1 -- 172.21.15.1:0/2504724668 shutdown_connections 2024-09-11T22:05:50.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.501+0000 7f7ba331c740 1 --2- 172.21.15.1:0/2504724668 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5617eb5ceea0 0x5617eb63bc60 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:50.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.501+0000 7f7ba331c740 1 --2- 172.21.15.1:0/2504724668 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5617eb5bf370 0x5617eb5c29e0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:50.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.501+0000 7f7ba331c740 1 --2- 172.21.15.1:0/2504724668 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5617eb5b5b40 0x5617eb632260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:50.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.501+0000 7f7ba331c740 1 --2- 172.21.15.1:0/2504724668 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5617eb5b45d0 0x5617eb631d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:50.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.501+0000 7f7b81ffb640 0 client.17508 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:05:50.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.501+0000 7f7ba331c740 1 -- 172.21.15.1:0/2504724668 >> 172.21.15.1:0/2504724668 conn(0x5617eb57d310 msgr2=0x5617eb5b6110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:50.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.501+0000 7f7ba331c740 1 -- 172.21.15.1:0/2504724668 shutdown_connections 2024-09-11T22:05:50.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.501+0000 7f7ba331c740 1 -- 172.21.15.1:0/2504724668 wait complete. 2024-09-11T22:05:50.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.501+0000 7f7ba331c740 20 client.17508 trim_cache size 0 max 16384 2024-09-11T22:05:50.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.501+0000 7f7ba0f43640 1 -- 172.21.15.1:0/35825327 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f7b840a2d70 msgr2=0x7f7b840a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:50.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.501+0000 7f7ba0f43640 1 --2- 172.21.15.1:0/35825327 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f7b840a2d70 0x7f7b840a5230 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7b940ccce0 tx=0x7f7b9406a9e0 comp rx=0 tx=0).stop 2024-09-11T22:05:50.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.501+0000 7f7ba0f43640 1 -- 172.21.15.1:0/35825327 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b9c106a90 msgr2=0x7f7b9c1a4310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:50.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.501+0000 7f7ba0f43640 1 --2- 172.21.15.1:0/35825327 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b9c106a90 0x7f7b9c1a4310 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f7b9400eaa0 tx=0x7f7b9400ef60 comp rx=0 tx=0).stop 2024-09-11T22:05:50.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.501+0000 7f7ba0f43640 1 -- 172.21.15.1:0/35825327 shutdown_connections 2024-09-11T22:05:50.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.501+0000 7f7ba0f43640 1 --2- 172.21.15.1:0/35825327 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f7b840a2d70 0x7f7b840a5230 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:50.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.501+0000 7f7ba0f43640 1 --2- 172.21.15.1:0/35825327 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b9c106a90 0x7f7b9c1a4310 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:50.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.501+0000 7f7ba0f43640 1 --2- 172.21.15.1:0/35825327 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b9c101430 0x7f7b9c1997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:50.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.501+0000 7f7ba0f43640 1 --2- 172.21.15.1:0/35825327 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b9c100a90 0x7f7b9c199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:50.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.501+0000 7f7ba0f43640 1 -- 172.21.15.1:0/35825327 >> 172.21.15.1:0/35825327 conn(0x7f7b9c0fc7d0 msgr2=0x7f7b9c0fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:50.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.502+0000 7f7ba0f43640 1 -- 172.21.15.1:0/35825327 shutdown_connections 2024-09-11T22:05:50.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:50.502+0000 7f7ba0f43640 1 -- 172.21.15.1:0/35825327 wait complete. 2024-09-11T22:05:51.240 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:50 smithi001 ceph-mon[30935]: pgmap v269: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 32 KiB/s wr, 25 op/s 2024-09-11T22:05:51.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:50 smithi053 ceph-mon[34908]: pgmap v269: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 32 KiB/s wr, 25 op/s 2024-09-11T22:05:51.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:50 smithi161 ceph-mon[35615]: pgmap v269: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 32 KiB/s wr, 25 op/s 2024-09-11T22:05:52.273 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T22:05:52.341 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T22:05:52.616 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:05:53.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:52 smithi001 ceph-mon[30935]: pgmap v270: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 22 op/s 2024-09-11T22:05:53.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.200+0000 7ff63a1ed640 1 -- 172.21.15.1:0/2877811494 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff634102d90 msgr2=0x7ff634103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:53.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.200+0000 7ff63a1ed640 1 --2- 172.21.15.1:0/2877811494 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff634102d90 0x7ff634103170 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7ff624009930 tx=0x7ff62402a420 comp rx=0 tx=0).stop 2024-09-11T22:05:53.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.201+0000 7ff63a1ed640 1 -- 172.21.15.1:0/2877811494 shutdown_connections 2024-09-11T22:05:53.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.201+0000 7ff63a1ed640 1 --2- 172.21.15.1:0/2877811494 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff634108290 0x7ff63410ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:53.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.201+0000 7ff63a1ed640 1 --2- 172.21.15.1:0/2877811494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff634103740 0x7ff634107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:53.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.201+0000 7ff63a1ed640 1 --2- 172.21.15.1:0/2877811494 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff634102d90 0x7ff634103170 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:53.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.201+0000 7ff63a1ed640 1 -- 172.21.15.1:0/2877811494 >> 172.21.15.1:0/2877811494 conn(0x7ff6340fc7d0 msgr2=0x7ff6340febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:53.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.201+0000 7ff63a1ed640 1 -- 172.21.15.1:0/2877811494 shutdown_connections 2024-09-11T22:05:53.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.201+0000 7ff63a1ed640 1 -- 172.21.15.1:0/2877811494 wait complete. 2024-09-11T22:05:53.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.201+0000 7ff63a1ed640 1 Processor -- start 2024-09-11T22:05:53.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.202+0000 7ff63a1ed640 1 -- start start 2024-09-11T22:05:53.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.202+0000 7ff63a1ed640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff634102d90 0x7ff6341985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:53.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.202+0000 7ff63a1ed640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff634103740 0x7ff634198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:53.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.202+0000 7ff63a1ed640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff634108290 0x7ff63419cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:53.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.202+0000 7ff63a1ed640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff634112090 con 0x7ff634108290 2024-09-11T22:05:53.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.202+0000 7ff63a1ed640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff634111f10 con 0x7ff634103740 2024-09-11T22:05:53.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.202+0000 7ff63a1ed640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff634112210 con 0x7ff634102d90 2024-09-11T22:05:53.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.202+0000 7ff633fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff634108290 0x7ff63419cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:53.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.202+0000 7ff633fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff634108290 0x7ff63419cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56676/0 (socket says 172.21.15.1:56676) 2024-09-11T22:05:53.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.202+0000 7ff633fff640 1 -- 172.21.15.1:0/1827943840 learned_addr learned my addr 172.21.15.1:0/1827943840 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:53.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.202+0000 7ff632ffd640 1 --2- 172.21.15.1:0/1827943840 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff634103740 0x7ff634198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:53.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.202+0000 7ff6337fe640 1 --2- 172.21.15.1:0/1827943840 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff634102d90 0x7ff6341985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:53.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.202+0000 7ff633fff640 1 -- 172.21.15.1:0/1827943840 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff634102d90 msgr2=0x7ff6341985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:53.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.202+0000 7ff633fff640 1 --2- 172.21.15.1:0/1827943840 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff634102d90 0x7ff6341985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:53.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.202+0000 7ff633fff640 1 -- 172.21.15.1:0/1827943840 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff634103740 msgr2=0x7ff634198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:53.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.202+0000 7ff633fff640 1 --2- 172.21.15.1:0/1827943840 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff634103740 0x7ff634198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:53.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.202+0000 7ff633fff640 1 -- 172.21.15.1:0/1827943840 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff63419d580 con 0x7ff634108290 2024-09-11T22:05:53.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.202+0000 7ff632ffd640 1 --2- 172.21.15.1:0/1827943840 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff634103740 0x7ff634198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:05:53.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.202+0000 7ff633fff640 1 --2- 172.21.15.1:0/1827943840 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff634108290 0x7ff63419cea0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7ff62c00eb10 tx=0x7ff62c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:53.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.203+0000 7ff630ff9640 1 -- 172.21.15.1:0/1827943840 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62c004b20 con 0x7ff634108290 2024-09-11T22:05:53.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.203+0000 7ff63a1ed640 1 -- 172.21.15.1:0/1827943840 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff63419d870 con 0x7ff634108290 2024-09-11T22:05:53.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.203+0000 7ff630ff9640 1 -- 172.21.15.1:0/1827943840 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff62c004cc0 con 0x7ff634108290 2024-09-11T22:05:53.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.203+0000 7ff630ff9640 1 -- 172.21.15.1:0/1827943840 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62c010760 con 0x7ff634108290 2024-09-11T22:05:53.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.203+0000 7ff63a1ed640 1 -- 172.21.15.1:0/1827943840 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6341a5150 con 0x7ff634108290 2024-09-11T22:05:53.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.205+0000 7ff630ff9640 1 -- 172.21.15.1:0/1827943840 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7ff62c005020 con 0x7ff634108290 2024-09-11T22:05:53.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.206+0000 7ff630ff9640 1 --2- 172.21.15.1:0/1827943840 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7ff6200a2cf0 0x7ff6200a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:53.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.206+0000 7ff6337fe640 1 --2- 172.21.15.1:0/1827943840 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7ff6200a2cf0 0x7ff6200a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:53.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.206+0000 7ff630ff9640 1 -- 172.21.15.1:0/1827943840 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7ff62c0cfb90 con 0x7ff634108290 2024-09-11T22:05:53.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.206+0000 7ff63c5c6740 1 Processor -- start 2024-09-11T22:05:53.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.206+0000 7ff63c5c6740 1 -- start start 2024-09-11T22:05:53.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.206+0000 7ff6337fe640 1 --2- 172.21.15.1:0/1827943840 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7ff6200a2cf0 0x7ff6200a51b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff624000c00 tx=0x7ff624005a20 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:53.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.207+0000 7ff63c5c6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55974860aa60 0x55974860ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:53.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.207+0000 7ff63c5c6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55974860b380 0x559748619c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:53.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.207+0000 7ff63c5c6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55974861a150 0x55974861c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:53.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.207+0000 7ff63c5c6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x559748570280 con 0x55974860b380 2024-09-11T22:05:53.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.207+0000 7ff63c5c6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x559748517b10 con 0x55974861a150 2024-09-11T22:05:53.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.207+0000 7ff63c5c6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55974856c240 con 0x55974860aa60 2024-09-11T22:05:53.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.207+0000 7ff632ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55974860b380 0x559748619c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:53.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.207+0000 7ff632ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55974860b380 0x559748619c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56680/0 (socket says 172.21.15.1:56680) 2024-09-11T22:05:53.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.207+0000 7ff632ffd640 1 -- 172.21.15.1:0/3072089982 learned_addr learned my addr 172.21.15.1:0/3072089982 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:53.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.207+0000 7ff6337fe640 1 --2- 172.21.15.1:0/3072089982 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55974860aa60 0x55974860ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:53.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.207+0000 7ff633fff640 1 --2- 172.21.15.1:0/3072089982 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55974861a150 0x55974861c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:53.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.207+0000 7ff632ffd640 1 -- 172.21.15.1:0/3072089982 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55974860aa60 msgr2=0x55974860ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:53.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.207+0000 7ff632ffd640 1 --2- 172.21.15.1:0/3072089982 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55974860aa60 0x55974860ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:53.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.207+0000 7ff632ffd640 1 -- 172.21.15.1:0/3072089982 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55974861a150 msgr2=0x55974861c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:53.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.207+0000 7ff632ffd640 1 --2- 172.21.15.1:0/3072089982 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55974861a150 0x55974861c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:53.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.207+0000 7ff632ffd640 1 -- 172.21.15.1:0/3072089982 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597485241f0 con 0x55974860b380 2024-09-11T22:05:53.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.207+0000 7ff6337fe640 1 --2- 172.21.15.1:0/3072089982 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55974860aa60 0x55974860ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:05:53.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.207+0000 7ff632ffd640 1 --2- 172.21.15.1:0/3072089982 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55974860b380 0x559748619c10 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7ff61c00bce0 tx=0x7ff61c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91632fc7482146d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:53.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.208+0000 7ff629ffb640 1 -- 172.21.15.1:0/3072089982 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff61c015070 con 0x55974860b380 2024-09-11T22:05:53.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.208+0000 7ff629ffb640 1 -- 172.21.15.1:0/3072089982 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff61c01a940 con 0x55974860b380 2024-09-11T22:05:53.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.208+0000 7ff629ffb640 1 -- 172.21.15.1:0/3072089982 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff61c011aa0 con 0x55974860b380 2024-09-11T22:05:53.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.208+0000 7ff63c5c6740 1 -- 172.21.15.1:0/3072089982 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55974860b380 msgr2=0x559748619c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:53.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.208+0000 7ff63c5c6740 1 --2- 172.21.15.1:0/3072089982 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55974860b380 0x559748619c10 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7ff61c00bce0 tx=0x7ff61c00f730 comp rx=0 tx=0).stop 2024-09-11T22:05:53.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.208+0000 7ff63c5c6740 1 -- 172.21.15.1:0/3072089982 shutdown_connections 2024-09-11T22:05:53.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.208+0000 7ff63c5c6740 1 --2- 172.21.15.1:0/3072089982 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55974861a150 0x55974861c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:53.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.208+0000 7ff63c5c6740 1 --2- 172.21.15.1:0/3072089982 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55974860b380 0x559748619c10 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:53.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.208+0000 7ff63c5c6740 1 --2- 172.21.15.1:0/3072089982 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55974860aa60 0x55974860ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:53.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.208+0000 7ff63c5c6740 1 -- 172.21.15.1:0/3072089982 >> 172.21.15.1:0/3072089982 conn(0x5597486070f0 msgr2=0x5597486074d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:53.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.208+0000 7ff63c5c6740 1 -- 172.21.15.1:0/3072089982 shutdown_connections 2024-09-11T22:05:53.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.209+0000 7ff63c5c6740 1 -- 172.21.15.1:0/3072089982 wait complete. 2024-09-11T22:05:53.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.209+0000 7ff63c5c6740 1 -- start start 2024-09-11T22:05:53.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.209+0000 7ff63c5c6740 1 Processor -- start 2024-09-11T22:05:53.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.209+0000 7ff63c5c6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55974860aa60 0x559748617090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:53.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.209+0000 7ff63c5c6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55974860b380 0x5597486175d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:53.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.209+0000 7ff63c5c6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55974861a150 0x559748617b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:53.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.209+0000 7ff63c5c6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55974861cd80 con 0x55974860b380 2024-09-11T22:05:53.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.209+0000 7ff63c5c6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55974852af10 con 0x55974860aa60 2024-09-11T22:05:53.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.209+0000 7ff63c5c6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55974861cf00 con 0x55974861a150 2024-09-11T22:05:53.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.209+0000 7ff63c5c6740 10 client.0 _ll_register_callbacks cb 0x5597484f3a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:05:53.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.209+0000 7ff632ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55974860b380 0x5597486175d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:53.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.209+0000 7ff632ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55974860b380 0x5597486175d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56684/0 (socket says 172.21.15.1:56684) 2024-09-11T22:05:53.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.210+0000 7ff632ffd640 1 -- 172.21.15.1:0/847027391 learned_addr learned my addr 172.21.15.1:0/847027391 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:53.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.210+0000 7ff633fff640 1 --2- 172.21.15.1:0/847027391 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55974861a150 0x559748617b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:53.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.210+0000 7ff6337fe640 1 --2- 172.21.15.1:0/847027391 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55974860aa60 0x559748617090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:53.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.210+0000 7ff632ffd640 1 -- 172.21.15.1:0/847027391 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55974861a150 msgr2=0x559748617b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:53.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.210+0000 7ff632ffd640 1 --2- 172.21.15.1:0/847027391 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55974861a150 0x559748617b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:53.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.210+0000 7ff632ffd640 1 -- 172.21.15.1:0/847027391 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55974860aa60 msgr2=0x559748617090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:53.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.210+0000 7ff632ffd640 1 --2- 172.21.15.1:0/847027391 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55974860aa60 0x559748617090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:53.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.210+0000 7ff632ffd640 1 -- 172.21.15.1:0/847027391 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597482e16a0 con 0x55974860b380 2024-09-11T22:05:53.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.210+0000 7ff632ffd640 1 --2- 172.21.15.1:0/847027391 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55974860b380 0x5597486175d0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7ff61c01a690 tx=0x7ff61c01a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bec7b02d54e30689 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:53.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.210+0000 7ff612ffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:05:53.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.210+0000 7ff612ffd640 1 -- 172.21.15.1:0/847027391 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff61c015070 con 0x55974860b380 2024-09-11T22:05:53.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.210+0000 7ff612ffd640 1 -- 172.21.15.1:0/847027391 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7ff61c00bdd0 con 0x55974860b380 2024-09-11T22:05:53.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.210+0000 7ff63c5c6740 1 -- 172.21.15.1:0/847027391 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff61c01a940 con 0x55974860b380 2024-09-11T22:05:53.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.210+0000 7ff612ffd640 1 -- 172.21.15.1:0/847027391 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7ff61c01ad00 con 0x55974860b380 2024-09-11T22:05:53.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.210+0000 7ff63c5c6740 10 client.17520 fetch_fsmap learned FSMap version 45 2024-09-11T22:05:53.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.210+0000 7ff63c5c6740 1 -- 172.21.15.1:0/847027391 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x559748628a20 con 0x55974860b380 2024-09-11T22:05:53.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.211+0000 7ff612ffd640 1 -- 172.21.15.1:0/847027391 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff61c01a940 con 0x55974860b380 2024-09-11T22:05:53.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.211+0000 7ff612ffd640 1 -- 172.21.15.1:0/847027391 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 45) ==== 8442+0+0 (secure 0 0 0) 0x7ff61c02a030 con 0x55974860b380 2024-09-11T22:05:53.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.211+0000 7ff63c5c6740 10 client.17520 fetch_fsmap finished waiting for FSMap version 45 2024-09-11T22:05:53.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.211+0000 7ff63c5c6740 10 client.17520 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:05:53.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.211+0000 7ff63c5c6740 20 client.17520 populate_metadata read hostname 'smithi001' 2024-09-11T22:05:53.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.211+0000 7ff63c5c6740 1 --2- 172.21.15.1:0/847027391 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x559748629e80 0x55974862c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:53.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.211+0000 7ff63c5c6740 4 client.17520 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:05:53.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.211+0000 7ff63c5c6740 1 -- 172.21.15.1:0/847027391 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55974857d7f0 con 0x559748629e80 2024-09-11T22:05:53.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.211+0000 7ff6337fe640 1 --2- 172.21.15.1:0/847027391 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x559748629e80 0x55974862c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:53.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.212+0000 7ff612ffd640 10 client.17520 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:05:53.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.212+0000 7ff6337fe640 1 --2- 172.21.15.1:0/847027391 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x559748629e80 0x55974862c270 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48156a074b738b8f server_cookie=62e6d86037755971 in_seq=0 out_seq=0 2024-09-11T22:05:53.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.212+0000 7ff612ffd640 1 -- 172.21.15.1:0/847027391 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55974857d7f0 con 0x559748629e80 2024-09-11T22:05:53.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.212+0000 7ff612ffd640 10 client.17520 handle_command_reply: tid=0 2024-09-11T22:05:53.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.212+0000 7ff63c5c6740 1 client.17520 shutdown 2024-09-11T22:05:53.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.213+0000 7ff63c5c6740 1 -- 172.21.15.1:0/847027391 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55974860b380 msgr2=0x5597486175d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:53.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.213+0000 7ff63c5c6740 1 --2- 172.21.15.1:0/847027391 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55974860b380 0x5597486175d0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7ff61c01a690 tx=0x7ff61c01a6c0 comp rx=0 tx=0).stop 2024-09-11T22:05:53.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.213+0000 7ff63c5c6740 1 -- 172.21.15.1:0/847027391 shutdown_connections 2024-09-11T22:05:53.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.213+0000 7ff63c5c6740 1 --2- 172.21.15.1:0/847027391 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x559748629e80 0x55974862c270 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:53.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.213+0000 7ff63c5c6740 1 --2- 172.21.15.1:0/847027391 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55974861a150 0x559748617b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:53.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.213+0000 7ff63c5c6740 1 --2- 172.21.15.1:0/847027391 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55974860b380 0x5597486175d0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:53.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.213+0000 7ff63c5c6740 1 --2- 172.21.15.1:0/847027391 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55974860aa60 0x559748617090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:53.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.213+0000 7ff63c5c6740 1 -- 172.21.15.1:0/847027391 >> 172.21.15.1:0/847027391 conn(0x5597486070f0 msgr2=0x55974860cdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:53.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.213+0000 7ff612ffd640 0 client.17520 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:05:53.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.213+0000 7ff63c5c6740 1 -- 172.21.15.1:0/847027391 shutdown_connections 2024-09-11T22:05:53.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.213+0000 7ff63c5c6740 1 -- 172.21.15.1:0/847027391 wait complete. 2024-09-11T22:05:53.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.213+0000 7ff63c5c6740 20 client.17520 trim_cache size 0 max 16384 2024-09-11T22:05:53.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.245+0000 7ff63c5c6740 1 Processor -- start 2024-09-11T22:05:53.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.245+0000 7ff63c5c6740 1 -- start start 2024-09-11T22:05:53.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.246+0000 7ff63c5c6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55974860aa60 0x559748618ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:53.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.246+0000 7ff63c5c6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55974860b380 0x559748631a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:53.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.246+0000 7ff63c5c6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55974861a150 0x559748631f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:53.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.246+0000 7ff63c5c6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff6341ab7c0 con 0x55974861a150 2024-09-11T22:05:53.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.246+0000 7ff63c5c6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff6341ad600 con 0x55974860b380 2024-09-11T22:05:53.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.246+0000 7ff63c5c6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff6341abf50 con 0x55974860aa60 2024-09-11T22:05:53.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.246+0000 7ff6337fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55974860aa60 0x559748618ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:53.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.246+0000 7ff6337fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55974860aa60 0x559748618ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:53004/0 (socket says 172.21.15.1:53004) 2024-09-11T22:05:53.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.246+0000 7ff632ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55974860b380 0x559748631a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:53.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.246+0000 7ff6337fe640 1 -- 172.21.15.1:0/1691862597 learned_addr learned my addr 172.21.15.1:0/1691862597 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:53.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.246+0000 7ff633fff640 1 --2- 172.21.15.1:0/1691862597 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55974861a150 0x559748631f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:53.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.246+0000 7ff6337fe640 1 -- 172.21.15.1:0/1691862597 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55974860b380 msgr2=0x559748631a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:53.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.246+0000 7ff6337fe640 1 --2- 172.21.15.1:0/1691862597 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55974860b380 0x559748631a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:53.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.246+0000 7ff6337fe640 1 -- 172.21.15.1:0/1691862597 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55974861a150 msgr2=0x559748631f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:53.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.246+0000 7ff6337fe640 1 --2- 172.21.15.1:0/1691862597 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55974861a150 0x559748631f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:53.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.246+0000 7ff6337fe640 1 -- 172.21.15.1:0/1691862597 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559748616040 con 0x55974860aa60 2024-09-11T22:05:53.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.246+0000 7ff633fff640 1 --2- 172.21.15.1:0/1691862597 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55974861a150 0x559748631f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:05:53.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.246+0000 7ff632ffd640 1 --2- 172.21.15.1:0/1691862597 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55974860b380 0x559748631a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:05:53.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.247+0000 7ff6337fe640 1 --2- 172.21.15.1:0/1691862597 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55974860aa60 0x559748618ef0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff624009930 tx=0x7ff62403c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e43d4f1ed9393691 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:53.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.247+0000 7ff6297fa640 1 -- 172.21.15.1:0/1691862597 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff624055070 con 0x55974860aa60 2024-09-11T22:05:53.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.247+0000 7ff6297fa640 1 -- 172.21.15.1:0/1691862597 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6240041c0 con 0x55974860aa60 2024-09-11T22:05:53.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.247+0000 7ff6297fa640 1 -- 172.21.15.1:0/1691862597 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff624035050 con 0x55974860aa60 2024-09-11T22:05:53.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.248+0000 7ff63c5c6740 1 -- 172.21.15.1:0/1691862597 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55974860aa60 msgr2=0x559748618ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:53.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.248+0000 7ff63c5c6740 1 --2- 172.21.15.1:0/1691862597 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55974860aa60 0x559748618ef0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff624009930 tx=0x7ff62403c000 comp rx=0 tx=0).stop 2024-09-11T22:05:53.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.248+0000 7ff63c5c6740 1 -- 172.21.15.1:0/1691862597 shutdown_connections 2024-09-11T22:05:53.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.248+0000 7ff63c5c6740 1 --2- 172.21.15.1:0/1691862597 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55974861a150 0x559748631f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:53.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.248+0000 7ff63c5c6740 1 --2- 172.21.15.1:0/1691862597 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55974860b380 0x559748631a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:53.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.248+0000 7ff63c5c6740 1 --2- 172.21.15.1:0/1691862597 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55974860aa60 0x559748618ef0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:53.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.248+0000 7ff63c5c6740 1 -- 172.21.15.1:0/1691862597 >> 172.21.15.1:0/1691862597 conn(0x5597486070f0 msgr2=0x5597486187e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:53.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.248+0000 7ff63c5c6740 1 -- 172.21.15.1:0/1691862597 shutdown_connections 2024-09-11T22:05:53.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.248+0000 7ff63c5c6740 1 -- 172.21.15.1:0/1691862597 wait complete. 2024-09-11T22:05:53.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.248+0000 7ff63c5c6740 1 -- start start 2024-09-11T22:05:53.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.248+0000 7ff63c5c6740 1 Processor -- start 2024-09-11T22:05:53.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.249+0000 7ff63c5c6740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55974860aa60 0x55974861dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:53.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.249+0000 7ff63c5c6740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55974860b380 0x55974861e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:53.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.249+0000 7ff63c5c6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55974861a150 0x55974861ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:53.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.249+0000 7ff63c5c6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff6341ace70 con 0x55974861a150 2024-09-11T22:05:53.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.249+0000 7ff63c5c6740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff6341ac6e0 con 0x55974860aa60 2024-09-11T22:05:53.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.249+0000 7ff63c5c6740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff6341a97a0 con 0x55974860b380 2024-09-11T22:05:53.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.249+0000 7ff63c5c6740 10 client.0 _ll_register_callbacks cb 0x7ff61c00bb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:05:53.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.249+0000 7ff6337fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55974860aa60 0x55974861dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:53.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.249+0000 7ff633fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55974861a150 0x55974861ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:53.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.249+0000 7ff633fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55974861a150 0x55974861ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56694/0 (socket says 172.21.15.1:56694) 2024-09-11T22:05:53.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.249+0000 7ff6337fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55974860aa60 0x55974861dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:43718/0 (socket says 172.21.15.1:43718) 2024-09-11T22:05:53.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.249+0000 7ff633fff640 1 -- 172.21.15.1:0/391440590 learned_addr learned my addr 172.21.15.1:0/391440590 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:53.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.249+0000 7ff632ffd640 1 --2- 172.21.15.1:0/391440590 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55974860b380 0x55974861e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:53.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.249+0000 7ff6337fe640 1 -- 172.21.15.1:0/391440590 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55974860b380 msgr2=0x55974861e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:53.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.249+0000 7ff6337fe640 1 --2- 172.21.15.1:0/391440590 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55974860b380 0x55974861e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:53.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.249+0000 7ff6337fe640 1 -- 172.21.15.1:0/391440590 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55974861a150 msgr2=0x55974861ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:53.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.249+0000 7ff6337fe640 1 --2- 172.21.15.1:0/391440590 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55974861a150 0x55974861ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:53.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.249+0000 7ff6337fe640 1 -- 172.21.15.1:0/391440590 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559748693aa0 con 0x55974860aa60 2024-09-11T22:05:53.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.249+0000 7ff633fff640 1 --2- 172.21.15.1:0/391440590 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55974861a150 0x55974861ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:05:53.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.249+0000 7ff632ffd640 1 --2- 172.21.15.1:0/391440590 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55974860b380 0x55974861e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:05:53.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.250+0000 7ff6337fe640 1 --2- 172.21.15.1:0/391440590 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55974860aa60 0x55974861dfb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff624009900 tx=0x7ff624032e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c21c1e667fb294ba server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:53.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.250+0000 7ff612ffd640 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2024-09-11T22:05:53.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.250+0000 7ff612ffd640 1 -- 172.21.15.1:0/391440590 <== mon.2 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff624055030 con 0x55974860aa60 2024-09-11T22:05:53.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.250+0000 7ff63c5c6740 1 -- 172.21.15.1:0/391440590 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6240041c0 con 0x55974860aa60 2024-09-11T22:05:53.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.250+0000 7ff612ffd640 1 -- 172.21.15.1:0/391440590 <== mon.2 v2:172.21.15.53:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7ff6240035f0 con 0x55974860aa60 2024-09-11T22:05:53.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.250+0000 7ff612ffd640 1 -- 172.21.15.1:0/391440590 <== mon.2 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7ff624035040 con 0x55974860aa60 2024-09-11T22:05:53.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.250+0000 7ff63c5c6740 10 client.26921 fetch_fsmap learned FSMap version 45 2024-09-11T22:05:53.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.250+0000 7ff63c5c6740 1 -- 172.21.15.1:0/391440590 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x559748694d30 con 0x55974860aa60 2024-09-11T22:05:53.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.250+0000 7ff612ffd640 1 -- 172.21.15.1:0/391440590 <== mon.2 v2:172.21.15.53:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6240041c0 con 0x55974860aa60 2024-09-11T22:05:53.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.250+0000 7ff612ffd640 1 -- 172.21.15.1:0/391440590 <== mon.2 v2:172.21.15.53:3300/0 5 ==== fsmap(e 45) ==== 8442+0+0 (secure 0 0 0) 0x7ff624049c90 con 0x55974860aa60 2024-09-11T22:05:53.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.251+0000 7ff63c5c6740 10 client.26921 fetch_fsmap finished waiting for FSMap version 45 2024-09-11T22:05:53.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.251+0000 7ff63c5c6740 10 client.26921 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:05:53.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.251+0000 7ff63c5c6740 20 client.26921 populate_metadata read hostname 'smithi001' 2024-09-11T22:05:53.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.251+0000 7ff63c5c6740 1 --2- 172.21.15.1:0/391440590 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5597486954c0 0x5597486978e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:53.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.251+0000 7ff63c5c6740 4 client.26921 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T22:05:53.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.251+0000 7ff63c5c6740 1 -- 172.21.15.1:0/391440590 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559748577d00 con 0x5597486954c0 2024-09-11T22:05:53.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.251+0000 7ff632ffd640 1 --2- 172.21.15.1:0/391440590 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5597486954c0 0x5597486978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:53.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.251+0000 7ff612ffd640 10 client.26921 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:05:53.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.251+0000 7ff632ffd640 1 --2- 172.21.15.1:0/391440590 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5597486954c0 0x5597486978e0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7931535cc5ec7471 server_cookie=773d8a26f580fc5 in_seq=0 out_seq=0 2024-09-11T22:05:53.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.252+0000 7ff612ffd640 1 -- 172.21.15.1:0/391440590 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559748577d00 con 0x5597486954c0 2024-09-11T22:05:53.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.253+0000 7ff612ffd640 10 client.26921 handle_command_reply: tid=0 2024-09-11T22:05:53.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.253+0000 7ff63c5c6740 1 client.26921 shutdown 2024-09-11T22:05:53.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.253+0000 7ff63c5c6740 1 -- 172.21.15.1:0/391440590 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55974860aa60 msgr2=0x55974861dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:53.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.253+0000 7ff63c5c6740 1 --2- 172.21.15.1:0/391440590 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55974860aa60 0x55974861dfb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff624009900 tx=0x7ff624032e50 comp rx=0 tx=0).stop 2024-09-11T22:05:53.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.253+0000 7ff63c5c6740 1 -- 172.21.15.1:0/391440590 shutdown_connections 2024-09-11T22:05:53.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.253+0000 7ff63c5c6740 1 --2- 172.21.15.1:0/391440590 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5597486954c0 0x5597486978e0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:53.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.253+0000 7ff63c5c6740 1 --2- 172.21.15.1:0/391440590 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55974861a150 0x55974861ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:53.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.253+0000 7ff63c5c6740 1 --2- 172.21.15.1:0/391440590 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55974860b380 0x55974861e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:53.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.253+0000 7ff63c5c6740 1 --2- 172.21.15.1:0/391440590 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55974860aa60 0x55974861dfb0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:53.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.253+0000 7ff63c5c6740 1 -- 172.21.15.1:0/391440590 >> 172.21.15.1:0/391440590 conn(0x5597486070f0 msgr2=0x55974860cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:53.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.253+0000 7ff612ffd640 0 client.26921 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:05:53.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.253+0000 7ff63c5c6740 1 -- 172.21.15.1:0/391440590 shutdown_connections 2024-09-11T22:05:53.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.253+0000 7ff63c5c6740 1 -- 172.21.15.1:0/391440590 wait complete. 2024-09-11T22:05:53.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.253+0000 7ff63c5c6740 20 client.26921 trim_cache size 0 max 16384 2024-09-11T22:05:53.271 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-11T22:05:53.271 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-09-11T22:05:53.271 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "a4068e5d-93d7-44ce-98dd-791c2ba148cc", 2024-09-11T22:05:53.271 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-09-11T22:05:53.271 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-11T22:05:53.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.254+0000 7ff63a1ed640 1 -- 172.21.15.1:0/1827943840 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7ff6200a2cf0 msgr2=0x7ff6200a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:53.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.254+0000 7ff63a1ed640 1 --2- 172.21.15.1:0/1827943840 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7ff6200a2cf0 0x7ff6200a51b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff624000c00 tx=0x7ff624005a20 comp rx=0 tx=0).stop 2024-09-11T22:05:53.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.254+0000 7ff63a1ed640 1 -- 172.21.15.1:0/1827943840 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff634108290 msgr2=0x7ff63419cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:53.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.254+0000 7ff63a1ed640 1 --2- 172.21.15.1:0/1827943840 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff634108290 0x7ff63419cea0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7ff62c00eb10 tx=0x7ff62c00c2d0 comp rx=0 tx=0).stop 2024-09-11T22:05:53.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.254+0000 7ff63a1ed640 1 -- 172.21.15.1:0/1827943840 shutdown_connections 2024-09-11T22:05:53.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.254+0000 7ff63a1ed640 1 --2- 172.21.15.1:0/1827943840 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7ff6200a2cf0 0x7ff6200a51b0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:53.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.254+0000 7ff63a1ed640 1 --2- 172.21.15.1:0/1827943840 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff634108290 0x7ff63419cea0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:53.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.254+0000 7ff63a1ed640 1 --2- 172.21.15.1:0/1827943840 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff634103740 0x7ff634198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:53.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.254+0000 7ff63a1ed640 1 --2- 172.21.15.1:0/1827943840 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff634102d90 0x7ff6341985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:53.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.254+0000 7ff63a1ed640 1 -- 172.21.15.1:0/1827943840 >> 172.21.15.1:0/1827943840 conn(0x7ff6340fc7d0 msgr2=0x7ff634105e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:53.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.254+0000 7ff63a1ed640 1 -- 172.21.15.1:0/1827943840 shutdown_connections 2024-09-11T22:05:53.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:53.254+0000 7ff63a1ed640 1 -- 172.21.15.1:0/1827943840 wait complete. 2024-09-11T22:05:53.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:52 smithi053 ceph-mon[34908]: pgmap v270: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 22 op/s 2024-09-11T22:05:53.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:52 smithi161 ceph-mon[35615]: pgmap v270: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 22 op/s 2024-09-11T22:05:54.245 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:53 smithi001 ceph-mon[30935]: scrub queued for path: / 2024-09-11T22:05:54.245 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:53 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:05:54.246 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:53 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:05:54.246 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:53 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:05:54.247 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:05:54.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:53 smithi053 ceph-mon[34908]: scrub queued for path: / 2024-09-11T22:05:54.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:53 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:05:54.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:53 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:05:54.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:53 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:05:54.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:53 smithi161 ceph-mon[35615]: scrub queued for path: / 2024-09-11T22:05:54.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:53 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:05:54.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:53 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:05:54.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:53 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:05:54.507 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:05:55.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.157+0000 7f7a0632d640 1 -- 172.21.15.1:0/1559050640 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7a00100a60 msgr2=0x7f7a00100ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:55.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.157+0000 7f7a0632d640 1 --2- 172.21.15.1:0/1559050640 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7a00100a60 0x7f7a00100ec0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f79e8009930 tx=0x7f79e802a420 comp rx=0 tx=0).stop 2024-09-11T22:05:55.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.157+0000 7f7a0632d640 1 -- 172.21.15.1:0/1559050640 shutdown_connections 2024-09-11T22:05:55.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.157+0000 7f7a0632d640 1 --2- 172.21.15.1:0/1559050640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a00101400 0x7f7a0010d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:55.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.157+0000 7f7a0632d640 1 --2- 172.21.15.1:0/1559050640 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7a00100a60 0x7f7a00100ec0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:55.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.157+0000 7f7a0632d640 1 --2- 172.21.15.1:0/1559050640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a00106a60 0x7f7a00106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:55.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.157+0000 7f7a0632d640 1 -- 172.21.15.1:0/1559050640 >> 172.21.15.1:0/1559050640 conn(0x7f7a000fc780 msgr2=0x7f7a000feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:55.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.157+0000 7f7a0632d640 1 -- 172.21.15.1:0/1559050640 shutdown_connections 2024-09-11T22:05:55.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.157+0000 7f7a0632d640 1 -- 172.21.15.1:0/1559050640 wait complete. 2024-09-11T22:05:55.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.158+0000 7f7a0632d640 1 Processor -- start 2024-09-11T22:05:55.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.158+0000 7f7a0632d640 1 -- start start 2024-09-11T22:05:55.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.158+0000 7f7a0632d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7a00100a60 0x7f7a001973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:55.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.158+0000 7f7a0632d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a00101400 0x7f7a00197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:55.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.158+0000 7f7a0632d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a00106a60 0x7f7a0019bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:55.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.158+0000 7f7a0632d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7a00110e60 con 0x7f7a00100a60 2024-09-11T22:05:55.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.158+0000 7f7a0632d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7a00110ce0 con 0x7f7a00101400 2024-09-11T22:05:55.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.158+0000 7f7a0632d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7a00110fe0 con 0x7f7a00106a60 2024-09-11T22:05:55.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.158+0000 7f79fffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7a00100a60 0x7f7a001973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:55.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.159+0000 7f79fffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7a00100a60 0x7f7a001973e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56704/0 (socket says 172.21.15.1:56704) 2024-09-11T22:05:55.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.159+0000 7f79fffff640 1 -- 172.21.15.1:0/3994690361 learned_addr learned my addr 172.21.15.1:0/3994690361 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:55.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.159+0000 7f7a048ca640 1 --2- 172.21.15.1:0/3994690361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a00106a60 0x7f7a0019bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:55.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.159+0000 7f79f7fff640 1 --2- 172.21.15.1:0/3994690361 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a00101400 0x7f7a00197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:55.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.159+0000 7f79fffff640 1 -- 172.21.15.1:0/3994690361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a00106a60 msgr2=0x7f7a0019bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:55.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.159+0000 7f79fffff640 1 --2- 172.21.15.1:0/3994690361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a00106a60 0x7f7a0019bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:55.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.159+0000 7f79fffff640 1 -- 172.21.15.1:0/3994690361 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a00101400 msgr2=0x7f7a00197920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:55.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.159+0000 7f79fffff640 1 --2- 172.21.15.1:0/3994690361 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a00101400 0x7f7a00197920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:55.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.159+0000 7f79fffff640 1 -- 172.21.15.1:0/3994690361 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a0019c390 con 0x7f7a00100a60 2024-09-11T22:05:55.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.159+0000 7f79fffff640 1 --2- 172.21.15.1:0/3994690361 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7a00100a60 0x7f7a001973e0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f79f0009970 tx=0x7f79f0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:55.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.159+0000 7f79fdffb640 1 -- 172.21.15.1:0/3994690361 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f000d020 con 0x7f7a00100a60 2024-09-11T22:05:55.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.159+0000 7f7a0632d640 1 -- 172.21.15.1:0/3994690361 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a0019c680 con 0x7f7a00100a60 2024-09-11T22:05:55.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.159+0000 7f79fdffb640 1 -- 172.21.15.1:0/3994690361 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79f0007d40 con 0x7f7a00100a60 2024-09-11T22:05:55.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.159+0000 7f79fdffb640 1 -- 172.21.15.1:0/3994690361 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f000ccc0 con 0x7f7a00100a60 2024-09-11T22:05:55.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.159+0000 7f7a0632d640 1 -- 172.21.15.1:0/3994690361 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a0006a790 con 0x7f7a00100a60 2024-09-11T22:05:55.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.161+0000 7f79fdffb640 1 -- 172.21.15.1:0/3994690361 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f79f000c460 con 0x7f7a00100a60 2024-09-11T22:05:55.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.162+0000 7f7a08706740 1 Processor -- start 2024-09-11T22:05:55.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.162+0000 7f7a08706740 1 -- start start 2024-09-11T22:05:55.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.162+0000 7f7a08706740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd4e1644a0 0x55cd4e164880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:55.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.162+0000 7f7a08706740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd4e164dc0 0x55cd4e16f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:55.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.162+0000 7f7a08706740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd4e16f680 0x55cd4e171a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:55.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.162+0000 7f7a08706740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55cd4e0ca2e0 con 0x55cd4e164dc0 2024-09-11T22:05:55.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.162+0000 7f7a08706740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55cd4e071ce0 con 0x55cd4e1644a0 2024-09-11T22:05:55.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.162+0000 7f7a08706740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55cd4e0c62a0 con 0x55cd4e16f680 2024-09-11T22:05:55.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.162+0000 7f79fdffb640 1 --2- 172.21.15.1:0/3994690361 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f79ec0a2cf0 0x7f79ec0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:55.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.162+0000 7f79fffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd4e164dc0 0x55cd4e16f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:55.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.162+0000 7f79fffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd4e164dc0 0x55cd4e16f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56710/0 (socket says 172.21.15.1:56710) 2024-09-11T22:05:55.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.162+0000 7f79fffff640 1 -- 172.21.15.1:0/487981208 learned_addr learned my addr 172.21.15.1:0/487981208 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:55.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.162+0000 7f7a048ca640 1 --2- 172.21.15.1:0/487981208 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd4e1644a0 0x55cd4e164880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:55.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.163+0000 7f7a048ca640 1 --2- 172.21.15.1:0/3994690361 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f79ec0a2cf0 0x7f79ec0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:55.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.163+0000 7f79f7fff640 1 --2- 172.21.15.1:0/487981208 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd4e16f680 0x55cd4e171a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:55.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.163+0000 7f79fdffb640 1 -- 172.21.15.1:0/3994690361 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7f79f00d07a0 con 0x7f7a00100a60 2024-09-11T22:05:55.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.163+0000 7f79fffff640 1 -- 172.21.15.1:0/487981208 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd4e16f680 msgr2=0x55cd4e171a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:55.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.163+0000 7f79fffff640 1 --2- 172.21.15.1:0/487981208 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd4e16f680 0x55cd4e171a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:55.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.163+0000 7f79fffff640 1 -- 172.21.15.1:0/487981208 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd4e1644a0 msgr2=0x55cd4e164880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:55.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.163+0000 7f79fffff640 1 --2- 172.21.15.1:0/487981208 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd4e1644a0 0x55cd4e164880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:55.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.163+0000 7f79fffff640 1 -- 172.21.15.1:0/487981208 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd4e056440 con 0x55cd4e164dc0 2024-09-11T22:05:55.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.163+0000 7f7a048ca640 1 --2- 172.21.15.1:0/487981208 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd4e1644a0 0x55cd4e164880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:05:55.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.163+0000 7f79fffff640 1 --2- 172.21.15.1:0/487981208 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd4e164dc0 0x55cd4e16f140 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f79f00cde20 tx=0x7f79f0009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12dba51dd8fc69ab server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:55.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.163+0000 7f7a048ca640 1 --2- 172.21.15.1:0/3994690361 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f79ec0a2cf0 0x7f79ec0a51b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f79f800a780 tx=0x7f79f800a4d0 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:55.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.163+0000 7f79f67fc640 1 -- 172.21.15.1:0/487981208 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f009cd30 con 0x55cd4e164dc0 2024-09-11T22:05:55.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.163+0000 7f79f67fc640 1 -- 172.21.15.1:0/487981208 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79f00a1030 con 0x55cd4e164dc0 2024-09-11T22:05:55.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.163+0000 7f79f67fc640 1 -- 172.21.15.1:0/487981208 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f009c180 con 0x55cd4e164dc0 2024-09-11T22:05:55.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.164+0000 7f7a08706740 1 -- 172.21.15.1:0/487981208 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd4e164dc0 msgr2=0x55cd4e16f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:55.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.164+0000 7f7a08706740 1 --2- 172.21.15.1:0/487981208 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd4e164dc0 0x55cd4e16f140 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f79f00cde20 tx=0x7f79f0009f60 comp rx=0 tx=0).stop 2024-09-11T22:05:55.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.164+0000 7f7a08706740 1 -- 172.21.15.1:0/487981208 shutdown_connections 2024-09-11T22:05:55.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.164+0000 7f7a08706740 1 --2- 172.21.15.1:0/487981208 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd4e16f680 0x55cd4e171a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:55.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.164+0000 7f7a08706740 1 --2- 172.21.15.1:0/487981208 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd4e164dc0 0x55cd4e16f140 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:55.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.164+0000 7f7a08706740 1 --2- 172.21.15.1:0/487981208 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd4e1644a0 0x55cd4e164880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:55.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.164+0000 7f7a08706740 1 -- 172.21.15.1:0/487981208 >> 172.21.15.1:0/487981208 conn(0x55cd4e160a30 msgr2=0x55cd4e160e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:55.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.164+0000 7f7a08706740 1 -- 172.21.15.1:0/487981208 shutdown_connections 2024-09-11T22:05:55.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.164+0000 7f7a08706740 1 -- 172.21.15.1:0/487981208 wait complete. 2024-09-11T22:05:55.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.164+0000 7f7a08706740 1 -- start start 2024-09-11T22:05:55.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.165+0000 7f7a08706740 1 Processor -- start 2024-09-11T22:05:55.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.165+0000 7f7a08706740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd4e1644a0 0x55cd4e16f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:55.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.165+0000 7f7a08706740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd4e164dc0 0x55cd4e16c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:55.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.165+0000 7f7a08706740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd4e16f680 0x55cd4e16c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:55.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.165+0000 7f7a08706740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55cd4e1721f0 con 0x55cd4e16f680 2024-09-11T22:05:55.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.165+0000 7f7a08706740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55cd4e085050 con 0x55cd4e164dc0 2024-09-11T22:05:55.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.165+0000 7f7a08706740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55cd4e172370 con 0x55cd4e1644a0 2024-09-11T22:05:55.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.165+0000 7f7a08706740 10 client.0 _ll_register_callbacks cb 0x55cd4e0c5160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:05:55.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.165+0000 7f7a048ca640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd4e16f680 0x55cd4e16c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:55.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.165+0000 7f7a048ca640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd4e16f680 0x55cd4e16c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56712/0 (socket says 172.21.15.1:56712) 2024-09-11T22:05:55.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.165+0000 7f7a048ca640 1 -- 172.21.15.1:0/2973729881 learned_addr learned my addr 172.21.15.1:0/2973729881 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:55.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.165+0000 7f79fffff640 1 --2- 172.21.15.1:0/2973729881 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd4e1644a0 0x55cd4e16f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:55.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.165+0000 7f79f7fff640 1 --2- 172.21.15.1:0/2973729881 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd4e164dc0 0x55cd4e16c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:55.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.165+0000 7f7a048ca640 1 -- 172.21.15.1:0/2973729881 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd4e1644a0 msgr2=0x55cd4e16f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:55.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.165+0000 7f7a048ca640 1 --2- 172.21.15.1:0/2973729881 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd4e1644a0 0x55cd4e16f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:55.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.165+0000 7f7a048ca640 1 -- 172.21.15.1:0/2973729881 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd4e164dc0 msgr2=0x55cd4e16c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:55.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.165+0000 7f7a048ca640 1 --2- 172.21.15.1:0/2973729881 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd4e164dc0 0x55cd4e16c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:55.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.165+0000 7f7a048ca640 1 -- 172.21.15.1:0/2973729881 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd4de3b3c0 con 0x55cd4e16f680 2024-09-11T22:05:55.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.165+0000 7f79f7fff640 1 --2- 172.21.15.1:0/2973729881 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd4e164dc0 0x55cd4e16c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:05:55.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.166+0000 7f7a048ca640 1 --2- 172.21.15.1:0/2973729881 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd4e16f680 0x55cd4e16c830 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f79f8005fb0 tx=0x7f79f8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=902b4d8ad534ea9b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:55.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.166+0000 7f79df7fe640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:05:55.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.166+0000 7f79df7fe640 1 -- 172.21.15.1:0/2973729881 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f8026070 con 0x55cd4e16f680 2024-09-11T22:05:55.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.166+0000 7f79df7fe640 1 -- 172.21.15.1:0/2973729881 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7f79f8004d20 con 0x55cd4e16f680 2024-09-11T22:05:55.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.166+0000 7f7a08706740 1 -- 172.21.15.1:0/2973729881 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79f00a1030 con 0x55cd4e16f680 2024-09-11T22:05:55.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.166+0000 7f79df7fe640 1 -- 172.21.15.1:0/2973729881 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f79f8030050 con 0x55cd4e16f680 2024-09-11T22:05:55.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.166+0000 7f7a08706740 10 client.17535 fetch_fsmap learned FSMap version 46 2024-09-11T22:05:55.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.166+0000 7f7a08706740 1 -- 172.21.15.1:0/2973729881 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55cd4e17dbe0 con 0x55cd4e16f680 2024-09-11T22:05:55.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.166+0000 7f79df7fe640 1 -- 172.21.15.1:0/2973729881 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79f00a1030 con 0x55cd4e16f680 2024-09-11T22:05:55.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.166+0000 7f79df7fe640 1 -- 172.21.15.1:0/2973729881 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 46) ==== 8442+0+0 (secure 0 0 0) 0x7f79f8020450 con 0x55cd4e16f680 2024-09-11T22:05:55.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.166+0000 7f7a08706740 10 client.17535 fetch_fsmap finished waiting for FSMap version 46 2024-09-11T22:05:55.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.166+0000 7f7a08706740 10 client.17535 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:05:55.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.166+0000 7f7a08706740 20 client.17535 populate_metadata read hostname 'smithi001' 2024-09-11T22:05:55.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.166+0000 7f7a08706740 1 --2- 172.21.15.1:0/2973729881 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55cd4e17f140 0x55cd4e1815c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:55.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.166+0000 7f7a08706740 4 client.17535 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:05:55.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.166+0000 7f7a08706740 1 -- 172.21.15.1:0/2973729881 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd4e0d7850 con 0x55cd4e17f140 2024-09-11T22:05:55.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.167+0000 7f79fffff640 1 --2- 172.21.15.1:0/2973729881 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55cd4e17f140 0x55cd4e1815c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:55.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.167+0000 7f79df7fe640 10 client.17535 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:05:55.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.167+0000 7f79fffff640 1 --2- 172.21.15.1:0/2973729881 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55cd4e17f140 0x55cd4e1815c0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=557529d1284567ab server_cookie=28e6ba53e40b8589 in_seq=0 out_seq=0 2024-09-11T22:05:55.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.168+0000 7f79df7fe640 1 -- 172.21.15.1:0/2973729881 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cd4e0d7850 con 0x55cd4e17f140 2024-09-11T22:05:55.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.168+0000 7f79df7fe640 10 client.17535 handle_command_reply: tid=0 2024-09-11T22:05:55.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.168+0000 7f7a08706740 1 client.17535 shutdown 2024-09-11T22:05:55.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.168+0000 7f7a08706740 1 -- 172.21.15.1:0/2973729881 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd4e16f680 msgr2=0x55cd4e16c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:55.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.168+0000 7f7a08706740 1 --2- 172.21.15.1:0/2973729881 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd4e16f680 0x55cd4e16c830 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f79f8005fb0 tx=0x7f79f8004a40 comp rx=0 tx=0).stop 2024-09-11T22:05:55.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.168+0000 7f7a08706740 1 -- 172.21.15.1:0/2973729881 shutdown_connections 2024-09-11T22:05:55.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.168+0000 7f7a08706740 1 --2- 172.21.15.1:0/2973729881 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55cd4e17f140 0x55cd4e1815c0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:55.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.168+0000 7f7a08706740 1 --2- 172.21.15.1:0/2973729881 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd4e16f680 0x55cd4e16c830 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:55.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.168+0000 7f7a08706740 1 --2- 172.21.15.1:0/2973729881 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd4e164dc0 0x55cd4e16c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:55.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.168+0000 7f7a08706740 1 --2- 172.21.15.1:0/2973729881 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd4e1644a0 0x55cd4e16f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:55.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.168+0000 7f79df7fe640 0 client.17535 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:05:55.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.168+0000 7f7a08706740 1 -- 172.21.15.1:0/2973729881 >> 172.21.15.1:0/2973729881 conn(0x55cd4e160a30 msgr2=0x55cd4e166760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:55.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.168+0000 7f7a08706740 1 -- 172.21.15.1:0/2973729881 shutdown_connections 2024-09-11T22:05:55.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.168+0000 7f7a08706740 1 -- 172.21.15.1:0/2973729881 wait complete. 2024-09-11T22:05:55.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.169+0000 7f7a08706740 20 client.17535 trim_cache size 0 max 16384 2024-09-11T22:05:55.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.186+0000 7f7a08706740 1 Processor -- start 2024-09-11T22:05:55.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.186+0000 7f7a08706740 1 -- start start 2024-09-11T22:05:55.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.186+0000 7f7a08706740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd4e1644a0 0x55cd4e17b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:55.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.186+0000 7f7a08706740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd4e164dc0 0x55cd4e1779a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:55.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.186+0000 7f7a08706740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd4e16f680 0x55cd4e177ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:55.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.186+0000 7f7a08706740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7a001ab920 con 0x55cd4e164dc0 2024-09-11T22:05:55.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.186+0000 7f7a08706740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7a001ad760 con 0x55cd4e1644a0 2024-09-11T22:05:55.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.186+0000 7f7a08706740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7a001ac0b0 con 0x55cd4e16f680 2024-09-11T22:05:55.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.186+0000 7f79f7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd4e164dc0 0x55cd4e1779a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:55.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.186+0000 7f79f7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd4e164dc0 0x55cd4e1779a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56726/0 (socket says 172.21.15.1:56726) 2024-09-11T22:05:55.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.186+0000 7f79f7fff640 1 -- 172.21.15.1:0/1737658384 learned_addr learned my addr 172.21.15.1:0/1737658384 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:55.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.186+0000 7f7a048ca640 1 --2- 172.21.15.1:0/1737658384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd4e16f680 0x55cd4e177ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:55.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.186+0000 7f79fffff640 1 --2- 172.21.15.1:0/1737658384 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd4e1644a0 0x55cd4e17b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:55.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.186+0000 7f79f7fff640 1 -- 172.21.15.1:0/1737658384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd4e16f680 msgr2=0x55cd4e177ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:55.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.186+0000 7f79f7fff640 1 --2- 172.21.15.1:0/1737658384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd4e16f680 0x55cd4e177ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:55.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.186+0000 7f79f7fff640 1 -- 172.21.15.1:0/1737658384 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd4e1644a0 msgr2=0x55cd4e17b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:55.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.186+0000 7f79f7fff640 1 --2- 172.21.15.1:0/1737658384 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd4e1644a0 0x55cd4e17b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:55.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.186+0000 7f79f7fff640 1 -- 172.21.15.1:0/1737658384 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd4e1604d0 con 0x55cd4e164dc0 2024-09-11T22:05:55.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.187+0000 7f79f7fff640 1 --2- 172.21.15.1:0/1737658384 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd4e164dc0 0x55cd4e1779a0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f79e8000c00 tx=0x7f79e80316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44d8a4e6c302a9c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:55.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.187+0000 7f79f5ffb640 1 -- 172.21.15.1:0/1737658384 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e8035040 con 0x55cd4e164dc0 2024-09-11T22:05:55.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.187+0000 7f79f5ffb640 1 -- 172.21.15.1:0/1737658384 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79e80035f0 con 0x55cd4e164dc0 2024-09-11T22:05:55.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.187+0000 7f79f5ffb640 1 -- 172.21.15.1:0/1737658384 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e803d690 con 0x55cd4e164dc0 2024-09-11T22:05:55.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.187+0000 7f7a08706740 1 -- 172.21.15.1:0/1737658384 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd4e164dc0 msgr2=0x55cd4e1779a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:55.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.187+0000 7f7a08706740 1 --2- 172.21.15.1:0/1737658384 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd4e164dc0 0x55cd4e1779a0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f79e8000c00 tx=0x7f79e80316f0 comp rx=0 tx=0).stop 2024-09-11T22:05:55.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.187+0000 7f7a08706740 1 -- 172.21.15.1:0/1737658384 shutdown_connections 2024-09-11T22:05:55.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.187+0000 7f7a08706740 1 --2- 172.21.15.1:0/1737658384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd4e16f680 0x55cd4e177ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:55.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.187+0000 7f7a08706740 1 --2- 172.21.15.1:0/1737658384 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd4e164dc0 0x55cd4e1779a0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:55.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.187+0000 7f7a08706740 1 --2- 172.21.15.1:0/1737658384 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd4e1644a0 0x55cd4e17b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:55.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.187+0000 7f7a08706740 1 -- 172.21.15.1:0/1737658384 >> 172.21.15.1:0/1737658384 conn(0x55cd4e160a30 msgr2=0x55cd4e16b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:55.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.188+0000 7f7a08706740 1 -- 172.21.15.1:0/1737658384 shutdown_connections 2024-09-11T22:05:55.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.188+0000 7f7a08706740 1 -- 172.21.15.1:0/1737658384 wait complete. 2024-09-11T22:05:55.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.188+0000 7f7a08706740 1 -- start start 2024-09-11T22:05:55.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.188+0000 7f7a08706740 1 Processor -- start 2024-09-11T22:05:55.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.188+0000 7f7a08706740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd4e1644a0 0x55cd4e1e1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:55.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.188+0000 7f7a08706740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd4e164dc0 0x55cd4e1e23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:55.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.188+0000 7f7a08706740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd4e16f680 0x55cd4e1e28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:55.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.188+0000 7f7a08706740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7a001acfd0 con 0x55cd4e16f680 2024-09-11T22:05:55.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.188+0000 7f7a08706740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f7a001ac840 con 0x55cd4e1644a0 2024-09-11T22:05:55.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.188+0000 7f7a08706740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f7a001a9900 con 0x55cd4e164dc0 2024-09-11T22:05:55.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.188+0000 7f7a08706740 10 client.0 _ll_register_callbacks cb 0x7f79f8014830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:05:55.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.188+0000 7f7a048ca640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd4e16f680 0x55cd4e1e28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:55.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.189+0000 7f7a048ca640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd4e16f680 0x55cd4e1e28e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56734/0 (socket says 172.21.15.1:56734) 2024-09-11T22:05:55.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.189+0000 7f7a048ca640 1 -- 172.21.15.1:0/755088435 learned_addr learned my addr 172.21.15.1:0/755088435 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:05:55.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.189+0000 7f79fffff640 1 --2- 172.21.15.1:0/755088435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd4e1644a0 0x55cd4e1e1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:55.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.189+0000 7f79f7fff640 1 --2- 172.21.15.1:0/755088435 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd4e164dc0 0x55cd4e1e23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:55.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.189+0000 7f7a048ca640 1 -- 172.21.15.1:0/755088435 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd4e164dc0 msgr2=0x55cd4e1e23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:55.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.189+0000 7f7a048ca640 1 --2- 172.21.15.1:0/755088435 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd4e164dc0 0x55cd4e1e23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:55.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.189+0000 7f7a048ca640 1 -- 172.21.15.1:0/755088435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd4e1644a0 msgr2=0x55cd4e1e1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:05:55.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.189+0000 7f7a048ca640 1 --2- 172.21.15.1:0/755088435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd4e1644a0 0x55cd4e1e1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:55.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.189+0000 7f7a048ca640 1 -- 172.21.15.1:0/755088435 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd4e1ed180 con 0x55cd4e16f680 2024-09-11T22:05:55.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.189+0000 7f7a048ca640 1 --2- 172.21.15.1:0/755088435 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd4e16f680 0x55cd4e1e28e0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x55cd4e16f650 tx=0x7f79f800f9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdc4f552447b4342 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:05:55.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.189+0000 7f79df7fe640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:05:55.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.189+0000 7f79df7fe640 1 -- 172.21.15.1:0/755088435 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f8026040 con 0x55cd4e16f680 2024-09-11T22:05:55.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.189+0000 7f79df7fe640 1 -- 172.21.15.1:0/755088435 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7f79f8014040 con 0x55cd4e16f680 2024-09-11T22:05:55.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.189+0000 7f7a08706740 1 -- 172.21.15.1:0/755088435 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79e80035f0 con 0x55cd4e16f680 2024-09-11T22:05:55.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.189+0000 7f79df7fe640 1 -- 172.21.15.1:0/755088435 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f79f8041050 con 0x55cd4e16f680 2024-09-11T22:05:55.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.189+0000 7f7a08706740 10 client.17541 fetch_fsmap learned FSMap version 46 2024-09-11T22:05:55.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.189+0000 7f7a08706740 1 -- 172.21.15.1:0/755088435 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55cd4e1ee450 con 0x55cd4e16f680 2024-09-11T22:05:55.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.190+0000 7f79df7fe640 1 -- 172.21.15.1:0/755088435 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79e80035f0 con 0x55cd4e16f680 2024-09-11T22:05:55.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.190+0000 7f79df7fe640 1 -- 172.21.15.1:0/755088435 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 46) ==== 8442+0+0 (secure 0 0 0) 0x7f79f8004020 con 0x55cd4e16f680 2024-09-11T22:05:55.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.190+0000 7f7a08706740 10 client.17541 fetch_fsmap finished waiting for FSMap version 46 2024-09-11T22:05:55.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.190+0000 7f7a08706740 10 client.17541 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:05:55.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.190+0000 7f7a08706740 20 client.17541 populate_metadata read hostname 'smithi001' 2024-09-11T22:05:55.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.190+0000 7f7a08706740 1 --2- 172.21.15.1:0/755088435 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55cd4e1eebf0 0x55cd4e1f1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:05:55.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.190+0000 7f7a08706740 4 client.17541 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:05:55.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.190+0000 7f7a08706740 1 -- 172.21.15.1:0/755088435 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cd4e0d1d60 con 0x55cd4e1eebf0 2024-09-11T22:05:55.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.190+0000 7f79fffff640 1 --2- 172.21.15.1:0/755088435 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55cd4e1eebf0 0x55cd4e1f1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:05:55.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.191+0000 7f79df7fe640 10 client.17541 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:05:55.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.191+0000 7f79fffff640 1 --2- 172.21.15.1:0/755088435 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55cd4e1eebf0 0x55cd4e1f1010 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aace8f88132ad1b5 server_cookie=a26fcae1a924bd7e in_seq=0 out_seq=0 2024-09-11T22:05:55.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.191+0000 7f79df7fe640 1 -- 172.21.15.1:0/755088435 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55cd4e0d1d60 con 0x55cd4e1eebf0 2024-09-11T22:05:55.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.191+0000 7f79df7fe640 10 client.17541 handle_command_reply: tid=0 2024-09-11T22:05:55.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.191+0000 7f7a08706740 1 client.17541 shutdown 2024-09-11T22:05:55.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.191+0000 7f7a08706740 1 -- 172.21.15.1:0/755088435 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd4e16f680 msgr2=0x55cd4e1e28e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:55.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.191+0000 7f7a08706740 1 --2- 172.21.15.1:0/755088435 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd4e16f680 0x55cd4e1e28e0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x55cd4e16f650 tx=0x7f79f800f9f0 comp rx=0 tx=0).stop 2024-09-11T22:05:55.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.191+0000 7f7a08706740 1 -- 172.21.15.1:0/755088435 shutdown_connections 2024-09-11T22:05:55.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.191+0000 7f7a08706740 1 --2- 172.21.15.1:0/755088435 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55cd4e1eebf0 0x55cd4e1f1010 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:55.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.191+0000 7f7a08706740 1 --2- 172.21.15.1:0/755088435 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd4e16f680 0x55cd4e1e28e0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:55.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.191+0000 7f79df7fe640 0 client.17541 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:05:55.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.191+0000 7f7a08706740 1 --2- 172.21.15.1:0/755088435 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55cd4e164dc0 0x55cd4e1e23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:55.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.191+0000 7f7a08706740 1 --2- 172.21.15.1:0/755088435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd4e1644a0 0x55cd4e1e1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:55.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.191+0000 7f7a08706740 1 -- 172.21.15.1:0/755088435 >> 172.21.15.1:0/755088435 conn(0x55cd4e160a30 msgr2=0x55cd4e16db10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:55.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.191+0000 7f7a08706740 1 -- 172.21.15.1:0/755088435 shutdown_connections 2024-09-11T22:05:55.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.191+0000 7f7a08706740 1 -- 172.21.15.1:0/755088435 wait complete. 2024-09-11T22:05:55.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.191+0000 7f7a08706740 20 client.17541 trim_cache size 0 max 16384 2024-09-11T22:05:55.235 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-11T22:05:55.235 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T22:05:55.235 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-09-11T22:05:55.235 INFO:teuthology.orchestra.run.smithi001.stdout: "a4068e5d-93d7-44ce-98dd-791c2ba148cc": { 2024-09-11T22:05:55.235 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2024-09-11T22:05:55.235 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "a4068e5d-93d7-44ce-98dd-791c2ba148cc", 2024-09-11T22:05:55.235 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2024-09-11T22:05:55.235 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:05:55.235 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:05:55.236 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-11T22:05:55.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.192+0000 7f7a0632d640 1 -- 172.21.15.1:0/3994690361 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f79ec0a2cf0 msgr2=0x7f79ec0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:55.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.192+0000 7f7a0632d640 1 --2- 172.21.15.1:0/3994690361 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f79ec0a2cf0 0x7f79ec0a51b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f79f800a780 tx=0x7f79f800a4d0 comp rx=0 tx=0).stop 2024-09-11T22:05:55.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.192+0000 7f7a0632d640 1 -- 172.21.15.1:0/3994690361 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7a00100a60 msgr2=0x7f7a001973e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:05:55.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.192+0000 7f7a0632d640 1 --2- 172.21.15.1:0/3994690361 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7a00100a60 0x7f7a001973e0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f79f0009970 tx=0x7f79f0009e30 comp rx=0 tx=0).stop 2024-09-11T22:05:55.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.192+0000 7f7a0632d640 1 -- 172.21.15.1:0/3994690361 shutdown_connections 2024-09-11T22:05:55.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.192+0000 7f7a0632d640 1 --2- 172.21.15.1:0/3994690361 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f79ec0a2cf0 0x7f79ec0a51b0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:55.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.192+0000 7f7a0632d640 1 --2- 172.21.15.1:0/3994690361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a00106a60 0x7f7a0019bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:55.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.192+0000 7f7a0632d640 1 --2- 172.21.15.1:0/3994690361 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7a00101400 0x7f7a00197920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:55.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.192+0000 7f7a0632d640 1 --2- 172.21.15.1:0/3994690361 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7a00100a60 0x7f7a001973e0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:05:55.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.192+0000 7f7a0632d640 1 -- 172.21.15.1:0/3994690361 >> 172.21.15.1:0/3994690361 conn(0x7f7a000fc780 msgr2=0x7f7a000fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:05:55.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.192+0000 7f7a0632d640 1 -- 172.21.15.1:0/3994690361 shutdown_connections 2024-09-11T22:05:55.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:05:55.192+0000 7f7a0632d640 1 -- 172.21.15.1:0/3994690361 wait complete. 2024-09-11T22:05:55.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:54 smithi053 ceph-mon[34908]: pgmap v271: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 KiB/s wr, 18 op/s 2024-09-11T22:05:55.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:54 smithi053 ceph-mon[34908]: Health check failed: 1 MDSs are read only (MDS_READ_ONLY) 2024-09-11T22:05:55.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:54 smithi053 ceph-mon[34908]: mds.? [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] up:active 2024-09-11T22:05:55.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:54 smithi053 ceph-mon[34908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:05:55.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:54 smithi161 ceph-mon[35615]: pgmap v271: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 KiB/s wr, 18 op/s 2024-09-11T22:05:55.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:54 smithi161 ceph-mon[35615]: Health check failed: 1 MDSs are read only (MDS_READ_ONLY) 2024-09-11T22:05:55.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:54 smithi161 ceph-mon[35615]: mds.? [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] up:active 2024-09-11T22:05:55.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:54 smithi161 ceph-mon[35615]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:05:55.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:54 smithi001 ceph-mon[30935]: pgmap v271: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 KiB/s wr, 18 op/s 2024-09-11T22:05:55.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:54 smithi001 ceph-mon[30935]: Health check failed: 1 MDSs are read only (MDS_READ_ONLY) 2024-09-11T22:05:55.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:54 smithi001 ceph-mon[30935]: mds.? [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] up:active 2024-09-11T22:05:55.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:54 smithi001 ceph-mon[30935]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:05:56.196 INFO:tasks.cephfs.filesystem:scrub status for tag:a4068e5d-93d7-44ce-98dd-791c2ba148cc - {'path': '/', 'tag': 'a4068e5d-93d7-44ce-98dd-791c2ba148cc', 'options': 'recursive,force'} 2024-09-11T22:05:57.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:56 smithi053 ceph-mon[34908]: scrub summary: idle 2024-09-11T22:05:57.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:56 smithi053 ceph-mon[34908]: pgmap v272: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-09-11T22:05:57.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:56 smithi161 ceph-mon[35615]: scrub summary: idle 2024-09-11T22:05:57.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:56 smithi161 ceph-mon[35615]: pgmap v272: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-09-11T22:05:57.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:56 smithi001 ceph-mon[30935]: scrub summary: idle 2024-09-11T22:05:57.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:56 smithi001 ceph-mon[30935]: pgmap v272: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-09-11T22:05:59.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:05:58 smithi053 ceph-mon[34908]: pgmap v273: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-09-11T22:05:59.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:05:58 smithi161 ceph-mon[35615]: pgmap v273: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-09-11T22:05:59.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:05:58 smithi001 ceph-mon[30935]: pgmap v273: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-09-11T22:06:01.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:06:01 smithi053 ceph-mon[34908]: pgmap v274: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 21 op/s 2024-09-11T22:06:01.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:06:01 smithi161 ceph-mon[35615]: pgmap v274: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 21 op/s 2024-09-11T22:06:01.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:06:01 smithi001 ceph-mon[30935]: pgmap v274: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 21 op/s 2024-09-11T22:06:03.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:06:03 smithi053 ceph-mon[34908]: pgmap v275: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-11T22:06:03.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:06:03 smithi161 ceph-mon[35615]: pgmap v275: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-11T22:06:03.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:06:03 smithi001 ceph-mon[30935]: pgmap v275: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-11T22:06:05.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:06:05 smithi053 ceph-mon[34908]: pgmap v276: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-11T22:06:05.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:06:05 smithi161 ceph-mon[35615]: pgmap v276: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-11T22:06:05.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:06:05 smithi001 ceph-mon[30935]: pgmap v276: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-11T22:06:07.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:06:07 smithi053 ceph-mon[34908]: pgmap v277: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 255 B/s wr, 29 op/s 2024-09-11T22:06:07.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:06:07 smithi161 ceph-mon[35615]: pgmap v277: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 255 B/s wr, 29 op/s 2024-09-11T22:06:07.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:06:07 smithi001 ceph-mon[30935]: pgmap v277: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 255 B/s wr, 29 op/s 2024-09-11T22:06:07.666 INFO:tasks.quiescer.fs.[cephfs]:Total duration (38.8) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-11T22:06:07.666 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.4 2024-09-11T22:06:07.667 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:06:07.941 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:06:09.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:06:09 smithi053 ceph-mon[34908]: pgmap v278: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 0 B/s wr, 22 op/s 2024-09-11T22:06:09.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:06:09 smithi161 ceph-mon[35615]: pgmap v278: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 0 B/s wr, 22 op/s 2024-09-11T22:06:09.362 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:06:09 smithi001 ceph-mon[30935]: pgmap v278: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 0 B/s wr, 22 op/s 2024-09-11T22:06:11.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:10.785+0000 7f6c62701640 1 -- 172.21.15.1:0/2638331994 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c5c102570 msgr2=0x7f6c5c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:11.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:10.785+0000 7f6c62701640 1 --2- 172.21.15.1:0/2638331994 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c5c102570 0x7f6c5c1029b0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f6c44009900 tx=0x7f6c4402a480 comp rx=0 tx=0).stop 2024-09-11T22:06:11.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:10.785+0000 7f6c62701640 1 -- 172.21.15.1:0/2638331994 shutdown_connections 2024-09-11T22:06:11.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:10.785+0000 7f6c62701640 1 --2- 172.21.15.1:0/2638331994 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c5c102ef0 0x7f6c5c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:11.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:10.785+0000 7f6c62701640 1 --2- 172.21.15.1:0/2638331994 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c5c102570 0x7f6c5c1029b0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:11.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:10.785+0000 7f6c62701640 1 --2- 172.21.15.1:0/2638331994 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c5c100e20 0x7f6c5c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:11.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:10.785+0000 7f6c62701640 1 -- 172.21.15.1:0/2638331994 >> 172.21.15.1:0/2638331994 conn(0x7f6c5c0fc7d0 msgr2=0x7f6c5c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:11.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:10.785+0000 7f6c62701640 1 -- 172.21.15.1:0/2638331994 shutdown_connections 2024-09-11T22:06:11.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:10.786+0000 7f6c62701640 1 -- 172.21.15.1:0/2638331994 wait complete. 2024-09-11T22:06:11.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:10.786+0000 7f6c62701640 1 Processor -- start 2024-09-11T22:06:11.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:10.786+0000 7f6c62701640 1 -- start start 2024-09-11T22:06:11.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:10.786+0000 7f6c62701640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c5c100e20 0x7f6c5c1984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:11.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:10.786+0000 7f6c62701640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c5c102570 0x7f6c5c198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:11.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:10.786+0000 7f6c62701640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c5c102ef0 0x7f6c5c19cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:11.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:10.786+0000 7f6c62701640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6c5c110f80 con 0x7f6c5c100e20 2024-09-11T22:06:11.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:10.786+0000 7f6c62701640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f6c5c110e00 con 0x7f6c5c102570 2024-09-11T22:06:11.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:10.786+0000 7f6c62701640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f6c5c111100 con 0x7f6c5c102ef0 2024-09-11T22:06:11.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:10.787+0000 7f6c5bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c5c100e20 0x7f6c5c1984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:11.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:10.787+0000 7f6c5bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c5c100e20 0x7f6c5c1984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36392/0 (socket says 172.21.15.1:36392) 2024-09-11T22:06:11.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:10.787+0000 7f6c5bfff640 1 -- 172.21.15.1:0/3973279709 learned_addr learned my addr 172.21.15.1:0/3973279709 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:06:11.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:10.787+0000 7f6c60c9e640 1 --2- 172.21.15.1:0/3973279709 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c5c102ef0 0x7f6c5c19cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:11.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:10.787+0000 7f6c53fff640 1 --2- 172.21.15.1:0/3973279709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c5c102570 0x7f6c5c198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:11.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:10.787+0000 7f6c5bfff640 1 -- 172.21.15.1:0/3973279709 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c5c102ef0 msgr2=0x7f6c5c19cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:11.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:10.787+0000 7f6c5bfff640 1 --2- 172.21.15.1:0/3973279709 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c5c102ef0 0x7f6c5c19cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:11.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:10.787+0000 7f6c5bfff640 1 -- 172.21.15.1:0/3973279709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c5c102570 msgr2=0x7f6c5c198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:11.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:10.787+0000 7f6c5bfff640 1 --2- 172.21.15.1:0/3973279709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c5c102570 0x7f6c5c198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:11.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:10.787+0000 7f6c5bfff640 1 -- 172.21.15.1:0/3973279709 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c5c19d480 con 0x7f6c5c100e20 2024-09-11T22:06:11.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:10.787+0000 7f6c5bfff640 1 --2- 172.21.15.1:0/3973279709 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c5c100e20 0x7f6c5c1984d0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c00d770 tx=0x7f6c4c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:11.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:10.787+0000 7f6c59ffb640 1 -- 172.21.15.1:0/3973279709 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c017070 con 0x7f6c5c100e20 2024-09-11T22:06:11.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:10.787+0000 7f6c62701640 1 -- 172.21.15.1:0/3973279709 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c5c19d770 con 0x7f6c5c100e20 2024-09-11T22:06:11.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:10.787+0000 7f6c59ffb640 1 -- 172.21.15.1:0/3973279709 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c4c005500 con 0x7f6c5c100e20 2024-09-11T22:06:11.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:10.787+0000 7f6c62701640 1 -- 172.21.15.1:0/3973279709 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c5c1a5050 con 0x7f6c5c100e20 2024-09-11T22:06:11.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:10.787+0000 7f6c59ffb640 1 -- 172.21.15.1:0/3973279709 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c004e00 con 0x7f6c5c100e20 2024-09-11T22:06:11.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:10.788+0000 7f6c62701640 1 -- 172.21.15.1:0/3973279709 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c5c101200 con 0x7f6c5c100e20 2024-09-11T22:06:11.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:10.790+0000 7f6c59ffb640 1 -- 172.21.15.1:0/3973279709 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f6c4c004fa0 con 0x7f6c5c100e20 2024-09-11T22:06:11.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:10.790+0000 7f6c59ffb640 1 --2- 172.21.15.1:0/3973279709 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f6c480a2d70 0x7f6c480a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:11.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:10.790+0000 7f6c53fff640 1 --2- 172.21.15.1:0/3973279709 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f6c480a2d70 0x7f6c480a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:11.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:10.790+0000 7f6c59ffb640 1 -- 172.21.15.1:0/3973279709 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7f6c4c0cf370 con 0x7f6c5c100e20 2024-09-11T22:06:11.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:10.791+0000 7f6c53fff640 1 --2- 172.21.15.1:0/3973279709 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f6c480a2d70 0x7f6c480a5230 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c0ff060 tx=0x7f6c44005a20 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:11.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:10.794+0000 7f6c59ffb640 1 -- 172.21.15.1:0/3973279709 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6c4c09bfd0 con 0x7f6c5c100e20 2024-09-11T22:06:11.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:11.009+0000 7f6c62701640 1 -- 172.21.15.1:0/3973279709 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6c5c199830 con 0x7f6c5c100e20 2024-09-11T22:06:11.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:11.010+0000 7f6c59ffb640 1 -- 172.21.15.1:0/3973279709 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+11986 (secure 0 0 0) 0x7f6c4c09b8d0 con 0x7f6c5c100e20 2024-09-11T22:06:11.264 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 46 2024-09-11T22:06:11.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:11.012+0000 7f6c62701640 1 -- 172.21.15.1:0/3973279709 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f6c480a2d70 msgr2=0x7f6c480a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:11.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:11.012+0000 7f6c62701640 1 --2- 172.21.15.1:0/3973279709 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f6c480a2d70 0x7f6c480a5230 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c0ff060 tx=0x7f6c44005a20 comp rx=0 tx=0).stop 2024-09-11T22:06:11.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:11.012+0000 7f6c62701640 1 -- 172.21.15.1:0/3973279709 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c5c100e20 msgr2=0x7f6c5c1984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:11.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:11.012+0000 7f6c62701640 1 --2- 172.21.15.1:0/3973279709 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c5c100e20 0x7f6c5c1984d0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c00d770 tx=0x7f6c4c00dc30 comp rx=0 tx=0).stop 2024-09-11T22:06:11.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:11.012+0000 7f6c62701640 1 -- 172.21.15.1:0/3973279709 shutdown_connections 2024-09-11T22:06:11.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:11.012+0000 7f6c62701640 1 --2- 172.21.15.1:0/3973279709 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f6c480a2d70 0x7f6c480a5230 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:11.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:11.012+0000 7f6c62701640 1 --2- 172.21.15.1:0/3973279709 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c5c102ef0 0x7f6c5c19cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:11.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:11.012+0000 7f6c62701640 1 --2- 172.21.15.1:0/3973279709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c5c102570 0x7f6c5c198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:11.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:11.012+0000 7f6c62701640 1 --2- 172.21.15.1:0/3973279709 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c5c100e20 0x7f6c5c1984d0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:11.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:11.012+0000 7f6c62701640 1 -- 172.21.15.1:0/3973279709 >> 172.21.15.1:0/3973279709 conn(0x7f6c5c0fc7d0 msgr2=0x7f6c5c06a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:11.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:11.012+0000 7f6c62701640 1 -- 172.21.15.1:0/3973279709 shutdown_connections 2024-09-11T22:06:11.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:11.013+0000 7f6c62701640 1 -- 172.21.15.1:0/3973279709 wait complete. 2024-09-11T22:06:11.265 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:06:11 smithi001 ceph-mon[30935]: pgmap v279: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 0 B/s wr, 27 op/s 2024-09-11T22:06:11.266 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:06:11 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/3973279709' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:06:11.266 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T22:06:11.266 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":46,"btime":"2024-09-11T22:05:53:986700+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/421542832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":421542832},{"type":"v1","addr":"172.21.15.1:6835","nonce":421542832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2836864406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2836864406},{"type":"v1","addr":"172.21.15.1:6837","nonce":2836864406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":46,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T21:55:20.791263+0000","modified":"2024-09-11T22:05:53.152083+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24484,"mds_2":24467,"mds_3":24466,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14595":{"gid":14595,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/407834672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":407834672},{"type":"v1","addr":"172.21.15.1:6839","nonce":407834672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/1068827678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":1068827678},{"type":"v1","addr":"172.21.15.1:6841","nonce":1068827678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/872410024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":872410024},{"type":"v1","addr":"172.21.15.161:6833","nonce":872410024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/1648944570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1648944570},{"type":"v1","addr":"172.21.15.53:6833","nonce":1648944570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/2126933993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2126933993},{"type":"v1","addr":"172.21.15.53:6835","nonce":2126933993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.161:6835/828381592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":828381592},{"type":"v1","addr":"172.21.15.161:6835","nonce":828381592}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.161:6837/1799318656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1799318656},{"type":"v1","addr":"172.21.15.161:6837","nonce":1799318656}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.53:6837/3457131830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3457131830},{"type":"v1","addr":"172.21.15.53:6837","nonce":3457131830}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/455365383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":455365383},{"type":"v1","addr":"172.21.15.161:6839","nonce":455365383}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":162,"addr":"172.21.15.53:6839/1378584447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1378584447},{"type":"v1","addr":"172.21.15.53:6839","nonce":1378584447}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24460,24466,24467,24484,24494]},"id":1}]} 2024-09-11T22:06:11.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:06:11 smithi053 ceph-mon[34908]: pgmap v279: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 0 B/s wr, 27 op/s 2024-09-11T22:06:11.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:06:11 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/3973279709' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:06:11.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:06:11 smithi161 ceph-mon[35615]: pgmap v279: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 0 B/s wr, 27 op/s 2024-09-11T22:06:11.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:06:11 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/3973279709' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:06:13.118 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:06:13 smithi001 ceph-mon[30935]: pgmap v280: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 0 B/s wr, 22 op/s 2024-09-11T22:06:13.153 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db / --timeout 90 --expiration 139.4 --await-for 45.0' 2024-09-11T22:06:13.154 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db / --timeout 90 --expiration 139.4 --await-for 45.0 2024-09-11T22:06:13.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:06:13 smithi053 ceph-mon[34908]: pgmap v280: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 0 B/s wr, 22 op/s 2024-09-11T22:06:13.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:06:13 smithi161 ceph-mon[35615]: pgmap v280: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 0 B/s wr, 22 op/s 2024-09-11T22:06:13.422 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:06:13.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.945+0000 7faf65fef640 1 -- 172.21.15.1:0/3500208968 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf60103ff0 msgr2=0x7faf601043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:13.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.945+0000 7faf65fef640 1 --2- 172.21.15.1:0/3500208968 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf60103ff0 0x7faf601043d0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7faf50009900 tx=0x7faf5002a410 comp rx=0 tx=0).stop 2024-09-11T22:06:13.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.945+0000 7faf65fef640 1 -- 172.21.15.1:0/3500208968 shutdown_connections 2024-09-11T22:06:13.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.945+0000 7faf65fef640 1 --2- 172.21.15.1:0/3500208968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faf60109120 0x7faf60110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:13.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.945+0000 7faf65fef640 1 --2- 172.21.15.1:0/3500208968 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf601049a0 0x7faf601089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:13.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.945+0000 7faf65fef640 1 --2- 172.21.15.1:0/3500208968 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf60103ff0 0x7faf601043d0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:13.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.945+0000 7faf65fef640 1 -- 172.21.15.1:0/3500208968 >> 172.21.15.1:0/3500208968 conn(0x7faf600ffa90 msgr2=0x7faf60101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:13.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.945+0000 7faf65fef640 1 -- 172.21.15.1:0/3500208968 shutdown_connections 2024-09-11T22:06:13.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.945+0000 7faf65fef640 1 -- 172.21.15.1:0/3500208968 wait complete. 2024-09-11T22:06:13.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.946+0000 7faf65fef640 1 Processor -- start 2024-09-11T22:06:13.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.946+0000 7faf65fef640 1 -- start start 2024-09-11T22:06:13.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.946+0000 7faf65fef640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faf60103ff0 0x7faf6019a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:13.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.946+0000 7faf65fef640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf601049a0 0x7faf6019ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:13.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.946+0000 7faf65fef640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf60109120 0x7faf6019ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:13.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.946+0000 7faf65fef640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7faf601141a0 con 0x7faf601049a0 2024-09-11T22:06:13.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.946+0000 7faf65fef640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7faf60114020 con 0x7faf60109120 2024-09-11T22:06:13.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.946+0000 7faf65fef640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7faf60114320 con 0x7faf60103ff0 2024-09-11T22:06:13.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.946+0000 7faf5effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf601049a0 0x7faf6019ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:13.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.946+0000 7faf5effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf601049a0 0x7faf6019ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36406/0 (socket says 172.21.15.1:36406) 2024-09-11T22:06:13.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.946+0000 7faf5effd640 1 -- 172.21.15.1:0/921117382 learned_addr learned my addr 172.21.15.1:0/921117382 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:06:13.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.946+0000 7faf5f7fe640 1 --2- 172.21.15.1:0/921117382 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faf60103ff0 0x7faf6019a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:13.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.946+0000 7faf5ffff640 1 --2- 172.21.15.1:0/921117382 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf60109120 0x7faf6019ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:13.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.946+0000 7faf5effd640 1 -- 172.21.15.1:0/921117382 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faf60103ff0 msgr2=0x7faf6019a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:13.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.947+0000 7faf5effd640 1 --2- 172.21.15.1:0/921117382 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faf60103ff0 0x7faf6019a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:13.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.947+0000 7faf5effd640 1 -- 172.21.15.1:0/921117382 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf60109120 msgr2=0x7faf6019ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:13.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.947+0000 7faf5effd640 1 --2- 172.21.15.1:0/921117382 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf60109120 0x7faf6019ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:13.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.947+0000 7faf5effd640 1 -- 172.21.15.1:0/921117382 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf6019f5f0 con 0x7faf601049a0 2024-09-11T22:06:13.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.947+0000 7faf5f7fe640 1 --2- 172.21.15.1:0/921117382 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faf60103ff0 0x7faf6019a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:06:13.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.947+0000 7faf5ffff640 1 --2- 172.21.15.1:0/921117382 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf60109120 0x7faf6019ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:06:13.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.947+0000 7faf5effd640 1 --2- 172.21.15.1:0/921117382 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf601049a0 0x7faf6019ab80 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7faf5400c890 tx=0x7faf5400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:13.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.947+0000 7faf5cff9640 1 -- 172.21.15.1:0/921117382 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf54004220 con 0x7faf601049a0 2024-09-11T22:06:13.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.947+0000 7faf65fef640 1 -- 172.21.15.1:0/921117382 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf6019f8e0 con 0x7faf601049a0 2024-09-11T22:06:13.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.947+0000 7faf5cff9640 1 -- 172.21.15.1:0/921117382 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf540043c0 con 0x7faf601049a0 2024-09-11T22:06:13.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.947+0000 7faf5cff9640 1 -- 172.21.15.1:0/921117382 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf54004a00 con 0x7faf601049a0 2024-09-11T22:06:13.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.947+0000 7faf65fef640 1 -- 172.21.15.1:0/921117382 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf601a71c0 con 0x7faf601049a0 2024-09-11T22:06:13.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.949+0000 7faf5cff9640 1 -- 172.21.15.1:0/921117382 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7faf540079b0 con 0x7faf601049a0 2024-09-11T22:06:13.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.950+0000 7faf683c8740 1 Processor -- start 2024-09-11T22:06:13.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.950+0000 7faf683c8740 1 -- start start 2024-09-11T22:06:13.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.950+0000 7faf683c8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611a25e1760 0x5611a25e1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:13.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.950+0000 7faf683c8740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5611a25e2080 0x5611a25ec400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:13.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.950+0000 7faf683c8740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611a25ec940 0x5611a25eed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:13.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.950+0000 7faf683c8740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5611a25472a0 con 0x5611a25ec940 2024-09-11T22:06:13.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.950+0000 7faf683c8740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5611a24eeb10 con 0x5611a25e2080 2024-09-11T22:06:13.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.950+0000 7faf683c8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5611a2543260 con 0x5611a25e1760 2024-09-11T22:06:13.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.950+0000 7faf5effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611a25ec940 0x5611a25eed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:13.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.950+0000 7faf5effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611a25ec940 0x5611a25eed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36416/0 (socket says 172.21.15.1:36416) 2024-09-11T22:06:13.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.950+0000 7faf5effd640 1 -- 172.21.15.1:0/2588687436 learned_addr learned my addr 172.21.15.1:0/2588687436 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:06:13.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.950+0000 7faf5cff9640 1 --2- 172.21.15.1:0/921117382 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7faf480a2cf0 0x7faf480a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:13.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.950+0000 7faf5f7fe640 1 --2- 172.21.15.1:0/2588687436 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5611a25e2080 0x5611a25ec400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:13.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.950+0000 7faf5ffff640 1 --2- 172.21.15.1:0/2588687436 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611a25e1760 0x5611a25e1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:13.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.950+0000 7faf5ffff640 1 --2- 172.21.15.1:0/921117382 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7faf480a2cf0 0x7faf480a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:13.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.951+0000 7faf5effd640 1 -- 172.21.15.1:0/2588687436 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611a25e1760 msgr2=0x5611a25e1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:13.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.951+0000 7faf5effd640 1 --2- 172.21.15.1:0/2588687436 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611a25e1760 0x5611a25e1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:13.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.951+0000 7faf5effd640 1 -- 172.21.15.1:0/2588687436 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5611a25e2080 msgr2=0x5611a25ec400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:13.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.951+0000 7faf5effd640 1 --2- 172.21.15.1:0/2588687436 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5611a25e2080 0x5611a25ec400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:13.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.951+0000 7faf5cff9640 1 -- 172.21.15.1:0/921117382 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7faf540cf240 con 0x7faf601049a0 2024-09-11T22:06:13.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.951+0000 7faf5effd640 1 -- 172.21.15.1:0/2588687436 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611a22b8b90 con 0x5611a25ec940 2024-09-11T22:06:13.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.951+0000 7faf5effd640 1 --2- 172.21.15.1:0/2588687436 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611a25ec940 0x5611a25eed30 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7faf5409b6b0 tx=0x7faf54005300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1366a3212150b6b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:13.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.951+0000 7faf45ffb640 1 -- 172.21.15.1:0/2588687436 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf5401d070 con 0x5611a25ec940 2024-09-11T22:06:13.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.951+0000 7faf5ffff640 1 --2- 172.21.15.1:0/921117382 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7faf480a2cf0 0x7faf480a51b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7faf58009440 tx=0x7faf5800c040 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:13.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.951+0000 7faf45ffb640 1 -- 172.21.15.1:0/2588687436 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf540860f0 con 0x5611a25ec940 2024-09-11T22:06:13.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.951+0000 7faf45ffb640 1 -- 172.21.15.1:0/2588687436 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf540ccbd0 con 0x5611a25ec940 2024-09-11T22:06:13.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.952+0000 7faf683c8740 1 -- 172.21.15.1:0/2588687436 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611a25ec940 msgr2=0x5611a25eed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:13.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.952+0000 7faf683c8740 1 --2- 172.21.15.1:0/2588687436 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611a25ec940 0x5611a25eed30 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7faf5409b6b0 tx=0x7faf54005300 comp rx=0 tx=0).stop 2024-09-11T22:06:13.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.952+0000 7faf683c8740 1 -- 172.21.15.1:0/2588687436 shutdown_connections 2024-09-11T22:06:13.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.952+0000 7faf683c8740 1 --2- 172.21.15.1:0/2588687436 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611a25ec940 0x5611a25eed30 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:13.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.952+0000 7faf683c8740 1 --2- 172.21.15.1:0/2588687436 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5611a25e2080 0x5611a25ec400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:13.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.952+0000 7faf683c8740 1 --2- 172.21.15.1:0/2588687436 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611a25e1760 0x5611a25e1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:13.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.952+0000 7faf683c8740 1 -- 172.21.15.1:0/2588687436 >> 172.21.15.1:0/2588687436 conn(0x5611a25ddcf0 msgr2=0x5611a25de0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:13.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.952+0000 7faf683c8740 1 -- 172.21.15.1:0/2588687436 shutdown_connections 2024-09-11T22:06:13.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.952+0000 7faf683c8740 1 -- 172.21.15.1:0/2588687436 wait complete. 2024-09-11T22:06:13.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.952+0000 7faf683c8740 1 -- start start 2024-09-11T22:06:13.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.952+0000 7faf683c8740 1 Processor -- start 2024-09-11T22:06:13.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.953+0000 7faf683c8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611a25e1760 0x5611a25e97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:13.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.953+0000 7faf683c8740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611a25e2080 0x5611a25e9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:13.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.953+0000 7faf683c8740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5611a25ec940 0x5611a25ea240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:13.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.953+0000 7faf683c8740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5611a25ef510 con 0x5611a25e2080 2024-09-11T22:06:13.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.953+0000 7faf683c8740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x5611a24cf6e0 con 0x5611a25ec940 2024-09-11T22:06:13.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.953+0000 7faf683c8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x5611a25ef690 con 0x5611a25e1760 2024-09-11T22:06:13.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.953+0000 7faf683c8740 10 client.0 _ll_register_callbacks cb 0x5611a24f3a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:06:13.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.953+0000 7faf5effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611a25e2080 0x5611a25e9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:13.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.953+0000 7faf5effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611a25e2080 0x5611a25e9d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36430/0 (socket says 172.21.15.1:36430) 2024-09-11T22:06:13.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.953+0000 7faf5effd640 1 -- 172.21.15.1:0/1256194688 learned_addr learned my addr 172.21.15.1:0/1256194688 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:06:13.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.953+0000 7faf5f7fe640 1 --2- 172.21.15.1:0/1256194688 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611a25e1760 0x5611a25e97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:13.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.953+0000 7faf5ffff640 1 --2- 172.21.15.1:0/1256194688 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5611a25ec940 0x5611a25ea240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:13.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.953+0000 7faf5effd640 1 -- 172.21.15.1:0/1256194688 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611a25e1760 msgr2=0x5611a25e97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:13.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.953+0000 7faf5effd640 1 --2- 172.21.15.1:0/1256194688 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611a25e1760 0x5611a25e97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:13.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.953+0000 7faf5effd640 1 -- 172.21.15.1:0/1256194688 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5611a25ec940 msgr2=0x5611a25ea240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:13.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.953+0000 7faf5effd640 1 --2- 172.21.15.1:0/1256194688 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5611a25ec940 0x5611a25ea240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:13.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.953+0000 7faf5effd640 1 -- 172.21.15.1:0/1256194688 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611a25f89b0 con 0x5611a25e2080 2024-09-11T22:06:13.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.953+0000 7faf5f7fe640 1 --2- 172.21.15.1:0/1256194688 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611a25e1760 0x5611a25e97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:06:13.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.953+0000 7faf5ffff640 1 --2- 172.21.15.1:0/1256194688 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5611a25ec940 0x5611a25ea240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:06:13.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.953+0000 7faf5effd640 1 --2- 172.21.15.1:0/1256194688 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611a25e2080 0x5611a25e9d00 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7faf5409b420 tx=0x7faf540a4500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5065ad7d4563570 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:13.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.953+0000 7faf3effd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:06:13.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.954+0000 7faf3effd640 1 -- 172.21.15.1:0/1256194688 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf5401d040 con 0x5611a25e2080 2024-09-11T22:06:13.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.954+0000 7faf3effd640 1 -- 172.21.15.1:0/1256194688 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7faf540d2be0 con 0x5611a25e2080 2024-09-11T22:06:13.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.954+0000 7faf683c8740 1 -- 172.21.15.1:0/1256194688 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf540860f0 con 0x5611a25e2080 2024-09-11T22:06:13.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.954+0000 7faf3effd640 1 -- 172.21.15.1:0/1256194688 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7faf5409be20 con 0x5611a25e2080 2024-09-11T22:06:13.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.954+0000 7faf683c8740 10 client.17559 fetch_fsmap learned FSMap version 46 2024-09-11T22:06:13.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.954+0000 7faf683c8740 1 -- 172.21.15.1:0/1256194688 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5611a25fb320 con 0x5611a25e2080 2024-09-11T22:06:13.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.954+0000 7faf3effd640 1 -- 172.21.15.1:0/1256194688 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf540860f0 con 0x5611a25e2080 2024-09-11T22:06:13.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.954+0000 7faf3effd640 1 -- 172.21.15.1:0/1256194688 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 46) ==== 8442+0+0 (secure 0 0 0) 0x7faf5409ab70 con 0x5611a25e2080 2024-09-11T22:06:13.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.954+0000 7faf683c8740 10 client.17559 fetch_fsmap finished waiting for FSMap version 46 2024-09-11T22:06:13.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.954+0000 7faf683c8740 10 client.17559 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-11T22:06:13.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.954+0000 7faf683c8740 20 client.17559 populate_metadata read hostname 'smithi001' 2024-09-11T22:06:13.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.954+0000 7faf683c8740 1 --2- 172.21.15.1:0/1256194688 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5611a25fc670 0x5611a25fea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:13.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.954+0000 7faf683c8740 4 client.17559 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:06:13.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.954+0000 7faf683c8740 1 -- 172.21.15.1:0/1256194688 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5611a25549d0 con 0x5611a25fc670 2024-09-11T22:06:13.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.955+0000 7faf5f7fe640 1 --2- 172.21.15.1:0/1256194688 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5611a25fc670 0x5611a25fea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:13.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.955+0000 7faf5f7fe640 1 --2- 172.21.15.1:0/1256194688 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5611a25fc670 0x5611a25fea60 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=123cd8c1688200cb server_cookie=21e09de33bb972aa in_seq=0 out_seq=0 2024-09-11T22:06:13.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.955+0000 7faf3effd640 10 client.17559 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:06:13.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.956+0000 7faf3effd640 1 -- 172.21.15.1:0/1256194688 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5611a25549d0 con 0x5611a25fc670 2024-09-11T22:06:13.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.956+0000 7faf3effd640 10 client.17559 handle_command_reply: tid=0 2024-09-11T22:06:13.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.956+0000 7faf683c8740 1 client.17559 shutdown 2024-09-11T22:06:13.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.956+0000 7faf683c8740 1 -- 172.21.15.1:0/1256194688 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611a25e2080 msgr2=0x5611a25e9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:13.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.956+0000 7faf683c8740 1 --2- 172.21.15.1:0/1256194688 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611a25e2080 0x5611a25e9d00 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7faf5409b420 tx=0x7faf540a4500 comp rx=0 tx=0).stop 2024-09-11T22:06:13.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.956+0000 7faf683c8740 1 -- 172.21.15.1:0/1256194688 shutdown_connections 2024-09-11T22:06:13.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.956+0000 7faf683c8740 1 --2- 172.21.15.1:0/1256194688 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5611a25fc670 0x5611a25fea60 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:13.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.956+0000 7faf683c8740 1 --2- 172.21.15.1:0/1256194688 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5611a25ec940 0x5611a25ea240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:13.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.956+0000 7faf683c8740 1 --2- 172.21.15.1:0/1256194688 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611a25e2080 0x5611a25e9d00 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:13.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.956+0000 7faf683c8740 1 --2- 172.21.15.1:0/1256194688 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611a25e1760 0x5611a25e97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:13.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.956+0000 7faf683c8740 1 -- 172.21.15.1:0/1256194688 >> 172.21.15.1:0/1256194688 conn(0x5611a25ddcf0 msgr2=0x5611a25e3a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:13.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.956+0000 7faf3effd640 0 client.17559 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:06:13.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.956+0000 7faf683c8740 1 -- 172.21.15.1:0/1256194688 shutdown_connections 2024-09-11T22:06:13.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.956+0000 7faf683c8740 1 -- 172.21.15.1:0/1256194688 wait complete. 2024-09-11T22:06:13.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.956+0000 7faf683c8740 20 client.17559 trim_cache size 0 max 16384 2024-09-11T22:06:13.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.973+0000 7faf683c8740 1 Processor -- start 2024-09-11T22:06:13.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.973+0000 7faf683c8740 1 -- start start 2024-09-11T22:06:13.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.974+0000 7faf683c8740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5611a25e1760 0x5611a25dcd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:13.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.974+0000 7faf683c8740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611a25e2080 0x5611a25ea090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:13.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.974+0000 7faf683c8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611a25ec940 0x5611a2604000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:13.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.974+0000 7faf683c8740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7faf601afcf0 con 0x5611a25e2080 2024-09-11T22:06:13.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.974+0000 7faf683c8740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7faf601b1b30 con 0x5611a25e1760 2024-09-11T22:06:13.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.974+0000 7faf683c8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7faf601b0480 con 0x5611a25ec940 2024-09-11T22:06:13.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.974+0000 7faf5effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611a25e2080 0x5611a25ea090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:13.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.974+0000 7faf5effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611a25e2080 0x5611a25ea090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36436/0 (socket says 172.21.15.1:36436) 2024-09-11T22:06:13.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.974+0000 7faf5effd640 1 -- 172.21.15.1:0/1099656301 learned_addr learned my addr 172.21.15.1:0/1099656301 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:06:13.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.974+0000 7faf5f7fe640 1 --2- 172.21.15.1:0/1099656301 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5611a25e1760 0x5611a25dcd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:13.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.974+0000 7faf5ffff640 1 --2- 172.21.15.1:0/1099656301 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611a25ec940 0x5611a2604000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:13.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.974+0000 7faf5effd640 1 -- 172.21.15.1:0/1099656301 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611a25ec940 msgr2=0x5611a2604000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:13.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.974+0000 7faf5effd640 1 --2- 172.21.15.1:0/1099656301 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611a25ec940 0x5611a2604000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:13.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.974+0000 7faf5effd640 1 -- 172.21.15.1:0/1099656301 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5611a25e1760 msgr2=0x5611a25dcd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:13.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.974+0000 7faf5effd640 1 --2- 172.21.15.1:0/1099656301 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5611a25e1760 0x5611a25dcd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:13.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.974+0000 7faf5effd640 1 -- 172.21.15.1:0/1099656301 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611a25ddb20 con 0x5611a25e2080 2024-09-11T22:06:13.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.974+0000 7faf5ffff640 1 --2- 172.21.15.1:0/1099656301 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611a25ec940 0x5611a2604000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:06:13.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.974+0000 7faf5effd640 1 --2- 172.21.15.1:0/1099656301 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611a25e2080 0x5611a25ea090 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7faf54097b80 tx=0x7faf54097af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1229faefbb8bada9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:13.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.975+0000 7faf457fa640 1 -- 172.21.15.1:0/1099656301 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf5401d050 con 0x5611a25e2080 2024-09-11T22:06:13.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.975+0000 7faf457fa640 1 -- 172.21.15.1:0/1099656301 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf540860f0 con 0x5611a25e2080 2024-09-11T22:06:13.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.975+0000 7faf457fa640 1 -- 172.21.15.1:0/1099656301 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf540a3f00 con 0x5611a25e2080 2024-09-11T22:06:13.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.975+0000 7faf683c8740 1 -- 172.21.15.1:0/1099656301 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611a25e2080 msgr2=0x5611a25ea090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:13.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.975+0000 7faf683c8740 1 --2- 172.21.15.1:0/1099656301 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611a25e2080 0x5611a25ea090 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7faf54097b80 tx=0x7faf54097af0 comp rx=0 tx=0).stop 2024-09-11T22:06:13.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.975+0000 7faf683c8740 1 -- 172.21.15.1:0/1099656301 shutdown_connections 2024-09-11T22:06:13.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.975+0000 7faf683c8740 1 --2- 172.21.15.1:0/1099656301 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611a25ec940 0x5611a2604000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:13.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.975+0000 7faf683c8740 1 --2- 172.21.15.1:0/1099656301 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611a25e2080 0x5611a25ea090 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:13.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.975+0000 7faf683c8740 1 --2- 172.21.15.1:0/1099656301 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5611a25e1760 0x5611a25dcd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:13.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.975+0000 7faf683c8740 1 -- 172.21.15.1:0/1099656301 >> 172.21.15.1:0/1099656301 conn(0x5611a25ddcf0 msgr2=0x5611a25dc650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:13.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.975+0000 7faf683c8740 1 -- 172.21.15.1:0/1099656301 shutdown_connections 2024-09-11T22:06:13.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.975+0000 7faf683c8740 1 -- 172.21.15.1:0/1099656301 wait complete. 2024-09-11T22:06:13.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.976+0000 7faf683c8740 1 -- start start 2024-09-11T22:06:13.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.976+0000 7faf683c8740 1 Processor -- start 2024-09-11T22:06:13.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.976+0000 7faf683c8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611a25e1760 0x5611a25f48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:13.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.976+0000 7faf683c8740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5611a25e2080 0x5611a25f4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:13.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.976+0000 7faf683c8740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611a25ec940 0x5611a25f18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:13.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.976+0000 7faf683c8740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7faf601b13a0 con 0x5611a25ec940 2024-09-11T22:06:13.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.976+0000 7faf683c8740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7faf601b0c10 con 0x5611a25e2080 2024-09-11T22:06:13.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.976+0000 7faf683c8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7faf601adcd0 con 0x5611a25e1760 2024-09-11T22:06:13.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.976+0000 7faf683c8740 10 client.0 _ll_register_callbacks cb 0x7faf5409b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:06:13.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.976+0000 7faf5ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611a25ec940 0x5611a25f18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:13.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.976+0000 7faf5ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611a25ec940 0x5611a25f18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36452/0 (socket says 172.21.15.1:36452) 2024-09-11T22:06:13.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.977+0000 7faf5ffff640 1 -- 172.21.15.1:0/887261551 learned_addr learned my addr 172.21.15.1:0/887261551 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:06:13.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.977+0000 7faf5f7fe640 1 --2- 172.21.15.1:0/887261551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611a25e1760 0x5611a25f48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:13.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.977+0000 7faf5effd640 1 --2- 172.21.15.1:0/887261551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5611a25e2080 0x5611a25f4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:13.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.977+0000 7faf5ffff640 1 -- 172.21.15.1:0/887261551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611a25e1760 msgr2=0x5611a25f48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:13.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.977+0000 7faf5ffff640 1 --2- 172.21.15.1:0/887261551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611a25e1760 0x5611a25f48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:13.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.977+0000 7faf5ffff640 1 -- 172.21.15.1:0/887261551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5611a25e2080 msgr2=0x5611a25f4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:13.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.977+0000 7faf5ffff640 1 --2- 172.21.15.1:0/887261551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5611a25e2080 0x5611a25f4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:13.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.977+0000 7faf5ffff640 1 -- 172.21.15.1:0/887261551 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611a25e45d0 con 0x5611a25ec940 2024-09-11T22:06:13.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.977+0000 7faf5ffff640 1 --2- 172.21.15.1:0/887261551 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611a25ec940 0x5611a25f18b0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7faf58017a30 tx=0x7faf58004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cd85f91cf7417ea server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:13.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.977+0000 7faf3effd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:06:13.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.977+0000 7faf3effd640 1 -- 172.21.15.1:0/887261551 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf58028020 con 0x5611a25ec940 2024-09-11T22:06:13.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.977+0000 7faf3effd640 1 -- 172.21.15.1:0/887261551 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7faf58016b50 con 0x5611a25ec940 2024-09-11T22:06:13.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.977+0000 7faf683c8740 1 -- 172.21.15.1:0/887261551 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf540860f0 con 0x5611a25ec940 2024-09-11T22:06:13.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.978+0000 7faf3effd640 1 -- 172.21.15.1:0/887261551 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7faf58033050 con 0x5611a25ec940 2024-09-11T22:06:13.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.978+0000 7faf683c8740 10 client.17565 fetch_fsmap learned FSMap version 46 2024-09-11T22:06:13.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.978+0000 7faf683c8740 1 -- 172.21.15.1:0/887261551 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5611a25d9260 con 0x5611a25ec940 2024-09-11T22:06:13.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.978+0000 7faf3effd640 1 -- 172.21.15.1:0/887261551 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf540860f0 con 0x5611a25ec940 2024-09-11T22:06:13.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.978+0000 7faf3effd640 1 -- 172.21.15.1:0/887261551 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 46) ==== 8442+0+0 (secure 0 0 0) 0x7faf5802c3a0 con 0x5611a25ec940 2024-09-11T22:06:13.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.978+0000 7faf683c8740 10 client.17565 fetch_fsmap finished waiting for FSMap version 46 2024-09-11T22:06:13.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.978+0000 7faf683c8740 10 client.17565 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-11T22:06:13.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.978+0000 7faf683c8740 20 client.17565 populate_metadata read hostname 'smithi001' 2024-09-11T22:06:13.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.978+0000 7faf683c8740 1 --2- 172.21.15.1:0/887261551 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5611a2667d70 0x5611a266a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:13.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.978+0000 7faf683c8740 4 client.17565 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.4, "await_for": 45.0}] 2024-09-11T22:06:13.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.978+0000 7faf683c8740 1 -- 172.21.15.1:0/887261551 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.4, "await_for": 45.0}) -- 0x5611a254ed20 con 0x5611a2667d70 2024-09-11T22:06:13.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.978+0000 7faf5f7fe640 1 --2- 172.21.15.1:0/887261551 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5611a2667d70 0x5611a266a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:13.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.979+0000 7faf5f7fe640 1 --2- 172.21.15.1:0/887261551 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5611a2667d70 0x5611a266a190 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa41e0a26d00d24e server_cookie=1910198f7bbc2d57 in_seq=0 out_seq=0 2024-09-11T22:06:13.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.979+0000 7faf3effd640 10 client.17565 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:06:13.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.988+0000 7faf3effd640 1 -- 172.21.15.1:0/887261551 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5611a254ed20 con 0x5611a2667d70 2024-09-11T22:06:13.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.988+0000 7faf3effd640 10 client.17565 handle_command_reply: tid=0 2024-09-11T22:06:13.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.988+0000 7faf683c8740 1 client.17565 shutdown 2024-09-11T22:06:13.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.988+0000 7faf683c8740 1 -- 172.21.15.1:0/887261551 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611a25ec940 msgr2=0x5611a25f18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:13.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.988+0000 7faf683c8740 1 --2- 172.21.15.1:0/887261551 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611a25ec940 0x5611a25f18b0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7faf58017a30 tx=0x7faf58004b50 comp rx=0 tx=0).stop 2024-09-11T22:06:13.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.988+0000 7faf683c8740 1 -- 172.21.15.1:0/887261551 shutdown_connections 2024-09-11T22:06:13.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.988+0000 7faf683c8740 1 --2- 172.21.15.1:0/887261551 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x5611a2667d70 0x5611a266a190 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:13.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.988+0000 7faf683c8740 1 --2- 172.21.15.1:0/887261551 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611a25ec940 0x5611a25f18b0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:13.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.988+0000 7faf683c8740 1 --2- 172.21.15.1:0/887261551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5611a25e2080 0x5611a25f4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:13.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.988+0000 7faf3effd640 0 client.17565 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:06:13.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.988+0000 7faf683c8740 1 --2- 172.21.15.1:0/887261551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x5611a25e1760 0x5611a25f48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:13.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.988+0000 7faf683c8740 1 -- 172.21.15.1:0/887261551 >> 172.21.15.1:0/887261551 conn(0x5611a25ddcf0 msgr2=0x5611a25dce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:13.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.988+0000 7faf683c8740 1 -- 172.21.15.1:0/887261551 shutdown_connections 2024-09-11T22:06:13.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.988+0000 7faf683c8740 1 -- 172.21.15.1:0/887261551 wait complete. 2024-09-11T22:06:13.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.988+0000 7faf683c8740 20 client.17565 trim_cache size 0 max 16384 2024-09-11T22:06:13.991 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-11T22:06:13.991 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 46, 2024-09-11T22:06:13.991 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24494, 2024-09-11T22:06:13.991 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 30, 2024-09-11T22:06:13.991 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-09-11T22:06:13.991 INFO:teuthology.orchestra.run.smithi001.stdout: "e9045ef6": { 2024-09-11T22:06:13.991 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 30, 2024-09-11T22:06:13.991 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-09-11T22:06:13.991 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-11T22:06:13.991 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-09-11T22:06:13.992 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-11T22:06:13.992 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:06:13.992 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 90.0, 2024-09-11T22:06:13.992 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 139.4, 2024-09-11T22:06:13.992 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-09-11T22:06:13.992 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-09-11T22:06:13.992 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-09-11T22:06:13.992 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-09-11T22:06:13.993 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-09-11T22:06:13.993 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-09-11T22:06:13.993 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:06:13.993 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:06:13.993 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:06:13.993 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:06:13.993 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:06:13.993 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-11T22:06:13.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.989+0000 7faf65fef640 1 -- 172.21.15.1:0/921117382 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7faf480a2cf0 msgr2=0x7faf480a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:13.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.989+0000 7faf65fef640 1 --2- 172.21.15.1:0/921117382 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7faf480a2cf0 0x7faf480a51b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7faf58009440 tx=0x7faf5800c040 comp rx=0 tx=0).stop 2024-09-11T22:06:13.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.989+0000 7faf65fef640 1 -- 172.21.15.1:0/921117382 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf601049a0 msgr2=0x7faf6019ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:13.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.989+0000 7faf65fef640 1 --2- 172.21.15.1:0/921117382 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf601049a0 0x7faf6019ab80 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7faf5400c890 tx=0x7faf5400cd50 comp rx=0 tx=0).stop 2024-09-11T22:06:13.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.989+0000 7faf65fef640 1 -- 172.21.15.1:0/921117382 shutdown_connections 2024-09-11T22:06:13.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.989+0000 7faf65fef640 1 --2- 172.21.15.1:0/921117382 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7faf480a2cf0 0x7faf480a51b0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:13.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.989+0000 7faf65fef640 1 --2- 172.21.15.1:0/921117382 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faf60109120 0x7faf6019ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:13.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.989+0000 7faf65fef640 1 --2- 172.21.15.1:0/921117382 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf601049a0 0x7faf6019ab80 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:13.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.989+0000 7faf65fef640 1 --2- 172.21.15.1:0/921117382 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faf60103ff0 0x7faf6019a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:13.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.989+0000 7faf65fef640 1 -- 172.21.15.1:0/921117382 >> 172.21.15.1:0/921117382 conn(0x7faf600ffa90 msgr2=0x7faf60100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:13.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.989+0000 7faf65fef640 1 -- 172.21.15.1:0/921117382 shutdown_connections 2024-09-11T22:06:13.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:13.989+0000 7faf65fef640 1 -- 172.21.15.1:0/921117382 wait complete. 2024-09-11T22:06:14.755 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'e9045ef6', quiesce took 7.1 seconds. Will release after: 12.299999999999999 2024-09-11T22:06:15.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:06:15 smithi053 ceph-mon[34908]: pgmap v281: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 0 B/s wr, 22 op/s 2024-09-11T22:06:15.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:06:15 smithi161 ceph-mon[35615]: pgmap v281: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 0 B/s wr, 22 op/s 2024-09-11T22:06:15.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:06:15 smithi001 ceph-mon[30935]: pgmap v281: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 0 B/s wr, 22 op/s 2024-09-11T22:06:17.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:06:17 smithi053 ceph-mon[34908]: pgmap v282: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 0 B/s wr, 27 op/s 2024-09-11T22:06:17.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:06:17 smithi161 ceph-mon[35615]: pgmap v282: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 0 B/s wr, 27 op/s 2024-09-11T22:06:17.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:06:17 smithi001 ceph-mon[30935]: pgmap v282: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 0 B/s wr, 27 op/s 2024-09-11T22:06:17.918 INFO:tasks.workunit.client.0.smithi001.stderr:+ set -e 2024-09-11T22:06:17.919 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo rsync -av /tmp/multiple_rsync_payload.201961 payload.1 2024-09-11T22:06:17.942 INFO:tasks.workunit.client.0.smithi001.stdout:sending incremental file list 2024-09-11T22:06:17.943 INFO:tasks.workunit.client.0.smithi001.stderr:rsync: [Receiver] mkdir "/home/ubuntu/cephtest/mnt.0/client.0/tmp/payload.1" failed: Read-only file system (30) 2024-09-11T22:06:17.943 INFO:tasks.workunit.client.0.smithi001.stderr:rsync error: error in file IO (code 11) at main.c(783) [Receiver=3.2.3] 2024-09-11T22:06:17.949 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-09-11T22:06:18.045 INFO:tasks.workunit:Stopping ['fs/misc'] on client.0... 2024-09-11T22:06:18.045 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-11T22:06:18.969 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/workunit.py", line 293, in _spawn_on_all_clients with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test fs/misc/multiple_rsync.sh) on smithi001 with status 11: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=086c737fa75c22d521a3932c59b31c5e1d1c9ef4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/multiple_rsync.sh' 2024-09-11T22:06:21.101 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=c8f8877416a44baa9ff51a080f7bb66d Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/workunit.py", line 293, in _spawn_on_all_clients with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test fs/misc/multiple_rsync.sh) on smithi001 with status 11: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=086c737fa75c22d521a3932c59b31c5e1d1c9ef4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/multiple_rsync.sh' 2024-09-11T22:06:21.104 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-09-11T22:06:21.114 INFO:tasks.quiescer:joining Quiescers 2024-09-11T22:06:21.114 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-09-11T22:06:21.114 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:06:21.121 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:06:19 smithi161 ceph-mon[35615]: pgmap v283: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:06:21.122 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:06:19 smithi053 ceph-mon[34908]: pgmap v283: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:06:21.122 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:06:19 smithi001 ceph-mon[30935]: pgmap v283: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:06:21.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:06:21 smithi053 ceph-mon[34908]: pgmap v284: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 0 B/s wr, 15 op/s 2024-09-11T22:06:21.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:06:21 smithi161 ceph-mon[35615]: pgmap v284: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 0 B/s wr, 15 op/s 2024-09-11T22:06:21.399 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:06:21.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:06:21 smithi001 ceph-mon[30935]: pgmap v284: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 0 B/s wr, 15 op/s 2024-09-11T22:06:21.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:21.895+0000 7f5a5deec640 1 -- 172.21.15.1:0/3186804749 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a58102dd0 msgr2=0x7f5a581031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:21.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:21.895+0000 7f5a5deec640 1 --2- 172.21.15.1:0/3186804749 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a58102dd0 0x7f5a581031b0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f5a48009930 tx=0x7f5a4802a420 comp rx=0 tx=0).stop 2024-09-11T22:06:21.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:21.895+0000 7f5a5deec640 1 -- 172.21.15.1:0/3186804749 shutdown_connections 2024-09-11T22:06:21.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:21.895+0000 7f5a5deec640 1 --2- 172.21.15.1:0/3186804749 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a58108300 0x7f5a5810ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:21.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:21.895+0000 7f5a5deec640 1 --2- 172.21.15.1:0/3186804749 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a58103780 0x7f5a58107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:21.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:21.895+0000 7f5a5deec640 1 --2- 172.21.15.1:0/3186804749 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a58102dd0 0x7f5a581031b0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:21.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:21.895+0000 7f5a5deec640 1 -- 172.21.15.1:0/3186804749 >> 172.21.15.1:0/3186804749 conn(0x7f5a580fc7d0 msgr2=0x7f5a580febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:21.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:21.895+0000 7f5a5deec640 1 -- 172.21.15.1:0/3186804749 shutdown_connections 2024-09-11T22:06:21.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:21.895+0000 7f5a5deec640 1 -- 172.21.15.1:0/3186804749 wait complete. 2024-09-11T22:06:21.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:21.896+0000 7f5a5deec640 1 Processor -- start 2024-09-11T22:06:21.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:21.896+0000 7f5a5deec640 1 -- start start 2024-09-11T22:06:21.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:21.896+0000 7f5a5deec640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a58102dd0 0x7f5a58100a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:21.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:21.896+0000 7f5a5deec640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a58103780 0x7f5a58100f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:21.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:21.896+0000 7f5a5deec640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a58108300 0x7f5a581014b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:21.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:21.896+0000 7f5a5deec640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5a58112090 con 0x7f5a58103780 2024-09-11T22:06:21.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:21.896+0000 7f5a5deec640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5a58111f10 con 0x7f5a58108300 2024-09-11T22:06:21.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:21.896+0000 7f5a5deec640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5a58112210 con 0x7f5a58102dd0 2024-09-11T22:06:21.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:21.896+0000 7f5a4effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a58103780 0x7f5a58100f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:21.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:21.896+0000 7f5a4effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a58103780 0x7f5a58100f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59798/0 (socket says 172.21.15.1:59798) 2024-09-11T22:06:21.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:21.896+0000 7f5a4effd640 1 -- 172.21.15.1:0/3451382166 learned_addr learned my addr 172.21.15.1:0/3451382166 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:06:21.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:21.896+0000 7f5a577fe640 1 --2- 172.21.15.1:0/3451382166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a58102dd0 0x7f5a58100a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:21.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:21.896+0000 7f5a57fff640 1 --2- 172.21.15.1:0/3451382166 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a58108300 0x7f5a581014b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:21.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:21.897+0000 7f5a4effd640 1 -- 172.21.15.1:0/3451382166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a58102dd0 msgr2=0x7f5a58100a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:21.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:21.897+0000 7f5a4effd640 1 --2- 172.21.15.1:0/3451382166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a58102dd0 0x7f5a58100a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:21.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:21.897+0000 7f5a4effd640 1 -- 172.21.15.1:0/3451382166 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a58108300 msgr2=0x7f5a581014b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:21.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:21.897+0000 7f5a4effd640 1 --2- 172.21.15.1:0/3451382166 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a58108300 0x7f5a581014b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:21.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:21.897+0000 7f5a4effd640 1 -- 172.21.15.1:0/3451382166 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a581a6a10 con 0x7f5a58103780 2024-09-11T22:06:21.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:21.897+0000 7f5a4effd640 1 --2- 172.21.15.1:0/3451382166 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a58103780 0x7f5a58100f70 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f5a4000d950 tx=0x7f5a4000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:21.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:21.897+0000 7f5a557fa640 1 -- 172.21.15.1:0/3451382166 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a40017070 con 0x7f5a58103780 2024-09-11T22:06:21.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:21.897+0000 7f5a557fa640 1 -- 172.21.15.1:0/3451382166 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a40009c30 con 0x7f5a58103780 2024-09-11T22:06:21.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:21.897+0000 7f5a557fa640 1 -- 172.21.15.1:0/3451382166 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a400047a0 con 0x7f5a58103780 2024-09-11T22:06:21.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:21.897+0000 7f5a5deec640 1 -- 172.21.15.1:0/3451382166 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a581a6d00 con 0x7f5a58103780 2024-09-11T22:06:21.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:21.897+0000 7f5a5deec640 1 -- 172.21.15.1:0/3451382166 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a581a7190 con 0x7f5a58103780 2024-09-11T22:06:21.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:21.898+0000 7f5a5deec640 1 -- 172.21.15.1:0/3451382166 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a581031b0 con 0x7f5a58103780 2024-09-11T22:06:21.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:21.899+0000 7f5a557fa640 1 -- 172.21.15.1:0/3451382166 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f5a40004020 con 0x7f5a58103780 2024-09-11T22:06:21.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:21.900+0000 7f5a557fa640 1 --2- 172.21.15.1:0/3451382166 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f5a440a2cf0 0x7f5a440a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:21.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:21.900+0000 7f5a557fa640 1 -- 172.21.15.1:0/3451382166 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7f5a400cf190 con 0x7f5a58103780 2024-09-11T22:06:21.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:21.900+0000 7f5a577fe640 1 --2- 172.21.15.1:0/3451382166 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f5a440a2cf0 0x7f5a440a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:21.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:21.901+0000 7f5a577fe640 1 --2- 172.21.15.1:0/3451382166 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f5a440a2cf0 0x7f5a440a51b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5a48000c00 tx=0x7f5a48005a20 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:21.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:21.905+0000 7f5a557fa640 1 -- 172.21.15.1:0/3451382166 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5a40020020 con 0x7f5a58103780 2024-09-11T22:06:22.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:22.121+0000 7f5a5deec640 1 -- 172.21.15.1:0/3451382166 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5a5810a2c0 con 0x7f5a58103780 2024-09-11T22:06:22.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:22.122+0000 7f5a557fa640 1 -- 172.21.15.1:0/3451382166 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+11986 (secure 0 0 0) 0x7f5a4009bd70 con 0x7f5a58103780 2024-09-11T22:06:22.123 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T22:06:22.123 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":46,"btime":"2024-09-11T22:05:53:986700+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/421542832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":421542832},{"type":"v1","addr":"172.21.15.1:6835","nonce":421542832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2836864406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2836864406},{"type":"v1","addr":"172.21.15.1:6837","nonce":2836864406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":46,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T21:55:20.791263+0000","modified":"2024-09-11T22:05:53.152083+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24484,"mds_2":24467,"mds_3":24466,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14595":{"gid":14595,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/407834672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":407834672},{"type":"v1","addr":"172.21.15.1:6839","nonce":407834672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/1068827678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":1068827678},{"type":"v1","addr":"172.21.15.1:6841","nonce":1068827678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/872410024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":872410024},{"type":"v1","addr":"172.21.15.161:6833","nonce":872410024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/1648944570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1648944570},{"type":"v1","addr":"172.21.15.53:6833","nonce":1648944570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/2126933993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2126933993},{"type":"v1","addr":"172.21.15.53:6835","nonce":2126933993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.161:6835/828381592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":828381592},{"type":"v1","addr":"172.21.15.161:6835","nonce":828381592}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.161:6837/1799318656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1799318656},{"type":"v1","addr":"172.21.15.161:6837","nonce":1799318656}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.53:6837/3457131830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3457131830},{"type":"v1","addr":"172.21.15.53:6837","nonce":3457131830}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/455365383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":455365383},{"type":"v1","addr":"172.21.15.161:6839","nonce":455365383}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":162,"addr":"172.21.15.53:6839/1378584447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1378584447},{"type":"v1","addr":"172.21.15.53:6839","nonce":1378584447}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24460,24466,24467,24484,24494]},"id":1}]} 2024-09-11T22:06:22.123 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 46 2024-09-11T22:06:22.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:22.124+0000 7f5a5deec640 1 -- 172.21.15.1:0/3451382166 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f5a440a2cf0 msgr2=0x7f5a440a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:22.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:22.124+0000 7f5a5deec640 1 --2- 172.21.15.1:0/3451382166 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f5a440a2cf0 0x7f5a440a51b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5a48000c00 tx=0x7f5a48005a20 comp rx=0 tx=0).stop 2024-09-11T22:06:22.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:22.124+0000 7f5a5deec640 1 -- 172.21.15.1:0/3451382166 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a58103780 msgr2=0x7f5a58100f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:22.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:22.124+0000 7f5a5deec640 1 --2- 172.21.15.1:0/3451382166 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a58103780 0x7f5a58100f70 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f5a4000d950 tx=0x7f5a4000de10 comp rx=0 tx=0).stop 2024-09-11T22:06:22.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:22.124+0000 7f5a5deec640 1 -- 172.21.15.1:0/3451382166 shutdown_connections 2024-09-11T22:06:22.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:22.124+0000 7f5a5deec640 1 --2- 172.21.15.1:0/3451382166 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f5a440a2cf0 0x7f5a440a51b0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:22.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:22.124+0000 7f5a5deec640 1 --2- 172.21.15.1:0/3451382166 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5a58108300 0x7f5a581014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:22.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:22.124+0000 7f5a5deec640 1 --2- 172.21.15.1:0/3451382166 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a58103780 0x7f5a58100f70 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:22.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:22.124+0000 7f5a5deec640 1 --2- 172.21.15.1:0/3451382166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5a58102dd0 0x7f5a58100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:22.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:22.124+0000 7f5a5deec640 1 -- 172.21.15.1:0/3451382166 >> 172.21.15.1:0/3451382166 conn(0x7f5a580fc7d0 msgr2=0x7f5a58105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:22.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:22.124+0000 7f5a5deec640 1 -- 172.21.15.1:0/3451382166 shutdown_connections 2024-09-11T22:06:22.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:22.124+0000 7f5a5deec640 1 -- 172.21.15.1:0/3451382166 wait complete. 2024-09-11T22:06:22.938 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db --cancel --all' 2024-09-11T22:06:22.939 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db --cancel --all 2024-09-11T22:06:23.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:06:23 smithi001 ceph-mon[30935]: pgmap v285: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-11T22:06:23.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:06:23 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/3451382166' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:06:23.199 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:06:23.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:06:23 smithi053 ceph-mon[34908]: pgmap v285: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-11T22:06:23.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:06:23 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/3451382166' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:06:23.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:06:23 smithi161 ceph-mon[35615]: pgmap v285: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-11T22:06:23.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:06:23 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/3451382166' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:06:23.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.692+0000 7ff16d626640 1 -- 172.21.15.1:0/747381310 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1681049a0 msgr2=0x7ff1681089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:23.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.692+0000 7ff16d626640 1 --2- 172.21.15.1:0/747381310 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1681049a0 0x7ff1681089f0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7ff15c009900 tx=0x7ff15c02a410 comp rx=0 tx=0).stop 2024-09-11T22:06:23.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.692+0000 7ff16d626640 1 -- 172.21.15.1:0/747381310 shutdown_connections 2024-09-11T22:06:23.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.692+0000 7ff16d626640 1 --2- 172.21.15.1:0/747381310 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff168109120 0x7ff168110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:23.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.692+0000 7ff16d626640 1 --2- 172.21.15.1:0/747381310 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1681049a0 0x7ff1681089f0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:23.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.692+0000 7ff16d626640 1 --2- 172.21.15.1:0/747381310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff168103ff0 0x7ff1681043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:23.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.692+0000 7ff16d626640 1 -- 172.21.15.1:0/747381310 >> 172.21.15.1:0/747381310 conn(0x7ff1680ffa90 msgr2=0x7ff168101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:23.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.692+0000 7ff16d626640 1 -- 172.21.15.1:0/747381310 shutdown_connections 2024-09-11T22:06:23.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.692+0000 7ff16d626640 1 -- 172.21.15.1:0/747381310 wait complete. 2024-09-11T22:06:23.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.692+0000 7ff16d626640 1 Processor -- start 2024-09-11T22:06:23.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.692+0000 7ff16d626640 1 -- start start 2024-09-11T22:06:23.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.693+0000 7ff16d626640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff168103ff0 0x7ff16819a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:23.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.693+0000 7ff16d626640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1681049a0 0x7ff16819ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:23.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.693+0000 7ff16d626640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff168109120 0x7ff16819eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:23.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.693+0000 7ff16d626640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff168114120 con 0x7ff1681049a0 2024-09-11T22:06:23.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.693+0000 7ff16d626640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff168113fa0 con 0x7ff168109120 2024-09-11T22:06:23.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.693+0000 7ff16d626640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff1681142a0 con 0x7ff168103ff0 2024-09-11T22:06:23.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.693+0000 7ff1667fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1681049a0 0x7ff16819ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:23.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.693+0000 7ff1667fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1681049a0 0x7ff16819ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59806/0 (socket says 172.21.15.1:59806) 2024-09-11T22:06:23.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.693+0000 7ff1667fc640 1 -- 172.21.15.1:0/2206073242 learned_addr learned my addr 172.21.15.1:0/2206073242 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:06:23.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.693+0000 7ff166ffd640 1 --2- 172.21.15.1:0/2206073242 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff168103ff0 0x7ff16819a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:23.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.693+0000 7ff1677fe640 1 --2- 172.21.15.1:0/2206073242 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff168109120 0x7ff16819eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:23.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.693+0000 7ff1667fc640 1 -- 172.21.15.1:0/2206073242 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff168103ff0 msgr2=0x7ff16819a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:23.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.693+0000 7ff1667fc640 1 --2- 172.21.15.1:0/2206073242 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff168103ff0 0x7ff16819a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:23.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.693+0000 7ff1667fc640 1 -- 172.21.15.1:0/2206073242 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff168109120 msgr2=0x7ff16819eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:23.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.693+0000 7ff1667fc640 1 --2- 172.21.15.1:0/2206073242 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff168109120 0x7ff16819eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:23.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.693+0000 7ff1667fc640 1 -- 172.21.15.1:0/2206073242 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff16819f5d0 con 0x7ff1681049a0 2024-09-11T22:06:23.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.693+0000 7ff1677fe640 1 --2- 172.21.15.1:0/2206073242 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff168109120 0x7ff16819eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:06:23.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.693+0000 7ff1667fc640 1 --2- 172.21.15.1:0/2206073242 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1681049a0 0x7ff16819ab60 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7ff15c006010 tx=0x7ff15c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:23.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.694+0000 7ff14ffff640 1 -- 172.21.15.1:0/2206073242 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15c03f030 con 0x7ff1681049a0 2024-09-11T22:06:23.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.694+0000 7ff16d626640 1 -- 172.21.15.1:0/2206073242 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff16819f860 con 0x7ff1681049a0 2024-09-11T22:06:23.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.694+0000 7ff14ffff640 1 -- 172.21.15.1:0/2206073242 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff15c003ce0 con 0x7ff1681049a0 2024-09-11T22:06:23.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.694+0000 7ff16d626640 1 -- 172.21.15.1:0/2206073242 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1681a71a0 con 0x7ff1681049a0 2024-09-11T22:06:23.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.694+0000 7ff14ffff640 1 -- 172.21.15.1:0/2206073242 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15c049060 con 0x7ff1681049a0 2024-09-11T22:06:23.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.696+0000 7ff14ffff640 1 -- 172.21.15.1:0/2206073242 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7ff15c042030 con 0x7ff1681049a0 2024-09-11T22:06:23.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.696+0000 7ff16f9ff740 1 Processor -- start 2024-09-11T22:06:23.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.696+0000 7ff16f9ff740 1 -- start start 2024-09-11T22:06:23.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.697+0000 7ff16f9ff740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5e7e43a00 0x55c5e7e43de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:23.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.697+0000 7ff16f9ff740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5e7e44320 0x55c5e7e4e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:23.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.697+0000 7ff16f9ff740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5e7e4ebe0 0x55c5e7e50fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:23.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.697+0000 7ff16f9ff740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55c5e7da8200 con 0x55c5e7e44320 2024-09-11T22:06:23.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.697+0000 7ff16f9ff740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55c5e7d4edc0 con 0x55c5e7e4ebe0 2024-09-11T22:06:23.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.697+0000 7ff16f9ff740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c5e7da41c0 con 0x55c5e7e43a00 2024-09-11T22:06:23.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.697+0000 7ff14ffff640 1 --2- 172.21.15.1:0/2206073242 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7ff1500a2cf0 0x7ff1500a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:23.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.697+0000 7ff166ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5e7e44320 0x55c5e7e4e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:23.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.697+0000 7ff166ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5e7e44320 0x55c5e7e4e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59812/0 (socket says 172.21.15.1:59812) 2024-09-11T22:06:23.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.697+0000 7ff166ffd640 1 -- 172.21.15.1:0/3765042604 learned_addr learned my addr 172.21.15.1:0/3765042604 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:06:23.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.697+0000 7ff1677fe640 1 --2- 172.21.15.1:0/3765042604 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5e7e43a00 0x55c5e7e43de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:23.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.697+0000 7ff1677fe640 1 --2- 172.21.15.1:0/2206073242 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7ff1500a2cf0 0x7ff1500a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:23.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.697+0000 7ff14ffff640 1 -- 172.21.15.1:0/2206073242 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7ff15c0ef990 con 0x7ff1681049a0 2024-09-11T22:06:23.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.697+0000 7ff166ffd640 1 -- 172.21.15.1:0/3765042604 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5e7e43a00 msgr2=0x55c5e7e43de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:23.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.697+0000 7ff166ffd640 1 --2- 172.21.15.1:0/3765042604 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5e7e43a00 0x55c5e7e43de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:23.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.697+0000 7ff166ffd640 1 -- 172.21.15.1:0/3765042604 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5e7e4ebe0 msgr2=0x55c5e7e50fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:23.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.697+0000 7ff1667fc640 1 --2- 172.21.15.1:0/3765042604 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5e7e4ebe0 0x55c5e7e50fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:23.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.697+0000 7ff166ffd640 1 --2- 172.21.15.1:0/3765042604 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5e7e4ebe0 0x55c5e7e50fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:23.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.697+0000 7ff166ffd640 1 -- 172.21.15.1:0/3765042604 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5e7b19620 con 0x55c5e7e44320 2024-09-11T22:06:23.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.697+0000 7ff1667fc640 1 --2- 172.21.15.1:0/3765042604 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5e7e4ebe0 0x55c5e7e50fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:06:23.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.697+0000 7ff166ffd640 1 --2- 172.21.15.1:0/3765042604 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5e7e44320 0x55c5e7e4e6a0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7ff15800bd10 tx=0x7ff1580107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b841b368e1d1f21 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:23.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.697+0000 7ff1677fe640 1 --2- 172.21.15.1:0/2206073242 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7ff1500a2cf0 0x7ff1500a51b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff160009370 tx=0x7ff160009240 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:23.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.698+0000 7ff14d7fa640 1 -- 172.21.15.1:0/3765042604 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff158004030 con 0x55c5e7e44320 2024-09-11T22:06:23.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.698+0000 7ff14d7fa640 1 -- 172.21.15.1:0/3765042604 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1580041d0 con 0x55c5e7e44320 2024-09-11T22:06:23.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.698+0000 7ff14d7fa640 1 -- 172.21.15.1:0/3765042604 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1580117e0 con 0x55c5e7e44320 2024-09-11T22:06:23.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.698+0000 7ff16f9ff740 1 -- 172.21.15.1:0/3765042604 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5e7e44320 msgr2=0x55c5e7e4e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:23.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.698+0000 7ff16f9ff740 1 --2- 172.21.15.1:0/3765042604 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5e7e44320 0x55c5e7e4e6a0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7ff15800bd10 tx=0x7ff1580107a0 comp rx=0 tx=0).stop 2024-09-11T22:06:23.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.698+0000 7ff16f9ff740 1 -- 172.21.15.1:0/3765042604 shutdown_connections 2024-09-11T22:06:23.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.698+0000 7ff16f9ff740 1 --2- 172.21.15.1:0/3765042604 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5e7e4ebe0 0x55c5e7e50fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:23.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.698+0000 7ff16f9ff740 1 --2- 172.21.15.1:0/3765042604 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5e7e44320 0x55c5e7e4e6a0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:23.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.698+0000 7ff16f9ff740 1 --2- 172.21.15.1:0/3765042604 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5e7e43a00 0x55c5e7e43de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:23.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.698+0000 7ff16f9ff740 1 -- 172.21.15.1:0/3765042604 >> 172.21.15.1:0/3765042604 conn(0x55c5e7e3ff90 msgr2=0x55c5e7e40370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:23.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.698+0000 7ff16f9ff740 1 -- 172.21.15.1:0/3765042604 shutdown_connections 2024-09-11T22:06:23.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.699+0000 7ff16f9ff740 1 -- 172.21.15.1:0/3765042604 wait complete. 2024-09-11T22:06:23.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.699+0000 7ff16f9ff740 1 -- start start 2024-09-11T22:06:23.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.699+0000 7ff16f9ff740 1 Processor -- start 2024-09-11T22:06:23.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.699+0000 7ff16f9ff740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5e7e43a00 0x55c5e7e4e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:23.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.699+0000 7ff16f9ff740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5e7e44320 0x55c5e7e4b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:23.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.699+0000 7ff16f9ff740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5e7e4ebe0 0x55c5e7e4bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:23.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.699+0000 7ff16f9ff740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55c5e7e517b0 con 0x55c5e7e4ebe0 2024-09-11T22:06:23.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.699+0000 7ff16f9ff740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55c5e7d42c50 con 0x55c5e7e43a00 2024-09-11T22:06:23.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.699+0000 7ff16f9ff740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c5e7e51930 con 0x55c5e7e44320 2024-09-11T22:06:23.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.699+0000 7ff16f9ff740 10 client.0 _ll_register_callbacks cb 0x55c5e7d54ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:06:23.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.699+0000 7ff1677fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5e7e4ebe0 0x55c5e7e4bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:23.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.699+0000 7ff1677fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5e7e4ebe0 0x55c5e7e4bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59814/0 (socket says 172.21.15.1:59814) 2024-09-11T22:06:23.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.699+0000 7ff1677fe640 1 -- 172.21.15.1:0/1302730018 learned_addr learned my addr 172.21.15.1:0/1302730018 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:06:23.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.699+0000 7ff1667fc640 1 --2- 172.21.15.1:0/1302730018 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5e7e44320 0x55c5e7e4b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:23.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.700+0000 7ff166ffd640 1 --2- 172.21.15.1:0/1302730018 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5e7e43a00 0x55c5e7e4e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:23.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.700+0000 7ff1677fe640 1 -- 172.21.15.1:0/1302730018 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5e7e44320 msgr2=0x55c5e7e4b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:23.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.700+0000 7ff1677fe640 1 --2- 172.21.15.1:0/1302730018 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5e7e44320 0x55c5e7e4b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:23.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.700+0000 7ff1677fe640 1 -- 172.21.15.1:0/1302730018 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5e7e43a00 msgr2=0x55c5e7e4e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:23.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.700+0000 7ff1677fe640 1 --2- 172.21.15.1:0/1302730018 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5e7e43a00 0x55c5e7e4e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:23.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.700+0000 7ff1677fe640 1 -- 172.21.15.1:0/1302730018 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5e7e4c610 con 0x55c5e7e4ebe0 2024-09-11T22:06:23.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.700+0000 7ff1677fe640 1 --2- 172.21.15.1:0/1302730018 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5e7e4ebe0 0x55c5e7e4bde0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7ff160014660 tx=0x7ff160004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecc8731084d67949 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:23.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.700+0000 7ff14aff5640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:06:23.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.700+0000 7ff14aff5640 1 -- 172.21.15.1:0/1302730018 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff160004b20 con 0x55c5e7e4ebe0 2024-09-11T22:06:23.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.700+0000 7ff14aff5640 1 -- 172.21.15.1:0/1302730018 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7ff160004cc0 con 0x55c5e7e4ebe0 2024-09-11T22:06:23.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.700+0000 7ff16f9ff740 1 -- 172.21.15.1:0/1302730018 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1580041d0 con 0x55c5e7e4ebe0 2024-09-11T22:06:23.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.700+0000 7ff14aff5640 1 -- 172.21.15.1:0/1302730018 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7ff160003970 con 0x55c5e7e4ebe0 2024-09-11T22:06:23.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.700+0000 7ff16f9ff740 10 client.17583 fetch_fsmap learned FSMap version 46 2024-09-11T22:06:23.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.700+0000 7ff16f9ff740 1 -- 172.21.15.1:0/1302730018 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55c5e7e5d370 con 0x55c5e7e4ebe0 2024-09-11T22:06:23.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.701+0000 7ff14aff5640 1 -- 172.21.15.1:0/1302730018 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1580041d0 con 0x55c5e7e4ebe0 2024-09-11T22:06:23.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.701+0000 7ff14aff5640 1 -- 172.21.15.1:0/1302730018 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 46) ==== 8442+0+0 (secure 0 0 0) 0x7ff16001d040 con 0x55c5e7e4ebe0 2024-09-11T22:06:23.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.701+0000 7ff16f9ff740 10 client.17583 fetch_fsmap finished waiting for FSMap version 46 2024-09-11T22:06:23.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.701+0000 7ff16f9ff740 10 client.17583 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-11T22:06:23.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.701+0000 7ff16f9ff740 20 client.17583 populate_metadata read hostname 'smithi001' 2024-09-11T22:06:23.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.701+0000 7ff16f9ff740 1 --2- 172.21.15.1:0/1302730018 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55c5e7e5e870 0x55c5e7e60cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:23.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.701+0000 7ff16f9ff740 4 client.17583 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:06:23.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.701+0000 7ff16f9ff740 1 -- 172.21.15.1:0/1302730018 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c5e7db5770 con 0x55c5e7e5e870 2024-09-11T22:06:23.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.701+0000 7ff166ffd640 1 --2- 172.21.15.1:0/1302730018 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55c5e7e5e870 0x55c5e7e60cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:23.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.701+0000 7ff14aff5640 10 client.17583 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:06:23.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.701+0000 7ff166ffd640 1 --2- 172.21.15.1:0/1302730018 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55c5e7e5e870 0x55c5e7e60cf0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e5f960df16b4d53 server_cookie=a8b2751395818c5e in_seq=0 out_seq=0 2024-09-11T22:06:23.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.702+0000 7ff14aff5640 1 -- 172.21.15.1:0/1302730018 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c5e7db5770 con 0x55c5e7e5e870 2024-09-11T22:06:23.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.702+0000 7ff14aff5640 10 client.17583 handle_command_reply: tid=0 2024-09-11T22:06:23.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.702+0000 7ff16f9ff740 1 client.17583 shutdown 2024-09-11T22:06:23.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.702+0000 7ff16f9ff740 1 -- 172.21.15.1:0/1302730018 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5e7e4ebe0 msgr2=0x55c5e7e4bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:23.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.702+0000 7ff16f9ff740 1 --2- 172.21.15.1:0/1302730018 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5e7e4ebe0 0x55c5e7e4bde0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7ff160014660 tx=0x7ff160004a40 comp rx=0 tx=0).stop 2024-09-11T22:06:23.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.702+0000 7ff16f9ff740 1 -- 172.21.15.1:0/1302730018 shutdown_connections 2024-09-11T22:06:23.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.702+0000 7ff16f9ff740 1 --2- 172.21.15.1:0/1302730018 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55c5e7e5e870 0x55c5e7e60cf0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:23.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.702+0000 7ff16f9ff740 1 --2- 172.21.15.1:0/1302730018 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5e7e4ebe0 0x55c5e7e4bde0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:23.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.702+0000 7ff16f9ff740 1 --2- 172.21.15.1:0/1302730018 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5e7e44320 0x55c5e7e4b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:23.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.702+0000 7ff16f9ff740 1 --2- 172.21.15.1:0/1302730018 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5e7e43a00 0x55c5e7e4e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:23.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.702+0000 7ff16f9ff740 1 -- 172.21.15.1:0/1302730018 >> 172.21.15.1:0/1302730018 conn(0x55c5e7e3ff90 msgr2=0x55c5e7e3f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:23.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.702+0000 7ff14aff5640 0 client.17583 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:06:23.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.702+0000 7ff16f9ff740 1 -- 172.21.15.1:0/1302730018 shutdown_connections 2024-09-11T22:06:23.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.702+0000 7ff16f9ff740 1 -- 172.21.15.1:0/1302730018 wait complete. 2024-09-11T22:06:23.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.703+0000 7ff16f9ff740 20 client.17583 trim_cache size 0 max 16384 2024-09-11T22:06:23.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.720+0000 7ff16f9ff740 1 Processor -- start 2024-09-11T22:06:23.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.720+0000 7ff16f9ff740 1 -- start start 2024-09-11T22:06:23.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.720+0000 7ff16f9ff740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5e7e43a00 0x55c5e7e40370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:23.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.720+0000 7ff16f9ff740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5e7e44320 0x55c5e7e665c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:23.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.720+0000 7ff16f9ff740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5e7e4ebe0 0x55c5e7e66b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:23.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.720+0000 7ff16f9ff740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff1681afc60 con 0x55c5e7e44320 2024-09-11T22:06:23.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.720+0000 7ff16f9ff740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff1681b1aa0 con 0x55c5e7e43a00 2024-09-11T22:06:23.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.720+0000 7ff16f9ff740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff1681b03f0 con 0x55c5e7e4ebe0 2024-09-11T22:06:23.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.720+0000 7ff1667fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5e7e44320 0x55c5e7e665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:23.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.720+0000 7ff1667fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5e7e44320 0x55c5e7e665c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59826/0 (socket says 172.21.15.1:59826) 2024-09-11T22:06:23.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.720+0000 7ff1667fc640 1 -- 172.21.15.1:0/870926874 learned_addr learned my addr 172.21.15.1:0/870926874 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:06:23.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.720+0000 7ff166ffd640 1 --2- 172.21.15.1:0/870926874 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5e7e43a00 0x55c5e7e40370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:23.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.720+0000 7ff1677fe640 1 --2- 172.21.15.1:0/870926874 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5e7e4ebe0 0x55c5e7e66b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:23.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.720+0000 7ff1667fc640 1 -- 172.21.15.1:0/870926874 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5e7e4ebe0 msgr2=0x55c5e7e66b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:23.754 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-11T22:06:23.755 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 46, 2024-09-11T22:06:23.755 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24494, 2024-09-11T22:06:23.755 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 31, 2024-09-11T22:06:23.755 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": {} 2024-09-11T22:06:23.755 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-11T22:06:23.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.720+0000 7ff1667fc640 1 --2- 172.21.15.1:0/870926874 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5e7e4ebe0 0x55c5e7e66b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:23.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.720+0000 7ff1667fc640 1 -- 172.21.15.1:0/870926874 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5e7e43a00 msgr2=0x55c5e7e40370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:23.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.720+0000 7ff1667fc640 1 --2- 172.21.15.1:0/870926874 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5e7e43a00 0x55c5e7e40370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:23.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.720+0000 7ff1667fc640 1 -- 172.21.15.1:0/870926874 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5e7e43830 con 0x55c5e7e44320 2024-09-11T22:06:23.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.720+0000 7ff1677fe640 1 --2- 172.21.15.1:0/870926874 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5e7e4ebe0 0x55c5e7e66b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:06:23.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.720+0000 7ff1667fc640 1 --2- 172.21.15.1:0/870926874 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5e7e44320 0x55c5e7e665c0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7ff15c0bbb70 tx=0x7ff15c0bba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33d46ce72f22683c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:23.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.721+0000 7ff14cff9640 1 -- 172.21.15.1:0/870926874 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15c035040 con 0x55c5e7e44320 2024-09-11T22:06:23.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.721+0000 7ff14cff9640 1 -- 172.21.15.1:0/870926874 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff15c0bc570 con 0x55c5e7e44320 2024-09-11T22:06:23.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.721+0000 7ff14cff9640 1 -- 172.21.15.1:0/870926874 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15c0b84e0 con 0x55c5e7e44320 2024-09-11T22:06:23.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.721+0000 7ff16f9ff740 1 -- 172.21.15.1:0/870926874 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5e7e44320 msgr2=0x55c5e7e665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:23.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.721+0000 7ff16f9ff740 1 --2- 172.21.15.1:0/870926874 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5e7e44320 0x55c5e7e665c0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7ff15c0bbb70 tx=0x7ff15c0bba70 comp rx=0 tx=0).stop 2024-09-11T22:06:23.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.721+0000 7ff16f9ff740 1 -- 172.21.15.1:0/870926874 shutdown_connections 2024-09-11T22:06:23.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.721+0000 7ff16f9ff740 1 --2- 172.21.15.1:0/870926874 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5e7e4ebe0 0x55c5e7e66b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:23.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.721+0000 7ff16f9ff740 1 --2- 172.21.15.1:0/870926874 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5e7e44320 0x55c5e7e665c0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:23.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.721+0000 7ff16f9ff740 1 --2- 172.21.15.1:0/870926874 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5e7e43a00 0x55c5e7e40370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:23.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.721+0000 7ff16f9ff740 1 -- 172.21.15.1:0/870926874 >> 172.21.15.1:0/870926874 conn(0x55c5e7e3ff90 msgr2=0x55c5e7e4d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:23.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.721+0000 7ff16f9ff740 1 -- 172.21.15.1:0/870926874 shutdown_connections 2024-09-11T22:06:23.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.721+0000 7ff16f9ff740 1 -- 172.21.15.1:0/870926874 wait complete. 2024-09-11T22:06:23.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.722+0000 7ff16f9ff740 1 -- start start 2024-09-11T22:06:23.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.722+0000 7ff16f9ff740 1 Processor -- start 2024-09-11T22:06:23.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.722+0000 7ff16f9ff740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5e7e43a00 0x55c5e7e48810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:23.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.722+0000 7ff16f9ff740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5e7e44320 0x55c5e7e48d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:23.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.722+0000 7ff16f9ff740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5e7e4ebe0 0x55c5e7e49290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:23.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.722+0000 7ff16f9ff740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff1681b1310 con 0x55c5e7e4ebe0 2024-09-11T22:06:23.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.722+0000 7ff16f9ff740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff1681b0b80 con 0x55c5e7e44320 2024-09-11T22:06:23.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.722+0000 7ff16f9ff740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff1681adc40 con 0x55c5e7e43a00 2024-09-11T22:06:23.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.722+0000 7ff16f9ff740 10 client.0 _ll_register_callbacks cb 0x7ff160014960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:06:23.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.722+0000 7ff1677fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5e7e4ebe0 0x55c5e7e49290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:23.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.722+0000 7ff1677fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5e7e4ebe0 0x55c5e7e49290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59836/0 (socket says 172.21.15.1:59836) 2024-09-11T22:06:23.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.722+0000 7ff1677fe640 1 -- 172.21.15.1:0/1462500548 learned_addr learned my addr 172.21.15.1:0/1462500548 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:06:23.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.722+0000 7ff166ffd640 1 --2- 172.21.15.1:0/1462500548 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5e7e43a00 0x55c5e7e48810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:23.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.722+0000 7ff1667fc640 1 --2- 172.21.15.1:0/1462500548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5e7e44320 0x55c5e7e48d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:23.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.722+0000 7ff1677fe640 1 -- 172.21.15.1:0/1462500548 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5e7e43a00 msgr2=0x55c5e7e48810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:23.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.722+0000 7ff1677fe640 1 --2- 172.21.15.1:0/1462500548 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5e7e43a00 0x55c5e7e48810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:23.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.722+0000 7ff1677fe640 1 -- 172.21.15.1:0/1462500548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5e7e44320 msgr2=0x55c5e7e48d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:23.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.722+0000 7ff1677fe640 1 --2- 172.21.15.1:0/1462500548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5e7e44320 0x55c5e7e48d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:23.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.722+0000 7ff1677fe640 1 -- 172.21.15.1:0/1462500548 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5e7ed0da0 con 0x55c5e7e4ebe0 2024-09-11T22:06:23.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.722+0000 7ff1677fe640 1 --2- 172.21.15.1:0/1462500548 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5e7e4ebe0 0x55c5e7e49290 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7ff160009a60 tx=0x7ff16000f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcf72c741b354633 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:23.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.722+0000 7ff14aff5640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:06:23.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.723+0000 7ff14aff5640 1 -- 172.21.15.1:0/1462500548 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff160004ab0 con 0x55c5e7e4ebe0 2024-09-11T22:06:23.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.723+0000 7ff14aff5640 1 -- 172.21.15.1:0/1462500548 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7ff160004c50 con 0x55c5e7e4ebe0 2024-09-11T22:06:23.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.723+0000 7ff16f9ff740 1 -- 172.21.15.1:0/1462500548 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff15c0bc570 con 0x55c5e7e4ebe0 2024-09-11T22:06:23.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.723+0000 7ff14aff5640 1 -- 172.21.15.1:0/1462500548 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7ff160004e50 con 0x55c5e7e4ebe0 2024-09-11T22:06:23.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.723+0000 7ff16f9ff740 10 client.17589 fetch_fsmap learned FSMap version 46 2024-09-11T22:06:23.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.723+0000 7ff16f9ff740 1 -- 172.21.15.1:0/1462500548 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55c5e7ed1fe0 con 0x55c5e7e4ebe0 2024-09-11T22:06:23.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.723+0000 7ff14aff5640 1 -- 172.21.15.1:0/1462500548 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff15c0bc570 con 0x55c5e7e4ebe0 2024-09-11T22:06:23.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.723+0000 7ff14aff5640 1 -- 172.21.15.1:0/1462500548 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 46) ==== 8442+0+0 (secure 0 0 0) 0x7ff160005730 con 0x55c5e7e4ebe0 2024-09-11T22:06:23.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.723+0000 7ff16f9ff740 10 client.17589 fetch_fsmap finished waiting for FSMap version 46 2024-09-11T22:06:23.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.723+0000 7ff16f9ff740 10 client.17589 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-11T22:06:23.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.723+0000 7ff16f9ff740 20 client.17589 populate_metadata read hostname 'smithi001' 2024-09-11T22:06:23.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.723+0000 7ff16f9ff740 1 --2- 172.21.15.1:0/1462500548 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55c5e7ed2710 0x55c5e7ed4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:23.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.723+0000 7ff16f9ff740 4 client.17589 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-09-11T22:06:23.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.723+0000 7ff16f9ff740 1 -- 172.21.15.1:0/1462500548 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x55c5e7e40b40 con 0x55c5e7ed2710 2024-09-11T22:06:23.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.723+0000 7ff166ffd640 1 --2- 172.21.15.1:0/1462500548 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55c5e7ed2710 0x55c5e7ed4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:23.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.724+0000 7ff14aff5640 10 client.17589 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:06:23.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.724+0000 7ff166ffd640 1 --2- 172.21.15.1:0/1462500548 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55c5e7ed2710 0x55c5e7ed4b30 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=567841a00859378b server_cookie=e93575a844acbbe9 in_seq=0 out_seq=0 2024-09-11T22:06:23.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.724+0000 7ff14aff5640 1 -- 172.21.15.1:0/1462500548 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x55c5e7e40b40 con 0x55c5e7ed2710 2024-09-11T22:06:23.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.724+0000 7ff14aff5640 10 client.17589 handle_command_reply: tid=0 2024-09-11T22:06:23.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.724+0000 7ff16f9ff740 1 client.17589 shutdown 2024-09-11T22:06:23.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.724+0000 7ff16f9ff740 1 -- 172.21.15.1:0/1462500548 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5e7e4ebe0 msgr2=0x55c5e7e49290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:23.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.724+0000 7ff16f9ff740 1 --2- 172.21.15.1:0/1462500548 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5e7e4ebe0 0x55c5e7e49290 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7ff160009a60 tx=0x7ff16000f810 comp rx=0 tx=0).stop 2024-09-11T22:06:23.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.724+0000 7ff16f9ff740 1 -- 172.21.15.1:0/1462500548 shutdown_connections 2024-09-11T22:06:23.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.724+0000 7ff16f9ff740 1 --2- 172.21.15.1:0/1462500548 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55c5e7ed2710 0x55c5e7ed4b30 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:23.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.724+0000 7ff16f9ff740 1 --2- 172.21.15.1:0/1462500548 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5e7e4ebe0 0x55c5e7e49290 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:23.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.724+0000 7ff16f9ff740 1 --2- 172.21.15.1:0/1462500548 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5e7e44320 0x55c5e7e48d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:23.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.724+0000 7ff16f9ff740 1 --2- 172.21.15.1:0/1462500548 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c5e7e43a00 0x55c5e7e48810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:23.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.724+0000 7ff16f9ff740 1 -- 172.21.15.1:0/1462500548 >> 172.21.15.1:0/1462500548 conn(0x55c5e7e3ff90 msgr2=0x55c5e7e45450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:23.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.724+0000 7ff14aff5640 0 client.17589 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:06:23.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.725+0000 7ff16f9ff740 1 -- 172.21.15.1:0/1462500548 shutdown_connections 2024-09-11T22:06:23.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.725+0000 7ff16f9ff740 1 -- 172.21.15.1:0/1462500548 wait complete. 2024-09-11T22:06:23.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.725+0000 7ff16f9ff740 20 client.17589 trim_cache size 0 max 16384 2024-09-11T22:06:23.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.725+0000 7ff16d626640 1 -- 172.21.15.1:0/2206073242 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7ff1500a2cf0 msgr2=0x7ff1500a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:23.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.725+0000 7ff16d626640 1 --2- 172.21.15.1:0/2206073242 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7ff1500a2cf0 0x7ff1500a51b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff160009370 tx=0x7ff160009240 comp rx=0 tx=0).stop 2024-09-11T22:06:23.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.725+0000 7ff16d626640 1 -- 172.21.15.1:0/2206073242 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1681049a0 msgr2=0x7ff16819ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:23.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.725+0000 7ff16d626640 1 --2- 172.21.15.1:0/2206073242 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1681049a0 0x7ff16819ab60 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7ff15c006010 tx=0x7ff15c0042f0 comp rx=0 tx=0).stop 2024-09-11T22:06:23.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.725+0000 7ff16d626640 1 -- 172.21.15.1:0/2206073242 shutdown_connections 2024-09-11T22:06:23.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.725+0000 7ff16d626640 1 --2- 172.21.15.1:0/2206073242 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7ff1500a2cf0 0x7ff1500a51b0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:23.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.725+0000 7ff16d626640 1 --2- 172.21.15.1:0/2206073242 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff168109120 0x7ff16819eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:23.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.725+0000 7ff16d626640 1 --2- 172.21.15.1:0/2206073242 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1681049a0 0x7ff16819ab60 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:23.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.725+0000 7ff16d626640 1 --2- 172.21.15.1:0/2206073242 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff168103ff0 0x7ff16819a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:23.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.725+0000 7ff16d626640 1 -- 172.21.15.1:0/2206073242 >> 172.21.15.1:0/2206073242 conn(0x7ff1680ffa90 msgr2=0x7ff168100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:23.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.725+0000 7ff16d626640 1 -- 172.21.15.1:0/2206073242 shutdown_connections 2024-09-11T22:06:23.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:23.725+0000 7ff16d626640 1 -- 172.21.15.1:0/2206073242 wait complete. 2024-09-11T22:06:24.468 INFO:tasks.quiescer:done joining Quiescers 2024-09-11T22:06:24.468 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-09-11T22:06:24.484 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi001.front.sepia.ceph.com 2024-09-11T22:06:24.493 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-09-11T22:06:24.502 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-09-11T22:06:24.503 INFO:teuthology.task.exec:Executing custom commands... 2024-09-11T22:06:24.503 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi001.front.sepia.ceph.com 2024-09-11T22:06:24.503 DEBUG:teuthology.orchestra.run.smithi001:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-09-11T22:06:24.574 DEBUG:teuthology.orchestra.run.smithi001:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-09-11T22:06:25.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:06:25 smithi053 ceph-mon[34908]: pgmap v286: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-11T22:06:25.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:06:25 smithi161 ceph-mon[35615]: pgmap v286: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-11T22:06:25.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:06:25 smithi001 ceph-mon[30935]: pgmap v286: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-11T22:06:26.197 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:06:26.469 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:06:27.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.983+0000 7f29a4737640 1 -- 172.21.15.1:0/1344006881 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f299c10c1a0 msgr2=0x7f299c10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:27.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.983+0000 7f29a4737640 1 --2- 172.21.15.1:0/1344006881 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f299c10c1a0 0x7f299c10e610 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f2994009ca0 tx=0x7f299402a950 comp rx=0 tx=0).stop 2024-09-11T22:06:27.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.983+0000 7f29a4737640 1 -- 172.21.15.1:0/1344006881 shutdown_connections 2024-09-11T22:06:27.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.983+0000 7f29a4737640 1 --2- 172.21.15.1:0/1344006881 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f299c10c1a0 0x7f299c10e610 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:27.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.983+0000 7f29a4737640 1 --2- 172.21.15.1:0/1344006881 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f299c103720 0x7f299c10bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:27.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.983+0000 7f29a4737640 1 --2- 172.21.15.1:0/1344006881 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f299c102e00 0x7f299c1031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:27.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.983+0000 7f29a4737640 1 -- 172.21.15.1:0/1344006881 >> 172.21.15.1:0/1344006881 conn(0x7f299c0fc7a0 msgr2=0x7f299c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:27.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.983+0000 7f29a4737640 1 -- 172.21.15.1:0/1344006881 shutdown_connections 2024-09-11T22:06:27.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.983+0000 7f29a4737640 1 -- 172.21.15.1:0/1344006881 wait complete. 2024-09-11T22:06:27.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.984+0000 7f29a4737640 1 Processor -- start 2024-09-11T22:06:27.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.984+0000 7f29a4737640 1 -- start start 2024-09-11T22:06:27.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.984+0000 7f29a4737640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f299c102e00 0x7f299c1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:27.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.984+0000 7f29a4737640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f299c103720 0x7f299c198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:27.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.984+0000 7f29a4737640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f299c10c1a0 0x7f299c19cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:27.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.984+0000 7f29a4737640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f299c111f70 con 0x7f299c103720 2024-09-11T22:06:27.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.984+0000 7f29a4737640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f299c111df0 con 0x7f299c102e00 2024-09-11T22:06:27.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.984+0000 7f29a4737640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f299c1120f0 con 0x7f299c10c1a0 2024-09-11T22:06:27.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.984+0000 7f29a1cd2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f299c103720 0x7f299c198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:27.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.984+0000 7f29a1cd2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f299c103720 0x7f299c198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37984/0 (socket says 172.21.15.1:37984) 2024-09-11T22:06:27.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.984+0000 7f29a1cd2640 1 -- 172.21.15.1:0/3280232944 learned_addr learned my addr 172.21.15.1:0/3280232944 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:06:27.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.984+0000 7f29a2cd4640 1 --2- 172.21.15.1:0/3280232944 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f299c10c1a0 0x7f299c19cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:27.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.984+0000 7f29a24d3640 1 --2- 172.21.15.1:0/3280232944 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f299c102e00 0x7f299c1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:27.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.984+0000 7f29a1cd2640 1 -- 172.21.15.1:0/3280232944 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f299c10c1a0 msgr2=0x7f299c19cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:27.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.984+0000 7f29a1cd2640 1 --2- 172.21.15.1:0/3280232944 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f299c10c1a0 0x7f299c19cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:27.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.984+0000 7f29a1cd2640 1 -- 172.21.15.1:0/3280232944 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f299c102e00 msgr2=0x7f299c1983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:27.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.984+0000 7f29a1cd2640 1 --2- 172.21.15.1:0/3280232944 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f299c102e00 0x7f299c1983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:27.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.985+0000 7f29a1cd2640 1 -- 172.21.15.1:0/3280232944 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f299c19d380 con 0x7f299c103720 2024-09-11T22:06:27.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.985+0000 7f29a1cd2640 1 --2- 172.21.15.1:0/3280232944 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f299c103720 0x7f299c198910 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f298c00c920 tx=0x7f298c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:27.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.985+0000 7f29937fe640 1 -- 172.21.15.1:0/3280232944 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298c007c70 con 0x7f299c103720 2024-09-11T22:06:27.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.985+0000 7f29a4737640 1 -- 172.21.15.1:0/3280232944 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f299c19d670 con 0x7f299c103720 2024-09-11T22:06:27.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.985+0000 7f29937fe640 1 -- 172.21.15.1:0/3280232944 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f298c007e10 con 0x7f299c103720 2024-09-11T22:06:27.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.985+0000 7f29937fe640 1 -- 172.21.15.1:0/3280232944 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298c018890 con 0x7f299c103720 2024-09-11T22:06:27.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.985+0000 7f29a4737640 1 -- 172.21.15.1:0/3280232944 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f299c1a4f50 con 0x7f299c103720 2024-09-11T22:06:27.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.987+0000 7f29937fe640 1 -- 172.21.15.1:0/3280232944 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f298c018a70 con 0x7f299c103720 2024-09-11T22:06:27.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.987+0000 7f29a6b10740 1 Processor -- start 2024-09-11T22:06:27.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.988+0000 7f29a6b10740 1 -- start start 2024-09-11T22:06:27.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.988+0000 7f29a6b10740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563a0ce5d4a0 0x563a0ce5d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:27.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.988+0000 7f29a6b10740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a0ce5ddc0 0x563a0ce68140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:27.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.988+0000 7f29a6b10740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a0ce68680 0x563a0ce6aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:27.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.988+0000 7f29a6b10740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x563a0cdc32e0 con 0x563a0ce68680 2024-09-11T22:06:27.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.988+0000 7f29a6b10740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x563a0cd6ace0 con 0x563a0ce5d4a0 2024-09-11T22:06:27.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.988+0000 7f29a6b10740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x563a0cdbf2a0 con 0x563a0ce5ddc0 2024-09-11T22:06:27.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.988+0000 7f29937fe640 1 --2- 172.21.15.1:0/3280232944 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f29880a2cf0 0x7f29880a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:27.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.988+0000 7f29a1cd2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a0ce68680 0x563a0ce6aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:27.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.988+0000 7f29a24d3640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a0ce5ddc0 0x563a0ce68140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:27.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.988+0000 7f29a24d3640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a0ce5ddc0 0x563a0ce68140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:33216/0 (socket says 172.21.15.1:33216) 2024-09-11T22:06:27.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.988+0000 7f29a24d3640 1 -- 172.21.15.1:0/3334509568 learned_addr learned my addr 172.21.15.1:0/3334509568 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:06:27.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.988+0000 7f29a2cd4640 1 --2- 172.21.15.1:0/3334509568 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563a0ce5d4a0 0x563a0ce5d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:27.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.988+0000 7f29a2cd4640 1 --2- 172.21.15.1:0/3280232944 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f29880a2cf0 0x7f29880a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:27.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.988+0000 7f29937fe640 1 -- 172.21.15.1:0/3280232944 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7f298c0cf610 con 0x7f299c103720 2024-09-11T22:06:27.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.988+0000 7f29a24d3640 1 -- 172.21.15.1:0/3334509568 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563a0ce5d4a0 msgr2=0x563a0ce5d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:27.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.988+0000 7f29a24d3640 1 --2- 172.21.15.1:0/3334509568 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563a0ce5d4a0 0x563a0ce5d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:27.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.988+0000 7f29a24d3640 1 -- 172.21.15.1:0/3334509568 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a0ce68680 msgr2=0x563a0ce6aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:27.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.988+0000 7f29a24d3640 1 --2- 172.21.15.1:0/3334509568 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a0ce68680 0x563a0ce6aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:27.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.988+0000 7f29a24d3640 1 -- 172.21.15.1:0/3334509568 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a0cd4f440 con 0x563a0ce5ddc0 2024-09-11T22:06:27.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.989+0000 7f29a2cd4640 1 --2- 172.21.15.1:0/3280232944 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f29880a2cf0 0x7f29880a51b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2994009a60 tx=0x7f29940095c0 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:27.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.989+0000 7f29a24d3640 1 --2- 172.21.15.1:0/3334509568 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a0ce5ddc0 0x563a0ce68140 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f299800cbf0 tx=0x7f299800e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5441d7b20ca33dc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:27.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.989+0000 7f2990ff9640 1 -- 172.21.15.1:0/3334509568 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2998010630 con 0x563a0ce5ddc0 2024-09-11T22:06:27.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.989+0000 7f2990ff9640 1 -- 172.21.15.1:0/3334509568 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2998010cb0 con 0x563a0ce5ddc0 2024-09-11T22:06:27.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.989+0000 7f2990ff9640 1 -- 172.21.15.1:0/3334509568 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f299800fc60 con 0x563a0ce5ddc0 2024-09-11T22:06:27.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.990+0000 7f29a6b10740 1 -- 172.21.15.1:0/3334509568 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a0ce5ddc0 msgr2=0x563a0ce68140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:27.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.990+0000 7f29a6b10740 1 --2- 172.21.15.1:0/3334509568 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a0ce5ddc0 0x563a0ce68140 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f299800cbf0 tx=0x7f299800e720 comp rx=0 tx=0).stop 2024-09-11T22:06:27.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.990+0000 7f29a6b10740 1 -- 172.21.15.1:0/3334509568 shutdown_connections 2024-09-11T22:06:27.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.990+0000 7f29a6b10740 1 --2- 172.21.15.1:0/3334509568 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a0ce68680 0x563a0ce6aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:27.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.990+0000 7f29a6b10740 1 --2- 172.21.15.1:0/3334509568 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a0ce5ddc0 0x563a0ce68140 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:27.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.990+0000 7f29a6b10740 1 --2- 172.21.15.1:0/3334509568 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563a0ce5d4a0 0x563a0ce5d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:27.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.990+0000 7f29a6b10740 1 -- 172.21.15.1:0/3334509568 >> 172.21.15.1:0/3334509568 conn(0x563a0ce59a30 msgr2=0x563a0ce59e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:27.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.990+0000 7f29a6b10740 1 -- 172.21.15.1:0/3334509568 shutdown_connections 2024-09-11T22:06:27.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.990+0000 7f29a6b10740 1 -- 172.21.15.1:0/3334509568 wait complete. 2024-09-11T22:06:27.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.990+0000 7f29a6b10740 1 -- start start 2024-09-11T22:06:27.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.990+0000 7f29a6b10740 1 Processor -- start 2024-09-11T22:06:27.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.991+0000 7f29a6b10740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563a0ce5d4a0 0x563a0ce67f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:27.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.991+0000 7f29a6b10740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a0ce5ddc0 0x563a0ce651f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:27.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.991+0000 7f29a6b10740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a0ce68680 0x563a0ce65730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:27.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.991+0000 7f29a6b10740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x563a0ce6b250 con 0x563a0ce68680 2024-09-11T22:06:27.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.991+0000 7f29a6b10740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x563a0cd7e050 con 0x563a0ce5d4a0 2024-09-11T22:06:27.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.991+0000 7f29a6b10740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x563a0ce6b3d0 con 0x563a0ce5ddc0 2024-09-11T22:06:27.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.991+0000 7f29a6b10740 10 client.0 _ll_register_callbacks cb 0x563a0cdbe160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:06:27.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.991+0000 7f29a2cd4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a0ce68680 0x563a0ce65730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:27.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.991+0000 7f29a2cd4640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a0ce68680 0x563a0ce65730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38002/0 (socket says 172.21.15.1:38002) 2024-09-11T22:06:27.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.991+0000 7f29a2cd4640 1 -- 172.21.15.1:0/1608375240 learned_addr learned my addr 172.21.15.1:0/1608375240 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:06:27.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.991+0000 7f29a1cd2640 1 --2- 172.21.15.1:0/1608375240 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a0ce5ddc0 0x563a0ce651f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:27.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.991+0000 7f29a24d3640 1 --2- 172.21.15.1:0/1608375240 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563a0ce5d4a0 0x563a0ce67f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:27.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.991+0000 7f29a2cd4640 1 -- 172.21.15.1:0/1608375240 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a0ce5ddc0 msgr2=0x563a0ce651f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:27.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.991+0000 7f29a2cd4640 1 --2- 172.21.15.1:0/1608375240 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a0ce5ddc0 0x563a0ce651f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:27.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.991+0000 7f29a2cd4640 1 -- 172.21.15.1:0/1608375240 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563a0ce5d4a0 msgr2=0x563a0ce67f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:27.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.991+0000 7f29a2cd4640 1 --2- 172.21.15.1:0/1608375240 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563a0ce5d4a0 0x563a0ce67f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:27.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.991+0000 7f29a2cd4640 1 -- 172.21.15.1:0/1608375240 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a0cb343c0 con 0x563a0ce68680 2024-09-11T22:06:27.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.991+0000 7f29a2cd4640 1 --2- 172.21.15.1:0/1608375240 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a0ce68680 0x563a0ce65730 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f2994005f40 tx=0x7f299402a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bbb9e91a660fab7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:27.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.991+0000 7f2985ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:06:27.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.991+0000 7f2985ffb640 1 -- 172.21.15.1:0/1608375240 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f299404f030 con 0x563a0ce68680 2024-09-11T22:06:27.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.991+0000 7f2985ffb640 1 -- 172.21.15.1:0/1608375240 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7f2994004040 con 0x563a0ce68680 2024-09-11T22:06:27.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.991+0000 7f29a6b10740 1 -- 172.21.15.1:0/1608375240 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2998010cb0 con 0x563a0ce68680 2024-09-11T22:06:27.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.992+0000 7f2985ffb640 1 -- 172.21.15.1:0/1608375240 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f2994034040 con 0x563a0ce68680 2024-09-11T22:06:27.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.992+0000 7f29a6b10740 10 client.17601 fetch_fsmap learned FSMap version 46 2024-09-11T22:06:27.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.992+0000 7f29a6b10740 1 -- 172.21.15.1:0/1608375240 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x563a0ce76ba0 con 0x563a0ce68680 2024-09-11T22:06:27.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.992+0000 7f2985ffb640 1 -- 172.21.15.1:0/1608375240 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2998010cb0 con 0x563a0ce68680 2024-09-11T22:06:27.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.992+0000 7f2985ffb640 1 -- 172.21.15.1:0/1608375240 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 46) ==== 8442+0+0 (secure 0 0 0) 0x7f2994056520 con 0x563a0ce68680 2024-09-11T22:06:27.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.992+0000 7f29a6b10740 10 client.17601 fetch_fsmap finished waiting for FSMap version 46 2024-09-11T22:06:27.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.992+0000 7f29a6b10740 10 client.17601 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:06:27.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.992+0000 7f29a6b10740 20 client.17601 populate_metadata read hostname 'smithi001' 2024-09-11T22:06:27.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.992+0000 7f29a6b10740 1 --2- 172.21.15.1:0/1608375240 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x563a0ce780a0 0x563a0ce7a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:27.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.992+0000 7f29a6b10740 4 client.17601 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:06:27.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.992+0000 7f29a6b10740 1 -- 172.21.15.1:0/1608375240 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a0cdd0850 con 0x563a0ce780a0 2024-09-11T22:06:27.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.993+0000 7f29a24d3640 1 --2- 172.21.15.1:0/1608375240 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x563a0ce780a0 0x563a0ce7a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:27.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.993+0000 7f29a24d3640 1 --2- 172.21.15.1:0/1608375240 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x563a0ce780a0 0x563a0ce7a540 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f8ca6e37811070d server_cookie=717adb68abcba25a in_seq=0 out_seq=0 2024-09-11T22:06:27.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.993+0000 7f2985ffb640 10 client.17601 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:06:27.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.994+0000 7f2985ffb640 1 -- 172.21.15.1:0/1608375240 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563a0cdd0850 con 0x563a0ce780a0 2024-09-11T22:06:27.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.994+0000 7f2985ffb640 10 client.17601 handle_command_reply: tid=0 2024-09-11T22:06:27.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.994+0000 7f29a6b10740 1 client.17601 shutdown 2024-09-11T22:06:27.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.994+0000 7f29a6b10740 1 -- 172.21.15.1:0/1608375240 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a0ce68680 msgr2=0x563a0ce65730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:27.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.994+0000 7f29a6b10740 1 --2- 172.21.15.1:0/1608375240 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a0ce68680 0x563a0ce65730 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f2994005f40 tx=0x7f299402a950 comp rx=0 tx=0).stop 2024-09-11T22:06:27.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.994+0000 7f29a6b10740 1 -- 172.21.15.1:0/1608375240 shutdown_connections 2024-09-11T22:06:27.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.994+0000 7f29a6b10740 1 --2- 172.21.15.1:0/1608375240 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x563a0ce780a0 0x563a0ce7a540 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:27.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.994+0000 7f29a6b10740 1 --2- 172.21.15.1:0/1608375240 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a0ce68680 0x563a0ce65730 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:27.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.994+0000 7f29a6b10740 1 --2- 172.21.15.1:0/1608375240 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a0ce5ddc0 0x563a0ce651f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:27.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.994+0000 7f29a6b10740 1 --2- 172.21.15.1:0/1608375240 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563a0ce5d4a0 0x563a0ce67f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:27.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.994+0000 7f29a6b10740 1 -- 172.21.15.1:0/1608375240 >> 172.21.15.1:0/1608375240 conn(0x563a0ce59a30 msgr2=0x563a0ce5f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:27.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.994+0000 7f2985ffb640 0 client.17601 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:06:27.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.994+0000 7f29a6b10740 1 -- 172.21.15.1:0/1608375240 shutdown_connections 2024-09-11T22:06:27.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.994+0000 7f29a6b10740 1 -- 172.21.15.1:0/1608375240 wait complete. 2024-09-11T22:06:27.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:26.994+0000 7f29a6b10740 20 client.17601 trim_cache size 0 max 16384 2024-09-11T22:06:27.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.012+0000 7f29a6b10740 1 Processor -- start 2024-09-11T22:06:27.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.012+0000 7f29a6b10740 1 -- start start 2024-09-11T22:06:27.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.012+0000 7f29a6b10740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a0ce5d4a0 0x563a0ce744b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:27.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.012+0000 7f29a6b10740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a0ce5ddc0 0x563a0ce7fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:27.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.012+0000 7f29a6b10740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563a0ce68680 0x563a0ce80350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:27.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.012+0000 7f29a6b10740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f299c1ad930 con 0x563a0ce5ddc0 2024-09-11T22:06:27.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.012+0000 7f29a6b10740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f299c1af770 con 0x563a0ce68680 2024-09-11T22:06:27.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.012+0000 7f29a6b10740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f299c1ae0c0 con 0x563a0ce5d4a0 2024-09-11T22:06:27.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.012+0000 7f29a1cd2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a0ce5ddc0 0x563a0ce7fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:27.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.012+0000 7f29a1cd2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a0ce5ddc0 0x563a0ce7fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38010/0 (socket says 172.21.15.1:38010) 2024-09-11T22:06:27.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.012+0000 7f29a1cd2640 1 -- 172.21.15.1:0/3977653800 learned_addr learned my addr 172.21.15.1:0/3977653800 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:06:27.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.012+0000 7f29a24d3640 1 --2- 172.21.15.1:0/3977653800 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a0ce5d4a0 0x563a0ce744b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:27.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.012+0000 7f29a2cd4640 1 --2- 172.21.15.1:0/3977653800 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563a0ce68680 0x563a0ce80350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:27.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.013+0000 7f29a1cd2640 1 -- 172.21.15.1:0/3977653800 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a0ce5d4a0 msgr2=0x563a0ce744b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:27.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.013+0000 7f29a1cd2640 1 --2- 172.21.15.1:0/3977653800 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a0ce5d4a0 0x563a0ce744b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:27.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.013+0000 7f29a1cd2640 1 -- 172.21.15.1:0/3977653800 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563a0ce68680 msgr2=0x563a0ce80350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:27.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.013+0000 7f29a1cd2640 1 --2- 172.21.15.1:0/3977653800 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563a0ce68680 0x563a0ce80350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:27.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.013+0000 7f29a1cd2640 1 -- 172.21.15.1:0/3977653800 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a0ce5dbf0 con 0x563a0ce5ddc0 2024-09-11T22:06:27.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.013+0000 7f29a1cd2640 1 --2- 172.21.15.1:0/3977653800 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a0ce5ddc0 0x563a0ce7fe10 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f298c0cd320 tx=0x7f298c00cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d2501f17cb2dc82 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:27.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.013+0000 7f2987fff640 1 -- 172.21.15.1:0/3977653800 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298c0a4760 con 0x563a0ce5ddc0 2024-09-11T22:06:27.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.013+0000 7f2987fff640 1 -- 172.21.15.1:0/3977653800 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f298c018d60 con 0x563a0ce5ddc0 2024-09-11T22:06:27.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.013+0000 7f2987fff640 1 -- 172.21.15.1:0/3977653800 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298c0a42d0 con 0x563a0ce5ddc0 2024-09-11T22:06:27.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.014+0000 7f29a6b10740 1 -- 172.21.15.1:0/3977653800 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a0ce5ddc0 msgr2=0x563a0ce7fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:27.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.014+0000 7f29a6b10740 1 --2- 172.21.15.1:0/3977653800 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a0ce5ddc0 0x563a0ce7fe10 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f298c0cd320 tx=0x7f298c00cf10 comp rx=0 tx=0).stop 2024-09-11T22:06:27.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.014+0000 7f29a6b10740 1 -- 172.21.15.1:0/3977653800 shutdown_connections 2024-09-11T22:06:27.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.014+0000 7f29a6b10740 1 --2- 172.21.15.1:0/3977653800 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563a0ce68680 0x563a0ce80350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:27.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.014+0000 7f29a6b10740 1 --2- 172.21.15.1:0/3977653800 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a0ce5ddc0 0x563a0ce7fe10 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:27.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.014+0000 7f29a6b10740 1 --2- 172.21.15.1:0/3977653800 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a0ce5d4a0 0x563a0ce744b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:27.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.014+0000 7f29a6b10740 1 -- 172.21.15.1:0/3977653800 >> 172.21.15.1:0/3977653800 conn(0x563a0ce59a30 msgr2=0x563a0ce642e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:27.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.014+0000 7f29a6b10740 1 -- 172.21.15.1:0/3977653800 shutdown_connections 2024-09-11T22:06:27.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.014+0000 7f29a6b10740 1 -- 172.21.15.1:0/3977653800 wait complete. 2024-09-11T22:06:27.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.014+0000 7f29a6b10740 1 -- start start 2024-09-11T22:06:27.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.014+0000 7f29a6b10740 1 Processor -- start 2024-09-11T22:06:27.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.015+0000 7f29a6b10740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a0ce5d4a0 0x563a0ce6dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:27.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.015+0000 7f29a6b10740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a0ce5ddc0 0x563a0ce6e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:27.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.015+0000 7f29a6b10740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563a0ce68680 0x563a0cedf230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:27.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.015+0000 7f29a6b10740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f299c1aefe0 con 0x563a0ce5ddc0 2024-09-11T22:06:27.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.015+0000 7f29a6b10740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f299c1ae850 con 0x563a0ce68680 2024-09-11T22:06:27.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.015+0000 7f29a6b10740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f299c1ab910 con 0x563a0ce5d4a0 2024-09-11T22:06:27.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.015+0000 7f29a6b10740 10 client.0 _ll_register_callbacks cb 0x7f29940098d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:06:27.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.015+0000 7f29a1cd2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a0ce5ddc0 0x563a0ce6e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:27.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.015+0000 7f29a1cd2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a0ce5ddc0 0x563a0ce6e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38018/0 (socket says 172.21.15.1:38018) 2024-09-11T22:06:27.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.015+0000 7f29a24d3640 1 --2- 172.21.15.1:0/3855202224 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a0ce5d4a0 0x563a0ce6dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:27.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.015+0000 7f29a1cd2640 1 -- 172.21.15.1:0/3855202224 learned_addr learned my addr 172.21.15.1:0/3855202224 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:06:27.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.015+0000 7f29a2cd4640 1 --2- 172.21.15.1:0/3855202224 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563a0ce68680 0x563a0cedf230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:27.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.015+0000 7f29a1cd2640 1 -- 172.21.15.1:0/3855202224 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a0ce5d4a0 msgr2=0x563a0ce6dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:27.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.015+0000 7f29a1cd2640 1 --2- 172.21.15.1:0/3855202224 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a0ce5d4a0 0x563a0ce6dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:27.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.015+0000 7f29a1cd2640 1 -- 172.21.15.1:0/3855202224 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563a0ce68680 msgr2=0x563a0cedf230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:27.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.015+0000 7f29a1cd2640 1 --2- 172.21.15.1:0/3855202224 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563a0ce68680 0x563a0cedf230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:27.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.015+0000 7f29a1cd2640 1 -- 172.21.15.1:0/3855202224 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a0cedf8a0 con 0x563a0ce5ddc0 2024-09-11T22:06:27.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.015+0000 7f29a24d3640 1 --2- 172.21.15.1:0/3855202224 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a0ce5d4a0 0x563a0ce6dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:06:27.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.015+0000 7f29a2cd4640 1 --2- 172.21.15.1:0/3855202224 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563a0ce68680 0x563a0cedf230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:06:27.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.015+0000 7f29a1cd2640 1 --2- 172.21.15.1:0/3855202224 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a0ce5ddc0 0x563a0ce6e170 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f298c007590 tx=0x7f298c09b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c313f5659a5f113 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:27.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.015+0000 7f2985ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:06:27.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.015+0000 7f2985ffb640 1 -- 172.21.15.1:0/3855202224 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298c098160 con 0x563a0ce5ddc0 2024-09-11T22:06:27.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.016+0000 7f2985ffb640 1 -- 172.21.15.1:0/3855202224 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7f298c0ba340 con 0x563a0ce5ddc0 2024-09-11T22:06:27.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.016+0000 7f29a6b10740 1 -- 172.21.15.1:0/3855202224 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f298c018d60 con 0x563a0ce5ddc0 2024-09-11T22:06:27.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.016+0000 7f2985ffb640 1 -- 172.21.15.1:0/3855202224 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f298c098300 con 0x563a0ce5ddc0 2024-09-11T22:06:27.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.016+0000 7f29a6b10740 10 client.17607 fetch_fsmap learned FSMap version 46 2024-09-11T22:06:27.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.016+0000 7f29a6b10740 1 -- 172.21.15.1:0/3855202224 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x563a0cee6c10 con 0x563a0ce5ddc0 2024-09-11T22:06:27.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.016+0000 7f2985ffb640 1 -- 172.21.15.1:0/3855202224 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f298c018d60 con 0x563a0ce5ddc0 2024-09-11T22:06:27.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.016+0000 7f2985ffb640 1 -- 172.21.15.1:0/3855202224 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 46) ==== 8442+0+0 (secure 0 0 0) 0x7f298c097840 con 0x563a0ce5ddc0 2024-09-11T22:06:27.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.016+0000 7f29a6b10740 10 client.17607 fetch_fsmap finished waiting for FSMap version 46 2024-09-11T22:06:27.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.016+0000 7f29a6b10740 10 client.17607 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:06:27.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.016+0000 7f29a6b10740 20 client.17607 populate_metadata read hostname 'smithi001' 2024-09-11T22:06:27.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.016+0000 7f29a6b10740 1 --2- 172.21.15.1:0/3855202224 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x563a0cee73e0 0x563a0cee9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:27.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.016+0000 7f29a6b10740 4 client.17607 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:06:27.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.016+0000 7f29a6b10740 1 -- 172.21.15.1:0/3855202224 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563a0cdcad60 con 0x563a0cee73e0 2024-09-11T22:06:27.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.016+0000 7f29a24d3640 1 --2- 172.21.15.1:0/3855202224 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x563a0cee73e0 0x563a0cee9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:27.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.017+0000 7f29a24d3640 1 --2- 172.21.15.1:0/3855202224 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x563a0cee73e0 0x563a0cee9800 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd3c51983133c0f2 server_cookie=4bcbdd24bb18ee65 in_seq=0 out_seq=0 2024-09-11T22:06:27.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.017+0000 7f2985ffb640 10 client.17607 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:06:27.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.017+0000 7f2985ffb640 1 -- 172.21.15.1:0/3855202224 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563a0cdcad60 con 0x563a0cee73e0 2024-09-11T22:06:27.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.017+0000 7f2985ffb640 10 client.17607 handle_command_reply: tid=0 2024-09-11T22:06:27.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.017+0000 7f29a6b10740 1 client.17607 shutdown 2024-09-11T22:06:27.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.017+0000 7f29a6b10740 1 -- 172.21.15.1:0/3855202224 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a0ce5ddc0 msgr2=0x563a0ce6e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:27.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.017+0000 7f29a6b10740 1 --2- 172.21.15.1:0/3855202224 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a0ce5ddc0 0x563a0ce6e170 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f298c007590 tx=0x7f298c09b930 comp rx=0 tx=0).stop 2024-09-11T22:06:27.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.017+0000 7f29a6b10740 1 -- 172.21.15.1:0/3855202224 shutdown_connections 2024-09-11T22:06:27.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.017+0000 7f29a6b10740 1 --2- 172.21.15.1:0/3855202224 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x563a0cee73e0 0x563a0cee9800 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:27.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.017+0000 7f29a6b10740 1 --2- 172.21.15.1:0/3855202224 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563a0ce68680 0x563a0cedf230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:27.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.017+0000 7f29a6b10740 1 --2- 172.21.15.1:0/3855202224 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a0ce5ddc0 0x563a0ce6e170 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:27.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.017+0000 7f29a6b10740 1 --2- 172.21.15.1:0/3855202224 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x563a0ce5d4a0 0x563a0ce6dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:27.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.017+0000 7f29a6b10740 1 -- 172.21.15.1:0/3855202224 >> 172.21.15.1:0/3855202224 conn(0x563a0ce59a30 msgr2=0x563a0ce58350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:27.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.017+0000 7f2985ffb640 0 client.17607 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:06:27.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.017+0000 7f29a6b10740 1 -- 172.21.15.1:0/3855202224 shutdown_connections 2024-09-11T22:06:27.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.017+0000 7f29a6b10740 1 -- 172.21.15.1:0/3855202224 wait complete. 2024-09-11T22:06:27.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.017+0000 7f29a6b10740 20 client.17607 trim_cache size 0 max 16384 2024-09-11T22:06:27.038 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-11T22:06:27.038 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-09-11T22:06:27.039 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-09-11T22:06:27.039 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-11T22:06:27.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.018+0000 7f29a4737640 1 -- 172.21.15.1:0/3280232944 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f29880a2cf0 msgr2=0x7f29880a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:27.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.018+0000 7f29a4737640 1 --2- 172.21.15.1:0/3280232944 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f29880a2cf0 0x7f29880a51b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2994009a60 tx=0x7f29940095c0 comp rx=0 tx=0).stop 2024-09-11T22:06:27.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.018+0000 7f29a4737640 1 -- 172.21.15.1:0/3280232944 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f299c103720 msgr2=0x7f299c198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:27.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.018+0000 7f29a4737640 1 --2- 172.21.15.1:0/3280232944 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f299c103720 0x7f299c198910 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f298c00c920 tx=0x7f298c00cde0 comp rx=0 tx=0).stop 2024-09-11T22:06:27.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.018+0000 7f29a4737640 1 -- 172.21.15.1:0/3280232944 shutdown_connections 2024-09-11T22:06:27.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.018+0000 7f29a4737640 1 --2- 172.21.15.1:0/3280232944 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f29880a2cf0 0x7f29880a51b0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:27.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.018+0000 7f29a4737640 1 --2- 172.21.15.1:0/3280232944 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f299c10c1a0 0x7f299c19cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:27.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.018+0000 7f29a4737640 1 --2- 172.21.15.1:0/3280232944 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f299c103720 0x7f299c198910 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:27.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.018+0000 7f29a4737640 1 --2- 172.21.15.1:0/3280232944 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f299c102e00 0x7f299c1983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:27.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.018+0000 7f29a4737640 1 -- 172.21.15.1:0/3280232944 >> 172.21.15.1:0/3280232944 conn(0x7f299c0fc7a0 msgr2=0x7f299c0fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:27.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.018+0000 7f29a4737640 1 -- 172.21.15.1:0/3280232944 shutdown_connections 2024-09-11T22:06:27.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:27.018+0000 7f29a4737640 1 -- 172.21.15.1:0/3280232944 wait complete. 2024-09-11T22:06:27.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:06:27 smithi001 ceph-mon[30935]: pgmap v287: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 0 B/s wr, 15 op/s 2024-09-11T22:06:27.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:06:27 smithi161 ceph-mon[35615]: pgmap v287: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 0 B/s wr, 15 op/s 2024-09-11T22:06:27.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:06:27 smithi053 ceph-mon[34908]: pgmap v287: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 0 B/s wr, 15 op/s 2024-09-11T22:06:27.745 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T22:06:27.746 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:06:28.009 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:06:28.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:28.516+0000 7fd09755b640 1 -- 172.21.15.1:0/356065096 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0901082a0 msgr2=0x7fd09010ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:28.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:28.516+0000 7fd09755b640 1 --2- 172.21.15.1:0/356065096 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0901082a0 0x7fd09010ec40 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fd088009cd0 tx=0x7fd08802a910 comp rx=0 tx=0).stop 2024-09-11T22:06:28.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:28.516+0000 7fd09755b640 1 -- 172.21.15.1:0/356065096 shutdown_connections 2024-09-11T22:06:28.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:28.516+0000 7fd09755b640 1 --2- 172.21.15.1:0/356065096 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0901082a0 0x7fd09010ec40 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:28.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:28.516+0000 7fd09755b640 1 --2- 172.21.15.1:0/356065096 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd0901036e0 0x7fd090107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:28.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:28.516+0000 7fd09755b640 1 --2- 172.21.15.1:0/356065096 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd090102d30 0x7fd090103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:28.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:28.516+0000 7fd09755b640 1 -- 172.21.15.1:0/356065096 >> 172.21.15.1:0/356065096 conn(0x7fd0900fc7d0 msgr2=0x7fd0900febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:28.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:28.516+0000 7fd09755b640 1 -- 172.21.15.1:0/356065096 shutdown_connections 2024-09-11T22:06:28.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:28.516+0000 7fd09755b640 1 -- 172.21.15.1:0/356065096 wait complete. 2024-09-11T22:06:28.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:28.517+0000 7fd09755b640 1 Processor -- start 2024-09-11T22:06:28.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:28.517+0000 7fd09755b640 1 -- start start 2024-09-11T22:06:28.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:28.517+0000 7fd09755b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd090102d30 0x7fd090100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:28.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:28.517+0000 7fd09755b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0901036e0 0x7fd090100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:28.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:28.517+0000 7fd09755b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0901082a0 0x7fd090101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:28.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:28.517+0000 7fd09755b640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd090112160 con 0x7fd0901036e0 2024-09-11T22:06:28.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:28.517+0000 7fd09755b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd090111fe0 con 0x7fd0901082a0 2024-09-11T22:06:28.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:28.517+0000 7fd09755b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd0901122e0 con 0x7fd090102d30 2024-09-11T22:06:28.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:28.517+0000 7fd094af6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0901036e0 0x7fd090100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:28.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:28.517+0000 7fd094af6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0901036e0 0x7fd090100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38050/0 (socket says 172.21.15.1:38050) 2024-09-11T22:06:28.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:28.517+0000 7fd094af6640 1 -- 172.21.15.1:0/3833338923 learned_addr learned my addr 172.21.15.1:0/3833338923 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:06:28.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:28.517+0000 7fd0952f7640 1 --2- 172.21.15.1:0/3833338923 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd090102d30 0x7fd090100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:28.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:28.517+0000 7fd095af8640 1 --2- 172.21.15.1:0/3833338923 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0901082a0 0x7fd090101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:28.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:28.517+0000 7fd094af6640 1 -- 172.21.15.1:0/3833338923 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd090102d30 msgr2=0x7fd090100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:28.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:28.517+0000 7fd094af6640 1 --2- 172.21.15.1:0/3833338923 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd090102d30 0x7fd090100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:28.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:28.517+0000 7fd094af6640 1 -- 172.21.15.1:0/3833338923 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0901082a0 msgr2=0x7fd090101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:28.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:28.517+0000 7fd094af6640 1 --2- 172.21.15.1:0/3833338923 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0901082a0 0x7fd090101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:28.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:28.517+0000 7fd094af6640 1 -- 172.21.15.1:0/3833338923 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0901a6c00 con 0x7fd0901036e0 2024-09-11T22:06:28.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:28.518+0000 7fd094af6640 1 --2- 172.21.15.1:0/3833338923 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0901036e0 0x7fd090100ed0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fd080006fd0 tx=0x7fd08000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:28.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:28.518+0000 7fd0867fc640 1 -- 172.21.15.1:0/3833338923 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd080011830 con 0x7fd0901036e0 2024-09-11T22:06:28.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:28.518+0000 7fd0867fc640 1 -- 172.21.15.1:0/3833338923 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd080016070 con 0x7fd0901036e0 2024-09-11T22:06:28.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:28.518+0000 7fd09755b640 1 -- 172.21.15.1:0/3833338923 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0901a6ef0 con 0x7fd0901036e0 2024-09-11T22:06:28.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:28.518+0000 7fd0867fc640 1 -- 172.21.15.1:0/3833338923 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08000f2e0 con 0x7fd0901036e0 2024-09-11T22:06:28.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:28.518+0000 7fd09755b640 1 -- 172.21.15.1:0/3833338923 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0901a7350 con 0x7fd0901036e0 2024-09-11T22:06:28.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:28.519+0000 7fd09755b640 1 -- 172.21.15.1:0/3833338923 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd090103190 con 0x7fd0901036e0 2024-09-11T22:06:28.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:28.521+0000 7fd0867fc640 1 -- 172.21.15.1:0/3833338923 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7fd0800119d0 con 0x7fd0901036e0 2024-09-11T22:06:28.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:28.522+0000 7fd0867fc640 1 --2- 172.21.15.1:0/3833338923 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fd07c0a2cf0 0x7fd07c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:28.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:28.522+0000 7fd0952f7640 1 --2- 172.21.15.1:0/3833338923 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fd07c0a2cf0 0x7fd07c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:28.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:28.522+0000 7fd0867fc640 1 -- 172.21.15.1:0/3833338923 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7fd0800cf140 con 0x7fd0901036e0 2024-09-11T22:06:28.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:28.522+0000 7fd0952f7640 1 --2- 172.21.15.1:0/3833338923 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fd07c0a2cf0 0x7fd07c0a51b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd08c009890 tx=0x7fd08c008040 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:28.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:28.525+0000 7fd0867fc640 1 -- 172.21.15.1:0/3833338923 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd08009bdd0 con 0x7fd0901036e0 2024-09-11T22:06:28.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:28.737+0000 7fd09755b640 1 -- 172.21.15.1:0/3833338923 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd090101ec0 con 0x7fd0901036e0 2024-09-11T22:06:28.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:28.738+0000 7fd0867fc640 1 -- 172.21.15.1:0/3833338923 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+11986 (secure 0 0 0) 0x7fd08009b6d0 con 0x7fd0901036e0 2024-09-11T22:06:28.739 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T22:06:28.739 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":46,"btime":"2024-09-11T22:05:53:986700+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/421542832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":421542832},{"type":"v1","addr":"172.21.15.1:6835","nonce":421542832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2836864406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2836864406},{"type":"v1","addr":"172.21.15.1:6837","nonce":2836864406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":46,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T21:55:20.791263+0000","modified":"2024-09-11T22:05:53.152083+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24484,"mds_2":24467,"mds_3":24466,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14595":{"gid":14595,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/407834672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":407834672},{"type":"v1","addr":"172.21.15.1:6839","nonce":407834672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/1068827678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":1068827678},{"type":"v1","addr":"172.21.15.1:6841","nonce":1068827678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/872410024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":872410024},{"type":"v1","addr":"172.21.15.161:6833","nonce":872410024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/1648944570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1648944570},{"type":"v1","addr":"172.21.15.53:6833","nonce":1648944570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/2126933993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2126933993},{"type":"v1","addr":"172.21.15.53:6835","nonce":2126933993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.161:6835/828381592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":828381592},{"type":"v1","addr":"172.21.15.161:6835","nonce":828381592}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.161:6837/1799318656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1799318656},{"type":"v1","addr":"172.21.15.161:6837","nonce":1799318656}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.53:6837/3457131830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3457131830},{"type":"v1","addr":"172.21.15.53:6837","nonce":3457131830}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/455365383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":455365383},{"type":"v1","addr":"172.21.15.161:6839","nonce":455365383}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":162,"addr":"172.21.15.53:6839/1378584447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1378584447},{"type":"v1","addr":"172.21.15.53:6839","nonce":1378584447}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24460,24466,24467,24484,24494]},"id":1}]} 2024-09-11T22:06:28.739 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 46 2024-09-11T22:06:28.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:28.740+0000 7fd09755b640 1 -- 172.21.15.1:0/3833338923 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fd07c0a2cf0 msgr2=0x7fd07c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:28.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:28.740+0000 7fd09755b640 1 --2- 172.21.15.1:0/3833338923 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fd07c0a2cf0 0x7fd07c0a51b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd08c009890 tx=0x7fd08c008040 comp rx=0 tx=0).stop 2024-09-11T22:06:28.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:28.740+0000 7fd09755b640 1 -- 172.21.15.1:0/3833338923 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0901036e0 msgr2=0x7fd090100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:28.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:28.740+0000 7fd09755b640 1 --2- 172.21.15.1:0/3833338923 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0901036e0 0x7fd090100ed0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fd080006fd0 tx=0x7fd08000b8e0 comp rx=0 tx=0).stop 2024-09-11T22:06:28.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:28.740+0000 7fd09755b640 1 -- 172.21.15.1:0/3833338923 shutdown_connections 2024-09-11T22:06:28.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:28.740+0000 7fd09755b640 1 --2- 172.21.15.1:0/3833338923 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fd07c0a2cf0 0x7fd07c0a51b0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:28.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:28.740+0000 7fd09755b640 1 --2- 172.21.15.1:0/3833338923 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0901082a0 0x7fd090101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:28.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:28.740+0000 7fd09755b640 1 --2- 172.21.15.1:0/3833338923 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0901036e0 0x7fd090100ed0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:28.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:28.740+0000 7fd09755b640 1 --2- 172.21.15.1:0/3833338923 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd090102d30 0x7fd090100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:28.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:28.740+0000 7fd09755b640 1 -- 172.21.15.1:0/3833338923 >> 172.21.15.1:0/3833338923 conn(0x7fd0900fc7d0 msgr2=0x7fd090106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:28.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:28.740+0000 7fd09755b640 1 -- 172.21.15.1:0/3833338923 shutdown_connections 2024-09-11T22:06:28.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:28.740+0000 7fd09755b640 1 -- 172.21.15.1:0/3833338923 wait complete. 2024-09-11T22:06:29.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:06:29 smithi161 ceph-mon[35615]: pgmap v288: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-11T22:06:29.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:06:29 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/3833338923' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:06:29.373 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:06:29 smithi001 ceph-mon[30935]: pgmap v288: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-11T22:06:29.373 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:06:29 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/3833338923' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:06:29.404 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T22:06:29.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:06:29 smithi053 ceph-mon[34908]: pgmap v288: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-11T22:06:29.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:06:29 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/3833338923' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:06:29.666 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:06:30.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.177+0000 7f2c9880f640 1 -- 172.21.15.1:0/2221636768 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c94100e20 msgr2=0x7f2c94101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:30.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.177+0000 7f2c9880f640 1 --2- 172.21.15.1:0/2221636768 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c94100e20 0x7f2c94101200 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f2c84009900 tx=0x7f2c8402a480 comp rx=0 tx=0).stop 2024-09-11T22:06:30.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.177+0000 7f2c9880f640 1 -- 172.21.15.1:0/2221636768 shutdown_connections 2024-09-11T22:06:30.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.177+0000 7f2c9880f640 1 --2- 172.21.15.1:0/2221636768 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c94102ef0 0x7f2c9410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:30.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.177+0000 7f2c9880f640 1 --2- 172.21.15.1:0/2221636768 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c94102570 0x7f2c941029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:30.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.177+0000 7f2c9880f640 1 --2- 172.21.15.1:0/2221636768 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c94100e20 0x7f2c94101200 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:30.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.177+0000 7f2c9880f640 1 -- 172.21.15.1:0/2221636768 >> 172.21.15.1:0/2221636768 conn(0x7f2c940fc7d0 msgr2=0x7f2c940febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:30.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.177+0000 7f2c9880f640 1 -- 172.21.15.1:0/2221636768 shutdown_connections 2024-09-11T22:06:30.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.177+0000 7f2c9880f640 1 -- 172.21.15.1:0/2221636768 wait complete. 2024-09-11T22:06:30.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.178+0000 7f2c9880f640 1 Processor -- start 2024-09-11T22:06:30.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.178+0000 7f2c9880f640 1 -- start start 2024-09-11T22:06:30.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.178+0000 7f2c9880f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c94100e20 0x7f2c9419a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:30.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.178+0000 7f2c9880f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c94102570 0x7f2c9419ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:30.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.178+0000 7f2c9880f640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c94102ef0 0x7f2c9419efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:30.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.178+0000 7f2c9880f640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2c94110f50 con 0x7f2c94100e20 2024-09-11T22:06:30.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.178+0000 7f2c9880f640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2c94110dd0 con 0x7f2c94102ef0 2024-09-11T22:06:30.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.178+0000 7f2c9880f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2c941110d0 con 0x7f2c94102570 2024-09-11T22:06:30.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.178+0000 7f2c9259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c94100e20 0x7f2c9419a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:30.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.178+0000 7f2c9259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c94100e20 0x7f2c9419a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38062/0 (socket says 172.21.15.1:38062) 2024-09-11T22:06:30.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.178+0000 7f2c9259c640 1 -- 172.21.15.1:0/3913485293 learned_addr learned my addr 172.21.15.1:0/3913485293 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:06:30.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.178+0000 7f2c8bfff640 1 --2- 172.21.15.1:0/3913485293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c94102570 0x7f2c9419ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:30.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.178+0000 7f2c92d9d640 1 --2- 172.21.15.1:0/3913485293 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c94102ef0 0x7f2c9419efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:30.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.179+0000 7f2c9259c640 1 -- 172.21.15.1:0/3913485293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c94102570 msgr2=0x7f2c9419ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:30.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.179+0000 7f2c9259c640 1 --2- 172.21.15.1:0/3913485293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c94102570 0x7f2c9419ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:30.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.179+0000 7f2c9259c640 1 -- 172.21.15.1:0/3913485293 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c94102ef0 msgr2=0x7f2c9419efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:30.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.179+0000 7f2c9259c640 1 --2- 172.21.15.1:0/3913485293 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c94102ef0 0x7f2c9419efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:30.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.179+0000 7f2c9259c640 1 -- 172.21.15.1:0/3913485293 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c9419f690 con 0x7f2c94100e20 2024-09-11T22:06:30.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.179+0000 7f2c8bfff640 1 --2- 172.21.15.1:0/3913485293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c94102570 0x7f2c9419ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:06:30.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.179+0000 7f2c92d9d640 1 --2- 172.21.15.1:0/3913485293 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c94102ef0 0x7f2c9419efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:06:30.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.179+0000 7f2c9259c640 1 --2- 172.21.15.1:0/3913485293 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c94100e20 0x7f2c9419a6e0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f2c84006010 tx=0x7f2c84003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:30.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.179+0000 7f2c8b7fe640 1 -- 172.21.15.1:0/3913485293 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8403f070 con 0x7f2c94100e20 2024-09-11T22:06:30.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.179+0000 7f2c8b7fe640 1 -- 172.21.15.1:0/3913485293 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c84031e30 con 0x7f2c94100e20 2024-09-11T22:06:30.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.179+0000 7f2c9880f640 1 -- 172.21.15.1:0/3913485293 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c9419f920 con 0x7f2c94100e20 2024-09-11T22:06:30.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.179+0000 7f2c8b7fe640 1 -- 172.21.15.1:0/3913485293 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c84035030 con 0x7f2c94100e20 2024-09-11T22:06:30.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.179+0000 7f2c9880f640 1 -- 172.21.15.1:0/3913485293 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c941a7260 con 0x7f2c94100e20 2024-09-11T22:06:30.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.182+0000 7f2c8b7fe640 1 -- 172.21.15.1:0/3913485293 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f2c84042070 con 0x7f2c94100e20 2024-09-11T22:06:30.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.182+0000 7f2c9abe8740 1 Processor -- start 2024-09-11T22:06:30.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.182+0000 7f2c9abe8740 1 -- start start 2024-09-11T22:06:30.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.182+0000 7f2c9abe8740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8c233bb40 0x55c8c233a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:30.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.182+0000 7f2c9abe8740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c8c233a5d0 0x55c8c233aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:30.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.182+0000 7f2c9abe8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c2345370 0x55c8c2347760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:30.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.182+0000 7f2c9abe8740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55c8c22a02f0 con 0x55c8c233a5d0 2024-09-11T22:06:30.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.182+0000 7f2c9abe8740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55c8c2247cf0 con 0x55c8c233bb40 2024-09-11T22:06:30.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.182+0000 7f2c9abe8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c8c229c2b0 con 0x55c8c2345370 2024-09-11T22:06:30.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.182+0000 7f2c9259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c8c233a5d0 0x55c8c233aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:30.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.182+0000 7f2c9259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c8c233a5d0 0x55c8c233aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38064/0 (socket says 172.21.15.1:38064) 2024-09-11T22:06:30.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.182+0000 7f2c9259c640 1 -- 172.21.15.1:0/1224414706 learned_addr learned my addr 172.21.15.1:0/1224414706 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:06:30.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.182+0000 7f2c8bfff640 1 --2- 172.21.15.1:0/1224414706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c2345370 0x55c8c2347760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:30.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.182+0000 7f2c92d9d640 1 --2- 172.21.15.1:0/1224414706 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8c233bb40 0x55c8c233a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:30.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.182+0000 7f2c8b7fe640 1 --2- 172.21.15.1:0/3913485293 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f2c800a2cf0 0x7f2c800a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:30.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.183+0000 7f2c9259c640 1 -- 172.21.15.1:0/1224414706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c2345370 msgr2=0x55c8c2347760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:30.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.183+0000 7f2c9259c640 1 --2- 172.21.15.1:0/1224414706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c2345370 0x55c8c2347760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:30.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.183+0000 7f2c9259c640 1 -- 172.21.15.1:0/1224414706 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8c233bb40 msgr2=0x55c8c233a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:30.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.183+0000 7f2c9259c640 1 --2- 172.21.15.1:0/1224414706 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8c233bb40 0x55c8c233a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:30.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.183+0000 7f2c9259c640 1 -- 172.21.15.1:0/1224414706 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8c222c3a0 con 0x55c8c233a5d0 2024-09-11T22:06:30.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.183+0000 7f2c92d9d640 1 --2- 172.21.15.1:0/3913485293 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f2c800a2cf0 0x7f2c800a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:30.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.183+0000 7f2c8b7fe640 1 -- 172.21.15.1:0/3913485293 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7f2c840efe90 con 0x7f2c94100e20 2024-09-11T22:06:30.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.183+0000 7f2c9259c640 1 --2- 172.21.15.1:0/1224414706 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c8c233a5d0 0x55c8c233aa10 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f2c840ed510 tx=0x7f2c84037f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a059f3ca61c3880f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:30.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.183+0000 7f2c88ff9640 1 -- 172.21.15.1:0/1224414706 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c840bbcd0 con 0x55c8c233a5d0 2024-09-11T22:06:30.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.183+0000 7f2c88ff9640 1 -- 172.21.15.1:0/1224414706 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c84087400 con 0x55c8c233a5d0 2024-09-11T22:06:30.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.183+0000 7f2c88ff9640 1 -- 172.21.15.1:0/1224414706 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c840bca70 con 0x55c8c233a5d0 2024-09-11T22:06:30.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.184+0000 7f2c92d9d640 1 --2- 172.21.15.1:0/3913485293 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f2c800a2cf0 0x7f2c800a51b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c005fd0 tx=0x7f2c8c004a40 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:30.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.184+0000 7f2c9abe8740 1 -- 172.21.15.1:0/1224414706 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c8c233a5d0 msgr2=0x55c8c233aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:30.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.184+0000 7f2c9abe8740 1 --2- 172.21.15.1:0/1224414706 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c8c233a5d0 0x55c8c233aa10 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f2c840ed510 tx=0x7f2c84037f40 comp rx=0 tx=0).stop 2024-09-11T22:06:30.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.184+0000 7f2c9abe8740 1 -- 172.21.15.1:0/1224414706 shutdown_connections 2024-09-11T22:06:30.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.184+0000 7f2c9abe8740 1 --2- 172.21.15.1:0/1224414706 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c2345370 0x55c8c2347760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:30.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.184+0000 7f2c9abe8740 1 --2- 172.21.15.1:0/1224414706 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c8c233a5d0 0x55c8c233aa10 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:30.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.184+0000 7f2c9abe8740 1 --2- 172.21.15.1:0/1224414706 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8c233bb40 0x55c8c233a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:30.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.184+0000 7f2c9abe8740 1 -- 172.21.15.1:0/1224414706 >> 172.21.15.1:0/1224414706 conn(0x55c8c2303310 msgr2=0x55c8c2336c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:30.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.184+0000 7f2c9abe8740 1 -- 172.21.15.1:0/1224414706 shutdown_connections 2024-09-11T22:06:30.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.184+0000 7f2c9abe8740 1 -- 172.21.15.1:0/1224414706 wait complete. 2024-09-11T22:06:30.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.184+0000 7f2c9abe8740 1 -- start start 2024-09-11T22:06:30.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.184+0000 7f2c9abe8740 1 Processor -- start 2024-09-11T22:06:30.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.185+0000 7f2c9abe8740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8c233a5d0 0x55c8c2344d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:30.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.185+0000 7f2c9abe8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c233bb40 0x55c8c2342020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:30.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.185+0000 7f2c9abe8740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c8c2345370 0x55c8c2342610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:30.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.185+0000 7f2c9abe8740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55c8c22a02f0 con 0x55c8c2345370 2024-09-11T22:06:30.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.185+0000 7f2c9abe8740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55c8c225b060 con 0x55c8c233a5d0 2024-09-11T22:06:30.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.185+0000 7f2c9abe8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c8c2347e30 con 0x55c8c233bb40 2024-09-11T22:06:30.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.185+0000 7f2c9abe8740 10 client.0 _ll_register_callbacks cb 0x55c8c1fae8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:06:30.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.185+0000 7f2c92d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c8c2345370 0x55c8c2342610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:30.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.185+0000 7f2c92d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c8c2345370 0x55c8c2342610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38080/0 (socket says 172.21.15.1:38080) 2024-09-11T22:06:30.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.185+0000 7f2c9259c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8c233a5d0 0x55c8c2344d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:30.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.185+0000 7f2c92d9d640 1 -- 172.21.15.1:0/3788486854 learned_addr learned my addr 172.21.15.1:0/3788486854 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:06:30.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.185+0000 7f2c8bfff640 1 --2- 172.21.15.1:0/3788486854 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c233bb40 0x55c8c2342020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:30.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.185+0000 7f2c92d9d640 1 -- 172.21.15.1:0/3788486854 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c233bb40 msgr2=0x55c8c2342020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:30.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.185+0000 7f2c92d9d640 1 --2- 172.21.15.1:0/3788486854 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c233bb40 0x55c8c2342020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:30.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.185+0000 7f2c92d9d640 1 -- 172.21.15.1:0/3788486854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8c233a5d0 msgr2=0x55c8c2344d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:30.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.185+0000 7f2c92d9d640 1 --2- 172.21.15.1:0/3788486854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8c233a5d0 0x55c8c2344d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:30.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.185+0000 7f2c92d9d640 1 -- 172.21.15.1:0/3788486854 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8c20113c0 con 0x55c8c2345370 2024-09-11T22:06:30.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.185+0000 7f2c92d9d640 1 --2- 172.21.15.1:0/3788486854 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c8c2345370 0x55c8c2342610 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c01be30 tx=0x7f2c8c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc0a78e476e751bf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:30.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.186+0000 7f2c79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:06:30.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.186+0000 7f2c79ffb640 1 -- 172.21.15.1:0/3788486854 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c024440 con 0x55c8c2345370 2024-09-11T22:06:30.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.186+0000 7f2c79ffb640 1 -- 172.21.15.1:0/3788486854 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7f2c8c018420 con 0x55c8c2345370 2024-09-11T22:06:30.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.186+0000 7f2c9abe8740 1 -- 172.21.15.1:0/3788486854 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c84087400 con 0x55c8c2345370 2024-09-11T22:06:30.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.186+0000 7f2c79ffb640 1 -- 172.21.15.1:0/3788486854 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f2c8c018620 con 0x55c8c2345370 2024-09-11T22:06:30.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.186+0000 7f2c9abe8740 10 client.17625 fetch_fsmap learned FSMap version 46 2024-09-11T22:06:30.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.186+0000 7f2c9abe8740 1 -- 172.21.15.1:0/3788486854 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55c8c23537b0 con 0x55c8c2345370 2024-09-11T22:06:30.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.186+0000 7f2c79ffb640 1 -- 172.21.15.1:0/3788486854 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c84087400 con 0x55c8c2345370 2024-09-11T22:06:30.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.186+0000 7f2c79ffb640 1 -- 172.21.15.1:0/3788486854 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 46) ==== 8442+0+0 (secure 0 0 0) 0x7f2c8c02a030 con 0x55c8c2345370 2024-09-11T22:06:30.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.186+0000 7f2c9abe8740 10 client.17625 fetch_fsmap finished waiting for FSMap version 46 2024-09-11T22:06:30.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.186+0000 7f2c9abe8740 10 client.17625 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-11T22:06:30.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.186+0000 7f2c9abe8740 20 client.17625 populate_metadata read hostname 'smithi001' 2024-09-11T22:06:30.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.186+0000 7f2c9abe8740 1 --2- 172.21.15.1:0/3788486854 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x55c8c2354d10 0x55c8c2357170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:30.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.186+0000 7f2c9abe8740 4 client.17625 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:06:30.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.186+0000 7f2c9abe8740 1 -- 172.21.15.1:0/3788486854 --> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c8c22ad860 con 0x55c8c2354d10 2024-09-11T22:06:30.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.187+0000 7f2c9259c640 1 --2- 172.21.15.1:0/3788486854 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x55c8c2354d10 0x55c8c2357170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:30.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.187+0000 7f2c9259c640 1 --2- 172.21.15.1:0/3788486854 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x55c8c2354d10 0x55c8c2357170 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c1d4288a1815ad33 server_cookie=8a3f6c63eaceda42 in_seq=0 out_seq=0 2024-09-11T22:06:30.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.187+0000 7f2c79ffb640 10 client.17625 ms_handle_connect on v2:172.21.15.161:6834/828381592 2024-09-11T22:06:30.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.188+0000 7f2c79ffb640 1 -- 172.21.15.1:0/3788486854 <== mds.4 v2:172.21.15.161:6834/828381592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c8c22ad860 con 0x55c8c2354d10 2024-09-11T22:06:30.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.188+0000 7f2c79ffb640 10 client.17625 handle_command_reply: tid=0 2024-09-11T22:06:30.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.188+0000 7f2c9abe8740 1 client.17625 shutdown 2024-09-11T22:06:30.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.188+0000 7f2c9abe8740 1 -- 172.21.15.1:0/3788486854 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c8c2345370 msgr2=0x55c8c2342610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:30.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.188+0000 7f2c9abe8740 1 --2- 172.21.15.1:0/3788486854 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c8c2345370 0x55c8c2342610 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c01be30 tx=0x7f2c8c004ab0 comp rx=0 tx=0).stop 2024-09-11T22:06:30.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.188+0000 7f2c9abe8740 1 -- 172.21.15.1:0/3788486854 shutdown_connections 2024-09-11T22:06:30.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.188+0000 7f2c9abe8740 1 --2- 172.21.15.1:0/3788486854 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c8c2345370 0x55c8c2342610 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:30.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.188+0000 7f2c9abe8740 1 --2- 172.21.15.1:0/3788486854 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c233bb40 0x55c8c2342020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:30.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.188+0000 7f2c9abe8740 1 --2- 172.21.15.1:0/3788486854 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x55c8c2354d10 0x55c8c2357170 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:30.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.188+0000 7f2c9abe8740 1 --2- 172.21.15.1:0/3788486854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8c233a5d0 0x55c8c2344d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:30.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.188+0000 7f2c79ffb640 0 client.17625 ms_handle_reset on v2:172.21.15.161:6834/828381592 2024-09-11T22:06:30.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.188+0000 7f2c9abe8740 1 -- 172.21.15.1:0/3788486854 >> 172.21.15.1:0/3788486854 conn(0x55c8c2303310 msgr2=0x55c8c233c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:30.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.188+0000 7f2c9abe8740 1 -- 172.21.15.1:0/3788486854 shutdown_connections 2024-09-11T22:06:30.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.188+0000 7f2c9abe8740 1 -- 172.21.15.1:0/3788486854 wait complete. 2024-09-11T22:06:30.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.188+0000 7f2c9abe8740 20 client.17625 trim_cache size 0 max 16384 2024-09-11T22:06:30.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.201+0000 7f2c9abe8740 1 Processor -- start 2024-09-11T22:06:30.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.201+0000 7f2c9abe8740 1 -- start start 2024-09-11T22:06:30.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.202+0000 7f2c9abe8740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c8c233a5d0 0x55c8c2350f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:30.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.202+0000 7f2c9abe8740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8c233bb40 0x55c8c2336c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:30.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.202+0000 7f2c9abe8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c2345370 0x55c8c235ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:30.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.202+0000 7f2c9abe8740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2c941afc40 con 0x55c8c233a5d0 2024-09-11T22:06:30.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.202+0000 7f2c9abe8740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2c941b1a80 con 0x55c8c233bb40 2024-09-11T22:06:30.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.202+0000 7f2c9abe8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2c941b03d0 con 0x55c8c2345370 2024-09-11T22:06:30.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.202+0000 7f2c9259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c8c233a5d0 0x55c8c2350f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:30.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.202+0000 7f2c9259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c8c233a5d0 0x55c8c2350f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38096/0 (socket says 172.21.15.1:38096) 2024-09-11T22:06:30.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.202+0000 7f2c9259c640 1 -- 172.21.15.1:0/3610997703 learned_addr learned my addr 172.21.15.1:0/3610997703 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:06:30.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.202+0000 7f2c8bfff640 1 --2- 172.21.15.1:0/3610997703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8c233bb40 0x55c8c2336c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:30.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.202+0000 7f2c92d9d640 1 --2- 172.21.15.1:0/3610997703 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c2345370 0x55c8c235ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:30.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.202+0000 7f2c9259c640 1 -- 172.21.15.1:0/3610997703 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c2345370 msgr2=0x55c8c235ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:30.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.202+0000 7f2c9259c640 1 --2- 172.21.15.1:0/3610997703 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c2345370 0x55c8c235ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:30.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.202+0000 7f2c9259c640 1 -- 172.21.15.1:0/3610997703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8c233bb40 msgr2=0x55c8c2336c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:30.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.202+0000 7f2c9259c640 1 --2- 172.21.15.1:0/3610997703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8c233bb40 0x55c8c2336c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:30.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.202+0000 7f2c9259c640 1 -- 172.21.15.1:0/3610997703 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8c2331a30 con 0x55c8c233a5d0 2024-09-11T22:06:30.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.202+0000 7f2c8bfff640 1 --2- 172.21.15.1:0/3610997703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8c233bb40 0x55c8c2336c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:06:30.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.202+0000 7f2c9259c640 1 --2- 172.21.15.1:0/3610997703 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c8c233a5d0 0x55c8c2350f50 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f2c840bc300 tx=0x7f2c840f3440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db6da4ce890ac0c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:30.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.202+0000 7f2c7bfff640 1 -- 172.21.15.1:0/3610997703 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c840bbcd0 con 0x55c8c233a5d0 2024-09-11T22:06:30.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.203+0000 7f2c7bfff640 1 -- 172.21.15.1:0/3610997703 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c840bca70 con 0x55c8c233a5d0 2024-09-11T22:06:30.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.203+0000 7f2c7bfff640 1 -- 172.21.15.1:0/3610997703 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c840bbeb0 con 0x55c8c233a5d0 2024-09-11T22:06:30.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.203+0000 7f2c9abe8740 1 -- 172.21.15.1:0/3610997703 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c8c233a5d0 msgr2=0x55c8c2350f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:30.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.203+0000 7f2c9abe8740 1 --2- 172.21.15.1:0/3610997703 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c8c233a5d0 0x55c8c2350f50 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f2c840bc300 tx=0x7f2c840f3440 comp rx=0 tx=0).stop 2024-09-11T22:06:30.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.203+0000 7f2c9abe8740 1 -- 172.21.15.1:0/3610997703 shutdown_connections 2024-09-11T22:06:30.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.203+0000 7f2c9abe8740 1 --2- 172.21.15.1:0/3610997703 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c2345370 0x55c8c235ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:30.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.203+0000 7f2c9abe8740 1 --2- 172.21.15.1:0/3610997703 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8c233bb40 0x55c8c2336c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:30.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.203+0000 7f2c9abe8740 1 --2- 172.21.15.1:0/3610997703 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c8c233a5d0 0x55c8c2350f50 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:30.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.203+0000 7f2c9abe8740 1 -- 172.21.15.1:0/3610997703 >> 172.21.15.1:0/3610997703 conn(0x55c8c2303310 msgr2=0x55c8c2341110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:30.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.203+0000 7f2c9abe8740 1 -- 172.21.15.1:0/3610997703 shutdown_connections 2024-09-11T22:06:30.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.203+0000 7f2c9abe8740 1 -- 172.21.15.1:0/3610997703 wait complete. 2024-09-11T22:06:30.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.204+0000 7f2c9abe8740 1 -- start start 2024-09-11T22:06:30.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.204+0000 7f2c9abe8740 1 Processor -- start 2024-09-11T22:06:30.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.204+0000 7f2c9abe8740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c8c233a5d0 0x55c8c2349250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:30.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.204+0000 7f2c9abe8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c233bb40 0x55c8c2349790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:30.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.204+0000 7f2c9abe8740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8c2345370 0x55c8c23bea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:30.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.204+0000 7f2c9abe8740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2c941b12f0 con 0x55c8c233a5d0 2024-09-11T22:06:30.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.204+0000 7f2c9abe8740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f2c941b0b60 con 0x55c8c2345370 2024-09-11T22:06:30.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.204+0000 7f2c9abe8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f2c941adc20 con 0x55c8c233bb40 2024-09-11T22:06:30.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.204+0000 7f2c9abe8740 10 client.0 _ll_register_callbacks cb 0x7f2c8c00ba10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:06:30.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.204+0000 7f2c9259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c8c233a5d0 0x55c8c2349250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:30.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.204+0000 7f2c9259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c8c233a5d0 0x55c8c2349250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38106/0 (socket says 172.21.15.1:38106) 2024-09-11T22:06:30.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.204+0000 7f2c92d9d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8c2345370 0x55c8c23bea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:30.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.204+0000 7f2c9259c640 1 -- 172.21.15.1:0/664923776 learned_addr learned my addr 172.21.15.1:0/664923776 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:06:30.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.204+0000 7f2c8bfff640 1 --2- 172.21.15.1:0/664923776 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c233bb40 0x55c8c2349790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:30.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.204+0000 7f2c9259c640 1 -- 172.21.15.1:0/664923776 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c233bb40 msgr2=0x55c8c2349790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:30.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.204+0000 7f2c9259c640 1 --2- 172.21.15.1:0/664923776 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c233bb40 0x55c8c2349790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:30.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.204+0000 7f2c9259c640 1 -- 172.21.15.1:0/664923776 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8c2345370 msgr2=0x55c8c23bea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:30.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.204+0000 7f2c9259c640 1 --2- 172.21.15.1:0/664923776 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8c2345370 0x55c8c23bea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:30.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.204+0000 7f2c9259c640 1 -- 172.21.15.1:0/664923776 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8c23bef90 con 0x55c8c233a5d0 2024-09-11T22:06:30.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.204+0000 7f2c8bfff640 1 --2- 172.21.15.1:0/664923776 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c233bb40 0x55c8c2349790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:06:30.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.205+0000 7f2c9259c640 1 --2- 172.21.15.1:0/664923776 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c8c233a5d0 0x55c8c2349250 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f2c84086360 tx=0x7f2c840f34b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4bcc75b3898ee11 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:30.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.205+0000 7f2c79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:06:30.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.205+0000 7f2c79ffb640 1 -- 172.21.15.1:0/664923776 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c840bbeb0 con 0x55c8c233a5d0 2024-09-11T22:06:30.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.205+0000 7f2c79ffb640 1 -- 172.21.15.1:0/664923776 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7f2c840f3670 con 0x55c8c233a5d0 2024-09-11T22:06:30.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.205+0000 7f2c9abe8740 1 -- 172.21.15.1:0/664923776 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c840bca70 con 0x55c8c233a5d0 2024-09-11T22:06:30.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.205+0000 7f2c79ffb640 1 -- 172.21.15.1:0/664923776 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f2c840bbcd0 con 0x55c8c233a5d0 2024-09-11T22:06:30.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.205+0000 7f2c9abe8740 10 client.17631 fetch_fsmap learned FSMap version 46 2024-09-11T22:06:30.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.205+0000 7f2c9abe8740 1 -- 172.21.15.1:0/664923776 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55c8c23c00e0 con 0x55c8c233a5d0 2024-09-11T22:06:30.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.205+0000 7f2c79ffb640 1 -- 172.21.15.1:0/664923776 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c840bca70 con 0x55c8c233a5d0 2024-09-11T22:06:30.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.205+0000 7f2c79ffb640 1 -- 172.21.15.1:0/664923776 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 46) ==== 8442+0+0 (secure 0 0 0) 0x7f2c840ed820 con 0x55c8c233a5d0 2024-09-11T22:06:30.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.205+0000 7f2c9abe8740 10 client.17631 fetch_fsmap finished waiting for FSMap version 46 2024-09-11T22:06:30.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.205+0000 7f2c9abe8740 10 client.17631 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-11T22:06:30.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.205+0000 7f2c9abe8740 20 client.17631 populate_metadata read hostname 'smithi001' 2024-09-11T22:06:30.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.205+0000 7f2c9abe8740 1 --2- 172.21.15.1:0/664923776 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x55c8c23c04f0 0x55c8c23c2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:30.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.205+0000 7f2c9abe8740 4 client.17631 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:06:30.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.205+0000 7f2c9abe8740 1 -- 172.21.15.1:0/664923776 --> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c8c22a7d70 con 0x55c8c23c04f0 2024-09-11T22:06:30.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.206+0000 7f2c8bfff640 1 --2- 172.21.15.1:0/664923776 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x55c8c23c04f0 0x55c8c23c2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:30.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.206+0000 7f2c8bfff640 1 --2- 172.21.15.1:0/664923776 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x55c8c23c04f0 0x55c8c23c2910 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2341fe56763a6ff8 server_cookie=c30ae06071f9544d in_seq=0 out_seq=0 2024-09-11T22:06:30.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.206+0000 7f2c79ffb640 10 client.17631 ms_handle_connect on v2:172.21.15.161:6834/828381592 2024-09-11T22:06:30.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.206+0000 7f2c79ffb640 1 -- 172.21.15.1:0/664923776 <== mds.4 v2:172.21.15.161:6834/828381592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c8c22a7d70 con 0x55c8c23c04f0 2024-09-11T22:06:30.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.206+0000 7f2c79ffb640 10 client.17631 handle_command_reply: tid=0 2024-09-11T22:06:30.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.206+0000 7f2c9abe8740 1 client.17631 shutdown 2024-09-11T22:06:30.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.206+0000 7f2c9abe8740 1 -- 172.21.15.1:0/664923776 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c8c233a5d0 msgr2=0x55c8c2349250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:30.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.206+0000 7f2c9abe8740 1 --2- 172.21.15.1:0/664923776 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c8c233a5d0 0x55c8c2349250 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f2c84086360 tx=0x7f2c840f34b0 comp rx=0 tx=0).stop 2024-09-11T22:06:30.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.207+0000 7f2c9abe8740 1 -- 172.21.15.1:0/664923776 shutdown_connections 2024-09-11T22:06:30.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.207+0000 7f2c9abe8740 1 --2- 172.21.15.1:0/664923776 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x55c8c23c04f0 0x55c8c23c2910 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:30.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.207+0000 7f2c79ffb640 0 client.17631 ms_handle_reset on v2:172.21.15.161:6834/828381592 2024-09-11T22:06:30.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.207+0000 7f2c9abe8740 1 --2- 172.21.15.1:0/664923776 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c8c2345370 0x55c8c23bea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:30.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.207+0000 7f2c9abe8740 1 --2- 172.21.15.1:0/664923776 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c8c233bb40 0x55c8c2349790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:30.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.207+0000 7f2c9abe8740 1 --2- 172.21.15.1:0/664923776 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c8c233a5d0 0x55c8c2349250 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:30.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.207+0000 7f2c9abe8740 1 -- 172.21.15.1:0/664923776 >> 172.21.15.1:0/664923776 conn(0x55c8c2303310 msgr2=0x55c8c2343880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:30.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.207+0000 7f2c9abe8740 1 -- 172.21.15.1:0/664923776 shutdown_connections 2024-09-11T22:06:30.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.207+0000 7f2c9abe8740 1 -- 172.21.15.1:0/664923776 wait complete. 2024-09-11T22:06:30.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.207+0000 7f2c9abe8740 20 client.17631 trim_cache size 0 max 16384 2024-09-11T22:06:30.227 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-11T22:06:30.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.207+0000 7f2c9880f640 1 -- 172.21.15.1:0/3913485293 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f2c800a2cf0 msgr2=0x7f2c800a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:30.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.207+0000 7f2c9880f640 1 --2- 172.21.15.1:0/3913485293 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f2c800a2cf0 0x7f2c800a51b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c005fd0 tx=0x7f2c8c004a40 comp rx=0 tx=0).stop 2024-09-11T22:06:30.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.207+0000 7f2c9880f640 1 -- 172.21.15.1:0/3913485293 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c94100e20 msgr2=0x7f2c9419a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:30.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.207+0000 7f2c9880f640 1 --2- 172.21.15.1:0/3913485293 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c94100e20 0x7f2c9419a6e0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f2c84006010 tx=0x7f2c84003c80 comp rx=0 tx=0).stop 2024-09-11T22:06:30.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.207+0000 7f2c9880f640 1 -- 172.21.15.1:0/3913485293 shutdown_connections 2024-09-11T22:06:30.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.207+0000 7f2c9880f640 1 --2- 172.21.15.1:0/3913485293 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f2c800a2cf0 0x7f2c800a51b0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:30.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.208+0000 7f2c9880f640 1 --2- 172.21.15.1:0/3913485293 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c94102ef0 0x7f2c9419efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:30.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.208+0000 7f2c9880f640 1 --2- 172.21.15.1:0/3913485293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c94102570 0x7f2c9419ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:30.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.208+0000 7f2c9880f640 1 --2- 172.21.15.1:0/3913485293 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c94100e20 0x7f2c9419a6e0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:30.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.208+0000 7f2c9880f640 1 -- 172.21.15.1:0/3913485293 >> 172.21.15.1:0/3913485293 conn(0x7f2c940fc7d0 msgr2=0x7f2c9406a070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:30.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.208+0000 7f2c9880f640 1 -- 172.21.15.1:0/3913485293 shutdown_connections 2024-09-11T22:06:30.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:30.208+0000 7f2c9880f640 1 -- 172.21.15.1:0/3913485293 wait complete. 2024-09-11T22:06:30.949 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T22:06:31.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:06:31 smithi001 ceph-mon[30935]: pgmap v289: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 0 B/s wr, 15 op/s 2024-09-11T22:06:31.218 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:06:31.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:06:31 smithi161 ceph-mon[35615]: pgmap v289: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 0 B/s wr, 15 op/s 2024-09-11T22:06:31.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:06:31 smithi053 ceph-mon[34908]: pgmap v289: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 0 B/s wr, 15 op/s 2024-09-11T22:06:31.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.779+0000 7f74d56cf640 1 -- 172.21.15.1:0/1014366236 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f74d0106a80 msgr2=0x7f74d0106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:31.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.779+0000 7f74d56cf640 1 --2- 172.21.15.1:0/1014366236 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f74d0106a80 0x7f74d0106e60 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f74c0009930 tx=0x7f74c002a420 comp rx=0 tx=0).stop 2024-09-11T22:06:31.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.779+0000 7f74d56cf640 1 -- 172.21.15.1:0/1014366236 shutdown_connections 2024-09-11T22:06:31.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.779+0000 7f74d56cf640 1 --2- 172.21.15.1:0/1014366236 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74d0101420 0x7f74d010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:31.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.779+0000 7f74d56cf640 1 --2- 172.21.15.1:0/1014366236 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f74d0100a80 0x7f74d0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:31.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.779+0000 7f74d56cf640 1 --2- 172.21.15.1:0/1014366236 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f74d0106a80 0x7f74d0106e60 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:31.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.779+0000 7f74d56cf640 1 -- 172.21.15.1:0/1014366236 >> 172.21.15.1:0/1014366236 conn(0x7f74d00fc7a0 msgr2=0x7f74d00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:31.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.779+0000 7f74d56cf640 1 -- 172.21.15.1:0/1014366236 shutdown_connections 2024-09-11T22:06:31.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.779+0000 7f74d56cf640 1 -- 172.21.15.1:0/1014366236 wait complete. 2024-09-11T22:06:31.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.780+0000 7f74d56cf640 1 Processor -- start 2024-09-11T22:06:31.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.780+0000 7f74d56cf640 1 -- start start 2024-09-11T22:06:31.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.780+0000 7f74d56cf640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f74d0100a80 0x7f74d0198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:31.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.780+0000 7f74d56cf640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f74d0101420 0x7f74d01989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:31.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.780+0000 7f74d56cf640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74d0106a80 0x7f74d019cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:31.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.780+0000 7f74d56cf640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f74d0110e80 con 0x7f74d0100a80 2024-09-11T22:06:31.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.780+0000 7f74d56cf640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f74d0110d00 con 0x7f74d0106a80 2024-09-11T22:06:31.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.780+0000 7f74d56cf640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f74d0111000 con 0x7f74d0101420 2024-09-11T22:06:31.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.780+0000 7f74ceffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f74d0100a80 0x7f74d0198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:31.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.780+0000 7f74ceffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f74d0100a80 0x7f74d0198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38132/0 (socket says 172.21.15.1:38132) 2024-09-11T22:06:31.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.780+0000 7f74ceffd640 1 -- 172.21.15.1:0/1714801139 learned_addr learned my addr 172.21.15.1:0/1714801139 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:06:31.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.780+0000 7f74c7fff640 1 --2- 172.21.15.1:0/1714801139 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f74d0101420 0x7f74d01989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:31.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.781+0000 7f74cf7fe640 1 --2- 172.21.15.1:0/1714801139 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74d0106a80 0x7f74d019cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:31.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.781+0000 7f74ceffd640 1 -- 172.21.15.1:0/1714801139 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f74d0101420 msgr2=0x7f74d01989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:31.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.781+0000 7f74ceffd640 1 --2- 172.21.15.1:0/1714801139 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f74d0101420 0x7f74d01989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:31.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.781+0000 7f74ceffd640 1 -- 172.21.15.1:0/1714801139 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74d0106a80 msgr2=0x7f74d019cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:31.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.781+0000 7f74ceffd640 1 --2- 172.21.15.1:0/1714801139 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74d0106a80 0x7f74d019cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:31.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.781+0000 7f74ceffd640 1 -- 172.21.15.1:0/1714801139 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74d019d420 con 0x7f74d0100a80 2024-09-11T22:06:31.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.781+0000 7f74ceffd640 1 --2- 172.21.15.1:0/1714801139 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f74d0100a80 0x7f74d0198470 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f74c0009230 tx=0x7f74c0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:31.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.781+0000 7f74ccff9640 1 -- 172.21.15.1:0/1714801139 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c003f030 con 0x7f74d0100a80 2024-09-11T22:06:31.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.781+0000 7f74ccff9640 1 -- 172.21.15.1:0/1714801139 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74c0003ad0 con 0x7f74d0100a80 2024-09-11T22:06:31.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.781+0000 7f74d56cf640 1 -- 172.21.15.1:0/1714801139 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74d019d6b0 con 0x7f74d0100a80 2024-09-11T22:06:31.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.781+0000 7f74ccff9640 1 -- 172.21.15.1:0/1714801139 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c0035070 con 0x7f74d0100a80 2024-09-11T22:06:31.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.781+0000 7f74d56cf640 1 -- 172.21.15.1:0/1714801139 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74d01a4ff0 con 0x7f74d0100a80 2024-09-11T22:06:31.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.784+0000 7f74ccff9640 1 -- 172.21.15.1:0/1714801139 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f74c0042030 con 0x7f74d0100a80 2024-09-11T22:06:31.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.784+0000 7f74d7aa8740 1 Processor -- start 2024-09-11T22:06:31.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.784+0000 7f74d7aa8740 1 -- start start 2024-09-11T22:06:31.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.784+0000 7f74d7aa8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ffa92afb40 0x55ffa92ae090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:31.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.784+0000 7f74d7aa8740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ffa92ae5d0 0x55ffa92aea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:31.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.784+0000 7f74d7aa8740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ffa92b9370 0x55ffa92bb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:31.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.784+0000 7f74d7aa8740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55ffa92142f0 con 0x55ffa92ae5d0 2024-09-11T22:06:31.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.784+0000 7f74d7aa8740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55ffa91bbcf0 con 0x55ffa92b9370 2024-09-11T22:06:31.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.784+0000 7f74d7aa8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ffa92102b0 con 0x55ffa92afb40 2024-09-11T22:06:31.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.784+0000 7f74ceffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ffa92ae5d0 0x55ffa92aea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:31.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.784+0000 7f74ccff9640 1 --2- 172.21.15.1:0/1714801139 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f74bc0a2cf0 0x7f74bc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:31.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.784+0000 7f74ceffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ffa92ae5d0 0x55ffa92aea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38140/0 (socket says 172.21.15.1:38140) 2024-09-11T22:06:31.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.785+0000 7f74ceffd640 1 -- 172.21.15.1:0/2609888159 learned_addr learned my addr 172.21.15.1:0/2609888159 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:06:31.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.785+0000 7f74cf7fe640 1 --2- 172.21.15.1:0/2609888159 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ffa92afb40 0x55ffa92ae090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:31.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.785+0000 7f74cf7fe640 1 --2- 172.21.15.1:0/1714801139 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f74bc0a2cf0 0x7f74bc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:31.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.785+0000 7f74c7fff640 1 --2- 172.21.15.1:0/2609888159 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ffa92b9370 0x55ffa92bb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:31.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.785+0000 7f74ccff9640 1 -- 172.21.15.1:0/1714801139 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7f74c00efa20 con 0x7f74d0100a80 2024-09-11T22:06:31.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.785+0000 7f74ceffd640 1 -- 172.21.15.1:0/2609888159 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ffa92afb40 msgr2=0x55ffa92ae090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:31.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.785+0000 7f74ceffd640 1 --2- 172.21.15.1:0/2609888159 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ffa92afb40 0x55ffa92ae090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:31.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.785+0000 7f74ceffd640 1 -- 172.21.15.1:0/2609888159 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ffa92b9370 msgr2=0x55ffa92bb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:31.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.785+0000 7f74ceffd640 1 --2- 172.21.15.1:0/2609888159 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ffa92b9370 0x55ffa92bb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:31.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.785+0000 7f74ceffd640 1 -- 172.21.15.1:0/2609888159 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffa91a03a0 con 0x55ffa92ae5d0 2024-09-11T22:06:31.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.785+0000 7f74c7fff640 1 --2- 172.21.15.1:0/2609888159 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ffa92b9370 0x55ffa92bb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:06:31.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.785+0000 7f74ceffd640 1 --2- 172.21.15.1:0/2609888159 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ffa92ae5d0 0x55ffa92aea10 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f74c00bbcb0 tx=0x7f74c0009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b268e93541b711c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:31.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.785+0000 7f74cf7fe640 1 --2- 172.21.15.1:0/1714801139 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f74bc0a2cf0 0x7f74bc0a51b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f74c8009990 tx=0x7f74c8009aa0 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:31.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.785+0000 7f74c57fa640 1 -- 172.21.15.1:0/2609888159 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c0049070 con 0x55ffa92ae5d0 2024-09-11T22:06:31.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.785+0000 7f74c57fa640 1 -- 172.21.15.1:0/2609888159 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74c00bbfb0 con 0x55ffa92ae5d0 2024-09-11T22:06:31.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.785+0000 7f74c57fa640 1 -- 172.21.15.1:0/2609888159 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c0003c80 con 0x55ffa92ae5d0 2024-09-11T22:06:31.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.786+0000 7f74d7aa8740 1 -- 172.21.15.1:0/2609888159 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ffa92ae5d0 msgr2=0x55ffa92aea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:31.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.786+0000 7f74d7aa8740 1 --2- 172.21.15.1:0/2609888159 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ffa92ae5d0 0x55ffa92aea10 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f74c00bbcb0 tx=0x7f74c0009680 comp rx=0 tx=0).stop 2024-09-11T22:06:31.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.786+0000 7f74d7aa8740 1 -- 172.21.15.1:0/2609888159 shutdown_connections 2024-09-11T22:06:31.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.786+0000 7f74d7aa8740 1 --2- 172.21.15.1:0/2609888159 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ffa92b9370 0x55ffa92bb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:31.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.786+0000 7f74d7aa8740 1 --2- 172.21.15.1:0/2609888159 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ffa92ae5d0 0x55ffa92aea10 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:31.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.786+0000 7f74d7aa8740 1 --2- 172.21.15.1:0/2609888159 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ffa92afb40 0x55ffa92ae090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:31.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.786+0000 7f74d7aa8740 1 -- 172.21.15.1:0/2609888159 >> 172.21.15.1:0/2609888159 conn(0x55ffa9277310 msgr2=0x55ffa92aac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:31.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.786+0000 7f74d7aa8740 1 -- 172.21.15.1:0/2609888159 shutdown_connections 2024-09-11T22:06:31.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.786+0000 7f74d7aa8740 1 -- 172.21.15.1:0/2609888159 wait complete. 2024-09-11T22:06:31.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.786+0000 7f74d7aa8740 1 -- start start 2024-09-11T22:06:31.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.787+0000 7f74d7aa8740 1 Processor -- start 2024-09-11T22:06:31.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.787+0000 7f74d7aa8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ffa92ae5d0 0x55ffa92b8d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:31.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.787+0000 7f74d7aa8740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ffa92afb40 0x55ffa92b6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:31.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.787+0000 7f74d7aa8740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ffa92b9370 0x55ffa92b6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:31.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.787+0000 7f74d7aa8740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55ffa92142f0 con 0x55ffa92afb40 2024-09-11T22:06:31.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.787+0000 7f74d7aa8740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55ffa91cf060 con 0x55ffa92b9370 2024-09-11T22:06:31.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.787+0000 7f74d7aa8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55ffa92bbe30 con 0x55ffa92ae5d0 2024-09-11T22:06:31.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.787+0000 7f74d7aa8740 10 client.0 _ll_register_callbacks cb 0x55ffa8f228c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:06:31.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.787+0000 7f74c7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ffa92afb40 0x55ffa92b6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:31.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.787+0000 7f74c7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ffa92afb40 0x55ffa92b6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38150/0 (socket says 172.21.15.1:38150) 2024-09-11T22:06:31.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.787+0000 7f74c7fff640 1 -- 172.21.15.1:0/2018886477 learned_addr learned my addr 172.21.15.1:0/2018886477 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:06:31.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.787+0000 7f74cf7fe640 1 --2- 172.21.15.1:0/2018886477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ffa92b9370 0x55ffa92b6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:31.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.787+0000 7f74ceffd640 1 --2- 172.21.15.1:0/2018886477 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ffa92ae5d0 0x55ffa92b8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:31.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.787+0000 7f74c7fff640 1 -- 172.21.15.1:0/2018886477 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ffa92ae5d0 msgr2=0x55ffa92b8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:31.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.787+0000 7f74c7fff640 1 --2- 172.21.15.1:0/2018886477 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ffa92ae5d0 0x55ffa92b8d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:31.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.787+0000 7f74c7fff640 1 -- 172.21.15.1:0/2018886477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ffa92b9370 msgr2=0x55ffa92b6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:31.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.787+0000 7f74c7fff640 1 --2- 172.21.15.1:0/2018886477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ffa92b9370 0x55ffa92b6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:31.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.787+0000 7f74c7fff640 1 -- 172.21.15.1:0/2018886477 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffa8f853c0 con 0x55ffa92afb40 2024-09-11T22:06:31.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.787+0000 7f74cf7fe640 1 --2- 172.21.15.1:0/2018886477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ffa92b9370 0x55ffa92b6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:06:31.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.787+0000 7f74c7fff640 1 --2- 172.21.15.1:0/2018886477 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ffa92afb40 0x55ffa92b6020 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f74b8012700 tx=0x7f74b8012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=535f10853ef07335 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:31.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.787+0000 7f74b67fc640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:06:31.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.788+0000 7f74b67fc640 1 -- 172.21.15.1:0/2018886477 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b8004050 con 0x55ffa92afb40 2024-09-11T22:06:31.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.788+0000 7f74b67fc640 1 -- 172.21.15.1:0/2018886477 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7f74b80041f0 con 0x55ffa92afb40 2024-09-11T22:06:31.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.788+0000 7f74d7aa8740 1 -- 172.21.15.1:0/2018886477 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74c00bbfb0 con 0x55ffa92afb40 2024-09-11T22:06:31.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.788+0000 7f74b67fc640 1 -- 172.21.15.1:0/2018886477 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f74b80043f0 con 0x55ffa92afb40 2024-09-11T22:06:31.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.788+0000 7f74d7aa8740 10 client.17643 fetch_fsmap learned FSMap version 46 2024-09-11T22:06:31.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.788+0000 7f74d7aa8740 1 -- 172.21.15.1:0/2018886477 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55ffa92c77b0 con 0x55ffa92afb40 2024-09-11T22:06:31.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.788+0000 7f74b67fc640 1 -- 172.21.15.1:0/2018886477 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74c00bbfb0 con 0x55ffa92afb40 2024-09-11T22:06:31.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.788+0000 7f74b67fc640 1 -- 172.21.15.1:0/2018886477 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 46) ==== 8442+0+0 (secure 0 0 0) 0x7f74b8003c60 con 0x55ffa92afb40 2024-09-11T22:06:31.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.788+0000 7f74d7aa8740 10 client.17643 fetch_fsmap finished waiting for FSMap version 46 2024-09-11T22:06:31.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.788+0000 7f74d7aa8740 10 client.17643 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-11T22:06:31.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.788+0000 7f74d7aa8740 20 client.17643 populate_metadata read hostname 'smithi001' 2024-09-11T22:06:31.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.788+0000 7f74d7aa8740 1 --2- 172.21.15.1:0/2018886477 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x55ffa92c8d60 0x55ffa92cb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:31.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.788+0000 7f74d7aa8740 4 client.17643 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:06:31.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.788+0000 7f74d7aa8740 1 -- 172.21.15.1:0/2018886477 --> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ffa9221860 con 0x55ffa92c8d60 2024-09-11T22:06:31.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.789+0000 7f74ceffd640 1 --2- 172.21.15.1:0/2018886477 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x55ffa92c8d60 0x55ffa92cb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:31.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.789+0000 7f74b67fc640 10 client.17643 ms_handle_connect on v2:172.21.15.161:6836/1799318656 2024-09-11T22:06:31.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.789+0000 7f74ceffd640 1 --2- 172.21.15.1:0/2018886477 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x55ffa92c8d60 0x55ffa92cb1c0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d453edc952f44308 server_cookie=cf0216a1e833590 in_seq=0 out_seq=0 2024-09-11T22:06:31.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.790+0000 7f74b67fc640 1 -- 172.21.15.1:0/2018886477 <== mds.3 v2:172.21.15.161:6836/1799318656 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ffa9221860 con 0x55ffa92c8d60 2024-09-11T22:06:31.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.790+0000 7f74b67fc640 10 client.17643 handle_command_reply: tid=0 2024-09-11T22:06:31.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.790+0000 7f74d7aa8740 1 client.17643 shutdown 2024-09-11T22:06:31.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.790+0000 7f74d7aa8740 1 -- 172.21.15.1:0/2018886477 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ffa92afb40 msgr2=0x55ffa92b6020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:31.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.790+0000 7f74d7aa8740 1 --2- 172.21.15.1:0/2018886477 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ffa92afb40 0x55ffa92b6020 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f74b8012700 tx=0x7f74b8012bc0 comp rx=0 tx=0).stop 2024-09-11T22:06:31.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.790+0000 7f74d7aa8740 1 -- 172.21.15.1:0/2018886477 shutdown_connections 2024-09-11T22:06:31.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.790+0000 7f74d7aa8740 1 --2- 172.21.15.1:0/2018886477 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x55ffa92c8d60 0x55ffa92cb1c0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:31.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.790+0000 7f74d7aa8740 1 --2- 172.21.15.1:0/2018886477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ffa92b9370 0x55ffa92b6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:31.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.790+0000 7f74b67fc640 0 client.17643 ms_handle_reset on v2:172.21.15.161:6836/1799318656 2024-09-11T22:06:31.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.790+0000 7f74d7aa8740 1 --2- 172.21.15.1:0/2018886477 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ffa92afb40 0x55ffa92b6020 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:31.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.790+0000 7f74d7aa8740 1 --2- 172.21.15.1:0/2018886477 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ffa92ae5d0 0x55ffa92b8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:31.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.790+0000 7f74d7aa8740 1 -- 172.21.15.1:0/2018886477 >> 172.21.15.1:0/2018886477 conn(0x55ffa9277310 msgr2=0x55ffa92b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:31.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.790+0000 7f74d7aa8740 1 -- 172.21.15.1:0/2018886477 shutdown_connections 2024-09-11T22:06:31.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.790+0000 7f74d7aa8740 1 -- 172.21.15.1:0/2018886477 wait complete. 2024-09-11T22:06:31.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.790+0000 7f74d7aa8740 20 client.17643 trim_cache size 0 max 16384 2024-09-11T22:06:31.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.803+0000 7f74d7aa8740 1 Processor -- start 2024-09-11T22:06:31.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.803+0000 7f74d7aa8740 1 -- start start 2024-09-11T22:06:31.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.803+0000 7f74d7aa8740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ffa92ae5d0 0x55ffa92c4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:31.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.803+0000 7f74d7aa8740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ffa92afb40 0x55ffa92b1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:31.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.803+0000 7f74d7aa8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ffa92b9370 0x55ffa92b16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:31.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.803+0000 7f74d7aa8740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f74d01ada00 con 0x55ffa92afb40 2024-09-11T22:06:31.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.803+0000 7f74d7aa8740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f74d01af840 con 0x55ffa92ae5d0 2024-09-11T22:06:31.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.803+0000 7f74d7aa8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f74d01ae190 con 0x55ffa92b9370 2024-09-11T22:06:31.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.804+0000 7f74c7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ffa92afb40 0x55ffa92b1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:31.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.804+0000 7f74c7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ffa92afb40 0x55ffa92b1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38162/0 (socket says 172.21.15.1:38162) 2024-09-11T22:06:31.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.804+0000 7f74c7fff640 1 -- 172.21.15.1:0/896724522 learned_addr learned my addr 172.21.15.1:0/896724522 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:06:31.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.804+0000 7f74cf7fe640 1 --2- 172.21.15.1:0/896724522 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ffa92b9370 0x55ffa92b16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:31.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.804+0000 7f74c7fff640 1 -- 172.21.15.1:0/896724522 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ffa92b9370 msgr2=0x55ffa92b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:31.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.804+0000 7f74c7fff640 1 --2- 172.21.15.1:0/896724522 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ffa92b9370 0x55ffa92b16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:31.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.804+0000 7f74ceffd640 1 --2- 172.21.15.1:0/896724522 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ffa92ae5d0 0x55ffa92c4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:31.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.804+0000 7f74c7fff640 1 -- 172.21.15.1:0/896724522 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ffa92ae5d0 msgr2=0x55ffa92c4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:31.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.804+0000 7f74c7fff640 1 --2- 172.21.15.1:0/896724522 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ffa92ae5d0 0x55ffa92c4f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:31.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.804+0000 7f74c7fff640 1 -- 172.21.15.1:0/896724522 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffa92cb840 con 0x55ffa92afb40 2024-09-11T22:06:31.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.804+0000 7f74c7fff640 1 --2- 172.21.15.1:0/896724522 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ffa92afb40 0x55ffa92b1180 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f74b8010500 tx=0x7f74b8009d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dc68f69b6cb2006 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:31.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.804+0000 7f74c4ff9640 1 -- 172.21.15.1:0/896724522 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b8028040 con 0x55ffa92afb40 2024-09-11T22:06:31.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.804+0000 7f74c4ff9640 1 -- 172.21.15.1:0/896724522 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74b800fe80 con 0x55ffa92afb40 2024-09-11T22:06:31.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.804+0000 7f74c4ff9640 1 -- 172.21.15.1:0/896724522 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b802d6d0 con 0x55ffa92afb40 2024-09-11T22:06:31.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.805+0000 7f74d7aa8740 1 -- 172.21.15.1:0/896724522 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ffa92afb40 msgr2=0x55ffa92b1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:31.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.805+0000 7f74d7aa8740 1 --2- 172.21.15.1:0/896724522 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ffa92afb40 0x55ffa92b1180 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f74b8010500 tx=0x7f74b8009d10 comp rx=0 tx=0).stop 2024-09-11T22:06:31.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.805+0000 7f74d7aa8740 1 -- 172.21.15.1:0/896724522 shutdown_connections 2024-09-11T22:06:31.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.805+0000 7f74d7aa8740 1 --2- 172.21.15.1:0/896724522 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ffa92b9370 0x55ffa92b16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:31.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.805+0000 7f74d7aa8740 1 --2- 172.21.15.1:0/896724522 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ffa92afb40 0x55ffa92b1180 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:31.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.805+0000 7f74d7aa8740 1 --2- 172.21.15.1:0/896724522 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ffa92ae5d0 0x55ffa92c4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:31.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.805+0000 7f74d7aa8740 1 -- 172.21.15.1:0/896724522 >> 172.21.15.1:0/896724522 conn(0x55ffa9277310 msgr2=0x55ffa92b5110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:31.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.805+0000 7f74d7aa8740 1 -- 172.21.15.1:0/896724522 shutdown_connections 2024-09-11T22:06:31.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.805+0000 7f74d7aa8740 1 -- 172.21.15.1:0/896724522 wait complete. 2024-09-11T22:06:31.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.805+0000 7f74d7aa8740 1 -- start start 2024-09-11T22:06:31.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.805+0000 7f74d7aa8740 1 Processor -- start 2024-09-11T22:06:31.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.806+0000 7f74d7aa8740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ffa92ae5d0 0x55ffa92bced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:31.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.806+0000 7f74d7aa8740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ffa92afb40 0x55ffa92bd410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:31.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.806+0000 7f74d7aa8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ffa92b9370 0x55ffa92bd950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:31.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.806+0000 7f74d7aa8740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f74d01af0b0 con 0x55ffa92afb40 2024-09-11T22:06:31.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.806+0000 7f74d7aa8740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f74d01ae920 con 0x55ffa92ae5d0 2024-09-11T22:06:31.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.806+0000 7f74d7aa8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f74d01ab9e0 con 0x55ffa92b9370 2024-09-11T22:06:31.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.806+0000 7f74d7aa8740 10 client.0 _ll_register_callbacks cb 0x7f74b8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:06:31.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.806+0000 7f74c7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ffa92afb40 0x55ffa92bd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:31.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.806+0000 7f74c7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ffa92afb40 0x55ffa92bd410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38176/0 (socket says 172.21.15.1:38176) 2024-09-11T22:06:31.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.806+0000 7f74c7fff640 1 -- 172.21.15.1:0/1835640073 learned_addr learned my addr 172.21.15.1:0/1835640073 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:06:31.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.806+0000 7f74ceffd640 1 --2- 172.21.15.1:0/1835640073 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ffa92ae5d0 0x55ffa92bced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:31.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.806+0000 7f74cf7fe640 1 --2- 172.21.15.1:0/1835640073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ffa92b9370 0x55ffa92bd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:31.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.806+0000 7f74c7fff640 1 -- 172.21.15.1:0/1835640073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ffa92b9370 msgr2=0x55ffa92bd950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:31.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.806+0000 7f74c7fff640 1 --2- 172.21.15.1:0/1835640073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ffa92b9370 0x55ffa92bd950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:31.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.806+0000 7f74c7fff640 1 -- 172.21.15.1:0/1835640073 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ffa92ae5d0 msgr2=0x55ffa92bced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:31.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.806+0000 7f74c7fff640 1 --2- 172.21.15.1:0/1835640073 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ffa92ae5d0 0x55ffa92bced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:31.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.806+0000 7f74c7fff640 1 -- 172.21.15.1:0/1835640073 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffa9332b10 con 0x55ffa92afb40 2024-09-11T22:06:31.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.806+0000 7f74cf7fe640 1 --2- 172.21.15.1:0/1835640073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ffa92b9370 0x55ffa92bd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:06:31.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.806+0000 7f74c7fff640 1 --2- 172.21.15.1:0/1835640073 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ffa92afb40 0x55ffa92bd410 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f74b8010ce0 tx=0x7f74b800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2602ee107ce1852 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:31.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.806+0000 7f74b67fc640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:06:31.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.806+0000 7f74b67fc640 1 -- 172.21.15.1:0/1835640073 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b8028070 con 0x55ffa92afb40 2024-09-11T22:06:31.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.806+0000 7f74b67fc640 1 -- 172.21.15.1:0/1835640073 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7f74b800fbb0 con 0x55ffa92afb40 2024-09-11T22:06:31.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.806+0000 7f74d7aa8740 1 -- 172.21.15.1:0/1835640073 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ffa921bd70 con 0x55ffa92afb40 2024-09-11T22:06:31.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.807+0000 7f74b67fc640 1 -- 172.21.15.1:0/1835640073 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f74b80106f0 con 0x55ffa92afb40 2024-09-11T22:06:31.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.807+0000 7f74d7aa8740 10 client.17649 fetch_fsmap learned FSMap version 46 2024-09-11T22:06:31.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.807+0000 7f74d7aa8740 1 -- 172.21.15.1:0/1835640073 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55ffa9333df0 con 0x55ffa92afb40 2024-09-11T22:06:31.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.807+0000 7f74b67fc640 1 -- 172.21.15.1:0/1835640073 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55ffa921bd70 con 0x55ffa92afb40 2024-09-11T22:06:31.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.807+0000 7f74b67fc640 1 -- 172.21.15.1:0/1835640073 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 46) ==== 8442+0+0 (secure 0 0 0) 0x7f74b802d470 con 0x55ffa92afb40 2024-09-11T22:06:31.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.807+0000 7f74d7aa8740 10 client.17649 fetch_fsmap finished waiting for FSMap version 46 2024-09-11T22:06:31.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.807+0000 7f74d7aa8740 10 client.17649 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-11T22:06:31.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.807+0000 7f74d7aa8740 20 client.17649 populate_metadata read hostname 'smithi001' 2024-09-11T22:06:31.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.807+0000 7f74d7aa8740 1 --2- 172.21.15.1:0/1835640073 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x55ffa9334570 0x55ffa9336990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:31.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.807+0000 7f74d7aa8740 4 client.17649 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:06:31.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.807+0000 7f74d7aa8740 1 -- 172.21.15.1:0/1835640073 --> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ffa9198170 con 0x55ffa9334570 2024-09-11T22:06:31.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.807+0000 7f74ceffd640 1 --2- 172.21.15.1:0/1835640073 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x55ffa9334570 0x55ffa9336990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:31.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.808+0000 7f74ceffd640 1 --2- 172.21.15.1:0/1835640073 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x55ffa9334570 0x55ffa9336990 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8935dadab8916af2 server_cookie=eae80b17a4c3fe62 in_seq=0 out_seq=0 2024-09-11T22:06:31.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.808+0000 7f74b67fc640 10 client.17649 ms_handle_connect on v2:172.21.15.161:6836/1799318656 2024-09-11T22:06:31.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.808+0000 7f74b67fc640 1 -- 172.21.15.1:0/1835640073 <== mds.3 v2:172.21.15.161:6836/1799318656 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ffa9198170 con 0x55ffa9334570 2024-09-11T22:06:31.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.808+0000 7f74b67fc640 10 client.17649 handle_command_reply: tid=0 2024-09-11T22:06:31.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.808+0000 7f74d7aa8740 1 client.17649 shutdown 2024-09-11T22:06:31.851 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-11T22:06:31.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.808+0000 7f74d7aa8740 1 -- 172.21.15.1:0/1835640073 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ffa92afb40 msgr2=0x55ffa92bd410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:31.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.808+0000 7f74d7aa8740 1 --2- 172.21.15.1:0/1835640073 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ffa92afb40 0x55ffa92bd410 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f74b8010ce0 tx=0x7f74b800f790 comp rx=0 tx=0).stop 2024-09-11T22:06:31.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.808+0000 7f74d7aa8740 1 -- 172.21.15.1:0/1835640073 shutdown_connections 2024-09-11T22:06:31.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.808+0000 7f74d7aa8740 1 --2- 172.21.15.1:0/1835640073 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x55ffa9334570 0x55ffa9336990 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:31.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.808+0000 7f74d7aa8740 1 --2- 172.21.15.1:0/1835640073 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55ffa92b9370 0x55ffa92bd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:31.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.808+0000 7f74d7aa8740 1 --2- 172.21.15.1:0/1835640073 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ffa92afb40 0x55ffa92bd410 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:31.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.808+0000 7f74d7aa8740 1 --2- 172.21.15.1:0/1835640073 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ffa92ae5d0 0x55ffa92bced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:31.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.808+0000 7f74b67fc640 0 client.17649 ms_handle_reset on v2:172.21.15.161:6836/1799318656 2024-09-11T22:06:31.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.808+0000 7f74d7aa8740 1 -- 172.21.15.1:0/1835640073 >> 172.21.15.1:0/1835640073 conn(0x55ffa9277310 msgr2=0x55ffa92b7880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:31.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.808+0000 7f74d7aa8740 1 -- 172.21.15.1:0/1835640073 shutdown_connections 2024-09-11T22:06:31.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.808+0000 7f74d7aa8740 1 -- 172.21.15.1:0/1835640073 wait complete. 2024-09-11T22:06:31.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.808+0000 7f74d7aa8740 20 client.17649 trim_cache size 0 max 16384 2024-09-11T22:06:31.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.809+0000 7f74d56cf640 1 -- 172.21.15.1:0/1714801139 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f74bc0a2cf0 msgr2=0x7f74bc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:31.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.809+0000 7f74d56cf640 1 --2- 172.21.15.1:0/1714801139 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f74bc0a2cf0 0x7f74bc0a51b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f74c8009990 tx=0x7f74c8009aa0 comp rx=0 tx=0).stop 2024-09-11T22:06:31.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.809+0000 7f74d56cf640 1 -- 172.21.15.1:0/1714801139 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f74d0100a80 msgr2=0x7f74d0198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:31.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.809+0000 7f74d56cf640 1 --2- 172.21.15.1:0/1714801139 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f74d0100a80 0x7f74d0198470 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f74c0009230 tx=0x7f74c0004270 comp rx=0 tx=0).stop 2024-09-11T22:06:31.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.809+0000 7f74d56cf640 1 -- 172.21.15.1:0/1714801139 shutdown_connections 2024-09-11T22:06:31.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.809+0000 7f74d56cf640 1 --2- 172.21.15.1:0/1714801139 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f74bc0a2cf0 0x7f74bc0a51b0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:31.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.809+0000 7f74d56cf640 1 --2- 172.21.15.1:0/1714801139 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f74d0106a80 0x7f74d019cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:31.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.809+0000 7f74d56cf640 1 --2- 172.21.15.1:0/1714801139 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f74d0101420 0x7f74d01989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:31.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.809+0000 7f74d56cf640 1 --2- 172.21.15.1:0/1714801139 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f74d0100a80 0x7f74d0198470 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:31.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.809+0000 7f74d56cf640 1 -- 172.21.15.1:0/1714801139 >> 172.21.15.1:0/1714801139 conn(0x7f74d00fc7a0 msgr2=0x7f74d01048d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:31.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.809+0000 7f74d56cf640 1 -- 172.21.15.1:0/1714801139 shutdown_connections 2024-09-11T22:06:31.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:31.809+0000 7f74d56cf640 1 -- 172.21.15.1:0/1714801139 wait complete. 2024-09-11T22:06:32.530 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T22:06:32.786 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:06:33.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:06:33 smithi001 ceph-mon[30935]: pgmap v290: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:06:33.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.298+0000 7fc15515b640 1 -- 172.21.15.1:0/351654891 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc150101420 msgr2=0x7fc15010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:33.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.298+0000 7fc15515b640 1 --2- 172.21.15.1:0/351654891 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc150101420 0x7fc15010d950 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fc148009cd0 tx=0x7fc14802a8e0 comp rx=0 tx=0).stop 2024-09-11T22:06:33.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.298+0000 7fc15515b640 1 -- 172.21.15.1:0/351654891 shutdown_connections 2024-09-11T22:06:33.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.298+0000 7fc15515b640 1 --2- 172.21.15.1:0/351654891 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc150101420 0x7fc15010d950 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:33.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.298+0000 7fc15515b640 1 --2- 172.21.15.1:0/351654891 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc150100a80 0x7fc150100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:33.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.298+0000 7fc15515b640 1 --2- 172.21.15.1:0/351654891 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc150106a80 0x7fc150106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:33.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.298+0000 7fc15515b640 1 -- 172.21.15.1:0/351654891 >> 172.21.15.1:0/351654891 conn(0x7fc1500fc7a0 msgr2=0x7fc1500febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:33.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.298+0000 7fc15515b640 1 -- 172.21.15.1:0/351654891 shutdown_connections 2024-09-11T22:06:33.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.298+0000 7fc15515b640 1 -- 172.21.15.1:0/351654891 wait complete. 2024-09-11T22:06:33.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.299+0000 7fc15515b640 1 Processor -- start 2024-09-11T22:06:33.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.299+0000 7fc15515b640 1 -- start start 2024-09-11T22:06:33.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.300+0000 7fc15515b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc150100a80 0x7fc1501972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:33.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.300+0000 7fc15515b640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc150101420 0x7fc150197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:33.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.300+0000 7fc15515b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc150106a80 0x7fc15019bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:33.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.300+0000 7fc15515b640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc150110e80 con 0x7fc150106a80 2024-09-11T22:06:33.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.300+0000 7fc15515b640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc150110d00 con 0x7fc150101420 2024-09-11T22:06:33.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.300+0000 7fc15515b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc150111000 con 0x7fc150100a80 2024-09-11T22:06:33.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.300+0000 7fc14f59e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc150106a80 0x7fc15019bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:33.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.300+0000 7fc14f59e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc150106a80 0x7fc15019bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38192/0 (socket says 172.21.15.1:38192) 2024-09-11T22:06:33.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.300+0000 7fc14f59e640 1 -- 172.21.15.1:0/1464331222 learned_addr learned my addr 172.21.15.1:0/1464331222 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:06:33.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.300+0000 7fc14ed9d640 1 --2- 172.21.15.1:0/1464331222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc150100a80 0x7fc1501972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:33.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.300+0000 7fc147fff640 1 --2- 172.21.15.1:0/1464331222 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc150101420 0x7fc150197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:33.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.300+0000 7fc14f59e640 1 -- 172.21.15.1:0/1464331222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc150100a80 msgr2=0x7fc1501972e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:33.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.300+0000 7fc14f59e640 1 --2- 172.21.15.1:0/1464331222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc150100a80 0x7fc1501972e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:33.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.300+0000 7fc14f59e640 1 -- 172.21.15.1:0/1464331222 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc150101420 msgr2=0x7fc150197820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:33.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.300+0000 7fc14f59e640 1 --2- 172.21.15.1:0/1464331222 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc150101420 0x7fc150197820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:33.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.300+0000 7fc14f59e640 1 -- 172.21.15.1:0/1464331222 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc15019c290 con 0x7fc150106a80 2024-09-11T22:06:33.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.300+0000 7fc14ed9d640 1 --2- 172.21.15.1:0/1464331222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc150100a80 0x7fc1501972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:06:33.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.300+0000 7fc14f59e640 1 --2- 172.21.15.1:0/1464331222 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc150106a80 0x7fc15019bbb0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fc148005cc0 tx=0x7fc148003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:33.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.300+0000 7fc14cd99640 1 -- 172.21.15.1:0/1464331222 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc148004040 con 0x7fc150106a80 2024-09-11T22:06:33.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.301+0000 7fc15515b640 1 -- 172.21.15.1:0/1464331222 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc15019c520 con 0x7fc150106a80 2024-09-11T22:06:33.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.301+0000 7fc14cd99640 1 -- 172.21.15.1:0/1464331222 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1480046c0 con 0x7fc150106a80 2024-09-11T22:06:33.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.301+0000 7fc14cd99640 1 -- 172.21.15.1:0/1464331222 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc148046480 con 0x7fc150106a80 2024-09-11T22:06:33.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.301+0000 7fc15515b640 1 -- 172.21.15.1:0/1464331222 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1501a3e60 con 0x7fc150106a80 2024-09-11T22:06:33.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.303+0000 7fc14cd99640 1 -- 172.21.15.1:0/1464331222 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7fc1480305a0 con 0x7fc150106a80 2024-09-11T22:06:33.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.303+0000 7fc157534740 1 Processor -- start 2024-09-11T22:06:33.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.303+0000 7fc157534740 1 -- start start 2024-09-11T22:06:33.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.304+0000 7fc157534740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af40535b40 0x55af40534090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:33.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.304+0000 7fc157534740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55af405345d0 0x55af40534a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:33.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.304+0000 7fc157534740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af4053f370 0x55af40541760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:33.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.304+0000 7fc157534740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55af4049a2f0 con 0x55af4053f370 2024-09-11T22:06:33.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.304+0000 7fc157534740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55af40441cf0 con 0x55af40535b40 2024-09-11T22:06:33.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.304+0000 7fc157534740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55af404962b0 con 0x55af405345d0 2024-09-11T22:06:33.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.304+0000 7fc14cd99640 1 --2- 172.21.15.1:0/1464331222 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fc13c0a2cf0 0x7fc13c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:33.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.304+0000 7fc147fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af4053f370 0x55af40541760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:33.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.304+0000 7fc147fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af4053f370 0x55af40541760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38200/0 (socket says 172.21.15.1:38200) 2024-09-11T22:06:33.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.304+0000 7fc147fff640 1 -- 172.21.15.1:0/2313825191 learned_addr learned my addr 172.21.15.1:0/2313825191 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:06:33.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.304+0000 7fc14f59e640 1 --2- 172.21.15.1:0/1464331222 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fc13c0a2cf0 0x7fc13c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:33.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.304+0000 7fc14ed9d640 1 --2- 172.21.15.1:0/2313825191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55af405345d0 0x55af40534a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:33.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.304+0000 7fc147fff640 1 --2- 172.21.15.1:0/2313825191 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af40535b40 0x55af40534090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:33.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.304+0000 7fc14cd99640 1 -- 172.21.15.1:0/1464331222 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7fc14802e030 con 0x7fc150106a80 2024-09-11T22:06:33.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.304+0000 7fc147fff640 1 -- 172.21.15.1:0/2313825191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55af405345d0 msgr2=0x55af40534a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:33.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.304+0000 7fc147fff640 1 --2- 172.21.15.1:0/2313825191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55af405345d0 0x55af40534a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:33.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.304+0000 7fc147fff640 1 -- 172.21.15.1:0/2313825191 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af40535b40 msgr2=0x55af40534090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:33.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.304+0000 7fc147fff640 1 --2- 172.21.15.1:0/2313825191 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af40535b40 0x55af40534090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:33.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.304+0000 7fc147fff640 1 -- 172.21.15.1:0/2313825191 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af404263a0 con 0x55af4053f370 2024-09-11T22:06:33.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.304+0000 7fc147fff640 1 --2- 172.21.15.1:0/2313825191 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af4053f370 0x55af40541760 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fc1380149c0 tx=0x7fc13800c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=224a15ae5282f4c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:33.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.305+0000 7fc14f59e640 1 --2- 172.21.15.1:0/1464331222 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fc13c0a2cf0 0x7fc13c0a51b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc1480bd210 tx=0x7fc1480dd040 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:33.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.305+0000 7fc1457fa640 1 -- 172.21.15.1:0/2313825191 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1380142d0 con 0x55af4053f370 2024-09-11T22:06:33.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.305+0000 7fc1457fa640 1 -- 172.21.15.1:0/2313825191 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc13800fc40 con 0x55af4053f370 2024-09-11T22:06:33.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.305+0000 7fc1457fa640 1 -- 172.21.15.1:0/2313825191 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc138012a90 con 0x55af4053f370 2024-09-11T22:06:33.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.305+0000 7fc157534740 1 -- 172.21.15.1:0/2313825191 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af4053f370 msgr2=0x55af40541760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:33.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.305+0000 7fc157534740 1 --2- 172.21.15.1:0/2313825191 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af4053f370 0x55af40541760 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fc1380149c0 tx=0x7fc13800c4d0 comp rx=0 tx=0).stop 2024-09-11T22:06:33.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.306+0000 7fc157534740 1 -- 172.21.15.1:0/2313825191 shutdown_connections 2024-09-11T22:06:33.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.306+0000 7fc157534740 1 --2- 172.21.15.1:0/2313825191 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af4053f370 0x55af40541760 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:33.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.306+0000 7fc157534740 1 --2- 172.21.15.1:0/2313825191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55af405345d0 0x55af40534a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:33.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.306+0000 7fc157534740 1 --2- 172.21.15.1:0/2313825191 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af40535b40 0x55af40534090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:33.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.306+0000 7fc157534740 1 -- 172.21.15.1:0/2313825191 >> 172.21.15.1:0/2313825191 conn(0x55af404fd310 msgr2=0x55af40530c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:33.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.306+0000 7fc157534740 1 -- 172.21.15.1:0/2313825191 shutdown_connections 2024-09-11T22:06:33.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.306+0000 7fc157534740 1 -- 172.21.15.1:0/2313825191 wait complete. 2024-09-11T22:06:33.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.306+0000 7fc157534740 1 -- start start 2024-09-11T22:06:33.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.306+0000 7fc157534740 1 Processor -- start 2024-09-11T22:06:33.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.306+0000 7fc157534740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af405345d0 0x55af4053ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:33.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.306+0000 7fc157534740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af40535b40 0x55af4053c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:33.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.306+0000 7fc157534740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55af4053f370 0x55af4053c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:33.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.306+0000 7fc157534740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55af40542070 con 0x55af405345d0 2024-09-11T22:06:33.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.306+0000 7fc157534740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55af40455060 con 0x55af40535b40 2024-09-11T22:06:33.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.306+0000 7fc157534740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55af405421f0 con 0x55af4053f370 2024-09-11T22:06:33.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.306+0000 7fc157534740 10 client.0 _ll_register_callbacks cb 0x55af401a88c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:06:33.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.306+0000 7fc147fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af405345d0 0x55af4053ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:33.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.306+0000 7fc147fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af405345d0 0x55af4053ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38202/0 (socket says 172.21.15.1:38202) 2024-09-11T22:06:33.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.306+0000 7fc147fff640 1 -- 172.21.15.1:0/1367691002 learned_addr learned my addr 172.21.15.1:0/1367691002 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:06:33.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.307+0000 7fc14ed9d640 1 --2- 172.21.15.1:0/1367691002 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af40535b40 0x55af4053c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:33.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.307+0000 7fc147fff640 1 --2- 172.21.15.1:0/1367691002 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55af4053f370 0x55af4053c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:33.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.307+0000 7fc147fff640 1 -- 172.21.15.1:0/1367691002 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55af4053f370 msgr2=0x55af4053c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:33.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.307+0000 7fc147fff640 1 --2- 172.21.15.1:0/1367691002 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55af4053f370 0x55af4053c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:33.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.307+0000 7fc147fff640 1 -- 172.21.15.1:0/1367691002 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af40535b40 msgr2=0x55af4053c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:33.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.307+0000 7fc147fff640 1 --2- 172.21.15.1:0/1367691002 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af40535b40 0x55af4053c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:33.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.307+0000 7fc147fff640 1 -- 172.21.15.1:0/1367691002 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af4020b3c0 con 0x55af405345d0 2024-09-11T22:06:33.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.307+0000 7fc147fff640 1 --2- 172.21.15.1:0/1367691002 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af405345d0 0x55af4053ed70 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fc13800c840 tx=0x7fc138012730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4df401886552a184 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:33.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.307+0000 7fc1367fc640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:06:33.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.307+0000 7fc1367fc640 1 -- 172.21.15.1:0/1367691002 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1380142d0 con 0x55af405345d0 2024-09-11T22:06:33.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.307+0000 7fc1367fc640 1 -- 172.21.15.1:0/1367691002 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7fc13800f760 con 0x55af405345d0 2024-09-11T22:06:33.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.307+0000 7fc157534740 1 -- 172.21.15.1:0/1367691002 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc13800fc40 con 0x55af405345d0 2024-09-11T22:06:33.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.307+0000 7fc1367fc640 1 -- 172.21.15.1:0/1367691002 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fc138033050 con 0x55af405345d0 2024-09-11T22:06:33.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.307+0000 7fc157534740 10 client.17661 fetch_fsmap learned FSMap version 46 2024-09-11T22:06:33.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.307+0000 7fc157534740 1 -- 172.21.15.1:0/1367691002 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55af4054db10 con 0x55af405345d0 2024-09-11T22:06:33.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.307+0000 7fc1367fc640 1 -- 172.21.15.1:0/1367691002 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc13800fc40 con 0x55af405345d0 2024-09-11T22:06:33.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.308+0000 7fc1367fc640 1 -- 172.21.15.1:0/1367691002 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 46) ==== 8442+0+0 (secure 0 0 0) 0x7fc138012c10 con 0x55af405345d0 2024-09-11T22:06:33.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.308+0000 7fc157534740 10 client.17661 fetch_fsmap finished waiting for FSMap version 46 2024-09-11T22:06:33.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.308+0000 7fc157534740 10 client.17661 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-11T22:06:33.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.308+0000 7fc157534740 20 client.17661 populate_metadata read hostname 'smithi001' 2024-09-11T22:06:33.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.308+0000 7fc157534740 1 --2- 172.21.15.1:0/1367691002 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x55af4054ee60 0x55af40551250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:33.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.308+0000 7fc157534740 4 client.17661 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:06:33.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.308+0000 7fc157534740 1 -- 172.21.15.1:0/1367691002 --> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af404a7860 con 0x55af4054ee60 2024-09-11T22:06:33.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.308+0000 7fc14ed9d640 1 --2- 172.21.15.1:0/1367691002 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x55af4054ee60 0x55af40551250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:33.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.308+0000 7fc1367fc640 10 client.17661 ms_handle_connect on v2:172.21.15.53:6836/3457131830 2024-09-11T22:06:33.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.308+0000 7fc14ed9d640 1 --2- 172.21.15.1:0/1367691002 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x55af4054ee60 0x55af40551250 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=197482b5fa920b6 server_cookie=554873c9ba333478 in_seq=0 out_seq=0 2024-09-11T22:06:33.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.309+0000 7fc1367fc640 1 -- 172.21.15.1:0/1367691002 <== mds.2 v2:172.21.15.53:6836/3457131830 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55af404a7860 con 0x55af4054ee60 2024-09-11T22:06:33.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.309+0000 7fc1367fc640 10 client.17661 handle_command_reply: tid=0 2024-09-11T22:06:33.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.309+0000 7fc157534740 1 client.17661 shutdown 2024-09-11T22:06:33.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.309+0000 7fc157534740 1 -- 172.21.15.1:0/1367691002 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af405345d0 msgr2=0x55af4053ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:33.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.309+0000 7fc157534740 1 --2- 172.21.15.1:0/1367691002 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af405345d0 0x55af4053ed70 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fc13800c840 tx=0x7fc138012730 comp rx=0 tx=0).stop 2024-09-11T22:06:33.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.310+0000 7fc157534740 1 -- 172.21.15.1:0/1367691002 shutdown_connections 2024-09-11T22:06:33.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.310+0000 7fc157534740 1 --2- 172.21.15.1:0/1367691002 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x55af4054ee60 0x55af40551250 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:33.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.310+0000 7fc157534740 1 --2- 172.21.15.1:0/1367691002 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55af4053f370 0x55af4053c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:33.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.310+0000 7fc157534740 1 --2- 172.21.15.1:0/1367691002 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af40535b40 0x55af4053c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:33.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.310+0000 7fc1367fc640 0 client.17661 ms_handle_reset on v2:172.21.15.53:6836/3457131830 2024-09-11T22:06:33.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.310+0000 7fc157534740 1 --2- 172.21.15.1:0/1367691002 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af405345d0 0x55af4053ed70 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:33.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.310+0000 7fc157534740 1 -- 172.21.15.1:0/1367691002 >> 172.21.15.1:0/1367691002 conn(0x55af404fd310 msgr2=0x55af40536960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:33.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.310+0000 7fc157534740 1 -- 172.21.15.1:0/1367691002 shutdown_connections 2024-09-11T22:06:33.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.310+0000 7fc157534740 1 -- 172.21.15.1:0/1367691002 wait complete. 2024-09-11T22:06:33.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.310+0000 7fc157534740 20 client.17661 trim_cache size 0 max 16384 2024-09-11T22:06:33.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.323+0000 7fc157534740 1 Processor -- start 2024-09-11T22:06:33.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.323+0000 7fc157534740 1 -- start start 2024-09-11T22:06:33.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.323+0000 7fc157534740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55af405345d0 0x55af40530df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:33.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.323+0000 7fc157534740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af40535b40 0x55af4054b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:33.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.323+0000 7fc157534740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af4053f370 0x55af40556910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:33.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.323+0000 7fc157534740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc1501ab730 con 0x55af4053f370 2024-09-11T22:06:33.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.323+0000 7fc157534740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc1501ad570 con 0x55af40535b40 2024-09-11T22:06:33.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.323+0000 7fc157534740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc1501abec0 con 0x55af405345d0 2024-09-11T22:06:33.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.323+0000 7fc147fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af4053f370 0x55af40556910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:33.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.323+0000 7fc147fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af4053f370 0x55af40556910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38212/0 (socket says 172.21.15.1:38212) 2024-09-11T22:06:33.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.323+0000 7fc147fff640 1 -- 172.21.15.1:0/3689907748 learned_addr learned my addr 172.21.15.1:0/3689907748 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:06:33.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.323+0000 7fc147fff640 1 --2- 172.21.15.1:0/3689907748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55af405345d0 0x55af40530df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:33.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.323+0000 7fc14ed9d640 1 --2- 172.21.15.1:0/3689907748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af40535b40 0x55af4054b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:33.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.323+0000 7fc147fff640 1 -- 172.21.15.1:0/3689907748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55af405345d0 msgr2=0x55af40530df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:33.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.323+0000 7fc147fff640 1 --2- 172.21.15.1:0/3689907748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55af405345d0 0x55af40530df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:33.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.323+0000 7fc147fff640 1 -- 172.21.15.1:0/3689907748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af40535b40 msgr2=0x55af4054b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:33.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.323+0000 7fc147fff640 1 --2- 172.21.15.1:0/3689907748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af40535b40 0x55af4054b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:33.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.323+0000 7fc147fff640 1 -- 172.21.15.1:0/3689907748 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af405516b0 con 0x55af4053f370 2024-09-11T22:06:33.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.324+0000 7fc147fff640 1 --2- 172.21.15.1:0/3689907748 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af4053f370 0x55af40556910 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fc138003710 tx=0x7fc138005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3041b23691301c0d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:33.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.324+0000 7fc144ff9640 1 -- 172.21.15.1:0/3689907748 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13800cd00 con 0x55af4053f370 2024-09-11T22:06:33.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.324+0000 7fc144ff9640 1 -- 172.21.15.1:0/3689907748 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc138005550 con 0x55af4053f370 2024-09-11T22:06:33.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.324+0000 7fc144ff9640 1 -- 172.21.15.1:0/3689907748 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1380127a0 con 0x55af4053f370 2024-09-11T22:06:33.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.324+0000 7fc157534740 1 -- 172.21.15.1:0/3689907748 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af4053f370 msgr2=0x55af40556910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:33.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.324+0000 7fc157534740 1 --2- 172.21.15.1:0/3689907748 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af4053f370 0x55af40556910 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fc138003710 tx=0x7fc138005970 comp rx=0 tx=0).stop 2024-09-11T22:06:33.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.325+0000 7fc157534740 1 -- 172.21.15.1:0/3689907748 shutdown_connections 2024-09-11T22:06:33.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.325+0000 7fc157534740 1 --2- 172.21.15.1:0/3689907748 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af4053f370 0x55af40556910 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:33.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.325+0000 7fc157534740 1 --2- 172.21.15.1:0/3689907748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af40535b40 0x55af4054b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:33.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.325+0000 7fc157534740 1 --2- 172.21.15.1:0/3689907748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55af405345d0 0x55af40530df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:33.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.325+0000 7fc157534740 1 -- 172.21.15.1:0/3689907748 >> 172.21.15.1:0/3689907748 conn(0x55af404fd310 msgr2=0x55af405306e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:33.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.325+0000 7fc157534740 1 -- 172.21.15.1:0/3689907748 shutdown_connections 2024-09-11T22:06:33.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.325+0000 7fc157534740 1 -- 172.21.15.1:0/3689907748 wait complete. 2024-09-11T22:06:33.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.325+0000 7fc157534740 1 -- start start 2024-09-11T22:06:33.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.325+0000 7fc157534740 1 Processor -- start 2024-09-11T22:06:33.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.325+0000 7fc157534740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55af405345d0 0x55af40539090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:33.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.325+0000 7fc157534740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af40535b40 0x55af405395d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:33.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.325+0000 7fc157534740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af4053f370 0x55af40539b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:33.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.325+0000 7fc157534740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc1501acde0 con 0x55af4053f370 2024-09-11T22:06:33.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.325+0000 7fc157534740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc1501ac650 con 0x55af40535b40 2024-09-11T22:06:33.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.325+0000 7fc157534740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc1501a9710 con 0x55af405345d0 2024-09-11T22:06:33.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.325+0000 7fc157534740 10 client.0 _ll_register_callbacks cb 0x7fc13800cf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:06:33.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.325+0000 7fc147fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af4053f370 0x55af40539b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:33.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.325+0000 7fc147fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af4053f370 0x55af40539b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38226/0 (socket says 172.21.15.1:38226) 2024-09-11T22:06:33.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.326+0000 7fc147fff640 1 -- 172.21.15.1:0/3238946983 learned_addr learned my addr 172.21.15.1:0/3238946983 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:06:33.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.326+0000 7fc147fff640 1 --2- 172.21.15.1:0/3238946983 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55af405345d0 0x55af40539090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:33.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.326+0000 7fc14ed9d640 1 --2- 172.21.15.1:0/3238946983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af40535b40 0x55af405395d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:33.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.326+0000 7fc147fff640 1 -- 172.21.15.1:0/3238946983 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55af405345d0 msgr2=0x55af40539090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:33.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.326+0000 7fc147fff640 1 --2- 172.21.15.1:0/3238946983 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55af405345d0 0x55af40539090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:33.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.326+0000 7fc147fff640 1 -- 172.21.15.1:0/3238946983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af40535b40 msgr2=0x55af405395d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:33.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.326+0000 7fc147fff640 1 --2- 172.21.15.1:0/3238946983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af40535b40 0x55af405395d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:33.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.326+0000 7fc147fff640 1 -- 172.21.15.1:0/3238946983 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af40537120 con 0x55af4053f370 2024-09-11T22:06:33.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.326+0000 7fc147fff640 1 --2- 172.21.15.1:0/3238946983 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af4053f370 0x55af40539b10 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fc1380144e0 tx=0x7fc1380143e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1602748d9e8272d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:33.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.326+0000 7fc1367fc640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:06:33.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.326+0000 7fc1367fc640 1 -- 172.21.15.1:0/3238946983 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc138041040 con 0x55af4053f370 2024-09-11T22:06:33.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.326+0000 7fc1367fc640 1 -- 172.21.15.1:0/3238946983 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7fc13800f9a0 con 0x55af4053f370 2024-09-11T22:06:33.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.326+0000 7fc157534740 1 -- 172.21.15.1:0/3238946983 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc138005550 con 0x55af4053f370 2024-09-11T22:06:33.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.326+0000 7fc1367fc640 1 -- 172.21.15.1:0/3238946983 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fc138012710 con 0x55af4053f370 2024-09-11T22:06:33.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.326+0000 7fc157534740 10 client.17667 fetch_fsmap learned FSMap version 46 2024-09-11T22:06:33.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.326+0000 7fc157534740 1 -- 172.21.15.1:0/3238946983 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55af405b9a40 con 0x55af4053f370 2024-09-11T22:06:33.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.327+0000 7fc1367fc640 1 -- 172.21.15.1:0/3238946983 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc138005550 con 0x55af4053f370 2024-09-11T22:06:33.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.327+0000 7fc1367fc640 1 -- 172.21.15.1:0/3238946983 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 46) ==== 8442+0+0 (secure 0 0 0) 0x7fc138047070 con 0x55af4053f370 2024-09-11T22:06:33.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.327+0000 7fc157534740 10 client.17667 fetch_fsmap finished waiting for FSMap version 46 2024-09-11T22:06:33.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.327+0000 7fc157534740 10 client.17667 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-11T22:06:33.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.327+0000 7fc157534740 20 client.17667 populate_metadata read hostname 'smithi001' 2024-09-11T22:06:33.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.327+0000 7fc157534740 1 --2- 172.21.15.1:0/3238946983 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x55af405ba110 0x55af405bc530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:33.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.327+0000 7fc157534740 4 client.17667 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:06:33.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.327+0000 7fc157534740 1 -- 172.21.15.1:0/3238946983 --> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55af404a1d70 con 0x55af405ba110 2024-09-11T22:06:33.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.327+0000 7fc14ed9d640 1 --2- 172.21.15.1:0/3238946983 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x55af405ba110 0x55af405bc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:33.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.327+0000 7fc14ed9d640 1 --2- 172.21.15.1:0/3238946983 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x55af405ba110 0x55af405bc530 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6605a56c6149c61c server_cookie=3f7bef8345644aee in_seq=0 out_seq=0 2024-09-11T22:06:33.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.327+0000 7fc1367fc640 10 client.17667 ms_handle_connect on v2:172.21.15.53:6836/3457131830 2024-09-11T22:06:33.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.328+0000 7fc1367fc640 1 -- 172.21.15.1:0/3238946983 <== mds.2 v2:172.21.15.53:6836/3457131830 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55af404a1d70 con 0x55af405ba110 2024-09-11T22:06:33.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.328+0000 7fc1367fc640 10 client.17667 handle_command_reply: tid=0 2024-09-11T22:06:33.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.328+0000 7fc157534740 1 client.17667 shutdown 2024-09-11T22:06:33.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.328+0000 7fc157534740 1 -- 172.21.15.1:0/3238946983 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af4053f370 msgr2=0x55af40539b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:33.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.328+0000 7fc157534740 1 --2- 172.21.15.1:0/3238946983 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af4053f370 0x55af40539b10 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fc1380144e0 tx=0x7fc1380143e0 comp rx=0 tx=0).stop 2024-09-11T22:06:33.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.328+0000 7fc157534740 1 -- 172.21.15.1:0/3238946983 shutdown_connections 2024-09-11T22:06:33.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.328+0000 7fc157534740 1 --2- 172.21.15.1:0/3238946983 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x55af405ba110 0x55af405bc530 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:33.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.328+0000 7fc157534740 1 --2- 172.21.15.1:0/3238946983 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af4053f370 0x55af40539b10 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:33.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.328+0000 7fc157534740 1 --2- 172.21.15.1:0/3238946983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55af40535b40 0x55af405395d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:33.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.328+0000 7fc157534740 1 --2- 172.21.15.1:0/3238946983 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55af405345d0 0x55af40539090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:33.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.328+0000 7fc157534740 1 -- 172.21.15.1:0/3238946983 >> 172.21.15.1:0/3238946983 conn(0x55af404fd310 msgr2=0x55af4053d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:33.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.328+0000 7fc1367fc640 0 client.17667 ms_handle_reset on v2:172.21.15.53:6836/3457131830 2024-09-11T22:06:33.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.328+0000 7fc157534740 1 -- 172.21.15.1:0/3238946983 shutdown_connections 2024-09-11T22:06:33.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.328+0000 7fc157534740 1 -- 172.21.15.1:0/3238946983 wait complete. 2024-09-11T22:06:33.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.328+0000 7fc157534740 20 client.17667 trim_cache size 0 max 16384 2024-09-11T22:06:33.359 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-11T22:06:33.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.329+0000 7fc15515b640 1 -- 172.21.15.1:0/1464331222 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fc13c0a2cf0 msgr2=0x7fc13c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:33.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.329+0000 7fc15515b640 1 --2- 172.21.15.1:0/1464331222 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fc13c0a2cf0 0x7fc13c0a51b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc1480bd210 tx=0x7fc1480dd040 comp rx=0 tx=0).stop 2024-09-11T22:06:33.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.329+0000 7fc15515b640 1 -- 172.21.15.1:0/1464331222 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc150106a80 msgr2=0x7fc15019bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:33.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.329+0000 7fc15515b640 1 --2- 172.21.15.1:0/1464331222 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc150106a80 0x7fc15019bbb0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fc148005cc0 tx=0x7fc148003970 comp rx=0 tx=0).stop 2024-09-11T22:06:33.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.329+0000 7fc15515b640 1 -- 172.21.15.1:0/1464331222 shutdown_connections 2024-09-11T22:06:33.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.329+0000 7fc15515b640 1 --2- 172.21.15.1:0/1464331222 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fc13c0a2cf0 0x7fc13c0a51b0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:33.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.329+0000 7fc15515b640 1 --2- 172.21.15.1:0/1464331222 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc150106a80 0x7fc15019bbb0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:33.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.329+0000 7fc15515b640 1 --2- 172.21.15.1:0/1464331222 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc150101420 0x7fc150197820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:33.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.329+0000 7fc15515b640 1 --2- 172.21.15.1:0/1464331222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc150100a80 0x7fc1501972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:33.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.329+0000 7fc15515b640 1 -- 172.21.15.1:0/1464331222 >> 172.21.15.1:0/1464331222 conn(0x7fc1500fc7a0 msgr2=0x7fc1500fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:33.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.329+0000 7fc15515b640 1 -- 172.21.15.1:0/1464331222 shutdown_connections 2024-09-11T22:06:33.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:33.329+0000 7fc15515b640 1 -- 172.21.15.1:0/1464331222 wait complete. 2024-09-11T22:06:33.362 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:06:33 smithi161 ceph-mon[35615]: pgmap v290: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:06:33.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:06:33 smithi053 ceph-mon[34908]: pgmap v290: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:06:35.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:06:35 smithi161 ceph-mon[35615]: pgmap v291: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:06:35.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:06:35 smithi001 ceph-mon[30935]: pgmap v291: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:06:35.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:06:35 smithi053 ceph-mon[34908]: pgmap v291: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:06:36.831 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T22:06:37.093 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:06:37.336 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:06:37 smithi001 ceph-mon[30935]: pgmap v292: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 14 op/s 2024-09-11T22:06:37.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:06:37 smithi161 ceph-mon[35615]: pgmap v292: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 14 op/s 2024-09-11T22:06:37.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:06:37 smithi053 ceph-mon[34908]: pgmap v292: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 14 op/s 2024-09-11T22:06:37.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.568+0000 7eff24cd7640 1 -- 172.21.15.1:0/3410702846 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff201036e0 msgr2=0x7eff20107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:37.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.568+0000 7eff24cd7640 1 --2- 172.21.15.1:0/3410702846 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff201036e0 0x7eff20107b90 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7eff14009930 tx=0x7eff1402a420 comp rx=0 tx=0).stop 2024-09-11T22:06:37.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.568+0000 7eff24cd7640 1 -- 172.21.15.1:0/3410702846 shutdown_connections 2024-09-11T22:06:37.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.568+0000 7eff24cd7640 1 --2- 172.21.15.1:0/3410702846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff201082a0 0x7eff2010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:37.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.568+0000 7eff24cd7640 1 --2- 172.21.15.1:0/3410702846 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff201036e0 0x7eff20107b90 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:37.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.568+0000 7eff24cd7640 1 --2- 172.21.15.1:0/3410702846 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff20102d30 0x7eff20103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:37.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.568+0000 7eff24cd7640 1 -- 172.21.15.1:0/3410702846 >> 172.21.15.1:0/3410702846 conn(0x7eff200fc7d0 msgr2=0x7eff200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:37.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.568+0000 7eff24cd7640 1 -- 172.21.15.1:0/3410702846 shutdown_connections 2024-09-11T22:06:37.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.568+0000 7eff24cd7640 1 -- 172.21.15.1:0/3410702846 wait complete. 2024-09-11T22:06:37.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.569+0000 7eff24cd7640 1 Processor -- start 2024-09-11T22:06:37.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.569+0000 7eff24cd7640 1 -- start start 2024-09-11T22:06:37.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.569+0000 7eff24cd7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff20102d30 0x7eff20198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:37.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.569+0000 7eff24cd7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff201036e0 0x7eff20198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:37.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.569+0000 7eff24cd7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff201082a0 0x7eff2019cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:37.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.569+0000 7eff24cd7640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7eff201120a0 con 0x7eff20102d30 2024-09-11T22:06:37.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.569+0000 7eff24cd7640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7eff20111f20 con 0x7eff201036e0 2024-09-11T22:06:37.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.569+0000 7eff24cd7640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7eff20112220 con 0x7eff201082a0 2024-09-11T22:06:37.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.569+0000 7eff1e59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff20102d30 0x7eff20198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:37.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.569+0000 7eff1e59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff20102d30 0x7eff20198520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41878/0 (socket says 172.21.15.1:41878) 2024-09-11T22:06:37.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.569+0000 7eff1e59c640 1 -- 172.21.15.1:0/3493128984 learned_addr learned my addr 172.21.15.1:0/3493128984 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:06:37.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.569+0000 7eff1ed9d640 1 --2- 172.21.15.1:0/3493128984 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff201082a0 0x7eff2019cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:37.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.569+0000 7eff1dd9b640 1 --2- 172.21.15.1:0/3493128984 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff201036e0 0x7eff20198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:37.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.570+0000 7eff1e59c640 1 -- 172.21.15.1:0/3493128984 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff201082a0 msgr2=0x7eff2019cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:37.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.570+0000 7eff1e59c640 1 --2- 172.21.15.1:0/3493128984 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff201082a0 0x7eff2019cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:37.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.570+0000 7eff1e59c640 1 -- 172.21.15.1:0/3493128984 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff201036e0 msgr2=0x7eff20198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:37.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.570+0000 7eff1e59c640 1 --2- 172.21.15.1:0/3493128984 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff201036e0 0x7eff20198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:37.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.570+0000 7eff1e59c640 1 -- 172.21.15.1:0/3493128984 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff2019d4d0 con 0x7eff20102d30 2024-09-11T22:06:37.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.570+0000 7eff1ed9d640 1 --2- 172.21.15.1:0/3493128984 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff201082a0 0x7eff2019cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:06:37.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.570+0000 7eff1dd9b640 1 --2- 172.21.15.1:0/3493128984 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff201036e0 0x7eff20198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:06:37.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.570+0000 7eff1e59c640 1 --2- 172.21.15.1:0/3493128984 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff20102d30 0x7eff20198520 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7eff1000cce0 tx=0x7eff1000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:37.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.570+0000 7eff0f7fe640 1 -- 172.21.15.1:0/3493128984 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff1000f650 con 0x7eff20102d30 2024-09-11T22:06:37.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.570+0000 7eff0f7fe640 1 -- 172.21.15.1:0/3493128984 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff1000fcd0 con 0x7eff20102d30 2024-09-11T22:06:37.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.570+0000 7eff24cd7640 1 -- 172.21.15.1:0/3493128984 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff2019d7c0 con 0x7eff20102d30 2024-09-11T22:06:37.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.570+0000 7eff0f7fe640 1 -- 172.21.15.1:0/3493128984 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff10018c70 con 0x7eff20102d30 2024-09-11T22:06:37.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.570+0000 7eff24cd7640 1 -- 172.21.15.1:0/3493128984 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff201a50a0 con 0x7eff20102d30 2024-09-11T22:06:37.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.572+0000 7eff0f7fe640 1 -- 172.21.15.1:0/3493128984 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7eff1000f7f0 con 0x7eff20102d30 2024-09-11T22:06:37.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.573+0000 7eff270b0740 1 Processor -- start 2024-09-11T22:06:37.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.573+0000 7eff270b0740 1 -- start start 2024-09-11T22:06:37.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.573+0000 7eff270b0740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d7e802fb40 0x55d7e802e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:37.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.573+0000 7eff270b0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7e802e5d0 0x55d7e802ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:37.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.573+0000 7eff270b0740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7e8039370 0x55d7e803b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:37.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.573+0000 7eff270b0740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55d7e7f942f0 con 0x55d7e8039370 2024-09-11T22:06:37.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.573+0000 7eff270b0740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55d7e7f3bcf0 con 0x55d7e802fb40 2024-09-11T22:06:37.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.573+0000 7eff270b0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d7e7f902b0 con 0x55d7e802e5d0 2024-09-11T22:06:37.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.573+0000 7eff0f7fe640 1 --2- 172.21.15.1:0/3493128984 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7eff080a2cf0 0x7eff080a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:37.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.573+0000 7eff1dd9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7e8039370 0x55d7e803b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:37.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.573+0000 7eff1dd9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7e8039370 0x55d7e803b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41890/0 (socket says 172.21.15.1:41890) 2024-09-11T22:06:37.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.573+0000 7eff1dd9b640 1 -- 172.21.15.1:0/4196934325 learned_addr learned my addr 172.21.15.1:0/4196934325 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:06:37.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.573+0000 7eff1ed9d640 1 --2- 172.21.15.1:0/4196934325 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d7e802fb40 0x55d7e802e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:37.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.573+0000 7eff0f7fe640 1 -- 172.21.15.1:0/3493128984 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7eff100cfa50 con 0x7eff20102d30 2024-09-11T22:06:37.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.573+0000 7eff1ed9d640 1 --2- 172.21.15.1:0/3493128984 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7eff080a2cf0 0x7eff080a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:37.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.573+0000 7eff1e59c640 1 --2- 172.21.15.1:0/4196934325 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7e802e5d0 0x55d7e802ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:37.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.574+0000 7eff1dd9b640 1 -- 172.21.15.1:0/4196934325 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7e802e5d0 msgr2=0x55d7e802ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:37.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.574+0000 7eff1dd9b640 1 --2- 172.21.15.1:0/4196934325 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7e802e5d0 0x55d7e802ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:37.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.574+0000 7eff1dd9b640 1 -- 172.21.15.1:0/4196934325 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d7e802fb40 msgr2=0x55d7e802e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:37.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.574+0000 7eff1dd9b640 1 --2- 172.21.15.1:0/4196934325 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d7e802fb40 0x55d7e802e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:37.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.574+0000 7eff1dd9b640 1 -- 172.21.15.1:0/4196934325 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7e7f203a0 con 0x55d7e8039370 2024-09-11T22:06:37.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.574+0000 7eff1ed9d640 1 --2- 172.21.15.1:0/3493128984 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7eff080a2cf0 0x7eff080a51b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7eff180128f0 tx=0x7eff18005f90 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:37.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.574+0000 7eff1dd9b640 1 --2- 172.21.15.1:0/4196934325 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7e8039370 0x55d7e803b760 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7eff14006010 tx=0x7eff14009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e85b0fe9437a94d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:37.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.574+0000 7eff0cff9640 1 -- 172.21.15.1:0/4196934325 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff1403f070 con 0x55d7e8039370 2024-09-11T22:06:37.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.574+0000 7eff0cff9640 1 -- 172.21.15.1:0/4196934325 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff14004030 con 0x55d7e8039370 2024-09-11T22:06:37.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.574+0000 7eff0cff9640 1 -- 172.21.15.1:0/4196934325 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff14035070 con 0x55d7e8039370 2024-09-11T22:06:37.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.575+0000 7eff270b0740 1 -- 172.21.15.1:0/4196934325 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7e8039370 msgr2=0x55d7e803b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:37.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.575+0000 7eff270b0740 1 --2- 172.21.15.1:0/4196934325 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7e8039370 0x55d7e803b760 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7eff14006010 tx=0x7eff14009680 comp rx=0 tx=0).stop 2024-09-11T22:06:37.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.575+0000 7eff270b0740 1 -- 172.21.15.1:0/4196934325 shutdown_connections 2024-09-11T22:06:37.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.575+0000 7eff270b0740 1 --2- 172.21.15.1:0/4196934325 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7e8039370 0x55d7e803b760 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:37.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.575+0000 7eff270b0740 1 --2- 172.21.15.1:0/4196934325 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7e802e5d0 0x55d7e802ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:37.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.575+0000 7eff270b0740 1 --2- 172.21.15.1:0/4196934325 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d7e802fb40 0x55d7e802e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:37.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.575+0000 7eff270b0740 1 -- 172.21.15.1:0/4196934325 >> 172.21.15.1:0/4196934325 conn(0x55d7e7ff7310 msgr2=0x55d7e802ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:37.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.575+0000 7eff270b0740 1 -- 172.21.15.1:0/4196934325 shutdown_connections 2024-09-11T22:06:37.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.575+0000 7eff270b0740 1 -- 172.21.15.1:0/4196934325 wait complete. 2024-09-11T22:06:37.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.575+0000 7eff270b0740 1 -- start start 2024-09-11T22:06:37.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.575+0000 7eff270b0740 1 Processor -- start 2024-09-11T22:06:37.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.576+0000 7eff270b0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7e802e5d0 0x55d7e8038cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:37.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.576+0000 7eff270b0740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7e802fb40 0x55d7e8035f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:37.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.576+0000 7eff270b0740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d7e8039370 0x55d7e80364e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:37.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.576+0000 7eff270b0740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55d7e803c010 con 0x55d7e802fb40 2024-09-11T22:06:37.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.576+0000 7eff270b0740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55d7e7f4f060 con 0x55d7e8039370 2024-09-11T22:06:37.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.576+0000 7eff270b0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d7e803c190 con 0x55d7e802e5d0 2024-09-11T22:06:37.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.576+0000 7eff270b0740 10 client.0 _ll_register_callbacks cb 0x55d7e7ca28c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:06:37.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.576+0000 7eff1dd9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7e802fb40 0x55d7e8035f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:37.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.576+0000 7eff1dd9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7e802fb40 0x55d7e8035f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41896/0 (socket says 172.21.15.1:41896) 2024-09-11T22:06:37.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.576+0000 7eff1dd9b640 1 -- 172.21.15.1:0/3829680914 learned_addr learned my addr 172.21.15.1:0/3829680914 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:06:37.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.576+0000 7eff1ed9d640 1 --2- 172.21.15.1:0/3829680914 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d7e8039370 0x55d7e80364e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:37.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.576+0000 7eff1e59c640 1 --2- 172.21.15.1:0/3829680914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7e802e5d0 0x55d7e8038cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:37.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.576+0000 7eff1dd9b640 1 -- 172.21.15.1:0/3829680914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7e802e5d0 msgr2=0x55d7e8038cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:37.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.576+0000 7eff1dd9b640 1 --2- 172.21.15.1:0/3829680914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7e802e5d0 0x55d7e8038cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:37.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.576+0000 7eff1dd9b640 1 -- 172.21.15.1:0/3829680914 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d7e8039370 msgr2=0x55d7e80364e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:37.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.576+0000 7eff1dd9b640 1 --2- 172.21.15.1:0/3829680914 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d7e8039370 0x55d7e80364e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:37.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.576+0000 7eff1dd9b640 1 -- 172.21.15.1:0/3829680914 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7e7d053c0 con 0x55d7e802fb40 2024-09-11T22:06:37.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.576+0000 7eff1e59c640 1 --2- 172.21.15.1:0/3829680914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7e802e5d0 0x55d7e8038cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:06:37.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.576+0000 7eff1dd9b640 1 --2- 172.21.15.1:0/3829680914 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7e802fb40 0x55d7e8035f80 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7eff1402abc0 tx=0x7eff140054d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3afea4e4dbfd9902 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:37.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.576+0000 7eff05ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:06:37.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.576+0000 7eff05ffb640 1 -- 172.21.15.1:0/3829680914 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff1404b020 con 0x55d7e802fb40 2024-09-11T22:06:37.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.577+0000 7eff05ffb640 1 -- 172.21.15.1:0/3829680914 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7eff140315b0 con 0x55d7e802fb40 2024-09-11T22:06:37.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.577+0000 7eff270b0740 1 -- 172.21.15.1:0/3829680914 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff14004030 con 0x55d7e802fb40 2024-09-11T22:06:37.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.577+0000 7eff05ffb640 1 -- 172.21.15.1:0/3829680914 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7eff14056050 con 0x55d7e802fb40 2024-09-11T22:06:37.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.577+0000 7eff270b0740 10 client.17679 fetch_fsmap learned FSMap version 46 2024-09-11T22:06:37.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.577+0000 7eff270b0740 1 -- 172.21.15.1:0/3829680914 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55d7e8047a20 con 0x55d7e802fb40 2024-09-11T22:06:37.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.577+0000 7eff05ffb640 1 -- 172.21.15.1:0/3829680914 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff14004030 con 0x55d7e802fb40 2024-09-11T22:06:37.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.577+0000 7eff05ffb640 1 -- 172.21.15.1:0/3829680914 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 46) ==== 8442+0+0 (secure 0 0 0) 0x7eff1403b4d0 con 0x55d7e802fb40 2024-09-11T22:06:37.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.577+0000 7eff270b0740 10 client.17679 fetch_fsmap finished waiting for FSMap version 46 2024-09-11T22:06:37.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.577+0000 7eff270b0740 10 client.17679 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T22:06:37.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.577+0000 7eff270b0740 20 client.17679 populate_metadata read hostname 'smithi001' 2024-09-11T22:06:37.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.577+0000 7eff270b0740 1 --2- 172.21.15.1:0/3829680914 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x55d7e8048d70 0x55d7e804b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:37.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.577+0000 7eff270b0740 4 client.17679 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:06:37.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.577+0000 7eff270b0740 1 -- 172.21.15.1:0/3829680914 --> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7e7fa1860 con 0x55d7e8048d70 2024-09-11T22:06:37.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.577+0000 7eff1e59c640 1 --2- 172.21.15.1:0/3829680914 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x55d7e8048d70 0x55d7e804b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:37.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.578+0000 7eff1e59c640 1 --2- 172.21.15.1:0/3829680914 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x55d7e8048d70 0x55d7e804b160 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=48bd87bb033f3e44 server_cookie=18a3998fd6df6e8d in_seq=0 out_seq=0 2024-09-11T22:06:37.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.578+0000 7eff05ffb640 10 client.17679 ms_handle_connect on v2:172.21.15.161:6838/455365383 2024-09-11T22:06:37.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.579+0000 7eff05ffb640 1 -- 172.21.15.1:0/3829680914 <== mds.1 v2:172.21.15.161:6838/455365383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d7e7fa1860 con 0x55d7e8048d70 2024-09-11T22:06:37.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.579+0000 7eff05ffb640 10 client.17679 handle_command_reply: tid=0 2024-09-11T22:06:37.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.579+0000 7eff270b0740 1 client.17679 shutdown 2024-09-11T22:06:37.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.579+0000 7eff270b0740 1 -- 172.21.15.1:0/3829680914 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7e802fb40 msgr2=0x55d7e8035f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:37.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.579+0000 7eff270b0740 1 --2- 172.21.15.1:0/3829680914 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7e802fb40 0x55d7e8035f80 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7eff1402abc0 tx=0x7eff140054d0 comp rx=0 tx=0).stop 2024-09-11T22:06:37.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.579+0000 7eff270b0740 1 -- 172.21.15.1:0/3829680914 shutdown_connections 2024-09-11T22:06:37.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.579+0000 7eff270b0740 1 --2- 172.21.15.1:0/3829680914 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x55d7e8048d70 0x55d7e804b160 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:37.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.579+0000 7eff05ffb640 0 client.17679 ms_handle_reset on v2:172.21.15.161:6838/455365383 2024-09-11T22:06:37.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.579+0000 7eff270b0740 1 --2- 172.21.15.1:0/3829680914 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d7e8039370 0x55d7e80364e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:37.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.579+0000 7eff270b0740 1 --2- 172.21.15.1:0/3829680914 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7e802fb40 0x55d7e8035f80 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:37.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.579+0000 7eff270b0740 1 --2- 172.21.15.1:0/3829680914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7e802e5d0 0x55d7e8038cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:37.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.579+0000 7eff270b0740 1 -- 172.21.15.1:0/3829680914 >> 172.21.15.1:0/3829680914 conn(0x55d7e7ff7310 msgr2=0x55d7e8030960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:37.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.579+0000 7eff270b0740 1 -- 172.21.15.1:0/3829680914 shutdown_connections 2024-09-11T22:06:37.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.579+0000 7eff270b0740 1 -- 172.21.15.1:0/3829680914 wait complete. 2024-09-11T22:06:37.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.579+0000 7eff270b0740 20 client.17679 trim_cache size 0 max 16384 2024-09-11T22:06:37.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.592+0000 7eff270b0740 1 Processor -- start 2024-09-11T22:06:37.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.592+0000 7eff270b0740 1 -- start start 2024-09-11T22:06:37.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.593+0000 7eff270b0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7e802e5d0 0x55d7e80452b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:37.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.593+0000 7eff270b0740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d7e802fb40 0x55d7e8035840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:37.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.593+0000 7eff270b0740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7e8039370 0x55d7e803a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:37.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.593+0000 7eff270b0740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7eff201ad9f0 con 0x55d7e8039370 2024-09-11T22:06:37.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.593+0000 7eff270b0740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7eff201af830 con 0x55d7e802fb40 2024-09-11T22:06:37.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.593+0000 7eff270b0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7eff201ae180 con 0x55d7e802e5d0 2024-09-11T22:06:37.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.593+0000 7eff1ed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7e8039370 0x55d7e803a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:37.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.593+0000 7eff1ed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7e8039370 0x55d7e803a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41910/0 (socket says 172.21.15.1:41910) 2024-09-11T22:06:37.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.593+0000 7eff1ed9d640 1 -- 172.21.15.1:0/2944385014 learned_addr learned my addr 172.21.15.1:0/2944385014 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:06:37.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.593+0000 7eff1dd9b640 1 --2- 172.21.15.1:0/2944385014 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d7e802fb40 0x55d7e8035840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:37.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.593+0000 7eff1e59c640 1 --2- 172.21.15.1:0/2944385014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7e802e5d0 0x55d7e80452b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:37.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.593+0000 7eff1ed9d640 1 -- 172.21.15.1:0/2944385014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7e802e5d0 msgr2=0x55d7e80452b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:37.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.593+0000 7eff1ed9d640 1 --2- 172.21.15.1:0/2944385014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7e802e5d0 0x55d7e80452b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:37.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.593+0000 7eff1ed9d640 1 -- 172.21.15.1:0/2944385014 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d7e802fb40 msgr2=0x55d7e8035840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:37.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.593+0000 7eff1ed9d640 1 --2- 172.21.15.1:0/2944385014 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d7e802fb40 0x55d7e8035840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:37.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.593+0000 7eff1ed9d640 1 -- 172.21.15.1:0/2944385014 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7e804b5c0 con 0x55d7e8039370 2024-09-11T22:06:37.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.593+0000 7eff1e59c640 1 --2- 172.21.15.1:0/2944385014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7e802e5d0 0x55d7e80452b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:06:37.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.594+0000 7eff1ed9d640 1 --2- 172.21.15.1:0/2944385014 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7e8039370 0x55d7e803a760 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7eff18012fa0 tx=0x7eff18009610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25f14ec4f3103e2a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:37.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.594+0000 7eff07fff640 1 -- 172.21.15.1:0/2944385014 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff1800dd60 con 0x55d7e8039370 2024-09-11T22:06:37.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.594+0000 7eff07fff640 1 -- 172.21.15.1:0/2944385014 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff1800f930 con 0x55d7e8039370 2024-09-11T22:06:37.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.594+0000 7eff07fff640 1 -- 172.21.15.1:0/2944385014 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff1802b7b0 con 0x55d7e8039370 2024-09-11T22:06:37.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.594+0000 7eff270b0740 1 -- 172.21.15.1:0/2944385014 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7e8039370 msgr2=0x55d7e803a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:37.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.594+0000 7eff270b0740 1 --2- 172.21.15.1:0/2944385014 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7e8039370 0x55d7e803a760 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7eff18012fa0 tx=0x7eff18009610 comp rx=0 tx=0).stop 2024-09-11T22:06:37.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.595+0000 7eff270b0740 1 -- 172.21.15.1:0/2944385014 shutdown_connections 2024-09-11T22:06:37.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.595+0000 7eff270b0740 1 --2- 172.21.15.1:0/2944385014 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7e8039370 0x55d7e803a760 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:37.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.595+0000 7eff270b0740 1 --2- 172.21.15.1:0/2944385014 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d7e802fb40 0x55d7e8035840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:37.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.595+0000 7eff270b0740 1 --2- 172.21.15.1:0/2944385014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7e802e5d0 0x55d7e80452b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:37.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.595+0000 7eff270b0740 1 -- 172.21.15.1:0/2944385014 >> 172.21.15.1:0/2944385014 conn(0x55d7e7ff7310 msgr2=0x55d7e8035070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:37.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.595+0000 7eff270b0740 1 -- 172.21.15.1:0/2944385014 shutdown_connections 2024-09-11T22:06:37.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.595+0000 7eff270b0740 1 -- 172.21.15.1:0/2944385014 wait complete. 2024-09-11T22:06:37.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.595+0000 7eff270b0740 1 -- start start 2024-09-11T22:06:37.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.595+0000 7eff270b0740 1 Processor -- start 2024-09-11T22:06:37.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.595+0000 7eff270b0740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7e802e5d0 0x55d7e8050440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:37.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.595+0000 7eff270b0740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7e802fb40 0x55d7e8050980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:37.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.595+0000 7eff270b0740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d7e8039370 0x55d7e8050ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:37.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.595+0000 7eff270b0740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7eff201af0a0 con 0x55d7e802e5d0 2024-09-11T22:06:37.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.595+0000 7eff270b0740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7eff201ae910 con 0x55d7e8039370 2024-09-11T22:06:37.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.595+0000 7eff270b0740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7eff201ab9d0 con 0x55d7e802fb40 2024-09-11T22:06:37.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.595+0000 7eff270b0740 10 client.0 _ll_register_callbacks cb 0x7eff18009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:06:37.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.595+0000 7eff1e59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7e802e5d0 0x55d7e8050440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:37.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.596+0000 7eff1e59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7e802e5d0 0x55d7e8050440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41916/0 (socket says 172.21.15.1:41916) 2024-09-11T22:06:37.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.596+0000 7eff1e59c640 1 -- 172.21.15.1:0/3513423880 learned_addr learned my addr 172.21.15.1:0/3513423880 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:06:37.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.596+0000 7eff1dd9b640 1 --2- 172.21.15.1:0/3513423880 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7e802fb40 0x55d7e8050980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:37.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.596+0000 7eff1ed9d640 1 --2- 172.21.15.1:0/3513423880 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d7e8039370 0x55d7e8050ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:37.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.596+0000 7eff1e59c640 1 -- 172.21.15.1:0/3513423880 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7e802fb40 msgr2=0x55d7e8050980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:37.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.596+0000 7eff1e59c640 1 --2- 172.21.15.1:0/3513423880 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7e802fb40 0x55d7e8050980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:37.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.596+0000 7eff1e59c640 1 -- 172.21.15.1:0/3513423880 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d7e8039370 msgr2=0x55d7e8050ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:37.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.596+0000 7eff1e59c640 1 --2- 172.21.15.1:0/3513423880 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d7e8039370 0x55d7e8050ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:37.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.596+0000 7eff1e59c640 1 -- 172.21.15.1:0/3513423880 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7e803f930 con 0x55d7e802e5d0 2024-09-11T22:06:37.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.596+0000 7eff1dd9b640 1 --2- 172.21.15.1:0/3513423880 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7e802fb40 0x55d7e8050980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:06:37.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.596+0000 7eff1ed9d640 1 --2- 172.21.15.1:0/3513423880 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d7e8039370 0x55d7e8050ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:06:37.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.596+0000 7eff1e59c640 1 --2- 172.21.15.1:0/3513423880 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7e802e5d0 0x55d7e8050440 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7eff100988a0 tx=0x7eff1006de90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab32314dabbfa5b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:37.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.596+0000 7eff05ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:06:37.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.596+0000 7eff05ffb640 1 -- 172.21.15.1:0/3513423880 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff100b7070 con 0x55d7e802e5d0 2024-09-11T22:06:37.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.596+0000 7eff270b0740 1 -- 172.21.15.1:0/3513423880 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff1800f930 con 0x55d7e802e5d0 2024-09-11T22:06:37.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.596+0000 7eff05ffb640 1 -- 172.21.15.1:0/3513423880 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7eff100cd380 con 0x55d7e802e5d0 2024-09-11T22:06:37.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.597+0000 7eff05ffb640 1 -- 172.21.15.1:0/3513423880 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7eff1007f050 con 0x55d7e802e5d0 2024-09-11T22:06:37.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.597+0000 7eff270b0740 10 client.17685 fetch_fsmap learned FSMap version 46 2024-09-11T22:06:37.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.597+0000 7eff270b0740 1 -- 172.21.15.1:0/3513423880 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55d7e80b7f00 con 0x55d7e802e5d0 2024-09-11T22:06:37.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.597+0000 7eff05ffb640 1 -- 172.21.15.1:0/3513423880 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff1800f930 con 0x55d7e802e5d0 2024-09-11T22:06:37.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.597+0000 7eff05ffb640 1 -- 172.21.15.1:0/3513423880 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 46) ==== 8442+0+0 (secure 0 0 0) 0x7eff100a46b0 con 0x55d7e802e5d0 2024-09-11T22:06:37.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.597+0000 7eff270b0740 10 client.17685 fetch_fsmap finished waiting for FSMap version 46 2024-09-11T22:06:37.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.597+0000 7eff270b0740 10 client.17685 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T22:06:37.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.597+0000 7eff270b0740 20 client.17685 populate_metadata read hostname 'smithi001' 2024-09-11T22:06:37.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.597+0000 7eff270b0740 1 --2- 172.21.15.1:0/3513423880 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x55d7e8048d70 0x55d7e80ba640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:37.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.597+0000 7eff270b0740 4 client.17685 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:06:37.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.597+0000 7eff270b0740 1 -- 172.21.15.1:0/3513423880 --> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d7e7f9bd70 con 0x55d7e8048d70 2024-09-11T22:06:37.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.597+0000 7eff1dd9b640 1 --2- 172.21.15.1:0/3513423880 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x55d7e8048d70 0x55d7e80ba640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:37.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.598+0000 7eff1dd9b640 1 --2- 172.21.15.1:0/3513423880 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x55d7e8048d70 0x55d7e80ba640 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=aa3e59279da7b7bc server_cookie=d6c3cfcd47f3a6b0 in_seq=0 out_seq=0 2024-09-11T22:06:37.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.598+0000 7eff05ffb640 10 client.17685 ms_handle_connect on v2:172.21.15.161:6838/455365383 2024-09-11T22:06:37.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.598+0000 7eff05ffb640 1 -- 172.21.15.1:0/3513423880 <== mds.1 v2:172.21.15.161:6838/455365383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d7e7f9bd70 con 0x55d7e8048d70 2024-09-11T22:06:37.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.598+0000 7eff05ffb640 10 client.17685 handle_command_reply: tid=0 2024-09-11T22:06:37.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.598+0000 7eff270b0740 1 client.17685 shutdown 2024-09-11T22:06:37.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.598+0000 7eff270b0740 1 -- 172.21.15.1:0/3513423880 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7e802e5d0 msgr2=0x55d7e8050440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:37.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.598+0000 7eff270b0740 1 --2- 172.21.15.1:0/3513423880 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7e802e5d0 0x55d7e8050440 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7eff100988a0 tx=0x7eff1006de90 comp rx=0 tx=0).stop 2024-09-11T22:06:37.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.598+0000 7eff270b0740 1 -- 172.21.15.1:0/3513423880 shutdown_connections 2024-09-11T22:06:37.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.598+0000 7eff270b0740 1 --2- 172.21.15.1:0/3513423880 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x55d7e8048d70 0x55d7e80ba640 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:37.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.598+0000 7eff270b0740 1 --2- 172.21.15.1:0/3513423880 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d7e8039370 0x55d7e8050ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:37.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.598+0000 7eff270b0740 1 --2- 172.21.15.1:0/3513423880 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7e802fb40 0x55d7e8050980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:37.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.598+0000 7eff270b0740 1 --2- 172.21.15.1:0/3513423880 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7e802e5d0 0x55d7e8050440 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:37.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.598+0000 7eff05ffb640 0 client.17685 ms_handle_reset on v2:172.21.15.161:6838/455365383 2024-09-11T22:06:37.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.598+0000 7eff270b0740 1 -- 172.21.15.1:0/3513423880 >> 172.21.15.1:0/3513423880 conn(0x55d7e7ff7310 msgr2=0x55d7e80377e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:37.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.598+0000 7eff270b0740 1 -- 172.21.15.1:0/3513423880 shutdown_connections 2024-09-11T22:06:37.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.598+0000 7eff270b0740 1 -- 172.21.15.1:0/3513423880 wait complete. 2024-09-11T22:06:37.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.599+0000 7eff270b0740 20 client.17685 trim_cache size 0 max 16384 2024-09-11T22:06:37.631 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-11T22:06:37.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.599+0000 7eff24cd7640 1 -- 172.21.15.1:0/3493128984 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7eff080a2cf0 msgr2=0x7eff080a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:37.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.599+0000 7eff24cd7640 1 --2- 172.21.15.1:0/3493128984 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7eff080a2cf0 0x7eff080a51b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7eff180128f0 tx=0x7eff18005f90 comp rx=0 tx=0).stop 2024-09-11T22:06:37.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.599+0000 7eff24cd7640 1 -- 172.21.15.1:0/3493128984 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff20102d30 msgr2=0x7eff20198520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:37.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.599+0000 7eff24cd7640 1 --2- 172.21.15.1:0/3493128984 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff20102d30 0x7eff20198520 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7eff1000cce0 tx=0x7eff1000a620 comp rx=0 tx=0).stop 2024-09-11T22:06:37.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.599+0000 7eff24cd7640 1 -- 172.21.15.1:0/3493128984 shutdown_connections 2024-09-11T22:06:37.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.599+0000 7eff24cd7640 1 --2- 172.21.15.1:0/3493128984 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7eff080a2cf0 0x7eff080a51b0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:37.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.599+0000 7eff24cd7640 1 --2- 172.21.15.1:0/3493128984 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff201082a0 0x7eff2019cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:37.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.599+0000 7eff24cd7640 1 --2- 172.21.15.1:0/3493128984 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff201036e0 0x7eff20198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:37.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.599+0000 7eff24cd7640 1 --2- 172.21.15.1:0/3493128984 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff20102d30 0x7eff20198520 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:37.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.599+0000 7eff24cd7640 1 -- 172.21.15.1:0/3493128984 >> 172.21.15.1:0/3493128984 conn(0x7eff200fc7d0 msgr2=0x7eff20105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:37.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.599+0000 7eff24cd7640 1 -- 172.21.15.1:0/3493128984 shutdown_connections 2024-09-11T22:06:37.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:37.600+0000 7eff24cd7640 1 -- 172.21.15.1:0/3493128984 wait complete. 2024-09-11T22:06:38.296 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T22:06:38.320 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:06:38 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:06:38.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:06:38 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:06:38.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:06:38 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:06:38.561 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:06:39.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.056+0000 7f36e5e0a640 1 -- 172.21.15.1:0/1674592113 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f36e010c180 msgr2=0x7f36e010e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:39.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.056+0000 7f36e5e0a640 1 --2- 172.21.15.1:0/1674592113 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f36e010c180 0x7f36e010e600 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f36d8009900 tx=0x7f36d802a480 comp rx=0 tx=0).stop 2024-09-11T22:06:39.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.056+0000 7f36e5e0a640 1 -- 172.21.15.1:0/1674592113 shutdown_connections 2024-09-11T22:06:39.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.056+0000 7f36e5e0a640 1 --2- 172.21.15.1:0/1674592113 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f36e010c180 0x7f36e010e600 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:39.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.056+0000 7f36e5e0a640 1 --2- 172.21.15.1:0/1674592113 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36e01036c0 0x7f36e010bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:39.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.056+0000 7f36e5e0a640 1 --2- 172.21.15.1:0/1674592113 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36e0102da0 0x7f36e0103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:39.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.056+0000 7f36e5e0a640 1 -- 172.21.15.1:0/1674592113 >> 172.21.15.1:0/1674592113 conn(0x7f36e00fc7a0 msgr2=0x7f36e00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:39.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.056+0000 7f36e5e0a640 1 -- 172.21.15.1:0/1674592113 shutdown_connections 2024-09-11T22:06:39.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.056+0000 7f36e5e0a640 1 -- 172.21.15.1:0/1674592113 wait complete. 2024-09-11T22:06:39.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.056+0000 7f36e5e0a640 1 Processor -- start 2024-09-11T22:06:39.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.057+0000 7f36e5e0a640 1 -- start start 2024-09-11T22:06:39.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.057+0000 7f36e5e0a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36e0102da0 0x7f36e019a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:39.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.057+0000 7f36e5e0a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f36e01036c0 0x7f36e019ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:39.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.057+0000 7f36e5e0a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36e010c180 0x7f36e019eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:39.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.057+0000 7f36e5e0a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f36e0111f40 con 0x7f36e01036c0 2024-09-11T22:06:39.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.057+0000 7f36e5e0a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f36e0111dc0 con 0x7f36e010c180 2024-09-11T22:06:39.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.057+0000 7f36e5e0a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f36e01120c0 con 0x7f36e0102da0 2024-09-11T22:06:39.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.057+0000 7f36deffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f36e01036c0 0x7f36e019ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:39.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.057+0000 7f36df7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36e0102da0 0x7f36e019a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:39.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.057+0000 7f36deffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f36e01036c0 0x7f36e019ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41940/0 (socket says 172.21.15.1:41940) 2024-09-11T22:06:39.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.057+0000 7f36deffd640 1 -- 172.21.15.1:0/1095543043 learned_addr learned my addr 172.21.15.1:0/1095543043 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:06:39.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.057+0000 7f36dffff640 1 --2- 172.21.15.1:0/1095543043 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36e010c180 0x7f36e019eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:39.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.057+0000 7f36deffd640 1 -- 172.21.15.1:0/1095543043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36e0102da0 msgr2=0x7f36e019a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:39.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.057+0000 7f36deffd640 1 --2- 172.21.15.1:0/1095543043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36e0102da0 0x7f36e019a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:39.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.057+0000 7f36deffd640 1 -- 172.21.15.1:0/1095543043 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36e010c180 msgr2=0x7f36e019eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:39.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.057+0000 7f36deffd640 1 --2- 172.21.15.1:0/1095543043 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36e010c180 0x7f36e019eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:39.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.057+0000 7f36deffd640 1 -- 172.21.15.1:0/1095543043 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36e019f5b0 con 0x7f36e01036c0 2024-09-11T22:06:39.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.057+0000 7f36df7fe640 1 --2- 172.21.15.1:0/1095543043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36e0102da0 0x7f36e019a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:06:39.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.058+0000 7f36deffd640 1 --2- 172.21.15.1:0/1095543043 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f36e01036c0 0x7f36e019ab40 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f36d400e9f0 tx=0x7f36d400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:39.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.058+0000 7f36dcff9640 1 -- 172.21.15.1:0/1095543043 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d400cd80 con 0x7f36e01036c0 2024-09-11T22:06:39.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.058+0000 7f36dcff9640 1 -- 172.21.15.1:0/1095543043 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36d4005500 con 0x7f36e01036c0 2024-09-11T22:06:39.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.058+0000 7f36e5e0a640 1 -- 172.21.15.1:0/1095543043 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36e019f8a0 con 0x7f36e01036c0 2024-09-11T22:06:39.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.058+0000 7f36dcff9640 1 -- 172.21.15.1:0/1095543043 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d4010690 con 0x7f36e01036c0 2024-09-11T22:06:39.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.058+0000 7f36e5e0a640 1 -- 172.21.15.1:0/1095543043 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36e01a7180 con 0x7f36e01036c0 2024-09-11T22:06:39.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.060+0000 7f36dcff9640 1 -- 172.21.15.1:0/1095543043 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f36d40046c0 con 0x7f36e01036c0 2024-09-11T22:06:39.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.060+0000 7f36e81e3740 1 Processor -- start 2024-09-11T22:06:39.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.061+0000 7f36e81e3740 1 -- start start 2024-09-11T22:06:39.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.061+0000 7f36e81e3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9b9765b40 0x55c9b9764090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:39.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.061+0000 7f36e81e3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9b97645d0 0x55c9b9764a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:39.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.061+0000 7f36e81e3740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c9b976f370 0x55c9b9771760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:39.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.061+0000 7f36e81e3740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55c9b96ca2f0 con 0x55c9b976f370 2024-09-11T22:06:39.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.061+0000 7f36e81e3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55c9b9671cf0 con 0x55c9b9765b40 2024-09-11T22:06:39.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.061+0000 7f36e81e3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c9b96c62b0 con 0x55c9b97645d0 2024-09-11T22:06:39.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.061+0000 7f36dcff9640 1 --2- 172.21.15.1:0/1095543043 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f36c80a2d70 0x7f36c80a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:39.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.061+0000 7f36deffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c9b976f370 0x55c9b9771760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:39.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.061+0000 7f36deffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c9b976f370 0x55c9b9771760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41948/0 (socket says 172.21.15.1:41948) 2024-09-11T22:06:39.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.061+0000 7f36deffd640 1 -- 172.21.15.1:0/1529205593 learned_addr learned my addr 172.21.15.1:0/1529205593 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:06:39.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.061+0000 7f36dffff640 1 --2- 172.21.15.1:0/1095543043 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f36c80a2d70 0x7f36c80a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:39.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.061+0000 7f36df7fe640 1 --2- 172.21.15.1:0/1529205593 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9b97645d0 0x55c9b9764a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:39.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.061+0000 7f36dffff640 1 --2- 172.21.15.1:0/1529205593 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9b9765b40 0x55c9b9764090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:39.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.061+0000 7f36dcff9640 1 -- 172.21.15.1:0/1095543043 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7f36d40cf7e0 con 0x7f36e01036c0 2024-09-11T22:06:39.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.061+0000 7f36deffd640 1 -- 172.21.15.1:0/1529205593 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9b97645d0 msgr2=0x55c9b9764a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:39.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.061+0000 7f36deffd640 1 --2- 172.21.15.1:0/1529205593 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9b97645d0 0x55c9b9764a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:39.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.061+0000 7f36deffd640 1 -- 172.21.15.1:0/1529205593 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9b9765b40 msgr2=0x55c9b9764090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:39.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.061+0000 7f36deffd640 1 --2- 172.21.15.1:0/1529205593 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9b9765b40 0x55c9b9764090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:39.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.061+0000 7f36deffd640 1 -- 172.21.15.1:0/1529205593 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9b96563a0 con 0x55c9b976f370 2024-09-11T22:06:39.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.062+0000 7f36dffff640 1 --2- 172.21.15.1:0/1095543043 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f36c80a2d70 0x7f36c80a5230 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f36d8031550 tx=0x7f36d8005d50 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:39.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.062+0000 7f36deffd640 1 --2- 172.21.15.1:0/1529205593 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c9b976f370 0x55c9b9771760 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f36d40ccf50 tx=0x7f36d4004ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a4c8b12054b284d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:39.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.062+0000 7f36c5ffb640 1 -- 172.21.15.1:0/1529205593 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d409b620 con 0x55c9b976f370 2024-09-11T22:06:39.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.062+0000 7f36c5ffb640 1 -- 172.21.15.1:0/1529205593 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36d409c3c0 con 0x55c9b976f370 2024-09-11T22:06:39.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.062+0000 7f36c5ffb640 1 -- 172.21.15.1:0/1529205593 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d4098330 con 0x55c9b976f370 2024-09-11T22:06:39.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.063+0000 7f36e81e3740 1 -- 172.21.15.1:0/1529205593 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c9b976f370 msgr2=0x55c9b9771760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:39.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.063+0000 7f36e81e3740 1 --2- 172.21.15.1:0/1529205593 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c9b976f370 0x55c9b9771760 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f36d40ccf50 tx=0x7f36d4004ae0 comp rx=0 tx=0).stop 2024-09-11T22:06:39.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.063+0000 7f36e81e3740 1 -- 172.21.15.1:0/1529205593 shutdown_connections 2024-09-11T22:06:39.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.063+0000 7f36e81e3740 1 --2- 172.21.15.1:0/1529205593 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c9b976f370 0x55c9b9771760 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:39.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.063+0000 7f36e81e3740 1 --2- 172.21.15.1:0/1529205593 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9b97645d0 0x55c9b9764a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:39.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.063+0000 7f36e81e3740 1 --2- 172.21.15.1:0/1529205593 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9b9765b40 0x55c9b9764090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:39.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.063+0000 7f36e81e3740 1 -- 172.21.15.1:0/1529205593 >> 172.21.15.1:0/1529205593 conn(0x55c9b972d310 msgr2=0x55c9b9760c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:39.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.063+0000 7f36e81e3740 1 -- 172.21.15.1:0/1529205593 shutdown_connections 2024-09-11T22:06:39.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.063+0000 7f36e81e3740 1 -- 172.21.15.1:0/1529205593 wait complete. 2024-09-11T22:06:39.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.063+0000 7f36e81e3740 1 -- start start 2024-09-11T22:06:39.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.063+0000 7f36e81e3740 1 Processor -- start 2024-09-11T22:06:39.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.064+0000 7f36e81e3740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c9b97645d0 0x55c9b976ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:39.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.064+0000 7f36e81e3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9b9765b40 0x55c9b976c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:39.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.064+0000 7f36e81e3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9b976f370 0x55c9b976c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:39.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.064+0000 7f36e81e3740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55c9b9772070 con 0x55c9b97645d0 2024-09-11T22:06:39.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.064+0000 7f36e81e3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55c9b9685060 con 0x55c9b976f370 2024-09-11T22:06:39.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.064+0000 7f36e81e3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55c9b97721f0 con 0x55c9b9765b40 2024-09-11T22:06:39.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.064+0000 7f36e81e3740 10 client.0 _ll_register_callbacks cb 0x55c9b93d88c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:06:39.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.064+0000 7f36df7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c9b97645d0 0x55c9b976ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:39.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.064+0000 7f36df7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c9b97645d0 0x55c9b976ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41958/0 (socket says 172.21.15.1:41958) 2024-09-11T22:06:39.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.064+0000 7f36df7fe640 1 -- 172.21.15.1:0/1239325807 learned_addr learned my addr 172.21.15.1:0/1239325807 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:06:39.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.064+0000 7f36deffd640 1 --2- 172.21.15.1:0/1239325807 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9b9765b40 0x55c9b976c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:39.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.064+0000 7f36dffff640 1 --2- 172.21.15.1:0/1239325807 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9b976f370 0x55c9b976c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:39.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.064+0000 7f36df7fe640 1 -- 172.21.15.1:0/1239325807 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9b9765b40 msgr2=0x55c9b976c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:39.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.064+0000 7f36df7fe640 1 --2- 172.21.15.1:0/1239325807 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9b9765b40 0x55c9b976c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:39.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.064+0000 7f36df7fe640 1 -- 172.21.15.1:0/1239325807 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9b976f370 msgr2=0x55c9b976c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:39.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.064+0000 7f36df7fe640 1 --2- 172.21.15.1:0/1239325807 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9b976f370 0x55c9b976c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:39.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.064+0000 7f36df7fe640 1 -- 172.21.15.1:0/1239325807 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9b943b3c0 con 0x55c9b97645d0 2024-09-11T22:06:39.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.064+0000 7f36df7fe640 1 --2- 172.21.15.1:0/1239325807 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c9b97645d0 0x55c9b976ed80 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f36d0010b80 tx=0x7f36d0011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de27f66faaac00be server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:39.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.064+0000 7f36beffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:06:39.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.064+0000 7f36beffd640 1 -- 172.21.15.1:0/1239325807 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d0011e60 con 0x55c9b97645d0 2024-09-11T22:06:39.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.064+0000 7f36beffd640 1 -- 172.21.15.1:0/1239325807 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7f36d0003970 con 0x55c9b97645d0 2024-09-11T22:06:39.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.064+0000 7f36e81e3740 1 -- 172.21.15.1:0/1239325807 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36d409c3c0 con 0x55c9b97645d0 2024-09-11T22:06:39.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.065+0000 7f36beffd640 1 -- 172.21.15.1:0/1239325807 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f36d0003b70 con 0x55c9b97645d0 2024-09-11T22:06:39.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.065+0000 7f36e81e3740 10 client.17697 fetch_fsmap learned FSMap version 46 2024-09-11T22:06:39.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.065+0000 7f36e81e3740 1 -- 172.21.15.1:0/1239325807 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55c9b977dba0 con 0x55c9b97645d0 2024-09-11T22:06:39.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.065+0000 7f36beffd640 1 -- 172.21.15.1:0/1239325807 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36d409c3c0 con 0x55c9b97645d0 2024-09-11T22:06:39.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.065+0000 7f36beffd640 1 -- 172.21.15.1:0/1239325807 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 46) ==== 8442+0+0 (secure 0 0 0) 0x7f36d001b620 con 0x55c9b97645d0 2024-09-11T22:06:39.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.065+0000 7f36e81e3740 10 client.17697 fetch_fsmap finished waiting for FSMap version 46 2024-09-11T22:06:39.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.065+0000 7f36e81e3740 10 client.17697 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:06:39.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.065+0000 7f36e81e3740 20 client.17697 populate_metadata read hostname 'smithi001' 2024-09-11T22:06:39.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.065+0000 7f36e81e3740 1 --2- 172.21.15.1:0/1239325807 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55c9b977f100 0x55c9b9781540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:39.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.065+0000 7f36e81e3740 4 client.17697 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:06:39.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.065+0000 7f36e81e3740 1 -- 172.21.15.1:0/1239325807 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c9b96d7860 con 0x55c9b977f100 2024-09-11T22:06:39.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.065+0000 7f36deffd640 1 --2- 172.21.15.1:0/1239325807 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55c9b977f100 0x55c9b9781540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:39.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.066+0000 7f36deffd640 1 --2- 172.21.15.1:0/1239325807 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55c9b977f100 0x55c9b9781540 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d70b27b2deb373d server_cookie=b82f7d4b0ada62cc in_seq=0 out_seq=0 2024-09-11T22:06:39.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.066+0000 7f36beffd640 10 client.17697 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:06:39.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.066+0000 7f36beffd640 1 -- 172.21.15.1:0/1239325807 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c9b96d7860 con 0x55c9b977f100 2024-09-11T22:06:39.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.066+0000 7f36beffd640 10 client.17697 handle_command_reply: tid=0 2024-09-11T22:06:39.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.066+0000 7f36e81e3740 1 client.17697 shutdown 2024-09-11T22:06:39.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.067+0000 7f36e81e3740 1 -- 172.21.15.1:0/1239325807 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c9b97645d0 msgr2=0x55c9b976ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:39.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.067+0000 7f36e81e3740 1 --2- 172.21.15.1:0/1239325807 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c9b97645d0 0x55c9b976ed80 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f36d0010b80 tx=0x7f36d0011440 comp rx=0 tx=0).stop 2024-09-11T22:06:39.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.067+0000 7f36e81e3740 1 -- 172.21.15.1:0/1239325807 shutdown_connections 2024-09-11T22:06:39.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.067+0000 7f36e81e3740 1 --2- 172.21.15.1:0/1239325807 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55c9b977f100 0x55c9b9781540 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:39.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.067+0000 7f36e81e3740 1 --2- 172.21.15.1:0/1239325807 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9b976f370 0x55c9b976c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:39.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.067+0000 7f36e81e3740 1 --2- 172.21.15.1:0/1239325807 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9b9765b40 0x55c9b976c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:39.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.067+0000 7f36beffd640 0 client.17697 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:06:39.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.067+0000 7f36e81e3740 1 --2- 172.21.15.1:0/1239325807 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c9b97645d0 0x55c9b976ed80 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:39.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.067+0000 7f36e81e3740 1 -- 172.21.15.1:0/1239325807 >> 172.21.15.1:0/1239325807 conn(0x55c9b972d310 msgr2=0x55c9b9766990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:39.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.067+0000 7f36e81e3740 1 -- 172.21.15.1:0/1239325807 shutdown_connections 2024-09-11T22:06:39.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.067+0000 7f36e81e3740 1 -- 172.21.15.1:0/1239325807 wait complete. 2024-09-11T22:06:39.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.067+0000 7f36e81e3740 20 client.17697 trim_cache size 0 max 16384 2024-09-11T22:06:39.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.080+0000 7f36e81e3740 1 Processor -- start 2024-09-11T22:06:39.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.080+0000 7f36e81e3740 1 -- start start 2024-09-11T22:06:39.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.081+0000 7f36e81e3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9b97645d0 0x55c9b9760e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:39.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.081+0000 7f36e81e3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9b9765b40 0x55c9b977b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:39.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.081+0000 7f36e81e3740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c9b976f370 0x55c9b9786e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:39.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.081+0000 7f36e81e3740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f36e01afca0 con 0x55c9b976f370 2024-09-11T22:06:39.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.081+0000 7f36e81e3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f36e01b1ae0 con 0x55c9b97645d0 2024-09-11T22:06:39.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.081+0000 7f36e81e3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f36e01b0430 con 0x55c9b9765b40 2024-09-11T22:06:39.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.081+0000 7f36dffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c9b976f370 0x55c9b9786e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:39.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.081+0000 7f36dffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c9b976f370 0x55c9b9786e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41962/0 (socket says 172.21.15.1:41962) 2024-09-11T22:06:39.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.081+0000 7f36dffff640 1 -- 172.21.15.1:0/267104139 learned_addr learned my addr 172.21.15.1:0/267104139 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:06:39.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.081+0000 7f36deffd640 1 --2- 172.21.15.1:0/267104139 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9b9765b40 0x55c9b977b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:39.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.081+0000 7f36df7fe640 1 --2- 172.21.15.1:0/267104139 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9b97645d0 0x55c9b9760e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:39.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.081+0000 7f36dffff640 1 -- 172.21.15.1:0/267104139 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9b9765b40 msgr2=0x55c9b977b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:39.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.081+0000 7f36dffff640 1 --2- 172.21.15.1:0/267104139 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9b9765b40 0x55c9b977b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:39.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.081+0000 7f36dffff640 1 -- 172.21.15.1:0/267104139 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9b97645d0 msgr2=0x55c9b9760e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:39.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.081+0000 7f36dffff640 1 --2- 172.21.15.1:0/267104139 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9b97645d0 0x55c9b9760e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:39.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.081+0000 7f36dffff640 1 -- 172.21.15.1:0/267104139 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9b9761960 con 0x55c9b976f370 2024-09-11T22:06:39.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.081+0000 7f36dffff640 1 --2- 172.21.15.1:0/267104139 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c9b976f370 0x55c9b9786e50 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f36d8009900 tx=0x7f36d802a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80bee05e63ff44c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:39.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.082+0000 7f36c57fa640 1 -- 172.21.15.1:0/267104139 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d8050070 con 0x55c9b976f370 2024-09-11T22:06:39.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.082+0000 7f36c57fa640 1 -- 172.21.15.1:0/267104139 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36d80047a0 con 0x55c9b976f370 2024-09-11T22:06:39.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.082+0000 7f36c57fa640 1 -- 172.21.15.1:0/267104139 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d8003830 con 0x55c9b976f370 2024-09-11T22:06:39.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.082+0000 7f36e81e3740 1 -- 172.21.15.1:0/267104139 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c9b976f370 msgr2=0x55c9b9786e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:39.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.082+0000 7f36e81e3740 1 --2- 172.21.15.1:0/267104139 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c9b976f370 0x55c9b9786e50 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f36d8009900 tx=0x7f36d802a480 comp rx=0 tx=0).stop 2024-09-11T22:06:39.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.082+0000 7f36e81e3740 1 -- 172.21.15.1:0/267104139 shutdown_connections 2024-09-11T22:06:39.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.082+0000 7f36e81e3740 1 --2- 172.21.15.1:0/267104139 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c9b976f370 0x55c9b9786e50 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:39.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.082+0000 7f36e81e3740 1 --2- 172.21.15.1:0/267104139 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9b9765b40 0x55c9b977b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:39.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.082+0000 7f36e81e3740 1 --2- 172.21.15.1:0/267104139 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9b97645d0 0x55c9b9760e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:39.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.082+0000 7f36e81e3740 1 -- 172.21.15.1:0/267104139 >> 172.21.15.1:0/267104139 conn(0x55c9b972d310 msgr2=0x55c9b9760700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:39.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.082+0000 7f36e81e3740 1 -- 172.21.15.1:0/267104139 shutdown_connections 2024-09-11T22:06:39.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.082+0000 7f36e81e3740 1 -- 172.21.15.1:0/267104139 wait complete. 2024-09-11T22:06:39.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.083+0000 7f36e81e3740 1 -- start start 2024-09-11T22:06:39.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.083+0000 7f36e81e3740 1 Processor -- start 2024-09-11T22:06:39.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.083+0000 7f36e81e3740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9b97645d0 0x55c9b97734e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:39.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.083+0000 7f36e81e3740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9b9765b40 0x55c9b9773a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:39.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.083+0000 7f36e81e3740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c9b976f370 0x55c9b9773fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:39.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.083+0000 7f36e81e3740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f36e01b1350 con 0x55c9b976f370 2024-09-11T22:06:39.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.083+0000 7f36e81e3740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f36e01b0bc0 con 0x55c9b9765b40 2024-09-11T22:06:39.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.083+0000 7f36e81e3740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f36e01adc80 con 0x55c9b97645d0 2024-09-11T22:06:39.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.083+0000 7f36e81e3740 10 client.0 _ll_register_callbacks cb 0x7f36d0010510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:06:39.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.084+0000 7f36dffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c9b976f370 0x55c9b9773fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:39.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.084+0000 7f36df7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9b97645d0 0x55c9b97734e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:39.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.084+0000 7f36df7fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9b97645d0 0x55c9b97734e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.1:47714/0 (socket says 172.21.15.1:47714) 2024-09-11T22:06:39.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.084+0000 7f36dffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c9b976f370 0x55c9b9773fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41964/0 (socket says 172.21.15.1:41964) 2024-09-11T22:06:39.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.084+0000 7f36df7fe640 1 -- 172.21.15.1:0/495414448 learned_addr learned my addr 172.21.15.1:0/495414448 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:06:39.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.084+0000 7f36deffd640 1 --2- 172.21.15.1:0/495414448 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9b9765b40 0x55c9b9773a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:39.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.084+0000 7f36dffff640 1 -- 172.21.15.1:0/495414448 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9b97645d0 msgr2=0x55c9b97734e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:39.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.084+0000 7f36dffff640 1 --2- 172.21.15.1:0/495414448 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9b97645d0 0x55c9b97734e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:39.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.084+0000 7f36dffff640 1 -- 172.21.15.1:0/495414448 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9b9765b40 msgr2=0x55c9b9773a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:39.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.084+0000 7f36dffff640 1 --2- 172.21.15.1:0/495414448 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9b9765b40 0x55c9b9773a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:39.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.084+0000 7f36dffff640 1 -- 172.21.15.1:0/495414448 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9b97e9110 con 0x55c9b976f370 2024-09-11T22:06:39.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.084+0000 7f36df7fe640 1 --2- 172.21.15.1:0/495414448 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9b97645d0 0x55c9b97734e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:06:39.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.084+0000 7f36dffff640 1 --2- 172.21.15.1:0/495414448 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c9b976f370 0x55c9b9773fa0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f36d8006010 tx=0x7f36d8004b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8abf93b4242fcda server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:39.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.084+0000 7f36beffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:06:39.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.084+0000 7f36beffd640 1 -- 172.21.15.1:0/495414448 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d8050050 con 0x55c9b976f370 2024-09-11T22:06:39.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.084+0000 7f36beffd640 1 -- 172.21.15.1:0/495414448 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7f36d8005380 con 0x55c9b976f370 2024-09-11T22:06:39.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.084+0000 7f36e81e3740 1 -- 172.21.15.1:0/495414448 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36d80047a0 con 0x55c9b976f370 2024-09-11T22:06:39.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.085+0000 7f36beffd640 1 -- 172.21.15.1:0/495414448 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f36d8065050 con 0x55c9b976f370 2024-09-11T22:06:39.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.085+0000 7f36e81e3740 10 client.17703 fetch_fsmap learned FSMap version 46 2024-09-11T22:06:39.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.085+0000 7f36e81e3740 1 -- 172.21.15.1:0/495414448 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55c9b97ea410 con 0x55c9b976f370 2024-09-11T22:06:39.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.085+0000 7f36beffd640 1 -- 172.21.15.1:0/495414448 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36d80047a0 con 0x55c9b976f370 2024-09-11T22:06:39.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.085+0000 7f36beffd640 1 -- 172.21.15.1:0/495414448 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 46) ==== 8442+0+0 (secure 0 0 0) 0x7f36d804c480 con 0x55c9b976f370 2024-09-11T22:06:39.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.085+0000 7f36e81e3740 10 client.17703 fetch_fsmap finished waiting for FSMap version 46 2024-09-11T22:06:39.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.085+0000 7f36e81e3740 10 client.17703 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:06:39.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.085+0000 7f36e81e3740 20 client.17703 populate_metadata read hostname 'smithi001' 2024-09-11T22:06:39.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.085+0000 7f36e81e3740 1 --2- 172.21.15.1:0/495414448 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55c9b97eab90 0x55c9b97ecfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:39.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.085+0000 7f36e81e3740 4 client.17703 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:06:39.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.085+0000 7f36e81e3740 1 -- 172.21.15.1:0/495414448 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c9b96d1d70 con 0x55c9b97eab90 2024-09-11T22:06:39.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.085+0000 7f36df7fe640 1 --2- 172.21.15.1:0/495414448 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55c9b97eab90 0x55c9b97ecfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:39.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.086+0000 7f36df7fe640 1 --2- 172.21.15.1:0/495414448 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55c9b97eab90 0x55c9b97ecfb0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a7ad0e6006777e4 server_cookie=c7173689eda1be99 in_seq=0 out_seq=0 2024-09-11T22:06:39.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.086+0000 7f36beffd640 10 client.17703 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:06:39.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.086+0000 7f36beffd640 1 -- 172.21.15.1:0/495414448 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c9b96d1d70 con 0x55c9b97eab90 2024-09-11T22:06:39.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.086+0000 7f36beffd640 10 client.17703 handle_command_reply: tid=0 2024-09-11T22:06:39.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.086+0000 7f36e81e3740 1 client.17703 shutdown 2024-09-11T22:06:39.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.086+0000 7f36e81e3740 1 -- 172.21.15.1:0/495414448 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c9b976f370 msgr2=0x55c9b9773fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:39.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.086+0000 7f36e81e3740 1 --2- 172.21.15.1:0/495414448 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c9b976f370 0x55c9b9773fa0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f36d8006010 tx=0x7f36d8004b80 comp rx=0 tx=0).stop 2024-09-11T22:06:39.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.086+0000 7f36e81e3740 1 -- 172.21.15.1:0/495414448 shutdown_connections 2024-09-11T22:06:39.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.086+0000 7f36e81e3740 1 --2- 172.21.15.1:0/495414448 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55c9b97eab90 0x55c9b97ecfb0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:39.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.086+0000 7f36e81e3740 1 --2- 172.21.15.1:0/495414448 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c9b976f370 0x55c9b9773fa0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:39.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.086+0000 7f36e81e3740 1 --2- 172.21.15.1:0/495414448 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c9b9765b40 0x55c9b9773a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:39.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.086+0000 7f36e81e3740 1 --2- 172.21.15.1:0/495414448 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c9b97645d0 0x55c9b97734e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:39.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.086+0000 7f36e81e3740 1 -- 172.21.15.1:0/495414448 >> 172.21.15.1:0/495414448 conn(0x55c9b972d310 msgr2=0x55c9b975f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:39.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.086+0000 7f36beffd640 0 client.17703 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:06:39.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.086+0000 7f36e81e3740 1 -- 172.21.15.1:0/495414448 shutdown_connections 2024-09-11T22:06:39.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.086+0000 7f36e81e3740 1 -- 172.21.15.1:0/495414448 wait complete. 2024-09-11T22:06:39.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.086+0000 7f36e81e3740 20 client.17703 trim_cache size 0 max 16384 2024-09-11T22:06:39.114 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-11T22:06:39.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.087+0000 7f36e5e0a640 1 -- 172.21.15.1:0/1095543043 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f36c80a2d70 msgr2=0x7f36c80a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:39.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.087+0000 7f36e5e0a640 1 --2- 172.21.15.1:0/1095543043 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f36c80a2d70 0x7f36c80a5230 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f36d8031550 tx=0x7f36d8005d50 comp rx=0 tx=0).stop 2024-09-11T22:06:39.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.087+0000 7f36e5e0a640 1 -- 172.21.15.1:0/1095543043 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f36e01036c0 msgr2=0x7f36e019ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:39.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.087+0000 7f36e5e0a640 1 --2- 172.21.15.1:0/1095543043 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f36e01036c0 0x7f36e019ab40 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f36d400e9f0 tx=0x7f36d400edf0 comp rx=0 tx=0).stop 2024-09-11T22:06:39.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.087+0000 7f36e5e0a640 1 -- 172.21.15.1:0/1095543043 shutdown_connections 2024-09-11T22:06:39.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.087+0000 7f36e5e0a640 1 --2- 172.21.15.1:0/1095543043 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f36c80a2d70 0x7f36c80a5230 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:39.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.087+0000 7f36e5e0a640 1 --2- 172.21.15.1:0/1095543043 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f36e010c180 0x7f36e019eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:39.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.087+0000 7f36e5e0a640 1 --2- 172.21.15.1:0/1095543043 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f36e01036c0 0x7f36e019ab40 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:39.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.087+0000 7f36e5e0a640 1 --2- 172.21.15.1:0/1095543043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f36e0102da0 0x7f36e019a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:39.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.087+0000 7f36e5e0a640 1 -- 172.21.15.1:0/1095543043 >> 172.21.15.1:0/1095543043 conn(0x7f36e00fc7a0 msgr2=0x7f36e0101b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:39.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.087+0000 7f36e5e0a640 1 -- 172.21.15.1:0/1095543043 shutdown_connections 2024-09-11T22:06:39.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:39.087+0000 7f36e5e0a640 1 -- 172.21.15.1:0/1095543043 wait complete. 2024-09-11T22:06:39.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:06:39 smithi161 ceph-mon[35615]: pgmap v293: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:06:39.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:06:39 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:06:39.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:06:39 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:06:39.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:06:39 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T22:06:39.393 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:06:39 smithi001 ceph-mon[30935]: pgmap v293: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:06:39.393 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:06:39 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:06:39.393 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:06:39 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:06:39.393 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:06:39 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T22:06:39.449 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:06:39 smithi053 ceph-mon[34908]: pgmap v293: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:06:39.449 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:06:39 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:06:39.449 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:06:39 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:06:39.449 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:06:39 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T22:06:40.819 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T22:06:40.819 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T22:06:41.091 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:06:41.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:06:41 smithi001 ceph-mon[30935]: pgmap v294: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 14 op/s 2024-09-11T22:06:41.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:06:41 smithi161 ceph-mon[35615]: pgmap v294: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 14 op/s 2024-09-11T22:06:41.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:06:41 smithi053 ceph-mon[34908]: pgmap v294: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 14 op/s 2024-09-11T22:06:41.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.593+0000 7ff055f20640 1 -- 172.21.15.1:0/3148284930 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff050103660 msgr2=0x7ff05010bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:41.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.593+0000 7ff055f20640 1 --2- 172.21.15.1:0/3148284930 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff050103660 0x7ff05010bb50 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7ff038009900 tx=0x7ff03802a480 comp rx=0 tx=0).stop 2024-09-11T22:06:41.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.593+0000 7ff055f20640 1 -- 172.21.15.1:0/3148284930 shutdown_connections 2024-09-11T22:06:41.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.593+0000 7ff055f20640 1 --2- 172.21.15.1:0/3148284930 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff05010c170 0x7ff05010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:41.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.593+0000 7ff055f20640 1 --2- 172.21.15.1:0/3148284930 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff050103660 0x7ff05010bb50 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:41.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.593+0000 7ff055f20640 1 --2- 172.21.15.1:0/3148284930 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff050102d40 0x7ff050103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:41.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.593+0000 7ff055f20640 1 -- 172.21.15.1:0/3148284930 >> 172.21.15.1:0/3148284930 conn(0x7ff0500fc7a0 msgr2=0x7ff0500febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:41.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.593+0000 7ff055f20640 1 -- 172.21.15.1:0/3148284930 shutdown_connections 2024-09-11T22:06:41.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.593+0000 7ff055f20640 1 -- 172.21.15.1:0/3148284930 wait complete. 2024-09-11T22:06:41.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.594+0000 7ff055f20640 1 Processor -- start 2024-09-11T22:06:41.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.594+0000 7ff055f20640 1 -- start start 2024-09-11T22:06:41.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.595+0000 7ff055f20640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff050102d40 0x7ff050198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:41.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.595+0000 7ff055f20640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff050103660 0x7ff050198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:41.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.595+0000 7ff055f20640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff05010c170 0x7ff05019cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:41.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.595+0000 7ff055f20640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff050111fb0 con 0x7ff05010c170 2024-09-11T22:06:41.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.595+0000 7ff055f20640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff050111e30 con 0x7ff050103660 2024-09-11T22:06:41.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.595+0000 7ff055f20640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff050112130 con 0x7ff050102d40 2024-09-11T22:06:41.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.595+0000 7ff04ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff05010c170 0x7ff05019cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:41.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.595+0000 7ff04ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff05010c170 0x7ff05019cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41994/0 (socket says 172.21.15.1:41994) 2024-09-11T22:06:41.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.595+0000 7ff04ffff640 1 -- 172.21.15.1:0/2874115684 learned_addr learned my addr 172.21.15.1:0/2874115684 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:06:41.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.595+0000 7ff04f7fe640 1 --2- 172.21.15.1:0/2874115684 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff050102d40 0x7ff050198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:41.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.595+0000 7ff046ffd640 1 --2- 172.21.15.1:0/2874115684 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff050103660 0x7ff050198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:41.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.595+0000 7ff04ffff640 1 -- 172.21.15.1:0/2874115684 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff050102d40 msgr2=0x7ff050198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:41.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.595+0000 7ff04ffff640 1 --2- 172.21.15.1:0/2874115684 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff050102d40 0x7ff050198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:41.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.595+0000 7ff04ffff640 1 -- 172.21.15.1:0/2874115684 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff050103660 msgr2=0x7ff050198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:41.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.595+0000 7ff04ffff640 1 --2- 172.21.15.1:0/2874115684 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff050103660 0x7ff050198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:41.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.595+0000 7ff04ffff640 1 -- 172.21.15.1:0/2874115684 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff05019d3c0 con 0x7ff05010c170 2024-09-11T22:06:41.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.595+0000 7ff046ffd640 1 --2- 172.21.15.1:0/2874115684 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff050103660 0x7ff050198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:06:41.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.595+0000 7ff04ffff640 1 --2- 172.21.15.1:0/2874115684 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff05010c170 0x7ff05019cce0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7ff04800b970 tx=0x7ff04800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:41.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.595+0000 7ff04d7fa640 1 -- 172.21.15.1:0/2874115684 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff048019070 con 0x7ff05010c170 2024-09-11T22:06:41.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.595+0000 7ff04d7fa640 1 -- 172.21.15.1:0/2874115684 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff048014440 con 0x7ff05010c170 2024-09-11T22:06:41.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.595+0000 7ff04d7fa640 1 -- 172.21.15.1:0/2874115684 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0480075d0 con 0x7ff05010c170 2024-09-11T22:06:41.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.595+0000 7ff055f20640 1 -- 172.21.15.1:0/2874115684 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff05019d6b0 con 0x7ff05010c170 2024-09-11T22:06:41.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.596+0000 7ff055f20640 1 -- 172.21.15.1:0/2874115684 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0501a4f90 con 0x7ff05010c170 2024-09-11T22:06:41.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.598+0000 7ff04d7fa640 1 -- 172.21.15.1:0/2874115684 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7ff04801c070 con 0x7ff05010c170 2024-09-11T22:06:41.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.598+0000 7ff0582f9740 1 Processor -- start 2024-09-11T22:06:41.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.598+0000 7ff0582f9740 1 -- start start 2024-09-11T22:06:41.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.599+0000 7ff0582f9740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56154cf27b60 0x56154cf27f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:41.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.599+0000 7ff0582f9740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56154cf28480 0x56154cf32800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:41.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.599+0000 7ff0582f9740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56154cf32d40 0x56154cf35130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:41.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.599+0000 7ff0582f9740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56154ce8d280 con 0x56154cf28480 2024-09-11T22:06:41.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.599+0000 7ff0582f9740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56154ce34b10 con 0x56154cf32d40 2024-09-11T22:06:41.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.599+0000 7ff0582f9740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56154ce89240 con 0x56154cf27b60 2024-09-11T22:06:41.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.599+0000 7ff04f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56154cf28480 0x56154cf32800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:41.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.599+0000 7ff046ffd640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56154cf32d40 0x56154cf35130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:41.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.599+0000 7ff04f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56154cf28480 0x56154cf32800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42008/0 (socket says 172.21.15.1:42008) 2024-09-11T22:06:41.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.599+0000 7ff04f7fe640 1 -- 172.21.15.1:0/765543722 learned_addr learned my addr 172.21.15.1:0/765543722 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:06:41.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.599+0000 7ff046ffd640 1 --2- 172.21.15.1:0/765543722 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56154cf27b60 0x56154cf27f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:41.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.600+0000 7ff04f7fe640 1 -- 172.21.15.1:0/765543722 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56154cf27b60 msgr2=0x56154cf27f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:41.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.600+0000 7ff04f7fe640 1 --2- 172.21.15.1:0/765543722 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56154cf27b60 0x56154cf27f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:41.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.600+0000 7ff04f7fe640 1 -- 172.21.15.1:0/765543722 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56154cf32d40 msgr2=0x56154cf35130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:41.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.600+0000 7ff04f7fe640 1 --2- 172.21.15.1:0/765543722 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56154cf32d40 0x56154cf35130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:41.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.600+0000 7ff04f7fe640 1 -- 172.21.15.1:0/765543722 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56154ce411f0 con 0x56154cf28480 2024-09-11T22:06:41.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.600+0000 7ff04d7fa640 1 --2- 172.21.15.1:0/2874115684 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7ff03c0a2cf0 0x7ff03c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:41.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.600+0000 7ff04f7fe640 1 --2- 172.21.15.1:0/765543722 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56154cf28480 0x56154cf32800 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7ff040011d80 tx=0x7ff04000aa20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29d234b827be1362 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:41.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.600+0000 7ff046ffd640 1 --2- 172.21.15.1:0/2874115684 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7ff03c0a2cf0 0x7ff03c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:41.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.600+0000 7ff04d7fa640 1 -- 172.21.15.1:0/2874115684 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7ff0480ce970 con 0x7ff05010c170 2024-09-11T22:06:41.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.600+0000 7ff045ffb640 1 -- 172.21.15.1:0/765543722 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff04001f070 con 0x56154cf28480 2024-09-11T22:06:41.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.600+0000 7ff045ffb640 1 -- 172.21.15.1:0/765543722 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff04000cb40 con 0x56154cf28480 2024-09-11T22:06:41.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.600+0000 7ff045ffb640 1 -- 172.21.15.1:0/765543722 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff04001ac20 con 0x56154cf28480 2024-09-11T22:06:41.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.601+0000 7ff0582f9740 1 -- 172.21.15.1:0/765543722 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56154cf28480 msgr2=0x56154cf32800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:41.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.601+0000 7ff046ffd640 1 --2- 172.21.15.1:0/2874115684 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7ff03c0a2cf0 0x7ff03c0a51b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff05010d4e0 tx=0x7ff03802a480 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:41.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.601+0000 7ff0582f9740 1 --2- 172.21.15.1:0/765543722 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56154cf28480 0x56154cf32800 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7ff040011d80 tx=0x7ff04000aa20 comp rx=0 tx=0).stop 2024-09-11T22:06:41.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.601+0000 7ff0582f9740 1 -- 172.21.15.1:0/765543722 shutdown_connections 2024-09-11T22:06:41.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.601+0000 7ff0582f9740 1 --2- 172.21.15.1:0/765543722 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56154cf32d40 0x56154cf35130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:41.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.601+0000 7ff0582f9740 1 --2- 172.21.15.1:0/765543722 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56154cf28480 0x56154cf32800 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:41.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.601+0000 7ff0582f9740 1 --2- 172.21.15.1:0/765543722 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56154cf27b60 0x56154cf27f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:41.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.601+0000 7ff0582f9740 1 -- 172.21.15.1:0/765543722 >> 172.21.15.1:0/765543722 conn(0x56154cf240f0 msgr2=0x56154cf244d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:41.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.601+0000 7ff0582f9740 1 -- 172.21.15.1:0/765543722 shutdown_connections 2024-09-11T22:06:41.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.601+0000 7ff0582f9740 1 -- 172.21.15.1:0/765543722 wait complete. 2024-09-11T22:06:41.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.601+0000 7ff0582f9740 1 -- start start 2024-09-11T22:06:41.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.602+0000 7ff0582f9740 1 Processor -- start 2024-09-11T22:06:41.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.602+0000 7ff0582f9740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56154cf27b60 0x56154cf2fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:41.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.602+0000 7ff0582f9740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56154cf28480 0x56154cf30180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:41.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.602+0000 7ff0582f9740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56154cf32d40 0x56154cf306c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:41.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.602+0000 7ff0582f9740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56154cf46920 con 0x56154cf27b60 2024-09-11T22:06:41.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.602+0000 7ff0582f9740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x56154ce47f10 con 0x56154cf28480 2024-09-11T22:06:41.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.602+0000 7ff0582f9740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x56154cf46aa0 con 0x56154cf32d40 2024-09-11T22:06:41.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.602+0000 7ff0582f9740 10 client.0 _ll_register_callbacks cb 0x56154ce10a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:06:41.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.602+0000 7ff04f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56154cf27b60 0x56154cf2fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:41.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.602+0000 7ff04f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56154cf27b60 0x56154cf2fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42010/0 (socket says 172.21.15.1:42010) 2024-09-11T22:06:41.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.602+0000 7ff04f7fe640 1 -- 172.21.15.1:0/1766595638 learned_addr learned my addr 172.21.15.1:0/1766595638 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:06:41.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.602+0000 7ff04ffff640 1 --2- 172.21.15.1:0/1766595638 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56154cf32d40 0x56154cf306c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:41.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.602+0000 7ff046ffd640 1 --2- 172.21.15.1:0/1766595638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56154cf28480 0x56154cf30180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:41.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.602+0000 7ff04f7fe640 1 -- 172.21.15.1:0/1766595638 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56154cf32d40 msgr2=0x56154cf306c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:41.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.602+0000 7ff04f7fe640 1 --2- 172.21.15.1:0/1766595638 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56154cf32d40 0x56154cf306c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:41.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.603+0000 7ff04f7fe640 1 -- 172.21.15.1:0/1766595638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56154cf28480 msgr2=0x56154cf30180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:41.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.603+0000 7ff04f7fe640 1 --2- 172.21.15.1:0/1766595638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56154cf28480 0x56154cf30180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:41.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.603+0000 7ff04f7fe640 1 -- 172.21.15.1:0/1766595638 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56154cbfe6a0 con 0x56154cf27b60 2024-09-11T22:06:41.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.603+0000 7ff046ffd640 1 --2- 172.21.15.1:0/1766595638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56154cf28480 0x56154cf30180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:06:41.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.603+0000 7ff04f7fe640 1 --2- 172.21.15.1:0/1766595638 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56154cf27b60 0x56154cf2fc40 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7ff04001d040 tx=0x7ff04001a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e9cf362c9ebbd53 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:41.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.603+0000 7ff02effd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:06:41.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.603+0000 7ff02effd640 1 -- 172.21.15.1:0/1766595638 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff04001f040 con 0x56154cf27b60 2024-09-11T22:06:41.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.603+0000 7ff0582f9740 1 -- 172.21.15.1:0/1766595638 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff04000cb40 con 0x56154cf27b60 2024-09-11T22:06:41.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.603+0000 7ff02effd640 1 -- 172.21.15.1:0/1766595638 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7ff04000c660 con 0x56154cf27b60 2024-09-11T22:06:41.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.603+0000 7ff02effd640 1 -- 172.21.15.1:0/1766595638 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7ff040031050 con 0x56154cf27b60 2024-09-11T22:06:41.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.603+0000 7ff0582f9740 10 client.17715 fetch_fsmap learned FSMap version 46 2024-09-11T22:06:41.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.603+0000 7ff0582f9740 1 -- 172.21.15.1:0/1766595638 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x56154cf52550 con 0x56154cf27b60 2024-09-11T22:06:41.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.603+0000 7ff02effd640 1 -- 172.21.15.1:0/1766595638 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff04000cb40 con 0x56154cf27b60 2024-09-11T22:06:41.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.603+0000 7ff02effd640 1 -- 172.21.15.1:0/1766595638 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 46) ==== 8442+0+0 (secure 0 0 0) 0x7ff040022070 con 0x56154cf27b60 2024-09-11T22:06:41.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.604+0000 7ff0582f9740 10 client.17715 fetch_fsmap finished waiting for FSMap version 46 2024-09-11T22:06:41.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.604+0000 7ff0582f9740 10 client.17715 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:06:41.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.604+0000 7ff0582f9740 20 client.17715 populate_metadata read hostname 'smithi001' 2024-09-11T22:06:41.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.604+0000 7ff0582f9740 1 --2- 172.21.15.1:0/1766595638 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x56154cf539b0 0x56154cf55da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:41.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.604+0000 7ff0582f9740 4 client.17715 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:06:41.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.604+0000 7ff0582f9740 1 -- 172.21.15.1:0/1766595638 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56154ce9a7f0 con 0x56154cf539b0 2024-09-11T22:06:41.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.604+0000 7ff046ffd640 1 --2- 172.21.15.1:0/1766595638 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x56154cf539b0 0x56154cf55da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:41.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.604+0000 7ff02effd640 10 client.17715 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:06:41.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.604+0000 7ff046ffd640 1 --2- 172.21.15.1:0/1766595638 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x56154cf539b0 0x56154cf55da0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d986c986da3ca5e server_cookie=51fd6c29619bdb32 in_seq=0 out_seq=0 2024-09-11T22:06:41.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.605+0000 7ff02effd640 1 -- 172.21.15.1:0/1766595638 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56154ce9a7f0 con 0x56154cf539b0 2024-09-11T22:06:41.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.605+0000 7ff02effd640 10 client.17715 handle_command_reply: tid=0 2024-09-11T22:06:41.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.605+0000 7ff0582f9740 1 client.17715 shutdown 2024-09-11T22:06:41.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.605+0000 7ff0582f9740 1 -- 172.21.15.1:0/1766595638 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56154cf27b60 msgr2=0x56154cf2fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:41.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.605+0000 7ff0582f9740 1 --2- 172.21.15.1:0/1766595638 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56154cf27b60 0x56154cf2fc40 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7ff04001d040 tx=0x7ff04001a640 comp rx=0 tx=0).stop 2024-09-11T22:06:41.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.605+0000 7ff0582f9740 1 -- 172.21.15.1:0/1766595638 shutdown_connections 2024-09-11T22:06:41.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.605+0000 7ff0582f9740 1 --2- 172.21.15.1:0/1766595638 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x56154cf539b0 0x56154cf55da0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:41.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.605+0000 7ff0582f9740 1 --2- 172.21.15.1:0/1766595638 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56154cf32d40 0x56154cf306c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:41.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.605+0000 7ff0582f9740 1 --2- 172.21.15.1:0/1766595638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56154cf28480 0x56154cf30180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:41.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.605+0000 7ff02effd640 0 client.17715 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:06:41.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.605+0000 7ff0582f9740 1 --2- 172.21.15.1:0/1766595638 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56154cf27b60 0x56154cf2fc40 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:41.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.605+0000 7ff0582f9740 1 -- 172.21.15.1:0/1766595638 >> 172.21.15.1:0/1766595638 conn(0x56154cf240f0 msgr2=0x56154cf23270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:41.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.605+0000 7ff0582f9740 1 -- 172.21.15.1:0/1766595638 shutdown_connections 2024-09-11T22:06:41.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.606+0000 7ff0582f9740 1 -- 172.21.15.1:0/1766595638 wait complete. 2024-09-11T22:06:41.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.606+0000 7ff0582f9740 20 client.17715 trim_cache size 0 max 16384 2024-09-11T22:06:41.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.623+0000 7ff0582f9740 1 Processor -- start 2024-09-11T22:06:41.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.623+0000 7ff0582f9740 1 -- start start 2024-09-11T22:06:41.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.623+0000 7ff0582f9740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56154cf27b60 0x56154cf5b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:41.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.623+0000 7ff0582f9740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56154cf28480 0x56154cf5ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:41.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.623+0000 7ff0582f9740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56154cf32d40 0x56154cf4bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:41.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.623+0000 7ff0582f9740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff0500722c0 con 0x56154cf28480 2024-09-11T22:06:41.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.623+0000 7ff0582f9740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff050074100 con 0x56154cf32d40 2024-09-11T22:06:41.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.623+0000 7ff0582f9740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff050072a50 con 0x56154cf27b60 2024-09-11T22:06:41.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.623+0000 7ff046ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56154cf28480 0x56154cf5ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:41.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.623+0000 7ff046ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56154cf28480 0x56154cf5ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42012/0 (socket says 172.21.15.1:42012) 2024-09-11T22:06:41.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.623+0000 7ff046ffd640 1 -- 172.21.15.1:0/1563625035 learned_addr learned my addr 172.21.15.1:0/1563625035 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:06:41.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.623+0000 7ff04f7fe640 1 --2- 172.21.15.1:0/1563625035 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56154cf27b60 0x56154cf5b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:41.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.624+0000 7ff046ffd640 1 -- 172.21.15.1:0/1563625035 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56154cf27b60 msgr2=0x56154cf5b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:41.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.624+0000 7ff046ffd640 1 --2- 172.21.15.1:0/1563625035 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56154cf27b60 0x56154cf5b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:41.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.624+0000 7ff046ffd640 1 -- 172.21.15.1:0/1563625035 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56154cf32d40 msgr2=0x56154cf4bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:41.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.624+0000 7ff046ffd640 1 --2- 172.21.15.1:0/1563625035 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56154cf32d40 0x56154cf4bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:41.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.624+0000 7ff046ffd640 1 -- 172.21.15.1:0/1563625035 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56154cf278e0 con 0x56154cf28480 2024-09-11T22:06:41.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.624+0000 7ff046ffd640 1 --2- 172.21.15.1:0/1563625035 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56154cf28480 0x56154cf5ba00 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7ff038009900 tx=0x7ff03803f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a63a9ec70c347239 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:41.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.624+0000 7ff0457fa640 1 -- 172.21.15.1:0/1563625035 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff038035040 con 0x56154cf28480 2024-09-11T22:06:41.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.624+0000 7ff0457fa640 1 -- 172.21.15.1:0/1563625035 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff038059070 con 0x56154cf28480 2024-09-11T22:06:41.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.624+0000 7ff0457fa640 1 -- 172.21.15.1:0/1563625035 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff03803d450 con 0x56154cf28480 2024-09-11T22:06:41.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.625+0000 7ff0582f9740 1 -- 172.21.15.1:0/1563625035 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56154cf28480 msgr2=0x56154cf5ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:41.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.625+0000 7ff0582f9740 1 --2- 172.21.15.1:0/1563625035 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56154cf28480 0x56154cf5ba00 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7ff038009900 tx=0x7ff03803f000 comp rx=0 tx=0).stop 2024-09-11T22:06:41.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.625+0000 7ff0582f9740 1 -- 172.21.15.1:0/1563625035 shutdown_connections 2024-09-11T22:06:41.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.625+0000 7ff0582f9740 1 --2- 172.21.15.1:0/1563625035 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56154cf32d40 0x56154cf4bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:41.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.625+0000 7ff0582f9740 1 --2- 172.21.15.1:0/1563625035 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56154cf28480 0x56154cf5ba00 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:41.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.625+0000 7ff0582f9740 1 --2- 172.21.15.1:0/1563625035 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56154cf27b60 0x56154cf5b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:41.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.625+0000 7ff0582f9740 1 -- 172.21.15.1:0/1563625035 >> 172.21.15.1:0/1563625035 conn(0x56154cf240f0 msgr2=0x56154cf31bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:41.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.625+0000 7ff0582f9740 1 -- 172.21.15.1:0/1563625035 shutdown_connections 2024-09-11T22:06:41.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.625+0000 7ff0582f9740 1 -- 172.21.15.1:0/1563625035 wait complete. 2024-09-11T22:06:41.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.625+0000 7ff0582f9740 1 -- start start 2024-09-11T22:06:41.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.625+0000 7ff0582f9740 1 Processor -- start 2024-09-11T22:06:41.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.626+0000 7ff0582f9740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56154cf27b60 0x56154cf4c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:41.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.626+0000 7ff0582f9740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56154cf28480 0x56154cf49700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:41.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.626+0000 7ff0582f9740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56154cf32d40 0x56154cf49c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:41.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.626+0000 7ff0582f9740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff050073970 con 0x56154cf27b60 2024-09-11T22:06:41.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.626+0000 7ff0582f9740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ff0500731e0 con 0x56154cf32d40 2024-09-11T22:06:41.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.626+0000 7ff0582f9740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ff0500702a0 con 0x56154cf28480 2024-09-11T22:06:41.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.626+0000 7ff0582f9740 10 client.0 _ll_register_callbacks cb 0x7ff04001d070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:06:41.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.626+0000 7ff04f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56154cf27b60 0x56154cf4c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:41.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.626+0000 7ff04f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56154cf27b60 0x56154cf4c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42024/0 (socket says 172.21.15.1:42024) 2024-09-11T22:06:41.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.626+0000 7ff04f7fe640 1 -- 172.21.15.1:0/62954076 learned_addr learned my addr 172.21.15.1:0/62954076 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:06:41.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.626+0000 7ff046ffd640 1 --2- 172.21.15.1:0/62954076 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56154cf28480 0x56154cf49700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:41.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.626+0000 7ff04ffff640 1 --2- 172.21.15.1:0/62954076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56154cf32d40 0x56154cf49c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:41.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.626+0000 7ff04f7fe640 1 -- 172.21.15.1:0/62954076 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56154cf28480 msgr2=0x56154cf49700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:41.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.626+0000 7ff04f7fe640 1 --2- 172.21.15.1:0/62954076 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56154cf28480 0x56154cf49700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:41.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.626+0000 7ff04f7fe640 1 -- 172.21.15.1:0/62954076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56154cf32d40 msgr2=0x56154cf49c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:41.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.626+0000 7ff04f7fe640 1 --2- 172.21.15.1:0/62954076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56154cf32d40 0x56154cf49c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:41.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.626+0000 7ff04f7fe640 1 -- 172.21.15.1:0/62954076 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56154cf4a450 con 0x56154cf27b60 2024-09-11T22:06:41.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.626+0000 7ff04f7fe640 1 --2- 172.21.15.1:0/62954076 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56154cf27b60 0x56154cf4c3b0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7ff0400055e0 tx=0x7ff040011d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b87bd7f0334f4772 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:41.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.626+0000 7ff02effd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:06:41.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.627+0000 7ff02effd640 1 -- 172.21.15.1:0/62954076 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff04001f040 con 0x56154cf27b60 2024-09-11T22:06:41.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.627+0000 7ff02effd640 1 -- 172.21.15.1:0/62954076 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7ff04000c660 con 0x56154cf27b60 2024-09-11T22:06:41.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.627+0000 7ff0582f9740 1 -- 172.21.15.1:0/62954076 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff038059070 con 0x56154cf27b60 2024-09-11T22:06:41.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.627+0000 7ff02effd640 1 -- 172.21.15.1:0/62954076 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7ff040043050 con 0x56154cf27b60 2024-09-11T22:06:41.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.627+0000 7ff0582f9740 10 client.17721 fetch_fsmap learned FSMap version 46 2024-09-11T22:06:41.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.627+0000 7ff0582f9740 1 -- 172.21.15.1:0/62954076 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x56154cfbe450 con 0x56154cf27b60 2024-09-11T22:06:41.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.627+0000 7ff02effd640 1 -- 172.21.15.1:0/62954076 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff038059070 con 0x56154cf27b60 2024-09-11T22:06:41.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.627+0000 7ff02effd640 1 -- 172.21.15.1:0/62954076 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 46) ==== 8442+0+0 (secure 0 0 0) 0x7ff040022050 con 0x56154cf27b60 2024-09-11T22:06:41.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.627+0000 7ff0582f9740 10 client.17721 fetch_fsmap finished waiting for FSMap version 46 2024-09-11T22:06:41.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.627+0000 7ff0582f9740 10 client.17721 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:06:41.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.627+0000 7ff0582f9740 20 client.17721 populate_metadata read hostname 'smithi001' 2024-09-11T22:06:41.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.627+0000 7ff0582f9740 1 --2- 172.21.15.1:0/62954076 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x56154cfbebc0 0x56154cfc0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:41.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.627+0000 7ff0582f9740 4 client.17721 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T22:06:41.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.627+0000 7ff0582f9740 1 -- 172.21.15.1:0/62954076 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56154ce94d00 con 0x56154cfbebc0 2024-09-11T22:06:41.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.627+0000 7ff046ffd640 1 --2- 172.21.15.1:0/62954076 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x56154cfbebc0 0x56154cfc0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:41.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.628+0000 7ff046ffd640 1 --2- 172.21.15.1:0/62954076 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x56154cfbebc0 0x56154cfc0fe0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec2e84ee137bfb32 server_cookie=258a230789dba079 in_seq=0 out_seq=0 2024-09-11T22:06:41.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.628+0000 7ff02effd640 10 client.17721 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:06:41.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.629+0000 7ff02effd640 1 -- 172.21.15.1:0/62954076 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56154ce94d00 con 0x56154cfbebc0 2024-09-11T22:06:41.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.629+0000 7ff02effd640 10 client.17721 handle_command_reply: tid=0 2024-09-11T22:06:41.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.629+0000 7ff0582f9740 1 client.17721 shutdown 2024-09-11T22:06:41.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.629+0000 7ff0582f9740 1 -- 172.21.15.1:0/62954076 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56154cf27b60 msgr2=0x56154cf4c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:41.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.629+0000 7ff0582f9740 1 --2- 172.21.15.1:0/62954076 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56154cf27b60 0x56154cf4c3b0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7ff0400055e0 tx=0x7ff040011d00 comp rx=0 tx=0).stop 2024-09-11T22:06:41.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.629+0000 7ff0582f9740 1 -- 172.21.15.1:0/62954076 shutdown_connections 2024-09-11T22:06:41.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.629+0000 7ff0582f9740 1 --2- 172.21.15.1:0/62954076 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x56154cfbebc0 0x56154cfc0fe0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:41.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.629+0000 7ff0582f9740 1 --2- 172.21.15.1:0/62954076 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56154cf32d40 0x56154cf49c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:41.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.629+0000 7ff0582f9740 1 --2- 172.21.15.1:0/62954076 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56154cf28480 0x56154cf49700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:41.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.629+0000 7ff0582f9740 1 --2- 172.21.15.1:0/62954076 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56154cf27b60 0x56154cf4c3b0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:41.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.629+0000 7ff0582f9740 1 -- 172.21.15.1:0/62954076 >> 172.21.15.1:0/62954076 conn(0x56154cf240f0 msgr2=0x56154cf22a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:41.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.629+0000 7ff02effd640 0 client.17721 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:06:41.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.630+0000 7ff0582f9740 1 -- 172.21.15.1:0/62954076 shutdown_connections 2024-09-11T22:06:41.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.630+0000 7ff0582f9740 1 -- 172.21.15.1:0/62954076 wait complete. 2024-09-11T22:06:41.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.630+0000 7ff0582f9740 20 client.17721 trim_cache size 0 max 16384 2024-09-11T22:06:41.639 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-11T22:06:41.640 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-09-11T22:06:41.640 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "cf120c2a-3202-4382-943e-41da97a17bf6", 2024-09-11T22:06:41.640 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-09-11T22:06:41.640 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-11T22:06:41.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.630+0000 7ff055f20640 1 -- 172.21.15.1:0/2874115684 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7ff03c0a2cf0 msgr2=0x7ff03c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:41.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.630+0000 7ff055f20640 1 --2- 172.21.15.1:0/2874115684 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7ff03c0a2cf0 0x7ff03c0a51b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff05010d4e0 tx=0x7ff03802a480 comp rx=0 tx=0).stop 2024-09-11T22:06:41.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.630+0000 7ff055f20640 1 -- 172.21.15.1:0/2874115684 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff05010c170 msgr2=0x7ff05019cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:41.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.630+0000 7ff055f20640 1 --2- 172.21.15.1:0/2874115684 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff05010c170 0x7ff05019cce0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7ff04800b970 tx=0x7ff04800bd70 comp rx=0 tx=0).stop 2024-09-11T22:06:41.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.630+0000 7ff055f20640 1 -- 172.21.15.1:0/2874115684 shutdown_connections 2024-09-11T22:06:41.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.630+0000 7ff055f20640 1 --2- 172.21.15.1:0/2874115684 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7ff03c0a2cf0 0x7ff03c0a51b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:41.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.631+0000 7ff055f20640 1 --2- 172.21.15.1:0/2874115684 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff05010c170 0x7ff05019cce0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:41.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.631+0000 7ff055f20640 1 --2- 172.21.15.1:0/2874115684 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff050103660 0x7ff050198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:41.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.631+0000 7ff055f20640 1 --2- 172.21.15.1:0/2874115684 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff050102d40 0x7ff050198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:41.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.631+0000 7ff055f20640 1 -- 172.21.15.1:0/2874115684 >> 172.21.15.1:0/2874115684 conn(0x7ff0500fc7a0 msgr2=0x7ff0500fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:41.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.631+0000 7ff055f20640 1 -- 172.21.15.1:0/2874115684 shutdown_connections 2024-09-11T22:06:41.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:41.631+0000 7ff055f20640 1 -- 172.21.15.1:0/2874115684 wait complete. 2024-09-11T22:06:42.474 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:06:42.736 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:06:43.152 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:06:43 smithi001 ceph-mon[30935]: scrub queued for path: / 2024-09-11T22:06:43.153 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:06:43 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:06:43.153 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:06:43 smithi001 ceph-mon[30935]: scrub summary: active paths [/] 2024-09-11T22:06:43.153 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:06:43 smithi001 ceph-mon[30935]: scrub summary: idle+waiting paths [/] 2024-09-11T22:06:43.153 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:06:43 smithi001 ceph-mon[30935]: pgmap v295: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:06:43.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.234+0000 7fb015e40640 1 -- 172.21.15.1:0/783644058 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb010074f40 msgr2=0x7fb01010de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:43.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.234+0000 7fb015e40640 1 --2- 172.21.15.1:0/783644058 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb010074f40 0x7fb01010de80 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7faff8009900 tx=0x7faff802a480 comp rx=0 tx=0).stop 2024-09-11T22:06:43.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.234+0000 7fb015e40640 1 -- 172.21.15.1:0/783644058 shutdown_connections 2024-09-11T22:06:43.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.234+0000 7fb015e40640 1 --2- 172.21.15.1:0/783644058 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb01010e3c0 0x7fb010110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:43.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.234+0000 7fb015e40640 1 --2- 172.21.15.1:0/783644058 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb010074f40 0x7fb01010de80 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:43.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.234+0000 7fb015e40640 1 --2- 172.21.15.1:0/783644058 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb010074620 0x7fb010074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:43.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.234+0000 7fb015e40640 1 -- 172.21.15.1:0/783644058 >> 172.21.15.1:0/783644058 conn(0x7fb0100fd8c0 msgr2=0x7fb0100ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:43.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.234+0000 7fb015e40640 1 -- 172.21.15.1:0/783644058 shutdown_connections 2024-09-11T22:06:43.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.234+0000 7fb015e40640 1 -- 172.21.15.1:0/783644058 wait complete. 2024-09-11T22:06:43.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.234+0000 7fb015e40640 1 Processor -- start 2024-09-11T22:06:43.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.234+0000 7fb015e40640 1 -- start start 2024-09-11T22:06:43.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.235+0000 7fb015e40640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb010074620 0x7fb01019a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:43.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.235+0000 7fb015e40640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb010074f40 0x7fb01019ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:43.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.235+0000 7fb015e40640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb01010e3c0 0x7fb01019ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:43.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.235+0000 7fb015e40640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb010114140 con 0x7fb010074f40 2024-09-11T22:06:43.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.235+0000 7fb015e40640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb010113fc0 con 0x7fb010074620 2024-09-11T22:06:43.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.235+0000 7fb015e40640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb0101142c0 con 0x7fb01010e3c0 2024-09-11T22:06:43.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.235+0000 7fb006ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb010074f40 0x7fb01019ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:43.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.235+0000 7fb006ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb010074f40 0x7fb01019ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42050/0 (socket says 172.21.15.1:42050) 2024-09-11T22:06:43.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.235+0000 7fb006ffd640 1 -- 172.21.15.1:0/2953348544 learned_addr learned my addr 172.21.15.1:0/2953348544 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:06:43.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.235+0000 7fb00f7fe640 1 --2- 172.21.15.1:0/2953348544 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb010074620 0x7fb01019a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:43.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.235+0000 7fb00ffff640 1 --2- 172.21.15.1:0/2953348544 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb01010e3c0 0x7fb01019ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:43.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.235+0000 7fb006ffd640 1 -- 172.21.15.1:0/2953348544 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb01010e3c0 msgr2=0x7fb01019ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:43.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.235+0000 7fb006ffd640 1 --2- 172.21.15.1:0/2953348544 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb01010e3c0 0x7fb01019ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:43.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.235+0000 7fb006ffd640 1 -- 172.21.15.1:0/2953348544 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb010074620 msgr2=0x7fb01019a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:43.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.235+0000 7fb006ffd640 1 --2- 172.21.15.1:0/2953348544 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb010074620 0x7fb01019a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:43.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.235+0000 7fb006ffd640 1 -- 172.21.15.1:0/2953348544 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb01019f5f0 con 0x7fb010074f40 2024-09-11T22:06:43.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.235+0000 7fb00ffff640 1 --2- 172.21.15.1:0/2953348544 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb01010e3c0 0x7fb01019ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:06:43.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.236+0000 7fb006ffd640 1 --2- 172.21.15.1:0/2953348544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb010074f40 0x7fb01019ab80 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7faff8006010 tx=0x7faff80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:43.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.236+0000 7fb00d7fa640 1 -- 172.21.15.1:0/2953348544 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff803f030 con 0x7fb010074f40 2024-09-11T22:06:43.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.236+0000 7fb015e40640 1 -- 172.21.15.1:0/2953348544 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb01019f880 con 0x7fb010074f40 2024-09-11T22:06:43.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.236+0000 7fb00d7fa640 1 -- 172.21.15.1:0/2953348544 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faff8003ce0 con 0x7fb010074f40 2024-09-11T22:06:43.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.236+0000 7fb00d7fa640 1 -- 172.21.15.1:0/2953348544 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff8035070 con 0x7fb010074f40 2024-09-11T22:06:43.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.236+0000 7fb015e40640 1 -- 172.21.15.1:0/2953348544 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0101a71c0 con 0x7fb010074f40 2024-09-11T22:06:43.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.238+0000 7fb00d7fa640 1 -- 172.21.15.1:0/2953348544 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7faff8042030 con 0x7fb010074f40 2024-09-11T22:06:43.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.238+0000 7fb018219740 1 Processor -- start 2024-09-11T22:06:43.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.238+0000 7fb018219740 1 -- start start 2024-09-11T22:06:43.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.239+0000 7fb018219740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6e23044a0 0x55f6e2304880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:43.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.239+0000 7fb018219740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6e2304dc0 0x55f6e230f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:43.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.239+0000 7fb018219740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6e230f680 0x55f6e2311a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:43.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.239+0000 7fb018219740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55f6e226a2e0 con 0x55f6e2304dc0 2024-09-11T22:06:43.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.239+0000 7fb018219740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55f6e2211ce0 con 0x55f6e230f680 2024-09-11T22:06:43.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.239+0000 7fb018219740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f6e22662a0 con 0x55f6e23044a0 2024-09-11T22:06:43.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.239+0000 7fb00f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6e2304dc0 0x55f6e230f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:43.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.239+0000 7fb00f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6e2304dc0 0x55f6e230f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42064/0 (socket says 172.21.15.1:42064) 2024-09-11T22:06:43.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.239+0000 7fb00f7fe640 1 -- 172.21.15.1:0/629327670 learned_addr learned my addr 172.21.15.1:0/629327670 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:06:43.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.239+0000 7fb00ffff640 1 --2- 172.21.15.1:0/629327670 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6e23044a0 0x55f6e2304880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:43.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.239+0000 7fb006ffd640 1 --2- 172.21.15.1:0/629327670 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6e230f680 0x55f6e2311a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:43.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.239+0000 7fb00d7fa640 1 --2- 172.21.15.1:0/2953348544 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7faffc0a2cf0 0x7faffc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:43.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.239+0000 7fb00f7fe640 1 -- 172.21.15.1:0/629327670 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6e23044a0 msgr2=0x55f6e2304880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:43.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.239+0000 7fb00f7fe640 1 --2- 172.21.15.1:0/629327670 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6e23044a0 0x55f6e2304880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:43.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.239+0000 7fb00f7fe640 1 -- 172.21.15.1:0/629327670 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6e230f680 msgr2=0x55f6e2311a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:43.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.239+0000 7fb00f7fe640 1 --2- 172.21.15.1:0/629327670 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6e230f680 0x55f6e2311a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:43.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.239+0000 7fb00f7fe640 1 -- 172.21.15.1:0/629327670 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6e21f6440 con 0x55f6e2304dc0 2024-09-11T22:06:43.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.239+0000 7fb006ffd640 1 --2- 172.21.15.1:0/629327670 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6e230f680 0x55f6e2311a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:06:43.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.239+0000 7fb00ffff640 1 --2- 172.21.15.1:0/2953348544 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7faffc0a2cf0 0x7faffc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:43.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.239+0000 7fb00d7fa640 1 -- 172.21.15.1:0/2953348544 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7faff80efb50 con 0x7fb010074f40 2024-09-11T22:06:43.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.239+0000 7fb00f7fe640 1 --2- 172.21.15.1:0/629327670 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6e2304dc0 0x55f6e230f140 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fb00000bd10 tx=0x7fb0000107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b39b46cf60216be7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:43.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.240+0000 7fb005ffb640 1 -- 172.21.15.1:0/629327670 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb000004030 con 0x55f6e2304dc0 2024-09-11T22:06:43.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.240+0000 7fb005ffb640 1 -- 172.21.15.1:0/629327670 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0000041d0 con 0x55f6e2304dc0 2024-09-11T22:06:43.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.240+0000 7fb005ffb640 1 -- 172.21.15.1:0/629327670 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0000117e0 con 0x55f6e2304dc0 2024-09-11T22:06:43.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.240+0000 7fb018219740 1 -- 172.21.15.1:0/629327670 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6e2304dc0 msgr2=0x55f6e230f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:43.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.240+0000 7fb00ffff640 1 --2- 172.21.15.1:0/2953348544 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7faffc0a2cf0 0x7faffc0a51b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb008005fd0 tx=0x7fb008004a40 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:43.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.240+0000 7fb018219740 1 --2- 172.21.15.1:0/629327670 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6e2304dc0 0x55f6e230f140 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fb00000bd10 tx=0x7fb0000107a0 comp rx=0 tx=0).stop 2024-09-11T22:06:43.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.240+0000 7fb018219740 1 -- 172.21.15.1:0/629327670 shutdown_connections 2024-09-11T22:06:43.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.240+0000 7fb018219740 1 --2- 172.21.15.1:0/629327670 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6e230f680 0x55f6e2311a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:43.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.240+0000 7fb018219740 1 --2- 172.21.15.1:0/629327670 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6e2304dc0 0x55f6e230f140 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:43.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.240+0000 7fb018219740 1 --2- 172.21.15.1:0/629327670 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6e23044a0 0x55f6e2304880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:43.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.240+0000 7fb018219740 1 -- 172.21.15.1:0/629327670 >> 172.21.15.1:0/629327670 conn(0x55f6e2300a30 msgr2=0x55f6e2300e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:43.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.240+0000 7fb018219740 1 -- 172.21.15.1:0/629327670 shutdown_connections 2024-09-11T22:06:43.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.240+0000 7fb018219740 1 -- 172.21.15.1:0/629327670 wait complete. 2024-09-11T22:06:43.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.241+0000 7fb018219740 1 -- start start 2024-09-11T22:06:43.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.241+0000 7fb018219740 1 Processor -- start 2024-09-11T22:06:43.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.241+0000 7fb018219740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6e23044a0 0x55f6e230efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:43.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.241+0000 7fb018219740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6e2304dc0 0x55f6e230c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:43.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.241+0000 7fb018219740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6e230f680 0x55f6e230c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:43.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.241+0000 7fb018219740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55f6e2312250 con 0x55f6e230f680 2024-09-11T22:06:43.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.241+0000 7fb018219740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55f6e2225050 con 0x55f6e2304dc0 2024-09-11T22:06:43.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.241+0000 7fb018219740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f6e23123d0 con 0x55f6e23044a0 2024-09-11T22:06:43.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.241+0000 7fb018219740 10 client.0 _ll_register_callbacks cb 0x55f6e2265160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:06:43.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.241+0000 7fb00ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6e230f680 0x55f6e230c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:43.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.241+0000 7fb00ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6e230f680 0x55f6e230c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42078/0 (socket says 172.21.15.1:42078) 2024-09-11T22:06:43.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.241+0000 7fb00ffff640 1 -- 172.21.15.1:0/72435393 learned_addr learned my addr 172.21.15.1:0/72435393 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:06:43.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.241+0000 7fb00f7fe640 1 --2- 172.21.15.1:0/72435393 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6e23044a0 0x55f6e230efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:43.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.242+0000 7fb006ffd640 1 --2- 172.21.15.1:0/72435393 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6e2304dc0 0x55f6e230c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:43.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.242+0000 7fb00ffff640 1 -- 172.21.15.1:0/72435393 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6e23044a0 msgr2=0x55f6e230efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:43.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.242+0000 7fb00ffff640 1 --2- 172.21.15.1:0/72435393 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6e23044a0 0x55f6e230efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:43.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.242+0000 7fb00ffff640 1 -- 172.21.15.1:0/72435393 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6e2304dc0 msgr2=0x55f6e230c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:43.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.242+0000 7fb00ffff640 1 --2- 172.21.15.1:0/72435393 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6e2304dc0 0x55f6e230c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:43.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.242+0000 7fb00ffff640 1 -- 172.21.15.1:0/72435393 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6e1fdb3c0 con 0x55f6e230f680 2024-09-11T22:06:43.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.242+0000 7fb00ffff640 1 --2- 172.21.15.1:0/72435393 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6e230f680 0x55f6e230c790 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fb008009b60 tx=0x7fb008004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fa3da86fe32af51 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:43.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.242+0000 7fafeeffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:06:43.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.242+0000 7fafeeffd640 1 -- 172.21.15.1:0/72435393 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb00802b060 con 0x55f6e230f680 2024-09-11T22:06:43.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.242+0000 7fafeeffd640 1 -- 172.21.15.1:0/72435393 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7fb008018070 con 0x55f6e230f680 2024-09-11T22:06:43.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.242+0000 7fb018219740 1 -- 172.21.15.1:0/72435393 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0000041d0 con 0x55f6e230f680 2024-09-11T22:06:43.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.242+0000 7fafeeffd640 1 -- 172.21.15.1:0/72435393 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fb00801e6f0 con 0x55f6e230f680 2024-09-11T22:06:43.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.242+0000 7fb018219740 10 client.17733 fetch_fsmap learned FSMap version 46 2024-09-11T22:06:43.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.242+0000 7fb018219740 1 -- 172.21.15.1:0/72435393 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55f6e231dc30 con 0x55f6e230f680 2024-09-11T22:06:43.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.243+0000 7fafeeffd640 1 -- 172.21.15.1:0/72435393 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0000041d0 con 0x55f6e230f680 2024-09-11T22:06:43.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.243+0000 7fafeeffd640 1 -- 172.21.15.1:0/72435393 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 46) ==== 8442+0+0 (secure 0 0 0) 0x7fb00800f040 con 0x55f6e230f680 2024-09-11T22:06:43.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.243+0000 7fb018219740 10 client.17733 fetch_fsmap finished waiting for FSMap version 46 2024-09-11T22:06:43.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.243+0000 7fb018219740 10 client.17733 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:06:43.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.243+0000 7fb018219740 20 client.17733 populate_metadata read hostname 'smithi001' 2024-09-11T22:06:43.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.243+0000 7fb018219740 1 --2- 172.21.15.1:0/72435393 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55f6e231f190 0x55f6e2321610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:43.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.243+0000 7fb018219740 4 client.17733 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:06:43.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.243+0000 7fb018219740 1 -- 172.21.15.1:0/72435393 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f6e2277850 con 0x55f6e231f190 2024-09-11T22:06:43.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.243+0000 7fb00f7fe640 1 --2- 172.21.15.1:0/72435393 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55f6e231f190 0x55f6e2321610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:43.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.243+0000 7fb00f7fe640 1 --2- 172.21.15.1:0/72435393 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55f6e231f190 0x55f6e2321610 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7de805fb20233698 server_cookie=99d1551c1512576a in_seq=0 out_seq=0 2024-09-11T22:06:43.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.243+0000 7fafeeffd640 10 client.17733 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:06:43.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.244+0000 7fafeeffd640 1 -- 172.21.15.1:0/72435393 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f6e2277850 con 0x55f6e231f190 2024-09-11T22:06:43.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.244+0000 7fafeeffd640 10 client.17733 handle_command_reply: tid=0 2024-09-11T22:06:43.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.244+0000 7fb018219740 1 client.17733 shutdown 2024-09-11T22:06:43.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.244+0000 7fb018219740 1 -- 172.21.15.1:0/72435393 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6e230f680 msgr2=0x55f6e230c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:43.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.244+0000 7fb018219740 1 --2- 172.21.15.1:0/72435393 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6e230f680 0x55f6e230c790 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fb008009b60 tx=0x7fb008004ab0 comp rx=0 tx=0).stop 2024-09-11T22:06:43.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.244+0000 7fb018219740 1 -- 172.21.15.1:0/72435393 shutdown_connections 2024-09-11T22:06:43.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.244+0000 7fb018219740 1 --2- 172.21.15.1:0/72435393 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55f6e231f190 0x55f6e2321610 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:43.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.244+0000 7fb018219740 1 --2- 172.21.15.1:0/72435393 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6e230f680 0x55f6e230c790 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:43.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.244+0000 7fb018219740 1 --2- 172.21.15.1:0/72435393 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6e2304dc0 0x55f6e230c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:43.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.244+0000 7fafeeffd640 0 client.17733 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:06:43.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.244+0000 7fb018219740 1 --2- 172.21.15.1:0/72435393 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6e23044a0 0x55f6e230efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:43.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.244+0000 7fb018219740 1 -- 172.21.15.1:0/72435393 >> 172.21.15.1:0/72435393 conn(0x55f6e2300a30 msgr2=0x55f6e2306710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:43.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.244+0000 7fb018219740 1 -- 172.21.15.1:0/72435393 shutdown_connections 2024-09-11T22:06:43.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.244+0000 7fb018219740 1 -- 172.21.15.1:0/72435393 wait complete. 2024-09-11T22:06:43.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.245+0000 7fb018219740 20 client.17733 trim_cache size 0 max 16384 2024-09-11T22:06:43.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.262+0000 7fb018219740 1 Processor -- start 2024-09-11T22:06:43.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.262+0000 7fb018219740 1 -- start start 2024-09-11T22:06:43.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.262+0000 7fb018219740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6e2300a30 0x55f6e231b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:43.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.262+0000 7fb018219740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6e23044a0 0x55f6e230c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:43.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.262+0000 7fb018219740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6e2304dc0 0x55f6e2307740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:43.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.262+0000 7fb018219740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb0101afc40 con 0x55f6e23044a0 2024-09-11T22:06:43.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.262+0000 7fb018219740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb0101b1a80 con 0x55f6e2300a30 2024-09-11T22:06:43.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.262+0000 7fb018219740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb0101b03d0 con 0x55f6e2304dc0 2024-09-11T22:06:43.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.262+0000 7fb006ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6e23044a0 0x55f6e230c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:43.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.262+0000 7fb006ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6e23044a0 0x55f6e230c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42090/0 (socket says 172.21.15.1:42090) 2024-09-11T22:06:43.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.262+0000 7fb006ffd640 1 -- 172.21.15.1:0/1525751777 learned_addr learned my addr 172.21.15.1:0/1525751777 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:06:43.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.262+0000 7fb00ffff640 1 --2- 172.21.15.1:0/1525751777 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6e2304dc0 0x55f6e2307740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:43.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.262+0000 7fb00f7fe640 1 --2- 172.21.15.1:0/1525751777 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6e2300a30 0x55f6e231b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:43.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.262+0000 7fb006ffd640 1 -- 172.21.15.1:0/1525751777 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6e2304dc0 msgr2=0x55f6e2307740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:43.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.262+0000 7fb006ffd640 1 --2- 172.21.15.1:0/1525751777 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6e2304dc0 0x55f6e2307740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:43.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.262+0000 7fb006ffd640 1 -- 172.21.15.1:0/1525751777 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6e2300a30 msgr2=0x55f6e231b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:43.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.262+0000 7fb006ffd640 1 --2- 172.21.15.1:0/1525751777 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6e2300a30 0x55f6e231b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:43.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.262+0000 7fb006ffd640 1 -- 172.21.15.1:0/1525751777 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6e230f310 con 0x55f6e23044a0 2024-09-11T22:06:43.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.262+0000 7fb006ffd640 1 --2- 172.21.15.1:0/1525751777 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6e23044a0 0x55f6e230c210 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7faff80b8550 tx=0x7faff8005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d735517e7e336fb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:43.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.263+0000 7fb0057fa640 1 -- 172.21.15.1:0/1525751777 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff8049040 con 0x55f6e23044a0 2024-09-11T22:06:43.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.263+0000 7fb0057fa640 1 -- 172.21.15.1:0/1525751777 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faff803b580 con 0x55f6e23044a0 2024-09-11T22:06:43.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.263+0000 7fb0057fa640 1 -- 172.21.15.1:0/1525751777 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff80c04a0 con 0x55f6e23044a0 2024-09-11T22:06:43.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.263+0000 7fb018219740 1 -- 172.21.15.1:0/1525751777 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6e23044a0 msgr2=0x55f6e230c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:43.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.263+0000 7fb018219740 1 --2- 172.21.15.1:0/1525751777 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6e23044a0 0x55f6e230c210 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7faff80b8550 tx=0x7faff8005b50 comp rx=0 tx=0).stop 2024-09-11T22:06:43.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.263+0000 7fb018219740 1 -- 172.21.15.1:0/1525751777 shutdown_connections 2024-09-11T22:06:43.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.263+0000 7fb018219740 1 --2- 172.21.15.1:0/1525751777 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6e2304dc0 0x55f6e2307740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:43.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.263+0000 7fb018219740 1 --2- 172.21.15.1:0/1525751777 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6e23044a0 0x55f6e230c210 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:43.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.263+0000 7fb018219740 1 --2- 172.21.15.1:0/1525751777 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6e2300a30 0x55f6e231b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:43.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.263+0000 7fb018219740 1 -- 172.21.15.1:0/1525751777 >> 172.21.15.1:0/1525751777 conn(0x55f6e230c8f0 msgr2=0x55f6e2307030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:43.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.263+0000 7fb018219740 1 -- 172.21.15.1:0/1525751777 shutdown_connections 2024-09-11T22:06:43.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.263+0000 7fb018219740 1 -- 172.21.15.1:0/1525751777 wait complete. 2024-09-11T22:06:43.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.264+0000 7fb018219740 1 -- start start 2024-09-11T22:06:43.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.264+0000 7fb018219740 1 Processor -- start 2024-09-11T22:06:43.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.264+0000 7fb018219740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6e2300a30 0x55f6e23134f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:43.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.264+0000 7fb018219740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6e23044a0 0x55f6e2313a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:43.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.264+0000 7fb018219740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6e2304dc0 0x55f6e2313f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:43.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.264+0000 7fb018219740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb0101b12f0 con 0x55f6e2300a30 2024-09-11T22:06:43.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.264+0000 7fb018219740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb0101b0b60 con 0x55f6e2304dc0 2024-09-11T22:06:43.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.264+0000 7fb018219740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb0101adc20 con 0x55f6e23044a0 2024-09-11T22:06:43.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.264+0000 7fb018219740 10 client.0 _ll_register_callbacks cb 0x7fb008009d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:06:43.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.264+0000 7fb00f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6e2300a30 0x55f6e23134f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:43.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.264+0000 7fb00f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6e2300a30 0x55f6e23134f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42098/0 (socket says 172.21.15.1:42098) 2024-09-11T22:06:43.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.264+0000 7fb00f7fe640 1 -- 172.21.15.1:0/1060161709 learned_addr learned my addr 172.21.15.1:0/1060161709 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:06:43.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.264+0000 7fb00ffff640 1 --2- 172.21.15.1:0/1060161709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6e2304dc0 0x55f6e2313f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:43.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.264+0000 7fb006ffd640 1 --2- 172.21.15.1:0/1060161709 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6e23044a0 0x55f6e2313a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:43.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.264+0000 7fb00f7fe640 1 -- 172.21.15.1:0/1060161709 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6e23044a0 msgr2=0x55f6e2313a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:43.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.264+0000 7fb00f7fe640 1 --2- 172.21.15.1:0/1060161709 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6e23044a0 0x55f6e2313a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:43.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.264+0000 7fb00f7fe640 1 -- 172.21.15.1:0/1060161709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6e2304dc0 msgr2=0x55f6e2313f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:06:43.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.264+0000 7fb00f7fe640 1 --2- 172.21.15.1:0/1060161709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6e2304dc0 0x55f6e2313f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:43.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.264+0000 7fb00f7fe640 1 -- 172.21.15.1:0/1060161709 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6e2389090 con 0x55f6e2300a30 2024-09-11T22:06:43.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.264+0000 7fb006ffd640 1 --2- 172.21.15.1:0/1060161709 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6e23044a0 0x55f6e2313a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:06:43.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.265+0000 7fb00f7fe640 1 --2- 172.21.15.1:0/1060161709 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6e2300a30 0x55f6e23134f0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fb00000bd10 tx=0x7fb0000058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9e479cc2662ad14 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:06:43.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.265+0000 7fafeeffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:06:43.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.265+0000 7fafeeffd640 1 -- 172.21.15.1:0/1060161709 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb000005240 con 0x55f6e2300a30 2024-09-11T22:06:43.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.265+0000 7fafeeffd640 1 -- 172.21.15.1:0/1060161709 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7fb0000053e0 con 0x55f6e2300a30 2024-09-11T22:06:43.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.265+0000 7fb018219740 1 -- 172.21.15.1:0/1060161709 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f6e2271d60 con 0x55f6e2300a30 2024-09-11T22:06:43.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.265+0000 7fafeeffd640 1 -- 172.21.15.1:0/1060161709 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fb000037050 con 0x55f6e2300a30 2024-09-11T22:06:43.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.265+0000 7fb018219740 10 client.17739 fetch_fsmap learned FSMap version 46 2024-09-11T22:06:43.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.265+0000 7fb018219740 1 -- 172.21.15.1:0/1060161709 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55f6e238a340 con 0x55f6e2300a30 2024-09-11T22:06:43.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.265+0000 7fafeeffd640 1 -- 172.21.15.1:0/1060161709 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55f6e2271d60 con 0x55f6e2300a30 2024-09-11T22:06:43.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.265+0000 7fafeeffd640 1 -- 172.21.15.1:0/1060161709 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 46) ==== 8442+0+0 (secure 0 0 0) 0x7fb000011670 con 0x55f6e2300a30 2024-09-11T22:06:43.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.265+0000 7fb018219740 10 client.17739 fetch_fsmap finished waiting for FSMap version 46 2024-09-11T22:06:43.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.265+0000 7fb018219740 10 client.17739 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:06:43.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.265+0000 7fb018219740 20 client.17739 populate_metadata read hostname 'smithi001' 2024-09-11T22:06:43.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.265+0000 7fb018219740 1 --2- 172.21.15.1:0/1060161709 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55f6e230f680 0x55f6e238c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:06:43.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.265+0000 7fb018219740 4 client.17739 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:06:43.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.265+0000 7fb018219740 1 -- 172.21.15.1:0/1060161709 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f6e21ee210 con 0x55f6e230f680 2024-09-11T22:06:43.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.266+0000 7fb006ffd640 1 --2- 172.21.15.1:0/1060161709 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55f6e230f680 0x55f6e238c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:06:43.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.266+0000 7fb006ffd640 1 --2- 172.21.15.1:0/1060161709 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55f6e230f680 0x55f6e238c920 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=740c3e544f710135 server_cookie=e5608a82b4cc8ea4 in_seq=0 out_seq=0 2024-09-11T22:06:43.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.266+0000 7fafeeffd640 10 client.17739 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:06:43.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.266+0000 7fafeeffd640 1 -- 172.21.15.1:0/1060161709 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55f6e21ee210 con 0x55f6e230f680 2024-09-11T22:06:43.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.266+0000 7fafeeffd640 10 client.17739 handle_command_reply: tid=0 2024-09-11T22:06:43.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.266+0000 7fb018219740 1 client.17739 shutdown 2024-09-11T22:06:43.297 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-11T22:06:43.297 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T22:06:43.297 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-09-11T22:06:43.297 INFO:teuthology.orchestra.run.smithi001.stdout: "cf120c2a-3202-4382-943e-41da97a17bf6": { 2024-09-11T22:06:43.297 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2024-09-11T22:06:43.297 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "cf120c2a-3202-4382-943e-41da97a17bf6", 2024-09-11T22:06:43.298 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2024-09-11T22:06:43.298 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:06:43.298 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:06:43.298 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-11T22:06:43.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.266+0000 7fb018219740 1 -- 172.21.15.1:0/1060161709 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6e2300a30 msgr2=0x55f6e23134f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:43.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.266+0000 7fb018219740 1 --2- 172.21.15.1:0/1060161709 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6e2300a30 0x55f6e23134f0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fb00000bd10 tx=0x7fb0000058e0 comp rx=0 tx=0).stop 2024-09-11T22:06:43.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.266+0000 7fb018219740 1 -- 172.21.15.1:0/1060161709 shutdown_connections 2024-09-11T22:06:43.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.266+0000 7fb018219740 1 --2- 172.21.15.1:0/1060161709 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55f6e230f680 0x55f6e238c920 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:43.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.266+0000 7fb018219740 1 --2- 172.21.15.1:0/1060161709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f6e2304dc0 0x55f6e2313f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:43.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.266+0000 7fafeeffd640 0 client.17739 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:06:43.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.266+0000 7fb018219740 1 --2- 172.21.15.1:0/1060161709 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f6e23044a0 0x55f6e2313a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:43.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.266+0000 7fb018219740 1 --2- 172.21.15.1:0/1060161709 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6e2300a30 0x55f6e23134f0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:43.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.267+0000 7fb018219740 1 -- 172.21.15.1:0/1060161709 >> 172.21.15.1:0/1060161709 conn(0x55f6e230c8f0 msgr2=0x55f6e22ffb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:43.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.267+0000 7fb018219740 1 -- 172.21.15.1:0/1060161709 shutdown_connections 2024-09-11T22:06:43.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.267+0000 7fb018219740 1 -- 172.21.15.1:0/1060161709 wait complete. 2024-09-11T22:06:43.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.267+0000 7fb018219740 20 client.17739 trim_cache size 0 max 16384 2024-09-11T22:06:43.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.267+0000 7fb015e40640 1 -- 172.21.15.1:0/2953348544 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7faffc0a2cf0 msgr2=0x7faffc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:43.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.267+0000 7fb015e40640 1 --2- 172.21.15.1:0/2953348544 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7faffc0a2cf0 0x7faffc0a51b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb008005fd0 tx=0x7fb008004a40 comp rx=0 tx=0).stop 2024-09-11T22:06:43.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.267+0000 7fb015e40640 1 -- 172.21.15.1:0/2953348544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb010074f40 msgr2=0x7fb01019ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:06:43.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.267+0000 7fb015e40640 1 --2- 172.21.15.1:0/2953348544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb010074f40 0x7fb01019ab80 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7faff8006010 tx=0x7faff80042f0 comp rx=0 tx=0).stop 2024-09-11T22:06:43.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.267+0000 7fb015e40640 1 -- 172.21.15.1:0/2953348544 shutdown_connections 2024-09-11T22:06:43.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.267+0000 7fb015e40640 1 --2- 172.21.15.1:0/2953348544 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7faffc0a2cf0 0x7faffc0a51b0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:43.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.267+0000 7fb015e40640 1 --2- 172.21.15.1:0/2953348544 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb01010e3c0 0x7fb01019ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:43.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.267+0000 7fb015e40640 1 --2- 172.21.15.1:0/2953348544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb010074f40 0x7fb01019ab80 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:43.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.267+0000 7fb015e40640 1 --2- 172.21.15.1:0/2953348544 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb010074620 0x7fb01019a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:06:43.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.267+0000 7fb015e40640 1 -- 172.21.15.1:0/2953348544 >> 172.21.15.1:0/2953348544 conn(0x7fb0100fd8c0 msgr2=0x7fb0100fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:06:43.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.268+0000 7fb015e40640 1 -- 172.21.15.1:0/2953348544 shutdown_connections 2024-09-11T22:06:43.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:06:43.268+0000 7fb015e40640 1 -- 172.21.15.1:0/2953348544 wait complete. 2024-09-11T22:06:43.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:06:43 smithi161 ceph-mon[35615]: scrub queued for path: / 2024-09-11T22:06:43.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:06:43 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:06:43.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:06:43 smithi161 ceph-mon[35615]: scrub summary: active paths [/] 2024-09-11T22:06:43.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:06:43 smithi161 ceph-mon[35615]: scrub summary: idle+waiting paths [/] 2024-09-11T22:06:43.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:06:43 smithi161 ceph-mon[35615]: pgmap v295: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:06:43.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:06:43 smithi053 ceph-mon[34908]: scrub queued for path: / 2024-09-11T22:06:43.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:06:43 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:06:43.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:06:43 smithi053 ceph-mon[34908]: scrub summary: active paths [/] 2024-09-11T22:06:43.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:06:43 smithi053 ceph-mon[34908]: scrub summary: idle+waiting paths [/] 2024-09-11T22:06:43.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:06:43 smithi053 ceph-mon[34908]: pgmap v295: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:06:43.968 INFO:tasks.cephfs.filesystem:scrub status for tag:cf120c2a-3202-4382-943e-41da97a17bf6 - {'path': '/', 'tag': 'cf120c2a-3202-4382-943e-41da97a17bf6', 'options': 'recursive,force'} 2024-09-11T22:06:45.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:06:45 smithi161 ceph-mon[35615]: pgmap v296: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:06:45.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:06:45 smithi001 ceph-mon[30935]: pgmap v296: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:06:45.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:06:45 smithi053 ceph-mon[34908]: pgmap v296: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:06:47.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:06:47 smithi161 ceph-mon[35615]: scrub summary: idle 2024-09-11T22:06:47.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:06:47 smithi161 ceph-mon[35615]: pgmap v297: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 16 op/s 2024-09-11T22:06:47.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:06:47 smithi001 ceph-mon[30935]: scrub summary: idle 2024-09-11T22:06:47.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:06:47 smithi001 ceph-mon[30935]: pgmap v297: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 16 op/s 2024-09-11T22:06:47.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:06:47 smithi053 ceph-mon[34908]: scrub summary: idle 2024-09-11T22:06:47.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:06:47 smithi053 ceph-mon[34908]: pgmap v297: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 16 op/s 2024-09-11T22:06:49.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:06:49 smithi161 ceph-mon[35615]: pgmap v298: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-09-11T22:06:49.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:06:49 smithi001 ceph-mon[30935]: pgmap v298: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-09-11T22:06:49.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:06:49 smithi053 ceph-mon[34908]: pgmap v298: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-09-11T22:06:51.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:06:51 smithi161 ceph-mon[35615]: pgmap v299: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 16 op/s 2024-09-11T22:06:51.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:06:51 smithi001 ceph-mon[30935]: pgmap v299: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 16 op/s 2024-09-11T22:06:51.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:06:51 smithi053 ceph-mon[34908]: pgmap v299: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 16 op/s 2024-09-11T22:06:53.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:06:53 smithi161 ceph-mon[35615]: pgmap v300: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-09-11T22:06:53.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:06:53 smithi001 ceph-mon[30935]: pgmap v300: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-09-11T22:06:53.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:06:53 smithi053 ceph-mon[34908]: pgmap v300: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-09-11T22:06:55.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:06:55 smithi161 ceph-mon[35615]: pgmap v301: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-09-11T22:06:55.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:06:55 smithi001 ceph-mon[30935]: pgmap v301: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-09-11T22:06:55.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:06:55 smithi053 ceph-mon[34908]: pgmap v301: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-09-11T22:06:57.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:06:57 smithi001 ceph-mon[30935]: pgmap v302: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 16 op/s 2024-09-11T22:06:57.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:06:57 smithi053 ceph-mon[34908]: pgmap v302: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 16 op/s 2024-09-11T22:06:57.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:06:57 smithi161 ceph-mon[35615]: pgmap v302: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 16 op/s 2024-09-11T22:06:59.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:06:59 smithi001 ceph-mon[30935]: pgmap v303: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:06:59.578 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:06:59 smithi053 ceph-mon[34908]: pgmap v303: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:06:59.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:06:59 smithi161 ceph-mon[35615]: pgmap v303: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:07:01.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:01 smithi001 ceph-mon[30935]: pgmap v304: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 14 op/s 2024-09-11T22:07:01.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:01 smithi053 ceph-mon[34908]: pgmap v304: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 14 op/s 2024-09-11T22:07:01.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:01 smithi161 ceph-mon[35615]: pgmap v304: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 14 op/s 2024-09-11T22:07:03.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:03 smithi001 ceph-mon[30935]: pgmap v305: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:07:03.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:03 smithi053 ceph-mon[34908]: pgmap v305: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:07:03.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:03 smithi161 ceph-mon[35615]: pgmap v305: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:07:05.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:05 smithi001 ceph-mon[30935]: pgmap v306: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:07:05.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:05 smithi053 ceph-mon[34908]: pgmap v306: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:07:05.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:05 smithi161 ceph-mon[35615]: pgmap v306: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:07:07.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:07 smithi001 ceph-mon[30935]: pgmap v307: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 0 B/s wr, 27 op/s 2024-09-11T22:07:07.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:07 smithi053 ceph-mon[34908]: pgmap v307: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 0 B/s wr, 27 op/s 2024-09-11T22:07:07.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:07 smithi161 ceph-mon[35615]: pgmap v307: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 0 B/s wr, 27 op/s 2024-09-11T22:07:09.394 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:09 smithi161 ceph-mon[35615]: pgmap v308: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 0 B/s wr, 22 op/s 2024-09-11T22:07:09.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:09 smithi001 ceph-mon[30935]: pgmap v308: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 0 B/s wr, 22 op/s 2024-09-11T22:07:09.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:09 smithi053 ceph-mon[34908]: pgmap v308: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 0 B/s wr, 22 op/s 2024-09-11T22:07:11.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:11 smithi001 ceph-mon[30935]: pgmap v309: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 0 B/s wr, 27 op/s 2024-09-11T22:07:11.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:11 smithi053 ceph-mon[34908]: pgmap v309: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 0 B/s wr, 27 op/s 2024-09-11T22:07:11.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:11 smithi161 ceph-mon[35615]: pgmap v309: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 0 B/s wr, 27 op/s 2024-09-11T22:07:12.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:12 smithi001 ceph-mon[30935]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:07:12.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:12 smithi053 ceph-mon[34908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:07:12.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:12 smithi161 ceph-mon[35615]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:07:13.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:13 smithi001 ceph-mon[30935]: pgmap v310: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 0 B/s wr, 22 op/s 2024-09-11T22:07:13.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:13 smithi053 ceph-mon[34908]: pgmap v310: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 0 B/s wr, 22 op/s 2024-09-11T22:07:13.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:13 smithi161 ceph-mon[35615]: pgmap v310: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 0 B/s wr, 22 op/s 2024-09-11T22:07:13.970 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:07:14.245 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:07:14.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.768+0000 7ffba4e56640 1 -- 172.21.15.1:0/3571135528 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffba0102ef0 msgr2=0x7ffba010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:14.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.768+0000 7ffba4e56640 1 --2- 172.21.15.1:0/3571135528 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffba0102ef0 0x7ffba010da40 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7ffb98009ca0 tx=0x7ffb9802a950 comp rx=0 tx=0).stop 2024-09-11T22:07:14.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.768+0000 7ffba4e56640 1 -- 172.21.15.1:0/3571135528 shutdown_connections 2024-09-11T22:07:14.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.768+0000 7ffba4e56640 1 --2- 172.21.15.1:0/3571135528 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffba0102ef0 0x7ffba010da40 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:14.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.768+0000 7ffba4e56640 1 --2- 172.21.15.1:0/3571135528 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffba0102570 0x7ffba01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:14.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.768+0000 7ffba4e56640 1 --2- 172.21.15.1:0/3571135528 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffba0100e20 0x7ffba0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:14.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.768+0000 7ffba4e56640 1 -- 172.21.15.1:0/3571135528 >> 172.21.15.1:0/3571135528 conn(0x7ffba00fc7d0 msgr2=0x7ffba00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:07:14.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.768+0000 7ffba4e56640 1 -- 172.21.15.1:0/3571135528 shutdown_connections 2024-09-11T22:07:14.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.768+0000 7ffba4e56640 1 -- 172.21.15.1:0/3571135528 wait complete. 2024-09-11T22:07:14.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.769+0000 7ffba4e56640 1 Processor -- start 2024-09-11T22:07:14.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.769+0000 7ffba4e56640 1 -- start start 2024-09-11T22:07:14.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.769+0000 7ffba4e56640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffba0100e20 0x7ffba01971a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:14.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.769+0000 7ffba4e56640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffba0102570 0x7ffba01976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:14.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.769+0000 7ffba4e56640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffba0102ef0 0x7ffba0197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:14.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.769+0000 7ffba4e56640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ffba0110fa0 con 0x7ffba0100e20 2024-09-11T22:07:14.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.769+0000 7ffba4e56640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ffba0110e20 con 0x7ffba0102570 2024-09-11T22:07:14.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.769+0000 7ffba4e56640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ffba0111120 con 0x7ffba0102ef0 2024-09-11T22:07:14.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.769+0000 7ffb9e59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffba0100e20 0x7ffba01971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:14.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.769+0000 7ffb9e59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffba0100e20 0x7ffba01971a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57380/0 (socket says 172.21.15.1:57380) 2024-09-11T22:07:14.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.769+0000 7ffb9e59c640 1 -- 172.21.15.1:0/1938083099 learned_addr learned my addr 172.21.15.1:0/1938083099 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:07:14.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.769+0000 7ffb97fff640 1 --2- 172.21.15.1:0/1938083099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffba0102570 0x7ffba01976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:14.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.769+0000 7ffb9ed9d640 1 --2- 172.21.15.1:0/1938083099 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffba0102ef0 0x7ffba0197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:14.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.769+0000 7ffb9e59c640 1 -- 172.21.15.1:0/1938083099 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffba0102ef0 msgr2=0x7ffba0197c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:14.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.769+0000 7ffb9e59c640 1 --2- 172.21.15.1:0/1938083099 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffba0102ef0 0x7ffba0197c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:14.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.769+0000 7ffb9e59c640 1 -- 172.21.15.1:0/1938083099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffba0102570 msgr2=0x7ffba01976e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:14.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.769+0000 7ffb9e59c640 1 --2- 172.21.15.1:0/1938083099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffba0102570 0x7ffba01976e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:14.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.769+0000 7ffb9e59c640 1 -- 172.21.15.1:0/1938083099 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffba01a6a10 con 0x7ffba0100e20 2024-09-11T22:07:14.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.770+0000 7ffb9e59c640 1 --2- 172.21.15.1:0/1938083099 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffba0100e20 0x7ffba01971a0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7ffb9000b4a0 tx=0x7ffb9000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:07:14.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.770+0000 7ffb977fe640 1 -- 172.21.15.1:0/1938083099 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb90014070 con 0x7ffba0100e20 2024-09-11T22:07:14.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.770+0000 7ffba4e56640 1 -- 172.21.15.1:0/1938083099 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffba01a6d00 con 0x7ffba0100e20 2024-09-11T22:07:14.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.770+0000 7ffb977fe640 1 -- 172.21.15.1:0/1938083099 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb90010af0 con 0x7ffba0100e20 2024-09-11T22:07:14.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.770+0000 7ffb977fe640 1 -- 172.21.15.1:0/1938083099 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb90017d10 con 0x7ffba0100e20 2024-09-11T22:07:14.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.770+0000 7ffba4e56640 1 -- 172.21.15.1:0/1938083099 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffba01a71e0 con 0x7ffba0100e20 2024-09-11T22:07:14.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.772+0000 7ffb977fe640 1 -- 172.21.15.1:0/1938083099 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7ffb90017070 con 0x7ffba0100e20 2024-09-11T22:07:14.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.772+0000 7ffba722f740 1 Processor -- start 2024-09-11T22:07:14.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.773+0000 7ffba722f740 1 -- start start 2024-09-11T22:07:14.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.773+0000 7ffba722f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b529bd64a0 0x55b529bd6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:14.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.773+0000 7ffba722f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b529bd6dc0 0x55b529be1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:14.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.773+0000 7ffba722f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b529be1680 0x55b529be3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:14.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.773+0000 7ffba722f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55b529b3c2e0 con 0x55b529bd6dc0 2024-09-11T22:07:14.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.773+0000 7ffba722f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55b529ae3ce0 con 0x55b529be1680 2024-09-11T22:07:14.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.773+0000 7ffba722f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b529b382a0 con 0x55b529bd64a0 2024-09-11T22:07:14.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.773+0000 7ffb9e59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b529bd6dc0 0x55b529be1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:14.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.773+0000 7ffb977fe640 1 --2- 172.21.15.1:0/1938083099 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7ffb8c0a2cf0 0x7ffb8c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:14.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.773+0000 7ffb9e59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b529bd6dc0 0x55b529be1140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57384/0 (socket says 172.21.15.1:57384) 2024-09-11T22:07:14.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.773+0000 7ffb9e59c640 1 -- 172.21.15.1:0/3973812751 learned_addr learned my addr 172.21.15.1:0/3973812751 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:07:14.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.773+0000 7ffb97fff640 1 --2- 172.21.15.1:0/3973812751 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b529be1680 0x55b529be3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:14.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.773+0000 7ffb9ed9d640 1 --2- 172.21.15.1:0/1938083099 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7ffb8c0a2cf0 0x7ffb8c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:14.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.773+0000 7ffb9ed9d640 1 --2- 172.21.15.1:0/3973812751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b529bd64a0 0x55b529bd6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:14.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.773+0000 7ffb977fe640 1 -- 172.21.15.1:0/1938083099 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7ffb900ce6f0 con 0x7ffba0100e20 2024-09-11T22:07:14.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.774+0000 7ffb9e59c640 1 -- 172.21.15.1:0/3973812751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b529bd64a0 msgr2=0x55b529bd6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:07:14.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.774+0000 7ffb9e59c640 1 --2- 172.21.15.1:0/3973812751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b529bd64a0 0x55b529bd6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:14.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.774+0000 7ffb9e59c640 1 -- 172.21.15.1:0/3973812751 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b529be1680 msgr2=0x55b529be3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:07:14.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.774+0000 7ffb9e59c640 1 --2- 172.21.15.1:0/3973812751 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b529be1680 0x55b529be3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:14.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.774+0000 7ffb9e59c640 1 -- 172.21.15.1:0/3973812751 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b529ac8440 con 0x55b529bd6dc0 2024-09-11T22:07:14.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.774+0000 7ffb97fff640 1 --2- 172.21.15.1:0/3973812751 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b529be1680 0x55b529be3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:07:14.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.774+0000 7ffb9ed9d640 1 --2- 172.21.15.1:0/1938083099 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7ffb8c0a2cf0 0x7ffb8c0a51b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ffb98005e30 tx=0x7ffb98005d80 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:07:14.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.774+0000 7ffb9e59c640 1 --2- 172.21.15.1:0/3973812751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b529bd6dc0 0x55b529be1140 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7ffb900cbf70 tx=0x7ffb90017730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdc559ead812922 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:07:14.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.774+0000 7ffb94ff9640 1 -- 172.21.15.1:0/3973812751 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9001e070 con 0x55b529bd6dc0 2024-09-11T22:07:14.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.774+0000 7ffb94ff9640 1 -- 172.21.15.1:0/3973812751 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb9009b3e0 con 0x55b529bd6dc0 2024-09-11T22:07:14.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.774+0000 7ffb94ff9640 1 -- 172.21.15.1:0/3973812751 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb90010ca0 con 0x55b529bd6dc0 2024-09-11T22:07:14.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.775+0000 7ffba722f740 1 -- 172.21.15.1:0/3973812751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b529bd6dc0 msgr2=0x55b529be1140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:14.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.775+0000 7ffba722f740 1 --2- 172.21.15.1:0/3973812751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b529bd6dc0 0x55b529be1140 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7ffb900cbf70 tx=0x7ffb90017730 comp rx=0 tx=0).stop 2024-09-11T22:07:14.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.775+0000 7ffba722f740 1 -- 172.21.15.1:0/3973812751 shutdown_connections 2024-09-11T22:07:14.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.775+0000 7ffba722f740 1 --2- 172.21.15.1:0/3973812751 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b529be1680 0x55b529be3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:14.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.775+0000 7ffba722f740 1 --2- 172.21.15.1:0/3973812751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b529bd6dc0 0x55b529be1140 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:14.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.775+0000 7ffba722f740 1 --2- 172.21.15.1:0/3973812751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b529bd64a0 0x55b529bd6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:14.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.775+0000 7ffba722f740 1 -- 172.21.15.1:0/3973812751 >> 172.21.15.1:0/3973812751 conn(0x55b529bd2a30 msgr2=0x55b529bd2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:07:14.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.775+0000 7ffba722f740 1 -- 172.21.15.1:0/3973812751 shutdown_connections 2024-09-11T22:07:14.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.775+0000 7ffba722f740 1 -- 172.21.15.1:0/3973812751 wait complete. 2024-09-11T22:07:14.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.775+0000 7ffba722f740 1 -- start start 2024-09-11T22:07:14.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.775+0000 7ffba722f740 1 Processor -- start 2024-09-11T22:07:14.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.776+0000 7ffba722f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b529bd64a0 0x55b529be0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:14.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.776+0000 7ffba722f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b529bd6dc0 0x55b529bde260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:14.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.776+0000 7ffba722f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b529be1680 0x55b529bde7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:14.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.776+0000 7ffba722f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55b529be4250 con 0x55b529bd6dc0 2024-09-11T22:07:14.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.776+0000 7ffba722f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55b529af7050 con 0x55b529be1680 2024-09-11T22:07:14.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.776+0000 7ffba722f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55b529be43d0 con 0x55b529bd64a0 2024-09-11T22:07:14.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.776+0000 7ffba722f740 10 client.0 _ll_register_callbacks cb 0x55b529b37160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:07:14.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.776+0000 7ffb97fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b529bd6dc0 0x55b529bde260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:14.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.776+0000 7ffb97fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b529bd6dc0 0x55b529bde260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57388/0 (socket says 172.21.15.1:57388) 2024-09-11T22:07:14.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.776+0000 7ffb97fff640 1 -- 172.21.15.1:0/1318880595 learned_addr learned my addr 172.21.15.1:0/1318880595 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:07:14.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.776+0000 7ffb9ed9d640 1 --2- 172.21.15.1:0/1318880595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b529be1680 0x55b529bde7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:14.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.776+0000 7ffb9e59c640 1 --2- 172.21.15.1:0/1318880595 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b529bd64a0 0x55b529be0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:14.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.776+0000 7ffb97fff640 1 -- 172.21.15.1:0/1318880595 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b529bd64a0 msgr2=0x55b529be0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:07:14.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.776+0000 7ffb97fff640 1 --2- 172.21.15.1:0/1318880595 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b529bd64a0 0x55b529be0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:14.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.776+0000 7ffb97fff640 1 -- 172.21.15.1:0/1318880595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b529be1680 msgr2=0x55b529bde7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:07:14.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.776+0000 7ffb97fff640 1 --2- 172.21.15.1:0/1318880595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b529be1680 0x55b529bde7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:14.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.776+0000 7ffb97fff640 1 -- 172.21.15.1:0/1318880595 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5298ad3c0 con 0x55b529bd6dc0 2024-09-11T22:07:14.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.776+0000 7ffb9e59c640 1 --2- 172.21.15.1:0/1318880595 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b529bd64a0 0x55b529be0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:07:14.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.776+0000 7ffb97fff640 1 --2- 172.21.15.1:0/1318880595 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b529bd6dc0 0x55b529bde260 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7ffb8800bad0 tx=0x7ffb8800bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=648ebf041d03f28c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:07:14.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.776+0000 7ffb85ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:07:14.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.776+0000 7ffb85ffb640 1 -- 172.21.15.1:0/1318880595 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb88003e20 con 0x55b529bd6dc0 2024-09-11T22:07:14.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.776+0000 7ffb85ffb640 1 -- 172.21.15.1:0/1318880595 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7ffb88011440 con 0x55b529bd6dc0 2024-09-11T22:07:14.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.776+0000 7ffba722f740 1 -- 172.21.15.1:0/1318880595 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb9009b3e0 con 0x55b529bd6dc0 2024-09-11T22:07:14.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.777+0000 7ffb85ffb640 1 -- 172.21.15.1:0/1318880595 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7ffb88011640 con 0x55b529bd6dc0 2024-09-11T22:07:14.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.777+0000 7ffba722f740 10 client.17751 fetch_fsmap learned FSMap version 47 2024-09-11T22:07:14.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.777+0000 7ffba722f740 1 -- 172.21.15.1:0/1318880595 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55b529befc90 con 0x55b529bd6dc0 2024-09-11T22:07:14.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.777+0000 7ffb85ffb640 1 -- 172.21.15.1:0/1318880595 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb9009b3e0 con 0x55b529bd6dc0 2024-09-11T22:07:14.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.777+0000 7ffb85ffb640 1 -- 172.21.15.1:0/1318880595 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 47) ==== 8434+0+0 (secure 0 0 0) 0x7ffb88021030 con 0x55b529bd6dc0 2024-09-11T22:07:14.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.777+0000 7ffba722f740 10 client.17751 fetch_fsmap finished waiting for FSMap version 47 2024-09-11T22:07:14.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.777+0000 7ffba722f740 10 client.17751 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:07:14.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.777+0000 7ffba722f740 20 client.17751 populate_metadata read hostname 'smithi001' 2024-09-11T22:07:14.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.777+0000 7ffba722f740 1 --2- 172.21.15.1:0/1318880595 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55b529bf1260 0x55b529bf36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:14.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.777+0000 7ffba722f740 4 client.17751 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:07:14.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.777+0000 7ffba722f740 1 -- 172.21.15.1:0/1318880595 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b529b49850 con 0x55b529bf1260 2024-09-11T22:07:14.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.778+0000 7ffb9e59c640 1 --2- 172.21.15.1:0/1318880595 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55b529bf1260 0x55b529bf36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:14.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.778+0000 7ffb9e59c640 1 --2- 172.21.15.1:0/1318880595 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55b529bf1260 0x55b529bf36b0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4eaacff625f871f server_cookie=5c49f27484731870 in_seq=0 out_seq=0 2024-09-11T22:07:14.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.778+0000 7ffb85ffb640 10 client.17751 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:07:14.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.779+0000 7ffb85ffb640 1 -- 172.21.15.1:0/1318880595 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b529b49850 con 0x55b529bf1260 2024-09-11T22:07:14.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.779+0000 7ffb85ffb640 10 client.17751 handle_command_reply: tid=0 2024-09-11T22:07:14.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.779+0000 7ffba722f740 1 client.17751 shutdown 2024-09-11T22:07:14.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.779+0000 7ffba722f740 1 -- 172.21.15.1:0/1318880595 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b529bd6dc0 msgr2=0x55b529bde260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:14.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.779+0000 7ffba722f740 1 --2- 172.21.15.1:0/1318880595 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b529bd6dc0 0x55b529bde260 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7ffb8800bad0 tx=0x7ffb8800bf90 comp rx=0 tx=0).stop 2024-09-11T22:07:14.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.779+0000 7ffba722f740 1 -- 172.21.15.1:0/1318880595 shutdown_connections 2024-09-11T22:07:14.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.779+0000 7ffba722f740 1 --2- 172.21.15.1:0/1318880595 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55b529bf1260 0x55b529bf36b0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:14.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.779+0000 7ffba722f740 1 --2- 172.21.15.1:0/1318880595 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b529be1680 0x55b529bde7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:14.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.779+0000 7ffb85ffb640 0 client.17751 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:07:14.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.779+0000 7ffba722f740 1 --2- 172.21.15.1:0/1318880595 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b529bd6dc0 0x55b529bde260 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:14.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.779+0000 7ffba722f740 1 --2- 172.21.15.1:0/1318880595 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b529bd64a0 0x55b529be0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:14.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.779+0000 7ffba722f740 1 -- 172.21.15.1:0/1318880595 >> 172.21.15.1:0/1318880595 conn(0x55b529bd2a30 msgr2=0x55b529bd8710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:07:14.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.779+0000 7ffba722f740 1 -- 172.21.15.1:0/1318880595 shutdown_connections 2024-09-11T22:07:14.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.779+0000 7ffba722f740 1 -- 172.21.15.1:0/1318880595 wait complete. 2024-09-11T22:07:14.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.779+0000 7ffba722f740 20 client.17751 trim_cache size 0 max 16384 2024-09-11T22:07:14.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.796+0000 7ffba722f740 1 Processor -- start 2024-09-11T22:07:14.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.796+0000 7ffba722f740 1 -- start start 2024-09-11T22:07:14.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.796+0000 7ffba722f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b529bd2a30 0x55b529bed4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:14.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.796+0000 7ffba722f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b529bd64a0 0x55b529bf8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:14.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.796+0000 7ffba722f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b529bd6dc0 0x55b529bf94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:14.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.796+0000 7ffba722f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ffba01afcf0 con 0x55b529bd6dc0 2024-09-11T22:07:14.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.796+0000 7ffba722f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ffba01b1ad0 con 0x55b529bd2a30 2024-09-11T22:07:14.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.796+0000 7ffba722f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ffba01b0480 con 0x55b529bd64a0 2024-09-11T22:07:14.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.796+0000 7ffb9ed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b529bd6dc0 0x55b529bf94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:14.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.797+0000 7ffb9ed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b529bd6dc0 0x55b529bf94c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57402/0 (socket says 172.21.15.1:57402) 2024-09-11T22:07:14.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.797+0000 7ffb9ed9d640 1 -- 172.21.15.1:0/1322692471 learned_addr learned my addr 172.21.15.1:0/1322692471 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:07:14.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.797+0000 7ffb9e59c640 1 --2- 172.21.15.1:0/1322692471 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b529bd2a30 0x55b529bed4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:14.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.797+0000 7ffb97fff640 1 --2- 172.21.15.1:0/1322692471 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b529bd64a0 0x55b529bf8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:14.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.797+0000 7ffb9ed9d640 1 -- 172.21.15.1:0/1322692471 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b529bd64a0 msgr2=0x55b529bf8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:07:14.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.797+0000 7ffb9ed9d640 1 --2- 172.21.15.1:0/1322692471 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b529bd64a0 0x55b529bf8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:14.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.797+0000 7ffb9ed9d640 1 -- 172.21.15.1:0/1322692471 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b529bd2a30 msgr2=0x55b529bed4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:07:14.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.797+0000 7ffb9ed9d640 1 --2- 172.21.15.1:0/1322692471 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b529bd2a30 0x55b529bed4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:14.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.797+0000 7ffb9ed9d640 1 -- 172.21.15.1:0/1322692471 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b529bdc7b0 con 0x55b529bd6dc0 2024-09-11T22:07:14.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.797+0000 7ffb9ed9d640 1 --2- 172.21.15.1:0/1322692471 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b529bd6dc0 0x55b529bf94c0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7ffb98035000 tx=0x7ffb9803d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0b3a2d860444c77 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:07:14.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.797+0000 7ffb87fff640 1 -- 172.21.15.1:0/1322692471 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb98034040 con 0x55b529bd6dc0 2024-09-11T22:07:14.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.797+0000 7ffb87fff640 1 -- 172.21.15.1:0/1322692471 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb9802f590 con 0x55b529bd6dc0 2024-09-11T22:07:14.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.797+0000 7ffb87fff640 1 -- 172.21.15.1:0/1322692471 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9804c860 con 0x55b529bd6dc0 2024-09-11T22:07:14.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.798+0000 7ffba722f740 1 -- 172.21.15.1:0/1322692471 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b529bd6dc0 msgr2=0x55b529bf94c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:14.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.798+0000 7ffba722f740 1 --2- 172.21.15.1:0/1322692471 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b529bd6dc0 0x55b529bf94c0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7ffb98035000 tx=0x7ffb9803d070 comp rx=0 tx=0).stop 2024-09-11T22:07:14.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.798+0000 7ffba722f740 1 -- 172.21.15.1:0/1322692471 shutdown_connections 2024-09-11T22:07:14.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.798+0000 7ffba722f740 1 --2- 172.21.15.1:0/1322692471 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b529bd6dc0 0x55b529bf94c0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:14.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.798+0000 7ffba722f740 1 --2- 172.21.15.1:0/1322692471 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b529bd64a0 0x55b529bf8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:14.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.798+0000 7ffba722f740 1 --2- 172.21.15.1:0/1322692471 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b529bd2a30 0x55b529bed4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:14.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.798+0000 7ffba722f740 1 -- 172.21.15.1:0/1322692471 >> 172.21.15.1:0/1322692471 conn(0x55b529bde900 msgr2=0x55b529bdd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:07:14.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.798+0000 7ffba722f740 1 -- 172.21.15.1:0/1322692471 shutdown_connections 2024-09-11T22:07:14.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.798+0000 7ffba722f740 1 -- 172.21.15.1:0/1322692471 wait complete. 2024-09-11T22:07:14.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.798+0000 7ffba722f740 1 -- start start 2024-09-11T22:07:14.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.798+0000 7ffba722f740 1 Processor -- start 2024-09-11T22:07:14.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.799+0000 7ffba722f740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b529bd2a30 0x55b529be97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:14.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.799+0000 7ffba722f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b529bd64a0 0x55b529be9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:14.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.799+0000 7ffba722f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b529bd6dc0 0x55b529be6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:14.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.799+0000 7ffba722f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ffba01b13a0 con 0x55b529bd64a0 2024-09-11T22:07:14.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.799+0000 7ffba722f740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7ffba01b0c10 con 0x55b529bd2a30 2024-09-11T22:07:14.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.799+0000 7ffba722f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7ffba01adcd0 con 0x55b529bd6dc0 2024-09-11T22:07:14.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.799+0000 7ffba722f740 10 client.0 _ll_register_callbacks cb 0x7ffb88004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:07:14.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.799+0000 7ffb97fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b529bd64a0 0x55b529be9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:14.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.799+0000 7ffb97fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b529bd64a0 0x55b529be9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57414/0 (socket says 172.21.15.1:57414) 2024-09-11T22:07:14.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.799+0000 7ffb97fff640 1 -- 172.21.15.1:0/2584902452 learned_addr learned my addr 172.21.15.1:0/2584902452 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:07:14.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.799+0000 7ffb9e59c640 1 --2- 172.21.15.1:0/2584902452 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b529bd2a30 0x55b529be97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:14.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.799+0000 7ffb9ed9d640 1 --2- 172.21.15.1:0/2584902452 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b529bd6dc0 0x55b529be6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:14.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.799+0000 7ffb97fff640 1 -- 172.21.15.1:0/2584902452 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b529bd6dc0 msgr2=0x55b529be6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:07:14.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.799+0000 7ffb97fff640 1 --2- 172.21.15.1:0/2584902452 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b529bd6dc0 0x55b529be6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:14.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.799+0000 7ffb97fff640 1 -- 172.21.15.1:0/2584902452 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b529bd2a30 msgr2=0x55b529be97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:07:14.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.799+0000 7ffb97fff640 1 --2- 172.21.15.1:0/2584902452 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b529bd2a30 0x55b529be97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:14.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.799+0000 7ffb97fff640 1 -- 172.21.15.1:0/2584902452 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b529be73c0 con 0x55b529bd64a0 2024-09-11T22:07:14.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.799+0000 7ffb97fff640 1 --2- 172.21.15.1:0/2584902452 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b529bd64a0 0x55b529be9cf0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7ffb8801b9f0 tx=0x7ffb88004420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a45c6269c91324c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:07:14.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.799+0000 7ffb85ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:07:14.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.800+0000 7ffb85ffb640 1 -- 172.21.15.1:0/2584902452 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb88005070 con 0x55b529bd64a0 2024-09-11T22:07:14.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.800+0000 7ffb85ffb640 1 -- 172.21.15.1:0/2584902452 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7ffb88005210 con 0x55b529bd64a0 2024-09-11T22:07:14.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.800+0000 7ffba722f740 1 -- 172.21.15.1:0/2584902452 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb9802f590 con 0x55b529bd64a0 2024-09-11T22:07:14.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.800+0000 7ffb85ffb640 1 -- 172.21.15.1:0/2584902452 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7ffb88005410 con 0x55b529bd64a0 2024-09-11T22:07:14.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.800+0000 7ffba722f740 10 client.17757 fetch_fsmap learned FSMap version 47 2024-09-11T22:07:14.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.800+0000 7ffba722f740 1 -- 172.21.15.1:0/2584902452 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55b529c5ba10 con 0x55b529bd64a0 2024-09-11T22:07:14.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.800+0000 7ffb85ffb640 1 -- 172.21.15.1:0/2584902452 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb9802f590 con 0x55b529bd64a0 2024-09-11T22:07:14.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.800+0000 7ffb85ffb640 1 -- 172.21.15.1:0/2584902452 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 47) ==== 8434+0+0 (secure 0 0 0) 0x7ffb88031030 con 0x55b529bd64a0 2024-09-11T22:07:14.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.800+0000 7ffba722f740 10 client.17757 fetch_fsmap finished waiting for FSMap version 47 2024-09-11T22:07:14.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.800+0000 7ffba722f740 10 client.17757 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:07:14.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.800+0000 7ffba722f740 20 client.17757 populate_metadata read hostname 'smithi001' 2024-09-11T22:07:14.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.800+0000 7ffba722f740 1 --2- 172.21.15.1:0/2584902452 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55b529be1680 0x55b529c5e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:14.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.800+0000 7ffba722f740 4 client.17757 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:07:14.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.800+0000 7ffba722f740 1 -- 172.21.15.1:0/2584902452 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b529b43d60 con 0x55b529be1680 2024-09-11T22:07:14.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.801+0000 7ffb9e59c640 1 --2- 172.21.15.1:0/2584902452 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55b529be1680 0x55b529c5e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:14.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.801+0000 7ffb9e59c640 1 --2- 172.21.15.1:0/2584902452 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55b529be1680 0x55b529c5e2b0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d166d0f4a4a8eb75 server_cookie=320788a2d1199ebc in_seq=0 out_seq=0 2024-09-11T22:07:14.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.801+0000 7ffb85ffb640 10 client.17757 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:07:14.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.801+0000 7ffb85ffb640 1 -- 172.21.15.1:0/2584902452 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b529b43d60 con 0x55b529be1680 2024-09-11T22:07:14.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.801+0000 7ffb85ffb640 10 client.17757 handle_command_reply: tid=0 2024-09-11T22:07:14.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.801+0000 7ffba722f740 1 client.17757 shutdown 2024-09-11T22:07:14.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.801+0000 7ffba722f740 1 -- 172.21.15.1:0/2584902452 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b529bd64a0 msgr2=0x55b529be9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:14.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.801+0000 7ffba722f740 1 --2- 172.21.15.1:0/2584902452 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b529bd64a0 0x55b529be9cf0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7ffb8801b9f0 tx=0x7ffb88004420 comp rx=0 tx=0).stop 2024-09-11T22:07:14.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.801+0000 7ffba722f740 1 -- 172.21.15.1:0/2584902452 shutdown_connections 2024-09-11T22:07:14.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.801+0000 7ffba722f740 1 --2- 172.21.15.1:0/2584902452 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55b529be1680 0x55b529c5e2b0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:14.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.801+0000 7ffba722f740 1 --2- 172.21.15.1:0/2584902452 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55b529bd6dc0 0x55b529be6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:14.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.801+0000 7ffba722f740 1 --2- 172.21.15.1:0/2584902452 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b529bd64a0 0x55b529be9cf0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:14.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.801+0000 7ffba722f740 1 --2- 172.21.15.1:0/2584902452 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b529bd2a30 0x55b529be97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:14.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.801+0000 7ffba722f740 1 -- 172.21.15.1:0/2584902452 >> 172.21.15.1:0/2584902452 conn(0x55b529bde900 msgr2=0x55b529bdfac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:07:14.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.801+0000 7ffb85ffb640 0 client.17757 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:07:14.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.801+0000 7ffba722f740 1 -- 172.21.15.1:0/2584902452 shutdown_connections 2024-09-11T22:07:14.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.801+0000 7ffba722f740 1 -- 172.21.15.1:0/2584902452 wait complete. 2024-09-11T22:07:14.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.802+0000 7ffba722f740 20 client.17757 trim_cache size 0 max 16384 2024-09-11T22:07:14.817 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-11T22:07:14.818 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-09-11T22:07:14.818 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-09-11T22:07:14.818 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-11T22:07:14.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.802+0000 7ffba4e56640 1 -- 172.21.15.1:0/1938083099 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7ffb8c0a2cf0 msgr2=0x7ffb8c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:14.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.802+0000 7ffba4e56640 1 --2- 172.21.15.1:0/1938083099 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7ffb8c0a2cf0 0x7ffb8c0a51b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ffb98005e30 tx=0x7ffb98005d80 comp rx=0 tx=0).stop 2024-09-11T22:07:14.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.802+0000 7ffba4e56640 1 -- 172.21.15.1:0/1938083099 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffba0100e20 msgr2=0x7ffba01971a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:14.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.802+0000 7ffba4e56640 1 --2- 172.21.15.1:0/1938083099 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffba0100e20 0x7ffba01971a0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7ffb9000b4a0 tx=0x7ffb9000b960 comp rx=0 tx=0).stop 2024-09-11T22:07:14.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.802+0000 7ffba4e56640 1 -- 172.21.15.1:0/1938083099 shutdown_connections 2024-09-11T22:07:14.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.802+0000 7ffba4e56640 1 --2- 172.21.15.1:0/1938083099 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7ffb8c0a2cf0 0x7ffb8c0a51b0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:14.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.802+0000 7ffba4e56640 1 --2- 172.21.15.1:0/1938083099 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffba0102ef0 0x7ffba0197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:14.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.802+0000 7ffba4e56640 1 --2- 172.21.15.1:0/1938083099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffba0102570 0x7ffba01976e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:14.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.802+0000 7ffba4e56640 1 --2- 172.21.15.1:0/1938083099 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffba0100e20 0x7ffba01971a0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:14.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.802+0000 7ffba4e56640 1 -- 172.21.15.1:0/1938083099 >> 172.21.15.1:0/1938083099 conn(0x7ffba00fc7d0 msgr2=0x7ffba0106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:07:14.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.802+0000 7ffba4e56640 1 -- 172.21.15.1:0/1938083099 shutdown_connections 2024-09-11T22:07:14.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:14.802+0000 7ffba4e56640 1 -- 172.21.15.1:0/1938083099 wait complete. 2024-09-11T22:07:15.391 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:15 smithi001 ceph-mon[30935]: pgmap v311: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 0 B/s wr, 22 op/s 2024-09-11T22:07:15.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:15 smithi053 ceph-mon[34908]: pgmap v311: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 0 B/s wr, 22 op/s 2024-09-11T22:07:15.590 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T22:07:15.590 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:07:15.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:15 smithi161 ceph-mon[35615]: pgmap v311: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 0 B/s wr, 22 op/s 2024-09-11T22:07:15.860 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:07:16.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:16.315+0000 7f5e93fba640 1 -- 172.21.15.1:0/302030556 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e8c1055b0 msgr2=0x7f5e8c10d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:16.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:16.315+0000 7f5e93fba640 1 --2- 172.21.15.1:0/302030556 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e8c1055b0 0x7f5e8c10d130 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f5e84009900 tx=0x7f5e8402a410 comp rx=0 tx=0).stop 2024-09-11T22:07:16.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:16.315+0000 7f5e93fba640 1 -- 172.21.15.1:0/302030556 shutdown_connections 2024-09-11T22:07:16.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:16.315+0000 7f5e93fba640 1 --2- 172.21.15.1:0/302030556 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e8c1055b0 0x7f5e8c10d130 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:16.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:16.315+0000 7f5e93fba640 1 --2- 172.21.15.1:0/302030556 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e8c100e30 0x7f5e8c104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:16.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:16.315+0000 7f5e93fba640 1 --2- 172.21.15.1:0/302030556 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e8c100480 0x7f5e8c100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:16.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:16.315+0000 7f5e93fba640 1 -- 172.21.15.1:0/302030556 >> 172.21.15.1:0/302030556 conn(0x7f5e8c077210 msgr2=0x7f5e8c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:07:16.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:16.315+0000 7f5e93fba640 1 -- 172.21.15.1:0/302030556 shutdown_connections 2024-09-11T22:07:16.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:16.315+0000 7f5e93fba640 1 -- 172.21.15.1:0/302030556 wait complete. 2024-09-11T22:07:16.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:16.316+0000 7f5e93fba640 1 Processor -- start 2024-09-11T22:07:16.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:16.316+0000 7f5e93fba640 1 -- start start 2024-09-11T22:07:16.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:16.316+0000 7f5e93fba640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e8c100480 0x7f5e8c197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:16.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:16.316+0000 7f5e93fba640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e8c100e30 0x7f5e8c197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:16.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:16.316+0000 7f5e93fba640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e8c1055b0 0x7f5e8c19bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:16.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:16.316+0000 7f5e93fba640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5e8c110ee0 con 0x7f5e8c1055b0 2024-09-11T22:07:16.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:16.316+0000 7f5e93fba640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f5e8c110d60 con 0x7f5e8c100e30 2024-09-11T22:07:16.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:16.316+0000 7f5e93fba640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f5e8c111060 con 0x7f5e8c100480 2024-09-11T22:07:16.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:16.316+0000 7f5e92557640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e8c1055b0 0x7f5e8c19bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:16.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:16.316+0000 7f5e92557640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e8c1055b0 0x7f5e8c19bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57440/0 (socket says 172.21.15.1:57440) 2024-09-11T22:07:16.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:16.316+0000 7f5e92557640 1 -- 172.21.15.1:0/1672196409 learned_addr learned my addr 172.21.15.1:0/1672196409 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:07:16.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:16.316+0000 7f5e91d56640 1 --2- 172.21.15.1:0/1672196409 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e8c100480 0x7f5e8c197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:16.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:16.317+0000 7f5e91555640 1 --2- 172.21.15.1:0/1672196409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e8c100e30 0x7f5e8c197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:16.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:16.317+0000 7f5e92557640 1 -- 172.21.15.1:0/1672196409 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e8c100480 msgr2=0x7f5e8c197300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:16.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:16.317+0000 7f5e92557640 1 --2- 172.21.15.1:0/1672196409 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e8c100480 0x7f5e8c197300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:16.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:16.317+0000 7f5e92557640 1 -- 172.21.15.1:0/1672196409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e8c100e30 msgr2=0x7f5e8c197840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:16.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:16.317+0000 7f5e92557640 1 --2- 172.21.15.1:0/1672196409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e8c100e30 0x7f5e8c197840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:16.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:16.317+0000 7f5e92557640 1 -- 172.21.15.1:0/1672196409 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e8c19c2b0 con 0x7f5e8c1055b0 2024-09-11T22:07:16.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:16.317+0000 7f5e92557640 1 --2- 172.21.15.1:0/1672196409 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e8c1055b0 0x7f5e8c19bbd0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f5e84003b90 tx=0x7f5e84003bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:07:16.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:16.317+0000 7f5e8affd640 1 -- 172.21.15.1:0/1672196409 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e8403f070 con 0x7f5e8c1055b0 2024-09-11T22:07:16.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:16.317+0000 7f5e8affd640 1 -- 172.21.15.1:0/1672196409 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e84031c90 con 0x7f5e8c1055b0 2024-09-11T22:07:16.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:16.317+0000 7f5e93fba640 1 -- 172.21.15.1:0/1672196409 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e8c19c540 con 0x7f5e8c1055b0 2024-09-11T22:07:16.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:16.317+0000 7f5e8affd640 1 -- 172.21.15.1:0/1672196409 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e84035030 con 0x7f5e8c1055b0 2024-09-11T22:07:16.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:16.317+0000 7f5e93fba640 1 -- 172.21.15.1:0/1672196409 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e8c1a3e80 con 0x7f5e8c1055b0 2024-09-11T22:07:16.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:16.318+0000 7f5e93fba640 1 -- 172.21.15.1:0/1672196409 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e8c101960 con 0x7f5e8c1055b0 2024-09-11T22:07:16.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:16.319+0000 7f5e8affd640 1 -- 172.21.15.1:0/1672196409 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f5e84042070 con 0x7f5e8c1055b0 2024-09-11T22:07:16.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:16.320+0000 7f5e8affd640 1 --2- 172.21.15.1:0/1672196409 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f5e740a2cf0 0x7f5e740a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:16.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:16.320+0000 7f5e91d56640 1 --2- 172.21.15.1:0/1672196409 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f5e740a2cf0 0x7f5e740a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:16.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:16.320+0000 7f5e8affd640 1 -- 172.21.15.1:0/1672196409 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7f5e840eff80 con 0x7f5e8c1055b0 2024-09-11T22:07:16.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:16.321+0000 7f5e91d56640 1 --2- 172.21.15.1:0/1672196409 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f5e740a2cf0 0x7f5e740a51b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c0058c0 tx=0x7f5e7c00a2d0 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:07:16.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:16.325+0000 7f5e8affd640 1 -- 172.21.15.1:0/1672196409 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5e840bcb60 con 0x7f5e8c1055b0 2024-09-11T22:07:16.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:16.536+0000 7f5e93fba640 1 -- 172.21.15.1:0/1672196409 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5e8c198660 con 0x7f5e8c1055b0 2024-09-11T22:07:16.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:16.537+0000 7f5e8affd640 1 -- 172.21.15.1:0/1672196409 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+11980 (secure 0 0 0) 0x7f5e840bc460 con 0x7f5e8c1055b0 2024-09-11T22:07:16.539 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T22:07:16.539 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":48,"btime":"2024-09-11T22:07:16:128439+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/421542832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":421542832},{"type":"v1","addr":"172.21.15.1:6835","nonce":421542832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2836864406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2836864406},{"type":"v1","addr":"172.21.15.1:6837","nonce":2836864406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":48,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T21:55:20.791263+0000","modified":"2024-09-11T22:07:15.670855+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24484,"mds_2":24467,"mds_3":24466,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14595":{"gid":14595,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/407834672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":407834672},{"type":"v1","addr":"172.21.15.1:6839","nonce":407834672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/1068827678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":1068827678},{"type":"v1","addr":"172.21.15.1:6841","nonce":1068827678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/872410024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":872410024},{"type":"v1","addr":"172.21.15.161:6833","nonce":872410024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/1648944570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1648944570},{"type":"v1","addr":"172.21.15.53:6833","nonce":1648944570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/2126933993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2126933993},{"type":"v1","addr":"172.21.15.53:6835","nonce":2126933993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.161:6835/828381592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":828381592},{"type":"v1","addr":"172.21.15.161:6835","nonce":828381592}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.161:6837/1799318656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1799318656},{"type":"v1","addr":"172.21.15.161:6837","nonce":1799318656}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.53:6837/3457131830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3457131830},{"type":"v1","addr":"172.21.15.53:6837","nonce":3457131830}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/455365383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":455365383},{"type":"v1","addr":"172.21.15.161:6839","nonce":455365383}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":162,"addr":"172.21.15.53:6839/1378584447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1378584447},{"type":"v1","addr":"172.21.15.53:6839","nonce":1378584447}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24460,24466,24467,24484,24494]},"id":1}]} 2024-09-11T22:07:16.539 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 48 2024-09-11T22:07:16.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:16.539+0000 7f5e93fba640 1 -- 172.21.15.1:0/1672196409 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f5e740a2cf0 msgr2=0x7f5e740a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:16.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:16.539+0000 7f5e93fba640 1 --2- 172.21.15.1:0/1672196409 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f5e740a2cf0 0x7f5e740a51b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c0058c0 tx=0x7f5e7c00a2d0 comp rx=0 tx=0).stop 2024-09-11T22:07:16.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:16.539+0000 7f5e93fba640 1 -- 172.21.15.1:0/1672196409 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e8c1055b0 msgr2=0x7f5e8c19bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:16.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:16.539+0000 7f5e93fba640 1 --2- 172.21.15.1:0/1672196409 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e8c1055b0 0x7f5e8c19bbd0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f5e84003b90 tx=0x7f5e84003bc0 comp rx=0 tx=0).stop 2024-09-11T22:07:16.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:16.539+0000 7f5e93fba640 1 -- 172.21.15.1:0/1672196409 shutdown_connections 2024-09-11T22:07:16.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:16.539+0000 7f5e93fba640 1 --2- 172.21.15.1:0/1672196409 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f5e740a2cf0 0x7f5e740a51b0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:16.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:16.539+0000 7f5e93fba640 1 --2- 172.21.15.1:0/1672196409 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e8c1055b0 0x7f5e8c19bbd0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:16.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:16.539+0000 7f5e93fba640 1 --2- 172.21.15.1:0/1672196409 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e8c100e30 0x7f5e8c197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:16.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:16.539+0000 7f5e93fba640 1 --2- 172.21.15.1:0/1672196409 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e8c100480 0x7f5e8c197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:16.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:16.539+0000 7f5e93fba640 1 -- 172.21.15.1:0/1672196409 >> 172.21.15.1:0/1672196409 conn(0x7f5e8c077210 msgr2=0x7f5e8c103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:07:16.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:16.539+0000 7f5e93fba640 1 -- 172.21.15.1:0/1672196409 shutdown_connections 2024-09-11T22:07:16.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:16.539+0000 7f5e93fba640 1 -- 172.21.15.1:0/1672196409 wait complete. 2024-09-11T22:07:17.166 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T22:07:17.426 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:07:17.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:17 smithi001 ceph-mon[30935]: pgmap v312: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 0 B/s wr, 27 op/s 2024-09-11T22:07:17.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:17 smithi001 ceph-mon[30935]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:07:17.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:17 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/1672196409' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:07:17.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:17 smithi053 ceph-mon[34908]: pgmap v312: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 0 B/s wr, 27 op/s 2024-09-11T22:07:17.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:17 smithi053 ceph-mon[34908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:07:17.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:17 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/1672196409' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:07:17.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:17 smithi161 ceph-mon[35615]: pgmap v312: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 0 B/s wr, 27 op/s 2024-09-11T22:07:17.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:17 smithi161 ceph-mon[35615]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:07:17.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:17 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/1672196409' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:07:17.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.901+0000 7f624d57c640 1 -- 172.21.15.1:0/3305700286 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6248101380 msgr2=0x7f624810dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:17.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.901+0000 7f624d57c640 1 --2- 172.21.15.1:0/3305700286 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6248101380 0x7f624810dfb0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f623c009900 tx=0x7f623c02a410 comp rx=0 tx=0).stop 2024-09-11T22:07:17.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.901+0000 7f624d57c640 1 -- 172.21.15.1:0/3305700286 shutdown_connections 2024-09-11T22:07:17.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.901+0000 7f624d57c640 1 --2- 172.21.15.1:0/3305700286 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f624810e4f0 0x7f6248110900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:17.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.901+0000 7f624d57c640 1 --2- 172.21.15.1:0/3305700286 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6248101380 0x7f624810dfb0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:17.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.901+0000 7f624d57c640 1 --2- 172.21.15.1:0/3305700286 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62481009d0 0x7f6248100db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:17.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.901+0000 7f624d57c640 1 -- 172.21.15.1:0/3305700286 >> 172.21.15.1:0/3305700286 conn(0x7f62480fc7d0 msgr2=0x7f62480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:07:17.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.901+0000 7f624d57c640 1 -- 172.21.15.1:0/3305700286 shutdown_connections 2024-09-11T22:07:17.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.901+0000 7f624d57c640 1 -- 172.21.15.1:0/3305700286 wait complete. 2024-09-11T22:07:17.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.902+0000 7f624d57c640 1 Processor -- start 2024-09-11T22:07:17.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.902+0000 7f624d57c640 1 -- start start 2024-09-11T22:07:17.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.902+0000 7f624d57c640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62481009d0 0x7f624819a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:17.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.902+0000 7f624d57c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6248101380 0x7f624819ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:17.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.902+0000 7f624d57c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f624810e4f0 0x7f624819efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:17.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.902+0000 7f624d57c640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6248114240 con 0x7f6248101380 2024-09-11T22:07:17.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.902+0000 7f624d57c640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f62481140c0 con 0x7f62481009d0 2024-09-11T22:07:17.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.902+0000 7f624d57c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f62481143c0 con 0x7f624810e4f0 2024-09-11T22:07:17.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.902+0000 7f62467fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6248101380 0x7f624819ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:17.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.902+0000 7f62467fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6248101380 0x7f624819ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41784/0 (socket says 172.21.15.1:41784) 2024-09-11T22:07:17.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.902+0000 7f62467fc640 1 -- 172.21.15.1:0/846121945 learned_addr learned my addr 172.21.15.1:0/846121945 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:07:17.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.902+0000 7f62477fe640 1 --2- 172.21.15.1:0/846121945 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f624810e4f0 0x7f624819efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:17.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.902+0000 7f6246ffd640 1 --2- 172.21.15.1:0/846121945 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62481009d0 0x7f624819a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:17.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.902+0000 7f62467fc640 1 -- 172.21.15.1:0/846121945 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f624810e4f0 msgr2=0x7f624819efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:17.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.902+0000 7f62467fc640 1 --2- 172.21.15.1:0/846121945 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f624810e4f0 0x7f624819efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:17.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.902+0000 7f62467fc640 1 -- 172.21.15.1:0/846121945 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62481009d0 msgr2=0x7f624819a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:17.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.902+0000 7f62467fc640 1 --2- 172.21.15.1:0/846121945 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62481009d0 0x7f624819a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:17.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.902+0000 7f62467fc640 1 -- 172.21.15.1:0/846121945 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f624819f6a0 con 0x7f6248101380 2024-09-11T22:07:17.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.903+0000 7f62467fc640 1 --2- 172.21.15.1:0/846121945 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6248101380 0x7f624819ac30 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f623c005ab0 tx=0x7f623c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:07:17.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.903+0000 7f622ffff640 1 -- 172.21.15.1:0/846121945 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f623c03f030 con 0x7f6248101380 2024-09-11T22:07:17.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.903+0000 7f624d57c640 1 -- 172.21.15.1:0/846121945 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f624819f930 con 0x7f6248101380 2024-09-11T22:07:17.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.903+0000 7f622ffff640 1 -- 172.21.15.1:0/846121945 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f623c003ce0 con 0x7f6248101380 2024-09-11T22:07:17.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.903+0000 7f622ffff640 1 -- 172.21.15.1:0/846121945 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f623c035070 con 0x7f6248101380 2024-09-11T22:07:17.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.903+0000 7f624d57c640 1 -- 172.21.15.1:0/846121945 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62481a7270 con 0x7f6248101380 2024-09-11T22:07:17.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.905+0000 7f622ffff640 1 -- 172.21.15.1:0/846121945 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f623c042030 con 0x7f6248101380 2024-09-11T22:07:17.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.906+0000 7f624f955740 1 Processor -- start 2024-09-11T22:07:17.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.906+0000 7f624f955740 1 -- start start 2024-09-11T22:07:17.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.906+0000 7f624f955740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560eb5167b40 0x560eb5166090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:17.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.906+0000 7f624f955740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560eb51665d0 0x560eb5166a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:17.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.906+0000 7f624f955740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560eb5171370 0x560eb5173760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:17.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.906+0000 7f624f955740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x560eb50cc2f0 con 0x560eb5171370 2024-09-11T22:07:17.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.906+0000 7f624f955740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x560eb5073cf0 con 0x560eb5167b40 2024-09-11T22:07:17.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.906+0000 7f624f955740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x560eb50c82b0 con 0x560eb51665d0 2024-09-11T22:07:17.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.906+0000 7f62467fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560eb5171370 0x560eb5173760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:17.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.906+0000 7f62467fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560eb5171370 0x560eb5173760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41796/0 (socket says 172.21.15.1:41796) 2024-09-11T22:07:17.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.906+0000 7f62467fc640 1 -- 172.21.15.1:0/829796217 learned_addr learned my addr 172.21.15.1:0/829796217 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:07:17.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.906+0000 7f6246ffd640 1 --2- 172.21.15.1:0/829796217 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560eb51665d0 0x560eb5166a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:17.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.906+0000 7f62477fe640 1 --2- 172.21.15.1:0/829796217 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560eb5167b40 0x560eb5166090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:17.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.906+0000 7f622ffff640 1 --2- 172.21.15.1:0/846121945 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f62300a2cf0 0x7f62300a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:17.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.906+0000 7f622ffff640 1 -- 172.21.15.1:0/846121945 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7f623c0efcd0 con 0x7f6248101380 2024-09-11T22:07:17.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.906+0000 7f6246ffd640 1 -- 172.21.15.1:0/829796217 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560eb5167b40 msgr2=0x560eb5166090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:07:17.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.906+0000 7f6246ffd640 1 --2- 172.21.15.1:0/829796217 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560eb5167b40 0x560eb5166090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:17.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.906+0000 7f62477fe640 1 --2- 172.21.15.1:0/846121945 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f62300a2cf0 0x7f62300a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:17.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.906+0000 7f6246ffd640 1 -- 172.21.15.1:0/829796217 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560eb5171370 msgr2=0x560eb5173760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:07:17.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.906+0000 7f6246ffd640 1 --2- 172.21.15.1:0/829796217 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560eb5171370 0x560eb5173760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:17.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.906+0000 7f6246ffd640 1 -- 172.21.15.1:0/829796217 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560eb50583a0 con 0x560eb51665d0 2024-09-11T22:07:17.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.907+0000 7f62467fc640 1 --2- 172.21.15.1:0/829796217 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560eb5171370 0x560eb5173760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:07:17.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.907+0000 7f6246ffd640 1 --2- 172.21.15.1:0/829796217 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560eb51665d0 0x560eb5166a10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f623800ab40 tx=0x7f623800f3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8daa985f3f4b0551 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:07:17.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.907+0000 7f62477fe640 1 --2- 172.21.15.1:0/846121945 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f62300a2cf0 0x7f62300a51b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6240007310 tx=0x7f6240004ab0 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:07:17.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.907+0000 7f622d7fa640 1 -- 172.21.15.1:0/829796217 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6238004020 con 0x560eb51665d0 2024-09-11T22:07:17.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.907+0000 7f622d7fa640 1 -- 172.21.15.1:0/829796217 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62380041c0 con 0x560eb51665d0 2024-09-11T22:07:17.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.907+0000 7f622d7fa640 1 -- 172.21.15.1:0/829796217 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6238018860 con 0x560eb51665d0 2024-09-11T22:07:17.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.908+0000 7f624f955740 1 -- 172.21.15.1:0/829796217 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560eb51665d0 msgr2=0x560eb5166a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:17.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.908+0000 7f624f955740 1 --2- 172.21.15.1:0/829796217 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560eb51665d0 0x560eb5166a10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f623800ab40 tx=0x7f623800f3c0 comp rx=0 tx=0).stop 2024-09-11T22:07:17.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.908+0000 7f624f955740 1 -- 172.21.15.1:0/829796217 shutdown_connections 2024-09-11T22:07:17.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.908+0000 7f624f955740 1 --2- 172.21.15.1:0/829796217 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560eb5171370 0x560eb5173760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:17.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.908+0000 7f624f955740 1 --2- 172.21.15.1:0/829796217 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560eb51665d0 0x560eb5166a10 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:17.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.908+0000 7f624f955740 1 --2- 172.21.15.1:0/829796217 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560eb5167b40 0x560eb5166090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:17.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.908+0000 7f624f955740 1 -- 172.21.15.1:0/829796217 >> 172.21.15.1:0/829796217 conn(0x560eb512f310 msgr2=0x560eb5162c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:07:17.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.908+0000 7f624f955740 1 -- 172.21.15.1:0/829796217 shutdown_connections 2024-09-11T22:07:17.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.908+0000 7f624f955740 1 -- 172.21.15.1:0/829796217 wait complete. 2024-09-11T22:07:17.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.908+0000 7f624f955740 1 -- start start 2024-09-11T22:07:17.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.908+0000 7f624f955740 1 Processor -- start 2024-09-11T22:07:17.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.908+0000 7f624f955740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560eb51665d0 0x560eb5170ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:17.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.909+0000 7f624f955740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560eb5167b40 0x560eb516df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:17.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.909+0000 7f624f955740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560eb5171370 0x560eb516e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:17.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.909+0000 7f624f955740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x560eb5174070 con 0x560eb5171370 2024-09-11T22:07:17.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.909+0000 7f624f955740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x560eb5087060 con 0x560eb5167b40 2024-09-11T22:07:17.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.909+0000 7f624f955740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x560eb51741f0 con 0x560eb51665d0 2024-09-11T22:07:17.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.909+0000 7f624f955740 10 client.0 _ll_register_callbacks cb 0x560eb4dda8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:07:17.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.909+0000 7f62477fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560eb5171370 0x560eb516e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:17.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.909+0000 7f62477fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560eb5171370 0x560eb516e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41806/0 (socket says 172.21.15.1:41806) 2024-09-11T22:07:17.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.909+0000 7f62477fe640 1 -- 172.21.15.1:0/2366648980 learned_addr learned my addr 172.21.15.1:0/2366648980 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:07:17.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.909+0000 7f6246ffd640 1 --2- 172.21.15.1:0/2366648980 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560eb51665d0 0x560eb5170ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:17.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.909+0000 7f62467fc640 1 --2- 172.21.15.1:0/2366648980 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560eb5167b40 0x560eb516df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:17.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.909+0000 7f62477fe640 1 -- 172.21.15.1:0/2366648980 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560eb51665d0 msgr2=0x560eb5170ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:07:17.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.909+0000 7f62477fe640 1 --2- 172.21.15.1:0/2366648980 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560eb51665d0 0x560eb5170ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:17.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.909+0000 7f62477fe640 1 -- 172.21.15.1:0/2366648980 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560eb5167b40 msgr2=0x560eb516df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:07:17.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.909+0000 7f62477fe640 1 --2- 172.21.15.1:0/2366648980 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560eb5167b40 0x560eb516df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:17.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.909+0000 7f62477fe640 1 -- 172.21.15.1:0/2366648980 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560eb4e3d3c0 con 0x560eb5171370 2024-09-11T22:07:17.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.909+0000 7f6246ffd640 1 --2- 172.21.15.1:0/2366648980 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560eb51665d0 0x560eb5170ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:07:17.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.909+0000 7f62467fc640 1 --2- 172.21.15.1:0/2366648980 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560eb5167b40 0x560eb516df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:07:17.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.909+0000 7f62477fe640 1 --2- 172.21.15.1:0/2366648980 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560eb5171370 0x560eb516e4f0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f6240011ec0 tx=0x7f6240004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b406f4df995c636 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:07:17.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.909+0000 7f622aff5640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:07:17.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.909+0000 7f622aff5640 1 -- 172.21.15.1:0/2366648980 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6240025020 con 0x560eb5171370 2024-09-11T22:07:17.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.909+0000 7f622aff5640 1 -- 172.21.15.1:0/2366648980 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7f6240021820 con 0x560eb5171370 2024-09-11T22:07:17.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.909+0000 7f624f955740 1 -- 172.21.15.1:0/2366648980 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62380041c0 con 0x560eb5171370 2024-09-11T22:07:17.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.910+0000 7f622aff5640 1 -- 172.21.15.1:0/2366648980 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f6240030050 con 0x560eb5171370 2024-09-11T22:07:17.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.910+0000 7f624f955740 10 client.17775 fetch_fsmap learned FSMap version 48 2024-09-11T22:07:17.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.910+0000 7f624f955740 1 -- 172.21.15.1:0/2366648980 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x560eb517fae0 con 0x560eb5171370 2024-09-11T22:07:17.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.910+0000 7f622aff5640 1 -- 172.21.15.1:0/2366648980 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62380041c0 con 0x560eb5171370 2024-09-11T22:07:17.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.910+0000 7f622aff5640 1 -- 172.21.15.1:0/2366648980 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 48) ==== 8430+0+0 (secure 0 0 0) 0x7f624000f7e0 con 0x560eb5171370 2024-09-11T22:07:17.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.910+0000 7f624f955740 10 client.17775 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T22:07:17.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.910+0000 7f624f955740 10 client.17775 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-11T22:07:17.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.910+0000 7f624f955740 20 client.17775 populate_metadata read hostname 'smithi001' 2024-09-11T22:07:17.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.910+0000 7f624f955740 1 --2- 172.21.15.1:0/2366648980 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x560eb5181040 0x560eb51834c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:17.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.910+0000 7f624f955740 4 client.17775 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:07:17.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.910+0000 7f624f955740 1 -- 172.21.15.1:0/2366648980 --> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560eb50d9860 con 0x560eb5181040 2024-09-11T22:07:17.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.910+0000 7f6246ffd640 1 --2- 172.21.15.1:0/2366648980 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x560eb5181040 0x560eb51834c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:17.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.911+0000 7f622aff5640 10 client.17775 ms_handle_connect on v2:172.21.15.161:6834/828381592 2024-09-11T22:07:17.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.911+0000 7f6246ffd640 1 --2- 172.21.15.1:0/2366648980 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x560eb5181040 0x560eb51834c0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bd7214f6a3b743ea server_cookie=7e8f1ad4501a48fd in_seq=0 out_seq=0 2024-09-11T22:07:17.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.912+0000 7f622aff5640 1 -- 172.21.15.1:0/2366648980 <== mds.4 v2:172.21.15.161:6834/828381592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560eb50d9860 con 0x560eb5181040 2024-09-11T22:07:17.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.912+0000 7f622aff5640 10 client.17775 handle_command_reply: tid=0 2024-09-11T22:07:17.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.912+0000 7f624f955740 1 client.17775 shutdown 2024-09-11T22:07:17.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.912+0000 7f624f955740 1 -- 172.21.15.1:0/2366648980 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560eb5171370 msgr2=0x560eb516e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:17.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.912+0000 7f624f955740 1 --2- 172.21.15.1:0/2366648980 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560eb5171370 0x560eb516e4f0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f6240011ec0 tx=0x7f6240004a40 comp rx=0 tx=0).stop 2024-09-11T22:07:17.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.912+0000 7f624f955740 1 -- 172.21.15.1:0/2366648980 shutdown_connections 2024-09-11T22:07:17.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.912+0000 7f624f955740 1 --2- 172.21.15.1:0/2366648980 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560eb5171370 0x560eb516e4f0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:17.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.912+0000 7f624f955740 1 --2- 172.21.15.1:0/2366648980 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x560eb5181040 0x560eb51834c0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:17.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.912+0000 7f624f955740 1 --2- 172.21.15.1:0/2366648980 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560eb5167b40 0x560eb516df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:17.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.912+0000 7f624f955740 1 --2- 172.21.15.1:0/2366648980 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560eb51665d0 0x560eb5170ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:17.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.912+0000 7f622aff5640 0 client.17775 ms_handle_reset on v2:172.21.15.161:6834/828381592 2024-09-11T22:07:17.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.912+0000 7f624f955740 1 -- 172.21.15.1:0/2366648980 >> 172.21.15.1:0/2366648980 conn(0x560eb512f310 msgr2=0x560eb5168960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:07:17.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.912+0000 7f624f955740 1 -- 172.21.15.1:0/2366648980 shutdown_connections 2024-09-11T22:07:17.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.912+0000 7f624f955740 1 -- 172.21.15.1:0/2366648980 wait complete. 2024-09-11T22:07:17.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.912+0000 7f624f955740 20 client.17775 trim_cache size 0 max 16384 2024-09-11T22:07:17.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.925+0000 7f624f955740 1 Processor -- start 2024-09-11T22:07:17.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.925+0000 7f624f955740 1 -- start start 2024-09-11T22:07:17.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.925+0000 7f624f955740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560eb51665d0 0x560eb516ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:17.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.925+0000 7f624f955740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560eb5167b40 0x560eb5169120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:17.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.925+0000 7f624f955740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560eb5171370 0x560eb5169660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:17.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.925+0000 7f624f955740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f62481afb70 con 0x560eb5167b40 2024-09-11T22:07:17.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.925+0000 7f624f955740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f62481b1920 con 0x560eb51665d0 2024-09-11T22:07:17.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.925+0000 7f624f955740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f62481b0300 con 0x560eb5171370 2024-09-11T22:07:17.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.925+0000 7f62467fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560eb5167b40 0x560eb5169120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:17.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.925+0000 7f62467fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560eb5167b40 0x560eb5169120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41814/0 (socket says 172.21.15.1:41814) 2024-09-11T22:07:17.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.925+0000 7f62467fc640 1 -- 172.21.15.1:0/3268072439 learned_addr learned my addr 172.21.15.1:0/3268072439 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:07:17.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.925+0000 7f6246ffd640 1 --2- 172.21.15.1:0/3268072439 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560eb51665d0 0x560eb516ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:17.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.926+0000 7f62467fc640 1 -- 172.21.15.1:0/3268072439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560eb5171370 msgr2=0x560eb5169660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:07:17.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.926+0000 7f62477fe640 1 --2- 172.21.15.1:0/3268072439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560eb5171370 0x560eb5169660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:17.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.926+0000 7f62467fc640 1 --2- 172.21.15.1:0/3268072439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560eb5171370 0x560eb5169660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:17.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.926+0000 7f62467fc640 1 -- 172.21.15.1:0/3268072439 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560eb51665d0 msgr2=0x560eb516ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:07:17.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.926+0000 7f62467fc640 1 --2- 172.21.15.1:0/3268072439 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560eb51665d0 0x560eb516ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:17.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.926+0000 7f62467fc640 1 -- 172.21.15.1:0/3268072439 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560eb5169ba0 con 0x560eb5167b40 2024-09-11T22:07:17.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.926+0000 7f6246ffd640 1 --2- 172.21.15.1:0/3268072439 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560eb51665d0 0x560eb516ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:07:17.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.926+0000 7f62467fc640 1 --2- 172.21.15.1:0/3268072439 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560eb5167b40 0x560eb5169120 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f623c005940 tx=0x7f623c03b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c412a189b60f67db server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:07:17.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.926+0000 7f622cff9640 1 -- 172.21.15.1:0/3268072439 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f623c049070 con 0x560eb5167b40 2024-09-11T22:07:17.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.926+0000 7f622cff9640 1 -- 172.21.15.1:0/3268072439 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f623c0f1040 con 0x560eb5167b40 2024-09-11T22:07:17.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.926+0000 7f622cff9640 1 -- 172.21.15.1:0/3268072439 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f623c0ed660 con 0x560eb5167b40 2024-09-11T22:07:17.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.926+0000 7f624f955740 1 -- 172.21.15.1:0/3268072439 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560eb5167b40 msgr2=0x560eb5169120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:17.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.926+0000 7f624f955740 1 --2- 172.21.15.1:0/3268072439 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560eb5167b40 0x560eb5169120 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f623c005940 tx=0x7f623c03b800 comp rx=0 tx=0).stop 2024-09-11T22:07:17.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.927+0000 7f624f955740 1 -- 172.21.15.1:0/3268072439 shutdown_connections 2024-09-11T22:07:17.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.927+0000 7f624f955740 1 --2- 172.21.15.1:0/3268072439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560eb5171370 0x560eb5169660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:17.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.927+0000 7f624f955740 1 --2- 172.21.15.1:0/3268072439 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560eb5167b40 0x560eb5169120 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:17.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.927+0000 7f624f955740 1 --2- 172.21.15.1:0/3268072439 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560eb51665d0 0x560eb516ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:17.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.927+0000 7f624f955740 1 -- 172.21.15.1:0/3268072439 >> 172.21.15.1:0/3268072439 conn(0x560eb512f310 msgr2=0x560eb516f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:07:17.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.927+0000 7f624f955740 1 -- 172.21.15.1:0/3268072439 shutdown_connections 2024-09-11T22:07:17.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.927+0000 7f624f955740 1 -- 172.21.15.1:0/3268072439 wait complete. 2024-09-11T22:07:17.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.927+0000 7f624f955740 1 -- start start 2024-09-11T22:07:17.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.927+0000 7f624f955740 1 Processor -- start 2024-09-11T22:07:17.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.927+0000 7f624f955740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560eb51665d0 0x560eb516b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:17.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.927+0000 7f624f955740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560eb5167b40 0x560eb516b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:17.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.927+0000 7f624f955740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560eb5171370 0x560eb516bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:17.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.927+0000 7f624f955740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f62481b1220 con 0x560eb5167b40 2024-09-11T22:07:17.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.927+0000 7f624f955740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f62481b0a90 con 0x560eb51665d0 2024-09-11T22:07:17.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.927+0000 7f624f955740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f62481adb50 con 0x560eb5171370 2024-09-11T22:07:17.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.927+0000 7f624f955740 10 client.0 _ll_register_callbacks cb 0x7f6240007e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:07:17.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.927+0000 7f62467fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560eb5167b40 0x560eb516b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:17.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.927+0000 7f62467fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560eb5167b40 0x560eb516b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41822/0 (socket says 172.21.15.1:41822) 2024-09-11T22:07:17.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.927+0000 7f62467fc640 1 -- 172.21.15.1:0/1419932391 learned_addr learned my addr 172.21.15.1:0/1419932391 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:07:17.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.927+0000 7f6246ffd640 1 --2- 172.21.15.1:0/1419932391 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560eb51665d0 0x560eb516b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:17.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.928+0000 7f62467fc640 1 -- 172.21.15.1:0/1419932391 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560eb5171370 msgr2=0x560eb516bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:07:17.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.928+0000 7f62477fe640 1 --2- 172.21.15.1:0/1419932391 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560eb5171370 0x560eb516bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:17.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.928+0000 7f62467fc640 1 --2- 172.21.15.1:0/1419932391 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560eb5171370 0x560eb516bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:17.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.928+0000 7f62467fc640 1 -- 172.21.15.1:0/1419932391 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560eb51665d0 msgr2=0x560eb516b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:07:17.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.928+0000 7f62467fc640 1 --2- 172.21.15.1:0/1419932391 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560eb51665d0 0x560eb516b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:17.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.928+0000 7f62467fc640 1 -- 172.21.15.1:0/1419932391 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560eb51eaf60 con 0x560eb5167b40 2024-09-11T22:07:17.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.928+0000 7f62477fe640 1 --2- 172.21.15.1:0/1419932391 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560eb5171370 0x560eb516bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:07:17.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.928+0000 7f6246ffd640 1 --2- 172.21.15.1:0/1419932391 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560eb51665d0 0x560eb516b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:07:17.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.928+0000 7f62467fc640 1 --2- 172.21.15.1:0/1419932391 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560eb5167b40 0x560eb516b680 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f623c0b86c0 tx=0x7f623c0c4e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=566723aa4111ee03 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:07:17.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.928+0000 7f622aff5640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:07:17.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.928+0000 7f622aff5640 1 -- 172.21.15.1:0/1419932391 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f623c049070 con 0x560eb5167b40 2024-09-11T22:07:17.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.928+0000 7f622aff5640 1 -- 172.21.15.1:0/1419932391 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7f623c03bd70 con 0x560eb5167b40 2024-09-11T22:07:17.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.928+0000 7f624f955740 1 -- 172.21.15.1:0/1419932391 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f623c0f1040 con 0x560eb5167b40 2024-09-11T22:07:17.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.928+0000 7f622aff5640 1 -- 172.21.15.1:0/1419932391 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f623c0c0560 con 0x560eb5167b40 2024-09-11T22:07:17.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.928+0000 7f624f955740 10 client.17781 fetch_fsmap learned FSMap version 48 2024-09-11T22:07:17.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.928+0000 7f624f955740 1 -- 172.21.15.1:0/1419932391 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x560eb51ec150 con 0x560eb5167b40 2024-09-11T22:07:17.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.929+0000 7f622aff5640 1 -- 172.21.15.1:0/1419932391 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f623c0f1040 con 0x560eb5167b40 2024-09-11T22:07:17.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.929+0000 7f622aff5640 1 -- 172.21.15.1:0/1419932391 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 48) ==== 8430+0+0 (secure 0 0 0) 0x7f623c03b460 con 0x560eb5167b40 2024-09-11T22:07:17.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.929+0000 7f624f955740 10 client.17781 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T22:07:17.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.929+0000 7f624f955740 10 client.17781 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-11T22:07:17.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.929+0000 7f624f955740 20 client.17781 populate_metadata read hostname 'smithi001' 2024-09-11T22:07:17.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.929+0000 7f624f955740 1 --2- 172.21.15.1:0/1419932391 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x560eb51ec8c0 0x560eb51eece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:17.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.929+0000 7f624f955740 4 client.17781 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:07:17.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.929+0000 7f624f955740 1 -- 172.21.15.1:0/1419932391 --> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560eb50d3d70 con 0x560eb51ec8c0 2024-09-11T22:07:17.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.929+0000 7f6246ffd640 1 --2- 172.21.15.1:0/1419932391 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x560eb51ec8c0 0x560eb51eece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:17.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.930+0000 7f6246ffd640 1 --2- 172.21.15.1:0/1419932391 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x560eb51ec8c0 0x560eb51eece0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=425b3bd45dfe11f1 server_cookie=1ca823df52f61c27 in_seq=0 out_seq=0 2024-09-11T22:07:17.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.930+0000 7f622aff5640 10 client.17781 ms_handle_connect on v2:172.21.15.161:6834/828381592 2024-09-11T22:07:17.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.930+0000 7f622aff5640 1 -- 172.21.15.1:0/1419932391 <== mds.4 v2:172.21.15.161:6834/828381592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560eb50d3d70 con 0x560eb51ec8c0 2024-09-11T22:07:17.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.930+0000 7f622aff5640 10 client.17781 handle_command_reply: tid=0 2024-09-11T22:07:17.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.930+0000 7f624f955740 1 client.17781 shutdown 2024-09-11T22:07:17.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.930+0000 7f624f955740 1 -- 172.21.15.1:0/1419932391 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560eb5167b40 msgr2=0x560eb516b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:17.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.930+0000 7f624f955740 1 --2- 172.21.15.1:0/1419932391 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560eb5167b40 0x560eb516b680 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f623c0b86c0 tx=0x7f623c0c4e20 comp rx=0 tx=0).stop 2024-09-11T22:07:17.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.930+0000 7f624f955740 1 -- 172.21.15.1:0/1419932391 shutdown_connections 2024-09-11T22:07:17.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.930+0000 7f624f955740 1 --2- 172.21.15.1:0/1419932391 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x560eb5171370 0x560eb516bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:17.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.930+0000 7f624f955740 1 --2- 172.21.15.1:0/1419932391 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560eb5167b40 0x560eb516b680 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:17.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.930+0000 7f624f955740 1 --2- 172.21.15.1:0/1419932391 >> [v2:172.21.15.161:6834/828381592,v1:172.21.15.161:6835/828381592] conn(0x560eb51ec8c0 0x560eb51eece0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:17.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.930+0000 7f624f955740 1 --2- 172.21.15.1:0/1419932391 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560eb51665d0 0x560eb516b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:17.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.930+0000 7f622aff5640 0 client.17781 ms_handle_reset on v2:172.21.15.161:6834/828381592 2024-09-11T22:07:17.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.930+0000 7f624f955740 1 -- 172.21.15.1:0/1419932391 >> 172.21.15.1:0/1419932391 conn(0x560eb512f310 msgr2=0x560eb517c300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:07:17.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.930+0000 7f624f955740 1 -- 172.21.15.1:0/1419932391 shutdown_connections 2024-09-11T22:07:17.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.930+0000 7f624f955740 1 -- 172.21.15.1:0/1419932391 wait complete. 2024-09-11T22:07:17.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.930+0000 7f624f955740 20 client.17781 trim_cache size 0 max 16384 2024-09-11T22:07:17.953 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-11T22:07:17.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.931+0000 7f624d57c640 1 -- 172.21.15.1:0/846121945 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f62300a2cf0 msgr2=0x7f62300a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:17.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.931+0000 7f624d57c640 1 --2- 172.21.15.1:0/846121945 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f62300a2cf0 0x7f62300a51b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6240007310 tx=0x7f6240004ab0 comp rx=0 tx=0).stop 2024-09-11T22:07:17.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.931+0000 7f624d57c640 1 -- 172.21.15.1:0/846121945 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6248101380 msgr2=0x7f624819ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:17.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.931+0000 7f624d57c640 1 --2- 172.21.15.1:0/846121945 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6248101380 0x7f624819ac30 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f623c005ab0 tx=0x7f623c0042f0 comp rx=0 tx=0).stop 2024-09-11T22:07:17.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.931+0000 7f624d57c640 1 -- 172.21.15.1:0/846121945 shutdown_connections 2024-09-11T22:07:17.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.931+0000 7f624d57c640 1 --2- 172.21.15.1:0/846121945 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f62300a2cf0 0x7f62300a51b0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:17.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.931+0000 7f624d57c640 1 --2- 172.21.15.1:0/846121945 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f624810e4f0 0x7f624819efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:17.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.931+0000 7f624d57c640 1 --2- 172.21.15.1:0/846121945 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6248101380 0x7f624819ac30 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:17.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.931+0000 7f624d57c640 1 --2- 172.21.15.1:0/846121945 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f62481009d0 0x7f624819a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:17.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.931+0000 7f624d57c640 1 -- 172.21.15.1:0/846121945 >> 172.21.15.1:0/846121945 conn(0x7f62480fc7d0 msgr2=0x7f62480febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:07:17.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.931+0000 7f624d57c640 1 -- 172.21.15.1:0/846121945 shutdown_connections 2024-09-11T22:07:17.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:17.931+0000 7f624d57c640 1 -- 172.21.15.1:0/846121945 wait complete. 2024-09-11T22:07:18.664 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T22:07:18.933 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:07:19.339 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:19 smithi001 ceph-mon[30935]: pgmap v313: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:07:19.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:19 smithi053 ceph-mon[34908]: pgmap v313: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:07:19.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.544+0000 7f1c61abe640 1 -- 172.21.15.1:0/2008617006 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c5c10c170 msgr2=0x7f1c5c10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:19.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.544+0000 7f1c61abe640 1 --2- 172.21.15.1:0/2008617006 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c5c10c170 0x7f1c5c10e610 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f1c54009ca0 tx=0x7f1c5402a950 comp rx=0 tx=0).stop 2024-09-11T22:07:19.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.544+0000 7f1c61abe640 1 -- 172.21.15.1:0/2008617006 shutdown_connections 2024-09-11T22:07:19.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.544+0000 7f1c61abe640 1 --2- 172.21.15.1:0/2008617006 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c5c10c170 0x7f1c5c10e610 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:19.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.544+0000 7f1c61abe640 1 --2- 172.21.15.1:0/2008617006 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c5c103660 0x7f1c5c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:19.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.544+0000 7f1c61abe640 1 --2- 172.21.15.1:0/2008617006 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c5c102d40 0x7f1c5c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:19.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.544+0000 7f1c61abe640 1 -- 172.21.15.1:0/2008617006 >> 172.21.15.1:0/2008617006 conn(0x7f1c5c0fc7a0 msgr2=0x7f1c5c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:07:19.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.544+0000 7f1c61abe640 1 -- 172.21.15.1:0/2008617006 shutdown_connections 2024-09-11T22:07:19.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.544+0000 7f1c61abe640 1 -- 172.21.15.1:0/2008617006 wait complete. 2024-09-11T22:07:19.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.545+0000 7f1c61abe640 1 Processor -- start 2024-09-11T22:07:19.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.545+0000 7f1c61abe640 1 -- start start 2024-09-11T22:07:19.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.545+0000 7f1c61abe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c5c102d40 0x7f1c5c198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:19.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.545+0000 7f1c61abe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c5c103660 0x7f1c5c1989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:19.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.545+0000 7f1c61abe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c5c10c170 0x7f1c5c19cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:19.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.545+0000 7f1c61abe640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1c5c111f50 con 0x7f1c5c10c170 2024-09-11T22:07:19.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.545+0000 7f1c61abe640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1c5c111dd0 con 0x7f1c5c102d40 2024-09-11T22:07:19.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.545+0000 7f1c61abe640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1c5c1120d0 con 0x7f1c5c103660 2024-09-11T22:07:19.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.545+0000 7f1c5bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c5c10c170 0x7f1c5c19cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:19.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.545+0000 7f1c5bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c5c10c170 0x7f1c5c19cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41830/0 (socket says 172.21.15.1:41830) 2024-09-11T22:07:19.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.545+0000 7f1c5bfff640 1 -- 172.21.15.1:0/3068753645 learned_addr learned my addr 172.21.15.1:0/3068753645 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:07:19.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.545+0000 7f1c5b7fe640 1 --2- 172.21.15.1:0/3068753645 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c5c102d40 0x7f1c5c198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:19.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.546+0000 7f1c5affd640 1 --2- 172.21.15.1:0/3068753645 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c5c103660 0x7f1c5c1989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:19.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.546+0000 7f1c5bfff640 1 -- 172.21.15.1:0/3068753645 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c5c103660 msgr2=0x7f1c5c1989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:19.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.546+0000 7f1c5bfff640 1 --2- 172.21.15.1:0/3068753645 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c5c103660 0x7f1c5c1989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:19.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.546+0000 7f1c5bfff640 1 -- 172.21.15.1:0/3068753645 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c5c102d40 msgr2=0x7f1c5c198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:19.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.546+0000 7f1c5bfff640 1 --2- 172.21.15.1:0/3068753645 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c5c102d40 0x7f1c5c198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:19.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.546+0000 7f1c5bfff640 1 -- 172.21.15.1:0/3068753645 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c5c19d410 con 0x7f1c5c10c170 2024-09-11T22:07:19.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.546+0000 7f1c5b7fe640 1 --2- 172.21.15.1:0/3068753645 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c5c102d40 0x7f1c5c198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:07:19.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.546+0000 7f1c5bfff640 1 --2- 172.21.15.1:0/3068753645 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c5c10c170 0x7f1c5c19cd30 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f1c5402ae70 tx=0x7f1c5402f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:07:19.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.546+0000 7f1c58ff9640 1 -- 172.21.15.1:0/3068753645 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c54034040 con 0x7f1c5c10c170 2024-09-11T22:07:19.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.546+0000 7f1c61abe640 1 -- 172.21.15.1:0/3068753645 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c5c19d6a0 con 0x7f1c5c10c170 2024-09-11T22:07:19.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.546+0000 7f1c58ff9640 1 -- 172.21.15.1:0/3068753645 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c54030920 con 0x7f1c5c10c170 2024-09-11T22:07:19.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.546+0000 7f1c58ff9640 1 -- 172.21.15.1:0/3068753645 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c54045480 con 0x7f1c5c10c170 2024-09-11T22:07:19.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.546+0000 7f1c61abe640 1 -- 172.21.15.1:0/3068753645 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c5c1a4fe0 con 0x7f1c5c10c170 2024-09-11T22:07:19.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.548+0000 7f1c58ff9640 1 -- 172.21.15.1:0/3068753645 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f1c54037020 con 0x7f1c5c10c170 2024-09-11T22:07:19.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.549+0000 7f1c63e97740 1 Processor -- start 2024-09-11T22:07:19.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.549+0000 7f1c63e97740 1 -- start start 2024-09-11T22:07:19.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.549+0000 7f1c63e97740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1e332ab40 0x55e1e3329090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:19.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.549+0000 7f1c63e97740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e1e33295d0 0x55e1e3329a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:19.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.549+0000 7f1c63e97740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1e3338780 0x55e1e333ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:19.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.549+0000 7f1c63e97740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55e1e328f2f0 con 0x55e1e332ab40 2024-09-11T22:07:19.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.549+0000 7f1c63e97740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55e1e3236cf0 con 0x55e1e3338780 2024-09-11T22:07:19.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.549+0000 7f1c63e97740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55e1e328b2b0 con 0x55e1e33295d0 2024-09-11T22:07:19.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.549+0000 7f1c5affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1e332ab40 0x55e1e3329090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:19.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.549+0000 7f1c58ff9640 1 --2- 172.21.15.1:0/3068753645 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f1c440a2cf0 0x7f1c440a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:19.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.549+0000 7f1c5affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1e332ab40 0x55e1e3329090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41838/0 (socket says 172.21.15.1:41838) 2024-09-11T22:07:19.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.549+0000 7f1c5affd640 1 -- 172.21.15.1:0/3030702639 learned_addr learned my addr 172.21.15.1:0/3030702639 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:07:19.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.549+0000 7f1c5b7fe640 1 --2- 172.21.15.1:0/3030702639 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e1e33295d0 0x55e1e3329a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:19.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.549+0000 7f1c5bfff640 1 --2- 172.21.15.1:0/3068753645 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f1c440a2cf0 0x7f1c440a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:19.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.550+0000 7f1c5affd640 1 -- 172.21.15.1:0/3030702639 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e1e33295d0 msgr2=0x55e1e3329a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:07:19.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.550+0000 7f1c5affd640 1 --2- 172.21.15.1:0/3030702639 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e1e33295d0 0x55e1e3329a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:19.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.550+0000 7f1c5affd640 1 -- 172.21.15.1:0/3030702639 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1e3338780 msgr2=0x55e1e333ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:07:19.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.550+0000 7f1c5affd640 1 --2- 172.21.15.1:0/3030702639 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1e3338780 0x55e1e333ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:19.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.550+0000 7f1c58ff9640 1 -- 172.21.15.1:0/3068753645 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7f1c540f03a0 con 0x7f1c5c10c170 2024-09-11T22:07:19.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.550+0000 7f1c5affd640 1 -- 172.21.15.1:0/3030702639 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1e321b3a0 con 0x55e1e332ab40 2024-09-11T22:07:19.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.550+0000 7f1c5affd640 1 --2- 172.21.15.1:0/3030702639 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1e332ab40 0x55e1e3329090 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f1c5000bc80 tx=0x7f1c50011750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcde8600b6e8b256 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:07:19.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.550+0000 7f1c5bfff640 1 --2- 172.21.15.1:0/3068753645 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f1c440a2cf0 0x7f1c440a51b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1c540b8df0 tx=0x7f1c54009950 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:07:19.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.550+0000 7f1c41ffb640 1 -- 172.21.15.1:0/3030702639 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c5001a400 con 0x55e1e332ab40 2024-09-11T22:07:19.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.550+0000 7f1c41ffb640 1 -- 172.21.15.1:0/3030702639 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c500128d0 con 0x55e1e332ab40 2024-09-11T22:07:19.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.550+0000 7f1c41ffb640 1 -- 172.21.15.1:0/3030702639 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c5000f800 con 0x55e1e332ab40 2024-09-11T22:07:19.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.551+0000 7f1c63e97740 1 -- 172.21.15.1:0/3030702639 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1e332ab40 msgr2=0x55e1e3329090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:19.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.551+0000 7f1c63e97740 1 --2- 172.21.15.1:0/3030702639 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1e332ab40 0x55e1e3329090 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f1c5000bc80 tx=0x7f1c50011750 comp rx=0 tx=0).stop 2024-09-11T22:07:19.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.551+0000 7f1c63e97740 1 -- 172.21.15.1:0/3030702639 shutdown_connections 2024-09-11T22:07:19.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.551+0000 7f1c63e97740 1 --2- 172.21.15.1:0/3030702639 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1e3338780 0x55e1e333ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:19.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.551+0000 7f1c63e97740 1 --2- 172.21.15.1:0/3030702639 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e1e33295d0 0x55e1e3329a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:19.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.551+0000 7f1c63e97740 1 --2- 172.21.15.1:0/3030702639 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1e332ab40 0x55e1e3329090 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:19.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.551+0000 7f1c63e97740 1 -- 172.21.15.1:0/3030702639 >> 172.21.15.1:0/3030702639 conn(0x55e1e32f2310 msgr2=0x55e1e3325c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:07:19.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.551+0000 7f1c63e97740 1 -- 172.21.15.1:0/3030702639 shutdown_connections 2024-09-11T22:07:19.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.551+0000 7f1c63e97740 1 -- 172.21.15.1:0/3030702639 wait complete. 2024-09-11T22:07:19.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.551+0000 7f1c63e97740 1 -- start start 2024-09-11T22:07:19.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.552+0000 7f1c63e97740 1 Processor -- start 2024-09-11T22:07:19.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.552+0000 7f1c63e97740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1e33295d0 0x55e1e33381b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:19.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.552+0000 7f1c63e97740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e1e332ab40 0x55e1e3335460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:19.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.552+0000 7f1c63e97740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1e3338780 0x55e1e3335bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:19.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.552+0000 7f1c63e97740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55e1e333b480 con 0x55e1e3338780 2024-09-11T22:07:19.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.552+0000 7f1c63e97740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55e1e324a060 con 0x55e1e33295d0 2024-09-11T22:07:19.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.552+0000 7f1c63e97740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55e1e333b600 con 0x55e1e332ab40 2024-09-11T22:07:19.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.552+0000 7f1c63e97740 10 client.0 _ll_register_callbacks cb 0x55e1e2f9d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:07:19.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.552+0000 7f1c5affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1e3338780 0x55e1e3335bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:19.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.552+0000 7f1c5affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1e3338780 0x55e1e3335bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41854/0 (socket says 172.21.15.1:41854) 2024-09-11T22:07:19.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.552+0000 7f1c5affd640 1 -- 172.21.15.1:0/3582926423 learned_addr learned my addr 172.21.15.1:0/3582926423 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:07:19.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.552+0000 7f1c5b7fe640 1 --2- 172.21.15.1:0/3582926423 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e1e332ab40 0x55e1e3335460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:19.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.552+0000 7f1c5affd640 1 --2- 172.21.15.1:0/3582926423 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1e33295d0 0x55e1e33381b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:19.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.552+0000 7f1c5affd640 1 -- 172.21.15.1:0/3582926423 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e1e332ab40 msgr2=0x55e1e3335460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:07:19.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.552+0000 7f1c5affd640 1 --2- 172.21.15.1:0/3582926423 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e1e332ab40 0x55e1e3335460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:19.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.552+0000 7f1c5affd640 1 -- 172.21.15.1:0/3582926423 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1e33295d0 msgr2=0x55e1e33381b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:07:19.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.552+0000 7f1c5affd640 1 --2- 172.21.15.1:0/3582926423 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1e33295d0 0x55e1e33381b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:19.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.552+0000 7f1c5affd640 1 -- 172.21.15.1:0/3582926423 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1e30003c0 con 0x55e1e3338780 2024-09-11T22:07:19.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.553+0000 7f1c5affd640 1 --2- 172.21.15.1:0/3582926423 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1e3338780 0x55e1e3335bf0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f1c5000b5d0 tx=0x7f1c5000f420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82eb5068422cc7e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:07:19.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.553+0000 7f1c3affd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:07:19.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.553+0000 7f1c3affd640 1 -- 172.21.15.1:0/3582926423 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c5001e020 con 0x55e1e3338780 2024-09-11T22:07:19.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.553+0000 7f1c3affd640 1 -- 172.21.15.1:0/3582926423 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7f1c50005050 con 0x55e1e3338780 2024-09-11T22:07:19.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.553+0000 7f1c63e97740 1 -- 172.21.15.1:0/3582926423 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c500128d0 con 0x55e1e3338780 2024-09-11T22:07:19.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.553+0000 7f1c3affd640 1 -- 172.21.15.1:0/3582926423 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f1c50024040 con 0x55e1e3338780 2024-09-11T22:07:19.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.553+0000 7f1c63e97740 10 client.17793 fetch_fsmap learned FSMap version 48 2024-09-11T22:07:19.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.553+0000 7f1c63e97740 1 -- 172.21.15.1:0/3582926423 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55e1e3346f60 con 0x55e1e3338780 2024-09-11T22:07:19.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.553+0000 7f1c3affd640 1 -- 172.21.15.1:0/3582926423 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c500128d0 con 0x55e1e3338780 2024-09-11T22:07:19.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.553+0000 7f1c3affd640 1 -- 172.21.15.1:0/3582926423 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 48) ==== 8430+0+0 (secure 0 0 0) 0x7f1c50004c80 con 0x55e1e3338780 2024-09-11T22:07:19.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.553+0000 7f1c63e97740 10 client.17793 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T22:07:19.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.553+0000 7f1c63e97740 10 client.17793 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-11T22:07:19.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.554+0000 7f1c63e97740 20 client.17793 populate_metadata read hostname 'smithi001' 2024-09-11T22:07:19.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.554+0000 7f1c63e97740 1 --2- 172.21.15.1:0/3582926423 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x55e1e3348310 0x55e1e334a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:19.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.554+0000 7f1c63e97740 4 client.17793 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:07:19.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.554+0000 7f1c63e97740 1 -- 172.21.15.1:0/3582926423 --> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e1e329c860 con 0x55e1e3348310 2024-09-11T22:07:19.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.554+0000 7f1c5b7fe640 1 --2- 172.21.15.1:0/3582926423 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x55e1e3348310 0x55e1e334a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:19.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.554+0000 7f1c3affd640 10 client.17793 ms_handle_connect on v2:172.21.15.161:6836/1799318656 2024-09-11T22:07:19.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.554+0000 7f1c5b7fe640 1 --2- 172.21.15.1:0/3582926423 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x55e1e3348310 0x55e1e334a700 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b81b025d0a070feb server_cookie=c96304db621f1903 in_seq=0 out_seq=0 2024-09-11T22:07:19.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.555+0000 7f1c3affd640 1 -- 172.21.15.1:0/3582926423 <== mds.3 v2:172.21.15.161:6836/1799318656 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e1e329c860 con 0x55e1e3348310 2024-09-11T22:07:19.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.555+0000 7f1c3affd640 10 client.17793 handle_command_reply: tid=0 2024-09-11T22:07:19.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.555+0000 7f1c63e97740 1 client.17793 shutdown 2024-09-11T22:07:19.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.555+0000 7f1c63e97740 1 -- 172.21.15.1:0/3582926423 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1e3338780 msgr2=0x55e1e3335bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:19.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.555+0000 7f1c63e97740 1 --2- 172.21.15.1:0/3582926423 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1e3338780 0x55e1e3335bf0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f1c5000b5d0 tx=0x7f1c5000f420 comp rx=0 tx=0).stop 2024-09-11T22:07:19.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.555+0000 7f1c63e97740 1 -- 172.21.15.1:0/3582926423 shutdown_connections 2024-09-11T22:07:19.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.555+0000 7f1c63e97740 1 --2- 172.21.15.1:0/3582926423 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x55e1e3348310 0x55e1e334a700 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:19.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.555+0000 7f1c63e97740 1 --2- 172.21.15.1:0/3582926423 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1e3338780 0x55e1e3335bf0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:19.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.555+0000 7f1c63e97740 1 --2- 172.21.15.1:0/3582926423 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e1e332ab40 0x55e1e3335460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:19.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.555+0000 7f1c63e97740 1 --2- 172.21.15.1:0/3582926423 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1e33295d0 0x55e1e33381b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:19.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.555+0000 7f1c63e97740 1 -- 172.21.15.1:0/3582926423 >> 172.21.15.1:0/3582926423 conn(0x55e1e32f2310 msgr2=0x55e1e332b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:07:19.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.555+0000 7f1c3affd640 0 client.17793 ms_handle_reset on v2:172.21.15.161:6836/1799318656 2024-09-11T22:07:19.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.555+0000 7f1c63e97740 1 -- 172.21.15.1:0/3582926423 shutdown_connections 2024-09-11T22:07:19.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.555+0000 7f1c63e97740 1 -- 172.21.15.1:0/3582926423 wait complete. 2024-09-11T22:07:19.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.555+0000 7f1c63e97740 20 client.17793 trim_cache size 0 max 16384 2024-09-11T22:07:19.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.568+0000 7f1c63e97740 1 Processor -- start 2024-09-11T22:07:19.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.569+0000 7f1c63e97740 1 -- start start 2024-09-11T22:07:19.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.569+0000 7f1c63e97740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1e33295d0 0x55e1e3334d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:19.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.569+0000 7f1c63e97740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e1e332ab40 0x55e1e33373d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:19.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.569+0000 7f1c63e97740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1e3338780 0x55e1e334ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:19.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.569+0000 7f1c63e97740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1c5c1ab790 con 0x55e1e3338780 2024-09-11T22:07:19.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.569+0000 7f1c63e97740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1c5c1ad540 con 0x55e1e33295d0 2024-09-11T22:07:19.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.569+0000 7f1c63e97740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1c5c1abf20 con 0x55e1e332ab40 2024-09-11T22:07:19.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.569+0000 7f1c5affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1e3338780 0x55e1e334ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:19.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.569+0000 7f1c5affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1e3338780 0x55e1e334ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41866/0 (socket says 172.21.15.1:41866) 2024-09-11T22:07:19.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.569+0000 7f1c5affd640 1 -- 172.21.15.1:0/396745557 learned_addr learned my addr 172.21.15.1:0/396745557 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:07:19.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.569+0000 7f1c5affd640 1 --2- 172.21.15.1:0/396745557 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1e33295d0 0x55e1e3334d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:19.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.569+0000 7f1c5b7fe640 1 --2- 172.21.15.1:0/396745557 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e1e332ab40 0x55e1e33373d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:19.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.569+0000 7f1c5affd640 1 -- 172.21.15.1:0/396745557 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e1e332ab40 msgr2=0x55e1e33373d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:07:19.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.569+0000 7f1c5affd640 1 --2- 172.21.15.1:0/396745557 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e1e332ab40 0x55e1e33373d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:19.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.569+0000 7f1c5affd640 1 -- 172.21.15.1:0/396745557 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1e33295d0 msgr2=0x55e1e3334d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:07:19.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.569+0000 7f1c5affd640 1 --2- 172.21.15.1:0/396745557 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1e33295d0 0x55e1e3334d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:19.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.569+0000 7f1c5affd640 1 -- 172.21.15.1:0/396745557 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1e3334410 con 0x55e1e3338780 2024-09-11T22:07:19.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.569+0000 7f1c5affd640 1 --2- 172.21.15.1:0/396745557 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1e3338780 0x55e1e334ff20 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f1c50012550 tx=0x7f1c50005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43531dba7e3e5311 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:07:19.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.570+0000 7f1c417fa640 1 -- 172.21.15.1:0/396745557 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c50037040 con 0x55e1e3338780 2024-09-11T22:07:19.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.570+0000 7f1c417fa640 1 -- 172.21.15.1:0/396745557 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c50011c50 con 0x55e1e3338780 2024-09-11T22:07:19.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.570+0000 7f1c417fa640 1 -- 172.21.15.1:0/396745557 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c5001e040 con 0x55e1e3338780 2024-09-11T22:07:19.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.570+0000 7f1c63e97740 1 -- 172.21.15.1:0/396745557 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1e3338780 msgr2=0x55e1e334ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:19.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.570+0000 7f1c63e97740 1 --2- 172.21.15.1:0/396745557 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1e3338780 0x55e1e334ff20 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f1c50012550 tx=0x7f1c50005970 comp rx=0 tx=0).stop 2024-09-11T22:07:19.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.570+0000 7f1c63e97740 1 -- 172.21.15.1:0/396745557 shutdown_connections 2024-09-11T22:07:19.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.570+0000 7f1c63e97740 1 --2- 172.21.15.1:0/396745557 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1e3338780 0x55e1e334ff20 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:19.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.570+0000 7f1c63e97740 1 --2- 172.21.15.1:0/396745557 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e1e332ab40 0x55e1e33373d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:19.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.570+0000 7f1c63e97740 1 --2- 172.21.15.1:0/396745557 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1e33295d0 0x55e1e3334d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:19.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.570+0000 7f1c63e97740 1 -- 172.21.15.1:0/396745557 >> 172.21.15.1:0/396745557 conn(0x55e1e32f2310 msgr2=0x55e1e3336cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:07:19.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.570+0000 7f1c63e97740 1 -- 172.21.15.1:0/396745557 shutdown_connections 2024-09-11T22:07:19.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.570+0000 7f1c63e97740 1 -- 172.21.15.1:0/396745557 wait complete. 2024-09-11T22:07:19.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.571+0000 7f1c63e97740 1 -- start start 2024-09-11T22:07:19.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.571+0000 7f1c63e97740 1 Processor -- start 2024-09-11T22:07:19.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.571+0000 7f1c63e97740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1e33295d0 0x55e1e3340a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:19.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.571+0000 7f1c63e97740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1e332ab40 0x55e1e3340f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:19.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.571+0000 7f1c63e97740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e1e3338780 0x55e1e333be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:19.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.571+0000 7f1c63e97740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1c5c1ace40 con 0x55e1e33295d0 2024-09-11T22:07:19.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.571+0000 7f1c63e97740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1c5c1ac6b0 con 0x55e1e332ab40 2024-09-11T22:07:19.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.571+0000 7f1c63e97740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1c5c1a9770 con 0x55e1e3338780 2024-09-11T22:07:19.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.571+0000 7f1c63e97740 10 client.0 _ll_register_callbacks cb 0x7f1c4c00b440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:07:19.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.571+0000 7f1c5affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1e33295d0 0x55e1e3340a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:19.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.571+0000 7f1c5affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1e33295d0 0x55e1e3340a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41880/0 (socket says 172.21.15.1:41880) 2024-09-11T22:07:19.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.571+0000 7f1c5affd640 1 -- 172.21.15.1:0/86900044 learned_addr learned my addr 172.21.15.1:0/86900044 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:07:19.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.571+0000 7f1c5affd640 1 --2- 172.21.15.1:0/86900044 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e1e3338780 0x55e1e333be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:19.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.571+0000 7f1c5b7fe640 1 --2- 172.21.15.1:0/86900044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1e332ab40 0x55e1e3340f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:19.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.572+0000 7f1c5affd640 1 -- 172.21.15.1:0/86900044 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e1e3338780 msgr2=0x55e1e333be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:07:19.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.572+0000 7f1c5affd640 1 --2- 172.21.15.1:0/86900044 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e1e3338780 0x55e1e333be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:19.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.572+0000 7f1c5affd640 1 -- 172.21.15.1:0/86900044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1e332ab40 msgr2=0x55e1e3340f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:07:19.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.572+0000 7f1c5affd640 1 --2- 172.21.15.1:0/86900044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1e332ab40 0x55e1e3340f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:19.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.572+0000 7f1c5affd640 1 -- 172.21.15.1:0/86900044 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1e333c660 con 0x55e1e33295d0 2024-09-11T22:07:19.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.572+0000 7f1c5affd640 1 --2- 172.21.15.1:0/86900044 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1e33295d0 0x55e1e3340a30 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f1c50012010 tx=0x7f1c50011fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f6a89bbd947c8b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:07:19.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.572+0000 7f1c3affd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:07:19.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.572+0000 7f1c3affd640 1 -- 172.21.15.1:0/86900044 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c50037070 con 0x55e1e33295d0 2024-09-11T22:07:19.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.572+0000 7f1c63e97740 1 -- 172.21.15.1:0/86900044 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c50011c50 con 0x55e1e33295d0 2024-09-11T22:07:19.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.572+0000 7f1c3affd640 1 -- 172.21.15.1:0/86900044 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7f1c5000fb80 con 0x55e1e33295d0 2024-09-11T22:07:19.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.572+0000 7f1c3affd640 1 -- 172.21.15.1:0/86900044 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f1c5001e040 con 0x55e1e33295d0 2024-09-11T22:07:19.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.572+0000 7f1c63e97740 10 client.17799 fetch_fsmap learned FSMap version 48 2024-09-11T22:07:19.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.572+0000 7f1c63e97740 1 -- 172.21.15.1:0/86900044 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55e1e33b2a00 con 0x55e1e33295d0 2024-09-11T22:07:19.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.572+0000 7f1c3affd640 1 -- 172.21.15.1:0/86900044 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c50011c50 con 0x55e1e33295d0 2024-09-11T22:07:19.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.573+0000 7f1c3affd640 1 -- 172.21.15.1:0/86900044 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 48) ==== 8430+0+0 (secure 0 0 0) 0x7f1c5003d070 con 0x55e1e33295d0 2024-09-11T22:07:19.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.573+0000 7f1c63e97740 10 client.17799 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T22:07:19.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.573+0000 7f1c63e97740 10 client.17799 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-11T22:07:19.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.573+0000 7f1c63e97740 20 client.17799 populate_metadata read hostname 'smithi001' 2024-09-11T22:07:19.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.573+0000 7f1c63e97740 1 --2- 172.21.15.1:0/86900044 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x55e1e33b3060 0x55e1e33b5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:19.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.573+0000 7f1c63e97740 4 client.17799 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:07:19.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.573+0000 7f1c63e97740 1 -- 172.21.15.1:0/86900044 --> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e1e3296d70 con 0x55e1e33b3060 2024-09-11T22:07:19.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.573+0000 7f1c5b7fe640 1 --2- 172.21.15.1:0/86900044 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x55e1e33b3060 0x55e1e33b5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:19.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.573+0000 7f1c5b7fe640 1 --2- 172.21.15.1:0/86900044 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x55e1e33b3060 0x55e1e33b5480 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=52f1a13a1f5aa471 server_cookie=890848b4d99bb9c7 in_seq=0 out_seq=0 2024-09-11T22:07:19.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.573+0000 7f1c3affd640 10 client.17799 ms_handle_connect on v2:172.21.15.161:6836/1799318656 2024-09-11T22:07:19.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.574+0000 7f1c3affd640 1 -- 172.21.15.1:0/86900044 <== mds.3 v2:172.21.15.161:6836/1799318656 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e1e3296d70 con 0x55e1e33b3060 2024-09-11T22:07:19.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.574+0000 7f1c3affd640 10 client.17799 handle_command_reply: tid=0 2024-09-11T22:07:19.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.574+0000 7f1c63e97740 1 client.17799 shutdown 2024-09-11T22:07:19.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.574+0000 7f1c63e97740 1 -- 172.21.15.1:0/86900044 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1e33295d0 msgr2=0x55e1e3340a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:19.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.574+0000 7f1c63e97740 1 --2- 172.21.15.1:0/86900044 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1e33295d0 0x55e1e3340a30 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f1c50012010 tx=0x7f1c50011fa0 comp rx=0 tx=0).stop 2024-09-11T22:07:19.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.574+0000 7f1c63e97740 1 -- 172.21.15.1:0/86900044 shutdown_connections 2024-09-11T22:07:19.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.574+0000 7f1c63e97740 1 --2- 172.21.15.1:0/86900044 >> [v2:172.21.15.161:6836/1799318656,v1:172.21.15.161:6837/1799318656] conn(0x55e1e33b3060 0x55e1e33b5480 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:19.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.574+0000 7f1c63e97740 1 --2- 172.21.15.1:0/86900044 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55e1e3338780 0x55e1e333be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:19.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.574+0000 7f1c63e97740 1 --2- 172.21.15.1:0/86900044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e1e332ab40 0x55e1e3340f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:19.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.574+0000 7f1c63e97740 1 --2- 172.21.15.1:0/86900044 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1e33295d0 0x55e1e3340a30 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:19.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.574+0000 7f1c3affd640 0 client.17799 ms_handle_reset on v2:172.21.15.161:6836/1799318656 2024-09-11T22:07:19.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.574+0000 7f1c63e97740 1 -- 172.21.15.1:0/86900044 >> 172.21.15.1:0/86900044 conn(0x55e1e32f2310 msgr2=0x55e1e33374e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:07:19.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.574+0000 7f1c63e97740 1 -- 172.21.15.1:0/86900044 shutdown_connections 2024-09-11T22:07:19.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.574+0000 7f1c63e97740 1 -- 172.21.15.1:0/86900044 wait complete. 2024-09-11T22:07:19.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.574+0000 7f1c63e97740 20 client.17799 trim_cache size 0 max 16384 2024-09-11T22:07:19.611 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-11T22:07:19.612 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:19 smithi161 ceph-mon[35615]: pgmap v313: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:07:19.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.575+0000 7f1c61abe640 1 -- 172.21.15.1:0/3068753645 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f1c440a2cf0 msgr2=0x7f1c440a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:19.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.575+0000 7f1c61abe640 1 --2- 172.21.15.1:0/3068753645 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f1c440a2cf0 0x7f1c440a51b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1c540b8df0 tx=0x7f1c54009950 comp rx=0 tx=0).stop 2024-09-11T22:07:19.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.575+0000 7f1c61abe640 1 -- 172.21.15.1:0/3068753645 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c5c10c170 msgr2=0x7f1c5c19cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:19.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.575+0000 7f1c61abe640 1 --2- 172.21.15.1:0/3068753645 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c5c10c170 0x7f1c5c19cd30 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f1c5402ae70 tx=0x7f1c5402f5b0 comp rx=0 tx=0).stop 2024-09-11T22:07:19.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.575+0000 7f1c61abe640 1 -- 172.21.15.1:0/3068753645 shutdown_connections 2024-09-11T22:07:19.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.575+0000 7f1c61abe640 1 --2- 172.21.15.1:0/3068753645 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f1c440a2cf0 0x7f1c440a51b0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:19.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.575+0000 7f1c61abe640 1 --2- 172.21.15.1:0/3068753645 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c5c10c170 0x7f1c5c19cd30 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:19.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.575+0000 7f1c61abe640 1 --2- 172.21.15.1:0/3068753645 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c5c103660 0x7f1c5c1989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:19.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.575+0000 7f1c61abe640 1 --2- 172.21.15.1:0/3068753645 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c5c102d40 0x7f1c5c198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:19.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.575+0000 7f1c61abe640 1 -- 172.21.15.1:0/3068753645 >> 172.21.15.1:0/3068753645 conn(0x7f1c5c0fc7a0 msgr2=0x7f1c5c0fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:07:19.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.575+0000 7f1c61abe640 1 -- 172.21.15.1:0/3068753645 shutdown_connections 2024-09-11T22:07:19.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:19.575+0000 7f1c61abe640 1 -- 172.21.15.1:0/3068753645 wait complete. 2024-09-11T22:07:20.394 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T22:07:20.658 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:07:21.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.141+0000 7f04f029d640 1 -- 172.21.15.1:0/3676275515 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04e810c170 msgr2=0x7f04e810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:21.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.141+0000 7f04f029d640 1 --2- 172.21.15.1:0/3676275515 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04e810c170 0x7f04e810e610 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f04e0009ca0 tx=0x7f04e002a950 comp rx=0 tx=0).stop 2024-09-11T22:07:21.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.141+0000 7f04f029d640 1 -- 172.21.15.1:0/3676275515 shutdown_connections 2024-09-11T22:07:21.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.141+0000 7f04f029d640 1 --2- 172.21.15.1:0/3676275515 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04e810c170 0x7f04e810e610 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:21.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.141+0000 7f04f029d640 1 --2- 172.21.15.1:0/3676275515 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04e8103660 0x7f04e810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:21.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.141+0000 7f04f029d640 1 --2- 172.21.15.1:0/3676275515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04e8102d40 0x7f04e8103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:21.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.141+0000 7f04f029d640 1 -- 172.21.15.1:0/3676275515 >> 172.21.15.1:0/3676275515 conn(0x7f04e80fc7a0 msgr2=0x7f04e80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:07:21.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.141+0000 7f04f029d640 1 -- 172.21.15.1:0/3676275515 shutdown_connections 2024-09-11T22:07:21.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.141+0000 7f04f029d640 1 -- 172.21.15.1:0/3676275515 wait complete. 2024-09-11T22:07:21.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.142+0000 7f04f029d640 1 Processor -- start 2024-09-11T22:07:21.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.142+0000 7f04f029d640 1 -- start start 2024-09-11T22:07:21.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.142+0000 7f04f029d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04e8102d40 0x7f04e8198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:21.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.142+0000 7f04f029d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04e8103660 0x7f04e8198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:21.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.142+0000 7f04f029d640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04e810c170 0x7f04e819cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:21.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.142+0000 7f04f029d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f04e8111f70 con 0x7f04e8102d40 2024-09-11T22:07:21.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.142+0000 7f04f029d640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f04e8111df0 con 0x7f04e810c170 2024-09-11T22:07:21.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.142+0000 7f04f029d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f04e81120f0 con 0x7f04e8103660 2024-09-11T22:07:21.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.142+0000 7f04ee039640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04e8102d40 0x7f04e8198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:21.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.142+0000 7f04ee039640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04e8102d40 0x7f04e8198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41892/0 (socket says 172.21.15.1:41892) 2024-09-11T22:07:21.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.142+0000 7f04ee039640 1 -- 172.21.15.1:0/3554529698 learned_addr learned my addr 172.21.15.1:0/3554529698 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:07:21.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.142+0000 7f04ee83a640 1 --2- 172.21.15.1:0/3554529698 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04e810c170 0x7f04e819cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:21.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.142+0000 7f04ed838640 1 --2- 172.21.15.1:0/3554529698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04e8103660 0x7f04e8198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:21.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.143+0000 7f04ee039640 1 -- 172.21.15.1:0/3554529698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04e8103660 msgr2=0x7f04e8198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:21.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.143+0000 7f04ee039640 1 --2- 172.21.15.1:0/3554529698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04e8103660 0x7f04e8198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:21.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.143+0000 7f04ee039640 1 -- 172.21.15.1:0/3554529698 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04e810c170 msgr2=0x7f04e819cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:21.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.143+0000 7f04ee039640 1 --2- 172.21.15.1:0/3554529698 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04e810c170 0x7f04e819cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:21.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.143+0000 7f04ee039640 1 -- 172.21.15.1:0/3554529698 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04e819d400 con 0x7f04e8102d40 2024-09-11T22:07:21.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.143+0000 7f04ee039640 1 --2- 172.21.15.1:0/3554529698 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04e8102d40 0x7f04e8198450 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f04e400e9f0 tx=0x7f04e400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:07:21.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.143+0000 7f04df7fe640 1 -- 172.21.15.1:0/3554529698 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e400cd80 con 0x7f04e8102d40 2024-09-11T22:07:21.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.143+0000 7f04df7fe640 1 -- 172.21.15.1:0/3554529698 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04e4010910 con 0x7f04e8102d40 2024-09-11T22:07:21.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.143+0000 7f04f029d640 1 -- 172.21.15.1:0/3554529698 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04e819d6f0 con 0x7f04e8102d40 2024-09-11T22:07:21.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.143+0000 7f04df7fe640 1 -- 172.21.15.1:0/3554529698 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e4018990 con 0x7f04e8102d40 2024-09-11T22:07:21.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.143+0000 7f04f029d640 1 -- 172.21.15.1:0/3554529698 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04e81a4fd0 con 0x7f04e8102d40 2024-09-11T22:07:21.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.146+0000 7f04df7fe640 1 -- 172.21.15.1:0/3554529698 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f04e4010ac0 con 0x7f04e8102d40 2024-09-11T22:07:21.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.146+0000 7f04f2676740 1 Processor -- start 2024-09-11T22:07:21.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.147+0000 7f04f2676740 1 -- start start 2024-09-11T22:07:21.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.147+0000 7f04f2676740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f2202b7b40 0x55f2202b6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:21.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.147+0000 7f04f2676740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f2202b65d0 0x55f2202b6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:21.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.147+0000 7f04f2676740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f2202c1370 0x55f2202c3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:21.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.147+0000 7f04f2676740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55f22021c2f0 con 0x55f2202b7b40 2024-09-11T22:07:21.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.147+0000 7f04f2676740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55f2201c3cf0 con 0x55f2202b65d0 2024-09-11T22:07:21.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.147+0000 7f04f2676740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f2202182b0 con 0x55f2202c1370 2024-09-11T22:07:21.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.147+0000 7f04ee83a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f2202b7b40 0x55f2202b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:21.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.147+0000 7f04ee83a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f2202b7b40 0x55f2202b6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41908/0 (socket says 172.21.15.1:41908) 2024-09-11T22:07:21.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.147+0000 7f04ee83a640 1 -- 172.21.15.1:0/332870374 learned_addr learned my addr 172.21.15.1:0/332870374 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:07:21.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.147+0000 7f04ee039640 1 --2- 172.21.15.1:0/332870374 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f2202b65d0 0x55f2202b6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:21.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.147+0000 7f04df7fe640 1 --2- 172.21.15.1:0/3554529698 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f04d40a2d70 0x7f04d40a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:21.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.147+0000 7f04ed838640 1 --2- 172.21.15.1:0/332870374 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f2202c1370 0x55f2202c3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:21.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.147+0000 7f04ee83a640 1 -- 172.21.15.1:0/332870374 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f2202c1370 msgr2=0x55f2202c3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:07:21.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.147+0000 7f04df7fe640 1 -- 172.21.15.1:0/3554529698 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7f04e40cf700 con 0x7f04e8102d40 2024-09-11T22:07:21.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.147+0000 7f04ee83a640 1 --2- 172.21.15.1:0/332870374 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f2202c1370 0x55f2202c3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:21.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.147+0000 7f04ee83a640 1 -- 172.21.15.1:0/332870374 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f2202b65d0 msgr2=0x55f2202b6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:07:21.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.147+0000 7f04ee83a640 1 --2- 172.21.15.1:0/332870374 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f2202b65d0 0x55f2202b6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:21.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.147+0000 7f04ee83a640 1 -- 172.21.15.1:0/332870374 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2201a83a0 con 0x55f2202b7b40 2024-09-11T22:07:21.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.147+0000 7f04ee039640 1 --2- 172.21.15.1:0/332870374 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f2202b65d0 0x55f2202b6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:07:21.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.148+0000 7f04ee83a640 1 --2- 172.21.15.1:0/3554529698 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f04d40a2d70 0x7f04d40a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:21.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.148+0000 7f04ee83a640 1 --2- 172.21.15.1:0/332870374 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f2202b7b40 0x55f2202b6090 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f04e0009ca0 tx=0x7f04e0009950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a88bab1630ee706f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:07:21.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.148+0000 7f04dcdea640 1 -- 172.21.15.1:0/332870374 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e0034040 con 0x55f2202b7b40 2024-09-11T22:07:21.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.148+0000 7f04dcdea640 1 -- 172.21.15.1:0/332870374 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04e0003c30 con 0x55f2202b7b40 2024-09-11T22:07:21.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.148+0000 7f04dcdea640 1 -- 172.21.15.1:0/332870374 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e0046670 con 0x55f2202b7b40 2024-09-11T22:07:21.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.149+0000 7f04f2676740 1 -- 172.21.15.1:0/332870374 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f2202b7b40 msgr2=0x55f2202b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:21.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.149+0000 7f04f2676740 1 --2- 172.21.15.1:0/332870374 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f2202b7b40 0x55f2202b6090 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f04e0009ca0 tx=0x7f04e0009950 comp rx=0 tx=0).stop 2024-09-11T22:07:21.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.149+0000 7f04ee83a640 1 --2- 172.21.15.1:0/3554529698 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f04d40a2d70 0x7f04d40a5230 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f04e004d000 tx=0x7f04e004e000 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:07:21.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.149+0000 7f04f2676740 1 -- 172.21.15.1:0/332870374 shutdown_connections 2024-09-11T22:07:21.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.149+0000 7f04f2676740 1 --2- 172.21.15.1:0/332870374 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f2202c1370 0x55f2202c3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:21.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.149+0000 7f04f2676740 1 --2- 172.21.15.1:0/332870374 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f2202b65d0 0x55f2202b6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:21.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.149+0000 7f04f2676740 1 --2- 172.21.15.1:0/332870374 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f2202b7b40 0x55f2202b6090 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:21.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.149+0000 7f04f2676740 1 -- 172.21.15.1:0/332870374 >> 172.21.15.1:0/332870374 conn(0x55f22027f310 msgr2=0x55f2202b2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:07:21.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.149+0000 7f04f2676740 1 -- 172.21.15.1:0/332870374 shutdown_connections 2024-09-11T22:07:21.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.149+0000 7f04f2676740 1 -- 172.21.15.1:0/332870374 wait complete. 2024-09-11T22:07:21.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.149+0000 7f04f2676740 1 -- start start 2024-09-11T22:07:21.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.149+0000 7f04f2676740 1 Processor -- start 2024-09-11T22:07:21.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.150+0000 7f04f2676740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f2202b65d0 0x55f2202c0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:21.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.150+0000 7f04f2676740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f2202b7b40 0x55f2202bdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:21.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.150+0000 7f04f2676740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f2202c1370 0x55f2202be5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:21.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.150+0000 7f04f2676740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55f2202c4070 con 0x55f2202c1370 2024-09-11T22:07:21.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.150+0000 7f04f2676740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55f2201d7060 con 0x55f2202b7b40 2024-09-11T22:07:21.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.150+0000 7f04f2676740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55f2202c41f0 con 0x55f2202b65d0 2024-09-11T22:07:21.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.150+0000 7f04f2676740 10 client.0 _ll_register_callbacks cb 0x55f21ff2a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:07:21.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.150+0000 7f04ee83a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f2202c1370 0x55f2202be5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:21.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.150+0000 7f04ee83a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f2202c1370 0x55f2202be5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41914/0 (socket says 172.21.15.1:41914) 2024-09-11T22:07:21.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.150+0000 7f04ee83a640 1 -- 172.21.15.1:0/2995746814 learned_addr learned my addr 172.21.15.1:0/2995746814 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:07:21.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.150+0000 7f04ed838640 1 --2- 172.21.15.1:0/2995746814 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f2202b7b40 0x55f2202bdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:21.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.150+0000 7f04ee039640 1 --2- 172.21.15.1:0/2995746814 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f2202b65d0 0x55f2202c0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:21.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.150+0000 7f04ee83a640 1 -- 172.21.15.1:0/2995746814 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f2202b65d0 msgr2=0x55f2202c0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:07:21.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.150+0000 7f04ee83a640 1 --2- 172.21.15.1:0/2995746814 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f2202b65d0 0x55f2202c0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:21.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.150+0000 7f04ee83a640 1 -- 172.21.15.1:0/2995746814 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f2202b7b40 msgr2=0x55f2202bdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:07:21.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.150+0000 7f04ee83a640 1 --2- 172.21.15.1:0/2995746814 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f2202b7b40 0x55f2202bdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:21.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.150+0000 7f04ee83a640 1 -- 172.21.15.1:0/2995746814 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f21ff8d3c0 con 0x55f2202c1370 2024-09-11T22:07:21.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.150+0000 7f04ee83a640 1 --2- 172.21.15.1:0/2995746814 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f2202c1370 0x55f2202be5d0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f04e00093e0 tx=0x7f04e0004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc556458630bbd94 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:07:21.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.150+0000 7f04d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:07:21.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.150+0000 7f04d1ffb640 1 -- 172.21.15.1:0/2995746814 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e0058070 con 0x55f2202c1370 2024-09-11T22:07:21.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.151+0000 7f04d1ffb640 1 -- 172.21.15.1:0/2995746814 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7f04e0046c20 con 0x55f2202c1370 2024-09-11T22:07:21.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.151+0000 7f04f2676740 1 -- 172.21.15.1:0/2995746814 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04e0003c30 con 0x55f2202c1370 2024-09-11T22:07:21.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.151+0000 7f04d1ffb640 1 -- 172.21.15.1:0/2995746814 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f04e0064050 con 0x55f2202c1370 2024-09-11T22:07:21.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.151+0000 7f04f2676740 10 client.17811 fetch_fsmap learned FSMap version 48 2024-09-11T22:07:21.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.151+0000 7f04f2676740 1 -- 172.21.15.1:0/2995746814 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55f2202c4070 con 0x55f2202c1370 2024-09-11T22:07:21.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.151+0000 7f04d1ffb640 1 -- 172.21.15.1:0/2995746814 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04e0003c30 con 0x55f2202c1370 2024-09-11T22:07:21.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.151+0000 7f04d1ffb640 1 -- 172.21.15.1:0/2995746814 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 48) ==== 8430+0+0 (secure 0 0 0) 0x7f04e00306a0 con 0x55f2202c1370 2024-09-11T22:07:21.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.151+0000 7f04f2676740 10 client.17811 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T22:07:21.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.151+0000 7f04f2676740 10 client.17811 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-11T22:07:21.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.151+0000 7f04f2676740 20 client.17811 populate_metadata read hostname 'smithi001' 2024-09-11T22:07:21.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.151+0000 7f04f2676740 1 --2- 172.21.15.1:0/2995746814 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x55f2202d0c50 0x55f2202d3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:21.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.151+0000 7f04f2676740 4 client.17811 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:07:21.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.151+0000 7f04f2676740 1 -- 172.21.15.1:0/2995746814 --> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f220229860 con 0x55f2202d0c50 2024-09-11T22:07:21.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.152+0000 7f04ee039640 1 --2- 172.21.15.1:0/2995746814 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x55f2202d0c50 0x55f2202d3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:21.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.152+0000 7f04d1ffb640 10 client.17811 ms_handle_connect on v2:172.21.15.53:6836/3457131830 2024-09-11T22:07:21.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.152+0000 7f04ee039640 1 --2- 172.21.15.1:0/2995746814 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x55f2202d0c50 0x55f2202d3080 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7b53eedb997f6658 server_cookie=acaa99d9b60d83ac in_seq=0 out_seq=0 2024-09-11T22:07:21.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.153+0000 7f04d1ffb640 1 -- 172.21.15.1:0/2995746814 <== mds.2 v2:172.21.15.53:6836/3457131830 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f220229860 con 0x55f2202d0c50 2024-09-11T22:07:21.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.153+0000 7f04d1ffb640 10 client.17811 handle_command_reply: tid=0 2024-09-11T22:07:21.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.153+0000 7f04f2676740 1 client.17811 shutdown 2024-09-11T22:07:21.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.153+0000 7f04f2676740 1 -- 172.21.15.1:0/2995746814 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f2202c1370 msgr2=0x55f2202be5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:21.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.153+0000 7f04f2676740 1 --2- 172.21.15.1:0/2995746814 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f2202c1370 0x55f2202be5d0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f04e00093e0 tx=0x7f04e0004860 comp rx=0 tx=0).stop 2024-09-11T22:07:21.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.153+0000 7f04f2676740 1 -- 172.21.15.1:0/2995746814 shutdown_connections 2024-09-11T22:07:21.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.153+0000 7f04f2676740 1 --2- 172.21.15.1:0/2995746814 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x55f2202d0c50 0x55f2202d3080 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:21.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.153+0000 7f04f2676740 1 --2- 172.21.15.1:0/2995746814 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f2202c1370 0x55f2202be5d0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:21.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.153+0000 7f04d1ffb640 0 client.17811 ms_handle_reset on v2:172.21.15.53:6836/3457131830 2024-09-11T22:07:21.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.153+0000 7f04f2676740 1 --2- 172.21.15.1:0/2995746814 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f2202b7b40 0x55f2202bdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:21.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.153+0000 7f04f2676740 1 --2- 172.21.15.1:0/2995746814 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f2202b65d0 0x55f2202c0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:21.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.153+0000 7f04f2676740 1 -- 172.21.15.1:0/2995746814 >> 172.21.15.1:0/2995746814 conn(0x55f22027f310 msgr2=0x55f2202b8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:07:21.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.153+0000 7f04f2676740 1 -- 172.21.15.1:0/2995746814 shutdown_connections 2024-09-11T22:07:21.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.153+0000 7f04f2676740 1 -- 172.21.15.1:0/2995746814 wait complete. 2024-09-11T22:07:21.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.153+0000 7f04f2676740 20 client.17811 trim_cache size 0 max 16384 2024-09-11T22:07:21.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.167+0000 7f04f2676740 1 Processor -- start 2024-09-11T22:07:21.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.167+0000 7f04f2676740 1 -- start start 2024-09-11T22:07:21.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.167+0000 7f04f2676740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f2202b65d0 0x55f2202bff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:21.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.167+0000 7f04f2676740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f2202b7b40 0x55f2202b9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:21.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.167+0000 7f04f2676740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f2202c1370 0x55f2202b9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:21.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.167+0000 7f04f2676740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f04e81ad940 con 0x55f2202b65d0 2024-09-11T22:07:21.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.167+0000 7f04f2676740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f04e81af6f0 con 0x55f2202c1370 2024-09-11T22:07:21.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.167+0000 7f04f2676740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f04e81ae0d0 con 0x55f2202b7b40 2024-09-11T22:07:21.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.167+0000 7f04ee039640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f2202b65d0 0x55f2202bff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:21.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.167+0000 7f04ee039640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f2202b65d0 0x55f2202bff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41924/0 (socket says 172.21.15.1:41924) 2024-09-11T22:07:21.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.167+0000 7f04ee039640 1 -- 172.21.15.1:0/2734035413 learned_addr learned my addr 172.21.15.1:0/2734035413 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:07:21.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.168+0000 7f04ee039640 1 -- 172.21.15.1:0/2734035413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f2202b7b40 msgr2=0x55f2202b9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:07:21.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.168+0000 7f04ed838640 1 --2- 172.21.15.1:0/2734035413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f2202b7b40 0x55f2202b9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:21.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.168+0000 7f04ee83a640 1 --2- 172.21.15.1:0/2734035413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f2202c1370 0x55f2202b9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:21.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.168+0000 7f04ee039640 1 --2- 172.21.15.1:0/2734035413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f2202b7b40 0x55f2202b9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:21.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.168+0000 7f04ee039640 1 -- 172.21.15.1:0/2734035413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f2202c1370 msgr2=0x55f2202b9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:07:21.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.168+0000 7f04ee039640 1 --2- 172.21.15.1:0/2734035413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f2202c1370 0x55f2202b9660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:21.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.168+0000 7f04ee039640 1 -- 172.21.15.1:0/2734035413 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2202b24f0 con 0x55f2202b65d0 2024-09-11T22:07:21.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.168+0000 7f04ed838640 1 --2- 172.21.15.1:0/2734035413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f2202b7b40 0x55f2202b9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:07:21.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.168+0000 7f04ee039640 1 --2- 172.21.15.1:0/2734035413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f2202b65d0 0x55f2202bff50 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f04e40cce70 tx=0x7f04e409bb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=106e658abddf8c0a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:07:21.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.168+0000 7f04d3fff640 1 -- 172.21.15.1:0/2734035413 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e4099d30 con 0x55f2202b65d0 2024-09-11T22:07:21.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.168+0000 7f04d3fff640 1 -- 172.21.15.1:0/2734035413 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04e4099f90 con 0x55f2202b65d0 2024-09-11T22:07:21.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.168+0000 7f04d3fff640 1 -- 172.21.15.1:0/2734035413 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e4018b30 con 0x55f2202b65d0 2024-09-11T22:07:21.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.169+0000 7f04f2676740 1 -- 172.21.15.1:0/2734035413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f2202b65d0 msgr2=0x55f2202bff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:21.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.169+0000 7f04f2676740 1 --2- 172.21.15.1:0/2734035413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f2202b65d0 0x55f2202bff50 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f04e40cce70 tx=0x7f04e409bb90 comp rx=0 tx=0).stop 2024-09-11T22:07:21.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.169+0000 7f04f2676740 1 -- 172.21.15.1:0/2734035413 shutdown_connections 2024-09-11T22:07:21.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.169+0000 7f04f2676740 1 --2- 172.21.15.1:0/2734035413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f2202c1370 0x55f2202b9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:21.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.169+0000 7f04f2676740 1 --2- 172.21.15.1:0/2734035413 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f2202b7b40 0x55f2202b9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:21.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.169+0000 7f04f2676740 1 --2- 172.21.15.1:0/2734035413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f2202b65d0 0x55f2202bff50 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:21.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.169+0000 7f04f2676740 1 -- 172.21.15.1:0/2734035413 >> 172.21.15.1:0/2734035413 conn(0x55f22027f310 msgr2=0x55f2202bf840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:07:21.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.169+0000 7f04f2676740 1 -- 172.21.15.1:0/2734035413 shutdown_connections 2024-09-11T22:07:21.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.169+0000 7f04f2676740 1 -- 172.21.15.1:0/2734035413 wait complete. 2024-09-11T22:07:21.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.169+0000 7f04f2676740 1 -- start start 2024-09-11T22:07:21.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.169+0000 7f04f2676740 1 Processor -- start 2024-09-11T22:07:21.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.170+0000 7f04f2676740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f2202b65d0 0x55f2202c5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:21.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.170+0000 7f04f2676740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f2202b7b40 0x55f2202c59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:21.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.170+0000 7f04f2676740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f2202c1370 0x55f2202c5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:21.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.170+0000 7f04f2676740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f04e81aeff0 con 0x55f2202b7b40 2024-09-11T22:07:21.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.170+0000 7f04f2676740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f04e81ae860 con 0x55f2202b65d0 2024-09-11T22:07:21.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.170+0000 7f04f2676740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f04e81ab920 con 0x55f2202c1370 2024-09-11T22:07:21.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.170+0000 7f04f2676740 10 client.0 _ll_register_callbacks cb 0x7f04e409ba60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:07:21.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.170+0000 7f04ed838640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f2202b7b40 0x55f2202c59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:21.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.170+0000 7f04ed838640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f2202b7b40 0x55f2202c59b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41934/0 (socket says 172.21.15.1:41934) 2024-09-11T22:07:21.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.170+0000 7f04ed838640 1 -- 172.21.15.1:0/3337235176 learned_addr learned my addr 172.21.15.1:0/3337235176 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:07:21.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.170+0000 7f04ee039640 1 --2- 172.21.15.1:0/3337235176 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f2202b65d0 0x55f2202c5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:21.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.170+0000 7f04ee83a640 1 --2- 172.21.15.1:0/3337235176 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f2202c1370 0x55f2202c5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:21.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.170+0000 7f04ed838640 1 -- 172.21.15.1:0/3337235176 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f2202c1370 msgr2=0x55f2202c5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:07:21.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.170+0000 7f04ed838640 1 --2- 172.21.15.1:0/3337235176 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f2202c1370 0x55f2202c5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:21.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.170+0000 7f04ed838640 1 -- 172.21.15.1:0/3337235176 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f2202b65d0 msgr2=0x55f2202c5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:07:21.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.170+0000 7f04ed838640 1 --2- 172.21.15.1:0/3337235176 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f2202b65d0 0x55f2202c5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:21.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.170+0000 7f04ed838640 1 -- 172.21.15.1:0/3337235176 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f22033abc0 con 0x55f2202b7b40 2024-09-11T22:07:21.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.170+0000 7f04ed838640 1 --2- 172.21.15.1:0/3337235176 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f2202b7b40 0x55f2202c59b0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f04d8005d00 tx=0x7f04d8019970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95e26cecb13328cc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:07:21.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.170+0000 7f04d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:07:21.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.170+0000 7f04d1ffb640 1 -- 172.21.15.1:0/3337235176 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d8026060 con 0x55f2202b7b40 2024-09-11T22:07:21.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.170+0000 7f04d1ffb640 1 -- 172.21.15.1:0/3337235176 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7f04d8004020 con 0x55f2202b7b40 2024-09-11T22:07:21.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.170+0000 7f04f2676740 1 -- 172.21.15.1:0/3337235176 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04e4099f90 con 0x55f2202b7b40 2024-09-11T22:07:21.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.171+0000 7f04d1ffb640 1 -- 172.21.15.1:0/3337235176 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f04d8004220 con 0x55f2202b7b40 2024-09-11T22:07:21.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.171+0000 7f04f2676740 10 client.17817 fetch_fsmap learned FSMap version 48 2024-09-11T22:07:21.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.171+0000 7f04f2676740 1 -- 172.21.15.1:0/3337235176 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55f22033be70 con 0x55f2202b7b40 2024-09-11T22:07:21.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.171+0000 7f04d1ffb640 1 -- 172.21.15.1:0/3337235176 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04e4099f90 con 0x55f2202b7b40 2024-09-11T22:07:21.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.171+0000 7f04d1ffb640 1 -- 172.21.15.1:0/3337235176 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 48) ==== 8430+0+0 (secure 0 0 0) 0x7f04d8021070 con 0x55f2202b7b40 2024-09-11T22:07:21.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.171+0000 7f04f2676740 10 client.17817 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T22:07:21.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.171+0000 7f04f2676740 10 client.17817 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-11T22:07:21.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.171+0000 7f04f2676740 20 client.17817 populate_metadata read hostname 'smithi001' 2024-09-11T22:07:21.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.171+0000 7f04f2676740 1 --2- 172.21.15.1:0/3337235176 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x55f2202d0c50 0x55f22033e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:21.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.171+0000 7f04f2676740 4 client.17817 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:07:21.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.171+0000 7f04f2676740 1 -- 172.21.15.1:0/3337235176 --> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f220223d70 con 0x55f2202d0c50 2024-09-11T22:07:21.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.171+0000 7f04ee039640 1 --2- 172.21.15.1:0/3337235176 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x55f2202d0c50 0x55f22033e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:21.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.172+0000 7f04ee039640 1 --2- 172.21.15.1:0/3337235176 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x55f2202d0c50 0x55f22033e720 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f192ba0969a06d2d server_cookie=bcb773a06d37284 in_seq=0 out_seq=0 2024-09-11T22:07:21.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.172+0000 7f04d1ffb640 10 client.17817 ms_handle_connect on v2:172.21.15.53:6836/3457131830 2024-09-11T22:07:21.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.172+0000 7f04d1ffb640 1 -- 172.21.15.1:0/3337235176 <== mds.2 v2:172.21.15.53:6836/3457131830 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f220223d70 con 0x55f2202d0c50 2024-09-11T22:07:21.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.172+0000 7f04d1ffb640 10 client.17817 handle_command_reply: tid=0 2024-09-11T22:07:21.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.172+0000 7f04f2676740 1 client.17817 shutdown 2024-09-11T22:07:21.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.172+0000 7f04f2676740 1 -- 172.21.15.1:0/3337235176 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f2202b7b40 msgr2=0x55f2202c59b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:21.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.172+0000 7f04f2676740 1 --2- 172.21.15.1:0/3337235176 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f2202b7b40 0x55f2202c59b0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f04d8005d00 tx=0x7f04d8019970 comp rx=0 tx=0).stop 2024-09-11T22:07:21.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.172+0000 7f04f2676740 1 -- 172.21.15.1:0/3337235176 shutdown_connections 2024-09-11T22:07:21.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.172+0000 7f04f2676740 1 --2- 172.21.15.1:0/3337235176 >> [v2:172.21.15.53:6836/3457131830,v1:172.21.15.53:6837/3457131830] conn(0x55f2202d0c50 0x55f22033e720 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:21.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.172+0000 7f04f2676740 1 --2- 172.21.15.1:0/3337235176 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f2202c1370 0x55f2202c5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:21.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.172+0000 7f04f2676740 1 --2- 172.21.15.1:0/3337235176 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f2202b7b40 0x55f2202c59b0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:21.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.172+0000 7f04d1ffb640 0 client.17817 ms_handle_reset on v2:172.21.15.53:6836/3457131830 2024-09-11T22:07:21.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.172+0000 7f04f2676740 1 --2- 172.21.15.1:0/3337235176 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f2202b65d0 0x55f2202c5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:21.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.172+0000 7f04f2676740 1 -- 172.21.15.1:0/3337235176 >> 172.21.15.1:0/3337235176 conn(0x55f22027f310 msgr2=0x55f2202c0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:07:21.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.172+0000 7f04f2676740 1 -- 172.21.15.1:0/3337235176 shutdown_connections 2024-09-11T22:07:21.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.172+0000 7f04f2676740 1 -- 172.21.15.1:0/3337235176 wait complete. 2024-09-11T22:07:21.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.173+0000 7f04f2676740 20 client.17817 trim_cache size 0 max 16384 2024-09-11T22:07:21.195 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-11T22:07:21.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.173+0000 7f04f029d640 1 -- 172.21.15.1:0/3554529698 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f04d40a2d70 msgr2=0x7f04d40a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:21.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.173+0000 7f04f029d640 1 --2- 172.21.15.1:0/3554529698 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f04d40a2d70 0x7f04d40a5230 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f04e004d000 tx=0x7f04e004e000 comp rx=0 tx=0).stop 2024-09-11T22:07:21.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.173+0000 7f04f029d640 1 -- 172.21.15.1:0/3554529698 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04e8102d40 msgr2=0x7f04e8198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:21.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.173+0000 7f04f029d640 1 --2- 172.21.15.1:0/3554529698 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04e8102d40 0x7f04e8198450 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f04e400e9f0 tx=0x7f04e400eeb0 comp rx=0 tx=0).stop 2024-09-11T22:07:21.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.173+0000 7f04f029d640 1 -- 172.21.15.1:0/3554529698 shutdown_connections 2024-09-11T22:07:21.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.173+0000 7f04f029d640 1 --2- 172.21.15.1:0/3554529698 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f04d40a2d70 0x7f04d40a5230 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:21.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.173+0000 7f04f029d640 1 --2- 172.21.15.1:0/3554529698 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f04e810c170 0x7f04e819cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:21.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.173+0000 7f04f029d640 1 --2- 172.21.15.1:0/3554529698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f04e8103660 0x7f04e8198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:21.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.173+0000 7f04f029d640 1 --2- 172.21.15.1:0/3554529698 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04e8102d40 0x7f04e8198450 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:21.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.173+0000 7f04f029d640 1 -- 172.21.15.1:0/3554529698 >> 172.21.15.1:0/3554529698 conn(0x7f04e80fc7a0 msgr2=0x7f04e80fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:07:21.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.173+0000 7f04f029d640 1 -- 172.21.15.1:0/3554529698 shutdown_connections 2024-09-11T22:07:21.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:21.173+0000 7f04f029d640 1 -- 172.21.15.1:0/3554529698 wait complete. 2024-09-11T22:07:21.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:21 smithi001 ceph-mon[30935]: pgmap v314: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 14 op/s 2024-09-11T22:07:21.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:21 smithi053 ceph-mon[34908]: pgmap v314: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 14 op/s 2024-09-11T22:07:21.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:21 smithi161 ceph-mon[35615]: pgmap v314: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 14 op/s 2024-09-11T22:07:21.944 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T22:07:22.212 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:07:22.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.703+0000 7f888fed3640 1 -- 172.21.15.1:0/2086130743 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f888810e3c0 msgr2=0x7f8888110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:22.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.703+0000 7f888fed3640 1 --2- 172.21.15.1:0/2086130743 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f888810e3c0 0x7f8888110810 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f8880009ca0 tx=0x7f888002a950 comp rx=0 tx=0).stop 2024-09-11T22:07:22.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.703+0000 7f888fed3640 1 -- 172.21.15.1:0/2086130743 shutdown_connections 2024-09-11T22:07:22.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.703+0000 7f888fed3640 1 --2- 172.21.15.1:0/2086130743 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f888810e3c0 0x7f8888110810 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:22.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.703+0000 7f888fed3640 1 --2- 172.21.15.1:0/2086130743 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8888074f40 0x7f888810de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:22.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.703+0000 7f888fed3640 1 --2- 172.21.15.1:0/2086130743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8888074620 0x7f8888074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:22.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.703+0000 7f888fed3640 1 -- 172.21.15.1:0/2086130743 >> 172.21.15.1:0/2086130743 conn(0x7f88880fd8c0 msgr2=0x7f88880ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:07:22.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.703+0000 7f888fed3640 1 -- 172.21.15.1:0/2086130743 shutdown_connections 2024-09-11T22:07:22.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.703+0000 7f888fed3640 1 -- 172.21.15.1:0/2086130743 wait complete. 2024-09-11T22:07:22.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.704+0000 7f888fed3640 1 Processor -- start 2024-09-11T22:07:22.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.704+0000 7f888fed3640 1 -- start start 2024-09-11T22:07:22.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.704+0000 7f888fed3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8888074620 0x7f888819a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:22.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.704+0000 7f888fed3640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8888074f40 0x7f888819abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:22.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.704+0000 7f888fed3640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f888810e3c0 0x7f888819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:22.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.704+0000 7f888fed3640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8888114170 con 0x7f8888074620 2024-09-11T22:07:22.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.704+0000 7f888fed3640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f8888113ff0 con 0x7f8888074f40 2024-09-11T22:07:22.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.704+0000 7f888fed3640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f88881142f0 con 0x7f888810e3c0 2024-09-11T22:07:22.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.704+0000 7f888dc6f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8888074620 0x7f888819a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:22.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.704+0000 7f888dc6f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8888074620 0x7f888819a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41956/0 (socket says 172.21.15.1:41956) 2024-09-11T22:07:22.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.704+0000 7f888dc6f640 1 -- 172.21.15.1:0/2216626690 learned_addr learned my addr 172.21.15.1:0/2216626690 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:07:22.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.705+0000 7f888d46e640 1 --2- 172.21.15.1:0/2216626690 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8888074f40 0x7f888819abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:22.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.705+0000 7f888e470640 1 --2- 172.21.15.1:0/2216626690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f888810e3c0 0x7f888819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:22.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.705+0000 7f888dc6f640 1 -- 172.21.15.1:0/2216626690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f888810e3c0 msgr2=0x7f888819ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:22.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.705+0000 7f888dc6f640 1 --2- 172.21.15.1:0/2216626690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f888810e3c0 0x7f888819ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:22.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.705+0000 7f888dc6f640 1 -- 172.21.15.1:0/2216626690 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8888074f40 msgr2=0x7f888819abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:22.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.705+0000 7f888dc6f640 1 --2- 172.21.15.1:0/2216626690 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8888074f40 0x7f888819abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:22.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.705+0000 7f888dc6f640 1 -- 172.21.15.1:0/2216626690 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f888819f620 con 0x7f8888074620 2024-09-11T22:07:22.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.705+0000 7f888e470640 1 --2- 172.21.15.1:0/2216626690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f888810e3c0 0x7f888819ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:07:22.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.705+0000 7f888dc6f640 1 --2- 172.21.15.1:0/2216626690 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8888074620 0x7f888819a670 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f888400c890 tx=0x7f888400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:07:22.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.705+0000 7f887effd640 1 -- 172.21.15.1:0/2216626690 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8884004220 con 0x7f8888074620 2024-09-11T22:07:22.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.705+0000 7f887effd640 1 -- 172.21.15.1:0/2216626690 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88840043c0 con 0x7f8888074620 2024-09-11T22:07:22.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.705+0000 7f888fed3640 1 -- 172.21.15.1:0/2216626690 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f888819f910 con 0x7f8888074620 2024-09-11T22:07:22.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.705+0000 7f887effd640 1 -- 172.21.15.1:0/2216626690 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88840049e0 con 0x7f8888074620 2024-09-11T22:07:22.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.705+0000 7f888fed3640 1 -- 172.21.15.1:0/2216626690 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88881a71f0 con 0x7f8888074620 2024-09-11T22:07:22.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.707+0000 7f887effd640 1 -- 172.21.15.1:0/2216626690 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f88840079b0 con 0x7f8888074620 2024-09-11T22:07:22.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.708+0000 7f88922ac740 1 Processor -- start 2024-09-11T22:07:22.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.708+0000 7f88922ac740 1 -- start start 2024-09-11T22:07:22.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.708+0000 7f88922ac740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564d7ec89b40 0x564d7ec88090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:22.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.708+0000 7f88922ac740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d7ec885d0 0x564d7ec88a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:22.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.708+0000 7f88922ac740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d7ec93370 0x564d7ec95760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:22.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.708+0000 7f88922ac740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x564d7ebee2f0 con 0x564d7ec93370 2024-09-11T22:07:22.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.708+0000 7f88922ac740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x564d7eb95cf0 con 0x564d7ec885d0 2024-09-11T22:07:22.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.708+0000 7f88922ac740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x564d7ebea2b0 con 0x564d7ec89b40 2024-09-11T22:07:22.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.708+0000 7f888d46e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d7ec93370 0x564d7ec95760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:22.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.708+0000 7f888d46e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d7ec93370 0x564d7ec95760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41966/0 (socket says 172.21.15.1:41966) 2024-09-11T22:07:22.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.708+0000 7f888d46e640 1 -- 172.21.15.1:0/884727464 learned_addr learned my addr 172.21.15.1:0/884727464 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:07:22.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.708+0000 7f888dc6f640 1 --2- 172.21.15.1:0/884727464 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d7ec885d0 0x564d7ec88a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:22.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.708+0000 7f887effd640 1 --2- 172.21.15.1:0/2216626690 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f88740a2d70 0x7f88740a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:22.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.709+0000 7f888e470640 1 --2- 172.21.15.1:0/884727464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564d7ec89b40 0x564d7ec88090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:22.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.709+0000 7f888e470640 1 --2- 172.21.15.1:0/2216626690 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f88740a2d70 0x7f88740a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:22.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.709+0000 7f888d46e640 1 -- 172.21.15.1:0/884727464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564d7ec89b40 msgr2=0x564d7ec88090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:07:22.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.709+0000 7f888d46e640 1 --2- 172.21.15.1:0/884727464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564d7ec89b40 0x564d7ec88090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:22.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.709+0000 7f888d46e640 1 -- 172.21.15.1:0/884727464 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d7ec885d0 msgr2=0x564d7ec88a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:07:22.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.709+0000 7f888d46e640 1 --2- 172.21.15.1:0/884727464 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d7ec885d0 0x564d7ec88a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:22.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.709+0000 7f888d46e640 1 -- 172.21.15.1:0/884727464 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d7eb7a3a0 con 0x564d7ec93370 2024-09-11T22:07:22.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.709+0000 7f887effd640 1 -- 172.21.15.1:0/2216626690 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7f88840cf510 con 0x7f8888074620 2024-09-11T22:07:22.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.709+0000 7f888d46e640 1 --2- 172.21.15.1:0/884727464 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d7ec93370 0x564d7ec95760 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f887800ab60 tx=0x7f887800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31dfda31f75d5ee9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:07:22.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.709+0000 7f8873fff640 1 -- 172.21.15.1:0/884727464 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f887801d070 con 0x564d7ec93370 2024-09-11T22:07:22.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.709+0000 7f888e470640 1 --2- 172.21.15.1:0/2216626690 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f88740a2d70 0x7f88740a5230 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8880030040 tx=0x7f88800095c0 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:07:22.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.709+0000 7f8873fff640 1 -- 172.21.15.1:0/884727464 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8878018b20 con 0x564d7ec93370 2024-09-11T22:07:22.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.709+0000 7f8873fff640 1 -- 172.21.15.1:0/884727464 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8878010b90 con 0x564d7ec93370 2024-09-11T22:07:22.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.710+0000 7f88922ac740 1 -- 172.21.15.1:0/884727464 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d7ec93370 msgr2=0x564d7ec95760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:22.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.710+0000 7f88922ac740 1 --2- 172.21.15.1:0/884727464 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d7ec93370 0x564d7ec95760 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f887800ab60 tx=0x7f887800f790 comp rx=0 tx=0).stop 2024-09-11T22:07:22.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.710+0000 7f88922ac740 1 -- 172.21.15.1:0/884727464 shutdown_connections 2024-09-11T22:07:22.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.710+0000 7f88922ac740 1 --2- 172.21.15.1:0/884727464 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d7ec93370 0x564d7ec95760 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:22.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.710+0000 7f88922ac740 1 --2- 172.21.15.1:0/884727464 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d7ec885d0 0x564d7ec88a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:22.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.710+0000 7f88922ac740 1 --2- 172.21.15.1:0/884727464 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564d7ec89b40 0x564d7ec88090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:22.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.710+0000 7f88922ac740 1 -- 172.21.15.1:0/884727464 >> 172.21.15.1:0/884727464 conn(0x564d7ec51310 msgr2=0x564d7ec84c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:07:22.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.710+0000 7f88922ac740 1 -- 172.21.15.1:0/884727464 shutdown_connections 2024-09-11T22:07:22.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.710+0000 7f88922ac740 1 -- 172.21.15.1:0/884727464 wait complete. 2024-09-11T22:07:22.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.710+0000 7f88922ac740 1 -- start start 2024-09-11T22:07:22.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.711+0000 7f88922ac740 1 Processor -- start 2024-09-11T22:07:22.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.711+0000 7f88922ac740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d7ec885d0 0x564d7ec92d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:22.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.711+0000 7f88922ac740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564d7ec89b40 0x564d7ec90000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:22.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.711+0000 7f88922ac740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d7ec93370 0x564d7ec905d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:22.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.711+0000 7f88922ac740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x564d7ec96070 con 0x564d7ec885d0 2024-09-11T22:07:22.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.711+0000 7f88922ac740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x564d7eba9060 con 0x564d7ec93370 2024-09-11T22:07:22.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.711+0000 7f88922ac740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x564d7ec961f0 con 0x564d7ec89b40 2024-09-11T22:07:22.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.711+0000 7f88922ac740 10 client.0 _ll_register_callbacks cb 0x564d7e8fc8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:07:22.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.711+0000 7f888dc6f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d7ec885d0 0x564d7ec92d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:22.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.711+0000 7f888dc6f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d7ec885d0 0x564d7ec92d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41974/0 (socket says 172.21.15.1:41974) 2024-09-11T22:07:22.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.711+0000 7f888dc6f640 1 -- 172.21.15.1:0/1481907087 learned_addr learned my addr 172.21.15.1:0/1481907087 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:07:22.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.711+0000 7f888d46e640 1 --2- 172.21.15.1:0/1481907087 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564d7ec89b40 0x564d7ec90000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:22.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.711+0000 7f888e470640 1 --2- 172.21.15.1:0/1481907087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d7ec93370 0x564d7ec905d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:22.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.711+0000 7f888dc6f640 1 -- 172.21.15.1:0/1481907087 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564d7ec89b40 msgr2=0x564d7ec90000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:07:22.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.711+0000 7f888dc6f640 1 --2- 172.21.15.1:0/1481907087 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564d7ec89b40 0x564d7ec90000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:22.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.711+0000 7f888dc6f640 1 -- 172.21.15.1:0/1481907087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d7ec93370 msgr2=0x564d7ec905d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:07:22.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.711+0000 7f888dc6f640 1 --2- 172.21.15.1:0/1481907087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d7ec93370 0x564d7ec905d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:22.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.711+0000 7f888dc6f640 1 -- 172.21.15.1:0/1481907087 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d7e95f3c0 con 0x564d7ec885d0 2024-09-11T22:07:22.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.711+0000 7f888d46e640 1 --2- 172.21.15.1:0/1481907087 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564d7ec89b40 0x564d7ec90000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:07:22.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.711+0000 7f888e470640 1 --2- 172.21.15.1:0/1481907087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d7ec93370 0x564d7ec905d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:07:22.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.712+0000 7f888dc6f640 1 --2- 172.21.15.1:0/1481907087 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d7ec885d0 0x564d7ec92d50 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f888409b690 tx=0x7f888409b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e5f052ba71dc5ff server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:07:22.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.712+0000 7f88717fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:07:22.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.712+0000 7f88717fa640 1 -- 172.21.15.1:0/1481907087 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f888401d070 con 0x564d7ec885d0 2024-09-11T22:07:22.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.712+0000 7f88717fa640 1 -- 172.21.15.1:0/1481907087 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7f88840976e0 con 0x564d7ec885d0 2024-09-11T22:07:22.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.712+0000 7f88922ac740 1 -- 172.21.15.1:0/1481907087 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8878018b20 con 0x564d7ec885d0 2024-09-11T22:07:22.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.712+0000 7f88717fa640 1 -- 172.21.15.1:0/1481907087 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f888409c090 con 0x564d7ec885d0 2024-09-11T22:07:22.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.712+0000 7f88922ac740 10 client.17829 fetch_fsmap learned FSMap version 48 2024-09-11T22:07:22.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.712+0000 7f88922ac740 1 -- 172.21.15.1:0/1481907087 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x564d7eca1b50 con 0x564d7ec885d0 2024-09-11T22:07:22.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.712+0000 7f88717fa640 1 -- 172.21.15.1:0/1481907087 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8878018b20 con 0x564d7ec885d0 2024-09-11T22:07:22.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.712+0000 7f88717fa640 1 -- 172.21.15.1:0/1481907087 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 48) ==== 8430+0+0 (secure 0 0 0) 0x7f88840cce40 con 0x564d7ec885d0 2024-09-11T22:07:22.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.712+0000 7f88922ac740 10 client.17829 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T22:07:22.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.712+0000 7f88922ac740 10 client.17829 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T22:07:22.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.712+0000 7f88922ac740 20 client.17829 populate_metadata read hostname 'smithi001' 2024-09-11T22:07:22.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.712+0000 7f88922ac740 1 --2- 172.21.15.1:0/1481907087 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x564d7eca2f60 0x564d7eca5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:22.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.712+0000 7f88922ac740 4 client.17829 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:07:22.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.712+0000 7f88922ac740 1 -- 172.21.15.1:0/1481907087 --> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d7ebfb860 con 0x564d7eca2f60 2024-09-11T22:07:22.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.713+0000 7f888d46e640 1 --2- 172.21.15.1:0/1481907087 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x564d7eca2f60 0x564d7eca5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:22.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.713+0000 7f888d46e640 1 --2- 172.21.15.1:0/1481907087 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x564d7eca2f60 0x564d7eca5400 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2d7eae4379cb9bfa server_cookie=3d5d214264f166f7 in_seq=0 out_seq=0 2024-09-11T22:07:22.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.713+0000 7f88717fa640 10 client.17829 ms_handle_connect on v2:172.21.15.161:6838/455365383 2024-09-11T22:07:22.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.714+0000 7f88717fa640 1 -- 172.21.15.1:0/1481907087 <== mds.1 v2:172.21.15.161:6838/455365383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564d7ebfb860 con 0x564d7eca2f60 2024-09-11T22:07:22.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.714+0000 7f88717fa640 10 client.17829 handle_command_reply: tid=0 2024-09-11T22:07:22.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.714+0000 7f88922ac740 1 client.17829 shutdown 2024-09-11T22:07:22.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.714+0000 7f88922ac740 1 -- 172.21.15.1:0/1481907087 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d7ec885d0 msgr2=0x564d7ec92d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:22.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.714+0000 7f88922ac740 1 --2- 172.21.15.1:0/1481907087 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d7ec885d0 0x564d7ec92d50 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f888409b690 tx=0x7f888409b7c0 comp rx=0 tx=0).stop 2024-09-11T22:07:22.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.714+0000 7f88922ac740 1 -- 172.21.15.1:0/1481907087 shutdown_connections 2024-09-11T22:07:22.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.714+0000 7f88922ac740 1 --2- 172.21.15.1:0/1481907087 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x564d7eca2f60 0x564d7eca5400 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:22.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.714+0000 7f88922ac740 1 --2- 172.21.15.1:0/1481907087 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d7ec93370 0x564d7ec905d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:22.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.714+0000 7f88922ac740 1 --2- 172.21.15.1:0/1481907087 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564d7ec89b40 0x564d7ec90000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:22.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.714+0000 7f88922ac740 1 --2- 172.21.15.1:0/1481907087 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d7ec885d0 0x564d7ec92d50 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:22.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.714+0000 7f88717fa640 0 client.17829 ms_handle_reset on v2:172.21.15.161:6838/455365383 2024-09-11T22:07:22.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.714+0000 7f88922ac740 1 -- 172.21.15.1:0/1481907087 >> 172.21.15.1:0/1481907087 conn(0x564d7ec51310 msgr2=0x564d7ec8a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:07:22.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.714+0000 7f88922ac740 1 -- 172.21.15.1:0/1481907087 shutdown_connections 2024-09-11T22:07:22.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.714+0000 7f88922ac740 1 -- 172.21.15.1:0/1481907087 wait complete. 2024-09-11T22:07:22.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.715+0000 7f88922ac740 20 client.17829 trim_cache size 0 max 16384 2024-09-11T22:07:22.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.727+0000 7f88922ac740 1 Processor -- start 2024-09-11T22:07:22.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.728+0000 7f88922ac740 1 -- start start 2024-09-11T22:07:22.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.728+0000 7f88922ac740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564d7ec885d0 0x564d7ec91f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:22.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.728+0000 7f88922ac740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d7ec89b40 0x564d7ec94760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:22.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.728+0000 7f88922ac740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d7ec93370 0x564d7ec94ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:22.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.728+0000 7f88922ac740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f88881afcf0 con 0x564d7ec93370 2024-09-11T22:07:22.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.728+0000 7f88922ac740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f88881b1aa0 con 0x564d7ec89b40 2024-09-11T22:07:22.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.728+0000 7f88922ac740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f88881b0480 con 0x564d7ec885d0 2024-09-11T22:07:22.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.728+0000 7f888e470640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d7ec93370 0x564d7ec94ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:22.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.728+0000 7f888e470640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d7ec93370 0x564d7ec94ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41976/0 (socket says 172.21.15.1:41976) 2024-09-11T22:07:22.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.728+0000 7f888e470640 1 -- 172.21.15.1:0/1436499421 learned_addr learned my addr 172.21.15.1:0/1436499421 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:07:22.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.728+0000 7f888d46e640 1 --2- 172.21.15.1:0/1436499421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d7ec89b40 0x564d7ec94760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:22.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.728+0000 7f888e470640 1 -- 172.21.15.1:0/1436499421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564d7ec885d0 msgr2=0x564d7ec91f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:07:22.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.728+0000 7f888dc6f640 1 --2- 172.21.15.1:0/1436499421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564d7ec885d0 0x564d7ec91f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:22.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.728+0000 7f888e470640 1 --2- 172.21.15.1:0/1436499421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564d7ec885d0 0x564d7ec91f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:22.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.728+0000 7f888e470640 1 -- 172.21.15.1:0/1436499421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d7ec89b40 msgr2=0x564d7ec94760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:07:22.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.728+0000 7f888e470640 1 --2- 172.21.15.1:0/1436499421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d7ec89b40 0x564d7ec94760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:22.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.728+0000 7f888e470640 1 -- 172.21.15.1:0/1436499421 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d7ec84ed0 con 0x564d7ec93370 2024-09-11T22:07:22.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.728+0000 7f888dc6f640 1 --2- 172.21.15.1:0/1436499421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564d7ec885d0 0x564d7ec91f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:07:22.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.728+0000 7f888d46e640 1 --2- 172.21.15.1:0/1436499421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d7ec89b40 0x564d7ec94760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:07:22.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.728+0000 7f888e470640 1 --2- 172.21.15.1:0/1436499421 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d7ec93370 0x564d7ec94ca0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f888002f550 tx=0x7f888003b070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8108f20be07429b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:07:22.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.728+0000 7f88737fe640 1 -- 172.21.15.1:0/1436499421 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f888004e020 con 0x564d7ec93370 2024-09-11T22:07:22.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.729+0000 7f88737fe640 1 -- 172.21.15.1:0/1436499421 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8880004520 con 0x564d7ec93370 2024-09-11T22:07:22.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.729+0000 7f88737fe640 1 -- 172.21.15.1:0/1436499421 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8880034040 con 0x564d7ec93370 2024-09-11T22:07:22.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.729+0000 7f88922ac740 1 -- 172.21.15.1:0/1436499421 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d7ec93370 msgr2=0x564d7ec94ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:22.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.729+0000 7f88922ac740 1 --2- 172.21.15.1:0/1436499421 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d7ec93370 0x564d7ec94ca0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f888002f550 tx=0x7f888003b070 comp rx=0 tx=0).stop 2024-09-11T22:07:22.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.729+0000 7f88922ac740 1 -- 172.21.15.1:0/1436499421 shutdown_connections 2024-09-11T22:07:22.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.729+0000 7f88922ac740 1 --2- 172.21.15.1:0/1436499421 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d7ec93370 0x564d7ec94ca0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:22.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.729+0000 7f88922ac740 1 --2- 172.21.15.1:0/1436499421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d7ec89b40 0x564d7ec94760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:22.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.729+0000 7f88922ac740 1 --2- 172.21.15.1:0/1436499421 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564d7ec885d0 0x564d7ec91f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:22.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.729+0000 7f88922ac740 1 -- 172.21.15.1:0/1436499421 >> 172.21.15.1:0/1436499421 conn(0x564d7ec51310 msgr2=0x564d7ec91860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:07:22.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.729+0000 7f88922ac740 1 -- 172.21.15.1:0/1436499421 shutdown_connections 2024-09-11T22:07:22.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.729+0000 7f88922ac740 1 -- 172.21.15.1:0/1436499421 wait complete. 2024-09-11T22:07:22.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.729+0000 7f88922ac740 1 -- start start 2024-09-11T22:07:22.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.730+0000 7f88922ac740 1 Processor -- start 2024-09-11T22:07:22.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.730+0000 7f88922ac740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d7ec885d0 0x564d7ec9b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:22.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.730+0000 7f88922ac740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564d7ec89b40 0x564d7ecaa360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:22.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.730+0000 7f88922ac740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d7ec93370 0x564d7ecaa8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:22.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.730+0000 7f88922ac740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f88881b13a0 con 0x564d7ec885d0 2024-09-11T22:07:22.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.730+0000 7f88922ac740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f88881b0c10 con 0x564d7ec93370 2024-09-11T22:07:22.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.730+0000 7f88922ac740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f88881adcd0 con 0x564d7ec89b40 2024-09-11T22:07:22.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.730+0000 7f88922ac740 10 client.0 _ll_register_callbacks cb 0x7f8884004570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:07:22.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.730+0000 7f888dc6f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d7ec885d0 0x564d7ec9b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:22.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.730+0000 7f888dc6f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d7ec885d0 0x564d7ec9b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41978/0 (socket says 172.21.15.1:41978) 2024-09-11T22:07:22.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.730+0000 7f888dc6f640 1 -- 172.21.15.1:0/2848579158 learned_addr learned my addr 172.21.15.1:0/2848579158 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:07:22.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.730+0000 7f888d46e640 1 --2- 172.21.15.1:0/2848579158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564d7ec89b40 0x564d7ecaa360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:22.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.730+0000 7f888e470640 1 --2- 172.21.15.1:0/2848579158 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d7ec93370 0x564d7ecaa8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:22.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.730+0000 7f888dc6f640 1 -- 172.21.15.1:0/2848579158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564d7ec89b40 msgr2=0x564d7ecaa360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:07:22.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.730+0000 7f888dc6f640 1 --2- 172.21.15.1:0/2848579158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564d7ec89b40 0x564d7ecaa360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:22.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.730+0000 7f888dc6f640 1 -- 172.21.15.1:0/2848579158 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d7ec93370 msgr2=0x564d7ecaa8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:07:22.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.730+0000 7f888dc6f640 1 --2- 172.21.15.1:0/2848579158 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d7ec93370 0x564d7ecaa8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:22.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.730+0000 7f888dc6f640 1 -- 172.21.15.1:0/2848579158 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d7ecaae10 con 0x564d7ec885d0 2024-09-11T22:07:22.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.730+0000 7f888e470640 1 --2- 172.21.15.1:0/2848579158 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d7ec93370 0x564d7ecaa8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:07:22.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.731+0000 7f888dc6f640 1 --2- 172.21.15.1:0/2848579158 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d7ec885d0 0x564d7ec9b950 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f8884005fd0 tx=0x7f88840cf790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd5a7d520a4d05b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:07:22.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.731+0000 7f88717fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:07:22.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.731+0000 7f88717fa640 1 -- 172.21.15.1:0/2848579158 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f888401d070 con 0x564d7ec885d0 2024-09-11T22:07:22.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.731+0000 7f88717fa640 1 -- 172.21.15.1:0/2848579158 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7f88840862c0 con 0x564d7ec885d0 2024-09-11T22:07:22.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.731+0000 7f88922ac740 1 -- 172.21.15.1:0/2848579158 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8880004520 con 0x564d7ec885d0 2024-09-11T22:07:22.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.731+0000 7f88717fa640 1 -- 172.21.15.1:0/2848579158 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f888409b990 con 0x564d7ec885d0 2024-09-11T22:07:22.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.731+0000 7f88922ac740 10 client.17835 fetch_fsmap learned FSMap version 48 2024-09-11T22:07:22.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.731+0000 7f88922ac740 1 -- 172.21.15.1:0/2848579158 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x564d7ec9a1b0 con 0x564d7ec885d0 2024-09-11T22:07:22.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.731+0000 7f88717fa640 1 -- 172.21.15.1:0/2848579158 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88840ba1a0 con 0x564d7ec885d0 2024-09-11T22:07:22.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.731+0000 7f88717fa640 1 -- 172.21.15.1:0/2848579158 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 48) ==== 8430+0+0 (secure 0 0 0) 0x7f888409aee0 con 0x564d7ec885d0 2024-09-11T22:07:22.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.731+0000 7f88922ac740 10 client.17835 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T22:07:22.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.731+0000 7f88922ac740 10 client.17835 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T22:07:22.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.731+0000 7f88922ac740 20 client.17835 populate_metadata read hostname 'smithi001' 2024-09-11T22:07:22.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.731+0000 7f88922ac740 1 --2- 172.21.15.1:0/2848579158 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x564d7ed0e010 0x564d7ed10430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:22.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.731+0000 7f88922ac740 4 client.17835 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:07:22.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.731+0000 7f88922ac740 1 -- 172.21.15.1:0/2848579158 --> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f8880004520 con 0x564d7ed0e010 2024-09-11T22:07:22.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.732+0000 7f888d46e640 1 --2- 172.21.15.1:0/2848579158 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x564d7ed0e010 0x564d7ed10430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:22.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.732+0000 7f888d46e640 1 --2- 172.21.15.1:0/2848579158 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x564d7ed0e010 0x564d7ed10430 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=48efa053b450640c server_cookie=dbe8f32645c66eb1 in_seq=0 out_seq=0 2024-09-11T22:07:22.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.732+0000 7f88717fa640 10 client.17835 ms_handle_connect on v2:172.21.15.161:6838/455365383 2024-09-11T22:07:22.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.732+0000 7f88717fa640 1 -- 172.21.15.1:0/2848579158 <== mds.1 v2:172.21.15.161:6838/455365383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f8880004520 con 0x564d7ed0e010 2024-09-11T22:07:22.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.732+0000 7f88717fa640 10 client.17835 handle_command_reply: tid=0 2024-09-11T22:07:22.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.732+0000 7f88922ac740 1 client.17835 shutdown 2024-09-11T22:07:22.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.733+0000 7f88922ac740 1 -- 172.21.15.1:0/2848579158 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d7ec885d0 msgr2=0x564d7ec9b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:22.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.733+0000 7f88922ac740 1 --2- 172.21.15.1:0/2848579158 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d7ec885d0 0x564d7ec9b950 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f8884005fd0 tx=0x7f88840cf790 comp rx=0 tx=0).stop 2024-09-11T22:07:22.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.733+0000 7f88922ac740 1 -- 172.21.15.1:0/2848579158 shutdown_connections 2024-09-11T22:07:22.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.733+0000 7f88922ac740 1 --2- 172.21.15.1:0/2848579158 >> [v2:172.21.15.161:6838/455365383,v1:172.21.15.161:6839/455365383] conn(0x564d7ed0e010 0x564d7ed10430 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:22.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.733+0000 7f88922ac740 1 --2- 172.21.15.1:0/2848579158 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d7ec93370 0x564d7ecaa8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:22.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.733+0000 7f88922ac740 1 --2- 172.21.15.1:0/2848579158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x564d7ec89b40 0x564d7ecaa360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:22.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.733+0000 7f88717fa640 0 client.17835 ms_handle_reset on v2:172.21.15.161:6838/455365383 2024-09-11T22:07:22.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.733+0000 7f88922ac740 1 --2- 172.21.15.1:0/2848579158 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d7ec885d0 0x564d7ec9b950 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:22.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.733+0000 7f88922ac740 1 -- 172.21.15.1:0/2848579158 >> 172.21.15.1:0/2848579158 conn(0x564d7ec51310 msgr2=0x564d7ec83d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:07:22.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.733+0000 7f88922ac740 1 -- 172.21.15.1:0/2848579158 shutdown_connections 2024-09-11T22:07:22.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.733+0000 7f88922ac740 1 -- 172.21.15.1:0/2848579158 wait complete. 2024-09-11T22:07:22.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.733+0000 7f88922ac740 20 client.17835 trim_cache size 0 max 16384 2024-09-11T22:07:22.769 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-11T22:07:22.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.733+0000 7f888fed3640 1 -- 172.21.15.1:0/2216626690 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f88740a2d70 msgr2=0x7f88740a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:22.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.733+0000 7f888fed3640 1 --2- 172.21.15.1:0/2216626690 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f88740a2d70 0x7f88740a5230 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8880030040 tx=0x7f88800095c0 comp rx=0 tx=0).stop 2024-09-11T22:07:22.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.733+0000 7f888fed3640 1 -- 172.21.15.1:0/2216626690 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8888074620 msgr2=0x7f888819a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:22.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.733+0000 7f888fed3640 1 --2- 172.21.15.1:0/2216626690 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8888074620 0x7f888819a670 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f888400c890 tx=0x7f888400cd50 comp rx=0 tx=0).stop 2024-09-11T22:07:22.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.733+0000 7f888fed3640 1 -- 172.21.15.1:0/2216626690 shutdown_connections 2024-09-11T22:07:22.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.733+0000 7f888fed3640 1 --2- 172.21.15.1:0/2216626690 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f88740a2d70 0x7f88740a5230 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:22.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.733+0000 7f888fed3640 1 --2- 172.21.15.1:0/2216626690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f888810e3c0 0x7f888819ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:22.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.733+0000 7f888fed3640 1 --2- 172.21.15.1:0/2216626690 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8888074f40 0x7f888819abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:22.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.733+0000 7f888fed3640 1 --2- 172.21.15.1:0/2216626690 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8888074620 0x7f888819a670 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:22.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.733+0000 7f888fed3640 1 -- 172.21.15.1:0/2216626690 >> 172.21.15.1:0/2216626690 conn(0x7f88880fd8c0 msgr2=0x7f88880fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:07:22.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.734+0000 7f888fed3640 1 -- 172.21.15.1:0/2216626690 shutdown_connections 2024-09-11T22:07:22.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:22.734+0000 7f888fed3640 1 -- 172.21.15.1:0/2216626690 wait complete. 2024-09-11T22:07:23.354 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:23 smithi001 ceph-mon[30935]: pgmap v315: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:07:23.383 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T22:07:23.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:23 smithi053 ceph-mon[34908]: pgmap v315: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:07:23.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:23 smithi161 ceph-mon[35615]: pgmap v315: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:07:23.645 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:07:23.655 DEBUG:teuthology.orchestra.run.smithi001:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-11T22:07:23.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:23.761+0000 7f9546c5e640 1 -- 172.21.15.1:0/1412807749 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9540101ac0 msgr2=0x7f9540101f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:23.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:23.761+0000 7f9546c5e640 1 --2- 172.21.15.1:0/1412807749 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9540101ac0 0x7f9540101f20 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f9534009930 tx=0x7f953402a420 comp rx=0 tx=0).stop 2024-09-11T22:07:23.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:23.761+0000 7f9546c5e640 1 -- 172.21.15.1:0/1412807749 shutdown_connections 2024-09-11T22:07:23.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:23.761+0000 7f9546c5e640 1 --2- 172.21.15.1:0/1412807749 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9540102460 0x7f954010e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:23.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:23.761+0000 7f9546c5e640 1 --2- 172.21.15.1:0/1412807749 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9540101ac0 0x7f9540101f20 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:23.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:23.761+0000 7f9546c5e640 1 --2- 172.21.15.1:0/1412807749 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9540107ad0 0x7f9540107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:23.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:23.761+0000 7f9546c5e640 1 -- 172.21.15.1:0/1412807749 >> 172.21.15.1:0/1412807749 conn(0x7f95400fd7e0 msgr2=0x7f95400ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:07:23.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:23.761+0000 7f9546c5e640 1 -- 172.21.15.1:0/1412807749 shutdown_connections 2024-09-11T22:07:23.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:23.761+0000 7f9546c5e640 1 -- 172.21.15.1:0/1412807749 wait complete. 2024-09-11T22:07:23.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:23.762+0000 7f9546c5e640 1 Processor -- start 2024-09-11T22:07:23.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:23.762+0000 7f9546c5e640 1 -- start start 2024-09-11T22:07:23.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:23.762+0000 7f9546c5e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9540101ac0 0x7f9540198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:23.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:23.762+0000 7f9546c5e640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9540102460 0x7f95401989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:23.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:23.762+0000 7f9546c5e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9540107ad0 0x7f954019cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:23.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:23.762+0000 7f9546c5e640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9540111f60 con 0x7f9540101ac0 2024-09-11T22:07:23.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:23.762+0000 7f9546c5e640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9540111de0 con 0x7f9540102460 2024-09-11T22:07:23.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:23.762+0000 7f9546c5e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f95401120e0 con 0x7f9540107ad0 2024-09-11T22:07:23.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:23.762+0000 7f95449fa640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9540101ac0 0x7f9540198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:23.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:23.762+0000 7f95449fa640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9540101ac0 0x7f9540198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41996/0 (socket says 172.21.15.1:41996) 2024-09-11T22:07:23.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:23.762+0000 7f95449fa640 1 -- 172.21.15.1:0/1727242847 learned_addr learned my addr 172.21.15.1:0/1727242847 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:07:23.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:23.762+0000 7f953ffff640 1 --2- 172.21.15.1:0/1727242847 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9540102460 0x7f95401989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:23.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:23.762+0000 7f95451fb640 1 --2- 172.21.15.1:0/1727242847 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9540107ad0 0x7f954019cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:23.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:23.762+0000 7f95449fa640 1 -- 172.21.15.1:0/1727242847 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9540107ad0 msgr2=0x7f954019cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:23.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:23.762+0000 7f95449fa640 1 --2- 172.21.15.1:0/1727242847 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9540107ad0 0x7f954019cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:23.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:23.762+0000 7f95449fa640 1 -- 172.21.15.1:0/1727242847 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9540102460 msgr2=0x7f95401989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:23.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:23.762+0000 7f95449fa640 1 --2- 172.21.15.1:0/1727242847 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9540102460 0x7f95401989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:23.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:23.762+0000 7f95449fa640 1 -- 172.21.15.1:0/1727242847 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f954019d470 con 0x7f9540101ac0 2024-09-11T22:07:23.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:23.762+0000 7f95449fa640 1 --2- 172.21.15.1:0/1727242847 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9540101ac0 0x7f9540198480 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f953000d950 tx=0x7f953000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:07:23.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:23.763+0000 7f953dffb640 1 -- 172.21.15.1:0/1727242847 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9530017070 con 0x7f9540101ac0 2024-09-11T22:07:23.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:23.763+0000 7f953dffb640 1 -- 172.21.15.1:0/1727242847 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9530009c30 con 0x7f9540101ac0 2024-09-11T22:07:23.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:23.763+0000 7f953dffb640 1 -- 172.21.15.1:0/1727242847 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953000f6b0 con 0x7f9540101ac0 2024-09-11T22:07:23.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:23.763+0000 7f9546c5e640 1 -- 172.21.15.1:0/1727242847 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f954019d760 con 0x7f9540101ac0 2024-09-11T22:07:23.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:23.763+0000 7f9546c5e640 1 -- 172.21.15.1:0/1727242847 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95401a4ff0 con 0x7f9540101ac0 2024-09-11T22:07:23.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:23.764+0000 7f9546c5e640 1 -- 172.21.15.1:0/1727242847 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9540103200 con 0x7f9540101ac0 2024-09-11T22:07:23.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:23.765+0000 7f953dffb640 1 -- 172.21.15.1:0/1727242847 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f95300109d0 con 0x7f9540101ac0 2024-09-11T22:07:23.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:23.766+0000 7f953dffb640 1 --2- 172.21.15.1:0/1727242847 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f95280a2cf0 0x7f95280a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:23.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:23.766+0000 7f953ffff640 1 --2- 172.21.15.1:0/1727242847 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f95280a2cf0 0x7f95280a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:23.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:23.766+0000 7f953dffb640 1 -- 172.21.15.1:0/1727242847 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7f95300cf0e0 con 0x7f9540101ac0 2024-09-11T22:07:23.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:23.766+0000 7f953ffff640 1 --2- 172.21.15.1:0/1727242847 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f95280a2cf0 0x7f95280a51b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9534006010 tx=0x7f9534000bc0 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:07:23.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:23.770+0000 7f953dffb640 1 -- 172.21.15.1:0/1727242847 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9530009de0 con 0x7f9540101ac0 2024-09-11T22:07:23.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:23.973+0000 7f9546c5e640 1 -- 172.21.15.1:0/1727242847 --> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f954010a0c0 con 0x7f95280a2cf0 2024-09-11T22:07:23.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:23.982+0000 7f953dffb640 1 -- 172.21.15.1:0/1727242847 <== mgr.15204 v2:172.21.15.1:6800/3543150772 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+349 (secure 0 0 0) 0x7f9534004270 con 0x7f95280a2cf0 2024-09-11T22:07:23.983 INFO:teuthology.orchestra.run.smithi001.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-11T00:00:00", "created": "2024-09-11T21:57:21", "first": "2024-09-11T21:58:00", "last": "2024-09-11T22:05:01", "last_pruned": "2024-09-11T22:04:01", "created_count": 8, "pruned_count": 1, "active": true} 2024-09-11T22:07:23.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:23.984+0000 7f9546c5e640 1 -- 172.21.15.1:0/1727242847 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f95280a2cf0 msgr2=0x7f95280a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:23.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:23.984+0000 7f9546c5e640 1 --2- 172.21.15.1:0/1727242847 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f95280a2cf0 0x7f95280a51b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9534006010 tx=0x7f9534000bc0 comp rx=0 tx=0).stop 2024-09-11T22:07:23.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:23.984+0000 7f9546c5e640 1 -- 172.21.15.1:0/1727242847 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9540101ac0 msgr2=0x7f9540198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:23.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:23.984+0000 7f9546c5e640 1 --2- 172.21.15.1:0/1727242847 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9540101ac0 0x7f9540198480 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f953000d950 tx=0x7f953000de10 comp rx=0 tx=0).stop 2024-09-11T22:07:23.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:23.984+0000 7f9546c5e640 1 -- 172.21.15.1:0/1727242847 shutdown_connections 2024-09-11T22:07:23.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:23.984+0000 7f9546c5e640 1 --2- 172.21.15.1:0/1727242847 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f95280a2cf0 0x7f95280a51b0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:23.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:23.984+0000 7f9546c5e640 1 --2- 172.21.15.1:0/1727242847 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9540107ad0 0x7f954019cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:23.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:23.984+0000 7f9546c5e640 1 --2- 172.21.15.1:0/1727242847 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9540102460 0x7f95401989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:23.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:23.984+0000 7f9546c5e640 1 --2- 172.21.15.1:0/1727242847 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9540101ac0 0x7f9540198480 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:23.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:23.984+0000 7f9546c5e640 1 -- 172.21.15.1:0/1727242847 >> 172.21.15.1:0/1727242847 conn(0x7f95400fd7e0 msgr2=0x7f95400fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:07:23.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:23.984+0000 7f9546c5e640 1 -- 172.21.15.1:0/1727242847 shutdown_connections 2024-09-11T22:07:23.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:23.984+0000 7f9546c5e640 1 -- 172.21.15.1:0/1727242847 wait complete. 2024-09-11T22:07:23.997 DEBUG:teuthology.orchestra.run.smithi001:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-11T22:07:24.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.101+0000 7fcf16d31640 1 -- 172.21.15.1:0/171650293 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcf101035c0 msgr2=0x7fcf1010baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:24.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.101+0000 7fcf16d31640 1 --2- 172.21.15.1:0/171650293 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcf101035c0 0x7fcf1010baa0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fcefc009900 tx=0x7fcefc02a480 comp rx=0 tx=0).stop 2024-09-11T22:07:24.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.101+0000 7fcf16d31640 1 -- 172.21.15.1:0/171650293 shutdown_connections 2024-09-11T22:07:24.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.101+0000 7fcf16d31640 1 --2- 172.21.15.1:0/171650293 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf1010c0f0 0x7fcf1010e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:24.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.101+0000 7fcf16d31640 1 --2- 172.21.15.1:0/171650293 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcf101035c0 0x7fcf1010baa0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:24.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.101+0000 7fcf16d31640 1 --2- 172.21.15.1:0/171650293 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf10102ca0 0x7fcf10103080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:24.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.101+0000 7fcf16d31640 1 -- 172.21.15.1:0/171650293 >> 172.21.15.1:0/171650293 conn(0x7fcf100fc6b0 msgr2=0x7fcf100fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:07:24.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.102+0000 7fcf16d31640 1 -- 172.21.15.1:0/171650293 shutdown_connections 2024-09-11T22:07:24.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.102+0000 7fcf16d31640 1 -- 172.21.15.1:0/171650293 wait complete. 2024-09-11T22:07:24.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.102+0000 7fcf16d31640 1 Processor -- start 2024-09-11T22:07:24.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.102+0000 7fcf16d31640 1 -- start start 2024-09-11T22:07:24.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.102+0000 7fcf16d31640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf10102ca0 0x7fcf10198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:24.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.102+0000 7fcf16d31640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf101035c0 0x7fcf101989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:24.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.102+0000 7fcf16d31640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcf1010c0f0 0x7fcf1019cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:24.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.102+0000 7fcf16d31640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcf10111ed0 con 0x7fcf1010c0f0 2024-09-11T22:07:24.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.102+0000 7fcf16d31640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcf10111d50 con 0x7fcf101035c0 2024-09-11T22:07:24.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.102+0000 7fcf16d31640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcf10112050 con 0x7fcf10102ca0 2024-09-11T22:07:24.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.103+0000 7fcf152ce640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcf1010c0f0 0x7fcf1019cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:24.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.103+0000 7fcf152ce640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcf1010c0f0 0x7fcf1019cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42020/0 (socket says 172.21.15.1:42020) 2024-09-11T22:07:24.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.103+0000 7fcf152ce640 1 -- 172.21.15.1:0/2939180009 learned_addr learned my addr 172.21.15.1:0/2939180009 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:07:24.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.103+0000 7fcf14acd640 1 --2- 172.21.15.1:0/2939180009 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf10102ca0 0x7fcf10198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:24.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.103+0000 7fcf0ffff640 1 --2- 172.21.15.1:0/2939180009 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf101035c0 0x7fcf101989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:24.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.103+0000 7fcf152ce640 1 -- 172.21.15.1:0/2939180009 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf10102ca0 msgr2=0x7fcf10198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:24.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.103+0000 7fcf152ce640 1 --2- 172.21.15.1:0/2939180009 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf10102ca0 0x7fcf10198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:24.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.103+0000 7fcf152ce640 1 -- 172.21.15.1:0/2939180009 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf101035c0 msgr2=0x7fcf101989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:24.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.103+0000 7fcf152ce640 1 --2- 172.21.15.1:0/2939180009 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf101035c0 0x7fcf101989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:24.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.103+0000 7fcf152ce640 1 -- 172.21.15.1:0/2939180009 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf1019d450 con 0x7fcf1010c0f0 2024-09-11T22:07:24.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.103+0000 7fcf0ffff640 1 --2- 172.21.15.1:0/2939180009 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf101035c0 0x7fcf101989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:07:24.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.103+0000 7fcf152ce640 1 --2- 172.21.15.1:0/2939180009 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcf1010c0f0 0x7fcf1019cd20 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fcf0800eaa0 tx=0x7fcf0800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:07:24.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.103+0000 7fcf0dffb640 1 -- 172.21.15.1:0/2939180009 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0800cdb0 con 0x7fcf1010c0f0 2024-09-11T22:07:24.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.103+0000 7fcf0dffb640 1 -- 172.21.15.1:0/2939180009 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf08005020 con 0x7fcf1010c0f0 2024-09-11T22:07:24.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.103+0000 7fcf16d31640 1 -- 172.21.15.1:0/2939180009 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf1019d740 con 0x7fcf1010c0f0 2024-09-11T22:07:24.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.103+0000 7fcf0dffb640 1 -- 172.21.15.1:0/2939180009 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf080116b0 con 0x7fcf1010c0f0 2024-09-11T22:07:24.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.103+0000 7fcf16d31640 1 -- 172.21.15.1:0/2939180009 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf101a4fd0 con 0x7fcf1010c0f0 2024-09-11T22:07:24.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.105+0000 7fcf16d31640 1 -- 172.21.15.1:0/2939180009 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf10103080 con 0x7fcf1010c0f0 2024-09-11T22:07:24.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.106+0000 7fcf0dffb640 1 -- 172.21.15.1:0/2939180009 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7fcf08005560 con 0x7fcf1010c0f0 2024-09-11T22:07:24.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.107+0000 7fcf0dffb640 1 --2- 172.21.15.1:0/2939180009 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fcf000a2cf0 0x7fcf000a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:24.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.107+0000 7fcf0dffb640 1 -- 172.21.15.1:0/2939180009 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7fcf080cf670 con 0x7fcf1010c0f0 2024-09-11T22:07:24.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.107+0000 7fcf14acd640 1 --2- 172.21.15.1:0/2939180009 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fcf000a2cf0 0x7fcf000a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:24.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.107+0000 7fcf14acd640 1 --2- 172.21.15.1:0/2939180009 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fcf000a2cf0 0x7fcf000a51b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fcf04009bd0 tx=0x7fcf04005f90 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:07:24.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.111+0000 7fcf0dffb640 1 -- 172.21.15.1:0/2939180009 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcf08016030 con 0x7fcf1010c0f0 2024-09-11T22:07:24.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.128+0000 7eff77065640 1 -- 172.21.15.1:0/414125072 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff70102d90 msgr2=0x7eff70103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:24.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.128+0000 7eff77065640 1 --2- 172.21.15.1:0/414125072 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff70102d90 0x7eff70103170 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7eff64009930 tx=0x7eff6402a420 comp rx=0 tx=0).stop 2024-09-11T22:07:24.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.128+0000 7eff77065640 1 -- 172.21.15.1:0/414125072 shutdown_connections 2024-09-11T22:07:24.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.128+0000 7eff77065640 1 --2- 172.21.15.1:0/414125072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff70108290 0x7eff7010ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:24.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.128+0000 7eff77065640 1 --2- 172.21.15.1:0/414125072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff70103740 0x7eff70107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:24.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.128+0000 7eff77065640 1 --2- 172.21.15.1:0/414125072 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff70102d90 0x7eff70103170 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:24.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.128+0000 7eff77065640 1 -- 172.21.15.1:0/414125072 >> 172.21.15.1:0/414125072 conn(0x7eff700fc7d0 msgr2=0x7eff700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:07:24.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.128+0000 7eff77065640 1 -- 172.21.15.1:0/414125072 shutdown_connections 2024-09-11T22:07:24.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.128+0000 7eff77065640 1 -- 172.21.15.1:0/414125072 wait complete. 2024-09-11T22:07:24.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.129+0000 7eff77065640 1 Processor -- start 2024-09-11T22:07:24.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.129+0000 7eff77065640 1 -- start start 2024-09-11T22:07:24.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.129+0000 7eff77065640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff70102d90 0x7eff701009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:24.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.129+0000 7eff77065640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff70103740 0x7eff70100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:24.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.129+0000 7eff77065640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff70108290 0x7eff70101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:24.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.129+0000 7eff77065640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7eff701120c0 con 0x7eff70103740 2024-09-11T22:07:24.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.129+0000 7eff77065640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7eff70111f40 con 0x7eff70102d90 2024-09-11T22:07:24.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.129+0000 7eff77065640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7eff70112240 con 0x7eff70108290 2024-09-11T22:07:24.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.129+0000 7eff6ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff70103740 0x7eff70100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:24.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.129+0000 7eff6ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff70103740 0x7eff70100f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42036/0 (socket says 172.21.15.1:42036) 2024-09-11T22:07:24.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.129+0000 7eff6ffff640 1 -- 172.21.15.1:0/1345573723 learned_addr learned my addr 172.21.15.1:0/1345573723 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:07:24.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.129+0000 7eff74e01640 1 --2- 172.21.15.1:0/1345573723 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff70102d90 0x7eff701009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:24.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.129+0000 7eff75602640 1 --2- 172.21.15.1:0/1345573723 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff70108290 0x7eff70101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:24.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.130+0000 7eff6ffff640 1 -- 172.21.15.1:0/1345573723 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff70108290 msgr2=0x7eff70101470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:24.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.130+0000 7eff6ffff640 1 --2- 172.21.15.1:0/1345573723 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff70108290 0x7eff70101470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:24.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.130+0000 7eff6ffff640 1 -- 172.21.15.1:0/1345573723 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff70102d90 msgr2=0x7eff701009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:24.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.130+0000 7eff6ffff640 1 --2- 172.21.15.1:0/1345573723 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff70102d90 0x7eff701009f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:24.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.130+0000 7eff6ffff640 1 -- 172.21.15.1:0/1345573723 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff701a6a50 con 0x7eff70103740 2024-09-11T22:07:24.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.130+0000 7eff74e01640 1 --2- 172.21.15.1:0/1345573723 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff70102d90 0x7eff701009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:07:24.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.130+0000 7eff6ffff640 1 --2- 172.21.15.1:0/1345573723 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff70103740 0x7eff70100f30 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7eff5c00e9c0 tx=0x7eff5c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:07:24.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.130+0000 7eff6dffb640 1 -- 172.21.15.1:0/1345573723 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5c00cd80 con 0x7eff70103740 2024-09-11T22:07:24.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.130+0000 7eff77065640 1 -- 172.21.15.1:0/1345573723 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff701a6d40 con 0x7eff70103740 2024-09-11T22:07:24.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.130+0000 7eff6dffb640 1 -- 172.21.15.1:0/1345573723 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff5c005500 con 0x7eff70103740 2024-09-11T22:07:24.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.130+0000 7eff6dffb640 1 -- 172.21.15.1:0/1345573723 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5c010690 con 0x7eff70103740 2024-09-11T22:07:24.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.131+0000 7eff77065640 1 -- 172.21.15.1:0/1345573723 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff701a7220 con 0x7eff70103740 2024-09-11T22:07:24.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.133+0000 7eff6dffb640 1 -- 172.21.15.1:0/1345573723 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7eff5c0046c0 con 0x7eff70103740 2024-09-11T22:07:24.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.133+0000 7eff6dffb640 1 --2- 172.21.15.1:0/1345573723 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7eff600a2cf0 0x7eff600a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:24.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.133+0000 7eff6dffb640 1 -- 172.21.15.1:0/1345573723 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7eff5c0cf870 con 0x7eff70103740 2024-09-11T22:07:24.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.133+0000 7eff74e01640 1 --2- 172.21.15.1:0/1345573723 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7eff600a2cf0 0x7eff600a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:24.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.134+0000 7eff74e01640 1 --2- 172.21.15.1:0/1345573723 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7eff600a2cf0 0x7eff600a51b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7eff64006010 tx=0x7eff64031510 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:07:24.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.134+0000 7eff7943e740 1 Processor -- start 2024-09-11T22:07:24.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.134+0000 7eff7943e740 1 -- start start 2024-09-11T22:07:24.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.134+0000 7eff7943e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2a505eb40 0x55d2a505d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:24.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.135+0000 7eff7943e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d2a505d5d0 0x55d2a505da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:24.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.135+0000 7eff7943e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d2a5068370 0x55d2a506a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:24.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.135+0000 7eff7943e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55d2a4fc32f0 con 0x55d2a5068370 2024-09-11T22:07:24.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.135+0000 7eff7943e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55d2a4f6acf0 con 0x55d2a505d5d0 2024-09-11T22:07:24.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.135+0000 7eff7943e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d2a4fbf2b0 con 0x55d2a505eb40 2024-09-11T22:07:24.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.135+0000 7eff75602640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d2a5068370 0x55d2a506a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:24.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.135+0000 7eff74e01640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2a505eb40 0x55d2a505d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:24.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.135+0000 7eff75602640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d2a5068370 0x55d2a506a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42046/0 (socket says 172.21.15.1:42046) 2024-09-11T22:07:24.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.135+0000 7eff75602640 1 -- 172.21.15.1:0/264605679 learned_addr learned my addr 172.21.15.1:0/264605679 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:07:24.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.135+0000 7eff6ffff640 1 --2- 172.21.15.1:0/264605679 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d2a505d5d0 0x55d2a505da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:24.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.135+0000 7eff74e01640 1 -- 172.21.15.1:0/264605679 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d2a505d5d0 msgr2=0x55d2a505da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:07:24.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.135+0000 7eff74e01640 1 --2- 172.21.15.1:0/264605679 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d2a505d5d0 0x55d2a505da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:24.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.135+0000 7eff74e01640 1 -- 172.21.15.1:0/264605679 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d2a5068370 msgr2=0x55d2a506a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:07:24.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.135+0000 7eff74e01640 1 --2- 172.21.15.1:0/264605679 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d2a5068370 0x55d2a506a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:24.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.135+0000 7eff74e01640 1 -- 172.21.15.1:0/264605679 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2a4f4f3a0 con 0x55d2a505eb40 2024-09-11T22:07:24.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.135+0000 7eff75602640 1 --2- 172.21.15.1:0/264605679 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d2a5068370 0x55d2a506a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:07:24.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.135+0000 7eff74e01640 1 --2- 172.21.15.1:0/264605679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2a505eb40 0x55d2a505d090 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7eff6402a940 tx=0x7eff6403d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b8fee029e68c0fb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:07:24.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.136+0000 7eff5affd640 1 -- 172.21.15.1:0/264605679 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff64035040 con 0x55d2a505eb40 2024-09-11T22:07:24.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.136+0000 7eff5affd640 1 -- 172.21.15.1:0/264605679 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff6404e070 con 0x55d2a505eb40 2024-09-11T22:07:24.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.136+0000 7eff5affd640 1 -- 172.21.15.1:0/264605679 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff640035f0 con 0x55d2a505eb40 2024-09-11T22:07:24.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.136+0000 7eff7943e740 1 -- 172.21.15.1:0/264605679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2a505eb40 msgr2=0x55d2a505d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:24.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.136+0000 7eff7943e740 1 --2- 172.21.15.1:0/264605679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2a505eb40 0x55d2a505d090 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7eff6402a940 tx=0x7eff6403d000 comp rx=0 tx=0).stop 2024-09-11T22:07:24.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.136+0000 7eff7943e740 1 -- 172.21.15.1:0/264605679 shutdown_connections 2024-09-11T22:07:24.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.136+0000 7eff7943e740 1 --2- 172.21.15.1:0/264605679 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d2a5068370 0x55d2a506a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:24.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.136+0000 7eff7943e740 1 --2- 172.21.15.1:0/264605679 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d2a505d5d0 0x55d2a505da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:24.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.136+0000 7eff7943e740 1 --2- 172.21.15.1:0/264605679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2a505eb40 0x55d2a505d090 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:24.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.136+0000 7eff7943e740 1 -- 172.21.15.1:0/264605679 >> 172.21.15.1:0/264605679 conn(0x55d2a5026310 msgr2=0x55d2a5059c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:07:24.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.136+0000 7eff7943e740 1 -- 172.21.15.1:0/264605679 shutdown_connections 2024-09-11T22:07:24.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.137+0000 7eff7943e740 1 -- 172.21.15.1:0/264605679 wait complete. 2024-09-11T22:07:24.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.137+0000 7eff7943e740 1 -- start start 2024-09-11T22:07:24.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.137+0000 7eff7943e740 1 Processor -- start 2024-09-11T22:07:24.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.137+0000 7eff7943e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2a505d5d0 0x55d2a5067cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:24.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.137+0000 7eff7943e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d2a505eb40 0x55d2a5064fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:24.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.137+0000 7eff7943e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d2a5068370 0x55d2a5065520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:24.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.137+0000 7eff7943e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55d2a506b010 con 0x55d2a5068370 2024-09-11T22:07:24.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.137+0000 7eff75602640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d2a5068370 0x55d2a5065520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:24.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.137+0000 7eff75602640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d2a5068370 0x55d2a5065520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42056/0 (socket says 172.21.15.1:42056) 2024-09-11T22:07:24.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.137+0000 7eff75602640 1 -- 172.21.15.1:0/1157207570 learned_addr learned my addr 172.21.15.1:0/1157207570 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:07:24.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.137+0000 7eff7943e740 1 -- 172.21.15.1:0/1157207570 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x55d2a4f7e060 con 0x55d2a505eb40 2024-09-11T22:07:24.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.137+0000 7eff7943e740 1 -- 172.21.15.1:0/1157207570 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x55d2a506b190 con 0x55d2a505d5d0 2024-09-11T22:07:24.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.138+0000 7eff74e01640 1 --2- 172.21.15.1:0/1157207570 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2a505d5d0 0x55d2a5067cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:24.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.138+0000 7eff7943e740 10 client.0 _ll_register_callbacks cb 0x55d2a4cd18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:07:24.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.138+0000 7eff6ffff640 1 --2- 172.21.15.1:0/1157207570 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d2a505eb40 0x55d2a5064fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:24.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.138+0000 7eff75602640 1 -- 172.21.15.1:0/1157207570 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2a505d5d0 msgr2=0x55d2a5067cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:07:24.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.138+0000 7eff75602640 1 --2- 172.21.15.1:0/1157207570 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2a505d5d0 0x55d2a5067cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:24.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.138+0000 7eff75602640 1 -- 172.21.15.1:0/1157207570 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d2a505eb40 msgr2=0x55d2a5064fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:07:24.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.138+0000 7eff75602640 1 --2- 172.21.15.1:0/1157207570 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d2a505eb40 0x55d2a5064fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:24.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.138+0000 7eff75602640 1 -- 172.21.15.1:0/1157207570 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2a4d343c0 con 0x55d2a5068370 2024-09-11T22:07:24.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.138+0000 7eff6ffff640 1 --2- 172.21.15.1:0/1157207570 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d2a505eb40 0x55d2a5064fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:07:24.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.138+0000 7eff74e01640 1 --2- 172.21.15.1:0/1157207570 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2a505d5d0 0x55d2a5067cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:07:24.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.138+0000 7eff75602640 1 --2- 172.21.15.1:0/1157207570 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d2a5068370 0x55d2a5065520 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7eff680139d0 tx=0x7eff68013e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd2d4282d821121b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:07:24.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.138+0000 7eff4ffff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:07:24.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.138+0000 7eff4ffff640 1 -- 172.21.15.1:0/1157207570 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff68003970 con 0x55d2a5068370 2024-09-11T22:07:24.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.138+0000 7eff4ffff640 1 -- 172.21.15.1:0/1157207570 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7eff68003b10 con 0x55d2a5068370 2024-09-11T22:07:24.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.138+0000 7eff7943e740 1 -- 172.21.15.1:0/1157207570 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff6404e070 con 0x55d2a5068370 2024-09-11T22:07:24.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.138+0000 7eff4ffff640 1 -- 172.21.15.1:0/1157207570 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7eff68003d10 con 0x55d2a5068370 2024-09-11T22:07:24.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.138+0000 7eff7943e740 10 client.17859 fetch_fsmap learned FSMap version 48 2024-09-11T22:07:24.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.138+0000 7eff7943e740 1 -- 172.21.15.1:0/1157207570 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55d2a5076a80 con 0x55d2a5068370 2024-09-11T22:07:24.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.139+0000 7eff4ffff640 1 -- 172.21.15.1:0/1157207570 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff6404e070 con 0x55d2a5068370 2024-09-11T22:07:24.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.139+0000 7eff4ffff640 1 -- 172.21.15.1:0/1157207570 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 48) ==== 8430+0+0 (secure 0 0 0) 0x7eff68022030 con 0x55d2a5068370 2024-09-11T22:07:24.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.139+0000 7eff7943e740 10 client.17859 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T22:07:24.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.139+0000 7eff7943e740 10 client.17859 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:07:24.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.139+0000 7eff7943e740 20 client.17859 populate_metadata read hostname 'smithi001' 2024-09-11T22:07:24.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.139+0000 7eff7943e740 1 --2- 172.21.15.1:0/1157207570 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55d2a5077f90 0x55d2a507a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:24.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.139+0000 7eff7943e740 4 client.17859 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:07:24.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.139+0000 7eff7943e740 1 -- 172.21.15.1:0/1157207570 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2a4fd0860 con 0x55d2a5077f90 2024-09-11T22:07:24.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.139+0000 7eff74e01640 1 --2- 172.21.15.1:0/1157207570 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55d2a5077f90 0x55d2a507a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:24.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.140+0000 7eff74e01640 1 --2- 172.21.15.1:0/1157207570 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55d2a5077f90 0x55d2a507a3c0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec97f47da60ca999 server_cookie=4b3a73d1282ac12d in_seq=0 out_seq=0 2024-09-11T22:07:24.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.140+0000 7eff4ffff640 10 client.17859 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:07:24.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.140+0000 7eff4ffff640 1 -- 172.21.15.1:0/1157207570 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d2a4fd0860 con 0x55d2a5077f90 2024-09-11T22:07:24.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.140+0000 7eff4ffff640 10 client.17859 handle_command_reply: tid=0 2024-09-11T22:07:24.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.140+0000 7eff7943e740 1 client.17859 shutdown 2024-09-11T22:07:24.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.141+0000 7eff7943e740 1 -- 172.21.15.1:0/1157207570 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d2a5068370 msgr2=0x55d2a5065520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:24.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.141+0000 7eff7943e740 1 --2- 172.21.15.1:0/1157207570 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d2a5068370 0x55d2a5065520 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7eff680139d0 tx=0x7eff68013e90 comp rx=0 tx=0).stop 2024-09-11T22:07:24.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.141+0000 7eff7943e740 1 -- 172.21.15.1:0/1157207570 shutdown_connections 2024-09-11T22:07:24.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.141+0000 7eff7943e740 1 --2- 172.21.15.1:0/1157207570 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55d2a5077f90 0x55d2a507a3c0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:24.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.141+0000 7eff7943e740 1 --2- 172.21.15.1:0/1157207570 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d2a5068370 0x55d2a5065520 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:24.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.141+0000 7eff7943e740 1 --2- 172.21.15.1:0/1157207570 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d2a505eb40 0x55d2a5064fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:24.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.141+0000 7eff4ffff640 0 client.17859 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:07:24.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.141+0000 7eff7943e740 1 --2- 172.21.15.1:0/1157207570 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2a505d5d0 0x55d2a5067cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:24.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.141+0000 7eff7943e740 1 -- 172.21.15.1:0/1157207570 >> 172.21.15.1:0/1157207570 conn(0x55d2a5026310 msgr2=0x55d2a505f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:07:24.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.141+0000 7eff7943e740 1 -- 172.21.15.1:0/1157207570 shutdown_connections 2024-09-11T22:07:24.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.141+0000 7eff7943e740 1 -- 172.21.15.1:0/1157207570 wait complete. 2024-09-11T22:07:24.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.141+0000 7eff7943e740 20 client.17859 trim_cache size 0 max 16384 2024-09-11T22:07:24.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.155+0000 7eff7943e740 1 Processor -- start 2024-09-11T22:07:24.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.155+0000 7eff7943e740 1 -- start start 2024-09-11T22:07:24.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.155+0000 7eff7943e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d2a505d5d0 0x55d2a50742b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:24.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.155+0000 7eff7943e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d2a505eb40 0x55d2a5063870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:24.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.155+0000 7eff7943e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2a5068370 0x55d2a5063db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:24.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.155+0000 7eff7943e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7eff701afc40 con 0x55d2a505d5d0 2024-09-11T22:07:24.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.155+0000 7eff7943e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7eff701b19f0 con 0x55d2a505eb40 2024-09-11T22:07:24.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.155+0000 7eff7943e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7eff701b03d0 con 0x55d2a5068370 2024-09-11T22:07:24.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.155+0000 7eff74e01640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d2a505d5d0 0x55d2a50742b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:24.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.155+0000 7eff74e01640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d2a505d5d0 0x55d2a50742b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42068/0 (socket says 172.21.15.1:42068) 2024-09-11T22:07:24.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.155+0000 7eff74e01640 1 -- 172.21.15.1:0/4291434381 learned_addr learned my addr 172.21.15.1:0/4291434381 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:07:24.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.155+0000 7eff6ffff640 1 --2- 172.21.15.1:0/4291434381 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d2a505eb40 0x55d2a5063870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:24.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.155+0000 7eff74e01640 1 -- 172.21.15.1:0/4291434381 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2a5068370 msgr2=0x55d2a5063db0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T22:07:24.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.155+0000 7eff74e01640 1 --2- 172.21.15.1:0/4291434381 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2a5068370 0x55d2a5063db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:24.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.155+0000 7eff74e01640 1 -- 172.21.15.1:0/4291434381 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d2a505eb40 msgr2=0x55d2a5063870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:07:24.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.155+0000 7eff74e01640 1 --2- 172.21.15.1:0/4291434381 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d2a505eb40 0x55d2a5063870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:24.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.155+0000 7eff74e01640 1 -- 172.21.15.1:0/4291434381 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2a505a960 con 0x55d2a505d5d0 2024-09-11T22:07:24.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.155+0000 7eff6ffff640 1 --2- 172.21.15.1:0/4291434381 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d2a505eb40 0x55d2a5063870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:07:24.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.156+0000 7eff74e01640 1 --2- 172.21.15.1:0/4291434381 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d2a505d5d0 0x55d2a50742b0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7eff64033000 tx=0x7eff64032cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfce7d531bd4c693 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:07:24.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.156+0000 7eff5a7fc640 1 -- 172.21.15.1:0/4291434381 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff64040050 con 0x55d2a505d5d0 2024-09-11T22:07:24.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.156+0000 7eff5a7fc640 1 -- 172.21.15.1:0/4291434381 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff6404e040 con 0x55d2a505d5d0 2024-09-11T22:07:24.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.156+0000 7eff5a7fc640 1 -- 172.21.15.1:0/4291434381 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff64035040 con 0x55d2a505d5d0 2024-09-11T22:07:24.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.156+0000 7eff7943e740 1 -- 172.21.15.1:0/4291434381 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d2a505d5d0 msgr2=0x55d2a50742b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:24.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.156+0000 7eff7943e740 1 --2- 172.21.15.1:0/4291434381 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d2a505d5d0 0x55d2a50742b0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7eff64033000 tx=0x7eff64032cb0 comp rx=0 tx=0).stop 2024-09-11T22:07:24.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.157+0000 7eff7943e740 1 -- 172.21.15.1:0/4291434381 shutdown_connections 2024-09-11T22:07:24.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.157+0000 7eff7943e740 1 --2- 172.21.15.1:0/4291434381 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2a5068370 0x55d2a5063db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:24.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.157+0000 7eff7943e740 1 --2- 172.21.15.1:0/4291434381 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d2a505eb40 0x55d2a5063870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:24.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.157+0000 7eff7943e740 1 --2- 172.21.15.1:0/4291434381 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d2a505d5d0 0x55d2a50742b0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:24.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.157+0000 7eff7943e740 1 -- 172.21.15.1:0/4291434381 >> 172.21.15.1:0/4291434381 conn(0x55d2a5026310 msgr2=0x55d2a5064f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:07:24.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.157+0000 7eff7943e740 1 -- 172.21.15.1:0/4291434381 shutdown_connections 2024-09-11T22:07:24.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.157+0000 7eff7943e740 1 -- 172.21.15.1:0/4291434381 wait complete. 2024-09-11T22:07:24.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.157+0000 7eff7943e740 1 -- start start 2024-09-11T22:07:24.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.157+0000 7eff7943e740 1 Processor -- start 2024-09-11T22:07:24.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.157+0000 7eff7943e740 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d2a505d5d0 0x55d2a5070830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:24.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.157+0000 7eff7943e740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2a505eb40 0x55d2a506bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:24.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.157+0000 7eff7943e740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d2a5068370 0x55d2a506c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:24.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.157+0000 7eff7943e740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7eff701b12f0 con 0x55d2a5068370 2024-09-11T22:07:24.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.157+0000 7eff7943e740 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7eff701b0b60 con 0x55d2a505d5d0 2024-09-11T22:07:24.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.157+0000 7eff7943e740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7eff701adc20 con 0x55d2a505eb40 2024-09-11T22:07:24.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.158+0000 7eff7943e740 10 client.0 _ll_register_callbacks cb 0x7eff68009bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:07:24.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.158+0000 7eff75602640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d2a5068370 0x55d2a506c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:24.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.158+0000 7eff75602640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d2a5068370 0x55d2a506c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42082/0 (socket says 172.21.15.1:42082) 2024-09-11T22:07:24.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.158+0000 7eff75602640 1 -- 172.21.15.1:0/3962345121 learned_addr learned my addr 172.21.15.1:0/3962345121 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:07:24.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.158+0000 7eff74e01640 1 --2- 172.21.15.1:0/3962345121 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d2a505d5d0 0x55d2a5070830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:24.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.158+0000 7eff75602640 1 -- 172.21.15.1:0/3962345121 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2a505eb40 msgr2=0x55d2a506bb60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T22:07:24.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.158+0000 7eff75602640 1 --2- 172.21.15.1:0/3962345121 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2a505eb40 0x55d2a506bb60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:24.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.158+0000 7eff75602640 1 -- 172.21.15.1:0/3962345121 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d2a505d5d0 msgr2=0x55d2a5070830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:07:24.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.158+0000 7eff75602640 1 --2- 172.21.15.1:0/3962345121 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d2a505d5d0 0x55d2a5070830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:24.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.158+0000 7eff75602640 1 -- 172.21.15.1:0/3962345121 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2a5060120 con 0x55d2a5068370 2024-09-11T22:07:24.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.158+0000 7eff74e01640 1 --2- 172.21.15.1:0/3962345121 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d2a505d5d0 0x55d2a5070830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:07:24.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.158+0000 7eff75602640 1 --2- 172.21.15.1:0/3962345121 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d2a5068370 0x55d2a506c0a0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7eff68005cc0 tx=0x7eff680118a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bf59acae33656aa server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:07:24.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.158+0000 7eff4ffff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-09-11T22:07:24.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.158+0000 7eff4ffff640 1 -- 172.21.15.1:0/3962345121 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff6801c640 con 0x55d2a5068370 2024-09-11T22:07:24.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.158+0000 7eff4ffff640 1 -- 172.21.15.1:0/3962345121 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7eff680050c0 con 0x55d2a5068370 2024-09-11T22:07:24.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.158+0000 7eff7943e740 1 -- 172.21.15.1:0/3962345121 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff6404e040 con 0x55d2a5068370 2024-09-11T22:07:24.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.159+0000 7eff4ffff640 1 -- 172.21.15.1:0/3962345121 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7eff68011e70 con 0x55d2a5068370 2024-09-11T22:07:24.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.159+0000 7eff7943e740 10 client.17865 fetch_fsmap learned FSMap version 48 2024-09-11T22:07:24.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.159+0000 7eff7943e740 1 -- 172.21.15.1:0/3962345121 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55d2a50eaee0 con 0x55d2a5068370 2024-09-11T22:07:24.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.159+0000 7eff4ffff640 1 -- 172.21.15.1:0/3962345121 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff6404e040 con 0x55d2a5068370 2024-09-11T22:07:24.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.159+0000 7eff4ffff640 1 -- 172.21.15.1:0/3962345121 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 48) ==== 8430+0+0 (secure 0 0 0) 0x7eff6801a070 con 0x55d2a5068370 2024-09-11T22:07:24.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.159+0000 7eff7943e740 10 client.17865 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T22:07:24.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.159+0000 7eff7943e740 10 client.17865 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:07:24.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.159+0000 7eff7943e740 20 client.17865 populate_metadata read hostname 'smithi001' 2024-09-11T22:07:24.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.159+0000 7eff7943e740 1 --2- 172.21.15.1:0/3962345121 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55d2a50eb5f0 0x55d2a50eda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:24.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.159+0000 7eff7943e740 4 client.17865 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:07:24.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.159+0000 7eff7943e740 1 -- 172.21.15.1:0/3962345121 --> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d2a4fcad70 con 0x55d2a50eb5f0 2024-09-11T22:07:24.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.159+0000 7eff74e01640 1 --2- 172.21.15.1:0/3962345121 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55d2a50eb5f0 0x55d2a50eda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:24.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.160+0000 7eff4ffff640 10 client.17865 ms_handle_connect on v2:172.21.15.53:6838/1378584447 2024-09-11T22:07:24.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.160+0000 7eff74e01640 1 --2- 172.21.15.1:0/3962345121 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55d2a50eb5f0 0x55d2a50eda30 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccb9a361ac47812b server_cookie=af0abbdc173ea255 in_seq=0 out_seq=0 2024-09-11T22:07:24.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.160+0000 7eff4ffff640 1 -- 172.21.15.1:0/3962345121 <== mds.0 v2:172.21.15.53:6838/1378584447 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d2a4fcad70 con 0x55d2a50eb5f0 2024-09-11T22:07:24.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.160+0000 7eff4ffff640 10 client.17865 handle_command_reply: tid=0 2024-09-11T22:07:24.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.160+0000 7eff7943e740 1 client.17865 shutdown 2024-09-11T22:07:24.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.160+0000 7eff7943e740 1 -- 172.21.15.1:0/3962345121 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d2a5068370 msgr2=0x55d2a506c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:24.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.160+0000 7eff7943e740 1 --2- 172.21.15.1:0/3962345121 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d2a5068370 0x55d2a506c0a0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7eff68005cc0 tx=0x7eff680118a0 comp rx=0 tx=0).stop 2024-09-11T22:07:24.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.160+0000 7eff7943e740 1 -- 172.21.15.1:0/3962345121 shutdown_connections 2024-09-11T22:07:24.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.160+0000 7eff7943e740 1 --2- 172.21.15.1:0/3962345121 >> [v2:172.21.15.53:6838/1378584447,v1:172.21.15.53:6839/1378584447] conn(0x55d2a50eb5f0 0x55d2a50eda30 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:24.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.160+0000 7eff7943e740 1 --2- 172.21.15.1:0/3962345121 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d2a5068370 0x55d2a506c0a0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:24.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.160+0000 7eff7943e740 1 --2- 172.21.15.1:0/3962345121 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d2a505eb40 0x55d2a506bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:24.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.160+0000 7eff7943e740 1 --2- 172.21.15.1:0/3962345121 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d2a505d5d0 0x55d2a5070830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:24.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.160+0000 7eff7943e740 1 -- 172.21.15.1:0/3962345121 >> 172.21.15.1:0/3962345121 conn(0x55d2a5026310 msgr2=0x55d2a5064b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:07:24.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.160+0000 7eff4ffff640 0 client.17865 ms_handle_reset on v2:172.21.15.53:6838/1378584447 2024-09-11T22:07:24.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.160+0000 7eff7943e740 1 -- 172.21.15.1:0/3962345121 shutdown_connections 2024-09-11T22:07:24.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.160+0000 7eff7943e740 1 -- 172.21.15.1:0/3962345121 wait complete. 2024-09-11T22:07:24.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.160+0000 7eff7943e740 20 client.17865 trim_cache size 0 max 16384 2024-09-11T22:07:24.183 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-09-11T22:07:24.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.161+0000 7eff77065640 1 -- 172.21.15.1:0/1345573723 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7eff600a2cf0 msgr2=0x7eff600a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:24.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.161+0000 7eff77065640 1 --2- 172.21.15.1:0/1345573723 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7eff600a2cf0 0x7eff600a51b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7eff64006010 tx=0x7eff64031510 comp rx=0 tx=0).stop 2024-09-11T22:07:24.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.161+0000 7eff77065640 1 -- 172.21.15.1:0/1345573723 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff70103740 msgr2=0x7eff70100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:24.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.161+0000 7eff77065640 1 --2- 172.21.15.1:0/1345573723 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff70103740 0x7eff70100f30 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7eff5c00e9c0 tx=0x7eff5c00ee80 comp rx=0 tx=0).stop 2024-09-11T22:07:24.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.161+0000 7eff77065640 1 -- 172.21.15.1:0/1345573723 shutdown_connections 2024-09-11T22:07:24.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.161+0000 7eff77065640 1 --2- 172.21.15.1:0/1345573723 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7eff600a2cf0 0x7eff600a51b0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:24.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.161+0000 7eff77065640 1 --2- 172.21.15.1:0/1345573723 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff70108290 0x7eff70101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:24.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.161+0000 7eff77065640 1 --2- 172.21.15.1:0/1345573723 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff70103740 0x7eff70100f30 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:24.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.161+0000 7eff77065640 1 --2- 172.21.15.1:0/1345573723 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7eff70102d90 0x7eff701009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:24.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.161+0000 7eff77065640 1 -- 172.21.15.1:0/1345573723 >> 172.21.15.1:0/1345573723 conn(0x7eff700fc7d0 msgr2=0x7eff70106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:07:24.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.161+0000 7eff77065640 1 -- 172.21.15.1:0/1345573723 shutdown_connections 2024-09-11T22:07:24.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.161+0000 7eff77065640 1 -- 172.21.15.1:0/1345573723 wait complete. 2024-09-11T22:07:24.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.318+0000 7fcf16d31640 1 -- 172.21.15.1:0/2939180009 --> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fcf10062630 con 0x7fcf000a2cf0 2024-09-11T22:07:24.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.325+0000 7fcf0dffb640 1 -- 172.21.15.1:0/2939180009 <== mgr.15204 v2:172.21.15.1:6800/3543150772 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fcf10062630 con 0x7fcf000a2cf0 2024-09-11T22:07:24.327 INFO:teuthology.orchestra.run.smithi001.stdout:/ 1m 6m3h 2024-09-11T22:07:24.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.327+0000 7fcf16d31640 1 -- 172.21.15.1:0/2939180009 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fcf000a2cf0 msgr2=0x7fcf000a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:24.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.327+0000 7fcf16d31640 1 --2- 172.21.15.1:0/2939180009 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fcf000a2cf0 0x7fcf000a51b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fcf04009bd0 tx=0x7fcf04005f90 comp rx=0 tx=0).stop 2024-09-11T22:07:24.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.327+0000 7fcf16d31640 1 -- 172.21.15.1:0/2939180009 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcf1010c0f0 msgr2=0x7fcf1019cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:24.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.327+0000 7fcf16d31640 1 --2- 172.21.15.1:0/2939180009 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcf1010c0f0 0x7fcf1019cd20 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fcf0800eaa0 tx=0x7fcf0800ef60 comp rx=0 tx=0).stop 2024-09-11T22:07:24.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.327+0000 7fcf16d31640 1 -- 172.21.15.1:0/2939180009 shutdown_connections 2024-09-11T22:07:24.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.327+0000 7fcf16d31640 1 --2- 172.21.15.1:0/2939180009 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fcf000a2cf0 0x7fcf000a51b0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:24.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.327+0000 7fcf16d31640 1 --2- 172.21.15.1:0/2939180009 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcf1010c0f0 0x7fcf1019cd20 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:24.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.327+0000 7fcf16d31640 1 --2- 172.21.15.1:0/2939180009 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcf101035c0 0x7fcf101989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:24.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.327+0000 7fcf16d31640 1 --2- 172.21.15.1:0/2939180009 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf10102ca0 0x7fcf10198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:24.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.327+0000 7fcf16d31640 1 -- 172.21.15.1:0/2939180009 >> 172.21.15.1:0/2939180009 conn(0x7fcf100fc6b0 msgr2=0x7fcf100fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:07:24.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.327+0000 7fcf16d31640 1 -- 172.21.15.1:0/2939180009 shutdown_connections 2024-09-11T22:07:24.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:24.327+0000 7fcf16d31640 1 -- 172.21.15.1:0/2939180009 wait complete. 2024-09-11T22:07:24.340 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-09-11T22:07:24.354 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-09-11T22:07:24.935 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-09-11T22:07:24.941 INFO:tasks.fwd_scrub:done joining 2024-09-11T22:07:24.941 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-11T22:07:24.952 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T22:07:25.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.057+0000 7fb62dcd2640 1 -- 172.21.15.1:0/3631850710 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb6281069e0 msgr2=0x7fb628106dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:25.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.057+0000 7fb62dcd2640 1 --2- 172.21.15.1:0/3631850710 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb6281069e0 0x7fb628106dc0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fb618009930 tx=0x7fb61802a420 comp rx=0 tx=0).stop 2024-09-11T22:07:25.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.057+0000 7fb62dcd2640 1 -- 172.21.15.1:0/3631850710 shutdown_connections 2024-09-11T22:07:25.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.057+0000 7fb62dcd2640 1 --2- 172.21.15.1:0/3631850710 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb628101370 0x7fb62810d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:25.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.057+0000 7fb62dcd2640 1 --2- 172.21.15.1:0/3631850710 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6281009d0 0x7fb628100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:25.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.057+0000 7fb62dcd2640 1 --2- 172.21.15.1:0/3631850710 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb6281069e0 0x7fb628106dc0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:25.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.057+0000 7fb62dcd2640 1 -- 172.21.15.1:0/3631850710 >> 172.21.15.1:0/3631850710 conn(0x7fb6280fc6b0 msgr2=0x7fb6280fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:07:25.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.057+0000 7fb62dcd2640 1 -- 172.21.15.1:0/3631850710 shutdown_connections 2024-09-11T22:07:25.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.057+0000 7fb62dcd2640 1 -- 172.21.15.1:0/3631850710 wait complete. 2024-09-11T22:07:25.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.058+0000 7fb62dcd2640 1 Processor -- start 2024-09-11T22:07:25.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.058+0000 7fb62dcd2640 1 -- start start 2024-09-11T22:07:25.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.058+0000 7fb62dcd2640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6281009d0 0x7fb628197390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:25.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.058+0000 7fb62dcd2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb628101370 0x7fb6281978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:25.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.058+0000 7fb62dcd2640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6281069e0 0x7fb62819bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:25.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.058+0000 7fb62dcd2640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb628110e20 con 0x7fb628101370 2024-09-11T22:07:25.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.058+0000 7fb62dcd2640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fb628110ca0 con 0x7fb6281009d0 2024-09-11T22:07:25.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.058+0000 7fb62dcd2640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fb628110fa0 con 0x7fb6281069e0 2024-09-11T22:07:25.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.059+0000 7fb626ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb628101370 0x7fb6281978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:25.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.059+0000 7fb626ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb628101370 0x7fb6281978d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42092/0 (socket says 172.21.15.1:42092) 2024-09-11T22:07:25.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.059+0000 7fb626ffd640 1 -- 172.21.15.1:0/1298310486 learned_addr learned my addr 172.21.15.1:0/1298310486 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:07:25.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.059+0000 7fb627fff640 1 --2- 172.21.15.1:0/1298310486 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6281069e0 0x7fb62819bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:25.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.059+0000 7fb6277fe640 1 --2- 172.21.15.1:0/1298310486 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6281009d0 0x7fb628197390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:25.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.059+0000 7fb626ffd640 1 -- 172.21.15.1:0/1298310486 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6281069e0 msgr2=0x7fb62819bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:25.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.059+0000 7fb626ffd640 1 --2- 172.21.15.1:0/1298310486 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6281069e0 0x7fb62819bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:25.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.059+0000 7fb626ffd640 1 -- 172.21.15.1:0/1298310486 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6281009d0 msgr2=0x7fb628197390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:25.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.059+0000 7fb626ffd640 1 --2- 172.21.15.1:0/1298310486 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6281009d0 0x7fb628197390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:25.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.059+0000 7fb626ffd640 1 -- 172.21.15.1:0/1298310486 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb62819c380 con 0x7fb628101370 2024-09-11T22:07:25.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.059+0000 7fb6277fe640 1 --2- 172.21.15.1:0/1298310486 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6281009d0 0x7fb628197390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:07:25.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.059+0000 7fb627fff640 1 --2- 172.21.15.1:0/1298310486 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6281069e0 0x7fb62819bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:07:25.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.059+0000 7fb626ffd640 1 --2- 172.21.15.1:0/1298310486 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb628101370 0x7fb6281978d0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fb61c00d950 tx=0x7fb61c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:07:25.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.059+0000 7fb624ff9640 1 -- 172.21.15.1:0/1298310486 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb61c017030 con 0x7fb628101370 2024-09-11T22:07:25.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.059+0000 7fb62dcd2640 1 -- 172.21.15.1:0/1298310486 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb62819c670 con 0x7fb628101370 2024-09-11T22:07:25.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.059+0000 7fb624ff9640 1 -- 172.21.15.1:0/1298310486 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb61c004020 con 0x7fb628101370 2024-09-11T22:07:25.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.059+0000 7fb624ff9640 1 -- 172.21.15.1:0/1298310486 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb61c0047a0 con 0x7fb628101370 2024-09-11T22:07:25.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.059+0000 7fb62dcd2640 1 -- 172.21.15.1:0/1298310486 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6281a3f00 con 0x7fb628101370 2024-09-11T22:07:25.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.060+0000 7fb62dcd2640 1 -- 172.21.15.1:0/1298310486 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb628100e30 con 0x7fb628101370 2024-09-11T22:07:25.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.061+0000 7fb624ff9640 1 -- 172.21.15.1:0/1298310486 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7fb61c0041d0 con 0x7fb628101370 2024-09-11T22:07:25.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.062+0000 7fb624ff9640 1 --2- 172.21.15.1:0/1298310486 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fb6100a2d00 0x7fb6100a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:25.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.062+0000 7fb624ff9640 1 -- 172.21.15.1:0/1298310486 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7fb61c0cefe0 con 0x7fb628101370 2024-09-11T22:07:25.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.062+0000 7fb6277fe640 1 --2- 172.21.15.1:0/1298310486 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fb6100a2d00 0x7fb6100a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:25.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.063+0000 7fb6277fe640 1 --2- 172.21.15.1:0/1298310486 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fb6100a2d00 0x7fb6100a51c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb618000c00 tx=0x7fb618032c70 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:07:25.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.066+0000 7fb624ff9640 1 -- 172.21.15.1:0/1298310486 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb61c09bb60 con 0x7fb628101370 2024-09-11T22:07:25.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:25 smithi001 ceph-mon[30935]: pgmap v316: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:07:25.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:25 smithi001 ceph-mon[30935]: from='client.17841 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:07:25.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.244+0000 7fb62dcd2640 1 -- 172.21.15.1:0/1298310486 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb6281089b0 con 0x7fb628101370 2024-09-11T22:07:25.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.245+0000 7fb624ff9640 1 -- 172.21.15.1:0/1298310486 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+23213 (secure 0 0 0) 0x7fb61c09b460 con 0x7fb628101370 2024-09-11T22:07:25.248 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T22:07:25.248 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":95,"fsid":"3ffca83c-7087-11ef-bcea-c7b262605968","created":"2024-09-11T21:47:28.868707+0000","modified":"2024-09-11T22:05:45.511990+0000","last_up_change":"2024-09-11T21:54:29.319441+0000","last_in_change":"2024-09-11T21:54:05.532281+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T21:50:41.026490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T21:55:16.961999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T21:55:19.066228+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":9,"snap_epoch":90,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"19e56504-18f1-4234-b2d6-f193ff47d8eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":4012142023},{"type":"v1","addr":"172.21.15.1:6803","nonce":4012142023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":4012142023},{"type":"v1","addr":"172.21.15.1:6805","nonce":4012142023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":4012142023},{"type":"v1","addr":"172.21.15.1:6809","nonce":4012142023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":4012142023},{"type":"v1","addr":"172.21.15.1:6807","nonce":4012142023}]},"public_addr":"172.21.15.1:6803/4012142023","cluster_addr":"172.21.15.1:6805/4012142023","heartbeat_back_addr":"172.21.15.1:6809/4012142023","heartbeat_front_addr":"172.21.15.1:6807/4012142023","state":["exists","up"]},{"osd":1,"uuid":"996c9f0c-c5a4-43b8-b8e0-d712f27c5dfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3039391924},{"type":"v1","addr":"172.21.15.53:6801","nonce":3039391924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3039391924},{"type":"v1","addr":"172.21.15.53:6803","nonce":3039391924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3039391924},{"type":"v1","addr":"172.21.15.53:6807","nonce":3039391924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3039391924},{"type":"v1","addr":"172.21.15.53:6805","nonce":3039391924}]},"public_addr":"172.21.15.53:6801/3039391924","cluster_addr":"172.21.15.53:6803/3039391924","heartbeat_back_addr":"172.21.15.53:6807/3039391924","heartbeat_front_addr":"172.21.15.53:6805/3039391924","state":["exists","up"]},{"osd":2,"uuid":"578d4260-41e1-4645-b9b1-1c5445712c13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1707914885},{"type":"v1","addr":"172.21.15.161:6801","nonce":1707914885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1707914885},{"type":"v1","addr":"172.21.15.161:6803","nonce":1707914885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1707914885},{"type":"v1","addr":"172.21.15.161:6807","nonce":1707914885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1707914885},{"type":"v1","addr":"172.21.15.161:6805","nonce":1707914885}]},"public_addr":"172.21.15.161:6801/1707914885","cluster_addr":"172.21.15.161:6803/1707914885","heartbeat_back_addr":"172.21.15.161:6807/1707914885","heartbeat_front_addr":"172.21.15.161:6805/1707914885","state":["exists","up"]},{"osd":3,"uuid":"73f78766-9662-4aa3-841c-fe2ce654ee9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1541142398},{"type":"v1","addr":"172.21.15.1:6811","nonce":1541142398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1541142398},{"type":"v1","addr":"172.21.15.1:6813","nonce":1541142398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1541142398},{"type":"v1","addr":"172.21.15.1:6817","nonce":1541142398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1541142398},{"type":"v1","addr":"172.21.15.1:6815","nonce":1541142398}]},"public_addr":"172.21.15.1:6811/1541142398","cluster_addr":"172.21.15.1:6813/1541142398","heartbeat_back_addr":"172.21.15.1:6817/1541142398","heartbeat_front_addr":"172.21.15.1:6815/1541142398","state":["exists","up"]},{"osd":4,"uuid":"56c808bb-d4cd-4969-b49e-adef49ecd0f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":3081402681},{"type":"v1","addr":"172.21.15.53:6809","nonce":3081402681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3081402681},{"type":"v1","addr":"172.21.15.53:6811","nonce":3081402681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3081402681},{"type":"v1","addr":"172.21.15.53:6815","nonce":3081402681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3081402681},{"type":"v1","addr":"172.21.15.53:6813","nonce":3081402681}]},"public_addr":"172.21.15.53:6809/3081402681","cluster_addr":"172.21.15.53:6811/3081402681","heartbeat_back_addr":"172.21.15.53:6815/3081402681","heartbeat_front_addr":"172.21.15.53:6813/3081402681","state":["exists","up"]},{"osd":5,"uuid":"7b8266d6-b507-47c9-b4fa-ff16c4a74d35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":1622081327},{"type":"v1","addr":"172.21.15.161:6809","nonce":1622081327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":1622081327},{"type":"v1","addr":"172.21.15.161:6811","nonce":1622081327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":1622081327},{"type":"v1","addr":"172.21.15.161:6815","nonce":1622081327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":1622081327},{"type":"v1","addr":"172.21.15.161:6813","nonce":1622081327}]},"public_addr":"172.21.15.161:6809/1622081327","cluster_addr":"172.21.15.161:6811/1622081327","heartbeat_back_addr":"172.21.15.161:6815/1622081327","heartbeat_front_addr":"172.21.15.161:6813/1622081327","state":["exists","up"]},{"osd":6,"uuid":"3af5925d-a749-4849-a076-80580222cad4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":158434853},{"type":"v1","addr":"172.21.15.1:6819","nonce":158434853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":158434853},{"type":"v1","addr":"172.21.15.1:6821","nonce":158434853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":158434853},{"type":"v1","addr":"172.21.15.1:6825","nonce":158434853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":158434853},{"type":"v1","addr":"172.21.15.1:6823","nonce":158434853}]},"public_addr":"172.21.15.1:6819/158434853","cluster_addr":"172.21.15.1:6821/158434853","heartbeat_back_addr":"172.21.15.1:6825/158434853","heartbeat_front_addr":"172.21.15.1:6823/158434853","state":["exists","up"]},{"osd":7,"uuid":"f8f26c3c-17e9-4e89-8392-289796e0c8a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2132057697},{"type":"v1","addr":"172.21.15.53:6817","nonce":2132057697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2132057697},{"type":"v1","addr":"172.21.15.53:6819","nonce":2132057697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2132057697},{"type":"v1","addr":"172.21.15.53:6823","nonce":2132057697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2132057697},{"type":"v1","addr":"172.21.15.53:6821","nonce":2132057697}]},"public_addr":"172.21.15.53:6817/2132057697","cluster_addr":"172.21.15.53:6819/2132057697","heartbeat_back_addr":"172.21.15.53:6823/2132057697","heartbeat_front_addr":"172.21.15.53:6821/2132057697","state":["exists","up"]},{"osd":8,"uuid":"36ea401b-3c93-4b9d-8880-298d1eb6e30b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1838248669},{"type":"v1","addr":"172.21.15.161:6817","nonce":1838248669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1838248669},{"type":"v1","addr":"172.21.15.161:6819","nonce":1838248669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1838248669},{"type":"v1","addr":"172.21.15.161:6823","nonce":1838248669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1838248669},{"type":"v1","addr":"172.21.15.161:6821","nonce":1838248669}]},"public_addr":"172.21.15.161:6817/1838248669","cluster_addr":"172.21.15.161:6819/1838248669","heartbeat_back_addr":"172.21.15.161:6823/1838248669","heartbeat_front_addr":"172.21.15.161:6821/1838248669","state":["exists","up"]},{"osd":9,"uuid":"4b788d3f-a3bc-4463-bd99-95fdb7a52768","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2806274589},{"type":"v1","addr":"172.21.15.1:6827","nonce":2806274589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2806274589},{"type":"v1","addr":"172.21.15.1:6829","nonce":2806274589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":2806274589},{"type":"v1","addr":"172.21.15.1:6833","nonce":2806274589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2806274589},{"type":"v1","addr":"172.21.15.1:6831","nonce":2806274589}]},"public_addr":"172.21.15.1:6827/2806274589","cluster_addr":"172.21.15.1:6829/2806274589","heartbeat_back_addr":"172.21.15.1:6833/2806274589","heartbeat_front_addr":"172.21.15.1:6831/2806274589","state":["exists","up"]},{"osd":10,"uuid":"70549796-5270-4ae2-ac3f-f61f5de105d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":761923580},{"type":"v1","addr":"172.21.15.53:6825","nonce":761923580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":761923580},{"type":"v1","addr":"172.21.15.53:6827","nonce":761923580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":761923580},{"type":"v1","addr":"172.21.15.53:6831","nonce":761923580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":761923580},{"type":"v1","addr":"172.21.15.53:6829","nonce":761923580}]},"public_addr":"172.21.15.53:6825/761923580","cluster_addr":"172.21.15.53:6827/761923580","heartbeat_back_addr":"172.21.15.53:6831/761923580","heartbeat_front_addr":"172.21.15.53:6829/761923580","state":["exists","up"]},{"osd":11,"uuid":"48f17c4a-370a-493e-956e-bf6ad3dcc594","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1586352190},{"type":"v1","addr":"172.21.15.161:6825","nonce":1586352190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":1586352190},{"type":"v1","addr":"172.21.15.161:6827","nonce":1586352190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":1586352190},{"type":"v1","addr":"172.21.15.161:6831","nonce":1586352190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":1586352190},{"type":"v1","addr":"172.21.15.161:6829","nonce":1586352190}]},"public_addr":"172.21.15.161:6825/1586352190","cluster_addr":"172.21.15.161:6827/1586352190","heartbeat_back_addr":"172.21.15.161:6831/1586352190","heartbeat_front_addr":"172.21.15.161:6829/1586352190","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:49:46.539613+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:50:11.912300+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:50:36.955490+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:51:02.200554+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:51:27.860585+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:51:54.694099+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:52:18.962351+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:52:44.881923+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:53:10.301390+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:53:35.417647+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:54:01.492219+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:54:26.938338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.26","mappings":[{"from":1,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/4286250488":"2024-09-12T21:57:13.664392+0000","172.21.15.1:0/2039944931":"2024-09-12T21:57:13.664392+0000","172.21.15.1:0/3202962495":"2024-09-12T21:48:00.674728+0000","172.21.15.1:6800/2311309446":"2024-09-12T21:48:00.674728+0000","172.21.15.1:6801/3890350886":"2024-09-12T21:57:13.664392+0000","172.21.15.1:6801/2311309446":"2024-09-12T21:48:00.674728+0000","172.21.15.1:0/2599895977":"2024-09-12T21:48:00.674728+0000","172.21.15.1:0/1278877732":"2024-09-12T21:48:24.524913+0000","172.21.15.1:0/276770268":"2024-09-12T21:48:00.674728+0000","172.21.15.1:0/1699984777":"2024-09-12T21:48:24.524913+0000","172.21.15.1:6800/2821761445":"2024-09-12T21:48:24.524913+0000","172.21.15.1:0/2797780151":"2024-09-12T21:57:13.664392+0000","172.21.15.1:6801/2821761445":"2024-09-12T21:48:24.524913+0000","172.21.15.1:0/1749020850":"2024-09-12T21:48:24.524913+0000","172.21.15.1:0/1202232270":"2024-09-12T21:57:13.664392+0000","172.21.15.1:6800/3890350886":"2024-09-12T21:57:13.664392+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T22:07:25.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.247+0000 7fb62dcd2640 1 -- 172.21.15.1:0/1298310486 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fb6100a2d00 msgr2=0x7fb6100a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:25.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.247+0000 7fb62dcd2640 1 --2- 172.21.15.1:0/1298310486 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fb6100a2d00 0x7fb6100a51c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb618000c00 tx=0x7fb618032c70 comp rx=0 tx=0).stop 2024-09-11T22:07:25.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.247+0000 7fb62dcd2640 1 -- 172.21.15.1:0/1298310486 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb628101370 msgr2=0x7fb6281978d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:25.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.247+0000 7fb62dcd2640 1 --2- 172.21.15.1:0/1298310486 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb628101370 0x7fb6281978d0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fb61c00d950 tx=0x7fb61c00de10 comp rx=0 tx=0).stop 2024-09-11T22:07:25.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.247+0000 7fb62dcd2640 1 -- 172.21.15.1:0/1298310486 shutdown_connections 2024-09-11T22:07:25.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.247+0000 7fb62dcd2640 1 --2- 172.21.15.1:0/1298310486 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fb6100a2d00 0x7fb6100a51c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:25.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.247+0000 7fb62dcd2640 1 --2- 172.21.15.1:0/1298310486 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6281069e0 0x7fb62819bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:25.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.247+0000 7fb62dcd2640 1 --2- 172.21.15.1:0/1298310486 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb628101370 0x7fb6281978d0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:25.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.247+0000 7fb62dcd2640 1 --2- 172.21.15.1:0/1298310486 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb6281009d0 0x7fb628197390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:25.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.247+0000 7fb62dcd2640 1 -- 172.21.15.1:0/1298310486 >> 172.21.15.1:0/1298310486 conn(0x7fb6280fc6b0 msgr2=0x7fb6280fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:07:25.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.247+0000 7fb62dcd2640 1 -- 172.21.15.1:0/1298310486 shutdown_connections 2024-09-11T22:07:25.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.247+0000 7fb62dcd2640 1 -- 172.21.15.1:0/1298310486 wait complete. 2024-09-11T22:07:25.261 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T21:50:41.026490+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-11T21:55:16.961999+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-11T21:55:19.066228+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 9, 'snap_epoch': 90, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-11T22:07:25.261 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-11T22:07:25.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.368+0000 7f95c33f8640 1 -- 172.21.15.1:0/3025770055 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f95bc105f50 msgr2=0x7f95bc106400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:25.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.368+0000 7f95c33f8640 1 --2- 172.21.15.1:0/3025770055 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f95bc105f50 0x7f95bc106400 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f95b4009cd0 tx=0x7f95b402a8e0 comp rx=0 tx=0).stop 2024-09-11T22:07:25.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.368+0000 7f95c33f8640 1 -- 172.21.15.1:0/3025770055 shutdown_connections 2024-09-11T22:07:25.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.368+0000 7f95c33f8640 1 --2- 172.21.15.1:0/3025770055 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f95bc105f50 0x7f95bc106400 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:25.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.368+0000 7f95c33f8640 1 --2- 172.21.15.1:0/3025770055 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95bc101380 0x7f95bc105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:25.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.368+0000 7f95c33f8640 1 --2- 172.21.15.1:0/3025770055 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95bc1009d0 0x7f95bc100db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:25.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.368+0000 7f95c33f8640 1 -- 172.21.15.1:0/3025770055 >> 172.21.15.1:0/3025770055 conn(0x7f95bc0fc6a0 msgr2=0x7f95bc0feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:07:25.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.368+0000 7f95c33f8640 1 -- 172.21.15.1:0/3025770055 shutdown_connections 2024-09-11T22:07:25.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.368+0000 7f95c33f8640 1 -- 172.21.15.1:0/3025770055 wait complete. 2024-09-11T22:07:25.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.368+0000 7f95c33f8640 1 Processor -- start 2024-09-11T22:07:25.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.368+0000 7f95c33f8640 1 -- start start 2024-09-11T22:07:25.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.368+0000 7f95c33f8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95bc1009d0 0x7f95bc1a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:25.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.369+0000 7f95c33f8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f95bc101380 0x7f95bc1a4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:25.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.369+0000 7f95c33f8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95bc105f50 0x7f95bc19e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:25.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.369+0000 7f95c33f8640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f95bc118490 con 0x7f95bc101380 2024-09-11T22:07:25.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.369+0000 7f95c33f8640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f95bc118310 con 0x7f95bc1009d0 2024-09-11T22:07:25.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.369+0000 7f95c33f8640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f95bc118610 con 0x7f95bc105f50 2024-09-11T22:07:25.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.369+0000 7f95c0993640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f95bc101380 0x7f95bc1a4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:25.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.369+0000 7f95c0993640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f95bc101380 0x7f95bc1a4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42106/0 (socket says 172.21.15.1:42106) 2024-09-11T22:07:25.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.369+0000 7f95c0993640 1 -- 172.21.15.1:0/4262110506 learned_addr learned my addr 172.21.15.1:0/4262110506 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:07:25.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.369+0000 7f95c1194640 1 --2- 172.21.15.1:0/4262110506 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95bc1009d0 0x7f95bc1a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:25.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.369+0000 7f95c1995640 1 --2- 172.21.15.1:0/4262110506 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95bc105f50 0x7f95bc19e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:25.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.369+0000 7f95c0993640 1 -- 172.21.15.1:0/4262110506 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95bc105f50 msgr2=0x7f95bc19e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:25.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.369+0000 7f95c0993640 1 --2- 172.21.15.1:0/4262110506 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95bc105f50 0x7f95bc19e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:25.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.369+0000 7f95c0993640 1 -- 172.21.15.1:0/4262110506 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95bc1009d0 msgr2=0x7f95bc1a46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:25.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.369+0000 7f95c0993640 1 --2- 172.21.15.1:0/4262110506 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95bc1009d0 0x7f95bc1a46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:25.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.369+0000 7f95c0993640 1 -- 172.21.15.1:0/4262110506 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95bc19f020 con 0x7f95bc101380 2024-09-11T22:07:25.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.369+0000 7f95c1995640 1 --2- 172.21.15.1:0/4262110506 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95bc105f50 0x7f95bc19e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:07:25.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.369+0000 7f95c0993640 1 --2- 172.21.15.1:0/4262110506 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f95bc101380 0x7f95bc1a4be0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f95ac00d770 tx=0x7f95ac00db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:07:25.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.369+0000 7f95b27fc640 1 -- 172.21.15.1:0/4262110506 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ac017070 con 0x7f95bc101380 2024-09-11T22:07:25.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.369+0000 7f95b27fc640 1 -- 172.21.15.1:0/4262110506 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95ac009820 con 0x7f95bc101380 2024-09-11T22:07:25.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.369+0000 7f95c33f8640 1 -- 172.21.15.1:0/4262110506 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95bc19f310 con 0x7f95bc101380 2024-09-11T22:07:25.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.369+0000 7f95b27fc640 1 -- 172.21.15.1:0/4262110506 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ac004cd0 con 0x7f95bc101380 2024-09-11T22:07:25.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.369+0000 7f95c33f8640 1 -- 172.21.15.1:0/4262110506 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95bc1ab620 con 0x7f95bc101380 2024-09-11T22:07:25.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.370+0000 7f95c33f8640 1 -- 172.21.15.1:0/4262110506 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95bc06ab50 con 0x7f95bc101380 2024-09-11T22:07:25.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.372+0000 7f95b27fc640 1 -- 172.21.15.1:0/4262110506 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f95ac0099d0 con 0x7f95bc101380 2024-09-11T22:07:25.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.372+0000 7f95b27fc640 1 --2- 172.21.15.1:0/4262110506 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f95a80a2d00 0x7f95a80a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:25.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.372+0000 7f95b27fc640 1 -- 172.21.15.1:0/4262110506 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7f95ac0cf250 con 0x7f95bc101380 2024-09-11T22:07:25.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.372+0000 7f95c1194640 1 --2- 172.21.15.1:0/4262110506 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f95a80a2d00 0x7f95a80a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:25.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.373+0000 7f95c1194640 1 --2- 172.21.15.1:0/4262110506 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f95a80a2d00 0x7f95a80a51c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f95b800abe0 tx=0x7f95b800a2d0 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:07:25.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.375+0000 7f95b27fc640 1 -- 172.21.15.1:0/4262110506 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f95ac09be30 con 0x7f95bc101380 2024-09-11T22:07:25.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:25 smithi053 ceph-mon[34908]: pgmap v316: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:07:25.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:25 smithi053 ceph-mon[34908]: from='client.17841 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:07:25.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.551+0000 7f95c33f8640 1 -- 172.21.15.1:0/4262110506 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f95bc19f610 con 0x7f95bc101380 2024-09-11T22:07:25.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.551+0000 7f95b27fc640 1 -- 172.21.15.1:0/4262110506 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) ==== 93+0+10 (secure 0 0 0) 0x7f95ac09b730 con 0x7f95bc101380 2024-09-11T22:07:25.552 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-09-11T22:07:25.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.553+0000 7f95c33f8640 1 -- 172.21.15.1:0/4262110506 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f95a80a2d00 msgr2=0x7f95a80a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:25.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.553+0000 7f95c33f8640 1 --2- 172.21.15.1:0/4262110506 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f95a80a2d00 0x7f95a80a51c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f95b800abe0 tx=0x7f95b800a2d0 comp rx=0 tx=0).stop 2024-09-11T22:07:25.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.553+0000 7f95c33f8640 1 -- 172.21.15.1:0/4262110506 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f95bc101380 msgr2=0x7f95bc1a4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:25.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.553+0000 7f95c33f8640 1 --2- 172.21.15.1:0/4262110506 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f95bc101380 0x7f95bc1a4be0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f95ac00d770 tx=0x7f95ac00db70 comp rx=0 tx=0).stop 2024-09-11T22:07:25.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.553+0000 7f95c33f8640 1 -- 172.21.15.1:0/4262110506 shutdown_connections 2024-09-11T22:07:25.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.553+0000 7f95c33f8640 1 --2- 172.21.15.1:0/4262110506 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f95a80a2d00 0x7f95a80a51c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:25.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.553+0000 7f95c33f8640 1 --2- 172.21.15.1:0/4262110506 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95bc105f50 0x7f95bc19e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:25.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.553+0000 7f95c33f8640 1 --2- 172.21.15.1:0/4262110506 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f95bc101380 0x7f95bc1a4be0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:25.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.553+0000 7f95c33f8640 1 --2- 172.21.15.1:0/4262110506 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f95bc1009d0 0x7f95bc1a46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:25.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.553+0000 7f95c33f8640 1 -- 172.21.15.1:0/4262110506 >> 172.21.15.1:0/4262110506 conn(0x7f95bc0fc6a0 msgr2=0x7f95bc103810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:07:25.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.553+0000 7f95c33f8640 1 -- 172.21.15.1:0/4262110506 shutdown_connections 2024-09-11T22:07:25.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.554+0000 7f95c33f8640 1 -- 172.21.15.1:0/4262110506 wait complete. 2024-09-11T22:07:25.566 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-11T22:07:25.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:25 smithi161 ceph-mon[35615]: pgmap v316: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:07:25.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:25 smithi161 ceph-mon[35615]: from='client.17841 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:07:25.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.672+0000 7f9868f66640 1 -- 172.21.15.1:0/3627796515 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9864101710 msgr2=0x7f9864101b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:25.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.672+0000 7f9868f66640 1 --2- 172.21.15.1:0/3627796515 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9864101710 0x7f9864101b50 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f9858009900 tx=0x7f985802a410 comp rx=0 tx=0).stop 2024-09-11T22:07:25.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.672+0000 7f9868f66640 1 -- 172.21.15.1:0/3627796515 shutdown_connections 2024-09-11T22:07:25.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.672+0000 7f9868f66640 1 --2- 172.21.15.1:0/3627796515 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9864102090 0x7f986410d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:25.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.672+0000 7f9868f66640 1 --2- 172.21.15.1:0/3627796515 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9864101710 0x7f9864101b50 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:25.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.672+0000 7f9868f66640 1 --2- 172.21.15.1:0/3627796515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9864106b40 0x7f9864106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:25.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.672+0000 7f9868f66640 1 -- 172.21.15.1:0/3627796515 >> 172.21.15.1:0/3627796515 conn(0x7f9864075d90 msgr2=0x7f98640761a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:07:25.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.672+0000 7f9868f66640 1 -- 172.21.15.1:0/3627796515 shutdown_connections 2024-09-11T22:07:25.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.672+0000 7f9868f66640 1 -- 172.21.15.1:0/3627796515 wait complete. 2024-09-11T22:07:25.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.673+0000 7f9868f66640 1 Processor -- start 2024-09-11T22:07:25.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.673+0000 7f9868f66640 1 -- start start 2024-09-11T22:07:25.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.673+0000 7f9868f66640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9864101710 0x7f986419a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:25.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.673+0000 7f9868f66640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9864102090 0x7f986419ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:25.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.673+0000 7f9868f66640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9864106b40 0x7f986419eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:25.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.673+0000 7f9868f66640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9864110ea0 con 0x7f9864102090 2024-09-11T22:07:25.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.673+0000 7f9868f66640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f9864110d20 con 0x7f9864101710 2024-09-11T22:07:25.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.673+0000 7f9868f66640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f9864111020 con 0x7f9864106b40 2024-09-11T22:07:25.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.673+0000 7f9861d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9864102090 0x7f986419ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:25.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.673+0000 7f9861d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9864102090 0x7f986419ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42134/0 (socket says 172.21.15.1:42134) 2024-09-11T22:07:25.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.673+0000 7f9861d9b640 1 -- 172.21.15.1:0/426354021 learned_addr learned my addr 172.21.15.1:0/426354021 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:07:25.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.674+0000 7f986259c640 1 --2- 172.21.15.1:0/426354021 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9864101710 0x7f986419a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:25.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.674+0000 7f9862d9d640 1 --2- 172.21.15.1:0/426354021 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9864106b40 0x7f986419eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:25.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.674+0000 7f9861d9b640 1 -- 172.21.15.1:0/426354021 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9864106b40 msgr2=0x7f986419eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:25.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.674+0000 7f9861d9b640 1 --2- 172.21.15.1:0/426354021 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9864106b40 0x7f986419eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:25.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.674+0000 7f9861d9b640 1 -- 172.21.15.1:0/426354021 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9864101710 msgr2=0x7f986419a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:25.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.674+0000 7f9861d9b640 1 --2- 172.21.15.1:0/426354021 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9864101710 0x7f986419a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:25.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.674+0000 7f9861d9b640 1 -- 172.21.15.1:0/426354021 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f986419f610 con 0x7f9864102090 2024-09-11T22:07:25.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.674+0000 7f986259c640 1 --2- 172.21.15.1:0/426354021 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9864101710 0x7f986419a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:07:25.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.674+0000 7f9862d9d640 1 --2- 172.21.15.1:0/426354021 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9864106b40 0x7f986419eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:07:25.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.674+0000 7f9861d9b640 1 --2- 172.21.15.1:0/426354021 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9864102090 0x7f986419ab60 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f9858006010 tx=0x7f98580042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:07:25.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.674+0000 7f98537fe640 1 -- 172.21.15.1:0/426354021 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985803f030 con 0x7f9864102090 2024-09-11T22:07:25.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.674+0000 7f98537fe640 1 -- 172.21.15.1:0/426354021 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9858003ce0 con 0x7f9864102090 2024-09-11T22:07:25.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.674+0000 7f98537fe640 1 -- 172.21.15.1:0/426354021 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9858035070 con 0x7f9864102090 2024-09-11T22:07:25.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.674+0000 7f9868f66640 1 -- 172.21.15.1:0/426354021 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f986419f8a0 con 0x7f9864102090 2024-09-11T22:07:25.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.674+0000 7f9868f66640 1 -- 172.21.15.1:0/426354021 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98641a7190 con 0x7f9864102090 2024-09-11T22:07:25.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.675+0000 7f9868f66640 1 -- 172.21.15.1:0/426354021 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9864101b50 con 0x7f9864102090 2024-09-11T22:07:25.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.676+0000 7f98537fe640 1 -- 172.21.15.1:0/426354021 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f9858042030 con 0x7f9864102090 2024-09-11T22:07:25.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.677+0000 7f98537fe640 1 --2- 172.21.15.1:0/426354021 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f984c0a2d00 0x7f984c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:25.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.677+0000 7f98537fe640 1 -- 172.21.15.1:0/426354021 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7f98580efde0 con 0x7f9864102090 2024-09-11T22:07:25.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.677+0000 7f986259c640 1 --2- 172.21.15.1:0/426354021 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f984c0a2d00 0x7f984c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:25.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.677+0000 7f986259c640 1 --2- 172.21.15.1:0/426354021 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f984c0a2d00 0x7f984c0a51c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f985400baa0 tx=0x7f9854005d20 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:07:25.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.680+0000 7f98537fe640 1 -- 172.21.15.1:0/426354021 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f98580bca80 con 0x7f9864102090 2024-09-11T22:07:25.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.857+0000 7f9868f66640 1 -- 172.21.15.1:0/426354021 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f98641a7360 con 0x7f9864102090 2024-09-11T22:07:25.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.858+0000 7f98537fe640 1 -- 172.21.15.1:0/426354021 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) ==== 104+0+11 (secure 0 0 0) 0x7f98580bc380 con 0x7f9864102090 2024-09-11T22:07:25.859 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-09-11T22:07:25.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.860+0000 7f9868f66640 1 -- 172.21.15.1:0/426354021 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f984c0a2d00 msgr2=0x7f984c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:25.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.860+0000 7f9868f66640 1 --2- 172.21.15.1:0/426354021 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f984c0a2d00 0x7f984c0a51c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f985400baa0 tx=0x7f9854005d20 comp rx=0 tx=0).stop 2024-09-11T22:07:25.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.860+0000 7f9868f66640 1 -- 172.21.15.1:0/426354021 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9864102090 msgr2=0x7f986419ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:25.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.860+0000 7f9868f66640 1 --2- 172.21.15.1:0/426354021 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9864102090 0x7f986419ab60 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f9858006010 tx=0x7f98580042f0 comp rx=0 tx=0).stop 2024-09-11T22:07:25.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.860+0000 7f9868f66640 1 -- 172.21.15.1:0/426354021 shutdown_connections 2024-09-11T22:07:25.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.860+0000 7f9868f66640 1 --2- 172.21.15.1:0/426354021 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f984c0a2d00 0x7f984c0a51c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:25.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.860+0000 7f9868f66640 1 --2- 172.21.15.1:0/426354021 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9864106b40 0x7f986419eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:25.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.860+0000 7f9868f66640 1 --2- 172.21.15.1:0/426354021 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9864102090 0x7f986419ab60 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:25.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.860+0000 7f9868f66640 1 --2- 172.21.15.1:0/426354021 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9864101710 0x7f986419a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:25.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.861+0000 7f9868f66640 1 -- 172.21.15.1:0/426354021 >> 172.21.15.1:0/426354021 conn(0x7f9864075d90 msgr2=0x7f986410bc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:07:25.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.861+0000 7f9868f66640 1 -- 172.21.15.1:0/426354021 shutdown_connections 2024-09-11T22:07:25.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.861+0000 7f9868f66640 1 -- 172.21.15.1:0/426354021 wait complete. 2024-09-11T22:07:25.873 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-11T22:07:25.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.979+0000 7fce629d8640 1 -- 172.21.15.1:0/4175059385 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fce5c100e00 msgr2=0x7fce5c101260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:25.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.979+0000 7fce629d8640 1 --2- 172.21.15.1:0/4175059385 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fce5c100e00 0x7fce5c101260 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fce50009930 tx=0x7fce5002a420 comp rx=0 tx=0).stop 2024-09-11T22:07:25.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.979+0000 7fce629d8640 1 -- 172.21.15.1:0/4175059385 shutdown_connections 2024-09-11T22:07:25.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.979+0000 7fce629d8640 1 --2- 172.21.15.1:0/4175059385 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce5c1017a0 0x7fce5c10d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:25.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.979+0000 7fce629d8640 1 --2- 172.21.15.1:0/4175059385 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fce5c100e00 0x7fce5c101260 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:25.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.979+0000 7fce629d8640 1 --2- 172.21.15.1:0/4175059385 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce5c1069b0 0x7fce5c106d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:25.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.979+0000 7fce629d8640 1 -- 172.21.15.1:0/4175059385 >> 172.21.15.1:0/4175059385 conn(0x7fce5c0766f0 msgr2=0x7fce5c076b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:07:25.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.979+0000 7fce629d8640 1 -- 172.21.15.1:0/4175059385 shutdown_connections 2024-09-11T22:07:25.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.979+0000 7fce629d8640 1 -- 172.21.15.1:0/4175059385 wait complete. 2024-09-11T22:07:25.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.980+0000 7fce629d8640 1 Processor -- start 2024-09-11T22:07:25.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.980+0000 7fce629d8640 1 -- start start 2024-09-11T22:07:25.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.980+0000 7fce629d8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fce5c100e00 0x7fce5c198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:25.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.980+0000 7fce629d8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce5c1017a0 0x7fce5c198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:25.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.980+0000 7fce629d8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce5c1069b0 0x7fce5c19ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:25.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.980+0000 7fce629d8640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fce5c06a3f0 con 0x7fce5c100e00 2024-09-11T22:07:25.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.980+0000 7fce629d8640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fce5c06a270 con 0x7fce5c1017a0 2024-09-11T22:07:25.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.980+0000 7fce629d8640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fce5c112150 con 0x7fce5c1069b0 2024-09-11T22:07:25.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.980+0000 7fce5bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fce5c100e00 0x7fce5c198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:25.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.980+0000 7fce5bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fce5c100e00 0x7fce5c198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42158/0 (socket says 172.21.15.1:42158) 2024-09-11T22:07:25.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.980+0000 7fce5bfff640 1 -- 172.21.15.1:0/446349685 learned_addr learned my addr 172.21.15.1:0/446349685 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:07:25.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.980+0000 7fce5b7fe640 1 --2- 172.21.15.1:0/446349685 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce5c1017a0 0x7fce5c198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:25.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.980+0000 7fce60f75640 1 --2- 172.21.15.1:0/446349685 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce5c1069b0 0x7fce5c19ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:25.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.981+0000 7fce5bfff640 1 -- 172.21.15.1:0/446349685 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce5c1069b0 msgr2=0x7fce5c19ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:25.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.981+0000 7fce5bfff640 1 --2- 172.21.15.1:0/446349685 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce5c1069b0 0x7fce5c19ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:25.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.981+0000 7fce5bfff640 1 -- 172.21.15.1:0/446349685 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce5c1017a0 msgr2=0x7fce5c198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:25.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.981+0000 7fce5bfff640 1 --2- 172.21.15.1:0/446349685 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce5c1017a0 0x7fce5c198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:25.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.981+0000 7fce5bfff640 1 -- 172.21.15.1:0/446349685 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce5c19d400 con 0x7fce5c100e00 2024-09-11T22:07:25.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.981+0000 7fce5bfff640 1 --2- 172.21.15.1:0/446349685 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fce5c100e00 0x7fce5c198410 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fce4c00c890 tx=0x7fce4c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:07:25.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.981+0000 7fce597fa640 1 -- 172.21.15.1:0/446349685 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce4c004220 con 0x7fce5c100e00 2024-09-11T22:07:25.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.981+0000 7fce597fa640 1 -- 172.21.15.1:0/446349685 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce4c0043c0 con 0x7fce5c100e00 2024-09-11T22:07:25.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.981+0000 7fce629d8640 1 -- 172.21.15.1:0/446349685 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce5c19d6f0 con 0x7fce5c100e00 2024-09-11T22:07:25.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.981+0000 7fce597fa640 1 -- 172.21.15.1:0/446349685 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce4c0049e0 con 0x7fce5c100e00 2024-09-11T22:07:25.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.981+0000 7fce629d8640 1 -- 172.21.15.1:0/446349685 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce5c1a4f80 con 0x7fce5c100e00 2024-09-11T22:07:25.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.982+0000 7fce629d8640 1 -- 172.21.15.1:0/446349685 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce5c101260 con 0x7fce5c100e00 2024-09-11T22:07:25.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.983+0000 7fce597fa640 1 -- 172.21.15.1:0/446349685 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7fce4c0079b0 con 0x7fce5c100e00 2024-09-11T22:07:25.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.984+0000 7fce597fa640 1 --2- 172.21.15.1:0/446349685 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fce440a2d00 0x7fce440a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:25.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.984+0000 7fce5b7fe640 1 --2- 172.21.15.1:0/446349685 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fce440a2d00 0x7fce440a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:25.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.984+0000 7fce597fa640 1 -- 172.21.15.1:0/446349685 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7fce4c0cf400 con 0x7fce5c100e00 2024-09-11T22:07:25.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.985+0000 7fce5b7fe640 1 --2- 172.21.15.1:0/446349685 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fce440a2d00 0x7fce440a51c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fce50006010 tx=0x7fce50000bc0 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:07:25.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:25.988+0000 7fce597fa640 1 -- 172.21.15.1:0/446349685 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fce4c014030 con 0x7fce5c100e00 2024-09-11T22:07:26.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:26.164+0000 7fce629d8640 1 -- 172.21.15.1:0/446349685 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fce5c02d7c0 con 0x7fce5c100e00 2024-09-11T22:07:26.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:26.165+0000 7fce597fa640 1 -- 172.21.15.1:0/446349685 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) ==== 100+0+11 (secure 0 0 0) 0x7fce4c09b990 con 0x7fce5c100e00 2024-09-11T22:07:26.166 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-09-11T22:07:26.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:26.167+0000 7fce629d8640 1 -- 172.21.15.1:0/446349685 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fce440a2d00 msgr2=0x7fce440a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:26.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:26.167+0000 7fce629d8640 1 --2- 172.21.15.1:0/446349685 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fce440a2d00 0x7fce440a51c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fce50006010 tx=0x7fce50000bc0 comp rx=0 tx=0).stop 2024-09-11T22:07:26.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:26.167+0000 7fce629d8640 1 -- 172.21.15.1:0/446349685 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fce5c100e00 msgr2=0x7fce5c198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:26.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:26.167+0000 7fce629d8640 1 --2- 172.21.15.1:0/446349685 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fce5c100e00 0x7fce5c198410 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fce4c00c890 tx=0x7fce4c00cd50 comp rx=0 tx=0).stop 2024-09-11T22:07:26.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:26.167+0000 7fce629d8640 1 -- 172.21.15.1:0/446349685 shutdown_connections 2024-09-11T22:07:26.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:26.167+0000 7fce629d8640 1 --2- 172.21.15.1:0/446349685 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fce440a2d00 0x7fce440a51c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:26.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:26.167+0000 7fce629d8640 1 --2- 172.21.15.1:0/446349685 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fce5c1069b0 0x7fce5c19ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:26.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:26.167+0000 7fce629d8640 1 --2- 172.21.15.1:0/446349685 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fce5c1017a0 0x7fce5c198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:26.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:26.167+0000 7fce629d8640 1 --2- 172.21.15.1:0/446349685 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fce5c100e00 0x7fce5c198410 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:26.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:26.167+0000 7fce629d8640 1 -- 172.21.15.1:0/446349685 >> 172.21.15.1:0/446349685 conn(0x7fce5c0766f0 msgr2=0x7fce5c0fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:07:26.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:26.167+0000 7fce629d8640 1 -- 172.21.15.1:0/446349685 shutdown_connections 2024-09-11T22:07:26.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:26.167+0000 7fce629d8640 1 -- 172.21.15.1:0/446349685 wait complete. 2024-09-11T22:07:26.179 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-11T22:07:26.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:26.282+0000 7f307ea74640 1 -- 172.21.15.1:0/2642856441 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f307810c100 msgr2=0x7f307810e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:26.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:26.282+0000 7f307ea74640 1 --2- 172.21.15.1:0/2642856441 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f307810c100 0x7f307810e550 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f3070009900 tx=0x7f307002a410 comp rx=0 tx=0).stop 2024-09-11T22:07:26.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:26.282+0000 7f307ea74640 1 -- 172.21.15.1:0/2642856441 shutdown_connections 2024-09-11T22:07:26.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:26.282+0000 7f307ea74640 1 --2- 172.21.15.1:0/2642856441 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f307810c100 0x7f307810e550 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:26.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:26.282+0000 7f307ea74640 1 --2- 172.21.15.1:0/2642856441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f30781035b0 0x7f307810ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:26.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:26.282+0000 7f307ea74640 1 --2- 172.21.15.1:0/2642856441 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3078102c00 0x7f3078102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:26.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:26.282+0000 7f307ea74640 1 -- 172.21.15.1:0/2642856441 >> 172.21.15.1:0/2642856441 conn(0x7f30780fc6e0 msgr2=0x7f30780feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:07:26.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:26.282+0000 7f307ea74640 1 -- 172.21.15.1:0/2642856441 shutdown_connections 2024-09-11T22:07:26.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:26.282+0000 7f307ea74640 1 -- 172.21.15.1:0/2642856441 wait complete. 2024-09-11T22:07:26.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:26.282+0000 7f307ea74640 1 Processor -- start 2024-09-11T22:07:26.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:26.283+0000 7f307ea74640 1 -- start start 2024-09-11T22:07:26.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:26.283+0000 7f307ea74640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3078102c00 0x7f30781982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:26.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:26.283+0000 7f307ea74640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f30781035b0 0x7f30781987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:26.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:26.283+0000 7f307ea74640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f307810c100 0x7f307819cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:26.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:26.283+0000 7f307ea74640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3078111eb0 con 0x7f30781035b0 2024-09-11T22:07:26.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:26.283+0000 7f307ea74640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f3078111d30 con 0x7f3078102c00 2024-09-11T22:07:26.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:26.283+0000 7f307ea74640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f3078112030 con 0x7f307810c100 2024-09-11T22:07:26.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:26.283+0000 7f3077fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f30781035b0 0x7f30781987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:26.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:26.283+0000 7f3077fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f30781035b0 0x7f30781987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42174/0 (socket says 172.21.15.1:42174) 2024-09-11T22:07:26.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:26.283+0000 7f307c810640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3078102c00 0x7f30781982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:26.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:26.283+0000 7f3077fff640 1 -- 172.21.15.1:0/227232084 learned_addr learned my addr 172.21.15.1:0/227232084 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:07:26.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:26.283+0000 7f307d011640 1 --2- 172.21.15.1:0/227232084 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f307810c100 0x7f307819cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:26.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:26.283+0000 7f3077fff640 1 -- 172.21.15.1:0/227232084 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f307810c100 msgr2=0x7f307819cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:26.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:26.283+0000 7f3077fff640 1 --2- 172.21.15.1:0/227232084 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f307810c100 0x7f307819cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:26.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:26.283+0000 7f3077fff640 1 -- 172.21.15.1:0/227232084 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3078102c00 msgr2=0x7f30781982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:26.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:26.283+0000 7f3077fff640 1 --2- 172.21.15.1:0/227232084 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3078102c00 0x7f30781982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:26.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:26.283+0000 7f3077fff640 1 -- 172.21.15.1:0/227232084 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f307819d290 con 0x7f30781035b0 2024-09-11T22:07:26.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:26.283+0000 7f3077fff640 1 --2- 172.21.15.1:0/227232084 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f30781035b0 0x7f30781987e0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f306800cce0 tx=0x7f306800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:07:26.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:26.284+0000 7f3075ffb640 1 -- 172.21.15.1:0/227232084 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3068004b20 con 0x7f30781035b0 2024-09-11T22:07:26.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:26.284+0000 7f307ea74640 1 -- 172.21.15.1:0/227232084 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f307819d580 con 0x7f30781035b0 2024-09-11T22:07:26.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:26.284+0000 7f3075ffb640 1 -- 172.21.15.1:0/227232084 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3068004cc0 con 0x7f30781035b0 2024-09-11T22:07:26.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:26.284+0000 7f3075ffb640 1 -- 172.21.15.1:0/227232084 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306800f850 con 0x7f30781035b0 2024-09-11T22:07:26.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:26.284+0000 7f307ea74640 1 -- 172.21.15.1:0/227232084 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30781a4e10 con 0x7f30781035b0 2024-09-11T22:07:26.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:26.285+0000 7f307ea74640 1 -- 172.21.15.1:0/227232084 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f307806abf0 con 0x7f30781035b0 2024-09-11T22:07:26.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:26.286+0000 7f3075ffb640 1 -- 172.21.15.1:0/227232084 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f3068005590 con 0x7f30781035b0 2024-09-11T22:07:26.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:26.287+0000 7f3075ffb640 1 --2- 172.21.15.1:0/227232084 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f30640a2d00 0x7f30640a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:26.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:26.287+0000 7f307c810640 1 --2- 172.21.15.1:0/227232084 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f30640a2d00 0x7f30640a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:26.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:26.287+0000 7f3075ffb640 1 -- 172.21.15.1:0/227232084 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7f30680cf8b0 con 0x7f30781035b0 2024-09-11T22:07:26.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:26.287+0000 7f307c810640 1 --2- 172.21.15.1:0/227232084 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f30640a2d00 0x7f30640a51c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f306000abe0 tx=0x7f306000a2d0 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:07:26.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:26.290+0000 7f3075ffb640 1 -- 172.21.15.1:0/227232084 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3068014030 con 0x7f30781035b0 2024-09-11T22:07:26.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:26 smithi001 ceph-mon[30935]: from='client.17847 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:07:26.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:26 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/1298310486' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:07:26.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:26 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/4262110506' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T22:07:26.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:26 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/426354021' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-11T22:07:26.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:26.514+0000 7f307ea74640 1 -- 172.21.15.1:0/227232084 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f3078102fe0 con 0x7f30781035b0 2024-09-11T22:07:26.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:26.522+0000 7f3075ffb640 1 -- 172.21.15.1:0/227232084 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+740892 (secure 0 0 0) 0x7f3078102fe0 con 0x7f30781035b0 2024-09-11T22:07:26.532 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T22:07:26.532 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-11T22:07:26.533 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 26, 2024-09-11T22:07:26.533 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.533 INFO:teuthology.orchestra.run.smithi001.stdout: "active_gid": 15204, 2024-09-11T22:07:26.533 INFO:teuthology.orchestra.run.smithi001.stdout: "active_name": "x", 2024-09-11T22:07:26.533 INFO:teuthology.orchestra.run.smithi001.stdout: "active_addrs": { 2024-09-11T22:07:26.533 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-11T22:07:26.533 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:26.533 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-11T22:07:26.534 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6800", 2024-09-11T22:07:26.534 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 3543150772 2024-09-11T22:07:26.534 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.534 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:26.534 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-09-11T22:07:26.534 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6801", 2024-09-11T22:07:26.534 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 3543150772 2024-09-11T22:07:26.534 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:26.535 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-11T22:07:26.535 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.535 INFO:teuthology.orchestra.run.smithi001.stdout: "active_addr": "172.21.15.1:6801/3543150772", 2024-09-11T22:07:26.535 INFO:teuthology.orchestra.run.smithi001.stdout: "active_change": "2024-09-11T21:57:13.664606+0000", 2024-09-11T22:07:26.535 INFO:teuthology.orchestra.run.smithi001.stdout: "active_mgr_features": 4540701547738038271, 2024-09-11T22:07:26.535 INFO:teuthology.orchestra.run.smithi001.stdout: "available": true, 2024-09-11T22:07:26.535 INFO:teuthology.orchestra.run.smithi001.stdout: "standbys": [ 2024-09-11T22:07:26.536 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:26.536 INFO:teuthology.orchestra.run.smithi001.stdout: "gid": 24869, 2024-09-11T22:07:26.536 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "y", 2024-09-11T22:07:26.536 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_features": 4540701547738038271, 2024-09-11T22:07:26.536 INFO:teuthology.orchestra.run.smithi001.stdout: "available_modules": [ 2024-09-11T22:07:26.536 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:26.536 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "alerts", 2024-09-11T22:07:26.536 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:26.537 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:26.537 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:26.537 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-09-11T22:07:26.537 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-09-11T22:07:26.537 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-11T22:07:26.537 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.537 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.538 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-09-11T22:07:26.538 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.538 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.538 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.538 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How frequently to reexamine health status", 2024-09-11T22:07:26.538 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.538 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.538 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.539 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.539 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:26.539 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:26.539 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.539 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.539 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.539 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:26.539 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.540 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.540 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:26.540 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:26.540 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:26.540 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:26.540 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:26.540 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:26.541 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:26.541 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:26.541 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.541 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.541 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.541 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.541 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.541 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:26.542 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:26.542 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.542 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.542 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.542 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:26.542 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.542 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.542 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.543 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.543 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.543 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.543 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.543 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.543 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:26.543 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:26.543 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.544 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.544 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.544 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:26.544 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.544 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.544 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:26.544 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:26.544 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:26.545 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:26.545 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:26.545 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:26.545 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:26.545 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:26.545 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.545 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.545 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.546 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.546 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.546 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:26.546 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:26.546 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.546 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.546 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.546 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:26.546 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.547 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.547 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.547 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.547 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.547 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.547 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.547 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.547 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_destination": { 2024-09-11T22:07:26.548 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_destination", 2024-09-11T22:07:26.548 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.548 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.548 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.548 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:26.548 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.548 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.548 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.549 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Email address to send alerts to", 2024-09-11T22:07:26.549 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.549 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.549 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.549 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.549 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_from_name": { 2024-09-11T22:07:26.549 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_from_name", 2024-09-11T22:07:26.549 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.550 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.550 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.550 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "Ceph", 2024-09-11T22:07:26.550 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.550 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.550 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.550 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Email From: name", 2024-09-11T22:07:26.550 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.551 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.551 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.551 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.551 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_host": { 2024-09-11T22:07:26.551 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_host", 2024-09-11T22:07:26.551 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.551 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.551 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.552 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:26.552 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.552 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.552 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.552 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SMTP server", 2024-09-11T22:07:26.552 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.552 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.552 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.553 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.553 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_password": { 2024-09-11T22:07:26.553 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_password", 2024-09-11T22:07:26.553 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.553 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.553 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.553 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:26.553 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.554 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.554 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.554 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Password to authenticate with", 2024-09-11T22:07:26.554 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.554 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.554 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.554 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.554 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_port": { 2024-09-11T22:07:26.555 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_port", 2024-09-11T22:07:26.555 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:26.555 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.555 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.555 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "465", 2024-09-11T22:07:26.555 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.555 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.555 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.556 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SMTP port", 2024-09-11T22:07:26.556 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.556 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.556 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.556 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.556 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_sender": { 2024-09-11T22:07:26.556 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_sender", 2024-09-11T22:07:26.556 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.556 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.557 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.557 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:26.557 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.557 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.557 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.557 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SMTP envelope sender", 2024-09-11T22:07:26.557 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.557 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.558 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.558 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.558 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_ssl": { 2024-09-11T22:07:26.558 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_ssl", 2024-09-11T22:07:26.558 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.558 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.558 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.559 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:26.559 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.559 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.559 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.559 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-11T22:07:26.559 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.559 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.559 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.560 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.560 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_user": { 2024-09-11T22:07:26.560 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_user", 2024-09-11T22:07:26.560 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.560 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.560 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.560 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:26.560 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.561 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.561 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.561 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "User to authenticate as", 2024-09-11T22:07:26.561 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.561 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.561 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.561 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:26.561 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:26.562 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.562 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:26.562 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "balancer", 2024-09-11T22:07:26.562 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:26.562 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:26.562 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:26.562 INFO:teuthology.orchestra.run.smithi001.stdout: "active": { 2024-09-11T22:07:26.562 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "active", 2024-09-11T22:07:26.563 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.563 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.563 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.563 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:26.563 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.563 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.563 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.563 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "automatically balance PGs across cluster", 2024-09-11T22:07:26.564 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.564 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.564 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.564 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.564 INFO:teuthology.orchestra.run.smithi001.stdout: "begin_time": { 2024-09-11T22:07:26.564 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "begin_time", 2024-09-11T22:07:26.564 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.564 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.565 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.565 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0000", 2024-09-11T22:07:26.565 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.565 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.565 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.565 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "beginning time of day to automatically balance", 2024-09-11T22:07:26.565 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-11T22:07:26.565 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.566 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.566 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.566 INFO:teuthology.orchestra.run.smithi001.stdout: "begin_weekday": { 2024-09-11T22:07:26.566 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "begin_weekday", 2024-09-11T22:07:26.566 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-09-11T22:07:26.566 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.566 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.566 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-09-11T22:07:26.567 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "0", 2024-09-11T22:07:26.567 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "6", 2024-09-11T22:07:26.567 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.567 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-11T22:07:26.567 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-11T22:07:26.567 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.567 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.567 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.568 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_compat_max_iterations": { 2024-09-11T22:07:26.568 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crush_compat_max_iterations", 2024-09-11T22:07:26.568 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-09-11T22:07:26.568 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.568 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.568 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "25", 2024-09-11T22:07:26.568 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-09-11T22:07:26.568 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "250", 2024-09-11T22:07:26.569 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.569 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-11T22:07:26.569 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.569 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.569 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.569 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.569 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_compat_metrics": { 2024-09-11T22:07:26.569 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crush_compat_metrics", 2024-09-11T22:07:26.570 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.570 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.570 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.570 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "pgs,objects,bytes", 2024-09-11T22:07:26.570 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.570 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.570 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.570 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-11T22:07:26.571 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-11T22:07:26.571 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.571 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.571 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.571 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_compat_step": { 2024-09-11T22:07:26.571 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crush_compat_step", 2024-09-11T22:07:26.571 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-09-11T22:07:26.571 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.572 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.572 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0.5", 2024-09-11T22:07:26.572 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "0.001", 2024-09-11T22:07:26.572 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "0.999", 2024-09-11T22:07:26.572 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.572 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "aggressiveness of optimization", 2024-09-11T22:07:26.572 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-11T22:07:26.572 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.573 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.573 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.573 INFO:teuthology.orchestra.run.smithi001.stdout: "end_time": { 2024-09-11T22:07:26.573 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "end_time", 2024-09-11T22:07:26.573 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.573 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.573 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.573 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "2359", 2024-09-11T22:07:26.574 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.574 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.574 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.574 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "ending time of day to automatically balance", 2024-09-11T22:07:26.574 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-11T22:07:26.574 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.574 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.575 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.575 INFO:teuthology.orchestra.run.smithi001.stdout: "end_weekday": { 2024-09-11T22:07:26.575 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "end_weekday", 2024-09-11T22:07:26.575 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-09-11T22:07:26.575 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.575 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.575 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-09-11T22:07:26.575 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "0", 2024-09-11T22:07:26.576 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "6", 2024-09-11T22:07:26.576 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.576 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-11T22:07:26.576 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-11T22:07:26.576 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.576 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.576 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.577 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:26.577 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:26.577 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.577 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.577 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.577 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:26.577 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.577 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.577 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:26.578 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:26.578 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:26.578 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:26.578 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:26.578 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:26.578 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:26.578 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:26.578 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.579 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.579 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.579 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.579 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.579 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:26.579 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:26.579 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.579 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.580 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.580 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:26.580 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.580 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.580 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.580 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.581 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.581 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.581 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.581 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.581 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:26.581 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:26.581 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.581 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.581 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.582 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:26.582 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.582 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.582 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:26.582 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:26.582 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:26.582 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:26.582 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:26.583 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:26.583 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:26.583 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:26.583 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.583 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.583 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.583 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.583 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.584 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:26.584 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:26.584 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.584 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.584 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.584 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:26.584 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.584 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.585 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.585 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.585 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.585 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.585 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.585 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.585 INFO:teuthology.orchestra.run.smithi001.stdout: "min_score": { 2024-09-11T22:07:26.585 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "min_score", 2024-09-11T22:07:26.586 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-09-11T22:07:26.586 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.586 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.586 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-09-11T22:07:26.586 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.586 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.586 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.586 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-11T22:07:26.587 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.587 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.587 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.587 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.587 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": { 2024-09-11T22:07:26.587 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mode", 2024-09-11T22:07:26.587 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.587 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.587 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.588 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "upmap", 2024-09-11T22:07:26.588 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.588 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.588 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:26.588 INFO:teuthology.orchestra.run.smithi001.stdout: "crush-compat", 2024-09-11T22:07:26.588 INFO:teuthology.orchestra.run.smithi001.stdout: "none", 2024-09-11T22:07:26.588 INFO:teuthology.orchestra.run.smithi001.stdout: "read", 2024-09-11T22:07:26.588 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap", 2024-09-11T22:07:26.589 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap-read" 2024-09-11T22:07:26.589 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:26.589 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Balancer mode", 2024-09-11T22:07:26.589 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.589 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.589 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.589 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.589 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_ids": { 2024-09-11T22:07:26.590 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pool_ids", 2024-09-11T22:07:26.590 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.590 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.590 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.590 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:26.590 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.590 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.590 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.591 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-11T22:07:26.591 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.591 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.591 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.591 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.591 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-09-11T22:07:26.591 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-09-11T22:07:26.591 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-11T22:07:26.592 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.592 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.592 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-09-11T22:07:26.592 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.592 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.592 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.592 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-11T22:07:26.593 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.593 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.593 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.593 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.593 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap_max_deviation": { 2024-09-11T22:07:26.593 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "upmap_max_deviation", 2024-09-11T22:07:26.593 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:26.593 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.594 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.594 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-09-11T22:07:26.594 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-09-11T22:07:26.594 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.594 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.594 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-11T22:07:26.594 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-11T22:07:26.594 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.595 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.595 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.595 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap_max_optimizations": { 2024-09-11T22:07:26.595 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "upmap_max_optimizations", 2024-09-11T22:07:26.595 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-09-11T22:07:26.595 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.595 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.595 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-11T22:07:26.596 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.596 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.596 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.596 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-11T22:07:26.596 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.596 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.596 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.597 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:26.597 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:26.597 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.597 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:26.597 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cephadm", 2024-09-11T22:07:26.597 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:26.597 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:26.597 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:26.598 INFO:teuthology.orchestra.run.smithi001.stdout: "agent_down_multiplier": { 2024-09-11T22:07:26.598 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "agent_down_multiplier", 2024-09-11T22:07:26.598 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-09-11T22:07:26.598 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.598 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.598 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "3.0", 2024-09-11T22:07:26.598 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.598 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.599 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.599 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-11T22:07:26.599 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.599 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.599 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.599 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.599 INFO:teuthology.orchestra.run.smithi001.stdout: "agent_refresh_rate": { 2024-09-11T22:07:26.600 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "agent_refresh_rate", 2024-09-11T22:07:26.600 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-11T22:07:26.600 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.600 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.600 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "20", 2024-09-11T22:07:26.600 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.600 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.601 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.601 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-11T22:07:26.601 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.601 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.601 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.601 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.601 INFO:teuthology.orchestra.run.smithi001.stdout: "agent_starting_port": { 2024-09-11T22:07:26.602 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "agent_starting_port", 2024-09-11T22:07:26.602 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:26.602 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.602 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.602 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "4721", 2024-09-11T22:07:26.602 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.602 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.602 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.603 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-11T22:07:26.603 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.603 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.603 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.603 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.603 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_ptrace": { 2024-09-11T22:07:26.603 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "allow_ptrace", 2024-09-11T22:07:26.604 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.604 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.604 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.604 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:26.604 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.604 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.604 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.605 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-11T22:07:26.605 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-11T22:07:26.605 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.605 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.605 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.605 INFO:teuthology.orchestra.run.smithi001.stdout: "autotune_interval": { 2024-09-11T22:07:26.605 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "autotune_interval", 2024-09-11T22:07:26.605 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-11T22:07:26.606 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.606 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.606 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-09-11T22:07:26.606 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.606 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.606 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.606 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-11T22:07:26.606 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.607 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.607 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.607 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.607 INFO:teuthology.orchestra.run.smithi001.stdout: "autotune_memory_target_ratio": { 2024-09-11T22:07:26.607 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "autotune_memory_target_ratio", 2024-09-11T22:07:26.607 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-09-11T22:07:26.607 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.607 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.608 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0.7", 2024-09-11T22:07:26.608 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.608 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.608 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.608 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-11T22:07:26.608 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.608 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.609 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.609 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.609 INFO:teuthology.orchestra.run.smithi001.stdout: "cephadm_log_destination": { 2024-09-11T22:07:26.609 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cephadm_log_destination", 2024-09-11T22:07:26.609 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.609 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.609 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.609 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:26.610 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.610 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.610 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:26.610 INFO:teuthology.orchestra.run.smithi001.stdout: "file", 2024-09-11T22:07:26.610 INFO:teuthology.orchestra.run.smithi001.stdout: "file,syslog", 2024-09-11T22:07:26.610 INFO:teuthology.orchestra.run.smithi001.stdout: "syslog" 2024-09-11T22:07:26.610 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:26.610 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-11T22:07:26.611 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.611 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.611 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.611 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.611 INFO:teuthology.orchestra.run.smithi001.stdout: "cgroups_split": { 2024-09-11T22:07:26.611 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cgroups_split", 2024-09-11T22:07:26.611 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.611 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.612 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.612 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:26.612 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.612 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.612 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.612 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-11T22:07:26.612 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.613 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.613 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.613 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.613 INFO:teuthology.orchestra.run.smithi001.stdout: "config_checks_enabled": { 2024-09-11T22:07:26.613 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "config_checks_enabled", 2024-09-11T22:07:26.613 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.613 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.613 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.614 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:26.614 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.614 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.614 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.614 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-11T22:07:26.614 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.614 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.614 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.615 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.615 INFO:teuthology.orchestra.run.smithi001.stdout: "config_dashboard": { 2024-09-11T22:07:26.615 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "config_dashboard", 2024-09-11T22:07:26.615 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.615 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.615 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.615 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:26.615 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.616 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.616 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.616 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-11T22:07:26.616 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.616 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.616 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.616 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.616 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_alertmanager": { 2024-09-11T22:07:26.617 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_alertmanager", 2024-09-11T22:07:26.617 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.617 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.617 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.617 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-11T22:07:26.617 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.617 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.617 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.618 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-09-11T22:07:26.618 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.618 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.618 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.618 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.618 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_base": { 2024-09-11T22:07:26.618 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_base", 2024-09-11T22:07:26.618 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.619 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.619 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.619 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-11T22:07:26.619 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.619 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.619 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.619 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Container image name, without the tag", 2024-09-11T22:07:26.619 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.619 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.620 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.620 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.620 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_elasticsearch": { 2024-09-11T22:07:26.620 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_elasticsearch", 2024-09-11T22:07:26.620 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.620 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.620 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.620 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-11T22:07:26.621 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.621 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.621 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.621 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "elasticsearch container image", 2024-09-11T22:07:26.621 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.621 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.621 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.621 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.622 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_grafana": { 2024-09-11T22:07:26.622 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_grafana", 2024-09-11T22:07:26.622 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.622 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.622 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.622 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-11T22:07:26.622 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.622 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.623 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.623 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-09-11T22:07:26.623 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.623 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.623 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.623 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.623 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_haproxy": { 2024-09-11T22:07:26.623 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_haproxy", 2024-09-11T22:07:26.624 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.624 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.624 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.624 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-11T22:07:26.624 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.624 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.624 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.625 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "HAproxy container image", 2024-09-11T22:07:26.625 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.625 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.625 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.625 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.625 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_jaeger_agent": { 2024-09-11T22:07:26.625 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_jaeger_agent", 2024-09-11T22:07:26.625 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.626 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.626 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.626 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-11T22:07:26.626 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.626 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.626 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.626 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Jaeger agent container image", 2024-09-11T22:07:26.627 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.627 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.627 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.627 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.627 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_jaeger_collector": { 2024-09-11T22:07:26.627 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_jaeger_collector", 2024-09-11T22:07:26.627 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.627 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.628 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.628 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-11T22:07:26.628 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.628 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.628 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.628 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Jaeger collector container image", 2024-09-11T22:07:26.628 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.629 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.629 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.629 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.629 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_jaeger_query": { 2024-09-11T22:07:26.629 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_jaeger_query", 2024-09-11T22:07:26.629 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.629 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.629 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.630 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-11T22:07:26.630 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.630 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.630 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.630 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Jaeger query container image", 2024-09-11T22:07:26.630 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.630 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.630 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.631 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.631 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_keepalived": { 2024-09-11T22:07:26.631 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_keepalived", 2024-09-11T22:07:26.631 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.631 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.631 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.631 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-11T22:07:26.631 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.632 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.632 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.632 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Keepalived container image", 2024-09-11T22:07:26.632 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.632 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.632 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.632 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.633 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_loki": { 2024-09-11T22:07:26.633 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_loki", 2024-09-11T22:07:26.633 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.633 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.633 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.633 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-11T22:07:26.633 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.633 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.634 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.634 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Loki container image", 2024-09-11T22:07:26.634 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.634 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.634 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.634 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.634 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_node_exporter": { 2024-09-11T22:07:26.634 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_node_exporter", 2024-09-11T22:07:26.635 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.635 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.635 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.635 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-11T22:07:26.635 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.635 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.635 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.635 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-09-11T22:07:26.636 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.636 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.636 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.636 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.636 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_nvmeof": { 2024-09-11T22:07:26.636 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_nvmeof", 2024-09-11T22:07:26.636 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.637 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.637 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.637 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-11T22:07:26.637 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.637 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.637 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.637 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Nvme-of container image", 2024-09-11T22:07:26.637 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.638 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.638 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.638 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.638 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_prometheus": { 2024-09-11T22:07:26.638 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_prometheus", 2024-09-11T22:07:26.638 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.638 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.639 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.639 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-11T22:07:26.639 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.639 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.639 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.639 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-09-11T22:07:26.639 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.640 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.640 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.640 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.640 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_promtail": { 2024-09-11T22:07:26.640 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_promtail", 2024-09-11T22:07:26.640 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.640 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.641 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.641 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-11T22:07:26.641 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.641 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.641 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.641 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Promtail container image", 2024-09-11T22:07:26.641 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.642 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.642 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.642 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.642 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_samba": { 2024-09-11T22:07:26.642 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_samba", 2024-09-11T22:07:26.642 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.642 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.643 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.643 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-11T22:07:26.643 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.643 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.643 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.643 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Samba/SMB container image", 2024-09-11T22:07:26.643 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.643 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.644 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.644 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.644 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_snmp_gateway": { 2024-09-11T22:07:26.644 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_snmp_gateway", 2024-09-11T22:07:26.644 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.644 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.645 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.645 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-11T22:07:26.645 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.645 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.645 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.645 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SNMP Gateway container image", 2024-09-11T22:07:26.645 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.646 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.646 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.646 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.646 INFO:teuthology.orchestra.run.smithi001.stdout: "container_init": { 2024-09-11T22:07:26.646 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_init", 2024-09-11T22:07:26.646 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.646 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.646 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.647 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:26.647 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.647 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.647 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.647 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Run podman/docker with `--init`", 2024-09-11T22:07:26.647 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.647 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.648 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.648 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.648 INFO:teuthology.orchestra.run.smithi001.stdout: "daemon_cache_timeout": { 2024-09-11T22:07:26.648 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "daemon_cache_timeout", 2024-09-11T22:07:26.648 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-11T22:07:26.648 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.648 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.649 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-09-11T22:07:26.649 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.649 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.649 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.649 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-11T22:07:26.649 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.649 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.649 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.650 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.650 INFO:teuthology.orchestra.run.smithi001.stdout: "default_cephadm_command_timeout": { 2024-09-11T22:07:26.650 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "default_cephadm_command_timeout", 2024-09-11T22:07:26.650 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:26.650 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.650 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.650 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "900", 2024-09-11T22:07:26.650 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.651 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.651 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.651 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-11T22:07:26.651 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.651 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.651 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.651 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.651 INFO:teuthology.orchestra.run.smithi001.stdout: "default_registry": { 2024-09-11T22:07:26.652 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "default_registry", 2024-09-11T22:07:26.652 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.652 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.652 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.652 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io", 2024-09-11T22:07:26.652 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.652 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.652 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.653 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-11T22:07:26.653 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.653 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.653 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.653 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.653 INFO:teuthology.orchestra.run.smithi001.stdout: "device_cache_timeout": { 2024-09-11T22:07:26.653 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "device_cache_timeout", 2024-09-11T22:07:26.653 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-11T22:07:26.654 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.654 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.654 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "1800", 2024-09-11T22:07:26.654 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.654 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.654 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.654 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "seconds to cache device inventory", 2024-09-11T22:07:26.654 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.655 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.655 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.655 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.655 INFO:teuthology.orchestra.run.smithi001.stdout: "device_enhanced_scan": { 2024-09-11T22:07:26.655 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "device_enhanced_scan", 2024-09-11T22:07:26.655 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.655 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.655 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.656 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:26.656 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.656 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.656 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.656 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-11T22:07:26.656 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.656 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.657 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.657 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.657 INFO:teuthology.orchestra.run.smithi001.stdout: "facts_cache_timeout": { 2024-09-11T22:07:26.657 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "facts_cache_timeout", 2024-09-11T22:07:26.657 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-11T22:07:26.657 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.657 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.657 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-09-11T22:07:26.658 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.658 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.658 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.658 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "seconds to cache host facts data", 2024-09-11T22:07:26.658 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.658 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.658 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.658 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.659 INFO:teuthology.orchestra.run.smithi001.stdout: "grafana_dashboards_path": { 2024-09-11T22:07:26.659 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "grafana_dashboards_path", 2024-09-11T22:07:26.659 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.659 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.659 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.659 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-11T22:07:26.659 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.660 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.660 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.660 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-11T22:07:26.660 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.660 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.660 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.660 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.661 INFO:teuthology.orchestra.run.smithi001.stdout: "host_check_interval": { 2024-09-11T22:07:26.661 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "host_check_interval", 2024-09-11T22:07:26.661 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-11T22:07:26.661 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.661 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.661 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-09-11T22:07:26.661 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.662 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.662 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.662 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to perform a host check", 2024-09-11T22:07:26.662 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.662 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.662 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.662 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.662 INFO:teuthology.orchestra.run.smithi001.stdout: "hw_monitoring": { 2024-09-11T22:07:26.663 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "hw_monitoring", 2024-09-11T22:07:26.663 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.663 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.663 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.663 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:26.663 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.663 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.664 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.664 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-11T22:07:26.664 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.664 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.664 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.664 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.664 INFO:teuthology.orchestra.run.smithi001.stdout: "inventory_list_all": { 2024-09-11T22:07:26.664 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "inventory_list_all", 2024-09-11T22:07:26.665 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.665 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.665 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.665 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:26.665 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.665 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.665 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.665 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-11T22:07:26.666 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.666 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.666 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.666 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.666 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:26.666 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:26.666 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.667 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.667 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.667 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:26.667 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.667 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.667 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:26.667 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:26.667 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:26.668 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:26.668 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:26.668 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:26.668 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:26.668 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:26.668 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.668 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.669 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.669 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.669 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.669 INFO:teuthology.orchestra.run.smithi001.stdout: "log_refresh_metadata": { 2024-09-11T22:07:26.669 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_refresh_metadata", 2024-09-11T22:07:26.669 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.669 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.670 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.670 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:26.670 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.670 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.670 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.670 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-11T22:07:26.670 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.671 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.671 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.671 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.671 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:26.671 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:26.671 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.671 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.671 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.672 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:26.672 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.672 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.672 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.672 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-11T22:07:26.672 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.672 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.672 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.673 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.673 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:26.673 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:26.673 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.673 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.673 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.673 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:26.674 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.674 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.674 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:26.674 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:26.674 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:26.674 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:26.674 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:26.674 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:26.674 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:26.675 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:26.675 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.675 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.675 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.675 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.675 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.675 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:26.675 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:26.676 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.676 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.676 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.676 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:26.676 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.676 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.676 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.676 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.676 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.677 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.677 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.677 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.677 INFO:teuthology.orchestra.run.smithi001.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-11T22:07:26.677 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-11T22:07:26.677 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.677 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.677 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.678 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:26.678 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.678 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.678 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.678 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-11T22:07:26.678 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.678 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.678 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.679 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.679 INFO:teuthology.orchestra.run.smithi001.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-11T22:07:26.679 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-11T22:07:26.679 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.679 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.679 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.679 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "*", 2024-09-11T22:07:26.679 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.680 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.680 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.680 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-11T22:07:26.680 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.680 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.680 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.680 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.680 INFO:teuthology.orchestra.run.smithi001.stdout: "max_count_per_host": { 2024-09-11T22:07:26.681 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_count_per_host", 2024-09-11T22:07:26.681 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:26.681 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.681 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.681 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-11T22:07:26.681 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.681 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.681 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.682 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "max number of daemons per service per host", 2024-09-11T22:07:26.682 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.682 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.682 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.682 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.682 INFO:teuthology.orchestra.run.smithi001.stdout: "max_osd_draining_count": { 2024-09-11T22:07:26.682 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_osd_draining_count", 2024-09-11T22:07:26.682 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:26.682 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.683 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.683 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-11T22:07:26.683 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.683 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.683 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.683 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-11T22:07:26.683 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.684 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.684 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.684 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.684 INFO:teuthology.orchestra.run.smithi001.stdout: "migration_current": { 2024-09-11T22:07:26.684 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "migration_current", 2024-09-11T22:07:26.684 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:26.684 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.684 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.685 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:26.685 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.685 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.685 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.685 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "internal - do not modify", 2024-09-11T22:07:26.685 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.685 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.685 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.686 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.686 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": { 2024-09-11T22:07:26.686 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mode", 2024-09-11T22:07:26.686 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.686 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.686 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.686 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "root", 2024-09-11T22:07:26.686 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.686 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.687 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:26.687 INFO:teuthology.orchestra.run.smithi001.stdout: "cephadm-package", 2024-09-11T22:07:26.687 INFO:teuthology.orchestra.run.smithi001.stdout: "root" 2024-09-11T22:07:26.687 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:26.687 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "mode for remote execution of cephadm", 2024-09-11T22:07:26.687 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.687 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.687 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.688 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.688 INFO:teuthology.orchestra.run.smithi001.stdout: "oob_default_addr": { 2024-09-11T22:07:26.688 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "oob_default_addr", 2024-09-11T22:07:26.688 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.688 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.688 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.688 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "169.254.1.1", 2024-09-11T22:07:26.688 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.689 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.689 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.689 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-11T22:07:26.689 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.689 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.689 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.689 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.689 INFO:teuthology.orchestra.run.smithi001.stdout: "prometheus_alerts_path": { 2024-09-11T22:07:26.690 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "prometheus_alerts_path", 2024-09-11T22:07:26.690 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.690 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.690 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.690 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-11T22:07:26.690 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.690 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.690 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.691 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-11T22:07:26.691 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.691 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.691 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.691 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.691 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_insecure": { 2024-09-11T22:07:26.692 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_insecure", 2024-09-11T22:07:26.692 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.692 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.692 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.692 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:26.692 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.692 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.692 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.693 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-11T22:07:26.693 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.693 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.693 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.693 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.693 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_password": { 2024-09-11T22:07:26.693 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_password", 2024-09-11T22:07:26.693 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.694 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.694 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.694 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:26.694 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.694 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.694 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.694 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-11T22:07:26.695 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.695 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.695 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.695 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.695 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_url": { 2024-09-11T22:07:26.695 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_url", 2024-09-11T22:07:26.695 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.695 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.696 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.696 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:26.696 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.696 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.696 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.696 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-11T22:07:26.696 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.696 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.697 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.697 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.697 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_username": { 2024-09-11T22:07:26.697 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_username", 2024-09-11T22:07:26.697 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.697 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.697 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.698 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:26.698 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.698 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.698 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.698 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-11T22:07:26.698 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.698 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.698 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.699 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.699 INFO:teuthology.orchestra.run.smithi001.stdout: "secure_monitoring_stack": { 2024-09-11T22:07:26.699 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "secure_monitoring_stack", 2024-09-11T22:07:26.699 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.699 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.699 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.699 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:26.699 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.700 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.700 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.700 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-11T22:07:26.700 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.700 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.700 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.700 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.701 INFO:teuthology.orchestra.run.smithi001.stdout: "service_discovery_port": { 2024-09-11T22:07:26.701 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "service_discovery_port", 2024-09-11T22:07:26.701 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:26.701 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.701 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.701 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8765", 2024-09-11T22:07:26.702 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.702 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.702 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.702 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "cephadm service discovery port", 2024-09-11T22:07:26.702 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.702 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.702 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.702 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.702 INFO:teuthology.orchestra.run.smithi001.stdout: "ssh_config_file": { 2024-09-11T22:07:26.703 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssh_config_file", 2024-09-11T22:07:26.703 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.703 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.703 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.703 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:26.703 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.703 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.703 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.703 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-11T22:07:26.704 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.704 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.704 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.704 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.704 INFO:teuthology.orchestra.run.smithi001.stdout: "use_agent": { 2024-09-11T22:07:26.704 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "use_agent", 2024-09-11T22:07:26.704 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.704 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.705 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.705 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:26.705 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.705 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.705 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.705 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-11T22:07:26.705 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.705 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.705 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.706 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.706 INFO:teuthology.orchestra.run.smithi001.stdout: "use_repo_digest": { 2024-09-11T22:07:26.706 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "use_repo_digest", 2024-09-11T22:07:26.706 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.706 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.706 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.706 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:26.706 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.707 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.707 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.707 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-11T22:07:26.707 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.707 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.707 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.707 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.707 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_on_failed_host_check": { 2024-09-11T22:07:26.707 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_on_failed_host_check", 2024-09-11T22:07:26.708 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.708 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.708 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.708 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:26.708 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.708 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.708 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.708 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise a health warning if the host check fails", 2024-09-11T22:07:26.709 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.709 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.709 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.709 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.709 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_on_stray_daemons": { 2024-09-11T22:07:26.709 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_on_stray_daemons", 2024-09-11T22:07:26.709 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.709 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.710 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.710 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:26.710 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.710 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.710 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.710 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-11T22:07:26.710 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.710 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.710 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.711 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.711 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_on_stray_hosts": { 2024-09-11T22:07:26.711 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_on_stray_hosts", 2024-09-11T22:07:26.711 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.711 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.711 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.711 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:26.711 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.712 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.712 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.712 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-11T22:07:26.712 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.712 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.712 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.712 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:26.713 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:26.713 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.713 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:26.713 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crash", 2024-09-11T22:07:26.713 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:26.713 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:26.713 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:26.714 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:26.714 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:26.714 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.714 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.714 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.714 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:26.714 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.714 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.715 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:26.715 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:26.715 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:26.715 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:26.715 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:26.715 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:26.715 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:26.715 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:26.716 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.716 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.716 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.716 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.716 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.716 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:26.716 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:26.717 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.717 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.717 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.717 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:26.717 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.717 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.717 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.717 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.718 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.718 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.718 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.718 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.718 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:26.718 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:26.718 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.718 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.719 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.719 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:26.719 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.719 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.719 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:26.719 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:26.719 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:26.719 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:26.720 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:26.720 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:26.720 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:26.720 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:26.720 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.720 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.720 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.721 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.721 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.721 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:26.721 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:26.721 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.721 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.721 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.721 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:26.722 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.722 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.722 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.722 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.722 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.722 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.722 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.722 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.723 INFO:teuthology.orchestra.run.smithi001.stdout: "retain_interval": { 2024-09-11T22:07:26.723 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "retain_interval", 2024-09-11T22:07:26.723 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-11T22:07:26.723 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.723 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.723 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "31536000", 2024-09-11T22:07:26.723 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.723 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.724 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.724 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-11T22:07:26.724 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.724 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.724 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.724 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.724 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_recent_interval": { 2024-09-11T22:07:26.725 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_recent_interval", 2024-09-11T22:07:26.725 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-11T22:07:26.725 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.725 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.725 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "1209600", 2024-09-11T22:07:26.725 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.725 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.726 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.726 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-11T22:07:26.726 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.726 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.726 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.726 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:26.726 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:26.726 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.726 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:26.727 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "dashboard", 2024-09-11T22:07:26.727 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:26.727 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:26.727 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:26.727 INFO:teuthology.orchestra.run.smithi001.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-11T22:07:26.727 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-11T22:07:26.727 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:26.727 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.728 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.728 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-11T22:07:26.728 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.728 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.728 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.728 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.728 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.728 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.729 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.729 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.729 INFO:teuthology.orchestra.run.smithi001.stdout: "ALERTMANAGER_API_HOST": { 2024-09-11T22:07:26.729 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-11T22:07:26.729 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.729 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.729 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.729 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:26.729 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.730 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.730 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.730 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.730 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.730 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.730 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.730 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.730 INFO:teuthology.orchestra.run.smithi001.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-11T22:07:26.730 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-11T22:07:26.731 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.731 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.731 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.731 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:26.731 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.731 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.731 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.731 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.732 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.732 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.732 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.732 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.732 INFO:teuthology.orchestra.run.smithi001.stdout: "AUDIT_API_ENABLED": { 2024-09-11T22:07:26.732 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "AUDIT_API_ENABLED", 2024-09-11T22:07:26.732 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.732 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.733 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.733 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:26.733 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.733 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.733 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.733 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.733 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.733 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.734 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.734 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.734 INFO:teuthology.orchestra.run.smithi001.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-11T22:07:26.734 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-11T22:07:26.734 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.734 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.734 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.734 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:26.735 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.735 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.735 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.735 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.735 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.735 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.735 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.735 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.736 INFO:teuthology.orchestra.run.smithi001.stdout: "ENABLE_BROWSABLE_API": { 2024-09-11T22:07:26.736 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-11T22:07:26.736 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.736 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.736 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.736 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:26.736 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.736 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.737 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.737 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.737 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.737 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.737 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.737 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.737 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-11T22:07:26.737 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-11T22:07:26.738 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.738 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.738 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.738 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:26.738 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.738 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.738 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.738 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.739 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.739 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.739 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.739 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.739 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-11T22:07:26.739 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-11T22:07:26.739 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.739 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.740 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.740 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:26.740 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.740 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.740 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.740 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.740 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.740 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.741 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.741 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.741 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-11T22:07:26.741 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-11T22:07:26.741 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.741 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.741 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.741 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:26.741 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.742 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.742 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.742 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.742 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.742 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.742 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.742 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.742 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-11T22:07:26.743 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-11T22:07:26.743 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.743 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.743 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.743 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:26.743 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.743 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.743 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.744 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.744 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.744 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.744 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.744 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.744 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-11T22:07:26.744 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-11T22:07:26.744 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.745 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.745 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.745 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:26.745 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.745 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.745 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.745 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.745 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.746 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.746 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.746 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.746 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-11T22:07:26.746 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-11T22:07:26.746 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.746 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.746 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.746 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:26.747 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.747 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.747 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.747 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.747 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.747 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.747 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.747 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.748 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-11T22:07:26.748 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-11T22:07:26.748 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.748 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.748 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.748 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:26.748 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.749 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.749 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.749 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.749 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.749 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.749 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.749 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.749 INFO:teuthology.orchestra.run.smithi001.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-11T22:07:26.750 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-11T22:07:26.750 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.750 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.750 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.750 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:26.750 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.750 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.750 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.751 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.751 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.751 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.751 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.751 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.751 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_PASSWORD": { 2024-09-11T22:07:26.751 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-11T22:07:26.751 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.752 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.752 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.752 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2024-09-11T22:07:26.752 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.752 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.752 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.752 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.753 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.753 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.753 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.753 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.753 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-11T22:07:26.753 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-11T22:07:26.753 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.753 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.754 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.754 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:26.754 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.754 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.754 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.754 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.754 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.754 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.755 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.755 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.755 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_URL": { 2024-09-11T22:07:26.755 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_URL", 2024-09-11T22:07:26.755 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.755 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.755 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.755 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:26.755 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.756 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.756 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.756 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.756 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.756 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.756 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.756 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.756 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_USERNAME": { 2024-09-11T22:07:26.757 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-11T22:07:26.757 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.757 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.757 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.757 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2024-09-11T22:07:26.757 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.757 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.757 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.758 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.758 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.758 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.758 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.758 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.758 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-11T22:07:26.758 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-11T22:07:26.758 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.758 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.759 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.759 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:26.759 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.759 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.759 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.759 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.759 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.760 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.760 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.760 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.760 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-11T22:07:26.760 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-11T22:07:26.760 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.760 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.761 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.761 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:26.761 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.761 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.761 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.761 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.761 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.761 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.761 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.762 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.762 INFO:teuthology.orchestra.run.smithi001.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-11T22:07:26.762 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-11T22:07:26.762 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.762 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.762 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.762 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:26.762 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.763 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.763 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.763 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.763 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.763 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.763 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.763 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.764 INFO:teuthology.orchestra.run.smithi001.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-11T22:07:26.764 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-11T22:07:26.764 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.764 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.764 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.764 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:26.764 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.765 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.765 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.765 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.765 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.765 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.765 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.765 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.766 INFO:teuthology.orchestra.run.smithi001.stdout: "PROMETHEUS_API_HOST": { 2024-09-11T22:07:26.766 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-11T22:07:26.766 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.766 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.766 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.766 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:26.766 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.766 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.766 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.767 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.767 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.767 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.767 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.767 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.767 INFO:teuthology.orchestra.run.smithi001.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-11T22:07:26.767 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-11T22:07:26.768 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.768 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.768 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.768 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:26.768 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.768 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.768 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.768 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.769 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.769 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.769 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.769 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.769 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-11T22:07:26.769 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-11T22:07:26.769 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.770 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.770 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.770 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:26.770 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.770 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.770 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.770 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.770 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.771 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.771 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.771 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.771 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-11T22:07:26.771 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-11T22:07:26.771 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.771 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.771 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.772 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:26.772 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.772 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.772 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.772 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.772 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.772 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.772 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.773 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.773 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-11T22:07:26.773 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-11T22:07:26.773 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.773 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.773 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.773 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:26.773 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.774 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.774 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.774 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.774 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.774 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.774 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.774 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.774 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-11T22:07:26.775 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-11T22:07:26.775 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.775 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.775 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.775 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:26.775 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.775 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.775 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.776 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.776 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.776 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.776 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.776 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.776 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-11T22:07:26.776 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-11T22:07:26.776 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.777 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.777 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.777 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:26.777 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.777 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.777 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.777 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.778 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.778 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.778 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.778 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.778 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-11T22:07:26.778 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-11T22:07:26.778 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.778 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.779 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.779 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:26.779 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.779 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.779 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.779 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.779 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.779 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.780 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.780 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.780 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-11T22:07:26.780 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-11T22:07:26.780 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.780 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.780 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.781 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:26.781 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.781 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.781 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.781 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.781 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.781 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.782 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.782 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.782 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_ENABLED": { 2024-09-11T22:07:26.782 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-11T22:07:26.782 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.782 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.782 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.782 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:26.783 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.783 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.783 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.783 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.783 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.783 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.783 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.784 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.784 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-11T22:07:26.784 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-11T22:07:26.784 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.784 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.784 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.784 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-11T22:07:26.784 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.785 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.785 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.785 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.785 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.785 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.785 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.785 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.786 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-11T22:07:26.786 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-11T22:07:26.786 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:26.786 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.786 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.786 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-11T22:07:26.786 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.786 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.787 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.787 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.787 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.787 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.787 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.787 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.787 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-11T22:07:26.787 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-11T22:07:26.788 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:26.788 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.788 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.788 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8", 2024-09-11T22:07:26.788 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.788 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.788 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.788 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.789 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.789 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.789 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.789 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.789 INFO:teuthology.orchestra.run.smithi001.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-11T22:07:26.789 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-11T22:07:26.789 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:26.789 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.790 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.790 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "45", 2024-09-11T22:07:26.790 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.790 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.790 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.790 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.790 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.790 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.791 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.791 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.791 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_ACCESS_KEY": { 2024-09-11T22:07:26.791 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-11T22:07:26.791 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.791 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.791 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.791 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:26.791 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.792 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.792 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.792 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.792 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.792 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.792 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.792 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.792 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-11T22:07:26.793 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-11T22:07:26.793 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.793 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.793 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.793 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2024-09-11T22:07:26.793 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.793 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.794 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.794 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.794 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.794 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.794 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.794 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.794 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_SECRET_KEY": { 2024-09-11T22:07:26.794 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-11T22:07:26.795 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.795 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.795 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.795 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:26.795 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.795 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.795 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.796 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.796 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.796 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.796 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.796 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.796 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_SSL_VERIFY": { 2024-09-11T22:07:26.796 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-11T22:07:26.797 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.797 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.797 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.797 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:26.797 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.797 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.797 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.798 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.798 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.798 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.798 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.798 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.798 INFO:teuthology.orchestra.run.smithi001.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-11T22:07:26.798 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-11T22:07:26.799 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:26.799 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.799 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.799 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-09-11T22:07:26.799 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.799 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.799 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.799 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.800 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.800 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.800 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.800 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.800 INFO:teuthology.orchestra.run.smithi001.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-11T22:07:26.800 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-11T22:07:26.800 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:26.801 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.801 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.801 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-11T22:07:26.801 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.801 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.801 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.801 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.802 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.802 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.802 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.802 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.802 INFO:teuthology.orchestra.run.smithi001.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-11T22:07:26.802 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-11T22:07:26.802 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:26.802 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.803 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.803 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-09-11T22:07:26.803 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.803 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.803 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.803 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.803 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.803 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.804 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.804 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.804 INFO:teuthology.orchestra.run.smithi001.stdout: "cross_origin_url": { 2024-09-11T22:07:26.804 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cross_origin_url", 2024-09-11T22:07:26.804 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.804 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.804 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.804 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:26.805 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.805 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.805 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.805 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.805 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.805 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.805 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.806 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.806 INFO:teuthology.orchestra.run.smithi001.stdout: "crt_file": { 2024-09-11T22:07:26.806 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crt_file", 2024-09-11T22:07:26.806 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.806 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.806 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.806 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:26.806 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.807 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.807 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.807 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.807 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.807 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.807 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.807 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.808 INFO:teuthology.orchestra.run.smithi001.stdout: "debug": { 2024-09-11T22:07:26.808 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "debug", 2024-09-11T22:07:26.808 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.808 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.808 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.808 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:26.808 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.809 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.809 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.809 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Enable/disable debug options", 2024-09-11T22:07:26.809 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.809 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.809 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.809 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.810 INFO:teuthology.orchestra.run.smithi001.stdout: "jwt_token_ttl": { 2024-09-11T22:07:26.810 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "jwt_token_ttl", 2024-09-11T22:07:26.810 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:26.810 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.810 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.810 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "28800", 2024-09-11T22:07:26.810 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.810 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.811 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.811 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.811 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.811 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.811 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.811 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.811 INFO:teuthology.orchestra.run.smithi001.stdout: "key_file": { 2024-09-11T22:07:26.812 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "key_file", 2024-09-11T22:07:26.812 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.812 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.812 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.812 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:26.812 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.812 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.813 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.813 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.813 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.813 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.813 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.813 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.813 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:26.813 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:26.814 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.814 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.814 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.814 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:26.814 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.814 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.814 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:26.814 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:26.815 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:26.815 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:26.815 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:26.815 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:26.815 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:26.815 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:26.815 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.815 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.816 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.816 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.816 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.816 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:26.816 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:26.816 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.816 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.817 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.817 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:26.817 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.817 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.817 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.817 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.817 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.817 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.818 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.818 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.818 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:26.818 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:26.818 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.818 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.818 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.818 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:26.818 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.819 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.819 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:26.819 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:26.819 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:26.819 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:26.819 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:26.819 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:26.819 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:26.820 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:26.820 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.820 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.820 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.820 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.820 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.820 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:26.820 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:26.821 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.821 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.821 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.821 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:26.821 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.821 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.821 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.822 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.822 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.822 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.822 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.822 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.822 INFO:teuthology.orchestra.run.smithi001.stdout: "motd": { 2024-09-11T22:07:26.822 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "motd", 2024-09-11T22:07:26.822 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.822 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.823 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.823 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:26.823 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.823 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.823 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.823 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "The message of the day", 2024-09-11T22:07:26.823 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.823 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.824 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.824 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.824 INFO:teuthology.orchestra.run.smithi001.stdout: "redirect_resolve_ip_addr": { 2024-09-11T22:07:26.824 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "redirect_resolve_ip_addr", 2024-09-11T22:07:26.824 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.824 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.824 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.825 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:26.825 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.825 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.825 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.825 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.825 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.825 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.825 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.826 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.826 INFO:teuthology.orchestra.run.smithi001.stdout: "server_addr": { 2024-09-11T22:07:26.826 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_addr", 2024-09-11T22:07:26.826 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.826 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.826 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.826 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "::", 2024-09-11T22:07:26.826 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.827 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.827 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.827 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.827 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.827 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.827 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.827 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.827 INFO:teuthology.orchestra.run.smithi001.stdout: "server_port": { 2024-09-11T22:07:26.828 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_port", 2024-09-11T22:07:26.828 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:26.828 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.828 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.828 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8080", 2024-09-11T22:07:26.828 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.828 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.828 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.829 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.829 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.829 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.829 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.829 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.829 INFO:teuthology.orchestra.run.smithi001.stdout: "ssl": { 2024-09-11T22:07:26.829 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssl", 2024-09-11T22:07:26.829 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.829 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.830 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.830 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:26.830 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.830 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.830 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.830 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.830 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.830 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.831 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.831 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.831 INFO:teuthology.orchestra.run.smithi001.stdout: "ssl_server_port": { 2024-09-11T22:07:26.831 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssl_server_port", 2024-09-11T22:07:26.831 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:26.831 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.831 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.831 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8443", 2024-09-11T22:07:26.832 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.832 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.832 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.832 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.832 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.832 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.832 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.832 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.833 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_behaviour": { 2024-09-11T22:07:26.833 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_behaviour", 2024-09-11T22:07:26.833 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.833 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.833 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.833 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "redirect", 2024-09-11T22:07:26.833 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.833 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.834 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:26.834 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:26.834 INFO:teuthology.orchestra.run.smithi001.stdout: "redirect" 2024-09-11T22:07:26.834 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:26.834 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.834 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.834 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.834 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.835 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.835 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_error_status_code": { 2024-09-11T22:07:26.835 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_error_status_code", 2024-09-11T22:07:26.835 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:26.835 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.835 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.835 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "500", 2024-09-11T22:07:26.835 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "400", 2024-09-11T22:07:26.836 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "599", 2024-09-11T22:07:26.836 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.836 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.836 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.836 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.836 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.836 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.836 INFO:teuthology.orchestra.run.smithi001.stdout: "url_prefix": { 2024-09-11T22:07:26.837 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "url_prefix", 2024-09-11T22:07:26.837 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.837 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.837 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.837 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:26.837 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.837 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.837 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.838 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.838 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.838 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.838 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.838 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:26.838 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:26.838 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.838 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:26.839 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "devicehealth", 2024-09-11T22:07:26.839 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:26.839 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:26.839 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:26.839 INFO:teuthology.orchestra.run.smithi001.stdout: "enable_monitoring": { 2024-09-11T22:07:26.839 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enable_monitoring", 2024-09-11T22:07:26.839 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.840 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.840 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.840 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:26.840 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.840 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.840 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.840 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "monitor device health metrics", 2024-09-11T22:07:26.840 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.841 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.841 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.841 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.841 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:26.841 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:26.841 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.841 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.841 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.842 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:26.842 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.842 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.842 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:26.842 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:26.842 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:26.842 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:26.842 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:26.843 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:26.843 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:26.843 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:26.843 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.843 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.843 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.843 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.844 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.844 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:26.844 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:26.844 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.844 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.844 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.844 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:26.844 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.845 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.845 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.845 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.845 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.845 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.845 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.845 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.845 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:26.846 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:26.846 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.846 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.846 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.846 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:26.846 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.846 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.846 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:26.846 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:26.847 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:26.847 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:26.847 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:26.847 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:26.847 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:26.847 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:26.847 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.847 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.848 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.848 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.848 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.848 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:26.848 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:26.848 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.848 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.848 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.849 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:26.849 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.849 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.849 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.849 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.849 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.849 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.849 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.850 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.850 INFO:teuthology.orchestra.run.smithi001.stdout: "mark_out_threshold": { 2024-09-11T22:07:26.850 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mark_out_threshold", 2024-09-11T22:07:26.850 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-11T22:07:26.850 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.850 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.850 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "2419200", 2024-09-11T22:07:26.851 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.851 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.851 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.851 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-11T22:07:26.851 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.851 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.851 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.851 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.852 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_name": { 2024-09-11T22:07:26.852 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pool_name", 2024-09-11T22:07:26.852 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.852 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.852 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.852 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "device_health_metrics", 2024-09-11T22:07:26.852 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.852 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.853 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.853 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-11T22:07:26.853 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.853 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.853 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.853 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.853 INFO:teuthology.orchestra.run.smithi001.stdout: "retention_period": { 2024-09-11T22:07:26.853 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "retention_period", 2024-09-11T22:07:26.854 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-11T22:07:26.854 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.854 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.854 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "15552000", 2024-09-11T22:07:26.854 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.854 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.854 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.854 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how long to retain device health metrics", 2024-09-11T22:07:26.855 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.855 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.855 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.855 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.855 INFO:teuthology.orchestra.run.smithi001.stdout: "scrape_frequency": { 2024-09-11T22:07:26.855 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "scrape_frequency", 2024-09-11T22:07:26.855 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-11T22:07:26.855 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.855 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.856 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "86400", 2024-09-11T22:07:26.856 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.856 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.856 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.856 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-11T22:07:26.856 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.856 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.857 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.857 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.857 INFO:teuthology.orchestra.run.smithi001.stdout: "self_heal": { 2024-09-11T22:07:26.857 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "self_heal", 2024-09-11T22:07:26.857 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.857 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.857 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.857 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:26.858 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.858 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.858 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.858 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-11T22:07:26.858 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.858 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.858 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.858 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.859 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-09-11T22:07:26.859 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-09-11T22:07:26.859 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-11T22:07:26.859 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.859 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.859 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-09-11T22:07:26.859 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.860 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.860 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.860 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to wake up and check device health", 2024-09-11T22:07:26.860 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.860 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.860 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.860 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.860 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_threshold": { 2024-09-11T22:07:26.861 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_threshold", 2024-09-11T22:07:26.861 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-11T22:07:26.861 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.861 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.861 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "7257600", 2024-09-11T22:07:26.861 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.861 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.861 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.862 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-11T22:07:26.862 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.862 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.862 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.862 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:26.862 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:26.862 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.862 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:26.863 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "diskprediction_local", 2024-09-11T22:07:26.863 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:26.863 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:26.863 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:26.863 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:26.863 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:26.863 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.863 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.864 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.864 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:26.864 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.864 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.864 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:26.864 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:26.864 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:26.864 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:26.865 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:26.865 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:26.865 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:26.865 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:26.865 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.865 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.865 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.865 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.865 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.866 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:26.866 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:26.866 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.866 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.866 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.866 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:26.866 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.866 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.867 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.867 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.867 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.867 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.867 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.867 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.867 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:26.867 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:26.868 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.868 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.868 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.868 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:26.868 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.868 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.868 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:26.869 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:26.869 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:26.869 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:26.869 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:26.869 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:26.869 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:26.869 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:26.870 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.870 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.870 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.870 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.870 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:26.870 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:26.870 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.870 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.871 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.871 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:26.871 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.871 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.871 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.871 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.871 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.871 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.872 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.872 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.872 INFO:teuthology.orchestra.run.smithi001.stdout: "predict_interval": { 2024-09-11T22:07:26.872 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "predict_interval", 2024-09-11T22:07:26.872 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.872 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.872 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.872 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "86400", 2024-09-11T22:07:26.873 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.873 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.873 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.873 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.873 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.873 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.873 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.873 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.873 INFO:teuthology.orchestra.run.smithi001.stdout: "predictor_model": { 2024-09-11T22:07:26.874 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "predictor_model", 2024-09-11T22:07:26.874 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.874 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.874 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.874 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "prophetstor", 2024-09-11T22:07:26.874 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.874 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.874 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.874 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.875 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.875 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.875 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.875 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.875 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-09-11T22:07:26.875 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-09-11T22:07:26.875 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.875 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.875 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.876 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-09-11T22:07:26.876 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.876 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.876 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.876 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.876 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.876 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.876 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.877 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:26.877 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:26.877 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.877 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:26.877 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "influx", 2024-09-11T22:07:26.877 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": false, 2024-09-11T22:07:26.877 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "influxdb python module not found", 2024-09-11T22:07:26.877 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:26.877 INFO:teuthology.orchestra.run.smithi001.stdout: "batch_size": { 2024-09-11T22:07:26.878 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "batch_size", 2024-09-11T22:07:26.878 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:26.878 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.878 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.878 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5000", 2024-09-11T22:07:26.878 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.878 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.878 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.879 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-11T22:07:26.879 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.879 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.879 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.879 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.879 INFO:teuthology.orchestra.run.smithi001.stdout: "database": { 2024-09-11T22:07:26.879 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "database", 2024-09-11T22:07:26.879 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.880 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.880 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.880 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "ceph", 2024-09-11T22:07:26.880 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.880 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.880 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.880 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-11T22:07:26.880 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.881 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.881 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.881 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.881 INFO:teuthology.orchestra.run.smithi001.stdout: "hostname": { 2024-09-11T22:07:26.881 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "hostname", 2024-09-11T22:07:26.881 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.881 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.882 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.882 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:26.882 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.882 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.882 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.882 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "InfluxDB server hostname", 2024-09-11T22:07:26.882 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.882 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.883 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.883 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.883 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-09-11T22:07:26.883 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-09-11T22:07:26.883 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-11T22:07:26.883 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.883 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.883 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "30", 2024-09-11T22:07:26.884 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "5", 2024-09-11T22:07:26.884 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.884 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.884 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-11T22:07:26.884 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.884 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.884 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.884 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.885 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:26.885 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:26.885 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.885 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.885 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.885 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:26.885 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.885 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.886 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:26.886 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:26.886 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:26.886 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:26.886 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:26.886 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:26.886 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:26.886 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:26.887 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.887 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.887 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.887 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.887 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.887 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:26.887 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:26.887 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.887 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.888 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.888 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:26.888 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.888 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.888 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.888 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.888 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.889 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.889 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.889 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.889 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:26.889 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:26.889 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.889 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.890 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.890 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:26.890 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.890 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.890 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:26.890 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:26.890 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:26.891 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:26.891 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:26.891 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:26.891 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:26.891 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:26.891 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.891 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.891 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.892 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.892 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.892 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:26.892 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:26.892 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.892 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.892 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.892 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:26.893 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.893 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.893 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.893 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.893 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.893 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.893 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.894 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.894 INFO:teuthology.orchestra.run.smithi001.stdout: "password": { 2024-09-11T22:07:26.894 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "password", 2024-09-11T22:07:26.894 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.894 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.894 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.894 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:26.894 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.895 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.895 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.895 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "password of InfluxDB server user", 2024-09-11T22:07:26.895 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.895 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.895 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.895 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.895 INFO:teuthology.orchestra.run.smithi001.stdout: "port": { 2024-09-11T22:07:26.896 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "port", 2024-09-11T22:07:26.896 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:26.896 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.896 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.896 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8086", 2024-09-11T22:07:26.896 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.896 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.896 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.897 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "InfluxDB server port", 2024-09-11T22:07:26.897 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.897 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.897 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.897 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.897 INFO:teuthology.orchestra.run.smithi001.stdout: "ssl": { 2024-09-11T22:07:26.897 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssl", 2024-09-11T22:07:26.898 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.898 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.898 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.898 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "false", 2024-09-11T22:07:26.898 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.898 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.898 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.898 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-11T22:07:26.899 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.899 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.899 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.899 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.899 INFO:teuthology.orchestra.run.smithi001.stdout: "threads": { 2024-09-11T22:07:26.899 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "threads", 2024-09-11T22:07:26.899 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:26.899 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.900 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.900 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-09-11T22:07:26.900 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-09-11T22:07:26.900 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "32", 2024-09-11T22:07:26.900 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.900 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-11T22:07:26.900 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.900 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.901 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.901 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.901 INFO:teuthology.orchestra.run.smithi001.stdout: "username": { 2024-09-11T22:07:26.901 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "username", 2024-09-11T22:07:26.901 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.901 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.901 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.901 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:26.902 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.902 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.902 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.902 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "username of InfluxDB server user", 2024-09-11T22:07:26.902 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.902 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.902 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.902 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.903 INFO:teuthology.orchestra.run.smithi001.stdout: "verify_ssl": { 2024-09-11T22:07:26.903 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "verify_ssl", 2024-09-11T22:07:26.903 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.903 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.903 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.903 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "true", 2024-09-11T22:07:26.903 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.904 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.904 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.904 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-11T22:07:26.904 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.904 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.904 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.904 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:26.904 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:26.905 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.905 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:26.905 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "insights", 2024-09-11T22:07:26.905 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:26.905 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:26.905 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:26.905 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:26.905 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:26.906 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.906 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.906 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.906 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:26.906 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.906 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.906 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:26.906 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:26.907 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:26.907 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:26.907 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:26.907 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:26.907 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:26.907 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:26.907 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.907 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.908 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.908 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.908 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.908 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:26.908 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:26.908 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.908 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.908 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.909 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:26.909 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.909 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.909 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.909 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.909 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.909 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.910 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.910 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.910 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:26.910 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:26.910 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.910 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.910 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.910 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:26.910 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.911 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.911 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:26.911 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:26.911 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:26.911 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:26.911 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:26.912 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:26.912 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:26.912 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:26.912 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.912 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.912 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.912 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.912 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.913 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:26.913 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:26.913 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.913 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.913 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.913 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:26.913 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.913 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.914 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.914 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.914 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.914 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.914 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.914 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:26.914 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:26.914 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.915 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:26.915 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "iostat", 2024-09-11T22:07:26.915 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:26.915 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:26.915 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:26.915 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:26.915 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:26.915 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.915 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.916 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.916 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:26.916 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.916 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.916 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:26.916 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:26.916 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:26.916 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:26.917 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:26.917 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:26.917 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:26.917 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:26.917 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.917 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.917 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.917 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.918 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.918 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:26.918 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:26.918 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.918 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.918 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.918 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:26.920 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.920 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.920 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.920 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.920 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.920 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.921 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.921 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.921 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:26.921 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:26.921 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.921 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.921 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.921 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:26.922 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.922 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.922 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:26.922 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:26.922 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:26.922 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:26.922 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:26.922 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:26.923 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:26.923 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:26.923 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.923 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.923 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.923 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.923 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.923 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:26.924 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:26.924 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.924 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.924 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.924 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:26.924 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.924 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.925 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.925 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.925 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.925 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.925 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.925 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:26.925 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:26.925 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.926 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:26.926 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "k8sevents", 2024-09-11T22:07:26.926 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:26.926 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:26.926 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:26.926 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_event_retention_days": { 2024-09-11T22:07:26.926 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ceph_event_retention_days", 2024-09-11T22:07:26.926 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:26.927 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.927 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.927 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "7", 2024-09-11T22:07:26.927 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.927 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.927 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.927 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-11T22:07:26.927 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.928 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.928 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.928 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.928 INFO:teuthology.orchestra.run.smithi001.stdout: "config_check_secs": { 2024-09-11T22:07:26.928 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "config_check_secs", 2024-09-11T22:07:26.928 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:26.928 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.928 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.929 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-11T22:07:26.929 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "10", 2024-09-11T22:07:26.929 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.929 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.929 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-11T22:07:26.929 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.930 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.930 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.930 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.930 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:26.930 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:26.930 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.930 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.930 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.930 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:26.931 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.931 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.931 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:26.931 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:26.931 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:26.931 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:26.931 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:26.931 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:26.932 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:26.932 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:26.932 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.932 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.932 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.932 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.932 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.932 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:26.933 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:26.933 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.933 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.933 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.933 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:26.933 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.933 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.933 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.934 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.934 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.934 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.934 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.934 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.934 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:26.934 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:26.934 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.935 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.935 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.935 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:26.935 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.935 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.935 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:26.935 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:26.935 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:26.936 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:26.936 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:26.936 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:26.936 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:26.936 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:26.936 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.936 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.936 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.937 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.937 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.937 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:26.937 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:26.937 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.937 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.937 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.937 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:26.938 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.938 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.938 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.938 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.938 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.938 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.938 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.938 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:26.939 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:26.939 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.939 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:26.939 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "localpool", 2024-09-11T22:07:26.939 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:26.939 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:26.939 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:26.939 INFO:teuthology.orchestra.run.smithi001.stdout: "failure_domain": { 2024-09-11T22:07:26.940 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "failure_domain", 2024-09-11T22:07:26.940 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.940 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.940 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.940 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "host", 2024-09-11T22:07:26.940 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.940 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.940 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.941 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "failure domain for any created local pool", 2024-09-11T22:07:26.941 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-11T22:07:26.941 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.941 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.941 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.941 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:26.941 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:26.941 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.942 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.942 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.942 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:26.942 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.942 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.942 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:26.942 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:26.942 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:26.943 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:26.943 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:26.943 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:26.943 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:26.943 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:26.943 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.943 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.943 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.944 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.944 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.944 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:26.944 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:26.944 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.944 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.944 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.944 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:26.945 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.945 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.945 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.945 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.945 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.945 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.945 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.945 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.946 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:26.946 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:26.946 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.946 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.946 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.946 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:26.946 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.946 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.947 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:26.947 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:26.947 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:26.947 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:26.947 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:26.947 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:26.947 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:26.947 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:26.948 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.948 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.948 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.948 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.948 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.948 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:26.948 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:26.949 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.949 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.949 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.949 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:26.949 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.949 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.949 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.949 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.950 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.950 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.950 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.950 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.950 INFO:teuthology.orchestra.run.smithi001.stdout: "min_size": { 2024-09-11T22:07:26.950 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "min_size", 2024-09-11T22:07:26.950 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:26.950 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.951 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.951 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:26.951 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.951 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.951 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.951 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "default min_size for any created local pool", 2024-09-11T22:07:26.951 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-11T22:07:26.951 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.952 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.952 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.952 INFO:teuthology.orchestra.run.smithi001.stdout: "num_rep": { 2024-09-11T22:07:26.952 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "num_rep", 2024-09-11T22:07:26.952 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:26.952 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.952 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.952 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "3", 2024-09-11T22:07:26.952 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.953 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.953 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.953 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "default replica count for any created local pool", 2024-09-11T22:07:26.953 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.953 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.953 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.953 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.954 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num": { 2024-09-11T22:07:26.954 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pg_num", 2024-09-11T22:07:26.954 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:26.954 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.954 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.954 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "128", 2024-09-11T22:07:26.954 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.954 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.955 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.955 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "default pg_num for any created local pool", 2024-09-11T22:07:26.955 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.955 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.955 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.955 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.955 INFO:teuthology.orchestra.run.smithi001.stdout: "prefix": { 2024-09-11T22:07:26.955 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "prefix", 2024-09-11T22:07:26.956 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.956 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.956 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.956 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:26.956 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.956 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.956 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.956 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "name prefix for any created local pool", 2024-09-11T22:07:26.957 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.957 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.957 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.957 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.957 INFO:teuthology.orchestra.run.smithi001.stdout: "subtree": { 2024-09-11T22:07:26.957 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "subtree", 2024-09-11T22:07:26.957 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.957 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.958 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.958 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "rack", 2024-09-11T22:07:26.958 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.958 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.958 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.958 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-11T22:07:26.958 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-11T22:07:26.958 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.959 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.959 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:26.959 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:26.959 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.959 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:26.959 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mds_autoscaler", 2024-09-11T22:07:26.959 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:26.959 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:26.960 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:26.960 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:26.960 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:26.960 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.960 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.960 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.960 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:26.961 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.961 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.961 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:26.961 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:26.961 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:26.961 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:26.961 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:26.961 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:26.962 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:26.962 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:26.962 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.962 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.962 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.962 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.962 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.962 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:26.963 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:26.963 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.963 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.963 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.963 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:26.963 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.963 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.964 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.964 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.964 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.964 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.964 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.964 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.964 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:26.964 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:26.965 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.965 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.965 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.965 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:26.965 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.965 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.965 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:26.965 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:26.966 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:26.966 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:26.966 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:26.966 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:26.966 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:26.966 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:26.966 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.966 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.966 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.967 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.967 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.967 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:26.967 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:26.967 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.967 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.967 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.967 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:26.967 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.968 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.968 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.968 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.968 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.968 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.968 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.968 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:26.968 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:26.969 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.969 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:26.969 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mirroring", 2024-09-11T22:07:26.969 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:26.969 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:26.969 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:26.969 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:26.969 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:26.970 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.970 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.970 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.970 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:26.970 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.970 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.970 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:26.970 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:26.971 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:26.971 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:26.971 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:26.971 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:26.971 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:26.971 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:26.971 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.971 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.971 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.972 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.972 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.972 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:26.972 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:26.972 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.972 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.972 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.972 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:26.973 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.973 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.973 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.973 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.973 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.973 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.973 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.973 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.974 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:26.974 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:26.974 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.974 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.974 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.974 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:26.974 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.974 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.975 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:26.975 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:26.975 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:26.975 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:26.975 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:26.975 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:26.975 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:26.975 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:26.976 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.976 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.976 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.976 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.976 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.976 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:26.976 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:26.976 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.977 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.977 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.977 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:26.977 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.977 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.977 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.977 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.978 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.978 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.978 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.978 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:26.978 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:26.978 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.978 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:26.978 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "nfs", 2024-09-11T22:07:26.978 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:26.979 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:26.979 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:26.979 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:26.979 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:26.979 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.979 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.979 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.979 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:26.980 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.980 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.980 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:26.980 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:26.980 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:26.980 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:26.980 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:26.980 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:26.981 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:26.981 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:26.981 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.981 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.981 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.981 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.981 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.981 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:26.982 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:26.982 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.982 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.982 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.982 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:26.982 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.982 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.982 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.983 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.983 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.983 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.983 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.983 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.983 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:26.983 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:26.983 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.984 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.984 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.984 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:26.984 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.984 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.984 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:26.984 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:26.984 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:26.985 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:26.985 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:26.985 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:26.985 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:26.985 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:26.985 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.985 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.985 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.986 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.986 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.986 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:26.986 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:26.986 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.986 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.986 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.987 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:26.987 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.987 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.987 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.987 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.987 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.987 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.987 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.988 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:26.988 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:26.988 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.988 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:26.988 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "orchestrator", 2024-09-11T22:07:26.988 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:26.988 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:26.988 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:26.989 INFO:teuthology.orchestra.run.smithi001.stdout: "fail_fs": { 2024-09-11T22:07:26.989 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "fail_fs", 2024-09-11T22:07:26.989 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.989 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.989 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:26.989 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:26.989 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.989 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.990 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.990 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-11T22:07:26.990 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.990 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.990 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.990 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.990 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:26.991 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:26.991 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.991 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.991 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.991 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:26.991 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.991 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.991 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:26.991 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:26.992 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:26.992 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:26.992 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:26.992 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:26.992 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:26.992 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:26.992 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.993 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.993 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.993 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.993 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.993 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:26.993 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:26.993 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.993 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.994 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.994 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:26.994 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.994 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.994 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.994 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.994 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.994 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.995 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.995 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.995 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:26.995 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:26.995 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:26.995 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.995 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.995 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:26.995 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.996 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.996 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:26.996 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:26.996 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:26.996 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:26.996 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:26.996 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:26.996 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:26.997 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:26.997 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.997 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.997 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.997 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.997 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.997 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:26.997 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:26.998 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:26.998 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:26.998 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:26.998 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:26.998 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:26.998 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:26.998 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:26.998 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:26.999 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:26.999 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:26.999 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:26.999 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:26.999 INFO:teuthology.orchestra.run.smithi001.stdout: "orchestrator": { 2024-09-11T22:07:26.999 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "orchestrator", 2024-09-11T22:07:26.999 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.000 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.000 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.000 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.000 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.000 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.000 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.000 INFO:teuthology.orchestra.run.smithi001.stdout: "cephadm", 2024-09-11T22:07:27.000 INFO:teuthology.orchestra.run.smithi001.stdout: "rook", 2024-09-11T22:07:27.001 INFO:teuthology.orchestra.run.smithi001.stdout: "test_orchestrator" 2024-09-11T22:07:27.001 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.001 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Orchestrator backend", 2024-09-11T22:07:27.001 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.001 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.001 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.001 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.001 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.002 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.002 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:27.002 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "osd_perf_query", 2024-09-11T22:07:27.002 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:27.002 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:27.002 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:27.002 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:27.003 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:27.003 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.003 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.003 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.003 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.003 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.003 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.003 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.004 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.004 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.004 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.004 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.004 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.004 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.004 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.004 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.005 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.005 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.005 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.005 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.005 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:27.005 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:27.005 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.005 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.006 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.006 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.006 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.006 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.006 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.006 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.006 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.006 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.006 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.007 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.007 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:27.007 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:27.007 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.007 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.007 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.007 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:27.007 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.008 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.008 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.008 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.008 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.008 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.008 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.008 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.008 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.009 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.009 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.009 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.009 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.009 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.009 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.009 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:27.009 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:27.010 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.010 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.010 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.010 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.010 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.010 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.010 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.010 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.011 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.011 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.011 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.011 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.011 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.011 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.011 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:27.011 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "osd_support", 2024-09-11T22:07:27.012 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:27.012 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:27.012 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:27.012 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:27.012 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:27.012 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.012 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.012 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.013 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.013 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.013 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.013 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.013 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.013 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.013 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.013 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.014 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.014 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.014 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.014 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.014 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.014 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.014 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.015 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.015 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:27.015 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:27.015 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.015 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.015 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.015 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.015 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.016 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.016 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.016 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.016 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.016 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.016 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.016 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.016 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:27.017 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:27.017 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.017 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.017 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.017 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:27.017 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.017 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.018 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.018 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.018 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.018 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.018 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.018 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.018 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.018 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.019 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.019 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.019 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.019 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.019 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.019 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:27.019 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:27.019 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.020 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.020 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.020 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.020 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.020 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.020 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.020 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.020 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.021 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.021 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.021 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.021 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.021 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.021 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:27.021 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pg_autoscaler", 2024-09-11T22:07:27.022 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:27.022 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:27.022 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:27.022 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:27.022 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:27.022 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.022 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.022 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.023 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.023 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.023 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.023 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.023 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.023 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.023 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.023 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.024 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.024 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.024 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.024 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.024 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.024 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.024 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.024 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.024 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:27.025 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:27.025 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.025 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.025 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.025 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.025 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.025 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.025 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.026 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.026 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.026 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.026 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.026 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.026 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:27.026 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:27.026 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.027 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.027 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.027 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:27.027 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.027 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.027 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.027 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.028 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.028 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.028 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.028 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.028 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.028 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.028 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.029 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.029 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.029 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.029 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.029 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:27.029 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:27.029 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.029 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.029 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.030 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.030 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.030 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.030 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.030 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.030 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.030 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.030 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.031 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.031 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-09-11T22:07:27.031 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-09-11T22:07:27.031 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-11T22:07:27.031 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.031 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.031 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-09-11T22:07:27.031 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.032 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.032 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.032 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.032 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.032 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.032 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.032 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.032 INFO:teuthology.orchestra.run.smithi001.stdout: "threshold": { 2024-09-11T22:07:27.033 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "threshold", 2024-09-11T22:07:27.033 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-09-11T22:07:27.033 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.033 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.033 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "3.0", 2024-09-11T22:07:27.033 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1.0", 2024-09-11T22:07:27.033 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.033 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.034 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "scaling threshold", 2024-09-11T22:07:27.034 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-11T22:07:27.034 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.034 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.034 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.034 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.034 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.034 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:27.035 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "progress", 2024-09-11T22:07:27.035 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:27.035 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:27.035 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:27.035 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_pg_recovery_event": { 2024-09-11T22:07:27.035 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "allow_pg_recovery_event", 2024-09-11T22:07:27.035 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.035 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.035 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.036 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.036 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.036 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.036 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.036 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-11T22:07:27.036 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.036 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.036 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.037 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.037 INFO:teuthology.orchestra.run.smithi001.stdout: "enabled": { 2024-09-11T22:07:27.037 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enabled", 2024-09-11T22:07:27.037 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.037 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.037 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.037 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:27.037 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.038 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.038 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.038 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.038 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.038 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.038 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.038 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.038 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:27.039 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:27.039 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.039 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.039 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.039 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.039 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.039 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.039 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.040 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.040 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.040 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.040 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.040 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.040 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.040 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.040 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.041 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.041 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.041 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.041 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.041 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:27.041 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:27.041 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.041 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.042 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.042 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.042 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.042 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.042 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.042 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.042 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.042 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.042 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.043 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.043 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:27.043 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:27.043 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.043 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.043 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.043 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:27.043 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.044 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.044 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.044 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.044 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.044 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.044 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.044 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.044 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.045 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.045 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.045 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.045 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.045 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.045 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.045 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:27.045 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:27.046 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.046 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.046 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.046 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.046 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.046 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.046 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.046 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.047 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.047 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.047 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.047 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.047 INFO:teuthology.orchestra.run.smithi001.stdout: "max_completed_events": { 2024-09-11T22:07:27.047 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_completed_events", 2024-09-11T22:07:27.047 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:27.047 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.048 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.048 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "50", 2024-09-11T22:07:27.048 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.048 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.048 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.048 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "number of past completed events to remember", 2024-09-11T22:07:27.048 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.048 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.049 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.049 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.049 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-09-11T22:07:27.049 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-09-11T22:07:27.049 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-11T22:07:27.049 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.049 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.050 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-09-11T22:07:27.050 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.050 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.050 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.050 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how long the module is going to sleep", 2024-09-11T22:07:27.050 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.050 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.050 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.051 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.051 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.051 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.051 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:27.051 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "prometheus", 2024-09-11T22:07:27.051 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:27.051 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:27.052 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:27.052 INFO:teuthology.orchestra.run.smithi001.stdout: "cache": { 2024-09-11T22:07:27.052 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cache", 2024-09-11T22:07:27.052 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.052 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.052 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.052 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:27.052 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.053 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.053 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.053 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.053 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.053 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.053 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.053 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.053 INFO:teuthology.orchestra.run.smithi001.stdout: "exclude_perf_counters": { 2024-09-11T22:07:27.054 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "exclude_perf_counters", 2024-09-11T22:07:27.054 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.054 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.054 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.054 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:27.054 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.054 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.054 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.055 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-11T22:07:27.055 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-11T22:07:27.055 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.055 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.055 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.055 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:27.055 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:27.055 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.056 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.056 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.056 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.056 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.056 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.056 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.056 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.056 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.057 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.057 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.057 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.057 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.057 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.057 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.057 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.057 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.058 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.058 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.058 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:27.058 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:27.058 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.058 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.058 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.058 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.059 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.059 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.059 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.059 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.059 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.059 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.059 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.059 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.060 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:27.060 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:27.060 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.060 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.060 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.060 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:27.060 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.060 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.060 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.061 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.061 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.061 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.061 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.061 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.061 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.061 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.061 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.062 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.062 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.062 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.062 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.062 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:27.062 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:27.062 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.062 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.063 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.063 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.063 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.063 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.063 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.063 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.063 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.063 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.064 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.064 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.064 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_stats_pools": { 2024-09-11T22:07:27.064 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rbd_stats_pools", 2024-09-11T22:07:27.064 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.064 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.064 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.064 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.065 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.065 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.065 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.065 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.065 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.065 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.065 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.065 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.066 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-11T22:07:27.066 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-11T22:07:27.066 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:27.066 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.066 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.066 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "300", 2024-09-11T22:07:27.066 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.066 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.067 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.067 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.067 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.067 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.067 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.067 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.067 INFO:teuthology.orchestra.run.smithi001.stdout: "scrape_interval": { 2024-09-11T22:07:27.067 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "scrape_interval", 2024-09-11T22:07:27.067 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-09-11T22:07:27.068 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.068 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.068 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "15.0", 2024-09-11T22:07:27.068 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.068 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.068 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.068 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.068 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.069 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.069 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.069 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.069 INFO:teuthology.orchestra.run.smithi001.stdout: "server_addr": { 2024-09-11T22:07:27.069 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_addr", 2024-09-11T22:07:27.069 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.069 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.069 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.070 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "::", 2024-09-11T22:07:27.070 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.070 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.070 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.070 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-11T22:07:27.070 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.070 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.070 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.071 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.071 INFO:teuthology.orchestra.run.smithi001.stdout: "server_port": { 2024-09-11T22:07:27.071 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_port", 2024-09-11T22:07:27.071 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:27.071 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.071 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.071 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "9283", 2024-09-11T22:07:27.071 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.072 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.072 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.072 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-11T22:07:27.072 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.072 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.072 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.072 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.072 INFO:teuthology.orchestra.run.smithi001.stdout: "stale_cache_strategy": { 2024-09-11T22:07:27.073 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "stale_cache_strategy", 2024-09-11T22:07:27.073 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.073 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.073 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.073 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "log", 2024-09-11T22:07:27.073 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.073 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.074 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.074 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.074 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.074 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.074 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.074 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.074 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_behaviour": { 2024-09-11T22:07:27.074 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_behaviour", 2024-09-11T22:07:27.075 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.075 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.075 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.075 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "default", 2024-09-11T22:07:27.075 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.075 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.075 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.076 INFO:teuthology.orchestra.run.smithi001.stdout: "default", 2024-09-11T22:07:27.076 INFO:teuthology.orchestra.run.smithi001.stdout: "error" 2024-09-11T22:07:27.076 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.076 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.076 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.076 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.076 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.076 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.077 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_error_status_code": { 2024-09-11T22:07:27.077 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_error_status_code", 2024-09-11T22:07:27.077 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:27.077 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.077 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.077 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "500", 2024-09-11T22:07:27.077 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "400", 2024-09-11T22:07:27.078 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "599", 2024-09-11T22:07:27.078 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.078 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.078 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.078 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.078 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.078 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.078 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.079 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.079 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:27.079 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rbd_support", 2024-09-11T22:07:27.079 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:27.079 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:27.079 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:27.080 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:27.080 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:27.080 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.080 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.080 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.080 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.080 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.080 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.081 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.081 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.081 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.081 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.081 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.081 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.081 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.081 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.082 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.082 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.082 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.082 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.082 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.082 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:27.082 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:27.082 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.083 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.083 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.083 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.083 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.083 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.083 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.083 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.083 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.084 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.084 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.084 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.084 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:27.084 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:27.084 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.084 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.084 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.085 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:27.085 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.085 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.085 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.085 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.085 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.085 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.085 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.086 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.086 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.086 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.086 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.086 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.086 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.086 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.086 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.087 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:27.087 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:27.087 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.087 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.087 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.087 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.087 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.087 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.088 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.088 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.088 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.088 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.088 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.088 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.088 INFO:teuthology.orchestra.run.smithi001.stdout: "max_concurrent_snap_create": { 2024-09-11T22:07:27.088 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_concurrent_snap_create", 2024-09-11T22:07:27.089 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:27.089 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.089 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.089 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-11T22:07:27.089 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.089 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.089 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.089 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.090 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.090 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.090 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.090 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.090 INFO:teuthology.orchestra.run.smithi001.stdout: "mirror_snapshot_schedule": { 2024-09-11T22:07:27.090 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mirror_snapshot_schedule", 2024-09-11T22:07:27.090 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.090 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.091 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.091 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.091 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.091 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.091 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.091 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.091 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.091 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.092 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.092 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.092 INFO:teuthology.orchestra.run.smithi001.stdout: "trash_purge_schedule": { 2024-09-11T22:07:27.092 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "trash_purge_schedule", 2024-09-11T22:07:27.092 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.092 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.092 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.092 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.093 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.093 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.093 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.093 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.093 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.093 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.093 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.093 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.094 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.094 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.094 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:27.094 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "restful", 2024-09-11T22:07:27.094 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:27.094 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:27.094 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:27.094 INFO:teuthology.orchestra.run.smithi001.stdout: "enable_auth": { 2024-09-11T22:07:27.095 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enable_auth", 2024-09-11T22:07:27.095 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.095 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.095 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.095 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:27.095 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.095 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.095 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.096 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.096 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.096 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.096 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.096 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.096 INFO:teuthology.orchestra.run.smithi001.stdout: "key_file": { 2024-09-11T22:07:27.096 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "key_file", 2024-09-11T22:07:27.096 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.096 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.097 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.097 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.097 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.097 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.097 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.097 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.097 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.098 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.098 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.098 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.098 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:27.098 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:27.098 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.099 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.099 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.099 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.099 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.099 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.099 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.099 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.099 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.100 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.100 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.100 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.100 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.100 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.100 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.100 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.100 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.101 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.101 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.101 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:27.101 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:27.101 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.101 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.101 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.101 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.102 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.102 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.102 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.102 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.102 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.102 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.102 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.102 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.103 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:27.103 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:27.103 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.103 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.103 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.103 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:27.103 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.103 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.104 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.104 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.104 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.104 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.104 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.104 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.104 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.104 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.105 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.105 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.105 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.105 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.105 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.105 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:27.105 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:27.105 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.106 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.106 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.106 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.106 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.106 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.106 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.106 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.106 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.107 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.107 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.107 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.107 INFO:teuthology.orchestra.run.smithi001.stdout: "server_addr": { 2024-09-11T22:07:27.107 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_addr", 2024-09-11T22:07:27.107 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.107 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.108 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.108 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.108 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.108 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.108 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.108 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.108 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.108 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.109 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.109 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.109 INFO:teuthology.orchestra.run.smithi001.stdout: "server_port": { 2024-09-11T22:07:27.109 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_port", 2024-09-11T22:07:27.109 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.109 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.109 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.109 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.110 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.110 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.110 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.110 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.110 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.110 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.110 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.110 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.111 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.111 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.111 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:27.111 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rgw", 2024-09-11T22:07:27.111 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:27.111 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:27.111 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:27.112 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:27.112 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:27.112 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.112 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.112 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.112 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.112 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.112 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.112 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.113 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.113 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.113 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.113 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.113 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.113 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.113 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.114 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.114 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.114 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.114 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.114 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.114 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:27.114 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:27.114 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.114 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.115 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.115 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.115 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.115 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.115 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.115 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.115 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.115 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.116 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.116 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.116 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:27.116 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:27.116 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.116 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.116 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.116 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:27.117 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.117 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.117 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.117 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.117 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.117 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.117 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.117 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.118 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.118 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.118 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.118 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.118 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.118 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.118 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.118 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:27.119 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:27.119 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.119 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.119 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.119 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.119 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.119 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.119 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.120 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.120 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.120 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.120 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.120 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.120 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.120 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.120 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:27.121 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rook", 2024-09-11T22:07:27.121 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:27.121 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:27.121 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:27.121 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:27.121 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:27.121 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.121 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.122 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.122 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.122 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.122 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.122 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.122 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.122 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.122 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.123 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.123 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.123 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.123 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.123 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.123 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.123 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.124 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.124 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.124 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:27.124 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:27.124 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.124 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.124 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.124 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.125 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.125 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.125 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.125 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.125 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.125 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.125 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.125 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.126 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:27.126 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:27.126 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.126 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.126 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.126 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:27.126 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.126 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.126 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.127 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.127 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.127 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.127 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.127 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.127 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.127 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.127 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.128 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.128 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.128 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.128 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.128 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:27.128 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:27.128 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.128 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.129 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.129 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.129 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.129 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.129 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.129 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.129 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.129 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.130 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.130 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.130 INFO:teuthology.orchestra.run.smithi001.stdout: "storage_class": { 2024-09-11T22:07:27.130 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "storage_class", 2024-09-11T22:07:27.130 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.130 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.130 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.130 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "local", 2024-09-11T22:07:27.131 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.131 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.131 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.131 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-11T22:07:27.131 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.131 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.131 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.132 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.132 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.132 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.132 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:27.132 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "selftest", 2024-09-11T22:07:27.132 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:27.132 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:27.132 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:27.133 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:27.133 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:27.133 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.133 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.133 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.133 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.133 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.133 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.134 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.134 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.134 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.134 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.134 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.134 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.134 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.135 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.135 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.135 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.135 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.135 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.135 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.135 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:27.135 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:27.136 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.136 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.136 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.136 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.136 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.136 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.136 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.136 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.137 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.137 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.137 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.137 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.137 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:27.137 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:27.137 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.138 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.138 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.138 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:27.138 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.138 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.138 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.138 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.138 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.139 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.139 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.139 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.139 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.139 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.139 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.139 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.139 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.140 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.140 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.140 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:27.140 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:27.140 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.140 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.140 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.140 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.141 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.141 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.141 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.141 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.141 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.141 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.141 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.141 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.142 INFO:teuthology.orchestra.run.smithi001.stdout: "roption1": { 2024-09-11T22:07:27.142 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "roption1", 2024-09-11T22:07:27.142 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.142 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.142 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.142 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.142 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.142 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.143 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.143 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.143 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.143 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.143 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.143 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.143 INFO:teuthology.orchestra.run.smithi001.stdout: "roption2": { 2024-09-11T22:07:27.143 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "roption2", 2024-09-11T22:07:27.144 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.144 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.144 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.144 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "xyz", 2024-09-11T22:07:27.144 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.144 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.144 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.144 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.144 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.145 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.145 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.145 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.145 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption1": { 2024-09-11T22:07:27.145 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption1", 2024-09-11T22:07:27.145 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.145 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.145 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.146 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.146 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.146 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.146 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.146 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.146 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.146 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.146 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.147 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.147 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption2": { 2024-09-11T22:07:27.147 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption2", 2024-09-11T22:07:27.147 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:27.147 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.147 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.147 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.148 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.148 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.148 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.148 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.148 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.148 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.148 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.148 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.149 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption3": { 2024-09-11T22:07:27.149 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption3", 2024-09-11T22:07:27.149 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-09-11T22:07:27.149 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.149 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.149 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.149 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.150 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.150 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.150 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.150 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.150 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.150 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.150 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.150 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption4": { 2024-09-11T22:07:27.151 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption4", 2024-09-11T22:07:27.151 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.151 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.151 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.151 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.151 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.151 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.151 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.152 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.152 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.152 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.152 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.152 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.152 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption5": { 2024-09-11T22:07:27.152 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption5", 2024-09-11T22:07:27.152 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.153 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.153 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.153 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.153 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.153 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.153 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.153 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.153 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.154 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.154 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.154 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.154 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption6": { 2024-09-11T22:07:27.154 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption6", 2024-09-11T22:07:27.154 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.154 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.154 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.155 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:27.155 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.155 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.155 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.155 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.155 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.155 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.155 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.155 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.156 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption7": { 2024-09-11T22:07:27.156 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption7", 2024-09-11T22:07:27.156 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:27.156 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.156 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.156 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.156 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-09-11T22:07:27.156 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "42", 2024-09-11T22:07:27.156 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.157 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.157 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.157 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.157 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.157 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.157 INFO:teuthology.orchestra.run.smithi001.stdout: "testkey": { 2024-09-11T22:07:27.157 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "testkey", 2024-09-11T22:07:27.157 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.158 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.158 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.158 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.158 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.158 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.158 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.158 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.158 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.159 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.159 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.159 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.159 INFO:teuthology.orchestra.run.smithi001.stdout: "testlkey": { 2024-09-11T22:07:27.159 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "testlkey", 2024-09-11T22:07:27.159 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.159 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.159 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.160 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.160 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.160 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.160 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.160 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.160 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.160 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.160 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.161 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.161 INFO:teuthology.orchestra.run.smithi001.stdout: "testnewline": { 2024-09-11T22:07:27.161 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "testnewline", 2024-09-11T22:07:27.161 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.161 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.161 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.161 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.161 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.162 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.162 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.162 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.162 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.162 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.162 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.162 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.162 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.163 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.163 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:27.163 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "snap_schedule", 2024-09-11T22:07:27.163 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:27.163 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:27.163 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:27.163 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_m_granularity": { 2024-09-11T22:07:27.163 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "allow_m_granularity", 2024-09-11T22:07:27.163 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.164 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.164 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.164 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.164 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.164 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.164 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.164 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "allow minute scheduled snapshots", 2024-09-11T22:07:27.165 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.165 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.165 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.165 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.165 INFO:teuthology.orchestra.run.smithi001.stdout: "dump_on_update": { 2024-09-11T22:07:27.165 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "dump_on_update", 2024-09-11T22:07:27.165 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.165 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.166 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.166 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.166 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.166 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.166 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.166 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "dump database to debug log on update", 2024-09-11T22:07:27.166 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.166 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.167 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.167 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.167 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:27.167 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:27.167 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.167 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.167 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.168 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.168 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.168 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.168 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.168 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.168 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.168 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.168 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.169 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.169 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.169 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.169 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.169 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.169 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.169 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.169 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.170 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:27.170 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:27.170 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.170 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.170 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.170 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.170 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.171 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.171 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.171 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.171 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.171 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.171 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.171 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.171 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:27.172 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:27.172 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.172 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.172 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.172 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:27.172 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.172 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.173 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.173 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.173 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.173 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.173 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.173 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.173 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.173 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.174 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.174 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.174 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.174 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.174 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.174 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:27.174 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:27.174 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.174 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.175 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.175 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.175 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.175 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.175 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.175 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.175 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.175 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.176 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.176 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.176 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.176 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.176 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:27.176 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "stats", 2024-09-11T22:07:27.176 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:27.176 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:27.177 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:27.177 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:27.177 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:27.177 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.177 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.177 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.177 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.177 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.178 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.178 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.178 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.178 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.178 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.178 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.178 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.178 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.179 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.179 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.179 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.179 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.179 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.179 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.179 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:27.179 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:27.180 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.180 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.180 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.180 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.180 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.180 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.180 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.180 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.181 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.181 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.181 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.181 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.181 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:27.181 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:27.181 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.181 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.182 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.182 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:27.182 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.182 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.182 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.182 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.182 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.182 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.183 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.183 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.183 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.183 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.183 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.183 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.183 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.183 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.184 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.184 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:27.184 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:27.184 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.184 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.184 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.184 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.184 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.185 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.185 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.185 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.185 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.185 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.185 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.185 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.186 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.186 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.186 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:27.186 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "status", 2024-09-11T22:07:27.186 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:27.186 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:27.186 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:27.186 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:27.186 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:27.187 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.187 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.187 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.187 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.187 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.187 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.187 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.187 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.188 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.188 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.188 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.188 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.188 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.188 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.188 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.189 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.189 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.189 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.189 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.189 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:27.189 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:27.189 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.189 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.190 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.190 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.190 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.190 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.190 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.190 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.190 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.190 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.191 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.191 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.191 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:27.191 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:27.191 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.191 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.191 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.191 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:27.192 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.192 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.192 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.192 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.192 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.192 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.192 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.192 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.192 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.193 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.193 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.193 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.193 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.193 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.193 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.193 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:27.193 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:27.194 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.194 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.194 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.194 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.194 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.194 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.194 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.194 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.195 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.195 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.195 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.195 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.195 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.195 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.195 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:27.195 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "telegraf", 2024-09-11T22:07:27.196 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:27.196 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:27.196 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:27.196 INFO:teuthology.orchestra.run.smithi001.stdout: "address": { 2024-09-11T22:07:27.196 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "address", 2024-09-11T22:07:27.196 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.196 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.197 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.197 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-11T22:07:27.197 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.197 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.197 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.197 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.197 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.198 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.198 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.198 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.198 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-09-11T22:07:27.198 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-09-11T22:07:27.198 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-11T22:07:27.198 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.198 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.199 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "15", 2024-09-11T22:07:27.199 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.199 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.199 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.199 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.199 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.199 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.199 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.200 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.200 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:27.200 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:27.200 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.200 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.200 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.200 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.201 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.201 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.201 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.201 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.201 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.201 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.201 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.201 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.202 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.202 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.202 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.202 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.202 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.202 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.202 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.202 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:27.203 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:27.203 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.203 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.203 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.203 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.203 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.203 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.203 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.203 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.204 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.204 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.204 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.204 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.204 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:27.204 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:27.204 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.204 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.205 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.205 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:27.205 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.205 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.205 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.205 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.205 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.205 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.206 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.206 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.206 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.206 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.206 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.206 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.206 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.206 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.207 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.207 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:27.207 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:27.207 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.207 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.207 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.207 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.207 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.208 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.208 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.208 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.208 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.208 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.208 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.208 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.208 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.209 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.209 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:27.209 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "telemetry", 2024-09-11T22:07:27.209 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:27.209 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:27.209 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:27.209 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_basic": { 2024-09-11T22:07:27.209 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_basic", 2024-09-11T22:07:27.210 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.210 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.210 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.210 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:27.210 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.210 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.210 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.211 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-11T22:07:27.211 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.211 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.211 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.211 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.211 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_crash": { 2024-09-11T22:07:27.211 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_crash", 2024-09-11T22:07:27.211 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.212 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.212 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.212 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:27.212 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.212 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.212 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.212 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-11T22:07:27.213 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.213 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.213 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.213 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.213 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_device": { 2024-09-11T22:07:27.213 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_device", 2024-09-11T22:07:27.213 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.213 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.214 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.214 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:27.214 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.214 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.214 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.214 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-11T22:07:27.214 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.214 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.215 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.215 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.215 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_ident": { 2024-09-11T22:07:27.215 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_ident", 2024-09-11T22:07:27.215 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.215 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.215 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.215 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.216 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.216 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.216 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.216 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-11T22:07:27.216 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.216 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.216 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.216 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.217 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_perf": { 2024-09-11T22:07:27.217 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_perf", 2024-09-11T22:07:27.217 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.217 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.217 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.217 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.217 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.217 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.218 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.218 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-11T22:07:27.218 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.218 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.218 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.218 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.218 INFO:teuthology.orchestra.run.smithi001.stdout: "contact": { 2024-09-11T22:07:27.218 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "contact", 2024-09-11T22:07:27.218 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.219 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.219 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.219 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.219 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.219 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.219 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.219 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.219 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.219 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.220 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.220 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.220 INFO:teuthology.orchestra.run.smithi001.stdout: "description": { 2024-09-11T22:07:27.220 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "description", 2024-09-11T22:07:27.220 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.220 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.220 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.221 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.221 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.221 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.221 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.221 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.221 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.221 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.222 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.222 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.222 INFO:teuthology.orchestra.run.smithi001.stdout: "device_url": { 2024-09-11T22:07:27.222 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "device_url", 2024-09-11T22:07:27.222 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.222 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.222 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.222 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-11T22:07:27.222 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.223 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.223 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.223 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.223 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.223 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.223 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.223 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.224 INFO:teuthology.orchestra.run.smithi001.stdout: "enabled": { 2024-09-11T22:07:27.224 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enabled", 2024-09-11T22:07:27.224 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.224 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.224 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.224 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.224 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.224 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.225 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.225 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.225 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.225 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.225 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.225 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.225 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-09-11T22:07:27.225 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-09-11T22:07:27.226 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:27.226 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.226 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.226 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "24", 2024-09-11T22:07:27.226 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "8", 2024-09-11T22:07:27.226 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.226 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.227 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.227 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.227 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.227 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.227 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.227 INFO:teuthology.orchestra.run.smithi001.stdout: "last_opt_revision": { 2024-09-11T22:07:27.227 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "last_opt_revision", 2024-09-11T22:07:27.227 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:27.228 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.228 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.228 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "1", 2024-09-11T22:07:27.228 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.228 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.228 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.228 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.229 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.229 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.229 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.229 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.229 INFO:teuthology.orchestra.run.smithi001.stdout: "leaderboard": { 2024-09-11T22:07:27.229 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "leaderboard", 2024-09-11T22:07:27.229 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.229 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.230 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.230 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.230 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.230 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.230 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.230 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.230 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.230 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.230 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.231 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.231 INFO:teuthology.orchestra.run.smithi001.stdout: "leaderboard_description": { 2024-09-11T22:07:27.231 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "leaderboard_description", 2024-09-11T22:07:27.231 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.231 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.231 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.231 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.231 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.232 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.232 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.232 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.232 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.232 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.232 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.232 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.233 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:27.233 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:27.233 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.233 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.233 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.233 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.233 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.233 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.233 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.234 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.234 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.234 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.234 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.234 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.234 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.234 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.235 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.235 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.235 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.235 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.235 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.235 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:27.235 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:27.235 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.236 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.236 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.236 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.236 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.236 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.236 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.236 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.236 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.237 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.237 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.237 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.237 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:27.237 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:27.237 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.237 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.237 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.238 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:27.238 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.238 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.238 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.238 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.238 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.238 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.239 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.239 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.239 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.239 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.239 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.239 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.239 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.239 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.240 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.240 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:27.240 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:27.240 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.240 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.240 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.240 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.240 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.241 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.241 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.241 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.241 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.241 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.241 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.241 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.241 INFO:teuthology.orchestra.run.smithi001.stdout: "organization": { 2024-09-11T22:07:27.242 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "organization", 2024-09-11T22:07:27.242 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.242 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.242 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.242 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.242 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.242 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.242 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.243 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.243 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.243 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.243 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.243 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.243 INFO:teuthology.orchestra.run.smithi001.stdout: "proxy": { 2024-09-11T22:07:27.243 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "proxy", 2024-09-11T22:07:27.243 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.243 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.244 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.244 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.244 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.244 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.244 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.244 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.244 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.245 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.245 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.245 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.245 INFO:teuthology.orchestra.run.smithi001.stdout: "url": { 2024-09-11T22:07:27.245 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "url", 2024-09-11T22:07:27.245 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.245 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.245 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.246 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-11T22:07:27.246 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.246 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.246 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.246 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.246 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.246 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.246 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.247 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.247 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.247 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.247 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:27.247 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "test_orchestrator", 2024-09-11T22:07:27.247 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:27.247 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:27.247 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:27.248 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:27.248 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:27.248 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.248 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.248 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.248 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.248 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.248 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.249 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.249 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.249 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.249 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.249 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.249 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.249 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.249 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.250 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.250 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.250 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.250 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.250 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.250 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:27.250 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:27.251 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.251 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.251 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.251 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.251 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.251 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.251 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.251 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.252 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.252 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.252 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.252 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.252 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:27.252 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:27.252 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.252 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.253 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.253 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:27.253 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.253 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.253 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.253 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.253 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.254 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.254 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.254 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.254 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.254 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.254 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.254 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.254 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.255 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.255 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.255 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:27.255 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:27.255 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.255 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.255 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.255 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.256 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.256 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.256 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.256 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.256 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.256 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.256 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.256 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.257 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.257 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.257 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:27.257 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "volumes", 2024-09-11T22:07:27.257 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:27.257 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:27.257 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:27.257 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:27.258 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:27.258 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.258 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.258 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.258 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.258 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.258 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.258 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.259 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.259 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.259 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.259 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.259 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.259 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.259 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.259 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.260 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.260 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.260 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.260 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.260 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:27.260 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:27.260 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.260 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.261 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.261 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.261 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.261 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.261 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.261 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.261 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.261 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.262 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.262 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.262 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:27.262 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:27.262 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.262 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.262 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.262 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:27.263 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.263 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.263 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.263 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.263 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.263 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.263 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.263 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.264 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.264 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.264 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.264 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.264 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.264 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.264 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.264 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:27.265 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:27.265 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.265 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.265 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.265 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.265 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.265 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.265 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.266 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.266 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.266 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.266 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.266 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.266 INFO:teuthology.orchestra.run.smithi001.stdout: "max_concurrent_clones": { 2024-09-11T22:07:27.266 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_concurrent_clones", 2024-09-11T22:07:27.266 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:27.267 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.267 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.267 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "4", 2024-09-11T22:07:27.267 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.267 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.267 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.267 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-11T22:07:27.267 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.267 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.268 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.268 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.268 INFO:teuthology.orchestra.run.smithi001.stdout: "periodic_async_work": { 2024-09-11T22:07:27.268 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "periodic_async_work", 2024-09-11T22:07:27.268 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.268 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.268 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.268 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.269 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.269 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.269 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.269 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Periodically check for async work", 2024-09-11T22:07:27.269 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.269 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.269 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.270 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.270 INFO:teuthology.orchestra.run.smithi001.stdout: "snapshot_clone_delay": { 2024-09-11T22:07:27.270 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "snapshot_clone_delay", 2024-09-11T22:07:27.270 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:27.270 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.270 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.270 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-09-11T22:07:27.270 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.271 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.271 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.271 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-11T22:07:27.271 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.271 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.271 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.271 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.271 INFO:teuthology.orchestra.run.smithi001.stdout: "snapshot_clone_no_wait": { 2024-09-11T22:07:27.271 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "snapshot_clone_no_wait", 2024-09-11T22:07:27.272 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.272 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.272 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.272 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:27.272 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.272 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.272 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.273 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-11T22:07:27.273 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.273 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.273 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.273 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.273 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.273 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.274 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:27.274 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix", 2024-09-11T22:07:27.274 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:27.274 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:27.274 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:27.274 INFO:teuthology.orchestra.run.smithi001.stdout: "discovery_interval": { 2024-09-11T22:07:27.274 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "discovery_interval", 2024-09-11T22:07:27.274 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-09-11T22:07:27.275 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.275 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.275 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "100", 2024-09-11T22:07:27.275 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.275 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.275 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.275 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.275 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.276 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.276 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.276 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.276 INFO:teuthology.orchestra.run.smithi001.stdout: "identifier": { 2024-09-11T22:07:27.276 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "identifier", 2024-09-11T22:07:27.276 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.276 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.276 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.277 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.277 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.277 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.277 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.277 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.277 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.277 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.277 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.278 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.278 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-09-11T22:07:27.278 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-09-11T22:07:27.278 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-11T22:07:27.278 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.278 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.278 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-09-11T22:07:27.278 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.279 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.279 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.279 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.279 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.279 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.279 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.279 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.279 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:27.280 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:27.280 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.280 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.280 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.280 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.280 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.280 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.280 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.281 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.281 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.281 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.281 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.281 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.281 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.281 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.281 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.282 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.282 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.282 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.282 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.282 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:27.282 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:27.282 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.283 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.283 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.283 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.283 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.283 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.283 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.283 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.283 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.284 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.284 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.284 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.284 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:27.284 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:27.284 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.284 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.284 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.285 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:27.285 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.285 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.285 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.285 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.285 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.285 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.286 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.286 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.286 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.286 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.286 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.286 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.286 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.286 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.287 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.287 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:27.287 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:27.287 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.287 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.287 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.287 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.287 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.288 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.288 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.288 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.288 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.288 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.288 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.288 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.288 INFO:teuthology.orchestra.run.smithi001.stdout: "zabbix_host": { 2024-09-11T22:07:27.289 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix_host", 2024-09-11T22:07:27.289 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.289 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.289 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.289 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.289 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.289 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.289 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.290 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.290 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.290 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.290 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.290 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.290 INFO:teuthology.orchestra.run.smithi001.stdout: "zabbix_port": { 2024-09-11T22:07:27.290 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix_port", 2024-09-11T22:07:27.290 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:27.291 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.291 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.291 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10051", 2024-09-11T22:07:27.291 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.291 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.291 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.291 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.291 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.292 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.292 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.292 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.292 INFO:teuthology.orchestra.run.smithi001.stdout: "zabbix_sender": { 2024-09-11T22:07:27.292 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix_sender", 2024-09-11T22:07:27.292 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.292 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.292 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.293 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-11T22:07:27.293 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.293 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.293 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.293 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.293 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.293 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.294 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.294 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.294 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.294 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.294 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-11T22:07:27.294 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.294 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:27.295 INFO:teuthology.orchestra.run.smithi001.stdout: "gid": 24874, 2024-09-11T22:07:27.295 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "z", 2024-09-11T22:07:27.295 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_features": 4540701547738038271, 2024-09-11T22:07:27.295 INFO:teuthology.orchestra.run.smithi001.stdout: "available_modules": [ 2024-09-11T22:07:27.295 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:27.295 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "alerts", 2024-09-11T22:07:27.295 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:27.295 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:27.296 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:27.296 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-09-11T22:07:27.296 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-09-11T22:07:27.296 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-11T22:07:27.296 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.296 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.296 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-09-11T22:07:27.296 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.296 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.297 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.297 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How frequently to reexamine health status", 2024-09-11T22:07:27.297 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.297 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.297 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.297 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.297 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:27.298 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:27.298 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.298 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.298 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.298 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.298 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.298 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.298 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.299 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.299 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.299 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.299 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.299 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.299 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.299 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.299 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.300 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.300 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.300 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.300 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.300 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:27.300 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:27.300 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.300 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.301 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.301 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.301 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.301 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.301 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.301 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.301 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.301 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.302 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.302 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.302 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:27.302 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:27.302 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.302 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.302 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.302 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:27.303 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.303 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.303 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.303 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.303 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.303 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.303 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.304 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.304 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.304 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.304 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.304 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.304 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.304 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.304 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.305 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:27.305 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:27.305 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.305 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.305 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.305 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.305 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.305 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.306 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.306 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.306 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.306 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.306 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.306 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.306 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_destination": { 2024-09-11T22:07:27.306 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_destination", 2024-09-11T22:07:27.307 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.307 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.307 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.307 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.307 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.307 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.307 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.307 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Email address to send alerts to", 2024-09-11T22:07:27.308 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.308 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.308 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.308 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.308 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_from_name": { 2024-09-11T22:07:27.308 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_from_name", 2024-09-11T22:07:27.308 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.309 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.309 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.309 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "Ceph", 2024-09-11T22:07:27.309 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.309 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.309 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.309 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Email From: name", 2024-09-11T22:07:27.309 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.310 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.310 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.310 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.310 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_host": { 2024-09-11T22:07:27.310 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_host", 2024-09-11T22:07:27.310 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.310 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.310 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.311 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.311 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.311 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.311 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.311 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SMTP server", 2024-09-11T22:07:27.311 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.311 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.311 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.312 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.312 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_password": { 2024-09-11T22:07:27.312 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_password", 2024-09-11T22:07:27.312 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.312 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.312 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.312 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.313 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.313 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.313 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.313 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Password to authenticate with", 2024-09-11T22:07:27.313 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.313 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.313 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.314 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.314 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_port": { 2024-09-11T22:07:27.314 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_port", 2024-09-11T22:07:27.314 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:27.314 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.314 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.314 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "465", 2024-09-11T22:07:27.314 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.315 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.315 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.315 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SMTP port", 2024-09-11T22:07:27.315 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.315 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.315 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.315 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.315 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_sender": { 2024-09-11T22:07:27.316 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_sender", 2024-09-11T22:07:27.316 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.316 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.316 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.316 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.316 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.316 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.317 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.317 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SMTP envelope sender", 2024-09-11T22:07:27.317 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.317 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.317 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.317 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.317 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_ssl": { 2024-09-11T22:07:27.317 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_ssl", 2024-09-11T22:07:27.318 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.318 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.318 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.318 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:27.318 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.318 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.318 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.318 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-11T22:07:27.319 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.319 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.319 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.319 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.319 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_user": { 2024-09-11T22:07:27.319 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_user", 2024-09-11T22:07:27.319 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.320 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.320 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.320 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.320 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.320 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.320 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.320 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "User to authenticate as", 2024-09-11T22:07:27.320 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.321 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.321 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.321 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.321 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.321 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.321 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:27.321 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "balancer", 2024-09-11T22:07:27.321 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:27.322 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:27.322 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:27.322 INFO:teuthology.orchestra.run.smithi001.stdout: "active": { 2024-09-11T22:07:27.322 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "active", 2024-09-11T22:07:27.322 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.322 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.322 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.322 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:27.322 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.323 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.323 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.323 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "automatically balance PGs across cluster", 2024-09-11T22:07:27.323 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.323 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.323 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.323 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.323 INFO:teuthology.orchestra.run.smithi001.stdout: "begin_time": { 2024-09-11T22:07:27.324 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "begin_time", 2024-09-11T22:07:27.324 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.324 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.324 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.324 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0000", 2024-09-11T22:07:27.324 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.324 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.324 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.325 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "beginning time of day to automatically balance", 2024-09-11T22:07:27.325 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-11T22:07:27.325 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.325 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.325 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.325 INFO:teuthology.orchestra.run.smithi001.stdout: "begin_weekday": { 2024-09-11T22:07:27.325 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "begin_weekday", 2024-09-11T22:07:27.325 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-09-11T22:07:27.326 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.326 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.326 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-09-11T22:07:27.326 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "0", 2024-09-11T22:07:27.326 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "6", 2024-09-11T22:07:27.326 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.326 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-11T22:07:27.326 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-11T22:07:27.327 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.327 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.327 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.327 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_compat_max_iterations": { 2024-09-11T22:07:27.327 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crush_compat_max_iterations", 2024-09-11T22:07:27.327 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-09-11T22:07:27.327 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.327 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.328 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "25", 2024-09-11T22:07:27.328 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-09-11T22:07:27.328 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "250", 2024-09-11T22:07:27.328 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.328 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-11T22:07:27.328 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.328 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.328 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.329 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.329 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_compat_metrics": { 2024-09-11T22:07:27.329 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crush_compat_metrics", 2024-09-11T22:07:27.329 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.329 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.329 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.329 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "pgs,objects,bytes", 2024-09-11T22:07:27.329 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.330 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.330 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.332 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-11T22:07:27.332 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-11T22:07:27.332 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.333 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.333 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.333 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_compat_step": { 2024-09-11T22:07:27.333 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crush_compat_step", 2024-09-11T22:07:27.333 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-09-11T22:07:27.333 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.333 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.334 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0.5", 2024-09-11T22:07:27.334 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "0.001", 2024-09-11T22:07:27.334 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "0.999", 2024-09-11T22:07:27.334 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.334 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "aggressiveness of optimization", 2024-09-11T22:07:27.334 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-11T22:07:27.334 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.334 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.335 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.335 INFO:teuthology.orchestra.run.smithi001.stdout: "end_time": { 2024-09-11T22:07:27.335 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "end_time", 2024-09-11T22:07:27.335 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.335 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.335 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.335 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "2359", 2024-09-11T22:07:27.335 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.336 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.336 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.336 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "ending time of day to automatically balance", 2024-09-11T22:07:27.336 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-11T22:07:27.336 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.336 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.336 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.336 INFO:teuthology.orchestra.run.smithi001.stdout: "end_weekday": { 2024-09-11T22:07:27.337 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "end_weekday", 2024-09-11T22:07:27.337 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-09-11T22:07:27.337 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.337 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.337 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-09-11T22:07:27.337 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "0", 2024-09-11T22:07:27.337 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "6", 2024-09-11T22:07:27.337 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.338 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-11T22:07:27.338 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-11T22:07:27.338 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.338 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.338 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.338 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:27.338 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:27.338 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.339 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.339 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.339 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.339 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.339 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.339 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.339 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.339 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.340 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.340 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.340 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.340 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.340 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.340 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.340 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.341 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.341 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.341 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.341 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:27.341 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:27.341 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.341 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.341 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.342 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.342 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.342 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.342 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.342 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.342 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.342 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.342 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.343 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.343 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:27.343 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:27.343 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.343 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.343 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.343 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:27.344 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.344 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.344 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.344 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.344 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.344 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.344 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.345 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.345 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.345 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.345 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.345 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.345 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.345 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.345 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.346 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:27.346 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:27.346 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.346 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.346 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.346 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.346 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.346 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.347 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.347 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.347 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.347 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.347 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.347 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.347 INFO:teuthology.orchestra.run.smithi001.stdout: "min_score": { 2024-09-11T22:07:27.347 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "min_score", 2024-09-11T22:07:27.348 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-09-11T22:07:27.348 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.348 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.348 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-09-11T22:07:27.348 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.348 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.348 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.349 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-11T22:07:27.349 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.349 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.349 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.349 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.349 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": { 2024-09-11T22:07:27.349 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mode", 2024-09-11T22:07:27.349 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.350 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.350 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.350 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "upmap", 2024-09-11T22:07:27.350 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.350 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.350 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.350 INFO:teuthology.orchestra.run.smithi001.stdout: "crush-compat", 2024-09-11T22:07:27.350 INFO:teuthology.orchestra.run.smithi001.stdout: "none", 2024-09-11T22:07:27.350 INFO:teuthology.orchestra.run.smithi001.stdout: "read", 2024-09-11T22:07:27.351 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap", 2024-09-11T22:07:27.351 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap-read" 2024-09-11T22:07:27.351 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.351 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Balancer mode", 2024-09-11T22:07:27.351 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.351 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.351 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.351 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.352 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_ids": { 2024-09-11T22:07:27.352 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pool_ids", 2024-09-11T22:07:27.352 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.352 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.352 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.352 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.352 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.352 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.352 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.353 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-11T22:07:27.353 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.353 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.353 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.353 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.353 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-09-11T22:07:27.353 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-09-11T22:07:27.353 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-11T22:07:27.354 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.354 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.354 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-09-11T22:07:27.354 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.354 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.354 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.354 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-11T22:07:27.354 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.355 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.355 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.355 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.355 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap_max_deviation": { 2024-09-11T22:07:27.355 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "upmap_max_deviation", 2024-09-11T22:07:27.355 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:27.355 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.355 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.355 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-09-11T22:07:27.356 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-09-11T22:07:27.356 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.356 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.356 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-11T22:07:27.356 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-11T22:07:27.356 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.356 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.357 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.357 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap_max_optimizations": { 2024-09-11T22:07:27.357 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "upmap_max_optimizations", 2024-09-11T22:07:27.357 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-09-11T22:07:27.357 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.357 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.357 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-11T22:07:27.357 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.357 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.358 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.358 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-11T22:07:27.358 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.358 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.358 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.358 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.358 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.358 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.359 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:27.359 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cephadm", 2024-09-11T22:07:27.359 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:27.359 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:27.359 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:27.359 INFO:teuthology.orchestra.run.smithi001.stdout: "agent_down_multiplier": { 2024-09-11T22:07:27.359 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "agent_down_multiplier", 2024-09-11T22:07:27.359 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-09-11T22:07:27.360 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.360 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.360 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "3.0", 2024-09-11T22:07:27.360 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.360 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.360 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.360 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-11T22:07:27.360 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.361 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.361 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.361 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.361 INFO:teuthology.orchestra.run.smithi001.stdout: "agent_refresh_rate": { 2024-09-11T22:07:27.361 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "agent_refresh_rate", 2024-09-11T22:07:27.361 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-11T22:07:27.361 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.361 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.362 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "20", 2024-09-11T22:07:27.362 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.362 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.362 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.362 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-11T22:07:27.362 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.362 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.362 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.362 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.363 INFO:teuthology.orchestra.run.smithi001.stdout: "agent_starting_port": { 2024-09-11T22:07:27.363 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "agent_starting_port", 2024-09-11T22:07:27.363 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:27.363 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.363 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.363 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "4721", 2024-09-11T22:07:27.363 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.363 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.364 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.364 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-11T22:07:27.364 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.364 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.364 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.364 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.364 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_ptrace": { 2024-09-11T22:07:27.364 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "allow_ptrace", 2024-09-11T22:07:27.365 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.365 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.365 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.365 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.365 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.365 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.365 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.366 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-11T22:07:27.366 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-11T22:07:27.366 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.366 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.366 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.366 INFO:teuthology.orchestra.run.smithi001.stdout: "autotune_interval": { 2024-09-11T22:07:27.366 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "autotune_interval", 2024-09-11T22:07:27.366 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-11T22:07:27.366 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.367 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.367 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-09-11T22:07:27.367 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.367 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.367 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.367 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-11T22:07:27.367 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.368 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.368 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.368 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.368 INFO:teuthology.orchestra.run.smithi001.stdout: "autotune_memory_target_ratio": { 2024-09-11T22:07:27.368 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "autotune_memory_target_ratio", 2024-09-11T22:07:27.368 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-09-11T22:07:27.368 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.368 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.369 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0.7", 2024-09-11T22:07:27.369 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.369 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.369 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.369 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-11T22:07:27.369 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.369 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.369 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.370 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.370 INFO:teuthology.orchestra.run.smithi001.stdout: "cephadm_log_destination": { 2024-09-11T22:07:27.370 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cephadm_log_destination", 2024-09-11T22:07:27.370 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.370 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.370 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.370 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.370 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.371 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.371 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.371 INFO:teuthology.orchestra.run.smithi001.stdout: "file", 2024-09-11T22:07:27.371 INFO:teuthology.orchestra.run.smithi001.stdout: "file,syslog", 2024-09-11T22:07:27.371 INFO:teuthology.orchestra.run.smithi001.stdout: "syslog" 2024-09-11T22:07:27.371 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.371 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-11T22:07:27.371 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.372 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.372 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.372 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.372 INFO:teuthology.orchestra.run.smithi001.stdout: "cgroups_split": { 2024-09-11T22:07:27.372 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cgroups_split", 2024-09-11T22:07:27.372 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.372 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.372 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.373 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:27.373 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.373 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.373 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.373 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-11T22:07:27.373 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.373 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.373 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.374 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.374 INFO:teuthology.orchestra.run.smithi001.stdout: "config_checks_enabled": { 2024-09-11T22:07:27.374 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "config_checks_enabled", 2024-09-11T22:07:27.374 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.374 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.374 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.374 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.374 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.375 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.375 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.375 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-11T22:07:27.375 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.375 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.375 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.375 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.375 INFO:teuthology.orchestra.run.smithi001.stdout: "config_dashboard": { 2024-09-11T22:07:27.375 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "config_dashboard", 2024-09-11T22:07:27.376 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.376 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.376 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.376 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:27.376 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.376 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.376 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.376 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-11T22:07:27.377 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.377 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.377 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.377 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.377 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_alertmanager": { 2024-09-11T22:07:27.377 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_alertmanager", 2024-09-11T22:07:27.377 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.377 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.378 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.378 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-11T22:07:27.378 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.378 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.378 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.378 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-09-11T22:07:27.378 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.378 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.379 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.379 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.379 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_base": { 2024-09-11T22:07:27.379 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_base", 2024-09-11T22:07:27.379 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.379 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.379 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.379 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-11T22:07:27.380 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.380 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.380 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.380 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Container image name, without the tag", 2024-09-11T22:07:27.380 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.380 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.380 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.380 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.381 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_elasticsearch": { 2024-09-11T22:07:27.381 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_elasticsearch", 2024-09-11T22:07:27.381 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.381 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.381 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.381 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-11T22:07:27.381 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.381 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.382 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.382 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "elasticsearch container image", 2024-09-11T22:07:27.382 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.382 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.382 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.382 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.382 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_grafana": { 2024-09-11T22:07:27.382 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_grafana", 2024-09-11T22:07:27.383 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.383 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.383 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.383 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-11T22:07:27.383 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.383 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.383 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.383 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-09-11T22:07:27.384 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.384 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.384 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.384 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.384 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_haproxy": { 2024-09-11T22:07:27.384 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_haproxy", 2024-09-11T22:07:27.384 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.384 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.385 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.385 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-11T22:07:27.385 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.385 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.385 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.385 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "HAproxy container image", 2024-09-11T22:07:27.385 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.385 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.386 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.386 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.386 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_jaeger_agent": { 2024-09-11T22:07:27.386 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_jaeger_agent", 2024-09-11T22:07:27.386 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.386 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.386 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.387 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-11T22:07:27.387 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.387 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.387 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.387 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Jaeger agent container image", 2024-09-11T22:07:27.387 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.387 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.387 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.388 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.388 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_jaeger_collector": { 2024-09-11T22:07:27.388 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_jaeger_collector", 2024-09-11T22:07:27.388 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.388 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.388 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.388 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-11T22:07:27.388 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.389 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.389 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.389 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Jaeger collector container image", 2024-09-11T22:07:27.389 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.389 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.389 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.389 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.389 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_jaeger_query": { 2024-09-11T22:07:27.390 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_jaeger_query", 2024-09-11T22:07:27.390 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.390 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.390 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.390 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-11T22:07:27.390 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.390 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.390 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.390 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Jaeger query container image", 2024-09-11T22:07:27.391 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.391 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.391 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.391 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.391 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_keepalived": { 2024-09-11T22:07:27.391 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_keepalived", 2024-09-11T22:07:27.391 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.391 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.391 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.392 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-11T22:07:27.392 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.392 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.392 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.392 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Keepalived container image", 2024-09-11T22:07:27.392 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.392 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.392 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.393 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.393 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_loki": { 2024-09-11T22:07:27.393 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_loki", 2024-09-11T22:07:27.393 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.393 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.393 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.393 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-11T22:07:27.393 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.394 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.394 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.394 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Loki container image", 2024-09-11T22:07:27.394 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.394 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.394 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.394 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.394 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_node_exporter": { 2024-09-11T22:07:27.395 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_node_exporter", 2024-09-11T22:07:27.395 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.395 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.395 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.395 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-11T22:07:27.395 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.395 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.395 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.396 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-09-11T22:07:27.396 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.396 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.396 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.396 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.396 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_nvmeof": { 2024-09-11T22:07:27.396 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_nvmeof", 2024-09-11T22:07:27.396 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.397 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.397 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.397 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-11T22:07:27.397 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.397 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.397 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.397 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Nvme-of container image", 2024-09-11T22:07:27.397 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.398 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.398 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.398 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.398 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_prometheus": { 2024-09-11T22:07:27.398 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_prometheus", 2024-09-11T22:07:27.398 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.398 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.398 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.399 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-11T22:07:27.399 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.399 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.399 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.399 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-09-11T22:07:27.399 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.399 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.399 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.400 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.400 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_promtail": { 2024-09-11T22:07:27.400 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_promtail", 2024-09-11T22:07:27.400 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.400 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.400 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.400 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-11T22:07:27.400 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.401 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.401 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.401 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Promtail container image", 2024-09-11T22:07:27.401 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.401 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.401 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.401 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.402 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_samba": { 2024-09-11T22:07:27.402 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_samba", 2024-09-11T22:07:27.402 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.402 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.402 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.402 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-11T22:07:27.402 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.402 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.403 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.403 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Samba/SMB container image", 2024-09-11T22:07:27.403 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.403 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.403 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.403 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.403 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_snmp_gateway": { 2024-09-11T22:07:27.403 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_snmp_gateway", 2024-09-11T22:07:27.404 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.404 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.404 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.404 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-11T22:07:27.404 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.404 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.404 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.404 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SNMP Gateway container image", 2024-09-11T22:07:27.405 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.405 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.405 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.405 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.405 INFO:teuthology.orchestra.run.smithi001.stdout: "container_init": { 2024-09-11T22:07:27.405 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_init", 2024-09-11T22:07:27.405 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.405 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.406 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.406 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:27.406 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.406 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.406 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.406 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Run podman/docker with `--init`", 2024-09-11T22:07:27.406 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.406 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.407 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.407 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.407 INFO:teuthology.orchestra.run.smithi001.stdout: "daemon_cache_timeout": { 2024-09-11T22:07:27.407 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "daemon_cache_timeout", 2024-09-11T22:07:27.407 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-11T22:07:27.407 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.407 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.408 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-09-11T22:07:27.408 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.408 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.408 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.408 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-11T22:07:27.408 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.408 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.408 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.409 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.409 INFO:teuthology.orchestra.run.smithi001.stdout: "default_cephadm_command_timeout": { 2024-09-11T22:07:27.409 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "default_cephadm_command_timeout", 2024-09-11T22:07:27.409 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:27.409 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.409 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.409 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "900", 2024-09-11T22:07:27.410 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.410 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.410 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.410 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-11T22:07:27.410 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.410 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.410 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.410 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.411 INFO:teuthology.orchestra.run.smithi001.stdout: "default_registry": { 2024-09-11T22:07:27.411 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "default_registry", 2024-09-11T22:07:27.411 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.411 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.411 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.411 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io", 2024-09-11T22:07:27.411 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.411 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.412 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.412 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-11T22:07:27.412 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.412 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.412 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.412 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.412 INFO:teuthology.orchestra.run.smithi001.stdout: "device_cache_timeout": { 2024-09-11T22:07:27.412 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "device_cache_timeout", 2024-09-11T22:07:27.413 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-11T22:07:27.413 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.413 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.413 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "1800", 2024-09-11T22:07:27.413 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.413 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.413 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.413 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "seconds to cache device inventory", 2024-09-11T22:07:27.414 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.414 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.414 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.414 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.414 INFO:teuthology.orchestra.run.smithi001.stdout: "device_enhanced_scan": { 2024-09-11T22:07:27.414 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "device_enhanced_scan", 2024-09-11T22:07:27.414 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.414 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.414 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.415 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.415 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.415 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.415 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.415 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-11T22:07:27.415 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.415 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.415 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.415 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.416 INFO:teuthology.orchestra.run.smithi001.stdout: "facts_cache_timeout": { 2024-09-11T22:07:27.416 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "facts_cache_timeout", 2024-09-11T22:07:27.416 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-11T22:07:27.416 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.416 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.416 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-09-11T22:07:27.416 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.416 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.417 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.417 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "seconds to cache host facts data", 2024-09-11T22:07:27.417 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.417 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.417 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.417 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.417 INFO:teuthology.orchestra.run.smithi001.stdout: "grafana_dashboards_path": { 2024-09-11T22:07:27.417 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "grafana_dashboards_path", 2024-09-11T22:07:27.418 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.418 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.418 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.418 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-11T22:07:27.418 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.418 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.418 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.418 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-11T22:07:27.419 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.419 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.419 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.419 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.419 INFO:teuthology.orchestra.run.smithi001.stdout: "host_check_interval": { 2024-09-11T22:07:27.419 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "host_check_interval", 2024-09-11T22:07:27.419 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-11T22:07:27.419 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.420 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.420 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-09-11T22:07:27.420 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.420 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.420 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.420 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to perform a host check", 2024-09-11T22:07:27.420 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.421 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.421 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.421 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.421 INFO:teuthology.orchestra.run.smithi001.stdout: "hw_monitoring": { 2024-09-11T22:07:27.421 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "hw_monitoring", 2024-09-11T22:07:27.421 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.421 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.421 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.422 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.422 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.422 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.422 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.422 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-11T22:07:27.422 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.422 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.422 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.422 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.423 INFO:teuthology.orchestra.run.smithi001.stdout: "inventory_list_all": { 2024-09-11T22:07:27.423 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "inventory_list_all", 2024-09-11T22:07:27.423 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.423 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.423 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.423 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.423 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.423 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.424 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.424 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-11T22:07:27.424 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.424 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.424 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.424 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.424 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:27.425 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:27.425 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.425 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.425 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.425 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.425 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.425 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.425 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.426 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.426 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.426 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.426 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.426 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.426 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.426 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.426 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.427 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.427 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.427 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.427 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.427 INFO:teuthology.orchestra.run.smithi001.stdout: "log_refresh_metadata": { 2024-09-11T22:07:27.427 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_refresh_metadata", 2024-09-11T22:07:27.427 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.427 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.428 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.428 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.428 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.428 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.428 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.428 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-11T22:07:27.428 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.429 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.429 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.429 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.429 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:27.429 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:27.429 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.429 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.429 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.430 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:27.430 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.430 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.430 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.430 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-11T22:07:27.430 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.430 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.431 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.431 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.431 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:27.431 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:27.431 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.431 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.431 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.431 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:27.432 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.432 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.432 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.432 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.432 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.432 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.432 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.433 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.433 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.433 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.433 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.433 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.433 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.433 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.433 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.434 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:27.434 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:27.434 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.434 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.434 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.434 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.434 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.435 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.435 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.435 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.435 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.435 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.435 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.435 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.435 INFO:teuthology.orchestra.run.smithi001.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-11T22:07:27.436 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-11T22:07:27.436 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.436 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.436 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.436 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.436 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.436 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.436 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.437 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-11T22:07:27.437 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.437 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.437 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.437 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.437 INFO:teuthology.orchestra.run.smithi001.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-11T22:07:27.437 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-11T22:07:27.437 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.438 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.438 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.438 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "*", 2024-09-11T22:07:27.438 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.438 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.438 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.438 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-11T22:07:27.439 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.439 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.439 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.439 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.439 INFO:teuthology.orchestra.run.smithi001.stdout: "max_count_per_host": { 2024-09-11T22:07:27.439 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_count_per_host", 2024-09-11T22:07:27.439 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:27.439 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.440 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.440 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-11T22:07:27.440 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.440 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.440 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.440 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "max number of daemons per service per host", 2024-09-11T22:07:27.440 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.440 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.441 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.441 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.441 INFO:teuthology.orchestra.run.smithi001.stdout: "max_osd_draining_count": { 2024-09-11T22:07:27.441 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_osd_draining_count", 2024-09-11T22:07:27.441 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:27.441 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.441 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.441 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-11T22:07:27.442 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.442 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.442 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.442 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-11T22:07:27.442 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.442 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.442 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.442 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.442 INFO:teuthology.orchestra.run.smithi001.stdout: "migration_current": { 2024-09-11T22:07:27.443 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "migration_current", 2024-09-11T22:07:27.443 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:27.443 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.443 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.443 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.443 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.443 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.444 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.444 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "internal - do not modify", 2024-09-11T22:07:27.444 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.444 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.444 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.444 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.444 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": { 2024-09-11T22:07:27.444 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mode", 2024-09-11T22:07:27.445 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.445 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.445 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.445 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "root", 2024-09-11T22:07:27.445 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.445 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.445 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.445 INFO:teuthology.orchestra.run.smithi001.stdout: "cephadm-package", 2024-09-11T22:07:27.446 INFO:teuthology.orchestra.run.smithi001.stdout: "root" 2024-09-11T22:07:27.446 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.446 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "mode for remote execution of cephadm", 2024-09-11T22:07:27.446 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.446 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.446 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.446 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.446 INFO:teuthology.orchestra.run.smithi001.stdout: "oob_default_addr": { 2024-09-11T22:07:27.447 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "oob_default_addr", 2024-09-11T22:07:27.447 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.447 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.447 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.447 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "169.254.1.1", 2024-09-11T22:07:27.447 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.447 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.447 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.448 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-11T22:07:27.448 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.448 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.448 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.448 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.448 INFO:teuthology.orchestra.run.smithi001.stdout: "prometheus_alerts_path": { 2024-09-11T22:07:27.448 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "prometheus_alerts_path", 2024-09-11T22:07:27.448 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.449 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.449 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.449 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-11T22:07:27.449 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.449 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.449 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.449 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-11T22:07:27.449 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.450 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.450 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.450 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.450 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_insecure": { 2024-09-11T22:07:27.450 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_insecure", 2024-09-11T22:07:27.450 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.450 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.450 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.451 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.451 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.451 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.451 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.451 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-11T22:07:27.451 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.451 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.451 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.452 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.452 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_password": { 2024-09-11T22:07:27.452 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_password", 2024-09-11T22:07:27.452 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.452 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.452 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.452 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.453 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.453 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.453 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.453 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-11T22:07:27.453 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.453 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.453 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.453 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.454 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_url": { 2024-09-11T22:07:27.454 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_url", 2024-09-11T22:07:27.454 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.454 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.454 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.454 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.454 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.454 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.455 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.455 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-11T22:07:27.455 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.455 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.455 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.455 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.455 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_username": { 2024-09-11T22:07:27.455 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_username", 2024-09-11T22:07:27.456 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.456 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.456 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.456 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.456 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.456 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.456 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.456 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-11T22:07:27.457 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.457 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.457 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.457 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.457 INFO:teuthology.orchestra.run.smithi001.stdout: "secure_monitoring_stack": { 2024-09-11T22:07:27.457 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "secure_monitoring_stack", 2024-09-11T22:07:27.457 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.457 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.458 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.458 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.458 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.458 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.458 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.458 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-11T22:07:27.458 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.458 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.458 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.459 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.459 INFO:teuthology.orchestra.run.smithi001.stdout: "service_discovery_port": { 2024-09-11T22:07:27.459 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "service_discovery_port", 2024-09-11T22:07:27.459 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:27.459 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.459 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.459 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8765", 2024-09-11T22:07:27.459 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.460 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.460 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.460 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "cephadm service discovery port", 2024-09-11T22:07:27.460 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.460 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.460 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.460 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.460 INFO:teuthology.orchestra.run.smithi001.stdout: "ssh_config_file": { 2024-09-11T22:07:27.461 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssh_config_file", 2024-09-11T22:07:27.461 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.461 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.461 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.461 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.461 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.461 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.461 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.462 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-11T22:07:27.462 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.462 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.462 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.462 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.462 INFO:teuthology.orchestra.run.smithi001.stdout: "use_agent": { 2024-09-11T22:07:27.462 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "use_agent", 2024-09-11T22:07:27.462 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.463 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.463 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.463 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.463 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.463 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.463 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.463 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-11T22:07:27.463 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.464 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.464 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.464 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.464 INFO:teuthology.orchestra.run.smithi001.stdout: "use_repo_digest": { 2024-09-11T22:07:27.464 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "use_repo_digest", 2024-09-11T22:07:27.464 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.464 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.464 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.465 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:27.465 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.465 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.465 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.465 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-11T22:07:27.465 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.465 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.465 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.466 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.466 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_on_failed_host_check": { 2024-09-11T22:07:27.466 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_on_failed_host_check", 2024-09-11T22:07:27.466 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.466 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.466 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.466 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:27.467 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.467 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.467 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.467 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise a health warning if the host check fails", 2024-09-11T22:07:27.467 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.467 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.467 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.467 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.468 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_on_stray_daemons": { 2024-09-11T22:07:27.468 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_on_stray_daemons", 2024-09-11T22:07:27.468 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.468 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.468 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.468 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:27.468 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.468 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.469 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.469 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-11T22:07:27.469 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.469 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.469 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.469 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.469 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_on_stray_hosts": { 2024-09-11T22:07:27.469 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_on_stray_hosts", 2024-09-11T22:07:27.470 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.470 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.470 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.470 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:27.470 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.470 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.470 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.470 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-11T22:07:27.471 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.471 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.471 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.471 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.471 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.471 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.471 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:27.471 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crash", 2024-09-11T22:07:27.472 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:27.472 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:27.472 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:27.472 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:27.472 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:27.472 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.472 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.472 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.473 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.473 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.473 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.473 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.473 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.473 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.473 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.473 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.474 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.474 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.474 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.474 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.474 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.474 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.474 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.474 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.475 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:27.475 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:27.475 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.475 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.475 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.475 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.475 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.475 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.476 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.476 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.476 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.476 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.476 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.476 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.476 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:27.476 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:27.476 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.477 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.477 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.477 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:27.477 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.477 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.477 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.477 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.477 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.478 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.478 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.478 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.478 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.478 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.478 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.478 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.479 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.479 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.479 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.479 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:27.479 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:27.479 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.479 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.479 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.480 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.480 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.480 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.480 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.480 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.480 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.480 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.480 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.481 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.481 INFO:teuthology.orchestra.run.smithi001.stdout: "retain_interval": { 2024-09-11T22:07:27.481 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "retain_interval", 2024-09-11T22:07:27.481 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-11T22:07:27.481 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.481 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.481 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "31536000", 2024-09-11T22:07:27.481 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.482 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.482 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.482 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-11T22:07:27.482 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.482 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.482 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.482 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.482 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_recent_interval": { 2024-09-11T22:07:27.483 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_recent_interval", 2024-09-11T22:07:27.483 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-11T22:07:27.483 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.483 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.483 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "1209600", 2024-09-11T22:07:27.483 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.483 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.483 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.484 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-11T22:07:27.484 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.484 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.484 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.484 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.484 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.484 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.484 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:27.485 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "dashboard", 2024-09-11T22:07:27.485 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:27.485 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:27.485 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:27.485 INFO:teuthology.orchestra.run.smithi001.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-11T22:07:27.485 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-11T22:07:27.485 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:27.485 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.486 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.486 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-11T22:07:27.486 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.486 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.486 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.486 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.486 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.486 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.487 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.487 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.487 INFO:teuthology.orchestra.run.smithi001.stdout: "ALERTMANAGER_API_HOST": { 2024-09-11T22:07:27.487 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-11T22:07:27.487 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.487 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.487 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.487 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.487 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.488 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.488 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.488 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.488 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.488 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.488 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.488 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.488 INFO:teuthology.orchestra.run.smithi001.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-11T22:07:27.489 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-11T22:07:27.489 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.489 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.489 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.489 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:27.489 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.489 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.490 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.490 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.490 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.490 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.490 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.490 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.490 INFO:teuthology.orchestra.run.smithi001.stdout: "AUDIT_API_ENABLED": { 2024-09-11T22:07:27.490 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "AUDIT_API_ENABLED", 2024-09-11T22:07:27.491 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.491 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.491 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.491 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.491 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.491 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.491 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.491 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.491 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.492 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.492 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.492 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.492 INFO:teuthology.orchestra.run.smithi001.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-11T22:07:27.492 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-11T22:07:27.492 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.492 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.492 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.493 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:27.493 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.493 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.493 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.493 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.493 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.493 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.494 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.494 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.494 INFO:teuthology.orchestra.run.smithi001.stdout: "ENABLE_BROWSABLE_API": { 2024-09-11T22:07:27.494 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-11T22:07:27.494 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.494 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.494 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.495 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:27.495 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.495 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.495 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.495 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.495 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.495 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.495 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.496 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.496 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-11T22:07:27.496 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-11T22:07:27.496 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.496 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.496 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.496 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:27.496 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.497 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.497 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.497 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.497 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.497 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.497 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.498 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.498 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-11T22:07:27.498 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-11T22:07:27.498 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.498 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.498 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.498 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:27.498 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.498 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.499 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.499 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.499 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.499 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.499 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.499 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.499 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-11T22:07:27.499 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-11T22:07:27.500 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.500 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.500 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.500 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:27.500 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.500 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.500 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.501 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.501 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.501 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.501 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.501 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.501 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-11T22:07:27.501 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-11T22:07:27.501 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.501 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.502 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.502 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:27.502 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.502 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.502 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.502 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.502 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.502 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.503 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.503 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.503 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-11T22:07:27.503 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-11T22:07:27.503 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.503 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.503 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.504 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:27.504 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.504 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.504 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.504 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.504 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.504 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.504 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.505 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.505 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-11T22:07:27.505 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-11T22:07:27.505 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.505 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.505 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.505 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:27.505 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.505 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.506 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.506 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.506 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.506 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.506 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.506 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.506 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-11T22:07:27.506 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-11T22:07:27.506 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.507 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.507 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.507 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:27.507 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.507 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.507 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.507 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.507 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.508 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.508 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.508 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.508 INFO:teuthology.orchestra.run.smithi001.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-11T22:07:27.508 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-11T22:07:27.508 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.508 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.509 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.509 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.509 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.509 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.509 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.509 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.509 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.509 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.510 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.510 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.510 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_PASSWORD": { 2024-09-11T22:07:27.510 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-11T22:07:27.510 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.510 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.510 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.510 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2024-09-11T22:07:27.511 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.511 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.511 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.511 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.511 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.511 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.511 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.511 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.512 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-11T22:07:27.512 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-11T22:07:27.512 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.512 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.512 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.512 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:27.512 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.512 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.513 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.513 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.513 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.513 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.513 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.513 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.513 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_URL": { 2024-09-11T22:07:27.513 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_URL", 2024-09-11T22:07:27.514 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.514 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.514 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.514 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.514 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.514 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.514 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.514 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.515 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.515 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.515 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.515 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.515 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_USERNAME": { 2024-09-11T22:07:27.515 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-11T22:07:27.515 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.515 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.516 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.516 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2024-09-11T22:07:27.516 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.516 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.516 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.516 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.516 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.516 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.517 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.517 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.517 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-11T22:07:27.517 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-11T22:07:27.517 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.517 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.517 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.517 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.518 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.518 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.518 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.518 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.518 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.518 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.518 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.518 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.519 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-11T22:07:27.519 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-11T22:07:27.519 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.519 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.519 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.519 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.519 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.519 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.519 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.520 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.520 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.520 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.520 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.520 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.520 INFO:teuthology.orchestra.run.smithi001.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-11T22:07:27.520 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-11T22:07:27.521 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.521 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.521 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.521 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:27.521 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.521 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.521 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.521 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.522 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.522 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.522 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.522 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.522 INFO:teuthology.orchestra.run.smithi001.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-11T22:07:27.522 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-11T22:07:27.522 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.522 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.523 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.523 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.523 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.523 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.523 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.523 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.523 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.523 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.524 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.524 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.524 INFO:teuthology.orchestra.run.smithi001.stdout: "PROMETHEUS_API_HOST": { 2024-09-11T22:07:27.524 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-11T22:07:27.524 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.524 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.524 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.525 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.525 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.525 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.525 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.525 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.525 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.525 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.525 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.526 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.526 INFO:teuthology.orchestra.run.smithi001.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-11T22:07:27.526 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-11T22:07:27.526 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.526 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.526 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.526 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:27.526 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.527 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.527 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.527 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.527 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.527 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.527 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.527 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.528 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-11T22:07:27.528 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-11T22:07:27.528 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.528 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.528 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.528 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.528 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.528 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.529 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.529 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.529 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.529 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.529 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.529 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.529 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-11T22:07:27.530 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-11T22:07:27.530 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.530 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.530 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.530 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.530 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.530 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.530 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.531 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.531 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.531 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.531 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.531 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.531 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-11T22:07:27.531 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-11T22:07:27.531 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.532 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.532 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.532 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:27.532 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.532 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.532 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.532 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.532 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.532 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.533 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.533 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.533 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-11T22:07:27.533 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-11T22:07:27.533 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.533 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.533 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.533 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:27.534 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.534 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.534 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.534 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.534 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.534 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.534 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.534 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.535 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-11T22:07:27.535 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-11T22:07:27.535 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.535 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.535 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.535 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.535 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.535 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.536 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.536 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.536 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.536 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.536 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.536 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.536 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-11T22:07:27.536 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-11T22:07:27.537 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.537 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.537 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.537 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.537 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.537 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.537 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.537 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.538 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.538 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.538 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.538 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.538 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-11T22:07:27.538 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-11T22:07:27.538 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.538 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.538 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.539 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.539 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.539 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.539 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.539 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.539 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.539 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.539 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.540 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.540 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_ENABLED": { 2024-09-11T22:07:27.540 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-11T22:07:27.540 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.540 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.540 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.540 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:27.540 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.541 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.541 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.541 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.541 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.541 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.541 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.541 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.541 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-11T22:07:27.542 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-11T22:07:27.542 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.542 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.542 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.542 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-11T22:07:27.542 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.542 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.542 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.543 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.543 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.543 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.543 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.543 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.543 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-11T22:07:27.543 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-11T22:07:27.544 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:27.544 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.544 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.544 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-11T22:07:27.544 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.544 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.544 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.544 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.545 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.545 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.545 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.545 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.545 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-11T22:07:27.545 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-11T22:07:27.545 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:27.545 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.546 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.546 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8", 2024-09-11T22:07:27.546 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.546 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.546 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.546 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.546 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.547 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.547 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.547 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.547 INFO:teuthology.orchestra.run.smithi001.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-11T22:07:27.547 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-11T22:07:27.547 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:27.547 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.547 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.548 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "45", 2024-09-11T22:07:27.548 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.548 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.548 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.548 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.548 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.548 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.548 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.549 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.549 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_ACCESS_KEY": { 2024-09-11T22:07:27.549 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-11T22:07:27.549 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.549 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.549 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.549 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.550 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.550 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.550 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.550 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.550 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.550 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.550 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.550 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.550 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-11T22:07:27.551 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-11T22:07:27.551 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.551 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.551 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.551 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2024-09-11T22:07:27.551 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.551 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.552 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.552 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.552 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.552 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.552 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.552 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.552 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_SECRET_KEY": { 2024-09-11T22:07:27.552 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-11T22:07:27.553 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.553 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.553 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.553 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.553 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.553 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.553 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.554 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.554 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.554 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.554 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.554 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.554 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_SSL_VERIFY": { 2024-09-11T22:07:27.554 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-11T22:07:27.554 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.555 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.555 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.555 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:27.555 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.555 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.555 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.555 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.555 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.556 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.556 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.556 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.556 INFO:teuthology.orchestra.run.smithi001.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-11T22:07:27.556 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-11T22:07:27.556 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:27.556 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.556 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.557 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-09-11T22:07:27.557 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.557 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.557 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.557 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.557 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.557 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.557 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.558 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.558 INFO:teuthology.orchestra.run.smithi001.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-11T22:07:27.558 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-11T22:07:27.558 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:27.558 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.558 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.558 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-11T22:07:27.558 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.559 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.559 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.559 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.559 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.559 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.559 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.559 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.559 INFO:teuthology.orchestra.run.smithi001.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-11T22:07:27.560 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-11T22:07:27.560 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:27.560 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.560 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.560 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-09-11T22:07:27.560 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.560 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.560 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.561 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.561 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.561 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.561 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.561 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.561 INFO:teuthology.orchestra.run.smithi001.stdout: "cross_origin_url": { 2024-09-11T22:07:27.561 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cross_origin_url", 2024-09-11T22:07:27.561 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.562 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.562 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.562 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.562 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.562 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.562 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.562 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.562 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.563 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.563 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.563 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.563 INFO:teuthology.orchestra.run.smithi001.stdout: "crt_file": { 2024-09-11T22:07:27.563 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crt_file", 2024-09-11T22:07:27.563 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.563 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.563 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.564 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.564 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.564 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.564 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.564 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.564 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.564 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.564 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.565 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.565 INFO:teuthology.orchestra.run.smithi001.stdout: "debug": { 2024-09-11T22:07:27.565 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "debug", 2024-09-11T22:07:27.565 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.565 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.565 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.565 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.565 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.566 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.566 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.566 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Enable/disable debug options", 2024-09-11T22:07:27.566 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.566 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.566 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.566 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.566 INFO:teuthology.orchestra.run.smithi001.stdout: "jwt_token_ttl": { 2024-09-11T22:07:27.567 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "jwt_token_ttl", 2024-09-11T22:07:27.567 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:27.567 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.567 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.567 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "28800", 2024-09-11T22:07:27.567 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.567 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.567 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.568 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.568 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.568 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.568 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.568 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.568 INFO:teuthology.orchestra.run.smithi001.stdout: "key_file": { 2024-09-11T22:07:27.568 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "key_file", 2024-09-11T22:07:27.568 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.569 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.569 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.569 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.569 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.569 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.569 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.569 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.570 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.570 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.570 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.570 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.570 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:27.570 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:27.570 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.570 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.570 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.571 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.571 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.571 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.571 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.571 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.571 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.571 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.571 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.572 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.572 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.572 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.572 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.572 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.572 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.572 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.572 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.573 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:27.573 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:27.573 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.573 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.573 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.573 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.573 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.573 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.574 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.574 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.574 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.574 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.574 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.574 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.574 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:27.574 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:27.575 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.575 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.575 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.575 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:27.575 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.575 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.575 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.575 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.576 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.576 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.576 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.576 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.576 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.576 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.576 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.576 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.577 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.577 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.577 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.577 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:27.577 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:27.577 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.577 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.577 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.578 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.578 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.578 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.578 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.578 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.578 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.578 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.579 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.579 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.579 INFO:teuthology.orchestra.run.smithi001.stdout: "motd": { 2024-09-11T22:07:27.579 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "motd", 2024-09-11T22:07:27.579 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.579 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.579 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.579 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.580 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.580 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.580 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.580 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "The message of the day", 2024-09-11T22:07:27.580 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.580 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.580 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.580 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.581 INFO:teuthology.orchestra.run.smithi001.stdout: "redirect_resolve_ip_addr": { 2024-09-11T22:07:27.581 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "redirect_resolve_ip_addr", 2024-09-11T22:07:27.581 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.581 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.581 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.581 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.581 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.581 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.582 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.582 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.582 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.582 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.582 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.582 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.582 INFO:teuthology.orchestra.run.smithi001.stdout: "server_addr": { 2024-09-11T22:07:27.583 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_addr", 2024-09-11T22:07:27.583 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.583 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.583 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.583 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "::", 2024-09-11T22:07:27.583 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.583 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.583 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.584 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.584 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.584 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.584 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.584 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.584 INFO:teuthology.orchestra.run.smithi001.stdout: "server_port": { 2024-09-11T22:07:27.584 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_port", 2024-09-11T22:07:27.584 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:27.585 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.585 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.585 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8080", 2024-09-11T22:07:27.585 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.585 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.585 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.585 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.585 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.586 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.586 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.586 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.586 INFO:teuthology.orchestra.run.smithi001.stdout: "ssl": { 2024-09-11T22:07:27.586 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssl", 2024-09-11T22:07:27.586 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.586 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.586 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.587 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:27.587 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.587 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.587 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.587 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.587 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.587 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.587 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.588 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.588 INFO:teuthology.orchestra.run.smithi001.stdout: "ssl_server_port": { 2024-09-11T22:07:27.588 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssl_server_port", 2024-09-11T22:07:27.588 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:27.588 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.588 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.588 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8443", 2024-09-11T22:07:27.589 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.589 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.589 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.589 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.589 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.589 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.589 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.589 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.589 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_behaviour": { 2024-09-11T22:07:27.590 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_behaviour", 2024-09-11T22:07:27.590 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.590 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.590 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.590 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "redirect", 2024-09-11T22:07:27.590 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.590 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.590 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.591 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.591 INFO:teuthology.orchestra.run.smithi001.stdout: "redirect" 2024-09-11T22:07:27.591 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.591 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.591 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.591 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.591 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.591 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.592 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_error_status_code": { 2024-09-11T22:07:27.592 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_error_status_code", 2024-09-11T22:07:27.592 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:27.592 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.592 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.592 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "500", 2024-09-11T22:07:27.592 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "400", 2024-09-11T22:07:27.592 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "599", 2024-09-11T22:07:27.592 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.593 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.593 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.593 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.593 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.593 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.593 INFO:teuthology.orchestra.run.smithi001.stdout: "url_prefix": { 2024-09-11T22:07:27.593 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "url_prefix", 2024-09-11T22:07:27.593 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.593 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.594 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.594 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.594 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.594 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.594 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.594 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.594 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.594 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.595 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.595 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.595 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.595 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.595 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:27.595 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "devicehealth", 2024-09-11T22:07:27.595 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:27.595 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:27.596 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:27.596 INFO:teuthology.orchestra.run.smithi001.stdout: "enable_monitoring": { 2024-09-11T22:07:27.596 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enable_monitoring", 2024-09-11T22:07:27.596 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.596 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.596 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.596 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:27.596 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.597 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.597 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.597 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "monitor device health metrics", 2024-09-11T22:07:27.597 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.597 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.597 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.597 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.597 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:27.598 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:27.598 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.598 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.598 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.598 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.598 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.598 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.598 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.598 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.599 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.599 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.599 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.599 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.599 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.599 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.599 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.599 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.600 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.600 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.600 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.600 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:27.600 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:27.600 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.600 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.601 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.601 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.601 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.601 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.601 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.601 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.601 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.601 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.602 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.602 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.602 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:27.602 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:27.602 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.602 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.602 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.602 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:27.603 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.603 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.603 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.603 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.603 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.603 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.603 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.604 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.604 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.604 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.604 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.604 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.604 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.604 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.605 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.605 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:27.605 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:27.605 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.605 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.605 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.605 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.605 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.605 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.606 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.606 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.606 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.606 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.606 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.606 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.606 INFO:teuthology.orchestra.run.smithi001.stdout: "mark_out_threshold": { 2024-09-11T22:07:27.606 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mark_out_threshold", 2024-09-11T22:07:27.607 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-11T22:07:27.607 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.607 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.607 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "2419200", 2024-09-11T22:07:27.607 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.607 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.607 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.607 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-11T22:07:27.607 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.608 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.608 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.608 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.608 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_name": { 2024-09-11T22:07:27.608 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pool_name", 2024-09-11T22:07:27.608 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.608 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.608 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.609 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "device_health_metrics", 2024-09-11T22:07:27.609 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.609 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.609 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.609 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-11T22:07:27.609 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.609 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.609 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.610 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.610 INFO:teuthology.orchestra.run.smithi001.stdout: "retention_period": { 2024-09-11T22:07:27.610 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "retention_period", 2024-09-11T22:07:27.610 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-11T22:07:27.610 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.610 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.610 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "15552000", 2024-09-11T22:07:27.611 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.611 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.611 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.611 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how long to retain device health metrics", 2024-09-11T22:07:27.611 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.611 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.611 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.612 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.612 INFO:teuthology.orchestra.run.smithi001.stdout: "scrape_frequency": { 2024-09-11T22:07:27.612 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "scrape_frequency", 2024-09-11T22:07:27.612 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-11T22:07:27.612 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.612 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.612 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "86400", 2024-09-11T22:07:27.612 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.613 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.613 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.613 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-11T22:07:27.613 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.613 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.613 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.613 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.614 INFO:teuthology.orchestra.run.smithi001.stdout: "self_heal": { 2024-09-11T22:07:27.614 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "self_heal", 2024-09-11T22:07:27.614 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.614 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.614 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.614 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:27.614 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.615 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.615 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.615 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-11T22:07:27.615 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.615 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.615 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.615 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.615 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-09-11T22:07:27.616 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-09-11T22:07:27.616 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-11T22:07:27.616 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.616 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.616 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-09-11T22:07:27.616 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.616 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.617 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.617 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to wake up and check device health", 2024-09-11T22:07:27.617 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.617 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.617 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.617 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.617 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_threshold": { 2024-09-11T22:07:27.618 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_threshold", 2024-09-11T22:07:27.618 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-11T22:07:27.618 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.618 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.618 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "7257600", 2024-09-11T22:07:27.618 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.618 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.619 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.619 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-11T22:07:27.619 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.619 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.619 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.619 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.619 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.619 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.620 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:27.620 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "diskprediction_local", 2024-09-11T22:07:27.620 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:27.620 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:27.620 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:27.620 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:27.620 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:27.620 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.620 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.621 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.621 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.621 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.621 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.621 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.621 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.621 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.621 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.622 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.622 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.622 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.622 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.622 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.622 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.622 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.622 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.622 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.623 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:27.623 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:27.623 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.623 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.623 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.623 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.623 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.623 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.624 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.624 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.624 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.624 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.624 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.624 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.624 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:27.624 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:27.624 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.625 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.625 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.625 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:27.625 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.625 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.625 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.625 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.625 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.626 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.626 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.626 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.626 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.626 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.626 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.626 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.626 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.626 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.627 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.627 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:27.627 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:27.627 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.627 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.627 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.627 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.627 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.628 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.628 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.628 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.628 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.628 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.628 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.628 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.628 INFO:teuthology.orchestra.run.smithi001.stdout: "predict_interval": { 2024-09-11T22:07:27.629 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "predict_interval", 2024-09-11T22:07:27.629 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.629 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.629 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.629 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "86400", 2024-09-11T22:07:27.629 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.629 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.629 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.630 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.630 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.630 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.630 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.630 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.630 INFO:teuthology.orchestra.run.smithi001.stdout: "predictor_model": { 2024-09-11T22:07:27.630 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "predictor_model", 2024-09-11T22:07:27.630 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.631 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.631 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.631 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "prophetstor", 2024-09-11T22:07:27.631 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.631 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.631 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.631 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.631 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.632 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.632 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.632 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.632 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-09-11T22:07:27.632 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-09-11T22:07:27.632 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.632 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.632 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.633 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-09-11T22:07:27.633 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.633 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.633 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.633 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.633 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.633 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.633 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.634 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.634 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.634 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.634 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:27.634 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "influx", 2024-09-11T22:07:27.634 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": false, 2024-09-11T22:07:27.634 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "influxdb python module not found", 2024-09-11T22:07:27.634 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:27.635 INFO:teuthology.orchestra.run.smithi001.stdout: "batch_size": { 2024-09-11T22:07:27.635 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "batch_size", 2024-09-11T22:07:27.635 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:27.635 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.635 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.635 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5000", 2024-09-11T22:07:27.635 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.635 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.635 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.636 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-11T22:07:27.636 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.636 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.636 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.636 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.636 INFO:teuthology.orchestra.run.smithi001.stdout: "database": { 2024-09-11T22:07:27.636 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "database", 2024-09-11T22:07:27.637 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.637 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.637 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.637 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "ceph", 2024-09-11T22:07:27.637 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.637 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.637 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.637 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-11T22:07:27.638 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.638 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.638 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.638 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.638 INFO:teuthology.orchestra.run.smithi001.stdout: "hostname": { 2024-09-11T22:07:27.638 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "hostname", 2024-09-11T22:07:27.638 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.638 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.638 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.639 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.639 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.639 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.639 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.639 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "InfluxDB server hostname", 2024-09-11T22:07:27.639 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.639 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.639 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.640 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.640 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-09-11T22:07:27.640 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-09-11T22:07:27.640 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-11T22:07:27.640 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.640 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.640 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "30", 2024-09-11T22:07:27.640 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "5", 2024-09-11T22:07:27.640 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.641 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.641 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-11T22:07:27.641 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.641 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.641 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.641 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.641 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:27.641 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:27.642 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.642 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.642 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.642 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.642 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.642 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.642 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.642 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.642 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.643 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.643 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.643 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.643 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.643 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.643 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.643 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.643 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.643 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.644 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.644 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:27.644 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:27.644 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.644 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.644 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.644 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.644 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.645 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.645 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.645 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.645 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.645 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.645 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.645 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.646 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:27.646 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:27.646 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.646 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.646 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.646 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:27.646 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.646 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.647 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.647 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.647 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.647 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.647 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.647 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.647 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.647 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.648 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.648 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.648 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.648 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.648 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.648 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:27.648 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:27.648 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.649 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.649 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.649 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.649 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.649 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.649 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.649 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.650 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.650 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.650 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.650 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.650 INFO:teuthology.orchestra.run.smithi001.stdout: "password": { 2024-09-11T22:07:27.650 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "password", 2024-09-11T22:07:27.650 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.650 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.651 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.651 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.651 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.651 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.651 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.651 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "password of InfluxDB server user", 2024-09-11T22:07:27.651 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.651 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.651 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.652 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.652 INFO:teuthology.orchestra.run.smithi001.stdout: "port": { 2024-09-11T22:07:27.652 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "port", 2024-09-11T22:07:27.652 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:27.652 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.652 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.652 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8086", 2024-09-11T22:07:27.652 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.653 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.653 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.653 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "InfluxDB server port", 2024-09-11T22:07:27.653 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.653 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.653 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.653 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.653 INFO:teuthology.orchestra.run.smithi001.stdout: "ssl": { 2024-09-11T22:07:27.653 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssl", 2024-09-11T22:07:27.654 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.654 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.654 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.654 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "false", 2024-09-11T22:07:27.654 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.654 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.654 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.654 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-11T22:07:27.654 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.655 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.655 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.655 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.655 INFO:teuthology.orchestra.run.smithi001.stdout: "threads": { 2024-09-11T22:07:27.655 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "threads", 2024-09-11T22:07:27.655 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:27.655 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.655 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.655 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-09-11T22:07:27.656 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-09-11T22:07:27.656 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "32", 2024-09-11T22:07:27.656 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.656 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-11T22:07:27.656 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.656 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.656 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.656 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.656 INFO:teuthology.orchestra.run.smithi001.stdout: "username": { 2024-09-11T22:07:27.657 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "username", 2024-09-11T22:07:27.657 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.657 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.657 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.657 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.657 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.657 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.657 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.658 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "username of InfluxDB server user", 2024-09-11T22:07:27.658 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.658 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.658 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.658 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.658 INFO:teuthology.orchestra.run.smithi001.stdout: "verify_ssl": { 2024-09-11T22:07:27.658 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "verify_ssl", 2024-09-11T22:07:27.658 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.659 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.659 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.659 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "true", 2024-09-11T22:07:27.659 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.659 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.659 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.659 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-11T22:07:27.659 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.660 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.660 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.660 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.660 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.660 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.660 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:27.660 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "insights", 2024-09-11T22:07:27.660 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:27.661 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:27.661 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:27.661 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:27.661 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:27.661 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.661 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.661 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.661 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.662 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.662 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.662 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.662 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.662 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.662 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.662 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.662 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.663 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.663 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.663 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.663 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.663 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.663 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.663 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.663 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:27.664 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:27.664 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.664 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.664 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.664 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.664 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.664 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.665 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.665 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.665 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.665 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.665 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.665 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.665 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:27.665 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:27.666 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.666 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.666 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.666 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:27.666 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.666 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.666 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.666 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.667 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.667 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.667 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.667 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.667 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.667 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.667 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.667 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.667 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.668 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.668 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.668 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:27.668 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:27.668 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.668 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.668 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.669 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.669 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.669 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.669 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.669 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.669 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.669 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.670 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.670 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.670 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.670 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.670 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:27.670 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "iostat", 2024-09-11T22:07:27.670 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:27.670 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:27.671 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:27.671 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:27.671 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:27.671 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.671 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.671 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.671 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.671 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.672 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.672 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.672 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.672 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.672 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.672 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.672 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.672 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.673 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.673 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.673 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.673 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.673 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.673 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.673 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:27.673 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:27.674 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.674 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.674 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.674 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.674 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.674 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.674 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.675 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.675 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.675 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.675 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.675 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.675 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:27.675 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:27.675 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.676 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.676 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.676 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:27.676 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.676 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.676 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.676 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.676 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.677 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.677 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.677 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.677 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.677 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.677 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.677 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.677 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.678 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.678 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.678 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:27.678 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:27.678 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.678 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.678 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.678 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.679 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.679 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.679 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.679 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.679 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.679 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.679 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.679 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.680 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.680 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.680 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:27.680 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "k8sevents", 2024-09-11T22:07:27.680 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:27.680 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:27.680 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:27.680 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_event_retention_days": { 2024-09-11T22:07:27.681 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ceph_event_retention_days", 2024-09-11T22:07:27.681 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:27.681 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.681 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.681 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "7", 2024-09-11T22:07:27.681 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.681 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.681 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.682 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-11T22:07:27.682 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.682 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.682 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.682 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.682 INFO:teuthology.orchestra.run.smithi001.stdout: "config_check_secs": { 2024-09-11T22:07:27.682 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "config_check_secs", 2024-09-11T22:07:27.682 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:27.683 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.683 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.683 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-11T22:07:27.683 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "10", 2024-09-11T22:07:27.683 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.683 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.683 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-11T22:07:27.683 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.684 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.684 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.684 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.684 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:27.684 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:27.684 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.684 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.685 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.685 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.685 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.685 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.685 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.685 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.685 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.686 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.686 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.686 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.686 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.686 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.686 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.686 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.686 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.686 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.687 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.687 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:27.687 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:27.687 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.687 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.687 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.687 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.687 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.688 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.688 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.688 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.688 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.688 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.688 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.688 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.688 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:27.689 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:27.689 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.689 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.689 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.689 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:27.689 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.689 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.689 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.690 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.690 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.690 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.690 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.690 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.690 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.690 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.690 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.691 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.691 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.691 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.691 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.691 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:27.691 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:27.691 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.691 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.692 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.692 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.692 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.692 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.692 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.692 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.692 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.692 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.693 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.693 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.693 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.693 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.693 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:27.693 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "localpool", 2024-09-11T22:07:27.693 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:27.693 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:27.694 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:27.694 INFO:teuthology.orchestra.run.smithi001.stdout: "failure_domain": { 2024-09-11T22:07:27.694 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "failure_domain", 2024-09-11T22:07:27.694 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.694 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.694 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.694 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "host", 2024-09-11T22:07:27.694 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.695 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.695 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.695 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "failure domain for any created local pool", 2024-09-11T22:07:27.695 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-11T22:07:27.695 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.695 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.695 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.696 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:27.696 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:27.696 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.696 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.696 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.696 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.696 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.696 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.697 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.697 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.697 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.697 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.697 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.697 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.697 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.697 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.698 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.698 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.698 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.698 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.698 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.698 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:27.698 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:27.698 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.699 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.699 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.699 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.699 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.699 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.699 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.699 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.699 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.699 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.700 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.700 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.700 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:27.700 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:27.700 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.700 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.700 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.701 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:27.701 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.701 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.701 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.701 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.701 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.701 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.701 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.702 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.702 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.702 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.702 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.702 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.702 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.702 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.702 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.703 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:27.703 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:27.703 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.703 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.703 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.703 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.703 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.703 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.704 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.704 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.704 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.704 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.704 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.704 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.704 INFO:teuthology.orchestra.run.smithi001.stdout: "min_size": { 2024-09-11T22:07:27.705 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "min_size", 2024-09-11T22:07:27.705 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:27.705 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.705 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.705 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.705 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.705 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.705 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.706 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "default min_size for any created local pool", 2024-09-11T22:07:27.706 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-11T22:07:27.706 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.706 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.706 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.706 INFO:teuthology.orchestra.run.smithi001.stdout: "num_rep": { 2024-09-11T22:07:27.706 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "num_rep", 2024-09-11T22:07:27.707 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:27.707 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.707 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.707 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "3", 2024-09-11T22:07:27.707 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.707 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.707 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.707 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "default replica count for any created local pool", 2024-09-11T22:07:27.708 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.708 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.708 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.708 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.708 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num": { 2024-09-11T22:07:27.708 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pg_num", 2024-09-11T22:07:27.708 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:27.708 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.709 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.709 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "128", 2024-09-11T22:07:27.709 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.709 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.709 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.709 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "default pg_num for any created local pool", 2024-09-11T22:07:27.710 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.710 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.710 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.710 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.710 INFO:teuthology.orchestra.run.smithi001.stdout: "prefix": { 2024-09-11T22:07:27.710 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "prefix", 2024-09-11T22:07:27.710 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.710 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.711 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.711 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.711 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.711 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.711 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.711 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "name prefix for any created local pool", 2024-09-11T22:07:27.711 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.712 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.712 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.712 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.712 INFO:teuthology.orchestra.run.smithi001.stdout: "subtree": { 2024-09-11T22:07:27.712 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "subtree", 2024-09-11T22:07:27.712 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.712 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.712 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.713 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "rack", 2024-09-11T22:07:27.713 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.713 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.713 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.713 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-11T22:07:27.713 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-11T22:07:27.713 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.714 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.714 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.714 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.714 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.714 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:27.714 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mds_autoscaler", 2024-09-11T22:07:27.714 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:27.714 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:27.715 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:27.715 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:27.715 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:27.715 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.715 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.715 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.715 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.715 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.716 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.716 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.716 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.716 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.716 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.716 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.716 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.716 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.717 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.717 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.717 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.717 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.717 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.717 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.717 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:27.717 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:27.718 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.718 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.718 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.718 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.718 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.718 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.718 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.718 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.719 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.719 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.719 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.719 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.719 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:27.719 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:27.719 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.719 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.719 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.720 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:27.720 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.720 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.720 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.720 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.720 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.720 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.720 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.721 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.721 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.721 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.721 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.721 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.721 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.721 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.722 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.722 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:27.722 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:27.722 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.722 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.722 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.722 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.722 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.723 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.723 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.723 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.723 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.723 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.723 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.723 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.723 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.723 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.724 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:27.724 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mirroring", 2024-09-11T22:07:27.724 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:27.724 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:27.724 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:27.724 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:27.724 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:27.725 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.725 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.725 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.725 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.725 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.725 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.725 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.725 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.726 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.726 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.726 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.726 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.726 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.726 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.726 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.726 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.727 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.727 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.727 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.727 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:27.727 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:27.727 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.727 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.727 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.728 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.728 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.728 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.728 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.728 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.728 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.728 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.728 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.729 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.729 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:27.729 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:27.729 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.729 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.729 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.729 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:27.729 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.730 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.730 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.730 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.730 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.730 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.730 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.730 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.730 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.731 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.731 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.731 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.731 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.731 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.731 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.731 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:27.731 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:27.732 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.732 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.732 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.732 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.732 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.732 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.732 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.732 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.733 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.733 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.733 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.733 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.733 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.733 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.733 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:27.733 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "nfs", 2024-09-11T22:07:27.734 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:27.734 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:27.734 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:27.734 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:27.734 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:27.734 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.734 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.735 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.735 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.735 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.735 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.735 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.735 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.735 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.736 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.736 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.736 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.736 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.736 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.736 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.736 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.736 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.737 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.737 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.737 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:27.737 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:27.737 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.737 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.737 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.737 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.738 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.738 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.738 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.738 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.738 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.738 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.738 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.738 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.739 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:27.739 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:27.739 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.739 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.739 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.739 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:27.739 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.739 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.740 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.740 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.740 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.740 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.740 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.740 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.740 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.740 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.741 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.741 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.741 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.741 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.741 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.741 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:27.741 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:27.741 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.742 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.742 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.742 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.742 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.742 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.742 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.742 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.742 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.743 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.743 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.743 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.743 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.743 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.743 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:27.743 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "orchestrator", 2024-09-11T22:07:27.743 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:27.744 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:27.744 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:27.744 INFO:teuthology.orchestra.run.smithi001.stdout: "fail_fs": { 2024-09-11T22:07:27.744 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "fail_fs", 2024-09-11T22:07:27.744 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.744 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.744 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.744 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.745 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.745 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.745 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.745 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-11T22:07:27.745 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.745 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.745 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.746 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.746 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:27.746 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:27.746 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.746 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.746 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.746 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.746 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.746 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.747 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.747 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.747 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.747 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.747 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.747 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.747 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.747 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.748 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.748 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.748 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.748 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.748 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.748 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:27.748 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:27.748 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.749 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.749 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.749 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.749 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.749 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.749 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.749 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.749 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.750 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.750 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.750 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.750 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:27.750 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:27.750 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.750 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.750 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.751 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:27.751 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.751 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.751 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.751 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.751 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.751 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.751 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.752 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.752 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.752 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.752 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.752 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.752 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.752 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.752 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.753 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:27.753 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:27.753 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.753 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.753 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.753 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.753 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.754 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.754 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.754 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.754 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.754 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.754 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.754 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.754 INFO:teuthology.orchestra.run.smithi001.stdout: "orchestrator": { 2024-09-11T22:07:27.755 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "orchestrator", 2024-09-11T22:07:27.755 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.755 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.755 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.755 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.755 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.755 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.755 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.755 INFO:teuthology.orchestra.run.smithi001.stdout: "cephadm", 2024-09-11T22:07:27.756 INFO:teuthology.orchestra.run.smithi001.stdout: "rook", 2024-09-11T22:07:27.756 INFO:teuthology.orchestra.run.smithi001.stdout: "test_orchestrator" 2024-09-11T22:07:27.756 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.756 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Orchestrator backend", 2024-09-11T22:07:27.756 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.756 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.756 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.756 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.757 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.757 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.757 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:27.757 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "osd_perf_query", 2024-09-11T22:07:27.757 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:27.757 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:27.757 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:27.757 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:27.757 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:27.758 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.758 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.758 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.758 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.758 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.758 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.758 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.759 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.759 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.759 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.759 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.759 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.759 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.759 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.759 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.760 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.760 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.760 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.760 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.760 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:27.760 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:27.760 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.760 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.761 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.761 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.761 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.761 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.761 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.761 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.761 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.761 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.761 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.762 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.762 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:27.762 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:27.762 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.762 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.762 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.762 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:27.763 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.763 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.763 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.763 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.763 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.763 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.763 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.763 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.764 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.764 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.764 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.764 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.764 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.764 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.764 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.765 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:27.765 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:27.765 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.765 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.765 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.765 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.765 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.765 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.766 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.766 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.766 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.766 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.766 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.766 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.766 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.766 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.767 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:27.767 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "osd_support", 2024-09-11T22:07:27.767 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:27.767 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:27.767 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:27.767 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:27.767 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:27.767 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.768 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.768 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.768 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.768 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.768 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.768 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.768 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.768 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.768 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.769 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.769 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.769 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.769 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.769 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.769 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.769 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.769 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.769 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.770 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:27.770 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:27.770 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.770 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.770 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.770 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.770 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.771 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.771 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.771 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.771 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.771 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.771 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.771 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.771 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:27.772 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:27.772 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.772 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.772 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.772 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:27.772 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.772 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.772 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.773 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.773 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.773 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.773 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.773 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.773 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.773 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.774 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.774 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.774 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.774 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.774 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.774 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:27.774 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:27.774 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.774 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.775 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.775 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.775 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.775 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.775 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.775 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.775 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.775 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.776 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.776 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.776 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.776 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.776 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:27.776 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pg_autoscaler", 2024-09-11T22:07:27.776 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:27.776 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:27.777 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:27.777 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:27.777 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:27.777 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.777 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.777 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.777 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.778 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.778 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.778 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.778 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.778 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.778 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.778 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.778 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.779 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.779 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.779 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.779 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.779 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.779 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.779 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.779 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:27.780 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:27.780 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.780 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.780 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.780 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.780 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.780 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.780 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.780 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.781 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.781 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.781 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.781 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.781 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:27.781 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:27.781 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.781 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.782 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.782 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:27.782 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.782 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.782 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.782 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.782 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.782 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.782 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.783 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.783 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.783 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.783 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.783 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.783 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.783 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.784 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.784 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:27.784 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:27.784 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.784 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.784 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.784 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.784 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.784 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.785 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.785 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.785 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.785 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.785 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.785 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.785 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-09-11T22:07:27.785 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-09-11T22:07:27.786 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-11T22:07:27.786 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.786 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.786 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-09-11T22:07:27.786 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.786 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.786 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.786 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.786 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.787 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.787 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.787 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.787 INFO:teuthology.orchestra.run.smithi001.stdout: "threshold": { 2024-09-11T22:07:27.787 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "threshold", 2024-09-11T22:07:27.787 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-09-11T22:07:27.787 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.787 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.788 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "3.0", 2024-09-11T22:07:27.788 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1.0", 2024-09-11T22:07:27.788 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.788 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.788 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "scaling threshold", 2024-09-11T22:07:27.788 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-11T22:07:27.788 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.788 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.789 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.789 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.789 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.789 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:27.789 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "progress", 2024-09-11T22:07:27.789 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:27.789 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:27.789 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:27.790 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_pg_recovery_event": { 2024-09-11T22:07:27.790 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "allow_pg_recovery_event", 2024-09-11T22:07:27.790 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.790 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.790 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.790 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.790 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.790 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.791 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.791 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-11T22:07:27.791 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.791 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.791 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.791 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.791 INFO:teuthology.orchestra.run.smithi001.stdout: "enabled": { 2024-09-11T22:07:27.791 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enabled", 2024-09-11T22:07:27.792 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.792 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.792 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.792 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:27.792 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.792 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.792 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.792 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.793 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.793 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.793 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.793 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.793 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:27.793 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:27.793 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.793 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.794 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.794 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.794 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.794 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.794 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.794 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.794 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.795 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.795 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.795 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.795 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.795 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.795 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.795 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.795 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.796 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.796 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.796 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:27.796 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:27.796 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.796 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.796 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.797 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.797 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.797 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.797 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.797 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.797 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.797 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.797 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.798 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.798 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:27.798 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:27.798 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.798 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.798 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.798 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:27.799 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.799 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.799 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.799 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.799 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.799 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.799 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.799 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.799 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.800 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.800 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.800 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.800 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.800 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.800 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.800 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:27.800 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:27.801 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.801 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.801 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.801 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.801 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.801 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.801 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.802 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.802 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.802 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.802 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.802 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.802 INFO:teuthology.orchestra.run.smithi001.stdout: "max_completed_events": { 2024-09-11T22:07:27.802 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_completed_events", 2024-09-11T22:07:27.802 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:27.802 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.803 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.803 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "50", 2024-09-11T22:07:27.803 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.803 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.803 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.803 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "number of past completed events to remember", 2024-09-11T22:07:27.803 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.803 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.804 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.804 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.804 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-09-11T22:07:27.804 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-09-11T22:07:27.804 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-11T22:07:27.804 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.804 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.804 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-09-11T22:07:27.805 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.805 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.805 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.805 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how long the module is going to sleep", 2024-09-11T22:07:27.805 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.805 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.805 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.805 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.806 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.806 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.806 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:27.806 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "prometheus", 2024-09-11T22:07:27.806 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:27.806 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:27.806 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:27.806 INFO:teuthology.orchestra.run.smithi001.stdout: "cache": { 2024-09-11T22:07:27.807 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cache", 2024-09-11T22:07:27.807 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.807 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.807 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.807 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:27.807 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.807 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.807 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.807 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.808 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.808 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.808 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.808 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.808 INFO:teuthology.orchestra.run.smithi001.stdout: "exclude_perf_counters": { 2024-09-11T22:07:27.808 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "exclude_perf_counters", 2024-09-11T22:07:27.808 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.808 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.809 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.809 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:27.809 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.809 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.809 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.809 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-11T22:07:27.809 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-11T22:07:27.810 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.810 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.810 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.810 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:27.810 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:27.810 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.810 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.811 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.811 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.811 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.811 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.811 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.811 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.811 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.811 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.812 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.812 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.812 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.812 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.812 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.812 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.812 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.812 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.813 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.813 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:27.813 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:27.813 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.813 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.813 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.813 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.813 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.814 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.814 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.814 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.814 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.814 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.814 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.814 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.814 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:27.814 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:27.815 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.815 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.815 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.815 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:27.815 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.815 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.815 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.815 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.816 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.816 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.816 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.816 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.816 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.816 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.816 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.816 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.817 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.817 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.817 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.817 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:27.817 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:27.817 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.817 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.817 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.818 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.818 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.818 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.818 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.818 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.818 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.818 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.818 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.819 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.819 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_stats_pools": { 2024-09-11T22:07:27.819 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rbd_stats_pools", 2024-09-11T22:07:27.819 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.819 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.819 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.819 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.819 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.820 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.820 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.820 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.820 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.820 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.820 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.820 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.821 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-11T22:07:27.821 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-11T22:07:27.821 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:27.821 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.821 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.821 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "300", 2024-09-11T22:07:27.821 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.821 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.822 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.822 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.822 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.822 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.822 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.822 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.822 INFO:teuthology.orchestra.run.smithi001.stdout: "scrape_interval": { 2024-09-11T22:07:27.822 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "scrape_interval", 2024-09-11T22:07:27.823 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-09-11T22:07:27.823 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.823 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.823 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "15.0", 2024-09-11T22:07:27.823 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.823 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.823 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.824 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.824 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.824 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.824 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.824 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.824 INFO:teuthology.orchestra.run.smithi001.stdout: "server_addr": { 2024-09-11T22:07:27.824 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_addr", 2024-09-11T22:07:27.824 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.825 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.825 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.825 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "::", 2024-09-11T22:07:27.825 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.825 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.825 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.825 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-11T22:07:27.826 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.826 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.826 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.826 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.826 INFO:teuthology.orchestra.run.smithi001.stdout: "server_port": { 2024-09-11T22:07:27.826 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_port", 2024-09-11T22:07:27.826 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:27.826 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.826 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.827 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "9283", 2024-09-11T22:07:27.827 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.827 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.827 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.827 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-11T22:07:27.827 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.827 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.828 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.828 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.828 INFO:teuthology.orchestra.run.smithi001.stdout: "stale_cache_strategy": { 2024-09-11T22:07:27.828 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "stale_cache_strategy", 2024-09-11T22:07:27.828 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.828 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.828 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.828 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "log", 2024-09-11T22:07:27.829 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.829 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.829 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.829 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.829 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.829 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.829 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.829 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.830 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_behaviour": { 2024-09-11T22:07:27.830 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_behaviour", 2024-09-11T22:07:27.830 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.830 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.830 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.830 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "default", 2024-09-11T22:07:27.830 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.831 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.831 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.831 INFO:teuthology.orchestra.run.smithi001.stdout: "default", 2024-09-11T22:07:27.831 INFO:teuthology.orchestra.run.smithi001.stdout: "error" 2024-09-11T22:07:27.831 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.831 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.831 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.831 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.832 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.832 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.832 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_error_status_code": { 2024-09-11T22:07:27.832 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_error_status_code", 2024-09-11T22:07:27.832 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:27.832 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.832 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.832 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "500", 2024-09-11T22:07:27.832 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "400", 2024-09-11T22:07:27.833 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "599", 2024-09-11T22:07:27.833 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.833 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.833 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.833 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.833 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.833 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.833 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.834 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.834 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:27.834 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rbd_support", 2024-09-11T22:07:27.834 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:27.834 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:27.834 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:27.834 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:27.834 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:27.835 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.835 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.835 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.835 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.835 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.835 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.835 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.835 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.836 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.836 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.836 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.836 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.836 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.836 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.836 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.837 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.837 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.837 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.837 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.837 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:27.837 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:27.837 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.837 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.838 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.838 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.838 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.838 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.838 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.838 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.838 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.838 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.839 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.839 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.839 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:27.839 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:27.839 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.839 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.839 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.840 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:27.840 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.840 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.840 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.840 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.840 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.840 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.840 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.841 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.841 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.841 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.841 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.841 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.841 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.841 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.841 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.841 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:27.842 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:27.842 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.842 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.842 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.842 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.842 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.842 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.842 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.842 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.843 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.843 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.843 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.843 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.843 INFO:teuthology.orchestra.run.smithi001.stdout: "max_concurrent_snap_create": { 2024-09-11T22:07:27.843 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_concurrent_snap_create", 2024-09-11T22:07:27.843 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:27.843 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.844 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.844 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-11T22:07:27.844 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.844 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.844 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.844 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.844 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.845 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.845 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.845 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.845 INFO:teuthology.orchestra.run.smithi001.stdout: "mirror_snapshot_schedule": { 2024-09-11T22:07:27.845 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mirror_snapshot_schedule", 2024-09-11T22:07:27.845 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.845 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.845 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.846 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.846 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.846 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.846 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.846 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.846 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.846 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.846 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.847 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.847 INFO:teuthology.orchestra.run.smithi001.stdout: "trash_purge_schedule": { 2024-09-11T22:07:27.847 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "trash_purge_schedule", 2024-09-11T22:07:27.847 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.847 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.847 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.847 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.847 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.847 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.848 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.848 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.848 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.848 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.848 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.848 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.848 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.849 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.849 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:27.849 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "restful", 2024-09-11T22:07:27.849 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:27.849 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:27.849 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:27.849 INFO:teuthology.orchestra.run.smithi001.stdout: "enable_auth": { 2024-09-11T22:07:27.849 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enable_auth", 2024-09-11T22:07:27.850 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.850 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.850 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.850 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:27.850 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.850 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.850 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.850 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.850 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.851 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.851 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.851 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.851 INFO:teuthology.orchestra.run.smithi001.stdout: "key_file": { 2024-09-11T22:07:27.851 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "key_file", 2024-09-11T22:07:27.851 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.851 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.851 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.852 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.852 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.852 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.852 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.852 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.852 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.852 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.853 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.853 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.853 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:27.853 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:27.853 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.853 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.853 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.853 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.854 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.854 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.854 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.854 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.854 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.854 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.854 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.855 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.855 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.855 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.855 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.855 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.855 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.855 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.855 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.856 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:27.856 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:27.856 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.856 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.856 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.856 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.856 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.856 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.857 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.857 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.857 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.857 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.857 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.857 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.857 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:27.858 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:27.858 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.858 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.858 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.858 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:27.858 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.858 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.858 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.859 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.859 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.859 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.859 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.859 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.859 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.859 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.859 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.860 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.860 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.860 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.860 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.860 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:27.860 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:27.860 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.861 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.861 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.861 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.861 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.861 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.861 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.861 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.861 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.862 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.862 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.862 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.862 INFO:teuthology.orchestra.run.smithi001.stdout: "server_addr": { 2024-09-11T22:07:27.862 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_addr", 2024-09-11T22:07:27.862 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.862 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.862 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.862 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.863 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.863 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.863 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.863 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.863 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.863 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.863 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.863 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.864 INFO:teuthology.orchestra.run.smithi001.stdout: "server_port": { 2024-09-11T22:07:27.864 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_port", 2024-09-11T22:07:27.864 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.864 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.864 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.864 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.864 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.865 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.865 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.865 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.865 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.865 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.865 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.865 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.865 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.865 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.866 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:27.866 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rgw", 2024-09-11T22:07:27.866 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:27.866 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:27.866 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:27.866 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:27.866 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:27.866 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.867 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.867 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.867 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.867 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.867 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.867 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.867 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.867 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.867 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.868 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.868 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.868 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.868 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.868 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.868 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.868 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.868 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.869 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.869 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:27.869 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:27.869 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.869 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.869 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.869 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.869 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.869 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.870 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.870 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.870 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.870 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.870 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.870 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:27.870 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:27.871 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.871 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.871 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.871 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:27.871 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.871 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.871 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.871 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.871 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.872 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.872 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.872 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.872 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.872 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.872 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.872 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.872 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.873 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.873 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.873 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:27.873 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:27.873 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.873 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.873 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.873 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.874 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.874 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.874 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.874 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.874 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.874 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.874 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.874 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.875 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.875 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.875 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:27.875 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rook", 2024-09-11T22:07:27.875 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:27.875 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:27.875 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:27.875 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:27.876 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:27.876 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.876 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.876 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.876 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.876 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.876 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.876 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.876 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.877 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.877 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.877 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.877 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.877 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.877 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.877 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.877 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.878 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.878 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.878 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.878 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:27.878 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:27.878 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.878 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.878 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.879 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.879 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.879 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.879 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.879 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.879 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.879 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.879 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.880 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.880 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:27.880 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:27.880 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.880 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.880 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.880 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:27.880 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.881 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.881 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.881 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.881 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.881 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.881 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.881 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.881 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.882 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.882 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.882 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.882 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.882 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.882 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.882 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:27.882 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:27.883 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.883 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.883 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.883 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.883 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.883 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.883 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.884 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.884 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.884 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.884 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.884 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.884 INFO:teuthology.orchestra.run.smithi001.stdout: "storage_class": { 2024-09-11T22:07:27.884 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "storage_class", 2024-09-11T22:07:27.885 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.885 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.885 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.885 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "local", 2024-09-11T22:07:27.885 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.885 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.885 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.885 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-11T22:07:27.886 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.886 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.886 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.886 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.886 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.886 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.886 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:27.886 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "selftest", 2024-09-11T22:07:27.887 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:27.887 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:27.887 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:27.887 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:27.887 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:27.887 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.887 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.887 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.888 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.888 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.888 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.888 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.888 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.888 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.888 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.888 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.889 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.889 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.889 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.889 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.889 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.889 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.889 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.889 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.890 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:27.890 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:27.890 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.890 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.890 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.890 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.890 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.890 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.891 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.891 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.891 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.891 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.891 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.891 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.891 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:27.891 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:27.892 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.892 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.892 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.892 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:27.892 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.892 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.892 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.892 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.893 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.893 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.893 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.893 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.893 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.893 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.893 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.893 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.894 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.894 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.894 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.894 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:27.894 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:27.894 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.894 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.894 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.895 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.895 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.895 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.895 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.895 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.895 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.895 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.895 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.896 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.896 INFO:teuthology.orchestra.run.smithi001.stdout: "roption1": { 2024-09-11T22:07:27.896 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "roption1", 2024-09-11T22:07:27.896 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.896 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.896 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.896 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.896 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.896 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.897 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.897 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.897 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.897 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.897 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.897 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.897 INFO:teuthology.orchestra.run.smithi001.stdout: "roption2": { 2024-09-11T22:07:27.898 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "roption2", 2024-09-11T22:07:27.898 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.898 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.898 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.898 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "xyz", 2024-09-11T22:07:27.898 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.898 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.898 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.899 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.899 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.899 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.899 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.899 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.899 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption1": { 2024-09-11T22:07:27.899 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption1", 2024-09-11T22:07:27.899 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.900 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.900 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.900 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.900 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.900 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.900 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.900 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.900 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.901 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.901 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.901 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.901 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption2": { 2024-09-11T22:07:27.901 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption2", 2024-09-11T22:07:27.901 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:27.901 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.902 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.902 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.902 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.902 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.902 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.902 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.902 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.902 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.902 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.903 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.903 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption3": { 2024-09-11T22:07:27.903 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption3", 2024-09-11T22:07:27.903 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-09-11T22:07:27.903 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.903 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.903 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.903 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.904 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.904 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.904 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.904 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.904 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.904 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.904 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.904 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption4": { 2024-09-11T22:07:27.905 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption4", 2024-09-11T22:07:27.905 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.905 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.905 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.905 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.905 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.905 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.906 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.906 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.906 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.906 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.906 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.906 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.906 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption5": { 2024-09-11T22:07:27.906 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption5", 2024-09-11T22:07:27.907 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.907 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.907 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.907 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.907 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.907 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.907 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.907 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.908 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.908 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.908 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.908 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.908 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption6": { 2024-09-11T22:07:27.908 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption6", 2024-09-11T22:07:27.908 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.908 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.909 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.909 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:27.909 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.909 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.909 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.909 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.909 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.909 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.910 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.910 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.910 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption7": { 2024-09-11T22:07:27.910 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption7", 2024-09-11T22:07:27.910 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:27.910 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.910 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.910 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.911 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-09-11T22:07:27.911 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "42", 2024-09-11T22:07:27.911 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.911 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.911 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.911 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.911 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.912 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.912 INFO:teuthology.orchestra.run.smithi001.stdout: "testkey": { 2024-09-11T22:07:27.912 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "testkey", 2024-09-11T22:07:27.912 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.912 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.912 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.912 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.912 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.913 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.913 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.913 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.913 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.913 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.913 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.913 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.914 INFO:teuthology.orchestra.run.smithi001.stdout: "testlkey": { 2024-09-11T22:07:27.914 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "testlkey", 2024-09-11T22:07:27.914 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.914 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.914 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.914 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.914 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.914 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.914 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.915 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.915 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.915 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.915 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.915 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.915 INFO:teuthology.orchestra.run.smithi001.stdout: "testnewline": { 2024-09-11T22:07:27.915 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "testnewline", 2024-09-11T22:07:27.915 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.916 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.916 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.916 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.916 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.916 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.916 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.916 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.916 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.917 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.917 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.917 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.917 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.917 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.917 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:27.917 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "snap_schedule", 2024-09-11T22:07:27.917 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:27.918 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:27.918 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:27.918 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_m_granularity": { 2024-09-11T22:07:27.918 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "allow_m_granularity", 2024-09-11T22:07:27.918 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.918 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.918 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.918 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.919 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.919 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.919 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.919 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "allow minute scheduled snapshots", 2024-09-11T22:07:27.919 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.919 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.919 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.919 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.920 INFO:teuthology.orchestra.run.smithi001.stdout: "dump_on_update": { 2024-09-11T22:07:27.920 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "dump_on_update", 2024-09-11T22:07:27.920 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.920 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.920 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.920 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.920 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.920 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.921 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.921 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "dump database to debug log on update", 2024-09-11T22:07:27.921 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.921 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.921 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.921 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.921 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:27.921 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:27.922 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.922 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.922 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.922 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.922 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.922 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.922 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.922 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.923 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.923 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.923 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.923 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.923 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.923 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.923 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.923 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.923 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.924 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.924 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.924 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:27.924 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:27.924 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.924 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.924 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.924 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.925 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.925 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.925 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.925 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.925 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.925 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.925 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.926 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.926 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:27.926 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:27.926 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.926 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.926 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.926 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:27.926 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.927 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.927 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.927 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.927 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.927 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.927 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.927 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.927 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.928 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.928 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.928 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.928 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.928 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.928 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.928 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:27.928 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:27.929 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.929 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.929 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.929 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.929 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.929 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.929 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.930 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.930 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.930 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.930 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.930 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.930 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.930 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.930 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:27.931 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "stats", 2024-09-11T22:07:27.931 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:27.931 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:27.931 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:27.931 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:27.931 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:27.931 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.931 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.931 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.932 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.932 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.932 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.932 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.932 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.932 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.932 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.932 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.933 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.933 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.933 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.933 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.933 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.933 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.933 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.933 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.934 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:27.934 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:27.934 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.934 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.934 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.934 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.934 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.934 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.935 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.935 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.935 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.935 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.935 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.935 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.935 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:27.936 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:27.936 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.936 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.936 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.936 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:27.936 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.936 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.936 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.937 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.937 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.937 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.937 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.937 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.937 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.937 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.937 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.938 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.938 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.938 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.938 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.938 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:27.938 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:27.938 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.938 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.939 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.939 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.939 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.939 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.939 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.939 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.939 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.939 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.940 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.940 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.940 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.940 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.940 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:27.940 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "status", 2024-09-11T22:07:27.940 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:27.941 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:27.941 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:27.941 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:27.941 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:27.941 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.941 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.941 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.941 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.942 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.942 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.942 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.942 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.942 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.942 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.942 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.942 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.943 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.943 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.943 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.943 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.943 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.943 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.943 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.943 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:27.944 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:27.944 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.944 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.944 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.944 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.944 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.944 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.945 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.945 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.945 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.945 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.945 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.945 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.945 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:27.945 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:27.946 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.946 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.946 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.946 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:27.946 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.946 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.946 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.946 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.947 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.947 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.947 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.947 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.947 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.947 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.947 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.947 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.948 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.948 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.948 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.948 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:27.948 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:27.948 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.948 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.949 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.949 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.949 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.949 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.949 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.949 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.949 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.949 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.950 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.950 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.950 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.950 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.950 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:27.950 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "telegraf", 2024-09-11T22:07:27.950 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:27.950 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:27.951 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:27.951 INFO:teuthology.orchestra.run.smithi001.stdout: "address": { 2024-09-11T22:07:27.951 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "address", 2024-09-11T22:07:27.951 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.951 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.951 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.951 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-11T22:07:27.951 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.952 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.952 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.952 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.952 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.952 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.952 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.952 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.952 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-09-11T22:07:27.953 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-09-11T22:07:27.953 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-11T22:07:27.953 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.953 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.953 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "15", 2024-09-11T22:07:27.953 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.953 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.954 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.954 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.954 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.954 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.954 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.954 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.954 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:27.954 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:27.954 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.955 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.955 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.955 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.955 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.955 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.955 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.955 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.955 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.956 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.956 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.956 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.956 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.956 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.956 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.956 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.956 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.957 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.957 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.957 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:27.957 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:27.957 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.957 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.957 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.957 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.957 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.958 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.958 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.958 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.958 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.958 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.958 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.958 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.958 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:27.959 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:27.959 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.959 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.959 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.959 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:27.959 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.959 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.959 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.960 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.960 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.960 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.960 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.960 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.960 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.960 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.960 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.961 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.961 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.961 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.961 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.961 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:27.961 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:27.961 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.961 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.962 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.962 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.962 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.962 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.962 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.962 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.962 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.962 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.963 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.963 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.963 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:27.963 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.963 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:27.963 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "telemetry", 2024-09-11T22:07:27.963 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:27.963 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:27.964 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:27.964 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_basic": { 2024-09-11T22:07:27.964 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_basic", 2024-09-11T22:07:27.964 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.964 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.964 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.964 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:27.964 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.965 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.965 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.965 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-11T22:07:27.965 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.965 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.965 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.965 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.965 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_crash": { 2024-09-11T22:07:27.966 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_crash", 2024-09-11T22:07:27.966 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.966 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.966 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.966 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:27.966 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.966 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.966 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.967 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-11T22:07:27.967 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.967 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.967 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.967 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.967 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_device": { 2024-09-11T22:07:27.967 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_device", 2024-09-11T22:07:27.967 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.968 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.968 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.968 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:27.968 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.968 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.968 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.968 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-11T22:07:27.968 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.969 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.969 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.969 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.969 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_ident": { 2024-09-11T22:07:27.969 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_ident", 2024-09-11T22:07:27.969 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.969 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.969 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.970 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.970 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.970 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.970 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.970 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-11T22:07:27.970 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.970 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.971 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.971 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.971 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_perf": { 2024-09-11T22:07:27.971 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_perf", 2024-09-11T22:07:27.971 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.971 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.971 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.972 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.972 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.972 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.972 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.972 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-11T22:07:27.972 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.972 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.972 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.973 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.973 INFO:teuthology.orchestra.run.smithi001.stdout: "contact": { 2024-09-11T22:07:27.973 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "contact", 2024-09-11T22:07:27.973 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.973 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.973 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.973 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.973 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.974 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.974 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.974 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.974 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.974 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.974 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.974 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.974 INFO:teuthology.orchestra.run.smithi001.stdout: "description": { 2024-09-11T22:07:27.975 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "description", 2024-09-11T22:07:27.975 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.975 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.975 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.975 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.975 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.975 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.975 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.976 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.976 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.976 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.976 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.976 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.976 INFO:teuthology.orchestra.run.smithi001.stdout: "device_url": { 2024-09-11T22:07:27.976 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "device_url", 2024-09-11T22:07:27.977 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.977 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.977 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.977 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-11T22:07:27.977 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.977 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.977 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.977 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.977 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.978 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.978 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.978 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.978 INFO:teuthology.orchestra.run.smithi001.stdout: "enabled": { 2024-09-11T22:07:27.978 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enabled", 2024-09-11T22:07:27.978 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.978 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.978 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.979 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.979 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.979 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.979 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.979 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.979 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.979 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.979 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.980 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.980 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-09-11T22:07:27.980 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-09-11T22:07:27.980 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:27.980 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.980 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.980 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "24", 2024-09-11T22:07:27.981 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "8", 2024-09-11T22:07:27.981 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.981 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.981 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.981 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.981 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.981 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.981 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.982 INFO:teuthology.orchestra.run.smithi001.stdout: "last_opt_revision": { 2024-09-11T22:07:27.982 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "last_opt_revision", 2024-09-11T22:07:27.982 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:27.982 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.982 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.982 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "1", 2024-09-11T22:07:27.982 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.982 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.983 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.983 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.983 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.983 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.983 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.983 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.983 INFO:teuthology.orchestra.run.smithi001.stdout: "leaderboard": { 2024-09-11T22:07:27.983 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "leaderboard", 2024-09-11T22:07:27.984 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.984 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.984 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.984 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.984 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.984 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.984 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.984 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.985 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.985 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.985 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.985 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.985 INFO:teuthology.orchestra.run.smithi001.stdout: "leaderboard_description": { 2024-09-11T22:07:27.985 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "leaderboard_description", 2024-09-11T22:07:27.985 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.985 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.986 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.986 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.986 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.986 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.986 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.986 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.986 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.986 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.987 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.987 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.987 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:27.987 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:27.987 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.987 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.987 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.987 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.988 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.988 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.988 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.988 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.988 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.988 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.988 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.988 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.989 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.989 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.989 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.989 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.989 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.989 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.989 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.989 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:27.990 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:27.990 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.990 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.990 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.990 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.990 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.990 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.990 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.991 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.991 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.991 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.991 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.991 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.991 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:27.991 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:27.991 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.992 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.992 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.992 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:27.992 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.992 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.992 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:27.992 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:27.992 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:27.993 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:27.993 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:27.993 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:27.993 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:27.993 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:27.993 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.993 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.993 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.994 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.994 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.994 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:27.994 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:27.994 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:27.994 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.994 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:27.994 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:27.995 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.995 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.995 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.995 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.995 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.995 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.995 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.995 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.996 INFO:teuthology.orchestra.run.smithi001.stdout: "organization": { 2024-09-11T22:07:27.996 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "organization", 2024-09-11T22:07:27.996 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.996 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.996 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.996 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.996 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.996 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.997 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.997 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.997 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.997 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.997 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.997 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.997 INFO:teuthology.orchestra.run.smithi001.stdout: "proxy": { 2024-09-11T22:07:27.998 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "proxy", 2024-09-11T22:07:27.998 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:27.998 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:27.998 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:27.998 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:27.998 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:27.998 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:27.998 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:27.999 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:27.999 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:27.999 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:27.999 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:27.999 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:27.999 INFO:teuthology.orchestra.run.smithi001.stdout: "url": { 2024-09-11T22:07:27.999 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "url", 2024-09-11T22:07:27.999 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.000 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.000 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.000 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-11T22:07:28.000 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.000 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.000 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.000 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.000 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.001 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.001 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.001 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.001 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.001 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.001 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:28.001 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "test_orchestrator", 2024-09-11T22:07:28.002 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:28.002 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:28.002 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:28.002 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:28.002 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:28.002 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.002 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.003 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.003 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.003 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.003 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.003 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.003 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.003 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.003 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.004 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.004 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.004 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.004 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.004 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.004 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.004 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.004 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.005 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.005 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:28.005 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:28.005 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.005 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.005 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.005 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.006 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.006 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.006 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.006 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.006 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.006 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.006 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.006 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.006 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:28.007 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:28.007 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.007 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.007 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.007 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:28.007 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.007 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.007 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.008 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.008 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.008 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.008 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.008 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.008 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.008 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.008 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.009 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.009 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.009 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.009 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.009 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:28.009 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:28.009 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.009 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.010 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.010 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.010 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.010 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.010 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.010 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.010 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.010 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.011 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.011 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.011 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.011 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.011 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:28.011 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "volumes", 2024-09-11T22:07:28.011 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:28.011 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:28.012 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:28.012 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:28.012 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:28.012 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.012 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.012 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.012 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.013 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.013 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.013 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.013 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.013 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.013 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.013 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.013 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.014 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.014 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.014 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.014 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.014 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.014 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.014 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.014 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:28.015 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:28.015 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.015 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.015 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.015 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.015 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.015 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.015 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.016 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.016 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.016 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.016 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.016 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.016 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:28.016 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:28.016 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.017 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.017 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.017 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:28.017 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.017 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.017 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.017 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.017 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.018 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.018 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.018 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.018 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.018 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.018 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.018 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.018 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.019 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.019 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.019 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:28.019 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:28.019 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.019 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.019 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.020 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.020 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.020 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.020 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.020 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.020 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.020 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.020 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.021 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.021 INFO:teuthology.orchestra.run.smithi001.stdout: "max_concurrent_clones": { 2024-09-11T22:07:28.021 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_concurrent_clones", 2024-09-11T22:07:28.021 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:28.021 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.021 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.021 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "4", 2024-09-11T22:07:28.021 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.022 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.022 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.022 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-11T22:07:28.022 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.022 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.022 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.022 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.022 INFO:teuthology.orchestra.run.smithi001.stdout: "periodic_async_work": { 2024-09-11T22:07:28.023 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "periodic_async_work", 2024-09-11T22:07:28.023 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.023 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.023 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.023 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.023 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.023 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.023 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.024 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Periodically check for async work", 2024-09-11T22:07:28.024 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.024 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.024 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.024 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.024 INFO:teuthology.orchestra.run.smithi001.stdout: "snapshot_clone_delay": { 2024-09-11T22:07:28.024 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "snapshot_clone_delay", 2024-09-11T22:07:28.024 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:28.025 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.025 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.025 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-09-11T22:07:28.025 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.025 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.025 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.025 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-11T22:07:28.026 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.026 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.026 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.026 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.026 INFO:teuthology.orchestra.run.smithi001.stdout: "snapshot_clone_no_wait": { 2024-09-11T22:07:28.026 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "snapshot_clone_no_wait", 2024-09-11T22:07:28.026 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.026 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.027 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.027 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:28.027 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.027 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.027 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.027 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-11T22:07:28.027 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.027 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.028 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.028 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.028 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.028 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.028 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:28.028 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix", 2024-09-11T22:07:28.028 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:28.028 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:28.029 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:28.029 INFO:teuthology.orchestra.run.smithi001.stdout: "discovery_interval": { 2024-09-11T22:07:28.029 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "discovery_interval", 2024-09-11T22:07:28.029 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-09-11T22:07:28.029 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.029 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.029 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "100", 2024-09-11T22:07:28.029 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.030 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.030 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.030 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.030 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.030 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.030 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.030 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.031 INFO:teuthology.orchestra.run.smithi001.stdout: "identifier": { 2024-09-11T22:07:28.031 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "identifier", 2024-09-11T22:07:28.031 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.031 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.031 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.031 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.031 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.031 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.032 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.032 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.032 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.032 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.032 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.032 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.032 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-09-11T22:07:28.033 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-09-11T22:07:28.033 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-11T22:07:28.033 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.033 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.033 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-09-11T22:07:28.033 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.033 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.033 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.034 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.034 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.034 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.034 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.034 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.034 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:28.034 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:28.034 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.035 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.035 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.035 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.035 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.035 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.035 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.035 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.035 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.036 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.036 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.036 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.036 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.036 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.036 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.036 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.036 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.037 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.037 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.037 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:28.037 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:28.037 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.037 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.037 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.037 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.038 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.038 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.038 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.038 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.038 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.038 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.038 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.038 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.039 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:28.039 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:28.039 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.039 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.039 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.039 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:28.039 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.039 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.040 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.040 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.040 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.040 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.040 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.040 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.040 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.040 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.041 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.041 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.041 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.041 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.041 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.041 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:28.041 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:28.041 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.042 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.042 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.042 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.042 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.042 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.042 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.042 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.042 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.043 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.043 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.043 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.043 INFO:teuthology.orchestra.run.smithi001.stdout: "zabbix_host": { 2024-09-11T22:07:28.043 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix_host", 2024-09-11T22:07:28.043 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.043 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.043 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.044 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.044 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.044 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.044 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.044 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.044 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.044 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.044 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.045 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.045 INFO:teuthology.orchestra.run.smithi001.stdout: "zabbix_port": { 2024-09-11T22:07:28.045 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix_port", 2024-09-11T22:07:28.045 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:28.045 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.045 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.045 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10051", 2024-09-11T22:07:28.045 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.046 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.046 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.046 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.046 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.046 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.046 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.046 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.046 INFO:teuthology.orchestra.run.smithi001.stdout: "zabbix_sender": { 2024-09-11T22:07:28.047 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix_sender", 2024-09-11T22:07:28.047 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.047 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.047 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.047 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-11T22:07:28.047 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.047 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.047 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.048 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.048 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.048 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.048 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.048 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.048 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.048 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.049 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-11T22:07:28.049 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.049 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.049 INFO:teuthology.orchestra.run.smithi001.stdout: "modules": [ 2024-09-11T22:07:28.049 INFO:teuthology.orchestra.run.smithi001.stdout: "cephadm", 2024-09-11T22:07:28.049 INFO:teuthology.orchestra.run.smithi001.stdout: "dashboard", 2024-09-11T22:07:28.049 INFO:teuthology.orchestra.run.smithi001.stdout: "iostat", 2024-09-11T22:07:28.049 INFO:teuthology.orchestra.run.smithi001.stdout: "nfs", 2024-09-11T22:07:28.050 INFO:teuthology.orchestra.run.smithi001.stdout: "restful", 2024-09-11T22:07:28.050 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_schedule" 2024-09-11T22:07:28.050 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.050 INFO:teuthology.orchestra.run.smithi001.stdout: "available_modules": [ 2024-09-11T22:07:28.050 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:28.050 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "alerts", 2024-09-11T22:07:28.050 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:28.050 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:28.050 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:28.051 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-09-11T22:07:28.051 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-09-11T22:07:28.051 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-11T22:07:28.051 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.051 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.051 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-09-11T22:07:28.051 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.052 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.052 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.052 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How frequently to reexamine health status", 2024-09-11T22:07:28.052 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.052 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.052 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.052 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.052 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:28.053 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:28.053 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.053 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.053 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.053 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.053 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.053 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.053 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.054 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.054 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.054 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.054 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.054 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.054 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.054 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.054 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.055 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.055 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.055 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.055 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.055 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:28.055 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:28.055 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.055 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.056 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.056 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.056 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.056 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.056 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.056 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.056 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.057 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.057 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.057 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.057 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:28.057 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:28.057 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.057 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.057 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.058 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:28.058 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.058 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.058 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.058 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.058 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.058 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.058 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.059 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.059 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.059 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.059 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.059 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.059 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.059 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.059 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.060 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:28.060 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:28.060 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.060 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.060 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.060 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.060 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.060 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.061 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.061 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.061 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.061 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.061 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.061 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.061 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_destination": { 2024-09-11T22:07:28.062 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_destination", 2024-09-11T22:07:28.062 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.062 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.062 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.062 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.062 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.062 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.062 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.063 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Email address to send alerts to", 2024-09-11T22:07:28.063 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.063 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.063 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.063 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.063 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_from_name": { 2024-09-11T22:07:28.063 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_from_name", 2024-09-11T22:07:28.064 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.064 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.064 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.064 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "Ceph", 2024-09-11T22:07:28.064 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.064 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.064 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.064 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Email From: name", 2024-09-11T22:07:28.065 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.065 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.065 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.065 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.065 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_host": { 2024-09-11T22:07:28.065 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_host", 2024-09-11T22:07:28.065 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.065 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.066 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.066 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.066 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.066 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.066 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.066 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SMTP server", 2024-09-11T22:07:28.066 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.066 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.067 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.067 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.067 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_password": { 2024-09-11T22:07:28.067 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_password", 2024-09-11T22:07:28.067 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.067 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.067 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.067 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.068 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.068 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.068 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.068 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Password to authenticate with", 2024-09-11T22:07:28.068 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.068 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.068 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.068 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.069 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_port": { 2024-09-11T22:07:28.069 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_port", 2024-09-11T22:07:28.069 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:28.069 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.069 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.069 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "465", 2024-09-11T22:07:28.069 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.069 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.069 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.070 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SMTP port", 2024-09-11T22:07:28.070 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.070 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.070 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.070 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.070 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_sender": { 2024-09-11T22:07:28.070 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_sender", 2024-09-11T22:07:28.070 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.071 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.071 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.071 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.071 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.071 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.071 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.071 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SMTP envelope sender", 2024-09-11T22:07:28.071 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.072 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.072 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.072 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.072 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_ssl": { 2024-09-11T22:07:28.072 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_ssl", 2024-09-11T22:07:28.072 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.072 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.072 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.073 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:28.073 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.073 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.073 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.073 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-11T22:07:28.073 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.073 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.073 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.074 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.074 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_user": { 2024-09-11T22:07:28.074 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_user", 2024-09-11T22:07:28.074 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.074 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.074 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.074 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.074 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.075 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.075 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.075 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "User to authenticate as", 2024-09-11T22:07:28.075 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.075 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.075 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.075 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.075 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.076 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.076 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:28.076 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "balancer", 2024-09-11T22:07:28.076 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:28.076 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:28.076 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:28.076 INFO:teuthology.orchestra.run.smithi001.stdout: "active": { 2024-09-11T22:07:28.076 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "active", 2024-09-11T22:07:28.077 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.077 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.077 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.077 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:28.077 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.077 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.077 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.077 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "automatically balance PGs across cluster", 2024-09-11T22:07:28.077 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.078 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.078 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.078 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.078 INFO:teuthology.orchestra.run.smithi001.stdout: "begin_time": { 2024-09-11T22:07:28.078 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "begin_time", 2024-09-11T22:07:28.078 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.078 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.079 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.079 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0000", 2024-09-11T22:07:28.079 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.079 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.079 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.079 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "beginning time of day to automatically balance", 2024-09-11T22:07:28.079 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-11T22:07:28.079 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.080 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.080 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.080 INFO:teuthology.orchestra.run.smithi001.stdout: "begin_weekday": { 2024-09-11T22:07:28.080 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "begin_weekday", 2024-09-11T22:07:28.080 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-09-11T22:07:28.080 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.080 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.081 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-09-11T22:07:28.081 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "0", 2024-09-11T22:07:28.081 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "6", 2024-09-11T22:07:28.081 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.081 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-11T22:07:28.081 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-11T22:07:28.081 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.081 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.082 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.082 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_compat_max_iterations": { 2024-09-11T22:07:28.082 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crush_compat_max_iterations", 2024-09-11T22:07:28.082 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-09-11T22:07:28.082 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.082 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.082 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "25", 2024-09-11T22:07:28.082 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-09-11T22:07:28.083 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "250", 2024-09-11T22:07:28.083 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.083 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-11T22:07:28.083 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.083 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.083 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.083 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.083 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_compat_metrics": { 2024-09-11T22:07:28.084 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crush_compat_metrics", 2024-09-11T22:07:28.084 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.084 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.084 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.084 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "pgs,objects,bytes", 2024-09-11T22:07:28.084 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.084 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.084 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.085 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-11T22:07:28.085 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-11T22:07:28.085 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.085 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.085 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.085 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_compat_step": { 2024-09-11T22:07:28.085 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crush_compat_step", 2024-09-11T22:07:28.085 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-09-11T22:07:28.086 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.086 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.086 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0.5", 2024-09-11T22:07:28.086 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "0.001", 2024-09-11T22:07:28.086 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "0.999", 2024-09-11T22:07:28.086 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.086 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "aggressiveness of optimization", 2024-09-11T22:07:28.086 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-11T22:07:28.087 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.087 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.087 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.087 INFO:teuthology.orchestra.run.smithi001.stdout: "end_time": { 2024-09-11T22:07:28.087 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "end_time", 2024-09-11T22:07:28.087 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.087 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.087 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.088 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "2359", 2024-09-11T22:07:28.088 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.088 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.088 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.088 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "ending time of day to automatically balance", 2024-09-11T22:07:28.088 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-11T22:07:28.088 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.089 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.089 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.089 INFO:teuthology.orchestra.run.smithi001.stdout: "end_weekday": { 2024-09-11T22:07:28.089 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "end_weekday", 2024-09-11T22:07:28.089 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-09-11T22:07:28.089 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.089 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.089 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-09-11T22:07:28.090 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "0", 2024-09-11T22:07:28.090 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "6", 2024-09-11T22:07:28.090 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.090 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-11T22:07:28.090 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-11T22:07:28.090 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.090 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.090 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.091 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:28.091 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:28.091 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.091 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.091 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.091 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.091 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.091 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.092 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.092 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.092 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.092 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.092 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.092 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.092 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.092 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.093 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.093 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.093 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.093 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.093 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.093 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:28.093 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:28.094 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.094 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.094 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.094 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.094 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.094 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.094 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.094 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.095 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.095 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.095 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.095 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.095 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:28.095 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:28.095 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.095 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.096 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.096 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:28.096 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.096 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.096 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.096 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.096 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.096 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.097 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.097 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.097 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.097 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.097 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.097 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.097 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.097 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.098 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.098 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:28.098 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:28.098 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.098 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.098 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.098 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.098 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.099 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.099 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.099 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.099 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.099 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.099 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.099 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.099 INFO:teuthology.orchestra.run.smithi001.stdout: "min_score": { 2024-09-11T22:07:28.099 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "min_score", 2024-09-11T22:07:28.100 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-09-11T22:07:28.100 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.100 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.100 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-09-11T22:07:28.100 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.100 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.100 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.101 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-11T22:07:28.101 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.101 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.101 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.101 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.101 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": { 2024-09-11T22:07:28.101 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mode", 2024-09-11T22:07:28.101 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.102 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.102 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.102 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "upmap", 2024-09-11T22:07:28.102 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.102 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.102 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.102 INFO:teuthology.orchestra.run.smithi001.stdout: "crush-compat", 2024-09-11T22:07:28.102 INFO:teuthology.orchestra.run.smithi001.stdout: "none", 2024-09-11T22:07:28.103 INFO:teuthology.orchestra.run.smithi001.stdout: "read", 2024-09-11T22:07:28.103 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap", 2024-09-11T22:07:28.103 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap-read" 2024-09-11T22:07:28.103 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.103 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Balancer mode", 2024-09-11T22:07:28.103 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.103 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.103 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.103 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.104 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_ids": { 2024-09-11T22:07:28.104 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pool_ids", 2024-09-11T22:07:28.104 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.104 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.104 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.104 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.104 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.104 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.105 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.105 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-11T22:07:28.105 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.105 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.105 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.105 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.105 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-09-11T22:07:28.105 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-09-11T22:07:28.106 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-11T22:07:28.106 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.106 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.106 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-09-11T22:07:28.106 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.106 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.106 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.106 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-11T22:07:28.107 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.107 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.107 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.107 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.107 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap_max_deviation": { 2024-09-11T22:07:28.107 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "upmap_max_deviation", 2024-09-11T22:07:28.107 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:28.107 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.108 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.108 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-09-11T22:07:28.108 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-09-11T22:07:28.108 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.108 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.108 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-11T22:07:28.108 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-11T22:07:28.109 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.109 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.109 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.109 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap_max_optimizations": { 2024-09-11T22:07:28.109 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "upmap_max_optimizations", 2024-09-11T22:07:28.109 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-09-11T22:07:28.109 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.109 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.110 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-11T22:07:28.110 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.110 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.110 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.110 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-11T22:07:28.110 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.110 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.110 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.110 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.111 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.111 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.111 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:28.111 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cephadm", 2024-09-11T22:07:28.111 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:28.111 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:28.111 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:28.111 INFO:teuthology.orchestra.run.smithi001.stdout: "agent_down_multiplier": { 2024-09-11T22:07:28.112 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "agent_down_multiplier", 2024-09-11T22:07:28.112 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-09-11T22:07:28.112 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.112 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.112 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "3.0", 2024-09-11T22:07:28.112 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.112 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.112 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.113 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-11T22:07:28.113 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.113 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.113 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.113 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.113 INFO:teuthology.orchestra.run.smithi001.stdout: "agent_refresh_rate": { 2024-09-11T22:07:28.113 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "agent_refresh_rate", 2024-09-11T22:07:28.113 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-11T22:07:28.114 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.114 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.114 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "20", 2024-09-11T22:07:28.114 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.114 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.114 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.114 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-11T22:07:28.114 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.115 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.115 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.115 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.115 INFO:teuthology.orchestra.run.smithi001.stdout: "agent_starting_port": { 2024-09-11T22:07:28.115 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "agent_starting_port", 2024-09-11T22:07:28.115 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:28.115 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.115 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.115 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "4721", 2024-09-11T22:07:28.116 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.116 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.116 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.116 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-11T22:07:28.116 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.116 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.116 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.116 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.117 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_ptrace": { 2024-09-11T22:07:28.117 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "allow_ptrace", 2024-09-11T22:07:28.117 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.117 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.117 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.117 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.117 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.117 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.118 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.118 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-11T22:07:28.118 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-11T22:07:28.118 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.118 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.118 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.118 INFO:teuthology.orchestra.run.smithi001.stdout: "autotune_interval": { 2024-09-11T22:07:28.118 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "autotune_interval", 2024-09-11T22:07:28.119 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-11T22:07:28.119 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.119 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.119 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-09-11T22:07:28.119 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.119 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.119 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.119 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-11T22:07:28.120 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.120 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.120 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.120 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.120 INFO:teuthology.orchestra.run.smithi001.stdout: "autotune_memory_target_ratio": { 2024-09-11T22:07:28.120 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "autotune_memory_target_ratio", 2024-09-11T22:07:28.120 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-09-11T22:07:28.121 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.121 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.121 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0.7", 2024-09-11T22:07:28.121 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.121 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.121 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.121 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-11T22:07:28.121 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.122 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.122 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.122 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.122 INFO:teuthology.orchestra.run.smithi001.stdout: "cephadm_log_destination": { 2024-09-11T22:07:28.122 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cephadm_log_destination", 2024-09-11T22:07:28.122 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.122 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.122 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.123 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.123 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.123 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.123 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.123 INFO:teuthology.orchestra.run.smithi001.stdout: "file", 2024-09-11T22:07:28.123 INFO:teuthology.orchestra.run.smithi001.stdout: "file,syslog", 2024-09-11T22:07:28.123 INFO:teuthology.orchestra.run.smithi001.stdout: "syslog" 2024-09-11T22:07:28.123 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.123 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-11T22:07:28.124 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.124 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.124 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.124 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.124 INFO:teuthology.orchestra.run.smithi001.stdout: "cgroups_split": { 2024-09-11T22:07:28.124 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cgroups_split", 2024-09-11T22:07:28.124 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.124 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.125 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.125 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:28.125 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.125 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.125 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.125 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-11T22:07:28.125 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.125 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.126 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.126 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.126 INFO:teuthology.orchestra.run.smithi001.stdout: "config_checks_enabled": { 2024-09-11T22:07:28.126 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "config_checks_enabled", 2024-09-11T22:07:28.126 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.126 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.126 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.126 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.126 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.127 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.127 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.127 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-11T22:07:28.127 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.127 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.127 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.127 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.127 INFO:teuthology.orchestra.run.smithi001.stdout: "config_dashboard": { 2024-09-11T22:07:28.128 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "config_dashboard", 2024-09-11T22:07:28.128 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.128 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.128 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.128 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:28.128 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.128 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.129 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.129 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-11T22:07:28.129 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.129 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.129 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.129 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.129 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_alertmanager": { 2024-09-11T22:07:28.129 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_alertmanager", 2024-09-11T22:07:28.130 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.130 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.130 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.130 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-11T22:07:28.130 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.130 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.130 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.130 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-09-11T22:07:28.131 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.131 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.131 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.131 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.131 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_base": { 2024-09-11T22:07:28.131 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_base", 2024-09-11T22:07:28.131 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.131 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.132 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.132 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-11T22:07:28.132 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.132 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.132 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.132 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Container image name, without the tag", 2024-09-11T22:07:28.132 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.133 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.133 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.133 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.133 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_elasticsearch": { 2024-09-11T22:07:28.133 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_elasticsearch", 2024-09-11T22:07:28.133 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.133 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.134 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.134 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-11T22:07:28.134 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.134 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.134 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.134 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "elasticsearch container image", 2024-09-11T22:07:28.134 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.134 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.135 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.135 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.135 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_grafana": { 2024-09-11T22:07:28.135 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_grafana", 2024-09-11T22:07:28.135 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.135 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.135 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.135 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-11T22:07:28.136 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.136 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.136 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.136 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-09-11T22:07:28.136 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.136 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.136 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.136 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.137 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_haproxy": { 2024-09-11T22:07:28.137 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_haproxy", 2024-09-11T22:07:28.137 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.137 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.137 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.137 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-11T22:07:28.137 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.137 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.138 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.138 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "HAproxy container image", 2024-09-11T22:07:28.138 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.138 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.138 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.138 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.138 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_jaeger_agent": { 2024-09-11T22:07:28.138 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_jaeger_agent", 2024-09-11T22:07:28.139 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.139 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.139 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.139 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-11T22:07:28.139 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.139 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.139 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.140 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Jaeger agent container image", 2024-09-11T22:07:28.140 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.140 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.140 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.140 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.140 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_jaeger_collector": { 2024-09-11T22:07:28.140 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_jaeger_collector", 2024-09-11T22:07:28.140 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.141 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.141 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.141 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-11T22:07:28.141 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.141 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.141 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.141 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Jaeger collector container image", 2024-09-11T22:07:28.141 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.141 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.142 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.142 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.142 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_jaeger_query": { 2024-09-11T22:07:28.142 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_jaeger_query", 2024-09-11T22:07:28.142 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.142 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.142 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.142 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-11T22:07:28.143 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.143 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.143 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.143 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Jaeger query container image", 2024-09-11T22:07:28.143 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.143 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.143 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.143 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.143 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_keepalived": { 2024-09-11T22:07:28.144 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_keepalived", 2024-09-11T22:07:28.144 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.144 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.144 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.144 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-11T22:07:28.144 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.144 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.144 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.145 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Keepalived container image", 2024-09-11T22:07:28.145 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.145 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.145 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.145 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.145 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_loki": { 2024-09-11T22:07:28.145 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_loki", 2024-09-11T22:07:28.145 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.146 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.146 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.146 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-11T22:07:28.146 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.146 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.146 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.146 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Loki container image", 2024-09-11T22:07:28.147 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.147 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.147 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.147 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.147 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_node_exporter": { 2024-09-11T22:07:28.147 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_node_exporter", 2024-09-11T22:07:28.147 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.147 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.148 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.148 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-11T22:07:28.148 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.148 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.148 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.148 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-09-11T22:07:28.148 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.149 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.149 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.149 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.149 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_nvmeof": { 2024-09-11T22:07:28.149 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_nvmeof", 2024-09-11T22:07:28.149 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.149 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.149 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.150 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-11T22:07:28.150 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.150 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.150 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.150 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Nvme-of container image", 2024-09-11T22:07:28.150 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.150 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.150 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.151 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.151 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_prometheus": { 2024-09-11T22:07:28.151 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_prometheus", 2024-09-11T22:07:28.151 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.151 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.151 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.151 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-11T22:07:28.151 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.151 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.152 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.152 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-09-11T22:07:28.152 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.152 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.152 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.152 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.153 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_promtail": { 2024-09-11T22:07:28.153 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_promtail", 2024-09-11T22:07:28.153 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.153 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.153 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.153 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-11T22:07:28.154 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.154 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.154 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.154 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Promtail container image", 2024-09-11T22:07:28.154 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.154 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.154 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.155 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.155 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_samba": { 2024-09-11T22:07:28.155 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_samba", 2024-09-11T22:07:28.155 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.155 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.155 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.155 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-11T22:07:28.155 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.156 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.156 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.156 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Samba/SMB container image", 2024-09-11T22:07:28.156 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.156 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.156 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.156 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.156 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_snmp_gateway": { 2024-09-11T22:07:28.157 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_snmp_gateway", 2024-09-11T22:07:28.157 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.157 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.157 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.157 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-11T22:07:28.157 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.157 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.157 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.158 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SNMP Gateway container image", 2024-09-11T22:07:28.158 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.158 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.158 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.158 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.158 INFO:teuthology.orchestra.run.smithi001.stdout: "container_init": { 2024-09-11T22:07:28.158 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_init", 2024-09-11T22:07:28.159 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.159 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.159 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.159 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:28.159 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.159 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.159 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.159 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Run podman/docker with `--init`", 2024-09-11T22:07:28.160 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.160 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.160 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.160 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.160 INFO:teuthology.orchestra.run.smithi001.stdout: "daemon_cache_timeout": { 2024-09-11T22:07:28.160 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "daemon_cache_timeout", 2024-09-11T22:07:28.160 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-11T22:07:28.160 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.161 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.161 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-09-11T22:07:28.161 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.161 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.161 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.161 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-11T22:07:28.161 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.161 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.161 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.162 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.162 INFO:teuthology.orchestra.run.smithi001.stdout: "default_cephadm_command_timeout": { 2024-09-11T22:07:28.162 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "default_cephadm_command_timeout", 2024-09-11T22:07:28.162 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:28.162 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.162 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.162 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "900", 2024-09-11T22:07:28.162 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.163 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.163 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.163 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-11T22:07:28.163 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.163 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.163 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.163 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.163 INFO:teuthology.orchestra.run.smithi001.stdout: "default_registry": { 2024-09-11T22:07:28.164 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "default_registry", 2024-09-11T22:07:28.164 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.164 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.164 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.164 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io", 2024-09-11T22:07:28.164 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.164 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.164 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.165 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-11T22:07:28.165 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.165 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.165 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.165 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.165 INFO:teuthology.orchestra.run.smithi001.stdout: "device_cache_timeout": { 2024-09-11T22:07:28.165 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "device_cache_timeout", 2024-09-11T22:07:28.165 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-11T22:07:28.166 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.166 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.166 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "1800", 2024-09-11T22:07:28.166 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.166 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.166 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.166 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "seconds to cache device inventory", 2024-09-11T22:07:28.166 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.167 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.167 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.167 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.167 INFO:teuthology.orchestra.run.smithi001.stdout: "device_enhanced_scan": { 2024-09-11T22:07:28.167 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "device_enhanced_scan", 2024-09-11T22:07:28.167 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.167 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.167 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.168 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.168 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.168 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.168 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.168 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-11T22:07:28.168 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.168 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.168 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.169 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.169 INFO:teuthology.orchestra.run.smithi001.stdout: "facts_cache_timeout": { 2024-09-11T22:07:28.169 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "facts_cache_timeout", 2024-09-11T22:07:28.169 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-11T22:07:28.169 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.169 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.169 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-09-11T22:07:28.169 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.170 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.170 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.170 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "seconds to cache host facts data", 2024-09-11T22:07:28.170 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.170 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.170 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.170 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.170 INFO:teuthology.orchestra.run.smithi001.stdout: "grafana_dashboards_path": { 2024-09-11T22:07:28.170 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "grafana_dashboards_path", 2024-09-11T22:07:28.171 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.171 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.171 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.171 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-11T22:07:28.171 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.171 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.171 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.172 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-11T22:07:28.172 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.172 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.172 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.172 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.172 INFO:teuthology.orchestra.run.smithi001.stdout: "host_check_interval": { 2024-09-11T22:07:28.172 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "host_check_interval", 2024-09-11T22:07:28.172 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-11T22:07:28.173 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.173 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.173 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-09-11T22:07:28.173 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.173 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.173 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.173 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to perform a host check", 2024-09-11T22:07:28.173 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.174 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.174 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.174 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.174 INFO:teuthology.orchestra.run.smithi001.stdout: "hw_monitoring": { 2024-09-11T22:07:28.174 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "hw_monitoring", 2024-09-11T22:07:28.174 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.174 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.174 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.175 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.175 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.175 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.175 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.175 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-11T22:07:28.175 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.175 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.175 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.176 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.176 INFO:teuthology.orchestra.run.smithi001.stdout: "inventory_list_all": { 2024-09-11T22:07:28.176 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "inventory_list_all", 2024-09-11T22:07:28.176 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.176 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.176 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.176 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.176 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.177 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.177 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.177 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-11T22:07:28.177 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.177 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.177 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.177 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.177 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:28.178 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:28.178 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.178 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.178 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.178 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.178 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.178 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.178 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.179 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.179 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.179 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.179 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.179 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.179 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.179 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.179 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.180 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.180 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.180 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.180 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.180 INFO:teuthology.orchestra.run.smithi001.stdout: "log_refresh_metadata": { 2024-09-11T22:07:28.180 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_refresh_metadata", 2024-09-11T22:07:28.180 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.181 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.181 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.181 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.181 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.181 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.181 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.181 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-11T22:07:28.181 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.182 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.182 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.182 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.182 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:28.182 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:28.182 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.182 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.182 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.183 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:28.183 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.183 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.183 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.183 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-11T22:07:28.183 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.183 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.183 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.184 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.184 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:28.184 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:28.184 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.184 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.184 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.184 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:28.185 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.185 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.185 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.185 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.185 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.185 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.185 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.186 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.186 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.186 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.186 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.186 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.186 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.186 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.186 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.187 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:28.187 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:28.187 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.187 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.187 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.187 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.187 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.187 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.188 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.188 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.188 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.188 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.188 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.188 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.188 INFO:teuthology.orchestra.run.smithi001.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-11T22:07:28.188 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-11T22:07:28.189 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.189 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.189 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.189 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.189 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.189 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.189 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.189 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-11T22:07:28.190 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.190 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.190 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.190 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.190 INFO:teuthology.orchestra.run.smithi001.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-11T22:07:28.190 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-11T22:07:28.190 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.190 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.191 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.191 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "*", 2024-09-11T22:07:28.191 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.191 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.191 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.191 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-11T22:07:28.191 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.191 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.192 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.192 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.192 INFO:teuthology.orchestra.run.smithi001.stdout: "max_count_per_host": { 2024-09-11T22:07:28.192 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_count_per_host", 2024-09-11T22:07:28.192 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:28.192 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.192 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.192 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-11T22:07:28.193 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.193 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.193 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.193 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "max number of daemons per service per host", 2024-09-11T22:07:28.193 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.193 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.193 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.193 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.194 INFO:teuthology.orchestra.run.smithi001.stdout: "max_osd_draining_count": { 2024-09-11T22:07:28.194 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_osd_draining_count", 2024-09-11T22:07:28.194 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:28.194 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.194 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.194 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-11T22:07:28.194 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.194 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.195 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.195 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-11T22:07:28.195 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.195 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.195 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.195 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.195 INFO:teuthology.orchestra.run.smithi001.stdout: "migration_current": { 2024-09-11T22:07:28.195 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "migration_current", 2024-09-11T22:07:28.196 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:28.196 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.196 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.196 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.196 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.196 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.196 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.196 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "internal - do not modify", 2024-09-11T22:07:28.197 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.197 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.197 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.197 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.197 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": { 2024-09-11T22:07:28.197 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mode", 2024-09-11T22:07:28.197 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.198 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.198 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.198 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "root", 2024-09-11T22:07:28.198 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.198 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.198 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.198 INFO:teuthology.orchestra.run.smithi001.stdout: "cephadm-package", 2024-09-11T22:07:28.198 INFO:teuthology.orchestra.run.smithi001.stdout: "root" 2024-09-11T22:07:28.198 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.199 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "mode for remote execution of cephadm", 2024-09-11T22:07:28.199 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.199 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.199 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.199 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.199 INFO:teuthology.orchestra.run.smithi001.stdout: "oob_default_addr": { 2024-09-11T22:07:28.199 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "oob_default_addr", 2024-09-11T22:07:28.199 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.200 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.200 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.200 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "169.254.1.1", 2024-09-11T22:07:28.200 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.200 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.200 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.200 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-11T22:07:28.200 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.201 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.201 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.201 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.201 INFO:teuthology.orchestra.run.smithi001.stdout: "prometheus_alerts_path": { 2024-09-11T22:07:28.201 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "prometheus_alerts_path", 2024-09-11T22:07:28.201 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.201 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.201 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.202 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-11T22:07:28.202 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.202 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.202 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.202 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-11T22:07:28.202 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.202 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.202 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.203 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.203 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_insecure": { 2024-09-11T22:07:28.203 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_insecure", 2024-09-11T22:07:28.203 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.203 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.203 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.203 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.203 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.204 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.204 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.204 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-11T22:07:28.204 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.204 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.204 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.204 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.204 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_password": { 2024-09-11T22:07:28.205 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_password", 2024-09-11T22:07:28.205 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.205 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.205 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.205 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.205 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.205 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.205 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.206 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-11T22:07:28.206 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.206 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.206 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.206 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.206 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_url": { 2024-09-11T22:07:28.206 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_url", 2024-09-11T22:07:28.206 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.207 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.207 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.207 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.207 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.207 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.207 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.207 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-11T22:07:28.207 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.208 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.208 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.208 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.208 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_username": { 2024-09-11T22:07:28.208 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_username", 2024-09-11T22:07:28.208 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.208 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.208 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.209 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.209 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.209 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.209 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.209 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-11T22:07:28.209 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.209 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.209 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.210 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.210 INFO:teuthology.orchestra.run.smithi001.stdout: "secure_monitoring_stack": { 2024-09-11T22:07:28.210 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "secure_monitoring_stack", 2024-09-11T22:07:28.210 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.210 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.210 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.210 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.210 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.211 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.211 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.211 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-11T22:07:28.211 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.211 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.211 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.211 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.212 INFO:teuthology.orchestra.run.smithi001.stdout: "service_discovery_port": { 2024-09-11T22:07:28.212 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "service_discovery_port", 2024-09-11T22:07:28.212 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:28.212 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.212 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.212 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8765", 2024-09-11T22:07:28.212 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.212 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.213 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.213 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "cephadm service discovery port", 2024-09-11T22:07:28.213 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.213 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.213 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.213 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.213 INFO:teuthology.orchestra.run.smithi001.stdout: "ssh_config_file": { 2024-09-11T22:07:28.214 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssh_config_file", 2024-09-11T22:07:28.214 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.214 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.214 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.214 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.214 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.214 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.215 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.215 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-11T22:07:28.215 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.215 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.215 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.215 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.215 INFO:teuthology.orchestra.run.smithi001.stdout: "use_agent": { 2024-09-11T22:07:28.215 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "use_agent", 2024-09-11T22:07:28.216 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.216 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.216 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.216 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.216 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.216 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.216 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.216 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-11T22:07:28.217 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.217 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.217 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.217 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.217 INFO:teuthology.orchestra.run.smithi001.stdout: "use_repo_digest": { 2024-09-11T22:07:28.217 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "use_repo_digest", 2024-09-11T22:07:28.217 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.217 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.218 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.218 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:28.218 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.218 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.218 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.218 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-11T22:07:28.218 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.218 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.219 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.219 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.219 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_on_failed_host_check": { 2024-09-11T22:07:28.219 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_on_failed_host_check", 2024-09-11T22:07:28.219 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.219 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.219 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.219 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:28.220 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.220 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.220 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.220 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise a health warning if the host check fails", 2024-09-11T22:07:28.220 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.220 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.220 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.221 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.221 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_on_stray_daemons": { 2024-09-11T22:07:28.221 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_on_stray_daemons", 2024-09-11T22:07:28.221 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.221 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.221 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.221 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:28.221 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.222 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.222 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.222 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-11T22:07:28.222 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.222 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.222 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.222 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.222 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_on_stray_hosts": { 2024-09-11T22:07:28.223 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_on_stray_hosts", 2024-09-11T22:07:28.223 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.223 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.223 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.223 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:28.223 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.223 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.223 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.224 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-11T22:07:28.224 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.224 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.224 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.224 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.224 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.224 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.225 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:28.225 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crash", 2024-09-11T22:07:28.225 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:28.225 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:28.225 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:28.225 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:28.225 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:28.225 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.226 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.226 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.226 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.226 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.226 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.226 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.226 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.226 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.227 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.227 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.227 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.227 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.227 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.227 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.227 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.227 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.228 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.228 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.228 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:28.228 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:28.228 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.228 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.228 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.228 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.229 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.229 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.229 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.229 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.229 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.229 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.229 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.229 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.230 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:28.230 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:28.230 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.230 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.230 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.230 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:28.230 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.230 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.231 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.231 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.231 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.231 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.231 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.231 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.231 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.231 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.232 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.232 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.232 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.232 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.232 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.232 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:28.232 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:28.232 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.233 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.233 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.233 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.233 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.233 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.233 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.233 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.234 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.234 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.234 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.234 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.234 INFO:teuthology.orchestra.run.smithi001.stdout: "retain_interval": { 2024-09-11T22:07:28.234 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "retain_interval", 2024-09-11T22:07:28.234 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-11T22:07:28.234 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.235 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.235 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "31536000", 2024-09-11T22:07:28.235 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.235 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.235 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.235 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-11T22:07:28.235 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.235 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.236 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.236 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.236 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_recent_interval": { 2024-09-11T22:07:28.236 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_recent_interval", 2024-09-11T22:07:28.236 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-11T22:07:28.236 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.236 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.236 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "1209600", 2024-09-11T22:07:28.237 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.237 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.237 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.237 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-11T22:07:28.237 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.237 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.237 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.237 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.238 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.238 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.238 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:28.238 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "dashboard", 2024-09-11T22:07:28.238 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:28.238 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:28.238 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:28.238 INFO:teuthology.orchestra.run.smithi001.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-11T22:07:28.239 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-11T22:07:28.239 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:28.239 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.239 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.239 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-11T22:07:28.239 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.239 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.240 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.240 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.240 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.240 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.240 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.240 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.240 INFO:teuthology.orchestra.run.smithi001.stdout: "ALERTMANAGER_API_HOST": { 2024-09-11T22:07:28.240 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-11T22:07:28.241 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.241 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.241 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.241 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.241 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.241 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.241 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.241 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.242 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.242 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.242 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.242 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.242 INFO:teuthology.orchestra.run.smithi001.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-11T22:07:28.242 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-11T22:07:28.242 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.243 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.243 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.243 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:28.243 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.243 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.243 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.243 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.243 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.244 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.244 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.244 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.244 INFO:teuthology.orchestra.run.smithi001.stdout: "AUDIT_API_ENABLED": { 2024-09-11T22:07:28.244 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "AUDIT_API_ENABLED", 2024-09-11T22:07:28.244 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.244 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.244 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.245 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.245 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.245 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.245 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.245 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.245 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.245 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.246 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.246 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.246 INFO:teuthology.orchestra.run.smithi001.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-11T22:07:28.246 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-11T22:07:28.246 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.246 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.246 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.246 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:28.247 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.247 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.247 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.247 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.247 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.247 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.247 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.247 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.248 INFO:teuthology.orchestra.run.smithi001.stdout: "ENABLE_BROWSABLE_API": { 2024-09-11T22:07:28.248 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-11T22:07:28.248 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.248 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.248 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.248 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:28.248 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.248 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.249 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.249 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.249 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.249 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.249 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.249 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.249 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-11T22:07:28.249 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-11T22:07:28.249 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.250 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.250 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.250 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:28.250 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.250 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.250 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.250 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.250 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.251 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.251 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.251 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.251 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-11T22:07:28.251 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-11T22:07:28.251 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.251 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.251 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.252 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:28.252 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.252 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.252 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.252 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.252 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.252 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.252 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.252 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.253 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-11T22:07:28.253 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-11T22:07:28.253 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.253 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.253 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.253 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:28.253 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.254 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.254 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.254 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.254 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.254 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.254 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.254 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.254 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-11T22:07:28.254 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-11T22:07:28.255 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.255 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.255 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.255 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:28.255 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.255 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.255 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.255 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.256 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.256 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.256 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.256 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.256 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-11T22:07:28.256 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-11T22:07:28.256 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.256 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.257 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.257 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:28.257 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.257 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.257 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.257 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.257 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.257 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.258 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.258 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.258 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-11T22:07:28.258 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-11T22:07:28.258 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.258 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.258 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.258 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:28.259 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.259 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.259 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.259 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.259 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.259 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.259 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.259 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.260 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-11T22:07:28.260 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-11T22:07:28.260 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.260 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.260 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.260 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:28.260 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.261 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.261 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.261 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.261 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.261 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.261 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.261 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.261 INFO:teuthology.orchestra.run.smithi001.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-11T22:07:28.261 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-11T22:07:28.262 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.262 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.262 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.262 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.262 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.262 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.262 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.263 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.263 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.263 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.263 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.263 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.263 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_PASSWORD": { 2024-09-11T22:07:28.263 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-11T22:07:28.263 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.263 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.264 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.264 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2024-09-11T22:07:28.264 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.264 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.264 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.264 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.264 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.264 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.265 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.265 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.265 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-11T22:07:28.265 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-11T22:07:28.265 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.265 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.265 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.265 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:28.266 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.266 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.266 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.266 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.266 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.266 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.266 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.266 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.267 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_URL": { 2024-09-11T22:07:28.267 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_URL", 2024-09-11T22:07:28.267 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.267 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.267 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.267 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.267 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.268 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.268 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.268 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.268 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.268 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.268 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.268 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.269 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_USERNAME": { 2024-09-11T22:07:28.269 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-11T22:07:28.269 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.269 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.269 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.269 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2024-09-11T22:07:28.269 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.269 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.270 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.270 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.270 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.270 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.270 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.270 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.270 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-11T22:07:28.271 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-11T22:07:28.271 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.271 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.271 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.271 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.271 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.271 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.271 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.272 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.272 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.272 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.272 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.272 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.272 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-11T22:07:28.272 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-11T22:07:28.272 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.273 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.273 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.273 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.273 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.273 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.273 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.273 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.274 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.274 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.274 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.274 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.274 INFO:teuthology.orchestra.run.smithi001.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-11T22:07:28.274 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-11T22:07:28.274 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.274 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.275 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.275 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:28.275 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.275 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.275 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.275 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.275 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.275 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.276 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.276 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.276 INFO:teuthology.orchestra.run.smithi001.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-11T22:07:28.276 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-11T22:07:28.276 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.276 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.276 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.276 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.277 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.277 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.277 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.277 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.277 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.277 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.277 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.277 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.278 INFO:teuthology.orchestra.run.smithi001.stdout: "PROMETHEUS_API_HOST": { 2024-09-11T22:07:28.278 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-11T22:07:28.278 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.278 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.278 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.278 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.278 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.278 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.279 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.279 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.279 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.279 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.279 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.279 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.279 INFO:teuthology.orchestra.run.smithi001.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-11T22:07:28.279 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-11T22:07:28.280 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.280 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.280 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.280 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:28.280 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.280 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.280 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.280 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.281 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.281 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.281 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.281 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.281 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-11T22:07:28.281 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-11T22:07:28.281 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.281 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.282 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.282 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.282 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.282 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.282 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.282 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.282 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.282 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.283 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.283 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.283 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-11T22:07:28.283 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-11T22:07:28.283 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.283 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.283 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.283 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.284 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.284 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.284 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.284 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.284 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.284 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.284 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.284 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.285 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-11T22:07:28.285 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-11T22:07:28.285 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.285 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.285 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.285 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:28.285 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.285 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.286 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.286 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.286 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.286 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.286 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.286 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.286 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-11T22:07:28.286 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-11T22:07:28.287 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.287 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.287 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.287 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:28.287 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.287 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.287 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.287 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.288 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.288 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.288 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.288 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.288 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-11T22:07:28.288 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-11T22:07:28.288 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.288 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.289 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.289 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.289 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.289 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.289 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.289 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.289 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.289 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.290 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.290 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.290 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-11T22:07:28.290 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-11T22:07:28.290 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.290 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.290 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.290 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.291 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.291 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.291 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.291 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.291 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.291 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.291 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.292 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.292 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-11T22:07:28.292 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-11T22:07:28.292 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.292 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.292 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.292 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.293 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.293 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.293 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.293 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.293 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.293 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.293 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.293 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.294 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_ENABLED": { 2024-09-11T22:07:28.294 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-11T22:07:28.294 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.294 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.294 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.294 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:28.294 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.294 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.295 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.295 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.295 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.295 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.295 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.295 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.295 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-11T22:07:28.295 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-11T22:07:28.296 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.296 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.296 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.296 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-11T22:07:28.296 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.296 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.296 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.296 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.297 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.297 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.297 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.297 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.297 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-11T22:07:28.297 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-11T22:07:28.297 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:28.297 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.298 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.298 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-11T22:07:28.298 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.298 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.298 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.298 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.298 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.298 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.299 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.299 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.299 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-11T22:07:28.299 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-11T22:07:28.299 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:28.299 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.299 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.299 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8", 2024-09-11T22:07:28.300 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.300 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.300 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.300 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.300 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.300 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.300 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.301 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.301 INFO:teuthology.orchestra.run.smithi001.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-11T22:07:28.301 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-11T22:07:28.301 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:28.301 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.301 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.301 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "45", 2024-09-11T22:07:28.301 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.302 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.302 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.302 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.302 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.302 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.302 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.302 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.303 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_ACCESS_KEY": { 2024-09-11T22:07:28.303 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-11T22:07:28.303 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.303 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.303 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.303 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.304 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.304 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.304 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.304 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.304 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.304 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.304 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.305 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.305 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-11T22:07:28.305 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-11T22:07:28.305 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.305 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.305 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.305 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2024-09-11T22:07:28.305 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.306 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.306 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.306 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.306 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.306 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.306 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.306 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.306 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_SECRET_KEY": { 2024-09-11T22:07:28.307 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-11T22:07:28.307 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.307 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.307 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.307 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.307 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.307 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.308 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.308 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.308 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.308 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.308 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.308 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.309 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_SSL_VERIFY": { 2024-09-11T22:07:28.309 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-11T22:07:28.309 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.309 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.309 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.309 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:28.310 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.310 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.310 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.310 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.310 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.310 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.310 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.310 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.311 INFO:teuthology.orchestra.run.smithi001.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-11T22:07:28.311 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-11T22:07:28.311 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:28.311 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.311 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.311 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-09-11T22:07:28.311 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.312 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.312 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.312 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.312 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.312 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.312 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.312 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.313 INFO:teuthology.orchestra.run.smithi001.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-11T22:07:28.313 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-11T22:07:28.313 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:28.313 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.313 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.313 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-11T22:07:28.313 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.314 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.314 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.314 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.314 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.314 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.314 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.314 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.314 INFO:teuthology.orchestra.run.smithi001.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-11T22:07:28.315 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-11T22:07:28.315 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:28.315 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.315 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.315 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-09-11T22:07:28.315 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.315 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.315 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.316 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.316 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.316 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.316 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.316 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.316 INFO:teuthology.orchestra.run.smithi001.stdout: "cross_origin_url": { 2024-09-11T22:07:28.316 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cross_origin_url", 2024-09-11T22:07:28.316 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.317 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.317 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.317 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.317 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.317 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.317 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.317 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.317 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.318 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.318 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.318 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.318 INFO:teuthology.orchestra.run.smithi001.stdout: "crt_file": { 2024-09-11T22:07:28.318 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crt_file", 2024-09-11T22:07:28.318 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.318 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.318 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.319 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.319 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.319 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.319 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.319 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.319 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.319 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.319 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.320 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.320 INFO:teuthology.orchestra.run.smithi001.stdout: "debug": { 2024-09-11T22:07:28.320 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "debug", 2024-09-11T22:07:28.320 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.320 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.320 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.320 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.321 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.321 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.321 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.321 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Enable/disable debug options", 2024-09-11T22:07:28.321 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.321 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.321 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.322 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.322 INFO:teuthology.orchestra.run.smithi001.stdout: "jwt_token_ttl": { 2024-09-11T22:07:28.322 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "jwt_token_ttl", 2024-09-11T22:07:28.322 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:28.322 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.322 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.322 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "28800", 2024-09-11T22:07:28.322 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.323 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.323 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.323 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.323 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.323 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.323 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.323 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.324 INFO:teuthology.orchestra.run.smithi001.stdout: "key_file": { 2024-09-11T22:07:28.324 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "key_file", 2024-09-11T22:07:28.324 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.324 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.324 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.324 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.324 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.325 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.325 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.325 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.325 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.325 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.325 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.325 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.325 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:28.326 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:28.326 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.326 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.326 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.326 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.326 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.326 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.327 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.327 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.327 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.327 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.327 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.327 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.327 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.328 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.328 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.328 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.328 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.328 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.328 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.328 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:28.328 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:28.329 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.329 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.329 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.329 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.329 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.329 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.329 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.329 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.330 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.330 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.330 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.330 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.330 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:28.330 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:28.330 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.331 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.331 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.331 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:28.331 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.331 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.331 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.331 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.331 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.332 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.332 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.332 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.332 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.332 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.332 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.332 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.333 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.333 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.333 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.333 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:28.333 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:28.333 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.333 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.334 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.334 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.334 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.334 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.334 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.334 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.334 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.334 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.335 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.335 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.335 INFO:teuthology.orchestra.run.smithi001.stdout: "motd": { 2024-09-11T22:07:28.335 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "motd", 2024-09-11T22:07:28.335 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.335 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.335 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.335 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.336 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.336 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.336 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.336 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "The message of the day", 2024-09-11T22:07:28.336 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.336 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.336 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.337 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.337 INFO:teuthology.orchestra.run.smithi001.stdout: "redirect_resolve_ip_addr": { 2024-09-11T22:07:28.337 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "redirect_resolve_ip_addr", 2024-09-11T22:07:28.337 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.337 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.337 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.337 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.337 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.338 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.338 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.338 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.338 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.338 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.338 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.338 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.338 INFO:teuthology.orchestra.run.smithi001.stdout: "server_addr": { 2024-09-11T22:07:28.338 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_addr", 2024-09-11T22:07:28.339 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.339 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.339 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.339 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "::", 2024-09-11T22:07:28.339 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.339 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.339 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.339 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.340 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.340 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.340 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.340 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.340 INFO:teuthology.orchestra.run.smithi001.stdout: "server_port": { 2024-09-11T22:07:28.340 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_port", 2024-09-11T22:07:28.340 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:28.341 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.341 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.341 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8080", 2024-09-11T22:07:28.341 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.341 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.341 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.341 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.341 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.342 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.342 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.342 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.342 INFO:teuthology.orchestra.run.smithi001.stdout: "ssl": { 2024-09-11T22:07:28.342 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssl", 2024-09-11T22:07:28.342 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.342 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.343 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.343 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:28.343 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.343 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.343 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.343 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.343 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.343 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.343 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.344 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.344 INFO:teuthology.orchestra.run.smithi001.stdout: "ssl_server_port": { 2024-09-11T22:07:28.344 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssl_server_port", 2024-09-11T22:07:28.344 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:28.344 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.344 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.344 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8443", 2024-09-11T22:07:28.345 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.345 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.345 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.345 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.345 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.345 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.345 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.345 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.345 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_behaviour": { 2024-09-11T22:07:28.346 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_behaviour", 2024-09-11T22:07:28.346 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.346 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.346 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.346 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "redirect", 2024-09-11T22:07:28.346 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.346 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.346 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.347 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.347 INFO:teuthology.orchestra.run.smithi001.stdout: "redirect" 2024-09-11T22:07:28.347 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.347 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.347 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.347 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.347 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.348 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.348 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_error_status_code": { 2024-09-11T22:07:28.348 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_error_status_code", 2024-09-11T22:07:28.348 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:28.348 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.348 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.348 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "500", 2024-09-11T22:07:28.348 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "400", 2024-09-11T22:07:28.349 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "599", 2024-09-11T22:07:28.349 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.349 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.349 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.349 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.349 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.349 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.349 INFO:teuthology.orchestra.run.smithi001.stdout: "url_prefix": { 2024-09-11T22:07:28.350 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "url_prefix", 2024-09-11T22:07:28.350 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.350 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.350 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.350 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.350 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.350 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.350 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.351 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.351 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.351 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.351 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.351 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.351 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.351 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.351 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:28.352 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "devicehealth", 2024-09-11T22:07:28.352 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:28.352 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:28.352 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:28.352 INFO:teuthology.orchestra.run.smithi001.stdout: "enable_monitoring": { 2024-09-11T22:07:28.352 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enable_monitoring", 2024-09-11T22:07:28.352 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.352 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.353 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.353 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:28.353 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.353 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.353 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.353 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "monitor device health metrics", 2024-09-11T22:07:28.353 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.354 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.354 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.354 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.354 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:28.354 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:28.354 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.354 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.354 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.355 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.355 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.355 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.355 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.355 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.355 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.355 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.356 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.356 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.356 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.356 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.356 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.356 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.356 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.357 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.357 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.357 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:28.357 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:28.357 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.357 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.357 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.358 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.358 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.358 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.358 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.358 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.358 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.358 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.358 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.359 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.359 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:28.359 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:28.359 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.359 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.359 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.359 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:28.359 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.360 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.360 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.360 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.360 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.360 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.360 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.360 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.360 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.361 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.361 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.361 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.361 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.361 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.361 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.361 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:28.361 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:28.362 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.362 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.362 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.362 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.362 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.362 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.362 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.363 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.363 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.363 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.363 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.363 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.363 INFO:teuthology.orchestra.run.smithi001.stdout: "mark_out_threshold": { 2024-09-11T22:07:28.363 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mark_out_threshold", 2024-09-11T22:07:28.363 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-11T22:07:28.364 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.364 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.364 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "2419200", 2024-09-11T22:07:28.364 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.364 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.364 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.364 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-11T22:07:28.364 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.365 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.365 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.365 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.365 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_name": { 2024-09-11T22:07:28.365 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pool_name", 2024-09-11T22:07:28.365 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.365 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.365 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.366 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "device_health_metrics", 2024-09-11T22:07:28.366 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.366 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.366 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.366 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-11T22:07:28.366 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.366 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.367 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.367 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.367 INFO:teuthology.orchestra.run.smithi001.stdout: "retention_period": { 2024-09-11T22:07:28.367 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "retention_period", 2024-09-11T22:07:28.367 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-11T22:07:28.367 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.367 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.367 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "15552000", 2024-09-11T22:07:28.368 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.368 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.368 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.368 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how long to retain device health metrics", 2024-09-11T22:07:28.368 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.368 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.368 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.369 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.369 INFO:teuthology.orchestra.run.smithi001.stdout: "scrape_frequency": { 2024-09-11T22:07:28.369 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "scrape_frequency", 2024-09-11T22:07:28.369 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-11T22:07:28.369 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.369 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.369 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "86400", 2024-09-11T22:07:28.369 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.370 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.370 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.370 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-11T22:07:28.370 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.370 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.370 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.370 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.370 INFO:teuthology.orchestra.run.smithi001.stdout: "self_heal": { 2024-09-11T22:07:28.371 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "self_heal", 2024-09-11T22:07:28.371 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.371 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.371 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.371 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:28.371 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.371 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.371 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.372 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-11T22:07:28.372 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.372 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.372 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.372 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.372 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-09-11T22:07:28.372 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-09-11T22:07:28.372 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-11T22:07:28.373 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.373 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.373 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-09-11T22:07:28.373 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.373 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.373 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.373 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to wake up and check device health", 2024-09-11T22:07:28.373 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.374 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.374 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.374 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.374 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_threshold": { 2024-09-11T22:07:28.374 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_threshold", 2024-09-11T22:07:28.374 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-11T22:07:28.374 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.375 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.375 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "7257600", 2024-09-11T22:07:28.375 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.375 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.375 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.375 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-11T22:07:28.375 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.375 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.376 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.376 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.376 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.376 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.376 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:28.376 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "diskprediction_local", 2024-09-11T22:07:28.376 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:28.376 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:28.377 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:28.377 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:28.377 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:28.377 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.377 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.377 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.377 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.378 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.378 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.378 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.378 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.378 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.378 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.378 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.378 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.379 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.379 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.379 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.379 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.379 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.379 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.379 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.379 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:28.380 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:28.380 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.380 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.380 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.380 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.380 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.380 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.380 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.381 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.381 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.381 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.381 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.381 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.381 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:28.381 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:28.381 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.382 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.382 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.382 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:28.382 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.382 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.382 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.382 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.383 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.383 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.383 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.383 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.383 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.383 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.383 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.384 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.384 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.384 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.384 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.384 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:28.384 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:28.384 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.384 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.385 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.385 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.385 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.385 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.385 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.385 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.385 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.385 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.386 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.386 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.386 INFO:teuthology.orchestra.run.smithi001.stdout: "predict_interval": { 2024-09-11T22:07:28.386 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "predict_interval", 2024-09-11T22:07:28.386 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.386 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.386 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.387 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "86400", 2024-09-11T22:07:28.387 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.387 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.387 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.387 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.387 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.388 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.388 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.388 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.388 INFO:teuthology.orchestra.run.smithi001.stdout: "predictor_model": { 2024-09-11T22:07:28.388 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "predictor_model", 2024-09-11T22:07:28.388 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.388 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.388 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.389 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "prophetstor", 2024-09-11T22:07:28.389 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.389 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.389 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.389 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.389 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.389 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.390 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.390 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.390 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-09-11T22:07:28.390 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-09-11T22:07:28.390 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.390 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.390 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.390 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-09-11T22:07:28.391 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.391 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.391 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.391 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.391 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.391 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.391 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.391 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.392 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.392 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.392 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:28.392 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "influx", 2024-09-11T22:07:28.392 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": false, 2024-09-11T22:07:28.392 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "influxdb python module not found", 2024-09-11T22:07:28.392 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:28.392 INFO:teuthology.orchestra.run.smithi001.stdout: "batch_size": { 2024-09-11T22:07:28.393 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "batch_size", 2024-09-11T22:07:28.393 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:28.393 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.393 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.393 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5000", 2024-09-11T22:07:28.393 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.393 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.393 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.394 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-11T22:07:28.394 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.394 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.394 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.394 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.394 INFO:teuthology.orchestra.run.smithi001.stdout: "database": { 2024-09-11T22:07:28.394 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "database", 2024-09-11T22:07:28.394 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.395 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.395 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.395 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "ceph", 2024-09-11T22:07:28.395 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.395 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.395 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.395 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-11T22:07:28.396 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.396 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.396 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.396 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.396 INFO:teuthology.orchestra.run.smithi001.stdout: "hostname": { 2024-09-11T22:07:28.396 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "hostname", 2024-09-11T22:07:28.396 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.396 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.397 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.397 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.397 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.397 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.397 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.397 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "InfluxDB server hostname", 2024-09-11T22:07:28.397 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.397 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.398 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.398 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.398 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-09-11T22:07:28.398 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-09-11T22:07:28.398 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-11T22:07:28.398 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.398 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.398 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "30", 2024-09-11T22:07:28.399 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "5", 2024-09-11T22:07:28.399 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.399 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.399 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-11T22:07:28.399 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.399 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.399 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.399 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.400 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:28.400 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:28.400 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.400 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.400 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.400 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.400 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.400 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.401 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.401 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.401 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.401 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.401 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.401 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.401 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.401 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.402 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.402 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.402 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.402 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.402 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.402 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:28.402 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:28.402 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.403 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.403 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.403 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.403 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.403 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.403 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.403 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.403 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.404 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.404 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.404 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.404 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:28.404 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:28.404 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.404 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.404 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.405 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:28.405 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.405 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.405 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.405 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.405 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.405 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.406 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.406 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.406 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.406 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.406 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.406 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.406 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.406 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.407 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.407 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:28.407 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:28.407 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.407 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.407 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.407 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.407 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.408 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.408 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.408 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.408 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.408 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.408 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.408 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.408 INFO:teuthology.orchestra.run.smithi001.stdout: "password": { 2024-09-11T22:07:28.409 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "password", 2024-09-11T22:07:28.409 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.409 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.409 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.409 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.409 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.409 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.409 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.410 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "password of InfluxDB server user", 2024-09-11T22:07:28.410 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.410 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.410 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.410 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.410 INFO:teuthology.orchestra.run.smithi001.stdout: "port": { 2024-09-11T22:07:28.410 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "port", 2024-09-11T22:07:28.410 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:28.411 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.411 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.411 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8086", 2024-09-11T22:07:28.411 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.411 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.411 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.411 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "InfluxDB server port", 2024-09-11T22:07:28.411 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.412 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.412 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.412 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.412 INFO:teuthology.orchestra.run.smithi001.stdout: "ssl": { 2024-09-11T22:07:28.412 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssl", 2024-09-11T22:07:28.412 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.412 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.413 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.413 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "false", 2024-09-11T22:07:28.413 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.413 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.413 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.413 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-11T22:07:28.413 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.413 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.414 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.414 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.414 INFO:teuthology.orchestra.run.smithi001.stdout: "threads": { 2024-09-11T22:07:28.414 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "threads", 2024-09-11T22:07:28.414 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:28.414 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.414 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.414 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-09-11T22:07:28.415 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-09-11T22:07:28.415 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "32", 2024-09-11T22:07:28.415 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.415 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-11T22:07:28.415 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.415 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.415 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.415 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.416 INFO:teuthology.orchestra.run.smithi001.stdout: "username": { 2024-09-11T22:07:28.416 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "username", 2024-09-11T22:07:28.416 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.416 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.416 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.416 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.416 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.417 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.417 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.417 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "username of InfluxDB server user", 2024-09-11T22:07:28.417 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.417 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.417 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.417 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.417 INFO:teuthology.orchestra.run.smithi001.stdout: "verify_ssl": { 2024-09-11T22:07:28.418 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "verify_ssl", 2024-09-11T22:07:28.418 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.418 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.418 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.418 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "true", 2024-09-11T22:07:28.418 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.418 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.419 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.419 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-11T22:07:28.419 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.419 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.419 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.419 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.419 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.419 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.420 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:28.420 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "insights", 2024-09-11T22:07:28.420 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:28.420 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:28.420 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:28.420 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:28.420 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:28.421 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.421 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.421 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.421 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.421 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.421 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.421 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.421 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.421 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.422 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.422 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.422 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.422 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.422 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.422 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.422 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.422 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.423 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.423 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.423 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:28.423 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:28.423 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.423 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.423 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.423 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.424 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.424 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.424 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.424 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.424 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.424 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.424 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.424 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.425 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:28.425 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:28.425 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.425 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.425 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.425 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:28.425 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.426 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.426 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.426 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.426 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.426 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.426 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.426 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.426 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.427 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.427 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.427 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.427 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.427 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.427 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.427 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:28.427 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:28.428 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.428 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.428 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.428 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.428 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.428 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.428 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.428 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.429 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.429 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.429 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.429 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.429 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.429 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.429 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:28.429 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "iostat", 2024-09-11T22:07:28.430 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:28.430 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:28.430 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:28.430 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:28.430 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:28.430 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.430 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.431 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.431 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.431 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.431 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.431 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.431 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.431 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.431 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.432 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.432 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.432 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.432 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.432 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.432 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.432 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.432 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.432 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.433 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:28.433 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:28.433 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.433 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.433 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.433 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.433 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.433 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.434 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.434 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.434 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.434 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.434 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.434 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.434 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:28.434 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:28.435 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.435 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.435 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.435 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:28.435 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.435 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.435 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.435 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.436 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.436 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.436 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.436 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.436 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.436 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.436 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.437 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.437 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.437 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.437 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.437 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:28.437 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:28.437 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.437 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.438 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.438 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.438 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.438 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.438 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.438 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.438 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.438 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.439 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.439 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.439 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.439 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.439 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:28.439 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "k8sevents", 2024-09-11T22:07:28.439 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:28.440 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:28.440 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:28.440 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_event_retention_days": { 2024-09-11T22:07:28.440 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ceph_event_retention_days", 2024-09-11T22:07:28.440 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:28.440 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.440 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.440 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "7", 2024-09-11T22:07:28.441 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.441 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.441 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.441 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-11T22:07:28.441 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.441 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.441 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.442 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.442 INFO:teuthology.orchestra.run.smithi001.stdout: "config_check_secs": { 2024-09-11T22:07:28.442 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "config_check_secs", 2024-09-11T22:07:28.442 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:28.442 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.442 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.442 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-11T22:07:28.442 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "10", 2024-09-11T22:07:28.443 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.443 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.443 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-11T22:07:28.443 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.443 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.443 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.443 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.443 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:28.444 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:28.444 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.444 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.444 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.444 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.444 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.444 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.444 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.445 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.445 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.445 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.445 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.445 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.445 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.445 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.445 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.446 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.446 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.446 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.446 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.446 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:28.446 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:28.446 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.447 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.447 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.447 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.447 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.447 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.447 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.447 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.447 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.447 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.448 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.448 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.448 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:28.448 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:28.448 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.448 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.448 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.448 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:28.449 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.449 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.449 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.449 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.449 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.449 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.449 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.450 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.450 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.450 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.450 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.450 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.450 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.450 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.450 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.451 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:28.451 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:28.451 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.451 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.451 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.451 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.451 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.452 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.452 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.452 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.452 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.452 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.452 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.452 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.452 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.453 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.453 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:28.453 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "localpool", 2024-09-11T22:07:28.453 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:28.453 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:28.453 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:28.453 INFO:teuthology.orchestra.run.smithi001.stdout: "failure_domain": { 2024-09-11T22:07:28.453 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "failure_domain", 2024-09-11T22:07:28.454 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.454 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.454 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.454 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "host", 2024-09-11T22:07:28.454 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.454 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.454 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.455 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "failure domain for any created local pool", 2024-09-11T22:07:28.455 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-11T22:07:28.455 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.455 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.455 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.455 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:28.455 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:28.455 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.456 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.456 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.456 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.456 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.456 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.456 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.456 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.457 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.457 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.457 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.457 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.457 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.457 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.457 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.457 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.458 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.458 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.458 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.458 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:28.458 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:28.458 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.458 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.458 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.459 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.459 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.459 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.459 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.459 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.459 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.459 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.459 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.460 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.460 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:28.460 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:28.460 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.460 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.460 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.460 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:28.461 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.461 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.461 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.461 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.461 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.461 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.461 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.461 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.462 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.462 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.462 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.462 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.462 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.462 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.462 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.462 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:28.463 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:28.463 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.463 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.463 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.463 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.463 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.463 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.463 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.464 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.464 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.464 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.464 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.464 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.464 INFO:teuthology.orchestra.run.smithi001.stdout: "min_size": { 2024-09-11T22:07:28.464 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "min_size", 2024-09-11T22:07:28.464 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:28.465 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.465 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.465 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.465 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.465 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.465 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.465 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "default min_size for any created local pool", 2024-09-11T22:07:28.466 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-11T22:07:28.466 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.466 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.466 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.466 INFO:teuthology.orchestra.run.smithi001.stdout: "num_rep": { 2024-09-11T22:07:28.466 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "num_rep", 2024-09-11T22:07:28.466 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:28.466 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.467 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.467 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "3", 2024-09-11T22:07:28.467 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.467 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.467 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.467 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "default replica count for any created local pool", 2024-09-11T22:07:28.467 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.467 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.468 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.468 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.468 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num": { 2024-09-11T22:07:28.468 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pg_num", 2024-09-11T22:07:28.468 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:28.468 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.468 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.469 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "128", 2024-09-11T22:07:28.469 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.469 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.469 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.469 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "default pg_num for any created local pool", 2024-09-11T22:07:28.469 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.469 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.469 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.469 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.470 INFO:teuthology.orchestra.run.smithi001.stdout: "prefix": { 2024-09-11T22:07:28.470 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "prefix", 2024-09-11T22:07:28.470 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.470 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.470 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.470 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.470 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.470 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.471 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.471 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "name prefix for any created local pool", 2024-09-11T22:07:28.471 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.471 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.471 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.471 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.471 INFO:teuthology.orchestra.run.smithi001.stdout: "subtree": { 2024-09-11T22:07:28.471 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "subtree", 2024-09-11T22:07:28.472 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.472 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.472 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.472 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "rack", 2024-09-11T22:07:28.472 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.472 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.472 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.472 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-11T22:07:28.473 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-11T22:07:28.473 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.473 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.473 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.473 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.473 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.473 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:28.473 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mds_autoscaler", 2024-09-11T22:07:28.474 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:28.474 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:28.474 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:28.474 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:28.474 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:28.474 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.474 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.474 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.475 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.475 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.475 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.475 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.475 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.475 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.475 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.475 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.476 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.476 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.476 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.476 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.476 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.476 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.476 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.477 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.477 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:28.477 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:28.477 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.477 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.477 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.477 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.477 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.478 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.478 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.478 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.478 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.478 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.478 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.478 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.478 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:28.479 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:28.479 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.479 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.479 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.479 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:28.479 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.479 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.480 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.480 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.480 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.480 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.480 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.480 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.480 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.480 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.481 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.481 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.481 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.481 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.481 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.481 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:28.481 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:28.481 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.482 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.482 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.482 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.482 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.482 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.482 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.482 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.483 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.483 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.483 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.483 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.483 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.483 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.483 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:28.483 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mirroring", 2024-09-11T22:07:28.484 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:28.484 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:28.484 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:28.484 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:28.484 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:28.484 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.484 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.484 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.485 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.485 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.485 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.485 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.485 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.485 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.485 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.486 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.486 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.486 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.486 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.486 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.486 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.486 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.486 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.487 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.487 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:28.487 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:28.487 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.487 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.487 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.487 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.488 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.488 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.488 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.488 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.488 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.488 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.488 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.488 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.489 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:28.489 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:28.489 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.489 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.489 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.489 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:28.489 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.489 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.490 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.490 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.490 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.490 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.490 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.490 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.490 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.490 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.491 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.491 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.491 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.491 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.491 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.491 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:28.491 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:28.492 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.492 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.492 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.492 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.492 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.492 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.492 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.492 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.493 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.493 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.493 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.493 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.493 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.493 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.493 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:28.494 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "nfs", 2024-09-11T22:07:28.494 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:28.494 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:28.494 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:28.494 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:28.494 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:28.494 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.494 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.495 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.495 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.495 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.495 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.495 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.495 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.495 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.495 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.496 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.496 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.496 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.496 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.496 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.496 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.496 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.496 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.497 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.497 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:28.497 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:28.497 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.497 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.497 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.497 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.497 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.498 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.498 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.498 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.498 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.498 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.498 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.498 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.498 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:28.499 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:28.499 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.499 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.499 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.499 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:28.499 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.499 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.499 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.500 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.500 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.500 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.500 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.500 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.500 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.500 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.500 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.501 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.501 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.501 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.501 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.501 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:28.501 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:28.501 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.501 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.501 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.502 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.502 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.502 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.502 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.502 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.502 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.502 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.503 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.503 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.503 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.503 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.503 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:28.503 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "orchestrator", 2024-09-11T22:07:28.503 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:28.503 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:28.503 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:28.504 INFO:teuthology.orchestra.run.smithi001.stdout: "fail_fs": { 2024-09-11T22:07:28.504 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "fail_fs", 2024-09-11T22:07:28.504 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.504 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.504 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.504 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.504 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.505 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.505 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.505 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-11T22:07:28.505 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.505 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.505 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.505 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.505 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:28.506 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:28.506 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.506 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.506 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.506 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.506 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.506 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.507 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.507 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.507 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.507 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.507 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.507 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.507 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.507 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.508 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.508 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.508 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.508 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.508 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.508 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:28.508 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:28.508 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.509 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.509 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.509 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.509 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.509 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.509 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.509 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.509 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.509 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.510 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.510 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.510 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:28.510 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:28.510 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.510 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.510 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.510 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:28.511 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.511 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.511 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.511 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.511 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.511 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.511 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.511 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.512 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.512 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.512 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.512 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.512 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.512 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.512 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.513 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:28.513 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:28.513 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.513 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.513 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.513 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.513 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.513 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.514 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.514 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.514 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.514 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.514 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.514 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.514 INFO:teuthology.orchestra.run.smithi001.stdout: "orchestrator": { 2024-09-11T22:07:28.514 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "orchestrator", 2024-09-11T22:07:28.515 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.515 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.515 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.515 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.515 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.515 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.515 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.515 INFO:teuthology.orchestra.run.smithi001.stdout: "cephadm", 2024-09-11T22:07:28.516 INFO:teuthology.orchestra.run.smithi001.stdout: "rook", 2024-09-11T22:07:28.516 INFO:teuthology.orchestra.run.smithi001.stdout: "test_orchestrator" 2024-09-11T22:07:28.516 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.516 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Orchestrator backend", 2024-09-11T22:07:28.516 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.516 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.516 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.516 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.517 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.517 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.517 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:28.517 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "osd_perf_query", 2024-09-11T22:07:28.517 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:28.517 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:28.517 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:28.518 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:28.518 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:28.518 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.518 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.518 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.518 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.518 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.518 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.519 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.519 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.519 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.519 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.519 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.519 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.519 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.519 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.520 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.520 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.520 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.520 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.520 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.520 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:28.520 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:28.521 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.521 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.521 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.521 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.521 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.521 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.521 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.521 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.522 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.522 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.522 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.522 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.522 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:28.522 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:28.522 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.522 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.523 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.523 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:28.523 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.523 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.523 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.523 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.523 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.523 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.523 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.524 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.524 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.524 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.524 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.524 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.524 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.524 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.524 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.525 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:28.525 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:28.525 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.525 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.525 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.525 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.525 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.525 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.526 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.526 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.526 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.526 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.526 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.526 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.526 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.526 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.527 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:28.527 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "osd_support", 2024-09-11T22:07:28.527 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:28.527 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:28.527 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:28.527 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:28.527 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:28.527 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.528 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.528 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.528 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.528 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.528 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.528 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.528 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.528 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.529 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.529 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.529 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.529 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.529 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.529 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.529 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.529 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.530 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.530 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.530 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:28.530 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:28.530 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.530 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.530 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.530 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.531 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.531 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.531 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.531 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.531 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.531 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.531 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.531 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.532 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:28.532 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:28.532 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.532 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.532 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.532 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:28.532 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.532 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.533 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.533 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.533 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.533 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.533 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.533 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.533 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.533 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.533 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.534 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.534 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.534 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.534 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.534 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:28.534 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:28.534 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.534 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.535 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.535 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.535 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.535 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.535 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.535 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.535 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.535 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.536 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.536 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.536 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.536 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.536 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:28.536 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pg_autoscaler", 2024-09-11T22:07:28.536 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:28.536 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:28.537 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:28.537 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:28.537 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:28.537 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.537 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.537 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.537 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.537 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.538 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.538 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.538 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.538 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.538 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.538 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.538 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.539 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.539 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.539 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.539 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.539 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.539 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.539 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.539 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:28.540 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:28.540 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.540 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.540 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.540 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.540 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.540 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.541 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.541 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.541 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.541 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.541 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.541 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.541 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:28.541 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:28.542 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.542 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.542 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.542 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:28.542 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.542 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.542 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.542 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.543 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.543 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.543 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.543 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.543 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.543 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.543 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.544 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.544 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.544 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.544 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.544 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:28.544 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:28.544 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.544 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.545 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.545 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.545 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.545 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.545 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.545 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.545 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.546 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.546 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.546 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.546 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-09-11T22:07:28.546 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-09-11T22:07:28.546 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-11T22:07:28.546 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.546 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.547 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-09-11T22:07:28.547 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.547 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.547 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.547 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.547 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.547 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.547 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.548 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.548 INFO:teuthology.orchestra.run.smithi001.stdout: "threshold": { 2024-09-11T22:07:28.548 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "threshold", 2024-09-11T22:07:28.548 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-09-11T22:07:28.548 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.548 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.548 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "3.0", 2024-09-11T22:07:28.548 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1.0", 2024-09-11T22:07:28.549 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.549 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.549 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "scaling threshold", 2024-09-11T22:07:28.549 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-11T22:07:28.549 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.549 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.549 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.550 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.550 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.550 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:28.550 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "progress", 2024-09-11T22:07:28.550 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:28.550 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:28.550 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:28.550 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_pg_recovery_event": { 2024-09-11T22:07:28.550 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "allow_pg_recovery_event", 2024-09-11T22:07:28.551 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.551 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.551 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.551 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.551 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.551 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.551 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.551 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-11T22:07:28.552 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.552 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.552 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.552 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.552 INFO:teuthology.orchestra.run.smithi001.stdout: "enabled": { 2024-09-11T22:07:28.552 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enabled", 2024-09-11T22:07:28.552 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.552 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.553 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.553 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:28.553 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.553 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.553 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.553 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.553 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.553 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.554 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.554 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.554 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:28.554 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:28.554 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.554 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.554 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.554 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.554 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.555 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.555 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.555 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.555 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.555 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.555 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.555 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.555 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.556 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.556 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.556 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.556 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.556 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.556 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.556 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:28.556 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:28.557 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.557 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.557 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.557 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.557 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.557 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.557 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.558 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.558 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.558 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.558 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.558 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.558 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:28.558 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:28.558 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.559 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.559 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.559 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:28.559 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.559 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.559 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.559 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.559 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.560 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.560 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.560 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.560 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.560 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.560 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.560 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.560 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.561 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.561 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.561 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:28.561 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:28.561 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.561 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.561 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.562 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.562 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.562 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.562 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.562 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.562 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.562 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.562 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.563 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.563 INFO:teuthology.orchestra.run.smithi001.stdout: "max_completed_events": { 2024-09-11T22:07:28.563 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_completed_events", 2024-09-11T22:07:28.563 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:28.563 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.563 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.563 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "50", 2024-09-11T22:07:28.563 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.564 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.564 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.564 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "number of past completed events to remember", 2024-09-11T22:07:28.564 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.564 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.564 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.564 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.564 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-09-11T22:07:28.565 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-09-11T22:07:28.565 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-11T22:07:28.565 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.565 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.565 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-09-11T22:07:28.565 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.565 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.565 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.566 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how long the module is going to sleep", 2024-09-11T22:07:28.566 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.566 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.566 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.566 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.566 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.566 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.566 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:28.566 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "prometheus", 2024-09-11T22:07:28.567 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:28.567 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:28.567 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:28.567 INFO:teuthology.orchestra.run.smithi001.stdout: "cache": { 2024-09-11T22:07:28.567 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cache", 2024-09-11T22:07:28.567 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.567 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.568 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.568 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:28.568 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.568 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.568 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.568 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.568 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.569 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.569 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.569 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.569 INFO:teuthology.orchestra.run.smithi001.stdout: "exclude_perf_counters": { 2024-09-11T22:07:28.569 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "exclude_perf_counters", 2024-09-11T22:07:28.569 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.569 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.569 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.570 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:28.570 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.570 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.570 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.570 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-11T22:07:28.570 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-11T22:07:28.570 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.571 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.571 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.571 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:28.571 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:28.571 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.571 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.571 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.571 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.572 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.572 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.572 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.572 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.572 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.572 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.572 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.572 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.573 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.573 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.573 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.573 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.573 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.573 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.573 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.573 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:28.574 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:28.574 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.574 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.574 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.574 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.574 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.574 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.575 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.575 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.575 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.575 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.575 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.575 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.575 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:28.575 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:28.576 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.576 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.576 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.576 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:28.576 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.576 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.576 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.576 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.577 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.577 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.577 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.577 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.577 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.577 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.577 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.577 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.578 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.578 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.578 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.578 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:28.578 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:28.578 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.578 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.579 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.579 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.579 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.579 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.579 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.579 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.579 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.579 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.580 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.580 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.580 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_stats_pools": { 2024-09-11T22:07:28.580 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rbd_stats_pools", 2024-09-11T22:07:28.580 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.580 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.580 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.580 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.581 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.581 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.581 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.581 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.581 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.581 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.581 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.581 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.582 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-11T22:07:28.582 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-11T22:07:28.582 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:28.582 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.582 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.582 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "300", 2024-09-11T22:07:28.582 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.582 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.583 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.583 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.583 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.583 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.583 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.583 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.583 INFO:teuthology.orchestra.run.smithi001.stdout: "scrape_interval": { 2024-09-11T22:07:28.584 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "scrape_interval", 2024-09-11T22:07:28.584 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-09-11T22:07:28.584 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.584 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.584 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "15.0", 2024-09-11T22:07:28.584 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.584 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.584 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.585 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.585 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.585 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.585 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.585 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.585 INFO:teuthology.orchestra.run.smithi001.stdout: "server_addr": { 2024-09-11T22:07:28.585 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_addr", 2024-09-11T22:07:28.585 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.586 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.586 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.586 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "::", 2024-09-11T22:07:28.586 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.586 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.586 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.586 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-11T22:07:28.586 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.587 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.587 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.587 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.587 INFO:teuthology.orchestra.run.smithi001.stdout: "server_port": { 2024-09-11T22:07:28.587 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_port", 2024-09-11T22:07:28.587 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:28.587 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.587 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.588 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "9283", 2024-09-11T22:07:28.588 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.588 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.588 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.588 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-11T22:07:28.588 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.588 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.589 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.589 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.589 INFO:teuthology.orchestra.run.smithi001.stdout: "stale_cache_strategy": { 2024-09-11T22:07:28.589 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "stale_cache_strategy", 2024-09-11T22:07:28.589 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.589 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.589 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.589 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "log", 2024-09-11T22:07:28.590 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.590 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.590 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.590 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.590 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.590 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.590 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.590 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.591 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_behaviour": { 2024-09-11T22:07:28.591 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_behaviour", 2024-09-11T22:07:28.591 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.591 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.591 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.591 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "default", 2024-09-11T22:07:28.591 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.591 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.592 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.592 INFO:teuthology.orchestra.run.smithi001.stdout: "default", 2024-09-11T22:07:28.592 INFO:teuthology.orchestra.run.smithi001.stdout: "error" 2024-09-11T22:07:28.592 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.592 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.592 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.592 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.593 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.593 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.593 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_error_status_code": { 2024-09-11T22:07:28.593 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_error_status_code", 2024-09-11T22:07:28.593 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:28.593 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.593 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.593 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "500", 2024-09-11T22:07:28.594 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "400", 2024-09-11T22:07:28.594 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "599", 2024-09-11T22:07:28.594 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.594 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.594 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.594 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.594 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.594 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.595 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.595 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.595 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:28.595 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rbd_support", 2024-09-11T22:07:28.595 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:28.595 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:28.595 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:28.595 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:28.596 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:28.596 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.596 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.596 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.596 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.596 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.596 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.596 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.597 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.597 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.597 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.597 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.597 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.597 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.597 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.597 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.598 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.598 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.598 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.598 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.598 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:28.598 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:28.598 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.598 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.599 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.599 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.599 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.599 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.599 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.599 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.599 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.599 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.600 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.600 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.600 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:28.600 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:28.600 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.600 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.600 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.600 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:28.601 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.601 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.601 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.601 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.601 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.601 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.601 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.602 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.602 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.602 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.602 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.602 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.602 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.602 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.602 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.602 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:28.603 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:28.603 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.603 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.603 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.603 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.603 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.603 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.603 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.604 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.604 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.604 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.604 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.604 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.604 INFO:teuthology.orchestra.run.smithi001.stdout: "max_concurrent_snap_create": { 2024-09-11T22:07:28.604 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_concurrent_snap_create", 2024-09-11T22:07:28.604 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:28.605 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.605 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.605 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-09-11T22:07:28.605 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.605 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.605 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.605 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.606 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.606 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.606 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.606 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.606 INFO:teuthology.orchestra.run.smithi001.stdout: "mirror_snapshot_schedule": { 2024-09-11T22:07:28.606 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mirror_snapshot_schedule", 2024-09-11T22:07:28.606 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.606 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.607 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.607 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.607 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.607 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.607 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.607 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.607 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.607 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.608 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.608 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.608 INFO:teuthology.orchestra.run.smithi001.stdout: "trash_purge_schedule": { 2024-09-11T22:07:28.608 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "trash_purge_schedule", 2024-09-11T22:07:28.608 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.608 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.608 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.608 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.609 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.609 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.609 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.609 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.609 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.609 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.609 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.609 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.610 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.610 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.610 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:28.610 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "restful", 2024-09-11T22:07:28.610 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:28.610 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:28.610 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:28.610 INFO:teuthology.orchestra.run.smithi001.stdout: "enable_auth": { 2024-09-11T22:07:28.611 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enable_auth", 2024-09-11T22:07:28.611 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.611 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.611 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.611 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:28.611 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.611 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.611 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.612 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.612 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.612 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.612 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.612 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.612 INFO:teuthology.orchestra.run.smithi001.stdout: "key_file": { 2024-09-11T22:07:28.612 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "key_file", 2024-09-11T22:07:28.612 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.613 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.613 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.613 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.613 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.613 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.613 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.613 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.613 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.614 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.614 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.614 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.614 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:28.614 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:28.614 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.614 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.614 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.615 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.615 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.615 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.615 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.615 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.615 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.615 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.615 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.616 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.616 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.616 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.616 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.616 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.616 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.616 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.617 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.617 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:28.617 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:28.617 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.617 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.617 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.617 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.617 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.618 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.618 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.618 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.618 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.618 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.618 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.618 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.618 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:28.619 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:28.619 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.619 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.619 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.619 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:28.619 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.619 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.620 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.620 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.620 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.620 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.620 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.620 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.620 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.620 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.621 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.621 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.621 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.621 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.621 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.621 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:28.621 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:28.622 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.622 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.622 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.622 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.622 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.622 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.622 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.622 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.623 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.623 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.623 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.623 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.623 INFO:teuthology.orchestra.run.smithi001.stdout: "server_addr": { 2024-09-11T22:07:28.623 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_addr", 2024-09-11T22:07:28.623 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.623 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.624 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.624 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.624 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.624 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.624 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.624 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.624 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.625 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.625 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.625 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.625 INFO:teuthology.orchestra.run.smithi001.stdout: "server_port": { 2024-09-11T22:07:28.625 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_port", 2024-09-11T22:07:28.625 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.625 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.625 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.626 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.626 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.626 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.626 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.626 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.626 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.626 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.627 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.627 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.627 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.627 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.627 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:28.627 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rgw", 2024-09-11T22:07:28.627 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:28.627 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:28.628 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:28.628 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:28.628 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:28.628 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.628 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.628 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.628 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.628 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.629 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.629 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.629 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.629 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.629 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.629 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.629 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.629 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.630 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.630 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.630 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.630 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.630 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.630 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.630 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:28.631 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:28.631 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.631 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.631 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.631 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.631 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.631 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.631 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.632 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.632 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.632 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.632 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.632 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.632 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:28.632 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:28.633 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.633 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.633 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.633 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:28.633 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.633 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.633 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.633 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.634 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.634 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.634 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.634 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.634 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.634 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.634 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.635 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.635 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.635 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.635 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.635 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:28.635 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:28.635 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.635 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.636 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.636 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.636 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.636 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.636 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.636 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.636 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.636 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.637 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.637 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.637 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.637 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.637 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:28.637 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rook", 2024-09-11T22:07:28.637 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:28.638 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:28.638 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:28.638 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:28.638 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:28.638 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.638 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.638 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.638 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.639 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.639 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.639 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.639 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.639 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.639 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.639 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.639 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.640 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.640 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.640 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.640 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.640 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.640 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.640 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.641 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:28.641 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:28.641 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.641 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.641 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.641 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.641 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.641 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.642 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.642 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.642 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.642 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.642 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.642 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.642 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:28.643 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:28.643 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.643 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.643 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.643 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:28.643 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.643 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.643 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.644 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.644 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.644 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.644 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.644 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.644 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.644 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.644 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.645 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.645 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.645 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.645 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.645 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:28.645 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:28.645 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.646 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.646 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.646 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.646 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.646 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.646 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.646 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.646 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.647 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.647 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.647 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.647 INFO:teuthology.orchestra.run.smithi001.stdout: "storage_class": { 2024-09-11T22:07:28.647 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "storage_class", 2024-09-11T22:07:28.647 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.647 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.647 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.648 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "local", 2024-09-11T22:07:28.648 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.648 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.648 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.648 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-11T22:07:28.648 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.648 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.649 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.649 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.649 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.649 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.649 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:28.649 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "selftest", 2024-09-11T22:07:28.649 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:28.649 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:28.650 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:28.650 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:28.650 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:28.650 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.650 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.650 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.650 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.650 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.651 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.651 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.651 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.651 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.651 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.651 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.651 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.651 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.652 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.652 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.652 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.652 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.652 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.652 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.652 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:28.652 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:28.652 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.653 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.653 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.653 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.653 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.653 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.653 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.653 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.654 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.654 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.654 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.654 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.654 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:28.654 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:28.654 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.654 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.655 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.655 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:28.655 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.655 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.655 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.655 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.655 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.655 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.656 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.656 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.656 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.656 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.656 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.656 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.656 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.657 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.657 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.657 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:28.657 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:28.657 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.657 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.657 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.657 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.658 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.658 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.658 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.658 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.658 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.658 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.658 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.658 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.659 INFO:teuthology.orchestra.run.smithi001.stdout: "roption1": { 2024-09-11T22:07:28.659 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "roption1", 2024-09-11T22:07:28.659 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.659 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.659 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.659 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.659 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.659 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.660 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.660 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.660 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.660 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.660 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.660 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.660 INFO:teuthology.orchestra.run.smithi001.stdout: "roption2": { 2024-09-11T22:07:28.661 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "roption2", 2024-09-11T22:07:28.661 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.661 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.661 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.661 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "xyz", 2024-09-11T22:07:28.661 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.661 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.661 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.662 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.662 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.662 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.662 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.662 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.662 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption1": { 2024-09-11T22:07:28.662 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption1", 2024-09-11T22:07:28.662 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.663 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.663 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.663 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.663 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.663 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.663 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.663 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.663 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.663 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.664 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.664 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.664 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption2": { 2024-09-11T22:07:28.664 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption2", 2024-09-11T22:07:28.664 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:28.664 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.664 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.664 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.665 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.665 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.665 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.665 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.665 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.665 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.665 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.666 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.666 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption3": { 2024-09-11T22:07:28.666 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption3", 2024-09-11T22:07:28.666 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-09-11T22:07:28.666 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.666 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.666 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.666 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.667 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.667 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.667 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.667 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.667 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.667 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.667 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.667 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption4": { 2024-09-11T22:07:28.668 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption4", 2024-09-11T22:07:28.668 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.668 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.668 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.668 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.668 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.668 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.668 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.669 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.669 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.669 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.669 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.669 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.669 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption5": { 2024-09-11T22:07:28.669 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption5", 2024-09-11T22:07:28.669 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.670 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.670 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.670 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.670 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.670 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.670 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.670 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.670 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.671 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.671 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.671 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.671 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption6": { 2024-09-11T22:07:28.671 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption6", 2024-09-11T22:07:28.671 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.671 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.671 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.672 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:28.672 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.672 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.672 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.672 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.672 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.672 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.672 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.673 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.673 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption7": { 2024-09-11T22:07:28.673 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption7", 2024-09-11T22:07:28.673 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:28.673 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.673 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.673 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.673 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-09-11T22:07:28.674 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "42", 2024-09-11T22:07:28.674 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.674 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.674 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.674 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.674 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.674 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.674 INFO:teuthology.orchestra.run.smithi001.stdout: "testkey": { 2024-09-11T22:07:28.675 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "testkey", 2024-09-11T22:07:28.675 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.675 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.675 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.675 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.675 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.675 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.675 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.676 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.676 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.676 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.676 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.676 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.676 INFO:teuthology.orchestra.run.smithi001.stdout: "testlkey": { 2024-09-11T22:07:28.676 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "testlkey", 2024-09-11T22:07:28.676 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.677 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.677 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.677 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.677 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.677 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.677 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.677 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.677 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.678 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.678 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.678 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.678 INFO:teuthology.orchestra.run.smithi001.stdout: "testnewline": { 2024-09-11T22:07:28.678 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "testnewline", 2024-09-11T22:07:28.678 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.678 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.678 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.679 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.679 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.679 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.679 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.679 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.679 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.679 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.679 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.680 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.680 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.680 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.680 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:28.680 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "snap_schedule", 2024-09-11T22:07:28.680 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:28.680 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:28.680 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:28.681 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_m_granularity": { 2024-09-11T22:07:28.681 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "allow_m_granularity", 2024-09-11T22:07:28.681 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.681 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.681 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.681 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.681 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.682 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.682 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.682 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "allow minute scheduled snapshots", 2024-09-11T22:07:28.682 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.682 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.682 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.682 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.682 INFO:teuthology.orchestra.run.smithi001.stdout: "dump_on_update": { 2024-09-11T22:07:28.683 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "dump_on_update", 2024-09-11T22:07:28.683 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.683 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.683 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.683 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.683 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.683 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.684 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.684 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "dump database to debug log on update", 2024-09-11T22:07:28.684 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.684 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.684 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.684 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.684 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:28.685 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:28.685 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.685 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.685 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.685 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.685 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.685 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.685 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.686 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.686 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.686 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.686 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.686 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.686 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.686 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.686 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.686 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.687 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.687 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.687 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.687 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:28.687 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:28.687 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.687 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.687 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.688 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.688 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.688 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.688 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.688 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.688 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.688 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.688 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.689 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.689 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:28.689 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:28.689 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.689 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.689 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.689 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:28.689 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.690 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.690 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.690 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.690 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.690 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.690 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.690 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.690 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.691 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.691 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.691 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.691 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.691 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.691 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.691 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:28.691 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:28.692 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.692 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.692 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.692 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.692 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.692 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.692 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.692 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.693 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.693 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.693 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.693 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.693 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.693 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.693 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:28.693 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "stats", 2024-09-11T22:07:28.694 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:28.694 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:28.694 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:28.694 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:28.694 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:28.694 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.694 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.694 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.695 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.695 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.695 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.695 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.695 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.695 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.695 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.695 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.696 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.696 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.696 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.696 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.696 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.696 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.696 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.696 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.697 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:28.697 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:28.697 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.697 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.697 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.697 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.697 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.697 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.698 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.698 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.698 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.698 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.698 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.698 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.698 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:28.698 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:28.699 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.699 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.699 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.699 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:28.699 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.699 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.699 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.700 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.700 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.700 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.700 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.700 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.700 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.700 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.700 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.701 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.701 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.701 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.701 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.701 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:28.701 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:28.701 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.701 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.702 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.702 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.702 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.702 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.702 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.702 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.702 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.703 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.703 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.703 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.703 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.703 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.703 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:28.703 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "status", 2024-09-11T22:07:28.704 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:28.704 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:28.704 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:28.704 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:28.704 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:28.704 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.704 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.704 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.705 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.705 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.705 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.705 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.705 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.705 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.705 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.705 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.706 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.706 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.706 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.706 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.706 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.706 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.706 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.706 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.707 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:28.707 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:28.707 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.707 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.707 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.707 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.707 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.707 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.708 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.708 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.708 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.708 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.708 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.708 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.708 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:28.709 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:28.709 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.709 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.709 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.709 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:28.709 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.709 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.709 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.710 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.710 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.710 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.710 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.710 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.710 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.710 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.711 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.711 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.711 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.711 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.711 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.711 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:28.711 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:28.711 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.712 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.712 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.712 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.712 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.712 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.712 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.712 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.712 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.713 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.713 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.713 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.713 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.713 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.713 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:28.713 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "telegraf", 2024-09-11T22:07:28.713 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:28.714 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:28.714 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:28.714 INFO:teuthology.orchestra.run.smithi001.stdout: "address": { 2024-09-11T22:07:28.714 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "address", 2024-09-11T22:07:28.714 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.714 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.714 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.714 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-11T22:07:28.715 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.715 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.715 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.715 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.715 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.715 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.715 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.715 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.715 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-09-11T22:07:28.716 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-09-11T22:07:28.716 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-11T22:07:28.716 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.716 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.716 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "15", 2024-09-11T22:07:28.716 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.716 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.717 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.717 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.717 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.717 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.717 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.717 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.717 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:28.717 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:28.718 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.718 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.718 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.718 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.718 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.718 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.718 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.718 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.719 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.719 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.719 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.719 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.719 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.719 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.719 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.719 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.720 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.720 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.720 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.720 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:28.720 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:28.720 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.720 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.720 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.721 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.721 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.721 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.721 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.721 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.721 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.721 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.721 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.722 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.722 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:28.722 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:28.722 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.722 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.722 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.722 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:28.722 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.723 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.723 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.723 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.723 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.723 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.723 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.723 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.723 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.724 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.724 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.724 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.724 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.724 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.724 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.724 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:28.724 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:28.725 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.725 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.725 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.725 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.725 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.725 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.725 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.725 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.726 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.726 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.726 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.726 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.726 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.726 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.726 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:28.727 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "telemetry", 2024-09-11T22:07:28.727 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:28.727 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:28.727 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:28.727 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_basic": { 2024-09-11T22:07:28.727 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_basic", 2024-09-11T22:07:28.727 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.727 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.728 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.728 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:28.728 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.728 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.728 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.728 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-11T22:07:28.728 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.728 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.729 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.729 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.729 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_crash": { 2024-09-11T22:07:28.729 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_crash", 2024-09-11T22:07:28.729 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.729 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.729 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.729 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:28.730 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.730 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.730 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.730 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-11T22:07:28.730 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.730 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.730 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.730 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.731 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_device": { 2024-09-11T22:07:28.731 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_device", 2024-09-11T22:07:28.731 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.731 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.731 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.731 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:28.731 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.732 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.732 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.732 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-11T22:07:28.732 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.732 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.732 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.732 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.732 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_ident": { 2024-09-11T22:07:28.733 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_ident", 2024-09-11T22:07:28.733 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.733 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.733 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.733 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.733 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.733 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.733 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.734 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-11T22:07:28.734 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.734 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.734 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.734 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.734 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_perf": { 2024-09-11T22:07:28.734 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_perf", 2024-09-11T22:07:28.734 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.735 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.735 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.735 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.735 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.735 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.735 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.735 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-11T22:07:28.735 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.736 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.736 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.736 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.736 INFO:teuthology.orchestra.run.smithi001.stdout: "contact": { 2024-09-11T22:07:28.736 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "contact", 2024-09-11T22:07:28.736 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.736 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.736 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.737 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.737 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.737 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.737 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.737 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.737 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.737 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.737 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.738 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.738 INFO:teuthology.orchestra.run.smithi001.stdout: "description": { 2024-09-11T22:07:28.738 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "description", 2024-09-11T22:07:28.738 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.738 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.738 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.738 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.738 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.739 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.739 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.739 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.739 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.739 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.739 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.739 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.739 INFO:teuthology.orchestra.run.smithi001.stdout: "device_url": { 2024-09-11T22:07:28.740 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "device_url", 2024-09-11T22:07:28.740 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.740 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.740 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.740 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-11T22:07:28.740 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.740 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.740 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.741 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.741 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.741 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.741 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.741 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.741 INFO:teuthology.orchestra.run.smithi001.stdout: "enabled": { 2024-09-11T22:07:28.741 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enabled", 2024-09-11T22:07:28.741 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.742 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.742 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.742 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.742 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.742 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.742 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.742 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.742 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.743 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.743 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.743 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.743 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-09-11T22:07:28.743 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-09-11T22:07:28.743 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:28.743 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.743 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.744 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "24", 2024-09-11T22:07:28.744 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "8", 2024-09-11T22:07:28.744 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.744 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.744 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.744 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.744 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.744 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.745 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.745 INFO:teuthology.orchestra.run.smithi001.stdout: "last_opt_revision": { 2024-09-11T22:07:28.745 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "last_opt_revision", 2024-09-11T22:07:28.745 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:28.745 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.745 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.745 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "1", 2024-09-11T22:07:28.745 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.746 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.746 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.746 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.746 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.746 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.746 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.746 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.746 INFO:teuthology.orchestra.run.smithi001.stdout: "leaderboard": { 2024-09-11T22:07:28.747 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "leaderboard", 2024-09-11T22:07:28.747 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.747 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.747 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.747 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.747 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.747 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.747 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.748 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.748 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.748 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.748 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.748 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.748 INFO:teuthology.orchestra.run.smithi001.stdout: "leaderboard_description": { 2024-09-11T22:07:28.748 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "leaderboard_description", 2024-09-11T22:07:28.748 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.749 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.749 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.749 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.749 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.749 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.749 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.749 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.750 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.750 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.750 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.750 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.750 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:28.750 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:28.750 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.750 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.751 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.751 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.751 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.751 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.751 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.751 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.751 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.751 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.752 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.752 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.752 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.752 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.752 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.752 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.752 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.752 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.753 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.753 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:28.753 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:28.753 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.753 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.753 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.753 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.753 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.753 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.754 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.754 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.754 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.754 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.754 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.754 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.754 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:28.754 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:28.755 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.755 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.755 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.755 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:28.755 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.755 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.755 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.755 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.756 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.756 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.756 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.756 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.756 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.756 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.756 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.756 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.757 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.757 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.757 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.757 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:28.757 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:28.757 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.757 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.758 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.758 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.758 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.758 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.758 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.758 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.758 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.759 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.759 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.759 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.759 INFO:teuthology.orchestra.run.smithi001.stdout: "organization": { 2024-09-11T22:07:28.759 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "organization", 2024-09-11T22:07:28.759 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.759 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.759 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.760 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.760 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.760 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.760 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.760 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.760 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.760 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.760 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.761 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.761 INFO:teuthology.orchestra.run.smithi001.stdout: "proxy": { 2024-09-11T22:07:28.761 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "proxy", 2024-09-11T22:07:28.761 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.761 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.761 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.761 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.761 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.762 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.762 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.762 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.762 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.762 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.762 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.762 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.762 INFO:teuthology.orchestra.run.smithi001.stdout: "url": { 2024-09-11T22:07:28.763 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "url", 2024-09-11T22:07:28.763 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.763 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.763 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.763 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-11T22:07:28.763 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.763 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.764 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.764 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.764 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.764 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.764 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.764 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.764 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.764 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.765 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:28.765 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "test_orchestrator", 2024-09-11T22:07:28.765 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:28.765 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:28.765 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:28.765 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:28.765 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:28.765 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.766 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.766 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.766 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.766 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.766 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.766 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.766 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.766 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.767 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.767 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.767 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.767 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.767 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.767 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.767 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.767 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.768 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.768 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.768 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:28.768 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:28.768 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.768 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.768 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.769 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.769 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.769 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.769 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.769 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.769 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.769 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.769 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.770 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.770 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:28.770 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:28.770 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.770 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.770 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.770 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:28.770 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.771 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.771 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.771 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.771 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.771 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.771 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.771 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.771 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.771 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.772 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.772 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.772 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.772 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.772 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.772 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:28.772 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:28.772 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.773 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.773 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.773 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.773 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.773 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.773 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.773 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.774 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.774 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.774 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.774 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.774 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.774 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.774 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:28.774 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "volumes", 2024-09-11T22:07:28.775 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:28.775 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:28.775 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:28.775 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:28.775 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:28.775 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.775 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.776 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.776 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.776 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.776 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.776 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.776 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.776 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.776 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.777 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.777 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.777 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.777 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.777 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.777 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.777 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.777 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.778 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.778 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:28.778 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:28.778 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.778 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.778 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.778 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.778 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.779 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.779 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.779 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.779 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.779 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.779 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.779 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.779 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:28.780 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:28.780 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.780 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.780 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.780 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:28.780 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.780 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.780 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.781 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.781 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.781 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.781 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.781 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.781 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.781 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.781 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.782 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.782 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.782 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.782 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.782 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:28.782 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:28.782 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.782 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.783 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.783 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.783 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.783 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.783 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.783 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.783 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.783 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.784 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.784 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.784 INFO:teuthology.orchestra.run.smithi001.stdout: "max_concurrent_clones": { 2024-09-11T22:07:28.784 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_concurrent_clones", 2024-09-11T22:07:28.784 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:28.784 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.784 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.784 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "4", 2024-09-11T22:07:28.785 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.785 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.785 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.785 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-11T22:07:28.785 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.785 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.785 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.785 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.786 INFO:teuthology.orchestra.run.smithi001.stdout: "periodic_async_work": { 2024-09-11T22:07:28.786 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "periodic_async_work", 2024-09-11T22:07:28.786 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.786 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.786 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.786 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.786 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.786 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.787 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.787 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Periodically check for async work", 2024-09-11T22:07:28.787 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.787 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.787 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.787 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.787 INFO:teuthology.orchestra.run.smithi001.stdout: "snapshot_clone_delay": { 2024-09-11T22:07:28.787 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "snapshot_clone_delay", 2024-09-11T22:07:28.788 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:28.788 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.788 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.788 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-09-11T22:07:28.788 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.788 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.788 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.788 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-11T22:07:28.789 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.789 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.789 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.789 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.789 INFO:teuthology.orchestra.run.smithi001.stdout: "snapshot_clone_no_wait": { 2024-09-11T22:07:28.789 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "snapshot_clone_no_wait", 2024-09-11T22:07:28.789 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.789 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.790 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.790 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-09-11T22:07:28.790 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.790 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.790 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.790 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-11T22:07:28.790 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.790 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.791 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.791 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.791 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.791 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.791 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:28.791 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix", 2024-09-11T22:07:28.791 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-09-11T22:07:28.791 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-09-11T22:07:28.792 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-09-11T22:07:28.792 INFO:teuthology.orchestra.run.smithi001.stdout: "discovery_interval": { 2024-09-11T22:07:28.792 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "discovery_interval", 2024-09-11T22:07:28.792 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-09-11T22:07:28.792 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.792 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.792 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "100", 2024-09-11T22:07:28.792 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.793 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.793 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.793 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.793 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.793 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.793 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.793 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.793 INFO:teuthology.orchestra.run.smithi001.stdout: "identifier": { 2024-09-11T22:07:28.794 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "identifier", 2024-09-11T22:07:28.794 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.794 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.794 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.794 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.794 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.794 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.794 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.795 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.795 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.795 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.795 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.795 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.795 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-09-11T22:07:28.795 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-09-11T22:07:28.795 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-09-11T22:07:28.796 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.796 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.796 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-09-11T22:07:28.796 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.796 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.796 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.796 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.797 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.797 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.797 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.797 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.797 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-09-11T22:07:28.797 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-09-11T22:07:28.797 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.797 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.798 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.798 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.798 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.798 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.798 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.798 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.798 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.798 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.799 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.799 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.799 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.799 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.799 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.799 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.799 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.799 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.800 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.800 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-09-11T22:07:28.800 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-09-11T22:07:28.800 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.800 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.800 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.800 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.800 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.801 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.801 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.801 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.801 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.801 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.801 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.801 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.802 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-09-11T22:07:28.802 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-09-11T22:07:28.802 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.802 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.802 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.802 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-09-11T22:07:28.802 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.802 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.803 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-09-11T22:07:28.803 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-09-11T22:07:28.803 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-09-11T22:07:28.803 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-09-11T22:07:28.803 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-09-11T22:07:28.803 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-09-11T22:07:28.803 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-09-11T22:07:28.803 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.804 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.804 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.804 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.804 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.804 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.804 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-09-11T22:07:28.804 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-09-11T22:07:28.804 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-09-11T22:07:28.805 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.805 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-09-11T22:07:28.805 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-09-11T22:07:28.805 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.805 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.805 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.805 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.805 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.806 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.806 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.806 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.806 INFO:teuthology.orchestra.run.smithi001.stdout: "zabbix_host": { 2024-09-11T22:07:28.806 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix_host", 2024-09-11T22:07:28.806 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.806 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.806 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.807 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-09-11T22:07:28.807 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.807 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.807 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.807 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.807 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.807 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.807 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.808 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.808 INFO:teuthology.orchestra.run.smithi001.stdout: "zabbix_port": { 2024-09-11T22:07:28.808 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix_port", 2024-09-11T22:07:28.808 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-09-11T22:07:28.808 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.808 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.808 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10051", 2024-09-11T22:07:28.809 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.809 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.809 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.809 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.809 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.809 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.809 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.809 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.810 INFO:teuthology.orchestra.run.smithi001.stdout: "zabbix_sender": { 2024-09-11T22:07:28.810 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix_sender", 2024-09-11T22:07:28.810 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-09-11T22:07:28.810 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-09-11T22:07:28.810 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-09-11T22:07:28.810 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-11T22:07:28.810 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-09-11T22:07:28.810 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-09-11T22:07:28.811 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-09-11T22:07:28.811 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-09-11T22:07:28.811 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-09-11T22:07:28.811 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-09-11T22:07:28.811 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-09-11T22:07:28.811 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.811 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.811 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.812 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.812 INFO:teuthology.orchestra.run.smithi001.stdout: "services": { 2024-09-11T22:07:28.812 INFO:teuthology.orchestra.run.smithi001.stdout: "dashboard": "https://172.21.15.1:8443/" 2024-09-11T22:07:28.812 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.812 INFO:teuthology.orchestra.run.smithi001.stdout: "always_on_modules": { 2024-09-11T22:07:28.812 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus": [ 2024-09-11T22:07:28.812 INFO:teuthology.orchestra.run.smithi001.stdout: "balancer", 2024-09-11T22:07:28.812 INFO:teuthology.orchestra.run.smithi001.stdout: "crash", 2024-09-11T22:07:28.813 INFO:teuthology.orchestra.run.smithi001.stdout: "devicehealth", 2024-09-11T22:07:28.813 INFO:teuthology.orchestra.run.smithi001.stdout: "orchestrator", 2024-09-11T22:07:28.813 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscaler", 2024-09-11T22:07:28.813 INFO:teuthology.orchestra.run.smithi001.stdout: "progress", 2024-09-11T22:07:28.813 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_support", 2024-09-11T22:07:28.813 INFO:teuthology.orchestra.run.smithi001.stdout: "status", 2024-09-11T22:07:28.813 INFO:teuthology.orchestra.run.smithi001.stdout: "telemetry", 2024-09-11T22:07:28.813 INFO:teuthology.orchestra.run.smithi001.stdout: "volumes" 2024-09-11T22:07:28.814 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.814 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific": [ 2024-09-11T22:07:28.814 INFO:teuthology.orchestra.run.smithi001.stdout: "balancer", 2024-09-11T22:07:28.814 INFO:teuthology.orchestra.run.smithi001.stdout: "crash", 2024-09-11T22:07:28.814 INFO:teuthology.orchestra.run.smithi001.stdout: "devicehealth", 2024-09-11T22:07:28.814 INFO:teuthology.orchestra.run.smithi001.stdout: "orchestrator", 2024-09-11T22:07:28.814 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscaler", 2024-09-11T22:07:28.814 INFO:teuthology.orchestra.run.smithi001.stdout: "progress", 2024-09-11T22:07:28.815 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_support", 2024-09-11T22:07:28.815 INFO:teuthology.orchestra.run.smithi001.stdout: "status", 2024-09-11T22:07:28.815 INFO:teuthology.orchestra.run.smithi001.stdout: "telemetry", 2024-09-11T22:07:28.815 INFO:teuthology.orchestra.run.smithi001.stdout: "volumes" 2024-09-11T22:07:28.815 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.815 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy": [ 2024-09-11T22:07:28.815 INFO:teuthology.orchestra.run.smithi001.stdout: "balancer", 2024-09-11T22:07:28.815 INFO:teuthology.orchestra.run.smithi001.stdout: "crash", 2024-09-11T22:07:28.816 INFO:teuthology.orchestra.run.smithi001.stdout: "devicehealth", 2024-09-11T22:07:28.816 INFO:teuthology.orchestra.run.smithi001.stdout: "orchestrator", 2024-09-11T22:07:28.816 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscaler", 2024-09-11T22:07:28.816 INFO:teuthology.orchestra.run.smithi001.stdout: "progress", 2024-09-11T22:07:28.816 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_support", 2024-09-11T22:07:28.816 INFO:teuthology.orchestra.run.smithi001.stdout: "status", 2024-09-11T22:07:28.816 INFO:teuthology.orchestra.run.smithi001.stdout: "telemetry", 2024-09-11T22:07:28.816 INFO:teuthology.orchestra.run.smithi001.stdout: "volumes" 2024-09-11T22:07:28.817 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.817 INFO:teuthology.orchestra.run.smithi001.stdout: "reef": [ 2024-09-11T22:07:28.817 INFO:teuthology.orchestra.run.smithi001.stdout: "balancer", 2024-09-11T22:07:28.817 INFO:teuthology.orchestra.run.smithi001.stdout: "crash", 2024-09-11T22:07:28.817 INFO:teuthology.orchestra.run.smithi001.stdout: "devicehealth", 2024-09-11T22:07:28.817 INFO:teuthology.orchestra.run.smithi001.stdout: "orchestrator", 2024-09-11T22:07:28.817 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscaler", 2024-09-11T22:07:28.817 INFO:teuthology.orchestra.run.smithi001.stdout: "progress", 2024-09-11T22:07:28.818 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_support", 2024-09-11T22:07:28.818 INFO:teuthology.orchestra.run.smithi001.stdout: "status", 2024-09-11T22:07:28.818 INFO:teuthology.orchestra.run.smithi001.stdout: "telemetry", 2024-09-11T22:07:28.818 INFO:teuthology.orchestra.run.smithi001.stdout: "volumes" 2024-09-11T22:07:28.818 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-09-11T22:07:28.818 INFO:teuthology.orchestra.run.smithi001.stdout: "squid": [ 2024-09-11T22:07:28.818 INFO:teuthology.orchestra.run.smithi001.stdout: "balancer", 2024-09-11T22:07:28.818 INFO:teuthology.orchestra.run.smithi001.stdout: "crash", 2024-09-11T22:07:28.819 INFO:teuthology.orchestra.run.smithi001.stdout: "devicehealth", 2024-09-11T22:07:28.819 INFO:teuthology.orchestra.run.smithi001.stdout: "orchestrator", 2024-09-11T22:07:28.819 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscaler", 2024-09-11T22:07:28.819 INFO:teuthology.orchestra.run.smithi001.stdout: "progress", 2024-09-11T22:07:28.819 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_support", 2024-09-11T22:07:28.819 INFO:teuthology.orchestra.run.smithi001.stdout: "status", 2024-09-11T22:07:28.819 INFO:teuthology.orchestra.run.smithi001.stdout: "telemetry", 2024-09-11T22:07:28.820 INFO:teuthology.orchestra.run.smithi001.stdout: "volumes" 2024-09-11T22:07:28.820 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-11T22:07:28.820 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.820 INFO:teuthology.orchestra.run.smithi001.stdout: "last_failure_osd_epoch": 87, 2024-09-11T22:07:28.820 INFO:teuthology.orchestra.run.smithi001.stdout: "active_clients": [ 2024-09-11T22:07:28.820 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:28.820 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "devicehealth", 2024-09-11T22:07:28.820 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-11T22:07:28.821 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:28.821 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-11T22:07:28.821 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:0", 2024-09-11T22:07:28.821 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2228130772 2024-09-11T22:07:28.821 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.821 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-11T22:07:28.821 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.821 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:28.822 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "libcephsqlite", 2024-09-11T22:07:28.822 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-11T22:07:28.822 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:28.822 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-11T22:07:28.822 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:0", 2024-09-11T22:07:28.822 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2636039805 2024-09-11T22:07:28.822 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.822 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-11T22:07:28.823 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.823 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:28.823 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rbd_support", 2024-09-11T22:07:28.823 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-11T22:07:28.823 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:28.823 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-11T22:07:28.823 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:0", 2024-09-11T22:07:28.823 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2287164575 2024-09-11T22:07:28.824 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.824 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-11T22:07:28.824 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.824 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:28.824 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "snap_schedule", 2024-09-11T22:07:28.824 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-11T22:07:28.824 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:28.825 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-11T22:07:28.825 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:0", 2024-09-11T22:07:28.825 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 367460565 2024-09-11T22:07:28.825 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.825 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-11T22:07:28.825 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.825 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:28.825 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "snap_schedule", 2024-09-11T22:07:28.825 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-11T22:07:28.826 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:28.826 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-11T22:07:28.826 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:0", 2024-09-11T22:07:28.826 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1887930862 2024-09-11T22:07:28.826 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.826 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-11T22:07:28.826 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:28.827 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:28.827 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "volumes", 2024-09-11T22:07:28.827 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-09-11T22:07:28.827 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-09-11T22:07:28.827 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-09-11T22:07:28.827 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:0", 2024-09-11T22:07:28.827 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2364596759 2024-09-11T22:07:28.827 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.827 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-11T22:07:28.828 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:28.828 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-09-11T22:07:28.828 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-11T22:07:28.830 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:26 smithi161 ceph-mon[35615]: from='client.17847 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:07:28.830 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:26 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/1298310486' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:07:28.830 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:26 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/4262110506' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T22:07:28.830 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:26 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/426354021' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-11T22:07:28.831 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:27 smithi161 ceph-mon[35615]: pgmap v317: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 17 op/s 2024-09-11T22:07:28.831 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:27 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/446349685' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-11T22:07:28.831 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:27 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/227232084' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-11T22:07:28.831 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:28 smithi161 ceph-mon[35615]: pgmap v318: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-11T22:07:28.831 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:26 smithi053 ceph-mon[34908]: from='client.17847 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:07:28.831 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:26 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/1298310486' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:07:28.831 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:26 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/4262110506' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T22:07:28.831 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:26 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/426354021' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-11T22:07:28.832 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:27 smithi053 ceph-mon[34908]: pgmap v317: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 17 op/s 2024-09-11T22:07:28.832 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:27 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/446349685' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-11T22:07:28.832 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:27 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/227232084' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-11T22:07:28.832 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:28 smithi053 ceph-mon[34908]: pgmap v318: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-11T22:07:28.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:26.527+0000 7f307ea74640 1 -- 172.21.15.1:0/227232084 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f30640a2d00 msgr2=0x7f30640a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:28.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:26.527+0000 7f307ea74640 1 --2- 172.21.15.1:0/227232084 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f30640a2d00 0x7f30640a51c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f306000abe0 tx=0x7f306000a2d0 comp rx=0 tx=0).stop 2024-09-11T22:07:28.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:26.527+0000 7f307ea74640 1 -- 172.21.15.1:0/227232084 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f30781035b0 msgr2=0x7f30781987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:28.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:26.527+0000 7f307ea74640 1 --2- 172.21.15.1:0/227232084 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f30781035b0 0x7f30781987e0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f306800cce0 tx=0x7f306800a620 comp rx=0 tx=0).stop 2024-09-11T22:07:28.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:26.527+0000 7f307ea74640 1 -- 172.21.15.1:0/227232084 shutdown_connections 2024-09-11T22:07:28.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:26.527+0000 7f307ea74640 1 --2- 172.21.15.1:0/227232084 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f30640a2d00 0x7f30640a51c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:28.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:26.527+0000 7f307ea74640 1 --2- 172.21.15.1:0/227232084 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f307810c100 0x7f307819cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:28.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:26.527+0000 7f307ea74640 1 --2- 172.21.15.1:0/227232084 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f30781035b0 0x7f30781987e0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:28.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:26.527+0000 7f307ea74640 1 --2- 172.21.15.1:0/227232084 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3078102c00 0x7f30781982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:28.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:26.527+0000 7f307ea74640 1 -- 172.21.15.1:0/227232084 >> 172.21.15.1:0/227232084 conn(0x7f30780fc6e0 msgr2=0x7f30780fdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:07:28.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:26.528+0000 7f307ea74640 1 -- 172.21.15.1:0/227232084 shutdown_connections 2024-09-11T22:07:28.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:26.528+0000 7f307ea74640 1 -- 172.21.15.1:0/227232084 wait complete. 2024-09-11T22:07:28.840 DEBUG:tasks.check_counter:Getting stats from a 2024-09-11T22:07:28.840 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-11T22:07:28.841 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:27 smithi001 ceph-mon[30935]: pgmap v317: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 17 op/s 2024-09-11T22:07:28.841 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:27 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/446349685' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-11T22:07:28.841 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:27 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/227232084' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-11T22:07:28.841 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:28 smithi001 ceph-mon[30935]: pgmap v318: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-11T22:07:29.112 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:07:29.661 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-11T22:07:29.662 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T22:07:29.662 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 1261, 2024-09-11T22:07:29.662 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 1261, 2024-09-11T22:07:29.662 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 29744961, 2024-09-11T22:07:29.662 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 400175, 2024-09-11T22:07:29.662 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 8, 2024-09-11T22:07:29.663 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 4, 2024-09-11T22:07:29.663 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.139331668, 2024-09-11T22:07:29.663 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.037026942, 2024-09-11T22:07:29.663 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.054547108, 2024-09-11T22:07:29.663 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.036142956, 2024-09-11T22:07:29.663 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:07:29.663 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 1260, 2024-09-11T22:07:29.664 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.052595005, 2024-09-11T22:07:29.664 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000041742 2024-09-11T22:07:29.664 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:29.664 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:07:29.664 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:29.664 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:29.664 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:29.664 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:29.665 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-11T22:07:29.665 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 448 2024-09-11T22:07:29.665 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:29.665 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T22:07:29.665 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 963, 2024-09-11T22:07:29.665 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 1114, 2024-09-11T22:07:29.665 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 61503839, 2024-09-11T22:07:29.665 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 739474, 2024-09-11T22:07:29.666 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 8, 2024-09-11T22:07:29.666 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 6, 2024-09-11T22:07:29.666 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.134752412, 2024-09-11T22:07:29.666 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.029308268, 2024-09-11T22:07:29.666 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.066968900, 2024-09-11T22:07:29.666 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.029200511, 2024-09-11T22:07:29.666 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:07:29.667 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 1111, 2024-09-11T22:07:29.667 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.046384671, 2024-09-11T22:07:29.667 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000041750 2024-09-11T22:07:29.667 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:29.667 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:07:29.667 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:29.667 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:29.668 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:29.668 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:29.668 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-11T22:07:29.668 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 433808 2024-09-11T22:07:29.668 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:29.668 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T22:07:29.668 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 320, 2024-09-11T22:07:29.668 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 288, 2024-09-11T22:07:29.669 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 14345240, 2024-09-11T22:07:29.669 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 148448, 2024-09-11T22:07:29.669 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 5, 2024-09-11T22:07:29.669 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 4, 2024-09-11T22:07:29.669 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.054488986, 2024-09-11T22:07:29.669 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.016934322, 2024-09-11T22:07:29.669 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.048752609, 2024-09-11T22:07:29.670 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.001708426, 2024-09-11T22:07:29.670 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:07:29.670 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 287, 2024-09-11T22:07:29.670 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.014885490, 2024-09-11T22:07:29.670 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000051865 2024-09-11T22:07:29.670 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:29.670 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:07:29.671 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:29.671 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:29.671 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:29.671 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:29.671 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 1742400, 2024-09-11T22:07:29.671 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 125552 2024-09-11T22:07:29.671 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:29.672 INFO:teuthology.orchestra.run.smithi001.stdout: "cct": { 2024-09-11T22:07:29.672 INFO:teuthology.orchestra.run.smithi001.stdout: "total_workers": 1, 2024-09-11T22:07:29.672 INFO:teuthology.orchestra.run.smithi001.stdout: "unhealthy_workers": 0 2024-09-11T22:07:29.672 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:29.672 INFO:teuthology.orchestra.run.smithi001.stdout: "finisher-MDSRank": { 2024-09-11T22:07:29.672 INFO:teuthology.orchestra.run.smithi001.stdout: "queue_len": 0, 2024-09-11T22:07:29.672 INFO:teuthology.orchestra.run.smithi001.stdout: "complete_latency": { 2024-09-11T22:07:29.673 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 3703, 2024-09-11T22:07:29.673 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.099260207, 2024-09-11T22:07:29.673 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000026805 2024-09-11T22:07:29.673 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:29.673 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:29.673 INFO:teuthology.orchestra.run.smithi001.stdout: "finisher-PurgeQueue": { 2024-09-11T22:07:29.673 INFO:teuthology.orchestra.run.smithi001.stdout: "queue_len": 0, 2024-09-11T22:07:29.674 INFO:teuthology.orchestra.run.smithi001.stdout: "complete_latency": { 2024-09-11T22:07:29.674 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:29.674 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:29.674 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:29.674 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:29.674 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:29.674 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": { 2024-09-11T22:07:29.674 INFO:teuthology.orchestra.run.smithi001.stdout: "request": 0, 2024-09-11T22:07:29.675 INFO:teuthology.orchestra.run.smithi001.stdout: "reply": 0, 2024-09-11T22:07:29.675 INFO:teuthology.orchestra.run.smithi001.stdout: "reply_latency": { 2024-09-11T22:07:29.675 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:29.675 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:29.675 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:29.675 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:29.675 INFO:teuthology.orchestra.run.smithi001.stdout: "slow_reply": 0, 2024-09-11T22:07:29.675 INFO:teuthology.orchestra.run.smithi001.stdout: "forward": 0, 2024-09-11T22:07:29.676 INFO:teuthology.orchestra.run.smithi001.stdout: "dir_fetch_complete": 0, 2024-09-11T22:07:29.676 INFO:teuthology.orchestra.run.smithi001.stdout: "dir_fetch_keys": 0, 2024-09-11T22:07:29.676 INFO:teuthology.orchestra.run.smithi001.stdout: "dir_commit": 0, 2024-09-11T22:07:29.676 INFO:teuthology.orchestra.run.smithi001.stdout: "dir_split": 0, 2024-09-11T22:07:29.676 INFO:teuthology.orchestra.run.smithi001.stdout: "dir_merge": 0, 2024-09-11T22:07:29.676 INFO:teuthology.orchestra.run.smithi001.stdout: "inodes": 31209, 2024-09-11T22:07:29.676 INFO:teuthology.orchestra.run.smithi001.stdout: "inodes_top": 105, 2024-09-11T22:07:29.676 INFO:teuthology.orchestra.run.smithi001.stdout: "inodes_bottom": 31104, 2024-09-11T22:07:29.677 INFO:teuthology.orchestra.run.smithi001.stdout: "inodes_pin_tail": 0, 2024-09-11T22:07:29.677 INFO:teuthology.orchestra.run.smithi001.stdout: "inodes_pinned": 31059, 2024-09-11T22:07:29.677 INFO:teuthology.orchestra.run.smithi001.stdout: "inodes_expired": 0, 2024-09-11T22:07:29.677 INFO:teuthology.orchestra.run.smithi001.stdout: "inodes_with_caps": 0, 2024-09-11T22:07:29.677 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": 0, 2024-09-11T22:07:29.677 INFO:teuthology.orchestra.run.smithi001.stdout: "subtrees": 1, 2024-09-11T22:07:29.677 INFO:teuthology.orchestra.run.smithi001.stdout: "traverse": 0, 2024-09-11T22:07:29.678 INFO:teuthology.orchestra.run.smithi001.stdout: "traverse_hit": 0, 2024-09-11T22:07:29.678 INFO:teuthology.orchestra.run.smithi001.stdout: "traverse_forward": 0, 2024-09-11T22:07:29.678 INFO:teuthology.orchestra.run.smithi001.stdout: "traverse_discover": 0, 2024-09-11T22:07:29.678 INFO:teuthology.orchestra.run.smithi001.stdout: "traverse_dir_fetch": 0, 2024-09-11T22:07:29.678 INFO:teuthology.orchestra.run.smithi001.stdout: "traverse_remote_ino": 0, 2024-09-11T22:07:29.678 INFO:teuthology.orchestra.run.smithi001.stdout: "traverse_lock": 0, 2024-09-11T22:07:29.678 INFO:teuthology.orchestra.run.smithi001.stdout: "load_cent": 0, 2024-09-11T22:07:29.678 INFO:teuthology.orchestra.run.smithi001.stdout: "q": 0, 2024-09-11T22:07:29.679 INFO:teuthology.orchestra.run.smithi001.stdout: "exported": 0, 2024-09-11T22:07:29.679 INFO:teuthology.orchestra.run.smithi001.stdout: "exported_inodes": 0, 2024-09-11T22:07:29.679 INFO:teuthology.orchestra.run.smithi001.stdout: "imported": 0, 2024-09-11T22:07:29.679 INFO:teuthology.orchestra.run.smithi001.stdout: "imported_inodes": 0, 2024-09-11T22:07:29.679 INFO:teuthology.orchestra.run.smithi001.stdout: "openino_dir_fetch": 0, 2024-09-11T22:07:29.679 INFO:teuthology.orchestra.run.smithi001.stdout: "openino_backtrace_fetch": 0, 2024-09-11T22:07:29.679 INFO:teuthology.orchestra.run.smithi001.stdout: "openino_peer_discover": 0, 2024-09-11T22:07:29.680 INFO:teuthology.orchestra.run.smithi001.stdout: "root_rfiles": 0, 2024-09-11T22:07:29.680 INFO:teuthology.orchestra.run.smithi001.stdout: "root_rbytes": 0, 2024-09-11T22:07:29.680 INFO:teuthology.orchestra.run.smithi001.stdout: "root_rsnaps": 0, 2024-09-11T22:07:29.680 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_backtrace_fetch": 0, 2024-09-11T22:07:29.680 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_set_tag": 0, 2024-09-11T22:07:29.680 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T22:07:29.680 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_inotable_repaired": 0, 2024-09-11T22:07:29.680 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_dir_inodes": 0, 2024-09-11T22:07:29.681 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T22:07:29.681 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T22:07:29.681 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_file_inodes": 0, 2024-09-11T22:07:29.681 INFO:teuthology.orchestra.run.smithi001.stdout: "handle_inode_file_caps": 0, 2024-09-11T22:07:29.681 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_cap_op_revoke": 0, 2024-09-11T22:07:29.681 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_cap_op_grant": 0, 2024-09-11T22:07:29.681 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T22:07:29.681 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T22:07:29.682 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T22:07:29.682 INFO:teuthology.orchestra.run.smithi001.stdout: "handle_client_caps": 0, 2024-09-11T22:07:29.682 INFO:teuthology.orchestra.run.smithi001.stdout: "handle_client_caps_dirty": 0, 2024-09-11T22:07:29.682 INFO:teuthology.orchestra.run.smithi001.stdout: "handle_client_cap_release": 0, 2024-09-11T22:07:29.682 INFO:teuthology.orchestra.run.smithi001.stdout: "process_request_cap_release": 0 2024-09-11T22:07:29.682 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:29.682 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_cache": { 2024-09-11T22:07:29.682 INFO:teuthology.orchestra.run.smithi001.stdout: "dir_update": 0, 2024-09-11T22:07:29.683 INFO:teuthology.orchestra.run.smithi001.stdout: "dir_update_receipt": 0, 2024-09-11T22:07:29.683 INFO:teuthology.orchestra.run.smithi001.stdout: "dir_try_discover": 0, 2024-09-11T22:07:29.683 INFO:teuthology.orchestra.run.smithi001.stdout: "dir_send_discover": 0, 2024-09-11T22:07:29.683 INFO:teuthology.orchestra.run.smithi001.stdout: "dir_handle_discover": 0, 2024-09-11T22:07:29.683 INFO:teuthology.orchestra.run.smithi001.stdout: "num_strays": 0, 2024-09-11T22:07:29.683 INFO:teuthology.orchestra.run.smithi001.stdout: "num_strays_delayed": 0, 2024-09-11T22:07:29.683 INFO:teuthology.orchestra.run.smithi001.stdout: "num_strays_enqueuing": 0, 2024-09-11T22:07:29.683 INFO:teuthology.orchestra.run.smithi001.stdout: "strays_created": 0, 2024-09-11T22:07:29.684 INFO:teuthology.orchestra.run.smithi001.stdout: "strays_enqueued": 0, 2024-09-11T22:07:29.684 INFO:teuthology.orchestra.run.smithi001.stdout: "strays_reintegrated": 0, 2024-09-11T22:07:29.684 INFO:teuthology.orchestra.run.smithi001.stdout: "strays_migrated": 0, 2024-09-11T22:07:29.684 INFO:teuthology.orchestra.run.smithi001.stdout: "num_recovering_processing": 0, 2024-09-11T22:07:29.684 INFO:teuthology.orchestra.run.smithi001.stdout: "num_recovering_enqueued": 0, 2024-09-11T22:07:29.684 INFO:teuthology.orchestra.run.smithi001.stdout: "num_recovering_prioritized": 0, 2024-09-11T22:07:29.684 INFO:teuthology.orchestra.run.smithi001.stdout: "recovery_started": 0, 2024-09-11T22:07:29.684 INFO:teuthology.orchestra.run.smithi001.stdout: "recovery_completed": 0, 2024-09-11T22:07:29.685 INFO:teuthology.orchestra.run.smithi001.stdout: "ireq_quiesce_path": 0, 2024-09-11T22:07:29.685 INFO:teuthology.orchestra.run.smithi001.stdout: "ireq_quiesce_inode": 0, 2024-09-11T22:07:29.685 INFO:teuthology.orchestra.run.smithi001.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T22:07:29.685 INFO:teuthology.orchestra.run.smithi001.stdout: "ireq_exportdir": 0, 2024-09-11T22:07:29.685 INFO:teuthology.orchestra.run.smithi001.stdout: "ireq_flush": 0, 2024-09-11T22:07:29.685 INFO:teuthology.orchestra.run.smithi001.stdout: "ireq_fragmentdir": 0, 2024-09-11T22:07:29.685 INFO:teuthology.orchestra.run.smithi001.stdout: "ireq_fragstats": 0, 2024-09-11T22:07:29.685 INFO:teuthology.orchestra.run.smithi001.stdout: "ireq_inodestats": 0 2024-09-11T22:07:29.686 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:29.686 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_log": { 2024-09-11T22:07:29.686 INFO:teuthology.orchestra.run.smithi001.stdout: "evlrg": 0, 2024-09-11T22:07:29.686 INFO:teuthology.orchestra.run.smithi001.stdout: "evadd": 0, 2024-09-11T22:07:29.686 INFO:teuthology.orchestra.run.smithi001.stdout: "evex": 0, 2024-09-11T22:07:29.686 INFO:teuthology.orchestra.run.smithi001.stdout: "evtrm": 0, 2024-09-11T22:07:29.686 INFO:teuthology.orchestra.run.smithi001.stdout: "ev": 36610, 2024-09-11T22:07:29.686 INFO:teuthology.orchestra.run.smithi001.stdout: "evexg": 0, 2024-09-11T22:07:29.687 INFO:teuthology.orchestra.run.smithi001.stdout: "evexd": 0, 2024-09-11T22:07:29.687 INFO:teuthology.orchestra.run.smithi001.stdout: "segadd": 0, 2024-09-11T22:07:29.687 INFO:teuthology.orchestra.run.smithi001.stdout: "segex": 0, 2024-09-11T22:07:29.687 INFO:teuthology.orchestra.run.smithi001.stdout: "segtrm": 0, 2024-09-11T22:07:29.687 INFO:teuthology.orchestra.run.smithi001.stdout: "seg": 51, 2024-09-11T22:07:29.687 INFO:teuthology.orchestra.run.smithi001.stdout: "segmjr": 4, 2024-09-11T22:07:29.687 INFO:teuthology.orchestra.run.smithi001.stdout: "segexg": 0, 2024-09-11T22:07:29.687 INFO:teuthology.orchestra.run.smithi001.stdout: "segexd": 0, 2024-09-11T22:07:29.688 INFO:teuthology.orchestra.run.smithi001.stdout: "expos": 4194304, 2024-09-11T22:07:29.688 INFO:teuthology.orchestra.run.smithi001.stdout: "wrpos": 107392014, 2024-09-11T22:07:29.688 INFO:teuthology.orchestra.run.smithi001.stdout: "rdpos": 107211669, 2024-09-11T22:07:29.688 INFO:teuthology.orchestra.run.smithi001.stdout: "jlat": { 2024-09-11T22:07:29.688 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:29.688 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:29.688 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:29.688 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:29.689 INFO:teuthology.orchestra.run.smithi001.stdout: "replayed": 36610 2024-09-11T22:07:29.689 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:29.689 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_mem": { 2024-09-11T22:07:29.689 INFO:teuthology.orchestra.run.smithi001.stdout: "ino": 14087, 2024-09-11T22:07:29.689 INFO:teuthology.orchestra.run.smithi001.stdout: "ino+": 32877, 2024-09-11T22:07:29.689 INFO:teuthology.orchestra.run.smithi001.stdout: "ino-": 18790, 2024-09-11T22:07:29.689 INFO:teuthology.orchestra.run.smithi001.stdout: "dir": 523, 2024-09-11T22:07:29.689 INFO:teuthology.orchestra.run.smithi001.stdout: "dir+": 1454, 2024-09-11T22:07:29.690 INFO:teuthology.orchestra.run.smithi001.stdout: "dir-": 931, 2024-09-11T22:07:29.690 INFO:teuthology.orchestra.run.smithi001.stdout: "dn": 31209, 2024-09-11T22:07:29.690 INFO:teuthology.orchestra.run.smithi001.stdout: "dn+": 46790, 2024-09-11T22:07:29.690 INFO:teuthology.orchestra.run.smithi001.stdout: "dn-": 15581, 2024-09-11T22:07:29.690 INFO:teuthology.orchestra.run.smithi001.stdout: "cap": 0, 2024-09-11T22:07:29.690 INFO:teuthology.orchestra.run.smithi001.stdout: "cap+": 0, 2024-09-11T22:07:29.690 INFO:teuthology.orchestra.run.smithi001.stdout: "cap-": 0, 2024-09-11T22:07:29.690 INFO:teuthology.orchestra.run.smithi001.stdout: "rss": 127004, 2024-09-11T22:07:29.691 INFO:teuthology.orchestra.run.smithi001.stdout: "heap": 215328 2024-09-11T22:07:29.691 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:29.691 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_server": { 2024-09-11T22:07:29.691 INFO:teuthology.orchestra.run.smithi001.stdout: "dispatch_client_request": 0, 2024-09-11T22:07:29.691 INFO:teuthology.orchestra.run.smithi001.stdout: "dispatch_server_request": 0, 2024-09-11T22:07:29.691 INFO:teuthology.orchestra.run.smithi001.stdout: "handle_client_request": 0, 2024-09-11T22:07:29.691 INFO:teuthology.orchestra.run.smithi001.stdout: "handle_client_session": 0, 2024-09-11T22:07:29.691 INFO:teuthology.orchestra.run.smithi001.stdout: "handle_peer_request": 0, 2024-09-11T22:07:29.692 INFO:teuthology.orchestra.run.smithi001.stdout: "req_create_latency": { 2024-09-11T22:07:29.692 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:29.692 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:29.692 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:29.692 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:29.692 INFO:teuthology.orchestra.run.smithi001.stdout: "req_getattr_latency": { 2024-09-11T22:07:29.692 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:29.692 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:29.693 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:29.693 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:29.693 INFO:teuthology.orchestra.run.smithi001.stdout: "req_getfilelock_latency": { 2024-09-11T22:07:29.693 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:29.693 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:29.693 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:29.693 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:29.693 INFO:teuthology.orchestra.run.smithi001.stdout: "req_link_latency": { 2024-09-11T22:07:29.694 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:29.694 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:29.694 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:29.694 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:29.694 INFO:teuthology.orchestra.run.smithi001.stdout: "req_lookup_latency": { 2024-09-11T22:07:29.694 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:29.694 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:29.694 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:29.695 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:29.695 INFO:teuthology.orchestra.run.smithi001.stdout: "req_lookuphash_latency": { 2024-09-11T22:07:29.695 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:29.695 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:29.695 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:29.695 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:29.695 INFO:teuthology.orchestra.run.smithi001.stdout: "req_lookupino_latency": { 2024-09-11T22:07:29.695 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:29.696 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:29.696 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:29.696 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:29.696 INFO:teuthology.orchestra.run.smithi001.stdout: "req_lookupname_latency": { 2024-09-11T22:07:29.696 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:29.696 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:29.696 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:29.697 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:29.697 INFO:teuthology.orchestra.run.smithi001.stdout: "req_lookupparent_latency": { 2024-09-11T22:07:29.697 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:29.697 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:29.697 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:29.697 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:29.697 INFO:teuthology.orchestra.run.smithi001.stdout: "req_lookupsnap_latency": { 2024-09-11T22:07:29.697 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:29.698 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:29.698 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:29.698 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:29.698 INFO:teuthology.orchestra.run.smithi001.stdout: "req_lssnap_latency": { 2024-09-11T22:07:29.698 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:29.698 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:29.698 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:29.698 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:29.699 INFO:teuthology.orchestra.run.smithi001.stdout: "req_mkdir_latency": { 2024-09-11T22:07:29.699 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:29.699 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:29.699 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:29.700 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:29.700 INFO:teuthology.orchestra.run.smithi001.stdout: "req_mknod_latency": { 2024-09-11T22:07:29.700 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:29.700 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:29.700 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:29.700 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:29.700 INFO:teuthology.orchestra.run.smithi001.stdout: "req_mksnap_latency": { 2024-09-11T22:07:29.700 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:29.701 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:29.701 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:29.701 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:29.701 INFO:teuthology.orchestra.run.smithi001.stdout: "req_open_latency": { 2024-09-11T22:07:29.701 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:29.701 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:29.701 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:29.702 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:29.702 INFO:teuthology.orchestra.run.smithi001.stdout: "req_readdir_latency": { 2024-09-11T22:07:29.702 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:29.702 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:29.702 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:29.702 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:29.702 INFO:teuthology.orchestra.run.smithi001.stdout: "req_rename_latency": { 2024-09-11T22:07:29.703 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:29.703 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:29.703 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:29.703 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:29.703 INFO:teuthology.orchestra.run.smithi001.stdout: "req_renamesnap_latency": { 2024-09-11T22:07:29.703 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:29.703 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:29.704 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:29.704 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:29.704 INFO:teuthology.orchestra.run.smithi001.stdout: "req_snapdiff_latency": { 2024-09-11T22:07:29.704 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:29.704 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:29.704 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:29.704 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:29.704 INFO:teuthology.orchestra.run.smithi001.stdout: "req_rmdir_latency": { 2024-09-11T22:07:29.705 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:29.705 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:29.705 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:29.705 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:29.705 INFO:teuthology.orchestra.run.smithi001.stdout: "req_rmsnap_latency": { 2024-09-11T22:07:29.705 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:29.705 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:29.706 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:29.706 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:29.706 INFO:teuthology.orchestra.run.smithi001.stdout: "req_rmxattr_latency": { 2024-09-11T22:07:29.706 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:29.706 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:29.706 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:29.706 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:29.706 INFO:teuthology.orchestra.run.smithi001.stdout: "req_setattr_latency": { 2024-09-11T22:07:29.707 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:29.707 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:29.707 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:29.707 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:29.707 INFO:teuthology.orchestra.run.smithi001.stdout: "req_setdirlayout_latency": { 2024-09-11T22:07:29.707 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:29.707 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:29.707 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:29.708 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:29.708 INFO:teuthology.orchestra.run.smithi001.stdout: "req_setfilelock_latency": { 2024-09-11T22:07:29.708 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:29.708 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:29.708 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:29.708 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:29.708 INFO:teuthology.orchestra.run.smithi001.stdout: "req_setlayout_latency": { 2024-09-11T22:07:29.708 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:29.709 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:29.709 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:29.709 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:29.709 INFO:teuthology.orchestra.run.smithi001.stdout: "req_setxattr_latency": { 2024-09-11T22:07:29.709 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:29.709 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:29.709 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:29.710 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:29.710 INFO:teuthology.orchestra.run.smithi001.stdout: "req_symlink_latency": { 2024-09-11T22:07:29.710 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:29.710 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:29.710 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:29.710 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:29.710 INFO:teuthology.orchestra.run.smithi001.stdout: "req_unlink_latency": { 2024-09-11T22:07:29.710 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:29.711 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:29.711 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:29.711 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:29.711 INFO:teuthology.orchestra.run.smithi001.stdout: "cap_revoke_eviction": 0, 2024-09-11T22:07:29.711 INFO:teuthology.orchestra.run.smithi001.stdout: "cap_acquisition_throttle": 0, 2024-09-11T22:07:29.711 INFO:teuthology.orchestra.run.smithi001.stdout: "req_getvxattr_latency": { 2024-09-11T22:07:29.711 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:29.711 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:29.712 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:29.712 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:29.712 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:29.712 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_sessions": { 2024-09-11T22:07:29.712 INFO:teuthology.orchestra.run.smithi001.stdout: "session_count": 2, 2024-09-11T22:07:29.712 INFO:teuthology.orchestra.run.smithi001.stdout: "session_add": 4, 2024-09-11T22:07:29.712 INFO:teuthology.orchestra.run.smithi001.stdout: "session_remove": 2, 2024-09-11T22:07:29.713 INFO:teuthology.orchestra.run.smithi001.stdout: "sessions_open": 3, 2024-09-11T22:07:29.713 INFO:teuthology.orchestra.run.smithi001.stdout: "sessions_stale": 0, 2024-09-11T22:07:29.713 INFO:teuthology.orchestra.run.smithi001.stdout: "total_load": 0, 2024-09-11T22:07:29.713 INFO:teuthology.orchestra.run.smithi001.stdout: "average_load": 0, 2024-09-11T22:07:29.713 INFO:teuthology.orchestra.run.smithi001.stdout: "avg_session_uptime": 2189, 2024-09-11T22:07:29.713 INFO:teuthology.orchestra.run.smithi001.stdout: "mdthresh_evicted": 0 2024-09-11T22:07:29.713 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:29.713 INFO:teuthology.orchestra.run.smithi001.stdout: "mempool": { 2024-09-11T22:07:29.714 INFO:teuthology.orchestra.run.smithi001.stdout: "bloom_filter_bytes": 0, 2024-09-11T22:07:29.714 INFO:teuthology.orchestra.run.smithi001.stdout: "bloom_filter_items": 0, 2024-09-11T22:07:29.714 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T22:07:29.714 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_alloc_items": 0, 2024-09-11T22:07:29.714 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T22:07:29.714 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_data_items": 0, 2024-09-11T22:07:29.714 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T22:07:29.715 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T22:07:29.715 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T22:07:29.715 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T22:07:29.715 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T22:07:29.715 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_other_items": 0, 2024-09-11T22:07:29.715 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T22:07:29.715 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T22:07:29.715 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_extent_bytes": 0, 2024-09-11T22:07:29.716 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_extent_items": 0, 2024-09-11T22:07:29.716 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_blob_bytes": 0, 2024-09-11T22:07:29.716 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_blob_items": 0, 2024-09-11T22:07:29.716 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T22:07:29.716 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T22:07:29.716 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T22:07:29.716 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T22:07:29.717 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T22:07:29.717 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_items": 0, 2024-09-11T22:07:29.717 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_txc_bytes": 0, 2024-09-11T22:07:29.717 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_txc_items": 0, 2024-09-11T22:07:29.717 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T22:07:29.717 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T22:07:29.717 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_bytes": 0, 2024-09-11T22:07:29.717 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_items": 0, 2024-09-11T22:07:29.718 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_bytes": 0, 2024-09-11T22:07:29.718 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_items": 0, 2024-09-11T22:07:29.718 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T22:07:29.718 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_reader_items": 0, 2024-09-11T22:07:29.718 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T22:07:29.718 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_writer_items": 0, 2024-09-11T22:07:29.718 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_anon_bytes": 751503, 2024-09-11T22:07:29.719 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_anon_items": 14152, 2024-09-11T22:07:29.719 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_meta_bytes": 0, 2024-09-11T22:07:29.719 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_meta_items": 0, 2024-09-11T22:07:29.719 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_bytes": 0, 2024-09-11T22:07:29.719 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_items": 0, 2024-09-11T22:07:29.719 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mapbl_bytes": 0, 2024-09-11T22:07:29.719 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mapbl_items": 0, 2024-09-11T22:07:29.719 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pglog_bytes": 0, 2024-09-11T22:07:29.720 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pglog_items": 0, 2024-09-11T22:07:29.720 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_bytes": 8392, 2024-09-11T22:07:29.720 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_items": 184, 2024-09-11T22:07:29.720 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T22:07:29.720 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_mapping_items": 0, 2024-09-11T22:07:29.720 INFO:teuthology.orchestra.run.smithi001.stdout: "pgmap_bytes": 0, 2024-09-11T22:07:29.720 INFO:teuthology.orchestra.run.smithi001.stdout: "pgmap_items": 0, 2024-09-11T22:07:29.720 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_co_bytes": 54767914, 2024-09-11T22:07:29.721 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_co_items": 434515, 2024-09-11T22:07:29.721 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_1_bytes": 0, 2024-09-11T22:07:29.721 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_1_items": 0, 2024-09-11T22:07:29.721 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_2_bytes": 0, 2024-09-11T22:07:29.721 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_2_items": 0 2024-09-11T22:07:29.721 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:29.721 INFO:teuthology.orchestra.run.smithi001.stdout: "objecter": { 2024-09-11T22:07:29.722 INFO:teuthology.orchestra.run.smithi001.stdout: "op_active": 0, 2024-09-11T22:07:29.722 INFO:teuthology.orchestra.run.smithi001.stdout: "op_laggy": 0, 2024-09-11T22:07:29.722 INFO:teuthology.orchestra.run.smithi001.stdout: "op_send": 2290, 2024-09-11T22:07:29.722 INFO:teuthology.orchestra.run.smithi001.stdout: "op_send_bytes": 21, 2024-09-11T22:07:29.722 INFO:teuthology.orchestra.run.smithi001.stdout: "op_resend": 3, 2024-09-11T22:07:29.722 INFO:teuthology.orchestra.run.smithi001.stdout: "op_reply": 2290, 2024-09-11T22:07:29.722 INFO:teuthology.orchestra.run.smithi001.stdout: "op_latency": { 2024-09-11T22:07:29.722 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 2290, 2024-09-11T22:07:29.723 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 4.172947558, 2024-09-11T22:07:29.723 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.001822247 2024-09-11T22:07:29.723 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:29.723 INFO:teuthology.orchestra.run.smithi001.stdout: "op_inflight": 0, 2024-09-11T22:07:29.723 INFO:teuthology.orchestra.run.smithi001.stdout: "oplen_avg": { 2024-09-11T22:07:29.723 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 2290, 2024-09-11T22:07:29.723 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 2291 2024-09-11T22:07:29.724 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:29.724 INFO:teuthology.orchestra.run.smithi001.stdout: "op": 2290, 2024-09-11T22:07:29.724 INFO:teuthology.orchestra.run.smithi001.stdout: "op_r": 2290, 2024-09-11T22:07:29.724 INFO:teuthology.orchestra.run.smithi001.stdout: "op_w": 0, 2024-09-11T22:07:29.724 INFO:teuthology.orchestra.run.smithi001.stdout: "op_rmw": 0, 2024-09-11T22:07:29.724 INFO:teuthology.orchestra.run.smithi001.stdout: "op_pg": 0, 2024-09-11T22:07:29.724 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_stat": 1432, 2024-09-11T22:07:29.724 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_create": 0, 2024-09-11T22:07:29.725 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_read": 856, 2024-09-11T22:07:29.725 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_write": 0, 2024-09-11T22:07:29.725 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_writefull": 0, 2024-09-11T22:07:29.725 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_writesame": 0, 2024-09-11T22:07:29.725 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_append": 0, 2024-09-11T22:07:29.725 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_zero": 0, 2024-09-11T22:07:29.725 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_truncate": 0, 2024-09-11T22:07:29.725 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_delete": 0, 2024-09-11T22:07:29.726 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_mapext": 0, 2024-09-11T22:07:29.726 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_sparse_read": 0, 2024-09-11T22:07:29.726 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_clonerange": 0, 2024-09-11T22:07:29.726 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_getxattr": 1, 2024-09-11T22:07:29.726 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_setxattr": 0, 2024-09-11T22:07:29.726 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_cmpxattr": 0, 2024-09-11T22:07:29.726 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_rmxattr": 0, 2024-09-11T22:07:29.726 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_resetxattrs": 0, 2024-09-11T22:07:29.727 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_call": 0, 2024-09-11T22:07:29.727 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_watch": 0, 2024-09-11T22:07:29.727 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_notify": 0, 2024-09-11T22:07:29.727 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T22:07:29.727 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_pgls": 0, 2024-09-11T22:07:29.727 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_pgls_filter": 0, 2024-09-11T22:07:29.727 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_other": 0, 2024-09-11T22:07:29.728 INFO:teuthology.orchestra.run.smithi001.stdout: "linger_active": 0, 2024-09-11T22:07:29.728 INFO:teuthology.orchestra.run.smithi001.stdout: "linger_send": 0, 2024-09-11T22:07:29.728 INFO:teuthology.orchestra.run.smithi001.stdout: "linger_resend": 0, 2024-09-11T22:07:29.728 INFO:teuthology.orchestra.run.smithi001.stdout: "linger_ping": 0, 2024-09-11T22:07:29.728 INFO:teuthology.orchestra.run.smithi001.stdout: "poolop_active": 0, 2024-09-11T22:07:29.728 INFO:teuthology.orchestra.run.smithi001.stdout: "poolop_send": 0, 2024-09-11T22:07:29.728 INFO:teuthology.orchestra.run.smithi001.stdout: "poolop_resend": 0, 2024-09-11T22:07:29.728 INFO:teuthology.orchestra.run.smithi001.stdout: "poolstat_active": 0, 2024-09-11T22:07:29.729 INFO:teuthology.orchestra.run.smithi001.stdout: "poolstat_send": 0, 2024-09-11T22:07:29.729 INFO:teuthology.orchestra.run.smithi001.stdout: "poolstat_resend": 0, 2024-09-11T22:07:29.729 INFO:teuthology.orchestra.run.smithi001.stdout: "statfs_active": 0, 2024-09-11T22:07:29.729 INFO:teuthology.orchestra.run.smithi001.stdout: "statfs_send": 0, 2024-09-11T22:07:29.729 INFO:teuthology.orchestra.run.smithi001.stdout: "statfs_resend": 0, 2024-09-11T22:07:29.729 INFO:teuthology.orchestra.run.smithi001.stdout: "command_active": 0, 2024-09-11T22:07:29.729 INFO:teuthology.orchestra.run.smithi001.stdout: "command_send": 0, 2024-09-11T22:07:29.729 INFO:teuthology.orchestra.run.smithi001.stdout: "command_resend": 0, 2024-09-11T22:07:29.730 INFO:teuthology.orchestra.run.smithi001.stdout: "map_epoch": 95, 2024-09-11T22:07:29.730 INFO:teuthology.orchestra.run.smithi001.stdout: "map_full": 0, 2024-09-11T22:07:29.730 INFO:teuthology.orchestra.run.smithi001.stdout: "map_inc": 11, 2024-09-11T22:07:29.730 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_sessions": 10, 2024-09-11T22:07:29.730 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_session_open": 10, 2024-09-11T22:07:29.730 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_session_close": 0, 2024-09-11T22:07:29.730 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_laggy": 0, 2024-09-11T22:07:29.730 INFO:teuthology.orchestra.run.smithi001.stdout: "omap_wr": 0, 2024-09-11T22:07:29.731 INFO:teuthology.orchestra.run.smithi001.stdout: "omap_rd": 2, 2024-09-11T22:07:29.731 INFO:teuthology.orchestra.run.smithi001.stdout: "omap_del": 0 2024-09-11T22:07:29.731 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:29.731 INFO:teuthology.orchestra.run.smithi001.stdout: "oft": { 2024-09-11T22:07:29.731 INFO:teuthology.orchestra.run.smithi001.stdout: "omap_total_objs": 0, 2024-09-11T22:07:29.731 INFO:teuthology.orchestra.run.smithi001.stdout: "omap_total_kv_pairs": 0, 2024-09-11T22:07:29.731 INFO:teuthology.orchestra.run.smithi001.stdout: "omap_total_updates": 0, 2024-09-11T22:07:29.731 INFO:teuthology.orchestra.run.smithi001.stdout: "omap_total_removes": 0 2024-09-11T22:07:29.732 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:29.732 INFO:teuthology.orchestra.run.smithi001.stdout: "osd-slow-ops": { 2024-09-11T22:07:29.732 INFO:teuthology.orchestra.run.smithi001.stdout: "slow_ops_count": 0 2024-09-11T22:07:29.732 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:29.732 INFO:teuthology.orchestra.run.smithi001.stdout: "purge_queue": { 2024-09-11T22:07:29.732 INFO:teuthology.orchestra.run.smithi001.stdout: "pq_executing_ops": 0, 2024-09-11T22:07:29.732 INFO:teuthology.orchestra.run.smithi001.stdout: "pq_executing_ops_high_water": 0, 2024-09-11T22:07:29.732 INFO:teuthology.orchestra.run.smithi001.stdout: "pq_executing": 0, 2024-09-11T22:07:29.733 INFO:teuthology.orchestra.run.smithi001.stdout: "pq_executing_high_water": 0, 2024-09-11T22:07:29.733 INFO:teuthology.orchestra.run.smithi001.stdout: "pq_executed_ops": 0, 2024-09-11T22:07:29.733 INFO:teuthology.orchestra.run.smithi001.stdout: "pq_executed": 0, 2024-09-11T22:07:29.733 INFO:teuthology.orchestra.run.smithi001.stdout: "pq_item_in_journal": 0 2024-09-11T22:07:29.733 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:29.733 INFO:teuthology.orchestra.run.smithi001.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T22:07:29.733 INFO:teuthology.orchestra.run.smithi001.stdout: "val": 0, 2024-09-11T22:07:29.733 INFO:teuthology.orchestra.run.smithi001.stdout: "max": 104857600, 2024-09-11T22:07:29.734 INFO:teuthology.orchestra.run.smithi001.stdout: "get_started": 0, 2024-09-11T22:07:29.734 INFO:teuthology.orchestra.run.smithi001.stdout: "get": 2540, 2024-09-11T22:07:29.734 INFO:teuthology.orchestra.run.smithi001.stdout: "get_sum": 105349845, 2024-09-11T22:07:29.734 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_fail": 0, 2024-09-11T22:07:29.734 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_success": 2540, 2024-09-11T22:07:29.734 INFO:teuthology.orchestra.run.smithi001.stdout: "take": 0, 2024-09-11T22:07:29.734 INFO:teuthology.orchestra.run.smithi001.stdout: "take_sum": 0, 2024-09-11T22:07:29.734 INFO:teuthology.orchestra.run.smithi001.stdout: "put": 2540, 2024-09-11T22:07:29.735 INFO:teuthology.orchestra.run.smithi001.stdout: "put_sum": 105349845, 2024-09-11T22:07:29.735 INFO:teuthology.orchestra.run.smithi001.stdout: "wait": { 2024-09-11T22:07:29.735 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:29.735 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:29.735 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:29.735 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:29.735 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:29.735 INFO:teuthology.orchestra.run.smithi001.stdout: "throttle-objecter_bytes": { 2024-09-11T22:07:29.736 INFO:teuthology.orchestra.run.smithi001.stdout: "val": 0, 2024-09-11T22:07:29.736 INFO:teuthology.orchestra.run.smithi001.stdout: "max": 104857600, 2024-09-11T22:07:29.736 INFO:teuthology.orchestra.run.smithi001.stdout: "get_started": 0, 2024-09-11T22:07:29.736 INFO:teuthology.orchestra.run.smithi001.stdout: "get": 0, 2024-09-11T22:07:29.736 INFO:teuthology.orchestra.run.smithi001.stdout: "get_sum": 0, 2024-09-11T22:07:29.736 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_fail": 0, 2024-09-11T22:07:29.736 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_success": 0, 2024-09-11T22:07:29.736 INFO:teuthology.orchestra.run.smithi001.stdout: "take": 2290, 2024-09-11T22:07:29.737 INFO:teuthology.orchestra.run.smithi001.stdout: "take_sum": 103206026, 2024-09-11T22:07:29.737 INFO:teuthology.orchestra.run.smithi001.stdout: "put": 2290, 2024-09-11T22:07:29.737 INFO:teuthology.orchestra.run.smithi001.stdout: "put_sum": 103206026, 2024-09-11T22:07:29.737 INFO:teuthology.orchestra.run.smithi001.stdout: "wait": { 2024-09-11T22:07:29.737 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:29.737 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:29.738 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:29.738 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:29.738 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:29.738 INFO:teuthology.orchestra.run.smithi001.stdout: "throttle-objecter_ops": { 2024-09-11T22:07:29.738 INFO:teuthology.orchestra.run.smithi001.stdout: "val": 0, 2024-09-11T22:07:29.738 INFO:teuthology.orchestra.run.smithi001.stdout: "max": 1024, 2024-09-11T22:07:29.738 INFO:teuthology.orchestra.run.smithi001.stdout: "get_started": 0, 2024-09-11T22:07:29.739 INFO:teuthology.orchestra.run.smithi001.stdout: "get": 0, 2024-09-11T22:07:29.739 INFO:teuthology.orchestra.run.smithi001.stdout: "get_sum": 0, 2024-09-11T22:07:29.739 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_fail": 0, 2024-09-11T22:07:29.739 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_success": 0, 2024-09-11T22:07:29.739 INFO:teuthology.orchestra.run.smithi001.stdout: "take": 2290, 2024-09-11T22:07:29.739 INFO:teuthology.orchestra.run.smithi001.stdout: "take_sum": 2290, 2024-09-11T22:07:29.739 INFO:teuthology.orchestra.run.smithi001.stdout: "put": 2290, 2024-09-11T22:07:29.739 INFO:teuthology.orchestra.run.smithi001.stdout: "put_sum": 2290, 2024-09-11T22:07:29.739 INFO:teuthology.orchestra.run.smithi001.stdout: "wait": { 2024-09-11T22:07:29.740 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:29.740 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:29.740 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:29.740 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:29.740 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:29.740 INFO:teuthology.orchestra.run.smithi001.stdout: "throttle-write_buf_throttle": { 2024-09-11T22:07:29.740 INFO:teuthology.orchestra.run.smithi001.stdout: "val": 0, 2024-09-11T22:07:29.740 INFO:teuthology.orchestra.run.smithi001.stdout: "max": 3758096384, 2024-09-11T22:07:29.741 INFO:teuthology.orchestra.run.smithi001.stdout: "get_started": 0, 2024-09-11T22:07:29.741 INFO:teuthology.orchestra.run.smithi001.stdout: "get": 0, 2024-09-11T22:07:29.741 INFO:teuthology.orchestra.run.smithi001.stdout: "get_sum": 0, 2024-09-11T22:07:29.741 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_fail": 0, 2024-09-11T22:07:29.741 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_success": 0, 2024-09-11T22:07:29.741 INFO:teuthology.orchestra.run.smithi001.stdout: "take": 0, 2024-09-11T22:07:29.741 INFO:teuthology.orchestra.run.smithi001.stdout: "take_sum": 0, 2024-09-11T22:07:29.741 INFO:teuthology.orchestra.run.smithi001.stdout: "put": 0, 2024-09-11T22:07:29.742 INFO:teuthology.orchestra.run.smithi001.stdout: "put_sum": 0, 2024-09-11T22:07:29.742 INFO:teuthology.orchestra.run.smithi001.stdout: "wait": { 2024-09-11T22:07:29.742 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:29.742 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:29.742 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:29.742 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:29.742 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:29.742 INFO:teuthology.orchestra.run.smithi001.stdout: "throttle-write_buf_throttle-0x5598b628c6e0": { 2024-09-11T22:07:29.743 INFO:teuthology.orchestra.run.smithi001.stdout: "val": 0, 2024-09-11T22:07:29.743 INFO:teuthology.orchestra.run.smithi001.stdout: "max": 3758096384, 2024-09-11T22:07:29.743 INFO:teuthology.orchestra.run.smithi001.stdout: "get_started": 0, 2024-09-11T22:07:29.743 INFO:teuthology.orchestra.run.smithi001.stdout: "get": 0, 2024-09-11T22:07:29.743 INFO:teuthology.orchestra.run.smithi001.stdout: "get_sum": 0, 2024-09-11T22:07:29.743 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_fail": 0, 2024-09-11T22:07:29.743 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_success": 0, 2024-09-11T22:07:29.743 INFO:teuthology.orchestra.run.smithi001.stdout: "take": 0, 2024-09-11T22:07:29.744 INFO:teuthology.orchestra.run.smithi001.stdout: "take_sum": 0, 2024-09-11T22:07:29.744 INFO:teuthology.orchestra.run.smithi001.stdout: "put": 0, 2024-09-11T22:07:29.744 INFO:teuthology.orchestra.run.smithi001.stdout: "put_sum": 0, 2024-09-11T22:07:29.744 INFO:teuthology.orchestra.run.smithi001.stdout: "wait": { 2024-09-11T22:07:29.744 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:29.744 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:29.744 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:29.744 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:29.745 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:29.745 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-11T22:07:30.599 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-09-11T22:07:30.600 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-09-11T22:07:30.600 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2024-09-11T22:07:30.600 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=0 2024-09-11T22:07:30.600 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2024-09-11T22:07:30.600 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-09-11T22:07:30.600 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-09-11T22:07:30.601 INFO:tasks.check_counter:Daemon mds.a mds.dir_split=0 2024-09-11T22:07:30.601 DEBUG:tasks.check_counter:Getting stats from d 2024-09-11T22:07:30.601 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-11T22:07:30.859 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:07:31.249 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:31 smithi001 ceph-mon[30935]: pgmap v319: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 17 op/s 2024-09-11T22:07:31.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:31 smithi053 ceph-mon[34908]: pgmap v319: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 17 op/s 2024-09-11T22:07:31.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:31 smithi161 ceph-mon[35615]: pgmap v319: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 17 op/s 2024-09-11T22:07:31.357 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-11T22:07:31.357 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T22:07:31.358 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 216, 2024-09-11T22:07:31.358 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 205, 2024-09-11T22:07:31.358 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 1579136, 2024-09-11T22:07:31.358 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 125984, 2024-09-11T22:07:31.358 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 4, 2024-09-11T22:07:31.358 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 0, 2024-09-11T22:07:31.358 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.040275442, 2024-09-11T22:07:31.359 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.014294805, 2024-09-11T22:07:31.359 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.038559044, 2024-09-11T22:07:31.359 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T22:07:31.359 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:07:31.359 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 204, 2024-09-11T22:07:31.359 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.010796283, 2024-09-11T22:07:31.359 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000052922 2024-09-11T22:07:31.359 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:31.360 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:07:31.360 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:31.360 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:31.360 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:31.360 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:31.360 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 1579136, 2024-09-11T22:07:31.360 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 125984 2024-09-11T22:07:31.361 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:31.361 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T22:07:31.361 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 2, 2024-09-11T22:07:31.361 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 157, 2024-09-11T22:07:31.361 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 320, 2024-09-11T22:07:31.361 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 184864, 2024-09-11T22:07:31.361 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 5, 2024-09-11T22:07:31.361 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 3, 2024-09-11T22:07:31.362 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.011989544, 2024-09-11T22:07:31.362 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.008113414, 2024-09-11T22:07:31.362 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.001743788, 2024-09-11T22:07:31.362 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T22:07:31.362 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:07:31.362 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 154, 2024-09-11T22:07:31.362 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.011633880, 2024-09-11T22:07:31.362 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000075544 2024-09-11T22:07:31.363 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:31.363 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:07:31.363 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:31.363 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:31.363 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:31.363 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:31.363 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-11T22:07:31.363 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 184864 2024-09-11T22:07:31.364 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:31.364 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T22:07:31.364 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 4, 2024-09-11T22:07:31.364 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 4, 2024-09-11T22:07:31.364 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 3024, 2024-09-11T22:07:31.364 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 448, 2024-09-11T22:07:31.364 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 2, 2024-09-11T22:07:31.364 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 1, 2024-09-11T22:07:31.365 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.002297770, 2024-09-11T22:07:31.365 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.000031521, 2024-09-11T22:07:31.365 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.002232796, 2024-09-11T22:07:31.365 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T22:07:31.365 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:07:31.365 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 3, 2024-09-11T22:07:31.365 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.004074957, 2024-09-11T22:07:31.365 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.001358319 2024-09-11T22:07:31.366 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:31.366 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:07:31.366 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:31.366 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:31.366 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:31.366 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:31.366 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-11T22:07:31.366 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 448 2024-09-11T22:07:31.367 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:31.367 INFO:teuthology.orchestra.run.smithi001.stdout: "cct": { 2024-09-11T22:07:31.367 INFO:teuthology.orchestra.run.smithi001.stdout: "total_workers": 0, 2024-09-11T22:07:31.367 INFO:teuthology.orchestra.run.smithi001.stdout: "unhealthy_workers": 0 2024-09-11T22:07:31.367 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:31.367 INFO:teuthology.orchestra.run.smithi001.stdout: "mempool": { 2024-09-11T22:07:31.367 INFO:teuthology.orchestra.run.smithi001.stdout: "bloom_filter_bytes": 0, 2024-09-11T22:07:31.367 INFO:teuthology.orchestra.run.smithi001.stdout: "bloom_filter_items": 0, 2024-09-11T22:07:31.368 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T22:07:31.368 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_alloc_items": 0, 2024-09-11T22:07:31.368 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T22:07:31.368 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_data_items": 0, 2024-09-11T22:07:31.368 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T22:07:31.368 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T22:07:31.368 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T22:07:31.368 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T22:07:31.369 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T22:07:31.369 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_other_items": 0, 2024-09-11T22:07:31.369 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T22:07:31.369 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T22:07:31.369 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_extent_bytes": 0, 2024-09-11T22:07:31.369 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_extent_items": 0, 2024-09-11T22:07:31.369 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_blob_bytes": 0, 2024-09-11T22:07:31.370 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_blob_items": 0, 2024-09-11T22:07:31.370 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T22:07:31.370 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T22:07:31.370 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T22:07:31.370 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T22:07:31.370 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T22:07:31.370 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_items": 0, 2024-09-11T22:07:31.370 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_txc_bytes": 0, 2024-09-11T22:07:31.371 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_txc_items": 0, 2024-09-11T22:07:31.371 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T22:07:31.371 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T22:07:31.371 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_bytes": 0, 2024-09-11T22:07:31.371 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_items": 0, 2024-09-11T22:07:31.371 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_bytes": 0, 2024-09-11T22:07:31.371 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_items": 0, 2024-09-11T22:07:31.372 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T22:07:31.372 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_reader_items": 0, 2024-09-11T22:07:31.372 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T22:07:31.372 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_writer_items": 0, 2024-09-11T22:07:31.372 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_anon_bytes": 148994, 2024-09-11T22:07:31.372 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_anon_items": 31, 2024-09-11T22:07:31.372 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_meta_bytes": 0, 2024-09-11T22:07:31.372 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_meta_items": 0, 2024-09-11T22:07:31.373 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_bytes": 0, 2024-09-11T22:07:31.373 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_items": 0, 2024-09-11T22:07:31.373 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mapbl_bytes": 0, 2024-09-11T22:07:31.373 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mapbl_items": 0, 2024-09-11T22:07:31.373 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pglog_bytes": 0, 2024-09-11T22:07:31.373 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pglog_items": 0, 2024-09-11T22:07:31.373 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_bytes": 0, 2024-09-11T22:07:31.374 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_items": 0, 2024-09-11T22:07:31.374 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T22:07:31.374 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_mapping_items": 0, 2024-09-11T22:07:31.374 INFO:teuthology.orchestra.run.smithi001.stdout: "pgmap_bytes": 0, 2024-09-11T22:07:31.374 INFO:teuthology.orchestra.run.smithi001.stdout: "pgmap_items": 0, 2024-09-11T22:07:31.374 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_co_bytes": 928, 2024-09-11T22:07:31.375 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_co_items": 2, 2024-09-11T22:07:31.375 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_1_bytes": 0, 2024-09-11T22:07:31.375 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_1_items": 0, 2024-09-11T22:07:31.375 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_2_bytes": 0, 2024-09-11T22:07:31.375 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_2_items": 0 2024-09-11T22:07:31.375 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:31.375 INFO:teuthology.orchestra.run.smithi001.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T22:07:31.375 INFO:teuthology.orchestra.run.smithi001.stdout: "val": 0, 2024-09-11T22:07:31.376 INFO:teuthology.orchestra.run.smithi001.stdout: "max": 104857600, 2024-09-11T22:07:31.376 INFO:teuthology.orchestra.run.smithi001.stdout: "get_started": 0, 2024-09-11T22:07:31.376 INFO:teuthology.orchestra.run.smithi001.stdout: "get": 218, 2024-09-11T22:07:31.376 INFO:teuthology.orchestra.run.smithi001.stdout: "get_sum": 1548710, 2024-09-11T22:07:31.376 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_fail": 0, 2024-09-11T22:07:31.376 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_success": 218, 2024-09-11T22:07:31.376 INFO:teuthology.orchestra.run.smithi001.stdout: "take": 0, 2024-09-11T22:07:31.377 INFO:teuthology.orchestra.run.smithi001.stdout: "take_sum": 0, 2024-09-11T22:07:31.377 INFO:teuthology.orchestra.run.smithi001.stdout: "put": 218, 2024-09-11T22:07:31.377 INFO:teuthology.orchestra.run.smithi001.stdout: "put_sum": 1548710, 2024-09-11T22:07:31.377 INFO:teuthology.orchestra.run.smithi001.stdout: "wait": { 2024-09-11T22:07:31.377 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:31.377 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:31.377 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:31.377 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:31.378 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:31.378 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-11T22:07:32.062 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-11T22:07:32.063 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-11T22:07:32.063 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-09-11T22:07:32.063 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-09-11T22:07:32.063 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-09-11T22:07:32.063 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-09-11T22:07:32.063 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-09-11T22:07:32.063 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-09-11T22:07:32.064 DEBUG:tasks.check_counter:Getting stats from g 2024-09-11T22:07:32.064 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-11T22:07:32.327 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:07:32.857 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-11T22:07:32.858 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T22:07:32.858 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 901, 2024-09-11T22:07:32.858 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 901, 2024-09-11T22:07:32.858 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 42235013, 2024-09-11T22:07:32.858 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 285695, 2024-09-11T22:07:32.859 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 8, 2024-09-11T22:07:32.859 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 4, 2024-09-11T22:07:32.859 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.115910549, 2024-09-11T22:07:32.859 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.037003691, 2024-09-11T22:07:32.859 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.046458703, 2024-09-11T22:07:32.859 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.023215178, 2024-09-11T22:07:32.859 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:07:32.859 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 900, 2024-09-11T22:07:32.860 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.042678936, 2024-09-11T22:07:32.860 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000047421 2024-09-11T22:07:32.860 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:32.860 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:07:32.860 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:32.860 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:32.860 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:32.860 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:32.861 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-11T22:07:32.861 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 448 2024-09-11T22:07:32.861 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:32.861 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T22:07:32.861 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 1255, 2024-09-11T22:07:32.861 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 1407, 2024-09-11T22:07:32.861 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 47775367, 2024-09-11T22:07:32.861 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 832495, 2024-09-11T22:07:32.862 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 8, 2024-09-11T22:07:32.862 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 6, 2024-09-11T22:07:32.862 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.125561426, 2024-09-11T22:07:32.862 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.034177038, 2024-09-11T22:07:32.862 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.054417688, 2024-09-11T22:07:32.862 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.028467075, 2024-09-11T22:07:32.862 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:07:32.863 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 1404, 2024-09-11T22:07:32.863 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.043992987, 2024-09-11T22:07:32.863 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000031334 2024-09-11T22:07:32.863 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:32.863 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:07:32.863 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:32.863 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:32.863 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:32.864 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:32.864 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-11T22:07:32.864 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 433984 2024-09-11T22:07:32.864 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:32.864 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T22:07:32.864 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 384, 2024-09-11T22:07:32.864 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 351, 2024-09-11T22:07:32.864 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 35338902, 2024-09-11T22:07:32.865 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 168799, 2024-09-11T22:07:32.865 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 5, 2024-09-11T22:07:32.865 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 4, 2024-09-11T22:07:32.865 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.072314137, 2024-09-11T22:07:32.865 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.017735795, 2024-09-11T22:07:32.865 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.063613061, 2024-09-11T22:07:32.866 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.003251262, 2024-09-11T22:07:32.866 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:07:32.866 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 350, 2024-09-11T22:07:32.866 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.015372141, 2024-09-11T22:07:32.866 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000043920 2024-09-11T22:07:32.866 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:32.866 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:07:32.867 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:32.867 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:32.867 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:32.867 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:32.867 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 1749456, 2024-09-11T22:07:32.867 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 126176 2024-09-11T22:07:32.867 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:32.868 INFO:teuthology.orchestra.run.smithi001.stdout: "cct": { 2024-09-11T22:07:32.868 INFO:teuthology.orchestra.run.smithi001.stdout: "total_workers": 1, 2024-09-11T22:07:32.868 INFO:teuthology.orchestra.run.smithi001.stdout: "unhealthy_workers": 0 2024-09-11T22:07:32.868 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:32.868 INFO:teuthology.orchestra.run.smithi001.stdout: "finisher-MDSRank": { 2024-09-11T22:07:32.868 INFO:teuthology.orchestra.run.smithi001.stdout: "queue_len": 0, 2024-09-11T22:07:32.868 INFO:teuthology.orchestra.run.smithi001.stdout: "complete_latency": { 2024-09-11T22:07:32.868 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 3802, 2024-09-11T22:07:32.869 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.091620091, 2024-09-11T22:07:32.869 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000024097 2024-09-11T22:07:32.869 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:32.869 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:32.869 INFO:teuthology.orchestra.run.smithi001.stdout: "finisher-PurgeQueue": { 2024-09-11T22:07:32.869 INFO:teuthology.orchestra.run.smithi001.stdout: "queue_len": 0, 2024-09-11T22:07:32.869 INFO:teuthology.orchestra.run.smithi001.stdout: "complete_latency": { 2024-09-11T22:07:32.870 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:32.870 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:32.870 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:32.870 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:32.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:32.870 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": { 2024-09-11T22:07:32.870 INFO:teuthology.orchestra.run.smithi001.stdout: "request": 0, 2024-09-11T22:07:32.871 INFO:teuthology.orchestra.run.smithi001.stdout: "reply": 0, 2024-09-11T22:07:32.871 INFO:teuthology.orchestra.run.smithi001.stdout: "reply_latency": { 2024-09-11T22:07:32.871 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:32.871 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:32.871 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:32.871 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:32.871 INFO:teuthology.orchestra.run.smithi001.stdout: "slow_reply": 0, 2024-09-11T22:07:32.871 INFO:teuthology.orchestra.run.smithi001.stdout: "forward": 0, 2024-09-11T22:07:32.872 INFO:teuthology.orchestra.run.smithi001.stdout: "dir_fetch_complete": 0, 2024-09-11T22:07:32.872 INFO:teuthology.orchestra.run.smithi001.stdout: "dir_fetch_keys": 0, 2024-09-11T22:07:32.872 INFO:teuthology.orchestra.run.smithi001.stdout: "dir_commit": 0, 2024-09-11T22:07:32.872 INFO:teuthology.orchestra.run.smithi001.stdout: "dir_split": 0, 2024-09-11T22:07:32.872 INFO:teuthology.orchestra.run.smithi001.stdout: "dir_merge": 0, 2024-09-11T22:07:32.872 INFO:teuthology.orchestra.run.smithi001.stdout: "inodes": 38998, 2024-09-11T22:07:32.872 INFO:teuthology.orchestra.run.smithi001.stdout: "inodes_top": 30, 2024-09-11T22:07:32.873 INFO:teuthology.orchestra.run.smithi001.stdout: "inodes_bottom": 38968, 2024-09-11T22:07:32.873 INFO:teuthology.orchestra.run.smithi001.stdout: "inodes_pin_tail": 0, 2024-09-11T22:07:32.873 INFO:teuthology.orchestra.run.smithi001.stdout: "inodes_pinned": 38954, 2024-09-11T22:07:32.873 INFO:teuthology.orchestra.run.smithi001.stdout: "inodes_expired": 0, 2024-09-11T22:07:32.873 INFO:teuthology.orchestra.run.smithi001.stdout: "inodes_with_caps": 0, 2024-09-11T22:07:32.873 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": 0, 2024-09-11T22:07:32.873 INFO:teuthology.orchestra.run.smithi001.stdout: "subtrees": 1, 2024-09-11T22:07:32.874 INFO:teuthology.orchestra.run.smithi001.stdout: "traverse": 0, 2024-09-11T22:07:32.874 INFO:teuthology.orchestra.run.smithi001.stdout: "traverse_hit": 0, 2024-09-11T22:07:32.874 INFO:teuthology.orchestra.run.smithi001.stdout: "traverse_forward": 0, 2024-09-11T22:07:32.874 INFO:teuthology.orchestra.run.smithi001.stdout: "traverse_discover": 0, 2024-09-11T22:07:32.874 INFO:teuthology.orchestra.run.smithi001.stdout: "traverse_dir_fetch": 0, 2024-09-11T22:07:32.874 INFO:teuthology.orchestra.run.smithi001.stdout: "traverse_remote_ino": 0, 2024-09-11T22:07:32.874 INFO:teuthology.orchestra.run.smithi001.stdout: "traverse_lock": 0, 2024-09-11T22:07:32.874 INFO:teuthology.orchestra.run.smithi001.stdout: "load_cent": 0, 2024-09-11T22:07:32.875 INFO:teuthology.orchestra.run.smithi001.stdout: "q": 0, 2024-09-11T22:07:32.875 INFO:teuthology.orchestra.run.smithi001.stdout: "exported": 0, 2024-09-11T22:07:32.875 INFO:teuthology.orchestra.run.smithi001.stdout: "exported_inodes": 0, 2024-09-11T22:07:32.875 INFO:teuthology.orchestra.run.smithi001.stdout: "imported": 0, 2024-09-11T22:07:32.875 INFO:teuthology.orchestra.run.smithi001.stdout: "imported_inodes": 0, 2024-09-11T22:07:32.875 INFO:teuthology.orchestra.run.smithi001.stdout: "openino_dir_fetch": 0, 2024-09-11T22:07:32.875 INFO:teuthology.orchestra.run.smithi001.stdout: "openino_backtrace_fetch": 0, 2024-09-11T22:07:32.875 INFO:teuthology.orchestra.run.smithi001.stdout: "openino_peer_discover": 0, 2024-09-11T22:07:32.876 INFO:teuthology.orchestra.run.smithi001.stdout: "root_rfiles": 0, 2024-09-11T22:07:32.876 INFO:teuthology.orchestra.run.smithi001.stdout: "root_rbytes": 0, 2024-09-11T22:07:32.876 INFO:teuthology.orchestra.run.smithi001.stdout: "root_rsnaps": 0, 2024-09-11T22:07:32.876 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_backtrace_fetch": 0, 2024-09-11T22:07:32.876 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_set_tag": 0, 2024-09-11T22:07:32.876 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T22:07:32.876 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_inotable_repaired": 0, 2024-09-11T22:07:32.877 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_dir_inodes": 0, 2024-09-11T22:07:32.877 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T22:07:32.877 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T22:07:32.877 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_file_inodes": 0, 2024-09-11T22:07:32.877 INFO:teuthology.orchestra.run.smithi001.stdout: "handle_inode_file_caps": 0, 2024-09-11T22:07:32.877 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_cap_op_revoke": 0, 2024-09-11T22:07:32.877 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_cap_op_grant": 0, 2024-09-11T22:07:32.877 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T22:07:32.878 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T22:07:32.878 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T22:07:32.878 INFO:teuthology.orchestra.run.smithi001.stdout: "handle_client_caps": 0, 2024-09-11T22:07:32.878 INFO:teuthology.orchestra.run.smithi001.stdout: "handle_client_caps_dirty": 0, 2024-09-11T22:07:32.878 INFO:teuthology.orchestra.run.smithi001.stdout: "handle_client_cap_release": 0, 2024-09-11T22:07:32.878 INFO:teuthology.orchestra.run.smithi001.stdout: "process_request_cap_release": 0 2024-09-11T22:07:32.878 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:32.879 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_cache": { 2024-09-11T22:07:32.879 INFO:teuthology.orchestra.run.smithi001.stdout: "dir_update": 0, 2024-09-11T22:07:32.879 INFO:teuthology.orchestra.run.smithi001.stdout: "dir_update_receipt": 0, 2024-09-11T22:07:32.879 INFO:teuthology.orchestra.run.smithi001.stdout: "dir_try_discover": 0, 2024-09-11T22:07:32.879 INFO:teuthology.orchestra.run.smithi001.stdout: "dir_send_discover": 0, 2024-09-11T22:07:32.879 INFO:teuthology.orchestra.run.smithi001.stdout: "dir_handle_discover": 0, 2024-09-11T22:07:32.879 INFO:teuthology.orchestra.run.smithi001.stdout: "num_strays": 0, 2024-09-11T22:07:32.879 INFO:teuthology.orchestra.run.smithi001.stdout: "num_strays_delayed": 0, 2024-09-11T22:07:32.880 INFO:teuthology.orchestra.run.smithi001.stdout: "num_strays_enqueuing": 0, 2024-09-11T22:07:32.880 INFO:teuthology.orchestra.run.smithi001.stdout: "strays_created": 0, 2024-09-11T22:07:32.880 INFO:teuthology.orchestra.run.smithi001.stdout: "strays_enqueued": 0, 2024-09-11T22:07:32.880 INFO:teuthology.orchestra.run.smithi001.stdout: "strays_reintegrated": 0, 2024-09-11T22:07:32.880 INFO:teuthology.orchestra.run.smithi001.stdout: "strays_migrated": 0, 2024-09-11T22:07:32.880 INFO:teuthology.orchestra.run.smithi001.stdout: "num_recovering_processing": 0, 2024-09-11T22:07:32.880 INFO:teuthology.orchestra.run.smithi001.stdout: "num_recovering_enqueued": 0, 2024-09-11T22:07:32.881 INFO:teuthology.orchestra.run.smithi001.stdout: "num_recovering_prioritized": 0, 2024-09-11T22:07:32.881 INFO:teuthology.orchestra.run.smithi001.stdout: "recovery_started": 0, 2024-09-11T22:07:32.881 INFO:teuthology.orchestra.run.smithi001.stdout: "recovery_completed": 0, 2024-09-11T22:07:32.881 INFO:teuthology.orchestra.run.smithi001.stdout: "ireq_quiesce_path": 0, 2024-09-11T22:07:32.881 INFO:teuthology.orchestra.run.smithi001.stdout: "ireq_quiesce_inode": 0, 2024-09-11T22:07:32.881 INFO:teuthology.orchestra.run.smithi001.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T22:07:32.881 INFO:teuthology.orchestra.run.smithi001.stdout: "ireq_exportdir": 0, 2024-09-11T22:07:32.881 INFO:teuthology.orchestra.run.smithi001.stdout: "ireq_flush": 0, 2024-09-11T22:07:32.882 INFO:teuthology.orchestra.run.smithi001.stdout: "ireq_fragmentdir": 0, 2024-09-11T22:07:32.882 INFO:teuthology.orchestra.run.smithi001.stdout: "ireq_fragstats": 0, 2024-09-11T22:07:32.882 INFO:teuthology.orchestra.run.smithi001.stdout: "ireq_inodestats": 0 2024-09-11T22:07:32.882 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:32.882 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_log": { 2024-09-11T22:07:32.882 INFO:teuthology.orchestra.run.smithi001.stdout: "evlrg": 0, 2024-09-11T22:07:32.882 INFO:teuthology.orchestra.run.smithi001.stdout: "evadd": 0, 2024-09-11T22:07:32.883 INFO:teuthology.orchestra.run.smithi001.stdout: "evex": 0, 2024-09-11T22:07:32.883 INFO:teuthology.orchestra.run.smithi001.stdout: "evtrm": 0, 2024-09-11T22:07:32.883 INFO:teuthology.orchestra.run.smithi001.stdout: "ev": 62942, 2024-09-11T22:07:32.883 INFO:teuthology.orchestra.run.smithi001.stdout: "evexg": 0, 2024-09-11T22:07:32.883 INFO:teuthology.orchestra.run.smithi001.stdout: "evexd": 0, 2024-09-11T22:07:32.883 INFO:teuthology.orchestra.run.smithi001.stdout: "segadd": 0, 2024-09-11T22:07:32.883 INFO:teuthology.orchestra.run.smithi001.stdout: "segex": 0, 2024-09-11T22:07:32.884 INFO:teuthology.orchestra.run.smithi001.stdout: "segtrm": 0, 2024-09-11T22:07:32.884 INFO:teuthology.orchestra.run.smithi001.stdout: "seg": 79, 2024-09-11T22:07:32.884 INFO:teuthology.orchestra.run.smithi001.stdout: "segmjr": 7, 2024-09-11T22:07:32.884 INFO:teuthology.orchestra.run.smithi001.stdout: "segexg": 0, 2024-09-11T22:07:32.884 INFO:teuthology.orchestra.run.smithi001.stdout: "segexd": 0, 2024-09-11T22:07:32.884 INFO:teuthology.orchestra.run.smithi001.stdout: "expos": 4194304, 2024-09-11T22:07:32.884 INFO:teuthology.orchestra.run.smithi001.stdout: "wrpos": 127143670, 2024-09-11T22:07:32.885 INFO:teuthology.orchestra.run.smithi001.stdout: "rdpos": 127141109, 2024-09-11T22:07:32.885 INFO:teuthology.orchestra.run.smithi001.stdout: "jlat": { 2024-09-11T22:07:32.885 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:32.885 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:32.885 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:32.885 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:32.885 INFO:teuthology.orchestra.run.smithi001.stdout: "replayed": 62942 2024-09-11T22:07:32.885 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:32.886 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_mem": { 2024-09-11T22:07:32.886 INFO:teuthology.orchestra.run.smithi001.stdout: "ino": 16623, 2024-09-11T22:07:32.886 INFO:teuthology.orchestra.run.smithi001.stdout: "ino+": 47707, 2024-09-11T22:07:32.886 INFO:teuthology.orchestra.run.smithi001.stdout: "ino-": 31084, 2024-09-11T22:07:32.886 INFO:teuthology.orchestra.run.smithi001.stdout: "dir": 523, 2024-09-11T22:07:32.886 INFO:teuthology.orchestra.run.smithi001.stdout: "dir+": 1242, 2024-09-11T22:07:32.886 INFO:teuthology.orchestra.run.smithi001.stdout: "dir-": 719, 2024-09-11T22:07:32.887 INFO:teuthology.orchestra.run.smithi001.stdout: "dn": 38998, 2024-09-11T22:07:32.887 INFO:teuthology.orchestra.run.smithi001.stdout: "dn+": 70518, 2024-09-11T22:07:32.887 INFO:teuthology.orchestra.run.smithi001.stdout: "dn-": 31520, 2024-09-11T22:07:32.887 INFO:teuthology.orchestra.run.smithi001.stdout: "cap": 0, 2024-09-11T22:07:32.887 INFO:teuthology.orchestra.run.smithi001.stdout: "cap+": 0, 2024-09-11T22:07:32.887 INFO:teuthology.orchestra.run.smithi001.stdout: "cap-": 0, 2024-09-11T22:07:32.887 INFO:teuthology.orchestra.run.smithi001.stdout: "rss": 136396, 2024-09-11T22:07:32.887 INFO:teuthology.orchestra.run.smithi001.stdout: "heap": 215328 2024-09-11T22:07:32.888 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:32.888 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_server": { 2024-09-11T22:07:32.888 INFO:teuthology.orchestra.run.smithi001.stdout: "dispatch_client_request": 0, 2024-09-11T22:07:32.888 INFO:teuthology.orchestra.run.smithi001.stdout: "dispatch_server_request": 0, 2024-09-11T22:07:32.888 INFO:teuthology.orchestra.run.smithi001.stdout: "handle_client_request": 0, 2024-09-11T22:07:32.888 INFO:teuthology.orchestra.run.smithi001.stdout: "handle_client_session": 0, 2024-09-11T22:07:32.888 INFO:teuthology.orchestra.run.smithi001.stdout: "handle_peer_request": 0, 2024-09-11T22:07:32.889 INFO:teuthology.orchestra.run.smithi001.stdout: "req_create_latency": { 2024-09-11T22:07:32.889 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:32.889 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:32.889 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:32.889 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:32.889 INFO:teuthology.orchestra.run.smithi001.stdout: "req_getattr_latency": { 2024-09-11T22:07:32.889 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:32.889 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:32.890 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:32.890 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:32.890 INFO:teuthology.orchestra.run.smithi001.stdout: "req_getfilelock_latency": { 2024-09-11T22:07:32.890 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:32.890 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:32.890 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:32.890 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:32.891 INFO:teuthology.orchestra.run.smithi001.stdout: "req_link_latency": { 2024-09-11T22:07:32.891 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:32.891 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:32.891 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:32.891 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:32.891 INFO:teuthology.orchestra.run.smithi001.stdout: "req_lookup_latency": { 2024-09-11T22:07:32.891 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:32.891 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:32.892 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:32.892 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:32.892 INFO:teuthology.orchestra.run.smithi001.stdout: "req_lookuphash_latency": { 2024-09-11T22:07:32.892 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:32.892 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:32.892 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:32.892 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:32.892 INFO:teuthology.orchestra.run.smithi001.stdout: "req_lookupino_latency": { 2024-09-11T22:07:32.893 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:32.893 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:32.893 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:32.893 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:32.893 INFO:teuthology.orchestra.run.smithi001.stdout: "req_lookupname_latency": { 2024-09-11T22:07:32.893 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:32.893 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:32.894 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:32.894 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:32.894 INFO:teuthology.orchestra.run.smithi001.stdout: "req_lookupparent_latency": { 2024-09-11T22:07:32.894 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:32.894 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:32.894 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:32.894 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:32.895 INFO:teuthology.orchestra.run.smithi001.stdout: "req_lookupsnap_latency": { 2024-09-11T22:07:32.895 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:32.895 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:32.895 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:32.895 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:32.895 INFO:teuthology.orchestra.run.smithi001.stdout: "req_lssnap_latency": { 2024-09-11T22:07:32.895 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:32.896 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:32.896 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:32.896 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:32.896 INFO:teuthology.orchestra.run.smithi001.stdout: "req_mkdir_latency": { 2024-09-11T22:07:32.896 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:32.896 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:32.897 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:32.897 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:32.897 INFO:teuthology.orchestra.run.smithi001.stdout: "req_mknod_latency": { 2024-09-11T22:07:32.897 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:32.897 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:32.897 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:32.897 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:32.897 INFO:teuthology.orchestra.run.smithi001.stdout: "req_mksnap_latency": { 2024-09-11T22:07:32.898 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:32.898 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:32.898 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:32.898 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:32.898 INFO:teuthology.orchestra.run.smithi001.stdout: "req_open_latency": { 2024-09-11T22:07:32.898 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:32.898 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:32.899 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:32.899 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:32.899 INFO:teuthology.orchestra.run.smithi001.stdout: "req_readdir_latency": { 2024-09-11T22:07:32.899 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:32.899 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:32.899 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:32.899 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:32.900 INFO:teuthology.orchestra.run.smithi001.stdout: "req_rename_latency": { 2024-09-11T22:07:32.900 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:32.900 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:32.900 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:32.900 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:32.900 INFO:teuthology.orchestra.run.smithi001.stdout: "req_renamesnap_latency": { 2024-09-11T22:07:32.900 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:32.900 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:32.901 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:32.901 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:32.901 INFO:teuthology.orchestra.run.smithi001.stdout: "req_snapdiff_latency": { 2024-09-11T22:07:32.901 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:32.901 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:32.901 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:32.901 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:32.902 INFO:teuthology.orchestra.run.smithi001.stdout: "req_rmdir_latency": { 2024-09-11T22:07:32.902 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:32.902 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:32.902 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:32.902 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:32.902 INFO:teuthology.orchestra.run.smithi001.stdout: "req_rmsnap_latency": { 2024-09-11T22:07:32.903 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:32.903 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:32.903 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:32.903 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:32.903 INFO:teuthology.orchestra.run.smithi001.stdout: "req_rmxattr_latency": { 2024-09-11T22:07:32.903 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:32.903 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:32.904 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:32.904 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:32.904 INFO:teuthology.orchestra.run.smithi001.stdout: "req_setattr_latency": { 2024-09-11T22:07:32.904 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:32.904 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:32.904 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:32.904 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:32.904 INFO:teuthology.orchestra.run.smithi001.stdout: "req_setdirlayout_latency": { 2024-09-11T22:07:32.905 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:32.905 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:32.905 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:32.905 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:32.905 INFO:teuthology.orchestra.run.smithi001.stdout: "req_setfilelock_latency": { 2024-09-11T22:07:32.905 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:32.905 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:32.906 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:32.906 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:32.906 INFO:teuthology.orchestra.run.smithi001.stdout: "req_setlayout_latency": { 2024-09-11T22:07:32.906 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:32.906 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:32.906 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:32.906 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:32.906 INFO:teuthology.orchestra.run.smithi001.stdout: "req_setxattr_latency": { 2024-09-11T22:07:32.907 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:32.907 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:32.907 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:32.907 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:32.907 INFO:teuthology.orchestra.run.smithi001.stdout: "req_symlink_latency": { 2024-09-11T22:07:32.907 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:32.907 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:32.908 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:32.908 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:32.908 INFO:teuthology.orchestra.run.smithi001.stdout: "req_unlink_latency": { 2024-09-11T22:07:32.908 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:32.908 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:32.908 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:32.908 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:32.909 INFO:teuthology.orchestra.run.smithi001.stdout: "cap_revoke_eviction": 0, 2024-09-11T22:07:32.909 INFO:teuthology.orchestra.run.smithi001.stdout: "cap_acquisition_throttle": 0, 2024-09-11T22:07:32.909 INFO:teuthology.orchestra.run.smithi001.stdout: "req_getvxattr_latency": { 2024-09-11T22:07:32.909 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:32.909 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:32.909 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:32.909 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:32.909 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:32.910 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_sessions": { 2024-09-11T22:07:32.910 INFO:teuthology.orchestra.run.smithi001.stdout: "session_count": 1, 2024-09-11T22:07:32.910 INFO:teuthology.orchestra.run.smithi001.stdout: "session_add": 4, 2024-09-11T22:07:32.910 INFO:teuthology.orchestra.run.smithi001.stdout: "session_remove": 3, 2024-09-11T22:07:32.910 INFO:teuthology.orchestra.run.smithi001.stdout: "sessions_open": 3, 2024-09-11T22:07:32.910 INFO:teuthology.orchestra.run.smithi001.stdout: "sessions_stale": 0, 2024-09-11T22:07:32.910 INFO:teuthology.orchestra.run.smithi001.stdout: "total_load": 0, 2024-09-11T22:07:32.910 INFO:teuthology.orchestra.run.smithi001.stdout: "average_load": 0, 2024-09-11T22:07:32.911 INFO:teuthology.orchestra.run.smithi001.stdout: "avg_session_uptime": 3767, 2024-09-11T22:07:32.911 INFO:teuthology.orchestra.run.smithi001.stdout: "mdthresh_evicted": 0 2024-09-11T22:07:32.911 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:32.911 INFO:teuthology.orchestra.run.smithi001.stdout: "mempool": { 2024-09-11T22:07:32.911 INFO:teuthology.orchestra.run.smithi001.stdout: "bloom_filter_bytes": 0, 2024-09-11T22:07:32.911 INFO:teuthology.orchestra.run.smithi001.stdout: "bloom_filter_items": 0, 2024-09-11T22:07:32.911 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T22:07:32.911 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_alloc_items": 0, 2024-09-11T22:07:32.912 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T22:07:32.912 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_data_items": 0, 2024-09-11T22:07:32.912 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T22:07:32.912 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T22:07:32.912 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T22:07:32.912 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T22:07:32.912 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T22:07:32.912 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_other_items": 0, 2024-09-11T22:07:32.913 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T22:07:32.913 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T22:07:32.913 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_extent_bytes": 0, 2024-09-11T22:07:32.913 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_extent_items": 0, 2024-09-11T22:07:32.913 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_blob_bytes": 0, 2024-09-11T22:07:32.913 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_blob_items": 0, 2024-09-11T22:07:32.913 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T22:07:32.913 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T22:07:32.914 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T22:07:32.914 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T22:07:32.914 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T22:07:32.914 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_items": 0, 2024-09-11T22:07:32.914 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_txc_bytes": 0, 2024-09-11T22:07:32.915 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_txc_items": 0, 2024-09-11T22:07:32.915 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T22:07:32.915 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T22:07:32.915 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_bytes": 0, 2024-09-11T22:07:32.915 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_items": 0, 2024-09-11T22:07:32.915 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_bytes": 0, 2024-09-11T22:07:32.915 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_items": 0, 2024-09-11T22:07:32.916 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T22:07:32.916 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_reader_items": 0, 2024-09-11T22:07:32.916 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T22:07:32.916 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_writer_items": 0, 2024-09-11T22:07:32.916 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_anon_bytes": 845258, 2024-09-11T22:07:32.916 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_anon_items": 16686, 2024-09-11T22:07:32.916 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_meta_bytes": 0, 2024-09-11T22:07:32.917 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_meta_items": 0, 2024-09-11T22:07:32.917 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_bytes": 0, 2024-09-11T22:07:32.917 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_items": 0, 2024-09-11T22:07:32.917 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mapbl_bytes": 0, 2024-09-11T22:07:32.917 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mapbl_items": 0, 2024-09-11T22:07:32.917 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pglog_bytes": 0, 2024-09-11T22:07:32.917 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pglog_items": 0, 2024-09-11T22:07:32.918 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_bytes": 8392, 2024-09-11T22:07:32.918 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_items": 184, 2024-09-11T22:07:32.918 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T22:07:32.918 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_mapping_items": 0, 2024-09-11T22:07:32.918 INFO:teuthology.orchestra.run.smithi001.stdout: "pgmap_bytes": 0, 2024-09-11T22:07:32.918 INFO:teuthology.orchestra.run.smithi001.stdout: "pgmap_items": 0, 2024-09-11T22:07:32.918 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_co_bytes": 65938753, 2024-09-11T22:07:32.918 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_co_items": 514971, 2024-09-11T22:07:32.919 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_1_bytes": 0, 2024-09-11T22:07:32.919 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_1_items": 0, 2024-09-11T22:07:32.919 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_2_bytes": 0, 2024-09-11T22:07:32.919 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_2_items": 0 2024-09-11T22:07:32.920 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:32.920 INFO:teuthology.orchestra.run.smithi001.stdout: "objecter": { 2024-09-11T22:07:32.920 INFO:teuthology.orchestra.run.smithi001.stdout: "op_active": 0, 2024-09-11T22:07:32.920 INFO:teuthology.orchestra.run.smithi001.stdout: "op_laggy": 0, 2024-09-11T22:07:32.920 INFO:teuthology.orchestra.run.smithi001.stdout: "op_send": 2284, 2024-09-11T22:07:32.920 INFO:teuthology.orchestra.run.smithi001.stdout: "op_send_bytes": 21, 2024-09-11T22:07:32.921 INFO:teuthology.orchestra.run.smithi001.stdout: "op_resend": 3, 2024-09-11T22:07:32.921 INFO:teuthology.orchestra.run.smithi001.stdout: "op_reply": 2284, 2024-09-11T22:07:32.921 INFO:teuthology.orchestra.run.smithi001.stdout: "op_latency": { 2024-09-11T22:07:32.921 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 2284, 2024-09-11T22:07:32.921 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 7.411906831, 2024-09-11T22:07:32.921 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.003245143 2024-09-11T22:07:32.921 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:32.922 INFO:teuthology.orchestra.run.smithi001.stdout: "op_inflight": 0, 2024-09-11T22:07:32.922 INFO:teuthology.orchestra.run.smithi001.stdout: "oplen_avg": { 2024-09-11T22:07:32.922 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 2284, 2024-09-11T22:07:32.922 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 2285 2024-09-11T22:07:32.922 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:32.922 INFO:teuthology.orchestra.run.smithi001.stdout: "op": 2284, 2024-09-11T22:07:32.922 INFO:teuthology.orchestra.run.smithi001.stdout: "op_r": 2284, 2024-09-11T22:07:32.923 INFO:teuthology.orchestra.run.smithi001.stdout: "op_w": 0, 2024-09-11T22:07:32.923 INFO:teuthology.orchestra.run.smithi001.stdout: "op_rmw": 0, 2024-09-11T22:07:32.923 INFO:teuthology.orchestra.run.smithi001.stdout: "op_pg": 0, 2024-09-11T22:07:32.923 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_stat": 1412, 2024-09-11T22:07:32.923 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_create": 0, 2024-09-11T22:07:32.923 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_read": 870, 2024-09-11T22:07:32.923 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_write": 0, 2024-09-11T22:07:32.923 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_writefull": 0, 2024-09-11T22:07:32.924 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_writesame": 0, 2024-09-11T22:07:32.924 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_append": 0, 2024-09-11T22:07:32.924 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_zero": 0, 2024-09-11T22:07:32.924 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_truncate": 0, 2024-09-11T22:07:32.924 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_delete": 0, 2024-09-11T22:07:32.924 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_mapext": 0, 2024-09-11T22:07:32.924 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_sparse_read": 0, 2024-09-11T22:07:32.924 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_clonerange": 0, 2024-09-11T22:07:32.925 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_getxattr": 1, 2024-09-11T22:07:32.925 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_setxattr": 0, 2024-09-11T22:07:32.925 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_cmpxattr": 0, 2024-09-11T22:07:32.925 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_rmxattr": 0, 2024-09-11T22:07:32.925 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_resetxattrs": 0, 2024-09-11T22:07:32.925 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_call": 0, 2024-09-11T22:07:32.925 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_watch": 0, 2024-09-11T22:07:32.926 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_notify": 0, 2024-09-11T22:07:32.926 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T22:07:32.926 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_pgls": 0, 2024-09-11T22:07:32.926 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_pgls_filter": 0, 2024-09-11T22:07:32.926 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_other": 0, 2024-09-11T22:07:32.926 INFO:teuthology.orchestra.run.smithi001.stdout: "linger_active": 0, 2024-09-11T22:07:32.927 INFO:teuthology.orchestra.run.smithi001.stdout: "linger_send": 0, 2024-09-11T22:07:32.927 INFO:teuthology.orchestra.run.smithi001.stdout: "linger_resend": 0, 2024-09-11T22:07:32.927 INFO:teuthology.orchestra.run.smithi001.stdout: "linger_ping": 0, 2024-09-11T22:07:32.927 INFO:teuthology.orchestra.run.smithi001.stdout: "poolop_active": 0, 2024-09-11T22:07:32.927 INFO:teuthology.orchestra.run.smithi001.stdout: "poolop_send": 0, 2024-09-11T22:07:32.927 INFO:teuthology.orchestra.run.smithi001.stdout: "poolop_resend": 0, 2024-09-11T22:07:32.927 INFO:teuthology.orchestra.run.smithi001.stdout: "poolstat_active": 0, 2024-09-11T22:07:32.927 INFO:teuthology.orchestra.run.smithi001.stdout: "poolstat_send": 0, 2024-09-11T22:07:32.928 INFO:teuthology.orchestra.run.smithi001.stdout: "poolstat_resend": 0, 2024-09-11T22:07:32.928 INFO:teuthology.orchestra.run.smithi001.stdout: "statfs_active": 0, 2024-09-11T22:07:32.928 INFO:teuthology.orchestra.run.smithi001.stdout: "statfs_send": 0, 2024-09-11T22:07:32.928 INFO:teuthology.orchestra.run.smithi001.stdout: "statfs_resend": 0, 2024-09-11T22:07:32.928 INFO:teuthology.orchestra.run.smithi001.stdout: "command_active": 0, 2024-09-11T22:07:32.928 INFO:teuthology.orchestra.run.smithi001.stdout: "command_send": 0, 2024-09-11T22:07:32.928 INFO:teuthology.orchestra.run.smithi001.stdout: "command_resend": 0, 2024-09-11T22:07:32.929 INFO:teuthology.orchestra.run.smithi001.stdout: "map_epoch": 95, 2024-09-11T22:07:32.929 INFO:teuthology.orchestra.run.smithi001.stdout: "map_full": 0, 2024-09-11T22:07:32.929 INFO:teuthology.orchestra.run.smithi001.stdout: "map_inc": 11, 2024-09-11T22:07:32.929 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_sessions": 10, 2024-09-11T22:07:32.929 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_session_open": 10, 2024-09-11T22:07:32.929 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_session_close": 0, 2024-09-11T22:07:32.930 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_laggy": 0, 2024-09-11T22:07:32.930 INFO:teuthology.orchestra.run.smithi001.stdout: "omap_wr": 0, 2024-09-11T22:07:32.930 INFO:teuthology.orchestra.run.smithi001.stdout: "omap_rd": 2, 2024-09-11T22:07:32.930 INFO:teuthology.orchestra.run.smithi001.stdout: "omap_del": 0 2024-09-11T22:07:32.930 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:32.930 INFO:teuthology.orchestra.run.smithi001.stdout: "oft": { 2024-09-11T22:07:32.930 INFO:teuthology.orchestra.run.smithi001.stdout: "omap_total_objs": 0, 2024-09-11T22:07:32.931 INFO:teuthology.orchestra.run.smithi001.stdout: "omap_total_kv_pairs": 0, 2024-09-11T22:07:32.931 INFO:teuthology.orchestra.run.smithi001.stdout: "omap_total_updates": 0, 2024-09-11T22:07:32.931 INFO:teuthology.orchestra.run.smithi001.stdout: "omap_total_removes": 0 2024-09-11T22:07:32.931 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:32.931 INFO:teuthology.orchestra.run.smithi001.stdout: "osd-slow-ops": { 2024-09-11T22:07:32.931 INFO:teuthology.orchestra.run.smithi001.stdout: "slow_ops_count": 0 2024-09-11T22:07:32.931 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:32.932 INFO:teuthology.orchestra.run.smithi001.stdout: "purge_queue": { 2024-09-11T22:07:32.932 INFO:teuthology.orchestra.run.smithi001.stdout: "pq_executing_ops": 0, 2024-09-11T22:07:32.932 INFO:teuthology.orchestra.run.smithi001.stdout: "pq_executing_ops_high_water": 0, 2024-09-11T22:07:32.932 INFO:teuthology.orchestra.run.smithi001.stdout: "pq_executing": 0, 2024-09-11T22:07:32.932 INFO:teuthology.orchestra.run.smithi001.stdout: "pq_executing_high_water": 0, 2024-09-11T22:07:32.932 INFO:teuthology.orchestra.run.smithi001.stdout: "pq_executed_ops": 0, 2024-09-11T22:07:32.932 INFO:teuthology.orchestra.run.smithi001.stdout: "pq_executed": 0, 2024-09-11T22:07:32.932 INFO:teuthology.orchestra.run.smithi001.stdout: "pq_item_in_journal": 0 2024-09-11T22:07:32.933 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:32.933 INFO:teuthology.orchestra.run.smithi001.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T22:07:32.933 INFO:teuthology.orchestra.run.smithi001.stdout: "val": 0, 2024-09-11T22:07:32.933 INFO:teuthology.orchestra.run.smithi001.stdout: "max": 104857600, 2024-09-11T22:07:32.933 INFO:teuthology.orchestra.run.smithi001.stdout: "get_started": 0, 2024-09-11T22:07:32.933 INFO:teuthology.orchestra.run.smithi001.stdout: "get": 2536, 2024-09-11T22:07:32.933 INFO:teuthology.orchestra.run.smithi001.stdout: "get_sum": 125105346, 2024-09-11T22:07:32.934 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_fail": 0, 2024-09-11T22:07:32.934 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_success": 2536, 2024-09-11T22:07:32.934 INFO:teuthology.orchestra.run.smithi001.stdout: "take": 0, 2024-09-11T22:07:32.934 INFO:teuthology.orchestra.run.smithi001.stdout: "take_sum": 0, 2024-09-11T22:07:32.934 INFO:teuthology.orchestra.run.smithi001.stdout: "put": 2536, 2024-09-11T22:07:32.934 INFO:teuthology.orchestra.run.smithi001.stdout: "put_sum": 125105346, 2024-09-11T22:07:32.934 INFO:teuthology.orchestra.run.smithi001.stdout: "wait": { 2024-09-11T22:07:32.934 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:32.935 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:32.935 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:32.935 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:32.935 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:32.935 INFO:teuthology.orchestra.run.smithi001.stdout: "throttle-objecter_bytes": { 2024-09-11T22:07:32.935 INFO:teuthology.orchestra.run.smithi001.stdout: "val": 0, 2024-09-11T22:07:32.935 INFO:teuthology.orchestra.run.smithi001.stdout: "max": 104857600, 2024-09-11T22:07:32.935 INFO:teuthology.orchestra.run.smithi001.stdout: "get_started": 0, 2024-09-11T22:07:32.936 INFO:teuthology.orchestra.run.smithi001.stdout: "get": 0, 2024-09-11T22:07:32.936 INFO:teuthology.orchestra.run.smithi001.stdout: "get_sum": 0, 2024-09-11T22:07:32.936 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_fail": 0, 2024-09-11T22:07:32.936 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_success": 0, 2024-09-11T22:07:32.936 INFO:teuthology.orchestra.run.smithi001.stdout: "take": 2284, 2024-09-11T22:07:32.936 INFO:teuthology.orchestra.run.smithi001.stdout: "take_sum": 122953100, 2024-09-11T22:07:32.936 INFO:teuthology.orchestra.run.smithi001.stdout: "put": 2284, 2024-09-11T22:07:32.936 INFO:teuthology.orchestra.run.smithi001.stdout: "put_sum": 122953100, 2024-09-11T22:07:32.937 INFO:teuthology.orchestra.run.smithi001.stdout: "wait": { 2024-09-11T22:07:32.937 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:32.937 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:32.937 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:32.937 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:32.937 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:32.937 INFO:teuthology.orchestra.run.smithi001.stdout: "throttle-objecter_ops": { 2024-09-11T22:07:32.938 INFO:teuthology.orchestra.run.smithi001.stdout: "val": 0, 2024-09-11T22:07:32.938 INFO:teuthology.orchestra.run.smithi001.stdout: "max": 1024, 2024-09-11T22:07:32.938 INFO:teuthology.orchestra.run.smithi001.stdout: "get_started": 0, 2024-09-11T22:07:32.938 INFO:teuthology.orchestra.run.smithi001.stdout: "get": 0, 2024-09-11T22:07:32.938 INFO:teuthology.orchestra.run.smithi001.stdout: "get_sum": 0, 2024-09-11T22:07:32.938 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_fail": 0, 2024-09-11T22:07:32.939 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_success": 0, 2024-09-11T22:07:32.939 INFO:teuthology.orchestra.run.smithi001.stdout: "take": 2284, 2024-09-11T22:07:32.939 INFO:teuthology.orchestra.run.smithi001.stdout: "take_sum": 2284, 2024-09-11T22:07:32.939 INFO:teuthology.orchestra.run.smithi001.stdout: "put": 2284, 2024-09-11T22:07:32.939 INFO:teuthology.orchestra.run.smithi001.stdout: "put_sum": 2284, 2024-09-11T22:07:32.939 INFO:teuthology.orchestra.run.smithi001.stdout: "wait": { 2024-09-11T22:07:32.939 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:32.940 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:32.940 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:32.940 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:32.940 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:32.940 INFO:teuthology.orchestra.run.smithi001.stdout: "throttle-write_buf_throttle": { 2024-09-11T22:07:32.940 INFO:teuthology.orchestra.run.smithi001.stdout: "val": 0, 2024-09-11T22:07:32.941 INFO:teuthology.orchestra.run.smithi001.stdout: "max": 3758096384, 2024-09-11T22:07:32.941 INFO:teuthology.orchestra.run.smithi001.stdout: "get_started": 0, 2024-09-11T22:07:32.941 INFO:teuthology.orchestra.run.smithi001.stdout: "get": 0, 2024-09-11T22:07:32.941 INFO:teuthology.orchestra.run.smithi001.stdout: "get_sum": 0, 2024-09-11T22:07:32.941 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_fail": 0, 2024-09-11T22:07:32.941 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_success": 0, 2024-09-11T22:07:32.941 INFO:teuthology.orchestra.run.smithi001.stdout: "take": 0, 2024-09-11T22:07:32.942 INFO:teuthology.orchestra.run.smithi001.stdout: "take_sum": 0, 2024-09-11T22:07:32.942 INFO:teuthology.orchestra.run.smithi001.stdout: "put": 0, 2024-09-11T22:07:32.942 INFO:teuthology.orchestra.run.smithi001.stdout: "put_sum": 0, 2024-09-11T22:07:32.942 INFO:teuthology.orchestra.run.smithi001.stdout: "wait": { 2024-09-11T22:07:32.942 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:32.942 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:32.942 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:32.943 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:32.943 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:32.943 INFO:teuthology.orchestra.run.smithi001.stdout: "throttle-write_buf_throttle-0x55ba85fe86e0": { 2024-09-11T22:07:32.943 INFO:teuthology.orchestra.run.smithi001.stdout: "val": 0, 2024-09-11T22:07:32.943 INFO:teuthology.orchestra.run.smithi001.stdout: "max": 3758096384, 2024-09-11T22:07:32.943 INFO:teuthology.orchestra.run.smithi001.stdout: "get_started": 0, 2024-09-11T22:07:32.943 INFO:teuthology.orchestra.run.smithi001.stdout: "get": 0, 2024-09-11T22:07:32.944 INFO:teuthology.orchestra.run.smithi001.stdout: "get_sum": 0, 2024-09-11T22:07:32.944 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_fail": 0, 2024-09-11T22:07:32.944 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_success": 0, 2024-09-11T22:07:32.944 INFO:teuthology.orchestra.run.smithi001.stdout: "take": 0, 2024-09-11T22:07:32.944 INFO:teuthology.orchestra.run.smithi001.stdout: "take_sum": 0, 2024-09-11T22:07:32.944 INFO:teuthology.orchestra.run.smithi001.stdout: "put": 0, 2024-09-11T22:07:32.944 INFO:teuthology.orchestra.run.smithi001.stdout: "put_sum": 0, 2024-09-11T22:07:32.945 INFO:teuthology.orchestra.run.smithi001.stdout: "wait": { 2024-09-11T22:07:32.945 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:32.945 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:32.945 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:32.945 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:32.945 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:32.945 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-11T22:07:33.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:33 smithi053 ceph-mon[34908]: pgmap v320: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-11T22:07:33.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:33 smithi053 ceph-mon[34908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:07:33.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:33 smithi161 ceph-mon[35615]: pgmap v320: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-11T22:07:33.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:33 smithi161 ceph-mon[35615]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:07:33.415 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:33 smithi001 ceph-mon[30935]: pgmap v320: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-11T22:07:33.416 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:33 smithi001 ceph-mon[30935]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:07:33.845 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-09-11T22:07:33.846 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-09-11T22:07:33.846 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2024-09-11T22:07:33.846 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=0 2024-09-11T22:07:33.846 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-09-11T22:07:33.846 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-09-11T22:07:33.846 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-09-11T22:07:33.847 INFO:tasks.check_counter:Daemon mds.g mds.dir_split=0 2024-09-11T22:07:33.847 DEBUG:tasks.check_counter:Getting stats from j 2024-09-11T22:07:33.847 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-11T22:07:34.114 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:07:34.629 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-09-11T22:07:34.629 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T22:07:34.629 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 4, 2024-09-11T22:07:34.630 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 4, 2024-09-11T22:07:34.630 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 3024, 2024-09-11T22:07:34.630 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 448, 2024-09-11T22:07:34.630 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 4, 2024-09-11T22:07:34.630 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 0, 2024-09-11T22:07:34.631 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.003493193, 2024-09-11T22:07:34.631 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.000041168, 2024-09-11T22:07:34.631 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.002322133, 2024-09-11T22:07:34.631 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T22:07:34.631 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:07:34.631 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 3, 2024-09-11T22:07:34.631 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.004196702, 2024-09-11T22:07:34.632 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.001398900 2024-09-11T22:07:34.632 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:34.632 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:07:34.632 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:34.632 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:34.632 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:34.633 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:34.633 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-11T22:07:34.633 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 448 2024-09-11T22:07:34.633 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:34.633 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T22:07:34.633 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 218, 2024-09-11T22:07:34.634 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 207, 2024-09-11T22:07:34.634 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 1579680, 2024-09-11T22:07:34.634 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 127248, 2024-09-11T22:07:34.634 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 2, 2024-09-11T22:07:34.634 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 1, 2024-09-11T22:07:34.634 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.041082541, 2024-09-11T22:07:34.634 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.014082238, 2024-09-11T22:07:34.635 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.039826543, 2024-09-11T22:07:34.635 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T22:07:34.635 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:07:34.635 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 206, 2024-09-11T22:07:34.635 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.010699939, 2024-09-11T22:07:34.636 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000051941 2024-09-11T22:07:34.636 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:34.636 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:07:34.636 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:34.636 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:34.636 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:34.636 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:34.637 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 1579680, 2024-09-11T22:07:34.637 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 127248 2024-09-11T22:07:34.637 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:34.637 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T22:07:34.637 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 2, 2024-09-11T22:07:34.637 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 158, 2024-09-11T22:07:34.638 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 320, 2024-09-11T22:07:34.638 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 185040, 2024-09-11T22:07:34.638 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 5, 2024-09-11T22:07:34.638 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 3, 2024-09-11T22:07:34.638 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.013970641, 2024-09-11T22:07:34.639 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.009048531, 2024-09-11T22:07:34.639 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.002484568, 2024-09-11T22:07:34.639 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T22:07:34.639 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:07:34.639 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 155, 2024-09-11T22:07:34.639 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.012481792, 2024-09-11T22:07:34.639 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000080527 2024-09-11T22:07:34.639 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:34.639 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:07:34.640 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:34.640 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:34.640 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:34.640 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:34.640 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-11T22:07:34.640 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 185040 2024-09-11T22:07:34.640 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:34.641 INFO:teuthology.orchestra.run.smithi001.stdout: "cct": { 2024-09-11T22:07:34.641 INFO:teuthology.orchestra.run.smithi001.stdout: "total_workers": 0, 2024-09-11T22:07:34.641 INFO:teuthology.orchestra.run.smithi001.stdout: "unhealthy_workers": 0 2024-09-11T22:07:34.641 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:34.641 INFO:teuthology.orchestra.run.smithi001.stdout: "mempool": { 2024-09-11T22:07:34.641 INFO:teuthology.orchestra.run.smithi001.stdout: "bloom_filter_bytes": 0, 2024-09-11T22:07:34.641 INFO:teuthology.orchestra.run.smithi001.stdout: "bloom_filter_items": 0, 2024-09-11T22:07:34.642 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T22:07:34.642 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_alloc_items": 0, 2024-09-11T22:07:34.642 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T22:07:34.642 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_data_items": 0, 2024-09-11T22:07:34.642 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T22:07:34.642 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T22:07:34.642 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T22:07:34.642 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T22:07:34.643 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T22:07:34.643 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_other_items": 0, 2024-09-11T22:07:34.643 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T22:07:34.643 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T22:07:34.643 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_extent_bytes": 0, 2024-09-11T22:07:34.643 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_extent_items": 0, 2024-09-11T22:07:34.643 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_blob_bytes": 0, 2024-09-11T22:07:34.643 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_blob_items": 0, 2024-09-11T22:07:34.644 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T22:07:34.644 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T22:07:34.644 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T22:07:34.644 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T22:07:34.644 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T22:07:34.644 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_items": 0, 2024-09-11T22:07:34.644 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_txc_bytes": 0, 2024-09-11T22:07:34.645 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_txc_items": 0, 2024-09-11T22:07:34.645 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T22:07:34.645 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T22:07:34.645 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_bytes": 0, 2024-09-11T22:07:34.645 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_items": 0, 2024-09-11T22:07:34.645 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_bytes": 0, 2024-09-11T22:07:34.645 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_items": 0, 2024-09-11T22:07:34.646 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T22:07:34.646 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_reader_items": 0, 2024-09-11T22:07:34.646 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T22:07:34.646 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_writer_items": 0, 2024-09-11T22:07:34.646 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_anon_bytes": 149017, 2024-09-11T22:07:34.646 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_anon_items": 31, 2024-09-11T22:07:34.646 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_meta_bytes": 0, 2024-09-11T22:07:34.646 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_meta_items": 0, 2024-09-11T22:07:34.647 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_bytes": 0, 2024-09-11T22:07:34.647 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_items": 0, 2024-09-11T22:07:34.647 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mapbl_bytes": 0, 2024-09-11T22:07:34.647 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mapbl_items": 0, 2024-09-11T22:07:34.647 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pglog_bytes": 0, 2024-09-11T22:07:34.647 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pglog_items": 0, 2024-09-11T22:07:34.647 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_bytes": 0, 2024-09-11T22:07:34.647 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_items": 0, 2024-09-11T22:07:34.648 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T22:07:34.648 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_mapping_items": 0, 2024-09-11T22:07:34.648 INFO:teuthology.orchestra.run.smithi001.stdout: "pgmap_bytes": 0, 2024-09-11T22:07:34.648 INFO:teuthology.orchestra.run.smithi001.stdout: "pgmap_items": 0, 2024-09-11T22:07:34.648 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_co_bytes": 928, 2024-09-11T22:07:34.648 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_co_items": 2, 2024-09-11T22:07:34.648 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_1_bytes": 0, 2024-09-11T22:07:34.648 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_1_items": 0, 2024-09-11T22:07:34.649 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_2_bytes": 0, 2024-09-11T22:07:34.649 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_2_items": 0 2024-09-11T22:07:34.649 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-09-11T22:07:34.649 INFO:teuthology.orchestra.run.smithi001.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T22:07:34.649 INFO:teuthology.orchestra.run.smithi001.stdout: "val": 0, 2024-09-11T22:07:34.649 INFO:teuthology.orchestra.run.smithi001.stdout: "max": 104857600, 2024-09-11T22:07:34.649 INFO:teuthology.orchestra.run.smithi001.stdout: "get_started": 0, 2024-09-11T22:07:34.650 INFO:teuthology.orchestra.run.smithi001.stdout: "get": 220, 2024-09-11T22:07:34.650 INFO:teuthology.orchestra.run.smithi001.stdout: "get_sum": 1548970, 2024-09-11T22:07:34.650 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_fail": 0, 2024-09-11T22:07:34.650 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_success": 220, 2024-09-11T22:07:34.650 INFO:teuthology.orchestra.run.smithi001.stdout: "take": 0, 2024-09-11T22:07:34.650 INFO:teuthology.orchestra.run.smithi001.stdout: "take_sum": 0, 2024-09-11T22:07:34.650 INFO:teuthology.orchestra.run.smithi001.stdout: "put": 220, 2024-09-11T22:07:34.650 INFO:teuthology.orchestra.run.smithi001.stdout: "put_sum": 1548970, 2024-09-11T22:07:34.651 INFO:teuthology.orchestra.run.smithi001.stdout: "wait": { 2024-09-11T22:07:34.651 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-09-11T22:07:34.651 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-09-11T22:07:34.651 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-09-11T22:07:34.651 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:34.651 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-09-11T22:07:34.651 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-09-11T22:07:34.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:34 smithi001 ceph-mon[30935]: pgmap v321: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-11T22:07:35.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:34 smithi053 ceph-mon[34908]: pgmap v321: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-11T22:07:35.091 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:34 smithi161 ceph-mon[35615]: pgmap v321: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-11T22:07:35.319 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-11T22:07:35.319 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-11T22:07:35.319 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-09-11T22:07:35.319 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-09-11T22:07:35.319 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-09-11T22:07:35.319 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-09-11T22:07:35.320 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-09-11T22:07:35.320 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-09-11T22:07:35.320 DEBUG:tasks.check_counter:Getting stats from b 2024-09-11T22:07:35.320 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-11T22:07:35.586 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T22:07:36.233 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-11T22:07:36.233 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T22:07:36.233 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 23477, 2024-09-11T22:07:36.233 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 21301, 2024-09-11T22:07:36.233 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 17546069, 2024-09-11T22:07:36.234 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 60137839, 2024-09-11T22:07:36.234 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 41, 2024-09-11T22:07:36.234 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 3, 2024-09-11T22:07:36.234 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 1.556883072, 2024-09-11T22:07:36.234 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.569658933, 2024-09-11T22:07:36.234 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.670385043, 2024-09-11T22:07:36.234 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.385011456, 2024-09-11T22:07:36.235 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:07:36.235 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 21302, 2024-09-11T22:07:36.235 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 15.459849475, 2024-09-11T22:07:36.235 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000725746 2024-09-11T22:07:36.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:36.236 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:07:36.236 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 18309, 2024-09-11T22:07:36.236 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.003293407, 2024-09-11T22:07:36.236 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000179 2024-09-11T22:07:36.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:36.237 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 1891872, 2024-09-11T22:07:36.237 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 494992 2024-09-11T22:07:36.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:36.237 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T22:07:36.237 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 127538, 2024-09-11T22:07:36.237 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 205001, 2024-09-11T22:07:36.237 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 53858375, 2024-09-11T22:07:36.238 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 199962069, 2024-09-11T22:07:36.238 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 25, 2024-09-11T22:07:36.238 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 7, 2024-09-11T22:07:36.238 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 8.160982682, 2024-09-11T22:07:36.238 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 3.561235770, 2024-09-11T22:07:36.238 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 155.345967869, 2024-09-11T22:07:36.238 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.806677388, 2024-09-11T22:07:36.239 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:07:36.239 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 205003, 2024-09-11T22:07:36.239 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 32.326274754, 2024-09-11T22:07:36.239 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000157686 2024-09-11T22:07:36.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:36.239 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:07:36.239 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 258519, 2024-09-11T22:07:36.239 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.047054390, 2024-09-11T22:07:36.240 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000182 2024-09-11T22:07:36.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:36.240 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 306208, 2024-09-11T22:07:36.240 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 210672 2024-09-11T22:07:36.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:36.240 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T22:07:36.240 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 71002, 2024-09-11T22:07:36.241 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 45375, 2024-09-11T22:07:36.241 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 34647926, 2024-09-11T22:07:36.241 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 120922137, 2024-09-11T22:07:36.241 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 148, 2024-09-11T22:07:36.241 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 10, 2024-09-11T22:07:36.241 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 3.695678303, 2024-09-11T22:07:36.242 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 1.499882394, 2024-09-11T22:07:36.242 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 2.165157281, 2024-09-11T22:07:36.242 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.606082730, 2024-09-11T22:07:36.242 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:07:36.242 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 45367, 2024-09-11T22:07:36.242 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 19.936761578, 2024-09-11T22:07:36.242 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000439455 2024-09-11T22:07:36.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:36.243 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:07:36.243 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 77630, 2024-09-11T22:07:36.243 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.010570462, 2024-09-11T22:07:36.243 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000136 2024-09-11T22:07:36.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:36.243 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 42544, 2024-09-11T22:07:36.244 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 1136 2024-09-11T22:07:36.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:36.244 INFO:teuthology.orchestra.run.smithi053.stdout: "cct": { 2024-09-11T22:07:36.245 INFO:teuthology.orchestra.run.smithi053.stdout: "total_workers": 1, 2024-09-11T22:07:36.245 INFO:teuthology.orchestra.run.smithi053.stdout: "unhealthy_workers": 0 2024-09-11T22:07:36.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:36.246 INFO:teuthology.orchestra.run.smithi053.stdout: "finisher-MDSRank": { 2024-09-11T22:07:36.246 INFO:teuthology.orchestra.run.smithi053.stdout: "queue_len": 0, 2024-09-11T22:07:36.246 INFO:teuthology.orchestra.run.smithi053.stdout: "complete_latency": { 2024-09-11T22:07:36.246 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 13947, 2024-09-11T22:07:36.246 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 60.179926914, 2024-09-11T22:07:36.247 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.004314901 2024-09-11T22:07:36.247 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-11T22:07:36.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:36.247 INFO:teuthology.orchestra.run.smithi053.stdout: "finisher-PurgeQueue": { 2024-09-11T22:07:36.247 INFO:teuthology.orchestra.run.smithi053.stdout: "queue_len": 0, 2024-09-11T22:07:36.247 INFO:teuthology.orchestra.run.smithi053.stdout: "complete_latency": { 2024-09-11T22:07:36.247 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 13606, 2024-09-11T22:07:36.247 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 2.815875129, 2024-09-11T22:07:36.248 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000206958 2024-09-11T22:07:36.248 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-11T22:07:36.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:36.248 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": { 2024-09-11T22:07:36.248 INFO:teuthology.orchestra.run.smithi053.stdout: "request": 45445, 2024-09-11T22:07:36.248 INFO:teuthology.orchestra.run.smithi053.stdout: "reply": 42208, 2024-09-11T22:07:36.248 INFO:teuthology.orchestra.run.smithi053.stdout: "reply_latency": { 2024-09-11T22:07:36.249 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 42208, 2024-09-11T22:07:36.249 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 209.853063339, 2024-09-11T22:07:36.249 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.004971878 2024-09-11T22:07:36.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:36.249 INFO:teuthology.orchestra.run.smithi053.stdout: "slow_reply": 0, 2024-09-11T22:07:36.249 INFO:teuthology.orchestra.run.smithi053.stdout: "forward": 3237, 2024-09-11T22:07:36.249 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_fetch_complete": 259, 2024-09-11T22:07:36.250 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_fetch_keys": 1, 2024-09-11T22:07:36.250 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_commit": 7285, 2024-09-11T22:07:36.250 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_split": 110, 2024-09-11T22:07:36.250 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_merge": 0, 2024-09-11T22:07:36.250 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes": 14318, 2024-09-11T22:07:36.250 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_top": 10003, 2024-09-11T22:07:36.250 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_bottom": 4304, 2024-09-11T22:07:36.250 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_pin_tail": 11, 2024-09-11T22:07:36.251 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_pinned": 22, 2024-09-11T22:07:36.251 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_expired": 32018, 2024-09-11T22:07:36.251 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_with_caps": 2, 2024-09-11T22:07:36.251 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": 2, 2024-09-11T22:07:36.251 INFO:teuthology.orchestra.run.smithi053.stdout: "subtrees": 6, 2024-09-11T22:07:36.251 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse": 54123, 2024-09-11T22:07:36.251 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_hit": 50288, 2024-09-11T22:07:36.251 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_forward": 3197, 2024-09-11T22:07:36.252 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_discover": 5, 2024-09-11T22:07:36.252 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_dir_fetch": 0, 2024-09-11T22:07:36.252 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_remote_ino": 0, 2024-09-11T22:07:36.252 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_lock": 0, 2024-09-11T22:07:36.252 INFO:teuthology.orchestra.run.smithi053.stdout: "load_cent": 0, 2024-09-11T22:07:36.252 INFO:teuthology.orchestra.run.smithi053.stdout: "q": 0, 2024-09-11T22:07:36.252 INFO:teuthology.orchestra.run.smithi053.stdout: "exported": 430, 2024-09-11T22:07:36.252 INFO:teuthology.orchestra.run.smithi053.stdout: "exported_inodes": 29437, 2024-09-11T22:07:36.253 INFO:teuthology.orchestra.run.smithi053.stdout: "imported": 294, 2024-09-11T22:07:36.253 INFO:teuthology.orchestra.run.smithi053.stdout: "imported_inodes": 13844, 2024-09-11T22:07:36.253 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_dir_fetch": 0, 2024-09-11T22:07:36.253 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_backtrace_fetch": 9980, 2024-09-11T22:07:36.253 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_peer_discover": 0, 2024-09-11T22:07:36.253 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rfiles": 4, 2024-09-11T22:07:36.253 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rbytes": 234, 2024-09-11T22:07:36.253 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rsnaps": 7, 2024-09-11T22:07:36.254 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_backtrace_fetch": 9980, 2024-09-11T22:07:36.254 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_set_tag": 0, 2024-09-11T22:07:36.254 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T22:07:36.254 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_inotable_repaired": 0, 2024-09-11T22:07:36.254 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dir_inodes": 82, 2024-09-11T22:07:36.254 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dir_base_inodes": 9, 2024-09-11T22:07:36.254 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dirfrag_rstats": 73, 2024-09-11T22:07:36.255 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_file_inodes": 9898, 2024-09-11T22:07:36.255 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_inode_file_caps": 17, 2024-09-11T22:07:36.255 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_revoke": 7640, 2024-09-11T22:07:36.255 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_grant": 34310, 2024-09-11T22:07:36.255 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_trunc": 1, 2024-09-11T22:07:36.255 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T22:07:36.256 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T22:07:36.256 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_caps": 23504, 2024-09-11T22:07:36.256 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_caps_dirty": 21808, 2024-09-11T22:07:36.257 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_cap_release": 319, 2024-09-11T22:07:36.257 INFO:teuthology.orchestra.run.smithi053.stdout: "process_request_cap_release": 15661 2024-09-11T22:07:36.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:36.257 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_cache": { 2024-09-11T22:07:36.257 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_update": 1439, 2024-09-11T22:07:36.257 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_update_receipt": 2207, 2024-09-11T22:07:36.257 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_try_discover": 1106, 2024-09-11T22:07:36.258 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_send_discover": 2221, 2024-09-11T22:07:36.258 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_handle_discover": 2390, 2024-09-11T22:07:36.258 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays": 11, 2024-09-11T22:07:36.258 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays_delayed": 0, 2024-09-11T22:07:36.258 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays_enqueuing": 0, 2024-09-11T22:07:36.258 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_created": 7462, 2024-09-11T22:07:36.258 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_enqueued": 7451, 2024-09-11T22:07:36.259 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_reintegrated": 0, 2024-09-11T22:07:36.259 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_migrated": 0, 2024-09-11T22:07:36.259 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_processing": 0, 2024-09-11T22:07:36.259 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_enqueued": 0, 2024-09-11T22:07:36.259 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_prioritized": 0, 2024-09-11T22:07:36.259 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_started": 0, 2024-09-11T22:07:36.259 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_completed": 0, 2024-09-11T22:07:36.259 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_quiesce_path": 8, 2024-09-11T22:07:36.260 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_quiesce_inode": 23321, 2024-09-11T22:07:36.260 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_enqueue_scrub": 9, 2024-09-11T22:07:36.260 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_exportdir": 430, 2024-09-11T22:07:36.260 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_flush": 0, 2024-09-11T22:07:36.260 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_fragmentdir": 113, 2024-09-11T22:07:36.260 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_fragstats": 0, 2024-09-11T22:07:36.260 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_inodestats": 0 2024-09-11T22:07:36.260 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:36.261 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_log": { 2024-09-11T22:07:36.261 INFO:teuthology.orchestra.run.smithi053.stdout: "evlrg": 4, 2024-09-11T22:07:36.261 INFO:teuthology.orchestra.run.smithi053.stdout: "evadd": 65031, 2024-09-11T22:07:36.261 INFO:teuthology.orchestra.run.smithi053.stdout: "evex": 64965, 2024-09-11T22:07:36.261 INFO:teuthology.orchestra.run.smithi053.stdout: "evtrm": 64964, 2024-09-11T22:07:36.261 INFO:teuthology.orchestra.run.smithi053.stdout: "ev": 67, 2024-09-11T22:07:36.261 INFO:teuthology.orchestra.run.smithi053.stdout: "evexg": 58, 2024-09-11T22:07:36.262 INFO:teuthology.orchestra.run.smithi053.stdout: "evexd": 1, 2024-09-11T22:07:36.262 INFO:teuthology.orchestra.run.smithi053.stdout: "segadd": 114, 2024-09-11T22:07:36.262 INFO:teuthology.orchestra.run.smithi053.stdout: "segex": 112, 2024-09-11T22:07:36.262 INFO:teuthology.orchestra.run.smithi053.stdout: "segtrm": 111, 2024-09-11T22:07:36.262 INFO:teuthology.orchestra.run.smithi053.stdout: "seg": 3, 2024-09-11T22:07:36.262 INFO:teuthology.orchestra.run.smithi053.stdout: "segmjr": 2, 2024-09-11T22:07:36.263 INFO:teuthology.orchestra.run.smithi053.stdout: "segexg": 1, 2024-09-11T22:07:36.263 INFO:teuthology.orchestra.run.smithi053.stdout: "segexd": 1, 2024-09-11T22:07:36.263 INFO:teuthology.orchestra.run.smithi053.stdout: "expos": 196897656, 2024-09-11T22:07:36.263 INFO:teuthology.orchestra.run.smithi053.stdout: "wrpos": 197222657, 2024-09-11T22:07:36.263 INFO:teuthology.orchestra.run.smithi053.stdout: "rdpos": 0, 2024-09-11T22:07:36.263 INFO:teuthology.orchestra.run.smithi053.stdout: "jlat": { 2024-09-11T22:07:36.263 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 8175, 2024-09-11T22:07:36.263 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 1487.698761729, 2024-09-11T22:07:36.264 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.181981499 2024-09-11T22:07:36.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:36.264 INFO:teuthology.orchestra.run.smithi053.stdout: "replayed": 0 2024-09-11T22:07:36.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:36.264 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_mem": { 2024-09-11T22:07:36.264 INFO:teuthology.orchestra.run.smithi053.stdout: "ino": 14295, 2024-09-11T22:07:36.264 INFO:teuthology.orchestra.run.smithi053.stdout: "ino+": 48888, 2024-09-11T22:07:36.264 INFO:teuthology.orchestra.run.smithi053.stdout: "ino-": 34593, 2024-09-11T22:07:36.265 INFO:teuthology.orchestra.run.smithi053.stdout: "dir": 580, 2024-09-11T22:07:36.265 INFO:teuthology.orchestra.run.smithi053.stdout: "dir+": 3757, 2024-09-11T22:07:36.265 INFO:teuthology.orchestra.run.smithi053.stdout: "dir-": 3177, 2024-09-11T22:07:36.265 INFO:teuthology.orchestra.run.smithi053.stdout: "dn": 14318, 2024-09-11T22:07:36.265 INFO:teuthology.orchestra.run.smithi053.stdout: "dn+": 63790, 2024-09-11T22:07:36.265 INFO:teuthology.orchestra.run.smithi053.stdout: "dn-": 49472, 2024-09-11T22:07:36.265 INFO:teuthology.orchestra.run.smithi053.stdout: "cap": 2, 2024-09-11T22:07:36.265 INFO:teuthology.orchestra.run.smithi053.stdout: "cap+": 33862, 2024-09-11T22:07:36.266 INFO:teuthology.orchestra.run.smithi053.stdout: "cap-": 33860, 2024-09-11T22:07:36.266 INFO:teuthology.orchestra.run.smithi053.stdout: "rss": 213088, 2024-09-11T22:07:36.266 INFO:teuthology.orchestra.run.smithi053.stdout: "heap": 231712 2024-09-11T22:07:36.266 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:36.266 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_server": { 2024-09-11T22:07:36.266 INFO:teuthology.orchestra.run.smithi053.stdout: "dispatch_client_request": 46396, 2024-09-11T22:07:36.266 INFO:teuthology.orchestra.run.smithi053.stdout: "dispatch_server_request": 7870, 2024-09-11T22:07:36.266 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_request": 45445, 2024-09-11T22:07:36.267 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_session": 400, 2024-09-11T22:07:36.267 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_peer_request": 15785, 2024-09-11T22:07:36.267 INFO:teuthology.orchestra.run.smithi053.stdout: "req_create_latency": { 2024-09-11T22:07:36.267 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 26464, 2024-09-11T22:07:36.267 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 140.487286920, 2024-09-11T22:07:36.267 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.005308618 2024-09-11T22:07:36.267 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:36.268 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getattr_latency": { 2024-09-11T22:07:36.268 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 208, 2024-09-11T22:07:36.268 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.074170440, 2024-09-11T22:07:36.268 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000356588 2024-09-11T22:07:36.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:36.268 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getfilelock_latency": { 2024-09-11T22:07:36.268 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:36.269 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:36.269 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:36.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:36.269 INFO:teuthology.orchestra.run.smithi053.stdout: "req_link_latency": { 2024-09-11T22:07:36.269 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:36.269 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:36.269 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:36.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:36.270 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookup_latency": { 2024-09-11T22:07:36.270 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 6984, 2024-09-11T22:07:36.270 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 5.117063073, 2024-09-11T22:07:36.270 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000732683 2024-09-11T22:07:36.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:36.270 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookuphash_latency": { 2024-09-11T22:07:36.270 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:36.270 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:36.271 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:36.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:36.271 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupino_latency": { 2024-09-11T22:07:36.271 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:36.271 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:36.271 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:36.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:36.271 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupname_latency": { 2024-09-11T22:07:36.272 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:36.272 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:36.272 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:36.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:36.272 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupparent_latency": { 2024-09-11T22:07:36.273 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:36.273 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:36.273 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:36.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:36.273 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupsnap_latency": { 2024-09-11T22:07:36.274 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:36.274 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:36.274 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:36.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:36.275 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lssnap_latency": { 2024-09-11T22:07:36.275 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 10, 2024-09-11T22:07:36.275 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.002056999, 2024-09-11T22:07:36.275 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000205699 2024-09-11T22:07:36.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:36.275 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mkdir_latency": { 2024-09-11T22:07:36.275 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 130, 2024-09-11T22:07:36.275 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 16.030378235, 2024-09-11T22:07:36.276 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.123310601 2024-09-11T22:07:36.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:36.276 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mknod_latency": { 2024-09-11T22:07:36.276 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:36.276 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:36.276 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:36.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:36.277 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mksnap_latency": { 2024-09-11T22:07:36.277 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 10, 2024-09-11T22:07:36.277 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 2.923853977, 2024-09-11T22:07:36.277 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.292385397 2024-09-11T22:07:36.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:36.277 INFO:teuthology.orchestra.run.smithi053.stdout: "req_open_latency": { 2024-09-11T22:07:36.278 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 1, 2024-09-11T22:07:36.278 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000214676, 2024-09-11T22:07:36.278 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000214676 2024-09-11T22:07:36.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:36.278 INFO:teuthology.orchestra.run.smithi053.stdout: "req_readdir_latency": { 2024-09-11T22:07:36.278 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 880, 2024-09-11T22:07:36.278 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.739686265, 2024-09-11T22:07:36.279 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000840552 2024-09-11T22:07:36.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:36.279 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rename_latency": { 2024-09-11T22:07:36.279 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 4, 2024-09-11T22:07:36.279 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.006749628, 2024-09-11T22:07:36.279 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.001687407 2024-09-11T22:07:36.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:36.279 INFO:teuthology.orchestra.run.smithi053.stdout: "req_renamesnap_latency": { 2024-09-11T22:07:36.280 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:36.280 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:36.280 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:36.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:36.280 INFO:teuthology.orchestra.run.smithi053.stdout: "req_snapdiff_latency": { 2024-09-11T22:07:36.280 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:36.280 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:36.281 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:36.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:36.281 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmdir_latency": { 2024-09-11T22:07:36.281 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 117, 2024-09-11T22:07:36.281 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 10.350107144, 2024-09-11T22:07:36.281 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.088462454 2024-09-11T22:07:36.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:36.282 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmsnap_latency": { 2024-09-11T22:07:36.282 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 1, 2024-09-11T22:07:36.282 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.470704306, 2024-09-11T22:07:36.282 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.470704306 2024-09-11T22:07:36.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:36.282 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmxattr_latency": { 2024-09-11T22:07:36.282 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 3, 2024-09-11T22:07:36.282 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.010528449, 2024-09-11T22:07:36.283 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.003509483 2024-09-11T22:07:36.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:36.283 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setattr_latency": { 2024-09-11T22:07:36.283 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 4, 2024-09-11T22:07:36.283 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 5.549850889, 2024-09-11T22:07:36.283 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 1.387462722 2024-09-11T22:07:36.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:36.283 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setdirlayout_latency": { 2024-09-11T22:07:36.284 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:36.284 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:36.284 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:36.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:36.284 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setfilelock_latency": { 2024-09-11T22:07:36.284 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 22, 2024-09-11T22:07:36.284 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 10.012074531, 2024-09-11T22:07:36.284 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.455094296 2024-09-11T22:07:36.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:36.285 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setlayout_latency": { 2024-09-11T22:07:36.285 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:36.285 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:36.285 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:36.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:36.285 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setxattr_latency": { 2024-09-11T22:07:36.285 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 24, 2024-09-11T22:07:36.286 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.080627841, 2024-09-11T22:07:36.286 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.003359493 2024-09-11T22:07:36.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:36.286 INFO:teuthology.orchestra.run.smithi053.stdout: "req_symlink_latency": { 2024-09-11T22:07:36.286 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:36.286 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:36.286 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:36.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:36.286 INFO:teuthology.orchestra.run.smithi053.stdout: "req_unlink_latency": { 2024-09-11T22:07:36.287 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 7344, 2024-09-11T22:07:36.287 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 17.997434316, 2024-09-11T22:07:36.287 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.002450631 2024-09-11T22:07:36.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:36.287 INFO:teuthology.orchestra.run.smithi053.stdout: "cap_revoke_eviction": 0, 2024-09-11T22:07:36.287 INFO:teuthology.orchestra.run.smithi053.stdout: "cap_acquisition_throttle": 0, 2024-09-11T22:07:36.287 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getvxattr_latency": { 2024-09-11T22:07:36.287 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 2, 2024-09-11T22:07:36.288 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000275650, 2024-09-11T22:07:36.288 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000137825 2024-09-11T22:07:36.288 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-11T22:07:36.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:36.288 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_sessions": { 2024-09-11T22:07:36.288 INFO:teuthology.orchestra.run.smithi053.stdout: "session_count": 2, 2024-09-11T22:07:36.288 INFO:teuthology.orchestra.run.smithi053.stdout: "session_add": 5, 2024-09-11T22:07:36.288 INFO:teuthology.orchestra.run.smithi053.stdout: "session_remove": 3, 2024-09-11T22:07:36.289 INFO:teuthology.orchestra.run.smithi053.stdout: "sessions_open": 2, 2024-09-11T22:07:36.289 INFO:teuthology.orchestra.run.smithi053.stdout: "sessions_stale": 0, 2024-09-11T22:07:36.289 INFO:teuthology.orchestra.run.smithi053.stdout: "total_load": 2310, 2024-09-11T22:07:36.289 INFO:teuthology.orchestra.run.smithi053.stdout: "average_load": 1155, 2024-09-11T22:07:36.289 INFO:teuthology.orchestra.run.smithi053.stdout: "avg_session_uptime": 599, 2024-09-11T22:07:36.289 INFO:teuthology.orchestra.run.smithi053.stdout: "mdthresh_evicted": 0 2024-09-11T22:07:36.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:36.290 INFO:teuthology.orchestra.run.smithi053.stdout: "mempool": { 2024-09-11T22:07:36.290 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_bytes": 360, 2024-09-11T22:07:36.290 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_items": 360, 2024-09-11T22:07:36.290 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T22:07:36.290 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_items": 0, 2024-09-11T22:07:36.290 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T22:07:36.290 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_items": 0, 2024-09-11T22:07:36.291 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T22:07:36.291 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T22:07:36.291 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T22:07:36.291 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T22:07:36.292 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T22:07:36.292 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_items": 0, 2024-09-11T22:07:36.292 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T22:07:36.292 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T22:07:36.292 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_bytes": 0, 2024-09-11T22:07:36.292 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_items": 0, 2024-09-11T22:07:36.292 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_bytes": 0, 2024-09-11T22:07:36.292 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_items": 0, 2024-09-11T22:07:36.293 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T22:07:36.293 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T22:07:36.293 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T22:07:36.293 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T22:07:36.293 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T22:07:36.293 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_items": 0, 2024-09-11T22:07:36.293 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_bytes": 0, 2024-09-11T22:07:36.293 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_items": 0, 2024-09-11T22:07:36.294 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T22:07:36.294 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T22:07:36.294 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_bytes": 0, 2024-09-11T22:07:36.294 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_items": 0, 2024-09-11T22:07:36.294 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_bytes": 0, 2024-09-11T22:07:36.294 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_items": 0, 2024-09-11T22:07:36.294 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T22:07:36.295 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_items": 0, 2024-09-11T22:07:36.295 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T22:07:36.295 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_items": 0, 2024-09-11T22:07:36.295 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_bytes": 738304, 2024-09-11T22:07:36.295 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_items": 11008, 2024-09-11T22:07:36.295 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_bytes": 88, 2024-09-11T22:07:36.295 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_items": 1, 2024-09-11T22:07:36.295 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_bytes": 0, 2024-09-11T22:07:36.296 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_items": 0, 2024-09-11T22:07:36.296 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_bytes": 0, 2024-09-11T22:07:36.296 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_items": 0, 2024-09-11T22:07:36.296 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_bytes": 0, 2024-09-11T22:07:36.296 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_items": 0, 2024-09-11T22:07:36.296 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_bytes": 8392, 2024-09-11T22:07:36.296 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_items": 184, 2024-09-11T22:07:36.297 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T22:07:36.297 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_items": 0, 2024-09-11T22:07:36.297 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_bytes": 0, 2024-09-11T22:07:36.297 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_items": 0, 2024-09-11T22:07:36.297 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_bytes": 43046267, 2024-09-11T22:07:36.297 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_items": 338477, 2024-09-11T22:07:36.297 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_bytes": 0, 2024-09-11T22:07:36.298 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_items": 0, 2024-09-11T22:07:36.298 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_bytes": 0, 2024-09-11T22:07:36.298 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_items": 0 2024-09-11T22:07:36.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:36.299 INFO:teuthology.orchestra.run.smithi053.stdout: "objecter": { 2024-09-11T22:07:36.299 INFO:teuthology.orchestra.run.smithi053.stdout: "op_active": 0, 2024-09-11T22:07:36.299 INFO:teuthology.orchestra.run.smithi053.stdout: "op_laggy": 0, 2024-09-11T22:07:36.299 INFO:teuthology.orchestra.run.smithi053.stdout: "op_send": 56278, 2024-09-11T22:07:36.299 INFO:teuthology.orchestra.run.smithi053.stdout: "op_send_bytes": 244262172, 2024-09-11T22:07:36.299 INFO:teuthology.orchestra.run.smithi053.stdout: "op_resend": 37, 2024-09-11T22:07:36.300 INFO:teuthology.orchestra.run.smithi053.stdout: "op_reply": 56278, 2024-09-11T22:07:36.300 INFO:teuthology.orchestra.run.smithi053.stdout: "op_latency": { 2024-09-11T22:07:36.300 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 56278, 2024-09-11T22:07:36.300 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 7021.660642148, 2024-09-11T22:07:36.300 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.124767416 2024-09-11T22:07:36.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:36.301 INFO:teuthology.orchestra.run.smithi053.stdout: "op_inflight": 0, 2024-09-11T22:07:36.301 INFO:teuthology.orchestra.run.smithi053.stdout: "oplen_avg": { 2024-09-11T22:07:36.301 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 56279, 2024-09-11T22:07:36.301 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 110474 2024-09-11T22:07:36.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:36.302 INFO:teuthology.orchestra.run.smithi053.stdout: "op": 56279, 2024-09-11T22:07:36.302 INFO:teuthology.orchestra.run.smithi053.stdout: "op_r": 10438, 2024-09-11T22:07:36.302 INFO:teuthology.orchestra.run.smithi053.stdout: "op_w": 45841, 2024-09-11T22:07:36.302 INFO:teuthology.orchestra.run.smithi053.stdout: "op_rmw": 0, 2024-09-11T22:07:36.302 INFO:teuthology.orchestra.run.smithi053.stdout: "op_pg": 0, 2024-09-11T22:07:36.302 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_stat": 6366, 2024-09-11T22:07:36.302 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_create": 21419, 2024-09-11T22:07:36.303 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_read": 189, 2024-09-11T22:07:36.303 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_write": 8355, 2024-09-11T22:07:36.303 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_writefull": 237, 2024-09-11T22:07:36.303 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_writesame": 0, 2024-09-11T22:07:36.303 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_append": 0, 2024-09-11T22:07:36.303 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_zero": 0, 2024-09-11T22:07:36.303 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_truncate": 5, 2024-09-11T22:07:36.303 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_delete": 8231, 2024-09-11T22:07:36.304 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_mapext": 0, 2024-09-11T22:07:36.304 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_sparse_read": 0, 2024-09-11T22:07:36.304 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_clonerange": 0, 2024-09-11T22:07:36.304 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_getxattr": 9998, 2024-09-11T22:07:36.304 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_setxattr": 42837, 2024-09-11T22:07:36.304 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_cmpxattr": 0, 2024-09-11T22:07:36.304 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_rmxattr": 0, 2024-09-11T22:07:36.305 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_resetxattrs": 0, 2024-09-11T22:07:36.305 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_call": 0, 2024-09-11T22:07:36.305 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_watch": 0, 2024-09-11T22:07:36.305 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_notify": 0, 2024-09-11T22:07:36.305 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T22:07:36.305 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_pgls": 0, 2024-09-11T22:07:36.305 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_pgls_filter": 0, 2024-09-11T22:07:36.305 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_other": 1, 2024-09-11T22:07:36.305 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_active": 0, 2024-09-11T22:07:36.306 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_send": 0, 2024-09-11T22:07:36.306 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_resend": 0, 2024-09-11T22:07:36.306 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_ping": 0, 2024-09-11T22:07:36.306 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_active": 0, 2024-09-11T22:07:36.306 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_send": 0, 2024-09-11T22:07:36.306 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_resend": 0, 2024-09-11T22:07:36.306 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_active": 0, 2024-09-11T22:07:36.306 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_send": 0, 2024-09-11T22:07:36.307 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_resend": 0, 2024-09-11T22:07:36.307 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_active": 0, 2024-09-11T22:07:36.307 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_send": 0, 2024-09-11T22:07:36.307 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_resend": 0, 2024-09-11T22:07:36.307 INFO:teuthology.orchestra.run.smithi053.stdout: "command_active": 0, 2024-09-11T22:07:36.307 INFO:teuthology.orchestra.run.smithi053.stdout: "command_send": 0, 2024-09-11T22:07:36.307 INFO:teuthology.orchestra.run.smithi053.stdout: "command_resend": 0, 2024-09-11T22:07:36.307 INFO:teuthology.orchestra.run.smithi053.stdout: "map_epoch": 95, 2024-09-11T22:07:36.308 INFO:teuthology.orchestra.run.smithi053.stdout: "map_full": 0, 2024-09-11T22:07:36.308 INFO:teuthology.orchestra.run.smithi053.stdout: "map_inc": 11, 2024-09-11T22:07:36.308 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_sessions": 12, 2024-09-11T22:07:36.308 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_session_open": 12, 2024-09-11T22:07:36.308 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_session_close": 0, 2024-09-11T22:07:36.308 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_laggy": 0, 2024-09-11T22:07:36.308 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_wr": 11642, 2024-09-11T22:07:36.309 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_rd": 520, 2024-09-11T22:07:36.309 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_del": 674 2024-09-11T22:07:36.309 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:36.309 INFO:teuthology.orchestra.run.smithi053.stdout: "oft": { 2024-09-11T22:07:36.309 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_objs": 1, 2024-09-11T22:07:36.309 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_kv_pairs": 0, 2024-09-11T22:07:36.310 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_updates": 0, 2024-09-11T22:07:36.310 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_removes": 0 2024-09-11T22:07:36.310 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:36.310 INFO:teuthology.orchestra.run.smithi053.stdout: "osd-slow-ops": { 2024-09-11T22:07:36.311 INFO:teuthology.orchestra.run.smithi053.stdout: "slow_ops_count": 23299 2024-09-11T22:07:36.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:36.311 INFO:teuthology.orchestra.run.smithi053.stdout: "purge_queue": { 2024-09-11T22:07:36.311 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_ops": 0, 2024-09-11T22:07:36.311 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_ops_high_water": 64, 2024-09-11T22:07:36.311 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing": 0, 2024-09-11T22:07:36.311 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_high_water": 6, 2024-09-11T22:07:36.311 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executed_ops": 7521, 2024-09-11T22:07:36.312 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executed": 7451, 2024-09-11T22:07:36.312 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_item_in_journal": 0 2024-09-11T22:07:36.312 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:36.312 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T22:07:36.312 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-09-11T22:07:36.312 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 104857600, 2024-09-11T22:07:36.312 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-09-11T22:07:36.312 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 222013, 2024-09-11T22:07:36.313 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 86005907, 2024-09-11T22:07:36.313 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-09-11T22:07:36.313 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 222013, 2024-09-11T22:07:36.313 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-09-11T22:07:36.313 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-09-11T22:07:36.313 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 222013, 2024-09-11T22:07:36.313 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 86005907, 2024-09-11T22:07:36.313 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-09-11T22:07:36.314 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:36.314 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:36.314 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:36.314 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-11T22:07:36.314 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:36.314 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-objecter_bytes": { 2024-09-11T22:07:36.314 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-09-11T22:07:36.314 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 104857600, 2024-09-11T22:07:36.315 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-09-11T22:07:36.315 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-09-11T22:07:36.315 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-09-11T22:07:36.315 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-09-11T22:07:36.315 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-09-11T22:07:36.315 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 56279, 2024-09-11T22:07:36.315 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 245099742, 2024-09-11T22:07:36.316 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 56279, 2024-09-11T22:07:36.316 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 245099742, 2024-09-11T22:07:36.316 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-09-11T22:07:36.316 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:36.316 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:36.317 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:36.317 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-11T22:07:36.317 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:36.317 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-objecter_ops": { 2024-09-11T22:07:36.317 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-09-11T22:07:36.317 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 1024, 2024-09-11T22:07:36.317 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-09-11T22:07:36.317 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-09-11T22:07:36.318 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-09-11T22:07:36.318 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-09-11T22:07:36.318 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-09-11T22:07:36.318 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 56279, 2024-09-11T22:07:36.318 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 56279, 2024-09-11T22:07:36.318 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 56279, 2024-09-11T22:07:36.318 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 56279, 2024-09-11T22:07:36.318 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-09-11T22:07:36.319 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:36.319 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:36.319 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:36.319 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-11T22:07:36.319 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:36.319 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-write_buf_throttle": { 2024-09-11T22:07:36.320 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-09-11T22:07:36.320 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 3758096384, 2024-09-11T22:07:36.320 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-09-11T22:07:36.320 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 7451, 2024-09-11T22:07:36.321 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 841569, 2024-09-11T22:07:36.321 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-09-11T22:07:36.321 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 7451, 2024-09-11T22:07:36.322 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-09-11T22:07:36.322 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-09-11T22:07:36.322 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 180, 2024-09-11T22:07:36.322 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 841569, 2024-09-11T22:07:36.322 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-09-11T22:07:36.322 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:36.322 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:36.322 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:36.323 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-11T22:07:36.323 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:36.323 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-write_buf_throttle-0x562d9c84a500": { 2024-09-11T22:07:36.323 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-09-11T22:07:36.323 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 3758096384, 2024-09-11T22:07:36.323 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-09-11T22:07:36.323 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 65031, 2024-09-11T22:07:36.324 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 193028353, 2024-09-11T22:07:36.324 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-09-11T22:07:36.324 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 65031, 2024-09-11T22:07:36.324 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-09-11T22:07:36.324 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-09-11T22:07:36.324 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 8175, 2024-09-11T22:07:36.324 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 193028353, 2024-09-11T22:07:36.324 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-09-11T22:07:36.325 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:36.325 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:36.325 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:36.325 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-11T22:07:36.325 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-11T22:07:36.325 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-11T22:07:37.047 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:36 smithi053 ceph-mon[34908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:07:37.048 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:36 smithi053 ceph-mon[34908]: pgmap v322: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 17 op/s 2024-09-11T22:07:37.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:36 smithi161 ceph-mon[35615]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:07:37.092 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:36 smithi161 ceph-mon[35615]: pgmap v322: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 17 op/s 2024-09-11T22:07:37.134 INFO:tasks.check_counter:Daemon mds.b mds.exported=430 2024-09-11T22:07:37.134 INFO:tasks.check_counter:Daemon mds.b mds.imported=294 2024-09-11T22:07:37.134 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=1439 2024-09-11T22:07:37.134 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=2207 2024-09-11T22:07:37.134 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=7 2024-09-11T22:07:37.134 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=10 2024-09-11T22:07:37.134 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=1 2024-09-11T22:07:37.135 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=110 2024-09-11T22:07:37.135 DEBUG:tasks.check_counter:Getting stats from e 2024-09-11T22:07:37.135 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-11T22:07:37.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:36 smithi001 ceph-mon[30935]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:07:37.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:36 smithi001 ceph-mon[30935]: pgmap v322: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 17 op/s 2024-09-11T22:07:37.395 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T22:07:37.873 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-11T22:07:37.874 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T22:07:37.874 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 2014, 2024-09-11T22:07:37.874 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 1979, 2024-09-11T22:07:37.874 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 48501353, 2024-09-11T22:07:37.874 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 687964, 2024-09-11T22:07:37.874 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 7, 2024-09-11T22:07:37.874 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 3, 2024-09-11T22:07:37.875 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.210106534, 2024-09-11T22:07:37.875 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.065904575, 2024-09-11T22:07:37.875 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.105607220, 2024-09-11T22:07:37.875 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.041274238, 2024-09-11T22:07:37.875 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:07:37.875 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 1978, 2024-09-11T22:07:37.875 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.071494206, 2024-09-11T22:07:37.875 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000036144 2024-09-11T22:07:37.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:37.876 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:07:37.876 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:37.876 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:37.876 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:37.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:37.876 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 1757760, 2024-09-11T22:07:37.876 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 129552 2024-09-11T22:07:37.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:37.877 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T22:07:37.877 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 283, 2024-09-11T22:07:37.877 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 437, 2024-09-11T22:07:37.877 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 83998074, 2024-09-11T22:07:37.877 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 524904, 2024-09-11T22:07:37.877 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 9, 2024-09-11T22:07:37.878 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 7, 2024-09-11T22:07:37.878 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.078433943, 2024-09-11T22:07:37.878 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.011474252, 2024-09-11T22:07:37.878 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.057656412, 2024-09-11T22:07:37.878 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.005852145, 2024-09-11T22:07:37.878 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:07:37.878 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 435, 2024-09-11T22:07:37.878 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.031801881, 2024-09-11T22:07:37.879 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000073107 2024-09-11T22:07:37.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:37.879 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:07:37.879 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:37.879 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:37.879 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:37.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:37.879 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-09-11T22:07:37.880 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 436752 2024-09-11T22:07:37.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:37.880 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T22:07:37.880 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 560, 2024-09-11T22:07:37.880 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 562, 2024-09-11T22:07:37.880 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 63070600, 2024-09-11T22:07:37.880 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 178087, 2024-09-11T22:07:37.880 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 6, 2024-09-11T22:07:37.881 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 5, 2024-09-11T22:07:37.882 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.083994573, 2024-09-11T22:07:37.882 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.012203185, 2024-09-11T22:07:37.882 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.051280070, 2024-09-11T22:07:37.882 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.016163277, 2024-09-11T22:07:37.882 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:07:37.883 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 560, 2024-09-11T22:07:37.883 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.020540810, 2024-09-11T22:07:37.883 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000036680 2024-09-11T22:07:37.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:37.883 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:07:37.883 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:37.884 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:37.884 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:37.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:37.884 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-11T22:07:37.884 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 0 2024-09-11T22:07:37.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:37.884 INFO:teuthology.orchestra.run.smithi053.stdout: "cct": { 2024-09-11T22:07:37.884 INFO:teuthology.orchestra.run.smithi053.stdout: "total_workers": 1, 2024-09-11T22:07:37.885 INFO:teuthology.orchestra.run.smithi053.stdout: "unhealthy_workers": 0 2024-09-11T22:07:37.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:37.885 INFO:teuthology.orchestra.run.smithi053.stdout: "finisher-MDSRank": { 2024-09-11T22:07:37.886 INFO:teuthology.orchestra.run.smithi053.stdout: "queue_len": 0, 2024-09-11T22:07:37.886 INFO:teuthology.orchestra.run.smithi053.stdout: "complete_latency": { 2024-09-11T22:07:37.886 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 4218, 2024-09-11T22:07:37.886 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.611090840, 2024-09-11T22:07:37.886 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000144876 2024-09-11T22:07:37.886 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-11T22:07:37.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:37.887 INFO:teuthology.orchestra.run.smithi053.stdout: "finisher-PurgeQueue": { 2024-09-11T22:07:37.887 INFO:teuthology.orchestra.run.smithi053.stdout: "queue_len": 0, 2024-09-11T22:07:37.887 INFO:teuthology.orchestra.run.smithi053.stdout: "complete_latency": { 2024-09-11T22:07:37.887 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:37.887 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:37.887 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:37.887 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-11T22:07:37.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:37.888 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": { 2024-09-11T22:07:37.888 INFO:teuthology.orchestra.run.smithi053.stdout: "request": 0, 2024-09-11T22:07:37.888 INFO:teuthology.orchestra.run.smithi053.stdout: "reply": 0, 2024-09-11T22:07:37.888 INFO:teuthology.orchestra.run.smithi053.stdout: "reply_latency": { 2024-09-11T22:07:37.888 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:37.888 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:37.888 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:37.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:37.889 INFO:teuthology.orchestra.run.smithi053.stdout: "slow_reply": 0, 2024-09-11T22:07:37.889 INFO:teuthology.orchestra.run.smithi053.stdout: "forward": 0, 2024-09-11T22:07:37.889 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_fetch_complete": 0, 2024-09-11T22:07:37.889 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_fetch_keys": 0, 2024-09-11T22:07:37.889 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_commit": 0, 2024-09-11T22:07:37.889 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_split": 0, 2024-09-11T22:07:37.889 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_merge": 0, 2024-09-11T22:07:37.890 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes": 30925, 2024-09-11T22:07:37.890 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_top": 21625, 2024-09-11T22:07:37.890 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_bottom": 9300, 2024-09-11T22:07:37.890 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_pin_tail": 0, 2024-09-11T22:07:37.890 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_pinned": 32, 2024-09-11T22:07:37.890 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_expired": 0, 2024-09-11T22:07:37.890 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_with_caps": 0, 2024-09-11T22:07:37.890 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": 0, 2024-09-11T22:07:37.891 INFO:teuthology.orchestra.run.smithi053.stdout: "subtrees": 2, 2024-09-11T22:07:37.891 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse": 0, 2024-09-11T22:07:37.891 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_hit": 0, 2024-09-11T22:07:37.891 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_forward": 0, 2024-09-11T22:07:37.891 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_discover": 0, 2024-09-11T22:07:37.891 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_dir_fetch": 0, 2024-09-11T22:07:37.891 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_remote_ino": 0, 2024-09-11T22:07:37.891 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_lock": 0, 2024-09-11T22:07:37.891 INFO:teuthology.orchestra.run.smithi053.stdout: "load_cent": 0, 2024-09-11T22:07:37.892 INFO:teuthology.orchestra.run.smithi053.stdout: "q": 0, 2024-09-11T22:07:37.892 INFO:teuthology.orchestra.run.smithi053.stdout: "exported": 0, 2024-09-11T22:07:37.892 INFO:teuthology.orchestra.run.smithi053.stdout: "exported_inodes": 0, 2024-09-11T22:07:37.892 INFO:teuthology.orchestra.run.smithi053.stdout: "imported": 0, 2024-09-11T22:07:37.892 INFO:teuthology.orchestra.run.smithi053.stdout: "imported_inodes": 0, 2024-09-11T22:07:37.892 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_dir_fetch": 0, 2024-09-11T22:07:37.892 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_backtrace_fetch": 0, 2024-09-11T22:07:37.893 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_peer_discover": 0, 2024-09-11T22:07:37.893 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rfiles": 4, 2024-09-11T22:07:37.893 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rbytes": 234, 2024-09-11T22:07:37.893 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rsnaps": 7, 2024-09-11T22:07:37.893 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_backtrace_fetch": 0, 2024-09-11T22:07:37.893 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_set_tag": 0, 2024-09-11T22:07:37.894 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T22:07:37.894 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_inotable_repaired": 0, 2024-09-11T22:07:37.894 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dir_inodes": 0, 2024-09-11T22:07:37.894 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T22:07:37.894 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T22:07:37.894 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_file_inodes": 0, 2024-09-11T22:07:37.894 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_inode_file_caps": 0, 2024-09-11T22:07:37.895 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_revoke": 0, 2024-09-11T22:07:37.895 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_grant": 0, 2024-09-11T22:07:37.895 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T22:07:37.895 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T22:07:37.895 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T22:07:37.895 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_caps": 0, 2024-09-11T22:07:37.895 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_caps_dirty": 0, 2024-09-11T22:07:37.895 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_cap_release": 0, 2024-09-11T22:07:37.896 INFO:teuthology.orchestra.run.smithi053.stdout: "process_request_cap_release": 0 2024-09-11T22:07:37.896 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:37.896 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_cache": { 2024-09-11T22:07:37.896 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_update": 0, 2024-09-11T22:07:37.896 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_update_receipt": 0, 2024-09-11T22:07:37.896 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_try_discover": 0, 2024-09-11T22:07:37.896 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_send_discover": 0, 2024-09-11T22:07:37.896 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_handle_discover": 0, 2024-09-11T22:07:37.897 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays": 0, 2024-09-11T22:07:37.897 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays_delayed": 0, 2024-09-11T22:07:37.897 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays_enqueuing": 0, 2024-09-11T22:07:37.897 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_created": 0, 2024-09-11T22:07:37.897 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_enqueued": 0, 2024-09-11T22:07:37.897 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_reintegrated": 0, 2024-09-11T22:07:37.897 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_migrated": 0, 2024-09-11T22:07:37.897 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_processing": 0, 2024-09-11T22:07:37.898 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_enqueued": 0, 2024-09-11T22:07:37.898 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_prioritized": 0, 2024-09-11T22:07:37.898 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_started": 0, 2024-09-11T22:07:37.898 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_completed": 0, 2024-09-11T22:07:37.898 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_quiesce_path": 0, 2024-09-11T22:07:37.898 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_quiesce_inode": 0, 2024-09-11T22:07:37.898 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T22:07:37.898 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_exportdir": 0, 2024-09-11T22:07:37.899 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_flush": 0, 2024-09-11T22:07:37.899 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_fragmentdir": 0, 2024-09-11T22:07:37.899 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_fragstats": 0, 2024-09-11T22:07:37.899 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_inodestats": 0 2024-09-11T22:07:37.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:37.899 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_log": { 2024-09-11T22:07:37.899 INFO:teuthology.orchestra.run.smithi053.stdout: "evlrg": 0, 2024-09-11T22:07:37.899 INFO:teuthology.orchestra.run.smithi053.stdout: "evadd": 0, 2024-09-11T22:07:37.900 INFO:teuthology.orchestra.run.smithi053.stdout: "evex": 0, 2024-09-11T22:07:37.900 INFO:teuthology.orchestra.run.smithi053.stdout: "evtrm": 0, 2024-09-11T22:07:37.900 INFO:teuthology.orchestra.run.smithi053.stdout: "ev": 65031, 2024-09-11T22:07:37.900 INFO:teuthology.orchestra.run.smithi053.stdout: "evexg": 0, 2024-09-11T22:07:37.900 INFO:teuthology.orchestra.run.smithi053.stdout: "evexd": 0, 2024-09-11T22:07:37.901 INFO:teuthology.orchestra.run.smithi053.stdout: "segadd": 0, 2024-09-11T22:07:37.901 INFO:teuthology.orchestra.run.smithi053.stdout: "segex": 0, 2024-09-11T22:07:37.901 INFO:teuthology.orchestra.run.smithi053.stdout: "segtrm": 0, 2024-09-11T22:07:37.901 INFO:teuthology.orchestra.run.smithi053.stdout: "seg": 3, 2024-09-11T22:07:37.901 INFO:teuthology.orchestra.run.smithi053.stdout: "segmjr": 36, 2024-09-11T22:07:37.901 INFO:teuthology.orchestra.run.smithi053.stdout: "segexg": 0, 2024-09-11T22:07:37.901 INFO:teuthology.orchestra.run.smithi053.stdout: "segexd": 0, 2024-09-11T22:07:37.901 INFO:teuthology.orchestra.run.smithi053.stdout: "expos": 196897656, 2024-09-11T22:07:37.902 INFO:teuthology.orchestra.run.smithi053.stdout: "wrpos": 197222657, 2024-09-11T22:07:37.902 INFO:teuthology.orchestra.run.smithi053.stdout: "rdpos": 197214115, 2024-09-11T22:07:37.902 INFO:teuthology.orchestra.run.smithi053.stdout: "jlat": { 2024-09-11T22:07:37.902 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:37.902 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:37.902 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:37.902 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:37.903 INFO:teuthology.orchestra.run.smithi053.stdout: "replayed": 65031 2024-09-11T22:07:37.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:37.903 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_mem": { 2024-09-11T22:07:37.903 INFO:teuthology.orchestra.run.smithi053.stdout: "ino": 13457, 2024-09-11T22:07:37.903 INFO:teuthology.orchestra.run.smithi053.stdout: "ino+": 47747, 2024-09-11T22:07:37.903 INFO:teuthology.orchestra.run.smithi053.stdout: "ino-": 34290, 2024-09-11T22:07:37.903 INFO:teuthology.orchestra.run.smithi053.stdout: "dir": 541, 2024-09-11T22:07:37.903 INFO:teuthology.orchestra.run.smithi053.stdout: "dir+": 2123, 2024-09-11T22:07:37.904 INFO:teuthology.orchestra.run.smithi053.stdout: "dir-": 1582, 2024-09-11T22:07:37.904 INFO:teuthology.orchestra.run.smithi053.stdout: "dn": 30925, 2024-09-11T22:07:37.904 INFO:teuthology.orchestra.run.smithi053.stdout: "dn+": 60400, 2024-09-11T22:07:37.904 INFO:teuthology.orchestra.run.smithi053.stdout: "dn-": 29475, 2024-09-11T22:07:37.904 INFO:teuthology.orchestra.run.smithi053.stdout: "cap": 0, 2024-09-11T22:07:37.904 INFO:teuthology.orchestra.run.smithi053.stdout: "cap+": 0, 2024-09-11T22:07:37.904 INFO:teuthology.orchestra.run.smithi053.stdout: "cap-": 0, 2024-09-11T22:07:37.904 INFO:teuthology.orchestra.run.smithi053.stdout: "rss": 122336, 2024-09-11T22:07:37.905 INFO:teuthology.orchestra.run.smithi053.stdout: "heap": 215328 2024-09-11T22:07:37.905 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:37.905 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_server": { 2024-09-11T22:07:37.905 INFO:teuthology.orchestra.run.smithi053.stdout: "dispatch_client_request": 0, 2024-09-11T22:07:37.905 INFO:teuthology.orchestra.run.smithi053.stdout: "dispatch_server_request": 0, 2024-09-11T22:07:37.905 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_request": 0, 2024-09-11T22:07:37.905 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_session": 0, 2024-09-11T22:07:37.905 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_peer_request": 0, 2024-09-11T22:07:37.906 INFO:teuthology.orchestra.run.smithi053.stdout: "req_create_latency": { 2024-09-11T22:07:37.906 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:37.906 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:37.906 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:37.906 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:37.906 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getattr_latency": { 2024-09-11T22:07:37.906 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:37.906 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:37.907 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:37.907 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:37.907 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getfilelock_latency": { 2024-09-11T22:07:37.907 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:37.907 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:37.907 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:37.907 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:37.908 INFO:teuthology.orchestra.run.smithi053.stdout: "req_link_latency": { 2024-09-11T22:07:37.908 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:37.908 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:37.908 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:37.908 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:37.908 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookup_latency": { 2024-09-11T22:07:37.908 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:37.909 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:37.909 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:37.909 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:37.909 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookuphash_latency": { 2024-09-11T22:07:37.909 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:37.909 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:37.909 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:37.909 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:37.910 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupino_latency": { 2024-09-11T22:07:37.910 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:37.910 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:37.910 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:37.910 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:37.910 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupname_latency": { 2024-09-11T22:07:37.910 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:37.910 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:37.911 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:37.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:37.911 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupparent_latency": { 2024-09-11T22:07:37.911 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:37.911 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:37.911 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:37.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:37.911 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupsnap_latency": { 2024-09-11T22:07:37.912 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:37.912 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:37.912 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:37.912 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:37.912 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lssnap_latency": { 2024-09-11T22:07:37.912 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:37.912 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:37.913 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:37.913 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:37.914 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mkdir_latency": { 2024-09-11T22:07:37.914 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:37.914 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:37.914 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:37.914 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:37.914 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mknod_latency": { 2024-09-11T22:07:37.914 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:37.915 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:37.915 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:37.915 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:37.915 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mksnap_latency": { 2024-09-11T22:07:37.915 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:37.915 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:37.915 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:37.915 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:37.916 INFO:teuthology.orchestra.run.smithi053.stdout: "req_open_latency": { 2024-09-11T22:07:37.916 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:37.916 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:37.916 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:37.916 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:37.916 INFO:teuthology.orchestra.run.smithi053.stdout: "req_readdir_latency": { 2024-09-11T22:07:37.916 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:37.917 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:37.917 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:37.917 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:37.917 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rename_latency": { 2024-09-11T22:07:37.917 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:37.917 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:37.917 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:37.917 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:37.918 INFO:teuthology.orchestra.run.smithi053.stdout: "req_renamesnap_latency": { 2024-09-11T22:07:37.918 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:37.918 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:37.918 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:37.918 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:37.918 INFO:teuthology.orchestra.run.smithi053.stdout: "req_snapdiff_latency": { 2024-09-11T22:07:37.918 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:37.918 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:37.919 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:37.919 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:37.919 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmdir_latency": { 2024-09-11T22:07:37.919 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:37.919 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:37.919 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:37.919 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:37.920 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmsnap_latency": { 2024-09-11T22:07:37.920 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:37.920 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:37.920 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:37.920 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:37.920 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmxattr_latency": { 2024-09-11T22:07:37.920 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:37.920 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:37.921 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:37.921 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:37.921 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setattr_latency": { 2024-09-11T22:07:37.921 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:37.921 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:37.921 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:37.921 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:37.921 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setdirlayout_latency": { 2024-09-11T22:07:37.922 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:37.922 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:37.922 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:37.922 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:37.922 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setfilelock_latency": { 2024-09-11T22:07:37.922 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:37.922 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:37.923 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:37.923 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:37.923 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setlayout_latency": { 2024-09-11T22:07:37.923 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:37.923 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:37.923 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:37.923 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:37.923 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setxattr_latency": { 2024-09-11T22:07:37.924 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:37.924 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:37.924 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:37.924 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:37.924 INFO:teuthology.orchestra.run.smithi053.stdout: "req_symlink_latency": { 2024-09-11T22:07:37.924 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:37.924 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:37.925 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:37.925 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:37.925 INFO:teuthology.orchestra.run.smithi053.stdout: "req_unlink_latency": { 2024-09-11T22:07:37.925 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:37.925 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:37.925 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:37.925 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:37.925 INFO:teuthology.orchestra.run.smithi053.stdout: "cap_revoke_eviction": 0, 2024-09-11T22:07:37.926 INFO:teuthology.orchestra.run.smithi053.stdout: "cap_acquisition_throttle": 0, 2024-09-11T22:07:37.926 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getvxattr_latency": { 2024-09-11T22:07:37.926 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:37.926 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:37.926 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:37.926 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-11T22:07:37.926 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:37.926 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_sessions": { 2024-09-11T22:07:37.927 INFO:teuthology.orchestra.run.smithi053.stdout: "session_count": 2, 2024-09-11T22:07:37.927 INFO:teuthology.orchestra.run.smithi053.stdout: "session_add": 5, 2024-09-11T22:07:37.927 INFO:teuthology.orchestra.run.smithi053.stdout: "session_remove": 3, 2024-09-11T22:07:37.927 INFO:teuthology.orchestra.run.smithi053.stdout: "sessions_open": 4, 2024-09-11T22:07:37.927 INFO:teuthology.orchestra.run.smithi053.stdout: "sessions_stale": 0, 2024-09-11T22:07:37.927 INFO:teuthology.orchestra.run.smithi053.stdout: "total_load": 0, 2024-09-11T22:07:37.927 INFO:teuthology.orchestra.run.smithi053.stdout: "average_load": 0, 2024-09-11T22:07:37.927 INFO:teuthology.orchestra.run.smithi053.stdout: "avg_session_uptime": 3353, 2024-09-11T22:07:37.928 INFO:teuthology.orchestra.run.smithi053.stdout: "mdthresh_evicted": 0 2024-09-11T22:07:37.928 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:37.928 INFO:teuthology.orchestra.run.smithi053.stdout: "mempool": { 2024-09-11T22:07:37.928 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_bytes": 0, 2024-09-11T22:07:37.928 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_items": 0, 2024-09-11T22:07:37.928 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T22:07:37.928 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_items": 0, 2024-09-11T22:07:37.928 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T22:07:37.929 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_items": 0, 2024-09-11T22:07:37.929 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T22:07:37.929 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T22:07:37.929 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T22:07:37.929 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T22:07:37.929 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T22:07:37.929 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_items": 0, 2024-09-11T22:07:37.929 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T22:07:37.930 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T22:07:37.930 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_bytes": 0, 2024-09-11T22:07:37.930 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_items": 0, 2024-09-11T22:07:37.930 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_bytes": 0, 2024-09-11T22:07:37.930 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_items": 0, 2024-09-11T22:07:37.930 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T22:07:37.930 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T22:07:37.931 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T22:07:37.931 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T22:07:37.931 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T22:07:37.931 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_items": 0, 2024-09-11T22:07:37.931 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_bytes": 0, 2024-09-11T22:07:37.931 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_items": 0, 2024-09-11T22:07:37.931 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T22:07:37.931 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T22:07:37.931 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_bytes": 0, 2024-09-11T22:07:37.932 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_items": 0, 2024-09-11T22:07:37.932 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_bytes": 0, 2024-09-11T22:07:37.932 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_items": 0, 2024-09-11T22:07:37.932 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T22:07:37.932 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_items": 0, 2024-09-11T22:07:37.932 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T22:07:37.933 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_items": 0, 2024-09-11T22:07:37.933 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_bytes": 736969, 2024-09-11T22:07:37.933 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_items": 13543, 2024-09-11T22:07:37.933 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_bytes": 0, 2024-09-11T22:07:37.933 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_items": 0, 2024-09-11T22:07:37.933 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_bytes": 0, 2024-09-11T22:07:37.933 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_items": 0, 2024-09-11T22:07:37.934 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_bytes": 0, 2024-09-11T22:07:37.934 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_items": 0, 2024-09-11T22:07:37.934 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_bytes": 0, 2024-09-11T22:07:37.934 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_items": 0, 2024-09-11T22:07:37.934 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_bytes": 8392, 2024-09-11T22:07:37.934 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_items": 184, 2024-09-11T22:07:37.934 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T22:07:37.935 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_items": 0, 2024-09-11T22:07:37.935 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_bytes": 0, 2024-09-11T22:07:37.935 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_items": 0, 2024-09-11T22:07:37.935 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_bytes": 53165651, 2024-09-11T22:07:37.935 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_items": 418677, 2024-09-11T22:07:37.935 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_bytes": 0, 2024-09-11T22:07:37.935 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_items": 0, 2024-09-11T22:07:37.935 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_bytes": 0, 2024-09-11T22:07:37.936 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_items": 0 2024-09-11T22:07:37.936 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:37.936 INFO:teuthology.orchestra.run.smithi053.stdout: "objecter": { 2024-09-11T22:07:37.936 INFO:teuthology.orchestra.run.smithi053.stdout: "op_active": 0, 2024-09-11T22:07:37.936 INFO:teuthology.orchestra.run.smithi053.stdout: "op_laggy": 0, 2024-09-11T22:07:37.936 INFO:teuthology.orchestra.run.smithi053.stdout: "op_send": 2593, 2024-09-11T22:07:37.936 INFO:teuthology.orchestra.run.smithi053.stdout: "op_send_bytes": 26, 2024-09-11T22:07:37.937 INFO:teuthology.orchestra.run.smithi053.stdout: "op_resend": 4, 2024-09-11T22:07:37.937 INFO:teuthology.orchestra.run.smithi053.stdout: "op_reply": 2593, 2024-09-11T22:07:37.937 INFO:teuthology.orchestra.run.smithi053.stdout: "op_latency": { 2024-09-11T22:07:37.937 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 2593, 2024-09-11T22:07:37.937 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 5.647882357, 2024-09-11T22:07:37.937 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.002178126 2024-09-11T22:07:37.937 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:37.937 INFO:teuthology.orchestra.run.smithi053.stdout: "op_inflight": 0, 2024-09-11T22:07:37.938 INFO:teuthology.orchestra.run.smithi053.stdout: "oplen_avg": { 2024-09-11T22:07:37.938 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 2593, 2024-09-11T22:07:37.938 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 2594 2024-09-11T22:07:37.938 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:37.938 INFO:teuthology.orchestra.run.smithi053.stdout: "op": 2593, 2024-09-11T22:07:37.938 INFO:teuthology.orchestra.run.smithi053.stdout: "op_r": 2593, 2024-09-11T22:07:37.938 INFO:teuthology.orchestra.run.smithi053.stdout: "op_w": 0, 2024-09-11T22:07:37.938 INFO:teuthology.orchestra.run.smithi053.stdout: "op_rmw": 0, 2024-09-11T22:07:37.939 INFO:teuthology.orchestra.run.smithi053.stdout: "op_pg": 0, 2024-09-11T22:07:37.939 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_stat": 1603, 2024-09-11T22:07:37.939 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_create": 0, 2024-09-11T22:07:37.939 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_read": 987, 2024-09-11T22:07:37.939 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_write": 0, 2024-09-11T22:07:37.939 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_writefull": 0, 2024-09-11T22:07:37.939 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_writesame": 0, 2024-09-11T22:07:37.940 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_append": 0, 2024-09-11T22:07:37.940 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_zero": 0, 2024-09-11T22:07:37.940 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_truncate": 0, 2024-09-11T22:07:37.940 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_delete": 0, 2024-09-11T22:07:37.940 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_mapext": 0, 2024-09-11T22:07:37.940 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_sparse_read": 0, 2024-09-11T22:07:37.940 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_clonerange": 0, 2024-09-11T22:07:37.940 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_getxattr": 2, 2024-09-11T22:07:37.941 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_setxattr": 0, 2024-09-11T22:07:37.941 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_cmpxattr": 0, 2024-09-11T22:07:37.941 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_rmxattr": 0, 2024-09-11T22:07:37.941 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_resetxattrs": 0, 2024-09-11T22:07:37.941 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_call": 0, 2024-09-11T22:07:37.941 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_watch": 0, 2024-09-11T22:07:37.941 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_notify": 0, 2024-09-11T22:07:37.942 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T22:07:37.942 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_pgls": 0, 2024-09-11T22:07:37.942 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_pgls_filter": 0, 2024-09-11T22:07:37.942 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_other": 0, 2024-09-11T22:07:37.942 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_active": 0, 2024-09-11T22:07:37.942 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_send": 0, 2024-09-11T22:07:37.942 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_resend": 0, 2024-09-11T22:07:37.943 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_ping": 0, 2024-09-11T22:07:37.943 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_active": 0, 2024-09-11T22:07:37.943 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_send": 0, 2024-09-11T22:07:37.943 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_resend": 0, 2024-09-11T22:07:37.943 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_active": 0, 2024-09-11T22:07:37.943 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_send": 0, 2024-09-11T22:07:37.943 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_resend": 0, 2024-09-11T22:07:37.944 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_active": 0, 2024-09-11T22:07:37.944 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_send": 0, 2024-09-11T22:07:37.944 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_resend": 0, 2024-09-11T22:07:37.944 INFO:teuthology.orchestra.run.smithi053.stdout: "command_active": 0, 2024-09-11T22:07:37.944 INFO:teuthology.orchestra.run.smithi053.stdout: "command_send": 0, 2024-09-11T22:07:37.944 INFO:teuthology.orchestra.run.smithi053.stdout: "command_resend": 0, 2024-09-11T22:07:37.944 INFO:teuthology.orchestra.run.smithi053.stdout: "map_epoch": 95, 2024-09-11T22:07:37.944 INFO:teuthology.orchestra.run.smithi053.stdout: "map_full": 0, 2024-09-11T22:07:37.945 INFO:teuthology.orchestra.run.smithi053.stdout: "map_inc": 11, 2024-09-11T22:07:37.945 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_sessions": 11, 2024-09-11T22:07:37.945 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_session_open": 11, 2024-09-11T22:07:37.945 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_session_close": 0, 2024-09-11T22:07:37.945 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_laggy": 0, 2024-09-11T22:07:37.945 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_wr": 0, 2024-09-11T22:07:37.945 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_rd": 2, 2024-09-11T22:07:37.945 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_del": 0 2024-09-11T22:07:37.946 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:37.946 INFO:teuthology.orchestra.run.smithi053.stdout: "oft": { 2024-09-11T22:07:37.946 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_objs": 0, 2024-09-11T22:07:37.946 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_kv_pairs": 0, 2024-09-11T22:07:37.946 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_updates": 0, 2024-09-11T22:07:37.946 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_removes": 0 2024-09-11T22:07:37.946 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:37.947 INFO:teuthology.orchestra.run.smithi053.stdout: "osd-slow-ops": { 2024-09-11T22:07:37.947 INFO:teuthology.orchestra.run.smithi053.stdout: "slow_ops_count": 0 2024-09-11T22:07:37.947 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:37.947 INFO:teuthology.orchestra.run.smithi053.stdout: "purge_queue": { 2024-09-11T22:07:37.947 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_ops": 0, 2024-09-11T22:07:37.947 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_ops_high_water": 0, 2024-09-11T22:07:37.947 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing": 0, 2024-09-11T22:07:37.947 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_high_water": 0, 2024-09-11T22:07:37.948 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executed_ops": 0, 2024-09-11T22:07:37.948 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executed": 0, 2024-09-11T22:07:37.948 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_item_in_journal": 0 2024-09-11T22:07:37.948 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:37.948 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T22:07:37.948 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-09-11T22:07:37.948 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 104857600, 2024-09-11T22:07:37.948 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-09-11T22:07:37.949 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 2853, 2024-09-11T22:07:37.949 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 195297202, 2024-09-11T22:07:37.949 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-09-11T22:07:37.949 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 2853, 2024-09-11T22:07:37.949 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-09-11T22:07:37.949 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-09-11T22:07:37.949 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 2853, 2024-09-11T22:07:37.950 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 195297202, 2024-09-11T22:07:37.950 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-09-11T22:07:37.950 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:37.950 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:37.950 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:37.950 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-11T22:07:37.950 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:37.950 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-objecter_bytes": { 2024-09-11T22:07:37.950 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-09-11T22:07:37.951 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 104857600, 2024-09-11T22:07:37.951 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-09-11T22:07:37.951 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-09-11T22:07:37.951 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-09-11T22:07:37.951 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-09-11T22:07:37.951 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-09-11T22:07:37.952 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 2593, 2024-09-11T22:07:37.952 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 193087523, 2024-09-11T22:07:37.952 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 2593, 2024-09-11T22:07:37.952 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 193087523, 2024-09-11T22:07:37.952 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-09-11T22:07:37.952 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:37.953 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:37.953 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:37.953 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-11T22:07:37.953 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:37.953 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-objecter_ops": { 2024-09-11T22:07:37.953 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-09-11T22:07:37.953 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 1024, 2024-09-11T22:07:37.953 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-09-11T22:07:37.953 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-09-11T22:07:37.954 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-09-11T22:07:37.954 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-09-11T22:07:37.954 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-09-11T22:07:37.954 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 2593, 2024-09-11T22:07:37.954 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 2593, 2024-09-11T22:07:37.954 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 2593, 2024-09-11T22:07:37.954 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 2593, 2024-09-11T22:07:37.954 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-09-11T22:07:37.955 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:37.955 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:37.955 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:37.955 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-11T22:07:37.955 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:37.955 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-write_buf_throttle": { 2024-09-11T22:07:37.955 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-09-11T22:07:37.956 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 3758096384, 2024-09-11T22:07:37.956 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-09-11T22:07:37.956 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-09-11T22:07:37.956 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-09-11T22:07:37.956 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-09-11T22:07:37.956 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-09-11T22:07:37.956 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-09-11T22:07:37.956 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-09-11T22:07:37.957 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 0, 2024-09-11T22:07:37.957 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 0, 2024-09-11T22:07:37.957 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-09-11T22:07:37.957 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:37.957 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:37.957 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:37.957 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-11T22:07:37.958 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:37.958 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-write_buf_throttle-0x5609af5806e0": { 2024-09-11T22:07:37.958 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-09-11T22:07:37.958 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 3758096384, 2024-09-11T22:07:37.958 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-09-11T22:07:37.958 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-09-11T22:07:37.958 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-09-11T22:07:37.958 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-09-11T22:07:37.958 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-09-11T22:07:37.959 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-09-11T22:07:37.959 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-09-11T22:07:37.959 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 0, 2024-09-11T22:07:37.959 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 0, 2024-09-11T22:07:37.959 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-09-11T22:07:37.959 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:37.959 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:37.959 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:37.960 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-11T22:07:37.960 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-11T22:07:37.960 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-11T22:07:38.606 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-11T22:07:38.606 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-11T22:07:38.606 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-09-11T22:07:38.607 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-09-11T22:07:38.607 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=7 2024-09-11T22:07:38.607 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-09-11T22:07:38.607 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-09-11T22:07:38.607 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2024-09-11T22:07:38.608 DEBUG:tasks.check_counter:Getting stats from h 2024-09-11T22:07:38.608 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-11T22:07:38.871 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T22:07:39.252 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:39 smithi053 ceph-mon[34908]: pgmap v323: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:07:39.252 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:39 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:07:39.263 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:39 smithi001 ceph-mon[30935]: pgmap v323: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:07:39.263 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:39 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:07:39.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:39 smithi161 ceph-mon[35615]: pgmap v323: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:07:39.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:39 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:07:39.494 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-11T22:07:39.495 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T22:07:39.495 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 81893, 2024-09-11T22:07:39.495 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 145389, 2024-09-11T22:07:39.495 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 41196845, 2024-09-11T22:07:39.495 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 116898291, 2024-09-11T22:07:39.495 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 27, 2024-09-11T22:07:39.496 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 5, 2024-09-11T22:07:39.496 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 5.385779625, 2024-09-11T22:07:39.496 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 2.346715555, 2024-09-11T22:07:39.496 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 24.164328423, 2024-09-11T22:07:39.496 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.268595534, 2024-09-11T22:07:39.496 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:07:39.496 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 145389, 2024-09-11T22:07:39.497 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 7.244409293, 2024-09-11T22:07:39.497 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000049827 2024-09-11T22:07:39.497 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:39.497 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:07:39.497 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 180130, 2024-09-11T22:07:39.497 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.035427068, 2024-09-11T22:07:39.497 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000196 2024-09-11T22:07:39.497 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:39.498 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-11T22:07:39.498 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 0 2024-09-11T22:07:39.498 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:39.498 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T22:07:39.498 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 36544, 2024-09-11T22:07:39.498 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 36945, 2024-09-11T22:07:39.498 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 24189914, 2024-09-11T22:07:39.498 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 90222961, 2024-09-11T22:07:39.499 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 15, 2024-09-11T22:07:39.499 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 6, 2024-09-11T22:07:39.499 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 2.451193967, 2024-09-11T22:07:39.499 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.897600609, 2024-09-11T22:07:39.499 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 1.069918362, 2024-09-11T22:07:39.499 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.536973728, 2024-09-11T22:07:39.499 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:07:39.499 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 36942, 2024-09-11T22:07:39.500 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 2.258856512, 2024-09-11T22:07:39.500 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000061146 2024-09-11T22:07:39.500 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:39.500 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:07:39.500 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 32887, 2024-09-11T22:07:39.500 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.006066073, 2024-09-11T22:07:39.500 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000184 2024-09-11T22:07:39.500 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:39.501 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 1248, 2024-09-11T22:07:39.501 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 438992 2024-09-11T22:07:39.501 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:39.501 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T22:07:39.501 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 30123, 2024-09-11T22:07:39.501 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 45389, 2024-09-11T22:07:39.501 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 26926198, 2024-09-11T22:07:39.502 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 38141441, 2024-09-11T22:07:39.502 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 16, 2024-09-11T22:07:39.502 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 7, 2024-09-11T22:07:39.502 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 2.867375013, 2024-09-11T22:07:39.502 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 1.433638891, 2024-09-11T22:07:39.502 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 1.118315544, 2024-09-11T22:07:39.502 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.358297584, 2024-09-11T22:07:39.502 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:07:39.503 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 45387, 2024-09-11T22:07:39.503 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 2.545515876, 2024-09-11T22:07:39.503 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000056084 2024-09-11T22:07:39.503 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:39.503 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:07:39.503 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 45046, 2024-09-11T22:07:39.503 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.011171649, 2024-09-11T22:07:39.503 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000248 2024-09-11T22:07:39.504 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:39.504 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 1787904, 2024-09-11T22:07:39.504 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 200800 2024-09-11T22:07:39.504 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:39.504 INFO:teuthology.orchestra.run.smithi053.stdout: "cct": { 2024-09-11T22:07:39.504 INFO:teuthology.orchestra.run.smithi053.stdout: "total_workers": 1, 2024-09-11T22:07:39.504 INFO:teuthology.orchestra.run.smithi053.stdout: "unhealthy_workers": 0 2024-09-11T22:07:39.504 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:39.505 INFO:teuthology.orchestra.run.smithi053.stdout: "finisher-MDSRank": { 2024-09-11T22:07:39.505 INFO:teuthology.orchestra.run.smithi053.stdout: "queue_len": 0, 2024-09-11T22:07:39.505 INFO:teuthology.orchestra.run.smithi053.stdout: "complete_latency": { 2024-09-11T22:07:39.505 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 17431, 2024-09-11T22:07:39.505 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 43.994837208, 2024-09-11T22:07:39.505 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.002523942 2024-09-11T22:07:39.506 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-11T22:07:39.506 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:39.506 INFO:teuthology.orchestra.run.smithi053.stdout: "finisher-PurgeQueue": { 2024-09-11T22:07:39.506 INFO:teuthology.orchestra.run.smithi053.stdout: "queue_len": 0, 2024-09-11T22:07:39.506 INFO:teuthology.orchestra.run.smithi053.stdout: "complete_latency": { 2024-09-11T22:07:39.506 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 24166, 2024-09-11T22:07:39.506 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 6.978179762, 2024-09-11T22:07:39.506 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000288760 2024-09-11T22:07:39.507 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-11T22:07:39.507 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:39.507 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": { 2024-09-11T22:07:39.507 INFO:teuthology.orchestra.run.smithi053.stdout: "request": 35231, 2024-09-11T22:07:39.507 INFO:teuthology.orchestra.run.smithi053.stdout: "reply": 33462, 2024-09-11T22:07:39.507 INFO:teuthology.orchestra.run.smithi053.stdout: "reply_latency": { 2024-09-11T22:07:39.507 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 33462, 2024-09-11T22:07:39.508 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 189.418946341, 2024-09-11T22:07:39.508 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.005660718 2024-09-11T22:07:39.508 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:39.508 INFO:teuthology.orchestra.run.smithi053.stdout: "slow_reply": 0, 2024-09-11T22:07:39.508 INFO:teuthology.orchestra.run.smithi053.stdout: "forward": 1769, 2024-09-11T22:07:39.509 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_fetch_complete": 876, 2024-09-11T22:07:39.509 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_fetch_keys": 0, 2024-09-11T22:07:39.509 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_commit": 507, 2024-09-11T22:07:39.509 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_split": 62, 2024-09-11T22:07:39.509 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_merge": 0, 2024-09-11T22:07:39.509 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes": 23827, 2024-09-11T22:07:39.509 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_top": 1959, 2024-09-11T22:07:39.509 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_bottom": 5557, 2024-09-11T22:07:39.510 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_pin_tail": 16311, 2024-09-11T22:07:39.510 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_pinned": 21026, 2024-09-11T22:07:39.510 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_expired": 33938, 2024-09-11T22:07:39.510 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_with_caps": 0, 2024-09-11T22:07:39.510 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": 0, 2024-09-11T22:07:39.510 INFO:teuthology.orchestra.run.smithi053.stdout: "subtrees": 6, 2024-09-11T22:07:39.510 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse": 67127, 2024-09-11T22:07:39.510 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_hit": 48578, 2024-09-11T22:07:39.511 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_forward": 1669, 2024-09-11T22:07:39.511 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_discover": 43, 2024-09-11T22:07:39.511 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_dir_fetch": 51, 2024-09-11T22:07:39.511 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_remote_ino": 0, 2024-09-11T22:07:39.511 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_lock": 1, 2024-09-11T22:07:39.511 INFO:teuthology.orchestra.run.smithi053.stdout: "load_cent": 0, 2024-09-11T22:07:39.511 INFO:teuthology.orchestra.run.smithi053.stdout: "q": 0, 2024-09-11T22:07:39.511 INFO:teuthology.orchestra.run.smithi053.stdout: "exported": 526, 2024-09-11T22:07:39.512 INFO:teuthology.orchestra.run.smithi053.stdout: "exported_inodes": 26360, 2024-09-11T22:07:39.512 INFO:teuthology.orchestra.run.smithi053.stdout: "imported": 500, 2024-09-11T22:07:39.512 INFO:teuthology.orchestra.run.smithi053.stdout: "imported_inodes": 27073, 2024-09-11T22:07:39.512 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_dir_fetch": 0, 2024-09-11T22:07:39.512 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_backtrace_fetch": 18122, 2024-09-11T22:07:39.512 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_peer_discover": 0, 2024-09-11T22:07:39.512 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rfiles": 4, 2024-09-11T22:07:39.512 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rbytes": 234, 2024-09-11T22:07:39.513 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rsnaps": 7, 2024-09-11T22:07:39.513 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_backtrace_fetch": 18122, 2024-09-11T22:07:39.513 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_set_tag": 0, 2024-09-11T22:07:39.513 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T22:07:39.513 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_inotable_repaired": 0, 2024-09-11T22:07:39.513 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dir_inodes": 4, 2024-09-11T22:07:39.513 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T22:07:39.514 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dirfrag_rstats": 4, 2024-09-11T22:07:39.514 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_file_inodes": 18118, 2024-09-11T22:07:39.514 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_inode_file_caps": 6, 2024-09-11T22:07:39.514 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_revoke": 9573, 2024-09-11T22:07:39.514 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_grant": 20847, 2024-09-11T22:07:39.514 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T22:07:39.514 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T22:07:39.514 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T22:07:39.515 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_caps": 17658, 2024-09-11T22:07:39.515 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_caps_dirty": 14724, 2024-09-11T22:07:39.515 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_cap_release": 502, 2024-09-11T22:07:39.515 INFO:teuthology.orchestra.run.smithi053.stdout: "process_request_cap_release": 27811 2024-09-11T22:07:39.515 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:39.515 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_cache": { 2024-09-11T22:07:39.515 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_update": 1983, 2024-09-11T22:07:39.516 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_update_receipt": 3980, 2024-09-11T22:07:39.516 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_try_discover": 1994, 2024-09-11T22:07:39.516 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_send_discover": 3072, 2024-09-11T22:07:39.516 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_handle_discover": 2357, 2024-09-11T22:07:39.516 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays": 1, 2024-09-11T22:07:39.516 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays_delayed": 0, 2024-09-11T22:07:39.516 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays_enqueuing": 0, 2024-09-11T22:07:39.516 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_created": 13495, 2024-09-11T22:07:39.517 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_enqueued": 13494, 2024-09-11T22:07:39.517 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_reintegrated": 0, 2024-09-11T22:07:39.517 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_migrated": 0, 2024-09-11T22:07:39.517 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_processing": 0, 2024-09-11T22:07:39.517 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_enqueued": 0, 2024-09-11T22:07:39.517 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_prioritized": 0, 2024-09-11T22:07:39.517 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_started": 0, 2024-09-11T22:07:39.518 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_completed": 0, 2024-09-11T22:07:39.518 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_quiesce_path": 8, 2024-09-11T22:07:39.518 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_quiesce_inode": 18816, 2024-09-11T22:07:39.518 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T22:07:39.518 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_exportdir": 526, 2024-09-11T22:07:39.518 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_flush": 0, 2024-09-11T22:07:39.518 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_fragmentdir": 63, 2024-09-11T22:07:39.518 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_fragstats": 0, 2024-09-11T22:07:39.519 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_inodestats": 0 2024-09-11T22:07:39.519 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:39.519 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_log": { 2024-09-11T22:07:39.519 INFO:teuthology.orchestra.run.smithi053.stdout: "evlrg": 4, 2024-09-11T22:07:39.519 INFO:teuthology.orchestra.run.smithi053.stdout: "evadd": 60090, 2024-09-11T22:07:39.519 INFO:teuthology.orchestra.run.smithi053.stdout: "evex": 0, 2024-09-11T22:07:39.519 INFO:teuthology.orchestra.run.smithi053.stdout: "evtrm": 0, 2024-09-11T22:07:39.520 INFO:teuthology.orchestra.run.smithi053.stdout: "ev": 60090, 2024-09-11T22:07:39.520 INFO:teuthology.orchestra.run.smithi053.stdout: "evexg": 0, 2024-09-11T22:07:39.520 INFO:teuthology.orchestra.run.smithi053.stdout: "evexd": 0, 2024-09-11T22:07:39.520 INFO:teuthology.orchestra.run.smithi053.stdout: "segadd": 77, 2024-09-11T22:07:39.520 INFO:teuthology.orchestra.run.smithi053.stdout: "segex": 0, 2024-09-11T22:07:39.520 INFO:teuthology.orchestra.run.smithi053.stdout: "segtrm": 0, 2024-09-11T22:07:39.520 INFO:teuthology.orchestra.run.smithi053.stdout: "seg": 77, 2024-09-11T22:07:39.520 INFO:teuthology.orchestra.run.smithi053.stdout: "segmjr": 6, 2024-09-11T22:07:39.521 INFO:teuthology.orchestra.run.smithi053.stdout: "segexg": 0, 2024-09-11T22:07:39.521 INFO:teuthology.orchestra.run.smithi053.stdout: "segexd": 0, 2024-09-11T22:07:39.521 INFO:teuthology.orchestra.run.smithi053.stdout: "expos": 4194304, 2024-09-11T22:07:39.521 INFO:teuthology.orchestra.run.smithi053.stdout: "wrpos": 136465521, 2024-09-11T22:07:39.521 INFO:teuthology.orchestra.run.smithi053.stdout: "rdpos": 0, 2024-09-11T22:07:39.521 INFO:teuthology.orchestra.run.smithi053.stdout: "jlat": { 2024-09-11T22:07:39.521 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 9235, 2024-09-11T22:07:39.522 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 5247.431812365, 2024-09-11T22:07:39.522 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.568211349 2024-09-11T22:07:39.522 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:39.522 INFO:teuthology.orchestra.run.smithi053.stdout: "replayed": 0 2024-09-11T22:07:39.522 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:39.522 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_mem": { 2024-09-11T22:07:39.522 INFO:teuthology.orchestra.run.smithi053.stdout: "ino": 13832, 2024-09-11T22:07:39.522 INFO:teuthology.orchestra.run.smithi053.stdout: "ino+": 51104, 2024-09-11T22:07:39.523 INFO:teuthology.orchestra.run.smithi053.stdout: "ino-": 37272, 2024-09-11T22:07:39.523 INFO:teuthology.orchestra.run.smithi053.stdout: "dir": 540, 2024-09-11T22:07:39.523 INFO:teuthology.orchestra.run.smithi053.stdout: "dir+": 2675, 2024-09-11T22:07:39.523 INFO:teuthology.orchestra.run.smithi053.stdout: "dir-": 2135, 2024-09-11T22:07:39.523 INFO:teuthology.orchestra.run.smithi053.stdout: "dn": 23827, 2024-09-11T22:07:39.523 INFO:teuthology.orchestra.run.smithi053.stdout: "dn+": 71259, 2024-09-11T22:07:39.523 INFO:teuthology.orchestra.run.smithi053.stdout: "dn-": 47432, 2024-09-11T22:07:39.524 INFO:teuthology.orchestra.run.smithi053.stdout: "cap": 0, 2024-09-11T22:07:39.524 INFO:teuthology.orchestra.run.smithi053.stdout: "cap+": 25896, 2024-09-11T22:07:39.524 INFO:teuthology.orchestra.run.smithi053.stdout: "cap-": 25896, 2024-09-11T22:07:39.524 INFO:teuthology.orchestra.run.smithi053.stdout: "rss": 189228, 2024-09-11T22:07:39.524 INFO:teuthology.orchestra.run.smithi053.stdout: "heap": 223520 2024-09-11T22:07:39.524 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:39.524 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_server": { 2024-09-11T22:07:39.524 INFO:teuthology.orchestra.run.smithi053.stdout: "dispatch_client_request": 55387, 2024-09-11T22:07:39.525 INFO:teuthology.orchestra.run.smithi053.stdout: "dispatch_server_request": 4140, 2024-09-11T22:07:39.525 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_request": 35231, 2024-09-11T22:07:39.525 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_session": 21, 2024-09-11T22:07:39.525 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_peer_request": 23598, 2024-09-11T22:07:39.525 INFO:teuthology.orchestra.run.smithi053.stdout: "req_create_latency": { 2024-09-11T22:07:39.525 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 5888, 2024-09-11T22:07:39.525 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 117.467924582, 2024-09-11T22:07:39.525 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.019950394 2024-09-11T22:07:39.526 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:39.526 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getattr_latency": { 2024-09-11T22:07:39.526 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 1, 2024-09-11T22:07:39.526 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.071812931, 2024-09-11T22:07:39.526 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.071812931 2024-09-11T22:07:39.526 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:39.526 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getfilelock_latency": { 2024-09-11T22:07:39.526 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:39.527 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:39.527 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:39.527 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:39.527 INFO:teuthology.orchestra.run.smithi053.stdout: "req_link_latency": { 2024-09-11T22:07:39.527 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:39.527 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:39.527 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:39.527 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:39.528 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookup_latency": { 2024-09-11T22:07:39.528 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 13012, 2024-09-11T22:07:39.528 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 16.667129761, 2024-09-11T22:07:39.528 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.001280904 2024-09-11T22:07:39.528 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:39.528 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookuphash_latency": { 2024-09-11T22:07:39.528 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:39.528 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:39.529 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:39.529 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:39.529 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupino_latency": { 2024-09-11T22:07:39.529 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:39.529 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:39.529 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:39.529 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:39.530 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupname_latency": { 2024-09-11T22:07:39.530 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:39.530 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:39.530 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:39.530 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:39.530 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupparent_latency": { 2024-09-11T22:07:39.530 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:39.530 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:39.531 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:39.531 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:39.531 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupsnap_latency": { 2024-09-11T22:07:39.531 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:39.532 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:39.532 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:39.532 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:39.532 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lssnap_latency": { 2024-09-11T22:07:39.532 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:39.532 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:39.533 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:39.533 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:39.533 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mkdir_latency": { 2024-09-11T22:07:39.533 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:39.533 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:39.533 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:39.533 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:39.533 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mknod_latency": { 2024-09-11T22:07:39.534 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:39.534 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:39.534 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:39.534 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:39.534 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mksnap_latency": { 2024-09-11T22:07:39.534 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:39.534 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:39.534 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:39.535 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:39.535 INFO:teuthology.orchestra.run.smithi053.stdout: "req_open_latency": { 2024-09-11T22:07:39.535 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:39.535 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:39.535 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:39.535 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:39.535 INFO:teuthology.orchestra.run.smithi053.stdout: "req_readdir_latency": { 2024-09-11T22:07:39.536 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 1066, 2024-09-11T22:07:39.536 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 1.047128506, 2024-09-11T22:07:39.536 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000982296 2024-09-11T22:07:39.536 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:39.536 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rename_latency": { 2024-09-11T22:07:39.536 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:39.536 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:39.537 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:39.537 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:39.537 INFO:teuthology.orchestra.run.smithi053.stdout: "req_renamesnap_latency": { 2024-09-11T22:07:39.537 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:39.537 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:39.537 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:39.537 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:39.537 INFO:teuthology.orchestra.run.smithi053.stdout: "req_snapdiff_latency": { 2024-09-11T22:07:39.538 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:39.538 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:39.538 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:39.538 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:39.538 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmdir_latency": { 2024-09-11T22:07:39.538 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 1, 2024-09-11T22:07:39.538 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.092915414, 2024-09-11T22:07:39.538 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.092915414 2024-09-11T22:07:39.539 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:39.539 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmsnap_latency": { 2024-09-11T22:07:39.539 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:39.539 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:39.539 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:39.539 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:39.539 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmxattr_latency": { 2024-09-11T22:07:39.540 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:39.540 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:39.540 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:39.540 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:39.540 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setattr_latency": { 2024-09-11T22:07:39.540 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:39.540 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:39.540 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:39.541 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:39.541 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setdirlayout_latency": { 2024-09-11T22:07:39.541 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:39.541 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:39.541 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:39.541 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:39.541 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setfilelock_latency": { 2024-09-11T22:07:39.541 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:39.542 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:39.542 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:39.542 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:39.542 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setlayout_latency": { 2024-09-11T22:07:39.542 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:39.542 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:39.542 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:39.543 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:39.543 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setxattr_latency": { 2024-09-11T22:07:39.543 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:39.543 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:39.543 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:39.543 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:39.543 INFO:teuthology.orchestra.run.smithi053.stdout: "req_symlink_latency": { 2024-09-11T22:07:39.543 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:39.544 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:39.544 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:39.544 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:39.544 INFO:teuthology.orchestra.run.smithi053.stdout: "req_unlink_latency": { 2024-09-11T22:07:39.544 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 13494, 2024-09-11T22:07:39.544 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 54.072035147, 2024-09-11T22:07:39.544 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.004007116 2024-09-11T22:07:39.544 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:39.545 INFO:teuthology.orchestra.run.smithi053.stdout: "cap_revoke_eviction": 0, 2024-09-11T22:07:39.545 INFO:teuthology.orchestra.run.smithi053.stdout: "cap_acquisition_throttle": 0, 2024-09-11T22:07:39.545 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getvxattr_latency": { 2024-09-11T22:07:39.545 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:39.545 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:39.545 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:39.545 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-11T22:07:39.545 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:39.546 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_sessions": { 2024-09-11T22:07:39.546 INFO:teuthology.orchestra.run.smithi053.stdout: "session_count": 2, 2024-09-11T22:07:39.546 INFO:teuthology.orchestra.run.smithi053.stdout: "session_add": 3, 2024-09-11T22:07:39.546 INFO:teuthology.orchestra.run.smithi053.stdout: "session_remove": 1, 2024-09-11T22:07:39.546 INFO:teuthology.orchestra.run.smithi053.stdout: "sessions_open": 2, 2024-09-11T22:07:39.546 INFO:teuthology.orchestra.run.smithi053.stdout: "sessions_stale": 0, 2024-09-11T22:07:39.546 INFO:teuthology.orchestra.run.smithi053.stdout: "total_load": 16166, 2024-09-11T22:07:39.546 INFO:teuthology.orchestra.run.smithi053.stdout: "average_load": 8083, 2024-09-11T22:07:39.546 INFO:teuthology.orchestra.run.smithi053.stdout: "avg_session_uptime": 603, 2024-09-11T22:07:39.547 INFO:teuthology.orchestra.run.smithi053.stdout: "mdthresh_evicted": 0 2024-09-11T22:07:39.547 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:39.547 INFO:teuthology.orchestra.run.smithi053.stdout: "mempool": { 2024-09-11T22:07:39.547 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_bytes": 0, 2024-09-11T22:07:39.547 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_items": 0, 2024-09-11T22:07:39.547 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T22:07:39.547 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_items": 0, 2024-09-11T22:07:39.547 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T22:07:39.547 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_items": 0, 2024-09-11T22:07:39.548 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T22:07:39.548 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T22:07:39.548 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T22:07:39.548 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T22:07:39.548 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T22:07:39.548 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_items": 0, 2024-09-11T22:07:39.548 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T22:07:39.549 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T22:07:39.549 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_bytes": 0, 2024-09-11T22:07:39.549 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_items": 0, 2024-09-11T22:07:39.549 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_bytes": 0, 2024-09-11T22:07:39.549 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_items": 0, 2024-09-11T22:07:39.549 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T22:07:39.549 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T22:07:39.549 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T22:07:39.550 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T22:07:39.550 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T22:07:39.550 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_items": 0, 2024-09-11T22:07:39.550 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_bytes": 0, 2024-09-11T22:07:39.550 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_items": 0, 2024-09-11T22:07:39.550 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T22:07:39.550 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T22:07:39.550 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_bytes": 0, 2024-09-11T22:07:39.551 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_items": 0, 2024-09-11T22:07:39.551 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_bytes": 0, 2024-09-11T22:07:39.551 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_items": 0, 2024-09-11T22:07:39.551 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T22:07:39.551 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_items": 0, 2024-09-11T22:07:39.551 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T22:07:39.551 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_items": 0, 2024-09-11T22:07:39.551 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_bytes": 826909, 2024-09-11T22:07:39.552 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_items": 13817, 2024-09-11T22:07:39.552 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_bytes": 88, 2024-09-11T22:07:39.552 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_items": 1, 2024-09-11T22:07:39.552 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_bytes": 0, 2024-09-11T22:07:39.552 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_items": 0, 2024-09-11T22:07:39.552 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_bytes": 0, 2024-09-11T22:07:39.552 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_items": 0, 2024-09-11T22:07:39.553 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_bytes": 0, 2024-09-11T22:07:39.553 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_items": 0, 2024-09-11T22:07:39.553 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_bytes": 8392, 2024-09-11T22:07:39.553 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_items": 184, 2024-09-11T22:07:39.553 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T22:07:39.553 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_items": 0, 2024-09-11T22:07:39.553 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_bytes": 0, 2024-09-11T22:07:39.553 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_items": 0, 2024-09-11T22:07:39.554 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_bytes": 49452056, 2024-09-11T22:07:39.554 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_items": 452317, 2024-09-11T22:07:39.554 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_bytes": 0, 2024-09-11T22:07:39.554 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_items": 0, 2024-09-11T22:07:39.554 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_bytes": 0, 2024-09-11T22:07:39.554 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_items": 0 2024-09-11T22:07:39.554 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:39.554 INFO:teuthology.orchestra.run.smithi053.stdout: "objecter": { 2024-09-11T22:07:39.555 INFO:teuthology.orchestra.run.smithi053.stdout: "op_active": 0, 2024-09-11T22:07:39.555 INFO:teuthology.orchestra.run.smithi053.stdout: "op_laggy": 0, 2024-09-11T22:07:39.555 INFO:teuthology.orchestra.run.smithi053.stdout: "op_send": 43247, 2024-09-11T22:07:39.555 INFO:teuthology.orchestra.run.smithi053.stdout: "op_send_bytes": 140656591, 2024-09-11T22:07:39.555 INFO:teuthology.orchestra.run.smithi053.stdout: "op_resend": 28, 2024-09-11T22:07:39.555 INFO:teuthology.orchestra.run.smithi053.stdout: "op_reply": 43247, 2024-09-11T22:07:39.555 INFO:teuthology.orchestra.run.smithi053.stdout: "op_latency": { 2024-09-11T22:07:39.556 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 43247, 2024-09-11T22:07:39.556 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 4417.328926738, 2024-09-11T22:07:39.556 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.102141857 2024-09-11T22:07:39.556 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:39.556 INFO:teuthology.orchestra.run.smithi053.stdout: "op_inflight": 0, 2024-09-11T22:07:39.556 INFO:teuthology.orchestra.run.smithi053.stdout: "oplen_avg": { 2024-09-11T22:07:39.557 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 43247, 2024-09-11T22:07:39.557 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 44841 2024-09-11T22:07:39.557 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:39.557 INFO:teuthology.orchestra.run.smithi053.stdout: "op": 43247, 2024-09-11T22:07:39.557 INFO:teuthology.orchestra.run.smithi053.stdout: "op_r": 19308, 2024-09-11T22:07:39.557 INFO:teuthology.orchestra.run.smithi053.stdout: "op_w": 23939, 2024-09-11T22:07:39.558 INFO:teuthology.orchestra.run.smithi053.stdout: "op_rmw": 0, 2024-09-11T22:07:39.558 INFO:teuthology.orchestra.run.smithi053.stdout: "op_pg": 0, 2024-09-11T22:07:39.558 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_stat": 0, 2024-09-11T22:07:39.558 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_create": 10, 2024-09-11T22:07:39.558 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_read": 310, 2024-09-11T22:07:39.558 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_write": 9545, 2024-09-11T22:07:39.558 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_writefull": 125, 2024-09-11T22:07:39.559 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_writesame": 0, 2024-09-11T22:07:39.559 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_append": 0, 2024-09-11T22:07:39.559 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_zero": 0, 2024-09-11T22:07:39.559 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_truncate": 0, 2024-09-11T22:07:39.559 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_delete": 13599, 2024-09-11T22:07:39.559 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_mapext": 0, 2024-09-11T22:07:39.559 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_sparse_read": 0, 2024-09-11T22:07:39.559 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_clonerange": 0, 2024-09-11T22:07:39.560 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_getxattr": 18124, 2024-09-11T22:07:39.560 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_setxattr": 20, 2024-09-11T22:07:39.560 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_cmpxattr": 0, 2024-09-11T22:07:39.560 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_rmxattr": 0, 2024-09-11T22:07:39.560 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_resetxattrs": 0, 2024-09-11T22:07:39.560 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_call": 0, 2024-09-11T22:07:39.560 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_watch": 0, 2024-09-11T22:07:39.560 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_notify": 0, 2024-09-11T22:07:39.561 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T22:07:39.561 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_pgls": 0, 2024-09-11T22:07:39.561 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_pgls_filter": 0, 2024-09-11T22:07:39.561 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_other": 0, 2024-09-11T22:07:39.561 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_active": 0, 2024-09-11T22:07:39.561 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_send": 0, 2024-09-11T22:07:39.561 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_resend": 0, 2024-09-11T22:07:39.562 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_ping": 0, 2024-09-11T22:07:39.562 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_active": 0, 2024-09-11T22:07:39.562 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_send": 0, 2024-09-11T22:07:39.562 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_resend": 0, 2024-09-11T22:07:39.562 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_active": 0, 2024-09-11T22:07:39.562 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_send": 0, 2024-09-11T22:07:39.562 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_resend": 0, 2024-09-11T22:07:39.562 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_active": 0, 2024-09-11T22:07:39.563 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_send": 0, 2024-09-11T22:07:39.563 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_resend": 0, 2024-09-11T22:07:39.563 INFO:teuthology.orchestra.run.smithi053.stdout: "command_active": 0, 2024-09-11T22:07:39.563 INFO:teuthology.orchestra.run.smithi053.stdout: "command_send": 0, 2024-09-11T22:07:39.563 INFO:teuthology.orchestra.run.smithi053.stdout: "command_resend": 0, 2024-09-11T22:07:39.563 INFO:teuthology.orchestra.run.smithi053.stdout: "map_epoch": 95, 2024-09-11T22:07:39.563 INFO:teuthology.orchestra.run.smithi053.stdout: "map_full": 0, 2024-09-11T22:07:39.563 INFO:teuthology.orchestra.run.smithi053.stdout: "map_inc": 11, 2024-09-11T22:07:39.563 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_sessions": 12, 2024-09-11T22:07:39.564 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_session_open": 12, 2024-09-11T22:07:39.564 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_session_close": 0, 2024-09-11T22:07:39.564 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_laggy": 0, 2024-09-11T22:07:39.564 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_wr": 1281, 2024-09-11T22:07:39.564 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_rd": 1752, 2024-09-11T22:07:39.564 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_del": 75 2024-09-11T22:07:39.564 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:39.565 INFO:teuthology.orchestra.run.smithi053.stdout: "oft": { 2024-09-11T22:07:39.565 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_objs": 1, 2024-09-11T22:07:39.565 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_kv_pairs": 0, 2024-09-11T22:07:39.565 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_updates": 0, 2024-09-11T22:07:39.565 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_removes": 0 2024-09-11T22:07:39.565 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:39.565 INFO:teuthology.orchestra.run.smithi053.stdout: "osd-slow-ops": { 2024-09-11T22:07:39.565 INFO:teuthology.orchestra.run.smithi053.stdout: "slow_ops_count": 18812 2024-09-11T22:07:39.566 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:39.566 INFO:teuthology.orchestra.run.smithi053.stdout: "purge_queue": { 2024-09-11T22:07:39.566 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_ops": 0, 2024-09-11T22:07:39.566 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_ops_high_water": 6, 2024-09-11T22:07:39.566 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing": 0, 2024-09-11T22:07:39.566 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_high_water": 6, 2024-09-11T22:07:39.566 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executed_ops": 13494, 2024-09-11T22:07:39.567 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executed": 13494, 2024-09-11T22:07:39.567 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_item_in_journal": 0 2024-09-11T22:07:39.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:39.567 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T22:07:39.567 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-09-11T22:07:39.567 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 104857600, 2024-09-11T22:07:39.567 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-09-11T22:07:39.567 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 148556, 2024-09-11T22:07:39.568 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 78924314, 2024-09-11T22:07:39.568 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-09-11T22:07:39.568 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 148556, 2024-09-11T22:07:39.568 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-09-11T22:07:39.568 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-09-11T22:07:39.568 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 148556, 2024-09-11T22:07:39.568 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 78924314, 2024-09-11T22:07:39.569 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-09-11T22:07:39.569 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:39.569 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:39.569 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:39.569 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-11T22:07:39.569 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:39.569 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-objecter_bytes": { 2024-09-11T22:07:39.569 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-09-11T22:07:39.570 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 104857600, 2024-09-11T22:07:39.570 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-09-11T22:07:39.570 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-09-11T22:07:39.570 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-09-11T22:07:39.570 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-09-11T22:07:39.570 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-09-11T22:07:39.570 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 43247, 2024-09-11T22:07:39.571 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 142213709, 2024-09-11T22:07:39.571 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 43247, 2024-09-11T22:07:39.571 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 142213709, 2024-09-11T22:07:39.571 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-09-11T22:07:39.571 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:39.571 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:39.571 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:39.572 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-11T22:07:39.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:39.572 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-objecter_ops": { 2024-09-11T22:07:39.572 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-09-11T22:07:39.572 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 1024, 2024-09-11T22:07:39.572 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-09-11T22:07:39.572 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-09-11T22:07:39.572 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-09-11T22:07:39.573 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-09-11T22:07:39.573 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-09-11T22:07:39.573 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 43247, 2024-09-11T22:07:39.573 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 43247, 2024-09-11T22:07:39.573 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 43247, 2024-09-11T22:07:39.573 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 43247, 2024-09-11T22:07:39.573 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-09-11T22:07:39.573 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:39.573 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:39.574 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:39.574 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-11T22:07:39.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:39.574 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-write_buf_throttle": { 2024-09-11T22:07:39.574 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-09-11T22:07:39.574 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 3758096384, 2024-09-11T22:07:39.574 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-09-11T22:07:39.575 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 13494, 2024-09-11T22:07:39.575 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 1571134, 2024-09-11T22:07:39.575 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-09-11T22:07:39.575 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 13494, 2024-09-11T22:07:39.575 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-09-11T22:07:39.575 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-09-11T22:07:39.575 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 310, 2024-09-11T22:07:39.575 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 1571134, 2024-09-11T22:07:39.576 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-09-11T22:07:39.576 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:39.576 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:39.576 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:39.576 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-11T22:07:39.576 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:39.576 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-write_buf_throttle-0x5578dee90500": { 2024-09-11T22:07:39.576 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-09-11T22:07:39.577 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 3758096384, 2024-09-11T22:07:39.577 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-09-11T22:07:39.577 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 60090, 2024-09-11T22:07:39.577 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 132271217, 2024-09-11T22:07:39.577 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-09-11T22:07:39.577 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 60090, 2024-09-11T22:07:39.577 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-09-11T22:07:39.578 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-09-11T22:07:39.578 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 9235, 2024-09-11T22:07:39.578 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 132271217, 2024-09-11T22:07:39.578 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-09-11T22:07:39.578 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:39.578 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:39.578 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:39.578 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-11T22:07:39.579 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-11T22:07:39.579 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-11T22:07:41.033 INFO:tasks.check_counter:Daemon mds.h mds.exported=526 2024-09-11T22:07:41.034 INFO:tasks.check_counter:Daemon mds.h mds.imported=500 2024-09-11T22:07:41.034 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=1983 2024-09-11T22:07:41.034 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=3980 2024-09-11T22:07:41.034 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=7 2024-09-11T22:07:41.034 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-09-11T22:07:41.034 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-09-11T22:07:41.035 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=62 2024-09-11T22:07:41.035 DEBUG:tasks.check_counter:Getting stats from k 2024-09-11T22:07:41.035 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-11T22:07:41.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:41 smithi053 ceph-mon[34908]: pgmap v324: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 14 op/s 2024-09-11T22:07:41.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:41 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T22:07:41.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:41 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T22:07:41.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:41 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T22:07:41.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:41 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T22:07:41.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:41 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:41.298 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:41 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:41.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:41 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:41.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:41 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:41.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:41 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:41.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:41 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:41.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:41 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:41.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:41 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:41.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:41 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:41.299 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:41 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:41.300 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:41 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:41.300 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:41 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:41.300 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:41 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:41.300 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:41 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:41.300 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:41 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:41.300 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:41 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:41.313 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T22:07:41.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:41 smithi161 ceph-mon[35615]: pgmap v324: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 14 op/s 2024-09-11T22:07:41.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:41 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T22:07:41.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:41 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T22:07:41.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:41 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T22:07:41.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:41 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T22:07:41.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:41 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:41.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:41 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:41.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:41 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:41.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:41 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:41.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:41 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:41.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:41 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:41.343 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:41 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:41.344 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:41 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:41.344 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:41 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:41.344 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:41 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:41.344 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:41 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:41.344 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:41 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:41.344 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:41 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:41.344 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:41 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:41.345 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:41 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:41.345 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:41 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:41.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:41 smithi001 ceph-mon[30935]: pgmap v324: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 14 op/s 2024-09-11T22:07:41.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:41 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T22:07:41.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:41 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T22:07:41.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:41 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T22:07:41.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:41 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T22:07:41.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:41 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:41.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:41 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:41.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:41 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:41.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:41 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:41.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:41 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:41.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:41 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:41.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:41 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:41.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:41 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:41.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:41 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:41.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:41 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:41.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:41 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:41.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:41 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:41.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:41 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:41.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:41 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:41.446 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:41 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:41.447 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:41 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:41.814 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-11T22:07:41.814 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T22:07:41.814 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 1788, 2024-09-11T22:07:41.814 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 1790, 2024-09-11T22:07:41.815 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 43898606, 2024-09-11T22:07:41.815 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 568591, 2024-09-11T22:07:41.815 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 8, 2024-09-11T22:07:41.815 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 4, 2024-09-11T22:07:41.815 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.171413431, 2024-09-11T22:07:41.815 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.048143043, 2024-09-11T22:07:41.815 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.065212422, 2024-09-11T22:07:41.816 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.045193249, 2024-09-11T22:07:41.816 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:07:41.816 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 1788, 2024-09-11T22:07:41.816 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.063546978, 2024-09-11T22:07:41.816 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000035540 2024-09-11T22:07:41.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:41.817 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:07:41.817 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:41.817 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:41.817 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:41.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:41.817 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-11T22:07:41.817 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 0 2024-09-11T22:07:41.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:41.818 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T22:07:41.818 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 347, 2024-09-11T22:07:41.818 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 504, 2024-09-11T22:07:41.818 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 37837422, 2024-09-11T22:07:41.818 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 547396, 2024-09-11T22:07:41.818 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 8, 2024-09-11T22:07:41.818 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 6, 2024-09-11T22:07:41.819 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.060296795, 2024-09-11T22:07:41.819 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.014108014, 2024-09-11T22:07:41.819 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.031832229, 2024-09-11T22:07:41.819 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.010189022, 2024-09-11T22:07:41.819 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:07:41.819 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 502, 2024-09-11T22:07:41.819 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.024429416, 2024-09-11T22:07:41.820 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000048664 2024-09-11T22:07:41.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:41.820 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:07:41.820 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:41.820 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:41.820 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:41.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:41.820 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-09-11T22:07:41.821 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 438896 2024-09-11T22:07:41.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:41.821 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T22:07:41.821 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 471, 2024-09-11T22:07:41.821 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 437, 2024-09-11T22:07:41.821 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 43763862, 2024-09-11T22:07:41.821 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 199050, 2024-09-11T22:07:41.822 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 5, 2024-09-11T22:07:41.822 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 4, 2024-09-11T22:07:41.822 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.078872030, 2024-09-11T22:07:41.822 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.018803524, 2024-09-11T22:07:41.822 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.070629064, 2024-09-11T22:07:41.822 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.004594945, 2024-09-11T22:07:41.822 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:07:41.823 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 436, 2024-09-11T22:07:41.823 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.020062418, 2024-09-11T22:07:41.823 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000046014 2024-09-11T22:07:41.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:41.823 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:07:41.823 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:41.824 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:41.824 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:41.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:41.824 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 1758688, 2024-09-11T22:07:41.824 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 131952 2024-09-11T22:07:41.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:41.824 INFO:teuthology.orchestra.run.smithi053.stdout: "cct": { 2024-09-11T22:07:41.824 INFO:teuthology.orchestra.run.smithi053.stdout: "total_workers": 1, 2024-09-11T22:07:41.825 INFO:teuthology.orchestra.run.smithi053.stdout: "unhealthy_workers": 0 2024-09-11T22:07:41.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:41.825 INFO:teuthology.orchestra.run.smithi053.stdout: "finisher-MDSRank": { 2024-09-11T22:07:41.825 INFO:teuthology.orchestra.run.smithi053.stdout: "queue_len": 0, 2024-09-11T22:07:41.825 INFO:teuthology.orchestra.run.smithi053.stdout: "complete_latency": { 2024-09-11T22:07:41.825 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 3839, 2024-09-11T22:07:41.825 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.092068192, 2024-09-11T22:07:41.825 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000023982 2024-09-11T22:07:41.826 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-11T22:07:41.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:41.826 INFO:teuthology.orchestra.run.smithi053.stdout: "finisher-PurgeQueue": { 2024-09-11T22:07:41.826 INFO:teuthology.orchestra.run.smithi053.stdout: "queue_len": 0, 2024-09-11T22:07:41.826 INFO:teuthology.orchestra.run.smithi053.stdout: "complete_latency": { 2024-09-11T22:07:41.826 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:41.826 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:41.826 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:41.826 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-11T22:07:41.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:41.827 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": { 2024-09-11T22:07:41.827 INFO:teuthology.orchestra.run.smithi053.stdout: "request": 0, 2024-09-11T22:07:41.827 INFO:teuthology.orchestra.run.smithi053.stdout: "reply": 0, 2024-09-11T22:07:41.827 INFO:teuthology.orchestra.run.smithi053.stdout: "reply_latency": { 2024-09-11T22:07:41.827 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:41.827 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:41.827 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:41.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:41.828 INFO:teuthology.orchestra.run.smithi053.stdout: "slow_reply": 0, 2024-09-11T22:07:41.828 INFO:teuthology.orchestra.run.smithi053.stdout: "forward": 0, 2024-09-11T22:07:41.828 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_fetch_complete": 0, 2024-09-11T22:07:41.828 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_fetch_keys": 0, 2024-09-11T22:07:41.828 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_commit": 0, 2024-09-11T22:07:41.828 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_split": 0, 2024-09-11T22:07:41.828 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_merge": 0, 2024-09-11T22:07:41.828 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes": 33441, 2024-09-11T22:07:41.829 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_top": 114, 2024-09-11T22:07:41.829 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_bottom": 33327, 2024-09-11T22:07:41.829 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_pin_tail": 0, 2024-09-11T22:07:41.829 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_pinned": 33278, 2024-09-11T22:07:41.829 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_expired": 0, 2024-09-11T22:07:41.829 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_with_caps": 0, 2024-09-11T22:07:41.829 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": 0, 2024-09-11T22:07:41.829 INFO:teuthology.orchestra.run.smithi053.stdout: "subtrees": 1, 2024-09-11T22:07:41.830 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse": 0, 2024-09-11T22:07:41.830 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_hit": 0, 2024-09-11T22:07:41.830 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_forward": 0, 2024-09-11T22:07:41.830 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_discover": 0, 2024-09-11T22:07:41.830 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_dir_fetch": 0, 2024-09-11T22:07:41.830 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_remote_ino": 0, 2024-09-11T22:07:41.830 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_lock": 0, 2024-09-11T22:07:41.830 INFO:teuthology.orchestra.run.smithi053.stdout: "load_cent": 0, 2024-09-11T22:07:41.831 INFO:teuthology.orchestra.run.smithi053.stdout: "q": 0, 2024-09-11T22:07:41.831 INFO:teuthology.orchestra.run.smithi053.stdout: "exported": 0, 2024-09-11T22:07:41.831 INFO:teuthology.orchestra.run.smithi053.stdout: "exported_inodes": 0, 2024-09-11T22:07:41.831 INFO:teuthology.orchestra.run.smithi053.stdout: "imported": 0, 2024-09-11T22:07:41.831 INFO:teuthology.orchestra.run.smithi053.stdout: "imported_inodes": 0, 2024-09-11T22:07:41.831 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_dir_fetch": 0, 2024-09-11T22:07:41.831 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_backtrace_fetch": 0, 2024-09-11T22:07:41.831 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_peer_discover": 0, 2024-09-11T22:07:41.832 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rfiles": 0, 2024-09-11T22:07:41.832 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rbytes": 0, 2024-09-11T22:07:41.832 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rsnaps": 0, 2024-09-11T22:07:41.832 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_backtrace_fetch": 0, 2024-09-11T22:07:41.832 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_set_tag": 0, 2024-09-11T22:07:41.832 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T22:07:41.832 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_inotable_repaired": 0, 2024-09-11T22:07:41.833 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dir_inodes": 0, 2024-09-11T22:07:41.833 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T22:07:41.833 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T22:07:41.833 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_file_inodes": 0, 2024-09-11T22:07:41.833 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_inode_file_caps": 0, 2024-09-11T22:07:41.833 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_revoke": 0, 2024-09-11T22:07:41.833 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_grant": 0, 2024-09-11T22:07:41.834 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T22:07:41.834 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T22:07:41.834 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T22:07:41.834 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_caps": 0, 2024-09-11T22:07:41.834 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_caps_dirty": 0, 2024-09-11T22:07:41.834 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_cap_release": 0, 2024-09-11T22:07:41.834 INFO:teuthology.orchestra.run.smithi053.stdout: "process_request_cap_release": 0 2024-09-11T22:07:41.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:41.834 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_cache": { 2024-09-11T22:07:41.835 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_update": 0, 2024-09-11T22:07:41.835 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_update_receipt": 0, 2024-09-11T22:07:41.835 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_try_discover": 0, 2024-09-11T22:07:41.835 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_send_discover": 0, 2024-09-11T22:07:41.835 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_handle_discover": 0, 2024-09-11T22:07:41.836 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays": 0, 2024-09-11T22:07:41.836 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays_delayed": 0, 2024-09-11T22:07:41.836 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays_enqueuing": 0, 2024-09-11T22:07:41.836 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_created": 0, 2024-09-11T22:07:41.836 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_enqueued": 0, 2024-09-11T22:07:41.836 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_reintegrated": 0, 2024-09-11T22:07:41.836 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_migrated": 0, 2024-09-11T22:07:41.836 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_processing": 0, 2024-09-11T22:07:41.837 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_enqueued": 0, 2024-09-11T22:07:41.837 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_prioritized": 0, 2024-09-11T22:07:41.837 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_started": 0, 2024-09-11T22:07:41.837 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_completed": 0, 2024-09-11T22:07:41.837 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_quiesce_path": 0, 2024-09-11T22:07:41.837 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_quiesce_inode": 0, 2024-09-11T22:07:41.837 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T22:07:41.838 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_exportdir": 0, 2024-09-11T22:07:41.838 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_flush": 0, 2024-09-11T22:07:41.838 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_fragmentdir": 0, 2024-09-11T22:07:41.838 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_fragstats": 0, 2024-09-11T22:07:41.838 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_inodestats": 0 2024-09-11T22:07:41.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:41.838 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_log": { 2024-09-11T22:07:41.838 INFO:teuthology.orchestra.run.smithi053.stdout: "evlrg": 0, 2024-09-11T22:07:41.839 INFO:teuthology.orchestra.run.smithi053.stdout: "evadd": 0, 2024-09-11T22:07:41.839 INFO:teuthology.orchestra.run.smithi053.stdout: "evex": 0, 2024-09-11T22:07:41.839 INFO:teuthology.orchestra.run.smithi053.stdout: "evtrm": 0, 2024-09-11T22:07:41.839 INFO:teuthology.orchestra.run.smithi053.stdout: "ev": 51376, 2024-09-11T22:07:41.839 INFO:teuthology.orchestra.run.smithi053.stdout: "evexg": 0, 2024-09-11T22:07:41.839 INFO:teuthology.orchestra.run.smithi053.stdout: "evexd": 0, 2024-09-11T22:07:41.839 INFO:teuthology.orchestra.run.smithi053.stdout: "segadd": 0, 2024-09-11T22:07:41.840 INFO:teuthology.orchestra.run.smithi053.stdout: "segex": 0, 2024-09-11T22:07:41.840 INFO:teuthology.orchestra.run.smithi053.stdout: "segtrm": 0, 2024-09-11T22:07:41.840 INFO:teuthology.orchestra.run.smithi053.stdout: "seg": 71, 2024-09-11T22:07:41.840 INFO:teuthology.orchestra.run.smithi053.stdout: "segmjr": 6, 2024-09-11T22:07:41.840 INFO:teuthology.orchestra.run.smithi053.stdout: "segexg": 0, 2024-09-11T22:07:41.840 INFO:teuthology.orchestra.run.smithi053.stdout: "segexd": 0, 2024-09-11T22:07:41.840 INFO:teuthology.orchestra.run.smithi053.stdout: "expos": 4194304, 2024-09-11T22:07:41.841 INFO:teuthology.orchestra.run.smithi053.stdout: "wrpos": 127267996, 2024-09-11T22:07:41.841 INFO:teuthology.orchestra.run.smithi053.stdout: "rdpos": 127089688, 2024-09-11T22:07:41.841 INFO:teuthology.orchestra.run.smithi053.stdout: "jlat": { 2024-09-11T22:07:41.841 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:41.841 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:41.841 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:41.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:41.841 INFO:teuthology.orchestra.run.smithi053.stdout: "replayed": 51376 2024-09-11T22:07:41.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:41.842 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_mem": { 2024-09-11T22:07:41.842 INFO:teuthology.orchestra.run.smithi053.stdout: "ino": 13762, 2024-09-11T22:07:41.842 INFO:teuthology.orchestra.run.smithi053.stdout: "ino+": 38804, 2024-09-11T22:07:41.842 INFO:teuthology.orchestra.run.smithi053.stdout: "ino-": 25042, 2024-09-11T22:07:41.842 INFO:teuthology.orchestra.run.smithi053.stdout: "dir": 523, 2024-09-11T22:07:41.842 INFO:teuthology.orchestra.run.smithi053.stdout: "dir+": 1658, 2024-09-11T22:07:41.843 INFO:teuthology.orchestra.run.smithi053.stdout: "dir-": 1135, 2024-09-11T22:07:41.843 INFO:teuthology.orchestra.run.smithi053.stdout: "dn": 33441, 2024-09-11T22:07:41.843 INFO:teuthology.orchestra.run.smithi053.stdout: "dn+": 55648, 2024-09-11T22:07:41.843 INFO:teuthology.orchestra.run.smithi053.stdout: "dn-": 22207, 2024-09-11T22:07:41.843 INFO:teuthology.orchestra.run.smithi053.stdout: "cap": 0, 2024-09-11T22:07:41.843 INFO:teuthology.orchestra.run.smithi053.stdout: "cap+": 0, 2024-09-11T22:07:41.843 INFO:teuthology.orchestra.run.smithi053.stdout: "cap-": 0, 2024-09-11T22:07:41.844 INFO:teuthology.orchestra.run.smithi053.stdout: "rss": 130948, 2024-09-11T22:07:41.844 INFO:teuthology.orchestra.run.smithi053.stdout: "heap": 215328 2024-09-11T22:07:41.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:41.844 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_server": { 2024-09-11T22:07:41.844 INFO:teuthology.orchestra.run.smithi053.stdout: "dispatch_client_request": 0, 2024-09-11T22:07:41.844 INFO:teuthology.orchestra.run.smithi053.stdout: "dispatch_server_request": 0, 2024-09-11T22:07:41.844 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_request": 0, 2024-09-11T22:07:41.845 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_session": 0, 2024-09-11T22:07:41.845 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_peer_request": 0, 2024-09-11T22:07:41.845 INFO:teuthology.orchestra.run.smithi053.stdout: "req_create_latency": { 2024-09-11T22:07:41.845 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:41.845 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:41.845 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:41.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:41.845 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getattr_latency": { 2024-09-11T22:07:41.846 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:41.848 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:41.848 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:41.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:41.848 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getfilelock_latency": { 2024-09-11T22:07:41.848 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:41.848 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:41.848 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:41.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:41.849 INFO:teuthology.orchestra.run.smithi053.stdout: "req_link_latency": { 2024-09-11T22:07:41.849 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:41.849 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:41.849 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:41.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:41.850 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookup_latency": { 2024-09-11T22:07:41.850 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:41.850 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:41.850 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:41.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:41.850 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookuphash_latency": { 2024-09-11T22:07:41.850 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:41.850 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:41.851 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:41.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:41.851 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupino_latency": { 2024-09-11T22:07:41.851 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:41.851 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:41.851 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:41.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:41.851 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupname_latency": { 2024-09-11T22:07:41.852 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:41.852 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:41.852 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:41.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:41.852 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupparent_latency": { 2024-09-11T22:07:41.852 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:41.852 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:41.852 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:41.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:41.853 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupsnap_latency": { 2024-09-11T22:07:41.853 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:41.853 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:41.853 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:41.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:41.853 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lssnap_latency": { 2024-09-11T22:07:41.853 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:41.854 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:41.854 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:41.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:41.854 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mkdir_latency": { 2024-09-11T22:07:41.854 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:41.854 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:41.854 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:41.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:41.855 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mknod_latency": { 2024-09-11T22:07:41.855 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:41.855 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:41.855 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:41.855 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:41.855 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mksnap_latency": { 2024-09-11T22:07:41.855 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:41.855 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:41.856 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:41.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:41.856 INFO:teuthology.orchestra.run.smithi053.stdout: "req_open_latency": { 2024-09-11T22:07:41.856 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:41.856 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:41.856 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:41.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:41.856 INFO:teuthology.orchestra.run.smithi053.stdout: "req_readdir_latency": { 2024-09-11T22:07:41.857 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:41.857 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:41.857 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:41.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:41.857 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rename_latency": { 2024-09-11T22:07:41.857 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:41.857 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:41.858 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:41.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:41.858 INFO:teuthology.orchestra.run.smithi053.stdout: "req_renamesnap_latency": { 2024-09-11T22:07:41.858 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:41.858 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:41.858 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:41.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:41.858 INFO:teuthology.orchestra.run.smithi053.stdout: "req_snapdiff_latency": { 2024-09-11T22:07:41.859 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:41.859 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:41.859 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:41.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:41.859 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmdir_latency": { 2024-09-11T22:07:41.859 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:41.859 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:41.859 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:41.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:41.860 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmsnap_latency": { 2024-09-11T22:07:41.860 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:41.860 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:41.860 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:41.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:41.861 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmxattr_latency": { 2024-09-11T22:07:41.861 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:41.861 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:41.861 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:41.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:41.861 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setattr_latency": { 2024-09-11T22:07:41.862 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:41.862 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:41.862 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:41.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:41.862 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setdirlayout_latency": { 2024-09-11T22:07:41.862 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:41.862 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:41.862 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:41.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:41.863 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setfilelock_latency": { 2024-09-11T22:07:41.863 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:41.863 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:41.863 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:41.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:41.864 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setlayout_latency": { 2024-09-11T22:07:41.864 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:41.864 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:41.864 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:41.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:41.864 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setxattr_latency": { 2024-09-11T22:07:41.864 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:41.864 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:41.865 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:41.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:41.865 INFO:teuthology.orchestra.run.smithi053.stdout: "req_symlink_latency": { 2024-09-11T22:07:41.865 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:41.865 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:41.865 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:41.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:41.865 INFO:teuthology.orchestra.run.smithi053.stdout: "req_unlink_latency": { 2024-09-11T22:07:41.866 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:41.866 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:41.866 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:41.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:41.866 INFO:teuthology.orchestra.run.smithi053.stdout: "cap_revoke_eviction": 0, 2024-09-11T22:07:41.866 INFO:teuthology.orchestra.run.smithi053.stdout: "cap_acquisition_throttle": 0, 2024-09-11T22:07:41.866 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getvxattr_latency": { 2024-09-11T22:07:41.866 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:41.867 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:41.867 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:41.867 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-11T22:07:41.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:41.867 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_sessions": { 2024-09-11T22:07:41.867 INFO:teuthology.orchestra.run.smithi053.stdout: "session_count": 1, 2024-09-11T22:07:41.867 INFO:teuthology.orchestra.run.smithi053.stdout: "session_add": 3, 2024-09-11T22:07:41.867 INFO:teuthology.orchestra.run.smithi053.stdout: "session_remove": 2, 2024-09-11T22:07:41.868 INFO:teuthology.orchestra.run.smithi053.stdout: "sessions_open": 2, 2024-09-11T22:07:41.868 INFO:teuthology.orchestra.run.smithi053.stdout: "sessions_stale": 0, 2024-09-11T22:07:41.868 INFO:teuthology.orchestra.run.smithi053.stdout: "total_load": 0, 2024-09-11T22:07:41.868 INFO:teuthology.orchestra.run.smithi053.stdout: "average_load": 0, 2024-09-11T22:07:41.868 INFO:teuthology.orchestra.run.smithi053.stdout: "avg_session_uptime": 3358, 2024-09-11T22:07:41.868 INFO:teuthology.orchestra.run.smithi053.stdout: "mdthresh_evicted": 0 2024-09-11T22:07:41.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:41.868 INFO:teuthology.orchestra.run.smithi053.stdout: "mempool": { 2024-09-11T22:07:41.869 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_bytes": 0, 2024-09-11T22:07:41.869 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_items": 0, 2024-09-11T22:07:41.869 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T22:07:41.869 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_items": 0, 2024-09-11T22:07:41.869 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T22:07:41.869 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_items": 0, 2024-09-11T22:07:41.869 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T22:07:41.869 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T22:07:41.870 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T22:07:41.870 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T22:07:41.870 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T22:07:41.870 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_items": 0, 2024-09-11T22:07:41.870 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T22:07:41.870 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T22:07:41.870 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_bytes": 0, 2024-09-11T22:07:41.871 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_items": 0, 2024-09-11T22:07:41.871 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_bytes": 0, 2024-09-11T22:07:41.871 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_items": 0, 2024-09-11T22:07:41.871 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T22:07:41.871 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T22:07:41.871 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T22:07:41.871 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T22:07:41.871 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T22:07:41.872 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_items": 0, 2024-09-11T22:07:41.872 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_bytes": 0, 2024-09-11T22:07:41.872 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_items": 0, 2024-09-11T22:07:41.872 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T22:07:41.872 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T22:07:41.872 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_bytes": 0, 2024-09-11T22:07:41.872 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_items": 0, 2024-09-11T22:07:41.872 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_bytes": 0, 2024-09-11T22:07:41.873 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_items": 0, 2024-09-11T22:07:41.873 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T22:07:41.873 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_items": 0, 2024-09-11T22:07:41.873 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T22:07:41.873 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_items": 0, 2024-09-11T22:07:41.873 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_bytes": 739408, 2024-09-11T22:07:41.873 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_items": 13825, 2024-09-11T22:07:41.874 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_bytes": 0, 2024-09-11T22:07:41.874 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_items": 0, 2024-09-11T22:07:41.874 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_bytes": 0, 2024-09-11T22:07:41.874 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_items": 0, 2024-09-11T22:07:41.874 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_bytes": 0, 2024-09-11T22:07:41.874 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_items": 0, 2024-09-11T22:07:41.874 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_bytes": 0, 2024-09-11T22:07:41.874 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_items": 0, 2024-09-11T22:07:41.875 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_bytes": 8392, 2024-09-11T22:07:41.875 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_items": 184, 2024-09-11T22:07:41.875 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T22:07:41.875 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_items": 0, 2024-09-11T22:07:41.875 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_bytes": 0, 2024-09-11T22:07:41.875 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_items": 0, 2024-09-11T22:07:41.875 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_bytes": 55539935, 2024-09-11T22:07:41.875 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_items": 432621, 2024-09-11T22:07:41.876 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_bytes": 0, 2024-09-11T22:07:41.876 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_items": 0, 2024-09-11T22:07:41.876 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_bytes": 0, 2024-09-11T22:07:41.876 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_items": 0 2024-09-11T22:07:41.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:41.876 INFO:teuthology.orchestra.run.smithi053.stdout: "objecter": { 2024-09-11T22:07:41.876 INFO:teuthology.orchestra.run.smithi053.stdout: "op_active": 0, 2024-09-11T22:07:41.876 INFO:teuthology.orchestra.run.smithi053.stdout: "op_laggy": 0, 2024-09-11T22:07:41.877 INFO:teuthology.orchestra.run.smithi053.stdout: "op_send": 2340, 2024-09-11T22:07:41.877 INFO:teuthology.orchestra.run.smithi053.stdout: "op_send_bytes": 21, 2024-09-11T22:07:41.877 INFO:teuthology.orchestra.run.smithi053.stdout: "op_resend": 3, 2024-09-11T22:07:41.877 INFO:teuthology.orchestra.run.smithi053.stdout: "op_reply": 2340, 2024-09-11T22:07:41.877 INFO:teuthology.orchestra.run.smithi053.stdout: "op_latency": { 2024-09-11T22:07:41.877 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 2340, 2024-09-11T22:07:41.878 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 4.164913275, 2024-09-11T22:07:41.878 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.001779877 2024-09-11T22:07:41.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:41.878 INFO:teuthology.orchestra.run.smithi053.stdout: "op_inflight": 0, 2024-09-11T22:07:41.878 INFO:teuthology.orchestra.run.smithi053.stdout: "oplen_avg": { 2024-09-11T22:07:41.878 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 2340, 2024-09-11T22:07:41.878 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 2341 2024-09-11T22:07:41.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:41.879 INFO:teuthology.orchestra.run.smithi053.stdout: "op": 2340, 2024-09-11T22:07:41.879 INFO:teuthology.orchestra.run.smithi053.stdout: "op_r": 2340, 2024-09-11T22:07:41.879 INFO:teuthology.orchestra.run.smithi053.stdout: "op_w": 0, 2024-09-11T22:07:41.879 INFO:teuthology.orchestra.run.smithi053.stdout: "op_rmw": 0, 2024-09-11T22:07:41.879 INFO:teuthology.orchestra.run.smithi053.stdout: "op_pg": 0, 2024-09-11T22:07:41.879 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_stat": 1445, 2024-09-11T22:07:41.879 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_create": 0, 2024-09-11T22:07:41.879 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_read": 893, 2024-09-11T22:07:41.880 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_write": 0, 2024-09-11T22:07:41.880 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_writefull": 0, 2024-09-11T22:07:41.880 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_writesame": 0, 2024-09-11T22:07:41.880 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_append": 0, 2024-09-11T22:07:41.880 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_zero": 0, 2024-09-11T22:07:41.880 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_truncate": 0, 2024-09-11T22:07:41.880 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_delete": 0, 2024-09-11T22:07:41.880 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_mapext": 0, 2024-09-11T22:07:41.881 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_sparse_read": 0, 2024-09-11T22:07:41.881 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_clonerange": 0, 2024-09-11T22:07:41.881 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_getxattr": 1, 2024-09-11T22:07:41.881 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_setxattr": 0, 2024-09-11T22:07:41.881 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_cmpxattr": 0, 2024-09-11T22:07:41.881 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_rmxattr": 0, 2024-09-11T22:07:41.881 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_resetxattrs": 0, 2024-09-11T22:07:41.881 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_call": 0, 2024-09-11T22:07:41.882 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_watch": 0, 2024-09-11T22:07:41.882 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_notify": 0, 2024-09-11T22:07:41.882 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T22:07:41.882 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_pgls": 0, 2024-09-11T22:07:41.882 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_pgls_filter": 0, 2024-09-11T22:07:41.882 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_other": 0, 2024-09-11T22:07:41.882 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_active": 0, 2024-09-11T22:07:41.883 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_send": 0, 2024-09-11T22:07:41.883 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_resend": 0, 2024-09-11T22:07:41.883 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_ping": 0, 2024-09-11T22:07:41.883 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_active": 0, 2024-09-11T22:07:41.883 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_send": 0, 2024-09-11T22:07:41.883 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_resend": 0, 2024-09-11T22:07:41.884 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_active": 0, 2024-09-11T22:07:41.884 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_send": 0, 2024-09-11T22:07:41.884 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_resend": 0, 2024-09-11T22:07:41.884 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_active": 0, 2024-09-11T22:07:41.884 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_send": 0, 2024-09-11T22:07:41.885 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_resend": 0, 2024-09-11T22:07:41.885 INFO:teuthology.orchestra.run.smithi053.stdout: "command_active": 0, 2024-09-11T22:07:41.885 INFO:teuthology.orchestra.run.smithi053.stdout: "command_send": 0, 2024-09-11T22:07:41.885 INFO:teuthology.orchestra.run.smithi053.stdout: "command_resend": 0, 2024-09-11T22:07:41.885 INFO:teuthology.orchestra.run.smithi053.stdout: "map_epoch": 95, 2024-09-11T22:07:41.885 INFO:teuthology.orchestra.run.smithi053.stdout: "map_full": 0, 2024-09-11T22:07:41.885 INFO:teuthology.orchestra.run.smithi053.stdout: "map_inc": 11, 2024-09-11T22:07:41.885 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_sessions": 10, 2024-09-11T22:07:41.886 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_session_open": 10, 2024-09-11T22:07:41.886 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_session_close": 0, 2024-09-11T22:07:41.886 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_laggy": 0, 2024-09-11T22:07:41.886 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_wr": 0, 2024-09-11T22:07:41.886 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_rd": 2, 2024-09-11T22:07:41.886 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_del": 0 2024-09-11T22:07:41.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:41.886 INFO:teuthology.orchestra.run.smithi053.stdout: "oft": { 2024-09-11T22:07:41.887 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_objs": 0, 2024-09-11T22:07:41.887 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_kv_pairs": 0, 2024-09-11T22:07:41.887 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_updates": 0, 2024-09-11T22:07:41.887 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_removes": 0 2024-09-11T22:07:41.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:41.887 INFO:teuthology.orchestra.run.smithi053.stdout: "osd-slow-ops": { 2024-09-11T22:07:41.887 INFO:teuthology.orchestra.run.smithi053.stdout: "slow_ops_count": 0 2024-09-11T22:07:41.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:41.888 INFO:teuthology.orchestra.run.smithi053.stdout: "purge_queue": { 2024-09-11T22:07:41.888 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_ops": 0, 2024-09-11T22:07:41.888 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_ops_high_water": 0, 2024-09-11T22:07:41.888 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing": 0, 2024-09-11T22:07:41.888 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_high_water": 0, 2024-09-11T22:07:41.888 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executed_ops": 0, 2024-09-11T22:07:41.888 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executed": 0, 2024-09-11T22:07:41.889 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_item_in_journal": 0 2024-09-11T22:07:41.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:41.889 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T22:07:41.889 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-09-11T22:07:41.889 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 104857600, 2024-09-11T22:07:41.889 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-09-11T22:07:41.889 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 2602, 2024-09-11T22:07:41.890 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 125249493, 2024-09-11T22:07:41.890 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-09-11T22:07:41.890 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 2602, 2024-09-11T22:07:41.890 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-09-11T22:07:41.890 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-09-11T22:07:41.890 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 2602, 2024-09-11T22:07:41.890 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 125249493, 2024-09-11T22:07:41.890 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-09-11T22:07:41.890 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:41.891 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:41.891 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:41.891 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-11T22:07:41.891 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:41.891 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-objecter_bytes": { 2024-09-11T22:07:41.891 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-09-11T22:07:41.891 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 104857600, 2024-09-11T22:07:41.891 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-09-11T22:07:41.892 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-09-11T22:07:41.892 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-09-11T22:07:41.892 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-09-11T22:07:41.892 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-09-11T22:07:41.892 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 2340, 2024-09-11T22:07:41.892 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 123082260, 2024-09-11T22:07:41.892 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 2340, 2024-09-11T22:07:41.893 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 123082260, 2024-09-11T22:07:41.893 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-09-11T22:07:41.893 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:41.893 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:41.893 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:41.893 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-11T22:07:41.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:41.893 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-objecter_ops": { 2024-09-11T22:07:41.894 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-09-11T22:07:41.894 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 1024, 2024-09-11T22:07:41.894 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-09-11T22:07:41.894 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-09-11T22:07:41.894 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-09-11T22:07:41.894 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-09-11T22:07:41.894 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-09-11T22:07:41.894 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 2340, 2024-09-11T22:07:41.895 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 2340, 2024-09-11T22:07:41.895 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 2340, 2024-09-11T22:07:41.895 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 2340, 2024-09-11T22:07:41.895 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-09-11T22:07:41.895 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:41.895 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:41.895 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:41.895 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-11T22:07:41.896 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:41.896 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-write_buf_throttle": { 2024-09-11T22:07:41.896 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-09-11T22:07:41.896 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 3758096384, 2024-09-11T22:07:41.896 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-09-11T22:07:41.896 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-09-11T22:07:41.896 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-09-11T22:07:41.896 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-09-11T22:07:41.897 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-09-11T22:07:41.897 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-09-11T22:07:41.897 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-09-11T22:07:41.897 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 0, 2024-09-11T22:07:41.897 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 0, 2024-09-11T22:07:41.897 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-09-11T22:07:41.897 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:41.897 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:41.898 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:41.898 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-11T22:07:41.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-11T22:07:41.898 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-write_buf_throttle-0x55dd2dba86e0": { 2024-09-11T22:07:41.898 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2024-09-11T22:07:41.898 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 3758096384, 2024-09-11T22:07:41.898 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2024-09-11T22:07:41.898 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2024-09-11T22:07:41.899 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2024-09-11T22:07:41.899 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2024-09-11T22:07:41.899 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2024-09-11T22:07:41.899 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2024-09-11T22:07:41.899 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2024-09-11T22:07:41.899 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 0, 2024-09-11T22:07:41.899 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 0, 2024-09-11T22:07:41.899 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2024-09-11T22:07:41.900 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2024-09-11T22:07:41.900 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2024-09-11T22:07:41.900 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2024-09-11T22:07:41.900 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-11T22:07:41.900 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-11T22:07:41.900 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-11T22:07:42.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:42 smithi161 ceph-mon[35615]: Adjusting osd_memory_target on smithi161 to 176.0M 2024-09-11T22:07:42.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:42 smithi161 ceph-mon[35615]: Unable to set osd_memory_target on smithi161 to 184591769: error parsing value: Value '184591769' is below minimum 939524096 2024-09-11T22:07:42.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:42 smithi161 ceph-mon[35615]: Adjusting osd_memory_target on smithi001 to 177.0M 2024-09-11T22:07:42.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:42 smithi161 ceph-mon[35615]: Unable to set osd_memory_target on smithi001 to 185689190: error parsing value: Value '185689190' is below minimum 939524096 2024-09-11T22:07:42.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:42 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T22:07:42.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:42 smithi001 ceph-mon[30935]: Adjusting osd_memory_target on smithi161 to 176.0M 2024-09-11T22:07:42.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:42 smithi001 ceph-mon[30935]: Unable to set osd_memory_target on smithi161 to 184591769: error parsing value: Value '184591769' is below minimum 939524096 2024-09-11T22:07:42.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:42 smithi001 ceph-mon[30935]: Adjusting osd_memory_target on smithi001 to 177.0M 2024-09-11T22:07:42.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:42 smithi001 ceph-mon[30935]: Unable to set osd_memory_target on smithi001 to 185689190: error parsing value: Value '185689190' is below minimum 939524096 2024-09-11T22:07:42.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:42 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T22:07:42.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:42 smithi053 ceph-mon[34908]: Adjusting osd_memory_target on smithi161 to 176.0M 2024-09-11T22:07:42.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:42 smithi053 ceph-mon[34908]: Unable to set osd_memory_target on smithi161 to 184591769: error parsing value: Value '184591769' is below minimum 939524096 2024-09-11T22:07:42.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:42 smithi053 ceph-mon[34908]: Adjusting osd_memory_target on smithi001 to 177.0M 2024-09-11T22:07:42.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:42 smithi053 ceph-mon[34908]: Unable to set osd_memory_target on smithi001 to 185689190: error parsing value: Value '185689190' is below minimum 939524096 2024-09-11T22:07:42.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:42 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T22:07:42.588 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-09-11T22:07:42.588 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-09-11T22:07:42.588 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-09-11T22:07:42.589 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2024-09-11T22:07:42.589 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2024-09-11T22:07:42.589 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-09-11T22:07:42.589 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-09-11T22:07:42.589 INFO:tasks.check_counter:Daemon mds.k mds.dir_split=0 2024-09-11T22:07:42.589 DEBUG:tasks.check_counter:Getting stats from c 2024-09-11T22:07:42.590 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-11T22:07:42.853 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T22:07:43.267 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:43 smithi161 ceph-mon[35615]: pgmap v325: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:07:43.267 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:43 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T22:07:43.268 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:43 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:43.268 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:43 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:43.268 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:43 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:43.268 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:43 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:43.268 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:43 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:43.268 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:43 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:43.268 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:43 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:43.269 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:43 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:43.269 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:43 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:07:43.269 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:43 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:07:43.269 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:43 smithi161 ceph-mon[35615]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T22:07:43.372 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-11T22:07:43.373 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T22:07:43.373 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 57283, 2024-09-11T22:07:43.373 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 54673, 2024-09-11T22:07:43.373 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 41546615, 2024-09-11T22:07:43.373 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 52018688, 2024-09-11T22:07:43.373 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 22, 2024-09-11T22:07:43.373 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 5, 2024-09-11T22:07:43.374 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 3.185896660, 2024-09-11T22:07:43.374 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 1.376224701, 2024-09-11T22:07:43.374 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 2.590271918, 2024-09-11T22:07:43.374 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.124720170, 2024-09-11T22:07:43.374 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:07:43.374 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 54677, 2024-09-11T22:07:43.374 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 6.407198273, 2024-09-11T22:07:43.374 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000117182 2024-09-11T22:07:43.375 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:43.375 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:07:43.375 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 98762, 2024-09-11T22:07:43.375 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.017352093, 2024-09-11T22:07:43.375 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000175 2024-09-11T22:07:43.375 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:43.375 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-11T22:07:43.375 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 0 2024-09-11T22:07:43.376 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:43.376 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T22:07:43.376 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 19167, 2024-09-11T22:07:43.376 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 15528, 2024-09-11T22:07:43.376 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 12402736, 2024-09-11T22:07:43.376 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 50739308, 2024-09-11T22:07:43.376 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 13, 2024-09-11T22:07:43.376 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 8, 2024-09-11T22:07:43.377 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 1.631801232, 2024-09-11T22:07:43.377 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.627881040, 2024-09-11T22:07:43.377 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.655086734, 2024-09-11T22:07:43.377 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.455480234, 2024-09-11T22:07:43.377 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:07:43.377 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 15525, 2024-09-11T22:07:43.377 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 1.866834309, 2024-09-11T22:07:43.377 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000120246 2024-09-11T22:07:43.378 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:43.378 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:07:43.378 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 18436, 2024-09-11T22:07:43.378 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.002980352, 2024-09-11T22:07:43.378 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000161 2024-09-11T22:07:43.378 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:43.378 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 1782672, 2024-09-11T22:07:43.378 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 380704 2024-09-11T22:07:43.379 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:43.379 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T22:07:43.379 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 57583, 2024-09-11T22:07:43.379 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 98070, 2024-09-11T22:07:43.380 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 20654803, 2024-09-11T22:07:43.380 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 91650540, 2024-09-11T22:07:43.380 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 24, 2024-09-11T22:07:43.380 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 8, 2024-09-11T22:07:43.380 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 4.460854673, 2024-09-11T22:07:43.380 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 1.948637613, 2024-09-11T22:07:43.381 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 24.961324431, 2024-09-11T22:07:43.381 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.586375359, 2024-09-11T22:07:43.381 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:07:43.381 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 98064, 2024-09-11T22:07:43.381 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 11.379336614, 2024-09-11T22:07:43.381 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000116039 2024-09-11T22:07:43.381 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:43.381 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:07:43.382 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 101776, 2024-09-11T22:07:43.382 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.021544862, 2024-09-11T22:07:43.382 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000211 2024-09-11T22:07:43.382 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:43.382 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 5200, 2024-09-11T22:07:43.382 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 442144 2024-09-11T22:07:43.382 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:43.383 INFO:teuthology.orchestra.run.smithi161.stdout: "cct": { 2024-09-11T22:07:43.383 INFO:teuthology.orchestra.run.smithi161.stdout: "total_workers": 1, 2024-09-11T22:07:43.383 INFO:teuthology.orchestra.run.smithi161.stdout: "unhealthy_workers": 0 2024-09-11T22:07:43.383 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:43.383 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-MDSRank": { 2024-09-11T22:07:43.383 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-09-11T22:07:43.383 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-09-11T22:07:43.383 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 13181, 2024-09-11T22:07:43.384 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 34.904515359, 2024-09-11T22:07:43.384 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.002648093 2024-09-11T22:07:43.384 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-11T22:07:43.384 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:43.384 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-PurgeQueue": { 2024-09-11T22:07:43.384 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-09-11T22:07:43.385 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-09-11T22:07:43.385 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 12139, 2024-09-11T22:07:43.385 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 2.627034485, 2024-09-11T22:07:43.385 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000216412 2024-09-11T22:07:43.385 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-11T22:07:43.385 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:43.385 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": { 2024-09-11T22:07:43.386 INFO:teuthology.orchestra.run.smithi161.stdout: "request": 24803, 2024-09-11T22:07:43.386 INFO:teuthology.orchestra.run.smithi161.stdout: "reply": 19652, 2024-09-11T22:07:43.386 INFO:teuthology.orchestra.run.smithi161.stdout: "reply_latency": { 2024-09-11T22:07:43.386 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 19652, 2024-09-11T22:07:43.386 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 75.745376680, 2024-09-11T22:07:43.386 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.003854334 2024-09-11T22:07:43.387 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:43.387 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_reply": 0, 2024-09-11T22:07:43.387 INFO:teuthology.orchestra.run.smithi161.stdout: "forward": 5151, 2024-09-11T22:07:43.387 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_complete": 304, 2024-09-11T22:07:43.387 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_keys": 0, 2024-09-11T22:07:43.387 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_commit": 547, 2024-09-11T22:07:43.387 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_split": 67, 2024-09-11T22:07:43.388 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_merge": 0, 2024-09-11T22:07:43.388 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes": 24104, 2024-09-11T22:07:43.388 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_top": 765, 2024-09-11T22:07:43.388 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_bottom": 12084, 2024-09-11T22:07:43.388 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pin_tail": 11255, 2024-09-11T22:07:43.388 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pinned": 23010, 2024-09-11T22:07:43.388 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_expired": 17829, 2024-09-11T22:07:43.388 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_with_caps": 1, 2024-09-11T22:07:43.389 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": 2, 2024-09-11T22:07:43.389 INFO:teuthology.orchestra.run.smithi161.stdout: "subtrees": 6, 2024-09-11T22:07:43.389 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse": 36808, 2024-09-11T22:07:43.389 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_hit": 28121, 2024-09-11T22:07:43.389 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_forward": 5011, 2024-09-11T22:07:43.389 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_discover": 75, 2024-09-11T22:07:43.389 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_dir_fetch": 0, 2024-09-11T22:07:43.389 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_remote_ino": 0, 2024-09-11T22:07:43.390 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_lock": 3, 2024-09-11T22:07:43.390 INFO:teuthology.orchestra.run.smithi161.stdout: "load_cent": 0, 2024-09-11T22:07:43.390 INFO:teuthology.orchestra.run.smithi161.stdout: "q": 0, 2024-09-11T22:07:43.390 INFO:teuthology.orchestra.run.smithi161.stdout: "exported": 270, 2024-09-11T22:07:43.390 INFO:teuthology.orchestra.run.smithi161.stdout: "exported_inodes": 15202, 2024-09-11T22:07:43.390 INFO:teuthology.orchestra.run.smithi161.stdout: "imported": 416, 2024-09-11T22:07:43.390 INFO:teuthology.orchestra.run.smithi161.stdout: "imported_inodes": 23239, 2024-09-11T22:07:43.390 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_dir_fetch": 0, 2024-09-11T22:07:43.391 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_backtrace_fetch": 11042, 2024-09-11T22:07:43.391 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_peer_discover": 0, 2024-09-11T22:07:43.391 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rfiles": 4, 2024-09-11T22:07:43.391 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rbytes": 234, 2024-09-11T22:07:43.391 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rsnaps": 7, 2024-09-11T22:07:43.391 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_fetch": 11042, 2024-09-11T22:07:43.391 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_set_tag": 0, 2024-09-11T22:07:43.391 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T22:07:43.392 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_inotable_repaired": 0, 2024-09-11T22:07:43.392 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_inodes": 4, 2024-09-11T22:07:43.392 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T22:07:43.392 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dirfrag_rstats": 4, 2024-09-11T22:07:43.392 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_file_inodes": 11038, 2024-09-11T22:07:43.392 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_inode_file_caps": 10, 2024-09-11T22:07:43.392 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_revoke": 9017, 2024-09-11T22:07:43.393 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_grant": 13356, 2024-09-11T22:07:43.393 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T22:07:43.393 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T22:07:43.393 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T22:07:43.393 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps": 15848, 2024-09-11T22:07:43.393 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps_dirty": 12998, 2024-09-11T22:07:43.393 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_cap_release": 251, 2024-09-11T22:07:43.393 INFO:teuthology.orchestra.run.smithi161.stdout: "process_request_cap_release": 15436 2024-09-11T22:07:43.394 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:43.394 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_cache": { 2024-09-11T22:07:43.394 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update": 1393, 2024-09-11T22:07:43.394 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update_receipt": 3320, 2024-09-11T22:07:43.394 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_try_discover": 1664, 2024-09-11T22:07:43.394 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_send_discover": 3236, 2024-09-11T22:07:43.394 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_handle_discover": 4042, 2024-09-11T22:07:43.394 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays": 1, 2024-09-11T22:07:43.395 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_delayed": 0, 2024-09-11T22:07:43.395 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_enqueuing": 0, 2024-09-11T22:07:43.395 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_created": 7124, 2024-09-11T22:07:43.395 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_enqueued": 7123, 2024-09-11T22:07:43.395 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_reintegrated": 0, 2024-09-11T22:07:43.395 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_migrated": 0, 2024-09-11T22:07:43.395 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_processing": 0, 2024-09-11T22:07:43.396 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_enqueued": 0, 2024-09-11T22:07:43.396 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_prioritized": 0, 2024-09-11T22:07:43.396 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_started": 0, 2024-09-11T22:07:43.396 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_completed": 0, 2024-09-11T22:07:43.396 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_quiesce_path": 8, 2024-09-11T22:07:43.396 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_quiesce_inode": 15314, 2024-09-11T22:07:43.396 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T22:07:43.396 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_exportdir": 273, 2024-09-11T22:07:43.397 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_flush": 0, 2024-09-11T22:07:43.397 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragmentdir": 67, 2024-09-11T22:07:43.397 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragstats": 0, 2024-09-11T22:07:43.397 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_inodestats": 0 2024-09-11T22:07:43.397 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:43.397 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_log": { 2024-09-11T22:07:43.397 INFO:teuthology.orchestra.run.smithi161.stdout: "evlrg": 3, 2024-09-11T22:07:43.397 INFO:teuthology.orchestra.run.smithi161.stdout: "evadd": 36610, 2024-09-11T22:07:43.398 INFO:teuthology.orchestra.run.smithi161.stdout: "evex": 0, 2024-09-11T22:07:43.398 INFO:teuthology.orchestra.run.smithi161.stdout: "evtrm": 0, 2024-09-11T22:07:43.398 INFO:teuthology.orchestra.run.smithi161.stdout: "ev": 36610, 2024-09-11T22:07:43.398 INFO:teuthology.orchestra.run.smithi161.stdout: "evexg": 0, 2024-09-11T22:07:43.398 INFO:teuthology.orchestra.run.smithi161.stdout: "evexd": 0, 2024-09-11T22:07:43.398 INFO:teuthology.orchestra.run.smithi161.stdout: "segadd": 51, 2024-09-11T22:07:43.398 INFO:teuthology.orchestra.run.smithi161.stdout: "segex": 0, 2024-09-11T22:07:43.398 INFO:teuthology.orchestra.run.smithi161.stdout: "segtrm": 0, 2024-09-11T22:07:43.399 INFO:teuthology.orchestra.run.smithi161.stdout: "seg": 51, 2024-09-11T22:07:43.399 INFO:teuthology.orchestra.run.smithi161.stdout: "segmjr": 4, 2024-09-11T22:07:43.399 INFO:teuthology.orchestra.run.smithi161.stdout: "segexg": 0, 2024-09-11T22:07:43.399 INFO:teuthology.orchestra.run.smithi161.stdout: "segexd": 0, 2024-09-11T22:07:43.399 INFO:teuthology.orchestra.run.smithi161.stdout: "expos": 4194304, 2024-09-11T22:07:43.399 INFO:teuthology.orchestra.run.smithi161.stdout: "wrpos": 107392014, 2024-09-11T22:07:43.399 INFO:teuthology.orchestra.run.smithi161.stdout: "rdpos": 0, 2024-09-11T22:07:43.399 INFO:teuthology.orchestra.run.smithi161.stdout: "jlat": { 2024-09-11T22:07:43.400 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 7822, 2024-09-11T22:07:43.400 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 2768.496067095, 2024-09-11T22:07:43.400 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.353937109 2024-09-11T22:07:43.400 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:43.400 INFO:teuthology.orchestra.run.smithi161.stdout: "replayed": 0 2024-09-11T22:07:43.400 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:43.400 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_mem": { 2024-09-11T22:07:43.400 INFO:teuthology.orchestra.run.smithi161.stdout: "ino": 14109, 2024-09-11T22:07:43.401 INFO:teuthology.orchestra.run.smithi161.stdout: "ino+": 34627, 2024-09-11T22:07:43.401 INFO:teuthology.orchestra.run.smithi161.stdout: "ino-": 20518, 2024-09-11T22:07:43.401 INFO:teuthology.orchestra.run.smithi161.stdout: "dir": 559, 2024-09-11T22:07:43.401 INFO:teuthology.orchestra.run.smithi161.stdout: "dir+": 3176, 2024-09-11T22:07:43.401 INFO:teuthology.orchestra.run.smithi161.stdout: "dir-": 2617, 2024-09-11T22:07:43.401 INFO:teuthology.orchestra.run.smithi161.stdout: "dn": 24104, 2024-09-11T22:07:43.401 INFO:teuthology.orchestra.run.smithi161.stdout: "dn+": 49056, 2024-09-11T22:07:43.401 INFO:teuthology.orchestra.run.smithi161.stdout: "dn-": 24952, 2024-09-11T22:07:43.402 INFO:teuthology.orchestra.run.smithi161.stdout: "cap": 2, 2024-09-11T22:07:43.402 INFO:teuthology.orchestra.run.smithi161.stdout: "cap+": 18979, 2024-09-11T22:07:43.402 INFO:teuthology.orchestra.run.smithi161.stdout: "cap-": 18977, 2024-09-11T22:07:43.402 INFO:teuthology.orchestra.run.smithi161.stdout: "rss": 194536, 2024-09-11T22:07:43.402 INFO:teuthology.orchestra.run.smithi161.stdout: "heap": 223520 2024-09-11T22:07:43.402 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:43.402 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_server": { 2024-09-11T22:07:43.402 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_client_request": 29071, 2024-09-11T22:07:43.403 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_server_request": 13787, 2024-09-11T22:07:43.403 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_request": 24803, 2024-09-11T22:07:43.403 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_session": 203, 2024-09-11T22:07:43.403 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_peer_request": 30916, 2024-09-11T22:07:43.403 INFO:teuthology.orchestra.run.smithi161.stdout: "req_create_latency": { 2024-09-11T22:07:43.403 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 4808, 2024-09-11T22:07:43.403 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 34.542060366, 2024-09-11T22:07:43.403 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.007184288 2024-09-11T22:07:43.404 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:43.404 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getattr_latency": { 2024-09-11T22:07:43.404 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 4, 2024-09-11T22:07:43.404 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.042921238, 2024-09-11T22:07:43.404 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.010730309 2024-09-11T22:07:43.404 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:43.404 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getfilelock_latency": { 2024-09-11T22:07:43.405 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:43.405 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:43.405 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:43.405 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:43.405 INFO:teuthology.orchestra.run.smithi161.stdout: "req_link_latency": { 2024-09-11T22:07:43.405 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:43.405 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:43.405 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:43.406 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:43.406 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookup_latency": { 2024-09-11T22:07:43.406 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 6662, 2024-09-11T22:07:43.406 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 16.378267328, 2024-09-11T22:07:43.406 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.002458461 2024-09-11T22:07:43.406 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:43.406 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookuphash_latency": { 2024-09-11T22:07:43.406 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:43.407 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:43.407 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:43.407 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:43.407 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupino_latency": { 2024-09-11T22:07:43.407 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1, 2024-09-11T22:07:43.407 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000116418, 2024-09-11T22:07:43.407 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000116418 2024-09-11T22:07:43.407 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:43.407 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupname_latency": { 2024-09-11T22:07:43.408 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:43.408 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:43.408 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:43.408 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:43.408 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupparent_latency": { 2024-09-11T22:07:43.408 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:43.408 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:43.408 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:43.409 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:43.409 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupsnap_latency": { 2024-09-11T22:07:43.409 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:43.409 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:43.410 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:43.410 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:43.410 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lssnap_latency": { 2024-09-11T22:07:43.410 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:43.410 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:43.410 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:43.411 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:43.411 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mkdir_latency": { 2024-09-11T22:07:43.411 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:43.411 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:43.411 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:43.411 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:43.411 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mknod_latency": { 2024-09-11T22:07:43.411 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:43.412 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:43.412 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:43.412 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:43.412 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mksnap_latency": { 2024-09-11T22:07:43.412 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:43.412 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:43.412 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:43.413 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:43.413 INFO:teuthology.orchestra.run.smithi161.stdout: "req_open_latency": { 2024-09-11T22:07:43.413 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:43.413 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:43.413 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:43.413 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:43.413 INFO:teuthology.orchestra.run.smithi161.stdout: "req_readdir_latency": { 2024-09-11T22:07:43.413 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1053, 2024-09-11T22:07:43.414 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 1.739163863, 2024-09-11T22:07:43.414 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.001651627 2024-09-11T22:07:43.414 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:43.414 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rename_latency": { 2024-09-11T22:07:43.414 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:43.414 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:43.414 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:43.414 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:43.415 INFO:teuthology.orchestra.run.smithi161.stdout: "req_renamesnap_latency": { 2024-09-11T22:07:43.415 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:43.415 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:43.415 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:43.415 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:43.415 INFO:teuthology.orchestra.run.smithi161.stdout: "req_snapdiff_latency": { 2024-09-11T22:07:43.415 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:43.415 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:43.416 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:43.416 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:43.416 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmdir_latency": { 2024-09-11T22:07:43.416 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1, 2024-09-11T22:07:43.416 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.069186554, 2024-09-11T22:07:43.416 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.069186554 2024-09-11T22:07:43.416 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:43.416 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmsnap_latency": { 2024-09-11T22:07:43.417 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:43.417 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:43.417 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:43.417 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:43.417 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmxattr_latency": { 2024-09-11T22:07:43.417 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:43.417 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:43.418 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:43.418 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:43.418 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setattr_latency": { 2024-09-11T22:07:43.418 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:43.418 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:43.418 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:43.418 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:43.418 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setdirlayout_latency": { 2024-09-11T22:07:43.419 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:43.419 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:43.419 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:43.419 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:43.419 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setfilelock_latency": { 2024-09-11T22:07:43.419 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:43.420 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:43.420 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:43.420 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:43.420 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setlayout_latency": { 2024-09-11T22:07:43.420 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:43.420 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:43.420 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:43.420 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:43.421 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setxattr_latency": { 2024-09-11T22:07:43.421 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:43.421 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:43.421 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:43.421 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:43.421 INFO:teuthology.orchestra.run.smithi161.stdout: "req_symlink_latency": { 2024-09-11T22:07:43.421 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:43.421 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:43.422 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:43.422 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:43.422 INFO:teuthology.orchestra.run.smithi161.stdout: "req_unlink_latency": { 2024-09-11T22:07:43.422 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 7123, 2024-09-11T22:07:43.422 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 22.973660913, 2024-09-11T22:07:43.422 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.003225278 2024-09-11T22:07:43.422 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:43.422 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_revoke_eviction": 0, 2024-09-11T22:07:43.423 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_acquisition_throttle": 0, 2024-09-11T22:07:43.423 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getvxattr_latency": { 2024-09-11T22:07:43.423 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:43.423 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:43.423 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:43.423 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-11T22:07:43.423 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:43.423 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_sessions": { 2024-09-11T22:07:43.424 INFO:teuthology.orchestra.run.smithi161.stdout: "session_count": 2, 2024-09-11T22:07:43.424 INFO:teuthology.orchestra.run.smithi161.stdout: "session_add": 4, 2024-09-11T22:07:43.424 INFO:teuthology.orchestra.run.smithi161.stdout: "session_remove": 2, 2024-09-11T22:07:43.424 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_open": 2, 2024-09-11T22:07:43.424 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_stale": 0, 2024-09-11T22:07:43.424 INFO:teuthology.orchestra.run.smithi161.stdout: "total_load": 4358, 2024-09-11T22:07:43.424 INFO:teuthology.orchestra.run.smithi161.stdout: "average_load": 2179, 2024-09-11T22:07:43.424 INFO:teuthology.orchestra.run.smithi161.stdout: "avg_session_uptime": 593, 2024-09-11T22:07:43.425 INFO:teuthology.orchestra.run.smithi161.stdout: "mdthresh_evicted": 0 2024-09-11T22:07:43.425 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:43.425 INFO:teuthology.orchestra.run.smithi161.stdout: "mempool": { 2024-09-11T22:07:43.425 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_bytes": 0, 2024-09-11T22:07:43.425 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_items": 0, 2024-09-11T22:07:43.425 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T22:07:43.425 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_items": 0, 2024-09-11T22:07:43.425 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T22:07:43.426 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_items": 0, 2024-09-11T22:07:43.426 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T22:07:43.426 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T22:07:43.426 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T22:07:43.426 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T22:07:43.426 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T22:07:43.426 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_items": 0, 2024-09-11T22:07:43.426 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T22:07:43.427 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T22:07:43.427 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_bytes": 0, 2024-09-11T22:07:43.427 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_items": 0, 2024-09-11T22:07:43.427 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_bytes": 0, 2024-09-11T22:07:43.427 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_items": 0, 2024-09-11T22:07:43.427 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T22:07:43.427 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T22:07:43.427 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T22:07:43.428 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T22:07:43.428 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T22:07:43.428 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_items": 0, 2024-09-11T22:07:43.428 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_bytes": 0, 2024-09-11T22:07:43.428 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_items": 0, 2024-09-11T22:07:43.428 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T22:07:43.428 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T22:07:43.429 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_bytes": 0, 2024-09-11T22:07:43.429 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_items": 0, 2024-09-11T22:07:43.429 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_bytes": 0, 2024-09-11T22:07:43.429 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_items": 0, 2024-09-11T22:07:43.429 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T22:07:43.429 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_items": 0, 2024-09-11T22:07:43.429 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T22:07:43.429 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_items": 0, 2024-09-11T22:07:43.430 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_bytes": 714110, 2024-09-11T22:07:43.430 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_items": 11415, 2024-09-11T22:07:43.430 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_bytes": 0, 2024-09-11T22:07:43.430 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_items": 0, 2024-09-11T22:07:43.430 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_bytes": 0, 2024-09-11T22:07:43.430 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_items": 0, 2024-09-11T22:07:43.430 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_bytes": 0, 2024-09-11T22:07:43.430 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_items": 0, 2024-09-11T22:07:43.431 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_bytes": 0, 2024-09-11T22:07:43.431 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_items": 0, 2024-09-11T22:07:43.431 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_bytes": 8392, 2024-09-11T22:07:43.431 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_items": 184, 2024-09-11T22:07:43.431 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T22:07:43.431 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_items": 0, 2024-09-11T22:07:43.431 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_bytes": 0, 2024-09-11T22:07:43.431 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_items": 0, 2024-09-11T22:07:43.432 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_bytes": 49898260, 2024-09-11T22:07:43.432 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_items": 424924, 2024-09-11T22:07:43.432 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_bytes": 0, 2024-09-11T22:07:43.432 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_items": 0, 2024-09-11T22:07:43.432 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_bytes": 0, 2024-09-11T22:07:43.432 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_items": 0 2024-09-11T22:07:43.432 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:43.432 INFO:teuthology.orchestra.run.smithi161.stdout: "objecter": { 2024-09-11T22:07:43.433 INFO:teuthology.orchestra.run.smithi161.stdout: "op_active": 0, 2024-09-11T22:07:43.433 INFO:teuthology.orchestra.run.smithi161.stdout: "op_laggy": 0, 2024-09-11T22:07:43.433 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send": 27396, 2024-09-11T22:07:43.433 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send_bytes": 111285767, 2024-09-11T22:07:43.433 INFO:teuthology.orchestra.run.smithi161.stdout: "op_resend": 29, 2024-09-11T22:07:43.433 INFO:teuthology.orchestra.run.smithi161.stdout: "op_reply": 27396, 2024-09-11T22:07:43.433 INFO:teuthology.orchestra.run.smithi161.stdout: "op_latency": { 2024-09-11T22:07:43.433 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 27396, 2024-09-11T22:07:43.434 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 2702.664578209, 2024-09-11T22:07:43.434 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.098651795 2024-09-11T22:07:43.434 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:43.434 INFO:teuthology.orchestra.run.smithi161.stdout: "op_inflight": 0, 2024-09-11T22:07:43.434 INFO:teuthology.orchestra.run.smithi161.stdout: "oplen_avg": { 2024-09-11T22:07:43.434 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 27396, 2024-09-11T22:07:43.434 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 28451 2024-09-11T22:07:43.434 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:43.435 INFO:teuthology.orchestra.run.smithi161.stdout: "op": 27396, 2024-09-11T22:07:43.435 INFO:teuthology.orchestra.run.smithi161.stdout: "op_r": 11467, 2024-09-11T22:07:43.435 INFO:teuthology.orchestra.run.smithi161.stdout: "op_w": 15929, 2024-09-11T22:07:43.435 INFO:teuthology.orchestra.run.smithi161.stdout: "op_rmw": 0, 2024-09-11T22:07:43.435 INFO:teuthology.orchestra.run.smithi161.stdout: "op_pg": 0, 2024-09-11T22:07:43.435 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_stat": 0, 2024-09-11T22:07:43.435 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_create": 10, 2024-09-11T22:07:43.435 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_read": 121, 2024-09-11T22:07:43.436 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_write": 7943, 2024-09-11T22:07:43.436 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writefull": 61, 2024-09-11T22:07:43.436 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writesame": 0, 2024-09-11T22:07:43.436 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_append": 0, 2024-09-11T22:07:43.436 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_zero": 0, 2024-09-11T22:07:43.436 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_truncate": 0, 2024-09-11T22:07:43.436 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_delete": 7226, 2024-09-11T22:07:43.437 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_mapext": 0, 2024-09-11T22:07:43.437 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_sparse_read": 0, 2024-09-11T22:07:43.437 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_clonerange": 0, 2024-09-11T22:07:43.437 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_getxattr": 11042, 2024-09-11T22:07:43.437 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_setxattr": 20, 2024-09-11T22:07:43.437 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_cmpxattr": 0, 2024-09-11T22:07:43.437 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_rmxattr": 0, 2024-09-11T22:07:43.437 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_resetxattrs": 0, 2024-09-11T22:07:43.438 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_call": 0, 2024-09-11T22:07:43.438 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_watch": 0, 2024-09-11T22:07:43.438 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_notify": 0, 2024-09-11T22:07:43.438 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T22:07:43.438 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls": 0, 2024-09-11T22:07:43.438 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls_filter": 0, 2024-09-11T22:07:43.438 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_other": 0, 2024-09-11T22:07:43.438 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_active": 0, 2024-09-11T22:07:43.439 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_send": 0, 2024-09-11T22:07:43.439 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_resend": 0, 2024-09-11T22:07:43.439 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_ping": 0, 2024-09-11T22:07:43.439 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_active": 0, 2024-09-11T22:07:43.439 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_send": 0, 2024-09-11T22:07:43.439 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_resend": 0, 2024-09-11T22:07:43.439 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_active": 0, 2024-09-11T22:07:43.439 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_send": 0, 2024-09-11T22:07:43.440 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_resend": 0, 2024-09-11T22:07:43.440 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_active": 0, 2024-09-11T22:07:43.440 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_send": 0, 2024-09-11T22:07:43.440 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_resend": 0, 2024-09-11T22:07:43.440 INFO:teuthology.orchestra.run.smithi161.stdout: "command_active": 0, 2024-09-11T22:07:43.440 INFO:teuthology.orchestra.run.smithi161.stdout: "command_send": 0, 2024-09-11T22:07:43.440 INFO:teuthology.orchestra.run.smithi161.stdout: "command_resend": 0, 2024-09-11T22:07:43.440 INFO:teuthology.orchestra.run.smithi161.stdout: "map_epoch": 95, 2024-09-11T22:07:43.441 INFO:teuthology.orchestra.run.smithi161.stdout: "map_full": 0, 2024-09-11T22:07:43.441 INFO:teuthology.orchestra.run.smithi161.stdout: "map_inc": 11, 2024-09-11T22:07:43.441 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_sessions": 12, 2024-09-11T22:07:43.441 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_open": 12, 2024-09-11T22:07:43.441 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_close": 0, 2024-09-11T22:07:43.441 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_laggy": 0, 2024-09-11T22:07:43.441 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_wr": 1348, 2024-09-11T22:07:43.441 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_rd": 608, 2024-09-11T22:07:43.442 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_del": 72 2024-09-11T22:07:43.442 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:43.442 INFO:teuthology.orchestra.run.smithi161.stdout: "oft": { 2024-09-11T22:07:43.442 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_objs": 1, 2024-09-11T22:07:43.442 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_kv_pairs": 0, 2024-09-11T22:07:43.442 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_updates": 0, 2024-09-11T22:07:43.442 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_removes": 0 2024-09-11T22:07:43.442 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:43.443 INFO:teuthology.orchestra.run.smithi161.stdout: "osd-slow-ops": { 2024-09-11T22:07:43.443 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_ops_count": 15306 2024-09-11T22:07:43.443 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:43.443 INFO:teuthology.orchestra.run.smithi161.stdout: "purge_queue": { 2024-09-11T22:07:43.443 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops": 0, 2024-09-11T22:07:43.443 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops_high_water": 6, 2024-09-11T22:07:43.443 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing": 0, 2024-09-11T22:07:43.443 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_high_water": 6, 2024-09-11T22:07:43.444 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed_ops": 7123, 2024-09-11T22:07:43.444 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed": 7123, 2024-09-11T22:07:43.444 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_item_in_journal": 0 2024-09-11T22:07:43.444 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:43.444 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T22:07:43.444 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-09-11T22:07:43.444 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-09-11T22:07:43.444 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-09-11T22:07:43.445 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 134029, 2024-09-11T22:07:43.445 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 62522349, 2024-09-11T22:07:43.445 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-09-11T22:07:43.445 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 134029, 2024-09-11T22:07:43.445 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-09-11T22:07:43.445 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-09-11T22:07:43.445 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 134029, 2024-09-11T22:07:43.445 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 62522349, 2024-09-11T22:07:43.446 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-09-11T22:07:43.446 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:43.446 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:43.446 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:43.446 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-11T22:07:43.446 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:43.446 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_bytes": { 2024-09-11T22:07:43.446 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-09-11T22:07:43.447 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-09-11T22:07:43.447 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-09-11T22:07:43.447 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-09-11T22:07:43.447 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-09-11T22:07:43.447 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-09-11T22:07:43.447 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-09-11T22:07:43.447 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 27396, 2024-09-11T22:07:43.447 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 112086630, 2024-09-11T22:07:43.448 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 27396, 2024-09-11T22:07:43.448 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 112086630, 2024-09-11T22:07:43.448 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-09-11T22:07:43.448 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:43.448 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:43.448 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:43.448 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-11T22:07:43.448 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:43.449 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_ops": { 2024-09-11T22:07:43.449 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-09-11T22:07:43.449 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 1024, 2024-09-11T22:07:43.449 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-09-11T22:07:43.449 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-09-11T22:07:43.449 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-09-11T22:07:43.449 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-09-11T22:07:43.450 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-09-11T22:07:43.450 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 27396, 2024-09-11T22:07:43.450 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 27396, 2024-09-11T22:07:43.450 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 27396, 2024-09-11T22:07:43.450 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 27396, 2024-09-11T22:07:43.450 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-09-11T22:07:43.450 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:43.450 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:43.451 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:43.451 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-11T22:07:43.451 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:43.451 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle": { 2024-09-11T22:07:43.451 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-09-11T22:07:43.451 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-09-11T22:07:43.451 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-09-11T22:07:43.451 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 7123, 2024-09-11T22:07:43.452 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 805727, 2024-09-11T22:07:43.452 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-09-11T22:07:43.452 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 7123, 2024-09-11T22:07:43.452 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-09-11T22:07:43.452 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-09-11T22:07:43.452 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 121, 2024-09-11T22:07:43.452 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 805727, 2024-09-11T22:07:43.452 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-09-11T22:07:43.453 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:43.453 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:43.453 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:43.453 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-11T22:07:43.453 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:43.453 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle-0x559a5a2ec500": { 2024-09-11T22:07:43.453 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-09-11T22:07:43.453 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-09-11T22:07:43.454 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-09-11T22:07:43.454 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 36610, 2024-09-11T22:07:43.454 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 103197710, 2024-09-11T22:07:43.454 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-09-11T22:07:43.454 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 36610, 2024-09-11T22:07:43.454 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-09-11T22:07:43.454 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-09-11T22:07:43.454 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 7822, 2024-09-11T22:07:43.455 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 103197710, 2024-09-11T22:07:43.455 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-09-11T22:07:43.455 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:43.455 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:43.455 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:43.455 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-11T22:07:43.455 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-11T22:07:43.455 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-11T22:07:43.456 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:43 smithi001 ceph-mon[30935]: pgmap v325: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:07:43.456 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:43 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T22:07:43.456 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:43 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:43.456 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:43 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:43.457 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:43 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:43.457 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:43 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:43.457 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:43 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:43.457 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:43 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:43.457 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:43 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:43.457 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:43 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:43.457 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:43 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:07:43.457 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:43 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:07:43.458 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:43 smithi001 ceph-mon[30935]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T22:07:43.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:43 smithi053 ceph-mon[34908]: pgmap v325: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:07:43.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:43 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T22:07:43.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:43 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:43.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:43 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:43.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:43 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:43.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:43 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:43.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:43 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:43.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:43 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:43.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:43 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:43.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:43 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:07:43.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:43 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:07:43.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:43 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:07:43.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:43 smithi053 ceph-mon[34908]: from='mgr.15204 ' entity='mgr.x' 2024-09-11T22:07:44.209 INFO:tasks.check_counter:Daemon mds.c mds.exported=270 2024-09-11T22:07:44.209 INFO:tasks.check_counter:Daemon mds.c mds.imported=416 2024-09-11T22:07:44.210 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=1393 2024-09-11T22:07:44.210 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=3320 2024-09-11T22:07:44.210 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=7 2024-09-11T22:07:44.210 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-09-11T22:07:44.210 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-09-11T22:07:44.210 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=67 2024-09-11T22:07:44.210 DEBUG:tasks.check_counter:Getting stats from f 2024-09-11T22:07:44.211 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-11T22:07:44.341 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:44 smithi161 ceph-mon[35615]: Adjusting osd_memory_target on smithi053 to 177.0M 2024-09-11T22:07:44.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:44 smithi161 ceph-mon[35615]: Unable to set osd_memory_target on smithi053 to 185699942: error parsing value: Value '185699942' is below minimum 939524096 2024-09-11T22:07:44.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:44 smithi001 ceph-mon[30935]: Adjusting osd_memory_target on smithi053 to 177.0M 2024-09-11T22:07:44.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:44 smithi001 ceph-mon[30935]: Unable to set osd_memory_target on smithi053 to 185699942: error parsing value: Value '185699942' is below minimum 939524096 2024-09-11T22:07:44.465 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T22:07:44.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:44 smithi053 ceph-mon[34908]: Adjusting osd_memory_target on smithi053 to 177.0M 2024-09-11T22:07:44.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:44 smithi053 ceph-mon[34908]: Unable to set osd_memory_target on smithi053 to 185699942: error parsing value: Value '185699942' is below minimum 939524096 2024-09-11T22:07:45.052 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-11T22:07:45.052 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T22:07:45.053 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 82903, 2024-09-11T22:07:45.053 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 133349, 2024-09-11T22:07:45.053 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 39728221, 2024-09-11T22:07:45.053 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 111381429, 2024-09-11T22:07:45.054 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 16, 2024-09-11T22:07:45.054 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 6, 2024-09-11T22:07:45.054 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 5.534139488, 2024-09-11T22:07:45.054 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 2.424447672, 2024-09-11T22:07:45.054 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 34.788518148, 2024-09-11T22:07:45.055 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.466080979, 2024-09-11T22:07:45.055 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:07:45.055 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 133346, 2024-09-11T22:07:45.055 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 13.427665031, 2024-09-11T22:07:45.055 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000100697 2024-09-11T22:07:45.056 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:45.056 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:07:45.056 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 159311, 2024-09-11T22:07:45.056 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.033918900, 2024-09-11T22:07:45.056 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000212 2024-09-11T22:07:45.056 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:45.056 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 1788144, 2024-09-11T22:07:45.057 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 211536 2024-09-11T22:07:45.057 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:45.057 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T22:07:45.057 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 37355, 2024-09-11T22:07:45.057 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 54070, 2024-09-11T22:07:45.057 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 29521774, 2024-09-11T22:07:45.058 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 44882202, 2024-09-11T22:07:45.058 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 20, 2024-09-11T22:07:45.058 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 6, 2024-09-11T22:07:45.058 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 2.870565093, 2024-09-11T22:07:45.058 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 1.153023072, 2024-09-11T22:07:45.058 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 1.342671025, 2024-09-11T22:07:45.058 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.439725232, 2024-09-11T22:07:45.059 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:07:45.059 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 54070, 2024-09-11T22:07:45.059 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 5.729516373, 2024-09-11T22:07:45.059 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000105964 2024-09-11T22:07:45.059 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:45.059 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:07:45.059 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 55159, 2024-09-11T22:07:45.060 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.013267524, 2024-09-11T22:07:45.060 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000240 2024-09-11T22:07:45.060 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:45.060 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 1088, 2024-09-11T22:07:45.060 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 124768 2024-09-11T22:07:45.060 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:45.060 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T22:07:45.061 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 35759, 2024-09-11T22:07:45.061 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 43118, 2024-09-11T22:07:45.061 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 22778241, 2024-09-11T22:07:45.061 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 81738450, 2024-09-11T22:07:45.061 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 23, 2024-09-11T22:07:45.061 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 9, 2024-09-11T22:07:45.061 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 3.542757778, 2024-09-11T22:07:45.062 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 1.852424606, 2024-09-11T22:07:45.062 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 1.310135792, 2024-09-11T22:07:45.062 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.489072909, 2024-09-11T22:07:45.062 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:07:45.062 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 43116, 2024-09-11T22:07:45.062 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 3.750588774, 2024-09-11T22:07:45.063 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000086988 2024-09-11T22:07:45.063 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:45.063 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:07:45.063 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 53817, 2024-09-11T22:07:45.063 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.010251263, 2024-09-11T22:07:45.063 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000190 2024-09-11T22:07:45.063 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:45.064 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-11T22:07:45.064 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 316368 2024-09-11T22:07:45.064 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:45.064 INFO:teuthology.orchestra.run.smithi161.stdout: "cct": { 2024-09-11T22:07:45.064 INFO:teuthology.orchestra.run.smithi161.stdout: "total_workers": 1, 2024-09-11T22:07:45.064 INFO:teuthology.orchestra.run.smithi161.stdout: "unhealthy_workers": 0 2024-09-11T22:07:45.065 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:45.065 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-MDSRank": { 2024-09-11T22:07:45.065 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-09-11T22:07:45.065 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-09-11T22:07:45.065 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 18392, 2024-09-11T22:07:45.065 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 46.929030182, 2024-09-11T22:07:45.065 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.002551600 2024-09-11T22:07:45.066 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-11T22:07:45.066 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:45.066 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-PurgeQueue": { 2024-09-11T22:07:45.066 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-09-11T22:07:45.066 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-09-11T22:07:45.066 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 21363, 2024-09-11T22:07:45.066 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 4.995543908, 2024-09-11T22:07:45.066 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000233840 2024-09-11T22:07:45.066 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-11T22:07:45.067 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:45.067 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": { 2024-09-11T22:07:45.067 INFO:teuthology.orchestra.run.smithi161.stdout: "request": 31274, 2024-09-11T22:07:45.067 INFO:teuthology.orchestra.run.smithi161.stdout: "reply": 30993, 2024-09-11T22:07:45.067 INFO:teuthology.orchestra.run.smithi161.stdout: "reply_latency": { 2024-09-11T22:07:45.067 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 30993, 2024-09-11T22:07:45.067 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 168.017896701, 2024-09-11T22:07:45.068 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.005421156 2024-09-11T22:07:45.068 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:45.068 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_reply": 0, 2024-09-11T22:07:45.068 INFO:teuthology.orchestra.run.smithi161.stdout: "forward": 281, 2024-09-11T22:07:45.068 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_complete": 558, 2024-09-11T22:07:45.068 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_keys": 0, 2024-09-11T22:07:45.068 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_commit": 515, 2024-09-11T22:07:45.068 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_split": 63, 2024-09-11T22:07:45.069 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_merge": 0, 2024-09-11T22:07:45.069 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes": 26588, 2024-09-11T22:07:45.069 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_top": 4, 2024-09-11T22:07:45.069 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_bottom": 8, 2024-09-11T22:07:45.069 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pin_tail": 26576, 2024-09-11T22:07:45.069 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pinned": 26581, 2024-09-11T22:07:45.069 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_expired": 34993, 2024-09-11T22:07:45.069 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_with_caps": 1, 2024-09-11T22:07:45.070 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": 1, 2024-09-11T22:07:45.070 INFO:teuthology.orchestra.run.smithi161.stdout: "subtrees": 5, 2024-09-11T22:07:45.070 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse": 65110, 2024-09-11T22:07:45.070 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_hit": 45030, 2024-09-11T22:07:45.070 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_forward": 47, 2024-09-11T22:07:45.070 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_discover": 17, 2024-09-11T22:07:45.070 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_dir_fetch": 105, 2024-09-11T22:07:45.070 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_remote_ino": 0, 2024-09-11T22:07:45.071 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_lock": 0, 2024-09-11T22:07:45.071 INFO:teuthology.orchestra.run.smithi161.stdout: "load_cent": 0, 2024-09-11T22:07:45.071 INFO:teuthology.orchestra.run.smithi161.stdout: "q": 0, 2024-09-11T22:07:45.071 INFO:teuthology.orchestra.run.smithi161.stdout: "exported": 629, 2024-09-11T22:07:45.071 INFO:teuthology.orchestra.run.smithi161.stdout: "exported_inodes": 29894, 2024-09-11T22:07:45.071 INFO:teuthology.orchestra.run.smithi161.stdout: "imported": 597, 2024-09-11T22:07:45.071 INFO:teuthology.orchestra.run.smithi161.stdout: "imported_inodes": 34683, 2024-09-11T22:07:45.072 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_dir_fetch": 0, 2024-09-11T22:07:45.072 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_backtrace_fetch": 18556, 2024-09-11T22:07:45.072 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_peer_discover": 0, 2024-09-11T22:07:45.072 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rfiles": 4, 2024-09-11T22:07:45.073 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rbytes": 234, 2024-09-11T22:07:45.073 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rsnaps": 7, 2024-09-11T22:07:45.073 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_fetch": 18556, 2024-09-11T22:07:45.073 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_set_tag": 0, 2024-09-11T22:07:45.074 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T22:07:45.074 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_inotable_repaired": 0, 2024-09-11T22:07:45.074 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_inodes": 2, 2024-09-11T22:07:45.075 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T22:07:45.075 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dirfrag_rstats": 2, 2024-09-11T22:07:45.075 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_file_inodes": 18554, 2024-09-11T22:07:45.075 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_inode_file_caps": 4, 2024-09-11T22:07:45.076 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_revoke": 9775, 2024-09-11T22:07:45.076 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_grant": 19693, 2024-09-11T22:07:45.076 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T22:07:45.077 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T22:07:45.077 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T22:07:45.077 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps": 19030, 2024-09-11T22:07:45.077 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps_dirty": 16226, 2024-09-11T22:07:45.077 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_cap_release": 462, 2024-09-11T22:07:45.077 INFO:teuthology.orchestra.run.smithi161.stdout: "process_request_cap_release": 25285 2024-09-11T22:07:45.077 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:45.078 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_cache": { 2024-09-11T22:07:45.078 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update": 1203, 2024-09-11T22:07:45.078 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update_receipt": 4182, 2024-09-11T22:07:45.078 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_try_discover": 2098, 2024-09-11T22:07:45.078 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_send_discover": 4089, 2024-09-11T22:07:45.078 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_handle_discover": 2173, 2024-09-11T22:07:45.078 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays": 1, 2024-09-11T22:07:45.079 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_delayed": 0, 2024-09-11T22:07:45.079 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_enqueuing": 0, 2024-09-11T22:07:45.079 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_created": 12379, 2024-09-11T22:07:45.079 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_enqueued": 12378, 2024-09-11T22:07:45.079 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_reintegrated": 0, 2024-09-11T22:07:45.079 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_migrated": 0, 2024-09-11T22:07:45.080 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_processing": 0, 2024-09-11T22:07:45.080 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_enqueued": 0, 2024-09-11T22:07:45.080 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_prioritized": 0, 2024-09-11T22:07:45.080 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_started": 0, 2024-09-11T22:07:45.080 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_completed": 0, 2024-09-11T22:07:45.080 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_quiesce_path": 8, 2024-09-11T22:07:45.080 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_quiesce_inode": 19143, 2024-09-11T22:07:45.080 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T22:07:45.081 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_exportdir": 631, 2024-09-11T22:07:45.081 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_flush": 0, 2024-09-11T22:07:45.081 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragmentdir": 67, 2024-09-11T22:07:45.081 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragstats": 0, 2024-09-11T22:07:45.081 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_inodestats": 0 2024-09-11T22:07:45.081 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:45.081 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_log": { 2024-09-11T22:07:45.082 INFO:teuthology.orchestra.run.smithi161.stdout: "evlrg": 5, 2024-09-11T22:07:45.082 INFO:teuthology.orchestra.run.smithi161.stdout: "evadd": 62942, 2024-09-11T22:07:45.082 INFO:teuthology.orchestra.run.smithi161.stdout: "evex": 0, 2024-09-11T22:07:45.082 INFO:teuthology.orchestra.run.smithi161.stdout: "evtrm": 0, 2024-09-11T22:07:45.082 INFO:teuthology.orchestra.run.smithi161.stdout: "ev": 62942, 2024-09-11T22:07:45.082 INFO:teuthology.orchestra.run.smithi161.stdout: "evexg": 0, 2024-09-11T22:07:45.082 INFO:teuthology.orchestra.run.smithi161.stdout: "evexd": 0, 2024-09-11T22:07:45.083 INFO:teuthology.orchestra.run.smithi161.stdout: "segadd": 79, 2024-09-11T22:07:45.083 INFO:teuthology.orchestra.run.smithi161.stdout: "segex": 0, 2024-09-11T22:07:45.083 INFO:teuthology.orchestra.run.smithi161.stdout: "segtrm": 0, 2024-09-11T22:07:45.083 INFO:teuthology.orchestra.run.smithi161.stdout: "seg": 79, 2024-09-11T22:07:45.083 INFO:teuthology.orchestra.run.smithi161.stdout: "segmjr": 7, 2024-09-11T22:07:45.083 INFO:teuthology.orchestra.run.smithi161.stdout: "segexg": 0, 2024-09-11T22:07:45.083 INFO:teuthology.orchestra.run.smithi161.stdout: "segexd": 0, 2024-09-11T22:07:45.084 INFO:teuthology.orchestra.run.smithi161.stdout: "expos": 4194304, 2024-09-11T22:07:45.084 INFO:teuthology.orchestra.run.smithi161.stdout: "wrpos": 127143670, 2024-09-11T22:07:45.084 INFO:teuthology.orchestra.run.smithi161.stdout: "rdpos": 0, 2024-09-11T22:07:45.084 INFO:teuthology.orchestra.run.smithi161.stdout: "jlat": { 2024-09-11T22:07:45.084 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 9417, 2024-09-11T22:07:45.084 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 6257.488973991, 2024-09-11T22:07:45.085 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.664488581 2024-09-11T22:07:45.085 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:45.085 INFO:teuthology.orchestra.run.smithi161.stdout: "replayed": 0 2024-09-11T22:07:45.085 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:45.085 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_mem": { 2024-09-11T22:07:45.085 INFO:teuthology.orchestra.run.smithi161.stdout: "ino": 16595, 2024-09-11T22:07:45.085 INFO:teuthology.orchestra.run.smithi161.stdout: "ino+": 51186, 2024-09-11T22:07:45.086 INFO:teuthology.orchestra.run.smithi161.stdout: "ino-": 34591, 2024-09-11T22:07:45.086 INFO:teuthology.orchestra.run.smithi161.stdout: "dir": 538, 2024-09-11T22:07:45.086 INFO:teuthology.orchestra.run.smithi161.stdout: "dir+": 3287, 2024-09-11T22:07:45.086 INFO:teuthology.orchestra.run.smithi161.stdout: "dir-": 2749, 2024-09-11T22:07:45.086 INFO:teuthology.orchestra.run.smithi161.stdout: "dn": 26588, 2024-09-11T22:07:45.086 INFO:teuthology.orchestra.run.smithi161.stdout: "dn+": 73959, 2024-09-11T22:07:45.086 INFO:teuthology.orchestra.run.smithi161.stdout: "dn-": 47371, 2024-09-11T22:07:45.087 INFO:teuthology.orchestra.run.smithi161.stdout: "cap": 1, 2024-09-11T22:07:45.087 INFO:teuthology.orchestra.run.smithi161.stdout: "cap+": 24983, 2024-09-11T22:07:45.087 INFO:teuthology.orchestra.run.smithi161.stdout: "cap-": 24982, 2024-09-11T22:07:45.087 INFO:teuthology.orchestra.run.smithi161.stdout: "rss": 189072, 2024-09-11T22:07:45.087 INFO:teuthology.orchestra.run.smithi161.stdout: "heap": 223520 2024-09-11T22:07:45.087 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:45.087 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_server": { 2024-09-11T22:07:45.087 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_client_request": 53825, 2024-09-11T22:07:45.087 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_server_request": 7617, 2024-09-11T22:07:45.088 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_request": 31274, 2024-09-11T22:07:45.088 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_session": 21, 2024-09-11T22:07:45.088 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_peer_request": 33923, 2024-09-11T22:07:45.088 INFO:teuthology.orchestra.run.smithi161.stdout: "req_create_latency": { 2024-09-11T22:07:45.088 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 6343, 2024-09-11T22:07:45.088 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 71.618468718, 2024-09-11T22:07:45.088 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.011290945 2024-09-11T22:07:45.088 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:45.089 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getattr_latency": { 2024-09-11T22:07:45.089 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 2, 2024-09-11T22:07:45.089 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.001487927, 2024-09-11T22:07:45.089 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000743963 2024-09-11T22:07:45.089 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:45.089 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getfilelock_latency": { 2024-09-11T22:07:45.090 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:45.090 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:45.090 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:45.090 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:45.090 INFO:teuthology.orchestra.run.smithi161.stdout: "req_link_latency": { 2024-09-11T22:07:45.090 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:45.090 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:45.091 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:45.091 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:45.091 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookup_latency": { 2024-09-11T22:07:45.091 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 11699, 2024-09-11T22:07:45.091 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 15.156883177, 2024-09-11T22:07:45.091 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.001295570 2024-09-11T22:07:45.091 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:45.091 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookuphash_latency": { 2024-09-11T22:07:45.092 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:45.092 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:45.092 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:45.092 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:45.092 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupino_latency": { 2024-09-11T22:07:45.092 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1, 2024-09-11T22:07:45.093 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000108767, 2024-09-11T22:07:45.093 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000108767 2024-09-11T22:07:45.093 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:45.093 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupname_latency": { 2024-09-11T22:07:45.093 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:45.093 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:45.094 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:45.094 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:45.094 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupparent_latency": { 2024-09-11T22:07:45.094 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:45.094 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:45.095 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:45.095 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:45.095 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupsnap_latency": { 2024-09-11T22:07:45.095 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:45.095 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:45.095 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:45.096 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:45.096 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lssnap_latency": { 2024-09-11T22:07:45.096 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:45.096 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:45.096 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:45.097 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:45.097 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mkdir_latency": { 2024-09-11T22:07:45.097 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:45.097 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:45.097 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:45.097 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:45.097 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mknod_latency": { 2024-09-11T22:07:45.097 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:45.098 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:45.098 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:45.098 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:45.098 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mksnap_latency": { 2024-09-11T22:07:45.098 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:45.098 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:45.098 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:45.099 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:45.099 INFO:teuthology.orchestra.run.smithi161.stdout: "req_open_latency": { 2024-09-11T22:07:45.099 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:45.099 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:45.099 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:45.099 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:45.099 INFO:teuthology.orchestra.run.smithi161.stdout: "req_readdir_latency": { 2024-09-11T22:07:45.099 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 569, 2024-09-11T22:07:45.100 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 1.151681960, 2024-09-11T22:07:45.100 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.002024045 2024-09-11T22:07:45.100 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:45.100 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rename_latency": { 2024-09-11T22:07:45.100 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:45.100 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:45.100 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:45.100 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:45.101 INFO:teuthology.orchestra.run.smithi161.stdout: "req_renamesnap_latency": { 2024-09-11T22:07:45.101 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:45.101 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:45.101 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:45.101 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:45.101 INFO:teuthology.orchestra.run.smithi161.stdout: "req_snapdiff_latency": { 2024-09-11T22:07:45.101 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:45.101 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:45.102 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:45.102 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:45.102 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmdir_latency": { 2024-09-11T22:07:45.102 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1, 2024-09-11T22:07:45.102 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.032821066, 2024-09-11T22:07:45.102 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.032821066 2024-09-11T22:07:45.102 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:45.102 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmsnap_latency": { 2024-09-11T22:07:45.102 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:45.103 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:45.103 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:45.103 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:45.103 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmxattr_latency": { 2024-09-11T22:07:45.103 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:45.103 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:45.103 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:45.103 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:45.104 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setattr_latency": { 2024-09-11T22:07:45.104 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:45.104 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:45.104 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:45.104 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:45.104 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setdirlayout_latency": { 2024-09-11T22:07:45.104 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:45.105 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:45.105 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:45.105 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:45.105 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setfilelock_latency": { 2024-09-11T22:07:45.105 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:45.105 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:45.105 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:45.106 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:45.106 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setlayout_latency": { 2024-09-11T22:07:45.106 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:45.106 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:45.106 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:45.106 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:45.107 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setxattr_latency": { 2024-09-11T22:07:45.107 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:45.107 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:45.107 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:45.107 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:45.107 INFO:teuthology.orchestra.run.smithi161.stdout: "req_symlink_latency": { 2024-09-11T22:07:45.107 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:45.108 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:45.108 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:45.108 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:45.108 INFO:teuthology.orchestra.run.smithi161.stdout: "req_unlink_latency": { 2024-09-11T22:07:45.108 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 12378, 2024-09-11T22:07:45.108 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 80.056445086, 2024-09-11T22:07:45.108 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.006467639 2024-09-11T22:07:45.109 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:45.109 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_revoke_eviction": 0, 2024-09-11T22:07:45.109 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_acquisition_throttle": 0, 2024-09-11T22:07:45.109 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getvxattr_latency": { 2024-09-11T22:07:45.109 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:45.109 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:45.109 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:45.110 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-11T22:07:45.110 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:45.110 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_sessions": { 2024-09-11T22:07:45.110 INFO:teuthology.orchestra.run.smithi161.stdout: "session_count": 1, 2024-09-11T22:07:45.110 INFO:teuthology.orchestra.run.smithi161.stdout: "session_add": 4, 2024-09-11T22:07:45.110 INFO:teuthology.orchestra.run.smithi161.stdout: "session_remove": 3, 2024-09-11T22:07:45.110 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_open": 1, 2024-09-11T22:07:45.111 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_stale": 0, 2024-09-11T22:07:45.111 INFO:teuthology.orchestra.run.smithi161.stdout: "total_load": 14520, 2024-09-11T22:07:45.111 INFO:teuthology.orchestra.run.smithi161.stdout: "average_load": 14520, 2024-09-11T22:07:45.111 INFO:teuthology.orchestra.run.smithi161.stdout: "avg_session_uptime": 609, 2024-09-11T22:07:45.111 INFO:teuthology.orchestra.run.smithi161.stdout: "mdthresh_evicted": 0 2024-09-11T22:07:45.111 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:45.111 INFO:teuthology.orchestra.run.smithi161.stdout: "mempool": { 2024-09-11T22:07:45.111 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_bytes": 0, 2024-09-11T22:07:45.112 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_items": 0, 2024-09-11T22:07:45.112 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T22:07:45.112 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_items": 0, 2024-09-11T22:07:45.112 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T22:07:45.112 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_items": 0, 2024-09-11T22:07:45.112 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T22:07:45.113 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T22:07:45.113 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T22:07:45.113 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T22:07:45.113 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T22:07:45.113 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_items": 0, 2024-09-11T22:07:45.113 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T22:07:45.113 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T22:07:45.113 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_bytes": 0, 2024-09-11T22:07:45.114 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_items": 0, 2024-09-11T22:07:45.114 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_bytes": 0, 2024-09-11T22:07:45.114 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_items": 0, 2024-09-11T22:07:45.114 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T22:07:45.114 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T22:07:45.114 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T22:07:45.114 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T22:07:45.114 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T22:07:45.115 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_items": 0, 2024-09-11T22:07:45.115 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_bytes": 0, 2024-09-11T22:07:45.115 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_items": 0, 2024-09-11T22:07:45.115 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T22:07:45.115 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T22:07:45.115 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_bytes": 0, 2024-09-11T22:07:45.115 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_items": 0, 2024-09-11T22:07:45.116 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_bytes": 0, 2024-09-11T22:07:45.116 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_items": 0, 2024-09-11T22:07:45.116 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T22:07:45.116 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_items": 0, 2024-09-11T22:07:45.116 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T22:07:45.116 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_items": 0, 2024-09-11T22:07:45.116 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_bytes": 908859, 2024-09-11T22:07:45.117 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_items": 16676, 2024-09-11T22:07:45.117 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_bytes": 88, 2024-09-11T22:07:45.117 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_items": 1, 2024-09-11T22:07:45.117 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_bytes": 0, 2024-09-11T22:07:45.117 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_items": 0, 2024-09-11T22:07:45.117 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_bytes": 0, 2024-09-11T22:07:45.118 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_items": 0, 2024-09-11T22:07:45.118 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_bytes": 0, 2024-09-11T22:07:45.118 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_items": 0, 2024-09-11T22:07:45.121 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_bytes": 8392, 2024-09-11T22:07:45.121 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_items": 184, 2024-09-11T22:07:45.121 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T22:07:45.121 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_items": 0, 2024-09-11T22:07:45.121 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_bytes": 0, 2024-09-11T22:07:45.122 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_items": 0, 2024-09-11T22:07:45.122 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_bytes": 57568533, 2024-09-11T22:07:45.122 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_items": 514016, 2024-09-11T22:07:45.122 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_bytes": 0, 2024-09-11T22:07:45.122 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_items": 0, 2024-09-11T22:07:45.122 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_bytes": 0, 2024-09-11T22:07:45.123 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_items": 0 2024-09-11T22:07:45.123 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:45.123 INFO:teuthology.orchestra.run.smithi161.stdout: "objecter": { 2024-09-11T22:07:45.123 INFO:teuthology.orchestra.run.smithi161.stdout: "op_active": 0, 2024-09-11T22:07:45.123 INFO:teuthology.orchestra.run.smithi161.stdout: "op_laggy": 0, 2024-09-11T22:07:45.123 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send": 42502, 2024-09-11T22:07:45.124 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send_bytes": 131031798, 2024-09-11T22:07:45.124 INFO:teuthology.orchestra.run.smithi161.stdout: "op_resend": 33, 2024-09-11T22:07:45.124 INFO:teuthology.orchestra.run.smithi161.stdout: "op_reply": 42502, 2024-09-11T22:07:45.124 INFO:teuthology.orchestra.run.smithi161.stdout: "op_latency": { 2024-09-11T22:07:45.124 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 42502, 2024-09-11T22:07:45.124 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 6155.373673643, 2024-09-11T22:07:45.125 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.144825506 2024-09-11T22:07:45.125 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:45.125 INFO:teuthology.orchestra.run.smithi161.stdout: "op_inflight": 0, 2024-09-11T22:07:45.125 INFO:teuthology.orchestra.run.smithi161.stdout: "oplen_avg": { 2024-09-11T22:07:45.126 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 42502, 2024-09-11T22:07:45.126 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 43792 2024-09-11T22:07:45.126 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:45.126 INFO:teuthology.orchestra.run.smithi161.stdout: "op": 42502, 2024-09-11T22:07:45.126 INFO:teuthology.orchestra.run.smithi161.stdout: "op_r": 19448, 2024-09-11T22:07:45.126 INFO:teuthology.orchestra.run.smithi161.stdout: "op_w": 23054, 2024-09-11T22:07:45.126 INFO:teuthology.orchestra.run.smithi161.stdout: "op_rmw": 0, 2024-09-11T22:07:45.127 INFO:teuthology.orchestra.run.smithi161.stdout: "op_pg": 0, 2024-09-11T22:07:45.127 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_stat": 0, 2024-09-11T22:07:45.127 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_create": 10, 2024-09-11T22:07:45.128 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_read": 334, 2024-09-11T22:07:45.128 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_write": 9751, 2024-09-11T22:07:45.128 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writefull": 147, 2024-09-11T22:07:45.128 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writesame": 0, 2024-09-11T22:07:45.129 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_append": 0, 2024-09-11T22:07:45.129 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_zero": 0, 2024-09-11T22:07:45.129 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_truncate": 0, 2024-09-11T22:07:45.130 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_delete": 12482, 2024-09-11T22:07:45.130 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_mapext": 0, 2024-09-11T22:07:45.130 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_sparse_read": 0, 2024-09-11T22:07:45.130 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_clonerange": 0, 2024-09-11T22:07:45.130 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_getxattr": 18556, 2024-09-11T22:07:45.130 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_setxattr": 20, 2024-09-11T22:07:45.131 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_cmpxattr": 0, 2024-09-11T22:07:45.131 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_rmxattr": 0, 2024-09-11T22:07:45.131 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_resetxattrs": 0, 2024-09-11T22:07:45.132 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_call": 0, 2024-09-11T22:07:45.132 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_watch": 0, 2024-09-11T22:07:45.132 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_notify": 0, 2024-09-11T22:07:45.132 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T22:07:45.132 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls": 0, 2024-09-11T22:07:45.132 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls_filter": 0, 2024-09-11T22:07:45.133 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_other": 0, 2024-09-11T22:07:45.133 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_active": 0, 2024-09-11T22:07:45.133 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_send": 0, 2024-09-11T22:07:45.133 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_resend": 0, 2024-09-11T22:07:45.133 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_ping": 0, 2024-09-11T22:07:45.133 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_active": 0, 2024-09-11T22:07:45.133 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_send": 0, 2024-09-11T22:07:45.134 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_resend": 0, 2024-09-11T22:07:45.134 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_active": 0, 2024-09-11T22:07:45.134 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_send": 0, 2024-09-11T22:07:45.134 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_resend": 0, 2024-09-11T22:07:45.134 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_active": 0, 2024-09-11T22:07:45.134 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_send": 0, 2024-09-11T22:07:45.134 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_resend": 0, 2024-09-11T22:07:45.134 INFO:teuthology.orchestra.run.smithi161.stdout: "command_active": 0, 2024-09-11T22:07:45.135 INFO:teuthology.orchestra.run.smithi161.stdout: "command_send": 0, 2024-09-11T22:07:45.135 INFO:teuthology.orchestra.run.smithi161.stdout: "command_resend": 0, 2024-09-11T22:07:45.135 INFO:teuthology.orchestra.run.smithi161.stdout: "map_epoch": 95, 2024-09-11T22:07:45.135 INFO:teuthology.orchestra.run.smithi161.stdout: "map_full": 0, 2024-09-11T22:07:45.135 INFO:teuthology.orchestra.run.smithi161.stdout: "map_inc": 11, 2024-09-11T22:07:45.135 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_sessions": 12, 2024-09-11T22:07:45.135 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_open": 12, 2024-09-11T22:07:45.135 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_close": 0, 2024-09-11T22:07:45.136 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_laggy": 0, 2024-09-11T22:07:45.136 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_wr": 1296, 2024-09-11T22:07:45.136 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_rd": 1116, 2024-09-11T22:07:45.136 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_del": 80 2024-09-11T22:07:45.137 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:45.137 INFO:teuthology.orchestra.run.smithi161.stdout: "oft": { 2024-09-11T22:07:45.137 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_objs": 1, 2024-09-11T22:07:45.138 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_kv_pairs": 0, 2024-09-11T22:07:45.138 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_updates": 0, 2024-09-11T22:07:45.138 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_removes": 0 2024-09-11T22:07:45.138 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:45.138 INFO:teuthology.orchestra.run.smithi161.stdout: "osd-slow-ops": { 2024-09-11T22:07:45.138 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_ops_count": 19137 2024-09-11T22:07:45.139 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:45.139 INFO:teuthology.orchestra.run.smithi161.stdout: "purge_queue": { 2024-09-11T22:07:45.139 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops": 0, 2024-09-11T22:07:45.139 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops_high_water": 6, 2024-09-11T22:07:45.140 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing": 0, 2024-09-11T22:07:45.140 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_high_water": 6, 2024-09-11T22:07:45.140 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed_ops": 12378, 2024-09-11T22:07:45.140 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed": 12378, 2024-09-11T22:07:45.141 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_item_in_journal": 0 2024-09-11T22:07:45.141 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:45.141 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T22:07:45.141 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-09-11T22:07:45.141 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-09-11T22:07:45.141 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-09-11T22:07:45.141 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 156013, 2024-09-11T22:07:45.142 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 77967679, 2024-09-11T22:07:45.142 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-09-11T22:07:45.142 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 156013, 2024-09-11T22:07:45.143 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-09-11T22:07:45.143 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-09-11T22:07:45.143 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 156013, 2024-09-11T22:07:45.143 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 77967679, 2024-09-11T22:07:45.143 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-09-11T22:07:45.143 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:45.143 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:45.144 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:45.144 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-11T22:07:45.144 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:45.144 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_bytes": { 2024-09-11T22:07:45.144 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-09-11T22:07:45.144 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-09-11T22:07:45.144 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-09-11T22:07:45.144 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-09-11T22:07:45.145 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-09-11T22:07:45.145 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-09-11T22:07:45.145 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-09-11T22:07:45.145 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 42502, 2024-09-11T22:07:45.145 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 132453656, 2024-09-11T22:07:45.145 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 42502, 2024-09-11T22:07:45.146 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 132453656, 2024-09-11T22:07:45.146 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-09-11T22:07:45.146 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:45.146 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:45.146 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:45.146 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-11T22:07:45.146 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:45.146 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_ops": { 2024-09-11T22:07:45.147 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-09-11T22:07:45.147 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 1024, 2024-09-11T22:07:45.147 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-09-11T22:07:45.147 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-09-11T22:07:45.147 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-09-11T22:07:45.147 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-09-11T22:07:45.147 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-09-11T22:07:45.148 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 42502, 2024-09-11T22:07:45.148 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 42502, 2024-09-11T22:07:45.148 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 42502, 2024-09-11T22:07:45.148 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 42502, 2024-09-11T22:07:45.148 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-09-11T22:07:45.149 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:45.149 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:45.149 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:45.149 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-11T22:07:45.149 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:45.149 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle": { 2024-09-11T22:07:45.149 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-09-11T22:07:45.150 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-09-11T22:07:45.150 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-09-11T22:07:45.150 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 12378, 2024-09-11T22:07:45.150 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 1430786, 2024-09-11T22:07:45.151 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-09-11T22:07:45.151 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 12378, 2024-09-11T22:07:45.151 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-09-11T22:07:45.152 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-09-11T22:07:45.152 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 334, 2024-09-11T22:07:45.152 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 1430786, 2024-09-11T22:07:45.152 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-09-11T22:07:45.153 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:45.153 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:45.153 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:45.153 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-11T22:07:45.153 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:45.153 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle-0x564a337d2500": { 2024-09-11T22:07:45.154 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-09-11T22:07:45.154 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-09-11T22:07:45.154 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-09-11T22:07:45.154 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 62942, 2024-09-11T22:07:45.154 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 122949366, 2024-09-11T22:07:45.154 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-09-11T22:07:45.154 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 62942, 2024-09-11T22:07:45.155 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-09-11T22:07:45.155 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-09-11T22:07:45.155 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 9417, 2024-09-11T22:07:45.155 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 122949366, 2024-09-11T22:07:45.155 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-09-11T22:07:45.155 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:45.155 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:45.156 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:45.156 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-11T22:07:45.156 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-11T22:07:45.156 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-11T22:07:45.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:45 smithi161 ceph-mon[35615]: pgmap v326: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:07:45.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:45 smithi161 ceph-mon[35615]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:07:45.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:45 smithi001 ceph-mon[30935]: pgmap v326: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:07:45.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:45 smithi001 ceph-mon[30935]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:07:45.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:45 smithi053 ceph-mon[34908]: pgmap v326: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:07:45.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:45 smithi053 ceph-mon[34908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:07:45.893 INFO:tasks.check_counter:Daemon mds.f mds.exported=629 2024-09-11T22:07:45.893 INFO:tasks.check_counter:Daemon mds.f mds.imported=597 2024-09-11T22:07:45.893 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=1203 2024-09-11T22:07:45.893 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=4182 2024-09-11T22:07:45.893 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=7 2024-09-11T22:07:45.894 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-09-11T22:07:45.894 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-09-11T22:07:45.894 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=63 2024-09-11T22:07:45.894 DEBUG:tasks.check_counter:Getting stats from i 2024-09-11T22:07:45.894 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-11T22:07:46.139 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T22:07:46.705 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-11T22:07:46.705 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T22:07:46.705 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 35412, 2024-09-11T22:07:46.705 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 42996, 2024-09-11T22:07:46.706 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 29773596, 2024-09-11T22:07:46.706 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 72904485, 2024-09-11T22:07:46.706 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 11, 2024-09-11T22:07:46.706 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 4, 2024-09-11T22:07:46.706 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 2.783617223, 2024-09-11T22:07:46.706 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 1.222226137, 2024-09-11T22:07:46.706 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 1.216692685, 2024-09-11T22:07:46.706 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.433381911, 2024-09-11T22:07:46.706 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:07:46.707 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 42995, 2024-09-11T22:07:46.707 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 318.860540687, 2024-09-11T22:07:46.707 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.007416223 2024-09-11T22:07:46.707 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:46.707 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:07:46.707 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 48456, 2024-09-11T22:07:46.707 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.009057685, 2024-09-11T22:07:46.707 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000186 2024-09-11T22:07:46.708 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:46.708 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-11T22:07:46.708 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 448 2024-09-11T22:07:46.708 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:46.708 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T22:07:46.708 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 44340, 2024-09-11T22:07:46.708 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 37199, 2024-09-11T22:07:46.708 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 27791810, 2024-09-11T22:07:46.709 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 57346906, 2024-09-11T22:07:46.709 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 15, 2024-09-11T22:07:46.709 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 5, 2024-09-11T22:07:46.709 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 3.162851823, 2024-09-11T22:07:46.709 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 1.355114107, 2024-09-11T22:07:46.709 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 1.537030194, 2024-09-11T22:07:46.709 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.521263347, 2024-09-11T22:07:46.709 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:07:46.709 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 37198, 2024-09-11T22:07:46.710 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 2.317357369, 2024-09-11T22:07:46.710 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000062297 2024-09-11T22:07:46.710 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:46.710 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:07:46.710 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 53899, 2024-09-11T22:07:46.710 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.008619181, 2024-09-11T22:07:46.710 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000159 2024-09-11T22:07:46.710 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:46.711 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 1794928, 2024-09-11T22:07:46.711 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 456528 2024-09-11T22:07:46.711 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:46.711 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T22:07:46.711 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 72180, 2024-09-11T22:07:46.711 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 119774, 2024-09-11T22:07:46.711 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 32394693, 2024-09-11T22:07:46.711 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 98464892, 2024-09-11T22:07:46.712 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 32, 2024-09-11T22:07:46.712 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 8, 2024-09-11T22:07:46.712 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 4.919577626, 2024-09-11T22:07:46.712 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 2.234089113, 2024-09-11T22:07:46.712 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 67.217782435, 2024-09-11T22:07:46.712 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.353002959, 2024-09-11T22:07:46.712 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:07:46.712 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 119771, 2024-09-11T22:07:46.712 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 12.004419334, 2024-09-11T22:07:46.713 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000100228 2024-09-11T22:07:46.713 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:46.713 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:07:46.713 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 143734, 2024-09-11T22:07:46.713 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.030702391, 2024-09-11T22:07:46.713 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000213 2024-09-11T22:07:46.713 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:46.713 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 1088, 2024-09-11T22:07:46.714 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 126208 2024-09-11T22:07:46.714 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:46.714 INFO:teuthology.orchestra.run.smithi161.stdout: "cct": { 2024-09-11T22:07:46.714 INFO:teuthology.orchestra.run.smithi161.stdout: "total_workers": 1, 2024-09-11T22:07:46.714 INFO:teuthology.orchestra.run.smithi161.stdout: "unhealthy_workers": 0 2024-09-11T22:07:46.714 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:46.714 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-MDSRank": { 2024-09-11T22:07:46.714 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-09-11T22:07:46.715 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-09-11T22:07:46.715 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 17007, 2024-09-11T22:07:46.715 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 44.723515467, 2024-09-11T22:07:46.715 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.002629712 2024-09-11T22:07:46.715 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-11T22:07:46.715 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:46.715 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-PurgeQueue": { 2024-09-11T22:07:46.715 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-09-11T22:07:46.716 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-09-11T22:07:46.716 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 16659, 2024-09-11T22:07:46.716 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 4.657021460, 2024-09-11T22:07:46.716 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000279549 2024-09-11T22:07:46.716 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-11T22:07:46.716 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:46.716 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": { 2024-09-11T22:07:46.716 INFO:teuthology.orchestra.run.smithi161.stdout: "request": 33288, 2024-09-11T22:07:46.716 INFO:teuthology.orchestra.run.smithi161.stdout: "reply": 26740, 2024-09-11T22:07:46.717 INFO:teuthology.orchestra.run.smithi161.stdout: "reply_latency": { 2024-09-11T22:07:46.717 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 26740, 2024-09-11T22:07:46.717 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 99.805235875, 2024-09-11T22:07:46.717 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.003732432 2024-09-11T22:07:46.717 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:46.717 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_reply": 0, 2024-09-11T22:07:46.717 INFO:teuthology.orchestra.run.smithi161.stdout: "forward": 6548, 2024-09-11T22:07:46.717 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_complete": 1166, 2024-09-11T22:07:46.718 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_keys": 0, 2024-09-11T22:07:46.718 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_commit": 515, 2024-09-11T22:07:46.718 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_split": 63, 2024-09-11T22:07:46.718 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_merge": 0, 2024-09-11T22:07:46.718 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes": 24217, 2024-09-11T22:07:46.718 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_top": 928, 2024-09-11T22:07:46.718 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_bottom": 449, 2024-09-11T22:07:46.718 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pin_tail": 22840, 2024-09-11T22:07:46.719 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pinned": 22889, 2024-09-11T22:07:46.719 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_expired": 25915, 2024-09-11T22:07:46.719 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_with_caps": 0, 2024-09-11T22:07:46.719 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": 0, 2024-09-11T22:07:46.719 INFO:teuthology.orchestra.run.smithi161.stdout: "subtrees": 6, 2024-09-11T22:07:46.719 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse": 54538, 2024-09-11T22:07:46.719 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_hit": 37823, 2024-09-11T22:07:46.719 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_forward": 6176, 2024-09-11T22:07:46.719 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_discover": 12, 2024-09-11T22:07:46.720 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_dir_fetch": 58, 2024-09-11T22:07:46.720 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_remote_ino": 0, 2024-09-11T22:07:46.720 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_lock": 4, 2024-09-11T22:07:46.720 INFO:teuthology.orchestra.run.smithi161.stdout: "load_cent": 0, 2024-09-11T22:07:46.720 INFO:teuthology.orchestra.run.smithi161.stdout: "q": 0, 2024-09-11T22:07:46.720 INFO:teuthology.orchestra.run.smithi161.stdout: "exported": 458, 2024-09-11T22:07:46.720 INFO:teuthology.orchestra.run.smithi161.stdout: "exported_inodes": 21987, 2024-09-11T22:07:46.721 INFO:teuthology.orchestra.run.smithi161.stdout: "imported": 506, 2024-09-11T22:07:46.721 INFO:teuthology.orchestra.run.smithi161.stdout: "imported_inodes": 24041, 2024-09-11T22:07:46.721 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_dir_fetch": 0, 2024-09-11T22:07:46.721 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_backtrace_fetch": 16581, 2024-09-11T22:07:46.721 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_peer_discover": 0, 2024-09-11T22:07:46.721 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rfiles": 4, 2024-09-11T22:07:46.721 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rbytes": 234, 2024-09-11T22:07:46.721 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rsnaps": 7, 2024-09-11T22:07:46.721 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_fetch": 16581, 2024-09-11T22:07:46.722 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_set_tag": 0, 2024-09-11T22:07:46.722 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T22:07:46.722 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_inotable_repaired": 0, 2024-09-11T22:07:46.722 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_inodes": 3, 2024-09-11T22:07:46.722 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T22:07:46.722 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dirfrag_rstats": 3, 2024-09-11T22:07:46.722 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_file_inodes": 16578, 2024-09-11T22:07:46.722 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_inode_file_caps": 10, 2024-09-11T22:07:46.723 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_revoke": 10286, 2024-09-11T22:07:46.723 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_grant": 17371, 2024-09-11T22:07:46.723 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T22:07:46.723 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T22:07:46.723 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T22:07:46.723 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps": 18419, 2024-09-11T22:07:46.723 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps_dirty": 15815, 2024-09-11T22:07:46.723 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_cap_release": 620, 2024-09-11T22:07:46.724 INFO:teuthology.orchestra.run.smithi161.stdout: "process_request_cap_release": 20727 2024-09-11T22:07:46.724 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:46.724 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_cache": { 2024-09-11T22:07:46.724 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update": 2031, 2024-09-11T22:07:46.724 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update_receipt": 2452, 2024-09-11T22:07:46.724 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_try_discover": 1230, 2024-09-11T22:07:46.724 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_send_discover": 2218, 2024-09-11T22:07:46.724 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_handle_discover": 3874, 2024-09-11T22:07:46.725 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays": 1, 2024-09-11T22:07:46.725 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_delayed": 0, 2024-09-11T22:07:46.725 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_enqueuing": 0, 2024-09-11T22:07:46.725 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_created": 9681, 2024-09-11T22:07:46.725 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_enqueued": 9680, 2024-09-11T22:07:46.725 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_reintegrated": 0, 2024-09-11T22:07:46.725 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_migrated": 0, 2024-09-11T22:07:46.725 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_processing": 0, 2024-09-11T22:07:46.726 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_enqueued": 0, 2024-09-11T22:07:46.726 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_prioritized": 0, 2024-09-11T22:07:46.726 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_started": 0, 2024-09-11T22:07:46.726 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_completed": 0, 2024-09-11T22:07:46.726 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_quiesce_path": 8, 2024-09-11T22:07:46.726 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_quiesce_inode": 18419, 2024-09-11T22:07:46.726 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T22:07:46.726 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_exportdir": 479, 2024-09-11T22:07:46.727 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_flush": 0, 2024-09-11T22:07:46.727 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragmentdir": 63, 2024-09-11T22:07:46.727 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragstats": 0, 2024-09-11T22:07:46.727 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_inodestats": 0 2024-09-11T22:07:46.727 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:46.727 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_log": { 2024-09-11T22:07:46.727 INFO:teuthology.orchestra.run.smithi161.stdout: "evlrg": 6, 2024-09-11T22:07:46.727 INFO:teuthology.orchestra.run.smithi161.stdout: "evadd": 51376, 2024-09-11T22:07:46.728 INFO:teuthology.orchestra.run.smithi161.stdout: "evex": 0, 2024-09-11T22:07:46.728 INFO:teuthology.orchestra.run.smithi161.stdout: "evtrm": 0, 2024-09-11T22:07:46.728 INFO:teuthology.orchestra.run.smithi161.stdout: "ev": 51376, 2024-09-11T22:07:46.728 INFO:teuthology.orchestra.run.smithi161.stdout: "evexg": 0, 2024-09-11T22:07:46.728 INFO:teuthology.orchestra.run.smithi161.stdout: "evexd": 0, 2024-09-11T22:07:46.728 INFO:teuthology.orchestra.run.smithi161.stdout: "segadd": 71, 2024-09-11T22:07:46.728 INFO:teuthology.orchestra.run.smithi161.stdout: "segex": 0, 2024-09-11T22:07:46.728 INFO:teuthology.orchestra.run.smithi161.stdout: "segtrm": 0, 2024-09-11T22:07:46.729 INFO:teuthology.orchestra.run.smithi161.stdout: "seg": 71, 2024-09-11T22:07:46.729 INFO:teuthology.orchestra.run.smithi161.stdout: "segmjr": 6, 2024-09-11T22:07:46.729 INFO:teuthology.orchestra.run.smithi161.stdout: "segexg": 0, 2024-09-11T22:07:46.729 INFO:teuthology.orchestra.run.smithi161.stdout: "segexd": 0, 2024-09-11T22:07:46.729 INFO:teuthology.orchestra.run.smithi161.stdout: "expos": 4194304, 2024-09-11T22:07:46.729 INFO:teuthology.orchestra.run.smithi161.stdout: "wrpos": 127267996, 2024-09-11T22:07:46.729 INFO:teuthology.orchestra.run.smithi161.stdout: "rdpos": 0, 2024-09-11T22:07:46.729 INFO:teuthology.orchestra.run.smithi161.stdout: "jlat": { 2024-09-11T22:07:46.730 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 9483, 2024-09-11T22:07:46.730 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 2680.562975341, 2024-09-11T22:07:46.730 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.282670354 2024-09-11T22:07:46.730 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:46.730 INFO:teuthology.orchestra.run.smithi161.stdout: "replayed": 0 2024-09-11T22:07:46.730 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:46.730 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_mem": { 2024-09-11T22:07:46.730 INFO:teuthology.orchestra.run.smithi161.stdout: "ino": 14222, 2024-09-11T22:07:46.731 INFO:teuthology.orchestra.run.smithi161.stdout: "ino+": 42838, 2024-09-11T22:07:46.731 INFO:teuthology.orchestra.run.smithi161.stdout: "ino-": 28616, 2024-09-11T22:07:46.731 INFO:teuthology.orchestra.run.smithi161.stdout: "dir": 540, 2024-09-11T22:07:46.731 INFO:teuthology.orchestra.run.smithi161.stdout: "dir+": 2772, 2024-09-11T22:07:46.731 INFO:teuthology.orchestra.run.smithi161.stdout: "dir-": 2232, 2024-09-11T22:07:46.731 INFO:teuthology.orchestra.run.smithi161.stdout: "dn": 24217, 2024-09-11T22:07:46.731 INFO:teuthology.orchestra.run.smithi161.stdout: "dn+": 59812, 2024-09-11T22:07:46.731 INFO:teuthology.orchestra.run.smithi161.stdout: "dn-": 35595, 2024-09-11T22:07:46.732 INFO:teuthology.orchestra.run.smithi161.stdout: "cap": 0, 2024-09-11T22:07:46.732 INFO:teuthology.orchestra.run.smithi161.stdout: "cap+": 23203, 2024-09-11T22:07:46.732 INFO:teuthology.orchestra.run.smithi161.stdout: "cap-": 23203, 2024-09-11T22:07:46.732 INFO:teuthology.orchestra.run.smithi161.stdout: "rss": 197740, 2024-09-11T22:07:46.732 INFO:teuthology.orchestra.run.smithi161.stdout: "heap": 223520 2024-09-11T22:07:46.732 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:46.732 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_server": { 2024-09-11T22:07:46.732 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_client_request": 44896, 2024-09-11T22:07:46.733 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_server_request": 14111, 2024-09-11T22:07:46.733 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_request": 33288, 2024-09-11T22:07:46.733 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_session": 144, 2024-09-11T22:07:46.733 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_peer_request": 38335, 2024-09-11T22:07:46.733 INFO:teuthology.orchestra.run.smithi161.stdout: "req_create_latency": { 2024-09-11T22:07:46.733 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 6521, 2024-09-11T22:07:46.733 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 21.791461929, 2024-09-11T22:07:46.733 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.003341736 2024-09-11T22:07:46.734 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:46.734 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getattr_latency": { 2024-09-11T22:07:46.734 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1, 2024-09-11T22:07:46.734 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.299485541, 2024-09-11T22:07:46.734 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.299485541 2024-09-11T22:07:46.734 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:46.734 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getfilelock_latency": { 2024-09-11T22:07:46.734 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:46.735 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:46.735 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:46.735 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:46.735 INFO:teuthology.orchestra.run.smithi161.stdout: "req_link_latency": { 2024-09-11T22:07:46.735 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:46.735 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:46.735 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:46.735 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:46.735 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookup_latency": { 2024-09-11T22:07:46.736 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 9480, 2024-09-11T22:07:46.736 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 15.979854439, 2024-09-11T22:07:46.736 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.001685638 2024-09-11T22:07:46.736 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:46.736 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookuphash_latency": { 2024-09-11T22:07:46.736 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:46.736 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:46.737 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:46.737 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:46.737 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupino_latency": { 2024-09-11T22:07:46.737 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:46.737 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:46.737 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:46.737 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:46.737 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupname_latency": { 2024-09-11T22:07:46.737 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:46.738 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:46.738 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:46.738 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:46.738 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupparent_latency": { 2024-09-11T22:07:46.738 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:46.738 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:46.738 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:46.738 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:46.739 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupsnap_latency": { 2024-09-11T22:07:46.739 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:46.739 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:46.739 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:46.739 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:46.739 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lssnap_latency": { 2024-09-11T22:07:46.739 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:46.739 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:46.740 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:46.740 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:46.740 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mkdir_latency": { 2024-09-11T22:07:46.740 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:46.740 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:46.740 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:46.741 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:46.741 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mknod_latency": { 2024-09-11T22:07:46.741 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:46.741 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:46.741 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:46.741 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:46.741 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mksnap_latency": { 2024-09-11T22:07:46.742 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:46.742 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:46.742 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:46.742 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:46.742 INFO:teuthology.orchestra.run.smithi161.stdout: "req_open_latency": { 2024-09-11T22:07:46.742 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:46.742 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:46.742 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:46.742 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:46.743 INFO:teuthology.orchestra.run.smithi161.stdout: "req_readdir_latency": { 2024-09-11T22:07:46.743 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1057, 2024-09-11T22:07:46.743 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 1.991507838, 2024-09-11T22:07:46.743 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.001884113 2024-09-11T22:07:46.743 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:46.743 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rename_latency": { 2024-09-11T22:07:46.743 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:46.743 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:46.744 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:46.744 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:46.744 INFO:teuthology.orchestra.run.smithi161.stdout: "req_renamesnap_latency": { 2024-09-11T22:07:46.744 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:46.744 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:46.744 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:46.744 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:46.744 INFO:teuthology.orchestra.run.smithi161.stdout: "req_snapdiff_latency": { 2024-09-11T22:07:46.745 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:46.745 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:46.745 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:46.745 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:46.745 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmdir_latency": { 2024-09-11T22:07:46.745 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1, 2024-09-11T22:07:46.745 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.082453263, 2024-09-11T22:07:46.745 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.082453263 2024-09-11T22:07:46.746 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:46.746 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmsnap_latency": { 2024-09-11T22:07:46.746 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:46.746 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:46.746 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:46.746 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:46.746 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmxattr_latency": { 2024-09-11T22:07:46.746 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:46.747 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:46.747 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:46.747 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:46.747 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setattr_latency": { 2024-09-11T22:07:46.747 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:46.747 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:46.747 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:46.748 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:46.748 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setdirlayout_latency": { 2024-09-11T22:07:46.748 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:46.748 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:46.748 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:46.748 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:46.748 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setfilelock_latency": { 2024-09-11T22:07:46.748 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:46.749 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:46.749 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:46.749 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:46.749 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setlayout_latency": { 2024-09-11T22:07:46.749 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:46.749 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:46.749 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:46.749 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:46.750 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setxattr_latency": { 2024-09-11T22:07:46.750 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:46.752 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:46.752 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:46.753 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:46.753 INFO:teuthology.orchestra.run.smithi161.stdout: "req_symlink_latency": { 2024-09-11T22:07:46.753 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:46.753 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:46.753 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:46.753 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:46.753 INFO:teuthology.orchestra.run.smithi161.stdout: "req_unlink_latency": { 2024-09-11T22:07:46.753 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 9680, 2024-09-11T22:07:46.754 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 59.660472865, 2024-09-11T22:07:46.754 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.006163271 2024-09-11T22:07:46.754 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:46.754 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_revoke_eviction": 0, 2024-09-11T22:07:46.754 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_acquisition_throttle": 0, 2024-09-11T22:07:46.754 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getvxattr_latency": { 2024-09-11T22:07:46.754 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:46.754 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:46.755 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:46.755 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-11T22:07:46.755 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:46.755 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_sessions": { 2024-09-11T22:07:46.755 INFO:teuthology.orchestra.run.smithi161.stdout: "session_count": 1, 2024-09-11T22:07:46.755 INFO:teuthology.orchestra.run.smithi161.stdout: "session_add": 3, 2024-09-11T22:07:46.756 INFO:teuthology.orchestra.run.smithi161.stdout: "session_remove": 2, 2024-09-11T22:07:46.756 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_open": 1, 2024-09-11T22:07:46.756 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_stale": 0, 2024-09-11T22:07:46.756 INFO:teuthology.orchestra.run.smithi161.stdout: "total_load": 6057, 2024-09-11T22:07:46.756 INFO:teuthology.orchestra.run.smithi161.stdout: "average_load": 6057, 2024-09-11T22:07:46.756 INFO:teuthology.orchestra.run.smithi161.stdout: "avg_session_uptime": 543, 2024-09-11T22:07:46.756 INFO:teuthology.orchestra.run.smithi161.stdout: "mdthresh_evicted": 0 2024-09-11T22:07:46.756 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:46.757 INFO:teuthology.orchestra.run.smithi161.stdout: "mempool": { 2024-09-11T22:07:46.757 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_bytes": 0, 2024-09-11T22:07:46.757 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_items": 0, 2024-09-11T22:07:46.757 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T22:07:46.757 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_items": 0, 2024-09-11T22:07:46.757 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T22:07:46.757 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_items": 0, 2024-09-11T22:07:46.757 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T22:07:46.758 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T22:07:46.758 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T22:07:46.758 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T22:07:46.758 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T22:07:46.758 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_items": 0, 2024-09-11T22:07:46.758 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T22:07:46.758 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T22:07:46.758 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_bytes": 0, 2024-09-11T22:07:46.758 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_items": 0, 2024-09-11T22:07:46.759 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_bytes": 0, 2024-09-11T22:07:46.759 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_items": 0, 2024-09-11T22:07:46.759 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T22:07:46.759 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T22:07:46.759 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T22:07:46.759 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T22:07:46.759 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T22:07:46.759 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_items": 0, 2024-09-11T22:07:46.760 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_bytes": 0, 2024-09-11T22:07:46.760 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_items": 0, 2024-09-11T22:07:46.760 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T22:07:46.760 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T22:07:46.760 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_bytes": 0, 2024-09-11T22:07:46.760 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_items": 0, 2024-09-11T22:07:46.760 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_bytes": 0, 2024-09-11T22:07:46.760 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_items": 0, 2024-09-11T22:07:46.761 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T22:07:46.761 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_items": 0, 2024-09-11T22:07:46.761 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T22:07:46.761 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_items": 0, 2024-09-11T22:07:46.761 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_bytes": 810279, 2024-09-11T22:07:46.761 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_items": 13367, 2024-09-11T22:07:46.761 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_bytes": 0, 2024-09-11T22:07:46.761 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_items": 0, 2024-09-11T22:07:46.761 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_bytes": 0, 2024-09-11T22:07:46.762 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_items": 0, 2024-09-11T22:07:46.762 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_bytes": 0, 2024-09-11T22:07:46.762 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_items": 0, 2024-09-11T22:07:46.762 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_bytes": 0, 2024-09-11T22:07:46.762 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_items": 0, 2024-09-11T22:07:46.762 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_bytes": 8392, 2024-09-11T22:07:46.762 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_items": 184, 2024-09-11T22:07:46.762 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T22:07:46.763 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_items": 0, 2024-09-11T22:07:46.763 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_bytes": 0, 2024-09-11T22:07:46.763 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_items": 0, 2024-09-11T22:07:46.763 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_bytes": 50384451, 2024-09-11T22:07:46.763 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_items": 444146, 2024-09-11T22:07:46.763 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_bytes": 0, 2024-09-11T22:07:46.763 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_items": 0, 2024-09-11T22:07:46.763 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_bytes": 0, 2024-09-11T22:07:46.764 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_items": 0 2024-09-11T22:07:46.764 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:46.764 INFO:teuthology.orchestra.run.smithi161.stdout: "objecter": { 2024-09-11T22:07:46.764 INFO:teuthology.orchestra.run.smithi161.stdout: "op_active": 0, 2024-09-11T22:07:46.764 INFO:teuthology.orchestra.run.smithi161.stdout: "op_laggy": 0, 2024-09-11T22:07:46.764 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send": 38213, 2024-09-11T22:07:46.764 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send_bytes": 130653505, 2024-09-11T22:07:46.764 INFO:teuthology.orchestra.run.smithi161.stdout: "op_resend": 30, 2024-09-11T22:07:46.765 INFO:teuthology.orchestra.run.smithi161.stdout: "op_reply": 38213, 2024-09-11T22:07:46.765 INFO:teuthology.orchestra.run.smithi161.stdout: "op_latency": { 2024-09-11T22:07:46.765 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 38213, 2024-09-11T22:07:46.765 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 2489.147274909, 2024-09-11T22:07:46.765 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.065138755 2024-09-11T22:07:46.765 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:46.765 INFO:teuthology.orchestra.run.smithi161.stdout: "op_inflight": 0, 2024-09-11T22:07:46.765 INFO:teuthology.orchestra.run.smithi161.stdout: "oplen_avg": { 2024-09-11T22:07:46.765 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 38213, 2024-09-11T22:07:46.766 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 40106 2024-09-11T22:07:46.766 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:46.766 INFO:teuthology.orchestra.run.smithi161.stdout: "op": 38213, 2024-09-11T22:07:46.766 INFO:teuthology.orchestra.run.smithi161.stdout: "op_r": 17964, 2024-09-11T22:07:46.766 INFO:teuthology.orchestra.run.smithi161.stdout: "op_w": 20249, 2024-09-11T22:07:46.766 INFO:teuthology.orchestra.run.smithi161.stdout: "op_rmw": 0, 2024-09-11T22:07:46.766 INFO:teuthology.orchestra.run.smithi161.stdout: "op_pg": 0, 2024-09-11T22:07:46.766 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_stat": 0, 2024-09-11T22:07:46.767 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_create": 10, 2024-09-11T22:07:46.767 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_read": 217, 2024-09-11T22:07:46.767 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_write": 9700, 2024-09-11T22:07:46.767 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writefull": 86, 2024-09-11T22:07:46.767 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writesame": 0, 2024-09-11T22:07:46.767 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_append": 0, 2024-09-11T22:07:46.767 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_zero": 0, 2024-09-11T22:07:46.767 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_truncate": 0, 2024-09-11T22:07:46.768 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_delete": 9784, 2024-09-11T22:07:46.768 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_mapext": 0, 2024-09-11T22:07:46.768 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_sparse_read": 0, 2024-09-11T22:07:46.768 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_clonerange": 0, 2024-09-11T22:07:46.768 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_getxattr": 16581, 2024-09-11T22:07:46.768 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_setxattr": 20, 2024-09-11T22:07:46.768 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_cmpxattr": 0, 2024-09-11T22:07:46.768 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_rmxattr": 0, 2024-09-11T22:07:46.769 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_resetxattrs": 0, 2024-09-11T22:07:46.769 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_call": 0, 2024-09-11T22:07:46.769 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_watch": 0, 2024-09-11T22:07:46.769 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_notify": 0, 2024-09-11T22:07:46.769 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T22:07:46.769 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls": 0, 2024-09-11T22:07:46.769 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls_filter": 0, 2024-09-11T22:07:46.769 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_other": 0, 2024-09-11T22:07:46.769 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_active": 0, 2024-09-11T22:07:46.770 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_send": 0, 2024-09-11T22:07:46.770 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_resend": 0, 2024-09-11T22:07:46.770 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_ping": 0, 2024-09-11T22:07:46.770 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_active": 0, 2024-09-11T22:07:46.770 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_send": 0, 2024-09-11T22:07:46.770 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_resend": 0, 2024-09-11T22:07:46.770 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_active": 0, 2024-09-11T22:07:46.770 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_send": 0, 2024-09-11T22:07:46.771 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_resend": 0, 2024-09-11T22:07:46.771 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_active": 0, 2024-09-11T22:07:46.771 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_send": 0, 2024-09-11T22:07:46.771 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_resend": 0, 2024-09-11T22:07:46.771 INFO:teuthology.orchestra.run.smithi161.stdout: "command_active": 0, 2024-09-11T22:07:46.771 INFO:teuthology.orchestra.run.smithi161.stdout: "command_send": 0, 2024-09-11T22:07:46.771 INFO:teuthology.orchestra.run.smithi161.stdout: "command_resend": 0, 2024-09-11T22:07:46.771 INFO:teuthology.orchestra.run.smithi161.stdout: "map_epoch": 95, 2024-09-11T22:07:46.772 INFO:teuthology.orchestra.run.smithi161.stdout: "map_full": 0, 2024-09-11T22:07:46.772 INFO:teuthology.orchestra.run.smithi161.stdout: "map_inc": 11, 2024-09-11T22:07:46.772 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_sessions": 12, 2024-09-11T22:07:46.772 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_open": 12, 2024-09-11T22:07:46.772 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_close": 0, 2024-09-11T22:07:46.772 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_laggy": 0, 2024-09-11T22:07:46.772 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_wr": 1301, 2024-09-11T22:07:46.772 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_rd": 2332, 2024-09-11T22:07:46.773 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_del": 75 2024-09-11T22:07:46.773 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:46.773 INFO:teuthology.orchestra.run.smithi161.stdout: "oft": { 2024-09-11T22:07:46.773 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_objs": 1, 2024-09-11T22:07:46.773 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_kv_pairs": 0, 2024-09-11T22:07:46.773 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_updates": 0, 2024-09-11T22:07:46.773 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_removes": 0 2024-09-11T22:07:46.773 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:46.773 INFO:teuthology.orchestra.run.smithi161.stdout: "osd-slow-ops": { 2024-09-11T22:07:46.774 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_ops_count": 18412 2024-09-11T22:07:46.774 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:46.774 INFO:teuthology.orchestra.run.smithi161.stdout: "purge_queue": { 2024-09-11T22:07:46.774 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops": 0, 2024-09-11T22:07:46.774 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops_high_water": 6, 2024-09-11T22:07:46.774 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing": 0, 2024-09-11T22:07:46.774 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_high_water": 6, 2024-09-11T22:07:46.774 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed_ops": 9680, 2024-09-11T22:07:46.775 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed": 9680, 2024-09-11T22:07:46.775 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_item_in_journal": 0 2024-09-11T22:07:46.775 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:46.775 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T22:07:46.775 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-09-11T22:07:46.775 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-09-11T22:07:46.775 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-09-11T22:07:46.775 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 151928, 2024-09-11T22:07:46.776 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 76268079, 2024-09-11T22:07:46.776 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-09-11T22:07:46.776 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 151928, 2024-09-11T22:07:46.776 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-09-11T22:07:46.776 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-09-11T22:07:46.776 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 151928, 2024-09-11T22:07:46.776 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 76268079, 2024-09-11T22:07:46.776 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-09-11T22:07:46.777 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:46.777 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:46.777 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:46.777 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-11T22:07:46.777 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:46.777 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_bytes": { 2024-09-11T22:07:46.777 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-09-11T22:07:46.777 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-09-11T22:07:46.778 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-09-11T22:07:46.778 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-09-11T22:07:46.778 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-09-11T22:07:46.778 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-09-11T22:07:46.778 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-09-11T22:07:46.778 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 38213, 2024-09-11T22:07:46.778 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 131766321, 2024-09-11T22:07:46.778 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 38213, 2024-09-11T22:07:46.779 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 131766321, 2024-09-11T22:07:46.779 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-09-11T22:07:46.779 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:46.779 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:46.779 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:46.779 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-11T22:07:46.779 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:46.779 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_ops": { 2024-09-11T22:07:46.780 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-09-11T22:07:46.780 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 1024, 2024-09-11T22:07:46.780 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-09-11T22:07:46.780 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-09-11T22:07:46.780 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-09-11T22:07:46.780 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-09-11T22:07:46.780 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-09-11T22:07:46.780 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 38213, 2024-09-11T22:07:46.781 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 38213, 2024-09-11T22:07:46.781 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 38213, 2024-09-11T22:07:46.781 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 38213, 2024-09-11T22:07:46.781 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-09-11T22:07:46.781 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:46.781 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:46.781 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:46.781 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-11T22:07:46.781 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:46.782 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle": { 2024-09-11T22:07:46.782 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-09-11T22:07:46.782 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-09-11T22:07:46.782 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-09-11T22:07:46.782 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 9680, 2024-09-11T22:07:46.782 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 1131472, 2024-09-11T22:07:46.782 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-09-11T22:07:46.782 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 9680, 2024-09-11T22:07:46.783 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-09-11T22:07:46.783 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-09-11T22:07:46.783 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 217, 2024-09-11T22:07:46.783 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 1131472, 2024-09-11T22:07:46.783 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-09-11T22:07:46.783 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:46.783 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:46.783 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:46.784 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-11T22:07:46.784 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:46.784 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle-0x556e2019e500": { 2024-09-11T22:07:46.784 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-09-11T22:07:46.784 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-09-11T22:07:46.784 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-09-11T22:07:46.784 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 51376, 2024-09-11T22:07:46.784 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 123073692, 2024-09-11T22:07:46.785 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-09-11T22:07:46.785 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 51376, 2024-09-11T22:07:46.785 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-09-11T22:07:46.785 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-09-11T22:07:46.785 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 9483, 2024-09-11T22:07:46.785 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 123073692, 2024-09-11T22:07:46.785 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-09-11T22:07:46.785 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:46.785 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:46.786 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:46.786 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-11T22:07:46.786 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-11T22:07:46.786 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-11T22:07:47.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:47 smithi161 ceph-mon[35615]: pgmap v327: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 14 op/s 2024-09-11T22:07:47.411 INFO:tasks.check_counter:Daemon mds.i mds.exported=458 2024-09-11T22:07:47.411 INFO:tasks.check_counter:Daemon mds.i mds.imported=506 2024-09-11T22:07:47.411 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=2031 2024-09-11T22:07:47.411 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=2452 2024-09-11T22:07:47.411 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=7 2024-09-11T22:07:47.411 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-09-11T22:07:47.412 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-09-11T22:07:47.412 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=63 2024-09-11T22:07:47.412 DEBUG:tasks.check_counter:Getting stats from l 2024-09-11T22:07:47.412 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-11T22:07:47.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:47 smithi001 ceph-mon[30935]: pgmap v327: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 14 op/s 2024-09-11T22:07:47.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:47 smithi053 ceph-mon[34908]: pgmap v327: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 14 op/s 2024-09-11T22:07:47.658 INFO:teuthology.orchestra.run.smithi161.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T22:07:48.236 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-09-11T22:07:48.236 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T22:07:48.236 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 561, 2024-09-11T22:07:48.237 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 561, 2024-09-11T22:07:48.237 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 33705143, 2024-09-11T22:07:48.237 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 177638, 2024-09-11T22:07:48.237 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 7, 2024-09-11T22:07:48.237 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 3, 2024-09-11T22:07:48.237 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.076062279, 2024-09-11T22:07:48.237 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.015815382, 2024-09-11T22:07:48.237 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.039798786, 2024-09-11T22:07:48.238 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.015064133, 2024-09-11T22:07:48.238 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:07:48.238 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 560, 2024-09-11T22:07:48.238 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.029748614, 2024-09-11T22:07:48.238 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000053122 2024-09-11T22:07:48.238 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:48.238 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:07:48.239 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:48.239 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:48.239 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:48.239 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:48.239 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-11T22:07:48.239 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 448 2024-09-11T22:07:48.239 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:48.240 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T22:07:48.240 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 972, 2024-09-11T22:07:48.240 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 936, 2024-09-11T22:07:48.240 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 60669726, 2024-09-11T22:07:48.240 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 358381, 2024-09-11T22:07:48.240 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 5, 2024-09-11T22:07:48.240 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 4, 2024-09-11T22:07:48.241 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.136537725, 2024-09-11T22:07:48.241 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.035087614, 2024-09-11T22:07:48.241 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.089566742, 2024-09-11T22:07:48.241 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.023474927, 2024-09-11T22:07:48.241 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:07:48.241 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 935, 2024-09-11T22:07:48.241 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.045383707, 2024-09-11T22:07:48.242 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000048538 2024-09-11T22:07:48.242 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:48.242 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:07:48.242 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:48.242 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:48.242 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:48.242 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:48.242 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 1772768, 2024-09-11T22:07:48.243 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 133232 2024-09-11T22:07:48.243 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:48.243 INFO:teuthology.orchestra.run.smithi161.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T22:07:48.243 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_messages": 1074, 2024-09-11T22:07:48.243 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages": 1234, 2024-09-11T22:07:48.243 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_bytes": 40330027, 2024-09-11T22:07:48.244 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_bytes": 781136, 2024-09-11T22:07:48.244 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_created_connections": 8, 2024-09-11T22:07:48.244 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_active_connections": 6, 2024-09-11T22:07:48.244 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_total_time": 0.144691871, 2024-09-11T22:07:48.244 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_send_time": 0.041970720, 2024-09-11T22:07:48.244 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_recv_time": 0.056404046, 2024-09-11T22:07:48.244 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_running_fast_dispatch_time": 0.034601747, 2024-09-11T22:07:48.244 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:07:48.245 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 1231, 2024-09-11T22:07:48.245 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.058895755, 2024-09-11T22:07:48.245 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000047843 2024-09-11T22:07:48.245 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:48.245 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:07:48.245 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:48.245 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:48.246 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:48.246 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:48.246 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-11T22:07:48.246 INFO:teuthology.orchestra.run.smithi161.stdout: "msgr_send_encrypted_bytes": 440240 2024-09-11T22:07:48.246 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:48.246 INFO:teuthology.orchestra.run.smithi161.stdout: "cct": { 2024-09-11T22:07:48.246 INFO:teuthology.orchestra.run.smithi161.stdout: "total_workers": 1, 2024-09-11T22:07:48.246 INFO:teuthology.orchestra.run.smithi161.stdout: "unhealthy_workers": 0 2024-09-11T22:07:48.247 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:48.247 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-MDSRank": { 2024-09-11T22:07:48.247 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-09-11T22:07:48.247 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-09-11T22:07:48.247 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 3833, 2024-09-11T22:07:48.247 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.108369384, 2024-09-11T22:07:48.248 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000028272 2024-09-11T22:07:48.248 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-11T22:07:48.248 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:48.248 INFO:teuthology.orchestra.run.smithi161.stdout: "finisher-PurgeQueue": { 2024-09-11T22:07:48.248 INFO:teuthology.orchestra.run.smithi161.stdout: "queue_len": 0, 2024-09-11T22:07:48.248 INFO:teuthology.orchestra.run.smithi161.stdout: "complete_latency": { 2024-09-11T22:07:48.248 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:48.249 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:48.249 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:48.249 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-11T22:07:48.249 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:48.249 INFO:teuthology.orchestra.run.smithi161.stdout: "mds": { 2024-09-11T22:07:48.249 INFO:teuthology.orchestra.run.smithi161.stdout: "request": 0, 2024-09-11T22:07:48.249 INFO:teuthology.orchestra.run.smithi161.stdout: "reply": 0, 2024-09-11T22:07:48.249 INFO:teuthology.orchestra.run.smithi161.stdout: "reply_latency": { 2024-09-11T22:07:48.250 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:48.250 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:48.250 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:48.250 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:48.250 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_reply": 0, 2024-09-11T22:07:48.250 INFO:teuthology.orchestra.run.smithi161.stdout: "forward": 0, 2024-09-11T22:07:48.250 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_complete": 0, 2024-09-11T22:07:48.250 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_fetch_keys": 0, 2024-09-11T22:07:48.251 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_commit": 0, 2024-09-11T22:07:48.251 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_split": 0, 2024-09-11T22:07:48.251 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_merge": 0, 2024-09-11T22:07:48.251 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes": 37595, 2024-09-11T22:07:48.251 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_top": 1509, 2024-09-11T22:07:48.251 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_bottom": 36086, 2024-09-11T22:07:48.251 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pin_tail": 0, 2024-09-11T22:07:48.251 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_pinned": 35438, 2024-09-11T22:07:48.252 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_expired": 0, 2024-09-11T22:07:48.252 INFO:teuthology.orchestra.run.smithi161.stdout: "inodes_with_caps": 0, 2024-09-11T22:07:48.252 INFO:teuthology.orchestra.run.smithi161.stdout: "caps": 0, 2024-09-11T22:07:48.252 INFO:teuthology.orchestra.run.smithi161.stdout: "subtrees": 1, 2024-09-11T22:07:48.252 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse": 0, 2024-09-11T22:07:48.252 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_hit": 0, 2024-09-11T22:07:48.252 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_forward": 0, 2024-09-11T22:07:48.252 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_discover": 0, 2024-09-11T22:07:48.253 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_dir_fetch": 0, 2024-09-11T22:07:48.253 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_remote_ino": 0, 2024-09-11T22:07:48.253 INFO:teuthology.orchestra.run.smithi161.stdout: "traverse_lock": 0, 2024-09-11T22:07:48.253 INFO:teuthology.orchestra.run.smithi161.stdout: "load_cent": 0, 2024-09-11T22:07:48.253 INFO:teuthology.orchestra.run.smithi161.stdout: "q": 0, 2024-09-11T22:07:48.253 INFO:teuthology.orchestra.run.smithi161.stdout: "exported": 0, 2024-09-11T22:07:48.253 INFO:teuthology.orchestra.run.smithi161.stdout: "exported_inodes": 0, 2024-09-11T22:07:48.254 INFO:teuthology.orchestra.run.smithi161.stdout: "imported": 0, 2024-09-11T22:07:48.254 INFO:teuthology.orchestra.run.smithi161.stdout: "imported_inodes": 0, 2024-09-11T22:07:48.254 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_dir_fetch": 0, 2024-09-11T22:07:48.254 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_backtrace_fetch": 0, 2024-09-11T22:07:48.254 INFO:teuthology.orchestra.run.smithi161.stdout: "openino_peer_discover": 0, 2024-09-11T22:07:48.254 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rfiles": 0, 2024-09-11T22:07:48.255 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rbytes": 0, 2024-09-11T22:07:48.255 INFO:teuthology.orchestra.run.smithi161.stdout: "root_rsnaps": 0, 2024-09-11T22:07:48.255 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_fetch": 0, 2024-09-11T22:07:48.255 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_set_tag": 0, 2024-09-11T22:07:48.255 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T22:07:48.255 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_inotable_repaired": 0, 2024-09-11T22:07:48.255 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_inodes": 0, 2024-09-11T22:07:48.255 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T22:07:48.256 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T22:07:48.256 INFO:teuthology.orchestra.run.smithi161.stdout: "scrub_file_inodes": 0, 2024-09-11T22:07:48.256 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_inode_file_caps": 0, 2024-09-11T22:07:48.256 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_revoke": 0, 2024-09-11T22:07:48.256 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_grant": 0, 2024-09-11T22:07:48.256 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T22:07:48.257 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T22:07:48.257 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T22:07:48.257 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps": 0, 2024-09-11T22:07:48.257 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_caps_dirty": 0, 2024-09-11T22:07:48.257 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_cap_release": 0, 2024-09-11T22:07:48.257 INFO:teuthology.orchestra.run.smithi161.stdout: "process_request_cap_release": 0 2024-09-11T22:07:48.257 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:48.257 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_cache": { 2024-09-11T22:07:48.258 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update": 0, 2024-09-11T22:07:48.258 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_update_receipt": 0, 2024-09-11T22:07:48.258 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_try_discover": 0, 2024-09-11T22:07:48.258 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_send_discover": 0, 2024-09-11T22:07:48.258 INFO:teuthology.orchestra.run.smithi161.stdout: "dir_handle_discover": 0, 2024-09-11T22:07:48.258 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays": 0, 2024-09-11T22:07:48.258 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_delayed": 0, 2024-09-11T22:07:48.259 INFO:teuthology.orchestra.run.smithi161.stdout: "num_strays_enqueuing": 0, 2024-09-11T22:07:48.259 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_created": 0, 2024-09-11T22:07:48.259 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_enqueued": 0, 2024-09-11T22:07:48.259 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_reintegrated": 0, 2024-09-11T22:07:48.259 INFO:teuthology.orchestra.run.smithi161.stdout: "strays_migrated": 0, 2024-09-11T22:07:48.259 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_processing": 0, 2024-09-11T22:07:48.259 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_enqueued": 0, 2024-09-11T22:07:48.260 INFO:teuthology.orchestra.run.smithi161.stdout: "num_recovering_prioritized": 0, 2024-09-11T22:07:48.260 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_started": 0, 2024-09-11T22:07:48.260 INFO:teuthology.orchestra.run.smithi161.stdout: "recovery_completed": 0, 2024-09-11T22:07:48.260 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_quiesce_path": 0, 2024-09-11T22:07:48.260 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_quiesce_inode": 0, 2024-09-11T22:07:48.260 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T22:07:48.260 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_exportdir": 0, 2024-09-11T22:07:48.261 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_flush": 0, 2024-09-11T22:07:48.261 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragmentdir": 0, 2024-09-11T22:07:48.261 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_fragstats": 0, 2024-09-11T22:07:48.261 INFO:teuthology.orchestra.run.smithi161.stdout: "ireq_inodestats": 0 2024-09-11T22:07:48.261 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:48.261 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_log": { 2024-09-11T22:07:48.261 INFO:teuthology.orchestra.run.smithi161.stdout: "evlrg": 0, 2024-09-11T22:07:48.262 INFO:teuthology.orchestra.run.smithi161.stdout: "evadd": 0, 2024-09-11T22:07:48.262 INFO:teuthology.orchestra.run.smithi161.stdout: "evex": 0, 2024-09-11T22:07:48.262 INFO:teuthology.orchestra.run.smithi161.stdout: "evtrm": 0, 2024-09-11T22:07:48.262 INFO:teuthology.orchestra.run.smithi161.stdout: "ev": 60090, 2024-09-11T22:07:48.262 INFO:teuthology.orchestra.run.smithi161.stdout: "evexg": 0, 2024-09-11T22:07:48.262 INFO:teuthology.orchestra.run.smithi161.stdout: "evexd": 0, 2024-09-11T22:07:48.262 INFO:teuthology.orchestra.run.smithi161.stdout: "segadd": 0, 2024-09-11T22:07:48.262 INFO:teuthology.orchestra.run.smithi161.stdout: "segex": 0, 2024-09-11T22:07:48.263 INFO:teuthology.orchestra.run.smithi161.stdout: "segtrm": 0, 2024-09-11T22:07:48.263 INFO:teuthology.orchestra.run.smithi161.stdout: "seg": 77, 2024-09-11T22:07:48.263 INFO:teuthology.orchestra.run.smithi161.stdout: "segmjr": 6, 2024-09-11T22:07:48.263 INFO:teuthology.orchestra.run.smithi161.stdout: "segexg": 0, 2024-09-11T22:07:48.263 INFO:teuthology.orchestra.run.smithi161.stdout: "segexd": 0, 2024-09-11T22:07:48.263 INFO:teuthology.orchestra.run.smithi161.stdout: "expos": 4194304, 2024-09-11T22:07:48.263 INFO:teuthology.orchestra.run.smithi161.stdout: "wrpos": 136465521, 2024-09-11T22:07:48.263 INFO:teuthology.orchestra.run.smithi161.stdout: "rdpos": 136461497, 2024-09-11T22:07:48.264 INFO:teuthology.orchestra.run.smithi161.stdout: "jlat": { 2024-09-11T22:07:48.264 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:48.264 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:48.264 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:48.264 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:48.264 INFO:teuthology.orchestra.run.smithi161.stdout: "replayed": 60090 2024-09-11T22:07:48.264 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:48.264 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_mem": { 2024-09-11T22:07:48.265 INFO:teuthology.orchestra.run.smithi161.stdout: "ino": 14102, 2024-09-11T22:07:48.265 INFO:teuthology.orchestra.run.smithi161.stdout: "ino+": 44040, 2024-09-11T22:07:48.265 INFO:teuthology.orchestra.run.smithi161.stdout: "ino-": 29938, 2024-09-11T22:07:48.265 INFO:teuthology.orchestra.run.smithi161.stdout: "dir": 523, 2024-09-11T22:07:48.265 INFO:teuthology.orchestra.run.smithi161.stdout: "dir+": 1113, 2024-09-11T22:07:48.265 INFO:teuthology.orchestra.run.smithi161.stdout: "dir-": 590, 2024-09-11T22:07:48.265 INFO:teuthology.orchestra.run.smithi161.stdout: "dn": 37595, 2024-09-11T22:07:48.265 INFO:teuthology.orchestra.run.smithi161.stdout: "dn+": 64593, 2024-09-11T22:07:48.266 INFO:teuthology.orchestra.run.smithi161.stdout: "dn-": 26998, 2024-09-11T22:07:48.266 INFO:teuthology.orchestra.run.smithi161.stdout: "cap": 0, 2024-09-11T22:07:48.266 INFO:teuthology.orchestra.run.smithi161.stdout: "cap+": 0, 2024-09-11T22:07:48.266 INFO:teuthology.orchestra.run.smithi161.stdout: "cap-": 0, 2024-09-11T22:07:48.266 INFO:teuthology.orchestra.run.smithi161.stdout: "rss": 133836, 2024-09-11T22:07:48.266 INFO:teuthology.orchestra.run.smithi161.stdout: "heap": 215328 2024-09-11T22:07:48.266 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:48.266 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_server": { 2024-09-11T22:07:48.267 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_client_request": 0, 2024-09-11T22:07:48.267 INFO:teuthology.orchestra.run.smithi161.stdout: "dispatch_server_request": 0, 2024-09-11T22:07:48.267 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_request": 0, 2024-09-11T22:07:48.267 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_client_session": 0, 2024-09-11T22:07:48.267 INFO:teuthology.orchestra.run.smithi161.stdout: "handle_peer_request": 0, 2024-09-11T22:07:48.267 INFO:teuthology.orchestra.run.smithi161.stdout: "req_create_latency": { 2024-09-11T22:07:48.267 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:48.267 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:48.268 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:48.268 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:48.268 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getattr_latency": { 2024-09-11T22:07:48.268 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:48.268 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:48.268 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:48.268 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:48.268 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getfilelock_latency": { 2024-09-11T22:07:48.269 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:48.269 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:48.269 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:48.269 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:48.269 INFO:teuthology.orchestra.run.smithi161.stdout: "req_link_latency": { 2024-09-11T22:07:48.269 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:48.269 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:48.270 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:48.270 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:48.270 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookup_latency": { 2024-09-11T22:07:48.270 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:48.270 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:48.270 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:48.270 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:48.271 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookuphash_latency": { 2024-09-11T22:07:48.271 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:48.271 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:48.271 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:48.271 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:48.271 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupino_latency": { 2024-09-11T22:07:48.271 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:48.271 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:48.272 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:48.272 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:48.272 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupname_latency": { 2024-09-11T22:07:48.272 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:48.272 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:48.272 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:48.272 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:48.273 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupparent_latency": { 2024-09-11T22:07:48.273 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:48.273 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:48.273 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:48.273 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:48.273 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lookupsnap_latency": { 2024-09-11T22:07:48.273 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:48.273 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:48.274 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:48.274 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:48.274 INFO:teuthology.orchestra.run.smithi161.stdout: "req_lssnap_latency": { 2024-09-11T22:07:48.274 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:48.274 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:48.274 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:48.274 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:48.275 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mkdir_latency": { 2024-09-11T22:07:48.275 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:48.275 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:48.275 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:48.275 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:48.275 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mknod_latency": { 2024-09-11T22:07:48.275 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:48.275 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:48.276 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:48.276 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:48.276 INFO:teuthology.orchestra.run.smithi161.stdout: "req_mksnap_latency": { 2024-09-11T22:07:48.276 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:48.276 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:48.276 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:48.276 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:48.276 INFO:teuthology.orchestra.run.smithi161.stdout: "req_open_latency": { 2024-09-11T22:07:48.277 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:48.277 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:48.277 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:48.277 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:48.277 INFO:teuthology.orchestra.run.smithi161.stdout: "req_readdir_latency": { 2024-09-11T22:07:48.277 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:48.277 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:48.278 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:48.278 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:48.278 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rename_latency": { 2024-09-11T22:07:48.278 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:48.278 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:48.278 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:48.278 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:48.279 INFO:teuthology.orchestra.run.smithi161.stdout: "req_renamesnap_latency": { 2024-09-11T22:07:48.279 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:48.279 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:48.279 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:48.279 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:48.279 INFO:teuthology.orchestra.run.smithi161.stdout: "req_snapdiff_latency": { 2024-09-11T22:07:48.279 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:48.279 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:48.280 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:48.280 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:48.280 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmdir_latency": { 2024-09-11T22:07:48.280 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:48.280 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:48.280 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:48.280 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:48.280 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmsnap_latency": { 2024-09-11T22:07:48.281 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:48.281 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:48.281 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:48.281 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:48.281 INFO:teuthology.orchestra.run.smithi161.stdout: "req_rmxattr_latency": { 2024-09-11T22:07:48.281 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:48.281 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:48.281 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:48.282 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:48.282 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setattr_latency": { 2024-09-11T22:07:48.282 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:48.282 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:48.282 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:48.282 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:48.282 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setdirlayout_latency": { 2024-09-11T22:07:48.282 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:48.283 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:48.283 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:48.283 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:48.283 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setfilelock_latency": { 2024-09-11T22:07:48.283 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:48.283 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:48.283 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:48.283 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:48.284 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setlayout_latency": { 2024-09-11T22:07:48.284 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:48.284 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:48.284 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:48.284 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:48.284 INFO:teuthology.orchestra.run.smithi161.stdout: "req_setxattr_latency": { 2024-09-11T22:07:48.284 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:48.285 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:48.285 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:48.285 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:48.285 INFO:teuthology.orchestra.run.smithi161.stdout: "req_symlink_latency": { 2024-09-11T22:07:48.285 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:48.285 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:48.285 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:48.285 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:48.286 INFO:teuthology.orchestra.run.smithi161.stdout: "req_unlink_latency": { 2024-09-11T22:07:48.286 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:48.286 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:48.286 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:48.286 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:48.286 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_revoke_eviction": 0, 2024-09-11T22:07:48.286 INFO:teuthology.orchestra.run.smithi161.stdout: "cap_acquisition_throttle": 0, 2024-09-11T22:07:48.286 INFO:teuthology.orchestra.run.smithi161.stdout: "req_getvxattr_latency": { 2024-09-11T22:07:48.287 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:48.287 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:48.287 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:48.287 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-11T22:07:48.287 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:48.287 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_sessions": { 2024-09-11T22:07:48.287 INFO:teuthology.orchestra.run.smithi161.stdout: "session_count": 2, 2024-09-11T22:07:48.288 INFO:teuthology.orchestra.run.smithi161.stdout: "session_add": 3, 2024-09-11T22:07:48.288 INFO:teuthology.orchestra.run.smithi161.stdout: "session_remove": 1, 2024-09-11T22:07:48.288 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_open": 3, 2024-09-11T22:07:48.288 INFO:teuthology.orchestra.run.smithi161.stdout: "sessions_stale": 0, 2024-09-11T22:07:48.288 INFO:teuthology.orchestra.run.smithi161.stdout: "total_load": 0, 2024-09-11T22:07:48.288 INFO:teuthology.orchestra.run.smithi161.stdout: "average_load": 0, 2024-09-11T22:07:48.288 INFO:teuthology.orchestra.run.smithi161.stdout: "avg_session_uptime": 2606, 2024-09-11T22:07:48.288 INFO:teuthology.orchestra.run.smithi161.stdout: "mdthresh_evicted": 0 2024-09-11T22:07:48.289 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:48.289 INFO:teuthology.orchestra.run.smithi161.stdout: "mempool": { 2024-09-11T22:07:48.289 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_bytes": 0, 2024-09-11T22:07:48.289 INFO:teuthology.orchestra.run.smithi161.stdout: "bloom_filter_items": 0, 2024-09-11T22:07:48.289 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T22:07:48.289 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_alloc_items": 0, 2024-09-11T22:07:48.289 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T22:07:48.290 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_data_items": 0, 2024-09-11T22:07:48.290 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T22:07:48.290 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T22:07:48.290 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T22:07:48.290 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T22:07:48.290 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T22:07:48.290 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_other_items": 0, 2024-09-11T22:07:48.291 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T22:07:48.291 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T22:07:48.291 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_bytes": 0, 2024-09-11T22:07:48.291 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_extent_items": 0, 2024-09-11T22:07:48.291 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_bytes": 0, 2024-09-11T22:07:48.291 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_blob_items": 0, 2024-09-11T22:07:48.291 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T22:07:48.291 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T22:07:48.292 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T22:07:48.292 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T22:07:48.292 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T22:07:48.292 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_fsck_items": 0, 2024-09-11T22:07:48.292 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_bytes": 0, 2024-09-11T22:07:48.292 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_txc_items": 0, 2024-09-11T22:07:48.292 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T22:07:48.292 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T22:07:48.293 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_bytes": 0, 2024-09-11T22:07:48.293 INFO:teuthology.orchestra.run.smithi161.stdout: "bluestore_writing_items": 0, 2024-09-11T22:07:48.293 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_bytes": 0, 2024-09-11T22:07:48.293 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_items": 0, 2024-09-11T22:07:48.293 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T22:07:48.293 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_reader_items": 0, 2024-09-11T22:07:48.293 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T22:07:48.293 INFO:teuthology.orchestra.run.smithi161.stdout: "bluefs_file_writer_items": 0, 2024-09-11T22:07:48.294 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_bytes": 744692, 2024-09-11T22:07:48.294 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_anon_items": 14163, 2024-09-11T22:07:48.294 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_bytes": 0, 2024-09-11T22:07:48.294 INFO:teuthology.orchestra.run.smithi161.stdout: "buffer_meta_items": 0, 2024-09-11T22:07:48.294 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_bytes": 0, 2024-09-11T22:07:48.294 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_items": 0, 2024-09-11T22:07:48.294 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_bytes": 0, 2024-09-11T22:07:48.294 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_mapbl_items": 0, 2024-09-11T22:07:48.295 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_bytes": 0, 2024-09-11T22:07:48.295 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_pglog_items": 0, 2024-09-11T22:07:48.295 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_bytes": 8392, 2024-09-11T22:07:48.295 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_items": 184, 2024-09-11T22:07:48.295 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T22:07:48.295 INFO:teuthology.orchestra.run.smithi161.stdout: "osdmap_mapping_items": 0, 2024-09-11T22:07:48.295 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_bytes": 0, 2024-09-11T22:07:48.295 INFO:teuthology.orchestra.run.smithi161.stdout: "pgmap_items": 0, 2024-09-11T22:07:48.296 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_bytes": 59221297, 2024-09-11T22:07:48.296 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_co_items": 450466, 2024-09-11T22:07:48.296 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_bytes": 0, 2024-09-11T22:07:48.296 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_1_items": 0, 2024-09-11T22:07:48.296 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_bytes": 0, 2024-09-11T22:07:48.296 INFO:teuthology.orchestra.run.smithi161.stdout: "unittest_2_items": 0 2024-09-11T22:07:48.296 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:48.297 INFO:teuthology.orchestra.run.smithi161.stdout: "objecter": { 2024-09-11T22:07:48.297 INFO:teuthology.orchestra.run.smithi161.stdout: "op_active": 0, 2024-09-11T22:07:48.297 INFO:teuthology.orchestra.run.smithi161.stdout: "op_laggy": 0, 2024-09-11T22:07:48.297 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send": 2337, 2024-09-11T22:07:48.297 INFO:teuthology.orchestra.run.smithi161.stdout: "op_send_bytes": 21, 2024-09-11T22:07:48.297 INFO:teuthology.orchestra.run.smithi161.stdout: "op_resend": 3, 2024-09-11T22:07:48.298 INFO:teuthology.orchestra.run.smithi161.stdout: "op_reply": 2337, 2024-09-11T22:07:48.298 INFO:teuthology.orchestra.run.smithi161.stdout: "op_latency": { 2024-09-11T22:07:48.298 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 2337, 2024-09-11T22:07:48.298 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 6.950847322, 2024-09-11T22:07:48.298 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.002974260 2024-09-11T22:07:48.298 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:48.298 INFO:teuthology.orchestra.run.smithi161.stdout: "op_inflight": 0, 2024-09-11T22:07:48.298 INFO:teuthology.orchestra.run.smithi161.stdout: "oplen_avg": { 2024-09-11T22:07:48.299 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 2337, 2024-09-11T22:07:48.299 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 2338 2024-09-11T22:07:48.299 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:48.299 INFO:teuthology.orchestra.run.smithi161.stdout: "op": 2337, 2024-09-11T22:07:48.299 INFO:teuthology.orchestra.run.smithi161.stdout: "op_r": 2337, 2024-09-11T22:07:48.299 INFO:teuthology.orchestra.run.smithi161.stdout: "op_w": 0, 2024-09-11T22:07:48.299 INFO:teuthology.orchestra.run.smithi161.stdout: "op_rmw": 0, 2024-09-11T22:07:48.300 INFO:teuthology.orchestra.run.smithi161.stdout: "op_pg": 0, 2024-09-11T22:07:48.300 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_stat": 1452, 2024-09-11T22:07:48.300 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_create": 0, 2024-09-11T22:07:48.300 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_read": 883, 2024-09-11T22:07:48.300 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_write": 0, 2024-09-11T22:07:48.300 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writefull": 0, 2024-09-11T22:07:48.300 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_writesame": 0, 2024-09-11T22:07:48.300 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_append": 0, 2024-09-11T22:07:48.301 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_zero": 0, 2024-09-11T22:07:48.301 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_truncate": 0, 2024-09-11T22:07:48.301 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_delete": 0, 2024-09-11T22:07:48.301 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_mapext": 0, 2024-09-11T22:07:48.301 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_sparse_read": 0, 2024-09-11T22:07:48.301 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_clonerange": 0, 2024-09-11T22:07:48.301 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_getxattr": 1, 2024-09-11T22:07:48.302 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_setxattr": 0, 2024-09-11T22:07:48.302 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_cmpxattr": 0, 2024-09-11T22:07:48.302 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_rmxattr": 0, 2024-09-11T22:07:48.302 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_resetxattrs": 0, 2024-09-11T22:07:48.302 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_call": 0, 2024-09-11T22:07:48.302 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_watch": 0, 2024-09-11T22:07:48.302 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_notify": 0, 2024-09-11T22:07:48.303 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T22:07:48.303 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls": 0, 2024-09-11T22:07:48.303 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_pgls_filter": 0, 2024-09-11T22:07:48.303 INFO:teuthology.orchestra.run.smithi161.stdout: "osdop_other": 0, 2024-09-11T22:07:48.303 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_active": 0, 2024-09-11T22:07:48.303 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_send": 0, 2024-09-11T22:07:48.303 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_resend": 0, 2024-09-11T22:07:48.303 INFO:teuthology.orchestra.run.smithi161.stdout: "linger_ping": 0, 2024-09-11T22:07:48.304 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_active": 0, 2024-09-11T22:07:48.304 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_send": 0, 2024-09-11T22:07:48.304 INFO:teuthology.orchestra.run.smithi161.stdout: "poolop_resend": 0, 2024-09-11T22:07:48.304 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_active": 0, 2024-09-11T22:07:48.304 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_send": 0, 2024-09-11T22:07:48.304 INFO:teuthology.orchestra.run.smithi161.stdout: "poolstat_resend": 0, 2024-09-11T22:07:48.304 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_active": 0, 2024-09-11T22:07:48.305 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_send": 0, 2024-09-11T22:07:48.305 INFO:teuthology.orchestra.run.smithi161.stdout: "statfs_resend": 0, 2024-09-11T22:07:48.305 INFO:teuthology.orchestra.run.smithi161.stdout: "command_active": 0, 2024-09-11T22:07:48.305 INFO:teuthology.orchestra.run.smithi161.stdout: "command_send": 0, 2024-09-11T22:07:48.305 INFO:teuthology.orchestra.run.smithi161.stdout: "command_resend": 0, 2024-09-11T22:07:48.305 INFO:teuthology.orchestra.run.smithi161.stdout: "map_epoch": 95, 2024-09-11T22:07:48.305 INFO:teuthology.orchestra.run.smithi161.stdout: "map_full": 0, 2024-09-11T22:07:48.305 INFO:teuthology.orchestra.run.smithi161.stdout: "map_inc": 11, 2024-09-11T22:07:48.306 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_sessions": 9, 2024-09-11T22:07:48.306 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_open": 9, 2024-09-11T22:07:48.306 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_session_close": 0, 2024-09-11T22:07:48.306 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_laggy": 0, 2024-09-11T22:07:48.306 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_wr": 0, 2024-09-11T22:07:48.306 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_rd": 2, 2024-09-11T22:07:48.306 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_del": 0 2024-09-11T22:07:48.306 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:48.307 INFO:teuthology.orchestra.run.smithi161.stdout: "oft": { 2024-09-11T22:07:48.307 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_objs": 0, 2024-09-11T22:07:48.307 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_kv_pairs": 0, 2024-09-11T22:07:48.307 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_updates": 0, 2024-09-11T22:07:48.307 INFO:teuthology.orchestra.run.smithi161.stdout: "omap_total_removes": 0 2024-09-11T22:07:48.307 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:48.308 INFO:teuthology.orchestra.run.smithi161.stdout: "osd-slow-ops": { 2024-09-11T22:07:48.308 INFO:teuthology.orchestra.run.smithi161.stdout: "slow_ops_count": 0 2024-09-11T22:07:48.308 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:48.308 INFO:teuthology.orchestra.run.smithi161.stdout: "purge_queue": { 2024-09-11T22:07:48.308 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops": 0, 2024-09-11T22:07:48.308 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_ops_high_water": 0, 2024-09-11T22:07:48.308 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing": 0, 2024-09-11T22:07:48.308 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executing_high_water": 0, 2024-09-11T22:07:48.309 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed_ops": 0, 2024-09-11T22:07:48.309 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_executed": 0, 2024-09-11T22:07:48.309 INFO:teuthology.orchestra.run.smithi161.stdout: "pq_item_in_journal": 0 2024-09-11T22:07:48.309 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:48.309 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T22:07:48.309 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-09-11T22:07:48.309 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-09-11T22:07:48.309 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-09-11T22:07:48.310 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 2603, 2024-09-11T22:07:48.310 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 134454215, 2024-09-11T22:07:48.310 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-09-11T22:07:48.310 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 2603, 2024-09-11T22:07:48.310 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-09-11T22:07:48.310 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-09-11T22:07:48.310 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 2603, 2024-09-11T22:07:48.311 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 134454215, 2024-09-11T22:07:48.311 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-09-11T22:07:48.311 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:48.311 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:48.311 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:48.311 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-11T22:07:48.311 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:48.311 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_bytes": { 2024-09-11T22:07:48.312 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-09-11T22:07:48.312 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 104857600, 2024-09-11T22:07:48.312 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-09-11T22:07:48.312 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-09-11T22:07:48.312 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-09-11T22:07:48.312 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-09-11T22:07:48.312 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-09-11T22:07:48.313 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 2337, 2024-09-11T22:07:48.313 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 132273248, 2024-09-11T22:07:48.313 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 2337, 2024-09-11T22:07:48.313 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 132273248, 2024-09-11T22:07:48.313 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-09-11T22:07:48.313 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:48.313 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:48.313 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:48.314 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-11T22:07:48.314 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:48.314 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-objecter_ops": { 2024-09-11T22:07:48.314 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-09-11T22:07:48.314 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 1024, 2024-09-11T22:07:48.314 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-09-11T22:07:48.314 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-09-11T22:07:48.314 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-09-11T22:07:48.315 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-09-11T22:07:48.315 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-09-11T22:07:48.315 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 2337, 2024-09-11T22:07:48.315 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 2337, 2024-09-11T22:07:48.315 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 2337, 2024-09-11T22:07:48.315 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 2337, 2024-09-11T22:07:48.315 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-09-11T22:07:48.315 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:48.316 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:48.316 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:48.316 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-11T22:07:48.316 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:48.316 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle": { 2024-09-11T22:07:48.316 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-09-11T22:07:48.316 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-09-11T22:07:48.317 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-09-11T22:07:48.317 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-09-11T22:07:48.317 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-09-11T22:07:48.317 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-09-11T22:07:48.317 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-09-11T22:07:48.317 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-09-11T22:07:48.317 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-09-11T22:07:48.317 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 0, 2024-09-11T22:07:48.318 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 0, 2024-09-11T22:07:48.318 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-09-11T22:07:48.318 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:48.318 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:48.318 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:48.318 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-11T22:07:48.318 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-09-11T22:07:48.318 INFO:teuthology.orchestra.run.smithi161.stdout: "throttle-write_buf_throttle-0x55d684f9a6e0": { 2024-09-11T22:07:48.319 INFO:teuthology.orchestra.run.smithi161.stdout: "val": 0, 2024-09-11T22:07:48.319 INFO:teuthology.orchestra.run.smithi161.stdout: "max": 3758096384, 2024-09-11T22:07:48.319 INFO:teuthology.orchestra.run.smithi161.stdout: "get_started": 0, 2024-09-11T22:07:48.319 INFO:teuthology.orchestra.run.smithi161.stdout: "get": 0, 2024-09-11T22:07:48.319 INFO:teuthology.orchestra.run.smithi161.stdout: "get_sum": 0, 2024-09-11T22:07:48.319 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_fail": 0, 2024-09-11T22:07:48.319 INFO:teuthology.orchestra.run.smithi161.stdout: "get_or_fail_success": 0, 2024-09-11T22:07:48.319 INFO:teuthology.orchestra.run.smithi161.stdout: "take": 0, 2024-09-11T22:07:48.320 INFO:teuthology.orchestra.run.smithi161.stdout: "take_sum": 0, 2024-09-11T22:07:48.320 INFO:teuthology.orchestra.run.smithi161.stdout: "put": 0, 2024-09-11T22:07:48.320 INFO:teuthology.orchestra.run.smithi161.stdout: "put_sum": 0, 2024-09-11T22:07:48.320 INFO:teuthology.orchestra.run.smithi161.stdout: "wait": { 2024-09-11T22:07:48.320 INFO:teuthology.orchestra.run.smithi161.stdout: "avgcount": 0, 2024-09-11T22:07:48.320 INFO:teuthology.orchestra.run.smithi161.stdout: "sum": 0.000000000, 2024-09-11T22:07:48.320 INFO:teuthology.orchestra.run.smithi161.stdout: "avgtime": 0.000000000 2024-09-11T22:07:48.320 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-11T22:07:48.321 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-09-11T22:07:48.321 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-09-11T22:07:48.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:48 smithi001 ceph-mon[30935]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:07:48.517 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:48 smithi161 ceph-mon[35615]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:07:48.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:48 smithi053 ceph-mon[34908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:07:49.004 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-09-11T22:07:49.005 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-09-11T22:07:49.005 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-09-11T22:07:49.005 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2024-09-11T22:07:49.005 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-09-11T22:07:49.006 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-09-11T22:07:49.006 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-09-11T22:07:49.006 INFO:tasks.check_counter:Daemon mds.l mds.dir_split=0 2024-09-11T22:07:49.006 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-11T22:07:49.019 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-11T22:07:49.020 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-11T22:07:49.020 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T22:07:49.047 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-09-11T22:07:49.047 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T22:07:49.108 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-11T22:07:49.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:49 smithi161 ceph-mon[35615]: pgmap v328: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:07:49.422 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:07:49.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:49 smithi001 ceph-mon[30935]: pgmap v328: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:07:49.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:49 smithi053 ceph-mon[34908]: pgmap v328: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:07:49.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:49.920+0000 7f885171a640 1 -- 172.21.15.1:0/1923735502 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f884c102d40 msgr2=0x7f884c103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:49.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:49.920+0000 7f885171a640 1 --2- 172.21.15.1:0/1923735502 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f884c102d40 0x7f884c103120 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f883c009900 tx=0x7f883c02a480 comp rx=0 tx=0).stop 2024-09-11T22:07:49.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:49.920+0000 7f885171a640 1 -- 172.21.15.1:0/1923735502 shutdown_connections 2024-09-11T22:07:49.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:49.920+0000 7f885171a640 1 --2- 172.21.15.1:0/1923735502 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f884c10c170 0x7f884c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:49.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:49.920+0000 7f885171a640 1 --2- 172.21.15.1:0/1923735502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f884c103660 0x7f884c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:49.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:49.920+0000 7f885171a640 1 --2- 172.21.15.1:0/1923735502 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f884c102d40 0x7f884c103120 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:49.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:49.920+0000 7f885171a640 1 -- 172.21.15.1:0/1923735502 >> 172.21.15.1:0/1923735502 conn(0x7f884c0fc7a0 msgr2=0x7f884c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:07:49.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:49.920+0000 7f885171a640 1 -- 172.21.15.1:0/1923735502 shutdown_connections 2024-09-11T22:07:49.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:49.920+0000 7f885171a640 1 -- 172.21.15.1:0/1923735502 wait complete. 2024-09-11T22:07:49.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:49.920+0000 7f885171a640 1 Processor -- start 2024-09-11T22:07:49.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:49.921+0000 7f885171a640 1 -- start start 2024-09-11T22:07:49.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:49.921+0000 7f885171a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f884c102d40 0x7f884c1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:49.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:49.921+0000 7f885171a640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f884c103660 0x7f884c100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:49.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:49.921+0000 7f885171a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f884c10c170 0x7f884c101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:49.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:49.921+0000 7f885171a640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f884c111f70 con 0x7f884c102d40 2024-09-11T22:07:49.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:49.921+0000 7f885171a640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f884c111df0 con 0x7f884c103660 2024-09-11T22:07:49.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:49.921+0000 7f885171a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f884c1120f0 con 0x7f884c10c170 2024-09-11T22:07:49.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:49.921+0000 7f884affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f884c102d40 0x7f884c1009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:49.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:49.921+0000 7f884affd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f884c102d40 0x7f884c1009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59348/0 (socket says 172.21.15.1:59348) 2024-09-11T22:07:49.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:49.921+0000 7f884affd640 1 -- 172.21.15.1:0/1063166423 learned_addr learned my addr 172.21.15.1:0/1063166423 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:07:49.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:49.921+0000 7f884a7fc640 1 --2- 172.21.15.1:0/1063166423 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f884c103660 0x7f884c100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:49.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:49.921+0000 7f884b7fe640 1 --2- 172.21.15.1:0/1063166423 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f884c10c170 0x7f884c101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:49.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:49.921+0000 7f884affd640 1 -- 172.21.15.1:0/1063166423 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f884c10c170 msgr2=0x7f884c101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:49.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:49.921+0000 7f884affd640 1 --2- 172.21.15.1:0/1063166423 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f884c10c170 0x7f884c101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:49.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:49.921+0000 7f884affd640 1 -- 172.21.15.1:0/1063166423 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f884c103660 msgr2=0x7f884c100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:49.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:49.921+0000 7f884affd640 1 --2- 172.21.15.1:0/1063166423 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f884c103660 0x7f884c100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:49.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:49.921+0000 7f884affd640 1 -- 172.21.15.1:0/1063166423 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f884c1a68f0 con 0x7f884c102d40 2024-09-11T22:07:49.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:49.921+0000 7f884affd640 1 --2- 172.21.15.1:0/1063166423 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f884c102d40 0x7f884c1009a0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f883c006010 tx=0x7f883c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:07:49.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:49.922+0000 7f8833fff640 1 -- 172.21.15.1:0/1063166423 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883c03f030 con 0x7f884c102d40 2024-09-11T22:07:49.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:49.922+0000 7f885171a640 1 -- 172.21.15.1:0/1063166423 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f884c1a6b80 con 0x7f884c102d40 2024-09-11T22:07:49.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:49.922+0000 7f8833fff640 1 -- 172.21.15.1:0/1063166423 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f883c003ce0 con 0x7f884c102d40 2024-09-11T22:07:49.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:49.922+0000 7f8833fff640 1 -- 172.21.15.1:0/1063166423 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883c035070 con 0x7f884c102d40 2024-09-11T22:07:49.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:49.922+0000 7f885171a640 1 -- 172.21.15.1:0/1063166423 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f884c1a70b0 con 0x7f884c102d40 2024-09-11T22:07:49.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:49.923+0000 7f885171a640 1 -- 172.21.15.1:0/1063166423 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f884c103120 con 0x7f884c102d40 2024-09-11T22:07:49.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:49.924+0000 7f8833fff640 1 -- 172.21.15.1:0/1063166423 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f883c042030 con 0x7f884c102d40 2024-09-11T22:07:49.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:49.925+0000 7f8833fff640 1 --2- 172.21.15.1:0/1063166423 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f88340a2cf0 0x7f88340a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:49.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:49.925+0000 7f884a7fc640 1 --2- 172.21.15.1:0/1063166423 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f88340a2cf0 0x7f88340a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:49.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:49.925+0000 7f8833fff640 1 -- 172.21.15.1:0/1063166423 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7f883c0efc90 con 0x7f884c102d40 2024-09-11T22:07:49.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:49.925+0000 7f884a7fc640 1 --2- 172.21.15.1:0/1063166423 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f88340a2cf0 0x7f88340a51b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f884c102430 tx=0x7f884000b2e0 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:07:49.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:49.928+0000 7f8833fff640 1 -- 172.21.15.1:0/1063166423 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f883c0bc8d0 con 0x7f884c102d40 2024-09-11T22:07:50.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:50.116+0000 7f885171a640 1 -- 172.21.15.1:0/1063166423 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f884c107210 con 0x7f884c102d40 2024-09-11T22:07:50.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:50.116+0000 7f8833fff640 1 -- 172.21.15.1:0/1063166423 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v95) ==== 81+0+929 (secure 0 0 0) 0x7f883c0bc1d0 con 0x7f884c102d40 2024-09-11T22:07:50.117 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/4286250488 2024-09-12T21:57:13.664392+0000 2024-09-11T22:07:50.118 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/2039944931 2024-09-12T21:57:13.664392+0000 2024-09-11T22:07:50.118 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/3202962495 2024-09-12T21:48:00.674728+0000 2024-09-11T22:07:50.118 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:6800/2311309446 2024-09-12T21:48:00.674728+0000 2024-09-11T22:07:50.118 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:6801/3890350886 2024-09-12T21:57:13.664392+0000 2024-09-11T22:07:50.118 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:6801/2311309446 2024-09-12T21:48:00.674728+0000 2024-09-11T22:07:50.118 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/2599895977 2024-09-12T21:48:00.674728+0000 2024-09-11T22:07:50.118 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/1278877732 2024-09-12T21:48:24.524913+0000 2024-09-11T22:07:50.119 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/276770268 2024-09-12T21:48:00.674728+0000 2024-09-11T22:07:50.119 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/1699984777 2024-09-12T21:48:24.524913+0000 2024-09-11T22:07:50.119 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:6800/2821761445 2024-09-12T21:48:24.524913+0000 2024-09-11T22:07:50.119 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/2797780151 2024-09-12T21:57:13.664392+0000 2024-09-11T22:07:50.119 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:6801/2821761445 2024-09-12T21:48:24.524913+0000 2024-09-11T22:07:50.119 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/1749020850 2024-09-12T21:48:24.524913+0000 2024-09-11T22:07:50.119 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/1202232270 2024-09-12T21:57:13.664392+0000 2024-09-11T22:07:50.119 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:6800/3890350886 2024-09-12T21:57:13.664392+0000 2024-09-11T22:07:50.120 INFO:teuthology.orchestra.run.smithi001.stderr:listed 16 entries 2024-09-11T22:07:50.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:50.119+0000 7f885171a640 1 -- 172.21.15.1:0/1063166423 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f88340a2cf0 msgr2=0x7f88340a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:50.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:50.119+0000 7f885171a640 1 --2- 172.21.15.1:0/1063166423 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f88340a2cf0 0x7f88340a51b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f884c102430 tx=0x7f884000b2e0 comp rx=0 tx=0).stop 2024-09-11T22:07:50.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:50.119+0000 7f885171a640 1 -- 172.21.15.1:0/1063166423 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f884c102d40 msgr2=0x7f884c1009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:50.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:50.119+0000 7f885171a640 1 --2- 172.21.15.1:0/1063166423 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f884c102d40 0x7f884c1009a0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f883c006010 tx=0x7f883c0042f0 comp rx=0 tx=0).stop 2024-09-11T22:07:50.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:50.119+0000 7f885171a640 1 -- 172.21.15.1:0/1063166423 shutdown_connections 2024-09-11T22:07:50.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:50.119+0000 7f885171a640 1 --2- 172.21.15.1:0/1063166423 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f88340a2cf0 0x7f88340a51b0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:50.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:50.119+0000 7f885171a640 1 --2- 172.21.15.1:0/1063166423 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f884c10c170 0x7f884c101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:50.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:50.119+0000 7f885171a640 1 --2- 172.21.15.1:0/1063166423 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f884c103660 0x7f884c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:50.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:50.119+0000 7f885171a640 1 --2- 172.21.15.1:0/1063166423 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f884c102d40 0x7f884c1009a0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:50.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:50.119+0000 7f885171a640 1 -- 172.21.15.1:0/1063166423 >> 172.21.15.1:0/1063166423 conn(0x7f884c0fc7a0 msgr2=0x7f884c0fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:07:50.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:50.119+0000 7f885171a640 1 -- 172.21.15.1:0/1063166423 shutdown_connections 2024-09-11T22:07:50.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:50.119+0000 7f885171a640 1 -- 172.21.15.1:0/1063166423 wait complete. 2024-09-11T22:07:50.297 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:50 smithi053 ceph-mon[34908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:07:50.445 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:50 smithi001 ceph-mon[30935]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:07:50.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:50 smithi161 ceph-mon[35615]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:07:50.971 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-11T22:07:50.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T22:07:50.971 DEBUG:teuthology.orchestra.run.smithi001:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-11T22:07:51.169 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi001.front.sepia.ceph.com 2024-09-11T22:07:51.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T22:07:51.170 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-11T22:07:51.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:51 smithi001 ceph-mon[30935]: pgmap v329: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 14 op/s 2024-09-11T22:07:51.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:51 smithi001 ceph-mon[30935]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:07:51.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:51 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/1063166423' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-11T22:07:51.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T22:07:51.224 DEBUG:teuthology.orchestra.run.smithi001:> ip netns list 2024-09-11T22:07:51.356 INFO:teuthology.orchestra.run.smithi001.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-11T22:07:51.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T22:07:51.357 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-11T22:07:51.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T22:07:51.403 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link delete ceph-brx 2024-09-11T22:07:51.465 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-11T22:07:51.480 INFO:tasks.cephadm:Teardown begin 2024-09-11T22:07:51.481 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-11T22:07:51.481 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:07:51.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:51 smithi053 ceph-mon[34908]: pgmap v329: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 14 op/s 2024-09-11T22:07:51.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:51 smithi053 ceph-mon[34908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:07:51.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:51 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/1063166423' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-11T22:07:51.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:51 smithi161 ceph-mon[35615]: pgmap v329: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 14 op/s 2024-09-11T22:07:51.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:51 smithi161 ceph-mon[35615]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:07:51.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:51 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/1063166423' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-11T22:07:51.740 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:07:52.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:52.247+0000 7fc8b10b7640 1 -- 172.21.15.1:0/257913104 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc8ac102790 msgr2=0x7fc8ac1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:52.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:52.247+0000 7fc8b10b7640 1 --2- 172.21.15.1:0/257913104 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc8ac102790 0x7fc8ac1067e0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fc8a0009900 tx=0x7fc8a002a410 comp rx=0 tx=0).stop 2024-09-11T22:07:52.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:52.247+0000 7fc8b10b7640 1 -- 172.21.15.1:0/257913104 shutdown_connections 2024-09-11T22:07:52.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:52.247+0000 7fc8b10b7640 1 --2- 172.21.15.1:0/257913104 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8ac106f10 0x7fc8ac10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:52.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:52.247+0000 7fc8b10b7640 1 --2- 172.21.15.1:0/257913104 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc8ac102790 0x7fc8ac1067e0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:52.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:52.247+0000 7fc8b10b7640 1 --2- 172.21.15.1:0/257913104 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8ac101de0 0x7fc8ac1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:52.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:52.247+0000 7fc8b10b7640 1 -- 172.21.15.1:0/257913104 >> 172.21.15.1:0/257913104 conn(0x7fc8ac0fd8c0 msgr2=0x7fc8ac0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:07:52.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:52.247+0000 7fc8b10b7640 1 -- 172.21.15.1:0/257913104 shutdown_connections 2024-09-11T22:07:52.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:52.247+0000 7fc8b10b7640 1 -- 172.21.15.1:0/257913104 wait complete. 2024-09-11T22:07:52.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:52.248+0000 7fc8b10b7640 1 Processor -- start 2024-09-11T22:07:52.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:52.248+0000 7fc8b10b7640 1 -- start start 2024-09-11T22:07:52.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:52.249+0000 7fc8b10b7640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8ac101de0 0x7fc8ac198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:52.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:52.249+0000 7fc8b10b7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8ac102790 0x7fc8ac198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:52.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:52.249+0000 7fc8b10b7640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc8ac106f10 0x7fc8ac19cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:52.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:52.249+0000 7fc8b10b7640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ac111f70 con 0x7fc8ac106f10 2024-09-11T22:07:52.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:52.249+0000 7fc8b10b7640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ac111df0 con 0x7fc8ac101de0 2024-09-11T22:07:52.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:52.249+0000 7fc8b10b7640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ac1120f0 con 0x7fc8ac102790 2024-09-11T22:07:52.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:52.249+0000 7fc8ab59e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc8ac106f10 0x7fc8ac19cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:52.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:52.249+0000 7fc8ab59e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc8ac106f10 0x7fc8ac19cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59362/0 (socket says 172.21.15.1:59362) 2024-09-11T22:07:52.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:52.249+0000 7fc8ab59e640 1 -- 172.21.15.1:0/575986264 learned_addr learned my addr 172.21.15.1:0/575986264 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:07:52.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:52.249+0000 7fc8aad9d640 1 --2- 172.21.15.1:0/575986264 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8ac101de0 0x7fc8ac198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:52.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:52.249+0000 7fc8aa59c640 1 --2- 172.21.15.1:0/575986264 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8ac102790 0x7fc8ac198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:52.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:52.249+0000 7fc8ab59e640 1 -- 172.21.15.1:0/575986264 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8ac102790 msgr2=0x7fc8ac198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:52.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:52.249+0000 7fc8ab59e640 1 --2- 172.21.15.1:0/575986264 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8ac102790 0x7fc8ac198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:52.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:52.249+0000 7fc8ab59e640 1 -- 172.21.15.1:0/575986264 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8ac101de0 msgr2=0x7fc8ac198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:52.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:52.249+0000 7fc8ab59e640 1 --2- 172.21.15.1:0/575986264 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8ac101de0 0x7fc8ac198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:52.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:52.249+0000 7fc8ab59e640 1 -- 172.21.15.1:0/575986264 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8ac19d400 con 0x7fc8ac106f10 2024-09-11T22:07:52.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:52.249+0000 7fc8ab59e640 1 --2- 172.21.15.1:0/575986264 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc8ac106f10 0x7fc8ac19cd20 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fc8a400eaa0 tx=0x7fc8a400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:07:52.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:52.249+0000 7fc893fff640 1 -- 172.21.15.1:0/575986264 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a400cdb0 con 0x7fc8ac106f10 2024-09-11T22:07:52.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:52.250+0000 7fc8b10b7640 1 -- 172.21.15.1:0/575986264 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8ac19d6f0 con 0x7fc8ac106f10 2024-09-11T22:07:52.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:52.250+0000 7fc893fff640 1 -- 172.21.15.1:0/575986264 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8a4005020 con 0x7fc8ac106f10 2024-09-11T22:07:52.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:52.250+0000 7fc893fff640 1 -- 172.21.15.1:0/575986264 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a40116d0 con 0x7fc8ac106f10 2024-09-11T22:07:52.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:52.250+0000 7fc8b10b7640 1 -- 172.21.15.1:0/575986264 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8ac1a4fd0 con 0x7fc8ac106f10 2024-09-11T22:07:52.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:52.251+0000 7fc8b10b7640 1 -- 172.21.15.1:0/575986264 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8ac1021c0 con 0x7fc8ac106f10 2024-09-11T22:07:52.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:52.252+0000 7fc893fff640 1 -- 172.21.15.1:0/575986264 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7fc8a40054d0 con 0x7fc8ac106f10 2024-09-11T22:07:52.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:52.253+0000 7fc893fff640 1 --2- 172.21.15.1:0/575986264 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fc8940a2d70 0x7fc8940a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:52.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:52.253+0000 7fc8aad9d640 1 --2- 172.21.15.1:0/575986264 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fc8940a2d70 0x7fc8940a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:52.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:52.253+0000 7fc893fff640 1 -- 172.21.15.1:0/575986264 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7fc8a40cf580 con 0x7fc8ac106f10 2024-09-11T22:07:52.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:52.253+0000 7fc8aad9d640 1 --2- 172.21.15.1:0/575986264 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fc8940a2d70 0x7fc8940a5230 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc89c0058c0 tx=0x7fc89c00a2d0 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:07:52.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:52.257+0000 7fc893fff640 1 -- 172.21.15.1:0/575986264 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc8a4016030 con 0x7fc8ac106f10 2024-09-11T22:07:52.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:52.471+0000 7fc8b10b7640 1 -- 172.21.15.1:0/575986264 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc8ac02d7c0 con 0x7fc8ac106f10 2024-09-11T22:07:52.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:52.471+0000 7fc893fff640 1 -- 172.21.15.1:0/575986264 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+11957 (secure 0 0 0) 0x7fc8a409bab0 con 0x7fc8ac106f10 2024-09-11T22:07:52.472 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T22:07:52.473 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":55,"btime":"2024-09-11T22:07:52:111655+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/421542832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":421542832},{"type":"v1","addr":"172.21.15.1:6835","nonce":421542832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2836864406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2836864406},{"type":"v1","addr":"172.21.15.1:6837","nonce":2836864406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":55,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T21:55:20.791263+0000","modified":"2024-09-11T22:07:51.374882+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24484,"mds_2":24467,"mds_3":24466,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14595":{"gid":14595,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/407834672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":407834672},{"type":"v1","addr":"172.21.15.1:6839","nonce":407834672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/1068827678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":1068827678},{"type":"v1","addr":"172.21.15.1:6841","nonce":1068827678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/872410024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":872410024},{"type":"v1","addr":"172.21.15.161:6833","nonce":872410024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/1648944570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1648944570},{"type":"v1","addr":"172.21.15.53:6833","nonce":1648944570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/2126933993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2126933993},{"type":"v1","addr":"172.21.15.53:6835","nonce":2126933993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.161:6835/828381592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":828381592},{"type":"v1","addr":"172.21.15.161:6835","nonce":828381592}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.161:6837/1799318656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1799318656},{"type":"v1","addr":"172.21.15.161:6837","nonce":1799318656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.53:6837/3457131830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3457131830},{"type":"v1","addr":"172.21.15.53:6837","nonce":3457131830}]},"join_fscid":-1,"export_targets":[1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/455365383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":455365383},{"type":"v1","addr":"172.21.15.161:6839","nonce":455365383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":162,"addr":"172.21.15.53:6839/1378584447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1378584447},{"type":"v1","addr":"172.21.15.53:6839","nonce":1378584447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24484,24467,24466,24460]},"id":1}]} 2024-09-11T22:07:52.473 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 55 2024-09-11T22:07:52.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:52.474+0000 7fc8b10b7640 1 -- 172.21.15.1:0/575986264 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fc8940a2d70 msgr2=0x7fc8940a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:52.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:52.474+0000 7fc8b10b7640 1 --2- 172.21.15.1:0/575986264 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fc8940a2d70 0x7fc8940a5230 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc89c0058c0 tx=0x7fc89c00a2d0 comp rx=0 tx=0).stop 2024-09-11T22:07:52.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:52.474+0000 7fc8b10b7640 1 -- 172.21.15.1:0/575986264 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc8ac106f10 msgr2=0x7fc8ac19cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:52.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:52.474+0000 7fc8b10b7640 1 --2- 172.21.15.1:0/575986264 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc8ac106f10 0x7fc8ac19cd20 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fc8a400eaa0 tx=0x7fc8a400ef60 comp rx=0 tx=0).stop 2024-09-11T22:07:52.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:52.474+0000 7fc8b10b7640 1 -- 172.21.15.1:0/575986264 shutdown_connections 2024-09-11T22:07:52.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:52.474+0000 7fc8b10b7640 1 --2- 172.21.15.1:0/575986264 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fc8940a2d70 0x7fc8940a5230 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:52.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:52.474+0000 7fc8b10b7640 1 --2- 172.21.15.1:0/575986264 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc8ac106f10 0x7fc8ac19cd20 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:52.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:52.474+0000 7fc8b10b7640 1 --2- 172.21.15.1:0/575986264 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc8ac102790 0x7fc8ac198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:52.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:52.474+0000 7fc8b10b7640 1 --2- 172.21.15.1:0/575986264 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc8ac101de0 0x7fc8ac198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:52.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:52.474+0000 7fc8b10b7640 1 -- 172.21.15.1:0/575986264 >> 172.21.15.1:0/575986264 conn(0x7fc8ac0fd8c0 msgr2=0x7fc8ac0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:07:52.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:52.474+0000 7fc8b10b7640 1 -- 172.21.15.1:0/575986264 shutdown_connections 2024-09-11T22:07:52.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:52.474+0000 7fc8b10b7640 1 -- 172.21.15.1:0/575986264 wait complete. 2024-09-11T22:07:53.110 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:07:53.370 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:07:53.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:53 smithi001 ceph-mon[30935]: pgmap v330: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:07:53.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:53 smithi001 ceph-mon[30935]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:07:53.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:53 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/575986264' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:07:53.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:53 smithi053 ceph-mon[34908]: pgmap v330: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:07:53.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:53 smithi053 ceph-mon[34908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:07:53.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:53 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/575986264' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:07:53.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:53 smithi161 ceph-mon[35615]: pgmap v330: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:07:53.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:53 smithi161 ceph-mon[35615]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:07:53.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:53 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/575986264' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:07:53.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:53.845+0000 7f0e93716640 1 -- 172.21.15.1:0/3663366537 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e8c074620 msgr2=0x7f0e8c074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:53.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:53.845+0000 7f0e93716640 1 --2- 172.21.15.1:0/3663366537 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e8c074620 0x7f0e8c074a00 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f0e88009900 tx=0x7f0e8802a480 comp rx=0 tx=0).stop 2024-09-11T22:07:53.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:53.845+0000 7f0e93716640 1 -- 172.21.15.1:0/3663366537 shutdown_connections 2024-09-11T22:07:53.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:53.845+0000 7f0e93716640 1 --2- 172.21.15.1:0/3663366537 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0e8c10e3c0 0x7f0e8c110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:53.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:53.845+0000 7f0e93716640 1 --2- 172.21.15.1:0/3663366537 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e8c074f40 0x7f0e8c10de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:53.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:53.845+0000 7f0e93716640 1 --2- 172.21.15.1:0/3663366537 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e8c074620 0x7f0e8c074a00 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:53.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:53.845+0000 7f0e93716640 1 -- 172.21.15.1:0/3663366537 >> 172.21.15.1:0/3663366537 conn(0x7f0e8c0fd8c0 msgr2=0x7f0e8c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:07:53.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:53.845+0000 7f0e93716640 1 -- 172.21.15.1:0/3663366537 shutdown_connections 2024-09-11T22:07:53.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:53.845+0000 7f0e93716640 1 -- 172.21.15.1:0/3663366537 wait complete. 2024-09-11T22:07:53.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:53.846+0000 7f0e93716640 1 Processor -- start 2024-09-11T22:07:53.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:53.846+0000 7f0e93716640 1 -- start start 2024-09-11T22:07:53.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:53.846+0000 7f0e93716640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e8c074620 0x7f0e8c19a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:53.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:53.846+0000 7f0e93716640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e8c074f40 0x7f0e8c19abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:53.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:53.846+0000 7f0e93716640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0e8c10e3c0 0x7f0e8c19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:53.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:53.846+0000 7f0e93716640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8c114170 con 0x7f0e8c074f40 2024-09-11T22:07:53.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:53.846+0000 7f0e93716640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8c113ff0 con 0x7f0e8c10e3c0 2024-09-11T22:07:53.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:53.846+0000 7f0e93716640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8c1142f0 con 0x7f0e8c074620 2024-09-11T22:07:53.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:53.846+0000 7f0e90cb1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e8c074f40 0x7f0e8c19abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:53.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:53.846+0000 7f0e90cb1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e8c074f40 0x7f0e8c19abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59384/0 (socket says 172.21.15.1:59384) 2024-09-11T22:07:53.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:53.846+0000 7f0e90cb1640 1 -- 172.21.15.1:0/422473536 learned_addr learned my addr 172.21.15.1:0/422473536 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:07:53.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:53.846+0000 7f0e914b2640 1 --2- 172.21.15.1:0/422473536 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e8c074620 0x7f0e8c19a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:53.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:53.847+0000 7f0e91cb3640 1 --2- 172.21.15.1:0/422473536 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0e8c10e3c0 0x7f0e8c19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:53.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:53.847+0000 7f0e90cb1640 1 -- 172.21.15.1:0/422473536 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e8c074620 msgr2=0x7f0e8c19a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:53.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:53.847+0000 7f0e90cb1640 1 --2- 172.21.15.1:0/422473536 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e8c074620 0x7f0e8c19a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:53.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:53.847+0000 7f0e90cb1640 1 -- 172.21.15.1:0/422473536 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0e8c10e3c0 msgr2=0x7f0e8c19ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:53.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:53.847+0000 7f0e90cb1640 1 --2- 172.21.15.1:0/422473536 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0e8c10e3c0 0x7f0e8c19ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:53.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:53.847+0000 7f0e90cb1640 1 -- 172.21.15.1:0/422473536 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e8c19f620 con 0x7f0e8c074f40 2024-09-11T22:07:53.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:53.847+0000 7f0e91cb3640 1 --2- 172.21.15.1:0/422473536 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0e8c10e3c0 0x7f0e8c19ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:07:53.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:53.847+0000 7f0e90cb1640 1 --2- 172.21.15.1:0/422473536 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e8c074f40 0x7f0e8c19abb0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c00d770 tx=0x7f0e7c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:07:53.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:53.847+0000 7f0e827fc640 1 -- 172.21.15.1:0/422473536 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c017070 con 0x7f0e8c074f40 2024-09-11T22:07:53.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:53.847+0000 7f0e827fc640 1 -- 172.21.15.1:0/422473536 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e7c005500 con 0x7f0e8c074f40 2024-09-11T22:07:53.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:53.847+0000 7f0e93716640 1 -- 172.21.15.1:0/422473536 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e8c19f910 con 0x7f0e8c074f40 2024-09-11T22:07:53.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:53.847+0000 7f0e827fc640 1 -- 172.21.15.1:0/422473536 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c004e00 con 0x7f0e8c074f40 2024-09-11T22:07:53.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:53.847+0000 7f0e93716640 1 -- 172.21.15.1:0/422473536 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e8c1a71f0 con 0x7f0e8c074f40 2024-09-11T22:07:53.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:53.848+0000 7f0e93716640 1 -- 172.21.15.1:0/422473536 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e8c074a00 con 0x7f0e8c074f40 2024-09-11T22:07:53.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:53.849+0000 7f0e827fc640 1 -- 172.21.15.1:0/422473536 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f0e7c004fa0 con 0x7f0e8c074f40 2024-09-11T22:07:53.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:53.850+0000 7f0e827fc640 1 --2- 172.21.15.1:0/422473536 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f0e780a2d70 0x7f0e780a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:53.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:53.851+0000 7f0e914b2640 1 --2- 172.21.15.1:0/422473536 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f0e780a2d70 0x7f0e780a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:53.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:53.851+0000 7f0e827fc640 1 -- 172.21.15.1:0/422473536 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7f0e7c0cf2d0 con 0x7f0e8c074f40 2024-09-11T22:07:53.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:53.851+0000 7f0e914b2640 1 --2- 172.21.15.1:0/422473536 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f0e780a2d70 0x7f0e780a5230 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0e88009230 tx=0x7f0e880058d0 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:07:53.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:53.854+0000 7f0e827fc640 1 -- 172.21.15.1:0/422473536 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0e7c09bed0 con 0x7f0e8c074f40 2024-09-11T22:07:54.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:54.065+0000 7f0e93716640 1 -- 172.21.15.1:0/422473536 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0e8c02d7c0 con 0x7f0e8c074f40 2024-09-11T22:07:54.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:54.066+0000 7f0e827fc640 1 -- 172.21.15.1:0/422473536 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+11957 (secure 0 0 0) 0x7f0e7c09b7d0 con 0x7f0e8c074f40 2024-09-11T22:07:54.067 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T22:07:54.067 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":55,"btime":"2024-09-11T22:07:52:111655+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/421542832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":421542832},{"type":"v1","addr":"172.21.15.1:6835","nonce":421542832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2836864406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2836864406},{"type":"v1","addr":"172.21.15.1:6837","nonce":2836864406}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":55,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T21:55:20.791263+0000","modified":"2024-09-11T22:07:51.374882+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24494,"mds_1":24484,"mds_2":24467,"mds_3":24466,"mds_4":24460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14595":{"gid":14595,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/407834672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":407834672},{"type":"v1","addr":"172.21.15.1:6839","nonce":407834672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14601":{"gid":14601,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/1068827678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":1068827678},{"type":"v1","addr":"172.21.15.1:6841","nonce":1068827678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.161:6833/872410024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":872410024},{"type":"v1","addr":"172.21.15.161:6833","nonce":872410024}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6833/1648944570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":1648944570},{"type":"v1","addr":"172.21.15.53:6833","nonce":1648944570}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/2126933993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2126933993},{"type":"v1","addr":"172.21.15.53:6835","nonce":2126933993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.161:6835/828381592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":828381592},{"type":"v1","addr":"172.21.15.161:6835","nonce":828381592}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.161:6837/1799318656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1799318656},{"type":"v1","addr":"172.21.15.161:6837","nonce":1799318656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.53:6837/3457131830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3457131830},{"type":"v1","addr":"172.21.15.53:6837","nonce":3457131830}]},"join_fscid":-1,"export_targets":[1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.161:6839/455365383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6838","nonce":455365383},{"type":"v1","addr":"172.21.15.161:6839","nonce":455365383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":162,"addr":"172.21.15.53:6839/1378584447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1378584447},{"type":"v1","addr":"172.21.15.53:6839","nonce":1378584447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24484,24467,24466,24460]},"id":1}]} 2024-09-11T22:07:54.068 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 55 2024-09-11T22:07:54.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:54.068+0000 7f0e93716640 1 -- 172.21.15.1:0/422473536 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f0e780a2d70 msgr2=0x7f0e780a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:54.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:54.068+0000 7f0e93716640 1 --2- 172.21.15.1:0/422473536 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f0e780a2d70 0x7f0e780a5230 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0e88009230 tx=0x7f0e880058d0 comp rx=0 tx=0).stop 2024-09-11T22:07:54.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:54.068+0000 7f0e93716640 1 -- 172.21.15.1:0/422473536 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e8c074f40 msgr2=0x7f0e8c19abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:54.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:54.068+0000 7f0e93716640 1 --2- 172.21.15.1:0/422473536 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e8c074f40 0x7f0e8c19abb0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c00d770 tx=0x7f0e7c00dc30 comp rx=0 tx=0).stop 2024-09-11T22:07:54.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:54.068+0000 7f0e93716640 1 -- 172.21.15.1:0/422473536 shutdown_connections 2024-09-11T22:07:54.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:54.068+0000 7f0e93716640 1 --2- 172.21.15.1:0/422473536 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f0e780a2d70 0x7f0e780a5230 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:54.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:54.068+0000 7f0e93716640 1 --2- 172.21.15.1:0/422473536 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0e8c10e3c0 0x7f0e8c19ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:54.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:54.068+0000 7f0e93716640 1 --2- 172.21.15.1:0/422473536 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e8c074f40 0x7f0e8c19abb0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:54.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:54.068+0000 7f0e93716640 1 --2- 172.21.15.1:0/422473536 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e8c074620 0x7f0e8c19a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:54.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:54.068+0000 7f0e93716640 1 -- 172.21.15.1:0/422473536 >> 172.21.15.1:0/422473536 conn(0x7f0e8c0fd8c0 msgr2=0x7f0e8c0fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:07:54.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:54.068+0000 7f0e93716640 1 -- 172.21.15.1:0/422473536 shutdown_connections 2024-09-11T22:07:54.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:54.068+0000 7f0e93716640 1 -- 172.21.15.1:0/422473536 wait complete. 2024-09-11T22:07:54.342 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:54 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/422473536' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:07:54.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:54 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/422473536' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:07:54.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:54 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/422473536' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:07:54.728 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T22:07:54.985 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:07:55.367 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:55 smithi001 ceph-mon[30935]: pgmap v331: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:07:55.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:55.449+0000 7f3291e32640 1 -- 172.21.15.1:0/2263735367 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f328c106a60 msgr2=0x7f328c106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:55.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:55.449+0000 7f3291e32640 1 --2- 172.21.15.1:0/2263735367 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f328c106a60 0x7f328c106e40 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f327c009930 tx=0x7f327c02a420 comp rx=0 tx=0).stop 2024-09-11T22:07:55.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:55.449+0000 7f3291e32640 1 -- 172.21.15.1:0/2263735367 shutdown_connections 2024-09-11T22:07:55.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:55.449+0000 7f3291e32640 1 --2- 172.21.15.1:0/2263735367 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f328c101400 0x7f328c10d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:55.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:55.449+0000 7f3291e32640 1 --2- 172.21.15.1:0/2263735367 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f328c100a60 0x7f328c100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:55.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:55.449+0000 7f3291e32640 1 --2- 172.21.15.1:0/2263735367 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f328c106a60 0x7f328c106e40 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:55.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:55.449+0000 7f3291e32640 1 -- 172.21.15.1:0/2263735367 >> 172.21.15.1:0/2263735367 conn(0x7f328c0fc780 msgr2=0x7f328c0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:07:55.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:55.449+0000 7f3291e32640 1 -- 172.21.15.1:0/2263735367 shutdown_connections 2024-09-11T22:07:55.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:55.449+0000 7f3291e32640 1 -- 172.21.15.1:0/2263735367 wait complete. 2024-09-11T22:07:55.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:55.449+0000 7f3291e32640 1 Processor -- start 2024-09-11T22:07:55.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:55.449+0000 7f3291e32640 1 -- start start 2024-09-11T22:07:55.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:55.450+0000 7f3291e32640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f328c100a60 0x7f328c10e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:55.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:55.450+0000 7f3291e32640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f328c101400 0x7f328c10e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:55.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:55.450+0000 7f3291e32640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f328c106a60 0x7f328c10ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:55.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:55.450+0000 7f3291e32640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f328c111f80 con 0x7f328c106a60 2024-09-11T22:07:55.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:55.450+0000 7f3291e32640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f328c111e00 con 0x7f328c100a60 2024-09-11T22:07:55.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:55.450+0000 7f3291e32640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f328c112100 con 0x7f328c101400 2024-09-11T22:07:55.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:55.450+0000 7f328bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f328c106a60 0x7f328c10ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:55.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:55.450+0000 7f328bfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f328c106a60 0x7f328c10ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59404/0 (socket says 172.21.15.1:59404) 2024-09-11T22:07:55.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:55.450+0000 7f328bfff640 1 -- 172.21.15.1:0/2119863806 learned_addr learned my addr 172.21.15.1:0/2119863806 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:07:55.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:55.450+0000 7f328b7fe640 1 --2- 172.21.15.1:0/2119863806 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f328c100a60 0x7f328c10e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:55.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:55.450+0000 7f3282ffd640 1 --2- 172.21.15.1:0/2119863806 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f328c101400 0x7f328c10e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:55.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:55.450+0000 7f328bfff640 1 -- 172.21.15.1:0/2119863806 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f328c101400 msgr2=0x7f328c10e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:55.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:55.450+0000 7f328bfff640 1 --2- 172.21.15.1:0/2119863806 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f328c101400 0x7f328c10e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:55.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:55.450+0000 7f328bfff640 1 -- 172.21.15.1:0/2119863806 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f328c100a60 msgr2=0x7f328c10e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:55.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:55.450+0000 7f328bfff640 1 --2- 172.21.15.1:0/2119863806 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f328c100a60 0x7f328c10e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:55.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:55.450+0000 7f328bfff640 1 -- 172.21.15.1:0/2119863806 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f328c1a6a00 con 0x7f328c106a60 2024-09-11T22:07:55.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:55.451+0000 7f328bfff640 1 --2- 172.21.15.1:0/2119863806 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f328c106a60 0x7f328c10ee40 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f328400cad0 tx=0x7f328400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:07:55.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:55.451+0000 7f32897fa640 1 -- 172.21.15.1:0/2119863806 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f328400ae20 con 0x7f328c106a60 2024-09-11T22:07:55.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:55.451+0000 7f3291e32640 1 -- 172.21.15.1:0/2119863806 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f328c1a6cf0 con 0x7f328c106a60 2024-09-11T22:07:55.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:55.451+0000 7f32897fa640 1 -- 172.21.15.1:0/2119863806 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3284005020 con 0x7f328c106a60 2024-09-11T22:07:55.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:55.451+0000 7f32897fa640 1 -- 172.21.15.1:0/2119863806 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f328400f6f0 con 0x7f328c106a60 2024-09-11T22:07:55.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:55.451+0000 7f3291e32640 1 -- 172.21.15.1:0/2119863806 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f328c1a71e0 con 0x7f328c106a60 2024-09-11T22:07:55.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:55.452+0000 7f3291e32640 1 -- 172.21.15.1:0/2119863806 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f328c100ec0 con 0x7f328c106a60 2024-09-11T22:07:55.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:55.454+0000 7f32897fa640 1 -- 172.21.15.1:0/2119863806 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f32840051d0 con 0x7f328c106a60 2024-09-11T22:07:55.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:55.454+0000 7f32897fa640 1 --2- 172.21.15.1:0/2119863806 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f32780a2cf0 0x7f32780a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:55.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:55.455+0000 7f328b7fe640 1 --2- 172.21.15.1:0/2119863806 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f32780a2cf0 0x7f32780a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:55.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:55.455+0000 7f32897fa640 1 -- 172.21.15.1:0/2119863806 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7f32840cf890 con 0x7f328c106a60 2024-09-11T22:07:55.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:55.455+0000 7f328b7fe640 1 --2- 172.21.15.1:0/2119863806 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f32780a2cf0 0x7f32780a51b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f327c009900 tx=0x7f327c0058d0 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:07:55.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:55.458+0000 7f32897fa640 1 -- 172.21.15.1:0/2119863806 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3284014030 con 0x7f328c106a60 2024-09-11T22:07:55.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:55 smithi053 ceph-mon[34908]: pgmap v331: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:07:55.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:55 smithi161 ceph-mon[35615]: pgmap v331: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:07:55.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:55.634+0000 7f3291e32640 1 -- 172.21.15.1:0/2119863806 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f328c10f8f0 con 0x7f328c106a60 2024-09-11T22:07:55.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:55.635+0000 7f32897fa640 1 -- 172.21.15.1:0/2119863806 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+23213 (secure 0 0 0) 0x7f328c10f8f0 con 0x7f328c106a60 2024-09-11T22:07:55.636 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-09-11T22:07:55.637 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":95,"fsid":"3ffca83c-7087-11ef-bcea-c7b262605968","created":"2024-09-11T21:47:28.868707+0000","modified":"2024-09-11T22:05:45.511990+0000","last_up_change":"2024-09-11T21:54:29.319441+0000","last_in_change":"2024-09-11T21:54:05.532281+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T21:50:41.026490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T21:55:16.961999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T21:55:19.066228+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":9,"snap_epoch":90,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"19e56504-18f1-4234-b2d6-f193ff47d8eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":4012142023},{"type":"v1","addr":"172.21.15.1:6803","nonce":4012142023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":4012142023},{"type":"v1","addr":"172.21.15.1:6805","nonce":4012142023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":4012142023},{"type":"v1","addr":"172.21.15.1:6809","nonce":4012142023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":4012142023},{"type":"v1","addr":"172.21.15.1:6807","nonce":4012142023}]},"public_addr":"172.21.15.1:6803/4012142023","cluster_addr":"172.21.15.1:6805/4012142023","heartbeat_back_addr":"172.21.15.1:6809/4012142023","heartbeat_front_addr":"172.21.15.1:6807/4012142023","state":["exists","up"]},{"osd":1,"uuid":"996c9f0c-c5a4-43b8-b8e0-d712f27c5dfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":3039391924},{"type":"v1","addr":"172.21.15.53:6801","nonce":3039391924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":3039391924},{"type":"v1","addr":"172.21.15.53:6803","nonce":3039391924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":3039391924},{"type":"v1","addr":"172.21.15.53:6807","nonce":3039391924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":3039391924},{"type":"v1","addr":"172.21.15.53:6805","nonce":3039391924}]},"public_addr":"172.21.15.53:6801/3039391924","cluster_addr":"172.21.15.53:6803/3039391924","heartbeat_back_addr":"172.21.15.53:6807/3039391924","heartbeat_front_addr":"172.21.15.53:6805/3039391924","state":["exists","up"]},{"osd":2,"uuid":"578d4260-41e1-4645-b9b1-1c5445712c13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":1707914885},{"type":"v1","addr":"172.21.15.161:6801","nonce":1707914885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":1707914885},{"type":"v1","addr":"172.21.15.161:6803","nonce":1707914885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":1707914885},{"type":"v1","addr":"172.21.15.161:6807","nonce":1707914885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":1707914885},{"type":"v1","addr":"172.21.15.161:6805","nonce":1707914885}]},"public_addr":"172.21.15.161:6801/1707914885","cluster_addr":"172.21.15.161:6803/1707914885","heartbeat_back_addr":"172.21.15.161:6807/1707914885","heartbeat_front_addr":"172.21.15.161:6805/1707914885","state":["exists","up"]},{"osd":3,"uuid":"73f78766-9662-4aa3-841c-fe2ce654ee9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1541142398},{"type":"v1","addr":"172.21.15.1:6811","nonce":1541142398}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1541142398},{"type":"v1","addr":"172.21.15.1:6813","nonce":1541142398}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1541142398},{"type":"v1","addr":"172.21.15.1:6817","nonce":1541142398}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1541142398},{"type":"v1","addr":"172.21.15.1:6815","nonce":1541142398}]},"public_addr":"172.21.15.1:6811/1541142398","cluster_addr":"172.21.15.1:6813/1541142398","heartbeat_back_addr":"172.21.15.1:6817/1541142398","heartbeat_front_addr":"172.21.15.1:6815/1541142398","state":["exists","up"]},{"osd":4,"uuid":"56c808bb-d4cd-4969-b49e-adef49ecd0f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":3081402681},{"type":"v1","addr":"172.21.15.53:6809","nonce":3081402681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3081402681},{"type":"v1","addr":"172.21.15.53:6811","nonce":3081402681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3081402681},{"type":"v1","addr":"172.21.15.53:6815","nonce":3081402681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3081402681},{"type":"v1","addr":"172.21.15.53:6813","nonce":3081402681}]},"public_addr":"172.21.15.53:6809/3081402681","cluster_addr":"172.21.15.53:6811/3081402681","heartbeat_back_addr":"172.21.15.53:6815/3081402681","heartbeat_front_addr":"172.21.15.53:6813/3081402681","state":["exists","up"]},{"osd":5,"uuid":"7b8266d6-b507-47c9-b4fa-ff16c4a74d35","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":1622081327},{"type":"v1","addr":"172.21.15.161:6809","nonce":1622081327}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":1622081327},{"type":"v1","addr":"172.21.15.161:6811","nonce":1622081327}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":1622081327},{"type":"v1","addr":"172.21.15.161:6815","nonce":1622081327}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":1622081327},{"type":"v1","addr":"172.21.15.161:6813","nonce":1622081327}]},"public_addr":"172.21.15.161:6809/1622081327","cluster_addr":"172.21.15.161:6811/1622081327","heartbeat_back_addr":"172.21.15.161:6815/1622081327","heartbeat_front_addr":"172.21.15.161:6813/1622081327","state":["exists","up"]},{"osd":6,"uuid":"3af5925d-a749-4849-a076-80580222cad4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":158434853},{"type":"v1","addr":"172.21.15.1:6819","nonce":158434853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":158434853},{"type":"v1","addr":"172.21.15.1:6821","nonce":158434853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":158434853},{"type":"v1","addr":"172.21.15.1:6825","nonce":158434853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":158434853},{"type":"v1","addr":"172.21.15.1:6823","nonce":158434853}]},"public_addr":"172.21.15.1:6819/158434853","cluster_addr":"172.21.15.1:6821/158434853","heartbeat_back_addr":"172.21.15.1:6825/158434853","heartbeat_front_addr":"172.21.15.1:6823/158434853","state":["exists","up"]},{"osd":7,"uuid":"f8f26c3c-17e9-4e89-8392-289796e0c8a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2132057697},{"type":"v1","addr":"172.21.15.53:6817","nonce":2132057697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2132057697},{"type":"v1","addr":"172.21.15.53:6819","nonce":2132057697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2132057697},{"type":"v1","addr":"172.21.15.53:6823","nonce":2132057697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2132057697},{"type":"v1","addr":"172.21.15.53:6821","nonce":2132057697}]},"public_addr":"172.21.15.53:6817/2132057697","cluster_addr":"172.21.15.53:6819/2132057697","heartbeat_back_addr":"172.21.15.53:6823/2132057697","heartbeat_front_addr":"172.21.15.53:6821/2132057697","state":["exists","up"]},{"osd":8,"uuid":"36ea401b-3c93-4b9d-8880-298d1eb6e30b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":1838248669},{"type":"v1","addr":"172.21.15.161:6817","nonce":1838248669}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1838248669},{"type":"v1","addr":"172.21.15.161:6819","nonce":1838248669}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1838248669},{"type":"v1","addr":"172.21.15.161:6823","nonce":1838248669}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1838248669},{"type":"v1","addr":"172.21.15.161:6821","nonce":1838248669}]},"public_addr":"172.21.15.161:6817/1838248669","cluster_addr":"172.21.15.161:6819/1838248669","heartbeat_back_addr":"172.21.15.161:6823/1838248669","heartbeat_front_addr":"172.21.15.161:6821/1838248669","state":["exists","up"]},{"osd":9,"uuid":"4b788d3f-a3bc-4463-bd99-95fdb7a52768","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2806274589},{"type":"v1","addr":"172.21.15.1:6827","nonce":2806274589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2806274589},{"type":"v1","addr":"172.21.15.1:6829","nonce":2806274589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":2806274589},{"type":"v1","addr":"172.21.15.1:6833","nonce":2806274589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2806274589},{"type":"v1","addr":"172.21.15.1:6831","nonce":2806274589}]},"public_addr":"172.21.15.1:6827/2806274589","cluster_addr":"172.21.15.1:6829/2806274589","heartbeat_back_addr":"172.21.15.1:6833/2806274589","heartbeat_front_addr":"172.21.15.1:6831/2806274589","state":["exists","up"]},{"osd":10,"uuid":"70549796-5270-4ae2-ac3f-f61f5de105d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":761923580},{"type":"v1","addr":"172.21.15.53:6825","nonce":761923580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":761923580},{"type":"v1","addr":"172.21.15.53:6827","nonce":761923580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":761923580},{"type":"v1","addr":"172.21.15.53:6831","nonce":761923580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":761923580},{"type":"v1","addr":"172.21.15.53:6829","nonce":761923580}]},"public_addr":"172.21.15.53:6825/761923580","cluster_addr":"172.21.15.53:6827/761923580","heartbeat_back_addr":"172.21.15.53:6831/761923580","heartbeat_front_addr":"172.21.15.53:6829/761923580","state":["exists","up"]},{"osd":11,"uuid":"48f17c4a-370a-493e-956e-bf6ad3dcc594","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1586352190},{"type":"v1","addr":"172.21.15.161:6825","nonce":1586352190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":1586352190},{"type":"v1","addr":"172.21.15.161:6827","nonce":1586352190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":1586352190},{"type":"v1","addr":"172.21.15.161:6831","nonce":1586352190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":1586352190},{"type":"v1","addr":"172.21.15.161:6829","nonce":1586352190}]},"public_addr":"172.21.15.161:6825/1586352190","cluster_addr":"172.21.15.161:6827/1586352190","heartbeat_back_addr":"172.21.15.161:6831/1586352190","heartbeat_front_addr":"172.21.15.161:6829/1586352190","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:49:46.539613+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:50:11.912300+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:50:36.955490+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:51:02.200554+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:51:27.860585+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:51:54.694099+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:52:18.962351+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:52:44.881923+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:53:10.301390+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:53:35.417647+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:54:01.492219+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T21:54:26.938338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.26","mappings":[{"from":1,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/4286250488":"2024-09-12T21:57:13.664392+0000","172.21.15.1:0/2039944931":"2024-09-12T21:57:13.664392+0000","172.21.15.1:0/3202962495":"2024-09-12T21:48:00.674728+0000","172.21.15.1:6800/2311309446":"2024-09-12T21:48:00.674728+0000","172.21.15.1:6801/3890350886":"2024-09-12T21:57:13.664392+0000","172.21.15.1:6801/2311309446":"2024-09-12T21:48:00.674728+0000","172.21.15.1:0/2599895977":"2024-09-12T21:48:00.674728+0000","172.21.15.1:0/1278877732":"2024-09-12T21:48:24.524913+0000","172.21.15.1:0/276770268":"2024-09-12T21:48:00.674728+0000","172.21.15.1:0/1699984777":"2024-09-12T21:48:24.524913+0000","172.21.15.1:6800/2821761445":"2024-09-12T21:48:24.524913+0000","172.21.15.1:0/2797780151":"2024-09-12T21:57:13.664392+0000","172.21.15.1:6801/2821761445":"2024-09-12T21:48:24.524913+0000","172.21.15.1:0/1749020850":"2024-09-12T21:48:24.524913+0000","172.21.15.1:0/1202232270":"2024-09-12T21:57:13.664392+0000","172.21.15.1:6800/3890350886":"2024-09-12T21:57:13.664392+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T22:07:55.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:55.638+0000 7f3291e32640 1 -- 172.21.15.1:0/2119863806 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f32780a2cf0 msgr2=0x7f32780a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:55.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:55.638+0000 7f3291e32640 1 --2- 172.21.15.1:0/2119863806 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f32780a2cf0 0x7f32780a51b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f327c009900 tx=0x7f327c0058d0 comp rx=0 tx=0).stop 2024-09-11T22:07:55.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:55.638+0000 7f3291e32640 1 -- 172.21.15.1:0/2119863806 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f328c106a60 msgr2=0x7f328c10ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:55.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:55.638+0000 7f3291e32640 1 --2- 172.21.15.1:0/2119863806 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f328c106a60 0x7f328c10ee40 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f328400cad0 tx=0x7f328400cf90 comp rx=0 tx=0).stop 2024-09-11T22:07:55.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:55.638+0000 7f3291e32640 1 -- 172.21.15.1:0/2119863806 shutdown_connections 2024-09-11T22:07:55.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:55.638+0000 7f3291e32640 1 --2- 172.21.15.1:0/2119863806 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f32780a2cf0 0x7f32780a51b0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:55.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:55.638+0000 7f3291e32640 1 --2- 172.21.15.1:0/2119863806 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f328c106a60 0x7f328c10ee40 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:55.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:55.638+0000 7f3291e32640 1 --2- 172.21.15.1:0/2119863806 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f328c101400 0x7f328c10e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:55.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:55.638+0000 7f3291e32640 1 --2- 172.21.15.1:0/2119863806 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f328c100a60 0x7f328c10e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:55.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:55.638+0000 7f3291e32640 1 -- 172.21.15.1:0/2119863806 >> 172.21.15.1:0/2119863806 conn(0x7f328c0fc780 msgr2=0x7f328c0fd140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:07:55.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:55.638+0000 7f3291e32640 1 -- 172.21.15.1:0/2119863806 shutdown_connections 2024-09-11T22:07:55.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:55.638+0000 7f3291e32640 1 -- 172.21.15.1:0/2119863806 wait complete. 2024-09-11T22:07:56.280 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:56 smithi001 ceph-mon[30935]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:07:56.280 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:56 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/2119863806' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:07:56.311 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph fs fail cephfs 2024-09-11T22:07:56.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:56 smithi053 ceph-mon[34908]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:07:56.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:56 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/2119863806' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:07:56.572 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:07:56.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:56 smithi161 ceph-mon[35615]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:07:56.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:56 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/2119863806' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:07:57.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:57.075+0000 7fcc6dc43640 1 -- 172.21.15.1:0/959925067 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc68102790 msgr2=0x7fcc681067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:57.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:57.075+0000 7fcc6dc43640 1 --2- 172.21.15.1:0/959925067 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc68102790 0x7fcc681067e0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c009900 tx=0x7fcc5c02a410 comp rx=0 tx=0).stop 2024-09-11T22:07:57.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:57.075+0000 7fcc6dc43640 1 -- 172.21.15.1:0/959925067 shutdown_connections 2024-09-11T22:07:57.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:57.075+0000 7fcc6dc43640 1 --2- 172.21.15.1:0/959925067 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc68106f10 0x7fcc6810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:57.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:57.075+0000 7fcc6dc43640 1 --2- 172.21.15.1:0/959925067 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc68102790 0x7fcc681067e0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:57.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:57.075+0000 7fcc6dc43640 1 --2- 172.21.15.1:0/959925067 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc68101de0 0x7fcc681021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:57.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:57.075+0000 7fcc6dc43640 1 -- 172.21.15.1:0/959925067 >> 172.21.15.1:0/959925067 conn(0x7fcc680fd8c0 msgr2=0x7fcc680ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:07:57.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:57.075+0000 7fcc6dc43640 1 -- 172.21.15.1:0/959925067 shutdown_connections 2024-09-11T22:07:57.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:57.076+0000 7fcc6dc43640 1 -- 172.21.15.1:0/959925067 wait complete. 2024-09-11T22:07:57.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:57.076+0000 7fcc6dc43640 1 Processor -- start 2024-09-11T22:07:57.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:57.076+0000 7fcc6dc43640 1 -- start start 2024-09-11T22:07:57.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:57.076+0000 7fcc6dc43640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc68101de0 0x7fcc68198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:57.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:57.076+0000 7fcc6dc43640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc68102790 0x7fcc681989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:57.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:57.076+0000 7fcc6dc43640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc68106f10 0x7fcc6819cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:57.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:57.076+0000 7fcc6dc43640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcc68111f70 con 0x7fcc68106f10 2024-09-11T22:07:57.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:57.076+0000 7fcc6dc43640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fcc68111df0 con 0x7fcc68101de0 2024-09-11T22:07:57.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:57.076+0000 7fcc6dc43640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fcc681120f0 con 0x7fcc68102790 2024-09-11T22:07:57.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:57.077+0000 7fcc67fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc68106f10 0x7fcc6819cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:57.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:57.077+0000 7fcc677fe640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc68101de0 0x7fcc68198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:57.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:57.077+0000 7fcc67fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc68106f10 0x7fcc6819cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43388/0 (socket says 172.21.15.1:43388) 2024-09-11T22:07:57.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:57.077+0000 7fcc67fff640 1 -- 172.21.15.1:0/4198651266 learned_addr learned my addr 172.21.15.1:0/4198651266 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:07:57.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:57.077+0000 7fcc66ffd640 1 --2- 172.21.15.1:0/4198651266 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc68102790 0x7fcc681989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:57.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:57.077+0000 7fcc67fff640 1 -- 172.21.15.1:0/4198651266 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc68102790 msgr2=0x7fcc681989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:57.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:57.077+0000 7fcc67fff640 1 --2- 172.21.15.1:0/4198651266 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc68102790 0x7fcc681989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:57.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:57.077+0000 7fcc67fff640 1 -- 172.21.15.1:0/4198651266 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc68101de0 msgr2=0x7fcc68198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:57.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:57.077+0000 7fcc67fff640 1 --2- 172.21.15.1:0/4198651266 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc68101de0 0x7fcc68198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:57.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:57.077+0000 7fcc67fff640 1 -- 172.21.15.1:0/4198651266 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc6819d440 con 0x7fcc68106f10 2024-09-11T22:07:57.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:57.077+0000 7fcc67fff640 1 --2- 172.21.15.1:0/4198651266 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc68106f10 0x7fcc6819cd60 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fcc6000cad0 tx=0x7fcc6000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:07:57.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:57.077+0000 7fcc64ff9640 1 -- 172.21.15.1:0/4198651266 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6000ae20 con 0x7fcc68106f10 2024-09-11T22:07:57.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:57.077+0000 7fcc64ff9640 1 -- 172.21.15.1:0/4198651266 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc60005020 con 0x7fcc68106f10 2024-09-11T22:07:57.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:57.077+0000 7fcc6dc43640 1 -- 172.21.15.1:0/4198651266 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc6819d730 con 0x7fcc68106f10 2024-09-11T22:07:57.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:57.077+0000 7fcc64ff9640 1 -- 172.21.15.1:0/4198651266 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6000f6f0 con 0x7fcc68106f10 2024-09-11T22:07:57.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:57.077+0000 7fcc6dc43640 1 -- 172.21.15.1:0/4198651266 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc681a5010 con 0x7fcc68106f10 2024-09-11T22:07:57.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:57.079+0000 7fcc6dc43640 1 -- 172.21.15.1:0/4198651266 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc681021c0 con 0x7fcc68106f10 2024-09-11T22:07:57.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:57.080+0000 7fcc64ff9640 1 -- 172.21.15.1:0/4198651266 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7fcc600051d0 con 0x7fcc68106f10 2024-09-11T22:07:57.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:57.080+0000 7fcc64ff9640 1 --2- 172.21.15.1:0/4198651266 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fcc500a2d70 0x7fcc500a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:57.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:57.080+0000 7fcc64ff9640 1 -- 172.21.15.1:0/4198651266 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8585+0+0 (secure 0 0 0) 0x7fcc600cf360 con 0x7fcc68106f10 2024-09-11T22:07:57.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:57.080+0000 7fcc677fe640 1 --2- 172.21.15.1:0/4198651266 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fcc500a2d70 0x7fcc500a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:57.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:57.081+0000 7fcc677fe640 1 --2- 172.21.15.1:0/4198651266 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fcc500a2d70 0x7fcc500a5230 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fcc58005fd0 tx=0x7fcc5800b390 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:07:57.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:57.084+0000 7fcc64ff9640 1 -- 172.21.15.1:0/4198651266 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcc60014030 con 0x7fcc68106f10 2024-09-11T22:07:57.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:57.296+0000 7fcc6dc43640 1 -- 172.21.15.1:0/4198651266 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fcc6810a150 con 0x7fcc68106f10 2024-09-11T22:07:57.297 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:57 smithi001 ceph-mon[30935]: pgmap v332: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.2 KiB/s wr, 16 op/s 2024-09-11T22:07:57.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:57.298+0000 7fcc64ff9640 1 -- 172.21.15.1:0/4198651266 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v57) ==== 161+0+0 (secure 0 0 0) 0x7fcc6009b8f0 con 0x7fcc68106f10 2024-09-11T22:07:57.299 INFO:teuthology.orchestra.run.smithi001.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-11T22:07:57.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:57.302+0000 7fcc6dc43640 1 -- 172.21.15.1:0/4198651266 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fcc500a2d70 msgr2=0x7fcc500a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:57.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:57.302+0000 7fcc6dc43640 1 --2- 172.21.15.1:0/4198651266 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fcc500a2d70 0x7fcc500a5230 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fcc58005fd0 tx=0x7fcc5800b390 comp rx=0 tx=0).stop 2024-09-11T22:07:57.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:57.302+0000 7fcc6dc43640 1 -- 172.21.15.1:0/4198651266 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc68106f10 msgr2=0x7fcc6819cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:57.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:57.302+0000 7fcc6dc43640 1 --2- 172.21.15.1:0/4198651266 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc68106f10 0x7fcc6819cd60 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fcc6000cad0 tx=0x7fcc6000cf90 comp rx=0 tx=0).stop 2024-09-11T22:07:57.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:57.302+0000 7fcc6dc43640 1 -- 172.21.15.1:0/4198651266 shutdown_connections 2024-09-11T22:07:57.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:57.302+0000 7fcc6dc43640 1 --2- 172.21.15.1:0/4198651266 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fcc500a2d70 0x7fcc500a5230 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:57.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:57.302+0000 7fcc6dc43640 1 --2- 172.21.15.1:0/4198651266 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc68106f10 0x7fcc6819cd60 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:57.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:57.302+0000 7fcc6dc43640 1 --2- 172.21.15.1:0/4198651266 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcc68102790 0x7fcc681989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:57.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:57.302+0000 7fcc6dc43640 1 --2- 172.21.15.1:0/4198651266 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcc68101de0 0x7fcc68198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:57.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:57.302+0000 7fcc6dc43640 1 -- 172.21.15.1:0/4198651266 >> 172.21.15.1:0/4198651266 conn(0x7fcc680fd8c0 msgr2=0x7fcc680fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:07:57.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:57.302+0000 7fcc6dc43640 1 -- 172.21.15.1:0/4198651266 shutdown_connections 2024-09-11T22:07:57.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:57.303+0000 7fcc6dc43640 1 -- 172.21.15.1:0/4198651266 wait complete. 2024-09-11T22:07:57.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:57 smithi053 ceph-mon[34908]: pgmap v332: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.2 KiB/s wr, 16 op/s 2024-09-11T22:07:57.548 INFO:journalctl@ceph.mds.e.smithi053.stdout:Sep 11 22:07:57 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-e[63763]: ignoring --setuser ceph since I am not root 2024-09-11T22:07:57.548 INFO:journalctl@ceph.mds.e.smithi053.stdout:Sep 11 22:07:57 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-e[63763]: ignoring --setgroup ceph since I am not root 2024-09-11T22:07:57.548 INFO:journalctl@ceph.mds.e.smithi053.stdout:Sep 11 22:07:57 smithi053 ceph-mds[63767]: -- 172.21.15.53:0/3902978177 <== mon.0 v2:172.21.15.1:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55588fbeb860 con 0x55588fbc9000 2024-09-11T22:07:57.548 INFO:journalctl@ceph.mds.e.smithi053.stdout:Sep 11 22:07:57 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-e[63763]: starting mds.e at 2024-09-11T22:07:57.549 INFO:journalctl@ceph.mds.b.smithi053.stdout:Sep 11 22:07:57 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-b[64373]: ignoring --setuser ceph since I am not root 2024-09-11T22:07:57.549 INFO:journalctl@ceph.mds.b.smithi053.stdout:Sep 11 22:07:57 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-b[64373]: ignoring --setgroup ceph since I am not root 2024-09-11T22:07:57.549 INFO:journalctl@ceph.mds.b.smithi053.stdout:Sep 11 22:07:57 smithi053 ceph-mds[64377]: -- 172.21.15.53:0/376855204 <== mon.2 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5640d62f5860 con 0x5640d62d2800 2024-09-11T22:07:57.549 INFO:journalctl@ceph.mds.b.smithi053.stdout:Sep 11 22:07:57 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-b[64373]: starting mds.b at 2024-09-11T22:07:57.550 INFO:journalctl@ceph.mds.h.smithi053.stdout:Sep 11 22:07:57 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-h[64063]: ignoring --setuser ceph since I am not root 2024-09-11T22:07:57.550 INFO:journalctl@ceph.mds.h.smithi053.stdout:Sep 11 22:07:57 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-h[64063]: ignoring --setgroup ceph since I am not root 2024-09-11T22:07:57.550 INFO:journalctl@ceph.mds.h.smithi053.stdout:Sep 11 22:07:57 smithi053 ceph-mds[64067]: -- 172.21.15.53:0/2449387759 <== mon.0 v2:172.21.15.1:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e57beb7860 con 0x55e57be95400 2024-09-11T22:07:57.550 INFO:journalctl@ceph.mds.h.smithi053.stdout:Sep 11 22:07:57 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-h[64063]: starting mds.h at 2024-09-11T22:07:57.550 INFO:journalctl@ceph.mds.k.smithi053.stdout:Sep 11 22:07:57 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-k[63453]: ignoring --setuser ceph since I am not root 2024-09-11T22:07:57.551 INFO:journalctl@ceph.mds.k.smithi053.stdout:Sep 11 22:07:57 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-k[63453]: ignoring --setgroup ceph since I am not root 2024-09-11T22:07:57.551 INFO:journalctl@ceph.mds.k.smithi053.stdout:Sep 11 22:07:57 smithi053 ceph-mds[63457]: -- 172.21.15.53:0/3204431578 <== mon.2 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560bbc183860 con 0x560bbc160800 2024-09-11T22:07:57.551 INFO:journalctl@ceph.mds.k.smithi053.stdout:Sep 11 22:07:57 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-k[63453]: starting mds.k at 2024-09-11T22:07:57.592 INFO:journalctl@ceph.mds.i.smithi161.stdout:Sep 11 22:07:57 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-i[65965]: ignoring --setuser ceph since I am not root 2024-09-11T22:07:57.592 INFO:journalctl@ceph.mds.i.smithi161.stdout:Sep 11 22:07:57 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-i[65965]: ignoring --setgroup ceph since I am not root 2024-09-11T22:07:57.592 INFO:journalctl@ceph.mds.i.smithi161.stdout:Sep 11 22:07:57 smithi161 ceph-mds[65969]: -- 172.21.15.161:0/3981677402 <== mon.0 v2:172.21.15.1:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d1e55e3860 con 0x55d1e55c0800 2024-09-11T22:07:57.592 INFO:journalctl@ceph.mds.i.smithi161.stdout:Sep 11 22:07:57 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-i[65965]: starting mds.i at 2024-09-11T22:07:57.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:57 smithi161 ceph-mon[35615]: pgmap v332: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.2 KiB/s wr, 16 op/s 2024-09-11T22:07:57.593 INFO:journalctl@ceph.mds.c.smithi161.stdout:Sep 11 22:07:57 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-c[65663]: ignoring --setuser ceph since I am not root 2024-09-11T22:07:57.593 INFO:journalctl@ceph.mds.c.smithi161.stdout:Sep 11 22:07:57 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-c[65663]: ignoring --setgroup ceph since I am not root 2024-09-11T22:07:57.593 INFO:journalctl@ceph.mds.c.smithi161.stdout:Sep 11 22:07:57 smithi161 ceph-mds[65667]: -- 172.21.15.161:0/3723732692 <== mon.1 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55662aaf3860 con 0x55662aad0800 2024-09-11T22:07:57.594 INFO:journalctl@ceph.mds.c.smithi161.stdout:Sep 11 22:07:57 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-c[65663]: starting mds.c at 2024-09-11T22:07:57.594 INFO:journalctl@ceph.mds.f.smithi161.stdout:Sep 11 22:07:57 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-f[65361]: ignoring --setuser ceph since I am not root 2024-09-11T22:07:57.595 INFO:journalctl@ceph.mds.f.smithi161.stdout:Sep 11 22:07:57 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-f[65361]: ignoring --setgroup ceph since I am not root 2024-09-11T22:07:57.595 INFO:journalctl@ceph.mds.f.smithi161.stdout:Sep 11 22:07:57 smithi161 ceph-mds[65365]: -- 172.21.15.161:0/3984711125 <== mon.1 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5650690af860 con 0x56506908d000 2024-09-11T22:07:57.595 INFO:journalctl@ceph.mds.f.smithi161.stdout:Sep 11 22:07:57 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-f[65361]: starting mds.f at 2024-09-11T22:07:57.595 INFO:journalctl@ceph.mds.l.smithi161.stdout:Sep 11 22:07:57 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-l[65060]: ignoring --setuser ceph since I am not root 2024-09-11T22:07:57.595 INFO:journalctl@ceph.mds.l.smithi161.stdout:Sep 11 22:07:57 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-l[65060]: ignoring --setgroup ceph since I am not root 2024-09-11T22:07:57.596 INFO:journalctl@ceph.mds.l.smithi161.stdout:Sep 11 22:07:57 smithi161 ceph-mds[65064]: -- 172.21.15.161:0/2941672502 <== mon.1 v2:172.21.15.161:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a750143860 con 0x55a750120800 2024-09-11T22:07:57.596 INFO:journalctl@ceph.mds.l.smithi161.stdout:Sep 11 22:07:57 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-l[65060]: starting mds.l at 2024-09-11T22:07:57.689 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:57 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mon-a[30931]: 2024-09-11T22:07:57.296+0000 7f2348aa7640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-11T22:07:57.690 INFO:journalctl@ceph.mds.a.smithi001.stdout:Sep 11 22:07:57 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-a[69747]: ignoring --setuser ceph since I am not root 2024-09-11T22:07:57.690 INFO:journalctl@ceph.mds.a.smithi001.stdout:Sep 11 22:07:57 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-a[69747]: ignoring --setgroup ceph since I am not root 2024-09-11T22:07:57.690 INFO:journalctl@ceph.mds.a.smithi001.stdout:Sep 11 22:07:57 smithi001 ceph-mds[69763]: -- 172.21.15.1:0/27531871 <== mon.0 v2:172.21.15.1:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55871163bc20 con 0x558711619400 2024-09-11T22:07:57.690 INFO:journalctl@ceph.mds.a.smithi001.stdout:Sep 11 22:07:57 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-a[69747]: starting mds.a at 2024-09-11T22:07:57.690 INFO:journalctl@ceph.mds.g.smithi001.stdout:Sep 11 22:07:57 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-g[69388]: ignoring --setuser ceph since I am not root 2024-09-11T22:07:57.691 INFO:journalctl@ceph.mds.g.smithi001.stdout:Sep 11 22:07:57 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-g[69388]: ignoring --setgroup ceph since I am not root 2024-09-11T22:07:57.691 INFO:journalctl@ceph.mds.g.smithi001.stdout:Sep 11 22:07:57 smithi001 ceph-mds[69413]: -- 172.21.15.1:0/3258082240 <== mon.0 v2:172.21.15.1:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5576c27f9860 con 0x5576c27d7000 2024-09-11T22:07:57.691 INFO:journalctl@ceph.mds.g.smithi001.stdout:Sep 11 22:07:57 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-g[69388]: starting mds.g at 2024-09-11T22:07:57.691 INFO:journalctl@ceph.mds.g.smithi001.stdout:Sep 11 22:07:57 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-g[69388]: 2024-09-11T22:07:57.480+0000 7ff0ae8f0640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6838/0: (98) Address already in use 2024-09-11T22:07:58.037 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-11T22:07:58.301 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:07:58.425 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:58 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/4198651266' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-11T22:07:58.426 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:58 smithi001 ceph-mon[30935]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-11T22:07:58.426 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:58 smithi001 ceph-mon[30935]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-11T22:07:58.426 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:58 smithi001 ceph-mon[30935]: Health check cleared: MDS_READ_ONLY (was: 1 MDSs are read only) 2024-09-11T22:07:58.426 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:58 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/4198651266' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-11T22:07:58.426 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:58 smithi001 ceph-mon[30935]: osdmap e96: 12 total, 12 up, 12 in 2024-09-11T22:07:58.426 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:58 smithi001 ceph-mon[30935]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-11T22:07:58.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:58 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/4198651266' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-11T22:07:58.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:58 smithi053 ceph-mon[34908]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-11T22:07:58.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:58 smithi053 ceph-mon[34908]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-11T22:07:58.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:58 smithi053 ceph-mon[34908]: Health check cleared: MDS_READ_ONLY (was: 1 MDSs are read only) 2024-09-11T22:07:58.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:58 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/4198651266' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-11T22:07:58.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:58 smithi053 ceph-mon[34908]: osdmap e96: 12 total, 12 up, 12 in 2024-09-11T22:07:58.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:58 smithi053 ceph-mon[34908]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-11T22:07:58.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:58 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/4198651266' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-11T22:07:58.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:58 smithi161 ceph-mon[35615]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-11T22:07:58.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:58 smithi161 ceph-mon[35615]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-11T22:07:58.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:58 smithi161 ceph-mon[35615]: Health check cleared: MDS_READ_ONLY (was: 1 MDSs are read only) 2024-09-11T22:07:58.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:58 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/4198651266' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-11T22:07:58.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:58 smithi161 ceph-mon[35615]: osdmap e96: 12 total, 12 up, 12 in 2024-09-11T22:07:58.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:58 smithi161 ceph-mon[35615]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-11T22:07:58.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:58.785+0000 7f1bb4fb8640 1 -- 172.21.15.1:0/4244137418 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1bb0102d40 msgr2=0x7f1bb0103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:58.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:58.785+0000 7f1bb4fb8640 1 --2- 172.21.15.1:0/4244137418 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1bb0102d40 0x7f1bb0103120 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f1ba0009900 tx=0x7f1ba002a480 comp rx=0 tx=0).stop 2024-09-11T22:07:58.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:58.785+0000 7f1bb4fb8640 1 -- 172.21.15.1:0/4244137418 shutdown_connections 2024-09-11T22:07:58.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:58.785+0000 7f1bb4fb8640 1 --2- 172.21.15.1:0/4244137418 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1bb010c170 0x7f1bb010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:58.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:58.785+0000 7f1bb4fb8640 1 --2- 172.21.15.1:0/4244137418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1bb0103660 0x7f1bb010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:58.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:58.785+0000 7f1bb4fb8640 1 --2- 172.21.15.1:0/4244137418 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1bb0102d40 0x7f1bb0103120 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:58.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:58.785+0000 7f1bb4fb8640 1 -- 172.21.15.1:0/4244137418 >> 172.21.15.1:0/4244137418 conn(0x7f1bb00fc7a0 msgr2=0x7f1bb00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:07:58.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:58.786+0000 7f1bb4fb8640 1 -- 172.21.15.1:0/4244137418 shutdown_connections 2024-09-11T22:07:58.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:58.786+0000 7f1bb4fb8640 1 -- 172.21.15.1:0/4244137418 wait complete. 2024-09-11T22:07:58.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:58.786+0000 7f1bb4fb8640 1 Processor -- start 2024-09-11T22:07:58.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:58.786+0000 7f1bb4fb8640 1 -- start start 2024-09-11T22:07:58.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:58.786+0000 7f1bb4fb8640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1bb0102d40 0x7f1bb0198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:58.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:58.786+0000 7f1bb4fb8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1bb0103660 0x7f1bb0198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:58.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:58.786+0000 7f1bb4fb8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1bb010c170 0x7f1bb019cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:58.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:58.787+0000 7f1bb4fb8640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb0111fb0 con 0x7f1bb010c170 2024-09-11T22:07:58.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:58.787+0000 7f1bb4fb8640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb0111e30 con 0x7f1bb0102d40 2024-09-11T22:07:58.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:58.787+0000 7f1bb4fb8640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb0112130 con 0x7f1bb0103660 2024-09-11T22:07:58.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:58.787+0000 7f1baed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1bb010c170 0x7f1bb019cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:58.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:58.787+0000 7f1baed9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1bb010c170 0x7f1bb019cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43464/0 (socket says 172.21.15.1:43464) 2024-09-11T22:07:58.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:58.787+0000 7f1baed9d640 1 -- 172.21.15.1:0/216085279 learned_addr learned my addr 172.21.15.1:0/216085279 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:07:58.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:58.787+0000 7f1badd9b640 1 --2- 172.21.15.1:0/216085279 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1bb0103660 0x7f1bb0198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:58.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:58.787+0000 7f1bae59c640 1 --2- 172.21.15.1:0/216085279 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1bb0102d40 0x7f1bb0198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:58.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:58.787+0000 7f1baed9d640 1 -- 172.21.15.1:0/216085279 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1bb0103660 msgr2=0x7f1bb0198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:58.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:58.787+0000 7f1baed9d640 1 --2- 172.21.15.1:0/216085279 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1bb0103660 0x7f1bb0198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:58.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:58.787+0000 7f1baed9d640 1 -- 172.21.15.1:0/216085279 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1bb0102d40 msgr2=0x7f1bb0198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:58.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:58.787+0000 7f1baed9d640 1 --2- 172.21.15.1:0/216085279 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1bb0102d40 0x7f1bb0198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:58.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:58.787+0000 7f1baed9d640 1 -- 172.21.15.1:0/216085279 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bb019d3e0 con 0x7f1bb010c170 2024-09-11T22:07:58.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:58.787+0000 7f1bae59c640 1 --2- 172.21.15.1:0/216085279 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1bb0102d40 0x7f1bb0198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:07:58.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:58.787+0000 7f1baed9d640 1 --2- 172.21.15.1:0/216085279 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1bb010c170 0x7f1bb019cd00 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f1ba800cce0 tx=0x7f1ba800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:07:58.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:58.787+0000 7f1ba77fe640 1 -- 172.21.15.1:0/216085279 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba800f450 con 0x7f1bb010c170 2024-09-11T22:07:58.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:58.787+0000 7f1ba77fe640 1 -- 172.21.15.1:0/216085279 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ba800fad0 con 0x7f1bb010c170 2024-09-11T22:07:58.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:58.787+0000 7f1bb4fb8640 1 -- 172.21.15.1:0/216085279 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bb019d6d0 con 0x7f1bb010c170 2024-09-11T22:07:58.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:58.787+0000 7f1ba77fe640 1 -- 172.21.15.1:0/216085279 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba8018c10 con 0x7f1bb010c170 2024-09-11T22:07:58.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:58.787+0000 7f1bb4fb8640 1 -- 172.21.15.1:0/216085279 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bb01a4fb0 con 0x7f1bb010c170 2024-09-11T22:07:58.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:58.789+0000 7f1bb4fb8640 1 -- 172.21.15.1:0/216085279 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bb0103120 con 0x7f1bb010c170 2024-09-11T22:07:58.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:58.790+0000 7f1ba77fe640 1 -- 172.21.15.1:0/216085279 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7f1ba800f5f0 con 0x7f1bb010c170 2024-09-11T22:07:58.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:58.790+0000 7f1ba77fe640 1 --2- 172.21.15.1:0/216085279 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f1b9c0a2d70 0x7f1b9c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:07:58.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:58.790+0000 7f1bae59c640 1 --2- 172.21.15.1:0/216085279 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f1b9c0a2d70 0x7f1b9c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:07:58.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:58.791+0000 7f1ba77fe640 1 -- 172.21.15.1:0/216085279 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9015+0+0 (secure 0 0 0) 0x7f1ba80cfd90 con 0x7f1bb010c170 2024-09-11T22:07:58.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:58.791+0000 7f1bae59c640 1 --2- 172.21.15.1:0/216085279 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f1b9c0a2d70 0x7f1b9c0a5230 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1ba0000c00 tx=0x7f1ba0032d50 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:07:58.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:58.795+0000 7f1ba77fe640 1 -- 172.21.15.1:0/216085279 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1ba8014030 con 0x7f1bb010c170 2024-09-11T22:07:59.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:59.004+0000 7f1bb4fb8640 1 -- 172.21.15.1:0/216085279 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f1bb002d7c0 con 0x7f1bb010c170 2024-09-11T22:07:59.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:59.305+0000 7f1ba77fe640 1 -- 172.21.15.1:0/216085279 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v59) ==== 104+0+0 (secure 0 0 0) 0x7f1ba809c760 con 0x7f1bb010c170 2024-09-11T22:07:59.306 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:59 smithi001 ceph-mon[30935]: pgmap v334: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-09-11T22:07:59.306 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:59 smithi001 ceph-mon[30935]: mds.? [v2:172.21.15.161:6832/2860232815,v1:172.21.15.161:6833/2860232815] up:boot 2024-09-11T22:07:59.306 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:59 smithi001 ceph-mon[30935]: mds.? [v2:172.21.15.161:6835/3139612192,v1:172.21.15.161:6838/3139612192] up:boot 2024-09-11T22:07:59.306 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:59 smithi001 ceph-mon[30935]: mds.? [v2:172.21.15.161:6834/2512663809,v1:172.21.15.161:6836/2512663809] up:boot 2024-09-11T22:07:59.307 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:59 smithi001 ceph-mon[30935]: mds.? [v2:172.21.15.161:6837/759103681,v1:172.21.15.161:6839/759103681] up:boot 2024-09-11T22:07:59.307 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:59 smithi001 ceph-mon[30935]: mds.? [v2:172.21.15.53:6833/1043001809,v1:172.21.15.53:6835/1043001809] up:boot 2024-09-11T22:07:59.307 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:59 smithi001 ceph-mon[30935]: mds.? [v2:172.21.15.53:6832/83423464,v1:172.21.15.53:6834/83423464] up:boot 2024-09-11T22:07:59.307 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:59 smithi001 ceph-mon[30935]: mds.? [v2:172.21.15.53:6836/3046498668,v1:172.21.15.53:6838/3046498668] up:boot 2024-09-11T22:07:59.307 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:59 smithi001 ceph-mon[30935]: mds.? [v2:172.21.15.53:6837/1315438779,v1:172.21.15.53:6839/1315438779] up:boot 2024-09-11T22:07:59.307 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:59 smithi001 ceph-mon[30935]: mds.? [v2:172.21.15.1:6838/981698941,v1:172.21.15.1:6840/981698941] up:boot 2024-09-11T22:07:59.307 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:59 smithi001 ceph-mon[30935]: mds.? [v2:172.21.15.1:6839/2441822821,v1:172.21.15.1:6841/2441822821] up:boot 2024-09-11T22:07:59.307 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:59 smithi001 ceph-mon[30935]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-11T22:07:59.308 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:59 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-11T22:07:59.308 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:59 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-11T22:07:59.308 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:59 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-11T22:07:59.308 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:59 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-11T22:07:59.308 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:59 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-11T22:07:59.308 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:59 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T22:07:59.308 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:59 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-11T22:07:59.308 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:59 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-11T22:07:59.309 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:59 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T22:07:59.309 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:59 smithi001 ceph-mon[30935]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-11T22:07:59.309 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:07:59 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/216085279' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T22:07:59.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:59.307+0000 7f1bb4fb8640 1 -- 172.21.15.1:0/216085279 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f1b9c0a2d70 msgr2=0x7f1b9c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:59.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:59.307+0000 7f1bb4fb8640 1 --2- 172.21.15.1:0/216085279 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f1b9c0a2d70 0x7f1b9c0a5230 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1ba0000c00 tx=0x7f1ba0032d50 comp rx=0 tx=0).stop 2024-09-11T22:07:59.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:59.307+0000 7f1bb4fb8640 1 -- 172.21.15.1:0/216085279 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1bb010c170 msgr2=0x7f1bb019cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:07:59.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:59.307+0000 7f1bb4fb8640 1 --2- 172.21.15.1:0/216085279 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1bb010c170 0x7f1bb019cd00 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f1ba800cce0 tx=0x7f1ba800a620 comp rx=0 tx=0).stop 2024-09-11T22:07:59.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:59.307+0000 7f1bb4fb8640 1 -- 172.21.15.1:0/216085279 shutdown_connections 2024-09-11T22:07:59.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:59.307+0000 7f1bb4fb8640 1 --2- 172.21.15.1:0/216085279 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7f1b9c0a2d70 0x7f1b9c0a5230 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:59.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:59.307+0000 7f1bb4fb8640 1 --2- 172.21.15.1:0/216085279 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1bb010c170 0x7f1bb019cd00 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:59.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:59.307+0000 7f1bb4fb8640 1 --2- 172.21.15.1:0/216085279 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1bb0103660 0x7f1bb0198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:59.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:59.307+0000 7f1bb4fb8640 1 --2- 172.21.15.1:0/216085279 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1bb0102d40 0x7f1bb0198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:07:59.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:59.307+0000 7f1bb4fb8640 1 -- 172.21.15.1:0/216085279 >> 172.21.15.1:0/216085279 conn(0x7f1bb00fc7a0 msgr2=0x7f1bb00fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:07:59.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:59.307+0000 7f1bb4fb8640 1 -- 172.21.15.1:0/216085279 shutdown_connections 2024-09-11T22:07:59.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:07:59.307+0000 7f1bb4fb8640 1 -- 172.21.15.1:0/216085279 wait complete. 2024-09-11T22:07:59.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:59 smithi053 ceph-mon[34908]: pgmap v334: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-09-11T22:07:59.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:59 smithi053 ceph-mon[34908]: mds.? [v2:172.21.15.161:6832/2860232815,v1:172.21.15.161:6833/2860232815] up:boot 2024-09-11T22:07:59.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:59 smithi053 ceph-mon[34908]: mds.? [v2:172.21.15.161:6835/3139612192,v1:172.21.15.161:6838/3139612192] up:boot 2024-09-11T22:07:59.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:59 smithi053 ceph-mon[34908]: mds.? [v2:172.21.15.161:6834/2512663809,v1:172.21.15.161:6836/2512663809] up:boot 2024-09-11T22:07:59.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:59 smithi053 ceph-mon[34908]: mds.? [v2:172.21.15.161:6837/759103681,v1:172.21.15.161:6839/759103681] up:boot 2024-09-11T22:07:59.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:59 smithi053 ceph-mon[34908]: mds.? [v2:172.21.15.53:6833/1043001809,v1:172.21.15.53:6835/1043001809] up:boot 2024-09-11T22:07:59.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:59 smithi053 ceph-mon[34908]: mds.? [v2:172.21.15.53:6832/83423464,v1:172.21.15.53:6834/83423464] up:boot 2024-09-11T22:07:59.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:59 smithi053 ceph-mon[34908]: mds.? [v2:172.21.15.53:6836/3046498668,v1:172.21.15.53:6838/3046498668] up:boot 2024-09-11T22:07:59.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:59 smithi053 ceph-mon[34908]: mds.? [v2:172.21.15.53:6837/1315438779,v1:172.21.15.53:6839/1315438779] up:boot 2024-09-11T22:07:59.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:59 smithi053 ceph-mon[34908]: mds.? [v2:172.21.15.1:6838/981698941,v1:172.21.15.1:6840/981698941] up:boot 2024-09-11T22:07:59.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:59 smithi053 ceph-mon[34908]: mds.? [v2:172.21.15.1:6839/2441822821,v1:172.21.15.1:6841/2441822821] up:boot 2024-09-11T22:07:59.549 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:59 smithi053 ceph-mon[34908]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-11T22:07:59.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:59 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-11T22:07:59.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:59 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-11T22:07:59.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:59 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-11T22:07:59.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:59 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-11T22:07:59.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:59 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-11T22:07:59.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:59 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T22:07:59.550 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:59 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-11T22:07:59.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:59 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-11T22:07:59.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:59 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T22:07:59.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:59 smithi053 ceph-mon[34908]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-11T22:07:59.551 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:07:59 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/216085279' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T22:07:59.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:59 smithi161 ceph-mon[35615]: pgmap v334: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-09-11T22:07:59.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:59 smithi161 ceph-mon[35615]: mds.? [v2:172.21.15.161:6832/2860232815,v1:172.21.15.161:6833/2860232815] up:boot 2024-09-11T22:07:59.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:59 smithi161 ceph-mon[35615]: mds.? [v2:172.21.15.161:6835/3139612192,v1:172.21.15.161:6838/3139612192] up:boot 2024-09-11T22:07:59.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:59 smithi161 ceph-mon[35615]: mds.? [v2:172.21.15.161:6834/2512663809,v1:172.21.15.161:6836/2512663809] up:boot 2024-09-11T22:07:59.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:59 smithi161 ceph-mon[35615]: mds.? [v2:172.21.15.161:6837/759103681,v1:172.21.15.161:6839/759103681] up:boot 2024-09-11T22:07:59.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:59 smithi161 ceph-mon[35615]: mds.? [v2:172.21.15.53:6833/1043001809,v1:172.21.15.53:6835/1043001809] up:boot 2024-09-11T22:07:59.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:59 smithi161 ceph-mon[35615]: mds.? [v2:172.21.15.53:6832/83423464,v1:172.21.15.53:6834/83423464] up:boot 2024-09-11T22:07:59.593 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:59 smithi161 ceph-mon[35615]: mds.? [v2:172.21.15.53:6836/3046498668,v1:172.21.15.53:6838/3046498668] up:boot 2024-09-11T22:07:59.594 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:59 smithi161 ceph-mon[35615]: mds.? [v2:172.21.15.53:6837/1315438779,v1:172.21.15.53:6839/1315438779] up:boot 2024-09-11T22:07:59.594 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:59 smithi161 ceph-mon[35615]: mds.? [v2:172.21.15.1:6838/981698941,v1:172.21.15.1:6840/981698941] up:boot 2024-09-11T22:07:59.594 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:59 smithi161 ceph-mon[35615]: mds.? [v2:172.21.15.1:6839/2441822821,v1:172.21.15.1:6841/2441822821] up:boot 2024-09-11T22:07:59.594 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:59 smithi161 ceph-mon[35615]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-11T22:07:59.595 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:59 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-11T22:07:59.595 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:59 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-11T22:07:59.595 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:59 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-11T22:07:59.595 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:59 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-11T22:07:59.595 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:59 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-11T22:07:59.595 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:59 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T22:07:59.596 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:59 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-11T22:07:59.596 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:59 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-11T22:07:59.596 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:59 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T22:07:59.596 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:59 smithi161 ceph-mon[35615]: from='mgr.15204 172.21.15.1:0/263731203' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-11T22:07:59.597 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:07:59 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/216085279' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T22:08:00.064 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-11T22:08:00.329 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:08:00.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:08:00 smithi001 ceph-mon[30935]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-11T22:08:00.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:08:00 smithi001 ceph-mon[30935]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-11T22:08:00.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:08:00 smithi001 ceph-mon[30935]: Cluster is now healthy 2024-09-11T22:08:00.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:08:00 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/216085279' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-11T22:08:00.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:08:00 smithi001 ceph-mon[30935]: fsmap 12 up:standby 2024-09-11T22:08:00.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:08:00 smithi053 ceph-mon[34908]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-11T22:08:00.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:08:00 smithi053 ceph-mon[34908]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-11T22:08:00.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:08:00 smithi053 ceph-mon[34908]: Cluster is now healthy 2024-09-11T22:08:00.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:08:00 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/216085279' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-11T22:08:00.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:08:00 smithi053 ceph-mon[34908]: fsmap 12 up:standby 2024-09-11T22:08:00.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:08:00 smithi161 ceph-mon[35615]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-11T22:08:00.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:08:00 smithi161 ceph-mon[35615]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-11T22:08:00.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:08:00 smithi161 ceph-mon[35615]: Cluster is now healthy 2024-09-11T22:08:00.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:08:00 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/216085279' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-11T22:08:00.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:08:00 smithi161 ceph-mon[35615]: fsmap 12 up:standby 2024-09-11T22:08:00.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:00.850+0000 7fd17c885640 1 -- 172.21.15.1:0/1694275266 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd178103760 msgr2=0x7fd178107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:08:00.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:00.850+0000 7fd17c885640 1 --2- 172.21.15.1:0/1694275266 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd178103760 0x7fd178107ba0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fd16c009930 tx=0x7fd16c02a420 comp rx=0 tx=0).stop 2024-09-11T22:08:00.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:00.850+0000 7fd17c885640 1 -- 172.21.15.1:0/1694275266 shutdown_connections 2024-09-11T22:08:00.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:00.850+0000 7fd17c885640 1 --2- 172.21.15.1:0/1694275266 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1781082e0 0x7fd17810ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:08:00.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:00.850+0000 7fd17c885640 1 --2- 172.21.15.1:0/1694275266 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd178103760 0x7fd178107ba0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:08:00.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:00.850+0000 7fd17c885640 1 --2- 172.21.15.1:0/1694275266 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd178102db0 0x7fd178103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:08:00.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:00.850+0000 7fd17c885640 1 -- 172.21.15.1:0/1694275266 >> 172.21.15.1:0/1694275266 conn(0x7fd1780fc7d0 msgr2=0x7fd1780febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:08:00.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:00.850+0000 7fd17c885640 1 -- 172.21.15.1:0/1694275266 shutdown_connections 2024-09-11T22:08:00.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:00.850+0000 7fd17c885640 1 -- 172.21.15.1:0/1694275266 wait complete. 2024-09-11T22:08:00.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:00.851+0000 7fd17c885640 1 Processor -- start 2024-09-11T22:08:00.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:00.851+0000 7fd17c885640 1 -- start start 2024-09-11T22:08:00.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:00.851+0000 7fd17c885640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd178102db0 0x7fd178198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:08:00.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:00.851+0000 7fd17c885640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd178103760 0x7fd178198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:08:00.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:00.851+0000 7fd17c885640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1781082e0 0x7fd17819cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:08:00.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:00.851+0000 7fd17c885640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd178112070 con 0x7fd178103760 2024-09-11T22:08:00.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:00.851+0000 7fd17c885640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fd178111ef0 con 0x7fd1781082e0 2024-09-11T22:08:00.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:00.851+0000 7fd17c885640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fd1781121f0 con 0x7fd178102db0 2024-09-11T22:08:00.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:00.851+0000 7fd175d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd178103760 0x7fd178198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:08:00.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:00.851+0000 7fd175d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd178103760 0x7fd178198b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43480/0 (socket says 172.21.15.1:43480) 2024-09-11T22:08:00.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:00.851+0000 7fd175d9b640 1 -- 172.21.15.1:0/3071194625 learned_addr learned my addr 172.21.15.1:0/3071194625 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:08:00.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:00.851+0000 7fd17659c640 1 --2- 172.21.15.1:0/3071194625 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd178102db0 0x7fd178198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:08:00.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:00.851+0000 7fd176d9d640 1 --2- 172.21.15.1:0/3071194625 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1781082e0 0x7fd17819cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:08:00.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:00.851+0000 7fd175d9b640 1 -- 172.21.15.1:0/3071194625 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd178102db0 msgr2=0x7fd178198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:08:00.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:00.851+0000 7fd175d9b640 1 --2- 172.21.15.1:0/3071194625 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd178102db0 0x7fd178198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:08:00.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:00.851+0000 7fd175d9b640 1 -- 172.21.15.1:0/3071194625 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1781082e0 msgr2=0x7fd17819cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:08:00.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:00.851+0000 7fd175d9b640 1 --2- 172.21.15.1:0/3071194625 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1781082e0 0x7fd17819cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:08:00.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:00.851+0000 7fd175d9b640 1 -- 172.21.15.1:0/3071194625 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd17819d5e0 con 0x7fd178103760 2024-09-11T22:08:00.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:00.851+0000 7fd17659c640 1 --2- 172.21.15.1:0/3071194625 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd178102db0 0x7fd178198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:08:00.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:00.852+0000 7fd175d9b640 1 --2- 172.21.15.1:0/3071194625 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd178103760 0x7fd178198b70 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fd16c005940 tx=0x7fd16c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:08:00.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:00.852+0000 7fd1677fe640 1 -- 172.21.15.1:0/3071194625 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd16c03f070 con 0x7fd178103760 2024-09-11T22:08:00.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:00.852+0000 7fd1677fe640 1 -- 172.21.15.1:0/3071194625 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd16c031c30 con 0x7fd178103760 2024-09-11T22:08:00.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:00.852+0000 7fd17c885640 1 -- 172.21.15.1:0/3071194625 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd17819d870 con 0x7fd178103760 2024-09-11T22:08:00.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:00.852+0000 7fd1677fe640 1 -- 172.21.15.1:0/3071194625 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd16c035030 con 0x7fd178103760 2024-09-11T22:08:00.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:00.852+0000 7fd17c885640 1 -- 172.21.15.1:0/3071194625 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1781a51b0 con 0x7fd178103760 2024-09-11T22:08:00.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:00.853+0000 7fd17c885640 1 -- 172.21.15.1:0/3071194625 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd17806a7d0 con 0x7fd178103760 2024-09-11T22:08:00.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:00.854+0000 7fd1677fe640 1 -- 172.21.15.1:0/3071194625 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7fd16c042070 con 0x7fd178103760 2024-09-11T22:08:00.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:00.855+0000 7fd1677fe640 1 --2- 172.21.15.1:0/3071194625 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fd1600a2cf0 0x7fd1600a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:08:00.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:00.855+0000 7fd1677fe640 1 -- 172.21.15.1:0/3071194625 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9015+0+0 (secure 0 0 0) 0x7fd16c0efec0 con 0x7fd178103760 2024-09-11T22:08:00.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:00.855+0000 7fd17659c640 1 --2- 172.21.15.1:0/3071194625 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fd1600a2cf0 0x7fd1600a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:08:00.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:00.855+0000 7fd17659c640 1 --2- 172.21.15.1:0/3071194625 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fd1600a2cf0 0x7fd1600a51b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd168009c20 tx=0x7fd168005f90 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:08:00.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:00.859+0000 7fd1677fe640 1 -- 172.21.15.1:0/3071194625 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd16c031de0 con 0x7fd178103760 2024-09-11T22:08:01.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:01.034+0000 7fd17c885640 1 -- 172.21.15.1:0/3071194625 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd17810a2c0 con 0x7fd178103760 2024-09-11T22:08:01.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:01.307+0000 7fd1677fe640 1 -- 172.21.15.1:0/3071194625 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v97) ==== 181+0+0 (secure 0 0 0) 0x7fd16c0bc8f0 con 0x7fd178103760 2024-09-11T22:08:01.308 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_metadata' removed 2024-09-11T22:08:01.309 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:08:01 smithi001 ceph-mon[30935]: pgmap v335: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-09-11T22:08:01.309 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:08:01 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/3071194625' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T22:08:01.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:01.310+0000 7fd17c885640 1 -- 172.21.15.1:0/3071194625 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fd1600a2cf0 msgr2=0x7fd1600a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:08:01.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:01.310+0000 7fd17c885640 1 --2- 172.21.15.1:0/3071194625 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fd1600a2cf0 0x7fd1600a51b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd168009c20 tx=0x7fd168005f90 comp rx=0 tx=0).stop 2024-09-11T22:08:01.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:01.311+0000 7fd17c885640 1 -- 172.21.15.1:0/3071194625 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd178103760 msgr2=0x7fd178198b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:08:01.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:01.311+0000 7fd17c885640 1 --2- 172.21.15.1:0/3071194625 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd178103760 0x7fd178198b70 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fd16c005940 tx=0x7fd16c003c10 comp rx=0 tx=0).stop 2024-09-11T22:08:01.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:01.311+0000 7fd17c885640 1 -- 172.21.15.1:0/3071194625 shutdown_connections 2024-09-11T22:08:01.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:01.311+0000 7fd17c885640 1 --2- 172.21.15.1:0/3071194625 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fd1600a2cf0 0x7fd1600a51b0 secure :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd168009c20 tx=0x7fd168005f90 comp rx=0 tx=0).stop 2024-09-11T22:08:01.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:01.311+0000 7fd17c885640 1 --2- 172.21.15.1:0/3071194625 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd1781082e0 0x7fd17819cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:08:01.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:01.311+0000 7fd17c885640 1 --2- 172.21.15.1:0/3071194625 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd178103760 0x7fd178198b70 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:08:01.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:01.311+0000 7fd17c885640 1 --2- 172.21.15.1:0/3071194625 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd178102db0 0x7fd178198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:08:01.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:01.311+0000 7fd17c885640 1 -- 172.21.15.1:0/3071194625 >> 172.21.15.1:0/3071194625 conn(0x7fd1780fc7d0 msgr2=0x7fd178105e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:08:01.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:01.312+0000 7fd17c885640 1 -- 172.21.15.1:0/3071194625 shutdown_connections 2024-09-11T22:08:01.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:01.312+0000 7fd17c885640 1 -- 172.21.15.1:0/3071194625 wait complete. 2024-09-11T22:08:01.547 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:08:01 smithi053 ceph-mon[34908]: pgmap v335: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-09-11T22:08:01.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:08:01 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/3071194625' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T22:08:01.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:08:01 smithi161 ceph-mon[35615]: pgmap v335: 97 pgs: 97 active+clean; 470 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-09-11T22:08:01.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:08:01 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/3071194625' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T22:08:02.310 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-11T22:08:02.585 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:08:02.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:08:02 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/3071194625' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T22:08:02.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:08:02 smithi161 ceph-mon[35615]: osdmap e97: 12 total, 12 up, 12 in 2024-09-11T22:08:02.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:08:02 smithi161 ceph-mon[35615]: pgmap v337: 65 pgs: 65 active+clean; 577 KiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:08:02.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:08:02 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/3071194625' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T22:08:02.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:08:02 smithi001 ceph-mon[30935]: osdmap e97: 12 total, 12 up, 12 in 2024-09-11T22:08:02.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:08:02 smithi001 ceph-mon[30935]: pgmap v337: 65 pgs: 65 active+clean; 577 KiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:08:02.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:08:02 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/3071194625' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T22:08:02.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:08:02 smithi053 ceph-mon[34908]: osdmap e97: 12 total, 12 up, 12 in 2024-09-11T22:08:02.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:08:02 smithi053 ceph-mon[34908]: pgmap v337: 65 pgs: 65 active+clean; 577 KiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:08:03.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:03.067+0000 7fa2372bb640 1 -- 172.21.15.1:0/1110437620 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa230102570 msgr2=0x7fa2301029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:08:03.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:03.067+0000 7fa2372bb640 1 --2- 172.21.15.1:0/1110437620 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa230102570 0x7fa2301029b0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fa220009900 tx=0x7fa22002a480 comp rx=0 tx=0).stop 2024-09-11T22:08:03.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:03.067+0000 7fa2372bb640 1 -- 172.21.15.1:0/1110437620 shutdown_connections 2024-09-11T22:08:03.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:03.067+0000 7fa2372bb640 1 --2- 172.21.15.1:0/1110437620 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa230102ef0 0x7fa23010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:08:03.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:03.067+0000 7fa2372bb640 1 --2- 172.21.15.1:0/1110437620 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa230102570 0x7fa2301029b0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:08:03.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:03.067+0000 7fa2372bb640 1 --2- 172.21.15.1:0/1110437620 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa230100e20 0x7fa230101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:08:03.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:03.067+0000 7fa2372bb640 1 -- 172.21.15.1:0/1110437620 >> 172.21.15.1:0/1110437620 conn(0x7fa2300fc7d0 msgr2=0x7fa2300febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:08:03.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:03.067+0000 7fa2372bb640 1 -- 172.21.15.1:0/1110437620 shutdown_connections 2024-09-11T22:08:03.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:03.067+0000 7fa2372bb640 1 -- 172.21.15.1:0/1110437620 wait complete. 2024-09-11T22:08:03.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:03.068+0000 7fa2372bb640 1 Processor -- start 2024-09-11T22:08:03.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:03.068+0000 7fa2372bb640 1 -- start start 2024-09-11T22:08:03.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:03.068+0000 7fa2372bb640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa230100e20 0x7fa230198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:08:03.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:03.068+0000 7fa2372bb640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa230102570 0x7fa230198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:08:03.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:03.068+0000 7fa2372bb640 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa230102ef0 0x7fa23019cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:08:03.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:03.068+0000 7fa2372bb640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa230110f50 con 0x7fa230100e20 2024-09-11T22:08:03.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:03.068+0000 7fa2372bb640 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 -- 0x7fa230110dd0 con 0x7fa230102ef0 2024-09-11T22:08:03.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:03.068+0000 7fa2372bb640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 -- 0x7fa2301110d0 con 0x7fa230102570 2024-09-11T22:08:03.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:03.068+0000 7fa235057640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa230100e20 0x7fa230198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:08:03.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:03.068+0000 7fa235057640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa230100e20 0x7fa230198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43496/0 (socket says 172.21.15.1:43496) 2024-09-11T22:08:03.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:03.068+0000 7fa235057640 1 -- 172.21.15.1:0/271388742 learned_addr learned my addr 172.21.15.1:0/271388742 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-09-11T22:08:03.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:03.068+0000 7fa235858640 1 --2- 172.21.15.1:0/271388742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa230102ef0 0x7fa23019cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:08:03.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:03.068+0000 7fa234856640 1 --2- 172.21.15.1:0/271388742 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa230102570 0x7fa230198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:08:03.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:03.068+0000 7fa235057640 1 -- 172.21.15.1:0/271388742 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa230102570 msgr2=0x7fa230198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:08:03.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:03.068+0000 7fa235057640 1 --2- 172.21.15.1:0/271388742 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa230102570 0x7fa230198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:08:03.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:03.068+0000 7fa235057640 1 -- 172.21.15.1:0/271388742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa230102ef0 msgr2=0x7fa23019cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:08:03.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:03.068+0000 7fa235057640 1 --2- 172.21.15.1:0/271388742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa230102ef0 0x7fa23019cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:08:03.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:03.069+0000 7fa235057640 1 -- 172.21.15.1:0/271388742 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa23019d3f0 con 0x7fa230100e20 2024-09-11T22:08:03.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:03.069+0000 7fa235057640 1 --2- 172.21.15.1:0/271388742 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa230100e20 0x7fa230198440 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fa22c00c920 tx=0x7fa22c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:08:03.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:03.069+0000 7fa2267fc640 1 -- 172.21.15.1:0/271388742 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa22c007c70 con 0x7fa230100e20 2024-09-11T22:08:03.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:03.069+0000 7fa2372bb640 1 -- 172.21.15.1:0/271388742 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa23019d6e0 con 0x7fa230100e20 2024-09-11T22:08:03.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:03.069+0000 7fa2267fc640 1 -- 172.21.15.1:0/271388742 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa22c007e10 con 0x7fa230100e20 2024-09-11T22:08:03.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:03.069+0000 7fa2372bb640 1 -- 172.21.15.1:0/271388742 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2301a4fc0 con 0x7fa230100e20 2024-09-11T22:08:03.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:03.069+0000 7fa2267fc640 1 -- 172.21.15.1:0/271388742 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa22c018890 con 0x7fa230100e20 2024-09-11T22:08:03.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:03.070+0000 7fa2372bb640 1 -- 172.21.15.1:0/271388742 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa230101200 con 0x7fa230100e20 2024-09-11T22:08:03.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:03.072+0000 7fa2267fc640 1 -- 172.21.15.1:0/271388742 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) ==== 137963+0+0 (secure 0 0 0) 0x7fa22c018a30 con 0x7fa230100e20 2024-09-11T22:08:03.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:03.072+0000 7fa2267fc640 1 --2- 172.21.15.1:0/271388742 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fa21c0a2d70 0x7fa21c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:08:03.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:03.073+0000 7fa2267fc640 1 -- 172.21.15.1:0/271388742 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8490+0+0 (secure 0 0 0) 0x7fa22c0cf890 con 0x7fa230100e20 2024-09-11T22:08:03.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:03.073+0000 7fa234856640 1 --2- 172.21.15.1:0/271388742 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fa21c0a2d70 0x7fa21c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:08:03.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:03.073+0000 7fa234856640 1 --2- 172.21.15.1:0/271388742 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fa21c0a2d70 0x7fa21c0a5230 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa2200096c0 tx=0x7fa220000bc0 comp rx=0 tx=0).ready entity=mgr.15204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:08:03.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:03.076+0000 7fa2267fc640 1 -- 172.21.15.1:0/271388742 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa22c014030 con 0x7fa230100e20 2024-09-11T22:08:03.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:03.252+0000 7fa2372bb640 1 -- 172.21.15.1:0/271388742 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa23010a0a0 con 0x7fa230100e20 2024-09-11T22:08:03.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:03.318+0000 7fa2267fc640 1 -- 172.21.15.1:0/271388742 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v98) ==== 169+0+0 (secure 0 0 0) 0x7fa22c00de20 con 0x7fa230100e20 2024-09-11T22:08:03.319 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_data' removed 2024-09-11T22:08:03.323 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:08:03 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/271388742' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T22:08:03.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:03.322+0000 7fa2372bb640 1 -- 172.21.15.1:0/271388742 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fa21c0a2d70 msgr2=0x7fa21c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:08:03.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:03.322+0000 7fa2372bb640 1 --2- 172.21.15.1:0/271388742 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fa21c0a2d70 0x7fa21c0a5230 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa2200096c0 tx=0x7fa220000bc0 comp rx=0 tx=0).stop 2024-09-11T22:08:03.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:03.322+0000 7fa2372bb640 1 -- 172.21.15.1:0/271388742 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa230100e20 msgr2=0x7fa230198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:08:03.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:03.322+0000 7fa2372bb640 1 --2- 172.21.15.1:0/271388742 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa230100e20 0x7fa230198440 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fa22c00c920 tx=0x7fa22c00cde0 comp rx=0 tx=0).stop 2024-09-11T22:08:03.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:03.323+0000 7fa2372bb640 1 -- 172.21.15.1:0/271388742 shutdown_connections 2024-09-11T22:08:03.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:03.323+0000 7fa2372bb640 1 --2- 172.21.15.1:0/271388742 >> [v2:172.21.15.1:6800/3543150772,v1:172.21.15.1:6801/3543150772] conn(0x7fa21c0a2d70 0x7fa21c0a5230 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:08:03.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:03.323+0000 7fa2372bb640 1 --2- 172.21.15.1:0/271388742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa230102ef0 0x7fa23019cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:08:03.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:03.323+0000 7fa2372bb640 1 --2- 172.21.15.1:0/271388742 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa230102570 0x7fa230198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:08:03.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:03.323+0000 7fa2372bb640 1 --2- 172.21.15.1:0/271388742 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa230100e20 0x7fa230198440 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:08:03.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:03.323+0000 7fa2372bb640 1 -- 172.21.15.1:0/271388742 >> 172.21.15.1:0/271388742 conn(0x7fa2300fc7d0 msgr2=0x7fa23006a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:08:03.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:03.323+0000 7fa2372bb640 1 -- 172.21.15.1:0/271388742 shutdown_connections 2024-09-11T22:08:03.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-09-11T22:08:03.323+0000 7fa2372bb640 1 -- 172.21.15.1:0/271388742 wait complete. 2024-09-11T22:08:03.591 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:08:03 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/271388742' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T22:08:03.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:08:03 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/271388742' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T22:08:03.990 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T22:08:04.018 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T22:08:04.047 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T22:08:04.074 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-11T22:08:04.074 DEBUG:teuthology.orchestra.run.smithi001:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-11T22:08:04.090 DEBUG:teuthology.orchestra.run.smithi053:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-11T22:08:04.109 DEBUG:teuthology.orchestra.run.smithi161:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-11T22:08:04.129 INFO:tasks.cephadm:Stopping all daemons... 2024-09-11T22:08:04.130 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-11T22:08:04.130 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mon.a 2024-09-11T22:08:04.324 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:08:04 smithi001 systemd[1]: Stopping Ceph mon.a for 3ffca83c-7087-11ef-bcea-c7b262605968... 2024-09-11T22:08:04.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:08:04 smithi161 ceph-mon[35615]: from='client.? 172.21.15.1:0/271388742' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T22:08:04.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:08:04 smithi161 ceph-mon[35615]: osdmap e98: 12 total, 12 up, 12 in 2024-09-11T22:08:04.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:08:04 smithi161 ceph-mon[35615]: pgmap v339: 1 pgs: 1 active+clean; 577 KiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:08:04.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:08:04 smithi001 ceph-mon[30935]: from='client.? 172.21.15.1:0/271388742' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T22:08:04.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:08:04 smithi001 ceph-mon[30935]: osdmap e98: 12 total, 12 up, 12 in 2024-09-11T22:08:04.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:08:04 smithi001 ceph-mon[30935]: pgmap v339: 1 pgs: 1 active+clean; 577 KiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:08:04.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:08:04 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mon-a[30931]: 2024-09-11T22:08:04.465+0000 7f234e2b2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:08:04.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:08:04 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mon-a[30931]: 2024-09-11T22:08:04.465+0000 7f234e2b2640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-11T22:08:04.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:08:04 smithi001 podman[207760]: 2024-09-11 22:08:04.527512571 +0000 UTC m=+0.159088114 container died f1d1cf7d2034ea29ba929dbc44cd9681238de202ff525f06a6580de4455f2057 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mon-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-11T22:08:04.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:08:04 smithi053 ceph-mon[34908]: from='client.? 172.21.15.1:0/271388742' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T22:08:04.797 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:08:04 smithi053 ceph-mon[34908]: osdmap e98: 12 total, 12 up, 12 in 2024-09-11T22:08:04.798 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:08:04 smithi053 ceph-mon[34908]: pgmap v339: 1 pgs: 1 active+clean; 577 KiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:08:05.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:08:04 smithi001 podman[207760]: 2024-09-11 22:08:04.941970621 +0000 UTC m=+0.573546161 container cleanup f1d1cf7d2034ea29ba929dbc44cd9681238de202ff525f06a6580de4455f2057 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True) 2024-09-11T22:08:05.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:08:04 smithi001 bash[207760]: ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mon-a 2024-09-11T22:08:05.517 INFO:journalctl@ceph.mon.a.smithi001.stdout:Sep 11 22:08:05 smithi001 podman[207773]: 2024-09-11 22:08:05.260353005 +0000 UTC m=+0.724414970 container remove f1d1cf7d2034ea29ba929dbc44cd9681238de202ff525f06a6580de4455f2057 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mon-a, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-11T22:08:05.532 DEBUG:teuthology.orchestra.run.smithi001:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mon.a.service' 2024-09-11T22:08:05.587 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:08:05.587 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-11T22:08:05.587 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-11T22:08:05.588 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mon.b 2024-09-11T22:08:05.918 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:08:05 smithi053 systemd[1]: Stopping Ceph mon.b for 3ffca83c-7087-11ef-bcea-c7b262605968... 2024-09-11T22:08:06.293 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:08:05 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mon-b[34904]: 2024-09-11T22:08:05.916+0000 7fa944615640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:08:06.293 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:08:05 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mon-b[34904]: 2024-09-11T22:08:05.916+0000 7fa944615640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-11T22:08:06.294 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:08:05 smithi053 podman[69942]: 2024-09-11 22:08:05.999034214 +0000 UTC m=+0.214968041 container died 90807c5d7261dec19a11fb43cfc5fef75a61da9d0d264811240bc195a36a4604 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mon-b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3) 2024-09-11T22:08:06.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:08:06 smithi053 podman[69942]: 2024-09-11 22:08:06.392659762 +0000 UTC m=+0.608593624 container cleanup 90807c5d7261dec19a11fb43cfc5fef75a61da9d0d264811240bc195a36a4604 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=) 2024-09-11T22:08:06.548 INFO:journalctl@ceph.mon.b.smithi053.stdout:Sep 11 22:08:06 smithi053 bash[69942]: ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mon-b 2024-09-11T22:08:06.776 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mon.b.service' 2024-09-11T22:08:06.824 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:08:06.824 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-11T22:08:06.824 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-11T22:08:06.824 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mon.c 2024-09-11T22:08:07.148 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:08:06 smithi161 systemd[1]: Stopping Ceph mon.c for 3ffca83c-7087-11ef-bcea-c7b262605968... 2024-09-11T22:08:07.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:08:07 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mon-c[35611]: 2024-09-11T22:08:07.146+0000 7f200c0bf640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:08:07.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:08:07 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mon-c[35611]: 2024-09-11T22:08:07.146+0000 7f200c0bf640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-11T22:08:07.592 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:08:07 smithi161 podman[71403]: 2024-09-11 22:08:07.362171595 +0000 UTC m=+0.341494358 container died 04ee519b67d41cfb5ad7250ad2b5fb9d31a0d7ece7bb50ee0db8e31731495e60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mon-c, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-11T22:08:08.016 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:08:07 smithi161 podman[71403]: 2024-09-11 22:08:07.730751369 +0000 UTC m=+0.710074132 container cleanup 04ee519b67d41cfb5ad7250ad2b5fb9d31a0d7ece7bb50ee0db8e31731495e60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mon-c, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3) 2024-09-11T22:08:08.016 INFO:journalctl@ceph.mon.c.smithi161.stdout:Sep 11 22:08:07 smithi161 bash[71403]: ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mon-c 2024-09-11T22:08:08.139 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mon.c.service' 2024-09-11T22:08:08.182 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:08:08.183 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-11T22:08:08.183 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-11T22:08:08.183 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mgr.x 2024-09-11T22:08:08.506 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 22:08:08 smithi001 systemd[1]: Stopping Ceph mgr.x for 3ffca83c-7087-11ef-bcea-c7b262605968... 2024-09-11T22:08:08.823 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 22:08:08 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x[31161]: 2024-09-11T22:08:08.504+0000 7f5fab8c7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:08:08.824 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 22:08:08 smithi001 podman[207880]: 2024-09-11 22:08:08.535905279 +0000 UTC m=+0.146539522 container died 4bf5fcbd71e5d307481af2d58c5ea88a95d452fe77949423dcca832fe91c6b02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-b53b6d3, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-11T22:08:09.193 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 22:08:08 smithi001 podman[207880]: 2024-09-11 22:08:08.923095507 +0000 UTC m=+0.533729749 container cleanup 4bf5fcbd71e5d307481af2d58c5ea88a95d452fe77949423dcca832fe91c6b02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T22:08:09.193 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Sep 11 22:08:08 smithi001 bash[207880]: ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-x 2024-09-11T22:08:09.404 DEBUG:teuthology.orchestra.run.smithi001:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mgr.x.service' 2024-09-11T22:08:09.440 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:08:09.440 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-11T22:08:09.440 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-11T22:08:09.440 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mgr.y 2024-09-11T22:08:09.720 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 11 22:08:09 smithi053 systemd[1]: Stopping Ceph mgr.y for 3ffca83c-7087-11ef-bcea-c7b262605968... 2024-09-11T22:08:09.981 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 11 22:08:09 smithi053 podman[70066]: 2024-09-11 22:08:09.785851179 +0000 UTC m=+0.144061064 container died a86754c804b97724293d3c5f6bf1074957dd6f15f6f9d931dd3897af437e001a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-y, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-11T22:08:10.659 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 11 22:08:10 smithi053 podman[70066]: 2024-09-11 22:08:10.298507682 +0000 UTC m=+0.656717561 container cleanup a86754c804b97724293d3c5f6bf1074957dd6f15f6f9d931dd3897af437e001a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-y, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-11T22:08:10.659 INFO:journalctl@ceph.mgr.y.smithi053.stdout:Sep 11 22:08:10 smithi053 bash[70066]: ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-y 2024-09-11T22:08:10.875 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mgr.y.service' 2024-09-11T22:08:10.913 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:08:10.914 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-11T22:08:10.914 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-11T22:08:10.914 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mgr.z 2024-09-11T22:08:11.248 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 11 22:08:10 smithi161 systemd[1]: Stopping Ceph mgr.z for 3ffca83c-7087-11ef-bcea-c7b262605968... 2024-09-11T22:08:11.521 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 11 22:08:11 smithi161 podman[71521]: 2024-09-11 22:08:11.24771504 +0000 UTC m=+0.135755731 container stop 57051efff98290a971d35f0fab755f4dfaa93b5030c769aabd1c1f25e0cab756 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-z, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T22:08:11.522 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 11 22:08:11 smithi161 podman[71521]: 2024-09-11 22:08:11.2755999 +0000 UTC m=+0.163640603 container died 57051efff98290a971d35f0fab755f4dfaa93b5030c769aabd1c1f25e0cab756 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-09-11T22:08:11.842 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 11 22:08:11 smithi161 podman[71521]: 2024-09-11 22:08:11.694461026 +0000 UTC m=+0.582501714 container cleanup 57051efff98290a971d35f0fab755f4dfaa93b5030c769aabd1c1f25e0cab756 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-z, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T22:08:11.842 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 11 22:08:11 smithi161 bash[71521]: ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-z 2024-09-11T22:08:12.116 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mgr.z.service' 2024-09-11T22:08:12.138 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 11 22:08:11 smithi161 podman[71532]: 2024-09-11 22:08:11.879720814 +0000 UTC m=+0.624795909 container remove 57051efff98290a971d35f0fab755f4dfaa93b5030c769aabd1c1f25e0cab756 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mgr-z, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-11T22:08:12.138 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 11 22:08:11 smithi161 systemd[1]: ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-09-11T22:08:12.139 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 11 22:08:12 smithi161 systemd[1]: ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mgr.z.service: Failed with result 'exit-code'. 2024-09-11T22:08:12.139 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 11 22:08:12 smithi161 systemd[1]: Stopped Ceph mgr.z for 3ffca83c-7087-11ef-bcea-c7b262605968. 2024-09-11T22:08:12.139 INFO:journalctl@ceph.mgr.z.smithi161.stdout:Sep 11 22:08:12 smithi161 systemd[1]: ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mgr.z.service: Consumed 11.527s CPU time. 2024-09-11T22:08:12.153 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:08:12.154 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-11T22:08:12.154 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-11T22:08:12.154 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ceph-3ffca83c-7087-11ef-bcea-c7b262605968@osd.0 2024-09-11T22:08:12.443 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 11 22:08:12 smithi001 systemd[1]: Stopping Ceph osd.0 for 3ffca83c-7087-11ef-bcea-c7b262605968... 2024-09-11T22:08:12.943 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 11 22:08:12 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-0[42460]: 2024-09-11T22:08:12.486+0000 7f9c0748d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:08:12.943 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 11 22:08:12 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-0[42460]: 2024-09-11T22:08:12.486+0000 7f9c0748d640 -1 osd.0 98 *** Got signal Terminated *** 2024-09-11T22:08:12.944 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 11 22:08:12 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-0[42460]: 2024-09-11T22:08:12.486+0000 7f9c0748d640 -1 osd.0 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T22:08:18.263 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 11 22:08:17 smithi001 podman[208001]: 2024-09-11 22:08:17.989162538 +0000 UTC m=+5.637148640 container died 975edddfaf6ebd8fde5196c078d0f90c75cccb4bf9f25ed37eb3d0c8c4e6eceb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True) 2024-09-11T22:08:18.693 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 11 22:08:18 smithi001 podman[208001]: 2024-09-11 22:08:18.462308248 +0000 UTC m=+6.110294347 container cleanup 975edddfaf6ebd8fde5196c078d0f90c75cccb4bf9f25ed37eb3d0c8c4e6eceb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T22:08:18.694 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 11 22:08:18 smithi001 bash[208001]: ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-0 2024-09-11T22:08:19.094 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 11 22:08:18 smithi001 podman[208016]: 2024-09-11 22:08:18.774873563 +0000 UTC m=+0.785173558 container remove 975edddfaf6ebd8fde5196c078d0f90c75cccb4bf9f25ed37eb3d0c8c4e6eceb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T22:08:19.353 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 11 22:08:19 smithi001 podman[208110]: 2024-09-11 22:08:18.996925338 +0000 UTC m=+0.016538797 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T22:08:19.354 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 11 22:08:19 smithi001 podman[208110]: 2024-09-11 22:08:19.226254964 +0000 UTC m=+0.245868452 container create 6f654f2d7c7715e0a006358852a989fd3c2f1689fee1d0dafc4e1cf460f9f6d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-0-deactivate, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-11T22:08:19.693 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 11 22:08:19 smithi001 podman[208110]: 2024-09-11 22:08:19.462474359 +0000 UTC m=+0.482087823 container init 6f654f2d7c7715e0a006358852a989fd3c2f1689fee1d0dafc4e1cf460f9f6d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-11T22:08:19.693 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 11 22:08:19 smithi001 podman[208110]: 2024-09-11 22:08:19.466054389 +0000 UTC m=+0.485667842 container start 6f654f2d7c7715e0a006358852a989fd3c2f1689fee1d0dafc4e1cf460f9f6d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T22:08:19.694 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 11 22:08:19 smithi001 podman[208110]: 2024-09-11 22:08:19.579058162 +0000 UTC m=+0.598671621 container attach 6f654f2d7c7715e0a006358852a989fd3c2f1689fee1d0dafc4e1cf460f9f6d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-11T22:08:20.693 INFO:journalctl@ceph.osd.0.smithi001.stdout:Sep 11 22:08:20 smithi001 podman[208110]: 2024-09-11 22:08:20.413165692 +0000 UTC m=+1.432779141 container died 6f654f2d7c7715e0a006358852a989fd3c2f1689fee1d0dafc4e1cf460f9f6d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0) 2024-09-11T22:08:21.301 DEBUG:teuthology.orchestra.run.smithi001:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3ffca83c-7087-11ef-bcea-c7b262605968@osd.0.service' 2024-09-11T22:08:21.344 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:08:21.344 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-11T22:08:21.345 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-11T22:08:21.345 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-3ffca83c-7087-11ef-bcea-c7b262605968@osd.1 2024-09-11T22:08:21.671 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 11 22:08:21 smithi053 systemd[1]: Stopping Ceph osd.1 for 3ffca83c-7087-11ef-bcea-c7b262605968... 2024-09-11T22:08:22.047 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 11 22:08:21 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-1[39386]: 2024-09-11T22:08:21.670+0000 7f6c7017e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:08:22.048 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 11 22:08:21 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-1[39386]: 2024-09-11T22:08:21.670+0000 7f6c7017e640 -1 osd.1 98 *** Got signal Terminated *** 2024-09-11T22:08:22.048 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 11 22:08:21 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-1[39386]: 2024-09-11T22:08:21.670+0000 7f6c7017e640 -1 osd.1 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T22:08:27.298 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 11 22:08:27 smithi053 podman[70202]: 2024-09-11 22:08:27.007023752 +0000 UTC m=+5.460250928 container died 6e03d87d492f56ad4a34c22e21c9438271c164b11ba0ae0d978420022991a7ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-1, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T22:08:27.606 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 11 22:08:27 smithi053 podman[70202]: 2024-09-11 22:08:27.413030726 +0000 UTC m=+5.866257891 container cleanup 6e03d87d492f56ad4a34c22e21c9438271c164b11ba0ae0d978420022991a7ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-1, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-11T22:08:27.606 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 11 22:08:27 smithi053 bash[70202]: ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-1 2024-09-11T22:08:28.040 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 11 22:08:27 smithi053 podman[70217]: 2024-09-11 22:08:27.606004756 +0000 UTC m=+0.595625165 container remove 6e03d87d492f56ad4a34c22e21c9438271c164b11ba0ae0d978420022991a7ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-1, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-11T22:08:28.297 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 11 22:08:28 smithi053 podman[70307]: 2024-09-11 22:08:27.942967115 +0000 UTC m=+0.016352720 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T22:08:28.297 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 11 22:08:28 smithi053 podman[70307]: 2024-09-11 22:08:28.0572162 +0000 UTC m=+0.130601837 container create f5c42de5a9dd638e964b744e7c42866970c2523f06659ca5da7f55c686ac1f19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-11T22:08:28.798 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 11 22:08:28 smithi053 podman[70307]: 2024-09-11 22:08:28.318590595 +0000 UTC m=+0.391976193 container init f5c42de5a9dd638e964b744e7c42866970c2523f06659ca5da7f55c686ac1f19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-11T22:08:28.798 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 11 22:08:28 smithi053 podman[70307]: 2024-09-11 22:08:28.321980863 +0000 UTC m=+0.395366466 container start f5c42de5a9dd638e964b744e7c42866970c2523f06659ca5da7f55c686ac1f19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T22:08:28.798 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 11 22:08:28 smithi053 podman[70307]: 2024-09-11 22:08:28.418608103 +0000 UTC m=+0.491993712 container attach f5c42de5a9dd638e964b744e7c42866970c2523f06659ca5da7f55c686ac1f19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T22:08:28.798 INFO:journalctl@ceph.osd.1.smithi053.stdout:Sep 11 22:08:28 smithi053 podman[70307]: 2024-09-11 22:08:28.511140139 +0000 UTC m=+0.584525735 container died f5c42de5a9dd638e964b744e7c42866970c2523f06659ca5da7f55c686ac1f19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-1-deactivate, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-11T22:08:29.142 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3ffca83c-7087-11ef-bcea-c7b262605968@osd.1.service' 2024-09-11T22:08:29.182 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:08:29.183 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-11T22:08:29.183 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-11T22:08:29.183 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-3ffca83c-7087-11ef-bcea-c7b262605968@osd.2 2024-09-11T22:08:29.508 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 11 22:08:29 smithi161 systemd[1]: Stopping Ceph osd.2 for 3ffca83c-7087-11ef-bcea-c7b262605968... 2024-09-11T22:08:29.841 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 11 22:08:29 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-2[40567]: 2024-09-11T22:08:29.507+0000 7fc56c51d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:08:29.842 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 11 22:08:29 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-2[40567]: 2024-09-11T22:08:29.507+0000 7fc56c51d640 -1 osd.2 98 *** Got signal Terminated *** 2024-09-11T22:08:29.842 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 11 22:08:29 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-2[40567]: 2024-09-11T22:08:29.507+0000 7fc56c51d640 -1 osd.2 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T22:08:35.342 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 11 22:08:35 smithi161 podman[71658]: 2024-09-11 22:08:35.069460674 +0000 UTC m=+5.682696586 container died 681a2fb4e57df557d39b0b30ecece919589540b35e76355f7f17c9ebccb9e568 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3) 2024-09-11T22:08:35.767 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 11 22:08:35 smithi161 podman[71658]: 2024-09-11 22:08:35.473851033 +0000 UTC m=+6.087086945 container cleanup 681a2fb4e57df557d39b0b30ecece919589540b35e76355f7f17c9ebccb9e568 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T22:08:35.767 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 11 22:08:35 smithi161 bash[71658]: ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-2 2024-09-11T22:08:36.084 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 11 22:08:35 smithi161 podman[71676]: 2024-09-11 22:08:35.767032914 +0000 UTC m=+0.692179876 container remove 681a2fb4e57df557d39b0b30ecece919589540b35e76355f7f17c9ebccb9e568 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-11T22:08:36.341 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 11 22:08:36 smithi161 podman[71762]: 2024-09-11 22:08:35.986430517 +0000 UTC m=+0.015860503 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T22:08:36.342 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 11 22:08:36 smithi161 podman[71762]: 2024-09-11 22:08:36.126364325 +0000 UTC m=+0.155794311 container create 55f53658d340a5b7d2db926e9a4b30043eef15eb5dfeaa67df288242bbaf2133 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-11T22:08:36.602 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 11 22:08:36 smithi161 podman[71762]: 2024-09-11 22:08:36.404201048 +0000 UTC m=+0.433631034 container init 55f53658d340a5b7d2db926e9a4b30043eef15eb5dfeaa67df288242bbaf2133 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-2-deactivate, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-11T22:08:36.603 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 11 22:08:36 smithi161 podman[71762]: 2024-09-11 22:08:36.407521971 +0000 UTC m=+0.436951966 container start 55f53658d340a5b7d2db926e9a4b30043eef15eb5dfeaa67df288242bbaf2133 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T22:08:36.603 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 11 22:08:36 smithi161 podman[71762]: 2024-09-11 22:08:36.504018302 +0000 UTC m=+0.533448294 container attach 55f53658d340a5b7d2db926e9a4b30043eef15eb5dfeaa67df288242bbaf2133 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3) 2024-09-11T22:08:36.891 INFO:journalctl@ceph.osd.2.smithi161.stdout:Sep 11 22:08:36 smithi161 podman[71762]: 2024-09-11 22:08:36.60272206 +0000 UTC m=+0.632152056 container died 55f53658d340a5b7d2db926e9a4b30043eef15eb5dfeaa67df288242bbaf2133 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-2-deactivate, ceph=True, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T22:08:37.228 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3ffca83c-7087-11ef-bcea-c7b262605968@osd.2.service' 2024-09-11T22:08:37.269 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:08:37.269 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-11T22:08:37.270 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-11T22:08:37.270 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ceph-3ffca83c-7087-11ef-bcea-c7b262605968@osd.3 2024-09-11T22:08:37.595 INFO:journalctl@ceph.osd.3.smithi001.stdout:Sep 11 22:08:37 smithi001 systemd[1]: Stopping Ceph osd.3 for 3ffca83c-7087-11ef-bcea-c7b262605968... 2024-09-11T22:08:37.943 INFO:journalctl@ceph.osd.3.smithi001.stdout:Sep 11 22:08:37 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-3[49085]: 2024-09-11T22:08:37.593+0000 7eff39155640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:08:37.943 INFO:journalctl@ceph.osd.3.smithi001.stdout:Sep 11 22:08:37 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-3[49085]: 2024-09-11T22:08:37.593+0000 7eff39155640 -1 osd.3 98 *** Got signal Terminated *** 2024-09-11T22:08:37.944 INFO:journalctl@ceph.osd.3.smithi001.stdout:Sep 11 22:08:37 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-3[49085]: 2024-09-11T22:08:37.593+0000 7eff39155640 -1 osd.3 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T22:08:39.193 INFO:journalctl@ceph.osd.3.smithi001.stdout:Sep 11 22:08:38 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-3[49085]: 2024-09-11T22:08:38.782+0000 7eff3576b640 -1 osd.3 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:15.799140+0000 front 2024-09-11T22:08:15.799101+0000 (oldest deadline 2024-09-11T22:08:38.699233+0000) 2024-09-11T22:08:39.842 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:08:39 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:08:39.567+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.893190+0000 front 2024-09-11T22:08:16.893259+0000 (oldest deadline 2024-09-11T22:08:39.193066+0000) 2024-09-11T22:08:40.193 INFO:journalctl@ceph.osd.3.smithi001.stdout:Sep 11 22:08:39 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-3[49085]: 2024-09-11T22:08:39.801+0000 7eff3576b640 -1 osd.3 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:15.799140+0000 front 2024-09-11T22:08:15.799101+0000 (oldest deadline 2024-09-11T22:08:38.699233+0000) 2024-09-11T22:08:41.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:08:40 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:08:40.613+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.893190+0000 front 2024-09-11T22:08:16.893259+0000 (oldest deadline 2024-09-11T22:08:39.193066+0000) 2024-09-11T22:08:41.193 INFO:journalctl@ceph.osd.3.smithi001.stdout:Sep 11 22:08:40 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-3[49085]: 2024-09-11T22:08:40.827+0000 7eff3576b640 -1 osd.3 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:15.799140+0000 front 2024-09-11T22:08:15.799101+0000 (oldest deadline 2024-09-11T22:08:38.699233+0000) 2024-09-11T22:08:41.986 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:08:41 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:08:41.630+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.893190+0000 front 2024-09-11T22:08:16.893259+0000 (oldest deadline 2024-09-11T22:08:39.193066+0000) 2024-09-11T22:08:42.193 INFO:journalctl@ceph.osd.3.smithi001.stdout:Sep 11 22:08:41 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-3[49085]: 2024-09-11T22:08:41.837+0000 7eff3576b640 -1 osd.3 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:15.799140+0000 front 2024-09-11T22:08:15.799101+0000 (oldest deadline 2024-09-11T22:08:38.699233+0000) 2024-09-11T22:08:42.342 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 11 22:08:41 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-5[47459]: 2024-09-11T22:08:41.985+0000 7ff94e3fd640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.528685+0000 front 2024-09-11T22:08:16.528663+0000 (oldest deadline 2024-09-11T22:08:41.828661+0000) 2024-09-11T22:08:42.943 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:08:42 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:08:42.468+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.497679+0000 front 2024-09-11T22:08:16.497606+0000 (oldest deadline 2024-09-11T22:08:42.397744+0000) 2024-09-11T22:08:42.990 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:08:42 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:08:42.658+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.893190+0000 front 2024-09-11T22:08:16.893259+0000 (oldest deadline 2024-09-11T22:08:39.193066+0000) 2024-09-11T22:08:43.342 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 11 22:08:42 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-5[47459]: 2024-09-11T22:08:42.989+0000 7ff94e3fd640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.528685+0000 front 2024-09-11T22:08:16.528663+0000 (oldest deadline 2024-09-11T22:08:41.828661+0000) 2024-09-11T22:08:43.362 INFO:journalctl@ceph.osd.3.smithi001.stdout:Sep 11 22:08:43 smithi001 podman[208211]: 2024-09-11 22:08:43.074124247 +0000 UTC m=+5.597549180 container died d5ace9b3ae60f2c45339527afd4fab286101758f685a31ac5fbe6cd70052ef38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-3, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-11T22:08:43.363 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 11 22:08:43 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-6[55973]: 2024-09-11T22:08:43.007+0000 7f9c8be0c640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.487210+0000 front 2024-09-11T22:08:17.487317+0000 (oldest deadline 2024-09-11T22:08:42.187568+0000) 2024-09-11T22:08:43.547 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:08:43 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:08:43.102+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.481596+0000 front 2024-09-11T22:08:17.481499+0000 (oldest deadline 2024-09-11T22:08:42.181347+0000) 2024-09-11T22:08:43.693 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:08:43 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:08:43.485+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.497679+0000 front 2024-09-11T22:08:16.497606+0000 (oldest deadline 2024-09-11T22:08:42.397744+0000) 2024-09-11T22:08:43.693 INFO:journalctl@ceph.osd.3.smithi001.stdout:Sep 11 22:08:43 smithi001 podman[208211]: 2024-09-11 22:08:43.570451167 +0000 UTC m=+6.093876146 container cleanup d5ace9b3ae60f2c45339527afd4fab286101758f685a31ac5fbe6cd70052ef38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-3, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-11T22:08:43.694 INFO:journalctl@ceph.osd.3.smithi001.stdout:Sep 11 22:08:43 smithi001 bash[208211]: ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-3 2024-09-11T22:08:43.972 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:08:43 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:08:43.707+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.893190+0000 front 2024-09-11T22:08:16.893259+0000 (oldest deadline 2024-09-11T22:08:39.193066+0000) 2024-09-11T22:08:44.193 INFO:journalctl@ceph.osd.3.smithi001.stdout:Sep 11 22:08:43 smithi001 podman[208229]: 2024-09-11 22:08:43.880924605 +0000 UTC m=+0.800968744 container remove d5ace9b3ae60f2c45339527afd4fab286101758f685a31ac5fbe6cd70052ef38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-3, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True) 2024-09-11T22:08:44.194 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 11 22:08:43 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-6[55973]: 2024-09-11T22:08:43.960+0000 7f9c8be0c640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.487210+0000 front 2024-09-11T22:08:17.487317+0000 (oldest deadline 2024-09-11T22:08:42.187568+0000) 2024-09-11T22:08:44.341 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 11 22:08:43 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-5[47459]: 2024-09-11T22:08:43.971+0000 7ff94e3fd640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.528685+0000 front 2024-09-11T22:08:16.528663+0000 (oldest deadline 2024-09-11T22:08:41.828661+0000) 2024-09-11T22:08:44.454 INFO:journalctl@ceph.osd.3.smithi001.stdout:Sep 11 22:08:44 smithi001 podman[208315]: 2024-09-11 22:08:44.104868769 +0000 UTC m=+0.016760750 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T22:08:44.455 INFO:journalctl@ceph.osd.3.smithi001.stdout:Sep 11 22:08:44 smithi001 podman[208315]: 2024-09-11 22:08:44.215154979 +0000 UTC m=+0.127047010 container create 2734fa01528c40e694b77bd33b6c882ad3aa006d4555ef707b80726398ad6f11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-3-deactivate, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, ceph=True, GIT_CLEAN=True) 2024-09-11T22:08:44.455 INFO:journalctl@ceph.osd.3.smithi001.stdout:Sep 11 22:08:44 smithi001 podman[208315]: 2024-09-11 22:08:44.451024841 +0000 UTC m=+0.362916810 container init 2734fa01528c40e694b77bd33b6c882ad3aa006d4555ef707b80726398ad6f11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-11T22:08:44.455 INFO:journalctl@ceph.osd.3.smithi001.stdout:Sep 11 22:08:44 smithi001 podman[208315]: 2024-09-11 22:08:44.454618516 +0000 UTC m=+0.366510495 container start 2734fa01528c40e694b77bd33b6c882ad3aa006d4555ef707b80726398ad6f11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T22:08:44.547 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:08:44 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:08:44.089+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.481596+0000 front 2024-09-11T22:08:17.481499+0000 (oldest deadline 2024-09-11T22:08:42.181347+0000) 2024-09-11T22:08:44.943 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:08:44 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:08:44.526+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.497679+0000 front 2024-09-11T22:08:16.497606+0000 (oldest deadline 2024-09-11T22:08:42.397744+0000) 2024-09-11T22:08:44.944 INFO:journalctl@ceph.osd.3.smithi001.stdout:Sep 11 22:08:44 smithi001 podman[208315]: 2024-09-11 22:08:44.559526882 +0000 UTC m=+0.471418860 container attach 2734fa01528c40e694b77bd33b6c882ad3aa006d4555ef707b80726398ad6f11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-3-deactivate, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-11T22:08:44.944 INFO:journalctl@ceph.osd.3.smithi001.stdout:Sep 11 22:08:44 smithi001 podman[208315]: 2024-09-11 22:08:44.653814175 +0000 UTC m=+0.565706154 container died 2734fa01528c40e694b77bd33b6c882ad3aa006d4555ef707b80726398ad6f11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-11T22:08:44.961 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:08:44 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:08:44.703+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.893190+0000 front 2024-09-11T22:08:16.893259+0000 (oldest deadline 2024-09-11T22:08:39.193066+0000) 2024-09-11T22:08:45.322 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 11 22:08:44 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-6[55973]: 2024-09-11T22:08:44.976+0000 7f9c8be0c640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.487210+0000 front 2024-09-11T22:08:17.487317+0000 (oldest deadline 2024-09-11T22:08:42.187568+0000) 2024-09-11T22:08:45.342 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 11 22:08:44 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-5[47459]: 2024-09-11T22:08:44.960+0000 7ff94e3fd640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.528685+0000 front 2024-09-11T22:08:16.528663+0000 (oldest deadline 2024-09-11T22:08:41.828661+0000) 2024-09-11T22:08:45.351 DEBUG:teuthology.orchestra.run.smithi001:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3ffca83c-7087-11ef-bcea-c7b262605968@osd.3.service' 2024-09-11T22:08:45.394 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:08:45.394 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-11T22:08:45.395 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-11T22:08:45.395 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-3ffca83c-7087-11ef-bcea-c7b262605968@osd.4 2024-09-11T22:08:45.423 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:08:45 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:08:45.095+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.481596+0000 front 2024-09-11T22:08:17.481499+0000 (oldest deadline 2024-09-11T22:08:42.181347+0000) 2024-09-11T22:08:45.693 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:08:45 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:08:45.519+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.497679+0000 front 2024-09-11T22:08:16.497606+0000 (oldest deadline 2024-09-11T22:08:42.397744+0000) 2024-09-11T22:08:45.710 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 11 22:08:45 smithi053 systemd[1]: Stopping Ceph osd.4 for 3ffca83c-7087-11ef-bcea-c7b262605968... 2024-09-11T22:08:46.047 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 11 22:08:45 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-4[45992]: 2024-09-11T22:08:45.709+0000 7fef05ea1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:08:46.048 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 11 22:08:45 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-4[45992]: 2024-09-11T22:08:45.709+0000 7fef05ea1640 -1 osd.4 98 *** Got signal Terminated *** 2024-09-11T22:08:46.048 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 11 22:08:45 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-4[45992]: 2024-09-11T22:08:45.709+0000 7fef05ea1640 -1 osd.4 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T22:08:46.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:08:45 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:08:45.713+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.893190+0000 front 2024-09-11T22:08:16.893259+0000 (oldest deadline 2024-09-11T22:08:39.193066+0000) 2024-09-11T22:08:46.092 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 11 22:08:45 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-5[47459]: 2024-09-11T22:08:45.942+0000 7ff94e3fd640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.528685+0000 front 2024-09-11T22:08:16.528663+0000 (oldest deadline 2024-09-11T22:08:41.828661+0000) 2024-09-11T22:08:46.443 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 11 22:08:46 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-6[55973]: 2024-09-11T22:08:46.020+0000 7f9c8be0c640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.487210+0000 front 2024-09-11T22:08:17.487317+0000 (oldest deadline 2024-09-11T22:08:42.187568+0000) 2024-09-11T22:08:46.547 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:08:46 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:08:46.051+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.481596+0000 front 2024-09-11T22:08:17.481499+0000 (oldest deadline 2024-09-11T22:08:42.181347+0000) 2024-09-11T22:08:46.943 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:08:46 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:08:46.525+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.497679+0000 front 2024-09-11T22:08:16.497606+0000 (oldest deadline 2024-09-11T22:08:42.397744+0000) 2024-09-11T22:08:47.092 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 11 22:08:46 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-5[47459]: 2024-09-11T22:08:46.947+0000 7ff94e3fd640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.528685+0000 front 2024-09-11T22:08:16.528663+0000 (oldest deadline 2024-09-11T22:08:41.828661+0000) 2024-09-11T22:08:47.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:08:46 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:08:46.706+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.893190+0000 front 2024-09-11T22:08:16.893259+0000 (oldest deadline 2024-09-11T22:08:39.193066+0000) 2024-09-11T22:08:47.443 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 11 22:08:47 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-6[55973]: 2024-09-11T22:08:47.033+0000 7f9c8be0c640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.487210+0000 front 2024-09-11T22:08:17.487317+0000 (oldest deadline 2024-09-11T22:08:42.187568+0000) 2024-09-11T22:08:47.547 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:08:47 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:08:47.082+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.481596+0000 front 2024-09-11T22:08:17.481499+0000 (oldest deadline 2024-09-11T22:08:42.181347+0000) 2024-09-11T22:08:47.943 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:08:47 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:08:47.546+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.497679+0000 front 2024-09-11T22:08:16.497606+0000 (oldest deadline 2024-09-11T22:08:42.397744+0000) 2024-09-11T22:08:47.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:08:47 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:08:47.546+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:22.398445+0000 front 2024-09-11T22:08:22.398495+0000 (oldest deadline 2024-09-11T22:08:47.098448+0000) 2024-09-11T22:08:47.960 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:08:47 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:08:47.701+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.893190+0000 front 2024-09-11T22:08:16.893259+0000 (oldest deadline 2024-09-11T22:08:39.193066+0000) 2024-09-11T22:08:47.960 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:08:47 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:08:47.701+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:21.494266+0000 front 2024-09-11T22:08:21.494360+0000 (oldest deadline 2024-09-11T22:08:47.394416+0000) 2024-09-11T22:08:48.342 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 11 22:08:47 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-5[47459]: 2024-09-11T22:08:47.959+0000 7ff94e3fd640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.528685+0000 front 2024-09-11T22:08:16.528663+0000 (oldest deadline 2024-09-11T22:08:41.828661+0000) 2024-09-11T22:08:48.342 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 11 22:08:47 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-5[47459]: 2024-09-11T22:08:47.959+0000 7ff94e3fd640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:23.529853+0000 front 2024-09-11T22:08:23.530033+0000 (oldest deadline 2024-09-11T22:08:47.029940+0000) 2024-09-11T22:08:48.443 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 11 22:08:47 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-6[55973]: 2024-09-11T22:08:47.985+0000 7f9c8be0c640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.487210+0000 front 2024-09-11T22:08:17.487317+0000 (oldest deadline 2024-09-11T22:08:42.187568+0000) 2024-09-11T22:08:48.547 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:08:48 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:08:48.120+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.481596+0000 front 2024-09-11T22:08:17.481499+0000 (oldest deadline 2024-09-11T22:08:42.181347+0000) 2024-09-11T22:08:48.943 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:08:48 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:08:48.532+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.497679+0000 front 2024-09-11T22:08:16.497606+0000 (oldest deadline 2024-09-11T22:08:42.397744+0000) 2024-09-11T22:08:48.943 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:08:48 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:08:48.532+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:22.398445+0000 front 2024-09-11T22:08:22.398495+0000 (oldest deadline 2024-09-11T22:08:47.098448+0000) 2024-09-11T22:08:49.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:08:48 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:08:48.739+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.893190+0000 front 2024-09-11T22:08:16.893259+0000 (oldest deadline 2024-09-11T22:08:39.193066+0000) 2024-09-11T22:08:49.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:08:48 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:08:48.739+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:21.494266+0000 front 2024-09-11T22:08:21.494360+0000 (oldest deadline 2024-09-11T22:08:47.394416+0000) 2024-09-11T22:08:49.092 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 11 22:08:48 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-5[47459]: 2024-09-11T22:08:48.976+0000 7ff94e3fd640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.528685+0000 front 2024-09-11T22:08:16.528663+0000 (oldest deadline 2024-09-11T22:08:41.828661+0000) 2024-09-11T22:08:49.093 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 11 22:08:48 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-5[47459]: 2024-09-11T22:08:48.977+0000 7ff94e3fd640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:23.529853+0000 front 2024-09-11T22:08:23.530033+0000 (oldest deadline 2024-09-11T22:08:47.029940+0000) 2024-09-11T22:08:49.128 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 11 22:08:48 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-4[45992]: 2024-09-11T22:08:48.871+0000 7fef024b7640 -1 osd.4 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:23.549725+0000 front 2024-09-11T22:08:23.549749+0000 (oldest deadline 2024-09-11T22:08:48.249970+0000) 2024-09-11T22:08:49.443 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 11 22:08:49 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-6[55973]: 2024-09-11T22:08:49.001+0000 7f9c8be0c640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.487210+0000 front 2024-09-11T22:08:17.487317+0000 (oldest deadline 2024-09-11T22:08:42.187568+0000) 2024-09-11T22:08:49.547 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:08:49 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:08:49.126+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.481596+0000 front 2024-09-11T22:08:17.481499+0000 (oldest deadline 2024-09-11T22:08:42.181347+0000) 2024-09-11T22:08:49.548 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 22:08:49 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-7[52494]: 2024-09-11T22:08:49.212+0000 7f29f1026640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:25.674626+0000 front 2024-09-11T22:08:25.674669+0000 (oldest deadline 2024-09-11T22:08:49.174727+0000) 2024-09-11T22:08:49.943 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:08:49 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:08:49.494+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.497679+0000 front 2024-09-11T22:08:16.497606+0000 (oldest deadline 2024-09-11T22:08:42.397744+0000) 2024-09-11T22:08:49.943 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:08:49 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:08:49.494+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:22.398445+0000 front 2024-09-11T22:08:22.398495+0000 (oldest deadline 2024-09-11T22:08:47.098448+0000) 2024-09-11T22:08:50.092 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 11 22:08:49 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-5[47459]: 2024-09-11T22:08:49.965+0000 7ff94e3fd640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.528685+0000 front 2024-09-11T22:08:16.528663+0000 (oldest deadline 2024-09-11T22:08:41.828661+0000) 2024-09-11T22:08:50.092 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 11 22:08:49 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-5[47459]: 2024-09-11T22:08:49.965+0000 7ff94e3fd640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:23.529853+0000 front 2024-09-11T22:08:23.530033+0000 (oldest deadline 2024-09-11T22:08:47.029940+0000) 2024-09-11T22:08:50.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:08:49 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:08:49.717+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.893190+0000 front 2024-09-11T22:08:16.893259+0000 (oldest deadline 2024-09-11T22:08:39.193066+0000) 2024-09-11T22:08:50.093 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:08:49 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:08:49.717+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:21.494266+0000 front 2024-09-11T22:08:21.494360+0000 (oldest deadline 2024-09-11T22:08:47.394416+0000) 2024-09-11T22:08:50.131 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 11 22:08:49 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-4[45992]: 2024-09-11T22:08:49.873+0000 7fef024b7640 -1 osd.4 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:23.549725+0000 front 2024-09-11T22:08:23.549749+0000 (oldest deadline 2024-09-11T22:08:48.249970+0000) 2024-09-11T22:08:50.443 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 11 22:08:50 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-6[55973]: 2024-09-11T22:08:50.011+0000 7f9c8be0c640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.487210+0000 front 2024-09-11T22:08:17.487317+0000 (oldest deadline 2024-09-11T22:08:42.187568+0000) 2024-09-11T22:08:50.547 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 22:08:50 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-7[52494]: 2024-09-11T22:08:50.249+0000 7f29f1026640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:25.674626+0000 front 2024-09-11T22:08:25.674669+0000 (oldest deadline 2024-09-11T22:08:49.174727+0000) 2024-09-11T22:08:50.548 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:08:50 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:08:50.131+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.481596+0000 front 2024-09-11T22:08:17.481499+0000 (oldest deadline 2024-09-11T22:08:42.181347+0000) 2024-09-11T22:08:50.943 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:08:50 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:08:50.532+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.497679+0000 front 2024-09-11T22:08:16.497606+0000 (oldest deadline 2024-09-11T22:08:42.397744+0000) 2024-09-11T22:08:50.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:08:50 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:08:50.532+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:22.398445+0000 front 2024-09-11T22:08:22.398495+0000 (oldest deadline 2024-09-11T22:08:47.098448+0000) 2024-09-11T22:08:51.009 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 11 22:08:50 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-5[47459]: 2024-09-11T22:08:50.962+0000 7ff94e3fd640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.528685+0000 front 2024-09-11T22:08:16.528663+0000 (oldest deadline 2024-09-11T22:08:41.828661+0000) 2024-09-11T22:08:51.010 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 11 22:08:50 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-5[47459]: 2024-09-11T22:08:50.962+0000 7ff94e3fd640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:23.529853+0000 front 2024-09-11T22:08:23.530033+0000 (oldest deadline 2024-09-11T22:08:47.029940+0000) 2024-09-11T22:08:51.010 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:08:50 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:08:50.730+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.893190+0000 front 2024-09-11T22:08:16.893259+0000 (oldest deadline 2024-09-11T22:08:39.193066+0000) 2024-09-11T22:08:51.010 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:08:50 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:08:50.730+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:21.494266+0000 front 2024-09-11T22:08:21.494360+0000 (oldest deadline 2024-09-11T22:08:47.394416+0000) 2024-09-11T22:08:51.342 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:08:51 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:08:51.008+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:26.210992+0000 front 2024-09-11T22:08:26.211085+0000 (oldest deadline 2024-09-11T22:08:50.310996+0000) 2024-09-11T22:08:51.443 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 11 22:08:50 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-6[55973]: 2024-09-11T22:08:50.985+0000 7f9c8be0c640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.487210+0000 front 2024-09-11T22:08:17.487317+0000 (oldest deadline 2024-09-11T22:08:42.187568+0000) 2024-09-11T22:08:51.469 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:08:51 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:08:51.151+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.481596+0000 front 2024-09-11T22:08:17.481499+0000 (oldest deadline 2024-09-11T22:08:42.181347+0000) 2024-09-11T22:08:51.469 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 11 22:08:51 smithi053 podman[70423]: 2024-09-11 22:08:51.172069586 +0000 UTC m=+5.577195644 container died bff20c95cd68ca63209afefb7b6ee3dcdf145754c9852db4209c42d8b94c5f24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-4, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T22:08:51.469 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 22:08:51 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-7[52494]: 2024-09-11T22:08:51.298+0000 7f29f1026640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:25.674626+0000 front 2024-09-11T22:08:25.674669+0000 (oldest deadline 2024-09-11T22:08:49.174727+0000) 2024-09-11T22:08:51.761 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 11 22:08:51 smithi053 podman[70423]: 2024-09-11 22:08:51.56823421 +0000 UTC m=+5.973360239 container cleanup bff20c95cd68ca63209afefb7b6ee3dcdf145754c9852db4209c42d8b94c5f24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-11T22:08:51.762 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 11 22:08:51 smithi053 bash[70423]: ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-4 2024-09-11T22:08:51.943 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:08:51 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:08:51.568+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.497679+0000 front 2024-09-11T22:08:16.497606+0000 (oldest deadline 2024-09-11T22:08:42.397744+0000) 2024-09-11T22:08:51.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:08:51 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:08:51.569+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:22.398445+0000 front 2024-09-11T22:08:22.398495+0000 (oldest deadline 2024-09-11T22:08:47.098448+0000) 2024-09-11T22:08:52.026 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 11 22:08:51 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-5[47459]: 2024-09-11T22:08:51.927+0000 7ff94e3fd640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.528685+0000 front 2024-09-11T22:08:16.528663+0000 (oldest deadline 2024-09-11T22:08:41.828661+0000) 2024-09-11T22:08:52.027 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 11 22:08:51 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-5[47459]: 2024-09-11T22:08:51.927+0000 7ff94e3fd640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:23.529853+0000 front 2024-09-11T22:08:23.530033+0000 (oldest deadline 2024-09-11T22:08:47.029940+0000) 2024-09-11T22:08:52.027 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:08:51 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:08:51.728+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.893190+0000 front 2024-09-11T22:08:16.893259+0000 (oldest deadline 2024-09-11T22:08:39.193066+0000) 2024-09-11T22:08:52.027 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:08:51 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:08:51.728+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:21.494266+0000 front 2024-09-11T22:08:21.494360+0000 (oldest deadline 2024-09-11T22:08:47.394416+0000) 2024-09-11T22:08:52.047 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 11 22:08:51 smithi053 podman[70438]: 2024-09-11 22:08:51.761346104 +0000 UTC m=+0.587006214 container remove bff20c95cd68ca63209afefb7b6ee3dcdf145754c9852db4209c42d8b94c5f24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-4, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-11T22:08:52.342 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:08:52 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:08:52.025+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:26.210992+0000 front 2024-09-11T22:08:26.211085+0000 (oldest deadline 2024-09-11T22:08:50.310996+0000) 2024-09-11T22:08:52.419 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:08:52 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:08:52.117+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.481596+0000 front 2024-09-11T22:08:17.481499+0000 (oldest deadline 2024-09-11T22:08:42.181347+0000) 2024-09-11T22:08:52.420 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 22:08:52 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-7[52494]: 2024-09-11T22:08:52.327+0000 7f29f1026640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:25.674626+0000 front 2024-09-11T22:08:25.674669+0000 (oldest deadline 2024-09-11T22:08:49.174727+0000) 2024-09-11T22:08:52.420 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 11 22:08:52 smithi053 podman[70530]: 2024-09-11 22:08:52.071413706 +0000 UTC m=+0.015750735 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T22:08:52.420 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 11 22:08:52 smithi053 podman[70530]: 2024-09-11 22:08:52.179070654 +0000 UTC m=+0.123407668 container create 58ba7fdff6949e1fabecc3456c169715bd6840a828e81a7f502a656978abe5bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-4-deactivate, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-11T22:08:52.443 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 11 22:08:51 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-6[55973]: 2024-09-11T22:08:51.997+0000 7f9c8be0c640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.487210+0000 front 2024-09-11T22:08:17.487317+0000 (oldest deadline 2024-09-11T22:08:42.187568+0000) 2024-09-11T22:08:52.443 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 11 22:08:51 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-6[55973]: 2024-09-11T22:08:51.997+0000 7f9c8be0c640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:26.288972+0000 front 2024-09-11T22:08:26.288673+0000 (oldest deadline 2024-09-11T22:08:50.988740+0000) 2024-09-11T22:08:52.738 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 11 22:08:52 smithi053 podman[70530]: 2024-09-11 22:08:52.548663638 +0000 UTC m=+0.493000655 container init 58ba7fdff6949e1fabecc3456c169715bd6840a828e81a7f502a656978abe5bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-4-deactivate, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-11T22:08:52.738 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 11 22:08:52 smithi053 podman[70530]: 2024-09-11 22:08:52.551976111 +0000 UTC m=+0.496313126 container start 58ba7fdff6949e1fabecc3456c169715bd6840a828e81a7f502a656978abe5bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-11T22:08:52.739 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 11 22:08:52 smithi053 podman[70530]: 2024-09-11 22:08:52.648698043 +0000 UTC m=+0.593035076 container attach 58ba7fdff6949e1fabecc3456c169715bd6840a828e81a7f502a656978abe5bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3) 2024-09-11T22:08:52.943 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:08:52 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:08:52.538+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.497679+0000 front 2024-09-11T22:08:16.497606+0000 (oldest deadline 2024-09-11T22:08:42.397744+0000) 2024-09-11T22:08:52.943 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:08:52 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:08:52.538+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:22.398445+0000 front 2024-09-11T22:08:22.398495+0000 (oldest deadline 2024-09-11T22:08:47.098448+0000) 2024-09-11T22:08:53.037 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 11 22:08:52 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-5[47459]: 2024-09-11T22:08:52.962+0000 7ff94e3fd640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.528685+0000 front 2024-09-11T22:08:16.528663+0000 (oldest deadline 2024-09-11T22:08:41.828661+0000) 2024-09-11T22:08:53.037 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 11 22:08:52 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-5[47459]: 2024-09-11T22:08:52.962+0000 7ff94e3fd640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:23.529853+0000 front 2024-09-11T22:08:23.530033+0000 (oldest deadline 2024-09-11T22:08:47.029940+0000) 2024-09-11T22:08:53.037 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:08:52 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:08:52.741+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.893190+0000 front 2024-09-11T22:08:16.893259+0000 (oldest deadline 2024-09-11T22:08:39.193066+0000) 2024-09-11T22:08:53.038 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:08:52 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:08:52.741+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:21.494266+0000 front 2024-09-11T22:08:21.494360+0000 (oldest deadline 2024-09-11T22:08:47.394416+0000) 2024-09-11T22:08:53.047 INFO:journalctl@ceph.osd.4.smithi053.stdout:Sep 11 22:08:52 smithi053 podman[70530]: 2024-09-11 22:08:52.738855122 +0000 UTC m=+0.683192138 container died 58ba7fdff6949e1fabecc3456c169715bd6840a828e81a7f502a656978abe5bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-4-deactivate, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3) 2024-09-11T22:08:53.341 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:08:53 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:08:53.036+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:26.210992+0000 front 2024-09-11T22:08:26.211085+0000 (oldest deadline 2024-09-11T22:08:50.310996+0000) 2024-09-11T22:08:53.369 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 22:08:53 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-7[52494]: 2024-09-11T22:08:53.304+0000 7f29f1026640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:25.674626+0000 front 2024-09-11T22:08:25.674669+0000 (oldest deadline 2024-09-11T22:08:49.174727+0000) 2024-09-11T22:08:53.370 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:08:53 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:08:53.128+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.481596+0000 front 2024-09-11T22:08:17.481499+0000 (oldest deadline 2024-09-11T22:08:42.181347+0000) 2024-09-11T22:08:53.389 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3ffca83c-7087-11ef-bcea-c7b262605968@osd.4.service' 2024-09-11T22:08:53.443 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 11 22:08:52 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-6[55973]: 2024-09-11T22:08:52.997+0000 7f9c8be0c640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.487210+0000 front 2024-09-11T22:08:17.487317+0000 (oldest deadline 2024-09-11T22:08:42.187568+0000) 2024-09-11T22:08:53.443 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 11 22:08:52 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-6[55973]: 2024-09-11T22:08:52.997+0000 7f9c8be0c640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:26.288972+0000 front 2024-09-11T22:08:26.288673+0000 (oldest deadline 2024-09-11T22:08:50.988740+0000) 2024-09-11T22:08:53.474 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:08:53.474 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-11T22:08:53.474 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-11T22:08:53.475 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-3ffca83c-7087-11ef-bcea-c7b262605968@osd.5 2024-09-11T22:08:53.844 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 11 22:08:53 smithi161 systemd[1]: Stopping Ceph osd.5 for 3ffca83c-7087-11ef-bcea-c7b262605968... 2024-09-11T22:08:53.845 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:08:53 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:08:53.711+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.893190+0000 front 2024-09-11T22:08:16.893259+0000 (oldest deadline 2024-09-11T22:08:39.193066+0000) 2024-09-11T22:08:53.845 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:08:53 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:08:53.711+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:21.494266+0000 front 2024-09-11T22:08:21.494360+0000 (oldest deadline 2024-09-11T22:08:47.394416+0000) 2024-09-11T22:08:53.943 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:08:53 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:08:53.545+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.497679+0000 front 2024-09-11T22:08:16.497606+0000 (oldest deadline 2024-09-11T22:08:42.397744+0000) 2024-09-11T22:08:53.943 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:08:53 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:08:53.545+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:22.398445+0000 front 2024-09-11T22:08:22.398495+0000 (oldest deadline 2024-09-11T22:08:47.098448+0000) 2024-09-11T22:08:54.342 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:08:54 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:08:54.030+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:26.210992+0000 front 2024-09-11T22:08:26.211085+0000 (oldest deadline 2024-09-11T22:08:50.310996+0000) 2024-09-11T22:08:54.342 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 11 22:08:53 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-5[47459]: 2024-09-11T22:08:53.878+0000 7ff951de7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:08:54.342 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 11 22:08:53 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-5[47459]: 2024-09-11T22:08:53.878+0000 7ff951de7640 -1 osd.5 98 *** Got signal Terminated *** 2024-09-11T22:08:54.342 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 11 22:08:53 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-5[47459]: 2024-09-11T22:08:53.878+0000 7ff951de7640 -1 osd.5 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T22:08:54.342 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 11 22:08:53 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-5[47459]: 2024-09-11T22:08:53.986+0000 7ff94e3fd640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.528685+0000 front 2024-09-11T22:08:16.528663+0000 (oldest deadline 2024-09-11T22:08:41.828661+0000) 2024-09-11T22:08:54.343 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 11 22:08:53 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-5[47459]: 2024-09-11T22:08:53.986+0000 7ff94e3fd640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:23.529853+0000 front 2024-09-11T22:08:23.530033+0000 (oldest deadline 2024-09-11T22:08:47.029940+0000) 2024-09-11T22:08:54.443 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 11 22:08:53 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-6[55973]: 2024-09-11T22:08:53.978+0000 7f9c8be0c640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.487210+0000 front 2024-09-11T22:08:17.487317+0000 (oldest deadline 2024-09-11T22:08:42.187568+0000) 2024-09-11T22:08:54.443 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 11 22:08:53 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-6[55973]: 2024-09-11T22:08:53.978+0000 7f9c8be0c640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:26.288972+0000 front 2024-09-11T22:08:26.288673+0000 (oldest deadline 2024-09-11T22:08:50.988740+0000) 2024-09-11T22:08:54.547 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 22:08:54 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-7[52494]: 2024-09-11T22:08:54.264+0000 7f29f1026640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:25.674626+0000 front 2024-09-11T22:08:25.674669+0000 (oldest deadline 2024-09-11T22:08:49.174727+0000) 2024-09-11T22:08:54.548 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:08:54 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:08:54.157+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.481596+0000 front 2024-09-11T22:08:17.481499+0000 (oldest deadline 2024-09-11T22:08:42.181347+0000) 2024-09-11T22:08:54.937 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:08:54 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:08:54.550+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.497679+0000 front 2024-09-11T22:08:16.497606+0000 (oldest deadline 2024-09-11T22:08:42.397744+0000) 2024-09-11T22:08:54.937 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:08:54 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:08:54.550+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:22.398445+0000 front 2024-09-11T22:08:22.398495+0000 (oldest deadline 2024-09-11T22:08:47.098448+0000) 2024-09-11T22:08:54.978 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:08:54 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:08:54.684+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.893190+0000 front 2024-09-11T22:08:16.893259+0000 (oldest deadline 2024-09-11T22:08:39.193066+0000) 2024-09-11T22:08:54.979 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:08:54 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:08:54.684+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:21.494266+0000 front 2024-09-11T22:08:21.494360+0000 (oldest deadline 2024-09-11T22:08:47.394416+0000) 2024-09-11T22:08:55.193 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 11 22:08:54 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-6[55973]: 2024-09-11T22:08:54.936+0000 7f9c8be0c640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.487210+0000 front 2024-09-11T22:08:17.487317+0000 (oldest deadline 2024-09-11T22:08:42.187568+0000) 2024-09-11T22:08:55.193 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 11 22:08:54 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-6[55973]: 2024-09-11T22:08:54.936+0000 7f9c8be0c640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:26.288972+0000 front 2024-09-11T22:08:26.288673+0000 (oldest deadline 2024-09-11T22:08:50.988740+0000) 2024-09-11T22:08:55.342 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:08:54 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:08:54.987+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:26.210992+0000 front 2024-09-11T22:08:26.211085+0000 (oldest deadline 2024-09-11T22:08:50.310996+0000) 2024-09-11T22:08:55.342 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 11 22:08:54 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-5[47459]: 2024-09-11T22:08:54.978+0000 7ff94e3fd640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.528685+0000 front 2024-09-11T22:08:16.528663+0000 (oldest deadline 2024-09-11T22:08:41.828661+0000) 2024-09-11T22:08:55.342 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 11 22:08:54 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-5[47459]: 2024-09-11T22:08:54.978+0000 7ff94e3fd640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:23.529853+0000 front 2024-09-11T22:08:23.530033+0000 (oldest deadline 2024-09-11T22:08:47.029940+0000) 2024-09-11T22:08:55.547 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 22:08:55 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-7[52494]: 2024-09-11T22:08:55.246+0000 7f29f1026640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:25.674626+0000 front 2024-09-11T22:08:25.674669+0000 (oldest deadline 2024-09-11T22:08:49.174727+0000) 2024-09-11T22:08:55.548 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:08:55 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:08:55.163+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.481596+0000 front 2024-09-11T22:08:17.481499+0000 (oldest deadline 2024-09-11T22:08:42.181347+0000) 2024-09-11T22:08:55.935 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:08:55 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:08:55.643+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.893190+0000 front 2024-09-11T22:08:16.893259+0000 (oldest deadline 2024-09-11T22:08:39.193066+0000) 2024-09-11T22:08:55.935 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:08:55 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:08:55.643+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:21.494266+0000 front 2024-09-11T22:08:21.494360+0000 (oldest deadline 2024-09-11T22:08:47.394416+0000) 2024-09-11T22:08:55.943 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:08:55 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:08:55.523+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.497679+0000 front 2024-09-11T22:08:16.497606+0000 (oldest deadline 2024-09-11T22:08:42.397744+0000) 2024-09-11T22:08:55.943 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:08:55 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:08:55.523+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:22.398445+0000 front 2024-09-11T22:08:22.398495+0000 (oldest deadline 2024-09-11T22:08:47.098448+0000) 2024-09-11T22:08:56.341 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 11 22:08:55 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-5[47459]: 2024-09-11T22:08:55.934+0000 7ff94e3fd640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.528685+0000 front 2024-09-11T22:08:16.528663+0000 (oldest deadline 2024-09-11T22:08:41.828661+0000) 2024-09-11T22:08:56.342 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 11 22:08:55 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-5[47459]: 2024-09-11T22:08:55.934+0000 7ff94e3fd640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:23.529853+0000 front 2024-09-11T22:08:23.530033+0000 (oldest deadline 2024-09-11T22:08:47.029940+0000) 2024-09-11T22:08:56.342 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:08:55 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:08:55.962+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:26.210992+0000 front 2024-09-11T22:08:26.211085+0000 (oldest deadline 2024-09-11T22:08:50.310996+0000) 2024-09-11T22:08:56.443 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 11 22:08:55 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-6[55973]: 2024-09-11T22:08:55.955+0000 7f9c8be0c640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.487210+0000 front 2024-09-11T22:08:17.487317+0000 (oldest deadline 2024-09-11T22:08:42.187568+0000) 2024-09-11T22:08:56.443 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 11 22:08:55 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-6[55973]: 2024-09-11T22:08:55.955+0000 7f9c8be0c640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:26.288972+0000 front 2024-09-11T22:08:26.288673+0000 (oldest deadline 2024-09-11T22:08:50.988740+0000) 2024-09-11T22:08:56.547 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:08:56 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:08:56.117+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.481596+0000 front 2024-09-11T22:08:17.481499+0000 (oldest deadline 2024-09-11T22:08:42.181347+0000) 2024-09-11T22:08:56.548 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:08:56 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:08:56.117+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:31.383481+0000 front 2024-09-11T22:08:31.383448+0000 (oldest deadline 2024-09-11T22:08:56.083565+0000) 2024-09-11T22:08:56.548 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 22:08:56 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-7[52494]: 2024-09-11T22:08:56.262+0000 7f29f1026640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:25.674626+0000 front 2024-09-11T22:08:25.674669+0000 (oldest deadline 2024-09-11T22:08:49.174727+0000) 2024-09-11T22:08:56.548 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 22:08:56 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-7[52494]: 2024-09-11T22:08:56.262+0000 7f29f1026640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:31.976401+0000 front 2024-09-11T22:08:31.976263+0000 (oldest deadline 2024-09-11T22:08:55.377028+0000) 2024-09-11T22:08:56.913 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:08:56 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:08:56.608+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.893190+0000 front 2024-09-11T22:08:16.893259+0000 (oldest deadline 2024-09-11T22:08:39.193066+0000) 2024-09-11T22:08:56.913 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:08:56 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:08:56.608+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:21.494266+0000 front 2024-09-11T22:08:21.494360+0000 (oldest deadline 2024-09-11T22:08:47.394416+0000) 2024-09-11T22:08:56.937 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:08:56 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:08:56.473+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.497679+0000 front 2024-09-11T22:08:16.497606+0000 (oldest deadline 2024-09-11T22:08:42.397744+0000) 2024-09-11T22:08:56.937 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:08:56 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:08:56.473+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:22.398445+0000 front 2024-09-11T22:08:22.398495+0000 (oldest deadline 2024-09-11T22:08:47.098448+0000) 2024-09-11T22:08:57.193 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 11 22:08:56 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-6[55973]: 2024-09-11T22:08:56.936+0000 7f9c8be0c640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.487210+0000 front 2024-09-11T22:08:17.487317+0000 (oldest deadline 2024-09-11T22:08:42.187568+0000) 2024-09-11T22:08:57.193 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 11 22:08:56 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-6[55973]: 2024-09-11T22:08:56.936+0000 7f9c8be0c640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:26.288972+0000 front 2024-09-11T22:08:26.288673+0000 (oldest deadline 2024-09-11T22:08:50.988740+0000) 2024-09-11T22:08:57.193 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 11 22:08:56 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-6[55973]: 2024-09-11T22:08:56.936+0000 7f9c8be0c640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:34.490028+0000 front 2024-09-11T22:08:34.489988+0000 (oldest deadline 2024-09-11T22:08:56.789739+0000) 2024-09-11T22:08:57.342 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:08:56 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:08:56.920+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:26.210992+0000 front 2024-09-11T22:08:26.211085+0000 (oldest deadline 2024-09-11T22:08:50.310996+0000) 2024-09-11T22:08:57.342 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 11 22:08:56 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-5[47459]: 2024-09-11T22:08:56.912+0000 7ff94e3fd640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.528685+0000 front 2024-09-11T22:08:16.528663+0000 (oldest deadline 2024-09-11T22:08:41.828661+0000) 2024-09-11T22:08:57.342 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 11 22:08:56 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-5[47459]: 2024-09-11T22:08:56.912+0000 7ff94e3fd640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:23.529853+0000 front 2024-09-11T22:08:23.530033+0000 (oldest deadline 2024-09-11T22:08:47.029940+0000) 2024-09-11T22:08:57.547 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 22:08:57 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-7[52494]: 2024-09-11T22:08:57.239+0000 7f29f1026640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:25.674626+0000 front 2024-09-11T22:08:25.674669+0000 (oldest deadline 2024-09-11T22:08:49.174727+0000) 2024-09-11T22:08:57.548 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 22:08:57 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-7[52494]: 2024-09-11T22:08:57.239+0000 7f29f1026640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:31.976401+0000 front 2024-09-11T22:08:31.976263+0000 (oldest deadline 2024-09-11T22:08:55.377028+0000) 2024-09-11T22:08:57.548 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:08:57 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:08:57.162+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.481596+0000 front 2024-09-11T22:08:17.481499+0000 (oldest deadline 2024-09-11T22:08:42.181347+0000) 2024-09-11T22:08:57.548 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:08:57 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:08:57.162+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:31.383481+0000 front 2024-09-11T22:08:31.383448+0000 (oldest deadline 2024-09-11T22:08:56.083565+0000) 2024-09-11T22:08:57.901 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:08:57 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:08:57.604+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.893190+0000 front 2024-09-11T22:08:16.893259+0000 (oldest deadline 2024-09-11T22:08:39.193066+0000) 2024-09-11T22:08:57.901 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:08:57 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:08:57.604+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:21.494266+0000 front 2024-09-11T22:08:21.494360+0000 (oldest deadline 2024-09-11T22:08:47.394416+0000) 2024-09-11T22:08:57.901 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:08:57 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:08:57.604+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:33.295337+0000 front 2024-09-11T22:08:33.295289+0000 (oldest deadline 2024-09-11T22:08:56.795428+0000) 2024-09-11T22:08:57.943 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:08:57 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:08:57.462+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.497679+0000 front 2024-09-11T22:08:16.497606+0000 (oldest deadline 2024-09-11T22:08:42.397744+0000) 2024-09-11T22:08:57.943 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:08:57 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:08:57.462+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:22.398445+0000 front 2024-09-11T22:08:22.398495+0000 (oldest deadline 2024-09-11T22:08:47.098448+0000) 2024-09-11T22:08:58.342 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:08:57 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:08:57.926+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:26.210992+0000 front 2024-09-11T22:08:26.211085+0000 (oldest deadline 2024-09-11T22:08:50.310996+0000) 2024-09-11T22:08:58.342 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 11 22:08:57 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-5[47459]: 2024-09-11T22:08:57.900+0000 7ff94e3fd640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.528685+0000 front 2024-09-11T22:08:16.528663+0000 (oldest deadline 2024-09-11T22:08:41.828661+0000) 2024-09-11T22:08:58.342 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 11 22:08:57 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-5[47459]: 2024-09-11T22:08:57.900+0000 7ff94e3fd640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:23.529853+0000 front 2024-09-11T22:08:23.530033+0000 (oldest deadline 2024-09-11T22:08:47.029940+0000) 2024-09-11T22:08:58.342 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 11 22:08:57 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-5[47459]: 2024-09-11T22:08:57.900+0000 7ff94e3fd640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:32.831667+0000 front 2024-09-11T22:08:32.831608+0000 (oldest deadline 2024-09-11T22:08:57.531690+0000) 2024-09-11T22:08:58.443 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 11 22:08:57 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-6[55973]: 2024-09-11T22:08:57.979+0000 7f9c8be0c640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.487210+0000 front 2024-09-11T22:08:17.487317+0000 (oldest deadline 2024-09-11T22:08:42.187568+0000) 2024-09-11T22:08:58.444 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 11 22:08:57 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-6[55973]: 2024-09-11T22:08:57.979+0000 7f9c8be0c640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:26.288972+0000 front 2024-09-11T22:08:26.288673+0000 (oldest deadline 2024-09-11T22:08:50.988740+0000) 2024-09-11T22:08:58.444 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 11 22:08:57 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-6[55973]: 2024-09-11T22:08:57.979+0000 7f9c8be0c640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:34.490028+0000 front 2024-09-11T22:08:34.489988+0000 (oldest deadline 2024-09-11T22:08:56.789739+0000) 2024-09-11T22:08:58.547 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 22:08:58 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-7[52494]: 2024-09-11T22:08:58.286+0000 7f29f1026640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:25.674626+0000 front 2024-09-11T22:08:25.674669+0000 (oldest deadline 2024-09-11T22:08:49.174727+0000) 2024-09-11T22:08:58.547 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 22:08:58 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-7[52494]: 2024-09-11T22:08:58.286+0000 7f29f1026640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:31.976401+0000 front 2024-09-11T22:08:31.976263+0000 (oldest deadline 2024-09-11T22:08:55.377028+0000) 2024-09-11T22:08:58.548 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:08:58 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:08:58.172+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.481596+0000 front 2024-09-11T22:08:17.481499+0000 (oldest deadline 2024-09-11T22:08:42.181347+0000) 2024-09-11T22:08:58.548 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:08:58 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:08:58.172+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:31.383481+0000 front 2024-09-11T22:08:31.383448+0000 (oldest deadline 2024-09-11T22:08:56.083565+0000) 2024-09-11T22:08:58.842 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:08:58 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:08:58.563+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.893190+0000 front 2024-09-11T22:08:16.893259+0000 (oldest deadline 2024-09-11T22:08:39.193066+0000) 2024-09-11T22:08:58.842 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:08:58 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:08:58.563+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:21.494266+0000 front 2024-09-11T22:08:21.494360+0000 (oldest deadline 2024-09-11T22:08:47.394416+0000) 2024-09-11T22:08:58.843 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:08:58 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:08:58.563+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:33.295337+0000 front 2024-09-11T22:08:33.295289+0000 (oldest deadline 2024-09-11T22:08:56.795428+0000) 2024-09-11T22:08:58.943 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:08:58 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:08:58.490+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.497679+0000 front 2024-09-11T22:08:16.497606+0000 (oldest deadline 2024-09-11T22:08:42.397744+0000) 2024-09-11T22:08:58.943 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:08:58 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:08:58.490+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:22.398445+0000 front 2024-09-11T22:08:22.398495+0000 (oldest deadline 2024-09-11T22:08:47.098448+0000) 2024-09-11T22:08:58.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:08:58 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:08:58.490+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:32.999892+0000 front 2024-09-11T22:08:32.999692+0000 (oldest deadline 2024-09-11T22:08:58.299748+0000) 2024-09-11T22:08:59.342 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:08:58 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:08:58.975+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:26.210992+0000 front 2024-09-11T22:08:26.211085+0000 (oldest deadline 2024-09-11T22:08:50.310996+0000) 2024-09-11T22:08:59.443 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 11 22:08:58 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-6[55973]: 2024-09-11T22:08:58.949+0000 7f9c8be0c640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.487210+0000 front 2024-09-11T22:08:17.487317+0000 (oldest deadline 2024-09-11T22:08:42.187568+0000) 2024-09-11T22:08:59.444 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 11 22:08:58 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-6[55973]: 2024-09-11T22:08:58.949+0000 7f9c8be0c640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:26.288972+0000 front 2024-09-11T22:08:26.288673+0000 (oldest deadline 2024-09-11T22:08:50.988740+0000) 2024-09-11T22:08:59.444 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 11 22:08:58 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-6[55973]: 2024-09-11T22:08:58.949+0000 7f9c8be0c640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:34.490028+0000 front 2024-09-11T22:08:34.489988+0000 (oldest deadline 2024-09-11T22:08:56.789739+0000) 2024-09-11T22:08:59.547 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:08:59 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:08:59.221+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.481596+0000 front 2024-09-11T22:08:17.481499+0000 (oldest deadline 2024-09-11T22:08:42.181347+0000) 2024-09-11T22:08:59.548 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:08:59 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:08:59.221+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:31.383481+0000 front 2024-09-11T22:08:31.383448+0000 (oldest deadline 2024-09-11T22:08:56.083565+0000) 2024-09-11T22:08:59.548 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 22:08:59 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-7[52494]: 2024-09-11T22:08:59.238+0000 7f29f1026640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:25.674626+0000 front 2024-09-11T22:08:25.674669+0000 (oldest deadline 2024-09-11T22:08:49.174727+0000) 2024-09-11T22:08:59.548 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 22:08:59 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-7[52494]: 2024-09-11T22:08:59.238+0000 7f29f1026640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:31.976401+0000 front 2024-09-11T22:08:31.976263+0000 (oldest deadline 2024-09-11T22:08:55.377028+0000) 2024-09-11T22:08:59.646 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 11 22:08:59 smithi161 podman[71877]: 2024-09-11 22:08:59.391729968 +0000 UTC m=+5.720807658 container died 3c0b50492934954f15e4408ad0e6b8a94ebc8ebb0c2e28714a2ab09ecda7fa92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T22:08:59.646 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:08:59 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:08:59.526+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.893190+0000 front 2024-09-11T22:08:16.893259+0000 (oldest deadline 2024-09-11T22:08:39.193066+0000) 2024-09-11T22:08:59.647 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:08:59 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:08:59.526+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:21.494266+0000 front 2024-09-11T22:08:21.494360+0000 (oldest deadline 2024-09-11T22:08:47.394416+0000) 2024-09-11T22:08:59.647 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:08:59 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:08:59.526+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:33.295337+0000 front 2024-09-11T22:08:33.295289+0000 (oldest deadline 2024-09-11T22:08:56.795428+0000) 2024-09-11T22:08:59.926 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:08:59 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:08:59.516+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.497679+0000 front 2024-09-11T22:08:16.497606+0000 (oldest deadline 2024-09-11T22:08:42.397744+0000) 2024-09-11T22:08:59.926 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:08:59 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:08:59.516+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:22.398445+0000 front 2024-09-11T22:08:22.398495+0000 (oldest deadline 2024-09-11T22:08:47.098448+0000) 2024-09-11T22:08:59.926 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:08:59 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:08:59.516+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:32.999892+0000 front 2024-09-11T22:08:32.999692+0000 (oldest deadline 2024-09-11T22:08:58.299748+0000) 2024-09-11T22:09:00.006 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 11 22:08:59 smithi161 podman[71877]: 2024-09-11 22:08:59.744291772 +0000 UTC m=+6.073369467 container cleanup 3c0b50492934954f15e4408ad0e6b8a94ebc8ebb0c2e28714a2ab09ecda7fa92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-5, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-11T22:09:00.007 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 11 22:08:59 smithi161 bash[71877]: ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-5 2024-09-11T22:09:00.193 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 11 22:08:59 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-6[55973]: 2024-09-11T22:08:59.925+0000 7f9c8be0c640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.487210+0000 front 2024-09-11T22:08:17.487317+0000 (oldest deadline 2024-09-11T22:08:42.187568+0000) 2024-09-11T22:09:00.193 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 11 22:08:59 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-6[55973]: 2024-09-11T22:08:59.925+0000 7f9c8be0c640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:26.288972+0000 front 2024-09-11T22:08:26.288673+0000 (oldest deadline 2024-09-11T22:08:50.988740+0000) 2024-09-11T22:09:00.193 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 11 22:08:59 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-6[55973]: 2024-09-11T22:08:59.925+0000 7f9c8be0c640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:34.490028+0000 front 2024-09-11T22:08:34.489988+0000 (oldest deadline 2024-09-11T22:08:56.789739+0000) 2024-09-11T22:09:00.342 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 11 22:09:00 smithi161 podman[71892]: 2024-09-11 22:09:00.020557872 +0000 UTC m=+0.623006271 container remove 3c0b50492934954f15e4408ad0e6b8a94ebc8ebb0c2e28714a2ab09ecda7fa92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-5, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T22:09:00.342 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:00 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:00.006+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:26.210992+0000 front 2024-09-11T22:08:26.211085+0000 (oldest deadline 2024-09-11T22:08:50.310996+0000) 2024-09-11T22:09:00.342 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:00 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:00.006+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:30.311301+0000 front 2024-09-11T22:08:30.311235+0000 (oldest deadline 2024-09-11T22:08:59.712086+0000) 2024-09-11T22:09:00.547 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 22:09:00 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-7[52494]: 2024-09-11T22:09:00.265+0000 7f29f1026640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:25.674626+0000 front 2024-09-11T22:08:25.674669+0000 (oldest deadline 2024-09-11T22:08:49.174727+0000) 2024-09-11T22:09:00.548 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 22:09:00 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-7[52494]: 2024-09-11T22:09:00.265+0000 7f29f1026640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:31.976401+0000 front 2024-09-11T22:08:31.976263+0000 (oldest deadline 2024-09-11T22:08:55.377028+0000) 2024-09-11T22:09:00.548 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:00 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:00.255+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.481596+0000 front 2024-09-11T22:08:17.481499+0000 (oldest deadline 2024-09-11T22:08:42.181347+0000) 2024-09-11T22:09:00.548 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:00 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:00.255+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:31.383481+0000 front 2024-09-11T22:08:31.383448+0000 (oldest deadline 2024-09-11T22:08:56.083565+0000) 2024-09-11T22:09:00.616 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 11 22:09:00 smithi161 podman[71978]: 2024-09-11 22:09:00.261300545 +0000 UTC m=+0.016056909 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T22:09:00.616 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 11 22:09:00 smithi161 podman[71978]: 2024-09-11 22:09:00.396638499 +0000 UTC m=+0.151394894 container create acf08793ac97d30559732e2783a7230c264c8145f02c677a27750744d4d4e61e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-5-deactivate, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T22:09:00.616 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:00 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:00.531+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.893190+0000 front 2024-09-11T22:08:16.893259+0000 (oldest deadline 2024-09-11T22:08:39.193066+0000) 2024-09-11T22:09:00.616 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:00 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:00.531+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:21.494266+0000 front 2024-09-11T22:08:21.494360+0000 (oldest deadline 2024-09-11T22:08:47.394416+0000) 2024-09-11T22:09:00.617 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:00 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:00.531+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:33.295337+0000 front 2024-09-11T22:08:33.295289+0000 (oldest deadline 2024-09-11T22:08:56.795428+0000) 2024-09-11T22:09:00.943 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:00 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:00.562+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.497679+0000 front 2024-09-11T22:08:16.497606+0000 (oldest deadline 2024-09-11T22:08:42.397744+0000) 2024-09-11T22:09:00.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:00 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:00.562+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:22.398445+0000 front 2024-09-11T22:08:22.398495+0000 (oldest deadline 2024-09-11T22:08:47.098448+0000) 2024-09-11T22:09:00.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:00 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:00.562+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:32.999892+0000 front 2024-09-11T22:08:32.999692+0000 (oldest deadline 2024-09-11T22:08:58.299748+0000) 2024-09-11T22:09:01.017 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 11 22:09:00 smithi161 podman[71978]: 2024-09-11 22:09:00.615780886 +0000 UTC m=+0.370537239 container init acf08793ac97d30559732e2783a7230c264c8145f02c677a27750744d4d4e61e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-5-deactivate, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-11T22:09:01.018 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 11 22:09:00 smithi161 podman[71978]: 2024-09-11 22:09:00.619104937 +0000 UTC m=+0.373861298 container start acf08793ac97d30559732e2783a7230c264c8145f02c677a27750744d4d4e61e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-11T22:09:01.018 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 11 22:09:00 smithi161 podman[71978]: 2024-09-11 22:09:00.707235711 +0000 UTC m=+0.461992064 container attach acf08793ac97d30559732e2783a7230c264c8145f02c677a27750744d4d4e61e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-11T22:09:01.018 INFO:journalctl@ceph.osd.5.smithi161.stdout:Sep 11 22:09:00 smithi161 podman[71978]: 2024-09-11 22:09:00.803672788 +0000 UTC m=+0.558429158 container died acf08793ac97d30559732e2783a7230c264c8145f02c677a27750744d4d4e61e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-11T22:09:01.341 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:01 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:01.017+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:26.210992+0000 front 2024-09-11T22:08:26.211085+0000 (oldest deadline 2024-09-11T22:08:50.310996+0000) 2024-09-11T22:09:01.342 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:01 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:01.017+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:30.311301+0000 front 2024-09-11T22:08:30.311235+0000 (oldest deadline 2024-09-11T22:08:59.712086+0000) 2024-09-11T22:09:01.443 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 11 22:09:00 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-6[55973]: 2024-09-11T22:09:00.949+0000 7f9c8be0c640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.487210+0000 front 2024-09-11T22:08:17.487317+0000 (oldest deadline 2024-09-11T22:08:42.187568+0000) 2024-09-11T22:09:01.443 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 11 22:09:00 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-6[55973]: 2024-09-11T22:09:00.949+0000 7f9c8be0c640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:26.288972+0000 front 2024-09-11T22:08:26.288673+0000 (oldest deadline 2024-09-11T22:08:50.988740+0000) 2024-09-11T22:09:01.443 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 11 22:09:00 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-6[55973]: 2024-09-11T22:09:00.949+0000 7f9c8be0c640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:34.490028+0000 front 2024-09-11T22:08:34.489988+0000 (oldest deadline 2024-09-11T22:08:56.789739+0000) 2024-09-11T22:09:01.507 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3ffca83c-7087-11ef-bcea-c7b262605968@osd.5.service' 2024-09-11T22:09:01.546 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:09:01.547 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-11T22:09:01.547 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-11T22:09:01.547 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ceph-3ffca83c-7087-11ef-bcea-c7b262605968@osd.6 2024-09-11T22:09:01.548 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 22:09:01 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-7[52494]: 2024-09-11T22:09:01.289+0000 7f29f1026640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:25.674626+0000 front 2024-09-11T22:08:25.674669+0000 (oldest deadline 2024-09-11T22:08:49.174727+0000) 2024-09-11T22:09:01.548 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 22:09:01 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-7[52494]: 2024-09-11T22:09:01.289+0000 7f29f1026640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:31.976401+0000 front 2024-09-11T22:08:31.976263+0000 (oldest deadline 2024-09-11T22:08:55.377028+0000) 2024-09-11T22:09:01.549 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:01 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:01.245+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.481596+0000 front 2024-09-11T22:08:17.481499+0000 (oldest deadline 2024-09-11T22:08:42.181347+0000) 2024-09-11T22:09:01.549 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:01 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:01.245+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:31.383481+0000 front 2024-09-11T22:08:31.383448+0000 (oldest deadline 2024-09-11T22:08:56.083565+0000) 2024-09-11T22:09:01.842 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:01 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:01.580+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.893190+0000 front 2024-09-11T22:08:16.893259+0000 (oldest deadline 2024-09-11T22:08:39.193066+0000) 2024-09-11T22:09:01.842 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:01 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:01.580+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:21.494266+0000 front 2024-09-11T22:08:21.494360+0000 (oldest deadline 2024-09-11T22:08:47.394416+0000) 2024-09-11T22:09:01.842 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:01 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:01.580+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:33.295337+0000 front 2024-09-11T22:08:33.295289+0000 (oldest deadline 2024-09-11T22:08:56.795428+0000) 2024-09-11T22:09:01.866 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:01 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:01.585+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.497679+0000 front 2024-09-11T22:08:16.497606+0000 (oldest deadline 2024-09-11T22:08:42.397744+0000) 2024-09-11T22:09:01.866 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:01 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:01.585+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:22.398445+0000 front 2024-09-11T22:08:22.398495+0000 (oldest deadline 2024-09-11T22:08:47.098448+0000) 2024-09-11T22:09:01.866 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:01 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:01.585+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:32.999892+0000 front 2024-09-11T22:08:32.999692+0000 (oldest deadline 2024-09-11T22:08:58.299748+0000) 2024-09-11T22:09:01.867 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 11 22:09:01 smithi001 systemd[1]: Stopping Ceph osd.6 for 3ffca83c-7087-11ef-bcea-c7b262605968... 2024-09-11T22:09:01.867 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 11 22:09:01 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-6[55973]: 2024-09-11T22:09:01.865+0000 7f9c8f7f6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:09:02.193 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 11 22:09:01 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-6[55973]: 2024-09-11T22:09:01.865+0000 7f9c8f7f6640 -1 osd.6 98 *** Got signal Terminated *** 2024-09-11T22:09:02.194 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 11 22:09:01 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-6[55973]: 2024-09-11T22:09:01.865+0000 7f9c8f7f6640 -1 osd.6 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T22:09:02.194 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 11 22:09:01 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-6[55973]: 2024-09-11T22:09:01.967+0000 7f9c8be0c640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.487210+0000 front 2024-09-11T22:08:17.487317+0000 (oldest deadline 2024-09-11T22:08:42.187568+0000) 2024-09-11T22:09:02.194 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 11 22:09:01 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-6[55973]: 2024-09-11T22:09:01.967+0000 7f9c8be0c640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:26.288972+0000 front 2024-09-11T22:08:26.288673+0000 (oldest deadline 2024-09-11T22:08:50.988740+0000) 2024-09-11T22:09:02.194 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 11 22:09:01 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-6[55973]: 2024-09-11T22:09:01.967+0000 7f9c8be0c640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:34.490028+0000 front 2024-09-11T22:08:34.489988+0000 (oldest deadline 2024-09-11T22:08:56.789739+0000) 2024-09-11T22:09:02.342 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:02 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:02.016+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:26.210992+0000 front 2024-09-11T22:08:26.211085+0000 (oldest deadline 2024-09-11T22:08:50.310996+0000) 2024-09-11T22:09:02.342 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:02 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:02.016+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:30.311301+0000 front 2024-09-11T22:08:30.311235+0000 (oldest deadline 2024-09-11T22:08:59.712086+0000) 2024-09-11T22:09:02.547 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 22:09:02 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-7[52494]: 2024-09-11T22:09:02.257+0000 7f29f1026640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:25.674626+0000 front 2024-09-11T22:08:25.674669+0000 (oldest deadline 2024-09-11T22:08:49.174727+0000) 2024-09-11T22:09:02.548 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 22:09:02 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-7[52494]: 2024-09-11T22:09:02.257+0000 7f29f1026640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:31.976401+0000 front 2024-09-11T22:08:31.976263+0000 (oldest deadline 2024-09-11T22:08:55.377028+0000) 2024-09-11T22:09:02.548 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:02 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:02.252+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.481596+0000 front 2024-09-11T22:08:17.481499+0000 (oldest deadline 2024-09-11T22:08:42.181347+0000) 2024-09-11T22:09:02.548 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:02 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:02.252+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:31.383481+0000 front 2024-09-11T22:08:31.383448+0000 (oldest deadline 2024-09-11T22:08:56.083565+0000) 2024-09-11T22:09:02.943 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:02 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:02.635+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.497679+0000 front 2024-09-11T22:08:16.497606+0000 (oldest deadline 2024-09-11T22:08:42.397744+0000) 2024-09-11T22:09:02.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:02 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:02.635+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:22.398445+0000 front 2024-09-11T22:08:22.398495+0000 (oldest deadline 2024-09-11T22:08:47.098448+0000) 2024-09-11T22:09:02.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:02 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:02.635+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:32.999892+0000 front 2024-09-11T22:08:32.999692+0000 (oldest deadline 2024-09-11T22:08:58.299748+0000) 2024-09-11T22:09:03.048 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:02 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:02.623+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.893190+0000 front 2024-09-11T22:08:16.893259+0000 (oldest deadline 2024-09-11T22:08:39.193066+0000) 2024-09-11T22:09:03.048 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:02 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:02.623+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:21.494266+0000 front 2024-09-11T22:08:21.494360+0000 (oldest deadline 2024-09-11T22:08:47.394416+0000) 2024-09-11T22:09:03.049 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:02 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:02.623+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:33.295337+0000 front 2024-09-11T22:08:33.295289+0000 (oldest deadline 2024-09-11T22:08:56.795428+0000) 2024-09-11T22:09:03.342 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:03 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:03.048+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:26.210992+0000 front 2024-09-11T22:08:26.211085+0000 (oldest deadline 2024-09-11T22:08:50.310996+0000) 2024-09-11T22:09:03.342 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:03 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:03.048+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:30.311301+0000 front 2024-09-11T22:08:30.311235+0000 (oldest deadline 2024-09-11T22:08:59.712086+0000) 2024-09-11T22:09:03.443 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 11 22:09:02 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-6[55973]: 2024-09-11T22:09:02.982+0000 7f9c8be0c640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.487210+0000 front 2024-09-11T22:08:17.487317+0000 (oldest deadline 2024-09-11T22:08:42.187568+0000) 2024-09-11T22:09:03.444 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 11 22:09:02 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-6[55973]: 2024-09-11T22:09:02.982+0000 7f9c8be0c640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:26.288972+0000 front 2024-09-11T22:08:26.288673+0000 (oldest deadline 2024-09-11T22:08:50.988740+0000) 2024-09-11T22:09:03.444 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 11 22:09:02 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-6[55973]: 2024-09-11T22:09:02.982+0000 7f9c8be0c640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:34.490028+0000 front 2024-09-11T22:08:34.489988+0000 (oldest deadline 2024-09-11T22:08:56.789739+0000) 2024-09-11T22:09:03.547 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 22:09:03 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-7[52494]: 2024-09-11T22:09:03.235+0000 7f29f1026640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:25.674626+0000 front 2024-09-11T22:08:25.674669+0000 (oldest deadline 2024-09-11T22:08:49.174727+0000) 2024-09-11T22:09:03.548 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 22:09:03 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-7[52494]: 2024-09-11T22:09:03.235+0000 7f29f1026640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:31.976401+0000 front 2024-09-11T22:08:31.976263+0000 (oldest deadline 2024-09-11T22:08:55.377028+0000) 2024-09-11T22:09:03.548 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:03 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:03.265+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.481596+0000 front 2024-09-11T22:08:17.481499+0000 (oldest deadline 2024-09-11T22:08:42.181347+0000) 2024-09-11T22:09:03.548 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:03 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:03.265+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:31.383481+0000 front 2024-09-11T22:08:31.383448+0000 (oldest deadline 2024-09-11T22:08:56.083565+0000) 2024-09-11T22:09:03.943 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:03 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:03.615+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.497679+0000 front 2024-09-11T22:08:16.497606+0000 (oldest deadline 2024-09-11T22:08:42.397744+0000) 2024-09-11T22:09:03.943 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:03 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:03.615+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:22.398445+0000 front 2024-09-11T22:08:22.398495+0000 (oldest deadline 2024-09-11T22:08:47.098448+0000) 2024-09-11T22:09:03.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:03 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:03.615+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:32.999892+0000 front 2024-09-11T22:08:32.999692+0000 (oldest deadline 2024-09-11T22:08:58.299748+0000) 2024-09-11T22:09:03.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:03 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:03.615+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:40.000748+0000 front 2024-09-11T22:08:40.000701+0000 (oldest deadline 2024-09-11T22:09:03.500864+0000) 2024-09-11T22:09:04.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:03 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:03.598+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.893190+0000 front 2024-09-11T22:08:16.893259+0000 (oldest deadline 2024-09-11T22:08:39.193066+0000) 2024-09-11T22:09:04.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:03 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:03.598+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:21.494266+0000 front 2024-09-11T22:08:21.494360+0000 (oldest deadline 2024-09-11T22:08:47.394416+0000) 2024-09-11T22:09:04.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:03 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:03.598+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:33.295337+0000 front 2024-09-11T22:08:33.295289+0000 (oldest deadline 2024-09-11T22:08:56.795428+0000) 2024-09-11T22:09:04.443 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 11 22:09:04 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-6[55973]: 2024-09-11T22:09:04.017+0000 7f9c8be0c640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.487210+0000 front 2024-09-11T22:08:17.487317+0000 (oldest deadline 2024-09-11T22:08:42.187568+0000) 2024-09-11T22:09:04.443 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 11 22:09:04 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-6[55973]: 2024-09-11T22:09:04.017+0000 7f9c8be0c640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:26.288972+0000 front 2024-09-11T22:08:26.288673+0000 (oldest deadline 2024-09-11T22:08:50.988740+0000) 2024-09-11T22:09:04.444 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 11 22:09:04 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-6[55973]: 2024-09-11T22:09:04.017+0000 7f9c8be0c640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:34.490028+0000 front 2024-09-11T22:08:34.489988+0000 (oldest deadline 2024-09-11T22:08:56.789739+0000) 2024-09-11T22:09:04.547 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 22:09:04 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-7[52494]: 2024-09-11T22:09:04.262+0000 7f29f1026640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:25.674626+0000 front 2024-09-11T22:08:25.674669+0000 (oldest deadline 2024-09-11T22:08:49.174727+0000) 2024-09-11T22:09:04.548 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 22:09:04 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-7[52494]: 2024-09-11T22:09:04.262+0000 7f29f1026640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:31.976401+0000 front 2024-09-11T22:08:31.976263+0000 (oldest deadline 2024-09-11T22:08:55.377028+0000) 2024-09-11T22:09:04.548 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:04 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:04.239+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.481596+0000 front 2024-09-11T22:08:17.481499+0000 (oldest deadline 2024-09-11T22:08:42.181347+0000) 2024-09-11T22:09:04.548 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:04 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:04.239+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:31.383481+0000 front 2024-09-11T22:08:31.383448+0000 (oldest deadline 2024-09-11T22:08:56.083565+0000) 2024-09-11T22:09:04.548 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:04 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:04.239+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:38.384885+0000 front 2024-09-11T22:08:38.385010+0000 (oldest deadline 2024-09-11T22:09:03.684801+0000) 2024-09-11T22:09:04.592 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:04 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:04.093+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:26.210992+0000 front 2024-09-11T22:08:26.211085+0000 (oldest deadline 2024-09-11T22:08:50.310996+0000) 2024-09-11T22:09:04.592 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:04 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:04.093+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:30.311301+0000 front 2024-09-11T22:08:30.311235+0000 (oldest deadline 2024-09-11T22:08:59.712086+0000) 2024-09-11T22:09:04.592 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:04 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:04.093+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:39.712759+0000 front 2024-09-11T22:08:39.712634+0000 (oldest deadline 2024-09-11T22:09:03.812485+0000) 2024-09-11T22:09:04.943 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:04 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:04.574+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.497679+0000 front 2024-09-11T22:08:16.497606+0000 (oldest deadline 2024-09-11T22:08:42.397744+0000) 2024-09-11T22:09:04.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:04 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:04.574+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:22.398445+0000 front 2024-09-11T22:08:22.398495+0000 (oldest deadline 2024-09-11T22:08:47.098448+0000) 2024-09-11T22:09:04.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:04 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:04.574+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:32.999892+0000 front 2024-09-11T22:08:32.999692+0000 (oldest deadline 2024-09-11T22:08:58.299748+0000) 2024-09-11T22:09:04.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:04 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:04.574+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:40.000748+0000 front 2024-09-11T22:08:40.000701+0000 (oldest deadline 2024-09-11T22:09:03.500864+0000) 2024-09-11T22:09:05.069 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:04 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:04.616+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.893190+0000 front 2024-09-11T22:08:16.893259+0000 (oldest deadline 2024-09-11T22:08:39.193066+0000) 2024-09-11T22:09:05.070 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:04 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:04.616+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:21.494266+0000 front 2024-09-11T22:08:21.494360+0000 (oldest deadline 2024-09-11T22:08:47.394416+0000) 2024-09-11T22:09:05.070 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:04 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:04.616+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:33.295337+0000 front 2024-09-11T22:08:33.295289+0000 (oldest deadline 2024-09-11T22:08:56.795428+0000) 2024-09-11T22:09:05.342 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:05 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:05.069+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:26.210992+0000 front 2024-09-11T22:08:26.211085+0000 (oldest deadline 2024-09-11T22:08:50.310996+0000) 2024-09-11T22:09:05.342 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:05 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:05.069+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:30.311301+0000 front 2024-09-11T22:08:30.311235+0000 (oldest deadline 2024-09-11T22:08:59.712086+0000) 2024-09-11T22:09:05.342 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:05 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:05.069+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:39.712759+0000 front 2024-09-11T22:08:39.712634+0000 (oldest deadline 2024-09-11T22:09:03.812485+0000) 2024-09-11T22:09:05.443 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 11 22:09:04 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-6[55973]: 2024-09-11T22:09:04.996+0000 7f9c8be0c640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.487210+0000 front 2024-09-11T22:08:17.487317+0000 (oldest deadline 2024-09-11T22:08:42.187568+0000) 2024-09-11T22:09:05.443 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 11 22:09:04 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-6[55973]: 2024-09-11T22:09:04.996+0000 7f9c8be0c640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:26.288972+0000 front 2024-09-11T22:08:26.288673+0000 (oldest deadline 2024-09-11T22:08:50.988740+0000) 2024-09-11T22:09:05.443 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 11 22:09:04 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-6[55973]: 2024-09-11T22:09:04.996+0000 7f9c8be0c640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:34.490028+0000 front 2024-09-11T22:08:34.489988+0000 (oldest deadline 2024-09-11T22:08:56.789739+0000) 2024-09-11T22:09:05.444 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 11 22:09:04 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-6[55973]: 2024-09-11T22:09:04.996+0000 7f9c8be0c640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:40.290501+0000 front 2024-09-11T22:08:40.290470+0000 (oldest deadline 2024-09-11T22:09:04.390550+0000) 2024-09-11T22:09:05.547 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 22:09:05 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-7[52494]: 2024-09-11T22:09:05.256+0000 7f29f1026640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:25.674626+0000 front 2024-09-11T22:08:25.674669+0000 (oldest deadline 2024-09-11T22:08:49.174727+0000) 2024-09-11T22:09:05.547 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 22:09:05 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-7[52494]: 2024-09-11T22:09:05.256+0000 7f29f1026640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:31.976401+0000 front 2024-09-11T22:08:31.976263+0000 (oldest deadline 2024-09-11T22:08:55.377028+0000) 2024-09-11T22:09:05.548 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:05 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:05.277+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.481596+0000 front 2024-09-11T22:08:17.481499+0000 (oldest deadline 2024-09-11T22:08:42.181347+0000) 2024-09-11T22:09:05.548 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:05 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:05.277+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:31.383481+0000 front 2024-09-11T22:08:31.383448+0000 (oldest deadline 2024-09-11T22:08:56.083565+0000) 2024-09-11T22:09:05.548 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:05 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:05.277+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:38.384885+0000 front 2024-09-11T22:08:38.385010+0000 (oldest deadline 2024-09-11T22:09:03.684801+0000) 2024-09-11T22:09:05.943 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:05 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:05.544+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.497679+0000 front 2024-09-11T22:08:16.497606+0000 (oldest deadline 2024-09-11T22:08:42.397744+0000) 2024-09-11T22:09:05.943 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:05 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:05.544+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:22.398445+0000 front 2024-09-11T22:08:22.398495+0000 (oldest deadline 2024-09-11T22:08:47.098448+0000) 2024-09-11T22:09:05.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:05 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:05.544+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:32.999892+0000 front 2024-09-11T22:08:32.999692+0000 (oldest deadline 2024-09-11T22:08:58.299748+0000) 2024-09-11T22:09:05.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:05 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:05.544+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:40.000748+0000 front 2024-09-11T22:08:40.000701+0000 (oldest deadline 2024-09-11T22:09:03.500864+0000) 2024-09-11T22:09:06.046 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:05 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:05.620+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.893190+0000 front 2024-09-11T22:08:16.893259+0000 (oldest deadline 2024-09-11T22:08:39.193066+0000) 2024-09-11T22:09:06.046 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:05 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:05.620+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:21.494266+0000 front 2024-09-11T22:08:21.494360+0000 (oldest deadline 2024-09-11T22:08:47.394416+0000) 2024-09-11T22:09:06.047 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:05 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:05.620+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:33.295337+0000 front 2024-09-11T22:08:33.295289+0000 (oldest deadline 2024-09-11T22:08:56.795428+0000) 2024-09-11T22:09:06.047 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:05 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:05.620+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:39.497398+0000 front 2024-09-11T22:08:39.497364+0000 (oldest deadline 2024-09-11T22:09:04.797240+0000) 2024-09-11T22:09:06.342 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:06 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:06.046+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:26.210992+0000 front 2024-09-11T22:08:26.211085+0000 (oldest deadline 2024-09-11T22:08:50.310996+0000) 2024-09-11T22:09:06.342 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:06 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:06.046+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:30.311301+0000 front 2024-09-11T22:08:30.311235+0000 (oldest deadline 2024-09-11T22:08:59.712086+0000) 2024-09-11T22:09:06.342 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:06 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:06.046+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:39.712759+0000 front 2024-09-11T22:08:39.712634+0000 (oldest deadline 2024-09-11T22:09:03.812485+0000) 2024-09-11T22:09:06.443 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 11 22:09:05 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-6[55973]: 2024-09-11T22:09:05.995+0000 7f9c8be0c640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.487210+0000 front 2024-09-11T22:08:17.487317+0000 (oldest deadline 2024-09-11T22:08:42.187568+0000) 2024-09-11T22:09:06.443 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 11 22:09:05 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-6[55973]: 2024-09-11T22:09:05.995+0000 7f9c8be0c640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:26.288972+0000 front 2024-09-11T22:08:26.288673+0000 (oldest deadline 2024-09-11T22:08:50.988740+0000) 2024-09-11T22:09:06.443 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 11 22:09:05 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-6[55973]: 2024-09-11T22:09:05.995+0000 7f9c8be0c640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:34.490028+0000 front 2024-09-11T22:08:34.489988+0000 (oldest deadline 2024-09-11T22:08:56.789739+0000) 2024-09-11T22:09:06.443 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 11 22:09:05 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-6[55973]: 2024-09-11T22:09:05.995+0000 7f9c8be0c640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:40.290501+0000 front 2024-09-11T22:08:40.290470+0000 (oldest deadline 2024-09-11T22:09:04.390550+0000) 2024-09-11T22:09:06.547 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 22:09:06 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-7[52494]: 2024-09-11T22:09:06.265+0000 7f29f1026640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:25.674626+0000 front 2024-09-11T22:08:25.674669+0000 (oldest deadline 2024-09-11T22:08:49.174727+0000) 2024-09-11T22:09:06.548 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 22:09:06 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-7[52494]: 2024-09-11T22:09:06.265+0000 7f29f1026640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:31.976401+0000 front 2024-09-11T22:08:31.976263+0000 (oldest deadline 2024-09-11T22:08:55.377028+0000) 2024-09-11T22:09:06.548 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 22:09:06 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-7[52494]: 2024-09-11T22:09:06.265+0000 7f29f1026640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:38.278154+0000 front 2024-09-11T22:08:38.278126+0000 (oldest deadline 2024-09-11T22:09:05.878759+0000) 2024-09-11T22:09:06.548 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:06 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:06.243+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.481596+0000 front 2024-09-11T22:08:17.481499+0000 (oldest deadline 2024-09-11T22:08:42.181347+0000) 2024-09-11T22:09:06.548 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:06 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:06.243+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:31.383481+0000 front 2024-09-11T22:08:31.383448+0000 (oldest deadline 2024-09-11T22:08:56.083565+0000) 2024-09-11T22:09:06.548 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:06 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:06.243+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:38.384885+0000 front 2024-09-11T22:08:38.385010+0000 (oldest deadline 2024-09-11T22:09:03.684801+0000) 2024-09-11T22:09:06.943 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:06 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:06.546+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.497679+0000 front 2024-09-11T22:08:16.497606+0000 (oldest deadline 2024-09-11T22:08:42.397744+0000) 2024-09-11T22:09:06.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:06 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:06.546+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:22.398445+0000 front 2024-09-11T22:08:22.398495+0000 (oldest deadline 2024-09-11T22:08:47.098448+0000) 2024-09-11T22:09:06.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:06 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:06.546+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:32.999892+0000 front 2024-09-11T22:08:32.999692+0000 (oldest deadline 2024-09-11T22:08:58.299748+0000) 2024-09-11T22:09:06.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:06 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:06.546+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:40.000748+0000 front 2024-09-11T22:08:40.000701+0000 (oldest deadline 2024-09-11T22:09:03.500864+0000) 2024-09-11T22:09:07.012 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:06 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:06.633+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.893190+0000 front 2024-09-11T22:08:16.893259+0000 (oldest deadline 2024-09-11T22:08:39.193066+0000) 2024-09-11T22:09:07.012 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:06 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:06.633+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:21.494266+0000 front 2024-09-11T22:08:21.494360+0000 (oldest deadline 2024-09-11T22:08:47.394416+0000) 2024-09-11T22:09:07.012 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:06 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:06.633+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:33.295337+0000 front 2024-09-11T22:08:33.295289+0000 (oldest deadline 2024-09-11T22:08:56.795428+0000) 2024-09-11T22:09:07.012 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:06 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:06.633+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:39.497398+0000 front 2024-09-11T22:08:39.497364+0000 (oldest deadline 2024-09-11T22:09:04.797240+0000) 2024-09-11T22:09:07.341 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:07 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:07.011+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:26.210992+0000 front 2024-09-11T22:08:26.211085+0000 (oldest deadline 2024-09-11T22:08:50.310996+0000) 2024-09-11T22:09:07.342 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:07 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:07.011+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:30.311301+0000 front 2024-09-11T22:08:30.311235+0000 (oldest deadline 2024-09-11T22:08:59.712086+0000) 2024-09-11T22:09:07.342 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:07 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:07.011+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:39.712759+0000 front 2024-09-11T22:08:39.712634+0000 (oldest deadline 2024-09-11T22:09:03.812485+0000) 2024-09-11T22:09:07.548 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 22:09:07 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-7[52494]: 2024-09-11T22:09:07.260+0000 7f29f1026640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:25.674626+0000 front 2024-09-11T22:08:25.674669+0000 (oldest deadline 2024-09-11T22:08:49.174727+0000) 2024-09-11T22:09:07.548 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 22:09:07 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-7[52494]: 2024-09-11T22:09:07.260+0000 7f29f1026640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:31.976401+0000 front 2024-09-11T22:08:31.976263+0000 (oldest deadline 2024-09-11T22:08:55.377028+0000) 2024-09-11T22:09:07.548 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 22:09:07 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-7[52494]: 2024-09-11T22:09:07.260+0000 7f29f1026640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:38.278154+0000 front 2024-09-11T22:08:38.278126+0000 (oldest deadline 2024-09-11T22:09:05.878759+0000) 2024-09-11T22:09:07.548 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:07 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:07.265+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.481596+0000 front 2024-09-11T22:08:17.481499+0000 (oldest deadline 2024-09-11T22:08:42.181347+0000) 2024-09-11T22:09:07.548 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:07 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:07.265+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:31.383481+0000 front 2024-09-11T22:08:31.383448+0000 (oldest deadline 2024-09-11T22:08:56.083565+0000) 2024-09-11T22:09:07.549 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:07 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:07.265+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:38.384885+0000 front 2024-09-11T22:08:38.385010+0000 (oldest deadline 2024-09-11T22:09:03.684801+0000) 2024-09-11T22:09:07.643 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 11 22:09:07 smithi001 podman[208418]: 2024-09-11 22:09:07.352264584 +0000 UTC m=+5.598376089 container died 03adbad21ad78665d086fc274f7b619f6df9b31773cfb2b0720fe25a7a5e69ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-6, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-11T22:09:07.643 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:07 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:07.588+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.497679+0000 front 2024-09-11T22:08:16.497606+0000 (oldest deadline 2024-09-11T22:08:42.397744+0000) 2024-09-11T22:09:07.644 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:07 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:07.588+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:22.398445+0000 front 2024-09-11T22:08:22.398495+0000 (oldest deadline 2024-09-11T22:08:47.098448+0000) 2024-09-11T22:09:07.644 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:07 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:07.588+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:32.999892+0000 front 2024-09-11T22:08:32.999692+0000 (oldest deadline 2024-09-11T22:08:58.299748+0000) 2024-09-11T22:09:07.644 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:07 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:07.588+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:40.000748+0000 front 2024-09-11T22:08:40.000701+0000 (oldest deadline 2024-09-11T22:09:03.500864+0000) 2024-09-11T22:09:07.943 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 11 22:09:07 smithi001 podman[208418]: 2024-09-11 22:09:07.841277173 +0000 UTC m=+6.087388675 container cleanup 03adbad21ad78665d086fc274f7b619f6df9b31773cfb2b0720fe25a7a5e69ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-6, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-11T22:09:07.943 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 11 22:09:07 smithi001 bash[208418]: ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-6 2024-09-11T22:09:07.985 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:07 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:07.674+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.893190+0000 front 2024-09-11T22:08:16.893259+0000 (oldest deadline 2024-09-11T22:08:39.193066+0000) 2024-09-11T22:09:07.985 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:07 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:07.674+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:21.494266+0000 front 2024-09-11T22:08:21.494360+0000 (oldest deadline 2024-09-11T22:08:47.394416+0000) 2024-09-11T22:09:07.985 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:07 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:07.674+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:33.295337+0000 front 2024-09-11T22:08:33.295289+0000 (oldest deadline 2024-09-11T22:08:56.795428+0000) 2024-09-11T22:09:07.985 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:07 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:07.674+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:39.497398+0000 front 2024-09-11T22:08:39.497364+0000 (oldest deadline 2024-09-11T22:09:04.797240+0000) 2024-09-11T22:09:08.342 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:07 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:07.984+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:26.210992+0000 front 2024-09-11T22:08:26.211085+0000 (oldest deadline 2024-09-11T22:08:50.310996+0000) 2024-09-11T22:09:08.342 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:07 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:07.984+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:30.311301+0000 front 2024-09-11T22:08:30.311235+0000 (oldest deadline 2024-09-11T22:08:59.712086+0000) 2024-09-11T22:09:08.342 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:07 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:07.984+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:39.712759+0000 front 2024-09-11T22:08:39.712634+0000 (oldest deadline 2024-09-11T22:09:03.812485+0000) 2024-09-11T22:09:08.443 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 11 22:09:08 smithi001 podman[208434]: 2024-09-11 22:09:08.109674897 +0000 UTC m=+0.751971615 container remove 03adbad21ad78665d086fc274f7b619f6df9b31773cfb2b0720fe25a7a5e69ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-6, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-11T22:09:08.548 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 22:09:08 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-7[52494]: 2024-09-11T22:09:08.283+0000 7f29f1026640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:25.674626+0000 front 2024-09-11T22:08:25.674669+0000 (oldest deadline 2024-09-11T22:08:49.174727+0000) 2024-09-11T22:09:08.548 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 22:09:08 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-7[52494]: 2024-09-11T22:09:08.283+0000 7f29f1026640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:31.976401+0000 front 2024-09-11T22:08:31.976263+0000 (oldest deadline 2024-09-11T22:08:55.377028+0000) 2024-09-11T22:09:08.548 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 22:09:08 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-7[52494]: 2024-09-11T22:09:08.283+0000 7f29f1026640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:38.278154+0000 front 2024-09-11T22:08:38.278126+0000 (oldest deadline 2024-09-11T22:09:05.878759+0000) 2024-09-11T22:09:08.548 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:08 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:08.244+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.481596+0000 front 2024-09-11T22:08:17.481499+0000 (oldest deadline 2024-09-11T22:08:42.181347+0000) 2024-09-11T22:09:08.548 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:08 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:08.244+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:31.383481+0000 front 2024-09-11T22:08:31.383448+0000 (oldest deadline 2024-09-11T22:08:56.083565+0000) 2024-09-11T22:09:08.549 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:08 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:08.244+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:38.384885+0000 front 2024-09-11T22:08:38.385010+0000 (oldest deadline 2024-09-11T22:09:03.684801+0000) 2024-09-11T22:09:08.730 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 11 22:09:08 smithi001 podman[208522]: 2024-09-11 22:09:08.366855282 +0000 UTC m=+0.016442270 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T22:09:08.730 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 11 22:09:08 smithi001 podman[208522]: 2024-09-11 22:09:08.485718515 +0000 UTC m=+0.135305544 container create b0f6048523b1ffe5a1794990ef027f7ab2eff98fbb55f0e948cafe7e668704de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-6-deactivate, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS) 2024-09-11T22:09:08.730 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:08 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:08.628+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.497679+0000 front 2024-09-11T22:08:16.497606+0000 (oldest deadline 2024-09-11T22:08:42.397744+0000) 2024-09-11T22:09:08.730 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:08 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:08.628+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:22.398445+0000 front 2024-09-11T22:08:22.398495+0000 (oldest deadline 2024-09-11T22:08:47.098448+0000) 2024-09-11T22:09:08.730 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:08 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:08.628+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:32.999892+0000 front 2024-09-11T22:08:32.999692+0000 (oldest deadline 2024-09-11T22:08:58.299748+0000) 2024-09-11T22:09:08.731 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:08 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:08.628+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:40.000748+0000 front 2024-09-11T22:08:40.000701+0000 (oldest deadline 2024-09-11T22:09:03.500864+0000) 2024-09-11T22:09:09.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:08 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:08.709+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.893190+0000 front 2024-09-11T22:08:16.893259+0000 (oldest deadline 2024-09-11T22:08:39.193066+0000) 2024-09-11T22:09:09.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:08 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:08.709+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:21.494266+0000 front 2024-09-11T22:08:21.494360+0000 (oldest deadline 2024-09-11T22:08:47.394416+0000) 2024-09-11T22:09:09.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:08 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:08.709+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:33.295337+0000 front 2024-09-11T22:08:33.295289+0000 (oldest deadline 2024-09-11T22:08:56.795428+0000) 2024-09-11T22:09:09.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:08 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:08.709+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:39.497398+0000 front 2024-09-11T22:08:39.497364+0000 (oldest deadline 2024-09-11T22:09:04.797240+0000) 2024-09-11T22:09:09.093 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:08 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:08.945+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:26.210992+0000 front 2024-09-11T22:08:26.211085+0000 (oldest deadline 2024-09-11T22:08:50.310996+0000) 2024-09-11T22:09:09.093 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:08 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:08.945+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:30.311301+0000 front 2024-09-11T22:08:30.311235+0000 (oldest deadline 2024-09-11T22:08:59.712086+0000) 2024-09-11T22:09:09.093 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:08 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:08.945+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:39.712759+0000 front 2024-09-11T22:08:39.712634+0000 (oldest deadline 2024-09-11T22:09:03.812485+0000) 2024-09-11T22:09:09.197 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 11 22:09:08 smithi001 podman[208522]: 2024-09-11 22:09:08.729891188 +0000 UTC m=+0.379478187 container init b0f6048523b1ffe5a1794990ef027f7ab2eff98fbb55f0e948cafe7e668704de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-11T22:09:09.197 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 11 22:09:08 smithi001 podman[208522]: 2024-09-11 22:09:08.733521038 +0000 UTC m=+0.383108027 container start b0f6048523b1ffe5a1794990ef027f7ab2eff98fbb55f0e948cafe7e668704de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux ) 2024-09-11T22:09:09.197 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 11 22:09:08 smithi001 podman[208522]: 2024-09-11 22:09:08.829988637 +0000 UTC m=+0.479575639 container attach b0f6048523b1ffe5a1794990ef027f7ab2eff98fbb55f0e948cafe7e668704de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-11T22:09:09.197 INFO:journalctl@ceph.osd.6.smithi001.stdout:Sep 11 22:09:08 smithi001 podman[208522]: 2024-09-11 22:09:08.925686008 +0000 UTC m=+0.575272996 container died b0f6048523b1ffe5a1794990ef027f7ab2eff98fbb55f0e948cafe7e668704de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2) 2024-09-11T22:09:09.547 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:09 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:09.272+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.481596+0000 front 2024-09-11T22:08:17.481499+0000 (oldest deadline 2024-09-11T22:08:42.181347+0000) 2024-09-11T22:09:09.548 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:09 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:09.272+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:31.383481+0000 front 2024-09-11T22:08:31.383448+0000 (oldest deadline 2024-09-11T22:08:56.083565+0000) 2024-09-11T22:09:09.548 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:09 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:09.272+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:38.384885+0000 front 2024-09-11T22:08:38.385010+0000 (oldest deadline 2024-09-11T22:09:03.684801+0000) 2024-09-11T22:09:09.548 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 22:09:09 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-7[52494]: 2024-09-11T22:09:09.274+0000 7f29f1026640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:25.674626+0000 front 2024-09-11T22:08:25.674669+0000 (oldest deadline 2024-09-11T22:08:49.174727+0000) 2024-09-11T22:09:09.548 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 22:09:09 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-7[52494]: 2024-09-11T22:09:09.274+0000 7f29f1026640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:31.976401+0000 front 2024-09-11T22:08:31.976263+0000 (oldest deadline 2024-09-11T22:08:55.377028+0000) 2024-09-11T22:09:09.548 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 22:09:09 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-7[52494]: 2024-09-11T22:09:09.274+0000 7f29f1026640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:38.278154+0000 front 2024-09-11T22:08:38.278126+0000 (oldest deadline 2024-09-11T22:09:05.878759+0000) 2024-09-11T22:09:09.698 DEBUG:teuthology.orchestra.run.smithi001:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3ffca83c-7087-11ef-bcea-c7b262605968@osd.6.service' 2024-09-11T22:09:09.745 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:09:09.746 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-11T22:09:09.746 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-11T22:09:09.746 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-3ffca83c-7087-11ef-bcea-c7b262605968@osd.7 2024-09-11T22:09:09.943 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:09 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:09.647+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.497679+0000 front 2024-09-11T22:08:16.497606+0000 (oldest deadline 2024-09-11T22:08:42.397744+0000) 2024-09-11T22:09:09.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:09 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:09.647+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:22.398445+0000 front 2024-09-11T22:08:22.398495+0000 (oldest deadline 2024-09-11T22:08:47.098448+0000) 2024-09-11T22:09:09.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:09 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:09.647+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:32.999892+0000 front 2024-09-11T22:08:32.999692+0000 (oldest deadline 2024-09-11T22:08:58.299748+0000) 2024-09-11T22:09:09.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:09 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:09.647+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:40.000748+0000 front 2024-09-11T22:08:40.000701+0000 (oldest deadline 2024-09-11T22:09:03.500864+0000) 2024-09-11T22:09:10.047 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 22:09:09 smithi053 systemd[1]: Stopping Ceph osd.7 for 3ffca83c-7087-11ef-bcea-c7b262605968... 2024-09-11T22:09:10.092 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:09 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:09.947+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:26.210992+0000 front 2024-09-11T22:08:26.211085+0000 (oldest deadline 2024-09-11T22:08:50.310996+0000) 2024-09-11T22:09:10.092 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:09 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:09.947+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:30.311301+0000 front 2024-09-11T22:08:30.311235+0000 (oldest deadline 2024-09-11T22:08:59.712086+0000) 2024-09-11T22:09:10.092 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:09 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:09.947+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:39.712759+0000 front 2024-09-11T22:08:39.712634+0000 (oldest deadline 2024-09-11T22:09:03.812485+0000) 2024-09-11T22:09:10.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:09 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:09.716+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.893190+0000 front 2024-09-11T22:08:16.893259+0000 (oldest deadline 2024-09-11T22:08:39.193066+0000) 2024-09-11T22:09:10.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:09 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:09.716+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:21.494266+0000 front 2024-09-11T22:08:21.494360+0000 (oldest deadline 2024-09-11T22:08:47.394416+0000) 2024-09-11T22:09:10.093 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:09 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:09.716+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:33.295337+0000 front 2024-09-11T22:08:33.295289+0000 (oldest deadline 2024-09-11T22:08:56.795428+0000) 2024-09-11T22:09:10.093 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:09 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:09.716+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:39.497398+0000 front 2024-09-11T22:08:39.497364+0000 (oldest deadline 2024-09-11T22:09:04.797240+0000) 2024-09-11T22:09:10.305 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 22:09:10 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-7[52494]: 2024-09-11T22:09:10.074+0000 7f29f8a18640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:09:10.305 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 22:09:10 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-7[52494]: 2024-09-11T22:09:10.074+0000 7f29f8a18640 -1 osd.7 98 *** Got signal Terminated *** 2024-09-11T22:09:10.305 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 22:09:10 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-7[52494]: 2024-09-11T22:09:10.074+0000 7f29f8a18640 -1 osd.7 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T22:09:10.306 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 22:09:10 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-7[52494]: 2024-09-11T22:09:10.281+0000 7f29f1026640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:25.674626+0000 front 2024-09-11T22:08:25.674669+0000 (oldest deadline 2024-09-11T22:08:49.174727+0000) 2024-09-11T22:09:10.306 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 22:09:10 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-7[52494]: 2024-09-11T22:09:10.281+0000 7f29f1026640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:31.976401+0000 front 2024-09-11T22:08:31.976263+0000 (oldest deadline 2024-09-11T22:08:55.377028+0000) 2024-09-11T22:09:10.306 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 22:09:10 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-7[52494]: 2024-09-11T22:09:10.281+0000 7f29f1026640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:38.278154+0000 front 2024-09-11T22:08:38.278126+0000 (oldest deadline 2024-09-11T22:09:05.878759+0000) 2024-09-11T22:09:10.797 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:10 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:10.304+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.481596+0000 front 2024-09-11T22:08:17.481499+0000 (oldest deadline 2024-09-11T22:08:42.181347+0000) 2024-09-11T22:09:10.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:10 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:10.304+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:31.383481+0000 front 2024-09-11T22:08:31.383448+0000 (oldest deadline 2024-09-11T22:08:56.083565+0000) 2024-09-11T22:09:10.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:10 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:10.304+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:38.384885+0000 front 2024-09-11T22:08:38.385010+0000 (oldest deadline 2024-09-11T22:09:03.684801+0000) 2024-09-11T22:09:10.943 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:10 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:10.606+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.497679+0000 front 2024-09-11T22:08:16.497606+0000 (oldest deadline 2024-09-11T22:08:42.397744+0000) 2024-09-11T22:09:10.943 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:10 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:10.606+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:22.398445+0000 front 2024-09-11T22:08:22.398495+0000 (oldest deadline 2024-09-11T22:08:47.098448+0000) 2024-09-11T22:09:10.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:10 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:10.606+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:32.999892+0000 front 2024-09-11T22:08:32.999692+0000 (oldest deadline 2024-09-11T22:08:58.299748+0000) 2024-09-11T22:09:10.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:10 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:10.606+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:40.000748+0000 front 2024-09-11T22:08:40.000701+0000 (oldest deadline 2024-09-11T22:09:03.500864+0000) 2024-09-11T22:09:10.994 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:10 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:10.725+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.893190+0000 front 2024-09-11T22:08:16.893259+0000 (oldest deadline 2024-09-11T22:08:39.193066+0000) 2024-09-11T22:09:10.994 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:10 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:10.725+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:21.494266+0000 front 2024-09-11T22:08:21.494360+0000 (oldest deadline 2024-09-11T22:08:47.394416+0000) 2024-09-11T22:09:10.994 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:10 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:10.725+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:33.295337+0000 front 2024-09-11T22:08:33.295289+0000 (oldest deadline 2024-09-11T22:08:56.795428+0000) 2024-09-11T22:09:10.995 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:10 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:10.725+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:39.497398+0000 front 2024-09-11T22:08:39.497364+0000 (oldest deadline 2024-09-11T22:09:04.797240+0000) 2024-09-11T22:09:11.336 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 22:09:11 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-7[52494]: 2024-09-11T22:09:11.273+0000 7f29f1026640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:25.674626+0000 front 2024-09-11T22:08:25.674669+0000 (oldest deadline 2024-09-11T22:08:49.174727+0000) 2024-09-11T22:09:11.336 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 22:09:11 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-7[52494]: 2024-09-11T22:09:11.273+0000 7f29f1026640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:31.976401+0000 front 2024-09-11T22:08:31.976263+0000 (oldest deadline 2024-09-11T22:08:55.377028+0000) 2024-09-11T22:09:11.336 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 22:09:11 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-7[52494]: 2024-09-11T22:09:11.273+0000 7f29f1026640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:38.278154+0000 front 2024-09-11T22:08:38.278126+0000 (oldest deadline 2024-09-11T22:09:05.878759+0000) 2024-09-11T22:09:11.342 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:10 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:10.993+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:26.210992+0000 front 2024-09-11T22:08:26.211085+0000 (oldest deadline 2024-09-11T22:08:50.310996+0000) 2024-09-11T22:09:11.342 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:10 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:10.993+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:30.311301+0000 front 2024-09-11T22:08:30.311235+0000 (oldest deadline 2024-09-11T22:08:59.712086+0000) 2024-09-11T22:09:11.342 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:10 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:10.993+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:39.712759+0000 front 2024-09-11T22:08:39.712634+0000 (oldest deadline 2024-09-11T22:09:03.812485+0000) 2024-09-11T22:09:11.797 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:11 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:11.344+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.481596+0000 front 2024-09-11T22:08:17.481499+0000 (oldest deadline 2024-09-11T22:08:42.181347+0000) 2024-09-11T22:09:11.797 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:11 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:11.344+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:31.383481+0000 front 2024-09-11T22:08:31.383448+0000 (oldest deadline 2024-09-11T22:08:56.083565+0000) 2024-09-11T22:09:11.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:11 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:11.344+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:38.384885+0000 front 2024-09-11T22:08:38.385010+0000 (oldest deadline 2024-09-11T22:09:03.684801+0000) 2024-09-11T22:09:11.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:11 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:11.344+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.686290+0000 front 2024-09-11T22:08:50.686225+0000 (oldest deadline 2024-09-11T22:09:11.186328+0000) 2024-09-11T22:09:11.943 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:11 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:11.565+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.497679+0000 front 2024-09-11T22:08:16.497606+0000 (oldest deadline 2024-09-11T22:08:42.397744+0000) 2024-09-11T22:09:11.943 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:11 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:11.565+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:22.398445+0000 front 2024-09-11T22:08:22.398495+0000 (oldest deadline 2024-09-11T22:08:47.098448+0000) 2024-09-11T22:09:11.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:11 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:11.565+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:32.999892+0000 front 2024-09-11T22:08:32.999692+0000 (oldest deadline 2024-09-11T22:08:58.299748+0000) 2024-09-11T22:09:11.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:11 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:11.565+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:40.000748+0000 front 2024-09-11T22:08:40.000701+0000 (oldest deadline 2024-09-11T22:09:03.500864+0000) 2024-09-11T22:09:12.018 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:11 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:11.733+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.893190+0000 front 2024-09-11T22:08:16.893259+0000 (oldest deadline 2024-09-11T22:08:39.193066+0000) 2024-09-11T22:09:12.018 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:11 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:11.733+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:21.494266+0000 front 2024-09-11T22:08:21.494360+0000 (oldest deadline 2024-09-11T22:08:47.394416+0000) 2024-09-11T22:09:12.018 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:11 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:11.733+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:33.295337+0000 front 2024-09-11T22:08:33.295289+0000 (oldest deadline 2024-09-11T22:08:56.795428+0000) 2024-09-11T22:09:12.018 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:11 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:11.733+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:39.497398+0000 front 2024-09-11T22:08:39.497364+0000 (oldest deadline 2024-09-11T22:09:04.797240+0000) 2024-09-11T22:09:12.342 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:12 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:12.017+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:26.210992+0000 front 2024-09-11T22:08:26.211085+0000 (oldest deadline 2024-09-11T22:08:50.310996+0000) 2024-09-11T22:09:12.342 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:12 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:12.017+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:30.311301+0000 front 2024-09-11T22:08:30.311235+0000 (oldest deadline 2024-09-11T22:08:59.712086+0000) 2024-09-11T22:09:12.342 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:12 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:12.017+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:39.712759+0000 front 2024-09-11T22:08:39.712634+0000 (oldest deadline 2024-09-11T22:09:03.812485+0000) 2024-09-11T22:09:12.548 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 22:09:12 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-7[52494]: 2024-09-11T22:09:12.262+0000 7f29f1026640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:25.674626+0000 front 2024-09-11T22:08:25.674669+0000 (oldest deadline 2024-09-11T22:08:49.174727+0000) 2024-09-11T22:09:12.548 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 22:09:12 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-7[52494]: 2024-09-11T22:09:12.262+0000 7f29f1026640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:31.976401+0000 front 2024-09-11T22:08:31.976263+0000 (oldest deadline 2024-09-11T22:08:55.377028+0000) 2024-09-11T22:09:12.548 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 22:09:12 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-7[52494]: 2024-09-11T22:09:12.262+0000 7f29f1026640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:38.278154+0000 front 2024-09-11T22:08:38.278126+0000 (oldest deadline 2024-09-11T22:09:05.878759+0000) 2024-09-11T22:09:12.549 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:12 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:12.307+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.481596+0000 front 2024-09-11T22:08:17.481499+0000 (oldest deadline 2024-09-11T22:08:42.181347+0000) 2024-09-11T22:09:12.549 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:12 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:12.307+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:31.383481+0000 front 2024-09-11T22:08:31.383448+0000 (oldest deadline 2024-09-11T22:08:56.083565+0000) 2024-09-11T22:09:12.549 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:12 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:12.307+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:38.384885+0000 front 2024-09-11T22:08:38.385010+0000 (oldest deadline 2024-09-11T22:09:03.684801+0000) 2024-09-11T22:09:12.549 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:12 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:12.307+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.686290+0000 front 2024-09-11T22:08:50.686225+0000 (oldest deadline 2024-09-11T22:09:11.186328+0000) 2024-09-11T22:09:12.842 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:12 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:12.688+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.893190+0000 front 2024-09-11T22:08:16.893259+0000 (oldest deadline 2024-09-11T22:08:39.193066+0000) 2024-09-11T22:09:12.842 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:12 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:12.688+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:21.494266+0000 front 2024-09-11T22:08:21.494360+0000 (oldest deadline 2024-09-11T22:08:47.394416+0000) 2024-09-11T22:09:12.842 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:12 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:12.688+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:33.295337+0000 front 2024-09-11T22:08:33.295289+0000 (oldest deadline 2024-09-11T22:08:56.795428+0000) 2024-09-11T22:09:12.842 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:12 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:12.688+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:39.497398+0000 front 2024-09-11T22:08:39.497364+0000 (oldest deadline 2024-09-11T22:09:04.797240+0000) 2024-09-11T22:09:12.943 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:12 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:12.520+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.497679+0000 front 2024-09-11T22:08:16.497606+0000 (oldest deadline 2024-09-11T22:08:42.397744+0000) 2024-09-11T22:09:12.943 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:12 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:12.520+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:22.398445+0000 front 2024-09-11T22:08:22.398495+0000 (oldest deadline 2024-09-11T22:08:47.098448+0000) 2024-09-11T22:09:12.943 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:12 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:12.520+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:32.999892+0000 front 2024-09-11T22:08:32.999692+0000 (oldest deadline 2024-09-11T22:08:58.299748+0000) 2024-09-11T22:09:12.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:12 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:12.520+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:40.000748+0000 front 2024-09-11T22:08:40.000701+0000 (oldest deadline 2024-09-11T22:09:03.500864+0000) 2024-09-11T22:09:13.342 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:13 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:13.008+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:26.210992+0000 front 2024-09-11T22:08:26.211085+0000 (oldest deadline 2024-09-11T22:08:50.310996+0000) 2024-09-11T22:09:13.342 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:13 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:13.008+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:30.311301+0000 front 2024-09-11T22:08:30.311235+0000 (oldest deadline 2024-09-11T22:08:59.712086+0000) 2024-09-11T22:09:13.342 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:13 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:13.008+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:39.712759+0000 front 2024-09-11T22:08:39.712634+0000 (oldest deadline 2024-09-11T22:09:03.812485+0000) 2024-09-11T22:09:13.342 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:13 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:13.008+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:47.214214+0000 front 2024-09-11T22:08:47.214135+0000 (oldest deadline 2024-09-11T22:09:12.513937+0000) 2024-09-11T22:09:13.548 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 22:09:13 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-7[52494]: 2024-09-11T22:09:13.255+0000 7f29f1026640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:25.674626+0000 front 2024-09-11T22:08:25.674669+0000 (oldest deadline 2024-09-11T22:08:49.174727+0000) 2024-09-11T22:09:13.548 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 22:09:13 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-7[52494]: 2024-09-11T22:09:13.255+0000 7f29f1026640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:31.976401+0000 front 2024-09-11T22:08:31.976263+0000 (oldest deadline 2024-09-11T22:08:55.377028+0000) 2024-09-11T22:09:13.548 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 22:09:13 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-7[52494]: 2024-09-11T22:09:13.255+0000 7f29f1026640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:38.278154+0000 front 2024-09-11T22:08:38.278126+0000 (oldest deadline 2024-09-11T22:09:05.878759+0000) 2024-09-11T22:09:13.548 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 22:09:13 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-7[52494]: 2024-09-11T22:09:13.255+0000 7f29f1026640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:46.979426+0000 front 2024-09-11T22:08:46.979447+0000 (oldest deadline 2024-09-11T22:09:12.879659+0000) 2024-09-11T22:09:13.548 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:13 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:13.276+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.481596+0000 front 2024-09-11T22:08:17.481499+0000 (oldest deadline 2024-09-11T22:08:42.181347+0000) 2024-09-11T22:09:13.549 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:13 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:13.276+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:31.383481+0000 front 2024-09-11T22:08:31.383448+0000 (oldest deadline 2024-09-11T22:08:56.083565+0000) 2024-09-11T22:09:13.549 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:13 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:13.276+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:38.384885+0000 front 2024-09-11T22:08:38.385010+0000 (oldest deadline 2024-09-11T22:09:03.684801+0000) 2024-09-11T22:09:13.549 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:13 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:13.276+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.686290+0000 front 2024-09-11T22:08:50.686225+0000 (oldest deadline 2024-09-11T22:09:11.186328+0000) 2024-09-11T22:09:13.943 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:13 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:13.496+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.497679+0000 front 2024-09-11T22:08:16.497606+0000 (oldest deadline 2024-09-11T22:08:42.397744+0000) 2024-09-11T22:09:13.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:13 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:13.496+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:22.398445+0000 front 2024-09-11T22:08:22.398495+0000 (oldest deadline 2024-09-11T22:08:47.098448+0000) 2024-09-11T22:09:13.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:13 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:13.496+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:32.999892+0000 front 2024-09-11T22:08:32.999692+0000 (oldest deadline 2024-09-11T22:08:58.299748+0000) 2024-09-11T22:09:13.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:13 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:13.496+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:40.000748+0000 front 2024-09-11T22:08:40.000701+0000 (oldest deadline 2024-09-11T22:09:03.500864+0000) 2024-09-11T22:09:14.022 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:13 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:13.736+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.893190+0000 front 2024-09-11T22:08:16.893259+0000 (oldest deadline 2024-09-11T22:08:39.193066+0000) 2024-09-11T22:09:14.022 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:13 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:13.736+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:21.494266+0000 front 2024-09-11T22:08:21.494360+0000 (oldest deadline 2024-09-11T22:08:47.394416+0000) 2024-09-11T22:09:14.022 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:13 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:13.736+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:33.295337+0000 front 2024-09-11T22:08:33.295289+0000 (oldest deadline 2024-09-11T22:08:56.795428+0000) 2024-09-11T22:09:14.022 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:13 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:13.736+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:39.497398+0000 front 2024-09-11T22:08:39.497364+0000 (oldest deadline 2024-09-11T22:09:04.797240+0000) 2024-09-11T22:09:14.342 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:14 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:14.021+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:26.210992+0000 front 2024-09-11T22:08:26.211085+0000 (oldest deadline 2024-09-11T22:08:50.310996+0000) 2024-09-11T22:09:14.342 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:14 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:14.021+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:30.311301+0000 front 2024-09-11T22:08:30.311235+0000 (oldest deadline 2024-09-11T22:08:59.712086+0000) 2024-09-11T22:09:14.342 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:14 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:14.021+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:39.712759+0000 front 2024-09-11T22:08:39.712634+0000 (oldest deadline 2024-09-11T22:09:03.812485+0000) 2024-09-11T22:09:14.342 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:14 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:14.021+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:47.214214+0000 front 2024-09-11T22:08:47.214135+0000 (oldest deadline 2024-09-11T22:09:12.513937+0000) 2024-09-11T22:09:14.548 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 22:09:14 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-7[52494]: 2024-09-11T22:09:14.212+0000 7f29f1026640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:25.674626+0000 front 2024-09-11T22:08:25.674669+0000 (oldest deadline 2024-09-11T22:08:49.174727+0000) 2024-09-11T22:09:14.548 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 22:09:14 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-7[52494]: 2024-09-11T22:09:14.212+0000 7f29f1026640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:31.976401+0000 front 2024-09-11T22:08:31.976263+0000 (oldest deadline 2024-09-11T22:08:55.377028+0000) 2024-09-11T22:09:14.548 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 22:09:14 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-7[52494]: 2024-09-11T22:09:14.212+0000 7f29f1026640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:38.278154+0000 front 2024-09-11T22:08:38.278126+0000 (oldest deadline 2024-09-11T22:09:05.878759+0000) 2024-09-11T22:09:14.548 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 22:09:14 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-7[52494]: 2024-09-11T22:09:14.212+0000 7f29f1026640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:46.979426+0000 front 2024-09-11T22:08:46.979447+0000 (oldest deadline 2024-09-11T22:09:12.879659+0000) 2024-09-11T22:09:14.549 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:14 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:14.309+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.481596+0000 front 2024-09-11T22:08:17.481499+0000 (oldest deadline 2024-09-11T22:08:42.181347+0000) 2024-09-11T22:09:14.549 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:14 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:14.309+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:31.383481+0000 front 2024-09-11T22:08:31.383448+0000 (oldest deadline 2024-09-11T22:08:56.083565+0000) 2024-09-11T22:09:14.549 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:14 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:14.309+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:38.384885+0000 front 2024-09-11T22:08:38.385010+0000 (oldest deadline 2024-09-11T22:09:03.684801+0000) 2024-09-11T22:09:14.549 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:14 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:14.309+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.686290+0000 front 2024-09-11T22:08:50.686225+0000 (oldest deadline 2024-09-11T22:09:11.186328+0000) 2024-09-11T22:09:14.943 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:14 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:14.461+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.497679+0000 front 2024-09-11T22:08:16.497606+0000 (oldest deadline 2024-09-11T22:08:42.397744+0000) 2024-09-11T22:09:14.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:14 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:14.461+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:22.398445+0000 front 2024-09-11T22:08:22.398495+0000 (oldest deadline 2024-09-11T22:08:47.098448+0000) 2024-09-11T22:09:14.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:14 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:14.461+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:32.999892+0000 front 2024-09-11T22:08:32.999692+0000 (oldest deadline 2024-09-11T22:08:58.299748+0000) 2024-09-11T22:09:14.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:14 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:14.461+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:40.000748+0000 front 2024-09-11T22:08:40.000701+0000 (oldest deadline 2024-09-11T22:09:03.500864+0000) 2024-09-11T22:09:14.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:14 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:14.461+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.502512+0000 front 2024-09-11T22:08:50.502535+0000 (oldest deadline 2024-09-11T22:09:14.002461+0000) 2024-09-11T22:09:15.027 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:14 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:14.760+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.893190+0000 front 2024-09-11T22:08:16.893259+0000 (oldest deadline 2024-09-11T22:08:39.193066+0000) 2024-09-11T22:09:15.027 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:14 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:14.760+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:21.494266+0000 front 2024-09-11T22:08:21.494360+0000 (oldest deadline 2024-09-11T22:08:47.394416+0000) 2024-09-11T22:09:15.028 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:14 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:14.760+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:33.295337+0000 front 2024-09-11T22:08:33.295289+0000 (oldest deadline 2024-09-11T22:08:56.795428+0000) 2024-09-11T22:09:15.028 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:14 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:14.760+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:39.497398+0000 front 2024-09-11T22:08:39.497364+0000 (oldest deadline 2024-09-11T22:09:04.797240+0000) 2024-09-11T22:09:15.342 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:15 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:15.027+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:26.210992+0000 front 2024-09-11T22:08:26.211085+0000 (oldest deadline 2024-09-11T22:08:50.310996+0000) 2024-09-11T22:09:15.342 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:15 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:15.027+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:30.311301+0000 front 2024-09-11T22:08:30.311235+0000 (oldest deadline 2024-09-11T22:08:59.712086+0000) 2024-09-11T22:09:15.342 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:15 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:15.027+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:39.712759+0000 front 2024-09-11T22:08:39.712634+0000 (oldest deadline 2024-09-11T22:09:03.812485+0000) 2024-09-11T22:09:15.342 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:15 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:15.027+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:47.214214+0000 front 2024-09-11T22:08:47.214135+0000 (oldest deadline 2024-09-11T22:09:12.513937+0000) 2024-09-11T22:09:15.638 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:15 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:15.303+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.481596+0000 front 2024-09-11T22:08:17.481499+0000 (oldest deadline 2024-09-11T22:08:42.181347+0000) 2024-09-11T22:09:15.639 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:15 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:15.303+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:31.383481+0000 front 2024-09-11T22:08:31.383448+0000 (oldest deadline 2024-09-11T22:08:56.083565+0000) 2024-09-11T22:09:15.639 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:15 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:15.303+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:38.384885+0000 front 2024-09-11T22:08:38.385010+0000 (oldest deadline 2024-09-11T22:09:03.684801+0000) 2024-09-11T22:09:15.639 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:15 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:15.303+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.686290+0000 front 2024-09-11T22:08:50.686225+0000 (oldest deadline 2024-09-11T22:09:11.186328+0000) 2024-09-11T22:09:15.943 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:15 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:15.464+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.497679+0000 front 2024-09-11T22:08:16.497606+0000 (oldest deadline 2024-09-11T22:08:42.397744+0000) 2024-09-11T22:09:15.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:15 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:15.464+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:22.398445+0000 front 2024-09-11T22:08:22.398495+0000 (oldest deadline 2024-09-11T22:08:47.098448+0000) 2024-09-11T22:09:15.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:15 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:15.464+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:32.999892+0000 front 2024-09-11T22:08:32.999692+0000 (oldest deadline 2024-09-11T22:08:58.299748+0000) 2024-09-11T22:09:15.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:15 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:15.464+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:40.000748+0000 front 2024-09-11T22:08:40.000701+0000 (oldest deadline 2024-09-11T22:09:03.500864+0000) 2024-09-11T22:09:15.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:15 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:15.464+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.502512+0000 front 2024-09-11T22:08:50.502535+0000 (oldest deadline 2024-09-11T22:09:14.002461+0000) 2024-09-11T22:09:16.032 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 22:09:15 smithi053 podman[70632]: 2024-09-11 22:09:15.638394859 +0000 UTC m=+5.693844477 container died 74733ddfb279a1012e83184de5dd95bb7e2608a66eba63300e8cdc925bad74db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-7, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3) 2024-09-11T22:09:16.091 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:15 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:15.996+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:26.210992+0000 front 2024-09-11T22:08:26.211085+0000 (oldest deadline 2024-09-11T22:08:50.310996+0000) 2024-09-11T22:09:16.092 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:15 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:15.996+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:30.311301+0000 front 2024-09-11T22:08:30.311235+0000 (oldest deadline 2024-09-11T22:08:59.712086+0000) 2024-09-11T22:09:16.092 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:15 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:15.996+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:39.712759+0000 front 2024-09-11T22:08:39.712634+0000 (oldest deadline 2024-09-11T22:09:03.812485+0000) 2024-09-11T22:09:16.092 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:15 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:15.996+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:47.214214+0000 front 2024-09-11T22:08:47.214135+0000 (oldest deadline 2024-09-11T22:09:12.513937+0000) 2024-09-11T22:09:16.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:15 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:15.758+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.893190+0000 front 2024-09-11T22:08:16.893259+0000 (oldest deadline 2024-09-11T22:08:39.193066+0000) 2024-09-11T22:09:16.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:15 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:15.758+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:21.494266+0000 front 2024-09-11T22:08:21.494360+0000 (oldest deadline 2024-09-11T22:08:47.394416+0000) 2024-09-11T22:09:16.093 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:15 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:15.758+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:33.295337+0000 front 2024-09-11T22:08:33.295289+0000 (oldest deadline 2024-09-11T22:08:56.795428+0000) 2024-09-11T22:09:16.093 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:15 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:15.758+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:39.497398+0000 front 2024-09-11T22:08:39.497364+0000 (oldest deadline 2024-09-11T22:09:04.797240+0000) 2024-09-11T22:09:16.093 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:15 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:15.758+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.598954+0000 front 2024-09-11T22:08:50.598798+0000 (oldest deadline 2024-09-11T22:09:15.199166+0000) 2024-09-11T22:09:16.297 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 22:09:16 smithi053 podman[70632]: 2024-09-11 22:09:16.131000918 +0000 UTC m=+6.186450537 container cleanup 74733ddfb279a1012e83184de5dd95bb7e2608a66eba63300e8cdc925bad74db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-7, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-11T22:09:16.298 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 22:09:16 smithi053 bash[70632]: ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-7 2024-09-11T22:09:16.706 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 22:09:16 smithi053 podman[70659]: 2024-09-11 22:09:16.365794031 +0000 UTC m=+0.723560954 container remove 74733ddfb279a1012e83184de5dd95bb7e2608a66eba63300e8cdc925bad74db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-11T22:09:16.706 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:16 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:16.316+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.481596+0000 front 2024-09-11T22:08:17.481499+0000 (oldest deadline 2024-09-11T22:08:42.181347+0000) 2024-09-11T22:09:16.706 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:16 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:16.316+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:31.383481+0000 front 2024-09-11T22:08:31.383448+0000 (oldest deadline 2024-09-11T22:08:56.083565+0000) 2024-09-11T22:09:16.706 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:16 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:16.316+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:38.384885+0000 front 2024-09-11T22:08:38.385010+0000 (oldest deadline 2024-09-11T22:09:03.684801+0000) 2024-09-11T22:09:16.707 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:16 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:16.316+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.686290+0000 front 2024-09-11T22:08:50.686225+0000 (oldest deadline 2024-09-11T22:09:11.186328+0000) 2024-09-11T22:09:16.943 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:16 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:16.505+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.497679+0000 front 2024-09-11T22:08:16.497606+0000 (oldest deadline 2024-09-11T22:08:42.397744+0000) 2024-09-11T22:09:16.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:16 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:16.505+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:22.398445+0000 front 2024-09-11T22:08:22.398495+0000 (oldest deadline 2024-09-11T22:08:47.098448+0000) 2024-09-11T22:09:16.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:16 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:16.505+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:32.999892+0000 front 2024-09-11T22:08:32.999692+0000 (oldest deadline 2024-09-11T22:08:58.299748+0000) 2024-09-11T22:09:16.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:16 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:16.505+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:40.000748+0000 front 2024-09-11T22:08:40.000701+0000 (oldest deadline 2024-09-11T22:09:03.500864+0000) 2024-09-11T22:09:16.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:16 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:16.505+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.502512+0000 front 2024-09-11T22:08:50.502535+0000 (oldest deadline 2024-09-11T22:09:14.002461+0000) 2024-09-11T22:09:17.039 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:16 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:16.788+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.893190+0000 front 2024-09-11T22:08:16.893259+0000 (oldest deadline 2024-09-11T22:08:39.193066+0000) 2024-09-11T22:09:17.039 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:16 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:16.788+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:21.494266+0000 front 2024-09-11T22:08:21.494360+0000 (oldest deadline 2024-09-11T22:08:47.394416+0000) 2024-09-11T22:09:17.039 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:16 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:16.788+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:33.295337+0000 front 2024-09-11T22:08:33.295289+0000 (oldest deadline 2024-09-11T22:08:56.795428+0000) 2024-09-11T22:09:17.040 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:16 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:16.788+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:39.497398+0000 front 2024-09-11T22:08:39.497364+0000 (oldest deadline 2024-09-11T22:09:04.797240+0000) 2024-09-11T22:09:17.040 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:16 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:16.788+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.598954+0000 front 2024-09-11T22:08:50.598798+0000 (oldest deadline 2024-09-11T22:09:15.199166+0000) 2024-09-11T22:09:17.047 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 22:09:16 smithi053 podman[70751]: 2024-09-11 22:09:16.608564188 +0000 UTC m=+0.015948400 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T22:09:17.047 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 22:09:16 smithi053 podman[70751]: 2024-09-11 22:09:16.716734495 +0000 UTC m=+0.124118739 container create de41817a1d99773ab44c0c76cf7d56ed35bbef9748ffe4627cb5e1d7db29c674 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T22:09:17.301 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:17 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:17.280+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.481596+0000 front 2024-09-11T22:08:17.481499+0000 (oldest deadline 2024-09-11T22:08:42.181347+0000) 2024-09-11T22:09:17.302 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:17 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:17.280+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:31.383481+0000 front 2024-09-11T22:08:31.383448+0000 (oldest deadline 2024-09-11T22:08:56.083565+0000) 2024-09-11T22:09:17.302 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:17 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:17.280+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:38.384885+0000 front 2024-09-11T22:08:38.385010+0000 (oldest deadline 2024-09-11T22:09:03.684801+0000) 2024-09-11T22:09:17.302 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:17 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:17.280+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.686290+0000 front 2024-09-11T22:08:50.686225+0000 (oldest deadline 2024-09-11T22:09:11.186328+0000) 2024-09-11T22:09:17.302 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 22:09:17 smithi053 podman[70751]: 2024-09-11 22:09:17.102737431 +0000 UTC m=+0.510121632 container init de41817a1d99773ab44c0c76cf7d56ed35bbef9748ffe4627cb5e1d7db29c674 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-11T22:09:17.303 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 22:09:17 smithi053 podman[70751]: 2024-09-11 22:09:17.106522174 +0000 UTC m=+0.513906385 container start de41817a1d99773ab44c0c76cf7d56ed35bbef9748ffe4627cb5e1d7db29c674 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-11T22:09:17.303 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 22:09:17 smithi053 podman[70751]: 2024-09-11 22:09:17.186018297 +0000 UTC m=+0.593402496 container attach de41817a1d99773ab44c0c76cf7d56ed35bbef9748ffe4627cb5e1d7db29c674 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-11T22:09:17.303 INFO:journalctl@ceph.osd.7.smithi053.stdout:Sep 11 22:09:17 smithi053 podman[70751]: 2024-09-11 22:09:17.301615579 +0000 UTC m=+0.708999803 container died de41817a1d99773ab44c0c76cf7d56ed35bbef9748ffe4627cb5e1d7db29c674 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-7-deactivate, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3) 2024-09-11T22:09:17.341 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:17 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:17.038+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:26.210992+0000 front 2024-09-11T22:08:26.211085+0000 (oldest deadline 2024-09-11T22:08:50.310996+0000) 2024-09-11T22:09:17.342 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:17 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:17.038+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:30.311301+0000 front 2024-09-11T22:08:30.311235+0000 (oldest deadline 2024-09-11T22:08:59.712086+0000) 2024-09-11T22:09:17.342 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:17 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:17.038+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:39.712759+0000 front 2024-09-11T22:08:39.712634+0000 (oldest deadline 2024-09-11T22:09:03.812485+0000) 2024-09-11T22:09:17.342 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:17 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:17.038+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:47.214214+0000 front 2024-09-11T22:08:47.214135+0000 (oldest deadline 2024-09-11T22:09:12.513937+0000) 2024-09-11T22:09:17.943 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:17 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:17.543+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.497679+0000 front 2024-09-11T22:08:16.497606+0000 (oldest deadline 2024-09-11T22:08:42.397744+0000) 2024-09-11T22:09:17.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:17 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:17.543+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:22.398445+0000 front 2024-09-11T22:08:22.398495+0000 (oldest deadline 2024-09-11T22:08:47.098448+0000) 2024-09-11T22:09:17.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:17 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:17.543+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:32.999892+0000 front 2024-09-11T22:08:32.999692+0000 (oldest deadline 2024-09-11T22:08:58.299748+0000) 2024-09-11T22:09:17.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:17 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:17.543+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:40.000748+0000 front 2024-09-11T22:08:40.000701+0000 (oldest deadline 2024-09-11T22:09:03.500864+0000) 2024-09-11T22:09:17.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:17 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:17.543+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.502512+0000 front 2024-09-11T22:08:50.502535+0000 (oldest deadline 2024-09-11T22:09:14.002461+0000) 2024-09-11T22:09:18.048 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:17 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:17.741+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.893190+0000 front 2024-09-11T22:08:16.893259+0000 (oldest deadline 2024-09-11T22:08:39.193066+0000) 2024-09-11T22:09:18.049 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:17 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:17.741+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:21.494266+0000 front 2024-09-11T22:08:21.494360+0000 (oldest deadline 2024-09-11T22:08:47.394416+0000) 2024-09-11T22:09:18.049 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:17 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:17.741+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:33.295337+0000 front 2024-09-11T22:08:33.295289+0000 (oldest deadline 2024-09-11T22:08:56.795428+0000) 2024-09-11T22:09:18.049 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:17 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:17.741+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:39.497398+0000 front 2024-09-11T22:08:39.497364+0000 (oldest deadline 2024-09-11T22:09:04.797240+0000) 2024-09-11T22:09:18.049 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:17 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:17.741+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.598954+0000 front 2024-09-11T22:08:50.598798+0000 (oldest deadline 2024-09-11T22:09:15.199166+0000) 2024-09-11T22:09:18.077 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3ffca83c-7087-11ef-bcea-c7b262605968@osd.7.service' 2024-09-11T22:09:18.124 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:09:18.124 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-11T22:09:18.124 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-11T22:09:18.125 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-3ffca83c-7087-11ef-bcea-c7b262605968@osd.8 2024-09-11T22:09:18.342 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:18 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:18.048+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:26.210992+0000 front 2024-09-11T22:08:26.211085+0000 (oldest deadline 2024-09-11T22:08:50.310996+0000) 2024-09-11T22:09:18.342 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:18 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:18.048+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:30.311301+0000 front 2024-09-11T22:08:30.311235+0000 (oldest deadline 2024-09-11T22:08:59.712086+0000) 2024-09-11T22:09:18.342 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:18 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:18.048+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:39.712759+0000 front 2024-09-11T22:08:39.712634+0000 (oldest deadline 2024-09-11T22:09:03.812485+0000) 2024-09-11T22:09:18.342 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:18 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:18.048+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:47.214214+0000 front 2024-09-11T22:08:47.214135+0000 (oldest deadline 2024-09-11T22:09:12.513937+0000) 2024-09-11T22:09:18.342 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:18 smithi161 systemd[1]: Stopping Ceph osd.8 for 3ffca83c-7087-11ef-bcea-c7b262605968... 2024-09-11T22:09:18.548 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:18 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:18.243+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.481596+0000 front 2024-09-11T22:08:17.481499+0000 (oldest deadline 2024-09-11T22:08:42.181347+0000) 2024-09-11T22:09:18.548 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:18 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:18.243+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:31.383481+0000 front 2024-09-11T22:08:31.383448+0000 (oldest deadline 2024-09-11T22:08:56.083565+0000) 2024-09-11T22:09:18.548 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:18 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:18.243+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:38.384885+0000 front 2024-09-11T22:08:38.385010+0000 (oldest deadline 2024-09-11T22:09:03.684801+0000) 2024-09-11T22:09:18.548 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:18 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:18.243+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.686290+0000 front 2024-09-11T22:08:50.686225+0000 (oldest deadline 2024-09-11T22:09:11.186328+0000) 2024-09-11T22:09:18.779 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:18 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:18.437+0000 7f350fa59640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:09:18.779 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:18 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:18.437+0000 7f350fa59640 -1 osd.8 98 *** Got signal Terminated *** 2024-09-11T22:09:18.779 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:18 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:18.437+0000 7f350fa59640 -1 osd.8 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T22:09:18.943 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:18 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:18.588+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.497679+0000 front 2024-09-11T22:08:16.497606+0000 (oldest deadline 2024-09-11T22:08:42.397744+0000) 2024-09-11T22:09:18.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:18 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:18.588+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:22.398445+0000 front 2024-09-11T22:08:22.398495+0000 (oldest deadline 2024-09-11T22:08:47.098448+0000) 2024-09-11T22:09:18.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:18 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:18.588+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:32.999892+0000 front 2024-09-11T22:08:32.999692+0000 (oldest deadline 2024-09-11T22:08:58.299748+0000) 2024-09-11T22:09:18.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:18 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:18.588+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:40.000748+0000 front 2024-09-11T22:08:40.000701+0000 (oldest deadline 2024-09-11T22:09:03.500864+0000) 2024-09-11T22:09:18.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:18 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:18.588+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.502512+0000 front 2024-09-11T22:08:50.502535+0000 (oldest deadline 2024-09-11T22:09:14.002461+0000) 2024-09-11T22:09:19.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:18 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:18.777+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.893190+0000 front 2024-09-11T22:08:16.893259+0000 (oldest deadline 2024-09-11T22:08:39.193066+0000) 2024-09-11T22:09:19.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:18 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:18.777+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:21.494266+0000 front 2024-09-11T22:08:21.494360+0000 (oldest deadline 2024-09-11T22:08:47.394416+0000) 2024-09-11T22:09:19.093 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:18 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:18.777+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:33.295337+0000 front 2024-09-11T22:08:33.295289+0000 (oldest deadline 2024-09-11T22:08:56.795428+0000) 2024-09-11T22:09:19.093 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:18 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:18.777+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:39.497398+0000 front 2024-09-11T22:08:39.497364+0000 (oldest deadline 2024-09-11T22:09:04.797240+0000) 2024-09-11T22:09:19.093 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:18 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:18.777+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.598954+0000 front 2024-09-11T22:08:50.598798+0000 (oldest deadline 2024-09-11T22:09:15.199166+0000) 2024-09-11T22:09:19.093 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:19 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:19.015+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:26.210992+0000 front 2024-09-11T22:08:26.211085+0000 (oldest deadline 2024-09-11T22:08:50.310996+0000) 2024-09-11T22:09:19.093 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:19 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:19.015+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:30.311301+0000 front 2024-09-11T22:08:30.311235+0000 (oldest deadline 2024-09-11T22:08:59.712086+0000) 2024-09-11T22:09:19.093 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:19 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:19.015+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:39.712759+0000 front 2024-09-11T22:08:39.712634+0000 (oldest deadline 2024-09-11T22:09:03.812485+0000) 2024-09-11T22:09:19.094 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:19 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:19.015+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:47.214214+0000 front 2024-09-11T22:08:47.214135+0000 (oldest deadline 2024-09-11T22:09:12.513937+0000) 2024-09-11T22:09:19.547 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:19 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:19.209+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.481596+0000 front 2024-09-11T22:08:17.481499+0000 (oldest deadline 2024-09-11T22:08:42.181347+0000) 2024-09-11T22:09:19.548 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:19 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:19.209+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:31.383481+0000 front 2024-09-11T22:08:31.383448+0000 (oldest deadline 2024-09-11T22:08:56.083565+0000) 2024-09-11T22:09:19.548 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:19 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:19.209+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:38.384885+0000 front 2024-09-11T22:08:38.385010+0000 (oldest deadline 2024-09-11T22:09:03.684801+0000) 2024-09-11T22:09:19.548 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:19 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:19.209+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.686290+0000 front 2024-09-11T22:08:50.686225+0000 (oldest deadline 2024-09-11T22:09:11.186328+0000) 2024-09-11T22:09:19.943 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:19 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:19.626+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.497679+0000 front 2024-09-11T22:08:16.497606+0000 (oldest deadline 2024-09-11T22:08:42.397744+0000) 2024-09-11T22:09:19.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:19 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:19.626+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:22.398445+0000 front 2024-09-11T22:08:22.398495+0000 (oldest deadline 2024-09-11T22:08:47.098448+0000) 2024-09-11T22:09:19.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:19 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:19.626+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:32.999892+0000 front 2024-09-11T22:08:32.999692+0000 (oldest deadline 2024-09-11T22:08:58.299748+0000) 2024-09-11T22:09:19.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:19 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:19.626+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:40.000748+0000 front 2024-09-11T22:08:40.000701+0000 (oldest deadline 2024-09-11T22:09:03.500864+0000) 2024-09-11T22:09:19.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:19 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:19.626+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.502512+0000 front 2024-09-11T22:08:50.502535+0000 (oldest deadline 2024-09-11T22:09:14.002461+0000) 2024-09-11T22:09:20.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:19 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:19.779+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.893190+0000 front 2024-09-11T22:08:16.893259+0000 (oldest deadline 2024-09-11T22:08:39.193066+0000) 2024-09-11T22:09:20.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:19 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:19.779+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:21.494266+0000 front 2024-09-11T22:08:21.494360+0000 (oldest deadline 2024-09-11T22:08:47.394416+0000) 2024-09-11T22:09:20.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:19 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:19.779+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:33.295337+0000 front 2024-09-11T22:08:33.295289+0000 (oldest deadline 2024-09-11T22:08:56.795428+0000) 2024-09-11T22:09:20.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:19 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:19.779+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:39.497398+0000 front 2024-09-11T22:08:39.497364+0000 (oldest deadline 2024-09-11T22:09:04.797240+0000) 2024-09-11T22:09:20.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:19 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:19.779+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.598954+0000 front 2024-09-11T22:08:50.598798+0000 (oldest deadline 2024-09-11T22:09:15.199166+0000) 2024-09-11T22:09:20.093 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:20 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:20.004+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:26.210992+0000 front 2024-09-11T22:08:26.211085+0000 (oldest deadline 2024-09-11T22:08:50.310996+0000) 2024-09-11T22:09:20.093 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:20 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:20.004+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:30.311301+0000 front 2024-09-11T22:08:30.311235+0000 (oldest deadline 2024-09-11T22:08:59.712086+0000) 2024-09-11T22:09:20.093 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:20 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:20.004+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:39.712759+0000 front 2024-09-11T22:08:39.712634+0000 (oldest deadline 2024-09-11T22:09:03.812485+0000) 2024-09-11T22:09:20.093 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:20 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:20.004+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:47.214214+0000 front 2024-09-11T22:08:47.214135+0000 (oldest deadline 2024-09-11T22:09:12.513937+0000) 2024-09-11T22:09:20.548 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:20 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:20.240+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.481596+0000 front 2024-09-11T22:08:17.481499+0000 (oldest deadline 2024-09-11T22:08:42.181347+0000) 2024-09-11T22:09:20.548 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:20 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:20.240+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:31.383481+0000 front 2024-09-11T22:08:31.383448+0000 (oldest deadline 2024-09-11T22:08:56.083565+0000) 2024-09-11T22:09:20.548 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:20 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:20.240+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:38.384885+0000 front 2024-09-11T22:08:38.385010+0000 (oldest deadline 2024-09-11T22:09:03.684801+0000) 2024-09-11T22:09:20.548 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:20 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:20.240+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.686290+0000 front 2024-09-11T22:08:50.686225+0000 (oldest deadline 2024-09-11T22:09:11.186328+0000) 2024-09-11T22:09:20.943 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:20 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:20.665+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.497679+0000 front 2024-09-11T22:08:16.497606+0000 (oldest deadline 2024-09-11T22:08:42.397744+0000) 2024-09-11T22:09:20.943 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:20 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:20.665+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:22.398445+0000 front 2024-09-11T22:08:22.398495+0000 (oldest deadline 2024-09-11T22:08:47.098448+0000) 2024-09-11T22:09:20.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:20 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:20.665+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:32.999892+0000 front 2024-09-11T22:08:32.999692+0000 (oldest deadline 2024-09-11T22:08:58.299748+0000) 2024-09-11T22:09:20.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:20 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:20.665+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:40.000748+0000 front 2024-09-11T22:08:40.000701+0000 (oldest deadline 2024-09-11T22:09:03.500864+0000) 2024-09-11T22:09:20.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:20 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:20.665+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.502512+0000 front 2024-09-11T22:08:50.502535+0000 (oldest deadline 2024-09-11T22:09:14.002461+0000) 2024-09-11T22:09:21.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:20 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:20.811+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.893190+0000 front 2024-09-11T22:08:16.893259+0000 (oldest deadline 2024-09-11T22:08:39.193066+0000) 2024-09-11T22:09:21.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:20 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:20.811+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:21.494266+0000 front 2024-09-11T22:08:21.494360+0000 (oldest deadline 2024-09-11T22:08:47.394416+0000) 2024-09-11T22:09:21.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:20 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:20.811+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:33.295337+0000 front 2024-09-11T22:08:33.295289+0000 (oldest deadline 2024-09-11T22:08:56.795428+0000) 2024-09-11T22:09:21.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:20 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:20.811+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:39.497398+0000 front 2024-09-11T22:08:39.497364+0000 (oldest deadline 2024-09-11T22:09:04.797240+0000) 2024-09-11T22:09:21.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:20 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:20.811+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.598954+0000 front 2024-09-11T22:08:50.598798+0000 (oldest deadline 2024-09-11T22:09:15.199166+0000) 2024-09-11T22:09:21.093 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:20 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:20.995+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:26.210992+0000 front 2024-09-11T22:08:26.211085+0000 (oldest deadline 2024-09-11T22:08:50.310996+0000) 2024-09-11T22:09:21.093 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:20 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:20.995+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:30.311301+0000 front 2024-09-11T22:08:30.311235+0000 (oldest deadline 2024-09-11T22:08:59.712086+0000) 2024-09-11T22:09:21.093 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:20 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:20.995+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:39.712759+0000 front 2024-09-11T22:08:39.712634+0000 (oldest deadline 2024-09-11T22:09:03.812485+0000) 2024-09-11T22:09:21.093 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:20 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:20.995+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:47.214214+0000 front 2024-09-11T22:08:47.214135+0000 (oldest deadline 2024-09-11T22:09:12.513937+0000) 2024-09-11T22:09:21.548 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:21 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:21.284+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.481596+0000 front 2024-09-11T22:08:17.481499+0000 (oldest deadline 2024-09-11T22:08:42.181347+0000) 2024-09-11T22:09:21.548 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:21 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:21.284+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:31.383481+0000 front 2024-09-11T22:08:31.383448+0000 (oldest deadline 2024-09-11T22:08:56.083565+0000) 2024-09-11T22:09:21.548 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:21 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:21.284+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:38.384885+0000 front 2024-09-11T22:08:38.385010+0000 (oldest deadline 2024-09-11T22:09:03.684801+0000) 2024-09-11T22:09:21.548 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:21 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:21.284+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.686290+0000 front 2024-09-11T22:08:50.686225+0000 (oldest deadline 2024-09-11T22:09:11.186328+0000) 2024-09-11T22:09:21.943 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:21 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:21.620+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.497679+0000 front 2024-09-11T22:08:16.497606+0000 (oldest deadline 2024-09-11T22:08:42.397744+0000) 2024-09-11T22:09:21.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:21 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:21.620+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:22.398445+0000 front 2024-09-11T22:08:22.398495+0000 (oldest deadline 2024-09-11T22:08:47.098448+0000) 2024-09-11T22:09:21.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:21 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:21.620+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:32.999892+0000 front 2024-09-11T22:08:32.999692+0000 (oldest deadline 2024-09-11T22:08:58.299748+0000) 2024-09-11T22:09:21.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:21 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:21.620+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:40.000748+0000 front 2024-09-11T22:08:40.000701+0000 (oldest deadline 2024-09-11T22:09:03.500864+0000) 2024-09-11T22:09:21.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:21 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:21.620+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.502512+0000 front 2024-09-11T22:08:50.502535+0000 (oldest deadline 2024-09-11T22:09:14.002461+0000) 2024-09-11T22:09:21.945 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:21 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:21.620+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-11T22:08:58.103409+0000 front 2024-09-11T22:08:58.103489+0000 (oldest deadline 2024-09-11T22:09:21.504260+0000) 2024-09-11T22:09:22.342 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:21 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:21.849+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.893190+0000 front 2024-09-11T22:08:16.893259+0000 (oldest deadline 2024-09-11T22:08:39.193066+0000) 2024-09-11T22:09:22.342 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:21 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:21.849+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:21.494266+0000 front 2024-09-11T22:08:21.494360+0000 (oldest deadline 2024-09-11T22:08:47.394416+0000) 2024-09-11T22:09:22.342 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:21 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:21.849+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:33.295337+0000 front 2024-09-11T22:08:33.295289+0000 (oldest deadline 2024-09-11T22:08:56.795428+0000) 2024-09-11T22:09:22.342 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:21 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:21.849+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:39.497398+0000 front 2024-09-11T22:08:39.497364+0000 (oldest deadline 2024-09-11T22:09:04.797240+0000) 2024-09-11T22:09:22.342 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:21 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:21.849+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.598954+0000 front 2024-09-11T22:08:50.598798+0000 (oldest deadline 2024-09-11T22:09:15.199166+0000) 2024-09-11T22:09:22.343 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:22 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:22.040+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:26.210992+0000 front 2024-09-11T22:08:26.211085+0000 (oldest deadline 2024-09-11T22:08:50.310996+0000) 2024-09-11T22:09:22.343 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:22 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:22.040+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:30.311301+0000 front 2024-09-11T22:08:30.311235+0000 (oldest deadline 2024-09-11T22:08:59.712086+0000) 2024-09-11T22:09:22.343 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:22 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:22.040+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:39.712759+0000 front 2024-09-11T22:08:39.712634+0000 (oldest deadline 2024-09-11T22:09:03.812485+0000) 2024-09-11T22:09:22.343 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:22 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:22.040+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:47.214214+0000 front 2024-09-11T22:08:47.214135+0000 (oldest deadline 2024-09-11T22:09:12.513937+0000) 2024-09-11T22:09:22.343 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:22 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:22.040+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-11T22:08:58.116025+0000 front 2024-09-11T22:08:58.116043+0000 (oldest deadline 2024-09-11T22:09:21.016226+0000) 2024-09-11T22:09:22.797 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:22 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:22.321+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.481596+0000 front 2024-09-11T22:08:17.481499+0000 (oldest deadline 2024-09-11T22:08:42.181347+0000) 2024-09-11T22:09:22.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:22 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:22.321+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:31.383481+0000 front 2024-09-11T22:08:31.383448+0000 (oldest deadline 2024-09-11T22:08:56.083565+0000) 2024-09-11T22:09:22.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:22 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:22.321+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:38.384885+0000 front 2024-09-11T22:08:38.385010+0000 (oldest deadline 2024-09-11T22:09:03.684801+0000) 2024-09-11T22:09:22.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:22 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:22.321+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.686290+0000 front 2024-09-11T22:08:50.686225+0000 (oldest deadline 2024-09-11T22:09:11.186328+0000) 2024-09-11T22:09:22.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:22 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:22.321+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-11T22:08:53.487319+0000 front 2024-09-11T22:08:53.487287+0000 (oldest deadline 2024-09-11T22:09:22.287607+0000) 2024-09-11T22:09:22.943 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:22 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:22.611+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.497679+0000 front 2024-09-11T22:08:16.497606+0000 (oldest deadline 2024-09-11T22:08:42.397744+0000) 2024-09-11T22:09:22.943 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:22 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:22.611+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:22.398445+0000 front 2024-09-11T22:08:22.398495+0000 (oldest deadline 2024-09-11T22:08:47.098448+0000) 2024-09-11T22:09:22.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:22 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:22.611+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:32.999892+0000 front 2024-09-11T22:08:32.999692+0000 (oldest deadline 2024-09-11T22:08:58.299748+0000) 2024-09-11T22:09:22.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:22 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:22.611+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:40.000748+0000 front 2024-09-11T22:08:40.000701+0000 (oldest deadline 2024-09-11T22:09:03.500864+0000) 2024-09-11T22:09:22.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:22 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:22.611+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.502512+0000 front 2024-09-11T22:08:50.502535+0000 (oldest deadline 2024-09-11T22:09:14.002461+0000) 2024-09-11T22:09:22.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:22 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:22.611+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-11T22:08:58.103409+0000 front 2024-09-11T22:08:58.103489+0000 (oldest deadline 2024-09-11T22:09:21.504260+0000) 2024-09-11T22:09:23.089 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:22 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:22.876+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.893190+0000 front 2024-09-11T22:08:16.893259+0000 (oldest deadline 2024-09-11T22:08:39.193066+0000) 2024-09-11T22:09:23.089 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:22 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:22.876+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:21.494266+0000 front 2024-09-11T22:08:21.494360+0000 (oldest deadline 2024-09-11T22:08:47.394416+0000) 2024-09-11T22:09:23.089 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:22 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:22.876+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:33.295337+0000 front 2024-09-11T22:08:33.295289+0000 (oldest deadline 2024-09-11T22:08:56.795428+0000) 2024-09-11T22:09:23.089 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:22 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:22.876+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:39.497398+0000 front 2024-09-11T22:08:39.497364+0000 (oldest deadline 2024-09-11T22:09:04.797240+0000) 2024-09-11T22:09:23.090 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:22 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:22.876+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.598954+0000 front 2024-09-11T22:08:50.598798+0000 (oldest deadline 2024-09-11T22:09:15.199166+0000) 2024-09-11T22:09:23.342 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:23 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:23.088+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:26.210992+0000 front 2024-09-11T22:08:26.211085+0000 (oldest deadline 2024-09-11T22:08:50.310996+0000) 2024-09-11T22:09:23.342 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:23 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:23.088+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:30.311301+0000 front 2024-09-11T22:08:30.311235+0000 (oldest deadline 2024-09-11T22:08:59.712086+0000) 2024-09-11T22:09:23.342 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:23 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:23.088+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:39.712759+0000 front 2024-09-11T22:08:39.712634+0000 (oldest deadline 2024-09-11T22:09:03.812485+0000) 2024-09-11T22:09:23.342 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:23 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:23.088+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:47.214214+0000 front 2024-09-11T22:08:47.214135+0000 (oldest deadline 2024-09-11T22:09:12.513937+0000) 2024-09-11T22:09:23.342 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:23 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8[54268]: 2024-09-11T22:09:23.088+0000 7f3508067640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-11T22:08:58.116025+0000 front 2024-09-11T22:08:58.116043+0000 (oldest deadline 2024-09-11T22:09:21.016226+0000) 2024-09-11T22:09:23.797 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:23 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:23.326+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.481596+0000 front 2024-09-11T22:08:17.481499+0000 (oldest deadline 2024-09-11T22:08:42.181347+0000) 2024-09-11T22:09:23.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:23 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:23.326+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:31.383481+0000 front 2024-09-11T22:08:31.383448+0000 (oldest deadline 2024-09-11T22:08:56.083565+0000) 2024-09-11T22:09:23.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:23 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:23.326+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:38.384885+0000 front 2024-09-11T22:08:38.385010+0000 (oldest deadline 2024-09-11T22:09:03.684801+0000) 2024-09-11T22:09:23.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:23 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:23.326+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.686290+0000 front 2024-09-11T22:08:50.686225+0000 (oldest deadline 2024-09-11T22:09:11.186328+0000) 2024-09-11T22:09:23.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:23 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:23.326+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-11T22:08:53.487319+0000 front 2024-09-11T22:08:53.487287+0000 (oldest deadline 2024-09-11T22:09:22.287607+0000) 2024-09-11T22:09:23.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:23 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:23.644+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.497679+0000 front 2024-09-11T22:08:16.497606+0000 (oldest deadline 2024-09-11T22:08:42.397744+0000) 2024-09-11T22:09:23.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:23 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:23.644+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:22.398445+0000 front 2024-09-11T22:08:22.398495+0000 (oldest deadline 2024-09-11T22:08:47.098448+0000) 2024-09-11T22:09:23.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:23 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:23.644+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:32.999892+0000 front 2024-09-11T22:08:32.999692+0000 (oldest deadline 2024-09-11T22:08:58.299748+0000) 2024-09-11T22:09:23.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:23 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:23.644+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:40.000748+0000 front 2024-09-11T22:08:40.000701+0000 (oldest deadline 2024-09-11T22:09:03.500864+0000) 2024-09-11T22:09:23.945 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:23 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:23.644+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.502512+0000 front 2024-09-11T22:08:50.502535+0000 (oldest deadline 2024-09-11T22:09:14.002461+0000) 2024-09-11T22:09:23.945 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:23 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:23.644+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-11T22:08:58.103409+0000 front 2024-09-11T22:08:58.103489+0000 (oldest deadline 2024-09-11T22:09:21.504260+0000) 2024-09-11T22:09:24.092 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:23 smithi161 podman[72092]: 2024-09-11 22:09:23.974038179 +0000 UTC m=+5.649196789 container died 603b13a6fa1ca5f32fb6acf4c0cac7e0493d3d0fd86531843f5d36c335664b23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=) 2024-09-11T22:09:24.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:23 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:23.837+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.893190+0000 front 2024-09-11T22:08:16.893259+0000 (oldest deadline 2024-09-11T22:08:39.193066+0000) 2024-09-11T22:09:24.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:23 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:23.837+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:21.494266+0000 front 2024-09-11T22:08:21.494360+0000 (oldest deadline 2024-09-11T22:08:47.394416+0000) 2024-09-11T22:09:24.093 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:23 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:23.837+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:33.295337+0000 front 2024-09-11T22:08:33.295289+0000 (oldest deadline 2024-09-11T22:08:56.795428+0000) 2024-09-11T22:09:24.093 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:23 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:23.837+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:39.497398+0000 front 2024-09-11T22:08:39.497364+0000 (oldest deadline 2024-09-11T22:09:04.797240+0000) 2024-09-11T22:09:24.093 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:23 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:23.837+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.598954+0000 front 2024-09-11T22:08:50.598798+0000 (oldest deadline 2024-09-11T22:09:15.199166+0000) 2024-09-11T22:09:24.093 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:23 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:23.837+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-11T22:08:58.100016+0000 front 2024-09-11T22:08:58.100085+0000 (oldest deadline 2024-09-11T22:09:23.400061+0000) 2024-09-11T22:09:24.592 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:24 smithi161 podman[72092]: 2024-09-11 22:09:24.338597413 +0000 UTC m=+6.013756008 container cleanup 603b13a6fa1ca5f32fb6acf4c0cac7e0493d3d0fd86531843f5d36c335664b23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3) 2024-09-11T22:09:24.592 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:24 smithi161 bash[72092]: ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8 2024-09-11T22:09:24.797 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:24 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:24.360+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.481596+0000 front 2024-09-11T22:08:17.481499+0000 (oldest deadline 2024-09-11T22:08:42.181347+0000) 2024-09-11T22:09:24.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:24 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:24.360+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:31.383481+0000 front 2024-09-11T22:08:31.383448+0000 (oldest deadline 2024-09-11T22:08:56.083565+0000) 2024-09-11T22:09:24.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:24 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:24.360+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:38.384885+0000 front 2024-09-11T22:08:38.385010+0000 (oldest deadline 2024-09-11T22:09:03.684801+0000) 2024-09-11T22:09:24.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:24 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:24.360+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.686290+0000 front 2024-09-11T22:08:50.686225+0000 (oldest deadline 2024-09-11T22:09:11.186328+0000) 2024-09-11T22:09:24.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:24 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:24.360+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-11T22:08:53.487319+0000 front 2024-09-11T22:08:53.487287+0000 (oldest deadline 2024-09-11T22:09:22.287607+0000) 2024-09-11T22:09:24.943 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:24 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:24.654+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.497679+0000 front 2024-09-11T22:08:16.497606+0000 (oldest deadline 2024-09-11T22:08:42.397744+0000) 2024-09-11T22:09:24.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:24 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:24.654+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:22.398445+0000 front 2024-09-11T22:08:22.398495+0000 (oldest deadline 2024-09-11T22:08:47.098448+0000) 2024-09-11T22:09:24.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:24 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:24.654+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:32.999892+0000 front 2024-09-11T22:08:32.999692+0000 (oldest deadline 2024-09-11T22:08:58.299748+0000) 2024-09-11T22:09:24.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:24 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:24.654+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:40.000748+0000 front 2024-09-11T22:08:40.000701+0000 (oldest deadline 2024-09-11T22:09:03.500864+0000) 2024-09-11T22:09:24.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:24 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:24.654+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.502512+0000 front 2024-09-11T22:08:50.502535+0000 (oldest deadline 2024-09-11T22:09:14.002461+0000) 2024-09-11T22:09:24.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:24 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:24.654+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-11T22:08:58.103409+0000 front 2024-09-11T22:08:58.103489+0000 (oldest deadline 2024-09-11T22:09:21.504260+0000) 2024-09-11T22:09:25.007 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:24 smithi161 podman[72108]: 2024-09-11 22:09:24.656776697 +0000 UTC m=+0.678960368 container remove 603b13a6fa1ca5f32fb6acf4c0cac7e0493d3d0fd86531843f5d36c335664b23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-b53b6d3, ceph=True) 2024-09-11T22:09:25.007 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:24 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:24.829+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.893190+0000 front 2024-09-11T22:08:16.893259+0000 (oldest deadline 2024-09-11T22:08:39.193066+0000) 2024-09-11T22:09:25.007 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:24 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:24.829+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:21.494266+0000 front 2024-09-11T22:08:21.494360+0000 (oldest deadline 2024-09-11T22:08:47.394416+0000) 2024-09-11T22:09:25.007 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:24 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:24.829+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:33.295337+0000 front 2024-09-11T22:08:33.295289+0000 (oldest deadline 2024-09-11T22:08:56.795428+0000) 2024-09-11T22:09:25.008 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:24 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:24.829+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:39.497398+0000 front 2024-09-11T22:08:39.497364+0000 (oldest deadline 2024-09-11T22:09:04.797240+0000) 2024-09-11T22:09:25.008 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:24 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:24.829+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.598954+0000 front 2024-09-11T22:08:50.598798+0000 (oldest deadline 2024-09-11T22:09:15.199166+0000) 2024-09-11T22:09:25.008 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:24 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:24.829+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-11T22:08:58.100016+0000 front 2024-09-11T22:08:58.100085+0000 (oldest deadline 2024-09-11T22:09:23.400061+0000) 2024-09-11T22:09:25.301 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:25 smithi161 podman[72197]: 2024-09-11 22:09:24.909552954 +0000 UTC m=+0.016088363 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T22:09:25.302 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:25 smithi161 podman[72197]: 2024-09-11 22:09:25.032698823 +0000 UTC m=+0.139234287 container create 277fc20f547447b9b115f0dc0e0ac24a86efbe2a3c0f8d9d4cc9e9bb762bf3b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-11T22:09:25.302 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:25 smithi161 podman[72197]: 2024-09-11 22:09:25.243136349 +0000 UTC m=+0.349671755 container init 277fc20f547447b9b115f0dc0e0ac24a86efbe2a3c0f8d9d4cc9e9bb762bf3b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8-deactivate, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True) 2024-09-11T22:09:25.302 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:25 smithi161 podman[72197]: 2024-09-11 22:09:25.246418446 +0000 UTC m=+0.352953844 container start 277fc20f547447b9b115f0dc0e0ac24a86efbe2a3c0f8d9d4cc9e9bb762bf3b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD) 2024-09-11T22:09:25.302 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:25 smithi161 podman[72197]: 2024-09-11 22:09:25.301443804 +0000 UTC m=+0.407979219 container attach 277fc20f547447b9b115f0dc0e0ac24a86efbe2a3c0f8d9d4cc9e9bb762bf3b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-11T22:09:25.592 INFO:journalctl@ceph.osd.8.smithi161.stdout:Sep 11 22:09:25 smithi161 podman[72197]: 2024-09-11 22:09:25.434956789 +0000 UTC m=+0.541492187 container died 277fc20f547447b9b115f0dc0e0ac24a86efbe2a3c0f8d9d4cc9e9bb762bf3b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-8-deactivate, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T22:09:25.797 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:25 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:25.336+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.481596+0000 front 2024-09-11T22:08:17.481499+0000 (oldest deadline 2024-09-11T22:08:42.181347+0000) 2024-09-11T22:09:25.797 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:25 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:25.336+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:31.383481+0000 front 2024-09-11T22:08:31.383448+0000 (oldest deadline 2024-09-11T22:08:56.083565+0000) 2024-09-11T22:09:25.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:25 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:25.336+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:38.384885+0000 front 2024-09-11T22:08:38.385010+0000 (oldest deadline 2024-09-11T22:09:03.684801+0000) 2024-09-11T22:09:25.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:25 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:25.336+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.686290+0000 front 2024-09-11T22:08:50.686225+0000 (oldest deadline 2024-09-11T22:09:11.186328+0000) 2024-09-11T22:09:25.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:25 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:25.336+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-11T22:08:53.487319+0000 front 2024-09-11T22:08:53.487287+0000 (oldest deadline 2024-09-11T22:09:22.287607+0000) 2024-09-11T22:09:25.943 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:25 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:25.665+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.497679+0000 front 2024-09-11T22:08:16.497606+0000 (oldest deadline 2024-09-11T22:08:42.397744+0000) 2024-09-11T22:09:25.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:25 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:25.665+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:22.398445+0000 front 2024-09-11T22:08:22.398495+0000 (oldest deadline 2024-09-11T22:08:47.098448+0000) 2024-09-11T22:09:25.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:25 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:25.665+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:32.999892+0000 front 2024-09-11T22:08:32.999692+0000 (oldest deadline 2024-09-11T22:08:58.299748+0000) 2024-09-11T22:09:25.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:25 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:25.665+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:40.000748+0000 front 2024-09-11T22:08:40.000701+0000 (oldest deadline 2024-09-11T22:09:03.500864+0000) 2024-09-11T22:09:25.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:25 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:25.665+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.502512+0000 front 2024-09-11T22:08:50.502535+0000 (oldest deadline 2024-09-11T22:09:14.002461+0000) 2024-09-11T22:09:25.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:25 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:25.665+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-11T22:08:58.103409+0000 front 2024-09-11T22:08:58.103489+0000 (oldest deadline 2024-09-11T22:09:21.504260+0000) 2024-09-11T22:09:25.988 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:25 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:25.814+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.893190+0000 front 2024-09-11T22:08:16.893259+0000 (oldest deadline 2024-09-11T22:08:39.193066+0000) 2024-09-11T22:09:25.988 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:25 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:25.814+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:21.494266+0000 front 2024-09-11T22:08:21.494360+0000 (oldest deadline 2024-09-11T22:08:47.394416+0000) 2024-09-11T22:09:25.989 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:25 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:25.814+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:33.295337+0000 front 2024-09-11T22:08:33.295289+0000 (oldest deadline 2024-09-11T22:08:56.795428+0000) 2024-09-11T22:09:25.989 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:25 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:25.814+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:39.497398+0000 front 2024-09-11T22:08:39.497364+0000 (oldest deadline 2024-09-11T22:09:04.797240+0000) 2024-09-11T22:09:25.989 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:25 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:25.814+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.598954+0000 front 2024-09-11T22:08:50.598798+0000 (oldest deadline 2024-09-11T22:09:15.199166+0000) 2024-09-11T22:09:25.989 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:25 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:25.814+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-11T22:08:58.100016+0000 front 2024-09-11T22:08:58.100085+0000 (oldest deadline 2024-09-11T22:09:23.400061+0000) 2024-09-11T22:09:26.017 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3ffca83c-7087-11ef-bcea-c7b262605968@osd.8.service' 2024-09-11T22:09:26.057 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:09:26.057 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-11T22:09:26.058 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-11T22:09:26.058 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ceph-3ffca83c-7087-11ef-bcea-c7b262605968@osd.9 2024-09-11T22:09:26.388 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:26 smithi001 systemd[1]: Stopping Ceph osd.9 for 3ffca83c-7087-11ef-bcea-c7b262605968... 2024-09-11T22:09:26.388 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:26 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:26.386+0000 7fac62c9a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:09:26.693 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:26 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:26.386+0000 7fac62c9a640 -1 osd.9 98 *** Got signal Terminated *** 2024-09-11T22:09:26.694 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:26 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:26.386+0000 7fac62c9a640 -1 osd.9 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T22:09:26.694 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:26 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:26.619+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.497679+0000 front 2024-09-11T22:08:16.497606+0000 (oldest deadline 2024-09-11T22:08:42.397744+0000) 2024-09-11T22:09:26.694 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:26 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:26.619+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:22.398445+0000 front 2024-09-11T22:08:22.398495+0000 (oldest deadline 2024-09-11T22:08:47.098448+0000) 2024-09-11T22:09:26.694 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:26 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:26.619+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:32.999892+0000 front 2024-09-11T22:08:32.999692+0000 (oldest deadline 2024-09-11T22:08:58.299748+0000) 2024-09-11T22:09:26.694 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:26 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:26.619+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:40.000748+0000 front 2024-09-11T22:08:40.000701+0000 (oldest deadline 2024-09-11T22:09:03.500864+0000) 2024-09-11T22:09:26.694 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:26 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:26.619+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.502512+0000 front 2024-09-11T22:08:50.502535+0000 (oldest deadline 2024-09-11T22:09:14.002461+0000) 2024-09-11T22:09:26.694 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:26 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:26.619+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-11T22:08:58.103409+0000 front 2024-09-11T22:08:58.103489+0000 (oldest deadline 2024-09-11T22:09:21.504260+0000) 2024-09-11T22:09:26.797 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:26 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:26.358+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.481596+0000 front 2024-09-11T22:08:17.481499+0000 (oldest deadline 2024-09-11T22:08:42.181347+0000) 2024-09-11T22:09:26.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:26 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:26.358+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:31.383481+0000 front 2024-09-11T22:08:31.383448+0000 (oldest deadline 2024-09-11T22:08:56.083565+0000) 2024-09-11T22:09:26.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:26 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:26.358+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:38.384885+0000 front 2024-09-11T22:08:38.385010+0000 (oldest deadline 2024-09-11T22:09:03.684801+0000) 2024-09-11T22:09:26.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:26 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:26.358+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.686290+0000 front 2024-09-11T22:08:50.686225+0000 (oldest deadline 2024-09-11T22:09:11.186328+0000) 2024-09-11T22:09:26.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:26 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:26.358+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-11T22:08:53.487319+0000 front 2024-09-11T22:08:53.487287+0000 (oldest deadline 2024-09-11T22:09:22.287607+0000) 2024-09-11T22:09:27.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:26 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:26.771+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.893190+0000 front 2024-09-11T22:08:16.893259+0000 (oldest deadline 2024-09-11T22:08:39.193066+0000) 2024-09-11T22:09:27.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:26 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:26.771+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:21.494266+0000 front 2024-09-11T22:08:21.494360+0000 (oldest deadline 2024-09-11T22:08:47.394416+0000) 2024-09-11T22:09:27.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:26 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:26.771+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:33.295337+0000 front 2024-09-11T22:08:33.295289+0000 (oldest deadline 2024-09-11T22:08:56.795428+0000) 2024-09-11T22:09:27.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:26 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:26.771+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:39.497398+0000 front 2024-09-11T22:08:39.497364+0000 (oldest deadline 2024-09-11T22:09:04.797240+0000) 2024-09-11T22:09:27.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:26 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:26.771+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.598954+0000 front 2024-09-11T22:08:50.598798+0000 (oldest deadline 2024-09-11T22:09:15.199166+0000) 2024-09-11T22:09:27.093 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:26 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:26.771+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-11T22:08:58.100016+0000 front 2024-09-11T22:08:58.100085+0000 (oldest deadline 2024-09-11T22:09:23.400061+0000) 2024-09-11T22:09:27.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:27 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:27.371+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.481596+0000 front 2024-09-11T22:08:17.481499+0000 (oldest deadline 2024-09-11T22:08:42.181347+0000) 2024-09-11T22:09:27.799 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:27 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:27.371+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:31.383481+0000 front 2024-09-11T22:08:31.383448+0000 (oldest deadline 2024-09-11T22:08:56.083565+0000) 2024-09-11T22:09:27.799 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:27 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:27.371+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:38.384885+0000 front 2024-09-11T22:08:38.385010+0000 (oldest deadline 2024-09-11T22:09:03.684801+0000) 2024-09-11T22:09:27.799 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:27 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:27.371+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.686290+0000 front 2024-09-11T22:08:50.686225+0000 (oldest deadline 2024-09-11T22:09:11.186328+0000) 2024-09-11T22:09:27.799 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:27 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:27.371+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-11T22:08:53.487319+0000 front 2024-09-11T22:08:53.487287+0000 (oldest deadline 2024-09-11T22:09:22.287607+0000) 2024-09-11T22:09:27.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:27 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:27.605+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.497679+0000 front 2024-09-11T22:08:16.497606+0000 (oldest deadline 2024-09-11T22:08:42.397744+0000) 2024-09-11T22:09:27.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:27 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:27.605+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:22.398445+0000 front 2024-09-11T22:08:22.398495+0000 (oldest deadline 2024-09-11T22:08:47.098448+0000) 2024-09-11T22:09:27.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:27 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:27.605+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:32.999892+0000 front 2024-09-11T22:08:32.999692+0000 (oldest deadline 2024-09-11T22:08:58.299748+0000) 2024-09-11T22:09:27.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:27 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:27.605+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:40.000748+0000 front 2024-09-11T22:08:40.000701+0000 (oldest deadline 2024-09-11T22:09:03.500864+0000) 2024-09-11T22:09:27.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:27 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:27.605+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.502512+0000 front 2024-09-11T22:08:50.502535+0000 (oldest deadline 2024-09-11T22:09:14.002461+0000) 2024-09-11T22:09:27.945 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:27 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:27.605+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-11T22:08:58.103409+0000 front 2024-09-11T22:08:58.103489+0000 (oldest deadline 2024-09-11T22:09:21.504260+0000) 2024-09-11T22:09:27.945 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:27 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:27.605+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-11T22:09:01.504582+0000 front 2024-09-11T22:09:01.504637+0000 (oldest deadline 2024-09-11T22:09:27.404578+0000) 2024-09-11T22:09:28.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:27 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:27.800+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.893190+0000 front 2024-09-11T22:08:16.893259+0000 (oldest deadline 2024-09-11T22:08:39.193066+0000) 2024-09-11T22:09:28.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:27 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:27.800+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:21.494266+0000 front 2024-09-11T22:08:21.494360+0000 (oldest deadline 2024-09-11T22:08:47.394416+0000) 2024-09-11T22:09:28.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:27 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:27.800+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:33.295337+0000 front 2024-09-11T22:08:33.295289+0000 (oldest deadline 2024-09-11T22:08:56.795428+0000) 2024-09-11T22:09:28.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:27 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:27.800+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:39.497398+0000 front 2024-09-11T22:08:39.497364+0000 (oldest deadline 2024-09-11T22:09:04.797240+0000) 2024-09-11T22:09:28.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:27 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:27.800+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.598954+0000 front 2024-09-11T22:08:50.598798+0000 (oldest deadline 2024-09-11T22:09:15.199166+0000) 2024-09-11T22:09:28.093 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:27 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:27.800+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-11T22:08:58.100016+0000 front 2024-09-11T22:08:58.100085+0000 (oldest deadline 2024-09-11T22:09:23.400061+0000) 2024-09-11T22:09:28.797 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:28 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:28.394+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.481596+0000 front 2024-09-11T22:08:17.481499+0000 (oldest deadline 2024-09-11T22:08:42.181347+0000) 2024-09-11T22:09:28.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:28 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:28.394+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:31.383481+0000 front 2024-09-11T22:08:31.383448+0000 (oldest deadline 2024-09-11T22:08:56.083565+0000) 2024-09-11T22:09:28.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:28 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:28.394+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:38.384885+0000 front 2024-09-11T22:08:38.385010+0000 (oldest deadline 2024-09-11T22:09:03.684801+0000) 2024-09-11T22:09:28.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:28 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:28.394+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.686290+0000 front 2024-09-11T22:08:50.686225+0000 (oldest deadline 2024-09-11T22:09:11.186328+0000) 2024-09-11T22:09:28.799 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:28 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:28.394+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-11T22:08:53.487319+0000 front 2024-09-11T22:08:53.487287+0000 (oldest deadline 2024-09-11T22:09:22.287607+0000) 2024-09-11T22:09:28.799 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:28 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:28.394+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-11T22:09:06.388601+0000 front 2024-09-11T22:09:06.388631+0000 (oldest deadline 2024-09-11T22:09:27.988689+0000) 2024-09-11T22:09:28.943 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:28 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:28.584+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.497679+0000 front 2024-09-11T22:08:16.497606+0000 (oldest deadline 2024-09-11T22:08:42.397744+0000) 2024-09-11T22:09:28.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:28 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:28.584+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:22.398445+0000 front 2024-09-11T22:08:22.398495+0000 (oldest deadline 2024-09-11T22:08:47.098448+0000) 2024-09-11T22:09:28.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:28 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:28.584+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:32.999892+0000 front 2024-09-11T22:08:32.999692+0000 (oldest deadline 2024-09-11T22:08:58.299748+0000) 2024-09-11T22:09:28.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:28 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:28.584+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:40.000748+0000 front 2024-09-11T22:08:40.000701+0000 (oldest deadline 2024-09-11T22:09:03.500864+0000) 2024-09-11T22:09:28.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:28 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:28.584+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.502512+0000 front 2024-09-11T22:08:50.502535+0000 (oldest deadline 2024-09-11T22:09:14.002461+0000) 2024-09-11T22:09:28.945 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:28 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:28.584+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-11T22:08:58.103409+0000 front 2024-09-11T22:08:58.103489+0000 (oldest deadline 2024-09-11T22:09:21.504260+0000) 2024-09-11T22:09:28.945 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:28 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:28.584+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-11T22:09:01.504582+0000 front 2024-09-11T22:09:01.504637+0000 (oldest deadline 2024-09-11T22:09:27.404578+0000) 2024-09-11T22:09:29.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:28 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:28.801+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.893190+0000 front 2024-09-11T22:08:16.893259+0000 (oldest deadline 2024-09-11T22:08:39.193066+0000) 2024-09-11T22:09:29.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:28 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:28.801+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:21.494266+0000 front 2024-09-11T22:08:21.494360+0000 (oldest deadline 2024-09-11T22:08:47.394416+0000) 2024-09-11T22:09:29.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:28 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:28.801+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:33.295337+0000 front 2024-09-11T22:08:33.295289+0000 (oldest deadline 2024-09-11T22:08:56.795428+0000) 2024-09-11T22:09:29.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:28 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:28.801+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:39.497398+0000 front 2024-09-11T22:08:39.497364+0000 (oldest deadline 2024-09-11T22:09:04.797240+0000) 2024-09-11T22:09:29.093 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:28 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:28.801+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.598954+0000 front 2024-09-11T22:08:50.598798+0000 (oldest deadline 2024-09-11T22:09:15.199166+0000) 2024-09-11T22:09:29.093 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:28 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:28.801+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-11T22:08:58.100016+0000 front 2024-09-11T22:08:58.100085+0000 (oldest deadline 2024-09-11T22:09:23.400061+0000) 2024-09-11T22:09:29.797 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:29 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:29.368+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.481596+0000 front 2024-09-11T22:08:17.481499+0000 (oldest deadline 2024-09-11T22:08:42.181347+0000) 2024-09-11T22:09:29.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:29 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:29.368+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:31.383481+0000 front 2024-09-11T22:08:31.383448+0000 (oldest deadline 2024-09-11T22:08:56.083565+0000) 2024-09-11T22:09:29.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:29 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:29.368+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:38.384885+0000 front 2024-09-11T22:08:38.385010+0000 (oldest deadline 2024-09-11T22:09:03.684801+0000) 2024-09-11T22:09:29.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:29 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:29.368+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.686290+0000 front 2024-09-11T22:08:50.686225+0000 (oldest deadline 2024-09-11T22:09:11.186328+0000) 2024-09-11T22:09:29.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:29 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:29.368+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-11T22:08:53.487319+0000 front 2024-09-11T22:08:53.487287+0000 (oldest deadline 2024-09-11T22:09:22.287607+0000) 2024-09-11T22:09:29.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:29 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:29.368+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-11T22:09:06.388601+0000 front 2024-09-11T22:09:06.388631+0000 (oldest deadline 2024-09-11T22:09:27.988689+0000) 2024-09-11T22:09:29.943 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:29 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:29.629+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.497679+0000 front 2024-09-11T22:08:16.497606+0000 (oldest deadline 2024-09-11T22:08:42.397744+0000) 2024-09-11T22:09:29.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:29 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:29.629+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:22.398445+0000 front 2024-09-11T22:08:22.398495+0000 (oldest deadline 2024-09-11T22:08:47.098448+0000) 2024-09-11T22:09:29.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:29 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:29.629+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:32.999892+0000 front 2024-09-11T22:08:32.999692+0000 (oldest deadline 2024-09-11T22:08:58.299748+0000) 2024-09-11T22:09:29.945 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:29 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:29.629+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:40.000748+0000 front 2024-09-11T22:08:40.000701+0000 (oldest deadline 2024-09-11T22:09:03.500864+0000) 2024-09-11T22:09:29.945 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:29 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:29.629+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.502512+0000 front 2024-09-11T22:08:50.502535+0000 (oldest deadline 2024-09-11T22:09:14.002461+0000) 2024-09-11T22:09:29.945 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:29 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:29.629+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-11T22:08:58.103409+0000 front 2024-09-11T22:08:58.103489+0000 (oldest deadline 2024-09-11T22:09:21.504260+0000) 2024-09-11T22:09:29.945 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:29 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:29.629+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-11T22:09:01.504582+0000 front 2024-09-11T22:09:01.504637+0000 (oldest deadline 2024-09-11T22:09:27.404578+0000) 2024-09-11T22:09:30.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:29 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:29.816+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.893190+0000 front 2024-09-11T22:08:16.893259+0000 (oldest deadline 2024-09-11T22:08:39.193066+0000) 2024-09-11T22:09:30.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:29 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:29.816+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:21.494266+0000 front 2024-09-11T22:08:21.494360+0000 (oldest deadline 2024-09-11T22:08:47.394416+0000) 2024-09-11T22:09:30.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:29 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:29.816+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:33.295337+0000 front 2024-09-11T22:08:33.295289+0000 (oldest deadline 2024-09-11T22:08:56.795428+0000) 2024-09-11T22:09:30.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:29 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:29.816+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:39.497398+0000 front 2024-09-11T22:08:39.497364+0000 (oldest deadline 2024-09-11T22:09:04.797240+0000) 2024-09-11T22:09:30.093 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:29 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:29.816+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.598954+0000 front 2024-09-11T22:08:50.598798+0000 (oldest deadline 2024-09-11T22:09:15.199166+0000) 2024-09-11T22:09:30.093 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:29 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:29.816+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-11T22:08:58.100016+0000 front 2024-09-11T22:08:58.100085+0000 (oldest deadline 2024-09-11T22:09:23.400061+0000) 2024-09-11T22:09:30.797 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:30 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:30.387+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.481596+0000 front 2024-09-11T22:08:17.481499+0000 (oldest deadline 2024-09-11T22:08:42.181347+0000) 2024-09-11T22:09:30.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:30 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:30.387+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:31.383481+0000 front 2024-09-11T22:08:31.383448+0000 (oldest deadline 2024-09-11T22:08:56.083565+0000) 2024-09-11T22:09:30.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:30 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:30.387+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:38.384885+0000 front 2024-09-11T22:08:38.385010+0000 (oldest deadline 2024-09-11T22:09:03.684801+0000) 2024-09-11T22:09:30.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:30 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:30.387+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.686290+0000 front 2024-09-11T22:08:50.686225+0000 (oldest deadline 2024-09-11T22:09:11.186328+0000) 2024-09-11T22:09:30.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:30 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:30.387+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-11T22:08:53.487319+0000 front 2024-09-11T22:08:53.487287+0000 (oldest deadline 2024-09-11T22:09:22.287607+0000) 2024-09-11T22:09:30.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:30 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:30.387+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-11T22:09:06.388601+0000 front 2024-09-11T22:09:06.388631+0000 (oldest deadline 2024-09-11T22:09:27.988689+0000) 2024-09-11T22:09:30.944 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:30 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:30.647+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.497679+0000 front 2024-09-11T22:08:16.497606+0000 (oldest deadline 2024-09-11T22:08:42.397744+0000) 2024-09-11T22:09:30.945 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:30 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:30.647+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:22.398445+0000 front 2024-09-11T22:08:22.398495+0000 (oldest deadline 2024-09-11T22:08:47.098448+0000) 2024-09-11T22:09:30.945 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:30 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:30.647+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:32.999892+0000 front 2024-09-11T22:08:32.999692+0000 (oldest deadline 2024-09-11T22:08:58.299748+0000) 2024-09-11T22:09:30.945 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:30 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:30.647+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:40.000748+0000 front 2024-09-11T22:08:40.000701+0000 (oldest deadline 2024-09-11T22:09:03.500864+0000) 2024-09-11T22:09:30.945 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:30 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:30.647+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.502512+0000 front 2024-09-11T22:08:50.502535+0000 (oldest deadline 2024-09-11T22:09:14.002461+0000) 2024-09-11T22:09:30.945 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:30 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:30.647+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-11T22:08:58.103409+0000 front 2024-09-11T22:08:58.103489+0000 (oldest deadline 2024-09-11T22:09:21.504260+0000) 2024-09-11T22:09:30.945 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:30 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9[62541]: 2024-09-11T22:09:30.647+0000 7fac5b2a8640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-11T22:09:01.504582+0000 front 2024-09-11T22:09:01.504637+0000 (oldest deadline 2024-09-11T22:09:27.404578+0000) 2024-09-11T22:09:31.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:30 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:30.822+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.893190+0000 front 2024-09-11T22:08:16.893259+0000 (oldest deadline 2024-09-11T22:08:39.193066+0000) 2024-09-11T22:09:31.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:30 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:30.822+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:21.494266+0000 front 2024-09-11T22:08:21.494360+0000 (oldest deadline 2024-09-11T22:08:47.394416+0000) 2024-09-11T22:09:31.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:30 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:30.822+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:33.295337+0000 front 2024-09-11T22:08:33.295289+0000 (oldest deadline 2024-09-11T22:08:56.795428+0000) 2024-09-11T22:09:31.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:30 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:30.822+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:39.497398+0000 front 2024-09-11T22:08:39.497364+0000 (oldest deadline 2024-09-11T22:09:04.797240+0000) 2024-09-11T22:09:31.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:30 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:30.822+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.598954+0000 front 2024-09-11T22:08:50.598798+0000 (oldest deadline 2024-09-11T22:09:15.199166+0000) 2024-09-11T22:09:31.093 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:30 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:30.822+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-11T22:08:58.100016+0000 front 2024-09-11T22:08:58.100085+0000 (oldest deadline 2024-09-11T22:09:23.400061+0000) 2024-09-11T22:09:31.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:31 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:31.435+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.481596+0000 front 2024-09-11T22:08:17.481499+0000 (oldest deadline 2024-09-11T22:08:42.181347+0000) 2024-09-11T22:09:31.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:31 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:31.435+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:31.383481+0000 front 2024-09-11T22:08:31.383448+0000 (oldest deadline 2024-09-11T22:08:56.083565+0000) 2024-09-11T22:09:31.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:31 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:31.435+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:38.384885+0000 front 2024-09-11T22:08:38.385010+0000 (oldest deadline 2024-09-11T22:09:03.684801+0000) 2024-09-11T22:09:31.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:31 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:31.435+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.686290+0000 front 2024-09-11T22:08:50.686225+0000 (oldest deadline 2024-09-11T22:09:11.186328+0000) 2024-09-11T22:09:31.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:31 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:31.435+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-11T22:08:53.487319+0000 front 2024-09-11T22:08:53.487287+0000 (oldest deadline 2024-09-11T22:09:22.287607+0000) 2024-09-11T22:09:31.799 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:31 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:31.435+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-11T22:09:06.388601+0000 front 2024-09-11T22:09:06.388631+0000 (oldest deadline 2024-09-11T22:09:27.988689+0000) 2024-09-11T22:09:32.096 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:31 smithi001 podman[208626]: 2024-09-11 22:09:31.834467382 +0000 UTC m=+5.575144509 container died f668411bc676437ed81f97f55d189b333e0fe043637ae81b83eb2b1c28ce5916 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T22:09:32.342 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:31 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:31.859+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.893190+0000 front 2024-09-11T22:08:16.893259+0000 (oldest deadline 2024-09-11T22:08:39.193066+0000) 2024-09-11T22:09:32.342 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:31 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:31.859+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:21.494266+0000 front 2024-09-11T22:08:21.494360+0000 (oldest deadline 2024-09-11T22:08:47.394416+0000) 2024-09-11T22:09:32.342 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:31 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:31.859+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:33.295337+0000 front 2024-09-11T22:08:33.295289+0000 (oldest deadline 2024-09-11T22:08:56.795428+0000) 2024-09-11T22:09:32.342 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:31 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:31.859+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:39.497398+0000 front 2024-09-11T22:08:39.497364+0000 (oldest deadline 2024-09-11T22:09:04.797240+0000) 2024-09-11T22:09:32.342 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:31 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:31.859+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.598954+0000 front 2024-09-11T22:08:50.598798+0000 (oldest deadline 2024-09-11T22:09:15.199166+0000) 2024-09-11T22:09:32.342 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:31 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:31.859+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-11T22:08:58.100016+0000 front 2024-09-11T22:08:58.100085+0000 (oldest deadline 2024-09-11T22:09:23.400061+0000) 2024-09-11T22:09:32.443 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:32 smithi001 podman[208626]: 2024-09-11 22:09:32.253311006 +0000 UTC m=+5.993988133 container cleanup f668411bc676437ed81f97f55d189b333e0fe043637ae81b83eb2b1c28ce5916 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-11T22:09:32.444 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:32 smithi001 bash[208626]: ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9 2024-09-11T22:09:32.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:32 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:32.462+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.481596+0000 front 2024-09-11T22:08:17.481499+0000 (oldest deadline 2024-09-11T22:08:42.181347+0000) 2024-09-11T22:09:32.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:32 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:32.462+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:31.383481+0000 front 2024-09-11T22:08:31.383448+0000 (oldest deadline 2024-09-11T22:08:56.083565+0000) 2024-09-11T22:09:32.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:32 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:32.462+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:38.384885+0000 front 2024-09-11T22:08:38.385010+0000 (oldest deadline 2024-09-11T22:09:03.684801+0000) 2024-09-11T22:09:32.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:32 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:32.462+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.686290+0000 front 2024-09-11T22:08:50.686225+0000 (oldest deadline 2024-09-11T22:09:11.186328+0000) 2024-09-11T22:09:32.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:32 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:32.462+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-11T22:08:53.487319+0000 front 2024-09-11T22:08:53.487287+0000 (oldest deadline 2024-09-11T22:09:22.287607+0000) 2024-09-11T22:09:32.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:32 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:32.462+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-11T22:09:06.388601+0000 front 2024-09-11T22:09:06.388631+0000 (oldest deadline 2024-09-11T22:09:27.988689+0000) 2024-09-11T22:09:32.930 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:32 smithi001 podman[208642]: 2024-09-11 22:09:32.505000081 +0000 UTC m=+0.666504385 container remove f668411bc676437ed81f97f55d189b333e0fe043637ae81b83eb2b1c28ce5916 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-11T22:09:33.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:32 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:32.821+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.893190+0000 front 2024-09-11T22:08:16.893259+0000 (oldest deadline 2024-09-11T22:08:39.193066+0000) 2024-09-11T22:09:33.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:32 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:32.821+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:21.494266+0000 front 2024-09-11T22:08:21.494360+0000 (oldest deadline 2024-09-11T22:08:47.394416+0000) 2024-09-11T22:09:33.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:32 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:32.821+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:33.295337+0000 front 2024-09-11T22:08:33.295289+0000 (oldest deadline 2024-09-11T22:08:56.795428+0000) 2024-09-11T22:09:33.093 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:32 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:32.821+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:39.497398+0000 front 2024-09-11T22:08:39.497364+0000 (oldest deadline 2024-09-11T22:09:04.797240+0000) 2024-09-11T22:09:33.093 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:32 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:32.821+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.598954+0000 front 2024-09-11T22:08:50.598798+0000 (oldest deadline 2024-09-11T22:09:15.199166+0000) 2024-09-11T22:09:33.093 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:32 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:32.821+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-11T22:08:58.100016+0000 front 2024-09-11T22:08:58.100085+0000 (oldest deadline 2024-09-11T22:09:23.400061+0000) 2024-09-11T22:09:33.193 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:32 smithi001 podman[208733]: 2024-09-11 22:09:32.93080328 +0000 UTC m=+0.110429547 container create 1cc2ccae3ac3bd2ca6e24e605ce1989c527f25d10f0b73e5d672fedc3a0d9801 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9-deactivate, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T22:09:33.193 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:32 smithi001 podman[208733]: 2024-09-11 22:09:32.836888013 +0000 UTC m=+0.016514239 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T22:09:33.194 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:33 smithi001 podman[208733]: 2024-09-11 22:09:33.17644266 +0000 UTC m=+0.356068888 container init 1cc2ccae3ac3bd2ca6e24e605ce1989c527f25d10f0b73e5d672fedc3a0d9801 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T22:09:33.194 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:33 smithi001 podman[208733]: 2024-09-11 22:09:33.179953288 +0000 UTC m=+0.359579514 container start 1cc2ccae3ac3bd2ca6e24e605ce1989c527f25d10f0b73e5d672fedc3a0d9801 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T22:09:33.653 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:33 smithi001 podman[208733]: 2024-09-11 22:09:33.250498381 +0000 UTC m=+0.430124615 container attach 1cc2ccae3ac3bd2ca6e24e605ce1989c527f25d10f0b73e5d672fedc3a0d9801 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T22:09:33.653 INFO:journalctl@ceph.osd.9.smithi001.stdout:Sep 11 22:09:33 smithi001 podman[208733]: 2024-09-11 22:09:33.369954451 +0000 UTC m=+0.549580694 container died 1cc2ccae3ac3bd2ca6e24e605ce1989c527f25d10f0b73e5d672fedc3a0d9801 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-11T22:09:33.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:33 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:33.497+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.481596+0000 front 2024-09-11T22:08:17.481499+0000 (oldest deadline 2024-09-11T22:08:42.181347+0000) 2024-09-11T22:09:33.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:33 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:33.497+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:31.383481+0000 front 2024-09-11T22:08:31.383448+0000 (oldest deadline 2024-09-11T22:08:56.083565+0000) 2024-09-11T22:09:33.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:33 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:33.497+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:38.384885+0000 front 2024-09-11T22:08:38.385010+0000 (oldest deadline 2024-09-11T22:09:03.684801+0000) 2024-09-11T22:09:33.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:33 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:33.497+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.686290+0000 front 2024-09-11T22:08:50.686225+0000 (oldest deadline 2024-09-11T22:09:11.186328+0000) 2024-09-11T22:09:33.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:33 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:33.497+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-11T22:08:53.487319+0000 front 2024-09-11T22:08:53.487287+0000 (oldest deadline 2024-09-11T22:09:22.287607+0000) 2024-09-11T22:09:33.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:33 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:33.497+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-11T22:09:06.388601+0000 front 2024-09-11T22:09:06.388631+0000 (oldest deadline 2024-09-11T22:09:27.988689+0000) 2024-09-11T22:09:33.940 DEBUG:teuthology.orchestra.run.smithi001:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3ffca83c-7087-11ef-bcea-c7b262605968@osd.9.service' 2024-09-11T22:09:33.985 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:09:33.985 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-11T22:09:33.985 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-11T22:09:33.986 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-3ffca83c-7087-11ef-bcea-c7b262605968@osd.10 2024-09-11T22:09:34.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:33 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:33.825+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.893190+0000 front 2024-09-11T22:08:16.893259+0000 (oldest deadline 2024-09-11T22:08:39.193066+0000) 2024-09-11T22:09:34.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:33 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:33.825+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:21.494266+0000 front 2024-09-11T22:08:21.494360+0000 (oldest deadline 2024-09-11T22:08:47.394416+0000) 2024-09-11T22:09:34.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:33 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:33.825+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:33.295337+0000 front 2024-09-11T22:08:33.295289+0000 (oldest deadline 2024-09-11T22:08:56.795428+0000) 2024-09-11T22:09:34.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:33 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:33.825+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:39.497398+0000 front 2024-09-11T22:08:39.497364+0000 (oldest deadline 2024-09-11T22:09:04.797240+0000) 2024-09-11T22:09:34.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:33 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:33.825+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.598954+0000 front 2024-09-11T22:08:50.598798+0000 (oldest deadline 2024-09-11T22:09:15.199166+0000) 2024-09-11T22:09:34.093 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:33 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:33.825+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-11T22:08:58.100016+0000 front 2024-09-11T22:08:58.100085+0000 (oldest deadline 2024-09-11T22:09:23.400061+0000) 2024-09-11T22:09:34.268 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:34 smithi053 systemd[1]: Stopping Ceph osd.10 for 3ffca83c-7087-11ef-bcea-c7b262605968... 2024-09-11T22:09:34.269 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:34 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:34.267+0000 7f5cf24e2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:09:34.533 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:34 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:34.267+0000 7f5cf24e2640 -1 osd.10 98 *** Got signal Terminated *** 2024-09-11T22:09:34.533 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:34 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:34.267+0000 7f5cf24e2640 -1 osd.10 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T22:09:34.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:34 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:34.532+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.481596+0000 front 2024-09-11T22:08:17.481499+0000 (oldest deadline 2024-09-11T22:08:42.181347+0000) 2024-09-11T22:09:34.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:34 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:34.532+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:31.383481+0000 front 2024-09-11T22:08:31.383448+0000 (oldest deadline 2024-09-11T22:08:56.083565+0000) 2024-09-11T22:09:34.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:34 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:34.532+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:38.384885+0000 front 2024-09-11T22:08:38.385010+0000 (oldest deadline 2024-09-11T22:09:03.684801+0000) 2024-09-11T22:09:34.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:34 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:34.532+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.686290+0000 front 2024-09-11T22:08:50.686225+0000 (oldest deadline 2024-09-11T22:09:11.186328+0000) 2024-09-11T22:09:34.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:34 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:34.532+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-11T22:08:53.487319+0000 front 2024-09-11T22:08:53.487287+0000 (oldest deadline 2024-09-11T22:09:22.287607+0000) 2024-09-11T22:09:34.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:34 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:34.532+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-11T22:09:06.388601+0000 front 2024-09-11T22:09:06.388631+0000 (oldest deadline 2024-09-11T22:09:27.988689+0000) 2024-09-11T22:09:35.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:34 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:34.813+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.893190+0000 front 2024-09-11T22:08:16.893259+0000 (oldest deadline 2024-09-11T22:08:39.193066+0000) 2024-09-11T22:09:35.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:34 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:34.813+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:21.494266+0000 front 2024-09-11T22:08:21.494360+0000 (oldest deadline 2024-09-11T22:08:47.394416+0000) 2024-09-11T22:09:35.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:34 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:34.813+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:33.295337+0000 front 2024-09-11T22:08:33.295289+0000 (oldest deadline 2024-09-11T22:08:56.795428+0000) 2024-09-11T22:09:35.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:34 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:34.813+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:39.497398+0000 front 2024-09-11T22:08:39.497364+0000 (oldest deadline 2024-09-11T22:09:04.797240+0000) 2024-09-11T22:09:35.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:34 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:34.813+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.598954+0000 front 2024-09-11T22:08:50.598798+0000 (oldest deadline 2024-09-11T22:09:15.199166+0000) 2024-09-11T22:09:35.093 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:34 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:34.813+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-11T22:08:58.100016+0000 front 2024-09-11T22:08:58.100085+0000 (oldest deadline 2024-09-11T22:09:23.400061+0000) 2024-09-11T22:09:36.047 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:35 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:35.556+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.481596+0000 front 2024-09-11T22:08:17.481499+0000 (oldest deadline 2024-09-11T22:08:42.181347+0000) 2024-09-11T22:09:36.048 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:35 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:35.556+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:31.383481+0000 front 2024-09-11T22:08:31.383448+0000 (oldest deadline 2024-09-11T22:08:56.083565+0000) 2024-09-11T22:09:36.048 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:35 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:35.556+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:38.384885+0000 front 2024-09-11T22:08:38.385010+0000 (oldest deadline 2024-09-11T22:09:03.684801+0000) 2024-09-11T22:09:36.048 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:35 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:35.556+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.686290+0000 front 2024-09-11T22:08:50.686225+0000 (oldest deadline 2024-09-11T22:09:11.186328+0000) 2024-09-11T22:09:36.048 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:35 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:35.556+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-11T22:08:53.487319+0000 front 2024-09-11T22:08:53.487287+0000 (oldest deadline 2024-09-11T22:09:22.287607+0000) 2024-09-11T22:09:36.048 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:35 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:35.556+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-11T22:09:06.388601+0000 front 2024-09-11T22:09:06.388631+0000 (oldest deadline 2024-09-11T22:09:27.988689+0000) 2024-09-11T22:09:36.091 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:35 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:35.828+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.893190+0000 front 2024-09-11T22:08:16.893259+0000 (oldest deadline 2024-09-11T22:08:39.193066+0000) 2024-09-11T22:09:36.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:35 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:35.828+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:21.494266+0000 front 2024-09-11T22:08:21.494360+0000 (oldest deadline 2024-09-11T22:08:47.394416+0000) 2024-09-11T22:09:36.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:35 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:35.828+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:33.295337+0000 front 2024-09-11T22:08:33.295289+0000 (oldest deadline 2024-09-11T22:08:56.795428+0000) 2024-09-11T22:09:36.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:35 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:35.828+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:39.497398+0000 front 2024-09-11T22:08:39.497364+0000 (oldest deadline 2024-09-11T22:09:04.797240+0000) 2024-09-11T22:09:36.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:35 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:35.828+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.598954+0000 front 2024-09-11T22:08:50.598798+0000 (oldest deadline 2024-09-11T22:09:15.199166+0000) 2024-09-11T22:09:36.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:35 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:35.828+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-11T22:08:58.100016+0000 front 2024-09-11T22:08:58.100085+0000 (oldest deadline 2024-09-11T22:09:23.400061+0000) 2024-09-11T22:09:37.047 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:36 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:36.574+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.481596+0000 front 2024-09-11T22:08:17.481499+0000 (oldest deadline 2024-09-11T22:08:42.181347+0000) 2024-09-11T22:09:37.048 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:36 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:36.574+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:31.383481+0000 front 2024-09-11T22:08:31.383448+0000 (oldest deadline 2024-09-11T22:08:56.083565+0000) 2024-09-11T22:09:37.048 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:36 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:36.574+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:38.384885+0000 front 2024-09-11T22:08:38.385010+0000 (oldest deadline 2024-09-11T22:09:03.684801+0000) 2024-09-11T22:09:37.048 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:36 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:36.574+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.686290+0000 front 2024-09-11T22:08:50.686225+0000 (oldest deadline 2024-09-11T22:09:11.186328+0000) 2024-09-11T22:09:37.048 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:36 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:36.574+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-11T22:08:53.487319+0000 front 2024-09-11T22:08:53.487287+0000 (oldest deadline 2024-09-11T22:09:22.287607+0000) 2024-09-11T22:09:37.048 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:36 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:36.574+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-11T22:09:06.388601+0000 front 2024-09-11T22:09:06.388631+0000 (oldest deadline 2024-09-11T22:09:27.988689+0000) 2024-09-11T22:09:37.091 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:36 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:36.798+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.893190+0000 front 2024-09-11T22:08:16.893259+0000 (oldest deadline 2024-09-11T22:08:39.193066+0000) 2024-09-11T22:09:37.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:36 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:36.798+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:21.494266+0000 front 2024-09-11T22:08:21.494360+0000 (oldest deadline 2024-09-11T22:08:47.394416+0000) 2024-09-11T22:09:37.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:36 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:36.798+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:33.295337+0000 front 2024-09-11T22:08:33.295289+0000 (oldest deadline 2024-09-11T22:08:56.795428+0000) 2024-09-11T22:09:37.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:36 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:36.798+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:39.497398+0000 front 2024-09-11T22:08:39.497364+0000 (oldest deadline 2024-09-11T22:09:04.797240+0000) 2024-09-11T22:09:37.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:36 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:36.798+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.598954+0000 front 2024-09-11T22:08:50.598798+0000 (oldest deadline 2024-09-11T22:09:15.199166+0000) 2024-09-11T22:09:37.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:36 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:36.798+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-11T22:08:58.100016+0000 front 2024-09-11T22:08:58.100085+0000 (oldest deadline 2024-09-11T22:09:23.400061+0000) 2024-09-11T22:09:37.797 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:37 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:37.537+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.481596+0000 front 2024-09-11T22:08:17.481499+0000 (oldest deadline 2024-09-11T22:08:42.181347+0000) 2024-09-11T22:09:37.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:37 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:37.537+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:31.383481+0000 front 2024-09-11T22:08:31.383448+0000 (oldest deadline 2024-09-11T22:08:56.083565+0000) 2024-09-11T22:09:37.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:37 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:37.537+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:38.384885+0000 front 2024-09-11T22:08:38.385010+0000 (oldest deadline 2024-09-11T22:09:03.684801+0000) 2024-09-11T22:09:37.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:37 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:37.537+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.686290+0000 front 2024-09-11T22:08:50.686225+0000 (oldest deadline 2024-09-11T22:09:11.186328+0000) 2024-09-11T22:09:37.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:37 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:37.537+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-11T22:08:53.487319+0000 front 2024-09-11T22:08:53.487287+0000 (oldest deadline 2024-09-11T22:09:22.287607+0000) 2024-09-11T22:09:37.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:37 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:37.537+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-11T22:09:06.388601+0000 front 2024-09-11T22:09:06.388631+0000 (oldest deadline 2024-09-11T22:09:27.988689+0000) 2024-09-11T22:09:37.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:37 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:37.537+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.53:6820 osd.7 since back 2024-09-11T22:09:14.989890+0000 front 2024-09-11T22:09:14.989851+0000 (oldest deadline 2024-09-11T22:09:37.289815+0000) 2024-09-11T22:09:38.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:37 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:37.811+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.893190+0000 front 2024-09-11T22:08:16.893259+0000 (oldest deadline 2024-09-11T22:08:39.193066+0000) 2024-09-11T22:09:38.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:37 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:37.811+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:21.494266+0000 front 2024-09-11T22:08:21.494360+0000 (oldest deadline 2024-09-11T22:08:47.394416+0000) 2024-09-11T22:09:38.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:37 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:37.811+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:33.295337+0000 front 2024-09-11T22:08:33.295289+0000 (oldest deadline 2024-09-11T22:08:56.795428+0000) 2024-09-11T22:09:38.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:37 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:37.811+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:39.497398+0000 front 2024-09-11T22:08:39.497364+0000 (oldest deadline 2024-09-11T22:09:04.797240+0000) 2024-09-11T22:09:38.093 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:37 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:37.811+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.598954+0000 front 2024-09-11T22:08:50.598798+0000 (oldest deadline 2024-09-11T22:09:15.199166+0000) 2024-09-11T22:09:38.093 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:37 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:37.811+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-11T22:08:58.100016+0000 front 2024-09-11T22:08:58.100085+0000 (oldest deadline 2024-09-11T22:09:23.400061+0000) 2024-09-11T22:09:38.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:38 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:38.539+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:17.481596+0000 front 2024-09-11T22:08:17.481499+0000 (oldest deadline 2024-09-11T22:08:42.181347+0000) 2024-09-11T22:09:38.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:38 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:38.539+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:31.383481+0000 front 2024-09-11T22:08:31.383448+0000 (oldest deadline 2024-09-11T22:08:56.083565+0000) 2024-09-11T22:09:38.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:38 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:38.539+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:38.384885+0000 front 2024-09-11T22:08:38.385010+0000 (oldest deadline 2024-09-11T22:09:03.684801+0000) 2024-09-11T22:09:38.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:38 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:38.539+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.686290+0000 front 2024-09-11T22:08:50.686225+0000 (oldest deadline 2024-09-11T22:09:11.186328+0000) 2024-09-11T22:09:38.798 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:38 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:38.539+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-11T22:08:53.487319+0000 front 2024-09-11T22:08:53.487287+0000 (oldest deadline 2024-09-11T22:09:22.287607+0000) 2024-09-11T22:09:38.799 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:38 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:38.539+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-09-11T22:09:06.388601+0000 front 2024-09-11T22:09:06.388631+0000 (oldest deadline 2024-09-11T22:09:27.988689+0000) 2024-09-11T22:09:38.799 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:38 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10[59186]: 2024-09-11T22:09:38.539+0000 7f5ceaaf0640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.53:6820 osd.7 since back 2024-09-11T22:09:14.989890+0000 front 2024-09-11T22:09:14.989851+0000 (oldest deadline 2024-09-11T22:09:37.289815+0000) 2024-09-11T22:09:39.342 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:38 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:38.852+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.893190+0000 front 2024-09-11T22:08:16.893259+0000 (oldest deadline 2024-09-11T22:08:39.193066+0000) 2024-09-11T22:09:39.342 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:38 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:38.852+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:21.494266+0000 front 2024-09-11T22:08:21.494360+0000 (oldest deadline 2024-09-11T22:08:47.394416+0000) 2024-09-11T22:09:39.342 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:38 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:38.852+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:33.295337+0000 front 2024-09-11T22:08:33.295289+0000 (oldest deadline 2024-09-11T22:08:56.795428+0000) 2024-09-11T22:09:39.342 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:38 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:38.852+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:39.497398+0000 front 2024-09-11T22:08:39.497364+0000 (oldest deadline 2024-09-11T22:09:04.797240+0000) 2024-09-11T22:09:39.342 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:38 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:38.852+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.598954+0000 front 2024-09-11T22:08:50.598798+0000 (oldest deadline 2024-09-11T22:09:15.199166+0000) 2024-09-11T22:09:39.343 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:38 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:38.852+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-11T22:08:58.100016+0000 front 2024-09-11T22:08:58.100085+0000 (oldest deadline 2024-09-11T22:09:23.400061+0000) 2024-09-11T22:09:39.343 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:38 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:38.852+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6820 osd.7 since back 2024-09-11T22:09:14.402414+0000 front 2024-09-11T22:09:14.402488+0000 (oldest deadline 2024-09-11T22:09:38.502275+0000) 2024-09-11T22:09:40.070 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:39 smithi053 podman[70855]: 2024-09-11 22:09:39.819579114 +0000 UTC m=+5.629718527 container died e5d06c9fd99ce04c65d94f926a14e40b0ce307534ef792a1c24368a7879b633e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T22:09:40.342 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:39 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:39.885+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.893190+0000 front 2024-09-11T22:08:16.893259+0000 (oldest deadline 2024-09-11T22:08:39.193066+0000) 2024-09-11T22:09:40.342 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:39 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:39.885+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:21.494266+0000 front 2024-09-11T22:08:21.494360+0000 (oldest deadline 2024-09-11T22:08:47.394416+0000) 2024-09-11T22:09:40.342 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:39 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:39.885+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:33.295337+0000 front 2024-09-11T22:08:33.295289+0000 (oldest deadline 2024-09-11T22:08:56.795428+0000) 2024-09-11T22:09:40.342 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:39 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:39.885+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:39.497398+0000 front 2024-09-11T22:08:39.497364+0000 (oldest deadline 2024-09-11T22:09:04.797240+0000) 2024-09-11T22:09:40.343 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:39 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:39.885+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.598954+0000 front 2024-09-11T22:08:50.598798+0000 (oldest deadline 2024-09-11T22:09:15.199166+0000) 2024-09-11T22:09:40.343 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:39 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:39.885+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-11T22:08:58.100016+0000 front 2024-09-11T22:08:58.100085+0000 (oldest deadline 2024-09-11T22:09:23.400061+0000) 2024-09-11T22:09:40.343 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:39 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:39.885+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6820 osd.7 since back 2024-09-11T22:09:14.402414+0000 front 2024-09-11T22:09:14.402488+0000 (oldest deadline 2024-09-11T22:09:38.502275+0000) 2024-09-11T22:09:40.444 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:40 smithi053 podman[70855]: 2024-09-11 22:09:40.167689657 +0000 UTC m=+5.977829080 container cleanup e5d06c9fd99ce04c65d94f926a14e40b0ce307534ef792a1c24368a7879b633e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-11T22:09:40.444 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:40 smithi053 bash[70855]: ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10 2024-09-11T22:09:40.787 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:40 smithi053 podman[70869]: 2024-09-11 22:09:40.443821292 +0000 UTC m=+0.621714144 container remove e5d06c9fd99ce04c65d94f926a14e40b0ce307534ef792a1c24368a7879b633e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-11T22:09:41.047 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:40 smithi053 podman[70958]: 2024-09-11 22:09:40.690206341 +0000 UTC m=+0.015877060 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T22:09:41.048 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:40 smithi053 podman[70958]: 2024-09-11 22:09:40.794759225 +0000 UTC m=+0.120429985 container create 80ea54c11c078fabe53d20b396c8068fb54731eb172c757a18ceabca5e7c1ba0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10-deactivate, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3) 2024-09-11T22:09:41.048 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:41 smithi053 podman[70958]: 2024-09-11 22:09:41.030728739 +0000 UTC m=+0.356399458 container init 80ea54c11c078fabe53d20b396c8068fb54731eb172c757a18ceabca5e7c1ba0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-11T22:09:41.048 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:41 smithi053 podman[70958]: 2024-09-11 22:09:41.03394858 +0000 UTC m=+0.359619308 container start 80ea54c11c078fabe53d20b396c8068fb54731eb172c757a18ceabca5e7c1ba0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T22:09:41.243 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:40 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:40.920+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.893190+0000 front 2024-09-11T22:08:16.893259+0000 (oldest deadline 2024-09-11T22:08:39.193066+0000) 2024-09-11T22:09:41.244 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:40 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:40.920+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:21.494266+0000 front 2024-09-11T22:08:21.494360+0000 (oldest deadline 2024-09-11T22:08:47.394416+0000) 2024-09-11T22:09:41.244 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:40 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:40.920+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:33.295337+0000 front 2024-09-11T22:08:33.295289+0000 (oldest deadline 2024-09-11T22:08:56.795428+0000) 2024-09-11T22:09:41.244 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:40 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:40.920+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:39.497398+0000 front 2024-09-11T22:08:39.497364+0000 (oldest deadline 2024-09-11T22:09:04.797240+0000) 2024-09-11T22:09:41.244 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:40 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:40.920+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.598954+0000 front 2024-09-11T22:08:50.598798+0000 (oldest deadline 2024-09-11T22:09:15.199166+0000) 2024-09-11T22:09:41.244 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:40 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:40.920+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-11T22:08:58.100016+0000 front 2024-09-11T22:08:58.100085+0000 (oldest deadline 2024-09-11T22:09:23.400061+0000) 2024-09-11T22:09:41.244 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:40 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:40.920+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6820 osd.7 since back 2024-09-11T22:09:14.402414+0000 front 2024-09-11T22:09:14.402488+0000 (oldest deadline 2024-09-11T22:09:38.502275+0000) 2024-09-11T22:09:41.509 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:41 smithi053 podman[70958]: 2024-09-11 22:09:41.130682696 +0000 UTC m=+0.456353414 container attach 80ea54c11c078fabe53d20b396c8068fb54731eb172c757a18ceabca5e7c1ba0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-11T22:09:41.510 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:41 smithi053 podman[70958]: 2024-09-11 22:09:41.227341749 +0000 UTC m=+0.553012468 container died 80ea54c11c078fabe53d20b396c8068fb54731eb172c757a18ceabca5e7c1ba0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux ) 2024-09-11T22:09:41.735 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3ffca83c-7087-11ef-bcea-c7b262605968@osd.10.service' 2024-09-11T22:09:41.760 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:41 smithi053 podman[70981]: 2024-09-11 22:09:41.726112243 +0000 UTC m=+0.488583373 container remove 80ea54c11c078fabe53d20b396c8068fb54731eb172c757a18ceabca5e7c1ba0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T22:09:41.760 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:41 smithi053 systemd[1]: ceph-3ffca83c-7087-11ef-bcea-c7b262605968@osd.10.service: Deactivated successfully. 2024-09-11T22:09:41.760 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:41 smithi053 systemd[1]: Stopped Ceph osd.10 for 3ffca83c-7087-11ef-bcea-c7b262605968. 2024-09-11T22:09:41.760 INFO:journalctl@ceph.osd.10.smithi053.stdout:Sep 11 22:09:41 smithi053 systemd[1]: ceph-3ffca83c-7087-11ef-bcea-c7b262605968@osd.10.service: Consumed 1min 4.739s CPU time. 2024-09-11T22:09:41.773 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:09:41.773 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-11T22:09:41.773 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-11T22:09:41.774 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-3ffca83c-7087-11ef-bcea-c7b262605968@osd.11 2024-09-11T22:09:42.083 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:41 smithi161 systemd[1]: Stopping Ceph osd.11 for 3ffca83c-7087-11ef-bcea-c7b262605968... 2024-09-11T22:09:42.083 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:41 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:41.891+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.893190+0000 front 2024-09-11T22:08:16.893259+0000 (oldest deadline 2024-09-11T22:08:39.193066+0000) 2024-09-11T22:09:42.083 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:41 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:41.891+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:21.494266+0000 front 2024-09-11T22:08:21.494360+0000 (oldest deadline 2024-09-11T22:08:47.394416+0000) 2024-09-11T22:09:42.084 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:41 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:41.891+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:33.295337+0000 front 2024-09-11T22:08:33.295289+0000 (oldest deadline 2024-09-11T22:08:56.795428+0000) 2024-09-11T22:09:42.084 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:41 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:41.891+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:39.497398+0000 front 2024-09-11T22:08:39.497364+0000 (oldest deadline 2024-09-11T22:09:04.797240+0000) 2024-09-11T22:09:42.084 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:41 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:41.891+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.598954+0000 front 2024-09-11T22:08:50.598798+0000 (oldest deadline 2024-09-11T22:09:15.199166+0000) 2024-09-11T22:09:42.084 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:41 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:41.891+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-11T22:08:58.100016+0000 front 2024-09-11T22:08:58.100085+0000 (oldest deadline 2024-09-11T22:09:23.400061+0000) 2024-09-11T22:09:42.084 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:41 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:41.891+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6820 osd.7 since back 2024-09-11T22:09:14.402414+0000 front 2024-09-11T22:09:14.402488+0000 (oldest deadline 2024-09-11T22:09:38.502275+0000) 2024-09-11T22:09:42.084 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:42 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:42.082+0000 7f67ab4e4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:09:42.084 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:42 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:42.082+0000 7f67ab4e4640 -1 osd.11 98 *** Got signal Terminated *** 2024-09-11T22:09:42.085 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:42 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:42.082+0000 7f67ab4e4640 -1 osd.11 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T22:09:43.342 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:42 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:42.843+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.893190+0000 front 2024-09-11T22:08:16.893259+0000 (oldest deadline 2024-09-11T22:08:39.193066+0000) 2024-09-11T22:09:43.342 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:42 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:42.843+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:21.494266+0000 front 2024-09-11T22:08:21.494360+0000 (oldest deadline 2024-09-11T22:08:47.394416+0000) 2024-09-11T22:09:43.342 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:42 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:42.843+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:33.295337+0000 front 2024-09-11T22:08:33.295289+0000 (oldest deadline 2024-09-11T22:08:56.795428+0000) 2024-09-11T22:09:43.342 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:42 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:42.843+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:39.497398+0000 front 2024-09-11T22:08:39.497364+0000 (oldest deadline 2024-09-11T22:09:04.797240+0000) 2024-09-11T22:09:43.342 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:42 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:42.843+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.598954+0000 front 2024-09-11T22:08:50.598798+0000 (oldest deadline 2024-09-11T22:09:15.199166+0000) 2024-09-11T22:09:43.342 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:42 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:42.843+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-11T22:08:58.100016+0000 front 2024-09-11T22:08:58.100085+0000 (oldest deadline 2024-09-11T22:09:23.400061+0000) 2024-09-11T22:09:43.343 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:42 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:42.843+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6820 osd.7 since back 2024-09-11T22:09:14.402414+0000 front 2024-09-11T22:09:14.402488+0000 (oldest deadline 2024-09-11T22:09:38.502275+0000) 2024-09-11T22:09:44.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:43 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:43.816+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.893190+0000 front 2024-09-11T22:08:16.893259+0000 (oldest deadline 2024-09-11T22:08:39.193066+0000) 2024-09-11T22:09:44.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:43 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:43.816+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:21.494266+0000 front 2024-09-11T22:08:21.494360+0000 (oldest deadline 2024-09-11T22:08:47.394416+0000) 2024-09-11T22:09:44.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:43 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:43.816+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:33.295337+0000 front 2024-09-11T22:08:33.295289+0000 (oldest deadline 2024-09-11T22:08:56.795428+0000) 2024-09-11T22:09:44.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:43 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:43.816+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:39.497398+0000 front 2024-09-11T22:08:39.497364+0000 (oldest deadline 2024-09-11T22:09:04.797240+0000) 2024-09-11T22:09:44.093 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:43 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:43.816+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.598954+0000 front 2024-09-11T22:08:50.598798+0000 (oldest deadline 2024-09-11T22:09:15.199166+0000) 2024-09-11T22:09:44.093 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:43 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:43.816+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-11T22:08:58.100016+0000 front 2024-09-11T22:08:58.100085+0000 (oldest deadline 2024-09-11T22:09:23.400061+0000) 2024-09-11T22:09:44.093 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:43 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:43.816+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6820 osd.7 since back 2024-09-11T22:09:14.402414+0000 front 2024-09-11T22:09:14.402488+0000 (oldest deadline 2024-09-11T22:09:38.502275+0000) 2024-09-11T22:09:45.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:44 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:44.809+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.893190+0000 front 2024-09-11T22:08:16.893259+0000 (oldest deadline 2024-09-11T22:08:39.193066+0000) 2024-09-11T22:09:45.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:44 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:44.809+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:21.494266+0000 front 2024-09-11T22:08:21.494360+0000 (oldest deadline 2024-09-11T22:08:47.394416+0000) 2024-09-11T22:09:45.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:44 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:44.809+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:33.295337+0000 front 2024-09-11T22:08:33.295289+0000 (oldest deadline 2024-09-11T22:08:56.795428+0000) 2024-09-11T22:09:45.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:44 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:44.809+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:39.497398+0000 front 2024-09-11T22:08:39.497364+0000 (oldest deadline 2024-09-11T22:09:04.797240+0000) 2024-09-11T22:09:45.093 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:44 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:44.809+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.598954+0000 front 2024-09-11T22:08:50.598798+0000 (oldest deadline 2024-09-11T22:09:15.199166+0000) 2024-09-11T22:09:45.093 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:44 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:44.809+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-11T22:08:58.100016+0000 front 2024-09-11T22:08:58.100085+0000 (oldest deadline 2024-09-11T22:09:23.400061+0000) 2024-09-11T22:09:45.093 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:44 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:44.809+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6820 osd.7 since back 2024-09-11T22:09:14.402414+0000 front 2024-09-11T22:09:14.402488+0000 (oldest deadline 2024-09-11T22:09:38.502275+0000) 2024-09-11T22:09:46.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:45 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:45.802+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.893190+0000 front 2024-09-11T22:08:16.893259+0000 (oldest deadline 2024-09-11T22:08:39.193066+0000) 2024-09-11T22:09:46.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:45 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:45.802+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:21.494266+0000 front 2024-09-11T22:08:21.494360+0000 (oldest deadline 2024-09-11T22:08:47.394416+0000) 2024-09-11T22:09:46.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:45 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:45.802+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:33.295337+0000 front 2024-09-11T22:08:33.295289+0000 (oldest deadline 2024-09-11T22:08:56.795428+0000) 2024-09-11T22:09:46.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:45 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:45.802+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:39.497398+0000 front 2024-09-11T22:08:39.497364+0000 (oldest deadline 2024-09-11T22:09:04.797240+0000) 2024-09-11T22:09:46.093 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:45 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:45.802+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.598954+0000 front 2024-09-11T22:08:50.598798+0000 (oldest deadline 2024-09-11T22:09:15.199166+0000) 2024-09-11T22:09:46.093 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:45 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:45.802+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-11T22:08:58.100016+0000 front 2024-09-11T22:08:58.100085+0000 (oldest deadline 2024-09-11T22:09:23.400061+0000) 2024-09-11T22:09:46.093 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:45 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:45.802+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6820 osd.7 since back 2024-09-11T22:09:14.402414+0000 front 2024-09-11T22:09:14.402488+0000 (oldest deadline 2024-09-11T22:09:38.502275+0000) 2024-09-11T22:09:47.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:46 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:46.766+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-09-11T22:08:16.893190+0000 front 2024-09-11T22:08:16.893259+0000 (oldest deadline 2024-09-11T22:08:39.193066+0000) 2024-09-11T22:09:47.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:46 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:46.766+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6804 osd.1 since back 2024-09-11T22:08:21.494266+0000 front 2024-09-11T22:08:21.494360+0000 (oldest deadline 2024-09-11T22:08:47.394416+0000) 2024-09-11T22:09:47.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:46 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:46.766+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6804 osd.2 since back 2024-09-11T22:08:33.295337+0000 front 2024-09-11T22:08:33.295289+0000 (oldest deadline 2024-09-11T22:08:56.795428+0000) 2024-09-11T22:09:47.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:46 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:46.766+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-09-11T22:08:39.497398+0000 front 2024-09-11T22:08:39.497364+0000 (oldest deadline 2024-09-11T22:09:04.797240+0000) 2024-09-11T22:09:47.092 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:46 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:46.766+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6812 osd.4 since back 2024-09-11T22:08:50.598954+0000 front 2024-09-11T22:08:50.598798+0000 (oldest deadline 2024-09-11T22:09:15.199166+0000) 2024-09-11T22:09:47.093 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:46 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:46.766+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6812 osd.5 since back 2024-09-11T22:08:58.100016+0000 front 2024-09-11T22:08:58.100085+0000 (oldest deadline 2024-09-11T22:09:23.400061+0000) 2024-09-11T22:09:47.093 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:46 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:46.766+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.53:6820 osd.7 since back 2024-09-11T22:09:14.402414+0000 front 2024-09-11T22:09:14.402488+0000 (oldest deadline 2024-09-11T22:09:38.502275+0000) 2024-09-11T22:09:47.093 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:46 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11[60831]: 2024-09-11T22:09:46.766+0000 7f67a3af2640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.161:6820 osd.8 since back 2024-09-11T22:09:21.403015+0000 front 2024-09-11T22:09:21.403051+0000 (oldest deadline 2024-09-11T22:09:46.702988+0000) 2024-09-11T22:09:47.873 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:47 smithi161 podman[72312]: 2024-09-11 22:09:47.595437559 +0000 UTC m=+5.619110962 container died 8ab1b5a399c9686dfdf553b0478339db30235888c420ed868feff7cdcc198c36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-11T22:09:47.873 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:47 smithi161 podman[72312]: 2024-09-11 22:09:47.872988632 +0000 UTC m=+5.896662034 container cleanup 8ab1b5a399c9686dfdf553b0478339db30235888c420ed868feff7cdcc198c36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True) 2024-09-11T22:09:48.342 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:47 smithi161 bash[72312]: ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11 2024-09-11T22:09:48.342 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:48 smithi161 podman[72330]: 2024-09-11 22:09:48.099136303 +0000 UTC m=+0.498257603 container remove 8ab1b5a399c9686dfdf553b0478339db30235888c420ed868feff7cdcc198c36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-11T22:09:48.702 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:48 smithi161 podman[72419]: 2024-09-11 22:09:48.399904981 +0000 UTC m=+0.101608541 container create 139bb6ba7e98da20b68df1f6243594de8d195d533427a26e53a2430331490cfd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T22:09:48.702 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:48 smithi161 podman[72419]: 2024-09-11 22:09:48.314061342 +0000 UTC m=+0.015764834 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T22:09:48.703 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:48 smithi161 podman[72419]: 2024-09-11 22:09:48.627362182 +0000 UTC m=+0.329065670 container init 139bb6ba7e98da20b68df1f6243594de8d195d533427a26e53a2430331490cfd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-11T22:09:48.703 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:48 smithi161 podman[72419]: 2024-09-11 22:09:48.630601759 +0000 UTC m=+0.332305246 container start 139bb6ba7e98da20b68df1f6243594de8d195d533427a26e53a2430331490cfd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS) 2024-09-11T22:09:48.703 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:48 smithi161 podman[72419]: 2024-09-11 22:09:48.702137405 +0000 UTC m=+0.403840907 container attach 139bb6ba7e98da20b68df1f6243594de8d195d533427a26e53a2430331490cfd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-11T22:09:49.071 INFO:journalctl@ceph.osd.11.smithi161.stdout:Sep 11 22:09:48 smithi161 podman[72419]: 2024-09-11 22:09:48.8226696 +0000 UTC m=+0.524373092 container died 139bb6ba7e98da20b68df1f6243594de8d195d533427a26e53a2430331490cfd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-osd-11-deactivate, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-11T22:09:49.340 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3ffca83c-7087-11ef-bcea-c7b262605968@osd.11.service' 2024-09-11T22:09:49.378 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:09:49.378 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-11T22:09:49.378 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-11T22:09:49.378 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mds.a 2024-09-11T22:09:49.681 INFO:journalctl@ceph.mds.a.smithi001.stdout:Sep 11 22:09:49 smithi001 systemd[1]: Stopping Ceph mds.a for 3ffca83c-7087-11ef-bcea-c7b262605968... 2024-09-11T22:09:49.943 INFO:journalctl@ceph.mds.a.smithi001.stdout:Sep 11 22:09:49 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-a[69747]: 2024-09-11T22:09:49.680+0000 7f4f16765640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:09:49.943 INFO:journalctl@ceph.mds.a.smithi001.stdout:Sep 11 22:09:49 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-a[69747]: 2024-09-11T22:09:49.680+0000 7f4f16765640 -1 mds.a *** got signal Terminated *** 2024-09-11T22:09:53.889 INFO:journalctl@ceph.mds.a.smithi001.stdout:Sep 11 22:09:53 smithi001 podman[208834]: 2024-09-11 22:09:53.498535066 +0000 UTC m=+3.921043349 container died 78f9f386febc1c5c88a73a3f42496dba74df06ad21f429c7706e26a5fd30a4cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-a, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-11T22:09:54.193 INFO:journalctl@ceph.mds.a.smithi001.stdout:Sep 11 22:09:54 smithi001 podman[208834]: 2024-09-11 22:09:54.038370431 +0000 UTC m=+4.460878704 container cleanup 78f9f386febc1c5c88a73a3f42496dba74df06ad21f429c7706e26a5fd30a4cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-a, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-11T22:09:54.193 INFO:journalctl@ceph.mds.a.smithi001.stdout:Sep 11 22:09:54 smithi001 bash[208834]: ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-a 2024-09-11T22:09:54.475 DEBUG:teuthology.orchestra.run.smithi001:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mds.a.service' 2024-09-11T22:09:54.500 INFO:journalctl@ceph.mds.a.smithi001.stdout:Sep 11 22:09:54 smithi001 podman[208848]: 2024-09-11 22:09:54.24820515 +0000 UTC m=+0.746238903 container remove 78f9f386febc1c5c88a73a3f42496dba74df06ad21f429c7706e26a5fd30a4cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-a, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T22:09:54.500 INFO:journalctl@ceph.mds.a.smithi001.stdout:Sep 11 22:09:54 smithi001 systemd[1]: ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mds.a.service: Deactivated successfully. 2024-09-11T22:09:54.500 INFO:journalctl@ceph.mds.a.smithi001.stdout:Sep 11 22:09:54 smithi001 systemd[1]: Stopped Ceph mds.a for 3ffca83c-7087-11ef-bcea-c7b262605968. 2024-09-11T22:09:54.500 INFO:journalctl@ceph.mds.a.smithi001.stdout:Sep 11 22:09:54 smithi001 systemd[1]: ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mds.a.service: Consumed 10.754s CPU time. 2024-09-11T22:09:54.510 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:09:54.511 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-11T22:09:54.511 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-11T22:09:54.511 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mds.d 2024-09-11T22:09:54.876 INFO:journalctl@ceph.mds.d.smithi001.stdout:Sep 11 22:09:54 smithi001 systemd[1]: Stopping Ceph mds.d for 3ffca83c-7087-11ef-bcea-c7b262605968... 2024-09-11T22:09:55.193 INFO:journalctl@ceph.mds.d.smithi001.stdout:Sep 11 22:09:54 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-d[68547]: 2024-09-11T22:09:54.874+0000 7f6e39311640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:09:55.193 INFO:journalctl@ceph.mds.d.smithi001.stdout:Sep 11 22:09:54 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-d[68547]: 2024-09-11T22:09:54.874+0000 7f6e39311640 -1 mds.d *** got signal Terminated *** 2024-09-11T22:09:56.593 INFO:journalctl@ceph.mds.d.smithi001.stdout:Sep 11 22:09:56 smithi001 podman[208954]: 2024-09-11 22:09:56.322849546 +0000 UTC m=+1.592640836 container died 31b9b50c2867b44155d02bfa2c4a6cff7d99c16bd81796cbb713126450eb852d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-d, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux ) 2024-09-11T22:09:56.943 INFO:journalctl@ceph.mds.d.smithi001.stdout:Sep 11 22:09:56 smithi001 podman[208954]: 2024-09-11 22:09:56.767620194 +0000 UTC m=+2.037411527 container cleanup 31b9b50c2867b44155d02bfa2c4a6cff7d99c16bd81796cbb713126450eb852d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-d, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-11T22:09:56.943 INFO:journalctl@ceph.mds.d.smithi001.stdout:Sep 11 22:09:56 smithi001 bash[208954]: ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-d 2024-09-11T22:09:57.201 DEBUG:teuthology.orchestra.run.smithi001:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mds.d.service' 2024-09-11T22:09:57.224 INFO:journalctl@ceph.mds.d.smithi001.stdout:Sep 11 22:09:56 smithi001 podman[208967]: 2024-09-11 22:09:56.969002294 +0000 UTC m=+0.643676392 container remove 31b9b50c2867b44155d02bfa2c4a6cff7d99c16bd81796cbb713126450eb852d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-d, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-11T22:09:57.224 INFO:journalctl@ceph.mds.d.smithi001.stdout:Sep 11 22:09:57 smithi001 systemd[1]: ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mds.d.service: Deactivated successfully. 2024-09-11T22:09:57.224 INFO:journalctl@ceph.mds.d.smithi001.stdout:Sep 11 22:09:57 smithi001 systemd[1]: Stopped Ceph mds.d for 3ffca83c-7087-11ef-bcea-c7b262605968. 2024-09-11T22:09:57.224 INFO:journalctl@ceph.mds.d.smithi001.stdout:Sep 11 22:09:57 smithi001 systemd[1]: ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mds.d.service: Consumed 1.054s CPU time. 2024-09-11T22:09:57.236 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:09:57.236 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-11T22:09:57.236 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-11T22:09:57.237 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mds.g 2024-09-11T22:09:57.693 INFO:journalctl@ceph.mds.g.smithi001.stdout:Sep 11 22:09:57 smithi001 systemd[1]: Stopping Ceph mds.g for 3ffca83c-7087-11ef-bcea-c7b262605968... 2024-09-11T22:09:58.193 INFO:journalctl@ceph.mds.g.smithi001.stdout:Sep 11 22:09:57 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-g[69388]: 2024-09-11T22:09:57.707+0000 7ff0ac8ec640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:09:58.193 INFO:journalctl@ceph.mds.g.smithi001.stdout:Sep 11 22:09:57 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-g[69388]: 2024-09-11T22:09:57.707+0000 7ff0ac8ec640 -1 mds.g *** got signal Terminated *** 2024-09-11T22:10:01.897 INFO:journalctl@ceph.mds.g.smithi001.stdout:Sep 11 22:10:01 smithi001 podman[209074]: 2024-09-11 22:10:01.504195299 +0000 UTC m=+4.048413946 container died c4b4c7930b9d77ff9418c8cc566a167e5da1135bfdb188a10fbb77e87b4db5c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-g, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-11T22:10:02.193 INFO:journalctl@ceph.mds.g.smithi001.stdout:Sep 11 22:10:02 smithi001 podman[209074]: 2024-09-11 22:10:02.046697674 +0000 UTC m=+4.590916323 container cleanup c4b4c7930b9d77ff9418c8cc566a167e5da1135bfdb188a10fbb77e87b4db5c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-g, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-11T22:10:02.194 INFO:journalctl@ceph.mds.g.smithi001.stdout:Sep 11 22:10:02 smithi001 bash[209074]: ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-g 2024-09-11T22:10:02.470 DEBUG:teuthology.orchestra.run.smithi001:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mds.g.service' 2024-09-11T22:10:02.493 INFO:journalctl@ceph.mds.g.smithi001.stdout:Sep 11 22:10:02 smithi001 podman[209087]: 2024-09-11 22:10:02.231482426 +0000 UTC m=+0.725544662 container remove c4b4c7930b9d77ff9418c8cc566a167e5da1135bfdb188a10fbb77e87b4db5c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T22:10:02.493 INFO:journalctl@ceph.mds.g.smithi001.stdout:Sep 11 22:10:02 smithi001 systemd[1]: ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mds.g.service: Deactivated successfully. 2024-09-11T22:10:02.493 INFO:journalctl@ceph.mds.g.smithi001.stdout:Sep 11 22:10:02 smithi001 systemd[1]: Stopped Ceph mds.g for 3ffca83c-7087-11ef-bcea-c7b262605968. 2024-09-11T22:10:02.493 INFO:journalctl@ceph.mds.g.smithi001.stdout:Sep 11 22:10:02 smithi001 systemd[1]: ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mds.g.service: Consumed 15.067s CPU time. 2024-09-11T22:10:02.506 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:10:02.506 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-11T22:10:02.506 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-11T22:10:02.506 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mds.j 2024-09-11T22:10:02.818 INFO:journalctl@ceph.mds.j.smithi001.stdout:Sep 11 22:10:02 smithi001 systemd[1]: Stopping Ceph mds.j for 3ffca83c-7087-11ef-bcea-c7b262605968... 2024-09-11T22:10:03.193 INFO:journalctl@ceph.mds.j.smithi001.stdout:Sep 11 22:10:02 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-j[68146]: 2024-09-11T22:10:02.816+0000 7ff80ba64640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:10:03.193 INFO:journalctl@ceph.mds.j.smithi001.stdout:Sep 11 22:10:02 smithi001 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-j[68146]: 2024-09-11T22:10:02.816+0000 7ff80ba64640 -1 mds.j *** got signal Terminated *** 2024-09-11T22:10:04.943 INFO:journalctl@ceph.mds.j.smithi001.stdout:Sep 11 22:10:04 smithi001 podman[209197]: 2024-09-11 22:10:04.567181441 +0000 UTC m=+1.836626155 container died 7c0ac8cf66ff7cb265a0652f4dd1051db87f60a2eb23d0da76f696e5f7da89b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-j, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS) 2024-09-11T22:10:05.427 INFO:journalctl@ceph.mds.j.smithi001.stdout:Sep 11 22:10:05 smithi001 podman[209197]: 2024-09-11 22:10:05.151086765 +0000 UTC m=+2.420531540 container cleanup 7c0ac8cf66ff7cb265a0652f4dd1051db87f60a2eb23d0da76f696e5f7da89b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-j, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T22:10:05.427 INFO:journalctl@ceph.mds.j.smithi001.stdout:Sep 11 22:10:05 smithi001 bash[209197]: ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-j 2024-09-11T22:10:05.553 DEBUG:teuthology.orchestra.run.smithi001:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mds.j.service' 2024-09-11T22:10:05.586 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:10:05.586 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-11T22:10:05.586 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-11T22:10:05.587 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mds.b 2024-09-11T22:10:05.878 INFO:journalctl@ceph.mds.b.smithi053.stdout:Sep 11 22:10:05 smithi053 systemd[1]: Stopping Ceph mds.b for 3ffca83c-7087-11ef-bcea-c7b262605968... 2024-09-11T22:10:06.297 INFO:journalctl@ceph.mds.b.smithi053.stdout:Sep 11 22:10:05 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-b[64373]: 2024-09-11T22:10:05.877+0000 7f94a6c80640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:10:06.297 INFO:journalctl@ceph.mds.b.smithi053.stdout:Sep 11 22:10:05 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-b[64373]: 2024-09-11T22:10:05.877+0000 7f94a6c80640 -1 mds.b *** got signal Terminated *** 2024-09-11T22:10:09.668 INFO:journalctl@ceph.mds.b.smithi053.stdout:Sep 11 22:10:09 smithi053 podman[71073]: 2024-09-11 22:10:09.373051834 +0000 UTC m=+3.588004971 container died 7e08278586c4a359fdb0e1ef579061b2014c783e233d0cd77992014fdda689ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-11T22:10:09.669 INFO:journalctl@ceph.mds.b.smithi053.stdout:Sep 11 22:10:09 smithi053 podman[71073]: 2024-09-11 22:10:09.66848487 +0000 UTC m=+3.883438014 container cleanup 7e08278586c4a359fdb0e1ef579061b2014c783e233d0cd77992014fdda689ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-11T22:10:10.037 INFO:journalctl@ceph.mds.b.smithi053.stdout:Sep 11 22:10:09 smithi053 bash[71073]: ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-b 2024-09-11T22:10:10.038 INFO:journalctl@ceph.mds.b.smithi053.stdout:Sep 11 22:10:09 smithi053 podman[71085]: 2024-09-11 22:10:09.903097289 +0000 UTC m=+0.527990385 container remove 7e08278586c4a359fdb0e1ef579061b2014c783e233d0cd77992014fdda689ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-b, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=) 2024-09-11T22:10:10.052 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mds.b.service' 2024-09-11T22:10:10.086 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:10:10.086 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-11T22:10:10.087 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-11T22:10:10.087 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mds.e 2024-09-11T22:10:10.297 INFO:journalctl@ceph.mds.e.smithi053.stdout:Sep 11 22:10:10 smithi053 systemd[1]: Stopping Ceph mds.e for 3ffca83c-7087-11ef-bcea-c7b262605968... 2024-09-11T22:10:10.797 INFO:journalctl@ceph.mds.e.smithi053.stdout:Sep 11 22:10:10 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-e[63763]: 2024-09-11T22:10:10.388+0000 7f18afcec640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:10:10.797 INFO:journalctl@ceph.mds.e.smithi053.stdout:Sep 11 22:10:10 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-e[63763]: 2024-09-11T22:10:10.388+0000 7f18afcec640 -1 mds.e *** got signal Terminated *** 2024-09-11T22:10:13.666 INFO:journalctl@ceph.mds.e.smithi053.stdout:Sep 11 22:10:13 smithi053 podman[71192]: 2024-09-11 22:10:13.376081813 +0000 UTC m=+3.071299032 container died 43bc380734bae6bd44f3327ac5b7a1f2d8df2f9d31d4a7bbaf54341a01bcc6b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-e, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-11T22:10:14.007 INFO:journalctl@ceph.mds.e.smithi053.stdout:Sep 11 22:10:13 smithi053 podman[71192]: 2024-09-11 22:10:13.781465649 +0000 UTC m=+3.476682914 container cleanup 43bc380734bae6bd44f3327ac5b7a1f2d8df2f9d31d4a7bbaf54341a01bcc6b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-e, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-11T22:10:14.008 INFO:journalctl@ceph.mds.e.smithi053.stdout:Sep 11 22:10:13 smithi053 bash[71192]: ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-e 2024-09-11T22:10:14.153 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mds.e.service' 2024-09-11T22:10:14.187 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:10:14.187 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-11T22:10:14.187 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-11T22:10:14.187 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mds.h 2024-09-11T22:10:14.547 INFO:journalctl@ceph.mds.h.smithi053.stdout:Sep 11 22:10:14 smithi053 systemd[1]: Stopping Ceph mds.h for 3ffca83c-7087-11ef-bcea-c7b262605968... 2024-09-11T22:10:14.548 INFO:journalctl@ceph.mds.h.smithi053.stdout:Sep 11 22:10:14 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-h[64063]: 2024-09-11T22:10:14.509+0000 7fb87ff85640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:10:14.548 INFO:journalctl@ceph.mds.h.smithi053.stdout:Sep 11 22:10:14 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-h[64063]: 2024-09-11T22:10:14.509+0000 7fb87ff85640 -1 mds.h *** got signal Terminated *** 2024-09-11T22:10:17.693 INFO:journalctl@ceph.mds.h.smithi053.stdout:Sep 11 22:10:17 smithi053 podman[71326]: 2024-09-11 22:10:17.369255022 +0000 UTC m=+2.961709021 container died 84569b980f2d2d0c3c8221ccebaa3469e7e744a0fd3eab0c353314de2699ce90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-h, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-11T22:10:18.047 INFO:journalctl@ceph.mds.h.smithi053.stdout:Sep 11 22:10:17 smithi053 podman[71326]: 2024-09-11 22:10:17.693384845 +0000 UTC m=+3.285838844 container cleanup 84569b980f2d2d0c3c8221ccebaa3469e7e744a0fd3eab0c353314de2699ce90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-h, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T22:10:18.047 INFO:journalctl@ceph.mds.h.smithi053.stdout:Sep 11 22:10:17 smithi053 bash[71326]: ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-h 2024-09-11T22:10:18.285 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mds.h.service' 2024-09-11T22:10:18.318 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:10:18.318 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-11T22:10:18.318 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-11T22:10:18.318 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mds.k 2024-09-11T22:10:18.639 INFO:journalctl@ceph.mds.k.smithi053.stdout:Sep 11 22:10:18 smithi053 systemd[1]: Stopping Ceph mds.k for 3ffca83c-7087-11ef-bcea-c7b262605968... 2024-09-11T22:10:19.047 INFO:journalctl@ceph.mds.k.smithi053.stdout:Sep 11 22:10:18 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-k[63453]: 2024-09-11T22:10:18.638+0000 7ff6cd1a1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:10:19.047 INFO:journalctl@ceph.mds.k.smithi053.stdout:Sep 11 22:10:18 smithi053 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-k[63453]: 2024-09-11T22:10:18.638+0000 7ff6cd1a1640 -1 mds.k *** got signal Terminated *** 2024-09-11T22:10:21.657 INFO:journalctl@ceph.mds.k.smithi053.stdout:Sep 11 22:10:21 smithi053 podman[71447]: 2024-09-11 22:10:21.380530271 +0000 UTC m=+2.843394299 container died 18b148130eb910259e6b0a67e898f13147ce401da6740458fa48b06bada9a0b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-11T22:10:21.919 INFO:journalctl@ceph.mds.k.smithi053.stdout:Sep 11 22:10:21 smithi053 podman[71447]: 2024-09-11 22:10:21.789302451 +0000 UTC m=+3.252166469 container cleanup 18b148130eb910259e6b0a67e898f13147ce401da6740458fa48b06bada9a0b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T22:10:21.919 INFO:journalctl@ceph.mds.k.smithi053.stdout:Sep 11 22:10:21 smithi053 bash[71447]: ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-k 2024-09-11T22:10:22.153 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mds.k.service' 2024-09-11T22:10:22.175 INFO:journalctl@ceph.mds.k.smithi053.stdout:Sep 11 22:10:22 smithi053 podman[71459]: 2024-09-11 22:10:22.01524719 +0000 UTC m=+0.635232636 container remove 18b148130eb910259e6b0a67e898f13147ce401da6740458fa48b06bada9a0b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-k, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-11T22:10:22.175 INFO:journalctl@ceph.mds.k.smithi053.stdout:Sep 11 22:10:22 smithi053 systemd[1]: ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mds.k.service: Deactivated successfully. 2024-09-11T22:10:22.175 INFO:journalctl@ceph.mds.k.smithi053.stdout:Sep 11 22:10:22 smithi053 systemd[1]: Stopped Ceph mds.k for 3ffca83c-7087-11ef-bcea-c7b262605968. 2024-09-11T22:10:22.175 INFO:journalctl@ceph.mds.k.smithi053.stdout:Sep 11 22:10:22 smithi053 systemd[1]: ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mds.k.service: Consumed 13.601s CPU time. 2024-09-11T22:10:22.186 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:10:22.186 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-11T22:10:22.186 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-11T22:10:22.187 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mds.c 2024-09-11T22:10:22.505 INFO:journalctl@ceph.mds.c.smithi161.stdout:Sep 11 22:10:22 smithi161 systemd[1]: Stopping Ceph mds.c for 3ffca83c-7087-11ef-bcea-c7b262605968... 2024-09-11T22:10:22.762 INFO:journalctl@ceph.mds.c.smithi161.stdout:Sep 11 22:10:22 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-c[65663]: 2024-09-11T22:10:22.503+0000 7f8830a0d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:10:22.762 INFO:journalctl@ceph.mds.c.smithi161.stdout:Sep 11 22:10:22 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-c[65663]: 2024-09-11T22:10:22.503+0000 7f8830a0d640 -1 mds.c *** got signal Terminated *** 2024-09-11T22:10:25.655 INFO:journalctl@ceph.mds.c.smithi161.stdout:Sep 11 22:10:25 smithi161 podman[72522]: 2024-09-11 22:10:25.370849221 +0000 UTC m=+2.988309024 container died a2ca22e4e25759fd85e8e5b69fb05c5a85b0a156951178b058779845d4e31e91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-c, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-09-11T22:10:26.021 INFO:journalctl@ceph.mds.c.smithi161.stdout:Sep 11 22:10:25 smithi161 podman[72522]: 2024-09-11 22:10:25.745448432 +0000 UTC m=+3.362908234 container cleanup a2ca22e4e25759fd85e8e5b69fb05c5a85b0a156951178b058779845d4e31e91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-c, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-11T22:10:26.022 INFO:journalctl@ceph.mds.c.smithi161.stdout:Sep 11 22:10:25 smithi161 bash[72522]: ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-c 2024-09-11T22:10:26.184 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mds.c.service' 2024-09-11T22:10:26.220 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:10:26.221 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-11T22:10:26.221 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-11T22:10:26.221 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mds.f 2024-09-11T22:10:26.542 INFO:journalctl@ceph.mds.f.smithi161.stdout:Sep 11 22:10:26 smithi161 systemd[1]: Stopping Ceph mds.f for 3ffca83c-7087-11ef-bcea-c7b262605968... 2024-09-11T22:10:26.842 INFO:journalctl@ceph.mds.f.smithi161.stdout:Sep 11 22:10:26 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-f[65361]: 2024-09-11T22:10:26.540+0000 7f84a0fea640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:10:26.842 INFO:journalctl@ceph.mds.f.smithi161.stdout:Sep 11 22:10:26 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-f[65361]: 2024-09-11T22:10:26.540+0000 7f84a0fea640 -1 mds.f *** got signal Terminated *** 2024-09-11T22:10:29.691 INFO:journalctl@ceph.mds.f.smithi161.stdout:Sep 11 22:10:29 smithi161 podman[72655]: 2024-09-11 22:10:29.369026622 +0000 UTC m=+2.931731218 container died dc51508f85587e6e1f75facf4de66d2fea202a2c48e31a1af1c69a1676096f87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-f, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-11T22:10:30.092 INFO:journalctl@ceph.mds.f.smithi161.stdout:Sep 11 22:10:29 smithi161 podman[72655]: 2024-09-11 22:10:29.690664355 +0000 UTC m=+3.253368943 container cleanup dc51508f85587e6e1f75facf4de66d2fea202a2c48e31a1af1c69a1676096f87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-f, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-11T22:10:30.092 INFO:journalctl@ceph.mds.f.smithi161.stdout:Sep 11 22:10:29 smithi161 bash[72655]: ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-f 2024-09-11T22:10:30.092 INFO:journalctl@ceph.mds.f.smithi161.stdout:Sep 11 22:10:29 smithi161 podman[72668]: 2024-09-11 22:10:29.875178542 +0000 UTC m=+0.501632212 container remove dc51508f85587e6e1f75facf4de66d2fea202a2c48e31a1af1c69a1676096f87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-f, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-11T22:10:30.113 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mds.f.service' 2024-09-11T22:10:30.146 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:10:30.147 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-11T22:10:30.147 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-11T22:10:30.147 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mds.i 2024-09-11T22:10:30.445 INFO:journalctl@ceph.mds.i.smithi161.stdout:Sep 11 22:10:30 smithi161 systemd[1]: Stopping Ceph mds.i for 3ffca83c-7087-11ef-bcea-c7b262605968... 2024-09-11T22:10:30.841 INFO:journalctl@ceph.mds.i.smithi161.stdout:Sep 11 22:10:30 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-i[65965]: 2024-09-11T22:10:30.444+0000 7eff82060640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:10:30.842 INFO:journalctl@ceph.mds.i.smithi161.stdout:Sep 11 22:10:30 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-i[65965]: 2024-09-11T22:10:30.444+0000 7eff82060640 -1 mds.i *** got signal Terminated *** 2024-09-11T22:10:33.592 INFO:journalctl@ceph.mds.i.smithi161.stdout:Sep 11 22:10:33 smithi161 podman[72778]: 2024-09-11 22:10:33.374539602 +0000 UTC m=+3.014366639 container died 55301ac4334fa3975f19c228a27daaea647193dc14a470c95c65e01d3544ed70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-i, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-11T22:10:33.928 INFO:journalctl@ceph.mds.i.smithi161.stdout:Sep 11 22:10:33 smithi161 podman[72778]: 2024-09-11 22:10:33.702296183 +0000 UTC m=+3.342123209 container cleanup 55301ac4334fa3975f19c228a27daaea647193dc14a470c95c65e01d3544ed70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-11T22:10:33.929 INFO:journalctl@ceph.mds.i.smithi161.stdout:Sep 11 22:10:33 smithi161 bash[72778]: ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-i 2024-09-11T22:10:34.144 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mds.i.service' 2024-09-11T22:10:34.177 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:10:34.177 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-11T22:10:34.177 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-11T22:10:34.177 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mds.l 2024-09-11T22:10:34.498 INFO:journalctl@ceph.mds.l.smithi161.stdout:Sep 11 22:10:34 smithi161 systemd[1]: Stopping Ceph mds.l for 3ffca83c-7087-11ef-bcea-c7b262605968... 2024-09-11T22:10:34.842 INFO:journalctl@ceph.mds.l.smithi161.stdout:Sep 11 22:10:34 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-l[65060]: 2024-09-11T22:10:34.497+0000 7fe604eed640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:10:34.842 INFO:journalctl@ceph.mds.l.smithi161.stdout:Sep 11 22:10:34 smithi161 ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-l[65060]: 2024-09-11T22:10:34.497+0000 7fe604eed640 -1 mds.l *** got signal Terminated *** 2024-09-11T22:10:37.648 INFO:journalctl@ceph.mds.l.smithi161.stdout:Sep 11 22:10:37 smithi161 podman[72901]: 2024-09-11 22:10:37.36579907 +0000 UTC m=+2.976071705 container died c8172226b47872aa8ed04235a4bed705e5eebae99826fb76141579bab1a1d985 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-l, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-11T22:10:37.998 INFO:journalctl@ceph.mds.l.smithi161.stdout:Sep 11 22:10:37 smithi161 podman[72901]: 2024-09-11 22:10:37.72251452 +0000 UTC m=+3.332787143 container cleanup c8172226b47872aa8ed04235a4bed705e5eebae99826fb76141579bab1a1d985 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-l, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True) 2024-09-11T22:10:37.998 INFO:journalctl@ceph.mds.l.smithi161.stdout:Sep 11 22:10:37 smithi161 bash[72901]: ceph-3ffca83c-7087-11ef-bcea-c7b262605968-mds-l 2024-09-11T22:10:38.156 DEBUG:teuthology.orchestra.run.smithi161:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3ffca83c-7087-11ef-bcea-c7b262605968@mds.l.service' 2024-09-11T22:10:38.188 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:10:38.189 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-11T22:10:38.189 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-11T22:10:38.401 INFO:teuthology.orchestra.run.smithi001.stdout:Deleting cluster with fsid: 3ffca83c-7087-11ef-bcea-c7b262605968 2024-09-11T22:10:41.304 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-11T22:10:41.513 INFO:teuthology.orchestra.run.smithi053.stdout:Deleting cluster with fsid: 3ffca83c-7087-11ef-bcea-c7b262605968 2024-09-11T22:10:44.017 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-11T22:10:44.217 INFO:teuthology.orchestra.run.smithi161.stdout:Deleting cluster with fsid: 3ffca83c-7087-11ef-bcea-c7b262605968 2024-09-11T22:10:46.596 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T22:10:46.630 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T22:10:46.663 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T22:10:46.690 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-11T22:10:46.692 DEBUG:teuthology.misc:Transferring archived files from smithi001:/var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901408/remote/smithi001/crash 2024-09-11T22:10:46.692 DEBUG:teuthology.orchestra.run.smithi001:> sudo tar c -f - -C /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/crash -- . 2024-09-11T22:10:46.763 INFO:teuthology.orchestra.run.smithi001.stderr:tar: /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-11T22:10:46.763 INFO:teuthology.orchestra.run.smithi001.stderr:tar: Error is not recoverable: exiting now 2024-09-11T22:10:46.765 DEBUG:teuthology.misc:Transferring archived files from smithi053:/var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901408/remote/smithi053/crash 2024-09-11T22:10:46.766 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar c -f - -C /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/crash -- . 2024-09-11T22:10:46.819 INFO:teuthology.orchestra.run.smithi053.stderr:tar: /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-11T22:10:46.819 INFO:teuthology.orchestra.run.smithi053.stderr:tar: Error is not recoverable: exiting now 2024-09-11T22:10:46.821 DEBUG:teuthology.misc:Transferring archived files from smithi161:/var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901408/remote/smithi161/crash 2024-09-11T22:10:46.822 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar c -f - -C /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/crash -- . 2024-09-11T22:10:46.859 INFO:teuthology.orchestra.run.smithi161.stderr:tar: /var/lib/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-11T22:10:46.860 INFO:teuthology.orchestra.run.smithi161.stderr:tar: Error is not recoverable: exiting now 2024-09-11T22:10:46.861 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-11T22:10:46.861 DEBUG:teuthology.orchestra.run.smithi001:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-11T22:10:46.956 INFO:teuthology.orchestra.run.smithi001.stdout:2024-09-11T22:05:46.557348+0000 mds.b (mds.0) 36 : cluster [ERR] failed to store backtrace on ino 0x100000069d1 object, pool 2, errno -2 2024-09-11T22:10:46.956 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-09-11T22:10:46.956 INFO:tasks.cephadm:Compressing logs... 2024-09-11T22:10:46.957 DEBUG:teuthology.orchestra.run.smithi001:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-11T22:10:46.959 DEBUG:teuthology.orchestra.run.smithi053:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-11T22:10:46.962 DEBUG:teuthology.orchestra.run.smithi161:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-11T22:10:46.986 INFO:teuthology.orchestra.run.smithi161.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-11T22:10:46.986 INFO:teuthology.orchestra.run.smithi161.stderr:: No such file or directory 2024-09-11T22:10:46.987 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-11T22:10:46.988 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-mds.i.log 2024-09-11T22:10:46.988 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-volume.log: /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-mon.c.log 2024-09-11T22:10:46.988 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-11T22:10:46.988 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-osd.11.log 2024-09-11T22:10:46.994 INFO:teuthology.orchestra.run.smithi053.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-11T22:10:46.994 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-osd.10.log 2024-09-11T22:10:46.994 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-osd.1.log 2024-09-11T22:10:46.994 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-osd.7.log 2024-09-11T22:10:46.995 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-11T22:10:46.995 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-mds.b.log 2024-09-11T22:10:46.995 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-osd.1.log: /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-osd.7.log: /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-mds.k.log 2024-09-11T22:10:46.995 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-osd.4.log 2024-09-11T22:10:46.995 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-11T22:10:46.995 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-11T22:10:46.997 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-mon.c.log: /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-osd.2.log 2024-09-11T22:10:46.997 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-osd.11.log: gzip 82.0% -- replaced with /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-11T22:10:46.997 INFO:teuthology.orchestra.run.smithi161.stderr: -5 --verbose -- /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-osd.5.log 2024-09-11T22:10:46.997 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph.log 2024-09-11T22:10:46.997 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-mds.l.log 2024-09-11T22:10:46.998 INFO:teuthology.orchestra.run.smithi161.stderr: 90.4% -- replaced with /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-11T22:10:46.998 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-osd.8.log 2024-09-11T22:10:46.998 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-mds.l.log: 89.8% -- replaced with /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-11T22:10:46.998 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-mgr.z.log 2024-09-11T22:10:46.999 INFO:teuthology.orchestra.run.smithi001.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-osd.0.log 2024-09-11T22:10:46.999 INFO:teuthology.orchestra.run.smithi001.stderr:: No such file or directory 2024-09-11T22:10:47.000 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph.audit.log: 90.4% -- replaced with /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-11T22:10:47.002 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-osd.8.log: 91.0% -- replaced with /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-11T22:10:47.002 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-mds.c.log 2024-09-11T22:10:47.002 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-mgr.y.log 2024-09-11T22:10:47.003 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph.cephadm.log: 82.1% -- replaced with /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-11T22:10:47.003 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph.log 2024-09-11T22:10:47.004 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-osd.9.log 2024-09-11T22:10:47.005 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-mgr.y.log: 90.8% -- replaced with /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-11T22:10:47.006 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-11T22:10:47.006 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-osd.0.log: /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-11T22:10:47.007 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-mds.a.log 2024-09-11T22:10:47.007 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-11T22:10:47.009 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-mon.a.log 2024-09-11T22:10:47.009 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-mds.g.log 2024-09-11T22:10:47.009 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-osd.6.log 2024-09-11T22:10:47.009 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph.log 2024-09-11T22:10:47.010 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-mds.g.log: /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-mds.d.log 2024-09-11T22:10:47.010 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-mgr.x.log 2024-09-11T22:10:47.010 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph.log: /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-mds.j.log 2024-09-11T22:10:47.010 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-osd.3.log 2024-09-11T22:10:47.010 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-11T22:10:47.011 INFO:teuthology.orchestra.run.smithi053.stderr: 91.8% -- replaced with /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-mgr.y.log.gz 2024-09-11T22:10:47.012 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-mds.h.log 2024-09-11T22:10:47.012 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-mgr.z.log: 91.9% -- replaced with /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-mgr.z.log.gz 2024-09-11T22:10:47.012 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-mds.f.log 2024-09-11T22:10:47.012 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-osd.3.log: /var/log/ceph/cephadm.log: 89.8% -- replaced with /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-11T22:10:47.016 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph.log: 89.9% -- replaced with /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-11T22:10:47.017 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-mon.b.log 2024-09-11T22:10:47.018 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-mds.e.log 2024-09-11T22:10:47.024 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-11T22:10:47.030 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-mon.b.log: /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-11T22:10:47.045 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-mds.f.log: /var/log/ceph/cephadm.log: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-11T22:10:47.065 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-mds.e.log: /var/log/ceph/cephadm.log: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-11T22:10:47.129 INFO:teuthology.orchestra.run.smithi001.stderr: 94.6% -- replaced with /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-mds.d.log.gz 2024-09-11T22:10:47.132 INFO:teuthology.orchestra.run.smithi001.stderr: 84.6% -- replaced with /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-11T22:10:47.160 INFO:teuthology.orchestra.run.smithi001.stderr: 94.5% -- replaced with /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-mds.j.log.gz 2024-09-11T22:10:47.239 INFO:teuthology.orchestra.run.smithi001.stderr: 90.3% -- replaced with /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-11T22:10:47.313 INFO:teuthology.orchestra.run.smithi001.stderr: 91.0% -- replaced with /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-11T22:10:47.358 INFO:teuthology.orchestra.run.smithi001.stderr: 94.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-11T22:10:47.460 INFO:teuthology.orchestra.run.smithi161.stderr: 91.7% -- replaced with /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-mon.c.log.gz 2024-09-11T22:10:47.581 INFO:teuthology.orchestra.run.smithi053.stderr: 91.6% -- replaced with /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-mon.b.log.gz 2024-09-11T22:10:48.468 INFO:teuthology.orchestra.run.smithi001.stderr: 90.2% -- replaced with /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-mgr.x.log.gz 2024-09-11T22:10:50.169 INFO:teuthology.orchestra.run.smithi001.stderr: 95.1% -- replaced with /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-mds.a.log.gz 2024-09-11T22:10:52.098 INFO:teuthology.orchestra.run.smithi001.stderr: 94.5% -- replaced with /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-mds.g.log.gz 2024-09-11T22:10:52.407 INFO:teuthology.orchestra.run.smithi053.stderr: 94.8% -- replaced with /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-mds.k.log.gz 2024-09-11T22:10:52.672 INFO:teuthology.orchestra.run.smithi001.stderr: 90.4% -- replaced with /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-mon.a.log.gz 2024-09-11T22:10:53.232 INFO:teuthology.orchestra.run.smithi161.stderr: 94.6% -- replaced with /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-mds.l.log.gz 2024-09-11T22:10:54.527 INFO:teuthology.orchestra.run.smithi053.stderr: 94.9% -- replaced with /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-mds.e.log.gz 2024-09-11T22:10:57.819 INFO:teuthology.orchestra.run.smithi053.stderr: 92.2% -- replaced with /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-osd.1.log.gz 2024-09-11T22:10:58.252 INFO:teuthology.orchestra.run.smithi053.stderr: 92.0% -- replaced with /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-osd.4.log.gz 2024-09-11T22:10:59.366 INFO:teuthology.orchestra.run.smithi001.stderr: 92.2% -- replaced with /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-osd.0.log.gz 2024-09-11T22:10:59.421 INFO:teuthology.orchestra.run.smithi001.stderr: 92.2% -- replaced with /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-osd.9.log.gz 2024-09-11T22:11:00.962 INFO:teuthology.orchestra.run.smithi161.stderr: 92.1% -- replaced with /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-osd.11.log.gz 2024-09-11T22:11:01.038 INFO:teuthology.orchestra.run.smithi001.stderr: 91.7% -- replaced with /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-osd.6.log.gz 2024-09-11T22:11:01.485 INFO:teuthology.orchestra.run.smithi161.stderr: 91.8% -- replaced with /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-osd.5.log.gz 2024-09-11T22:11:01.598 INFO:teuthology.orchestra.run.smithi161.stderr: 92.1% -- replaced with /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-osd.2.log.gz 2024-09-11T22:11:02.122 INFO:teuthology.orchestra.run.smithi053.stderr: 91.9% -- replaced with /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-osd.10.log.gz 2024-09-11T22:11:02.160 INFO:teuthology.orchestra.run.smithi001.stderr: 92.0% -- replaced with /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-osd.3.log.gz 2024-09-11T22:11:02.163 INFO:teuthology.orchestra.run.smithi001.stderr: 2024-09-11T22:11:02.163 INFO:teuthology.orchestra.run.smithi001.stderr:real 0m15.189s 2024-09-11T22:11:02.163 INFO:teuthology.orchestra.run.smithi001.stderr:user 0m46.317s 2024-09-11T22:11:02.163 INFO:teuthology.orchestra.run.smithi001.stderr:sys 0m1.583s 2024-09-11T22:11:04.465 INFO:teuthology.orchestra.run.smithi053.stderr: 91.8% -- replaced with /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-osd.7.log.gz 2024-09-11T22:11:04.661 INFO:teuthology.orchestra.run.smithi161.stderr: 91.9% -- replaced with /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-osd.8.log.gz 2024-09-11T22:11:27.518 INFO:teuthology.orchestra.run.smithi161.stderr: 94.1% -- replaced with /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-mds.c.log.gz 2024-09-11T22:11:35.204 INFO:teuthology.orchestra.run.smithi161.stderr: 93.9% -- replaced with /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-mds.f.log.gz 2024-09-11T22:11:36.445 INFO:teuthology.orchestra.run.smithi161.stderr: 93.9% -- replaced with /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-mds.i.log.gz 2024-09-11T22:11:36.447 INFO:teuthology.orchestra.run.smithi161.stderr: 2024-09-11T22:11:36.447 INFO:teuthology.orchestra.run.smithi161.stderr:real 0m49.470s 2024-09-11T22:11:36.447 INFO:teuthology.orchestra.run.smithi161.stderr:user 3m18.755s 2024-09-11T22:11:36.447 INFO:teuthology.orchestra.run.smithi161.stderr:sys 0m6.032s 2024-09-11T22:11:40.804 INFO:teuthology.orchestra.run.smithi053.stderr: 93.9% -- replaced with /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-mds.h.log.gz 2024-09-11T22:11:54.800 INFO:teuthology.orchestra.run.smithi053.stderr: 93.5% -- replaced with /var/log/ceph/3ffca83c-7087-11ef-bcea-c7b262605968/ceph-mds.b.log.gz 2024-09-11T22:11:54.802 INFO:teuthology.orchestra.run.smithi053.stderr: 2024-09-11T22:11:54.802 INFO:teuthology.orchestra.run.smithi053.stderr:real 1m7.825s 2024-09-11T22:11:54.802 INFO:teuthology.orchestra.run.smithi053.stderr:user 3m3.842s 2024-09-11T22:11:54.802 INFO:teuthology.orchestra.run.smithi053.stderr:sys 0m5.051s 2024-09-11T22:11:54.803 INFO:tasks.cephadm:Archiving logs... 2024-09-11T22:11:54.803 DEBUG:teuthology.misc:Transferring archived files from smithi001:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901408/remote/smithi001/log 2024-09-11T22:11:54.805 DEBUG:teuthology.orchestra.run.smithi001:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-11T22:12:02.078 DEBUG:teuthology.misc:Transferring archived files from smithi053:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901408/remote/smithi053/log 2024-09-11T22:12:02.079 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-11T22:12:25.658 DEBUG:teuthology.misc:Transferring archived files from smithi161:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901408/remote/smithi161/log 2024-09-11T22:12:25.659 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-11T22:12:50.611 INFO:tasks.cephadm:Removing cluster... 2024-09-11T22:12:50.612 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 --force 2024-09-11T22:12:50.827 INFO:teuthology.orchestra.run.smithi001.stdout:Deleting cluster with fsid: 3ffca83c-7087-11ef-bcea-c7b262605968 2024-09-11T22:12:51.155 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 --force 2024-09-11T22:12:51.365 INFO:teuthology.orchestra.run.smithi053.stdout:Deleting cluster with fsid: 3ffca83c-7087-11ef-bcea-c7b262605968 2024-09-11T22:12:51.820 DEBUG:teuthology.orchestra.run.smithi161:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3ffca83c-7087-11ef-bcea-c7b262605968 --force 2024-09-11T22:12:52.023 INFO:teuthology.orchestra.run.smithi161.stdout:Deleting cluster with fsid: 3ffca83c-7087-11ef-bcea-c7b262605968 2024-09-11T22:12:52.523 INFO:tasks.cephadm:Removing cephadm ... 2024-09-11T22:12:52.523 DEBUG:teuthology.orchestra.run.smithi001:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-11T22:12:52.543 DEBUG:teuthology.orchestra.run.smithi053:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-11T22:12:52.563 DEBUG:teuthology.orchestra.run.smithi161:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-11T22:12:52.579 INFO:tasks.cephadm:Teardown complete 2024-09-11T22:12:52.580 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-11T22:12:52.591 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-11T22:12:52.591 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-11T22:12:52.594 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-11T22:12:52.606 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-11T22:12:52.718 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-11T22:12:52.729 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-11T22:12:52.730 DEBUG:teuthology.orchestra.run.smithi001:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T22:12:52.733 DEBUG:teuthology.orchestra.run.smithi053:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T22:12:52.735 DEBUG:teuthology.orchestra.run.smithi161:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T22:12:52.748 INFO:teuthology.orchestra.run.smithi001.stderr:bash: line 1: ntpq: command not found 2024-09-11T22:12:52.749 INFO:teuthology.orchestra.run.smithi161.stderr:bash: line 1: ntpq: command not found 2024-09-11T22:12:52.750 INFO:teuthology.orchestra.run.smithi053.stderr:bash: line 1: ntpq: command not found 2024-09-11T22:12:52.753 INFO:teuthology.orchestra.run.smithi001.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T22:12:52.754 INFO:teuthology.orchestra.run.smithi001.stdout:=============================================================================== 2024-09-11T22:12:52.754 INFO:teuthology.orchestra.run.smithi001.stdout:^* hv01.front.sepia.ceph.com 3 6 377 23 +1073us[+1102us] +/- 60ms 2024-09-11T22:12:52.754 INFO:teuthology.orchestra.run.smithi001.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 23 +280us[ +280us] +/- 73ms 2024-09-11T22:12:52.754 INFO:teuthology.orchestra.run.smithi001.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 24 -1992us[-1963us] +/- 93ms 2024-09-11T22:12:52.754 INFO:teuthology.orchestra.run.smithi001.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T22:12:52.757 INFO:teuthology.orchestra.run.smithi053.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T22:12:52.757 INFO:teuthology.orchestra.run.smithi053.stdout:=============================================================================== 2024-09-11T22:12:52.757 INFO:teuthology.orchestra.run.smithi053.stdout:^* hv01.front.sepia.ceph.com 3 6 377 23 +1068us[+1087us] +/- 60ms 2024-09-11T22:12:52.757 INFO:teuthology.orchestra.run.smithi053.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 22 +255us[ +255us] +/- 73ms 2024-09-11T22:12:52.758 INFO:teuthology.orchestra.run.smithi053.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 24 -2007us[-1989us] +/- 93ms 2024-09-11T22:12:52.758 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T22:12:52.758 INFO:teuthology.orchestra.run.smithi161.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T22:12:52.758 INFO:teuthology.orchestra.run.smithi161.stdout:=============================================================================== 2024-09-11T22:12:52.758 INFO:teuthology.orchestra.run.smithi161.stdout:^* hv01.front.sepia.ceph.com 3 6 377 23 +1059us[+1050us] +/- 60ms 2024-09-11T22:12:52.759 INFO:teuthology.orchestra.run.smithi161.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 20 +267us[ +267us] +/- 73ms 2024-09-11T22:12:52.759 INFO:teuthology.orchestra.run.smithi161.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 22 -2005us[-2005us] +/- 93ms 2024-09-11T22:12:52.759 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T22:12:52.759 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-11T22:12:52.772 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-11T22:12:52.772 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-11T22:12:52.847 DEBUG:teuthology.orchestra.run.smithi001:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-11T22:12:52.999 DEBUG:teuthology.orchestra.run.smithi053:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-11T22:12:53.077 DEBUG:teuthology.orchestra.run.smithi161:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-11T22:12:53.146 DEBUG:teuthology.orchestra.run.smithi001:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-11T22:12:53.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T22:12:53.179 DEBUG:teuthology.orchestra.run.smithi053:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-11T22:12:53.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T22:12:53.209 DEBUG:teuthology.orchestra.run.smithi161:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-11T22:12:53.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T22:12:53.242 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-11T22:12:53.253 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-11T22:12:53.279 INFO:teuthology.task.internal:Duration was 1954.318968 seconds 2024-09-11T22:12:53.280 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-11T22:12:53.290 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-11T22:12:53.290 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-11T22:12:53.293 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-11T22:12:53.295 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-11T22:12:53.334 INFO:teuthology.orchestra.run.smithi161.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T22:12:53.335 INFO:teuthology.orchestra.run.smithi053.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T22:12:53.373 INFO:teuthology.orchestra.run.smithi001.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T22:12:53.838 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-11T22:12:53.839 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi001.front.sepia.ceph.com 2024-09-11T22:12:53.839 DEBUG:teuthology.orchestra.run.smithi001:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-11T22:12:53.864 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi053.front.sepia.ceph.com 2024-09-11T22:12:53.864 DEBUG:teuthology.orchestra.run.smithi053:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-11T22:12:53.907 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi161.front.sepia.ceph.com 2024-09-11T22:12:53.907 DEBUG:teuthology.orchestra.run.smithi161:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-11T22:12:53.936 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-11T22:12:53.936 DEBUG:teuthology.orchestra.run.smithi001:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-11T22:12:53.939 DEBUG:teuthology.orchestra.run.smithi053:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-11T22:12:53.950 DEBUG:teuthology.orchestra.run.smithi161:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-11T22:12:54.003 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-11T22:12:54.003 DEBUG:teuthology.orchestra.run.smithi001:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-11T22:12:54.010 DEBUG:teuthology.orchestra.run.smithi053:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-11T22:12:54.019 DEBUG:teuthology.orchestra.run.smithi161:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-11T22:12:55.141 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-11T22:12:55.154 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-11T22:12:55.155 DEBUG:teuthology.orchestra.run.smithi001:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-11T22:12:55.180 DEBUG:teuthology.orchestra.run.smithi053:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-11T22:12:55.210 DEBUG:teuthology.orchestra.run.smithi161:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-11T22:12:55.240 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-11T22:12:55.253 DEBUG:teuthology.orchestra.run.smithi001:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-11T22:12:55.255 DEBUG:teuthology.orchestra.run.smithi053:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-11T22:12:55.258 DEBUG:teuthology.orchestra.run.smithi161:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-11T22:12:55.281 INFO:teuthology.orchestra.run.smithi053.stdout:kernel.core_pattern = core 2024-09-11T22:12:55.283 INFO:teuthology.orchestra.run.smithi001.stdout:kernel.core_pattern = core 2024-09-11T22:12:55.308 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern = core 2024-09-11T22:12:55.328 DEBUG:teuthology.orchestra.run.smithi001:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-11T22:12:55.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T22:12:55.358 DEBUG:teuthology.orchestra.run.smithi053:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-11T22:12:55.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T22:12:55.373 DEBUG:teuthology.orchestra.run.smithi161:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-11T22:12:55.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T22:12:55.391 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-11T22:12:55.412 INFO:teuthology.task.internal:Transferring archived files... 2024-09-11T22:12:55.413 DEBUG:teuthology.misc:Transferring archived files from smithi001:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901408/remote/smithi001 2024-09-11T22:12:55.414 DEBUG:teuthology.orchestra.run.smithi001:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-11T22:12:55.491 DEBUG:teuthology.misc:Transferring archived files from smithi053:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901408/remote/smithi053 2024-09-11T22:12:55.491 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-11T22:12:55.530 DEBUG:teuthology.misc:Transferring archived files from smithi161:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901408/remote/smithi161 2024-09-11T22:12:55.530 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-11T22:12:55.572 INFO:teuthology.task.internal:Removing archive directory... 2024-09-11T22:12:55.573 DEBUG:teuthology.orchestra.run.smithi001:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-11T22:12:55.575 DEBUG:teuthology.orchestra.run.smithi053:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-11T22:12:55.578 DEBUG:teuthology.orchestra.run.smithi161:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-11T22:12:55.624 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-11T22:12:55.638 INFO:teuthology.task.internal:Not uploading archives. 2024-09-11T22:12:55.638 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-11T22:12:55.649 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-11T22:12:55.650 DEBUG:teuthology.orchestra.run.smithi001:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-11T22:12:55.653 DEBUG:teuthology.orchestra.run.smithi053:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-11T22:12:55.655 DEBUG:teuthology.orchestra.run.smithi161:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-11T22:12:55.665 INFO:teuthology.orchestra.run.smithi001.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 11 22:12 /home/ubuntu/cephtest 2024-09-11T22:12:55.668 INFO:teuthology.orchestra.run.smithi053.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 11 22:12 /home/ubuntu/cephtest 2024-09-11T22:12:55.681 INFO:teuthology.orchestra.run.smithi161.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 11 22:12 /home/ubuntu/cephtest 2024-09-11T22:12:55.683 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-11T22:12:55.694 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-11T22:12:55.723 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/fs/misc}} duration: 1954.318968296051 failure_reason: 'Command failed (workunit test fs/misc/multiple_rsync.sh) on smithi001 with status 11: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=086c737fa75c22d521a3932c59b31c5e1d1c9ef4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/multiple_rsync.sh''' flavor: default owner: scheduled_teuthology@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=c8f8877416a44baa9ff51a080f7bb66d status: fail success: false 2024-09-11T22:12:55.724 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-11T22:12:55.814 INFO:teuthology.run:FAIL